Archive | March, 2008

Flex Remoting – AMF

by on March 30, 2008 in Report

Open up FDS?

Adobe announced that it will open source the strip down version of LiveCycle Data Service called BlazeDS. From what I heard, BlazeDS provides AMF3 remoting without data management and its data push solution is not so scalable as LCDS. (features comparsion chart). The reason is that BlazeDB still uses the blocking IO to handle connections whereas LCDS uses Java NIO. The power of Java NIO is that the server thread will not be held up for the connection during the request is in process. (Detail here). So, BlazeDB is only good for tasting the power of AMG remoting and learning how AMG protocol being implemented.

Future of GraniteDS

However, if all you need a solid AMF solution to connect Flex 2 with Java, take a look at GraniteDS. I heard that it does a great job. However, GraniteDS may become obsolete soon as the founder, Franck Wolff, seemed a bit caught off guard and unsure what the future held for GraniteDS. However, finally he decided to continue GDS development. Here are why GDS is still better than BlazeDS:

  1. BlazeDS does not offer any data management features.
  2. FDS/LCDS has it own (proprietary and closed) data management system, offers (officially supported?) service factories for Spring and Hibernate (but without any lazy-loading support) and is, say, expensive.
  3. GraniteDS integrates with known enterprise J2EE persistence systems such as EJB3 (session and entity beans with full lazy-loading support) and with wildly used and open source frameworks such as Spring (with Acegi security) or JBoss/Seam (with conversation scope support – new, coming in the next release).
  4. GraniteDS provides a fast and powerful ActionScript3 code generator (Gas3): you just have to write your database model (EJB3 entity beans), Gas3 will generate corresponding ActionScript3 beans and Hibernate code generation tools will create your database tables. So, from an enterprise point of view, GraniteDS does not really compare to BlazeDS and, on the other hand, it seems to be still a powerful and free alternative to FDS/LCDS.

Update: GraniteDS 1.0 is released. Here is an article that discusses this release.

How about WebORB?

I also heard that WebORB is a powerful alternative to LCDS (compare them here). Again it is Free! However, it is not open source and its license may constraint you doing some interesting things. I will check this out more. However, here is a promising comment from the director of scrapblog.com – a fancy Flex example on Web. "Scrapblog.com needed a remoting solution to use with Flex 2, as it currently only supports Java. I didn’t think twice when choosing WebORB for this task. Version 3.0 [of WebORB] is simply amazing. You won’t experience the flexibility and responsiveness that WebORB offers with any other remoting solution. My hat goes off to Midnightcoders, keep the great work." – Omar Ramos, Scrapblog’s Director of R&D.

Leave a comment • Continue Reading →

Flex – Power of E4X

by on March 20, 2008 in Report

E4X is similar to XPath that is used to manipulate the structural xml in a scripting language. Here are some examples:

Delete node from XML

<root>
<level>
<detail></detail>
</level>
<level>
<detail></detail>
</level>
</root>

I want to make a copy of this XML (could be either of the three types of XML objects) and remove the <detail> nodes from every node no matter that hiarchy they are in.

private function testXML():void {
var xml:XML =
<root><level><detail></detail></level><level><detail></detail></level></root>
var test:XML = xml.copy();
delete test..*.detail;
trace(test.toXMLString());
}

Add new node to XML

var newItem:XML =
<level id=’1′>
<detail></detail>
</level>
test.appendChild(newItem);

Traverse

var myXML:XML =
<order>
<item id=’1′>
<menuName>burger</menuName>
<price>3.95</price>
</item>
<item id=’2′>
<menuName>fries</menuName>
<price>1.45</price>
</item>
</order>

myXML.item[0].menuName (for element traverse, use index to identify an element)
myXML.item.(@id=2).menuName (search an element using its attribute @)
myXML.item.(menuName==”xxx”).price

var total:Number = 0;
for each (var property:XML in myXML.item) {
var q:int = Number(property.@quantity);
var p:Number = Number(property.price);
var itemTotal:Number = q * p;
total += itemTotal;
trace (q + ” ” + property.menuName + ” $” + itemTotal.toFixed(2))
}
trace (“Total: $”, total.toFixed(2));

Assignment

myXML.item[0].menuName=”regular burger”;
myXML.item[1].menuName=”small fries”;
myXML.item[2].menuName=”medium cola”;

myXML.item.(menuName==”regular burger”).@quantity = “2″;
myXML.item.(menuName==”small fries”).@quantity = “2″;
myXML.item.(menuName==”medium cola”).@quantity = “2″;

var myXML:XML =
<order>
<book ISBN=”0942407296″>
<title>Baking Extravagant Pastries with Kumquats</title>
<author>
<lastName>Contino</lastName>
<firstName>Chuck</firstName>
</author>
<pageCount>238</pageCount>
</book>
<book ISBN=”0865436401″>
<title>Emu Care and Breeding</title>
<editor>
<lastName>Case</lastName>
<firstName>Justin</firstName>
</editor>
<pageCount>115</pageCount>
</book>
</order>

myXML.book –> XMLList (direct children of myXML object that has element name “book”)
myXML..lastName –> XMLList (any descendant that has element name “lastName”)
myXML.book.author.lastName –> XMLList
myXML.book[0]
myXML.book[0].title
myXML.child(“book”).title
myXML.book.(@ISBN=’0942407296′)
myXML.book[0].@ISBN –> output 0942407296
myXML.book.(title==’Emu Care and Breeding’)
myXML.book.(pageCount > 100)
myXML.book.(title.toString().search(‘Emu’) > -1)

var myXML:XML =
<order>
<item id=’1′ quantity=’2′>
<menuName>burger</menuName>
<price>3.95</price>
</item>
<item id=’2′ quantity=’2′>
<menuName>fries</menuName>
<price>1.45</price>
</item>
</order>;

var total2:Number = 0;
for each (var prop:XML in myXML.item) {
total2 += prop.@quantity * prop.price;
}

http://www.devx.com/Java/Article/21383/1763/page/1

Leave a comment • Continue Reading →

WebDav vs FTP

by on March 20, 2008 in System

Today, I have come across a technical issue that a process is taking too long to download a file from one of our file server. The reason is due to the number of the files of a folder is increased over time and finally reach to ~ 12000. If you use ftp, you need to establish tcp connection per file download.

  1. Webdav is a protocol. It enhances http, and works thru http, so you only need to open port 80 (or 443 if https)
  2. It allows you to work with remote files as if they were on your machine.
  3. Example, ftp, for you to edit a txt file on an ftp server, you need to download the file, edit, and upload the revised file. Webdav allows you to work with the file, fromthe server, without downloading.
  4. It also allows for “locking” of files, which means, multiple users can work with a file at the same time, but only 1 at a time can make changes.
  5. Wedav is also secure because it works along side ntfs permissions, and can be used over https, so everything is encrypted. With FTP you’d have to setup a vnp connection to the ftp server, and enable IPsec policies over that vpn.
  6. Webdav allows you to pipe multiple transfers, as opposed to ftp opening a new connection for every transfer.
  7. It is more efficient, and nativley support by windows (webfolders IS webdav) and ultimatly more secure than FTP, and the fact that when working with xp, every app sees the file as if it were local, makes it a much more efficient webmastering/file transfering solution.
  8. It means less ports to open on your firewall, and a smaller attack zone for hackers, cause it works from port 80 with http
Leave a comment • Continue Reading →

Art of using database indexes

by on March 13, 2008 in Data Store, Design, Scale

Need of Indexes

Image of you have a table of user info, if the table contains 50 million of rows. Without index, running a query like below will need a full table scan. Clearly it is not efficient as it is O(n) problem.

SELECT * FROM user_info WHERE last_name = “Tom”

But if we index it on last_name column, the last_name field will be sorted alphabetically. Now if you look up the last_name = ‘Tom’, you can go directly to the one starts with ‘T’. Internally, index table contains the fields you are indexed and the position of the matching records (ie. pointer).

Cost of Indexes

  1. Because database needs to maintain a separate list of indexes’ values, there is cost to keep them updated as your data changes
  2. Indexes cost space. It is a trade-off between space and time. Lets say the user_info table has 2 billions rows and last_name is 8 bytes long, you are looking at roughly 16 GB of space for the data portion of the index. Plus 4-8 bytes for each row pointer, it will go up to 32 GB of space.

With these cost, you don’t want to index every column in a table.

Type of Indexes

 

  1. Multicolumn indexes – reduce the set that matches single column only (ie. more selective). For example, if you have each field as index separately, database may not use them all at once at the same time. Like MySQL, it will only ever use one index per table per query. To choose which index to use, MySQL will make a decision about which index will return fewer rows via index statistics.
  2. Partial indexes – if you don’t have too much space for your index, you can specify a subset of bytes from your index value be used.
  3. Clustered indexes – In MySQL, for MyISAM, the indexes are kept completely separate from the row data. With clustered indexes, the index and the record itself are “clustered” together. InnoDB uses clustered indexes. In Oracle, clustered indexes are known as “index-organized tables”. The type of index will reduce two lookups (index and record data) to one. Internally, clustered index reorders the way records in the table are physically stored. Therefore table can have only one clustered index. The leaf nodes of a clustered index contain the data pages.

Deep look in MySQL: The InnoDB storage engine creates a clustered index for every table. If the table has a primary key, that is the clustered index. If not, InnoDB internally assigns a six-byte unique ID to every row and uses that as the clustered index. (don’t let it generate a useless one for you). All indexes are B-trees. In InnoDB, the primary key’s leaf nodes are the data. Secondary indexes have a pointer to the data at their leaf nodes.

Index Structure

  • B-Tree Indexes – (balanced tree indexes, a tree structure that will never become lopsided as new nodes are added and removed. It gives us O(log n) performance for single-record lookup). Unlike binary tree, B-trees have many keys per node and don’t grow tall or deep as quick as a binary tree. It is very good for range-based queries as well. For example, for quey below, server simply finds the first Ray record and last Robert record. It then knows everything in between are also matches. The same is true of virtually any query that involves understanding the range of values (>, <, MIN, MAX, BETWEEN xx AND yy)

SELECT * FROM user_info WHERE last_name BETWEEN ‘Ray’ AND ‘Robert’

  • Hash Indexes – It gives very fast lookups O(1) but it is less flexible and predictable than other indexes. First, the key is hashed and compare. So, the range-based queries cannot use it. Uniform distribution is the key here.

Index Limitation

  1. Index doesn’t work together with wildcard and regular expression search.
  2. If index selectively is like > 30% of rows (very low), table scan may be better.

 

Leave a comment • Continue Reading →

Flex 3 – Advanced DataGrid

by on March 3, 2008 in Report

Since I am leading the reporting team in Adconion, I would like to spend sometime on the new component “Advanced Data Grid” from Flex 3. Being excited to know that this component has added many cool features on top of the DataGrid, I would like to find out whether it meets all my needs. Before I introduce it to my company, I would like to write an example to exercise its power first. The example below will demonstrate some documented features in Advanced Data Grid and some custom features that I would like in my project. To make the example prettier, I would use Yahoo Flex Skin.

Yahoo Flex Skin

yahooflexskin.JPG

Apart from providing you flex skin, Yahoo also shares with you some custom-made flex components. Among them, I would like to look into Yahoo Map once I have time. (http://developer.yahoo.com/flash/maps/examples.html). However, it is out of my scope today.

What I want to show you in my example
There are some cool features I want to try out in this example. Here is the list:

  1. Grouping
  2. Tree view with “Expand All” and “Shrink All” features.
  3. Multiple columns sorting
  4. Auto-completed search filter
  5. Smart resizing data grid
Leave a comment • Continue Reading →