Update

SQLite 3 is released that addressed some of its issues in version 2.

• BLOB support
• Fulltext searching
• Connection shared between threads
• Improve concurrency

However, it still doesn’t support writeable view, nested transaction and foreign key.

Presentation

Here is a nice presentation from Paul Roberson, look at it first.

 Note from the video:

1. Warm up with general SQL Tips (Join favors subquery, Avoid IN, Avoid LIKE, Specify columns name in select and insert, Avoid unnecessary join)
2. AIR SQL connection can connect up to 10 databases at a time, you can use qualifier for your tables.
3. Don’t reuse the same SQL Statement for different prepared statements.
4. Use transaction to do batch insert/ update/ delete operations (48x faster!)
5. Index columns in WHERE clause, use together index together
6. Create table structure before you add data because internally SQLite…?
7. Handle large resultset in parts for perceived performance gain (detailed)

There are several things I want to find out:

1. Can SQLite handles large dataset?
   • Yes. According to spec, it can handle terabyte of data.
2. Does SQLite support pagination?
   • Yes. Look at SQL Statement object.
3. How SQLite synchronize with the updated data from the remote database?
   • Strategies: overwrite vs delta (timestamp, field by field comparison, dirty flag)
   • Live cycle data service has built-in SQLite synchronization support including offline caching and conflict management.
   • http://coenraets.org/blog/2008/05/insync-automatic-offline-data-synchronization-in-air-using-lcds-26/

Some notes about SQLite

Below are some SQLite tips and practices I obtained from different sources:

• A big advantage of sqlite above a flat file is the possibility to index your data.
• Using parameterize query protects against sql injection, and makes the ‘ problems go away. It is also much faster because sqlite can reuse the execution plan of statements when you use parameters.
• Make sure to import the records in a transaction so that it doesn’t spend a lot of time creating indexes until everything is imported.
• The SQLite documentation states that SQLite databases can be terabytes in size, and that the primary limitation of SQLite is concurrency (many users at the same time).
• “The SQLite database is pretty damn fast. I was getting near instantaneous searching with databases that were ~100,000 records. Somewhere around 800,000 – 1,000,000 records you start losing performance, waiting a few seconds for a search” – by Daniel
• Each database is contained within a single file.

Reference

Below are some good links I have found:

• Develop air application with large dataset
• SQLite at the Lake – Peter Elst (06/26/2009)
• SQLite admin tool by David Deraedt – not open source
• Another SQLite admin tool by Christophe Coenraets (10/2007) – open source
• Firefox SQLite Manager plugin – support csv import. However, the project is no longer active.
• Adobe Air and SQLite video session (4/18/2008) – it contains 3 interesting examples.
  
• Get started with adobe air and sqlite (1/28/2009)
• Develop the connection and statement pool

I always wonder how my Flex application displayed on the Flash Player in browser. Why decompile Flex SWF will give me 2 frames movie? What is SystemManager and how can I get a handle of it? Many of these kind of questions are at the lower level. The level that makes Flex application possible. Normally, we don’t need to dive into this to write a Flex application. However, it would make me feel more comfortable to understand this before I advocate Flex as the main part of our company’s UI strategy. 

To understand how your Flex application got loaded and display on Flash Player, you can read this great article. I am going to put down the sequences in steps below:

Flex is a 2-frame movie

The first frame of a Flex SWF contains the SystemManager, the Preloader, the DownloadProgressBar and some glue helper classes. Remember, the Preloader is what creates the DownloadProgressBar control which displays the progress of a Flex application downloading and being initialized. The second frame of a Flex SWF contains the rest of the Flex framework code, your application code and all of your application assets like embedded fonts, images, etc.

By creating a 2-frame movie, Flex applications can take advantage of the streaming support built into the Flash Player and a preloader can appear before all of the Flex framework code and your application code are downloaded.

The .swf format is a progressive download format which means that Flash player can access content on frames as they download without having to wait for the entire file to download.

Here are the steps:

1. First, enough bytes for frame 1 are streamed down to the Flash Player.
2. The Flash Player executes those bytes by creating a SystemManager instance.
3. SystemManager instruct the Flash Player to stop at the end of frame 1.
4. SystemManager then goes on to create the Preloader which creates the DownloadProgressBar control and pops that up on the client screen.
5. The Preloader then starts tracking the rest of the bytes streaming in from the Flex SWF. 
6. Once all the bytes for the Flex framework and application code are in, the System Manager goes on to frame 2 and instantiates the Application instance.
7. Once the Application instance has been created, the SystemManager sets Application.systemManager to itself. This is how you, the application developer, can access the SystemManager at a later time.
8. The Application dispatches the preinitialize event at the beginning of the initialization process.
9. Application goes on to create its children. The method createChildren() is called on the application. At this point each of the application’s components is being constructed, and each component’s createChildren() will be also called. For detail, look at component lifecycle section.
10. The Application dispatches the initialize event, which indicates that all application’s components have been initialized. However, at this state, all the components are not yet laid out.
11. Eventually, once all the Application child controls and containers have been created, sized and positioned, the Application dispatches the creationComplete event.
12. Once the creationComplete event has been dispatched, the Preloader removes the DownloadProgressBar control and the SystemManager adds the Application instance to the Flash Player display list. (The Flash Player display list is basically the tree of visible or potentially visible objects that make up your application. When you add and remove child components to your application, your basically just adding and removing them from the display list).
13. Once the Application is added to the Flash Player display list, the Application dispatches its applicationComplete event
14. The Application has been created and is up on the screen ready to be interacted with.
     

Component Architecture

 This is the best video I have found that discuss the component lifecycle in detail – Thanks for Deepa.

Below is summary I took from the video above, in case you don’t want to spend an hour to listen to this. However, I really think you should. To start out, Deepa introduced component and skinning architecture “Spark” that is part of Flex 4 Gumbo. Spark is built on top of Halo (ie. Flex 3 component architecture) and components using Halo or Spark can co-exist. Then,  Deepa put her focus to talk about how to develop her custom video component on top of Halo and Spark for comparison. Spart is great that it can factor out the layout code from component.

Halo component lifecycle can be separated into 3 phases:

Phase 1 – Initialization

1. Construction
   • Choose the right base class and provide default constructor (ie. zero argument). By extending UIComponent, you inherit all of the lifecycle methods, events and properties. Also, since UIComponent extends from EventDispatcher, your component inherits the ability to listen and dispatch events.
   • Best practice: Call super() and add event listener. Minimal work should occur here.
2. COnfiguration
   • setter and getter for properties.
   • Involve in invalidation and validation cycle. Detail later.
   • Best practice: Setter should not expect the internal children have been created and you want your setter to be fast. Use _xxx for storage variable and have dirty flag for your variable. Throw events out when the internal state of your component get changed.
3. Attachment
   • Component are added to the flash display list by its parent via addChild() call. Without attachment, component lifecycle will be stalled. Nothing is going to happen to your component. So, this is an important step.
   • Display list is a tree of visible or potentially visible objects in your application. At the root of the display list is your main application. Thing like containment hierarchy and rendering order are all maintained by the display list.
4. Initialization- 5 lifecycle actions occur here.
   • preinitialize event is dispatched. It signifies that you as component that has been added to the display list by your parent via addChild().
   • createChildren() – walk through all your children, create, configure and attach them to the display list. Best practice: Call super.createChildren(), construct if not exist and attach your children via addChild(). If your child component is dynamic and data driven, use commitProperties() b/c it gets called at every invalidation and validation cycle. Halo rules: Container –> nested structure of UIComponents –> MovieClip, Video, Shape and Sprite.
   • initialize event is dispatched. It signifies that you and all your children are created and attached.
   • First invalidation/ validation pass occurs,
   • creationComplete event is dispatched. Ready for prime time.

Phase 2 – Update

At this phase, your component is fully initialized and ready for usage. Now, it needs to know how to update. And update occurs when its internal state has been changed by like user interactions. To respond to the changes, component uses the invalidation and validation cycle. The key here is to flag the changed variable dirty during invalidation and later handles it during validation right before rendering. So, you can have many invalidation and one validation that gives better performance via avoiding repetitive work. To better understand this approach, Deepa talks about Flash Player Elastic Racetrack that has 2 parts: code execution and rendering. If either part taking too long, Flash player cannot get its job done faster than 1 frame per second. You will see your application with lag and halt, that is bad! 

1. Invalidation/ Validation can be split into 3 phase.
   • InvalidateProperties –> commitProperties
   • InvalidateSize –> measure (measure may not be called. Don’t have your code depends on it)
   • InvalidateDisplayLIst –> updateDisplayList

Phase 3 – Destruction

1. Detachment
2. Garbage Collection

What is Mixin?

When you put the [Mixin] metadata just above your class definition and add a static init function to the class like so,

[Mixin]
public class Model {
	public static function init (systemManager : ISystemManager)   {
		trace ("I get called first")
	}
}

The static init function will be called as soon as your application loads (assuming this class is referenced somewhere in the app), much like static initialization blocks in Java. This is useful if you have some code that you want to run before any of the other code in the class.

Reference

Below are some interesting articles I found related to this article

1. Create Custom Preloader, Ted has an article related to this too
2. Speed up startup loading time
3. Dynamic loading a new custom theme without fraction seconds delay
4. Deepa presentation in MAX
5. Introduction to mixins

1. Dow Jone Interactive Chart (commerical – it is exactly what I am looking for)
2. Interactive Bubble Chart (open – although it is not exactly want I want, but if I believe the code can benefit me if I need to customize line chart. I may just need to draw the interactive small bubble on the line to get my job done!)
3. This demo gives you tons of chart samples. They are all great example although none of them satisfy my current need.
4. This demo is close to what I want. From this demo, I notice I can use “annotationElement” to draw on top of the data series. However, the trick is to convert the data points to pixel coordinate in order for me to draw something that can move along with the graph even someone stretches it. To make thing easier, Ely Greenfield has created DataDrawingCanvas that helps us draw on the chart with only data points specified instead of pixel coordinates. This class extends the ChartElement like AnnotationElement does (blog). That is amazing!! Thanks!!!
5. Google Finance Chart (It is exactly what I want. I wonder I can get the source of it)
   • I have found the blog and powerpoint of this sample (1/7/2009)
   • Google uses the Flash/ JavaScript integrate kit to get it works (blog) – I heard that it is very nice combination of Flash and AJAX. This is similar to MeasureMap‘s use of the kit.
   • It is open source example!!! (code). Thanks for Brendan Meutzner!!
   • Brendan also shows us how he created his demo in 5 steps to help us understand how to build it ourselves.
   • Step 1, Step 2, Step 3, Step 4, Step 5 – enjoy!!

Reference

Useful resources:

1. Data Visualization by Tom Gonzalez. (Tom created an open source visualization framework named Axiis. It looks great. Once I get a chance, I will dig into it) – 7/31/2009
2. Building a Flex Component by Ely GreenField
3. Create component and enforce separation of concern
4. http://www.edwardtufte.com/tufte/ (Edward Tufte – famous guy in data visualization)
5. http://www.insideria.com/2008/03/image-manipulation-in-flex.html (Image Manipulation)

Recently, I found out that Adobe has released BlazeDS (subset of LiveCycleDS) that has 4 main advantages:

1. AS3 to Java object communication (no XML passes back and forth is needed!)
2. Boost up performance b/c AMF is a binary protocol
3. Built-in proxy support that gets around the cross domain security issue from Flex in ease.
4. Allow push messaging

I have followed the guideline and set it up. Now my Flex application can call my Java object method without passing xml back and forth. It is awesome! During the setup process, you may experience your flex cannot find the destination set up in the server.

The trick here is to add a services argument to the mxmlc call, something of the form below should do the trick! 

-services “[local path to your java project]/WEB-INF/flex/services-config.xml”

Now you may start enjoying how AS3 talks to your Java Object. However, if  we bypass the Servlet layer in the code, how can we carry session across remote method calls? Great that I have found out how to handle it via this article. In short, you can access Session from your Java object via:

FlexContext.getFlexSession()

Here is the quote I got from the BlazeDS developer guide.

The FlexContext class is useful for getting access to the session and the HTTP pieces of the session, such as the HTTP servlet request and response. This lets you access HTTP data when you use a Flex application in the context of a larger web application where other classes, such as JSPs or Struts actions, might have stored information.

The FlexSession class provides access to an ID and also provides setAttribute and getAttribute functionality. This is useful for storing data on the server that doesn’t have to go back to the client. However, FlexSession is not cluster-aware; if a client connects to a different server in the cluster, the client receives a new FlexSession. Nothing stored in the FlexSession attributes is persisted for clustering purposes. The FlexSessionListener class is useful for monitoring who is connected. You add a listener by using the static method to track new connections being made. You receive a reference to the session that was added. Each session can then report when it is destroyed to those same listeners. You use this for monitoring connections that close, and also to clean up resources.

When I looked into the source of FlexContext, I noticed that it leverages ThreadLocal to store context info like request, response and session.

    private static ThreadLocal sessions = new ThreadLocal();
    /**
     * The FlexSession for the current request.  Available for users.
     */
    public static FlexSession getFlexSession()
    {
        return (FlexSession)sessions.get();
    }

Reference

Below are some of the useful references I have read so far:

1. Jim Boone’s Blog

Follow the steps below to get your first Flex salesforce app up in Salesforce.

1. Register a developer edition account from Salesforce. Note: Dev account never expires but the account does come with a few limitations. You can only have two users, one an admin account so that you can build and install applications and the other a normal user account so you can test your work from the perspective of a normal user. The account has a 2MB data limit and you can send mass email. However, It is totally fine for playing around all the features that Salesforce provides.
   
2. Download the Flex Salesforce Toolkit (ie. force.com-air_flex-1.0.zip). This toolkit provides the needed libraries to communicate directly with your salesforce.com database records from within a Flex application, using native ActionScript packages and returning strongly typed classes.The documentation on the classes can be found here.
3. In this zip file, there is a library called as3Salesforce.swc in the bin directory just off the root of this zip file. It is the library you need to associate to your Flex project.
4. Create a Flex project and include the swc library in it.
5. You can follow this screencast to get your first project up.

What you can do after that?

Now you have your Flex application run locally in Salesforce. Here is my TODO list and the solutions of each one.

1. Run your app under your own website and pull info from Salesforce using the same api.
2. Run your app under Salesforce and have it pulled data from your system thru Flex Remoting (HTTPService, WebService, RemoteObject…etc).
3. Can we use the API to pull Salesforce metadata like SControl?
4. How can we provide our application via Apex Exchange?
   • Introduction of Salesforce’s AppExchange
   • Using Salesforce’s API
   • Packaging for salesforce’s appexchange
5. How can we use the Salesforce Flex AIR Toolkit to make have your application deal with Salesforce in offline mode? Look into this article.

What Flex gives you but not original Salesforce UI?

Now you know how to integrate Flex with Salesforce. But what problems that Flex helps us solving but not the original Salesforce UI?

1. Capture user events on the fly and display additional fields or populate bunch of fields.
2. Visual the data via Flex charting.
3. Full control on the layout, and look and feel.

OK. There is need for it. Although the need may go away once the whole world is wired by WiFi or other technologies (may still take a while), we may want to solve this problem to improve user experience. Here is how the problem get solved in high level?

1. User request is captured locally.
2. All your operation is handled locally to the local database.
3. When Internet access is available, synchronize the local database with the server database.

How Gear solves this? (I learn it here)

1. LocalServer module captures browser HTTP and HTTPS requests if there is no internet access (Browser plugin works only with Firefox 1.5+ and IE6).
2. Database module (uses SQLite) stores the updates and provides the information needed. (news). When Internet access is available, it will do the synchronization work for you automatically.
3. Google gear also provides background processing mechanism taht allows I/O or compute-intensive Javascript to run without blocking the user interface.

How AIR solves this?

1. Adobe AIR is desktop application that runs without browser.
2. Apollo provides SQLite as well for storing the updates and provides the information needed.
3. Sync module will do the synchronization work when Internet access is available.

My preference

I have to pick either one of them. I would select AIR b/c ilike the power of Flex in UI. So, if AIR solves the problem, I may just stick with it. Simple?!! Since I am working for the Performance Ad Network right now, if I can pull enough data across the wire and store it locally, then I can run OLAP and other complex queries and visualize the data with the advanced data grid and charting available in Flex, that will be awesome!

Update: Flex and Gear actually can work together! (Someone demonstrates this combination via building a sales force application)

Adobe announced that it will open source the strip down version of LiveCycle Data Service called BlazeDS. From what I heard, BlazeDS provides AMF3 remoting without data management and its data push solution is not so scalable as LCDS. (features comparsion chart). The reason is that BlazeDB still uses the blocking IO to handle connections whereas LCDS uses Java NIO. The power of Java NIO is that the server thread will not be held up for the connection during the request is in process. (Detail here). So, BlazeDB is only good for tasting the power of AMG remoting and learning how AMG protocol being implemented.

Future of GraniteDS

However, if all you need a solid AMF solution to connect Flex 2 with Java, take a look at GraniteDS. I heard that it does a great job. However, GraniteDS may become obsolete soon as the founder, Franck Wolff, seemed a bit caught off guard and unsure what the future held for GraniteDS. However, finally he decided to continue GDS development. Here are why GDS is still better than BlazeDS:

1. BlazeDS does not offer any data management features.
2. FDS/LCDS has it own (proprietary and closed) data management system, offers (officially supported?) service factories for Spring and Hibernate (but without any lazy-loading support) and is, say, expensive.
3. GraniteDS integrates with known enterprise J2EE persistence systems such as EJB3 (session and entity beans with full lazy-loading support) and with wildly used and open source frameworks such as Spring (with Acegi security) or JBoss/Seam (with conversation scope support – new, coming in the next release).
4. GraniteDS provides a fast and powerful ActionScript3 code generator (Gas3): you just have to write your database model (EJB3 entity beans), Gas3 will generate corresponding ActionScript3 beans and Hibernate code generation tools will create your database tables. So, from an enterprise point of view, GraniteDS does not really compare to BlazeDS and, on the other hand, it seems to be still a powerful and free alternative to FDS/LCDS.

Update: GraniteDS 1.0 is released. Here is an article that discusses this release.

How about WebORB?

I also heard that WebORB is a powerful alternative to LCDS (compare them here). Again it is Free! However, it is not open source and its license may constraint you doing some interesting things. I will check this out more. However, here is a promising comment from the director of scrapblog.com – a fancy Flex example on Web. "Scrapblog.com needed a remoting solution to use with Flex 2, as it currently only supports Java. I didn’t think twice when choosing WebORB for this task. Version 3.0 [of WebORB] is simply amazing. You won’t experience the flexibility and responsiveness that WebORB offers with any other remoting solution. My hat goes off to Midnightcoders, keep the great work." – Omar Ramos, Scrapblog’s Director of R&D.

Delete node from XML

<root>
<level>
<detail></detail>
</level>
<level>
<detail></detail>
</level>
</root>

I want to make a copy of this XML (could be either of the three types of XML objects) and remove the <detail> nodes from every node no matter that hiarchy they are in.

private function testXML():void {
var xml:XML =
<root><level><detail></detail></level><level><detail></detail></level></root>
var test:XML = xml.copy();
delete test..*.detail;
trace(test.toXMLString());
}

Add new node to XML

var newItem:XML =
<level id=’1′>
<detail></detail>
</level>
test.appendChild(newItem);

Traverse

var myXML:XML =
<order>
<item id=’1′>
<menuName>burger</menuName>
<price>3.95</price>
</item>
<item id=’2′>
<menuName>fries</menuName>
<price>1.45</price>
</item>
</order>

myXML.item[0].menuName (for element traverse, use index to identify an element)
myXML.item.(@id=2).menuName (search an element using its attribute @)
myXML.item.(menuName==”xxx”).price

var total:Number = 0;
for each (var property:XML in myXML.item) {
var q:int = Number(property.@quantity);
var p:Number = Number(property.price);
var itemTotal:Number = q * p;
total += itemTotal;
trace (q + ” ” + property.menuName + ” $” + itemTotal.toFixed(2))
}
trace (“Total: $”, total.toFixed(2));

Assignment

myXML.item[0].menuName=”regular burger”;
myXML.item[1].menuName=”small fries”;
myXML.item[2].menuName=”medium cola”;

myXML.item.(menuName==”regular burger”).@quantity = “2″;
myXML.item.(menuName==”small fries”).@quantity = “2″;
myXML.item.(menuName==”medium cola”).@quantity = “2″;

var myXML:XML =
<order>
<book ISBN=”0942407296″>
<title>Baking Extravagant Pastries with Kumquats</title>
<author>
<lastName>Contino</lastName>
<firstName>Chuck</firstName>
</author>
<pageCount>238</pageCount>
</book>
<book ISBN=”0865436401″>
<title>Emu Care and Breeding</title>
<editor>
<lastName>Case</lastName>
<firstName>Justin</firstName>
</editor>
<pageCount>115</pageCount>
</book>
</order>

myXML.book –> XMLList (direct children of myXML object that has element name “book”)
myXML..lastName –> XMLList (any descendant that has element name “lastName”)
myXML.book.author.lastName –> XMLList
myXML.book[0]
myXML.book[0].title
myXML.child(“book”).title
myXML.book.(@ISBN=’0942407296′)
myXML.book[0].@ISBN –> output 0942407296
myXML.book.(title==’Emu Care and Breeding’)
myXML.book.(pageCount > 100)
myXML.book.(title.toString().search(‘Emu’) > -1)

var myXML:XML =
<order>
<item id=’1′ quantity=’2′>
<menuName>burger</menuName>
<price>3.95</price>
</item>
<item id=’2′ quantity=’2′>
<menuName>fries</menuName>
<price>1.45</price>
</item>
</order>;

var total2:Number = 0;
for each (var prop:XML in myXML.item) {
total2 += prop.@quantity * prop.price;
}

http://www.devx.com/Java/Article/21383/1763/page/1

Yahoo Flex Skin

Apart from providing you flex skin, Yahoo also shares with you some custom-made flex components. Among them, I would like to look into Yahoo Map once I have time. (http://developer.yahoo.com/flash/maps/examples.html). However, it is out of my scope today.

What I want to show you in my example
There are some cool features I want to try out in this example. Here is the list:

1. Grouping
2. Tree view with “Expand All” and “Shrink All” features.
3. Multiple columns sorting
4. Auto-completed search filter
5. Smart resizing data grid

The SharedObject class functions like a browser cookie. You use the class to store data on the user’s local hard drive and call that data during the same session or in a later session. Below are the keys of using SharedObject:

1. Applications can only access their own SharedObject data, and only if they are running on the same domain. The data is not sent to the server and is not accessible by other Flex applications running on other domains.
2. When you create a SharedObject, Macromedia Flash Player creates a new directory for the application and domain, and creates an empty *.sol file that stores the SharedObject data. The default location of this file is in a subdirectory of the user’s home directory.
3. By default, Flash can save locally persistent SharedObjects of up to 100K in size. When you try to save a larger set of data, Flash Player displays the Local Storage dialog box, which lets the user allow or deny local storage for the domain that is requesting access.
4. Some important methods:
   • clear()
   • flush() – write to the file. If not, flex persists it when the application is closed. However, this does not provide the user with an opportunity to increase the available space
   • getLocal(“name”)
   • getSize()
5. You can store simple data types in a SharedObject. These types are Number, String, Boolean, XML, Date, Array, and Object. After you assign values to the data property, you must instruct Flash Player to write those values to the SharedObject’s file. To force Flash Player to write the values to the SharedObject’s file, use the flush() method:

sharedObject_ID.data.variable = value; //store data


LSO vs Cookie

1. LSOs are can store more data than cookies
2. LSOs never expire
3. LSOs arent transmitted between the client and the server
4. LSOs can store native ActionScript datatypes. 

Reference

http://www.adobe.com/support/documentation/en/flex/1/lsos/lsos5.html