Archive | Report RSS feed for this section

Reporting solution!

Open source reporting

My company needs a reporting engine but it doesn’t want to go for the expensive commerical ones like MicroStrategy. In fact, I don’t know why we need to pay so much because there are tools out there for FREE. As usual, I googled the Net and found out two seemingly promising open source reporting solution.

  1. Pentaho Reporting
  2. Jasper Reporting

Both of them are bundled with a suite of tools related to OLAP, Data Mining, ETL.. etc. To me, I just want an non-invasive reporting engine that can easily integrate into our architecture. To my dismay, I found out Pentaho doesn’t go this route. It basically gives you a reporting server configured. You could build your reports and deploy them following the manual. However, I hardly see a reporting solution that could satisfy all the business requirements without customization. All I expected from Pentaho is a jar file with documents that shows me how to use its api to generate reports in different formats and how to integrate with our database. I have attempted to look into the code and extracted the stuff I want from Pentaho. However, I found out the engine is actually not powerful. To strip out the workflow part, it is basically a simple SQL executor that later on will render the result according to the UI info embedded in the report definition. What is wrong with that?

  1. We want to handle pagination and data streaming as our data volume is huge. In Pentaho, you need to take care these yourself. So, you write your own sql, paginate yourself, stream it yourself if the resultset is huge. Isn’t it what we are doing without it? Apart from that, each report in Pentaho needs a report definition. It supports dynamic sql via token replacement. It is primitive as I want it to support control flow because I may decide what tables to join based on the input filters.
  2. On the UI side, Pentaho helps you to render your result into graph, table…etc. Again, I don’t like this UI solution as well. I found that JFreeChart is not as powerful as the Flex solution. I am adopting Flex and it gives me much powerful visualization tool. All I want is to ship my Flex app the data from my query’s result.

How about Japser? Pretty much the same but the good thing of Jasper is that it gives you the jar and document of how to use it instead of a reporting server like Pentaho. So, I can use it as report renderer to generate PDF and Excel like other utility libraries I use. So, what is my final solution?

I finally decide to create my own report definition that my Flex UI can take and render out the reporting interface. So, I don’t need to create form for each report. Apart from that, in my report definition, I have iBatis SQL template embedded. So, I can leverage its dynamic sql feature that supports control flow logic and the auto result class population. Yes, I still need to handle pagination and streaming myself. But, at least, it already saves up my time. The result object populated will return to Flex via AMF. So, I don’t need to marshal and de-marshal it in xml. It saves the processing time and costs less bandwidth. At the end, my solution combines the best in the market:

  1. Powerful reporting widgets provided by Flex
  2. Fast streaming and RPC protocol – AMF
  3. Good dynamic sql generation and mapping tool from iBatis
  4. Good reporting rendering tool from Jasper that helps me to do PDF and Excel generation

My solution is more flexible. As I can plugin hibernate map if I don’t want to write my own sql at all. Apart from that, no UI work is needed to deploy a new report unless my generic reporting interface is not enough.

Later, if I really need the workflow engine provided by Pentaho, I can plug it in. Again, the document provided doesn’t give us clear instruction or APIs of how to do it.

Reference

Below are references I used to build my solution:

  1. Flexible reporting with JasperReport and iBatis
  2. How Kodo JPA handles large result set (its optimization guide is good reference even you may not use Kodo)
  3. Process Large Result Sets in Java Web Application
  4. Streaming architecture

 

Leave a comment Continue Reading →

Flex Annotated Charting

Recently, I want to extend the LineChart in Flex. I want to have line chart with event annotated like Google Finance.

 
First of all, I googled the Net to see whether anyone had already done it. It was even better if I could find any open source project related to this. Below are the interesting things I found:
  1. Dow Jone Interactive Chart (commerical – it is exactly what I am looking for)
  2. Interactive Bubble Chart (open – although it is not exactly want I want, but if I believe the code can benefit me if I need to customize line chart. :cool: I may just need to draw the interactive small bubble on the line to get my job done!)
  3. This demo gives you tons of chart samples. They are all great example although none of them satisfy my current need.
  4. This demo is close to what I want. From this demo, I notice I can use “annotationElement” to draw on top of the data series. However, the trick is to convert the data points to pixel coordinate in order for me to draw something that can move along with the graph even someone stretches it. To make thing easier, Ely Greenfield has created DataDrawingCanvas that helps us draw on the chart with only data points specified instead of pixel coordinates. This class extends the ChartElement like AnnotationElement does (blog). That is amazing!! Thanks!!! :smile:
  5. Google Finance Chart (It is exactly what I want. I wonder I can get the source of it)
    • I have found the blog and powerpoint of this sample (1/7/2009)
    • Google uses the Flash/ JavaScript integrate kit to get it works (blog) – I heard that it is very nice combination of Flash and AJAX. This is similar to MeasureMap‘s use of the kit.
    • It is open source example!!! (code). Thanks for Brendan Meutzner!!
    • Brendan also shows us how he created his demo in 5 steps to help us understand how to build it ourselves.
    • Step 1, Step 2, Step 3, Step 4, Step 5 – enjoy!!

Reference

Useful resources:

  1. Data Visualization by Tom Gonzalez. (Tom created an open source visualization framework named Axiis. It looks great. Once I get a chance, I will dig into it) – 7/31/2009
  2. Building a Flex Component by Ely GreenField
  3. Create component and enforce separation of concern
  4. http://www.edwardtufte.com/tufte/ (Edward Tufte – famous guy in data visualization)
  5. http://www.insideria.com/2008/03/image-manipulation-in-flex.html (Image Manipulation)

 

Leave a comment Continue Reading →

Adobe AIR vs Google Gear

When you come to the world of offline/disconnected application, you will hear the new buzzwords – AIR and Gear. Before I talked about these technologies, I want to talk about what problem we are trying to solve first. What we need an application to be able to function offline? First, Internet access is not everywhere. When you are on the plane, at the coffee shop, on the Maui beach etc, you may not be able to connect to your favorite Web applications. What if you want to look at some of your reports? I would like to see the traffics statistics for my own little wedding website company – Jusproposed.com. However, I don’t see Google Analytics available offine :)

OK. There is need for it. Although the need may go away once the whole world is wired by WiFi or other technologies (may still take a while), we may want to solve this problem to improve user experience. Here is how the problem get solved in high level?

  1. User request is captured locally.
  2. All your operation is handled locally to the local database.
  3. When Internet access is available, synchronize the local database with the server database.

How Gear solves this? (I learn it here)

  1. LocalServer module captures browser HTTP and HTTPS requests if there is no internet access (Browser plugin works only with Firefox 1.5+ and IE6).
  2. Database module (uses SQLite) stores the updates and provides the information needed. (news). When Internet access is available, it will do the synchronization work for you automatically.
  3. Google gear also provides background processing mechanism taht allows I/O or compute-intensive Javascript to run without blocking the user interface.

How AIR solves this?

  1. Adobe AIR is desktop application that runs without browser.
  2. Apollo provides SQLite as well for storing the updates and provides the information needed.
  3. Sync module will do the synchronization work when Internet access is available.

My preference

I have to pick either one of them. I would select AIR b/c ilike the power of Flex in UI. So, if AIR solves the problem, I may just stick with it. Simple?!! :) Since I am working for the Performance Ad Network right now, if I can pull enough data across the wire and store it locally, then I can run OLAP and other complex queries and visualize the data with the advanced data grid and charting available in Flex, that will be awesome!

Update: Flex and Gear actually can work together! (Someone demonstrates this combination via building a sales force application)

Leave a comment Continue Reading →

Flex Remoting – AMF

Open up FDS?

Adobe announced that it will open source the strip down version of LiveCycle Data Service called BlazeDS. From what I heard, BlazeDS provides AMF3 remoting without data management and its data push solution is not so scalable as LCDS. (features comparsion chart). The reason is that BlazeDB still uses the blocking IO to handle connections whereas LCDS uses Java NIO. The power of Java NIO is that the server thread will not be held up for the connection during the request is in process. (Detail here). So, BlazeDB is only good for tasting the power of AMG remoting and learning how AMG protocol being implemented.

Future of GraniteDS

However, if all you need a solid AMF solution to connect Flex 2 with Java, take a look at GraniteDS. I heard that it does a great job. However, GraniteDS may become obsolete soon as the founder, Franck Wolff, seemed a bit caught off guard and unsure what the future held for GraniteDS. However, finally he decided to continue GDS development. Here are why GDS is still better than BlazeDS:

  1. BlazeDS does not offer any data management features.
  2. FDS/LCDS has it own (proprietary and closed) data management system, offers (officially supported?) service factories for Spring and Hibernate (but without any lazy-loading support) and is, say, expensive.
  3. GraniteDS integrates with known enterprise J2EE persistence systems such as EJB3 (session and entity beans with full lazy-loading support) and with wildly used and open source frameworks such as Spring (with Acegi security) or JBoss/Seam (with conversation scope support – new, coming in the next release).
  4. GraniteDS provides a fast and powerful ActionScript3 code generator (Gas3): you just have to write your database model (EJB3 entity beans), Gas3 will generate corresponding ActionScript3 beans and Hibernate code generation tools will create your database tables. So, from an enterprise point of view, GraniteDS does not really compare to BlazeDS and, on the other hand, it seems to be still a powerful and free alternative to FDS/LCDS.

Update: GraniteDS 1.0 is released. Here is an article that discusses this release.

How about WebORB?

I also heard that WebORB is a powerful alternative to LCDS (compare them here). Again it is Free! However, it is not open source and its license may constraint you doing some interesting things. I will check this out more. However, here is a promising comment from the director of scrapblog.com – a fancy Flex example on Web. "Scrapblog.com needed a remoting solution to use with Flex 2, as it currently only supports Java. I didn’t think twice when choosing WebORB for this task. Version 3.0 [of WebORB] is simply amazing. You won’t experience the flexibility and responsiveness that WebORB offers with any other remoting solution. My hat goes off to Midnightcoders, keep the great work." – Omar Ramos, Scrapblog’s Director of R&D.

Leave a comment Continue Reading →

Flex – Power of E4X

E4X is similar to XPath that is used to manipulate the structural xml in a scripting language. Here are some examples:

Delete node from XML

<root>
<level>
<detail></detail>
</level>
<level>
<detail></detail>
</level>
</root>

I want to make a copy of this XML (could be either of the three types of XML objects) and remove the <detail> nodes from every node no matter that hiarchy they are in.

private function testXML():void {
var xml:XML =
<root><level><detail></detail></level><level><detail></detail></level></root>
var test:XML = xml.copy();
delete test..*.detail;
trace(test.toXMLString());
}

Add new node to XML

var newItem:XML =
<level id=’1′>
<detail></detail>
</level>
test.appendChild(newItem);

Traverse

var myXML:XML =
<order>
<item id=’1′>
<menuName>burger</menuName>
<price>3.95</price>
</item>
<item id=’2′>
<menuName>fries</menuName>
<price>1.45</price>
</item>
</order>

myXML.item[0].menuName (for element traverse, use index to identify an element)
myXML.item.(@id=2).menuName (search an element using its attribute @)
myXML.item.(menuName==”xxx”).price

var total:Number = 0;
for each (var property:XML in myXML.item) {
var q:int = Number(property.@quantity);
var p:Number = Number(property.price);
var itemTotal:Number = q * p;
total += itemTotal;
trace (q + ” ” + property.menuName + ” $” + itemTotal.toFixed(2))
}
trace (“Total: $”, total.toFixed(2));

Assignment

myXML.item[0].menuName=”regular burger”;
myXML.item[1].menuName=”small fries”;
myXML.item[2].menuName=”medium cola”;

myXML.item.(menuName==”regular burger”).@quantity = “2″;
myXML.item.(menuName==”small fries”).@quantity = “2″;
myXML.item.(menuName==”medium cola”).@quantity = “2″;

var myXML:XML =
<order>
<book ISBN=”0942407296″>
<title>Baking Extravagant Pastries with Kumquats</title>
<author>
<lastName>Contino</lastName>
<firstName>Chuck</firstName>
</author>
<pageCount>238</pageCount>
</book>
<book ISBN=”0865436401″>
<title>Emu Care and Breeding</title>
<editor>
<lastName>Case</lastName>
<firstName>Justin</firstName>
</editor>
<pageCount>115</pageCount>
</book>
</order>

myXML.book –> XMLList (direct children of myXML object that has element name “book”)
myXML..lastName –> XMLList (any descendant that has element name “lastName”)
myXML.book.author.lastName –> XMLList
myXML.book[0]
myXML.book[0].title
myXML.child(“book”).title
myXML.book.(@ISBN=’0942407296′)
myXML.book[0].@ISBN –> output 0942407296
myXML.book.(title==’Emu Care and Breeding’)
myXML.book.(pageCount > 100)
myXML.book.(title.toString().search(‘Emu’) > -1)

var myXML:XML =
<order>
<item id=’1′ quantity=’2′>
<menuName>burger</menuName>
<price>3.95</price>
</item>
<item id=’2′ quantity=’2′>
<menuName>fries</menuName>
<price>1.45</price>
</item>
</order>;

var total2:Number = 0;
for each (var prop:XML in myXML.item) {
total2 += prop.@quantity * prop.price;
}

http://www.devx.com/Java/Article/21383/1763/page/1

Leave a comment Continue Reading →

Flex 3 – Advanced DataGrid

Since I am leading the reporting team in Adconion, I would like to spend sometime on the new component “Advanced Data Grid” from Flex 3. Being excited to know that this component has added many cool features on top of the DataGrid, I would like to find out whether it meets all my needs. Before I introduce it to my company, I would like to write an example to exercise its power first. The example below will demonstrate some documented features in Advanced Data Grid and some custom features that I would like in my project. To make the example prettier, I would use Yahoo Flex Skin.

Yahoo Flex Skin

yahooflexskin.JPG

Apart from providing you flex skin, Yahoo also shares with you some custom-made flex components. Among them, I would like to look into Yahoo Map once I have time. (http://developer.yahoo.com/flash/maps/examples.html). However, it is out of my scope today.

What I want to show you in my example
There are some cool features I want to try out in this example. Here is the list:

  1. Grouping
  2. Tree view with “Expand All” and “Shrink All” features.
  3. Multiple columns sorting
  4. Auto-completed search filter
  5. Smart resizing data grid
Leave a comment Continue Reading →

Why Flex for RIA, no AJAX?

Here is the list of reasons why I chose Flex for the RIA development.

  1. Write Once Deploy Everywhere – Flex generates SWF that runs on top of Flash Player VM and behaves consistently across different browsers, even mobile phones later. With this, all the browser compatibility issues are basically offloaded by Adobe.
  2. Solid programming model with rich widgets and libraries.
  3. AMF makes Flex object to Java POJO communication possible. No need to use verbose XML – Check out BlazeDS.
  4. Flex IDE is a plugin in Eclipse that gives stepwise debugging, UI design console, code completion and more. Working with Actionscript is like Java.
  5. Flex SDK is open source and free.
  6. Great support on video streaming
  7. Integrate with HTML, Javascript and CSS, so it is not invasive adoption.
  8. Support offline application via AIR – Adobe has been working on the Adobe Integrated Runtime (AIR) that allows for using existing web application development skills to build and deploy desktop applications. AIR is still in early development, but promises to allow developers to use their newly learned Flex skills to build desktop applications. No need to learn Swing, Applet…etc.
  9. Provide several RPC methods like HTTPService, WebService, AMF and JSON. AMF is 10x faster than SOAP. James Ward developed his Census Flex application to provide performance benchmarks for the different RPC methods in the mainstream RIA technologies. (Download)
  10. You can keep the state in the Flex app and have your server completely stateless.
  11. More to come! :)

 

Leave a comment Continue Reading →

Flex – Performance Guidelines

Flex Performance Guidelines

  1. Avoid nested container - it is resource intensive for Flex to figure out the x,y of the elements via sizing and measuring algorithm. Tips: hard-code x,y and width and height.
  2. Naviagtor containers have built-in deferred instantiation – eg. Accrdion, TabNavigator, ViewStack.
  3. Progressive layout improves initialization experience using queued creationPolicy in the deferred instantiation architecture. UI element are loaded in the successively fashion. Progressive layout does not quantifiably reduce application startup time, but it significantly improves the perceived startup time.
  4. Handle large data sets may make your Flex application appears slow. You can use pagination or other methods to address this. (reference).
  5. Improve charting performance
    • All charts cache intermediary values in the transformation from data to screen, so that only the minimum amount of recalculation occurs in response to any change to the data or chart.
    • The most expensive actions to perform in Flex charts is forcing a chart to redraw an axis, or forcing a chart to recalculate its labels. So, change dataProvider is costly in this sense.
  6. Use Runtime Shared Library (RSLs) to shrink the size of your application’s resulting SWF file. Because once you externalizing shared assets into standalone files, you can separately download and cache on the client. (reference)
  7. Use Profiler: getTimer() to record the time in millisecond in ActionScript.

Reference

Good Video: https://admin.adobe.acrobat.com/_a300965365/p71169528/

Leave a comment Continue Reading →

Flex Architecture – MVCS

I have come across an excellent article from Joe Berkovitz that talks about a “right” way to architect your application. I like this article a lot. It gives me a good refresh of what I have learnt in these years. I decide to spend a little time to go over some of the key points from this article.

Isolation and encapsulation are the keys. Isolating concerns opens the door to accommodating change, because it puts boundaries on the ripple effects of changes within the code. Isolation makes modular development and testing easy. Isolation makes it easy to have different people work on different pieces. Finally, isolation makes it easier to envision ways of improving the user experience that are generalized and powerful.

The first rule I learnt in component design is “Reduce coupling and increase cohesion”. The goal of this is to reduce the amount of interactions among components and in turn promote reusability and reduce complexity. This concept proven to be useful in my career life. The tricky part of it is to draw a line around a component and provide a clear responsibilities (ie. via interface) and interactions of it (ie. communication). A good architect can make it very clean and simple whereas a novice designer cannot make a component reusable without associated with different dependencies. In this article, the author focuses on the design in UI, but the rules apply to all aspects of design. A term “MVCS” is mentioned that adds “Services” to the common known MVC pattern. Service is responsible to encapsulate the communication with the external world. Here is the diagram of the MVCS:

MVCS

In MVCS, Model captures the state of the application. It doesn’t know any of the non-Model classes and it carries no intelligence (ie. no function but read/write for properties). According to the author:

In general, avoid functions, since they introduce more complexity, and stray interactions with non-Model objects can creep in. Complex logic for model objects usually belongs in the Controller.

However, from what I have learnt, there are certainly logics belongs to the model classes. A domain model without any behavior is termed as “Anemic Domain Model” by Martin Fowler. “…this is an anti-pattern because it’s so contrary to the basic idea of object-oriented design; which is to combine data and process together.” When I look closer, this is the model that supports the UI but not the domain model in the server side. Is the same rule applied in this area as well? On the other hand, how the model notifies the View if its state is changed? In Flex, you can define your properties as “Bindable”. Whenever the state of the model is change, an event will be fired to notify the associated View.

View references objects in Model via data binding. Bindings are used to move data from sources that change into destinations that need to be change-aware. Besides,View captures users interaction (via event handler registration) and hands off action to Controller (via calling Controller’s function in event handler). Controller, like Session, is application-wide object that should be obtained through a static instance variable to avoid having to pass all such objects all over the application.

Controller is the center of the application. It is the place where application logic and behavior are implemented. It is the only part of the architecture that communicates with all the other parts, thus relieving the View, Model, and Service packages from needing to know too much about each other.

Service objects make up a layer of objects that handle all communication with the outside world like Web Service and Http calls. Besides, Services act as factories for Operation objects; it is actually these Operations that do the work of communication by initiating requests, handling responses, and dispatching status and completion events. Why not just do it internally rather than generate the Operation for controller to call? According to the author, it gives more flexibility. However, I don’t see such much usage of this flexibility. So, I still prefer doing simple Service’s method call… :)

 

fig_09.gif

Leave a comment Continue Reading →