The key goals of load testing are:

1. Find out whether your website can support the expected # of concurrent users.
2. At what load does the app break?

To do that, you normally follow these steps:

1. Identifying the primary user path
2. Identifying the expected # of concurrent users. (Both now and future)
3. Set up virtual users to hit the app (load generation capability is the key factor to pick the right tool. You don’t want too much hardware investment to generate the load you want, right? )
4. Run the test
5. Analyze the result (throughput under load, avg response time under load)

Challenges to load test Flex app

We have a web application that has Flex frontend talks to J2EE web application backend via AMF. How do we load and stress test this system? We certainly can just perform load test against our backend. However, it may need to expose our service via Servlet and load test it as typical restful web service. If we want to simulate Flex and load test through AMF. We need to find a way to capture the AMF requests from Flex client and replay it in our load testing suite. To do that, we can use Charles http proxy to capture the AMF request and tell JMeter to load test our application via replaying this AMF request. This  article can give us the detail. However, Charles is a commerical product. If you want a free solution, you can try this.

I know that JMeter comes with proxy to record the request. You can try it out to see whether you can exam the AMF message. Let me know if it works!

JMeter

Load is all of the users using your web site at a point in time. Load includes users making requests to your web site as well as those reading pages from previous requests. However, we do need some way of differentiating between all clients and those clients actually making requests to our web site. We use the terms concurrent load and active load (making request) to make this distinction. We use the JMeter to help us to generate the load to our system. In term of load generation, we should make sure that we can simulate the peak load. JMeter is a great load testing tool. I have heard that Google is using this to load and stress test its application.

To use JMeter, follow the steps below:

1. Set up Thread Group – use to model concurrent virtual users and decide how you want the load be generated.
   • Number of threads.
   • The ramp-up period (it tells JMeter the amount of time for creating the total number of threads). At the beginning of a load test, if the ramp-up period is zero, JMeter will create all the threads at once and send out requests immediately, thus potentially saturating the server and, more importantly, deceivingly increasing the load. That is, the server could become overloaded, not because the average hit rate is high, but because you send all the threads’ first requests simultaneously, causing an unusual initial peak hit rate. The rule of thumb for determining a reasonable ramp-up period is to keep the initial hit rate close to the average hit rate.
   • The number of times to execute the test.
   • If the client machine running JMeter lacks enough computing power to model a heavy load, JMeter’s distributive testing feature allows you to control multiple remote JMeter engines from a single JMeter console.
2. Introduce user think time
3. Specify response-time requirements and validate test results.

To get familiar with it, here are some articles I found it useful

1. Stress testing with JMeter by Daniel Rubio – Linux.com
2. Loading testing with Apache JMeter by Kulvir Singh Bhogal, – Devx.com
3. JMeter distributed testing
4. JMeter recording testing
5. Load test your drupal app scalability with JMeter – Part 1, Part 2
6. Load testing with JMeter – powerpoint – very good!
7. Scalability Factors of JMeter in Performance Testing – response size, response time, protocol, hardware configuration, load generating tool architecture and configuration, complexity of client-side processing.
8. JMeter Tips – Javaworld

 Reference

1. Someone has written a nice article in sys-con to talk about how to load test Flex application.
2. Use FlexUnit for stress testing
3. Test Remote Data Service via FlexUnit
4. Load testing with log replay – interesting idea

My company needs a reporting engine but it doesn’t want to go for the expensive commerical ones like MicroStrategy. In fact, I don’t know why we need to pay so much because there are tools out there for FREE. As usual, I googled the Net and found out two seemingly promising open source reporting solution.

1. Pentaho Reporting
2. Jasper Reporting

Both of them are bundled with a suite of tools related to OLAP, Data Mining, ETL.. etc. To me, I just want an non-invasive reporting engine that can easily integrate into our architecture. To my dismay, I found out Pentaho doesn’t go this route. It basically gives you a reporting server configured. You could build your reports and deploy them following the manual. However, I hardly see a reporting solution that could satisfy all the business requirements without customization. All I expected from Pentaho is a jar file with documents that shows me how to use its api to generate reports in different formats and how to integrate with our database. I have attempted to look into the code and extracted the stuff I want from Pentaho. However, I found out the engine is actually not powerful. To strip out the workflow part, it is basically a simple SQL executor that later on will render the result according to the UI info embedded in the report definition. What is wrong with that?

1. We want to handle pagination and data streaming as our data volume is huge. In Pentaho, you need to take care these yourself. So, you write your own sql, paginate yourself, stream it yourself if the resultset is huge. Isn’t it what we are doing without it? Apart from that, each report in Pentaho needs a report definition. It supports dynamic sql via token replacement. It is primitive as I want it to support control flow because I may decide what tables to join based on the input filters.
2. On the UI side, Pentaho helps you to render your result into graph, table…etc. Again, I don’t like this UI solution as well. I found that JFreeChart is not as powerful as the Flex solution. I am adopting Flex and it gives me much powerful visualization tool. All I want is to ship my Flex app the data from my query’s result.

How about Japser? Pretty much the same but the good thing of Jasper is that it gives you the jar and document of how to use it instead of a reporting server like Pentaho. So, I can use it as report renderer to generate PDF and Excel like other utility libraries I use. So, what is my final solution?

I finally decide to create my own report definition that my Flex UI can take and render out the reporting interface. So, I don’t need to create form for each report. Apart from that, in my report definition, I have iBatis SQL template embedded. So, I can leverage its dynamic sql feature that supports control flow logic and the auto result class population. Yes, I still need to handle pagination and streaming myself. But, at least, it already saves up my time. The result object populated will return to Flex via AMF. So, I don’t need to marshal and de-marshal it in xml. It saves the processing time and costs less bandwidth. At the end, my solution combines the best in the market:

1. Powerful reporting widgets provided by Flex
2. Fast streaming and RPC protocol – AMF
3. Good dynamic sql generation and mapping tool from iBatis
4. Good reporting rendering tool from Jasper that helps me to do PDF and Excel generation

My solution is more flexible. As I can plugin hibernate map if I don’t want to write my own sql at all. Apart from that, no UI work is needed to deploy a new report unless my generic reporting interface is not enough.

Later, if I really need the workflow engine provided by Pentaho, I can plug it in. Again, the document provided doesn’t give us clear instruction or APIs of how to do it.

Reference

Below are references I used to build my solution:

1. Flexible reporting with JasperReport and iBatis
2. How Kodo JPA handles large result set (its optimization guide is good reference even you may not use Kodo)
3. Process Large Result Sets in Java Web Application
4. Streaming architecture

This article is written on top of the great work that Sébastien Arbogast has done. He has written 3 articles that showed you how to wire up Flex, BlazeDS, Spring, Hibernate and MySQL with Maven as build process. I have included his articles below as your reference.

1. The Flex, Spring, and BlazeDS full stack – Part 1: Creating a Flex module
2. The Flex, Spring and BlazeDS full stack – Part 2: Writing the to-do list server
3. The Flex, Spring and BlazeDS full stack – Part 3: Putting the application together

I have found Sebastien’s work as a good foundation for my own project. To contribute back to the community, I will write a series of articles to show you how can customize and extend the todolist sample.

What is in the Part 1 of the series…

1. Enhancements on the Maven build process
   • Leverage RSL to factor our the framework swc, so the size of the application swf will be reduced. Apart from that, I also take advantage of Flash Player Cache that is available after version 9 update 3 to cache the framework libraries.
   • Clean up the Flex and BlazeDS dependencies in POM as the latest version of the sdk is available and the BlazeDS dependencies are officially available.
   • Include some common reports for maven site generation
   • Embed Jetty web server in the build process for quick deployment and testing
2. Document how to get the sample up on Eclipse for development
   
3. Use Mate as Flex framework
   • Restructure ToDoList sample to leverage Mate framework
   • Factor out Mate as RSL and integrate it with Maven build process via Flex-mojo plugin.

What are in the coming articles…

1. In part 2 of this series, I will show you how to use flex-mojo to build a modular Flex application.
2. In part 3 of this series, I will show you how to test your flex app via FlexUnit (Unit test) and FlexMonkey (Functional test)
3. In part 4 or this series, I will work on server side. I am planning to add monitoring, caching and security to the server side.

Review “ToDoList” sample

Before I start my journey, let me highlight what Sebastien has done first:

1. Sebastien’s sample demonstrates how to use Maven as a build process. There are 3 parts or subprojects in his sample. They are:
   • todolist-config (configuration files shared by other subprojects)
   • todolist-ria (Flex frontend)
   • todolist-web (Server side that supports the Frontend)
2. All these subprojects are considered as modules of the main project (root POM). Finally, they are combined together into war artifact and ready to deploy to Tomcat or other J2EE webapp server.
3. Flex frontend and backend communicate through a binary RPC protocol – AMF. AMF is considered to be the simplest and fastest remoting approach available in Flex. Recently, Adobe has released BlazeDS as an open source implementation of AMF spec. In this sample, BlazeDS is used. To use BlazeDS, there are few things you need to do:
   • Externalize your POJO service via BlazeDS. This sample shows you how to integrate BlazeDS with Spring
   • Make BlazeDS endpoints availabe to the Net via Servlet.
   • Have frontend and backend shared the same BlazeDS configuration files.
4. In this sample, you can also find out how to use flex-mojo maven plugin to compile the Flex frontend code into swf. Apart from flex-mojo plugin, there are other two good plugins worth to mention:
   • maven-assembly-plugin - can be used to bundle all the files under a directory into a zip file. It is used by todolist-config to bundle all the configuration files (service-config.xml and remoting-config.xml) into a zip during the package phase.
   • maven-dependency-plugin – can be used to unpack the zip file and move to the place you want. It is used by todolist-web to unpack the config zip during the generate-resources phase.

Enhancements on maven POM

I have modified the sample’s maven pom as follows:

• Link to new repository “Sonatype Forge” in the root POM. So, I can use the new version of flex-mojo and simplify the todolist-ria adobe framework dependencies. Apart from that, I also take away the private repository from Sebastein because BlazeDS libraries are available in official maven repository (Note: The BlazeDS libraries available in official maven repo are in version 3.0 instead of 3.0.0.544. So, you need to modify the webapp pom correspondingly).

    <repositories>
        <repository>
            <id>flex-mojos-repository</id>
            <url>http://svn.sonatype.org/flexmojos/repository/</url>
            <releases>
                <enabled>true</enabled>
            </releases>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </repository>
    </repositories>

    <pluginRepositories>
        <pluginRepository>
            <id>flex-mojos-repository</id>
            <url>http://svn.sonatype.org/flexmojos/repository/</url>
            <releases>
                <enabled>true</enabled>
            </releases>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </pluginRepository>
    </pluginRepositories>

• Because I link to Sonatype repository, I can have my todolist-ria depends on one flex-framework pom dependency instead of all the swc dependencies. Note that the pom dependency is a way to factor out all the adobe swc dependencies that makes your pom easier to maintain.

        <dependency>
            <groupId>com.adobe.flex.framework</groupId>
            <artifactId>flex-framework</artifactId>
            <version>3.1.0.2710</version>
            <type>pom</type>
        </dependency>

• I include mysql driver as dependency in my webapp pom. I think it is cleaner to bundle it in war. I have also added jetty plugin in the POM so you have a web server embedded in the build process. With this, you can run this sample application right after you check it out from svn (assume you have maven 2 installed). To start jetty, you can issue the following maven command under your webapp project.

project_root> mvn clean install
project_root/jp-web> mvn jetty:run-war

• I have included some reports that will be shown after site generation. You may not be able to do mvn site-deploy because it is linked to my web hosting site. However, you can modify it for your own sake.

Get the sample up on Eclipse

To develop on Eclipse, you can follow the steps below:

1. Create Eclipse project file via running the command below at the project root. This will create 2 eclipse projects. One for todolist-ria and one for the webapp. You noticed that I use the -Declipse.downloadSource=true to include the source files of my dependencies in my eclipse project. Therefore, I can get to the source code if needed.

mvn -Declipse.downloadSource=true eclipse:eclipse

1. Import the projects into Eclipse
2. Add new variable M2_REPO and set it equals to [home]/.m2/repository
3. If you have installed Flex Builder plugin to your Eclipse, you can Add Flex Project Nature to the todolist-ria project.
   • Select Application Server Type: J2EE
   • Put check on “Use remote object access service” with LiveCycle Data Service selected.
   • Set up the path. I have my tomcat installed under C:\tools with default 8080 as port. You should make the changes if you installed it differently.
   • 
   • Remove the generated main.mxml under the src folder.
   • Set index.mxml under src folder as default Flex application file to run.
   • Use Default Flex SDK in Flex Compiler Configuration instead of Server Flex SDK
   • Right click and select Recreate HTML Template if you see error.
   • After all these, you have configured your Flex application pointing to the webapp server and sharing the BlazeDS configuration files. You can verify in Flex Compiler Configuration’s Additional Compiler Parameters. See whether you see this: -services “C:\tools\tomcat-6.0.16\webapps\jp\WEB-INF\flex\services-config.xml” -locale en_US
   • Move the war to your tomcat’s webapp folder and start it under remote debugging setting. If you are using window, set DEBUG_OPTS=-Xdebug -Xrunjdwp:transport=dt_socket,server=y,address=8787,suspend=n under your bin/catalina.bat.
   • Start your webapp via bin/startup.bat
   • Put breakpoint under TodoServiceImpl save method and start remote debugger on localhost:8787
   • Right click the index.mxml and Run As Flex Application.
   • Add a new entry and save it on the flex app. You should see your remote debugger halt at the breakpoint for you to debug.
   • Now you can change your flex code and test it out without leaving your Eclipse. However, if you modify the service in webapp, you need to run “mvn clean install” and deploy the war to the tomcat before your flex code can call your server-side code via AMF.

Use Mate as Framework

If you are not familiar with Mate, click the image below that moves you to a nice presentation.

What did I do to restructure the todolist sample to make it Mate app?

Download

I have made my work available at: www.solutionhacker.com/wp-content/uploads/todolist-jp-modified.zip

Reference

Below are the references I used for the article:

1. Flex mojo compiler user guide
2. Flex mojo dependency scope rules
3. Flex 3 feature introduction: Flex 3 RSL
4. Improving Flex application performance using Flash Player Cache
5. FNA archetype projects 

1. Write Once Deploy Everywhere – Flex generates SWF that runs on top of Flash Player VM and behaves consistently across different browsers, even mobile phones later. With this, all the browser compatibility issues are basically offloaded by Adobe.
2. Solid programming model with rich widgets and libraries.
3. AMF makes Flex object to Java POJO communication possible. No need to use verbose XML – Check out BlazeDS.
4. Flex IDE is a plugin in Eclipse that gives stepwise debugging, UI design console, code completion and more. Working with Actionscript is like Java.
5. Flex SDK is open source and free.
6. Great support on video streaming
7. Integrate with HTML, Javascript and CSS, so it is not invasive adoption.
8. Support offline application via AIR – Adobe has been working on the Adobe Integrated Runtime (AIR) that allows for using existing web application development skills to build and deploy desktop applications. AIR is still in early development, but promises to allow developers to use their newly learned Flex skills to build desktop applications. No need to learn Swing, Applet…etc.
9. Provide several RPC methods like HTTPService, WebService, AMF and JSON. AMF is 10x faster than SOAP. James Ward developed his Census Flex application to provide performance benchmarks for the different RPC methods in the mainstream RIA technologies. (Download)
10. You can keep the state in the Flex app and have your server completely stateless.
11. More to come!