Tag Archives: caching

Powerful combination: JMX + Annotation + AOP

What is AOP?

AOP is a way to modularize cross-cutting concerns. Ok, what does “modularize” really mean? Modularization is the encapsulation of a unit of functionality. It is exactly what “Class” is doing in OO world. How about “cross-cutting concerns”? Basically it means any functionalities that span multiple modules/ classes. They include Transaction Management, Security, Caching, Performance Monitoring and etc. To understand how AOP works, we first look at the common terms in this area:

  1. Join point – An identifiable point in the execution of a program like method invocation, exception thrown.
  2. Pointcut – Program construct that selects join points and collects context at those points. AspectJ has a rich pointcut expression language!
  3. Advice – Code to be executed at a join point that has been selected by a pointcut.

To me, I found it easier to understand these terms if I consider join point as event generated point in code, pointcut as a way to define what events to be captured and advice as event handler.

AOP is indeed a powerful way to factor out system or infrasturcture-related code from the business oriented code. Typically, we use it to take care of transaction, security and profiling aspects. But it doesn’t stop you putting creativity in this domain. With a bit more creativity, you can also do the following::

  1. Exception translation – checked to runtime
  2. Catch ConcurrencyFailureExceptions and transparently retry if an idempotent operation fails with, for example, a deadlock loser exception.

How I use Spring AOP in my project?

I have been told to report the elapsed time for all calls to the database. If I don’t know how to use AOP, I may end up putting code to measure time for every JDBC calls. It ends up tangling performance monitoring code with my main line business logic and the same logic will be scattered everywhere in my data access code. Bad!! That is why we need to know how to factor out the performance monitoring code into an aspect like below:

Here we use AspectJ annotation approach to implement the aspect. “Around” is to intercept start and end of any repository method. Here is what states in Spring 2.5 reference:

Spring 2.0 introduces a simpler and more powerful way of writing custom aspects using either a schema-based approach or the @AspectJ annotation style. Both of these styles offer fully typed advice and use of the AspectJ pointcut language, while still using Spring AOP for weaving.

If you use AspectJ annotation, you need to put <aop:aspectj-autoproxy/> in your application-context.xml. The limitation of Spring proxy-based AOP is that it is limited to method invocation interception. To get around that, you can use AspectJ syntax in your pointcut expression. You don’t need to build the application with ajc (the AspectJ compiler) even you are using AspectJ syntax. Spring AOP can also understand @AspectJ aspects. I strong suggest you use Annotation driven AOP because it is cleaner and simplier. Working with AOP, I have faced 2 questions.

  1. How to select the methods that I want to intercept without hardcoding the method or package name in my pointcut expression. So, my aspect or pointcut doesn’t contain application specific information – Look into annotation and AOP section.
  2. How to turn on and off AOP without restarting the web application? I would use JMX. Look into “What is JMX” section. 

Annotation and AOP

Annotation provides a better way other than code signature for selecting join point that leads to creating loosely coupled aspect. In fact, you can see annotation as another signature of a method in other dimension. And a method can have multiple annotations and each concern just bother its own annotation. It is called multidimensional signature space. For example,

@Authentication(“bankOperation”)
@Transactional(REQUIRED)
public void credit(){…}

Pointcut uses annotation to capture join points. For example:

execution(@Transactional * *.*(..)) Execution of a method annotated as Transactional
execution((@Trasactional *) *.*(..)) Execution of a method that returns object annotated as Transactional
execution(* (@Transactional *).*(..)) Execution of a method defined for type annotated as Transactional

Selection can use Annotation types and Annotation values.  What is more, annotation values can be used in Advice implementation.

Here is a great video from Parleys that talked about “Leveraging Annotation with AOP”. I have included some key points Ramnivas made here:

  • Write you pointcut in a smart way to avoid annotation mess. Try to use naming and package convention to help you. For example, if you want to write app log for all public facing service method, you can use “public” with package name containing “service” wildcard to help you.
  • If you really need to use annotation like @Transaction that designer has no way to define the pointcut beforehand, use annotation to describe what the join point is but not how to handle it. So, your transaction aspect only need to worry annotation @Transaction and decouple from the application.
  • You can piggyback annotation. For example, you can make all entities auditable via declare @type: @Entity *: @Auditable; 

How does Spring AOP work internally?

The magic behind AOP is the concept of Proxy/ Decorator/ Interceptor/ Filter pattern. To me, all those patterns are conceptually the same. They all try to present itself as target object (thru implementing the same interface), intercept method call and execute injected logics. And you can have more than one interceptors invoked in series. In Spring AOP, there is one thing we need to pay attention:

However, once the call has finally reached the target object, …any method calls that it may make on itself, such as this.bar() or this.foo(), are going to be invoked against the this reference, and not the proxy. This has important implications. It means that self-invocation is not going to result in the advice associated with a method invocation getting a chance to execute. To handle this, either you refactor your code such that the self-invocation does not happen (best approach) or you make self invocation call thru proxy like ((Pojo) AopContext.currentProxy()).bar() (invasive approach b/c it totally couples your code to Spring AOP, and it makes the class itself aware of the fact that it is being used in an AOP context, which flies in the face of AOP. Avoid using it).

However, it must be noted that AspectJ does not have this self-invocation issue because it is not a proxy-based AOP framework.

What is JMX?

In short, it is a way to enable management and monitoring of Java applications over a generic API. JMX has a simple architecture that contains instrumentation level, agent level and distribution service level. In instrumentation layer, we register MBean to the MBeanServer. In simple term, In simple term, MBean is a  JavaBean with defined management interface that exposes attributes and operations to the world. MBeanServer acts as a broker to decouple communication among application MBeans and/or remote clients.

Combine AOP and JMX

AOP is statically defined and intercept at the runtime. It is hard to take this out or add another aspect in after you start your machine. However, with JMX, you can enable and disable it via skipping the aspect code. :cool: On the other hand, you can also use JMX to configure and report SLA metrics like configure thresholds and send notifications of violations. That sounds very interesting to me. There are other interesting usages mentioned in the Parley’s video as well:

  1. Service blocking – throw an exception if particular service you don’t want to user to use it for a period of time esp during maintenance time.
  2. Caching management – I am currently using interceptor pattern and IoC to intercept dao method calls for cache lookup. 

Reference

  1. JavaOne 07 – JMX, AOP and Spring (Nice Presentation)
  2. Parley’s AOP and JMX (Video)
  3. Simplifying Enterprise Applications with Spring 2.0 and AspectJ
  4. Workflow Orchestration Using AOP
  5. Performance Monitoring with AOP and JMX

 

Leave a comment Continue Reading →

Speed up your website via caching

Introduction

Caching is a crucial performance tuning strategy, especially your system has high read to write ratio. You can perform caching strategy at different levels from client browser cache all the way to disk cache at server side. Lets take a brief look at where we can cache based on the invocation path for a request to be fulfilled:

  1. Client browser cache
  2. CDN network
    • A CDN is a network, like Akamai, where a web site such as JustProposed.com can offload high-bandwidth static files like photos and videos to another network, so that my web site doesn’t need to have such huge bandwidth to run. Since bandwidth is a major expense, especially as we grow or when we get slashdotted (in which case we run out of bandwidth), a CDN has looked interesting. However, Akamai is too expensive for us to use. So, we will go for the free network, Coral CDN.
    • Apart from the bandwidth, JustProposed.com has lots of non-USA users who sometimes find my site slow to use. So, CDN network gives us proximity advantages.
    • To use Coral CDN, you simply append nydu.net:8080 to the end of the hostname in the URL of your expensive resources. For example, http://www.justproposed.com/raydoris/myphoto.jpg to http://www.justproposed.com.nydu.net:8080/raydoris/myphoto.jpg
    • Coral looks great, the only problem I have with it is that it’s running on a high port, so that people behind proxy servers that don’t automatically support http over anything bug port 80 will have problems. To use Coral, follow this instruction.
  3. Reverse proxy server and content accelerator – Squid
    •  Why not use Apache as reverse proxy instead of putting Squid in front of Apache? Here are some of the benefits of this setup. The main reason is that Apache spawns out a new process per request that eats up lots of resources.
    •  

 

There are several things that you need to look at when you go for caching approach:

  1. What to cache? The data used by most web applications varies in its dynamicity, from completely static to always changing at every request. Everything that has some degree of stability can be cached. However, I always pick the ones that are most frequently access and/or expensive to compute and retrieve to cache because of the limited resource (ie. memory).

Application level caching (for J2EE)

JCS – Java Caching System

  1. Configuration
    • To understand the power of JCS, the best way is to look at its configuration file. To find out what is each configurable parameter does, take a look at this article.
  2. Integrate with Spring
    • To use JCS with Spring, take a look at this article. It talks about how to create a wrapper or Interceptor for your DAO and inject it to your service for caching purpose. To implement cache as an aspect with full control of what and how to cache, it doesn’t use the declarative Spring module caching approach. Regular dependency injection can do the trick!
  3. Distributed caching
    • JCS is a front-tier cache that can be configured to maintain consistency across multiple servers by using a centralized remote server (client-server) or by lateral distribution (peer-to-peer) of cache updates. 

Reference

  1. Speed up your LAMP stack with lighhttpd
  2. Squid and Apache on the same server – have squid listened on port 80 and apache listened on port 8080
  3. Squid configuration variable

 

Leave a comment Continue Reading →

Tomcat Performance Tuning

Most companies I have worked for use Tomcat as Servlet Container. It is de facto standard just like how Apache been used as Web Server. However, most of us just drag our war file to the webapp folder and use Tomcat with all the settings as default out of the box. It works fine in development environment but may not in production. This article will give you advice in several areas:

  1. Production Tomcat Architecture
  2. Tuning tomcat for performance
  3. Resolving problems which affect availability

 Production Tomcat Architecture

In production Tomcat relies on a number of resources which can impact its overall performance. Understanding the overall system architecture is key to tuning performance and troubleshooting problems.

  1. Hardware: CPU(s), memory, network IO and file IO
  2. OS: SMP (symmetric multiprocessing) and thread support
  3. JVM: version, tuning memory usage, and tuning GC
  4. Tomcat: version (example, Tomcat 6 supports NIO)
  5. Application: Application design can have the largest impact on overall performance
  6. Database: concurrent db connection is allowed (pooling and object caching)
  7. Web Server: Apache can sit in front of Tomcat and serves the static content. It also can do load balancing across multiple Tomcat instances.
  8. Network: Network delays.
  9. Remote Client: How fast is the communication protocol? Content can be compressed. 

Performance Tuning

How to measure and test performance

  • Request latency is key b/c it reflects the responsiveness of your site for visitors.
  • Test environment should match production as closely as possible.
  • The data volume is important to simulate in database side.
  • Test HTTP requests with different request parameters (test corner cases)
  • Use load test to simulate the traffics (ex. JMeter)
  • Final tests should be over longer periods like days because JVM performance changes over time and can actually improve if using HotSpot. Memory leaks, db temporary unavailable, etc can only be found when running longer tests.

JVM version, memory usage and GC

  • Sun Java 1.3 and later releases inlcude HotSpot profiling optimizer customized for long running server application.
  • Tomcat will freeze processing of all requests while the JVM is performing GC. On a poorly tuned JVM this can last 10′s of seconds. Most GC’s should take < 1 second and never exceed 10 seconds
  • Tune the -Xms (min) and -Xmx (max) java stack memory (set them to the same value can improve GC performance)
  • Make sure the java process always keeps the memory it uses resident in physical memory and not swapped out to virtual memory.
  • Use -Xincgc to enable incremental garbage collection
  • Try reducing -Xss thread stack memory usage

Tomcat version and configuration

  • Tomcat 6 supports NIO.
  • Set “reloadable” false – remove unnecessary detection overhead
  • Set “liveDeploy” to false – liveDeploy controls whether your webapps directory is periodically checked for new war files. This is done using background thread.
  • Set “debug” to 0
  • Set “swallowOutput” to true – This makes sure all output to stdout or stderr for a web application gets directed to the web application log rather than the console or catalina.out. This make it easier to troubleshoot problems.
  • Connector configuration – minProcessor, maxProcessor, acceptCount, enableLookups. Don’t set the acceptCount too high b/c this sets the number of pending requests awaiting processing. It is better deny few requests than overload Tomcat and cause problems for all requests. Set “enableLookups” to false b/c DNS lookups can add significant delays.

Database connection pool

  • We use connection pool provided by Spring instead
  • Using middleware to persist and cache objects from your database can significantly improve performance b/c of fewer db calls, less thrashing of the JVM for creation and subsequent GC of object craeted for resultset.

Application design and profiling

  • If the data used to generate a dynamic page rarely changes, modify it to a static page which you regenerate periodically.
  • Cache dynamic page
  • Use tool like JProble to profle your web applications during development phase
  • Look for possible thread synchronization bottlenecks
  • Date and Time thread synchronization bottleneck 

Troubleshooting

Collecting and analyzing log data

Common problems

  • Broken pipe – For HTTP Connector indicates that the remote client aborted the request. For web server JK Connector indicates that the web server process or thread was terminated. These are normal and rarely due to a problem with Tomcat. However, if you have long request, the connectionTimeout may close the connection before you send your response back.
  • Tomcat freezes or pauses with no request being processed – usually due to a long pause of JVM GC. A long pause can cause a cascading effect and high load once Tomcat starts handling requests again. Don’t set the “acceptCount” too high and use java -verbose:gc startup argument to collect GC data.
  • Out of Memory Exception – look into application code to fix the leak (profile tool can help). Increase available memory on the system via -Xmx. Restart tomcat!
  • Database connection failure – connection used up when traffic is spike.
  • Random connection close exception - when you close your connection twice. First close(), the connection returns to the pool. It may be picked up by another thread. Now, second close() may close a connection that is being used by other thread. Don’t close connection twice, use JDBC Template from Spring to avoid this problem. 

Reference

  1. JavaWorld GC Article
  2. Sun HotSpot Performance Document
  3. Tomcat Performance Slides

  

Leave a comment Continue Reading →