How to start our Architectural Review  

We can review our architecture in 3 different aspects to evaluate its scalability and performance.

1. Deployment and infrastructure
2. Technology Stacks
3. Performance and Scalability Frame

Deployment and Infrastructure

• Do you need a distributed architecture? If not, you better co-locate web and app server to eliminate the overhead via remote communications (ie. serialization cost and network latency). If you really need to separate them, use facade pattern to provide a coarse-grained interface and cache data locally where appropriate.
• What distributed communication should you use? In Java, you have choice like SOAP, XML, REST, Hessian, RMI, EJB…
• Do you have frequent interaction across boundaries? Reduce coupling and increase cohesion via encapsulation and component design.
• What restrictions does your infrastructure impose? Like internal and external firewall, SSL and etc.
• Do you consider network bandwidth restrictions? Evaluate the size of the average request and response and multiple it by the expected concurrent load of users. The total figure should be considerably lower than the available network bandwidth. There are various ways to minimize the client bandwidth like use data paging, enable HTTP 1.1 compression (eg. gzip).
• Do you share resources with other applications? CPU, Memory and IO are 3 major resources for an application.
• Does your design support scaling up and out? Scale out first then up. Avoid server affinity by designing appropriate state and caching mechanisms.

Performance and Scalability Frame

1. Communication
   • Chatty remote call? How you do remoting?
   • Secure communication?  Pick the right key size and encryption algorithm. If you use encryption, where possible (when both parties are known in advance), use symmetric encryption instead of asymmetric encryption. Asymmetric encryption provides improved security but has a greater negative impact on performance. A common approach is to use asymmetric only to exchange a secret key and then to use symmetric encryption.
   • Asynchronous messaging? queue vs topic. Do you make long running call? If yes, you should consider doing it asynchronously via messaging. If the client needs a way to get the result back and pick up for continue processing, register the callback for the return.
2. Concurrency
   • How well your design minimizes contention and maximizes concurrency?
   • Issues: Blocking calls, hold on the lock longer than necessary, inappropriate isolation level, misusing threads, nongranular lock
   • Do you need to execute tasks concurrently? Concurrent execution tends to be most suitable for tasks that are independent of each other. You do not benefit from asynchronous implementation if the work is CPU bound (especially for single processor servers) instead of I/O-bound. If the work is CPU bound, an asynchronous implementation results in increased utilization and thread switching on an already busy processor. This is likely to hurt performance and throughput. Consider using asynchronous invocation when the client can execute parallel tasks that are I/O-bound as part of the unit of work. For example, you can use an asynchronous call to a Web service to free up the executing thread to do some parallel work before blocking on the Web service call and waiting for the results.
   • Do you create threads on a per-request basis? Use thread pool is better option. Threads are shared resources and are expensive to initialize and manage. If you create threads on a per-request basis in a server-side application, this affects scalability by increasing the likelihood of thread starvation and affects performance, due to the increased overhead of thread creation, processor context switching, and garbage collection..
   • Do you design thread safe types by default? If it is unnecessary, use non-thread safe type (eg. use HashMap instead of Hashtable).
   • Do you use fine-grained locks?
   • Do you acquire late and release early? Acquiring late and releasing shared resources early is the key to reducing contention. For example, you may use synchronized block instead of synchronized method to reduce the contention.
   • Do you use the appropriate synchronization primitive? Read/Write lock, Mutex, Semaphore, atomic operation like increment on integer rather than double.
   • Do you use an appropriate transaction isolation level?
   • Does your design consider asynchronous execution?
3. Caching
   • Do you cache data?
   • Do you know which data to cache?
   • Do you cache volatile data?
   • Have you chosen the right cache location? Different cache layers.
   • What is your expiration policy?
4. State Management
   • Keep your server as stateless as possible.
   • If you cannot, pay attention what to put in the session in term of size and amount.
   • Expire the session once the user logout or you set the session expiration time for idle session to timeout.
   • The objects you put in the session need to be serializable.
5. Data structure and algorithm
   • Use the right data structure and algorithm can greatly improve the performance.