Performance Testing
The goal is to identify the bottleneck in your code and optimize it if possible. 

1. Test against a single box - white box testing.
2. Bottleneck can happen in different levels:
   • Network
   • Application (code efficiency)
   • Database
   • OS (memory, CPU, IO, Swap) 
3. Tuning
   • Start at application and database level 
   • For the bottleneck that you fixed, you can use JUnitPref to check this out. JUnitPref enhances unit test via putting in load and tested test capabilities
   • Modify one variable at a time and redo the measurements

If, after tuning the application and the database, the system still doesn’t meet its expected goals in terms of performance, a wide array of tuning procedures is available at the all the levels discussed before. Here are some examples of things you can do to enhance the performance of a Web application outside of the application code per se but still continue on a single box configuration:

1. Caching (Web cache - Squid, DB cache..etc)
2. Publish highly-requested Web pages statically, so that they don’t hit the database.
3. Scale the Web and database servers vertically, by adding more hardware resources (CPU, RAM, disks)
4. Increase the available network bandwidth
5. Design change

If you cannot optimize your app much, you can use the result you get from the performance test and figure out how many boxes you need to satisfy the business requirement. It is time we turn to load testing.

Load Testing
Simulate the load that you want your system to reach and see how your system behave. The goal of load testing is:

1. Expose bugs that do not surface in cursory testing, such as memory management bugs, memory leaks, buffer overflows, etc.
2. Ensure the application meets the performance baseline established during the performance testing.
3. Requirement: You need a large datasets available for testing.

Although performance testing and load testing can seem similar, their goals are different. On one hand, performance testing uses load testing techniques and tools for measurement and benchmarking purposes and uses various load levels. On the other hand, load testing operates at a predefined load level, usually the highest load that the system can accept while still functioning properly. Note that load testing does not aim to break the system by overwhelming it, but instead tries to keep the system constantly humming like a well-oiled machine. Here is the key things in the load testing.

1. Test against more than 1 box - black box testing.
2. Tool you can use is JMeter and Grinder

What you can do the solve the bottleneck in this level?

1. Scale the Web server farm horizontally via load balancing.
2. Scale the database servers horizontally and split them into read/write servers and read-only servers, then load balance the read-only servers

Stress Testing
The goal is to push your system to the limit and see what will be broken. The main purpose behind this madness is to make sure that the system fails and recovers gracefully — this quality is known as recoverability.

• double the baseline number for concurrent users/HTTP connections
• randomly shut down and restart ports on the network switches/routers that connect the servers (via SNMP commands for example)
• take the database offline, then restart it
• rebuild a RAID array while the system is running
• run processes that consume resources (CPU, memory, disk, network) on the Web and database servers