If we seriously consider scalability and performance issue, we have to set scalability and performance targets and test if
we have achieved the target.Say, for example, the performance target is average page loading time is 0.5 second on localhost, the scalability target is it can handle up to 200 concurrent users using developer's machine as server.
I would like to recommend 2 free tools to do such tests.
One is JUnitPerf http://www.clarkware.com/software/JUnitPerf.html
JUnitPerf 1.9.1 is the latest release. JUnitPerf consists of a small set of test decorators and related classes that extend the JUnit API; thus JUnitPerf tests extend a JUnit test by using the decorator pattern. We must have pre-existing JUnit tests(we do have) to use JUnitPerf. JUnitPerf allows us to easily load tests and have tests fail if they do not respond in a certain time. However we should be judicious in deciding when to use JUnitPerf. Using JUnitPerf to test every line of code is usually time-consuming and unnecessary. JUnitPerf is most effective in situations where we have specific performance criteria to meet. It is often a good idea to first use a performance-profiling tool, such as JProbe, to identify areas of code that have the highest potential for scalability and performance issues. Then we can focus your JUnitPerf tests on those relatively few bottleneck class methods.
We can use JUnitPerf in conjunction with HttpUnit and JMeter to test performance of a Web site under load. For example, write a set JUnit test that uses HttpUnit tests that simulate a user navigating to a product in a Web catalog. Use JMeter to flood the site with requests, while we run a navigation test under a JUnitPerf timed test and set the test to fail if it does not navigate the site to the product page in 0.5 seconds.
JMeter http://akarta.apache.org/jmeter/ mentioned above is just the second tool I recommend.
Typically, we use JMeter to measure your system performance. At first it may seem that JMeter overlaps with JUnitPerf or HttpUnit. Rather, it complements these tools: We can use JMeter to simulate load while we use JUnitPerf in conjunction with HttpUnit to ensure that your Web application still responds in a timely manner.
We can use JMeter to simulate a heavy load on a your system, server, and network. JMeter has a full multithreading framework that allows concurrent sampling by many threads and simultaneous sampling of different functions by separate ThreadGroups. Thus, we can also use JMeter to test system performance under different load types, such as heavy updates, heavy browsing, heavy transactions, or under different combination of load profiles simultaneously. An advantage of using JMeter is that we get instant visual feedback of system performance with its graphs and splines.
However, The 2 tools above is not a replacement of our TestHarness, but they would be very helpful if we use them properly.