JBoss5 Test Results

Intro

As part of the upgrade to JBoss5, we wanted to run some scalability tests to ensure that LAMS would perform as good or better than the previous JBoss 4.0.2 version we've been using until LAMS v2.3.

So we created a branch on CVS for the JBoss5 upgrade and Marcin has done the necessary modifications to run LAMS2.3 with JBoss5.1. We also had to do changes to our Test Harness to make it work with JBoss5. The Test Harness allows us to simulate a number of concurrent web users going thru a sequence in LAMS (or author a sequence, start a lesson, etc). So we can use it to do massive scalability tests.

LAMS server hardware specs

HP Compaq DC7800 Convertible Minitower P
HP Core 2 Duo E6750 Processor (2.66 GHz, 4 MB L2 cache, 1333 MHz FSB)
Ubuntu 8.04 Server
HP 160GB SATA3.0Gb/sNCQ 10K(16MB)2nd HDD
SATA Cable 2nd CMT Cable Kit
2GB PC2-5300 (DDR2-667)2x1GB Memory
NV GF 8400GS (256MB DH) PCIex1 1st Card
SATA 16X SuperMulti LightScribe1st Drive
HP USB Standard Keyboard
HP USB 2-Button Optical Scroll Mouse
Intel vPro AMT 3.0 Platform SW

Test harness clients hardware specs

Same as the LAMS server.

Important!
As you can see we are using our development desktops to run these test. The idea begin this is to show the level of concurrency that we can achieve with inexpensive hardware. We could have used fast and flashy servers, but we wanted to test what performance teachers could get running LAMS on desktop computers they might have in their schools.

Test configuration and setup

See JBoss5 Configuration

Testing methodology

Each test harness sets a lesson and the number of learners that will go thru it. So in each instance of this test, we have 10 lessons and we keep increasing the number of learners we put in each lesson. So for instance to achieve 500 concurrent users, we run 10 instances of the test harness, that will create a lesson each, and assign 50 learners to each of them. All lessons are run at the same time, so LAMS gets with by 500 simulated web users each of them going thru the activities in the lesson.

Results

We use a set of 10 test harness clients to hit the LAMS.

Concurrent Users Average Response Time (seconds)
500 (10 lessons with 50 learners each) 0.189552782
600 (10 lessons with 60 learners each) 0.281842308
800 (10 lessons with 80 learners each) 3.409386443
1000 (10 lessons with 100 learners each) 7.095551162
1200 (10 lessons with 120 learners each) 11.708171

You can read this as when there's 500 concurrent users, LAMS takes 0.189 seconds to respond a page on average. For 600 concurrent users, it teaks LAMS 0.281 seconds to respond a request on average, and so on.

Analysis

This shows that on a very simple desktop with 1GB RAM, LAMS can run over a 1000 concurrent users with reasonable response times (~7 seconds). For over 1000 concurrent users then a bigger server is required (or your students will get bored and drift off )

Detailed results

In this tables we show more information about this tests.

Concurrent users Test harness 1 Test harness 2 Test harness 3 Test harness 4 Test harness 5 Test harness 6 Test harness 7 Test harness 8 Test harness 9 Test harness 10 Average response
500 0.18441121 0.201907466 0.204111627 0.17733744 0.195068546 0.196534283 0.17419802 0.170333092 0.198519324 0.193106815 0.189552782
600 0.285396573 0.274368421 0.256729583 0.25139871 0.276855242 0.285705242 0.293563394 0.29339471 0.302918459 0.298092742 0.281842308
800 3.446443469 3.384630622 3.47818909 3.381823521 3.341239299 3.44665405 3.425849637 3.375886147 3.351797305 3.461351294 3.409386443
1000 6.878914203 6.563643558 6.755987761 6.643305192 7.414887529 7.236631681 7.259979808 7.473638468 7.341679966 7.386843451 7.095551162
1200 11.22182422 11.37581152 11.48213639 11.27639502 11.32057811 12.02008534 12.08427152 11.98061218 11.99809418 12.32190153 11.708171

Raw results

500 (50x10)

mitch2
 50 learner(s) attended, 50 finished and 0 aborted. Total time: 20 minutes 56 seconds
     [java] 15:24:31,732 INFO [main] TestReporter - Total response time: 763.278 seconds
     [java] 15:24:31,732 INFO [main] TestReporter - Average response time: 0.1844112104373037 seconds
     [java] 15:24:31,732 INFO [main] TestReporter - Average response time of joinLesson calls: 0.57092 seconds

luke1
 50 learner(s) attended, 50 finished and 0 aborted. Total time: 21 minutes 33 seconds
     [java] 15:25:06,929 INFO [main] TestReporter - Total response time: 835.695 seconds
     [java] 15:25:06,929 INFO [main] TestReporter - Average response time: 0.20190746557139408 seconds
     [java] 15:25:06,929 INFO [main] TestReporter - Average response time of joinLesson calls: 0.55344 seconds

luke2
 50 learner(s) attended, 49 finished and 1 aborted. Total time: 21 minutes 52 seconds
     [java] 15:25:28,414 INFO [main] TestReporter - Total response time: 835.633 seconds
     [java] 15:25:28,415 INFO [main] TestReporter - Average response time: 0.20411162677088424 seconds
     [java] 15:25:28,415 INFO [main] TestReporter - Average response time of joinLesson calls: 0.55372 seconds

mitch1
 50 learner(s) attended, 50 finished and 0 aborted. Total time: 22 minutes 34 seconds
     [java] 15:25:50,581 INFO [main] TestReporter - Total response time: 734.177 seconds
     [java] 15:25:50,581 INFO [main] TestReporter - Average response time: 0.17733743961352658 seconds
     [java] 15:25:50,581 INFO [main] TestReporter - Average response time of joinLesson calls: 0.55134 seconds

jun-dir2
 50 learner(s) attended, 49 finished and 1 aborted. Total time: 21 minutes 8 seconds
     [java] 15:24:32,540 INFO [main] TestReporter - Total response time: 802.512 seconds
     [java] 15:24:32,540 INFO [main] TestReporter - Average response time: 0.1950685464268352 seconds
     [java] 15:24:32,540 INFO [main] TestReporter - Average response time of joinLesson calls: 0.55006 seconds

jun-dir1
 50 learner(s) attended, 50 finished and 0 aborted. Total time: 22 minutes 2 seconds
     [java] 15:25:28,534 INFO [main] TestReporter - Total response time: 814.045 seconds
     [java] 15:25:28,534 INFO [main] TestReporter - Average response time: 0.19653428295509415 seconds
     [java] 15:25:28,534 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5536 seconds

antonio1
 50 learner(s) attended, 50 finished and 0 aborted. Total time: 22 minutes 19 seconds
     [java] 15:25:21,142 INFO [main] TestReporter - Total response time: 721.354 seconds
     [java] 15:25:21,142 INFO [main] TestReporter - Average response time: 0.17419801980198021 seconds
     [java] 15:25:21,142 INFO [main] TestReporter - Average response time of joinLesson calls: 0.55592 seconds

antonio2
 50 learner(s) attended, 50 finished and 0 aborted. Total time: 23 minutes 11 seconds
     [java] 15:26:15,216 INFO [main] TestReporter - Total response time: 705.179 seconds
     [java] 15:26:15,216 INFO [main] TestReporter - Average response time: 0.1703330917874396 seconds
     [java] 15:26:15,216 INFO [main] TestReporter - Average response time of joinLesson calls: 0.55182 seconds

daniel1
 50 learner(s) attended, 50 finished and 0 aborted. Total time: 22 minutes 48 seconds
     [java] 15:26:15,325 INFO [main] TestReporter - Total response time: 821.87 seconds
     [java] 15:26:15,325 INFO [main] TestReporter - Average response time: 0.19851932367149758 seconds
     [java] 15:26:15,325 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5538799999999999 seconds

daniel2
 50 learner(s) attended, 50 finished and 0 aborted. Total time: 22 minutes 7 seconds
     [java] 15:25:35,595 INFO [main] TestReporter - Total response time: 799.076 seconds
     [java] 15:25:35,595 INFO [main] TestReporter - Average response time: 0.19310681488641857 seconds
     [java] 15:25:35,596 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5531 seconds


600 (60x10)

mitch1
 60 learner(s) attended, 60 finished and 0 aborted. Total time: 21 minutes 8 seconds
     [java] 15:52:03,199 INFO [main] TestReporter - Total response time: 1415.567 seconds
     [java] 15:52:03,200 INFO [main] TestReporter - Average response time: 0.28539657258064516 seconds
     [java] 15:52:03,200 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5526833333333333 seconds

mitch2
 60 learner(s) attended, 60 finished and 0 aborted. Total time: 21 minutes 1 second
     [java] 15:51:53,551 INFO [main] TestReporter - Total response time: 1360.593 seconds
     [java] 15:51:53,551 INFO [main] TestReporter - Average response time: 0.2743684210526316 seconds
     [java] 15:51:53,551 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5592 seconds

antonio1
 60 learner(s) attended, 60 finished and 0 aborted. Total time: 21 minutes 21 seconds
     [java] 15:51:58,087 INFO [main] TestReporter - Total response time: 1273.122 seconds
     [java] 15:51:58,087 INFO [main] TestReporter - Average response time: 0.2567295825771325 seconds
     [java] 15:51:58,087 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5612666666666667 seconds

antonio2
 60 learner(s) attended, 60 finished and 0 aborted. Total time: 22 minutes 27 seconds
     [java] 15:53:05,677 INFO [main] TestReporter - Total response time: 1247.189 seconds
     [java] 15:53:05,677 INFO [main] TestReporter - Average response time: 0.2513987099375126 seconds
     [java] 15:53:05,677 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5531166666666666 seconds

daniel1
 60 learner(s) attended, 60 finished and 0 aborted. Total time: 22 minutes 38 seconds
     [java] 15:53:39,633 INFO [main] TestReporter - Total response time: 1373.202 seconds
     [java] 15:53:39,633 INFO [main] TestReporter - Average response time: 0.2768552419354839 seconds
     [java] 15:53:39,633 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5619 seconds

daniel2
 60 learner(s) attended, 60 finished and 0 aborted. Total time: 23 minutes 1 second
     [java] 15:54:03,367 INFO [main] TestReporter - Total response time: 1417.098 seconds
     [java] 15:54:03,367 INFO [main] TestReporter - Average response time: 0.28570524193548386 seconds
     [java] 15:54:03,367 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5559499999999999 seconds

luke1
 60 learner(s) attended, 60 finished and 0 aborted. Total time: 23 minutes 8 seconds
     [java] 15:54:00,680 INFO [main] TestReporter - Total response time: 1456.368 seconds
     [java] 15:54:00,680 INFO [main] TestReporter - Average response time: 0.29356339447692 seconds
     [java] 15:54:00,680 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5511166666666667 seconds

luke2
 60 learner(s) attended, 59 finished and 1 aborted. Total time: 23 minutes 4 seconds
     [java] 15:53:58,050 INFO [main] TestReporter - Total response time: 1442.035 seconds
     [java] 15:53:58,050 INFO [main] TestReporter - Average response time: 0.2933947100712106 seconds
     [java] 15:53:58,050 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5559999999999999 seconds

jun-dir2
 60 learner(s) attended, 59 finished and 1 aborted.  Total time: 23 minutes 7 seconds
     [java] 15:54:06,942 INFO [main] TestReporter - Total response time: 1478.545 seconds
     [java] 15:54:06,942 INFO [main] TestReporter - Average response time: 0.3029184593321041 seconds
     [java] 15:54:06,942 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5527457627118645 seconds

jun-dir1
 60 learner(s) attended, 60 finished and 0 aborted. Total time: 22 minutes 44 seconds
     [java] 15:53:45,526 INFO [main] TestReporter - Total response time: 1478.54 seconds
     [java] 15:53:45,527 INFO [main] TestReporter - Average response time: 0.29809274193548385 seconds
     [java] 15:53:45,527 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5572666666666667 seconds


800 (80x10)

mitch1
 80 learner(s) attended, 80 finished and 0 aborted. Total time: 27 minutes 44 seconds
     [java] 16:35:30,722 INFO [main] TestReporter - Total response time: 22770.652 seconds
     [java] 16:35:30,723 INFO [main] TestReporter - Average response time: 3.446443469047979 seconds
     [java] 16:35:30,723 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5861875000000001 seconds

mitch2
 80 learner(s) attended, 80 finished and 0 aborted. Total time: 25 minutes 55 seconds
     [java] 16:33:42,885 INFO [main] TestReporter - Total response time: 22348.716 seconds
     [java] 16:33:42,885 INFO [main] TestReporter - Average response time: 3.3846306224443437 seconds
     [java] 16:33:42,885 INFO [main] TestReporter - Average response time of joinLesson calls: 0.563875 seconds

luke2
 80 learner(s) attended, 79 finished and 1 aborted. Total time: 27 minutes 12 seconds
     [java] 16:34:49,947 INFO [main] TestReporter - Total response time: 22698.662 seconds
     [java] 16:34:49,947 INFO [main] TestReporter - Average response time: 3.4781890897946677 seconds
     [java] 16:34:49,948 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5573291139240507 seconds

luke1
 80 learner(s) attended, 80 finished and 0 aborted. Total time: 28 minutes 34 seconds
     [java] 16:36:12,924 INFO [main] TestReporter - Total response time: 22343.708 seconds
     [java] 16:36:12,924 INFO [main] TestReporter - Average response time: 3.381823520508551 seconds
     [java] 16:36:12,925 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5635625 seconds

daniel1
 80 learner(s) attended, 79 finished and 1 aborted. Total time: 26 minutes 11 seconds
     [java] 16:33:48,484 INFO [main] TestReporter - Total response time: 21935.236 seconds
     [java] 16:33:48,484 INFO [main] TestReporter - Average response time: 3.3412392993145468 seconds
     [java] 16:33:48,484 INFO [main] TestReporter - Average response time of joinLesson calls: 0.570775 seconds

daniel2
 80 learner(s) attended, 80 finished and 0 aborted. Total time: 27 minutes 8 seconds
     [java] 16:34:43,956 INFO [main] TestReporter - Total response time: 22765.15 seconds
     [java] 16:34:43,956 INFO [main] TestReporter - Average response time: 3.44665404996215 seconds
     [java] 16:34:43,956 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5612874999999999 seconds

antonio1
 80 learner(s) attended, 80 finished and 0 aborted. Total time: 27 minutes 20 seconds
     [java] 16:34:38,802 INFO [main] TestReporter - Total response time: 22624.311 seconds
     [java] 16:34:38,802 INFO [main] TestReporter - Average response time: 3.4258496365838886 seconds
     [java] 16:34:38,802 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5648500000000001 seconds

antonio2
 80 learner(s) attended, 80 finished and 0 aborted. Total time: 26 minutes 34 seconds
     [java] 16:33:50,913 INFO [main] TestReporter - Total response time: 22297.728 seconds
     [java] 16:33:50,913 INFO [main] TestReporter - Average response time: 3.3758861468584405 seconds
     [java] 16:33:50,914 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5681375 seconds

jun-dir1
 80 learner(s) attended, 80 finished and 0 aborted. Total time: 26 minutes 50 seconds
     [java] 16:34:36,064 INFO [main] TestReporter - Total response time: 22141.973 seconds
     [java] 16:34:36,064 INFO [main] TestReporter - Average response time: 3.351797305479867 seconds
     [java] 16:34:36,064 INFO [main] TestReporter - Average response time of joinLesson calls: 0.565925 seconds

jun-dir2
 80 learner(s) attended, 80 finished and 0 aborted. Total time: 28 minutes 0 seconds
     [java] 16:35:48,529 INFO [main] TestReporter - Total response time: 22869.148 seconds
     [java] 16:35:48,529 INFO [main] TestReporter - Average response time: 3.4613512940820343 seconds
     [java] 16:35:48,529 INFO [main] TestReporter - Average response time of joinLesson calls: 0.5623 seconds



1000 (100x10):

Antonio1
 100 learner(s) attended, 100 finished and 0 aborted. Total time: 34
minutes 20 seconds
    Total response time: 56764.8 seconds
    Average response time: 6.878914202617548 seconds
    Average response time of joinLesson calls: 0.56333 seconds

Antonio2
 100 learner(s) attended, 100 finished and 0 aborted. Total time: 33
minutes 19 seconds
    Total response time: 54156.623 seconds
    Average response time: 6.563643558356563 seconds
    Average response time of joinLesson calls: 0.58286 seconds

Mitch1
 100 learner(s) attended, 100 finished and 0 aborted. Total time: 33
minutes 36 seconds
    Total response time: 55750.411 seconds
    Average response time: 6.755987760542899 seconds
    Average response time of joinLesson calls: 0.56799 seconds

Mitch2
 100 learner(s) attended, 99 finished and 1 aborted. Total time: 31
minutes 59 seconds
    Total response time: 54375.453 seconds
    Average response time: 6.643305192425168 seconds
    Average response time of joinLesson calls: 0.57197 seconds

Daniel1
 100 learner(s) attended, 100 finished and 0 aborted. Total time: 33
minutes 26 seconds
    Total response time: 61180.237 seconds
    Average response time: 7.41488752878439 seconds
    Average response time of joinLesson calls: 0.61716 seconds

Daniel2
 100 learner(s) attended, 100 finished and 0 aborted. Total time: 33
minutes 13 seconds
    Total response time: 59709.448 seconds
    Average response time: 7.236631681008363 seconds
    Average response time of joinLesson calls: 0.57577 seconds

Jun-Dir1
 100 learner(s) attended, 98 finished and 2 aborted. Total time: 34
minutes 10 seconds
    Total response time: 58965.556 seconds
    Average response time: 7.259979807929081 seconds
    Average response time of joinLesson calls: 0.5715959595959597 seconds

Jun-Dir2
 100 learner(s) attended, 100 finished and 0 aborted. Total time: 33
minutes 16 seconds
    Total response time: 61664.991 seconds
    Average response time: 7.473638468064477 seconds
    Average response time of joinLesson calls: 0.56792 seconds

Luke1
 100 learner(s) attended, 98 finished and 2 aborted. Total time: 33
minutes 33 seconds
    Total response time: 60355.951 seconds
    Average response time: 7.341679965940883 seconds
    Average response time of joinLesson calls: 0.56561 seconds

Luke2
 100 learner(s) attended, 100 finished and 0 aborted. Total time: 35
minutes 5 seconds
    Total response time: 60963.619 seconds
    Average response time: 7.386843450866351 seconds
    Average response time of joinLesson calls: 0.57457 seconds



1200 (120x10)

Antonio1
 120 learner(s) attended, 119 finished and 1 aborted. Total time: 41 minutes 53 seconds
     Total response time: 110826.736 seconds
     Average response time: 11.221824220332119 seconds
     Average response time of joinLesson calls: 0.5861416666666667 seconds

Antonio2
 120 learner(s) attended, 120 finished and 0 aborted. Total time: 42 minutes 24 seconds
     Total response time: 112620.534 seconds
     Average response time: 11.375811515151515 seconds
     Average response time of joinLesson calls: 0.5797083333333333 seconds

Mitch1
 120 learner(s) attended, 120 finished and 0 aborted. Total time: 42 minutes 34 seconds
     Total response time: 113650.186 seconds
     Average response time: 11.48213639119014 seconds
     Average response time of joinLesson calls: 0.5804833333333334 seconds

Mitch2
 120 learner(s) attended, 119 finished and 1 aborted. Total time: 42 minutes 2 seconds
     Total response time: 110959.727 seconds
     Average response time: 11.276395020325204 seconds
     Average response time of joinLesson calls: 0.5875416666666666 seconds

Daniel1
 120 learner(s) attended, 120 finished and 0 aborted. Total time: 41 minutes 41 seconds
     Total response time: 112107.685 seconds
     Average response time: 11.320578107644147 seconds
     Average response time of joinLesson calls: 0.5843666666666666 seconds

Daniel2
 120 learner(s) attended, 120 finished and 0 aborted. Total time: 42 minutes 11 seconds
     Total response time: 119010.865 seconds
     Average response time: 12.020085344914655 seconds
     Average response time of joinLesson calls: 0.5851916666666667 seconds

Jun-Dir1
 120 learner(s) attended, 120 finished and 0 aborted. Total time: 41 minutes 44 seconds
     Total response time: 119634.288 seconds
     Average response time: 12.084271515151515 seconds
     Average response time of joinLesson calls: 0.5903333333333334 seconds

Jun-Dir2
 120 learner(s) attended, 120 finished and 0 aborted. Total time: 41 minutes 39 seconds
     Total response time: 118596.08 seconds
     Average response time: 11.980612183048793 seconds
     Average response time of joinLesson calls: 0.5857333333333333 seconds

Luke1
 120 learner(s) attended, 118 finished and 2 aborted. Total time: 42 minutes 18 seconds
     Total response time: 118349.201 seconds
     Average response time: 11.998094180859692 seconds
     Average response time of joinLesson calls: 0.5778916666666666 seconds

Luke2
 120 learner(s) attended, 119 finished and 1 aborted. Total time: 42 minutes 6 seconds
     Total response time: 121259.833 seconds
     Average response time: 12.321901534396911 seconds
     Average response time of joinLesson calls: 0.5814916666666666 seconds


Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.