Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 10 Next »

Clustering LAMS 2

This page is a collection of notes on load balancing and clustering LAMS on JBoss 4.0.2. There are 2 levels of clustering that we touch on, load balancing and session replication. There are a few things to overcome to get full session replication working (which we'll mention later), so for the most part we concentrate on load balancing. That is, several JBoss nodes sitting behind a load balancer, and writing to the same database.

Also note that this is more of a functional setup in order to run tests rather than the last word.


Roughly speaking, the steps below will set up a load balancer using Apache and mod_jk, providing sticky sessions to a group of JBoss nodes.

Setup the first node

  1. Make a copy of the 'all' server configuration and name it 'node1'. Deploy LAMS here.

    If using Linux...

    In ${JBOSS_HOME}/bin/run.conf, set in JAVA_OPTS
    See and for reference.

  2. Move lams.ear to /var/lib/lams/lams.ear
  3. Link ${JBOSS_HOME}/server/node1/deploy/lams.ear to /var/lib/lams/lams.ear
  4. In ${JBOSS_HOME}/server/node1/deploy/jbossweb-tomcat55.sar/META-INF/jboss-service.xml, set UseJK to true in order to enable sticky sessions from the Tomcat side.
  5. In ${JBOSS_HOME}/server/node1/deploy/jbossweb-tomcat55.sar/server.xml, give your node a jvmRoute name:
    <Engine name="jboss.web" defaultHost="localhost" jvmRoute="node1">
  6. Delete ${JBOSS_HOME}/server/node1/deploy/deploy.last/farm-service.xml to disable farming
  7. In ${JBOSS_HOME}/server/node1/deploy/tc5-cluster-service.xml and local-service.xml, set CacheMode to REPL_SYNC. We want entity cache replication to be synchronous so our hibernate objects are in a consistent state.
  8. (Optional - is this required?) In ${JBOSS_HOME}/server/node1/deploy/jbossweb-tomcat55.sar/context.xml, set distributable flag to true:
    <Context ... distributable="true">
  9. Start up LAMS by doing cd ${JBOSS_HOME}/bin; ./ -c node1 -b ${IP_ADDRESS}

    Make sure you specify the IP address in order for JBoss' auto discovery to work.

Share common files

Setup a network share for /var/lib/lams - this directory will contain lams.ear (application files + lams-www uploaded files). In terms of the configuration keys, this share will contain the EARDir, ContentRepositoryDir, TempDir, and DumpDir. The network user will need write access to these folders.

This path will need to be the same on all node servers, since all nodes use the same value from the database.

I've found that I needed to mount the lams.ear directly in the deploy folder, then link /var/lib/lams to the mount. The other way around jboss would continually undeploy and deploy lams.ear for some reason.

Setup the second node

  1. Unzip a new JBoss instance on a new machine
  2. Copy the 'node1' directory from the first node into ${JBOSS_HOME}/server/ directory, rename 'node2'
  3. Set preferIPv4Stack=true if Linux
  4. Mount the network share and link location of EARDir to ${JBOSS_HOME}/server/node2/deploy/lams.ear
  5. Modify ${JBOSS_HOME}/server/node1/deploy/jbossweb-tomcat55.sar/server.xml and change jvmRoute="node2"
  6. Start LAMS as above.

You should see something like the following on startup:

2008-10-08 14:38:46,184 INFO  [org.jboss.ha.framework.interfaces.HAPartition.DefaultPartition] Number of cluster members: 2
2008-10-08 14:38:46,185 INFO  [org.jboss.ha.framework.interfaces.HAPartition.DefaultPartition] Other members: 1
2008-10-08 14:38:46,185 INFO  [org.jboss.ha.framework.interfaces.HAPartition.DefaultPartition] Fetching state (will wait for 30000 milliseconds):
2008-10-08 14:38:46,186 INFO  [org.jboss.ha.framework.interfaces.HAPartition.DefaultPartition] New cluster view for partition DefaultPartition: 1 ([,] delta: 0)

This indicates that it has found node1. Similarly, node1 should show:

2008-10-08 14:42:18,442 INFO  [DefaultPartition] New cluster view for partition DefaultPartition (id: 1, delta: 1) : [,]
2008-10-08 14:42:24,749 INFO  [TreeCache] viewAccepted(): new members: [,]
2008-10-08 14:42:24,752 INFO  [TreeCache] locking the tree to obtain transient state
2008-10-08 14:42:24,754 INFO  [TreeCache] returning the transient state (140 bytes)

Setup the load balancer

  1. Setup Apache with mod_jk
  2. Set JkMount in your virtual server:
    JkMount /lams* loadbalancer
  3. Configure the


JBoss manual chapter on clustering,
Tomcat documentation on clustering,
Mod jk worker documentation,

Problems for LAMS 2

Configuration - config values are currently read from a Map static to each node. This needs to be cached across the nodes or read directly from the database in order for changes to config values to work. As a workaround, accessing the 'edit configuration details' updates the Map.

SessionMap - uses a unique counter static to each node, but since we have sticky sessions this isn't a problem.

SessionManager - same as for SessionMap.

The above classes will probably need to be changed to use jboss cache if we want session replication/failover/fault tolerance.

  • No labels