The Cluster Membership object

Table of Contents

Introduction

The membership component in the Apache Tribes Channel is responsible for dynamic discovery of other members(nodes) in the cluster. There are currently two different membership service, the org.apache.catalina.tribes.membership.McastService and the org.apache.catalina.tribes.membership.StaticMembershipService. The McastService builds a multicast based membership service that sends UDP packets to multicast IP addresses. The StaticMembershipService builds a unicast based membership service that sends TCP packets to predefined member address.

Default Implementation

The default implementation of the cluster group notification is built on top of multicast heartbeats sent using UDP packets to a multicast IP address. Cluster members are grouped together by using the same multicast address/port combination. Each member sends out a heartbeat with a given interval (frequency), and this heartbeat is used for dynamic discovery. In a similar fashion, if a heartbeat has not been received in a timeframe specified by dropTime ms. a member is considered suspect and the channel and any membership listener will be notified.

Attributes

Common Attributes

Attribute Description
className

The implementation of the membership component. Two implementations available, org.apache.catalina.tribes.membership.McastService and org.apache.catalina.tribes.membership.StaticMembershipService.

Multicast Attributes

Attribute Description
className

The value is org.apache.catalina.tribes.membership.McastService. This implementation uses multicast heartbeats for member discovery.

address

The multicast address that the membership will broadcast its presence and listen for other heartbeats on. The default value is 228.0.0.4 Make sure your network is enabled for multicast traffic.
The multicast address, in conjunction with the port is what creates a cluster group. To divide up your farm into several different group, or to split up QA from production, change the port or the address
Previously known as mcastAddr.

port

The multicast port, the default value is 45564
The multicast port, in conjunction with the address is what creates a cluster group. To divide up your farm into several different group, or to split up QA from production, change the port or the address

frequency

The frequency in milliseconds in which heartbeats are sent out. The default value is 500 ms.
In most cases the default value is sufficient. Changing this value, simply changes the interval in between heartbeats.

dropTime

The membership component will time out members and notify the Channel if a member fails to send a heartbeat within a give time. The default value is 3000 ms. This means, that if a heartbeat is not received from a member in that timeframe, the membership component will notify the cluster of this.
On a high latency network you may wish to increase this value, to protect against false positives.
Apache Tribes also provides a TcpFailureDetector that will verify a timeout using a TCP connection when a heartbeat timeout has occurred. This protects against false positives.

bind

Use this attribute if you wish to bind your multicast traffic to a specific network interface. By default, or when this attribute is unset, it tries to bind to 0.0.0.0 and sometimes on multihomed hosts this becomes a problem.

ttl

The time-to-live setting for the multicast heartbeats. This setting should be a value between 0 and 255. The default value is VM implementation specific.

domain

Apache Tribes has the ability to logically group members into domains, by using this domain attribute. The org.apache.catalina.tribes.Member.getDomain() method returns the value specified here.

soTimeout

The sending and receiving of heartbeats is done on a single thread, hence to avoid blocking this thread forever, you can control the SO_TIMEOUT value on this socket.
If a value smaller or equal to 0 is presented, the code will default this value to frequency

recoveryEnabled

In case of a network failure, Java multicast socket don't transparently fail over, instead the socket will continuously throw IOException upon each receive request. When recoveryEnabled is set to true, this will close the multicast socket and open a new socket with the same properties as defined above.
The default is true.

recoveryCounter

When recoveryEnabled==true this value indicates how many times an error has to occur before recovery is attempted. The default is 10.

recoverySleepTime

When recoveryEnabled==true this value indicates how long time (in milliseconds) the system will sleep in between recovery attempts, until it recovers successfully. The default is 5000 (5 seconds).

localLoopbackDisabled

Membership uses multicast, it will call java.net.MulticastSocket.setLoopbackMode(localLoopbackDisabled). When localLoopbackDisabled==true multicast messages will not reach other nodes on the same local machine. The default is false.

Static Membership Attributes

Attribute Description
className

The value is org.apache.catalina.tribes.membership.StaticMembershipService.

connectTimeout

Timeout for attempting a TCP connection to address of predefined static member. Default is 500 ms.

expirationTime

If members have failed to update their alive time within the given time, this membership will notify the memberDisappeared event to cluster. Default is 5000 ms.

rpcTimeout

Timeout for messages that used for member notification to/from othee nodes. Default is 3000 ms.

useThread

If set to true, this membership service will start a local thread for sending a ping message. if set to false, channel heartbeat will send a ping message. Default is false.

pingInterval

If useThread == true, defines the interval of sending a ping message. Default is 1000 ms.

Nested Components

Static Membership Service allows nesting of a <LocalMember> and <Member> element.

StaticMember Attributes

LocalMember:
Static member that is the local member of the static cluster group.

Note: In Tomcat 9.0.17 and later, The setting of local member is not required. It is possible to set up a list of all cluster members including local member instead of setting this components.

Attribute Description
className Only one implementation available:org.apache.catalina.tribes.membership.StaticMember
port There is no need to set. The value of this attribute inherits from the cluster receiver setting.
securePort There is no need to set. The value of this attribute inherits from the cluster receiver setting.
host There is no need to set. The value of this attribute inherits from the cluster receiver setting.
domain The logical cluster domain for that this static member listens for cluster messages. Two different type of values are possible:
1. Regular string values like "staging-domain" or "tomcat-cluster" will be converted into bytes using ISO-8859-1 encoding. 2. byte array in string form, for example {216,123,12,3}
uniqueId A universally uniqueId for this static member. The values must be 16 bytes in the following form:
1. byte array in string form, for example {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}

Member:
Static member that add to the static cluster group.

Attribute Description
className Only one implementation available:org.apache.catalina.tribes.membership.StaticMember
port The port that this static member listens to for cluster messages
securePort The secure port this static member listens to for encrypted cluster messages default value is -1, this value means the member is not listening on a secure port
host The host (or network interface) that this static member listens for cluster messages. Three different type of values are possible:
1. IP address in the form of "216.123.1.23"
2. Hostnames like "tomcat01.mydomain.com" or "tomcat01" as long as they resolve correctly
3. byte array in string form, for example {216,123,12,3}
domain The logical cluster domain for that this static member listens for cluster messages. Two different type of values are possible:
1. Regular string values like "staging-domain" or "tomcat-cluster" will be converted into bytes using ISO-8859-1 encoding.
2. byte array in string form, for example {216,123,12,3}
uniqueId A universally uniqueId for this static member. The values must be 16 bytes in the following form:
1. byte array in string form, for example {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}

Setting

Before Tomcat 9.0.16 

     <Membership className="org.apache.catalina.tribes.membership.StaticMembershipService">
       <LocalMember className="org.apache.catalina.tribes.membership.StaticMember"
                  uniqueId="{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}"/>
       <Member className="org.apache.catalina.tribes.membership.StaticMember"
                  port="4004"
                  host="tomcat02.mydomain.com"
                  uniqueId="{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,1}"/>
     </Membership>

Tomcat9.0.17 and later 

     <Membership className="org.apache.catalina.tribes.membership.StaticMembershipService">
       <Member className="org.apache.catalina.tribes.membership.StaticMember"
                  port="4004"
                  host="tomcat01.mydomain.com"
                  uniqueId="{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}"/>
       <Member className="org.apache.catalina.tribes.membership.StaticMember"
                  port="4004"
                  host="tomcat02.mydomain.com"
                  uniqueId="{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,1}"/>
     </Membership>