6. Cassandra Cluster Build¶
In this section the method of building Cassandra cluster is introduced.Following descriptions in this document are based on [[Table 1. Cassandra Connection Information Setting Example]].
1st Node (Seed Node)
192.168.0.1
2nd Node
192.168.0.2
3rd Node
192.168.0.3
Keyspace
default
Warning
In case replication-factor or host is changed for the cluster build, setting is necessary when intra-mart Accel Platform is set up.For the details, please refer to [Setup Guide - Apache Cassandra (in case IMBox is used)].
Topics
6.1. Cluster Configuration¶
While Cassandra can be operated with only 1 unit, it can also be built as a cluster configuration allowing load balancing and redundancy for handling large volume messages.As is described in Assuring Cassandra Consistency , since IMBox adopts [Quorum(consistency guaranteed for (number of nodes+1)/2)] for consistency guarantee, it is strongly recommended to make the number of nodes in the cluster to 3 units or more.
6.2. Node Addition¶
Changing replication-factor and host
replication-factor (hereafter denoted as RF) is the setting which shows how many data replicas are retained in the cluster.If the standard value of 1 is used, data reference would not be possible if the node of the subject fails.As was stated before, since IMBox adopts Quorum for consistency guarantee, it is recommended to use the RF value in the table below to guarantee consistency.
Number of Nodes
RF value(recommended value)
Description
1
1
3
3
If RF value is set to 2, consistency cannot be guaranteed if one of the nodes has stopped, and IMBox will cease to be operational.
4 or more
3 or more and not greater than the number of nodes
Please set arbitrary value considering failure proof factor and data size for each node.
Seed Node should be specified.
Please open <%CASSANDRA_HOME%/conf/cassandra.yaml> file of each node by the editor, and specify the first node (192.168.0.1) to [seeds] property as the seed node.seed_provider: # Addresses of hosts that are deemed contact points. # Cassandra nodes use this list of hosts to find each other and learn # the topology of the ring. You must change this if you are running # multiple nodes! - class_name: org.apache.cassandra.locator.SimpleSeedProvider parameters: # seeds is actually a comma-delimited list of addresses. # Ex: "<ip1>,<ip2>,<ip3>" - seeds: "192.168.0.1"After setting is made at each node, please start Cassandra.
Please execute the command below, and each node is displayed as follows. It is completed if the total sum of Effective-Ownership is confirmed to be equal to the value of [RF * 100%].
# /usr/local/cassandra/bin/nodetool ring (In case of Windows ：C:/ > %CASSANDRA_HOME%/bin/nodetool ring) Starting NodeTool Address DC Rack Status State Load Effective-Ownership Token 134073324684581098149220982299151052417 192.168.0.1 datacenter1 rack1 Up Normal 117.33 KB 88.35% 72029913264419379591431889264009303469 192.168.0.2 datacenter1 rack1 Up Normal 225.3 KB 59.54% 110436698385580468589163673403640885580 192.168.0.3 datacenter1 rack1 Up Normal 213.4 KB 52.11% 134073324684581098149220982299151052417Note
On Cassandra following ports are used by default to communicate with other Cassandra.Please change the setting of iptables or SElinux for Linux, and the setting of firewall for Windows on an as required basis.Client Connection Port and Inter-Node Communication Port are set in <%CASSANDRA_HOME%/conf/cassandra.yaml>.JMX Connection Port is set in <%CASSANDRA_HOME%/conf/cassandra-env.sh> for Linux and in <%CASSANDRA_HOME%/bin/cassandra.bat> for Windows.
[1] cannot be changed
[2] in case SSL is used for inter-node communication Warning
If the value other than 9160 is specified to the Client Connection Port, error will occur in the new node detection function.Please do not change the initial value of 9160, and use it.Note
What is [Seed Node] ?
This node functions as a center (hub) of Cassandra cluster. A node that is newly added to the cluster will obtain status information of other nodes or information about the overall cluster from the seed node. Functions themselves would work properly without a seed node, and hence the seed node itself would not be a single point of failure. However, without a seed node, it would take more time to detect the status change of nodes in the cluster.
6.2.1. In case IMBoxEnvironment has already been built¶
In case tenant environment setup has been already executed and IMBox environment has been built, please execute the following steps.
Execute cassandra-cli in the directory below.
# %CASSANDRA_HOME%/bin/cassandra-cli.sh (or cassandra-cli.bat)Connect to Cassandra with the command below.
[default@unknown] connect 127.0.0.1/9160;Specify the Keyspace used by IMBox with the command below.
[default@unknown] use default;Change replication_factor by IMBox with the command below.
[default@default] update keyspace default with strategy_options={replication_factor:2};It is successful if the message below is displayed.
Waiting for schema agreement... ... schemas agree across the clusterFinish cassandra-cli with the command below, and restart Cassandra.
[default@default] exit;After the restart, please execute the command below.
# /usr/local/cassandra/bin/nodetool repair (in case of Windows: C:/ > %CASSANDRA_HOME%/bin/nodetool repair)
6.3. Node Deletion¶
In case you need to delete any node in the cluster, please execute decommission on the node that should be deleted.It will take a while for this process to complete, because the data on the node to be deleted should be transferred to other nodes.# /usr/local/cassandra/bin/nodetool -h [IP address of the node to be deleted] decommission (in case of Windows : C:/ > %CASSANDRA_HOME%/bin/nodetool -h [IP address of the node to be deleted] decommission)