intra-mart Accel Platform / Cassandra Administrator Guide

Version 8 2014-04-01

«  4.3. Registration/Deletion to Windows Service   ::   Contents   ::   4.5. Registration/Deletion to Linux Daemon  »

4.4. Cassandra Setup for Linux

In this section the setup method of Cassandra to Linux environment is described.

4.4.1. File Expansion

<apache-cassandra-1.1.12-bin.tar.gz> file which has been downloaded by Obtaining Apache Cassandra should be expanded to the arbitrary path.

Note

In this document the directory below is specified as an example.

[/usr/local/cassandra]

Note

If you do the symbolic-link of the one expanded as /usr/local/apache-cassandra-1.1.12
to /usr/local/cassandra, you would be able to have easier access when the version is upgraded.
# ln -s /usr/local/apache-cassandra-1.1.12 /usr/local/cassandra

4.4.2. Various Settings

  • Following settings are necessary before starting Cassandra.

4.4.2.1. Cassandra Settings

  • Settings of Cassandra are stated in </usr/local/cassandra/conf/cassandra.yaml> file. Please open it by the editor, and edit it.
  • For the details about cassandra.yaml file, please refer to [cassandra.yaml Major Items List] of [Cassandra Reference Information].
  1. Specifying Cluster Name

    Arbitrary cluster name should be specified to the [cluster_name] property.
    cluster_name: 'IMBox Cluster'
    

    Warning

    In case multiple Cassandra servers exist, cluster may be configured by looking at the cluster name to judge if it is the Cassandra of the same cluster.
    Therefore, it is recommended to change the initial value, when intra-mart Accel Platform is newly built.
  2. Specifying Data Storage Location

    Please specify arbitrary path to [data_file_directories] property.
    If there is no directory specified, it will be automatically created at startup time.
    data_file_directories:
      - /var/lib/cassandra/data
    
  3. Specifying Commit Log Storage Location

    Please specify arbitrary path to [commitlog_directory] property.
    If there is no directory specified, it will be automatically created at startup time.
    # commit log
    commitlog_directory: /var/lib/cassandra/commit_log/1.1.12
    
  4. Specifying Cache Data Storage Location

    Please specify arbitrary path to [saved_caches_directory] property.
    If there is no directory specified, it will be automatically created at startup time.
    # saved caches
    saved_caches_directory: /var/lib/cassandra/saved_caches/1.1.12
    
  5. Specifying Communication Address for communication with other Cassandra

    Cassandra communicates with other Cassandra’s with the IP address obtained by Java InetAddress.getLocalHost(). If there are multiple addresses, it is necessary to explicitly specify the IP address to be used for such communication.
    listen_address: 192.168.xxx.xxx(arbitrary IP address)
    
    rpc_address: 192.168.xxx.xxx(arbitrary IP address)
    
  6. Specifying System Log

    Please open </usr/local/cassandra/conf/log4j-server.properties> file by the editor, and set arbitrary path to [log4j.appender.R.File] property.
    log4j.appender.R.File=/var/log/cassandra/1.1.12/system.log
    

Warning

Remarks for Settings

Depending on the combination of OS and JDK version, Xss of JVM parameters may become insufficient causing the failure of Cassandra startup.
We have observed that the error would occur for the combinations listed below.
  • OS: Linux type OS
  • JDK: Oracle JDK 1.7u40 , Oracle JDK 1.7u45

Circumvention

This symptom can be circumvented by changing the value of Xss which is set in ${cassandra}/conf/cassandra-env.sh.

  • In case Cassandra version is 1.1.4

       if [ "`uname`" = "Linux" ] ; then
       # reduce the per-thread stack size to minimize the impact of Thrift
       # thread-per-client.  (Best practice is for client connections to
       # be pooled anyway.) Only do so on Linux where it is known to be
       # supported.
         if startswith "$JVM_VERSION" '1.7.'
         then
    #         JVM_OPTS="$JVM_OPTS -Xss160k"
           JVM_OPTS="$JVM_OPTS -Xss228k"
         else
           JVM_OPTS="$JVM_OPTS -Xss128k"
         fi
     fi
    
  • In case Cassandra version is 1.1.12

     if [ "`uname`" = "Linux" ] ; then
       # reduce the per-thread stack size to minimize the impact of Thrift
       # thread-per-client.  (Best practice is for client connections to
       # be pooled anyway.) Only do so on Linux where it is known to be
       # supported.
       # u34 and greater need 180k
    
      #     JVM_OPTS="$JVM_OPTS -Xss180k"
      JVM_OPTS="$JVM_OPTS -Xss228k"
    fi
    

4.4.2.2. Changing Number of File Descriptors

Depending on the OS settings, IOException or FileNotFoundException may occur during the systems operations, and Cassandra may not work properly.
This problem occurs if the upper limit of the number of files that can be used (opened) by the Cassandra process exceeds the upper limit defined by OS.
Therefore, this problem can be circumvented by adding the following numbers for the respective environment to the number of OS file descriptors
which is set in /etc/system/limits.conf or /etc/security/limits.conf. So, please change the value accordingly.
* soft nofile 32768
* hard nofile 32768
root soft nofile 32768
root hard nofile 32768

[*] User name and the value are listed as examples. Please set the appropriate values for your environment.

Note

Current setting value of file descriptor can be

ulimit -n

found by the above.

4.4.2.3. Memory Setting

In the Linux environment the maximum heap size (Xmx) is automatically set with the value below by </usr/local/cassandra/conf/cassandra-env.sh> at startup time.
  • If installed memory size is 2048MB or less, it is half of installed memory size.
  • If installed memory size is 2049MB or more and 4099MB or less, it is 1024MB.
  • If installed memory size is 4100MB or more and 32768MB or less, it is one quarter of installed memory size.
  • If installed memory size is 32769MB or more, it is 8196MB from the installed memory.
If you want to set the arbitrary value, you should remove the comment-out of the following items in </usr/local/cassandra/conf/cassandra-env.sh>, and set the arbitrary value.
#MAX_HEAP_SIZE="4G"
#HEAP_NEWSIZE="800M"

Warning

The value which is set to MAX_HEAP_SIZE should be [512M] or more.
If the value smaller than [512M] is set, it may not function properly.
In Cassandra it is recommended to set the value no smaller than 1GB.
Appropriate value should be set for HEAP_NEWSIZE depending on the value set for MAX_HEAP_SIZE.

4.4.2.4. Environment Variable Setting

Home directory at which JDK has been installed should be added to the environment variable.
Following setting should be made to the environment variable of the user who executes Cassandra.
Variable JAVA_HOME
Value Home Directory at which JDK has been installed

«  4.3. Registration/Deletion to Windows Service   ::   Contents   ::   4.5. Registration/Deletion to Linux Daemon  »