Module Nn_config

module Nn_config: sig .. end

Configuration of name nodes

class type nn_node_config = object .. end

class type nn_min_config = object .. end

The subset of Nn_config.nn_node_config that is minimally required for contacting the namenode.

val extract_node_config : Netplex_types.config_file -> nn_node_config

Config section must look like (see Nn_config.nn_node_config for explanations):

         netplex {
           ...
           namenodes {
             clustername = "cluster1";  (* required *)
             node { addr = "hostname:port" };
             node { addr = "hostname:port" };
             ...
             rank = "0";                (* mandatory *)
             hostname = "hostname";     (* optional *)
             timeout = 60.0;            (* optional *)
             alive_min = 1;             (* optional *)
             alive_min_startup = 1;     (* optional *)
             elect_timeout 300.0;       (* optional *)
             replication = 3;           (* optional *)
             lock_timeout = 300;        (* optional *)
             pool_size = 268435456;     (* optional *)
           }
           datanodes {
             port = 2728;                      (* mandatory *)
             blocksize = 65536;                (* mandatory *)
             timeout = 60.0;                   (* optional *)
             security_level = "auth";          (* optional *) 
             discovery { addr = "224.0.0.1" }; (* optional *)
             ...
             discovery_period = 1.0;           (* optional *)
             multicast_ttl = 2;                (* optional *)
             max_errors = 1;                   (* optional *)
           }
           ...
         }
      

The parameter rank can also be given by running a script that outputs the rank to stdout. This script must be given as parameter rank_script.

Instead of describing the nodes directly in the config file using node subsections, one can also point to external file:

 
         node_list = "filename";
         port = <default_port>;
      

These two parameters can replace the node subsections for namenodes. The external file enumerates the hosts, one hostname a line (with shell-style comments).

Special values:

• alive_min and alive_min_startup can also be set to "all" (in double quotes). This means all configured name nodes must be available

val extract_min_config : Netplex_types.config_file -> nn_min_config

Config section must look like (see Nn_config.nn_node_config for explanations):

         netplex {
           ...
           namenodes {
             clustername = "cluster1";  (* required *)
             node { addr = "hostname:port" };
             node { addr = "hostname:port" };
             ...
             timeout = 60.0;            (* optional *)
           }
         }
      

val bmaprowsize : int

block map row size - how many bits are in one row of the block map. Changing this value also affects the database.