Index of class methods

Drops resources - intended to be used for error cleanup

Drops resources - intended to be used for error cleanup

Access a blockmap stored in DB row id and for datastore identity, with an initial size.

Adds the passed counters to the current set of counters, e.g.

whether this object refers to an existing row in the datastore table.

Add a group.

Add a user.

Write an admin table (this does not fit anywhere else)

The contents of this object as admin tables.

The contents of this object as admin tables.

Gets the allocinfo list for this inode

The owner tuple used by this view

The authentication ticket

Map/reduce processes files in units of bigblocks.

Same as Mapred_def.mapred_job_config.bigblock_size.

Returns the blocklist of a file like PlasmaFS does

Reserve blocks and allocate at commit time.

Returns the allocation size

Mark blocks and deallocate at commit time.

Get the contents of the blockalloc table for this datastore identity, wrapped into a blockmap_t object

If the blockmap exists, return it.

Ensures that all known blockmaps are loaded

Pin these blocks until commit/rollback.

This is used to synchronize accesses to the in-memory blockmap with the per-inode blocklists in the db.

All blockmaps so far loaded

Returns the block size for this path

Books this ID with this (total) size, and pushes the new entry

Cancels any ongoing write.

Return the cauth record

Check the config.

Deallocates resources

This filesystem will not be used anymore

Indicate that we are done with the file

Close the client (deallocate the message box)

Releases resources (e.g.

Releases resources (e.g.

The clustername

cmp s1 i1 l1 s2 i2 l2: Compare two keys as the sorter does.

The optional combiner is called for the internal shuffle passes.

The name of the started command

commit owner: commits owner's reservations, and makes them permanent.

Commits the transaction (prepared or not), and returns

Commits the transaction (prepared or not), but does not return the transaction

Renames a couple of files at once.

The commit monitor controls which commits can be run in parallel

Estimate where we are

The map/reduce server configuration

The config file containing config

The connection.

The connection.

continue_with f: Set f as the continuation.

Copies a file to a new name.

copy_in_from_buf path pos mem len: XXX

copy_out_to_buf path pos mem len: XXX

Copies the mentioned file to the task nodes, and places them into the local_directory.

Returns current revision identifier

Get a custom parameter or raise Not_found

The list of allowed custom parameters

Delete this row in the datastore table

Get the contents of the datastore table

Maximum ID used in the datastore table (None if use_id was never called)

The secret random number used for securing transactional writes

Change the row in datastore identified by id so that the fields identity, size, and enabled are updated.

Record that this ID is now also used in a transaction

Returns the db transaction that is used for commits

Delete the group.

Delete the user

A delayed allocation is not immediately pushed to the shared book.

Returns del_now, del_later: del_now are the inodes to be deleted at the end of the transaction.

Deletes the local directory on the task nodes.

Disable this master

disable_slave name

Expected blocksize of datanodes

The blocksize.

The blocksize.

The blocksize.

The clustername

The directory containing the store.

The directory containing the store.

The address to use for discovering datanodes.

Every how many seconds a new discovery messge is emitted

The number of dn_io processes.

For the liveliness monitoring: how many errors (missing response messages) are at most acceptable.

The multicast group to join

The number of hops (routers+1) for multicasting.

The timeout for connects and RPC calls to data nodes

The data node port

The security level the datanodes expect

The size of the shm queue, in blocks.

The size of the shm queue, in blocks.

How often the file is synced to disk

Get a dump of the internal state

Current sums for `Shm and `Buf as debug string

The sort limit after applying the factor, in bytes

If >0, enhanced map tasks are created.

Whether there are still enough enabled slaves

The common event system

The event system this transaction uses

The event system this transaction uses

Extracts the key from a record, and returns the position as pair (index,len).

The local directory corresponding to "file::/"

The filename

The count is the number of names for an inode (same as length of list returned by filename_rev_get_e)

Returns the inode of the member name of directory dir_inode

adds this filename.

Lists subnames of a potential directory name, together with inodes

Returns the parent directory

Works only for directories, and obtains an existence lock for the directory.

filename_rev_get does not acquire locks.

deletes this filename from this directory.

Access to the cluster filesystem

The filesystem

The filesystem

fill_blockmap index data: Fills the blockmap at index with data, a string consisting of 0 and 1.

Set that the map is filled, i.e.

fill_from_db index n: Fills the n rows at index from the db (synchronously)

whether the object is completely filled with data

Flushes the records from the buffer to disk

Forget everything about this datastore identity

free l owner: deallocates these blocks.

from_dev_e q dev size_limit lines_limit esys; Generalization of from_fd_e.

Outputs the records coming from this file.

Get the blockmap for this identity, or Not_found

get_changes owner: Returns the changes made by owner.

Find a group entry by name or by gid

Find a group entry by name or by gid

getgroups user: Returns the set of effective group memberships for user

getgroups user: Returns the set of effective group memberships for user

Find a passwd entry by name or by uid

Find a passwd entry by name or by uid

Return the users and groups

Return the users and groups

whether there are still free blocks

hash record key_index key_len: Takes key = String.sub record index key_len as key, and returns the hash value.

whether link is supported for this path

Whether this manager has allocations

the datastore db table id

This is the blockmap for this datastore identity

whether nothing is being executed

Inactivate

Sets active to false

Increases this shared counter

Inserts a new inode and returns the ID.

deletes the inode.

returns blocks.

Serializer for getting a new ID from the inode table

like inode_get_e but immediately acquires a write lock for the inode

Maximum ID used in the inode table (None if use_id was never called)

The use count for inodes:

Sets that the inode is anonymous

Fake that this inode is considered as used.

First delete the blocks in the del range, then add the blocks from add.

Updates the inode.

This update of mtime/ctime is lock-free

Returns (transactions) numbers (or raises Not_found)

Record that this ID is now also used in a transaction

This plasma directory contains the input files

How input_dir is interpreted

Reads one record, and advances the cursor.

How to split the input file into records

Reads some records and appends them to the passed queue.

How to represent records for internal files

whether the transaction has been returned

whether the transaction has been returned

Whether this slave is disabled

An automatically generated name.

link flags old_name new_name

The directory path on the task nodes for files that are needed by tasks

If the passed path can be mapped to the local filesystem, the corresponding path in the local filesystem is returned (w/o tree prefix)

Returns for the given path the association on which node a location resides.

lock_filename (dir_inode,name) lock_type owner: Locks the name member of directory dir_inode.

Returns Some unlock if successful where calling unlock removes the lock, or reverts to the previous state (1-step undo)

This plasma directory is used for log files.

The directory path on the task nodes where the tasks can put files, and these files are finally copied to PlasmaFS

The mapper reads records, maps them, and writes them into a second file.

The number of map tasks that should not be exceeded.

Whether only whole files are passed to a map job.

The master in host:port syntax

Whether the master (i.e.

How many files are merged at most by a shuffle task

How many enabled slaves we need at minimum

Creates a new directory.

High watermark for buffer memory.

Alternate way for setting the high watermark as fraction of available physical RAM.

Low watermark for buffer memory.

Alternate way for setting the low watermark as fraction of available physical RAM.

Maximum for buffer memory.

Alternate way for setting the maximum as fraction of available physical RAM.

Same as Mapred_config.mapred_config.mr_buffer_size.

The normal size of I/O buffers.

Same as Mapred_config.mapred_config.mr_buffer_size_tight.

The size of I/O buffers when RAM is tight.

The maximum number of pre-partitions.

High watermark for shared memory.

Alternate way for setting the high watermark as fraction of available shared memory.

Low watermark for shared memory.

Alternate way for setting the low watermark as fraction of available shared memory.

Maximum for shared memory.

Alternate way for setting the maximum as fraction of available shared memory.

The size of the buffers for sorting.

Task nodes (only hostname)

The port number

A directory where to put executables, logs, etc.

If set, only these Unix users may interact with the running task server

let se = serialized il f: Same as serializing over all values of il (in a nested way)

The name of the sorter, for debugging

A name for identifying this job, e.g.

allocates a new ID for an shm or buf object

Creates a new revision identifier

If there is a delayed continuation execute it now

The clustername

The clustername

Whether to do more consistency checks, even the time-consuming ones

Whether PlasmaFS is disabled.

For how many seconds to wait for neighbor namenodes

The hostname of this node.

Which inodecaches are announced to the clients

After these many seconds transactions are not retried on ECONFLICT

How many name nodes need to be alive to continue with operations

How many name nodes need to be alive at cluster startup

The timeout for connects and RPC calls to name nodes

The name nodes in "host:port" syntax

The name nodes in "host:port" syntax

The name nodes in "host:port" syntax

How much pool RAM is allocated (shared memory)

The rank of this node compared to other nodes.

Default replication factor

the configuration

Provided for compatibility with Netfs.stream_fs.

Returns a new plasma cluster for this path, if available

Returns an independent access object for the same filesystem.

This plasma directory will get output files.

Outputs these records.

How to write the output file from records.

Outputs the records in the queue.

The partition.

partition_of_key me jc s p l: Determines the partition of a key (which is supposed to occupy the range p to p+l-1 of s).

The number of partitions = number of reduce tasks

Provided for compatibility with Netfs.stream_fs.

Provided for compatibility with Netfs.stream_fs.

Reads the record, but does not advance the cursor, i.e.

Which phases are enabled

pin l owner: marks these blocks as "to keep" even if a competing transaction frees them.

The local directory corresponding to "plasma::/" (if PlasmaFS is NFS-mounted)

The ordinal number of the record that will be read next.

Needs to be called after a commit.

This is run by the job process after the last task is finished

Prepares the commit, and returns the journal to commit.

Prepares the commit, and returns additional journal entries to commit.

This is run by the job process before the first task is started

Prepares the transaction for commit

Push allocations to the shared book.

Hands over the records in the queue to the sorter.

read flags filename: Opens the file filename for reading, and returns the input stream.

read_file flags fs rc name block len: Reads from name, starting at block, ending at block+len-1 (plus the following overflow region).

read_file flags filename: Opens the file filename for reading, and returns the contents as a local_file.

Reads the contents of a directory.

Reads the target of a symlink

The record_config

The record_config

The reducer reads all the records of one partition, and puts them into an output file.

Release the pinned blocks

release owner: Unpins the blocks of this owner.

Removes the file or symlink.

Renames the file.

The request ID.

reserve n owner ri: finds n free blocks and returns their indices.

Resets this shared counter, and returns the value before resetting

return the transaction to the pool or closes it

return the transaction to the pool

Removes an empty directory

The same connection coerced to read-only

Rolls this view back

rollback owner: unreserves owner's blocks.

Rolls the transaction back, and returns

Unless noprepare is set, prepare the query first, and execute it then.

let se = serialized i f: If already a serializer is running for the passed value of i, it is waited until the previous engines reach a final state, and then it is run e = f esys.

This filesystem connection will not be used for some time, and resources should be given back to the OS if possible.

Sets the function for extracting the key.

Sets the identitifer returned by cur_rev_ident

Set whether zombie inodes exist

The manager for shared memory and large buffers

Shut the process down

Returns the size of a file.

Size of the blockmap in blocks

Connections to the name slaves as triples (name,client,enabled).

Sorts the records, and writes them to the passed writer.

Maximum number of lines for a given sort limit in bytes

Returns the duration of the last sort (w/o writing) in seconds

This is normally set to one of the sorters defined in Mapred_sorters.

How many files are created by a shuffle task.

returns statistics: write_blocks: how many blocks have been written out, write_lines: how many lines have been processed (unavailable if from_fd_e is used), write_bytes: how many bytes have been processed, write_fs_time: the time spent for waiting on the filesystem layer for writing out data

returns statistics: read_blocks: how many blocks have been read in, read_lines: how many lines have been processed (unavailable if to_fd_e is used), read_bytes: how many bytes have been processed, read_fs_time: the time spent for waiting on the filesystem layer for new data

Number of used, transitional, free blocks

symlink flags oldpath newpath: Creates a symlink.

Deletes the entire log directory.

These files are copied at job start to the "local" directory for this job on all task nodes.

Returns the files that are still in the log directory

This filename prefix is used for log files

Copies these files of the log directory into PlasmaFS.

Returns whether the test is true.

Similar to test but this function performs all tests in the list at once, and returns a bool for each test.

to_any_e dest esys: like to_fd_e but the data is not written to a file descriptor.

Similar to to_fd_e, only that this method writes to an Uq_io.out_device.

to_fd_e fd esys: The records are written to fd.

unlock all filenames of this owner

unlock all inodes of this owner

Effective watermarks

This plasma directory is used for temporary files.

write flags filename: Opens (and optionally creates) the filename for writing, and returns the output stream.

write_file fs rc name: Appends records to this file (which must already exist).

write_file flags filename localfile: Opens the file filename for writing, and copies the contents of the localfile to it.

The preferred suffix for written files

Whether zombie inodes exist (unused anonymous inodes)