Plasma GitLab Archive
Projects Blog Knowledge

Plasma Project:
Home 
Manual 
FAQ 
Slides 
Perf 
Releases 

Module Mapred_toolkit.Place

module Place: sig .. end


A place is a file, or directory containing files in PlasmaFS.
type 'a t 
A place for storing records of type 'a
type 'a codec = ('a -> string) Mapred_rfun.rfun *
(string -> int -> int -> 'a) Mapred_rfun.rfun
A codec (coder/decoder) says how to convert a record to string, and how to read it back. It is given as two functions (coder,decoder) which must both be registered (rfun). The decoder takes a string and a range with position and length.
type ext_location = [ `Deep_dir of string | `File of string | `Flat_dir of string ] 
type location = [ `Deep_dir of string | `File of string | `Flat_dir of string | `Notebook ] 
val create : ?prefix:string ->
?repl:int ->
Mapred_fs.filesystem ->
'a codec ->
Mapred_toolkit.format ->
Mapred_io.record_config ->
ext_location -> 'a t
Creates a new empty place. The file or directory is created. The place can be used to store new files

  • prefix: New files will get this prefix (ignored for `File location)

val from : ?prefix:string ->
?repl:int ->
Mapred_fs.filesystem ->
'a codec ->
Mapred_toolkit.format ->
Mapred_io.record_config ->
ext_location -> 'a t
Picks up an existing location, and makes the files there accessible as place

  • prefix: New files will get this prefix (ignored for `File location)

val notebook_place : unit -> 'a t
A new in-memory place for notebooks
val location : 'a t -> location
val get_codec : 'a t -> 'a codec
val get_format : 'a t -> Mapred_toolkit.format
val get_fs : 'a t -> Mapred_fs.filesystem
val get_rc : 'a t -> Mapred_io.record_config
val is_file_place : 'a t -> bool
Whether the place can store files
val files : 'a t -> string list
Returns the files of the place
val create_file : ?filename:string -> 'a t -> string
create_file pl:

Create a new empty file at the place. The file name is automatically determined, but starts with the previously set prefix. Also, the file gets a suffix that is derived from the format (".var" or ".fixed<n>").

If the place refers to a `File location, it is only possible to call create_file once.

This function fails for notebooks.

The optional filename argument influences the filename.

  • If passed, and if prefix ^ filename ^ suffix does not exist yes, this file name is chosen. Otherwise:
  • If passed, a fragment uuid is generated so that prefix ^ filename ^ uuid ^ suffix is the new file
  • If not passed, the name is prefix ^ uuid ^ suffix.

val clear : 'a t -> unit
Deletes all files at this place the files function returns. Note that this excludes ignored files starting with "_" and ".", and files in directories starting with these characters. Only files are deleted, and the directories are left in place.

This function works for notebook places, and just deletes the notebooks.

For deleting directories entirely, please use Mapred_io.delete_rec.

This web site is published by Informatikbüro Gerd Stolpmann
Powered by Caml