Omnicalc Modules

This documentation provides an overview of the omnicalc internal data structures. This page is not comprehensive, however the authors have selected the most important parts of the underlying codes. The main documentation provides a more user-friendly guide to the code.

Command-line interface

Omnicalc command-line interface.

cli.locate(keyword)

Find a function.

cli.set_config(*args, **kwargs)

Command-line interface to update configuration in config_fn (typically config.py). This function routes make set ...` requests to functions here in the acme.py module, which manages all experiments. Since ``set is a python type, we make use of the config.py alias scheme to map this function to make set .... This was adapted from the automacs.runner.acme version to be more generic.

cli.nuke(sure=False)

Reset this copy of omnicalc. Be careful!

cli.setup()

cli.compute(meta=None, confirm=False, kill_switch=None)

Expose the workspace to the command line. Note that the kill_switch is a dummy which is caught by makeface.py.

cli.plot(name, meta=None)

Plot something

cli.pipeline(name, meta=None)

Plot something

cli.clone_calcs(source)

Clone a calculations repository.

cli.look(method=None, **kwargs)

Inspect the workspace. Send a method name and we will run it for you.

cli.blank_meta()

Configuration

config.set_config(*args, **kwargs)

Command-line interface to update configuration in config_fn (typically config.py). This function routes make set ...` requests to functions here in the acme.py module, which manages all experiments. Since ``set is a python type, we make use of the config.py alias scheme to map this function to make set .... This was adapted from the automacs.runner.acme version to be more generic.

config.unset(*args)

Remove items from config.

Workspace

class omnicalc.WorkSpace(plot=None, plot_call=False, pipeline=None, meta=None, confirm_compute=False, cwd=None, do_slices=True, checkup=False)

User-facing calculation management. Style note: the workspace instance is passed around to many classes in maps.py. The author is aware that this is highly unusual, but it takes the place of a larger, more hierarchical class.

Methods

attach_standard_tools(mod)

Send standard tools to the calculation functions.

chase_upstream(specs, warn=False)

Fill in upstream information. Works in-place on specs.

compute(confirm=False, checkup=False, cleanup=None)

Run through computations.

compute_single(incoming)

SINGLE COMPUTATION. Note that this replaces (most of) computer from the original omnicalc.

find_script(name, root=’calcs’)

Find a generic script somewhere in the calculations folder.

get_calculation_function(calcname)

Search the calcs subdirectory for a calculation function. Note that this lookup function enforces the naming rule which is hard-coded: namely, that all calculations must be in a function in a script which each use the calculation name.

get_importer(silent=False)

Development purposes.

get_new_dat_name(base_name)

Get a new filename for the post-processing data. Assumes we already checked the data so we aren’t doing a redundant calculation.

get_simulations_in_collection(*names)

Read a collections list.

infer_calculation_order()

Needs tested and confirmed possibly with a safety check of some kind to avoid infinite recursion. Lifted directly from omnicalc workspace action function.

infer_group(calc)

Figure out groups for a downstream calculation.

infer_pbc(calc)

Figure out PBC condition for a downstream calculation.

Important note: calculations with uptype post will drop the group and pbc flags from their filenames. To identify a postprocessed data file, we need to infer the original simulation name from the short (or prefixed) name at the beginning of the data file’s name. We do this by making a map between simulation names in the metadata slices dictionary and their shortened names. This lets us perform a reverse lookup and figure out which simulations in the slices folder (and elsewhere in the metadata) are the parents of a postprocessed data file we found on the disk. For that reason, the back_namer below sweeps over all possible spot names and all possible slices to figure out the pbc flag for the upstream data. This allows us to drop the pbc flags on downstream calculations with uptype post.

job_print(job)

Describe the job.

load(name, cwd=None, verbose=False, exclude_slice_source=False, filename=False)

Wrap load which must be used by other modules.

nprocs = 4

pipeline(name, plot_call=False, meta=None)

Plot something. ! Get this out of the workspace.

plot(plotname, plot_call=False, meta=None)

Plot something. ! Get this out of the workspace.

plotload(plotname)

Get data for plotting programs.

prepare_calculations(calcnames=None, sns=None)

Match calculations with simulations. This function prepares all pending calculations unless you ask for a specific one.

read_specs(meta=None, merge_method=’careful’)

Read and interpret calculation specs. Lifted directly from old workspace.load_specs.

show_specs()

Print specs.

sns()

For backwards compatibility with plot programs, we serve the list of simulations for a particular plot using this function with no arguments.

specs_path = (‘calcs’, ‘specs’, ‘*.yaml’)

store(obj, name, path, attrs=None, print_types=False, verbose=True)

Wrap store which must be importable.

times(write_json=False)

Useful via make look times. Shows all of the edr times.

times_json()

Expose make look times to the factory. !Need a long-term solution for calling things like this.

variable_unpacker(specs)

Internal variable substitutions using the “+” syntax.

versioning = {‘spec_file’: 2}

omnicalc.compute(meta=None, confirm=False, kill_switch=None)

Expose the workspace to the command line. Note that the kill_switch is a dummy which is caught by makeface.py.

omnicalc.load(name, cwd=None, verbose=False, exclude_slice_source=False, filename=False)

Get binary data from a computation.

omnicalc.look(method=None, **kwargs)

Inspect the workspace. Send a method name and we will run it for you.

omnicalc.pipeline(name, meta=None)

Plot something

omnicalc.plot(name, meta=None)

Plot something

omnicalc.store(obj, name, path, attrs=None, print_types=False, verbose=True)

Use h5py to store a dictionary of data.

Data structures (maps)

OMNICALC DATA STRUCTURES Note that this file “maps” much of the metadata onto objects in memory and hence constitutes the core of the omnicalc functionality (other than omnicalc.py).

class maps.CalcMeta(meta, **kwargs)

Listing of calculations for cross-referencing. All calculations are identified by the name and the index of the “unrolled” version i.e. the specific calculation after loops are applied.

Methods

calcjobs(name)

Return calculations by name, including loops. Called by prepare_calculations.

find_calculation(name, specs)

Find a calculation in the master CalcMeta list by specs.

get_upstream(specs)

Get upstream calculations.

unroll_loops(details, return_stubs=False)

The jobs list may contain loops. We “unroll” them here.

class maps.Calculation(name, specs, stub)

A calculation, including settings.

class maps.ComputeJob(calc, sl, **kwargs)

A computation job joins a calculation with a simulation slice.

Methods

match_result()

Check if this job is done. This function requires a JOIN between postdat items of the datspec variety and this job. Currently there are two ways that a job is completed, either if the computation just ran or if we already linked the datspec with the job request. Both should result in the same object, namely, a “result” which might be a good way to name this.

class maps.DatSpec(fn=None, dn=None, job=None, work=None)

Bases: maps.NamingConvention

Parent class for identifying a piece of post-processed data. DatSpec instances can be picked up from a file on disk but also generated before running a job, which will inevitably write the post-processed data anyway.

Methods

basename()

Name this slice.

from_file(fn, dn)

DatSpec objects can be imported from file (including from previous versions of omnicalc) or they can be constructed in anticipation of finishing a calculation job and making the file (see from_job below). This function handles most backwards compatibility.

from_job(job)

Create datspec object from a job in preparation for running the calculation.

class maps.NamingConvention(**kwargs)

Organize the naming conventions for omnicalc. omni_namer data structure:

naming indexed by pairs: slice-type,data-type

dictionary-to-name name-to-dictionary

meta slice reading indexed by: slice-type

keys required in the specs

Several classes below inherit this to have easy access to the namers.

Methods

common_types = {‘wild’: ‘[A-Za-z0-9\-_]’, ‘float’: ‘\d+(?:(?:\.\d+))?’, ‘gmx_suffixes’: ‘(?:gro|xtc)’}

interpret_name(name)

Given a post-processing data file name, extract data and infer the version.

name_slice(kind, **kwargs)

Generate a slice name from metadata according to the slice type.

omni_namer = [((‘standard’, ‘gmx’), {‘n2d’: ‘^(?P<short_name>%(wild)s+)\.(?P<start>%(float)s)-(?P<end>%(float)s)-(?P<skip>%(float)s)\.(?P<group>%(wild)s+)\.pbc(?P<pbc>%(wild)s+)\.%(gmx_suffixes)s$’, ‘d2n’: ‘%(short_name)s.%(start)s-%(end)s-%(skip)s.%(group)s.pbc%(pbc)s.%(suffix)s’}), ((‘standard’, ‘datspec’), {‘n2d’: ‘^(?P<short_name>%(wild)s+)\.(?P<start>%(float)s)-(?P<end>%(float)s)-(?P<skip>%(float)s)\.(?P<group>%(wild)s+)\.pbc(?P<pbc>%(wild)s+)\.(?P<calc_name>%(wild)s+)\.n(?P<nnum>\d+)\.(dat|spec)$’, ‘d2n’: ‘%(short_name)s.%(start)s-%(end)s-%(skip)s.%(group)s.pbc%(pbc)s.%(calc_name)s’}), ((‘standard_obvious’, ‘datspec’), {‘n2d’: ‘^(?P<short_name>%(wild)s+)\.(?P<start>%(float)s)-(?P<end>%(float)s)-(?P<skip>%(float)s)\.(?P<calc_name>%(wild)s+)\.n(?P<nnum>\d+)\.(dat|spec)$’, ‘d2n’: ‘%(short_name)s.%(start)s-%(end)s-%(skip)s.%(calc_name)s’}), ((‘raw’, ‘datspec’), {‘n2d’: ‘^(?P<short_name>%(wild)s+)\.(?P<calc_name>%(wild)s+)\.n(?P<nnum>\d+)\.(dat|spec)$’, ‘d2n’: ‘%(short_name)s.%(calc_name)s’})]

omni_slicer_namer = {‘readymade_namd’: {‘slice_keys’: [‘readymade_namd’]}, ‘standard’: {‘slice_keys’: [‘groups’, ‘slices’]}}

parser = {(‘standard’, ‘gmx’): {‘n2d’: ‘^(?P<short_name>%(wild)s+)\.(?P<start>%(float)s)-(?P<end>%(float)s)-(?P<skip>%(float)s)\.(?P<group>%(wild)s+)\.pbc(?P<pbc>%(wild)s+)\.%(gmx_suffixes)s$’, ‘d2n’: ‘%(short_name)s.%(start)s-%(end)s-%(skip)s.%(group)s.pbc%(pbc)s.%(suffix)s’}, (‘standard_obvious’, ‘datspec’): {‘n2d’: ‘^(?P<short_name>%(wild)s+)\.(?P<start>%(float)s)-(?P<end>%(float)s)-(?P<skip>%(float)s)\.(?P<calc_name>%(wild)s+)\.n(?P<nnum>\d+)\.(dat|spec)$’, ‘d2n’: ‘%(short_name)s.%(start)s-%(end)s-%(skip)s.%(calc_name)s’}, (‘raw’, ‘datspec’): {‘n2d’: ‘^(?P<short_name>%(wild)s+)\.(?P<calc_name>%(wild)s+)\.n(?P<nnum>\d+)\.(dat|spec)$’, ‘d2n’: ‘%(short_name)s.%(calc_name)s’}, (‘standard’, ‘datspec’): {‘n2d’: ‘^(?P<short_name>%(wild)s+)\.(?P<start>%(float)s)-(?P<end>%(float)s)-(?P<skip>%(float)s)\.(?P<group>%(wild)s+)\.pbc(?P<pbc>%(wild)s+)\.(?P<calc_name>%(wild)s+)\.n(?P<nnum>\d+)\.(dat|spec)$’, ‘d2n’: ‘%(short_name)s.%(start)s-%(end)s-%(skip)s.%(group)s.pbc%(pbc)s.%(calc_name)s’}}

class maps.ParsedRawData(work)

Import raw simulation data.

Methods

divy_keys(spot)

The treeparser matches trajectory files with a combined regex. This function prepares a lambda that divides the combined regex into parts and reduces them to strings if there is only one part. The resulting regex groups serve as keys in the toc.

get_timeseries(sn_full)

Typically EDR times are stored in the toc for a particular spot. This function first figures out which spot you want and then returns the edr data.

keyfinder(spotname)

Decorate the keys_to_filename lookup function so it can be sent to e.g. slice_trajectory. If you are only working with a single spot, then this creates the file-name inference function for all data in that spot.

prefixer(sn)

Choose a prefix for naming post-processing files.

spotname_lookup(sn_full)

Find the spotname for a particular simulation. This is only used in a few places in maps.py: in the prefixer below and the portion of SliceMeta.__init__ which makes slices.

treeparser(spot, **kwargs)

This function parses simulation data which are organized into a “spot”. It writes the filenames to the table of contents (self.toc).

class maps.PostDat(where, namer=None, work=None)

Bases: maps.NamingConvention

A library of post-processed data. This class mirrors the data in the post_spot (aka post_data_spot). It includes both post-processing dat/spec file pairs, as well as sliced trajectories in gro/xtc or psf/dcd formats.

Methods

get_twin(name, pair)

Many slices files have natural twins e.g. dat/spec and gro/xtc. This function finds them.

limbo()

posts()

search_slices(**kwargs)

Find a specific slice.

slices()

class maps.Slice(**kwargs)

Parent class which holds several different representations of what we call a “slice”.

Methods

flat()

Reduce a slice into a more natural form.

class maps.SliceMeta(meta, **kwargs)

This class JOINs a metadata slice dictionary with slices in the postdat so that the actual slice can be retrieved for the calculation. It also calls the slicer.

Methods

get_slice(sn, group, slice_name)

Get slices. Group is permissive so we retrieve slices with this function.

maps.json_type_fixer(series)

Cast integer strings as integers, recursively. We also fix ‘None’.