Basecode

AMX Module

The basecode comes with the standard automacs distribution, and is included here so it makes it into the master index. This documentation section is included only so that the functions are loaded into the index. This documentation is included for clarity and should not replace the walkthrough.

automacs.commands_interpret(templates)

automacs.commands_interpret_deprecated(block)

Interpret a block of command templates for GROMACS.

automacs.component(name, count=None)

Add or modify the composition of the system and return the count if none is provided. Originally designed for protein_atomistic.py.

automacs.delve(o, *k)

automacs.get_last_frame(gro=’system-previous’, dest=None, source=None, tpr=False)

Prepare or locate a snapshot of the last state of the system.

This function makes “chaining” possible by connecting one step to the next (along with a portion of init which checks for previous states). It defaults to the current step, but you can also set the source and destination. The default output filename is “system-previous” because this is designed for chaining. The default source is obtained by first checking for a previous state, in which case we get the state from there. The primary alternate behavior is to get the last frame of the current directory, but this is typically suited to quick analysis, video-making, etc, in which case the user would have to explicitly set the source keyword to state.here. If no previous state is available, we fall back to using state.here.

automacs.get_trajectory(dest=None)

Convert the trajectory to reassemble broken molecules. Requires items from the history_gmx. Note that this is customized for vmdmake but it could be generalized and added to automacs.py.

automacs.include(name)

Add an ITP file to the itp (non-ff includes) list but avoid redundancies which cause errors in GROMACS.

automacs.init()

Initialize the automacs state. Note that all automacs variables are defined here. Structure of the state Cursors: here = current directory with trailing slash step = current step name stepno = current step number History: steps = list of steps in order State-only interactors: calls.py make_step

automacs.minimize_deprecated(name, method=’steep’, top=None)

minimize(name,method=’steep’) Standard minimization procedure.

automacs.q(key, val=None)

Check either settings or the state for a variable name. This takes the place of (indiscriminately) loading all settings into the state.

automacs.register(func)

Collect utility functions in the state. [Typically this would be a decorator, but the import scheme makes that unworkable.]

automacs.trim_waters_deprecated(structure=’solvate-dense’, gro=’solvate’, gap=3, boxvecs=None, method=’aamd’, boxcut=True)

trim_waters(structure=’solvate-dense’,gro=’solvate’,gap=3,boxvecs=None) Remove waters within a certain number of Angstroms of the protein. #### water and all (water and (same residue as water within 10 of not water)) note that we vided the solvate.gro as a default so this can be used with any output gro file

automacs.write_mdp(param_file=None, rootdir=’./’, outdir=”, extras=None)

Universal MDP file writer which creates input files based on a unified dictionary. All MDP parameters should be stored in mdpdefs within inputs/input_specs_mdp.py. We assemble MDP files according to rules that can be found in the mdp entry for the specs dictionary contained in the input_specs_PROCEDURE.py file. Other simulation steps may use this function to access the mdp_section entry from the same specs dictionary in order to write more MDP files for e.g. equilibration steps.

In the inputs/input_specs_mdp.py file we define the “mdpdefs” (read: Molecular Dynamics Parameter DEFinitionS) dictionary, which is a customizable description of how to run the GROMACS integrator. The mdpdefs variable has a few specific kinds of entries denoted by comments and and described here.

The top level of mdpdefs is a “group” set in the mdp entry of the specs dictionary. This allows us to organize our parameters into distinct groups depending on the task. For example, we might have different groups of parameters for coarse-grained and atomistic simulations. We whittle the full mdpspecs dictionary by the group name below.

Our whittled dictionary then contains three kinds of entries.

1. The entry called defaults is a dictionary which tells you which dictionaries to use if no extra information

   is provided. Each key in the defaults dictionary describes a type of parameters (e.g. “output” refers to the parameters that specify how to write files). If the associated value is “None” then we assume that the key is found at the top level of mdpdefs. Otherwise the value allows us to descend one more level and choose between competing sets of parameters.

2. Other entries with keys defined in defaults contain either a single dictionary for that default

   (recall that we just use None to refer to these) or multiple dictionaries with names referred to by the defaults. These entries are usually grouped by type e.g. output or coupling parameters.

3. Override keys at the top level of mdpdefs which do not appear in defaults contain dictionaries which

   are designed to override the default ones wholesale. They can be used by including their names in the list associated with a particular mdp file name in specs. If they contain a dictionary, then this dictionary will override the default dictionary with that key. Otherwise, they should contain key-value pairs that can lookup a default in the same way that the defaults section does.

Except for the “group” entry, the specs key mdp_section (remember that this is defined in input_specs_PROCEDURE.py) should include keys with desired MDP file names pointing to lists that contain override keys and dictionaries. If you include a dictionary in the value for a particular MDP file then its key-value pairs will either override an MDP setting directly or override a key-value pair in the defaults. Note that this function was updated from the original automacs to be more explicit.

calls.convert_gmx_template_to_call(spec, kwargs, strict=False)

Use gromacs call instructions along with kwargs to make a new command.

calls.copy_file(src, dst, **kwargs)

Wrapper for copying files.

calls.copy_files(src, dst)

Wrapper for copying files with a glob.

calls.get_gmx_paths(override=False, gmx_series=False, hostname=None)

!!!

calls.get_gmx_share()

Figure out the share/gromacs/top directory.

calls.get_last_gmx_call(name, this_state=None)

The gmx call history is loaded by convert_gmx_template_to_call. We can retrieve the last call to a particular gromacs utility using this function.

calls.get_machine_config(hostname=None)

!!!

calls.gmx(program, **kwargs)

Construct a GROMACS command and either run it or queue it in a script.

calls.gmx_run(cmd, log, nonessential=False, inpipe=None)

Run a GROMACS command instantly and log the results to a file.

calls.make_sidestep(name)

Make a new step folder which is off-pathway.

calls.make_step(name)

Make a new step folder. See automacs.py docstring for defining variables in the state.

calls.modules_load(machine_config)

Interact with environment modules to load software.

calls.move_file(src, dest, **kwargs)

Wrapper for moving files.

calls.register_file(fn)

Maintain a list of new, essential files. T hese have not specific categories (in contrast to e.g. ITP files)

calls.register_gmx_call(command, flag, value)

Register an automatic rule for adding flags to gromacs calls.

Command-line interface

A collection of (helpful) command-line utilities. The interface is managed by the makeface module.

cli.locate(keyword)

Locate the source code for a python function which is visible to the controller.

Parameters:

keyword : string

Any part of a function name (including regular expressions)

Notes

This controller script is grepped by the makefile in order to expose its python functions to the makefile interface so that users can run e.g. “make program protein” in orer to access the program function above. The makeface function routes makefile arguments and keyword arguments into python’s functions. The makefile also detects python functions from any scripts located in amx/procedures/extras. The locate function is useful for finding functions which may be found in many parts of the automacs directory structure.

cli.flag_search(keyword)

Search the codebase for cases where a settings flag is requested.

cli.config()

Print the config in a tree form using the (hidden) asciitree function above.

cli.watch()

Wrapper for a command which tails the oldest mdrun command.

cli.layout(name=None)

Map all teh codes.

cli.gromacs_config(where=None)

Make sure there is a gromacs configuration available.

cli.setup(name=”)

Run this after cloning a fresh copy of automacs in order to clone some standard

cli.notebook(procedure, rewrite=False, go=False, name=’notebook.ipynb’)

Make an IPython notebook for a particular procedure.

cli.upload(alias, path=’~’, sure=False, state_fn=’state.json’, bulk=False)

Upload the data to a supercomputer. !Bulk is not implemented yet. !Note that putting sure before path ruins the argument handling hence makeface needs fix

cli.download(sure=False)

Download a simulation from the cluster if it has already been uploaded.

cli.cluster(hostname=None, overwrite=True)

Write a cluster header according to the machine configuration. The machine configuration is read from ~/.automacs.py but can be overriden by a local config.py which can be created with make config local. This code will concatenate the cluster submission header with a continuation script. Note that we do not log this operation because it only manipulates BASH scripts.

cli.qsub()

Submits the job to the queue. This saves you from changing into the latest step directory.

cli.gitcheck()

Check all git repos for changes.

cli.gitpull()

Pull in each git module.

cli.rewrite_config(source=’config.py’)

Reformat the config.py file in case you change it and want it to look normal again.

cli.codecheck(fn)

Check a file for evaluation from the command line. This utility is useful when you want to see if a file with dict literals passes the JSON check.

cli.collect_parameters()

Scan for all parameters files.

cli.write_continue_script(hostname=None, overwrite=False)

Write the continue script if it does not yet exist. Note that this script wraps the same function in amx.continue_script for the CLI.

cli.show_kickstarters()

Print the available kickstarters for the user.

cli.hardstart()

Generate a state.json to initiate a hard restart. This functionality is useful for dropping CPT and TPR files into a new directory for a continuation.

COMMON

Extra functions common to many simulations. Generic extensions used by: proteins, bilayers Someday this might become part of the amx canon.

common.atomistic_or_coarse()

We rely on tags to identify atomistic and coarse-grained simulations rather than settings.

common.center_by_group(structure, gro, selection)

Center a particular selection in the box. Similar to “restuff” above.

common.contiguous_encode(items)

Handy method for encoding tuples into single objects in numpy for uniqueness checking.

common.count_molecules(structure, resname)

Count the number of molecules in a system using make_ndx.

common.counterions(structure, top)

Standard procedure for adding counterions. The resname must be understandable by “r RESNAME” in make_ndx and writes to the top file. Note that the top argument is not used and should be removed after checking downstream.

common.dotplace(n)

common.equilibrate(groups=None, structure=’system’, top=’system’, stages_only=False, seq=None)

Standard equilibration procedure.

common.equilibrate_check(name)

Check if the gro file for this step has been written.

common.extract_itp(topfile)

Extract a protein.itp file from a top file which is automatically generated by pdb2gmx. Note that parts of this function was poached to read_top in common. This always generates a single ITP file.

common.get_box_vectors(structure, gro=None, d=0, log=’checksize’)

Return the box vectors.

common.get_last(name, cwd=None)

Get the last of a particular file by extension.

common.get_last_file(name, cwd=None)

Get the last of a particular file by extension.

common.get_last_mdps()

Get all of the mdp files from a previous step.

common.get_last_sources()

Get the sources retrieved from the previous step (this gets the originals).

common.get_pdb(code)

Download a PDB from the database or copy from file.

common.get_start_structure(path)

Get a start structure or auto-detect a PDB in the inputs.

common.gro_combinator(*args, **kwargs)

Concatenate an arbitrary number of GRO files.

common.grouper(ndx=’system-groups’, protein=True, lipids=True)

Generic replacement for generating system-groups.ndx for various configurations.

common.minimize(name, method=’steep’)

Standard minimization procedure.

common.protein_laden(structure=’system’)

Infer if there are proteins in the simulation.

common.read_gro(gro, **kwargs)

Read a GRO file and return its XYZ coordinates and atomnames. !Note that this is highly redundant with a cgmd_bilayer.read_molecule so you might replace that one. !RE-ADD NUMPY AND CENTER FOR reionize/cgmd_bilayer !Note that we drop velocities which should be read separately or with a flag.

common.read_molecule(gro)

Read a molecule in GRO form and return its XYZ coordinates and atomnames. !REPLACE WITH common.read_gro

common.remove_hetero_atoms(structure, out)

Remove heteroatoms from a PDB.

common.restart_clean(part, structure, groups, posres_coords=None, mdp=’input-md-in’)

Perform a hard-restart that mimics the md.part0001 naming scheme.

common.restuff(structure, gro, tpr, ndx)

Restuff everything in the box. Used as a prelude for the generic solvate function. ! Desperately needs a better name.

common.solvate(structure, gro, edges=None, center=False)

Standard solvate procedure for atomistic protein in water. Often requires restuff above.

common.solvate_protein(structure, top)

solvate(structure,top) Standard solvate procedure for atomistic protein in water. ! STILL USING THIS BECAUSE NO scipy and because we cannot find spc216 in the other one

common.trim_waters(structure=’solvate-dense’, gro=’solvate’, gap=3, boxvecs=None, method=’aamd’, boxcut=True)

ABSOLUTE FINAL VERSION OF THIS FUNCTION HOPEFULLY GOD WILLING trim_waters(structure=’solvate-dense’,gro=’solvate’,gap=3,boxvecs=None) Remove waters within a certain number of Angstroms of the protein. #### water and all (water and (same residue as water within 10 of not water)) note that we vided the solvate.gro as a default so this can be used with any output gro file IS IT A PROBLEM THAT THIS DOESN’T TOUCH THE IONS??

common.unique(items)

…

common.write_gro(**kwargs)

Write a GRO file with new coordinates.

common.write_structure_pdb(structure, pdb)

Infer the starting residue from the original PDB and write structure.pdb with the correct indices according to the latest GRO structure (typically counterions.gro).

common.write_top(topfile)

Write a topology.

common.write_top_original(topfile)

write_top(topfile) Write the topology file.

common.write_topology(topfile)

Write a topology.

Force field tools mediate naming schemes between various force fields.

class force_field_tools.Landscape(ff=None, cwd=None)

Handle force field naming schemes.

! lay out the types in meta.json? This class reads the force field files according to types and exposes them to automacs in a way that makes it easy to ask for ion definitions by type.

Methods

anions()

cations()

ions()

lipids()

my(struct, cat)

Get the item names for a GMXStructure. Uses atom names. Returns any atom names that apply, so this is really ion-specific !!!

objects_by_category(cat)

Return all object names in a particular category.

protein_residues = [‘GLH’, ‘ILE’, ‘ALAD’, ‘GLUH’, ‘GLN’, ‘HISH’, ‘ASN1’, ‘HYP’, ‘GLY’, ‘HIP’, ‘ARGN’, ‘MSE’, ‘CYS1’, ‘GLU’, ‘CYS2’, ‘CYS’, ‘HISE’, ‘ASP’, ‘SER’, ‘HSD’, ‘HSE’, ‘PRO’, ‘CYX’, ‘ASPH’, ‘ORN’, ‘HSP’, ‘HID’, ‘HIE’, ‘LYN’, ‘DAB’, ‘ASN’, ‘CYM’, ‘HISD’, ‘VAL’, ‘THR’, ‘HISB’, ‘HIS’, ‘HIS1’, ‘HIS2’, ‘TRP’, ‘HISA’, ‘ACE’, ‘ASH’, ‘CYSH’, ‘PGLU’, ‘LYS’, ‘PHE’, ‘ALA’, ‘QLN’, ‘MET’, ‘LYSH’, ‘NME’, ‘LEU’, ‘ARG’, ‘TYR’]

protein_selection()

Return a protein selection string for make_ndx.

sterols()

force_field_tools.force_field_family()

Get the family name for the force field i.e. charmm or martini.

class structure_tools.GMXStructure(fn=None, center=False, **kwargs)

Methods

add(another, before=False, **kwargs)

Add another GRO to this one.

cog(*inds)

Given a set of index lists (presumably returned from select), return the centroid of centroids.

detect_composition()

Infer the topology.

detect_composition_NEWISH()

Detect the composition of a system in order to write an accurate topology file. pseudocode:

see if there are blocks with proteins if PROTEINS

search for ITP files if multiple ITP files throw an error if itp then

read in distinct protein objects scan resnames for residues that correspond to them if any gaps/etc throw an error otherwise log the proteins in their position in the sequence then recapitulate the original composition method

fix_residue_numbering()

Ensure coherent residue numbering.

get_landscape(fn=None)

REPLACE WITH force_field_tools.py

meta_keys = [‘atom_names’, ‘residue_names’, ‘residue_indices’, ‘points’]

read_box_vectors(string)

Interpret box vectors. The GRO box vector is a list of space-separated reals.

regroup()

Sort everything in the system so it follows the composition order.

remove(inds)

Remove atoms corresponding to a selection.

renumber()

Renumber residues.

select(text, return_bools=False)

Return atom indices for items that match a particular selection.

select_center(selections)

Wraps a common call to cog which retrieves the center of geometry for a list of selections.

trim(gap=0.3, subject=None, discard=None)

Remove nearby waters. Recapitulates much of trim_waters in common.py but is adapted slightly. The discard

write(out_fn)

Write a GRO file.

write_box()

Turn the box vectors from a triplet into a string.

structure_tools.operator(x, op, y, structure)

Perform a single logical operation on an atom selection.

structure_tools.operator_abstract(a, op, b, sentence, structure)

Recursive function which calls for operations on “a operator b” phrases in parentheses.

structure_tools.parse(text, structure)

Parse a sentence with explicit order of operations defined by parentheses.

class topology_tools.GMXForceField(dirname)

A collection of GMXTopology files

Methods

molecule(name)

molecules_list()

class topology_tools.GMXTopology(itp=None, **kwargs)

Represent a molecule topology.

Methods

add_molecule(**kwargs)

Add a new molecule.

dd = 3

details = {‘records’: ‘i j funct length’}

entry_pre_proc(match)

Catalogue and apply preprocessor flags, namely ifdef.

entry_proc(name, text)

Process a single entry type in ???

kw = ‘length’

molecule_rename(old, rename)

name = ‘constraints’

preproc()

process_moleculetype(text)

Interpret a single moleculetype entry from a GROMACS itp file.

restrain_atoms(*names, **kwargs)

Apply a restraint to a molecule.

write(fn, overwrite=False)

Write an ITP file.

utils.bash(command, log=None, cwd=None, inpipe=None)

Run a bash command

utils.status(string, i=0, looplen=None, bar_character=None, width=25, tag=”, start=None)

Show a status bar and counter for a fixed-length operation.

Runner

CONTROL

Specify all control flow for codes managed by ACME.

This module contains the entire command-line interface for ACME, and hence it should be found in “commands” setting inside config.py. It includes functions which prepare and execute the experiments. It consults controlspec.py, which describes the rules for writing experiment files loaded by acme.py. Each experiment typically comes with extra paths and imports, alongside a “settings” text block. The _keysets function validates all of the incoming experiments to make sure they have all of the required metadata.

control.look(state=’state.json’)

View the current state from the commandline. Useful for debugging.

control.quick(procname, **kwargs)

Prepare and run a script embedded inside an experiment file. If we supply a stepno as a keyword argument, then we assume this is a metarun, and we write the appropriate experiment file and script without executing it. Otherwise exeuction happens immediately.

control.prep(procname=None, noscript=False, v=False)

Prepare an experiment from inputs specified by the config. There are two modes: a “metarun” or a single program.

control.run(procname=None, over=’expt.json’, script=’script.py’, PYTHON_DEBUG=None, look=False)

Run a prepared acme script read from the experiment settings. Note that all scripts look for expt.json. So if you want to run things manually, just copy your experiments, and run their script. But the expts have the script name so we could write a wrapper that finds the right script from an expt. But this function does that and even more, so it’s pretty useful.

control.metarun()

Run a series of simulation steps.

control.clean(sure=False)

control.back(command=None, cwd=’.’, log=’log-back’, go=None)

Run a command in the background and generate a kill switch.

Parameters:

command : string

A terminal command used to execute the script in the background e.g. ./script-protein.py. You can also use other targets in the command e.g. make back command="make metarun <name>".

control.preplist(silent=False, verbose=False)

Identify all available experiments and print them for the user.

control.set_config(what, *args, **kwargs)

Command-line interface to update configuration in config_fn (typically config.py). This function routes make set ...` requests to functions here in the acme.py module, which manages all experiments. Since ``set is a python type, we make use of the config.py alias scheme to map this function to make set ....

control.go(procname, clean=False, back=False)

Sugar for running make prep (name) && make run which also works for metaruns or quick scripts.

control.prep_json()

Print the experiments list in JSON format. Useful for the factory.

CONTROL SPECIFICATION

This code includes only a single variable called controlspec, which provides instructions for validating experiments so that they are fully specified at runtime. The controlspec['keysets'] variables provides sets of keys (ironically for the _keysets function in control.py), at least one which must match with the keys in each experiment. See control.py for more details; this code uses the controlspec variable to direct the execution.

The controlspec variable below sets the required keys for different run types. For example, the first of the keysets tells you that any run must contain the following keys: ['script','extensions','params','tags','settings','cwd']. These keys are always required (though some can be None or empty lists) and ensure that the correct script is executed with the right settings.

    'run':{('script','extensions','params','tags','settings','cwd'):'std'},
    'metarun':{('metarun','cwd'):True},
    'metarun_steps':{
      ('step','do'):'simple',
      ('quick','settings'):'quick',
      ('quick','settings','jupyter_coda'):'quick',
      ('step','do','settings'):'settings',},
    'quick':{
      ('quick',):'quick',
      ('quick','jupyter_coda'):'quick',
      ('quick','cwd',):'quick',
      ('quick','settings','cwd'):'basic',
      ('quick','settings','cwd','params','extensions','tags'):'extensions',
      ('quick','settings','cwd','params','extensions','tags','imports'):'extensions_imports'},},}

ACME EXPERIMENTS

Codes in this module manage reading and writing experiments. Experiments for acme-inflected simulation runs are written to special data structures found in python scripts containing dictionary literals. The files are located by get_input_files.

Note! Your experiment files may use paths that depend on your specific configuration. You can manage these paths in two ways. You can either decide on a configuration (described largely by the modules list in config.py) and then if you decide to move something, perform batch replacements on your paths inside of the experiment files. A more advanced option is to label paths in other modules with the “@module” syntactic sugar. If “module” is the directory name for a module, then @module/path/to/file will be replaced with the correct path, regardless of where the module is. This is handled in loadstate.py, which is the only place where experiments are unpacked.

MAKEFACE (a.k.a. MAKEfile interFACE)

A crude but convenient way to make a command-line interface for python. This makeface.py connects the makefile to arbitrary python functions found in files specified by the commands key in config.py which is managed by the acme module.

Note: you avoid using make keywords as python argument names (e.g. “w” and “s”) and avoid using *args and **kwargs because arguments are passed from the Makefile to the python functions by using the inspect module for introspection. You can access python functions from the terminal by running make my_python_function some_true_bool_flag kwarg="some value". Set commands in the config.py managed by acme to specify which files provide functions to the interface. You can set __all__ in these files to hide extraneous functions from make.

makeface.abspath(path)

Get the right path.

makeface.fab(text, *flags)

Colorize the text.

makeface.fab_deprecated(text, *flags)

On the chopping block.

makeface.import_local(fn)

Import a local script manually.

makeface.import_remote(script, is_script=False)

Import a script as a module, directly, iff it is not in the path.

makeface.makeface(*arglist)

Route make commands into python.

makeface.str_or_list(x)

Turn a string or a list into a list.

makeface.strip_builtins(obj)

Remove builtins from a hash in place.

makeface.tracebacker(e)