schrodinger.application.matsci.jobutils module

Copyright Schrodinger, LLC. All rights reserved.

exception schrodinger.application.matsci.jobutils.LicenseError(token)

Bases: Exception

License error

__init__(token)

Initialize LicenseError object.

Parameters

token (schrodinger.infra.licensing) – License token that is not available

args
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

schrodinger.application.matsci.jobutils.get_logging_tag(job, tag='log_tag')

Get the logging tag of a job.

Parameters
Returns

the logging tag

Return type

str

schrodinger.application.matsci.jobutils.add_outfile_to_backend(file_fn, backend=None, set_structure_output=False, stream=False, wait=True)

Add output file or directory to the backend.

Parameters

  • file_fn (str) – File or directory name

  • backend (schrodinger.job._Backend or None) – Backend handle. If None, a backend will be checked for. If no backend is found, nothing will be done.

  • set_structure_output (bool) – If True, set this structure as output

  • stream (bool) – If True, stream the file to the submission host

  • wait (bool) – if True wait until the job has finished before the output file or directory is copied back to the launch host, otherwise copy the file or directory to the launch host when this function is called, only relevant if stream is False

schrodinger.application.matsci.jobutils.add_to_backend(*files, backend=None, **kwargs)

Add output file(s) or directory(s) to the backend. See add_outfile_to_backend docs for kwargs.

Parameters

  • files (str) – List of files/directories

  • backend (schrodinger.job._Backend or None) – Backend handle. If None, a backend will be checked for. If no backend is found, nothing will be done.

schrodinger.application.matsci.jobutils.log_structures_found(qjob, structure_files, log, jobstates_for_logging=None)

Log structures info for a job.

Parameters

  • qjob (schrodinger.job.queue.JobControlJob) – The subjob to find structures

  • structure_files (dict) – Keys are subjobs, values are sets of structure file names

  • log (callable) – function(msg) writes msg to log file

  • jobstates_for_logging (None or list of str) – Log info for subjobs in these states

schrodinger.application.matsci.jobutils.run_jobdj_and_add_files(jobdj, log, expect_exts=None, exclude_exts=None, jobdj_dir='.')

Run the subjobs currently queued up in the jobdj, adding their files to the current backend for copying back to the job directory and locating the expected structure output file for each job.

Parameters

  • jobdj (schrodinger.job.queue.JobDJ) – The JobDJ with queued up jobs to run

  • log (callable) – function(msg) writes msg to log file

  • expect_exts (None or list of str) – The expected extensions of the output files

  • exclude_exts (None or list of str) – The output files found with the excluded extensions are not copied back to the original job directory or documented into logger

  • jobdj_dir (str) – jobdj_dir is the relative path from where the backend is created to where the jobdj is created. Using /scr/user/jobname/subdir1/subdir2 as an example, normally backend and jobdj are created in /scr/user/jobname/, and thus jobdj_dir by default is os.curdir. If the jobdj is created inside /scr/user/jobname/subdir1/, the jobdj_dir is subdir1 as backend is created in /scr/user/jobname/. In this example, the backend is in /scr/user/jobname/, jobdj is in /scr/user/jobname/subdir1, and the subjob can run in /scr/user/jobname/subdir1/subdir2. (subjob.getCommandDir() gives subdir2)

Return type

list

Returns

A list of structure output file names, sorted alphabetically

schrodinger.application.matsci.jobutils.finalize_subjob(subjob, backend, structure_files, structure_extensions, log, exclude_exts=None, jobdj_dir='.')

Mark subjob output and log files for copying back to the job directory, and find the structure output file if there is one

Parameters

  • subjob (schrodinger.job.queue.JobControlJob or None) – The subjob to mark files from

  • backend (schrodinger.job.jobcontrol._Backend or None) – The current backend or None if there isn’t one

  • structure_files (dict) – If an output structure file is found, it will be added to this dict. Keys are subjobs, values are sets of structure file names

  • structure_extensions (list of str) – The expected extension of the structure files

  • log (function) – function(msg) writes msg to log file

  • exclude_exts (None or list of str) – The output files found with the excluded extensions are not copied back to the original job directory or documented into logger

  • jobdj_dir (str) – jobdj_dir is the relative path from where the backend is created to where the jobdj is created. Using /scr/user/jobname/subdir1/subdir2 as an example, normally backend and jobdj are created in /scr/user/jobname/, and thus jobdj_dir by default is os.curdir. If the jobdj is created inside /scr/user/jobname/subdir1/, the jobdj_dir is subdir1 as backend is created in /scr/user/jobname/. In this example, the backend is in /scr/user/jobname/, jobdj is in /scr/user/jobname/subdir1, and the subjob can run in /scr/user/jobname/subdir1/subdir2. (subjob.getCommandDir() gives subdir2)

Return type

bool

Returns

True if the job has completed, False if not

schrodinger.application.matsci.jobutils.add_subjob_files_to_backend(subjob, path=None, backend=None, exclude_exts=None, also_input=False, include_only_exts=None)

Add all the output and log files from a subjob to the backend of this job so that they get copied back to the original job directory.

Note

subjob log files are added as output files instead of log files. They will not be streamed back to the original job directory but instead copied back at the end of this job like a normal output file.

Parameters

  • subjob (schrodinger.job.jobcontrol.Job or schrodinger.job.queue.JobControlJob) – The subjob to add files from.

  • path (str) – The path to the subjob directory from where the backend is created if it was not run in the same directory as this job. Use FROM_SUBJOB to get the subjob directory from a JobControlJob object - this will be ignored if subjob is a Job object.

  • backend (schrodinger.job.jobcontrol._Backend) – The backend if one exists

  • exclude_exts (None or list of str) – The output files found with the excluded extensions are not copied back to the original job directory or documented into logger

  • also_input (bool) – Also add the job input files to the backend

  • include_only_exts (None or list of str) – Only the output files found with the included extensions are copied back to the original job directory or documented into logger

schrodinger.application.matsci.jobutils.determine_source_path(backend=None, job=None)

Determine the original job directory. This is obtained from the job control Job object for this process. If no Job object is found, the current directory is used.

Parameters

  • backend (schrodinger.job.jobcontrol._Backend) – The job control backend. Will be used to obtain the job object if no job is supplied. If neither backend or job are supplied, the backend will be obtained from job control (if one exists).

  • job (schrodinger.job.jobcontrol.Job) – The job control job for this process. If not supplied, will be obtained from the backend (if one exists).

Return type

str

Returns

The directory that is the source path. Will be either the OrigLaunchDir property of the job or the current directory if not running under job control.

schrodinger.application.matsci.jobutils.set_source_path(struct, backend=None, job=None, path=None)

Set the source path property to the original job directory. This is obtained from the job control Job object for this process. If no Job object is found, the current directory is used.

Parameters

  • struct (schrodinger.structure.Structure) – The structure to set the property on. Note that property setting works properly on both Structure and Cms objects.

  • backend (schrodinger.job.jobcontrol._Backend) – The job control backend. Will be used to obtain the job object if no job is supplied. If neither backend or job are supplied, the backend will be obtained from job control (if one exists).

  • job (schrodinger.job.jobcontrol.Job) – The job control job for this process. If not supplied, will be obtained from the backend (if one exists).

  • path (str) – Manually set this path to the source directory, overriding all other options

Return type

str

Returns

The directory set as the source path. Will be either the OrigLaunchDir property of the job or the current directory if not running under job control.

schrodinger.application.matsci.jobutils.get_source_path(source, existence_check=True)

Get the source path to the original job directory

Parameters

  • source (schrodinger.structure.Structure or schrodinger.project.ProjectRow) – Either the ProjectRow or the structure to obtain the source information from. If a structure, can be either a Structure or a Cms object.

  • existence_check (bool) – If True (default), a blank string will be returned if the source path does not exist. If False, the path is returned regardless of whether it exists or not.

Return type

str

Returns

The original job directory or a blank string if none is found

schrodinger.application.matsci.jobutils.get_file_path(struct, prop)

Get the path of the file defined by source path and property name

Parameters
Returns

path of the file if found

Return type

str or None

schrodinger.application.matsci.jobutils.prepare_job_spec_builder(argv, program_name, default_jobname, input_fn=None, set_driver_reserves_cores=False, schrodinger_product=None)

Prepare generic job specification builder. If set_driver_reserves_cores script (driver) is set to True, script is expected to use all the cores (cpus), similar to umbrella mode in multisim. For an example see stress-strain driver. For all other cases (such as in opto/hopping/amorphous) keep set_driver_reserves_cores to False.

Parameters

  • argv (list) – The list of command line arguments, including the script name at [0], similar to that returned by sys.argv

  • program_name (str) – Program name

  • default_jobname (str) – Default job name

  • input_fn (str) – Input filename

  • set_driver_reserves_cores (bool) – If True, enable launchapi.setDriverReservesCores

  • schrodinger_product (str) – A product directory to search for the script/executable. This should be the name of a directory under SCHRODINGER without the trailing version (i.e. the “-v*” part).

Return type

launchapi.JobSpecificationArgsBuilder

Returns

Job specification builder object

schrodinger.application.matsci.jobutils.add_desmond_license_to_job_builder(job_builder, license_host=None, toplevel_required=True)

Add desmond GPU license to job builder.

Parameters

  • job_builder (launchapi.JobSpecificationArgsBuilder) – Job specification builder object

  • license_host (str or None) – Host for which license should be generated. If None, use host from the top level args

  • toplevel_required (bool) – If true, it’s required that the current process was executed through toplevel.py, which causes TOPLEVEL_HOST_ARGS to be set

Return type

str or None

Returns

None on success, error message on error

schrodinger.application.matsci.jobutils.add_desmond_license_to_job_spec(job_spec, license_host)

Add desmond GPU license based on the license host to the job spec.

Parameters
Return type

str or None

Returns

None on success, error message on error

schrodinger.application.matsci.jobutils.parse_restart_parameters_file(path)

Parse parameters file.

Format of the file: 1st line is driver’s filename 2nd line is original arguments passed to the driver

Parameters

path (str) – Path to the file with original arguments

Return type

dict

Returns

Dictionary with parsed values

schrodinger.application.matsci.jobutils.write_restart_parameters_file(driver, args, outpath)

Write out original arguments to parameters file and add the file as an output file to any existing jobcontrol backend.

Parameters

  • driver (str) – Driver’s filename

  • args (list) – Original arguments passed to driver

  • outpath (str) – Path to the parameters file to write original arguments

schrodinger.application.matsci.jobutils.get_restart_id_filename(jobname)

For the given restart jobname, return the name of the file containing the job id.

Return type

str

Returns

The name of the restart jobid file

schrodinger.application.matsci.jobutils.extract_job_data(options, add_to_backend=False)

Unzip the archive in the current directory. All the error handling is on the caller function.

Param`argparse.Namespace` options

The object holding all the option values

Parameters

add_to_backend (bool) – Whether to add the files to the JC backend so that they are returned by the job

schrodinger.application.matsci.jobutils.archive_job_data(path, files_path)

Create a gzipped tar archive in the current directory from the provided list of file paths. All the error handling is on the caller function.

Parameters

  • path (files_path) – Path to the new archive to be created

  • path – List of files to be archived

schrodinger.application.matsci.jobutils.create_restart_launcher(script, prog, input_name, output_name, zip_name, options, args)
schrodinger.application.matsci.jobutils.create_restart_jobcmd(driver_path, zip_fn, restart_options, args, default_jobname)

Generate command-line list for the job restart.

Parameters

  • driver_path (str) – Path to the driver

  • zip_fn (str) – Filename of the archive with all restart files

  • restart_options (argparse.Namespace) – The object holding all the option values

  • args (list) – List of the arguments passed to the original run

  • default_jobname (str) – Default job name

Return type

list

Returns

List of parameters ready for job submission

schrodinger.application.matsci.jobutils.write_idfile(jobobj)

Store the job id in a file as a signal to the GUI that a new job has been launched.

Parameters

jobobj (schrodinger.job.jobcontrol.Job) – The object holding all the option values

schrodinger.application.matsci.jobutils.get_string_from_flag(flag)

Return the string from the flag

Parameters

flag (str) – The flag for the desired option

Return type

str

Returns

The string from the flag (the flag minus the leading dash)

schrodinger.application.matsci.jobutils.get_option(options, flag, default=None)

Return the option value associated with flag

Parameters

  • options (argparse.Namespace) – The object holding all the option values

  • flag (str) – The flag for the desired option.

  • default (any) – The default value to return if the flag is not found

Return type

any

Returns

The value associated with flag, or None if flag (minus any leading dashes) is not found as a property on options

schrodinger.application.matsci.jobutils.set_option(options, flag, value)

Set the option value associated with flag

Parameters

  • options (argparse.Namespace) – The object holding all the option values

  • flag (str) – The flag for the desired option. If the string starts with a ‘-’, the ‘-’ is removed.

  • value (any) – The value to set the option.flag value to

schrodinger.application.matsci.jobutils.get_restart_options(options)

Get the command line options from the -restart_x flags

Return type

tuple

Returns

tuple of strings (viewname, incorporation, project, host:cpu, jobname)

schrodinger.application.matsci.jobutils.seed_random_number_generator(options, log=None)

Seed the random number generator based on the command line options. If there is no seed in the command line options, a random value is used.

Parameters

  • options (argparse.Namespace) – The command line options from argparse. Note that passing in None for options will have the same affect as if the seed flag does not exist on options (i.e. a random value will be generated).

  • log (function) – A function to log the seed value. Should take a single str argument.

Return type

int

Returns

The seed used for the generator

schrodinger.application.matsci.jobutils.check_license(panel=None, token=94, name='', as_validator=False, fall_back_tokens=None, matsci_product=True)

Check if a valid token exists. If called from Maestro, also check out and hold a MATERIALSCIENCE_MAIN token.

Parameters

  • panel (schrodinger.ui.qt.appframework.AppFramework) – panel to use to put up an error dialog if no license

  • token (schrodinger.utils.license constant or None) – A token type from the schrodinger.utils.license module, such as MATERIALSCIENCE_MAIN or MATERIALSCIENCE_GA. If not provided, the default material science token will be used if is_cg is False, or the CG token will be used if is_cg is True.

  • name (str) – The user-identifiable name for the token - used for error messages. If not provided, the string used in the license module for this token (if one exists) will be used.

  • as_validator (bool) – If True, this function will work as an AF2 validation method. Instead of posting a dialog or printing a message for a failed license check, it will return (False, error_message).

  • fall_back_tokens (list or None) – if present specifies that if the intended license check from the given token or name fails then attempt to get a valid license by running through this ordered list of fall back tokens, each of which is a schrodinger.utils.license constant for a non-MATSCI product

  • matsci_product (bool) – True if this check is for a MatSci product, False if not. If True, and run from a Maestro session, this function will ensure that a single MS MAIN license token will be checked out and held for this Maestro session. False indicates that this call is from a non-MatSci panel and should not ensure that a MS MAIN token is being held for this Maestro session.

Return type

bool or (bool, str)

Returns

True if valid license exists. If no valid license exists, False will be returned by default, but (False, msg) will be returned if as_validator=True. Note that (False, msg) evalutes to True so must be handled by the calling routine as not a boolean if as_validator=True.

schrodinger.application.matsci.jobutils.check_licenses(*tokens, as_validator=False)

Check if valid tokens exist. If called from Maestro, also check out and hold a MATERIALSCIENCE_MAIN token.

Parameters

  • tokens (list[schrodinger.utils.license]) – List of license tokens

  • as_validator (bool) – If True, this function will work as an AF2 validation method. Instead of posting a dialog or printing a message for a failed license check, it will return (False, error_message).

Return type

bool or (bool, str)

Returns

True if valid license exists. If no valid license exists, False will be returned by default, but (False, msg) will be returned if as_validator=True. Note that (False, msg) evalutes to True so must be handled by the calling routine as not a boolean if as_validator=True.

schrodinger.application.matsci.jobutils.check_cg_license(*args, is_cg, **kwargs)

A wrapper around check_license that checks the appropriate license for the given the panel is using coarse grained structures or not.

Parameters

is_cg (bool) – True if the panel is using coarse grained structures, False if not

Return type

bool or (bool, str)

Returns

True if valid license exists. If no valid license exists, False will be returned by default, but (False, msg) will be returned if as_validator=True. Note that (False, msg) evalutes to True so must be handled by the calling routine as not a boolean if as_validator=True.

schrodinger.application.matsci.jobutils.create_run_dir(panel, jobname)

Create a subdirectory to run a job in, asking the user and removing existing directories if needed.

Parameters
Return type

str or None

Returns

The path to the directory or None if an existing directory was found and the user elected not to remove it

schrodinger.application.matsci.jobutils.string_to_value(string)

Change a text string from a file to a value. Converts string values of special Python tokens such as True, False or None to the Python tokens. Converts numbers to int or float if possible.

Parameters

string (str) – The string to convert

Returns

string converted to, in order of preference: [True|False|None], int, float, or input type

schrodinger.application.matsci.jobutils.working_directory(path)

A context manager which changes the working directory to the given path, and then changes it back to its previous value on exit.

class schrodinger.application.matsci.jobutils.StringCleaner(extra_replacement_pairs=None, separator='-')

Bases: object

Manages the cleaning of strings.

__init__(extra_replacement_pairs=None, separator='-')

Populate an instance with some defaults. The replacement dictionary needs to be set such that the most specific replacements occur last. This is because the replacements should be done in a certain order, for example (‘C:', ‘’) should be done before (‘:’, ‘’) and (’', ‘’), and because people tend to append to an iterable rather than prepend we will traverse the iterable backwards.

Parameters

  • extra_replacement_pairs (list of tuples) – each tuple in this list contains a single replacement pair, i.e. a single substring to be replaced and a single substring to replace it.

  • separator (str) – in the case of non-unique strings this is the string that separates the non-unique part from the number of times used which is the unique part.

cleanAndUniquify(input_str, clear_prev=False, max_len=100)

Shorten if necessary, replace certain characters in an input string and then uniquify the string by comparing with a dictionary of previous names and number of times used.

Parameters

  • input_str (str) – the input string we want cleaned and uniqified

  • clear_prev (bool) – specify if the dictionary of previous names should first be cleared

  • max_len (int) – maximum length of the input_str allowed, otherwise it will be shortened to the max_len value

Return type

str

Returns

the input string now cleaned and uniquified

schrodinger.application.matsci.jobutils.clean_string(string, default='title')

Cleans the given string by removing special characters to make it acceptable for a file name. If the string is blank, it will be replaced by the value of default.

Parameters

  • string (str) – The string to clean.

  • default (str) – The name to use if string is blank

Return type

str

Returns

A string usable as a filename

schrodinger.application.matsci.jobutils.zip_and_set_incorporation(zipname, filelist)

Zip up all the requested files and set the resulting archive as the job control backend structure output file (if runnning under job control).

Parameters

  • zipname (str) – The name of the archive to create

  • filelist (list) – Each item in filelist is the name of a file to add to file zipname

class schrodinger.application.matsci.jobutils.CellRunInfo(structs, basename, replicate, multiple_cells, component=None, repeat_unit_info=None)

Bases: object

Holds the information for the run for a single cell

__init__(structs, basename, replicate, multiple_cells, component=None, repeat_unit_info=None)

Create a CellRunInfo object

Parameters

  • structs (list) – The list of structures to include in the cell

  • basename (str) – The generic basename for job files. This will be modified based on the value of replicate, component and multiple_cells to form the base name for files for this specific cell.

  • replicate (int) – Which replicate this is for - 1-based

  • multiple_cells (bool) – Whether there will be multiple replicates of this cell

  • component (int or None) – The structure number this cell is for, or None if this is a mixed structure cell

  • repeat_unit_info (list or None) – Each item of the list is a tuple. The first item of the list is the sequence of monomer one-letter codes that give the repeat unit sequence. The second item is a tag to be added to the polymer name for that sequence (used for enumerated sequences)

class schrodinger.application.matsci.jobutils.MultijobDriver(runlist, options, args, log, remote_script, default_jobname, basename=None)

Bases: object

Resubmit the driver as subjobs

__init__(runlist, options, args, log, remote_script, default_jobname, basename=None)

Create multiple cells in parallel running a subjob for each cell. Zip up the resulting cms files into a jobname.zip file and set it as the structure output file to be incorporated.

Parameters

  • runlist (list of CellRunInfo) – Each item of runlist will generate a subjob and a single cell

  • options (argparse.Namespace) – The command line options

  • args (iterable) – The command line arguments as passed in by sys

  • log (function) – function(msg) writes msg to log file

  • remote_script (string) – the dir and name for driver to resubmit

  • default_jobname (string) – Default job name

  • basename (str or None) – The generic basename defined from inputfile.

removeFlag(args, flag, and_value=False)

Remove a flag from the command line flags

Parameters

  • args (list) – The list of command line arguments

  • flag (str) – The flag to remove

  • and_value (bool) – Also remove the value associated with the flag - it is assumed that this is the following list item

replaceValue(args, old_value, new_value)

Replace the list item with value=old_value with the new value

Parameters

  • args (list) – The list of command line arguments

  • old_value (str) – The value to replace

  • new_value (str) – The value to replace old_value with

removeUnwantedKeywords(subargs)

Remove keywords from the original command line arguments that should not be passed on to any subjobs. This modifies the subargs list in place.

Parameters

subargs (list) – The command line arguments for a subjob

removePerSubjobUnwantedKeywords(runinfo, subargs)

Remove keywords from the command line arguments that should not be passed on to this specific subjob. This modifies the subargs list in place.

Parameters

  • runinfo (CellRunInfo) – The info for this subjob

  • subargs (list) – The command line arguments for a subjob

multijobDriver(runlist, args, basename, log, remote_script)

Create multiple cells in parallel running a subjob for each cell. Zip up the resulting cms files into a jobname.zip file and set it as the structure output file to be incorporated.

Parameters

  • runlist (list of CellRunInfo) – Each item of runlist will generate a subjob and a single cell

  • args (iterable) – The command line arguments as passed in by sys

  • basename (str) – The zipped .zip or . maegz filename for all job files.

  • log (function) – function(msg) writes msg to log file

  • remote_script (string) – the dir and name for driver to resubmit

schrodinger.application.matsci.jobutils.get_jobname(default_jobname)

Return a the jobname from backend, command line (-NAMEJOB / environment), DEFAULT_JOBNAME

Parameters

default_jobname (str) – default_jobname of the current module

Return type

string

Returns

Jobname

schrodinger.application.matsci.jobutils.get_procs()

Get number of processors from backend or command-line arguments.

Return type

int

Returns

Number of processors

schrodinger.application.matsci.jobutils.memory_usage_psutil()

return the memory usage in MB

Return type

float

Rparam

memory usage in MB

schrodinger.application.matsci.jobutils.get_size_of(an_object, size_unit='megabytes')

Return the size of an object in size_unit. The object can be any type of object. All built-in objects will return correct results, but this does not have to hold true for third-party extensions as it is implementation specific.

Parameters

an_object (any type of python object) –

Returns

the size of an object in size_unit

Return type

float

schrodinger.application.matsci.jobutils.get_jobhosts(ncpu=None)

Return the job hosts from backend or command line.

Parameters

ncpu (int or None) – number of processors

Return type

None or list of tuple

Rparam

the hosts or None

schrodinger.application.matsci.jobutils.get_jobhost()

Return the first job hosts from backend or command line.

Return type

list or None

Rparam

the first host or None

schrodinger.application.matsci.jobutils.get_jobhost_name()

Return the job host name from backend or command line.

Return type

str or None

Rparam

the host name

schrodinger.application.matsci.jobutils.get_backend_hosts_str()

Get backend host(s) as passed to the -HOST flag. This can be useful when a subjob needs to resubmit its own subjobs to the original queue via subhost and the current job may be running on localhost due to smart distribution being enabled in the parent job.

For the hierarchy of submission: driver.py -> opto_driver/pdft_driver -> backend subjobs

opto_driver.py will get ‘localhost’ as HOST because of the smart distribution, its subjobs still need to go to the queue. In order to do this, opto_driver.py can be submitted with ‘-SUBHOST get_queue_host()’.

We want to keep smart distribution in the driver.py jobdj, so that opto_driver is not taking another slot in the queue.

Return type

str or None

Returns

Backend hosts or None if host is to determined by the JC

schrodinger.application.matsci.jobutils.is_jobhost_gpu_available(hostname=None)

Check whether the gpu is available on the host. If hostname is provided, check will be performed on given host. If hostname is not provided, check SUBHOST, if defined. Then, check HOST, if defined. At last, check localhost.

Parameters

hostname (str) – the hostname to check gpu availability

Return type

bool

Rparam

True means gpu is available.

schrodinger.application.matsci.jobutils.add_zipfile_to_backend(adir)

Add a zip file of the given directory to the job backend.

Parameters

adir (str) – the directory

schrodinger.application.matsci.jobutils.get_backend_first_host()

Get backend first host.

Return type

str, int

Returns

host, number of processors

schrodinger.application.matsci.jobutils.write_cms_with_wam(cms_model, filename, wam_type)

Write the cms model to a file with the provided WAM property

Parameters

  • cms_model (cms.Cms) – The cms model to write to file

  • filename (str) – The cms path

  • wam_type (int) – One of the enums defined in workflow_action_menu.h

schrodinger.application.matsci.jobutils.add_wam_to_cms(filename, wam_type)

Rewrite the cms with the WAM type added

Parameters

  • filename (str) – The cms path

  • wam_type (int) – One of the enums defined in workflow_action_menu.h

schrodinger.application.matsci.jobutils.write_mae_with_wam(structs, filename, wam_type=None)
Parameters

  • struct (list(structure.Structure)) – The structures to write to file

  • filename (str) – The mae path

  • wam_type (int) – One of the enums defined in workflow_action_menu.h

Raises

ValueError – If the file path is not for a Maestro file

schrodinger.application.matsci.jobutils.set_structure_wam(struct, wam_type)

Sets the WAM property for the passed structure

Parameters

  • struct (structure.Structure) – The structure to set the WAM for

  • wam_type (int) – One of the enums defined in workflow_action_menu.h

schrodinger.application.matsci.jobutils.remove_structure_wam(struct)

Remove the structure WAM property from the passed structure

Parameters

struct (structure.Structure) – The structure to remove property from

schrodinger.application.matsci.jobutils.get_smart_distribution_from_environ(default=True)

Get smart distribution of the queue value based on the environment variable.

Parameters

default (bool) – Default value if env variable is not set. Must be aligned with the default value of the queue.JobDJ.smart_distribution

Return type

bool

Returns

Smart distribution value

schrodinger.application.matsci.jobutils.set_smart_distribution_from_environ(jobq, log=None)

Set smart distribution of the queue to on/off based on the environment variable.

Parameters

  • jobq (schrodinger.job.queue.JobDJ) – The JobDJ object

  • log (function or None) – A function to log smart distribution status. Should take a single str argument.

class schrodinger.application.matsci.jobutils.LoggingSubprocessJob(cmd, subdir)

Bases: schrodinger.job.queue.SubprocessJob

A Subprocess job that stores its output in a log file

__init__(cmd, subdir)

Create an instance

Parameters

  • cmd (list) – The command as a list of strings

  • subdir (str) – The absolute subdirectory path to run the command in

doCommand(*args, **kwargs)

Overwrite the parent method to open the log file before running

postCommand()

Overwrite the parent method to close the log file

addFinalizer(function: Callable[[schrodinger.job.queue.BaseJob], None], run_dir: Optional[str] = None)

Add a function to be invoked when the job completes successfully.

See also the add_multi_job_finalizer function.

addGroupPrereq(job: schrodinger.job.queue.BaseJob)

Make all jobs connected to job prerequisites of all jobs connected to this Job.

addPrereq(job: schrodinger.job.queue.BaseJob)

Add a job that is an immediate prerequisite for this one.

cancel()

Send termination request to subprocess managed job. This method will eventually deprecate SubprocessJob.kill

finalize()

Clean up after a job successfully runs.

genAllJobs(seen: Optional[Set[schrodinger.job.queue.BaseJob]] = None) → Generator[schrodinger.job.queue.BaseJob, None, None]

A generator that yields all jobs connected to this one.

genAllPrereqs(seen=None) → Generator[schrodinger.job.queue.BaseJob, None, None]

A generator that yields all jobs that are prerequisites on this one.

getCommand() → List[str]

Return the command used to run this job.

getCommandDir() → str

Return the launch/command directory name. If None is returned, the job will be launched in the current directory.

getDuration() → Optional[int]

Return the CPU time of the job in seconds.

If the job is still running, returns None.

getJobDJ() → schrodinger.job.queue.JobDJ

Return the JobDJ instance that this job has been added to.

getPrereqs()

Return a set of all immediate prerequisites for this job.

getStatusStrings() → Tuple[str, str, str]

Return a tuple of status strings for printing by JobDJ.

The strings returned are (status, jobid, host).

hasExited() → bool

Returns True if this job finished, successfully or not.

hasStarted() → bool

Returns True if this job has started (not waiting)

init_count = 0
isComplete() → bool

Returns True if this job finished successfully

kill()

Send termination request to subprocess managed job.

maxFailuresReached(msg: str)

This is a method that will be called after the job has failed and the maximum number of failures per JobDJ run has been reached. After invoking this method, JobDJ will raise a RuntimeError and the process will exit.

preCommand()

A method to make pre-command changes, like cd’ing to the correct directory to run the command in.

run(*args, **kwargs)

Run the job.

The steps taken are as follows:

  1. Execute the preCommand method for things like changing the working directory.

  2. Call the doCommand to do the actual work of computation or job launching.

  3. Call the postCommand method to undo the changes from the preCommand that need to be undone.

runsLocally() → bool

Return True if the job runs on the JobDJ control host, False if not. Jobs that run locally don’t need hosts.

There is no limit on the number of locally run jobs.

setup()

A method to do initial setup; executed after preCommand, just before doCommand.

property state: schrodinger.job.queue.JobState

Return the current state of the job.

Note that this method can be overridden by subclasses that wish to provide for restartability at a higher level than unpickling BaseJob instances. For example, by examining some external condition (e.g. presence of output files) the state JobState.DONE could be returned immediately and the job would not run.

update()

Update the current job status and set state.

class schrodinger.application.matsci.jobutils.RobustSubmissionJob(command: List[str], command_dir: Optional[str] = None, name: Optional[str] = None, max_retries: Optional[int] = None, timeout: Optional[int] = None, launch_timeout: Optional[int] = None, launch_env_variables: Optional[Dict[str, str]] = None, resource_requirement: Optional[Union[str, schrodinger.job.resource.ComputeType]] = None, license_requirement: Optional[List[str]] = None, smart_dist_eligible: Optional[bool] = True, **kwargs)

Bases: schrodinger.job.queue.JobControlJob

A JobControlJob object that will resubmit multiple times for fizzled jobs (if the queue setting is such) but by default will not resubmit jobs that died.

retryFailure(max_retries=0)

Determine if the job should be retried or not. This overwrites the parent method to by default not retry jobs that have a status of “died” as that will indicate that Jaguar failed, which it almost certainly will again.

Parameters

max_retries (int) – The queue’s max_retries parameter

Return type

bool

Returns

True if the job should be retried, False if not

__init__(command: List[str], command_dir: Optional[str] = None, name: Optional[str] = None, max_retries: Optional[int] = None, timeout: Optional[int] = None, launch_timeout: Optional[int] = None, launch_env_variables: Optional[Dict[str, str]] = None, resource_requirement: Optional[Union[str, schrodinger.job.resource.ComputeType]] = None, license_requirement: Optional[List[str]] = None, smart_dist_eligible: Optional[bool] = True, **kwargs)

Job constructor.

Parameters

  • command – The command that runs the job.

  • command_dir – The directory from which to run the command.

  • name – The name of the job.

  • max_retries – Number of allowed retries for this job. If this is set, it is never overridden by the SCHRODINGER_MAX_RETRIES environment variable. If it is not set, the value of max_retries defined in JobDJ is used, and SCHRODINGER_MAX_RETRIES can be used to override this value at runtime. To prevent this job from being restarted altogether, set max_retries to zero.

  • timeout – Timeout (in seconds) after which the job will be killed. If None, the job is allowed to run indefinitely.

  • launch_timeout – Timeout (in seconds) for the job launch process to complete. If None, a default timeout will be used for jobserver and old jobcontrol jobs ( see get_default_timeout() ) unless a value for job timeout parameter is passed and is not greater than the default timeout.

  • launch_env_variables – A dictionary with the environment variables to add when the jobcontrol job is launched. The name of any additional variables to set should be in the keyword of the dict and the value should be the corresponding value. These will be added to any environment variables already present, but removed after the job has been launched.

  • kwargs – Additional keyword arguments. Provided for consistency of interface in subclasses.

  • resource_requirement – Whether the job will require special compute resources, such as GPU.

  • license_requirement – List of license tokens required for the job to be used for license checking when SMART_LICENSE_CHECK feature flag is turned on. This is useful for license checking the first job of the smart distribution launched directly to the localhost without canceling from the queue. The license requirements are not known until the job is launched. Each license token is in the form ‘TOKEN’ or ‘TOKEN:n’ where TOKEN is the name of the license, and n is the number of tokens.

  • smart_dist_eligible – Whether this job can be submitted via smart distribution (True) or not (False). This setting only comes into play if all other requirements (such as the resource_requirement, license requirement, number of processors, and smart distribution being turned on) are met. In other words, setting it to True will not force the job to run via smart distribution, but setting it to False will ensure that it does not.

acquireLicenseForSmartDistribution() → bool

Acquire and hold licenses for a smart distribution job. This makes sure the job won’t fail due to unavailable licenses.

Returns True if the licenses registered for the job are acquired, and False if they are not. If no licenses are registered, it always returns True to avoid preventing jobs from using the smart distribution feature. For legacy jobcontrol, the license check is not performed, and is always returned True. We want to use this feature as a pitch to move users to JOB_SERVER.

addFinalizer(function: Callable[[schrodinger.job.queue.BaseJob], None], run_dir: Optional[str] = None)

Add a function to be invoked when the job completes successfully.

See also the add_multi_job_finalizer function.

addGroupPrereq(job: schrodinger.job.queue.BaseJob)

Make all jobs connected to job prerequisites of all jobs connected to this Job.

addLaunchEnv(key: str, val: str)

Adds the given environment key and and value to the list of launch environment.

Parameters

  • key – environment key to add to the launch environment.

  • val – environment value associcated with the key to add to the launch environment.

addPrereq(job: schrodinger.job.queue.BaseJob)

Add a job that is an immediate prerequisite for this one.

cancel()

Send kill request to jobcontrol managed job. This method will eventually deprecate JobControlJob.kill

cancelSubmitted(do_license_check: bool = False) → schrodinger.job.queue.CancelSubmittedStatus

If the job is still in the ‘submitted’ state, cancel it, purge the jobrecord and set the job handle to None. This tries to acquire licenses for the job before canceling from the queue if do_license_check is turned on.

Parameters

do_license_check – Acquire licenses for the job before canceling from the queue.

Returns one of the status of CancelSubmittedStatus.

doCommand(host: str, local: bool = False)

Launch job on specified host using jobcontrol.launch_job().

Parameters

  • host – Host on which the job will be executed.

  • local – Removed in JOB_SERVER.

finalize()

Clean up after a job successfully runs.

genAllJobs(seen: Optional[Set[schrodinger.job.queue.BaseJob]] = None) → Generator[schrodinger.job.queue.BaseJob, None, None]

A generator that yields all jobs connected to this one.

genAllPrereqs(seen=None) → Generator[schrodinger.job.queue.BaseJob, None, None]

A generator that yields all jobs that are prerequisites on this one.

getCommand() → List[str]

Return the command used to run this job.

getCommandDir() → str

Return the launch/command directory name. If None is returned, the job will be launched in the current directory.

getDuration() → Optional[int]

Return the duration of the Job as recorded by job server. The duration does not include queue wait time.

If the job is running or has not launched, returns None.

Note that this method makes a blocking call to the job server.

getJob() → Optional[schrodinger.job.jobcontrol.Job]

Return the job record as a schrodinger.job.jobcontrol.Job instance.

Returns None if the job hasn’t been launched.

getJobDJ() → schrodinger.job.queue.JobDJ

Return the JobDJ instance that this job has been added to.

getPrereqs()

Return a set of all immediate prerequisites for this job.

getStatusStrings() → Tuple[str, str, str]

Return a tuple of status strings for printing by JobDJ.

The strings returned are (status, jobid, host).

hasExited() → bool

Returns True if this job finished, successfully or not.

hasStarted() → bool

Returns True if this job has started (not waiting)

init_count = 0
isComplete() → bool

Returns True if this job finished successfully

kill()

Send kill request to jobcontrol managed job

maxFailuresReached(msg: str)

Print an error summary, including the last 20 lines from each log file in the LogFiles list of the job record.

postCommand()

A method to restore things to the pre-command state.

preCommand()

A method to make pre-command changes, like cd’ing to the correct directory to run the command in.

run(*args, **kwargs)

Run the job.

The steps taken are as follows:

  1. Execute the preCommand method for things like changing the working directory.

  2. Call the doCommand to do the actual work of computation or job launching.

  3. Call the postCommand method to undo the changes from the preCommand that need to be undone.

runsLocally() → bool

Return True if the job runs on the JobDJ control host, False if not. Jobs that run locally don’t need hosts.

There is no limit on the number of locally run jobs.

setup()

A method to do initial setup; executed after preCommand, just before doCommand.

property state: schrodinger.job.queue.JobState

Return the current state of the job.

Note that this method can be overridden by subclasses that wish to provide for restartability at a higher level than unpickling BaseJob instances. For example, by examining some external condition (e.g. presence of output files) the state JobState.DONE could be returned immediately and the job would not run.

update()

Checks for changes in job status, and updates the object appropriately (marks for restart, etc).

Raises

RuntimeError – if an unknown Job Status or ExitStatus is encountered.

usesJobServer() → bool

Detect, by looking at the jobId, whether this job uses a job server.

schrodinger.application.matsci.jobutils.create_cpu_gpu_queue(options, sd_resources=None, **kwargs)

Create a JobDJ that can manage both a cpu and a gpu queue simultaneously

Parameters

Additional keyword arguments are passed to the create_queue function

Return type

schrodinger.job.queue.JobDJ

Returns

The JobDJ object

schrodinger.application.matsci.jobutils.create_queue(options=None, host=None, **kwargs)
Create a JobDJ job with some default values and the given keyword arguments

Current defaults: - verbosty: normal - max_failures: NOLIMIT - max_retries: 3

Parameters

  • options (argparse Namespace object) – the hostlist will be formed from the options.host property if not supplied by the host argument

  • host (str or list) – The host string to use to create the queue. If not provided, or if the module constant AUTOHOST, then the host will be determined automatically by job control from the toplevel -HOST and -SUBHOST arguments. AUTOHOST is synonymous with None. If host is a list, it will be used directly as the hosts argument for JobDJ.

All other keyword arguments will be passed on to the JobDJ object

Return type

schrodinger.job.queue.JobDJ

Returns

The JobDJ object

Raises

ValueError – if host has the wrong type

schrodinger.application.matsci.jobutils.get_all_subjobs(job)

Get all subjobs (and recursively all subjobs of subjobs) of the given job

Parameters

job (jobcontrol.Job) – The job to get subjobs of

Return type

list

Returns

A list of all subjobs stemming from the current job. Each item is a jobcontrol.Job object

schrodinger.application.matsci.jobutils.is_job_server_job(job)

Check if the job is/was run under Job Server

Parameters

job (jobcontrol.Job) – The job to check

Return type

bool

Returns

Whether the job was run under job server

schrodinger.application.matsci.jobutils.is_downloadable_job_server_job(job)

Check if the job is/was run under Job Server and has not yet downloaded its files

Parameters

job (jobcontrol.Job) – The job to check

Return type

bool

Returns

Whether the job was run under job server and has not downloaded files

schrodinger.application.matsci.jobutils.get_job_from_hub(jobid)

Get a job object for the given job id

Parameters

jobid (str) – The job id

Return type

jobcontrol.Job

Returns

The job object for this job id

Raise

jobhub.StdException If there is no job found for jobid. One possible reason for this is that JobDJ smart distribution cancelled that subjob after it was submitted.

class schrodinger.application.matsci.jobutils.JSFilePathData(path, job)

Bases: object

Manage info for a file that belongs to a currently running job server job

The object has attributes for the associated job and file name. Attributes requests not found on this class will be passed on to the pathlib.Path file name attribute.

__init__(path, job)

Create a JSFilePathData object

Parameters

  • path (str) – The name of the file

  • job (jobcontrol.Job) – The job this file is associated with

exception schrodinger.application.matsci.jobutils.FileDownloadError

Bases: Exception

Raised by FileDownloader for any error

__init__(*args, **kwargs)
args
with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

class schrodinger.application.matsci.jobutils.FileDownloader

Bases: object

Manage retrieving file information from a running Job Server job and downloaded associated files

__init__()

Create a FileDownloader instance

static interpretJobServerError(msg, checkhost=True)

Form a more user-friendly error message for some known Job Server errors

Parameters

  • msg (str) – The Job Server error message

  • checkhost (bool) – Whether to try to extract the host name from the message

Return type

str

Returns

A modified error message, or the original message if no modification is found

listAvailableFiles(job, subjobs=True, wait=False)

Get file names available on the server for the given job

Parameters

  • job (jobcontrol.Job) – The job to get file names for

  • subjobs (bool) – Also get file names for subjobs of the given job

  • wait (bool) – If True, wait until the process finishes and return the file info. If False, start a process to retrieve the file names and return. The calling function will need to call the parseAvailableFiles method when the process completes. Note that using wait=True can freeze a GUI for a long time until the network process times out. See the jobdirdlg.InteractiveFileDownloader class.

Return type

list or None

Returns

If wait is False, the process attribute is the running process. If wait is True, a list of JSFilePathData objects is returned for each file found on the server.

Raises

FileDownloadError – if an error occurs while retrieving names

raiseProcessError()

Raise an exception with the stderr text from the current process

Raises

FileDownloadError – Raised with stderr output from the process

parseAvailableFiles()

Parse the available file data from a process started by get_files_available_on_server

Return type

list

Returns

Each item of the list is a JSFilePathData object for a file found on the server

Raises

  • RuntimeError – If no process has been started

  • FileDownloadError – If unable to parse the data

downloadJobFileToTemp(job, filename, temp_path=None, wait=False)

Tail the given file on the server to a local file. This does not count as “downloading” the file in Job Server terms (i.e. the file will still be downloaded to the job directory at the end of the job if requested by Job Server settings).

It is up to the calling code to remove the temporary file created by this method.

Parameters

  • job (jobcontrol.Job) – The job the file is associated with

  • filename (str) – The name of the file

  • temp_path (str) – The path to the file to write. If not given, a temp path will be created.

  • wait (bool) – If wait is True, wait for the file to download and return the name of the temp file. If wait is False, start the download process and return None while the process runs. Note that using wait=True can freeze a GUI for a long time until the network process times out. See the jobdirdlg.InteractiveFileDownloader class

Return type

str or None

Returns

If wait is False, the process attribute is the running process and None is returned. If wait is True, the full path of the file created is returned.

Raises

FileDownloadError – if an error occurs while downloading

cleanFiles()

Remove any files created by this downloader

schrodinger.application.matsci.jobutils.register_driver_log(job_builder, default_job_name)

Register the driver log file upfront in the given job builder specfication.

Parameters

  • job_builder (launchapi.JobSpecificationArgsBuilder) – job specification builder object

  • default_job_name (str) – the default job name to fall back on

schrodinger.application.matsci.jobutils.add_jaguar_lic(job_builder, njobs=1)

Add a license file to the given job builder specification.

Parameters

  • job_builder (launchapi.JobSpecificationArgsBuilder) – job specification builder object

  • njobs (int) – The number of jobs on localhost.

schrodinger.application.matsci.jobutils.SubjobInfo

alias of schrodinger.application.matsci.jobutils.Subjob_Info

schrodinger.application.matsci.jobutils.create_subjob(script_path, cls, jobname, options=None, input_files=(), setup_logging=True, job_kwargs=None)

Create a subjob to run a class in a subdirectory of the current directory, with the subdirectory name being the jobname.

Parameters

  • script_path (str) – The path to the script containing the class

  • cls (class) – The class to run

  • jobname (str) – The name of the subjob

  • options (dict or None) – The options to pass to the class constructor, as a dict

  • input_files (tuple) – Tuple of input files to register with jobcontrol

  • setup_logging (bool) – If True, automatically set up logging for the subjob

  • job_kwargs (dict) – Keyword arguments to pass to the job constructor

schrodinger.application.matsci.jobutils.run_single_subjob(cmd, jdj, user_facing_name, job_kwargs=None, logger=None)

Run a single subjob and do error handling and logging for it

Parameters

  • cmd (list) – The command to run

  • jdj (queue.JobDJ) – The jobdj to use

  • user_facing_name (str) – The name of the subjob to display to the user

  • job_kwargs (dict) – Keyword arguments to pass to the job constructor

  • logger (logging.Logger or None) – The logger to use

Return type

bool

Returns

True if the job succeeded, False otherwise

schrodinger.application.matsci.jobutils.driver_main(driver_class, input_flag=None, license_token=None, des_platform_error=False)

Create a main function for a driver module

Parameters

  • driver_class (type) – The driver class. It should take (options, jobname) as constructor arguments and have a run method for running the calculations. The module is also found using this class.

  • input_flag (str) – The name of the input flag to use to get the jobname. If not provided, use module.JOBNAME instead of the input file basename.

  • license_token (schrodinger.utils.license) – The license token to use, if anything other than MS MAIN

  • des_platform_error (bool) – If True, exit with an error if Desmond cannot be run on this platform

Return type

callable

Returns

The main function for the driver module

schrodinger.application.matsci.jobutils.create_job_spec_function(driver_class, input_flags=None, md_umbrella=False, add_jag_lic_if_needed=False, custom_script=False, customize_func=None)

Create and return a get_job_spec_from_args function for a driver

Parameters

  • driver_class (type) – The driver class. The module is found using this class. Does not need to have any specific format.

  • input_flags (list) – List of input flags to use to register the input files. Should not include any optional inputs.

  • md_umbrella (bool) – If True, add desmond license for md umbrella

  • add_jag_lic_if_needed (bool) – If True, add Jaguar license to jobbuilder when TPP is 1

  • custom_script (bool) – If True, register the script as an input file

  • customize_func (callable) – A function to call to customize the jobbuidler, in case of optional input files for instance

Return type

callable

Returns

The get_job_spec_from_args function

schrodinger.application.matsci.jobutils.cms_trj_optional_job_spec(job_builder, options, parser)

Add optional cms and trj input files to job builder

Parameters