schrodinger.utils.installation_check module

schrodinger.utils.installation_check.get_distro_package() → str

Returns the distro package to use for package requirements check handling the special case for distros.

schrodinger.utils.installation_check.ensure_logfile(output_dir=None)

Context manager for adding a logfile to LOGGERS. Ensures that multiple nested entries into the context manager only add one logfile.

schrodinger.utils.installation_check.main() → int

Run installation check.

Returns

exit code

schrodinger.utils.installation_check.cd(path)

schrodinger.utils.installation_check.get_host_entries(args)

Get all the host entries (jobcontrol.Host objects) from the hosts file. If no -test arguments were given, return them all. If -test arguments were given, return entries whose names are given by those arguments and which are present in the hosts file (warn on absent ones)

schrodinger.utils.installation_check.run_test_jobs(entries, launch_dir='.', duration=10, timeout=None)

Run testapp jobs using a list of host entries (jobcontrol.Host objects). Return a list of all jobids, succeeded and failed, that were run

schrodinger.utils.installation_check.get_postmortem_command(jobids: List[str]) → List[str]

Return correct postmortem command.

schrodinger.utils.installation_check.run_postmortem(jobids)

Run postmortem for a given set of jobids, log the results into the “postmortem.log”

schrodinger.utils.installation_check.https_session(max_retries=5)

schrodinger.utils.installation_check.download_linux_packages_requirement(sess: requests.sessions.Session, file_to_download: str) → str

Download the given file using the given requests session to a temporary location and return the path.

schrodinger.utils.installation_check.env_update_and_run(cmd: List) → subprocess.CompletedProcess

Run bash script in an isolated environment.

schrodinger.utils.installation_check.test_deps()

Determine if we have required packages to satisfy the suite dependencies.

If test fails, raises RuntimeError with suggestion on how to fix.

schrodinger.utils.installation_check.get_installation_check_commands()

Returns a map of diagnostic commands with:

key: command description value: args list

schrodinger.utils.installation_check.run_installation_check(output_dir='.')

Runs diagnostic commands and writes output in the directory specified. Note: licadmin will write the license - info file in the current directory, not in output_dir. We may want to remove the output_dir argument and leave it to the caller to cd to the correct directory

schrodinger.utils.installation_check.print_installation_check(output_file=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

Runs diagnostic commands and writes output to the single file specified. This will not run licadmin to avoid generating any extraneous files.

schrodinger.utils.installation_check.queue_inst_file(queue_name, filename)

schrodinger.utils.installation_check.queue_orig_dir(queue_name)

schrodinger.utils.installation_check.queue_orig_file(queue_name, filename)

schrodinger.utils.installation_check.queue_config_diffs()

Return a collection of diffs of config and template queue files (original version from the installation vs what’s in the user’s queues dir)

schrodinger.utils.installation_check.queue_config_file_pairs()

Return a list of pairs of config/template queue files to be compared, and a list of warnings. A pair is one file from the user’s installation and the original version of the same file (from the ‘orig’ subdirectory). A warning is produced whenever the ‘orig’ subdirectory is not found, or when a user’s or original file that should be present, is not.

schrodinger.utils.installation_check.queue_config_files_diff(inst_filepath, orig_filepath)

Return a diff (list of strings) of two config/template queue files

schrodinger.utils.installation_check.log_banner(logger)

Log the basic information.

schrodinger.utils.installation_check.check_running_as_root(logger)

Produce a warning if we are running as root.

schrodinger.utils.installation_check.parse_args(args=None)

Parse cmdline arguments.

class schrodinger.utils.installation_check.TestJobDJ(entries, basedir=None, *args, **kwargs)

Bases: schrodinger.job.queue.JobDJ

A flavor of JobDJ that takes a list of host entries (jobcontrol.Host objects) and lets the caller schedule exactly one job per entry.

Create an instance of this class, add the jobs, and call run().

__init__(entries, basedir=None, *args, **kwargs)

Constructor.

Parameters

• hosts – A list of hosts to run on. Items of the list can be a (<hostname>, <maximum_concurrent_subjobs>) tuple that gives a host name and the maximum number of processors that can be used on that host. Or, items of the list can be a ResourceHost object that provides a detailed listing of the number of processors on each host that can be used for jobs requesting specific resources. The list can mix tuple and ResourceHost items. If a host appears more than once (whether as a tuple or ResourceHost), the information in the multiple entries is added together. Processors in the tuple form are only considered usable by jobs that have resource_requirement=None (The default state for jobs). The default value of None for this parameter means the host information is determined automatically from the command line and is usually desired.

• local – No longer functional in JOB_SERVER.

• max_retries – Number of allowed retries per subjob. If this is set, it is never overridden by the SCHRODINGER_MAX_RETRIES environment variable. If it is not set, the value in default_max_retries is used, and SCHRODINGER_MAX_RETRIES is allowed to override. If you wish to disable restarting altogether, set this value to zero.

• default_max_retries – Number of allowed retries per subjob. This value can always be overridden by the SCHRODINGER_MAX_RETRIES environment variable. Default is zero.

• max_failures – Total number of allowed subjob failures before JobDJ exits. If it is not defined, a default of zero will be used (exit on any failure after attempting to restart), but this can be overridden with the SCHRODINGER_MAX_FAILURES environment variable. To allow an unlimited number of subjob failures, set max_failures to the module level NOLIMIT constant.

• verbosity – There are three allowed verbosity levels: “quiet” - only warnings and errors are printed; “normal” - JobDJ progress is printed; and “verbose” - additional debugging info is printed. Default is “quiet”.

• job_class – The class to use as the default job constructor when the addJob argument is not a BaseJob instance.

• update_delay – The number of seconds to wait between job control database reads for JobControlJob jobs. (This delay is for an individual job, not for any job database read.) Default is None, which causes the module level constant UPDATE_DELAY to be used.

• smart_dist_resources – Custom resources that the localhost provides for smart distribution. Jobs that require a custom resource that is not included in this list will not be run on the localhost even if smart distribution is turned on. This information is unused if smart distribution is off. Each item of the list is a string that defines a custom resource.

Raises

HostTypeError – if the host argument or one of its items is of the wrong type

basedir()

entryByName(name)

completedJobids()

The jobids of all jobs that have completed (with success or failure)

property active_jobs: List[schrodinger.job.queue.BaseJob]

addJob(job: Union[schrodinger.job.queue.BaseJob, List], add_connected: bool = True, **kwargs)

Add a job to run. If job is not a BaseJob instance, a BaseJob instance is constructed with job as the first argument. The default BaseJob class for the JobDJ instance can be specified in the constructor for JobDJ.

Additional keyword arguments are passed on to the job constructor.

All job prerequisites and dependencies need to be specified before adding a job to JobDJ.

Parameters

add_connected – If True, for jobs with dependencies only one job per connected group should be added and all connected jobs will be discovered and added automatically. If False, it is the user’s responsibility to make sure that any prerequisites of a job are also added.

property all_jobs: List[schrodinger.job.queue.BaseJob]

static createResourceHost(host_info: Tuple[str, int], resource: Optional[str] = None) → schrodinger.job.queue.ResourceHost

Given a tuple of (hostname, procs), return a ResourceHost.

Parameters

• host_info (tuple) – The hostname and number of processors it has

• resource (str) – The resource the host provides, if any. If not given, the host will only run jobs that have no resource_requirement.

Return type

ResourceHost

Returns

The ResourceHost created from in the input data

Raises

HostTypeError – If the provided host_info has the wrong format

disableSmartDistribution()

Disable smart distribution of jobs.

Smart distribution allows subjobs to run on the machine that JobDJ is running on when JobDJ itself is running under a queuing system. This is usually desirable since the JobDJ process doesn’t generally consume significant computational resources and you don’t want to leave a queue slot mostly idle.

property done_jobs: List[schrodinger.job.queue.BaseJob]

Successfully completed jobs, sorted into the order they were marked as completed by JobDJ.

dump(filename: pathlib.Path)

Pickle the JobDJ instance to the specified file name.

property failed_jobs: List[schrodinger.job.queue.BaseJob]

getActiveProcCounts() → Dict[str, int]

Return a dictionary containing the number of active jobs on each host.

Any resource requirement of the active jobs is ignored.

getActiveProcsOnHost(hostname: str, requirement: Optional[Union[str, schrodinger.job.resource.ComputeType]]) → int

Get the current number of processors on the given host that are currently occupied by jobs that request the given requirement

Parameters

• hostname – The host to check

• requirement – The job requirement to check

Returns

The number of currently used processors on hostname that provide requirement

getFirstHost()

Get first host from the lists of hosts.

Return type

str

Returns

First host

hasStarted() → bool

Returns True if JobDJ has started already

isComplete() → bool

Returns True if JobDJ has completed, False otherwise.

killJobs()

Kill all active jobs

markForRestart(job: schrodinger.job.queue.BaseJob, action: str)

Mark a job as dead, but make sure that it gets restarted.

Parameters

action – Describes the reason the job is being restarted.

printStatus(job: Optional[schrodinger.job.queue.BaseJob] = None, action: Optional[str] = None)

Prints the status of JobDJ and the action/status for the job.

If no job is specified, prints the status header.

If no action is specified, the status_string attribute of the job is used.

run(*, status_change_callback: Optional[Callable[[schrodinger.job.queue.BaseJob], None]] = None, periodic_callback: Optional[Callable[[schrodinger.job.queue.BaseJob], None]] = None, callback_interval: int = 300, restart_failed: bool = True)

Call this method to run all jobs that have been added. The method will return control when all jobs have completed.

Parameters

• status_change_callback – A function to call every time a job status changes. For example, JobState.RUNNING->JobState.DONE. This function takes a single argument of a schrodinger.job.queue.BaseJob object.

• periodic_callback – A command to call periodically, regardless of whether job status has changed or not. The function will be called without any arguments.

• callback_interval – The interval at which the periodic interval will be called. This time is only approximately enforced and will depend on the timing delay settings (e.g. MONITOR_DELAY).

• restart_failed – True (default) if previously failed jobs should be restarted, False if not.

setHostList(host_list: List[Union[Tuple[str, int], schrodinger.job.queue.ResourceHost]])

Define compute hosts to run subjobs on.

Active jobs are not affected by a change in the host list.

Parameters

host_list – A list of hosts to run on. See the documentation for the hosts parameter to the __init__ method of this class for additional information.

Raises

HostTypeError – If one of the host_list items is not of the correct type

setSmartDistribution(state: bool)

Set smart distribution of jobs.

Parameters

state (bool) – Whether to enable smart distribution

property total_active: int

The number of jobs currently running.

property total_added: int

The number of individual jobs that have been added to the JobDJ instance.

property total_failed: int

The number of jobs that have failed.

property total_finished: int

The number of jobs that have finished successfully.

property waiting_jobs: List[schrodinger.job.queue.BaseJob]

Jobs waiting to be started.

class schrodinger.utils.installation_check.TestJob(command_dir=None, duration=10, timeout=None)

Bases: schrodinger.job.queue.JobControlJob

A flavor of JobControlJob to be used with TestJobDJ. Its command line will be determined when run() is called on it, by setup(); the job will get named after the host entry that it gets to run on.

__init__(command_dir=None, duration=10, timeout=None)

Parameters

• command_dir (str) – The launch directory of the job

• duration (int) – The duration of the (testapp) job

• timeout (int) – Timeout (in seconds) after which the job will be killed. If None, the job is allowed to run indefinitely.

doCommand(host, *args, **kwargs)

Launch job on specified host using jobcontrol.launch_job().

Parameters

• host – Host on which the job will be executed.

• local – Removed in JOB_SERVER.

run(host, *args, **kwargs)

Run the job.

The steps taken are as follows:

1. Execute the preCommand method for things like changing the working directory.

2. Call the doCommand to do the actual work of computation or job launching.

3. Call the postCommand method to undo the changes from the preCommand that need to be undone.

formCommand(entry)

Generate a command line appropriate for a given host entry

acquireLicenseForSmartDistribution() → bool

Acquire and hold licenses for a smart distribution job. This makes sure the job won’t fail due to unavailable licenses.

Returns True if the licenses registered for the job are acquired, and False if they are not. If no licenses are registered, it always returns True to avoid preventing jobs from using the smart distribution feature. For legacy jobcontrol, the license check is not performed, and is always returned True. We want to use this feature as a pitch to move users to JOB_SERVER.

addFinalizer(function: Callable[[schrodinger.job.queue.BaseJob], None], run_dir: Optional[str] = None)

Add a function to be invoked when the job completes successfully.

See also the add_multi_job_finalizer function.

addGroupPrereq(job: schrodinger.job.queue.BaseJob)

Make all jobs connected to job prerequisites of all jobs connected to this Job.

addLaunchEnv(key: str, val: str)

Adds the given environment key and and value to the list of launch environment.

Parameters

• key – environment key to add to the launch environment.

• val – environment value associcated with the key to add to the launch environment.

addPrereq(job: schrodinger.job.queue.BaseJob)

Add a job that is an immediate prerequisite for this one.

cancel()

Send kill request to jobcontrol managed job. This method will eventually deprecate JobControlJob.kill

cancelSubmitted(do_license_check: bool = False) → schrodinger.job.queue.CancelSubmittedStatus

If the job is still in the ‘submitted’ state, cancel it, purge the jobrecord and set the job handle to None. This tries to acquire licenses for the job before canceling from the queue if do_license_check is turned on.

Parameters

do_license_check – Acquire licenses for the job before canceling from the queue.

Returns one of the status of CancelSubmittedStatus.

finalize()

Clean up after a job successfully runs.

genAllJobs(seen: Optional[Set[schrodinger.job.queue.BaseJob]] = None) → Generator[schrodinger.job.queue.BaseJob, None, None]

A generator that yields all jobs connected to this one.

genAllPrereqs(seen=None) → Generator[schrodinger.job.queue.BaseJob, None, None]

A generator that yields all jobs that are prerequisites on this one.

getCommand() → List[str]

Return the command used to run this job.

getCommandDir() → str

Return the launch/command directory name. If None is returned, the job will be launched in the current directory.

getDuration() → Optional[int]

Return the duration of the Job as recorded by job server. The duration does not include queue wait time.

If the job is running or has not launched, returns None.

Note that this method makes a blocking call to the job server.

getJob() → Optional[schrodinger.job.jobcontrol.Job]

Return the job record as a schrodinger.job.jobcontrol.Job instance.

Returns None if the job hasn’t been launched.

getJobDJ() → schrodinger.job.queue.JobDJ

Return the JobDJ instance that this job has been added to.

getPrereqs()

Return a set of all immediate prerequisites for this job.

getStatusStrings() → Tuple[str, str, str]

Return a tuple of status strings for printing by JobDJ.

The strings returned are (status, jobid, host).

hasExited() → bool

Returns True if this job finished, successfully or not.

hasStarted() → bool

Returns True if this job has started (not waiting)

init_count = 0

isComplete() → bool

Returns True if this job finished successfully

kill()

Send kill request to jobcontrol managed job

maxFailuresReached(msg: str)

Print an error summary, including the last 20 lines from each log file in the LogFiles list of the job record.

postCommand()

A method to restore things to the pre-command state.

preCommand()

A method to make pre-command changes, like cd’ing to the correct directory to run the command in.

retryFailure(max_retries: int = 0) → bool

This method will be called when the job has failed, and JobDJ needs to know whether the job should be retried or not.

JobDJ’s value for the max_retries parameter is passed in, to be used when the job doesn’t have its own max_retries value.

Return True if this job should be retried, otherwise False.

runsLocally() → bool

Return True if the job runs on the JobDJ control host, False if not. Jobs that run locally don’t need hosts.

There is no limit on the number of locally run jobs.

setup()

A method to do initial setup; executed after preCommand, just before doCommand.

property state: schrodinger.job.queue.JobState

Return the current state of the job.

Note that this method can be overridden by subclasses that wish to provide for restartability at a higher level than unpickling BaseJob instances. For example, by examining some external condition (e.g. presence of output files) the state JobState.DONE could be returned immediately and the job would not run.

update()

Checks for changes in job status, and updates the object appropriately (marks for restart, etc).

Raises

RuntimeError – if an unknown Job Status or ExitStatus is encountered.

usesJobServer() → bool

Detect, by looking at the jobId, whether this job uses a job server.

schrodinger.utils.installation_check.find_user_libstdcpp()

Returns path to system libstdc++.so.6 (or default libstdc++.so.6 before SCHRODINGER environment was found.

schrodinger.utils.installation_check.get_libstdcpp_version(library)

Returns GLIBCXX minor version from a given library.

Parameters

library (str) – pathname to a libstdc++.so.6

schrodinger.utils.installation_check.get_bundled_libstdcpp()