schrodinger.job.launcher module

A class for launching individual executables under job control.

Launcher is a Python wrapper for using the external ‘jlaunch’ script. It simplifies the creation of the “startup script” for a script running under job control.

NOTE: Launcher is designed for use within startup scripts only. If there is a script/executable that needs to be submitted under job control by another script, please file a case to have a startup script written for it; then use jobcontrol.launch_job() to launch it.

Copyright Schrodinger, LLC. All rights reserved.

class schrodinger.job.launcher.Launcher(script, jobname=None, copyscript=False, usehostargs=None, runtoplevel=False, host=None, local=False, user=None, disp=None, proj=None, save=None, prog=None, nice=None, debugger=None, tmpdir=None, wait=False, no_redirect=False, localdriver=False, nproc=None, interval=None, loginterval=None, expandvars=None, viewname=None, oplsdir=None, product=None, stoppable=False)

Bases: object

Run a program (typically a Python or shell script) under job control.

The parameters describing the job are specified as attributes of the launcher object.

The launch() method translates the job parameters into commandline options for the jlaunch script, and executes that script. If the job is launched successfully, a job object is returned.

The getCommandArgs() method returns the jlaunch command without executing the job.

NOTE: Launcher is designed for use within startup scripts only.

__init__(script, jobname=None, copyscript=False, usehostargs=None, runtoplevel=False, host=None, local=False, user=None, disp=None, proj=None, save=None, prog=None, nice=None, debugger=None, tmpdir=None, wait=False, no_redirect=False, localdriver=False, nproc=None, interval=None, loginterval=None, expandvars=None, viewname=None, oplsdir=None, product=None, stoppable=False)
Parameters

  • script –

    The remote path to the script or program to run (the only required argument). Used by job control to locate the script on the remote host (which will be same as the launch host if host=”localhost” or no_redirect=True).

    Example: “$REMOTE_MMSHARE_EXEC/../../python/scripts/prepwizard_driver.py”

    If the script is located in a $SCHRODINGER/run path, then simply the name of the script can be passed in (e.g. “prepwizard_driver.py”), as $SCHRODINGER/run of the remote host will automatically find the script.

    If copyscript is True, must be set to the full local path of the script to be copied.

  • copyscript – Whether or not to copy the script or program to the job directory, to make it accessible on the compute host (default: False). Useful for allowing non-distributed scripts (e.g. from ScriptCenter) to run on remote hosts. Keep set to False when running scripts included in the Schrodinger distribution, including toplevel scripts, scripts under mmshare, or any other distributed repository. Avoid the hack of setting <script> to the local path of the script and enabling <copyscript> - instead set <script> to the remote path of the script - ask fellow developers for help if needed.

  • usehostargs – Deprecated and has no effect.

  • runtoplevel – Use $SCHRODINGER/run to reset the environment before submitting the script under job control. This is useful if the script that uses Launcher is not invoked with options (e.g., -HOST) appropriate for running the job, such as a script that both starts a GUI and runs the job. (default: False)

  • host – Run jobs on the specified host. By default, host specified on the original command-line will be used (JOBHOST env var is read, which is set by the original top-level script).

  • local – Run the job in the directory from which it was launched (deprecated)

  • user – Run the job as the specified user (deprecated).

  • proj – The absolute pathname for the Maestro project to which this job should be assigned. If None, jlaunch will use the project specified on the command line, or by the SCHRODINGER_PROJECT env var. (default: None)

  • disp – The “disposition” for this job’s outputs; supported settings are “NONE”, “NEW”, “APPEND”, or None. If None, jlaunch will use the disposition specified on the command line or by the SCHRODINGER_JOB_DISPOSITION env var. THis parameter is only meaningful if the job has been assigned to a project. (default: None)

  • save – Save the temporary job directory after the job completes (default: None). By default, will use what was specified on the command line (toplevel script will parse the -SAVE option and set the SCHRODINGER_SAVE_JOBDIR env var). Set to True or False to force saving or not saving the job directory.

  • prog – The name of the program being executed, for example, ‘Glide’. This is recorded in the Program field of the job record. (default: None)

  • nice – Run the program at reduced priority (default: False). By default, use nice if specified on the command line (toplevel script parses the -NICE option and sets the SCHRODINGER_NICE env var.)

  • debugger – Run the job locally under the specified debugger. (default: None)

  • tmpdir – The absolute pathname for the directory under which the temporary job directory should be created. (default: None) If set to None, then the usual procedures are followed for getting the tmpdir, that is, the ‘tmpdir:’ setting in the hosts file, the SCHRODINGER_TMPDIR env var, and the -TMPDIR commandline option.

  • wait – Do not return from launch() until the job completes. (default: False)

  • no_redirect – Do not run under job control (default: False). Set to True if the user specified the -NOJOBID option.

  • localdriver – Submit the main job (driver) to localhost, and use the -HOST command line argument only for any subjobs started by it. This overrides any -DRIVERHOST option on the command line. This is not compatible with ‘runtoplevel’ mode. (default: False)

  • nproc – The number of processors required for a parallel job. When a job is submitted to a queueing system, this many processors will be requested; it has no effect on simple local and remote jobs. For a serial job, this should be set to None. (default: None)

  • interval – The monitoring interval, in seconds. This controls how often monitor files are copied back during the job. If monitor files are specified, then nothing is copied back. If set to zero or None, monitoring is turned off altogether. Monitoring can be very resource-intensive, and so it should only be turned on at user request, through the Maestro monitor panel. This parameter should generally always be left at None. (default: None)

  • loginterval – The log interval, in seconds. This controls how frequently logfile changes are copied back during the job. If set to None, the standard log interval is used; if set to zero, the log file won’t be copied back until the backend finishes. Generally, you should let this be None. (default: None)

  • expandvars – Whether to expand environment variable references in the command that’s executed to launch the job. If set to None, defer to jobcontrol.launch_job(), which has its own default behavior for this. (default: None)

  • oplsdir – Path to the custom opls parameter directory for the -OPLSDIR flag (default: None)

  • product (str) – Product to pass in to $SCHRODINGER/run - determines what product directory is searched on the remote host. By default it’s mmshare.

  • stoppable (bool) – If True, stopping a job is handled by sending a message to the backend.

Warning

Monitoring should only be turned on when requested by the user, because it imposes a typically slows down the backend and consumes significant system resources.

addScriptArgs(args)

Specify arguments to add to the command line for the backend when it is executed on the compute node.

Parameters

args – A list of arguments.

addInputFile(filename)

Register an input file with job control. This file will be copied to the job directory before the backend is executed, unless the file is already accessible at the registered pathname. Don’t use this method to register the script or program being executed. Use the copyscript option for that.

Parameters

filename – The pathname of the input file.

addTempInputFile(filename, name='')

Register a temporary input file with job control.

These temporary files are stored in the JobDB with the name ‘<jobid>.TIN<name>.<ext>’, where the <name> is a tag specified via the ‘name’ arg. The ‘name’ can be omitted for one input file at most.

Parameters

  • filename – The name of the temporary input file.

  • name – The tag for this temporary input file.

Raises

Exception – if either the ‘name’ or ‘filename’ has already been added.

addForceInputFile(source_filename, dest_filename=None)

Register an input file with job control. Job control will copy this file to the job directory regardless of whether it’s already accessible on the compute node.

Parameters

  • source_filename – The pathname for the input file on launch node.

  • dest_filename – The name given to the copied file. By default, this will be the base name of source_filename.

addOutputFile(filename)

Register an output file with job control so it gets copied back to the launch directory. If the file is not found in the job directory after the job completes, this will be silently ignored.

Parameters

filename – The name of the output file.

addRequiredOutputFile(filename)

Register an output file with job control so it gets copied back to the launch directory. If the file does not exist at the end of the job, then job control will mark the job as “died”.

Parameters

filename – The name of the output file.

setStructureOutputFile(filename)

Register a structure output file with job control (i.e., the file that gets incorporated into Maestro upon job completion). This method also registers the file as an output file.

Parameters

filename – The name of the structure output file.

setStructureMonitorFile(filename)

Register a structure monitor file with job control. The name of the structure monitor file. If this job is monitored from Maestro, then this file will be used to get the structure to display.

Parameters

filename – The name of the structure monitor file.

setStdOutErrFile(filename)

Specify the name of the main log file, which captures the stdout and stderr of the backend. If no filename is specified, then the main log file will be named ‘<jobname>.log’.

Parameters

filename – The name of the main log file.

addLogFile(filename, default=False)

Register a log file with job control. This file will be copied back to the launch directory incrementally as the job runs. By default, the first log file registered will be displayed when the job is being monitored in Maestro. Use the ‘default’ option to specify that some particular file should be displayed.

Parameters

  • filename – The name of a log file.

  • default – Whether to display this log file in the monitor panel when the job is being monitored in Maestro. (default: False)

setNoLaunch(nolaunch)

Specifies whether should exit before actually launching the job. This can be useful for debugging issues in job launching.

Parameters

nolaunch (bool) – If true, exit just before the job would be submitted to the queue, or started on the compute node.

setAppendLogs(appendlogs)

Specifies whether to append to existing log files. Useful for restarting jobs.

Parameters

appendlogs (bool) – If true, append to existing logfiles

addEnv(env)

Specify an environment variable setting that should be added to the backend environment on the compute node.

Parameters

env – An environment variable setting, in the form ‘MYENV=value’.

addLicense(license)

Specify a license that’s required by the backend. If license checking is turned on, then the backend will not be started unless all required licenses are available.

Parameters

license – An license token in the form ‘TOKEN’ or ‘TOKEN:n’, where TOKEN is the name of license and n is the number of tokens.

getCommandArgs()

Returns the job invocation command. If the job is being run ‘no_redirect’, just the backend execution command is returned. Otherwise, the jlaunch invocation command is returned. The jlaunch command uses the new semantics (e.g., ‘-cmd’ instead of ‘-e’). Raises a RuntimeError if the SCHRODINGER and MMSHARE_EXEC env vars aren’t set.

launch(print_jobid=True, launch_dir=None)

Launch the script or program.

If the program is successfully launched under job control, the resulting Job object is returned.

With ‘no_redirect’ runs the backend directly and returns the subprocess exit status. With ‘debugger’ runs the backend using specified debugger and returns None.

Parameters

  • print_jobid – Whether to print the JobId of the newly launched job to the terminal. (default: True)

  • lauch_dir – Launch the job from the specified directory.

Raises

IOError – if any of the input files are missing. OSError: if the executable is missing. RuntimeError: If jlaunch failed to submit the job.

Returns

The job object of the launched job. If no_redirect is set, the subprocess exit status is returned; if debugger is set, None is returned.

schrodinger.job.launcher.launch(script, args=None, input_files=None, force_input_files=None, output_files=None, req_output_files=None, structure_output_file=None, structure_monitor_file=None, log_files=None, envs=None, licenses=None, print_jobid=True, appendlogs=False, nolaunch=None, **kwargs)

Launch a job directly, without creating an explicit Launcher instance. This function supports same arguments as the Launcher class constructor, as well as those listed below. These additional arguments are all optional.

NOTE: Launcher is designed for use within startup scripts only.

Parameters

  • args – A list of string arguments to pass to the backend program. (default: None)

  • input_files – A list of input files that are required for the job to run. These will be copied to the job directory unless they’re already accessible with the given pathnames. (default: None)

  • force_input_files – A list of input files that should be transfered to the job directory, regardless of whether they’re already accessible or not. (default: None)

  • output_files – A list of output files to register with job control. These files will be copied back to the launch directory after the backend completes. Missing files are silently ignored. (default: None)

  • req_output_files – A list of output files to register with job control. These files will be copied back to the launch directory after the backend completes. Missing files mark the job as “died”. (default: None)

  • structure_output_file – The name of the structure output file. If this job’s results are supposed to be incorporated into a Maestro project, then this is the file that will be incorporated. (default: None)

  • structure_monitor_file – The name of the structure monitor file. If this job is monitored from Maestro, then this file will be used to get the structure to display. (default: None)

  • log_files – A list of log files to register with job control. These files will be copied back to the launch directory continually while the job is running. (default: None)

  • envs – A list of environment variable settings to be made on the compute node. Each setting should be a string, in the form “MYENV=value”. (default: None)

  • licenses – A list of required licenses. Each requirement is specified as a string in the form ‘TOKEN’ or ‘TOKEN:n’, where TOKEN is the name of the license and n is the number of tokens checked out. (default: None)

  • print_jobid – Whether to print the JobId of the newly launched job to the terminal. (default: True)

  • appendlogs – If true, appends to log files rather than creating blank ones. (default: False)

Raises

IOError – if any of the input files are missing.

Returns

The job object, if launched successfully under job control, otherwise None.