schrodinger.tasks.queue module

class schrodinger.tasks.queue.QueueReport(*args, _param_type=<object object>, **kwargs)

Bases: schrodinger.models.parameters.CompoundParam

num_tasks: int

A parameter of the class.

num_done_tasks: int

A parameter of the class.

num_failed_tasks: int

A parameter of the class.

failure_infos: Dict[str, schrodinger.tasks.tasks.FailureInfo]

A parameter of the class.

failure_infosChanged

A pyqtSignal emitted by instances of the class.

failure_infosReplaced

A pyqtSignal emitted by instances of the class.

num_done_tasksChanged

A pyqtSignal emitted by instances of the class.

num_done_tasksReplaced

A pyqtSignal emitted by instances of the class.

num_failed_tasksChanged

A pyqtSignal emitted by instances of the class.

num_failed_tasksReplaced

A pyqtSignal emitted by instances of the class.

num_tasksChanged

A pyqtSignal emitted by instances of the class.

num_tasksReplaced

A pyqtSignal emitted by instances of the class.

class schrodinger.tasks.queue.QueueResults(*args, _param_type=<object object>, **kwargs)

Bases: schrodinger.models.parameters.CompoundParam

Data class to store the results of a queued task run and report them if requested.

We implement several dunder methods to make this class behave like a list of done tasks in order to maintain backwards compatibility with existing code.

done_tasks: List[schrodinger.tasks.tasks.AbstractTask]

A parameter of the class.

failed_tasks: List[schrodinger.tasks.tasks.AbstractTask]

A parameter of the class.

__len__() → int
addTasks(tasks: List[schrodinger.tasks.tasks.AbstractTask]) → None
makeReport() → schrodinger.tasks.queue.QueueReport
done_tasksChanged

A pyqtSignal emitted by instances of the class.

done_tasksReplaced

A pyqtSignal emitted by instances of the class.

failed_tasksChanged

A pyqtSignal emitted by instances of the class.

failed_tasksReplaced

A pyqtSignal emitted by instances of the class.

class schrodinger.tasks.queue.TaskQueue(*args, _param_type=<object object>, **kwargs)

Bases: schrodinger.tasks.tasks.SignalTask

A task that runs a queue of tasks. The TaskQueue is done when all its added tasks have completed, regardless of whether they completed successfully or failed. To use, add tasks with addTask and then start the task queue.

queuedTaskFinished

A pyqtSignal emitted by instances of the class.

queueDone

A pyqtSignal emitted by instances of the class.

max_running_tasks: int

A parameter of the class.

__init__(*args, **kwargs)
addTask(task)
getTasks()

Return all tasks in the queue.

Return type

tuple[tasks.AbstractTask]

setUpMain()
start(*args, **kwargs)

This is the main method for starting a task. Start will check if a task is not already running, run preprocessing, and then run the task.

Failures in preprocessing will interrupt the task start, and the task will never enter the RUNNING state.

Parameters

skip_preprocessing (bool) – whether to skip preprocessing. This can be useful if preprocessing was already performed prior to calling start.

calling_contextChanged

A pyqtSignal emitted by instances of the class.

calling_contextReplaced

A pyqtSignal emitted by instances of the class.

failure_infoChanged

A pyqtSignal emitted by instances of the class.

failure_infoReplaced

A pyqtSignal emitted by instances of the class.

inputChanged

A pyqtSignal emitted by instances of the class.

inputReplaced

A pyqtSignal emitted by instances of the class.

max_progressChanged

A pyqtSignal emitted by instances of the class.

max_progressReplaced

A pyqtSignal emitted by instances of the class.

max_running_tasksChanged

A pyqtSignal emitted by instances of the class.

max_running_tasksReplaced

A pyqtSignal emitted by instances of the class.

nameChanged

A pyqtSignal emitted by instances of the class.

nameReplaced

A pyqtSignal emitted by instances of the class.

outputChanged

A pyqtSignal emitted by instances of the class.

outputReplaced

A pyqtSignal emitted by instances of the class.

progressChanged

A pyqtSignal emitted by instances of the class.

progressReplaced

A pyqtSignal emitted by instances of the class.

progress_stringChanged

A pyqtSignal emitted by instances of the class.

progress_stringReplaced

A pyqtSignal emitted by instances of the class.

statusChanged

A pyqtSignal emitted by instances of the class.

statusReplaced

A pyqtSignal emitted by instances of the class.

class schrodinger.tasks.queue.TaskDJ(*args, task_timeout=None, **kwargs)

Bases: schrodinger.job.queue.JobDJ

WARNING: This class is slated for removal. To run tasks in parallel, consider using the run_tasks_in_parallel below, or else use JobDJ directly by using task.runToCmd and task.setJob

A subclass of JobDJ that supports running tasks.

NOTE: In order for remote jobs to be distributed properly, ensure that task.job_config.host_settings.host is set to None, otherwise TaskDJ will not add a -HOST option.

__init__(*args, task_timeout=None, **kwargs)

Constructor.

Parameters

  • hosts – A list of hosts to run on. Items of the list can be a (<hostname>, <maximum_concurrent_subjobs>) tuple that gives a host name and the maximum number of processors that can be used on that host. Or, items of the list can be a ResourceHost object that provides a detailed listing of the number of processors on each host that can be used for jobs requesting specific resources. The list can mix tuple and ResourceHost items. If a host appears more than once (whether as a tuple or ResourceHost), the information in the multiple entries is added together. Processors in the tuple form are only considered usable by jobs that have resource_requirement=None (The default state for jobs). The default value of None for this parameter means the host information is determined automatically from the command line and is usually desired.

  • local – No longer functional in JOB_SERVER.

  • max_retries – Number of allowed retries per subjob. If this is set, it is never overridden by the SCHRODINGER_MAX_RETRIES environment variable. If it is not set, the value in default_max_retries is used, and SCHRODINGER_MAX_RETRIES is allowed to override. If you wish to disable restarting altogether, set this value to zero.

  • default_max_retries – Number of allowed retries per subjob. This value can always be overridden by the SCHRODINGER_MAX_RETRIES environment variable. Default is zero.

  • max_failures – Total number of allowed subjob failures before JobDJ exits. If it is not defined, a default of zero will be used (exit on any failure after attempting to restart), but this can be overridden with the SCHRODINGER_MAX_FAILURES environment variable. To allow an unlimited number of subjob failures, set max_failures to the module level NOLIMIT constant.

  • verbosity – There are three allowed verbosity levels: “quiet” - only warnings and errors are printed; “normal” - JobDJ progress is printed; and “verbose” - additional debugging info is printed. Default is “quiet”.

  • job_class – The class to use as the default job constructor when the addJob argument is not a BaseJob instance.

  • update_delay – The number of seconds to wait between job control database reads for JobControlJob jobs. (This delay is for an individual job, not for any job database read.) Default is None, which causes the module level constant UPDATE_DELAY to be used.

  • smart_dist_resources – Custom resources that the localhost provides for smart distribution. Jobs that require a custom resource that is not included in this list will not be run on the localhost even if smart distribution is turned on. This information is unused if smart distribution is off. Each item of the list is a string that defines a custom resource.

Raises

HostTypeError – if the host argument or one of its items is of the wrong type

addTask(task)
updatedTasks()
class schrodinger.tasks.queue.AutoFileMode

Bases: enum.IntEnum

FAILED_LOG = 1
FAILED_ALL = 2
NONE = 3
LOG = 4
ALL = 5
schrodinger.tasks.queue.run_tasks_in_parallel(task_list, autoname=True, basename=None, basedir=None)

This functions provides a convenient way of launching multiple tasks in parallel while taking care of boilerplate and prevents common mistakes.

By default, this function will:

  • Give each task a unique name

  • Give each task its own directory

  • Create a new JobDJ or TaskQueue

  • Run it with the provided tasks

  • Register the log files of any failed tasks with jobserver if using

    jobtasks

This can be used with either JobTasks or SubprocessTasks. SubprocessTasks will be run on a TaskQueue while JobTasks will be run on JobDJ.

Parameters

  • task_list (list of tasks.AbstractTask) – the tasks to run

  • autoname (bool) – whether to automatically give each task a unique name

  • basename (str) – basename to be used in autonaming each task. Has no effect if autoname is set to False

  • basedir (str) – Base directory; all task taskdirs will be created inside. Defaults to cwd, and relative paths will be taken relative to cwd.

Returns

the queue results, which can be used to generate a queue report.

Return type

QueueResults

schrodinger.tasks.queue.run_tasks_on_queue(task_list, autoname=True, basename=None, basedir=None, max_running_tasks=None)

Launch multiple SubprocessTasks on a TaskQueue.

See run_tasks_in_parallel for more details. :param max_running_tasks: specifies how many tasks can be run simultaneously :type max_running_tasks: int

schrodinger.tasks.queue.run_tasks_on_dj(task_list, dj=None, autoname=True, basename=None, basedir=None, auto_file_mode=AutoFileMode.FAILED_LOG, dj_init_kwargs: dict = None, dj_addJob_kwargs: dict = None, dj_run_kwargs: dict = None)

Launch multiple jobtasks on a jobdj. Use this function over run_tasks_in_parallel if you know you’ll be running jobtasks and you want to further customize how they’re run.

Parameters

  • dj (schrodinger.job.queue.JobDJ) – A JobDJ to run the tasks on. This is optional and useful if want a jobdj with specific settings.

  • autoname (bool) – whether to automatically give each task a unique name

  • basename (str) – basename to be used in autonaming each task. Has no effect if autoname is set to False

  • basedir (str) – Base directory; all task taskdirs will be created inside. Defaults to cwd, and relative paths will be taken relative to cwd.

  • auto_file_mode (AutoFileMode) – what subtask files to register with the parent job to be copied back by jobcontrol. Only has an effect if this function is called from inside a jobcontrol backend.

  • dj_init_kwargs – A dictionary of keyword arguments to pass to JobDJ.__init__

  • dj_run_kwargs – A dictionary of keyword arguments to pass to JobDJ.run

  • dj_addJob_kwargs – A dictionary of keyword arguments to pass to JobDJ.addJob for each jobtask.

schrodinger.tasks.queue.autoname_tasks(task_list, basename=None)

Sets unique names on all the tasks provided with an incrementing integer suffix. See run_tasks_on_dj for details.

schrodinger.tasks.queue.configure_taskdirs(task_list, autoname=True, basename=None, basedir=None)

Automatically configure task names and taskdirs to commonly used values.

Parameters

  • task_list (list of tasks.AbstractTask) – the tasks to run

  • autoname (bool) – whether to automatically give each task a unique name

  • basename (str) – basename to be used in autonaming each task. Has no effect if autoname is set to False

  • basedir (str) – Base directory; all task taskdirs will be created inside. Defaults to cwd, and relative paths will be taken relative to cwd.

schrodinger.tasks.queue.auto_register_files(task_list, auto_file_mode=AutoFileMode.FAILED_LOG)

Automatically registers files from the subtask with the parent job (if applicable).