schrodinger.seam.options.pipeline_options module

class schrodinger.seam.options.pipeline_options.SeamOptions(flags: Optional[List[str]] = None, **kwargs: Any)

Bases: apache_beam.options.pipeline_options.PipelineOptions

validate(validator: apache_beam.options.pipeline_options_validator.PipelineOptionsValidator) → List[str]

getWorkerTimeoutInSeconds() → int

__init__(flags: Optional[List[str]] = None, **kwargs: Any) → None

Initialize an options class.

The initializer will traverse all subclasses, add all their argparse arguments and then parse the command line specified by flags or by default the one obtained from sys.argv.

The subclasses of PipelineOptions do not need to redefine __init__.

Args:

flags: An iterable of command line arguments to be used. If not specified

then sys.argv will be used as input for parsing arguments.

**kwargs: Add overrides for arguments passed in flags. For overrides

of arguments, please pass the option names instead of flag names. Option names: These are defined as dest in the parser.add_argument() for each flag. Passing flags like {no_use_public_ips: True}, for which the dest is defined to a different flag name in the parser, would be discarded. Instead, pass the dest of the flag (dest of no_use_public_ips is use_public_ips).

display_data()

Returns the display data associated to a pipeline component.

It should be reimplemented in pipeline components that wish to have static display data.

Returns:

Dict[str, Any]: A dictionary containing key:value pairs. The value might be an integer, float or string value; a DisplayDataItem for values that have more data (e.g. short value, label, url); or a HasDisplayData instance that has more display data that should be picked up. For example:

{
  'key1': 'string_value',
  'key2': 1234,
  'key3': 3.14159265,
  'key4': DisplayDataItem('apache.org', url='http://apache.org'),
  'key5': subComponent
}

classmethod from_dictionary(options)

Returns a PipelineOptions from a dictionary of arguments.

Args:

options: Dictionary of argument value pairs.

Returns:

A PipelineOptions object representing the given arguments.

get_all_options(drop_default=False, add_extra_args_fn: Optional[Callable[[apache_beam.options.pipeline_options._BeamArgumentParser], None]] = None, retain_unknown_options=False) → Dict[str, Any]

Returns a dictionary of all defined arguments.

Returns a dictionary of all defined arguments (arguments that are defined in any subclass of PipelineOptions) into a dictionary.

Args:

drop_default: If set to true, options that are equal to their default

values, are not returned as part of the result dictionary.

add_extra_args_fn: Callback to populate additional arguments, can be used

by runner to supply otherwise unknown args.

retain_unknown_options: If set to true, options not recognized by any

known pipeline options class will still be included in the result. If set to false, they will be discarded.

Returns:

Dictionary of all args and values.

view_as(cls: Type[apache_beam.options.pipeline_options.PipelineOptionsT]) → apache_beam.options.pipeline_options.PipelineOptionsT

Returns a view of current object as provided PipelineOption subclass.

Example Usage:

options = PipelineOptions(['--runner', 'Direct', '--streaming'])
standard_options = options.view_as(StandardOptions)
if standard_options.streaming:
  # ... start a streaming job ...

Note that options objects may have multiple views, and modifications of values in any view-object will apply to current object and other view-objects.

Args:

cls: PipelineOptions class or any of its subclasses.

Returns:

An instance of cls that is initialized using options contained in current object.