schrodinger.test.performance.reporter module

Basic client-side components for performance testing. Typical clients should only need to use Test class.

@copyright: (c) Schrodinger, LLC All rights reserved.

class schrodinger.test.performance.reporter.Test(name, product, description=None, scival=False, upload=True, baseClient=None)[source]

Bases: object

A performance test. name and product must uniquely specify a test. product is required to match an existing product name in the database. New tests require descriptions when uploaded. The descriptions of existing tests are not changed by result upload. The baseClient must be an instance of PerformanceTestClient.

Invididual results are added with addResult(). All results are uploaded to the database when report() is called.

Instantiate with scival set to True if you are working with scival performance tests.

Typical pattern:

baseClient = create_performance_test_reporter(...)
test = performance.Test(
    name = "distribution_size",
    product = "shared components",
    description = "Determine the size of the SCHRODINGER distribution and report it to the performance database.",
    baseClient = baseClient,
)
# Result with a metric name and value
test.addResult('file count', 200000)
# Result with a metric name, value, and units
test.addResult('size', 20000, 'MB')
test.report()
__init__(name, product, description=None, scival=False, upload=True, baseClient=None)[source]
addResult(name: str, value: float, units: Optional[str] = None)[source]

Add a result to the current test. Results are not uploaded until report() is called.

Parameters

  • name – Name of the metric being reported

  • value – Current value of the metric

  • units – (optional) units of the value.

report(build_id=None, buildtype=None, mmshare=None, release=None)[source]

Once all results have been added to the test, report them to the database.

schrodinger.test.performance.reporter.validate_types(name, value, units=None)[source]

Validate data types before attempting upload to the server.

schrodinger.test.performance.reporter.create_performance_test_reporter(name: str, product: str, description: str, scival: bool, upload: bool) → schrodinger.test.performance.reporter.Test[source]

Factory method for creating instances of PerformanceTestClient. This is the preferred method, over using the Test directly.

Parameters

  • name – name of the test

  • product – product being tested

  • description – description of the test

  • scival – is the test a scival (scientific validation) test

  • upload – should the results be uploaded to the STU service

schrodinger.test.performance.reporter.get_or_create_test(name, description, product_name, username=None, scival=False)[source]

Get or create a single test from the performance database.

Setting scival to True will add the ‘scival’ tag when creating a new test.

schrodinger.test.performance.reporter.api_url(resource_name, item_id=None, host=None)[source]

Get an address on the core server

schrodinger.test.performance.reporter.performance_api_url(resource_name, item_id=None, host=None)[source]

Get an address in the performance bit of the server.

schrodinger.test.performance.reporter.resource_id(uri)[source]

Get the resource’s ID number from a uri

schrodinger.test.performance.reporter.get_or_create(url, auth, params)[source]

Get or create a resource matching the parameters.

schrodinger.test.performance.reporter.system_information(host)[source]

System information required to report results.

schrodinger.test.performance.reporter.host_information()[source]
schrodinger.test.performance.reporter.post_system(auth: schrodinger.test.stu.client.ApiKeyAuth, baseClient)[source]

Post the current host’s system information to the performance test server.

:return URI for the new system.

schrodinger.test.performance.reporter.install_information(build_id=None, buildtype=None, mmshare=None, release=None)[source]

Execution environment information required to report results.

schrodinger.test.performance.reporter.guess_build_type_and_id(mmshare: int, buildtype=None) → Tuple[str, str][source]

Provide reasonable default values for the buildtype and build_id. When possible, reads from the environment variables SCHRODINGER_BUILDTYPE and SCHRODINGER_BUILD_ID. If SCHRODINGER_BUILDTYPE is not set then we assume the buildtype is NB.

Parameters

  • mmshare (int) – mmshare version number, e.g. 54053

  • buildtype (str) – NB or OB

Returns

Tuple of buildtype and build ID, e.g. ‘NB’, ‘build-075’

Return type

tuple