schrodinger.active_learning.al_report module

schrodinger.active_learning.al_report.get_ligand_ml_metric(ligand_ml_model_file: str) → Tuple

Extract the test set metrics, test set labels and predictions from ligand_ml model file.

Parameters

ligand_ml_model_file – ligand_ml .qzip model file.

Returns

r2, mae, rmse, labels and prediction of the test set

schrodinger.active_learning.al_report.make_train_report(ligand_ml_model_file: str, report_path: str, iter_num: int)

Generate a pdf file that records the test set metrics of the ligand_ml model.

Parameters

  • ligand_ml_model_file – ligand_ml .qzip model file.

  • report_path – path of the pdf report

  • iter_num – current iteration number

schrodinger.active_learning.al_report.make_score_hist_plot(name_to_scores: Dict, x_label: str, plot_path: str)

Generate a bar-type histogram plot which contains multiple data. The names of the data are used as the legends.

Parameters

  • name_to_scores – a dictionary where keys are used as the legends and values are the numbers to be plotted.

  • x_label – label of x-axis

  • plot_path – path of the saved plot

schrodinger.active_learning.al_report.perform_ttest(scores_by_node: Dict, alpha: float = 0.05, equal_var=False) → Dict

Perform unpaired t-test on two subsequent iterations of scores. If we accept the null hypothesis, the two iterations have statistically identical averages. If we reject the null hypothesis, the averages of the two iterations are statistically different. We assume sets of scores have unequal variances and uneven samples sizes.

Parameters

  • scores_by_node – map of node name to scores

  • alpha – significance level

  • equal_var – if True, perform a standard independent 2 sample test that assumes equal population variances and sample sizes

Returns

node name to t-test results

schrodinger.active_learning.al_report.calculate_kstest_results(scores_by_node: Dict, target: str) → Dict

Calculate one-sided KS test for scores between two iterations

Parameters

  • scores_by_node – map of node name to scores

  • target – optimization target

Returns

node name to KS test results

schrodinger.active_learning.al_report.perform_ks_test(reference_scores: List, scores: List, target: str, alpha=0.05) → Tuple

Perform one-sided KS (Kolmogorov-Smirnov) test on two subsequent iterations of scores.

In a one-sided two-sample KS test, we compare the cumulative distribution functions of two sets of scores to determine if the previous iteration’s CDF distribution is stochastically greater or less than the next iteration in a specific direction. For more details see https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.kstest.html

Interpreting the outcome of the KS test: dG FEP or Glide scores: ‘reject’ means that scores from the next iteration are statistically favored to be lower given a p-value less than the significance level alpha.

All other targets: ‘reject’ means that scores from the next iteration are statistically favored to be higher given a p-value less than the significance level alpha.

Parameters

  • reference_scores – list of scores from an iteration to use as reference

  • scores – list of scores from the iteration that follows reference_scores

  • target – optimization target

  • alpha – significance level

Returns

KS statistic, p-value, significance level, outcome of the test

schrodinger.active_learning.al_report.get_image(path: str, width: float = 72.0) → reportlab.platypus.flowables.Image

Convert image file to reportlab image object that has the same aspect ratio and specified width.

Parameters

  • path – path of the image file.

  • width (float) – width of the reportlab image.

Returns

reportlab image

schrodinger.active_learning.al_report.get_report_maker(active_learning_job)

Get corresponding report maker for the active learning job. It returns None for evaluate task since we do not have report for it yet.

Parameters

active_learning_job (ActiveLearningJob) – active learning job to be processed.

Returns

corresponding report maker

Return type

ALPilotReportMaker

schrodinger.active_learning.al_report.get_time_cost(nodes: Dict, node_name: str) → str

Return the time cost of a node. It returns ‘Unavailable’ if the time cost is not available.

Parameters

  • nodes – dict that maps node name to node object.

  • node_name – name of the active learning node of interest.

Returns

time cost in h/m/s format.

schrodinger.active_learning.al_report.get_score_pred_as_array(title_to_score: Dict, pred_score_file: str, discard_cutoff: float, ascending: bool = True) → numpy.ndarray

Return the score, predicted score, prediction uncertainty of the ligands as the N X 3 numpy array.

Parameters

  • title_to_score – dict that maps ligand title to score.

  • pred_score_file – path of the ligand ml prediction .csv file.

  • discard_cutoff – score cutoff for excluding the ligands in ML training set.

  • ascending – lower value means better ligand if ascending is True

Returns

numpy array of (num_of_ligands X (score, pred, uncertain))

schrodinger.active_learning.al_report.calculate_recovery_ratio(label_pred: numpy.ndarray, top_ratio: float) → numpy.ndarray

Calculate the recovery ratio of the best ligands based on label in different numbers of the top ligands predicted by ligand_ml. More negative value means better ligand.

Parameters

  • label_pred – array of dim number of ligands X 2 contains the (label, prediction)

  • top_ratio – top ratio of the ligands by label.

Returns

(screen ratio, recovery ratio of top ligands defined by top_ratio) of all the ligands.

schrodinger.active_learning.al_report.plot_correlation(x: List, y: List, fname: str, x_label: str, y_label: str, title: str) → None

Generate a simple correlation scatter plot :param x_label: label of x-axis :param y_label: label of y-axis :param fname: path of the saved plot :param x: list of x values :param y: list of y values :param title: title of the plot

schrodinger.active_learning.al_report.plot_regression(y_true: numpy.ndarray, y_pred: numpy.ndarray, fname: str)

Generate regression plot. This function is sightly modified from ligand_ml/plotting.py to change the labels of axis.

Parameters

  • y_true – 1D array test set label.

  • y_pred – 2D array of ligand_ml prediction and uncertainty

  • fname – filename to save the image

schrodinger.active_learning.al_report.plot_recovery(recovery_results: Dict, fname: str)

Generate and save recovery plot image.

Parameters

  • recovery_results – dict that maps top ratio to the recovery ratio numpy array.

  • fname – path of the saved image.

schrodinger.active_learning.al_report.make_regress_recovery_plots(y_true: numpy.ndarray, y_pred_uncertain: numpy.ndarray, top_ratio_samples: List, regress_text: str, recovery_text: str)

Generate regression plot and recovery plot and include both in a table. Also return the recovery results for the sampled top ratios as a dict.

schrodinger.active_learning.al_report.make_recovery_table(recovery_results: Dict, screen_ratio_samples: List) → List

Generate a list of list that contains the recovery ratio for certain top ratio and screen ratio.

Parameters

  • recovery_results – dict that maps top ratio to the recovery ratio numpy array.

  • screen_ratio_samples – list of screen ratios

Returns

table as a list of list, table caption, largest enrichment in the table.

schrodinger.active_learning.al_report.get_conclusion_string(best_enrichment: float, job_type: str, high_enrich: int = 10, low_enrich: int = 2) → str

Return the conclusion string based on the job type and the higheest enrichment we have in the recovery ratio table.

class schrodinger.active_learning.al_report.ALReportMaker(active_learning_job)

Bases: object

Base class for different types of AL report maker.

__init__(active_learning_job)

Initialize the report maker for an active learning job

initReport(header: str)

Initialize the report and add header information

class schrodinger.active_learning.al_report.ALPilotReportMaker(active_learning_job)

Bases: schrodinger.active_learning.al_report.ALReportMaker

__init__(active_learning_job)

Initialize the report maker for an active learning job

report()

Function for building the report

addRunDetail()

Add job specifications and running time cost information to the report

addRecoveryResults()

Add the regression plot, recovery plot, recovery table and conclusion to the report.

class schrodinger.active_learning.al_report.ALScreenReportMaker(active_learning_job)

Bases: schrodinger.active_learning.al_report.ALReportMaker

__init__(active_learning_job)

Initialize the report maker for an active learning job

report()

Function for building the report

addRunDetail()

Add job specifications and running time cost information to the report

addRecoveryResults()

Add the regression plot, recovery plot, recovery table and conclusion to the report.