Experiment#

The module tsl.experiment contains classes and utilities for experiment pipelining, scalability and reproducibility. The main class in the package is tsl.experiment.Experiment and relies on Hydra for managing configurations.

Experiment#

class Experiment(run_fn: Callable, config_path: Optional[str] = None, config_name: Optional[str] = None, pre_run_hooks: Optional[Union[Callable, List[Callable]]] = None)[source]#

Simple class to handle the routines used to run experiments.

This class relies heavily on the Hydra framework, check Hydra docs for usage information.

Hydra is an optional dependency of tsl, to install it using pip:

pip install hydra-core
Parameters:

  • run_fn (callable) – Python function that actually runs the experiment when called. The run function must accept a single argument, being the experiment configuration.

  • config_path (str, optional) – Path to configuration files. If not specified the default will be used.

  • config_name (str, optional) – Name of the configuration file in config_path to be used. The yaml extension can be omitted.

  • pre_run_hooks (list) – Ordered list of functions to call on run() before the run_fn. Every hook must accept a single argument, being the experiment configuration, and act in-place on the configuration.

property run_dir#

Directory of the current run, where logs and artifacts are stored.

log_config() → None[source]#

Save config as .yaml file in run_dir().

run()[source]#

Run the experiment routine.

Loggers#

NeptuneLogger

Extensions of PyTorch Lightning NeptuneLogger with useful logging functionalities.
class NeptuneLogger(api_key: Optional[str] = None, project_name: Optional[str] = None, experiment_name: Optional[str] = None, tags: Optional[Union[str, List]] = None, params: Optional[Mapping] = None, save_dir: Optional[str] = None, debug: bool = False, prefix: Optional[str] = 'logs', upload_stdout: bool = False, **kwargs)[source]#

Extensions of PyTorch Lightning NeptuneLogger with useful logging functionalities.

Parameters:

  • api_key (str, optional) – Neptune API token, found on https://neptune.ai upon registration. Read: how to find and set Neptune API token. It is recommended to keep it in the NEPTUNE_API_TOKEN environment variable, then you can drop api_key=None. (default: None)

  • project_name (str, optional) – Name of a project in a form of “my_workspace/my_project”. If None, the value of NEPTUNE_PROJECT environment variable will be taken. You need to create the project in https://neptune.ai first. (default: None)

  • experiment_name (str, optional) – Editable name of the run. Run name appears in the “all metadata/sys” section in Neptune UI. (default: None)

  • tags (list, optional) – List of tags of the run. (default: None)

  • params (Mapping, optional) – Mapping of the run’s parameters (are logged as "parameters" on Neptune). (default: None)

  • save_dir (str, optional) – Save directory of the experiment, used to temporarily log artifacts before upload. If None, then defaults to .neptune. (default: None)

  • debug (bool) – If True, then do not log online (i.e., log in "debug" mode). Otherwise log online in "async" mode. (default: False)

  • prefix (str, optional) – Root namespace for all metadata logging. (default: "logs")

  • upload_stdout (bool) – If True, then log also stdout on Neptune. (default: False)

  • **kwargs – Additional parameters for NeptuneLogger.

property save_dir: Optional[str]#

Gets the save directory of the experiment.

Returns:

the root directory where experiment logs get saved

log_numpy(array, name: str = 'array')[source]#

Log a numpy array object.

Parameters:

  • array (array_like) – The array to be logged.

  • name (str) – The name of the file. (default: 'array')

log_dataframe(df: DataFrame, name: str = 'dataframe')[source]#

Log a dataframe as csv.

Parameters:

  • df (DataFrame) – The dataframe to be logged.

  • name (str) – The name of the file. (default: 'dataframe')

log_figure(fig, name: str = 'figure')[source]#

Log a matplotlib figure as html.

Parameters:

  • fig (Figure) – The matplotlib figure to be logged.

  • name (str) – The name of the file. (default: 'figure')

log_pred_df(name, idx, y, yhat, label_y='true', label_yhat='pred')[source]#

Log a csv containing predictions and true values. Only works for univariate timeseries.

Parameters:

  • name – name of the file

  • idx – dataframe idx

  • y – true values

  • yhat – predictions

  • label_y – true values

  • label_yhat – predictions

Returns: