Skip to content

Anomaly Detection

After uploading a dataset, the user can start working with the uploaded dataset to build and apply anomaly detection models.

This section explains how to use the TIM Python client to create an anomaly detection job, execute such a job, combine both into a single method call and retrieve results. It also explains how to delete jobs from the TIM Repository.

build_anomaly_detection_model - create an anomaly detection model building job

build_anomaly_detection_model(self, dataset_id: str, job_configuration: Optional[tim.data_sources.anomaly_detection.types.BuildAnomalyDetectionModelConfiguration] = None) -> str

The build_anomaly_detection_model method registers an anomaly detection model building job in the TIM repository. This method is called on the authenticated instance of Tim created as described in Authentication ("client"), like in the following statement:

client.build_anomaly_detection_model(dataset_id = <dataset ID>, job_configuration = <job configuration>)

using keyword arguments, or in the following statement:

client.build_anomaly_detection_model(<dataset ID>, <job configuration>)

using positional arguments, where <dataset ID> and <job configuration> are replaced by the ID and Dictionary representing them, respectively.

The arguments are:

  • dataset_id: the ID of a dataset in the TIM repository,
  • job_configuration: model building and anomaly detection configuration for the TIM engine. This is an optional argument, available keys are:
    • name: the name to give to the anomaly detection job; this is an optional argument,
    • configuration: a Dictionary containing the configuration for the model building process. This is an optional argument. The following keys are available:
      • domainSpecifics: a list of Dictionaries enumerating all detection perspectives with their sensitivity-related settings entering the anomalous behavior model building process; this allows the user to adjust the anomaly detection to their preferences; available keys are:
        • perspective: a detection perspective to include as input of the anomalous behavior model building process; options are: "Residual", "ResidualChange", "Fluctuation", "FluctuationChange", Imbalance" and "ImbalanceChange"; by default "Residual" and "Imbalance" are included,
        • sensitivity: the sensitivity to anomalies for the given perspective; this represents the expected percentage (minimum 0, maximum 100) of anomalies in the model building (training) data; if not provided, TIM will automatically determine the optimal sensitivity taking into account minSensitivity and maxSensitivity,
        • minSensitivity: a lower boundary (minimum 0, maximum 100; default 0) that should be respected when determining the optimal sensitivity; if sensitivity is provided, this parameter will be ignored,
        • maxSensitivity: an upper boundary (minimum 0, maximum 100; default 5) that should be respected when determining the optimal sensitivity, if sensitivity is provided, this parameter will be ignored,
      • normalBehaviorModel: a Dictionary containing the settings for the normal behavior model; available keys are:
        • useNormalBehavior: a boolean indicating whether TIM should use normal behavior, the default value is True; when True, TIM finds the normal behavior of a given KPI and detects anomalous behavior based on the difference between the KPI and its normal behavior; when False TIM detects anomalies solely based on the KPI; if set to False, the rest of the normalBehaviorModel configuration parameters are irelevant, and normal behavior and residuals cannot be returned,
        • normalization: a boolean indicating whether influencers are scaled by their mean and standard deviation (normalized), the default value is True; switching normalization off may help when modeling data with structural changes,
        • maxModelComplexity: the maximal possible number of terms in the normal behavior model (minimum 1, maximum 100); if not set, TIM will calculate the complexity automatically based on the sampling period of the dataset,
        • features: a list enumerating all types of transformations TIM should use during feature engineering; if not provided, TIM will determine the optimal selection of features automatically; options are: "ExponentialMovingAverage", "RestOfWeek", "Periodic", "Intercept", "PiecewiseLinear", "TimeOffsets", "Polynomial", "Identity", "SimpleMovingAverage", "Month", "Trend", "DayOfWeek", "Fourier" and "PublicHolidays",
        • dailyCycle: a boolean indicating whether models should focus on the respective times within a day (specific hours, quarter-hours, etc.); if not set, TIM will determine this property automatically using autocorrelation analysis,
        • useKPIOffsets: a boolean indicating whether the normal behavior model should contain KPI offsets; if not provided, TIM will determine the optimal setting automatically,
        • allowOffsets: a boolean indicating whether TIM can use influencers that are not available throughout the whole detection period by creating their offset transformations; by default this is set to True,
      • anomalousBehaviorModel: a Dictionary containing the settings used for the anomalous behavior model, available keys are:
        • maxModelComplexity: the maximal complexity to search when building the anomalous behavior model (minimum 1, maximum 30; default 30),
        • detectionIntervals: a list of Dictionaries containing a cron-like notation indicating what parts of a day or week will be used by the anomalous behavior model; this setting can be beneficial in building more appropriate models and reducing model building time; by default, detection intervals are set to all samples; each Dictionary contains the following keys:
          • type: the type of interval, options are "Day", "Hour", "Minute" and "Second",
          • value: the exact value(s) or value range indicating the samples of type type for TIM to take into account,
    • data: a Dictionary containing the configuration for the data that is used for model building and application. This is an optional argument, available keys are:
      • versionId: the ID of the dataset version to use; if not set, the latest version will be used,
      • rows: a Dictionary or list of Dictionaries defining which samples should be used for model building; if not set, all observations will be used; either a relative range or (absolute) ranges can be defined:
        • for a relative range, the timestamps are defined as the last period of data counted backwards value baseUnits from the last value of the KPI variable. The available keys are:
          • baseUnit: base unit in which time is counted, one of "Day", "Hour", "Minute", "Second", "Month" and "Sample"; the default value is "Sample",
          • value: the number of base units; the minimum value is 0,
        • for (absolute) ranges, the timestamps of the in-sample range are defined as a list of explicit from-to ranges. A list of Dictionaries should be passed; the available keys in each Dictionary are:
          • from: the start timestamp of the range,
          • to: the end timestamp of the range,
      • columns: a list defining which columns (variables) should be used to build the model; if not set, all columns are selected. Each list item can either be a string (the name of the column) or an integer (the index of the column),
      • KPIColupmn: defines which column represents the KPI variable. The desired KPI has to be included in the "columns" selection; if not set, the first column in the "columns" selection is used. The KPI column can be indicated either by a string (the name of the column) or an integer (the index of the column),
      • holidayColumn: defines which column represents the holiday variable. This is an optional argument; if not set, no column is defined as holiday column. The holiday column can be indicated either by a string (the name of the column) or an integer (the index of the column),
      • imputation: a Dictionary to define the imputation of missing data in the influencer columns; if not provided, TIM will default to imputation of type "None" with a gap length of 0 samples (i.e. no imputation). This is an optional argument, available keys are:
        • type: the type of imputation to be applied; options are "Linear", "LOCF" (Last Observation Carried Forward) and "None",
        • maxGapLength: the maximum length of gaps that should be imputed; the minimum value is 0,
      • timeScale: a Dictionary defining the sampling period to which TIM should aggregate the dataset before model building and detecting. This is an optional parameter; if not set, TIM uses the dataset's sampling period. Available keys are:
        • baseUnit: base unit in which time is counted, one of "Day", "Hour", "Minute" and "Second",
        • value: the number of base units; the minimum value is 1,
      • aggregation: defines the aggregation function to be used for the KPI variable. The default aggregation is mean for numerical variables and maximum for boolean variables. This is an optional argument; options are "Mean", "Sum" "Maximum" and "Minimum",
      • updates: a list of Dictionaries containing the settings for managing data availabilities; if not set, TIM expects all columns in the dataset to be aligned with the last KPI observation; available keys in each Dictionary are:
        • column: defines the column represents the settings in this Dictionary relate to; the column can be indicated either by a string (the name of the column) or an integer (the index of the column),
        • updateTime: a list of Dictionaries indicating the time(s) of the update(s), available keys in each Dictionary are:
          • type: the type of interval, options are "Day", "Hour", "Minute" and "Second",
          • value: the exact value(s) or value range indicating the samples of type type for TIM to take into account,
        • updateUntil: a Dictionary indicating the relative position of the last updated (most recent) value to the update time; available keys are:
          • baseUnit: base unit in which the relative distance is counted, one of "Day", "Hour" and "Sample"; the default value is "Sample",
          • offset: the offset value (number of base units); the default value is 0.

This method returns the ID of the anomaly detection job that has been created.

create_anomaly_detection - create an anomaly detection detect job

create_anomaly_detection(self, parent_job_id: str, job_configuration: Optional[tim.data_sources.anomaly_detection.types.AnomalyDetectionDetectConfiguration] = None) -> str

The create_anomaly_detection method registers an anomaly detection detect job in the TIM repository. This method is called on the authenticated instance of Tim created as described in Authentication ("client"), like in the following statement:

client.create_anomaly_detection(parent_job_id = <parent job ID>, job_configuration = <job configuration>)

using keyword arguments, or in the following statement:

client.build_anomaly_detection_model(<parent job ID>, <job configuration>)

using positional arguments, where <parent job ID> and <job configuration> are replaced by the ID and Dictionary representing them, respectively.

The arguments are:

  • parent_job_id: the ID of the parent job, an anomaly detection model (re)building job from which the model should be used to detect with,
  • job_configuration: a Dictionary containing the anomaly detection configuration for the TIM engine. This is an optional argument, available keys are:
    • name: the name to give to the anomaly detection job; this is an optional argument,
    • data: a Dictionary containing the configuration for the data that is used for model application. This is an optional argument, available keys are:
      • version: a Dictionary containing the key id, refering to the ID of the dataset version to use; if not set, the latest version will be used,
      • rows: a Dictionary or list of Dictionaries defining which samples should be used for model application; if not set, all observations will be used; either a relative range or (absolute) ranges can be defined:
        • for a relative range, the timestamps are defined as the last period of data counted backwards value baseUnits from the last value of the KPI variable. The available keys are:
          • baseUnit: base unit in which time is counted, one of "Day", "Hour", "Minute", "Second", "Month" and "Sample"; the default value is "Sample",
          • value: the number of base units; the minimum value is 0,
        • for (absolute) ranges, the timestamps of the in-sample range are defined as a list of explicit from-to ranges. A list of Dictionaries should be passed; the available keys in each Dictionary are:
          • from: the start timestamp of the range,
          • to: the end timestamp of the range,
      • imputation: a Dictionary to define the imputation of missing data in the influencer columns; if not provided, TIM will default to imputation of type "None" with a gap length of 0 samples (i.e. no imputation). This is an optional argument, available keys are:
        • type: the type of imputation to be applied; options are "Linear", "LOCF" (Last Observation Carried Forward) and "None",
        • maxGapLength: the maximum length of gaps that should be imputed; the minimum value is 0,

This method returns the ID of the anomaly detection job that has been created.

execute_anomaly_detection - execute an anomaly detection job

execute_anomaly_detection(self, anomaly_detection_job_id: str, wait_to_finish: bool = True, handle_status_poll: Union[Callable[[tim.types.StatusResponse], NoneType], NoneType] = None) -> Union[tim.data_sources.anomaly_detection.types.AnomalyDetectionResultsResponse, tim.types.ExecuteResponse]

The execute_anomaly_detection method executes an anomaly detection job that has been registered. This method is called on the authenticated instance of Tim created as described in Authentication ("client"), like in the following statement:

client.execute_anomaly_detection(anomaly_detection_job_id = <anomaly detection job ID>, wait_to_finish = <wait to finish>, handle_status_poll = <callback function>)

using keyword arguments, or in the following statement:

client.execute_anomaly_detection(<anomaly detection job ID>, <wait to finish>, <callback function>)

using positional arguments, where <anomaly detection job ID> and <wait to finish> are replaced by the ID of the anomaly detection job to execute and an optional boolean indicating whether to wait for the execution to finish before returning, respectively, and <callback function> is replaced by an optional callback function for status polling.

The arguments are:

  • anomaly_detection_job_id: the ID of the anomaly detection job to execute,
  • wait_to_finish: a boolean indicating whether to wait for the execution to finish before returning; this is an optional parameter,
  • handle_status_poll: a callback function handling polling for the status and progress of the anomaly detection job execution.

If wait_to_finish is set to True, this method returns the following data:

  • metadata: a Dictionary containing metadata of the anomaly detection job (or None when the job failed); available keys are:
    • id: the ID of the anomaly detection job,
    • name: the name of the anomaly detection job,
    • type: the type of the anomaly detection job, options are "build-model", "rebuild-model", "detect, "rca",
    • status: the status of the anomaly detection job, options are "Registered", "Running", "Finished", "FinishedWithWarning", "Failed", "Queued",
    • parentId: the ID of the parent job of the anomaly detection job (if any),
    • experiment: a Dictionary containing the key id, refering to the ID of the experiment in which the anomaly detection job resides,
    • useCase: a Dictionary containing the key id, refering to the ID of the use case in which the experiment containing the anomaly detection job resides,
    • dataset: a Dictionary containing the key dataset, which in turn is a Dictionary containing the key id, refering to the ID of the dataset version the anomaly detection job is based on,
    • createdAt: the datetime at which the anomaly detection job was created,
    • executedAt: the datetime at which the anomaly detection job was executed,
    • completedAt: the datetime at which the anomaly detection job was completed,
    • workerVersion: the version of the worker the anomaly detection job has been executed with,
    • registrationBody: a Dictionary containing the settings that were used when registering the anomaly detection job (ex.g. by calling build_anomaly_detection_model()); available keys are:
      • name: the name given to the anomaly detection job upon registration,
      • useCase: a Dictionary containing the key id, refering to the ID of the use case the job was registered in,
      • configuration: the configuration of the job, see the key configuration in the argument job_configuration of build_anomaly_detection_model() for more information.
      • data: the data-related configuration of the job, see the key data in the argument job_configuration of build_anomaly_detection_model() for more information,
    • jobLoad: the load of the anomaly detection job, options are "Light" and "Heavy",
    • calculationTime: the amount of time that was required to execute the job,
  • model_result: a Dictionary containing information related to the model; available keys are:
    • modelVersion: the version of the model,
    • model: the model, containing the key normalBehaviorModel, which in turn contains the following keys:
      • samplingPeriod: the sampling period of the dataset the model is built with,
      • models: a list of Dictionaries containing the models themselves; each dictionary contains the following keys:
        • index: the index of the model in the modelZoo,
        • dayTime: the time of day the model is built to detect for in case of a daily cycle model, null in case of a non-daily cycle model,
        • terms: a list of Dictionaries containing the terms that compose the model; for each Dictionary the available keys are:
          • importance: the importance (level of contribution) of the term or feature in the model,
          • parts: a list of Dictionaries containing the parts that compose the term or feature; the following keys can be available:
            • type: the type of the term or feature; this determines the other key(s) that are present,
            • predictor: the predictor the term or feature relates to; used together with offset for some terms or features,
            • offset: the offset of the predictor the term or feature relates to; used together with predictor for some terms or features,
            • value: the value of the term or feature; used instead of predictor and offset for some terms or features,
            • window: the window of some transformation to a predictor creating some terms or features,
            • knot: a knot used in a piecewise linear transformation,
            • subtype: the subtype of the term or feature,
            • period: the period related to the term or feature; used together with unit for some terms or features,
            • cosOrder: a list of floats representing cosines,
            • sinOrder: a list of floats representing sines,
            • unit: the unit related to the period of a term or feature; used together with period for some terms or features,
            • day: a calendar transformation related to a day,
            • month: a calendar transformation related to a month,
      • variableProperties: a list of Dictionaries containing properties of each of the variables; each dictionary contains the following keys:
        • name: the name of the variable,
        • dataFrom: the largest offset of the variable used by the model,
        • dataTo: the smallest offset of the variable used by the model,
        • importance: the importance (level of contribution) of the variable in the model,
    • signature: the signature of the model,
  • table_result: a DataFrame containing the result table of the anomaly detection job; available columns are: timestamp, model_index, kpi, normal_behavior, anomaly_code, anomaly_indicator_residual, anomaly_indicator_residual_change, anomaly_indicator_fluctuation, anomaly_indicator_fluctuation_change, anomaly_indicator_imbalance and anomaly_indicator_imbalance_change; this table is empty for jobs of type "RCA",
  • logs: a list of Dictionaries, each of which containing the following keys:
    • message: the actual message of the log,
    • messageType: the type of the message; possible values are "Info", "Warning" and "Error",
    • createdAt: the time the message was raised,
    • origin: the phase of processing during which the message was raised; possible values are "Registration", "Execution" and "Validation".

Upon succesful execution of the anomaly detection job, metadata, model_result and table_result will be populated; logs will be returned for any job, including failed jobs.

If wait_to_finish is set to False, this method returns a Dictionary with the following keys:

  • message: a message indicating what has happened (the anomaly detection job has been posted to a queue),
  • code: a code providing more information on this message.

If an error is encountered, a similar Dictionary will be returned with the keys message and code containing additional information about the error.

build_anomaly_detection_model_and_execute - create and execute an anomaly detection model building job

build_anomaly_detection_model_and_execute(self, dataset_id: str, job_configuration: Union[tim.data_sources.anomaly_detection.types.BuildAnomalyDetectionModelConfiguration, NoneType] = None, wait_to_finish: bool = True, handle_status_poll: Union[Callable[[tim.types.StatusResponse], NoneType], NoneType] = None) -> Union[tim.data_sources.anomaly_detection.types.AnomalyDetectionResultsResponse, tim.types.ExecuteResponse]

The build_anomaly_detection_model_and_execute method registers an anomaly detection job in the TIM repository and then immediately executes it. It combines the functionality of build_anomaly_detection_model() and execute_anomaly_detection(). This method is called on the authenticated instance of Tim created as described in Authentication ("client"), like in the following statement:

client.build_anomaly_detection_model_and_execute(dataset_id = <dataset ID>, job_configuration = <job configuration>, wait_to_finish = <wait to finish>, handle_status_poll = <callback function>)

using keyword arguments, or in the following statement:

client.build_anomaly_detection_model_and_execute(<dataset ID>, <job configuration>, <wait to finish>, <callback function>)

using positional arguments, where <dataset ID> and <job configuration> are replaced by the ID and Dictionary representing them, <wait to finish> represents an optional boolean indicating whether to wait for the execution to finish before returning, and <callback function> is replaced by an optional callback function for status polling.

The arguments are:

  • dataset_id: the ID of a dataset in the TIM repository,
  • job_configuration: model building and anomaly detection configuration for the TIM engine. This is an optional argument, available keys are:
    • name: the name to give to the anopmaly detection job. This is an optional argument.
    • configuration: a Dictionary containing the configuration for the model building process. This is an optional argument. The following keys are available:
      • domainSpecifics: a list of Dictionaries enumerating all detection perspectives with their sensitivity-related settings entering the anomalous behavior model building process; this allows the user to adjust the anomaly detection to their preferences; available keys are:
        • perspective: a detection perspective to include as input of the anomalous behavior model building process; options are: "Residual", "ResidualChange", "Fluctuation", "FluctuationChange", Imbalance" and "ImbalanceChange"; by default "Residual" and "Imbalance" are included,
        • sensitivity: the sensitivity to anomalies for the given perspective; this represents the expected percentage (minimum 0, maximum 100) of anomalies in the model building (training) data; if not provided, TIM will automatically determine the optimal sensitivity taking into account minSensitivity and maxSensitivity,
        • minSensitivity: a lower boundary (minimum 0, maximum 100; default 0) that should be respected when determining the optimal sensitivity; if sensitivity is provided, this parameter will be ignored,
        • maxSensitivity: an upper boundary (minimum 0, maximum 100; default 5) that should be respected when determining the optimal sensitivity, if sensitivity is provided, this parameter will be ignored,
      • normalBehaviorModel: a Dictionary containing the settings for the normal behavior model; available keys are:
        • useNormalBehavior: a boolean indicating whether TIM should use normal behavior, the default value is True; when True, TIM finds the normal behavior of a given KPI and detects anomalous behavior based on the difference between the KPI and its normal behavior; when False TIM detects anomalies solely based on the KPI; if set to False, the rest of the normalBehaviorModel configuration parameters are irelevant, and normal behavior and residuals cannot be returned,
        • normalization: a boolean indicating whether influencers are scaled by their mean and standard deviation (normalized), the default value is True; switching normalization off may help when modeling data with structural changes,
        • maxModelComplexity: the maximal possible number of terms in the normal behavior model (minimum 1, maximum 100); if not set, TIM will calculate the complexity automatically based on the sampling period of the dataset,
        • features: a list enumerating all types of transformations TIM should use during feature engineering; if not provided, TIM will determine the optimal selection of features automatically; options are: "ExponentialMovingAverage", "RestOfWeek", "Periodic", "Intercept", "PiecewiseLinear", "TimeOffsets", "Polynomial", "Identity", "SimpleMovingAverage", "Month", "Trend", "DayOfWeek", "Fourier" and "PublicHolidays",
        • dailyCycle: a boolean indicating whether models should focus on the respective times within a day (specific hours, quarter-hours, etc.); if not set, TIM will determine this property automatically using autocorrelation analysis,
        • useKPIOffsets: a boolean indicating whether the normal behavior model should contain KPI offsets; if not provided, TIM will determine the optimal setting automatically,
        • allowOffsets: a boolean indicating whether TIM can use influencers that are not available throughout the whole detection period by creating their offset transformations; by default this is set to True,
      • anomalousBehaviorModel: a Dictionary containing the settings used for the anomalous behavior model, available keys are:
        • maxModelComplexity: the maximal complexity to search when building the anomalous behavior model (minimum 1, maximum 30; default 30),
        • detectionIntervals: a list of Dictionaries containing a cron-like notation indicating what parts of a day or week will be used by the anomalous behavior model; this setting can be beneficial in building more appropriate models and reducing model building time; by default, detection intervals are set to all samples; each Dictionary contains the following keys:
          • type: the type of interval, options are "Day", "Hour", "Minute" and "Second",
          • value: the exact value(s) or value range indicating the samples of type type for TIM to take into account,
    • data: a Dictionary containing the configuration for the data that is used for model building and application. This is an optional argument, available keys are:
      • version: a Dictionary containing the key id, refering to the ID of the dataset version to use; if not set, the latest version will be used,
      • rows: a Dictionary or list of Dictionaries defining which samples should be used for model building; if not set, all observations will be used; either a relative range or (absolute) ranges can be defined:
        • for a relative range, the timestamps are defined as the last period of data counted backwards value baseUnits from the last value of the KPI variable. The available keys are:
          • baseUnit: base unit in which time is counted, one of "Day", "Hour", "Minute", "Second", "Month" and "Sample"; the default value is "Sample",
          • value: the number of base units; the minimum value is 0,
        • for (absolute) ranges, the timestamps of the in-sample range ar defined as a list of explicit from-to ranges. A list of Dictionaries should be passed; the available keys in each Dictionary are:
          • from: the start timestamp of the range,
          • to: the end timestamp of the range,
      • columns: a list defining which columns (variables) should be used to build the model; if not set, all columns are selected. Each list item can either be a string (the name of the column) or an integer (the index of the column),
      • KPIColupmn: defines which column represents the KPI variable. The desired KPI has to be included in the "columns" selection; if not set, the first column in the "columns" selection is used. The KPI column can be indicated either by a string (the name of the column) or an integer (the index of the column),
      • holidayColumn: defines which column represents the holiday variable. This is an optional argument; if not set, no column is defined as holiday column. The holiday column can be indicated either by a string (the name of the column) or an integer (the index of the column),
      • imputation: a Dictionary to define the imputation of missing data in the influencer columns; if not provided, TIM will default to imputation of type "None" with a gap length of 0 samples (i.e. no imputation). This is an optional argument, available keys are:
        • type: the type of imputation to be applied; options are "Linear", "LOCF" (Last Observation Carried Forward) and "None",
        • maxGapLength: the maximum length of gaps that should be imputed; the minimum value is 0,
      • timeScale: a Dictionary defining the sampling period to which TIM should aggregate the dataset before model building and detecting. This is an optional parameter; if not set, TIM uses the dataset's sampling period. Available keys are:
        • baseUnit: base unit in which time is counted, one of "Day", "Hour", "Minute" and "Second",
        • value: the number of base units; the minimum value is 1,
      • aggregation: defines the aggregation function to be used for the KPI variable. The default aggregation is mean for numerical variables and maximum for boolean variables. This is an optional argument; options are "Mean", "Sum" "Maximum" and "Minimum",
      • updates: a list of Dictionaries containing the settings for managing data availabilities; if not set, TIM expects all columns in the dataset to be aligned with the last KPI observation; available keys in each Dictionary are:
        • column: defines the column represents the settings in this Dictionary relate to; the column can be indicated either by a string (the name of the column) or an integer (the index of the column),
        • updateTime: a list of Dictionaries indicating the time(s) of the update(s), available keys in each Dictionary are:
          • type: the type of interval, options are "Day", "Hour", "Minute" and "Second",
          • value: the exact value(s) or value range indicating the samples of type type for TIM to take into account,
        • updateUntil: a Dictionary indicating the relative position of the last updated (most recent) value to the update time; available keys are:
          • baseUnit: base unit in which the relative distance is counted, one of "Day", "Hour" and "Sample"; the default value is "Sample",
          • offset: the offset value (number of base units); the default value is 0.
  • wait_to_finish: a boolean indicating whether to wait for the execution to finish before returning; this is an optional parameter,
  • handle_status_poll: a callback function handling polling for the status and progress of the anomaly detection job execution.

If wait_to_finish is set to True, this method returns the following data:

  • metadata: a Dictionary containing metadata of the anomaly detection job (or None when the job failed); available keys are:
    • id: the ID of the anomaly detection job,
    • name: the name of the anomaly detection job,
    • type: the type of the anomaly detection job, options are "build-model", "rebuild-model", "detect, "rca",
    • status: the status of the anomaly detection job, options are "Registered", "Running", "Finished", "FinishedWithWarning", "Failed", "Queued",
    • parentId: the ID of the parent job of the anomaly detection job (if any),
    • experiment: a Dictionary containing the key id, refering to the ID of the experiment in which the anomaly detection job resides,
    • useCase: a Dictionary containing the key id, refering to the ID of the use case in which the experiment containing the anomaly detection job resides,
    • dataset: a Dictionary containing the key dataset, which in turn is a Dictionary containing the key id, refering to the ID of the dataset version the anomaly detection job is based on,
    • createdAt: the datetime at which the anomaly detection job was created,
    • executedAt: the datetime at which the anomaly detection job was executed,
    • completedAt: the datetime at which the anomaly detection job was completed,
    • workerVersion: the version of the worker the anomaly detection job has been executed with,
    • registrationBody: a Dictionary containing the settings that were used when registering the anomaly detection job (ex.g. by calling build_anomaly_detection_model()); available keys are:
      • name: the name given to the anomaly detection job upon registration,
      • useCase: a Dictionary containing the key id, refering to the ID of the use case the job was registered in,
      • configuration: the configuration of the job, see the key configuration in the argument job_configuration of build_anomaly_detection_model() for more information.
      • data: the data-related configuration of the job, see the key data in the argument job_configuration of build_anomaly_detection_model() for more information,
    • jobLoad: the load of the anomaly detection job, options are "Light" and "Heavy",
    • calculationTime: the amount of time that was required to execute the job,
  • model_result: a Dictionary containing information related to the model; available keys are:
    • modelVersion: the version of the model,
    • model: the model, containing the key normalBehaviorModel, which in turn contains the following keys:
      • samplingPeriod: the sampling period of the dataset the model is built with,
      • models: a list of Dictionaries containing the models themselves; each dictionary contains the following keys:
        • index: the index of the model in the modelZoo,
        • dayTime: the time of day the model is built to detect for in case of a daily cycle model, null in case of a non-daily cycle model,
        • terms: a list of Dictionaries containing the terms that compose the model; for each Dictionary the available keys are:
          • importance: the importance (level of contribution) of the term or feature in the model,
          • parts: a list of Dictionaries containing the parts that compose the term or feature; the following keys can be available:
            • type: the type of the term or feature; this determines the other key(s) that are present,
            • predictor: the predictor the term or feature relates to; used together with offset for some terms or features,
            • offset: the offset of the predictor the term or feature relates to; used together with predictor for some terms or features,
            • value: the value of the term or feature; used instead of predictor and offset for some terms or features,
            • window: the window of some transformation to a predictor creating some terms or features,
            • knot: a knot used in a piecewise linear transformation,
            • subtype: the subtype of the term or feature,
            • period: the period related to the term or feature; used together with unit for some terms or features,
            • cosOrder: a list of floats representing cosines,
            • sinOrder: a list of floats representing sines,
            • unit: the unit related to the period of a term or feature; used together with period for some terms or features,
            • day: a calendar transformation related to a day,
            • month: a calendar transformation related to a month,
      • variableProperties: a list of Dictionaries containing properties of each of the variables; each dictionary contains the following keys:
        • name: the name of the variable,
        • dataFrom: the largest offset of the variable used by the model,
        • dataTo: the smallest offset of the variable used by the model,
        • importance: the importance (level of contribution) of the variable in the model,
    • signature: the signature of the model,
  • table_result: a DataFrame containing the result table of the anomaly detection job; available columns are: timestamp, model_index, kpi, normal_behavior, anomaly_code, anomaly_indicator_residual, anomaly_indicator_residual_change, anomaly_indicator_fluctuation, anomaly_indicator_fluctuation_change, anomaly_indicator_imbalance and anomaly_indicator_imbalance_change; this table is empty for jobs of type "RCA",
  • logs: a list of Dictionaries, each of which containing the following keys:
    • message: the actual message of the log,
    • messageType: the type of the message; possible values are "Info", "Warning" and "Error",
    • createdAt: the time the message was raised,
    • origin: the phase of processing during which the message was raised; possible values are "Registration", "Execution" and "Validation".

Upon successful execution of the anomaly detection job, metadata, model_result and table_result will be populated; logs will be returned for any job, including failed jobs.

If wait_to_finish is set to False, this method returns a Dictionary with the following keys:

  • message: a message indicating what has happened (the anomaly detection job has been posted to a queue),
  • code: a code providing more information on this message.

If an error is encountered, a similar Dictionary will be returned with the keys message and code containing additional information about the error.

create_anomaly_detection_and_execute - create and execute an anomaly detection detect job

create_anomaly_detection_and_execute(self, parent_job_id: str, job_configuration: Union[tim.data_sources.anomaly_detection.types.AnomalyDetectionDetectConfiguration, NoneType] = None, wait_to_finish: bool = True, handle_status_poll: Union[Callable[[tim.types.StatusResponse], NoneType], NoneType] = None) -> Union[tim.data_sources.anomaly_detection.types.AnomalyDetectionResultsResponse, tim.types.ExecuteResponse]

The create_anomaly_detection_and_execute method registers an anomaly detection job in the TIM repository and then immediately executes it. It combines the functionality of create_anomaly_detection() and execute_anomaly_detection(). This method is called on the authenticated instance of Tim created as described in Authentication ("client"), like in the following statement:

client.create_anomaly_detection_and_execute(parent_job_id = <parent job ID>, job_configuration = <job configuration>, wait_to_finish = <wait to finish>, handle_status_poll = <callback function>)

using keyword arguments, or in the following statement:

client.create_anomaly_detection_and_execute(<dataset ID>, <job configuration>, <wait to finish>, <callback function>)

using positional arguments, where <dataset ID> and <job configuration> are replaced by the ID and Dictionary representing them, <wait to finish> represents an optional boolean indicating whether to wait for the execution to finish before returning, and <callback function> is replaced by an optional callback function for status polling.

The arguments are:

  • parent_job_id: the ID of the parent job, an anomaly detection model (re)building job from which the model should be used to detect with,
  • job_configuration: a Dictionary containing the anomaly detection configuration for the TIM engine. This is an optional argument, available keys are:
    • name: the name to give to the anomaly detection job; this is an optional argument,
    • data: a Dictionary containing the configuration for the data that is used for model application. This is an optional argument, available keys are:
      • version: a Dictionary containing the key id, refering to the ID of the dataset version to use; if not set, the latest version will be used,
      • rows: a Dictionary or list of Dictionaries defining which samples should be used for model application; if not set, all observations will be used; either a relative range or (absolute) ranges can be defined:
        • for a relative range, the timestamps are defined as the last period of data counted backwards value baseUnits from the last value of the KPI variable. The available keys are:
          • baseUnit: base unit in which time is counted, one of "Day", "Hour", "Minute", "Second", "Month" and "Sample"; the default value is "Sample",
          • value: the number of base units; the minimum value is 0,
        • for (absolute) ranges, the timestamps of the in-sample range are defined as a list of explicit from-to ranges. A list of Dictionaries should be passed; the available keys in each Dictionary are:
          • from: the start timestamp of the range,
          • to: the end timestamp of the range,
      • imputation: a Dictionary to define the imputation of missing data in the influencer columns; if not provided, TIM will default to imputation of type "None" with a gap length of 0 samples (i.e. no imputation). This is an optional argument, available keys are:
        • type: the type of imputation to be applied; options are "Linear", "LOCF" (Last Observation Carried Forward) and "None",
        • maxGapLength: the maximum length of gaps that should be imputed; the minimum value is 0,
  • wait_to_finish: a boolean indicating whether to wait for the execution to finish before returning; this is an optional parameter,
  • handle_status_poll: a callback function handling polling for the status and progress of the anomaly detection job execution.

If wait_to_finish is set to True, this method returns the following data:

  • metadata: a Dictionary containing metadata of the anomaly detection job (or None when the job failed); available keys are:
    • id: the ID of the anomaly detection job,
    • name: the name of the anomaly detection job,
    • type: the type of the anomaly detection job, options are "build-model", "rebuild-model", "detect, "rca",
    • status: the status of the anomaly detection job, options are "Registered", "Running", "Finished", "FinishedWithWarning", "Failed", "Queued",
    • parentId: the ID of the parent job of the anomaly detection job (if any),
    • experiment: a Dictionary containing the key id, refering to the ID of the experiment in which the anomaly detection job resides,
    • useCase: a Dictionary containing the key id, refering to the ID of the use case in which the experiment containing the anomaly detection job resides,
    • dataset: a Dictionary containing the key dataset, which in turn is a Dictionary containing the key id, refering to the ID of the dataset version the anomaly detection job is based on,
    • createdAt: the datetime at which the anomaly detection job was created,
    • executedAt: the datetime at which the anomaly detection job was executed,
    • completedAt: the datetime at which the anomaly detection job was completed,
    • workerVersion: the version of the worker the anomaly detection job has been executed with,
    • registrationBody: a Dictionary containing the settings that were used when registering the anomaly detection job (ex.g. by calling create_anomaly_detection()); available keys are:
      • name: the name given to the anomaly detection job upon registration,
      • data: the data-related configuration of the job, see the key data in the argument job_configuration of create_anomaly_detection() for more information,
    • jobLoad: the load of the anomaly detection job, options are "Light" and "Heavy",
    • calculationTime: the amount of time that was required to execute the job,
  • model_result: a Dictionary containing information related to the model; available keys are:
    • modelVersion: the version of the model,
    • model: the model, containing the key normalBehaviorModel, which in turn contains the following keys:
      • samplingPeriod: the sampling period of the dataset the model is built with,
      • models: a list of Dictionaries containing the models themselves; each dictionary contains the following keys:
        • index: the index of the model in the modelZoo,
        • dayTime: the time of day the model is built to detect for in case of a daily cycle model, null in case of a non-daily cycle model,
        • terms: a list of Dictionaries containing the terms that compose the model; for each Dictionary the available keys are:
          • importance: the importance (level of contribution) of the term or feature in the model,
          • parts: a list of Dictionaries containing the parts that compose the term or feature; the following keys can be available:
            • type: the type of the term or feature; this determines the other key(s) that are present,
            • predictor: the predictor the term or feature relates to; used together with offset for some terms or features,
            • offset: the offset of the predictor the term or feature relates to; used together with predictor for some terms or features,
            • value: the value of the term or feature; used instead of predictor and offset for some terms or features,
            • window: the window of some transformation to a predictor creating some terms or features,
            • knot: a knot used in a piecewise linear transformation,
            • subtype: the subtype of the term or feature,
            • period: the period related to the term or feature; used together with unit for some terms or features,
            • cosOrder: a list of floats representing cosines,
            • sinOrder: a list of floats representing sines,
            • unit: the unit related to the period of a term or feature; used together with period for some terms or features,
            • day: a calendar transformation related to a day,
            • month: a calendar transformation related to a month,
      • variableProperties: a list of Dictionaries containing properties of each of the variables; each dictionary contains the following keys:
        • name: the name of the variable,
        • dataFrom: the largest offset of the variable used by the model,
        • dataTo: the smallest offset of the variable used by the model,
        • importance: the importance (level of contribution) of the variable in the model,
    • signature: the signature of the model,
  • table_result: a DataFrame containing the result table of the anomaly detection job; available columns are: timestamp, model_index, kpi, normal_behavior, anomaly_code, anomaly_indicator_residual, anomaly_indicator_residual_change, anomaly_indicator_fluctuation, anomaly_indicator_fluctuation_change, anomaly_indicator_imbalance and anomaly_indicator_imbalance_change; this table is empty for jobs of type "RCA",
  • logs: a list of Dictionaries, each of which containing the following keys:
    • message: the actual message of the log,
    • messageType: the type of the message; possible values are "Info", "Warning" and "Error",
    • createdAt: the time the message was raised,
    • origin: the phase of processing during which the message was raised; possible values are "Registration", "Execution" and "Validation".

Upon successful execution of the anomaly detection job, metadata, model_result and table_result will be populated; logs will be returned for any job, including failed jobs.

If wait_to_finish is set to False, this method returns a Dictionary with the following keys:

  • message: a message indicating what has happened (the anomaly detection job has been posted to a queue),
  • code: a code providing more information on this message.

If an error is encountered, a similar Dictionary will be returned with the keys message and code containing additional information about the error.

get_anomaly_detection_results - retrieve the results of an executed anomaly detection job

get_anomaly_detection_results(self, anomaly_detection_job_id: str) -> tim.data_sources.anomaly_detection.types.AnomalyDetectionResultsResponse

The get_anomaly_detection_results method retrieves the results of an executed anomaly detection job from the TIM repository. This method is called on the authenticated instance of Tim created as described in Authentication ("client"), like in the following statement:

client.get_anomaly_detection_results(anomaly_detection_job_id = <anomaly detection job ID>)

using a keyword argument, or in the following statement:

client.get_anomaly_detection_results(<anomaly detection job ID>)

using a positional argument, where <anomaly detection job ID> is replaced by the ID of the anomaly detection job the results should be retrieved of.

The arguments are:

  • anomaly_detection_job_id: the ID of the anomaly detection job of which to retrieve the results.

This method returns the following data:

  • metadata: a Dictionary containing metadata of the anomaly detection job (or None when the job failed); available keys are:
    • id: the ID of the anomaly detection job,
    • name: the name of the anomaly detection job,
    • type: the type of the anomaly detection job, options are "build-model", "rebuild-model", "detect, "rca",
    • status: the status of the anomaly detection job, options are "Registered", "Running", "Finished", "FinishedWithWarning", "Failed", "Queued",
    • parentId: the ID of the parent job of the anomaly detection job (if any),
    • experiment: a Dictionary containing the key id, refering to the ID of the experiment in which the anomaly detection job resides,
    • useCase: a Dictionary containing the key id, refering to the ID of the use case in which the experiment containing the anomaly detection job resides,
    • dataset: a Dictionary containing the key dataset, which in turn is a Dictionary containing the key id, refering to the ID of the dataset version the anomaly detection job is based on,
    • createdAt: the datetime at which the anomaly detection job was created,
    • executedAt: the datetime at which the anomaly detection job was executed,
    • completedAt: the datetime at which the anomaly detection job was completed,
    • workerVersion: the version of the worker the anomaly detection job has been executed with,
    • registrationBody: a Dictionary containing the settings that were used when registering the anomaly detection job (ex.g. by calling build_anomaly_detection_model()); available keys are:
      • name: the name given to the anomaly detection job upon registration,
      • useCase: a Dictionary containing the key id, refering to the ID of the use case the job was registered in,
      • configuration: the configuration of the job, see the key configuration in the argument job_configuration of build_anomaly_detection_model() for more information,
      • data: the data-related configuration of the job, see the key data in the argument job_configuration of build_anomaly_detection_model() for more information.
    • jobLoad: the load of the anomaly detection job, options are "Light" and "Heavy",
    • calculationTime: the amount of time that was required to execute the job,
  • model_result: a Dictionary containing information related to the model; available keys are:
    • modelVersion: the version of the model,
    • model: the model, containing the key normalBehaviorModel, which in turn contains the following keys:
      • samplingPeriod: the sampling period of the dataset the model is built with,
      • models: a list of Dictionaries containing the models themselves; each dictionary contains the following keys:
        • index: the index of the model in the modelZoo,
        • dayTime: the time of day the model is built to detect for in case of a daily cycle model, null in case of a non-daily cycle model,
        • terms: a list of Dictionaries containing the terms that compose the model; for each Dictionary the available keys are:
          • importance: the importance (level of contribution) of the term or feature in the model,
          • parts: a list of Dictionaries containing the parts that compose the term or feature; the following keys can be available:
            • type: the type of the term or feature; this determines the other key(s) that are present,
            • predictor: the predictor the term or feature relates to; used together with offset for some terms or features,
            • offset: the offset of the predictor the term or feature relates to; used together with predictor for some terms or features,
            • value: the value of the term or feature; used instead of predictor and offset for some terms or features,
            • window: the window of some transformation to a predictor creating some terms or features,
            • knot: a knot used in a piecewise linear transformation,
            • subtype: the subtype of the term or feature,
            • period: the period related to the term or feature; used together with unit for some terms or features,
            • cosOrder: a list of floats representing cosines,
            • sinOrder: a list of floats representing sines,
            • unit: the unit related to the period of a term or feature; used together with period for some terms or features,
            • day: a calendar transformation related to a day,
            • month: a calendar transformation related to a month,
      • variableProperties: a list of Dictionaries containing properties of each of the variables; each dictionary contains the following keys:
        • name: the name of the variable,
        • dataFrom: the largest offset of the variable used by the model,
        • dataTo: the smallest offset of the variable used by the model,
        • importance: the importance (level of contribution) of the variable in the model,
    • signature: the signature of the model,
  • table_result: a DataFrame containing the result table of the anomaly detection job; available columns are: timestamp, model_index, kpi, normal_behavior, anomaly_code, anomaly_indicator_residual, anomaly_indicator_residual_change, anomaly_indicator_fluctuation, anomaly_indicator_fluctuation_change, anomaly_indicator_imbalance and anomaly_indicator_imbalance_change; this table is empty for jobs of type "RCA",
  • logs: a list of Dictionaries, each of which containing the following keys:
    • message: the actual message of the log,
    • messageType: the type of the message; possible values are "Info", "Warning" and "Error",
    • createdAt: the time the message was raised,
    • origin: the phase of processing during which the message was raised; possible values are "Registration", "Execution" and "Validation".

Upon succesful execution of the anomaly detection job, metadata, model_result and table_result will be populated; logs will be returned for any job, including failed jobs.

get_anomaly_detection_jobs - retrieve a list of anomaly detection jobs

get_anomaly_detection_jobs(self, offset: Optional[int] = None, limit: Optional[int] = None, sort: str = '-createdAt', experiment_id: Optional[str] = None, use_case_id: Optional[str] = None, type: Optional[str] = None, status: Optional[str] = None, parent_id: Optional[str] = None, from_datetime: Optional[str] = None, to_datetime: Optional[str] = None) -> List[tim.data_sources.anomaly_detection.types.AnomalyDetection]

The get_anomaly_detection_jobs method retrieves a list of available anomaly detection jobs from the TIM repository. This method is called on the authenticated instance of Tim created as described in Authentication ("client"), like in the following statement:

client.get_anomaly_detection_jobs(offset = <offset>, limit = <limit>, sort = <sorting order>, experiment_id = <experiment ID>, use_case_id = <use case ID>, type = <job type>, status = <job status>, parent_id = <parent job ID>, from_datetime = <from datetime>, to_datetime = <to datetime>)

using keyword arguments, or in the following statement:

client.get_anomaly_detection_jobs(<offset>, <limit>, <sorting order>, <experiment ID>, <use case ID>, <type>, <status>, <parent job ID>, <from datetime>, <to datetime>)

using positional arguments, where <offset>, <limit>, <sorting order>, <experiment ID>, <use case ID>, <type>, <status>, <parent job ID>, <from datetime> and <to datetime> are replaced by the relevant values.

The arguments are:

  • offset: the number of anomaly_detection jobs to be skipped from the beginning of the list (related to pagination), this is an optional argument with a default value of 0,
  • limit: the maximum number of anomaly_detection jobs to be returned, this is an optional argument with a default value of 10000,
  • sort: a sorting order to sort results by, possible values are "+createdAt", "-createdAt", "+executedAt", "-executedAt", "+completedAt", "-completedAt", "+priority" and "-priority", where "+" and "-" indicate ascending and descending order, respectively. This is an optional argument with a default value of "-createdAt" (most recently created anomaly_detection jobs are returned first),
  • experiment_id: a filter on the ID of the experiment a anomaly detection job resides in, this is an optional argument with a default value of None,
  • use_case_id: a filter on the ID of the use case a anomaly detection job resides in, this is an optional argument with a default value of None,
  • type: a filter on the type of the anomaly detection job, possible values are any comma separated list of "build-model", "rebuild-model", "detect" and "rca". This is an optional argument with a default value of None,
  • status: a filter on the status of the anomaly detection job, possible values are any comma separated list of "Registered", "Queued", "Running", "Finished", "FinishedWithWarning" and "Failed". This is an optional argument with a default value of None,
  • parent_id: a filter on the ID of the parent job of the anomaly detection job(s) to retrieve, this is an optional argument with a default value of None,
  • from_datetime: a filter for a minimal date and time of job creation, this is an optional argument with a default value of None,
  • to_datetime: a filter for a maximal date and time of job creation, this is an optional argument with a default value of None.

This method returns a list of Dictionaries, each of which can include the following data:

  • id: the ID of the anomaly_detection job,
  • name: the name of the anomaly_detection job,
  • type: the type of the anomaly_detection job,
  • status: the status of the anomaly_detection job,rrr
  • parentJob: a Dictionary containing the following keys:
    • id: the ID of the parent job of the anomaly_detection job,
  • useCase: a Dictionary containing the following keys:
    • id: the ID of the use case the anomaly_detection job resides in,
  • experiment: a Dictionary containing the following keys:
    • id: the ID of the experiment the anomaly_detection job resides in,
  • dataset: a Dictionary containing the following keys:
    • version: a Dictionary containing the following keys:
      • id: the ID of the dataset version the job is executed on,
  • createdAt: the date and time of creation of this anomaly_detection job,
  • completedAt: the date and time of completion of this anomaly_detection job,
  • executedAt: the date and time of execution of this anomaly_detection job,
  • workerVersion: the version of the worker used to execute this anomaly_detection job,
  • registrationBody: a Dictionary containing the settings that were used when registering the anomaly detection job (ex.g. by calling build_anomaly_detection_model()); available keys are:
    • name: the name given to the anomaly detection job upon registration,
    • useCase: a Dictionary containing the key id, refering to the ID of the use case the job was registered in,
    • configuration: the configuration of the job, see the key configuration in the argument job_configuration of build_anomaly_detection_model() for more information.
    • data: the data-related configuration of the job, see the key data in the argument job_configuration of build_anomaly_detection_model() for more information.

delete_anomaly_detection - delete an anomaly detection job

delete_anomaly_detection(self, anomaly_detection_job_id) -> tim.types.ExecuteResponse

The delete_anomaly_detection method deletes an anomaly detection job from the TIM repository. This method is called on the authenticated instance of Tim created as described in Authentication ("client"), like in the following statement:

client.delete_anomaly_detection(anomaly_detection_job_id = <anomaly detection job ID>)

using keyword arguments, or in the following statement:

client.delete_anomaly_detection(<anomaly detection job ID>)

using positional arguments, where <anomaly detection job ID> is replaced by the ID of the anomaly detection job to delete.

The argument is:

  • anomaly_detection_job_id: the ID of the anomaly detection job to delete.

This method returns a Dictionary with the following keys:

  • message: a message indicating what has happened (the anomaly detection job has successfully been deleted),
  • code: a code providing more information on this message; if the deletion was successful, this code will be "AD07034".

If an error is encountered, a similar Dictionary will be returned with the keys message and code containing additional information about the error.