Task Structure

There are two types of forecasting tasks: backtesting and production. Both have their own distinct purpose and structure. A backtesting task involves building a model on historical data and then assessing its quality by applying it to ‘unseen’ data. This gives an impression of how the model would behave in production. When a certain model produces satisfying results in backtesting, the model can be moved into production. A production task involves applying a model to real scenarios with new data, thus generating forecasts for the future which can be used to base decisions on. TIM allows two inherently different ways to go about production tasks: either one of the models built in during backtesting is used an applied to new data, or TIM instantly generates a forecast straight from the data through its InstantML capabilities. In the first scenario, users must take care to assure the current situation conforms to the assumptions made when building the model. If a user fails to do so, the quality of the resulting forecast might suffer from it. The second scenario is the most convenient use of TIM, as all available data can be used with no further risks. A user only needs to specify the desired forecasting horizon. In this scenario, no model is returned.

Although many options allow users to personalize their forecasting process, most of these options do not require manual setting. When the setting of a certain option is omitted, it will be set up automatically using a default value.

For anomaly detection tasks, many of the settings are similar to forecasting. Here are the examples of YAML files containing different tasks.

Complete example of a backtesting forecasting task

version: "1.0"
type: Forecasting
modelBuilding:
  data:
    rows:
    - from: 2009-01-01 00:00:00
      to:   2009-06-30 23:00:00
    - from: 2009-09-01 00:00:00
      to:   2010-12-31 23:00:00
    columns: 1:4
    updates:
    - uniqueName: ElectricityLoad
      updateUntil:
        baseUnit: Day
        offset: -1
      updateTime:
      - type: Hour
        value: "8"
      - type: Minute
        value: "30"
    - uniqueName: Temperature
      updateUntil:
        baseUnit: Day
        offset: 1
      updateTime:
      - type: Hour
        value: "8"
      - type: Minute
        value: "30"
  configuration:
    usage:
      usageType: Repeating
      usageTime:
      - type: Day
        value: "Mo-Fr"
      - type: Hour
        value: "8-16"
      - type: Minute
        value: "0"
      predictionFrom:
        baseUnit: Sample
        offset: 1
      predictionTo:
        baseUnit: Day
        offset: 2
      modelQuality:
      - day: 0
        quality: UltraHigh
      - day: 1
        quality: High
      - day: 2
        quality: Medium
    features: [MovingAverage, Intercept, PiecewiseLinear, TimeOffsets, Polynomial]
    dataNormalization: true
    maxModelComplexity: 50
    timeSpecificModels: false
    extendedOutputConfiguration:
      returnSimpleImportances: true 
      returnExtendedImportances: true
forecasting:
  data:
    columns: 1:3, 5
  configuration:
    predictionScope:
      type: Ranges
      ranges:
      - from: 2011-01-01 00:00:00
        to:   2011-12-31 23:00:00
    extendedOutputConfiguration:
      returnAggregatedPredictions: true
      returnRawPredictions: true

Minimal example of a backtesting forecasting task

Most of the settings can be omitted and they will be set up automatically. The minimalistic task can then look like this:

version: "1.0"
type: Forecasting
modelBuilding:
  configuration:
    usage:
      usageType: Repeating
      usageTime:
      - type: Hour
        value: "8"
      - type: Minute
        value: "30"
      predictionFrom:
        baseUnit: Sample
        offset: 1
      predictionTo:
        baseUnit: Day
        offset: 1
forecasting:

Example of Instant Forecasting

cast. All the math settings can be configured the same way as in the backtesting task. No model is returned.

version: "1.0"
type: Forecasting
forecasting:
  configuration:
    usage:
      predictionTo:
        baseUnit: Day
        offset: 3

Complete example of an anomaly detection task

version: "1.0"
type: AnomalyDetection
modelBuilding:
  data:
    rows:
    - from: 2009-01-01 00:00:00
      to:   2009-06-30 23:00:00
    - from: 2009-09-01 00:00:00
      to:   2010-12-31 23:00:00
    columns: 1:4
    updates:
    - uniqueName: Temperature
      updateUntil:
        baseUnit: Day
        offset: -1
      updateTime:
      - type: Hour
        value: "12"
      - type: Minute
        value: "0"
  configuration:
    sensitivity: 2
    detectionIntervals:
    - type: Day
      value: "*"
    - type: Hour
      value: "6-18"
    - type: Minute
      value: "0"
    normalBehaviorModellingConfiguration:
      dataNormalization: true
      maxModelComplexity: 50
      features:
      - Polynomial
      - TimeOffsets
      - PiecewiseLinear
      - Intercept
      - PeriodicComponents
      - DayOfWeek
      - MovingAverage
      timeSpecificModels: false
      useTargetOffsets: false
    abnormalBehaviorModellingConfiguration:
      dataNormalization: true
      maxModelComplexity: 20
      features:
      # - Residual
      - Fluctuation
      # - FluctuationChange
      # - Magnitude
      # - MagnitudeChange
      - SimpleImbalance
      - SimpleImbalanceChange
      # - Imbalance
      - Increments
      # - IncrementsChange
      # - OneSidedIncrementsChange
      # - Hour
      # - DayOfWeek
      # - Month
    extendedOutputConfiguration:
      returnAnomalyIndicator: true
      returnNormalBehavior: true 
      returnSimpleImportances: false
      returnExtendedImportances: false
anomalyDetection:
  data:
    reportRows:
    - from: 2011-01-01 00:00:00
      to:   2011-12-31 23:00:00
    columns: 1:3, 5

Minimal example of an anomaly detection task

Most of the settings can be omitted and they will be set up automatically. The minimalist task can then look like this:

version: "1.0"
type: AnomalyDetection
modelBuilding:
anomalyDetection: