Skip to main content

Filters

Filters are operations that can be defined under preprocessors. Preprocessors should be written as an array of filters. TIM applies alls filters in a specified order (i.e. with the AND operation in between array items). There are two main categories of filters: row filters and column filters. Currently only a categorical filter is supported.

Row filters

Category filter

For a specified categorical column, this filter enables the selection of only those rows with a specific value. Currently only group keys of panel data are supported. Training models only on a small subset of the existing groups saves training time, which can be useful when a user wants to forecast only those specific groups.

There are two possibilities for setting a categorical filter: a simple and complex filter.

Simple filter

A simple category filter contains only a single filter object, with the relevant column name and categories that should be filtered. Categories can be expressed both as Integers and as Strings.

{
"type" : "CategoryFilter",
"values": {
"column": "ColumnName_1",
"categories": [1, 2, 3]
}
}

Complex filter

A complex category filter allows for the definition of more complex filtering, enabling both AND and OR operations between filters. Such a filter is written as an array of arrays of filter values. In the inner arrays, filters are combined using the AND operation; between those inner arrays, (sets of) filters are combined using the OR operation.

The following example selects only those rows where

  • the values of ColumnName_1 are 1 or 2,
  • AND the values of ColumnName_2 are "a" or "b",

OR the rows where

  • the values of ColumnName_1 are 3 or 4,
  • AND the values of ColumnName_2 are "c" or "d".

Schematically, this means:

( ColumnName_1 in [1, 2] AND ColumnName_2 in ["a", "b"] )
OR
( ColumnName_1 in [3, 4] AND ColumnName_2 in ["c", "d"] )

The JSON for passing this set of filters to the TIM API looks like this:

{
"type" : "CategoryFilter",
"values": [
[
{
"column": "ColumnName_1",
"categories": [1, 2]
},
{
"column": "ColumnName_2",
"categories": ["a", "b"]
}
],
[
{
"column": "ColumnName_1",
"categories": [3, 4]
},
{
"column": "ColumnName_2",
"categories": ["c", "d"]
}
]
]
}

Column filters

These filters are currently not available in the preprocessors; it is only possible to manually list those columns that should be returned.