The job's result include its ModelZoo, which is visualized on the experiment page in TIM Studio.
By default, the predictor importances are shown, visualized in a treemap. This visualization represents the extent to which each predictor (each input variable) contribute to the models and thus the forecast or detection. The color scheme of the treemap matches the colors of the predictors in the line chart; hovering over the treemap will give the user more information about the relevant predictors and their importance.
Flipping the switch above the treemap changes to that of the feature importances. This visualization represents the extent to which each feature (generated transformation of input variables and other data) contribute to the model and thus the forecast or detection. If applicable, a slider bar will appear allowing users to browse through the different models used in calculating the requested forecast or detection. Again, the color scheme of the treemap matches the colors of the predictors in the line chart; this time a single feature may have multiple colors, meaning it represents an interaction between multiple (input or artificial) variables. Again, hovering over one of the treemaps will give the user more information about the relevant features and their importance.
The ModelZoo browser¶
The expand icon at the top right corner of the treemap can be used to open up the ModelZoo browser. As the name suggests, this pop-up allows the user to inspect the entire ModelZoo and all information that is contained in it.
The image below shows the overview of the ModelZoo in the ModelZoo browser. This overview is a larger version of the predictor importances described above.
The ModelZoo browser contains much more information than just the importance of the predictors; spread over different parts of the ModelZoo Browser. The next sections discuss them one by one.
The leftmost part of the ModelZoo browser is a navigation section. This section allows the user to intuitively navigate through the ModelZoo to find the specific model of interest.
Clicking "OVERVIEW" serves to return to the overall predictor importances of all the models in the ModelZoo. At the top right part of this navigation section, the download button allows the user to download the entire ModelZoo in JSON format for further use, ex.g. uploading it to a TIM Edge installation.
The rest of this section gives hierarchical overview of the model in the ModelZoo. The way this overview is structured in itself contains information about the ModelZoo: it's different for daily cycle and non-daily cycle ModelZoos.
For a non-daily cycle ModelZoo, the models are listed by their index, as shown in the image below.
For a daily cycle ModelZoo, there is yet another distinction between forecasting and anomaly detection ModelZoos. The models are listed by the hours they apply to. For forecasting ModelZoos, they are further divided over the number of 'days ahead' they are built to forecast to, as shown in the image below.
For anomaly detection ModelZoos, this 'days ahead' level isn't included as it doesn't make sense here: anomaly detection doesn't happen over a forecasting horizon. Therefore, there's only a single day's worth of models in a daily cycle anomaly detection ModelZoo, as shown in the image below.
When one particular model in the ModelZoo is selected, that model's feature importances are visualized. The title above the chart indicates the model's index; in the image below the model's index is 15. At the top right corner of the card, a breadcrumb shows the path towards the current model in the ModelZoo; in the image below the model for 14:00:00 for Day + 1. Again, the overview button allows the user to return to the overall predictor importances of all the models in the ModelZoo.
At the top right of the chart, the user can switch between views. Current supported views are the familiar treemap view - for clearly visualizing ratios and overall variable contribution - and a table view - for an organized view on the terms, their importance and their coefficients, and how importances and coefficients each compare. The image above shows a treemap; the image below shows the same model's table view.
Next to the menu allowing users to switch view, the timescale of the dataset the ModelZoo was built on is displayed - in this case, this is one hour.
At the top right part of the chart zone (right below the breadcrumb if present), an insights button can be found. This button opens the insights tab, which contains additional insights into the different variables usage in the model in question and on the model itself. The information in this tab covers multiple topics: variable offsets, unused variables, and for forecasting models also the data difficulty (overview) and model complexity (model).
The variable offsets section contains information about the offsets of the relevant variables. For each variable, the used offsets are shown on a time axis. On top of improving the insight into the model, the variable offsets used by a model gives the user knowledge of exactly what data should be included if they want to use the same model and apply it to new data (a new dataset version).
Both the scale and relative position to the anchor point of these axes are automatically selected for optimal viewing of the different variable offets; it is however possible for the user to change the scale if preferred.
For forecasting models, the available offsets can be shown next to the used offsets by toggling them on in the Use available offsets switch. This visualizes the differences (if any) between the offsets that were available to the model during training, and the offsets that are actually used by the model.
The unused variables section contains an overview of variables included in the dataset, but not used in this particular model. When on the overview of all the models in the ModelZoo (i.e. no model is selected), this section shows the variables that were not used throughout the entire ModelZoo. In the image below, the unused variables are Load and Temperature.
For forecasting models, the data difficulty is displayed in the insights tab in the ModelZoo Browser Overview
For each forecasting model, the model quality is also displayed in the insights tab of that model in the ModelZoo Browser.