Retail Sales

Problem description

Forecasting sales of a product or service plays an important role in the lifecycle of almost every retail company. There are multiple reasons why it is essential to know the future sales. It can drive plenty of management decisions such as efficient inventory management, preventing or early detection of potential issues, price setting, marketing and other.

Data recommendation template

The dataset can contain various factors that company tracks or estimates which could help forecasting future sales. These could be internal factors such as number of stores, number of employees, level of product/service, product portfolio size or external factors such as economic and industry conditions, rate of inflation and other. Of course, only those should be selected that are relevant for company’s business case and affect sales of the product. The second condition is not so strict – TIM engine can deal with irrelevant predictors, simply they won’t be included in model building. Some of these metrics can also be estimated into the future. For example, company probably knows price of the product or number of stores in a near future.

TIM setup

TIM requires no setup of TIM’s mathematical internals and works well in business user mode. All that is required from a user is to let TIM know forecasting routine and desired prediction horizon. It is often the case that there are some seasonal patterns in product sales time series, usually daily weekly or yearly. These will be detected by TIM engine automatically if the WeekRest, PeriodicComponents and TimeOffsets dictionaries are turned on (by default they are).

Demo using Python API Client

Set up Python Libraries

In [1]:
import logging
import pandas as pd
import plotly as plt
import as px
import plotly.graph_objects as go
import numpy as np
import json

import tim_client

Credentials and logging

(Do not forget to fill in your credentials in the credentials.json file)
In [2]:
with open('credentials.json') as f:
    credentials_json = json.load(f)                     # loading the credentials from credentials.json

TIM_URL = ''          # URL to which the requests are sent

SAVE_JSON = False                                       # if True - JSON requests and responses are saved to JSON_SAVING_FOLDER
JSON_SAVING_FOLDER = 'logs/'                            # folder where the requests and responses are stored

In [3]:
level = logging.getLevelName(LOGGING_LEVEL)
logging.basicConfig(level=level, format='[%(levelname)s] %(asctime)s - %(name)s:%(funcName)s:%(lineno)s - %(message)s')
logger = logging.getLogger(__name__)
In [4]:
credentials = tim_client.Credentials(credentials_json['license_key'], credentials_json['email'], credentials_json['password'], tim_url=TIM_URL)
api_client = tim_client.ApiClient(credentials)

api_client.save_json = SAVE_JSON
api_client.json_saving_folder_path = JSON_SAVING_FOLDER
[INFO] 2020-10-30 10:02:26,483 - tim_client.api_client:save_json:74 - Saving JSONs functionality has been disabled
[INFO] 2020-10-30 10:02:26,486 - tim_client.api_client:json_saving_folder_path:89 - JSON destination folder changed to logs

Specify configuration

To let TIM know that we want to forecast one week ahead we can set the "prediction to" to 7 samples. Model will be built using a range between 2014-01-01 00:00:00 – 2015-12-31 00:00:00. Out-of-sample forecasts are made on the rest - from 2016-01-01 00:00:00 – 2016-07-24 00:00:00. (the last 206 samples). To get better insights from our model we will also want extended importance and prediction intervals to be returned.

In [5]:
configuration_backtest = {
    'usage': {                                 
        'predictionTo': { 
            'baseUnit': 'Sample',                # units that are used for specifying the prediction horizon length (one of 'Day', 'Hour', 'QuarterHour', 'Sample')
            'offset': 7                       # number of units we want to predict into the future (24 hours in this case)
        'backtestLength': 206                 # number of samples that are used for backtesting (note that these samples are excluded from model building period)
    "predictionIntervals": {
        "confidenceLevel": 90                  # confidence level of the prediction intervals (in %)
    'extendedOutputConfiguration': {
        'returnExtendedImportances': True      # flag that specifies if the importances of features are returned in the response

Data description

Dataset used in this example has daily sampling rate and contains data from 2014-01-01 to 2016-07-31.


Target variable represents daily sales of a product.

Predictor candidates

There are two predictor candidates – stock price and product price.

Forecasting scenario

In this example we will simulate a week ahead scenario (7 samples - 7 days). Each day we wish to have forecasts for 7 days ahead starting from the next day. We suppose that the sales of the preceding day is already known. We assume that the last known value of the stock price predictor is from the last day and the last value of the target is from the current day. The value of the product price predictor is known for the entire forecasting horizon.

In [6]:
data = tim_client.load_dataset_from_csv_file('data.csv', sep=',')                                  # loading data from data.csv
data                                                                                               # quick look at the data
date product_sales stock_price product_price
0 2014-01-01 00:00:00 0.0 4972.0 1.29
1 2014-01-02 00:00:00 70.0 4902.0 1.29
2 2014-01-03 00:00:00 59.0 4843.0 1.29
3 2014-01-04 00:00:00 93.0 4750.0 1.29
4 2014-01-05 00:00:00 96.0 4654.0 1.29
... ... ... ... ...
938 2016-07-27 00:00:00 NaN NaN 2.39
939 2016-07-28 00:00:00 NaN NaN 2.39
940 2016-07-29 00:00:00 NaN NaN 2.39
941 2016-07-30 00:00:00 NaN NaN 2.39
942 2016-07-31 00:00:00 NaN NaN 2.39

943 rows × 4 columns


In [7]:
backtest = api_client.prediction_build_model_predict(data, configuration_backtest)                 # running the RTInstantML forecasting using data and defined configuration
backtest.status                                                                                    # status of the job

Visualize backtesting

In [8]:
fig = plt.subplots.make_subplots(rows=1, cols=1, shared_xaxes=True, vertical_spacing=0.02)      # plot initialization

fig.add_trace(go.Scatter(x = data.loc[:, "date"], y=data.loc[:, "product_sales"],
                         name = "target", line=dict(color='black')), row=1, col=1)              # plotting the target variable

fig.add_trace(go.Scatter(x = backtest.prediction.index, 
                         y = backtest.prediction.loc[:, 'Prediction'],
                         name = "production forecast", 
                         line = dict(color='purple')), row=1, col=1)                            # plotting production prediction

fig.add_trace(go.Scatter(x = backtest.prediction_intervals_upper_values.index,
                         y = backtest.prediction_intervals_upper_values.loc[:, 'UpperValues'],
                         marker = dict(color="#444"),
                         line = dict(width=0),
                         showlegend = False), row=1, col=1)                           
fig.add_trace(go.Scatter(x = backtest.prediction_intervals_lower_values.index,
                         y = backtest.prediction_intervals_lower_values.loc[:, 'LowerValues'],
                         fill = 'tonexty',
                         line = dict(width=0),
                         showlegend = False), row=1, col=1)                                     # plotting confidence intervals

fig.add_trace(go.Scatter(x = backtest.aggregated_predictions[0]['values'].index, 
                         y = backtest.aggregated_predictions[0]['values'].loc[:, 'Prediction'],
                         name = "in-sample MAPE: " + str(round(backtest.aggregated_predictions[0]['accuracyMetrics']['MAPE'], 2)), 
                         line=dict(color='goldenrod')), row=1, col=1)                           # plotting in-sample prediction

fig.add_trace(go.Scatter(x = backtest.aggregated_predictions[1]['values'].index, 
                         y = backtest.aggregated_predictions[1]['values'].loc[:, 'Prediction'],
                         name = "out-of-sample MAPE: " + str(round(backtest.aggregated_predictions[1]['accuracyMetrics']['MAPE'], 2)), 
                         line = dict(color='red')), row=1, col=1)                               # plotting out-of-sample-sample prediction

fig.update_layout(height=600, width=1000, 
                  title_text="Backtesting, modelling difficulty: " 
                  + str(round(backtest.data_difficulty, 2)) + "%" )                             # update size and title of the plot