Transfer Learning Techniques: Revolutionizing Time Series Forecasting with Lightning

Transfer learning is an exciting and under-explored technique in the field of time series forecasting. With it, we can achieve lightning-fast predictions with a fraction of the computational cost by pre-training a flexible model on a large dataset and then using it on another dataset with little to no additional training. Transfer learning thus demonstrates the potential to solve one of the most significant limitations of modern machine learning methods: the tradeoff between accuracy and speed.

Recently, large language models like ChatGPT and DALL-E have captured the attention not only of machine learning researchers and practitioners but a wide swathe of the general public as well. These models, trained on massive amounts of text data, have demonstrated the capability to perform a wide range of natural language tasks and generate text that closely resembles something a human may have written. Startups like OpenAI and Stability AI have successfully commercialized these models and made them accessible to a broader audience either as open-source code or as pre-trained artifacts. The success of these models and startups highlights the potential for transfer learning techniques to have a similar impact on the time series forecasting field.

One of the most exciting implications of time-series transfer learning is the ability to make “cold-start” predictions, which refers to creating accurate predictions for series with limited to no history. Examples of this in use are the creation of forecasts for novel products or items with no recorded past. This sort of problem arises in many industries like retail or fashion when new items are launched into the market.  

Another implication is the possibility of extremely low latency predictions. An immediate implication of the reduced optimization time of transfer-learning systems is the possibility of predictions for cases when one can’t continuously train models. An example of this kind of prediction would be the predictions on small computing units that sample sensor data in IOT on a millisecond frequency.

Finally, time series models pre-trained on large datasets can leverage repeated events across series that would otherwise be considered structural breaks or non-stationary behaviors and still provide robust and accurate predictions. Examples of non-stationary behaviors include promotions, non-seasonal calendar effects, or other conditions that could systematically change demand series predictions like pandemics. 

In addition to making pre-trained models publicly available, Nixtla has partnered with Lightning AI to create a demo app that allows data scientists and DevOps engineers to upload their own datasets and make 0-shot predictions. This demo app demonstrates the power of transfer learning techniques in making accurate predictions despite minimal training data. 

Nixtla’s use of the Lightning platform has proven to be highly beneficial in developing AI applications. Nixtla leveraged the power of PyTorch Lightning to train multiple deep-learning models for time series forecasting. With just a few lines of code, Nixtla could easily deploy these models in the lightning cloud by selecting the desired infrastructure (number of CPUs and RAM) and executing one command. As a result, the AI application was ready for user testing in no time. The partnership is contributing to the growth of this innovative field and bringing the benefits of transfer learning to a broader audience.

Transfer Learning Demo App

In collaboration with Lightning AI, a platform for building AI products and deploying models on the cloud without worrying about infrastructure, cost management, and scaling, Nixtla created an app that allows anyone to test the performance of the pre-trained models on their data. The app also allows users to modify the input data to remove outliers and see how the forecast accuracy improves.

This app makes it easy for anyone to take advantage of transfer learning in time series forecasting without worrying about the technical headaches of infrastructure, cost management, and scaling. Without this app, you would have to collect data from different sources to ensure the model has useful patterns to learn, choose the deep learning models to train, and train them efficiently using the right infrastructure (GPUS). This app allows you to test if transfer learning is the right technique for you, before wasting time and money.

You can easily test the pre-trained models by properly uploading your time series dataset. To ensure compatibility with the model, your dataset should have at least two columns: one for the temporal variable identifier and another for the target variable. You can see an example here. With just a few simple steps, you can gain insight into the predictions made by these powerful models.

You can test the forecasting abilities of a pre-trained model by uploading your time series dataset and comparing the predictions made by different models. By simply switching the selected model, you can gain a deeper understanding of the strengths and limitations of different pre-trained models, and choose the one that best fits your needs. This makes it easy to evaluate the performance of various models and make data-driven decisions. Try experimenting with multiple pre-trained models to improve efficiency and reduce costs and compare their forecasting capabilities. With this approach, you can avoid the need for training numerous models and datasets, which can be time-consuming and resource-intensive. By leveraging pre-trained models, you can quickly and easily evaluate various options, identify the best-performing models for your use case, and make informed decisions about your strategy.

The app can also forecast multiple time series at once. By adding an extra column to your dataset that identifies each time series, the app and the chosen model will forecast the full dataset. An example of a dataset with multiple time series can be found here. This allows you to quickly and easily forecast multiple time series with a single dataset, making it a powerful tool for data analysis and forecasting. With this feature, you can forecast different time series in one go, saving you time and effort.

With the collaboration between Nixtla and Lightning AI, anyone can quickly and easily test and improve their forecasts using the power of transfer learning and deep learning.