Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Articles by Nikos Kafritsas
Timer-XL: A Long-Context Foundation Model for Time-Series Forecasting
has been a shift in how foundation models operate. After establishing that pretraining a large deep learning model on a vast corpus of temporal data grants generalizable properties, pretrained TS models now aim to be even more versatile. For time series, this means supporting exogenous variables and allowing variable context and prediction lengths. This article discusses Timer-XL[1], an upgraded time series model based on Timer[2].
Toto: A Foundation Time-Series Model Optimized for Observability Data
Datadog open-sourced their pretrained time series model! Foundation time-series models keep pushing the frontier. Recently, Datadog Research open-sourced Toto[2] — Time Series Optimized Transformer for Observability, a model purpose-built for observability data. Toto is currently the most extensively pretrained time-series foundation model: The pretraining corpus contains 2.36 trillion tokens, with ~70% coming from Datadog’s private telemetry dataset.
Timer-XL: Long-Context Foundation Model for Time-Series Forecasting
Lately, there has been a shift in how foundation models operate. After establishing that pretraining a large Deep Learning model on a vast corpus of temporal data grants generalizable properties, pretrained TS models now aim to be even more versatile. For time series, this means supporting exogenous variables and allowing variable context and prediction lengths. This article discusses Timer-XL[1], an upgraded time series model based on Timer[2].
Retail Forecasting with IBM’s Tiny Time Mixers (TTM)
Putting the popular foundation model to the test! Foundation models are already reshaping time-series forecasting. Take the VN1 retail forecasting competition for example: Nixtla’s TimeGPT ranked 2nd using zero-shot forecasting — no training, ensembling, or postprocessing (e.g., no manual forecasting of zero-sales products). Similarly, finetuned MOIRAI-base achieved 1st place in the same competition! This shows strong potential for foundation models — when used correctly.
Influential Time-Series Forecasting Papers of 2023-2024: Part 2
Member-only story Nikos Kafritsas · Follow Published in The Forecaster · 12 min read · -- The 2nd part of innovations in time-series forecasting I hope your 2025 is off to a great start! This is Part 2 of my roundup of the most notable time-series forecasting papers from 2023–2024. In this second part, I’ll also discuss impactful papers that bring key innovations from other areas of machine learning into time-series forecasting.
Influential Time-Series Forecasting Papers of 2023-2024: Part 1
Exploring the latest advancements in time series Let’s kick off 2025 with a roundup of notable time-series forecasting papers. I’ve also included some key papers from 2023 as well. 2024 was the year of foundation forecasting models — but I’ll cover those in a separate article.
TSMixer: Google’s Innovative Deep Learning Forecasting Model
Combining lightweight design with high forecasting accuracy Many DL forecasting models tend to follow AI trends. But not TSMixer. Released by Google almost a year ago, it has gained significant traction and is widely used as an alternative to traditional ML-based models, thanks to these advantages: Lightweight: Its MLP-based architecture efficiently captures patterns in both time and feature dimensions.
Context-Aided Forecasting: Enhancing Forecasting with Textual Data
A promising alternative approach to improve forecasting The use of textual data to enhance forecasting performance isn’t new. In financial markets, text data and economic news often play a critical role in producing accurate forecasts — sometimes even more so than numeric historical data. Recently, many large language models (LLMs) have been fine-tuned on Fedspeak and news sentiment analysis. These models rely solely on text data to estimate market sentiment.
Context-Aided Forecasting: Enhancing Forecasting with Textual Data
The use of textual data to enhance forecasting performance isn’t new. In financial markets, text data and economic news often play a critical role in producing accurate forecasts – sometimes even more so than numeric historical data. Recently, many large language models (LLMs) have been fine-tuned on Fedspeak and news sentiment analysis. These models rely solely on text data to estimate market sentiment.
MOIRAI-MOE: Upgrading MOIRAI with Mixture-of-Experts for Enhanced Forecasting
The popular foundation time-series model just got an update The race to build the Top foundation forecasting model is on! Salesforce’s MOIRAI, one of the early foundation models, achieved high benchmark results and was open-sourced along with its pretraining dataset, LOTSA. We extensively analyzed how MOIRAI works here — and built an end-to-end project comparing MOIRAI with popular statistical models.
TIME-MOE: Billion-Scale Time Series Foundation Model with Mixture-of-Experts
And open-source as well! The Mixture-of-Experts (MOE) architecture has surged in popularity with the rise of large language models (LLMs). As time-series models adopt cutting-edge techniques, Mixture-of-Experts has naturally found its place in the time-series foundation space. This article discusses Time-MOE, a time-series foundation model that uses MOE to improve forecasting accuracy while reducing computational costs.
Qwen2-VL: Hands-On Guides for Invoice Data Extraction, Video Chatting, and Mutlimodal RAG with PDFs
How to use a top, open-source, vision language model The surge of new LLMs and vLLMs is unstoppable. One notable model is Qwen2-VL — a Vision LM with advanced image and video understanding. Key features of Qwen2-VL: Powerful architecture: Released by Alibaba, it’s built on the Qwen2 LM. Object and text recognition: Qwen2-VL recognizes complex object relationships, handwritten text, and multiple languages.
Forecasting with NHiTs: Uniting Deep Learning + Signal Processing Theory for Superior Accuracy
A high-performance DL model for all forecasting cases NHiTS was published 2 years ago — and has gained significant attention in the forecasting community since then. First, it’s a versatile model — accepting past observations, future known inputs, and static exogenous variables. It can be applied across various forecasting domains, including energy demand, retail, and financial markets. It’s lightweight, yet high-performance.
VisionTS: Building Superior Forecasting Models from Images
Leveraging the power of images for time-series forecasting Which is the biggest challenge when building a pretrained time-series model? Answer: Finding high-quality, diverse time-series data. We’ve discussed this in previous articles. There are 2 main approaches to building a foundation forecasting model: “Bootstrap” an LLM: Repurpose a pretrained LLM like GPT-4 or Llama by applying fine-tuning or tokenization strategies tailored for time-series tasks.
MOIRAI: Zero-Shot Forecasting Without Training
How can a model forecast time series data without being trained? How can a model forecast time series data without being trained? At first, it seems counter-intuitive. However, the recent foundation models have integrated many techniques from the LLM literature. Most importantly, scaling laws seem to apply to time series as well. Given more data, training time, and parameter size, these models perform better. We discussed how these models work in our previous article.
Will Transformers Revolutionize Time-Series Forecasting?
At the heart of the "Generative AI revolution" is the Transformer model - introduced by Google in 2017. But every tech revolution creates confusion. In a rapid growth environment, it's difficult to assess innovations unbiasedly - let alone estimate their impact. Zealous adopters: They use Transformers everywhere, including outside of NLP. They use them even if they can't or don't want to - they are forced by their employers, managers, investors, etc.
Exploring the Latest Advances in Foundation Time-series Models
Fast and accurate forecasting of new data — without training Recent advances in foundation time-series models are groundbreaking. TimeGPT was the first native foundation model. It was released this past August and shook the forecasting community. Since then, numerous other foundation models have been released, including: TimesFM MOIRAI Tiny Time Mixers (TTM) MOMENT We’ve covered these models in previous articles, but they’ve had many updates since their initial release.
Tiny Time Mixers (TTM): A Powerful Zero-Shot Forecasting Model by IBM
Tiny Time Mixers (TTM): A Powerful Zero-Shot Forecasting Model by IBM A new lightweight open-source foundation model Created by author using DALLE*3 If you follow the latest research on LLMs, you will notice two main approaches: First, researchers focus on building the largest models possible. Pretraining on next-word prediction is crucial for enhancing performance (and where the millions…
MOMENT: A Foundation Model for Time Series Forecasting, Classification, Anomaly Detection
MOMENT: A Foundation Model for Time Series Forecasting, Classification, Anomaly Detection A unified model that covers multiple time-series tasks Created by author using DALLE*3 Foundation models have ignited the applications of LLMs in time series. In the past few months, we saw the release of novel forecasting models like TimesFM, TimeGPT, and of course Salesforce’s…
MOMENT: A Foundation Model for Time Series Forecasting, Classification, Anomaly Detection
MOMENT: A Foundation Model for Time Series Forecasting, Classification, Anomaly Detection A unified model that covers multiple time-series tasks Created by author using DALLE*3 Foundation models have ignited the applications of LLMs in time series. In the past few months, we saw the release of novel forecasting models like TimesFM, TimeGPT, and of course Salesforce’s…
MOIRAI: Salesforce’s Foundation Model for Time-Series Forecasting
Code, model weights, and data will be released soon Time series foundation models are finally taking off! The previous articles explored 2 promising foundation forecasting models, TimeGPT and TimesFM. This article will explore MOIRAI [1], a groundbreaking TS foundation model by Salesforce.
TimesFM: Google’s Foundation Model For Time-Series Forecasting
A new age for time series Google just entered the race of foundation models for time-series forecasting. In August 2023, the time-series community was disrupted by the release of TimeGPT, Nixtla’s first foundation model for time series forecasting. Following TimeGPT, multiple foundation forecasting models were released, but there was one that stood out. Recently, Google unveiled TimesFM[1], a groundbreaking time-series model with phenomenal results.
TimesFM: Google’s Foundation Model For Time-Series Forecasting
A new age for time series Google just entered the race of foundation models for time-series forecasting. In August 2023, the time-series community was disrupted by the release of TimeGPT, Nixtla’s first foundation model for time series forecasting. Following TimeGPT, multiple foundation forecasting models were released, but there was one that stood out. Recently, Google unveiled TimesFM[1], a groundbreaking time-series model with phenomenal results.
AutoGluon-TimeSeries: Every Time Series Forecasting Model In One Library
A powerful library by Amazon — coding example included The open-source landscape for time series is heating up. This includes successful libraries such as Darts, GluonTS, and Nixtla. Last year, Amazon built an extension of its AutoGluon library, which focuses on time-series — known as AutoGluon-Timeseries(AG-TS)[1]. AG-TS leverages the expertise of other libraries: From Amazon itself (AutoGluon and GluonTS). From Nixtla (StatsForecast and MLForecast).
ToolFormer: Guiding AI Models To Use External Tools
Meta’s LLM teaches itself to call External APIs Now that the dust has settled, the weaknesses of LLMs are known. Even the powerful GPT-4 struggles with math operations. Also, the training cut-off time is an inherent weakness of every LLM. They struggle to answer queries on new things. A loose fix is to use external Plugins (e.g. ChatGPT plugins). Still, the user has to manually specify some actions, and these plugins are sometimes unreliable.
Using ChatGPT to Decode Stock Price Movements
Created with Stable Diffusion [1] New Large Language Models (LLMs) are uncharted territory in financial economics. Can new AI models provide a competitive edge? Of course, Language models in financial applications are not new. A notable example is FinBERT, a Transformer model based on Google's BERT architecture. However, with the recent breakthroughs of Generative Pretrained Transformers (like ChatGPT), the ability of LLMs and their impact on various fields should be discussed and re-evaluated.
Time-Series Forecasting: Deep Learning vs Statistics - Who Wins?
A comprehensive guide on the ultimate dilemma In recent years, Deep Learning has made remarkable progress in the field of NLP. Time series, also sequential in nature, raise the question: what happens if we bring the full power of pretrained transformers to time-series forecasting? However, some papers, such as [2] and [3] have scrutinized Deep Learning models. These papers do not present the full picture.
Time-Series Forecasting: Deep Learning vs Statistics
Created with Stable Diffusion [1] In recent years, Deep Learning has made remarkable progress in the field of NLP. Time series, also sequential in nature, raise the question: what happens if we bring the full power of pretrained transformers to time-series forecasting? However, some papers, such as [2] and [3] have scrutinized Deep Learning models. These papers do not present the full picture.
Deep GPVAR: Upgrading DeepAR For Multi-Dimensional Forecasting
Amazon’s new Time-Series Forecasting model What is the most enjoyable thing when you read a new paper? For me, this is the following: Imagine a popular model suddenly getting upgraded — with just a few elegant tweaks. Three years after DeepAR[1], Amazon engineers published its revamped version, known as Deep GPVAR [2] (Deep Gaussian-Process Vector Auto-regressive). This is a much-improved model of the original version. Plus, it is open-source.
Upgrading DeepAR For Multi-Dimensional Forecasting
Amazon's new Time-Series Forecasting model What is the most enjoyable thing when you read a new paper? For me, this is the following: Imagine a popular model suddenly getting upgraded - with just a few elegant tweaks. This is a much-improved model of the original version. Plus, it is open-source. In this article, we discuss: How Deep GPVAR works in depth. How DeepAR and Deep GPVAR are different. What problems does Deep GPVAR solve and why it's better than DeepAR?
An Essential Guide & Applications in Time Series Forecasting
What are copula functions and why do we need them Copulas are functions that model the dependency among several distributions. They are mostly used in financial applications like portfolio risk assessment and hedge fund management. They came to prominence in 2008, when it was discovered that quantitative scientists had misused copulas in their calculations, failing to predict significant events. Nonetheless, copulas are still great mathematical tools.
An Essential Guide & Applications in Time Series Forecasting
What are copula functions and why do we need them Copulas are functions that model the dependency among several distributions. They are mostly used in financial applications like portfolio risk assessment and hedge fund management. They came to prominence in 2008, when it was discovered that quantitative scientists had misused copulas in their calculations, failing to predict significant events. Nonetheless, copulas are still great mathematical tools.
N-BEATS : Time-Series Forecasting with Neural Basis Expansion
A Deep Learning model for zero-shot time-series forecasting There’s one thing that makes Time-Series Forecasting special. It was the only area of Data Science where Deep Learning and Transformers didn’t decisively outperform the other models. Let’s use the prestigious Makridakis M-competitions as a benchmark — a series of large-scale challenges that showcase the latest advances in the time-series forecasting area.
Time-Series Forecasting with Neural Basis Expansion
It was the only area of Data Science where Deep Learning and Transformers didn't decisively outperform the other models. Let's use the prestigious Makridakis M-competitions as a benchmark - a series of large-scale challenges that showcase the latest advances in the time-series forecasting area. In the fourth iteration of the competition, known as M4, the winning solution was ES-RNN [2], a hybrid LSTM & Exponential Smoothing model developed by Uber.
DeepAR: Mastering Time-Series Forecasting with Deep Learning
Amazon’s autoregressive deep network A few years ago, time-series models worked on a single sequence only. Hence, if we had multiple time-series, one option was to create one model per sequence. Or, if we could “tabularize” our data, we could apply the gradient-boosted tree models — which work very well even today. The first model that could natively work on multiple time-series was DeepAR[2], an autoregressive recurrent network developed by Amazon.
Mastering Time-Series Forecasting with Deep Learning
Hence, if we had multiple time-series, one option was to create one model per sequence. Or, if we could "tabularize" our data, we could apply the gradient-boosted tree models - which work very well even today. The first model that could natively work on multiple time-series was DeepAR[2], an autoregressive recurrent network developed by Amazon. In this article, we will see how DeepAR works in-depth and why it is a milestone for the time-series community.
Temporal Fusion Transformer: Time Series Forecasting with Deep Learning - Complete Tutorial
Create accurate and interpretable predictions According to [2],Temporal Fusion Transformer outperforms all prominent Deep Learning models for time series forecasting. Including a featured Gradient Boosting Tree model for tabular time series data. But what is Temporal Fusion Transformer (TFT)[3] and why is it so interesting? In this article, we briefly explain the novelties of Temporal Fusion Transformer and build an end-to-end project on Energy Demand Forecasting.
Temporal Fusion Transformer: Time Series Forecasting with Deep Learning
According to [2],Temporal Fusion Transformer outperforms all prominent Deep Learning models for time series forecasting. Including a featured Gradient Boosting Tree model for tabular time series data. But what is Temporal Fusion Transformer (TFT)[3] and why is it so interesting? In this article, we briefly explain the novelties of Temporal Fusion Transformer and build an end-to-end project on Energy Demand Forecasting. Specifically, we will cover: How to prepare our data for the TFT format.
Temporal Fusion Transformer: Time Series Forecasting with Deep Learning
According to [2],Temporal Fusion Transformer outperforms all prominent Deep Learning models for time series forecasting. Including a featured Gradient Boosting Tree model for tabular time series data. But what is Temporal Fusion Transformer (TFT)[3] and why is it so interesting? In this article, we briefly explain the novelties of Temporal Fusion Transformer and build an end-to-end project on Energy Demand Forecasting. Specifically, we will cover: How to prepare our data for the TFT format.
Whisper: Transcribe & Translate Audio Files With Human-Level Performance
An AI model compatible with 97 languages — and how to use it It is now clear that the general direction of Deep Learning research has fundamentally changed. A few years ago, the modus operandi of most innovative papers was this: Select a dataset X, build and train a novel model on that dataset, and prove your model was the best by reporting SOTA results. But what happens if we test the model on dataset Y? For example: Train a powerful ResNet on the ImageNet dataset.
Transcribe & Translate Audio Files With Human-Level Performance
A few years ago, the modus operandi of most innovative papers was this: Select a dataset X, build and train a novel model on that dataset, and prove your model was the best by reporting SOTA results. But what happens if we test the model on dataset Y? For example: Train a powerful ResNet on the ImageNet dataset. The model will excel in classifying cats and dogs, but it will have trouble classifying images such as cartoon dogs, hand-drawn dogs, or Van Gogh-style dogs.
CLIP: The Most Influential AI Model From OpenAI - And How To Use It
Find out how the model works — coding example included What do the recent AI breakthroughs, DALLE[1] and Stable Diffusion[2] have in common? They both use components of CLIP’s[3] architecture. Hence, if you want to grasp how those models work, understanding CLIP is a prerequisite. Besides, CLIP has been used to index photos on Unsplash. But what does CLIP do, and why it’s a milestone for the AI community?
The Most Influential AI Model From OpenAI- And How To Use It
What do the recent AI breakthroughs, and Stable Diffusion[2] have in common? They both use components of architecture. Hence, if you want to grasp how those models work, understanding CLIP is a prerequisite. Besides, CLIP has been used to index photos on Unsplash. But what does CLIP do, and why it's a milestone for the AI community? Let's dive in! Let's break down this description: Open Source: The model is created and open-sourced by OpenAI. We will later see a programming tutorial on how to use it.
Build a Named Entity Recognition App with Streamlit
From building the app to deployment — with code included In my previous article, we fine-tuned a Named Entity Recognition (NER) model, trained on the wnut_17[1] dataset. In this article, we show step-by-step how to integrate this model with Streamlit and deploy it using HugginFace Spaces. The goal of this app is to tag input sentences per user request in real time. Let’s dive in! Streamlit is an easy-to-use tool for creating interactive applications that sit on top of a data science project.
Named Entity Recognition with Deep Learning (BERT) - The Essential Guide
From data preparation to model training for NER tasks — and how to tag your own sentences Nowadays, NLP has become synonymous with Deep Learning. But, Deep Learning is not the ‘magic bullet’ for every NLP task. For example, in sentence classification tasks, a simple linear classifier could work reasonably well. Especially if you have a small training dataset. However, some NLP tasks flourish with Deep Learning.
Data Science: Write Robust Python With Static Typing
Updated for Python 3.10 There are two types of programming languages: Statically typed and dynamically typed languages. Python is a dynamically typed language. You don’t have to explicitly specify the data type of variables. The same is true for functions: You don’t have to specify the type of arguments or the function’s return type. In other words, there is no code compilation — the Python interpreter performs type checking at runtime — during code execution.
Block-Recurrent Transformer: LSTM and Transformer Combined
A powerful model that combines the best of both worlds The vanilla Transformer is no longer the all-mighty model that tackles any case in Deep Learning. In a previous article, we proved that for time series forecasting tasks, Transformers were struggling. That’s why Google created a hybrid Transformer-LSTM model that achieves SOTA results in time series forecasting tasks. After the hype ended, researchers started focusing on the shortcomings of Transformers.
Analyze Your Referred Memberships With This Maths Formula
By Nikos Kafritsas Works 100% Photo by Liviu C. on Unsplash Referred memberships are a significant boost for Medium writers. Many writers adapt their writing strategy to focus on referred memberships: They prioritize quantity and write shorter posts. This strategy supposedly increases your total reach, leading to more external views. Having more external views...
Become a Master Writer With 15 Powerful Writing Tips
By Nikos Kafritsas Photo by Lukas Blazek on Unsplash Fifteen years ago, I was studying for my advanced English certification exam. English is not my native language. The trickiest part of the exam was the essay: You had to writeat least 400 words in 30 minutes! It was impossible! One way to reach the word count quota was 'catchy phrases' - a lot. Also, you scored extra points for using fancy words...
Installing PyTorch on Apple M1 chip with GPU Acceleration
It finally arrived! The trajectory of Deep Learning support for the MacOS community has been amazing so far. Starting with the M1 devices, Apple introduced a built-in graphics processor that enables GPU acceleration. Hence, M1 Macbooks became suitable for deep learning tasks. No more Colab for MacOS data scientists! Next on the agenda was compatibility with the popular ML frameworks. Tensorflow was the first framework to become available in Apple Silicon devices.
Show More
loading
Actions
Is this you?
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.Get in touch with Nikos
Contact Nikos, search articles and posts on X, monitor coverage, and track replies from one place.
Learn more about Muck Rack