Financial markets generate enormous amounts of data every second. Prices change, volumes fluctuate, volatility expands and contracts, and relationships between assets evolve over time. For a data scientist, this makes financial data fascinating — but also unusually difficult to model.
Unlike a typical tabular dataset, financial observations are ordered in time, strongly dependent on market conditions, and affected by costs that may not appear in the raw price history. In foreign exchange, for example, a model evaluating positions held for several days may need to include overnight financing alongside spreads and commissions. Researchers working with FX data can use this tool to estimate the swap component before incorporating it into a more realistic strategy model.
That distinction matters. A model can predict prices reasonably well and still produce poor practical results if its assumptions about execution and holding costs are unrealistic.
This guide introduces the foundations of financial time series analysis, explains the most important statistical challenges, and shows how data scientists can build more reliable workflows for market research.
What Is a Financial Time Series?
A time series is simply a sequence of observations recorded in chronological order.
In finance, common examples include:
- stock prices;
- currency exchange rates;
- bond yields;
- cryptocurrency prices;
- trading volume;
- market volatility;
- interest rates;
- commodity prices.
At first glance, this looks similar to many other machine-learning datasets. The important difference is that observations cannot usually be treated as independent.
Today’s EUR/USD price, for example, is closely connected to yesterday’s price. Volatility during one hour may also influence volatility during the next.
This dependence changes how the data should be transformed, split and validated.
In financial modelling, time is not just another feature. It determines what information could realistically have been known when a prediction was made.
Ignoring that principle is one of the fastest ways to build a backtest that looks impressive but could never have existed in the real world.
Price Levels vs. Returns
One of the first decisions in financial time series analysis is whether to study prices directly or transform them into returns.
Suppose a currency pair moves from 1.1000 to 1.1110.
The simple return is:
Return = (New Price − Old Price) / Old Price
In this case:
(1.1110 − 1.1000) / 1.1000 = 1%
Analysts also frequently use logarithmic returns:
Log Return = ln(New Price / Old Price)
Why transform prices at all?
Raw financial prices often contain trends and are non-stationary. Returns usually behave more consistently over time and are therefore better suited to many statistical models.
| Data Type | Typical Use | Main Limitation |
| Price level | Charting, trend analysis | Often non-stationary |
| Simple return | Performance analysis | Less convenient for aggregation |
| Log return | Statistical modelling | Slightly less intuitive |
| Trading volume | Liquidity analysis | Market-dependent interpretation |
| Volatility | Risk and regime analysis | Must be estimated |
| Spread/cost data | Execution modelling | Can vary significantly over time |
The correct representation depends on the research question.
Why Stationarity Matters
Many classical statistical methods assume that the properties of a time series remain reasonably stable.
This idea is called stationarity.
A stationary process has statistical characteristics such as mean and variance that do not systematically drift over time.
Financial price levels usually violate this assumption.
Consider a stock that traded at $20 ten years ago and now trades at $150. Calculating one long-term average price is not particularly meaningful.
Returns are often closer to stationary, although even they can exhibit changing volatility and structural breaks.
Data scientists commonly investigate stationarity using:
- visual inspection;
- rolling mean and variance;
- autocorrelation plots;
- Augmented Dickey-Fuller tests;
- KPSS tests.
These tests are useful, but they should not replace economic reasoning. Financial markets can change regimes even when a statistical test suggests that a historical series is stationary.
Volatility Is Rarely Constant
One of the most distinctive properties of financial data is volatility clustering.
Large price movements tend to be followed by periods with more large movements. Quiet markets likewise often remain quiet for a while.
Imagine daily returns such as:
0.2%, -0.1%, 0.3%, 0.1%, -0.2%
followed by:
-3.1%, 2.4%, -2.7%, 4.0%, -1.9%
The average return might not change dramatically, but the risk environment clearly has.
This phenomenon is important because a model trained during a calm period may behave very differently during a crisis.
Common methods for studying volatility include:
- rolling standard deviation;
- exponentially weighted volatility;
- Average True Range;
- ARCH and GARCH models;
- implied volatility;
- machine-learning volatility models.
For many practical projects, a simple rolling volatility measure is a useful starting point before moving to more complex approaches.
Autocorrelation: Does the Past Predict the Future?
Autocorrelation measures the relationship between a time series and its own previous values.
If today’s return strongly predicted tomorrow’s return, trading would be comparatively easy.
In highly liquid markets, raw returns often show weak linear autocorrelation. However, other transformations may display stronger persistence.
For example:
- absolute returns may be autocorrelated;
- squared returns often show volatility clustering;
- trading volume can exhibit persistence;
- spreads can vary systematically by time of day.
This illustrates an important principle in financial data science:
The strongest information may not exist in the raw price direction itself.
Useful signals may instead appear in volatility, liquidity, market structure, correlations or conditional relationships.
A Basic Python Workflow
Python has become one of the most popular environments for financial research because its ecosystem includes tools for data manipulation, statistics, visualisation and machine learning.
A basic workflow might look like this:
import pandas as pd
import numpy as np
df = pd.read_csv(“market_data.csv”)
df[“return”] = df[“close”].pct_change()
df[“log_return”] = np.log(df[“close”] / df[“close”].shift(1))
df[“volatility_20”] = df[“log_return”].rolling(20).std()
df = df.dropna()
This produces only a few variables, but they already allow several useful questions to be explored:
- Are returns normally distributed?
- Does volatility cluster?
- Do high-volatility periods produce different returns?
- Are extreme moves becoming more frequent?
- Does behaviour change across trading sessions?
More sophisticated modelling should usually come after exploratory analysis, not before it.
The Biggest Trap: Data Leakage
Financial machine-learning projects are extremely vulnerable to data leakage.
Leakage occurs when a model receives information that would not have been available at prediction time.
Suppose you calculate a technical indicator using data from 10:00 through 11:00 and then use that indicator to simulate a trade placed at 10:30.
The model knows part of the future.
Another example occurs when preprocessing is performed on the complete dataset before it is divided into training and test periods.
If a scaler learns the mean and standard deviation from 2015–2026 and the model is then tested on 2020, information from future observations has indirectly entered the pipeline.
The correct order is generally:
- Split data chronologically.
- Fit transformations using the training period.
- Apply those transformations to later periods.
- Train the model.
- Evaluate it on genuinely unseen observations.
Small mistakes at this stage can produce enormous differences in apparent model performance.
Never Randomly Shuffle Financial Time Series
Random train-test splitting is standard practice for many machine-learning problems.
For market data, it can be misleading.
Suppose observations from 2022, 2024 and 2026 appear in the training sample while observations from 2023 and 2025 appear in the test set.
That is not how a real forecasting system operates.
In production, the relationship is always:
Past → Present → Future
Validation should preserve the same structure.
A basic chronological split might use:
- 2018–2022 for training;
- 2023–2024 for validation;
- 2025–2026 for testing.
An even better approach for many applications is walk-forward validation.
The model is repeatedly trained using historical data and evaluated on the period immediately following it.
This makes it possible to observe how performance changes through different market regimes.
Feature Engineering for Financial Data
Good features should represent plausible relationships rather than arbitrary mathematical transformations.
Useful categories include:
Momentum Features
Examples:
- 1-day return;
- 5-day return;
- 20-day return;
- distance from a moving average;
- rate of change.
Volatility Features
Examples:
- rolling standard deviation;
- average true range;
- intraday range;
- realised volatility.
Market Regime Features
These can include:
- trend vs. range conditions;
- high vs. low volatility;
- risk-on vs. risk-off periods.
Calendar Features
Market behaviour can vary by:
- hour;
- day of week;
- trading session;
- month;
- proximity to major announcements.
Cross-Asset Features
A currency model, for example, may benefit from information about:
- bond yields;
- equity indices;
- commodities;
- related currency pairs.
The goal is not to create as many variables as possible. It is to create features with a defensible reason for containing information.
Why Trading Costs Belong in the Dataset
A common mistake in financial research is evaluating predictions independently from the costs of acting on them.
Consider a model that generates hundreds of trades each month.
A backtest based only on midpoint prices might look profitable. In reality, every trade can involve costs.
Depending on the market and instrument, these may include:
- bid-ask spread;
- brokerage commission;
- slippage;
- exchange fees;
- overnight financing;
- borrowing costs.
Imagine two hypothetical strategies:
| Metric | Strategy A | Strategy B |
| Gross return per trade | 0.10% | 0.05% |
| Estimated transaction cost | 0.02% | 0.04% |
| Net return per trade | 0.08% | 0.01% |
| Trades per month | 20 | 200 |
Strategy B may appear attractive when costs are ignored, yet only a tiny expected margin remains after execution.
For longer-duration leveraged FX positions, overnight financing introduces another variable because carrying a position across rollover can create either a charge or a credit depending on the instrument and trading conditions.
This is why financial time series analysis should eventually move beyond price prediction toward decision modelling.
The more useful question is not:
“Will the price rise?”
It is:
“Is the expected move large enough to justify taking the position after realistic costs and uncertainty?”
Machine Learning Does Not Eliminate Market Noise
Financial datasets are attractive candidates for machine learning because they are large and continuously generated.
Unfortunately, more data does not automatically mean more predictable data.
Researchers may experiment with:
- logistic regression;
- random forests;
- gradient boosting;
- support vector machines;
- recurrent neural networks;
- LSTMs;
- transformers.
But model complexity should not be confused with predictive power.
A sophisticated neural network can overfit noise just as easily as a simpler model. In some cases, a transparent logistic regression baseline may be more valuable because its behaviour is easier to inspect.
Before adopting a complex model, compare it against simple baselines.
If the neural network cannot meaningfully outperform a naive benchmark out of sample, its additional complexity may not be justified.
Measure More Than Prediction Accuracy
Imagine a directional model with 55% accuracy.
Is it good?
There is no way to know from accuracy alone.
The model could correctly predict many tiny movements while missing the largest ones.
For trading-oriented financial analysis, researchers may also examine:
- average return per signal;
- profit factor;
- maximum drawdown;
- volatility of returns;
- Sharpe ratio;
- turnover;
- transaction costs;
- performance by market regime;
- stability over time.
A useful model should ideally remain reasonably robust when assumptions change.
If changing a lookback period from 20 to 21 days destroys all apparent performance, the result deserves scrutiny.
Watch for Regime Changes
Markets are not static systems.
Inflation changes. Central banks alter interest rates. Liquidity disappears and returns. Regulations evolve. Market participants adopt new technology.
A relationship discovered in one period may disappear in another.
This is known as concept drift or regime change.
A production model should therefore be monitored rather than deployed and forgotten.
Useful monitoring metrics include:
- prediction distributions;
- feature distributions;
- realised volatility;
- model error;
- trading frequency;
- expected vs. realised outcomes.
Significant deviations may indicate that the data-generating process has changed.
A Practical Financial Time Series Checklist
Before trusting the results of a financial model, ask:
- Is the data correctly ordered in time?
- Are missing observations handled consistently?
- Are features calculated without future information?
- Is the training/test split chronological?
- Have returns been considered instead of only raw prices?
- Has changing volatility been examined?
- Are results stable across different market periods?
- Are spreads, commissions and other relevant costs included?
- Has the strategy been tested on genuinely unseen data?
- Does the model outperform a simple baseline?
- Is there a plausible explanation for the relationship being modelled?
If several answers are “no,” improving the research design is usually more valuable than adding a more sophisticated algorithm.
Final Thoughts
Financial time series analysis sits at an interesting intersection of statistics, machine learning and real-world decision making.
The technical tools are increasingly accessible. A few lines of Python can download data, calculate returns, train a model and produce an attractive performance chart.
The difficult part is determining whether that chart represents a genuine relationship or an artefact of leakage, overfitting, changing market regimes or unrealistic assumptions.
Good financial data science therefore begins with disciplined methodology.
Understand how the observations were generated. Preserve the direction of time. Test models on unseen periods. Examine volatility and structural changes. And when the analysis is intended to represent an executable trading strategy, model the costs of acting on predictions rather than treating them as an afterthought.
In financial markets, the most complicated model is rarely the most important advantage. A realistic dataset and a well-designed experiment are usually far more valuable.