Moving from classical analysis to machine learning requires a fundamental shift in data representation. Raw time-series data, particularly from financial instruments like STIR futures and commodity spreads, is often non-stationary. Using such data directly in regression models leads to spurious correlations and unreliable performance. The goal is to engineer features that are stationary while preserving the essential information about market dynamics.

Integer differentiation, such as taking first differences to calculate returns, is a blunt instrument. While it achieves stationarity, it erases the series' memory, discarding valuable information about long-term trends. Fractional differentiation offers a more nuanced approach. By applying a fractional differencing operator, we can remove just enough of the trend to achieve stationarity while retaining the maximum possible memory of the original series. This is particularly effective for commodity spread data, where mean-reversion dynamics are critical but can be obscured by shocks and regime shifts.

Applying this to raw CQG TFX data, for example, allows the transformation of price series for individual futures contracts into stationary series that retain the memory of their term structure relationships. The result is a feature set that is statistically robust without sacrificing the economic intuition of carry and roll yield.

Beyond price, market microstructure data provides a higher-frequency view of supply and demand. Order flow imbalance (OFI) is a potent predictive feature derived from market depth metrics. By aggregating the net changes in limit orders at the best bid and ask, OFI captures the immediate pressure on price. Unlike simple volume, it provides a directional signal, indicating whether buyers or sellers are more aggressive.

To build a cohesive multi-asset strategy, these engineered features must be structured systematically. A tensor is the ideal data structure for this task. For a portfolio of STIR futures, we can construct a 3D tensor with dimensions representing time, contract maturity, and feature type. This allows the model to learn not just from the time-series of a single contract, but from the cross-sectional relationships across the entire yield curve.

The term structure curve itself contains predictive information. Changes in its shape—steepening, flattening, or inversions—reflect shifts in market expectations. To make this information digestible for a model, the curve dynamics must be normalised. A common approach is to decompose the curve's movements using Principal Component Analysis (PCA).

Typically, the first three principal components capture the vast majority of the curve's variance and correspond to intuitive economic factors:

• PC1 (Level): A parallel shift up or down in the entire curve.
• PC2 (Slope): A steepening or flattening of the curve.
• PC3 (Curvature): A change in the convexity or 'hump' of the curve.

By using the time-series of these component scores as features, we feed the model a low-dimensional, decorrelated, and economically meaningful representation of the term structure. These normalised inputs are far more stable for training than using the raw yields of dozens of individual contracts.

This architectural approach ensures that raw data is systematically transformed into a feature set that is stationary, information-rich, and structured for advanced model training. The resulting normalised tensors retain the economic logic of concepts like carry and basis while being statistically sound for machine learning applications.

With these techniques, raw market data transitions from a noisy, non-stationary collection of series into a structured, predictive input ready for sophisticated AI models.