/signal-classification

name: signal-classification
description: ML trading signal classifiers using XGBoost and LightGBM with walk-forward validation, SHAP feature importance, and threshold optimization

Signal Classification

Predict whether an asset's price will move up or down over a forward horizon using supervised machine learning classifiers. This skill covers the full pipeline: label creation, model training, walk-forward validation, feature importance analysis, and threshold optimization for trading applications.

Why Tree-Based Models Dominate Trading ML

XGBoost and LightGBM are the workhorses of quantitative trading ML for good reason:

• **Non-linear relationships**: Financial features interact in complex, non-linear ways that trees capture naturally
• **Robust to feature scale**: No need to normalize or standardize inputs — trees split on rank order
• **Built-in feature importance**: Understand which features drive predictions without separate analysis
• **Fast training and inference**: Train on thousands of samples in seconds, predict in microseconds
• **Handle missing values**: Native support for NaN without imputation hacks
• **Regularization built in**: max_depth, min_child_weight, subsample all prevent overfitting

Linear models and deep learning have their place, but for tabular trading features with fewer than 100k samples, gradient-boosted trees consistently outperform alternatives.

Classification Types

Binary Classification

The simplest and most common setup. Predict whether forward returns exceed a threshold:

• **Up signal**: forward return > +1%
• **Down signal**: forward return < -1%
• **Neutral (excluded)**: -1% to +1% — drop these from training to create cleaner labels

import numpy as np

def create_binary_labels(
    prices: np.ndarray, horizon: int = 24, threshold: float = 0.01
) -> np.ndarray:
    """Create binary labels from forward returns.

    Args:
        prices: Array of prices.
        horizon: Forward return lookback in bars.
        threshold: Minimum return magnitude for a label.

    Returns:
        Array of labels: 1 (up), 0 (down), NaN (neutral).
    """
    fwd_returns = np.roll(prices, -horizon) / prices - 1
    fwd_returns[-horizon:] = np.nan
    labels = np.where(fwd_returns > threshold, 1,
             np.where(fwd_returns < -threshold, 0, np.nan))
    return labels

Multi-Class Classification

Three classes for finer signal granularity:

| Class | Condition | Typical threshold | |-------|-----------|-------------------| | Strong Up | fwd_return > +2% | High confidence long | | Mild Up | +0.5% to +2% | Moderate confidence | | Down | fwd_return < -0.5% | Avoid / short |

Multi-class reduces per-class sample size. Use only with large datasets (1000+ samples per class).

Probability Calibration

Raw model probabilities from XGBoost/LightGBM are not well-calibrated. A predicted 0.7 probability does not mean 70% chance of being correct. Use calibration to fix this:

from sklearn.calibration import CalibratedClassifierCV

calibrated = CalibratedClassifierCV(base_model, cv=5, method="isotonic")
calibrated.fit(X_train, y_train)
probs = calibrated.predict_proba(X_test)[:, 1]

Isotonic calibration works better than Platt scaling for tree models.

Walk-Forward Validation

**This is the single most important concept in trading ML.** Standard cross-validation randomly shuffles data, which creates lookahead bias. Walk-forward validation respects time ordering.

How It Works

Window 1: [===TRAIN===][GAP][=TEST=]
Window 2:    [===TRAIN===][GAP][=TEST=]
Window 3:       [===TRAIN===][GAP][=TEST=]
Window 4:          [===TRAIN===][GAP][=TEST=]

Each window: 1. Train on past N bars 2. Skip a gap (embargo) equal to the forward return horizon 3. Predict on next M bars 4. Record out-of-sample predictions 5. Slide forward and repeat

Typical Parameters

| Parameter | Value | Rationale | |-----------|-------|-----------| | Train window | 30 days (720 hourly bars) | Enough data to learn, recent enough to be relevant | | Test window | 7 days (168 hourly bars) | Enough predictions for statistical significance | | Step size | 1 day (24 bars) | Overlap test windows for more data points | | Gap (embargo) | Same as forward horizon | Prevents label leakage |

Walk-Forward Implementation

from typing import Iterator

def walk_forward_splits(
    n_samples: int,
    train_size: int = 720,
    test_size: int = 168,
    step_size: int = 24,
    gap: int = 24,
) -> Iterator[tuple[np.ndarray, np.ndarray]]:
    """Generate walk-forward train/test index splits.

    Args:
        n_samples: Total number of samples.
        train_size: Number of training samples per window.
        test_size: Number of test samples per window.
        step_size: Step between successive windows.
        gap: Gap between train end and test start.

    Yields:
        Tuples of (train_indices, test_indices).
    """
    start = 0
    while start + train_size + gap + test_size <= n_samples:
        train_idx = np.arange(start, start + train_size)
        test_start = start + train_size + gap
        test_idx = np.arange(test_start, test_start + test_size)
        yield train_idx, test_idx
        start += step_size

See `references/validation_methods.md` for purged CV, CPCV, and evaluation metrics.

Model Training Pipeline

Full Pipeline Overview

1. **Feature engineering** — compute technical indicators, on-chain metrics, volume features (see `feature-engineering` skill) 2. **Label creation** — forward returns with threshold, drop neutral zone 3. **Walk-forward split** — time-ordered train/test windows with gap 4. **Train model** — XGBoost or LightGBM on each training window 5. **Predict on test** — generate out-of-sample probability predictions 6. **Aggregate predictions** — concatenate all out-of-sample results 7. **Evaluate** — accuracy, precision, recall, F1, AUC, profit factor

Quick Training Example