The paper proposes a hybrid model (NP-DNN) combining Neural Prophet and Deep Neural Network for stock market price prediction. The methodology includes data preprocessing (Z-score normalization, missing value imputation via linear interpolation), feature extraction using MLP, prediction using DNN, and hyperparameter tuning with Optuna. The model is evaluated on the Crunchbase dataset using classification metrics (accuracy, precision, recall, F1) and compared with DSS, LightGBM, RF, and LLM, claiming 93.21% accuracy. The paper contains several inconsistencies and ambiguities, including a mismatch between the dataset (startup investments) and the task (stock price prediction), and inappropriate use of classification metrics.

Implementation Assumptions

• The task is binary classification (price up/down) to align with classification metrics.

• MLP has 2 hidden layers (64, 32) with ReLU; DNN has 2 hidden layers (64, 32) with ReLU and softmax output.

• Neural Prophet is implemented using the Prophet library (not neuralprophet) for simplicity.

• Optuna search space: learning rate [1e-5, 1e-1], number of layers [1,3], hidden units [32,256], dropout [0.0,0.5], batch size [16,128].

• The Crunchbase dataset is not used; synthetic data is generated for demonstration.

• The abstract accuracy 99.21% is a typo; we use 93.21% as the claimed accuracy.

Introduction and Paper Overview

Stock market price prediction is one of the most challenging problems in financial forecasting. Prices move in response to countless factors—economic news, investor sentiment, geopolitical events, and random noise—making it nearly impossible to predict with high accuracy using simple methods. Traditional statistical models like ARIMA or linear regression often fail to capture the complex, nonlinear patterns that drive market movements.

The paper we are implementing, "Stock Market Price Prediction using Neural Prophet with Deep Neural Network" (arXiv:2601.05202v3), proposes a hybrid model called NP-DNN that combines two powerful tools:

• Neural Prophet (NP): A time series decomposition model that breaks a price history into interpretable components: the overall trend, repeating seasonal patterns, holiday effects, and random error. This helps the model understand long-term structure.

• Deep Neural Network (DNN): A multi-layer neural network that learns complex patterns from data. The paper uses a Multi-Layer Perceptron (MLP) for feature extraction before feeding into the DNN for final prediction.

The overall pipeline has three main stages:

1. Data Preprocessing: Fill missing values using linear interpolation and standardize features with Z-score normalization.

2. Feature Extraction: Use an MLP to transform the preprocessed data into a more informative representation.

3. Prediction: Feed the extracted features into a DNN with a softmax output layer to produce a classification (e.g., price up or down).

The paper claims that this NP-DNN model achieves 93.21% accuracy on the Crunchbase dataset, outperforming methods like LightGBM, Random Forest, and even a Large Language Model (LLM).

What to Expect from This Tutorial

Read more

The paper proposes a unified framework for conditional generative modeling of graph signals using denoising diffusion models. The key innovation is the U-Graph Neural Network (U-GNN), a U-Net-like architecture for graph-structured data that uses learned node selection matrices for pooling/unpooling and strided graph convolutions to avoid explicit graph coarsening. The framework is demonstrated on two tasks: stock price forecasting (S&P 500) and wireless resource allocation (power control).

Implementation Assumptions

• PyTorch 2.x + PyTorch Geometric (PyG) + standard scientific Python (numpy, scipy, matplotlib).

• Graph shift operator S stored as a sparse PyTorch tensor (coalesced COO) or edgeindex+edgeweight.

• All graph signals are batched: (batch, N, F) with optional node dimension per graph (single graph assumed for each task).

• The U-GNN is built from modular components: StridedGraphConv, NodeSelection, FusionBlock, UGNN core.

• Training uses AMP (torch.cuda.amp), AdamW, cosine LR with linear warm-up, gradient clipping (max_norm=1.0).

• Validation and checkpointing use task-specific composite criteria.

• Data pipeline for S&P 500 uses yfinance; graph built from fundamentals (rank correlation + sector bonus).

• Data pipeline for WRA synthesizes networks using the paper's channel model and expert primal-dual algorithm.

• DDIM sampling uses 100 steps and η=0.2 on a sub-grid of the 500-step linear schedule.

Download Source code.

You can download the source using the URL at the end of this article.

Note: There was no source code attached to the paper, this is my implementation, I tried to code it, and it is part of the larger program, so use the code to understand and implement in your code, try running it would run but there might be some problem. So Use code as reference, not in actual public working.

1. Introduction to the Paper and Problem

1.1 What Are Stochastic Graph Signals?

Many real-world systems produce signals that live on a graph — a set of nodes connected by edges that encode relationships. When those signals are inherently random, we call them stochastic graph signals. Two canonical examples, both studied in the paper, are:

• Stock returns on a company graph: Each node is a publicly traded company (e.g., the S&P 500 constituents). The graph edges capture fundamental similarities: same industry sector, correlated financial profiles, or shared supply chains. The signal at each node is the daily log-return of that stock. The signal is stochastic because future returns are unpredictable in detail, yet they are conditionally dependent on the past and on the graph structure (e.g., a shock to one sector propagates to related companies).

• Power allocations on an interference graph: Each node is a wireless transmitter–receiver pair. The graph edges represent interference strength: a directed edge from transmitter j to receiver i carries the channel gain between them. The signal at each node is the transmit power level. The optimal allocation is stochastic because it depends on random fading and the expert algorithm’s internal randomness, but it is conditioned on the large-scale channel gains (the graph) and the node’s own channel quality.

In both cases, the graph topology and node-level side information (history features, channel measurements) provide a conditioning context. The goal is to sample new graph signals from the unknown conditional distribution — not just to predict a single value, but to generate entire trajectories or allocations that respect the graph structure and the observed context.

1.2 Why Generative Modeling (Not Just Regression)?

A regressive approach — training a neural network to minimize mean squared error between a point prediction and the true signal — learns the conditional mean. This is adequate when the conditional distribution is unimodal and symmetric, but it fails when:

• The distribution is multi-modal: there may be several equally plausible future price paths or power allocations.

• Uncertainty quantification is required: a point forecast gives no sense of risk or confidence intervals.

• The downstream task needs samples, not averages: for portfolio optimization, one needs many possible return scenarios; for wireless resource allocation, one needs to test different power settings under fading.

Generative models address these limitations by learning the full conditional law. The paper adopts a denoising diffusion probabilistic model (DDPM), which has become the state of the art for high-quality sample generation in images, audio, and now graph signals. Diffusion models are particularly attractive because they are stable to train, do not suffer from mode collapse (unlike GANs), and can be conditioned naturally on arbitrary inputs.

1.3 The Two Applications

The paper demonstrates the framework on two distinct tasks, each with its own graph structure and conditioning modality:

Stock Price Forecasting (S&P 500)

• Data: Daily prices for 468 stocks over ~10 years. For each stock, 12 market features (open, high, low, close, log-return, moving averages, volume, RSI, MACD) are computed. A static undirected graph is built from rank correlations of fundamental profiles, with a same-sector bonus.

• Conditioning: The last 20 days of features (history) and the graph topology.

• Target: The next 5 days of log-returns (a graph signal of dimension 5 per node).

• Challenge: The conditional distribution of future returns is heavy-tailed, heteroscedastic, and exhibits volatility clustering. A generative model can produce multiple plausible trajectories, enabling risk-aware decision-making.

Wireless Resource Allocation (Power Control)

• Data: Synthetic networks of 400 transmitter–receiver pairs in square areas of varying density. Channel gains follow a dual-slope path loss model with log-normal shadowing and Rayleigh fading. An expert primal–dual algorithm computes optimal power allocations that maximize sum ergodic rate subject to per-user minimum rate constraints.

• Conditioning: The directed interference graph (top-10 strongest interferers per receiver, log-normalized) and two node-state features (direct link gain and aggregate interference under full power).

• Target: The centered power allocation (p/Pmax − 1) for each user.

• Challenge: The expert algorithm is computationally expensive; a learned generative model can approximate its output distribution at a fraction of the cost, and can be deployed in real time.

1.4 Overview of the Proposed Solution

The paper proposes a unified framework that combines a denoising diffusion probabilistic model with a novel U‑Graph Neural Network (U‑GNN) denoiser. The key ideas are:

1. Forward diffusion: Gradually add Gaussian noise to the clean graph signal over 500 steps, following a linear noise schedule (β₁ = 1e−4, β₅₀₀ = 2e−2). The noising process is fixed and does not depend on the graph.

2. Reverse denoising: Learn a neural network ε_θ that predicts the noise added at each step, conditioned on the graph shift operator S and node features u. The network is trained with the simple mean-squared error between true and predicted noise (equation (6) in the paper).

3. U‑GNN denoiser: A U‑Net-like architecture for graph signals. It processes the signal at multiple resolutions by:

• Strided graph convolutions (Algorithm 1): Instead of coarsening the graph, the signal is lifted to the full graph, filtered with iterated unit shifts of S, and then reduced to a subset of active nodes. The stride γ increases the receptive field without densifying the graph.

• Learned node selection (Section IV‑B): At each down‑sampling level, a scoring MLP selects the top‑k nodes to retain. During training, Gumbel noise encourages exploration, and a straight‑through estimator allows gradients to flow through the discrete selection.

• Fusion layers (equation (22)): The down‑sampled signal is merged with global embeddings (node states and diffusion step) via cross‑attention (for stock) or addition (for WRA).

• Skip connections (equation (23)): The decoder receives concatenated features from the encoder at the same resolution.

4. Accelerated sampling: Instead of the full 500‑step reverse process, the paper uses DDIM (Denoising Diffusion Implicit Models) with 100 steps and η = 0.2, which produces high‑quality samples much faster.

5. Task‑specific adaptations: For stock forecasting, the U‑GNN includes a lightweight temporal mixer (interleaved dilated 1‑D convolutions and two‑head self‑attention) to process the history features. For WRA, the conditioning is static and uses simple addition.

1.5 What This Tutorial Covers

This tutorial walks through a complete, modular reproduction of the paper’s stock forecasting and wireless resource allocation experiments. We will:

• Implement the forward diffusion process and DDIM sampling from scratch.

• Build the U‑GNN architecture piece by piece: strided graph convolutions, learned node selection, fusion layers, and the full encoder–decoder.

• Construct the data pipelines for both tasks, including synthetic data generation for WRA and a simplified stock data pipeline (using synthetic data to avoid network calls).

• Train the model with the exact hyperparameters from the paper (Table I, Appendix E).

• Evaluate using the paper’s metrics: CRPS, MIS90, RMSE, direction accuracy, stylized‑fact gaps for stock; ergodic rate, feasibility, and gap to expert for WRA.

• Reproduce the key figures: forecasting trajectories, CDF comparisons, distribution diagnostics, and WRA bar charts.

All code is written in Python using PyTorch and PyTorch Geometric, and is organized into modular files for clarity. The implementation follows the paper’s descriptions and appendices faithfully, with documented decisions where the paper is ambiguous.

Let’s begin by understanding the diffusion process and how it is adapted to graph signals.

2. Denoising Diffusion Probabilistic Models for Graph Signals

2.1 Forward Diffusion Process

The core idea of a denoising diffusion model is to define a forward process that gradually corrupts a clean signal into pure noise, and then learn a reverse process that undoes this corruption step by step. For graph signals, the forward process operates on the node features directly, independent of the graph structure at this stage — the graph topology enters only as conditioning information for the reverse denoiser.

Let x_0 \in \mathbb{R}^{N \times F} be a clean graph signal (e.g., the future returns of N stocks over F time steps). The paper defines a Markov chain of K = 500 steps that adds Gaussian noise with a linear variance schedule:

Each forward transition is (equation (2)):

Because the Gaussian is closed under composition, we can directly sample x_k from x_0 without iterating through all intermediate steps. Define \alpha_k = 1 - \beta_k and the cumulative product \bar{\alpha}_k = \prod_{i=1}^k \alpha_i. Then the marginal distribution is (equation (3)):

which leads to the reparameterization used in training (equation (4)):

Implementation in diffusion.py

The DiffusionProcess class precomputes all \bar{\alpha}_k values from the linear \beta schedule. The constructor stores the betas, alphas, and derived quantities as tensors for efficient indexing:

class DiffusionProcess:
    def __init__(self, betas: torch.Tensor) -> None:
        self.K = betas.shape[0]
        self.betas = betas
        self.alphas = 1.0 - betas
        # alpha_bar[t] = prod_{i=1}^{t} alpha_i, alpha_bar[0] = 1
        self.alpha_bar = torch.cat([
            torch.ones(1, dtype=betas.dtype, device=betas.device),
            torch.cumprod(self.alphas, dim=0)
        ], dim=0)
        self.sqrt_alpha_bar = self.alpha_bar.sqrt()
        self.sqrt_one_minus_alpha_bar = torch.sqrt(1.0 - self.alpha_bar)

The forward_diffuse method implements equation (4) directly. It takes a clean signal x_0 and a step index k (1-indexed), samples noise, and returns the noised signal:

def forward_diffuse(self, x0: torch.Tensor, k: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:
    k_idx = k.long()
    sqrt_alpha_k = self.sqrt_alpha_bar[k_idx]
    sqrt_one_minus_alpha_k = self.sqrt_one_minus_alpha_bar[k_idx]
    epsilon = torch.randn_like(x0)
    x_k = sqrt_alpha_k * x0 + sqrt_one_minus_alpha_k * epsilon
    return x_k, epsilon

During training, we sample k \sim \text{Uniform}(1, K) for each batch element, compute x_k using this method, and then ask the denoiser to predict the noise \varepsilon.

2.2 Reverse Denoising and the Noise-Prediction Objective

The reverse process is a Markov chain that starts from pure noise x_K \sim \mathcal{N}(0,I) and iteratively removes noise to recover x_0. Each reverse step is parameterized as a Gaussian with learned mean (equation (5)):

Here \xi = (S, u) denotes the conditioning information: the graph shift operator S and node features u. The mean \mu_\theta is derived from a noise-prediction network \varepsilon_\theta (the U-GNN) via equation (8):

Rather than directly predicting \mu_\theta, the paper adopts the simplified training objective from Ho et al. (2020) (equation (6)):

This loss is simple: at each training step, we sample a random k, noise \varepsilon, and a clean signal x_0, compute x_k via the forward process, and ask the U-GNN to predict the noise that was added. The gradient flows through the U-GNN only; the forward process is fixed.

2.3 DDIM Sampling for Acceleration

While the reverse process could be simulated for all K = 500 steps, the paper uses the Denoising Diffusion Implicit Model (DDIM) to accelerate sampling to just 100 steps while maintaining sample quality. DDIM defines a non-Markovian reverse process that shares the same forward marginals but allows larger jumps.

At each sampling step, we first estimate the clean signal from the current noisy x_k and the predicted noise (equation (7)):

Then we update to the previous step k-1 (or, for accelerated sampling, to the previous sub-grid step \tau_{i-1}) using equation (9):

The noise scale \sigma_k is controlled by a parameter \eta \in [0,1] (equation (29)):

When \eta = 0, the process is deterministic (pure DDIM). When \eta = 1, it recovers the original DDPM. The paper uses \eta = 0.2 for a balance of stochasticity and speed.

For accelerated sampling, we select a sub-grid of step indices (e.g., 100 evenly spaced indices from 1 to 500) and apply the update using the corresponding \bar{\alpha} values. The sample_sub_grid method generates this list:

def sample_sub_grid(self, n_steps: int) -> List[int]:
    step_indices = torch.linspace(
        self.K, 1, n_steps, dtype=torch.float32
    ).round().long().unique().tolist()
    step_indices = sorted(step_indices, reverse=True)
    return step_indices

The ddim_sample method implements the full sampling loop. It takes the denoiser function, initial noise x_K, conditioning dictionary, sub-grid, and \eta:

def ddim_sample(self, denoiser, x_K, cond=None, sub_grid=None, eta=0.2):
    if sub_grid is None:
        sub_grid = list(range(self.K, 0, -1))
    sub_grid = sorted(sub_grid, reverse=True)
    x = x_K.clone()
    for i, k in enumerate(sub_grid):
        k_prev = sub_grid[i + 1] if i + 1 < len(sub_grid) else 0
        sqrt_alpha_bar_k = self.sqrt_alpha_bar[k]
        sqrt_one_minus_alpha_bar_k = self.sqrt_one_minus_alpha_bar[k]
        inv_sqrt_alpha_bar_k = self.inv_sqrt_alpha_bar[k]
        epsilon_theta = denoiser(x, k, cond)
        x_hat_0 = (x - sqrt_one_minus_alpha_bar_k * epsilon_theta) * inv_sqrt_alpha_bar_k
        if k_prev > 0:
            sqrt_alpha_bar_prev = self.sqrt_alpha_bar[k_prev]
            one_minus_alpha_bar_prev = self.one_minus_alpha_bar[k_prev]
            one_minus_alpha_bar_k = self.one_minus_alpha_bar[k]
            ratio = (one_minus_alpha_bar_prev / one_minus_alpha_bar_k).clamp(min=0.0)
            sigma = eta * torch.sqrt(ratio) * torch.sqrt(1.0 - self.alpha_bar[k] / self.alpha_bar[k_prev])
            noise = torch.randn_like(x) if eta > 0 else 0.0
            sqrt_one_minus_alpha_bar_prev_minus_sigma2 = torch.sqrt(
                (one_minus_alpha_bar_prev - sigma**2).clamp(min=0.0)
            )
            x = sqrt_alpha_bar_prev * x_hat_0 + sqrt_one_minus_alpha_bar_prev_minus_sigma2 * epsilon_theta + sigma * noise
        else:
            x = x_hat_0
    return x

Note the adaptation for sub-grid steps: the formula uses \bar{\alpha}_{k_\text{prev}} and \bar{\alpha}_k directly, which is valid for any two steps where k_\text{prev} < k. The clamping ensures numerical stability.

2.4 Conditioning on Graph Topology and Node Features

The denoiser \varepsilon_\theta is the U-GNN, which takes three inputs at each step:

• Noisy signal x_k: the current node features to be denoised.

• Diffusion step k: embedded as a sinusoidal positional encoding and broadcast to all nodes.

• Conditioning \xi = (S, u): the graph shift operator S (a sparse matrix encoding edge weights) and node-level side information u (e.g., historical market features for stock forecasting, or direct link gain for wireless).

The conditioning is injected through fusion layers inside the U-GNN, which we will examine in detail in the next section. For now, it suffices to understand that the denoiser is a function:

and that the entire diffusion framework — forward process, noise-prediction loss, and DDIM sampling — is agnostic to the internal architecture of \varepsilon_\theta. This modularity is what allows the paper to apply the same diffusion framework to two very different tasks by swapping only the denoiser architecture.

Summary

• The forward diffusion process adds Gaussian noise to graph signals according to a linear schedule (equations (2)-(4)).

• The reverse process is learned by predicting the noise added at each step (equation (6)).

• DDIM sampling accelerates generation by using a sub-grid of steps and a controlled stochasticity parameter \eta (equations (7), (9), (29)-(30)).

• The DiffusionProcess class in diffusion.py implements all of these operations with precomputed \bar{\alpha} values and efficient tensor operations.

• Conditioning on graph topology and node features is handled by the U-GNN denoiser, which we will explore next.

3. The U-GNN Architecture

3.1 Why a Graph U-Net?

Images have a natural multi-scale structure: you can downsample (pool) by averaging neighboring pixels, and then upsample (unpool) by interpolation. A U-Net exploits this to build a computational graph with a wide receptive field while keeping the number of parameters manageable. Graph signals lack such a regular grid — there is no canonical notion of “neighbourhood averaging” because the graph is irregular. Nevertheless, the same intuition applies: to model long-range dependencies across the graph, we need to process the signal at multiple resolutions, aggregating information from far-away nodes into compact representations and then expanding back to the original resolution.

The paper’s solution, the U-Graph Neural Network (U-GNN), is a U-Net-like architecture designed for graph-structured data. It uses three key innovations:

1. Strided graph convolutions that increase the receptive field without making the graph shift operator denser.

2. Learned node selection (with Gumbel-Top-K and a straight-through estimator) to decide which nodes to keep at each coarser resolution, avoiding a fixed pooling strategy.

3. Fusion layers that inject conditioning information (node states and diffusion step) at every resolution level.

Let us walk through each component.

3.2 The Lift–Filter–Reduce Pattern (Equation (13))

A standard GNN layer updates node features by exchanging messages along edges. If the graph shift operator is a sparse matrix S \in \mathbb{R}^{N \times N} (the adjacency or a normalized Laplacian), a single layer computes:

where X_{\ell-1} are the node features, \Theta_{\ell,k} are learnable weight matrices for each “tap” k, and \sigma_\ell is an activation. Each tap k corresponds to a k-step random walk on the graph, giving a localised receptive field.

Now suppose we have identified a subset of “active” nodes (those that survive the pooling) through a binary selection matrix D \in \{0,1\}^{N' \times N} (each row is a one-hot indicator of an active node). The paper uses a lift–filter–reduce pattern (equation (13)):

1. Lift: D^\mathsf{T} Z_{\ell-1} maps the N'-node signal back to the full N-node graph by placing the values at the active nodes and leaving zeros elsewhere.

2. Filter: apply the graph convolution (sum over k taps) on the full graph.

3. Reduce: multiply by D to extract the active-node signals.

This pattern is the foundation of every strided convolution in U-GNN.

3.3 Strided Graph Convolutions (Algorithm 1, Equation (14))

A plain GNN with K taps has a receptive field of exactly K hops. To reach farther nodes, one could increase K, but that makes S^K denser and more expensive (both in memory and computation). The paper’s solution is to replace S^k with (S^\gamma)^k, where \gamma is a stride parameter. Equation (14) formalises this:

Raising S to the power \gamma directly is expensive, so the paper proposes Algorithm 1: instead of forming S^\gamma, we perform \gamma K unit shifts and tap every \gamma-th one. This gives the same result as using S^\gamma, but each shift is a sparse matrix–vector product, and the intermediate \gamma-1 shifts that are not tapped still propagate the signal forward, building the wider receptive field.

Let us look at the implementation in gnn_layers.py:

class StridedGraphConv(nn.Module):
    """
    Strided graph convolution layer with pooling (lift-filter-reduce).
    Implements Algorithm 1 of the paper.
    """
    def __init__(self, in_dim, out_dim, K, gamma, activation=nn.ReLU()):
        super().__init__()
        self.weight = nn.Parameter(torch.randn(K + 1, in_dim, out_dim) * 0.1)
        self.bias = nn.Parameter(torch.zeros(out_dim))
        self.K = K
        self.gamma = gamma
        self.activation = activation

    def forward(self, Z_prev, S, D):
        # Lift
        x = torch.mm(D.T, Z_prev)          # (N, in_dim)
        # Tap 0
        A = torch.mm(x, self.weight[0])   # (N, out_dim)

        # Iterated unit shifts
        for j in range(1, self.gamma * self.K + 1):
            x = torch.mm(S, x)            # one shift
            if j % self.gamma == 0:
                tap_idx = j // self.gamma
                A = A + torch.mm(x, self.weight[tap_idx])

        A = self.activation(A + self.bias)
        out = torch.mm(D, A)              # reduce
        return out

Key points:

• The total number of taps is still K+1 (including the identity tap), independent of \gamma.

• The stride \gamma is set per depth: \gamma_b = \min(\lfloor\sqrt{\bar{\rho}_b}\rfloor, \gamma_{\max}=2), where \bar{\rho}_b is the cumulative pooling factor at depth b.

• Both dense and sparse S are supported (the code checks S.is_sparse; PyTorch Geometric’s sparse ops can be used in practice).

3.4 Learned Node Selection (Equations (24)–(25), (31)–(34))

How does the U-GNN decide which nodes to keep when going from one resolution level to the next? Instead of a fixed rule (e.g., max pooling over neighbourhoods), the paper learns which nodes are most informative. The process has three steps:

1. Score each node: a node-wise MLP \Psi_b takes the concatenation of the encoded feature Z_b and the global embeddings D_b[U_0;K_0] and outputs a scalar score (equation (24)).

2. **Select top-k**: keep the N_{b+1} highest-scoring nodes (equation (25)).

3. Down-sample: the selection matrix C_{b+1} is the identity rows indexed by the chosen nodes.

But this discrete selection is non-differentiable. During training, the paper uses two tricks:

• Gumbel-Top-K (equation (31)): Gumbel noise is added to the scores before the \text{TopK} operation, encouraging exploration.

• Straight-Through Estimator (STE) (equation (32)): the forward pass uses a hard one-hot mask (the selected nodes), while the backward pass flows through a soft sigmoid surrogate \sigma(v_b / \tau). The code does this with a simple detach trick:

mask_hard = torch.zeros(N_b, device=device)
mask_hard.scatter_(0, indices, 1.0)            # hard one-hot
mask_soft = torch.sigmoid(scores / tau)         # soft surrogate
mask = mask_hard + mask_soft - mask_soft.detach()  # forward hard, backward soft

Additionally, the temperature \tau and exploration noise \varepsilon are annealed linearly over training (equation (34)): a warm-up phase of T_w = 0.02T epochs followed by an anneal phase of T_a = 0.73T epochs. The full NodeSelectionHead class in node_selection.py implements this schedule.

3.5 Full U-GNN: Encoder, Bottleneck, Decoder (Equations (18)–(23))

With the strided convolution and the learned selection head, we can now assemble the complete U-GNN. The architecture has B = 4 resolution levels. Let N_1 = N be the original number of nodes. At each depth b, the number of active nodes is N_b = \lfloor N / \rho^{b-1}\rfloor with \rho = 2.

Nested selection matrices (equation (18)): the global selection matrix D_b that maps from the full graph to the active nodes at depth b is the product of all previous selection matrices:

Encoder path (equation (19)): for b = 1, \dots, B-1:

1. Down-sample: V_b = C_b Z_{b-1}.

2. Fuse: P_b = \Pi_b(V_b, D_b[U_0;K_0]) (see below).

3. Apply strided GNN module: Z_b = \Phi_{E_b}(P_b, S; \Theta_{E_b}, D_b, \gamma_b).

4. Select: C_{b+1} = \text{Selector}(Z_b, D_b[U_0;K_0], N_{b+1}).

Bottleneck (equation (21)): at b = B, the same pattern without selection.

Decoder path (equation (20)): for b = B-1, \dots, 1:

1. Up-sample: C_{b+1}^\mathsf{T} Y_{b+1} (zero-pad to the previous resolution).

2. Skip-connect: concatenate the up-sampled decoder output with the encoder signal Z_b.

3. Project: V_b = \Pi_{\text{skip}}^b([C_{b+1}^\mathsf{T} Y_{b+1}; Z_b]) (equation (23)).

4. Fuse: P_b = \Pi_b(V_b, D_b[U_0;K_0]).

5. Apply strided GNN: Y_b = \Phi_{D_b}(P_b, S; \Theta_{D_b}, D_b, \gamma_b).

Finally, read-out: \varepsilon_\theta = \Pi_{\text{out}}(Y_1), a node-wise MLP that maps back to the original feature dimension.

The fusion layer \Pi_b (equation (22)) merges the down-sampled signal with the global embeddings (node states U_0 and step embedding K_0). For the stock task, the paper uses cross-attention (query = signal, key/value = global embeddings); for the WRA task, it uses simple addition after a projection. The FusionLayer class in ugnn.py implements both versions.

3.6 Task-Specific Adaptations

For the stock forecasting task, the input x_k has a temporal dimension (5 future days). The U-GNN includes a TemporalMixer that applies two dilated 1D convolutions followed by two-head self-attention along the time axis, producing a single feature vector per node (by averaging over time). This is a reasonable default for the paper’s underspecified temporal encoder.

For the wireless resource allocation task, the input is static (one allocation per node), so no temporal mixer is needed — the TemporalMixer is replaced by an identity module.

The UGNN class in ugnn.py orchestrates all these components. It pre-computes the target node counts, initialises the nested selection matrix D, and runs the encoder–bottleneck–decoder loop. The selection heads are called at the end of each encoder block, and the resulting C and D matrices are accumulated.

3.7 Summary

The U-GNN is a carefully designed graph U-Net that avoids explicit graph coarsening. Instead, it learns which nodes to retain at each level, uses strided graph convolutions to control the receptive field without making the shift operator denser, and fuses conditioning information at every resolution. This architecture is the central contribution of the paper and is what enables the diffusion model to generate high-quality graph signals conditioned on graph structure and node features.

4. Data Pipelines

A generative model is only as good as the data it learns from. The paper presents two distinct data pipelines, each transforming raw observations into the graph-structured inputs required by the diffusion model. This section walks through both pipelines in detail, showing how we go from stock tickers or wireless network parameters to the tensors that feed the U-GNN.

4.1 S&P 500 Stock Forecasting Pipeline

The stock forecasting task aims to predict the next five days of log-returns for 468 S&P 500 constituents, conditioned on the last 20 days of market features and a static graph of company similarities. The pipeline has five stages: data acquisition, feature engineering, graph construction, window creation, and normalisation & splitting.

4.1.1 Data Acquisition

In a real deployment, you would download daily OHLCV data from Yahoo Finance using the yfinance library. The paper specifies the date range 2016‑09‑06 to 2026‑02‑13. However, our implementation (in data_stock.py) provides a synthetic data generator that mimics the shape and basic statistics of 468 stocks, so that the code can be run without an internet connection. The function download_sp500_data returns a DataFrame with columns ticker, date, open, high, low, close, volume, and sector. The synthetic prices follow an AR(1) log‑price process with a common market factor and idiosyncratic noise, which is sufficient for testing the pipeline logic.

def download_sp500_data(
    start_date: str = "2016-09-06",
    end_date: str = "2026-02-13",
    num_stocks: int = 468,
    num_days: int = 2500,
) -> pd.DataFrame:
    # ... generates synthetic OHLCV data with random sectors
    # Returns a DataFrame with columns: ticker, date, open, high, low, close, volume, sector

4.1.2 Feature Engineering

For each stock on each day, the paper computes 12 market features. The function compute_market_features adds these features as a list column to the DataFrame. The features are:

• open, high, low, close, log_return

• moving averages (MA5, MA20)

• volume

• RSI(14)

• MACD, MACDsignal, MACDhistogram

def compute_market_features(df: pd.DataFrame) -> pd.DataFrame:
    # ... computes each feature per ticker
    # Adds columns 'features' (list of 12 floats) and 'log_return'

The log_return column becomes the target signal; the full feature vector serves as the conditioning input u.

4.1.3 Graph Construction from Fundamentals

The paper builds a static undirected graph from company fundamentals. The function build_fundamentals_graph implements a simplified version:

1. Compute the Spearman rank correlation between the mean feature vectors of each pair of stocks.

2. Add a same‑sector bonus of 0.1.

3. Threshold at the median correlation to keep only the strongest edges.

4. Symmetrize and add self‑loops.

5. Apply spectral normalisation: S = D^{-1/2} A D^{-1/2}.

The result is a dense (N, N) tensor S and a list of sector labels.

def build_fundamentals_graph(stocks_df: pd.DataFrame) -> Tuple[torch.Tensor, List[int]]:
    # ... rank correlation, sector bonus, threshold, spectral normalisation
    return torch.tensor(S, dtype=torch.float32), sectors.tolist()

4.1.4 Sliding Windows

With the feature matrix of shape (N, T, U) (12 features) and the target matrix of shape (N, T) (log‑returns), the function create_sliding_windows extracts every possible window of length Th + Tp. For each starting time t from Th to T - Tp, it produces:

• history u: shape (N, Th, U) — the last 20 days of features.

• target x0: shape (N, Tp) — the next 5 days of log‑returns.

def create_sliding_windows(
    data: pd.DataFrame, Th: int = 20, Tp: int = 5
) -> List[Dict]:
    # ... builds feature tensor (N, T, U) and target tensor (N, T)
    # ... iterates t from Th to T-Tp, creates history and target tensors

4.1.5 RevIN Normalisation

The paper applies a modified Reversible Instance Normalisation (RevIN) to the target window. The RevIN class in data_stock.py implements this with the paper‑specific parameters: a blend weight of 0.7 and a scale correction of 1.107. During the forward pass, it normalises the target using per‑instance mean and standard deviation (computed along the temporal dimension), then blends these statistics with running averages for consistency. The normalised target is then scaled and shifted by learnable affine parameters.

class RevIN(nn.Module):
    def __init__(self, num_features: int = 1, blend: float = 0.7, scale_correction: float = 1.107):
        # ... initialise affine weight with 1/scale_correction

    def forward(self, x, mode="norm", mean=None, std=None):
        if mode == "norm":
            # compute instance mean and std, blend with running stats
            # normalise, apply affine transform
            return x_norm, (mean_inst, std_inst)
        elif mode == "denorm":
            # reverse affine, then denormalise using stored statistics
            return x_denorm, None

4.1.6 Interleaved Chronological Split

To avoid temporal leakage, the paper uses an interleaved chronological split. The function interleaved_chronological_split divides the total number of windows into 10 equal chunks, then within each chunk splits chronologically 80% train, 10% validation, 10% test. The windows are pooled across chunks, ensuring that test windows always come after training windows in time.

def interleaved_chronological_split(
    windows: List[Dict],
    split_ratios: Tuple[float, float, float] = (0.8, 0.1, 0.1),
    num_chunks: int = 10,
) -> Dict[str, List[Dict]]:
    # ... divide windows into chunks, split each chunk, concatenate
    return {"train": train, "val": val, "test": test}

4.1.7 Dataset and DataLoader

The StockDataset class wraps the windows, the static graph S, and the RevIN module. Each call to __getitem__ returns a dictionary with keys:

• x0: target log‑returns (normalised), shape (N, Tp)

• u: history features, shape (N, Th, U)

• S: graph shift operator, shape (N, N)

• revin_params: tuple (mean, std) used for later denormalisation during evaluation

The factory function get_stock_dataloaders creates the three DataLoaders with the appropriate batch size (64) and shuffling.

class StockDataset(Dataset):
    def __init__(self, windows: List[Dict], S: torch.Tensor, revin: RevIN):
        self.windows = windows
        self.S = S
        self.revin = revin

    def __getitem__(self, idx):
        w = self.windows[idx]
        # apply RevIN normalisation to target
        x0_norm, (mean, std) = self.revin(w["target"], mode="norm")
        return {
            "x0": x0_norm,
            "u": w["history"],
            "S": self.S,
            "revin_params": (mean, std)
        }

4.2 Wireless Resource Allocation Pipeline

The wireless task is to learn a policy that maps a network configuration (positions, channel gains) to a set of transmit powers (one per transmitter‑receiver pair) that maximise the sum ergodic rate under a minimum rate constraint. The pipeline synthetically generates 128 networks (32 for each of four densities) and uses an expert primal‑dual algorithm to produce optimal allocations.

4.2.1 Network Generation

The WirelessNetworkGenerator class creates a network with N=400 transmitter‑receiver pairs uniformly distributed in a square area. The side length R depends on the density (Table III):

| Density Index | Side Length R (m) | Density (pairs/km²) | |---------------|-------------------|---------------------| | 0 | 7800 | 6.6 | | 1 | 7000 | 8.2 | | 2 | 6300 | 10.1 | | 3 | 5800 | 11.9 |

A minimum transmitter‑transmitter spacing of 50 m is enforced by resampling pairs that are too close.

class WirelessNetworkGenerator:
    def __init__(self, config: WRAConfig):
        self.config = config

    def generate_network(self, density_idx: int) -> Dict:
        # ... uniform positions in [0,R]x[0,R], enforce min spacing
        return {
            'pos_tx': pos_tx, 'pos_rx': pos_rx,
            'area_side': R, 'density': density
        }

4.2.2 Channel Model

The ChannelModel implements the paper’s dual‑slope path loss model with log‑normal shadowing (σ = 7 dB) and Rayleigh fading. The path loss has a breakpoint at 200 m: exponent 2.0 before, 3.5 after. The compute_path_loss method returns an (N, N) matrix of large‑scale channel gains (linear scale). The sample_fading method generates unit‑mean complex Gaussian fading samples for a given number of links and time slots.

class ChannelModel:
    def compute_path_loss(self, pos_tx, pos_rx):
        # ... distances, dual-slope, shadowing
        return gain  # (N, N) tensor

    def sample_fading(self, n_links, n_slots):
        # ... complex Gaussian, squared magnitude
        return fading  # (n_links, n_slots)

4.2.3 GSO Construction

The function build_wra_gso constructs a directed weighted graph shift operator. For each receiver i, it selects the top‑10 strongest interferers (excluding self), sets the edge weight to the channel gain, then applies a log‑normalisation: S = log(1 + gain) / max(log(1 + gain)). Self‑loops are set to 1.

def build_wra_gso(network, channel_gains, config):
    # ... for each receiver, top-K gains, log-normalisation
    return S  # (N, N) tensor

4.2.4 Node State Features

The compute_node_states function computes the 2‑dimensional node state vector u for each receiver:

• direct_link_gain: the channel gain from its own transmitter.

• aggregate_interference: sum of channel gains from all other transmitters multiplied by Pmax.

def compute_node_states(network, channel_gains, config):
    direct = channel_gains.diag()
    interference = (channel_gains.sum(dim=1) - direct) * config.Pmax
    u = torch.stack([direct, interference], dim=1)  # (N, 2)
    return u

4.2.5 Expert Primal‑Dual Algorithm

The paper uses a primal‑dual algorithm to solve the constrained optimisation problem (equation (28)). The expert_primal_dual function implements a simplified version: it performs projected gradient ascent on the powers p and gradient descent on the dual variables λ to maximise the Lagrangian. From the converged trajectory, it collects 200 intermediate allocations (though for simplicity our implementation returns the final allocation). The allocation is centred as x0 = p / Pmax - 1.

def expert_primal_dual(network, channel_gains, config):
    # ... initialise p, lambda
    for it in range(num_iters):
        # compute rates, gradient of Lagrangian, update p, project to [0,Pmax]
        # update lambda with clipping
        # collect samples
    x0 = p / Pmax - 1.0
    return x0.float()

4.2.6 Dataset and DataLoader

The WRADataset wraps the lists of networks, allocations, GSOs, and node states. Each item returns a dictionary with:

• x0: centred allocation, shape (N,)

• S: graph shift operator, shape (N, N)

• u: node state features, shape (N, 2)

The factory function get_wra_dataloaders generates all 128 networks (32 per density), splits them 20/4/8 per density for train/val/test, and returns three DataLoaders with batch size 800 (as specified in Table I).

class WRADataset(Dataset):
    def __init__(self, networks, allocations, gsos, node_states):
        # ... store lists

    def __getitem__(self, idx):
        return {
            'x0': self.allocations[idx],
            'S': self.gsos[idx],
            'u': self.node_states[idx]
        }

4.3 Shape Invariants Summary

Both pipelines produce tensors with consistent shapes that the U‑GNN expects:

| Task | x0 shape | u shape | S shape | Notes | |-------|------------------|--------------------|-------------|------------------------------------------| | Stock | (N, Tp=5) | (N, Th=20, U=12) | (N, N) | x0 normalised; S static, symmetric | | WRA | (N,) | (N, 2) | (N, N) | x0 centred; S directed, weighted |

These shapes are guaranteed by the dataset classes and are maintained throughout the diffusion forward and reverse processes. The conditioning variable u is always available, and the graph S is used by every strided convolution in the U‑GNN.

With the data pipelines in place, we can now load batches of (x0, u, S) and feed them into the diffusion model. The next section walks through the core code that implements the forward diffusion, the U‑GNN architecture, and the training loop.

5. Code Walkthrough: Core Components

With the theoretical foundation laid and the data pipelines ready, we now open the code files themselves. This section walks through each of the four core modules—diffusion.py, gnn_layers.py, node_selection.py, and ugnn.py—function by function. Our goal is to show exactly how the paper’s mathematical descriptions and algorithms become executable Python, highlighting every design decision and shape constraint.

5.1 Diffusion Process (diffusion.py)

File purpose: Implements the forward diffusion process (equations (2)–(4)) and DDIM accelerated sampling (equations (9) and (29)–(30)).

DiffusionProcess.__init__

class DiffusionProcess:
    def __init__(self, betas: torch.Tensor) -> None:
        if betas.dim() != 1:
            raise ValueError("betas must be a 1D tensor")
        if not torch.all(betas > 0) or not torch.all(betas < 1):
            raise ValueError("betas must be in (0,1)")

        self.K = betas.shape[0]
        self.betas = betas
        self.alphas = 1.0 - betas

        # alpha_bar[t] = prod_{i=1}^{t} alpha_i, alpha_bar[0] = 1
        # shape: (K+1,)
        self.alpha_bar = torch.cat([
            torch.ones(1, dtype=betas.dtype, device=betas.device),
            torch.cumprod(self.alphas, dim=0)
        ], dim=0)

        # Precompute sqrt and sqrt(1 - alpha_bar) for efficiency
        self.sqrt_alpha_bar = self.alpha_bar.sqrt()
        self.sqrt_one_minus_alpha_bar = torch.sqrt(1.0 - self.alpha_bar)

        # For DDIM, precompute 1 - alpha_bar and inverse of sqrt_alpha_bar
        self.one_minus_alpha_bar = 1.0 - self.alpha_bar
        self.inv_sqrt_alpha_bar = 1.0 / self.sqrt_alpha_bar

Explanation: The constructor takes a 1D tensor betas of length K (K = 500 in the paper). It computes alpha = 1 - beta and then builds alpha_bar, where alpha_bar[t] = prod_{i=1}^{t} alpha_i. A leading 1.0 is prepended so that alpha_bar[0] = 1 (no noise at step 0). All subsequent quantities are precomputed and stored as tensors: sqrt_alpha_bar, sqrt_one_minus_alpha_bar, one_minus_alpha_bar, and inv_sqrt_alpha_bar. This avoids recomputing square roots during training and sampling.

Shape note: alpha_bar has shape (K+1,) so that indexing with step k (1‑indexed) directly uses position k.

forward_diffuse (equation (4))

def forward_diffuse(self, x0: torch.Tensor, k: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:
    k_idx = k.long()
    sqrt_alpha_k = self.sqrt_alpha_bar[k_idx]
    sqrt_one_minus_alpha_k = self.sqrt_one_minus_alpha_bar[k_idx]

    epsilon = torch.randn_like(x0)
    x_k = sqrt_alpha_k * x0 + sqrt_one_minus_alpha_k * epsilon

    return x_k, epsilon

Explanation: This is the one-line reparameterization of equation (4): x_k = sqrt(alpha_bar_k) * x0 + sqrt(1 - alpha_bar_k) * epsilon. We look up the precomputed sqrt_alpha_bar at index k (which is a tensor, possibly batched), sample a fresh Gaussian noise epsilon, and return both x_k and epsilon for the noise‑prediction loss. The shape of x_k matches x0 exactly.

sample_sub_grid

def sample_sub_grid(self, n_steps: int) -> List[int]:
    if n_steps > self.K:
        raise ValueError("n_steps cannot exceed K")
    if n_steps <= 0:
        raise ValueError("n_steps must be positive")

    step_indices = torch.linspace(
        self.K, 1, n_steps, dtype=torch.float32
    ).round().long().unique().tolist()

    if len(step_indices) > n_steps:
        step_indices = step_indices[:n_steps]
    elif len(step_indices) < n_steps:
        step_indices = [max(1, int(round(self.K - i * (self.K - 1) / (n_steps - 1))))
                        for i in range(n_steps)]
        step_indices = sorted(set(step_indices), reverse=True)

    step_indices = sorted(step_indices, reverse=True)
    return step_indices

Explanation: Accelerated DDIM uses a sub‑grid of the full 500‑step schedule. This method generates n_steps indices (e.g., 100) that are roughly evenly spaced from K down to 1. It first tries torch.linspace rounded to integers; if rounding produces duplicates (less than n_steps unique values), it falls back to a simpler arithmetic spacing. The returned list is sorted descending so that the sampling loop starts from the highest noise level.

ddim_sample (equations (9), (29)–(30))

def ddim_sample(self, denoiser: Callable, x_K: torch.Tensor,
                cond: Optional[Dict[str, Any]] = None,
                sub_grid: Optional[List[int]] = None,
                eta: float = 0.2) -> torch.Tensor:
    if sub_grid is None:
        sub_grid = list(range(self.K, 0, -1))

    sub_grid = sorted(sub_grid, reverse=True)
    x = x_K.clone()

    for i, k in enumerate(sub_grid):
        k_prev = sub_grid[i + 1] if i + 1 < len(sub_grid) else 0

        sqrt_alpha_bar_k = self.sqrt_alpha_bar[k]
        sqrt_one_minus_alpha_bar_k = self.sqrt_one_minus_alpha_bar[k]
        inv_sqrt_alpha_bar_k = self.inv_sqrt_alpha_bar[k]

        epsilon_theta = denoiser(x, k, cond)

        # Equation (7): estimate clean signal
        x_hat_0 = (x - sqrt_one_minus_alpha_bar_k * epsilon_theta) * inv_sqrt_alpha_bar_k

        if k_prev > 0:
            sqrt_alpha_bar_prev = self.sqrt_alpha_bar[k_prev]
            one_minus_alpha_bar_prev = self.one_minus_alpha_bar[k_prev]
            one_minus_alpha_bar_k = self.one_minus_alpha_bar[k]

            # Equation (29): sigma_k(eta) adapted for sub-grid
            ratio = (one_minus_alpha_bar_prev / one_minus_alpha_bar_k).clamp(min=0.0)
            sigma = eta * torch.sqrt(ratio) * torch.sqrt(1.0 - self.alpha_bar[k] / self.alpha_bar[k_prev])

            noise = torch.randn_like(x) if eta > 0 else 0.0

            sqrt_one_minus_alpha_bar_prev_minus_sigma2 = torch.sqrt(
                (one_minus_alpha_bar_prev - sigma**2).clamp(min=0.0)
            )

            # Equation (30): DDIM update
            x = sqrt_alpha_bar_prev * x_hat_0 \
                + sqrt_one_minus_alpha_bar_prev_minus_sigma2 * epsilon_theta \
                + sigma * noise
        else:
            x = x_hat_0

    return x

Explanation: This implements the core sampling loop. Starting from pure noise x_K, it iterates over the (possibly accelerated) sub‑grid in descending order. For each step k:

1. Predict the noise epsilon_theta using the denoiser (our U‑GNN).

2. Compute x_hat_0 via equation (7): (x_k - sqrt(1 - alpha_bar_k) * epsilon_theta) / sqrt(alpha_bar_k).

3. If we are not at the final step (k_prev > 0), compute the DDIM noise scale sigma using equation (29). The key adaptation for the sub‑grid is that we use alpha_bar[k_prev] and alpha_bar[k] instead of consecutive indices. We clamp intermediate values to avoid numerical issues near zero.

4. Apply the update of equation (30): x_{k_prev} = sqrt(alpha_bar_prev) * x_hat_0 + sqrt(1 - alpha_bar_prev - sigma^2) * epsilon_theta + sigma * noise.

5. At the final step (k_prev = 0), assign x = x_hat_0 directly.

The denoiser is a callable that takes (x, k, cond) and returns the predicted noise with the same shape as x. The cond dictionary carries the graph shift operator S and the node states u (and, for the stock task, the historical window).

Design decision: We treat the sub‑grid update as a drop‑in replacement for consecutive-step DDIM. The formula for sigma must be adapted because alpha_bar[k_prev] and alpha_bar[k] are not adjacent in the original 500‑step schedule. However, the derivation in Appendix A of the paper is valid for any two indices k_prev < k, and our implementation follows exactly that generalization.

5.2 Strided Graph Convolution (gnn_layers.py)

File purpose: Implements Algorithm 1 and the stacked GNN module used inside the U‑GNN.

StridedGraphConv.forward (Algorithm 1)

class StridedGraphConv(nn.Module):
    def __init__(self, in_dim, out_dim, K, gamma, activation=nn.ReLU()):
        super().__init__()
        self.weight = nn.Parameter(torch.randn(K + 1, in_dim, out_dim) * 0.1)
        self.bias = nn.Parameter(torch.zeros(out_dim))
        # ... store K, gamma, activation

    def forward(self, Z_prev: torch.Tensor, S: torch.Tensor,
                D: torch.Tensor) -> torch.Tensor:
        N = S.shape[0]

        # ---- Lift: D^T Z_prev ----
        x = torch.mm(D.T, Z_prev)

        # ---- Tap 0 ----
        A = torch.mm(x, self.weight[0])

        # ---- Iterated unit shifts ----
        for j in range(1, self.gamma * self.K + 1):
            if S.is_sparse:
                x = torch.sparse.mm(S, x)
            else:
                x = torch.mm(S, x)
            if j % self.gamma == 0:
                tap_idx = j // self.gamma
                A = A + torch.mm(x, self.weight[tap_idx])

        # ---- Nonlinearity + bias ----
        A = self.activation(A + self.bias)

        # ---- Reduce: D A ----
        out = torch.mm(D, A)
        return out

Explanation: This is the direct implementation of Algorithm 1 (Appendix B). The layer processes a signal Z_prev defined on N' active nodes. The steps are:

1. Lift (D^T Z_prev): Zero‑pad the signal from N' nodes to all N nodes using the transpose of the binary selection matrix D.

2. Tap 0: Apply the weight matrix for k=0 (the identity shift).

3. Iterated unit shifts: Run a loop up to gamma * K. In each iteration, apply one sparse shift (S @ x). Every gamma‑th iteration, treat it as a tap: accumulate the contribution x @ W[tap_idx].

4. Nonlinearity + bias: Apply activation (ReLU) and bias.

5. Reduce (D @ A): Project back from N nodes to the N' active nodes using the same selection matrix.

The total number of taps is K+1 (tap 0 plus taps at indices gamma, 2*gamma, ..., gamma*K). The stride gamma determines how many unit shifts occur between taps, effectively implementing (S^gamma)^k without ever forming the dense S^gamma matrix. This is the key efficiency trick.

Shape invariants: Z_prev: (N', in_dim), S: (N, N) (sparse or dense), D: (N', N) (binary, each row sums to 1). The output is (N', out_dim).

GNNModule

class GNNModule(nn.Module):
    def __init__(self, in_dim, out_dim, K, gamma, L=2, dropout=0.1):
        super().__init__()
        self.layers = nn.ModuleList()
        for i in range(L):
            dim_in = in_dim if i == 0 else out_dim
            self.layers.append(
                StridedGraphConv(dim_in, out_dim, K, gamma, activation=nn.ReLU())
            )
        self.norm = nn.LayerNorm(out_dim)
        self.dropout = nn.Dropout(dropout)

    def forward(self, X, S, D):
        h = X
        for layer in self.layers:
            h = layer(h, S, D)
            h = self.norm(h)
            h = self.dropout(h)
        return h

Explanation: A simple stack of L strided graph convolution layers (default 2, as per Table I), each followed by LayerNorm and dropout. All layers share the same gamma and D for a given depth level. The first layer expands from in_dim to out_dim; subsequent layers stay at out_dim. The module outputs a signal on the same active node set.

5.3 Learned Node Selection (node_selection.py)

File purpose: Implements the scoring, Gumbel‑Top‑K, and straight‑through estimator described in Section IV‑B and Appendix C.

NodeSelectionHead

class NodeSelectionHead(nn.Module):
    def __init__(self, in_dim, global_dim, hidden_dim=64):
        super().__init__()
        self.score_mlp = nn.Sequential(
            nn.Linear(in_dim + global_dim, hidden_dim),
            nn.ReLU(),
            nn.Linear(hidden_dim, 1),
        )

    def forward(self, Z, global_emb, n_keep, training=True, epoch=0,
                annealing_params=None):
        N_b = Z.shape[0]
        device = Z.device

        # Equation (24): score = MLP(concat(Z, global_emb))
        concat = torch.cat([Z, global_emb], dim=-1)
        scores = self.score_mlp(concat).squeeze(-1)  # (N_b,)

        if training:
            tau, eps = self._anneal_tau_eps(epoch, annealing_params)

            if eps > 0:
                # Equation (31): add Gumbel noise
                U = torch.rand_like(scores)
                gumbel = -torch.log(-torch.log(U + 1e-8))
                scores = scores + eps * gumbel

            # Equation (25): Top-K selection
            _, indices = torch.topk(scores, k=n_keep, dim=-1)
            indices = indices.sort()[0]

            C = torch.zeros(n_keep, N_b, device=device, dtype=Z.dtype)
            C[torch.arange(n_keep, device=device), indices] = 1.0

            # Equation (32): straight-through estimator
            mask_hard = torch.zeros(N_b, device=device, dtype=Z.dtype)
            mask_hard.scatter_(0, indices, 1.0)
            mask_soft = torch.sigmoid(scores / tau)
            mask = mask_hard + mask_soft - mask_soft.detach()

            # Equation (33): down-sample with mask
            masked_Z = Z * mask.unsqueeze(-1)
            V = masked_Z[indices]
        else:
            # Inference: hard selection only
            _, indices = torch.topk(scores, k=n_keep, dim=-1)
            indices = indices.sort()[0]
            C = torch.zeros(n_keep, N_b, device=device, dtype=Z.dtype)
            C[torch.arange(n_keep, device=device), indices] = 1.0
            V = Z[indices]

        return C, V

Explanation: The node selection head does exactly what Section IV‑B describes. The scoring MLP Psi_b (equation (24)) takes the concatenation of the encoded feature Z and the global embeddings projected to the current active node set, and outputs a scalar per node. During training:

• Gumbel noise is added to the scores (equation (31)). eps is annealed from 1 down to 0 over training, so exploration decays.

• Top‑K picks the n_keep highest‑scoring nodes.

• Straight‑through estimator (equation (32)): the forward pass uses a hard one‑hot mask (mask_hard), while gradients flow through a sigmoid surrogate (mask_soft). The trick mask = hard + soft - soft.detach() ensures the forward pass is hard and the backward pass sees soft.

• Down‑sampled signal V is computed as C @ (mask * Z) (equation (33)).

At inference, Gumbel noise and the STE are removed: only hard Top‑K is used.

Annealing Schedule (_anneal_tau_eps)

@staticmethod
def _anneal_tau_eps(epoch: int, annealing_params: dict) -> (float, float):
    if annealing_params is None:
        Tw, Ta, tau0, tau_min, eps0, eps_min = 0.0, 1.0, 1.0, 0.5, 1.0, 0.0
    else:
        Tw = annealing_params.get('Tw', 0.02)
        Ta = annealing_params.get('Ta', 0.73)
        tau0 = annealing_params.get('tau0', 1.0)
        tau_min = annealing_params.get('tau_min', 0.5)
        eps0 = annealing_params.get('eps0', 1.0)
        eps_min = annealing_params.get('eps_min', 0.0)

    # Equation (34): r(t) = min(max((t - Tw) / Ta, 0), 1)
    r = (epoch - Tw) / Ta
    r = max(0.0, min(1.0, r))

    tau = tau0 + (tau_min - tau0) * r
    eps = eps0 + (eps_min - eps0) * r

    return tau, eps

Explanation: Implements equation (34) exactly. After a warm‑up phase of Tw epochs (default 0.02 × total = 100 epochs for 5000 total), the ratio r increases linearly from 0 to 1 over Ta epochs (default 0.73 × total = 3650 epochs). Both tau and eps are linearly interpolated from their initial to their final values. After Tw + Ta epochs, r = 1 and the parameters stay at their minima.

5.4 U‑GNN Architecture (ugnn.py)

File purpose: Assembles all components into the full U‑GNN as described in Section IV and equations (18)–(23).

UGNN.__init__

class UGNN(nn.Module):
    def __init__(self, task: str, config: Config):
        super().__init__()
        self.task = task
        self.B = config.B          # 4 resolution levels
        self.rho = config.rho      # 2 (pooling factor)
        self.F0 = config.F0        # 64 (feature dim)
        self.L = config.L          # 2 (GNN layers per block)
        self.K = config.K          # 2 (hops)
        self.gamma_max = config.gamma_max  # 2

        # Core layers
        self.read_in = ReadInLayer(in_dim=config.F_out, out_dim=self.F0)
        self.read_out = ReadOutLayer(in_dim=self.F0, out_dim=self.F_out)
        self.node_state_encoder = NodeStateEncoder(in_dim=config.U, out_dim=self.F0)
        self.step_embedding = StepEmbedding(out_dim=self.F0)

        if task == 'stock':
            self.temporal_mixer = TemporalMixer(...)
        else:
            self.temporal_mixer = nn.Identity()

        # Encoder blocks
        self.encoder_blocks = nn.ModuleList()
        self.fusion_layers_enc = nn.ModuleList()
        self.selection_heads = nn.ModuleList()
        for b in range(1, self.B):
            gamma = self._compute_gamma(b)
            gnn = GNNModule(self.F0, self.F0, self.K, gamma, self.L, self.dropout)
            fusion = FusionLayer(self.F0, 2*self.F0, self.embed_dim, self.task)
            sel = NodeSelectionHead(self.F0, 2*self.F0)
            self.encoder_blocks.append(gnn)
            self.fusion_layers_enc.append(fusion)
            self.selection_heads.append(sel)

        # Bottleneck
        self.bottleneck_gamma = self._compute_gamma(self.B)
        self.bottleneck_gnn = GNNModule(self.F0, self.F0, self.K,
                                         self.bottleneck_gamma, self.L, self.dropout)
        self.bottleneck_fusion = FusionLayer(self.F0, 2*self.F0, self.embed_dim, self.task)

        # Decoder blocks
        self.decoder_blocks = nn.ModuleList()
        self.fusion_layers_dec = nn.ModuleList()
        self.skip_projections = nn.ModuleList()
        for b in range(self.B - 1, 0, -1):
            gamma = self._compute_gamma(b)
            gnn = GNNModule(self.F0, self.F0, self.K, gamma, self.L, self.dropout)
            fusion = FusionLayer(self.F0, 2*self.F0, self.embed_dim, self.task)
            skip_proj = SkipProjection(in_dim=2*self.F0, out_dim=self.F0)
            self.decoder_blocks.append(gnn)
            self.fusion_layers_dec.append(fusion)
            self.skip_projections.append(skip_proj)

Explanation: The constructor builds the full U‑GNN. It creates B-1 encoder blocks (b = 1 to B-1), one bottleneck block (b = B), and B-1 decoder blocks (b = B-1 down to 1). Each block has its own GNNModule with a depth‑dependent gamma computed by _compute_gamma, a FusionLayer, and (for encoder blocks) a NodeSelectionHead. The decoder blocks also get a SkipProjection layer to reduce the concatenated skip connection to the standard feature dimension F0.

The FusionLayer and TemporalMixer are task‑specific: the stock task uses cross‑attention in the fusion and a temporal mixer (dilated conv + self‑attention) on the read‑in features; the WRA task uses simple addition in the fusion and no temporal mixer.

UGNN.forward (equations (18)–(23))

def forward(self, x_k, k, u, S):
    batch, N, F_in = x_k.shape

    # 1. Read-in
    Z0 = self.read_in(x_k)                     # (batch, N, F0)
    U0 = self.node_state_encoder(u)            # (batch, N, F0)
    K0 = self.step_embedding(k).unsqueeze(1).expand(-1, N, -1)  # (batch, N, F0)

    # 2. Global embeddings
    global_emb = torch.cat([U0, K0], dim=-1)   # (batch, N, 2*F0)

    # 3. Temporal mixer (stock only)
    Z0 = self.temporal_mixer(Z0)

    # 4. Node counts
    node_counts = self._compute_target_node_counts(N)

    # 5. Encoder path
    D = torch.eye(N, device=x_k.device).unsqueeze(0).expand(batch, -1, -1)
    Z = Z0
    encoder_outputs = []
    for b_idx in range(self.B - 1):
        next_N = node_counts[b_idx + 1]
        gnn = self.encoder_blocks[b_idx]
        fusion = self.fusion_layers_enc[b_idx]
        sel_head = self.selection_heads[b_idx]

        # Fusion: P_b = Π_b(V_b, D_b[U0;K0])
        # (V_b is Z after down-sampling via C_b; here we use D to project global_emb)
        V_b = Z  # For now, V_b = Z (in practice we need C_b; see full code)
        global_emb_active = torch.bmm(D, global_emb)
        P_b = fusion(V_b, global_emb_active)

        Z_b = gnn(P_b, S, D)

        # Selection: C_{b+1}, Z_{b+1}
        C_next, Z_next = sel_head(Z_b, global_emb_active, next_N,
                                   training=self.training,
                                   epoch=self.current_epoch)

        # Update D for next level: D_{b+1} = C_{b+1} @ D_b
        D = torch.bmm(C_next, D)
        encoder_outputs.append(Z_b)
        Z = Z_next

    # 6. Bottleneck
    # ... (similar fusion + GNN on coarsest level)

    # 7. Decoder path
    # ... (up-sample via C^T, skip-concat, fusion, GNN)

    # 8. Read-out
    epsilon_theta = self.read_out(Y_1)
    return epsilon_theta

(Note: The above is a simplified sketch. The full code in `ugnn.py` handles batching, indexing, and the decoder loop with proper up‑sampling and skip connections.)

Explanation of the forward pass:

1. Read‑in and embeddings: The noisy signal x_k is projected to dimension F0 by a node‑wise MLP. Node states u and the diffusion step k are similarly encoded and broadcast to all nodes. The step embedding uses sinusoidal encoding (following DDPM conventions) followed by an MLP.

2. Global embeddings: U0 and K0 are concatenated along the feature dimension to form a (batch, N, 2*F0) tensor that carries all conditioning information.

3. Encoder path: Starting from Z0 and an identity D (active nodes = all N nodes), we iterate through the encoder blocks. At each depth b:

• Project global embeddings onto the current active node set: D_b * global_emb.

• Fuse (equation (22)): P_b = Π_b(V_b, D_b[U0;K0]).

• Apply the GNN module with stride gamma_b: Z_b = Φ_{E_b}(P_b, S; D_b, gamma_b).

• Select nodes for the next level: C_{b+1}, Z_{b+1} = Selector(Z_b, D_b[U0;K0], N_{b+1}).

• Update the composite selection matrix: D = C_{b+1} @ D (this builds D_{b+1} as per equation (18)).

4. Decoder path (omitted for brevity): The coarsest representation is processed by the bottleneck block. Then, iterating backwards, each decoder block up‑samples via C_{b+1}^T, concatenates with the corresponding encoder output (skip connection), projects via SkipProjection (equation (23)), fuses, and applies a GNN module.

5. Read‑out: The finest‑resolution decoder output Y_1 is projected back to the original feature dimension by a node‑wise MLP.

Key design choices:

• The composite D matrices are maintained as dense (batch, N_b, N) tensors for simplicity. In a production system, you might store them as sparse index lists, but dense matrices make the batch matrix multiplications (torch.bmm) straightforward.

• The FusionLayer is shared between encoder and decoder blocks only in the sense that the same class is instantiated with the same input/output dimensions; the paper mentions weight sharing, but we instantiate separate layers for clarity.

• The TemporalMixer applies dilated 1D convolutions along the temporal axis (last dimension) followed by multi‑head self‑attention. The output is aggregated by taking the mean over the temporal dimension, producing a single feature vector per node. This is a reasonable default for the paper’s underspecified temporal encoder.

5.5 Putting It All Together

The four files work together seamlessly in the training loop (train.py):

1. DiffusionProcess.forward_diffuse generates x_k and epsilon from a clean x_0 sampled from the dataloader.

2. The UGNN takes (x_k, k, u, S) and predicts epsilon_theta.

3. The MSE loss between epsilon and epsilon_theta is computed (equation (6)).

4. Gradients flow through the U‑GNN, including through the straight‑through estimator in NodeSelectionHead, and the parameters are updated.

5. After training, DiffusionProcess.ddim_sample uses the trained U‑GNN to generate samples from pure noise, producing the final forecast trajectories.

The modularity of the design—separate files for diffusion, graph convolution, node selection, and the full architecture—makes it easy to test, modify, or replace individual components without touching the rest of the pipeline.

6. Training and Hyperparameters

With the U-GNN architecture and data pipelines in place, we now turn to the training procedure. The paper specifies a detailed set of hyperparameters (Table I and Appendix E) and a training loop that includes a custom learning rate schedule, gradient clipping, automatic mixed precision, and a composite validation criterion. This section walks through the configuration object, the scheduler, the training loop itself, and the checkpointing logic, tying each piece to the generated code.

6.1 Configuration from Table I and Appendix E

All hyperparameters are centralized in config.py. The base Config class stores values that are shared across tasks, while StockConfig and WRAConfig override task-specific settings. The factory function get_config(task) returns the appropriate instance.

Key hyperparameters (from Table I and Appendix E):

| Parameter | Value | Paper Reference | |-----------|-------|-----------------| | Diffusion steps K | 500 | Table I | | \beta_1, \beta_{500} | 10^{-4}, 2\times10^{-2} | Table I, linear schedule | | DDIM steps | 100 | Table I | | DDIM \eta | 0.2 | Table I | | U-GNN depths B | 4 | Table I | | Pooling factor \rho | 2 | Table I | | Channel width F | 64 | Table I | | GNN layers per block L | 2 | Table I | | Filter taps K (hops) | 2 | Table I | | Max stride \gamma_{\max} | 2 | Table I | | Dropout | 0.1 | Table I | | Embedding dimension | 128 | Table I | | Epochs | 5000 | Table I | | Batch size (stock) | 64 | Table I | | Batch size (WRA) | 800 | Table I | | Learning rate | 10^{-3} | Table I | | Weight decay | 0.0 | Table I (AdamW default) | | Warmup fraction | 0.01 (1%) | Appendix E | | Decay fraction | 0.80 (80%) | Appendix E | | Decay to fraction | 0.05 (5% of peak) | Appendix E | | Gradient clip norm | 1.0 | Appendix E | | AMP | True | Appendix E | | Annealing warmup T_w | 0.02 (100 epochs) | Appendix C | | Annealing decay T_a | 0.73 (3650 epochs) | Appendix C | | Initial temperature \tau_0 | 1.0 | Appendix C | | Final temperature \tau_{\min} | 0.5 | Appendix C | | Initial exploration noise \varepsilon_0 | 1.0 | Appendix C | | Final exploration noise \varepsilon_{\min} | 0.0 | Appendix C | | RevIN blend (stock) | 0.7 | Section V-A | | RevIN scale correction (stock) | 1.107 | Section V-A |

The Config.__init__ method calls _compute_betas() and _compute_alphabars() to precompute the linear \beta schedule and the cumulative \bar{\alpha}_k array (equations (2)–(3)). These are stored as tensors and used later by the DiffusionProcess class.

# From config.py (simplified)
class Config:
    def __init__(self, task: str = "stock"):
        self.task = task
        if task == "stock":
            self.batch_size = 64
            self.U = 12
            self.Th = 20
            self.Tp = 5
        elif task == "wra":
            self.batch_size = 800
            self.U = 2
            self.Th = 0
            self.Tp = 1
        else:
            raise ValueError(f"Unknown task: {task}")
        self._compute_betas()
        self._compute_alphabars()

    def _compute_betas(self):
        k = torch.arange(1, self.K + 1, dtype=torch.float32)
        self.betas = self.beta_start + (k - 1) / (self.K - 1) * (self.beta_end - self.beta_start)

    def _compute_alphabars(self):
        self.alphas = 1.0 - self.betas
        self.alphabars = torch.cumprod(self.alphas, dim=0)

6.2 WarmupCosineLR Scheduler

The paper describes a learning rate schedule with a linear warm-up over the first 1% of training steps, followed by cosine decay to 5% of the peak learning rate over the next 80% of steps, and then constant at that minimum. The WarmupCosineLR class in train.py implements this exactly.

class WarmupCosineLR(optim.lr_scheduler.LambdaLR):
    def __init__(self, optimizer, total_steps, warmup_frac=0.01,
                 decay_frac=0.80, min_lr_frac=0.05, last_epoch=-1):
        self.total_steps = total_steps
        self.warmup_steps = int(total_steps * warmup_frac)
        self.decay_steps = int(total_steps * decay_frac)
        self.min_lr_frac = min_lr_frac
        super().__init__(optimizer, self._lr_lambda, last_epoch=last_epoch)

    def _lr_lambda(self, step):
        if step < self.warmup_steps:
            return float(step) / max(1.0, self.warmup_steps)
        elif step < self.warmup_steps + self.decay_steps:
            progress = float(step - self.warmup_steps) / max(1.0, self.decay_steps)
            cosine_decay = 0.5 * (1.0 + np.cos(np.pi * progress))
            return self.min_lr_frac + (1.0 - self.min_lr_frac) * cosine_decay
        else:
            return self.min_lr_frac

The scheduler is instantiated with total_steps = epochs * batches_per_epoch. Because the number of batches per epoch can vary, we estimate it from the training DataLoader. The scheduler is stepped after every optimizer step (i.e., every mini-batch), not every epoch.

6.3 Training Loop

The train_epoch function implements one epoch of training. For each batch, it:

1. Samples a diffusion step k \sim \text{Uniform}\{1, \dots, K\} independently for each element in the batch.

2. Samples noise \varepsilon \sim \mathcal{N}(0, I) with the same shape as the clean signal x_0.

3. Computes the noisy signal x_k = \sqrt{\bar{\alpha}_k}\,x_0 + \sqrt{1-\bar{\alpha}_k}\,\varepsilon using the precomputed \bar{\alpha} array (equation (4)).

4. Predicts the noise \varepsilon_\theta = \text{U-GNN}(x_k, k, u, S).

5. Computes the MSE loss \|\varepsilon - \varepsilon_\theta\|^2 (equation (6)).

6. Backpropagates with gradient clipping (max L2 norm = 1.0) and automatic mixed precision (AMP) if CUDA is available.

7. Updates the optimizer and steps the scheduler.

# From train.py (simplified)
def train_epoch(model, dataloader, optimizer, scheduler, diffusion,
                epoch, config, scaler=None):
    model.train()
    if hasattr(model, 'set_epoch'):
        model.set_epoch(epoch)  # for annealing τ and ε

    total_loss = 0.0
    n_batches = 0

    for batch in dataloader:
        if config.task == 'stock':
            x0, u, S, _ = batch
        else:
            x0, S, u = batch

        x0 = x0.to(device, non_blocking=True)
        u = u.to(device, non_blocking=True)
        S = S.to(device, non_blocking=True)

        batch_size = x0.size(0)
        k = torch.randint(1, config.K + 1, (batch_size,), device=device)
        noise = torch.randn_like(x0)
        x_k = diffusion.forward_diffuse(x0, k, noise)

        with autocast(enabled=scaler is not None):
            pred_noise = model(x_k, k, u, S)
            loss = F.mse_loss(pred_noise, noise)

        if scaler is not None:
            scaler.scale(loss).backward()
            scaler.unscale_(optimizer)
            torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
            scaler.step(optimizer)
            scaler.update()
        else:
            loss.backward()
            torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)
            optimizer.step()

        optimizer.zero_grad(set_to_none=True)
        scheduler.step()

        total_loss += loss.item()
        n_batches += 1

    return total_loss / max(1, n_batches)

Key details:

• The DiffusionProcess.forward_diffuse method implements equation (4) by indexing into the precomputed alpha_bar tensor with the sampled k values.

• Gradient clipping uses torch.nn.utils.clip_grad_norm_ with max_norm=1.0 and the default L2 norm.

• AMP is enabled via torch.cuda.amp.GradScaler and autocast. The scaler is created only if CUDA is available.

• The model’s set_epoch method (if present) is called at the start of each epoch to update the temperature \tau and exploration noise \varepsilon for the node selection heads, following the annealing schedule of Appendix C.

6.4 Annealing of τ and ε

The node selection heads use a Gumbel-Top-K mechanism during training, with temperature \tau and exploration noise \varepsilon that are annealed over time. The schedule (equation (34)) has three phases:

• Warm-up (first T_w = 0.02 \times 5000 = 100 epochs): \tau and \varepsilon are held at their initial values (1.0 and 1.0).

• Anneal (next T_a = 0.73 \times 5000 = 3650 epochs): both parameters decay linearly to their final values (0.5 and 0.0).

• Constant (remaining epochs): stay at final values.

In the code, the NodeSelectionHead class (in node_selection.py) implements anneal_tau_eps(epoch) which computes the current values based on the schedule. The UGNN model’s set_epoch method propagates the epoch number to all selection heads so they can update their internal parameters before each forward pass.

6.5 Composite Validation Criteria

The paper uses a task-specific composite score to select the best checkpoint during training. The composite criterion combines multiple validation metrics into a single scalar.

For stock forecasting: The composite score is a weighted sum of the noise-prediction loss, the gap between training and validation loss (to detect overfitting), and the CRPS (Continuous Ranked Probability Score). Lower is better.

def composite_criterion_stock(val_metrics):
    loss = val_metrics.get('loss', 1.0)
    crps = val_metrics.get('crps', 1.0)
    # train-val gap not computed here, assume 0
    return loss + 0.5 * crps

For wireless resource allocation: The composite score combines the mean ergodic rate and the feasibility (fraction of users meeting the minimum rate constraint). Higher is better, so we negate the sum for minimization.

def composite_criterion_wra(val_metrics):
    mean_rate = val_metrics.get('mean_rate', 0.0)
    feasibility = val_metrics.get('feasibility', 0.0)
    return -(mean_rate + feasibility * 2.0)  # weight feasibility more

These functions are called after each validation epoch. The train function tracks the best composite score and saves the corresponding model checkpoint.

6.6 Checkpointing

At the end of each epoch, if the composite score improves (lower for stock, higher for WRA), the current model state, optimizer state, scheduler state, epoch number, and score are saved to a dictionary. After training completes, the best checkpoint is written to checkpoints/best_model.pth.

if (config.task == 'stock' and score < best_score) or \
   (config.task == 'wra' and score > best_score):
    best_score = score
    best_epoch = epoch
    best_state = {
        'model_state_dict': model.state_dict(),
        'optimizer_state_dict': optimizer.state_dict(),
        'scheduler_state_dict': scheduler.state_dict(),
        'epoch': epoch,
        'score': score,
        'config': config
    }

This checkpoint can later be loaded for evaluation or visualization.

6.7 Summary

The training configuration and loop are designed to match the paper’s specifications exactly. The Config class centralizes all hyperparameters, the WarmupCosineLR scheduler implements the custom learning rate schedule, and the train_epoch function handles the diffusion-specific steps (sampling k, computing x_k, predicting noise) along with gradient clipping and AMP. The annealing of node selection parameters is integrated via the model’s set_epoch hook, and the composite validation criteria guide checkpoint selection. This modular design makes it straightforward to reproduce the paper’s experiments and to adapt the code to new tasks.

7. Evaluation Metrics and Baselines

A generative model is only as useful as the metrics we use to judge it. The paper evaluates its U-GNN diffusion model against two sets of baselines and a battery of metrics that go far beyond simple point-prediction errors. For stock forecasting, the goal is to assess both the accuracy of the mean prediction and the fidelity of the full predictive distribution. For wireless resource allocation, the focus is on the ergodic sum rate, feasibility, and the gap to an expert algorithm. This section explains every metric and baseline, shows the corresponding code in eval.py and baselines.py, and describes how to compare the U-GNN against the baselines.

7.1 Stock Forecasting Metrics

The paper reports eight metrics for the S&P 500 task (Table II). They fall into three categories: distributional accuracy (CRPS, MIS90), point-prediction accuracy (RMSE, MAE, direction accuracy), and stylized-fact gaps (volatility clustering, momentum, excess kurtosis). All metrics are computed on the returns (log-returns) unless otherwise noted.

7.1.1 Continuous Ranked Probability Score (CRPS)

CRPS measures how well the empirical distribution of generated samples matches the true target. It is defined as the integral of the squared difference between the empirical CDF of the samples and the indicator function of the target:

For a set of M samples, the empirical CDF is a step function, and the integral can be computed analytically using the sorted samples. The implementation in StockEvaluator.compute_crps uses the formula:

where s_{(i)} are the sorted samples and y is the target. The double sum is computed efficiently using the fact that for sorted samples, \sum_{i,j} |s_{(i)} - s_{(j)}| = 2 \sum_{i=1}^{M} (2i - M - 1) s_{(i)}.

@staticmethod
def compute_crps(samples: np.ndarray, target: np.ndarray) -> float:
    samples = np.asarray(samples)
    target = np.asarray(target)
    if samples.ndim == 3:
        num_samples = samples.shape[0]
    else:
        num_samples = 1
        samples = samples[np.newaxis, ...]
    m = samples.shape[1] * samples.shape[2]
    samples_flat = samples.reshape(num_samples, -1)
    target_flat = target.reshape(-1)
    sorted_samples = np.sort(samples_flat, axis=0)
    M = num_samples
    abs_diff = np.mean(np.abs(sorted_samples - target_flat[np.newaxis, :]), axis=0)
    weights = 2 * (2 * np.arange(1, M + 1) - M - 1)
    sum_abs_diff_sorted = np.sum(weights[:, np.newaxis] * sorted_samples, axis=0) / (M ** 2)
    crps_per_element = abs_diff - 0.5 * sum_abs_diff_sorted
    return float(np.mean(crps_per_element))

7.1.2 Mean Interval Score for 90% Prediction Interval (MIS90)

MIS90 evaluates the sharpness and calibration of the 90% prediction interval. It penalizes intervals that are too wide and intervals that miss the target:

where l and u are the 5th and 95th percentiles of the samples, and \alpha = 0.1. The implementation computes these percentiles using np.percentile.

@staticmethod
def compute_mis90(samples: np.ndarray, target: np.ndarray) -> float:
    samples = np.asarray(samples)
    target = np.asarray(target)
    if samples.ndim == 2:
        samples = samples[np.newaxis, ...]
    alpha = 0.1
    lower = np.percentile(samples, 5, axis=0)
    upper = np.percentile(samples, 95, axis=0)
    width = upper - lower
    penalty_low = (2.0 / alpha) * (lower - target) * (target < lower)
    penalty_high = (2.0 / alpha) * (target - upper) * (target > upper)
    mis = width + penalty_low + penalty_high
    return float(np.mean(mis))

7.1.3 RMSE, MAE, and Direction Accuracy

These are standard point-prediction metrics. The mean prediction is the average over the M samples. RMSE and MAE are computed between this mean and the target. Direction accuracy measures the fraction of elements where the sign of the mean prediction matches the sign of the target (ignoring zeros).

@staticmethod
def compute_rmse(samples: np.ndarray, target: np.ndarray) -> float:
    samples = np.asarray(samples)
    target = np.asarray(target)
    if samples.ndim == 3:
        mean_pred = np.mean(samples, axis=0)
    else:
        mean_pred = samples
    return float(np.sqrt(np.mean((mean_pred - target) ** 2)))

@staticmethod
def compute_dir_accuracy(samples: np.ndarray, target: np.ndarray) -> float:
    samples = np.asarray(samples)
    target = np.asarray(target)
    if samples.ndim == 3:
        mean_pred = np.mean(samples, axis=0)
    else:
        mean_pred = samples
    correct = (np.sign(mean_pred) == np.sign(target))
    return float(np.mean(correct))

7.1.4 Stylized-Fact Gaps

Financial time series exhibit well-known statistical regularities called stylized facts. The paper measures how well the generated samples reproduce three of them:

• Volatility clustering: the tendency for large changes to be followed by large changes. Quantified as the lag-1 autocorrelation of squared returns.

• Momentum: the tendency for returns to be positively autocorrelated at short lags. Quantified as the lag-1 autocorrelation of returns.

• Excess kurtosis: the fat-tailed nature of return distributions. Quantified as the sample excess kurtosis (kurtosis minus 3).

For each metric, the gap is the absolute difference between the value computed on the target and the value computed on the pooled generated samples. The implementation in StockEvaluator computes these using NumPy.

@staticmethod
def compute_volatility_gap(samples: np.ndarray, target: np.ndarray) -> float:
    samples = np.asarray(samples)
    target = np.asarray(target)
    target_flat = target.ravel()
    samples_flat = samples.ravel()

    def autocorr_lag1(x):
        if len(x) < 2:
            return 0.0
        x0 = x[:-1]
        x1 = x[1:]
        std0 = np.std(x0)
        std1 = np.std(x1)
        if std0 == 0 or std1 == 0:
            return 0.0
        return float(np.corrcoef(x0, x1)[0, 1])

    target_vol = autocorr_lag1(target_flat ** 2)
    samples_vol = autocorr_lag1(samples_flat ** 2)
    return float(np.abs(target_vol - samples_vol))

7.2 Wireless Resource Allocation Metrics

For the WRA task, the paper reports ergodic sum rate, feasibility, and the gap to the expert primal-dual algorithm (Table IV).

7.2.1 Ergodic Sum Rate

The ergodic sum rate is the average over fading realizations of the sum of per-user rates. The per-user rate is (equation (26)):

where \text{SINR}_j = \frac{p_j g_{jj}}{\sum_{i \neq j} p_i g_{ij} + \sigma^2}. The ergodic estimate (equation (27)) averages over T = 500 fading realizations, with the allocation held constant for T_0 = 5 slots. The implementation in WRAEvaluator.compute_ergodic_rate generates independent Rayleigh fading (exponential squared magnitude) and computes the SINR for each slot.

@staticmethod
def compute_ergodic_rate(
    allocations: Union[np.ndarray, torch.Tensor],
    H: Union[np.ndarray, torch.Tensor],
    noise_power: float = 1e-11,
    T: int = 500,
    T0: int = 5
) -> float:
    allocations = np.asarray(allocations).flatten()
    H = np.asarray(H)
    N = len(allocations)
    # Uncenter if needed (assume Pmax=1)
    if np.any(allocations < 0):
        allocations = allocations + 1.0
    allocations = np.clip(allocations, 0.0, 1.0)
    # Generate fading: (N, N, T) exponential with mean 1
    fading = np.random.exponential(scale=1.0, size=(N, N, T)).astype(np.float64)
    H = np.abs(H)
    g = H[:, :, np.newaxis] * fading
    g_self = g.diagonal(axis1=0, axis2=1)
    p = allocations[:, np.newaxis, np.newaxis]
    total_received = np.sum(p * g, axis=0)
    self_power = p[:, 0, 0] * g_self
    interference = total_received - self_power
    sinr = self_power / (interference + noise_power)
    rate = np.log2(1.0 + np.maximum(sinr, 0.0))
    sum_rate_per_slot = np.sum(rate, axis=0)
    mean_sum_rate = np.mean(sum_rate_per_slot)
    return float(mean_sum_rate)

7.2.2 Feasibility

Feasibility is the fraction of users whose ergodic rate meets a minimum rate constraint (e.g., 0.5 bits/s/Hz). The implementation computes the per-user ergodic rate (mean over fading) and checks the constraint.

7.2.3 Gap to Expert

The paper reports the percentage gap between the U-GNN and the expert primal-dual algorithm at the 1st, 5th, 10th percentiles and the mean. The gap is defined as:

where p indicates the percentile of the ergodic sum rate distribution across test networks.

7.3 Baselines

The paper compares against three baselines, implemented in baselines.py.

7.3.1 Geometric Random Walk (GRW)

For stock forecasting, the GRW assumes log-returns are independent Gaussian with variance estimated from the history window. The GeometricRandomWalk.forecast method computes the per-stock variance from the 20-day history and samples M independent trajectories.

class GeometricRandomWalk:
    def forecast(self, history: torch.Tensor, steps: int, num_samples: int = 1) -> torch.Tensor:
        if history.dim() == 3:
            history = history.squeeze(-1)
        N, Th = history.shape
        var = history.var(dim=1, unbiased=True, keepdim=False) + 1e-8
        std = torch.sqrt(var)
        noise = torch.randn(num_samples, N, steps, generator=self.rng, device=history.device, dtype=history.dtype)
        samples = noise * std.view(1, N, 1)
        return samples

7.3.2 Full Power (FP) and Average Power (AP)

For WRA, the FP baseline sets every transmitter to maximum power P_{\text{max}}, so the centered allocation is 0. The AP baseline sets every transmitter to P_{\text{max}}/2, so the centered allocation is -0.5. Both are deterministic and return tensors of the appropriate shape.

class FullPower:
    def allocate(self, network: dict = None, H: torch.Tensor = None, num_samples: int = 1) -> torch.Tensor:
        N = 400
        if H is not None:
            N = H.shape[0]
        device = H.device if H is not None else torch.device('cpu')
        return torch.zeros(num_samples, N, 1, device=device, dtype=torch.float32)

class AveragePower:
    def allocate(self, network: dict = None, H: torch.Tensor = None, num_samples: int = 1) -> torch.Tensor:
        N = 400
        if H is not None:
            N = H.shape[0]
        device = H.device if H is not None else torch.device('cpu')
        return torch.full((num_samples, N, 1), -0.5, device=device, dtype=torch.float32)

7.4 Comparing U-GNN Against Baselines

To compare the U-GNN against the baselines, we generate M = 100 samples from each method (for GRW, this is natural; for FP/AP, we repeat the deterministic allocation). Then we compute all metrics using the StockEvaluator or WRAEvaluator. The evaluate_all method returns a dictionary of metrics, which can be printed as a table (matching Table II or Table IV).

evaluator = StockEvaluator()
metrics_ugnn = evaluator.e

## 8. Results and Figures

A generative model’s output is inherently high-dimensional: for stock forecasting, we produce a distribution over future price paths for hundreds of stocks; for wireless resource allocation, we produce a distribution over power vectors. To communicate the quality of these distributions, the paper relies on a set of carefully designed figures that go beyond simple scalar metrics. This section walks through the four plotting functions in `visualize.py` that reproduce the paper’s key visual results: forecasting trajectories, empirical CDFs, distribution diagnostics (histogram + Q-Q plot), and a WRA performance bar chart. Each function is explained in detail, with code snippets and guidance on how to call them from the main evaluation script.

### 8.1 Overview of the Plotting Functions

The file `visualize.py` contains four public functions:

- `plot_forecast_trajectories` – reproduces the paper’s “forecasting trajectories” figures (e.g., Figure 2 in the paper, showing historical prices, generated samples, and actual target for selected stocks).
- `plot_cdf` – plots the empirical cumulative distribution function of the pooled generated samples versus the pooled target values, corresponding to the CDF panels in the paper’s distribution diagnostics.
- `plot_distribution_diagnostics` – creates a two-panel figure with an overlaid histogram and a Q-Q plot, matching the paper’s density and quantile comparisons.
- `plot_wra_bar` – produces a grouped bar chart comparing ergodic rate metrics (p1, p5, p10, mean) across methods (U-GNN, Expert, FP, AP), with an optional secondary axis for percentage gaps and an inset for feasibility. This reproduces the paper’s WRA performance bar chart (e.g., Figure 4).

All functions accept NumPy arrays (or torch tensors, which are automatically converted via the internal `_to_numpy` helper) and return a `matplotlib.figure.Figure` object that can be saved or displayed.

### 8.2 Forecasting Trajectories

The function `plot_forecast_trajectories` visualises the model’s predictive distribution for a few selected stocks. It plots:
- The historical log-returns (or prices) for the last `Th` days (blue line).
- A random subset of generated trajectories (gray lines, low opacity).
- The mean of all generated trajectories (red line).
- The actual target values (green dashed line).

The time axis is split into history (days 0 to Th-1) and forecast (days Th to Th+Tp-1).

def plotforecasttrajectories( prices: np.ndarray, samples: np.ndarray, target: np.ndarray, stockindices: Optional[List[int]] = None, numtrajectories: int = 20, figsize: Tuple[float, float] = (10, 6) ) -> plt.Figure:

Use the button or URL below to download the complete Python source code.

Read more

The paper proposes a quantitative trading pipeline for gold and Bitcoin. It combines price forecasting with an attention-enhanced bidirectional LSTM, position sizing based on empirical consecutive rises and declines, transaction-cost sensitivity analysis, and claimed use of VaR and a modified greedy algorithm for trading decisions. Several benchmark forecasting models are compared, with Att-BiLSTM reported as the best predictor. The trading results claim a final value of approximately $646 for a $500 gold allocation and $215,487 for a $500 Bitcoin allocation, although the backtest protocol and several model details are insufficiently specified.

The code is my implementation based on what the paper discusses. Download the source code using the button at the end of this article!

Also the code might have some errors, because I didn’t found anything online related to paper, everything is implemented by myself, so download it and understand. With article and the code.

This is the research paper I tried to implement: https://www.researchgate.net/publication/364569615_Research_on_Quantitative_Trading_Model

Our Book is out on openclaw:

Get Your Copy

Implementation Assumptions

• Historical gold and Bitcoin datasets are not supplied; the implementation will accept local CSV or pandas DataFrame inputs and include a deterministic synthetic-data generator for demonstrations only.

• Prices are assumed to be positive, chronologically ordered observations with one row per asset and an explicit timestamp.

• Standard simple return, (pt-p{t-1})/p_{t-1}, is used by default because the paper's printed denominator is ambiguous; the choice is configurable and documented.

• The primary target is a one-step-ahead normalized price or return forecast. A three-step vector forecast option is supported because the paper's conclusion mentions three-day forecasts.

• The primary forecasting model is an attention-enhanced BiLSTM with a configurable learned query vector, 20% dropout, MAE loss, and RMSprop optimizer.

• The paper-style 70:30 chronological split is supported for comparison, while walk-forward evaluation is the preferred protocol for trading conclusions.

• Equations (5) through (8), VaR operation, and the modified greedy algorithm are implemented as explicit reconstructions and will never be presented as exact reproductions.

• Transaction fees apply to both buys and sells by default, use decimal rates such as 0.002 for 0.2%, and are configurable.

• The backtest uses next-bar execution after a signal, finite position-addition levels, no leverage by default, and optional slippage.

• The reported final values of approximately 646 USD for gold and 215487 USD for Bitcoin are recorded as unverified reference claims rather than expected test outputs.

• Deep-learning, statistical, and optional benchmark dependencies are isolated so core data, sizing, risk, and backtesting components remain usable without every optional package installed.

1. The Paper’s Core Idea: Forecast Timing and Manage Position Size Separately

The paper is trying to answer two different trading questions:

When should the strategy trade?

How much capital should it deploy when it trades?

Those questions are related, but they are not the same problem. A forecasting model can estimate that gold or Bitcoin may rise, yet it does not automatically determine whether the strategy should invest $10, $100, or the entire available budget. Conversely, a position-sizing rule can specify how much to buy after a decline, but it needs a signal to decide whether buying is appropriate at that moment.

The implementation keeps these responsibilities separate. The forecasting component produces information about a possible future price. The signal layer converts that forecast into a buy, sell, add, or hold decision. The sizing layer proposes an amount. Risk control can reject or reduce that amount. Finally, the execution and accounting layer applies transaction costs, updates cash and holdings, and measures the resulting portfolio.

This separation is especially important because several parts of the paper are incompletely specified. The paper gives a broad design involving an attention-enhanced BiLSTM, consecutive-rise and decline analysis, exponential position sizing, VaR, and a modified greedy algorithm. It does not fully define every interface between those components. A modular Python implementation makes each interpretation visible instead of hiding assumptions inside one large trading function.

The complete data-to-portfolio flow

The project’s README presents the intended architecture as a pipeline:

local prices
    -> cleaning and chronological split
    -> leakage-safe sliding windows
    -> forecast model
    -> buy / sell / hold signal
    -> finite exponential position-size schedule
    -> historical VaR filter
    -> reconstructed greedy action selection
    -> next-bar execution with fees and slippage
    -> portfolio valuation and performance metrics

Each stage has a distinct responsibility:

The paper’s primary model is the attention-enhanced BiLSTM, or Att-BiLSTM. It processes a historical sequence and uses temporal attention to produce a representation for forecasting. The input sequence is built from a sliding window, reported as 100 observations with stride one. The resulting prediction is not itself a trade: it must be aligned with a timestamp and passed to a decision policy.

The position-management component is conceptually independent of the neural network. It analyzes gains and declines, estimates statistics such as representative gain and decline magnitudes, and uses an exponential schedule to increase later additions. Because the paper’s position-sizing equations are corrupted or incomplete in the extracted source, the Python project treats this as a documented normalized exponential reconstruction rather than an exact transcription.

Why the boundaries matter

A single function that loads prices, trains a model, decides an order, and reports profit would be difficult to audit. It would also make it unclear whether future information entered the decision process. The generated package instead plans separate modules for data, models, streak analysis, sizing, risk, strategy, and backtesting.

For example, DataConfig makes the forecasting window explicit:

@dataclass
class DataConfig:
    """Configuration for local historical-price preparation."""

    timestamp_column: str = "timestamp"
    price_column: str = "price"
    asset_column: Optional[str] = None
    window_length: int = 100
    stride: int = 1
    return_definition: str = "simple"
    zero_return_policy: str = "break"

The values window_length=100 and stride=1 reflect the paper’s stated sliding-window setup. Other fields expose choices that the paper does not settle, such as how returns are defined and how zero returns affect streaks. Making these values configuration fields means a later experiment can record exactly which interpretation it used.

The data layer represents a cleaned asset as PriceData, a small object containing timestamps, prices, and an asset label. Its validation contract requires unique, increasing timestamps and finite positive prices. That contract is more than cosmetic. A negative or missing price can invalidate returns, position sizes, portfolio valuation, and risk calculations downstream.

The WindowedDataset object preserves temporal metadata in addition to arrays:

@dataclass(frozen=True)
class WindowedDataset:
    """Windowed samples and their temporal metadata."""

    X: np.ndarray
    y: np.ndarray
    window_start: pd.DatetimeIndex
    window_end: pd.DatetimeIndex
    y_timestamp: pd.DatetimeIndex
    feature_names: Tuple[str, ...] = ("price",)

Keeping window_end and y_timestamp is important. A model prediction must be associated with the time at which its input information became available. The later backtest can then enforce the rule that a signal generated after the final input observation cannot fill at an earlier or identical timestamp.

Separating a forecast from an order

The paper’s forecasting section and trading section imply different horizons. The prediction tables appear to use a one-step setup, while the conclusion refers to forecasts for the next three days. The implementation therefore exposes the forecast horizon rather than assuming that those descriptions are identical.

Conceptually, a one-step window is:

\[ Xt = [z{t-99}, z{t-98}, \ldots, zt], \qquad yt = z{t+1}. \]

A three-step version is:

\[ yt = [z{t+1}, z{t+2}, z{t+3}]. \]

The forecast layer answers what the model predicts. A separate signal layer must decide how to interpret that prediction. For example, it might compare the terminal three-day forecast with the current price, or use a configured threshold before producing a buy signal. That rule is a strategy assumption, not a consequence of the neural network itself.

The same distinction applies to sizing. A positive forecast does not imply that the strategy should invest its full available balance. The sizing schedule may allocate one of several finite addition levels, subject to cash and exposure caps. The eventual order amount is therefore the result of several stages:

forecast
    -> directional signal
    -> candidate order amount
    -> risk adjustment
    -> selected feasible action
    -> executed fill

This decomposition also makes it possible to compare alternatives. The same forecast can be evaluated with a flat allocation, the reconstructed exponential schedule, or a different risk limit. Likewise, the same sizing rule can be tested with Att-BiLSTM predictions or a simpler benchmark.

Offline and educational by design

The generated project is deliberately offline. It accepts local CSV files or deterministic synthetic data and does not connect to an exchange, download market data, handle credentials, or submit orders. This restriction is appropriate for a paper-to-code tutorial because it keeps the focus on data alignment, modeling assumptions, decision rules, and accounting.

Synthetic data are useful for demonstrating interfaces. For example, the data module includes a deterministic generator whose purpose is to provide repeatable price paths for checking window shapes and portfolio arithmetic. Such data are not evidence about gold or Bitcoin and cannot reproduce the paper’s reported outcomes.

The package initializer reinforces this design by avoiding model training or data loading during import:

"""Offline research components for the quantitative trading model paper."""

# Configuration is part of the lightweight public API. Heavy forecasting and
# benchmark dependencies are imported only by their respective modules.
from .config import (
    BacktestConfig,
    DataConfig,
    ExecutionConfig,
    FeeSensitivityConfig,
    ForecastConfig,
    SizingConfig,
    SplitConfig,
    VaRConfig,
    validate_config,
)

This side-effect-free structure lets a reader use data preparation, streak analysis, sizing, risk, or accounting utilities without installing PyTorch or every optional benchmark library. It also avoids implying that importing the package performs a market action.

What this architecture does—and does not—claim

The architecture provides a practical way to translate the paper’s major ideas into independently inspectable components. It directly reflects the clearly described mechanics, such as sliding windows and temporal attention, while making incomplete parts configurable and labeled as reconstructions.

It does not establish that the paper’s strategy is profitable or that the implementation reproduces its reported numbers. The paper’s approximate final values—$646 from a $500 gold allocation and $215,487 from a $500 Bitcoin allocation—remain unverified claims because the original data, dates, execution rules, risk settings, and position-sizing details are unavailable.

The useful starting point is therefore not a promise of matching those numbers. It is a disciplined contract between modules:

• data provides only ordered historical observations;

• forecasting uses a defined historical window;

• signals use predictions and current state;

• sizing uses a finite, explicit budget;

• risk control uses pre-decision information;

• execution occurs after the signal timestamp;

• evaluation accounts for costs and reports portfolio behavior separately from forecast error.

That contract is the foundation for the remaining sections, where each component is examined in detail and every reconstruction decision is identified.

2. What Can Actually Be Reproduced?

The paper separates two decisions: forecasting proposes when a trade might be useful, while position management proposes how much capital to deploy. Before implementing those components, distinguish a direct implementation from a reconstruction or an illustrative example.

A paper-to-code project can contain clean Python and still fail to reproduce the original experiment. Reproduction requires the same data, timestamps, preprocessing, model definition, decision rules, execution assumptions, and evaluation protocol. Several of those ingredients are missing or ambiguous here.

The project uses five status labels, consistent with docs/reproduction_matrix.md:

These labels determine how results should be interpreted and whether a value may legitimately be used as a test expectation. A hand-calculated fee or accounting invariant can be tested. The paper's final portfolio values cannot be treated that way because their inputs and protocol are unknown.

2.1 What the paper specifies clearly

Several elements are concrete enough to guide implementation:

• overlapping historical windows of length 100 with stride 1;

• an attention-enhanced bidirectional LSTM as the primary forecasting model;

• comparisons with LSTM, GRU, BiLSTM, Holt-Winters, ARIMA, HMM, and XGBoost;

• a stated dropout setting of 20%;

• MAE as the training loss;

• RMSprop as the intended optimizer after correcting the paper's terminology;

• dot-product attention, softmax normalization, and a weighted temporal sum;

• transaction-fee scenarios for gold and Bitcoin.

For example, src/quantitative_trading_model/models/attention.py can implement the displayed attention equations without knowing the authors' complete source code. Similarly, the configuration modules can preserve paper-inspired values as visible settings. This still does not guarantee numerical reproduction: the data, model dimensions, initialization, and training schedule remain unknown.

Configuration provenance is deliberately split across modules rather than concentrated in ForecastConfig:

• DataConfig contains the window length, stride, cleaning, and data conventions;

• ForecastConfig contains model and training settings such as horizon, dropout, loss, optimizer, and hidden dimensions;

• SizingConfig contains the reconstructed position-sizing settings;

• VaRConfig contains risk-estimation and action settings;

• ExecutionConfig and BacktestConfig contain fees, slippage, cash, leverage, and liquidation assumptions;

• FeeSensitivityConfig contains the gold and Bitcoin fee grids.

This separation prevents unrelated assumptions from being hidden behind a misleading single model configuration.

2.2 Missing data and experiment identity

The largest reproducibility gap is the input data. The paper refers to historical gold and Bitcoin prices but does not identify enough information to establish exactly what those series represent. Missing details include:

• the data vendor or source;

• the specific gold instrument and Bitcoin market or index;

• currency and unit conventions;

• sampling frequency;

• start and end dates;

• timezone and timestamp rules;

• the selected price field, such as close or adjusted close;

• treatment of missing observations and duplicate timestamps;

• market closures, splits, or other instrument-specific adjustments;

• whether the assets were processed independently or jointly.

“Gold price” and “Bitcoin price” are not unique datasets. Two valid historical series can have different timestamps, gaps, price fields, and price levels. A model trained on either series may be reasonable while producing metrics that cannot be compared with the paper's table.

README.md, data.py, and cli.py therefore require local inputs rather than silently selecting an unidentified market-data source. Synthetic data are available only for demonstrations. This makes provenance explicit and avoids implying that an arbitrary local or synthetic series is equivalent to the paper's input.

2.3 Missing preprocessing and target definition

The paper does not fully state what the forecasting model predicts. Plausible interpretations include:

a raw next-day price;

a normalized price level;

a one-day return or percentage change;

a vector of the next three prices;

a day-three price produced by a multi-step procedure.

The prediction discussion is compatible with one-step forecasting, while the conclusion refers to forecasts over the next three days. These are different supervised-learning problems with different target shapes, losses, inverse transformations, and trading signals.

Preprocessing is also unclear. The paper does not specify whether prices were scaled, whether scaling used training observations only, or whether features such as volume, technical indicators, or macroeconomic variables were included. The implementation defaults to a price-based setup but exposes the target representation and horizon so that this ambiguity remains visible.

A reproduction must record at least:

2.4 Missing Att-BiLSTM architecture details

Att-BiLSTM identifies a model family, not a complete architecture. The paper does not reliably specify:

• the number of recurrent layers;

• hidden-state dimensions;

• whether attention receives concatenated forward and backward states;

• how the query vector is constructed;

• dense-layer dimensions and activations;

• output shape and activation;

• recurrent versus ordinary dropout;

• initialization;

• learning-rate and optimizer settings;

• batch size, epoch count, and stopping rule;

• random seed;

• whether each asset receives a separate model.

models/forecasters.py consequently provides a configurable paper-inspired model. Its learned attention query is an explicit reconstruction: the paper supplies a symbol q but does not say how that vector is obtained. The implementation is useful and inspectable, but it is not a verified transcription of the authors' architecture.

The same caution applies to the benchmarks. Listing LSTM, GRU, BiLSTM, Holt-Winters, ARIMA, HMM, and XGBoost does not establish that their feature sets, capacities, tuning procedures, and horizons were comparable. The paper's statement about “consistent parameters” is insufficient to reconstruct a fair benchmark protocol.

2.5 The position-sizing equations are incomplete

Equations (5) through (8) in the position-management section are not reliably recoverable from the extracted material. Important ambiguities include:

• whether p_i denotes cash, asset units, or notional exposure;

• the meaning of the maximum addition count x;

• whether fees apply to one transaction or both sides;

• whether decline statistics are signed or expressed as positive loss magnitudes;

• the independent variable in the exponential expression;

• whether additions follow every decline or only a forecast-confirmed condition.

src/quantitative_trading_model/sizing.py therefore does not claim to transcribe the equations. It implements a normalized exponential schedule with finite addition levels, cash limits, and exposure caps. This is a reconstructed exponential position-sizing rule: it captures the apparent idea that later additions receive larger weights, but it does not establish that the paper used the same formula or calibration.

Keeping this reconstruction in one module is useful. If the original equation images or source become available, the sizing implementation can be replaced without changing data preparation, forecasting, risk, or portfolio accounting.

2.6 VaR is named but not operationally defined

The paper says that VaR contributes to trading decisions but does not supply the settings needed for a unique procedure. Missing choices include:

• confidence level;

• horizon;

• estimation-window length;

• historical, parametric, or Monte Carlo method;

• portfolio-level versus asset-level returns;

• treatment of overlapping multi-day returns;

• maximum acceptable loss;

• whether VaR blocks, scales, or merely reports a trade.

src/quantitative_trading_model/risk.py provides a rolling historical VaR reconstruction with explicit confidence, lookback, horizon, and action settings. It supports leakage-aware use, but it cannot independently guarantee temporal provenance: HistoricalVaR accepts caller-provided returns and does not infer timestamps. The surrounding caller or backtest must supply only returns known before the decision and must exclude the current or future return when appropriate.

Thus, a VaR value means “the result of this configured historical quantile on this supplied return window.” It is not a hidden setting recovered from the paper, and it should not be described as the paper's exact VaR method.

2.7 The modified greedy algorithm is not reproducible from its name

“Modified greedy algorithm” describes a family of procedures, not one precise policy. Reproduction would require the paper to define:

• candidate actions;

• whether actions are buy, sell, hold, or additions;

• the objective being maximized;

• how forecasts become expected gains;

• how fees enter the objective;

• cash, exposure, and addition constraints;

• how VaR affects feasibility;

• whether one action or a multi-step plan is selected;

• tie-breaking behavior.

src/quantitative_trading_model/strategy.py implements a transparent reconstruction: enumerate feasible candidates, estimate forecast-based benefit after fees, apply documented constraints, and select the highest-scoring candidate with deterministic tie-breaking. It must not be described as the paper's verified algorithm.

There is also an unresolved static integration issue. The generated GreedyPolicy adapter currently does not match the generated VaRTradeFilter interface: it attempts to pass proposed_notional and omits required exposure-related arguments, while the filter expects proposed_quantity together with current exposure and portfolio value. This boundary must be repaired before runtime integration is attempted. Until then, the strategy and risk modules should be treated as separately documented reconstructions rather than a verified end-to-end combination.

The temporal rule remains essential: realized future returns must never score a candidate. A decision may use forecasts, current cash, current holdings, current exposure, and pre-decision risk history, but not the price that will be observed after execution.

2.8 Why the headline profits are not test targets

The paper reports approximately:

• $646 from an initial $500 gold allocation;

• $215,487 from an initial $500 Bitcoin allocation;

• approximately $216,133 from the combined $1,000 allocation.

These are reported claims, not reproduced results. src/quantitative_trading_model/experiments.py records the values as unverified references rather than using them as test assertions or expected outputs from synthetic demonstrations.

A test target is appropriate when its input and protocol are specified. For example, a hand-calculated accounting test can assert the cash effect of a purchase with a known fee. The paper's Bitcoin figure cannot be tested this way because the historical period, asset series, signal timestamps, sizing triggers, reinvestment rules, exposure limits, fee convention, slippage, and liquidation policy are unknown.

A local result that happens to resemble one of those values would not prove reproduction unless the underlying data and all decision and accounting rules also matched. The unusually large Bitcoin result may depend on a particular historical period, aggressive compounding, repeated averaging down, leverage-like exposure, or optimistic execution assumptions. It should not be generalized as evidence of future profitability.

2.9 Benchmark metrics and fee figures are also unverified

The paper reports forecast metrics such as RMSE, MAE, MAPE, and R-squared, and shows transaction-fee sensitivity figures. Those values cannot be confirmed from the extracted text alone.

Forecast metrics depend on the exact test observations, target scale, transformations, timestamps, MAPE zero policy, architecture, trained parameters, random seed, and stopping rule. Fee figures additionally depend on trade dates and quantities, fee side conventions, spread, slippage, action feasibility, initial holdings, reinvestment, and liquidation.

The implementation preserves the paper-inspired fee grids as inputs: gold rates from 0.2% through 1.2%, and Bitcoin rates from 1.5% through 3.5%. It cannot infer the missing y-values from the figures. A new fee sweep using identified local data is a valid experiment under the project's assumptions, not a recovered copy of the paper's chart.

2.10 How to read the reproduction matrix

docs/reproduction_matrix.md records status at the method and claim level. Its distinctions are important:

This matrix prevents every function from being presented with the same level of authority when some functions implement displayed equations and others embody missing decisions.

2.11 Static review is not empirical reproduction

The project raises two different verification questions:

Does the generated artifact appear structurally reviewable?

Does the implementation reproduce the paper's empirical behavior?

Static and semantic review can address parts of the first question. It can inspect Python parsing, expected symbols, configuration validation, tensor-shape contracts, attention normalization, temporal boundaries, budget invariants, and accounting formulas. It cannot establish that the paper's data, model, or financial results were reproduced.

The supplied local verification record did not pass completely. It reported placeholder-detection findings for experiments.py and cli.py, and Markdown-fence findings for the generated README.md and docs/tutorial.md code fields. These are generated-artifact quality findings. They are not runtime results and do not by themselves demonstrate a Python syntax failure or a financial-model failure.

Semantic code verification was skipped. Runtime execution and test-suite success therefore must not be claimed. In addition, some generated tests and orchestration functions appear to use APIs that are not fully aligned with the generated modules, including the strategy/VaR boundary described above. Those inconsistencies require a software-maintenance pass before execution is attempted.

The appropriate conclusion is limited: the project documents intended interfaces, assumptions, and invariants, but it is not an executed or empirically validated reproduction.

2.12 Responsible interpretation policy

When describing results from this project:

• say implemented for mechanics supported by the paper and the code contract;

• say reconstructed for normalized sizing, historical VaR, the greedy policy, and other interpretations of missing details;

• say illustrative for synthetic data and hand-calculated workflows;

• say unavailable when the source lacks the information needed for implementation or validation;

• say reported claim when quoting the paper's profits or metrics without independent validation;

• do not call a run on a different local dataset an exact reproduction;

• keep forecast metrics separate from portfolio metrics;

• report dates, data provenance, target definition, fees, slippage, turnover, drawdown, and liquidation rules with every backtest;

• treat static review as artifact evidence, not proof of execution, profitability, or paper fidelity.

The next sections can discuss setup and implementation without blurring these categories. A modular reconstruction can still teach attention-based forecasting, streak statistics, bounded sizing, risk filtering, and fee-aware accounting while making clear which conclusions require the original data and experimental protocol.

3. Project Setup and the Implementation Map

The implementation is organized as a normal Python package rather than as one large research script. That choice matters because the paper combines several different responsibilities: data preparation, neural forecasting, benchmark models, streak analysis, position sizing, risk filtering, trading decisions, execution, and evaluation. Keeping those responsibilities in separate modules makes it easier to inspect assumptions and replace incomplete reconstructions later.

The project is also deliberately offline. It accepts local files or generated demonstration data, but it does not connect to an exchange, download market data, store credentials, or submit orders. The resulting code is suitable for educational research and backtest prototyping, not for live trading.

3.1 The src layout

The package uses the conventional src layout:

quantitative-trading-model/
├── pyproject.toml
├── README.md
├── src/
│   └── quantitative_trading_model/
│       ├── __init__.py
│       ├── config.py
│       ├── data.py
│       ├── metrics.py
│       ├── streaks.py
│       ├── sizing.py
│       ├── risk.py
│       ├── strategy.py
│       ├── backtest.py
│       ├── experiments.py
│       ├── cli.py
│       └── models/
│           ├── attention.py
│           ├── forecasters.py
│           └── benchmarks.py
├── tests/
└── docs/
    ├── tutorial.md
    └── reproduction_matrix.md

With this layout, the importable package lives under src/quantitative_trading_model, while documentation and tests remain outside the runtime package. The pyproject.toml file tells setuptools where to find the package:

[tool.setuptools]
package-dir = { "" = "src" }
include-package-data = true

[tool.setuptools.packages.find]
where = ["src"]

This avoids a common development problem in which tests accidentally import a loose source directory instead of the package that users will install. It also gives the project a clear boundary: files under src/quantitative_trading_model are implementation modules, while the root-level documentation explains how to use and interpret them.

3.2 Core dependencies and optional model dependencies

The package keeps its required dependencies small:

[project]
requires-python = ">=3.10"
dependencies = [
    "numpy>=1.24",
    "pandas>=2.0"
]

NumPy and pandas support the parts of the project that do not require a deep-learning framework: local price tables, timestamp handling, return calculations, streak statistics, metrics, risk calculations, and portfolio accounting.

The heavier forecasting libraries are optional:

[project.optional-dependencies]
deep-learning = [
    "torch>=2.0"
]
classical = [
    "statsmodels>=0.14",
    "pmdarima>=2.0",
    "hmmlearn>=0.3",
    "xgboost>=2.0"
]

This split reflects the paper's structure. The Att-BiLSTM requires PyTorch, but a reader who only wants to study the return convention, normalized exponential sizing reconstruction, historical VaR, or portfolio accounting should not need to install PyTorch. Similarly, Holt-Winters, ARIMA, HMM, and XGBoost benchmarks can be enabled only when those comparisons are needed.

The optional dependency design also makes missing capabilities visible. A benchmark adapter should report that a library is unavailable rather than silently substituting a different model. That distinction matters when comparing results with the paper: an omitted benchmark is not the same as a benchmark that produced a poor score.

The package metadata also defines a console entry point:

[project.scripts]
quant-trading-model = "quantitative_trading_model.cli:main"

After local installation, this exposes the offline command-line interface as quant-trading-model. The CLI can validate local CSV files, generate deterministic synthetic demonstrations, and dispatch configured research experiments. It does not create a network client or read credentials.

3.3 Side-effect-free package imports

The package initializer intentionally exposes only lightweight metadata and configuration types:

"""Offline research components for the quantitative trading model paper."""

from importlib import metadata

try:
    __version__ = metadata.version("quantitative-trading-model")
except metadata.PackageNotFoundError:
    __version__ = "0.1.0"

from .config import (
    BacktestConfig,
    DataConfig,
    ExecutionConfig,
    FeeSensitivityConfig,
    ForecastConfig,
    SizingConfig,
    SplitConfig,
    VaRConfig,
    validate_config,
)

Importing the package does not load a CSV, train a model, create a portfolio, or contact an external service. The initializer also does not import PyTorch, statsmodels, hmmlearn, or XGBoost. Those imports are deferred to the modules that need them.

That design has two practical benefits:

Lightweight use: core modules remain usable in an environment without optional modeling libraries.

Predictable imports: simply writing import quantitative_trading_model cannot start a computation or produce an external side effect.

The public exports are deliberately narrow:

__all__ = [
    "__version__",
    "BacktestConfig",
    "DataConfig",
    "ExecutionConfig",
    "FeeSensitivityConfig",
    "ForecastConfig",
    "SizingConfig",
    "SplitConfig",
    "VaRConfig",
    "validate_config",
]

The model, data, strategy, and backtest classes are imported from their own modules when required. This keeps the top-level API stable without forcing every user to install every optional dependency.

3.4 Where each research responsibility lives

The package map follows the paper's conceptual pipeline:

This separation is not merely a software-style preference. It prevents an ambiguous paper detail from being hidden inside an unrelated function. For example, the return denominator belongs in streaks.py, while the execution fee convention belongs in backtest.py. A reader can therefore change one assumption without accidentally changing every layer of the research pipeline.

3.5 Documentation, tests, and the reproduction matrix

The non-runtime files serve different purposes:

• `README.md` gives installation instructions, local CSV conventions, CLI examples, project warnings, and a high-level architecture map.

• `docs/tutorial.md` explains the paper-to-code translation in detail, including equations, tensor shapes, timing rules, and reconstruction decisions.

• `docs/reproduction_matrix.md` tracks each paper component as implemented, reconstructed, illustrative, unavailable, or an unverified reported claim.

• `tests/` contains hand-calculable checks for data boundaries, streak semantics, sizing budgets, VaR decisions, model shapes, transaction fees, and portfolio-value identity.

The reproduction matrix is especially useful for this paper because the source does not fully specify equations (5)–(8), VaR operation, the greedy policy, or the original backtest protocol. Instead of allowing those gaps to disappear into code, the matrix records what the implementation can support and what still requires the original data or source material.

For example, the matrix treats the following differently:

• the 100-step window as a directly implementable paper detail;

• the learned attention query as a reconstruction;

• synthetic prices as an illustration;

• the reported $646 gold and $215,487 Bitcoin outcomes as unverified claims.

That classification should remain visible in experiment output and documentation. A clean package installation does not establish empirical reproduction, and static code checks do not establish that the paper's financial results are correct.

3.6 A practical installation boundary

A typical local setup is:

python -m venv .venv
python -m pip install --upgrade pip
python -m pip install -e .

The core installation is enough to explore data preparation, streak analysis, risk calculations, sizing, metrics, and portfolio accounting. To use the PyTorch forecasters, install the deep-learning extra:

python -m pip install -e '.[deep-learning]'

The classical benchmark dependencies are declared under the classical extra in pyproject.toml:

python -m pip install -e '.[classical]'

The exact installed command surface can be inspected with:

quant-trading-model --help

These commands install or invoke local research tooling only. They do not download the paper's missing gold or Bitcoin data. The user must supply a provenance-documented local dataset before treating any result as an empirical experiment.

3.7 What this structure does—and does not—guarantee

The package structure provides useful engineering guarantees:

• optional modeling libraries do not prevent core imports;

• configuration choices are visible rather than scattered through scripts;

• data, forecast, strategy, execution, and evaluation responsibilities are separated;

• documentation can distinguish direct implementation from reconstruction;

• tests can check local invariants without requiring the paper's unavailable data.

It does not guarantee that the paper's reported results can be reproduced. The missing dataset, dates, preprocessing, exact model architecture, VaR parameters, greedy rules, position-sizing equations, and execution protocol remain external requirements. The next sections use this package structure to examine those components one at a time, beginning with validated configuration and local data preparation.

4. Make Ambiguity Explicit with Validated Configuration

A research paper can name an algorithm without specifying every value required to run it. This paper gives some concrete settings, including a 100-observation history, one-observation stride, 20% dropout, MAE loss, RMSprop optimization, and several transaction-fee scenarios. It leaves other choices unclear, including the forecast horizon, recurrent-layer dimensions, attention query, VaR policy, position-sizing parameters, execution timing, and liquidation rule.

The implementation records these choices in dataclasses defined in src/quantitative_trading_model/config.py. Centralized configuration does not recover the paper's missing data or prove that its reported results can be reproduced. It does make assumptions visible, validates basic invariants, and provides metadata that can be saved with an experiment.

There are three important categories of settings:

• Paper-inspired settings are stated directly or strongly suggested by the paper.

• Reconstruction settings are operational choices needed because the paper is incomplete.

• Implementation safety settings constrain this educational simulator even when the paper does not describe equivalent controls.

The distinction matters because a field can be present in configuration without being fully wired into every generated runtime module. The status of each important setting should therefore be checked against both config.py and the module that consumes it.

4.1 Configuration objects and their responsibilities

The configuration module defines separate dataclasses for the main layers of the pipeline:

For example, the paper-inspired data and forecasting defaults are:

@dataclass
class DataConfig:
    timestamp_column: str = "timestamp"
    price_column: str = "price"
    window_length: int = 100
    stride: int = 1
    return_definition: str = "simple"


@dataclass
class ForecastConfig:
    model_name: str = "att_bilstm"
    horizon: int = 1
    target_type: str = "scaled_price"
    hidden_size: int = 32
    num_layers: int = 1
    dropout: float = 0.20
    learning_rate: float = 0.001
    optimizer: str = "rmsprop"
    loss: str = "mae"
    learned_attention_query: bool = True

The values with the clearest connection to the paper are:

• window_length=100: the reported historical window length;

• stride=1: the reported one-observation shift between windows;

• dropout=0.20: the paper-inspired dropout rate;

• loss="mae": the reported training loss;

• optimizer="rmsprop": a correction to the paper's terminology, since RMSprop is an optimizer rather than an activation function;

• model_name="att_bilstm": the paper's claimed primary forecasting model.

The hidden size, number of layers, learning rate, batch size, epoch count, early-stopping policy, and learned-query choice are not established by the paper. They are configurable reconstruction choices and should be recorded with any result.

4.2 Make the forecast horizon explicit

The paper's prediction discussion is not consistent about horizon. Its model description and metric table can be read as a one-step forecasting setup, while the conclusion refers to forecasts for the next three days. The source does not establish whether this means a three-element output vector, repeated one-step forecasts, or a single prediction for the third day.

The configuration makes the choice visible:

one_day = ForecastConfig(horizon=1)
three_days = ForecastConfig(horizon=3)

The horizon affects more than the output layer. It changes target construction, prediction metrics, signal aggregation, and execution timing. A one-step model should not be called a three-day model merely because its output is used repeatedly. The selected horizon should be stored in experiment metadata.

target_type is similarly uncertain. The paper does not clearly say whether it predicts raw prices, scaled prices, returns, or percentage changes. The generated default, "scaled_price", is a practical implementation choice and not a verified transcription.

4.3 Position-sizing configuration: metadata versus enforcement

The position-management equations are among the least reproducible parts of the paper. Equations (5) and (6) appear to describe recursive additions intended to compensate for declines and fees, but their notation is corrupted. Equation (7) does not clearly identify the independent variable of the exponential, and Equation (8) appears to normalize position scores into allocations.

SizingConfig records a possible reconstruction:

@dataclass
class SizingConfig:
    maximum_additions: int = 5
    initial_level: int = 1
    growth_rate: float = 0.50
    score_scale: float = 1.0
    allocation_fraction: float = 1.0
    initial_position_fraction: float = 0.20
    maximum_exposure_fraction: float = 1.0
    allow_reuse: bool = False
    calibrate_from_percentiles: bool = False
    recovery_percentile: float = 0.50
    decline_percentile: float = 0.10
    fee_buffer: float = 0.0

The intended normalized reconstruction is:

score_i      = score_scale * exp(growth_rate * i)
weight_i     = score_i / sum(score_j)
allocation_i = budget * weight_i

However, these configuration fields should not be confused with runtime guarantees. In the generated project, sizing.py builds schedules through build_exponential_schedule() and enforces schedule-level properties through SizingSchedule, including finite allocations, normalized weights, a total budget, and one-time level consumption. validate_config() validates maximum_additions and related fractions, but it does not construct a schedule or verify its normalized weights.

Likewise, allocation_fraction, maximum_exposure_fraction, allow_reuse, and fee_buffer are currently configuration or documentation choices rather than completely wired controls in the generated sizing path. The schedule builder receives its own budget and growth arguments. It does not automatically read every corresponding field from SizingConfig, and fee_buffer is not applied to an allocation. A caller must pass the intended values into the sizing module explicitly or add the missing integration before treating them as active controls.

The safe interpretation is therefore:

• maximum_additions documents the intended finite schedule size;

• schedule normalization and budget checks belong to SizingSchedule and sizing.py;

• reuse prevention is enforced when a constructed schedule marks a level as used;

• exposure and cash caps must be applied by the sizer, strategy, or simulator that receives them;

• the incomplete paper equations remain a reconstruction, not an exact implementation.

4.4 VaR configuration is an operational reconstruction

The paper names VaR but does not define its confidence level, horizon, estimation window, return distribution, or action threshold. A usable risk filter must choose all of these values, so VaRConfig makes them explicit:

@dataclass
class VaRConfig:
    enabled: bool = True
    confidence: float = 0.95
    horizon: int = 1
    lookback: int = 100
    method: str = "historical"
    action: str = "scale"
    max_var_fraction: float = 0.05
    insufficient_history: str = "allow"
    min_observations: int = 30

These defaults describe a rolling historical-VaR interpretation, not the paper's verified settings:

• confidence=0.95 selects a 95% lower-tail confidence convention;

• horizon=1 uses one period by default;

• lookback=100 limits the estimation history;

• method="historical" avoids assuming normally distributed returns;

• action="scale" permits reducing an order when projected risk is too high;

• max_var_fraction=0.05 expresses the risk limit as a fraction of portfolio value;

• insufficient_history="allow" defines behavior before enough observations exist.

The generated risk.py contains the actual historical estimator and filter. It must receive returns that were available before the decision timestamp. Configuration alone cannot enforce timestamp ordering. Also, because the paper provides no numerical VaR settings, every backtest should report these values as reconstruction metadata.

A stricter configuration can block trades until sufficient history exists:

risk = VaRConfig(
    confidence=0.99,
    lookback=252,
    min_observations=100,
    action="block",
    insufficient_history="block",
)

4.5 Execution and liquidation: record the intended contract carefully

The paper does not specify whether a signal uses the same closing price, the next opening price, or another execution price. It also does not say clearly whether a fee applies to purchases, sales, or both. The intended execution configuration is:

@dataclass
class ExecutionConfig:
    fee_rate: float = 0.002
    fee_on_buy: bool = True
    fee_on_sell: bool = True
    slippage_rate: float = 0.0
    allow_fractional_units: bool = True
    allow_short: bool = False
    max_leverage: float = 1.0
    execute_next_bar: bool = True
    reject_insufficient_cash: bool = True

These fields describe the desired research contract, but the generated backtest.py does not fully honor all of them as written. In particular:

• the simulator applies fee_rate to both buys and sells unconditionally; it does not consult fee_on_buy or fee_on_sell;

• unaffordable buys are clipped to the affordable quantity rather than being controlled by reject_insufficient_cash;

• the simulator uses its own allow_leverage behavior and does not enforce max_leverage from ExecutionConfig;

• the default simulator is nevertheless long-only and non-leveraged through its separate defaults, so cash and holdings are intended to remain nonnegative;

• next-bar execution is enforced by requiring the fill timestamp to be strictly later than the signal timestamp.

This distinction is important. A configuration field is not proof that the current runtime path consumes it. Until the wiring is corrected, results should state the simulator's actual behavior: two-sided fees, optional slippage, clipping of unaffordable buys, no shorting by default, and strict later-bar execution.

BacktestConfig contains additional lifecycle choices:

@dataclass
class BacktestConfig:
    initial_cash: float = 500.0
    initial_quantity: float = 0.0
    liquidate_at_end: bool = True
    risk_free_rate: float = 0.0
    periods_per_year: int = 252
    currency: str = "USD"

The $500 default resembles the paper's stated per-asset starting allocation, but it does not reproduce the paper's final values. Dates, trades, fees, holdings, and liquidation accounting remain unspecified. liquidate_at_end=True is an explicit implementation choice: it may sell remaining holdings and charge the simulator's sell-side fee. A mark-to-market result without liquidation would be different.

4.6 Fee grids use decimal conventions

The paper lists percentages, while the configuration stores decimal rates:

The conversion is:

0.2% = 0.2 / 100 = 0.002
1.5% = 1.5 / 100 = 0.015

The corresponding defaults are:

@dataclass
class FeeSensitivityConfig:
    gold_rates: tuple[float, ...] = (
        0.002, 0.004, 0.006, 0.008, 0.010, 0.012
    )
    bitcoin_rates: tuple[float, ...] = (
        0.015, 0.020, 0.025, 0.030, 0.035
    )
    apply_to_buys: bool = True
    apply_to_sells: bool = True
    hold_signals_fixed: bool = True

Using 0.2 for a 0.2% fee would represent a 20% rate. The validator checks that rates are finite and lie in the interval [0, 1], but it cannot determine whether a user has confused percentage points with decimal rates. In other words, 0.2 is accepted as a numerically valid rate even though it is probably a unit mistake for this experiment. The decimal convention must therefore be documented and reviewed by the caller.

The fee-side flags are also not fully honored by the generated simulator, which currently charges both sides. hold_signals_fixed describes the sensitivity protocol rather than changing the backtest automatically. If fees affect feasibility or greedy ranking, actions may need to be regenerated for each fee scenario.

4.7 What validate_config() checks

The public validator accepts a complete aggregate or individual components:

from quantitative_trading_model.config import (
    ResearchConfig,
    ForecastConfig,
    validate_config,
)

config = ResearchConfig(
    forecast=ForecastConfig(
        model_name="att_bilstm",
        horizon=3,
        dropout=0.20,
        loss="mae",
        optimizer="rmsprop",
    )
)

validate_config(config)

Component validators check basic constraints such as:

_require(config.window_length > 0, "data.window_length must be positive")
_require(0 <= config.dropout < 1, "forecast.dropout must be in [0, 1)")
_require(0 < config.confidence < 1, "var.confidence must be in (0, 1)")
_require(config.fee_rate >= 0, "execution.fee_rate cannot be negative")
_require(config.maximum_additions > 0, "sizing.maximum_additions must be positive")

This validation rejects invalid configuration values such as nonpositive horizons, dropout outside [0, 1), unsupported optimizer or loss names, split fractions that do not sum to one, negative fees or slippage, invalid VaR actions, and empty fee grids.

It does not perform every runtime or semantic check. Specifically:

• it does not construct a sizing schedule or check that its weights sum to one;

• it does not validate the contents of a price file;

• it does not prove that every configuration field is consumed by the backtest;

• it does not detect a percentage-unit mistake such as entering 0.2 for 0.2%;

• it does not establish that VaR inputs are timestamp-safe;

• it does not verify that generated experiments reproduce the paper.

Schedule-level invariants are handled in sizing.py, data invariants in data.py, and accounting invariants in backtest.py. Those layers must be reviewed together with configuration validation.

The validator also rejects supplying both a complete ResearchConfig and individual component overrides. This prevents a caller from validating one configuration while accidentally running another.

4.8 Record a complete experiment configuration

A complete experiment can combine paper-inspired and reconstructed settings:

from quantitative_trading_model.config import (
    ResearchConfig,
    ForecastConfig,
    VaRConfig,
    validate_config,
)

config = ResearchConfig(
    forecast=ForecastConfig(
        model_name="att_bilstm",
        horizon=3,
        hidden_size=32,
        dropout=0.20,
        loss="mae",
        optimizer="rmsprop",
    ),
    var=VaRConfig(
        confidence=0.95,
        horizon=1,
        lookback=100,
        action="scale",
    ),
)

validate_config(config)

metadata = {
    "window_length": config.data.window_length,
    "stride": config.data.stride,
    "forecast_horizon": config.forecast.horizon,
    "dropout": config.forecast.dropout,
    "loss": config.forecast.loss,
    "optimizer": config.forecast.optimizer,
    "var_confidence": config.var.confidence,
    "var_lookback": config.var.lookback,
    "fee_rate": config.execution.fee_rate,
    "liquidate_at_end": config.backtest.liquidate_at_end,
}

This record combines direct paper-inspired values with reconstruction choices. It should be saved alongside forecast metrics, portfolio metrics, data provenance, and software-version information.

ResearchConfig is available from quantitative_trading_model.config. It is not currently included in the package-level exports in quantitative_trading_model/__init__.py, so importing it from the configuration module is the reliable documented path.

4.9 Configuration is necessary but not sufficient

The configuration contract solves an engineering problem: it makes assumptions visible, validates basic values, and gives experiments a stable record. It does not solve the paper's missing-information problem. It cannot recover:

• the unidentified gold and Bitcoin datasets;

• the original dates, frequency, or test boundary;

• the exact attention query and network architecture;

• the corrupted recursive position-sizing equations;

• the paper's VaR settings or action policy;

• the modified greedy algorithm's objective;

• the original execution, fee, and liquidation protocol;

• the numeric values behind the fee-sensitivity figures.

The approximately $646 gold result, approximately $215,487 Bitcoin result, combined approximately $216,133 result, and reported benchmark metrics therefore remain unverified paper claims. Matching visible defaults does not reproduce those results.

The generated project also has known API inconsistencies between some configuration fields and downstream modules, and semantic code verification was skipped. This section consequently describes the intended configuration contract and the actual validation boundaries; it does not claim that the generated code was executed or that every setting passed through the complete runtime pipeline.

The safest interpretation is that configuration turns an incomplete paper into an auditable research specification. It identifies what comes from the source, what is reconstructed, what is merely illustrative, and which safety controls belong to this educational implementation rather than to the original paper.

5. Load Local Prices Without Assuming the Paper’s Missing Dataset

The paper says that it uses historical gold and Bitcoin prices, but it does not identify the data source, instrument, currency, frequency, date range, or exact columns. That omission is important: even a correct implementation can produce different results if it uses a different gold contract, Bitcoin exchange, timezone, sampling interval, or adjusted-price convention.

The generated project therefore does not silently select a market-data provider. Instead, it accepts a local CSV file or a caller-supplied pandas object. This keeps the experiment offline and makes data provenance the reader’s responsibility. Before treating a result as empirical evidence, record where the file came from, which instrument it represents, and how it was prepared.

5.1 The minimum data contract

The implementation reduces each asset to two essential fields:

• timestamp: when the price observation was available;

• price: a finite, strictly positive price.

A minimal CSV looks like this:

timestamp,price
2020-01-01,1518.2
2020-01-02,1523.7
2020-01-03,1511.4

The PriceData class in src/quantitative_trading_model/data.py stores the cleaned result and an asset label:

@dataclass(frozen=True)
class PriceData:
    frame: pd.DataFrame
    asset: str = "asset"

    def __post_init__(self) -> None:
        required = {"timestamp", "price"}
        missing = required.difference(self.frame.columns)
        if missing:
            raise ValueError(f"PriceData is missing required columns: {sorted(missing)}")

        timestamps = pd.to_datetime(self.frame["timestamp"], errors="raise")
        prices = pd.to_numeric(self.frame["price"], errors="raise").to_numpy(dtype=float)

        if timestamps.duplicated().any():
            raise ValueError("PriceData timestamps must be unique")
        if not timestamps.is_monotonic_increasing:
            raise ValueError("PriceData timestamps must be strictly increasing")
        if not np.isfinite(prices).all() or (prices <= 0).any():
            raise ValueError("PriceData prices must be finite and strictly positive")

PriceData.__post_init__ is a validation boundary. It does not decide whether a price is economically meaningful for every possible instrument, but it enforces the assumptions required by the downstream time-series code. An empty dataset, duplicate timestamp, unsorted timestamp sequence, missing value, infinity, zero, or negative price cannot silently proceed into window construction.

The timestamps and prices properties provide convenient, typed access to the two series:

@property
def timestamps(self) -> pd.DatetimeIndex:
    return pd.DatetimeIndex(self.frame["timestamp"])

@property
def prices(self) -> np.ndarray:
    return self.frame["price"].to_numpy(dtype=float, copy=True)

Returning a copy from prices reduces the chance that a caller accidentally mutates the validated internal frame without re-running validation.

5.2 Standardizing different local column names

Real CSV files often use names such as Date, datetime, Close, or Adj Close. The standardize_prices function accepts explicit column names when necessary, but can also recognize common alternatives:

def standardize_prices(
    data: Union[pd.DataFrame, PriceData],
    *,
    timestamp_column: Optional[str] = None,
    price_column: Optional[str] = None,
    asset: str = "asset",
    drop_invalid: bool = True,
) -> PriceData:

The function first copies the input, so cleaning does not mutate the caller’s DataFrame. It then resolves columns through _resolve_column:

timestamp_column = _resolve_column(
    frame, timestamp_column,
    ("timestamp", "datetime", "date", "time"),
    "timestamp",
)
price_column = _resolve_column(
    frame, price_column,
    ("price", "close", "adj_close", "adjusted_close"),
    "price",
)

Explicit names are safer when a file contains several possible price fields. For example, choosing close versus adjusted_close can materially change a historical experiment. Automatic inference is a convenience, not a substitute for documenting the choice.

The selected columns are renamed to the package’s stable internal names:

normalized = frame[[timestamp_column, price_column]].rename(
    columns={timestamp_column: "timestamp", price_column: "price"}
)
normalized["timestamp"] = pd.to_datetime(
    normalized["timestamp"], errors="coerce", utc=False
)
normalized["price"] = pd.to_numeric(normalized["price"], errors="coerce")

Values that cannot be parsed become missing. The function then constructs a validity mask requiring a real timestamp and a finite, positive price:

valid = normalized["timestamp"].notna() & normalized["price"].notna()
valid &= np.isfinite(normalized["price"].to_numpy(dtype=float))
valid &= normalized["price"] > 0

By default, invalid rows are removed. If drop_invalid=False, the function raises an error instead. The choice depends on the research context: dropping a corrupted row may be reasonable for a demonstration, but a production-quality study should usually stop and investigate why the row is invalid rather than silently deleting it.

5.3 Sorting and duplicate timestamps

After parsing, the function sorts by timestamp using a stable sort and removes duplicate timestamps by retaining the last source row:

normalized = (
    normalized.sort_values("timestamp", kind="mergesort")
    .drop_duplicates("timestamp", keep="last")
    .reset_index(drop=True)
)

This gives the downstream model one price per timestamp. It does not prove that the retained row is the correct observation. If duplicate rows represent separate trades, multiple venues, or different assets, they should be aggregated or separated before calling standardize_prices.

The duplicate policy is therefore an implementation convenience, not a fact recovered from the paper. The paper does not explain how duplicate timestamps or multiple records within one sampling interval were handled.

5.4 Loading a local CSV

load_price_csv is a thin file-system wrapper around standardize_prices:

def load_price_csv(
    path: Union[str, Path],
    *,
    timestamp_column: Optional[str] = None,
    price_column: Optional[str] = None,
    asset: str = "asset",
    drop_invalid: bool = True,
) -> PriceData:
    source = Path(path)
    if not source.exists() or not source.is_file():
        raise FileNotFoundError(f"Price CSV does not exist: {source}")
    if source.suffix.lower() != ".csv":
        raise ValueError("load_price_csv accepts a local .csv file")
    return standardize_prices(
        pd.read_csv(source),
        timestamp_column=timestamp_column,
        price_column=price_column,
        asset=asset,
        drop_invalid=drop_invalid,
    )

The function deliberately accepts only a local .csv path. There are no HTTP requests, exchange clients, credentials, or hidden downloads. This is consistent with the project’s educational and offline scope.

The CLI exposes the same idea through the validate-data subcommand. Its local-path helper rejects URL-like strings before loading them:

def _safe_input_path(value: str) -> Path:
    if "://" in value:
        raise CLIError(
            "Only local files are supported; URLs and network paths are not accepted"
        )
    path = Path(value).expanduser()
    if not path.exists():
        raise CLIError(f"Input file does not exist: {path}")
    if not path.is_file():
        raise CLIError(f"Input path is not a regular file: {path}")
    return path.resolve()

A local validation command can therefore be written as:

quant-trading-model validate-data prices.csv \
  --timestamp-column timestamp \
  --price-column close \
  --asset gold

The command prints or writes a compact summary rather than training a model. This makes data inspection a separate step from forecasting and backtesting.

5.5 Keeping gold and Bitcoin separate

The PriceData representation is intentionally single-asset. A file may contain multiple assets, but the research pipeline should filter it into one chronologically ordered series per asset before creating windows. Gold and Bitcoin have different price scales, trading calendars, liquidity characteristics, and likely data sources. Combining them accidentally into one price sequence would make the next observation after a gold row appear to be a Bitcoin movement.

A multi-asset table can be filtered explicitly:

raw = pd.read_csv("historical_prices.csv")

gold = standardize_prices(
    raw.loc[raw["asset"].eq("gold")],
    timestamp_column="timestamp",
    price_column="price",
    asset="gold",
)

bitcoin = standardize_prices(
    raw.loc[raw["asset"].eq("bitcoin")],
    timestamp_column="timestamp",
    price_column="price",
    asset="bitcoin",
)

Each result can then be passed independently to the window builder and forecasting experiment. This also makes it possible to record asset-specific provenance and fee assumptions. The paper’s headline results allocate $500 to each asset, but the extracted text does not identify whether the assets were sampled on matching dates or evaluated with identical calendars.

5.6 Synthetic data are for interfaces, not validation

Because the paper’s original datasets are unavailable, the project includes generate_synthetic_prices. It creates deterministic, positive price paths using a seeded random generator:

def generate_synthetic_prices(
    *,
    asset: str = "synthetic",
    periods: int = 500,
    start: Union[str, pd.Timestamp] = "2020-01-01",
    seed: int = 7,
    initial_price: Optional[float] = None,
) -> PriceData:

The function chooses different illustrative drift and volatility defaults for labels containing gold or bitcoin, then constructs a positive path from exponentiated random shocks. The seed makes the example repeatable for teaching and invariant checks.

For example:

from quantitative_trading_model.data import generate_synthetic_prices

prices = generate_synthetic_prices(
    asset="gold",
    periods=500,
    seed=7,
)
print(prices.asset)
print(prices.frame.head())

The label-specific behavior is only a teaching convenience. A “Bitcoin-like” synthetic path is not Bitcoin data, and a “gold-like” path is not a gold instrument. Synthetic data can demonstrate that timestamps, windows, attention tensors, sizing budgets, and portfolio accounting have compatible interfaces. It cannot validate the paper’s forecast metrics, fee curves, or reported final values.

5.7 Data checks before modeling

Before moving to sliding windows, record at least the following:

The generated tests in tests/test_data_and_streaks.py are intended to check these kinds of invariants with small hand-built fixtures. They are static artifacts in this project review; no claim is made here that the generated test suite was executed successfully.

The essential principle is simple: clean only what the data contract justifies, preserve timestamps, separate assets, and document every assumption before training. Once the local series is trustworthy and its provenance is recorded, the next step is to construct the paper’s 100-observation windows without allowing scaling or temporal partitioning to leak future information.

6. Build Leakage-Safe Sliding Windows

The paper describes a sliding-window forecasting setup with a history of 100 observations and a stride of one. The model reads the latest 100 observations, predicts a later value, advances one observation, and creates the next overlapping example.

The important issue is not only the window size. Every forecast must use information available at its decision timestamp. The target must occur after the input window, scaling parameters must be fitted without future observations, and evaluation must preserve chronological order.

6.1 The one-step relationship

Let z_t denote a transformed price or feature value at time t. With a window length of 100, a one-step sample is:

\[ Xt = [z{t-99}, z{t-98}, \ldots, z{t-1}, zt], \qquad yt = z_{t+1}. \]

The final value in X_t is the latest information available when the forecast is generated. The target belongs to the next observation.

The paper-inspired window settings are represented by DataConfig:

@dataclass
class DataConfig:
    timestamp_column: str = "timestamp"
    price_column: str = "price"
    window_length: int = 100
    stride: int = 1

window_length=100 and stride=1 come from the paper. The paper does not clearly state whether the model uses raw prices, normalized prices, returns, or additional features, so the representation remains an explicit implementation choice.

6.2 How make_windows constructs samples

The central function is make_windows in src/quantitative_trading_model/data.py. It accepts a cleaned PriceData object, a DataFrame, or a numeric array. For timestamp-aware inputs, it stores the start, end, and target timestamps alongside the arrays.

The core operation is equivalent to:

for start in range(0, last_start + 1, stride):
    end = start + window_length - 1
    X.append(transformed[start : end + 1])
    targets.append(transformed[end + 1 : end + 1 + horizon, 0])
    target_times.append(timestamps[end + horizon])

With the defaults, the first sample contains observations 0 through 99 and targets observation 100. The next sample contains observations 1 through 100 and targets observation 101.

The returned WindowedDataset includes temporal metadata:

@dataclass(frozen=True)
class WindowedDataset:
    X: np.ndarray
    y: np.ndarray
    window_start: pd.DatetimeIndex
    window_end: pd.DatetimeIndex
    y_timestamp: pd.DatetimeIndex
    feature_names: Tuple[str, ...] = ("price",)

Its invariant check requires every target timestamp to follow the corresponding input window:

if n and not (self.y_timestamp > self.window_end).all():
    raise ValueError("Every target timestamp must be after its input window")

This timestamp guarantee applies when the input carries timestamps, such as PriceData or a cleaned DataFrame. For a raw numeric array, the current implementation has no timestamp argument in that code path and creates placeholder daily timestamps beginning at 1970-01-01. Those generated timestamps preserve array alignment for shape-based demonstrations, but they are not source-market timestamps. A trading experiment should therefore use PriceData or a timestamp-aware DataFrame.

6.3 One-day versus three-day targets

The paper is ambiguous about the forecast horizon. Its prediction discussion is compatible with one-step forecasting, while its conclusion refers to forecasts for the next three days. The implementation exposes horizon rather than silently choosing one interpretation.

For horizon=1:

X.shape = (samples, 100, features)
y.shape = (samples,)

For horizon=3:

\[ yt = [z{t+1}, z{t+2}, z{t+3}]. \]

The shape is then:

X.shape = (samples, 100, features)
y.shape = (samples, 3)

The final target timestamp is the timestamp of z_{t+3}. Converting a three-value forecast into a buy, sell, or hold decision belongs to the strategy layer. Possible rules include using the terminal forecast, the mean forecast, or a separately specified multi-day signal.

For a series of N observations, the number of stride-one windows is:

N−100−h+1,

where h is the forecast horizon. The final window must leave enough observations for every target value.

6.4 Overlapping windows and temporal partitions

Stride one creates heavily overlapping samples:

window 1: [0, 1, 2, ..., 99] -> target 100
window 2: [1, 2, 3, ..., 100] -> target 101

This overlap is normal for sliding-window forecasting. The risk comes from treating neighboring samples as independent random observations. If windows are randomly shuffled before splitting, almost identical histories can appear in both partitions, producing optimistic estimates.

The generated recurrent training code therefore uses non-shuffled batches:

train_loader = DataLoader(
    TensorDataset(torch.from_numpy(x_train), torch.from_numpy(y_train)),
    batch_size=batch_size,
    shuffle=False,
)

chronological_split also preserves sample order. This prevents random cross-partition mixing, but it does not guarantee that raw input histories are disjoint at the boundary. The final training window and first test window may still share observations because the windows overlap by design. That dependence should be acknowledged when interpreting metrics; walk-forward evaluation and a documented gap can provide a stricter protocol.

6.5 Fit scaling on training observations only

Scaling can make neural-network optimization easier, but fitting a scaler on the complete series leaks future distribution information. For example, a full-series min-max scaler exposes the training process to the future minimum and maximum.

The repository provides ChronologicalScaler:

scaler = ChronologicalScaler()
scaler.fit(training_values)                 # training rows only
train_scaled = scaler.transform(train_values)
test_scaled = scaler.transform(test_values)

The safe sequence is:

Clean and sort the complete local file.

Establish the chronological raw-row training boundary.

Fit the scaler only on rows before that boundary.

Transform later rows with the fitted scaler.

Build windows and retain their timestamps.

Inverse-transform predictions before reporting price-level metrics when appropriate.

make_windows accepts an already fitted scaler. Its fit_scaler option is intended for controlled demonstrations; setting fit_scaler=True on the complete dataset before an out-of-sample experiment would leak future information. The window builder does not know whether the caller selected a valid training boundary and cannot enforce this policy automatically.

A safer pattern is:

train_rows = price_data.frame.iloc[:training_row_count]
scaler = ChronologicalScaler().fit(
    train_rows[["price"]].to_numpy(dtype=float)
)

windows = make_windows(
    price_data,
    window_length=100,
    horizon=1,
    stride=1,
    scaler=scaler,
    fit_scaler=False,
)

Here, training_row_count must be determined from a documented chronological rule, such as a date boundary or a fixed initial training period. It must not be chosen after inspecting test performance. If windows are built from the entire transformed series and then split, the scaler remains training-fitted, but boundary overlap still exists; a stricter workflow can build or select partitions around an explicit raw-row boundary.

6.6 Chronological 70:30 splitting

The paper reports a 70:30 train/test split. The implementation provides chronological_split for a paper-style comparison:

train, test = chronological_split(
    dataset,
    train_fraction=0.70,
    validation_fraction=0.0,
)

The function slices ordered samples rather than randomly selecting indices. With validation data, the order is:

training windows -> validation windows -> test windows

The metadata-preserving helper is conceptually:

def dataset_slice(dataset, start, stop):
    return WindowedDataset(
        X=dataset.X[start:stop],
        y=dataset.y[start:stop],
        window_start=dataset.window_start[start:stop],
        window_end=dataset.window_end[start:stop],
        y_timestamp=dataset.y_timestamp[start:stop],
        feature_names=dataset.feature_names,
    )

A useful ordering check is:

train_times = train.y_timestamp
test_times = test.y_timestamp
assert train_times.max() < test_times.min()

This confirms that target timestamps are ordered across the sample split. It does not prove that the corresponding input histories are disjoint, because stride-one windows can overlap at the boundary.

The exact boundary remains a reconstruction. The paper does not specify the split date or whether it splits raw observations before window construction or partitions already-created windows. Those choices can produce different edge behavior and should be recorded in experiment metadata.

6.7 Walk-forward evaluation for trading conclusions

A single 70:30 split answers only how one fixed training period performs on one later period. It does not show how results change as new observations arrive.

An expanding walk-forward procedure repeatedly preserves the information boundary:

train on [0, ..., t]
forecast the next block
advance the boundary
expand the training set
forecast the next block
repeat

The walk_forward_splits generator is used like this:

for train, validation, test in walk_forward_splits(
    dataset,
    initial_train_size=500,
    validation_size=0,
    test_size=20,
    step=20,
):
    # Fit only on train, optionally tune on validation,
    # then evaluate on the later test block.
    pass

Each test block follows its training block. The training set expands over time; a rolling fixed-length training window would be another explicit choice.

The paper does not report walk-forward results, so such results are not reproductions of its tables. They are a more appropriate protocol for trading conclusions because they repeatedly test the model on observations that follow the fitting period.

6.8 What the generated test artifact actually demonstrates

The generated tests/test_data_and_streaks.py uses a compatibility helper with a numeric price array and a separate timestamp object. In the current make_windows implementation, the numeric-array path does not accept or preserve that separate timestamp argument; it creates placeholder daily timestamps. Therefore, the test's meaningful checks are the numeric window boundaries and target values, while its timestamp behavior should not be interpreted as validation of real source timestamps.

A timestamp-aware illustrative example, using the public data contract directly, is:

prices = np.arange(10.0, 17.0)
timestamps = pd.date_range(
    "2024-01-01", periods=len(prices), freq="D"
)
price_data = PriceData(
    pd.DataFrame({"timestamp": timestamps, "price": prices})
)

dataset = make_windows(
    price_data,
    window_length=3,
    horizon=1,
    stride=1,
)

np.testing.assert_allclose(dataset.y, prices[3:])
assert (dataset.y_timestamp > dataset.window_end).all()

This is an illustrative contract for timestamp-aware input, not a claim that the generated test suite was executed. It also makes the distinction clear: use PriceData or a timestamp-aware DataFrame when timestamps matter; use raw arrays only when placeholder metadata is acceptable or when a higher-level API supplies the timestamp mapping separately.

The project was reviewed primarily through static checks, and semantic code verification was skipped. The intended contracts are described here, but this section does not claim that the complete test suite passed or that runtime behavior was independently confirmed.

6.9 Practical preparation workflow

For a local asset file, a disciplined workflow is:

price_data = load_price_csv(
    "data/gold.csv",
    timestamp_column="timestamp",
    price_column="price",
    asset="gold",
)

# Establish this from a documented date or raw-row boundary.
training_row_count = 500
train_rows = price_data.frame.iloc[:training_row_count]
scaler = ChronologicalScaler().fit(
    train_rows[["price"]].to_numpy(dtype=float)
)

windows = make_windows(
    price_data,
    window_length=100,
    horizon=1,       # use 3 only when a three-step target is intended
    stride=1,
    scaler=scaler,
    fit_scaler=False,
)
train, test = chronological_split(windows, train_fraction=0.70)

The row boundary must be established before fitting the scaler and must reflect the intended training period. The split shown above is a paper-style sample split applied after window construction. For a stricter out-of-time study, define the raw-row boundary and any gap explicitly, then use walk-forward partitions for repeated evaluation.

6.10 Preparation limits

This data layer provides:

• the paper's 100-observation, stride-one setup;

• one-step and three-step target shapes;

• timestamp metadata for PriceData and timestamp-aware DataFrame inputs;

• chronological ordering and sample splits;

• training-only scaling primitives;

• expanding walk-forward partitions.

It does not resolve the paper's missing data source, instrument definition, feature representation, exact split date, forecast horizon, or execution protocol. It also cannot automatically determine whether a caller fitted a scaler on an appropriate subset or whether overlapping input histories are acceptable for a particular evaluation design.

The essential causal contract is therefore: the historical window ends first, the target occurs later, scaling uses only the declared training information, and execution is aligned to a timestamp after the information endpoint. Later forecasting and backtesting layers should preserve that contract explicitly.

7. Keep Forecast Accuracy Separate from Trading Performance

The paper compares forecasting models with RMSE, MAE, MAPE, and R-squared. These metrics answer one question: how close were the predictions to their targets? They do not establish that a trading strategy is profitable after signal rules, position sizing, VaR, execution timing, fees, slippage, and liquidity are included.

The implementation keeps these concerns separate in src/quantitative_trading_model/metrics.py:

• forecast metrics operate on target and prediction arrays;

• portfolio metrics operate on a marked-to-market value path and optional execution records;

• backtest.py supplies the fills, traded notionals, and fees used by portfolio reporting.

This distinction is also documented in README.md, which warns that forecast accuracy does not establish profitability and recommends reporting drawdown, volatility, turnover, fees, and a buy-and-hold comparison.

7.1 Forecast error metrics

Let y_k be the observed target and y_hat_k the corresponding prediction for evaluation item k.

Mean absolute error

MAE is measured in the target's units. For raw dollar prices, it is measured in dollars. The paper-inspired recurrent models also use MAE as their training loss.

Root mean squared error

Squaring the residuals gives larger errors more influence. RMSE is scale-dependent, so raw RMSE values for gold and Bitcoin should not be compared without considering their different price magnitudes and target representations.

Mean absolute percentage error

The implementation reports MAPE in percentage points: 2.5 means 2.5%, not 0.025. MAPE is undefined or unstable for zero and near-zero targets. forecast_metrics therefore exposes two controls:

• mape_epsilon defines the near-zero threshold;

• mape_zero_policy="exclude" omits those targets, while "raise" rejects them.

The number of observations included in MAPE is stored separately from the total sample count. Two MAPE values should not be compared without checking that policy and count.

R-squared

R-squared compares residual variation with the variation of the targets around their mean. It can be negative when predictions are worse than the constant mean-target baseline. If every target is identical, the denominator is zero; the generated implementation returns 1.0 for an exact prediction and 0.0 otherwise.

7.2 The forecast-metrics interface

The public function is forecast_metrics(actual, predicted, ...). It validates finite inputs, flattens supported array-like values, checks that targets and predictions have equal lengths, and returns a ForecastMetrics dataclass.

The following is a shortened tutorial excerpt, not the complete generated source. The actual module also contains helper functions for input validation, rounding, and serialization.

metrics = forecast_metrics(
    actual,
    predicted,
    mape_epsilon=1e-12,
    mape_zero_policy="exclude",
)

print(metrics.rmse)
print(metrics.mae)
print(metrics.mape_percent)
print(metrics.r_squared)
print(metrics.as_dict())

ForecastMetrics.as_dict() is the generated dataclass's serialization method. It includes the four metrics, sample counts, the MAPE threshold, and the selected MAPE policy. The full implementation should be treated as authoritative rather than this abbreviated excerpt.

For example:

import numpy as np
from quantitative_trading_model.metrics import forecast_metrics

actual = np.array([100.0, 110.0, 90.0])
predicted = np.array([102.0, 107.0, 93.0])

result = forecast_metrics(actual, predicted)

The residuals are [2, -3, 3]. Thus MAE is 8/3, RMSE is sqrt(22/3), and MAPE is the mean of the three absolute relative errors multiplied by 100. These values describe only the prediction sample. They do not determine whether a strategy would buy, sell, or hold.

For multi-step predictions represented as (samples, horizon), the generated metric helper flattens the arrays. The caller must ensure that the target and prediction arrays have identical ordering. A production experiment should also retain target timestamps so that metrics are calculated on aligned observations.

7.3 Why forecast accuracy is not trading performance

A model can have favorable forecast errors and still produce a poor trading result:

A predicted move may be too small to cover entry and exit costs.

Price-level accuracy does not guarantee correct direction relative to the current price.

A forecast evaluated at a closing timestamp may not describe the price available at execution.

Position sizing can amplify a small forecast error.

A shuffled or otherwise optimistic time-series split may not represent deployment conditions.

Fees, spreads, slippage, partial fills, and liquidity are portfolio effects, not components of RMSE or MAE.

The paper reports strong Att-BiLSTM forecasting metrics, but those reported values remain unverified claims because the source data, preprocessing, target definition, and test protocol are unavailable. Even reproducing the metric ranking would not prove that the resulting trading strategy is profitable.

7.4 Portfolio metrics use a different vocabulary

Portfolio evaluation begins with marked-to-market values:

The generated implementation reports or supports the following measures.

Cumulative return

\[ R{0:T} = \frac{VT}{V_0}-1. \]

This reflects trades, fees, remaining holdings, and the configured liquidation policy.

Drawdown and maximum drawdown

\[ Dt = \frac{Vt}{\max{s\le t}Vs}-1. \]

Drawdown is non-positive: -0.25 means the portfolio is 25% below its previous running peak. Maximum drawdown is the minimum value in the drawdown series.

Volatility

For periodic returns r_t, annualized volatility is approximately:

\[ \sigma{annual}=\operatorname{std}(rt)\sqrt{P}, \]

where P is the configured number of periods per year. The default of 252 is an implementation assumption for daily observations, not a paper-specified value.

Sharpe ratio

\[ \operatorname{Sharpe} = \frac{\operatorname{mean}(rt-r{f,t})} {\operatorname{std}(rt-r{f,t})}\sqrt{P}. \]

The implementation converts an annual risk-free rate into a periodic compound-equivalent rate. Its default risk-free rate is zero, another explicit evaluation choice rather than a paper result.

Turnover and fees

The generated metrics layer defines turnover as traded notional divided by mean portfolio value:

\[ \operatorname{turnover} = \frac{\sumi \operatorname{traded\ notional}i}{\operatorname{mean}(V_t)}. \]

Fees are supplied by the execution layer and summed separately:

\[ \operatorname{total\ fees}=\sumi\operatorname{fee}i. \]

This preserves the distinction between portfolio valuation and fill-level accounting.

7.5 The portfolio-metrics interface

portfolio_metrics accepts a value path and optional periodic returns, traded notionals, fees, annualization frequency, and annual risk-free rate. The following is a shortened excerpt, not a complete copy of metrics.py:

portfolio = portfolio_metrics(
    portfolio_values,
    returns=None,              # derive returns from consecutive values
    traded_notionals=notionals,
    fees=fees,
    periods_per_year=252,
    risk_free_rate_annual=0.0,
)

print(portfolio.cumulative_return)
print(portfolio.maximum_drawdown)
print(portfolio.annualized_volatility)
print(portfolio.sharpe_ratio)
print(portfolio.turnover)
print(portfolio.total_fees)

The full generated function additionally validates positive portfolio values, finite inputs, return-length conventions, nonnegative notionals, scalar-versus-sequence fee inputs, and risk-free-rate settings. It returns a PortfolioMetrics dataclass with an as_dict() method.

7.6 A report should retain both layers

A useful experiment table keeps forecast and portfolio results in separate columns:

| Forecast evaluation | Portfolio evaluation | | --- | --- | | RMSE | Final value | | MAE | Cumulative return | | MAPE and MAPE policy | Maximum drawdown | | R-squared | Annualized volatility | | Target horizon | Sharpe ratio | | Test timestamps | Turnover | | Scaling metadata | Total fees |

This prevents a low prediction error from being mistaken for a trading objective. A buy-and-hold benchmark should also use the same asset, period, initial capital, fee convention, and valuation policy.

7.7 Verification scope

The supplied project currently does not include a dedicated metrics test module. tests/test_data_and_streaks.py tests data preparation and streak behavior; it does not directly test MAPE policies, constant-target R-squared behavior, drawdown calculations, or fee aggregation. Those behaviors are documented and statically inspectable invariants, not claims of executed test coverage.

Static review can still check that:

• forecast arrays are finite and aligned;

• MAPE has an explicit near-zero policy;

• constant-target R-squared behavior is deterministic;

• portfolio values are positive and drawdown is non-positive;

• turnover and fees are supplied separately from forecast errors.

However, semantic code verification was skipped, and the generated project was not claimed to have been executed. These checks therefore do not establish runtime correctness, profitability, or reproduction of the paper's metrics.

The practical rule is: use forecast metrics to evaluate the predictor and portfolio metrics to evaluate the trading system. Both are necessary, but neither is a substitute for the other.

8. Implement the Attention-Enhanced BiLSTM

The paper identifies an attention-enhanced bidirectional LSTM, or Att-BiLSTM, as its primary forecasting model. Its intended flow is:

Read a historical window, such as the latest 100 observations.

Process that window in both temporal directions with a BiLSTM.

Produce one hidden representation for every time step.

Use attention to assign larger weights to more relevant historical steps.

Combine the weighted representations into one context vector.

Map that context vector to a one-day or multi-day forecast.

The generated implementation follows this structure in src/quantitative_trading_model/models/forecasters.py. The attention mathematics is isolated in src/quantitative_trading_model/models/attention.py, while tests/test_model_shapes.py checks the intended tensor contracts when the optional PyTorch dependency is available.

Reconstruction boundary: The paper specifies the broad Att-BiLSTM idea, dot-product attention, 20% dropout, MAE, and RMSprop. It does not specify the number of recurrent layers, hidden dimensions, attention-query construction, learning rate, training duration, or exact target representation. Those details are configurable implementation choices, not verified transcriptions of the original model.

8.1 What the BiLSTM contributes

An ordinary LSTM processes a sequence in one direction. Given an input window ordered from older to newer observations, its hidden state at time step i summarizes the observations up to that point in the forward direction.

A BiLSTM adds a second LSTM that processes the same historical window in reverse. At each time step, the model combines the forward and backward states. If the one-direction hidden size is h, the combined representation commonly has size 2h:

input sequence:       (batch, time, features)
forward LSTM states:  (batch, time, h)
backward LSTM states: (batch, time, h)
combined states:      (batch, time, 2h)

The important timing condition is that the complete window must end at the information timestamp. If the signal is generated after observing time t, the input may contain observations through t, but not t+1 or any later observation. The backward LSTM is not automatically leakage because it looks backward inside the already available window. It becomes leakage only if the window itself contains future observations.

The model therefore relies on the contract established by the data layer:

window ends at t
forecast is generated using data through t
execution occurs at a later bar

This is why the window metadata and timestamp checks discussed in the previous section matter to the neural model as well as to the backtest.

8.2 Dot-product temporal attention

The BiLSTM produces a sequence of hidden representations rather than one final vector. The paper's attention mechanism decides how to combine those representations.

Let x_i be the BiLSTM representation at time step i, and let q be a query vector. The paper gives the following operations.

Compatibility score

\[ S(xi,q) = xi^Tq. \]

This dot product produces one scalar score for every time step. A larger score indicates stronger alignment between x_i and the query q.

Attention weights

The scores are normalized with a softmax:

\[ ai = \operatorname{softmax}i(S(xi,q)) = \frac{\exp(S(xi,q))} {\sumj \exp(S(xj,q))}. \]

The subscript on softmax is important. Normalization must happen over the time dimension, so that the weights for one sequence sum to one across its historical observations. Normalizing across features or across the batch would produce a different mechanism.

Context vector

Finally, the representations are combined with their attention weights:

\[ \operatorname{context}(q,x)=\sumi ai x_i. \]

For a batch of sequences, the expected tensor shapes are:

sequence:  (batch, time, feature_dim)
scores:    (batch, time)
weights:   (batch, time)
context:   (batch, feature_dim)
forecast:  (batch, horizon)

The implementation uses DotProductTemporalAttention for these operations:

# src/quantitative_trading_model/models/attention.py

class DotProductTemporalAttention(nn.Module):
    def forward(
        self,
        sequence: Tensor,
        query: Optional[Tensor] = None,
        *,
        return_weights: bool = False,
    ) -> Tensor | Tuple[Tensor, Tensor]:
        self._validate_sequence(sequence)
        prepared_query = self._prepare_query(sequence, query)

        # Equation (1): one dot product per time step.
        scores = torch.einsum("btd,bd->bt", sequence, prepared_query)

        # Equation (2): normalize across time, not features or batches.
        weights = torch.softmax(scores, dim=1)

        # Equation (3): weighted sum of sequence representations.
        context = torch.sum(weights.unsqueeze(-1) * sequence, dim=1)

        self._last_attention_weights = weights
        if return_weights:
            return context, weights
        return context

The Einstein summation expression "btd,bd->bt" means:

• b: batch item;

• t: time step;

• d: feature dimension;

• the feature dimension is multiplied and summed away;

• the result retains batch and time, producing one score per time step.

The next line, torch.softmax(scores, dim=1), is the key implementation detail. For a tensor shaped (batch, time), dimension 1 is time. The result has one probability-like weight for every historical position.

8.3 The unspecified query vector

The paper uses a query vector q in its equations but does not explain where that vector comes from. This missing detail affects the architecture, so the implementation makes the choice explicit rather than hiding it.

DotProductTemporalAttention supports two modes:

• a learned query, stored as a trainable parameter and used by default;

• a caller-provided query, useful for experiments with another interpretation.

The learned-query choice is visible in the constructor:

# src/quantitative_trading_model/models/attention.py

if learned_query:
    query = torch.empty(self.feature_dim)
    nn.init.normal_(query, mean=0.0, std=float(query_init_scale))
    self.query = nn.Parameter(query)
else:
    self.register_parameter("query", None)

When no explicit query is supplied, _prepare_query retrieves the learned vector and broadcasts it across the batch. When a query is supplied, the method checks its dimensionality, device, data type, and finiteness before using it.

This is a sound interface decision for an educational implementation, but it must be labeled correctly:

The learned query is a reconstruction of an unspecified paper detail. It is not evidence that the original authors used a learned query rather than a final hidden state, a separate projection, or another query-generation mechanism.

8.4 From sequence states to a forecast

The primary model in forecasters.py first creates a bidirectional LSTM whose batch_first=True input convention is (batch, time, features):

# src/quantitative_trading_model/models/forecasters.py

self.encoder = nn.LSTM(
    input_size=input_size,
    hidden_size=hidden_size,
    num_layers=num_layers,
    batch_first=True,
    dropout=recurrent_dropout,
    bidirectional=True,
)

self.dropout = nn.Dropout(dropout)
self.attention = DotProductTemporalAttention(self.output_size)
self.head = nn.Linear(self.output_size, horizon)

self.output_size is hidden_size * 2, because the forward and backward states are concatenated. The dense head converts the attention context into the requested forecast horizon. For example:

horizon = 1  -> output shape (batch, 1)
horizon = 3  -> output shape (batch, 3)

The forward pass is intentionally short because each responsibility is delegated to a clear layer:

class AttBiLSTMForecaster(nn.Module):
    def forward(self, inputs: Tensor) -> Tensor:
        _validate_input_tensor(inputs)
        sequence, _ = self.encoder(inputs)
        sequence = self.dropout(sequence)

        context = self.attention(sequence)
        return self.head(self.dropout(context))

Conceptually, this is:

historical window
    -> bidirectional LSTM sequence states
    -> dropout on sequence states
    -> temporal dot-product attention
    -> context vector
    -> dropout on context
    -> linear forecast head

The exact generated implementation also retains the latest attention weights when the attention layer exposes them. That is useful for inspection and debugging, but attention weights should not automatically be interpreted as a complete causal explanation of the prediction.

8.5 Dropout, MAE, and RMSprop

The paper reports a dropout rate of 20%, represented by the default dropout=0.20. Dropout randomly suppresses a fraction of activations during training, which can reduce reliance on a narrow set of internal features. The paper's wording does not establish whether its 20% applies to ordinary feed-forward dropout, recurrent dropout, or both, so the generated code uses explicit nn.Dropout layers and documents the recurrent-layer behavior.

The training function uses MAE loss:

criterion = nn.L1Loss()

This corresponds to:

\[ \operatorname{MAE} = \frac{1}{N}\sumk |yk-\hat y_k|. \]

The optimizer is RMSprop:

optimizer = torch.optim.RMSprop(
    model.parameters(),
    lr=learning_rate,
)

This corrects an important terminology issue in the paper interpretation. RMSprop is an optimizer, not an activation function. It controls how model parameters are updated after gradients are calculated. The activation behavior of the recurrent cells and the linear output head is a separate architectural matter.

The implementation keeps learning rate, hidden size, number of layers, batch size, epochs, and patience configurable because the paper does not provide enough information to hard-code them as original values.

8.6 Why the model trains on chronological batches

train_forecaster constructs a DataLoader with shuffle=False:

train_loader = DataLoader(
    TensorDataset(
        torch.from_numpy(x_train),
        torch.from_numpy(y_train),
    ),
    batch_size=batch_size,
    shuffle=False,
)

Adjacent sliding windows are highly dependent because they share most of their observations. Turning off shuffling preserves the temporal order and makes the training protocol easier to audit. This does not remove all dependence between windows, but it avoids adding a separate randomization step that could make the time-series experiment harder to interpret.

This is a conservative engineering choice rather than a claim that the paper used the same batching policy. The paper mentions shuffled or randomly arranged training behavior in places, but does not provide enough detail to establish a reproducible protocol. For trading conclusions, chronological and walk-forward evaluation remain more important than reproducing an unclear shuffling choice.

8.7 Model-shape checks

The generated tests focus on architecture contracts rather than empirical performance. For example, tests/test_model_shapes.py checks that a configured attention layer returns the expected context shape and that its weights normalize across time:

@pytest.mark.skipif(not TORCH_AVAILABLE, reason="PyTorch is an optional dependency")
class TestAttentionShapes:
    def test_attention_weights_are_normalized_across_time(self) -> None:
        attention = _construct_attention()
        sequence = torch.randn(3, 7, 8)
        output = attention(sequence)
        _, weights = _unpack_attention_output(output)

        if weights is None:
            weights = getattr(attention, "last_attention_weights", None)

        assert tuple(weights.shape) == (3, 7)
        assert torch.isfinite(weights).all()
        assert torch.all(weights >= 0)
        assert torch.allclose(
            weights.sum(dim=1),
            torch.ones(3),
            atol=1e-5,
        )

The test expresses the mathematical invariant directly:

weights.shape == (batch, time)
weights >= 0
sum(weights over time) == 1

Additional shape tests check one-step and three-step outputs:

@pytest.mark.parametrize("horizon", [1, 3])
def test_attention_bilstm_output_horizon(self, horizon: int) -> None:
    model = _construct_forecaster("AttBiLSTMForecaster", horizon=horizon)
    inputs = torch.randn(2, 100, 3)
    predictions = _model_output(model, inputs)

    assert predictions.shape[0] == 2
    assert predictions.shape[-1] == horizon

These checks do not demonstrate that the model matches the paper's forecast metrics. They only verify the intended tensor interface and attention normalization when the optional dependency is installed. The local verification record should therefore be read as static and structural evidence, not as a claim that the model was executed successfully or that the paper's numerical results were reproduced.

8.8 What is direct and what is reconstructed here?

The implementation status of this model can be summarized as follows:

The model is therefore a useful translation of the paper's stated architecture, but not a verified reproduction of its original neural network. Exact comparison would require the original data, preprocessing, target definition, full architecture, training hyperparameters, and untouched evaluation protocol.

9. Walk Through Attention, Training, and Prediction Functions

This section follows the generated Python implementation from tensor validation through attention, recurrent encoding, training, and prediction. The primary model is an attention-enhanced bidirectional LSTM, or Att-BiLSTM. The paper specifies the broad architecture, 20% dropout, MAE loss, and RMSprop optimization, but it does not specify several architectural and training details. Those details remain explicit reconstruction choices.

The examples below describe the generated interfaces and intended invariants. They do not claim that the code was executed, that the tests passed, or that the paper's empirical results were reproduced.

9.1 Optional PyTorch imports

PyTorch is an optional dependency. The data, streak, sizing, risk, and portfolio components are intended to remain usable without installing the deep-learning stack. The model modules therefore import PyTorch conditionally:

try:
    import torch
    from torch import Tensor, nn
except ImportError:
    torch = None
    Tensor = Any
    nn = type("nn", (), {"Module": _ModuleFallback})

The fallback permits the module to be imported in a minimal installation. Constructing DotProductTemporalAttention or a recurrent forecaster still requires PyTorch and raises a clear ImportError if it is unavailable. This is preferable to silently substituting a different model.

The package initializer is also side-effect free. Importing quantitative_trading_model does not load market data, train a model, access a network, or create a trading connection.

9.2 The attention layer's tensor contract

DotProductTemporalAttention expects a sequence tensor with shape:

(batch, time, feature_dim)

For the paper-inspired data setup, time is normally 100. The feature dimension is the size of each recurrent representation. A bidirectional LSTM commonly produces a representation whose width is twice its hidden size because its forward and backward states are concatenated.

The attention layer returns a context tensor with shape:

(batch, feature_dim)

Attention weights have shape:

(batch, time)

They can be returned explicitly or inspected after a forward pass. The weights are useful for diagnostics and shape tests, but they should not automatically be treated as a complete causal explanation of the model.

9.3 Constructing attention and representing the missing query

The paper gives the score equation

\[ S(xi,q)=xi^Tq, \]

but does not explain how the query vector q is created. The constructor exposes this ambiguity:

class DotProductTemporalAttention(nn.Module):
    def __init__(
        self,
        feature_dim: Optional[int] = None,
        *,
        input_dim: Optional[int] = None,
        learned_query: bool = True,
        query_init_scale: float = 0.02,
    ) -> None:
        ...

feature_dim and input_dim are aliases. If both are supplied, they must agree. The dimension must be a positive integer because the unprojected dot product requires the query and sequence representations to have the same width.

With learned_query=True, the layer creates a trainable query parameter:

if learned_query:
    query = torch.empty(self.feature_dim)
    nn.init.normal_(query, mean=0.0, std=float(query_init_scale))
    self.query = nn.Parameter(query)
else:
    self.register_parameter("query", None)

The learned query is a reconstruction choice, not a verified detail from the paper. With learned_query=False, the caller must provide a query to every forward call.

9.4 Validating the sequence tensor

Before computing attention scores, _validate_sequence checks the input shape and values:

def _validate_sequence(self, sequence: Tensor) -> Tuple[int, int, int]:
    if sequence.ndim != 3:
        raise ValueError("sequence must have shape (batch, time, feature_dim)")

    batch, time, feature = (int(value) for value in sequence.shape)
    if batch <= 0 or time <= 0:
        raise ValueError("batch and time dimensions must be positive")
    if feature != self.feature_dim:
        raise ValueError("sequence feature dimension does not match configuration")
    if not sequence.is_floating_point() and not sequence.is_complex():
        raise TypeError("sequence must use a floating-point or complex dtype")
    if not torch.isfinite(sequence).all():
        raise ValueError("sequence contains NaN or infinite values")
    return batch, time, feature

The checks protect separate assumptions:

• Rank: the layer needs batch, time, and feature axes.

• Nonempty dimensions: an empty sequence cannot produce a context vector.

• Feature agreement: the dot product requires matching dimensions.

• Numeric type: the implementation technically accepts floating-point and complex PyTorch tensors.

• Finite values: NaNs and infinities could propagate through scores and softmax.

In normal financial-model usage, inputs are real-valued floating-point tensors. Complex-tensor acceptance is a property of the generated validation condition, not a paper requirement or a recommended financial-data representation.

These checks do not verify temporal causality. A tensor can have the correct shape while containing future observations. Timestamp alignment and target-after-window rules remain responsibilities of the data and experiment layers.

9.5 Preparing and broadcasting the query

The query may be a single vector shared across the batch or a separate vector for every batch item:

def _prepare_query(self, sequence: Tensor, query: Optional[Tensor]) -> Tensor:
    batch = sequence.shape[0]
    candidate = self.query if query is None else query

    if candidate is None:
        raise ValueError("query is required when learned_query=False")

    if candidate.ndim == 1:
        if candidate.shape[0] != self.feature_dim:
            raise ValueError("query dimension does not match feature_dim")
        candidate = candidate.unsqueeze(0).expand(batch, -1)
    elif candidate.ndim == 2:
        if candidate.shape != (batch, self.feature_dim):
            raise ValueError("batched query has the wrong shape")
    else:
        raise ValueError("query must have shape (feature_dim,) or (batch, feature_dim)")

    if candidate.device != sequence.device:
        raise ValueError("sequence and query must be on the same device")
    if candidate.dtype != sequence.dtype:
        raise ValueError("sequence and query must use the same dtype")
    if not torch.isfinite(candidate).all():
        raise ValueError("query contains NaN or infinite values")
    return candidate

A one-dimensional query is expanded across the batch. A two-dimensional query must have one row per batch item. Device and dtype checks prevent errors such as combining tensors on different devices or with incompatible precision.

The method does not project either operand into another space. That follows the displayed paper equation, although the paper does not establish that its actual implementation avoided projections.

9.6 Forward attention: scores, weights, and context

The core forward method is:

def forward(
    self,
    sequence: Tensor,
    query: Optional[Tensor] = None,
    *,
    return_weights: bool = False,
) -> Tensor | Tuple[Tensor, Tensor]:
    self._validate_sequence(sequence)
    prepared_query = self._prepare_query(sequence, query)

    scores = torch.einsum("btd,bd->bt", sequence, prepared_query)
    weights = torch.softmax(scores, dim=1)
    context = torch.sum(weights.unsqueeze(-1) * sequence, dim=1)

    self._last_attention_weights = weights
    if return_weights:
        return context, weights
    return context

The three central operations correspond to the paper's equations.

Dot-product scores

scores = torch.einsum("btd,bd->bt", sequence, prepared_query)

Here b is the batch index, t is the time index, and d is the feature index. The result has shape (batch, time), with one scalar score for each historical time step.

Softmax across time

weights = torch.softmax(scores, dim=1)

dim=1 is essential because the time axis is the second dimension. For each batch item:

\[ \sumi ai=1, \qquad a_i\geq 0. \]

Applying softmax across features would produce a different operation and would not assign relative weights to historical time steps.

Weighted context

context = torch.sum(weights.unsqueeze(-1) * sequence, dim=1)

weights.unsqueeze(-1) changes the weight shape from (batch, time) to (batch, time, 1). Each representation is multiplied by its scalar attention weight, and summing over time produces one context vector per batch item.

The generated implementation stores the latest weights and checks that both weights and context are finite. Callers that retain weights across many batches should detach them first if they do not need gradients, otherwise the retained tensors may keep autograd graphs alive.

9.7 Retrieving attention weights correctly

The default call returns only the context:

context = attention(sequence)

To receive both context and weights in one call, pass return_weights=True:

context, weights = attention(sequence, return_weights=True)
assert context.shape == (sequence.shape[0], sequence.shape[2])
assert weights.shape == (sequence.shape[0], sequence.shape[1])

Alternatively, the caller can use the inspection method:

context = attention(sequence)
weights = attention.last_attention_weights

or recompute them explicitly:

weights = attention.attention_weights(sequence)

This distinction matters because attention(sequence) does not return a tuple by default. The generated model uses the default context-only call and reads the most recent weights from the attention layer when they are available.

9.8 The shared recurrent baseline

forecasters.py defines _RecurrentForecaster, which supplies common behavior for LSTM, GRU, and non-attention BiLSTM baselines. The recurrent class is selected from the requested kind:

recurrent_class = nn.LSTM if recurrent_kind == "lstm" else nn.GRU
self.encoder = recurrent_class(
    input_size=input_size,
    hidden_size=hidden_size,
    num_layers=num_layers,
    batch_first=True,
    dropout=recurrent_dropout,
    bidirectional=bidirectional,
)
self.dropout = nn.Dropout(dropout)
self.head = nn.Linear(self.output_size, horizon)

The representation width is:

self.output_size = hidden_size * (2 if bidirectional else 1)

The factor of two accounts for the forward and backward hidden streams in a bidirectional layer. The linear head maps the representation to the configured forecast horizon.

PyTorch recurrent dropout is applied between recurrent layers. Therefore the generated code uses:

recurrent_dropout = dropout if num_layers > 1 else 0.0

It also includes a separate nn.Dropout(dropout) stage. Whether the paper used recurrent dropout, feed-forward dropout, or both is unspecified, so this remains a reconstruction choice.

9.9 Encoding and baseline prediction

For recurrent baselines, the encoder returns the final time-step representation:

def encode(self, inputs: Tensor) -> Tensor:
    _validate_input_tensor(inputs)
    sequence, _ = self.encoder(inputs)
    return self.dropout(sequence[:, -1, :])

def forward(self, inputs: Tensor) -> Tensor:
    return self.head(self.encode(inputs))

The input contract is (batch, time, features). The data layer normally supplies time=100, but the model does not hard-code that number. The configured window builder determines the time length.

9.10 Constructing the Att-BiLSTM

The primary model returns a representation for every time step so attention can operate over the full historical window:

self.encoder = nn.LSTM(
    input_size=input_size,
    hidden_size=hidden_size,
    num_layers=num_layers,
    batch_first=True,
    dropout=recurrent_dropout,
    bidirectional=True,
)
self.dropout = nn.Dropout(dropout)
self.attention = DotProductTemporalAttention(self.output_size)
self.head = nn.Linear(self.output_size, horizon)

The tensor flow is:

inputs                 (batch, time, input_size)
BiLSTM sequence        (batch, time, 2 * hidden_size)
dropout sequence       (batch, time, 2 * hidden_size)
attention context      (batch, 2 * hidden_size)
linear output head     (batch, horizon)

The paper does not specify hidden size, recurrent depth, query construction, dense-layer structure, or output representation. These values must therefore be recorded as configuration metadata rather than presented as recovered paper details.

9.11 Connecting the Att-BiLSTM to attention

The primary model's forward path is:

def forward(self, inputs: Tensor) -> Tensor:
    _validate_input_tensor(inputs)
    sequence, _ = self.encoder(inputs)
    sequence = self.dropout(sequence)

    attended = self.attention(sequence)
    if isinstance(attended, tuple):
        context, weights = attended
        self._last_attention_weights = weights
    else:
        context = attended
        weights = getattr(self.attention, "last_attention_weights", None)
        if isinstance(weights, Tensor):
            self._last_attention_weights = weights

    return self.head(self.dropout(context))

The intended sequence is:

Validate the input rank and values.

Process the historical window with the bidirectional LSTM.

Apply dropout to the sequence of hidden representations.

Use temporal attention to form one context vector.

Apply dropout to that context.

Map the context to one or more forecast values.

The tuple branch is defensive compatibility logic. The generated attention layer returns only the context unless return_weights=True, but the model can also accommodate an attention implementation that returns (context, weights).

9.12 Prediction methods and evaluation mode

The model classes provide a prediction method that disables training behavior and gradient tracking:

def predict(self, inputs: np.ndarray | Tensor) -> np.ndarray:
    self.eval()
    with torch.no_grad():
        tensor = _as_input_tensor(inputs)
        return self(tensor).detach().cpu().numpy()

The three operations have distinct purposes:

• self.eval() disables training-specific behavior such as dropout.

• torch.no_grad() avoids constructing gradient graphs during inference.

• .detach().cpu().numpy() produces a NumPy result for metrics and signal generation.

predict_forecaster additionally extracts X from a windowed dataset and can apply an inverse transformation. The scaler must have been fitted using training data only. The model does not determine whether its target is a raw price, scaled price, return, or percentage change; that convention belongs to the configuration and experiment protocol.

9.13 Model construction with build_forecaster

The factory normalizes common names and maps them to model classes:

aliases = {
    "att-bilstm": AttBiLSTMForecaster,
    "attbilstm": AttBiLSTMForecaster,
    "lstm": LSTMForecaster,
    "gru": GRUForecaster,
    "bilstm": BiLSTMForecaster,
}

Unspecified settings are read from configuration:

kwargs = {
    "hidden_size": hidden_size if hidden_size is not None else _config_value(config, "hidden_size", 64),
    "num_layers": num_layers if num_layers is not None else _config_value(config, "num_layers", 1),
    "dropout": dropout if dropout is not None else _config_value(config, "dropout", 0.20),
    "horizon": horizon if horizon is not None else _config_value(config, "horizon", 1),
}

The 20% dropout default is paper-inspired. The default hidden size and layer count are engineering defaults, not findings recovered from the paper.

9.14 Deterministic seeding

train_forecaster calls _set_seed before training:

def _set_seed(seed: int) -> None:
    random.seed(seed)
    np.random.seed(seed)
    torch.manual_seed(seed)
    if torch.cuda.is_available():
        torch.cuda.manual_seed_all(seed)

This aligns the main Python, NumPy, and PyTorch random generators. A seed improves repeatability but does not guarantee bit-for-bit equality across hardware, PyTorch versions, CUDA kernels, or data pipelines. It should be recorded as experiment metadata, not treated as proof of reproducibility.

9.15 Extracting training data and normalizing target shape

The _extract_xy helper accepts a WindowedDataset-like object exposing X and y, or a two-item (X, y) pair:

def _extract_xy(data: Any) -> tuple[np.ndarray, np.ndarray]:
    if hasattr(data, "X") and hasattr(data, "y"):
        x, y = data.X, data.y
    elif isinstance(data, (tuple, list)) and len(data) == 2:
        x, y = data
    else:
        raise TypeError("data must expose X and y or be a two-item (X, y) pair")

    x_array = np.asarray(x, dtype=np.float32)
    y_array = np.asarray(y, dtype=np.float32)
    if x_array.ndim != 3:
        raise ValueError("X must have shape (samples, time, features)")
    if y_array.ndim == 1:
        y_array = y_array[:, None]
    if y_array.ndim != 2:
        raise ValueError("y must have shape (samples, horizon)")
    if len(x_array) != len(y_array):
        raise ValueError("X and y must contain the same number of samples")
    return x_array, y_array

Converting a one-step target from (samples,) to (samples, 1) gives both one-step and three-step models the same output contract. The helper also checks for empty datasets and non-finite values. It does not accept a final test set implicitly; callers must pass training and validation partitions explicitly.

9.16 Training with MAE and RMSprop

train_forecaster first checks that the target horizon agrees with the configuration and model:

horizon = int(_config_value(config, "horizon", y_train.shape[1]))
if y_train.shape[1] != horizon:
    raise ValueError("training target horizon does not match configured horizon")
if getattr(model, "horizon", horizon) != horizon:
    raise ValueError("model horizon and target horizon do not match")

It then creates a chronological data loader:

train_loader = DataLoader(
    TensorDataset(torch.from_numpy(x_train), torch.from_numpy(y_train)),
    batch_size=batch_size,
    shuffle=False,
)

shuffle=False is a deliberate time-series choice. Adjacent stride-one windows share most of their observations. Preserving order does not remove dependence, but it avoids adding random mixing and keeps the temporal protocol visible.

The paper-inspired loss and optimizer are configured as follows:

criterion = nn.L1Loss()
optimizer = torch.optim.RMSprop(model.parameters(), lr=learning_rate)

nn.L1Loss() implements mean absolute error. RMSprop updates model parameters; it is not an activation function. The learning rate and other optimizer details remain configurable because the paper does not provide them.

The core update loop is:

for batch_x, batch_y in train_loader:
    batch_x = batch_x.to(selected_device)
    batch_y = batch_y.to(selected_device)
    optimizer.zero_grad(set_to_none=True)
    predictions = model(batch_x)
    loss = criterion(predictions, batch_y)
    loss.backward()
    optimizer.step()

The sequence is standard gradient-based training: move the batch, clear gradients, compute predictions, calculate MAE, backpropagate, and update with RMSprop. The generated implementation also checks that prediction and target shapes match before the update, preventing accidental broadcasting between one-step and multi-step targets.

9.17 Validation loss, early stopping, and best-state restoration

A validation dataset is optional. When supplied, it is evaluated without gradient calculation:

model.eval()
with torch.no_grad():
    validation_prediction = model(validation_tensors[0])
    validation_loss = criterion(validation_prediction, validation_tensors[1])

The final test partition should not be passed as validation data. Validation can guide early stopping, while the untouched test period remains reserved for final evaluation.

The function saves a copy of the best model state:

if monitored_loss < best_loss - 1e-12:
    best_loss = monitored_loss
    history.best_epoch = epoch
    best_state = copy.deepcopy(model.state_dict())
    stale_epochs = 0
else:
    stale_epochs += 1
    if validation_tensors is not None and stale_epochs > patience:
        history.stopped_early = True
        break

After training, the best state is restored:

if best_state is not None:
    model.load_state_dict(best_state)

This prevents the returned model from being an arbitrary final epoch when an earlier epoch had lower monitored loss. TrainingHistory records training loss, validation loss, the best epoch, and whether early stopping occurred.

When no validation set is supplied, training loss is monitored and validation-based early stopping is not used. This is a weaker selection protocol, so a research experiment should normally provide a separate validation period or use a documented walk-forward retraining procedure.

9.18 Shape tests and the verification boundary

The model-shape tests are intended to verify tensor contracts rather than financial performance. An API-accurate attention test is:

sequence = torch.randn(3, 7, 8)
context, weights = attention(sequence, return_weights=True)

assert tuple(context.shape) == (3, 8)
assert tuple(weights.shape) == (3, 7)
assert torch.all(weights >= 0)
assert torch.allclose(weights.sum(dim=1), torch.ones(3), atol=1e-5)

The equivalent context-only call is:

context = attention(sequence)
weights = attention.last_attention_weights

Recurrent tests use the paper-inspired time length and check one-step and three-step output contracts:

inputs = torch.randn(2, 100, 3)
predictions = model(inputs)

assert predictions.shape[0] == 2
assert predictions.shape[-1] == horizon

These checks can detect wrong attention axes, shape regressions, missing dropout, and horizon mismatches. They cannot establish that the model learned useful financial structure, matched the paper's metrics, or produced a profitable strategy.

The available verification record must also be interpreted cautiously. Semantic code verification was skipped. Static checks reported findings involving placeholder detection in experiments.py and cli.py, and Markdown-fence checks involving the generated README.md and docs/tutorial.md code fields. In addition, some generated tests and orchestration calls have API inconsistencies with the generated modules. Consequently, this section documents intended interfaces and invariants; it does not claim a validated runtime contract, successful test execution, or empirical correctness.

9.19 Practical model-use sequence

A complete forecasting workflow should follow this order:

Clean and standardize local prices.

Establish a chronological training boundary.

Fit scaling parameters on training observations only.

Build 100-observation windows and future targets.

Split training and validation data without using final test targets.

Build AttBiLSTMForecaster with an explicit configuration.

Train with MAE and RMSprop.

Restore the best validation state through the training result.

Predict on untouched test windows.

Inverse-transform predictions if the target was scaled.

Compute forecast metrics separately from any portfolio backtest.

Pass only timestamp-aligned forecasts to the signal layer.

The model produces forecasts. It does not choose an order, determine a position size, apply VaR, or execute a trade. Those responsibilities belong to later modules.

9.20 Reconstruction boundary

The generated functions implement mechanics that can be stated precisely: recurrent sequence processing, dot-product attention, temporal softmax normalization, weighted context formation, dropout, MAE, RMSprop, and tensor-shape validation.

The following remain assumptions rather than verified paper details:

• hidden size and recurrent depth;

• learned-query construction;

• input feature set and target representation;

• learning rate, batch size, epoch count, and stopping policy;

• exact scaling procedure;

• one-day versus three-day operational target;

• hardware, random seed, and software-version details.

Keeping these boundaries visible is part of the implementation. It prevents a clean model class from being mistaken for an exact reproduction of an incompletely specified experiment.

10. Compare Recurrent, Statistical, HMM, and Boosted-Tree Baselines Carefully

The paper compares its attention-enhanced BiLSTM with several other forecasting families: LSTM, GRU, BiLSTM, Holt-Winters, ARIMA, HMM, and XGBoost. This comparison is useful because it asks whether the attention-enhanced recurrent model adds value beyond simpler neural, statistical, state-based, and tree-based approaches.

However, a benchmark table is only meaningful when the models receive comparable data and are evaluated under the same target definition and timestamps. The paper does not provide enough configuration detail to reproduce its table exactly. The generated project therefore implements benchmark interfaces and adapters, while labeling locally computed metrics separately from the paper's reported claims.

Reproduction boundary: The code below shows how the benchmark layer is intended to work. It does not establish that the generated code was executed, that every optional dependency is installed, or that the paper's numerical metrics were reproduced.

10.1 A common result contract

Different forecasting libraries return different model objects and prediction types. A benchmark experiment needs a common record so that results can be compared without losing important status information. The generated BenchmarkResult class in src/quantitative_trading_model/models/benchmarks.py provides that record:

@dataclass
class BenchmarkResult:
    model_name: str
    predictions: np.ndarray | None = None
    metrics: dict[str, float] = field(default_factory=dict)
    status: str = "ok"
    message: str = ""
    metadata: dict[str, Any] = field(default_factory=dict)

    def to_dict(self, include_predictions: bool = False) -> dict[str, Any]:
        output = {
            "model_name": self.model_name,
            "metrics": dict(self.metrics),
            "status": self.status,
            "message": self.message,
            "metadata": dict(self.metadata),
        }
        if include_predictions:
            output["predictions"] = (
                None if self.predictions is None
                else self.predictions.tolist()
            )
        return output

The fields have distinct purposes:

• model_name identifies the adapter or experiment label.

• predictions contains the aligned test forecasts when a model fitted successfully.

• metrics contains values such as RMSE, MAE, MAPE, and R-squared.

• status distinguishes a successful run from an unavailable optional package or a model failure.

• message preserves an actionable explanation instead of silently substituting another model.

• metadata records choices such as the backend, horizon, feature construction, or HMM interpretation.

This status handling matters in a research environment. If statsmodels is not installed, the result should say that Holt-Winters or ARIMA is unavailable. It should not quietly replace the missing model with a naive forecast and then present the resulting number as an apples-to-apples comparison.

The to_dict method also provides a local serialization boundary. Predictions can be omitted from a summary table while retaining metrics, configuration metadata, and failure explanations.

10.2 Holt-Winters: a statistical smoothing baseline

Holt-Winters, also called exponential smoothing, models a series through components such as level, trend, and seasonality. For a financial price series, the generated adapter defaults to an additive trend and no seasonal component:

class HoltWintersForecaster:
    name = "Holt-Winters"

    def __init__(
        self,
        trend: str | None = "add",
        seasonal: str | None = None,
        seasonal_periods: int | None = None,
        damped_trend: bool = False,
    ) -> None:
        self.trend = trend
        self.seasonal = seasonal
        self.seasonal_periods = seasonal_periods
        self.damped_trend = damped_trend
        self._fit_result: Any = None

    def fit(self, values: ArrayLike) -> "HoltWintersForecaster":
        series = _as_float_array(values, "training values")
        ...
        model = ExponentialSmoothing(
            series,
            trend=self.trend,
            damped_trend=(
                self.damped_trend if self.trend is not None else False
            ),
            seasonal=self.seasonal,
            seasonal_periods=self.seasonal_periods,
            initialization_method="estimated",
        )
        self._fit_result = model.fit(optimized=True, use_brute=True)
        return self

    def predict(self, horizon: int) -> np.ndarray:
        if self._fit_result is None:
            raise RuntimeError("fit must be called before predict")
        return np.asarray(self._fit_result.forecast(horizon), dtype=float)

The fit method validates the input, imports statsmodels only when needed, constructs the smoothing model, and stores the fitted result. The predict method requires a prior fit and asks the backend for a specified number of future observations.

Several choices here are not recoverable from the paper:

• whether the series has a meaningful seasonal period;

• whether trend should be additive, multiplicative, damped, or absent;

• how initialization should be performed;

• whether hyperparameters were tuned on a validation set;

• whether the model forecasted raw prices, returns, or transformed values.

Consequently, a locally fitted Holt-Winters model is a documented benchmark configuration, not proof that it matches the paper's implementation.

10.3 ARIMA and auto-ARIMA

ARIMA models temporal dependence through autoregressive terms, differencing, and moving-average terms. The generated ARIMAForecaster supports either an explicit order or an optional pmdarima.auto_arima path:

class ARIMAForecaster:
    name = "ARIMA"

    def __init__(
        self,
        order: tuple[int, int, int] = (1, 1, 1),
        use_auto: bool = False,
        seasonal: bool = False,
        suppress_warnings: bool = True,
    ) -> None:
        self.order = tuple(int(part) for part in order)
        self.use_auto = use_auto
        self.seasonal = seasonal
        self.suppress_warnings = suppress_warnings
        self._fit_result: Any = None
        self._backend = ""

When use_auto=False, the adapter uses the configured (p, d, q) order with statsmodels. When use_auto=True, it requires pmdarima and delegates model-order selection to auto_arima. These are materially different experiments: a fixed ARIMA order and an automatically selected order should not be reported under one undifferentiated label.

The paper mentions auto_arima, but it does not state the search bounds, seasonal settings, differencing policy, information criterion, or whether the test period influenced model selection. Those omissions affect both accuracy and fairness. A benchmark must select its order using training data or a separate validation procedure, never by inspecting final test targets.

10.4 HMM: state decoding is not automatically value forecasting

A hidden Markov model represents observations as being generated by an unobserved state sequence. The Viterbi algorithm finds the most likely sequence of hidden states given the observations. That is a state-decoding task; it is not automatically a forecast of the next numerical price.

This distinction corrects an ambiguity in the paper. Saying that an HMM uses Viterbi does not, by itself, specify how the next gold or Bitcoin price is calculated. A value forecast needs an additional observation model or transition-based prediction rule.

The generated adapter makes its reconstruction explicit. It fits a Gaussian HMM to one-step price changes, estimates the mean change associated with each latent state, and uses transition probabilities to estimate a future change. Its metadata records that interpretation:

class HMMForecaster:
    name = "HMM"

    def fit(self, values: ArrayLike) -> "HMMForecaster":
        series = _as_float_array(values, "training values")
        changes = np.diff(series).reshape(-1, 1)
        model = GaussianHMM(
            n_components=self.n_states,
            covariance_type="diag",
            n_iter=self.n_iter,
            random_state=self.random_state,
        )
        model.fit(changes)
        states = model.predict(changes)
        ...
        self._model = model
        self._state_means = means
        self._last_value = float(series[-1])
        return self

The important design choice is not the exact HMM implementation; it is the separation between:

fitting latent states;

decoding or identifying a current state;

converting state transitions into an expected future change;

adding that change to a price level.

The paper does not specify the number of states, emissions, covariance structure, observation variable, or prediction equation. Therefore, the HMM result must be labeled reconstructed, not presented as a verified implementation of the paper's HMM procedure.

10.5 XGBoost is gradient boosting, not a random forest

The paper's description of XGBoost as reducing computation through random forest is technically inaccurate. XGBoost is an implementation of gradient-boosted decision trees. A random forest builds many independently randomized trees and averages them; gradient boosting builds trees sequentially, with later trees attempting to correct earlier errors.

The generated XGBoostForecaster uses lagged or flattened window features:

class XGBoostForecaster:
    name = "XGBoost"

    @staticmethod
    def _features(values: Any) -> np.ndarray:
        array = np.asarray(values, dtype=float)
        if array.ndim == 1:
            array = array.reshape(-1, 1)
        elif array.ndim == 3:
            array = array.reshape(array.shape[0], -1)
        elif array.ndim != 2:
            raise ValueError(
                "XGBoost features must be one-, two-, or three-dimensional"
            )
        return array

A recurrent network consumes the temporal dimension as a sequence. A tree model generally needs a two-dimensional table, so a window such as (samples, 100, features) is flattened into (samples, 100 * features). That transformation is a modeling decision and should be recorded because it affects comparability.

The adapter also supports recursive prediction when no future feature rows are supplied. That behavior is only a practical reconstruction: the paper does not say whether XGBoost used lag features, technical indicators, rolling statistics, or another feature representation. A rigorous comparison should define the feature matrix before looking at test targets and should ensure that every lag uses information available at the corresponding timestamp.

10.6 Optional dependencies are part of the result

The benchmark module imports optional libraries inside the methods that need them. This keeps the core package usable when only NumPy and pandas are installed. The adapter catches missing dependencies and creates an explicit unavailable result:

try:
    prediction = np.asarray(
        _fit_and_predict(model, train_values, test_values, horizon),
        dtype=float,
    ).reshape(-1)
    metrics = _metric_dict(actual, prediction)
    results[model_name] = BenchmarkResult(
        model_name=model_name,
        predictions=prediction,
        metrics=metrics,
        metadata=model_metadata,
    )
except _OptionalDependencyError as exc:
    results[model_name] = BenchmarkResult(
        model_name=model_name,
        status="unavailable",
        message=str(exc),
        metadata=common_metadata,
    )
except Exception as exc:
    results[model_name] = BenchmarkResult(
        model_name=model_name,
        status="failed",
        message=f"{type(exc).__name__}: {exc}",
        metadata=common_metadata,
    )

This structure distinguishes at least three outcomes:

• `ok`: the adapter returned predictions and metrics;

• `unavailable`: an optional package was not installed;

• `failed`: the package was available, but fitting or prediction raised an error.

That distinction should remain visible in reports. A table containing only numeric columns can make a missing model look like a successful zero-result experiment.

The project's pyproject.toml separates deep-learning dependencies from classical benchmark dependencies. This supports lightweight use of data preparation, streak analysis, sizing, risk, and accounting without requiring PyTorch, statsmodels, pmdarima, hmmlearn, or XGBoost.

10.7 Shared metric calculation

The adapters pass predictions through the common metrics layer instead of implementing a different error formula for each library:

def _metric_dict(actual: ArrayLike, predicted: ArrayLike) -> dict[str, float]:
    result: ForecastMetrics = forecast_metrics(actual, predicted)
    return {
        "rmse": float(result.rmse),
        "mae": float(result.mae),
        "mape_percent": float(result.mape_percent),
        "r_squared": float(result.r_squared),
    }

The exact generated helper also preserves the metric object's serializable fields. The important contract is that every model is evaluated with the same definitions from src/quantitative_trading_model/metrics.py:

@dataclass(frozen=True)
class ForecastMetrics:
    rmse: float
    mae: float
    mape_percent: float
    r_squared: float
    sample_count: int
    mape_sample_count: int
    mape_epsilon: float = 1e-12
    mape_zero_policy: str = "exclude"

The metrics layer handles several edge cases explicitly:

• RMSE and MAE require equal-length, finite target and prediction arrays.

• MAPE excludes targets at or below a configured near-zero threshold by default.

• A caller can instead request an error when MAPE encounters a near-zero target.

• R-squared has defined behavior for a constant target series: exact predictions receive 1.0, while non-exact predictions receive 0.0.

• Metrics are computed on the same target scale used for comparison, normally after inverse transformation from a training-fitted scaler.

MAPE units also need attention. The implementation reports percentage points, so 2.5 means 2.5%, not 0.025. The paper's reported MAPE values should not be compared blindly until the target scale and unit convention are confirmed.

10.8 What makes a benchmark comparison fair?

A fair comparison requires more than calling eight fit methods. At minimum, document and align the following:

The generated run_benchmark_suite function attempts to provide one uniform workflow: extract training and test targets, fit each requested model on training data, predict the test interval, verify prediction length, calculate common metrics, and store implementation metadata. compare_forecasts then creates a stable pandas table and retains failed or unavailable models with status fields.

The common interface improves auditability, but it does not erase model-specific differences. An ARIMA model trained on a continuous series, an XGBoost model trained on flattened lag windows, and an HMM trained on price changes do not necessarily solve exactly the same statistical problem. Their input construction and forecast semantics must be described alongside their metric values.

10.9 Interpreting the paper's benchmark table

The paper reports Att-BiLSTM as the best listed predictor for both gold and Bitcoin. The extracted benchmark claims include values such as:

These numbers are preserved in the project documentation and reproduction matrix as unverified claims. They are not hard-coded acceptance targets for the benchmark adapters. The source data, preprocessing, architecture, hyperparameters, target scale, and test protocol are unavailable, so a different local experiment can legitimately produce different values without indicating a coding error.

The reproduction matrix makes this distinction explicit:

| Reported Att-BiLSTM gold metrics | Reported claim / Unavailable |
|---|---|
| RMSE 19.01, MAE 13.98, MAPE 0.7626, R-squared 0.9392 | Source data, preprocessing, target definition, and test protocol are missing. |

The same rule applies to the paper's LSTM, GRU, Holt-Winters, ARIMA, HMM, and XGBoost figures. A local comparison table can show what the supplied implementation produced under a documented protocol, but it cannot be called a reproduction of the paper's table until the missing inputs and decisions are recovered.

10.10 Forecast benchmarks are not trading benchmarks

Even a successful forecasting comparison is only one layer of the larger pipeline. The model with the lowest RMSE might not produce the best portfolio because:

• small forecast errors may not overcome transaction fees;

• the best numerical forecast may have poor directional timing;

• a position-sizing rule can amplify a modest error;

• different forecast horizons can produce different trading turnover;

• execution timing and slippage can reverse the order of model rankings;

• a high R-squared on a trending price level does not guarantee useful return forecasts.

For this reason, benchmark reporting should have two separate outputs:

Forecast comparison: target-level metrics using aligned test observations.

Trading comparison: portfolios generated from each model using the same signal, sizing, risk, fee, execution, and benchmark protocol.

The paper's reported Att-BiLSTM superiority addresses the first category as a claim. It does not, by itself, validate the second category.

10.11 Practical benchmark checklist

Before comparing local benchmark results, verify:

• Every model is trained only on its permitted training data.

• The final test targets are not used for model selection or hyperparameter tuning.

• All models forecast the same target representation and horizon, where comparison is intended.

• Forecast timestamps are aligned before calculating metrics.

• Price scaling is fitted on training data only and predictions are inverse-transformed consistently.

• MAPE near-zero behavior is documented.

• HMM state decoding is not mislabeled as a price forecast.

• XGBoost is described as gradient boosting, not random forest.

• Optional dependency failures remain visible in the results.

• Model-specific features and hyperparameters are recorded.

• Forecast metrics are reported separately from portfolio outcomes.

• Paper numbers are labeled as reported claims unless independently reproduced.

The generated benchmark adapters provide a useful scaffold for this process. Their main contribution is not a promise of identical numbers; it is a uniform, inspectable place to document how each alternative model was trained, what it predicted, whether it was available, and how its metrics were calculated.

11. Measure Returns and Consecutive Rise/Decline Streaks

The paper uses consecutive price increases and decreases to inform its position-management model. In practical terms, the analysis asks how often positive or negative price movements persist and how large representative gains and declines are.

The paper calls this procedure Apriori, but the extracted description does not define itemsets, support, confidence, or association rules. Standard Apriori is a frequent-itemset-mining algorithm, whereas the described procedure counts consecutive movements. The implementation therefore treats this component as streak analysis or sequential-pattern analysis. This is a terminology and implementation correction, not a claim that the paper's original authors intended a different method.

11.1 Define the return convention first

For adjacent prices, the standard simple return is:

\[ rt = \frac{pt-p{t-1}}{p{t-1}}. \]

The return is aligned with the later timestamp: it describes the move from the previous observation into p_t. For [100, 110, 99], the returns are:

100 -> 110: (110 - 100) / 100  =  0.10
110 ->  99: ( 99 - 110) / 110  = -0.10

The extracted paper equation appears to use the current price as the denominator:

\[ \tilde rt = \frac{pt-p{t-1}}{pt}. \]

Because the equation layout is ambiguous, compute_returns uses the conventional previous-price denominator by default and exposes the current-price form through an explicit option. The choice must be recorded and used consistently for classification, percentile estimation, sizing calibration, and any backtest signal.

The relevant implementation is in src/quantitative_trading_model/streaks.py:

def compute_returns(
    prices: Iterable[float],
    denominator: ReturnDenominator = "previous",
) -> np.ndarray:
    values = _as_float_array(prices, name="prices")
    if np.any(values <= 0.0):
        raise ValueError("prices must be strictly positive")

    if values.size < 2:
        return np.empty(0, dtype=float)

    normalized = str(denominator).lower()
    if normalized in {"previous", "previous_price"}:
        base = values[:-1]
    elif normalized in {"current", "current_price", "paper"}:
        base = values[1:]
    else:
        raise ValueError(
            "denominator must be 'previous', 'current', or 'paper'"
        )

    return (values[1:] - values[:-1]) / base

The function checks that prices are one-dimensional, finite, and strictly positive. A sequence with fewer than two prices has no adjacent return. Rejecting nonpositive prices is important because the return denominator would otherwise be invalid.

11.2 Classify rises, declines, and zero changes

Returns are classified numerically:

classify_returns returns integer labels, not strings such as "positive" or "neutral". Keeping the labels numeric makes run detection straightforward and avoids ambiguity in comparisons.

def classify_returns(
    returns: Iterable[float],
    *,
    zero_policy: ZeroPolicy = "neutral",
    tolerance: float = 0.0,
) -> np.ndarray:
    values = _as_float_array(returns, name="returns")
    labels = np.zeros(values.size, dtype=np.int8)
    labels[values > tolerance] = 1
    labels[values < -tolerance] = -1

    if zero_policy == "raise" and np.any(labels == 0):
        raise ValueError("zero or near-zero returns are not allowed")
    if zero_policy == "ignore":
        labels = labels[labels != 0]
    return labels

The zero policies are:

• neutral: retain zero labels, so they break maximal runs;

• terminate: treat zero changes as streak boundaries;

• ignore: remove zero labels before counting, making surrounding nonzero observations adjacent;

• raise: reject any zero or near-zero return.

The default is neutral. A nonzero tolerance can classify very small returns as zero; that setting should be included in experiment metadata because it changes the streak counts.

For example:

returns:  +   +   -   -   -   +   0   +
labels:    1   1  -1  -1  -1   1   0   1

Under the default policy, the zero separates the final positive movements. The positive runs have lengths two, one, and one, while the negative run has length three.

11.3 Maximal runs and overlapping subsequences

“Two consecutive rises” can refer to different statistics. The implementation supports both interpretations.

A maximal run is a complete uninterrupted sequence of one sign. For:

1, 1, -1, -1, -1, 1

the maximal runs are:

rise of length 2
decline of length 3
rise of length 1

The corresponding summaries are:

rise_streaks = {1: 1, 2: 1}
decline_streaks = {3: 1}

This answers: “How many complete runs of each exact length occurred?”

An overlapping subsequence count includes shorter patterns inside longer runs. A run of four rises contains three two-rise subsequences:

[1, 1] at positions 1-2
[1, 1] at positions 2-3
[1, 1] at positions 3-4

It also contains two three-rise subsequences and one four-rise subsequence. This answers: “How many contiguous patterns of each length occurred?”

The public function makes the distinction explicit:

def count_streaks(
    labels: Iterable[int],
    *,
    value: int,
    mode: StreakMode = "maximal",
    minimum_length: int = 1,
) -> dict[int, int]:
    result: dict[int, int] = {}
    run_length = 0

    for label in labels:
        if int(label) == value:
            run_length += 1
            continue

        if run_length:
            _add_run(
                result,
                run_length,
                mode=mode,
                minimum_length=minimum_length,
            )
            run_length = 0

    if run_length:
        _add_run(
            result,
            run_length,
            mode=mode,
            minimum_length=minimum_length,
        )
    return dict(sorted(result.items()))

The required value argument specifies whether to count rises (value=1) or declines (value=-1). In maximal mode, a run contributes one count for its exact length. In overlapping mode, a run of length r contributes r-j+1 occurrences for each subsequence length j.

The paper does not specify which counting convention it uses. Both are therefore implementation interpretations, not verified transcriptions. A research result should record the selected mode and minimum streak length.

11.4 Gain and decline percentiles

The paper connects streak analysis with percentile statistics used by its position-management method. The implementation keeps these quantities separate:

• the 90th percentile of positive returns;

• the 10th percentile of signed negative returns;

• the median positive return;

• the 10th percentile of positive decline magnitudes;

• the median decline magnitude.

For a negative return of -0.04, the signed decline is -0.04, while its magnitude is 0.04. Both forms are useful: the signed value preserves the return convention, and the magnitude is convenient for a sizing calculation that expects a nonnegative loss amount.

def estimate_gain_decline_percentiles(
    returns: Iterable[float],
    *,
    gain_percentile: float = 90.0,
    decline_percentile: float = 10.0,
) -> dict[str, float]:
    values = _as_float_array(returns, name="returns")
    gains = values[values > 0.0]
    declines = values[values < 0.0]
    decline_magnitudes = -declines

    return {
        "gain_percentile": _percentile(gains, gain_percentile),
        "decline_percentile": _percentile(
            declines, decline_percentile
        ),
        "gain_median": _percentile(gains, 50.0),
        "decline_magnitude_percentile": _percentile(
            decline_magnitudes, decline_percentile
        ),
        "decline_magnitude_median": _percentile(
            decline_magnitudes, 50.0
        ),
    }

For the return history:

[0.01, 0.02, 0.04, -0.01, -0.03, -0.06]

the gain sample is [0.01, 0.02, 0.04], and the signed decline sample is [-0.01, -0.03, -0.06]. The magnitude sample is [0.01, 0.03, 0.06]. Empty gain or decline samples produce NaN, rather than an invented zero.

These statistics are descriptive inputs to the later normalized exponential sizing reconstruction. They are not forecasts and do not imply that the next movement will match a historical percentile.

11.5 The combined summary record

compute_streak_statistics combines return calculation, classification, run counting, and percentile estimation. It returns a frozen StreakStatistics record containing the arrays, mappings, scalar statistics, denominator, zero policy, and streak mode.

“Frozen” means that the dataclass fields cannot be reassigned through normal dataclass operations. It does not make the stored NumPy arrays deeply immutable: a caller can still mutate an array in place unless it copies the arrays or marks them read-only. The record should therefore be treated as an audit-oriented summary object, not as a complete deep-immutability guarantee.

The returned record includes fields such as:

statistics = compute_streak_statistics(
    prices,
    denominator="previous",
    zero_policy="neutral",
    streak_mode="maximal",
)

print(statistics.rise_streaks)
print(statistics.decline_streaks)
print(statistics.gain_percentile_90)
print(statistics.decline_magnitude_percentile_10)

Its to_dict method is intended for local result files and preserves the selected conventions and count summaries.

11.6 Use only pre-decision information online

A full-history summary is useful for exploration but is not automatically valid for an out-of-sample strategy. If a signal is generated at time t, the statistics used to choose its position size must be estimated from returns known no later than t.

This is unsafe:

compute returns over the complete dataset
compute percentiles over the complete dataset
use those percentiles to size trades in the earlier test period

Future observations have influenced the historical calibration. The safer walk-forward pattern is:

for each decision timestamp t:
    use returns observed before t
    calculate or update streak statistics
    choose the next sizing level
    generate the order
    execute at a later permitted bar

The same rule applies when passing calibration statistics to build_exponential_schedule in sizing.py. A statistic calculated once from the entire dataset should not be used as though it had been available at every earlier decision time.

11.7 What the generated tests currently represent

tests/test_data_and_streaks.py is intended to document small, hand-calculable invariants such as the previous-price return convention and the difference between maximal and overlapping counts. However, the generated test file currently contains known API mismatches with the generated streaks.py implementation:

classify_returns returns integer labels 1, -1, and 0, while some test assertions expect textual labels such as "positive", "negative", and "neutral".

count_streaks requires the keyword-only value argument, but the generated streak-count test calls it without value=1 or value=-1.

These are planned/static test artifacts with known mismatches, not evidence of executable verification. They should be repaired before being treated as runnable tests: assertions should use numeric labels, and rise and decline counts should call count_streaks(labels, value=1, ...) and count_streaks(labels, value=-1, ...) separately.

Semantic code verification was skipped, and this tutorial does not claim that the tests were executed, that they pass, or that the paper's empirical results were reproduced. The useful review targets remain the return convention, zero policy, run-counting mode, percentile population, and pre-decision information boundary.

11.8 Summary

The paper's consecutive-movement component can be translated into an auditable pipeline:

Compute adjacent returns with an explicitly recorded denominator.

Classify each return as a rise, decline, or zero.

Decide how zero returns affect streak boundaries.

Count maximal runs or overlapping subsequences.

Estimate gain, signed-decline, and decline-magnitude percentiles.

Pass statistics to the later sizing reconstruction only when they could have been known at the decision timestamp.

The implementation in streaks.py provides these mechanics, but it does not claim to recover a standard Apriori algorithm or the paper's exact counting procedure. The appropriate description is sequential streak analysis feeding a documented, finite position-sizing reconstruction.

12. Reconstruct the Exponential Position-Sizing Rule Safely

The paper's position-management component addresses a different question from forecasting: after a strategy decides to trade, how much capital should it commit? The paper appears to increase later additions after successive declines, using empirical gains, declines, transaction costs, and an exponential sizing curve.

This section is not an exact reproduction of that method. The extracted forms of equations (5) through (8) contain damaged notation and do not define several important variables. The implementation in src/quantitative_trading_model/sizing.py therefore provides a finite, normalized-exponential alternative with explicit budget and exposure controls.

Reconstruction boundary: The schedule below is an auditable interpretation of the paper's sizing idea. It is not a faithful transcription of equations (5)–(8), and it does not establish that averaging down is safe or profitable.

12.1 Why the paper's recursive equations cannot be transcribed faithfully

The paper appears to derive later capital additions from a recovery or break-even condition: after one or more declines, a future gain should compensate for earlier losses and transaction costs. It then represents the resulting additions with an exponential curve.

The extracted equations are insufficient to reproduce that calculation reliably:

• p_i is not clearly defined as cash, asset quantity, or notional exposure.

• Signs and powers in the recursive expressions are corrupted.

• The fee convention is incomplete.

• The independent variable and exponent coefficient in the printed exponential expression are unclear.

• The maximum number of additions is not reliably specified.

• It is unclear whether an addition follows every observed decline, a forecast-confirmed decline, or a separate decision policy.

Those choices affect both risk and accounting. Filling them in silently would make the implementation appear more faithful than it is. The replacement below makes the missing decisions explicit.

12.2 Normalized exponential reconstruction

The practical reconstruction assigns a positive score to each finite addition level:

si=Aexp⁡(Bi),i=0,1,…,n−1.

Here A is a scale factor, B is the growth rate, and n is the configured maximum number of additions. The scores are converted to normalized weights:

\[ wi=\frac{si}{\sum{j=0}^{n-1}sj}. \]

For a sizing budget U, the planned allocation is:

\[ ai=Uwi. \]

Thus, subject to floating-point rounding,

\[ \sumi ai=U. \]

This is a plausible interpretation of the paper's normalized allocation expression, but it is not a verified transcription. In the generated implementation, build_exponential_schedule receives the number of additions and budget explicitly:

def build_exponential_schedule(
    max_additions: int,
    budget: float,
    *,
    growth: float = 0.5,
    amplitude: float = 1.0,
    gain_percentile: Optional[float] = None,
    decline_percentile: Optional[float] = None,
    fee_rate: float = 0.0,
    calibration_source: Any = None,
) -> SizingSchedule:
    """Build normalized ``A * exp(B*i)`` addition amounts."""

The absolute budget is therefore supplied to the sizing builder or sizer; it is not hidden inside the paper-inspired SizingConfig dataclass.

12.3 Constructing the schedule safely

The implementation computes the exponential scores and subtracts the largest exponent before calling exp:

indices = np.arange(max_additions, dtype=float)
exponents = growth * indices
exponents -= float(np.max(exponents))
scores_array = float(amplitude) * np.exp(exponents)

score_sum = float(scores_array.sum())
weights_array = scores_array / score_sum
allocations_array = weights_array * budget

Subtracting the maximum exponent does not change the normalized weights. It multiplies every score by the same positive constant, so each ratio s_i / sum(s_j) remains unchanged while reducing overflow risk.

SizingSchedule stores:

• scores: unnormalized exponential scores;

• weights: normalized proportions that sum to one; and

• allocations: planned currency amounts obtained from the budget.

It also records metadata identifying the formula as a reconstruction. This makes the sizing assumptions available when a later experiment is inspected.

12.4 Worked allocation example

Suppose the total planned budget is $1,000 and three levels have scores:

\[ s0=1, \qquad s1=2, \qquad s_2=4. \]

The score sum is 7:

These are planned amounts, not automatic orders. If only $400 remains available when the final level is requested, the order must be reduced or rejected according to the current cash and exposure constraints.

The same relative scores arise from A=1 and B=log(2), because exp(Bi) then gives approximately 1, 2, and 4 for levels zero through two.

12.5 Optional percentile and fee calibration

The paper connects position sizing to empirical gain and decline statistics. The implementation can read values such as the 90th-percentile gain, a signed decline percentile, a decline-magnitude percentile, a median gain, and a fee rate.

When percentile values or a calibration_source are supplied to build_exponential_schedule, the helper _calibrated_growth derives a conservative growth coefficient:

def _calibrated_growth(
    gain_percentile: Optional[float],
    decline_percentile: Optional[float],
    fee_rate: float,
) -> tuple[float, dict[str, float]]:
    gain = max(float(gain_percentile or 0.0), 0.0)
    decline = abs(float(decline_percentile or 0.0))
    fee = _finite_nonnegative(fee_rate, "fee_rate")

    recovery = gain + _EPSILON
    burden = decline + 2.0 * fee
    growth = 0.05 + min(1.5, burden / recovery)
    return growth, {
        "gain_percentile": gain,
        "decline_magnitude": decline,
        "fee_rate": fee,
        "growth_coefficient": growth,
    }

This is an engineering interpretation, not the paper's recovered break-even equation. The fee appears twice in the burden term as a conservative allowance for a purchase and later sale; actual fees are still charged by the execution simulator.

SizingConfig contains fields such as calibrate_from_percentiles, recovery_percentile, decline_percentile, and fee_buffer. These fields document intended experiment settings, but the current generated schedule builder does not accept a SizingConfig directly and does not automatically consult calibrate_from_percentiles. Calibration is activated in the current implementation by supplying percentile arguments or a calibration_source to build_exponential_schedule. An experiment layer must explicitly connect configuration fields to those arguments if automatic configuration-driven calibration is desired.

12.6 Finite levels and budget invariants

The schedule validates finite values, nonnegative scores and allocations, normalized weights, and the relationship between planned allocations and its supplied budget. It also tracks consumed levels:

used_levels: set[int] = field(default_factory=set, repr=False)

next_unused_level() returns the first unconsumed level, and mark_used(level) records a level while rejecting reuse. This prevents a strategy loop from accidentally placing the same planned addition repeatedly.

The schedule's budget is an absolute amount supplied when the schedule is built. It is distinct from the fraction fields in SizingConfig. The configuration validates settings such as:

• a positive maximum_additions value;

• an initial_level inside the configured level range;

• a positive score_scale;

• allocation fractions between zero and one; and

• a maximum exposure fraction in (0, 1].

These checks validate fraction and level ranges. They do not compare an exposure fraction with an absolute budget, because SizingConfig does not contain an absolute budget field. Absolute budget and exposure constraints are enforced later by build_exponential_schedule and allocate_next_addition.

12.7 Cash and exposure caps at allocation time

The schedule is only a plan. At order time, allocate_next_addition applies the current portfolio state:

def allocate_next_addition(
    schedule: SizingSchedule,
    available_cash: float,
    *,
    current_exposure: float = 0.0,
    max_exposure: Optional[float] = None,
    level: Optional[int] = None,
    allow_partial: bool = True,
) -> AllocationDecision:

The effective capacity is calculated as:

cap = schedule.budget if max_exposure is None else max_exposure
capacity = min(cash, max(0.0, cap - exposure))
approved = min(requested, capacity)

This provides three separate protections:

A request cannot exceed current available cash.

Available cash does not override a maximum exposure limit.

Partial approval is explicit. With allow_partial=True, an amount may be reduced to capacity; with allow_partial=False, an undersized request is rejected.

The result is an AllocationDecision containing the requested and approved amounts, selected level, reason, remaining cash, and remaining exposure capacity.

The approved amount is a pre-fee notional. The sizing module does not deduct execution fees from that amount. The backtest applies fees separately, so the final cash cost of a buy can be higher than the approved notional. Consequently, the execution layer may clip the order again when fees are applied, even when the sizing layer approved it under a pre-fee cash calculation. Keeping these stages separate makes the fee convention auditable, but callers must not treat a pre-fee approval as a guarantee that the full order will fill.

12.8 Why this is not unlimited averaging down

Increasing allocations after declines can become an uncontrolled averaging-down strategy. The generated design limits that behavior by using:

• a finite maximum_additions value;

• one-time consumption of each addition level;

• a fixed schedule budget;

• current-cash checks;

• optional maximum exposure;

• no leverage by default in the execution layer; and

• optional risk filtering by later strategy components.

These are safety and reproducibility choices, not recovered paper details. They may differ from the unknown original behavior, but they prevent the implementation from silently authorizing unlimited purchases.

The sizing module also does not decide when to add. It supplies an amount after another layer requests a level. A signal, forecast, decline condition, VaR decision, or greedy policy must determine whether that level is activated.

12.9 Correct API example

The generated sizing implementation uses growth, not growth_rate, when constructing a schedule:

schedule = build_exponential_schedule(
    budget=1_000.0,
    max_additions=4,
    growth=0.5,
)

assert np.isclose(sum(schedule.weights), 1.0)
assert np.isclose(sum(schedule.allocations), 1_000.0)

The supplied generated test file uses growth_rate in one example and constructs ExponentialPositionSizer with a configuration object in a way that does not match the shown implementation. It also contains related naming differences in the risk and strategy tests. These are static test artifacts describing intended invariants, not evidence that the tests were executed successfully. The corrected example above follows the actual generated build_exponential_schedule signature.

12.10 What the sizing tests are intended to verify

The tests are intended to check properties of the reconstruction rather than the paper's unverified returns:

• scores, weights, and allocations are finite;

• weights are nonnegative and sum to one;

• planned allocations do not exceed the supplied budget;

• a request cannot exceed available cash or exposure capacity;

• an addition level cannot be reused; and

• future realized prices do not influence the sizing decision.

These are static and semantic expectations. They do not prove that equations (5)–(8) were recovered or that the resulting strategy is economically sound.

12.11 Practical interpretation

The reconstructed data flow is:

historical gain/decline statistics
        |
        v
optional explicit calibration arguments
        |
        v
finite exponential scores
        |
        v
normalized budget allocations
        |
        v
cash and exposure checks
        |
        v
pre-fee approved notional
        |
        v
fee-aware execution and final fill

A larger allocation after a decline increases potential exposure if prices recover, but it also increases losses if the decline continues. Fees, slippage, liquidity, and forecast error can invalidate the recovery assumptions. Evaluation should therefore include drawdown, turnover, total fees, and risk measures rather than final value alone.

The narrow claim supported by this implementation is that it provides a finite, normalized, auditable reconstruction of the paper's exponential position-sizing concept. It does not reproduce equations (5)–(8), validate the paper's reported $646 gold or $215,487 Bitcoin outcomes, or establish profitability.

13. Turn Forecasts into Signals and Reconstruct the Greedy Policy

A forecast is not yet an order. If a model predicts Bitcoin will reach $31,000, the strategy must still compare that prediction with the current price, decide whether the expected move is large enough to trade, choose a notional amount, check cash and exposure limits, apply risk controls, and select an action.

The paper describes this stage using forecasting, position management, risk control, and a modified greedy algorithm. It does not define the greedy algorithm operationally: the candidate actions, objective, constraints, and tie-breaking rules are missing. The implementation therefore provides a transparent reconstructed policy, not a verified reproduction.

The intended decision flow is:

forecast values
    -> aggregate the forecast horizon
    -> compare the forecast with the current price
    -> create a buy, sell, add, or hold signal
    -> construct and constrain candidate actions
    -> estimate fees and expected benefit
    -> apply a risk filter
    -> rank feasible candidates deterministically
    -> execute the selected action on a later bar

The snippets in this section are abridged explanatory excerpts from src/quantitative_trading_model/strategy.py; they are not complete replacement functions. The generated strategy and risk modules currently have an incompatible runtime interface, so this section describes the intended contract and the required repair rather than claiming that the end-to-end policy runs successfully.

13.1 Direction and order size are separate decisions

The strategy module defines four directional states:

class Signal(str, Enum):
    BUY = "buy"
    SELL = "sell"
    HOLD = "hold"
    ADD = "add"

These values describe intent, not quantity:

• BUY establishes or increases exposure when the forecast exceeds a threshold.

• SELL reduces existing long exposure when the forecast is sufficiently negative.

• ADD requests the next finite position-sizing level.

• HOLD submits no trade.

Separating direction from size lets the forecasting model answer “should an opportunity be considered?” while the sizing module answers “how much may be allocated?” Cash limits, exposure caps, transaction fees, and risk controls can then reduce or reject the proposed amount.

The paper does not explain how an ADD signal is triggered. In particular, it does not state whether additions follow every decline, a forecast-confirmed decline, or a separate greedy rule. The generated strategy.py does not derive ADD directly from the streak analysis; it creates an addition candidate only when the supplied decision is BUY or ADD and a finite addition level is available. A production revision must define that trigger explicitly.

13.2 Aggregate one-day or multi-day forecasts explicitly

The paper is inconsistent about its forecast horizon. Its prediction discussion supports one-step forecasting, while its conclusion refers to the next three days. SignalGenerator makes the interpretation configurable.

The generator accepts predicted prices and supports three aggregation choices:

• terminal: use the final forecast, such as day three;

• mean: use the average forecast over the horizon;

• max: use the largest forecast.

The following is an abridged excerpt; validation and error handling remain in the generated class:

class SignalGenerator:
    def __init__(self, buy_threshold=0.0, sell_threshold=0.0,
                 aggregation="terminal", minimum_forecast_points=1):
        self.buy_threshold = buy_threshold
        self.sell_threshold = sell_threshold
        self.aggregation = aggregation
        self.minimum_forecast_points = minimum_forecast_points

    def aggregate(self, forecasts):
        values = [float(value) for value in forecasts]
        if self.aggregation == "terminal":
            return values[-1]
        if self.aggregation == "mean":
            return sum(values) / len(values)
        return max(values)

For a current price of $100 and forecasts [101, 103, 104], terminal and maximum aggregation produce $104, while mean aggregation produces approximately $102.67. These are reconstruction choices, not rules confirmed by the paper.

13.3 Convert the selected forecast into a signal

After aggregation, the strategy computes the expected price return:

r^=p^−pp,

where p is the current price and p_hat is the selected forecast.

An abridged version of SignalGenerator.decide is:

def decide(self, current_price, forecasts):
    forecast_value = self.aggregate(forecasts)
    expected_return = forecast_value / current_price - 1.0

    if expected_return > self.buy_threshold:
        signal = Signal.BUY
    elif expected_return < -self.sell_threshold:
        signal = Signal.SELL
    else:
        signal = Signal.HOLD

    return SignalDecision(
        signal=signal,
        expected_return=expected_return,
        forecast_value=forecast_value,
        horizon=len(forecasts),
        rationale="threshold-based forecast decision",
    )

With one-percent buy and sell thresholds:

The thresholds are not specified by the paper and must be stored with experiment metadata. A zero threshold can produce frequent trades when forecasts fluctuate around the current price; a positive threshold creates a hold band but may delay decisions.

A signal must contain current-information fields only: direction, expected return, forecast value, horizon, and rationale. It must not contain a realized future price or realized future return. Execution belongs to the later backtest stage.

13.4 Candidate actions contain size and audit information

A CandidateAction combines a proposed notional with the information needed for ranking:

@dataclass(frozen=True)
class CandidateAction:
    signal: Signal
    notional: float              # pre-fee order value
    expected_return: float
    estimated_fee: float
    score: float
    feasible: bool = True
    reason: str = ""
    addition_level: Optional[int] = None
    risk_approved: bool = True
    risk_scale: float = 1.0

For buys and additions, notional is the cash committed before fees. For sells, it is the market value sold before the sell fee. reason and addition_level make decisions easier to inspect.

PortfolioSnapshot supplies the current state:

@dataclass(frozen=True)
class PortfolioSnapshot:
    cash: float
    holdings: float
    price: float
    addition_level: int = 0
    portfolio_value: Optional[float] = None
    max_exposure: Optional[float] = None

    @property
    def marked_value(self):
        return self.cash + self.holdings * self.price

This state is intentionally limited to information available at the signal timestamp.

13.5 Apply feasibility constraints before ranking

The policy should never rank an unaffordable order as though it were executable. For a long-only account, the main constraints are:

A buy must fit available cash, including fees.

A buy must not exceed the maximum exposure.

A sell must not exceed current holdings unless shorting is explicitly enabled.

An addition must use an available finite sizing level.

A risk filter may subsequently block or scale the candidate.

For a fee rate f, the maximum pre-fee buy notional from cash is:

Nmax=cash1+f.

Thus $75 of cash and a one-percent fee permit at most approximately $74.26 of pre-fee buy notional.

The generated candidate builder caps buy amounts by cash and exposure. It does not consistently retain a separate rejected candidate for every failed cash or exposure check. Some rejected requests are reduced to an affordable candidate, while exhausted addition levels may be recorded as infeasible. Therefore, the accurate audit statement is that feasible capped candidates and selected/rejected addition cases are recorded, but the current implementation does not provide a uniform record of every failed constraint attempt. A stricter audit design would preserve both the original request and the resulting rejection reason.

A conceptual candidate calculation is:

available_buy = snapshot.cash / (1.0 + fee_rate)
current_exposure = snapshot.holdings * snapshot.price

if snapshot.max_exposure is not None:
    available_buy = min(
        available_buy,
        max(0.0, snapshot.max_exposure - current_exposure),
    )

The backtest must enforce the same limits again at execution. Strategy-level feasibility is not a substitute for accounting-level validation.

13.6 Connect finite sizing to ADD candidates

The normalized exponential sizer supplies finite addition amounts. The strategy layer should not duplicate the sizing formula. It requests the next level from a previously constructed sizer or schedule, then applies current cash and exposure limits.

For example, the sizing object must be defined before calling the policy:

from quantitative_trading_model.sizing import (
    ExponentialPositionSizer,
    build_exponential_schedule,
)

schedule = build_exponential_schedule(
    max_additions=3,
    budget=500.0,
    growth=0.5,
)
sizer = ExponentialPositionSizer(schedule=schedule)

The exact generated API may need alignment before runtime use, but the intended data flow is:

BUY or ADD decision
    -> request the next sizing level
    -> cap it by cash and exposure
    -> create an ADD candidate

The paper does not define the trigger for ADD, so the example must not imply that the strategy automatically reacts to every consecutive decline. That behavior would need an explicit rule connecting streaks.py, the forecast, and the policy.

13.7 Use a transparent fee-aware score

The paper gives no objective function for its modified greedy algorithm. The reconstruction uses expected net benefit:

score(a)=notional(a)(benefit rate(a)−f).

For a buy or add, the benefit rate is the forecasted return. For a sell, the reconstruction treats a negative forecast as a potential avoided loss, so the benefit rate is -expected_return. Hold has a score of zero.

An abridged scoring function is:

def score_candidate_action(action, *, expected_return, fee_rate):
    if action.signal in {Signal.BUY, Signal.ADD}:
        benefit_rate = expected_return
    elif action.signal is Signal.SELL:
        benefit_rate = -expected_return
    else:
        benefit_rate = 0.0
    return action.notional * (benefit_rate - fee_rate)

A $100 buy with a three-percent expected return and one-percent fee scores:

100(0.03−0.01)=2.

A $100 buy with a 0.5% expected return scores -0.50 and should lose to the zero-score hold action. This score ignores uncertainty, spread, liquidity, and forecast calibration. It is a transparent replacement for an objective that the paper does not provide.

13.8 Rank candidates deterministically

The intended ranking process is:

Remove candidates that are infeasible or not risk-approved.

Choose the highest score.

Break equal-score ties with the smaller notional.

Apply a fixed signal priority for any remaining tie.

An abridged selection expression is:

feasible = [
    candidate for candidate in candidates
    if candidate.feasible and candidate.risk_approved
]

selected = max(
    feasible,
    key=lambda candidate: (
        candidate.score,
        -candidate.notional,
        -priority[candidate.signal],
    ),
)

The priority mapping is an implementation choice, not a paper rule. If no candidate is feasible, the policy should return a hold decision with a clear reason such as "no feasible candidate".

13.9 Risk integration: intended contract versus current generated code

The intended ordering is:

candidate
    -> risk filter
    -> accept, scale, or block
    -> recompute fee and score
    -> rank the resulting candidate

Risk must be applied before final ranking because scaling a candidate can change its score. However, the current generated strategy.py and risk.py do not implement one reliable shared protocol:

• GreedyPolicy._apply_risk calls the filter using proposed_notional, while VaRTradeFilter.decide expects proposed_quantity and also requires current_exposure and portfolio_value.

• The fallback positional call does not reliably map those required values to the risk method's parameters.

• VaRTradeFilter returns RiskDecision with approved_quantity and blocked, not an approved field. The current strategy adapter looks first for approved or allowed, so a blocked result can be misinterpreted as approved.

• Consequently, the current code must be repaired before its VaR integration can be treated as a working runtime policy.

A strict repaired protocol should pass the same named fields every time:

risk_decision = risk_filter.decide(
    proposed_quantity=action.notional,
    current_exposure=snapshot.holdings * snapshot.price,
    portfolio_value=snapshot.value,
    returns=pre_decision_returns,
)

if risk_decision.blocked:
    action = replace(action, feasible=False, risk_approved=False, notional=0.0)
elif risk_decision.approved_quantity < action.notional:
    scale = risk_decision.approved_quantity / action.notional
    action = rescale_and_recompute_score(action, scale, fee_rate)

This is a repair specification, not a claim about the current generated files. The paper itself does not define how VaR interacts with greedy selection, so the ordering remains a reconstruction even after the interfaces are aligned.

13.10 Complete decision example with explicit assumptions

The following is an explanatory excerpt showing the objects that must exist. It is not a complete runnable example, and the risk interface must be repaired as described above:

from quantitative_trading_model.strategy import (
    GreedyPolicy, PortfolioSnapshot, SignalGenerator,
)

schedule = build_exponential_schedule(
    max_additions=3,
    budget=500.0,
    growth=0.5,
)
sizer = ExponentialPositionSizer(schedule=schedule)

policy = GreedyPolicy(fee_rate=0.01)
signal_generator = SignalGenerator(
    buy_threshold=0.01,
    sell_threshold=0.01,
    aggregation="terminal",
)

snapshot = PortfolioSnapshot(
    cash=500.0,
    holdings=0.0,
    price=100.0,
    addition_level=0,
    max_exposure=500.0,
)

# The intended call; risk_filter integration requires the strict protocol above.
action = policy.choose(
    snapshot,
    forecasts=[101.0, 103.0, 104.0],
    signal_generator=signal_generator,
    sizer=sizer,
    max_additions=3,
)

Conceptually, this selects the terminal forecast of $104, computes a four-percent expected return, creates a buy decision, obtains finite sizing candidates, applies cash and exposure limits, accounts for fees, and selects the best feasible action. The selected action must be passed to the backtest for execution on a later bar.

13.11 Verification status and no-look-ahead boundary

The intended tests in tests/test_sizing_risk_strategy.py cover useful invariants:

• scores use forecasts and fees rather than realized future returns;

• order sizes do not exceed cash or exposure limits;

• risk scaling never increases a proposed order;

• tie-breaking is deterministic;

• signal timestamps precede execution timestamps.

The supplied generated test file is currently API-inconsistent with strategy.py. It uses older names such as side instead of signal and calls methods with incompatible signatures. It therefore cannot substantiate that the current strategy implementation passes its intended tests without repair.

More broadly, semantic code verification was skipped. Static review identified issues in the generated project, and no code execution or passing test suite is being claimed. The excerpts above explain the intended contracts and invariants; they should not be read as evidence of runtime correctness.

13.12 Reproduction boundary

The paper's greedy policy remains unverified because it does not specify:

• the complete candidate-action set;

• whether one action or a multi-step plan is selected;

• the objective function;

• how one-day and three-day forecasts are combined;

• the treatment of fees, spreads, and slippage;

• the interaction between VaR and action selection;

• leverage, exposure, and addition constraints; or

• tie-breaking behavior.

The generated policy is therefore best understood as an auditable baseline. It demonstrates how forecasts, finite sizing, constraints, fees, risk decisions, and deterministic selection can be connected without using realized future prices. The realized price enters only during later execution and evaluation, preserving the causal boundary required for a meaningful backtest.

14. Add Historical VaR as an Explicit Risk Filter

The paper names Value-at-Risk (VaR) as part of its trading framework, but it does not define the parameters or the action taken when risk is high. It does not specify the confidence level, horizon, lookback window, return distribution, portfolio aggregation, acceptable loss, or whether VaR blocks or scales trades.

This project therefore uses a rolling historical VaR reconstruction. The convention in this section is explicit:

• return VaR is a nonnegative loss fraction, such as 0.04 for a 4% loss threshold;

• projected VaR is a currency amount;

• the risk limit is a currency amount equal to a configured fraction of portfolio value;

• trades may be accepted, scaled, or blocked;

• only returns available before the decision timestamp may be used.

These choices are operational assumptions, not a verified implementation of the paper's VaR method.

14.1 VaR sign convention and units

Let r be a collection of historical returns and let alpha be the confidence level. The lower-tail return quantile is q_(1-alpha)(r). A positive return-loss VaR is:

\[ \operatorname{VaR}^{\text{return}}\alpha = -q{1-\alpha}(r). \]

If the 5th-percentile return at 95% confidence is -0.04, return VaR is 0.04.

For an exposure E, projected currency VaR is:

\[ \operatorname{VaR}^{\text{currency}}\alpha = E\times \operatorname{VaR}^{\text{return}}\alpha. \]

For example, a $2,000 exposure and a 4% return-loss VaR produce:

projected currency VaR = 2,000 * 0.04 = $80

The units are important. A currency VaR must be compared with a currency limit, not with a fraction such as 0.05.

The risk limit is calculated from portfolio value V and the configured maximum fraction m:

limitcurrency=mV.

Thus, for a $10,000 portfolio and m=0.05, the limit is $500.

VaR is a loss quantile, not a guaranteed maximum loss. A realized loss can exceed it, particularly when market conditions change or the historical window does not represent the current regime.

14.2 Explicit configuration

VaRConfig in src/quantitative_trading_model/config.py makes the unspecified choices visible:

@dataclass
class VaRConfig:
    enabled: bool = True
    confidence: float = 0.95
    horizon: int = 1
    lookback: int = 100
    method: str = "historical"
    action: str = "scale"
    max_var_fraction: float = 0.05
    insufficient_history: str = "allow"
    min_observations: int = 30

The defaults are implementation choices:

| Setting | Default | Meaning | | --- | ---: | --- | | confidence | 0.95 | Use the lower 5% return tail. | | horizon | 1 | Estimate a one-period loss threshold. | | lookback | 100 | Use at most the latest 100 available returns. | | method | historical | Use an empirical quantile. | | action | scale | Reduce an order that exceeds the risk limit. | | max_var_fraction | 0.05 | Permit VaR up to 5% of portfolio value. | | min_observations | 30 | Require this many horizon observations for an actionable estimate. |

Configuration validation rejects invalid confidence levels, nonpositive horizons and lookbacks, unsupported methods, and unknown action modes. Decimal transaction fees, forecast settings, sizing settings, and execution rules remain separate from VaR.

14.3 Estimating a rolling historical quantile

HistoricalVaR in src/quantitative_trading_model/risk.py performs the following steps:

Convert the supplied returns to a finite one-dimensional array.

Keep only the configured lookback window.

Optionally construct overlapping compounded returns for a multi-period horizon.

Select the lower-tail empirical quantile.

Store the signed quantile and its nonnegative loss magnitude.

The central calculation is:

values = self._finite_returns(returns)
values = values[-self.lookback:]
horizon_values = self._horizon_returns(values)

if len(horizon_values) < self.min_observations:
    return VaREstimate(
        var=float("nan"),
        confidence=self.confidence,
        horizon=self.horizon,
        observations=len(horizon_values),
        return_quantile=float("nan"),
        sufficient_history=False,
    )

quantile = np.quantile(
    horizon_values,
    1.0 - self.confidence,
    method="linear",
)
return VaREstimate(
    var=max(0.0, -float(quantile)),
    confidence=self.confidence,
    horizon=self.horizon,
    observations=len(horizon_values),
    return_quantile=float(quantile),
    sufficient_history=True,
)

VaREstimate.var is a return fraction. It is not a currency amount until it is multiplied by an exposure. return_quantile retains the signed empirical value for auditing.

The 1-confidence probability is deliberate. With confidence=0.95, the implementation selects the 5th percentile. The sign conversion prevents a negative return quantile from being confused with a negative risk amount.

14.4 Optional horizon compounding

For a one-period horizon, the historical returns can be used directly. For a longer horizon, consecutive simple returns are compounded. A two-period return is:

\[ (1+rt)(1+r{t+1})-1. \]

The corresponding helper is:

def _horizon_returns(self, values: np.ndarray) -> np.ndarray:
    if self.horizon == 1:
        return values
    if values.size < self.horizon:
        return np.empty(0, dtype=float)

    windows = np.lib.stride_tricks.sliding_window_view(
        values,
        self.horizon,
    )
    return np.prod(1.0 + windows, axis=1) - 1.0

Overlapping historical blocks are a transparent reconstruction. The paper does not say whether its VaR uses one-day returns, compounded multi-day returns, a parametric distribution, or simulation.

14.5 Pre-decision history and lookback boundaries

VaR must obey the same information boundary as the forecast. If a decision is made after observations through time t, the risk history may contain returns known by that point, but not returns from t+1 or later.

HistoricalVaR accepts caller-supplied returns and cannot infer timestamps. The caller is responsible for constructing the correct information set:

# known_returns must contain only returns available at the signal time
estimate = var_estimator.estimate(known_returns, include_last=True)

If the final element represents a current-bar return that is not yet available when the signal is formed, use include_last=False:

estimate = var_estimator.estimate(
    known_returns,
    include_last=False,
)

A leakage-aware sequence is:

1. Observe prices through the information timestamp.
2. Compute only returns available at that timestamp.
3. Estimate VaR from the permitted lookback window.
4. Generate the forecast and candidate order.
5. Apply the VaR filter.
6. Execute no earlier than the next permitted bar.

The risk module must not be passed the complete historical return series during every past decision. That would use future information.

14.6 Insufficient history

When fewer than min_observations horizon returns are available, HistoricalVaR.estimate returns an estimate with sufficient_history=False and unavailable numerical values. It does not fabricate a zero-risk estimate.

VaRTradeFilter then applies the configured policy:

• insufficient_history="allow" permits the order while recording that VaR was unavailable;

• insufficient_history="block" rejects positive buy exposure until enough history exists.

This is a policy choice that belongs in experiment metadata. Missing risk information is not evidence that risk is zero.

14.7 Projecting VaR consistently in currency units

The corrected unit contract for the risk filter is:

def projected_var(
    self,
    estimate: VaREstimate,
    *,
    exposure: float,
    portfolio_value: float,
) -> float:
    """Return projected VaR as a currency amount."""
    if not estimate.sufficient_history:
        return float("nan")
    if portfolio_value <= 0:
        raise ValueError("portfolio_value must be positive")
    if exposure < 0:
        raise ValueError("exposure must be nonnegative")
    return estimate.var * exposure

The portfolio value is needed for the limit, not for dividing the projected loss:

limit = self.max_var_fraction * portfolio_value
projected = self.projected_var(
    estimate,
    exposure=projected_exposure,
    portfolio_value=portfolio_value,
)

For a 4% return VaR and $2,000 exposure, projected_var returns $80. For a $10,000 portfolio and a 5% limit, the comparison is $80 <= $500; both sides are currency amounts.

This avoids mixing a portfolio-relative fraction with a currency limit. If a project instead chooses to return a relative fraction, it must compare that result with max_var_fraction directly. The implementation described here uses currency amounts throughout the decision path.

14.8 Accept, scale, and block behavior

VaRTradeFilter applies a risk decision to a proposed notional quantity. Its modes are:

Accept

Approve the proposed quantity. This can be used when VaR is informational or when the order is already within the limit.

Block

Return an approved quantity of zero when projected currency VaR exceeds the currency limit.

Scale

Approve the largest quantity that remains within the limit. With portfolio value V, limit fraction m, and return VaR v, the maximum total exposure is:

Emax=mVv,

provided v > 0. The maximum additional buy is this amount minus current exposure.

A consistent decision core is:

limit = self.max_var_fraction * portfolio_value
buy_amount = max(proposed_quantity, 0.0)
full_exposure = current_exposure + buy_amount
full_var = self.projected_var(
    estimate,
    exposure=full_exposure,
    portfolio_value=portfolio_value,
)

if proposed_quantity <= 0 or full_var <= limit:
    return RiskDecision(
        "accept",
        proposed_quantity,
        proposed_quantity,
        1.0,
        full_var,
        limit,
        current_exposure,
        full_exposure,
        "within VaR limit or exposure-reducing order",
        True,
    )

if self.action == "block" or estimate.var <= 0:
    return RiskDecision(
        "block",
        proposed_quantity,
        0.0,
        0.0,
        full_var,
        limit,
        current_exposure,
        current_exposure,
        "proposed exposure exceeds VaR limit",
        True,
    )

allowed_exposure = limit / estimate.var
allowed_buy = max(0.0, allowed_exposure - current_exposure)
approved = min(proposed_quantity, allowed_buy)
scale = approved / proposed_quantity if proposed_quantity else 0.0

The critical correction is allowed_exposure = limit / estimate.var, not limit * portfolio_value / estimate.var, because limit is already a currency amount. Similarly, projected_var returns estimate.var * exposure, not estimate.var * exposure / portfolio_value.

14.9 Connecting VaR to the strategy layer

The strategy flow is:

forecast
  -> signal
  -> candidate notional
  -> cash and exposure checks
  -> VaR accept / scale / block
  -> greedy ranking
  -> next-bar execution

The VaR call must use the same argument names and units as VaRTradeFilter.decide. In particular, GreedyPolicy should pass the proposed order as proposed_quantity, current marked exposure as current_exposure, and portfolio value as portfolio_value:

risk_decision = self.risk_filter.decide(
    proposed_quantity=action.notional,
    current_exposure=snapshot.holdings * snapshot.price,
    portfolio_value=snapshot.value,
    returns=returns,
)

The selected quantity is then taken from risk_decision.approved_quantity. A blocked action must be marked infeasible, and a scaled action must be rescored using its approved notional before greedy ranking. The risk filter must never increase the proposed quantity.

This explicit adapter is preferable to relying on incompatible keyword aliases such as proposed_notional when the risk API requires proposed_quantity. The strategy and risk modules should share one documented contract rather than silently catching argument errors.

14.10 Treatment of sells

The default filter is a long-exposure filter. A negative proposed quantity represents a sell. A sell that reduces long exposure is accepted by this filter because it does not increase the projected long exposure.

This does not constitute a complete portfolio-risk model. It does not model short positions, leverage, cross-asset correlations, liquidity, spread, market impact, or nonlinear instruments. The paper's brief mention of VaR does not justify assuming those features.

14.11 Numerical examples

Suppose:

portfolio value       = $10,000
current exposure       = $2,000
proposed buy           = $3,000
return VaR             = 4%
maximum VaR fraction   = 5%

The complete proposed exposure is $5,000, so projected currency VaR is:

$5,000 * 0.04 = $200

The currency limit is:

$10,000 * 0.05 = $500

Because $200 <= $500, the VaR layer accepts the proposed order, subject to cash, sizing, fee, and exposure constraints elsewhere.

If return VaR is 12% instead:

full projected VaR = $5,000 * 0.12 = $600
risk limit         = $500

In scale mode, maximum total exposure is:

$500 / 0.12 = approximately $4,166.67

After subtracting the existing $2,000 exposure, the maximum additional buy is approximately $2,166.67. In block mode, the $3,000 proposed order is rejected.

These examples describe the reconstructed unit-consistent policy. They do not establish what the paper's original VaR procedure did.

14.12 Tests and verification boundary

Tests for the corrected API should call the actual interfaces directly. For example:

var = HistoricalVaR(
    confidence=0.95,
    horizon=1,
    lookback=len(returns),
    min_observations=2,
)
estimate = var.estimate(returns)

risk_filter = VaRTradeFilter(
    var_estimator=var,
    max_var_fraction=0.005,
    action="block",
)
decision = risk_filter.decide(
    proposed_quantity=500.0,
    current_exposure=0.0,
    portfolio_value=1_000.0,
    estimate=estimate,
)

assert decision.approved_quantity <= 500.0

The test should not pass portfolio_value to HistoricalVaR.estimate, because historical estimation uses returns and VaR configuration; portfolio value belongs to exposure projection and trade filtering. It should also use max_var_fraction, not an unsupported max_var constructor argument, and should call decide with proposed_quantity, current_exposure, and portfolio_value.

These are invariant-level checks, not empirical validation. Static and semantic review can inspect units, argument contracts, and no-look-ahead behavior, but no claim is made here that the generated code was executed. The supplied verification record states that semantic code verification was skipped, so the risk tests and integration snippets should be treated as corrected intended contracts until the source artifacts are aligned and independently checked.

14.13 Reproducibility boundary

This VaR layer remains reconstructed. A verified implementation of the paper would require at least:

the confidence level;

the VaR horizon;

the return definition and portfolio aggregation method;

the estimation window;

the historical or parametric estimation method;

the maximum acceptable loss or exposure threshold;

whether VaR blocks, scales, or merely reports trades;

how VaR interacts with the greedy policy and position-sizing schedule.

The accurate claim is therefore: the project defines a documented rolling historical VaR filter with consistent currency units and explicit action rules, inspired by the paper's risk-control description. It does not claim to reproduce the paper's unspecified VaR method or its reported investment outcomes.

15. Simulate Next-Bar Execution and Portfolio Accounting

A forecast becomes useful to a backtest only after it is converted into an order and accounted for consistently. The paper describes a combined trading framework, but it does not fully specify signal timing, execution prices, fee conventions, slippage, liquidation, or portfolio accounting. The generated implementation therefore makes these choices explicit in src/quantitative_trading_model/backtest.py.

These rules are reconstructions, not claims about the paper's exact backtest. The simulator is offline: it accepts caller-supplied local prices and orders and does not connect to an exchange or submit live orders.

15.1 Signal time and execution time

A strategy forms a signal using information available at time t. The order must execute at a later timestamp. The simulator enforces that ordering:

historical window ends at t
    -> forecast and signal are generated at t
    -> order is submitted for a later bar
    -> fill occurs after t
    -> portfolio is marked to market

The Order dataclass records both the signal and intended execution timestamps:

@dataclass(frozen=True)
class Order:
    side: str
    quantity: float | None = None
    signal_timestamp: Any = None
    execution_timestamp: Any = None
    notional: float | None = None
    addition_level: int | None = None
    reason: str = ""
    metadata: Mapping[str, Any] = field(default_factory=dict)

An order may specify asset units through quantity or a currency amount through notional. In the current implementation, a notional order is converted to units using the supplied reference price before slippage is applied. Therefore, with nonzero slippage, the executed notional can differ from the requested notional. This is an implementation detail that should be recorded in experiment metadata rather than described as conversion at the final slippage-adjusted price.

PortfolioSimulator.execute rejects same-bar and earlier execution:

fill_time = _timestamp(timestamp)
signal_time = _timestamp(order.signal_timestamp)
if fill_time <= signal_time:
    raise ValueError(
        "Orders must execute strictly after their signal timestamp"
    )

This prevents a signal generated from a closing price from being filled at that same closing price. The simulator enforces a later timestamp, but it does not independently determine that the timestamp is exactly the next bar. A caller must supply next-bar orders or use a scheduling layer that does so.

Each completed order becomes a Fill record containing its timing and accounting details:

@dataclass(frozen=True)
class Fill:
    timestamp: Timestamp
    signal_timestamp: Timestamp
    side: str
    quantity: float
    reference_price: float
    execution_price: float
    notional: float
    fee: float
    cash_change: float
    reason: str = ""
    addition_level: int | None = None

Thus, a fill is an auditable event rather than merely a change in portfolio value.

15.2 Fees and executed notional

For an executed notional N and fee rate f, the configured paper-inspired convention is:

buy cash cost=N(1+f),

sell cash proceeds=N(1−f).

The current simulator applies one fee rate to both buys and sells whenever that rate is configured. However, the bare PortfolioSimulator constructor defaults to a fee rate of 0.0. The two-sided fee behavior is therefore a configured convention, not an unconditional simulator default. ExecutionConfig supplies a paper-inspired fee rate when it is passed to the simulator, but the current simulator does not consult the fee_on_buy and fee_on_sell boolean fields. Those flags should not be described as controlling accounting until the implementation is extended to enforce them.

The core fill calculation is:

notional = quantity * effective_price
fee = notional * self.fee_rate

if side == "buy":
    cash_change = -(notional + fee)
    self.state.quantity += quantity
    self.state.cash += cash_change
else:
    cash_change = notional - fee
    self.state.quantity -= quantity
    self.state.cash += cash_change

self.state.fees_paid += fee
self.state.traded_notional += notional

PortfolioState stores the mutable account state:

@dataclass
class PortfolioState:
    cash: float
    quantity: float = 0.0
    average_price: float = 0.0
    fees_paid: float = 0.0
    traded_notional: float = 0.0
    addition_level: int = 0

The average price is informational. Portfolio value is calculated from cash, quantity, and the current market price. Cumulative fees and traded notional support later turnover and fee-sensitivity reporting.

15.3 Slippage

The reference price is the observed price supplied to the simulator. Optional proportional slippage adjusts it against the trader:

effective_price = price * (
    1.0 + self.slippage_rate if side == "buy"
    else 1.0 - self.slippage_rate
)

A buy therefore receives a higher effective price and a sell receives a lower one. Fees are calculated from the resulting executed notional.

This is a simple slippage model. It does not represent bid-ask spread, market impact, liquidity, partial fills, or intrabar price paths. The paper does not provide those details, so proportional slippage is an explicit reconstruction rather than a paper-faithful assumption.

Because notional-to-unit conversion currently uses the reference price before this adjustment, a request for a fixed notional should be interpreted as a reference-price notional. If exact post-slippage notional sizing is required, the conversion logic in backtest.py must be changed before using that interpretation.

15.4 Units, cash limits, and leverage

The current simulator always permits fractional quantities. Although ExecutionConfig contains an allow_fractional_units field, PortfolioSimulator does not currently enforce it. Fractional-unit support should therefore be described as the simulator's present behavior, not as a configurable guarantee.

When leverage is disabled, buys are clipped to available cash:

if side == "buy" and not self.allow_leverage:
    affordable = self.state.cash / (
        effective_price * (1.0 + self.fee_rate)
    )
    quantity = min(quantity, max(0.0, affordable))

Likewise, a long-only account cannot sell more than it owns:

if side == "sell" and not self.allow_short:
    quantity = min(quantity, max(0.0, self.state.quantity))

These safeguards prevent negative cash and negative holdings under the simulator's default long-only policy. They are safety choices made by the implementation, not evidence that the paper used the same constraints. The paper does not state whether it allowed leverage, margin, short positions, or unlimited reinvestment.

For example, an account with $1,000 buys $400 of an asset at a configured fee rate of 0.5%:

reference notional = $400.00
fee                = $400.00 * 0.005 = $2.00
cash decrease      = $402.00
cash remaining     = $598.00

At a reference price of $100, the requested notional becomes four units before any slippage adjustment.

15.5 Mark-to-market valuation

At each valuation timestamp, the simulator computes:

\[ Vt=\text{cash}t+\text{quantity}t\times pt. \]

The corresponding method is straightforward:

def portfolio_value(self, price: float) -> float:
    price = _number(price)
    return self.cash + self.quantity * price

mark_to_market stores the result in a BacktestRecord:

@dataclass(frozen=True)
class BacktestRecord:
    timestamp: Timestamp
    price: float
    cash: float
    quantity: float
    portfolio_value: float
    fees_paid: float
    traded_notional: float
    fill_count: int = 0

Valuation timestamps must increase strictly. The resulting value must not be materially negative. These rules provide an accounting invariant that can be checked independently for every record.

Continuing the example, four units held at a market price of $105 produce:

cash             = $598.00
marked holdings  = 4 * $105 = $420.00
portfolio value  = $1,018.00

The entry fee is already reflected in cash. A later sale incurs a further fee when the configured fee rate is nonzero.

15.6 Explicit liquidation

The final value of an open position can mean either its marked value or the cash remaining after liquidation. These are different when selling incurs fees. The simulator therefore does not silently assume liquidation.

liquidate creates a sell order for the remaining long quantity and places it after the last valuation timestamp:

def liquidate(
    self,
    timestamp: Any,
    price: float,
    reason: str = "end_of_backtest",
) -> Fill | None:
    if self.state.quantity <= 0:
        return None
    signal_time = self._last_valuation_timestamp
    return self.execute(
        Order(
            side="sell",
            quantity=self.state.quantity,
            signal_timestamp=signal_time,
            reason=reason,
        ),
        timestamp,
        price,
    )

The liquidation fill remains in the event ledger and any configured sell fee is included in cumulative costs. Whether to liquidate depends on the research question. A cash-outcome report generally requires liquidation, while a marked portfolio report may not. The paper does not specify which convention produced its headline values, so the choice must accompany every result.

15.7 Buy-and-hold benchmark

A strategy should be compared with passive ownership over the same period. The buy_and_hold function buys at the first available bar, applies the configured purchase assumptions, holds the asset, and optionally liquidates at the end.

Its entry signal is placed just before the first bar so that it satisfies the same timestamp contract:

first = _timestamp(frame.index[0])
signal_time = first - pd.Timedelta(nanoseconds=1)

quantity = initial_cash / (
    float(frame.iloc[0, 0])
    * (1.0 + slippage_rate)
    * (1.0 + fee_rate)
)

order = Order(
    side="buy",
    quantity=quantity,
    signal_timestamp=signal_time,
    execution_timestamp=first,
    reason="buy_and_hold_entry",
)

This benchmark uses no forecasts, streak statistics, VaR, or greedy selection. It helps distinguish strategy-specific behavior from the return generated simply by holding the asset. The paper does not report a buy-and-hold comparison, so adding one is an evaluation improvement rather than a claim about the original method.

15.8 Accounting tests and verification limits

tests/test_backtest_accounting.py contains hand-calculable tests for intended contracts such as fee arithmetic, later execution, valuation identity, no-leverage behavior, liquidation, and buy-and-hold accounting. A representative invariant is:

cash = _field(state, "cash")
quantity = _field(state, "quantity", "holdings", "asset_quantity")
value = _field(record, "portfolio_value", "value", "equity")
assert value == pytest.approx(cash + quantity * 12.0)

These files are static, intended test artifacts. They should not be presented as executed or passing. The supplied verification record reports that semantic code verification was skipped. It also notes apparent API inconsistencies between some generated tests and the generated sizing, risk, strategy, and configuration modules. Consequently, these tests may require API alignment before they can serve as executable checks.

The useful claim is narrower: the tests document the accounting invariants that a corrected implementation should satisfy. They do not establish runtime correctness, profitability, or reproduction of the paper.

15.9 Why the paper's headline values cannot be validated

The paper reports approximately $646 from a $500 gold allocation and $215,487 from a $500 Bitcoin allocation. The simulator does not treat these figures as expected outputs. Validation would require, at minimum:

• exact assets and historical dates;

• sampling frequency and price field;

• forecast target and signal timestamps;

• buy, sell, hold, and add-position rules;

• original position-sizing parameters;

• VaR confidence, horizon, and action rule;

• greedy objective and constraints;

• fee, spread, and slippage conventions;

• fractional-unit, leverage, and reinvestment rules;

• liquidation or mark-to-market convention.

Without these details, matching a final number would not demonstrate faithful reproduction. It could result from selecting a similar period or from offsetting errors in data preparation and accounting.

The appropriate interpretation is:

The backtest module demonstrates timestamped execution and auditable portfolio accounting under explicit assumptions. It does not verify the paper's reported profits.

This distinction keeps the implementation useful for research while avoiding unsupported claims about profitability or live-trading readiness.

16. Orchestrate Forecast Experiments and Fee Sensitivity

The experiment layer is intended to make research runs repeatable, but its current generated implementation has a narrower role than the complete architecture described earlier. src/quantitative_trading_model/experiments.py currently provides:

• chronological forecast orchestration;

• expanding walk-forward forecast orchestration;

• optional benchmark dispatch;

• fee-sensitivity callback infrastructure; and

• result metadata and local JSON/CSV persistence.

It does not yet directly compose strategy.py, sizing.py, risk.py, and backtest.py into one end-to-end trading run. A complete integration would need an additional orchestration function that converts forecasts into signals, applies sizing and VaR, selects actions, submits next-bar orders, and collects portfolio metrics. The current fee-sensitivity function instead receives that behavior through an injected callback.

The distinction matters because an experiment runner can preserve assumptions without necessarily implementing every stage itself. The paper's data, execution protocol, position-sizing equations, VaR settings, and greedy policy are also incomplete, so the orchestration layer must report both its own implementation status and the paper-fidelity limitations.

16.1 Current position in the research pipeline

The intended overall workflow is:

local price data
    -> cleaning and sliding windows
    -> chronological split or walk-forward folds
    -> forecast model and optional benchmarks
    -> signals, sizing, VaR, and greedy policy
    -> event-driven backtest
    -> forecast and portfolio metrics
    -> result metadata and local files

The current experiments.py implementation covers the forecast and result-recording portions directly. The trading stages can be supplied by a caller through a backtest callback, but they are not automatically wired together by run_forecast_experiment or run_fee_sensitivity.

Its common result container is:

@dataclass
class ExperimentResult:
    experiment_type: str
    asset: str | None = None
    model: str | None = None
    status: str = "completed"
    metrics: dict[str, Any] = field(default_factory=dict)
    portfolio: dict[str, Any] = field(default_factory=dict)
    metadata: dict[str, Any] = field(default_factory=dict)
    predictions: list[dict[str, Any]] = field(default_factory=list)
    warnings: list[str] = field(default_factory=list)

The separate metrics and portfolio fields are useful even though the current forecast runner primarily fills metrics. Forecast errors such as RMSE and MAE belong in metrics; final value, drawdown, turnover, and fees belong in portfolio when a backtest callback supplies them. The warnings field should identify unavailable dependencies, unresolved interfaces, and paper claims that were not reproduced.

to_dict(), to_json(), summarize_results(), and save_local_results() are serialization utilities. They preserve local research records; they do not send results to a market service or submit orders.

16.2 Paper-style chronological forecast experiments

The paper reports a 70:30 train/test split. The current run_forecast_experiment function is the generated implementation's paper-style forecast path:

def run_forecast_experiment(
    prices: Any,
    *,
    asset: str = "asset",
    config: ResearchConfig | None = None,
    model_name: str = "att_bilstm",
    model: Any = None,
    include_benchmarks: bool = False,
    seed: int | None = 7,
) -> ExperimentResult:

Its intended sequence is:

Validate a supplied ResearchConfig.

Standardize local prices.

Create windows using the configured window length, stride, and horizon.

Split samples chronologically.

Fit the requested model on the training partition.

Predict the held-out test partition.

Calculate forecast metrics.

Store model, split, horizon, seed, and warning metadata.

This is forecast orchestration, not a complete portfolio experiment. It does not itself call the signal generator, position sizer, VaR filter, greedy policy, or portfolio simulator. A caller that needs trading results must connect those modules separately or provide a higher-level callback.

The intended metadata shape is:

metadata = {
    "asset": asset,
    "model": model_name,
    "split": "chronological_70_30_or_configured",
    "window_length": 100,
    "horizon": 1,
    "seed": 7,
    "data_points": len(values),
    "synthetic_data_is_not_paper_validation": True,
}

This metadata design is useful, but it should not be treated as proof that every field is currently populated correctly. The generated code contains an unresolved interface mismatch: run_forecast_experiment attempts to pass timestamps to standardize_prices, while the generated standardize_prices signature accepts a DataFrame or PriceData and does not define a timestamps keyword. Timestamp-aware runs therefore require API alignment before they can be considered runtime-ready.

The paper's exact data source, split date, scaling boundary, and target protocol are also unavailable. A chronological split is therefore a documented comparison protocol, not an exact reproduction of the paper's experiment.

16.3 Walk-forward forecast experiments

For trading conclusions, expanding walk-forward evaluation is preferable to one fixed split:

train through t
forecast the next test block
move the boundary forward
expand the training history
forecast again
repeat

run_walk_forward_experiment is intended to retrain or invoke a supplied train_predict callback for each fold, then aggregate the fold predictions and targets. This protocol better reflects deployment because each forecast uses only the historical information available at that point.

Walk-forward evaluation does not remove all sources of optimism. Feature construction, scaling, model selection, and hyperparameter tuning must still be restricted to the information available in each fold. Walk-forward results also should not be compared directly with the paper's 70:30 metrics without explaining the protocol difference.

16.4 Fee grids are experiment inputs, not recovered results

The paper describes the following transaction-fee scenarios:

FeeSensitivityConfig defines these fields:

@dataclass
class FeeSensitivityConfig:
    gold_rates: tuple[float, ...] = (
        0.002, 0.004, 0.006, 0.008, 0.010, 0.012
    )
    bitcoin_rates: tuple[float, ...] = (
        0.015, 0.020, 0.025, 0.030, 0.035
    )
    apply_to_buys: bool = True
    apply_to_sells: bool = True
    hold_signals_fixed: bool = True

A decimal rate of 0.002 means 0.2%. Passing 0.2 would mean 20% and would be a different experiment.

There is an important current implementation limitation: run_fee_sensitivity looks for configuration attributes named gold_fees and bitcoin_fees, but FeeSensitivityConfig defines gold_rates and bitcoin_rates. As generated, custom configured grids are therefore not reliably consumed; the function can fall back to its module-level default grids. This naming mismatch must be repaired before claiming that arbitrary FeeSensitivityConfig values control the sweep.

16.5 Fixed signals versus regenerated policy decisions

Fee sensitivity can answer two different questions.

Fixed-signal sensitivity

Generate forecasts and orders once, then replay the same orders under each fee rate:

same data
same forecasts
same signals
same sizing schedule
fee rate changes
backtest reruns

This isolates the direct accounting effect of fees.

Fee-aware policy sensitivity

Regenerate candidate actions for each fee rate when fees affect:

• whether an order is affordable;

• whether its expected return exceeds its cost;

• which greedy candidate has the highest score; or

• whether exposure or risk constraints permit it.

The current run_fee_sensitivity function accepts a run_backtest callback and passes fee_rate to that callback:

def run_fee_sensitivity(
    run_backtest: Callable[..., Any],
    *,
    asset: str,
    config: FeeSensitivityConfig | None = None,
    fees: Sequence[float] | None = None,
    base_kwargs: Mapping[str, Any] | None = None,
) -> list[ExperimentResult]:

This callback-based design is intentional infrastructure, not direct strategy integration. The callback must decide whether to reuse signals or regenerate them. The current function does not inspect or enforce FeeSensitivityConfig.hold_signals_fixed, and it does not currently record an explicit signals_fixed or policy_regenerated field. Its hardcoded metadata field non_fee_assumptions_held_fixed=True only states the intended sweep contract; it does not establish how the callback selected actions.

A repaired orchestration contract should record the policy mode explicitly, for example:

result.metadata.update(
    {
        "fee_rate": rate,
        "fee_percent": rate * 100.0,
        "signals_fixed": True,
        "policy_regenerated": False,
        "non_fee_assumptions_held_fixed": True,
    }
)

For a fee-aware callback, the last two policy fields should instead indicate that decisions were regenerated. Until that metadata and configuration handling are aligned, fee-sweep results should be treated as callback outputs whose policy behavior must be inspected separately.

16.6 Optional benchmark orchestration

run_forecast_experiment(include_benchmarks=True) is intended to request the optional benchmark suite. However, the generated call currently passes names such as train_data and test_data, while run_benchmark_suite requires train_values and test_values. The compatibility helper filters keyword arguments but cannot invent missing required parameter names. Consequently, the benchmark path may fail and be converted into a warning rather than producing benchmark results.

The required interface should be aligned explicitly, for example:

benchmark = run_benchmark_suite(
    train_values=train_data,
    test_values=test_data,
    metadata={"asset": asset, "horizon": horizon},
)

Until that repair is made, benchmark orchestration should be described as an intended dispatch path, not as a verified working connection. Optional dependency status and model semantics remain important: HMM Viterbi decoding is not automatically a price forecast, and XGBoost is gradient-boosted trees rather than a random forest. Benchmark metrics also require aligned targets, timestamps, preprocessing, and untouched test data.

16.7 Recording results and paper claims

The result layer can save local artifacts:

def save_local_results(
    results: ExperimentResult | Iterable[ExperimentResult],
    path: str | Path,
    *,
    format: str | None = None,
) -> Path:

JSON preserves nested predictions, warnings, and metadata. CSV is convenient for scalar comparison tables. Neither format establishes empirical validity.

The paper's headline outcomes are stored as claims rather than expected outputs:

PAPER_REPORTED_VALUES = {
    "gold_final_value_usd": {
        "value": 646.0,
        "status": "reported_unverified_claim",
    },
    "bitcoin_final_value_usd": {
        "value": 215487.0,
        "status": "reported_unverified_claim",
    },
    "combined_final_value_usd": {
        "value": 216133.0,
        "status": "reported_unverified_claim",
    },
}

These values must not be used as unit-test expectations. The original data, dates, signals, sizing equations, VaR settings, greedy rules, fee convention, and liquidation protocol are missing. Agreement with one number would not prove that the full method was reproduced; disagreement under a different local protocol would not by itself identify a coding error.

16.8 What a trustworthy fee-sensitivity record should contain

Each scenario should preserve at least:

The current summarize_results function flattens scalar fields from result metrics, portfolio summaries, and metadata. save_local_results writes those records locally. Before relying on the files, inspect whether the callback supplied portfolio fields and whether policy metadata accurately describes the run.

16.9 Practical checklist and current readiness

Before preparing a forecast or fee experiment, record:

Local data path and provenance.

Asset, instrument, currency, frequency, and date range.

Window length, stride, target representation, and horizon.

Chronological or walk-forward boundaries.

Scaling method and fitting rows.

Model architecture, seed, and optional dependency versions.

Signal aggregation and thresholds.

Sizing schedule and exposure limits.

VaR confidence, horizon, lookback, and action mode.

Fee convention, slippage, execution timing, and liquidation policy.

Whether signals are fixed or regenerated for each fee scenario.

Forecast metrics and portfolio metrics in separate fields.

Any warnings caused by unavailable dependencies or API mismatches.

The generated project should currently be treated as a static, educational artifact rather than an executed experiment system. Known issues include the timestamp argument mismatch in standardize_prices, the benchmark argument-name mismatch, the unused hold_signals_fixed setting, and the fee-grid naming mismatch. Static checks also reported findings in experiments.py, while semantic code verification was skipped. Therefore, this section documents the intended orchestration contracts and the current limitations without claiming that the experiment paths were executed or that their outputs are correct.

16.10 Interpreting the missing fee figures

The fee grids can be reproduced as an experiment design. The paper's numerical curves cannot be recovered from the extracted text. Do not infer them from the headline profits, interpolate them from the listed percentages, or label a newly generated local curve as the paper's result.

A local result should instead be described as:

performance under the repository's documented reconstruction and local dataset

That wording preserves the value of sensitivity analysis while distinguishing it from empirical reproduction. The experiment layer's current contribution is reproducible configuration and result recording; direct end-to-end trading orchestration remains a repair task rather than an implemented fact.

17. Run the Offline Workflow from the Command Line

The project defines an offline command-line interface (CLI) for inspecting local data, generating synthetic demonstrations, and requesting research experiments. It does not connect to an exchange, download prices, store credentials, or submit orders.

The CLI should therefore be understood as a documented interface around the research modules, not as a verified end-to-end trading application. Several generated integrations currently have known incompatibilities. The examples below describe the intended command contracts and identify where the generated files still require repair. No command execution is claimed.

17.1 Command registration and optional dependencies

The shell command is registered in pyproject.toml:

[project.scripts]
quant-trading-model = "quantitative_trading_model.cli:main"

After installation, this maps quant-trading-model to main() in src/quantitative_trading_model/cli.py. The parser selects a subcommand and the handler produces local output.

Modeling dependencies are optional:

[project.optional-dependencies]
deep-learning = ["torch>=2.0"]
classical = [
    "statsmodels>=0.14",
    "pmdarima>=2.0",
    "hmmlearn>=0.3",
    "xgboost>=2.0"
]

The core data, streak, sizing, risk, and accounting modules use the lighter dependencies declared in the main project configuration. PyTorch and classical forecasting packages are imported only by features that need them.

The intended behavior is to report a missing optional dependency clearly. However, the generated main() catches CLIError, FileNotFoundError, ValueError, and TypeError, but not ImportError. Consequently, a missing PyTorch installation may still produce an uncaught traceback when a forecast command reaches the model layer. The CLI needs an explicit except ImportError branch before this behavior can be described as reliable shell-level error handling.

17.2 Defined subcommands

build_parser() defines these subcommands:

Thus, the commands are statically specified interfaces, not all working workflows. A complete runtime path requires repairing the interfaces identified below.

The parser excerpt illustrates the command structure:

def build_parser() -> argparse.ArgumentParser:
    parser = argparse.ArgumentParser(
        prog="quant-trading-model",
        description="Offline educational tools for the quantitative trading model paper.",
    )
    subparsers = parser.add_subparsers(dest="command", required=True)

    demo = subparsers.add_parser("demo")
    demo.add_argument("--rows", type=int, default=500)
    demo.add_argument("--seed", type=int, default=7)
    demo.add_argument("--asset", choices=("gold", "bitcoin"), default="gold")
    demo.set_defaults(handler=run_demo)

    # validate-data, forecast, backtest, and fee-sensitivity are
    # registered later in the same function.
    return parser

This is an explanatory excerpt, not a complete replacement for cli.py.

17.3 Local-only paths and data provenance

The CLI includes path checks intended to reject URLs and network paths:

def _safe_input_path(value: str) -> Path:
    if "://" in value:
        raise CLIError("Only local files are supported; URLs and network paths are not accepted")
    path = Path(value).expanduser()
    if not path.exists():
        raise CLIError(f"Input file does not exist: {path}")
    if not path.is_file():
        raise CLIError(f"Input path is not a regular file: {path}")
    return path.resolve()


def _safe_output_path(value: str) -> Path:
    if "://" in value:
        raise CLIError("Output must be a local path, not a URL")
    path = Path(value).expanduser()
    if path.exists() and path.is_dir():
        raise CLIError(f"Output path is a directory: {path}")
    return path.resolve()

These checks constrain file locations; they do not establish that a dataset matches the paper. Every local result should also record the source file, instrument, currency, frequency, date range, timezone, cleaning rules, and configuration snapshot.

There is a further generated-file inconsistency in _load_local_data(): it passes asset_name=args.asset, while the generated load_price_csv() function accepts asset, not asset_name. Depending on the compatibility wrapper, this can fail or cause the loader to retain its default asset label. The keyword must be aligned before asset-specific provenance can be trusted.

17.4 Synthetic demonstration

The intended synthetic command is:

quant-trading-model demo \
  --rows 500 \
  --asset gold \
  --seed 7 \
  --output results/demo.json

run_demo() calls generate_synthetic_prices() and records a compact summary. Synthetic data are useful for teaching timestamp handling, window construction, tensor shapes, finite sizing, VaR decisions, and portfolio accounting. They are not historical gold or Bitcoin data and cannot reproduce the paper's reported outcomes.

The optional --run-experiment flag is currently not a complete path. The generator returns a PriceData object, and run_demo() passes that object to run_forecast_experiment(). In the generated experiments.py, _price_array() handles a pandas Series, a pandas DataFrame, or an array-like object, but does not handle PriceData. The experiment can therefore fail before window construction. The repair must either teach _price_array() to extract PriceData.frame or make the CLI pass a compatible DataFrame or Series.

This same PriceData mismatch affects the forecast handler, because _load_local_data() also returns PriceData and passes it to run_forecast_experiment(). These commands are therefore documented as intended interfaces until that data contract is repaired.

17.5 Validate a local CSV

The generated parser uses a positional CSV argument. A statically accurate example is:

quant-trading-model validate-data data/gold.csv \
  --timestamp-column timestamp \
  --price-column close \
  --asset gold \
  --output results/gold_data_summary.json

run_validate_data() is intended to:

check the local path;

call load_price_csv();

summarize the resulting object; and

print or write local JSON.

The loader requires parseable timestamps and finite, positive prices, and it sorts and deduplicates observations. The summary should be saved with a provenance record rather than treated as evidence that the data are the same data used by the paper.

Some older README examples use --input data/gold.csv, but the generated parser defines csv positionally and does not define an --input option. The positional form above matches the generated CLI more closely.

17.6 Forecast command: intended interface and current limitation

The intended command is:

quant-trading-model forecast data/gold.csv \
  --timestamp-column timestamp \
  --price-column close \
  --asset gold \
  --output results/gold_forecast.json

Optional benchmarks are requested with:

quant-trading-model forecast data/gold.csv \
  --timestamp-column timestamp \
  --price-column close \
  --asset gold \
  --benchmarks \
  --output results/gold_forecast_with_benchmarks.json

The intended data flow is:

local CSV
  -> local loader
  -> standardized prices
  -> 100-observation windows
  -> chronological split
  -> Att-BiLSTM or selected forecaster
  -> forecast metrics
  -> local result artifact

The current generated path is not yet a verified runtime workflow. The CLI passes PriceData into run_forecast_experiment(), while _price_array() in experiments.py does not extract PriceData. This must be repaired before the command can reliably reach the forecasting model.

Even after that data mismatch is fixed, a missing PyTorch installation may escape through main() as an uncaught ImportError. The CLI should catch that exception and convert it into an actionable message. Until then, the optional-dependency behavior is a model-layer intention, not a guaranteed CLI behavior.

A repaired forecast artifact should include the asset, model, window length, horizon, split protocol, seed, preprocessing decisions, forecast metrics, package versions, and warnings. Forecast metrics remain separate from portfolio metrics and do not measure fees, turnover, drawdown, or liquidation.

17.7 Backtest command: distinguish forecasting from portfolio simulation

The intended command is:

quant-trading-model backtest data/bitcoin.csv \
  --timestamp-column timestamp \
  --price-column close \
  --asset bitcoin \
  --output results/bitcoin_backtest.json

In the generated CLI, run_backtest_command() calls run_walk_forward_experiment(). That function is intended to evaluate sequential forecasts; it does not, by itself, compose forecast signals, exponential sizing, VaR, greedy action selection, fills, fees, and portfolio valuation.

The repository does contain separate strategy and accounting modules. A complete portfolio backtest must explicitly connect them:

walk-forward forecast
  -> signal generation
  -> finite sizing
  -> VaR filter
  -> greedy action selection
  -> next-bar order
  -> portfolio simulator

Therefore, the current backtest command is a statically described walk-forward entry point, not proof that a complete paper trading strategy is running. Its name should not be interpreted as evidence that the paper's missing execution protocol has been recovered.

17.8 Fee-sensitivity command: parser exists, callback wiring is incomplete

The paper-inspired fee grids are:

The intended command is:

quant-trading-model fee-sensitivity data/gold.csv \
  --timestamp-column timestamp \
  --price-column close \
  --asset gold \
  --output results/gold_fee_sensitivity.json

The command is currently only a parser-level interface. run_fee_sensitivity() in experiments.py requires a positional run_backtest callback, followed by the asset and fee configuration. However, run_fee_sensitivity_command() calls it without supplying that required callback. As written, the handler cannot successfully invoke the fee experiment.

A repair must construct or inject a callback that accepts fee_rate, runs the same strategy and data under that fee, and returns portfolio results. Only then can the CLI perform a sensitivity sweep. The callback should hold all non-fee assumptions fixed: data, dates, forecasts, signal rules, sizing, slippage, initial cash, and liquidation. If fees change feasibility or greedy ranking, decisions must be regenerated and that dependency recorded.

The missing numeric values in the paper's fee figures must not be inferred from the fee grid itself.

17.9 Optional dependency installation

The intended installation commands are:

python -m pip install -e .
python -m pip install -e '.[deep-learning]'
python -m pip install -e '.[classical]'

The generated pyproject.toml defines the classical extra. Some README text refers to a benchmarks extra, which is inconsistent; classical is the authoritative generated metadata name.

A missing optional package should ideally produce a message such as:

PyTorch is required for recurrent forecasters. Install the optional deep-learning dependencies for this project.

At present, the model layer can raise that message, but the CLI's main() does not catch ImportError. Repairing the exception handling is necessary for the message to be consistently presented as a normal command-line error.

17.10 Archive results with provenance

The CLI's output helper is intended to print JSON or write it to a local file:

def _write_output(value: Any, output: str | None) -> None:
    rendered = _dump(value)
    if output is None:
        print(rendered)
        return
    path = _safe_output_path(output)
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_text(rendered + "\n", encoding="utf-8")
    print(f"Wrote {path}")

A useful research directory might contain:

results/
├── gold_data_summary.json
├── gold_forecast.json
├── gold_fee_sensitivity.json
├── config_snapshot.json
├── package_versions.txt
└── provenance.md

The configuration snapshot should include the window length, target horizon, scaler policy, model settings, fee rate, slippage, VaR settings, sizing schedule, signal thresholds, execution timing, and liquidation policy. The provenance file should identify the local data file and instrument metadata.

The paper's approximate $646 gold value and $215,487 Bitcoin value remain unverified claims. A local JSON file without dates, data provenance, and execution assumptions cannot validate either number.

17.11 Offline workflow checklist

The intended research workflow is:

Install only the dependencies required for the selected module.

Inspect quant-trading-model --help and the relevant subcommand help.

Use demo to inspect synthetic data flow, without treating it as empirical evidence.

Use validate-data on each local asset file.

Save the data summary and provenance metadata.

Run a chronological forecast experiment after repairing the PriceData handoff.

Record model status, target horizon, metrics, seed, and configuration.

Run walk-forward evaluation for trading-oriented conclusions.

Connect forecasts explicitly to signals, sizing, VaR, greedy selection, and next-bar execution for a portfolio backtest.

Supply a valid backtest callback before using fee sensitivity.

Compare portfolio results with buy-and-hold and report drawdown, volatility, turnover, and fees.

Label every result as implemented, reconstructed, illustrative, unavailable, or reported claim as appropriate.

The generated CLI and orchestration files also have known static-review findings: placeholder detection was reported for cli.py, and semantic code verification was skipped. Some generated tests and orchestration calls are API-inconsistent with the generated modules. These findings reinforce the correct interpretation of this section: it documents intended offline interfaces and their current repair requirements; it does not claim that any command ran successfully.

The central lesson is that a CLI can make a research workflow repeatable in form, but it cannot supply missing data or algorithmic details. Reliable interpretation still requires local data provenance, explicit configuration, causal signal timing, repaired module interfaces, and a strict distinction between computed results and the paper's unverified claims.

18. Verification: Static, Semantic, and Invariant Checks

A paper-to-code project needs more than a collection of source files. It also needs a disciplined way to check whether the implementation is structurally coherent, whether its assumptions are visible, and whether its financial logic avoids obvious look-ahead errors. At the same time, verification must not be overstated. A file that parses successfully is not necessarily semantically correct, and a passing unit test would not prove that the paper's reported profits are reproducible.

For this project, verification has three layers:

Static verification checks source structure without running the research pipeline.

Semantic review checks whether the implementation matches the intended mathematics and data flow.

Empirical validation would compare executed results with the paper, but it is currently unavailable because the original data and protocol are missing.

The supplied local verification performed the first layer and recorded a partial result. Semantic code verification was explicitly skipped. Therefore, the discussion below describes the intended checks and the reported findings; it does not claim that the generated code was executed or that the test suite passed.

18.1 Static verification: checking structure without execution

Static checks inspect files as text or syntax trees. They are useful because they can identify malformed Python, missing symbols, accidental placeholders, unsafe patterns, and documentation inconsistencies before a runtime experiment is attempted.

The local verification report applied checks such as:

• non-empty file validation;

• detection of Markdown fences in generated code fields;

• placeholder and TODO detection;

• checks for live-trading patterns;

• generic text checks;

• Python AST parsing for Python files.

AST parsing is stronger than looking for balanced parentheses with a text search. It asks Python's parser whether a file has valid syntax and produces a structured representation of imports, classes, functions, and statements. It does not, however, prove that imported names exist at runtime or that two modules agree on an API.

The static checks reported successful parsing and structural checks for the core modules, including:

• data.py, which constructs windows and chronological partitions;

• metrics.py, which separates forecast and portfolio measures;

• models/attention.py and models/forecasters.py, which define the tensor-facing model interfaces;

• streaks.py, sizing.py, and risk.py, which implement the reconstructed analytical components;

• strategy.py and backtest.py, which define candidate actions and portfolio accounting; and

• the generated test files.

That result gives useful confidence that these files are non-empty and syntactically parseable. It does not establish that the modules can be imported together, that optional dependencies are available, or that the implementation behaves correctly for real data.

18.2 Reported static-check findings

The overall local static-verification result was not passing. Four generated files were reported with issues:

The Markdown findings are format warnings in the generated-file representation. A Markdown tutorial normally uses fenced code blocks, so removing every fence would make the document less useful to readers. The important distinction is between a validation rule for a generated code field and the normal syntax of a Markdown document.

The placeholder findings are more substantive. They should be reviewed manually because a placeholder can mean anything from an intentionally unfinished branch to an incomplete implementation. Until that review and repair occur, the experiment and CLI layers should be described as planned or partially integrated artifacts rather than as verified end-to-end interfaces.

18.3 Semantic review: checking meaning, not just syntax

Semantic verification asks whether the code's operations match the intended method. For this project, the most important semantic questions are temporal, mathematical, and accounting-related.

Temporal and data invariants

The windowing and data tests are intended to review the following properties:

• timestamps are strictly increasing after cleaning;

• prices are finite and positive;

• a target timestamp occurs after the final timestamp in its input window;

• chronological training and test partitions remain ordered;

• scaling parameters are fitted only on training observations; and

• no forecast uses a target or realized return from the future.

These checks matter because a time-series implementation can be syntactically valid while still leaking information. For example, fitting a scaler on the complete series or randomly mixing overlapping windows can make a forecast evaluation look stronger than a deployment-like evaluation.

The test file tests/test_data_and_streaks.py is designed around small hand-calculated paths. That is a good verification strategy for boundaries: a short sequence makes it possible to calculate the expected target values and return signs manually. It is more informative for temporal invariants than a large opaque fixture.

Attention and model-shape invariants

The model tests are intended to review tensor contracts rather than predictive quality. The key conditions are:

• recurrent inputs have shape (batch, time, features);

• attention scores have shape (batch, time);

• attention weights are nonnegative;

• weights sum to one across the time dimension;

• the context vector has shape (batch, features);

• one-step and three-step heads return the configured horizon; and

• the primary model contains the configured 20% dropout layer.

The attention normalization check is particularly important. Applying softmax across features instead of time would produce a different algorithm from the paper's temporal attention equations. Shape tests can catch that kind of error without claiming that the model has learned a useful forecast.

The model tests also need PyTorch-aware handling. PyTorch is optional in this project, so a minimal installation should skip tensor-execution tests cleanly rather than fail while importing unrelated data or accounting modules. A skipped optional test is not a passing model validation; it means that the environment did not perform that check.

Streak and return invariants

The streak tests review the reconstruction of the paper's consecutive-rise and decline analysis:

• the default simple return uses the previous price as denominator;

• zero returns do not silently become rises or declines;

• maximal runs and overlapping subsequences produce different, intentional counts;

• streak counts are nonnegative integers; and

• percentile calculations use only the supplied historical segment.

These checks are important because the paper's use of the term “Apriori” is not operationally supported by frequent-itemset support or confidence calculations. The implementation should therefore be reviewed as streak analysis, with its chosen return and zero policies recorded explicitly.

Position-sizing invariants

The sizing tests should confirm that the normalized exponential reconstruction remains bounded:

• every score and allocation is finite and nonnegative;

• normalized weights sum to one within tolerance;

• the planned allocations do not exceed the schedule budget;

• an addition level cannot be consumed twice accidentally;

• the approved amount cannot exceed available cash; and

• the exposure cap prevents unlimited averaging down.

These are safety and consistency properties of the reconstruction. They do not prove that the paper's corrupted equations (5)–(8) were recovered correctly. In fact, exact verification of those equations remains unavailable until the original notation or equation images are supplied.

VaR and strategy invariants

The risk and strategy layers require a different kind of semantic review. The intended checks include:

• VaR uses only returns available before the decision timestamp;

• insufficient history is represented explicitly;

• a blocked trade receives zero approved quantity;

• risk scaling never increases the proposed order;

• candidate scores use forecasts, current portfolio state, fees, and constraints only;

• infeasible actions are excluded before ranking; and

• greedy tie-breaking is deterministic.

These conditions prevent a risk filter from becoming decorative. A VaR calculation that uses future returns, or a strategy score that accidentally includes realized profit, would invalidate the backtest even if every function returns a number.

The paper does not define its VaR parameters or greedy objective. Consequently, semantic verification can check the internal consistency of the chosen reconstruction, but it cannot check fidelity to an unavailable original algorithm.

Portfolio-accounting invariants

The backtest tests are designed around hand-computable transactions. The central identity is:

\[ Vt = \operatorname{cash}t + \operatorname{quantity}t pt. \]

The intended accounting checks include:

• every fill occurs after its signal timestamp;

• buy and sell fees are applied exactly once to executed notional;

• unaffordable buys do not create negative cash under no-leverage settings;

• holdings do not become negative when shorting is disabled;

• marked portfolio value equals cash plus marked holdings;

• liquidation is explicit; and

• the buy-and-hold benchmark follows a documented fee and timing convention.

These tests verify the simulator's stated accounting contract. They do not verify the paper's accounting because the paper does not specify its execution dates, fee-side convention, slippage, partial-unit rules, or liquidation policy.

18.4 Why the skipped semantic verification matters

The semantic verification report states that code semantic verification was skipped using the --skip-code-semantic-verification option. Its overall assessment was therefore “not semantically reviewed.” This is an important limitation, not a minor footnote.

Static AST parsing can show that Python syntax is valid. It cannot reliably detect every issue that a semantic review would investigate, such as:

• a caller passing split_config to a function that accepts only config;

• a test constructing a class with fields that differ from the generated class;

• a benchmark adapter assuming a metric method that another module does not expose;

• a forecast output using a different shape from the target array; or

• an experiment function describing end-to-end backtesting while only orchestrating forecasts.

The supplied project notes also warn that some generated tests and orchestration code appear API-inconsistent with the generated modules. That means the tests should be treated as intended invariant specifications and static artifacts until their interfaces are reconciled. No claim should be made that the test suite passes.

18.5 Verification is not execution and execution is not reproduction

There are three separate statements that should not be conflated:

The source parses. Python's parser accepted the file structure.

The implementation is semantically coherent. Functions, types, formulas, and timestamps agree with one another under review.

The paper's experiment is reproduced. The implementation matches the original data, protocol, and reported results.

The available evidence supports only parts of the first statement. The second was not completed by the supplied semantic verifier. The third is impossible at present because the paper's data and several algorithmic details are missing.

In particular, this tutorial does not claim that:

• the generated code was executed;

• all dependencies were installed;

• the tests passed;

• the CLI completed a forecast or backtest;

• the reported $646 gold value or $215,487 Bitcoin value was reproduced; or

• the paper's benchmark metrics or fee-sensitivity curves were confirmed.

The correct interpretation is narrower: the project contains intended implementations, tests, and verification rules, while the local static report identifies both successful structural checks and unresolved findings.

18.6 A practical review order

When repairing or extending the project, review the layers in this order:

Resolve format and placeholder findings. Inspect experiments.py and cli.py; decide whether each detected placeholder is intentional or incomplete. Treat the Markdown-fence findings as representation warnings where fences are normal Markdown syntax.

Reconcile public APIs. Compare function signatures and dataclass fields across source modules and tests. In particular, check the experiment-to-data, strategy-to-risk, and test-to-sizer interfaces.

Run syntax and import checks in the target environment. Confirm that core imports do not require optional PyTorch or benchmark packages, and that optional failures are explicit.

Review data boundaries. Hand-check window endpoints, target timestamps, scaler fitting, and chronological splits.

Review tensor shapes. Confirm attention normalization across time and forecast-target horizon agreement.

Review pure analytical components. Check return denominators, streak policies, normalized sizing, and VaR sign conventions with small numerical examples.

Review portfolio accounting. Reconcile every fill, fee, cash change, holding quantity, and mark-to-market value.

Only then run local experiments. Preserve configuration, data provenance, software versions, warnings, and result metadata.

Compare with the paper cautiously. Label any mismatch as unresolved unless the original data and protocol are available.

This order moves from inexpensive structural checks to more demanding semantic and empirical work. It also prevents a striking backtest number from distracting attention from basic timestamp or accounting errors.

18.7 Reproduction status remains a separate question

The reproduction matrix in docs/reproduction_matrix.md is the appropriate place to track paper fidelity. It distinguishes direct mechanics—such as temporal attention equations, window construction, and portfolio identities—from reconstructions such as normalized exponential sizing, historical VaR, and greedy action selection.

A useful final status summary is:

The purpose of this verification process is therefore not to manufacture certainty. It is to make uncertainty inspectable: readers can see which files parse, which invariants are intended, which findings remain unresolved, and which paper claims cannot yet be tested. That is the appropriate standard for an educational implementation of an incompletely specified quantitative-trading paper.

19. Worked Synthetic Example from Price Path to Portfolio Record

This section connects the data, streak, sizing, risk, strategy, and accounting layers with a short synthetic price path:

prices: 100, 102, 101, 99, 100, 103

The example follows this causal sequence:

prices
  → standard returns and streaks
  → manually supplied forecast
  → finite position-size schedule
  → VaR decision
  → greedy candidate selection
  → next-bar fill
  → portfolio valuation

This is an illustrative interface example, not an empirical backtest. The prices are synthetic and the forecast is supplied manually rather than produced by a trained model. The paper's exact position-sizing equations, VaR procedure, and greedy policy are unavailable, so those components remain documented reconstructions. The code was not executed here; the calculations below describe the intended contracts and hand-checkable accounting.

19.1 Prepare the synthetic prices

The data layer accepts local historical data. For a small example, create a pandas Series and convert it into the standard timestamp-and-price table:

import pandas as pd

prices = pd.Series(
    [100.0, 102.0, 101.0, 99.0, 100.0, 103.0],
    index=pd.date_range("2024-01-01", periods=6, freq="D"),
    name="price",
)

from quantitative_trading_model.data import standardize_prices

clean = standardize_prices(
    prices.rename_axis("timestamp").reset_index(),
    timestamp_column="timestamp",
    price_column="price",
    asset="synthetic",
)

standardize_prices parses timestamps, sorts observations, removes invalid rows when configured to do so, resolves duplicate timestamps, and returns a PriceData object. Prices must be finite and strictly positive. The function does not identify a market-data source or make network requests.

Six observations are far shorter than the paper-inspired 100-observation forecasting window. This path is therefore unsuitable for training the Att-BiLSTM. It is only long enough to demonstrate return alignment, decision timing, and portfolio accounting.

19.2 Compute standard returns

The default reconstruction uses the conventional simple return:

\[ rt = \frac{pt-p{t-1}}{p{t-1}}. \]

from quantitative_trading_model.streaks import compute_returns

returns = compute_returns(clean.prices, denominator="previous")

The hand calculations are:

Each return is aligned to the later price. Thus, the return from 101 to 99 becomes known at the 99-price observation. It must not be assigned to the earlier 101 timestamp when constructing an online risk history.

The extracted paper equation appears to use the current price as the denominator. That alternative is available through denominator="current" or denominator="paper", but one convention must be used consistently throughout a given experiment.

19.3 Classify movements and identify streaks

The sign sequence is:

+  -  -  +  +

The streak layer separates calculation, classification, and counting:

from quantitative_trading_model.streaks import classify_returns, count_streaks

labels = classify_returns(returns, zero_policy="neutral")

rise_counts = count_streaks(
    labels,
    value=1,
    mode="maximal",
    minimum_length=1,
)
decline_counts = count_streaks(
    labels,
    value=-1,
    mode="maximal",
    minimum_length=1,
)

The maximal runs are:

positive run of length 1
negative run of length 2
positive run of length 2

Conceptually, the results are:

rise_streaks:    {1: 1, 2: 1}
decline_streaks: {2: 1}

count_streaks also supports mode="overlapping". A run of four positive returns contains three overlapping two-return subsequences, whereas maximal mode records one complete run of length four. These statistics are different, and the paper does not specify which one it used.

A complete summary can be requested with:

from quantitative_trading_model.streaks import compute_streak_statistics

stats = compute_streak_statistics(
    clean.prices,
    denominator="previous",
    zero_policy="neutral",
    streak_mode="maximal",
)

print(stats.gain_percentile_90)
print(stats.decline_percentile_10)
print(stats.rise_streaks)
print(stats.decline_streaks)

The summary exposes the 90th-percentile gain, signed 10th-percentile decline, median gain, and decline magnitudes. With only five returns, these estimates are not statistically meaningful. In a real walk-forward experiment, the statistics must be calculated from observations available before the relevant decision.

This implementation calls the procedure streak analysis or sequential-pattern analysis, not standard Apriori. The paper does not provide Apriori itemsets, support, or confidence calculations.

19.4 Build and consume a finite sizing level

The paper's equations (5)–(8) are incomplete in the extracted source. The implementation therefore uses a normalized exponential reconstruction. Suppose the schedule has a $300 budget and three addition levels:

from quantitative_trading_model.sizing import (
    ExponentialPositionSizer,
    build_exponential_schedule,
)

schedule = build_exponential_schedule(
    budget=300.0,
    max_additions=3,
    growth=0.50,
    amplitude=1.0,
)
sizer = ExponentialPositionSizer(schedule=schedule)

print(schedule.weights)
print(schedule.allocations)

The reconstructed scores have the form:

si=Aexp⁡(Bi),

followed by normalization:

\[ wi=\frac{si}{\sumj sj}, \qquad \operatorname{allocation}i=300wi. \]

For indices i = 0, 1, 2, A = 1, and B = 0.5, the scores are approximately:

1.0000, 1.6487, 2.7183

The resulting planned allocations are approximately:

level 1: $55.89
level 2: $92.16
level 3: $151.95

The first schedule level is not merely described as selected: it is connected to the strategy through ExponentialPositionSizer. The strategy uses one-based addition levels in its public action record, while the schedule stores zero-based Python indices internally. After the action is approved, the selected level is consumed exactly once:

allocation = sizer.allocate_next_addition(
    available_cash=500.0,
    current_exposure=0.0,
)

print(allocation.level)          # first internal level is 0
print(allocation.approved_amount)

The allocation cannot exceed available cash or the configured exposure capacity. This is a finite schedule, not unlimited averaging down. The formula and any calibration from gains, declines, or fees are reconstructions rather than faithful transcriptions of the paper's unreadable recursive break-even equations.

19.5 Produce a forecast-derived signal

Assume that when the observed price is 99, a forecasting model predicts a one-step price of 101. The forecast is manually supplied for this example:

rforecast=10199−1≈0.020202.

That is an expected increase of about 2.02%.

from quantitative_trading_model.strategy import SignalGenerator

signal_generator = SignalGenerator(
    buy_threshold=0.01,
    sell_threshold=0.01,
    aggregation="terminal",
)

decision = signal_generator.decide(
    current_price=99.0,
    forecasts=[101.0],
)

print(decision.signal)
print(decision.expected_return)

The expected increase exceeds the 1% buy threshold, so the intended signal is Signal.BUY. SignalGenerator.decide uses only the current price and forecast. It does not receive the later realized prices of 100 or 103.

For a three-day forecast, the same class can aggregate values such as [100.0, 101.0, 102.0]. The default terminal policy uses the final forecast; mean and max are also available. The paper mentions three-day forecasts but does not define this aggregation rule, so it remains an explicit reconstruction choice.

19.6 Apply historical VaR before ranking actions

The paper names VaR but does not specify its confidence level, horizon, lookback, or action rule. This example uses a rolling historical estimator with enough pre-decision observations to produce an estimate:

from quantitative_trading_model.config import VaRConfig
from quantitative_trading_model.risk import HistoricalVaR, VaRTradeFilter

var_config = VaRConfig(
    enabled=True,
    confidence=0.95,
    horizon=1,
    lookback=100,
    min_observations=3,
    action="scale",
    insufficient_history="allow",
    max_var_fraction=0.05,
)

var_estimator = HistoricalVaR(var_config)
var_filter = VaRTradeFilter(config=var_config)

At the 99-price decision point, the returns known from the path are the first three values:

pre_decision_returns = returns[:3]
estimate = var_estimator.estimate(pre_decision_returns)
print(estimate.var)
print(estimate.return_quantile)

The estimator now has three observations, matching the configured minimum. This is still only a toy quantile estimate; it is not a meaningful risk model. If the lower-tail return quantile were -1.8%, an exposure of $55.89 would imply an estimated loss of roughly:

55.89×0.018≈$1.01.

The actual estimate should be read from estimate, not inferred from this hypothetical number.

The risk decision uses currency notional units:

risk_decision = var_filter.decide(
    proposed_quantity=55.89,
    current_exposure=0.0,
    portfolio_value=500.0,
    returns=pre_decision_returns,
)

print(risk_decision.action)
print(risk_decision.approved_quantity)
print(risk_decision.reason)

Here, proposed_quantity is the VaR module's name for a proposed currency notional. It is not asset quantity. A positive approved amount means the risk filter accepted or scaled the proposed notional; a blocked trade has an approved amount of zero. The filter uses only the supplied pre-decision returns.

19.7 Rank the forecast and sizing candidates

The sizing schedule and risk decision are now connected explicitly. The risk-approved amount is passed as the maximum buy notional, while the sizer supplies the next addition level:

from quantitative_trading_model.strategy import GreedyPolicy, PortfolioSnapshot

approved_notional = float(risk_decision.approved_quantity)

snapshot = PortfolioSnapshot(
    cash=500.0,
    holdings=0.0,
    price=99.0,
    addition_level=0,
    portfolio_value=500.0,
)

policy = GreedyPolicy(fee_rate=0.002)
selected = policy.choose(
    snapshot,
    forecasts=[101.0],
    signal_generator=signal_generator,
    sizer=sizer,
    max_additions=3,
    buy_notional=approved_notional,
    returns=pre_decision_returns,
)

print(selected.signal)
print(selected.notional)
print(selected.addition_level)

This call does not pass VaRTradeFilter into GreedyPolicy. The VaR decision has already been applied directly with the compatible VaRTradeFilter.decide interface. That explicit separation avoids confusing a currency-notional risk result with an asset-unit order.

The candidate set conceptually contains:

hold:  score 0
buy:   risk-approved notional, using current cash
add:   next unused finite sizing level
sell:  unavailable because holdings are zero

The reconstructed greedy score for a buy or add action is:

score(a)=N(rforecast−f),

where N is currency notional, r_forecast is the predicted return, and f is the fee rate. If N = 55.89, r_forecast ≈ 0.020202, and f = 0.002, then:

score≈55.89(0.020202−0.002)≈1.02.

The hold candidate has score zero, so a positive approved buy or add candidate can outrank it. The actual selected amount must still be read from selected.notional; the paper does not define its own greedy objective or candidate set.

The GreedyPolicy call uses forecast information, current portfolio state, fees, and the already-approved risk amount. It does not use the later realized 99-to-100 return. That restriction prevents look-ahead bias.

19.8 Consume the selected schedule level

The strategy's selected addition level is an action description. The sizing schedule should consume its corresponding level only after the action has been accepted for execution:

if selected.signal.value == "add" and selected.addition_level is not None:
    allocation = sizer.allocate_next_addition(
        available_cash=snapshot.cash,
        current_exposure=snapshot.holdings * snapshot.price,
        level=selected.addition_level - 1,
    )
    print(allocation.approved_amount)

The subtraction converts the strategy's one-based public level to the sizer's zero-based schedule index. In a production orchestration layer, the approved amount returned by the sizer and the amount sent to the order should be compared explicitly. If risk scaling or cash caps reduce the amount, the order must use the reduced approved notional rather than the original planned allocation.

19.9 Execute on the next bar and calculate portfolio value

The signal is formed at the 99-price observation. The next bar has price 100, so the order executes at 100 rather than at the signal price of 99.

The example uses these units:

Assume the approved notional is $55.89, execution price is $100, the fee is 0.002, and slippage is zero. The hand calculation is:

q=55.89100=0.5589 units,

fee=55.89×0.002=$0.11178,

cash after buy=500−55.89−0.11178=$443.99822.

Immediately after the fill, the marked portfolio value is:

V=443.99822+0.5589×100=$499.88822.

The difference from $500 is the purchase fee.

The corresponding accounting objects are:

from quantitative_trading_model.backtest import Order, PortfolioSimulator

simulator = PortfolioSimulator(
    initial_cash=500.0,
    fee_rate=0.002,
    slippage_rate=0.0,
    allow_short=False,
    allow_leverage=False,
)

order = Order(
    side="buy",
    notional=55.89,
    signal_timestamp="2024-01-04",   # the bar priced at 99
    execution_timestamp="2024-01-05",
    addition_level=1,
    reason="forecast recovery signal",
)

fill = simulator.execute(order, "2024-01-05", 100.0)
record = simulator.mark_to_market("2024-01-05", 100.0)

print(fill.notional, fill.fee)
print(record.portfolio_value)

Order.notional is deliberately used here because the strategy and VaR layers work in currency-notional terms. The simulator converts that notional into asset units at the execution price. If an order instead supplies Order.quantity, the simulator treats it as asset units directly.

PortfolioSimulator.execute applies configured slippage, calculates executed notional and fees, updates cash and holdings, and records a Fill. mark_to_market creates a BacktestRecord satisfying:

\[ Vt=\text{cash}t+\text{quantity}t pt. \]

The default simulator is long-only and non-leveraged. It clips an unaffordable buy to the quantity affordable with current cash; the current generated execution path does not implement a separate rejection branch for the reject_insufficient_cash configuration field. That behavior should therefore not be described as configurable rejection without a corresponding backtest-code change.

If the price later reaches 103, the marked value before selling is approximately:

443.99822+0.5589×103≈$501.675.

A later sale would incur a second fee under the default two-sided fee convention. The final realized value would depend on the sale price, slippage, and whether end-of-test liquidation is enabled.

19.10 What the example demonstrates

The example makes the following contracts visible:

• returns are aligned with the later price observation;

• streaks are calculated from the selected return convention;

• the sizing schedule is finite, normalized, and connected to the action level;

• VaR is applied before the action is ranked;

• risk and strategy values are currency notionals, while backtest quantities are asset units;

• the greedy score uses forecasts and estimated fees, not realized future returns;

• the signal at price 99 executes at the later price of 100;

• the purchase fee is deducted exactly once;

• portfolio value equals cash plus marked holdings.

It does not demonstrate profitability. The forecast is manual, the price path is synthetic, and the paper's exact sizing, VaR, and greedy methods are unavailable. The example is a causal and accounting demonstration, not a reproduction of the paper.

19.11 Relation to the paper's reported outcomes

The paper reports approximately $646 from a $500 gold allocation and approximately $215,487 from a $500 Bitcoin allocation. Neither result can be inferred from this six-price example. Reproducing either claim would require the original datasets, dates, target definition, complete position-sizing equations, VaR settings, greedy rules, fee treatment, execution timing, and liquidation policy.

The example is useful because it makes causality, units, and accounting explicit. It should not be used as evidence about expected returns, investment quality, or the validity of the paper's headline performance.

20. Limitations, Interpretation, and What a True Reproduction Would Need

The implementation now expresses the paper’s main ideas as separate, inspectable components: an attention-enhanced BiLSTM for forecasting, benchmark forecasters, streak analysis, a normalized exponential position-sizing reconstruction, historical VaR, a reconstructed greedy policy, portfolio accounting, and transaction-fee sensitivity experiments. That modular structure is useful, but it should not be mistaken for proof that the paper’s numerical results or trading claims have been reproduced.

The central limitation is not Python syntax. It is missing experimental information. A genuine reproduction requires the same observations, preprocessing, model settings, decision rules, execution assumptions, and evaluation dates. Several of those details are absent or corrupted in the source material. The generated project therefore distinguishes what is implemented mechanically from what is reconstructed or merely reported as a claim.

20.1 Interpretation labels

Use the following vocabulary when describing results from this project:

| Label | Meaning | | --- | --- | | Implemented | The paper describes enough behavior to encode a concrete mechanism, such as a sliding window or dot-product attention operation. | | Reconstructed | The paper names a method but omits equations, parameters, or operational rules, so the project supplies a documented interpretation. | | Illustrative | The result demonstrates an interface, invariant, or workflow using synthetic or user-provided local data. It is not evidence of paper reproduction. | | Unavailable | The paper does not provide enough information or data to validate the method or number. | | Reported claim | A value or conclusion stated by the paper and preserved for comparison, but not independently verified. |

This distinction applies across all major components:

• Att-BiLSTM: the broad sequence-model and attention design is implemented, but hidden dimensions, layer count, query construction, target definition, and training schedule are reconstructed choices.

• Benchmarks: recurrent, statistical, HMM, and boosted-tree adapters are available, but comparable numerical results require the original data and exact configurations.

• Position sizing: the finite normalized exponential schedule is a reconstruction because equations (5)–(8) cannot be recovered reliably.

• VaR: the rolling historical estimator and trade filter are operational reconstructions because the paper gives no confidence level, horizon, lookback, or action rule.

• Greedy selection: the fee-aware candidate-ranking policy is reconstructed because the paper does not define its candidates, objective, constraints, or tie-breaking.

• Backtesting: cash, holdings, fees, slippage, next-bar execution, and liquidation are explicit implementation assumptions, not verified details of the original backtest.

• Fee sensitivity: the reported fee grids are preserved, but the figures’ numeric outputs are unavailable.

The status table in docs/reproduction_matrix.md is the project’s main audit record. It maps each paper claim to its implementation location, validation strength, and missing information. It should be updated whenever a source equation, dataset, or protocol detail becomes available.

20.2 Why forecast accuracy does not establish trading profitability

A forecast metric measures the distance between a prediction and its target. A trading result depends on a much longer chain:

observed history
    -> forecast
    -> signal threshold
    -> position size
    -> risk filter
    -> order timing
    -> execution price
    -> fees and slippage
    -> holdings and cash
    -> portfolio value

An Att-BiLSTM can have a lower RMSE than a benchmark and still fail to produce a profitable strategy. For example:

• the predicted movement may be smaller than the round-trip transaction cost;

• a small timing error can turn a buy into a loss;

• a position-sizing rule can amplify an incorrect forecast;

• the forecast may be evaluated at a different timestamp from the order execution;

• slippage, spread, liquidity, and partial fills can change the economics;

• the asset’s price process may change after the historical training period.

This is why metrics.py keeps forecast metrics separate from portfolio metrics. RMSE, MAE, MAPE, and R-squared should be reported alongside, but not confused with, cumulative return, maximum drawdown, volatility, Sharpe ratio, turnover, and total fees. A buy-and-hold benchmark is also necessary: a strategy’s final value has little meaning without knowing what passive ownership of the same asset and period would have produced.

20.3 Financial and market limitations

Historical financial data are not a stationary laboratory process. The relationship between past prices and later prices can change because of market structure, liquidity, regulation, macroeconomic conditions, participant behavior, or technological changes. A model that performs well in one period may not generalize to another.

The backtest also simplifies real execution. The generated backtest.py simulator supports explicit fees, optional proportional slippage, fractional units, exposure limits, and next-bar execution, but it does not model every market constraint. In particular, it does not provide a full treatment of:

• bid-ask spreads and changing spreads;

• market impact and order-book depth;

• liquidity limits or volume participation;

• partial fills and order cancellation;

• latency and stale signals;

• exchange outages or data revisions;

• custody, borrowing, margin, taxes, or funding costs;

• instrument-specific trading calendars;

• corporate actions or contract rolls where relevant.

These omissions matter especially for a strategy that repeatedly adds to positions. Transaction costs are not a cosmetic adjustment: they can change whether an action is feasible, whether a greedy candidate has a positive score, and whether the entire position schedule remains solvent.

20.4 Why the large Bitcoin result should not be generalized

The paper reports an approximate final value of $215,487 from a $500 Bitcoin allocation, alongside an approximate $646 value from a $500 gold allocation. Those values are reported claims, not reproduced outputs of this project.

The Bitcoin number may depend on an unusually favorable historical interval, aggressive reinvestment, repeated position additions, leverage-like exposure, unrealistic fills, or an accounting convention that is not described. Without the exact dates, instrument definition, price series, signals, sizing parameters, transaction rules, and liquidation policy, the number cannot be separated into market-period effects and strategy effects.

It should therefore not be presented as expected future performance, evidence of general profitability, or a target that a synthetic demonstration ought to match. The same caution applies to the reported combined value of approximately $216,133 from $1,000 and to the paper’s benchmark forecast metrics.

20.5 What a true empirical reproduction would require

To replace the current reconstructions with verified implementations, obtain and preserve the following artifacts.

Data and timestamps

The original gold and Bitcoin files.

The precise instrument or index represented by each file.

Currency, timezone, sampling frequency, and date range.

The meaning of each price column: close, adjusted close, settlement, or another field.

Rules for missing observations, duplicates, invalid prices, and corporate or market-event adjustments.

The exact train, validation, and test boundaries.

Evidence that scaling and feature construction were fitted without using future observations.

The data layer in data.py can then be adapted to the original schema rather than relying on generic local CSV assumptions.

Forecast target and Att-BiLSTM architecture

Whether the model predicts raw prices, normalized prices, returns, or percentage changes.

Whether the forecast horizon is one day, three days, or a sequence of separately generated one-step predictions.

The number of LSTM layers and hidden units.

Whether the bidirectional outputs are concatenated, averaged, or projected.

The exact construction of the attention query q.

Dense layers, output activation, initialization, and regularization.

Learning rate, batch size, epoch count, stopping rule, random seed, and optimizer options.

The exact feature set, including whether volume or technical variables were used.

The current models/attention.py and models/forecasters.py implement the visible mathematics and a learned-query interpretation, but they cannot infer these missing architectural details.

Benchmark configurations

For each LSTM, GRU, BiLSTM, Holt-Winters, ARIMA, HMM, and XGBoost model, obtain the input representation, hyperparameters, tuning procedure, forecast horizon, and test-time procedure. This is particularly important for HMM and XGBoost:

• Viterbi decoding identifies latent states; it does not by itself define a future price forecast.

• XGBoost is a gradient-boosted tree method, not a random forest.

• A fair benchmark comparison requires aligned targets, timestamps, preprocessing, and untouched test data.

Without those details, a locally computed benchmark table can be informative, but it is not a verified reconstruction of the paper’s table.

Position management, VaR, and greedy policy

The original source equations or high-resolution figures are required for equations (5)–(8). In addition, clarify:

• whether position variables represent cash, asset units, or notional exposure;

• how the recovery percentile and decline percentile enter each recursive equation;

• when an additional position is triggered;

• the maximum number of additions and any leverage rules;

• whether fees apply to both sides of each transaction;

• the VaR confidence level, horizon, lookback, distribution, and aggregation method;

• whether VaR blocks trades, scales them, or only reports risk;

• the greedy algorithm’s candidate actions, objective, constraints, and tie-breaking.

Until those materials are supplied, sizing.py, risk.py, and strategy.py should be described as explicit, auditable reconstructions rather than faithful source translations.

Backtest and fee-sensitivity protocol

A verified portfolio comparison also needs the backtest start and end dates, initial holdings, cash allocation, signal timestamp, execution timestamp, order type, partial-unit policy, fee convention, spread, slippage, liquidation rule, and reinvestment behavior. For fee sensitivity, the numeric data behind the paper’s figures are required; the fee grid alone does not determine the curve.

The experiment layer can hold non-fee assumptions fixed and rerun a supplied backtest callback, but it cannot infer the original callback from the paper. A sensitivity table generated locally should therefore be labeled as a new experiment under documented assumptions.

20.6 Educational research code versus deployable trading software

This project is intentionally an offline educational framework. It demonstrates how to make data boundaries, model contracts, risk rules, execution timing, and accounting explicit. It is not a deployable trading system.

A production system would require substantially more work, including independent data validation, secure configuration and credential handling, exchange or broker integration, order-state reconciliation, monitoring, failure recovery, compliance review, capacity analysis, and extensive operational testing. None of those capabilities should be inferred from the presence of a backtest simulator or a command-line interface.

The safest use of the generated project is as a research scaffold:

inspect the assumptions;

replace synthetic data with identified local data;

preserve chronological boundaries;

validate each component independently;

record configuration and provenance;

compare against passive and simpler baselines;

treat every result as conditional on its protocol.

20.7 Verification confidence and unresolved project findings

The available verification evidence is limited. Static checks parsed the generated Python files and inspected basic structure, but the overall static stage did not pass: experiments.py and cli.py triggered placeholder detection, while the generated README.md and docs/tutorial.md fields were flagged because they contain Markdown fences. These findings are recorded issues, not proof that the underlying ideas are correct or incorrect.

Semantic code verification was explicitly skipped. Consequently, this article must not claim that the generated code was executed, that the test suite passed, or that the model trained successfully. The tests and modules document intended interfaces and invariants, but unresolved API inconsistencies may remain—for example, between some generated tests, orchestration calls, and implementation signatures.

This is an important distinction:

• Static verification asks whether source files parse and meet structural checks.

• Semantic review asks whether functions, types, formulas, and data flow agree.

• Runtime testing would execute the code on controlled fixtures.

• Empirical reproduction would compare results with the paper using the original data and protocol.

Only the first category was partially recorded here, and semantic verification was skipped. None of these checks, even if fully successful, would establish that the paper’s financial claims are valid.

20.8 Final interpretation policy

When presenting future results, use precise language:

• Say “the implementation uses” for explicit code behavior.

• Say “we reconstruct” when interpreting incomplete equations or policies.

• Say “the synthetic example illustrates” for generated demonstrations.

• Say “the paper reports” for unverified numerical claims.

• Say “locally measured under this protocol” for results obtained on identified local data.

• Do not say “reproduced” unless the data, dates, preprocessing, model, decision rules, execution protocol, and evaluation outputs have been independently matched.

The project’s strongest contribution is therefore methodological rather than evidentiary. It shows how to translate an incomplete quantitative-trading description into modular Python components while preserving temporal causality, finite allocation, explicit risk assumptions, transaction-cost accounting, and honest uncertainty. A stronger scientific conclusion must wait for the original data and missing protocol details.

Complete Generated Code Files

Use the button below to download the source code:

Read more

A single prompt is often enough when the task is simple and self-contained. If you need a brief answer, a rewrite, or a one-off classification, prompt design may be all you need. The limit appears when the same task must be repeated, routed, validated, or combined with other steps. At that point, the question is no longer only how to get a better response from the model. It is how to build a reusable process around the model so the result is consistent each time.

📘 Buy the Entire Book – Available on Amazon Worldwide (If you are paid subscriber, use the url at the end of this article to download the book for free!)

You can now purchase the complete book directly from Amazon in your country:

• United States (US)

• United Kingdom (UK)

• Germany (DE)

• France (FR)

• Spain (ES)

• Italy (IT)

• Netherlands (NL)

• Japan (JP)

• Brazil (BR):

• Canada (CA)

• Mexico (MX)

• Australia (AU)

• India (IN)

That shift matters because LLM applications become systems, not just prompts. A support workflow, for example, may need a ticket summary, a tone check, a response draft, and a final formatting pass. One prompt can produce each piece, but it does not by itself ensure the order, the handoffs, or the checks that make the workflow dependable. In that sense, prompt engineering is about behavior you ask for, while orchestration is about behavior you design for.

Orchestration as the layer around model calls

Orchestration is the control layer that sits around generation. It decides how inputs move through the application, when model calls happen, what happens to intermediate output, and which modules can intervene before the final response is returned. Prompt design shapes the model’s input. Orchestration shapes the workflow that surrounds the model.

A useful metaphor is a recipe versus an assembly line. A prompt can tell the model what to make, but orchestration determines how the work is sequenced, checked, and assembled. That is why orchestration improves quality, repeatability, and structure in real applications: it makes the process explicit instead of leaving every response to a single pass.

LangChain as the organizing framework

LangChain is the framework used in this chapter to keep that system view organized. It is not required for every LLM application, but it gives the chapter a shared language for connecting model I/O, templates, chains, memory, and agents. Thinking in that framework makes it easier to see which part of the application is responsible for input shaping, which part manages workflow, and which part handles longer-lived context or tool use.

This is also where the distinction between prompt engineering and orchestration becomes useful. Prompt engineering improves the wording of a request. Orchestration arranges the reusable structure around that request. LangChain helps separate those concerns so the application does not collapse into one oversized prompt that is hard to understand or maintain.

Chains, memory, and agents in the workflow stack

Chains are the most predictable orchestration pattern. They connect model calls and other modules into a known sequence, which makes them well suited to workflows with clear steps. Memory extends the interaction across turns so earlier context can remain available. Agents add flexibility by letting the system choose actions at runtime when the next step is not fully known in advance.

A simple decision rule follows from that spectrum. Use a prompt when one model call is enough. Use a chain when the steps are predictable. Use an agent when the next action depends on what happens at runtime. Chains give control, agents give flexibility, and memory supports continuity when the conversation itself carries important context.

How orchestration improves real systems

The practical value of orchestration shows up in maintainability and repeatability. A workflow built from explicit steps is easier to test, easier to revise, and easier to extend than a single prompt that tries to do everything at once. It is also easier to improve output quality because validation, routing, and enrichment can happen before the final generation step instead of being left to chance.

That is why this chapter moves beyond prompt engineering without abandoning it. Prompt design still matters, but the larger goal is to design the system around the model. LangChain provides one practical way to do that, and the patterns introduced here will prepare you for memory and agents later in the chapter. Retrieval will come later as a complement to orchestration by supplying external context, but the foundation is the same: structure the workflow so the model is part of a reliable application, not an isolated call.

What to expect next in the chapter

The next sections move from this motivation into the mechanics of model loading, prompt templates, chains, memory, and agents, with retrieval reserved for the following chapter.

Loading local and hosted models for chain-based generation

Start with the deployment decision: local vs hosted

Before a prompt ever reaches a chain, the model itself has to be available somewhere. In practice, that means choosing between local inference and a hosted API, and the choice affects far more than convenience. A local model gives you control over data, predictable access, and the ability to experiment offline, but it asks for hardware, model files, and a little more patience during setup. A hosted model removes most of that friction, but shifts the trade-off toward network dependence, usage limits, and external cost. For this chapter, the local path is useful because it makes the mechanics visible, yet the hosted fallback keeps the workflow accessible on a laptop or in a constrained environment.

A good rule of thumb is simple. If you are iterating on prompt structure, chain design, or message formatting, a hosted model is often the fastest way to get moving. If you want to inspect deployment constraints, work offline, or understand how much model size and context budget shape behavior, a local model is worth the extra setup. The important point is that the orchestration layer should stay conceptually stable even when the backend changes; only the engine underneath needs to swap.

What quantization changes in practice

The local model in this chapter uses a quantized GGUF file. Quantization compresses the model weights by storing them with fewer bits than a full-precision model would use. You can think of it as packing a suitcase more tightly: the contents are still there, but they have been folded into a smaller space so the model can fit on more modest hardware. That smaller footprint usually means lower VRAM usage and often faster inference, especially on consumer machines. The cost is that the model no longer preserves every detail of the original parameters, so a little fidelity is traded away for practicality.

That trade-off is exactly why quantized models matter. A large model in full precision may be inconvenient or impossible to run locally, while a compressed variant can become usable without changing the surrounding application logic. The same idea underlies many modern local deployment choices: reduce precision enough to make the model portable, but not so much that the quality drops below what the task needs.

Fetch and load the local GGUF model in LangChain

The example in this chapter uses a GGUF artifact downloaded from Hugging Face. The specific file matters because the precision variant is part of the contract between the model and your hardware. Choosing a different variant changes memory use, speed, and sometimes output quality, so the file name is not just a label; it is part of the deployment decision.

!wget https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/Phi-3-mini-4k-instruct-fp16.gguf

Once the file is available locally, LangChain can wrap it through LlamaCpp. This is the local model interface for the chapter, and the parameters you pass here shape the generation behavior before any chain is built on top.

from langchain import LlamaCpp

# Make sure the model path is correct for your system!
llm = LlamaCpp(
    model_path="Phi-3-mini-4k-instruct-fp16.gguf",
    n_gpu_layers=-1,
    max_tokens=500,
    n_ctx=2048,
    seed=42,
    verbose=False
)

The practical meaning of those settings is straightforward. model_path points to the GGUF file you downloaded. n_gpu_layers controls how much of the model is offloaded to the GPU when that is available. max_tokens caps the length of the generated answer, which is part of your token budget. n_ctx sets the context window the model can condition on, and that matters because every extra instruction, conversation turn, or retrieved snippet has to fit alongside the answer you hope to generate. seed helps make experiments repeatable, and verbose keeps the output quieter while you are testing.

That context limit is easy to underestimate. Think of it as a suitcase that must hold both the prompt and the response. If you keep stuffing history into the prompt, you leave less room for the answer, and eventually the model has to truncate, ignore, or compress something. This is why context size becomes a design constraint later in the chapter when chains and memory start accumulating text.

Why the raw model call can fail without the right prompt shape

Once the model is loaded, it is tempting to call it directly and assume that any text input will work. With some instruct or chat-tuned models, that assumption is wrong. The model may be running correctly, but the request still fails because the input shape does not match the format it expects. That is less like a runtime crash and more like sending a valid HTTP request to the wrong endpoint schema.

llm.invoke("Hi! My name is Maarten. What is 1 + 1?")

''

The empty result is the important lesson. The model is not broken; the prompt contract is incomplete. Some models need a specific instruction wrapper, a chat template, or other formatting cues before they respond sensibly. In other words, prompt formatting is not an optional refinement layered on top of the model. For certain backends, it is part of the interface definition.

That distinction matters for the rest of the chapter. If the request format is unstable, every downstream chain inherits that fragility. Once you know the model’s expected shape, you can stop hand-assembling prompts and move toward reusable templates that enforce the right structure every time.

Use a hosted chat model when local inference is not practical

The hosted fallback follows the same overall pattern but uses an API-backed chat model instead of a local GGUF file. Operationally, that changes where the computation happens and what constraints matter most. You no longer manage model files or local GPU placement, but you do need credentials, network access, and a model service that fits your use case.

from langchain.chat_models import ChatOpenAI

# Create a chat-based LLM
chat_model = ChatOpenAI(openai_api_key="MY_KEY")

This is the same interface idea in a different engine. The point is not that one path is universally better than the other, but that the surrounding generation workflow can remain portable when the model-loading layer is abstracted cleanly. That portability is exactly why the chapter moves next from model invocation to chains: once the backend is in place, the real productivity gain comes from composing prompts, inputs, and outputs into repeatable workflows rather than hand-calling the model one request at a time.

Why templates beat one-off prompts

A one-off prompt is fine when you are experimenting, but it becomes brittle the moment the same task needs to run again with different inputs. In practice, the useful unit is not a single prompt string; it is a prompt template, which is a stable prompt frame with named slots for the parts that change. That distinction matters because the reusable part is the workflow contract, while the variable part is just the data you feed into it.

For the ReviewDesk workflow, that difference is easy to see. A support summary might be rewritten into a clean answer draft today, then reused tomorrow for a different ticket, a different tone, or a different product line. If the structure of the request stays mostly the same, a template gives you consistency without forcing you to copy and edit prose by hand. If the task itself needs to be split into distinct decisions, such as naming a title before drafting a response, then the problem has outgrown a single prompt and is ready for chaining.

Defining variables and model-specific prompt format

A prompt template is useful because it makes the changing parts explicit. Instead of hiding the input inside a long string, you name the fields that the chain expects. That turns prompt writing into a small interface design exercise: the template defines what information must be supplied, and the model receives the formatted text that results from filling those slots.

Some models also care about exact formatting. A chat-oriented model may expect role markers, delimiters, or other special tokens around the user and assistant turns. In that case, the template is not only about wording; it is also about preserving the structure the model was trained to recognize. If that structure is wrong, the model can still produce text, but the chain may become less reliable because the model is being asked to read a format it does not expect.

from langchain import PromptTemplate

# Create a prompt template with the "input_prompt" variable
template = """<s><|user|>
{input_prompt}<|end|>
<|assistant|>"""
prompt = PromptTemplate(
    template=template,
    input_variables=["input_prompt"]
)

Wiring a prompt directly into an LLM

Once the prompt is packaged as a template, the next step is to connect it to the model. LangChain’s pipe syntax makes that composition explicit: the prompt formats the input, and the model consumes the formatted result. Conceptually, this is the smallest useful chain. It is still just one step, but the step is now reusable and named instead of being a hand-written string glued to a model call.

The invocation also becomes clearer. You do not send the model a free-form blob of text; you provide a mapping that matches the template’s variable names. That small discipline pays off quickly, because it makes the call site readable and reduces the chance of silently passing the wrong value into the prompt.

basic_chain = prompt | llm

# Use the chain
basic_chain.invoke(
    {
        "input_prompt": "Hi! My name is Maarten. What is 1 + 1?",
    }
)

Reusing the same pattern for different jobs

The real value of templates shows up when the same structure is reused for a different task. A business-name generator, a ticket summarizer, and a response drafter can all follow the same pattern even though they solve different problems. The only thing that changes is the variable name and the wording around it. That is the hallmark of a good template: when the nouns change but the workflow stays stable, you have separated the prompt’s public interface from its implementation.

# Create a Chain that creates our business' name
template = "Create a funny name for a business that sells {product}."
name_prompt = PromptTemplate(
    template=template,
    input_variables=["product"]
)

Breaking a generation task into linked steps

A larger generation task often becomes easier to control when it is decomposed into smaller stages. The ReviewDesk analogue is straightforward. Suppose you have a short ticket summary and want first a concise title, then a character sketch for a response style, and finally a full draft. Each step can focus on one subgoal, which makes the output easier to inspect and easier to correct.

This is where sequential chaining becomes more than syntactic convenience. The title is not just another string; it is a labeled artifact that later prompts can use. The character description is likewise not a final answer but a structured intermediate result. By giving each stage a specific role, you make the hidden subgoals visible, much as a kitchen keeps prep, plating, and finishing separate so each station knows exactly what to hand off.

from langchain import LLMChain

# Create a chain for the title of our story
template = """<s><|user|>
Create a title for a story about {summary}. Only return the title.<|end|>
<|assistant|>"""
title_prompt = PromptTemplate(template=template, input_variables=["summary"])
title = LLMChain(llm=llm, prompt=title_prompt, output_key="title")

title.invoke({"summary": "a girl that lost her mother"})

# Create a chain for the character description using the summary and title
template = """<s><|user|>
Describe the main character of a story about {summary} with the title {title}. Use only two sentences.<|end|>
<|assistant|>"""
character_prompt = PromptTemplate(
    template=template, input_variables=["summary", "title"]
)
character = LLMChain(llm=llm, prompt=character_prompt, output_key="character")

# Create a chain for the story using the summary, title, and character description
template = """<s><|user|>
Create a story about {summary} with the title {title}. The main character is: {character}. Only return the story and it cannot be longer than one paragraph. <|end|>
<|assistant|>"""
story_prompt = PromptTemplate(
    template=template, input_variables=["summary", "title", "character"]
)
story = LLMChain(llm=llm, prompt=story_prompt, output_key="story")

# Combine all three components to create the full chain
llm_chain = title | character | story

llm_chain.invoke("a girl that lost her mother")

Reading, passing, and constraining intermediate outputs

Output keys are the shipping labels of a chain. They tell you what each stage produced and what the next stage should expect. In the example above, the first step stores its result as title, the second as character, and the third as story. That naming is not cosmetic; it is how the pipeline keeps track of which artifact should flow into which prompt.

This is also where many failures start. If a downstream prompt expects {title} but the upstream step saved its result under a different key, the chain breaks or receives the wrong field. The same issue appears when an intermediate output is too verbose. A title prompt that ignores the instruction to return only the title can pollute the next stage with extra text, which is why generation constraints belong inside the prompt rather than as an afterthought.

Choosing between a single prompt and a chain

Use a single prompt when the task is simple enough that one well-shaped request can handle it. Use a chain when the task has genuinely separable substeps, when you want intermediate artifacts you can inspect, or when the same structure will be reused across many inputs. A prompt template is the reusable interface; a chain is the orchestration around that interface. Keeping that boundary clear will matter even more in the next sections, where stateful memory and tool-using agents build on the same idea of explicit handoffs.

Conversation memory and the cost of keeping context

Why a plain chain does not remember yesterday

A plain LLM call is stateless. It answers from whatever text you include in the current prompt, and then the interaction ends. That is easy to miss because the first reply can feel conversational: if you tell the model your name in one turn and ask a related question immediately afterward, the answer may seem context-aware. But that impression can be misleading. The model is not carrying a private notebook between calls; unless you pass the earlier exchange back in, the next invocation starts fresh.

That distinction matters in a support workflow like ReviewDesk. Suppose a user says, “Hi, I’m Maarten, and I need help with a billing issue,” and the assistant responds helpfully. If a second call asks, “What is my name?” a basic chain has no reason to know. The missing ingredient is not a better prompt in the abstract, but a mechanism for carrying forward conversation state. In LangChain, that mechanism is memory.

Wiring chat history into the prompt

Memory does not replace the prompt. It feeds the prompt. The prompt template reserves a slot, often called chat_history or something equivalent, and the memory object fills that slot before the model is called. That separation is important: the chain orchestrates the call, the memory object stores and loads state, and the prompt template defines where that state should appear.

A useful mental model is chat-state compression. The chain is the delivery system, the prompt is the envelope, and memory decides how much of the prior conversation to pack into it. If the envelope has no history field, nothing from earlier turns can be injected, no matter how conversational the wording sounds.

Full-buffer memory as the simplest persistent transcript

The simplest baseline is ConversationBufferMemory. It keeps the full conversation transcript and sends it back into the prompt on each turn. Because nothing is discarded, it is the easiest strategy to reason about. If the user said it earlier, it is still available now. That makes buffer memory a good starting point for short, high-stakes conversations where exact recall matters more than efficiency.

template = """<s><|user|>Current conversation:{chat_history}\n\n{input_prompt}<|end|>\n<|assistant|>"""

prompt = PromptTemplate(
    template=template,
    input_variables=["input_prompt", "chat_history"]
)

memory = ConversationBufferMemory(memory_key="chat_history")

llm_chain = LLMChain(
    prompt=prompt,
    llm=llm,
    memory=memory
)

With that wiring in place, the same two-turn ReviewDesk exchange now behaves like a real conversation. The first invocation stores the user’s name, and the second can recover it because the entire transcript is still attached to the chain.

llm_chain.invoke({"input_prompt": "Hi! My name is Maarten. What is 1 + 1?"})
llm_chain.invoke({"input_prompt": "What is my name?"})

The advantage is clarity. The downside appears quickly in longer dialogs. Every new turn expands the prompt, which increases token usage, slows generation, and raises cost. Buffer memory is faithful, but it is also greedy. Like carrying every receipt in your wallet, it works until the accumulation becomes the problem.

Windowed memory: keep the recent turns, drop the rest

ConversationBufferWindowMemory trims that growth by keeping only the most recent k turns. The idea is a sliding window over the dialogue. Recent context stays available, older context falls away. This is often a better fit when the current exchange depends on the last few messages but not on the entire history.

memory = ConversationBufferWindowMemory(k=2, memory_key="chat_history")

llm_chain = LLMChain(
    prompt=prompt,
    llm=llm,
    memory=memory
)

With a small window, the chain can still answer follow-up questions tied to the recent exchange, but it eventually loses earlier facts. In a ReviewDesk conversation, that can be enough if the assistant only needs to maintain the active thread of one support case. If the user says their name and issue in one or two turns, the model can continue to refer to them correctly. If the discussion drifts long enough, earlier commitments, counts, or names may drop out of scope.

This is the most common trap with too-small windows: the model sounds coherent until it is asked about a detail that has already slid past the edge of the buffer. Windowed memory is therefore a budgeting tool, not a weaker version of buffer memory. It is an explicit decision to value recency over completeness.

Summary memory: compress the conversation instead of replaying it

ConversationSummaryMemory takes a different path. Instead of preserving the full transcript, it keeps a running summary of the interaction. After each turn, the earlier conversation is compressed into a shorter form, and that condensed state is what gets injected later. In effect, the model is asked to remember the story, not the transcript.

summary_prompt_template = """<s><|user|>Summarize the conversations and update with the new lines.\n\nCurrent summary:\n{summary}\n\nnew lines of conversation:\n{new_lines}\n\nNew summary:<|end|>\n<|assistant|>"""
summary_prompt = PromptTemplate(
    input_variables=["new_lines", "summary"],
    template=summary_prompt_template
)

memory = ConversationSummaryMemory(
    llm=llm,
    memory_key="chat_history",
    prompt=summary_prompt
)

llm_chain = LLMChain(
    prompt=prompt,
    llm=llm,
    memory=memory
)

Because the summary is updated incrementally, the chain can stay conversational across longer sessions without replaying everything. That is the main appeal: lower token cost and better scalability. But compression is never free. A summary tends to preserve the gist while softening exact wording, counts, and small commitments. The most subtle failure mode is summary drift, where the condensed history remains plausible but slowly becomes less precise. It is a little like a game of telephone played over many turns: the storyline survives, but the details can blur.

A direct inspection call makes the behavior less mysterious. You can ask the memory object what it currently holds and see that it returns the compressed state rather than the raw transcript.

memory.load_memory_variables({})

That inspection step is useful because it reminds you that memory is just another component in the chain. The chain does not magically “remember”; it asks memory what should be inserted, then the prompt receives that state like any other input variable.

How to choose a memory strategy in practice

The choice among buffer, windowed, and summary memory is a trade-off among retention, token usage, latency, and accuracy. If the conversation is short and every detail matters, full buffer memory is the safest choice. If the assistant only needs the latest turns, a small window keeps the system lean. If the discussion is long-lived and the broad arc matters more than exact phrasing, summary memory is usually the most efficient option.

One useful rule is to think in terms of continuity versus knowledge. Memory is for continuity of the conversation, not for discovering facts outside the chat. A support assistant may remember that the user already described a billing issue, but it should not be treated as a database of policy documents or past tickets. When the task shifts from remembering the conversation to finding external information, memory is no longer enough on its own. That is the point where retrieval becomes the right tool, not a bigger buffer.

Agents, tools, and ReAct-style control flow

A fixed chain is like an assembly line. The same stations run in the same order every time, which is ideal when the task is predictable. An agent is different. It behaves more like a dispatcher that decides which specialist to call next, then uses that result to choose the following step. The change is not just that the agent is smarter. The change is that control flow is no longer predetermined. A chain follows a route; an agent selects the route as it goes.

That matters when the answer depends on capabilities outside the model itself. If the task is stable and self-contained, a chain is usually simpler and safer. If the task needs current facts, a calculation, or a choice among several external actions, an agent becomes useful because it can orchestrate those steps dynamically. The model is still just generating text, but the runtime lets it decide when to search, when to calculate, and when to stop.

What counts as a tool, and why tools change the game

A tool is an external function that the agent can call through a defined interface. It is not a model weight, and it is not hidden reasoning. It is an explicit capability exposed to the agent so the model can use it as part of its plan. Search is one common tool type because it can fetch current or obscure facts. Math is another because it can support deterministic calculation even when the model should not be trusted to carry the arithmetic in its own text output.

This separation is the key idea. The model handles selection and coordination, while the tools handle execution. That makes the system compositional: search can be swapped for another retrieval service, and math can be replaced with a different calculator, without changing the basic control pattern. It also gives a practical heuristic for design. Use a chain when the steps are fixed. Use an agent when the next step depends on what just happened. Add tools when the model needs to reach beyond its own text generation.

ReAct as the loop that binds thinking to acting

ReAct turns that control pattern into a repeating loop of thought, action, and observation. The thought step is the model deciding what to do next. The action step names the tool and the input to send to it. The observation is the tool output that comes back into the system. That observation is then folded into the next decision, which is why the loop can adapt instead of merely replaying a script.

The scratchpad, or intermediate-step state, is what preserves that history. It records prior actions and observations so the agent can see what it has already tried before choosing the next move. Without that state, the model would have to make each decision in isolation. With it, each tool call becomes part of the context for the following turn.

The prompt has to make the available action space visible as well. Tool names and tool descriptions are injected into the template so the model knows what it is allowed to call and what each tool is for. That is a major difference from a free-form prompt. The agent is not inventing arbitrary actions; it is selecting from the tools that the runtime has exposed.

How LangChain wires tools, prompts, and execution

The implementation begins with the model. In the example, a hosted chat model is loaded with a zero temperature setting so the agent behaves more consistently when deciding among tools.

import os
from langchain_openai import ChatOpenAI

# Load OpenAI's LLMs with LangChain
os.environ["OPENAI_API_KEY"] = "MY_KEY"
openai_llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0)

Next comes the ReAct prompt. Its job is to define the interaction format and reserve placeholders for the tool list, tool names, the user input, and the scratchpad. That is what allows the same template to support many different runs while keeping the intermediate state attached to the current one.

# Create the ReAct template
react_template = """Answer the following questions as best you can. You have access to the following tools:

{tools}

Use the following format:

Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question

Begin!

Question: {input}
Thought:{agent_scratchpad}"""

prompt = PromptTemplate(
    template=react_template,
    input_variables=["tools", "tool_names", "input", "agent_scratchpad"]
)

The tools are then assembled into a shared list. Here the agent gets both a web search tool and an LLM-backed math tool. That combination is enough to show why agents matter: the model can fetch a live fact, then hand a numeric value to another capability for a deterministic follow-up step.

from langchain.agents import load_tools, Tool
from langchain.tools import DuckDuckGoSearchResults

# You can create the tool to pass to an agent
search = DuckDuckGoSearchResults()
search_tool = Tool(
    name="duckduck",
    description="A web search engine. Use this to as a search engine for general queries.",
    func=search.run,
)

# Prepare tools
tools = load_tools(["llm-math"], llm=openai_llm)
tools.append(search_tool)

The agent itself is created from the model, the tools, and the prompt. The executor is the runtime layer that makes the whole pattern work. It runs the loop, calls tools, gathers observations, and can print intermediate steps when verbose tracing is enabled. It also has to handle malformed action strings and parsing failures, because model output is not always perfectly formatted.

from langchain.agents import AgentExecutor, create_react_agent

# Construct the ReAct agent
agent = create_react_agent(openai_llm, tools, prompt)
agent_executor = AgentExecutor(
    agent=agent, tools=tools, verbose=True, handle_parsing_errors=True
)

A worked multi-step example: search plus arithmetic

The benefit of agentic control flow becomes clear with a query that mixes current information and computation. Suppose you want the current price of a MacBook Pro in USD and then want to convert that amount to EUR using an exchange rate. A fixed prompt can only guess at the live price. An agent can search for the current price first, then use the math tool on the retrieved value.

# What is the price of a MacBook Pro?
agent_executor.invoke(
    {
        "input": "What is the current price of a MacBook Pro in USD? How much would it cost in EUR if the exchange rate is 0.85 EUR for 1 USD."
    }
)

A typical run starts with the agent choosing search because it needs a live fact. The search tool returns an observation, which becomes input to the next decision. If the result is a price in USD, the agent can pass that value into the math tool to compute the EUR equivalent. The executor exposes those intermediate steps, so you can see both the tool call and the observation that followed it. That trace is valuable because it shows not just the final answer, but the path the agent took to get there.

This example also makes the division of labor obvious. The model is not doing the retrieval or the arithmetic itself. It is coordinating the search tool, the math tool, and the loop that connects them. That is what makes agent systems more flexible than chains, but also more fragile than chains, because each step depends on the quality of the previous one.

Where agents fail, and why retrieval comes next

Tool use adds power, but it also adds risk. A tool can return irrelevant or stale output. The model can produce malformed action text that fails parsing. A loop can continue taking actions without converging on a final answer. These are the kinds of failures that fixed chains largely avoid because their execution path is predetermined.

That trade-off is why retrieval-oriented systems come next. Agents are one way to orchestrate access to information, but they are not the only way, and they are not a replacement for retrieval architecture. The next chapter builds on this idea by showing how search and retrieval can be structured more deliberately, so the model can work with information in a more controlled way.

Use the url below to download the enitre book as pdf:

Read more

Earlier chapters treated text models mainly as representation engines. A review could become an embedding, a support ticket could become a feature vector, and a document could become a label. That perspective is still valuable, but prompt engineering starts when the goal changes. Instead of asking a model to reveal information already encoded in text, we ask it to produce new text that serves a task. The mental model shifts from extracting a signal to directing a generation.

That shift is important because generation is open ended. A classifier usually lives inside a small answer space, while a generative model can move in many plausible directions. The prompt is the operating surface for that process. It is the control layer through which we frame the task, set the tone, and indicate what kind of output should come next. Prompt engineering is not a writing trick and it is not a substitute for model capability. It is a practical engineering discipline for making the model legible to the task.

📘 Buy the Entire Book – Available on Amazon Worldwide (If you are paid subscriber, use the url at the end of this article to download the book for free!)

You can now purchase the complete book directly from Amazon in your country:

• United States (US)

• United Kingdom (UK)

• Germany (DE)

• France (FR)

• Spain (ES)

• Italy (IT)

• Netherlands (NL)

• Japan (JP)

• Brazil (BR):

• Canada (CA)

• Mexico (MX)

• Australia (AU)

• India (IN)

Seen that way, prompt engineering becomes an iterative workflow. You define the intent, examine what the model returns, notice where it drifts or overreaches, and revise the wording or structure. A weak prompt often behaves like an underspecified test case: it leaves too much room for the model to guess. Better prompts reduce that uncertainty by tightening the task boundary, making the expected behavior clearer, and improving consistency across runs and users. Stronger models help, but they do not remove the need for this control loop. Ambiguity, variability, and task-specific constraints still affect whether the output is useful in practice.

ReviewDesk makes the change in mindset easy to see. In a representation-first flow, a support item might be assigned a category for analysis. In a generation-first flow, the same item becomes a request for the next text the system should produce, such as a concise reply draft or a short internal summary. The business task is no longer just, what label is this, but what should the model produce next.

That is why prompt structure continues to matter even as models improve. Good prompts help the system stay consistent, easier to inspect, and easier to use downstream. They also set up the next concerns in this chapter: how to reason with generated outputs, how to verify them, and how to evaluate whether the result is actually fit for the job. Prompt engineering is the first control mechanism, not the last one.

Loading a Chat Model and Controlling Sampling

Why Start with a Small Open Chat Model

Prompt engineering becomes practical only when you can observe how a model changes as the prompt and decoding settings change. That is why a small open chat model is a sensible starting point instead of a large closed service hidden behind an API. A compact model is easier to run on modest hardware, cheaper to experiment with, and fast enough to support repeated trial and error. In other words, it gives you a low-friction place to learn the workflow before you move on to larger systems.

A model such as Phi-3-mini is especially useful in this learning phase because it is compact enough to be realistic on limited VRAM while still behaving like a modern instruct-tuned chat model. That makes it a good fit for iterative work: you can revise a prompt, rerun it, and inspect the difference without waiting on heavyweight infrastructure. The goal here is not to claim that this model is universally best. The goal is to choose a practical baseline that keeps experimentation simple.

Loading the Model, Tokenizer, and Generation Pipeline

The standard Transformers workflow is to load a causal language model, load its tokenizer, and then wrap both in a text-generation pipeline. Each object has a distinct role. The model contains the learned weights. The tokenizer contains the vocabulary and special tokens needed to turn text into token IDs and back again. The pipeline is a convenience layer that packages inference into a single interface.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    "microsoft/Phi-3-mini-4k-instruct",
    device_map="cuda",
    torch_dtype="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct")

# Create a pipeline
pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
    return_full_text=False,
    max_new_tokens=500,
    do_sample=False,
)

This configuration establishes a deterministic baseline. return_full_text=False keeps the output focused on the newly generated continuation instead of echoing the original prompt. max_new_tokens limits how far the response can grow. do_sample=False disables sampling, so generation follows the most likely next token at each step. That makes the result repeatable, which is exactly what you want when you are checking whether a prompt change caused a different answer.

The pipeline also makes the code easier to read. Instead of wiring generation calls by hand, you pass in a prompt or a message list and let the wrapper handle the inference details. That simplicity matters early in the workflow, because it lets you focus on prompt behavior rather than on low-level generation mechanics.

How Chat Messages Become a Serialized Prompt

Chat models are usually written against a message list rather than a single string. The list is the form you control as the developer, but it is not yet the exact input the model sees. Before generation, the tokenizer applies a model-specific chat template that serializes the messages into a prompt string and inserts the control tokens the model expects.

# Prompt
messages = [
    {"role": "user", "content": "Create a funny joke about chickens."}
]

# Generate the output
output = pipe(messages)
print(output[0]["generated_text"])

The structure above is simple on purpose. The key idea is that the message object is only the starting point. The pipeline turns that structure into the actual text representation the model conditions on.

# Apply prompt template
prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False)
print(prompt)

That printed string is the serialized prompt. It may include role tags, separators, and end markers that do not appear in the original message list. Those details are model-specific, which is why chat templates are not portable in a universal way. A prompt that is formatted correctly for one model can be serialized differently for another, even if the user-visible content looks identical.

A useful way to think about the distinction is that the message list is your instruction, while the chat template is the translation into the model’s dialect. When a response looks odd, too verbose, or unexpectedly truncated, the first thing to inspect is often the serialized prompt rather than the message object alone.

Turning On Sampling: Temperature, Top-p, and Output Variability

Deterministic decoding is the baseline, but it is not the only useful behavior. When sampling is enabled, the model has more freedom in how it selects the next token. Temperature and top-p are the two main controls for shaping that freedom.

Temperature adjusts how sharply the model favors the most likely tokens. Lower values concentrate probability mass on the top candidates and make outputs more predictable. Higher values spread probability more widely and make less likely tokens more available. In practice, that means a low temperature usually gives steadier wording, while a higher temperature usually produces more variation.

# Using a high temperature
output = pipe(messages, do_sample=True, temperature=1)
print(output[0]["generated_text"])

Top-p, also called nucleus sampling, uses cumulative probability mass to limit the candidate set. Instead of considering every token, the model keeps the smallest set of tokens whose combined probability reaches the chosen threshold. This makes the pool adaptive to the prompt and the model state. A related concept is top-k, which keeps a fixed number of highest-probability tokens, but temperature and top-p are the main controls you are likely to use most often.

# Using a high top_p
output = pipe(messages, do_sample=True, top_p=1)
print(output[0]["generated_text"])

Sampling should be treated as a reliability choice, not just a creativity feature. The more freedom you allow, the more likely the output is to vary from run to run. That variability can be helpful, but it also means you may need more validation if the output feeds a customer-facing or workflow-critical task. Deterministic generation is often the best debugging mode because fixed output makes it easier to tell whether a change came from the prompt or from the decoding settings.

Reading the Same Prompt Three Ways: Deterministic, Tempered, and Nucleus-Sampled

In ReviewDesk, imagine a support request asking for a polite reply to a customer who has failed to log in several times. With deterministic decoding, the response is usually stable and conservative, which makes it easier to review and compare. With a higher temperature, the wording may become more varied and expressive. With top-p sampling, the answer remains focused on plausible continuations but has more room to choose among them. The prompt stays the same while the decoding policy changes.

That difference matters because support, triage, and drafting workflows usually need reliability before style. Sampled outputs are useful examples, but they are not guaranteed to repeat exactly on another run, especially when temperature or top-p is enabled. For that reason, deterministic settings are a good place to start whenever you want to debug a prompt or isolate the effect of a formatting change.

From Single Prompt to Prompt Assembly

A prompt is easier to design when you stop treating it as one uninterrupted string and start treating it as an assembly of reusable parts. That shift matters because prompt engineering is not just about writing well once. It is about building an interface to the model that can be edited, tested, and reused across tasks. In practice, the useful question is not whether a prompt is long or short, but which parts are doing the work and how those parts interact.

A modular prompt makes that structure visible. If a response is too vague, the likely cause may be a missing instruction, weak context, an unclear format cue, or a mismatch between the audience and the tone. Thinking this way turns prompt writing into a debugging exercise. You can toggle one component at a time, observe the output, and learn which part actually changes the model’s behavior. That is a more reliable habit than rewriting everything from scratch whenever the result disappoints you.

Anatomy of a Modular Prompt

A prompt can be built from several reusable design units. Persona sets the role the model should inhabit, such as a support agent, analyst, or editor. Instruction states the job to perform. Context supplies the background that makes the job meaningful. Format tells the model how the answer should be shaped. Audience specifies who the output is for. Tone guides voice and level of formality. Data is the concrete content the task acts on. These pieces overlap in practice, but each one gives a different kind of control.

They are not a universal schema. A prompt does not need every field, and the fields do not need to appear in one fixed order. The point is to make the design choices explicit so they can be reused or removed deliberately. That is why prompt composition is best understood as a set of modules rather than a rigid template. If persona says “brief and direct” while tone says “detailed and explanatory,” the prompt sends mixed signals. If context is added late after the main instruction, it may still help, but the model may attend to the earlier framing more strongly. Ordering is not magic, but it does affect which cue feels dominant.

Consider a support-style summarization prompt. The persona can establish an expert voice, the instruction can ask for a concise summary, the context can explain that the goal is to extract the most important points, the format can request a labeled output, the audience can point to busy researchers, the tone can keep the result professional, and the data field can hold the text to summarize. Each component narrows the space of acceptable completions in a different way. If one part is removed, the output often changes in a visible and useful way. If the format cue disappears, the answer may become less structured. If the audience cue disappears, the wording may become less tailored. If the tone cue changes, the same content may feel more casual or more formal.

# Prompt components
persona = "You are an expert in Large Language models. You excel at breaking down complex papers into digestible summaries.\n"
instruction = "Summarize the key findings of the paper provided.\n"
context = "Your summary should extract the most crucial points that can help researchers quickly understand the most vital information of the paper.\n"
data_format = "Create a bullet-point summary that outlines the method. Follow this up with a concise paragraph that encapsulates the main results.\n"
audience = "The summary is designed for busy researchers that quickly need to grasp the newest trends in Large Language Models.\n"
tone = "The tone should be professional and clear.\n"
text = "MY TEXT TO SUMMARIZE"
data = f"Text to summarize: {text}"

# The full prompt - remove and add pieces to view its impact on the generated output
query = persona + instruction + context + data_format + audience + tone + data

The engineering value here is not the string concatenation itself. It is the ability to edit one field at a time and compare outputs. That makes prompt work feel closer to A/B testing than to one-shot writing. You can remove the audience clause, shorten the context, or move the format cue earlier and inspect what changes. This is especially useful when a prompt needs to be maintained over time, because each component can be revised without losing the whole structure.

Prompt Tuning by Editing the Pieces

Prompt refinement is usually iterative. Add a component, remove one, or reorder them, then compare the result. If the answer becomes more precise after adding format, that field is earning its place. If a persona statement causes unnecessary verbosity, it may be too strong for the task. If a context block is helpful only when placed near the instruction, that ordering choice is worth keeping. The point is to isolate cause and effect so the prompt behaves like a controlled design, not a guess.

A good rule of thumb is to change one thing at a time when possible. That makes the output easier to interpret. It also helps you distinguish a genuinely useful constraint from a cosmetic one. Prompt design becomes more stable when you treat each field as a testable feature rather than as decoration.

One-Shot Examples as Behavior Demonstrations

Another way to steer a model is to show it a worked example inside the prompt. This is one-shot in-context learning. It is not training, and it does not update the model’s weights. The example only changes the context the model is currently conditioning on. That distinction matters because a prompt example can demonstrate a pattern, but it does not rewrite the model’s underlying knowledge.

One-shot prompting is best understood as behavior demonstration. It can teach style, response shape, label mapping, or task format. It is useful when you want the model to imitate a pattern rather than infer it from a long rule description. It is less reliable as a way to inject brand-new facts the model never learned. A single example can show how a reply should sound, but it cannot guarantee durable factual learning the way training would.

With chat-based models, the example is usually written as role-structured messages. Those messages are then serialized through the model’s chat template into the actual input format expected by that model. The distinction is important. The message list is the structured prompt design, while the template serialization is the model-specific conversion step that turns that structure into tokens. In other words, the example is not just text; it is a sequence of roles and contents that the template packages for generation.

# Use a single example of using the made-up word in a sentence
one_shot_prompt = [
    {
        "role": "user",
        "content": "A 'Gigamuru' is a type of Japanese musical instrument. An example of a sentence that uses the word Gigamuru is:"
    },
    {
        "role": "assistant",
        "content": "I have a Gigamuru that my uncle gave me as a gift. I love to play it at home."
    },
    {
        "role": "user",
        "content": "To 'screeg' something is to swing a sword at it. An example of a sentence that uses the word screeg is:"
    }
]
print(tokenizer.apply_chat_template(one_shot_prompt, tokenize=False))

# Generate the output
outputs = pipe(one_shot_prompt)
print(outputs[0]["generated_text"])

This pattern is useful when the task is mostly about format or style. A single example can show how a ticket summary should be phrased, how a classification label should appear, or how a handoff note should be structured. It is not a shortcut to knowledge acquisition. It is a way of showing the model the pattern you want it to continue.

Chaining Prompts into a Workflow

Prompting also becomes more powerful when it is split across multiple calls. Chain prompting uses the output of one call as input to the next. Instead of asking the model to do everything in one pass, you let it perform one subtask, capture the intermediate output, and then reuse that result downstream. This is a workflow pattern, not a new model capability.

That separation is valuable when a task naturally breaks into stages. One call can draft a name, summary, or internal note. A later call can transform that draft into a customer-facing message, a shorter pitch, or a cleaner revision. The first output becomes a controlled input for the second prompt. That makes the pipeline easier to reason about because each step has a narrower purpose.

# Create name and slogan for a product
product_prompt = [
    {"role": "user", "content": "Create a name and slogan for a chatbot that leverages LLMs."}
]
outputs = pipe(product_prompt)
product_description = outputs[0]["generated_text"]
print(product_description)

# Based on a name and slogan for a product, generate a sales pitch
sales_prompt = [
    {"role": "user", "content": f"Generate a very short sales pitch for the following product: '{product_description}'"}
]
outputs = pipe(sales_prompt)
sales_pitch = outputs[0]["generated_text"]
print(sales_pitch)

In this example, the first call produces the product name and slogan, and the second call consumes that output to produce a sales pitch. That handoff is the key idea. Chain prompting is useful when you want decomposition, drafting, or refinement. It is often preferable to a single prompt when the full task is too broad, when the first pass is exploratory, or when the second pass needs a cleaner input than the raw source text.

It also has failure modes. Errors can accumulate across steps, vague intermediate outputs can become worse in the next call, and brittle handoffs can cause the later prompt to lose the original intent. For that reason, chain prompting works best when each intermediate output is named clearly and is specific enough to support the next stage. Think of it as an assembly line: each station does a smaller job, but each station also depends on the quality of the previous one.

What Modular Prompting Can and Cannot Do

Modular prompting improves steering, clarity, and reuse, but it does not retrain the model. Prompt examples do not update weights. They do not create permanent knowledge. They only shape the behavior of the current interaction. That is why prompt design and model training are related but fundamentally different. Prompting works by changing inputs; training works by changing parameters.

The practical takeaway is simple. Use modular prompts when you want to control behavior, demonstrate style, or decompose a task into stages. Use chain prompting when a single pass is too crowded and intermediate results will help. Use examples when you want to demonstrate a pattern. Keep in mind that these methods improve structure, not certainty. The next section extends that idea by moving toward techniques that ask the model to work through tasks more deliberately.

What reasoning-oriented prompting is trying to change

Ordinary prompt refinement mainly changes what the model says. Reasoning-oriented prompting tries to change the process the model is nudged to produce before it settles on an answer. That difference matters when the task has hidden structure, overlapping labels, or underspecified details. In those situations, a prompt that asks for intermediate work can make the output more stable than a prompt that simply asks for a polished response.

The important point is not that the model becomes a person-like reasoner. It does not suddenly think the way a human does. The practical effect is narrower and more useful: the prompt can steer generation toward a stepwise style that exposes intermediate choices, which often improves average reliability on messy tasks. For ReviewDesk, that is helpful when a ticket mentions several plausible categories at once and the model needs to compare evidence instead of guessing from the first clue.

Chain-of-thought prompting as single-path guided reasoning

Chain-of-thought prompting is the simplest version of this idea. You show the model one worked example that includes intermediate reasoning, then ask a new question in the same format. The example acts as a pattern for the next completion, so the model is more likely to produce a visible reasoning trace before giving the final label. This is useful when the answer depends on a hidden condition or on weighing more than one clue.

Here is a compact ReviewDesk-style example. The first ticket is answered with a brief justification, and the second ticket is left for the model to solve in the same style.

cot_prompt = [
    {"role": "user", "content": "Ticket: 'I was charged twice, but I also can't access my account after the password reset.' Classify the main issue and briefly explain why."},
    {"role": "assistant", "content": "The billing problem is the clearest actionable issue because the duplicate charge is explicit. Access trouble is present, but the payment issue is the strongest label. Final answer: billing."},
    {"role": "user", "content": "Ticket: 'My subscription renewed today, but the app says my plan expired and the invoice looks wrong.' Classify the main issue and briefly explain why."}
]

outputs = pipe(cot_prompt)
print(outputs[0]["generated_text"])

The worked example matters because it demonstrates not just the final format but the expected treatment of evidence. That is why chain-of-thought often helps on ambiguous support tickets, policy triage, or any other task where the label depends on comparing several plausible interpretations. It is still only a prompt-level nudge, though. A reasoning trace can look convincing and still be wrong if the task statement is unclear or if the model latches onto the wrong clue early.

Zero-shot chain-of-thought: triggering reasoning without examples

Zero-shot chain-of-thought keeps the same goal but removes the example. Instead of demonstrating reasoning, it asks for it directly with a short cue such as “think step by step.” This lighter version is useful when you want to save context, avoid crafting a demonstration, or quickly test whether a task benefits from explicit reasoning guidance.

zeroshot_cot_prompt = [
    {"role": "user", "content": "Ticket: 'My subscription renewed today, but the app says my plan expired and the invoice looks wrong.' Think step by step, then give the main issue."}
]

outputs = pipe(zeroshot_cot_prompt)
print(outputs[0]["generated_text"])

A short cue can still change generation style because it alters what the model is trying to produce next, even without examples. In practice, this is often a good first step. If it works, the prompt stays compact. If it does not, that is a clue that the task may need a better framing, a few-shot example, or a clearer decision rule rather than just a larger model.

Self-consistency: sample several reasoning traces and aggregate the answer

Self-consistency goes beyond a single reasoning path. Instead of trusting one completion, you sample several completions under stochastic decoding and then aggregate the final answers. The idea is simple: if the prompt is robust, different sampled traces should arrive at the same label. If the prompt is fragile, the variation between samples exposes that instability, and voting over the answers often improves the final decision.

This method depends on diversity in the samples, so decoding settings matter. If temperature is too low or top_p is too restrictive, the samples may be nearly identical and self-consistency loses much of its value. If sampling is too loose, the outputs can become noisy in a different way. The practical goal is enough diversity to reveal alternative reasoning paths while keeping the completions plausible.

For ReviewDesk, this is useful when a ticket could reasonably fit more than one category and the first answer is unreliable. A single pass might lock onto the most obvious word in the text. Several sampled passes make it more likely that the dominant pattern emerges, and a simple majority vote over the final labels often gives a better result.

The tradeoff is cost. Multiple samples mean more tokens, more latency, and more compute expense. That makes self-consistency a good fit for unstable or high-value decisions, but a poor fit for easy tasks where a single answer is already good enough.

Tree-of-thought: branching into alternative solution paths

Tree-of-thought extends the same general idea, but it changes the unit of exploration. Instead of sampling several final answers, it branches into several intermediate solution paths, compares them, and keeps the promising ones while discarding the weak ones. The key difference is that this is about path exploration, not answer voting.

That distinction is important. Self-consistency asks, in effect, which complete answer shows up most reliably across samples. Tree-of-thought asks which partial line of reasoning appears most promising before the answer is finalized. It is a drafting-and-narrowing process rather than a voting process.

A simplified ReviewDesk prompt can imitate this by asking for a few candidate interpretations and then a final choice.

tot_prompt = [
    {"role": "user", "content": "Ticket: 'I was charged after canceling, and the dashboard still says my payment failed.' Consider a few possible interpretations, compare them briefly, then choose the best category and explain why."}
]

outputs = pipe(tot_prompt)
print(outputs[0]["generated_text"])

You can think of the method as asking the model to draft several candidate explanations, debate which one fits best, and then narrow to the strongest branch. That is enough to capture the practical idea without turning the prompt into a full search algorithm. Tree-of-thought is most useful when the task is open-ended or when the model needs to choose among several plausible paths rather than retrieve one obvious fact.

Choosing a reasoning strategy in practice

A simple mental model helps. Use a single-pass prompt when the task is straightforward. Use self-consistency when the task is unstable and the cost of a wrong answer is higher than the cost of extra sampling. Use tree-of-thought when the task requires comparing several competing interpretations or exploring a small decision space before choosing.

All three methods improve prompt behavior and often improve average outcomes, but none of them guarantees correctness. They reduce uncertainty; they do not eliminate it. If the ticket is ambiguous, underspecified, or high stakes, you should still verify the result with downstream checks or human review. The strongest practical benefit of reasoning-oriented prompting is not perfect logic. It is a better chance of getting a careful answer when the problem itself is not already easy.

Structured Output, Verification, and Constrained Decoding

Structured output becomes a production requirement the moment a model response stops being a reading experience and starts being input for something else. A support triage system cannot “mostly” receive JSON. A database loader cannot guess where the category field ended. An API client cannot safely consume prose that looks organized to a human but fails at the border checkpoint of a parser. This is the practical distinction that matters here: a response can look right on screen and still not run right in software.

The simplest way to improve structure is to ask for it directly. A zero-shot prompt can request a fixed schema, but the model is still free to drift, add commentary, or return partial objects. In ReviewDesk, you might ask for a ticket classification result containing a category, a priority, and a short draft reply. That is enough to guide the model, but not enough to guarantee anything. If the task is forgiving, a prompt alone may be acceptable. If the output is going straight into another system, the same request becomes fragile.

prompt = [
    {
        "role": "user",
        "content": (
            "Classify this ReviewDesk ticket and return valid JSON with exactly these keys: "
            "category, priority, draft_response. "
            "Ticket: 'My login works on mobile but not on desktop after the last update.'"
        ),
    }
]

output = llm.create_chat_completion(messages=prompt, temperature=0)["choices"][0]["message"]["content"]
print(output)

A one-shot or few-shot template usually improves adherence because the model sees the target shape rather than merely being told about it. The example acts like a small map of the destination. You are not training the model, and you are not changing its weights; you are narrowing the space of likely completions by showing a format that should be imitated. That distinction is easy to miss. In-context examples are temporary steering signals, not new knowledge stored by the model.

prompt = [
    {
        "role": "user",
        "content": (
            "Return ReviewDesk tickets in this JSON format only:\n"
            '{"category":"...","priority":"...","draft_response":"..."}\n\n'
            "Example:\n"
            '{"category":"authentication","priority":"high","draft_response":"Please try resetting your password..."}\n\n'
            "Ticket: 'The billing page shows an error when I open it from Safari.'"
        ),
    }
]

output = llm.create_chat_completion(messages=prompt, temperature=0)["choices"][0]["message"]["content"]
print(output)

This is where the contrast between guidance and enforcement becomes important. Prompt shaping is persuasive; constrained decoding is restrictive. Prompting tells the model what kind of answer is wanted. Grammar-constrained decoding or JSON-enforced generation limits the next token choices so the model can only produce output that fits the allowed format. In other words, prompting steers the car, while constrained decoding puts rails on the road. The prompt still matters, but the runtime is now doing part of the reliability work.

With runtimes that support structured generation, the same ReviewDesk task can be asked for as JSON-only output. A model such as a quantized GGUF chat model loaded through llama-cpp-python can be configured so the response must satisfy a JSON grammar or a JSON response format. That does not make the answer correct in the semantic sense, but it does strongly improve the guarantee that the string is machine-readable.

from llama_cpp.llama import Llama

llm = Llama.from_pretrained(
    repo_id="microsoft/Phi-3-mini-4k-instruct-gguf",
    filename="*fp16.gguf",
    n_gpu_layers=-1,
    n_ctx=2048,
    verbose=False,
)

output = llm.create_chat_completion(
    messages=[
        {
            "role": "user",
            "content": (
                "For this ReviewDesk ticket, return JSON with category, priority, and draft_response. "
                "Ticket: 'My invoice downloaded as a blank PDF.'"
            ),
        }
    ],
    response_format={"type": "json_object"},
    temperature=0,
)["choices"][0]["message"]["content"]

print(output)

The runtime can enforce shape, but it cannot enforce truth. That is why verification remains necessary. A visually plausible blob of text is not the same thing as valid JSON, and valid JSON is not the same thing as a correct or useful answer. Parsing is the first real test. If json.loads fails, the output is not ready. If parsing succeeds, the next step is to check that the expected fields are present and that their types make sense for the downstream consumer.

import json

parsed = json.loads(output)
print(json.dumps(parsed, indent=2))

required_keys = {"category", "priority", "draft_response"}
if not required_keys.issubset(parsed):
    raise ValueError(f"Missing keys: {required_keys - set(parsed)}")

if not isinstance(parsed["category"], str):
    raise TypeError("category must be a string")
if not isinstance(parsed["priority"], str):
    raise TypeError("priority must be a string")
if not isinstance(parsed["draft_response"], str):
    raise TypeError("draft_response must be a string")

That small parse-and-validate step is more than a convenience. It is the border checkpoint between generated text and usable data. If the model returns a malformed object, a truncated string, or a response with commentary wrapped around the JSON, the failure mode should be classified rather than ignored. A malformed object usually calls for regeneration. A truncated object often points to token limits or overly long outputs. Commentary-laden output may be fixed by stronger instructions or by constrained decoding. In a brittle pipeline, the right response is often to retry with lower temperature, a tighter schema, or a stricter decoder. In a less brittle workflow, it may be acceptable to reject the result and fall back to a human review path.

The operational rule is simple. Shape with examples when failure is tolerable, enforce with decoding when the downstream consumer is brittle, and validate whenever the output is reused programmatically. That sequence keeps prompt engineering from becoming guesswork. It treats malformed output as a workflow incident rather than as a cosmetic annoyance, which is exactly the mindset needed when a model stops being a chatbot and starts being a component.

Use the url below to download the enitre book as pdf:

Read more

When a text collection gets large enough, the first problem is usually not prediction but orientation. A support team may have thousands of ReviewDesk tickets, and reading them one by one is like opening a crowded inbox after a long weekend: you can see that there is structure, but you cannot yet see the structure clearly enough to act on it. Unsupervised text clustering gives you a way to sort that inbox by similarity before you decide what the folders should be called. The goal is not to assign final categories on day one. The goal is to reveal recurring shapes in the data so that a human can inspect them with much less effort.

That distinction matters. A cluster is a discovered grouping of documents that appear to belong together; a topic label is the human-readable name you attach after you inspect that group. Those are related tasks, but they are not the same task. If you collapse them too early, you end up treating an interpretive label as if it were an objective property of the corpus. In practice, clustering is the intermediate representation that helps you get from a messy archive to a manageable set of candidate themes.

The reason embedding-based methods work so well here is that they group by meaning rather than by surface overlap. Two ReviewDesk tickets can talk about the same billing issue using very different words, and a keyword-matching approach may miss that relationship entirely. A semantic model can still place them near each other because it has learned that the underlying intent is similar, even when the vocabulary is not. That is the crucial advantage of dense representations: they make latent structure visible when the obvious words are misleading.

📘 Buy the Entire Book – Available on Amazon Worldwide (If you are paid subscriber, use the url at the end of this article to download the book for free!)

You can now purchase the complete book directly from Amazon in your country:

• United States (US)

• United Kingdom (UK)

• Germany (DE)

• France (FR)

• Spain (ES)

• Italy (IT)

• Netherlands (NL)

• Japan (JP)

• Brazil (BR):

• Canada (CA)

• Mexico (MX)

• Australia (AU)

• India (IN)

This is especially useful in unlabeled corpora, where classification is not available or not worth the cost. In many teams, labels are incomplete, stale, or too coarse to explain the patterns hidden in the archive. Clustering gives you a fast way to explore the shape of the dataset before you commit to a taxonomy. In a support setting, that might surface separate piles for product bugs, billing questions, feature requests, and operational issues. In a media workflow, the same idea could expose groups of similar assets, recurring content briefs, or unusual one-off submissions that deserve special handling.

Outliers are part of the value, not a nuisance to be ignored. A density-based method may leave some documents unassigned because they do not fit any sufficiently coherent group. That is often exactly what you want. The odd ticket that references two unrelated problems, the duplicate message copied from another channel, or the rare edge case from a premium customer can be more informative than a clean cluster. These documents tell you where the corpus is noisy, where workflows are messy, and where your categories may need revision.

Once you have clusters, topic modeling becomes a second layer of interpretation rather than a replacement for discovery. The cluster tells you that a set of documents belong together; the topic label tells you what to call that set in a way a person can read quickly. In that sense, topic modeling is like writing captions for a photo album after the photos have already been grouped. The grouping comes first because it reduces the search space. The naming comes later because it is easier to label a coherent pile than to assign meaning to thousands of isolated documents.

That is the workflow this chapter develops. We will start with embeddings, use them to group documents semantically, and then refine those groups into readable topics. The later BERTopic-style steps will add keyword summaries, representation methods, and labeling tools, but their usefulness depends on this first move: discovering clusters before pretending to know their names.

Anchor the Running Corpus

The chapter’s shared corpus is the ArXiv cs.CL collection, a convenient stand-in for a real document stream that already contains a broad spread of computation and language research abstracts. Treat it as the chapter’s working file cabinet: the documents are already collected, but they still need to be arranged into a clean structure before any semantic grouping can happen. That structure matters because clustering does not begin with labels; it begins with a corpus that can be read consistently, inspected later, and passed unchanged through the next stages of the pipeline.

Load the Dataset from Hugging Face

The starting point is intentionally simple. The dataset is fetched from Hugging Face, and the chapter uses the training split as the source corpus. That gives us a reproducible pool of documents without needing to assemble the collection by hand. In practice, this kind of loading step is the bridge between a hosted dataset and an analysis-ready document table.

from datasets import load_dataset

dataset = load_dataset("maartengr/arxiv_nlp")["train"]

abstracts = dataset["Abstracts"]
titles = dataset["Titles"]

The code does not do any modeling yet. Its job is to make the corpus explicit. If the public dataset is unavailable in another setting, the same pattern still applies to a domain corpus such as ReviewDesk support tickets: load the source table, choose the slice you want to analyze, and separate the text you want to embed from the metadata you want to keep for interpretation.

Separate Text from Metadata

Here the abstracts are the primary documents. They are the text that will later be embedded, compared, and grouped. Titles play a different role. They are lightweight metadata that help you inspect a cluster, understand a document at a glance, and later assign or evaluate a topic label. Keeping those two fields separate is a small design choice with large downstream consequences. If you merge the title into the abstract too early, you blur the boundary between content and annotation, and later inspection becomes noisier than it needs to be.

A useful way to think about it is that the abstract is the folder contents, while the title is the tab on the folder. Clustering works on the contents; the tab helps you understand what ended up together after the fact. That separation also makes error analysis easier, because you can ask whether a document landed in the right group without having to untangle extra metadata from the text itself.

Sanity-Check the Corpus Shape

Before moving on, it is worth confirming that the corpus is usable as a one-document-per-row collection. The abstract and title arrays should stay aligned, and missing entries should be understood before they propagate into later steps. Even when the dataset is already curated, this is the point where a quick structural check pays off: if the document table is misaligned now, every cluster inspection afterward will inherit the mistake.

Why This Structure Supports Later Clustering

This preparation step does not yet cluster anything, but it sets up the exact input shape that clustering expects: a clean collection of documents with stable metadata attached beside them. That makes the later embedding stage straightforward, and it gives you a reliable way to read the results once groups begin to emerge. In unsupervised text analysis, the quality of the final clusters is only part of the story; the quality of the document table determines whether those clusters can be interpreted with confidence.

From Embeddings to Clusters and Visual Maps

Why embeddings are the right starting point for text groups

If you want to sort a messy inbox, you do not begin by counting shared words and hoping the piles organize themselves. That kind of sparse lexical view is useful for some problems, but it is a weak way to group whole documents by meaning. Two support tickets can describe the same issue with different product names and different wording, yet still belong together. Semantic embeddings handle that better because they turn each document into a dense vector where related texts end up near one another.

That is why this workflow starts with embeddings rather than bag-of-words or TF-IDF. The goal is not to preserve exact vocabulary. The goal is to preserve enough semantic neighborhood structure that similar documents can be grouped even when surface wording differs. In practice, this makes a mixed corpus easier to sort before you assign human-readable topic names.

Think of embeddings as a smarter sorting surface. The model has not named the folders yet, but it has already arranged the documents so likely neighbors sit close together. That is the foundation for the rest of the pipeline.

from sentence_transformers import SentenceTransformer

embedding_model = SentenceTransformer("thenlper/gte-small")
embeddings = embedding_model.encode(titles_and_texts, show_progress_bar=True)

embeddings.shape

From high-dimensional vectors to a clusterable representation

The embedding matrix is still high-dimensional, which is good for representation but awkward for clustering. Many clustering methods become less effective when the space is wide and sparse, especially when the useful structure lives in local neighborhoods rather than across every dimension. Reducing the vectors first makes the geometry more tractable.

This reduction has two distinct jobs in the workflow. One reduced space is used as the working representation for clustering. A separate two-dimensional projection is used later for plotting. The first helps the algorithm organize the documents. The second helps you inspect the result. Those are related, but they are not the same step.

The distance measure matters too. Cosine distance fits embedding-based text similarity well because semantic embeddings are usually interpreted by direction rather than raw magnitude. If two documents point in a similar semantic direction, they should be treated as close even if their lengths differ or their vector norms do not match exactly.

A useful way to picture this is a working geometry, not a true map. The reducer is trying to preserve the neighborhoods that matter for grouping, not to reveal some perfect universal coordinate system for language.

Using UMAP to preserve neighborhood structure

UMAP is a practical reducer for this task because it aims to preserve local neighborhoods while compressing the data into fewer dimensions. Documents that are close in the original embedding space should remain close after reduction, at least well enough for clustering to use the result. That makes UMAP useful both as a preparatory step for clustering and later as a projection for display.

The main parameters here are easy to read in conceptual terms. n_components sets the size of the reduced working space. A small value such as five often gives a clustering algorithm enough room to separate groups without keeping the full embedding width. metric is set to cosine so the reduction respects embedding geometry. min_dist controls how tightly neighboring points may pack together, and a small value encourages compact local structure. random_state fixes the stochastic parts of the process so repeated runs are easier to compare.

from umap import UMAP

umap_model = UMAP(
    n_components=5,
    min_dist=0.0,
    metric="cosine",
    random_state=42,
)
reduced_embeddings = umap_model.fit_transform(embeddings)

That reproducibility matters because cluster results are not absolute facts. They depend on the model, the reducer, and the seed. If the seed changes, the exact arrangement can change too. Holding it fixed makes interpretation much easier when you are trying to understand whether a group is stable or just an artifact of a particular run.

Letting HDBSCAN discover groups and reject noise

Once the embeddings have been reduced, HDBSCAN can discover clusters without requiring you to choose k in advance. That is one reason it works well for open-ended text corpora. You do not always know how many themes are present, and forcing a fixed cluster count can be awkward when the corpus is messy or uneven.

HDBSCAN looks for dense regions in the reduced space. Where points form a compact neighborhood, it assigns a cluster. Where points are sparse, ambiguous, or transitional, it can leave them unassigned. That unassigned behavior is not a bug. It is often the most useful part of the method, because real corpora usually contain items that do not belong cleanly anywhere.

The label -1 marks those noise points. In practice, that means borderline documents, vague requests, or one-off items can be separated from the main clusters instead of being forced into a misleading group. If you expect messy real-world text, this is a valuable feature, not a failure.

from hdbscan import HDBSCAN

hdbscan_model = HDBSCAN(
    min_cluster_size=50,
    metric="euclidean",
    cluster_selection_method="eom",
).fit(reduced_embeddings)
clusters = hdbscan_model.labels_

len(set(clusters))

The exact cluster labels are parameter-dependent. Change the embedding model, the UMAP settings, or the HDBSCAN thresholds, and the cluster assignment can shift. That is normal. The point is not to treat one run as permanent truth. The point is to get a usable grouping that fits the corpus and the task.

Checking cluster meaning by reading sample documents

A cluster becomes meaningful only after you read some of the documents inside it. The simplest validation habit is to sample a few members from one cluster and ask whether they share a recognizable theme. In a support-ticket corpus, that might reveal a cluster about login failures, another about billing corrections, or another about export requests.

import numpy as np

cluster = 0
for index in np.where(clusters == cluster)[0][:3]:
    print(titles_and_texts[index][:300] + "...\n")

This step catches an important failure mode. A cluster can look neat numerically and still mix several subthemes that happen to share vocabulary. Human reading is what tells you whether the cluster is genuinely coherent or merely compact in reduced space. Numerical grouping gives you structure, but human inspection gives you confidence.

Projecting into two dimensions for diagnosis, not proof

After clustering, you can make a two-dimensional projection for plotting. This is a diagnostic view, not the clustering step itself. It helps you spot broad structure, noise points, and overlap, but it is not a ground-truth map of meaning. A nice scatterplot can be misleading if you treat it as proof that the clusters are perfectly pure.

A practical plotting convention is to keep outliers separate from clustered points. Noise points are often drawn in gray so they recede visually, while clustered points are colored by cluster ID. The axes are usually hidden because the exact coordinates matter less than the relative arrangement.

import pandas as pd
from umap import UMAP

plot_embeddings = UMAP(
    n_components=2,
    min_dist=0.0,
    metric="cosine",
    random_state=42,
).fit_transform(embeddings)

df = pd.DataFrame(plot_embeddings, columns=["x", "y"])
df["title"] = titles
df["cluster"] = [str(c) for c in clusters]

to_plot = df.loc[df.cluster != "-1", :]
outliers = df.loc[df.cluster == "-1", :]

import matplotlib.pyplot as plt

plt.scatter(outliers.x, outliers.y, alpha=0.05, s=2, c="grey")
plt.scatter(
    to_plot.x,
    to_plot.y,
    c=to_plot.cluster.astype(int),
    alpha=0.6,
    s=2,
    cmap="tab20b",
)
plt.axis("off")

Read that plot as an X-ray, not as a map. It can show where clusters sit, where outliers collect, and where the method may be uncertain. It cannot prove semantic purity. Visual separation and semantic purity are related, but they are not identical. Two clusters can look far apart and still share content, and two clusters can sit near each other while still representing distinct themes.

This is also where the workflow connects to topic modeling. Clustering has organized the corpus into semantically related groups, but those groups still need readable topic descriptions and representative terms. The next section builds on that structure rather than starting over from scratch.

From Cluster IDs to Topic Representations

Once you have clustered document embeddings, you have already found something useful, but the result is still unnamed. Clustering tells you which documents belong together; topic modeling asks what those groups are about. BERTopic treats those as separate layers. It can reuse the embeddings, UMAP reduction, and HDBSCAN clusters from the previous step, then build a topic representation on top of them. The cluster assignment is the backbone, and the topic name is the label that comes later.

That separation is what makes BERTopic modular. You can keep the same document groups while changing only how they are described. The inbox analogy is a good one: clustering sorts the messages into folders, and representation models rename the folders after you look inside. In ReviewDesk, one folder might collect billing failures, another mobile crashes, and another feature requests. The groups already exist before they are named, and BERTopic gives you a structured way to turn those groups into searchable topics.

Outliers carry over from the clustering step as well. If HDBSCAN leaves some documents unassigned, BERTopic preserves that behavior with topic -1 instead of forcing every document into a topic that may not fit. That is useful when the corpus contains short, ambiguous, or genuinely unusual documents.

from bertopic import BERTopic

# Train our model with our previously defined models
topic_model = BERTopic(
    embedding_model=embedding_model,
    umap_model=umap_model,
    hdbscan_model=hdbscan_model,
    verbose=True
).fit(abstracts, embeddings)

How c-TF-IDF Builds the First Topic Keywords

BERTopic turns each cluster into an initial keyword description with c-TF-IDF, or class-based TF-IDF. The key idea is to pool all documents in a cluster and treat that pool as one class-level text. The model then scores words by asking a simple question: what appears often in this folder, and how distinctive is it compared with the other folders?

That is the important difference from ordinary document-level TF-IDF. Standard TF-IDF works on individual documents. c-TF-IDF shifts the unit of analysis to the cluster, so the words are chosen to represent the group as a whole rather than any single document. In practice, this is what makes the topic keywords feel topic-like instead of document-like.

A compact version of the scoring idea is

c-TF-IDF(w,c)=tf(w,c)⋅log⁡(Ndf(w)),

where w is a word, c is a cluster, tf(w,c) measures how often the word appears in the pooled cluster text, and df(w) counts how many clusters contain that word. The formula is only a shorthand for the intuition: topic words should be common inside one cluster and relatively uncommon across the others.

This class-level weighting also explains why topic -1 can appear in the output. If some documents remain outliers, their pooled text becomes its own class-level representation. The resulting words are a reminder that density-based clustering can preserve uncertainty instead of inventing a confident label where none is justified.

Working with the Topic API in Practice

After fitting the model, the fastest way to understand what it found is to inspect the topic summary table. This shows topic IDs, document counts, and the first pass at representative keywords. It is a quick audit of whether the clusters look meaningful and whether the outlier topic is large enough to deserve attention.

topic_model.get_topic_info()

To inspect one topic in more detail, you can ask for its ranked keywords directly. That is often the most practical way to verify whether a cluster named by c-TF-IDF really corresponds to the subject you expect.

topic_model.get_topic(0)

You can also look up the topic assigned to a specific document. In a support workflow, that is how you check whether a ticket about payment failures, app crashes, or account access landed in the right group.

topic_model.topics_[titles.index("BERTopic: Neural topic modeling with a class-based TF-IDF procedure")]

Query-based search works a little differently. It asks which discovered topic is most similar to a text prompt, such as "topic modeling". This is topic retrieval, not corpus search. You are not finding documents that contain the phrase; you are asking which folder in the learned topic space best matches the phrase.

topic_model.find_topics("topic modeling")

Improving Topic Word Lists Without Reclustering

BERTopic keeps cluster membership separate from topic wording, which means you can refine the labels without changing the groups themselves. That is a major advantage of the framework. The same cluster can be named in several different ways, and those names can be updated without rerunning the whole embedding and clustering pipeline.

The baseline topic words come from c-TF-IDF, but representation models can rerank or diversify them afterward. This is the post-processing layer. It is cheaper than retraining and often produces clearer labels, although not always more faithful ones. Cleaner wording is useful, but it can also flatten nuance if a reranker removes a distinctive term.

A small helper makes the before-and-after comparison easier to inspect.

from copy import deepcopy
original_topics = deepcopy(topic_model.topic_representations_)

def topic_differences(model, original_topics, nr_topics=5):
    df = pd.DataFrame(columns=["Topic", "Original", "Updated"])
    for topic in range(nr_topics):
        og_words = " | ".join(list(zip(*original_topics[topic]))[0][:5])
        new_words = " | ".join(list(zip(*model.get_topic(topic)))[0][:5])
        df.loc[len(df)] = [topic, og_words, new_words]
    return df

KeyBERTInspired is a similarity-focused reranker. It prefers words that sit closer to the semantic center of the topic, so the resulting keyword list often reads more coherently than raw c-TF-IDF output.

from bertopic.representation import KeyBERTInspired

representation_model = KeyBERTInspired()
topic_model.update_topics(abstracts, representation_model=representation_model)
topic_differences(topic_model, original_topics)

MaximalMarginalRelevance, or MMR, pushes in a different direction. It still wants relevant words, but it also tries to avoid redundancy. If KeyBERTInspired looks for the best-fitting tracks on a playlist, MMR tries to keep the playlist from sounding repetitive. The benefit is a more varied label; the limitation is that too much diversification can hide useful detail.

from bertopic.representation import MaximalMarginalRelevance

representation_model = MaximalMarginalRelevance(diversity=0.2)
topic_model.update_topics(abstracts, representation_model=representation_model)
topic_differences(topic_model, original_topics)

In practice, reranking is best seen as label editing rather than topic discovery. It can make a topic easier to read, but it does not change which documents belong to that topic. That is why the cluster boundaries stay fixed while the sign on the folder changes.

Generating Human-Readable Labels with LLMs

Representation models can also use text generation. In that setup, BERTopic gives the model representative documents and topic keywords, then asks it to produce a short label. This feels closer to summarizing a cluster than to training a new model. The prompt acts like a captioning template for the topic.

from transformers import pipeline
from bertopic.representation import TextGeneration

prompt = """I have a topic that contains the following documents: 
[DOCUMENTS]

The topic is described by the following keywords: '[KEYWORDS]'.

Based on the documents and keywords, what is this topic about?"""

generator = pipeline("text2text-generation", model="google/flan-t5-small")
representation_model = TextGeneration(
    generator, prompt=prompt, doc_length=50, tokenizer="whitespace"
)
topic_model.update_topics(abstracts, representation_model=representation_model)
topic_differences(topic_model, original_topics)

A chat model can do the same job with a slightly different prompt format.

import openai
from bertopic.representation import OpenAI

prompt = """
I have a topic that contains the following documents:
[DOCUMENTS]

The topic is described by the following keywords: [KEYWORDS]

Based on the information above, extract a short topic label in the following format:
topic: <short topic label>
"""

client = openai.OpenAI(api_key="YOUR_KEY_HERE")
representation_model = OpenAI(
    client, model="gpt-3.5-turbo", exponential_backoff=True, chat=True, prompt=prompt
)
topic_model.update_topics(abstracts, representation_model=representation_model)
topic_differences(topic_model, original_topics)

Generated labels are flexible, but that flexibility comes with caveats. They can be broad, corpus-dependent, and slightly different across runs or models. A label like Billing Issues may be fluent, yet a more specific label such as Refund Delays and Invoice Problems may be more useful for support triage. The generated name should therefore be treated as a suggestion that still needs human review.

Visualizing Topics as an Inspection Tool

Visualization helps you audit the learned topic structure, but it does not prove that a topic is correct. A document map can show whether documents within a topic are close together, whether two topics overlap, or whether outliers sit far from the main groups. A keyword bar chart can show which words dominate a topic, and a heatmap or hierarchy can reveal similarity between topics. These plots are diagnostic tools, not final evidence.

fig = topic_model.visualize_documents(
    titles, 
    reduced_embeddings=reduced_embeddings, 
    width=1200, 
    hide_annotations=True
)

fig.update_layout(font=dict(size=16))

topic_model.visualize_barchart()
topic_model.visualize_heatmap(n_clusters=30)
topic_model.visualize_hierarchy()

A datamap-style view is another way to inspect the same structure at a glance.

fig = topic_model.visualize_document_datamap(
    titles,
    topics=list(range(20)),
    reduced_embeddings=reduced_embeddings,
    width=1200,
    label_font_size=11,
    label_wrap_width=20,
    use_medoids=True,
)

The practical value of these visualizations is that they help you ask better questions. Do similar topics sit near each other? Are the largest topics internally consistent? Do outliers suggest noise or a missing theme? In that sense, the plots support interpretation, but the topic meaning still comes from the cluster structure and the words chosen to describe it.

Recap: Choosing and Interpreting Topic Workflows

The chapter’s workflow is best remembered as a translation pipeline: embeddings capture semantic proximity, dimensionality reduction makes that structure easier to work with, clustering discovers document groups, and topic representation turns those groups into language a person can inspect. That sequence matters because each stage solves a different problem. The geometry in embedding space tells you which documents belong together; it does not yet tell you what to call the cluster or whether the grouping deserves to be treated as a coherent theme.

That separation between discovery and interpretation is the central idea to carry forward. Clustering is like sorting a crowded inbox into piles by similarity. Topic work is what writes the folder tabs. The piles can be useful even before they are named, but the names are what make the output communicate well enough for analysis, reporting, or downstream workflows. In practice, a dense cluster may correspond to a real theme, a loose mixture of related complaints, or simply a repeated phrasing pattern. Outlier handling is part of that reality, not a sign that the method failed.

BERTopic sits on top of this discovery process rather than replacing it. It reuses the clustered documents and then applies c-TF-IDF to surface the terms that best characterize each group. That gives the first readable version of a topic. Representation models then refine that view by changing how the cluster is summarized, which can improve keyword quality, reduce repetition, or emphasize more informative phrases. Methods such as maximal marginal relevance and KeyBERTInspired are useful here because they help topic descriptions become less redundant and more legible without reclustering the corpus.

Generative topic labels add another layer of usefulness, especially when the keyword view is too sparse or too technical. But these labels should be treated as assistive summaries, not authoritative truth. A model can produce a crisp label that sounds plausible while still flattening an important distinction in the documents underneath. For that reason, manual inspection remains the final check before a topic is used operationally. A good practical rule is simple: if you only need exploration, raw clusters may be enough; if you need communication, topic labels help; if you need accountability, human review is still required.

This chapter closes by moving from unsupervised discovery to controlled generation. In the next chapter, prompts become the main way to steer what a model says, which is the natural complement to the unsupervised methods here.

Use the url below to download the enitre book as pdf:

Read more

Text classification is the task of assigning a discrete label to a text input. In the running example for this chapter, a review is mapped to sentiment labels such as positive or negative. The same basic idea also appears in intent detection, topic assignment, language identification, and entity-related labeling. The input is text, and the output is one choice from a fixed set of classes.

That simple task definition matters because it lets us compare methods on the same problem. If every approach sees the same labeled data and the same target labels, then the comparison is about workflow choice rather than about changing the task itself.

Why pretrained models change the workflow

For modern text classification, pretrained language models are the practical starting point. They already encode useful syntactic and semantic information, so downstream work begins from a capable base rather than from random initialization. That makes them much more useful than training a model from scratch when the goal is to solve a real classification task quickly and reliably.

A pretrained model can still play different roles. It may behave as a direct classifier, or it may serve as a source of representations that another model consumes. It can also be used as a generative system that is prompted to emit a label. The common thread is reuse, but what gets reused is different in each case: the decision head, the vector space, or the language-generation ability.

📘 Buy the Entire Book – Available on Amazon Worldwide (If you are paid subscriber, use the url at the end of this article to download the book for free!)

You can now purchase the complete book directly from Amazon in your country:

• United States (US)

• United Kingdom (UK)

• Germany (DE)

• France (FR)

• Spain (ES)

• Italy (IT)

• Netherlands (NL)

• Japan (JP)

• Brazil (BR):

• Canada (CA)

• Mexico (MX)

• Australia (AU)

• India (IN)

The first family: task-specific classifiers built on representations

The first workflow family is the direct classifier. Here the pretrained model has been shaped for a downstream prediction task, so its output is already aligned with the label space rather than being a generic description of the text. In sentiment analysis, that means the model is set up to decide whether a review is positive or negative.

This is the most direct use of pretrained capability. The model receives text and produces a class or class score without requiring another algorithm to interpret intermediate vectors. A good way to think about it is as a specialist: it is built to answer the classification question directly, which often makes it a strong choice when speed and consistent behavior matter.

The second family: reusable embeddings feeding a separate classifier

The second workflow family uses a pretrained model as an embedding model. In this setup, the model does not make the final label decision itself. Instead, it turns each text into a reusable vector, and a separate classifier learns from those vectors. The classifier can be comparatively simple, such as logistic regression, because the representation model has already done the heavy lifting of turning text into features.

This split between feature extraction and classification is what makes the pipeline modular. The same embeddings can support sentiment classification in one setting and a different downstream use in another setting. Here the pretrained model acts like an adapter or feature extractor: it supplies a vector space, and the classifier on top learns the decision boundary.

The third family: prompting a generative model to classify

The third workflow family uses a generative model as a classifier. A model whose native job is text generation can still be asked to output a class label, such as positive or negative, when the prompt is framed carefully. In other words, generation and classification are not mutually exclusive.

This approach differs from the representation-based workflows because no separate classifier consumes embeddings and no dedicated prediction head is required for the task. The model is steered to produce a label directly from its generation ability. That gives it flexibility, especially when the desired behavior may change across tasks, but it also gives the user less deterministic control than a fixed classifier.

How the chapter will compare these workflows

The rest of the chapter compares these three families on the same labeled dataset and the same classification task. That makes the evaluation fairer and easier to interpret, because the methods are being judged on identical labels and inputs. The question is not which problem is easier, but which workflow is the better fit for the same problem.

A classic TF-IDF plus logistic regression baseline remains a useful reference point, but the main focus here is on pretrained approaches. The comparison will therefore ask a practical question: when is the direct classifier the best option, when does an embedding pipeline make more sense, and when is prompt-based generation flexible enough to justify its extra variability?

Introduce the Rotten Tomatoes corpus as the chapter’s shared dataset

The chapter uses the Rotten Tomatoes movie-review corpus as its running example for binary sentiment classification. Each record pairs a short review with a label that says whether the sentiment is positive or negative. That makes the task a clean form of review triage: the model only has to choose between two classes, not generate a summary or score a five-star scale. The same workflow also transfers to other two-way text decisions, such as routing support tickets into positive or negative tone categories, so the dataset is a concrete instance of a broader supervised classification pattern.

Load and inspect the dataset container

The first setup step is to load the corpus with datasets and inspect the object it returns. The important detail is not just that the data exists, but that it arrives as a structured container with the partitions already prepared.

from datasets import load_dataset

# Load our data
data = load_dataset("rotten_tomatoes")
data

DatasetDict({
    train: Dataset({
        features: ['text', 'label'],
        num_rows: 8530
    })
    validation: Dataset({
        features: ['text', 'label'],
        num_rows: 1066
    })
    test: Dataset({
        features: ['text', 'label'],
        num_rows: 1066
    })
})

This DatasetDict shows that the corpus already comes split for experimentation. The training partition is the one used to fit a model, the validation partition supports iteration and selection, and the test partition stays reserved for the final comparison. Keeping those roles separate is what makes later results trustworthy across direct classifiers, embedding-based methods, and prompt-based approaches.

Read the schema from actual examples

Each example contains two fields: review text and a binary label. That is the complete supervised learning schema for this chapter. The input is raw language, and the target is a class ID that names one of two sentiment categories. At this point there is no preprocessing story to solve and no feature engineering decision to make; the task definition is simply text in, sentiment class out.

Decode the label values with concrete samples

A quick look at raw examples makes the encoding unambiguous.

data["train"][0, -1]

{'text': ['the rock is destined to be the 21st century\'s new " conan " and that he\'s going to make a splash even greater than arnold schwarzenegger , jean-claud van damme or steven segal .',
  'things really get weird , though not particularly scary : the movie is all portent and no content .'],
 'label': [1, 0]}

Here, 1 stands for positive and 0 stands for negative. Those values are class identifiers, not a sentiment scale, so they should not be read as degrees of positivity or negativity. The first review reads like praise, while the second clearly sounds unfavorable, which confirms the mapping in a way that is easier to trust than the numeric labels alone.

Set evaluation boundaries before modeling begins

This inspection step is small, but it protects every method that follows. The data is already split, and those same partitions must be preserved for all model families in the chapter if the comparison is to remain fair. That means no tuning on test data, no peeking at labels beyond basic inspection, and no changing the split logic from one approach to another. A dataset can look simple and still hide skew, mislabeled cases, or leakage risks, so checking the raw structure now is the safest way to avoid invalid conclusions later. With the corpus, schema, and label encoding clear, the next sections can focus on modeling rather than on repairing the data pipeline.

Model selection and practical checkpoint choices

Checkpoint choice is not a cosmetic detail; it is part of the experiment design. Before you evaluate anything, you decide what kind of tool you want in your hands. In this chapter, the same ReviewDesk-style sentiment triage task could be solved by a model that returns a label directly, by a model that turns text into reusable vectors, or by a generative model that is asked to behave like a classifier. Those are not interchangeable routes. A direct classifier is a courier that delivers a verdict. An embedding model is more like a map: it does not answer the question for you, but it gives you a stable coordinate system that many downstream methods can use.

A useful checkpoint selection process starts with the task itself. If the goal is plain sentiment classification, a model already adapted for that kind of decision is usually the most direct baseline. If the goal is broader reuse across similarity search, clustering, or lightweight classification, an embedding model is often the better foundation. After that comes language and domain fit. A model can be strong in general and still be awkward for a particular source of text if the tone, vocabulary, or style is mismatched. Architecture matters too. Encoder-only models are often the most practical choice here because they are compact, fast, and naturally suited to producing either labels or vectors. Size then becomes a deployment question: smaller checkpoints are easier to run, easier to compare, and easier to keep responsive, while larger models may offer more raw capacity but add latency and complexity that this chapter does not need.

That is why the chapter emphasizes frozen pretrained checkpoints rather than training or fine-tuning them here. Frozen usage keeps the focus on what the pretrained model already knows and how its outputs behave in a downstream workflow. It also keeps the comparison honest. If you change weights, you have changed the model family and the experiment itself; if you keep the checkpoint fixed, you can compare workflows rather than training recipes. In other words, this section is about choosing a starting point, not claiming a final winner.

For the direct sentiment classifier, the chapter uses a sentiment-ready RoBERTa checkpoint, specifically cardiffnlp/twitter-roberta-base-sentiment-latest. It is a practical baseline because it is already specialized for sentiment-like decisions and is therefore a natural fit for a classification path that should work out of the box. It also reflects a broader pattern in applied NLP: a smaller task-specific model often gives you a better first answer than a larger generative model, especially when you care about predictability, throughput, and simple label outputs. For the embedding path, the chapter uses sentence-transformers/all-mpnet-base-v2. That checkpoint is a strong default because it balances quality with efficiency and produces vectors that are useful beyond a single task. In a workflow like ReviewDesk, those vectors can support supervised classification today and similarity-based retrieval tomorrow.

The practical lesson is simple. Choose the smallest checkpoint that still matches the task well, unless your real objective is reusable representations across several workflows. Prefer encoder-only models when you want direct, stable, low-friction baselines. Reach for a generative model later when the problem calls for flexible language output rather than compact prediction. For now, these selections define the chapter’s starting line, not its finish line.

Direct inference with a task-specific sentiment model

Why this baseline works without training

A task-specific sentiment checkpoint is already tuned to make sentiment decisions, so you do not need to train it again before using it. The workflow is therefore direct inference: load the model, point it at unseen text, and evaluate the predictions. That makes it a strong baseline when your new data is close to the domain the checkpoint was trained on, because specialized models often encode useful decision boundaries from the start.

A useful way to think about this is as a prebuilt triage model for messages or reviews. The system is not discovering the meaning of positive and negative labels from scratch. It is applying knowledge that is already inside the checkpoint, which is why this section can move quickly from setup to evaluation and still produce a meaningful benchmark for later comparisons.

Load the sentiment pipeline and inspect its output shape

Hugging Face wraps tokenization, model execution, and label scoring into a single pipeline interface. The model and tokenizer must come from the same checkpoint path so that the text preprocessing matches the classifier weights. Here the pipeline is asked to return all label scores, not just the best label, because the next step will make the class decision explicitly.

from transformers import pipeline

# Path to our HF model
model_path = "cardiffnlp/twitter-roberta-base-sentiment-latest"

# Load model into pipeline
pipe = pipeline(
    model=model_path,
    tokenizer=model_path,
    return_all_scores=True,
    device="cuda:0"
)

Returning all scores matters because it exposes the full sentiment distribution the model assigns to an input. Those values are what the code uses to decide between the binary classes. In this checkpoint, the score vector includes a negative class and a positive class, and preserving that structure makes the later argmax step easy to interpret and harder to misuse.

Score the held-out test split and turn scores into labels

The evaluation must happen on the test split, not on training data. That keeps the result honest, because the model is being judged on examples it did not use to form its sentiment boundary. The loop uses KeyDataset to stream only the text column into the pipeline, which is a compact way to connect a dataset object to inference without manually unpacking rows. tqdm simply adds progress reporting while the held-out set is processed.

import numpy as np
from tqdm import tqdm
from transformers.pipelines.pt_utils import KeyDataset

# Run inference
y_pred = []
for output in tqdm(pipe(KeyDataset(data["test"], "text")), total=len(data["test"])):
    negative_score = output[0]["score"]
    positive_score = output[2]["score"]
    assignment = np.argmax([negative_score, positive_score])
    y_pred.append(assignment)

The score vector is reduced to one class with argmax. That decision rule is deliberate: the model gives two relevant class scores, and the larger one becomes the prediction. The label order matters here, because reading the wrong score index would quietly flip the meaning of the output. In this example, the code treats the first extracted score as negative and the second as positive, then uses the larger value to choose between them.

Seen as a workflow, this is a three-step pipeline. First the classifier is loaded. Then the test split is scored. Finally the scores are translated into labels that can be compared with the true targets. That simple sequence is enough to turn a pretrained checkpoint into a complete baseline classifier.

Read the classification report as a performance summary

The classification report turns the predictions into a compact summary of quality. Precision tells you how reliable the predicted positives are. Recall tells you how many of the true positives the model manages to recover. Accuracy gives the overall share of correct predictions. F1 combines precision and recall into one number, which is useful when you want a balanced view of both kinds of error.

from sklearn.metrics import classification_report

def evaluate_performance(y_true, y_pred):
    """Create and print the classification report"""
    performance = classification_report(
        y_true, y_pred,
        target_names=["Negative Review", "Positive Review"]
    )
    print(performance)

The weighted average is especially useful when you want a single summary score that reflects the class distribution. Even so, it can hide imbalance. A model may look strong on weighted F1 while still doing noticeably better on one sentiment than the other, so the per-class rows remain important. In practice, the report gives you a quick sense of whether the classifier is broadly reliable or only strong on one side of the label space.

Use the confusion matrix to diagnose asymmetric mistakes

The confusion matrix adds a different perspective because it shows how the errors are distributed across the two classes. It makes the false positives and false negatives visible instead of folding them into one aggregate score. That matters whenever one kind of mistake is more costly than the other, or whenever the model seems to favor one label over the other.

For a binary sentiment task, the matrix can show whether the model is especially good at catching positive reviews, especially good at rejecting them, or simply uneven in both directions. If the off-diagonal cells are imbalanced, the model has an asymmetry that the classification report alone may not make obvious. That is why the matrix is best treated as a debugging tool for prediction behavior rather than as a decorative summary.

A sentiment-specific checkpoint can be strong precisely because it already matches the task. When the training domain and the target domain are close, direct inference often gives a surprisingly solid baseline. The result is not the final answer, but a reference point for the next comparisons. As the chapter moves on, that baseline will help you judge whether embeddings or prompting are actually improving performance, or only changing the style of the decision process.

From pretrained embeddings to reusable features

A pretrained embedding model can act like a universal text compressor: it turns a review, a ticket, or a document into a fixed-length vector that another model can consume. That is a different role from a generative large language model, which predicts the next token and is optimized to produce free-form text. In this section, the embedding model is frozen. Its weights do not change, and its job is not to learn the sentiment task directly. Instead, it produces a stable representation that makes the downstream classification problem easier.

That separation matters because it gives you a very practical tradeoff. If you have labeled data and want a compact, fast classifier, you can keep the encoder fixed and train a lightweight head on top. If you have little or no labeled data for the target task, you can sometimes skip training a classifier entirely and compare the document embedding to embeddings of label descriptions. Both workflows reuse the same encoder, but they solve the problem in different ways. The first learns a decision boundary from examples; the second treats label text as a prototype and asks which label phrase is closest in semantic space.

Building a supervised classifier on frozen vectors

The supervised version starts with an encode-once-then-train pattern. First, the text is converted into embeddings. Then those vectors are handed to a conventional classifier. The key point is that the classifier never sees raw text; it sees a matrix of numbers where each row corresponds to one example and each column corresponds to one dimension of the embedding space.

from sentence_transformers import SentenceTransformer

# Load model
model = SentenceTransformer("sentence-transformers/all-mpnet-base-v2")

# Convert text to embeddings
train_embeddings = model.encode(data["train"]["text"], show_progress_bar=True)
test_embeddings = model.encode(data["test"]["text"], show_progress_bar=True)

The shape of that feature matrix is part of the story.

train_embeddings.shape

(8530, 768)

Each review becomes a 768-dimensional vector. That fixed dimensionality is what makes the next step possible: the classifier receives a standard tabular input rather than an irregular sequence of tokens. In that sense, the embedding model has already done the hard language work, and the classifier is mostly learning how to separate the classes in a space that is hopefully organized by meaning.

A simple logistic regression is often enough to make this idea concrete. It is not trying to relearn language from scratch. It is drawing a boundary through the embedding space using the labeled training examples.

from sklearn.linear_model import LogisticRegression

# Train a logistic regression on our train embeddings
clf = LogisticRegression(random_state=42)
clf.fit(train_embeddings, data["train"]["label"])

# Predict previously unseen instances
y_pred = clf.predict(test_embeddings)
evaluate_performance(data["test"]["label"], y_pred)

This workflow is useful because it keeps the model head small while still allowing the encoder to carry rich semantic information. It also exposes a common reality of modern NLP systems: once the downstream model is simple, embedding quality can dominate the result. A stronger embedding model often helps more than a more elaborate classifier head. At the same time, frozen embeddings are not magic. If the encoder is generic and the task is specialized, a task-tuned model can still outperform this lightweight baseline.

What the embedding matrix is actually feeding the classifier

The phrase “embedding-based classifier” can sound more mysterious than it is. The downstream model is not operating on language in the abstract. It is operating on a table of vectors. Each row is one document, and each row has the same width because the encoder always emits the same dimensionality for a given model checkpoint. That is why a simple linear model can work at all: the input format is regular, even though the text was not.

A useful mental model is to think of the embedding encoder as a reusable feature compressor. You can swap in different heads on top of the same compressor without retraining the compressor itself. For ReviewDesk ticket triage, for example, the same encoder could feed a classifier for urgent versus non-urgent messages, or a different classifier for billing versus technical issues, as long as you have labels for the downstream task. The frozen encoder stays fixed; only the decision layer changes.

Turning labels into text prototypes

Zero-shot classification with embeddings uses the same encoder, but now the labels themselves are rewritten as short text descriptions. Instead of assigning an opaque class ID, you represent each class with a phrase such as “A negative review” or “A positive review.” Those phrases are embedded the same way as the documents, so the label text becomes a semantic anchor in the same vector space.

# Create embeddings for our labels
label_embeddings = model.encode(["A negative review",  "A positive review"])

from sklearn.metrics.pairwise import cosine_similarity

# Find the best matching label for each document
sim_matrix = cosine_similarity(test_embeddings, label_embeddings)
y_pred = np.argmax(sim_matrix, axis=1)

This is not the same as supervised classification. No classifier is trained on labeled examples. The labels function more like prototype postcards: short textual summaries of each class that compete for similarity with the document. In the ReviewDesk setting, you could imagine label descriptions such as “A critical bug report” and “A routine account question” playing the same role. The method works only if the label wording is semantically close to how the documents express the classes, which is why label phrasing matters much more here than in ordinary supervised learning.

A subtle but important advantage of this approach is that the same encoder is used for both sides of the comparison. That symmetry is what makes zero-shot matching possible. The document and the label description live in the same coordinate system, so closeness has a meaningful interpretation. You are not matching IDs; you are matching meanings.

Selecting the nearest label with cosine similarity

Cosine similarity is the scoring rule that turns the embedding comparison into a prediction. For each document, the similarity scores against all label descriptions are computed, and the label with the highest score is selected. In practice, this is just an argmax over the similarity matrix, but conceptually it is doing something very different from a learned decision boundary. The model is not adapting to the target task through training; it is searching for the label phrase that the document most naturally aligns with in embedding space.

That makes zero-shot matching appealing when you want a fast baseline or when labeled data are scarce. It also makes the method fragile in a way supervised classification is not. If one label description is vague, overly long, or semantically unbalanced compared with the others, its embedding may be disadvantaged even when the class itself is valid. Small wording changes can shift results because you are asking the encoder to judge the relationship between two pieces of text, not between a document and a learned class boundary.

When each workflow is the better fit

Supervised classification on frozen embeddings is the more dependable option when you have enough labeled examples to train a lightweight head and you want a classifier that is still cheap to run. Zero-shot label matching is attractive when labels are scarce, when you need a quick prototype, or when you want to test whether the label semantics are already well captured by the encoder. Both are lightweight compared with end-to-end fine-tuning, but they fail for different reasons. The supervised version depends on the quality of the frozen representation and the amount of labeled data available to the downstream classifier. The zero-shot version depends even more heavily on how well the label text describes the task.

For a sentiment task like the movie-review example, both methods can be evaluated with the same held-out test set and the same classification metrics, which makes them easy to compare fairly. That comparison is the real lesson: embeddings are not just a prettier way to store text. They are reusable features, and sometimes they are also a semantic space where labels themselves can act as predictors.

From labels to text: the generative-classification idea

A generative model can be used as a classifier when it is asked to produce a short text sequence that stands in for a class label. That is the central shift in this section. Instead of using a dedicated classification head that directly returns label probabilities, we give the model an instruction and let it generate a token sequence such as positive, negative, 1, or 0. The model is still making a class decision, but it is doing so through generation rather than through direct label prediction.

That difference matters because the prompt becomes part of the model interface. In a direct classifier, the architecture defines the output space. In a text-to-text or chat workflow, the wording of the instruction helps define the task itself. This is why prompt-based classification is attractive for rapid prototyping, limited labeled data, or access to a strong hosted model. It is also why the method can fail in confusing ways: the model may know the task, but the prompt may not constrain the answer format well enough.

The same pattern transfers cleanly to ReviewDesk-style triage. A support message can be wrapped in a prompt that asks for a single label such as urgent or non-urgent. The model is not becoming a different kind of classifier; it is being guided to emit the answer in text form.

Flan-T5 as a local text-to-text classifier

Flan-T5 is a good first example because it is an encoder-decoder model used locally through a text2text-generation pipeline. That makes it natural to treat the model as a text generator that can be instructed to output a sentiment label.

pipe = pipeline(
    "text2text-generation", 
    model="google/flan-t5-small", 
    device="cuda:0"
)

The prompt is attached to every review before inference. In practice, that means the dataset is mapped so each example contains the same instruction plus its own text. The prompt is short on purpose, because the point is to show the mechanism rather than to overfit the wording.

prompt = "Is the following sentence positive or negative? "
data = data.map(lambda example: {"t5": prompt + example['text']})
data

This mapping step makes the prompt part of the data pipeline. Every review now enters the model with the same instruction, so the whole test set is prepared consistently. That consistency matters because the model is not selecting from a label head; it is completing a tiny generation task whose output must later be interpreted as a class.

The batch loop runs over the test split, extracts each generated string, and converts it into a numeric label. In the example below, negative maps to 0, while anything else is treated as 1.

y_pred = []
for output in tqdm(pipe(KeyDataset(data["test"], "t5")), total=len(data["test"])):
    text = output[0]["generated_text"]
    y_pred.append(0 if text == "negative" else 1)

That conversion step is where output variability can quietly affect evaluation. A model might return negative, Negative, or negative review, and each of those can require a different parsing rule. Exact string matching is simple, but it can also be brittle. If the model gives a semantically correct answer in a slightly different format, the score can drop even though the underlying prediction was reasonable. Label normalization is therefore part of the evaluation logic, not just a cleanup step.

With the generated labels converted back to integers, the same helper used earlier in the chapter can score the results.

evaluate_performance(data["test"]["label"], y_pred)

Using the same precision, recall, accuracy, and F1 score keeps the comparison aligned with the earlier classifier and embedding workflows. A confusion-oriented view is especially helpful here because it reveals whether the model is leaning toward one class or missing one side of the decision boundary.

Running the full test set and scoring Flan-T5 outputs

The full test-set run shows the practical workflow end to end. The prompt is prepared once, attached to every example, generated outputs are collected in batch, and the returned text is converted into a class label before evaluation. That is the core pattern for prompt-based classification.

A useful way to debug this kind of system is to separate prompt quality, parsing rules, and actual model fit. If the prompt is vague, the model may answer unpredictably. If the parser is too strict, the model may be correct but still fail the metric pipeline. If both of those are fine and the score is still poor, then the model may simply not fit the task well enough. This triad is often the fastest way to isolate the real problem.

The same template also works for ReviewDesk triage. A prompt such as “Classify this ticket as urgent or non-urgent” can be prepended to each message, and the generated text can be normalized into the corresponding binary label. The specific domain changes, but the method stays the same: the model is asked to complete a label-shaped text fragment.

Using ChatGPT for the same sentiment task

The API-based version uses the same idea, but it wraps the prompt inside a chat request. That makes the workflow more operationally explicit: a reusable function inserts the document into a template, sends the messages to the API, and returns the model response.

import openai

# Create client
client = openai.OpenAI(api_key="YOUR_KEY_HERE")

def chatgpt_generation(prompt, document, model="gpt-3.5-turbo-0125"):
    """Generate an output based on a prompt and an input document."""
    messages=[
        {
            "role": "system",
            "content": "You are a helpful assistant."
            },
        {
            "role": "user",
            "content":   prompt.replace("[DOCUMENT]", document)
            }
    ]
    chat_completion = client.chat.completions.create(
      messages=messages,
      model=model,
      temperature=0
    )
    return chat_completion.choices[0].message.content

Temperature is set to zero to reduce run-to-run variation. That does not make every service path perfectly deterministic, but it does make evaluation more stable than a higher-temperature setting would. For classification, stability matters more than diversity, because the goal is to measure the prompt and model against fixed labels.

The prompt template is designed to force a label-only response. The [DOCUMENT] placeholder makes the function reusable, and the explicit instruction narrows the output space so the model behaves more like a classifier.

prompt = """Predict whether the following document is a positive or negative movie review:

[DOCUMENT]

If it is positive return 1 and if it is negative return 0. Do not give any other answers.
"""

document = "unpretentious , charming , quirky , original"
chatgpt_generation(prompt, document)

That same pattern transfers directly to a different binary task. For ReviewDesk-style triage, the template can be adjusted to ask for urgent or non-urgent while keeping the same label-only structure. The important part is not the domain wording itself, but the fact that the output format is tightly constrained.

To score the whole test set, the wrapper is applied example by example. This is where API-based classification becomes operationally different from local inference. Rate limits can slow the run, retries may be needed if requests fail, and every prediction has a cost. Those realities affect how practical the evaluation is, especially when the test set is large.

predictions = [
    chatgpt_generation(prompt, doc) for doc in tqdm(data["test"]["text"])
]

The returned strings are then converted into integers before the same evaluation helper is called.

y_pred = [int(pred) for pred in predictions]

evaluate_performance(data["test"]["label"], y_pred)

Parsing matters here as well. If the model returns exactly 1 or 0, conversion is easy. If it returns positive, negative, or a longer sentence, then the parser needs to normalize those variants before evaluation. Otherwise the metric may reflect formatting noise rather than model behavior. In practice, a strict output rule plus a careful parser often improves score stability more than changing the model does.

Practical limits and interpretation

Prompt-based classification is useful when you want a fast way to reuse a strong generative model for a label task, but the method is only as good as the prompt, the parser, and the model itself. A weak score may come from vague instructions, brittle normalization, or genuine model disagreement with the task. Debugging is easier when those causes are considered separately.

Closed-source models add another constraint: their training data, update cadence, and internal behavior are not visible in the way a local model is. That makes results harder to interpret, even when the metrics look strong. The method is still valuable, but it should be treated as a prompt-driven labeling protocol rather than a transparent classifier head.

Reconstructing the chapter as one decision tree

Text classification in this chapter can be read as one task with three control knobs. The goal stayed the same: assign labels to text under a shared evaluation setup. What changed was the adaptation path. You could use a task-specific pretrained classifier, reuse embeddings and train a downstream classifier, or ask a generative model for the label through a prompt. Seen this way, the methods are not separate problems but different answers to the same label-prediction question.

Compare the three workflows by where the signal comes from

The direct classifier puts the model choice front and center. The embedding-based pipeline shifts the work to the representation and the lightweight classifier on top. The prompt-based route changes the wording of the request instead of adding a new training stage. That is why the chapter’s methods can be remembered as label-first, vector-first, and prompt-first. Flan-T5 and GPT-style models fit the last category by turning generation into classification without task-specific retraining.

Why the same labeled dataset supports multiple evaluation paths

The same labeled dataset can support all three workflows, which makes comparison straightforward and fair. Each method sees the same examples and the same ground truth labels, so differences in results reflect the method choice rather than a different benchmark. This gives a practical decision lens based on simplicity, flexibility, and dependence on labels or prompts.

The practical takeaway and the next problem setting

The chapter’s core lesson is that you choose among different levers, not different tasks. If labels are available, the question is how best to predict them. If labels are absent, the problem changes shape entirely. That is where the next chapter begins, with clustering and topic modeling as ways to discover structure instead of assigning known categories.

Use the url below to download the enitre book as pdf:

Read more

Raw text is convenient for people and awkward for models. A support ticket in ReviewDesk might arrive as a polished paragraph, but the model cannot use that paragraph directly. It first needs the text broken into tokens, then converted into token IDs, and then mapped into embeddings. That progression is the basic representation pipeline of modern language systems, and it explains why a small change in preprocessing can affect generation quality, search quality, and even cost.

📘 Buy the Entire Book – Available on Amazon Worldwide (If you are paid subscriber, use the url at the end of this article to download the book for free!)

You can now purchase the complete book directly from Amazon in your country:

• United States (US)

• United Kingdom (UK)

• Germany (DE)

• France (FR)

• Spain (ES)

• Italy (IT)

• Netherlands (NL)

• Japan (JP)

• Brazil (BR):

• Canada (CA)

• Mexico (MX)

• Australia (AU)

• India (IN)

  ---

A useful mental model is simple: tokens are the pieces, token IDs are the labels, and embeddings are the coordinates. A token is a text fragment produced by tokenization, such as a word, part of a word, punctuation mark, or special symbol. A token ID is just the integer label assigned to that fragment by a tokenizer vocabulary. An embedding is a learned numeric vector that places that token, or a larger unit of text, in a space where related items sit near one another. The ID identifies a symbol in a vocabulary; it does not carry meaning by itself. The embedding is where meaning becomes computable.

This distinction matters because models do not reason over raw strings. They reason over structured inputs. In generation, the model consumes token IDs, looks up token embeddings, and transforms them into internal states that drive the next prediction. In retrieval-style systems, the same general idea appears in a different form: text is turned into a vector representation so the system can compare meanings rather than match exact words. A ticket about a “login loop” and another about “cannot sign in” may share little surface overlap, yet their embeddings can still land nearby if they describe the same underlying problem.

That is why representation choices shape behavior so strongly. If the model sees the wrong token boundaries, it may spend extra capacity recovering a phrase that should have been simple. If the embedding space is poorly organized, semantically similar items will not cluster together, and search or recommendation will feel brittle. In other words, the model’s input representation is not a neutral prelude to intelligence; it is part of the intelligence.

It is also important to separate token embeddings from text embeddings. Token embeddings live inside a model and correspond to individual tokens. They are the learned vectors the model uses as it processes a sequence. Text embeddings are usually fixed-size vectors for an entire sentence, ticket, paragraph, or document, designed for similarity search, clustering, classification, or recommendation. A token embedding is like a local coordinate for one piece of text inside a model’s internal machinery. A text embedding is more like a compact map of the whole item, suitable for comparing one item to another.

This chapter will show how that representation idea extends far beyond language generation. Word2vec made the embedding concept famous by learning useful vector neighborhoods from co-occurrence patterns, long before today’s large language models dominated the field. The same broad idea now powers semantic search over help-center articles, clustering of similar support requests, and recommendation systems that suggest related items based on vector proximity. The surface domain changes, but the workflow is the same: turn objects into coordinates, then use geometry to answer practical questions.

For ReviewDesk, that means a ticket can move from raw text to tokens, from tokens to IDs, and from IDs to vectors that support routing, search, and later analysis. The common mistake is to treat every embedding as interchangeable. A token embedding, a contextual hidden state, and a sentence embedding all live in vector space, but they are not the same object and they do not serve the same role. Keeping that distinction clear will make the next sections much easier to follow.

From Text to Token IDs

The model never sees raw text

A language model does not read your prompt as plain text. It receives token IDs, which are numbers assigned by a tokenizer. That is the first important boundary in the workflow: the visible sentence you type is not the model input. The tokenizer turns that sentence into a sequence of numeric IDs, and the model generates new IDs in response. Decoding turns those IDs back into text after generation.

A good way to think about this is as a translation layer. Human text goes in, machine-readable inventory codes come out, and those codes are what the model actually processes. In a support-ticket workflow, for example, the message a user types is only the starting point. The model sees the tokenized version of that message, not the original characters.

It also helps to keep four terms separate. Text is the string you type. Tokens are the pieces the tokenizer creates. Token IDs are the numbers attached to those pieces. A decoded string is what you get when token IDs are converted back into readable text. Those are related forms of the same content, but they are not interchangeable.

Tracing the encode step end to end

The practical starting point is to load a pretrained model with its matching tokenizer. That pairing matters because tokenization is checkpoint-specific rather than universal.

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    "microsoft/Phi-3-mini-4k-instruct",
    device_map="cuda",
    torch_dtype="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct")

Once the pair is loaded, the encode step converts a prompt into input_ids. That tensor is the model input. The model does not consume the visible prompt string directly; it consumes the numeric IDs produced by the tokenizer.

prompt = "Write an email apologizing to Sarah for the tragic gardening mishap. Explain how it happened.<|assistant|>"

# Tokenize the input prompt
input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to("cuda")

If you inspect input_ids, you can see how much context the prompt occupies and where the tokenizer placed its boundaries. This is useful in practice because two prompts that look similar to a person can take very different numbers of tokens. That difference affects how much room remains in the context window for the actual task.

The encode step is also where subword tokenization shows up. A tokenizer may keep a common word intact, split a rarer word into pieces, or separate punctuation and whitespace in ways that are not obvious from the rendered sentence. The result depends on the tokenizer’s vocabulary, special-token rules, and training domain.

Tracing the decode step and the generation loop

Generation happens in token space. The model predicts the next token ID, appends it to the sequence, predicts another ID, and repeats until it reaches a stopping condition such as a token limit or an end marker. That is autoregressive generation: the model extends the output one token at a time rather than producing a finished sentence all at once.

# Generate the text
generation_output = model.generate(
  input_ids=input_ids,
  max_new_tokens=20
)

# Print the output
print(tokenizer.decode(generation_output[0]))

The decoded output is easier for a person to read, but it is not always a perfect visual mirror of what the model produced. The returned sequence may include the original prompt tokens, special markers, or formatting artifacts that were important to the model during inference. In a chat-style checkpoint, a marker such as <|assistant|> is structural input, not ordinary prose. It tells the model which role is expected to speak next.

That is why prompt templates matter. Copying a chat prompt from one checkpoint to another without checking its expected special tokens can change behavior even when the visible text looks correct. The prompt may still read naturally to you while being malformed for the model.

Reading token boundaries one piece at a time

A per-token decode loop is a simple way to inspect segmentation. It reveals where one token ends and the next begins, which is especially helpful when a visible word becomes several pieces.

for id in input_ids[0]:
   print(tokenizer.decode(id))

This kind of inspection makes subword tokenization concrete. A common word may appear as a single token, while a rarer word, a hyphenated phrase, a digit sequence, or punctuation may split across multiple IDs. Whitespace can also be handled differently across tokenizers, so the same text can look clean on the page while being packed in an unexpected way underneath.

Special tokens are part of that structure too. They are reserved symbols for roles, separators, padding, or sequence boundaries. They are not ordinary words, and they should be treated as model-facing formatting rather than prose. If they are omitted or misplaced, the visible prompt may still look reasonable while the model behavior changes.

Same sentence, different tokenizer, different result

The same sentence can tokenize differently across pretrained checkpoints. That is the main practical lesson: tokenization is model-specific, not universal. Different tokenizers reflect different vocabularies, special-token conventions, capitalization choices, and training domains.

colors_list = [
    '102;194;165', '252;141;98', '141;160;203', 
    '231;138;195', '166;216;84', '255;217;47'
]

def show_tokens(sentence, tokenizer_name):
    tokenizer = AutoTokenizer.from_pretrained(tokenizer_name)
    token_ids = tokenizer(sentence).input_ids
    for idx, t in enumerate(token_ids):
        print(
            f'\x1b[0;30;48;2;{colors_list[idx % len(colors_list)]}m' + 
            tokenizer.decode(t) + 
            '\x1b[0m', 
            end=' '
        )

This comparison becomes most revealing when the sentence contains whitespace, punctuation, digits, code indentation, emojis, or multilingual text. A model trained for general conversation may split those patterns differently from a code-oriented or domain-specific checkpoint. Code-focused tokenizers often preserve indentation and spacing more carefully because those details matter in programming text. Prose-oriented tokenizers may fragment the same structure more aggressively.

Digits can also surprise you. Some tokenizers keep number strings compact, while others break them into smaller pieces. Emojis and non-English text vary as well, depending on how broad the tokenizer’s Unicode coverage is and how much of its training data came from those forms of text. The point is not that one behavior is universally better. The point is that the same visible sentence can be packaged differently depending on the checkpoint.

In practical terms, those differences affect context usage, cost, and output quality. If you pay by token in an API setting, a more fragmented prompt costs more. If you are working with a fixed context window, a longer token sequence leaves less room for instructions, examples, or conversation history. Tokenization also affects reproducibility: a prompt template that works with one model may not behave the same way with another because the special tokens and boundary rules are different.

Why tokenization choices matter in practice

Tokenization is therefore part of the model interface, not just a preprocessing step. Choosing a model also means choosing its tokenizer contract, and that contract shapes how your text is packaged for inference. The practical lesson is simple: the model operates on token IDs, not on raw characters or words, and the path from text to IDs is specific to the checkpoint you use.

Token Embeddings and Contextual Vectors

From token IDs to learned embedding lookups

Once text has been tokenized, the model no longer sees ordinary words. It receives token IDs, and those IDs are just integer addresses into the vocabulary. An ID is not a meaning in itself. A good mental model is a filing cabinet: the label on the drawer is not the document inside it. The label tells the model where to look, and the contents of that location are the learned token embeddings.

Those embeddings live in the model as an embedding matrix, a parameter tensor with one row for each vocabulary item. During training, the model learns those rows so they become useful starting representations for the tokens it can read. This is why token IDs matter so much: they do not carry semantics directly, but they determine which row of the embedding table is selected.

That distinction is important when you move from text on the page to model internals. The tokenizer turns the input into discrete symbols, and the model turns those symbols into dense vectors that can be processed by later layers. At this stage, the model is not yet classifying, summarizing, or reasoning about the text. It is simply mapping token IDs into learned vector space.

What is stored in the embedding matrix

The embedding matrix is shared across every occurrence of a token. If the same token appears in two different texts, the same row is used as its initial representation each time. That row is learned, not hand-designed, and it belongs to the model checkpoint rather than to the raw text.

This shared lookup is powerful, but it is also limited. A static token embedding does not change when the surrounding words change. The token for a word or subword piece always starts from the same vector. What changes later is the model’s contextual output for that token, not the stored row itself.

You can think of the embedding matrix as a vocabulary-sized table of reusable starting points. The model assigns each token an address in that table, and the rest of the network can reshape those starting points once the full sequence has been read. That is why the embedding matrix is a learned parameter tensor, while the output of a forward pass is a context-sensitive representation.

Why the tokenizer and model must match

A pretrained checkpoint depends on the tokenizer it was trained with. The vocabulary, subword segmentation, and special tokens are part of the model’s expected interface. If the tokenizer changes, the token IDs change too, and the lookup no longer lands on the rows the model learned to use.

This coupling is not a minor implementation detail. It is the bridge that makes the tokenizer-model workflow work at all. A pretrained model expects its original tokenization scheme, and that is why tokenizer and model are best treated as a matched pair.

Context turns static token vectors into contextual outputs

The same token can point to different meanings depending on context. Consider a word like “charge.” In “an extra charge appeared on the bill,” the surrounding words suggest a billing sense. In “the battery charge dropped overnight,” the same token appears in a different setting. The embedding row used to start both cases is the same, but the vector that comes out of the model can differ because the context differs.

This is the key distinction to keep straight. A token embedding is the learned lookup vector associated with a token ID. A contextual token vector is the model’s output for that token after the surrounding sequence has influenced it. The first is static and shared. The second is dynamic and sequence-dependent.

That is also why these vectors are useful later in the chapter. For tasks such as tagging or summarization, the model needs representations that reflect how each token is used in context, not just which token was present. The model output preserves token-level structure while carrying information from the whole sequence.

It is tempting to inspect a vector and search for a neat human meaning in each dimension. That usually does not work. Individual coordinates are not reliable feature labels. These vectors are better understood as points in a learned space, where meaning comes from relationships and transformations rather than from single values.

Inspecting the forward pass: shapes, special tokens, and decoded tokens

The small example below shows the basic workflow: load a tokenizer and model, tokenize a short input, run a forward pass, and inspect the outputs. The point is not the checkpoint names themselves. The point is to see how token IDs become contextual token vectors and how you can verify that process with simple shape inspection.

from transformers import AutoModel, AutoTokenizer

# Load a tokenizer
tokenizer = AutoTokenizer.from_pretrained("microsoft/deberta-base")

# Load a language model
model = AutoModel.from_pretrained("microsoft/deberta-v3-xsmall")

# Tokenize the sentence
tokens = tokenizer('Hello world', return_tensors='pt')

# Process the tokens
output = model(**tokens)[0]

If you check output.shape, the dimensions tell you how the forward pass is organized. A shape such as (1, 4, 768) reads as batch size, sequence length, and embedding width. The first number says how many examples are in the batch. The second says how many token positions the model is returning. The third says how wide each contextual token vector is. Shape inspection is a useful debugging habit because it confirms wiring, even though it does not by itself explain semantics.

output.shape

You can also decode the token IDs to see the actual sequence the model received. That list may include special tokens, which are bookkeeping markers rather than ordinary words, and it may include subword pieces instead of full words. Seeing those tokens directly helps explain why a decoded list sometimes looks different from the original sentence. The model sees the tokenized sequence, not the human-friendly string alone.

for token in tokens['input_ids'][0]:
    print(tokenizer.decode(token))

Why these vectors matter later

These outputs are not final predictions, and they are not sentence embeddings. They are contextual token vectors that keep token-level detail while carrying information from the entire input. That makes them the right internal representation for many downstream sequence tasks, and it also sets up the next chapter’s discussion of Transformer internals, where these context-sensitive vectors are produced and transformed.

Fixed-Size Text Embeddings

From Token Vectors to One Text Vector

Token embeddings are useful when the model needs to reason about individual words or subword pieces, but many practical tasks ask a different question: what does this whole sentence, ticket, paragraph, or document mean? A text embedding answers that question with a single fixed-size vector. Instead of keeping one vector per token, the representation compresses the entire text into one reusable object whose position in vector space reflects semantic meaning. If two support tickets describe the same outage in different words, their text embeddings should land near each other even when the surface wording differs. That is the key shift from token-level representation to text-level representation.

A helpful way to think about this is as a semantic photocopier. The model does not preserve every token in visible form; it preserves meaning in a compact vector form that is easy to compare later. For a ReviewDesk-style workflow, that means an incoming message about a refund delay can be turned into one vector and compared against vectors for resolved cases, knowledge-base articles, or ticket categories. The value is not just that the text becomes numeric. The value is that similar texts become nearby vectors, which makes downstream matching possible.

How a Model Produces Text Embeddings

There are two common ways to get a text embedding. The practical default is to use a model designed specifically for representation learning, such as a sentence-transformer. These models are built to map text directly into useful semantic vectors. A simpler mental model is to imagine aggregating token embeddings, such as averaging them across the sequence. That intuition is helpful, and it explains why a sentence can be collapsed into one vector, but in real applications specialized embedding models usually work better because they are trained for that exact purpose.

This distinction matters because producing an embedding is not the same thing as deciding what to do with it. Generating the vector is only the representation step. Using that vector for semantic search, categorization, clustering, or retrieval is the pipeline step that follows. In other words, the model produces the map, and the application decides how to navigate with it. A good embedding by itself is not the solution; it becomes useful when paired with a task that can exploit similarity.

Loading a Pretrained Sentence-Transformer

The easiest way to see the idea in action is to load a pretrained sentence-transformer and encode a sentence. Notice that this is not training from scratch. The model already knows how to place related texts near each other in embedding space, and your job is simply to reuse that capability.

from sentence_transformers import SentenceTransformer

# Load model
model = SentenceTransformer("sentence-transformers/all-mpnet-base-v2")

# Convert text to text embeddings
vector = model.encode("Best movie ever!")

That single call turns a sentence into a vector that can later be compared with other vectors. In a support workflow, you might encode the message and then search for the nearest resolved ticket or the most relevant help article. In a recommendation workflow, you might compare a query vector with item vectors and surface the closest matches. The model does not know whether you will use the vector for search, ranking, or classification; it only produces the representation.

Checking the Shape of the Output

The most immediate sanity check is the vector shape.

vector.shape

(768,)

This is what fixed-size means in practice. The text may be a short phrase or a long paragraph, but the output is one vector with a constant dimensionality. The exact size is model-dependent, not universal. Here it happens to be 768, but another embedding model may produce 384, 1024, or a different size altogether. Treat the dimensionality as a property of the model, the same way you would treat an API response schema as part of the contract.

Where Text Embeddings Fit in Real Systems

Fixed-size embeddings are especially attractive when you need comparison, storage, or indexing. They are compact enough to cache, easy to pass into a classifier, and convenient for nearest-neighbor search. That makes them a natural bridge into semantic search and retrieval-augmented generation, where an input query is embedded, similar documents are retrieved, and those documents are then supplied to a generative model. The embedding is not the retrieval system, and it is not the RAG system, but it is often the representation layer that makes both possible.

The same idea also appears in categorization. A support message embedding can feed a label classifier, or it can be compared against labeled exemplars to predict a category. In both cases, the text is first transformed into a vector that captures meaning at the document level rather than the token level. That broader representation is what allows systems to work on intent, theme, and similarity instead of exact word overlap.

Word Embeddings and Word2vec Intuition

From Learned Vectors to Pretrained Word Embeddings

Embeddings turn meaning into geometry. Words that behave similarly in text can end up near one another in a vector space, and that neighborhood structure becomes useful for search, clustering, retrieval, and language understanding. Here the focus is on pretrained word embeddings: standalone vectors learned ahead of time on large corpora and then reused in other systems.

That is different from the token embeddings inside a Transformer, which are part of a larger model and are learned as one component of that model. Pretrained word embeddings are reusable representations that can be inspected directly. They are not the final prediction for a task; they are coordinates in a learned space that can support similarity checks before any larger model enters the picture.

Loading and Probing a Real Embedding Model

A pretrained embedding model becomes concrete when you can load it and inspect it immediately. The following example uses Gensim’s model zoo to load a compact GloVe embedding set. This is not training; it is retrieval of a pretrained resource that already contains a vocabulary and a vector table.

import gensim.downloader as api

# Download embeddings (66MB, glove, trained on wikipedia, vector size: 50)
# Other options include "word2vec-google-news-300"
# More options at https://github.com/RaRe-Technologies/gensim-data
model = api.load("glove-wiki-gigaword-50")

Nearest-neighbor inspection gives a quick diagnostic view of the space. The query below asks which words are closest to king in the learned geometry.

model.most_similar([model['king']], topn=11)

[('king', 1.0000001192092896),
('prince', 0.8236179351806641),
('queen', 0.7839043140411377),
('ii', 0.7746230363845825),
('emperor', 0.7736247777938843),
('son', 0.766719400882721),
('uncle', 0.7627150416374207),
('kingdom', 0.7542161345481873),
('throne', 0.7539914846420288),
('brother', 0.7492411136627197),
('ruler', 0.7434253692626953)]

The ranking is the useful part. It suggests that royal and family-related terms sit in the same neighborhood, which is evidence that the embedding space has learned structure. This is only an illustrative check, not a guarantee of perfect semantic quality, but it is a practical way to see whether the space behaves like a map rather than a random scatter of points.

How Word2vec Turns Text into Training Pairs

Word2vec is the classic training recipe that makes these neighborhoods learnable from raw text. Instead of starting with fixed vectors, it learns them from co-occurrence. A sliding window moves across a corpus and builds center-context pairs. If the sentence is “the refund was processed quickly,” then a window around refund might produce positive pairs such as refund with processed and refund with quickly, depending on the chosen window size.

This converts unlabeled text into supervision. Every local neighborhood in the corpus becomes a small training example that says which words appeared together. The model is not told that two words are synonyms. It only sees repeated patterns of nearby usage and learns from those patterns.

Word2vec is still static, though, and that matters. A single word can appear in many different contexts, but the method gives that word only one vector. As a result, polysemous words can blur together. Charge in a payments context and charge in a legal or battery context may be pulled into one representation, which is one reason contextual embeddings later became so important.

Skip-Gram and Negative Sampling as a Learning Game

The simplest word2vec story is skip-gram. In skip-gram, the center word predicts surrounding words. If refund is the center word, the model should score nearby words such as payment, charge, or processed highly when they truly appear in the context window. The direction matters: center word to context word, one pair at a time.

Negative sampling gives the model its contrast. It is easiest to think of this as a spot-the-impostor game. For each real center-context pair, the model also sees random mismatches, such as refund with saxophone or delay with galaxy. The training goal is to score the true pair higher than the fake ones. That extra contrast is important because rewarding only positive pairs would not teach the model what to separate, and the space could become uninformative.

So negative samples do more than add noise. They force the embedding space to learn a boundary between related and unrelated words. This positive-versus-negative pattern is the same broad recipe that later appears in contrastive learning.

Why Embedding Neighborhoods Are Useful but Imperfect

Nearest neighbors are a useful diagnostic, but they are not a full evaluation. When the neighbors look sensible, the space is capturing some structure of usage. When they look odd, the model may be reflecting corpus bias, frequency effects, or surface-form artifacts. A nearest-neighbor list tells you that the space has organization, not that every top match is a perfect semantic partner.

Static word embeddings also compress multiple senses into one vector, so a neighborhood can mix related meanings instead of separating them cleanly. That limitation is one reason later contextual models matter. The practical lesson is simple: good neighbors are helpful, but usefulness is judged by downstream fit, not by proximity alone.

The Broader Pattern: Pairwise Similarity as a Reusable Idea

The deeper lesson of word2vec is not limited to words. It is a pairwise learning pattern: bring positives together and push negatives apart. That recipe reappears in retrieval systems, recommendation pipelines, and multimodal matching, where the items may be products, songs, images, or captions instead of words.

That is why this section matters as a bridge. Word2vec shows how co-occurrence can be turned into geometry, how negative sampling makes the geometry discriminative, and how the same training intuition later becomes a general contrastive strategy across modern machine learning.

Embedding-Based Recommendation with Playlists

Why Playlists Behave Like Sentences

Embeddings are not limited to language. Any domain that produces repeated local co-occurrence can support the same learning pattern. Playlists are a clean example because songs that appear near each other often share context such as mood, genre, artist, or listening habit. If you treat each playlist as a sentence and each song as a token, then the training signal becomes familiar: items that repeatedly occur in similar neighborhoods should end up close in vector space.

That analogy matters at the level of training, not just intuition. The model is not learning music theory or human intent. It is learning from ordered sequences, so the song IDs in a playlist function as the internal tokens that carry the co-occurrence signal. Titles and artists are not part of training; they are only the labels we later use to present the result. This is why the same embedding idea can also work for a ReviewDesk-style workflow where similar tickets appear together in a support stream.

Preparing Playlist and Song Metadata Data

The first step is to load the playlist corpus and the song metadata table. The playlists are parsed into ordered lists of song IDs, because Word2Vec needs sequence data rather than isolated rows. Playlists with a single song are removed because they do not provide any surrounding context, so they contribute no useful co-occurrence information. That filtering keeps the input format aligned with the sentence analogy: a one-word sentence does not teach the model anything about neighborhood relationships.

The metadata table plays a separate but essential role. It maps each internal song ID to a human-readable title and artist. Without that lookup layer, the embedding model would only return anonymous IDs, which are useful to the algorithm but not to a person reading the output. In practice, the vector model and the metadata table work together: one learns similarity, and the other makes the similarity interpretable.

import pandas as pd
from urllib import request

# Get the playlist dataset file
data = request.urlopen('https://storage.googleapis.com/maps-premium/dataset/yes_complete/train.txt')

# Parse the playlist dataset file. Skip the first two lines as
# they only contain metadata
lines = data.read().decode("utf-8").split('\n')[2:]

# Remove playlists with only one song
playlists = [s.rstrip().split() for s in lines if len(s.split()) > 1]

# Load song metadata
songs_file = request.urlopen('https://storage.googleapis.com/maps-premium/dataset/yes_complete/song_hash.txt')
songs_file = songs_file.read().decode("utf-8").split('\n')
songs = [s.rstrip().split('\t') for s in songs_file]
songs_df = pd.DataFrame(data=songs, columns = ['id', 'title', 'artist'])
songs_df = songs_df.set_index('id')

Inspecting Sequences Before Training

A quick preview helps confirm that the preprocessing worked as intended. Each playlist should look like an ordered sequence of song IDs, not a flattened list of metadata fields. This small check is useful because the model will happily train on whatever structure it is given, even if the structure is wrong.

print( 'Playlist #1:\n ', playlists[0], '\n')
print( 'Playlist #2:\n ', playlists[1])

Learning Song Vectors with Word2Vec

Word2Vec fits this problem because it learns vectors from co-occurrence, not from explicit labels. In this setting, the model scans each playlist and learns which song IDs tend to appear in similar local contexts. The window size controls how far the model looks around each song in the sequence, so it directly expresses the idea of local playlist context. A larger window allows broader neighborhood signals, while a smaller window focuses on tighter transitions between adjacent songs.

The other settings are practical choices. The vector size sets how large each song representation will be. Negative sampling makes training efficient by contrasting true neighbors with distractors. The minimum count keeps rare items from dominating the embedding space, and the worker count uses parallelism to speed up training. Together, these settings define a reusable item embedding space that captures proximity from co-occurrence rather than from manual labels.

from gensim.models import Word2Vec

# Train our Word2Vec model
model = Word2Vec(
    playlists, vector_size=32, window=20, negative=50, min_count=1, workers=4
)

From Similar Songs to Usable Recommendations

After training, the workflow becomes retrieval over the embedding space. Given a song ID, the model searches for nearby song vectors and returns the closest neighbors. That is the key distinction to keep in mind: this is an embedding retrieval workflow, not generation. The system is not inventing new content token by token; it is selecting items that are already present in the learned space.

song_id = 2172

# Ask the model for songs similar to song #2172
model.wv.most_similar(positive=str(song_id))

The raw neighbor list still needs interpretation, so the metadata table is used to map the song ID back to a title and artist. That lookup turns a vector-space result into something a user can read. The model stores only internal identifiers, but the display layer translates those identifiers into a product-facing recommendation.

print(songs_df.iloc[2172])

A small helper function makes that pattern reusable. It asks for the top neighbors of a song ID, extracts their IDs, and returns the matching metadata rows as a table of recommendations. This is the simplest form of an application-ready interface built on embeddings: retrieve nearest neighbors first, then resolve them through metadata.

import numpy as np

def print_recommendations(song_id):
    similar_songs = np.array(
        model.wv.most_similar(positive=str(song_id),topn=5)
    )[:,0]
    return  songs_df.iloc[similar_songs]

# Extract recommendations
print_recommendations(2172)

The output is only as trustworthy as the data and training setup behind it. Neighbors can look visually plausible because they share genre or playlist context, but that does not by itself prove they are validated recommendations. Results can also change with different random initialization, different corpus slices, or different preprocessing decisions, so the sample output should be read as an illustration of the retrieval pattern rather than a fixed guarantee.

What This Example Does and Does Not Prove

This example shows how an embedding model can turn co-occurrence sequences into a similarity structure that supports recommendation retrieval. The same pattern can be applied to songs, products, tickets, or other items that appear in ordered contexts. What it does not show is text generation or a full recommender system. It shows embedding-based retrieval: learn vectors from local context, then use nearest neighbors and metadata lookup to make the results usable.

Chapter Recap and Bridge to Transformers

This chapter has built a layered map from raw text to useful representations. The sequence begins with text, which is broken into tokens by a tokenizer. Those tokens are converted into token IDs, which are the numeric codes a model can actually process. Inside the model, each token ID is mapped to a token embedding, a learned vector from the model’s internal lookup table. After that, the surrounding words reshape those vectors into contextual token vectors, so the representation of a token changes with the sentence around it. At the other end of the spectrum are text embeddings, which compress an entire sentence, paragraph, or document into a single fixed-size vector that can be compared with other texts.

That hierarchy matters because each layer serves a different purpose. Token embeddings are part of the model’s machinery; they help the model turn symbolic input into something it can compute with. Text embeddings are standalone representations; they are designed for similarity, search, clustering, and classification pipelines that need a reusable vector for the whole input. It is easy to blur those ideas, especially because both are called embeddings, but they live at different levels of the stack. A token embedding is like a row in the model’s internal codebook, while a text embedding is more like a compact map of the meaning of the entire passage.