Rust's Iterator Adapters and flat_map for Pair Signal Computation

Lazy pair generation, spread scoring, and the upstream/downstream classifier — zero allocation before the gate.

When must you generate all possibilities, and when must you generate only the one you need?

The answer is architecture. A loop that generates every candidate before filtering any of them has made a decision about memory: all candidates live simultaneously. An adapter chain that generates one candidate, tests it, and discards it before generating the next has made the opposite decision: only the current candidate exists. For a 500-symbol universe and 124,750 candidate pairs, those two architectures allocate at ratios of 124,750-to-1.

The Ops lesson defined the pair-signal protocol and the data contract. This lesson implements both in Rust, using the iterator adapter system that Blandy and Orendorff ground in Chapter 15 (pp. 343–358). The language's iterator protocol is the natural shape for this computation. The flat_map adapter generates pairs lazily. The filter_map adapter gates and transforms in the same pass. The collect() terminal materializes only the winners.

§ IFrame: The Cartesian Stream and Classify-as-You-Go

Two coined terms name the architecture this lesson turns on.

The cartesian stream is the lazy pair generator. Given a slice of N symbols, the cartesian stream produces pairs in C(N, 2) = N(N-1)/2 order, one at a time, without allocating a Vec of all pairs first. Each pair is generated on demand when the downstream adapter calls next().

The classify-as-you-go discipline is the practice of testing and scoring inside the adapter chain — not after collection. Every gate runs in the stream. Pairs that fail any gate vanish from the chain without ever touching heap-allocated storage in a collection.

§ IIThe Iterator Protocol

The Iterator trait requires a single method:

pub trait Iterator {
    type Item;
    fn next(&mut self) -> Option<Self::Item>;
}

Every adapter wraps another iterator. flat_map takes a closure that maps each item to an inner iterator and flattens those inner iterators into a single stream. filter_map takes a closure returning Option<B>: Some(b) enters the stream, None is silently discarded. The adapter stack is compiled into a tight loop — no vtable dispatch, no runtime indirection.

§ IIIData Structures

use std::collections::BTreeMap;

#[derive(Debug, Clone, PartialEq, PartialOrd)]
pub struct Symbol(pub String);

#[derive(Debug, Clone, Copy, PartialEq)]
pub enum ChainPosition {
    Upstream,
    Downstream,
}

#[derive(Debug)]
pub struct PairSignal {
    pub long: Symbol,
    pub short: Symbol,
    pub zscore: f64,
    pub adf_pvalue: f64,
    pub long_position: ChainPosition,
    pub short_position: ChainPosition,
}

The ChainPosition enum names the input/output long-short structure from the Ops lesson. A PairSignal always carries long_position: Upstream and short_position: Downstream. The invariant lives in the type system, not in comments. A BTreeMap stores parameters rather than a HashMap: lexicographic ordering produces deterministic iteration across runs, which matters for audit trails in a financial pipeline.

§ IVThe Cartesian Stream

pub fn generate_pair_signals(
    symbols: &[Symbol],
    store: &CointegrationStore,
    current_prices: &BTreeMap<String, f64>,
    entry_zscore: f64,
    adf_threshold: f64,
) -> Vec<PairSignal> {
    symbols
        .iter()
        .enumerate()
        .flat_map(|(i, long)| {
            symbols[i + 1..].iter().map(move |short| (long, short))
        })
        .filter_map(|(long, short)| {
            let params = store.lookup(long, short)?;
            if params.adf_pvalue >= adf_threshold {
                return None;
            }
            let long_price = current_prices.get(&long.0)?;
            let short_price = current_prices.get(&short.0)?;
            let spread = short_price - params.beta * long_price;
            let zscore = (spread - params.spread_mean) / params.spread_std;
            if zscore.abs() < entry_zscore {
                return None;
            }
            Some(PairSignal {
                long: long.clone(),
                short: short.clone(),
                zscore,
                adf_pvalue: params.adf_pvalue,
                long_position: ChainPosition::Upstream,
                short_position: ChainPosition::Downstream,
            })
        })
        .collect()
}

§ VSorting by Signal Strength

pub fn rank_signals(mut signals: Vec<PairSignal>) -> Vec<PairSignal> {
    signals.sort_by(|a, b| {
        b.zscore
            .abs()
            .partial_cmp(&a.zscore.abs())
            .unwrap_or(std::cmp::Ordering::Equal)
    });
    signals
}

partial_cmp returns Option<Ordering> because f64 includes NaN, which is neither greater nor less than any value. The .unwrap_or(Equal) handles the NaN case without panic. In production, a PairSignal with a NaN z-score indicates a data-pipeline fault and should be logged and dropped rather than sorted to equal.

§ VIThe Classify-as-You-Go Discipline

State the principle plainly. Every gate runs inside the stream. No intermediate Vec holds pairs that have passed some gates but not all. No two-pass design collects all cointegrated pairs into one Vec and then filters by z-score.

The reason this matters beyond memory: intermediate collections create stale intermediate state. A Vec<CointegrationResult> populated at market open and held through the session carries parameters calibrated to the open snapshot. The classify-as-you-go discipline computes the spread and z-score in the same pass using the same current_prices map. There is no stale intermediate collection because there is no intermediate collection.

§ VIIConnection to Prior Lessons

06-01 trait objects. The 06-01 lesson built open scorer stacks using Box<dyn Scorer> — adapter composition at runtime. This lesson's adapter chain is the compile-time analog: the sequence is fixed at write time, so no vtable dispatch is needed. Both answer the same question; adapters pay nothing at runtime while trait objects carry vtable overhead on each next() call.

06-15 enum-driven state machines. The position lifecycle from 06-15 receives PairSignal structs as input events. The state machine carries states: AwaitingFill, Active { entry_zscore, entry_time }, ClosingPending. The PairSignal.zscore is the entry event; the running z-score from the monitoring loop is the exit event.

§ VIIIClosing

The iterator adapters are the shape the language provides for this class of computation. flat_map for cartesian expansion. filter_map for concurrent gating and transformation. collect() at the single decision point. The pipeline is proportional to signal density, not universe size.

The Ops lesson defined the protocol and the data contract. This lesson expressed the contract in Rust. The CointegrationStore holds exactly the fields test_cointegration computed in Python. The adapter chain implements exactly the three-step protocol in a lazy pipeline.

Examine this. Build the pipeline from a real symbol slice.