Cohort Dispersion as a Regime Classifier

Cross-sectional return dispersion, the rotation-not-exit signal, and the within-cohort long/short structure.

A flat index is the one number that hides a rotation completely. What did the average throw away?

The exit gate from two days ago watched one series over time: does the withdrawal regime end, yes or no. This lesson turns the camera ninety degrees. Instead of watching one thing across many days, it watches many things across one day, and asks what the spread between them says about the state of the whole.

The occasion is a live pattern. Through the first two days of Q3 the AI trade did not leave the market; it moved up its own stack. Semiconductor names that ran eighty percent in the first half took profits — Micron down nine, SanDisk down ten — while the hyperscalers that rent compute rose, Meta up roughly ten on a cloud pivot, Microsoft up three, the index itself close to flat. The bid did not exit. It rotated. A trader watching only the index return saw nothing. A trader watching the dispersion inside the cohort saw the whole event.

Call the instrument a cohort dispersion classifier: a rule that reads the cross-sectional spread of returns within a defined group and labels the regime the group is in. The classifier does not predict direction. It discriminates between three states a cohort can occupy — moving together, rotating internally, or breaking apart — and it hands that label to the systems that do size and direction.

§ IFrame

Entry and exit watched a level move through time. The cross-section watches an arrangement at a single instant. These are orthogonal reads of the same market, and the second one is native to rotation: when money leaves semis for hyperscalers, no aggregate moves much, but the ranking inverts, and ranking is what the cross-section measures. This lesson builds the classifier that reads the arrangement and names the regime hiding behind a flat tape.

§ IIFoundations

The cross-section is a different observation than the time series

A factor model reads a universe on a single day as a cross-section: every name carries an exposure and a return, and the model asks which exposures explain the spread of returns right now. Chan's treatment of factor models (Machine Trading, Ch 2, pp. 72–73) makes the operational point that the useful signal lives in the relative ordering across names on the same date, not in any one name's path. The cross-sectional view is native to rotation. When money leaves semis for hyperscalers, no aggregate moves much, but the ranking inverts, and ranking is exactly what the cross-section measures.

Dispersion is the width of that cross-section. Grinold and Kahn treat dispersion inside portfolio construction (Active Portfolio Management, pp. 402–408) as the spread of outcomes a portfolio can realize — the same statistic, read as opportunity. When dispersion is wide, selection matters, because the gap between the best and worst member is large and a correct ranking is worth capturing. When dispersion is narrow, selection is nearly free money left on the table, because every member is doing the same thing and there is nothing to select between.

Three regimes a cohort can occupy

A cohort is a named set of assets that share a driver: AI-compute equities, layer-1 tokens, stablecoin-quoted majors. On any day the cohort sits in one of three states, and dispersion plus correlation name which.

Coherence. Low dispersion, high pairwise correlation. Everything moves the same direction by a similar amount. The driver is dominating and idiosyncratic differences are suppressed. Risk-on and risk-off both look like this.

Rotation. High dispersion, but the spread is structured — winners and losers sort along a readable axis (up the stack, large over small, quality over junk). The cohort's total return can be near zero while its internal ordering churns violently. This is the Q3-open state.

Fracture. High dispersion with no structure — the spread is wide and the ordering is noise, no axis explains it. This is the cohort losing its shared driver, and it is the state that precedes a regime change rather than a rotation inside one.

The classifier's whole job is to tell rotation from fracture, because they share the symptom — wide dispersion — and demand opposite responses.

Why the aggregate hides all three

An index return is a cohort's first moment. Coherence, rotation, and fracture are statements about the second moment and the structure of the cross-section. A flat index is consistent with all three: everyone flat, or winners cancelling losers, or chaos averaging out. Reading regime from the aggregate is reading a distribution from its mean alone. The dispersion classifier reads the shape.

§ IIIMechanism

The classifier consumes a cohort's one-day cross-section and emits a label. Four measurements, computed each session.

The label follows from the combination. Low dispersion is coherence, regardless of the other three. High dispersion with high structure and persistence is rotation, and the classifier names the axis. High dispersion with low structure is fracture, and the classifier raises the flag that the cohort's driver is failing.

The rule that keeps this honest is that structure gates the rotation call. Wide dispersion alone never earns the rotation label, because fracture wears the same width. Only a spread that sorts along a readable axis, and sorts the same way two sessions running, is rotation. Absent structure, wide dispersion is fracture until proven otherwise — the conservative default, because fracture is the expensive state to misread.

§ IVWorked Example

Take the Q3-open AI cohort: a dozen names spanning the stack, from foundry and memory at the bottom through GPU vendors to the hyperscalers at the top. On July 1 the cohort's mean return was near zero. Read as an index, a non-event.

Now the cross-section. Dispersion is wide — Micron down nine and SanDisk down ten at one end, Meta up ten at the other, roughly nineteen points of spread across the cohort in a single session. So the cohort has left coherence. The question is rotation or fracture, and structure decides it.

Rank each name by position in the stack: raw-silicon and memory at the bottom, compute-renters at the top. The returns sort cleanly along that axis — the lower the layer, the worse the day; the higher, the better. Structure is high. Check persistence: 06-29 showed the same shape at smaller amplitude, so the ranking holds across two sessions, N=2. The classifier labels the cohort rotation, axis = up-the-stack, and it does so while the index says nothing happened.

The label drives the book. Rotation with a readable axis is a within-cohort long/short: long the top of the axis, short the bottom, net exposure near zero. Long hyperscalers, short semis, sized so the cohort's own beta cancels. Aldridge's statistical-arbitrage framing (HFT, Ch 8, pp. 212–213) is the sizing home — the tradeable object is the relationship between the two legs, not either leg's direction, so the position earns from the ranking being right even if the whole cohort goes nowhere. That is precisely the Q3-open condition.

Contrast the fracture branch. Suppose the same nineteen-point spread but the ranking is noise — memory up, one hyperscaler down, no axis fits, and yesterday's order does not predict today's. Structure is low, persistence fails. The classifier withholds the rotation label and flags fracture. The correct response is the opposite of a pair trade: reduce cohort exposure, because a group that has lost its shared driver is about to reprice as individual names, and the market-neutral structure that harvests rotation gets run over by fracture.

def classify_cohort(returns, axis_rank, prev_rank):
    disp = cross_sectional_std(returns)
    if disp < COHERENCE_FLOOR:
        return Label("COHERENCE", axis=None)
    structure = axis_fit(returns, axis_rank)
    persistence = rank_corr(rank_of(returns), prev_rank)
    if structure >= STRUCTURE_MIN and persistence >= PERSIST_MIN:
        return Label("ROTATION", axis=axis_rank.name)
    return Label("FRACTURE", axis=None)

The function reads as the discipline reads. Low width is coherence and nothing else is computed. Wide width is never rotation on its own — only structure and persistence together promote the label, and everything else falls through to fracture, the conservative default.

§ VConnection to Prior Lessons

06-30 multi-signal exit confirmation. That lesson read one series over time and refused to flip regime on a single print. This one reads one cross-section per day and refuses to flip from coherence to rotation on a single session — persistence is the same patience, rotated from the time axis onto the cross-sectional axis. The fake-reversal filter and the persistence gate are one discipline in two orientations.

06-19 regime detection and persistence. Detection labeled a market state from a time series with a hysteresis buffer against premature flips. The dispersion classifier is the cross-sectional member of the same family: another regime labeler, another buffer against noise, reading width and structure where the detector read level and duration. Together they give a two-axis read on the same market — where it is over time, and how its members are arranged today.

06-26 cross-asset pair signals. That lesson built a long/short from a structural relationship between two named assets. Rotation is the cohort-scale generalization: the same long-top short-bottom structure, but the pair is selected by the cross-sectional ranking rather than a cointegration test on a fixed duo. The pair lesson is this lesson with the cohort size set to two.

§ VIConnection to Today's Dev Lesson

The R and Python lesson builds the machinery this classifier assumes. It assembles the cohort return panel — members down the rows, sessions across the columns — and computes the cross-sectional dispersion, the rank correlation that measures persistence, and the axis-fit that measures structure. The exploration happens in R, where the distributional shape of the cross-section is visible at a glance; the production computation is enforced in Python, where the same metrics run inside the scheduled job. This lesson defines what the four measurements mean; the Dev lesson makes them reproducible on a panel of real numbers.

§ VIIClosing

The index is the cohort's mean, and the mean is the one statistic that hides a rotation completely. The regimes that matter for selection — coherence, rotation, fracture — are statements about the width and the shape of the cross-section, and they are invisible to anyone reading the aggregate. The dispersion classifier reads width first, gates the rotation call on structure, confirms with persistence, and hands a labeled regime with a named axis to the book. When the label is rotation, the axis is the trade: long its top, short its bottom, and let the cohort's own direction cancel.

When the group goes nowhere, look at how its members got there. The spread is the signal the average threw away.