Regime Detection and Persistence Estimation for Adversarial Markets

Online hidden-state inference, survival-hazard duration models, and the hysteresis discipline that keeps the read from twitching.

What does a strategy stand on that it did not choose?

It stands on the regime. A momentum book makes money while trends persist and bleeds while they chop; a mean-reversion book does the reverse. Neither chose the weather it trades in. The June 12 lesson decided which strategies keep capital by reading their attribution. This lesson reads the thing one level up — the market state every strategy is exposed to whether it models that state or not — and turns reading it into a running service the trading system can query before it sizes a single position.

Three lessons back, the pre-trade gate already used a regime score to bend Kelly sizing toward caution. That score was an input the lesson assumed. Today it becomes an output a system produces, with a probability attached, a dwell-time attached, and a discipline that keeps it from twitching.

§ IFrame

A regime is a stretch of time over which the rules a strategy depends on hold roughly steady. Call them by plain names: trending, choppy, risk-on, risk-off, high-volatility, calm. The market does not announce which one it is in. It emits prices, volumes, spreads, and fund flows, and the regime that produced them stays hidden behind that emission. The operator's job is to infer the hidden state from the visible stream, attach an honest probability to the inference, and estimate how long the state is likely to last before it flips.

Two questions sit at the center, and they are different questions. The first is which regime are we in now, a classification over the present. The second is how much longer will it hold, a forecast over the future. The June 4 lesson leaned on the first and ignored the second. A book that knows it is in a calm regime but not that the calm is in its third week of an average two-week life is sizing on half the picture. The duration estimate is the half the daily operational loop kept assuming and never produced.

§ IIFoundations: The Hidden State and the Visible Stream

Murphy frames the machinery exactly. A hidden Markov model posits a discrete hidden state that evolves as a Markov chain, where the state at each step depends only on the state before it, and an observation emitted at each step whose distribution depends only on the current hidden state (Machine Learning: A Probabilistic Perspective, Ch 17, p 657). The market is the textbook case. The hidden state is the regime. The observations are the day's returns, realized volatility, and flow prints. The transition matrix says how often a trending market stays trending versus tips into chop. The emission model says what returns and volatility look like conditional on each regime.

The inference the operator runs online is filtering: given every observation up to now, compute the probability distribution over the current hidden state. Murphy's state-space chapter generalizes the same recursion to continuous latent variables, where the Kalman filter plays the role the forward algorithm plays for the discrete chain (Ch 18, p 686). The shape is identical at both scales: a prior over the state, an observation, a correction of the prior by the observation, a posterior that becomes the next step's prior. The regime service is that recursion, run once per bar, forever.

Chan grounds the same idea on market data directly. His time-series treatment walks the practitioner from stationarity tests through the structure that lets a model separate signal from the random walk underneath it, and the regime question is the structural break that stationarity tests are built to catch (Machine Trading, Ch 3, pp 88-97). A series that looks like noise inside one regime and noise inside another can carry a sharp, tradeable shift at the boundary. Detecting the boundary is the whole game.

§ IIIMechanism: Three Reads the Service Produces

The regime service emits three numbers on every bar, and naming them keeps the consumers honest about what they are reading.

A knowledge-gap note belongs here in the open. The QuantOps shelf carries Murphy and Chan for the hidden-state and time-series machinery, but no dedicated survival-analysis tome answered the lattice. The duration read above is built from first principles rather than cited to a canonical source, and the gap is surfaced for tome acquisition. An operator should treat the hazard arithmetic as sound but unblessed until the shelf carries Klein-and-Moeschberger or its equal.

§ IVWorked Example: A Regime Read at the ETF-Flow Boundary

Take the situation the vault has been tracking. An ETF-flow regime had persisted, the book leaned on its persistence, and then a decision-day print plus a hawkish policy turn withdrew two of the legs the regime stood on. Read that morning through the three numbers.

Before the print, the Posterior-Over-Regimes sits heavy on risk-on, say 0.8. The Survival read says the regime is mature, already past the median life of past risk-on stretches, so its remaining-life estimate is short and its hazard is elevated by age. Transition-Pressure is low but not zero. The honest summary that morning is we are in a risk-on regime, but an old one, and old ones do not owe us more time. That summary alone tells the pre-trade gate to size as if the floor could move, which is exactly the regime-conditional caution the June 4 lesson reached for.

The print lands. Net complex outflow turns sharply negative; the policy dots flip hawkish. The next bar's observation is far in the tail of the risk-on emission model and squarely inside the risk-off one. The filter does what filters do: the posterior collapses off risk-on toward risk-off within one or two bars, Transition-Pressure spikes on the bar of the print and then falls as the new state settles, and the Survival read resets its clock to a fresh, young risk-off regime. The kill-switch discipline from June 5 reads the spike and the posterior collapse as its risk-off trigger and pulls exposure before the slower attribution layer has a single new data point.

§ VThe Hysteresis Discipline

Here is the failure mode that ruins a naive regime service, and the discipline that prevents it. A filter run bar by bar will, near a regime boundary, flip its argmax back and forth as noisy observations nudge the posterior across the 0.5 line. Trending, choppy, trending, choppy, four flips in six bars. Every consumer downstream re-sizes, re-hedges, and re-routes on each flip, paying transaction cost on noise and churning the book for nothing. The regime did not change four times. The estimate twitched.

The cure is hysteresis, and it has the shape of a thermostat. A thermostat does not toggle the furnace the instant the temperature crosses the setpoint; it waits for a band of separation so a half-degree wobble does not cycle the compressor to death. The regime service borrows the move directly. Coin it the Two-Threshold Dwell. A flip from regime A to regime B requires the posterior on B to clear a high entry threshold, say 0.7, not a bare majority, and to hold above a lower exit threshold, say 0.55, for a minimum dwell of several bars before the service publishes the new state. A wobble that pokes B to 0.6 for one bar never clears entry. A genuine shift that drives B to 0.85 and holds clears entry and survives the dwell. The service trades a bar or two of latency at the true boundary for immunity to dozens of false ones, and on adversarial-market data that trade is heavily favorable.

The Two-Threshold Dwell is the operator's discipline, not the model's. The filter keeps emitting its honest, twitchy posterior on every bar. The service layer applies the entry threshold, the exit threshold, and the dwell, and only it decides when the published regime label changes. The separation matters because the raw posterior is still available to the consumers that want it, the pre-trade gate that sizes on the full distribution, while the published label that gates the heavy machinery moves only when the dwell is satisfied. Two consumers, two latencies, one filter.

§ VIConnection to Today's Dev Lesson

Today's Go lesson builds the service layer this lesson specified. The published regime is a value that many consumers read constantly and one writer updates rarely, which is the exact shape Go's atomic.Pointer is built for: lock-free reads for the gate, the kill-switch, and the dashboards, with the single writer swapping in a new regime only when the Two-Threshold Dwell clears. The dwell itself becomes a Go timer that the writer arms when the posterior first crosses entry and disarms if the posterior falls back below exit before the timer fires. The transition broadcast, the moment the published label finally changes, fans out to consumers over a channel. The discipline this lesson named in market terms, the Go lesson renders in concurrency primitives.

§ VIIClosing

A strategy stands on a regime it did not choose. The operator who measures the regime gives every strategy a floor it can see. Produce three reads on every bar: the distribution over states, the survival of the current state, and the pressure toward the next one. Publish the state through a two-threshold dwell so the label moves on shifts and not on noise. Hand the raw posterior to the consumers that can use confidence directly, and hand the dwelled label to the machinery that must not churn.

The regime is the weather. A book that trades blind to the weather is a book that learns the storm arrived by getting wet. Measure the sky.

Examine well. Reflect on this.