Rust's Trait Objects and Enum Dispatch for a Model Cascade Router — Static versus Dynamic Dispatch, the Escalation Ladder, and Zero-Cost Tier Selection
§I — Frame
Today's Ops lesson built a cascade that routes each request to the cheapest model that can answer it. The router holds an ordered list of tiers and walks it until one clears the gate. In Rust, that list is the first design decision, and the decision is between two kinds of dispatch.
A cascade router does not know which tier will run until the estimator reads the request. The tier is a runtime choice. Rust gives two tools for holding a runtime choice of behavior: a trait object, which erases the concrete type behind a pointer and a vtable, and an enum, which names every possible type at compile time and matches over them. The two tools trade the same pair of properties in opposite directions. Trait objects buy open extensibility at the cost of an indirect call. Enums buy a direct, inlinable call at the cost of a closed, edit-the-enum-to-add-a-variant type set.
The router uses both, in the two places each fits. The escalation ladder — the list of tiers walked at runtime, extensible with new backends — is a Vec<Box<dyn Model>>. The difficulty estimator — a hot path over a fixed set of tier labels — is an enum matched directly. Blandy and Orendorff draw this exact line: a trait object is the right tool when you need a collection of mixed types behind one interface, and generics with static dispatch are the right tool when the type is known and the call is hot (Programming Rust, Ch. 11, pp. 266–271). The router needs both, so it holds both.
§II — Foundations
Rust resolves a method call in one of two ways. With static dispatch, the compiler knows the concrete type at the call site, so it emits a direct call to the exact function and can inline it. With dynamic dispatch, the compiler knows only that the value implements a trait, so it emits a call through a vtable — a table of function pointers carried alongside the value. The vtable call cannot inline and costs one pointer indirection.
A trait object is how Rust holds dynamic dispatch. Write Box<dyn Model> and you have a heap-allocated value of some type implementing Model, with a pointer to that value and a pointer to its vtable. A Vec<Box<dyn Model>> holds a mixed collection: rung one may be an OpenAISmall, rung two an AnthropicMid, rung three a FrontierModel, and the vector treats them uniformly because each satisfies the Model trait. Add a new backend by implementing the trait; the vector does not change.
An enum is how Rust holds a closed set with static dispatch. Write enum Tier { Small, Mid, Frontier } and every possible value is named at compile time. A match over the enum compiles to a direct branch — no vtable, no indirection, and the compiler checks that every variant is handled. Add a variant and every match that does not handle it fails to compile, which is a feature: the type system finds every site that must change.
The router's two needs map onto these two tools cleanly. The tier list must accept new backends without editing a central type, and it is walked at most a few times per request, so the vtable cost is nothing against a network call to a model. Dynamic dispatch, trait objects. The difficulty estimator runs on every request before any model call and works over a fixed, known set of tier labels, so it wants the direct branch and the exhaustiveness check. Static dispatch, enum.
§III — Mechanism
The trait and its objects
The Model trait names the operations the router needs from every tier: run the request and return an answer with a confidence the gate can read.
pub struct Response {
pub text: String,
pub confidence: f32,
}
pub trait Model {
fn name(&self) -> &str;
fn generate(&self, prompt: &str) -> Response;
}
pub struct SmallModel;
pub struct MidModel;
pub struct FrontierModel;
impl Model for SmallModel {
fn name(&self) -> &str { "small" }
fn generate(&self, prompt: &str) -> Response {
Response { text: cheap_infer(prompt), confidence: 0.62 }
}
}
impl Model for FrontierModel {
fn name(&self) -> &str { "frontier" }
fn generate(&self, prompt: &str) -> Response {
Response { text: frontier_infer(prompt), confidence: 0.95 }
}
}Each backend is a distinct type implementing one shared trait. The router never names these types again after construction. It holds them as Box<dyn Model> and speaks only through the trait.
The ladder as a trait-object vector
The escalation ladder is a vector of boxed trait objects, ordered cheapest first. Building it is the one place the concrete types appear.
pub struct Cascade {
ladder: Vec<Box<dyn Model>>,
}
impl Cascade {
pub fn standard() -> Self {
Cascade {
ladder: vec![
Box::new(SmallModel),
Box::new(MidModel),
Box::new(FrontierModel),
],
}
}
}The vector holds three different types uniformly. Adding a fourth backend is one more Box::new at construction and one more impl Model; the Cascade type and every method on it stay unchanged. This is the extensibility that justifies paying the vtable cost.
The gate and the ladder walk
The gate decides accept-or-escalate on each tier's answer. The walk starts at the estimator's chosen rung and climbs until a tier clears the gate or the ladder runs out, in which case the last tier's answer is final because there is nothing more expensive to try.
fn gate(resp: &Response, min_confidence: f32) -> bool {
!resp.text.trim().is_empty() && resp.confidence >= min_confidence
}
impl Cascade {
pub fn route(&self, prompt: &str, start: usize, min_conf: f32) -> Response {
let last = self.ladder.len() - 1;
for rung in start..=last {
let resp = self.ladder[rung].generate(prompt);
if rung == last || gate(&resp, min_conf) {
return resp;
}
}
unreachable!("loop returns at rung == last")
}
}Each self.ladder[rung].generate(prompt) is a dynamic call through the trait object's vtable. The router does not know or care whether it just called the small model or the frontier model; the trait erased that. The rung == last short-circuit encodes the top-rung rule from the Ops lesson: the final tier has no gate above it, so its answer returns unconditionally.
The estimator as a matched enum
The estimator picks the start rung. It runs on every request, works over a fixed set of tiers, and wants the direct branch, so it is an enum with a match.
pub enum Tier {
Small,
Mid,
Frontier,
}
impl Tier {
fn start_index(self) -> usize {
match self {
Tier::Small => 0,
Tier::Mid => 1,
Tier::Frontier => 2,
}
}
}
fn estimate(prompt: &str, tool_count: u8) -> Tier {
let long = prompt.len() > 800;
let multistep = prompt.contains("then") || tool_count > 1;
match (long, multistep) {
(false, false) => Tier::Small,
(false, true) => Tier::Mid,
(true, _) => Tier::Frontier,
}
}The match compiles to a direct branch with no vtable. If a fourth tier is added to the enum, both matches fail to compile until the new variant is handled, and the compiler names every site. That is the closed-set safety the trait object cannot give and the estimator wants.
§IV — Worked Example
Wire the estimator to the cascade. A request arrives; the estimator reads it and returns a Tier; the tier's start_index seeds the ladder walk; the cascade routes from there.
pub fn answer(cascade: &Cascade, prompt: &str, tool_count: u8) -> Response {
let tier = estimate(prompt, tool_count);
let start = tier.start_index();
cascade.route(prompt, start, 0.75)
}Three requests through this path. A short lookup with no tools: estimate returns Tier::Small, start rung 0, the small model answers, the gate checks confidence against 0.75, and if it clears, the answer returns after one dynamic call. A long synthesis request naming tools: estimate returns Tier::Frontier, start rung 2, which is also the last rung, so the frontier model answers and returns unconditionally — the cascade never touched the cheap tiers on a request it saw was hard.
The escalation case. A lookup reads short and toolless, so estimate returns Tier::Small and the walk starts at rung 0. The small model returns low confidence below 0.75. The gate rejects it. rung is 0, not last, so the loop continues to rung 1. The mid model answers above threshold; the gate accepts; the answer returns. Two dynamic calls, one wasted cheap call as toll, and no wrong answer shipped. The Vec<Box<dyn Model>> made the climb a plain loop over an index, and the trait erased which concrete model ran at each step.
The two dispatch styles did their two jobs. The enum estimator branched directly and exhaustively on a closed tier set, on the hot per-request path. The trait-object ladder walked a mixed, extensible collection at the cost of a vtable hop per rung, which is invisible against the model calls themselves.
§V — Connection to Prior Lessons
The enum-driven deployment state machine (2026-06-15) used an enum to make illegal transitions unrepresentable — an absent match arm was an enforced rule. The estimator here uses the same property: a tier not handled in the match is a compile error, so the closed set stays honest as it grows. Enums encode a closed world and the compiler guards its edges.
The iterator-adapter lesson (2026-06-26) built lazy pipelines with flat_map and filter_map for zero-cost composition. The ladder walk is the counterpart: where iterators inline a closed chain of transforms with static dispatch, the cascade deliberately gives up inlining to hold an open collection of backends. The two lessons bracket Rust's dispatch spectrum — static and inlined when the shape is closed and hot, dynamic and boxed when the shape is open and cold.
§VI — Connection to Today's Ops Lesson
The Ops lesson named the three parts of a cascade: difficulty estimation, the cheap-model-first gate, and the escalation ladder. This lesson gives each a Rust home. The estimator is an enum matched statically because it is a hot path over a closed set. The ladder is a Vec<Box<dyn Model>> walked dynamically because it is an open, extensible collection of backends. The gate is a free function returning a bool, called once per rung inside the walk. The Ops lesson's rule that the top rung returns unconditionally becomes the rung == last short-circuit in the loop. The whole cascade is a dispatch-design problem, and Rust makes the two dispatch choices explicit where other languages hide them.
§VII — Closing
The router holds a runtime choice of behavior, and Rust asks the same question twice: is the set of behaviors open or closed, and is the call hot or cold. Open and cold is a trait object. Closed and hot is an enum. The cascade is both at once, in two different places, so it holds both.
Reach for dyn when the collection must grow without editing a central type. Reach for the enum when the set is fixed and the compiler should guard every branch. Name which you are choosing, and why.
Examine well.
Related
- Prior arc: Rust's Iterator Adapters and flat_map (2026-06-26) — static, inlined composition; this lesson takes the dynamic, boxed side of the same dispatch spectrum
- Language hub: Rust (Backend Stack)
- Grounding tome: Programming Rust — Blandy, Orendorff, Tindall (Ch. 11 Traits and Generics, pp. 266–271)