By @pagemanSourceOpenVersions

from https://docs.google.com/document/d/1viJ6IRVPuBdg9zQ9BSsS7uztuoaxlmMZ7u8bJhL_dRU/edit?usp=sharing

Fork
Clone
git clone https://openversions.com/git/pageman/formal-names-for-context-window-strategies.git
Cite
Version link
https://openversions.com/pageman/formal-names-for-context-window-strategies/commit/d8caa3d05ec654c92fcf93d7be137190f2766f7b
Version identifier (commit hash)
d8caa3d05ec654c92fcf93d7be137190f2766f7b

Fork to create your own copy with credit to the source. Clone to get a Git copy on your computer.

Latest version

Checks complete

First version

Published

Claims

Findings

Headline finding

The strategy referred to as "O(n) Segmented State-Machine Streaming" is a bespoke, prompt-level branding of four established patterns in LLM systems engineering: Plan-and-Solve / Skeleton-of-Thought (SoT) Generation, State-Space / Scratchpad Checkpointing, Chunked Autoregressive Continuation (Stateful Paging), and Chunked / Segmented Complexity Amortization.

The Minimum Description Length (MDL) State-Machine Streaming prompt (~115 tokens) achieves ~85% compression compared to the full protocol while locking per-cell output to a flat ~60-token band with zero progressive bloat across 4x4 (16-cell) and 8x8 (64-cell) consensus algorithm grids.

In the 8x8 (64-cell) MDL consensus grid demonstration, the entire 64-cell matrix cleared a standard 4,096-token output ceiling in a single uninterrupted stream (~3,980 tokens) with 100% adherence to all seven protocol constraints.

Transitioning from a sequential state machine to an async parallel orchestrator using asyncio and Semaphores turns linear O(n) execution into O(1) wall-clock latency (bounded only by the concurrency limit), while preserving crash-recovery, deduplication, and schema guarantees.

The stateless Python orchestrator maintains O(1) constant input context per turn because messages are freshly constructed on every iteration with only system_prompt and the state_cursor JSON.

Methods

Minimum Description Length (MDL) State-Machine Streaming Prompt: A ~115-token prompt template that enforces a 1-line stage skeleton [S1..SK], a bounded per-cell schema (Title | Def (<=15w) | Steps (1->N) | Edge (<=12w)), rolling checkpoints [CP k/K | NEXT: S(k+1)|I1], and stateful paging [PAUSE Sk|Ij].

Source excerpts

“PROTOCOL: MDL STATE-MACHINE STREAMING GOAL: [OBJECTIVE] GRID: K=[STAGES] x M=[ITEMS/STAGE] RULES: 1. SKELETON: First emit 1-line stage index [S1..SK] (no prose). 2. BOUNDED SCHEMA (O(1)/cell): Prefix each cell [Sk|Ij]. Emit ONLY: Title | Def (<=15w) | Steps (1->N) | Edge (<=12w). 3. CHECKPOINT: After each stage, emit [CP k/K | NEXT: S(k+1)|I1]. Zero recap. 4. PAGING: Near token limit, finish cell, emit [PAUSE Sk|Ij], stop. On "CONTINUE", resume S(k)|I(j+1). START: Emit skeleton, then [S1|I1].” · Show in paper

Fault-Tolerant Stateless Python Orchestrator (resilient_orchestrator.py): A sequential Python pipeline using Pydantic Structured Outputs (BatchResponse), exponential backoff with jitter (call_with_backoff), atomic disk checkpointing (save_checkpoint_atomic), and coordinate deduplication/cursor reconciliation (validate_and_merge_batch).

Source excerpts

“Key Fault-Tolerance Additions ● Exponential Backoff with Jitter (call_with_backoff): Catches rate limits (429), transient server errors (5xx), and network drops, retrying with randomized exponential delays (base_delay * 2^attempt + jitter) so concurrent workers don't collide. ● Atomic Crash-Safe Checkpointing & Resume-from-Disk: Saves both records and the exact state_cursor into a unified state file using atomic writes (os.replace via a .tmp file). ● Duplicate & Out-of-Bounds Coordinate Validation: Tracks every committed (stage, item) tuple in a set.” · Show in paper

Async Parallel Python Orchestrator (async_parallel_orchestrator.py): An asyncio-based parallel orchestrator that dispatches per-stage worker coroutines (process_stage_worker) concurrently under asyncio.Semaphore(max_concurrency) and asyncio.Lock() for thread-safe state updates and atomic disk writes.

Source excerpts

“Per-Stage Worker Coroutines (process_stage_worker): Because each stage (1..K) has deterministic, non-overlapping coordinates, all incomplete stages are dispatched concurrently via asyncio.gather(). ... asyncio.Semaphore(max_concurrency) caps active in-flight API requests so you don't trigger 429 rate-limit cascades. asyncio.Lock() serializes state mutations and atomic disk writes (save_checkpoint_atomic)” · Show in paper

A/B Stress-Test Evaluation Protocol: A comparison methodology using an 8x10 (80-cell) Linux kernel taxonomy task to evaluate the MDL protocol against an unconstrained baseline across truncation cutoff point, early token bloat, resumption friction, and information density.

Source excerpts

“When you run both prompts in fresh Claude sessions, compare them across four concrete failure metrics: 1. Truncation Cutoff Point 2. Early Token Bloat (Items 1–10 vs. Items 35–45) 3. Resumption Friction 4. Information Density” · Show in paper