The 2023 ReAct paper Yao et al. (2023) gave us a loop: Thought → Action → Observation. It worked. It also baked in assumptions that are now bottlenecks: linear chains, single-tool steps, no explicit cost model, no recovery. Xu et al. (2026) survey 200+ papers and map the paradigm shift: single-call execution → multi-tool orchestration. The math makes the difference visible.

The Mathematical Gap

Single-call (≤1 tool):

(z, u, y) ∼ π_θ(·|q, o_0, Σ_D_ctx),  z ∈ D ∪ {∅}
(f, s_1) ← Exec(z, u; s_0),  y ∼ π_θ(·|q, o_0, f)
max_θ E[R(q, y)]

One decision. One tool. Reward = task success.

Multi-tool orchestration (variable horizon T):

a_t ∈ A := (D × U) ∪ {Finish(y)}
a_t ∼ π_θ(·|h_t, m_t)
(f_t, s_{t+1}) ← Exec(d_t, u_t; s_t)
h_{t+1} = h_t ⊕ (a_t, f_t, o_{t+1})
τ := ((a_0,f_0),...,(a_{T-1},f_{T-1}), Finish(y))
max_θ E[R_task(q,τ,y) - λ·Cost(τ)]

Variable horizon. Action space too large for context. Cross-tool dependencies. Reward = success − λ·cost. The λ term changes everything: every tool call has a price. The policy must schedule, not just choose.

Six Dimensions of the Shift

Dimension	Single-Call / ReAct	Multi-Tool Orchestration
Planning	Implicit, step-by-step	Explicit: DAGs, AND/OR trees, tool graphs
Execution	Sequential	Concurrent (GAP, ToolNet), batched (LLMCompiler)
Recovery	Retry / backtrack	Checkpoint/rollback (Smurfs), graph decomposition on failure (ADaPT)
Memory	Conversation history	Structured: episodic + semantic + procedural (A-MEM, MIRIX)
Self-improvement	Few-shot / prompt	Reflexion, SPIRAL, test-time tool evolution, MetaAgent distillation
Training	Prompt / ICL	Trajectory synthesis → SFT → RL (GRPO on perturbed trajectories)

Planning: Graphs Beat Chains

ReAct executes linearly. Modern planners build dependency graphs before executing:

• GAP / ToolNet / AutoTool: construct tool graph from specs or historical trajectories. AutoTool reports 30% cost reduction by reusing inertia-aware graphs.
• HIPLAN: bi-level — macro milestones (Sys2) + micro execution (Sys1).
• ADaPT: recursive — try flat execution, decompose to graph on failure.
• ARTIS: think-simulate-act — internal risk-aware simulator for irreversible ops (delete, purchase, deploy).
• Smurfs: context-efficient DFSDT with stable rollback + local context filtering.

The pattern: plan as data structure, execute as graph traversal. This enables concurrency (independent branches run in parallel) and surgical rollback (revert only the failed subgraph).

Execution: Dual-System Architectures

Single model = single speed. Production systems split:

System	Sys1 (Fast)	Sys2 (Slow)
MARS	Fast parser / router	Slow reasoner / planner
CodeTool	Env gatekeeper (syntactic checks)	Central model (latent rewards)
SwiftSage	Lightweight edge modules	Central large model
HuggingGPT	LLM router → specialized models	—

The fast path handles 80% of calls (lookup, format, validate). The slow path plans, recovers, synthesizes. Cost (λ) drops because expensive tokens are reserved for decisions that need them.

Memory for Orchestration

Orchestration needs structured memory, not chat history. Xu et al. (2026) categorize:

• COMPASS: context manager compresses subtasks → progress briefs (keeps context lean).
• A-MEM: semantic links via vector similarity; updatable historical nodes.
• MIRIX: granular taxonomy — Core, Episodic, Semantic, Procedural, Resource, Knowledge Vault.
• Tool-to-Agent Retrieval: bipartite knowledge graph (tools ↔ parent agents) for reuse.

The agent that remembers its own past plans (successes, failures, recovered states) outperforms the one that only remembers what the user said.

Self-Improvement: The Loop Closes

Method	Mechanism
Reflexion	Episodic memory of trial/error; verbal reflection → better next attempt
Failure Makes Agent Stronger	GRPO on perturbed trajectories; error isolation + verifiable repair
SPIRAL	Grounds reflection in formal logic (not free text)
PreFlect	Prospective error anticipation before acting
MetaAgent	Distills experiences → reusable knowledge + internal tools
Test-Time Tool Evolution	Dynamic synthesis/verification of new executable tools
DRAFT	Iterative tool description rewriting

The trajectory: reflect → distill → internalize as tool/skill. The agent writes its own standard library.

Training Paradigm Ladder

Training-free (Prompt / ICL / Retrieval)
    → Trajectory Data Synthesis (Seal-Tools, BUTTON, hybrid bottom-up/top-down)
    → Supervised Fine-Tuning (SFT) on curated trajectories
    → Reinforcement Learning (RL) — GRPO, PPO on multi-turn rewards

Key insight from Xu et al. (2026): trajectory synthesis moved from “generate random” → Synthesis-Validation-Expansion framework. Generate → verify with executor / judge → expand valid ones → iterate. Quality > quantity.

What This Means for You

If you are still prompting ReAct loops: you are building 2023 architecture. The shift is not “better prompts.” It is:

1. Explicit planning artifact (DAG/plan) before execution — inspectable, editable, versionable.
2. Cost-aware scheduling — λ is a real budget; parallelize independent calls; batch sequential ones.
3. Structured memory — not chat logs; episodic (what happened), semantic (facts), procedural (skills).
4. Recovery as first-class — checkpoint state; on failure, rollback subgraph, not whole conversation.
5. Self-improvement loop — log trajectories, judge them, distill into skills/tools, re-deploy.

The model is the same. The host changed. Build the host.