Agent Handoffs That Don't Drop Context

1. The Agent Handoff Problem

When a single AI agent handles a task from start to finish, context management is straightforward — the agent holds everything it needs in its own working memory. But modern production systems rarely operate that way. Complex workflows demand specialization: a research agent gathers data, an analysis agent interprets it, a code-generation agent writes the implementation, and a review agent validates the output. Each transition between agents is a handoff, and each handoff is an opportunity for context to degrade or disappear entirely.

The fundamental challenge is what we call the telephone game effect. In the children's game, a message passed through a chain of people arrives at the end barely recognizable. Multi-agent pipelines suffer the same distortion. Agent A understands the user's intent perfectly. It passes a summary to Agent B, which loses nuance. Agent B passes its own summary to Agent C, which now operates on a twice-compressed version of the original request. By Agent D, the system is solving a different problem than the one the user asked about.

The problem compounds with scale. A two-agent pipeline might lose 10% of relevant context per handoff. A five-agent pipeline operating at the same per-handoff loss rate retains only 66% of the original context by the final stage. In practice, the loss is not uniform — critical constraints and edge cases are the first details to be dropped, because they are often expressed subtly and require deliberate effort to propagate.

Without a formal handoff protocol, each agent makes ad-hoc decisions about what to pass along. Some pass too little, stripping away important constraints. Others pass too much, flooding the next agent with irrelevant context that consumes token budget and dilutes focus. Neither approach works reliably at production scale. What you need is a structured protocol — a contract between agents that defines exactly what gets transferred, in what format, and with what guarantees.

2. Anatomy of a Handoff Protocol

A well-designed handoff protocol is more than a serialized blob of text. It is a structured envelope that gives the receiving agent everything it needs to continue work without ambiguity. Think of it as an API contract between the sending and receiving agents — both sides agree on the schema, the required fields, and the semantics of each field.

The Protocol Envelope

Every handoff should include five core components packed into a single envelope:

• Task context — The original user intent, the current state of the task, and what specifically the receiving agent is expected to accomplish. This is not a full conversation history; it is a distilled statement of purpose.
• Memory references — Pointers to relevant memories stored in a persistent system like Memory Spine. Rather than copying large blocks of context inline, the envelope references memory IDs that the receiving agent can fetch on demand.
• Execution history — A structured log of what previous agents did, what they decided, and why. This prevents the receiving agent from repeating work or contradicting earlier decisions.
• Constraints and guardrails — Hard requirements that must be preserved across the entire pipeline: token budgets, security boundaries, formatting rules, user preferences, and any explicit "do not" instructions.
• Expected outputs — A clear specification of what the receiving agent should produce, including format, length, and quality criteria. This eliminates ambiguity about the handoff's purpose.

Protocol Versioning

The envelope should carry a version identifier. As your system evolves, the handoff schema will change — new fields get added, old fields become optional. Version numbers let receiving agents parse envelopes correctly even during rolling deployments where some agents run older code. A simple integer version with backward-compatible field additions covers most production needs.

3. Memory Spine Handoff API

Memory Spine provides a purpose-built agent_handoff() function that handles the mechanical complexity of assembling a handoff envelope. Instead of manually querying memories, formatting context, and managing token budgets, you call a single function that packages everything the receiving agent needs.

The function works by collecting three categories of data: recent memories (the last N interactions relevant to the current task), pinned context (critical facts that should always be included, like system configuration or user identity), and task state (the current progress, decisions made, and remaining work). It then serializes this data into a structured envelope, applying token-budget trimming to ensure the output fits within the receiving agent's context window.

# Initiate a handoff from an analysis agent to a code-generation agent
from memory_spine import MemorySpine

spine = MemorySpine(endpoint="http://127.0.0.1:8788")

# Build the handoff envelope
handoff = spine.agent_handoff(
    target_agent="code-generator",
    include_recent=20,           # Last 20 relevant memories
    include_pins=True,           # Always include pinned context
    task_summary="Implement the retry logic for the payment gateway "
                 "based on the analysis in memories ms-4821 through ms-4825. "
                 "Use exponential backoff with jitter. Max 3 retries.",
    constraints={
        "language": "python",
        "max_tokens": 4000,
        "security": "no plaintext credentials",
        "style": "follow existing patterns in src/payments/"
    },
    execution_history=[
        {"agent": "researcher", "action": "gathered API docs", "memory_id": "ms-4821"},
        {"agent": "analyst", "action": "identified failure modes", "memory_id": "ms-4824"},
        {"agent": "analyst", "action": "recommended retry strategy", "memory_id": "ms-4825"}
    ]
)

# The handoff object contains the full envelope
print(handoff.envelope_version)   # "2"
print(handoff.token_count)        # 3,247 (fits within budget)
print(handoff.memory_refs)        # ["ms-4821", "ms-4822", ... "ms-4825"]

The receiving agent consumes the envelope by parsing the structured fields. It does not need to understand how the envelope was assembled — it simply reads the task summary, fetches any referenced memories it needs, respects the constraints, and produces the expected output. This clean separation between envelope production and consumption is what makes the protocol robust across heterogeneous agent implementations.

4. Implementation Patterns

Not all handoffs follow the same pattern. The right choice depends on your pipeline topology, latency requirements, and failure recovery needs. Three patterns cover the vast majority of production use cases.

Direct Handoff

The simplest pattern: Agent A finishes its work and sends the envelope directly to Agent B. This is synchronous and blocking — Agent A waits for acknowledgment before releasing its resources. Direct handoff works well for linear pipelines where each stage has exactly one successor and you need strict ordering guarantees.

The downside is coupling. If Agent B is slow or unavailable, Agent A blocks. In practice, you add a timeout and fallback, but the fundamental architecture assumes both agents are available simultaneously.

Broadcast Handoff

Agent A publishes the handoff envelope to a shared channel, and multiple downstream agents consume it independently. This is the pattern you use when a single piece of work fans out into parallel tracks — for example, when a research agent's findings need to be simultaneously consumed by a code generator, a test writer, and a documentation agent.

# Broadcast handoff: one sender, multiple receivers
handoff_envelope = spine.agent_handoff(
    target_agent="broadcast",
    include_recent=15,
    task_summary="Research complete on OAuth 2.1 migration. "
                 "Findings stored in ms-7100 through ms-7108.",
    constraints={"deadline": "2026-01-30T00:00:00Z"}
)

# Publish to the handoff channel
channel = spine.get_channel("oauth-migration")
channel.publish(
    envelope=handoff_envelope,
    receivers=["code-generator", "test-writer", "doc-writer"],
    delivery="at-least-once"
)

# Each receiver picks up the envelope independently
# and works in parallel without blocking others

Broadcast handoffs require an at-least-once delivery guarantee — every intended receiver must get the envelope, even if that means some receivers see it twice. Idempotent processing on the receiver side handles duplicates gracefully.

Choreographed Handoff

In this pattern, there is no central orchestrator. Each agent knows who comes next in the pipeline and initiates the handoff autonomously. Agent A hands off to Agent B, which hands off to Agent C, and so on. The handoff chain is defined by configuration or convention, not by a coordinator process.

Choreographed handoffs excel in systems where the pipeline shape varies by task. A routing agent examines the task and decides the first handoff, and each subsequent agent decides its own successor based on the current state. This gives you dynamic pipelines that adapt to the work without requiring a static DAG definition.

5. Context Serialization

Deciding what to include in a handoff envelope is as important as the protocol itself. Include too much and you waste token budget, dilute the receiving agent's focus, and increase latency. Include too little and you lose critical context. The goal is a minimal-but-sufficient envelope that carries everything the receiving agent needs and nothing it doesn't.

What to Include vs. Exclude

Always include the task summary, active constraints, unresolved decisions, and any memory references the receiving agent will need. Exclude raw conversation logs (summarize instead), intermediate reasoning chains that led to already-finalized decisions, and any data that the receiving agent can fetch on demand from Memory Spine or another data source.

Token Budget Management

Every receiving agent has a finite context window. The handoff envelope must fit within a token budget that leaves room for the agent's system prompt, the task-specific instructions, and the agent's own working space. Memory Spine's agent_handoff() accepts a max_tokens constraint and applies priority-based trimming automatically — high-priority memories and pinned context are kept; older, lower-relevance memories are dropped first.

The lazy loading strategy is particularly powerful when all agents in the pipeline have access to Memory Spine. Instead of embedding full memories in the envelope, you embed only memory IDs. The receiving agent fetches the specific memories it needs during execution. This keeps the envelope small and ensures the receiving agent always gets the freshest version of each memory.

6. Error Handling and Recovery

Handoffs fail. Networks drop packets. Agents crash mid-processing. Memory Spine might be temporarily unavailable. A production handoff protocol must handle all of these scenarios gracefully, without losing work or corrupting state.

Failed Handoff Detection

The sending agent should always wait for an explicit acknowledgment from the receiver before considering a handoff complete. If no acknowledgment arrives within a configurable timeout (typically 10–30 seconds for synchronous handoffs), the sender retries. After a maximum number of retries (usually 3), the sender escalates — either routing to an alternative agent or parking the task in a dead-letter queue for human review.

# Robust handoff with retry and circuit breaker
import time

class HandoffError(Exception):
    pass

class CircuitOpenError(HandoffError):
    pass

class HandoffClient:
    def __init__(self, spine, max_retries=3, timeout=15):
        self.spine = spine
        self.max_retries = max_retries
        self.timeout = timeout
        self.failure_count = 0
        self.circuit_open = False
        self.circuit_reset_time = None

    def send_handoff(self, envelope, target_agent):
        # Circuit breaker: fast-fail if recent handoffs have failed
        if self.circuit_open:
            if time.time() < self.circuit_reset_time:
                raise CircuitOpenError(
                    f"Circuit open for {target_agent}. "
                    f"Resets at {self.circuit_reset_time}"
                )
            self.circuit_open = False
            self.failure_count = 0

        last_error = None
        for attempt in range(1, self.max_retries + 1):
            try:
                ack = self.spine.deliver_handoff(
                    envelope=envelope,
                    target=target_agent,
                    timeout=self.timeout
                )
                self.failure_count = 0  # Reset on success
                return ack
            except TimeoutError as e:
                last_error = e
                wait = min(2 ** attempt, 30)  # Exponential backoff
                time.sleep(wait)

        # All retries exhausted — open the circuit
        self.failure_count += 1
        if self.failure_count >= 3:
            self.circuit_open = True
            self.circuit_reset_time = time.time() + 60  # 1-min cooldown

        # Park the failed handoff for recovery
        self.spine.park_envelope(envelope, reason=str(last_error))
        raise HandoffError(
            f"Handoff to {target_agent} failed after "
            f"{self.max_retries} attempts: {last_error}"
        )

Partial Context Recovery

Sometimes a handoff succeeds but the envelope is incomplete — perhaps Memory Spine was briefly unavailable and some memory references could not be resolved during envelope assembly. The receiving agent should detect missing references and attempt to fetch them directly. If it cannot, it should continue with degraded context and flag the gaps in its output, rather than failing entirely. Graceful degradation is almost always better than a hard failure in multi-agent pipelines.

7. Monitoring Handoff Health

You cannot improve what you do not measure. Production handoff systems need dedicated monitoring that tracks three categories of metrics: latency, completeness, and reliability.

Key Metrics

• Handoff latency (p50, p95, p99) — How long it takes from the moment the sending agent initiates a handoff to the moment the receiving agent acknowledges it. Spikes in p99 latency often indicate Memory Spine connectivity issues or overloaded receiving agents.
• Context completeness score — The percentage of requested memory references that were successfully resolved and included in the envelope. A score below 95% should trigger investigation. Below 80% should trigger an alert.
• Handoff failure rate — The percentage of handoffs that exhaust all retries without successful delivery. Healthy systems operate below 0.5%. Above 2% indicates a systemic issue.
• Hop depth distribution — How many handoffs occur per task on average. Unexpected increases suggest routing loops or inefficient pipeline configurations.
• Token utilization ratio — The percentage of the token budget actually used by the envelope. Consistently low ratios mean you are over-budgeting; consistently hitting 100% means context is being aggressively trimmed.

Alerting Strategy

Set up tiered alerts based on severity. A single failed handoff is informational. Three consecutive failures to the same target agent is a warning. A circuit breaker opening is critical and should page the on-call engineer. Context completeness dropping below 80% across all handoffs is critical — it means the entire pipeline is operating on degraded context and output quality is likely compromised.

8. Production Best Practices

Teams that run multi-agent systems at scale converge on a set of practices that keep handoffs reliable as the system grows. Here are the patterns that matter most.

Version Your Protocol

Include a version number in every handoff envelope. When you add fields, increment the minor version and make new fields optional so older receivers can still parse the envelope. When you make breaking changes (renaming fields, changing semantics), increment the major version and run a migration period where senders emit both old and new formats.

Test Handoff Chains End-to-End

Unit testing individual agents is necessary but not sufficient. You need integration tests that exercise full handoff chains: Agent A hands off to Agent B, which hands off to Agent C, and you verify that the final output reflects the original intent. These tests catch serialization bugs, missing fields, and context-loss regressions that unit tests miss.

Load Test Your Handoff Infrastructure

Handoff performance degrades under load in ways that are hard to predict. Memory Spine queries slow down. Channel throughput drops. Envelope serialization becomes a bottleneck. Run load tests that simulate peak handoff volume and measure latency at every stage. Common targets are 100 concurrent handoffs for small systems and 1,000+ for platforms with many users triggering parallel pipelines.

Use Immutable Envelopes

Once assembled, a handoff envelope should never be mutated. If an agent needs to add information, it creates a new envelope that references the previous one. This gives you a full audit trail of how context evolved through the pipeline, which is invaluable for debugging failures and understanding agent behavior.

The best handoff protocol is one that the receiving agent does not even notice — it simply has the context it needs, in the format it expects, ready to work.

Design for Backward Compatibility

In a production environment, you will never upgrade all agents simultaneously. Some agents will be running version N of the handoff protocol while others are on version N+1. Design your envelopes so that older agents can safely ignore fields they don't recognize. Use optional fields with sensible defaults. Never remove fields in a minor version — deprecate them instead and remove them in the next major version after a full migration cycle.