Sync Protocol

The sync protocol ensures nodes converge on the same ledger state by exchanging missing blocks. It uses a frontier-based approach: each node tells its peer what it already has, and the peer sends back what it is missing.

Protocol ID: /xe/sync/1.0.0

#Workflow

  Client                                  Server  ──────                                  ──────     │                                       │     │  SyncRequest (frontiers, page_size)   │     │ ─────────────────────────────────────▶│     │                                       │     │                          Compare frontiers     │                          Walk chains for     │                          missing blocks     │                                       │     │  SyncResponse (blocks[], has_more)    │     │◀───────────────────────────────────── │     │                                       │     │  SyncResponse (blocks[], has_more)    │     │◀───────────────────────────────────── │     │                                       │     │  SyncResponse (blocks[], has_more=f)  │     │◀───────────────────────────────────── │     │                                       │     │  Validate and add blocks to ledger    │     │                                       │

1. Client sends frontiers -- A map of account -> latest block hash representing its current view of the ledger, plus a requested page size.
2. Server compares frontiers -- For each account, the server determines which blocks the client is missing.
3. Server streams pages -- Missing blocks are sent in paginated responses. Each page contains up to pageSize blocks and a HasMore flag.
4. Client receives and validates -- The client collects all blocks, then adds them to the ledger via AddSyncedBlock() with retry logic for cross-account dependencies. AddSyncedBlock skips the timestamp window check (which would reject historical blocks) while keeping all other validation (signatures, PoW, balances, chain integrity).
5. Stream closes -- After the final page (HasMore=false), the stream ends.

#Wire Types

#SyncRequest

type SyncRequest struct {    Frontiers map[string]string `json:"frontiers"` // account → frontier block hash    PageSize  int               `json:"page_size"`}

#SyncResponse

1type SyncResponse struct {2    Blocks  []*core.Block `json:"blocks"`3    HasMore bool          `json:"has_more"`4    Cursor  string        `json:"cursor"` // hash of last block sent5}

#Constants

#Triggers

Sync is triggered in three ways:

#1. On Peer Connection

When a new peer connects, the node immediately initiates a sync:

1h.Network().Notify(&network.NotifyBundle{2    ConnectedF: func(n network.Network, conn network.Conn) {3        go requestSync(h, conn.RemotePeer(), ledger)4    },5})

#2. Periodic Re-Sync (with frontier tracking)

A background goroutine checks peers every 10 seconds, but only syncs when something has changed. A SyncTracker records the frontiers last sent to each peer and a dirty flag that is set when a block is added locally (via gossip, API, or sync):

• Dirty flag set: sync all peers on the next tick, then clear the flag
• Dirty flag clear: skip all peers (frontiers haven't changed)
• Full resync: forced every 60 seconds as a safety net regardless of dirty state

This eliminates the constant stream-open/close chatter when the network is idle. With 5 peers and a clean ledger, the node produces zero sync traffic between the 60-second safety ticks.

ticker := time.NewTicker(periodicSyncInterval) // 10s$ for range ticker.C {    for _, pid := range h.Network().Peers() {        if tracker.shouldSync(pid, currentFrontiers) {            go requestSync(h, pid, ledger, quarantine)        }    }}

#3. Incoming Sync Requests

The node also serves sync requests from other peers via the stream handler registered at /xe/sync/1.0.0.

#Rate Limiting

Both inbound and outbound sync are rate-limited per peer with separate syncRateLimiter instances:

The rate limiter tracks the last sync timestamp per peer ID. Expired entries are evicted on each check to prevent unbounded memory growth.

func (rl *syncRateLimiter) allow(pid peer.ID) bool

#Server-Side Logic

The server handles an incoming sync stream as follows:

1. Decode request -- Read and JSON-decode the SyncRequest from a size-limited reader (maxSyncRequestBytes)
2. Cap frontiers -- If the request contains more than maxFrontiers entries, excess entries are silently trimmed
3. Clamp page size -- Page size is clamped to [1, maxPageSize]
4. Walk chains -- For each account in the server's ledger:
   • If the client has no frontier for the account, send the entire chain
   • If the client's frontier matches the server's, skip (already in sync)
   • If the client's frontier is behind, send blocks after the frontier
   • If the client's frontier is unrecognized, skip the account (prevents amplification attacks)
5. Send pages -- Blocks are grouped into pages of pageSize and streamed as JSON-encoded SyncResponse objects

#Client-Side Logic

The client side of a sync:

1. Open stream -- Create a new stream to the target peer with a 60-second timeout
2. Send frontiers -- JSON-encode SyncRequest with the ledger's current frontiers
3. Close write -- Signal to the server that the request is complete
4. Read pages -- Decode SyncResponse objects until HasMore=false or EOF, accumulating blocks up to maxTotalBlocks
5. Add blocks with retry -- Blocks are added to the ledger via AddSyncedBlock() in multiple passes (up to 3 retries) to handle cross-account dependencies. AddSyncedBlock bypasses the timestamp window check — synced blocks are historical data that was already validated when originally published. All other validation (signatures, PoW, balances, chain integrity) still applies.

#Cross-Account Dependencies

A receive block references a send block from a different account. If the send block arrives later in the sync stream, the receive block fails to validate. The retry mechanism handles this:

Pass 1: Add all blocks → some receive blocks fail (send not yet in ledger)Pass 2: Retry failed blocks → most succeed (sends now in ledger)Pass 3: Final retry → remaining edge cases

Retryable errors include:

If no progress is made in a retry pass (same number of failures), retrying stops.

#Block Quarantine

Blocks that fail with non-retryable errors (invalid signature, wrong hash, missing attestations) are added to an in-memory quarantine set. On subsequent sync rounds, quarantined blocks are skipped entirely — no validation attempt, no log output. This prevents repeated "block rejected" log spam from permanently-invalid blocks (e.g., blocks from a previous network epoch).

The quarantine resets on node restart, which is correct — new state after restart may make previously invalid blocks valid.

#Security Considerations

Mitigations in place:

• Request size limit (1 MiB) -- Prevents OOM from huge frontier maps
• Response size limit (10 MiB) -- Prevents OOM from oversized responses
• Frontier cap (10,000) -- Limits frontier entries per request
• Block caps (10,000 client + server) -- Prevents CPU/memory exhaustion from chain walking
• Rate limiting (5s cooldown) -- Prevents sync flooding
• Stream deadline (60s) -- Prevents hung connections
• Amplification resistance -- Unrecognized frontiers are skipped, not replied with full chains

#Relationship to Gossip

Sync and gossip are complementary:

Gossip handles the fast path -- new blocks propagate in near-real-time. Sync handles the slow path -- catching up after downtime, missed messages, or network partitions.