CortanaROI Brain

Dashboard real-time tick streaming - three layers that all must work for "live"

4 min read

Dashboard real-time tick streaming - three layers

A truly live trading dashboard (open positions panel reflecting broker within ≤ 1s) requires three independent layers to all work correctly. Failure of any one drops the system to “snapshot polling” mode where the operator can’t trust what they see.

The three layers

Layer 1 - IBKR tick subscription

ib.qualifyContractsAsync(contract)        # contract.conId must be set
  → ib.reqMktData(contract, '', False, False)  # keepUpToDate=True semantics
    → ib.pendingTickersEvent fires on every tick
      → ticker.bid / ticker.ask / ticker.last populate

Failure modes:

  • Contract not qualified before subscription → conId == 0 → callbacks never fire → bid/ask stay at NaN forever → silently degraded.
  • Multiple subscriptions on same clientId competing → some return data, some don’t, no error visible.
  • ib_async’s default Ticker initial state is NaN; tick callbacks may take seconds to land for low-volume contracts → premature read sees NaN, reports snapshot_fallback.

Validation: direct test from a fresh clientId - bid > 0 AND ask > 0 within 5s on a liquid 0DTE option.

Layer 2 - Backend SSE serialization

ticker_map[conid] = {bid, ask, last, mid, last_tick_ts}
  → build SSE payload
    → json.dumps(payload, allow_nan=False)  # NaN/inf → null
      → write to /api/positions/stream

Failure modes:

  • json.dumps default allow_nan=True emits bare NaN literals. JSON.parse in browser rejects → entire SSE event dropped silently.
  • Float NaN comparison gotchas leak into upstream payload (e.g., pnl = (current - entry) * qty where current = NaN → pnl = NaN).
  • Cache TTL > 1s means SSE pushes the same stale value repeatedly - the stream is alive, the data is dead.

Validation: curl /api/positions/stream | python -c 'json.loads(...)' must NEVER raise, and 'NaN' not in raw must be true.

Layer 3 - Frontend DOM diff and reconciliation

EventSource('/api/positions/stream').onmessage = (evt) => {
  d = JSON.parse(evt.data)
  applyPositionUpdates(d.positions)  # must replaceChildren on empty
}

Failure modes:

  • Empty positions: [] taken as no-op rather than authoritative empty. Stale cards never clear.
  • Competing data sources (WS path + SSE path) where one overwrites the other with stale snapshots.
  • Frontend computes P&L from cached entry × current instead of using backend-provided pnl_dollars → sign/color bugs on shorts.

Validation: open/close a paper position 3 times; card must appear in ≤ 2s of fill and disappear in ≤ 2s of close, no ghost cards.

The trap: each layer can silently fall back

If Layer 1 returns NaN, Layer 2 sees NaN, falls back to ib.positions().marketPrice polling, and stamps the SSE payload with tick_source: 'snapshot_fallback'. The system keeps “working” but with 1-5 second update latency instead of <1s. Visually, the operator sees a yellow badge instead of green - easy to miss in the middle of a trade.

The fallback is a feature (honest degradation > silent stale data), but it means EVERY shipped fix must be validated end-to-end with all three layers, not just at one layer.

How to test all three layers

Mocked unit tests cover Layer 2 and Layer 3 in isolation. They cannot test Layer 1. A real-broker integration harness is required:

  1. Open a 1-contract paper position on a liquid ATM call.
  2. Within 5s, GET /api/positions/stream and assert:
    • tick_source == 'stream' (Layer 1 healthy)
    • bid > 0 AND ask > 0 and NOT null/NaN (Layer 1+2 healthy)
    • market_price within $0.10 of live broker mid (cross-validates Layer 1 against the real Gateway)
  3. Wait 10s, GET again, assert market_price has changed at least once (proves tick stream, not constant fallback).
  4. Frontend DOM check via browser snapshot or gstack /qa: card renders with all fields populated and updates visibly.
  5. Cut the position. Card clears within 2s.

If any of these fails, the dashboard is not real-time - it’s snapshot-polling dressed in SSE clothing.

The MK3 alternative

This three-layer fragility is inherent to a polling/cache architecture retrofitted to look event-driven. MK3 on Nautilus Trader has event-driven streaming as a framework primitive - message-bus subscribers receive tick events directly, no polling, no cache TTL, no fallback mode. The MK3 dashboard piggybacks on the same message bus the engine reads from, so by construction the dashboard sees the same tick the engine sees, at the same time.

For MK2, the right answer is to make Layer 1 reliable, sanitize Layer 2 aggressively, and DOM-diff Layer 3 correctly - and to validate all three via a real-broker harness on every ship.

Graph View