Nautilus Trader - Developer Guide (Extension + Contribution)

Nautilus Trader - Developer Guide

The developer guide is the contract for anyone extending the platform. It assumes the Rust core / Python control plane / PyO3 bridge architecture from the concepts guide and tells you how to actually plug new venues, new strategies, and new data types into it without breaking determinism, replay, or the FFI memory model.

The headline rule that runs through every section: build the Rust core first, then the Python wiring. Adapters that try to live entirely in Python are explicitly second-class - for new work, the Rust + PyO3 stack is the supported path. New live adapter examples in the v2 path target nautilus_trader.live.LiveNode; the older TradingNode is legacy v1/Cython runtime only.

Sub-pages covered: Environment Setup, Design Principles, Coding Standards (Rust + Python), Testing, Test Datasets, Docs Style, Releases, Adapters, Data Testing Spec, Execution Testing Spec, Benchmarking, FFI Memory Contract.

Writing custom data adapters

This is the answer to “how would Cortana plug Unusual Whales into Nautilus.”

Layered structure (mandatory). Every adapter splits across two trees:

crates/adapters/your_adapter/        # Rust core
├── src/
│   ├── common/        # consts, credential, enums, parse, urls, retry, testing
│   ├── http/          # client.rs, error.rs, models.rs, parse.rs, query.rs
│   ├── websocket/     # client.rs, dispatch.rs, handler.rs, messages.rs, parse.rs
│   ├── python/        # PyO3 bindings (mod.rs, http.rs, websocket.rs, urls.rs, enums.rs)
│   ├── config.rs      # bon::Builder + Default
│   ├── data.rs        # Data client implementation
│   ├── execution.rs   # Execution client implementation
│   ├── factories.rs   # Factory functions
│   └── lib.rs
├── tests/             # Mock-Axum integration tests
└── test_data/         # Canonical venue payloads

nautilus_trader/adapters/your_adapter/   # Python wiring
├── config.py    # LiveDataClientConfig / LiveExecClientConfig subclasses
├── data.py      # LiveDataClient or LiveMarketDataClient
├── execution.py # LiveExecutionClient
├── factories.py # Convert venue payloads to Nautilus models
├── providers.py # InstrumentProvider
└── constants.py

The Rust layer owns HTTP/WS networking, request signing, rate limiting, and parsing. The Python layer owns the engine-facing interface (LiveDataClient, LiveMarketDataClient, LiveExecutionClient, InstrumentProvider) and the configuration the user actually instantiates.

Implementation sequence (dependency-driven). The guide is explicit that this is not optional ordering - each phase depends on the previous one:

1. Phase 1: Rust core infrastructure. HTTP error types → HTTP client → HTTP models → HTTP parsing → WS error types → WS client → WS messages → WS parsing → PyO3 bindings. Milestone: cargo test passes; client can authenticate and stream raw bytes.
2. Phase 2: Instruments. Parse venue instrument definitions into InstrumentAny variants (spot, perpetual, future, option). Implement InstrumentProvider. Handle symbol normalization (suffixes like -LINEAR, -PERP, -OPTION; Ustr interning; case normalization). Milestone: InstrumentProvider.load_all_async() returns valid Nautilus instruments.
3. Phase 3: Market data. Public WS streams (book/trades/tickers/quotes) + historical HTTP requests + Python LiveDataClient wiring. Milestone: subscribed data lands on the message bus.
4. Phase 4: Execution. Private WS streams (orders/fills/positions/account) + order submission/modify/cancel + LiveExecutionClient + reconciliation reports for startup. Milestone: orders submit, fills arrive, state reconciles on connect.
5. Phase 5: Advanced features. Conditional orders, brackets, batch ops, venue-specific data (funding rates, options chains, liquidations).
6. Phase 6: Configuration and factories. LiveDataClientConfig / LiveExecClientConfig subclasses, factory functions, env-var credential resolution.
7. Phase 7: Testing and documentation. See Testing strategy below.

Required Rust patterns.

• Symbol normalization in common/symbol.rs: implement format_instrument_id (venue → Nautilus InstrumentId, e.g. "BTCUSDT" + Linear → "BTCUSDT-LINEAR.BYBIT") and format_venue_symbol (Nautilus → venue). Wrap symbols in a thin newtype like BybitSymbol for validation.
• URL resolution in common/urls.rs with const fn get_ws_base_url(testnet: bool) helpers; config structs expose base_url_http / base_url_ws overrides that fall back to these constants.
• Configs use bon::Builder + Default where Default delegates to the builder so defaults are defined exactly once. Plain T for fields that always have a sensible default; Option<T> only when None carries semantic meaning (“disabled”, “unbounded”, “inherit from environment”).
• Error taxonomy in common/error.rs: top-level enum that wraps VenueHttpError, VenueWsError, VenueBuildError with #[from] so ? propagates cleanly.
• Retry classification in common/retry.rs with Retryable / NonRetryable / Fatal variants and an optional retry_after: Option<Duration> for venues that send rate-limit hints.
• Adapter runtime rule (load-bearing): spawn tasks via nautilus_common::live::get_runtime().spawn(...), never plain tokio::spawn. Plain tokio::spawn panics when called from Python threads because they have no Tokio thread-local context. The check_tokio_usage.sh pre-commit hook enforces this. For sync→async bridges in adapters, use get_runtime().block_on(...).
• PyO3 stub annotations are mandatory for every Python-exposed type so generated .pyi stubs stay in sync. Use gen_stub_pyclass, gen_stub_pyclass_enum, gen_stub_pymethods, gen_stub_pyfunction.

Registration flow. From the user’s Python entrypoint:

node = (
    LiveNode.builder("TESTER-001", TraderId("TESTER-001"), Environment.SANDBOX)
    .with_data_engine_config(LiveDataEngineConfig(...))
    .add_data_client(None, adapter_data_client_factory, data_client_config)
    .add_exec_client(None, adapter_exec_client_factory, exec_client_config)
    .build()
)

The factory functions live in nautilus_trader/adapters/<adapter>/factories.py and instantiate the live clients from the config. The PyO3 bindings exposed from crates/adapters/<adapter>/src/python/ are what the factory wires through.

Writing custom execution adapters

Same crate skeleton as the data adapter - execution shares crates/adapters/<adapter>/ with data; only src/execution.rs, websocket/dispatch.rs, and the private WS topics differ. The Python side adds LiveExecutionClient (in execution.py) plus an adapter_exec_client_factory.

Required minimum surface to call an execution adapter “baseline compliant”:

• Submit market and limit orders (HTTP and/or WS).
• Modify and cancel single orders.
• Subscribe to private WS streams for OrderUpdate, Fill, Position, AccountState.
• Execution reconciliation on connect - generate order/fill/position status reports so the engine can rebuild local state from broker truth after a restart. This is non-optional; the spec acceptance tests check it.

For a non-standard broker (e.g., Cortana wrapping a fintech with REST-only order submission and no WS for fills), you’d implement the same layered shape but the WS dispatch becomes a polling loop on order/account endpoints, emitting the same OrderFilled / OrderAccepted / AccountState events the engine expects. The strategy’s view stays unchanged.

Custom strategy patterns

In Nautilus terms, “Cortana’s scoring engine” doesn’t become a Strategy directly - it becomes either:

• A Strategy subclass when it owns order placement and lifecycle.
• An Actor subclass when it only consumes data and publishes signals (no order lifecycle). Most signal/scoring engines are better as actors composed under a thin order-placing strategy.

Strategy lifecycle hooks (the only methods Nautilus drives):

• on_start() / on_stop() / on_resume() / on_reset() / on_dispose() - lifecycle
• on_instrument(instrument) - instrument loaded
• on_quote_tick(tick) / on_trade_tick(tick) / on_bar(bar) - market data
• on_order_book(book) / on_order_book_deltas(deltas) - book data
• on_order_filled(fill) / on_order_accepted(order) / on_position_opened(pos) / on_position_closed(pos) - execution lifecycle
• on_save() -> dict[str, bytes] / on_load(state) - persistence across restarts

All callbacks must be type-annotated. Type hints are not optional - the codebase enforces PEP 604 union syntax (Instrument | None) and rejects Optional[...].

Cortana mapping (concrete proposal for MK3):

• The composite scorer becomes an Actor that subscribes to bars/quotes/UW custom data and publishes a ScoreUpdate custom message on the bus.
• A thin CortanaStrategy(Strategy) subscribes to ScoreUpdate, runs the meta-labeling secondary classifier, applies the win-prob gate, and submits bracket orders via self.submit_order(...) against the IBKR adapter.
• Position-manager logic (TP/SL fallback, time-in-trade exits) lives in the strategy’s on_quote_tick and on_position_opened.
• All scoring-engine inputs are immutable messages (Design Principles guide is explicit: messages are never mutated after creation). The actor must derive new local state if it needs a different representation, never edit the input.

Testing strategy

Nautilus treats tests as executable specifications - there’s a formal “mechanism ladder” you climb only as input space grows.

Test layers (climb only when the layer below stops detecting regressions):

Layer	Trigger
Unit test	Single function, enumerable cases.
Parametrized test	Same shape across discrete inputs (order side, status, instrument).
Property-based test	Invariant must hold for a whole class of inputs you cannot enumerate.
Integration test	Multiple modules interact through real (non-mocked) engine or runtime.
Fuzz test	Untrusted bytes cross a parser, decoder, or wire-format handler.
Spec acceptance test	Behavior depends on a live venue contract (DataTester / ExecTester).
Deterministic simulation	Correctness depends on task scheduling, timeouts, or wall-clock ordering (DST).
Formal verification	Pure function with crisp invariants and bounded input space.

Module shape → which layers apply:

Module shape	Layers	Example
Pure function, crisp invariants	Unit, parametrized, property, fuzz	Reconciliation, portfolio math
Pure function, no invariants	Unit, parametrized, property, fuzz	Codecs, adapter parsers
Stateful, synchronous	Unit, parametrized, property over transitions	Cache, order book
Stateful, async	Unit, integration, deterministic simulation	Live engine, execution manager
I/O bound, venue contract	Integration, spec acceptance, boundary fuzz	Adapter client loops

Backtest harness. Nautilus uses the same engine code in backtest, sandbox, and live - there is no separate backtest runtime. You build a BacktestNode, register your strategy, feed historical data via BacktestEngineConfig and the data engine writes to the same message bus the strategy reads from in production. This is the core determinism property: replay yields the same logical inputs as the original run.

Mock data. Test providers live in python/tests/providers.py: TestInstrumentProvider and TestDataProvider give you canonical instruments and bar/tick streams.

Adapter spec acceptance tests. DataTester and ExecTester are concrete strategies/actors that run a defined matrix against a live venue:

• Data spec (TC-D01..TC-D60+): instruments, order book deltas, book at interval, book depth, snapshots, managed-book-from-deltas, quotes, trades, bars, mark price, index price, funding rate, custom data.
• Execution spec (TC-E01..TC-E50+): market BUY/SELL, IOC/FOK/GTC TIF, limit orders, post-only, cancel, modify, stop-loss, take-profit, brackets, batch ops, rejection handling, reconciliation on reconnect.

Each adapter must pass the subset matching its supported capabilities - this is the contract for “your adapter is shippable.” The matrix doubles as a self-documenting capability table.

Anti-patterns the guide explicitly calls out:

• Don’t capture log output to assert on log messages - verify observable behavior instead. Log capture is fragile because loggers are global, test order is non-deterministic, and assertions break when wording changes.
• Don’t use pytest.raises(BaseException) against PyO3 panic paths in python/tests/. Debug builds may pass; release wheels abort the interpreter.
• Don’t add debug_assert! where no test reaches it - release builds strip the check, so unexercised assertions have zero signal.

Property tests. Use proptest in Rust for round-trip serialization, inverse operations, and transitivity over Price, Quantity, UnixNanos, matching engine, and state machines.

Fuzzing. Required at network boundaries and exchange data parsers (JSON, FIX, WebSocket feeds). The system must return Result::Err on malformed bytes - never panic, hang, or leak.

Deterministic simulation testing (DST) preconditions before promoting an async module to DST:

• Time/task/runtime/signal primitives route through nautilus_common::live::dst, not tokio directly.
• Wall-clock reads go through nautilus_core::time, not SystemTime::now().
• Ordering-dependent state uses IndexMap / IndexSet, not default hash collections.
• Every tokio::select! on control-plane paths sets biased.
• No escape hatches: no Instant::now(), SystemTime::now(), tokio::signal::ctrl_c, std::thread::spawn, or tokio::task::spawn_blocking outside the seam.
• Replay-sensitive IDs (trade_id, venue_order_id) are pure functions of inputs.

Performance / Rust extension points

Rust is mandatory for networking clients, parsers, matching engines, accounting engines, and any data-plane loop measured in microseconds. The benchmarking section distinguishes:

• Criterion (wall-clock, confidence bands, ≥100ns) - use for any user-visible number or comparison. HTML reports under target/criterion/.
• iai (retired CPU instructions via Cachegrind, sub-100ns) - use for CI regression detection. Deterministic but machine-specific; never use for cross-machine comparison.

Most hot paths get both. Each crate keeps benches in crates/<crate>/benches/, registered in Cargo.toml via [[bench]] blocks. Workspace Makefile has make cargo-ci-benches for the nightly CI performance workflow.

Profiling. cargo flamegraph --bench <name> -p <crate> --profile bench produces a flamegraph.svg. The custom [profile.bench] inherits from release but sets debug = "full" so symbols stay readable. macOS requires sudo (DTrace), which leaves root-owned files in target/ you’ll need to clean up.

Python is mandatory for strategy logic, configuration, orchestration, the user-facing API. Anything user-replaceable. The PEP-8 + ruff + numpy-docstring conventions are enforced by pre-commit hooks.

When to write Rust instead of Python:

• Anything in the data engine hot path (bar building, book maintenance, tick routing).
• Parsers and codecs (JSON, FIX, WebSocket frames, Arrow IPC).
• Matching logic, fill simulation, accounting math.
• Anywhere you’d otherwise hit tokio::spawn from a hot loop.

When to keep Python:

• Strategy on_* callbacks (the PyO3 boundary swallows roughly all of the observable per-event cost relative to actual decision logic).
• ML inference where the model itself dominates (sklearn, PyTorch).
• Configuration, scripting, glue code.

FFI memory contract (load-bearing for any Rust↔Python boundary you add):

• Rust panics must never unwind across extern "C". Wrap every exported symbol in crate::ffi::abort_on_panic(|| { ... }). The panic message logs before the abort.
• CVec is a repr(C) thin wrapper around Vec<T> passed by value. The receiving Python/Cython code must call the type-specific drop helper exactly once (vec_drop_book_levels, vec_drop_book_orders, etc.). The generic cvec_drop was removed because it always treated buffers as Vec<u8> and corrupted the allocator on type mismatch.
• New PyO3 capsules must use PyCapsule::new_with_destructor with a closure that reconstructs the Box<T> or Vec<T> and drops it. Never PyCapsule::new(..., None) - that variant has no destructor and leaks.
• *_API Box-backed wrappers (e.g., OrderBook_API) must ship paired *_new / *_drop exports. Validate parameters before heap allocation.

Contributing back

Three-branch model: develop (active dev, dev wheels to Cloudflare R2), nightly (pre-release alpha wheels and CLI binaries), master (stable; merges trigger PyPI publish, Docker image build, docs rebuild).

Two version numbers tracked independently: pyproject.toml (Python package, e.g. 1.223.0) and Cargo.toml workspace (Rust crates, e.g. 0.55.0). The Python version drives the release tag (v1.223.0).

Pre-commit hooks enforce most of the conventions: check_anyhow_usage.sh, check_logging_macro_usage.sh, check_error_conventions.sh, check_tokio_usage.sh, check_copyright_year.sh. PRs that bypass these will fail CI.

Every Rust file requires the LGPL-3.0 standardized copyright header (current year enforced by hook). Logging macros must be fully qualified (log::info!, not info!). Error context strings are lowercase to chain naturally (except proper nouns / acronyms). String formatting uses inline format arguments (anyhow::bail!("Failed to subtract {n} months from {datetime}")), not positional.

Adapters live under crates/adapters/ (Rust) and nautilus_trader/adapters/ (Python). Adapter-only deps go in the workspace Cargo.toml “Adapter dependencies” section - pre-commit blocks core crates from using them.

For Cortana MK3 specifically

REUSE (do not reimplement):

• LiveNode kernel + lifecycle (start/stop/dispose, signal handling).
• Message bus (immutable messages, deterministic replay, in-process pub/sub).
• Cache (instrument, account, position, order state).
• Execution engine (order routing, order state machine, position tracking).
• Risk engine (pre-trade limits, max position, max loss; we wire LiveRiskEngineConfig).
• Reconciliation framework (broker-truth-first writes - reconciliation reports on connect/reconnect already match GH #46 / feedback_dual_tp_defense_in_depth).
• IBKR adapter (under nautilus_trader/adapters/interactive_brokers/ - this is exactly the data + execution wiring we’re currently building from scratch).
• Backtest engine (same code as live; just swap the data source).
• DataTester / ExecTester spec suites (free integration testing for our IBKR adapter usage).

BUILD (Cortana-specific):

• Composite scoring Actor - subscribes to bars/quotes/option-chain data, publishes ScoreUpdate custom messages.
• Meta-labeling secondary classifier - wraps the composite score, gates entries by win-probability. Lives as a downstream actor or as a Strategy.on_score_update filter.
• UW data adapter - full crates/adapters/unusual_whales/ Rust crate per the Phase 1-7 sequence above. Polls REST for net-flow/expiry, GEX, charm/vanna magnitudes; subscribes to UW WebSocket for tape-print events. Publishes as CustomData so any actor can subscribe.
• Cyclical encoding pipeline - actor that derives sin/cos features for minute-of-day, day-of-week, days-to-expiry; publishes alongside core features.
• Brain integration - actor that, on on_position_closed, logs the trade outcome to ~/brain for later retrieval (the gbrain runtime DB stays separate - the actor writes markdown via a queue).
• ML dashboard - out-of-band consumer that reads the message bus over Redis (Nautilus supports remote message-bus consumers) and renders win-prob, feature attribution, and live signal cards.
• Cortana strategy - thin Strategy subclass that consumes ScoreUpdate, applies the gate, sizes via the meta-prob weighted sizer, and submits brackets via the IBKR exec client.

Translation table for current Cortana → Nautilus:

Cortana current	Nautilus equivalent
cortanaroi/engine/scoring_engine.py	Actor subclass publishing ScoreUpdate
cortanaroi/engine/position_manager.py	Strategy.on_quote_tick + bracket orders
cortanaroi/engine/cooldown_state.py	Strategy local state, persisted via on_save
cortanaroi/data/uw_*.py	crates/adapters/unusual_whales/ + Python factory
cortanaroi/data/ibkr_*.py	Use shipped IBKR adapter (replace, don’t port)
cortanaroi/db/*.py	Cache + reconciliation + custom persistence sink
cortanaroi/dashboard/*.py	Out-of-band Redis subscriber to message bus
cortanaroi/ml/*.py	Stays Python; called from strategy/actor

Net assessment: The amount of code we’d delete in MK3 is larger than the amount we’d write. Roughly all of the data ingestion plumbing, position lifecycle, broker reconciliation, and replay/backtest scaffolding becomes “call into Nautilus.” The remaining Cortana-specific work is what we should have been spending all our time on anyway: scoring features, meta-labeling, strike selection, and strategy logic.

See also

• nautilus-concepts.md
• nautilus-integrations.md
• nautilus-how-to.md