Arch Documentation

New Page

Architecture: current and target

This document describes the system's design: an Engine, any number of Providers and one Exchange service per trading venue. Phases 1–5 of the migration (§8) are implemented: the engine talks to the exchanges only through this contract, and §3 now describes the architecture before the migration. docs/provider-protocol.md stays the reference for the legacy PricingInfo subject that Internal and the bookmaker feeds still use (phase 6).

1. Purpose

Use exchanges and providers to find markets that are priced wrong, and buy them when they are cheaper than the real probability.

The system is run by one person: one operator, one wallet per exchange. There are no user accounts or roles.

2. Roles

Role What it does Talks over Owns
Engine Receives prices from exchanges and providers, matches feeds of the same game, builds the consensus line, runs strategies (paper and real), executes on an exchange, shows metrics (dashboard) and logs matched data to Influx. Never names a specific exchange in code. NATS (in), HTTP (to exchanges), NATS WebSocket (dashboard) GlobalGameId, game state (Redis), mappings, trades and paper fills (Mongo), price history (Influx)
Provider Unmanaged price source, possibly on a remote network. Publishes its games and prices; managing it is optional. NATS only (publish) Its own GameIds
Exchange A price provider that can also trade on its own platform (today: Polymarket). One instance per exchange. Publishes prices like a provider and serves an HTTP API for availability, subscriptions, trades and orders. NATS (publish) + HTTP API Its GameIds (e.g. a Polymarket slug and its -more-markets slug under one id), AssetIds, wallet, orders

What changes against today: trading (order placement, the wallet's fills, market resolution) moves from the engine into the exchange, and the engine stops knowing which exchange it talks to. Strategies, paper trading, matching and consensus stay in the engine.

3. Architecture before the migration

flowchart LR
    subgraph Providers
        INT["Internal (Override)"]
        BK["Bookmaker feeds (*-TX ...)"]
    end
    subgraph PP["PolymarketProvider"]
        GAM["Gamma discovery + mapper"]
        WS["CLOB WebSockets"]
        MON["Monitor :8080"]
    end
    subgraph ENG["ArbitrageEngine"]
        ROUTE["Routing / matching"]
        CONS["Consensus"]
        STRAT["Strategies"]
        PAPER["PaperTrader"]
        LIVE["LiveTrader / ManualTrader"]
        HIST["TradeHistory (wallet poll)"]
        RES["MarketResolver (Gamma)"]
        WEB["Dashboard :8088"]
    end
    EXEC["Python executor (FastAPI)"]
    PMAPI["Polymarket data-api / Gamma"]
    NATS(("NATS"))
    REDIS[("Redis")]
    MONGO[("Mongo")]
    INFLUX[("Influx")]

    INT -- "arbitrage.prices.internal" --> NATS
    BK -- "arbitrage.prices.*" --> NATS
    PP -- "prices, catalog, retired, trades" --> NATS
    NATS -- "subscriptions, refresh" --> PP
    NATS --> ROUTE --> CONS --> STRAT
    STRAT --> PAPER
    STRAT --> LIVE --> EXEC
    HIST --> PMAPI
    RES --> PMAPI
    ENG --> REDIS
    ENG --> MONGO
    ENG --> INFLUX
    WEB -. "arbitrage.ui.*" .- NATS

Today's control plane is all NATS (ControlSubjects in Arbitrage/PolymarketSportFeed/Engine.cs):

Subject Direction Content
arbitrage.prices.<source> provider → engine PricingInfo
arbitrage.catalog.polymarket provider → engine CatalogMessage, every 60 s
arbitrage.control.polymarket.subscriptions engine → provider SubscriptionSet (full wanted set, every 60 s and on change)
arbitrage.control.polymarket.sync provider → engine ask for the wanted set (startup)
arbitrage.control.polymarket.refresh engine → provider resend these events' books
arbitrage.control.polymarket.retired provider → engine events finished
arbitrage.trades.polymarket provider → engine MarketTrade, the public trade tape

Things in the current shape that the target removes:

4. Target architecture (implemented)

flowchart LR
    subgraph PROV["Providers (unmanaged, may be remote)"]
        P1["Internal"]
        P2["Bookmaker feeds"]
    end
    subgraph EXS["Exchanges (one per venue)"]
        X1["Polymarket Exchange"]
        X2["Future exchange"]
    end
    subgraph ENG["Engine"]
        ROUTE["Matching"]
        CONS["Consensus"]
        STRAT["Strategies"]
        PAPER["Paper trading"]
        EXEC["Execution client"]
        RECON["Subscription reconcile"]
        ORD["Order tracking"]
        WEB["Dashboard"]
    end
    NATS(("NATS"))
    REDIS[("Redis")]
    MONGO[("Mongo")]
    INFLUX[("Influx")]

    P1 -- "Book / Update" --> NATS
    P2 -- "Book / Update" --> NATS
    X1 -- "Book / Update" --> NATS
    X2 -- "Book / Update" --> NATS
    NATS --> ROUTE --> CONS --> STRAT
    STRAT --> PAPER
    STRAT --> EXEC
    EXEC -- "POST /trade" --> X1
    ORD -- "GET /orders" --> X1
    RECON -- "GET /availability, POST /subscription" --> X1
    RECON -. "same API" .-> X2
    ENG --> REDIS
    ENG --> MONGO
    ENG --> INFLUX

5. Contracts

Enum integer values are part of the wire format: never reorder them, only append. The market enums (Period, MarketType, MarketTarget) and their conventions (home-margin spreads, 3-way moneylines, team totals) stay exactly as in docs/provider-protocol.md.

5.1 Exchange HTTP API

Method Path Purpose
GET /health Liveness, identity and money available
GET /availability Every game the exchange can open, with its state
POST /subscription Open or close one game's price stream
POST /trade Place an order
GET /orders Our orders, their fills and settlement
GET /markets?assetIds=a,b Market state of any assets (positions that are not orders: paper fills)

The API moves money: it must only be reachable from the engine's network (see §9). An exchange may also require a shared key in the X-Api-Key header on /trade and /orders.

GET /health

{ "time": "2026-10-24T18:02:11Z", "name": "Polymarket", "liquidityAvailable": 412.37 }

liquidityAvailable is the balance usable for trading, in the exchange's settlement currency (USDC for Polymarket).

GET /availability

[
  {
    "gameId": "pm-7f3a1c",
    "teamNames": ["Boston Celtics", "New York Knicks"],
    "matchDate": "2026-10-25T00:00:00Z",
    "startTimeUtc": "2026-10-25T23:30:00Z",
    "updateType": "Accumulative",
    "isEnabled": true
  }
]

POST /subscription

{ "gameId": "pm-7f3a1c", "enable": true }

One call per game. enable: true opens the stream (if not open already) and the exchange starts publishing Updates for it; false stops it. The call is idempotent.

The engine does not trust its own memory of what is open: every 60 s it reads /availability, compares isEnabled with the games it wants, and sends /subscription for every difference. An exchange restart (everything isEnabled: false) therefore heals within a minute, and the exchange never has to persist subscriptions. The engine only disables games it enabled itself in this run, so a game enabled by hand on the exchange stays open.

enable: true on a game that already streams is a resend: the exchange forgets its last top-of-book for that game and publishes the whole book again (the engine asks for it after clearing a board, e.g. a series split, and at startup for the pregame games). Unknown gameId → 404.

POST /trade

Request (by AssetId, the normal path):

{
  "clientOrderId": "tennis-live-v2:9c1e...:1",
  "assetId": "71321045679252212594626385532706912750332728571942532289631379312455583992563",
  "amount": 5.0,
  "sell": false,
  "price": 0.43,
  "tradeType": "FAK",
  "expirationSeconds": 0
}

Or by market, when the caller has no AssetId: replace assetId with "target": { "gameId": "pm-7f3a1c", "period": "Game", "type": "Moneyline", "targetType": "Home", "points": 0 } and the exchange resolves it with its mapper.

Field Meaning
clientOrderId Generated by the engine (ui:<guid> for the dashboard, live:<strategy>:<asset>:<ticks> for real-money strategies). The exchange stores the order under it before sending it and answers Declined: duplicate clientOrderId (with the first order's orderId) to a second one, so a retry after a timeout can never buy twice.
amount Buy: money to spend (before the exchange's fee). Sell: shares to sell. Each exchange also has its own per-trade cap.
price Limit price in the asset's own terms (required). Rounded to the tick so the limit always holds: a buy is rounded down, a sell up.
tradeType FAK (fill what the book has at or under the limit, cancel the rest), FOK (all or nothing), GTD (rest on the book until expirationSeconds).
expirationSeconds Only for GTD. Polymarket clamps it to 120–600 s (the CLOB wants at least 180 s ahead and the executor adds 60 s).

Response:

{ "status": "Delayed", "message": "in-play order delay", "orderId": "0xab12...", "clientOrderId": "tennis-live-v2:9c1e...:1", "price": 0.43 }

status: Accepted (matched or resting), Declined (rejected, message says why; a FAK/FOK that matched nothing is Declined on Kalshi), Delayed (outcome pending: Polymarket holds in-play sports orders before matching, and a delayed order may never fill; a timeout or lost answer is Delayed too, also on the engine's side when the exchange doesn't answer). price is the limit actually sent. The response never carries amounts: the final outcome always comes from /orders.

GET /orders?since=<ISO time>

OrderStatus adds Resting (a GTD order on the book with nothing matched yet).

[
  {
    "orderId": "0xab12...",
    "clientOrderId": "tennis-live-v2:9c1e...:1",
    "gameId": "pm-7f3a1c",
    "assetId": "7132...2563",
    "sell": false,
    "status": "Filled",
    "shares": 11.62,
    "avgPrice": 0.43,
    "filledAt": "2026-10-24T18:02:14Z",
    "marketStatus": "Resolved",
    "payout": 1.0,
    "currentPrice": 1.0
  }
]
Field Meaning
status Delayed, Filled, Partial, Unfilled (delayed or GTD order that expired without a match), Rejected, Resting
marketStatus Open, Closed (trading stopped, result pending), Resolved
payout Per share once resolved: 1 won, 0 lost, 0.5 void
currentPrice Latest price of the token, to value open positions

This replaces the engine's old wallet poll and Gamma resolution. It also lists orders placed outside the engine (e.g. on the exchange's website), with no clientOrderId, so the Bets tab still sees the whole wallet. Fills never travel on NATS.

GET /markets?assetIds=a,b

[ { "assetId": "7132...2563", "marketStatus": "Resolved", "payout": 1.0, "currentPrice": 1.0, "title": "Celtics vs. Knicks" } ]

The same market state as /orders, for assets that are not our orders: the engine resolves paper fills with it (MarketResolver, every 5 min, batches of 50).

5.2 NATS messages

Every message uses one envelope:

{ "messageType": "Book", "publisher": "Polymarket", "content": { } }

Subject: arbitrage.<messageType lower case>.<publisher lower case>, e.g. arbitrage.update.polymarket, so the engine subscribes to arbitrage.book.> and arbitrage.update.>.

Book

content is the same list as /availability. An exchange lists only its streaming games; a provider lists every game it has. Sent at startup, whenever the list changes (at most one per 5 s) and about every 5 minutes.

Update

content is one game: its gameId and the list of its markets that changed. One Update message per game.

{
  "messageType": "Update",
  "publisher": "Polymarket",
  "content": {
    "gameId": "pm-7f3a1c",
    "markets": [
      {
        "period": 1, "type": 1, "targetType": 3, "points": 0,
        "price": 0.43,
        "bestBid": 0.41,
        "timestampMilliseconds": 1792951334000,
        "assetId": "7132...2563",
        "reference": "Boston Celtics"
      }
    ]
  }
}

5.3 Enums

Enum Values
UpdateType Accumulative = 0, Override = 1
TradeType FAK = 0, FOK = 1, GTD = 2
TradeStatus Accepted = 0, Declined = 1, Delayed = 2
OrderStatus Delayed = 0, Filled = 1, Partial = 2, Unfilled = 3, Rejected = 4, Resting = 5 (GTD on the book, nothing matched yet)
MarketStatus Open = 0, Closed = 1, Resolved = 2

6. Sequences

6.1 Discovery, matching and subscription

sequenceDiagram
    participant X as Exchange
    participant P as Provider
    participant N as NATS
    participant E as Engine

    P->>N: Book (all its games)
    N->>E: Book
    E->>E: match games (team names + date)
    X->>N: Book (enabled games)
    E->>X: GET /availability
    X-->>E: games, isEnabled
    E->>E: match, decide wanted set (game also has an Internal feed, or opened by hand)
    E->>X: POST /subscription {gameId, enable: true}
    X->>N: Update (full book first, then changes)
    N->>E: Update
    E->>E: consensus, strategies

    loop every 60 s
        E->>X: GET /availability
        X-->>E: isEnabled per game
        E->>X: POST /subscription for each difference
    end

    Note over X: restart: nothing enabled
    E->>X: GET /availability (next loop)
    E->>X: POST /subscription again for every wanted game

6.2 A strategy buy

sequenceDiagram
    participant S as Strategy
    participant E as Engine (execution)
    participant X as Exchange
    participant M as Mongo

    S->>E: signal (exchange, assetId, ask, fair)
    E->>X: POST /trade {clientOrderId = order id, assetId, amount, price, FAK}
    X->>X: store order by clientOrderId, send it
    X-->>E: Accepted / Declined / Delayed
    E->>M: StrategyOrders (strategy, fair, outcome)
    loop every 60 s (exchange): wallet fills
        X->>X: fill → oldest open order of ours on that token and side
    end
    loop every 60 s (engine, TradeHistory)
        E->>X: GET /orders?since=now-2d
        X-->>E: shares, avgPrice per order (+ fills placed elsewhere)
        E->>M: Trades: new fill tagged with the order's strategy + fair (by clientOrderId)
    end
    Note over E: no fill 5 min after a Delayed order: Unfilled, token and game slot free again (max 3 attempts)
    loop every 5 min (engine, MarketResolver)
        E->>X: GET /markets?assetIds=... (tokens not resolved yet, real and paper)
        X-->>E: marketStatus, payout, currentPrice
        E->>M: TradeAssets → Won / Lost / Void on the Bets tab
    end

6.3 Engine startup

sequenceDiagram
    participant E as Engine
    participant R as Redis
    participant X as Exchange
    participant N as NATS

    E->>X: GET /health (each host in "exchange", up to 10 s): its name
    E->>R: restore games and feed mappings
    E->>N: subscribe arbitrage.prices.>, arbitrage.book.>, arbitrage.update.>
    E->>X: GET /availability (reconcile loop, at once and every 60 s)
    Note over E: known games route at once (mapping restored)
    Note over E: a provider's new game waits for its next Book (at most ~5 min)
    E->>X: POST /subscription for wanted games not streaming
    E->>X: POST /subscription (resend) for the pregame games, 30 s later

7. Before → after the migration

Before Now
Arbitrage/PolymarketProvider Polymarket Exchange (same discovery, mapper and WebSockets, plus the HTTP API)
CatalogMessage on arbitrage.catalog.polymarket Book + GET /availability
SubscriptionSet on ...subscriptions / ...sync POST /subscription + the engine's 60 s reconcile
...retired game drops out of Book / /availability
...refresh (resend books) POST /subscription {enable: true} on an already open game resends its full book
arbitrage.prices.<source> + PricingInfo Update envelope; team names and dates move to Book
PricingInfo.InternalId (provider feed id) gameId (owned by the publisher)
MarketPrice.ConditionId/NegRisk/TickSize/BestAsk removed from the wire, kept inside the exchange
ManualTrader / LiveTrader → Python executor POST /trade on the exchange; the executor becomes the exchange's internal detail
TradeHistory (data-api wallet poll) GET /orders
MarketResolver (Gamma resolution) GET /orders (marketStatus, payout)
PaperTrader unchanged, stays in the engine
arbitrage.trades.polymarket (public tape) and maker-v3 removed; maker-v3 is retired, its paper fills stay in Mongo
"Polymarket" checks in the engine (strategies, dashboard, subscriptions) "is an exchange" (it has an HTTP API in config)
Engine.UpdateModes (hard-coded) updateType in Book / /availability

8. Migration phases

Each phase deploys on its own and keeps today's behaviour working. Status (2026-10-24): phases 1–5 are implemented together (the engine reads only Book/Update from exchanges; the Polymarket provider is now Arbitrage/PolymarketExchange); phase 6 waits on the Internal feed, which lives outside this repo.

  1. Envelope alongside the old subjects. The engine accepts Book/Update as well as PricingInfo. The Polymarket provider publishes both. updateType from Book replaces the hard-coded UpdateModes when present.
  2. Polymarket exchange API, read side. GET /health, GET /availability, POST /subscription in PolymarketProvider, with a stable gameId per event + companion (mapped from today's ids so Redis/Mongo/Influx history keeps working).
  3. Engine reconciles over HTTP. Exchange base URLs in config; the reconcile loop replaces SubscriptionLoop; catalog/subscriptions/sync/refresh/retired subjects are removed from both sides.
  4. Trading moves to the exchange. POST /trade and GET /orders in the exchange (it takes over the executor client, the wallet poll and Gamma resolution). The engine's LiveTrader, ManualTrader, TradeHistory and MarketResolver call them instead. maker-v3 is unregistered.
  5. Engine without exchange names. Replace the "Polymarket" checks with the exchange role; signals carry the exchange name. Remove the Polymarket-only fields from MarketPrice.
  6. Providers on the envelope. Internal and the bookmaker feeds switch to Book/Update; then the engine drops PricingInfo and docs/provider-protocol.md is rewritten from §5. (Not done: the engine already routes a provider's Book as its full game list and accepts its Updates.)

What the engine does with the contract

9. Production concerns

10. Implemented exchanges

Kalshi (Arbitrage/KalshiExchange)

The first service written against this contract (the shared types are in Arbitrage/Contracts). The engine consumes it like any exchange: its asks show on the dashboard and Top edges, and the paper strategies run on it (<profile>@kalshi); the dashboard's Buy is off for it unless engine:trading:exchanges lists it.

Polymarket (Arbitrage/PolymarketExchange)

The former PolymarketProvider. GameId = the Gamma event id (the feed id the engine already had, so Redis, Mongo and Influx history carry on); the -more-markets companion is merged under it.

Betfair (Arbitrage/BetfairExchange)

Betfair Exchange (global API-NG endpoints), port 8092, publisher Betfair. Paper only in the engine (the @betfair strategy copies); its own /trade is off unless betfair:trading is true.

SX Bet (Arbitrage/SxBetExchange)

SX Bet V3 order book (https://api.sx.bet, or the Toronto testnet with sxbet:testnet), port 8080 (sxbet:port), publisher SxBet. Paper only in the engine (the @sxbet strategy copies); its own /trade is off unless sxbet:trading is true and sxbet:privateKey is set. SX Bet is an on-chain peer-to-peer exchange: every market is a binary question with two outcomes, odds are implied probabilities on a 10^20 scale quoted from the maker's side, and the taker of a level pays 1 − maker. Funds live in a proxy wallet (USDC on SX's chain) deployed from sx.bet.

Limitless (Arbitrage/LimitlessExchange)

Limitless Exchange (https://api.limitless.exchange, Base mainnet, chain 8453, no testnet), port 8080 (limitless:port), publisher Limitless. Paper only in the engine (the @limitless strategy copies); its own /trade is off unless limitless:trading is true and limitless:privateKey is the EOA of the profile behind the token. Limitless is a CLOB prediction market: every market is a YES/NO pair of conditional tokens backed by 1 USDC, quoted 0–1 in USDC per share on one order book (the YES side; a YES bid at q is a NO ask at 1 − q). Its sports markets are replications of Polymarket's and of a sports data feed (metadata.externalProvider).

Novig (Arbitrage/NovigExchange)

Novig's exchange API v3 (https://api.novig.com, or the play-money exchange https://api.paper.novig.com with novig:paper; docs at docs.novig.com), port 8080 (novig:port), publisher Novig. Paper only in the engine (the @novig strategy copies); its own /trade is off unless novig:trading is true and the key is a trading key. Novig is a US peer-to-peer sports exchange: every market has two outcomes, each with its own ladder of bids (an order always buys an outcome), a price is the outcome's probability on a grid (0.001 steps up to 0.050 and from 0.950, 0.005 between) and one contract pays 1¢, so 100 contracts are one share of the model. Takers pay c × P (1 − P) per contract (fee.coefficient, 0.06 on every game market) but only while the event is in play (charged = WHEN_LIVE); makers pay nothing.

Pinnacle (Arbitrage/PinnacleExchange)

Pinnacle's official REST API (https://api.pinnacle.com, or pinnacle:baseUrl for a white-label such as ps3838), port 8093 (pinnacle:port), publisher Pinnacle. Paper only in the engine (the @pinnacle strategy copies); its own /trade is off unless pinnacle:trading is true. Pinnacle closed new API access to the public in July 2025: this needs an account that already has it (HTTP Basic auth, pinnacle:username / password). A 401/403 stops every call until a restart (repeated failed logins lock the account). Pinnacle is a bookmaker, not an order book: one price per outcome, its margin included, and a bet is taken whole at the current line or not at all.