Building a crypto market-making pipeline: from Elixir execution to ML signal generation on Ray

A market-making system is a split personality. One half lives in microseconds: hold quotes on both sides of the book, cancel and replace as the market moves, never freeze while money is on the table. The other half lives in seconds and gigabytes: ingest history, train models, score features, decide where the fair price sits and how wide to quote. Build both halves in the same runtime and one of them suffers — either the order loop stalls behind a model, or the model is starved to keep the order loop quick.

The design that worked split them by runtime. Elixir runs the execution layer, where its concurrency model and fault tolerance are exactly the right tool. Ray runs the signal layer, where Python's ML ecosystem and distributed compute live. A narrow, deliberate seam connects them. This is how the pieces fit and where the seam has to be defended.

Why Elixir owns execution

The execution layer's whole job is to stay responsive while juggling many independent, failure-prone things at once: a websocket per exchange, a process tracking each open order, timers for quote refresh, reconciliation against fills. This is the problem the BEAM was built for.

Each exchange connection is a supervised process. Each instrument's quoting logic is its own process holding its own state. A crash in one — a malformed message, a rejected order, a dropped socket — is isolated by the supervisor and restarted clean, while every other instrument keeps quoting. Clean means empty: the restarted quoter comes back with no fair value and holds no quotes until the next signal arrives, which fails safe rather than resuming on state that may have gone stale during the crash. There is no shared mutable book to corrupt and no global lock to stall the system when one venue misbehaves.

defmodule Execution.Quoter do
  use GenServer

  # One process per instrument, registered under its symbol so the signal
  # feed can address it by name. Its mailbox serializes every event for
  # that instrument, so quote state mutates without locks.
  def start_link(symbol) do
    GenServer.start_link(__MODULE__, symbol,
      name: {:via, Registry, {Quoters, symbol}})
  end

  def init(symbol) do
    {:ok, %{symbol: symbol, fair: nil, bid: nil, ask: nil}}
  end

  # A fresh fair value arrives from the signal layer.
  def handle_cast({:fair_value, fair, half_spread}, state) do
    bid = fair - half_spread
    ask = fair + half_spread
    :ok = Venue.replace_quotes(state.symbol, bid, ask)
    {:noreply, %{state | fair: fair, bid: bid, ask: ask}}
  end

  # A fill came back. React immediately; do not wait on the model.
  def handle_info({:fill, side, qty, price}, state) do
    Risk.record_fill(state.symbol, side, qty, price)
    {:noreply, state}
  end
end

The point of that snippet is what it does not do. It never calls a model. It never blocks on the network beyond the venue it owns. A fill is handled the instant it arrives, because reacting to a fill late is how a market maker accumulates the position it did not want.

TODO(founder): the concrete reason latency mattered here — the venues, the quote refresh rate you needed to hold, and an incident where a slow path cost a fill or a fee tier.

Why Ray owns signals

Fair value and spread are the output of real computation: feature pipelines over order-book and trade history, models that need training and periodic retraining, and scoring that is far too heavy to sit in the order loop. That is Ray's territory.

Ray distributes the work that does not fit on one core or one box. Feature computation fans out across the cluster. Model training runs as a Ray job. A long-lived Ray actor holds the loaded model in memory and answers scoring requests, so the model is loaded once, not per request.

import ray

@ray.remote(num_gpus=0.25)
class FairValueModel:
    def __init__(self, weights_uri: str):
        self.model = load_model(weights_uri)   # loaded once, held in the actor

    def score(self, features: dict) -> dict:
        fair = self.model.predict(features)
        half_spread = self.spread_policy(features, fair)
        return {"fair": fair, "half_spread": half_spread}

Holding the model in a Ray actor is the equivalent of the Elixir discipline on the other side: do the expensive setup once, then serve cheap requests. Retraining produces new weights, a new actor loads them, and traffic cuts over without the execution layer ever knowing the model changed underneath it.

TODO(founder): the real signal — what the model actually predicts, the features that mattered, and how often you retrained. Keep it as specific as you are comfortable publishing.

The seam between them

Two runtimes have to talk, and the seam is where this architecture is won or lost. The constraint is non-negotiable: nothing the signal layer does may ever block the execution layer. The model can be late. The order loop cannot.

So the seam is asynchronous and lossy by design. The signal layer pushes fair-value updates toward execution; the execution layer applies the latest one it has and never waits for the next. If a model update is slow, the quoter keeps quoting on the last good fair value rather than freezing for a fresh one. A stale quote is a managed risk. A frozen quoter is an unmanaged one.

Concretely, the two sides exchange messages over a transport that decouples their clocks — a message bus or a small socket protocol — carrying fair value and spread one way and, optionally, fills and inventory the other way so the model can condition on current position. The model returns {fair, half_spread} for a symbol; whatever transport carries it tags on the symbol, and the Elixir feed decodes the result into {:signal, symbol, fair, half_spread} — the symbol is what lets the feed route to the right quoter.

defmodule Execution.SignalFeed do
  use GenServer

  # Inbound fair-value updates from the Ray layer. Fan out to the quoter
  # for the symbol and move on. No blocking call crosses this boundary.
  def handle_info({:signal, symbol, fair, half_spread}, state) do
    GenServer.cast({:via, Registry, {Quoters, symbol}},
      {:fair_value, fair, half_spread})
    {:noreply, state}
  end
end

cast, not call. The execution side fires the update at the quoter and returns immediately. The boundary carries no synchronous call in the hot direction, so a stall on the Ray side can never propagate into the order loop. That single discipline — async in, latest-wins, never block — is what lets two very different runtimes share a system without one dragging down the other.

TODO(founder): the transport you actually used between Elixir and Ray, why you chose it, and any failure where the seam leaked latency or a message backed up.

What this buys

Splitting the system by runtime let each half be good at one thing. Elixir's execution layer stays responsive and fault-isolated under venue churn and message floods, because that is what the BEAM does. Ray's signal layer scales the heavy compute and serves models from memory, because that is what Ray does. The seam keeps them honest: asynchronous, latest-wins, and structurally incapable of letting a slow model freeze a live quote.

The trade is operational. Two runtimes mean two deployment stories, two monitoring surfaces, and a boundary to defend on every change. For a market maker, where the cost of a stalled order loop is paid in real inventory, that trade was clearly worth making.

TODO(founder): the bottom line — what this pipeline let you do that a single-runtime build could not, stated plainly and honestly.