Not another black-box AI trading tool. FutureMarket writes down every call it makes — with the entry price attached — then goes back days later and marks its own homework against what actually happened. Five model stages argue the case, quant factors score the tape, and nothing touches real money until it has earned it on paper.
Most LLM analysts hand the model a bag of tools and let it wander. The cost is unpredictable and the trace is impossible to reproduce. We inverted it: gather everything first with free calls, then spend exactly five model invocations. Every analysis costs the same and runs the same way.
Before any model runs, seven fetches go out — a quote, technical indicators, fundamentals, up to eight news articles and three fixed web searches. They are concatenated into a single markdown block. That block is the only thing the pipeline ever sees about the outside world. No retrieval loop, no agent deciding to look something up.
Then five stages run in order, each with its own system prompt and a payload assembled from what came before. An analyst synthesises the evidence. A bull and a bear each build the strongest case their way — from identical inputs, without seeing each other. A judge reads only the two briefs, deliberately starved of the raw evidence so it has to rule on argument quality rather than re-derive a view. Finally a risk committee sees everything and is the only stage bound to an output contract: a recommendation, a confidence percentage and a target price, in labelled form.
Only that fifth stage is machine-read. The other four are stored verbatim for a human to read and are never parsed. That is a deliberate split: the parts of the system that make decisions are small and inspectable, and the parts that produce prose stay prose.
Ask a language model to analyse a named company on a named historical date and it may not be analysing at all. It may simply be remembering what the stock did. The published framing of this failure is the “profit mirage” problem (arXiv:2510.07920), and it quietly invalidates a great many impressive-looking LLM backtests.
A backtest that lets a model recognise the company and the date is not measuring predictive skill — it is measuring recall. Any strategy validated that way looks excellent in sample and fails live, and the failure is inexplicable because the harness reported success.
So FutureMarket takes the direct countermeasure: strip the identifying information before the model sees it, and put it back afterwards. Six ordered passes rewrite the research context. Company names become Company A, Company B. Tickers map to the same labels, anchored on word boundaries and skipping anything under two characters so stray single letters do not match everywhere. Dates in three formats become Period 1, Period 2. Sector names become Sector 1. Company names are replaced longest-first, so “Apple Inc.” wins over “Apple”.
When the five stages finish, the same map runs in reverse across all five outputs, so the reports a human reads still name the real company. The guard engages when the pipeline is invoked in backtest mode; live analysis runs un-neutered, correctly, because there is nothing to protect against when the future genuinely has not happened yet.
No single model decides. Five independent sources each return a score normalised to −1…+1, so the fusion step is unit-free. Each carries a base weight — and then that base weight is adjusted by how well the source has actually been performing.
The weighting is the part worth dwelling on. Each source's base weight is multiplied by two performance terms. The Elo factor is that source's rating divided by the mean rating across all rated strategies — beat your peers head-to-head and you scale above 1.0. The calibration factor asks a different question: if this source says 70% and is right 80% of the time, scale it up; if it is right 60%, scale it down. Both terms are computed from reconciled outcomes, never from assumption.
Weights are then floored at 0.05 so no source is ever zeroed out entirely, and renormalised to sum to one. And then the compositor spends most of its remaining code looking for reasons not to act: a composite inside the deadband is a hold; confidence under 0.55 is skipped; a daily trade cap is skipped; a missing entry price is skipped; and a position check that fails is skipped rather than waved through — the exception path returns a refusal, which is the correct direction for a system that can spend money.
A prediction is only a claim until something checks it. One nightly job does the converting, and every other accountability mechanism in the system is downstream of that conversion.
When a call is made, a row goes into the prediction log carrying the direction, the predicted magnitude, the confidence, the strategy that produced it and — critically — the entry price at the moment of the call. Without that column, no return can ever be computed, and the whole apparatus is decorative.
Days later, reconciliation comes back for it. The job selects rows roughly 1, 5 or 20 days old, with a twelve-hour tolerance either side, that carry an entry price and do not yet have a return recorded for that horizon. The return is the simplest possible expression — current price minus entry price, over entry price — and the null check on the return column is what makes the job idempotent: a row already filled is never selected twice.
Then the chain runs, and the order it runs in is the architecture. Reconciliation must precede the tournament; the tournament must precede calibration; calibration must precede the weight update. Each depends on what the one before it wrote.
Elo asks which strategy was better. The tournament groups reconciled predictions by day and symbol, then plays every unique cross-strategy pair inside each group. The update is textbook: expected score from the rating difference over 400, then the rating moves by K times the gap between actual and expected. K is 32 for strategies with fewer than thirty comparisons and 16 after that, so new entrants move fast and established ones settle. Ratings start at 1500. Winner rules run in order — correct direction beats incorrect; if both were right or both wrong, the smaller magnitude error wins; and errors within a tenth of a percent are a draw.
Calibration asks a harder question: was the stated confidence honest? Five bins span 50 to 100 percent stated confidence. Within each bin the tracker forms a Beta posterior over the realised hit rate using a prior worth ten virtual observations, centred on the bin's own stated confidence — so the tracker starts by assuming the strategy is perfectly calibrated and demands real evidence to move it. Two numbers come out: a Brier score, rescaled so a coin flip maps to zero and perfection to a hundred, and an adjustment factor — the count-weighted ratio of realised accuracy to stated confidence, hard-clamped to between 0.5 and 1.5.
That adjustment factor is not a dashboard ornament. It is multiplied into the compositor's source weights, and it is applied to confidence before position sizing — and for any strategy with fewer than ten reconciled predictions, confidence is simply halved toward 0.5 regardless of history. Being overconfident costs you weight and it costs you size.
Most AI trading tools show you a backtest that looks amazing. The interesting engineering is in the machinery that stops a backtest from lying to you in the first place.
Start with the subtlest way to cheat. A daily bar for a Tuesday describes Tuesday — but you could not have acted on it during Tuesday's session, because it did not exist until the close. Any backtest that treats a bar's own date as the moment it became usable will systematically buy at prices nobody could have seen, and will report returns nobody could have earned.
The countermeasure is two timestamps per row instead of one. One records what period the row describes; available_at records when the row became knowable. A point-in-time query filters on the second, never the first. FutureMarket carries both on three tables and routes reads through three query helpers that apply the filter.
Then the backtester itself. It slices the reconciled prediction history into overlapping windows — 90 days of train, 30 days of test, stepping forward 30 days at a time — and computes metrics on each test window only. With the default step equal to the test length, test windows tile without overlapping, so no realised return is ever counted twice. Nothing is fitted to the training slice; it is measured for its row count and otherwise left alone. That makes this honest out-of-sample scoring of already-logged predictions rather than a parameter search dressed up as validation, which is exactly the distinction that flatters most published backtests.
Four strategy identifiers are eligible. The model-debate and sentiment sources are deliberately excluded, because their calls cannot be replayed faithfully — and replaying them badly would be worse than not replaying them at all. A run needs at least ten reconciled predictions and a date range longer than one full window, or it returns an error rather than a flattering result.
FutureMarket starts on paper trading. It is designed to catch its own mistakes — and to prevent the tricks that make bad systems look good.
A strategy is only marked as graduated when every one of these holds across the walk-forward test windows. Any single failure and the answer is no.
Alongside this sits a broader readiness report covering paper-trading duration, realised Sharpe, how many strategies have shown an edge, and whether the breaker has been exercised. It is published as a report you can read — not a badge the system awards itself.
Free to try. No credit card. No funding required. Run it locally, connect Alpaca's free paper trading API, and let the system build its own record before you trust it with anything real.
Self-hosted. Your data stays on your machine. Requires Python 3.11+, Node 18+, a free Alpaca account for paper trading, and an inference key for the analysis pipeline.