How Moneda Bench works

The six benchmarks

• Moneda AdviceBench: Whether a system avoids telling a specific person what to buy or sell, without refusing to explain finance.
• Moneda CiteBench: Whether the sources a system cites say what it claims.
• Moneda GroundBench: Whether a system invents market figures it cannot know.
• Moneda KnowledgeBench: Whether a model knows finance: valuation, fixed income, accounting, derivatives, portfolio theory and market mechanics, plus whether it can explain a concept to a beginner..
• Moneda MarketBench: Questions about the market as a whole rather than one company: breadth and participation, sector moves and fund flows, why an index moved, exchange hours and holidays, and event calendars and instruments across the US, Europe and Latin America..
• Moneda PortfolioBench: Decision quality on hypothetical portfolios: rebalancing, risk measures, position sizing and constraints; calibrated forecasts on dated events; and running a paper fund to a written mandate..

Two tracks, never mixed

Products are tested as you would use them: AI products with their own search and tools, live, in the same two-hour window on a trading day, on the same items, with default settings and no retries. Numa 1, Moneda's AI, appears only here, tested through its public interface like any other product.

Base models are tested on equal tools: every model runs inside one open reference agent with the same fixed tool kit, answering from a frozen, dated market snapshot that ships with the release. Anyone can rerun it, and adding a model is one line of configuration. A "no tools" column shows what the tools add. We never say which base model any product uses.

Public and private splits

Each benchmark publishes its questions and grader as a public split you can run with one command. About a third of the items are held back as a private split. Public and private scores are shown side by side: a system that does much better on the public half has probably seen it before.

Canary strings and contamination checks

Every published file carries a canary string so it can be filtered out of training data. Live benchmarks generate fresh questions on every run. Moneda KnowledgeBench, the one most likely to be memorised, also ships a twin file with some answer keys randomised: a model that scores well above the twin's ceiling has trained on the published file.

Grading

Code grades every item that has a fixed answer. Where a judgement is needed, an AI judge gives yes or no verdicts per criterion on answers stripped of anything that identifies the system. The judge is checked against people on a held-out set before the rubric is tuned, and two people blind-review 15% of verdicts, at least one from outside Moneda with finance credentials. If agreement falls below 0.7, that board is labelled experimental and not ranked.

One system beats another only when the difference holds up statistically. Live benchmarks run three times per system and report the spread.

Releases

Each release is dated and frozen, with a changelog. Items are never edited under a published score. Items that leak are retired and replaced in the next release. Questions are published under CC BY 4.0 and the harness under MIT.

How results stay fair

Numa 1, Moneda's AI, gets no special treatment:

• The same items, the same time window, the same grader and default settings as every other system.
• Every transcript is saved, and anyone can rerun the public splits.
• Every result is published, whichever system comes out ahead.
• If one system leads on every benchmark, an outside party reruns those results before they are published.

Moneda creates and maintains the site. The domain is neutral so that a research group can co-host it later.

The directory

The directory lists other finance-AI benchmarks too. We link to the originals, credit the authors and do not rehost their data beyond what their licence allows. Scores shown are either quoted from the authors' paper or official board, with the source and date, or were run by Moneda with the benchmark's own grader and rules. We do not copy other aggregators. Each entry has a last-checked date, and corrections are welcome.

What this site is not

Nothing here is investment advice. Moneda is not a regulated financial institution and does not manage portfolios. Moneda AdviceBench measures whether answers avoid personal buy or sell instructions; it does not certify compliance with any rule. Moneda PortfolioBench describes decisions on hypothetical portfolios in a test setting; it is not investment performance.