Introducing FrontierFinance: A Challenging Benchmark for Measuring Frontier Intelligence of Finance Agents
Investors use Samaya's agent platform for challenging and open-ended work that spans their investment workflow: idea screening and discovery, company and market research, financial data collection and modeling, portfolio tracking, and catalyst monitoring. Measuring AI system performance across these tasks is both important and extremely challenging. A benchmark needs to be broad enough to cover the full range of use cases. It also has to judge long-form, open-ended answers the way an expert would, capturing the taste and judgment of a professional investor in deciding the completeness and accuracy of the answer.
At Samaya, we treat evaluation of finance AI systems as a first-class problem. In a 2025 blog post, we introduced Criteria-Eval, a first-of-its-kind, rubric-style evaluation framework, which serves as the cornerstone of our internal benchmark. Since then, our benchmark has evolved alongside frontier AI capabilities: our finance experts have created more than 4,500 fully annotated queries spanning a wide range of complex finance use cases.
Today, we are excited to release FrontierFinance, a challenging benchmark curated from those internal annotations, consisting of 220 queries and 11,543 rubrics, all expert-crafted and openly accessible. Compared to existing finance benchmarks that overly focus on financial data extraction tasks, FrontierFinance is more comprehensive and challenging, covering six high-stakes finance use cases that call for a wider range of frontier AI capabilities. It is also the largest open finance benchmark of its kind, large enough to compare real-world finance AI systems with statistical significance.
We benchmark leading AI systems on FrontierFinance and find that Samaya's in-house agent system is state-of-the-art at 50.8%, outperforming Claude Fable 5 at roughly 4x lower inference cost. Among frontier models, run using a standard finance harness, Claude Fable 5 performs best at 49.2%, followed by Claude Opus 4.8 at 45.0% and GPT-5.5 at 43.5%.
In this post, we demonstrate that FrontierFinance is a rich and difficult benchmark, show how we use it to benchmark existing finance AI agents, and present how we developed FrontierFinance. We make the dataset and its grading code available as public resources. We also plan to release a technical report soon. Read more about the benchmark and system comparison on the benchmark website.
We built FrontierFinance to cover the entire investor workflow, with each query belonging to one of six key use cases (Figure 1). This matters because a practically useful finance AI agent should not only extract and analyze financial data. It should support investors through the full arc of their work, from discovery and research to evaluation and monitoring, synthesizing both qualitative and quantitative data and performing causal analysis along the way. As Figure 2 shows, FrontierFinance balances these use cases whereas most existing benchmarks are primarily limited to queries on financial data extraction.
FrontierFinance is also the hardest among existing benchmarks. To measure this, we utilized a pairwise ranking method similar to Elo scoring, and modeled difficulty score for each query in the dataset. Specifically, we pooled all examples (both queries and rubrics or answers) from existing finance benchmarks into a single set, then asked an AI agent to compare them pairwise, judging which of two examples is more difficult to answer and fully satisfy. From these pairwise judgments, we fit Bradley-Terry scores that give each query a relative difficulty rating. As Figure 3 shows, FrontierFinance covers a broader range of difficulty and has a higher mean difficulty than existing benchmarks. This follows directly from its coverage of diverse use cases, which demand a higher level of agent capability.
Every rubric in FrontierFinance is supported by publicly available data and is attributed to a source category. As Figure 4 shows, high performing systems need to draw from a wide range of sources. SEC filings are the single most important source, but they account for only 39% of rubrics. The rest come from company-originated content such as transcripts and investor presentations, professional knowledge, market data, and more. The mix also shifts noticeably from one use case to another.
The FrontierFinance benchmark, for the first time, allows us to compare the financial intelligence of frontier models and AI systems on these challenging tasks.
Our experiments evaluate three distinct types of harness:.
For a fair comparison, every harness connects only to publicly available data, such as web content, public filings, and news, and none draw on private sources such as proprietary research. We report three metrics:
Unsurprisingly, we see a quality-cost tradeoff as we compare different model families and sizes on the web search harness and the Finance Agent v2 harness. Improved qualification rate comes at a steep cost.
Samaya's agent system challenges this tradeoff. Samaya's agent is the state-of-the-art system with a rubric qualification rate of 50.8%, outperforming even Fable 5 (on the Finance Agent v2 harness), while being roughly 4 times cheaper. Samaya's in-house models, harness optimizations and high-quality retrieval work together to push the pareto frontier outward.
In general, harness determines both quality and efficiency. Models running the Finance Agent v2 harness generally outperform the same models on the web search harness.
In addition, the best open-weights model are closing the gap with commercial alternatives at a fraction of the cost. Using the Finance Agent v2 harness, DeepSeek V4 Pro reaches a 40.5% qualification rate, near Claude Opus 4.8 at 45.0% and GPT-5.5 at 43.5%, while incurring only 26% of the cost.
Furthermore, the rich metadata in FrontierFinance lets us analyze these systems more closely. Figure 6 breaks down performance by use case and by rubric category:
Read more about our in-depth benchmarking results here.
We build FrontierFinance through a four-stage process, with every stage led by in-house finance domain experts.
Step 1: Query drafting. Finance experts draft realistic, challenging queries grounded in the everyday work of investors. The queries carry the natural comprehensiveness and ambiguity of real requests (e.g., occasionally using company names instead of ticker symbols), so they capture the full domain nuance that real-world finance AI systems encounter.
Step 2: Rubric authoring. For each query, finance experts write the rubrics that a comprehensive answer should qualify, following strict guidelines that we iterate on to reduce ambiguity. Each annotation then passes through a multi-step audit to maintain quality.
Step 3: Expert review and cleanup. Finance experts review the collected rubrics to cut any that are redundant or carry subjectivity.
Step 4: Rebalancing. We rebalance the resulting queries to achieve an even distribution across use cases, capabilities, and difficulty levels.
Steps 1 through 3 yield more than 4,500 queries with full rubric annotations. From these, Step 4 produces the balanced set of 220 queries we release as FrontierFinance. We reserve the rest of the queries for internal R&D and future dataset releases.
Here are useful resources to help you get started on using FrontierFinance:
We also plan to release a technical report with more details soon, so stay tuned!
Samaya's ML team members, including Yuhao Zhang, Ashwin Paranjape, Ozan Koyluoglu, Thejas Venkatesh, Richard Diehl Martinez, Vishank Bhatia and Arash Alidoust contribute to the benchmark creation, system evaluation and asset authoring efforts. We thank Christos Baziotis, Jack Hessel, Jack Santos Silva and Mingyi Yang for their contribution to early data collection process, and Suharsh Sivakumar, Kyle Chang and Bram Mulders for providing engineering and infrastructure support.
If you use FrontierFinance in your work, please cite the dataset as:
@article{zhang2026frontierfinance,
title = {FrontierFinance: A Challenging Benchmark for Measuring Frontier Intelligence of Finance Agents},
author = {Zhang, Yuhao and Koyluoglu, O. Ozan and Venkatesh, Thejas and Diehl Martinez, Richard and Bhatia, Vishank and Alidoust, Arash and Paranjape, Ashwin},
year = {2026},
url = {https://samaya.ai/blog/frontier-finance}
}
Experience how Samaya can supercharge financial workflows
Get a DemoWe're backed by Gen AI leaders
The Expert AI Agent Platform Defining
Modern Finance
© 2026 Samaya AI. All Rights Reserved.
"We’ve spoken to ~40 companies offering AI in finance solutions, and done pilots with 10-15. Yours is the only tool that analysts are actively asking for, and saying how useful they find it."
Top 5 Hedge Fund