Data Collection
Expert
AI
Expert + AI
01
Expert
Expert-crafted queries
Finance experts draft realistic, challenging queries, guided by real-world financial workflows.
02
Expert
Expert rubric annotation
Finance experts author rubrics; supervisors verify.
03
Expert + AI
Expert-guided cleanup
Finance experts guide AI agents to clean up rubrics and remove subjectivity.
04
Expert + AI
Taxonomy-based rebalancing
Rebalance across use cases, capabilities, and difficulty.
Figure 7: Data collection process for FrontierFinance.