After Model Gaps Narrow, Arena Starts Funding Evaluation Methods

Arena.ai's Fall 2026 Academic Partnership Program has reached its final deadline. A reminder on X said proposals are due tonight at midnight. The program offers up to $50,000 per project and is aimed at tenure-track faculty at U.S. universities, meaning faculty on the tenure track. Research directions are limited to the scientific foundations of AI evaluation: evaluation and ranking methods, measurement and validity, learning from human preferences, and safety and alignment.

This tweet cites a September 23 reminder, which at the time stated the deadline was October 30, 2026. The final reminder now moves the time up to tonight at midnight. The original tweet is here.

Evaluation research is usually less visible than model releases. Once the gap between the top leaderboard entries narrows, ranking methods matter more. In the replies, Tanish Mehta said that the top models are very close, so how to rank them matters more. DARKKEVN Labs said measurement validity is especially timely: changes in leaderboards are only useful if you know what construct the evaluations are measuring; preference data, safety alignment, and capability benchmarks can easily contradict one another, and funding work on how they fit together would be valuable. kimiloo asked whether selected teams would get access to raw interaction logs; the tweet did not answer this.

For developers, the practical questions are how leaderboard scores are produced, how human preferences become signals, and how to interpret situations where safety alignment and capability benchmarks conflict. For general readers, it can be understood as funding the methods for writing exam questions and grading them. The closer the models get, the harder it becomes to fudge the test design and grading.