One strip per model and task family. Each square is a difficulty level, easiest on the left; the darker it is, the more of that level's tasks the model solved. The figure at the end of the strip is the level where it stops solving half of them.
Select a column heading to sort the table. Select it again to reverse the order.
Pick a battery and up to four setups, then choose what to look at. Every bar is drawn to the largest value among the ones you picked. On B2 you can only pick setups that ran under the same limits.
This table ranks served setups (model, route, effort, agent, limits) on two families of sealed puzzles. It is not a ranking of bare models and not a forecast of open-ended work.
Select a column heading to sort a group. Sorting never mixes groups.
Filled marks show where the model solves at least half the tasks every time; hollow marks show where it solves at least half at least once. Circles sit at a single level; rounded bars ("between 3 and 4") span the range between two levels, and arrows ("7 or above") mean the model cleared the hardest level on the test. Thin lines show uncertainty; faded marks and grey labels are not yet settled.
Up is better, left is cheaper. Bars show the uncertainty in the score. Tokens are counts and are not equal in price across providers.
Solved tries out of completed tries at each level. Darker means a higher pass rate.