Polyomino Packing
Pack every piece, rotated or reflected as you like, into the smallest rectangle. Every packing without overlap is valid, and a fuller rectangle scores higher.
Version 1.1 · Leaderboard updated: October 18, 2026
FrontierCS: Evolving Challenges for Evolving Intelligence
179 open-ended algorithmic problems. Any valid answer is accepted and a better one always scores higher, so an agent can keep improving for as long as it runs.
frontier-cs · harbor trial
$ uv run frontier harbor trial algorithmic 0 \
-a codex -m gpt-5.5
submit #1scored
submit #2scored
submit #3scored
submit #4scored
submit #5scored
✓ best submission kept
Leaderboard
| Rank | Agent | Score | Reasoning |
|---|---|---|---|
| 1 |  Claude Opus 5.5 Claude Code | 56.52 | High |
| 2 |  GPT-6 Astra Codex CLI | 53.98 | Low |
| 3 | Claude Sonnet 5.5 Claude Code | 53.90 | Max |
| 4 | GPT-6 Sol Codex CLI | 52.38 | High |
| 5 |  Gemini 3.7 Flash Gemini CLI | 47.97 | Medium |
| 6 | Gemini 3.8 Flash Gemini CLI | 47.79 | Medium |
| Rank | Agent | Score | Avg steps | Avg tool calls | Avg tokens |
|---|---|---|---|---|---|
| 1 | GPT-5.6 Sol Codex | 76.4 | 57.8 | 51.8 | 1.93M |
| 2 | Claude Opus 4.8 Claude Code | 74.5 | 355.4 | 145.7 | 14.72M |
| 3 | GPT-5.5 Codex | 72.1 | 48.3 | 55.8 | 2.02M |
| 4 | Gemini 3.1 Pro Amplio | 68.9 | — | — | — |
| 5 |  Qwen3.7 Max Claude Code | 61.9 | 133.9 | 139.1 | 13.85M |
| 6 | Gemini 3.1 Pro Gemini CLI | 60.2 | 74.3 | 41.6 | 2.00M |
| 7 |  Kimi K2.6 | 46.9 | 67.2 | 70.6 | 6.79M |
| 8 | Claude Opus 4.7 Claude Code | 43.0 | 77.2 | 42.2 | 10.96M |
| Rank | Model | Score@1 | Avg@5 | Score@5 | Elo |
|---|---|---|---|---|---|
| Human experts | 86.99 | — | — | — | |
| 1 | Gemini 3.0 Pro | 33.12 | 34.58 | 56.09 | 1265 |
| 2 | GPT-5.2 Thinking | 32.40 | 33.11 | 47.19 | 1242 |
| 3 | GPT-5 Thinking | 23.10 | 22.58 | 39.73 | 1196 |
| 4 |  DeepSeek 3.2 | 24.83 | 23.89 | 41.44 | 1193 |
| 5 |  Grok 4 | 24.04 | 22.98 | 36.81 | 1174 |
| 6 | Gemini 2.5 Pro | 20.34 | 19.32 | 36.65 | 1167 |
| 7 | GPT-5.1 Thinking | 20.64 | 21.49 | 34.76 | 1164 |
Problems
World Map
Given which of N countries share a border, colour a K by K grid so that two colours touch exactly when their countries are neighbours. Any such map is valid, and a smaller map scores higher.
Permutation
A permutation is hidden. Each query is a sequence, and the only answer is how many positions match. Any strategy that finds the permutation is valid, and fewer queries score higher.
StatementAdapted from Mastermind with a linear number of queries
Questions
What makes a problem open-ended?
Its checker accepts any valid answer and then measures how good it is. No known answer is optimal, so the reference marks a strong answer and not the end of the problem.
How is a problem scored?
Each problem has a reference worth 60 points: the human reference where the problem has one, and otherwise the objective value that earned 60 under the earlier rule. Up to the reference, the score follows the problem's own curve, set so that the reference earns 60. Beyond it, the score is 60 plus the natural logarithm of how many times better the answer is, capped at 100, so each further point takes an answer e times better. A model's score is the mean over problems.
What changed in version 1.1?
In version 1.0, many problems gave full marks to an answer that matched the reference, so an answer that beat it scored no higher. Version 1.1 sets the reference at 60 and keeps rewarding answers beyond it. It removes 8 problems that are solved or no longer tell models apart (64, 140, 179, 180, 181, 184, 185 and 228), which leaves 180. It also fixes evaluator errors, removes test cases in problems 23 and 303 that every answer passes or that no answer can score on, and repairs an input of problem 82. Problem 61 now asks for the schedule itself instead of its value, so version 1.1 leaves it out as well, for 179 problems. The version 1.1 results are agent runs, each model in its lab's official harness.
The version 1.0 leaderboards are kept as they were. Their scores use the earlier rule and cannot be compared with version 1.1.
How do the version 1.0 leaderboards differ?
In a single attempt, a model writes one program and that program is scored. An agent works in a sandbox through Harbor for up to 5 hours, can submit many times for feedback, and keeps its best score.
How do I run a problem?
Clone the repository and install it with uv sync. Score a solution of your own:
frontier eval algorithmic 0 your_solution.cpp
Or let an agent work on the problem:
uv run frontier harbor trial algorithmic 0 -a codex -m gpt-5.5
The test cases are public.
Where are the research problems?
Research and repository-level tasks continue in FrontierCS 2. This page covers the algorithmic problems.