BBQ, an NSDI 2024 circuit, picks the next packet a network card sends. Gemini 4 Argon and Claude Opus 5.5 each gave it a faster clock than its authors did, one with the same memory and one with registers in its place. We build the circuit up from a 16-priority example to show both.
Gemini 4 Argon and Claude Opus 5.5 each ran the BBQ task once in FrontierCS 2. Both gave a hardware priority queue a faster clock than its authors did. Both removed the same wait. Each level of the queue reads a counter from memory, and the clock must leave room for that read. Gemini 4 Argon kept every memory and made the choice that follows the read before the value lands. Its circuit handles up to 1.56 times the operations per second of the human reference, with about the same logic. Claude Opus 5.5 took the counters out of memory. Its circuit handles up to 1.84 times as many, with up to 16 times the logic. Of the twelve models on the leaderboard, these two and GPT-6.1 Sol passed.
FrontierCS 2 removes the core of a paper’s released code, and an agent writes it back. The authors’ code is the human reference. A run passes only if its final version beats it on every metric of every hidden workload.
BBQ is a priority queue built as a circuit, by Atre, Sadok and Sherry at NSDI 2024 [1]. A network card or a switch needs such a queue to schedule packets. Each packet gets a priority, and the queue must hand out the best one next, millions of times a second. On an FPGA, BBQ reports three times the packet rate of similar hardware queues [1].
The task keeps the circuit around the queue and removes the logic of its tree levels. That logic is a Python program that writes the circuit, and the agent writes it back. The judge checks every answer on nine workloads at five tree sizes. It then measures two speeds on each workload: operations per second, and the latency of the slowest operation. A pass needs both to beat the human reference on every workload.
Take 16 priorities, 0 to 15, with 0 the best. BBQ keeps them in a tree of 4-bit bitmaps, as Figure 1 shows. The bottom level has four rows of four bits, one bit per priority. A bit is 1 when a packet of that priority waits. The top level has a single row, and its bit i is 1 when row i below holds a 1.
Each bit also has a counter of the packets below it. In Figure 1 the queue holds two packets of priority 1, one of priority 2 and two of priority 9.
To remove the best packet, the circuit follows the first 1 at each level. At the top it finds bit 0, so it goes to row 0. In row 0 it finds bit 1: priority 1. It then subtracts one from each counter on its way.
A bit turns to 0 when its counter does. The first removal leaves one packet of priority 1, so no bit changes. The second takes the last one, and bit 1 of row 0 turns to 0. The third removal must then find priority 2, and it empties row 0. Each removal must know whether a count was 1. The next removal comes one cycle later and must see the result.
A circuit works in ticks of a clock. Between ticks, values wait in registers. In each tick, values leave the registers, pass through logic and must settle in the next registers. The clock period is the time of the slowest such stretch.
The task computes the period from the circuit with a fixed model. Synthesis cuts the logic into lookup tables, the basic gate of an FPGA. Each lookup table on a stretch costs 500 picoseconds, and every period carries a fixed 300 ps:
\[\text{period} = 300 + 500\,n \ \text{ps},\]where \(n\) is the number of lookup tables on the slowest stretch. A stretch that starts at a memory read costs 1,000 ps more, as Figure 2 shows.
The period sets the speed. A design finishes some number of operations per cycle, and its operations per second is that number over the period. Every design in this study finishes about 0.68 operations per cycle, so the period decides.
No design can go below 1,800 ps. The stage that reads the priority buckets is fixed, outside the agent’s part. It reads a memory and passes one lookup table: 300 + 1,000 + 500 = 1,800 ps.
The human reference keeps the top level in registers. Below it, the counters sit in memory blocks, and in a three-level tree the bottom bitmaps do too. A memory block packs thousands of values into little space. This is what lets BBQ reach 32K priorities on an FPGA [1].
A lower level takes three stages, shown in Figure 3. The first finds the first 1 of the row and asks memory for that bit’s counter. In the second, the counter arrives. The third adds or subtracts one, updates the bit and writes both back.
Deciding whether a bit clears costs little. Each counter carries one extra bit, which the paper calls the waterlevel bit [1]. It says whether the count is at least 2. A removal clears its bit when the waterlevel bit is 0, with no arithmetic on the count.
The wait is in reaching the counter. A memory read lands late in its cycle. By then, an operation just ahead may have changed the same counter. After the read, the level compares the counter with what the operations ahead wrote, and keeps the newest value.
That comparison sets the clock. We timed the human reference with the task’s own script and tools. On the three smallest trees, its slowest stretch starts at the counter memory. It passes three lookup tables: 300 + 1,000 + 3 × 500 = 2,800 ps. On the two largest, the search for the first 1 is slowest, at 3,300 and 3,800 ps.
Gemini 4 Argon kept every memory of the human reference and the three stages of each lower level. It changed what a level does while the read is in flight, as Figure 4 shows.
Which value is newest does not depend on the value itself. It depends on which operations ahead touched the same counter, and registers hold that. Gemini 4 Argon computes the choice from those registers in parallel with the read. When the counter lands, one lookup table picks it or a newer value. The stretch from memory drops from three lookup tables to one: 1,800 ps.
The search in the first stage has the same problem one stage earlier. It cannot yet know whether the operations ahead clear a bit in the row. Gemini 4 Argon searches three versions of the row at once. One is the row as stored. The other two are the row as the operations one and two ahead would leave it. It keeps the right result once those operations decide.
The top level got a parallel form. Its counters sit in registers, so every bit can read its own waterlevel bit at once. The new top row is the old row with the chosen bit cleared. Every bit whose count is at least 2 is then set back. The top level takes no stage of its own. Each operation needs one cycle fewer: 9 on two levels and 13 on three, against 10 and 14.
On trees of width 4 the clock reaches the floor of 1,800 ps. At widths 8 and 16 it is 2,300 and 2,800 ps. The search for the first 1 and the counter update set it there. The circuit spends about the logic of the human reference. At 4,096 priorities it uses 2,126 lookup tables and 2,295 registers, against 1,694 and 2,431. Every memory stays in use.
Claude Opus 5.5 removed the wait by removing the memory. Each bucket of a level gets three registers: its bit, its count, and a flag for a count of 1. The flag does the job of the waterlevel bit.
Registers can be read at any moment, so a level needs one stage, not three. In that stage the circuit finds the first 1 of the operation’s row. It updates every row of the level at once. A removal clears a bit when its flag is set, and the new flag says whether the count was 2.
One stage per level has a second effect. An operation leaves a level before the next one enters it. The next operation sees the newest state, so the comparison with the operations ahead disappears.
Claude Opus 5.5 kept this stage short. The stage records which bucket each operation hit, as one flag per bucket. A later stage turns these flags into the digit the next level needs, outside the loop between back-to-back operations.
The clock reaches the floor of 1,800 ps on all four trees of the development feedback. The hidden tests add width 16, which feedback never showed. For a bottom level of more than 1,024 buckets, Claude Opus 5.5 wrote a second form. It keeps the rows in a register file and reads one row per operation. At width 16 the clock is 2,300 ps.
Registers cost space. At 4,096 priorities the circuit uses 26,763 lookup tables and 48,454 registers, against 1,694 and 2,431 for the human reference. The state grows with the number of priorities. The patch handles at most 8,192 priorities, while the paper’s FPGA design holds 32K [1].
Both circuits answered every operation of all nine workloads correctly. Both beat the human reference on both speeds, on every workload. The chart below shows operations per second at the five tree sizes.
Claude Opus 5.5 handles 1.56 to 1.84 times the operations per second of the human reference. Gemini 4 Argon handles 1.22 to 1.56 times. On trees of width 4 the two tie, since both reach the floor. The gap opens at widths 8 and 16. The latency of the slowest operation falls by 36 to 53% for Claude Opus 5.5. It falls by 26 to 42% for Gemini 4 Argon. The task scores the runs 42.2 and 37.4, against 26.0 for the human reference.
What each design spends is the other half of the comparison. The chart below counts the logic of each at 4,096 priorities.
The task scores speed and correctness, not size. Within those rules, Claude Opus 5.5 bought its extra speed with logic. Gemini 4 Argon found its speed in the order of the work.
Claude Opus 5.5 scored 40.9 with its first submission, $7.28 into its run, and 43.3 with its second. Gemini 4 Argon started at 27.7 and reached 38.5 with its second submission. Both final scores sit a little below the best development scores, since the hidden tests add width 16.
The patches below are the final submissions, the code the hidden workloads scored.
GPT-6.1 Sol passed as well, with 36.7. At width 16 its circuit ran at 3,800 ps, the period of the human reference. It passed there on 1.007 times the operations per second of the human reference. GPT-6 Astra scored 40.4 but failed. It reached 1,800 ps on the four trees of the feedback. At width 16 its clock rose to 4,800 ps.
Muse Spark 1.3, Muse Wonder 1.0 and DeepSeek V4.1 Flash also moved the counters into registers. At width 16 their circuits ran at 5,300 to 8,300 ps, and all three failed. Registers alone did not make a fast clock. The short single stage did.
Kimi K3 kept the memories and ran at 4,300 ps. GLM 5.3, GLM 5.3 Flash, Qwen 3.8 Max and Step 5 Preview gave no correct circuit at width 16.
Twelve models tried to beat the code that the SVG-EAR authors released for their ECCV 2026 sparse-attention method, and none passed. We show what each run tried, which of its 48 checks it missed, and by how much.
QuarterPIR, a EUROCRYPT 2024 scheme, lets a client read a database without revealing what it reads. Muse Wonder 1.0 cut its bytes per lookup by 27%, with privacy unchanged. We build the scheme up from a 16-entry example to show how.