Twelve models tried to beat the code that the SVG-EAR authors released for their ECCV 2026 sparse-attention method, and none passed. We show what each run tried, which of its 48 checks it missed, and by how much.
SVG-EAR [1] makes video generation faster. It computes part of attention exactly and estimates the rest. Its authors released their code, and in FrontierCS 2 that code is the bar to beat. Twelve models tried. None passed.
Speed was within reach. Seven runs beat the human reference on time on all twelve hidden clips. Image quality was not: no run matched the reference on every clip. GPT-6 Astra came closest. It measured its own error as it ran and added a correction the paper lacks. It still missed five of its 48 checks. Three of them fell on one Wan clip, where its PSNR landed 0.80 dB below the bar.
Below we explain the task and the method behind the reference. We then follow four runs and measure how far each one fell short.
FrontierCS 2 builds each task from a recent paper with released code. We remove the part that implements the paper’s main idea, and an AI agent writes it back. The agent may rebuild the paper or try anything else behind the same interfaces. The authors’ code stays as the human reference. The judge compares the agent’s final version with it on workloads the agent never sees.
Video diffusion models are slow, and most of their time goes into attention. The SVG-EAR task keeps two open video models, Wan 2.1 and HunyuanVideo. It removes the SVG-EAR attention layer from both, along with three helper routines they share. The agent must put fast attention back without hurting the videos.
The judge makes twelve hidden clips with the agent’s code: eight with Wan and four with HunyuanVideo. For each clip it records the generation time. It also compares the clip with the one that full attention makes from the same prompt and seed. Three image scores measure the match. A run passes only if every clip clears four bars.
| Measure | What it tells you | Bar on each clip |
|---|---|---|
| Time | Seconds to generate the clip | At least 4% faster than the reference |
| PSNR | Pixel match to full attention, higher is better | At most 0.25 dB below the reference |
| SSIM | Structural match to full attention, higher is better | At most 0.01 below the reference |
| LPIPS | Perceptual distance from full attention, lower is better | At most 0.005 above the reference |
Four measures on twelve clips make 48 checks, and a pass needs all of them. A run that is faster and more faithful on average still fails if it misses one check on one clip.
The leaderboard also ranks runs by a score with no bars. The score rewards image quality, and speed measured against full attention rather than against the reference. A clip that matches full attention exactly, at the speed of full attention, scores 25. The human reference scores 33.9. A run can outscore the reference without passing, and on this task nine did.
The human reference is the authors’ tuned code for a method from ECCV 2026, which improves on an earlier sparse-attention method. To see what the models were up against, we build SVG-EAR up from a single query.
A video diffusion model makes a clip from noise in 50 steps. Each step runs a transformer over the clip’s tokens, and a 720p clip of 81 frames has 75,600 of them. In attention, every token looks at every other token, and that dominates the cost [1]. With full attention, one 720p clip from Wan 2.2 takes 29 minutes on an H100 GPU. SVG-EAR brings it to 16 minutes [1].
Take one query token and four key tokens. The query gives each key a score: the dot product of the two vectors, over \(\sqrt d\), where \(d\) is the vector length. Each key also carries a value vector. The output averages the values, and each value counts with weight \(e\) to its score:
\[o = \frac{\sum_j e^{s_j}\, v_j}{\sum_j e^{s_j}}, \qquad s_j = \frac{q \cdot k_j}{\sqrt d}.\]In our example the four keys fall into two clusters. Cluster A holds two keys that both score 1. Cluster B holds two keys that score 0 and 2. The weights come to about 5.4 in A and 8.4 in B, 13.8 in all.
The cheapest move is to skip a cluster. Sparse VideoGen2 [2], the method SVG-EAR builds on, computes the clusters with the most weight and drops the others. Drop A here, and 5.4 of the query’s 13.8 is gone.
SVG-EAR keeps an estimate of every cluster it skips. It replaces each key of the cluster with the cluster’s average key, its centroid. A skipped cluster of \(n\) keys, whose centroid scores \(\bar s\), then gets the weight
\[n\, e^{\bar s},\]and its values are replaced by their average. The estimate needs one score per cluster, not one per key.
In the example both centroids score 1, so the method estimates both clusters at about 5.4. For A that is exact, since its two keys score the same. For B it falls short of the true 8.4 by about 3, as Figure 1 shows.
Now suppose there is room to compute one cluster in full. Ranked by estimated weight, A and B tie at 5.4. SVG-EAR ranks them by how wrong the estimate would be. A’s estimate has no error and B’s is off by 3. It computes B, estimates A, and the toy query comes out exact.
Real attention has many queries, and they form clusters too. Sorting the tokens by cluster splits attention into blocks, one for each pair of a query cluster and a key cluster. The task uses 300 query clusters for Wan, 400 for HunyuanVideo and 1,000 key clusters. Figure 2 draws a small case.
Checking every query against every key would cost as much as attention itself. SVG-EAR checks one probe per query cluster: the cluster’s average query. It scores the probe against each key of a block and adds up the gap between estimate and truth,
\[\sum_{j \in \text{block}} \big\| e^{\bar s}\, \bar v - e^{s_j}\, v_j \big\|^2,\]where \(s_j\) is the probe’s score for key \(j\), and \(\bar s\) and \(\bar v\) belong to the key’s cluster. The method divides this error by the number of token pairs in the block [1].
The size of the budget comes from Sparse VideoGen2. For each query cluster, it counts the token pairs in the blocks that hold 90% of the estimated weight. SVG-EAR spends those pairs on the blocks with the most error per pair and estimates the rest. On Wan 2.2, the paper reports a PSNR of 24.995 dB against 23.556 for Sparse VideoGen2, at a lower density [1].
Three things combine. The reference is the newer of two published methods, tuned by the people who designed it. The pass rule asks every one of the twelve clips to be 4% faster and to stay inside three narrow image bands. Speed and quality pull against each other: computing fewer blocks saves time and adds estimation error. A run that gains on one axis tends to give ground on the other.
No model passed. Table 1 lists every run by score.
| Run | Approach | Score | Checks passed | Speed checks passed |
|---|---|---|---|---|
| GPT-6 Astra | Sparse, with an error budget measured per head | 48.59 | 43 | 12 |
| GPT-6.1 Sol | Exact attention, sparse path switched off | 38.42 | 30 | 0 |
| Claude Opus 5.5 | Sparse, the paper’s error-aware choice of blocks | 37.42 | 32 | 12 |
| Kimi K3 | 36.86 | 26 | 12 | |
| DeepSeek V4.1 Flash | 36.01 | 32 | 12 | |
| Gemini 4 Argon | Sparse, no estimate of skipped blocks | 34.98 | 26 | 12 |
| Muse Spark 1.3 | 34.88 | 29 | 10 | |
| GLM 5.3 Flash | 34.44 | 29 | 12 | |
| Muse Wonder 1.0 | 34.25 | 27 | 9 | |
| Human reference | SVG-EAR, the authors’ code | 33.9 | ||
| Qwen 3.8 Max | 20.91 | 28 | 12 | |
| GLM 5.3 | 15.35 | 9 | 0 | |
| Step 5 Preview | 7.39 | 18 | 0 |
Table 1. Every leaderboard run on SVG-EAR, by score. A check is one measure on one clip, and a pass needs all 48. Speed checks count the clips at least 4% faster than with the human reference. The approach is given for the four runs we examine below.
GPT-6 Astra stands apart. It scored 48.59, the highest on the task, and passed 43 checks, more than any other run. GPT-6.1 Sol ranks second by score. It ran exact attention, so it lost every speed check, as its section explains. Seven runs beat the reference’s time on every clip. Apart from Astra and Sol, no run passed more than 8 of the 12 checks of any image score.
Only Astra is both faster and closer to full attention than the reference on average. Averages hide the pass rule, which works clip by clip. The grids below show all 48 checks of every run.
Fidelity, not speed, decided the task. No run passed all 36 image checks. Wan clip 3 was the hardest clip: across the twelve runs, 33 of its 36 image checks missed.
Astra kept the design of SVG-EAR: cluster the tokens, compute some blocks exactly and estimate the rest from centroids. It changed two things in how that design runs.
SVG-EAR sizes its budget with one rule for every head of every layer. Heads differ. Some attend to a few nearby tokens, and others spread over the whole clip. A share of estimated weight says little about how much error a head will make.
Astra measures that error instead. Every fourth denoising step, it takes 32 query tokens as probes, at equal spacing over the tokens in cluster order. It computes their exact attention over every key. It then rebuilds their output the sparse way under three caps, which let each query cluster compute 12%, 25% or 50% of the keys.
Each head runs at the lowest cap that passes a bound. The mean squared gap must stay under 0.0005 times the mean square of the exact output. A head that fails even at 50% runs dense attention until the next check. Figure 3 follows one check.
Inside its cap, a head ranks blocks by estimated weight, the older rule of Sparse VideoGen2. It keeps blocks until they hold 98% of that weight or fill the cap. The paper checks error per block, with a cluster’s average query, inside a budget fixed in advance. Astra checks error per head, with real queries, and lets the measurement set the budget.
Return to cluster B, and suppose the budget leaves no room to compute it. Its estimate, 5.4, falls short of the true 8.4. The shortfall has a fixed sign. A score is linear in the key, so the centroid’s score is the average of the scores. The curve \(e^x\) bends upward, so the average of \(e^{s_j}\) is never below \(e^{\bar s}\). The centroid guesses low, and more so when the scores spread wide.
A second-order expansion of each \(e^{s_j}\) around \(\bar s\) gives the size of the gap:
\[\frac{1}{n} \sum_j e^{s_j} \approx e^{\bar s} \Big( 1 + \frac{\sigma^2}{2} \Big) \approx e^{\bar s + \sigma^2/2},\]where \(\sigma^2\) is the variance of the query’s scores over the cluster’s keys. The last form puts the fix inside the score. A kernel that already scores each centroid adds one term, and nothing else changes.
In terms of the keys, \(\sigma^2 = q^\top \Sigma\, q / d\), where \(\Sigma\) is the covariance of the cluster’s keys. Every run in this section keeps one variance per key coordinate, and ignores how coordinates move together:
\[\sigma^2 \approx \frac{1}{d} \sum_{i=1}^{d} q_i^2 \operatorname{Var}(k_i).\]B’s two scores, 0 and 2, have variance 1. The corrected estimate is about 9.0, against the true 8.4. It overshoots here, but it lands far closer than 5.4.
SVG-EAR gives a skipped cluster the weight \(n\, e^{\bar s}\), with no variance term [1]. Astra adds the term to every block it estimates, at a strength it sets per head. On the probes it tries the term at 0, ½, 1 and 1½ times its full size. It keeps the strength with the smallest error.
Two runs off the leaderboard found the same term on their own. They come from a batch of exploration runs, which the leaderboard does not count. GPT-6.1 Sol’s exploration run adds the full term to every estimated score, capped at 2. Claude Opus 5.5’s exploration run uses the same form at half strength, and on HunyuanVideo only. All three runs keep the plain average of the values. Our results cannot separate what the term added to Astra’s image quality from what the measured budget added.
GPT-6 Astra, leaderboard run · svg/solver_extensions/adaptive.py, lines 67–74
group_scores = (sampled_queries
@ key_means.float().transpose(-2, -1)) / dim**.5
variance_scores = (sampled_queries.square()
@ key_vars.float().transpose(-2, -1)) / (2 * dim)
group_scores += col_sizes.clamp_min(1).float().log().unsqueeze(-2)
tail_candidates = []
for factor in (0., .5, 1., 1.5):
group_weights = (group_scores + factor * variance_scores
- maximum).exp()
group_weights.masked_fill_(col_sizes.unsqueeze(-2) == 0, 0.)
tail_candidates.append(group_weights)
GPT-6.1 Sol, exploration run off the leaderboard · svg/solver_extensions/clustered.py, lines 85–89
scores = tl.dot(queries, keys) * (dim ** -0.5)
if corrected:
variances = tl.load(
variance + (head_idx * key_clusters + key_idx[None, :]) * dim
+ channels[:, None],
(key_idx[None, :] < key_clusters) & (channels[:, None] < dim), 0)
scores += 0.5 * tl.minimum(
tl.dot(queries_squared, variances) / dim, 4.0)
In both, the new term is the squared query against the per-coordinate key variances, over \(2d\). Astra’s excerpt is the probe check, which tries four strengths for each head. Its kernel in sparse.py then applies the chosen strength. Sol’s excerpt sits in its kernel and applies one strength everywhere. Both excerpts are wrapped to fit the page; the code is otherwise as submitted.
Astra also cut costs outside the estimate. Its dense steps and dense heads run on FlashInfer’s dense kernel. It reclusters the tokens every other step, with 12 k-means rounds at the start and 1 per step. The reference runs 50 and 2. Its final patch changes 8 files, adding 675 lines and removing 5.
Astra’s two ideas arrived late, one at a time. Its first version, three minutes in, ran dense attention everywhere and scored 51.05 on the development suite of four clips. Its clustered sparse versions then scored lower, and four of them scored 0: the judge reported that its GPU worker failed on them. A fixed variance term was already in some of those versions.
The budget measured per head came in the ninth version, after almost four hours. It lifted the development score from 33.61 to 65.62. The tenth version let each head choose the strength of the term. It passed all 16 development checks and scored 70.85, and it is the final submission. On the twelve hidden clips, which the agent never sees, it scored 48.59 and missed five checks.
Astra beat the reference’s time on all twelve clips, at 728.6 seconds per clip on average against 891.0. No run was faster on average. It also beat the reference on all three image scores on average: PSNR 23.77 dB against 21.67, SSIM 0.7973 against 0.7665 and LPIPS 0.1864 against 0.215. On HunyuanVideo clip 4 its PSNR was 28.96 dB, against 24.76.
Its five misses fell on two Wan clips. On Wan clip 3 it missed all three image scores by a wide margin. Its PSNR there was 19.42 dB against the reference’s 20.47, 0.80 dB below the bar. On Wan clip 6 it missed SSIM by 0.0007 and LPIPS by 0.012. Its speed margin was uneven. It took 0.78 to 0.80 of the reference’s time on the Wan clips, but 0.94 to 0.95 on HunyuanVideo, close to the limit of 0.96. The run cost $77.24, the most of any run on this task.
The patch below is the final submission, the code the hidden clips scored.
Of the runs we examined, Claude Opus 5.5 stayed closest to the paper. It rebuilt the error-aware choice of blocks. It changed one thing. In the paper, the budget comes from the weight rule of Sparse VideoGen2. Opus sizes the budget by error too: it computes blocks until they cover a set share of the estimated error, not of the weight.
It passed all twelve speed checks, at 803.9 seconds per clip on average against 891.0. Its image quality landed level with the reference’s on average, with a mean PSNR of 21.53 dB against 21.67. It passed the SSIM check on 8 clips, PSNR on 7 and LPIPS on 5. It scored 37.42, third on the task.
It missed 16 checks, all on image quality, spread over eight clips. The widest miss was PSNR on Wan clip 2, 1.32 dB below the bar. The narrowest was LPIPS on HunyuanVideo clip 1, 0.0003 above it. Its exploration run, which added half the variance term on HunyuanVideo, passed 39 of 48 checks. Its nine misses there fell on Wan clips 1, 2, 3 and 8.
Gemini 4 Argon went back to Sparse VideoGen2. Its SVG-EAR processors only call the Sparse VideoGen2 processors they inherit from. Those compute the blocks with the most weight and drop the rest. The routine meant to merge in the estimate computes the chosen blocks and nothing more. This is the first move of the toy example, where A’s weight was lost. It submitted this one version, 18 minutes into the run, and never changed it.
It passed all twelve speed checks, at 833.0 seconds per clip on average. It missed 22 image checks. On HunyuanVideo it missed all twelve, and its PSNR fell 0.44 to 1.22 dB below the bar. Astra’s PSNR rose above the reference’s on every HunyuanVideo clip. On Wan, Gemini 4 Argon missed all three image scores on clips 2, 3 and 7, and LPIPS on clip 5. Two of those Wan misses were narrow: its PSNR fell 0.003 dB short on clip 3 and 0.024 dB on clip 2.
Dropping the estimate did not buy the most speed. Astra’s clips, estimates and all, took 728.6 seconds. Gemini 4 Argon’s run scored 34.98.
GPT-6.1 Sol’s leaderboard run ended with no sparse attention at all. Both of its processors return exact attention before any clustering. They call FlashInfer’s dense kernel and ask first for its FlashAttention 3 backend. The sparse path is still in the files, behind a switch set to dense.
Sol ran dense attention from its first version, three minutes in. Three versions that skipped work came later: strided sampling of the keys at one hour, a clustered sparse version at two hours and an error-ranked one at three and a half. They scored 5.72, 34.93 and 25.27 on the development suite. Its notes say the clustered version took far longer on one Wan clip and still lost fidelity on HunyuanVideo, and that dense attention should hold its time better across clips.
A dense version on FlashInfer, three hours in, scored 53.16, the highest of the run. Its message said it missed all four development speed checks. Sol saved it as its final submission, and later dense kernels did not replace it.
Exact attention cannot meet the speed bar here. Sol’s clips took 941.8 seconds on average, against 891.0, and 1.02 to 1.17 times the reference’s time clip by clip. It lost all twelve speed checks. Its image quality was the best of any leaderboard run: PSNR 25.38 dB, SSIM 0.808 and LPIPS 0.1721. The run scored 38.42, second on the task. The score measures speed against the full-attention run that made each target, not against the reference. A fast dense kernel earns speed credit on that measure.
The run also serves as a yardstick for the image bars. Exact attention missed six image checks: all three on Wan clip 3, SSIM and LPIPS on Wan clip 6, and SSIM on Wan clip 2. The first five are the checks Astra missed, by similar margins. On Wan clip 3, Sol’s PSNR fell 0.62 dB short of the bar and Astra’s 0.80 dB.
Sol’s exploration run did build sparse attention, with the variance term at full strength. It passed 42 of 48 checks, one fewer than Astra, and every speed check. Its clips took 796.5 seconds on average, at a mean PSNR of 24.69 dB. Its six misses fell on Wan clips 2, 3 and 6. On Wan clip 3 its PSNR fell 0.80 dB short, as Astra’s did. The task scored that run 49.28.
Five more runs scored above the reference, from 34.25 to 36.86: Kimi K3, DeepSeek V4.1 Flash, Muse Spark 1.3, GLM 5.3 Flash and Muse Wonder 1.0. All five were faster than the reference on average, and Kimi K3, DeepSeek V4.1 Flash and GLM 5.3 Flash beat its time on every clip. All five sit below the reference’s mean PSNR, and each missed between 16 and 22 checks.
Qwen 3.8 Max beat the reference’s time on every clip. Its mean PSNR, 18.69 dB, was the lowest of any run on the chart, and it scored 20.91. GLM 5.3 and Step 5 Preview lost every speed check and scored 15.35 and 7.39.
Table 2 sets out where each run we examined missed, and its largest PSNR miss.
| Run | Speed misses | Image misses | Clips with a miss | Largest PSNR miss |
|---|---|---|---|---|
| GPT-6 Astra | 0 | 5 | Wan 3, 6 | 0.80 dB on Wan clip 3 |
| GPT-6.1 Sol, exploration run | 0 | 6 | Wan 2, 3, 6 | 0.80 dB on Wan clip 3 |
| Claude Opus 5.5 | 0 | 16 | Wan 2, 3, 5, 6, 7, 8; HunyuanVideo 1, 2 | 1.32 dB on Wan clip 2 |
| Gemini 4 Argon | 0 | 22 | Wan 2, 3, 5, 7; HunyuanVideo 1, 2, 3, 4 | 1.22 dB on HunyuanVideo clip 4 |
| GPT-6.1 Sol | 12 | 6 | Every clip | 0.62 dB on Wan clip 3 |
Table 2. Distance to a pass for the runs examined in detail, including GPT-6.1 Sol's exploration run, which the leaderboard does not count. The last column gives how far the run's PSNR fell below the bar, 0.25 dB under the reference, on its worst clip.
Seven runs beat the reference’s time on every clip, but no run passed all 36 image checks. Astra passed 31 of them, and exact attention 30. Wan clip 3 stopped every run we examined: each missed its PSNR bar there, by 0.003 to 0.80 dB.
A recent, well-tuned method and a pass rule applied clip by clip held off every model on SVG-EAR. GPT-6 Astra cut the mean time by 18%, improved all three mean image scores and found a correction the paper lacks. It still missed five checks, and exact attention missed the same five.
BBQ, an NSDI 2024 circuit, picks the next packet a network card sends. Gemini 4 Argon and Claude Opus 5.5 each gave it a faster clock than its authors did, one with the same memory and one with registers in its place. We build the circuit up from a 16-priority example to show both.
QuarterPIR, a EUROCRYPT 2024 scheme, lets a client read a database without revealing what it reads. Muse Wonder 1.0 cut its bytes per lookup by 27%, with privacy unchanged. We build the scheme up from a 16-entry example to show how.