Announcing AA-3D: a benchmark for AI models building interactive 3D experiences

Detailed scenes win more votes, and stronger models build them

The most detailed fifth of scenes in each task has screenshots twice as large (168 KB against 84 KB) and a third as empty (22% against 60%). It wins 63% of its votes, against 31% for the least detailed fifth. The models’ Elo alone predicts 61% and 33%, so the detailed scenes come mostly from the stronger models. These data cannot show whether detail is part of what makes those models strong. Within one model, the effect is small: a scene one standard deviation more detailed than the model’s other scenes wins about 1.5 points more of its votes (1.3 to 1.8, depending on the detail measure). Of 86 scene measures, 11 pass correction: the detail measures, render resolution and antialiasing, GPU time per frame and the number of page elements. None moves votes by more than 4 points.

AA-3D Votes Won by Scene Detail

Detail combines four screenshot measures, defined in What Detail Measures. Within the same model and task, one standard deviation more detail adds 1.5 points (p = 0.001, q = 0.011). 1,008 scenes, excluding 23 that break on the voting page.