Reproducibility is a core worry. Solaiman noted that trusting an evaluation depends on whether its results can be reproduced, and on having sufficient access to the model itself, which is where open weights help. Comparing different benchmarks is its own problem: in her words, they are an evolving science where nobody knows how to meaningfully compare them to each other, and they become ineffective relatively quickly.

The scope of testing is also shifting. Evaluations increasingly cover the whole system around a model, including the harness, the tool a user employs to access the model. Solaiman said a weaker model with a better harness could easily outperform a frontier model, yet there are far fewer evaluations for harnesses than for models, and agentic evaluations remain hard and expensive.

The embedded evaluator idea has gained traction as labs respond to a series of safety incidents, and Hugging Face itself was the target of one such incident. But the approach has drawn criticism: Anthropic's decision to hire Accenture as its first evaluator prompted backlash from critics who questioned whether a consulting firm was qualified for the job.