Paper
HivemindEval: Construction and Provisional Validation of an Open-Weight Verifier for UK/EU Compliance Analysis, with a Disclosed Rating-Collapse Failure Mode
Giovanni De Lillo, Hypereum Ltd
Current
Summary
We describe the construction, evaluation, and release of HivemindEval v2.1, an open-weight verifier that scores compliance findings for six UK and EU regulatory frameworks on six quality dimensions, producing structured JSON output with per-dimension reasoning. The model is a supervised fine-tune of Qwen3-8B, released under Apache-2.0 as merged bfloat16 weights, and runs on a single 24 GB GPU at approximately 40 tokens per second.

Limits
The report publishes the validation method in full: benchmark construction, contamination controls, gold policy, metrics, acceptance gates, results including losses, and the design of the human validation. It withholds the training pipeline.