NBER researchers testing 13 proprietary and open-weight LLMs on HouseholdBench, which unites 6 US household surveys and 32 prediction tasks, found the best model lowered error for numeric outcomes by 12.2%, and that a billion parameter open-weight model matched proprietary models after fine-tuning and aggregating 16 predictions per observation A fine-tuned 4 billion parameter open model matched proprietary LLMs at predicting household outcomes
NBER researchers Jin Huang, Diego Ferreras Garrucho, Yutong Xie, Walter M. Yuan, Qiaozhu Mei, Chen Lian, and Jonathon Hazell measured the performance of 13 proprietary and open-weight LLMs on HouseholdBench, which unites 6 U.S. household surveys and 32 prediction tasks. They found that a 4 billion parameter open-weight model matched proprietary models’ performance after fine-tuning and aggregating 16 predictions per observation.
Verified 2026-11-02
Track with InsiderWhy it matters
Small open models can catch up with costly proprietary ones on specialist tasks once tuned, which pressures pricing at the top of the model market.
More on Models
- AI models can now win gold at the International Mathematical Olympiad, yet Stanford's 2026 AI Index found they still fumble a task most children master early. Across ClockBench's 180 clock designs and 720 questions, the top model, GPT-5.4 High, read analog clocks correctly only 50.6% of the time, versus 90.1% for humans
- After briefly nearly closing in 2024, the performance gap between the best closed-weight and open-weight AI models widened again on the Arena leaderboard. As of March 2026, Stanford's 2026 AI Index finds the top closed model leads the top open model by 3.3%, up from a 0.5% lead in August 2024
- Stanford's 2026 AI Index tracks how AI models perform on GPQA Diamond, a set of PhD-level chemistry and physics questions designed to resist Google searches. Mean model accuracy on the benchmark reached 93% in 2025, compared with an expert human validator baseline of 81.2% -- AI models now beat PhD-level experts by 12 points on GPQA Diamond
FREE · NO CARD
Facts like this, six days a week
The Daily Receipts on weekdays, The Weekly Brief on Saturday — six sourced facts a day by email, two more than the free site shows. Every one traced to the filing it came from. Free.
Get the free daily briefFree · No card · Unsubscribe anytime
Join now and you start with 100 Receipts on The Receipts Ledger — your first step toward a free year of Insider or Pro. How it works →
See today's issue →