Can we trust LLMs to judge whether a research idea is novel?
Old Ideas, Novel Problems tests how reliably LLMs judge the novelty of research ideas, finding that small changes to the evaluation setup can produce very different verdicts.
Sources
Related
- Sources: Fujifilm plans to increase magnetic tape production capacity fivefold to meet AI-driven storage demand, its biggest production expansion in ~20 years (Nikkei Asia) OCT 28, 2026
- Anthropic's S-1 implies a $40T+ annual TAM based on the economic value of knowledge work in 2025, topping SpaceX's $28.5T estimate, without explicitly naming it (Wall Street Journal) OCT 28, 2026
- Attackers compromised 42 GitHub repositories by using files too large to be searched OCT 27, 2026
- Attackers built a ‘Skynet’ AI agent that hijacks exposed models to run cyberattacks OCT 27, 2026
- AMD says it will invest $1B in Meta's equity offering, its first Helios rackscale systems are installed at Meta, and its EPYC CPUs power Meta's Muse AI agent (Kai Nicol-Schwarz/CNBC) OCT 27, 2026