Crowdsourced AI benchmarks have serious flaws, some experts say

TechCrunch
By Kyle WiggersApr 22, 2025, 8:30 am148 pts
AI labs are increasingly relying on crowdsourced benchmarking platforms such as Chatbot Arena to probe the strengths and weaknesses of their latest models. But some experts say that there are serious problems with this approach from an ethical and academic perspective. Over the past few years, labs including…

Read Article Share
Share Article
- email
- x.com
- facebook
- pocket
- reddit
- tumblr
- linkedin
- pinterest

Trending Today on Tech News Tube

Commentary: Taiwan-US form high-tech strategic partnership to counter China's rise

Digitimes

Commentary: Taiwan-US form high-tech strategic partnership to counter China's rise 129

AI video startup, Higgsfield, founded by ex-Snap exec, lands $1.3B valuation

TechCrunch

AI video startup, Higgsfield, founded by ex-Snap exec, lands $1.3B valuation 123

Amazon’s New World: Aeternum MMO will shut down next year

The Verge

Amazon’s New World: Aeternum MMO will shut down next year 116

The AI lab revolving door spins ever faster

TechCrunch

The AI lab revolving door spins ever faster 114

Iran's Internet Shutdown Is Now One of the Longest Ever

Slashdot

Iran's Internet Shutdown Is Now One of the Longest Ever 111

Taiwan's semiconductor test solution sector set for rapid growth amid AI chip demand surge

Digitimes

Taiwan's semiconductor test solution sector set for rapid growth amid AI chip demand surge 109

Kathleen Kennedy steps down as Lucasfilm president, marking a new era for the Star Wars franchise

Engadget

Kathleen Kennedy steps down as Lucasfilm president, marking a new era for the Star Wars franchise 98

TSMC sees no signs of the AI boom slowing for at least two or three years

The Register

TSMC sees no signs of the AI boom slowing for at least two or three years 98

About Tech News Tube

Tech News Tube is a real time news feed of the latest technology news headlines.

Follow all of the top tech sites in one place, on the web or your mobile device.

Featured

German cops add Black Basta boss to EU…

German cops add Black Basta boss to EU most-wanted list

Grubhub says hackers stole company data…

Grubhub says hackers stole company data in recent breach - here's what we know

Code.org: Use AI In an Interview Without…

Code.org: Use AI In an Interview Without Our OK and You're Dead To Us

RondoDox botnet linked to large-scale…

RondoDox botnet linked to large-scale exploit of critical HPE OneView bug

US imposes 25% chip tariff under Section…

US imposes 25% chip tariff under Section 232, but Taiwan secures strategic…

US cuts Taiwan tariffs to 15%, ties…

US cuts Taiwan tariffs to 15%, ties relief to US chip factory expansion

Meta has discontinued its metaverse for…

Meta has discontinued its metaverse for work, too

OpenAI invests in brain-interface biz…

OpenAI invests in brain-interface biz co-founded by CEO Sam Altman

OpenAI's acquisition of Torch signals…

OpenAI's acquisition of Torch signals intensifying AI competition in healthcare…

Airbnb seemingly confirms Apple’s…

Airbnb seemingly confirms Apple’s self-driving car — and what it reveals…