A few months ago, we added private data to the Open ASR Leaderboard: 11 splits from Appen and DataOceanAI covering scripted and conversational speech from five accents (blog). The community response was great, with many highlighting the importance of held-out sets for trustworthy evaluations. Since then, we added one more private split for Indian-accented English with Voice Arena (blog).
Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Articles by Eric Bezzam
Open TTS Leaderboard: Scalable Evaluation for Multilingual Text-to-Speech and Voice Cloning
The pace of open-source text-to-speech (TTS) model releases has been incredible. On the Hugging Face Hub (as of Sep 30, 2026) there are more than 8K TTS models available 🚀 Evaluation, however, hasn't kept pace: it remains fragmented and unstandardized. The gold standard is human preference scores such as MOS or MUSHRA (more on metrics).
The Open ASR Leaderboard Adds Its First Global South Language
Voice Arena and Hugging Face partner to launch open ASR evaluation for Hindi and Indian English Benchmarks decide what gets built. A model that scores well on the Open ASR Leaderboard gets adopted and iterated on, while capabilities the leaderboard does not measure tend not to improve. Much of the recent work on the leaderboard has gone into making the evaluation metrics more trustworthy: Held-out private splits.
Measuring benchmark optimization in speech recognition
Public voice AI benchmarks increasingly suggest that models are performing at human levels. Yet those scores don't always reflect how models work in the real-world. Since public benchmarks are open and widely used, models can also become optimized for the tests themselves. Their scores may improve because they have learned benchmark-specific patterns and not because they have become better at the underlying task.
Chromatix: a differentiable, GPU-accelerated wave-optics library
|
By Eric Bezzam, Amit Kohli, Siqi Yang, Amey Chaware, Cedric Allier, Changjia Cai, Geneva Anderberg, Magdalena Schneider, Rainer Heintzmann, Corey Simmerer, Seung Yun Han, Yewon Kim, Benedict Diederich, Chulmin Joo, Laura Waller, Nicholas J. Durr, Nicolas C. Pégard, Patrick La Riviere, Roarke Horstmeyer, Shwetadwip Chowdhury, Srinivas C. Turaga, Diptodip Deb, Gert-Jan Both, M. Hossein Eybposh, Fabrizio A. Rivera-Sanchez, Guanghan Meng, Jovan Tormes-Vaquerano, Sibi Chakravarthy Shanmugavel, Teja Maruvada
|
Nature
Verified
Abstract Many current microscopy methods incorporate computational modeling as an integral part of the imaging process, either to solve inverse problems or optimize the optical system design itself. These methods often depend on differentiable optics simulations, yet no standardized framework exists, forcing computational optics researchers to repeatedly and independently implement simulations with limited reusability and performance.
Adding Benchmaxxer Repellant to the Open ASR Leaderboard
"When a measure becomes a target, it ceases to be a good measure." (Goodhart’s Law) TLDR: Appen Inc. and DataoceanAI have provided high-quality English ASR datasets covering scripted and conversational speech over multiple accents. To prevent potential risks of benchmaxxing or test-set contamination, we will keep these datasets private for a high-quality measure of performance on multiple tasks.
A complete lensless imaging tutorial - hardware, software and algorithms
In this post, we go through all the steps to build and use a lensless camera, specifically the DiffuserCam proposed by Prof. Waller’s group at UC Berkeley. This tutorial is useful for educators, hobbyists, and researchers interested in exploring lensless imaging with off-the-shelf, cheap components (about $80 for a Raspberry Pi, SD card, and the HQ sensor).
Show More
loading
Actions
Is this you?
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.Get in touch with Eric
Contact Eric, search articles and posts on X, monitor coverage, and track replies from one place.
Learn more about Muck Rack