Sunishchal Dev

Paskov, Patricia, Christopher Rodriguez, Sunishchal Dev, and Stephen Casper, Open-Weight AI Models Require Proportional Evaluation Approaches, RAND Corporation, PE-A4886-1, May 2026. As of May 4, 2026: https://www.rand.org/pubs/perspectives/PEA4886-1.html Paskov, Patricia, Christopher Rodriguez, Sunishchal Dev, and Stephen Casper, Open-Weight AI Models Require Proportional Evaluation Approaches. Santa Monica, CA: RAND Corporation, 2026. https://www.rand.org/pubs/perspectives/PEA4886-1.html.

Key Findings The single-rubric method consistently provided the largest and most statistically significant reduction in normalized mean absolute error (9 to 25 percentage points), outperforming more-complex autograding pipelines. In several cases, the single-rubric method matched or exceeded the accuracy of nonexpert human graders but cost more than a thousand times less in time and money.

This tool was developed by RAND as a subcontractor for a Software Engineering Institute (SEI) project, part of Carnegie Mellon University (CMU). At RAND, this work was performed within the RAND National Security Research Division. This publication is part of the RAND tool series. RAND tools include models, databases, calculators, computer code, GIS mapping tools, practitioner guidelines, web applications, and various other toolkits and applied research products.