The Salt - Curated AI

The Weekly Salt #128 This week, we review: Schrödinger’s Code Repository: Have LLMs Learned SWE-bench or Memorized It? Calibration as a First-Class Criterion in LLM Evaluation Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs Harness-Zero: Harness Distillation via Agent-as-Harness The Salt - Curated AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

This week, we review: Unlocking Lossless Speedups in LLMs via Discrete Diffusion Don’t Drop Dropout: Optimizing Layer Sparsity for Efficient LLM Training and Inference Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning Rethinking On-Policy Distillation of Large Language Models II: One Training Example Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM There is a nice theme across these papers: several expensive pieces of...

This week, we review: Let’s Scale Step by Step: Compute-Efficient Hyperparameter Transfer for Large-Scale Mixture-of-Experts SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science?

This week, we review: RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction Know When to Stop: Segment-Level Credit Assignment for Reducing Overthinking When Attention Goes Blind: Numerical Failure in ALiBi Positional Encodings The Salt - Curated AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

This week, we review: LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks Distilled Reinforcement Learning for LLM Post-training DeepLoop: Depth Scaling for Looped Transformers The Salt - Curated AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Google’s Gemma 4 technical report is surprisingly brief. Compared with the documentation released alongside many competing open-weight models, it leaves out much of the information that researchers and practitioners would normally expect, particularly about pre-training, data composition, optimization, and post-training. That makes the report disappointing as a source of training transparency.

This week, we review: LLM-as-a-Verifier: A General-Purpose Verification Framework Morphing into Hybrid Attention Models AI translation of literary texts is “fine”, but readers still prefer human translations Google also finally published the Gemma 4 Technical Report. I’ll publish a complete review next week! The Salt - Curated AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

This week, we review: Rethinking the Role of Efficient Attention in Hybrid Architectures Variable-Width Transformers The Hidden Power of Scaling Factor in LoRA Optimization The Salt - Curated AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

This week, we review: ⭐How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMs Dynamic Linear Attention Compress-Distill: Reasoning Trace Compression for Efficient Knowledge Distillation The Salt - Curated AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. ⭐How Does Reasoning Flow?

The Weekly Salt #119 This week, we review: ⭐Share More, Search Less: Collaborative Parallel Thinking for Efficient Test-Time Scaling Less is More: Early Stopping Rollout for On-Policy Distillation Negligible in Size, Significant in Effect: On Scale Vectors in Large Language Models The Salt - Curated AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.