Prajun Trital

Abstract High-quality openly-accessible machine learning (ML)-ready datasets play a foundational role in developing new artificial intelligence (AI) models or fine-tuning existing models for scientific applications such as weather and climate analysis. However, despite the growing development of new deep learning models for weather and climate, there is a scarcity of curated, pre-processed ML-ready datasets.

Before diving into Cross-Entropy, it’s important to first understand some foundational concepts, such as Huffman Coding and Shannon’s Information Theory. Press enter or click to view image in full size Huffman Coding A lossless data compression algorithm that assigns variable-length codes to characters Prefix-free coding (no code is a prefix of another). It does that by building a binary tree from bottom up (combining the lowest-probability symbols first).

In our previous article, we explored the differences between Parametric and Non-Parametric models. "Parametric models have a fixed number of parameters learned from training". This means we must: - Need a Loss Function: Since we're "learning" parameters, we need a way to measure what "good learning" looks like. How else would we know if our slope and intercept are optimal? - Must Choose Function Class: We're committing to a "fixed" structure (linear, quadratic, etc.), so we must decide this upfront.