Ehssan Khan

Small language models (SLMs) have gained significant traction recently thanks to their efficiency and strong performance compared to their larger counterparts. This is especially important in situations such as edge servers or offline agents, where deploying a large language model (LLM) on GPUs isn’t feasible. In this blog, I show you how to run responsive, CPU-only applications using a quantized SLM in the GPT-Generated Unified Format (GGUF).

Authors: Ehssan Khan, AI Software Solutions Engineer Ben Consolvo, AI Software Solutions Engineer With the launch of the C4 series, Google Cloud now offers access to Intel® Xeon® 6 processors with P-cores (code-named Granite Rapids) CPUs. This complements the C4D AMD EPYC 5 and the Arm-based C4A Google Axion offerings.

Authors: Shin Wei Lim, AI Software Engineer, Intel Jocelyn Teh, Machine Learning Engineer, Intel Ehssan Khan, AI Software Engineer, Intel Gary Koay, Software Engineering Manager, Intel Richard Chuang, Principal AI Engineer, Intel One of the most pressing challenges for media and entertainment today is navigating between the explosion of data and the need to deliver personalized, monetizable content experiences at scale and in real time.