Is this you? As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.
Claim your profile
Articles by Sid Shanker
Baseten on Hugging Face Inference Providers 🔥
We're thrilled to share that Baseten is now a supported Inference Provider on the Hugging Face Hub! Baseten joins our growing ecosystem, enhancing the breadth and capabilities of serverless inference directly on the Hub's model pages. Inference Providers are also seamlessly integrated into our client SDKs (for both JS and Python), making it super easy to use a wide variety of models with your preferred providers.
Rolling deployments for zero-downtime model updates
We heard from customers coming from other inference platforms that while updating their models, they were stuck choosing between blue-green deployments and hard cutovers. Blue-green deployments require a full parallel fleet alongside the existing one, effectively doubling compute spend for the duration of the rollout (and challenging to do during a capacity crunch). Hard cutovers are cheaper but all-or-nothing: if you run into issues, there's no pausing mid-rollout.
Create custom environments for deployments on Baseten
Developers want the ability to test production workflows and workloads without the consequences of failing in production.
Introducing canary deployments on Baseten
We’re excited to launch canary deployments, enabling you to gradually shift traffic to new model deployments over custom time windows! Rolling out new deployments is a fraught process, especially when it comes to complex generative AI models. From the model quality to performing at scale and interacting with other moving parts in your pipeline, there’s always a non-negligible risk that something could go wrong.
Introducing Baseten Chains
Today, we're excited to announce the beta release of Chains, a framework and SDK designed to simplify the creation and deployment of compound AI systems featuring multiple models and components. We’re committed to continuously evolving our platform to deliver the best performance, reliability, and efficiency for the sophisticated AI products our customers build. Chains represents a giant leap forward in that commitment, to enhancing the performance and efficiency of AI infrastructure.
CI/CD for AI model deployments
Continuous integration and continuous deployment (CI/CD) is an industry standard for everything from web to mobile to API development. But for AI models, deployment is often a manual process. What would CI/CD look like for AI models? Broadly, AI models need a pipeline that looks like any other CI/CD setup: A safe environment to deploy model updates without affecting production. A way to test newly deployed models before promoting to production.
Streaming real-time text to speech with XTTS V2
In this tutorial, we deploy a streaming endpoint for XTTS V2, a state of the art open source text to speech model with voice cloning capabilities. The streaming endpoint has a total round-trip time to first chunk of as little as 200 milliseconds and delivers near real-time audio playback for a given text. A text to speech model (or speech synthesis model) is a type of generative AI model that creates a natural, humanlike narration of a text input.
The best open source large language model
Large language models (LLMs) are the definitive category in generative AI. But with tens of thousands of options, it can be hard to feel confident about making the right tradeoffs between output quality, speed, and cost — especially when models specialize in different tasks.
Show More
loading
Actions
Is this you?
As a journalist, you can create a free Muck Rack account to customize your profile, list your contact preferences, and upload a portfolio of your best work.Get in touch with Sid
Contact Sid, search articles and posts on X, monitor coverage, and track replies from one place.
Learn more about Muck Rack