Published 7/31/2026 Choosing scaling metrics, tuning windows, and budgeting for cold starts on dedicated inference. With Dedicated Model Inference on the Together AI platform you can get your deployments to autoscale on metrics the inference engine actually understands, such as in-flight requests, TTFT, GPU utilization, token throughput. You can set replica bounds, pick a metric and target, and then tune two windows that control how eagerly it scales up and how patiently it scales down.
