Parnab Basak

At AWS re:Invent 2025, which took place last December in Las Vegas, Amazon Web Services (AWS) unveiled an array of cutting-edge innovations during a week of keynotes, innovation talks, breakout sessions, product launches, interactive demos, and networking opportunities. For stakeholders in the public sector, the conference offered a whole host of important announcements, compelling themes, and inspiring customer stories.

Large language model (LLM) inference can quickly become expensive and slow, especially when serving the same or similar requests repeatedly. As more applications incorporate Artificial Intelligence (AI), organizations often face mounting costs from redundant computations and frustrated users waiting for responses. Smart caching strategies offer a powerful solution by storing and reusing previous results, dramatically reducing both response times and invocation overhead.

In Part 1 of this series, we discussed three patterns and general best practices for building real-time, interactive, generative AI applications. However, not all generative AI workflows require immediate responses. This post explores two complementary approaches for non-real-time scenarios: buffered asynchronous processing for time-intensive individual requests, and batch processing for scheduled or event-driven workflows.