Autoscaling Your AI Agent Under Load // TRAIN BRAIN

Autoscaling Your AI Agent Under Load

This video demonstrates how to effectively autoscale your AI agent under heavy user load. We simulate a stress test on a decoupled architecture, combining a GPU-powered Gemma LLM with a lightweight ADK agent on Google Cloud Run. Discover how Cloud Run intelligently provisions resources to handle high demand, ensuring graceful scaling and cost efficiency by only scaling the bottleneck component.
Chapters:
0:00 - Introduction: The Challenge of Load
0:19 - Load Testing with Locust
1:31 - Observing Autoscaling in Cloud Run
2:02 - Key Learnings: Decoupling and Cost Efficiency
2:31 - Conclusion
Resources:
Codelab → http://goo.gle/475sUpV
GitHub Repository → http://goo.gle/3KJVc1Y
Google Cloud Run GPU → http://goo.gle/48sn3NV
ADK Documentation → http://goo.gle/3LauFL8
Subscribe to Google Cloud Tech → https://goo.gle/GoogleCloudTech
#GoogleCloud #LLM #Gemma #ADK #CloudRun
Speakers: Amit Maraj
Products Mentioned: Cloud Run, Gemma, AI Infrastructure, Cloud GPUs

Google Cloud Tech

Helping you build what's next with secure infrastructure, developer tools, APIs, data analytics and machine learning....

Connecting ADK Agents to MCP Servers

Use the Gemini CLI Jules and Observability extensions together

Introduction to Vertex AI Agent Engine

Power your AI agents with MCP tools on Google Cloud Run

Use the Gemini CLI Jules and security extensions to fix security vulnerabilities in the background

Use the Jules extension for Gemini CLI to fix multiple GitHub issues

Dataplex fundamentals: Aspects & glossaries

We tried to jailbreak our AI (and Model Armor stopped it)

Parallel bug fixing & unit testing with Jules and Observability extensions for Gemini CLI

How to fix security vulnerabilities with the Jules and security extensions for Gemini CLI

How to fix multiple GitHub issues at once using the Jules extension for Gemini CLI

The path to AI inferencing on GKE Part 1: Guided model research

Vibe coding with Google AI Studio | The Agent Factory

Is it possible to create a model agnostic prompt?

Building agentic RAG for e-commerce with ADK and Vector Search

Demo: Vibe coding a command line Markdown viewer with the Gemini CLI

Don't guess: How to benchmark your AI prompts

Identity and Access Management for Agents

ComfyUI on GKE for Genmedia solutions

Meet Cloud SQL: Google Cloud's fully managed and intelligent relational database service

Autoscaling Your AI Agent Under Load

Common Looker CI errors (and how to tackle them)

Multi-agent vs. single-agent: Which should you use?

Spanner: The always-on, virtually unlimited scale database

Building an AI tutor that ACTUALLY remembers you

Agent Sessions and Tool Authentication

How to build a multi-agent app with ADK and Gemini

The Agent Factory - Episode 11: AI agents for data engineering and data science

How AI agents can remember you forever

Connecting Your AI Agent to a Cloud-Hosted LLM

My AI agent crashed and how to save its memory

Gemini vs. the clock: 5 hours to build a FinTech agent

This month in GKE: September edition

Giving your AI agent a "scratchpad" memory

Secure ADK elements with Secret Manager

Validate your Looker workflow from data to dashboard

AI Agent Dance Off: Comparing Design Approaches

The Agent Factory - Episode 10: Agent Security

Why do AI agents forget everything?

Workflow agents and communication in ADK

Build serverless applications in the Cloud Run hackathon

Common Cloud Run errors and how to fix them with Cloud Logging

Deploying A GPU-Powered LLM on Cloud Run

Build an AI app that watches videos using Gemini

Large scale data ingestion for MongoDB using Application Integration

Salesforce bulk data ingestion using Application Integration

How to enable app management for Google Cloud folder

The Agent Factory - Episode 9: Agent evaluation with ADK & Vertex AI

Protect your LookML: Continuous Integration for reliable data

Secure AI: De-identifying data with SDP

Gemini in Google Threat Intelligence

Foundations of multi-agent systems with ADK

What is Looker CI?

Programmable data quality with Dataplex and generative AI

The Looker MCP server: Your data's universal translator for AI

Observability in action: A Google Cloud Next demo

This month in GKE: August edition

Effortless data prep in BigQuery: A low coder's guide

Brunch, Automated: How to Architect AI Agents for Everyday Tasks

Say hi to the new GKE: A better default compute for every workload

The Agent Factory - Episode 8: Agent payments, can you do my shopping?

How to connect Cloud Run to NoSQL Firestore

They built an AI agent in 3 hours: Virtual try on with skin tone detection

The Agent Factory - Episode 7: Gemini CLI with Taylor Mullen

Configure a Java Producer for Google Cloud’s managed Apache Kafka service (step-by-step)

How memory makes AI agents more effective

Build AI Agents on GKE in the GKE Turns 10 Hackathon

Visualizing BigQuery geospatial data in Colab

How to deploy a container image to Cloud Run

The Agent Factory - Episode 4: Remember me: Memory in Agents

Container-Optimized Compute Platform for GKE Autopilot

The Future of Automation: How AI Agents are Revolutionizing Data and Robotics

Enable Looker Studio report generation with one click!

GKE Inference Reference Architecture, Your Blueprint for Production-Ready Inference

This Month in GKE: July Edition

The Agent Factory - Episode 3: Building Custom Tools for Agents

The future of AI architecture: Using agents to supervise other agents

What is Private Google Access?

Defining AI agents: From definitions to stopping conditions

The AI Agent Bake-Off Ep.1 Trailer / Can They Build an AI Agent in 3 Hours?