Brief

AWS has introduced a new framework for building scalable, serverless multi-agent generative AI systems. This solution integrates LangGraph Agents with Amazon Bedrock AgentCore's memory and observability features. The system leverages AWS Lambda and Step Functions for automatic scaling and efficient state management, enabling complex agent workflows with robust control and cost management. AI

A developer built an LLM router to optimize API costs by classifying prompt complexity and directing requests to the most cost-effective model. This system uses Pydantic AI and Claude 3.5 Haiku for classification, LiteLLM for routing, and tracks costs in real-time. The solution achieved a 62% cost reduction, saving $2,602 per month, while maintaining 99.2% quality, though it introduces a slight latency overhead. AI

A security audit of 35 Model Context Protocol (MCP) servers revealed widespread vulnerabilities, with 62% exhibiting issues. The most common problem was path traversal, allowing unauthorized file access, exacerbated by AI agents' potential manipulation through prompt injection. Other critical findings included shell metacharacters in configurations leading to remote code execution, exposed API keys in public repositories, and unpinned package dependencies that pose supply chain risks. AI

A software development team tested their LLM-based flake detection system by simulating an infrastructure failure, specifically by disabling an entire AWS Availability Zone. The initial test revealed a critical flaw: the flake detector, which relied on a single OpenAI endpoint, became unresponsive when the zone went down. To address this, the team integrated Bifrost, an AI gateway, as a sidecar to their agents, enabling failover to different providers and keys, and successfully mitigating the outage during a subsequent test. AI

This article details how to build a cross-cloud Retrieval-Augmented Generation (RAG) workflow using ChromaDB, a vector database, across Azure and AWS. It focuses on enhancing Large Language Model (LLM) capabilities by integrating external data sources. The guide aims to provide practical steps for developers looking to implement such a system in a multi-cloud environment. AI

A significant cost issue has emerged for teams using LLM tracing, primarily due to the large storage requirements of prompts and responses. Storing full LLM trace payloads without a retention policy can drastically increase AWS S3 bills. The article proposes three solutions: sampling successful traces while retaining all errors, implementing tiered storage with lifecycle policies for older data, and optimizing the data stored by focusing on critical information. AI

Modal has developed a system to achieve truly serverless GPUs for AI inference, addressing the challenge of rapidly scaling resources to meet variable demand. Their approach involves maintaining cloud buffers of idle GPUs, a custom filesystem for lazy container image serving, and efficient checkpoint/restore mechanisms for both CPU and GPU processes. This engineering effort, developed over five years, reduces AI inference replica scaling time from tens of minutes to mere seconds, aiming to maximize GPU Allocation Utilization. AI

A media company developed a serverless platform on AWS to automate the evaluation of AI-generated podcast summaries. The system sends articles to multiple foundation models simultaneously via AWS Bedrock, then uses a separate AI judge, Claude Haiku, to score each output based on criteria like accuracy and engagement. Finally, it generates an HTML report for visual comparison of the results, optimizing prompt refinement and parallel model invocation for efficiency. AI

A new approach allows running open-source LLMs like Llama 3 directly within AWS Lambda containers, bypassing traditional API providers for specific tasks. This method leverages model quantization and increased Lambda container limits to enable self-hosting of LLMs on serverless CPUs. While not universally cheaper than managed APIs, it offers significant cost savings and enhanced privacy for high-volume, low-reasoning workloads. AI

Amazon SageMaker AI now offers OpenAI-compatible API support for its real-time inference endpoints. This integration allows users to invoke models hosted on SageMaker using existing OpenAI SDKs, LangChain, or Strands Agents by simply updating the endpoint URL. The new feature supports bearer token authentication for secure access and enables multi-model hosting and the deployment of fine-tuned open-source models without requiring code modifications. AI

A developer has created a skill to improve AI-generated AWS architecture diagrams, addressing issues like manual cleanup, inconsistent styling, and overlapping elements. This skill, a markdown file with specific rules and reference data, enhances the output of AI tools like Claude Code and Kiro CLI by enforcing consistent layouts, icon usage, and edge routing. After five rounds of refinement, the skill ensures diagrams are professional-looking and usable for client presentations with minimal post-generation editing. AI

AWS has introduced a new integration that connects its Quick suite with AWS services via Bedrock AgentCore Runtime. This allows users to interact with AWS services using natural language, translating queries into AWS CLI commands without manual intervention. The system leverages Amazon Cognito for authentication and IAM for secure command execution, providing audit trails through CloudWatch Logs. AI

Amazon Nova Act, an AWS service for building and managing AI agents, has achieved HIPAA eligibility. This allows healthcare organizations to automate workflows involving protected health information (ePHI) through browser-based AI agents. The service can handle tasks like claims processing, referral coordination, and appointment scheduling, reducing administrative burden and improving efficiency while adhering to compliance requirements. AI

DeepL is now processing user data in the United States via Amazon Web Services. This change means that data entered into DeepL's services will be transferred to and handled within the US. The move is facilitated by DeepL's partnership with AWS. AI

AWS has introduced Amazon Bedrock AgentCore, a managed service designed to simplify the creation and deployment of multi-tenant AI agentic applications. This platform addresses key SaaS architectural challenges such as tenant isolation, data security, and cost attribution. By utilizing session-isolated microVMs, AgentCore offers robust security and operational efficiency for various use cases, including business intelligence, recruitment assistance, and dashboard automation. AI

AWS has developed an AI agent system to optimize radiology workflows, addressing inefficiencies in traditional worklist systems. These AI agents consider factors like radiologist specialization, workload, and fatigue to assign cases more effectively. This approach aims to reduce diagnostic delays and associated costs, with Radiology Partners collaborating on its adoption. AI

AWS has launched its Model Context Protocol (MCP) server, providing AI coding agents with a secure and auditable method to interact with AWS services. This managed server allows agents to access APIs, documentation, and operational workflows via a standardized interface, avoiding the need to expose broad credentials. AI

This article details the architecture of a serverless LLM classification pipeline built on AWS. It focuses on the practical steps and considerations for scaling such a system, emphasizing the ease of using LLMs for tasks like sentiment analysis. AI

Gu Fan, a former AWS Greater China executive, is joining Visa as its Vice President of Technology Ecosystem for Greater China. Fan previously held a senior role at AWS, leading strategic client accounts and reporting to Zhang Wenyi. Zhang, also an AWS alumna, now leads Visa's Greater China operations and will report to Fan. AI

Scuderia Ferrari is partnering with IBM to leverage artificial intelligence for enhanced Formula 1 fan engagement. The collaboration focuses on transforming vast amounts of race data into personalized content, aiming to make each fan feel uniquely connected to the team. This initiative includes AI-generated race summaries, interactive games, and an AI companion within the Ferrari fan app, which has already seen a 62% increase in engagement during race weekends. AI

Innovative Solutions, an AWS Premier Partner, has redesigned its enterprise services delivery by adopting Fireworks AI as its primary inference layer. This strategic shift addresses escalating AI inference costs and delivery complexity, which were previously limiting profit margins and operational flexibility. By moving its DarcyIQ platform to Fireworks AI, the company achieved predictable economics and enabled a transition from linear service models to parallel, agent-driven execution. AI

The AI Report is launching "The AI Executives Pass," a curated bundle of AI tools, partner perks, and resources valued at over $14,000. This pass aims to provide a practical AI stack for business leaders, founders, and teams, helping them cut costs on individual subscriptions and discover genuinely useful tools. Founding access is available for $199 per year, with the price set to increase to $299 after one week. AI

Amazon SageMaker AI has introduced new features to streamline the deployment of generative AI models. The platform now offers optimized inference recommendations, leveraging NVIDIA AIPerf to reduce the weeks-long manual benchmarking process for developers. Additionally, AWS has launched G7e instances powered by NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs, providing increased memory and networking throughput for faster and more cost-effective inference of large language models. AI

Arcee AI has migrated its specialized small language models (SLMs) from AWS to Together Dedicated Endpoints, seeking improved cost, performance, and operational agility. The company focuses on training efficient models under 72 billion parameters for specific tasks like coding and general text generation. Arcee AI also developed Arcee Conductor, an inference routing system that directs queries to the most suitable model, including third-party options like GPT-4.1 and Claude 3.7 Sonnet, to optimize cost and performance. AI

Fig, a startup developing a tool to enhance terminal workflows with visual applications, successfully built its initial Minimum Viable Product (MVP) in just two weeks. The company leveraged the Repl.it development platform for its rapid deployment capabilities, version control integration, and multiplayer features. While Repl.it was instrumental in their early stages, Fig eventually transitioned to Heroku and AWS as they scaled and encountered platform limitations. AI