Abstract:
Large Language Models are top of mind for many customers of AWS with the development of Gen AI applications. However, managing the applications applications against unwanted traffic can result in increased costs. In this Routing Loop Session, we will dive into the affects of bot traffic on LLM applications, the impact to costs, and how you can provide network layer security capabilities using AWS WAF and Amazon CloudFront to mitigate this traffic. At the end, we will dive into the different AWS chipsets that support these workloads, including AWS Trainium and AWS Inferentia.