Running State-of-Art Gen AI Models on-Device with NPU Acceleration - Felix Baum, Qualcomm
Running State-of-Art Gen AI Models on-Device with NPU Acceleration - Felix Baum, Qualcomm
Since the boom of generative AI, the industry is now moving towards on-device AI inferencing, as it is not only a trend but a necessity now in order to save costs, achieve the best inference performance, ultra-low latency at the lowest power possible. In this session we go over the new features added on the Qualcomm AI Stack and how it works with the public release of ExecuTorch 1.0. We will discuss how to run traditional workloads as well as GenAI use cases including the latest version of Llama on the Mobile device while using Qualcomm Hexagon NPU.
Since the boom of generative AI, the industry is now moving towards on-device AI inferencing, as it is not only a trend but a necessity now in order to save costs, achieve the best inference performance, ultra-low latency at the lowest power possible. In this session we go over the new features added on the Qualcomm AI Stack and how it works with the public release of ExecuTorch 1.0. We will discuss how to run traditional workloads as well as GenAI use cases including the latest version of Llama on the Mobile device while using Qualcomm Hexagon NPU.
PyTorch
Welcome to the official PyTorch YouTube Channel. Learn about the latest PyTorch tutorials, new, and more.
PyTorch is an open source machine learning framework that is used by both researchers and developers to build, train, and deploy ML systems that so...
torch.compile and Diffusers: A Hands-On Guide to Peak Performance - Sayak Paul, Hugging Face
PyTorch
torch compile and Diffusers - A Hands On Guide to Peak Performance - PyTorch Compiler Series
PyTorch
verl: An Open Source Large Scale LLM RL Framework for Agentic Tasks - Yuxuan Tong, Bytedance
PyTorch
Lightning Talk: Sparsifying Vision Transformers with Minimal Accuracy Loss - Jesse Cai, Meta
PyTorch