Making Flux Run Fast
This video showcases a series of PyTorch optimizations that bring Flux text-to-image generation inference time to under half a second. torch.compile / torch.export and quantization through the torchao library are crucial players to achieving these times. These optimizations generalize well and can be easily applied to other transformer-based models.
Speaker: Joel Schlosser
Joel is a PyTorch Core developer focusing on a variety of performance-based techniques, including torch.compile and torch.export.
PyTorch Compiler Series: In this video series, watch the PyTorch Compiler team share tips and tricks that help you get the max out of torch.compile, torch.export, and related technologies, while enjoying a glimpse into all the cool engineering work that goes on behind the scenes.