torch.compile Practice and Optimization in Different Scenarios - Yichen Yan, Alibaba Cloud
torch dynamo is the core compilation optimization component in pytorch 2.0. torch dynamo can optimize dynamic graphs with minor modifications, and fall back to the default implementation when it is not supported, which greatly reduces the difficulty of enabling compilation optimization.
However, in practice, due to the diversity of AI workloads, there is still a certain threshold for enabling torch dynamo. To address these issues, we have optimized torch.compile to make it easier for users of different models (LLM/others) and loads (inference/training) to benefit from compilation optimization.