PyTorch split rewrite breaks vLLM QK-norm RoPE fusion
A functionalization change that turns multi-output splits into slices leaves vLLM’s pattern matcher with nothing to fuse under Inductor graph partition.
A PyTorch nightly change in how multi-output views are functionalized has broken vLLM’s fused Q/K RMSNorm and rotary embedding path when Inductor graph partitioning is enabled.
vLLM’s fusion pass looks for a split_with_sizes on the combined QKV tensor, then RMSNorm and RoPE on the Q and K pieces. After the change, that split is rewritten during functionalization into contiguous sibling slices instead. The expected pattern no longer appears, so the matcher finds no fusion candidates and the optimized path silently fails to apply.
The rewrite is intentional: PyTorch now regenerates each output of ops such as split, split_with_sizes, and unbind directly as a slice or select, rather than replaying the full multi-output op. That removes redundant fake-tensor and proxy work, but it also invalidates downstream graph matchers that hard-coded the old form.
Laith Sakka reported the regression against PyTorch and sketched three paths forward: a temporary compatibility flag in PyTorch, updating vLLM to treat contiguous slices as equivalent to the old split, or rolling the functionalization change back. Matching on slices is the durable fix if the new replay behavior stays.