PyTorch proposal would cache shared pipeline activations per rank
A rank-local cache would serve reused residuals and return one summed gradient, replacing custom stage subclasses.
By tensorA rank-local cache would serve reused residuals and return one summed gradient, replacing custom stage subclasses.
By tensorA proposal would let memory-heavy pipeline stages park saved activations on spare peer GPU memory without changing the compute schedule.
By tensor