Section: New Software and Platforms

rotor

Re-materializing Optimally with pyTORch

Keywords: Deep learning - Optimization - Python - GPU - Automatic differentiation

Functional Description: Allows to train very large convolutional networks on limited memory by optimally selecting which activations should be kept and which should be recomputed. This code is meant to replace the checkpoint.py utility available in pytorch, by providing more efficient rematerialization strategies. The algorithm is easier to tune: the only required parameter is the available memory, instead of the number of segments.

• Contact: Lionel Eyraud-Dubois