Biography
Simone Scardapane is Assistant Professor at Sapienza University of Rome, working in several subfields of deep learning, including graph neural networks and continual learning. He has a strong interest in promoting machine learning in Italy, having contributed to several no-profit activities, including Meetups and podcasts. He is action editor of Neural Networks and Cognitive Computation, member of the IEEE CIS Social Media Sub-Committee, the IEEE Task Force on Reservoir Computing, the “Machine learning in geodesy” joint study group of the International Association of Geodesy, and chair of the Statistical Pattern Recognition Techniques TC of the International Association for Pattern Recognition.
Talk
Automatic differentiation (autodiff) is at the hearth of the deep learning “magic”, and it is also powering advances in multiple fields ranging from visual rendering to quantum chemistry. In the first part of this practical tutorial, we show some fundamental ideas from the autodiff field, and how they are implemented in several common frameworks, including TensorFlow, PyTorch, and JAX. In the second part, we show instead how to implement a custom PyTorch-like autodiff library from scratch. We conclude with some trends and advanced tools from the autodiff world, e.g., stateless models in PyTorch.