The Loss Landscape high resolution collection pieces are created with real data from the training processes of artificial deep neural networks (AI) through a complex process composed of multiple stages and contain the highest resolution and quality representations in the world of the surfaces that express the performance of the learning processes of artificial neural networks. In these pieces you are looking at the fingerprints of the learning process of artificial machines, the shadows of spaces that have billions of dimensions, spaces navigated by neural networks that are looking for the right combination of their parameters. To create this piece I slice the very high dimensional space to capture a dimensionality reduced representation that preserves the essence of its key features, combining engineering, scientific and creative disciplines to produce, through a number of complex stages, the final piece in very high detail and resolution, in a process that requires a lot of time, computation, effort and creativity. Because of the difficulty of the creation of these pieces in high resolution and detail, there are very few available. Icarus uses real data and showcases the training process that connects two optima through a pathway generated with a bezier curve. To create ICARUS, 15 GPUs were used over more than 2 weeks to produce over 50 million loss values. The entire process end to end took over 4 weeks of work. As Wikipedia states, “In Greek mythology, Icarus is the son of the master craftsman Daedalus, the creator of the Labyrinth. Icarus and his father attempt to escape from Crete by means of wings that his father constructed from feathers and wax.”. We can think of the loss landscape as another labyrinth where our “escape” is to find a low enough valley, one of those optimas we are searching for. But this is no ordinary labyrinth, for ours is highly dimensional, and unlike in traditional labyrinths, in our loss landscape it is possible to find shortcuts that can connect some of those optima. So just as Icarus and his father make use of special wings to escape Crete, the creators of the paper combine simple curves (a bezier in this specific video) and their custom training process to escape the isolation between the optima, demonstrating that even though straight lines between the optima must cross hills of very high loss values, there are other pathways that connect them, through which training and test accuracy remain nearly constant. On top of the above, the morphology of the two connected optima in this video, also resembles a set of wings. These wings come to life in the strategies used by these modern “Icarus” like scientists as they find new ways to escape the isolation of the optima present in these kinds of loss landscapes. This piece shows the top view of the middle stages of a training process that connects two minima. Mode Connectivity. Optima of complex loss functions connected by simple curves over which training and test accuracy are nearly constant. Visualization data generated through a collaboration between Pavel Izmailov (@Pavel_Izmailov), Timur Garipov (@tim_garipov) and Javier Ideami (@ideami). Based on the NeurIPS 2018 paper by Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry Vetrov, Andrew Gordon Wilson: https://arxiv.org/abs/1802.10026 | Creative visualization and artwork produced by Javier Ideami. Real data, resnet-20 no-skip, cifar10, sgd-mom, bs=128, wd=3e-4, mom=0.9, bn, train mod, 90k pts. log scaled (orig loss nums).