LUMI AI guide

Warning

This project is still work in progress and changes are made constantly. For well tested examples have a look at the LUMI AI workshop material: https://github.com/Lumi-supercomputer/Getting_Started_with_AI_workshop

This guide is designed to assist users in migrating their machine learning applications from smaller-scale computing environments to LUMI. We will walk you through a detailed example of training an image classification model using PyTorch's Vision Transformer (VIT) on the ImageNet dataset.

All Python and bash scripts referenced in this guide are accessible in this GitHub repository. We start with a basic python script, visualtransformer.py, that could run on your local machine and modify it over the next chapters to run it efficiently on LUMI.

Even though this guide uses PyTorch, most of the covered topics are independent of the used machine learning framework. We therefore believe this guide is helpful for all new ML users on LUMI while also providing a concrete example that runs on LUMI.

Requirements

Before proceeding, please ensure you meet the following prerequisites:

• A basic understanding of machine learning concepts and Python programming. This guide will focus primarily on aspects specific to training models on LUMI.
• An active user account on LUMI and familiarity with its basic operations.
• If you wish to run the included examples, you need to be part of a project with GPU hours on LUMI.

Table of contents

The guide is structured into the following sections:

• QuickStart
• Setting up your own environment
• File formats for training data
• Data Storage Options
• Multi-GPU and Multi-Node Training
• Monitoring and Profiling jobs
• TensorBoard visualization
• MLflow visualization

Further reading

• LUMI Documentation
• LUMI software library, PyTorch
• LUMI software library, TensorFlow
• LUMI software library, Jax
• Workshop material - Moving your AI training jobs to LUMI