Tech talk

A quick guide to LLM inference

Spotlight: Pi School of AI Alumni Success Stories

Online Tech Talk with Ivan Gentile

Abstract

This session provides a quick guide to essential tools for training and accelerating the inference of open-weight large language models (LLMs). We will cover an overview of open-weight LLMs, the Pytorch Profiler for performance optimisation, and techniques such as torch, compile, and quantisation to improve model efficiency. Attendees will gain practical insights into how to optimise their models using these tools. The session will include a Q&A session for further discussion. It is ideal for data scientists, machine learning engineers, and AI enthusiasts.

Bio

Ivan Gentile is a data scientist at the non-profit foundation IFAB in Bologna. He specialises in machine learning on HPC infrastructures and various aspects of LLM. With a background in physics, mathematics and complexity science, Ivan’s passion for philosophy led him to explore the vast depths of the IT world. Ivan was a Fellow of the Pi School of AI Session 13.

Are you interested in more Tech Talks? You can find them here.

Follow us on Pi School’s social media channels and keep updated on upcoming events.