📅 Day 1 (27 Sept): Fundamentals of Large Language Models
9:00 AM - 12:00 PM: Lecture
During this lecture, you will develop a comprehensive understanding of the fundamental concepts underlying language models.
The session will provide a comprehensive overview of language modeling basics, as well as the evolutionary journey of approaches within this domain. We will explore the progression from rule-based systems to the revolutionary attention mechanism and the subsequent models built upon it, such as Transformers. By delving into the intricacies of these modern Transformers-based language models, you will discover the remarkable power they possess and gain insights into how they have become the state-of-the-art technology in the field of nlp.
12:00 PM - 1:00 PM: Lunch Break
Sandwich lunch and refreshments (tea, coffee, buiscuts, etc.) will be provided free of cost!
1:00 PM - 4:00 PM: Coding Tutorial: Introduction to Transformer Models
- During this hands-on tutorial, you will learn how to implement and utilise
Transformer
models for various NLP tasks. You will be guided through the process of setting up the required software environment, loading andfine-tuning
pretrainedTransformer
models, and applying them to solve specific research problems in social sciences. Practical exercises and coding examples will be provided.
- During this hands-on tutorial, you will learn how to implement and utilise
📅 Day 2 (28 Sept): Model Interpretation and Debiasing
9:00 AM - 12:00 PM: Lecture
- This lecture will focus on techniques for interpreting the outputs of language models. You will learn about methods such as
attention visualization
andprobing
to understand how the models make predictions. Additionally, the session will cover the topic ofmodel bias
, its impact on social science research, and strategies to detect and mitigate bias in language models.
- This lecture will focus on techniques for interpreting the outputs of language models. You will learn about methods such as
12:00 PM - 1:00 PM: Lunch Break
Sandwich lunch and refreshments (tea, coffee, buiscuts, etc.) will be provided free of cost!
1:00 PM - 4:00 PM: Coding Tutorial: Interpreting and Debiasing Language Models
- During this tutorial, you will have the opportunity to apply the concepts covered in the lecture to interpret and debias language models. You will gain hands-on experience with various interpretation techniques and learn how to evaluate and address biases in model predictions. We will provide you with practical exercises and examples.
📅 Day 3 (29 Sept): Recent Developments: Large Language Models and Alignment
9:00 AM - 12:00 PM: Lecture
- In this lecture, we will focus on recent advancements in language models, specifically the emergence of large-scale models and the intriguing concept of reinforcement learning from human feedback (
RLHF
). You will have the opportunity to explore the capabilities and limitations of these powerful language models. We will delve into the concept ofRLHF
, discussing how it plays a pivotal role in improving model performance and aligning them with human feedback. Furthermore, we will cover the generation ofInstructGPT
models, including the highly popularChatGPT
. Through detailed examples and case studies, you will gain insights into the creation process and understand how these models can be effectively utilized. By the end of this session, you will have a comprehensive understanding ofRLHF
, along with practical knowledge of building and leveragingInstructGPT
models likeChatGPT
.
- In this lecture, we will focus on recent advancements in language models, specifically the emergence of large-scale models and the intriguing concept of reinforcement learning from human feedback (
12:00 PM - 1:00 PM: Lunch Break
Sandwich lunch and refreshments (tea, coffee, buiscuts, etc.) will be provided free of cost!
1:00 PM - 4:00 PM: Coding Tutorial: Advanced Techniques with Large Language Models and Alignment
- During this tutorial, you will explore advanced techniques with large language models and
RLHF
. You will gain hands-on experience with fine-tuning large-scale models usingRLHF
methods. Practical examples and exercises will be provided to help you apply these techniques to your own research questions and datasets.
- During this tutorial, you will explore advanced techniques with large language models and
4:00 PM - 6 PM: Coding Challenge and Awards Ceremony🏆
- In this practical part of our programme, you’ll join forces with your peers to form a set of teams, pitted against a demanding research problem. Each team will try to solve the challenge aided by the skills you have learned during the workshop. Success in this competition isn’t simply about finding the correct solution or achieving the best score; it’s about pushing boundaries, thinking outside the box, and showcasing ingenious problem-solving skills! Exciting prizes await the teams that demonstrate exceptional creativity, innovation, and deliver the most effective solutions. This coding challenge will serve as a great platform to test your skills, solidify your networks, cooperate with peers, and dive deep into the thrilling world of language models!