Programme

📅 Day 1 (27 Sept): Fundamentals of Large Language Models

9:00 AM - 12:00 PM: Lecture
- During this lecture, you will develop a comprehensive understanding of the fundamental concepts underlying language models.
  The session will provide a comprehensive overview of language modeling basics, as well as the evolutionary journey of approaches within this domain. We will explore the progression from rule-based systems to the revolutionary attention mechanism and the subsequent models built upon it, such as Transformers. By delving into the intricacies of these modern Transformers-based language models, you will discover the remarkable power they possess and gain insights into how they have become the state-of-the-art technology in the field of nlp.
12:00 PM - 1:00 PM: Lunch Break

Sandwich lunch and refreshments (tea, coffee, buiscuts, etc.) will be provided free of cost!

1:00 PM - 4:00 PM: Coding Tutorial: Introduction to Transformer Models
- During this hands-on tutorial, you will learn how to implement and utilise Transformer models for various NLP tasks. You will be guided through the process of setting up the required software environment, loading and fine-tuning pretrained Transformer models, and applying them to solve specific research problems in social sciences. Practical exercises and coding examples will be provided.

9:00 AM - 12:00 PM: Lecture
- This lecture will focus on techniques for interpreting the outputs of language models. You will learn about methods such as attention visualization and probing to understand how the models make predictions. Additionally, the session will cover the topic of model bias, its impact on social science research, and strategies to detect and mitigate bias in language models.
12:00 PM - 1:00 PM: Lunch Break

Sandwich lunch and refreshments (tea, coffee, buiscuts, etc.) will be provided free of cost!

1:00 PM - 4:00 PM: Coding Tutorial: Interpreting and Debiasing Language Models
- During this tutorial, you will have the opportunity to apply the concepts covered in the lecture to interpret and debias language models. You will gain hands-on experience with various interpretation techniques and learn how to evaluate and address biases in model predictions. We will provide you with practical exercises and examples.

9:00 AM - 12:00 PM: Lecture
- In this lecture, we will focus on recent advancements in language models, specifically the emergence of large-scale models and the intriguing concept of reinforcement learning from human feedback (RLHF). You will have the opportunity to explore the capabilities and limitations of these powerful language models. We will delve into the concept of RLHF, discussing how it plays a pivotal role in improving model performance and aligning them with human feedback. Furthermore, we will cover the generation of InstructGPT models, including the highly popular ChatGPT. Through detailed examples and case studies, you will gain insights into the creation process and understand how these models can be effectively utilized. By the end of this session, you will have a comprehensive understanding of RLHF, along with practical knowledge of building and leveraging InstructGPT models like ChatGPT.
12:00 PM - 1:00 PM: Lunch Break

Sandwich lunch and refreshments (tea, coffee, buiscuts, etc.) will be provided free of cost!

1:00 PM - 4:00 PM: Coding Tutorial: Advanced Techniques with Large Language Models and Alignment
- During this tutorial, you will explore advanced techniques with large language models and RLHF. You will gain hands-on experience with fine-tuning large-scale models using RLHF methods. Practical examples and exercises will be provided to help you apply these techniques to your own research questions and datasets.
4:00 PM - 6 PM: Coding Challenge and Awards Ceremony🏆
- In this practical part of our programme, you’ll join forces with your peers to form a set of teams, pitted against a demanding research problem. Each team will try to solve the challenge aided by the skills you have learned during the workshop. Success in this competition isn’t simply about finding the correct solution or achieving the best score; it’s about pushing boundaries, thinking outside the box, and showcasing ingenious problem-solving skills! Exciting prizes await the teams that demonstrate exceptional creativity, innovation, and deliver the most effective solutions. This coding challenge will serve as a great platform to test your skills, solidify your networks, cooperate with peers, and dive deep into the thrilling world of language models!