September 19, 2024
12:15 pm
Speaker:
Brent Harrison, Computer Science, University of Kentucky
Where:
327 McVey Hall
(Zoom link: https://uky.zoom.us/j/82467171189)
Title:
Practical Value Alignment Using GPT-4o
Abstract:
As artificial intelligence (AI) and machine learning (ML) systems grow in power, their risk of causing unintentional harm grows. This is because often these systems are optimizing for criteria that is different than our own, often meaning that it does not consider the many social and cultural norms that we implicitly and explicitly use when making decisions. In a value-aligned system, however, system behavior and objectives are aligned with expected human sociocultural norms. Creating a value-aligned system, however, can be difficult as sources of explicit value information are scarce.
In this tutorial, I will discuss how large language models, specifically GPT-4o, can be used to perform practical value alignment using a corpus of children’s stories. I will go over the entire learning pipeline, beginning from data curation and cleaning, all the way through zero and few-shot value extraction using GPT-4o. Ideally this will demonstrate the necessary steps to effectively use LLMs to perform complex learning tasks.
