Dissertation Talk: Designing and Evaluating AI Algorithms in Human-Centered Settings

Event details Date: Thursday, July 30, 2026 Time: 12:00 PM to 12:00 PM Location: Gateway 3355 Type: Sports Audience: Faculty,Students About this event As AI models are increasingly deployed in settings shaped by complex human behavior, there is a critical need for algorithmic principles that account for human values and strategic incentives. This dissertation develops theoretical foundations for designing and evaluating AI algorithms in human-centered settings, drawing on tools from algorithmic economics and learning theory. In this talk, I will present two central threads of this work. First, I will discuss incentive-aware evaluation, where I will introduce a framework for truthful sequential probability forecasting and algorithmic principles for designing calibration metrics that inherently incentivize high-quality predictions. Second, I will discuss how aligning AI with human preferences must account for preference heterogeneity, and introduce a framework called the distortion of AI alignment that characterizes information-theoretic limits of learning from heterogeneous human feedback and motivates robust, game-theoretic approaches to policy optimization. I will then discuss how these ideas extend to the design of pluralistic leaderboards for AI evaluation, which incorporate heterogeneous human preferences into model rankings. Official event details: https://events.berkeley.edu/eecs/event/324927-dissertation-talk-designing-and-evaluating-ai