Dear all, We are going to have 2 AI seminars this week. The first one will be on *"**Reinforcement Learning with Exogenous States and Rewards**" *by George Trimponias and is scheduled to be on November 3rd (Thursday), 9-10 AM PST. It will be followed by a 30-minute Q&A session with the graduate students. *Please note that the speaker will be on zoom for the event but it will be set up in Rogers 230 for everyone to attend.* *Reinforcement Learning with Exogenous States and Rewards* George Trimponias Sr. Applied Scientist Amazon *Abstract:* Exogenous state variables and rewards can slow down reinforcement learning by injecting uncontrolled variation into the reward signal. We formalize exogenous state variables and rewards and show that if the reward function decomposes additively into endogenous and exogenous components, the MDP can be decomposed into an exogenous Markov Reward Process (based on the exogenous reward) and an endogenous Markov Decision Process (optimizing the endogenous reward). Any optimal policy for the endogenous MDP is also an optimal policy for the original MDP, but because the endogenous reward typically has reduced variance, the endogenous MDP is easier to solve. We consider settings where the decomposition of the state space into exogenous and endogenous variables or subspaces is not given but must be discovered. We introduce and prove correctness of algorithms for discovering the exogenous and endogenous subspaces of the state space when they are mixed through linear combination. These algorithms can be applied during reinforcement learning to discover the exogenous space, remove the exogenous reward, and focus reinforcement learning on the endogenous MDP. Experiments on a variety of challenging synthetic MDPs show that these methods, applied online, discover surprisingly large exogenous subspaces and produce large speedups in reinforcement learning. *Speaker Bio:* George Trimponias has been an Applied Scientist with Amazon Search in Luxembourg since 2020, focusing on natural language processing for improved information retrieval. He was a Researcher at Huawei Noah’s Ark Lab in Hong Kong from 2015 to early 2020, where he conducted machine learning research for communication networks. He received his PhD in Computer Science and Engineering from the Hong Kong University of Science and Technology. His research interests include machine learning, game theory and optimization. His recent focus is on the design of efficient algorithms that can accelerate reinforcement learning in the presence of exogenous states and rewards. This will be followed by a talk on Friday, November 4th, 2022. The second talk will be on *"**Responsibility in AI for Fair and Transparent Search and Recommender Systems**" *by Chirag Shah and is scheduled to be on November 4th(Friday), 1-2 PM PST. It will be followed by a 30-minute Q&A session with the graduate students. *Please note that the speaker will be on zoom for the event but it will be set up in KEC 1001 for everyone to attend.* *Responsibility in AI for Fair and Transparent Search and Recommender Systems* Chirag Shah Professor Information School University of Washington *Abstract:* Bias in data as well as lack of transparency and fairness in algorithms are not new problems, but with the increasing scale, complexity, and adoption, most AI systems are suffering from these issues at a level unprecedented. Information access systems are not spared since these days, almost all large-scale systems of information access are mediated by algorithms. These algorithms are optimized not only for relevance, which is subjective to begin with, but also for measures of engagement and impressions. They are picking up signals of what may be 'good' from individuals and perpetuating that through learning methods that are opaque and hard to debug. Considering 'fairness' and introducing more transparency can help, but it can also backfire or create other issues. We also need to understand how and why users of these systems engage with content. In this talk, I will share some of our attempts for bringing fairness in ranking systems and then talk about how the solutions are not that simple. To really address the problems of misinformation and misrepresentation in information access, we need to look for human-AI synergy where the responsibility of fairness and transparency lies not only on systems, but also on developers, regulators, and end-users. *Speaker Bio:* Chirag Shah is Professor in Information School (iSchool) at University of Washington (UW) in Seattle. He is also Adjunct Professor with Paul G. Allen School of Computer Science & Engineering as well as Human Centered Design & Engineering (HCDE). He is the Founding Director for InfoSeeking Lab and Founding Co-Director of Center for Responsibility in AI Systems & Experiences (RAISE). His research interests include intelligent search and recommender systems, trying to understand the task a person is doing and providing proactive recommendations. In addition to creating task-based systems that provide more personalized reactive and proactive recommendations, he is also focusing on making such systems transparent, fair, and free of biases. He is the recipient of 2019 Microsoft BCS/BCS IRSG Karen Spärck Jones Award. *Please watch this space for future AI Seminars :* * https://eecs.oregonstate.edu/ai-events <https://eecs.oregonstate.edu/ai-events>* Rajesh Mangannavar, Graduate Student Oregon State University ---- AI Seminar Important Reminders: -> For graduate students in the AI program, attendance is strongly encouraged.