- Ai - Engineering Mailing Lists

AI alignment reading group: Deep reinforcement learning from human preferences
by Turner, Alex 06 May '21

06 May '21

This Friday, we'll discuss "Deep reinforcement learning from human preferences": "For sophisticated reinforcement learning (RL) systems to interact usefully with real-world environments, we need to communicate complex goals to these systems. In this work, we explore goals defined in terms of (non-expert) human preferences between pairs of trajectory segments. We show that this approach can effectively solve complex RL tasks without access to the reward function, including Atari games and simulated robot locomotion, while providing feedback on less than one percent of our agent's interactions with the environment. This reduces the cost of human oversight far enough that it can be practically applied to state-of-the-art RL systems. To demonstrate the flexibility of our approach, we show that we can successfully train complex novel behaviors with about an hour of human time. These behaviors and environments are considerably more complex than any that have been previously learned from human feedback. " Paper: https://arxiv.org/abs/1706.03741 We'll meet Friday at 1. https://oregonstate.zoom.us/j/2739792686?pwd=VkRUeHJkYnhvTzlvZzR6YnZWNERKQT…

1 0

OSU AI Seminar - **Today** 1PM | Dale Schuurmans - Google Brain/University of Alberta
by Patravali, Jay Sanjay 05 May '21

05 May '21

Dear All, A gentle reminder that we a seminar by Prof. Dale Schuurmans from Google Brain/University of Alberta. He will be presenting *"*Reductionism in Reinforcement Learning*"* *today 5/5 from 1.00-2.00PM PT.* Zoom Link: https://tinyurl.com/392tj3hk Please find attached the poster flyer for more information. *Title*: Reductionism in Reinforcement Learning *Abstract*: Learning to make sequential decisions from interaction raises numerous challenges, including temporal prediction, state and value estimation, sequential planning, exploration, and strategic interaction. Each challenge is independently difficult, yet reinforcement learning research has often sought holistic solutions that rely on unified learning principles. I will argue that such a holistic approach can hamper progress. To make the point, I focus on reinforcement learning in Markov Decision Processes, where the challenges of value estimation, sequential planning, and exploration are jointly raised. By eliminating exploration from consideration, recent work on off-line reinforcement learning has led to improved methods for value estimation, policy optimization, and sequential planning. Taking the reduction a step further, I reconsider the relationship between value estimation and sequential planning, and show that a unified approach faces unresolved difficulties whenever generalization is considered. Instead, by separating these two challenges, a clearer understanding of successful value estimation methods can be achieved, while the shortcomings of existing strategies can be overcome in some cases. I will attempt to illustrate how these reductions allow other areas of machine learning, optimization, control and planning to be better leveraged in reinforcement learning. *Bio*: Dale Schuurmans is a Research Scientist at Google Brain, Professor of Computing Science at the University of Alberta, a Canada CIFAR AI Chair, and a Fellow of AAAI. He has served as an Associate Editor in Chief for IEEE TPAMI, an Associate Editor for JMLR, AIJ, JAIR and MLJ, and as a Program Co-chair for AAAI-2016, NIPS-2008 and ICML-2004. He has worked in many areas of machine learning and artificial intelligence, including model selection, on-line learning, adversarial optimization, boolean satisfiability, sequential decision making, reinforcement learning, Bayesian optimization, semidefinite methods for unsupervised learning, dimensionality reduction, and robust estimation. He has published over 200 papers in these areas, and has received paper awards at NeurIPS, ICML, IJCAI, and AAAI. Please plan on attending! Thanks, Jay Patravali Graduate Student, CoRIS-EECS. Oregon State University ---- AI Seminar Important Reminders: -> The AI Seminar has a strict "no electronics" and "no recordings" policy. -> For graduate students in the AI program, attendance is strongly encouraged.

1 0

OSU AI Seminar - May 5 2021 | Dale Schuurmans - Google Brain/University of Alberta
by Patravali, Jay Sanjay 04 May '21

04 May '21

Dear All, Our next seminar will be by Prof. Dale Schuurmans from Google Brain/University of Alberta. He will be presenting *"*Reductionism in Reinforcement Learning*"* on *Wednesday 5/5 from 1.00-2.00PM PT.* Zoom Link: https://tinyurl.com/392tj3hk Please find attached the poster flyer for more information. *Title*: Reductionism in Reinforcement Learning *Abstract*: Learning to make sequential decisions from interaction raises numerous challenges, including temporal prediction, state and value estimation, sequential planning, exploration, and strategic interaction. Each challenge is independently difficult, yet reinforcement learning research has often sought holistic solutions that rely on unified learning principles. I will argue that such a holistic approach can hamper progress. To make the point, I focus on reinforcement learning in Markov Decision Processes, where the challenges of value estimation, sequential planning, and exploration are jointly raised. By eliminating exploration from consideration, recent work on off-line reinforcement learning has led to improved methods for value estimation, policy optimization, and sequential planning. Taking the reduction a step further, I reconsider the relationship between value estimation and sequential planning, and show that a unified approach faces unresolved difficulties whenever generalization is considered. Instead, by separating these two challenges, a clearer understanding of successful value estimation methods can be achieved, while the shortcomings of existing strategies can be overcome in some cases. I will attempt to illustrate how these reductions allow other areas of machine learning, optimization, control and planning to be better leveraged in reinforcement learning. *Bio*: Dale Schuurmans is a Research Scientist at Google Brain, Professor of Computing Science at the University of Alberta, a Canada CIFAR AI Chair, and a Fellow of AAAI. He has served as an Associate Editor in Chief for IEEE TPAMI, an Associate Editor for JMLR, AIJ, JAIR and MLJ, and as a Program Co-chair for AAAI-2016, NIPS-2008 and ICML-2004. He has worked in many areas of machine learning and artificial intelligence, including model selection, on-line learning, adversarial optimization, boolean satisfiability, sequential decision making, reinforcement learning, Bayesian optimization, semidefinite methods for unsupervised learning, dimensionality reduction, and robust estimation. He has published over 200 papers in these areas, and has received paper awards at NeurIPS, ICML, IJCAI, and AAAI. Please plan on attending! Thanks, Jay Patravali Graduate Student, CoRIS-EECS. Oregon State University ---- AI Seminar Important Reminders: -> The AI Seminar has a strict "no electronics" and "no recordings" policy. -> For graduate students in the AI program, attendance is strongly encouraged.

1 0

[CoRIS-List] Starting Now: Kory Mathewson
by Hatton, Ross 30 Apr '21

30 Apr '21

Hi All, Today's seminar is starting now: Kory Mathewson (DeepMind) Speaker bio: https://korymathewson.com Watch link: *https://oregonstate.zoom.us/j/97644079157?pwd=WXRZeEhOR0padXVzb2NkVkVkVEFmZz09 <https://oregonstate.zoom.us/j/97644079157?pwd=WXRZeEhOR0padXVzb2NkVkVkVEFmZ…>* --- Title: "Humour-in-the-loop: Spontaneous Storytelling with Interactive Machine Learning" Abstract: Improvisation is the ideal evaluation testbed for collaboration between humans and intelligent machines. In improvised theatre artists perform real-time, dynamic problem solving to collaboratively generate interesting narratives. I co-developed artificial improvisation: improvised theatre performed by humans alongside intelligent machines. I start by presenting the background underlying the art of improvisation and the scientific fields of interactive machine learning and dialogue generation. Then, I present Pyggy, the first experiment on live stage human-machine improvisation, and A.L.Ex., the Artificial Language Experiment, which addresses several critical technical improvements over Pyggy. "Improbotics" is then presented, which details audience evaluation of Turing test-inspired live improvised performance using A.L.Ex. The talk concludes by discussing public reflections on live artificial improvisation performances from around the world and interesting future directions to explore. I also discuss ongoing work at the intersection of intelligent minds both human and machine. --- Pre- and post-seminar Gather space for OSU Robotics attendees (with an integrated link to the main talk) is at https://gather.town/app/HeC6WAB1zL6cKWp5/OSU%20Robotics%20Seminar -Ross =========== Ross L. Hatton Associate Professor, Robotics and Mechanical Engineering Collaborative Robotics and Intelligent Systems Institute Oregon State University coris.oregonstate.edu research.engr.oregonstate.edu/lram/ rosslhatton.com ross.hatton(a)oregonstate.edu

1 0

AI alignment reading group: inaccessible information
by Turner, Alex 29 Apr '21

29 Apr '21

This Friday, we'll discuss "Inaccessible information": "Suppose that I have a great model for predicting “what will Alice say next?” I can evaluate and train this model by checking its predictions against reality, but there may be many facts this model “knows” that I can’t easily access. For example, the model might have a detailed representation of Alice’s thoughts which it uses to predict what Alice will say, *without* being able to directly answer “What is Alice thinking?” In this case, I can only access that knowledge indirectly, e.g. by asking about what Alice would say under different conditions. I’ll call information like “What is Alice thinking?” inaccessible. I think it’s very plausible that AI systems will build up important inaccessible knowledge, and that this may be a central feature of the AI alignment problem." Post: https://www.alignmentforum.org/posts/ZyWyAJbedvEgRT2uF/inaccessible-informa… We'll meet Friday at 1. https://oregonstate.zoom.us/j/2739792686?pwd=VkRUeHJkYnhvTzlvZzR6YnZWNERKQT…

1 0

[CoRIS-List] Reminder: Friday, 10am: Kory Mathewson
by Hatton, Ross 28 Apr '21

28 Apr '21

Hi All, As a reminder, we have an OSU Robotics Zoom seminar at 10am on Friday April 30: Kory Mathewson (DeepMind) Speaker bio: https://korymathewson.com Watch link: *https://oregonstate.zoom.us/j/97644079157?pwd=WXRZeEhOR0padXVzb2NkVkVkVEFmZz09 <https://oregonstate.zoom.us/j/97644079157?pwd=WXRZeEhOR0padXVzb2NkVkVkVEFmZ…>* --- Title: "Humour-in-the-loop: Spontaneous Storytelling with Interactive Machine Learning" Abstract: Improvisation is the ideal evaluation testbed for collaboration between humans and intelligent machines. In improvised theatre artists perform real-time, dynamic problem solving to collaboratively generate interesting narratives. I co-developed artificial improvisation: improvised theatre performed by humans alongside intelligent machines. I start by presenting the background underlying the art of improvisation and the scientific fields of interactive machine learning and dialogue generation. Then, I present Pyggy, the first experiment on live stage human-machine improvisation, and A.L.Ex., the Artificial Language Experiment, which addresses several critical technical improvements over Pyggy. "Improbotics" is then presented, which details audience evaluation of Turing test-inspired live improvised performance using A.L.Ex. The talk concludes by discussing public reflections on live artificial improvisation performances from around the world and interesting future directions to explore. I also discuss ongoing work at the intersection of intelligent minds both human and machine. --- Pre- and post-seminar Gather space for OSU Robotics attendees (with an integrated link to the main talk) is at https://gather.town/app/HeC6WAB1zL6cKWp5/OSU%20Robotics%20Seminar -Ross =========== Ross L. Hatton Associate Professor, Robotics and Mechanical Engineering Collaborative Robotics and Intelligent Systems Institute Oregon State University coris.oregonstate.edu research.engr.oregonstate.edu/lram/ rosslhatton.com ross.hatton(a)oregonstate.edu

1 0

OSU AI Seminar - April 28th 2021 | J. Andrew (Drew) Bagnell - Aurora.tech/CMU
by Patravali, Jay Sanjay 28 Apr '21

28 Apr '21

Dear All, Our first seminar for this spring term will be by Prof. J. Andrew (Drew) Bagnell from Aurora.tech/Carnegie Mellon University. He will be presenting *"*Machine Learning, Games, and the Aurora Driver*"* on *Wednesday 4/28 from 1.00-2.00PM PT.* Zoom Link: https://tinyurl.com/eeapzpxk Please find attached the poster flyer for more information. *Title*: Machine Learning, Games, and the Aurora Driver *Abstract*: In this talk, I’ll explore the power of a game-theoretic viewpoint in self-driving and in machine learning. We’ll begin by considering the application of machine learning to Aurora’s advanced self-driving system in perception, forecasting and decision-making. We’ll discuss particularly complexities that arise from multi-actor interaction. We will then explore the, perhaps surprising, role a game-theoretic view can take in developing algorithms for learning to make decisions. *Bio*: J. Andrew (Drew) Bagnell is Chief Scientist and co-founder of Aurora (aurora.tech) where he works to develop self-driving vehicles. He also serves as Consulting Professor at CMU’s Robotics Institute and Machine Learning Department. He has worked for two decades at the intersection of ML and robotics in industrial and academic roles. Bagnell’s group has received over a dozen research awards for publications in both the robotics and ML communities including best paper awards at ICML, RSS, and ICRA. He received the 2016 Ryan Award, CMU’s award for Meritorious Teaching, and served as the founding director of the Robotics Institute Summer Scholars program, a research experience that has enabled hundreds of undergraduates throughout the world to leap into robotics research. Please plan on attending! Thanks, Jay Patravali Graduate Student, CoRIS-EECS. Oregon State University ---- AI Seminar Important Reminders: -> The AI Seminar has a strict "no electronics" and "no recordings" policy. -> For graduate students in the AI program, attendance is strongly encouraged.

2 1

OSU AI Seminar - **Today** 1PM | J. Andrew (Drew) Bagnell - Aurora.tech/CMU
by Patravali, Jay Sanjay 28 Apr '21

28 Apr '21

Dear All, A gentle reminder for our first seminar this spring term by Prof. J. Andrew (Drew) Bagnell from Aurora.tech/Carnegie Mellon University. He will be presenting *"*Machine Learning, Games, and the Aurora Driver*"* on *Today 4/28 from 1.00-2.00PM PT.* Zoom Link: https://tinyurl.com/eeapzpxk Please find attached the poster flyer for more information. *Title*: Machine Learning, Games, and the Aurora Driver *Abstract*: In this talk, I’ll explore the power of a game-theoretic viewpoint in self-driving and in machine learning. We’ll begin by considering the application of machine learning to Aurora’s advanced self-driving system in perception, forecasting and decision-making. We’ll discuss particularly complexities that arise from multi-actor interaction. We will then explore the, perhaps surprising, role a game-theoretic view can take in developing algorithms for learning to make decisions. *Bio*: J. Andrew (Drew) Bagnell is Chief Scientist and co-founder of Aurora (aurora.tech) where he works to develop self-driving vehicles. He also serves as Consulting Professor at CMU’s Robotics Institute and Machine Learning Department. He has worked for two decades at the intersection of ML and robotics in industrial and academic roles. Bagnell’s group has received over a dozen research awards for publications in both the robotics and ML communities including best paper awards at ICML, RSS, and ICRA. He received the 2016 Ryan Award, CMU’s award for Meritorious Teaching, and served as the founding director of the Robotics Institute Summer Scholars program, a research experience that has enabled hundreds of undergraduates throughout the world to leap into robotics research. Please plan on attending! Thanks, Jay Patravali Graduate Student, CoRIS-EECS. Oregon State University ---- AI Seminar Important Reminders: -> The AI Seminar has a strict "no electronics" and "no recordings" policy. -> For graduate students in the AI program, attendance is strongly encouraged.

1 0

[CoRIS-List] Starting Now: Koushil Sreenath
by Hatton, Ross 23 Apr '21

23 Apr '21

Hi All, Today's seminar is starting now: Koushil Sreenath (UC Berkeley) Speaker bio: https://roboticstoday.github.io/ Watch link: https://roboticstoday.github.io/watch.html Pre- and post-seminar Gather space for OSU Robotics attendees (with an integrated link to the main talk) is at https://gather.town/app/HeC6WAB1zL6cKWp5/OSU%20Robotics%20Seminar -Ross =========== Ross L. Hatton Associate Professor, Robotics and Mechanical Engineering Collaborative Robotics and Intelligent Systems Institute Oregon State University coris.oregonstate.edu research.engr.oregonstate.edu/lram/ rosslhatton.com ross.hatton(a)oregonstate.edu

1 0

Meeting topic for April 23
by Quintin Pope 23 Apr '21

23 Apr '21

Hello everyone, This Friday, we’ll be discussing content from the “Circuits” thread on distill.pub: https://distill.pub/2020/circuits/ This is a series posts on reverse engineering the internal structure of vision nets by looking at the weights and features such networks learn. We’ll particularly focus on the “Branch Specialization” post: https://distill.pub/2020/circuits/branch-specialization/, which discusses how different network regions can specialize in different sorts of tasks. Neither Neale nor Alex will be at the meeting (one reason why the meeting topic went out so late). We’ll still be meeting at 1 PM and use the normal meeting room: https://oregonstate.zoom.us/j/2739792686?pwd=VkRUeHJkYnhvTzlvZzR6YnZWNERKQT… (I’ll send an email with a new room if this doesn’t work) Best, Quintin

1 0