- Ai - Engineering Mailing Lists

Schedule for AI Alignment Reading Group
by Pope, Quintin 15 Feb '22

15 Feb '22

Hello everyone, People have had schedule conflicts with our current meeting time, so I've decided to send out a survey to find a time that may work better. Please fill out the survey with your availability: http://whenisgood.net/swyfrcp Depending on responses, we may or may not be able to meet this week. I'll send out an update once the meeting time is confirmed. Our next paper will be "Red Teaming Language Models with Language Models <https://www.deepmind.com/research/publications/2022/Red-Teaming-Language-Mo…> ": Language Models (LMs) often cannot be deployed because of their potential > to harm users in ways that are hard to predict in advance. Prior work > identifies harmful behaviors before deployment by using human annotators to > hand-write test cases. However, human annotation is expensive, limiting the > number and diversity of test cases. In this work, we automatically find > cases where a target LM behaves in a harmful way, by generating test cases > (“red teaming”) using another LM. We evaluate the target LM’s replies to > generated test questions using a classifier trained to detect offensive > content, uncovering tens of thousands of offensive replies in a 280B > parameter LM chatbot. We explore several methods, from zero-shot generation > to reinforcement learning, for generating test cases with varying levels of > diversity and difficulty. Furthermore, we use prompt engineering to control > LM-generated test cases to uncover a variety of other harms, automatically > finding groups of people that the chatbot discusses in offensive ways, > personal and hospital phone numbers generated as the chatbot’s own contact > info, leakage of private training data in generated text, and harms that > occur over the course of a conversation. Overall, LM-based red teaming is > one promising tool (among many needed) for finding and fixing diverse, > undesirable LM behaviors before impacting users. > I hope as many people as possible are able to come to the new event. Join Zoom Meeting https://oregonstate.zoom.us/j/95843260079?pwd=TzZTN0xPaFZrazRGTElud0J1cnJLU… Password: 961594 Phone Dial-In Information +1 971 247 1195 US (Portland) +1 253 215 8782 US (Tacoma) +1 301 715 8592 US (Washington DC) Meeting ID: 958 4326 0079 All the best, Quintin

1 0

[CoRIS-List] OSU Robotics Seminar, Friday February 25: Nick Gravish
by Ross L. Hatton 14 Feb '22

14 Feb '22

Hi All, Our second winter Robotics Seminar will be on Friday, February 25 with a talk from Nick Gravish from the University of California, San Diego, who works in the area of bioinspired locomotion. The seminar will be from 10-11am, followed by an 11-11:30 student-only Q&A session. ** This seminar will be held in person in LINC 302. ** --- A simulcast of the seminar will be available via Zoom https://oregonstate.zoom.us/j/92183247338?pwd=MENOWHgvSFVGYzNjZExDT2hWRUNxd… --- -Ross =========== Ross L. Hatton Associate Professor, Robotics and Mechanical Engineering Collaborative Robotics and Intelligent Systems Institute Oregon State University coris.oregonstate.edu research.engr.oregonstate.edu/lram/ rosslhatton.com ross.hatton(a)oregonstate.edu =========== Title: Design and control of emergent oscillations for flapping-wing flyers and swarming snakes Abstract: Locomotion in living systems and bio-inspired robots requires the generation and control of oscillatory motion. While a common method to generate motion is through modulation of time-dependent “clock” signals, in this talk we will motivate and study an alternative method of oscillatory generation through autonomous limit-cycle systems. Limit-cycle oscillators for robotics have many desirable properties including adaptive behaviors, entrainment between oscillators, and potential simplification of motion control. I will present two examples of the generation and control of autonomous oscillatory motion in bio-inspired robotics. First, I will describe our recent work to study the dynamics of wingbeat oscillations in “asynchronous” insects and how we can build these behaviors into micro-aerial vehicles. In the second part of this talk I will describe how simple snake-like robots with limit-cycle gaits enable swarms to synchronize their movement through contact and without communication. More broadly in this talk I hope to motivate why we should look to autonomous dynamical systems for designing and controlling emergent locomotor behaviors in robotics. Bio: Dr. Nick Gravish received his PhD from Georgia Tech where he used robots as physical models to motivate and study aspects of biological locomotion. During his post-doc Gravish worked in the microrobotics lab of Rob Wood at Harvard, where he gained expertise in designing and studying insect-scale robots. Gravish is currently an assistant professor at UC San Diego in the Mechanical and Aerospace Engineering department. His lab bridges the gap between bio-inspiration, biomechanics, and robotics, towards the development of new bio-inspired robotic technologies to improve the adaptability and resilience of mobile robots.

1 0

Intel Internship in Deep RL
by Fern, Alan Paul 14 Feb '22

14 Feb '22

Hi Everyone, I was asked by a colleague at Intel Research to circulate this announcement for an Intel internship. https://jobs.intel.com/ShowJob/Id/3269293/AI-Deep-Learning-Graduate-Intern<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjobs.inte…> They are particularly interested in people with previous RL experience. There is the possibility of doing this internship remotely. -Alan

1 0

AI seminar: Feb 16 2022
by Mangannavar, Rajesh Devaraddi 10 Feb '22

10 Feb '22

Dear all, Our next AI seminar on *"Quo Vadis? Predicting Future Trajectories of Robots through Temporal Logics and Bayesian Inference" *by professor Sriram Sankaranarayanan is scheduled to be on February 16th, 1-2 PM PST. It will be followed by a 30 minute Q&A session by the graduate students. Zoom Link: https://oregonstate.zoom.us/j/93591935144?pwd=YjZaSjBYS0NmNUtjQzBEdzhPeDZ5U… *Quo Vadis? Predicting Future Trajectories of Robots through Temporal Logics and Bayesian Inference* Sriram Sankaranarayanan Associate Professor Computer Science University of Colorado, Boulder *Abstract:* Predicting the future states of robots through observation of its past and current actions is a very interesting problem. It is a fundamental "primitive" for applications such as runtime monitoring to prevent impending failures such as collisions or entering forbidden regions in the workspace. In this talk, we will present a few approaches to this problem beginning with a simple learning-based extrapolation of the robot's past positions to predict future trajectories, assuming a simple dynamical model for the robot. Unfortunately, such an extrapolation will remain valid only for relatively short time horizons. To improve upon this, we show how the "intent" of the agent can be represented and reasoned with. To represent intent, we use a restricted class of formulas from temporal logics as hypothesized intents. By combining these temporal logic representations through the machinery of Bayesian Inference, we show how we can predict future trajectories of robots rapidly through a combination of off-line pre-computations that enable cheaper real-time predictions. We conclude by describing ongoing work that develops a hierarchical approach wherein we separate "short-term" intents from "longer-term" intents that can be represented by the full strength of temporal logic-based specifications. This presentation is based on a series of joint works with Hansol Yoon and Chou Yi. *Speaker Bio:* Sriram Sankaranarayanan is an associate professor of Computer Science at the University of Colorado, Boulder. His research interests include automatic techniques for reasoning about the behavior of computer and cyber-physical systems. Sriram obtained a PhD in 2005 from Stanford University where he was advised by Zohar Manna and Henny Sipma. Subsequently he worked as a research staff member at NEC research labs in Princeton, NJ. He has been on the faculty at CU Boulder since 2009. Sriram has been the recipient of awards including the President's Gold Medal from IIT Kharagpur (2000), Siebel Scholarship (2005), the CAREER award from NSF (2009), Dean's award for outstanding junior faculty (2012), outstanding teaching (2014), and the Provost's faculty achievement award (2014). *Please watch this space for future AI Seminars :* * https://eecs.oregonstate.edu/ai-events <https://eecs.oregonstate.edu/ai-events>* Rajesh Mangannavar, Graduate Student Oregon State University ---- AI Seminar Important Reminders: -> For graduate students in the AI program, attendance is strongly encouraged.

1 0

AI Alignment Reading Group
by Pope, Quintin 09 Feb '22

09 Feb '22

Hello everyone, We'll be meeting Friday at 2 PM PST to discuss OpenAI's recent paper "Training language models to follow instructions with human feedback <https://cdn.openai.com/papers/Training_language_models_to_follow_instructio…> ". Making language models bigger does not inherently make them better at > following a user’s intent. For example, large language models can generate > outputs that are untruthful, toxic, or simply not helpful to the user. In > other words, these models are not aligned with their users. In this > paper, we show an avenue for aligning language models with user intent on a > wide range of tasks by fine-tuning with human feedback. Starting with a set > of labeler-written prompts and prompts submitted through the OpenAI API, we > collect a dataset of labeler demonstrations of the desired model behavior, > which we use to fine-tune GPT-3 using supervised learning. We then collect > a dataset of rankings of model outputs, which we use to further fine-tune > this supervised model using reinforcement learning from human feedback > (RLHF). We call the resulting models InstructGPT. In human evaluations on > our prompt distribution, outputs from the 1.3B parameter InstructGPT model > are preferred to outputs from the 175B GPT-3, despite having 100x fewer > parameters. Moreover, InstructGPT models show improvements in truthfulness > and reductions in toxic output generation while having minimal performance > regressions on public NLP datasets. Even though InstructGPT still makes > simple mistakes, our results show that fine-tuning with human feedback is a > promising direction for aligning language models with human intent. Anyone interested in language modeling, reinforcement learning, their intersection, or language models that can actually follow instructions should feel welcome to join! Join Zoom Meeting https://oregonstate.zoom.us/j/95843260079?pwd=TzZTN0xPaFZrazRGTElud0J1cnJLU… Password: 961594 Phone Dial-In Information +1 971 247 1195 US (Portland) +1 253 215 8782 US (Tacoma) +1 301 715 8592 US (Washington DC) Meeting ID: 958 4326 0079 All the best, Quintin

1 0

AI seminar: Feb 9 2022
by Mangannavar, Rajesh Devaraddi 08 Feb '22

08 Feb '22

Dear all, Our next AI seminar on *"Algorithmic Ethics for Autonomous Systems" *by professor Houssam Abbas is scheduled to be on February 9th(Tomorrow), 1-2 PM PST. It will be followed by a 30 minute Q&A session by the graduate students. Zoom Link: https://oregonstate.zoom.us/j/93591935144?pwd=YjZaSjBYS0NmNUtjQzBEdzhPeDZ5U… *Algorithmic Ethics for Autonomous Systems* Houssam Abbas Assistant Professor Electrical and Computer Engineering Oregon State University *Abstract:* The creation of intelligent systems that are autonomous, update their own objectives, and interact with humans in their daily lives, has long been a driving force in systems engineering, robotics, and Artificial Intelligence. Example systems include nursing robots in hospitals, self-driving vehicles, and worker bots collaborating with humans. An explicit ethical awareness in these systems is recognized as a necessary condition for successful daily interaction with humans. However, to this day, there are comparatively few algorithms, and even fewer tools, for designing ethics-equipped Autonomous Intelligent Systems (AIS), especially when integrated with a physical control loop. This research develops a computational theory and formal design tools for ethics-equipped embodied AIS. This talk will describe some of my research in developing engineering tools for automatic reasoning about ethical guidelines. Such guidelines take the form of statements of Obligation (`The robot ought to care for the patient in greater pain'), Permission (`The robot is permitted to offer a mask to a contagious patient') and Prohibition (`The robot is forbidden from factoring gender into care decisions'). We formalize such Obligations, Permissions and Prohibitions in deontic logic, and develop model-checking and learning algorithms for deontic properties of finite automata. I will then describe the road ahead for the formal study of ethical obligations in autonomous systems. *Speaker Bio:* Houssam Abbas is an Assistant Professor of Electrical Engineering and Computer Science at Oregon State University. His research interests are in the verification and control of cyber-physical systems and formal ethical theories for autonomous agents, with particular emphasis on unpiloted ground and aerial vehicles. He is a recipient of the NSF CAREER award. He was a postdoctoral fellow at the University of Pennsylvania, and a design automation engineer at Intel. *Please watch this space for future AI Seminars :* * https://eecs.oregonstate.edu/ai-events <https://eecs.oregonstate.edu/ai-events>* Rajesh Mangannavar, Graduate Student Oregon State University ---- AI Seminar Important Reminders: -> For graduate students in the AI program, attendance is strongly encouraged.

1 0

AI Alignment Reading Group Meeting
by Pope, Quintin 02 Feb '22

02 Feb '22

Hello everyone, We'll be discussing the paper On the Expressivity of Markov Reward <https://arxiv.org/abs/2111.00876>. Reward is the driving force for reinforcement-learning agents. This paper > is dedicated to understanding the expressivity of reward as a way to > capture tasks that we would want an agent to perform. We frame this study > around three new abstract notions of "task" that might be desirable: (1) a > set of acceptable behaviors, (2) a partial ordering over behaviors, or (3) > a partial ordering over trajectories. Our main results prove that while > reward can express many of these tasks, there exist instances of each task > type that no Markov reward function can capture. We then provide a set of > polynomial-time algorithms that construct a Markov reward function that > allows an agent to optimize tasks of each of these three types, and > correctly determine when no such reward function exists. We conclude with > an empirical study that corroborates and illustrates our theoretical > findings. (Thank you to Professor Dietterich for bringing this paper to my attention) We'll be sure to discuss the first post, and hopefully the other two if we have time. Anyone interested in trying to better understand goal directed behavior is welcome to join! We're meeting at 2 PM PST on Friday like normal. Anyone interested in reinforcement learning should feel welcome to attend! Join Zoom Meeting https://oregonstate.zoom.us/j/95843260079?pwd=TzZTN0xPaFZrazRGTElud0J1cnJLU… Password: 961594 Phone Dial-In Information +1 971 247 1195 US (Portland) +1 253 215 8782 US (Tacoma) +1 301 715 8592 US (Washington DC) Meeting ID: 958 4326 0079 All the best, Quintin

1 0

AI seminar: Feb 2 2022
by Mangannavar, Rajesh Devaraddi 01 Feb '22

01 Feb '22

Dear all, Our next AI seminar on *"The Familiarity Hypothesis: Explaining the Behavior of Deep Open Set Methods" *by professor Tom Dietterich is scheduled to be on February 2nd (Tomorrow), 1-2 PM PST. It will be followed by a 30 minute Q&A session by the graduate students. Zoom Link: https://oregonstate.zoom.us/j/93591935144?pwd=YjZaSjBYS0NmNUtjQzBEdzhPeDZ5U… *The Familiarity Hypothesis: Explaining the Behavior of Deep Open Set Methods* Tom Dietterich Distinguished Professor Emeritus Computer Science Oregon State University *Abstract:* In many applications, object recognition systems encounter objects belonging to categories unseen during training. Hence, the set of possible categories is an open set. Detecting such “novel category'” objects is usually formulated as an anomaly detection problem. Anomaly detection algorithms for feature-vector data identify anomalies as outliers, but outlier detection has not worked well in deep learning. Instead, methods based on the computed logits of object recognition systems give state-of-the-art performance. This talk proposes the Familiarity Hypothesis that these methods succeed because they are detecting the absence of familiar learned features. This talk will review evidence from the literature and from our own experiments that supports this hypothesis. It then experimentally tests a set of predicted consequences of this hypothesis that provide additional support. The talk will conclude with a discussion of whether familiarity detection is an inevitable consequence of representation learning and concludes that we can go beyond familiarity detection if we can learn to represent objects in terms of disentangled attributes that support outlier detection. *Speaker Bio:* Tom Dietterich (AB Oberlin College 1977; MS University of Illinois 1979; Ph.D. Stanford University 1984) is Distinguished Professor Emeritus in the School of Electrical Engineering and Computer Science at Oregon State University. He is one of the pioneers of the field of machine learning and has authored more than 225 refereed publications and two books. His current research topics include robust artificial intelligence, robust human-AI systems, and applications in sustainability. Dietterich has devoted many years of service to the research community. He is a former president of the Association for the Advancement of Artificial Intelligence and the founding president of the International Machine Learning Society. Other major roles include executive editor of the Journal Machine Learning, co-founder of the Journal for Machine Learning Research, and program chair of AAAI 1990 and NIPS 2000. He currently serves as one of the moderators for the cs.LG category on arXiv. *Please watch this space for future AI Seminars :* * https://eecs.oregonstate.edu/ai-events <https://eecs.oregonstate.edu/ai-events>* Rajesh Mangannavar, Graduate Student Oregon State University ---- AI Seminar Important Reminders: -> For graduate students in the AI program, attendance is strongly encouraged.

1 0

[CoRIS-List] Reminder: Today: OSU Robotics Seminar, Friday January 28: Sandhya Saisubramanian
by Ross L. Hatton 28 Jan '22

28 Jan '22

Hi All, As a reminder, we're kicking off the winter term of the Robotics Seminar today with a talk from Sandhya Saisubramanian, who recently joined the EECS faculty at Oregon State and works in the area of autonomous systems. The seminar will be from 10-11am, followed by an 11-11:30 student-only Q&A session. This first seminar will be conducted via Zoom. The seminar Zoom room can be reached at https://oregonstate.zoom.us/j/92183247338?pwd=MENOWHgvSFVGYzNjZExDT2hWRUNxd… I also have a Gather space (with a direct link to the Zoom session) set up at https://gather.town/app/HeC6WAB1zL6cKWp5/OSU%20Robotics%20Seminar for pre- and post-seminar socialization. -Ross =========== Ross L. Hatton Associate Professor, Robotics and Mechanical Engineering Collaborative Robotics and Intelligent Systems Institute Oregon State University coris.oregonstate.edu research.engr.oregonstate.edu/lram/ rosslhatton.com ross.hatton(a)oregonstate.edu =========== Title: Avoiding Negative Side Effects of Autonomous Systems in the Open World Abstract: Autonomous systems are increasingly deployed in open-world environments. These systems have broad societal impacts and it is critical to ensure that they operate safely when deployed. Due to the practical challenges in data collection and precise model specification, AI systems often operate based on imperfect models which may lead to undesirable behaviors. A particular form of unexpected, undesirable consequence is the negative side effects. In this talk, I will (1) discuss the challenges in identifying and mitigating the impacts of negative side effects; (2) examine the efficiency of different forms of feedback to learn the penalty associated with negative side effects; and (3) discuss a multi-objective approach to mitigate the side effects. Bio: Sandhya Saisubramanian is an Assistant Professor in the School of EECS at Oregon State University. She recently completed her Ph.D. from the University of Massachusetts Amherst. Her research interests are in developing general techniques for reliable decision-making in autonomous systems that operate in fully and partially observable open-world environments. She is a recipient of Distinguished Paper Award at IJCAI 2020 and was selected as a Rising Star in EECS by UC Berkeley in 2020.

1 0

AI seminar: Feb 2 2022
by Mangannavar, Rajesh Devaraddi 27 Jan '22

27 Jan '22

Dear all, Our next AI seminar on *"The Familiarity Hypothesis: Explaining the Behavior of Deep Open Set Methods" *by professor Tom Dietterich is scheduled to be on February 2nd, 1-2 PM PST. It will be followed by a 30 minute Q&A session by the graduate students. Zoom Link: https://oregonstate.zoom.us/j/93591935144?pwd=YjZaSjBYS0NmNUtjQzBEdzhPeDZ5U… *The Familiarity Hypothesis: Explaining the Behavior of Deep Open Set Methods* Tom Dietterich Distinguished Professor Emeritus Computer Science Oregon State University *Abstract:* In many applications, object recognition systems encounter objects belonging to categories unseen during training. Hence, the set of possible categories is an open set. Detecting such “novel category'” objects is usually formulated as an anomaly detection problem. Anomaly detection algorithms for feature-vector data identify anomalies as outliers, but outlier detection has not worked well in deep learning. Instead, methods based on the computed logits of object recognition systems give state-of-the-art performance. This talk proposes the Familiarity Hypothesis that these methods succeed because they are detecting the absence of familiar learned features. This talk will review evidence from the literature and from our own experiments that supports this hypothesis. It then experimentally tests a set of predicted consequences of this hypothesis that provide additional support. The talk will conclude with a discussion of whether familiarity detection is an inevitable consequence of representation learning and concludes that we can go beyond familiarity detection if we can learn to represent objects in terms of disentangled attributes that support outlier detection. *Speaker Bio:* Tom Dietterich (AB Oberlin College 1977; MS University of Illinois 1979; Ph.D. Stanford University 1984) is Distinguished Professor Emeritus in the School of Electrical Engineering and Computer Science at Oregon State University. He is one of the pioneers of the field of machine learning and has authored more than 225 refereed publications and two books. His current research topics include robust artificial intelligence, robust human-AI systems, and applications in sustainability. Dietterich has devoted many years of service to the research community. He is a former president of the Association for the Advancement of Artificial Intelligence and the founding president of the International Machine Learning Society. Other major roles include executive editor of the Journal Machine Learning, co-founder of the Journal for Machine Learning Research, and program chair of AAAI 1990 and NIPS 2000. He currently serves as one of the moderators for the cs.LG category on arXiv. *Please watch this space for future AI Seminars :* * https://eecs.oregonstate.edu/ai-events <https://eecs.oregonstate.edu/ai-events>* Rajesh Mangannavar, Graduate Student Oregon State University ---- AI Seminar Important Reminders: -> For graduate students in the AI program, attendance is strongly encouraged.

1 0