October 2022 - Ai - Engineering Mailing Lists

AI seminar: November 4, 2022
by Mangannavar, Rajesh Devaraddi 31 Oct '22

31 Oct '22

Dear all, We are going to have 2 AI seminars this week. The first one will be on *"**Reinforcement Learning with Exogenous States and Rewards**" *by George Trimponias and is scheduled to be on November 3rd (Thursday), 9-10 AM PST. It will be followed by a 30-minute Q&A session with the graduate students. *Please note that the speaker will be on zoom for the event but it will be set up in Rogers 230 for everyone to attend.* *Reinforcement Learning with Exogenous States and Rewards* George Trimponias Sr. Applied Scientist Amazon *Abstract:* Exogenous state variables and rewards can slow down reinforcement learning by injecting uncontrolled variation into the reward signal. We formalize exogenous state variables and rewards and show that if the reward function decomposes additively into endogenous and exogenous components, the MDP can be decomposed into an exogenous Markov Reward Process (based on the exogenous reward) and an endogenous Markov Decision Process (optimizing the endogenous reward). Any optimal policy for the endogenous MDP is also an optimal policy for the original MDP, but because the endogenous reward typically has reduced variance, the endogenous MDP is easier to solve. We consider settings where the decomposition of the state space into exogenous and endogenous variables or subspaces is not given but must be discovered. We introduce and prove correctness of algorithms for discovering the exogenous and endogenous subspaces of the state space when they are mixed through linear combination. These algorithms can be applied during reinforcement learning to discover the exogenous space, remove the exogenous reward, and focus reinforcement learning on the endogenous MDP. Experiments on a variety of challenging synthetic MDPs show that these methods, applied online, discover surprisingly large exogenous subspaces and produce large speedups in reinforcement learning. *Speaker Bio:* George Trimponias has been an Applied Scientist with Amazon Search in Luxembourg since 2020, focusing on natural language processing for improved information retrieval. He was a Researcher at Huawei Noah’s Ark Lab in Hong Kong from 2015 to early 2020, where he conducted machine learning research for communication networks. He received his PhD in Computer Science and Engineering from the Hong Kong University of Science and Technology. His research interests include machine learning, game theory and optimization. His recent focus is on the design of efficient algorithms that can accelerate reinforcement learning in the presence of exogenous states and rewards. This will be followed by a talk on Friday, November 4th, 2022. The second talk will be on *"**Responsibility in AI for Fair and Transparent Search and Recommender Systems**" *by Chirag Shah and is scheduled to be on November 4th(Friday), 1-2 PM PST. It will be followed by a 30-minute Q&A session with the graduate students. *Please note that the speaker will be on zoom for the event but it will be set up in KEC 1001 for everyone to attend.* *Responsibility in AI for Fair and Transparent Search and Recommender Systems* Chirag Shah Professor Information School University of Washington *Abstract:* Bias in data as well as lack of transparency and fairness in algorithms are not new problems, but with the increasing scale, complexity, and adoption, most AI systems are suffering from these issues at a level unprecedented. Information access systems are not spared since these days, almost all large-scale systems of information access are mediated by algorithms. These algorithms are optimized not only for relevance, which is subjective to begin with, but also for measures of engagement and impressions. They are picking up signals of what may be 'good' from individuals and perpetuating that through learning methods that are opaque and hard to debug. Considering 'fairness' and introducing more transparency can help, but it can also backfire or create other issues. We also need to understand how and why users of these systems engage with content. In this talk, I will share some of our attempts for bringing fairness in ranking systems and then talk about how the solutions are not that simple. To really address the problems of misinformation and misrepresentation in information access, we need to look for human-AI synergy where the responsibility of fairness and transparency lies not only on systems, but also on developers, regulators, and end-users. *Speaker Bio:* Chirag Shah is Professor in Information School (iSchool) at University of Washington (UW) in Seattle. He is also Adjunct Professor with Paul G. Allen School of Computer Science & Engineering as well as Human Centered Design & Engineering (HCDE). He is the Founding Director for InfoSeeking Lab and Founding Co-Director of Center for Responsibility in AI Systems & Experiences (RAISE). His research interests include intelligent search and recommender systems, trying to understand the task a person is doing and providing proactive recommendations. In addition to creating task-based systems that provide more personalized reactive and proactive recommendations, he is also focusing on making such systems transparent, fair, and free of biases. He is the recipient of 2019 Microsoft BCS/BCS IRSG Karen Spärck Jones Award. *Please watch this space for future AI Seminars :* * https://eecs.oregonstate.edu/ai-events <https://eecs.oregonstate.edu/ai-events>* Rajesh Mangannavar, Graduate Student Oregon State University ---- AI Seminar Important Reminders: -> For graduate students in the AI program, attendance is strongly encouraged.

1 0

AI seminar: October 28, 2022
by Mangannavar, Rajesh Devaraddi 27 Oct '22

27 Oct '22

Dear all, Our next AI seminar on *"Investigating Latent State and Uncertainty Representations in Reinforcement Learning" *by Anurag Koul is scheduled to be on October 28th (Tomorrow), 1-2 PM PST. It will be followed by a 30-minute Q&A session by the graduate students. *Please note that the speaker will be on zoom for the event but it will be set up in KEC 1001 for everyone to attend.* *Investigating Latent State and Uncertainty Representations in Reinforcement Learning* Anurag Koul Postdoctoral Researcher Microsoft Research (New York) *Abstract:* Learning latent space representations of high-dimensional world states has been at the core of recent rapid growth in reinforcement learning(RL). At the same time, RL algorithms have suffered from ignored uncertainties in the predicted estimates of model-free or model-based methods. In our work, we investigate both of these aspects independently. Firstly, we studied the explainability of policies learned over latent representations. In particular, we focus on control policies represented as recurrent neural networks (RNNs) which are difficult to explain, understand, and analyze due to their use of continuous-valued memory vectors and observation features. We introduced a new technique, Quantized Bottleneck Insertion, to learn finite representations of these vectors and features. This helped us to create a finite-state machine representation of the policies which we show improves their interpretability. Secondly, we studied model-based reinforcement learning approaches for continuous action spaces based on tree-based planning over learned latent dynamics. We demonstrate improvement in sample efficiency and performance on a majority of challenging continuous-control benchmarks compared to the state-of-the-art methods by including look-ahead search during decision-time planning. Thirdly, we study policy evaluation over offline historical data and highlight the need to couple confidence values with the estimated policy evaluations for capturing uncertainties. Towards this, we created a benchmark to study confidence estimation by offline reinforcement learning(ORL) methods. This benchmark is derived by adding sets of policy comparison queries to datasets from ORL and comes with a set of evaluation metrics. In addition, we present an empirical evaluation of a class of model-based baselines over our benchmark. These baselines learn ensembles of dynamics models, which are used in various ways to produce simulations for answering queries with confidence values. While our results suggested advantages for certain baseline variations, there appears to be significant room for improvement in future work. *Speaker Bio:* Anurag Koul is a PostDoctoral Researcher at Microsoft Research (New York). He recently graduated with a Ph.D. from Oregon State University under the supervision of Prof. Alan Fern. His research work revolves around reinforcement learning (RL) where he has worked on explainability of RL agents, planning with latent space models, and understanding uncertainty in offline reinforcement learning. *Please watch this space for future AI Seminars :* * https://eecs.oregonstate.edu/ai-events <https://eecs.oregonstate.edu/ai-events>* Rajesh Mangannavar, Graduate Student Oregon State University ---- AI Seminar Important Reminders: -> For graduate students in the AI program, attendance is strongly encouraged.

1 0

AI seminar: October 28, 2022
by Mangannavar, Rajesh Devaraddi 24 Oct '22

24 Oct '22

Dear all, Our next AI seminar on *"Investigating Latent State and Uncertainty Representations in Reinforcement Learning" *by Anurag Koul is scheduled to be on October 28th (Friday), 1-2 PM PST. It will be followed by a 30-minute Q&A session by the graduate students. *Please note that the speaker will be on zoom for the event but it will be set up in KEC 1001 for everyone to attend.* *Investigating Latent State and Uncertainty Representations in Reinforcement Learning* Anurag Koul Postdoctoral Researcher Microsoft Research (New York) *Abstract:* Learning latent space representations of high-dimensional world states has been at the core of recent rapid growth in reinforcement learning(RL). At the same time, RL algorithms have suffered from ignored uncertainties in the predicted estimates of model-free or model-based methods. In our work, we investigate both of these aspects independently. Firstly, we studied the explainability of policies learned over latent representations. In particular, we focus on control policies represented as recurrent neural networks (RNNs) which are difficult to explain, understand, and analyze due to their use of continuous-valued memory vectors and observation features. We introduced a new technique, Quantized Bottleneck Insertion, to learn finite representations of these vectors and features. This helped us to create a finite-state machine representation of the policies which we show improves their interpretability. Secondly, we studied model-based reinforcement learning approaches for continuous action spaces based on tree-based planning over learned latent dynamics. We demonstrate improvement in sample efficiency and performance on a majority of challenging continuous-control benchmarks compared to the state-of-the-art methods by including look-ahead search during decision-time planning. Thirdly, we study policy evaluation over offline historical data and highlight the need to couple confidence values with the estimated policy evaluations for capturing uncertainties. Towards this, we created a benchmark to study confidence estimation by offline reinforcement learning(ORL) methods. This benchmark is derived by adding sets of policy comparison queries to datasets from ORL and comes with a set of evaluation metrics. In addition, we present an empirical evaluation of a class of model-based baselines over our benchmark. These baselines learn ensembles of dynamics models, which are used in various ways to produce simulations for answering queries with confidence values. While our results suggested advantages for certain baseline variations, there appears to be significant room for improvement in future work. *Speaker Bio:* Anurag Koul is a PostDoctoral Researcher at Microsoft Research (New York). He recently graduated with a Ph.D. from Oregon State University under the supervision of Prof. Alan Fern. His research work revolves around reinforcement learning (RL) where he has worked on explainability of RL agents, planning with latent space models, and understanding uncertainty in offline reinforcement learning. *Please watch this space for future AI Seminars :* * https://eecs.oregonstate.edu/ai-events <https://eecs.oregonstate.edu/ai-events>* Rajesh Mangannavar, Graduate Student Oregon State University ---- AI Seminar Important Reminders: -> For graduate students in the AI program, attendance is strongly encouraged.

1 0

AI seminar: October 21, 2022
by Mangannavar, Rajesh Devaraddi 21 Oct '22

21 Oct '22

Dear all, Our next AI seminar on *"**Vulnerability and Robustness of Linear Bandits* *" *by Huazheng Wang is scheduled to be on October 21st (Tomorrow), 1-2 PM PST. It will be followed by a 30-minute Q&A session by the graduate students. This is an *in-person* event and will be held at *KEC 1001* *Vulnerability and Robustness of Linear Bandits* Huazheng Wang Assistant Professor Computer Science Oregon State University *Abstract:* Bandit algorithm has become a reference solution for sequential decision-making problems and has been applied in many real-world scenarios such as recommender systems, display advertisements, and clinical trials. Recent works showed that multi-armed (non-contextual) bandits are vulnerable to data poisoning attack: by manipulating a small amount of rewards, an adversary could control the behavior of the bandit algorithm. However, little is known about the vulnerability of contextual bandits. In this talk, I will first answer the question "When are linear stochastic bandits attackable?". I will introduce the complete necessity and sufficiency characterization of attackability of linear stochastic bandits, which is based on the geometry of the arms' context vectors. A practical two-stage attack method is then proposed following the condition. Finally, I will talk about our new result on defending against poisoning attack inspired by the condition. *Speaker Bio:* Huazheng Wang is an assistant professor in School of Electrical Engineering and Computer Science at Oregon State University. He was a postdoctoral research associate at Princeton University from 2021 to 2022. He received his PhD in Computer Science from University of Virginia in 2021, and his B.E. from University of Science and Technology of China in 2015. His research interests include machine learning, reinforcement learning and information retrieval. His recent focus is developing efficient and robust reinforcement learning and multi-armed bandit algorithms with applications to online recommendation and ranking systems. He is a recipient of SIGIR 2019 Best Paper Award. *Please watch this space for future AI Seminars :* * https://eecs.oregonstate.edu/ai-events <https://eecs.oregonstate.edu/ai-events>* Rajesh Mangannavar, Graduate Student Oregon State University ---- AI Seminar Important Reminders: -> For graduate students in the AI program, attendance is strongly encouraged.

1 0

AI seminar: October 21, 2022
by Mangannavar, Rajesh Devaraddi 17 Oct '22

17 Oct '22

Dear all, Our next AI seminar on *"**Vulnerability and Robustness of Linear Bandits* *" *by Huazheng Wang is scheduled to be on October 21st (Friday), 1-2 PM PST. It will be followed by a 30-minute Q&A session by the graduate students. This is an *in-person* event and will be held at *KEC 1001* *Vulnerability and Robustness of Linear Bandits* Huazheng Wang Assistant Professor Computer Science Oregon State University *Abstract:* Bandit algorithm has become a reference solution for sequential decision-making problems and has been applied in many real-world scenarios such as recommender systems, display advertisements, and clinical trials. Recent works showed that multi-armed (non-contextual) bandits are vulnerable to data poisoning attack: by manipulating a small amount of rewards, an adversary could control the behavior of the bandit algorithm. However, little is known about the vulnerability of contextual bandits. In this talk, I will first answer the question "When are linear stochastic bandits attackable?". I will introduce the complete necessity and sufficiency characterization of attackability of linear stochastic bandits, which is based on the geometry of the arms' context vectors. A practical two-stage attack method is then proposed following the condition. Finally, I will talk about our new result on defending against poisoning attack inspired by the condition. *Speaker Bio:* Huazheng Wang is an assistant professor in School of Electrical Engineering and Computer Science at Oregon State University. He was a postdoctoral research associate at Princeton University from 2021 to 2022. He received his PhD in Computer Science from University of Virginia in 2021, and his B.E. from University of Science and Technology of China in 2015. His research interests include machine learning, reinforcement learning and information retrieval. His recent focus is developing efficient and robust reinforcement learning and multi-armed bandit algorithms with applications to online recommendation and ranking systems. He is a recipient of SIGIR 2019 Best Paper Award. *Please watch this space for future AI Seminars :* * https://eecs.oregonstate.edu/ai-events <https://eecs.oregonstate.edu/ai-events>* Rajesh Mangannavar, Graduate Student Oregon State University ---- AI Seminar Important Reminders: -> For graduate students in the AI program, attendance is strongly encouraged.

1 0

AI seminar: October 14, 2022
by Mangannavar, Rajesh Devaraddi 13 Oct '22

13 Oct '22

Dear all, Our next AI seminar on *"Causal Inference and Data Fusion" *by Elias Bareinboim is scheduled to be on October 14th (Tomorrow), 1-2 PM PST. It will be followed by a 30-minute Q&A session by the graduate students. *Please note that the speaker will be on zoom for the event but it will be set up in KEC 1001 for everyone to attend.* *Causal Inference and Data Fusion* Elias Bareinboim Associate Professor, Department of Computer Science Director, Causal Artificial Intelligence Laboratory Columbia University *Abstract:* Causal inference is usually dichotomized into two categories, experimental (Fisher, Cox, Cochran) and observational (Neyman, Rubin, Robins, Dawid, Pearl) which, by and large, are studied separately. Understanding reality is more demanding. Experimental and observational studies are but two extremes of a rich spectrum of research designs that generate the bulk of the data available in practical, large-scale situations. In typical medical explorations, for example, data from multiple observations and experiments are collected, coming from distinct experimental setups, different sampling conditions, and heterogeneous populations. In this talk, I will introduce the data-fusion problem, which is concerned with piecing together multiple datasets collected under heterogeneous conditions (to be defined) so as to obtain valid answers to queries of interest. The availability of multiple heterogeneous datasets presents new opportunities to causal analysts since the knowledge that can be acquired from combined data would not be possible from any individual source alone. However, the biases that emerge in heterogeneous environments require new analytical tools. Some of these biases, including confounding, sampling selection, and cross-population biases, have been addressed in isolation, largely in restricted parametric models. I will present my work on a general, non-parametric framework for handling these biases and, ultimately, a theoretical solution to the problem of fusion in causal inference tasks. Suggested readings: 1. E. Bareinboim and J. Pearl. Causal inference and the Data-Fusion Problem. <http://https//www.pnas.org/content/113/27/7345> Proceedings of the National Academy of Sciences, 113(27): 7345-7352, 2016. 2. E. Bareinboim, J. Correa, D. Ibeling, T. Icard. On Pearl’s Hierarchy and the Foundations of Causal Inference <https://causalai.net/r60.pdf>. In “Probabilistic and Causal Inference: The Works of Judea Pearl”, In Probabilistic and Causal Inference: The Works of Judea Pearl (ACM, Special Turing Series), pp. 507-556, 2022. 3. K. Xia, K. Lee, Y. Bengio, E. Bareinboim. The Causal-Neural Connection: Expressiveness, Learnability, and Inference <https://causalai.net/r80.pdf>. In Proceedings of the 35th Annual Conference on Neural Information Processing Systems (NeurIPS), 2021. *Speaker Bio:* Elias Bareinboim is an associate professor in the Department of Computer Science and the director of the Causal Artificial Intelligence (CausalAI) Laboratory at Columbia University. His research focuses on causal and counterfactual inference and their applications to data-driven fields in the health and social sciences as well as artificial intelligence and machine learning. His work was the first to propose a general solution to the problem of ``data-fusion,'' providing practical methods for combining datasets generated under different experimental conditions and plagued with various biases. More recently, Bareinboim has been exploring the intersection of causal inference with decision-making (including reinforcement learning) and explainability (including fairness analysis). Bareinboim received his Ph.D. from the University of California, Los Angeles, where he was advised by Judea Pearl. Bareinboim was named one of ``AI's 10 to Watch'' by IEEE, and is a recipient of the NSF CAREER Award, the ONR Young Investigator Award, the Dan David Prize Scholarship, the 2014 AAAI Outstanding Paper Award, and the 2019 UAI Best Paper Award. *Please watch this space for future AI Seminars :* * https://eecs.oregonstate.edu/ai-events <https://eecs.oregonstate.edu/ai-events>* Rajesh Mangannavar, Graduate Student Oregon State University ---- AI Seminar Important Reminders: -> For graduate students in the AI program, attendance is strongly encouraged.

1 0

AI seminar: October 10, 2022
by Mangannavar, Rajesh Devaraddi 10 Oct '22

10 Oct '22

Dear all, We are going to have 2 AI seminars in the coming week. The first talk will be on *"Rigorous Experimentation For Reinforcement Learning" *by Scott Jordan and is scheduled to be on October 10th (Today), 1-2 PM PST. It will be followed by a 30-minute Q&A session with the graduate students. This is an *in-person* event and will be held at *KEAR 305* *Rigorous Experimentation For Reinforcement Learning* Scott Jordan Postdoc University of Alberta *Abstract:* Scientific fields make advancements by leveraging the knowledge created by others to push the boundary of understanding. The primary tool in many fields for generating knowledge is empirical experimentation. Although common, generating accurate knowledge from empirical experiments is often challenging due to inherent randomness in execution and confounding variables that can obscure the correct interpretation of the results. As such, researchers must hold themselves and others to a high degree of rigor when designing experiments. Unfortunately, most reinforcement learning (RL) experiments lack this rigor, making the knowledge generated from experiments dubious. This dissertation proposes methods to address central issues in RL experimentation. Evaluating the performance of an RL algorithm is the most common type of experiment in RL literature. Most performance evaluations are often incapable of answering a specific research question and produce misleading results. Thus, the first issue we address is how to create a performance evaluation procedure that holds up to scientific standards. Despite the prevalence of performance evaluation, these types of experiments produce limited knowledge, e.g., they can only show how well an algorithm worked and not why, and they require significant amounts of time and computational resources. As an alternative, this dissertation proposes that scientific testing, the process of conducting carefully controlled experiments designed to further the knowledge and understanding of how an algorithm works, should be the primary form of experimentation. Lastly, this dissertation provides a case study using policy gradient methods, showing how scientific testing can replace performance evaluation as the primary form of experimentation. As a result, this dissertation can motivate others in the field to adopt more rigorous experimental practices. *Speaker Bio:* Scott Jordan got his Batchelor's degree from Oregon State in 2015 while working with Tom Dietterich. He recently got his Ph.D. from the University of Massachusetts and is now a Postdoc at the University of Alberta working with Martha White. His research focuses on reinforcement learning with the goal of understanding the properties necessary for scalable and effective sequential decision-making, with works published in ICML, NeurIPS, and other ML venues. His dissertation addresses issues with poor experimentation practices common in reinforcement learning research. This will be followed by a talk on Friday, October 14th, 2022. The second talk will be on *"Causal Inference and Data Fusion" *by Elias Bareinboim and is scheduled to be on October 14th (Friday), 1-2 PM PST. It will be followed by a 30-minute Q&A session by the graduate students. This is an *in-person* event and will be held at *KEC 1001* *Causal Inference and Data Fusion* Elias Bareinboim Associate Professor, Department of Computer Science Director, Causal Artificial Intelligence Laboratory Columbia University *Abstract:* Causal inference is usually dichotomized into two categories, experimental (Fisher, Cox, Cochran) and observational (Neyman, Rubin, Robins, Dawid, Pearl) which, by and large, are studied separately. Understanding reality is more demanding. Experimental and observational studies are but two extremes of a rich spectrum of research designs that generate the bulk of the data available in practical, large-scale situations. In typical medical explorations, for example, data from multiple observations and experiments are collected, coming from distinct experimental setups, different sampling conditions, and heterogeneous populations. In this talk, I will introduce the data-fusion problem, which is concerned with piecing together multiple datasets collected under heterogeneous conditions (to be defined) so as to obtain valid answers to queries of interest. The availability of multiple heterogeneous datasets presents new opportunities to causal analysts since the knowledge that can be acquired from combined data would not be possible from any individual source alone. However, the biases that emerge in heterogeneous environments require new analytical tools. Some of these biases, including confounding, sampling selection, and cross-population biases, have been addressed in isolation, largely in restricted parametric models. I will present my work on a general, non-parametric framework for handling these biases and, ultimately, a theoretical solution to the problem of fusion in causal inference tasks. Suggested readings: 1. E. Bareinboim and J. Pearl. Causal inference and the Data-Fusion Problem. <http://https//www.pnas.org/content/113/27/7345> Proceedings of the National Academy of Sciences, 113(27): 7345-7352, 2016. 2. E. Bareinboim, J. Correa, D. Ibeling, T. Icard. On Pearl’s Hierarchy and the Foundations of Causal Inference <https://causalai.net/r60.pdf>. In “Probabilistic and Causal Inference: The Works of Judea Pearl”, In Probabilistic and Causal Inference: The Works of Judea Pearl (ACM, Special Turing Series), pp. 507-556, 2022. 3. K. Xia, K. Lee, Y. Bengio, E. Bareinboim. The Causal-Neural Connection: Expressiveness, Learnability, and Inference <https://causalai.net/r80.pdf>. In Proceedings of the 35th Annual Conference on Neural Information Processing Systems (NeurIPS), 2021. *Speaker Bio:* Elias Bareinboim is an associate professor in the Department of Computer Science and the director of the Causal Artificial Intelligence (CausalAI) Laboratory at Columbia University. His research focuses on causal and counterfactual inference and their applications to data-driven fields in the health and social sciences as well as artificial intelligence and machine learning. His work was the first to propose a general solution to the problem of ``data-fusion,'' providing practical methods for combining datasets generated under different experimental conditions and plagued with various biases. More recently, Bareinboim has been exploring the intersection of causal inference with decision-making (including reinforcement learning) and explainability (including fairness analysis). Bareinboim received his Ph.D. from the University of California, Los Angeles, where he was advised by Judea Pearl. Bareinboim was named one of ``AI's 10 to Watch'' by IEEE, and is a recipient of the NSF CAREER Award, the ONR Young Investigator Award, the Dan David Prize Scholarship, the 2014 AAAI Outstanding Paper Award, and the 2019 UAI Best Paper Award. *Please watch this space for future AI Seminars :* * https://eecs.oregonstate.edu/ai-events <https://eecs.oregonstate.edu/ai-events>* Rajesh Mangannavar, Graduate Student Oregon State University ---- AI Seminar Important Reminders: -> For graduate students in the AI program, attendance is strongly encouraged.

1 0

AI seminar: October 7, 2022 [LOCATION CORRECTION]
by Mangannavar, Rajesh Devaraddi 07 Oct '22

07 Oct '22

Dear all, We are going to have 2 AI seminars in the coming week. The first one will be on *"**AI and O.R. for Environmental Sustainability* *" *by Bistra Dilkina and is scheduled to be on October 7th (Tomorrow), 1-2 PM PST. It will be followed by a 30-minute Q&A session with the graduate students. This is an *in-person* event and will be held at *KEC 1001 (The previous mail said this would be at a different location. Please disregard that).* *AI and O.R. for Environmental Sustainability* Bistra Dilkina Associate Professor of Computer Science Co-Director, USC Center of AI in Society University of Southern California *Abstract:* With the increasing anthropogenic pressures of urbanization, agriculture, deforestation, other socio-economic drivers as well as climate change, biodiversity and habitat conservation is a key sustainable development goal. Techniques from AI and O.R. and their hybridization have an important role to play in providing both predictive and prescriptive tools to inform critical decision-making, which can help us do more with less in this important application domain. A prime example of the field of Computational Sustainability, this presentation will give several successful examples of the two-way street of research providing useful domain solutions to real-world problems, while advancing core methodology in AI and O.R. Key examples include using deep learning and satellite data for land cover mapping, predicting species distributions under climate change and optimizing spatial conservation planning, as well as developing data-driven techniques to curb illicit wildlife poaching and trafficking. *Speaker Bio:* Dr. Bistra Dilkina is an associate professor of computer science at the University of Southern California, co-director of the USC Center of AI in Society, and the inaugural Dr. Allen and Charlotte Ginsburg Early Career Chair at the USC Viterbi School of Engineering. Her research and teaching center around the integration of machine learning and discrete optimization, with a strong focus on AI applications in computational sustainability and social good. She received her Ph.D. from Cornell University in 2012 and was a post-doctoral associate at the Institute for Computational Sustainability. Her research has contributed significant advances to machine-learning-guided combinatorial solving including mathematical programming and planning, as well as decision-focused learning where combinatorial reasoning is integrated in machine learning pipelines. Her applied research in Computational Sustainability spans using AI for wildlife conservation planning, using AI to understand the impacts of climate change in terms of energy, water, habitat and human migration, and using AI to optimize fortification of lifeline infrastructures for disaster resilience. She has over 90 publications and has co-organized or served as a chair to numerous workshops, tutorials, and special tracks at major conferences. This will be followed by a talk on Monday, October 10th, 2022. The second talk will be on *"Rigorous Experimentation For Reinforcement Learning" *by Scott Jordan and is scheduled to be on October 10th, 1-2 PM PST. It will be followed by a 30-minute Q&A session with the graduate students. This is an *in-person* event and will be held at *KEAR 305* *Rigorous Experimentation For Reinforcement Learning* Scott Jordan Postdoc University of Alberta *Abstract:* Scientific fields make advancements by leveraging the knowledge created by others to push the boundary of understanding. The primary tool in many fields for generating knowledge is empirical experimentation. Although common, generating accurate knowledge from empirical experiments is often challenging due to inherent randomness in execution and confounding variables that can obscure the correct interpretation of the results. As such, researchers must hold themselves and others to a high degree of rigor when designing experiments. Unfortunately, most reinforcement learning (RL) experiments lack this rigor, making the knowledge generated from experiments dubious. This dissertation proposes methods to address central issues in RL experimentation. Evaluating the performance of an RL algorithm is the most common type of experiment in RL literature. Most performance evaluations are often incapable of answering a specific research question and produce misleading results. Thus, the first issue we address is how to create a performance evaluation procedure that holds up to scientific standards. Despite the prevalence of performance evaluation, these types of experiments produce limited knowledge, e.g., they can only show how well an algorithm worked and not why, and they require significant amounts of time and computational resources. As an alternative, this dissertation proposes that scientific testing, the process of conducting carefully controlled experiments designed to further the knowledge and understanding of how an algorithm works, should be the primary form of experimentation. Lastly, this dissertation provides a case study using policy gradient methods, showing how scientific testing can replace performance evaluation as the primary form of experimentation. As a result, this dissertation can motivate others in the field to adopt more rigorous experimental practices. *Speaker Bio:* Scott Jordan got his Batchelor's degree from Oregon State in 2015 while working with Tom Dietterich. He recently got his Ph.D. from the University of Massachusetts and is now a Postdoc at the University of Alberta working with Martha White. His research focuses on reinforcement learning with the goal of understanding the properties necessary for scalable and effective sequential decision-making, with works published in ICML, NeurIPS, and other ML venues. His dissertation addresses issues with poor experimentation practices common in reinforcement learning research. *Please watch this space for future AI Seminars :* * https://eecs.oregonstate.edu/ai-events <https://eecs.oregonstate.edu/ai-events>* Rajesh Mangannavar, Graduate Student Oregon State University ---- AI Seminar Important Reminders: -> For graduate students in the AI program, attendance is strongly encouraged.

1 0

AI seminar: October 7, 2022
by Mangannavar, Rajesh Devaraddi 06 Oct '22

06 Oct '22

Dear all, We are going to have 2 AI seminars in the coming week. The first one will be on *"**AI and O.R. for Environmental Sustainability* *" *by Bistra Dilkina is scheduled to be on October 7th (Tomorrow), 1-2 PM PST. It will be followed by a 30-minute Q&A session with the graduate students. This is an *in-person* event and will be held at *KEAR 1001* *AI and O.R. for Environmental Sustainability* Bistra Dilkina Associate Professor of Computer Science Co-Director, USC Center of AI in Society University of Southern California *Abstract:* With the increasing anthropogenic pressures of urbanization, agriculture, deforestation, other socio-economic drivers as well as climate change, biodiversity and habitat conservation is a key sustainable development goal. Techniques from AI and O.R. and their hybridization have an important role to play in providing both predictive and prescriptive tools to inform critical decision-making, which can help us do more with less in this important application domain. A prime example of the field of Computational Sustainability, this presentation will give several successful examples of the two-way street of research providing useful domain solutions to real-world problems, while advancing core methodology in AI and O.R. Key examples include using deep learning and satellite data for land cover mapping, predicting species distributions under climate change and optimizing spatial conservation planning, as well as developing data-driven techniques to curb illicit wildlife poaching and trafficking. *Speaker Bio:* Dr. Bistra Dilkina is an associate professor of computer science at the University of Southern California, co-director of the USC Center of AI in Society, and the inaugural Dr. Allen and Charlotte Ginsburg Early Career Chair at the USC Viterbi School of Engineering. Her research and teaching center around the integration of machine learning and discrete optimization, with a strong focus on AI applications in computational sustainability and social good. She received her Ph.D. from Cornell University in 2012 and was a post-doctoral associate at the Institute for Computational Sustainability. Her research has contributed significant advances to machine-learning-guided combinatorial solving including mathematical programming and planning, as well as decision-focused learning where combinatorial reasoning is integrated in machine learning pipelines. Her applied research in Computational Sustainability spans using AI for wildlife conservation planning, using AI to understand the impacts of climate change in terms of energy, water, habitat and human migration, and using AI to optimize fortification of lifeline infrastructures for disaster resilience. She has over 90 publications and has co-organized or served as a chair to numerous workshops, tutorials, and special tracks at major conferences. This will be followed by a talk on Monday, October 10th, 2022. The second talk will be on *"Rigorous Experimentation For Reinforcement Learning" *by Scott Jordan is scheduled to be on October 10th, 1-2 PM PST. It will be followed by a 30-minute Q&A session with the graduate students. This is an *in-person* event and will be held at *KEC 305* *Rigorous Experimentation For Reinforcement Learning* Scott Jordan Postdoc University of Alberta *Abstract:* Scientific fields make advancements by leveraging the knowledge created by others to push the boundary of understanding. The primary tool in many fields for generating knowledge is empirical experimentation. Although common, generating accurate knowledge from empirical experiments is often challenging due to inherent randomness in execution and confounding variables that can obscure the correct interpretation of the results. As such, researchers must hold themselves and others to a high degree of rigor when designing experiments. Unfortunately, most reinforcement learning (RL) experiments lack this rigor, making the knowledge generated from experiments dubious. This dissertation proposes methods to address central issues in RL experimentation. Evaluating the performance of an RL algorithm is the most common type of experiment in RL literature. Most performance evaluations are often incapable of answering a specific research question and produce misleading results. Thus, the first issue we address is how to create a performance evaluation procedure that holds up to scientific standards. Despite the prevalence of performance evaluation, these types of experiments produce limited knowledge, e.g., they can only show how well an algorithm worked and not why, and they require significant amounts of time and computational resources. As an alternative, this dissertation proposes that scientific testing, the process of conducting carefully controlled experiments designed to further the knowledge and understanding of how an algorithm works, should be the primary form of experimentation. Lastly, this dissertation provides a case study using policy gradient methods, showing how scientific testing can replace performance evaluation as the primary form of experimentation. As a result, this dissertation can motivate others in the field to adopt more rigorous experimental practices. *Speaker Bio:* Scott Jordan got his Batchelor's degree from Oregon State in 2015 while working with Tom Dietterich. He recently got his Ph.D. from the University of Massachusetts and is now a Postdoc at the University of Alberta working with Martha White. His research focuses on reinforcement learning with the goal of understanding the properties necessary for scalable and effective sequential decision-making, with works published in ICML, NeurIPS, and other ML venues. His dissertation addresses issues with poor experimentation practices common in reinforcement learning research. *Please watch this space for future AI Seminars :* * https://eecs.oregonstate.edu/ai-events <https://eecs.oregonstate.edu/ai-events>* Rajesh Mangannavar, Graduate Student Oregon State University ---- AI Seminar Important Reminders: -> For graduate students in the AI program, attendance is strongly encouraged.

1 0

AI seminar: October 7, 2022
by Mangannavar, Rajesh Devaraddi 03 Oct '22

03 Oct '22

Dear all, Our next AI seminar on *"**AI and O.R. for Environmental Sustainability**" *by Bistra Dilkina is scheduled to be on October 7th (Friday), 1-2 PM PST. It will be followed by a 30-minute Q&A session by the graduate students. This is an *in-person* event and will be held at *KEC 1001* *AI and O.R. for Environmental Sustainability* Bistra Dilkina Associate Professor of Computer Science Co-Director, USC Center of AI in Society University of Southern California *Abstract:* With the increasing anthropogenic pressures of urbanization, agriculture, deforestation, other socio-economic drivers as well as climate change, biodiversity and habitat conservation is a key sustainable development goal. Techniques from AI and O.R. and their hybridization have an important role to play in providing both predictive and prescriptive tools to inform critical decision-making, which can help us do more with less in this important application domain. A prime example of the field of Computational Sustainability, this presentation will give several successful examples of the two-way street of research providing useful domain solutions to real-world problems, while advancing core methodology in AI and O.R. Key examples include using deep learning and satellite data for land cover mapping, predicting species distributions under climate change and optimizing spatial conservation planning, as well as developing data-driven techniques to curb illicit wildlife poaching and trafficking. *Speaker Bio:* Dr. Bistra Dilkina is an associate professor of computer science at the University of Southern California, co-director of the USC Center of AI in Society, and the inaugural Dr. Allen and Charlotte Ginsburg Early Career Chair at the USC Viterbi School of Engineering. Her research and teaching center around the integration of machine learning and discrete optimization, with a strong focus on AI applications in computational sustainability and social good. She received her Ph.D. from Cornell University in 2012 and was a post-doctoral associate at the Institute for Computational Sustainability. Her research has contributed significant advances to machine-learning-guided combinatorial solving including mathematical programming and planning, as well as decision-focused learning where combinatorial reasoning is integrated in machine learning pipelines. Her applied research in Computational Sustainability spans using AI for wildlife conservation planning, using AI to understand the impacts of climate change in terms of energy, water, habitat and human migration, and using AI to optimize fortification of lifeline infrastructures for disaster resilience. She has over 90 publications and has co-organized or served as a chair to numerous workshops, tutorials, and special tracks at major conferences. *Please watch this space for future AI Seminars :* * https://eecs.oregonstate.edu/ai-events <https://eecs.oregonstate.edu/ai-events>* Rajesh Mangannavar, Graduate Student Oregon State University ---- AI Seminar Important Reminders: -> For graduate students in the AI program, attendance is strongly encouraged.

1 0