Dear all,
Our next AI seminar on *"Investigating Latent State and Uncertainty
Representations in Reinforcement Learning" *by Anurag Koul
is scheduled to be on October 28th (Tomorrow), 1-2 PM PST. It will be
followed by a 30-minute Q&A session by the graduate students.
*Please note that the speaker will be on zoom for the event but it will be
set up in KEC 1001 for everyone to attend.*
*Investigating Latent State and Uncertainty Representations in
Reinforcement Learning*
Anurag Koul
Postdoctoral Researcher
Microsoft Research (New York)
*Abstract:*
Learning latent space representations of high-dimensional world states has
been at the core of recent rapid growth in reinforcement learning(RL). At
the same time, RL algorithms have suffered from ignored uncertainties in
the predicted estimates of model-free or model-based methods. In our work,
we investigate both of these aspects independently.
Firstly, we studied the explainability of policies learned over latent
representations. In particular, we focus on control policies represented as
recurrent neural networks (RNNs) which are difficult to explain,
understand, and analyze due to their use of continuous-valued memory
vectors and observation features. We introduced a new technique, Quantized
Bottleneck Insertion, to learn finite representations of these vectors and
features. This helped us to create a finite-state machine representation of
the policies which we show improves their interpretability.
Secondly, we studied model-based reinforcement learning approaches for
continuous action spaces based on tree-based planning over learned latent
dynamics. We demonstrate improvement in sample efficiency and performance
on a majority of challenging continuous-control benchmarks compared to the
state-of-the-art methods by including look-ahead search during
decision-time planning.
Thirdly, we study policy evaluation over offline historical data and
highlight the need to couple confidence values with the estimated policy
evaluations for capturing uncertainties. Towards this, we created a
benchmark to study confidence estimation by offline reinforcement
learning(ORL) methods. This benchmark is derived by adding sets of policy
comparison queries to datasets from ORL and comes with a set of evaluation
metrics. In addition, we present an empirical evaluation of a class of
model-based baselines over our benchmark. These baselines learn ensembles
of dynamics models, which are used in various ways to produce simulations
for answering queries with confidence values. While our results suggested
advantages for certain baseline variations, there appears to be significant
room for improvement in future work.
*Speaker Bio:*
Anurag Koul is a PostDoctoral Researcher at Microsoft Research (New York).
He recently graduated with a Ph.D. from Oregon State University under the
supervision of Prof. Alan Fern. His research work revolves around
reinforcement learning (RL) where he has worked on explainability of RL
agents, planning with latent space models, and understanding uncertainty in
offline reinforcement learning.
*Please watch this space for future AI Seminars :*
* https://eecs.oregonstate.edu/ai-events
<https://eecs.oregonstate.edu/ai-events>*
Rajesh Mangannavar,
Graduate Student
Oregon State University
----
AI Seminar Important Reminders:
-> For graduate students in the AI program, attendance is strongly
encouraged.
Dear all,
Our next AI seminar on *"Investigating Latent State and Uncertainty
Representations in Reinforcement Learning" *by Anurag Koul
is scheduled to be on October 28th (Friday), 1-2 PM PST. It will be
followed by a 30-minute Q&A session by the graduate students.
*Please note that the speaker will be on zoom for the event but it will be
set up in KEC 1001 for everyone to attend.*
*Investigating Latent State and Uncertainty Representations in
Reinforcement Learning*
Anurag Koul
Postdoctoral Researcher
Microsoft Research (New York)
*Abstract:*
Learning latent space representations of high-dimensional world states has
been at the core of recent rapid growth in reinforcement learning(RL). At
the same time, RL algorithms have suffered from ignored uncertainties in
the predicted estimates of model-free or model-based methods. In our work,
we investigate both of these aspects independently.
Firstly, we studied the explainability of policies learned over latent
representations. In particular, we focus on control policies represented as
recurrent neural networks (RNNs) which are difficult to explain,
understand, and analyze due to their use of continuous-valued memory
vectors and observation features. We introduced a new technique, Quantized
Bottleneck Insertion, to learn finite representations of these vectors and
features. This helped us to create a finite-state machine representation of
the policies which we show improves their interpretability.
Secondly, we studied model-based reinforcement learning approaches for
continuous action spaces based on tree-based planning over learned latent
dynamics. We demonstrate improvement in sample efficiency and performance
on a majority of challenging continuous-control benchmarks compared to the
state-of-the-art methods by including look-ahead search during
decision-time planning.
Thirdly, we study policy evaluation over offline historical data and
highlight the need to couple confidence values with the estimated policy
evaluations for capturing uncertainties. Towards this, we created a
benchmark to study confidence estimation by offline reinforcement
learning(ORL) methods. This benchmark is derived by adding sets of policy
comparison queries to datasets from ORL and comes with a set of evaluation
metrics. In addition, we present an empirical evaluation of a class of
model-based baselines over our benchmark. These baselines learn ensembles
of dynamics models, which are used in various ways to produce simulations
for answering queries with confidence values. While our results suggested
advantages for certain baseline variations, there appears to be significant
room for improvement in future work.
*Speaker Bio:*
Anurag Koul is a PostDoctoral Researcher at Microsoft Research (New York).
He recently graduated with a Ph.D. from Oregon State University under the
supervision of Prof. Alan Fern. His research work revolves around
reinforcement learning (RL) where he has worked on explainability of RL
agents, planning with latent space models, and understanding uncertainty in
offline reinforcement learning.
*Please watch this space for future AI Seminars :*
* https://eecs.oregonstate.edu/ai-events
<https://eecs.oregonstate.edu/ai-events>*
Rajesh Mangannavar,
Graduate Student
Oregon State University
----
AI Seminar Important Reminders:
-> For graduate students in the AI program, attendance is strongly
encouraged.
Dear all,
Our next AI seminar on *"**Vulnerability and Robustness of Linear Bandits*
*" *by Huazheng Wang is scheduled to be on October 21st (Tomorrow), 1-2 PM
PST. It will be followed by a 30-minute Q&A session by the graduate
students.
This is an *in-person* event and will be held at *KEC 1001*
*Vulnerability and Robustness of Linear Bandits*
Huazheng Wang
Assistant Professor
Computer Science
Oregon State University
*Abstract:*
Bandit algorithm has become a reference solution for sequential
decision-making problems and has been applied in many real-world scenarios
such as recommender systems, display advertisements, and clinical trials.
Recent works showed that multi-armed (non-contextual) bandits are
vulnerable to data poisoning attack: by manipulating a small amount of
rewards, an adversary could control the behavior of the bandit algorithm.
However, little is known about the vulnerability of contextual bandits. In
this talk, I will first answer the question "When are linear stochastic
bandits attackable?". I will introduce the complete necessity and
sufficiency characterization of attackability of linear stochastic bandits,
which is based on the geometry of the arms' context vectors. A practical
two-stage attack method is then proposed following the condition. Finally,
I will talk about our new result on defending against poisoning attack
inspired by the condition.
*Speaker Bio:*
Huazheng Wang is an assistant professor in School of Electrical Engineering
and Computer Science at Oregon State University. He was a postdoctoral
research associate at Princeton University from 2021 to 2022. He received
his PhD in Computer Science from University of Virginia in 2021, and his
B.E. from University of Science and Technology of China in 2015. His
research interests include machine learning, reinforcement learning and
information retrieval. His recent focus is developing efficient and robust
reinforcement learning and multi-armed bandit algorithms with applications
to online recommendation and ranking systems. He is a recipient of SIGIR
2019 Best Paper Award.
*Please watch this space for future AI Seminars :*
* https://eecs.oregonstate.edu/ai-events
<https://eecs.oregonstate.edu/ai-events>*
Rajesh Mangannavar,
Graduate Student
Oregon State University
----
AI Seminar Important Reminders:
-> For graduate students in the AI program, attendance is strongly
encouraged.
Dear all,
Our next AI seminar on *"**Vulnerability and Robustness of Linear Bandits*
*" *by Huazheng Wang is scheduled to be on October 21st (Friday), 1-2 PM
PST. It will be followed by a 30-minute Q&A session by the graduate
students.
This is an *in-person* event and will be held at *KEC 1001*
*Vulnerability and Robustness of Linear Bandits*
Huazheng Wang
Assistant Professor
Computer Science
Oregon State University
*Abstract:*
Bandit algorithm has become a reference solution for sequential
decision-making problems and has been applied in many real-world scenarios
such as recommender systems, display advertisements, and clinical trials.
Recent works showed that multi-armed (non-contextual) bandits are
vulnerable to data poisoning attack: by manipulating a small amount of
rewards, an adversary could control the behavior of the bandit algorithm.
However, little is known about the vulnerability of contextual bandits. In
this talk, I will first answer the question "When are linear stochastic
bandits attackable?". I will introduce the complete necessity and
sufficiency characterization of attackability of linear stochastic bandits,
which is based on the geometry of the arms' context vectors. A practical
two-stage attack method is then proposed following the condition. Finally,
I will talk about our new result on defending against poisoning attack
inspired by the condition.
*Speaker Bio:*
Huazheng Wang is an assistant professor in School of Electrical Engineering
and Computer Science at Oregon State University. He was a postdoctoral
research associate at Princeton University from 2021 to 2022. He received
his PhD in Computer Science from University of Virginia in 2021, and his
B.E. from University of Science and Technology of China in 2015. His
research interests include machine learning, reinforcement learning and
information retrieval. His recent focus is developing efficient and robust
reinforcement learning and multi-armed bandit algorithms with applications
to online recommendation and ranking systems. He is a recipient of SIGIR
2019 Best Paper Award.
*Please watch this space for future AI Seminars :*
* https://eecs.oregonstate.edu/ai-events
<https://eecs.oregonstate.edu/ai-events>*
Rajesh Mangannavar,
Graduate Student
Oregon State University
----
AI Seminar Important Reminders:
-> For graduate students in the AI program, attendance is strongly
encouraged.
Dear all,
Our next AI seminar on *"Causal Inference and Data Fusion" *by Elias
Bareinboim
is scheduled to be on October 14th (Tomorrow), 1-2 PM PST. It will be
followed by a 30-minute Q&A session by the graduate students.
*Please note that the speaker will be on zoom for the event but it will be
set up in KEC 1001 for everyone to attend.*
*Causal Inference and Data Fusion*
Elias Bareinboim
Associate Professor, Department of Computer Science
Director, Causal Artificial Intelligence Laboratory
Columbia University
*Abstract:*
Causal inference is usually dichotomized into two categories, experimental
(Fisher, Cox, Cochran) and observational (Neyman, Rubin, Robins, Dawid,
Pearl) which, by and large, are studied separately. Understanding reality
is more demanding. Experimental and observational studies are but two
extremes of a rich spectrum of research designs that generate the bulk of
the data available in practical, large-scale situations. In typical medical
explorations, for example, data from multiple observations and experiments
are collected, coming from distinct experimental setups, different sampling
conditions, and heterogeneous populations.
In this talk, I will introduce the data-fusion problem, which is concerned
with piecing together multiple datasets collected under heterogeneous
conditions (to be defined) so as to obtain valid answers to queries of
interest. The availability of multiple heterogeneous datasets presents new
opportunities to causal analysts since the knowledge that can be acquired
from combined data would not be possible from any individual source alone.
However, the biases that emerge in heterogeneous environments require new
analytical tools. Some of these biases, including confounding, sampling
selection, and cross-population biases, have been addressed in isolation,
largely in restricted parametric models. I will present my work on a
general, non-parametric framework for handling these biases and,
ultimately, a theoretical solution to the problem of fusion in causal
inference tasks.
Suggested readings:
1. E. Bareinboim and J. Pearl. Causal inference and the Data-Fusion Problem.
<http://https//www.pnas.org/content/113/27/7345> Proceedings of the
National Academy of Sciences, 113(27): 7345-7352, 2016.
2. E. Bareinboim, J. Correa, D. Ibeling, T. Icard. On Pearl’s Hierarchy and
the Foundations of Causal Inference <https://causalai.net/r60.pdf>. In
“Probabilistic and Causal Inference: The Works of Judea Pearl”, In
Probabilistic and Causal Inference: The Works of Judea Pearl (ACM, Special
Turing Series), pp. 507-556, 2022.
3. K. Xia, K. Lee, Y. Bengio, E. Bareinboim. The Causal-Neural Connection:
Expressiveness, Learnability, and Inference <https://causalai.net/r80.pdf>.
In Proceedings of the 35th Annual Conference on Neural Information
Processing Systems (NeurIPS), 2021.
*Speaker Bio:*
Elias Bareinboim is an associate professor in the Department of Computer
Science and the director of the Causal Artificial Intelligence (CausalAI)
Laboratory at Columbia University. His research focuses on causal and
counterfactual inference and their applications to data-driven fields in
the health and social sciences as well as artificial intelligence and
machine learning. His work was the first to propose a general solution to
the problem of ``data-fusion,'' providing practical methods for combining
datasets generated under different experimental conditions and plagued with
various biases. More recently, Bareinboim has been exploring the
intersection of causal inference with decision-making (including
reinforcement learning) and explainability (including fairness analysis).
Bareinboim received his Ph.D. from the University of California, Los
Angeles, where he was advised by Judea Pearl. Bareinboim was named one of
``AI's 10 to Watch'' by IEEE, and is a recipient of the NSF CAREER Award,
the ONR Young Investigator Award, the Dan David Prize Scholarship, the 2014
AAAI Outstanding Paper Award, and the 2019 UAI Best Paper Award.
*Please watch this space for future AI Seminars :*
* https://eecs.oregonstate.edu/ai-events
<https://eecs.oregonstate.edu/ai-events>*
Rajesh Mangannavar,
Graduate Student
Oregon State University
----
AI Seminar Important Reminders:
-> For graduate students in the AI program, attendance is strongly
encouraged.
Dear all,
We are going to have 2 AI seminars in the coming week.
The first talk will be on *"Rigorous Experimentation For Reinforcement
Learning" *by Scott Jordan and is scheduled to be on October 10th (Today),
1-2 PM PST. It will be followed by a 30-minute Q&A session with the
graduate students.
This is an *in-person* event and will be held at *KEAR 305*
*Rigorous Experimentation For Reinforcement Learning*
Scott Jordan
Postdoc
University of Alberta
*Abstract:*
Scientific fields make advancements by leveraging the knowledge created by
others to push the boundary of understanding. The primary tool in many
fields for generating knowledge is empirical experimentation. Although
common, generating accurate knowledge from empirical experiments is often
challenging due to inherent randomness in execution and confounding
variables that can obscure the correct interpretation of the results. As
such, researchers must hold themselves and others to a high degree of rigor
when designing experiments. Unfortunately, most reinforcement learning (RL)
experiments lack this rigor, making the knowledge generated from
experiments dubious. This dissertation proposes methods to address central
issues in RL experimentation.
Evaluating the performance of an RL algorithm is the most common type of
experiment in RL literature. Most performance evaluations are often
incapable of answering a specific research question and produce misleading
results. Thus, the first issue we address is how to create a performance
evaluation procedure that holds up to scientific standards.
Despite the prevalence of performance evaluation, these types of
experiments produce limited knowledge, e.g., they can only show how well an
algorithm worked and not why, and they require significant amounts of time
and computational resources. As an alternative, this dissertation proposes
that scientific testing, the process of conducting carefully controlled
experiments designed to further the knowledge and understanding of how an
algorithm works, should be the primary form of experimentation.
Lastly, this dissertation provides a case study using policy gradient
methods, showing how scientific testing can replace performance evaluation
as the primary form of experimentation. As a result, this dissertation can
motivate others in the field to adopt more rigorous experimental practices.
*Speaker Bio:*
Scott Jordan got his Batchelor's degree from Oregon State in 2015 while
working with Tom Dietterich. He recently got his Ph.D. from the University
of Massachusetts and is now a Postdoc at the University of Alberta working
with Martha White. His research focuses on reinforcement learning with the
goal of understanding the properties necessary for scalable and effective
sequential decision-making, with works published in ICML, NeurIPS, and
other ML venues. His dissertation addresses issues with poor
experimentation practices common in reinforcement learning research.
This will be followed by a talk on Friday, October 14th, 2022.
The second talk will be on *"Causal Inference and Data Fusion" *by Elias
Bareinboim and is scheduled to be on October 14th (Friday), 1-2 PM PST. It
will be followed by a 30-minute Q&A session by the graduate students.
This is an *in-person* event and will be held at *KEC 1001*
*Causal Inference and Data Fusion*
Elias Bareinboim
Associate Professor, Department of Computer Science
Director, Causal Artificial Intelligence Laboratory
Columbia University
*Abstract:*
Causal inference is usually dichotomized into two categories, experimental
(Fisher, Cox, Cochran) and observational (Neyman, Rubin, Robins, Dawid,
Pearl) which, by and large, are studied separately. Understanding reality
is more demanding. Experimental and observational studies are but two
extremes of a rich spectrum of research designs that generate the bulk of
the data available in practical, large-scale situations. In typical medical
explorations, for example, data from multiple observations and experiments
are collected, coming from distinct experimental setups, different sampling
conditions, and heterogeneous populations.
In this talk, I will introduce the data-fusion problem, which is concerned
with piecing together multiple datasets collected under heterogeneous
conditions (to be defined) so as to obtain valid answers to queries of
interest. The availability of multiple heterogeneous datasets presents new
opportunities to causal analysts since the knowledge that can be acquired
from combined data would not be possible from any individual source alone.
However, the biases that emerge in heterogeneous environments require new
analytical tools. Some of these biases, including confounding, sampling
selection, and cross-population biases, have been addressed in isolation,
largely in restricted parametric models. I will present my work on a
general, non-parametric framework for handling these biases and,
ultimately, a theoretical solution to the problem of fusion in causal
inference tasks.
Suggested readings:
1. E. Bareinboim and J. Pearl. Causal inference and the Data-Fusion Problem.
<http://https//www.pnas.org/content/113/27/7345> Proceedings of the
National Academy of Sciences, 113(27): 7345-7352, 2016.
2. E. Bareinboim, J. Correa, D. Ibeling, T. Icard. On Pearl’s Hierarchy and
the Foundations of Causal Inference <https://causalai.net/r60.pdf>. In
“Probabilistic and Causal Inference: The Works of Judea Pearl”, In
Probabilistic and Causal Inference: The Works of Judea Pearl (ACM, Special
Turing Series), pp. 507-556, 2022.
3. K. Xia, K. Lee, Y. Bengio, E. Bareinboim. The Causal-Neural Connection:
Expressiveness, Learnability, and Inference <https://causalai.net/r80.pdf>.
In Proceedings of the 35th Annual Conference on Neural Information
Processing Systems (NeurIPS), 2021.
*Speaker Bio:*
Elias Bareinboim is an associate professor in the Department of Computer
Science and the director of the Causal Artificial Intelligence (CausalAI)
Laboratory at Columbia University. His research focuses on causal and
counterfactual inference and their applications to data-driven fields in
the health and social sciences as well as artificial intelligence and
machine learning. His work was the first to propose a general solution to
the problem of ``data-fusion,'' providing practical methods for combining
datasets generated under different experimental conditions and plagued with
various biases. More recently, Bareinboim has been exploring the
intersection of causal inference with decision-making (including
reinforcement learning) and explainability (including fairness analysis).
Bareinboim received his Ph.D. from the University of California, Los
Angeles, where he was advised by Judea Pearl. Bareinboim was named one of
``AI's 10 to Watch'' by IEEE, and is a recipient of the NSF CAREER Award,
the ONR Young Investigator Award, the Dan David Prize Scholarship, the 2014
AAAI Outstanding Paper Award, and the 2019 UAI Best Paper Award.
*Please watch this space for future AI Seminars :*
* https://eecs.oregonstate.edu/ai-events
<https://eecs.oregonstate.edu/ai-events>*
Rajesh Mangannavar,
Graduate Student
Oregon State University
----
AI Seminar Important Reminders:
-> For graduate students in the AI program, attendance is strongly
encouraged.
Dear all,
We are going to have 2 AI seminars in the coming week.
The first one will be on *"**AI and O.R. for Environmental Sustainability*
*" *by Bistra Dilkina and is scheduled to be on October 7th (Tomorrow), 1-2
PM PST. It will be followed by a 30-minute Q&A session with the graduate
students.
This is an *in-person* event and will be held at *KEC 1001 (The previous
mail said this would be at a different location. Please disregard that).*
*AI and O.R. for Environmental Sustainability*
Bistra Dilkina
Associate Professor of Computer Science
Co-Director, USC Center of AI in Society
University of Southern California
*Abstract:*
With the increasing anthropogenic pressures of urbanization, agriculture,
deforestation, other socio-economic drivers as well as climate change,
biodiversity and habitat conservation is a key sustainable development
goal. Techniques from AI and O.R. and their hybridization have an important
role to play in providing both predictive and prescriptive tools to inform
critical decision-making, which can help us do more with less in this
important application domain. A prime example of the field of Computational
Sustainability, this presentation will give several successful examples of
the two-way street of research providing useful domain solutions to
real-world problems, while advancing core methodology in AI and O.R. Key
examples include using deep learning and satellite data for land cover
mapping, predicting species distributions under climate change and
optimizing spatial conservation planning, as well as developing data-driven
techniques to curb illicit wildlife poaching and trafficking.
*Speaker Bio:*
Dr. Bistra Dilkina is an associate professor of computer science at the
University of Southern California, co-director of the USC Center of AI in
Society, and the inaugural Dr. Allen and Charlotte Ginsburg Early Career
Chair at the USC Viterbi School of Engineering. Her research and teaching
center around the integration of machine learning and discrete
optimization, with a strong focus on AI applications in computational
sustainability and social good. She received her Ph.D. from Cornell
University in 2012 and was a post-doctoral associate at the Institute for
Computational Sustainability. Her research has contributed significant
advances to machine-learning-guided combinatorial solving including
mathematical programming and planning, as well as decision-focused learning
where combinatorial reasoning is integrated in machine learning pipelines.
Her applied research in Computational Sustainability spans using AI for
wildlife conservation planning, using AI to understand the impacts of
climate change in terms of energy, water, habitat and human migration, and
using AI to optimize fortification of lifeline infrastructures for disaster
resilience. She has over 90 publications and has co-organized or served as
a chair to numerous workshops, tutorials, and special tracks at major
conferences.
This will be followed by a talk on Monday, October 10th, 2022.
The second talk will be on *"Rigorous Experimentation For Reinforcement
Learning" *by Scott Jordan and is scheduled to be on October 10th, 1-2 PM
PST. It will be followed by a 30-minute Q&A session with the graduate
students.
This is an *in-person* event and will be held at *KEAR 305*
*Rigorous Experimentation For Reinforcement Learning*
Scott Jordan
Postdoc
University of Alberta
*Abstract:*
Scientific fields make advancements by leveraging the knowledge created by
others to push the boundary of understanding. The primary tool in many
fields for generating knowledge is empirical experimentation. Although
common, generating accurate knowledge from empirical experiments is often
challenging due to inherent randomness in execution and confounding
variables that can obscure the correct interpretation of the results. As
such, researchers must hold themselves and others to a high degree of rigor
when designing experiments. Unfortunately, most reinforcement learning (RL)
experiments lack this rigor, making the knowledge generated from
experiments dubious. This dissertation proposes methods to address central
issues in RL experimentation.
Evaluating the performance of an RL algorithm is the most common type of
experiment in RL literature. Most performance evaluations are often
incapable of answering a specific research question and produce misleading
results. Thus, the first issue we address is how to create a performance
evaluation procedure that holds up to scientific standards.
Despite the prevalence of performance evaluation, these types of
experiments produce limited knowledge, e.g., they can only show how well an
algorithm worked and not why, and they require significant amounts of time
and computational resources. As an alternative, this dissertation proposes
that scientific testing, the process of conducting carefully controlled
experiments designed to further the knowledge and understanding of how an
algorithm works, should be the primary form of experimentation.
Lastly, this dissertation provides a case study using policy gradient
methods, showing how scientific testing can replace performance evaluation
as the primary form of experimentation. As a result, this dissertation can
motivate others in the field to adopt more rigorous experimental practices.
*Speaker Bio:*
Scott Jordan got his Batchelor's degree from Oregon State in 2015 while
working with Tom Dietterich. He recently got his Ph.D. from the University
of Massachusetts and is now a Postdoc at the University of Alberta working
with Martha White. His research focuses on reinforcement learning with the
goal of understanding the properties necessary for scalable and effective
sequential decision-making, with works published in ICML, NeurIPS, and
other ML venues. His dissertation addresses issues with poor
experimentation practices common in reinforcement learning research.
*Please watch this space for future AI Seminars :*
* https://eecs.oregonstate.edu/ai-events
<https://eecs.oregonstate.edu/ai-events>*
Rajesh Mangannavar,
Graduate Student
Oregon State University
----
AI Seminar Important Reminders:
-> For graduate students in the AI program, attendance is strongly
encouraged.
Dear all,
We are going to have 2 AI seminars in the coming week.
The first one will be on *"**AI and O.R. for Environmental Sustainability*
*" *by Bistra Dilkina is scheduled to be on October 7th (Tomorrow), 1-2 PM
PST. It will be followed by a 30-minute Q&A session with the graduate
students.
This is an *in-person* event and will be held at *KEAR 1001*
*AI and O.R. for Environmental Sustainability*
Bistra Dilkina
Associate Professor of Computer Science
Co-Director, USC Center of AI in Society
University of Southern California
*Abstract:*
With the increasing anthropogenic pressures of urbanization, agriculture,
deforestation, other socio-economic drivers as well as climate change,
biodiversity and habitat conservation is a key sustainable development
goal. Techniques from AI and O.R. and their hybridization have an important
role to play in providing both predictive and prescriptive tools to inform
critical decision-making, which can help us do more with less in this
important application domain. A prime example of the field of Computational
Sustainability, this presentation will give several successful examples of
the two-way street of research providing useful domain solutions to
real-world problems, while advancing core methodology in AI and O.R. Key
examples include using deep learning and satellite data for land cover
mapping, predicting species distributions under climate change and
optimizing spatial conservation planning, as well as developing data-driven
techniques to curb illicit wildlife poaching and trafficking.
*Speaker Bio:*
Dr. Bistra Dilkina is an associate professor of computer science at the
University of Southern California, co-director of the USC Center of AI in
Society, and the inaugural Dr. Allen and Charlotte Ginsburg Early Career
Chair at the USC Viterbi School of Engineering. Her research and teaching
center around the integration of machine learning and discrete
optimization, with a strong focus on AI applications in computational
sustainability and social good. She received her Ph.D. from Cornell
University in 2012 and was a post-doctoral associate at the Institute for
Computational Sustainability. Her research has contributed significant
advances to machine-learning-guided combinatorial solving including
mathematical programming and planning, as well as decision-focused learning
where combinatorial reasoning is integrated in machine learning pipelines.
Her applied research in Computational Sustainability spans using AI for
wildlife conservation planning, using AI to understand the impacts of
climate change in terms of energy, water, habitat and human migration, and
using AI to optimize fortification of lifeline infrastructures for disaster
resilience. She has over 90 publications and has co-organized or served as
a chair to numerous workshops, tutorials, and special tracks at major
conferences.
This will be followed by a talk on Monday, October 10th, 2022.
The second talk will be on *"Rigorous Experimentation For Reinforcement
Learning" *by Scott Jordan is scheduled to be on October 10th, 1-2 PM PST. It
will be followed by a 30-minute Q&A session with the graduate students.
This is an *in-person* event and will be held at *KEC 305*
*Rigorous Experimentation For Reinforcement Learning*
Scott Jordan
Postdoc
University of Alberta
*Abstract:*
Scientific fields make advancements by leveraging the knowledge created by
others to push the boundary of understanding. The primary tool in many
fields for generating knowledge is empirical experimentation. Although
common, generating accurate knowledge from empirical experiments is often
challenging due to inherent randomness in execution and confounding
variables that can obscure the correct interpretation of the results. As
such, researchers must hold themselves and others to a high degree of rigor
when designing experiments. Unfortunately, most reinforcement learning (RL)
experiments lack this rigor, making the knowledge generated from
experiments dubious. This dissertation proposes methods to address central
issues in RL experimentation.
Evaluating the performance of an RL algorithm is the most common type of
experiment in RL literature. Most performance evaluations are often
incapable of answering a specific research question and produce misleading
results. Thus, the first issue we address is how to create a performance
evaluation procedure that holds up to scientific standards.
Despite the prevalence of performance evaluation, these types of
experiments produce limited knowledge, e.g., they can only show how well an
algorithm worked and not why, and they require significant amounts of time
and computational resources. As an alternative, this dissertation proposes
that scientific testing, the process of conducting carefully controlled
experiments designed to further the knowledge and understanding of how an
algorithm works, should be the primary form of experimentation.
Lastly, this dissertation provides a case study using policy gradient
methods, showing how scientific testing can replace performance evaluation
as the primary form of experimentation. As a result, this dissertation can
motivate others in the field to adopt more rigorous experimental practices.
*Speaker Bio:*
Scott Jordan got his Batchelor's degree from Oregon State in 2015 while
working with Tom Dietterich. He recently got his Ph.D. from the University
of Massachusetts and is now a Postdoc at the University of Alberta working
with Martha White. His research focuses on reinforcement learning with the
goal of understanding the properties necessary for scalable and effective
sequential decision-making, with works published in ICML, NeurIPS, and
other ML venues. His dissertation addresses issues with poor
experimentation practices common in reinforcement learning research.
*Please watch this space for future AI Seminars :*
* https://eecs.oregonstate.edu/ai-events
<https://eecs.oregonstate.edu/ai-events>*
Rajesh Mangannavar,
Graduate Student
Oregon State University
----
AI Seminar Important Reminders:
-> For graduate students in the AI program, attendance is strongly
encouraged.
Dear all,
Our next AI seminar on *"**AI and O.R. for Environmental Sustainability**" *by
Bistra Dilkina is scheduled to be on October 7th (Friday), 1-2 PM PST. It
will be followed by a 30-minute Q&A session by the graduate students.
This is an *in-person* event and will be held at *KEC 1001*
*AI and O.R. for Environmental Sustainability*
Bistra Dilkina
Associate Professor of Computer Science
Co-Director, USC Center of AI in Society
University of Southern California
*Abstract:*
With the increasing anthropogenic pressures of urbanization, agriculture,
deforestation, other socio-economic drivers as well as climate change,
biodiversity and habitat conservation is a key sustainable development
goal. Techniques from AI and O.R. and their hybridization have an important
role to play in providing both predictive and prescriptive tools to inform
critical decision-making, which can help us do more with less in this
important application domain. A prime example of the field of Computational
Sustainability, this presentation will give several successful examples of
the two-way street of research providing useful domain solutions to
real-world problems, while advancing core methodology in AI and O.R. Key
examples include using deep learning and satellite data for land cover
mapping, predicting species distributions under climate change and
optimizing spatial conservation planning, as well as developing data-driven
techniques to curb illicit wildlife poaching and trafficking.
*Speaker Bio:*
Dr. Bistra Dilkina is an associate professor of computer science at the
University of Southern California, co-director of the USC Center of AI in
Society, and the inaugural Dr. Allen and Charlotte Ginsburg Early Career
Chair at the USC Viterbi School of Engineering. Her research and teaching
center around the integration of machine learning and discrete
optimization, with a strong focus on AI applications in computational
sustainability and social good. She received her Ph.D. from Cornell
University in 2012 and was a post-doctoral associate at the Institute for
Computational Sustainability. Her research has contributed significant
advances to machine-learning-guided combinatorial solving including
mathematical programming and planning, as well as decision-focused learning
where combinatorial reasoning is integrated in machine learning pipelines.
Her applied research in Computational Sustainability spans using AI for
wildlife conservation planning, using AI to understand the impacts of
climate change in terms of energy, water, habitat and human migration, and
using AI to optimize fortification of lifeline infrastructures for disaster
resilience. She has over 90 publications and has co-organized or served as
a chair to numerous workshops, tutorials, and special tracks at major
conferences.
*Please watch this space for future AI Seminars :*
* https://eecs.oregonstate.edu/ai-events
<https://eecs.oregonstate.edu/ai-events>*
Rajesh Mangannavar,
Graduate Student
Oregon State University
----
AI Seminar Important Reminders:
-> For graduate students in the AI program, attendance is strongly
encouraged.
Dear all,
Our next AI seminar on *"Understanding Multiview and Self-Supervised
Representation Learning: A Nonlinear Mixture Identification Perspective" *by
Xiao Fu is scheduled to be on September 30th (Tomorrow), 1-2 PM PST. It
will be followed by a 30-minute Q&A session by the graduate students.
This is an *in-person* event and will be held at *KEC 1001*
*Understanding Multiview and Self-Supervised Representation Learning: A
Nonlinear Mixture Identification Perspective*
Xiao Fu
Assistant Professor
School of Electrical Engineering and Computer Science
Oregon State University
*Abstract:*
Central to representation learning is to succinctly represent
high-dimensional data using the “essential information’’ while discarding
the “redundant information”. Properly formulating and approaching this
objective is critical to fending against overfitting, and can also benefit
many important tasks such as domain adaptation and transfer learning. This
talk aims to deepen understanding of representation learning and using the
gained insights to come up with a new learning method. In particular,
attention will be paid to two representation learning paradigms using
multiple views of data, as both naturally acquired (e.g., image and audio)
and artificially produced (e.g., via adding different noise to data
samples) multiview data have empirically proven useful in producing
essential information-reflecting vector representations. Natural views are
often handled by multiview analysis tools, e.g., (deep) canonical
correlation analysis [(D)CCA], while the artificial ones are frequently
used in self-supervised learning (SSL) paradigms, e.g., BYOL and Barlow
Twins. However, the effectiveness of these methods is mostly validated
empirically, and more insights and theoretical underpinnings remain to be
discovered. In this talk, an intuitive generative model of multiview data
is adopted, where the views are different nonlinear mixtures of shared and
private components. Since the shared components are
view/distortion-invariant, such components may serve for representing the
essential information of data in a non-redundant way. Under this model, a
key module used in a suite of DCCA and SSL paradigms, namely, latent
correlation maximization, is shown to guarantee the extraction of the
shared components across views (up to certain ambiguities). It is further
shown that the private information in each view can be provably
disentangled from the shared using proper regularization design---which can
facilitate tasks such cross-view translation and data generation. A finite
sample analysis, which has been rare in nonlinear mixture identifiability
study, is also presented. The theoretical results and newly designed
regularization are tested on a series of tasks.
*Speaker Bio:*
Xiao Fu received the Ph.D. degree in Electronic Engineering from The
Chinese University of Hong Kong (CUHK), Shatin, N.T., Hong Kong, in 2014.
He was a Postdoctoral Associate with the Department of Electrical and
Computer Engineering, University of Minnesota, Minneapolis, MN, USA, from
2014 to 2017. Since 2017, he has been an Assistant Professor with the
School of Electrical Engineering and Computer Science, Oregon State
University, Corvallis, OR, USA. His research interests include the broad
area of signal processing and machine learning.
Dr. Fu received a Best Student Paper Award at ICASSP 2014, and was a
recipient of the Outstanding Postdoctoral Scholar Award at University of
Minnesota in 2016. His coauthored papers received Best Student Paper Awards
from IEEE CAMSAP 2015 and IEEE MLSP 2019, respectively. He received the
National Science Foundation CAREER Award in 2022. He serves as a member of
the Sensor Array and Multichannel Technical Committee (SAM-TC) of the IEEE
Signal Processing Society (SPS). He is also a member of the Signal
Processing for Multisensor Systems Technical Area Committee (SPMuS-TAC) of
EURASIP. He is the Treasurer of the IEEE SPS Oregon Chapter. He serves as
an Editor of Signal Processing and an Associate Editor of IEEE Transactions
on Signal Processing. He was a tutorial speaker at ICASSP 2017 and SIAM
Conference on Applied Linear Algebra 2021.
*Please watch this space for future AI Seminars :*
* https://eecs.oregonstate.edu/ai-events
<https://eecs.oregonstate.edu/ai-events>*
Rajesh Mangannavar,
Graduate Student
Oregon State University
----
AI Seminar Important Reminders:
-> For graduate students in the AI program, attendance is strongly
encouraged.
