Dear all,
Our next AI seminar on *"Understanding Multiview and Self-Supervised
Representation Learning: A Nonlinear Mixture Identification Perspective" *by
Xiao Fu is scheduled to be on September 30th (Tomorrow), 1-2 PM PST. It
will be followed by a 30-minute Q&A session by the graduate students.
This is an *in-person* event and will be held at *KEC 1001*
*Understanding Multiview and Self-Supervised Representation Learning: A
Nonlinear Mixture Identification Perspective*
Xiao Fu
Assistant Professor
School of Electrical Engineering and Computer Science
Oregon State University
*Abstract:*
Central to representation learning is to succinctly represent
high-dimensional data using the “essential information’’ while discarding
the “redundant information”. Properly formulating and approaching this
objective is critical to fending against overfitting, and can also benefit
many important tasks such as domain adaptation and transfer learning. This
talk aims to deepen understanding of representation learning and using the
gained insights to come up with a new learning method. In particular,
attention will be paid to two representation learning paradigms using
multiple views of data, as both naturally acquired (e.g., image and audio)
and artificially produced (e.g., via adding different noise to data
samples) multiview data have empirically proven useful in producing
essential information-reflecting vector representations. Natural views are
often handled by multiview analysis tools, e.g., (deep) canonical
correlation analysis [(D)CCA], while the artificial ones are frequently
used in self-supervised learning (SSL) paradigms, e.g., BYOL and Barlow
Twins. However, the effectiveness of these methods is mostly validated
empirically, and more insights and theoretical underpinnings remain to be
discovered. In this talk, an intuitive generative model of multiview data
is adopted, where the views are different nonlinear mixtures of shared and
private components. Since the shared components are
view/distortion-invariant, such components may serve for representing the
essential information of data in a non-redundant way. Under this model, a
key module used in a suite of DCCA and SSL paradigms, namely, latent
correlation maximization, is shown to guarantee the extraction of the
shared components across views (up to certain ambiguities). It is further
shown that the private information in each view can be provably
disentangled from the shared using proper regularization design---which can
facilitate tasks such cross-view translation and data generation. A finite
sample analysis, which has been rare in nonlinear mixture identifiability
study, is also presented. The theoretical results and newly designed
regularization are tested on a series of tasks.
*Speaker Bio:*
Xiao Fu received the Ph.D. degree in Electronic Engineering from The
Chinese University of Hong Kong (CUHK), Shatin, N.T., Hong Kong, in 2014.
He was a Postdoctoral Associate with the Department of Electrical and
Computer Engineering, University of Minnesota, Minneapolis, MN, USA, from
2014 to 2017. Since 2017, he has been an Assistant Professor with the
School of Electrical Engineering and Computer Science, Oregon State
University, Corvallis, OR, USA. His research interests include the broad
area of signal processing and machine learning.
Dr. Fu received a Best Student Paper Award at ICASSP 2014, and was a
recipient of the Outstanding Postdoctoral Scholar Award at University of
Minnesota in 2016. His coauthored papers received Best Student Paper Awards
from IEEE CAMSAP 2015 and IEEE MLSP 2019, respectively. He received the
National Science Foundation CAREER Award in 2022. He serves as a member of
the Sensor Array and Multichannel Technical Committee (SAM-TC) of the IEEE
Signal Processing Society (SPS). He is also a member of the Signal
Processing for Multisensor Systems Technical Area Committee (SPMuS-TAC) of
EURASIP. He is the Treasurer of the IEEE SPS Oregon Chapter. He serves as
an Editor of Signal Processing and an Associate Editor of IEEE Transactions
on Signal Processing. He was a tutorial speaker at ICASSP 2017 and SIAM
Conference on Applied Linear Algebra 2021.
*Please watch this space for future AI Seminars :*
* https://eecs.oregonstate.edu/ai-events
<https://eecs.oregonstate.edu/ai-events>*
Rajesh Mangannavar,
Graduate Student
Oregon State University
----
AI Seminar Important Reminders:
-> For graduate students in the AI program, attendance is strongly
encouraged.
Dear all,
Our next AI seminar on *"Understanding Multiview and Self-Supervised
Representation Learning: A Nonlinear Mixture Identification Perspective" *by
Xiao Fu is scheduled to be on September 30th (Friday), 1-2 PM PST. It will
be followed by a 30-minute Q&A session by the graduate students.
This is an *in-person* event and will be held at *KEC 1001*
*Understanding Multiview and Self-Supervised Representation Learning: A
Nonlinear Mixture Identification Perspective*
Xiao Fu
Assistant Professor
School of Electrical Engineering and Computer Science
Oregon State University
*Abstract:*
Central to representation learning is to succinctly represent
high-dimensional data using the “essential information’’ while discarding
the “redundant information”. Properly formulating and approaching this
objective is critical to fending against overfitting, and can also benefit
many important tasks such as domain adaptation and transfer learning. This
talk aims to deepen understanding of representation learning and using the
gained insights to come up with a new learning method. In particular,
attention will be paid to two representation learning paradigms using
multiple views of data, as both naturally acquired (e.g., image and audio)
and artificially produced (e.g., via adding different noise to data
samples) multiview data have empirically proven useful in producing
essential information-reflecting vector representations. Natural views are
often handled by multiview analysis tools, e.g., (deep) canonical
correlation analysis [(D)CCA], while the artificial ones are frequently
used in self-supervised learning (SSL) paradigms, e.g., BYOL and Barlow
Twins. However, the effectiveness of these methods is mostly validated
empirically, and more insights and theoretical underpinnings remain to be
discovered. In this talk, an intuitive generative model of multiview data
is adopted, where the views are different nonlinear mixtures of shared and
private components. Since the shared components are
view/distortion-invariant, such components may serve for representing the
essential information of data in a non-redundant way. Under this model, a
key module used in a suite of DCCA and SSL paradigms, namely, latent
correlation maximization, is shown to guarantee the extraction of the
shared components across views (up to certain ambiguities). It is further
shown that the private information in each view can be provably
disentangled from the shared using proper regularization design---which can
facilitate tasks such cross-view translation and data generation. A finite
sample analysis, which has been rare in nonlinear mixture identifiability
study, is also presented. The theoretical results and newly designed
regularization are tested on a series of tasks.
*Speaker Bio:*
Xiao Fu received the Ph.D. degree in Electronic Engineering from The
Chinese University of Hong Kong (CUHK), Shatin, N.T., Hong Kong, in 2014.
He was a Postdoctoral Associate with the Department of Electrical and
Computer Engineering, University of Minnesota, Minneapolis, MN, USA, from
2014 to 2017. Since 2017, he has been an Assistant Professor with the
School of Electrical Engineering and Computer Science, Oregon State
University, Corvallis, OR, USA. His research interests include the broad
area of signal processing and machine learning.
Dr. Fu received a Best Student Paper Award at ICASSP 2014, and was a
recipient of the Outstanding Postdoctoral Scholar Award at University of
Minnesota in 2016. His coauthored papers received Best Student Paper Awards
from IEEE CAMSAP 2015 and IEEE MLSP 2019, respectively. He received the
National Science Foundation CAREER Award in 2022. He serves as a member of
the Sensor Array and Multichannel Technical Committee (SAM-TC) of the IEEE
Signal Processing Society (SPS). He is also a member of the Signal
Processing for Multisensor Systems Technical Area Committee (SPMuS-TAC) of
EURASIP. He is the Treasurer of the IEEE SPS Oregon Chapter. He serves as
an Editor of Signal Processing and an Associate Editor of IEEE Transactions
on Signal Processing. He was a tutorial speaker at ICASSP 2017 and SIAM
Conference on Applied Linear Algebra 2021.
*Please watch this space for future AI Seminars :*
* https://eecs.oregonstate.edu/ai-events
<https://eecs.oregonstate.edu/ai-events>*
Rajesh Mangannavar,
Graduate Student
Oregon State University
----
AI Seminar Important Reminders:
-> For graduate students in the AI program, attendance is strongly
encouraged.
Dear all,
Our next AI seminar on *"AI Seminar: Universal Representations for AI" *by
Sridhar Mahadevan is scheduled to be on September 23rd (Tomorrow), 1-2 PM
PST. It will be followed by a 30-minute Q&A session by the graduate
students.
This is an *in-person* event and will be held at *KEC 1001*
*AI Seminar: Universal Representations for AI*
Sridhar Mahadevan
Director, Adobe Research
Research Professor, University of Massachusetts, Amherst
*Abstract:*
Humans are uniquely endowed in biology with a singular cognitive ability to
pass on millennia of accumulated knowledge to our descendants, not solely
through our genetic endowment, but also through the manufacture of abstract
representations: art, culture, language and writing, music, science, and
technology. This ability to transfer non-genetic information has been
greatly amplified with the invention of decentralized information
repositories, such as the traditional web or the emerging cryptographically
more secure blockchain economy. In this talk, I will describe a long-term
research program on analyzing four central capabilities that underlie our
ability to manufacture representations:
Objectification: our ability to denote arbitrary entities as ``objects”,
from elementary particles to our society, the U.S. economy, or an entire
galaxy or a (meta)universe.
Social Interaction: our ability to confer agenthood on both inanimate and
animate objects, treating objects not as passive containers of data, but
actors defined by social interactions.
Metaphors: our ability to use analogies to map unfamiliar situations –
understanding an alien civilization with an unknown language -- to ones we
are familiar with from our experience.
Diagrammatic representations: our ability to draw figures, record a tune as
a formal musical score, or write equations reveal how diagrams are a
universal cognitive calculus for us.
We use the abstract mathematics of category theory to characterize
universal properties that define these four core capabilities. Objects in a
category are modeled as actors, whose properties emerge from their social
interaction with other objects, modeled as morphisms. Functors and natural
transformations play the role of metaphors in human language. Adjunctions
across categories plays the role of analogies. Diagrams are special
functors that map from indexing categories to domain categories and define
universal constructions for reasoning about interactions. We illustrate our
framework by manufacturing novel representations for modeling causality and
conditional independence, creativity and imagination, learning, language,
and decision-making.
*Speaker Bio:*
Sridhar Mahadevan has been thinking about AI every waking day for the past
40 years, since he wrote his first machine learning program in 1982 at the
Indian Institute of Technology, Kanpur. He was elected Fellow of AAAI for
significant contributions in several areas of machine learning. He has
published widely across many subfields of AI, including a book on robot
learning in 1993 (Springer), and a book on representation discovery in 2007
(Morgan-Claypool). He is currently working on his third book on Universal
AI. He is a Director at Adobe Research, and a Research Professor at the
University of Massachusetts, Amherst.
*Please watch this space for future AI Seminars :*
* https://eecs.oregonstate.edu/ai-events
<https://eecs.oregonstate.edu/ai-events>*
Rajesh Mangannavar,
Graduate Student
Oregon State University
----
AI Seminar Important Reminders:
-> For graduate students in the AI program, attendance is strongly
encouraged.
Dear all,
Our next AI seminar on *"AI Seminar: Universal Representations for AI" *by
Sridhar Mahadevan is scheduled to be on September 23rd (Friday), 1-2 PM
PST. It will be followed by a 30-minute Q&A session by the graduate
students.
This is an *in-person* event and will be held at *KEC 1001*
*AI Seminar: Universal Representations for AI*
Sridhar Mahadevan
Director, Adobe Research
Research Professor, University of Massachusetts, Amherst
*Abstract:*
Humans are uniquely endowed in biology with a singular cognitive ability to
pass on millennia of accumulated knowledge to our descendants, not solely
through our genetic endowment, but also through the manufacture of abstract
representations: art, culture, language and writing, music, science, and
technology. This ability to transfer non-genetic information has been
greatly amplified with the invention of decentralized information
repositories, such as the traditional web or the emerging cryptographically
more secure blockchain economy. In this talk, I will describe a long-term
research program on analyzing four central capabilities that underlie our
ability to manufacture representations:
Objectification: our ability to denote arbitrary entities as ``objects”,
from elementary particles to our society, the U.S. economy, or an entire
galaxy or a (meta)universe.
Social Interaction: our ability to confer agenthood on both inanimate and
animate objects, treating objects not as passive containers of data, but
actors defined by social interactions.
Metaphors: our ability to use analogies to map unfamiliar situations –
understanding an alien civilization with an unknown language -- to ones we
are familiar with from our experience.
Diagrammatic representations: our ability to draw figures, record a tune as
a formal musical score, or write equations reveal how diagrams are a
universal cognitive calculus for us.
We use the abstract mathematics of category theory to characterize
universal properties that define these four core capabilities. Objects in a
category are modeled as actors, whose properties emerge from their social
interaction with other objects, modeled as morphisms. Functors and natural
transformations play the role of metaphors in human language. Adjunctions
across categories plays the role of analogies. Diagrams are special
functors that map from indexing categories to domain categories and define
universal constructions for reasoning about interactions. We illustrate our
framework by manufacturing novel representations for modeling causality and
conditional independence, creativity and imagination, learning, language,
and decision-making.
*Speaker Bio:*
Sridhar Mahadevan has been thinking about AI every waking day for the past
40 years, since he wrote his first machine learning program in 1982 at the
Indian Institute of Technology, Kanpur. He was elected Fellow of AAAI for
significant contributions in several areas of machine learning. He has
published widely across many subfields of AI, including a book on robot
learning in 1993 (Springer), and a book on representation discovery in 2007
(Morgan-Claypool). He is currently working on his third book on Universal
AI. He is a Director at Adobe Research, and a Research Professor at the
University of Massachusetts, Amherst.
*Please watch this space for future AI Seminars :*
* https://eecs.oregonstate.edu/ai-events
<https://eecs.oregonstate.edu/ai-events>*
Rajesh Mangannavar,
Graduate Student
Oregon State University
----
AI Seminar Important Reminders:
-> For graduate students in the AI program, attendance is strongly
encouraged.
