June 2023 - Ai - Engineering Mailing Lists

information theory class
by Nguyen, Thinh 05 Jan '24

05 Jan '24

Hello everyone, I just want to bring to your attention that I will be teaching information theory class (ECE 566) next term. If you are interested in learning the theories and some foundational connections among various disciplines (compression, compression, machine learning, statistics, ...) then you might want to consider taking ECE 566. I am attaching the ECE 566 syllabus. If enough number of student register for ECE 566, I will offer the second … [View More]

1 1

AI Seminar : June 9, 2023
by Mangannavar, Rajesh Devaraddi 08 Jun '23

08 Jun '23

Dear all, Our next AI seminar on *"Lipstick on a Pig: Using Language Models as Few-Shot Learners" *by Sameer Singh is scheduled to be on June 9th (Tomorrow), 1-2 PM PST (Add to Google Calendar <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcalendar.…> ) *Location: * https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Foregonsta… Please note that this is a Zoom-only event *Lipstick on a Pig: Using Language Models as Few-Shot Learners* Sameer Singh … [View More]Associate Professor, Computer Science University of California, Irvine *Abstract:* Today's Natural Language Processing (NLP) strategies are heavily reliant on pre-trained language models due to their ability to deliver semantically rich representations. While these models provide impressive few-shot natural language understanding and reasoning capabilities, simply using them as "fill in the blank" prompts may not be a one-size-fits-all solution. Despite the allure of direct application of these models - an allure that grows with model and dataset sizes - the objectives of language modeling and few-shot learning are not perfectly aligned. Unpacking this disparity is crucial. In this talk, I will describe some of our work in characterizing the differences between language modeling and few-shot learning. I will show how language modeling comes with crucial shortcomings for few-shot adaptation and describe a simple approach to address them. Then, focusing on numerical reasoning, I will show that the reasoning ability of the language models depends strongly on simple statistics of the pretraining corpus, performing much more accurately for more common terms. These results suggest language modeling may not be sufficient to learn robust reasoners and that we need to take the pretraining data into account when interpreting few-shot evaluation results. While language models hold substantial promise, making these 'pigs' presentable with 'lipstick' may require a more comprehensive approach than currently anticipated. *Speaker Bio:* Dr. Sameer Singh is an Associate Professor of Computer Science at the University of California, Irvine (UCI). He is working primarily on the robustness and interpretability of machine learning algorithms and models that reason with text and structure for natural language processing. Sameer was a postdoctoral researcher at the University of Washington and received his Ph.D. from the University of Massachusetts, Amherst. He has received the NSF CAREER award, UCI Distinguished Early Career Faculty award, the Hellman Faculty Fellowship, and was selected as a DARPA Riser. His group has received funding from Allen Institute for AI, Amazon, NSF, DARPA, Adobe Research, Hasso Plattner Institute, NEC, Base 11, and FICO. Sameer has published extensively at machine learning and natural language processing venues and received conference paper awards at KDD 2016, ACL 2018, EMNLP 2019, AKBC 2020, ACL 2020, and NAACL 2022. (https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsameersin…) *Please watch this space for future AI Seminars :* https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fengineeri… Rajesh Mangannavar, Graduate Student Oregon State University ---- AI Seminar Important Reminders: -> For graduate students in the AI program, attendance is strongly encouraged [View Less]

1 0

AI Seminar : June 9, 2023
by Mangannavar, Rajesh Devaraddi 06 Jun '23

06 Jun '23

Dear all, Our next AI seminar on *"Lipstick on a Pig: Using Language Models as Few-Shot Learners" *by Sameer Singh is scheduled to be on June 9th (Friday), 1-2 PM PST (Add to Google Calendar <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcalendar.…> ) *Location: * https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Foregonsta… Please note that this is a Zoom-only event *Lipstick on a Pig: Using Language Models as Few-Shot Learners* Sameer Singh … [View More]Associate Professor, Computer Science University of California, Irvine *Abstract:* Today's Natural Language Processing (NLP) strategies are heavily reliant on pre-trained language models due to their ability to deliver semantically rich representations. While these models provide impressive few-shot natural language understanding and reasoning capabilities, simply using them as "fill in the blank" prompts may not be a one-size-fits-all solution. Despite the allure of direct application of these models - an allure that grows with model and dataset sizes - the objectives of language modeling and few-shot learning are not perfectly aligned. Unpacking this disparity is crucial. In this talk, I will describe some of our work in characterizing the differences between language modeling and few-shot learning. I will show how language modeling comes with crucial shortcomings for few-shot adaptation and describe a simple approach to address them. Then, focusing on numerical reasoning, I will show that the reasoning ability of the language models depends strongly on simple statistics of the pretraining corpus, performing much more accurately for more common terms. These results suggest language modeling may not be sufficient to learn robust reasoners and that we need to take the pretraining data into account when interpreting few-shot evaluation results. While language models hold substantial promise, making these 'pigs' presentable with 'lipstick' may require a more comprehensive approach than currently anticipated. *Speaker Bio:* Dr. Sameer Singh is an Associate Professor of Computer Science at the University of California, Irvine (UCI). He is working primarily on the robustness and interpretability of machine learning algorithms and models that reason with text and structure for natural language processing. Sameer was a postdoctoral researcher at the University of Washington and received his Ph.D. from the University of Massachusetts, Amherst. He has received the NSF CAREER award, UCI Distinguished Early Career Faculty award, the Hellman Faculty Fellowship, and was selected as a DARPA Riser. His group has received funding from Allen Institute for AI, Amazon, NSF, DARPA, Adobe Research, Hasso Plattner Institute, NEC, Base 11, and FICO. Sameer has published extensively at machine learning and natural language processing venues and received conference paper awards at KDD 2016, ACL 2018, EMNLP 2019, AKBC 2020, ACL 2020, and NAACL 2022. (https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsameersin…) *Please watch this space for future AI Seminars :* https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fengineeri… Rajesh Mangannavar, Graduate Student Oregon State University ---- AI Seminar Important Reminders: -> For graduate students in the AI program, attendance is strongly encouraged [View Less]

1 0

AI Seminar : June 2, 2023
by Mangannavar, Rajesh Devaraddi 01 Jun '23

01 Jun '23

Dear all, Our next AI seminar on *"Convergence Analysis Framework for Fixed-Point Algorithms in Machine Learning and Signal Processing" *by Raviv Raich is scheduled to be on June 2nd (Tomorrow), 1-2 PM PST (Add to Google Calendar <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcalendar.…> ). It will be followed by a 30-minute Q&A session with the graduate students. *Location: *Crop Science Building 122 *Convergence Analysis Framework for Fixed-Point Algorithms in … [View More]Machine Learning and Signal Processing* Raviv Raich Associate Professor Electrical and Computer Engineering Oregon State University *Abstract:* Many iterative algorithms are designed to solve challenging signal processing and machine learning optimization problems. Among such algorithms are iterative hard thresholding for sparse reconstruction, projected gradient descent for matrix completion, and alternating projection for phase retrieval. Such algorithms and others can be viewed through the fix-point iteration lens. In this talk, we will examine the fixed-point view of iterative algorithms. We will present results on the analysis of projected gradient descent for the well-known constrained least squares problem and show how such analysis can be used to optimize the iterative solution. As a special case of this framework, we will examine iterative solutions to the matrix completion problem. Our approach provides a stepping stone to the optimization of acceleration approaches across multiple algorithms. *Speaker Bio:* Raviv Raich received the B.Sc. and M.Sc. degrees in electrical engineering from Tel-Aviv University, Tel-Aviv, Israel, in 1994 and 1998, respectively, and the Ph.D. degree in electrical engineering from the Georgia Institute of Technology, Atlanta, GA, in 2004. Between 1999 and 2000, he was a Researcher with the Communications Team, Industrial Research, Ltd., Wellington, New Zealand. From 2004 to 2007, he was a Postdoctoral Fellow with the University of Michigan, Ann Arbor, MI. Raich has been an assistant professor (2007-2013) and an associate professor (2013-2023) with the School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, OR. His research interests include probabilistic modeling and optimization in signal processing and machine learning. From 2011 to 2014, he was an Associate Editor for the IEEE Transactions on Signal Processing. He was a Member during 2011–2016 and the Chair during 2017–2018 of the Machine Learning for Signal Processing Technical Committee (TC) of the IEEE Signal Processing Society. Since 2019, he has been a Member of the Signal Processing Theory and Methods, TC of the IEEE Signal Processing Society. *Please watch this space for future AI Seminars :* https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fengineeri… Rajesh Mangannavar, Graduate Student Oregon State University ---- AI Seminar Important Reminders: -> For graduate students in the AI program, attendance is strongly encouraged [View Less]

1 0