February 2025 - Ai - Engineering Mailing Lists

AI Seminar: Occam Gradient Descent
by Abou Mrad, Christian 27 Feb '25

27 Feb '25

Dear all, Our next AI seminar is scheduled for Friday, February 28th. Talk details: AI Seminar: Occam Gradient Descent Speaker: Balas Natarajan Kausik Time: 2:00 PM Location: KEC 1001 and Zoom Zoom link: https://oregonstate.zoom.us/s/98357211915<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Foregonsta…> Talk Abstract: Gradient descent on large neural networks such as transformers overfits early with many small weights, and therefore inefficient on both computing resources and training data. In response to these inefficiencies, we exploit learning theory to derive Occam Gradient Descent, which interleaves optimal reduction of model size to reduce test loss, with gradient descent on model weights to reduce training loss. Our experiments show that with respect to loss, compute and model size, Occam Gradient Descent outperforms traditional gradient descent across a range of problem domains. Speaker Bio: B.N. “Nat” Kausik is an independent researcher and entrepreneur. A graduate of Cornell, Princeton and IIT Madras, he has worked in both academia and industry. For future AI seminars, please visit: https://engineering.oregonstate.edu/EECS/research/AI-seminars<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fengineeri…>. Best, Christian Abou Mrad Graduate Student Oregon State University

1 1

AI Seminar: Towards Intrinsically Explainable Clustering Algorithms
by Abou Mrad, Christian 20 Feb '25

20 Feb '25

Dear all, Our next AI seminar is scheduled for Friday, February 21th. Talk details: AI Seminar: Towards Intrinsically Explainable Clustering Algorithms Speaker: Dr. Eldan Cohen, Assistant Professor of Industrial Engineering at The University of Toronto, The director of the Optimization and Machine Learning (OptiMaL) Lab Time: 2:00 PM Location: KEC 1001 and Zoom Zoom link: https://oregonstate.zoom.us/s/98357211915<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Foregonsta…> Talk Abstract: As machine learning has become more prevalent in society in recent years, the need for trustworthy models that stakeholders can audit has increased dramatically. In particular, one desirable aspect of trustworthiness in ML is that the approaches utilized are constrained so that their predictive mechanisms are innately explainable to humans. Cluster analysis is a fundamental task aims to uncover meaningful groups within data, with many high-stakes applications. Despite the growing interest in explainability in ML, relatively limited work has focused on explainable clustering and most clustering algorithms do not provide any explanation of the obtained partition, requiring post-hoc analysis to characterize the clusters. In this talk, I will describe recent work on clustering algorithms that construct explainable mappings from samples to clusters. We will discuss deep neural approaches, as well as approaches based on combinatorial optimization, to construct such mappings. We will also consider the ability to use clustering constraints to incorporate domain knowledge and improve clustering accuracy and present insight into the trade-off between explainability and the satisfaction of such constraints. Speaker Bio: Eldan Cohen is an Assistant Professor of Industrial Engineering at The University of Toronto and the director of the Optimization and Machine Learning (OptiMaL) Lab. His research interests include machine and deep learning, heuristic search and optimization, and natural language processing, with an emphasis on human compatible approaches. In addition, he has worked on applications of these techniques in various domains such as healthcare, manufacturing, and autonomous agents. He completed his PhD at the Department of Mechanical and Industrial Engineering at the University of Toronto and was a postdoctoral fellow at the department of Computer Science and the Vector Institute for Artificial Intelligence. All information can be found: https://events.oregonstate.edu/event/towards-intrinsically-explainable-clus…<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fevents.or…> For future AI seminars, please visit: https://engineering.oregonstate.edu/EECS/research/AI-seminars<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fengineeri…>. Best, Christian Abou Mrad Graduate Student Oregon State University

1 1

AI Seminar: Understanding the promises and limits of fine-tuning
by Abou Mrad, Christian 13 Feb '25

13 Feb '25

Dear all, Our next AI seminar is scheduled for Friday, February 14th. Talk details: AI Seminar: Understanding the promises and limits of fine-tuning Speaker: Dr. Aditi Raghunathan, Assistant Professor of Computer Science, Carnegie Mellon University Time: 2:00 PM Location: KEC 1001 and Zoom Zoom link: https://oregonstate.zoom.us/s/98357211915<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Foregonsta…> Talk Abstract: In recent years, foundation models—large pretrained models that can be adapted for a wide range of tasks—have achieved state-of-the-art performance on a variety of tasks. The adaptation or fine-tuning process is a crucial component that enables specialization to the task of interest, and is the de facto standard for mitigating risks such as reducing toxic and harmful generations from large language models. While the pretrained models are trained on broad data, the adaptation (or fine-tuning) process is often performed on limited well-curated data. How well does fine-tuning generalize beyond the narrow training distribution? Via theory and experiments, we show how to improve current fine-tuning approaches so that they can better leverage diverse pretraining knowledge and improve downstream performance across broader settings than the narrow fine-tuning data. On the flip side, we show that pretrained knowledge can be hard to get rid of, thereby underlining the potential perils of over reliance on fine-tuning for safety. Speaker Bio: Aditi Raghunathan is an Assistant Professor in the Computer Science Department at CMU. She received her PhD from Stanford in 2021, and Bachelor of Technology from IIT Madras in 2016. She is a recipient of the Okawa research grant, Schmidt AI2050 Early Career Fellowship, the Google Research Scholar Award, Rising Stars in EECS, Google PhD Fellowship, Open Philanthropy AI Fellowship, Stanford School of Engineering Fellowship and Google Anita Borg Memorial Fellowship. She was featured in the Forbes 30 under 30 list for her contributions to reliable machine learning. Her PhD thesis was awarded the Arthur Samuel Best Thesis Award at Stanford. Her research has also been recognized by multiple orals and spotlights at top conferences, and a Best Paper Award at Data Problems in ML Workshop at ICLR 2024. For future AI seminars, please visit: https://engineering.oregonstate.edu/EECS/research/AI-seminars<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fengineeri…>. Best, Christian Abou Mrad Graduate Student Oregon State University

1 1

AI Seminar: Active Learning For Human Preference Elicitation In LLMs
by Abou Mrad, Christian 06 Feb '25

06 Feb '25

Dear all, Our next AI seminar is scheduled for Friday, February 7th. Talk details: AI Seminar: Active Learning For Human Preference Elicitation In LLMs Speaker: Branislav Kveton, Adobe Research Time: 2:00 PM Location: KEC 1001 and Zoom Zoom link: https://oregonstate.zoom.us/s/98357211915<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Foregonsta…> Talk Abstract: Learning from human preferences has been central to recent advances in AI. Motivated by the cost of obtaining high-quality human feedback, we study active learning of large language models (LLMs) through adaptive human preference elicitation. The key idea in our work is to linearize LLMs, either the reward model or the optimized policy, and apply optimal designs, a methodology for computing optimal information-gathering policies in statistics. We design efficient algorithms, fundamentally leverage the autoregressive nature of the LLMs, and evaluate our algorithms on state-of-the-art models. Speaker Bio: Branislav (Brano) is a Principal Research Scientist at Adobe Research and leads their team on planning and reasoning with generative AI models. He proposes, analyzes, and applies algorithms that learn incrementally, run in real time, and converge to near-optimal solutions with more feedback. Brano has been an active researcher in the fields of ML and AI for more than 15 years. His works have been published at top-tier conferences in these fields and he serves on senior roles at these conferences. For future AI seminars, please visit: https://engineering.oregonstate.edu/EECS/research/AI-seminars<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fengineeri…>. Best, Christian Abou Mrad Graduate Student Oregon State University

1 1