Dear all, Our next AI seminar is scheduled for Friday, February 28th. Talk details: AI Seminar: Occam Gradient Descent Speaker: Balas Natarajan Kausik Time: 2:00 PM Location: KEC 1001 and Zoom Zoom link: https://oregonstate.zoom.us/s/98357211915<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Foregonstate.zoom.us%2Fs%2F98357211915&data=05%7C02%7Cai%40engr.oregonstate.edu%7C5b934469414843b04ba008dd550cf8eb%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C638760236375317246%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=RKb5aT4hqRgZ3qnPg3TpVy3ylpaXpwDCvJsveLdEEfU%3D&reserved=0> Talk Abstract: Gradient descent on large neural networks such as transformers overfits early with many small weights, and therefore inefficient on both computing resources and training data. In response to these inefficiencies, we exploit learning theory to derive Occam Gradient Descent, which interleaves optimal reduction of model size to reduce test loss, with gradient descent on model weights to reduce training loss. Our experiments show that with respect to loss, compute and model size, Occam Gradient Descent outperforms traditional gradient descent across a range of problem domains. Speaker Bio: B.N. “Nat” Kausik is an independent researcher and entrepreneur. A graduate of Cornell, Princeton and IIT Madras, he has worked in both academia and industry. For future AI seminars, please visit: https://engineering.oregonstate.edu/EECS/research/AI-seminars<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fengineering.oregonstate.edu%2FEECS%2Fresearch%2FAI-seminars&data=05%7C02%7Cai%40engr.oregonstate.edu%7C5b934469414843b04ba008dd550cf8eb%7Cce6d05e13c5e4d6287a84c4a2713c113%7C0%7C0%7C638760236375335668%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=mG5fIo9o7gcfNXmIxJXJulcVu4oIqHuUOqagXnpGRX4%3D&reserved=0>. Best, Christian Abou Mrad Graduate Student Oregon State University