Speaker: Praneeth Vepakomma (MIT/IDSS and MBZUAI)Title: Extremely-efficient fine-tuning of LLMsAbstract:Large Language Models (LLMs) have reshaped generative AI, but fully fine-tuning these massive architectures is quite expensive in computational and communication resources. Low-Rank Adaptation (LoRA) partially mitigates these challenges, yet conventional LoRA often struggles to match the performance of full fine-tuning. In this talk, I introduce LoRA-SB (LoRA Silver Bullet), a novel approach that injects a constrained update space into LoRA’s framework, enabling optimal scaling for high-rank gradient directions that mimic full fine-tuning in a low-rank space, and meets the performance of full fine-tuning. We theoretically prove that our initialization strategy provides an optimal low-rank approximation of the initial gradient and preserves critical update directions throughout training. Extensive experiments on mathematical reasoning, commonsense inference, and language understanding tasks show that LoRA-SB exceeds the performance of standard LoRA while requiring 27–90× fewer trainable parameters and comprehensively outperforms LoRA-XS. Our findings demonstrate that it is not only possible but also highly effective to simulate full fine-tuning in low-rank subspaces, offering significant efficiency gains at no loss in accuracy. Additionally, we introduce Fed-SB, a federated extension of LoRA-SB that employs direct averaging of the small matrix R to guarantee exact updates and drastically reduce communication costs—independent of the number of clients—by up to 230×. Fed-SB further enhances privacy-utility-communication efficiency trade-offs by lowering noise requirements and avoiding noise amplification. Overall, it establishes a new Pareto frontier for efficient, scalable federated fine-tuning in both private and non-private settings.Zoom link: https://mit.zoom.us/j/91933017072

A 50 minute, (usually) student-led tour of MIT D-Lab, D-Lab prototypes, and our workshop! Hear about the 23-year history of D-Lab, our founding director Amy Smith, our 12+ MIT classes, research groups, humanitarian innovation program and more! Not free at tour time? Stop by anytime to look around or email d-lab-tours@mit.edu.

Have some fun(nel) on a tunnel or sun(nel) walk! Join us for a 30-minute volunteer-led walk either through MIT's famous tunnel system or around Killian Court. As the weather gets warmer, walk leaders may choose to take the group outside. Is the weather warm and you missed the start? Find the group on Killian Court and join in!Location details: Meet in the lobby under the “Belonging + Community” banner. Location photo below.Sun(nel) Walk Leaders will identify themselves by holding a white at the meeting location.Prize Drawing: Attend a walk and scan a QR code from the walk leaders to be entered into a drawing for a getfit canvas boat tote bag at the end of the getfit challenge. The more walks you attend, the more entries you get. Winner will be drawn and notified at the end of April. Winner does not need to be a getfit participant.Disclaimer: Tunnel walks are led by volunteers. In the rare occasion when a volunteer isn’t able to make it, we will do our best to notify participants. In the event we are unable to notify participants and a walk leader does not show up, we encourage you to walk as much as you feel comfortable doing so. We recommend you check this calendar just before you head out!Getfit is a 12-week fitness challenge for the entire MIT community. These tunnel walks are open to the entire MIT community and you do not need to be a current getfit participant to join.

Register here.We will be making eclectic Crystal Rhinestone Painting Suncatcher. These decorative suncatcher ornaments will bring a sparkling touch to your interior decor. These Crafternoon events provide for a great opportunity to connect with fellow spouses and partners from MS&PC and create long-lasting memories. Children are welcome to join as well. We will be providing all supplies for this event, sponsored by MS&PC.If you have any questions, please contact Miyuki Ota: miyukichan8@hotmail.com.

Title: A relative trace formula approach to the stable trace formula on unitary groupSpeaker: Weixiao LuZoom Link: https://mit.zoom.us/j/3930038357

Have some fun(nel) on a tunnel or sun(nel) walk! Join us for a 30-minute volunteer-led walk either through MIT’s famous tunnel system or around Killian Court. As the weather gets warmer, walk leaders may choose to take the group outside. Is the weather warm and you missed the start? Find the group on Killian Court and join in!Location details: Meet in the lobby with the big mirror, right inside the Collier Memorial entrance to Stata. Location photo below.Sun(nel) Walk Leaders will identify themselves by holding a white flag at the meeting location.Prize Drawing: Attend a walk and scan a QR code from the walk leaders to be entered into a drawing for a getfit canvas boat tote bag at the end of the getfit challenge. The more walks you attend, the more entries you get. Winner will be drawn and notified at the end of April. Winner does not need to be a getfit participant.Disclaimer: These walks are led by volunteers. In the rare occasion when a volunteer isn’t able to make it, we will do our best to notify participants. In the event we are unable to notify participants and a walk leader does not show up, we encourage you to walk as much as you feel comfortable doing so. We recommend checking this calendar just before you head out!Getfit is a 12-week fitness challenge for the entire MIT community. These walks are open to the entire MIT community and you do not need to be a current getfit participant to join.