Papers

Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization

Noam Razin, Sadhika Malladi, Adithya Bhaskar, Danqi Chen, Sanjeev Arora, Boris Hanin.

Fine-Tuning in Modern Machine Learning (FITML) at NeurIPS 2024.
Attributing Model Behavior at Scale (ATTRIB) at NeurIPS 2024.

Progressive distillation induces an implicit curriculum

Abhishek Panigrahi*, Bingbin Liu*, Sadhika Malladi, Andrej Risteski, Surbhi Goel.

Math for Modern Machine Learning (M3L) Workshop at NeurIPS 2024.
Theoretical Foundations of Foundation Models Workshop at ICML 2024
Mechanistic Interpretability (MI) Workshop at ICML 2024 (initial version).

Adaptive Data Optimization: Dynamic Sample Selection with Scaling Laws

Yiding Jiang, Allan Zhou, Zhili Feng, Sadhika Malladi, J. Zico Kolter.

In submission.

Provable unlearning in topic modeling and downstream tasks.

Stanley Wei, Sadhika Malladi, Sanjeev Arora, Amartya Sanyal.

In submission.

CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs

Zirui Wang, Mengzhou Xia, Luxi He, Howard Chen, Yitao Liu, Richard Zhu, Kaiqu Liang, Xindi Wu, Haotian Liu, Sadhika Malladi, Alexis Chevalier, Sanjeev Arora, Danqi Chen.

NeurIPS 2024.
EVAL-FoMo Workshop at ECCV 2024.

Preference Learning Algorithms Do Not Learn Preference Rankings

Angelica Chen, Sadhika Malladi, Lily H. Zhang, Xinyi Chen, Qiuyi Zhang, Rajesh Ranganath, Kyunghyun Cho.

NeurIPS 2024.
Oral presentation at Models of Human Feedback for AI Alignment Workshop at ICML 2024.
Theoretical Foundations of Foundation Models at ICML 2024.

MUSE: Machine Unlearning Six-Way Evaluation for Language Models

Weijia Shi*, Jaechan Lee*, Yangsibo Huang*, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A. Smith, Chiyuan Zhang.

GenLaw Workshop at ICML 2024.

LESS: Selecting Influential Data for Targeted Instruction Tuning

Mengzhou Xia*, Sadhika Malladi*, Suchin Gururangan, Sanjeev Arora, Danqi Chen.

ICML 2024 (OpenReview).
Data Problems for Foundation Models (DPFM) Workshop at ICLR 2024.

Trainable Transformer in Transformer

Abhishek Panigrahi*, Sadhika Malladi*, Mengzhou Xia, Sanjeev Arora.

ICML 2024 (OpenReview).
Robustness of Few-Shot and Zero-Shot Learning in Foundation Models (R0-FoMo) Workshop at NeurIPS 2023.

The Marginal Value of Momentum for Small Learning Rate SGD

Runzhe Wang, Sadhika Malladi, Tianhao Wang, Kaifeng Lyu, Zhiyuan Li.

ICLR 2024 (OpenReview).
High-dimensional Learning Dynamics Workshop at ICML 2023.

Fine-Tuning Language Models with Just Forward Passes

Sadhika Malladi*, Tianyu Gao*, Eshaan Nichani, Alex Damian, Jason D. Lee, Danqi Chen, Sanjeev Arora.

Oral presentation at NeurIPS 2023 (OpenReview).
Oral presentation at Efficient Systems for Foundation Models (ES-FoMo) workshop at ICML 2023.
Differentiable Everything Workshop at ICML 2023.

A Kernel-Based View of Language Model Fine-Tuning

Sadhika Malladi, Alexander Wettig, Dingli Yu, Danqi Chen, Sanjeev Arora.

ICML 2023 (OpenReview).
Oral presentation at ICLR 2023 Foundation Models Workshop (ME-FoMo).

On the SDEs and Scaling Rules for Adaptive Gradient Algorithms

Sadhika Malladi*, Kaifeng Lyu*, Abhishek Panigrahi, Sanjeev Arora.

NeurIPS 2022 (OpenReview).
Oral presentation at ICML 2022 Workshop on Continuous Time Methods.

On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs)

Zhiyuan Li, Sadhika Malladi, Sanjeev Arora.

NeurIPS 2021 (OpenReview).

A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks

Nikunj Saunshi, Sadhika Malladi, Sanjeev Arora.

ICLR 2021 (OpenReview).
Oral presentation at NeurIPS 2020 Women in Machine Learning Workshop (Video).
Self-Supervised Learning Workshop at NeurIPS 2020.
Nikunj's Talk at the Vector Institute.

Prediction Propagation for Domain Adaptation in NLP

Bjarke Felbo, Michiel Bakker, Abhimanyu Dubey, Sadhika Malladi, Alex "Sandy" Pentland, Iyad Rahwan.

Towards learning with limited labels: Equivariance, Invariance, and Beyond at ICML 2018.

FastNorm: Improving Numerical Stability of Deep Network Training with Efficient Normalization

Sadhika Malladi, Ilya Sharapov

Women in Machine Learning Workshop at NeurIPS 2017.

Systematic Analysis of Sex-Linked Molecular Alterations and Therapies in Cancer

Sadhika Malladi*, Jonathan Ma*, Andrew H. Beck

Nature Scientific Reports, 2016.

EMDomics: a robust and powerful method for the identification of genes differentially expressed between heterogeneous classes

Daniel Schmolze, Mayineur Matituoheti, Sadhika Malladi, Andrew H. Beck.

Bioinformatics, 2016.

Assessing treatment response in triple-negative breast cancer from quantitative image analysis in perfusion magnetic resonance imaging

Imon Banerjee, Sadhika Malladi, Daniela Lee, Adrien Depeursinge, Melinda Telli, Jafi Lipson, Daniel Golden, Daniel L. Rubin.

Journal of Medical Imaging, 2017.