Prof. Pradeep Ravikumar, Carnegie Mellon University
Building Robust Ensembles via Margin Boosting
Monday Sept. 14, 2022 | 11:15
Hybrid
Room BM5202 | Zoom Link
In the context of adversarial robustness, a single model does not usually have enough power to defend against all possible adversarial attacks, and as a result, has sub-optimal robustness. Consequently, an emerging line of work has focused on learning an ensemble of neural networks to defend against adversarial attacks. In this work, we take a principled approach towards building robust ensembles. We view this problem from the perspective of solving a margin-boosting game, and develop an algorithm for learning an ensemble with maximum margin. Through extensive empirical evaluation on benchmark datasets, we show that our algorithm not only outperforms existing ensembling techniques, but also large models trained in an end-to-end fashion. An important byproduct of our work is a margin-maximizing cross-entropy (MCE) loss, which is a better alternative to the standard cross-entropy (CE) loss. Empirically, we show that replacing the CE loss in state-of-the-art adversarial training techniques with our MCE loss leads to significant performance improvement.
Joint work with Arun Sai Suggala, Dinghuai Zhang, Hongyang Zhang, Aaron Courville, Yoshua Bengio
Joint work with Arun Sai Suggala, Dinghuai Zhang, Hongyang Zhang, Aaron Courville, Yoshua Bengio
Pradeep Ravikumar is a Professor in the Machine Learning Department, School of Computer Science at Carnegie Mellon University. He was previously an Associate Director at the Center for Big Data Analytics, at the University of Texas at Austin. His thesis has received honorable mentions in the ACM SIGKDD Dissertation award and the CMU School of Computer Science Distinguished Dissertation award. He is a Sloan Fellow, a Siebel Scholar, a recipient of the NSF CAREER Award, and was Program Chair for the International Conference on Artificial Intelligence and Statistics (AISTATS) in 2013. He is Associate Editor-in-Chief for IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), and action editor for the Machine Learning journal, and the Journal of Machine Learning Research.
Dr. Ravikumar’s research group at CMU works on the foundations of statistical machine learning, with recent focus on “next generation” machine learning systems, that are explainable, robust to train and test time corruptions, and resilient to distribution shifts, and are learnt under resource constraints by leveraging or discovering various notions of “structure” and domain knowledge.
Dr. Ravikumar’s research group at CMU works on the foundations of statistical machine learning, with recent focus on “next generation” machine learning systems, that are explainable, robust to train and test time corruptions, and resilient to distribution shifts, and are learnt under resource constraints by leveraging or discovering various notions of “structure” and domain knowledge.
: