Portfolio item number 1
Short description of portfolio item number 1
Posts by Collection
portfolio
Portfolio item number 2
Short description of portfolio item number 2
publications
(Preprint) Towards Understanding Neural Collapse: The Effects of Batch Normalization and Weight Decay
Published in Arxiv Preprint, 2023
Neural Collapse (NC) is a geometric structure recently observed in the final layer of neural network classifiers. In this paper, we investigate the interrelationships between batch normalization (BN), weight decay, and proximity to the NC structure. Our work introduces the geometrically intuitive intra-class and inter-class cosine similarity measure, which encapsulates multiple core aspects of NC. Leveraging this measure, we establish theoretical guarantees for the emergence of NC under the influence of last-layer BN and weight decay, specifically in scenarios where the regularized cross-entropy loss is near-optimal. Experimental evidence substantiates our theoretical findings, revealing a pronounced occurrence of NC in models incorporating BN and appropriate weight-decay values. This combination of theoretical and empirical insights suggests a greatly influential role of BN and weight decay in the emergence of NC.
Recommended citation: Leyan Pan and Xinyuan Cao. Towards understanding neural collapse: The effects of Batch Normalization and Weight Decay. Arxiv Preprint, 2023. https://arxiv.org/abs/2309.04644
talks
Talk 1 on Relevant Topic in Your Field
Published:
This is a description of your talk, which is a markdown files that can be all markdown-ified like any other post. Yay markdown!
Conference Proceeding talk 3 on Relevant Topic in Your Field
Published:
This is a description of your conference proceedings talk, note the different field in type. You can put anything in this field.
teaching
Graduate Teaching Assistant for CS 4510 Automata and Complexity
Undergraduate course, Georgia Institute of Technology, School of Computer Science, 2022
Answers in-class questions, hosts office hours, and grade exams and homework for ~100 student for CS 4510 Automata and Complexity (Spring 2022, Fall 2022, Spring 2023) instructed by Prof. Zvi Galil at Georgia Tech.
Graduate Teaching Assistant of CS 6262 Network Security
graduate course, Georgia Institute of Technology, School of Cybersecurity and Privacy, 2023
Answered online questions, hosted office hours, and graded homeworks for CS 6262 Network Security instructed by Prof. Wenke Lee at Georgia Tech.