Guojun Zhang

I am an algorithmic engineer at MiniMax. My current research focus is on Alignment for LLMs, and transfer learning. I obtained my Ph.D [thesis] from the David R. Cheriton School of Computer Science at the University of Waterloo, also as a student affiliate of the Vector Institute. My PhD supervisors were Prof. Pascal Poupart and Prof. Yaoliang Yu. I obtained my master in theoretical physics from the Perimeter Institute, and I was fortunate to work with Prof. Freddy Cachazo on theoretical physics.

Research Interests: alignment, transfer learning, multimodality

News

• 2024.07 I started my new job at MiniMax. Excited to ramp up in LLMs.
• 2024.06 One paper accepted to ICML 2024.
• 2023.12 Our paper ''Calibrated One Round Federated Learning with Bayesian Inference in the Predictive Space'' has been accepted in AAAI 2024. Congrats Mohsin (now a PhD student of Yoshua Bengio)!
• 2023.08 I will serve as an Area Chair for AISTATS 2024.
• 2023.08 Our paper ''Private GANs, Revisited'' has been accepted to TPDP 2023 and TMLR.
• 2023.07 Our paper ''Understanding Hessian Alignment for Domain Generalization'' has been accepted to ICCV 2023.
• 2022.12 Our paper ''Proportional Fairness in Federated Learning'' has been accepted at TMLR.
• 2022.11 Two papers to present at NeurIPS 2022: our JMLR paper at the Journal-to-Conference Track and our paper ''Private GANs, Revisited'' at the SyntheticData4ML Workshop.
• 2022.11 Reviewing for ICLR 2023.
• 2022.10 Reviewing for SyntheticData4ML Workshop in NeurIPS 2022.
• 2022.08 I will serve as an Area Chair for AISTATS 2023.
• 2022.07 I will serve as a Session Chair in ICML 2022.
• 2022.05 I will give an invited talk at 2022 Optimization Days organized by HEC Montréal.
• 2022.04 Our paper ''Federated Learning Meets Multi-objective Optimization'' is accepted at IEEE Transactions on Network Science and Engineering!
• 2022.03 I will serve as a Program Committee in the FL-IJCAI workshop 2022.
• 2022.01 Our paper ''Domain Adversarial Training: A Game Perspective'' has been accepted at ICLR 2022.
• 2022.01 Our paper ''Optimality and Stability in Non-convex Smooth Games'' has been accepted to Journal of Machine Learning Research.
• 2021.11 Our paper ''f-Mutual Information Contrastive Learning'' has been accepted as an oral to NeurIPS 2021 workshop on self-supervised learning!
• 2021.09 Our paper ''Quantifying and Improving Transferability in Domain Generalization'' is accepted at NeurIPS 2021! Thanks to all my collaborators!
• 2021.09 Excited to start my new job as a senior researcher at Huawei Montreal!
• 2021.07 Our paper ''Newton-type Methods for Minimax Optimization'' is accepted at the ICML 2021 workshop ''Beyond first-order methods in ML systems''
• 2021.07 Completed my defence! Thanks to all committee members!
• 2021.05 Our paper ''f-Domain Adversarial Learning: Theory and Algorithms'' is accepted at ICML 2021
• 2021.05 Happy to serve as reviewers for CoRL 2021 and NeurIPS 2021
• 2020.08 Attending CIFAR Deep Learning + Reinforcement Learning Summer School
• 2020.05 I'm excited to join Prof. Sanja Fidler's group to do a summer intern at NVIDIA, Toronto.
• 2020.04 Attending ICLR 2020 and presenting my work on bilinear zero-sum games

Education

• 2017.09-2021.08 Ph.D. of Computer Science, University of Waterloo, Waterloo, Canada. Supervisors: Pascal Poupart and Yaoliang Yu
• 2015.08-2016.06 Master of Physics, University of Waterloo/Perimeter Institute, Waterloo, Canada. Supervisor: Freddy Cachazo
• 2011.09-2015.07 Bachelor of Physics, University of Science and Technology of China, Hefei, China. GPA—4.13/4.3

Awards

• ICML 2022 Top 10% Reviewer
• Cheriton Scholarship
• Guo Moruo Scholarship

Selected Publications

Large Language Model:
• 2025.01 MiniMax, Aonian Li, Bangwei Gong, Bo Yang, Boji Shan, Chang Liu, Cheng Zhu, Chunhao Zhang, Congchao Guo, Da Chen, Dong Li, Enwei Jiao, Gengxin Li, Guojun Zhang, Haohai Sun, Houze Dong, Jiadai Zhu, Jiaqi Zhuang, Jiayuan Song, Jin Zhu, Jingtao Han, Jingyang Li, Junbin Xie, Junhao Xu, Junjie Yan, Kaishun Zhang, Kecheng Xiao, Kexi Kang, Le Han, Leyang Wang, Lianfei Yu, Liheng Feng, Lin Zheng, Linbo Chai, Long Xing, Meizhi Ju, Mingyuan Chi, Mozhi Zhang, Peikai Huang, Pengcheng Niu, Pengfei Li, Pengyu Zhao, Qi Yang, Qidi Xu, Qiexiang Wang, Qin Wang, Qiuhui Li, Ruitao Leng, Shengmin Shi, Shuqi Yu, Sichen Li, Songquan Zhu, Tao Huang, Tianrun Liang, Weigao Sun, Weixuan Sun, Weiyu Cheng, Wenkai Li, Xiangjun Song, Xiao Su, Xiaodong Han, Xinjie Zhang, Xinzhu Hou, Xu Min, Xun Zou, Xuyang Shen, Yan Gong, Yingjie Zhu, Yipeng Zhou, Yiran Zhong, Yongyi Hu, Yuanxiang Fan, Yue Yu, Yufeng Yang, Yuhao Li, Yunan Huang, Yunji Li, Yunpeng Huang, Yunzhi Xu, Yuxin Mao, Zehan Li, Zekang Li, Zewei Tao, Zewen Ying, Zhaoyang Cong, Zhen Qin, Zhenhua Fan, Zhihang Yu, Zhuo Jiang, Zijia Wu. MiniMax-01: Scaling Foundation Models with Lightning Attention.
  [arxiv][GitHub][model][API]
  
  Transfer Learning (Domain Generalization/Adaptation/Multitask):
• 2023.05 Yifei He, Shiji Zhou, Guojun Zhang, Hyokun Yun, Yi Xu, Belinda Zeng, Trishul Chilimbi, Han Zhao. Robust Multi-Task Learning with Excess Risks. ICML 2024.
  [ICML][arxiv]
• 2023.12 (DG) Sobhan Hemati, Mahdi Beitollahi, Amir Hossein Estiri, Bassel Al Omari, Xi Chen, Guojun Zhang. Beyond Loss Functions: Exploring Data-Centric Approaches with Diffusion Model for Domain Generalization. TMLR 2024.
  [arxiv][TMLR]
• 2023.11 (OOD) Ahmad Rashid, Serena Hacker, Guojun Zhang, Agustinus Kristiadi, Pascal Poupart. Preventing Arbitrarily High Confidence on Far-Away Data in Point-Estimated Discriminative Neural Networks. AISTATS 2024.
  [AISTATS][arxiv]
• 2023.07 (DG) Sobhan Hemati*, Guojun Zhang*, Amir Estiri, Xi Chen. Understanding Hessian Alignment for Domain Generalization . ICCV 2023.
  [ICCV][arxiv][code][poster][video]
• 2022.11 (DA) Ehsan Imani, Guojun Zhang, Jun Luo, Pascal Poupart, Philip Torr, Yangchen Pan. Label Alignment Regularization for Distribution Shift. JMLR 2024
  [arxiv][JMLR]
• 2022.01 (DA) David Acnua, Marc Law, Guojun Zhang, Sanja Fidler. Domain Adversarial Training: A Game Perspective. ICLR 2022.
  [ICLR][arxiv]
• 2021.06 (DG) Guojun Zhang, Han Zhao, Yaoliang Yu and Pascal Poupart. Quantifying and Improving Transferability in Domain Generalization. NeurIPS 2021.
  [NeurIPS][arxiv][code][video]
• 2020.10 (DA) David Acuna, Guojun Zhang, Marc Law and Sanja Fidler. f-Domain Adversarial Learning: Theory and Algorithms. ICML 2021 (spotlight).
  [ICML][arxiv][code]
  
  Federated learning and privacy:
• 2024.08 Yasser H. Khalil, Amir Hossein Estiri, Mahdi Beitollahi, Nader Asadi, Sobhan Hemati, Xu Li, Guojun Zhang, Xi Chen. DFML: Decentralized Federated Mutual Learning. TMLR 2024.
[TMLR][arxiv]
• 2023.12 Mohsin Hasan, Guojun Zhang, Kaiyang Guo, Xi Chen, Pascal Poupart. Calibrated One Round Federated Learning with Bayesian Inference in the Predictive Space. AAAI 2024.
[AAAI][arxiv][code]
• 2023.08 Guojun Zhang, Mahdi Beitollahi, Alex Bie, Xi Chen. Understanding the Role of Layer Normalization in Label-Skewed Federated Learning. TMLR 2024.
  [TMLR][arxiv][code]
• 2022.11 Alex Bie, Gautam Kamath, Guojun Zhang. Private GANs, Revisited. TMLR 2023 ; NeurIPS 2022 SyntheticData4ML workshop; TPDP 2023.
  [TMLR][NeurIPS workshop][arxiv]
• 2022.06 Artur Back de Luca*, Guojun Zhang*, Xi Chen, Yaoliang Yu. Mitigating Data Heterogeneity in Federated Learning with Data Augmentation.
  [arxiv][code]
• 2022.02 Guojun Zhang, Saber Malekmohammadi, Xi Chen and Yaoliang Yu. Proportional Fairness in Federated Learning. TMLR 2023.
  [TMLR][arxiv][slides][code]
• 2020.06 Zeou Hu, Kiarash Shaloudegi, Guojun Zhang and Yaoliang Yu. Federated Learning meets Multi-objective Optimization. IEEE Transactions on Network Science and Engineering 2022.
  [IEEE TNSE][arxiv][code]
  
  Contrastive Learning:
• 2021.12 Yiwei Lu*, Guojun Zhang*, Sun Sun, Hongyu Guo and Yaoliang Yu. $f$-MICL: Understanding and Generalizing InfoNCE-based Contrastive Learning. TMLR 2023. NeurIPS 2021 workshop on self-supervised learning (contributed talk).
  [TMLR][NeurIPS workshop][poster]
  
  Minimax Optimization and Smooth Games:
• 2022.01 Guojun Zhang, Pascal Poupart and Yaoliang Yu. Optimality and Stability in Non-Convex Smooth Games. JMLR 2022.
  [JMLR][arxiv][bib]
• 2020.06 Guojun Zhang, Kaiwen Wu, Pascal Poupart and Yaoliang Yu. Newton-type Methods for Minimax Optimization. ICML 2021 workshop for ''Beyond first-order methods in ML systems.''
  [arxiv][workshop][code]
• 2019.08 Guojun Zhang and Yaoliang Yu. Convergence of Gradient Methods on Bilinear Zero-Sum Games.
  ICLR 2020, also presented at NeurIPS workshop SGO&ML 2019
  [ICLR][arxiv][workshop][code]
  
  Theoretical Physics:
• 2017.05 Sebastian Mizera and Guojun Zhang (α-β order). String-theoretical Deformation of the Parke-Taylor Factor. Phys. Rev. D 96 (2017) no.6, 066016.
  [PRD][arxiv]
• 2016.12 Humberto Gomez, Sebastian Mizera and Guojun Zhang (α-β order). CHY Loop Integrands from Holomorphic Forms. JHEP 1703 (2017) 092.
  [JHEP][arxiv]
• 2016.09 Freddy Cachazo, Sebastian Mizera and Guojun Zhang (α-β order). Scattering Equations: Real Solutions and Particles on a Line. JHEP 1703 (2017) 151.
  [JHEP][arxiv]
• 2015.05 Xin Wang, Guojun Zhang and Min-xin Huang, New Exact Quantization Condition for Toric Calabi-Yau Geometries. Phys. Rev. Lett. 115, 121601 (2015).
  [PRL][arxiv]

Academic Services

PC Member: IJCAI
Conference Reviewer: NeurIPS, ICML, ICLR, AISTATS, CoRL, AAAI
Program Committee: FL-IJCAI 2022, FL@FM-IJCAI 2024, FL@FM-ICME 2024
Journal Reviewer: Journal of Scientific Computing, TMLR, T-PAMI, ACM Transactions on Intelligent Systems and Technology
Workshop Reviewer: SyntheticData4ML Workshop NeurIPS 2022.

Misc

• General: How to read a paper; How to write an ML paper; How to write math; How to give a talk
• Math classics:
  • Analysis: Baby Rudin; Child Rudin (on it now); Adult Rudin
  • Probability and statistics: Stochastic Processes; Dembo's note
  • Matrices: Random matrix theory; Matrix analysis
  • Algebra: Invariant theory
  • (to be continued, suggestions are welcome)