Kaiyuan Zhang Kaiyuan Zhang
How to pronounce?

PhD candidate
Purdue University
Office: 305 N. University Street
West Lafayette, IN, 47907-2107
Email: [email protected]
Google Scholar | dblp | Twitter | GitHub | LinkedIn

I am a PhD candidate of Computer Science at Purdue University, fortunate to be advised by Prof. Ninghui Li and Prof. Xiangyu Zhang. I am supported by the Amazon Fellowship. My research interests focus on security and privacy in machine learning, recently in safe & responsible large language models.

I was a research intern at Amazon Bedrock and NEC Labs America during my PhD. Before joining Purdue, I spent a wonderful year as a visiting graduate student at the University of Illinois at Urbana-Champaign, co-advised by Prof. Tao Xie and Prof. Tianyin Xu. I received an M.S. in Computer Science from the University of Texas at Dallas, advised by Prof. Wei Yang. I was a full-time research assistant advised by Prof. Wei Chen for 1.5 years at Zhejiang University before starting grad school.

Research Opportunities: I am always happy to discuss and brainstorm; if you’re a researcher/student interested in privacy, security, or AI safety (LLMs most recent), please reach out to [email protected].

Office Hours: I owe a lot to my patient research mentors who generously spent their time helping me along the way. From Sept. 2024, I will dedicate 2 hours each week to offer mentorship to undergrad and grad students on any topics you’d like to discuss. If you’re interested, please fill out the AMA form.

Selected Works [Full List] [Google Scholar] [dblp] (*: equal contribution)

  • μKE: Matryoshka Unstructured Knowledge Editing of Large Language Models
    Zian Su*, Ziyang Huang*, Kaiyuan Zhang, Xiangyu Zhang
    Preprint 2025
    [paper] [code (coming soon)]
  • CENSOR: Defense Against Gradient Inversion via Orthogonal Subspace Bayesian Sampling
    Kaiyuan Zhang, Siyuan Cheng, Guangyu Shen, Bruno Ribeiro, Shengwei An, Pin-Yu Chen, Xiangyu Zhang, Ninghui Li
    Proceedings of the 32nd Network and Distributed System Security Symposium (NDSS 2025)
    [paper] [code] [slides] [twitter] [website]
  • ProSec: Fortifying Code LLMs with Proactive Security Alignment
    Xiangzhe Xu*, Zian Su*, Jinyao Guo, Kaiyuan Zhang, Zhenting Wang, Xiangyu Zhang
    Preprint 2024
    [paper] [code]
  • ProRec: Source Code Foundation Models are Transferable Binary Analysis Knowledge Bases
    Zian Su, Xiangzhe Xu, Ziyang Huang, Kaiyuan Zhang, Xiangyu Zhang
    Proceedings of the Thirty-Eighth Annual Conference on Neural Information Processing Systems (NeurIPS 2024)
    [paper] [code]
  • Exploring the Orthogonality and Linearity of Backdoor Attacks
    Kaiyuan Zhang*, Siyuan Cheng*, Guangyu Shen, Guanhong Tao, Shengwei An, Anuran Makur, Shiqing Ma, Xiangyu Zhang
    Proceedings of the 45th IEEE Symposium on Security and Privacy (S&P 2024)
    [paper] [code] [slides] [video] [poster] [twitter] [website]
  • FLIP: A Provable Defense Framework for Backdoor Mitigation in Federated Learning
    Kaiyuan Zhang, Guanhong Tao, Qiuling Xu, Siyuan Cheng, Shengwei An, Yingqi Liu, Shiwei Feng, Guangyu Shen, Pin-Yu Chen, Shiqing Ma, Xiangyu Zhang
    Proceedings of the Eleventh International Conference on Learning Representations (ICLR 2023)
    ECCV 2022 Workshop on Adversarial Robustness in the Real World (AROW 2022), Best Paper Award 🏆
    [paper] [code] [slides] [media coverage]

Selected Honors & Awards

  • Apple Scholar Nominee, 2025
  • Amazon Fellowship, 2024
  • InnovatED Award, Purdue University, 2023
  • Best Paper Award, ECCV AROW Workshop, 2022
  • Summer Research Grant, Purdue University, 2022
  • National Scholarship, 2016
  • Multiple Travel Grants: NDSS 2025, IEEE S&P 2024, ACM CCS 2020, etc.

Teaching

Guest Lecture

Teaching Assistant

Professional Services

  • Organizers
    • The 1st Workshop on Backdoor Attacks and Defenses in Machine Learning (BANDS) at ICLR, 2023
    • Machine Learning & Security Seminar at Purdue University, 2021, 2022, 2023
  • Program Committee Member
    • International Conference on Learning Representations (ICLR), 2024, 2025
    • International Conference on Machine Learning (ICML), 2023, 2024, 2025
    • Advances in Neural Information Processing Systems (NeurIPS), 2023, 2024, 2025
    • IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024
    • IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023
    • ICLR Workshop on Privacy Regulation and Protection in Machine Learning, 2024
    • ICML Workshop on Federated Learning and Analytics in Practice, 2023
    • ICML Workshop on Adversarial Machine Learning Frontiers, 2022
  • Journal Reviewers
    • IEEE Transactions on Information Forensics and Security (TIFS), 2023, 2024, 2025
    • IEEE Transactions on Dependable and Secure Computing (TDSC), 2023
    • ACM Transactions on Privacy and Security (TOPS), 2023
  • External Reviewers
    • ACM Conference on Computer and Communications Security (CCS), 2021, 2022, 2023, 2024, 2025
    • USENIX Security Symposium (USENIX Security), 2024, 2025
    • IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 2025
    • ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), 2024
    • IEEE/ACM International Conference on Automated Software Engineering (ASE), 2023
  • Student Volunteers
    • ICML 2021, ICLR 2021, ECOOP/ISSTA 2021, CCS 2020, SIGMOD 2020, etc.

Personal

  • Kaiyuan Zhang’s name is pronounced as
  • Basketball is my favorite, I’ve been playing it weekly for more than ten years.
  • I love tennis, especially the moments of hitting the ball with the right amount of spin and force.
  • I also enjoy kayaking, quietly exploring a creek, enjoy views that can’t be seen from shore.
  • I consider myself lucky to be guided by helpful advice and instructions. Here are some notes I collect from time to time.