Kaiyuan Zhang How to pronounce? PhD candidate Purdue University Office: 305 N. University Street West Lafayette, IN, 47907-2107 Email: [email protected] Google Scholar | dblp | Twitter | GitHub | LinkedIn

I am a PhD candidate of Computer Science at Purdue University, co-advised by Prof. Ninghui Li and Prof. Xiangyu Zhang. I am supported by the Amazon Fellowship and the Bilsland Dissertation Fellowship.

I am obsessed with simple solutions, passionate about deep understanding, and driven to build proactive systems. My research interests focus on security and privacy in machine learning. Currently, I study the security and privacy of large language models from an adversarial perspective, to understand and mitigate long-term threats and build proactive secure systems.

I will join Microsoft Research (NYC) as a Research Intern in Summer 2025. I was a research intern at Amazon AWS GenAI and NEC Labs America during my PhD. Before joining Purdue, I spent a wonderful year as a visiting graduate student at the University of Illinois at Urbana-Champaign, co-advised by Prof. Tao Xie and Prof. Tianyin Xu. I received an M.S. in Computer Science from the University of Texas at Dallas, advised by Prof. Wei Yang. I was a full-time research assistant advised by Prof. Wei Chen for 1.5 years at Zhejiang University before starting grad school.

Research Opportunities: I am always happy to discuss and brainstorm; if you’re a researcher/student interested in security, privacy, or GenAI (LLM agents most recent), please reach out to [email protected].

Office Hours: I owe a lot to my patient research mentors who generously spent their time helping me along the way. From September 2024, I will dedicate 2 hours each week to offer mentorship to undergrad and grad students on any topics you’d like to discuss. If you’re interested, please fill out the AMA form.

Selected Works [Full List] (*: equal contribution)

• LLM Agents Should Employ Security Principles
  Kaiyuan Zhang, Zian Su, Pin-Yu Chen, Elisa Bertino, Xiangyu Zhang, Ninghui Li
  CoRR 2025
  [paper] [slides] [twitter]
• SOFT: Selective Data Obfuscation for Protecting LLM Fine-tuning against Membership Inference Attacks
  Kaiyuan Zhang, Siyuan Cheng, Hanxi Guo, Yuetian Chen, Zian Su, Shengwei An, Yuntao Du, Charles Fleming, Ashish Kundu, Xiangyu Zhang, Ninghui Li
  The 34th USENIX Security Symposium (Security’25), Seattle, WA, August 2025
  [paper] [code (coming soon)]
• CENSOR: Defense Against Gradient Inversion via Orthogonal Subspace Bayesian Sampling
  Kaiyuan Zhang, Siyuan Cheng, Guangyu Shen, Bruno Ribeiro, Shengwei An, Pin-Yu Chen, Xiangyu Zhang, Ninghui Li
  The 32nd Network and Distributed System Security Symposium (NDSS’25), San Diego, CA, Feb 2025
  [paper] [code] [slides] [twitter] [website]
• Exploring the Orthogonality and Linearity of Backdoor Attacks
  Kaiyuan Zhang*, Siyuan Cheng*, Guangyu Shen, Guanhong Tao, Shengwei An, Anuran Makur, Shiqing Ma, Xiangyu Zhang
  The 45th IEEE Symposium on Security and Privacy (Oakland’24), San Francisco, CA, May 2024
  [paper] [code] [slides] [video] [poster] [twitter] [website]
• FLIP: A Provable Defense Framework for Backdoor Mitigation in Federated Learning
  Kaiyuan Zhang, Guanhong Tao, Qiuling Xu, Siyuan Cheng, Shengwei An, Yingqi Liu, Shiwei Feng, Guangyu Shen, Pin-Yu Chen, Shiqing Ma, Xiangyu Zhang
  The Eleventh International Conference on Learning Representations (ICLR’23), Kigali, Rwanda, May 2023
  Best Paper Award 🏆 in ECCV 2022 Workshop on Adversarial Robustness in the Real World (AROW’22), Virtually, October 2022
  [paper] [code] [slides] [media coverage]

Selected Honors & Awards

• Bilsland Dissertation Fellowship, 2026
• Apple Scholar Nominee, 2025
• NDSS Symposium Fellowship, 2025
• Graduate Student Travel Award, Purdue University, 2025
• Amazon Fellowship, 2024
• IEEE S&P Travel Grant, 2024
• InnovatED Award, Purdue University, 2023
• Best Paper Award, ECCV AROW Workshop, 2022
• Summer Research Grant, Purdue University, 2022
• ACM CCS Student Conference Grant, 2020
• National Scholarship, Ministry of Education of China (top 0.2% undergraduates nationwide), 2016

Teaching

Teaching Assistant

• CS 59200: AI And Security, Grad Level (Head TA), Prof. Xiangyu Zhang, Purdue University, Fall 2024.
• CS 37300: Data Mining and Machine Learning, Undergrad Level, Prof. Bruno Ribeiro and Prof. Dan Goldwasser, Purdue University, Fall 2022.
  
• CS 37300: Data Mining and Machine Learning, Undergrad Level, Prof. Steve Hanneke, Purdue University, Spring 2022.
  
• CS 47300: Web Information Search And Management, Undergrad Level, Prof. Hisham R. Benotman, Purdue University, Fall 2021.
  
• CS 18200: Foundations of Computer Science, Undergrad Level, Prof. Andres Bejarano, Purdue University, Summer 2021.
• CS 37300: Data Mining and Machine Learning, Undergrad Level, Prof. Jean Honorio, Purdue University, Spring 2021.
  
• CS 47300: Web Information Search And Management, Undergrad Level, Prof. Christopher W. Clifton, Purdue University, Fall 2020.
  
• Visual Analytics, Undergrad Level, Prof. Wei Chen, Zhejiang University, Summer 2018.

Guest Lectures

• CS 59200: AI And Security. Grad Level, Purdue University. Topic: LLM Memorization, invited by Prof. Xiangyu Zhang, Fall 2024.
• COMPSCI 360: Introduction to Computer and Network Security. Undergrad Level, University of Massachusetts, Amherst. Topic: Backdoor Attacks and Defenses in Machine Learning, invited by Prof. Shiqing Ma, Fall 2023.
  
• CS 52900: Security Analytics. Grad Level, Purdue University. Topic: Backdoor Attacks and Defenses in Machine Learning, invited by Prof. Berkay Celik, Fall 2022.
  

Professional Services

• Organizers
  • The 1st Workshop on Backdoor Attacks and Defenses in Machine Learning (BANDS) at ICLR, 2023
  • Machine Learning & Security Seminar at Purdue University, 2021, 2022, 2023
• Program Committee Member
  • International Conference on Learning Representations (ICLR), 2024, 2025
  • International Conference on Machine Learning (ICML), 2023, 2024, 2025
  • Advances in Neural Information Processing Systems (NeurIPS), 2023, 2024, 2025
  • IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024
  • IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023
  • ICLR Workshop on Privacy Regulation and Protection in Machine Learning, 2024
  • ICML Workshop on Federated Learning and Analytics in Practice, 2023
  • ICML Workshop on Adversarial Machine Learning Frontiers, 2022
• Journal Reviewers
  • IEEE Transactions on Information Forensics and Security (TIFS), 2023, 2024, 2025
  • IEEE Transactions on Dependable and Secure Computing (TDSC), 2023
  • ACM Transactions on Privacy and Security (TOPS), 2023
• External Reviewers
  • ACM Conference on Computer and Communications Security (CCS), 2021, 2022, 2023, 2024, 2025
  • USENIX Security Symposium (Security), 2024, 2025
  • IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 2025
  • ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), 2024
  • IEEE/ACM International Conference on Automated Software Engineering (ASE), 2023
• Student Volunteers
  • ICML 2021, ICLR 2021, ECOOP/ISSTA 2021, CCS 2020, SIGMOD 2020, etc.

Personal

• Kaiyuan Zhang’s name is pronounced as
• I love basketball and have been playing weekly for over ten years.
• I’m a big fan of tennis, especially the moments of hitting the ball with the right amount of spin and force.
• I enjoy kayaking, quietly exploring a creek, enjoy views that can’t be seen from shore.
• I also like reading, drinking, cooking, traveling, and, of course, boba :-)
• I consider myself lucky to be guided by helpful advice and instructions. Here are some notes I collect from time to time.