Yixiao HUANG


Kowloon, Hong Kong, China


I’m Yixiao HUANG (Chinese name: 黄一笑). I received my Bachelor’s degree in Informaton Engineering in June, 2023 from Department of Electrical Engineering, City University of Hong Kong. I’m generally intersted in ML/DL Theory and non-convex optimization.

Currently I’m a research assistant at City University of Hong Kong supervised by Dr. Chen Liu where I’m investigating the reason behind the superior performance of adaptive methods (e.g., Adam) in language models. I was fortunate to work with Dr. Rosa CHAN who patiently guided me to start my research adventure. I also had a great time at University of Michigan, working with Dr. Samet Oymak on the theoretical foundations of self-attention.


Apr 3, 2024 I will be joining UC Berkeley EECS as a first-year PhD student in the upcoming Fall. See you at Berkeley!
Feb 22, 2024 Our paper on the relationship between self-attention and markov models is available on arXiv!
Jan 10, 2024 Our paper on the implicit bias of next-token prediction has been accepted by AISTATS 2024! It’s also available on arXiv.

selected publications

* indicates equal contribution
  1. ArXiv
    From Self-Attention to Markov Models: Unveiling the Dynamics of Generative Transformers
    M. Emrullah IldizYixiao HuangYingcong Li, and 2 more authors
    arXiv preprint arXiv:2402.13512, 2024
  2. AISTATS 2024
    Mechanics of Next Token Prediction with Self-Attention
    Yingcong Li*Yixiao Huang*M. Emrullah Ildiz, and 2 more authors
    Accepted by International Conference on Artificial Intelligence and Statistics (AISTATS 2024), 2024
  3. In submission
    Sparse-PGD: An Effective and Efficient Attack for ł_0 Bounded Adversarial Perturbation
    Xuyang Zhong, Yixiao Huang, and Chen Liu
    In submission, 2023