Internship project topic: Inductive bias of Transformer
|
Research Interests
- Deep learning
- Over-parametrized models
|
Biography
Yongtao Wu received a Bachelor’s degree inTelecommunication Engineering from Sun Yat-sen University in 2020, and a Master’s degree in Machine Learning from KTH Royal Institute of Technology in 2022. From July to September 2022 he was an intern at LIONS. In September 2022 he started his PhD at LIONS.
On the Convergence of Encoder-only Shallow Transformers
Y. Wu; F. Liu; G. Chrysos; V. Cevher
2023. 37th Annual Conference on Neural Information Processing Systems, New Orleans, USA, December 10-16. 2023. Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study
Y. Wu; Z. Zhu; F. Liu; G. Chrysos; V. Cevher
2022. 36th Conference on Neural Information Processing Systems (NeurIPS), New Orleans, USA, November 28 – December 3, 2022.