Le Yu

Le Yu

Ph.D. student at Beihang University

Le Yu (于乐)’s Homepage

Brief Bio

I am an Algorithm Engineer of Qwen Team, Alibaba Group. I am primarily responsible for the alignment of the Qwen series models during the post-training phase. I received the Ph.D. degree in the School of Computer Science and Engineering at Beihang University in 2024, under the supervision of Prof. Weifeng Lv. I received the B.S. degree in the School of Computer Science and Engineering at Beihang University in 2019.

I focus on designing neural networks for representation learning on spatio-temporal data (including texts, sequences, and graphs) and using the learned representations to empower various scenarios. My research interests include (large) language models, graph neural networks, recommender systems, etc. I have published several peer-reviewed papers in top venues like ICML, NeurIPS, KDD, WWW, IEEE TKDE, ACM TOIS, EMNLP, AAAI, and IJCAI. I have also served as a PC member/reviewer for conferences/journals like ICLR, AAAI, LOG, ACM TOIS, ACM TKDD, IEEE TKDE, IEEE TNNLS, and Neural Networks.

News

[10/2024] One paper about delta parameter analysis was available on arXiv.
[08/2024] One paper about merging LLMs was available on arXiv.
[06/2024] I successfully defended my doctoral dissertation.
[06/2024] One paper was accepted by ACM TOIS.
[05/2024] One paper was accepted by ICML 2024.
[05/2024] One paper was accepted by KSEM 2024.
[10/2023] One paper was accepted by Findings of EMNLP 2023.
[09/2023] One paper was accepted by NeurIPS 2023.
[08/2023] Two papers were accepted by IEEE TKDE.
[04/2023] One paper was accepted by IJCAI 2023.
[12/2022] One paper was accepted by IEEE TKDE.
[11/2022] One paper was accepted by AAAI 2023.
[03/2022] One paper was accepted by IEEE TKDE.
[01/2022] One paper was accepted by WWW 2022.
[01/2022] One paper was accepted by Knowledge-Based Systems.
[04/2021] One paper was accepted by Knowledge-Based Systems.
[09/2020] One paper was accepted by Neurocomputing.
[05/2020] One paper was accepted by KDD 2020.

Publications

In the Year of 2024

[arXiv1] Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, Yongbin Li. Extend Model Merging from Fine-Tuned to Pre-Trained Large Language Models via Weight Disentanglement. arXiv 2024. Paper Code

[arXiv2] Qiaoyu Tang, Le Yu, Bowen Yu, Hongyu Lin, Keming Lu, Yaojie Lu, Xianpei Han, Le Sun. A Unified View of Delta Parameter Editing in Post-Trained Large-Scale Models. arXiv 2024. Paper Code

[C1] Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, Yongbin Li. Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch. ICML 2024. Paper Code

[C2] Zihang Liu, Le Yu, Weimiao Li, Tongyu Zhu, Leilei Sun. Multi-mode Spatial-Temporal Data Modeling with Fully Connected Networks. KSEM 2024. Paper Code

[J1] Le Yu, Zihang Liu, Leilei Sun, Bowen Du, Chuanren Liu, Weifeng Lv. Continuous-Time User Preference Modelling for Temporal Sets Prediction. IEEE TKDE. Paper Code

[J2] Tao Zou, Le Yu, Leilei Sun, Bowen Du, Deqing Wang, Fuzhen Zhuang. Event-based Dynamic Graph Representation Learning for Patent Application Trend Prediction. IEEE TKDE. Paper Code

[J3] Tao Zou, Le Yu, Junchen Ye, Leilei Sun, Bowen Du, Deqing Wang. Adaptive Taxonomy Learning and Historical Patterns Modelling for Patent Classification. ACM TOIS. Paper Code

In the Year of 2023

[C1] Tao Zou*, Le Yu*, Yifei Huang, Leilei Sun, Bowen Du. Pretraining Language Models with Text-Attributed Heterogeneous Graphs. EMNLP 2023 (Findings). Paper Code

[C2] Le Yu, Leilei Sun, Bowen Du, Weifeng Lv. Towards Better Dynamic Graph Learning: New Architecture and Unified Library. NeurIPS 2023 (Poster). Paper Code

[arXiv1] Le Yu. An Empirical Evaluation of Temporal Graph Benchmark. arXiv 2023. Paper Code

[C3] Le Yu, Zihang Liu, Tongyu Zhu, Leilei Sun, Bowen Du, Weifeng Lv. Predicting Temporal Sets with Simplified Fully Connected Networks. AAAI 2023 (Oral). Paper Code

[C4] Xiaodong Lu, Shuo Ji, Le Yu, Leilei Sun, Bowen Du, Tongyu Zhu. Continuous-Time Graph Learning for Cascade Popularity Prediction. IJCAI 2023. Paper Code

[J1] Le Yu, Leilei Sun, Bowen Du, Tongyu Zhu, Weifeng Lv. Label-Enhanced Graph Neural Network for Semi-supervised Node Classification. IEEE TKDE. Paper Code

[J2] Le Yu, Leilei Sun, Bowen Du, Chuanren Liu, Weifeng Lv, Hui Xiong. Heterogeneous Graph Representation Learning with Relation Awareness. IEEE TKDE. Paper Code

In the Year of 2022

[C1] Le Yu, Guanghui Wu, Leilei Sun, Bowen Du, Weifeng Lv. Element-guided Temporal Graph Representation Learning for Temporal Sets Prediction. Proceedings of the ACM Web Conference 2022 (WWW 2022). Paper Code

[J1] Xuxiang Ta, Zihan Liu, Xiao Hu, Le Yu, Leilei Sun, Bowen Du. Adaptive Spatio-temporal Graph Neural Network for Traffic Forecasting. Knowledge-Based Systems. Paper Code

In the Year of 2021

[J1] Le Yu, Bowen Du, Xiao Hu, Leilei Sun, Liangzhe Han, Weifeng Lv. Deep Spatio-temporal Graph Convolutional Network for Traffic Accident Prediction. Neurocomputing. Paper Code

[J2] Nannan Shi, Le Yu, Leilei Sun, Lihua Wang, Chunming Lin, Ruixing Zhang. Deep Heterogeneous Network for Temporal Set Prediction. Knowledge-Based Systems. Paper Code

In the Year of 2020

[C1] Le Yu, Leilei Sun, Bowen Du, Chuanren Liu, Hui Xiong, Weifeng Lv. Predicting Temporal Sets with Deep Neural Networks. (KDD 2020). Paper Code

[arXiv1] Le Yu, Leilei Sun, Bowen Du, Chuanren Liu, Weifeng Lv, Hui Xiong. Hybrid Micro/Macro Level Convolution for Heterogeneous Graph Learning. arXiv 2020. Paper Code

Education

  • Sept. 2019 - Jun. 2024 Ph.D., School of Computer Science and Engineering, Beihang University
    Supervisor: Prof. Weifeng Lv
  • Sept. 2015 - Jun. 2019 B.S., School of Computer Science and Engineering, Beihang University

Honors Awarded

  • 2020 & 2023 National Scholarship for Graduate Students, Beihang University
  • 2023 Huawei Scholarship, Beihang University
  • 2023 Guorui Scholarship, Beihang University
  • 2019 Graduate Freshman Scholarship, Beihang University
  • Merit Student & Outstanding Graduate Student (2020, 2022, 2023), Beihang University

Invited Talks

Professional Services

  • PC Member (or Reviewer):
    ICLR 2025; AAAI 2023, 2024, 2025; LOG 2022, 2023, 2024; ACM TOIS; ACM TKDD; IEEE TKDE; IEEE TNNLS; Neural Networks

  • Assistant PC Member (or Reviewer):
    SIGIR 2020, 2021, 2022, 2023; KDD 2020, 2021, 2022, 2023; AAAI 2020, 2021, 2022; IJCAI 2021, 2022, 2023; CIKM 2020, 2021; ICDM 2021; WSDM 2021, 2022; IEEE TITS; ACM TIST; Neurocomputing; Knowledge-Based Systems

Experiences

  • Jun. 2023 – Aug. 2024, Research Intern, Tongyi Laboratory, Alibaba Group
  • Sept. 2024 – now., Algorithm Engineer, Qwen Team, Alibaba Group