I was a research engineer affiliated with Shanghai Ark NLP (Shark NLP), Shanghai AI Lab, leading a research team on text generation and long-range language modeling. Previously, I worked as a researcher at Bytedance AI Lab. I also interned at Google AI China Center. I spent seven years at Fudan University pursuing my bachelor and master degrees, under the supervision by Prof. Xiaoqing Zheng. My research focuses on text generation and generative models. Currently, I feel interested in (large) language models on their backbone architectures, pretraining and instruction tuning. Welcome to have a chat! Email: fengjt16@fudan.edu.cn

Softwares

• OpenICL : an open-source framework to facilitate research, development, and prototyping of in-context learning..
• ParaGen : a PyTorch deep learning framework for parallel sequence generation and beyond.

Research Highlights (Google Scholar)

• Long-Range Language Modeling
  • In-Context Learning with Many Demonstration Examples, arXiv:2302.04931
  • CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling, ICML 2023
• Text generation & neural machine translation
  • Optimizing Non-Autoregressive Transformers with Contrastive Learning, arXiv:2305.13667
  • CoNT: Contrastive Neural Text Generation, NeurIPS 2022 (Spotlight)
  • The Volctrans GLAT System: Non-autoregressive Translation Meets WMT21, WMT 2021 (De-En champion)
• Generative models for sequence modeling
  • Attentive MLP for Non-Autoregressive Generation, ECML 2023
  • DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models, ICLR 2023
  • Neural Phrase-to-Phrase Machine Translation, arXiv:1811.02172
• Contextualized word representation learning
  • Geometric Relationship between Word and Context Representations, AAAI 2018 (Oral)
  • Learning Context-Specific Word/Character Embeddings, AAAI 2017