Large Language Models | Jiangjie Chen

Seed-Thinking-v1.5: Advancing Superb Reasoning Models with Reinforcement Learning

We introduce Seed-Thinking-v1.5, a Mixture-of-Experts (MoE) model with a relatively small size, featuring 20B activated and 200B total parameters, capable of reasoning through thinking before responding, resulting in improved performance on a widerange of benchmarks.

ByteDance Seed

Seed-Thinking-v1.5: Advancing Superb Reasoning Models with Reinforcement Learning

DAPO: An Open-source LLM Reinforcement Learning System At Scale

We introduce DAPO, a Decoupled Clip and Dynamic sAmpling Policy Optimization algorithm, and fully open-source a state-of-the-art large-scale RL system that achieves 50 points on AIME 2024 using Qwen2.5-32B base model.

Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Weinan Dai, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei-Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, Mingxuan Wang

PowerAttention: Exponentially Scaling of Receptive Fields for Effective Sparse Attention

We introduce PowerAttention, a novel sparse attention design that facilitates effective and complete context extension through the theoretical analysis.

Lida Chen, Dong Xu, Chenxin An, Xintao Wang, Yikai Zhang, Jiangjie Chen, Zujie Liang, Feng Wei, Jiaqing Liang, Yanghua Xiao, Wei Wang

PowerAttention: Exponentially Scaling of Receptive Fields for Effective Sparse Attention

DEEPER Insight into Your User: Directed Persona Refinement for Dynamic Persona Modeling

We introduce DEEPER, a novel approach for dynamic persona modeling that enables continual persona optimization.

Aili Chen, Chengyu Du, Jiangjie Chen, Jinghan Xu, Yikai Zhang, Siyu Yuan, Zulong Chen, Liangyue Li, Yanghua Xiao

DEEPER Insight into Your User: Directed Persona Refinement for Dynamic Persona Modeling

CoSER: Coordinating LLM-Based Persona Simulation of Established Roles

We introduce CoSER, a collection of a high-quality dataset, open models, and an evaluation protocol towards effective RPLAs of established characters.

Xintao Wang, Heng Wang, Yifei Zhang, Xinfeng Yuan, Rui Xu, Jen-Tse Huang, Siyu Yuan, Haoran Guo, Jiangjie Chen, Wei Wang, Yanghua Xiao, Shuchang Zhou

CoSER: Coordinating LLM-Based Persona Simulation of Established Roles

EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction

We proposes EASYTOOL, a method that simplifies tool documentation into concise instructions, improving tool use by language models.

Siyu Yuan, Kaitao Song, Jiangjie Chen, Xu Tan, Yongliang Shen, Ren Kan, Dongsheng Li, Deqing Yang

EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction

EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithms

We introduce EvoAgent, a method using evolutionary algorithms to automatically expand expert agents into multi-agent systems, enhancing the task-solving capabilities of large language model-based agents without additional human design.

Siyu Yuan, Kaitao Song, Jiangjie Chen, Xu Tan, Dongsheng Li, Deqing Yang

EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithms

Revealing the Barriers of Language Agents in Planning

We reveal the two key factors that hinder language agents from achieving human-level planning.

Jian Xie, Kexun Zhang, Jiangjie Chen, Siyu Yuan, Kai Zhang, Yikai Zhang, Lei Li, Yanghua Xiao

Revealing the Barriers of Language Agents in Planning

SelfGoal: Your Language Agents Already Know How to Achieve High-level Goals

We introduce SelfGoal, an automatic approach that enhances language agents’ capabilities to achieve high-level goals with limited instructions and delayed feedback by adaptively breaking down goals into practical subgoals.

Ruihan Yang, Jiangjie Chen, Yikai Zhang, Siyu Yuan, Aili Chen, Kyle Richardson, Yanghua Xiao, Deqing Yang

SelfGoal: Your Language Agents Already Know How to Achieve High-level Goals

SEGMENT+: Long Text Processing with Short-Context Language Models

We introduce SEGMENT+, a framework that enables LMs to efficiently handle extended inputs within limited context windows, improving performance in long-document tasks through structured notes and a filtering module.

Wei Shi, Shuang Li, Kerun Yu, Jinglei Chen, Zujie Liang, Xinhui Wu, Yuxi Qian, Feng Wei, Bo Zheng, Jiaqing Liang, Jiangjie Chen, Yanghua Xiao

SEGMENT+: Long Text Processing with Short-Context Language Models