Dataset

Character is Destiny: Can Large Language Models Simulate Persona-Driven Decisions in Role-Playing?

We evaluate the potential of LLMs to make decisions as literary characters, using a new dataset and improved method that enhances decision-making accuracy, with future work and resources to be shared publicly.

Rui Xu, Xintao Wang, Jiangjie Chen, Siyu Yuan, Xinfeng Yuan, Jiaqing Liang, Zulong Chen, Xiaoqing Dong, Yanghua Xiao

Character is Destiny: Can Large Language Models Simulate Persona-Driven Decisions in Role-Playing?

Evaluating Character Understanding of Large Language Models via Character Profiling from Fictional Works

We propose a new approach to evaluate LLMs’ understanding of fictional characters by summarizing character profiles, using a specially constructed dataset, and shows promising results for their application in role-playing agents.

Xinfeng Yuan, Siyu Yuan, Yuhan Cui, Tianhe Lin, Xintao Wang, Rui Xu, Jiangjie Chen, Deqing Yang

Evaluating Character Understanding of Large Language Models via Character Profiling from Fictional Works

Translate Meanings, Not Just Words: IdiomKB's Role in Optimizing Idiomatic Translation with Language Models

We propose a multilingual idiom KB (IdiomKB) developed using LLMs to facilitate better idiomatic translation by smaller models by retrieving idioms’ figurative meanings.

Shuang Li, Jiangjie Chen, Siyu Yuan, Xinyi Wu, Hao Yang, Shimin Tao, Yanghua Xiao

Translate Meanings, Not Just Words: IdiomKB's Role in Optimizing Idiomatic Translation with Language Models

Beneath Surface Similarity: Large Language Models Make Reasonable Scientific Analogies after Structure Abduction

We propose a scientific analogical reasoning benchmark with structure abduction, SCAR, and show that large language models make reasonable scientific analogies after structure abduction.

Siyu Yuan, Jiangjie Chen, Xuyang Ge, Yanghua Xiao, Deqing Yang

Beneath Surface Similarity: Large Language Models Make Reasonable Scientific Analogies after Structure Abduction

AnalogyKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Base

A million-scale analogy KB derived from existing KGs, to enable large language models to achieve analogical reasoning skills.

Siyu Yuan, Jiangjie Chen, Changzhi Sun, Jiaqing Liang, Yanghua Xiao, Deqing Yang

AnalogyKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Base

Distilling Script Knowledge from Large Language Models for Constrained Language Planning

We propose an over-generate-then-filter approach to improve large language models (LLMs) on constrained language planning, and use it to distill a novel constrained language planning dataset, CoScript.

Siyu Yuan, Jiangjie Chen, Ziquan Fu, Xuyang Ge, Soham Shah, Charles Robert Jankowski, Yanghua Xiao, Deqing Yang

Distilling Script Knowledge from Large Language Models for Constrained Language Planning

E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning

We benchmark knowledge-intensive analogical reasoning with human-annotated explanations.

Jiangjie Chen, Rui Xu, Ziquan Fu, Wei Shi, Zhongqiao Li, Xinbo Zhang, Changzhi Sun, Lei Li, Yanghua Xiao, Hao Zhou

E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning

CN-Probase: A Data-driven Approach for Large-scale Chinese Taxonomy Construction

We build a large-scale Chinese Taxonomy Knowledge Graph.

Jindong Chen, Ao Wang, Jiangjie Chen, Yanghua Xiao, Zhendong Chu, Jingping Liu, Jiaqing Liang, Wei Wang