Yao Xiao


2023

pdf bib
Decomposed Prompt Tuning via Low-Rank Reparameterization
Yao Xiao | Lu Xu | Jiaxi Li | Wei Lu | Xiaoli Li
Findings of the Association for Computational Linguistics: EMNLP 2023

While prompt tuning approaches have achieved competitive performance with high efficiency, we observe that they invariably employ the same initialization process, wherein the soft prompt is either randomly initialized or derived from an existing embedding vocabulary. In contrast to these conventional methods, this study aims to investigate an alternative way to derive soft prompt. Our empirical studies show that the soft prompt typically exhibits a low “intrinsic rank” characteristic. With such observations, we propose decomposed prompt tuning, a novel approach that utilizes low-rank matrices to initialize the soft prompt. Through the low-rank reparameterization, our method significantly reduces the number of trainable parameters while maintaining effectiveness. Experimental results on the SuperGLUE benchmark in both high-resource and low-resource scenarios demonstrate the effectiveness of the proposed method.

2015

pdf bib
Verbal and Nonverbal Clues for Real-life Deception Detection
Verónica Pérez-Rosas | Mohamed Abouelenien | Rada Mihalcea | Yao Xiao | CJ Linton | Mihai Burzo
Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing