Xinjie Zhou


2023

pdf bib
KBioXLM: A Knowledge-anchored Biomedical Multilingual Pretrained Language Model
Lei Geng | Xu Yan | Ziqiang Cao | Juntao Li | Wenjie Li | Sujian Li | Xinjie Zhou | Yang Yang | Jun Zhang
Findings of the Association for Computational Linguistics: EMNLP 2023

Most biomedical pretrained language models are monolingual and cannot handle the growing cross-lingual requirements. The scarcity of non-English domain corpora, not to mention parallel data, poses a significant hurdle in training multilingual biomedical models. Since knowledge forms the core of domain-specific corpora and can be translated into various languages accurately, we propose a model called KBioXLM, which transforms the multilingual pretrained model XLM-R into the biomedical domain using a knowledge-anchored approach. We achieve a biomedical multilingual corpus by incorporating three granularity knowledge alignments (entity, fact, and passage levels) into monolingual corpora. Then we design three corresponding training tasks (entity masking, relation masking, and passage relation prediction) and continue training on top of the XLM-R model to enhance its domain cross-lingual ability. To validate the effectiveness of our model, we translate the English benchmarks of multiple tasks into Chinese. Experimental results demonstrate that our model significantly outperforms monolingual and multilingual pretrained models in cross-lingual zero-shot and few-shot scenarios, achieving improvements of up to 10+ points.

2016

pdf bib
Cross-Lingual Sentiment Classification with Bilingual Document Representation Learning
Xinjie Zhou | Xiaojun Wan | Jianguo Xiao
Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

pdf bib
User Embedding for Scholarly Microblog Recommendation
Yang Yu | Xiaojun Wan | Xinjie Zhou
Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

pdf bib
Attention-based LSTM Network for Cross-Lingual Sentiment Classification
Xinjie Zhou | Xiaojun Wan | Jianguo Xiao
Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing

2013

pdf bib
Collective Opinion Target Extraction in Chinese Microblogs
Xinjie Zhou | Xiaojun Wan | Jianguo Xiao
Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing