Michael Elhadad


2023

pdf bib
Semantic Decomposition of Question and SQL for Text-to-SQL Parsing
Ben Eyal | Moran Mahabi | Ophir Haroche | Amir Bachar | Michael Elhadad
Findings of the Association for Computational Linguistics: EMNLP 2023

Text-to-SQL semantic parsing faces challenges in generalizing to cross-domain and complex queries. Recent research has employed a question decomposition strategy to enhance the parsing of complex SQL queries.However, this strategy encounters two major obstacles: (1) existing datasets lack question decomposition; (2) due to the syntactic complexity of SQL, most complex queries cannot be disentangled into sub-queries that can be readily recomposed. To address these challenges, we propose a new modular Query Plan Language (QPL) that systematically decomposes SQL queries into simple and regular sub-queries. We develop a translator from SQL to QPL by leveraging analysis of SQL server query optimization plans, and we augment the Spider dataset with QPL programs. Experimental results demonstrate that the modular nature of QPL benefits existing semantic-parsing architectures, and training text-to-QPL parsers is more effective than text-to-SQL parsing for semantically equivalent queries. The QPL approach offers two additional advantages: (1) QPL programs can be paraphrased as simple questions, which allows us to create a dataset of (complex question, decomposed questions). Training on this dataset, we obtain a Question Decomposer for data retrieval that is sensitive to database schemas. (2) QPL is more accessible to non-experts for complex queries, leading to more interpretable output from the semantic parser.

pdf bib
Emptying the Ocean with a Spoon: Should We Edit Models?
Yuval Pinter | Michael Elhadad
Findings of the Association for Computational Linguistics: EMNLP 2023

We call into question the recently popularized method of direct model editing as a means of correcting factual errors in LLM generations. We contrast model editing with three similar but distinct approaches that pursue better defined objectives: (1) retrieval-based architectures, which decouple factual memory from inference and linguistic capabilities embodied in LLMs; (2) concept erasure methods, which aim at preventing systemic bias in generated text; and (3) attribution methods, which aim at grounding generations into identified textual sources. We argue that direct model editing cannot be trusted as a systematic remedy for the disadvantages inherent to LLMs, and while it has proven potential in improving model explainability, it opens risks by reinforcing the notion that models can be trusted for factuality. We call for cautious promotion and application of model editing as part of the LLM deployment process, and for responsibly limiting the use cases of LLMs to those not relying on editing as a critical component.

2022

pdf bib
Cross-Lingual UMLS Named Entity Linking using UMLS Dictionary Fine-Tuning
Rina Galperin | Shachar Schnapp | Michael Elhadad
Findings of the Association for Computational Linguistics: ACL 2022

We study cross-lingual UMLS named entity linking, where mentions in a given source language are mapped to UMLS concepts, most of which are labeled in English. Our cross-lingual framework includes an offline unsupervised construction of a translated UMLS dictionary and a per-document pipeline which identifies UMLS candidate mentions and uses a fine-tuned pretrained transformer language model to filter candidates according to context. Our method exploits a small dataset of manually annotated UMLS mentions in the source language and uses this supervised data in two ways: to extend the unsupervised UMLS dictionary and to fine-tune the contextual filtering of candidate mentions in full documents. We demonstrate results of our approach on both Hebrew and English. We achieve new state-of-the-art (SOTA) results on the Hebrew Camoni corpus, +8.9 F1 on average across three communities in the dataset. We also achieve new SOTA on the English dataset MedMentions with +7.3 F1.

2021

pdf bib
Data Efficient Masked Language Modeling for Vision and Language
Yonatan Bitton | Michael Elhadad | Gabriel Stanovsky | Roy Schwartz
Findings of the Association for Computational Linguistics: EMNLP 2021

Masked language modeling (MLM) is one of the key sub-tasks in vision-language pretraining. In the cross-modal setting, tokens in the sentence are masked at random, and the model predicts the masked tokens given the image and the text. In this paper, we observe several key disadvantages of MLM in this setting. First, as captions tend to be short, in a third of the sentences no token is sampled. Second, the majority of masked tokens are stop-words and punctuation, leading to under-utilization of the image. We investigate a range of alternative masking strategies specific to the cross-modal setting that address these shortcomings, aiming for better fusion of text and image in the learned representation. When pre-training the LXMERT model, our alternative masking strategies consistently improve over the original masking strategy on three downstream tasks, especially in low resource settings. Further, our pre-training approach substantially outperforms the baseline model on a prompt-based probing task designed to elicit image objects. These results and our analysis indicate that our method allows for better utilization of the training data.

pdf bib
Automatic Generation of Contrast Sets from Scene Graphs: Probing the Compositional Consistency of GQA
Yonatan Bitton | Gabriel Stanovsky | Roy Schwartz | Michael Elhadad
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

Recent works have shown that supervised models often exploit data artifacts to achieve good test scores while their performance severely degrades on samples outside their training distribution. Contrast sets (Gardneret al., 2020) quantify this phenomenon by perturbing test samples in a minimal way such that the output label is modified. While most contrast sets were created manually, requiring intensive annotation effort, we present a novel method which leverages rich semantic input representation to automatically generate contrast sets for the visual question answering task. Our method computes the answer of perturbed questions, thus vastly reducing annotation cost and enabling thorough evaluation of models’ performance on various semantic aspects (e.g., spatial or relational reasoning). We demonstrate the effectiveness of our approach on the GQA dataset and its semantic scene graph image representation. We find that, despite GQA’s compositionality and carefully balanced label distribution, two high-performing models drop 13-17% in accuracy compared to the original test set. Finally, we show that our automatic perturbation can be applied to the training set to mitigate the degradation in performance, opening the door to more robust models.

pdf bib
Evaluation Guidelines to Deal with Implicit Phenomena to Assess Factuality in Data-to-Text Generation
Roy Eisenstadt | Michael Elhadad
Proceedings of the 1st Workshop on Understanding Implicit and Underspecified Language

Data-to-text generation systems are trained on large datasets, such as WebNLG, Ro-toWire, E2E or DART. Beyond traditional token-overlap evaluation metrics (BLEU or METEOR), a key concern faced by recent generators is to control the factuality of the generated text with respect to the input data specification. We report on our experience when developing an automatic factuality evaluation system for data-to-text generation that we are testing on WebNLG and E2E data. We aim to prepare gold data annotated manually to identify cases where the text communicates more information than is warranted based on the in-put data (extra) or fails to communicate data that is part of the input (missing). While analyzing reference (data, text) samples, we encountered a range of systematic uncertainties that are related to cases on implicit phenomena in text, and the nature of non-linguistic knowledge we expect to be involved when assessing factuality. We derive from our experience a set of evaluation guidelines to reach high inter-annotator agreement on such cases.

2020

pdf bib
Building a Hebrew Semantic Role Labeling Lexical Resource from Parallel Movie Subtitles
Ben Eyal | Michael Elhadad
Proceedings of the Twelfth Language Resources and Evaluation Conference

We present a semantic role labeling resource for Hebrew built semi-automatically through annotation projection from English. This corpus is derived from the multilingual OpenSubtitles dataset and includes short informal sentences, for which reliable linguistic annotations have been computed. We provide a fully annotated version of the data including morphological analysis, dependency syntax and semantic role labeling in both FrameNet and ProbBank styles. Sentences are aligned between English and Hebrew, both sides include full annotations and the explicit mapping from the English arguments to the Hebrew ones. We train a neural SRL model on this Hebrew resource exploiting the pre-trained multilingual BERT transformer model, and provide the first available baseline model for Hebrew SRL as a reference point. The code we provide is generic and can be adapted to other languages to bootstrap SRL resources.

pdf bib
Neural Micro-Planning for Data to Text Generation Produces more Cohesive Text
Roy Eisenstadt | Michael Elhadad
Proceedings of the Workshop on Discourse Theories for Text Planning

2019

pdf bib
Question Answering as an Automatic Evaluation Metric for News Article Summarization
Matan Eyal | Tal Baumel | Michael Elhadad
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)

Recent work in the field of automatic summarization and headline generation focuses on maximizing ROUGE scores for various news datasets. We present an alternative, extrinsic, evaluation metric for this task, Answering Performance for Evaluation of Summaries. APES utilizes recent progress in the field of reading-comprehension to quantify the ability of a summary to answer a set of manually created questions regarding central entities in the source article. We first analyze the strength of this metric by comparing it to known manual evaluation metrics. We then present an end-to-end neural abstractive model that maximizes APES, while increasing ROUGE scores to competitive results.

2016

pdf bib
The Hebrew FrameNet Project
Avi Hayoun | Michael Elhadad
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)

We present the Hebrew FrameNet project, describe the development and annotation processes and enumerate the challenges we faced along the way. We have developed semi-automatic tools to help speed the annotation and data collection process. The resource currently covers 167 frames, 3,000 lexical units and about 500 fully annotated sentences. We have started training and testing automatic SRL tools on the seed data.

pdf bib
Sentence Embedding Evaluation Using Pyramid Annotation
Tal Baumel | Raphael Cohen | Michael Elhadad
Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP

2014

pdf bib
Query-Chain Focused Summarization
Tal Baumel | Raphael Cohen | Michael Elhadad
Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

2013

pdf bib
Word Segmentation, Unknown-word Resolution, and Morphological Agreement in a Hebrew Parsing System
Yoav Goldberg | Michael Elhadad
Computational Linguistics, Volume 39, Issue 1 - March 2013

pdf bib
Effect of Out Of Vocabulary Terms on Inferring Eligibility Criteria for a Retrospective Study in Hebrew EHR
Raphael Cohen | Michael Elhadad
Proceedings of the 2013 Workshop on Biomedical Natural Language Processing

pdf bib
Multi-document multilingual summarization corpus preparation, Part 2: Czech, Hebrew and Spanish
Michael Elhadad | Sabino Miranda-Jiménez | Josef Steinberger | George Giannakopoulos
Proceedings of the MultiLing 2013 Workshop on Multilingual Multi-document Summarization

2012

pdf bib
Domain Adaptation of a Dependency Parser with a Class-Class Selectional Preference Model
Raphael Cohen | Yoav Goldberg | Michael Elhadad
Proceedings of ACL 2012 Student Research Workshop

2011

pdf bib
Joint Hebrew Segmentation and Parsing using a PCFGLA Lattice Parser
Yoav Goldberg | Michael Elhadad
Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies

2010

pdf bib
Easy-First Dependency Parsing of Modern Hebrew
Yoav Goldberg | Michael Elhadad
Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages

pdf bib
Inspecting the Structural Biases of Dependency Parsing Algorithms
Yoav Goldberg | Michael Elhadad
Proceedings of the Fourteenth Conference on Computational Natural Language Learning

pdf bib
Book Review: Natural Language Processing with Python by Steven Bird, Ewan Klein, and Edward Loper
Michael Elhadad
Computational Linguistics, Volume 36, Issue 4 - December 2010

pdf bib
An Efficient Algorithm for Easy-First Non-Directional Dependency Parsing
Yoav Goldberg | Michael Elhadad
Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics

2009

pdf bib
On the Role of Lexical Features in Sequence Labeling
Yoav Goldberg | Michael Elhadad
Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing

pdf bib
Gaiku : Generating Haiku with Word Associations Norms
Yael Netzer | David Gabay | Yoav Goldberg | Michael Elhadad
Proceedings of the Workshop on Computational Approaches to Linguistic Creativity

pdf bib
Hebrew Dependency Parsing: Initial Results
Yoav Goldberg | Michael Elhadad
Proceedings of the 11th International Conference on Parsing Technologies (IWPT’09)

pdf bib
Enhancing Unlexicalized Parsing Performance Using a Wide Coverage Lexicon, Fuzzy Tag-Set Mapping, and EM-HMM-Based Lexical Probabilities
Yoav Goldberg | Reut Tsarfaty | Meni Adler | Michael Elhadad
Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009)

2008

pdf bib
Unsupervised Lexicon-Based Resolution of Unknown Words for Full Morphological Analysis
Meni Adler | Yoav Goldberg | David Gabay | Michael Elhadad
Proceedings of ACL-08: HLT

pdf bib
EM Can Find Pretty Good HMM POS-Taggers (When Given a Good Start)
Yoav Goldberg | Meni Adler | Michael Elhadad
Proceedings of ACL-08: HLT

pdf bib
splitSVM: Fast, Space-Efficient, non-Heuristic, Polynomial Kernel Computation for NLP Applications
Yoav Goldberg | Michael Elhadad
Proceedings of ACL-08: HLT, Short Papers

pdf bib
Tagging a Hebrew Corpus: the Case of Participles
Meni Adler | Yael Netzer | Yoav Goldberg | David Gabay | Michael Elhadad
Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08)

We report on an effort to build a corpus of Modern Hebrew tagged with part-of-speech and morphology. We designed a tagset specific to Hebrew while focusing on four aspects: the tagset should be consistent with common linguistic knowledge; there should be maximal agreement among taggers as to the tags assigned to maintain consistency; the tagset should be useful for machine taggers and learning algorithms; and the tagset should be effective for applications relying on the tags as input features. In this paper, we illustrate these issues by explaining our decision to introduce a tag for beinoni forms in Hebrew. We explain how this tag is defined, and how it helped us improve manual tagging accuracy to a high-level, while improving automatic tagging and helping in the task of syntactic chunking.

2007

pdf bib
SVM Model Tampering and Anchored Learning: A Case Study in Hebrew NP Chunking
Yoav Goldberg | Michael Elhadad
Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics

pdf bib
Can You Tag the Modal? You Should.
Yael Netzer | Meni Adler | David Gabay | Michael Elhadad
Proceedings of the 2007 Workshop on Computational Approaches to Semitic Languages: Common Issues and Resources

2006

pdf bib
Using Semantic Authoring for Blissymbols Communication Boards
Yael Netzer | Michael Elhadad
Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers

pdf bib
An Unsupervised Morpheme-Based HMM for Hebrew Morphological Disambiguation
Meni Adler | Michael Elhadad
Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics

pdf bib
Noun Phrase Chunking in Hebrew: Influence of Lexical and Morphological Features
Yoav Goldberg | Meni Adler | Michael Elhadad
Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics

2005

pdf bib
Interactive Authoring of Logical Forms for Multilingual Generation
Ofer Biller | Michael Elhadad | Yael Netzer
Proceedings of the Tenth European Workshop on Natural Language Generation (ENLG-05)

2000

pdf bib
INLG’2000 Proceedings of the First International Conference on Natural Language Generation
Michael Elhadad
INLG’2000 Proceedings of the First International Conference on Natural Language Generation

pdf bib
Integrating a Large-Scale, Reusable Lexicon with a Natural Language Generator
Hongyan Jing | Yael Dahan | Michael Elhadad | Kathy McKeown
INLG’2000 Proceedings of the First International Conference on Natural Language Generation

1999

pdf bib
Bilingual Hebrew-English Generation of Possessives and Partitives: Raising the Input Abstraction Level
Yael Dahan Netzer | Michael Elhadad
Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics

pdf bib
Information Fusion in the Context of Multi-Document Summarization
Regina Barzilay | Kathleen R. McKeown | Michael Elhadad
Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics

1998

pdf bib
Generating Determiners and Quantifiers in Hebrew
Yael Dahan Netzer | Michael Elhadad
Computational Approaches to Semitic Languages

pdf bib
Generation of Noun Compounds in Hebrew: Can Syntactic Knowledge Be Fully Encapsulated?
Yael Dahan Netzer | Michael Elhadad
Natural Language Generation

1997

pdf bib
Floating Constraints in Lexical Choice
Michael Elhadad | Kathleen McKeown | Jacques Robin
Computational Linguistics, Volume 23, Number 2, June 1997

pdf bib
Using Lexical Chains for Text Summarization
Regina Barzilay | Michael Elhadad
Intelligent Scalable Text Summarization

1996

pdf bib
An Overview of SURGE: a Reusable Comprehensive Syntactic Realization Component
Michael Elhadad | Jacques Robin
Eighth International Natural Language Generation Workshop (Posters and Demonstrations)

1992

pdf bib
Generating Coherent Argumentative Paragraphs
Michael Elhadad
COLING 1992 Volume 2: The 14th International Conference on Computational Linguistics

1990

pdf bib
Generating Connectives
Michael Elhadad | Kathleen R. McKeown
COLING 1990 Volume 3: Papers presented to the 13th International Conference on Computational Linguistics

pdf bib
Types in Functional Unification Grammars
Michael Elhadad
28th Annual Meeting of the Association for Computational Linguistics