Alex Lascarides


2023

pdf bib
Learning the Effects of Physical Actions in a Multi-modal Environment
Gautier Dagan | Frank Keller | Alex Lascarides
Findings of the Association for Computational Linguistics: EACL 2023

Large Language Models (LLMs) handle physical commonsense information inadequately. As a result of being trained in a disembodied setting, LLMs often fail to predict an action’s outcome in a given environment. However, predicting the effects of an action before it is executed is crucial in planning, where coherent sequences of actions are often needed to achieve a goal. Therefore, we introduce the multi-modal task of predicting the outcomes of actions solely from realistic sensory inputs (images and text). Next, we extend an LLM to model latent representations of objects to better predict action outcomes in an environment. We show that multi-modal models can capture physical commonsense when augmented with visual information. Finally, we evaluate our model’s performance on novel actions and objects and find that combining modalities help models to generalize and learn physical commonsense reasoning better.

pdf bib
Interactive Acquisition of Fine-grained Visual Concepts by Exploiting Semantics of Generic Characterizations in Discourse
Jonghyuk Park | Alex Lascarides | Subramanian Ramamoorthy
Proceedings of the 15th International Conference on Computational Semantics

Interactive Task Learning (ITL) concerns learning about unforeseen domain concepts via natural interactions with human users. The learner faces a number of significant constraints: learning should be online, incremental and few-shot, as it is expected to perform tangible belief updates right after novel words denoting unforeseen concepts are introduced. In this work, we explore a challenging symbol grounding task—discriminating among object classes that look very similar—within the constraints imposed by ITL. We demonstrate empirically that more data-efficient grounding results from exploiting the truth-conditions of the teacher’s generic statements (e.g., “Xs have attribute Z.”) and their implicatures in context (e.g., as an answer to “How are Xs and Ys different?”, one infers Y lacks attribute Z).

pdf bib
Dialogue-based generation of self-driving simulation scenarios using Large Language Models
Antonio Valerio Miceli Barone | Craig Innes | Alex Lascarides
Proceedings of the 3rd Combined Workshop on Spatial Language Understanding and Grounded Communication for Robotics (SpLU-RoboNLP 2023)

Simulation is an invaluable tool for developing and evaluating controllers for self-driving cars. Current simulation frameworks are driven by highly-specialist domain specific languages, and so a natural language interface would greatly enhance usability. But there is often a gap, consisting of tacit assumptions the user is making, between a concise English utterance and the executable code that captures the user’s intent. In this paper we describe a system that addresses this issue by supporting an extended multimodal interaction: the user can follow up prior instructions with refinements or revisions, in reaction to the simulations that have been generated from their utterances so far. We use Large Language Models (LLMs) to map the user’s English utterances in this interaction into domain-specific code, and so we explore the extent to which LLMs capture the context sensitivity that’s necessary for computing the speaker’s intended message in discourse.

2022

pdf bib
Interactive Symbol Grounding with Complex Referential Expressions
Rimvydas Rubavicius | Alex Lascarides
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

We present a procedure for learning to ground symbols from a sequence of stimuli consisting of an arbitrarily complex noun phrase (e.g. “all but one green square above both red circles.”) and its designation in the visual scene. Our distinctive approach combines: a) lazy few-shot learning to relate open-class words like green and above to their visual percepts; and b) symbolic reasoning with closed-class word categories like quantifiers and negation. We use this combination to estimate new training examples for grounding symbols that occur within a noun phrase but aren’t designated by that noun phase (e.g, red in the above example), thereby potentially gaining data efficiency. We evaluate the approach in a visual reference resolution task, in which the learner starts out unaware of concepts that are part of the domain model and how they relate to visual percepts.

2021

pdf bib
Symbol Grounding and Task Learning from Imperfect Corrections
Mattias Appelgren | Alex Lascarides
Proceedings of Second International Combined Workshop on Spatial Language Understanding and Grounded Communication for Robotics

This paper describes a method for learning from a teacher’s potentially unreliable corrective feedback in an interactive task learning setting. The graphical model uses discourse coherence to jointly learn symbol grounding, domain concepts and valid plans. Our experiments show that the agent learns its domain-level task in spite of the teacher’s mistakes.

2017

pdf bib
Grounding Symbols in Multi-Modal Instructions
Yordan Hristov | Svetlin Penkov | Alex Lascarides | Subramanian Ramamoorthy
Proceedings of the First Workshop on Language Grounding for Robotics

As robots begin to cohabit with humans in semi-structured environments, the need arises to understand instructions involving rich variability—for instance, learning to ground symbols in the physical world. Realistically, this task must cope with small datasets consisting of a particular users’ contextual assignment of meaning to terms. We present a method for processing a raw stream of cross-modal input—i.e., linguistic instructions, visual perception of a scene and a concurrent trace of 3D eye tracking fixations—to produce the segmentation of objects with a correspondent association to high-level concepts. To test our framework we present experiments in a table-top object manipulation scenario. Our results show our model learns the user’s notion of colour and shape from a small number of physical demonstrations, generalising to identifying physical referents for novel combinations of the words.

pdf bib
Proceedings of the IWCS workshop on Foundations of Situated and Multimodal Communication
Nicholas Asher | Julie Hunter | Alex Lascarides
Proceedings of the IWCS workshop on Foundations of Situated and Multimodal Communication

pdf bib
Evaluating Persuasion Strategies and Deep Reinforcement Learning methods for Negotiation Dialogue agents
Simon Keizer | Markus Guhe | Heriberto Cuayáhuitl | Ioannis Efstathiou | Klaus-Peter Engelbrecht | Mihai Dobre | Alex Lascarides | Oliver Lemon
Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers

In this paper we present a comparative evaluation of various negotiation strategies within an online version of the game “Settlers of Catan”. The comparison is based on human subjects playing games against artificial game-playing agents (‘bots’) which implement different negotiation dialogue strategies, using a chat dialogue interface to negotiate trades. Our results suggest that a negotiation strategy that uses persuasion, as well as a strategy that is trained from data using Deep Reinforcement Learning, both lead to an improved win rate against humans, compared to previous rule-based and supervised learning baseline dialogue negotiators.

2015

pdf bib
Integrating Non-Linguistic Events into Discourse Structure
Julie Hunter | Nicholas Asher | Alex Lascarides
Proceedings of the 11th International Conference on Computational Semantics

2013

pdf bib
Grounding Strategic Conversation: Using Negotiation Dialogues to Predict Trades in a Win-Lose Game
Anaïs Cadilhac | Nicholas Asher | Farah Benamara | Alex Lascarides
Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing

2012

pdf bib
Multimodal Grammar Implementation
Katya Alahverdzhieva | Dan Flickinger | Alex Lascarides
Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

2011

pdf bib
Intégration de la parole et du geste déictique dans une grammaire multimodale (Integration of Speech and Deictic Gesture in a Multimodal Grammar)
Katya Alahverdzhieva | Alex Lascarides
Actes de la 18e conférence sur le Traitement Automatique des Langues Naturelles. Articles longs

Dans cet article, nous présentons une analyse à base de contraintes de la relation forme-sens des gestes déictiques et de leur signal de parole synchrone. En nous basant sur une étude empirique de corpus multimodaux, nous définissons quels énoncés multimodaux sont bien formés, et lesquels ne pourraient jamais produire le sens voulu dans la situation communicative. Plus précisément, nous formulons une grammaire multimodale dont les règles de construction utilisent la prosodie, la syntaxe et la sémantique de la parole, la forme et le sens du signal déictique, ainsi que la performance temporelle de la parole et la deixis afin de contraindre la production d’un arbre de syntaxe combinant parole et gesture déictique ainsi que la représentation unifiée du sens pour l’action multimodale correspondant à cet arbre. La contribution de notre projet est double : nous ajoutons aux ressources existantes pour le TAL un corpus annoté de parole et de gestes, et nous créons un cadre théorique pour la grammaire au sein duquel la composition sémantique d’un énoncé découle de la synchronie entre geste et parole.

pdf bib
Strategic Conversation
Alex Lascarides
Proceedings of the SIGDIAL 2011 Conference

pdf bib
Commitments to Preferences in Dialogue
Anais Cadilhac | Nicholas Asher | Farah Benamara | Alex Lascarides
Proceedings of the SIGDIAL 2011 Conference

2009

pdf bib
Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009)
Alex Lascarides | Claire Gardent | Joakim Nivre
Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009)

pdf bib
A Logic of Semantic Representations for Shallow Parsing
Alexander Koller | Alex Lascarides
Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009)

2008

pdf bib
Agreement and Disputes in Dialogue
Alex Lascarides | Nicholas Asher
Proceedings of the 9th SIGdial Workshop on Discourse and Dialogue

2005

pdf bib
Probabilistic Head-Driven Parsing for Discourse Structure
Jason Baldridge | Alex Lascarides
Proceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL-2005)

2004

pdf bib
Inferring Sentence-internal Temporal Relations
Mirella Lapata | Alex Lascarides
Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004

pdf bib
Combining Hierarchical Clustering and Machine Learning to Predict High-Level Discourse Structure
Caroline Sporleder | Alex Lascarides
COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics

2003

pdf bib
A Statistical Approach to the Semantics of Verb-Particles
Colin Bannard | Timothy Baldwin | Alex Lascarides
Proceedings of the ACL 2003 Workshop on Multiword Expressions: Analysis, Acquisition and Treatment

pdf bib
The interpretation of non-sentential utterances in dialogue
David Schlangen | Alex Lascarides
Proceedings of the Fourth SIGdial Workshop of Discourse and Dialogue

pdf bib
Detecting Novel Compounds: The Role of Distributional Evidence
Mirella Lapata | Alex Lascarides
10th Conference of the European Chapter of the Association for Computational Linguistics

pdf bib
A Probabilistic Account of Logical Metonymy
Maria Lapata | Alex Lascarides
Computational Linguistics, Volume 29, Number 2, June 2003

2002

pdf bib
XML-based NLP Tools for Analysing and Annotating Medical Language
Claire Grover | Ewan Klein | Mirella Lapata | Alex Lascarides
COLING-02: The 2nd Workshop on NLP and XML (NLPXML-2002)

2001

pdf bib
An Algebra for Semantic Construction in Constraint-based Grammars
Ann Copestake | Alex Lascarides | Dan Flickinger
Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics

pdf bib
XML-Based Data Preparation for Robust Deep Parsing
Claire Grover | Alex Lascarides
Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics

1999

pdf bib
Default Representation in Constraint-based Frameworks
Alex Lascarides | Ann Copestake
Computational Linguistics, Volume 25, Number 1, March 1999

pdf bib
Ninth Conference of the European Chapter of the Association for Computational Linguistics
Henry S. Thompson | Alex Lascarides
Ninth Conference of the European Chapter of the Association for Computational Linguistics

1997

pdf bib
Intergrating Symbolic and Statistical Representations: The Lexicon Pragmatics Interface
Ann Copestake | Alex Lascarides
35th Annual Meeting of the Association for Computational Linguistics and 8th Conference of the European Chapter of the Association for Computational Linguistics

1994

pdf bib
Intentions and Information in Discourse
Nicholas Asher | Alex Lascarides
32nd Annual Meeting of the Association for Computational Linguistics

1993

pdf bib
A Semantics and Pragmatics for the Pluperfect
Alex Lascarides | Nicholas Asher
Sixth Conference of the European Chapter of the Association for Computational Linguistics

pdf bib
Temporal Connectives in a Discourse Context
Alex Lascarides | Jon Oberlander
Sixth Conference of the European Chapter of the Association for Computational Linguistics

1992

pdf bib
Preventing False Temporal Implicatures: Interactive Defaults for Text Generation
Jon Oberlander | Alex Lascarides
COLING 1992 Volume 2: The 14th International Conference on Computational Linguistics

pdf bib
Inferring Discourse Relations in Context
Alex Lascarides | Nicholas Asher | Jon Oberlander
30th Annual Meeting of the Association for Computational Linguistics

1991

pdf bib
Discourse Relations and Defeasible Knowledge
Alex Lascarides | Nicholas Asher
29th Annual Meeting of the Association for Computational Linguistics