TU Darmstadt / ULB / TUbiblio

MetaQA: Combining Expert Agents for Multi-Skill Question Answering

Puerto, Haritz ; Şahin, Gözde Gül ; Gurevych, Iryna (2023)
MetaQA: Combining Expert Agents for Multi-Skill Question Answering.
17th Conference of the European Chapter of the Association for Computational Linguistics. Dubrovnik, Croatia (02.-06.05.2023)
Conference or Workshop Item, Bibliographie

Abstract

The recent explosion of question-answering (QA) datasets and models has increased the interest in the generalization of models across multiple domains and formats by either training on multiple datasets or combining multiple models. Despite the promising results of multi-dataset models, some domains or QA formats may require specific architectures, and thus the adaptability of these models might be limited. In addition, current approaches for combining models disregard cues such as question-answer compatibility. In this work, we propose to combine expert agents with a novel, flexible, and training-efficient architecture that considers questions, answer predictions, and answer-prediction confidence scores to select the best answer among a list of answer predictions. Through quantitative and qualitative experiments, we show that our model i) creates a collaboration between agents that outperforms previous multi-agent and multi-dataset approaches, ii) is highly data-efficient to train, and iii) can be adapted to any QA format. We release our code and a dataset of answer predictions from expert agents for 16 QA datasets to foster future research of multi-agent systems.

Item Type: Conference or Workshop Item
Erschienen: 2023
Creators: Puerto, Haritz ; Şahin, Gözde Gül ; Gurevych, Iryna
Type of entry: Bibliographie
Title: MetaQA: Combining Expert Agents for Multi-Skill Question Answering
Language: English
Date: 2 May 2023
Publisher: ACL
Book Title: The 17th Conference of the European Chapter of the Association for Computational Linguistics - proceedings of the conference
Event Title: 17th Conference of the European Chapter of the Association for Computational Linguistics
Event Location: Dubrovnik, Croatia
Event Dates: 02.-06.05.2023
URL / URN: https://aclanthology.org/2023.eacl-main.259/
Abstract:

The recent explosion of question-answering (QA) datasets and models has increased the interest in the generalization of models across multiple domains and formats by either training on multiple datasets or combining multiple models. Despite the promising results of multi-dataset models, some domains or QA formats may require specific architectures, and thus the adaptability of these models might be limited. In addition, current approaches for combining models disregard cues such as question-answer compatibility. In this work, we propose to combine expert agents with a novel, flexible, and training-efficient architecture that considers questions, answer predictions, and answer-prediction confidence scores to select the best answer among a list of answer predictions. Through quantitative and qualitative experiments, we show that our model i) creates a collaboration between agents that outperforms previous multi-agent and multi-dataset approaches, ii) is highly data-efficient to train, and iii) can be adapted to any QA format. We release our code and a dataset of answer predictions from expert agents for 16 QA datasets to foster future research of multi-agent systems.

Uncontrolled Keywords: UKP_p_square, UKP_p_seditrah_factcheck
Divisions: 20 Department of Computer Science
20 Department of Computer Science > Ubiquitous Knowledge Processing
Date Deposited: 12 Jun 2023 12:27
Last Modified: 04 Aug 2023 09:33
PPN: 510358594
Export:
Suche nach Titel in: TUfind oder in Google
Send an inquiry Send an inquiry

Options (only for editors)
Show editorial Details Show editorial Details