TU Darmstadt / ULB / TUbiblio

UKP-SQuARE v3: A Platform for Multi-Agent QA Research

Puerto, Haritz ; Baumgärtner, Tim ; Sachdeva, Rachneet ; Fang, Haishuo ; Zhang, Hao ; Tariverdian, Sewin ; Wang, Kexin ; Gurevych, Iryna (2023)
UKP-SQuARE v3: A Platform for Multi-Agent QA Research.
61st Annual Meeting of the Association for Computational Linguistics. Toronto, Canada (10.-12.07.2023)
Conference or Workshop Item, Bibliographie

Abstract

The continuous development of Question Answering (QA) datasets has drawn the research community’s attention toward multi-domain models. A popular approach is to use multi-dataset models, which are models trained on multiple datasets to learn their regularities and prevent overfitting to a single dataset. However, with the proliferation of QA models in online repositories such as GitHub or Hugging Face, an alternative is becoming viable. Recent works have demonstrated that combining expert agents can yield large performance gains over multi-dataset models. To ease research in multi-agent models, we extend UKP-SQuARE, an online platform for QA research, to support three families of multi-agent systems: i) agent selection, ii) early-fusion of agents, and iii) late-fusion of agents. We conduct experiments to evaluate their inference speed and discuss the performance vs. speed trade-off compared to multi-dataset models. UKP-SQuARE is open-source and publicly available.

Item Type: Conference or Workshop Item
Erschienen: 2023
Creators: Puerto, Haritz ; Baumgärtner, Tim ; Sachdeva, Rachneet ; Fang, Haishuo ; Zhang, Hao ; Tariverdian, Sewin ; Wang, Kexin ; Gurevych, Iryna
Type of entry: Bibliographie
Title: UKP-SQuARE v3: A Platform for Multi-Agent QA Research
Language: English
Date: 10 July 2023
Publisher: ACL
Book Title: The 61st Annual Meeting of the Association for Computational Linguistics: Proceedings of the Conference Volume 3: System Demonstrations
Event Title: 61st Annual Meeting of the Association for Computational Linguistics
Event Location: Toronto, Canada
Event Dates: 10.-12.07.2023
URL / URN: https://aclanthology.org/2023.acl-demo.55
Abstract:

The continuous development of Question Answering (QA) datasets has drawn the research community’s attention toward multi-domain models. A popular approach is to use multi-dataset models, which are models trained on multiple datasets to learn their regularities and prevent overfitting to a single dataset. However, with the proliferation of QA models in online repositories such as GitHub or Hugging Face, an alternative is becoming viable. Recent works have demonstrated that combining expert agents can yield large performance gains over multi-dataset models. To ease research in multi-agent models, we extend UKP-SQuARE, an online platform for QA research, to support three families of multi-agent systems: i) agent selection, ii) early-fusion of agents, and iii) late-fusion of agents. We conduct experiments to evaluate their inference speed and discuss the performance vs. speed trade-off compared to multi-dataset models. UKP-SQuARE is open-source and publicly available.

Uncontrolled Keywords: UKP_p_square, UKP_p_qa_sci_inf
Divisions: 20 Department of Computer Science
20 Department of Computer Science > Ubiquitous Knowledge Processing
Date Deposited: 26 Jul 2023 07:57
Last Modified: 27 Jul 2023 10:23
PPN: 509978096
Export:
Suche nach Titel in: TUfind oder in Google
Send an inquiry Send an inquiry

Options (only for editors)
Show editorial Details Show editorial Details