TU Darmstadt / ULB / TUbiblio

Netted?! How to Improve the Usefulness of Spider & Co.

Hättasch, Benjamin ; Geisler, Nadja ; Binnig, Carsten (2021):
Netted?! How to Improve the Usefulness of Spider & Co.
2nd International Biennial Conference on Design of Experimental Search & Information Retrieval Systems (DESIRES 2021), Padua, Italy, 15.-18.09.2021, [Conference or Workshop Item]

Abstract

Natural language interfaces for databases (NLIDBs) are an intuitive way to access and explore structured data. That makes challenges like Spider (Yale’s semantic parsing and text-to-SQL challenge) valuable, as they produce a series of approaches for NL-to-SQL-translation. However, the resulting contributions leave something to be desired. In this paper, we analyze the usefulness of those submissions to the leaderboard for future research. We also present a prototypical implementation called UniverSQL that makes these approaches easier to use in information access systems. We hope that this lowered barrier encourages (future) participants of these challenges to add support for actual usage of their submissions. Finally, we discuss what could be done to improve future benchmarks and shared tasks for (not only) NLIDBs.

Item Type: Conference or Workshop Item
Erschienen: 2021
Creators: Hättasch, Benjamin ; Geisler, Nadja ; Binnig, Carsten
Title: Netted?! How to Improve the Usefulness of Spider & Co.
Language: English
Abstract:

Natural language interfaces for databases (NLIDBs) are an intuitive way to access and explore structured data. That makes challenges like Spider (Yale’s semantic parsing and text-to-SQL challenge) valuable, as they produce a series of approaches for NL-to-SQL-translation. However, the resulting contributions leave something to be desired. In this paper, we analyze the usefulness of those submissions to the leaderboard for future research. We also present a prototypical implementation called UniverSQL that makes these approaches easier to use in information access systems. We hope that this lowered barrier encourages (future) participants of these challenges to add support for actual usage of their submissions. Finally, we discuss what could be done to improve future benchmarks and shared tasks for (not only) NLIDBs.

Uncontrolled Keywords: dm, dm_nhr4ces, dm_aiphes, dm_softwarecampus, dm_project_AIPHES_area_d2, dm_project_wannadb
Divisions: 20 Department of Computer Science
20 Department of Computer Science > Data Management
DFG-Graduiertenkollegs
DFG-Graduiertenkollegs > Research Training Group 1994 Adaptive Preparation of Information from Heterogeneous Sources
Event Title: 2nd International Biennial Conference on Design of Experimental Search & Information Retrieval Systems (DESIRES 2021)
Event Location: Padua, Italy
Event Dates: 15.-18.09.2021
Date Deposited: 10 Dec 2021 09:29
Official URL: https://http://desires.dei.unipd.it/
Export:
Suche nach Titel in: TUfind oder in Google
Send an inquiry Send an inquiry

Options (only for editors)
Show editorial Details Show editorial Details