Petrak, Dominic ; Moosavi, Nafise Sadat ; Tian, Ye ; Rozanov, Nikolai ; Gurevych, Iryna (2023)
Learning From Free-Text Human Feedback - Collect New Datasets Or Extend Existing Ones?
2023 Conference on Empirical Methods in Natural Language Processing. Singapore (06.-10.12.2023)
doi: 10.18653/v1/2023.emnlp-main.1011
Konferenzveröffentlichung, Bibliographie
Kurzbeschreibung (Abstract)
Continuous learning from free-text human feedback, such as error corrections, new knowledge, or alternative responses, is essential for today’s chatbots and virtual assistants to stay up-to-date, engaging, and socially acceptable. However, for research on methods for learning from such data, annotated data is scarce. To address this, we examine the error and user response types of six popular dialogue datasets from various types, including MultiWoZ, PersonaChat, Wizards-of-Wikipedia, and others, to assess their extendibility with the needed annotations. For this corpus study, we manually annotate a subset of each dataset with error and user response types using an improved version of the Integrated Error Taxonomy and a newly proposed user response type taxonomy. We provide the resulting dataset (EURTAD) to the community. Our findings provide new insights into dataset composition, including error types, user response types, and the relations between them.
| Typ des Eintrags: | Konferenzveröffentlichung |
|---|---|
| Erschienen: | 2023 |
| Autor(en): | Petrak, Dominic ; Moosavi, Nafise Sadat ; Tian, Ye ; Rozanov, Nikolai ; Gurevych, Iryna |
| Art des Eintrags: | Bibliographie |
| Titel: | Learning From Free-Text Human Feedback - Collect New Datasets Or Extend Existing Ones? |
| Sprache: | Englisch |
| Publikationsjahr: | Dezember 2023 |
| Verlag: | ACL |
| Buchtitel: | Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing |
| Veranstaltungstitel: | 2023 Conference on Empirical Methods in Natural Language Processing |
| Veranstaltungsort: | Singapore |
| Veranstaltungsdatum: | 06.-10.12.2023 |
| DOI: | 10.18653/v1/2023.emnlp-main.1011 |
| URL / URN: | https://aclanthology.org/2023.emnlp-main.1011/ |
| Kurzbeschreibung (Abstract): | Continuous learning from free-text human feedback, such as error corrections, new knowledge, or alternative responses, is essential for today’s chatbots and virtual assistants to stay up-to-date, engaging, and socially acceptable. However, for research on methods for learning from such data, annotated data is scarce. To address this, we examine the error and user response types of six popular dialogue datasets from various types, including MultiWoZ, PersonaChat, Wizards-of-Wikipedia, and others, to assess their extendibility with the needed annotations. For this corpus study, we manually annotate a subset of each dataset with error and user response types using an improved version of the Integrated Error Taxonomy and a newly proposed user response type taxonomy. We provide the resulting dataset (EURTAD) to the community. Our findings provide new insights into dataset composition, including error types, user response types, and the relations between them. |
| Freie Schlagworte: | UKP_p_SERMAS,UKP_p_LOEWE Spitzenprofessur |
| Fachbereich(e)/-gebiet(e): | 20 Fachbereich Informatik 20 Fachbereich Informatik > Ubiquitäre Wissensverarbeitung |
| Hinterlegungsdatum: | 18 Jan 2024 13:41 |
| Letzte Änderung: | 07 Mär 2024 12:17 |
| PPN: | 516071459 |
| Export: | |
| Suche nach Titel in: | TUfind oder in Google |
 |
Frage zum Eintrag |
Optionen (nur für Redakteure)
 |
Redaktionelle Details anzeigen |
Drucken
Impressum