Hou, Yufang ; Tran, Thy Thy ; Vu, Doan ; Cao, Yiwen ; Li, Kai ; Rohde, Lukas ; Gurevych, Iryna (2024)
A Course Shared Task on Evaluating LLM Output for Clinical Questions.
6th Workshop on Teaching Natural Language Processing (TeachingNLP 2024). Bangkok, Thailand (12.08.2024 - 16.08.2024)
Konferenzveröffentlichung, Bibliographie
Kurzbeschreibung (Abstract)
This paper presents a shared task that we organized at the Foundations of Language Technology (FoLT) course in 2023/2024 at the Technical University of Darmstadt, which focuses on evaluating the output of Large Language Models (LLMs) in generating harmful answers to health-related clinical questions. We describe the task design considerations and report the feedback we received from the students. We expect the task and the findings reported in this paper to be relevant for instructors teaching natural language processing (NLP).
| Typ des Eintrags: | Konferenzveröffentlichung |
|---|---|
| Erschienen: | 2024 |
| Autor(en): | Hou, Yufang ; Tran, Thy Thy ; Vu, Doan ; Cao, Yiwen ; Li, Kai ; Rohde, Lukas ; Gurevych, Iryna |
| Art des Eintrags: | Bibliographie |
| Titel: | A Course Shared Task on Evaluating LLM Output for Clinical Questions |
| Sprache: | Englisch |
| Publikationsjahr: | 15 August 2024 |
| Ort: | Bangkok, Thailand |
| Verlag: | ACL |
| Buchtitel: | TeachNLP 2024: The Sixth Workshop on Teaching NLP - Proceedings of the Workshop |
| Veranstaltungstitel: | 6th Workshop on Teaching Natural Language Processing (TeachingNLP 2024) |
| Veranstaltungsort: | Bangkok, Thailand |
| Veranstaltungsdatum: | 12.08.2024 - 16.08.2024 |
| URL / URN: | https://aclanthology.org/2024.teachingnlp-1.11/ |
| Kurzbeschreibung (Abstract): | This paper presents a shared task that we organized at the Foundations of Language Technology (FoLT) course in 2023/2024 at the Technical University of Darmstadt, which focuses on evaluating the output of Large Language Models (LLMs) in generating harmful answers to health-related clinical questions. We describe the task design considerations and report the feedback we received from the students. We expect the task and the findings reported in this paper to be relevant for instructors teaching natural language processing (NLP). |
| Fachbereich(e)/-gebiet(e): | 20 Fachbereich Informatik 20 Fachbereich Informatik > Ubiquitäre Wissensverarbeitung |
| Hinterlegungsdatum: | 20 Aug 2024 08:50 |
| Letzte Änderung: | 19 Nov 2024 14:18 |
| PPN: | 523650817 |
| Export: | |
| Suche nach Titel in: | TUfind oder in Google |
 |
Frage zum Eintrag |
Optionen (nur für Redakteure)
 |
Redaktionelle Details anzeigen |
Drucken
Impressum