Trick, Susanne ; Rothkopf, Constantin A. ; Jäkel, Frank (2023)
A Normative Model for Bayesian Combination of Subjective Probability Estimates.
In: Judgment and Decision Making, 2023 (18)
doi: 10.1017/jdm.2023.39
Article, Bibliographie
Abstract
Combining experts’ subjective probability estimates is a fundamental task with broad applicability in domains ranging from finance to public health. However, it is still an open question how to combine such estimates optimally. Since the beta distribution is a common choice for modeling uncertainty about probabilities, here we propose a family of normative Bayesian models for aggregating probability estimates based on beta distributions. We systematically derive and compare different variants, including hierarchical and non-hierarchical as well as asymmetric and symmetric beta fusion models. Using these models, we show how the beta calibration function naturally arises in this normative framework and how it is related to the widely used Linear-in-Log-Odds calibration function. For evaluation, we provide the new Knowledge Test Confidence data set consisting of subjective probability estimates of 85 forecasters on 180 queries. On this and another data set, we show that the hierarchical symmetric beta fusion model performs best of all beta fusion models and outperforms related Bayesian fusion models in terms of mean absolute error.
| Item Type: | Article |
|---|---|
| Erschienen: | 2023 |
| Creators: | Trick, Susanne ; Rothkopf, Constantin A. ; Jäkel, Frank |
| Type of entry: | Bibliographie |
| Title: | A Normative Model for Bayesian Combination of Subjective Probability Estimates |
| Language: | English |
| Date: | 2023 |
| Place of Publication: | Cambridge |
| Publisher: | Cambridge University Press |
| Journal or Publication Title: | Judgment and Decision Making |
| Volume of the journal: | 2023 |
| Issue Number: | 18 |
| DOI: | 10.1017/jdm.2023.39 |
| URL / URN: | https://www.cambridge.org/core/journals/judgment-and-decisio... |
| Abstract: | Combining experts’ subjective probability estimates is a fundamental task with broad applicability in domains ranging from finance to public health. However, it is still an open question how to combine such estimates optimally. Since the beta distribution is a common choice for modeling uncertainty about probabilities, here we propose a family of normative Bayesian models for aggregating probability estimates based on beta distributions. We systematically derive and compare different variants, including hierarchical and non-hierarchical as well as asymmetric and symmetric beta fusion models. Using these models, we show how the beta calibration function naturally arises in this normative framework and how it is related to the widely used Linear-in-Log-Odds calibration function. For evaluation, we provide the new Knowledge Test Confidence data set consisting of subjective probability estimates of 85 forecasters on 180 queries. On this and another data set, we show that the hierarchical symmetric beta fusion model performs best of all beta fusion models and outperforms related Bayesian fusion models in terms of mean absolute error. |
| Uncontrolled Keywords: | forecast aggregation, normative model, Bayesian inference, calibration, confidence, Projekt IKIDA 01IS20045 |
| Additional Information: | Article number: e40 |
| Divisions: | 03 Department of Human Sciences 03 Department of Human Sciences > Institute for Psychology 03 Department of Human Sciences > Institute for Psychology > Models of Higher Cognition 03 Department of Human Sciences > Institute for Psychology > Psychology of Information Processing Zentrale Einrichtungen Zentrale Einrichtungen > Centre for Cognitive Science (CCS) |
| Date Deposited: | 27 Nov 2023 13:34 |
| Last Modified: | 20 Feb 2024 08:50 |
| PPN: | 513497455 |
| Export: | |
| Suche nach Titel in: | TUfind oder in Google |
 |
Send an inquiry |
Options (only for editors)
 |
Show editorial Details |
Drucken
Impressum