huji
From Anomalies to Forecasts:
Choice Prediction Competition for Decisions under Risk and Ambiguity
(CPC2015)

Supported by the Max Wertheimer Minerva Center for Cognitive Processing and Human Performance
Organized by: Ido Erev, Eyal Ert, and Ori Plonsky
Submission deadline: May 17th, 2015  |  Early registration until April 1st, 2015

from http://blogs.ubc.ca/ubcowatershed/files/2013/09/Ellsberg-paradox1.jpg

Motivation and the basic idea

Experimental studies of human choice behavior have documented clear violations of rational economic theory and triggered the development of behavioral economics. Yet, the impact of these careful studies on applied economic analyses, and policy decisions, is not large. One justification for the tendency to ignore the experimental evidence involves the assertion that the behavioral literature highlights contradicting deviations from maximization, and it is not easy to predict which deviation is likely to be more important in specific situations.

To address this problem Kahneman and Tversky (1979) proposed a model (Prospect theory) that captures the joint effect of four of the most important deviations from maximization: the certainty effect (Allais paradox, Allais, 1953), the reflection effect, overweighting of low probability extreme events, and loss aversion (see top four rows in Table 1). The current paper extends this and similar efforts (see e.g., Thaler & Johnson, 1990; Brandstätter, Gigerenzer, & Hertwig, 2006; Birnbaum, 2008; Wakker, 2010; Erev et al., 2010) by facilitating the derivation and comparison of models that capture the joint impact of the four "prospect theory effects" and ten additional phenomena (see Table 1).

These choice phenomena were replicated under one "standard" setting (Hertwig & Ortmann, 2001): choice with real stakes in a space of experimental tasks wide enough to replicate all the phenomena illustrated in Table 1. The results suggest that all 14 phenomena emerge in our setting. Yet, their magnitude tends to be smaller than their magnitude in the original demonstrations.

The current choice prediction competition focuses on developing models that can capture all of these phenomena but also predict behavior in other choice problems. To calibrate the models we ran an “estimation set” study that included 60, randomly selected, choice problems.

The original description of the competition that was available when the competition was announced can be found here: Initial_Description_19.11.2014

For the paper that summarizes the competition, forthcoming in Psychological Review, click here: DEcompPaper_22.12.2016

And now the movie: a video featuring Ido's talk on the competition is available here CPC talk Jan 16 2017

Table 1. Typical Examples of Fourteen Phenomena and Their Replications in the Current Study

Classical Demonstration

Current Replication

Phenomenon

Problems

%B Choice

Problems

%B Choice

Description Phenomena

1. Certainty effect/Allais paradox (Kahneman & Tversky, 1979; following Allais, 1953)

A: 3000 with certainty

B: 4000, .8; 0 otherwise

A’: 3000, .25; 0 otherwise

B’: 4000, .20; 0 otherwise


20%


65%

A: 3 with certainty

B: 4, .8; 0 otherwise

Aʹ: 3, .25; 0 otherwise

Bʹ: 4, .20; 0 otherwise


42%

61%

2. Reflection effect (Kahneman & Tversky, 1979)

A: 3000 with certainty

B: 4000, .8; 0 otherwise

A’: -3000 with certainty

B’: -4000, .8; 0 otherwise


20%


92%

A: 3 with certainty

B: 4, .8; 0 otherwise

Aʹ: -3 with certainty

Bʹ: -4, .8; 0 otherwise


42%


49%

3. Over-weighting of rare events (Kahneman & Tversky, 1979)

A: 5 with certainty

B: 5000, .001; 0 otherwise


72%

A: 2 with certainty

B: 101, .01; 1 otherwise


55%

4. Loss aversion (Ert & Erev, 2013; following Kahneman & Tversky, 1979)

A: 0 with certainty

B: -100, .5; 100 otherwise


22%

A: 0 with certainty

B: -50, .5; 50 otherwise


34%

5. Low magnitude eliminates loss aversion (Ert & Erev, 2013)

A: 0 with certainty

B: -10, .5; 10 otherwise


48%

A: 0 with certainty

B: -1, .5; 1 otherwise


49%

6. St. Petersburg paradox/risk aversion (Bernoulli, 1738/1954)

A fair coin will be flipped until it comes up heads. The number of flips will be denoted by the letter k. The casino pays a gambler 2k. What is the maximal amount of money that you are willing to pay for playing this game?

Modal response: less than 8

A: 9 with certainty

B: 2, 1/2; 4, 1/4; 8; 1/8; 16, 1/16; 32, 1/32; 64, 1/64; 128, 1/128; 256 otherwise


38%

7. Ellsberg paradox/ Ambiguity aversion (Einhorn & Hogarth, 1986; following Ellsberg, 1961)

Urn K includes 50 Red, and 50 White balls. Urn U includes 100 balls, each either Red or White with unknown proportions. Choose between:

A: 100 if a ball drawn from K is Red; 0 otherwise

B: 100 if a ball drawn from U is Red; 0 otherwise

C: Indifference



47%

19%

34%

A: 10 with probability .5;

0 otherwise

B: 10 with probability ‘p’; 0 otherwise (‘p’ unknown constant)



37%

8. Break even effect (Thaler & Johnson, 1990)

A: -2.25 with certainty

B: -4.50, .5; 0 otherwise

A': -7.50 with certainty

B': -5.25 .5; -9.75 otherwise


87%


77%

A: -1 with certainty

B: -2, .5; 0 otherwise

Aʹ: -2 with certainty

Bʹ: -3 .5; -1 otherwise


58%


48%

9. Get something effect (Ert & Erev, 2013, following Payne, 2005)

A: 11, .5; 3 otherwise

B: 13, .5; 0 otherwise

A’: 12, .5; 4 otherwise

B’: 14, .5; 1 otherwise


21%


38%

A: 1 with certainty

B: 2, .5; 0 otherwise

Aʹ: 2 with certainty

Bʹ: 3 .5; 1 otherwise


35%

10. Splitting effect (Birnbaum, 2008)

A: 96; .90; 14, .05; 12 .05

B: 96; .85; 90, .05; 12, .10


73%

A: 16 with certainty

B: 1, .6; 50, .4

Aʹ: 16 with certainty

Bʹ: 1, .6; 44, .1; 48, .1; 50, .2


49.9%


50.4%

Experience Phenomena

11. Under-weighting of rare events (Barron & Erev, 2003)

A: 3 with certainty

B: 32, .1; 0 otherwise

A’: -3 with certainty

B’: -32, .1; 0 otherwise


32%


61%

A: 1 with certainty

B’: 20, .05; 0 otherwise

Aʹ: -1 with certainty

Bʹ: -20 .05; 0 otherwise

Aʺ: 2 with certainty

Bʺ: 101, .01; 0 otherwise


29%


64%


42%

12. Reversed reflection (Barron & Erev, 2003)

A: 3 with certainty

B: 4, .8; 0 otherwise

A’: -3 with certainty

B’: -4, .8; 0 otherwise


63%


40%

A: 3 with certainty

B: 4, .8; 0 otherwise

Aʹ: -3 with certainty

Bʹ: -4, .8; 0 otherwise


65%


36%

13. Payoff variability effect (Erev & Haruvy, 2010; following Busemeyer & Townsend, 1993)

A: 0 with certainty

B: 1 with certainty

A’: 0 with certainty

B’: -9, .5; 11 otherwise


96%


58%

A: 2 with certainty

B: 3 with certainty

A': 6 if E; 0 otherwise

B': 9 if not E; 0 otherwise


100%


84%

14. Correlation effect (Grosskopf et al., 2006; following Diederich & Busemeyer, 1999)

A: 150+N1 if E; 50+N1 otherwise

B: 160+N2 if E'; 70 +N2 otherwise

A': 150+N1 if E; 50+N1 otherwise

B': 160+N2 if E; 60 +N2 otherwise

Ni ~ N(0,20), P(E) = P(E') = .5


82%


98%

A: 6 if E; 0 otherwise

B: 9 if not E; 0 otherwise

A’: 6 if E; 0 otherwise

B’: 8 if E; 0 otherwise

P(E) =0.5


84%


98%

The basic task:

The participants in each competition will be allowed to study the results of the estimation set. Their goal will be to develop a model that will predict the results of the competition set. In order to qualify to the competition, the model will have to capture all 14 choice phenomena of Table 1. The model should be implemented in a computer program that reads the parameters of the problems as an input and predicts the proportion of choices of Option B as an output. Thus, we use the generalization criterion methodology (see Busemeyer & Wang, 2000).

Additional information concerning the competition can be found in the following pages:

  1. The “registration” and submission page explains the actions that have to be taken in order to participate in the competition.
  2. The “problem selection algorithm” page presents the algorithm that was used to select the problems in the estimation set, and is used to select the problems in the competition set.
  3. The “Method” page presents the experimental method.
  4. The “aggregated results” page presents the problems that were studied in the estimation set and the aggregated results.
  5. The “raw data” pages present the raw data for each of the participants.
  6. The “competition rules” page explains the time schedule, and the required features of the submissions.
  7. The “baseline model” page presents examples of possible submissions to the competition.

GOOD LUCK!

Department of Agricultural Economics and Management, Robert H. Smith Faculty of Agriculture, Food and Environment, The Hebrew University of Jerusalem
K&E Design
© The Hebrew University of Jerusalem  | P.O.Box 12, Rehovot, 76100, Israel  | Tel: +972-(0)8-9489375  |  Email: Eyal.Ert@mail.huji.ac.il