Eliciting truthful responses from people - who give subjective answers - is crucial to the surveys and expert analyses that determine government and financial policies. Finding out if someone is answering a question truthfully is a murky process, often inaccessible to the questioner - sometimes even the person answering cannot be sure.

Now, Drazen Prelec, a psychologist at the Massachusetts Institute of Technology in Cambridge, US, has devised a scoring system, or “Bayesian truth serum” to encourage people to divulge their honest opinions.

The executive summary of Mr Prelec's scoring system can be found here.

Remarks: If all respondents are rational expected value-maximizing Bayesians then truthful responding remains the correct strategy "even for someone who is sure that his answer represents a minority view", i.e. "the scoring function removes all bias in favor of consensus".

A closer look at the scoring function: Respondents are indexed by r ∈ {1,2,...} and are confronted with a m multiple choice question (example for m = 2: Have you had more than 20 sexual partners over the past vear? (yes/no). Each respondent assigns a value of 1 to the answer that applies to him, zero otherwise). Denote answers (x) and predictions (y) by
The total score for a respondent combines the information score with a seperate score for the accuracy of prediction:

Prelec writes: The first part of the equation selects a single information-score value, given that x_r,k=0 for all answers except the one endorsed by r. The second part is a penalty proportional to the relative entropy (or Kullback-Leibler divergence) between the empirical distribution and r's prediction of that distribution. The best prediction score is zero, attained when prediction exactly matches reality. Expected prediction score is maximized by reporting expected frequencies. The constant alpha fine-tunes the weight given to prediction error.

My take: I guess one can see how it ought to work but Prelec actually writes in his own paper why things are far from perfect:

Truth-telling is individually rational in the sense that a truthful answer maximizes expected information score, assuming that everyone is responding truthfully [hence, it is a Bayesian Nash-Equilibrium]. <> In actual applications of the method, one would not teach respondents the mathematics of scoring or explain the notion of equilibrium. Rather, one would like to be able to tell them that truthful answers will maximize their expected scores, and that in arriving at their personal true answer they are free to ignore what other respondents might say. (emphasis mine).

(via Marginal Revolution)

Source: A Bayesian Truth Serum for Subjective Data, Drazen Prelec