WARNING: Beware of Pretend Predictive Market Providers

by Prediki Market Supervision -- Vienna, 10 Dec 2016

It has come to the attention of Prediki Market Supervision that there may be ostensible “predictive market” providers which mislead clients and the public, purporting to employ the method to forecast and ensure new product or communication concept success but which in truth just run a survey.

Surveys can only elicit claimed intent which is well known to be contaminated with biases, variability, overstatement, and a low correlation to actual purchases. Unsuspecting clients’ negative experience with the fakes may slow the appreciation and adoption of the real technology.

Genuine prediction markets are virtual stock exchanges where (1) participants truly buy and sell shares using virtual money and where (2) rewards depend on good predictive performance. Fake providers merely ask survey questions about a hypothetical purchase or sale. “Would you buy or sell shares in this concept?” Respondents in these bogus markets obtain a reward irrespective of the quality of their answers, as long as they click through the complete exercise.

The fakes appear to target marketing executives who are more familiar with traditional questionnaires. They are made to believe that prediction markets should be just as simplistic as filling out a survey. Ironically, they hold the more complex user interaction required by the real method against it, despite actual participants’ appreciation of the gamification.

On the positive side, even the fake providers benefit from one of the many behavioural advantages of prediction markets. Their question form - even if just asked by questionnaire - already changes the respondents' perspective when looking at the concept, product, or advert. They assume a third party angle, as an investor rather than a buyer, which measurably improves accuracy versus traditional intent questions.

However, the fakes cannot profit from the mechanism responsible for the largest uplift in reliability, Trading.

1. Only this continuous interaction - a highly expressive numeric two-way conversation - enables participants to reflect on their first answer and improve their response if they realise they were wrong.

2. Only the final feedback about being right or wrong facilitates the associative learning of participants needed to maximise forecast accuracy and insight generation. Collective IQ rises as a predictive research project moves from idea stage to concept stage and so on until the final pricing and promotion test stages.

Pretend providers are reminiscent of “cargo cults” where primitive societies mimic an advanced technology in the deluded expectation of reaping its benefits. The term was coined after World War 2 when indigenous people in Eastern Indonesia observed Japanese and US army airstrips. Huge metal "birds" brought clothing, food, medicine, tents, and weapons. After the end of the war, the airbases were abandoned but native believers strived for a return of the cargo by performing rituals on mock airstrips, wearing radios made of coconuts and straw, and staging drills with wooden sticks as rifles.

In the same vein - and while they may not always be perfect - imagine the utter chaos of the world economy if the London, New York, or Frankfurt Stock Exchanges were to run just simple surveys instead of continuous real-time markets to decide the prices of their listed companies’ predicted cash-flows.

Tellingly, providers of fake prediction markets seek to “prove” the validity of their survey approach not by comparing the predictions with real-world sales or other objective performance indicators but by showing some similarity of the results to other survey-based tests. This similarity should not come as a surprise as their method is but a survey.

Despite increasing use of purchase intent surveys and due to their manifold validity problems for future and intent questions, the number of product launch failures has not declined over the past two decades. Therefore it is of tantamount importance to promote the adoption of genuine prediction markets and to expose the pretenders.

Without technical knowledge it is not always easy to distinguish fake and real prediction markets. If you suspect that an ostensible prediction market provider just asks survey questions instead of running an authentic virtual stock exchange feel free to check with Prediki Market Supervision at info@prediki.com.

• Back to the Management by Predictions Blog
• Follow this blog for more stories, new case studies and how-to's.