Expert testing

An expert provides probabilistic predictions about a sequence of future outcomes (for example, an outcome can be the daily price of a stock and the expert provides the distribution of the price). An inspector reviews the predictions made by the expert and the observed outcomes and applies some test to decide on the validity of the expert’s predictions. The expert testing literature asks whether there exists some test that distinguishes a “true expert”, who provided the correct predictions from a “charlatan”, who concoct predictions strategically to pass the test. I will give a survey of the literature, heavily biased towards my own papers.