I just wanted to get on the record that I don’t trust claims based on statistical data without being able to review in detail the methods used to obtain and interpret the data. Even with the best intentions, full integrity and honesty, it is not difficult to come to completely the wrong conclusions based on apparently compelling statistical evidence.
This isn’t just theory, I’ve seen it happen. Detailed percentages presented at upper governmental levels, based on huge sample sets, giving a totally false impression because of errors and assumptions that occur at various places in the process. The exact same question asked twice in the same survey, giving very different results depending on the section in which the question appeared, providing an unstated context to the question. The devil is in the details, and the details can be extremely subtle.
I have a “put up or shut up” rule that applies to anybody who makes claims based on unrevealed statistical evidence. It applies to corporations, news outlets, bloggers, government ministers, everybody. Without allowing scrutiny of the full details, all statistical claims are null and void, as far as I’m concerned. “Trust me” doesn’t cut it. Sorry, no exceptions.