Fact-checking should be a business

By tradition and by default, books aren’t verified to anything near the standard of a magazine piece.

I am continually amazed at how often mainstream, otherwise trust-worthy news sources get things wrong. As the quip goes, “I find that the New York Times is always right, except in areas where I have first-hand knowledge.” Even peer-reviewed scientific journals are not immune: only about 40% of results published in top-tier psychology science journals can be fully replicated.

There are plenty of everyday examples:

Medical and health information is notoriously inaccurate, even from sources you’d hope you can trust. For decades, the Center for Science in the Public Interest, as well as the National Academy of Sciences, encouraged the American public to eat trans-fats.
A quote regularly repeated by New York Times op-ed author David Brooks, about a rising sense of self-importance among American adolescents appears to be entirely wrong.
Many, many books have been retracted, or published with disclaimers

What if there were an organization, like UL (that approves electrical equipment) or Consumer Reports (that recommends a variety of household products), only instead of dealing in physical goods, they put their stamp on books or magazines? Of course we already have book reviews, often by people who are themselves experts in the subject, but how many of them go systematically through all the facts and references to be sure that every claim in the book is accurate?

Cochrane is one independent organization that tries to be systematic in its reviews of the trustworthiness of medical findings. Verificationist is a service that offers to do fact-checking for books on behalf of publishers or authors. Morningstar and many other investment advisory firms do this for stocks and bonds. Can’t we get something similar for books?

Unfortunately I think this would be a lousy business. Too few publishers or authors would be willing to pay to have their own work fact-checked, and most customers, if given a choice, would prefer a cheaper book with facts presented in “good faith” over a more expensive one that was independently vetted.