Howard’s Technical Writing Blog

A technical writer’s miscellany

Bad information

Posted by Howard Silcock on 1 July 2010

I just received an email from the Departmental Library in the government department where I work, summarising the results of a survey about the dangers of “bad information” on businesses and organisations. The summary lists a number of “facts” that it says were uncovered (the list is reproduced below) and provides the following link for further information: Recognise the name in the URL? Oh, Dow Jones—seems reputable enough, doesn’t it? I can just see people quoting the results as established facts about the problems of gathering and using information.

The trouble is, though, that if you attempt to discover any evidence or methodology behind these “facts”, you soon recognise a few danger signals. Indeed, for anyone interested in information management or related topics, the very reporting of this survey, both in the email I’m quoting and on the web page it links to, seems to me to serve as a remarkable—and presumably unintended—example of how to promulgate bad information. And you can be sure it will be promulgated—taken completely out of context, without even an indication of how the results were arrived at.

Let’s first look at the actual list of “facts” that supposedly emerged from the survey.

Most common types of bad information:

  • Opinion disguised as fact (72%)
  • Biased sources (69%)
  • Factual errors (59%)
  • Urban myths/legends/hoaxes (36%)
  • Deliberate misinformation (25%)

Top 3 bad info dangers:

  1. Wasted time double checking facts (69%)
  2. Wasted time vetting sources (49%)
  3. Bad decisions made based on bad information (41%)

Top 3 bad info sources:

  1. Blogs and other social media sites (61%)
  2. General websites (57%)
  3. Marketing materials (35%)

OK, that’s the list. The percentages provide an aura of precision, don’t they? And it’s a reputable source, isn’t it? But let’s just dig a little deeper.

First, how did they get those percentages? The Full Survey Results on the web page referred to above should give some idea. But if we look there we find that they simply surveyed a number of people and asked each of them to list—taking the first group of “facts” as an example—the types of bad information they thought were most common.  Furthermore, examining the report more closely, we see that they weren’t asked to come up with their own ideas—they were given a number of possibles to choose from. So when the results state, for example, that ‘Opinion disguised as fact’ is the most common source of bad information, what that really means is that the respondents picked this one most frequently from a number of suggested responses, based on nothing but their own judgment of what seemed true for them. There’s absolutely no requirement for respondents to back up their assertions with evidence.

And who were these respondents, anyway? The report lists some facts about them: the industries they came from, the countries represented, and so on. But there’s no attempt to say whether they in any way representative. In fact, they obviously aren’t in any sense a representative sample. There are many groups of people not included at all, even if we restrict ourselves to people who work with information rather than physical labour. Where are the universities represented, for example?

Interestingly, among the statements respondents were invited to endorse were “I spend a lot of time double checking information” and “I spend a lot of time wading through suspect sources”. After the little bit of investigation I’ve described here, I think I know just what kind of “suspect sources” they’re talking about!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: