Website Evaluation and Information Literacy

I know it’s probably going to be me against the world on this one, but here goes: I don’t really understand why librarians teach students “website evaluation.”

What I mean, more specifically, is this:

(1) There does not seem to me to be any special criteria that apply to evaluating websites as such, as opposed to evaluating and thinking critically about information more generally.

(2) We marginalize ourselves by doing things like teaching students “website evaluation” instead of teaching critical thinking more generally.

Let me explain.

(1) It seems to me* that, most of the time, when people teach website evaluation, they teach something like the C.R.A.A.P. test. And this is fine; I know that a lot of people like this. But I think it’s actually a pretty superficial way to teach students to think about information, because it focuses heavily on features that tend to be superficial with regard to evaluating information.

So, for example, we tell students to see whether the person who wrote it is credible, to check for bias, to see who published it, whether it’s got citations, whether it’s current, whatever. There’s nothing really wrong with this, per se. But I think it’s not the most helpful way to present this to students, because, by and large, many of these things focus on aspects of the information ultimately fail you when the chips are down. They’re mere indicators of credibility, not constitutive of it.

Consider, say, the institutional affiliation of the author. We teach this as one of the factors that goes into establishing whether something is reliable. But what about when you show the students an example of a piece of information written by a professor from Harvard, that identifies the professor as a professor from Harvard, that’s full of of a bunch of unsubstantiated claims and rhetoric? That criterion fails.

I guess you could say that, well, in that case, you can try to identify if they have an agenda, or what their purpose is. I guess. But a person can have a good agenda and no obvious bias and write a piece of unreliable crap. And they can be a professor at at Harvard and say a bunch of silly things. These indicators, in many cases, fail. They’re just indicators of credibility, nothing more.

Another way to put the point is that they don’t actually teach you to think critically at all. They don’t make you information literate. One reason for this is that they don’t focus on the deep structure of information: what, fundamentally, makes something the kind of thing you should believe, or not. So, although they are useful to talk about with students – I’m not denying this, I do it all the time – they are never the focus of my discussions about evaluating the quality of a piece of information . Instead, I’ve tried to devise some (currently top secret) lesson plans and active learning pieces that focus on the deep structure of information, that the stuff you ought to believe in this world is the stuff that’s based in evidence.

Now this, of course, is, indeed, one of the components of the C.R.A.A.P. test. But it gets lost in, for lack of better joke, all the other superficial crap that gets discussed along with it. What I think we need to be doing is focusing on how to teach students the threshold concept that good information is the stuff based on good evidence. This is the stuff you ought to believe.

It’s this very principle, incidentally, that fundamentally dictates the kinds of research skills professors typically want students to use in their papers. So when you “think like a professor” about information, think about the fact that faculty typically want students to use the stuff that’s in peer reviewed journals not because it’s peer reviewed as such, but because this is supposed to track that it’s based on good evidence and argumentation. And, when faculty want students to use sources in their papers, the real reason they’re asking for this is because they want students to mimic the behavior of scholars: whenever you make a claim, you’d better rock a citation that tells me where you got that evidence, and why I should believe the thing you just said. And you’d better use good evidence.

So back to (1): if what I just said is right, the same thing that establishes whether a peer-reviewed journal article is reliable is the same exact thing that establishes whether a website is credible: whether the author is giving you good reason to believe the stuff they say. That question turns on the evidence that they’ve given you to believe what they say. Thus, the vast majority of my efforts teaching information evaluation to students are spent trying to get them to see the deep structure of information quality, that the stuff you believe – whether it’s from CNN, or FOX News, or the stuff your racist uncle told you – is whether or not the person presented evidence for what they said. I think this explains not only why their professors are asking them to search for scholarly articles for this particular assignment – but it also gives them a basic applied epistemology for thinking about the kind of stuff they ought to believe out there in the world. It gives them a basic conceptual building block they need in order to think critically.

So to bring it back to website evaluation: I think it makes total sense to show them a website, or a few websites, as a concrete example so they can practice a more general principle of evaluating information based on evidence. But this idea that we do this special thing as librarians, teach students how to evaluate websites, is, I think, fundamentally misguided.

(2) It seems to me* that it’s also bad for the profession.

Here’s something an education professor at my alma mater said to me once when I said I was an instruction librarian:

“What, you teach students to like evaluate websites and stuff?”

One way to interpret this is that it totally misunderstands what librarians do.  However, I think it’s a pretty savvy assessment, shows a pretty good insight into what we do;  for it’s basically saying, “that’s a pretty superficial way to go about teaching students to think critically.”

Because it is.

David Foster Wallace once told his creative writing students that the best way to write witty, clever characters is to have your characters say witty, clever things. So too, the best way for faculty to think we are sophisticated academcs is to stop teaching their students superficial things.

We can take the lead from recent pedagogical literature which indicates that

the distinctive quality that separates those who do high level thinking from those who do low level thinking is effective use of evidence

and start helping students develop cognitive strategies to think about information in a sophisticated way. We can teach them information literacy.


*” ‘It seems to me’ = ‘I have no argument.””

-comment written in the margins of my philosophy master’s thesis by my adviser, Blacksburg, Virginia, some time in 2007.



Filed under Bibliographic Instruction is Dead, Library Instruction, Posts by Kevin Michael Klipfel

4 responses to “Website Evaluation and Information Literacy

  1. Kevin, this brings up something you and I have discussed several times before – as librarians, do we want to be viewed as educators? When I talk about instruction with other public service librarians, I bring up this question all the time. Because if we want to be taken seriously as educators by other members of the academy, which I would suggest should be the goal of any academic librarian in public services, then we need to start teaching like educators. And how do educators teach? Educators (1) present conceptual knowledge, (2) provide context for these concepts that is relevant for students, (3) model successful behavior based on these concepts, and then (4) provide time for guided practice of these concepts.

    What educators don’t do is spend their classroom time demonstrating how to use tools, which is exactly why classic bibliographic instruction is so problematic right? Instead of creating learning objectives like “students will be able to understand the importance of using evidence to support a rhetorical argument,” we create learning objectives of “students will be able to utilize MeSH within PubMed to create a literature review.” Which of those two learning objective looks like it was created by an educator, and which one looks like it was created by a technician conducting an informal workshop? It seems to me* that website evaluation lectures too often align with that second learning objective, rather than the first – the librarian is teaching a tool, rather than a concept. In your example, applying C.R.A.A.P. to a website is just a tool of potential criteria to apply. Meanwhile, the actual concept the librarian is trying to convey is, as you say, how someone can determine if a resource should be considered valid evidence.

    However, website evaluation can be used as a (2) context for applying these concepts in a way that is relevant for students. I think that* a personal anecdote of a class I taught to early career undergraduate science students demonstrates how this can be done effectively. For their assignment, students needed to examine controversial topics within science, and determine if there is any credible evidence supporting these topics. I brought up the topic of linking autism to vaccination, and as an example of resource evaluation, I showed them Generation Rescue (, the autism “research” organization created by Jenny McCarthy. What I mentioned about this website is how it has many of the “check list” characteristics of a “reliable” source. Its pages have citations, it suggests physicians in your area that can help with treatment, and parents can even apply for a “biomedical grant” from the organization. In fact, if we applied C.R.A.A.P., this website might even pass all the criteria. However, if we dig a little deeper past the superficial level, it becomes clear that Generation Rescue is supported almost entirely by nutritional supplement companies that use this organization as a marketing arm for their snake oil. The “biomedical grants” are in fact just 90 day supplies of said snake oil.

    So I think that these conversations on how “reliable looking” resources can be totally bunk have a place in the library classroom. But do I think that website evaluation is such a core component of information literacy that students must be exposed to it through a formal, dedicated lecture devoid of context within a specific assignment? Of course not.

  2. Pingback: Philosophies of Librarianship | Rule Number One: A Library Blog

  3. Pingback: Secondary Education Information Literacy #secondaryeducation #informationliteracy #schoollibraries | A Listly List

  4. Pingback: Fact Checking and Information Literacy, Again | Rule Number One: A Library Blog

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s