Problems of Generalization

* Ed Yong, ‘How a Focus on Rich Educated People Skews Brain Studies’, The Atlantic, October 31st 2017, https://www.theatlantic.com/science/archive/2017/10/how-a-focus-on-rich-educated-people-skews-brain-studies/544499/
* Joseph Henrich, Steven J. Heine, and Ara Norenzayan, ‘The Weirdest People in the World?’, Behavioural and Brain Sciences, 33 (2010), 61-83.
* Neil Stewart, Jesse Chandler, and Gabriele Paolacci, ‘Crowdsourcing Samples in Cognitive Science’, Trends in Cognitive Sciences, 21 (2017), 736-48.
* Lili Yu and Erik D. Reichle, ‘Chinese versus English: Insights on Cognition during Reading’, Trends in Cognitive Sciences, 21 (2017), 721-4.
* Felix Sprang, ‘The Confines of Cognitive Literary Studies: The Sonnet and a Cognitive Poetics of Form’, Journal of Literary Theory, 11 (2017), 240-254.

That’s a lot of references, because the problem of generalization is interesting, and everywhere, and circulates between literature and cognitive science in a scary feedback loop (the video kind, not the management consultants’ diagram kind; don’t watch the video below, by the way). I don’t think this is all well known to readers of this blog, so I thought I’d work through some of it.
      From the outset in my attempts to connect literature with cognitive science, I’ve been negotiating a worry from the direction of the humanities, which is that it might not be possible, or desirable, to characterize the human across cultures or historical periods. The conclusions made in cognitive science often tend to do this, implicitly at least. However, when someone in the humanities wants to capture (for example) the particulars of Caribbean experience, or of Medieval memory, there is understandable push back against an interest in how ‘the mind’ works.
      Nevertheless, I have often come to feel that there are thoughts about the species-level human that must be inflected (sometimes greatly) by specific circumstances, but not invalidated. I like the creative tension involved, I try not to take things for granted, but I don’t deny myself the things that seem to persist across (in my case) historical distance.
      There are, though, voices within cognitive science that are raising concerns about the data-gathering practices that dominate, which seems an interesting and important angle from which to address my problem of generalization, so in the post I’ll mention a couple of recent instances, and I’ll link it back to literature as well.

*

Ed Yong’s article in The Atlantic highlights the problem that research in psychology, and in other fields as well, is predominantly practised on people from WEIRD societies. (Cute acronym: WEIRD is Western, Educated, Industrialized, Rich, and Democratic; see the essay by Henrich et al. cited above.) Some fields seem to me more vulnerable to this than others; it’s true that at times I have felt that biggish conclusions in cognitive science are being drawn on the basis of a cohort of university students (standard experimental subjects) and that this is not acknowledged actively enough. This seems to me like something to be addressed in the processing of approaching those conclusions. Experiments that teach us something should not be disparaged because they don’t teach us everything, but claims can be made carefully, and with a lack of reach and diversity in mind. Future experiments could aspire to expand the scope, and to test the persistence of the effects; I have seen good examples of this sort of work.

*

Stewart et al. raise a different problem with the cohorts examined in experiments. They explore some consequences of crowdsourcing data, i.e. using large online surveys. There are, it seems, people who regularly take part in psychological surveys, and who become rather expert: ‘the population which we are sampling from is surprisingly small and highly experienced in cognitive science experiments, and this non-naïveté affects responses to frequently used measures’. Larger samples are good, and not always using college students is good, but these screen-mouse-mind-warriors need careful thought. For example, Stewart et al. say, if you’ve taken lots of attention tests, that will affect future performance; if an experiment conceals from its participants what is really being tested (as many do), then experienced people will be wise to these strategies, especially if they have been debriefed before (as many are). They say this is verging on ‘a tragedy of the commons, where studies run in one laboratory can contaminate the pool for other laboratories running other studies’. It never rains but it pours.

*

Don’t watch the video. It may actually be bad for you.


I told you not to watch the video.

*

Yu and Reichle (see above) provide another warning against generalization. Their short essay is about how experiments on reading (how eyes move, for example) reveal differences between Englsh and Chinese as it is experienced on the page. They raise lots of questions about the shapes of characters and the role of phonology, and it’s clear that there is a great deal to learn about what happens to readers when they read.
      This is something that I have to think about all the time: not the eye movements themselves, but the paths and destinations that literature provides. There is a danger in generalizing about ‘readers’ and ‘the reader’: this might just mean ‘what I think’. It might offer a restricted view of who ‘readers’ are or should be; I am thinking here, for example, about claims about what might attract or offend them. For the most part this is a convention to be used well or badly: it can just undeceptively mean ‘what I think’, or rather, ‘my claim about what this work makes happen’ — not so different from what a poem ‘means’, and offered as a contribution to a conversation in which assent and recognition from others (or lack thereof) will determine whether the argument is a good one.
      As the essay by my friend Felix Sprang (also cited above) shows, there may be a temptation within the field of cognitive literary studies to feel that we are getting closer to saying what happens to ‘the reader’. Some scholars do get rather categorical in claiming that in the light of what an experiment shows about the mind, we now know what the effect of a given literary form is. I feel like I am almost always trying to handle this carefully, and not closing off interpretation in this way, but I probably haven’t avoided the trap every time.
      Sprang makes clever use of some eye-tracking research to suggest that readers differ from one another rather a lot; and that, interestingly, that difference expands in re-reading, rather than evening out as variant readers coalesce. So while we may have tools that can tell us about what happens to readers individually when they read sonnets individually, and that’s a good thing, we may not be able to get closer to what happens to ‘the reader’ in the abstract in relation to ‘the sonnet’ in the abstract. This is a very useful and interesting challenge, and all part of the anxiety of generalization that I’m tracing in this post.

E-mail me at rtrl100[at]cam.ac.uk

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.