Grounded versus Speculative Reasoning in HCI

As readers of this blog are well aware, HCI is at an interesting cross-roads. The history of the discipline is fundamentally scientific, with primary inputs from psychology and computer science. The future of the discipline appears minimally to include cultural, with the rise of affective, entertainment, domestic, social, and other culturally dense forms of computing. In its main venues, from CHI to Interactions magazine, it is crying out for approaches that will help interaction designers and HCI researchers work seriously on problems like the experiential qualities of interaction, interaction aesthetics, and so on.

I consider myself one of many voices trying to respond to that call. And perhaps one of the greatest challenges this agenda faces is the unspoken but omnipresent expectation that whatever solutions are offered will meet similar standards of scientific “rigor” that have been in place for decades. One problem, of course, is that culturally relevant knowledge is not necessarily the same kind of knowledge as scientific understandings of problem spaces.

Another problem, perhaps even worse, is the normalizing notion that traditional science has a monopoly on rigor and “serious” practices of knowledge production. This is not asserted explicitly, as a form of intellectual bigotry, but rather it comes out unconsciously, through habits of mind. And the goal of this post is to expose and subject to critique that habit of mind.

I came up against this in the course of doing a review for CHI2009. I was asked to review a paper that I immediately recognized as a work of philosophy in HCI, the kind of work that to me is responsive to the call for a more humanistic HCI. The paper explicated several widely accepted knowledge constructs in HCI, each of which were presented as a collection of principles. The paper offered an abstract, yet reasoned, analysis of these principles, suggested the existence of a gap exposed by said analysis, and offered to fill that gap with a new set of principles.

I quite liked the paper, but not all of the reviewers agreed with me. One of the reviewers in particular repeatedly claimed that the argument was “not grounded.” (The reviewer had other — and legitimate — objections, which I am not talking about here; I just want to take on this single objection.) Initially, I inferred that the reviewer was criticizing the speculative nature of the entire argument; there was no data, no study to support it. Later, that reviewer clarified and suggested that the argument would have been better if the author’s definitions of key vocabulary had been derived from scholarly literature. Instead, in one key instance it was derived from a “dictionary” (which was implied to be lazy), and in another key instance no source was offered (which was considered to be opinion-based and arbitrary).

Now, I should make very clear that although the author was roundly criticized for using insufficiently serious sources, or no sources at all, neither the reviewer nor the meta-reviewer made any suggestion whatsoever that there was anything wrong with the author’s actual use of this terminology; they did not suggest that there was any limitation, distortion, incompleteness, conflation or any other legitimate problem with the terminology. The only thing wrong with it is that its provenance was not deemed “serious” enough. As I learned in a 100-level logic class in 1993, this argument is a logical fallacy (it depends on an “appeal to authority“). Put simply, the argument was deemed problematic not because anyone could find any fault with it anywhere, but simply because it failed to showcase the apparatus of intellectual seriousness.

As someone trained in philosophy (it was my Ph.D. minor and my dissertation, recently published by Routledge, is about the philosophy of language), I have a real problem with these objections. In logic, one is allowed to stipulate premises as a part of an argument, even far more outlandish ones than were found in this paper. And building on this stipulation, one can develop a rational argument. Obviously, anyone can reasonably question the stipulation, e.g., by saying that premise P stipulated as true is in fact not true, and therefore any subsequent reasoning that depends upon its truth is irrevocably damaged. All of that is fair game. But to say that premise P can’t be taken seriously because it came from source S, without even bothering to engage in whether premise P has any value in its own right, is just a poor philosophical response.

And therein lies the rub.

For what the other reviewer wrote was not a philosophical response. It was a response derived not from philosophy or the humanities, but rather from the social sciences (or so I infer–I don’t know who that reviewer is). And the social sciences are, for perfectly legitimate and appropriate reasons, much less speculative than philosophy. To oversimplify for purposes of clarity, social sciences are geared toward a rigorous and (as) objective (as possible) understanding of external reality, while the humanities are comparatively more oriented to rigorous critiques of knowledge / understanding itself.

Part of social science’s rigor is in “grounding.” There are two acceptable ways (well, more than two but I’m focusing on two here) to ground reasoning in social sciences: one is through the careful collection, analysis, and interpretation of data. One eye opener for me as a humanist entering HCI years ago was (to me, at that time) obsessive care with which claims were made. It seemed to me then that social scientists would only make the tiniest, safest, most conservative claims; they shied away from the bold and interesting ones that really push understanding. Now I understand why that is the case: when you are making truth claims about reality, unless you have that care, there can be serious consequences as a result of speculation not only to knowledge of a state of affairs, but also action taken based on the assumption that that knowledge is true (policymaking, design, and other human interventions intended to change our world for the better). The other acceptable way to ground reasoning is by appealing to some other authority who has already done such an analysis. In this special and limited context, appeals to authority in the social sciences are, if not logically airtight, at least able to provide the epistemological foundation required for the work of the field.

Philosophy and more generally the humanities, in contrast, are not as strongly oriented toward truth claims about the world as it actually is. Rather, they are comparatively more oriented toward changing and reconstructing knowledge. Not surprisingly, they are also more skeptical of appeals to authority. They are often at their best when they help us think in new ways, see things in a new light. Where science is comparatively more ambitious ontologically (understanding the world as it is), the humanities are comparatively more ambitious epistemologically (changing understanding itself). So the most important question of a philosophical paper about principles in HCI is not whether the argument is grounded (an ontological concern), but rather whether the paper helps us think more productively about our field (an epistemological concern). And no one can rationally evaluate whether it helps us think productively if we are dismissive of it, not on its merits as a coherent theory/argument, but rather on concerns about the provenance of particular parts of the argument.

The differences between ontological and epistemological orientations in reasoning can be seen in the diverse discourses on the body, for example. Medical researchers have recently identified a link between a gene and transsexuality. Postmodern feminists, such as Judith Butler, argue that gender is a social construct created and maintained by the gendered performances into which we all are forced to engage, and transsexuality is in Butler’s analysis powerful evidence of the social constructedness of gender. On their face, these two discourses seem to contradict each other, since how can a gene be causally linked to a social construction of knowledge? But both discourses are speaking at different levels, two ships passing in the night. Butler is not suggesting that the science here is simply wrong and delusional, and the medical researchers aren’t claiming that they know what causes a social construction of knowledge. Rather, Butler is offering an alternative explanation of gender as a means of helping all of us think more critically about the relationships between physical bodies (the male, the female) and the meanings we ascribe to them (masculinity, feminitiy, transsexuality, perversion, sickness, etc.), a concern that envelops all human intellectual activity, from medicine to literary theory.

Let me return to the CHI review. It is certainly the case that any CHI paper should be evaluated carefully for its rigor. As I have tried to show here, this particular standard of rigor (i.e., the need for every element of an argument to be grounded) applied to this paper by the other reviewer was, in my opinion, not “too high” a standard for philosophy, as if philosophy has lower standards of reasoning (!), but rather a priori the wrong standard. A philosophical argument was evaluated as a scientific argument; both its philosophical rigor and contribution were missed because it failed to meet the normative standard of scientific reasoning, even though it never presented itself as such. From a philosophical point of view, if the offending part of the argument is valid or correct, then it doesn’t matter where it came from; if it is wrongheaded, then it also doesn’t matter where it came from.

And this episode is a typical, not exceptional, example of a major intellectual problem in HCI. Normative notions of science are being used to dismiss legitimate humanistic work by the very same people who are crying out for better work on the cultural, experiential, and speculative dimensions of HCI’s enterprise. The converse is also true: humanistically marginal work is being accepted because it conforms to the veneer of scientific presentation. The result, as I have argued elsewhere, is that piecemeal and simplistic ideas from the humanities are informing HCI instead of rich and robust ones.

If HCI wants competence in the cultural dimensions of interaction design, it must first have literacy in the intellectual disciplines that specialize in them.