Should the critical-interpretavist researchers of CHI leave the design subcommunity?

In this month’s Interactions magazine, there is an article I feel ambivalent about. It is Gaver and Höök’s “In Search of the Elusive CHI Design Paper“. I am ambivalent because there is much that I agree with and support, but at the same time some of its language–and practices it seems to support that I have seen and experienced–marginalizes research that I believe in and that I believe contributes to design.

The Article’s Argument

Gaver and Höök’s primary purpose is to address the problem that that not a lot of design papers are submitted to or get through the Design subcommittee of ACM SIGCHI. I think the authors’ desire to get more design papers (which I understand in the sense of reflecting the practices of design as a well established discipline) in CHI is one I share with the authors, so I have no objection to the primary purpose of this short article.

The authors further argue (1) that what is submitted to the design subcommittee reflects an over-broad understanding of design and (2) that design subcommittee submissions suffer from inappropriate reviewing standards applied to design.

Here is a key passage about (1):

We worry that Design is becoming something of a catch-all category at CHI. In many ways, it’s a good thing we provide a home for submissions that don’t fit easily into other categories, particularly papers that are risky, transdisciplinary, or unconventional. But all too often we encountered submissions that seemed better suited to one of the other subcommittees at CHI: Understanding Users, for example, or Interaction Techniques, Devices and Modalities. Why do these get sent to Design?

Of course, design has always been a difficult discipline to define. After all, we talk about interaction design, but also user interface design, or the design of computer architectures, or the design of data structures. For that matter, it is not unusual to hear that everybody is a designer—though we tend to agree with Bill Buxton’s rejoinder that if everybody who chooses their own clothes is a designer, then anybody who can count change is a mathematician.

Scoping design so broadly doesn’t seem useful to us. Instead, we think of design as involving certain skills and practices, including, for instance, planning and making finished artifacts, creative processes for reframing problems and developing design spaces, engagement with settings, material explorations, and an attention to aesthetics that seeks not just to make things beautiful but also to convey cultural identity, guide expectations, and shape a dynamic gestalt. Design in this sense may be pursued by individuals, teams, or collaborative groups; design work is increasingly distributed, outsourced, or left open for completion by end users. Nonetheless there is a family resemblance to design practices that Nigel Cross characterized as a way of thinking, involving synthetic, proactive approaches to understanding and shaping the world through artifacts. These are the sorts of practices developed through specialist courses and educational institutions, and though they can be pursued independently, it is important that they speak to those communities: Not everybody who makes something is a designer, and not all studies with relevance to design are design research.

[Full disclosure: As its authors note, the Interactions article reflects a discussion held in one of the design subcommittee’s groups; I was at that meeting where this was discussed.]

Though I see their point about the risks of an overly broad understanding of design, I have reservations about blaming the problem of too few design papers at CHI on such definition. That seems to suggest that the “real design papers” [sic] are being crowded out by, I guess, “fake design papers” (?). At the most superficial level, if it’s the case that papers are submitted that are out of scope of the design subcommittee, then it should be a straightforward matter for the subcommittee to refuse to consider them–either by rejecting them or by passing them to a more appropriate committee–a suggestion the authors also make. So out-of-scope submissions is not the real problem, and so I’m not sure why it was so important excommunicate them, if the goal is to publish more “real design papers.”

For Gaver and Höök, “The more important question to us, however, is why we don’t see more ‘real design papers'” submitted in the first place.

At this point, the authors turn to point (2), offering an interesting and insightful account (which is partly an outcome of the discussion at the PC meeting) of why “real design papers” get shot down by reviewers–they are expected to meet dubiously applicable criteria:

CHI design papers should—according to folk wisdom—be framed in terms of an overarching design approach (ideally new and with a catchy name), motivated by a set of specific research questions, accompanied by an extensive literature review, and analyzed in a lengthy discussion to produce generalizable lessons, ideally in theoretical terms.

But so far, what Gaver and Höök have offered is an argument to shift the standards of reviewing of “real design papers,” and I agree that stakeholders in the design subcommunity within CHI should take up such a project in earnest. Playing devil’s advocate, I can also see a counterargument that says, if you want to be a part of CHI you need to meet CHI’s standards and be legible to CHI. And a counterargument to that would be that if CHI wants to be influenced by design, it needs to make room for, understand, and respect epistemologies, methods, and discursive genres from design. (Personally, I think a reasonable outcome is somewhere in the middle, and it not only will, but should be negotiated in an ongoing way over time.)

But anyway, the authors have not made a clear argument about how to get “real design papers” submitted in the first place. The article ended with a To Be Continued, so hopefully they will get to that.

What About Us “Fake Designers”?

I wonder, however, whether the authors have considered the political effects of the framing of their article in terms of “real” and implicitly “fake” design, because it seems to me that the authors are excluding or at least marginalizing certain types of contributions that have been an important part of the design subcommittee in recent years.

Why do I say that? Gaver and Höök’s strategy of asserting that “design” is over-broadly defined in HCI–they use the pejorative term “catch-all” and add the reductio ad absurdum Buxton quip–asserts strong and explicit boundaries between “real design” and everything else. Their actual and repeated use of the term “real design” furthers this agenda. They even go so far as to define “real design papers”:

what we mean by real design papers is ones that focus on one or several of the practices listed earlier—the making of artifacts, creative processes, material explorations, or aesthetic crafting. Insofar as such concerns are the heart of design, it makes sense that they should be the topic of design papers as well. [emphasis in original]

Further, the language of the article (e.g., “it’s a good thing we provide a home for submissions that don’t fit easily into other categories, particularly papers that are risky, transdisciplinary, or unconventional”) reinforces a normative understanding of what is core and what is peripheral. And decisions about exemplary papers, including which papers are offered up as good examples to the community in the CFP, also seem to reflect this understanding.

My conclusion reading this is that I am a “fake designer” and that my contributions to the design subcommittee have been welcome but also not part of its core. And, dear readers, many of you are fake designers, too.

I asked in that discussion whether the committee was proposing a divorce between design and the critical-interpretativist thread of research, which I identify as a part of, and which has long been aligned with design in CHI. I never got an explicit answer in the meeting, but I think there is an implicit answer in this article as written (as well as in choices made about what papers best exemplify the design community).

So unless one views design theory as a material, the construction of an essay as a creative and material exploration, and the construction of interpretative understandings of design contributions as a form of aesthetic crafting; and unless one considers design criticism as a means of exploring and developing design spaces that “make things beautiful [and] also to convey cultural identity, guide expectations, and shape a dynamic gestalt”–then the work in the critical-interpretativist tradition at CHI is not “real design.” I don’t think Nigel Cross would see it as real design, and I am not sure whether those in the room where this discussion unfolded did either (the agenda-setting and visionary contributions of some of them to that very tradition notwithstanding).

What Should CHI Design’s Borders Do?

Borders perform two types of work. First, they assert and maintain difference–ensuring that elements from both sides don’t intermingle with each other. Second, they do almost the opposite: they allow elements from each side to cross into the other, in a controlled and structured way.

I am not opposed to some notion of borders to help clarify and support distinctive contributions of design to CHI.

But I also believe those borders are presently ill-defined for historic and epistemological reasons: the whole CHI community is still working out how it relates to design, what that relation means, who wants to be a part of it, and how.

And I don’t think the design subcommunity should unilaterally define that relationship.

Further, if the design subcommunity is not seeing the papers it wants, I would think the best strategy would be to invite the kinds of contributions it wants by welcoming, collaborating with, and helping to cooperatively shape those of us who want to engage design. That was my experience 10 years ago in the design subcommunity of CHI.

More recently, I am witnessing and experiencing something different, where the design subcommunity seems to be denigrating research it doesn’t consider to be “real design.” I am far from alone in feeling this way. I don’t believe this is intentional in many cases, but it is making me–and many critical-interpretavist colleagues I have spoken to–ask: where should our work go, if we want it to be read as making a more serious contribution than “fake design”?

Should the critical-interpretavist researchers of CHI leave the design subcommunity?

Conclusion / Recommendation

My own view is that there must be a way to promote “real design papers” in a way that nonetheless includes and welcomes fellow travelers on the borders–shaping their work and pulling them into the project. So I hope as a community we can attract more design papers into CHI, but I hope we can do it in a way that doesn’t marginalize and alienate those who have given much to design research.

Whatever the outcome to that, I hope that everyone who has a stake in design at CHI has a chance to participate in shaping the borders around design at CHI, as well as who gets to cross them, and with what kinds of welcome once they are there.

Posted in Uncategorized | 3 Comments

The Materiality of Research Practice

This is a quickie today. Recently (readers might have noticed) I have been reflecting a lot on my research practice. I have been trying to find those moments when I got things right and better recognize when I did not. And I am sharing this reflective process publicly in hopes of helping others find ways to improve their reflections and practices.

Anyway, I came across a wonderful article on the materiality of research practice, which I highly recommend. (Indeed, I recommend it more than finishing this blog post, if you can do only one or the other…)

It is: The Materiality of Research: ‘On the Materiality of Writing in Academia or Remembering Where I Put My Thoughts’ by Ninna Meier

There was a passage in it that really resonated with me:

I actually like review processes or co-authors: they provide a much-needed break and opportunity to distance myself from the text (it’s at their desk, so to speak, and not on mine).

The scary part of this is when it comes back, the text. When this happens, I am often appalled at how unfinished it was when I submitted it and even more appalled that I couldn’t see it at the time (how can this be? I have never understood it). This is perhaps why I don’t trust the ‘current-author me’, because ‘future-author me’ will have read more and understood more and thus be able to write much better thoughts into the text and improve the overall result.

This passage helps explain a deep shift in how I view myself as a peer reviewer. When I was younger, I sought to “protect” the research community from “bad” research. It looks so ugly typed up like that, but there it is.

Continue reading

Posted in Humanistic HCI, Peer Reviewing, Prewriting, Writing Process | Leave a comment

Critiquing Scholarly Positions

If I am right that HCI and neighboring fields will increasingly rely on the essay as a means of scholarly contribution and debate in the future, then it follows that the construction, articulation, and criticism of intellectual positions will become increasingly important.

In Humanistic HCI, we talk about the essay, the epistemic roles of positions, and how they should be peer reviewed. We defined a position thus:

[A] position is not merely a proposition; it instead holistically comprises an expert-subjective voice; a theoretical-methodological stance; its own situatedness within a domain; and a pragmatic purpose. (73)

But as I read design research papers in the fragmented and emerging subdomain of research through design (and similar practices, including constructive design, critical design, and so forth), I have been frustrated with how researchers characterize others’ positions, especially ones they disagree with.

The purpose of this post is not to discourage disagreement.

It is, rather, to support disagreement in a scholarly way.

Continue reading

Posted in Design Process, HCI, Humanistic HCI, Interaction Design, Research Through Design | Leave a comment

A Dark Pattern in Humanistic HCI

I have noticed a dark pattern among papers that align themselves with critical or humanistic approaches to HCI. I myself have been guilty of contributing to that pattern (though I am trying to reform). But I still see it all the time as a peer reviewer and also as a Ph.D. supervisor.

And since I spend so much time evangelizing humanistic HCI, I thought it might also be good to point out one of its dark patterns, to encourage critical/humanist HCIers not to do it, and to encourage reviewers to call this out and use it as an argument against accepting the paper.

And of course I want to offer a positive way forward instead.

The dark pattern is:

“I love a critical theory/author; you in HCI should change your practice to use it, too.”

Characteristic features of this dark pattern include the following:

Continue reading

Posted in Criticism, HCI, Peer Reviewing, Rant | 6 Comments

The “Knowledge as a By-Product of Artistic Practice is Still Not Research” Objection to My “Criterial Knowledge” Post

I spoke to some colleagues about my earlier post, The Criterial Knowledge Argument for Research Through Design, who are themselves experts in research through design [EDIT: the researchers in question are Jodi Forlizzi and John Zimmerman]. While these colleagues were generally sympathetic to the claim that art and design can contribute to knowledge in general and even criterial knowledge in particular, they objected that the sorts of knowledge outcomes I describe in that post can be arrived at not intentionally but rather as a by-product of artistic practice, and therefore cannot be characterized as research.

This is a very reasonable objection. In fact, their objection could be stated even more forcefully: the knowledge outcomes I describe in that post are almost always achieved as a by-product of artistic practice and therefore almost never research as such (at least historically). We don’t view the poet Virgil as a “researcher,” even if his literary depiction of Dido has informed Western people’s cultural understanding of the character and qualities of jealousy for two millennia.

Their objection made me realize that I had left certain things tacit in that post that should have been made explicit. And upon further reflection, I developed some of my ideas about these issues further than I had before. So here are some amendments to the earlier post, addressing their objection:

  1. For a design researcher doing RtD to be able to invoke the Criterial Knowledge Argument for Research through Design in presenting/publishing her own work, she would have to justifiably claim that the processes, methods, and desired inquiry outcomes of the RtD project in question were designed to contribute to criterial knowledge of a given domain, concept, phenomenon, experience, etc., and she would likewise have to demonstrate that she did indeed achieve such knowledge.
  2. If, on the other hand, she was trying to do something entirely different (e.g., in the hope of contributing to “intermediate concepts” that characterize several successful designs) and as a by-product just so happened to contribute to criterial knowledge about X, then the latter knowledge outcome would obviously exist, but it wouldn’t be research (in the same sense that Virgil wasn’t a researcher when he wrote The Aeneid, even if the latter did result in criterial knowledge of jealousy).
  3. Let us consider another example, that of a design researcher theorizing a domain/phenomenon/matter of interest, whose methodology includes a critical examination of other individuals’ research through design projects (and sadly there is not enough of this at present, at least in HCI). If this design researcher is able critically and analytically to discover and to explicate that this corpus of designs do contribute to criterial knowledge of X in such-and-such ways, then that would be research, but it would be design criticism research, and also not research through design.

Centuries of critical interpretation and analysis of works of art as well as the theorization of that critical practice has shown that art works contribute to intellectual virtues in incredibly broad and diverse ways (e.g., criterial knowledge, improvement of our capacities of perception, rendering us more empathic and less egoistic, opening our minds to new ideas and values, and much more). In parallel ways, research through design is likely to achieve many of the same intellectual benefits.

My colleagues’ objection has helped me clarify that it is important to distinguish between those knowledge outcomes intentionally “baked into” a given RtD process as central to, even the point of, the research, and those knowledge outcomes that happen externally to such a research project. I’ve identified two categories of the latter: cognitive by-products of a design practice intended for some other purpose (e.g., to produce a good design), and subsequent critical discoveries, often made by third-parties.

I view it as a happy problem that RtD has such high potential to yield “surplus” knowledge outcomes, but less salutary that our theoretical and reporting vocabulary does not yet allow this community to do them justice.

Posted in Aesthetics, Design Process, HCI, Interaction Design, Research Through Design | 12 Comments

The Criterial Knowledge Argument for Research Through Design

NOTE: This post was prompted by a lively discussion on the PhD Design distribution list. Special thanks to Mike Zender for prompting me to flesh this out..

One of the methodological strategies I have been using for the past year and a half for my study of research through design is to construct an inventory of arguments that I find in philosophical aesthetics, literary and film criticism, and art history and criticism that assert that an artwork (literary, visual, cinematic, whatever) does inquiry, that is, it contributes to knowledge.

In this post, I sketch out one such argument, and then I apply it to research through design, and that is what I will call the Criterial Knowledge Argument for Research Through Design.

The Criterial Knowledge Argument for Literature

First, I sketch the argument as it has been developed in philosophy. Here I will sketch out the view of philosopher John Gibson, from his Fiction and the Weave of Life (Oxford UP, 2007)

Continue reading

Posted in Aesthetics, Criticism, HCI, Research Through Design | Tagged , , , , | 1 Comment

Research Through Design: A Humanistic Conception

Note #1: This is adapted from a post I submitted to the PhDDesign distribution list’s thread on “Research through design.”

Note #2: This is a preliminary attempt to formulate the perspective I am developing in my sabbatical monograph, Design As Inquiry. If you like what I’m doing, or if you think I’m nuts and want to argue with me, please contact me!

It is well known that in spite of considerable excitement about research through design (and constructive design, speculative design, etc.), that many questions remain about its status as “research,” including whether/how it conforms to our conceptions of research, how its impacts are to assessed, and how to distinguish good from bad design projects aspiring to one of these contribution types. This has major implications, e.g., for funding, publication, tenure, and indeed whether Design can justifiably award Ph.D. degrees.

One reason for the present difficulties is that it seems difficult to ask how design does “research” without becoming bewitched by scientific research norms or foundationalist attempts to find powerful arguments in Frayling that frankly aren’t there to be found. As someone trained in the humanities, I have been wondering–sort of an elaborate thought experiment–what RtD looks like if we imagine it in relation to a more humanistic conception of inquiry.

​One common argument views design inquiry as a form of “exploration.” I think such a view, unless carefully qualified, understates the rigor and knowledge contributions at stake. ​The novels of Henry James do not merely “​explore” moral life–they systematically interrogate it, situated in a complex world of particulars, of events, of emotional ​resonance​, in a way that rivals Kant’s and Rawls’ moral philosophy​ (Nussbaum, 1990). Warhol’s Brillo Box does not merely explore popular culture–it interrogates the ​very ​theories by which we are able to recognize art as art (Danto, 1981). The Aliens films do not merely explore personhood–they interrogate different formulations of it, working through the consequences of diverse positions, and clarify for us what is at stake in these formulations ​in our increasingly biotechnical world of the proximal future ​(Mulhall, 2002). ​The tragedies of Shakespeare do not merely explore skepticism; they interrogate its arguments, work through its psychological motivations and behavioral consequences, and reveal how disowning knowledge leads to the un-acknowledgment of our human relatedness and mutual obligations (Cavell, 1969). ​Is not the systematic consideration and critique of different positions and their consequences knowledge work?

Continue reading

Posted in Aesthetics, Criticism, Design Process, HCI, Research Through Design | Tagged , , , | 2 Comments