The Office of Digital Humanities at the NEH just released a new study analyzing the efficacy of their Digital Humanities grant program. The response of SUG recipients was uniformly positive and they described the program as “hugely successful”. But a cursory glance of the appendix of projects reveals that for every excellent inPho there are a handful of half-baked digital humanities projects that sadly litter the landscape. The Start-Up Grants were designed as High Risk / High Reward, and so failure is a definite possibility, but it’s been six years since A Companion to Digital Humanities was published and participants in the field are still uncomfortable criticizing the work of their peers and of themselves. As I said on HASTAC last year, if the Digital Humanities is to be taken seriously, then we need to be able to distinguish between success and failure. Everything isn’t successful, even if it’s innovative, just look at Google Wave.
Like many people, I always meant to use Google Wave. The initial beta was only open to select digerati and the press release described a transcendent, synthetic form of communication that would do to email what email did to the telephone. Wave was multimodal, real-time and still grounded in text–which sounds like most Digital Humanities project proposals–and, unlike so many of those proposals, Wave actually got up and running. But last month, Google pulled the plug, and the more I think about this ambitious project, the more I think humanities scholars could do well to learn from Google’s experience.
It’s possible Wave was axed because of the use of a non-compliant Java framework, which would open it up to legal action by Java’s new owners at Oracle, and while this may have contributed to Google’s decision to drop Wave, I tend to think it had more to do with the tension between Wave’s grand potential and its rather middling adoption.
Ambition and feature creep go hand-in-hand
Google Wave was an email and a document and an instant message and more. It integrated video with text and put the whole thing together in a wiki-like editable space. The framework was new and innovative, but it was fundamentally a synthetic project, which is very common in the Digital Humanities. These synthetic projects are easy to envision, but if not carefully considered, they can also resemble a certain meme known as Yo Dawg:
The “Yo dawg” or “Sup dawg” image macro first appeared on the ’chans in early 2007 and experienced a resurgence in late 2008 on Reddit, Tumblr, and other mainstream forums & blogs. It follows a simple format:
Standard: {yo,sup} dawg, I herd you like X, so I put an X in your Y so you can VERB while you VERB
Repetitive: {yo,sup} dawg, I herd you like X, so I put an X in your X so you can X while you X
Abstract: {yo,sup} dawg, I herd you like X, so I put an Y in your Z so you can VERB while you VERB
Now, I’m not saying that Google heard we like videos and wikis and texting so they put all that in an email so we can do all that while we do all that, but I can think of several Digital Humanities projects that consist of simply bolting one concept onto another without any theoretical or technical effort expended to present a unified structure. When Ruth Mostern and I built the Digital Gazetteer of the Song Dynasty, we’d considered also building a web interface allowing search and visualization but decided only to release the raw database, because the web app, while highly visible (and therefore professionally valuable), wandered too far afield from the project’s initial purpose: to build a rigorous and sophisticated representation of medieval Chinese political geography.
Acknowledge when you don’t have an audience.
Even the most tedious academic scholarship has metrics for success based on audience adoption. Sure, your latest monograph on 17th century French medallions may not show up on the New York Times Best Seller List, but if it didn’t even get bought by the UC library system, then maybe that’s a sign that you, too, should move on. In like manner, if you’ve built something that no one is utilizing, then maybe it’s time to phase it out. Which brings me to the most important lesson that Google Wave taught us:
Don’t just wander away from a project, definitively end it.
A colleague of mine pointed out that the Valley of the Shadow Project had a definitive end and that now the library has the (not easy) task of maintaining it as a finished work. This is decidedly not the norm with Digital Humanities projects. Even the most superficial survey of “current” Digital Humanities projects will reveal that many are vaporware or abandoned. The lack of a definitive end product short-circuits the capacity for outside scholars to review the work of their peers and determine its quality. The field cannot be taken seriously as long as this remains the standard practice of its participants.
Reuse, Reduce, Recycle
I started this post by writing that everything isn’t a success, and I know that one argument against that is even in failure there are lessons to be learned, methods and technologies developed. Google is opening up more of the source code and, we can be assured, folding into other projects lessons and code that they may not be making public. But these steps, whether it’s turning over created data to library curation or making public the code underpinning your Digital Humanities projects, will be hindered or completely unavailable if you cannot acknowledge that your project is over. There are three probable results for any valuable technical or methodological assets of failed Digital Humanities projects that fail to acknowledge their failure: Either they’re distributed arbitrarily in a piecemeal, shadow economy, or they’re only publicly released when they’re outdated and the value has significantly declined, or they vanish and all the work and effort and thought that went into them might as well have never occurred.
One of the most common responses about the Digital Humanities SUGs was that just winning the grant established credibility for the scholar in question. Credibility cannot rest solely on the ability to get grants. The academy has a long tradition of peer review in the establishment of scholarly credibility, and though peer review is problematic, it is focused on results and on the ability for the scholar to properly describe their results, even when the original goal was not achieved. We cannot achieve that ability to review works unless the creators of the work can define its completion. And we also cannot place value on works unless we can acknowledge failure when it occurs. Google Wave was a failure, but Google isn’t. Likewise, digital humanities projects can be failures without the Digital Humanities being a failure. It’s time to acknowledge that.
And it’s not just DH. The whole idea of what and how we “publish” needs to be rethought. Just as a tweet can be a reasonable means of communication, there are many ways to release and end a project, including doing things like calling it a proof-of-concept. The problem is the arena: journals, websites, blogposts, just to name a few, all have implications of how others interact with the ideas. I would argue, however, that a few paragraphs of reflection attached to any project – living, dead, or dormant – are an important piece that we often leave out.
Pingback: Tweets that mention What the Digital Humanities needs to learn from Google Wave | Digital Humanities Specialist -- Topsy.com
Agreed, in general. Digital humanities projects need much better protocols for endgame. But I’d like to poke at this a little, because the ODH report is a surprising impetus for your post.
One great thing about the NEH SUG application process is the requirement to articulate what the measurable outcomes of the proposed project will be, and how they will benefit the related fields. At a minimum, there’s a publication — often in the form of a white paper; and in fact many publications have come out of the SUGs.
But being *start-up* grants, these awards presume further work beyond the grant’s scope, and some of the funded projects did not progress beyond their funding stage. Do I understand correctly — is this what you’re calling out as a failure? If so, then I’d say wait. The knowledge produced needs to reach interested parties, and they need to do something with it, and that takes time. We are many overlapping communities struggling with information and work overload; if some projects haven’t been continued, they may later. Or others may learn from them in the future. The length of “later” and “future” can’t be determined. If they were good ideas in the first place, three years is not enough time to make the success-or-failure call. My question is: did they do what their proposal said they would do? And did they publish the results, making them openly accessible and discoverable?
I’m in fundamental agreement with you, Vika, and I’ll admit that I’m agitating here. But I’m unwilling to follow you into this idea that all digital humanities projects were good ideas or somehow fundamentally successful simply because they exist and may prove useful to others at some point in the future.
As to your questions, my suggestion would be to do what I’ve done, and flip to the appendix and take the time to click on every example site (and make sure to correct for typos) and publication. I realize we’re all very busy, but I find it troubling that it seems that the entire DH community is too busy to have done that. I think, very quickly, you’ll be able to answer your questions and also get a sense of the level of quality of the work that came out of the SUGs.
Yes, granted, I did not click on every single link in there — and I think expecting DH community members to do that would be unreasonable. At best, you’ll get people checking out projects at least tangentially relevant to their own work. But point taken.
I’m far from saying that all digital humanities projects are good ideas. The projects that get funded by the SUGs undergo a rigorous peer review process in the application stage. Having experienced this process from both sides, I’m comfortable saying that if they did get funding, they were deemed by scholarly peers to be good ideas. Execution, now, that’s a different story.
It’s not the SUG process that I’m taking issue with, though I do think one of the major problems with DH right now is that everything is front-loaded toward grants and as such is more about cool ideas and less about results. Rather, the ODH report serves as a list of DH projects, and a survey that gives the sense of boundless optimism in the digital humanities–both of those deserve some criticism.
As to reading through the projects, it doesn’t take much time, but you don’t even have to go through the whole list. Even the first five or ten will give a very telling sample. It worries me, though, that there’s a sense that we can’t review the product but we can review the proposal, and judge the latter as good but only be able to assume about the former.
Shouldn’t the same scholarly peers be able to come to an equally valid opinion about product as about plan?
Surely you don’t think that ANY digital humanist is qualified to review ANY grant proposal? Reviewers are selected according to fields of expertise. So yeah, with the projects I can speak about, I can certainly address from both the proposal and the “final” product end (though products of SUG are explicitly *not* final). Others, I can’t. This goes for most of your other colleagues.
The ODH grant list is certainly very visible, this being The Government Office Funding DH Right Now. And yes, people need to both eat and prove to their institutions that this is worthy stuff, and grants are great for that. But saying that “everything” in DH is grant-oriented is silly. That the cool ideas, and not just the good results, are visible is a boon to the field. Maybe some cool idea that isn’t well executed will be taken up later by someone else.
I hear your skepticism about the boundless optimism, and though you’ve criticized it already, I’d be interested in a more thorough critique.
One thing you’re getting at is an interesting idea: having another peer review process *following* the end of the grant period, and somehow (anonymously) publishing the results.
The more I manage projects in higher ed, the more I come to believe that what instructional design and educational theory identify as “learning objectives” really need to be applied to all projects we would like to have measurable results.
In short, higher education knows an even more powerful distinction between “two cultures” than the one that C. P. Snow wrote about. Namely, on the one hand, we have very fine specialists in all kinds of subjects who teach.
And on the other hand we have educational specialists, instructional designers, or even project managers, who are more comfortable with stating objectives in terms of measurable outcomes.
Certainly higher learning, notably the humanistic kind, must needs have the very highest aspirations our culture can imagine. On the other hand, a shared conversation about how you demonstrate that a goal has been achieved–that is something we could all use a bit more of.
Best,
Edward O’Neill
Twitter: @learningtech
Excellent point. This is a subject that I plan on exploring in much more detail, but my sense, especially from discussions about the creation of Digital Humanities labs and departments, is that the two cultures are seen as even more distinct than ever, and even though there are more collaborative projects involving humanists and CS types, there’s not a true dialectic going on.
My own rather odd position at Stanford was designed to help with this by being something of an academic corpus callosum, trying to improve communication between technical and humanities participants. A more traditional description would be that I’m a software developer with a domain specialty in the humanities, but my favorite was when a Classics professor referred to me as “one of those guys, except you don’t despise us”. Regardless, there’s an obvious situation wherein the technical experts aren’t aware of of the sophistication and complexity of the humanities knowledge being addressed and the humanities scholars aren’t aware of how nuanced and powerful software can be, and I think that it’s a direct result of an inability to communicate due to jargon, intellectual bigotry, laziness and tradition. To be clear, I’m not claiming that’s the entirety of the field, but it has been a pervasive theme among the scholars with which I’ve interacted.
Obviously, this line of inquiry requires more space and time than a comment field…