Monthly Archives: July 2012

Measuring the Immeasurable

Those of us who create behavioral change tools where the holy grail is adoption by academic institutions, non-profits, corporations, health and military organizations, the quest for legitimacy is a relentless task.  Fellow colleagues back in the 90s were simply called gamers.  Then came the millenium and we all got a nice promotion to “game designers.” But that was nothing when we somehow achieved a title bump to “serious game producers,” and then just recently, I heard an ARG serious game designer described as a “transformational game practitioner who impacts the digital media learning space.”  Now we’re hearing about “interactive architects building game-infused learning trajectories for core curriculum.”

Okaaay.  That just won’t fit on my business card.

All of this long-winded jargon provides gravitas to what was once considered a trivial pursuit.  A game.  No school superintendant or CEO or exec director in their right mind some years ago would have allowed a game in their institution.  A “game-infused learning trajectory” — yes.  But not a game.

What differentiates the two interventions?  Verifiable data.  Games merely entertain.  But a serious game lifts the learning environment.  And this lift must be quantifiable.

Lord Kelvin on an early “tablet computer”

In these contexts, you will often hear a well-worn saw generally attributed to the great 19th-Century physicist William Thomson, 1st Baron Kelvin, most frequently referred to as Lord Kelvin:  “If you can’t measure it, you can’t improve.”  And thus, on the backside of most good learning games (and certainly, this is true of Nuvana games), you will see reams and reams of data generated by the activities of the players, and sometimes, the mentors.  Our new digital platforms themselves can’t help but generate the data given their mathematically encoded make-up.  It’s why researchers, academics and evaluators love the medium.  Every good digital serious game generates an endless outpouring of whos and whens and wheres and hows that are easily sliced and diced by age group, sex, demographics, racial profiles, economic class, geographic pinpoints, times of day, device used, and of course, type of gum chewed.

Where is this leading?  The holier than holy grail:  assessment.

If admin can tell by a player’s score, or level, or game outcome HOW they are doing relative to their grade, their work force, their disease cohort, their political grouping, their target market peer group — then the presence of that game within an institution is not only legitimate — it’s necessary.

Or is it?

If there’s anything that’s truly measurable in the last three decades of American academics and the U.S. workforce, is that we have an alarming dropout rate and an even more disturbing unemployment rate.  The years of No Child Left Behind and rigorous standardized testing which gave us a certain kind of thinker — for better or worse — has helped to generate these stunning results.

To cloak an SAT score, a STAR test, an employee assessment (can you say “urine test”) — in the sheep’s clothing of a “transformational game” — transforms nothing.  Most users see through that kind of inauthentic camouflaging.  Interestingly, the games that give one the most “reliable” data are actually “closed loop” games.  That is, there are right and wrong answers, and thus, results are verifiable.  But those make for very dull games when everyone is aimed toward the same outcome.

The most interesting transformational games, like life, are where there are many correct answers.

In fact, if you really think about it, it is those things that we CAN’T measure that improve us, by and large.  Spiritual fulfillment, fun, love, creativity, inspiration, friendship, familial bliss, meditation, relaxation, focus.  Enthusiasm for problem solving (and that is not measurable by getting the “right solution”).  These are all things that are hard to measure.  (Yes, we can measure activity that generates and surrounds some of those qualities — but, for example, is there a reliable creativity quotient?)  And yet, there is no doubt that when we immerse ourselves in these kinds of activities — we do feel vastly improved.

So at Nuvana, we’re of two minds.  Yes, we love data and we provide it.  But we also know that at times, it can be misleading.  That the games we want to make should also put the player on a path to improvement that has impact on what is generally immeasurable.  Immeasurable impact.  When that unquantifiable area is attended to, the “player” feels a wholeness that is generally missing in many of the institutions where they normally inhabit:  school, work, government, etc.  If the “player” feels improvement in that area, they engage and apply themselves more effectively at those quantifiable activities.

Improving the measurable is what all transformational games aim to accomplish.

Improving the immeasurable is the Nuvana secret sauce.

And by the way, about that Lord Kelvin guy.  A brilliant man.  But before you put too much faith in his little aphorism about measuring and improvement, just remember, he’s the scientist who wrote to the Niagara Falls Power Company: “Trust you will avoid the gigantic mistake of alternating current.”  And also predicted: “Radio has no future.”

— JvK

Tagged , , , , ,