Monthly Archives: May 2012

Focus groups, polling and the “American Idol Delusion”

“I’m watching you!”

When we pre-tested our first game, “Flashback,” the high-schoolers we polled universally responded that they would NOT play an American history game.  It was the academic course many indicated they hated most, and that playing “a game” revolving around history sounded as attractive as drinking a brussel sprout milkshake.  Yet within days of launching Flashback, the site went wild and within 10 days had 200,000 page views.  In post-game interviews, 97% of the kids who said they would never play our game said they would definitely play it again.

What does this tell us?  About the same thing that election pollsters already know but rarely tell anyone otherwise they’d be out of a job:  polling is unreliable.  In a recent New York Magazine piece, Gavin Polone wrote about a similar mistrust he has of focus groups, the small test audiences (generally 48 men and women) that rate TV pilots.  Mr. Polone observes, “The idea that such a small sample can represent the whole market for a show does seem ridiculous and bound to deliver anomalous results.”  Some of those results are notorious.  Focus groups gave extremely low ratings to “Seinfeld” and “Friends.”

So what is wrong with polls and focus groups, and even more puzzling, why do we continue to use them?

Without doubt, where polling goes off the rails is that they rely on personal opinion — and opinion that is derived in a questionable context.  We call it, the American Idol Delusion.  You know what I’m talking about:  in the early episodes of any season of American Idol, a judge asks a contestant, “So do you think you’re the next American Idol?”  And the contestant brashly exclaims, “Absolutely, I’m a phenomenal singer.”  And then when that singer opens his or her mouth, the worst screeching and out-of-tune caterwauling gushes out.  Randy pales and moans, “No no no, dawg — that’s just horrible!  Can’t you hear yourself?!”

No, they can’t.  Hardly any of us can hear or see ourselves objectively — or anyone else, for that matter.  Prejudice and self-delusion color our opinions.  So why do we continue to use polls and focus groups?  Because they generate quantitative results.  “54% are fearful about the economy…  7% are undecided.”  It all sounds so scientific.  But what are they measuring?  Opinions.  Employing “tools” (i.e. polls and focus groups) that do no more than measure opinions in order to make decisions can result in epic disasters.  (48 men and women, somewhere, LOVED “Homeboys in Outer Space” — or don’t you remember that series?)

So what is worth measuring?  Actions.  Yup.  You know, those things that happen in the real world and speak louder than words.  You may think you’re the next Whitney Houston, but if when you open your mouth you are Roseanne Barr yowling the National Anthem — well, it really doesn’t matter what you think.

That’s why when we build games that also need to serve as assessment tools, we measure the data generated by actions.  Not opinions.

I also mentioned context as important to evaluating the action.  With the Nuvana platform, most of the actions generated by players happens in a social environment.  That is, you do something, and you publish it to the community.  Something very powerful and magical happens within that context:  authenticity.  People who fill out polling questionnaires in private have very different, and generally less honest answers, than when they answer in a community environment.  Let’s face it:  it’s much harder to b.s. peers in public.  Everyone is watching and listening.  If it’s a good community, they will call you out — but hopefully in a supportive way.  That’s what Nuvana strives for:  authentic actions and self-correction by the involvement of the community.

We see this happen all the time with Nuvana games.  The ethics of a social network based on authentic actions always trend toward the moral high ground.  It’s not totally surprising.  There was an interesting phenomenon discovered by Melissa Bateson, a behavioral biologist at the University of Newcastle-upon-Tyne who was in charge of her psych department’s coffee fund.  She found that by putting up a picture of a pair of eyes, donations to the coffee jar went up threefold.

Hmm.  Maybe we should change our name to NooVana.  With the double O’s in the shape of eyes.

Seriously, we have no interest in playing the role of Big Brother for the gamified universe — but we are seeking to drive authentic behaviors for ethical communities.  The actions that result can be transformed into verifiable data with which one can build trustworthy assessments.

What does that mean in real world terms?  Well, back to Mr. Polone who finishes his New York magazine piece by writing:  “Really, this [focus group] testing ritual seems pretty outdated in the Internet age. Why stick with such a small sample size when we have the technology to quickly get the opinions of a much bigger audience? A network could put all of its pilots up on its website and let the entire populace of interested viewers vote for their favorites and leave comments on how they would like to see the show improved before it went to series.”

Sounds like the perfect premise for a Nuvana game, doesn’t it?

— JvK

Advertisements
Tagged , , , , , , , , ,

Win State

If you ask a game designer, “What makes a game, a game?” — many will answer with some version of the following:  “If there’s a win state, it’s a game.”  And then you look at those activities that have a traditional win states and your list quickly bulges with the usual:  chess, WoW, baseball, tic-tac-toe, Angry Birds — the list of games is endless.

But then it’s easy to expand the list to other less traditional game activities that still possess a win state:  political elections, educational grading systems (“I’m in the top quintile!”), eBay bidding, war, dangerous medical procedures “Patient X beat the odds!,” foundation granting processes (“We won the grant!”), the human race (well there are always winners and losers in a race), and even love (“I am going to win her heart!”).

For better or worse, that parsing process has helped to fuel the proliferation of gamification.  Is it that I reside in West Coast Technoville or because we’re in this “space” — but it seems every day we hear the rallying cry of Gamification from some of the most unexpected quarters.  (I’d list some here, but even the most preposterous in conjunction with our tool makes some darn good sense).  Alongside this rush to gaming are the accessories and features that come with it, these new, shiny tokens of achievement:  badges.  New badging systems are cropping up like wildfire.  Some from traditional sources, like the scouting community which has always proudly worn their embroidered emblems on their ubiquitous sashes.  Other badges are attached to new systems and games in the digital media learning arena with a whole new set of graphics, titles and certifications — but all meant to be “worn” on virtual sashes over swelling digital chests.

With the Nuvana MVP, our users can import whatever badging system they prefer — or create their own, or use our default badge set.  That’s what certain communities need and demand.

But at Nuvana, we think it’s important to remember that there is a significant population of “active engagers” (our sometimes-used, less trivializing term for “gamers”) that actually hate badges and scores.  And that’s putting it mildly.  The ostentatious display of achievement only serves a certain kind of player.  The quest for badges and subsequent community admiration is in many cases, a turnoff to innovators, artists, out-of-box thinkers, and geniuses.  It’s also a turnoff to kids with already-low self-esteem who are one gold star-less day of becoming dropouts.  Do we really think that trading a letter grade for a badge or a snazzy score because a school is suddenly “gamified” is going to make a difference to that disengaged teen sitting in the back of the class with the hood over his head?

Marlon Brando and Woody Allen’s refusal to attend the Academy Awards to receive Oscars is legendary.  The greatest golfer of all time, Bobby Jones, the only man to win the Grand Slam, retired at 28, thoroughly uninspired by tournament play and the thought of winning more trophies.  Jean-Paul Sartre not only declined a membership to the Legion of Honor, he also refused the Nobel Prize in Literature in 1964 citing it went against his philosophy of how a true writer would behave.

What we strive for at Nuvana is to create games and tools that fit the unique ideals, principles and sensibilities of the organization that adopts it — and even more important, for the community it wants to engage.  You want a competitive game with high scores and badges?  Great, the Nuvana MVP can serve you.  And you over there — you want a tool that provides a thoughtful, collaborative journey without scores or medals?  Well, the Nuvana MVP works just as well for you too.

A deeper question might be, what does your community think a win looks like?

What we’ve realized over the years is that there is no one “win state.”  Belief in the One Win State creates too many losers.  The best definition of win state is derived through personal and/or institutional actions and behavior, and that individual’s or culture’s sense of achievement and quality.

I had the good fortune to go to an amazing high school that both my children were able to attend as well which embraced a controversial “no grade” policy.  Really.  You wrote papers, took tests — but there was never a letter or number stamped on the result.  Instead, assessment was a rich and involved process of self-evaluation mixed with detailed teacher commenting connected to an ever-present rubric.  The result?  There were no A-students.  There weren’t even D-students.  At least, they weren’t easily identified.  Still, every student knew where they stood — and the path they needed to follow.  This unique school fostered a culture of life-long learners who did not look to others for judgement and glorification, who were driven by a passion for inquiry.  A momentary failure was only an experience that deepened wisdom.  A so-called “success” was only a step on a path with many forks.

Rudyard Kipling’s words over the door to center court to Wimbledon are well known:  “If you can meet with Triumph and Disaster, And treat those two impostors just the same…”  Understanding those two lines will lead to a higher understanding of Win State, without doubt.  But do you know how Mr. Kipling’s poem ends?

“…If you can fill the unforgiving minute
With sixty seconds’ worth of distance run,
Yours is the Earth and everything that’s in it,
And—which is more—you’ll be a Man, my son!”

Totally committed action and relentless passionate engagement is the ultimate win state.  That’s why we at Nuvana believe…the New Win = greater good in the real world.

Any way we can get to that state, whether through a game or a collective journey where everyone is a winner — well, that’s our goal.

—  JvK

Tagged , , , , , , ,

The Best Game Platform

THE BEST GAME PLATFORM

When we began designing the Nuvana MVP (Mega-Versatile Platform), we spent a lot of time looking at one game platform that had a lot of the features we wanted to embed in our product.  The more we looked at it, the more we saw design genius.

It is:

  •          Inexpensive
  •          Supports individual and group play
  •          So simple, it never comes with instructions
  •          Yet generates thousands of different games
  •          Can be played for win state
  •          Or can be played for non-quantified amusement
  •          Playable by all ages – from toddlers to adults
  •          Male and female friendly
  •          Creates social networks
  •          But also suitable for solitary use
  •          Builds math skills
  •          Sharpens memory functions
  •          Is small and exceedingly mobile
  •          So indestructible and foolproof, makes “Help Desk” unnecessary
  •          Access to technology not a barrier, yet —
  •          Excellent for desktop and mobile conditions
  •          Easily adapted to myriad graphic designs
  •          Can support branding or advertising (on flip side)

What’s this awesome platform?

by http://bestdaybiomechanics.com/2012/03/28/deck-of-cards-workout/

  •          Oh, and it’s adaptable for real world activities too…

Hopefully, the Nuvana MVP is as fully featured.

—  JvK

Tagged , , , , , ,

CHANGE THE WHA–??

Three little words.  Barely enough to make a sentence.  But for a mission statement, it’s a huge mouthful:  “Change the world!”

Right from the beginning, we felt that our team, our platform, and our efforts should aim to accomplish a higher cause.

Believe me, there were skeptics.  Even the first coders we first hired could barely contain dubious snorts when we told them world change was our goal.  And who could blame them?  Some of these guys had built websites for global banks, coded for some corporate heavy hitters.

But within hours after launching our first game and the site was going wild with amazing activity, one of those skeptical techies called us, screaming over the phone, “Are you looking at the site!  Oh my God, you guys were right!  We’re changing the world!”

And that’s why, after all these years, we keep at it.  Each time we launch a Nuvana game, we see the incredible effect of individuals and communities making an impact on each other and the world around them.  What’s become clear and why our aphorism reads:  “Change behavior – change the world!” – is that powerful and lasting change in the real world comes about because of human actions.  And to create action where there has previously been inertia, you have to change behavior.

That’s what the Nuvana tool is all about:  changing behavior.  We use it to engage the failing student in a subject she previously hated.  We use it to change the eating habits of an obese individual.  We use it to encourage the researcher to share best practices and collaborate with someone he previously thought was a competitor.  We use it to gather apathetic residents and turn them into concerned citizens that work together to improve their community.

What behaviors are you looking to change?  What actions are you looking to provoke?  What will the world look like and how much better will it be when you transform it?

That’s why we named the company Nuvana.  It’s the new Nirvana that we all make together.

—  JvK

Tagged , , , , , , ,

The Birth of Nuvana

Back in 2009, when Jeff and I began Nuvana, we held a brainstorm with our talented team of designers, technologists, educators, gamers, and all-around, hardcore doers.  Our office was pretty much a big, empty space dominated by a ping-pong table we used as a conference table with nothing on the walls except for two huge white boards.  In the corner of one of the boards, I wrote what you see in the picture:

Image

1 TOOL = NUVANA

“Change behavior — Change the world!”

It’s been our mantra, our mission, the guiding principle that has led Nuvana from birth to where we are now.

Believe it or not, in the ensuing years, we’ve never erased that scrawl from the board.  At times, we trace over those words when they fade or when someone’s new idea bleeds into that box.  But it stays on the board and it will likely remain:  to remind us why we’re doing what we’re doing.

Don’t get me wrong:  we love games and we especially love having fun which has helped us to gather the kind of cool folk that make up the Nuvana team.  But we all know on some level that what we are trying to accomplish is more than just amusement.  At its core, Nuvana is about constructing tools that change the world…for the better.  If that tool is wrapped in a game that is fun and engaging — wonderful.  But the tool below that candy-coating better be kick-ass.

In general, we believe there are three principles that create good tools:

1)  A good tool is powerful.

2)  A good tool is absurdly simple.

3)  A good tool is achievable.

Creating a tool for the field of digital media learning while adhering to these principles is what drives us every day.  Can we make our platform more effective?  Can we make it easier to use?  Can we make it more cost effective and more easily distributed?

Yup, it’s the old Better, Faster, or Cheaper conflict – take your pick.  In most cases, you only get two.

Well, with Nuvana tools – we want our partners to get all three – with the added bonus of a fourth benefit:  fun.

— JvK

Tagged , , , , , , ,