Monday, June 14, 2010

Real Games Have Curves: Welcome to the Competence Zone

Let's make a graph. The horizontal axis is player skill. On the far left is no skill - just random button-pushing. On the far right is perfect videogame godhood, always doing exactly the correct thing at the correct time in the correct way. The first time you play a game, you'll probably be somewhere in the middle - farther right if you're a veteran gamer, farther left if you're a novice. As you play the game, and learn its mechanics, you'll trend right as you get better.

The vertical axis is performance level. At the very bottom is complete failure - game over as quickly as possible, not achieving any of the game's goals. Farther up is the passing line, separating failure below from success above. The line itself is a performance level of just barely passing a challenge - surviving the boss fight with one hit point left, clearing the race course just before the clock runs out, and so on. And at the very top of the axis is absolute perfect performance - winning by the largest margin possible.

Now we can chart the performance levels achievable with a particular amount of player skill: the "skill curve" for a given challenge.

Guitar Hero skill curve

Here's a skill curve for a song in Guitar Hero. The player must be fairly skilled to hold the right frets and strum at the right time, but does not have to be perfect in order to pass the song. Failing to hit notes drains a life meter, and when the meter empties, the player fails the song - but hitting notes correctly refills the meter. As long as the player doesn't miss too many notes in a row it's possible to miss a lot of them and still pass the song. This leaves plenty of room to improve - hitting a greater percentage of the notes earns the player a higher score, up to the perfect combo of hitting every single note.

The area between just barely passing and playing perfectly is the "competence zone." A player with skill in this range is competent to pass a challenge, but not necessarily expert in it.

Guitar Hero skill curve with competence zone

Guitar Hero songs can, of course, be played on different difficulty settings. Above might be the skill curve for a particular song on Medium. Switch to Easy and the curve moves left; switch to Hard and the curve moves right. The higher the difficulty is set, the more complex is the pattern of notes to be hit, and the more skill is required to perform at a given level. The game still allows the player to miss many of the notes and pass the song, so the competence zone is the same size and shape.

Let's compare this to Elite Beat Agents, a rhythm game for the Nintendo DS. Much like Guitar Hero, EBA features a sequence of notes that must be hit to pass a song, with missed notes draining a life meter and hit notes filling it. However, EBA introduces another factor - the life meter constantly drains over time. On low difficulties, the drain rate is slow, and there is not much effect on gameplay. But the life meter drains faster on higher difficulties, drastically reducing the safety net.

Here are skill curves for a song in EBA on the easiest difficulty and the hardest. On the harder difficulty, not only does the curve slide right because the note pattern has become more complex, but the curve also steepens because the player is allowed to miss far fewer notes before failure. The result is that the competence zone is much smaller - the player has to be much closer to perfect in order to simply pass.

Elite Beat Agents skill curves for Easy and Hard

It's a bit like moving an archery target farther away and making it smaller at the same time. It's harder to hit, and there is less margin for error. It requires more skill to perform at the same relative level - hit the same distance from the center of the target, or miss the same number of notes - but that level might also no longer be good enough.

Now let's consider Bit.Trip Runner.

Gaijin Games's fourth outing on WiiWare, Bit.Trip Runner has been well received. Like its predecessors, it's a graphically and musically nostalgic rhythm game - but Runner takes the form of a sidescrolling platformer. It's a bit like if Guitar Hero rotated the fretboard ninety degrees and instead of scrolling colored notes at you, disguised them as various hazards and obstacles that must be reacted to appropriately - but these actions still occur in rhythm, and create music.

My two favorite genres happen to be rhythm games and platformers. And if you're anything like me, the trailer alone demands that you purchase the game immediately - especially at its eight dollar price point.


If you are like me, you'll also notice the trailer doesn't show what happens when the player fails to avoid a hazard. That's nothing unusual, of course - marketing material generally shows success rather than failure, to make games look more fun. But because this is a rhythm game, the pacing is vital. If Commander Video (the player character) were to stumble on an obstacle, it would mess up the timing of the level and therefore the song. So the trailer left me rather curious as to how Runner maintains its momentum.

Its solution turns out to be quite simple: hit one obstacle, and Commander Video is immediately sent back to the very beginning of the level. This occurs no matter how much of the level the player has traversed - there are no mid-level checkpoints. Trip over a ledge, and you have to replay the entire level up to that point. It's a bit like if Guitar Hero restarted the song every time the player missed a note.


It's a punishing strategy, and it becomes more punishing the longer the levels get. It's enough to make Bit.Trip Runner qualify as a Bad Good Game. Since any failure to avoid an obstacle means restarting the level, the player's performance must be near-perfect in order to beat a level. In other words, the competence zone is vanishingly small.

Bit.Trip Runner skill curve

By showcasing the frustration that comes with a tiny competence zone, Bit.Trip Runner inadvertently demonstrates how important the competence zone really is - it's the only part of the skill curve in which flow can occur.

I've mentioned flow before - it's a fascinating, broadly useful concept. In games, flow is what happens when the challenge level is perfectly tuned to the player's skill and presents clear goals and clear feedback: the player loses themselves in the challenge and tunes out the rest of the world, and applies their skill to just barely manage to prevail. It's an incredibly satisfying feeling and it enhances learning - it's absolutely the best experience a skill-based game can deliver. Therefore, a well-designed skill-based game should provide as much opportunity for the player to achieve flow as it possibly can - which means it should have as large a competence zone as possible. There should be lots of room between passing and perfect.

Let's compare the experience of two players, one playing a song in Guitar Hero and the other playing a level in Bit.Trip Runner. Both start with skill too low to pass, but they practice and get better. They're both still making mistakes, but they are making fewer of them, and less often. Eventually, our Guitar Hero player is good enough to just barely pass the song. Eureka! Flow is achieved. Meanwhile, our Runner player is still failing, still having an experience interrupted by frustrating level restarts.

The Guitar Hero player now formulates a new goal - to score better on the song, perhaps earning four stars, then five. Each goal is just within reach, and flow is maintained as the player becomes increasingly skilled. The Runner player is still working on the same goal of simply passing the level. Flow has not yet been achieved.

Once skilled enough to achieve a nearly perfect performance, the Runner player finally achieves flow and beats the level. From this point, it's a fairly short journey for both players to become good enough to play flawlessly, after which flow is no longer possible because the challenge level cannot match player skill.

Guitar Hero and Bit.Trip Runner skill curves

The Guitar Hero player had much, much more opportunity for flow, due to the much larger competence zone. The Runner player had only a small window - assuming they persevered through the long period of frustration and didn't simply quit. By presenting a tiny competence zone, Bit.Trip Runner robs the player of a great deal of potential enjoyment.

So what happens when things are taken in the other direction, and the competence zone is expanded to huge dimensions?

Rock Band 3 might be about to show us.

In addition to its various other enhancements over previous installments (such as having a keyboard controller for piano parts), Rock Band 3 features a new "Pro" mode, which is actually designed to bridge the gap between playing fake plastic instruments and the real thing.

"Drums are fairly straightforward -- strap the three Mad Catz cymbals to your existing Rock Band 2 (or The Beatles: Rock Band) kit and you're good to go. . . . Pro Keyboards are a bit more interesting, taking advantage of the entire two-octave keyboard and having the player actually tapping out real notes. . . . But the real news here is the game's support for a Pro Guitar, a new peripheral distributed by Mad Catz. Based on the Fender Mustang, the guitar features 17 frets and six strings, for a total of -- are you sitting down? -- 102 buttons on the fretboard. The guitar also features six 'strings' for picking out and strumming notes.

In case you don't see where this is going -- like the Pro Keyboard, the guitar charts in this mode will actually have you playing the notes in the song. . . . [A]s a guitarist, it was easy to see that all of the notes were in the right places, from the power chords to the individual notes in the song's blistering solo.

Harmonix understands that these most difficult modes won't be for everyone, but it's an interesting bridge between simply playing a game and holding a real guitar in your hands. Starting at the easiest level, it's likely that most players (especially those who've played Rock Band guitar on expert) will feel at home. Before they know it, the idea is to get them more comfortable with a real six-string axe.

'Our perspective on this is to not necessarily to turn people into virtuoso guitar players,' Sussman says. 'I think the line that we use in the studio is that we want to get you to the point where you can beat out a song at a campfire, and not necessarily jump on stage. But what we've seen in our focus testing is that after an hour of, kind of, trainer development and a couple of songs, people actually come out of that one hour and they have basic guitar chops.'"
Nick Chester, Pre-E3: Hands-on with the 'disruptive' Rock Band 3


It's too early to say yet whether Rock Band 3 will succeed in its goal. But if it does, Harmonix will accomplish something amazing. They will solve the problem that it takes a long time and a lot of practice before learning an instrument becomes rewarding. They will connect the competence zones of the game and the real instruments. Learning will be a smooth, satisfying ride all the way up.

Rock Band 3 Normal Mode, Rock Band 3 Pro Mode, and Real Instrument skill curves, with an effectively continuous competence zone

I've mainly discussed the competence zone in relation to rhythm games. But it applies to any skill-based endeavor. The size of the competence zone is the answer to the question, "How badly and how many times can I screw up before I have failed?" Easy to overlook, but deeply important to enjoyment and learning, it's a significant facet of challenge.

17 comments:

  1. Hey Doc,

    I agree with your assessment about Runner. It is incredibly frustrating to hit something near the end of a stage and completely lose your flow. The levels are, thankfully, shorter than ones from the earlier games; but the pre-boss stages are still really easy to mess up in.

    ReplyDelete
  2. Your article is actually quite thought provoking. Good stuff.

    ReplyDelete
  3. I don't entirely agree with the way you assigned a competency zone to Bit Runner. I never played the game but from the trailer it looks like there are a bunch of coins (and probably other stuff to collect) that you need to get a perfect score.

    I just think that the competency zone could be bigger. It doesn't look like you need any coins to pass, just as long as you dont hit anything. And getting all the coins/collectables probably takes a crap ton of "player skill."

    ReplyDelete
  4. I disagree. Your lines seem to imply that once you become 'passable' at a hard game, it's actually very easy to suddenly excel at it. Surely it's the inverse of the graph you draw?

    ReplyDelete
  5. I have a slight disagreement with your last graph: There should be some overlap between the highest score of the lowest skill, and the lowest score of the middle skill. I don't think a player will play the easy score until he's gotten the absolutely highest score in easy, to nearly lose in medium, but expect to have a decent chance to win.

    @FinalSin: You're confusing "amount of skill" with "time to obtain skill". Once you're close to perfect, you have a high skill, and *raising your skill* is harder.

    @jxj: Same as FinalSin. The passing level requires a high amount of skill; the perfect score requires an even higher amount of skill, which is hard to achieve.

    ReplyDelete
  6. Interesting. Another type of game that can be considered in the context of this plot is fighting games. I'd imagine the graph for (particularly old-school, button combination heavy) fighting games might resemble more of an inverse bell than a (more or less) constant slope. That is, in some games (think Street Fighter 2, Mortal Kombat, and Soul Calibur), one can start the game with very little skill (iow, a "button masher") and be surprisingly successful at the game.

    As most players start to become more skillful, they begin to learn the button combinations, but may actually fall lower in performance level because the difficulty of learning so many button combinations and strategies tends to produce a very simple play style consisting of the few moves and / or combos the player knows, which can actually be less successful than the "random" strategy employed by the button masher.

    After more time the player tends to learn a wider variety of combos and strategies and their performance tends to climb back to (and, of course, above) the performance of the button masher.

    The interesting part of this type of game is considering where the "competence" zone(s) may fall, and how this could affect the player's satisfaction.

    ReplyDelete
  7. Your graphs are a bit shonky. For a start you should have numbers on the two axis to show that you are talking about percentages of how skillfull it is possible to be in the game so at 100% skill you get 100% score. Then what are these 'knees' in your graphs all about? You don't address why a game that fails you instantly for one mistake has a curving (!) line up to the start of the competency zone. Would it not be a flat line along the bottom or are you talking about a score rather than just finishing a level?

    If you are talking about score then you need to explain the scoring systems and things like multipliers come into effect, if you are talking about 'distance' through a level then the curves should be straight lines and if you are talking about completing a level then the area before the competency zone should be a flat line.

    TL;DR broken graphs.

    ReplyDelete
  8. @liamross: Not necessarily. After a certain amount of skill, more skill yields no gain at all; it just makes the game take less effort.

    Second: The graphs are abstract. In the precise example proposed by the article, you can think of the first half of the graph as "I reached x% of the end of the song" and the rest as "I reached Y score".

    TL;DR: Learn to read graphs.

    ReplyDelete
  9. The graphs would be easier to read if performance was the horizontal axis and skill level was vertical. Skill level is more of a 'quantity' that is built up, while performance is usually time or progress related (holding out for longer in Tetris, or progressing further through a song).

    ReplyDelete
  10. @Anonymous, in regards to fighting games
    To expand upon that, the upper levels of some of these games take an (in my opinion) frightening dedication to detail. You can find details on the number of frames of every move of every character for some games in the Tekken series (Google Tekken Frame Data).

    ReplyDelete
  11. Ahh, graphs pulled out of your a**. You really need measurable results before you can support any arguments based on comparing "graphs", otherwise they are just subjective sketches. Please preface the article with a note saying all graphs shown are not measurable but are an estimation based on ... (whatever experience is applicable).

    ReplyDelete
  12. The graphs are more intended to be conceptual than precise, folks..

    ReplyDelete
  13. The thing about Runner is that the majority of stages are actually really short, and they usually do a pretty good job of introducing new concepts gradually and letting you work on each aspect individually before bringing them all together.

    The trouble is, when they do bring them all together, in each of the pre-boss stages, the sheer length alone becomes a major obstacle. That and the boss stages themselves end up requiring rote memorization to get through, something that isn't really a problem on the smaller stages.

    It's painful, because these two stages per zone (six in total) drag down the whole rest of the game, I had no real trouble getting perfect scores on each of the smaller stages for the first two zones (if you haven't seen or had it spoiled already, the reward for collecting all the gold bars is totally worth it) and the second pre-boss stage just beat me down for literally hours, to the point where I don't really want to play anymore.

    What I really want is a proper version of Canabalt for WiiWare, the flash version can scarcely run on my ageing mac.

    ReplyDelete
  14. A great explanation and analysis as usual. Although I have to disagree with some of this - you won't be experiencing true 'flow' throughout the competence zone necessarily. Once you've got enough skill to get to the passing point, you may or may not be experiencing flow in getting your skill to the "perfect" ability. This could be dependant on the player, but it's very unlikely and I feel this is very likely to be linked to the game's (internal or potentially external - eg. an outside tournament) reward structure (& as you state goals) - most of which tend to be quite weak at the top end - the rewards, especially in gated games (eg. progression through levels) are usually essentially loaded around the pass point, where you'd be having the most flow-type fun ANYWAY.

    ReplyDelete
  15. Also.. for those interested in fighting games, I believe they are a lot more involved than this with regards to flow and even competency; because they are designed to be played against other people. (Although if you are playing them one player, then all the above article and comments apply).

    I think the flow in competitive fighting games can be broken into two main aspects.

    One is the "execution flow" which fits the concepts applied above:

    http://agoners.wordpress.com/2009/09/30/challenge-me-angel/

    and the other one is what I've dubbed "versus flow" which is related to the level of competition playing against another player gives you:

    http://agoners.wordpress.com/2009/10/09/versus-flow/

    ReplyDelete
  16. Interesting that the Y axis is the game's rating of your performance and the X axis is your actual skill-level (which presumably is pretty much the same thing as your actual performance). If you moved the pass-bar for the less forgiving game near the top, the lines would look about the same (and the X and Y axes would be the same thing since the game's performance rating scale would mooshed up at the top if it were unforgiving, so you could use X for time or something and show how hard it was to improve your skill level in the game), but then you wouldn't get to see the game's distorted evaluation of performance (the curvy line on the unforgiving game).

    ReplyDelete
  17. Bravo. I have never seen such in-depth and well thought out analysis of the difficulty levels of video games, but I think this is on the money. I work for a game design firm, and we pride ourselves on putting the same amount of analysis into all of the work that we do. Personally, I think this is somewhat rare in a game dev firm.

    ReplyDelete