I became a serious customizable card game playtester with the Babylon 5 CCG.  I don’t have a lot of recollections of playtesting the Shadows Expansion, but two of the game’s designers lived in the area, and we may have had some involvement.  The first big playtest job was Great War.

But, I’m not writing this to talk about the details of B5 playtesting.  Rather, there are things I learned as I playtested other card games about the development process and playtesting in particular with regards to card development.

A lot of playtesters I ran across back in the day were just people who wanted to see cards before everyone else.  Another group were those who wanted to express opinions based on existing notions.  What I discovered about useful playtesting was that it was a job.  Fortunately for Precedence Publishing, I had lots of free time back then and had a local group of people to playtest who made time.

Where does the time go?

Well, there is playing, but a lot of CCG players want to play CCGs, so however long this takes it doesn’t feel like much of a time sink.  However, good playtesting involves playing games that aren’t going to be all that fun/interesting.  A huge problem with niche CCG playtesting is sample size and regional metagame differences.  Playing one game with your “Turn one, fully upgrade my Subsidized Merchant” deck isn’t going to provide much in the way of useful data points to see if card changes should be made.

Thinking about decks.  This can vary as much as a playtester wants it to.  A simple starting point for playtesting is to take a known deck and modify it.  While that’s much harder with a CCG’s first set, sometimes the starter decks for a first set have largely been laid out and you have some sense what a deck is expected to look like.  Of course, the most innovative deck construction is when you “break the rules” and build something completely different from typical decks.

Both of those activities are activities assumed to be of interest to CCG players.  Here’s an activity that will make you feel like you are doing a job:  printing cards, cutting out card slips, sleeving decks.  I did so much of this in the ‘90s that it’s immediately a groan when I realize I need to do it, again.  Someday we will have discs that we hold that are programmed and can be changed on the fly, though already we have electronic card games where you could do something similar (except programming time is arguably more work than dealing with white slips of paper).

Multiple scissors or scissors plus papercutter.  Make sure to not cut through the text on the .pdfs/Word files/whatever.  What cards are lying around to use to back?  Well, with an existing game, can use cards for that game.  For Traveller playtesting, we used Magic, B5, Wheel of Time, and I think World of Warcraft and possibly Game of Thrones.

Good practice – playtesting by actually playing the cards.  You put the effort above into playtesting so that you aren’t just throwing out opinions that may be good but often are incomplete and possibly even counterproductive.  I still think about how I thought a Vampire: The Eternal Struggle Gargoyle card was costed okay in the Bloodlines Expansion when more actual play would have made me realize it was ridiculously overcosted and more than a decade later a card was made specifically to make that card and its ilk less costly.

So, you arrange playtest sessions – something that can be difficult as some of your play group may just want to play and not playtest.  You print out pages of cards, decide which ones to put into decks, cut those up, sleeve them, and you have playtest decks.

Good practice – have benchmarks.  There is value in playing playtest cards not just for power level but just for familiarity.  Is the wording clear?  Are there card combos you haven’t thought of?  Etc.  However, the first great fear with development is producing overpowered cards.  That is maybe not what should be the greatest fear, but, whether it’s the first fear or the second fear, the other great fear is producing cards nobody plays because they are underpowered and not interesting in another way.  That’s important.  CCGs are about variety and some cards will always be better at a given point in time than other cards.  But, … let me give an example.

Our Wheel of Time CCG playtest group was either the only or one of the few playtest groups for the first full set of WoT.  Other playtest groups had seen the starter decks for months, but there was a last minute playtest session with one of the designers to playtest everything else.  You may well imagine that for a 300+ card set that four people in one day did not completely playtest the set before it was “finalized” (card publishers routinely change cards after playtesters are done with them, in some cases in horrendous ways that frustrate the playtesters when they see the published product).

Getting a bit off track, though this example is relevant for another comment.  Benchmarks.  Playing all playtest decks against each other can give a warped idea of power levels.  Again sample size comes up, but the most correct thing to do is to go through a nigh infinite number of games where you play playtest cards against existing decks and tweak and tweak and tweak both playtest decks and the baseline decks (because the metagame will shift with new cards and the same decks shouldn’t remain just as effective).

So, I got off on a tangent there with cards being interesting.  That WoT playtest session is most memorable for me because it brought home how much players will gravitate towards the best cards or certain cards that appeal to them (major characters in an IP, for instance) and not get around to a lot of the other cards.  The first WoT set had some crazy broken cards, at least some of which we had serious concerns about, but it also had many terribly underpowered cards because we just didn’t have the time or the energy to push to get them improved.  Ever since then, I’ve had in mind during any playtest that it’s important to encourage players to look for cards that seem too weak or too boring.  Too weak is fine as long as players can enjoy the thematics or the mechanical quirkiness of a weak card, but I don’t want to see cards that don’t get played by anyone get made.

Good practice – adjust.  So often, given that there’s a limited window for playtesting (and there should be to force playtesters to get things done) and playtesters have limited time and energy, playtesting doesn’t get as deep as it should be.  I routinely focused on specific cards in playtesting to the exclusion of a lot of other cards later on, when we would divide up who worked on what to avoid having everyone gravitate towards the same cards.  But, even so, it’s typical for playtesters and the developers in charge not to go through enough permutations to make better cards.  I’m credited with designing B5’s Gerontocracy because I playtested it far more than anyone else (and playtested what was a pretty boring card at all) and proposed a far stronger card than what came out initially.  Gerontocracy is hardly a power card in B5, but it’s not unplayable.  Not many cards get the treatment of not only playing and saying “this card doesn’t do anything” but keep adjusting the mechanics until they are reasonable.

If the good practices are to treat playtesting as a job, actually playing with the cards rather than just sending emails with your thoughts from reading them, setting benchmarks for effectiveness, paying attention to the less interesting cards, and adjusting decks and games to go through a variety of possibilities, bad practices are often the opposite.

It’s extremely common to read a card and immediately start complaining.  Now, for wording issues, whether lack of clarity or poor templating or whatever, that’s great.  But, sometimes, you have to prove that a card is under/overpowered.

It’s extremely common to draw conclusions from small sample sizes and/or unusual metagames.  Yes, people have limited time/energy.  And, yes, as I mention above, sometimes you just want to get a sense of what the cards are doing before unleashing your tournament decks on playtest decks.  The other situation that can occur is when playtest groups choose not to play certain things because “they aren’t fun”.  When it comes to a game with a tournament component, can’t control for what some players dislike, have to bust out the cheese, the griefing, and so forth.

I think you get the point.

So, I’ve pretty much just talked about what development is like for the playtesters.  What about the developers who farm out playtesting to volunteers?  Where do they go wrong?

I already mentioned how cards will change after playtesters are done with them.  This can be really obnoxious even when it doesn’t lead to broken cards.  Obviously, the greater problem is leading to grossly overpowered cards or nerfing some card so hard that it only gets played as a joke.

Being slow to respond.  Playtesters can easily become idle if their comments aren’t acted upon or there aren’t new cards to test out.

I’m going kind of backwards here – not being organized to begin with.  It’s good when there’s a playtest coordinator who sets an agenda, lays out best practices, and communicates regularly with external playtesters.

Being stubborn.  Everyone will fixated on a particular opinion.  A lot of the time, that’s not a problem.  But, sometimes, you get a situation where the people in charge of publishing cards don’t want to listen or ever change their minds and it doesn’t end up well.  The Wheel of Time CCG had a promo card that we worked on.  The game manager wanted it to be a card draw or card searching effect.  The only thing more broken in WoT than card drawing is card searching or maybe it’s the other way around.  I proposed ideas in the draw/search milieu because that’s what the boss wanted.  Eventually ended up being one of, maybe the most, powerful card drawing cards (after some stuff from earlier sets got nerfed) in the game.  As someone who learned to be concerned about promo cards with B5, this did not make me happy.  As a player, of course, I played max copies of the card a lot.

Playtesters are people.  Some developers are people, too.  Mistakes happen.  On rare occasions, getting things right also happens.  I can see at least a few cards in Traveller I would change now, but I hope this article not only gives a bit more insight into my views on how development goes but also a bit more insight into how it may go for other games.