Q&A: MMPORG Alpha Testing

Hey folks 👋

Today’s newsletter is another dive into the world of gaming. If you missed the last one, here’s a link to it. It discussed pitching VCs pre-product.

Over the last couple of years, I’ve realised know-how from this industry can be widely applied to practically any other market with astonishing results. On so many levels.

Others have noticed, too. Up until recently, the gaming industry was seen as kind of an oddball outlier in the broader tech and startup ecosystem. A hits-driven business with distinct inputs and outputs. Not really taken seriously by those outside of it.

Now, smart people have started to *very successfully* apply game design principles and mechanics to traditionally stale products and markets from a UX standpoint.

Well-known examples are Superhuman, Robinhood, and Duolingo.

Bitkraft Ventures calls this “Applied Game Mechanics.”

At it’s core, it’s about creating strong engagement loops.

Why? The key to shipping and supporting a super engaging product is to optimize the experience for human psychology. Not utility.

Gaming companies do this *the best*.

They are light years ahead of most other industries in this regard. Everyone can learn from their software optimized for psychological engagement. Which, I’ll cover more in future posts.

At this point, you may be thinking about the term ‘gamification’.

“Hasn’t gamifying things been around for years?” Yes and no.

The term and surface-level attempts to do it have been around for over a decade, but the *real* application of this into the DNA of non-gaming products… not so much.

For sometime, game design principles and mechanics have been layered *on top* of already existing infrastructure in the name of ‘gamification’. Often, with less than stellar results.

As Bitkraft points out:

• Foursquare’s struggle to retain users after it became impossible for them to progress up leaderboards;

• Zappos’ introduction of customer loyalty badges that customers did not understand how to accumulate or value;

• Disneyland hotels’ “electronic whip”, which demotivated most employees by highlighting failure

Conversely, recent cohorts of startups like Superhuman and RobinHood have built them *natively* into the DNA of the experience since the beginning. That is the vital difference. It’s not an afterthought.

Bitkraft explains this well:

What we have seen so far is the tip of the iceberg.

Expect to see a ton of startups utilizing Applied Game Mechanics over the next decade. Particularly in markets with ‘low-hanging fruit’ in terms of UX.

On that note, I’ll crack on with today’s Q&A.

It’s with Tyler Cloutier, the co-founder and CEO of Clockwork Labs. Inside, he details how he conducted alpha testing for their new MMPORG, BitCraft.

Drawing on the themes I mentioned above, game testing best practices are also highly transferable and under-utilised in other markets. Particularly when it comes to fostering and growing a passionate community.

Let’s jump in.

Tyler Cloutier is a co-founder of Clockwork Labs—an indie studio on a mission to build “a new kind of MMORPG”.

That’s an acronym for Massively Multiplayer Online Role-Playing Game.

Tyler set up the company with his friend, Alessandro Asoni, back in 2019 to pursue a longtime dream of playing a video game that’s “truly open and free”.

What does that mean?

Tyler’s Medium post announcement back in 2019 painted a very compelling vision:

Their vision, team acumen, and concept art were solid enough to convince prominent gaming investors to back them in a pre-product seed round. This included big names such as Skycatcher and Supercell.

How did that come about?

Tyler informs me he fired off a cold email to Ilkka Paananen, CEO of Supercell, discussing his support of the Brawl Stars and Clash of Clans producer’s famous decentralized company culture. He responded, and, from there, Tyler was connected with Supercell’s investment team.

Here’s the concept art they used to pitch their BitCraft vision. 👇

Tyler says they “take inspiration from the ultimate sandbox world: the real world.” 👇

It certainly piqued my interest. I put my name down for their TestFlight alpha straight away.

A few months later, I received an invitation to download the game. I was immediately impressed by what they had put together in a relatively short period of time. Plus, an engaged community was brewing on their discord server.

It’s certainly an ambitious undertaking. Building the game is a big project in itself, but doing so in a manner harmonious to the experience of alpha testers must be the really tricky bit, I thought.

After all, it’s going to take a lot of time before it’s ready for the masses. Years.

Eight months after their first alpha concluded, Alpha2 launched last summer with some strikingly good new game art and visual style. 👇

Other updates included:

• Much faster loading times

• Terrain biomes and mountain ranges

• Town walls

• Town chat

• Player created quests

• A much larger world (4x bigger than Alpha1)

A year later, Tyler and Alessandro were still working hard on alpha testing. Now, they’ve raised a $4.3m Series A round to keep testing and building.

As you’ll read below, today the game mechanics have evolved into more a Runescape-type experience in “which players build the world themselves in a procedurally generated landscape.”

This got me curious.

What’s it like conducting a series of alpha tests for a complex game over a long period of time? What are the best practices in terms of balancing product progression with community management? Especially if the product needs to change materially from the initial pitch. What’s the latest thinking from the frontlines on this?

I sat down with Tyler and picked his brains. 👇

🔥 Q&A

Why did you found Clockwork Labs? And, why BitCraft as the first game?

Alessandro and I originally had planned to make a tool for research grad students to store the data and lab notebooks in.

At the time the company was called SkyLab. It's actually still an active site (https://skylab.io) which is used by grad students today.

However, it turns out grad students aren't the most lucrative target demographic!

Towards the end of SkyLab, I spoke with Alessandro about a game idea that I had been kicking around for a while (BitCraft). He thought it was interesting, so we thought we'd give it a go.

The motivation for BitCraft is to pick up where games like Runescape left off.

Tell me a bit more about yourself and your cofounder, Alessandro Asoni. What have you worked on before, and what do you work on now at Clockwork?

Alessandro and I are distributed systems engineers by background. We've both been working as software engineers in industry for >10 years.

Prior to SkyLab I worked at MachineZone and Alessandro worked at Bloomberg LP in New York. Alessandro is spending a lot of his time on the game engineering, and I'm spending most of my time on the game design, art direction, and administration, while working on our game's engine on the weekends.

Where is BitCraft today? Talk me through your alpha tests and milestones.

Thus far we haven't released much publically about our game, but what I can say is that we've had a playable version of the game since November 24th 2019.

Of course, at that time the world of BitCraft was populated with quite a few gray boxes and looked quite a bit different than it does today!

During that time we've given access to a small number of pre-alpha testers so that we know that the core game loop is progressing well.

We're planning on announcing the first public version of this game this summer so definitely stay tuned for that! Super excited to show where the game has gone.

How has the game evolved from a player perspective over this time?

The concept of the game has evolved pretty dramatically since the original idea, and the mechanics of the game have changed quite a bit since we got it into the hands of players.

The core vision largely remains unchanged though. The original idea for the game was closer to something like a Stardew Valley or even HayDay MMO where everyone has their own plot of land in a shared world.

These days the mechanics of the game more closely resemble a Runescape-like game in which players build the world themselves in a procedurally generated landscape.

What technology have you used to build the game? How many people have been involved in bringing the alpha to market?

The game client is built using the Unity engine. Our game server is built using our proprietary server-side game engine called the Clockwork Engine.

We are hopeful that one day we can make this available to other developers, although we don't have any current plans for this.

Currently, we have 13 full-time + part-time people working on the game!

Building a proprietary Clockwork Engine and opening it up to other devs sounds amazing. What made you choose that path?

We just knew we would need to develop our own server-side system to be able to support the kind of gameplay we wanted to have in the game.

We explored using SpatialOS, but we determined it wasn't a great fit for us. As we developed it became clear that what we were building for ourselves might one day be more widely applicable.

What has been your overall approach and strategy to conducting the alpha tests? i.e. copying proven frameworks, keeping it organic, etc.

Our general approach is: test early and often!

Even our best-laid plans don't always survive contact with the players, so as far as we can tell the best thing to do is get it in front of them as soon as possible.

Beyond that we try to make sure that our core mechanics are mechanics that have worked in other games, even if the complete game is totally new.

Do you have any unique insights from alpha testing that would be helpful for other founders to utilize?

Same as above: test as soon as possible, even if it's janky.

It might even change the entire way you're looking at the problem and give you the maximum amount of time to pivot if need be. I don't think this means you need to *release* your game early, but having a handful of testers goes a long way.

What 'best practices' learned at Machine Zone have helped you at Clockwork?

I learned a huge amount at Machine Zone. In several cases I learned what not to do from seeing Machine Zone's struggle with certain things, and that has given us some power of hindsight we otherwise would not have had.

I think one thing that Machine Zone did exceptionally well was realizing that shipping fast is hugely important, and that it matters much more to have a janky system that many people are using, than a beautiful system that is never finished.

What team hires did you prioritize making, and why?

We've prioritized game design, art, and engineering. Game design because if the game doesn't have a fun loop, we're wasting our time. Art because our game needs to turn some heads. Engineering because we need to be able to deliver on the vision.

How have you attracted alpha testers?

So far, we haven't done much!

We've had a small number of articles written about us, so some players have just come to us. In the coming months as we plan to go public with the next version of the game, we'll be getting more alpha testers.

What have you learned from alpha testing from a product standpoint?

This would probably get too deep in the mechanics of the game to go into here. I'll give some general learnings though. Grindy gameplay works to increase retention of hardcore users but isn't super fun. Some of the more complex things we added ended up not mattering. UX is important.

What are examples of ‘grindy’ gameplay and ‘complex things’ in this scenario?

In terms of grindy gameplay, in BitCraft that essentially means leveling. So for example, cutting down a thousand trees before you have the woodcutting skill to cut down a high-level tree.

One example of a complex system we built which ended up not mattering was a complex land ownership system where you could buy and sell land tiles. In the end, a more simple system worked a lot better with actual players.

What have you learned from alpha testing from a community standpoint?

I think knowing when to build hype for your game and growing your community is important. You don't want to have a huge amount of attention when you're building things out. I think this comes down to having the right balance of public and private info.

What has been the most challenging aspect of alpha testing, and how have you gone about tackling it?

I think one of the more challenging aspects has been to communicate with our testers about the changes that we're making and to set expectations appropriately.

So far we've tackled it by offering a weekly dev blog, but I think it would be good to have some resources they can reference about the game design choices we're making.

At this early stage, what methodology are you using to develop the game? KPI-driven, qualitative, something else?

In the very early stages, I think the purpose of testing is to answer questions, so in that way it definitely starts out qualitative.

Whenever we release an iteration of the game we have a hypothesis we want to test.

How is the core game loop? Do players interact with this content? Is the content balanced, etc. After that we look at retention. That's our main focus because it's the best indicator that players like our game.

Put another way, you have a grand vision for BitCraft. How do you balance working towards that vision versus player feedback and behavior that may contradict it?

Similar answer here to the above. I think it's important to have a vision for what you want to build, but any vision has a set of hypotheses that need to be tested and iterated on. It's possible the hypothesis for the entire vision is wrong, but more likely it will evolve as you test mechanics. The way we process player feedback is to always listen to their problems, but rarely directly implement the solutions they offer. Most often we have a higher-level understanding of the vision to see how we can address their issues another way.If we find that behavior is completely counter to what we intended we try to roll with it. We haven't found behavior that contradicts our vision, but if we did and we understood it, I don't see a problem with updating your vision.

Since you're in alpha testing, how do you define and map out progress?

We generally follow a four step cycle of progress: Overall goal → Milestones → Game Proposals → TicketsAfter each release (milestone) of the game to our testers, we reevaluate whether we're on the right track, and then we rinse and repeat.

You have a major update coming up. What's next? :)

It's for sure the most exciting update for the game yet!

This is the first time we plan to integrate everything together into a very tight game loop. We're introducing survival mechanics, combat, better onboarding, redesigned UI, biomes, clothing and a visual overhaul to boot.

We also have been developing a trailer so that's very exciting!

And, here is the trailer!

—

Thanks, Tyler! An amazing journey so far.

If you’d like to learn more, check out the links below

Until next time,

Martin 👋

More like this:

ℹ️ BitCraft website.

📰 Announcing BitCraft.

📰 BitCraft Alpha2 Playtest.