The Gaming Industry is Clueless about what Alpha and Beta Mean

[Icemania]

Gal Civ's so called Beta 1, which had a massive array of missing features, prompted some debate on what Alpha and Beta meant. Das has a good video on this subject below.

https://www.youtube.com/watch?v=NSwO_E5BLpA

Stardock have their own way of looking at it and unlike others they have a good track record so there is nothing to worry about. That said, given the number of games that are released in a Beta state in gaming generally, they could manage community expectations more easily.

The gaming industry is all over the place with how these terms are used. But how do you manage expectations with a smorgasboard?

Consider Scotty from the Enterprise. He always had extra warp speed behind the scenes. He had it right, underpromise and overdeliver.

With relatively minor variation I've seen the conventional definition used a lot in software development. I'm normally (not always) on the business owner side of the software development fence. A common characteristic in the best software development PM's is that they always understate the status. Why? They know good business owners can smell a rat from a mile away ... just like an experienced gamer.

The best strategy I've ever seen was calling software at conventional Beta an Alpha, calling software ready for release a Beta. That PM was considered a magician, just like Scotty. If you have enough time/budget to do it that way, it does wonders for perceptions ... and future business. Tough for indies, but eminently feasible for Stardock.

It's not just about the semantics of Alpha and Beta definitions.

[Nasarog]

Alpha is early access
Beta if feature complete and now you are running down the bugs and optimizing
Gold is the game being released.

If the game has all of it's features intact and nothing new is being added, just streamlined, that's is beta. If you are releasing parts of the game and testing compatability and gameplay, that's alpha.

So, GC3 is in late alpha. When everything is in and runing together, that's beta. In most cases, we are paying the developers to be beta testers. Then we keep paying for features to be added.

That's how I see it.

[Arioch]

Beta is supposed to mean feature complete, but almost nobody in the gaming industry uses that definition. I suppose in a world of continuous online updates, nothing is ever feature complete.

[daveyg2611]

I think there's definitely a marketing strategy in there, too. Given how long it takes for many titles to truly make it to "feature-complete", smart marketing would be to move it to Beta when the majority, but not all, core systems are implemented in some capacity. I don't think Early Access wins a lot of people over, so moving relatively quickly to Beta will probably inject some cash into an ongoing project.

[WabbaCat]

Early access can be at any stage of the game.... I dont think it should be shiped as an alpha status thing?

but alot of people could use to be more aware of what early access is... there tends to be alot of people that buy into early access and subsequently bash it for being incomplete, buggy or otherwise.

[Nasarog]

Wow, thank you.

[Mezmorki]

@ faijeya

Great and insightful post ... thank you for sharing!

One of the best quotes I heard from someone on the web in discussing the question of alpha/beta/release was quite simple, and brilliant. It was this:

A game is released the moment anyone can pay money for it and it play it.

In the era of early access this is worth considering. If a developer is charging money for access and making the game available to anyone immediately, then irrespective of what stage the developers are internally calling it, it is "released" in a sense, and much of the buyer-be-aware type issues that affect any other released game come into play, including early access reviews/previews, user opinions, marketing, etc.

I can certainly understand the appeal from the developer standpoint of doing early access - it makes tremendous sense provided you have something good enough to show for whatever it is you are releasing such that you can maintain some traction with your audience. But does this approach really lead to the best product at the end of the day?

There are enough games that have entered early access at some alpha or beta level, but were then "released" some months later with very few substantive improvements to the game to make me very cautious about the whole thing. As mentioned, the moment you start charging anyone in the public money for your product the whole development equilibrium would seem to shift.

Gosh - people used to be PAID to do beta testing ....

[Nasarog]

Originally posted by :i69:Mezmorki:

Gosh - people used to be PAID to do beta testing ....

YES! I've been feeling and saying this for a long time. The first time I really felt this was with Gal Civ 2. I've said as much on their forums back in the day and caught crap for it.

[Mezmorki]

Originally posted by faijeya:

By proper testing I mean:
- regression testing between builds
- UI and HW diversity
- assessment current build's risk areas
- proper dissection of look'n'feel feedback on subjective and objective (both are important and should not be mixed)
- having a habit to note what was done before encountering a particular bug and making the best possible effort to create steps to reproduce it
- having a habit to follow an opened bug and validate it when devs say it's fixed

Definitely. Good beta testing requires a lot of diligence on the part of the beta tester. It isn't "fun" necessarily to be diligent about debugging, taking notes, etc. And good testers can also help improve a game's fundamental design by having a large pool of experience to draw from and being in closer communication with the developer and what their objectives are.

I wonder to what extent some developers have replaced this older model of testing with the publically open "help up design the game you want!" (but please pay us first) approach. It's got design by committee written all over it - and for every good piece of input you get, there's 100's of ridiculous comments and suggestions from Laz0rBa$$ter65 pouring in.

[WabbaCat]

"if" I understand correctly,...
you still cant ship early access and release/beta/alpha stages together.... its not the same.

Early access is not "buying the game" therefore it doesnt follow with
"A game is released the moment anyone can pay money for it and it play it."

Early access is buying the right to access the development stages of the game with the promise of a copy of the game when its finished. It's specifically worded this way so that people dont confuse this with a finished product in any way(yet they do anyhow).

^is this not correct now?

[WabbaCat]

in other words "Dont buy early access, unless you really trust the developer" just like always? :)

[Mezmorki]

Originally posted by WabbaJack:

"if" I understand correctly,...
you still cant ship early access and release/beta/alpha stages together.... its not the same.

Early access is not "buying the game" therefore it doesnt follow with
"A game is released the moment anyone can pay money for it and it play it."

Early access is buying the right to access the development stages of the game with the promise of a copy of the game when its finished. It's specifically worded this way so that people dont confuse this with a finished product in any way(yet they do anyhow).

^is this not correct now?

Technically no one is "buying the game" anymore at all, we're all licensing the rights to access the game. Sadly. Whether that right is granted during a development stage of alpha/beta or release is a secondary point.

But, technically you are right according to the definitions of early access. The quote was really just stepping back and thinking more broadly about the transaction that occurs when you buy something. If a product is available for sale to anyone who wants to buy it, it's "released" in a certain sense, and completely up to the developer to do what they want as far as future development goes.

Increasingly a product being "released" hardly matters - it's just another point in time of the game's development. And I guess that's what I'm getting at.

Do you know what would help counter-balance the early release trend? Reviewers actually reviewing games in early release and calling them reviews. It’s no different than a reviewer waiting to review a game a few months post-release after a bunch of patching and coming to a different conclusion compared to people that reviewed it right at release. They are all moments in time in a continuous development cycle.

This probably sounds unfair, but developers would think twice about going for a money grab if they knew their half-finished product would be received poorly among critics.

[Mezmorki]

@faijeya


Interesting. As you have worked as a dev and in QA (correct?) - I'm curious what you think about another pet theory of mine.

So I come from a heavy boardgaming background both playing and designing - I even have a 4X boardgame that's been published! One thing in boardgame design is that making sweeping gameplay changes to the core design early in the process is very easy. It takes as long as it takes to imagine the change and scribble up some new components on pieces of cardstock. As a consequence it's much easier to explore with big changes in the core gameplay because you don't have 1000's of lines of code invested in a particular system. However - boardgames are far simpler mechanically then videogames since humans generally have to manage all the details - but the result is that the core gameplay REALLY REALLY matters because that's 95% of the experience.

Now, for videogames, I can't help but get the impression that a lot of newer developers come up with the best laid plan and initial design concept, and then spend a TON of time just implementing the idea and getting it into a complete and playable state just so that they can SEE if the design is even working or is even fun. For most of the smaller studios, they just don't have the budgets and resources to completely rework huge aspects of the gameplay to radically make the game better when they realize things aren't going so well. It just takes too much time to recode huge parts of the software just to experiment with different systems and design approaches.

I don't envy the task before video game developers. But at the same time, I look at a lot of the failed early access games (failed as in the final product was bad or forgettable) and can’t help but think: these guys spent all their time and resources just implementing the first pass of the design. A lot of panned games have gone from early access alpha to beta to release with almost no substantive core gameplay changes – it’s all just getting the thing to work and tweaking values here and there. But no amount of tweaking on that level will turn it into a good game if the core design ideas need a major overhaul.

I’m curious what you think of this!

[Icemania]

Beta Testing Best Practices

Hey All:

I've recently signed up to help Beta Test Distant Worlds for coming patches and also any future games in the series (despite my marketing concerns I love the game).

I've done plenty of testing in software development more generally (although these days it's always a Business Owner type role) but I'm new to the gaming scene.

I only intend to volunteer to Beta Test for other great or potentially great 4X games as I'd prefer to help in a few games rather than be spread too thin. After all time is short with families, work, exercise and forum posting getting in the way of gaming (except when I'm on leave like this last week).

We are all bound by Non-Disclosure Agreements, so I appreciate the need to generalise rather than be specific, but who else here volunteers as a Beta Tester and for which 4X games? What have you found to be Best Practice in the gaming world? The intent is a thread where we can share our experiences.

[Nasarog]

I'm on a MAC currently. We get no love.

[BobbyBullets]

I've got a lot to add here, but not the time just now. I'll add my write up here when I do find the time.