Tutorial: Organizing Your Own Playtest

Overview: The Importance of Playtests in Game Development

Organizing your own playtest is a daunting task, but it can provide valuable insights to guide product development and make decisions that will impact a game’s success with target audiences. However, it is often an underutilized development strategy due to budget constraints faced by some game developers.

In scenarios where developers juggle a limited budget while being aware of the value of Focus Groups, they may enlist family and friends, offering pizza and soft drinks in exchange for their role as testers and feedback providers on their experience. Depending on the relevance and feasibility of the collected feedback, development decisions can be judiciously made before the game is released.

While involving acquaintances in playtests is a commendable approach in budget-restricted or non-existent situations for this strategic development aspect, the added value of playtests designed and conducted by a professional team remains unparalleled. Regardless of the chosen approach for game Focus Groups – involving acquaintances or professional services – the process must be driven by specific, clearly identified objectives to maximize the quality and relevance of the gathered information.

In this article, for a bit of pizza and soft drinks, you will gain insights from a team of professionals experienced in playtests concerning design, preparation, and planning. This will help you confirm the effectiveness of your current approach and/or add elements here and there.

Understanding Your Players/Users: Creating User Profiles

If you haven’t already created player profiles, you will want to do so beforehand to ensure you solicit relevant profiles in your trial group. For information on this topic, Games User Research has published an excellent article on how to create user profiles. The following lines summarize the essential content of this article and also address trial planning over time.

Understanding Your Players: Key Behaviors to Know

When thinking about player profiles, we often consider general factors about the player that could provide information to marketing specialists to promote the game. However, in a game trial approach, it is the specific behaviors that players will adopt during the game that need to be identified to choose them based on your product. This is an important nuance. Here are some useful questions in this regard:

How do players behave in the type of game you are creating?
What is the perceived strength of your game in the user experience? Specific and innovative mechanics? A rich story?
Which features or mechanics are popular and appreciated in this type of game? Could your target players expect to find these features/mechanics in yours?
What is not appreciated by target players for this type of game?
What attracts target players to a new game of this genre?
What are the playing habits of target players? Short sessions? Prolonged sessions over an extended period?
What motivates target players to buy a game of this genre? Do you have examples of reasons that have prompted target players to buy other games of the same genre?

The answers to these questions will help you define clear and specific objectives to determine what elements your targeted playtest will focus on, based on the success criteria sought by your users!

Setting Clear Objectives for Playtests

With player profiles created and a good idea of the elements that players will appreciate, you move on to the next step: formulating the objectives of the playtest.

Firstly, an effective way to identify the objective or objectives of a trial begins with the following question: “What do I want to achieve with this playtest? What are the trial’s desired results and how will they serve me?”. For example, are you looking to measure user retention, overall replayability, immersion, or ease of integration? State objectives accompanied by key performance indicators (KPIs) and do not deviate from them unless necessary.

Secondly, take a moment with your development team (or trusted pizza/soft drink advisors) to identify potential objections to certain features and especially to the key mechanics of your game. Consider the return on investment for these objections by anticipating the costs of reorienting development if many objections arise during your trial.

While some features may be excellent to keep/incorporate into your game, they may also prove too costly to adjust/fix. Save yourself a headache and organize how you will prioritize recommendations from your player base by defining priorities on the types of recommendations that could result from playtests. By translating recommendations into prioritized actions, you will use your development budget wisely.

The step following the identification of precisely stated and measurable objectives is formulating hypotheses about the appreciation of your key elements and adjustments related to them.

Formulating Hypotheses about Key Adjustments

Hypotheses about your adjustment points must be formulated in an “educated” manner well before the playtest (the earlier in your development cycle, the better). By “educated,” we mean understanding the operational implications of redirecting your resources, as well as an approximation of the perceived importance of the redirection by your players. With all the work you have just done on your player profiles and the objectives of your trial, your mind should already be ready to follow some trains of thought. Ask yourself questions about:

The more innovative and central points of your game and how they compare to games of the same genre
The elements of your game that will likely be well-received by your player base (and vice versa)

Starting from the objectives aimed at the playtest, visualize the behaviors that participants might adopt during the trial session to anticipate certain points of friction in their appreciation or understanding of the elements subject to your hypotheses. Do they learn them without generating too much confusion in the playing habits of your target audience? Does your environment seem authentic enough without losing its individualism to maintain the interest of your players?

You will need to imagine how your players will react by asking questions to help predict their satisfaction and engagement. Then, it will be time to create surveys to measure and qualify their appreciation of your key elements.

Surveys are a topic in themselves. Here is a link to an article by Games User Research that provides excellent guidelines for survey design.

Planning the Playtest: A Low-Cost or No-Cost Approach

Now that you have the elements to measure for organizing a trial, you are ready for the planning stage! In the example presented in the following lines, we will assume that you have a “low to nonexistent” budget.

Pre-Playtest Planning: 2 to 4 Weeks Before the Event

If not already done, two to four weeks before the “D-day,” you will work on the analysis and documentation of your game, the creation of user profiles, the precise statement of measurable objectives, and the formulation of hypotheses about the appreciation of your key elements.

Very importantly, you will also want to explore whether you will use a playtesting platform! While we have one here at HUWIZ, Testify is an excellent alternative that we have heard a lot of good things about.

Playtest Day: Data Collection and Observation

On the day of the playtest, everything revolves around data collection and observation of participant behaviors. This is where your agenda and the surveys you created come into play. What we recommend is to let your participants try your game and only interact with them to help solve certain problems or provide answers if something is unclear. You want them focused and uninterrupted during the playtest session!

Make sure to carefully observe what they do, what they say, and their non-verbal cues that indicate what they are experiencing while playing, keeping your hypotheses about the appreciation of your key elements in mind. New hypotheses will inevitably emerge from these observations. Be sure to document these new hypotheses with your observations and validate/invalidate them in the planned individual and group discussions after the game session.

Hold individual and group discussions only after everyone has finished their game session to maximize information gathering on your hypotheses. For the various hypotheses you will measure throughout your playtest, plan multiple-choice questions (for the survey) and open-ended questions to facilitate individual and group feedback meetings.

While open-ended questions give you information about participants’ vision/appreciation, closed or multiple-choice questions confirm trends with yes/no, how much, when… Qualitative feedback questions – open-ended questions – are generally asked during individual and group meetings, while closed or multiple-choice questions are used in survey questionnaires to be filled out.

Playtest Data Analysis: Ideas and Recommendations

At the end of a playtest including a survey and interviews, you will have collected a lot of information about your game and what you wanted to measure. With all this data, you will be able to confirm whether your game will be well-received within your target audience based on the playtest participant sample and begin to associate them with different priorities related to your return on investment that you had envisioned before organizing the session.

Were your hypotheses about the appreciation of your key elements correct? Were you a bit off the mark… and in what way? Develop a report based on the analysis of the data you collected during the playtest. In pursuit of the return on investment you are aiming for, determine participant recommendations that would be relevant to activate while prioritizing them according to your budget.

Next Playtest: Decide on Additional Tests

Now that you have some results, you can decide whether to plan another playtest based on your confidence in the data you have collected. Nothing prevents you from revisiting certain topics during your development to solidify your confidence if your budget or timeline allows!

Conclusion: Harnessing Playtests for Informed Game Development

Playtests prove to be an excellent strategy to guide the development of a game towards the expectations of a target audience, to judiciously invest its budget in development, to prioritize actions that maximize return on investment, and to make a mark in an industry teeming with products. If your budget allows, using the services of a quality assurance company that offers playtest design, direction, and interpretation is the optimal option. Otherwise, a “homegrown playtest” that includes members of your circle can also provide valuable insights, provided that you have identified measurable objectives, have a player profile, stated your hypotheses, and prepared your questions. After all, many of us passionate about video games are capable of providing feedback on a product! Anyone on your team and their uncle might have an idea that can make all the difference in how your game leaves its mark.

There is so much more to say about the development strategy incorporating playtests, but by incorporating the steps and ingredients proposed in this article, you are already well on your way to organizing a playtest that will provide valuable information.