Third Party Embedded Testing: An Emerging Practice

The development of video games is an area where quality assurance (QA) plays a crucial role in the quality of the final products. With the often changing standards that are linked to the constant evolution of market trends and associated technologies, companies are seeking effective approaches to optimize the quality of their games (while remaining faithful to their planned schedules). Integrating external testing teams from the early stages of development proves to be a strategic practice to consider, and we’ve been hearing this more and more from our partners and industry friends. This “embedded testing teams” strategy, which we’ll explain here, aims to maximize the quality of the final product while optimizing development processes by involving external teams where some developers would have otherwise recruited additional members to their teams.

In this article, we will examine the benefits and operational implications of “embedding” external testing teams into the video game development process as early as possible. Through embedded testing, these tests involve external teams becoming familiar with their partners’ operations/internal development tools and integrating them punctually or continuously into all phases of game production as you would a regular team member.

We will highlight the key changes of embedded testing compared to “traditional” methods of outsourced testing. We will examine how this transition shapes the role of external quality assurance (QA) teams, as well as the responsibilities of analysts and project management. Finally, we will illustrate the tangible implications of embedded testing in terms of efficiency and overall quality of the final product.

What is Embedded Testing?

Embedded testing teams represent a shift in how external QA is commonly approached. In the traditional approach (which remains a viable outsourcing option), the focus is mainly on quality control performed later in the development cycles, where testers execute test cases to identify defects in the elements they are checking. They rely on their managers to then communicate their results to stakeholders.

Embedded QA, on the other hand, generally cuts out the middle man for communications, and encompasses proactive measures of communications. Rather than functioning as an external entity, each analyst becomes an integral part of the QA team and is responsible for specific features or components and communicating any obstacles or unforeseen cases. This configuration advocates for a more “in-depth” involvement throughout the development process.

In embedded testing, the tester performs these activities:

Inputs are made into a software.
A portion of the software is executed.
The state of the software is observed, and its outputs are confirmed to be either conforming or non-conforming to the expectations of proper functionality.

There are several types of embedded testing, but for this article, we will briefly discuss the implications from a team management perspective. For more in-depth explanations, Guru99 wrote an interesting read on the subject.

Key Implications and Considerations

The transition to an embedded testing model requires several adjustments:

Sustained integration vs. occasional: Analysts working closely with development teams must learn their development language, what tools their colleagues use, and their preferred methodologies if they are not part of the analyst’s expertise portfolio.
Targeted assignments: Analysts are assigned to specific features or components, communicating their results to developers collaborating with them.
Direct communication: Test requests are transmitted directly from feature owners to analysts, facilitating a faster feedback loop. Analysts are also responsible for communicating their results to the various developers they work with, as said above.
Adaptability: Analysts must be adaptable as workload fluctuates depending on internal production pace. During slow periods, they will engage in brainstorming to identify additional tasks.
Enhanced responsibility: Analysts take on increased responsibilities, including direct communication with clients and participation in client meetings, rather than relying on their team leads or project managers.

Required Skills for Embedded Testing

Embedded testing reshapes the role of external analysts and has a few demands of them:

Clear and precise communication ability: Analysts play a crucial role in disseminating test-related information during scrums, ensuring alignment with project objectives. Therefore, they must be clear and concise in their communications and concerned with the understanding of the information they share.
Client engagement: Analysts interact directly with clients and stakeholders, requiring professionalism and proactive communication.
Meeting preparedness: Analysts participate in meetings with clients, requiring thorough preparation and active participation.

Management Modalities

For management, the shift to embedded testing presents challenges and opportunities:

Supervision vs. control: Management focuses on supervision rather than task direction, maintaining a balance between autonomy and guidance.
Agile framework: Agile methodologies like Scrum are essential for tracking progress, with kickoff meetings replaced by more collaborative sessions.
Selective team composition: Analyst selection, as demonstrated above, typically demands people with a bit more experience. Their profile is carefully considered before being assigned to a team.
Personalized support: Management provides personalized support, adapting coaching strategies to meet individual analyst needs. For their clients, follow-ups are provided on analyst integration progress and appropriateness.

Conclusion

Embedded testing offers a quality assurance approach for external teams that differs from their typical mandates, encouraging and fostering closer collaboration between development teams and testers by integrating them into the development processes throughout all its stages. By doing so, they participate as test owners and communicators with the many stakeholders involved in the development process.

By adopting this model, studios can improve the efficiency of production and verification of their game components, delivering higher-quality games while outsourcing tests that would traditionally be performed in-house. As the industry evolves, adopting innovative approaches like embedded testing presents an interesting approach for any developer seeking to ally with good quality assurance partners.