#gamesUR Summit 2022 Talks

A Games User Researcher’s Toolkit: The benefits and challenges of using external tools and platforms

This session is about the tools that we - as people doing research and UX work in games - can use to do our jobs, how those tools are maturing, and how the tools themselves are changing the way that we work. We carried out interviews and surveys with user researchers and other staff members at game studios to look at their support needs. We saw increasing prevalence and use of tools and research platforms to support games user research programmes, at least partially in response to the pandemic.

This kind of rapid change can be difficult however. In this talk I will look at the potential benefits and challenges associated with adopting the use of tools, drawing on not only our research findings but also experience both inside and outside of the games industry.

* As a researcher, how should I handle increasing prevalence of tools and platforms? What are the implications for not only day-to-day work but my own skills and personal development?

*As a decision-maker at a game studio, how do I decide which of these platforms and tools we should be using? There are so many!

* How can we make sure that we’re adhering to good methodological practice and developing our UX maturity if much of our research work is done on external tools?

Back to top.
Gareth Lloyd

Playtest Cloud

Between Agency and In-house Research: How to Communicate With 41+ Dev Studios

Working at MY.GAMES, a product company that currently has 41+ studios of various sizes, we conduct research for everyone. Slowly but surely over the past 5 years, we've been introducing UX research culture into a company that rarely did game research before.

Initially, studios came to us when they faced issues with retention and could not explain "bad" metrics. However, now we have moved to another level: we discuss with the teams in advance the applicability of UX research at each stage of product development. At the moment, when faced with a variety of games (large PC / console projects, medium projects, small mobile, experimental projects), we have to optimize processes, adapt to teams and their resources, interact with them, integrating into the game development pipeline.

We are going to tell you how we have introduced UX research culture in recent years, how we communicate with all the studios and don’t go crazy in this chaos of a variety of products, different audiences (USA, Europe, CIS, Asia) and processes. Also we will share lifehacks on optimization.

Who will benefit from the presentation:

- UX researchers in individual studios and holdings: we will share information on how to build processes if there are more than one studio and you are inundated with projects.
- For those who want to implement a UX culture in a company: in order not to step on the same rake as we did, and to use the experience of our laboratory.
- Game designers, producers and anyone interested in UX research: we will share how we communicated the value of research to teams and what results this led to.

Back to top.
Ekaterina Lisovskaia

MY.GAMES

Maria Amirkhanyan

MY.GAMES

COVID Realities from the GamesUR Salary Survey

The COVID pandemic has has unprecedented global influence on the video games industry, turning production and communication norms on their heads as studios reacted.

The influence of this shift—technologically, socially, operationally and culturally—seems likely to leave lingering implications on the realities of games user research as a practice.

This talk brings together insights from the GamesUR Salary Survey, which collected anonymous reflections on changing research operations.

- How has gamesUR changed in this period?
- What have been the challenges?
- How has the pandemic impacted our work?

Seb will present themes and learnings sourced from the two gamesUR salary surveys conducted mid-pandemic.

This presentation is presented on behalf of the games user research salary survey team of Seb Long, Jonathan Dankoff and Andrew Menger-Ogle.

Back to top.
Sebastian Long

Player Research

GamesUR Salary Survey Team

Designer-inspired Maturity Framework for Sustainable Game Design Experiences

Game design professionals design memorable experiences for players. While games user research focuses on enhancing player experience, it is equally intriguing to study the experiences of professionals developing these games within the game development community. This presentation (1) summarizes factors that inspire and sustain professionals to design games, (2) highlights major challenges faced by professionals in the game industry and (3) proposes a design maturity framework that can help assess the maturity level of game development experiences. The design maturity framework measures various dimensions of game development in the key focus areas of game studio operations, game designer experiences, player experience research, quality of games as a product and game design scholarship. The objective of the session is to present game design professionals and studio executives with frameworks that they can use to evaluate their own practices for an enjoyable and sustainable workplace experience. The concepts presented in this talk will help build positive experiences within the game studio, highlight some of the challenges faced by game developers in the industry and open discussions on what can be done to improve game design practices in the gaming community.

Back to top.
Nandhini Giri

Purdue University

Flighting & UR: the Perfect Partnership. A retrospective from the development of Age of Empires IV

A major component of understanding player experiences during Age of Empires IV development was reaching out to specific audiences early on and with enough time to iterate. In this talk we will discuss how the UR and Flight teams worked together to get input from a council of highly engaged franchise community members, as well as volunteers from within the company, and how this feedback became even more important during the pandemic. We will cover ways in which Flighting is a parallel approach (along with traditional UR methods) to gathering player insights that runs throughout development, accommodations we made for our specific audiences, techniques for engaging the dev team in the process, and what we learned about how to improve the process in the future. We will also discuss how we iterated on the flighting process to support a live services model following the game’s launch. We hope that sharing this story can demonstrate the value of this partnership for both sides.

Back to top.
Savannah Harrison

World's Edge, Microsoft

Todd Kelley

Xbox Research, Microsoft

How do we run remote research with kids who are so over Zoom?

In-person research with children (<13 yrs old) is already a challenging task, given so many different factors (ex. kids' short attention span etc.). With the increasing shift from in-person to remote research, there is an even greater need and importance in establishing effective research practices to successfully conduct remote research with children.

Developmental Psychologists have conducted several rigorous studies to establish the viability of various remote testing methods with children. This talk aims to translate and contextualize these methods in relation to running remote user experience research with kids in hopes that researchers can become more successful in applying remote methods to obtain quality data with children, given all the constraints.

We discuss 3 practical tips for implementing interactive studies so that children remain engaged (and hopefully have fun!) with minimal parental involvement.

Back to top.
Carissa Kang

Roblox

How to playtest Dead by Daylight: an exploratory approach on a live asymmetrical multiplayer game

In this presentation, I wanted to share how our newly establish user research team at the time managed to put into place all those new processes and how we strengthen our practices with the game Dead by Daylight. The asymmetrical and multiplayer nature of the game was, at the time, unique in the game industry as well as challenging especially in a research context. To be able to provide useful and meaningful insights, the user research team explore different ways to establish ongoing conversations and collaborations with the design team. The continuous process to understand the creative and design intentions were key elements on how we could create tailored and appropriate protocols for upcoming features for this game.

The main take away of this presentation:

-Understand the different realities of a small and newly establish user research team.
-How we put together our playtest processes in a live asymmetrical multiplayer game with the collaboration of the dev. team and players' feedback.
-Best practices and avoided practices during our playtest.

Back to top.
Émilie Paquin

Behaviour Interactive

Industry-academic Partnerships to Train the Next Generation of Games User Researchers

Ubisoft Toronto and Ontario Tech University have recently partnered to co-run a games user research (GUR) training program where students or recent graduates get to participate in a paid opportunity to develop their practical GUR skills by working on project briefs provided to them by Ubisoft Toronto, while receiving training and mentorship from both Ubisoft and Ontario Tech researchers. This presentation highlights the motivations, challenges, and opportunities in establishing industry-academic partnerships. We will provide details on our GUR training approach and discuss lessons learned.

Back to top.
Ahmed Ghoneim

Ubisoft Toronto

Pejman Mirza-Babaei

UXR Lab, Ontario Tech University

LITE: Not Quite RITE

RITE testing is a well-established method in our field, for good reason! However, both RITE and standard usability can struggle in the face of systems that are spaced more than two hours apart, and RITE in particular can be taxing on teams and it can be hard to get the necessary team buy-in as a result. I’m going to talk through a modified version of RITE – which I call Long-term Iterative Testing and Evaluation, or LITE – this lighter-weight version embraces the flexibility of the original method, but is less resource-intensive for the team and can facilitate broader coverage.

Essentially, LITE is a method optimized to generate high quality usability data for games with complex systems that require long-term engagement to achieve mastery, particularly for chaotic projects that have a high risk of missing build deliveries. It does this through leveraging an engine: a standardized schedule of biweekly research sessions paired with weekly planning sessions and rapid, real-time, reporting of issues in order to facilitate broad topic coverage, a regular flow of data, the flexibility to adapt sessions to the team’s needs in real time, and the accountability to iterate in response to UR feedback.

In this talk, I’ll give an overview of how to run LITE studies – including discussion of when a LITE approach is likely to be helpful and when to turn off a LITE engine – as well as compare the method to another iterative testing model, Rapid Iterative Testing & Evaluation (RITE), and provide an overview of the history of the method with some examples of specific titles where it has been particularly useful.

Back to top.
Jonathan Ehrich

Xbox Research

Not just for climbing—Ladders in Games User Research: A Flexible Framework to Unify Stakeholders and Shift from a ‘Game Features’ to ‘Player Needs’ Mindset

Researchers often work with multiple game teams and encounter a wide variety of design philosophies and audience definitions (even within the same team). Designing with game-features first may limit the eventual appeal of a game. Starting with a motivation-based framework helps design teams recognize important player needs, align around player-centric design pillars, and provides a common language to inform future design decisions.

This presentation will outline a process to help you 1) develop an initial set of player motivations, and 2) connect those motivations to specific experiences and game-features to satisfy your future players’ needs by doing an informational deep dive into the 1:1 semi-structured in-depth interviewing technique of laddering. There will be an introduction to the technique and its underlying theory, tips to implement it into 1:1 interviews, discussion around why it’s a flexible method, and examples of ways to analyze, visualize, and communicate findings.

Lastly, takeaways will be discussed, such as laddering being a flexible framework for your research toolkit, as it can help stakeholders transition from a ‘game features’ to ‘player needs’ mindset, while also informing both development and marketing teams.

Back to top.
Bill Hardin

Amazon Games

Presence in Practice: Affinity vs. Contrast in Video Game UX

In 3D games we see many types of user interfaces. In this session you will learn what they are, and how to best select which type of interface should be presented to our players depending on the player's level of 'presence' while playing your game. You will see a case study covering a fictional abstract AAA open world game realised for this talk and how the UI is structured from the ground up while best respecting immersion in our game world and giving the game's core systems room to breathe.

Back to top.
Ahmed Salama

Ubisoft Stockholm

Seeing Yourself in the Game: A Guide to Inclusive Character Customization

Players take a variety of approaches to customizing game characters. Some prefer to make a character who looks like them to feel immersed in the game while others prefer the escape of role-playing as someone completely different. However, for underrepresented groups, character customization systems often lack options that allow them to make a character who fully represents them. In this presentation, I discuss research focused on which customization features players value most. The analysis focuses on 5 broad categories: body, face, skin, hair, and voice customization, and how gender identity and race/ethnicity impact how much players value those features.

Back to top.
Deanne Adams

Skills and methods as a Game User Researcher: Is the job still the same after COVID?

Interested in becoming a Game User Researcher, but you are not sure if you have the right skills?

This talk might be for you!

Based on my previous talk at the GRUX online (What your future employer wants you to do), I expanded my research around GRUX job posts as well as trying to compare pre- / post-COVID posts to see if anything in terms of skills, methods, and job requirements changed with the New Ways Of Working that came with this pandemic.

This talk is primarily valuable for newcomers who might be interested in learning what skills they need in order to get their first job, but also for everyone else, who is interested to see what the day-to-day and methods used are, for each level of seniority as a game user researcher. UX Leads, implicated in the recruitment process, can also find benefits in it by discovering what the general trends in skills and methods are, and what other companies consider useful skills to have as a GUR.

Back to top.
Raphaël Leroy

Anais digital

Taking Flight: Soaring into the Golden Age of Social Science

Tech and its place in our lives has exploded over the last few years, becoming deeply embedded into all aspect of the way we work, play, and live. The games industry, too, has reached a defining moment: ever more players are streaming into our gaming experiences and communities, and there are more and more ways to play and engage with games every day. As our industry becomes increasingly more complex - and as we begin to understand more about the challenges and opportunities that our new world order brings - the questions our partner teams want to answer about our players and their experiences have grown more complex as well.

Simply put, we're in a "golden age of social science." The challenges that face our field won't just be solved by tech stack or fancy economics - they'll be solved by us, behavioral scientists who can see the weave that underlies the reality of human behavior. Our field is more important than it's ever been, as we're uniquely equipped to help guide our game development teams through this crucial moment. In this keynote address, we'll discuss some guiding principles to help us completely seize this moment before us.  

Back to top.
Melissa Boone

Xbox Research, Microsoft

The Ethics of Our Art: Research Ethics and the Games User Experience

“Any tool can be used for good or bad. It’s really the ethics of the artist using it.” John Knoll, visual effect artist 

 Many professions have ethical principles or a code of conduct that they follow to make sure their work is credible and trustworthy. Adhering to these principles can also ensure that the people within the profession and interacting with the profession are treated with respect and care.  

When research is done in an academic setting, three ethical principles are used to guide researcher conduct:   (1) Respect for persons, also known as a person’s right to choose based on good information.   (2) Beneficence, the idea that we will ‘do no harm’ and try to benefit others.  (3) Justice and fairness. 

These same principles also have unique applications to the field of Games User Research. The purpose of this session is to explore why and how these ethical principles should be considered by the Games User Research community. The session will also propose actions Games User Research and Experience professionals can take to treat their users with respect and care.  

The session is led by two research ethics professionals who have a combined 30 years of experience in academia, including research ethics in the field of entertainment arts and engineering. 

Back to top.
Ann Johnson, PhD, MPH, CIP

University of Utah

Lisa Rigtrup, BS, CIP

University of Utah

User experience: ground for virtual and real-life taxes

The study addresses the user experience generating in-game and real-life tax obligations. The research compares these tax obligations and assesses them from a user's and tax perspective.

The presentation focuses on two areas. The first is in-game taxation as a part of the virtual world's economy from both a user and tax perspective. MMORPGs often use virtual taxes as one of the tools to maintain the virtual economy. This research addresses types of such virtual tax, their commonalities, and differences with the real-world taxes, mitigation of in-game taxation, and if it makes sense from a professional tax perspective.

Second, the study addresses the situation when the user's game activities and virtual earnings evolve into real-life tax obligations.

The study calls upon careful representation of taxes in games and consciousness of real-life tax obligations of gaming experience.

Back to top.
Anna Vvedenskaya

University of Amsterdam, the Netherlands

UX Research Practices and Challenges of Indie Developers

The organising team members of this roundtable have expertise in both academic and industry and have been working closely with several indie developers. Based on these experiences and recent discussions with developers, they have identified some common practices and the challenges associated with them that we would be the focus of this roundtable session:

1) Value of UX research for indie studios: Discussing areas that UX research may add value to indie studios, from improving the quality of their titles to helping team members better align on their KPIs.
2) Defining research objectives: Discussing how to help teams identify key research objectives.
3) Finding and vetting participants: Discussing best practices and resources for recruitment of participants.
4) Choosing appropriate research methods: Discussing adaptation of mixed methods to better align with indie studios' resources and research maturity.
5) Collecting and interpreting data: Discussing approaches and techniques for analysing and interpreting research data.
6) Dealing with feedback: Discussing reporting of research findings to the team and prioritising tasks and activities to act on the feedback.
7) Evaluating ROI: Discussing approaches for assessing game success and research ROI post-release.

Back to top.
Alena Denisova

University of York, UK

Jason Della Rocca

Execution Labs

Pejman Mirza-Babaei

UXR Lab, Ontario Tech University

Steve Bromley

www.gamesuserresearch.com