Background

We worked with VRT Twente to create a serious game that simulates a crisis situation - Power Outage in our case. This is a project that was done between October 2021 and January 2022. My team came up with a web based game that can be played across all platforms.

I worked as part of a small team of four people. All of us come from different backgrounds with two specialising in Programming and the other two having experience in Industrial Design.

My responsibilities included the following:

1. Devise a plan for the initial user research.

2. Plan a flow for the prototype with the results.

3. Build the web application.

Research Phase

We started off with Business Requirements, and followed it up with Idea Validation, User Research and Profiling.

Business Research

We investigated various games that already existed on the market, saw what methods they used and how we can improve upon those methods.

An example of a Serious game:

World Rescue

‍

Card Sorting Activity using MIRO

This was done to get an idea about various actions that should be incorporated into the game. This activity had some actions (on cards) that we believe the user would make when faced with a crisis. The user has to rank the cards based on importance; the user can also add their own cards. We also had some interventions with various scenarios to understand how and what sort of actions users take in these scenarios.

‍

Key Findings

We initially hypothesized that International residents and Dutch residents will have different plans of action, this turned out to be true. We now have a diverse list of choices that we plan to include in the game.

Ideate Phase

This Phase includes Sketching followed by LoFi Prototyping and HiFi Prototyping with usability testing after each prototype.

Sketching

We made some sketches of how we planned our game to look. The UX process of how the game is cycled through was also decided from this phase.

We did not see value in making proper paper prototypes as we will not be able to convey the severity of the crisis without actual visual cues and graphics. So we decided to go with an illustration-style 2D game. The activities we did in the card storing were taken as a base for the game screens, but still, they required some refinement. To refine this we defined the game and had a long in-person meeting where we discussed the activity that the user would do during each day, the mode of interaction, and made a user flow. This gave us an overview of how many screens and assets are required for the game. The user flow also highlights the key action at each screen.

Hifi Prototype

As we had to make a game, some of the parts cannot properly be simulated using a low fidelity prototype. Moreover, we had some time crunch issues. So we decided to go ahead with two phases of HiFi testing.

Survive the Crisis: A link for the final web based serious game programmed with the help of Figma design and vanilla JavaScript.

Conclusions

When we initially started this project, the main aim was pretty clear, but it was too broad. There were a lot of things that needed to be done before we could get started with developing the game. 

There were two main facets that needed to be addressed : 

1. The look of the game and the flow of the interactions in the game.
2. The game itself i.e. the mechanics, gamification elements etc.

Ideally, both of the above points are required for a complete game. But due to the time constraints we had we were focused more on the first point.

System Usability Scale (SUS):

Score: 75.63 Standard Deviation : 13.01

User Experience Questionnaire (UEQ):

We see very good results considering the fact that this is a serious game and we only had a very limited number of testers. We lacked a bit of time to include a higher number of participants, but the scores we achieved are more than respectable with almost every category ranging above average in metrics. We achieved our best results in Attractiveness, Perspicuity and Efficiency and sort of middling results in the other metrics Dependability, Stimulation and Novelty. These can be better explained once you consider the statistical significance of these scores. You can find that in the table below:

As we can see, a mean score above 1.5 is considered to be acceptable for publishing the application for the Attractiveness attribute and we comfortably achieved it with a reasonable variance considering the participant limitation(low number of participants) we had. Overall, we achieved a Good result as shown in the benchmark figure.

Next comes Perspicuity with a mean score of over 2 and a low variance of 0.45. A score over 2 is considered to be Excellent for this attribute and perhaps the most important attribute considering the fact that this is a serious game.

Efficiency is also once again scoring on above-average going good and this is a commendable thing considering we are taking in a  lot of answers to the questions from the participants.

Dependability is a metric that cannot be properly measured for the web application we made considering the fact that we never really focused on it as we mentioned to people that this was only testing and no data will be collected from them based on the choices they made in the game.

Stimulation and Novelty are our worst-performing attributes and also the most subjective of them. These expectations vary widely among users and the very high variance values basically prove the same. This high variance over this small value is typically not a good measure. Even so, the score does stay in the above-average range.

Usability Issues and Improvements

Some learnings about improving the clarity of the visuals learnt from Web prototype testing:

Accomplishments

• The intuitive nature of the game. People did not face any difficulties in understanding what needs to be done on any screen.
• The colorful graphics used in the game. These were meant to lighten the situation, give the feeling of a game and was really appreciated by almost everyone.
• The clarity about the information that was available in the game was also appreciated by almost everyone.
• The most important fact was that it was fun to answer the questions asked through this click and play visual format.

Limitations

• Some of the functions seemed half baked. This was true since we only managed to develop the game to just a certain extent due to various limitations.
• Graphics. While they were appreciated by most, some lamented that the intensity of the situation was reduced a bit.
• Lack of storytelling in the later parts of the game. This was overlooked by us in the later parts. This needs to be improved.
• The rigid nature of the game, lack of smooth transitions, progress indicators etc basically lack of gamification elements.

Further research

• Gamification and Storytelling. These are the most important things that need to be done as they are severely lacking at the moment.
• More scenarios - especially self related situations. The majority opinion was that the game was too short. It should have had more situations. The scope of the game was quite limited. Why not extend it over the whole city or country.
• Going back to previous answers, changing them and even though this was more about collecting information, give points and grade what all actions that can be graded and give solid feedback to the users.

‍

‍