CZ Intro: So this is my favorite talk so far. It was so cool what this team does with data to get sentiment about their game. Now both of the speakers work for big companies that have games where they earn hundreds of reviews a day.
If you are just a small indie with a tiny game most of what they do is not relevant. But if you are doing opposition or game research by looking at reviews, this would be so helpful. This is a great way to categorize what your competitors are doing.
Deciphering the data left behind in player reviews. Why data is important, how to process and understand this data visualizing and tabling data
Why Player Reviews?
- Helpful to identify bugs across new versions
- Share it with your team so coders know when there is a problem with a feature.
Note reviews are ”raw data” they call it raw data because it is unfiltered and emotional. What their goal is, is to turn this raw emotional data into quantifiable data that they can turn into actions for their dev team.
To deal with this emotionally straining data they create a nice routine for reading reviews – do them in small batches over time, drink tea, play Tetris after it.
Steam Reviews
- Steam reviews can be accessed via API calls
Oculus Reviews are actually easier to access for developers
- They have a bunch of really good tools.
- You can tag reviews in the back end (CZ side note: that would be awesome steam feature)
- You can directly download reviews
The first step is to turn emotional “raw” data into unemotional and they do that by tagging each review
- Don’t mention emotions, tag what it is for example: fitness, dear devs, music, price, good job, dance, music
Tags they use
- Accessibility
- Audio
- Bugs
- Community
- Competitor – see what they are comparing to the other games
- Demographic
- Difficulty
- Gameplay
- Price
- User content
- Story
- Graphics
- multiplayer
- User experience (UX) – they use this for any emotional things they are feeling by the player that they can’t quite explain. Basically means, “this experience is bad for me”
- Performance
- Networking
- Leaderboards
VR Tags:
- Guardian
- Fitness
- Motion Controls
- Motion Sickness
Niche:
- Trigger warnings
- More – this is used for if the user likes the game they just want more content
- Parking lot
- Hype
- Offtopic
After tagging they clean up the data
- Remove excess columns.
- Use pivot tables to convert tags into individual tables
- Do date conversions
- Add version numbers for which version of your game that review is about
Turned raw emotion into quantifiable information can be useful in this way:
- You have trends and intuition – keep it in the back of your mind, but dont turn this in
**SUPER COOL TIP: ** Google spreadsheets added this new tool called “explore button” and you can just ask intuitive questions and google sheets will auto interpret your request and give you the results: Example: “What is average rating per version” and the table will figure it out.
Tableau – Advanced data visualizations tool they use to turn tags into visualized data. Like a bubble word chart. (see the screenshot below)
- Then they dive in and sort based on review sentiment and version and tags like features
- They can do heat map of tags so you can identify that the bad reviews are because of content.
- This chart shows 3 different games where they cross reference the color to the tag. Isn’t that cool? You can see what the major issue is.
This is great if you are doing opposition research
How to take actions on the reviews:
- Know what your development team is going to be doing because you can tell them if something is coming.
- If you see a problem that was complained about for a long time and then the bug is fixed then do a big celebration about that bug being fixed. You can do an update marketing beat about it.
Celebrate
- Post good reviews there if they come in. Make it very good and proud.