How was the October 2024 Steam Next Fest

An entirely new Steam Next Fest

This year Valve fundamentally changed how developers manage their demos and how they are given visibility within the algorithm (I wrote about these changes here):

Also you should totally check out similar analysis from another data cruncher Adam Sharman here.

I think these changes are great and they cement what I have seen since 2020 which is that making and promoting your demo is fundamental to your marketing. The demo should be the cornerstone of your marketing. Not social media, not the press, demos! Releasing a demo and marketing it to streamers and in festivals is how you get wishlists.

Along with these demo changes, Valve also revamped how Steam Next Fest worked. The October 2024 Next Fest (which ran from October 14-October 21st) represented the biggest change to the festival since its inception during the pandemic.

These changes have left developers with a bunch of questions so I created a survey to help answer them.

6 Questions I had going into the new Next Fest

Were the Steam Next Fest changes good or bad?
Do people play your demo or just wishlists?
Separate Steam Page or Not?
When should you push the email wishlisters button?
Did Steam Next Fest Change mid-festival to reward top performers?
Does having more wishlists lead to more wishlists?

Question #1 Was this a bad Next Fest for developers?

Answer: Yes, games underperformed compared to previous Next Fests. And, it seems the biggest reduction in visibility was towards top-performing games. Basically, the new algorithm effectively lowered the maximum potential visibility while maintaining the minimum.

Here are some basic stats

	Number of Games	Games surveyed	Average Wishlists	Wishlists earned 70th percentile	Wishlists earned Median	Wishlists earned 30th percentile	Median Impressions	Median Views
2020	??	61	4046	1,870	520	176	1,519,697	4,748
2021	696	42	3,752	2,169	957	550	585,743	7,970
June 2022	1056	65	4803	2,964	1159	434	511,091	5,654
October 2022	673	41	17518	5077	1174	645	664560	6048
October 2024	3,000	86	2205	1864	944	473	N/A	N/A

As you can see, the median and 30th percentile were almost the same as previous Steam Next Fests. But the top percentile was quite a bit lower. Therefore, the average wishlists earned are much lower because the top-tier games just seemed to perform worse and didn’t sway that average.

Here is a graph of every game in my survey. Each blue bar is a separate game.

Compare the 2024 graph to one that I made for the October 2022 Steam Next Fest

Notice that 7 of the games in my survey from 2022 achieved more than 20,000 wishlists (if you read my 2022 blog post I took Manor Lords out of many graphs because it earned 309,463 wishlists during the week of Steam Next Fest 2022.)

None of the games in my survey earned more than 20,000 wishlists.

Maybe this is due to sampling? Maybe big publishers have stopped following me or are less willing to share data with me.

So I looked at some correlated data from the Top 50 most played demos chart that Valve publishes.

Sulfur was #48 on the list and I reached out to the developer and they earned about 10,000 wishlists as part of Steam Next Fest. They gained 731 followers during the week.
Keep Driving was #24 on the list and their PR Team at Future Friends confirmed that they got 66,157 wishlists. They gained 3411 followers during the week.

Ok! So Keep Driving wasn’t part of my survey but they did exceptionally well!

Looking at their wishlist to follower ratio, it is 19. That means for every 1 follower they also got 19 wishlists. That could be a good key to estimate wishlist / follower ratios for other games.

For instance, Popucon was not part of my survey, but was #3 on the most played chart gained 832 followers. Using the Keep Driving estimated wishlist ratio, that means they probably gained 15,808 wishlists. Good, but not FANTASTIC like the games in 2022.

The Precinct was #6 on the chart and gained 3823 followers and an estimated 72,637 wishlists. That is good!

So what is going on?

It seems the wishlist earnings are much more volatile. Some games on the “most-played” list did much better than others (even though their rank was lower). It seems as if having a “top performing” demo in Steam Next Fest doesn’t automatically guarantee 30,000-80,000 wishlists.

If you had a top-performing game in Next Fest and earned more than 20,000 wishlists, please message me. I would love to find more examples of hits.

Another sub question: How many wishlists will I earn after the first day?

I found that on average, if you multiply your day 1 wishlist earnings by 6.69 you will get the total wishlists earned during Steam Next Fest. Of course this is an average and the multiplier was much lower for the lower end of the scale.

Question #2 Do people play your demo or just wishlists?

Many developers fear that Steam makes it too easy to wishlists games during a festival so you get an inflated, low-quality wishlist during the festival.

Other developers fear that allowing players to play a game will make them “tired” of it and they won’t buy the game later.

Developers fear a lot of stuff.

So what should they fear?

Back in the June 2024 Steam Next Fest Valve started reporting clear metrics around demo plays:

How many people played your demo but didn’t wishlist the game
How many people wishlisted the game without even playing the demo
How many people played the demo AND wishlisted the game.

I found the median across all the games in the October 2024 Steam Next Fest survey and generated this venn diagram.

MOST Steam Next Fest shoppers are just wishlisting games as they see them. It is not about playing the demo.

MOST Steam Next Fest shoppers play the demo and move on. The median is 5.26. That means that overall for every 5 demo players of the demo, 1 will wishlist it.

Even the best performing game in my survey didn’t have a much different picture.This is the venn diagram for the best performing game in my survey:

The top performing game in my survey was a co-op adventure game so it kind of makes sense that there were more many more demo plays than wishlists. My theory is players were encouraging their friends to play and those friends were just going along with it even though it wasn’t their favorite genre.

I was so curious about the overall effect, I made a chart of every game and their ratio of Played but didn’t wishlist to Played and Wishlisted.

Each bar is a different game in my survey. The height of the bar is how high that ratio was. To put it bluntly: high bar = more people bounced after playing the demo without wishlisting it.

Games that earned the most wishlists are on the left of the chart. You can see that those games with the highest ratios are typically the lower-earning games on the right side of the graph. But overall the correlation between this ratio and total wishlists earned is approximately -0.28 which is a weak negative correlation. So don’t freak out if you have a ratio greater than 5. But maybe double check what is going on with your demo if you have something above 10.

After the June Steam Next Fest I asked for this same number and here is the median chart. You can read my full post on the June 2024 Next Fest here

The median ratio back in June was 4.19. I think the overwhelming number of games that participated in the October Next Fest meant that players were moving on faster without wishlisting.

Question #3 Separate Steam Page or Not?

The newest Steam Store feature is a separate page for your demo vs just having a button on your game’s store page that says “Download Demo.” When you have a separate demo page, players can leave reviews for your demo. Developers can choose between the two options. So which is better?

Answer: You should do the separate Steam page for your demo so you can see reviews, even though there is a SLIGHT difference in the number of wishlists you earn.

Here is a graph of how games performed with the two different options:

The 3 biggest games in my survey didn’t create a separate page for their demo. But, if I look at the median wishlists earned for the two options it is much less clear:

Demo on main page: 967 median wishlists earned
Separate demo page: 930 median wishlists earned

So it doesn’t really seem to matter that much.

But what do I recommend?

You should make a separate page for your demo!

Get those early reviews! I know many developers are afraid of bad reviews from Steam shoppers but it is better to hear the truth now before launch so that you have time to fix it. You are going to face bad reviews at some time.

If the reviews get really nasty you can always pull your Demo Steam page and all those reviews disappear.

So again, open yourself up for feedback! It hurts but the act of creation is always putting yourself out there. It is part of the deal.

Question #4 When should you push the “Email wishlisters” button?

One of the newest features with Steam Demos is that within the first 2 weeks after publishing your demo, you can push a button that will notify everyone who wishlisted your game that the demo is live (see image below)!

So should developers push the button with 2 weeks before Steam Next Fest to get “momentum” or should they wait to push the button the moment Steam Next Fest officially starts to get “visibility” during the event?

Answer: I don’t have enough data to give a good answer but my gut tells me it doesn’t matter. Instead, get back to working on your game.

From the survey data I drew this chart. The blue bar is the median wishlists earned by the games that pushed the button at various times. The red line is how many games in my survey did this.

If you just look at the chart, you might make the assumption it is best to push the button 1 week before the festival. But there is just not enough data in my survey. For most options there were fewer than 10 games that did it.

If I look at what the top 5 best-earning games did:

3 of the top games never pushed it because the demo had launched earlier or they didn’t know about the button.
1 of the top games pushed it the first day of the festival
1 of the top games pushed it two weeks before.

I think the decision of “when to push the button” is very similar to “when to launch your game” – it is a decision which doesn’t have much of an effect but because Valve asks you when you want to do it, you have to make a choice which leads to over-analysis over a trivial detail, aka bikeshedding.

The decision that has the biggest impact on how many wishlists you earn is what genre of game you make. So spend A LOT of time on that decision and almost no time on the precise day of when to push the “Email Wishlisters.” It doesn’t matter.

The bigger problem is that 11 of my survey respondents didn’t even know there was such a button. Even though the choice doesn’t matter, you should try to learn how the system works and use every option available to you. Make Valve earn that 30% you give them by using every feature they give you! I wrote about the new button on my blog in a post back in July when the changes were unveiled. Please don’t miss this stuff! You should subscribing to my newsletter and read every word of every blog post I write (I kid! I kid! But, seriously, please read my blog this stuff is important).

Question #5 Did Steam Next Fest Change mid-festival to reward top performers?

When Valve revealed that there was going to be a new Steam Next Fest algorithm, they hinted that they would give equal visibility to all games during the first few days of Next Fest. Then after a few days collecting data, the algorithm would start prioritizing the higher performing games.

It sounded like some sort of AI machine learning algorithm was being trotted out.

Midway though the festival On October 17th Alden Kroll from Valve tweeted that the change had occurred and Steam Next Fest would start recommending games based on prior data. In his statement (linked below) he clarified how the favoring works.

Based on his tweets it sounds like some mysterious AI magic that does the evaluation of what people want.

I was curious if I could see this change in recommendations on Steam.

Here is what I saw.

If I look at the top 5 wishlist-earning games in my survey then find the median of them for each day and graph it, here is what I get:

You can definitely see that some big increase happens on Day 3 and then the recommendations keep going up. Note that Day 8 count drops significantly; that’s because that day is when the festival ends mid day.

Although the graph goes up, this doesn’t mean the algorithm is now recommending the game. This steady increase could be because of the effects of being on the “Most Wishlisted” and “Most Played Demos” lists.

If I look at the daily median of the bottom 5 games I see this:

You can clearly see the steep dropoff in visibility after the first 2 days.

If I look at the day wishlist history for the middle 5 games in my survey I see this:

Also a dropoff, but doesn’t seem as steep as the bottom 5 games.

I was curious, where does this dropoff happen?

From Day 2 to Day 3 seems like a dramatic change.

So, I looked at all the games in my survey and looked at how the wishlists changed between Day 2 and Day 3.

Here is the graph of every game as it changes from Day 2 to Day 3. Each game is a blue bar. If a bar is under “0” that means it earned fewer wishlists on Day 3 than Day 2.

It is pretty clear, the top 4 games in my keep getting promoted on the following days while pretty much all the other games in the festival drop the next day.

Why such a dramatic drop for the #1 game in the survey? Here is its daily wishlist chart for just that game:

You can see a dip in Day 3, but the following days it bounces around 2500 per day. Also note how many daily wishlists it is getting; it gets an order of magnitude more wishlists than the other games so small dips look more dramatic by comparison. Overall it doesn’t look like any traffic is being cut to it, just noise.

So what is this big cutoff where games that earn less get cut the next day? 1831 wishlists on Day 2.

However, I don’t think that is caused by some “evolving recommendation algorithm” that Alden was tweeting about. I think it probably has to do with those top charts.

In my survey data, I can’t really see a clear reaction on day 2 or 3. Basically, every game that is not doing very very well in Next Fest gets reduced coverage in the following days. The top games seem to get steady visibility over the week instead of a reduction.

So maybe the Steam algorithm is doing some machine learning adaptive recommendation system, but I can’t see it having a major, isolated effect.

Maybe they are still tuning the machine learning algorithm and this was just a test. Maybe they chickened out and decided at the last minute not to do such a dramatic promotion of some games?

Who knows what Valve does.

Question #6 Does having more wishlists lead to more wishlists

In the old Steam Next Fest one of the best ways to ensure having a good Steam Next Fest was to try to have a wishlist total of at least 150K wishlists before the start of it so that you could be featured on the front page on the “Most Wishlisted” charts

Is that still true?

The correlation between number of wishlists before Steam Next Fest and performance had a correlation of 0.673 which is a moderate to strong positive correlation between two variables.

As you can see there were several games that were announced during SNF so had 0 wishlists before but still did well. However on the far right side of the graph, most of the games that underperformed also had low wishlist counts.

Valve did deprioritize the “most wishlisted” charts (It was now hidden on a sub tab) but there was still an effect on wishlists. You can also see this in the data for “Did Steam Next Fest Change to reward some games?”

Adam Sharman had a similar finding when he looked at several games sharing their wishlist numbers.

So I don’t think it is as important as it once was but still quite important to enter Steam Next Fest.

Summary

So what do I make of the new Steam Next Fest format?

The new Next Fest Format was kinda bad for top performing games and about the same for everyone else.

The games that went into steam next fest with the most wishlists were not necessarily rewarded for that. Many of the games that were in the 50 most-played demos only earned around 10,000-20,000 wishlists which is a lot but not as good as in previous SNFs.

I think it is great goal for Valve to try to increase exposure to more games, but the “middle-class” games seemed to earn just a bit less than in previous fests.

But keep in mind that this Next Fest had 3x as many games than in previous ones, so perhaps the tidal wave of new games flooded the system and this is a decent outcome considering the circumstances.

What to think about Steam Next Fest

Steam Next Fest has never been the #1 way to earn wishlists for most games.

I write about Steam Next Fest a lot not because it is the best way to get wishlists but because everyone gets to do it and therefore people have to make a choice of “when do you enter Next Fest.”

It is similar to when you launch your game.

I don’t think the day you launch matters as much as people think it does. But, because Steam forces you to make a decision it leads you to over-analyze that decision.

There are a few games that do VERY VERY well in Steam Next Fest but those are the outliers.

Really other smaller festivals that are more exclusive are better at getting you visibility. We track those festivals over at howtomarketagame.com/festivals

So what is my recommendation after all of this?

You should still do the last Next Fest before you launch, even if it doesn’t work as well as it used to.
Don’t debut your demo during Next Fest, instead do Alphas, Betas, and launch earlier as part of another festival.
Push the “Notify Wishlisters” button way before Steam Next Fest.
Market your game in the run up to Steam Next Fest (I will give you tips on how to market it in a follow-on post to release later this week.
Make a separate page for your demo so that you can get reviews that will help you improve your game.