The build is done. The screenshots are made. And somewhere between finishing and announcing it, somebody suggests a soft launch, usually in Canada or the Philippines, usually for a couple of weeks.
Then two weeks pass, a few hundred people install it, the numbers look roughly fine, and you go global.
Here is what is wrong with that, and it is the same arithmetic that ruins most app tests. A few hundred installs cannot tell you whether your conversion rate is good or bad. It can only tell you whether the app works. Those are completely different questions, and a soft launch is excellent at one of them.
So: what a soft launch can actually answer, and what it cannot. How long it has to run before the numbers mean anything. And the two store features that do the same job with one toggle.
Everything on this list is a yes-or-no question, which is why a few hundred people is enough to answer it.
None of those needs statistics. They need somebody who is not you, on a phone that is not yours, in a storefront that takes real payments.
The soft launch is usually sold as a way to tune your funnel before the real audience arrives. The arithmetic does not support that.
Freemium means an app you can use free and pay to unlock more. The median one converts 2.1% of downloads by day 35.
At that rate, telling a 10% difference from noise takes roughly 76,800 users per version. A soft launch with 500 installs produces about ten buyers. Ten.
So a soft-launch conversion rate is not a benchmark, it is an anecdote with a percent sign. If it comes out at 1% you have learned nothing, and if it comes out at 6% you have learned nothing more enjoyably.
What you can read at that volume are the big, early, binary things: did people get through onboarding, did the paywall appear, did anybody buy at all.
Not two weeks because two weeks is a round number. Run it until your own funnel has finished.
Working out your actual minimum
Add these together rather than picking a duration first.
Your trial length
Nobody who installs on the last day has converted. A 7-day trial means the final week of data is incomplete by definition.
Plus one renewal
If you want to know whether people stay, and not just whether they start, you need the first renewal to have happened.
Rounded to whole weeks
Weekends do not behave like weekdays. Ending a test on a Monday samples a different population from the one that started it.
A 7-day trial on a monthly plan therefore wants about six weeks, not two. If that sounds long, remember what the alternative is: finding the broken purchase flow on launch day.
The usual list is Canada, Australia, New Zealand, Ireland and the Philippines, and the usual reason given is cheap installs. That is the wrong reason.
What you want is a market whose behaviour resembles the one you are actually launching into. Price sensitivity, payment habits, language, and the kind of phone people are holding.
A test in a market where the median subscription price is a third of your target market’s tells you something true about that market and nothing reliable about yours. Cheap traffic is cheap for reasons, and those reasons show up in your conversion rate.
If your real launch is the United States, English-speaking markets with similar spending patterns are the closest proxies. If your real launch is Southeast Asia, the answer is completely different, and so is your price.
Both stores will do most of this for you, and almost nobody uses them.
Phased release on the App Store rolls an update out over seven days, to a growing share of users who have automatic updates switched on. You can pause it at any point. If something breaks on day one, it broke for 1% of them.
Staged rollout on Google Play does the same with a percentage you choose, and you can halt it.
Neither helps with a first launch, because there is nobody to roll out to. For every launch after that one, they are the soft launch, and they cost a toggle rather than a quarter.
Fix what broke. That is the whole point and it is enough.
Resist rebuilding the paywall because 500 people converted at 1.4%. You do not know that 1.4% is wrong, and you will not know until you have twenty times the traffic. Changing a screen on that basis is a judgement call dressed as a result, and the cost is that you can no longer tell which version you are measuring.
Then go. A soft launch that never ends is a launch you are avoiding.
Releasing your app in one or two small markets before the launch you actually care about, so that problems show up in front of a few hundred people rather than your whole audience. It is a bug-finding exercise far more than a tuning exercise.
Long enough for your own funnel to finish, which is your trial length plus one renewal period, rounded to whole weeks. A 7-day trial on a monthly plan therefore wants about six weeks. Two weeks is a round number rather than an answer, and the last week of any test is incomplete because nobody who installed on the final day has converted yet.
Not usefully. At the median freemium rate of 2.1%, 500 installs produce about ten buyers, and separating a 10% difference from noise takes tens of thousands of users per version. A soft-launch conversion rate is an anecdote with a percent sign on it. What it can tell you is whether purchases complete, whether anything crashes, and whether onboarding reaches the end.
One whose behaviour resembles the market you are really launching into: price sensitivity, payment habits, language and devices. Canada, Australia, New Zealand and Ireland are common proxies for a US launch. Choosing a market for cheap installs gives you behaviour you cannot generalise from, because the traffic is cheap for reasons that show up in your conversion rate.
Conversion design for subscription apps.
Screenshots, onboarding and paywalls, connected around one goal: getting more users to pay.