Customer Validation Examples: Attempting to Scale Customer Acquisition With 29 Marketing Experiments

I originally wrote this post in 2012 on the Backupify blog. It has since been lost and my friend and colleague Rob Stevens managed to record it. Rob and I both think it’s a useful example of how to think about scaling marketing through the next phase of a startup, so I’m re-posting it now so it is findable. Some of the methods are dying, or less relevant, but, hopefully you find some gems that help your company.

Almost everyone in technology now knows the Customer Discovery process. All I hear from startups is stuff about “product/market fit.” Unfortunately, not much has been written about the next step — Customer Validation.

The Customer Validation step is all about finding a scalable, repeatable sales model. Most startups rely on just a few customer validation experiments: PPC, SEO, viral, blogging and social media. But there are many other ways to acquire customers, and my goal with this post is to highlight some of the things we have tried at Backupify.

Since we are almost 4 years old as a company, our customer acquisition experiments have run the gamut. We have done most of the traditional startup things, experimented with many of the more traditional customer acquisition methods, worked partnerships, and even tried a few things out of left field. Given that we recently did a year-in-review of our 2012 experiments, I thought it would be helpful to the ecosystem for us to share what we learned so that other startups can get ideas as they go through their own customer validation process.

Over the past year we ran 32 different marketing experiments. The purpose of this post is to describe those experiments and explain our results. Most of them did not work well, but they may work well for you, so I encourage you to read them all with an open mind. There are two things to note about this list. The first is, we did have three experiments that worked very well. Those will not be covered here because we aren’t ready to expose that information. The second thing is that the vast majority of these experiments were focused on filling the top of the funnel. We also ran numerous sales experiments and some mid- to bottom- of-funnel experiments as well, which aren’t covered in this post, but all of our numbers are very good there and so most of our efforts are focused on top-of-funnel.

Before I go into the experiments, let me set the stage for our market so that you understand our unique marketing challenges. First of all, we are in a new market. SaaS itself is new so backing up SaaS applications is even newer. As of today, there are only a handful of companies that are doing it, and none of us are generating more than a few million dollars in revenue. Our products are sold to IT managers or CIOs, since they are the people who love backup. Whereas a typical person will ask “why do I need to back up Google Apps or Salesforce data,” an IT person will actually describe our products as “exciting” because they now get to have granular control over restoring data lost from user error, hacked accounts, or accidental deletion.

Our target customer is the IT manager at a company with 250 to 10,000 employees.

Companies over 10,000 employees have large IT departments that often build custom solutions, and aren’t adopting SaaS as rapidly as smaller companies. We have many companies under 250 employees who have bought our product, but the product is designed with larger companies in mind. As of today we have over 5,000 paying customers. About 90% of our revenue comes from Google Apps backup, but our Salesforce backup is growing at 3x the rate and will probably overtake our Google Apps revenue in a little more than a year after launch. Most of these experiments below were focused on customer acquisition for our Google Apps backup product.

We ranked all of our experiments on a scale of 0–3 depending on how well they worked. I will include some experiments below that scored a “3ʺʺ, but not the top three experiments we ran and continue to use for the majority of our leads.

The Experiments (ranked in descending order):

1. Reduce trial from 30 to 15 days. Score: 3 Notes: Sped up buying process with no impact on close rate.

2. Price elasticity test. Score: 3 Notes: We ran Google Apps backup offers of $1, $2, and $3 per user (our base Google Apps backup prices is $3 per user) to over 1000 trials who previously did not sign up. We got great results and learned that our price was relatively inelastic at those levels. This meant that while a lower price may attract more business, it wouldn’t attract so much more that it made up for price difference.

3. Backupify Migrator. Score: 2 Notes: This throws off a similar number of leads as our Snapshot Tool (which was a 2011 experiment that has gone well), but the companies that sign up for Backupify after experiencing Migrator tend to be smaller than those that come from Snapshot, so we get less MRR (monthly recurring revenue) from them. The plus side of this tool is that it effectively runs at a negative cost of customer acquisition because many companies try Migrator and buy some migrations, and that revenue (while one-time and not recurring) is significant.

4. Outsourced cold calling. Score: 2 Notes: This generated good leads, but was economically inefficient and led to very long, slow sales cycles for these leads.

5. Website improvements. Score: 2 Notes: We did a major website overhaul. Our conversion rates of visitors to trial went up slightly, and we got many more “contact us” leads.

6. Whitepaper networks. Score: 2 Notes: Two whitepaper networks that did provide some leads but had much lower conversion rates and were less scalable than we wanted.

7. Product reviews. Score: 2 Notes: Having an external party review our Google Apps backup product provided a decent return.

8. Website whitepaper leads. Score: 2 Notes: Adding whitepapers to our website led to leads that convert well. Unfortunately, scalability is tied to website traffic.

9. Infographics. Score: 2 Notes: Our best infographic of the year was our social media value infographic. It didn’t beat our 2011 infographic about data loss in Google Apps, from a lead perspective, but did from a PR and overall traffic perspective. Overall, infographics have become more competitive and less interesting to the media.

10. Dreamforce. Score: 2 Notes: Startups don’t usually do trade shows, but as the world’s biggest conference around a single ISV, Dreamforce was awesome and was a great source of leads. But, it was our first time and we didn’t play it all that well. Next time we will be more prepared.

11. EduCause. Score: 2 Notes: This is a great conference to get in front of EDU decision makers if you sell to that market, but they aren’t on Google Apps in large enough numbers to make it worth our while.

12. Webinar (company unnamed). Score: 2 Notes: Getting turnout to a webinar is tough, but those who come do become customers at decent rates.

13. Syndicating Bloggers. Score: 2 Notes: It works, but isn’t scalable enough to justify the heavy investment in managing and monitoring.

14. Unlimited Storage Promotion. Score: 2 Notes: Offering unlimited storage didn’t move the needle on leads, trials, or conversion rates.

15. B2B Whitepaper network. Score: 1 Notes: Mismanaged, poor lead flow, they refunded part of our payment.

16. Gmail Data Value Calculator (free tool). Score: 1 Notes: Too consumerish, poorly executed in that we didn’t make it shareable enough. We should have added a leaderboard and some other ideas to get a bit more from this. Traffic and PR performed OK, but not many leads.

17. On page SEO improvements. Score: 1 Notes: Search volume in our category, like most new markets, is low, so improving SEO is a low ROI activity in the near term.

18. Referral email campaign to current customers. Score: 1 Notes: This generally works well for small and mid-market customer acquisition, but without a built-in referral program, and given that Google Apps is rapidly growing but still minority market share product, it had limited effectiveness. We may return to this in 2014.

19. Upsell email campaign to Backupify for Personal Apps users. Score: 1 Notes: Limited overlap and limited success.

20. SEM/Keyword Search. Score: 1 Notes: This is expensive to setup and maintain for more than a few keywords, and given the low search volume in our market, it has limited impact. The other problem is that Google Apps customers span from 1 seat ($3/month revenue for us) to tens of thousands of seats (tens of thousands of dollars in revenue for us) and so SEM forces us to pay for tons of the smaller leads just to land the occasional large lead.

21. Salesforce Product Launch Announcement. Score: 1 Notes: Don’t do these during Dreamforce as you will get drowned out by all the other announcements.

22. Spiceworks promotion. Score: 1 Notes: Limited scalability and has to be actively managed. This will probably work well once Google Apps has a higher market penetration. However, if you sell an IT product, you should try it.

23. Offshore Teleprospecting. Score: 1 Notes: This call center that gets paid on trial installations would probably be great for most products, but they had a hard time finding the Google Apps admin once they reached an organization, and we require an admin to install Backupify on a Google Apps domain. Difficulty reaching an administrator is a common problem customer acquisition problem with add-on services like ours.

24. PR Stunts. Score: 0 Notes: We had a great idea for a public stunt, that I won’t mention, but was a massive fail. These are all or nothing propositions, so be careful how much you invest in them.

25. Affiliate program. Score: 0 Notes: We saw too much fraud on the affiliate network we tried, and it required much more policing than we anticipated. Not worth it given all the bad lead data.

26. Cold email campaign. Score: 0 Notes: No surprise here, but cold emailing a list of people when you are in a new market that requires an evangelical sales model has poor ROI.

27. SEO off page improvement. Score: 0 Notes: It turns out that we have a lot of great incoming links already, so hiring a consultant to help didn’t make a difference.

28. Gmail deletion stunt. Score: 0 Notes: I deleted my entire Gmail account (yes I really did) in honor of World Backup Day. This got no press pickup. I think the video straddled a bad line of too edited to seem off the cuff, and not edited enough to seem polished and professional. Or you could blame it on my lousy acting.

29. Tech-focused white paper program. Score: 0 Notes: This company turned out be a crappy content farm with no real value.

So there you have it: 29 of our marketing experiments for 2012. There were also three things that worked really well for us that I didn’t write about here, but may discuss at some point in the future. I hope you get some ideas from this as you think about your own customer validation process post product/market fit. Feel free to reach out to me, or anyone on the Backupify marketing team, if you have specific questions about one of these programs that you think might work for you. We love to help.