7 min read

The problem with engineering-led growth for early stage startups

The problem with engineering-led growth for early stage startups
Being “data-driven” does not mean running an experiment that will take 6 months to reach statistical significance (…at which point you’ve run out of money)

This post was originally published on February 7, 2022 for the Angular Ventures newsletter. Subscribe here to receive all new Angular blog posts, data reports, and newsletters directly to your inbox.

What I’ve always found frustrating about much of the writing about “growth” and growth teams is that there’s this implicit assumption that every company needs the same sort of team, no matter the company’s stage or product.

This mythical team is, of course, the engineering growth team. The mad scientists of Startup Land. Experiment-driven. Data-obsessed. Constantly iterating on new product features to create growth loops and drive adoption, activation and revenue.

There’s something alluring about this sort of team, especially for a founder thinking about how to hit increasingly aggressive growth targets, because their approach is experiment-driven. It feels like more science than art, which is oddly comforting when everything about your company’s future is so uncertain. If you hire the right growth people and let them loose, they’ll experiment away in their growth laboratory and your product will grow in front of your very eyes. Or so the legend goes.

This approach works for consumer products with millions of active users. And it can work for bottoms-up SaaS once the product has reached significant scale. But for the vast majority of early stage founders, this obsession with experiment-driven, engineering-led growth isn’t useful, and can often be actively harmful.

In reality, different growth teams, with different skill sets and profiles, are needed during each stage of a startup’s development. Engineering-led growth teams have their place, but if you’re thinking of hiring a growth PM to solve your growth problems and you only have a few hundred users…stop. And keep reading this post instead.

The founder-led growth team

Let’s start at the beginning.

You’re seeing anywhere from zero to a hundred sign ups per day. You have a few thousand weekly or monthly active users tops. You have no revenue, or you’ve maybe just started experimenting with monetization. You definitely haven’t reached product-market fit yet.

At this stage, growth is the job of the CEO. You can’t outsource it. Your goal is to find product-market fit as quickly as you can, and in so doing, create the foundation on which you can build a growth machine.

Enough has been written about the early days of starting a company and finding product-market fit. (Talk to your users, do things that don’t scale, etc.) I won’t rehash that here. But what I will note is that during this period you’re probably starting to get an intuition for which growth strategies may be most important for your company based on talking to your users every day. Do you think you’ll need to invest in building a community? Creating a content machine? Will paid advertising be important to your growth story? How about channel partners? Behind this intuition will be a list a hundred items long of ideas you wish you had the time to experiment with. Note these down.

The generalist-led growth team

You’re seeing hundreds of sign ups per day. Weekly or monthly active users are in the low to mid-four figures. Maybe you’ve started monetizing, maybe not, but you’re starting to see the glimmers of product-market fit, both through an increase in weekly active users and some organic growth through positive word-of-mouth. You have an intuition that certain growth channels might work really well for you, but you don’t have definitive evidence on any of them yet.

Unfortunately, all of a sudden you realize that you’re so busy hiring, selling, fundraising, and building the product, that you don’t have a spare moment to figure out which of these growth strategies will be most important for your business. You have too many ideas, and too little time. What you need is, well, another you.

This is when you need to hire what I call a “generalist” growth person.

The “generalist” growth team will be tasked with figuring out which growth strategies, whether top of the funnel or in the product, are most important to invest in. That means they may be tasked with writing content, drafting ad copy, managing Adwords or Facebook, buying ads on newsletters, onboarding users manually to identify issues with the onboarding or activation journey (and manually optimizing those flows), or anything else that helps them unlock growth for the company. For any of the above, if they don’t have the skills or knowledge themselves, they need to be comfortable contracting out to (and managing) somebody who does.

I’ve found that former founders are often the best fit for this sort of role, because it allows them to wear many hats, it gives them extreme autonomy and it ties their work directly to the most important metric in the company.

Just like you built an MVP of your product to find product-market fit, they’re building growth strategy MVPs to find product-channel fit. Once they find it, they’ll probably need to hand-off ownership to somebody focused exclusively on the channel. Sometimes somebody on the team might spin off and start that effort themselves (e.g. at Airtable, the first product marketer and the first growth marketer were originally members of the growth team), or somebody may need to be hired. Either way, if this team does its job, it should cease to exist within 2 years of being formed.

I’m unabashedly biased here because this was my job at Airtable, but the members of this team can be some of your most impactful early team members. They have deep context on the product, users and growth channels. They can go on to lead growth-related teams like product marketing, growth marketing, business operations, and growth product, or even customer-facing functions like customer success or implementation.

Sidebar: experimentation during the “generalist” phase

During this phase, you will likely get the urge to start experimenting with changes to the product. It’s natural! You’re talking to users every day, and you notice that there are some bleedingly obvious problems with the onboarding flow that you’d love to be able to fix. You’re convinced these changes would impact the activation rate.

Unfortunately, it’s likely that you don’t have enough users for most experiments to hit statistical significance in a reasonable amount of time. Here’s an example: let’s say you have a 2% upgrade rate from your free trial to a paid plan. You have an idea for a change that you think will increase the conversion rate by 20% (from 2% to 2.4%). You’ll need 20,000 users to see that experimental flow before you can detect that change with statistical significance. How many weeks will it take you to hit that number of users? Do you have the time to sit around and wait that long?

Given this reality, you have a few choices.

  1. Manual experimentation. Come up with ways to test your hypotheses manually. Let’s say you think building new data import functionality will improve onboarding success. You can’t test it, given user volume, and you don’t have the spare engineering capacity anyway. Can you test this hypothesis by importing data for a subset of your customers manually? You won’t reach statistical significance, but what might you learn?
  2. Trust your gut. Let’s say you have enough credibility with the product and engineering teams to get your ideas on the roadmap, but you’re wondering what to propose. Here’s the thing: you’ve spoken with thousands of users at this point. You’ve manually proven out a handful of hypotheses. You have great intuition at this point. Bring those customer stories to the product team and show them the way. Don’t waste time experimenting. Build the best possible user experience and keep going.
  3. Big changes only. If you have a bunch of changes you want to make, and you want to convince yourself that you’re creating a better or worse experience, consider combining them into one big launch. With a big launch, the expected relative shift in the outcome variable may be large enough that you can detect it, despite low user volume. Using the example from before, if the expected change in conversion rate was 50%, instead of 20%, you’d only need 2,900 users to see the variation.

The engineering-led growth team

At this point, many of the growth channels that the generalist growth team experimented with are now owned by the growth marketing, product marketing, community, sales or partnerships teams. What’s left is the product itself. And finally, the product is seeing enough sign ups and daily/weekly active users to provide fertile ground for feature-level experimentation.

You’re seeing thousands of sign ups per day. Weekly active users are above 50,000, at least. The onboarding flow, upgrade flow and other important growth surface areas (e.g. referrals) have been built, but not optimized. You long ago started monetizing, and revenue is pouring in, but you know that the experience has not been tuned or optimized. You get the sense that there’s low-hanging fruit everywhere, but you don’t know how to, or have the time to, prioritize it all.

Finally, the time has come to bring in the mad scientists. Once you’ve hired a product or engineering growth leader, my former colleague Darius Contractor recommends starting with a “Baby Growth” team. This team should ideally be made up of 2–4 existing team members who are analytical, business-focused and understand the users and product extremely well. Ideally, you’re able to pull together a product manager, front-end engineer, back-end engineer, and a data scientist.

Give them a focus. Darius suggests asking them to “start with a metric everyone cares about and a surface no one cares about.” Often this is the pricing page, or maybe the upgrade flow. This ensures that they don’t compete needlessly with existing engineering teams, but can also have an immediate and visible impact.

To set them up for success, ensure they have the data and analytics infrastructure required for identifying areas of the product ripe for experimentation and running experiments. (Or provide them with the resources they need to build this infrastructure themselves).

The most successful engineering growth teams I’ve seen are maniacally focused and incredibly structured. This is a process management system that Darius built to run product growth at Dropbox, Facebook and then Airtable, which is a good place to start. (You can hear more tactical suggestions from Darius during a recent Angular Insights session here).

And finally, as the team grows, you can create different pods to own different surface areas (e.g. onboarding flow, upgrade flow, referral flow), each with their own core metrics.

As you can see, as a company evolves, the job of the “growth” team changes drastically. Growth teams are hard enough to manage because they are meant to constantly move fast and change focus. At the very least, you can help yourself by knowing what kind of growth team you need to hire depending on the stage of your company.