Meaningful metrics for incubators and accelerators

Editor’s Note: This is cross posted from WhoYouCallingAJesse.com by Jesse Rodgers, who is a cofounder of TribeHR. He has been a key member of the Waterloo startup community hosting StartupCampWaterloo and other events to bring together and engage local entrepreneurs. Follow him on Twitter @jrodgers or WhoYouCallingAJesse.com.

CC-BY-20 Some rights reserved by Nathan E Photography
Attribution Some rights reserved by Nathan E Photography

Incubators and accelerators are businesses just like the businesses they intend to help develop as they travel through the startup lifecycle. As with any business, there are indicators that they can measure to give them a better idea of how they are performing besides the big public relations buzz around a company being funded.

You need to measure these numbers so that when a success happens you can hopefully gain some insights on how to help the other companies better. The problem is that even though the model of an incubator or accelerator is generally known, how to take 10 companies and have 10 successful growth companies come out the other side of the program is not.

The issue of what metrics to use is an important but complicated problem to solve.

Set the baseline at the application process (pre-program)

There are far more applicants than slots offered in an incubator or accelerator program. However, it is at this point that a program is gathering it’s best intelligence. You need a baseline measurement at the start of the program that you can measure every team against. What you should be tracking:

  • Who applied to the program that you didn’taccept (this is your control sample)
    • Track their progress on Angellist, Crunchbase, and/or go back to their web site in 3, 6, 12 months.
    • Keep a ratio of who is still in business and what their status is.
  • Maintain, in a CRM system, information on the applicant founders and their team members.

Measure the incubator/accelerator clients (in-program)

At this point there are X number of startups with Y number of founders and maybe Z employees. What you want to measure are things that demonstrate they have improved (or not) and which are things you would expect to see improve as a result of the services provided by any incubator or accelerator:

  • Current customers and revenue per customer (for most that will be 0 at the start) that will work across revenue models: CAC, ARPU, churn rate.
  • Sales funnel – do they have leads? How many? Are they qualified leads? What are they worth?
  • Average user growth in the last month.
  • What mentors or advisors did they meet through the program? What role did they take with the company?

Run these numbers at the start and at the end of the program. If you are a pure research focused incubator, ignore this section. You have a much longer time to see success – but few are truly research focused.

Monitor the graduates: Alumni (post-program)

This is a very important thing an incubator/accelerator can do — build and maintain its alumni connections. These folks not only help at every stage of running future programs but their success lifts the profile of the program, just like how alumni of prestigious business schools make the business schools prestigious.

There should be reporting milestones at a set interval (probably financial quarter based) where you gain the following insights on the company:

  • Customer growth percentage: CAC, ARPU, and churn rate all expressed as percentage growth.
  • Sales funnel growth expressed as a percentage.
  • Average user growth in the last month.
  • What mentors or advisors are currently active with the company?

Ideally you should have a position that is equivalent to a close advisor or board observer with the company once it graduates from the program.

Defining success

If an incubator or accelerator program is successful, the graphs should be heading up and to the right at a much faster pace than they would have been had startups not entered the program.

The only baseline data I know of is from the Startup Genome. In their report they explain the stages and the average length of time it takes a company to go through them. For an incubator or accelerator to demonstrate that they work, I would expect a successful company to move through the stages faster than the average. I would also expect them to fail faster than the average.

Tracking metrics puts a lot more overhead on an accelerator. It is likely more than they budgeted for to start. However, if you want to know if the program is successful it is worth the investment of an admin salary to track and crunch data. This is just a baseline, track more and figure out what the indicators of success are for you.

There are two types of startup incubators in the world: YCombinator or TechStars

Note: This is a guest post by Jesse Rodgers who is a cofounder of TribeHR. Jesse specializes in product design, web application development and emerging web technologies in higher education. He has been a key member of the Waterloo startup community hosting StartupCampWaterloo and other events to bring together and engage local entrepreneurs. Follow him on Twitter @jrodgers or WhoYouCallingAJesse.com.

CC-BY-SA  Some rights reserved by µµ
AttributionShare Alike Some rights reserved by µµ

Incubators and accelerators [Eds note: and cyclotrons] have but one purpose: move startups along in their life cycle at a faster pace than they would normally and increase the likelihood of a return by providing that service. If you are a startup looking at applying to an incubator you need to understand that the differences in how these programs differ go beyond the money they give you in exchange for equity.

An oversimplification of the incubator/accelerator space is to classify them as either a Y-Combinator (YC) or a TechStars (TS). If you really look at the booming world of incubators for high tech startups you see a model that either based on education and peers that is driven by a strong personality (YC) or a model that is more institutional, follows a script, and feels less personal but is more in line with how VC’s work (TS) (I would place 500 Startups right in the middle between YC & TS which is arguably representative of a third type). There is plenty to be found about the differences but here is a bit of a deeper exploration into the differences.

Startup lifecycle

Startups have a number of key phases in development that is best outlined in Fred Destin’s presentation on startup lifecycle.

  1. Start
  2. Launch
  3. Build
  4. Chasm
  5. Scale

With the 12-14 week cohort models, like YCombinator and TechStars, the focus should be on moving through starting and on to launch phase. There may be some that get into a build phase. The incubator or accelerator hopes that once they are done a 12-14 week program the startup will be in a much better position to move quickly through the build stage and at least take on the chasm phase.

Where I see the key difference between YC and TS is that YC seems to be able to get companies to go through stage 1 to 3 and they accept companies mainly in the start phase. TS seems to not attract a cluster of companies in a particular phase or not care about what phase a company is in.

The basics of an incubator/accelerator (whatever you want to call it)

Within the execution of any incubator or accelerator program there are, in my mind, 4 core stages in a typical cycle:

  • Recruitment
  • Onboarding
  • In the program
  • After the program

Within each of these of these stages there are a number of specific activities that all incubators do but in general they aren’t all that different.

Recruitment

YC currently leads the thought leadership with Hacker News, Paul Graham’s (PG) blog, and it’s success. Applicants fill out a form and once told they have an interview, travel to YC in Mountain View for an interview. They get just 15 min with a small panel and the panel does a bunch of tricks to the founders like carrying on side conversations – there are a lot of blog posts about that.

TechStars has adopted a more consistent process over it’s many affiliated programs (it appears) but they lack YC’s Hacker News or thought leadership (although they would claim otherwise). With Techstars there appears to be an affiliation with the Kauffman Foundation and the role they are taking in promoting the incubator model in general they have made themselves an authority in the space. From people I know that have been in the program it is a fairly standard process similar to raising Angel capital.

Onboarding

I am not sure on TS on-boarding but YC has a very short interview to decision to start of program window. YC has a little book that is like a long Wikipedia article written by Paul Graham that offers insights and baseline knowledge. From what I have been told the YC machine is pretty much immediately available to you when they say “you are in” — startups decide when to tell others. What is really interesting is that YC doesn’t announce it. They generally let a company know they are YC funded on the interview day but they don’t make a big announcement or anything.
Not having a big incubator announcement is a key difference here though. I will assume that with TS it is just like YC in that they have decided to fund you, they are now available to you. However, TechStars (it appears) doesn’t approach announcing the cohort in the same way as YC — they announce them ahead of the program.

In the program: peer mentorship, startup culture

Each program runs for roughly 3 months, 12-14 weeks, where mentorship, various events, and a demo day to close it off normally occur. Each week is important given that each team only has 3 months. Over three months there are phases you can generally identify:

  • Teams becoming familiar with each other, their mentors, and what they need to do (first 2 weeks).
  • The heads down getting stuff done phase (8-10 weeks).
  • Funding mode going into Demo Day (2 weeks).

Other incubator programs are fairly similar with any given week involving office hours (optional or required) and a speaker/dinner. The office hours are used to check in and place goals on the teams. Throughout the term there are demo nights, which are used by YC as a way to put peer pressure on other teams that might not be moving as fast as others.
Where they differ here is in the education of the founder(s). From everyone I have talked to that has gone through YC it seems to me it is a very challenging but rewarding relationship for a certain type of founder. That would make sense as a certain personality type will work best with Paul Graham’s way of doing things and will excel. I am not entirely sure it is simply a hacker/coder persona as most assume. I think it is a personality and learning style that goes a bit deeper.
TechStars has a co-working model with parts very similar to YC. The key difference is that TS doesn’t have the Paul Graham approach to educating founders so you will get very different details depending on who is running the program. TS also gives the startups a place to work where YC leaves them to find a house and work out of it.

After the program: Alumni network

The key value any incubator or accelerator provides after the program is the alumni network of companies that are now a few steps ahead (depending on the age of the incubator there could be alumni with very large companies) of the current cohort in the program. Over time these alumni are your best mentors and connectors.

It is at this phase where the greatest value is for the startup, I believe. You now have access to what the old folks call a big rolodex (social graph) that will open many doors which essentially leaves it up to the entrepreneur whether their company will succeed or not. There are few to no barriers, generally speaking.

Any alumni of YC or Techstars still have contact with the folks in their cohort and all cohorts along with Hacker News. Techstars Network is so big they have a conference just for alumni while YC taps its alumni for all kinds of things. Also, founders seems to find going through the program a second time is different but just as valuable. These massive networks of successful alumni with a flock of high profile admirers is very similar to that of Higher Education alumni networks, so much so it convinces me that this entire process is a form of higher education.

Programs that work copy YCombinator, even TechStars did

CC-BY-NC Some rights reserved by andi.vs.zf
AttributionNo Derivative Works Some rights reserved by andi.vs.zf

The current culture of education focused incubators started in my mind with YCombinator (started in 2005). I believe what we are seeing with the success of YC and TS is new take on graduate school. Both are different, both work, and people can have strong opinions either way. They feed a need that I don’t think people outside of incubators quite understand yet, learning to be a founder is really hard. Being a successful founder is even harder. The bet is that if you help young founders focus on what is important they will see success earlier or just simply see what success looks like.
If you are looking at an incubator anywhere (there are lots of great programs out there) you need to understand that the money is secondary. You need to find a program that will fit with the way you learn and has companies that you want to work with. It is just like how you picked your University or College except this time it can cost you a lot more (in equity) if you are successful.

Note: This is a guest post by Jesse Rodgers who is a cofounder of TribeHR. Jesse specializes in product design, web application development and emerging web technologies in higher education. He has been a key member of the Waterloo startup community hosting StartupCampWaterloo and other events to bring together and engage local entrepreneurs. Follow him on Twitter @jrodgers or WhoYouCallingAJesse.com.

Extreme Startups

Extreme Startups

Rob Lewis and TechVibes is reporting that ExtremeU (you can read our past coverage 2009, 2010, 2011) has launched a new Toronto based incubator that leverages their experience over the past 3 years. Mark Evans provides additional details that includes “$7-million in funding from Extreme Venture PartnersOMERS VenturesRho Canada VenturesBlackBerry Partners Fund and BDC.”

Extreme Startups includes a who’s who of  the Toronto startup scene as mentors:

  • David Ossip
  • Daniel Debow
  • Anand Agarwala
  • Michael McDermentt
  • Ameet Shah
  • Albert Lai
  • Leila Boujnane
  • Ali Asaria
  • Noah Godfrey
  • Ray Reddy
  • Rick Segal
  • Salim Teja
  • Derek Seto
  • Nick Koudas

Congrats to Andy Yang, Sunil Sharma and Amar Varma in getting this thing launched. Plus how can this not be awesome with Andy Yang as Harold and Sunil Sharma as Kumar in Extreme Startupping.

Andy Yang and Sunil Sharma go EXTREME STARTUPPING