1. 13:18 10th Jul 2014

    Notes: 2

    Customer churn can kill your startup

    Startups are about growth, and of all the different possible metrics, startups often focus on user growth - if people aren’t using your new product or service in greater and greater numbers, it’s a good sign that you’re not on the right track. And, as long as your business-model is solid, user growth is a leading indicator of revenue and profit growth.

    But there’s another key metric that can be the making of your company, or the death of it - churn, or the proportion of existing customers who become ex-customers during a period of time (generally a year).

    In the early days, it’s very hard to measure your churn because you simply haven’t been around long enough. The early revenue growth curves for high- and low-churn companies may look identical. 

    In fact, high-churn businesses often have exceptionally impressive growth numbers in their first months or years because they’ve found some viral loop, the crack cocaine of startup-land, which provides an explosive catalyst for new-user acquisition.

    However, problems often start to appear after the first year or so - occasional weak months look exceptionally bad, since your business model relies on new customers in the top of the funnel to replace those flooding out. And even if you manage to keep finding new people to push into the top of your funnel for several years, you will eventually churn through your entire addressable market.

    Then your business implodes.

    The prime examples are Zynga and Groupon - both of which have been catastrophically failing over the last year or two.

    Conversely, low-churn businesses often have relatively modest initial user-acquisition growth curves. Often, the nature of low-churn businesses (in particular, high switching costs) makes it incredibly hard to gain your first few users before you’ve established credibility. However, these same hurdles present considerable long-term advantages, and the occasional slow month in terms of user-acquisition still increases monthly revenue. Once you’ve built up a loyal customer base, it becomes an extremely attractive business model - costs are often fixed & up-front, and the marginal cost to serve each customer every month is minimal.

    When talking to startups, my favourite graph to look at is their revenue by monthly cohort. That is, splitting each months’ revenue by the month in which the users originally signed up to the service.

    In a low-churn business, the graph looks like a layer-cake, with each new month of users adding a layer of revenue on the top of the business.

    image

    The thin cohorts at the bottom of the graph are the loyal users who signed up at the start, and who are still paying for your service 18 months later. The orange tip in the top-right are the users who just signed up, and have only contributed to revenue in month 18.

    In the best low-churn businesses, the revenue provided by each cohort can actually grow over time - each cohort gets “fatter” - if you can successfully up-sell new services.

    In a high-churn business, the cohorts decay alarmingly quickly, and once new-user acquisition starts to slow, the business implodes.

    image

    If you simply looked at the top-line of the graph (total monthly revenue), without considering the cohorts, you’d think the high-churn business was doing much better until at least month 13-14. But one-and-a-half years in, it’s clear that the low-churn business is on a path to success, while the high-churn business needs to execute a substantial pivot to avoid failure.

    Indicators of Churn

    So what are the characteristics of high- and low-churn businesses, and what techniques can you use to minimise churn?

    High Switching Costs - if it’s slow, painful & costly to switch providers, you will experience very low customer churn. This is clearly the case with retail banks, email providers and recurring payment processors. I still bank with the same provider I chose 15 years ago, despite their crappy service and barely-functional website.

    New competitors have developed clever tactics to solve this problem - developing onboarding tools to reduce switching costs, or focussing on new customers before they’ve committed to an incumbent provider. This is one reason Stripe focusses so hard on serving other startups, and banks target teenagers.

    Sales Architecture - the way a product is priced & sold is often designed to minimise churn. Gyms & dating sites are the best examples here. People come out of the Christmas & New Year vacation period feeling unhealthy, lonely, and determined to change. One large dating site books over 70% of its annual revenue in the first quarter of the year. 

    Strong Brand Loyalty / Personality - the best brands position themselves as expressions of their customers’ personality.

    Cigarette companies and beverage manufacturers are the best example of this - products are often indistinguishable from each other, and the switching costs are minimal. The customer can just reach for a different can on the shelf. Companies combat this by investing heavily in marketing to position their products in customers’ mind as extensions of themselves.

    There’s an interesting phenomenon in the drinks industry where a couple of faceless mega-corps control a wide variety of incredibly powerful brands with basically indistinguishable underlying products - MillerCoors and Anheuser Busch. 

     
    Make Something People Love
     - after 
    some fantastic comments from readers, I’ve added this extra point. Perhaps the most difficult, but also most effective form of customer retention is making a product or service that is dramatically superior to the competition. As Barry put it, “The switching cost is missing out on a great service.” It’s subtly different from brand loyalty - Google has very low churn because its search engine is so much better than the competition. It arguably doesn’t have a very strong brand or “personality”.

    So What?

    To some extent, you’re stuck with the churn inherent in your industry. But you do have some level of control.

    If you enter a market like payment processing, be prepared for the long-haul. You’ll likely struggle to establish your credibility early-on, and finding innovative ways to acquire new customers be vital. You’ll need to raise significant amounts of early capital to invest in infrastructure that will only start to pay off years later.

    If you’re part of an industry in which you can acquire customers incredibly cheaply and monetise them quickly, you’ll generally need less early capital. You’ll often reach profitability very early on, and it may feel like you’re on a startup rocket-ship. But make sure you’re providing sustainable, long-term value to your customers, otherwise you risk running out of fuel and crashing painfully back to earth.

    Discuss this post on Hacker News

    Download the spreadsheet used to create these charts

    Update: I’ve got a script that will auto-generate these charts from your Stripe.com data. If you’d like to see your own churn graph, email me at tomblomfield@gmail.com for details. I’ll release the tool publicly once I’ve had a chance to test it some more.

     
  2. 17:33 29th Mar 2014

    Notes: 2

    Growth

    "When eating an elephant, take one bite at a time."

    - Creighton Abrams

    I was chatting with an investor a year or two ago, shortly after our Series A, and the subject of revenue goals came up.

    At the time, our revenue was pretty low - less than 10% of monthly costs. We estimated our business would only become viable when revenue reached a level 10 times greater than our current costs.

    The implication was that we needed grow by 100 X before our business had any significance at all, and I was struggling to come up with any credible plan for getting there.

    Her response was for us to simply put one foot in front of the other. Just focus on growing every month, and the rest will take care of itself.

    At 15% month-over-month growth, you’d grow by 100 X in around 33 months.

    At 20% month-over-month, it takes 25 months.

    At 25% month-over-month, it takes 21 months.

     
  3. Startup Series Part 4: Deadlines

    image

    The question of deadlines is one of the most divisive issues in software development today. Business-people despair at developers’ inability to meet goals, while developers lament the latest unachievable deadline imposed by their pointy-haired boss.

    In practice, there is often a common cause of this malaise. If your team doesn’t have an agreed scope with sufficiently granular time-estimates for each component piece of work, your project will inevitably overrun, and sometimes by an order of magnitude.

    Scoping a project

    The solution is to come up with a scoped deadline - a date in the future by which you will have implemented an agreed list of features or functionality. Crucially, each of the features needs an estimate of the amount of time or effort required to complete it, similar to some styles of Agile software development.

    The only effective way to estimate effort is to break the project down into chunks of work that can individually be estimated. In no case should any chunk be more than 2 developer-days, and ideally they’d be around ½ a day each (4 hours). There are lots of methods for actually coming up with these numbers - Planning poker is one neat collaborative solution, or Joel Spolsky’s Evidence-Based Estimation is also interesting.

    Fundamentally, developers should be the people saying how long a given chunk of work will take. Whatever system you use, it’s essential to have a relationship where developers feel like their judgment is respected and the business-side of the company trusts developers to put in a reasonable amount of effort. At the end of the day, even the best estimation techniques can’t fix a broken culture.

    The alternative, an “it’s done when it’s done” attitude, is dangerously counterproductive - if there’s no pressure to hit business goals, technical decisions will tend towards feature-creep and perfectionism.

    Why scoped-deadlines are useful

    As far as I can see, there are 3 main benefits of scoped deadlines:

    It makes you iterative - forcing everyone involved to ask “is this feature really necessary?”. You will always have more development work than time available, so choosing what to work on is a crucial skill. If you strive to keep your deadlines around 2 weeks, you will prototype very quickly, building only the features you really need. Half the time, customer feedback will change what you saw as “version 2” anyway.

    Second, it prevents scope creep during the project, as you’ve locked-in the features that are going to be delivered. This is good for both developers and business-people. For the former, it encourages a  pragmatic approach to the quality-vs-speed tradeoff - personally, it helps me check the urge to refactor code when it isn’t strictly necessary. For the latter, it provides a positive response to the perennial request of “can you just implement XYZ for me?”. Sure - in the next version.

    Third, it can sometimes good for motivation & productivity, for some combination of people, teams & problems. I find deadlines energising - crossing to-do items of a list, and delivering impressive new feature motivates me. It’s a very tangible way for a team to see that they’re making progress, and people love being part of success. But it’s certainly the least important of the three.

    Deadlines in practice

    As a project progresses, you should be regularly deploying your code - use a feature-switch to keep new work visible to only staff or small segments of your user base. This is very important because it’s a leading indicator of missed deadlines. If you’re halfway through your “sprint”, and no code has been deployed, you’re likely going to overrun. It also helps catch bugs early, and makes you consider how your code will interact with the existing codebase from the very start.

    Even with this advice, you sometimes just won’t hit your deadlines. It might turn out that the version on open-source library you’re using isn’t compatible with a new tool you want to implement. Unexpected problems crop up. While I don’t think it’s a terrible thing to work a couple of late nights if your project needs to get back on track, it’s very different if the scope of the project has fundamentally changed.

    Retrospectives are a good way to identify any patterns in scoping problems - if you highlight areas that consistently overran, you’ll get better at spotting them in future.

    On the other hand, be quick to celebrate your successes. Releasing code from behind a feature-switch to your general user-base is a great time to gather the team and reflect on your achievements.

    Discuss this post over at Hacker News

     
  4. 13:33 26th Sep 2013

    Notes: 8

    Startup Series Part 3: You make what you measure

    Merely measuring something has an uncanny tendency to improve it. If you want to make your user numbers go up, put a big piece of paper on your wall and every day plot the number of users. You’ll be delighted when it goes up and disappointed when it goes down. Pretty soon you’ll start noticing what makes the number go up, and you’ll start to do more of that. Corollary: be careful what you measure.

    - Paul Graham

    Metrics are one of the most powerful and most dangerous tools at a startup’s disposal. They can help focus a team around a common goal or push you off-course for months.

    Start by tracking one thing

    We realised pretty early at GoCardless that we needed to track something to work out if we were improving. As we are a payments processor, the first metric we chose was our total daily transaction value - how much money was going through the system. We chose this metric because, in the very early days, total daily transaction fees (ie our revenue) was embarrassingly low, whereas our total transactions were roughly 100x higher. This worked well for a few months - we hit milestones in encouragingly quick succession, moving from £1k-a-month to £1m-a-month to £1m-a-day. We’d display the daily number on a large board in the office, and send weekly and monthly summaries to everyone in the company, with the change over the previous period. It worked well both to motivate the team and act as a Pole Star when making decisions; “which of these options will maximise our monthly transaction value?”. When we hit 10% week-on-week growth, we were happy. When we didn’t, we redoubled our efforts.

    Choose a proxy for long-term value

    However, a year or so down the line, we realised that we’d encouraged some pretty perverse behaviour. A large minority of our transaction value was coming from a few merchants putting through very high-value transactions. Because of a quirk in our pricing model (1% capped at £2 per transaction), we were actually making relatively little revenue and assuming way more risk than we’d like on these customers. For a short time, we were actively pursuing more of these customers in order to boost transaction value. Yet, when we came to search for a solution, it was incredible how ingrained our hunt for transaction volume had become - any change we made to combat the problem would inevitably result in lower transaction values, although dramatically higher revenue - and yet, emotionally, we struggled to make the change. Once we swapped the key metric to daily transaction fees, the problem went away.

    Since it sometimes seems fashionable to run companies without revenue, you need to pick the best proxy for long-term value of your company, be that profit, revenue, installs, MAU or page-views. But this raises the next problem we encountered. Even if you could track company value on a day-to-day basis (and you can if you’ve gone public), it’s not an actionable metric. This is usually a problem for B2B companies since the enterprise sales cycle can often be several months - the work you do today won’t impact revenue for some time.

    Break out KPIs

    One solution is to have each team within the company break down that key number into its component drivers until you get to something actionable. A really neat example is an outbound telesales team. Ultimately, you want your sales team to be closing recurring deals, which will drive revenue in a predictable fashion. But, as any sales director will tell you, if you’re not closing the deals, you need to focus on the activities you’re performing. The stage before a closed deal might be some form of proposal, outlining the key SLAs you will deliver in return for committed fees. To deliver 5 closed deals this month, you might need to make 10 proposals. To make those 10 proposals, you need to attend pitch meetings with 20 companies. To get those pitch meetings, you’ll need to have spoken to 200 companies. Since only 1-in-3 cold-calls results in a conversation with a decision-maker, your telesales teams need to call 600 companies this month. Suddenly, you have some pretty actionable KPIs for your sales team. These are the numbers that you track in daily stand-ups, and measure conversion between each stage. Since they are actionable metrics, the entire process becomes an optimisation problem.

    As far as possible, each part of your business should have 2 or 3 actionable metrics that they examine on a daily or weekly basis. They’re the numbers that tell you whether you’re improving or not. An example might be website up-time for the devops team, or average customer wait-time for your customer support team. Whatever numbers you choose need to be objective - if people can massage up a bad month, it undermines everyone’s faith in the metric. These metrics shouldn’t be sticks to beat people with - it’s actually incredibly motivating to work very hard and see that you’re making progress. They’re also useful to highlight strategies or areas of the business that just aren’t working out, rather than allocating personal blame.

    Eric Ries spends a lot of time in the Lean Startup talking about actionable vs vanity metrics - it’s well worth a read.

    Implement your strategy with KPIs

    The choice of KPIs is also a very powerful tool for a founder or management team to set the direction of a company. There are many different ways to reach £1m in monthly revenue - the way you set up your KPIs and associated targets will have a huge influence over the behaviour of people in your company. As a simple example, are you aiming for 10,000 customers to pay you £100 each, or 100 who’ll pay £10,000 each?

    When metrics go wrong

    Even these actionable KPIs can produce perverse behaviour. I worked at a large national utilities supplier whose (very large) email customer service team committed to responding to 90% of emails within 24 hours. On the face of it, a great actionable metric - and the team were consistently replying to 89 or 90% of emails within the allotted time. But they’d also developed a practice of simply ignoring any email which exceed the 24-hour cutoff. Since these emails didn’t impact their targets (or their bonus), 10% of customers simply never got a response.

    A final issue is that, in some areas of your business, KPIs can be really tough or even impossible to pin down. While a front-end team working on your landing pages might measure conversion to signup, a PR team running a 6-month campaign may struggle to find an objective indicator of success. You can try to count the number of articles (and the readership of each), but you’d also need to assess the relevancy of the target audience. You might try to establish a baseline of page-views and count increases or spikes, but there’s so much noise that it quickly becomes subjective. While people have tried to to pin down the value of a “like” or a “re-tweet”, it’s only usually relevant for consumer-facing companies. We’ve never really managed to find a solution to this problem - I’d love to hear what other companies have done.

    In summary

    • Pick one number that’s your “Pole star” - the best proxy for long-term company value. Re-examine it periodically to see if it’s producing perverse outcomes.
    • Have each team break down the key metric into its component drivers or KPIs that individuals can influence. Measure team & individual performance & improvement on this basis.
    • Make sure the numbers are objective.
    • Make them visible - put the key metric on a big dashboard, and give each team a dashboard that lets them see how they’re doing. Send regular automated emails around your teams with a breakdown of their numbers.

    Follow the discussion over on Hacker News

     
  5. 10:20 22nd Sep 2013

    Notes: 1

    Startup Series Part 2: Attracting great engineers

    This is the second in a series of articles on the operational side of running startup - once you’ve found product-market fit and raised VC cash, how do you go about building a business?

    The first post in this series, on the hiring process, assumes you have a sufficient number of quality applicants. Once that’s true, hiring becomes a process optimisation problem - it might take a bit of effort to get right, but there aren’t too many unknowns. On the other hand, attracting top-quality developers to your company in the first place is a much tougher proposition. 

    In the long-term, you will attract outstanding talent if you; 

    • build a successful company with a strong vision
    • working on interesting technical challenges
    • with other outstanding developers
    • who value software development as a professional trade. 

    The often-omitted final step is to tell people about it! You could have the best culture in the world, but you won’t attract applicants if no-one knows about you. 

    Github is one of the best in the business when it comes to attracting great talent. People like Zach Holman and Tom Preston-Warner blog profusely and tour the world talking about software development. But, underlying every article and presentation is this message; Github is an awesome place to work.

    Since not every company can spend hundreds of thousands a year flying people around the world for conferences, or fit their reception area out as a replica of the Oval OfficeI wanted to share a few actionable tips for the first-time founder. These shorter-term fixes will provide value disproportionate to their cost. There are also a few gotchas - common practices that do more harm than good.

    Provide top-quality equipment

    Dollar-for-dollar, high-quality equipment was consistently the most sought-after perk when we’ve surveyed developers, above a swanky office, fancy snacks, and even salary. It makes sense to invest here.

    Some simple maths backs this up - depending on experience, you might pay anywhere from £30,000 to £70,000+ for a software developer in somewhere like London. Upgrading from a standard MacBook air to top-of the range model plus Thunderbolt display will cost you an additional £1,500. You’d need to achieve a 1.9% productivity gain (based on a £40k salary and a 2-year equipment lifetime) to pay back the investment. If this equipment reduces the running time of your test-suite by two minutes, you’d save about 4 days a year[1]  which, alone, is worth about 1.8% productivity boost.

    [1] 4 days = 2 minutes * 5 runs a day * 225 working days per year

    It then costs, conservatively, £5,000 to attract and hire a single developer. If top-quality equipment has any impact on hiring or retention, you’ve suddenly made a profitable investment. You also get to take lots of cool photos of your working environment and post them all over twitter and your hiring page - remember that telling people is the most important part.

    image

     

    Interesting observation; while we still let people choose their equipment, we strongly encourage people to choose between the MacBook Pro and Air. We’ve had several people originally opt for various flavours of Linux, but they’ve all swapped over to Macs within 12 months. The homogeneity means that equipment will work anywhere in the office - swapping desks to work with a new project team simply means plugging into a new Thunderbolt, rather than lugging tonnes of equipment around. It also means that useful internal tools & scripts that we’ve developed are more likely to work out-of-the-box.

    Work on awesome side-projects

    We’ve found that having the engineering team take a couple of days off work to hack on a fun side-project can be a great recruitment tool. Not only does it increase morale and let you try out new technologies, it also provides you with great content for blogging & conference talks.

    The best example is our pool-ball tracker we built at GoCardless. We mounted a webcam above our office pool table, and used OpenCV libraries to track the position of the balls. The ball coordinates were fed into a rules engine which then scored the game. It got a great reception on Hacker News and elsewhere, and resulted in an extra 9,000 unique visitors to our site on a single day. And it was awesome fun.

    Other examples are our GoCardless-branded beer and Raspberry Pi projects. 

    Our only failure here was a Spotify-powered office music system - we just never got around to blogging about it. As with everything on this list, it’s only effective if you tell people about it.

    Technical & Culture Blogging

    This is the single greatest recruitment tool we’ve found. If you’re using any kind of technology at all, or have views on development best-practices and startup culture, you should be able to write compelling content that will do well on places like Hacker News and Reddit. It’s also more more repeatable and has a higher ROI than side-projects, because it doesn’t require half-a-dozen people to take multiple days off work.

    For example, one particularly incendiary article on ditching Responsive Web Design garnered 34,000 unique visitors on a single day. We’ve had several applicants explicitly mention this article during interviews, even 9 months down the line.

    You can blog about pretty much anything you work on - meetings, org structure, a new javascript framework you’re using. Explaining how you’re different from just another BigCo is an extremely useful recruitment tool.

    Often, the key to getting traction on somewhere like Hacker News is choosing a punchy title and taking a slightly more extreme position (See "Hours are Bullshit") that you might otherwise do. Interesting technical posts do well, too.

    From a practical point of view, make sure you host the blog on your company’s domain (SEO benefits), mention your company’s name and link to your hiring page in an inoffensive way.

    Open-source your tools

    As a technology startup, you’ll be successful if you’re closer to your market & customers than your competitors, and you’re able to respond more quickly to their needs. As such, giving away tools that make you more productive initially seems counter-intuitive, but it’s not. Depending on your market, your competitors are likely to be slow-moving incumbents. If they’re still struggling with Visual Basic .Net and SOAP APIs, releasing a new Ruby deployment tool isn’t going to help them much. 

    Even if that’s not the case, open-sourcing useful little tools will dramatically help your recruitment processes - Github are the authors of the well-known Ruby tool Resque, for example.

    At GoCardless, we’ve not been as good as we’d like at open-sourcing our tools, but that’s changing. We’ve just released Hutch, a Ruby tool for RabbitMQ, and some development widgets for Dashing. Next on the list is a neat little tool written in Go that makes it simple to spin up multiple services in a local development environment.

    When you open-source your tools, don’t forget to tell people about it! At the very least, it’s a nice thing to do for the ecosystem.

    Speak at meetups & conferences

    This is really an extension of the previous three points - take all the great content you’ve produced from your fun side-projects and open-source contributions, and package them up into neat 20-minute presentations with some interesting slides.

    It’s great for your engineers’ personal development, and it really helps spread your name around the relevant communities. You don’t need to spend a lot of money on flights around the world, either. Many cities have very vibrant user-groups which hold meetups every few weeks - there’s an Angular JS group in 50 cities around the world, for example.

    Encourage all of your engineers to attend - and make sure you’re wearing your company t-shirts! It’s a great opportunity to talk to other like-minded developers about the interesting technology you’re using.

     

    Finally…

    These are five pretty simple things you can do to increase the number of quality applicants. The most important factor is simply to get your name out into whichever community you’re trying to recruit from. If you spend a couple of weeks building a cool new tool, don’t forget to spend the final hour or two writing a blog post about it.

    I’d love to hear about other tips people have - you can follow the discussion over at Hacker News

     
  6. Startup Series Part 1: Interviewing Engineers

    There’s been a huge volume of literature written on the subject of startup methodologies and philosophies. This is a positive thing for the ecosystem - just as new software tools mean it’s simpler and cheaper to start a company, these methodologies allow first-time entrepreneurs to avoid many of the pitfalls of the past.

    But I’d like to write a more practical guide - what happens once you’ve launched your MVP, iterated to a product that people actually want, and raised VC money to scale? What are the day-to-day challenges you’ll face and pitfalls you’ll encounter? Over the last 3 years at GoCardless, we’ve run into and subsequently dug ourselves out of many of these pitfalls. By making these mistakes, we’ve developed solutions that, at times, have dramatically increased our productivity.


    Startup Series Part 1: Interviewing Engineers

    The hiring process at technology companies has been the subject of a great number of blog posts, often written from the perspective of disgruntled candidates. They rail against what they see as capricious and ineffective techniques. This problem will probably exist as long as interviews produce unsuccessful candidates, but it is possible to design & run a process that will annoy a very minimal number of people, whilst selecting the best of the bunch to work at your company.

    Once you have a good number of capable people applying to work at your company (a much harder problem!), selecting the best of them should not be a terribly difficult problem.

    Over the last 3 years, we’ve seen around a thousand CVs, conducted hundreds of interviews, reviewed several dozen coding challenges and ultimately made 15 technical hires. As a result, we identified a number pitfalls in the interviewing and selection process. Usually we did this by stumbling into them head-first.

     

    Define a process and stick to it

    If you’re not absolutely clear on what each step of the interview process involves, and what you’re looking for at each stage, you’ll inevitably resort to weak, fluffy questions that don’t really reveal anything about candidate. If you throw a couple of your developers into an interview with no preparation, you’ve wasted 2 hours of engineering time at best. At worst, you’ve alienated the candidate and irritated your developers.

    Instead, invest a little time defining what systems and processes you’ll use. Write down this process and share it with your team and candidates. You don’t need to invent this from scratch - choose someone else’s and adapt it to your needs.

    Get an applicant-tracking system that stores CVs and the result of each interview (with detailed notes), so that you can compare candidates based on data. If you see a dozen developers in a week day, you’re unlikely to remember each of them by Friday.

    Our process was pretty simple: CV screen, 15-minute phone screen, 2-hour coding exercise, 1 in-person interview to debrief on the coding exercise, 2 subsequent “final-round” interviews, with a mix of technical and “fit” questions.

    Ultimately, the output of each stage should be “hire” or “no-hire”. Rack up enough “hire” votes from the different interviews, and you can make an informed decision. If you want, you can assign a more granular score, but it rarely yields a different outcome.

    If you find yourself coming out of an interview and not being able to offer a hire/no-hire opinion, you’re asking the wrong questions.

     

    Lack of preparation is unforgivable

    Before the interview, write a list of good questions for each aspect of a developer’s skillset - with a backend developer, for example, you might want to focus on “web technology”, “databases”, “system design and architecture” and “algorithms”. If you have 2 in-person interviews, split the topics between them so that you don’t end up covering the same ground.

    You can do the same thing for cultural & personality interviews, but you need to take care to avoid weak questions which either don’t reveal anything. Often, questions like “why do you want to work at a startup?” and “why do you want to work at this company?” are just an opportunity for the candidate to blow smoke up your arse. It’s a vanity exercise.

    Instead, I like questions like “which parts of your previous job did you most enjoy? Which parts would you avoid in future?”. If you really probe for specific examples, it can tell you a lot about a person’s personality and motivations.

    This list of questions should be shared with the company, and improved over time. I’d suggest it’s a key part of the company’s IP.


    Relevant coding challenges are the best indicator of ability

    In my experience, relevant, realistic coding challenges are by far the best indicator of ability. They should take roughly 2 hours, to be done in the candidate’s own time, and they should tell you:

    - at a basic level, can the candidate structure a simple system?

    - does the candidate write fluent, idiomatic code, in the context of whatever language is chosen?

    - can the candidate select & use appropriate libraries?

    - how much effort has gone into building the system?

    For example, we sometimes ask people to write a client library for a real-world API or web-service in a language of their choice. I really like the exercise because there are such a range of solutions and opportunities for people to demonstrate exceptional ability and effort. There are also a number of great extension questions when you ask the candidate to talk you through the solution. How would you speed-up the client or the API? What are the different approaches to caching? How might you fetch a great number records simultaneously? If you were maintaining the API, how would you deal with very high volumes of requests? What are the various ways of dealing with API versioning?

    We also tend to do some fairly lightweight algorithmic coding challenges in person - calculate row n of Pascal’s triangle or implement a basic sorting algorithm, for example. It’s important to provide a clear definition of the algorithm in question, allow the candidate use of a language of their choice and their own development environment, otherwise you’re not testing the right things. Interestingly, the optimal solution to many of these challenges involve recursion - a technique that seems to baffle some developers.

    These shorter tests tend to reveal whether people can think under pressure, communicate clearly and debug broken code in a sensible way. For example, faced with a half-working sorting algorithm, a bad candidate will start changing variables at random. A good candidate would work methodically through the problem, ideally adding debugging lines that output the values of relevant variables at each stage. It’s worth noting that, while useful, these tests are not as good an indicator of on-the-job capability as the longer coding challenge. Many fantastic engineers don’t excel under this kind of high-pressure situation, and actually it’s not the kind of scenario they’d encounter on-the-job very often.

    At the other end of the spectrum, asking someone to complete a 3 or 4-day project might be the ideal way to judge aptitude, but it just isn’t realistic in our experience.

    Practical Tips

    Finally, there are a number of simple practical tips that will make your life much easier:

    Wherever possible, try to bunch your interviews into blocks of time. At the most extreme, I met 8 applicants for in-person interviews in a single day. It’s exhausting, but it avoids constant distraction and context-switching, and allows you compare candidates more effectively. 

    Have one person in the company who is responsible for receiving & screening CVs. Do this in blocks of time, once or twice a week. Otherwise, it can get overwhelming 

    Be clear what you’re looking for from CV- and phone-screens. You could be wasting a lot of time. Write a checklist after you’ve seen a dozen or two CVs. We should have rejected way more people at this stage.

    Whatever the outcome, respond to candidates quickly and constructively. It makes them feel valued, and it’s simple good manners.

    Finally…

    Even using this process, you’ll make bad hires. It happens, and it’s not the end of the world. Fire fast and move on. Overwhelmingly, the bad hires we made were due to poor preparation, or, at times, wilfully skipping most of this process. Hiring well is one of the most important things you can do for your company. Don’t neglect it!

     

    I’d love to hear your thoughts - please comment below, or follow the discussion over at Hacker News

     
  7. 04:41 28th May 2013

    Notes: 1

    Advice

    Starting a company is a challenging process - you’re thrown into the deep-end with nothing but your wits to help you. Often, though, advice will be forthcoming. This post is about that advice, and how to spot the difference between good and bad.

    Some advice is expert - your lawyer talking about your Ts & Cs, or a sysadmin giving advice on web-service redundancy. In these cases, you’d do well to trust the person’s professional judgement and follow their advice.

    Other advice is not.

    Some people, as part of their jobs, are trained to give substantive, concrete, and often completely baseless advice. In my previous life as a Management Consultant, we’d routinely give advice like “sell these 400 under-performing stores”. There is a certain reassurance in the tangible, solid nature of the advice; “Do this and you’ll be fine”. It absolves the recipients from spending more time thinking about the problem - the experts had provided a solution. As long as the advice wasn’t demonstrably wrong, everyone was happy.

    Unfortunately, early-stage companies aren’t like this. If you waste 3 or 4 months misguidedly pursuing some market segment, or building a feature-set before you’ve determined that you’re really on to something, you’ll bankrupt your company.

    This is why advising early-stage startups is so fraught with difficulty - experts from fields like law & accounting give substantive advice without really understanding the nuance of the market or the product. In fact, almost no-one else in the world is close enough to your customers to really understand their problems. These are the problems that you, as a startup founder, should be trying to identify & solve.

    When my startup, GoCardless, was in Y Combinator, the advice we got was different. Despite having some of the best collective experience in early-stage startups on the planet, it was striking how rarely the partners would give this kind of substantive advice. Instead, they dealt in mantras & heuristics. At the time, this was incredibly frustrating - we wanted someone wise just to tell us what to do. It apparently doesn’t work like that.

    Instead, they presented a set of rules which appear to maximise (but not guarantee) a startup’s chances of survival:

    • "Launch early & talk to your users" (or, more completely “Build the absolute smallest thing that can be considered a complete application and ship it" )
    • "Make Something People Want" 
    • "Don’t Worry about Competitors"
    • "Focus 90% of your time on solving your one biggest problem"
    • "Don’t Talk To Investors (until demo-day)"
    • "Be a cockroach" (or “Assume you won’t get money, and if someone does offer you any, assume you’ll never get any more.”)

    Sam Altman recently published a list of these at http://blog.samaltman.com/startup-advice

    When people come to you with a problem, it’s always tempting to give substantive advice. This must be even truer for the Y Combinator partners - these are people who’ve built & sold a number of highly successful businesses over the last 15 years. Yet, by and large, they refrain. In retrospect, I find that a really interesting phenomenon.

    On the other hand, it’s troubling to see accelerators, especially in Europe, touting their vast “networks of mentors”. Experienced accountants give great tax advice, but dreadful startup advice. When it comes to acquiring your first 100 users, I’d stick with the mantras - “Launch early, talk to your users & make something people want.”

     
  8. 06:22 8th Nov 2012

    Notes: 10

    Making something people want - The GoCardless story

    I’m a co-founder of GoCardless, a direct debit payments company in London. This is the story of our first two years and the people we’ve met along the way.

    Making something no-one wants

    One startup I know very well in San Francisco was started by 3 ex-financiers. Their idea involved an industry in which none of the 3 had any experience - they wanted to build software to help building contractors bid for big construction projects.

    None of them knew how to code, so they specced up a “prototype” and engaged a web-development agency to build this “MVP”. Development was slower than hoped for, and the specs changed repeatedly, based on the evolving strategies of the founders and conversations they had with the “big players” in the industry. Almost 10 months later, this MVP was ready for a trial launch. Unfortunately, the product by this stage was so complex that it was almost impossible to use. Successful use of the product involved no fewer than 14-stages, involving 3-way communication between the bidder, the project-owners and referrals from previous projects.

    They’d spent so long story-boarding their product that they had a really good idea of how it would work once they’d achieved huge market penetration. According to their “vision”, all communication would be managed within the app, as everyone in the industry would be using it.

    They had some pretty great features that were novel for the industry at the time - sharing quotes electronically (step 8), exchanging &  signing contracts online (step 11) and eventually paying for the services (step 13) as part of the process. Unfortunately, it was impossible to use an individual component without completing all of the previous steps. Until they’d reached huge penetration amongst all 3 types of user, their product would be almost unusable.

    But, when looking back on the project, each of them professed that they thought they were building an MVP.

    The mistake they made was focussing too much on their “vision” or “strategy”, and not enough on the dirty, mundane work of making a product that addressed real needs or desires. Chances are that one or two of the 14 features were genuinely useful and would have made a fantastic starting point for a startup. Unfortunately, they were unable to work out which ones they were.

    Unless people are incredibly lucky, they’re often not making something people want.

    "Ideas for startups are worth something… but mainly as starting points"

    "An idea for a startup… is only a beginning. A lot of would-be startup founders think the key to the whole process is the initial idea, and from that point all you have to do is execute. Venture capitalists know better. If you go to VC firms with a brilliant idea that you’ll tell them about if they sign a nondisclosure agreement, most will tell you to get lost. That shows how much a mere idea is worth. The market price is less than the inconvenience of signing an NDA."- Paul Graham, How to Start a Startup

    "Making something people want" is not the product or conclusion of your idea plus development time. It’s not the end result - it’s the process. "Making something people want" means prototyping, launching early (often by faking large parts of your service) and talking to users.

    It’s the fastest, most cost-effective way of finding that smallest “quantum of utility” - the kernel of an idea that solves a burning problem for a tiny subset of people. 

    In other words, it’s a way of ensuring you’re not wasting your time. 

    People

    There are two kinds of people who most commonly have this problem. 

    The first is the “Business Strategist”. He’s the “ideas guy” who goes to meetups to tell people about his gleaming vision for his startup. He’s a management consultant, a banker, a lawyer. He doesn’t want to get his hands dirty with actual customers, and he’s looking for developers to build his “MVP”.

    The other is the “Technical Purist”. He’s often got a Computer Science Degree (or several), he’s a fantastic developer and he’s worked at a big tech company or in a well-respected development agency. He develops against the nightly builds of the hottest new graph databases. Everything is test-driven and built to scale to tens of millions of users. He loves building stuff so much that he doesn’t want to waste any of his time actually talking to people or selling his product.

    At my company, GoCardless, we had a combination of both of these problems.

    During the summer of 2011, we were 6 months into our startup. We’d built a product the previous January and launched after just 3 weeks of development. Our “vision” was to solve group payments - we were a platform for sports teams & university societies to organise & collect money. We managed to convince about 20 or 30 of our friends & family to collect money for their various causes - from rowing clubs to stag parties. It worked ok - people we knew used the product, and some used it repeatedly. It was enough to show a glimmer of possibility, and it helped us get onto Y Combinator that summer.

    By June, growth was stagnant. We spoke to our users every day, asked them what additional features they wanted to see, and then we built them. From scratch, we built a set of customisable forms for collecting information from group members (wufuu, anyone?), advanced calendaring and reminder functions (doodle.com) and the most complex set of functionality for managing large sports clubs with multiple teams, roles and permissions that anyone could possibly imagine. Our solution to crappy growth was to split out time roughly 50:50 between building more features and debating the vision of our company.

    After an afternoon of meetings with pg & the other Y Combinator founders, we resolved to take a different approach. We’d give ourselves 2 weeks to see if we could make this thing work. We paused development entirely and spent all of our time cold-calling new potential customers. And since our payments platform was UK-only, this meant getting up at 3am East-Coast time.

    One particularly memorable conversation involved 20 minutes speaking to a man who ran a local village cricket club in the South-West of England. I’d empathised with the strains of running a sports club (I’d run one at university), and the terrible administrative burden it came with. We’d discussed the trials of collecting subscription fees and the social anxiety caused by chasing up over-due debts. So I moved to close him; “We’ve built a website to solve all of these problems - give me your email address and you can try it out.”

    "What," he asked, "is an email address good for? Why would I have one of those?"

    "To communicate online," I spluttered, "to use the internet!"

    "The internet? I don’t use the internet. Goodbye, sir" he concluded.

    Our problem, we discovered, was two-fold. One, that the mass-market didn’t appear ready for our product. It turned out that the most successful Irish sports-team management tool (teamer.net) is still based almost entirely on SMS. The second problem was more fundamental; we hadn’t really discovered our “quantum of utility”. We didn’t have that tiny kernel of a product that solved a really burning problem. Collecting money for sports teams is annoying, but it’s only a small part of a few people’s lives.

    On the other hand, it turned out that businesses faced the problem of collecting money every day. And for particular people within those businesses - the accounts receivable department - this was their entire job. The core of our product - a Direct Debit API - was an ideal solution to this burning problem.

    "In nearly every failed startup, the real problem was that customers didn’t want the product… The only way to make something customers want is to get a prototype in front of them and refine it based on their reactions." - Paul Graham, How to Start a Startup

    To that quote I would add "and if it turns out that no-one cares, do something else"

    Most startups aren’t competing with other startups; they’re competing with no one giving a shit.

    A few years later…

    One of the founders of the construction startup (mentioned at the start of this article) came across another group of founders who looked like they were on the same path. This company had what seemed like a promising idea - an interesting take on loyalty-schemes & subscriptions for real-world products and services. Its non-technical founders were looking for a designer and a coder to build an iOS app & website, which would be their MVP. They’d planned out exactly how it would work when they’d achieved massive adoption.

    They had a long and frank discussion with the earlier founder. Heeding the words of warning, this company reformulated its plans. What was the bare minimum they needed to work out whether customers & businesses were interested? How could they simulate the benefits of their service without blowing tens of thousands of pounds on outsourced development?

    The founders decided to sell shop-by-shop in their local area, signing businesses up to their new “platform”, promising them new patrons & more repeat custom. They then quickly ran a local advertising campaign offering people great deals at their favourite cafes & bars. Instead of spending time & money building an app and a website, they simply distributed laminated cards printed with with unique IDs to their customers, and every morning took a paper lists to the local businesses with details of the discounts each person was entitled to. Every night, they’d collect the lists and manually enter the data back into an Excel spreadsheet. By the end of the month, the majority of the businesses reported remarkable increases in volume & cash-flow, in some cases up to 75% improvement. They used this proof to secure investment & build a team of developers and salespeople.

    Conclusions

    You should spend most of your time focussing on making something people want. This is a process, not a goal.

    If you’re worried you’re not making something people want, the solution is almost always the same - focus on what you’re shipping in the next month. Don’t obsess over long-term strategy, or stick dogmatically to a “vision”. Launch sooner than makes you comfortable. Fake as much of the service or product as you can to save time. Talk to your users to make sure you’re solving a burning problem. Once you have 20 or 30 regular users, do everything in your power to absolutely delight them.

    Ensure you’re making something people want. If you’re worried about this, ask them.

    Discuss this post on Hacker News

     
  9. 12:54 20th Apr 2012

    Notes: 14

    Reblogged from peternixey

    peternixey:

    I seem to have a lot of conversations recently with people who find themselves in the kitchen but without the means to cook anything.

    As a cook myself they often ask me how to go about finding themselves a chef who can cook something for them. I ask them why they want to do that and they tell…

     
  10. Don’t write tests - The Hidden Cost of TDD

    I’ve been following the TDD/testing debate with some interest; I work at an early-stage payments startup called GoCardless where we test religiously. We’re not dogmatic about whether the tests come first or not, but code doesn’t get deployed without full unit & acceptance tests. It’s saved our skins on more than one occasion

    I wanted to explore the benefits of testing and suggest some situations in which <heresy>testing may be counterproductive</heresy>  

    Testing - The Good Bits

    When you’re working as part of a team on a large codebase, comprehensive testing is invaluable. If you measure development velocity on a company-wide basis over the medium-long term, testing provides clear benefits.

    Engineers working on one part of the codebase don’t have to worry about their changes having unanticipated knock-on effects in another area. A good mix of unit tests and acceptance tests means that you can write & deploy code quickly, without a lengthly QA process.

    When stuff does break, unit tests allow you to identify the individual component that’s causing problems. It’s fantastic - at GoCardless, no code goes into production without comprehensive testing & peer code review.

    (If that sounds like a place you’d like to work, we’re hiring!)

    Testing slows down rapid prototyping

    Having said that, there are situations when comprehensive testing just isn’t the right solution. I’m not talking about whether tests should come before (TDD) or after writing code - there are situations where it might be beneficial to write no tests at all.

    The particular situation I have in mind is the ultra early-stage startup struggling to find traction. The number one priority in this situation is not to write beautiful code or solve hard technical problems in innovative ways; it’s to work out what people people want. You do this by iterating quickly & launching early - just as it’s often possible to fake a much more sophisticated or elegant solution by doing stuff manually, it’s also possible to write quick, scrappy code that works just about well enough.

    You’ll accumulate buckets of technical debt, but it’s generally ok. If you don’t find traction or “product-market-fit”, you declare bankruptcy on that technical debt and just move on to the next iteration.

    Once you’ve found traction, you’ll likely need to re-write your codebase, but that’s ok as well. At this point, you’ll have a much clearer idea of what the product actually does, so writing specs should be much simpler.

    As an addendum, I think this conundrum is one reason why outsourcing early-stage tech development is such a terrible idea - any decent web development shop will generally insist on writing comprehensive test suites, and they’ll react badly if the job spec changes dramatically every 2 weeks. But that’s sort of necessary when you’re writing code for an early-stage startup.

    Discuss this post at Hacker News !