Having to decide when to build versus when to buy is a common occurrence in software development. Build vs. buy scenarios usually occur when there is a product need for something new and significantly complex. Some examples are capitalizing on an opportunity to deliver exponential value to customers, needing to scale, better measuring your audience, etc... Some organizations always lean towards building while others always lean towards buying. It is important to know your organization's tendencies going in. My experiences from leading build vs. buy decisions on various product teams has helped me identify some common patterns that guide my current and future decision making.
To identify those patterns, I ask myself and the team a set of questions. Each question falls into one of four main categories.
- Reviewing the strategy
- Evaluating the product
- Identifying costs and timelines
- Looking at vendor relationships
The questions have helped guide my decisions -- which have led to success, for the most part :) -- in build vs. buy scenarios in the past, but they definitely do not apply to all situations. Up to this point, I've mainly been part of product teams that focus on consumer facing products. As a frame of reference, I've listed some of the more significant build vs. buy scenarios that I've been through. It is by no means a comprehensive list, since even small product features can sometimes be build vs. buy candidates, but hopefully the examples I reference will aid in getting my point across.
- Android & iOS video players
- Marketing websites for SaaS companies
- User Authentication systems
- Apps on platforms w/ small market share (e.g. Fire TV & Windows Phone)
- Video Quality of Experience analytics
Now, down to business: the questions. I usually collect the data for each question separately, but then try to go through them in their specific order so I can paint the whole picture.
Reviewing the strategy
Hopefully the new feature/product/service -- henceforth referred to as "it" -- was already researched and validated as a necessary addition. But, just in case, there are a few questions to ask yourself to make sure you're not getting too caught up in the "wow-factor" of some latest and greatest technology.
Do you envision it being a top 3 reason why users would use your product over a competitors?
Basically, is it going to move the needle? Determining the relative business importance of the new thing compared to other things in your product is always a good starting point. If you can't quantify its importance to the business, then you probably won't be able to convince the people controlling the purse strings to invest. As such, it is much harder to make a buy case for things that fall into the "customer delight" category since -- despite being necessary -- the ROI is hard to quantify. For example, if your product is a document annotation app, then the performance and flexibility of the annotation features is a key reason users would use your app over a competitors. On the other hand, the UI animations a user sees when choosing a different annotation tool are a nice touch, but won't help you retain users unless the base features are of high enough value.
Do my competitors already have it?
Just because your competitors have it doesn't mean your customers want it. But, if your competitors do have it, and you've done the research to know your customers want it, then you should probably have it too. In some cases, you simply need to deliver something that is just-as-good as your competitors offering in order to not bleed customers. For example, a core feature in all of the PBS apps is the video player. And although the video player itself is not the #1 reason someone uses the app, if it sucks enough, people will leave and never come back. If the player functions well enough compared to others, then PBS can keep those customers and compete on things it is more competent in -- and that it can afford -- such as content creation, discovery, and personalization.
Do you need it for a tactical win, a strategic advantage, or both?
If you need a tactical win -- for example, to get a product shipped or to block a competitor's advance -- then the timelines are usually shorter by default, which impacts your choices. If it is a strategic play to become first-in-class in a specified area, then there may be some more flexibility in the timeline. Sometimes you need both. I've been in situations where there was a desire to get something out the door quickly, but then there was also a desire to continuously invest until we had a first-in-class offering. One example was the desire to test user's desire to create an account with us by connecting via one of their social networks instead of using an email/password combo. It was too complex of a feature to quickly test, so a 3rd party solution to validate whether or not it was a strategic need. It was, so ended up building a better system ourselves to use long term.
Evaluating the product
Can it be built using one of our engineering core competencies?
It is incredibly important to know whether or not the item in question can be built using one of your core engineering competencies. Every organization has them. They could be Python, Java, iOS, etc... If it can be built using one, then you already have some upfront knowledge of the speed and quality at which it could be delivered. It also means you could possibly inherit a vendor's code base without bringing things to a screeching halt.
If it doesn't fit within our core competencies, do we want the new technology to become part of our core competencies?
If the new thing is important enough, you may decide to add a new technology to your list of engineering core competencies. THIS IS NOT A DECISION TO BE TAKEN LIGHTLY. It can have a serious impact on budgets, headcount, timelines, and the overall future of your engineering organization. Product Managers are not the people that will ultimately make these decisions, but they do have a seat at the table. I've experienced this challenge when switching from building native apps to building HTML5 hybrid apps. Our core competencies shifted from iOS/Android engineering to responsive web engineering. The people who ended up on the team reflected that shift. Eventually the decision was made to switch back to native apps, and it took over a year to build back up the necessary core competencies.
Are any vendors offering it yet?
If vendors are already offering it, you now have some pre-defined buy options on the table. Even if vendors are not yet offering it, that doesn't mean you couldn't find a new vendor that is willing to build the whole thing for you, or work with one of your current vendors to build a solution. For example, when faced with a Jan 1st 2014 FCC deadline, PBS worked with JW player to build out support for all the FCC mandated closed caption display configuration options.
Are there any open source solutions?
Open source solutions are a great starting point for build vs. buy evaluations. You can easily compare open source solutions to what vendors are offering to see if the added vendor costs are worth it. If you are leaning towards building, you can more easily identify what development land mines have already been stepped on and quickly paint a picture of works & what doesn't work. For example, when searching for an Android video player, PBS compared open source options such as the ExoPlayer to vendor options such as the VisualOn player to see which had better HTTP Live Streaming support and whether or not PBS could build it themselves.
Does it require direct integration with your current product or is it a standalone solution your product will communicate with?
Both scenarios comes with their fair share of communication overhead. Direct integration requires almost daily back and forth between development teams. Standalone services many not require as much day-to-day, but it is important the engineering teams agree on a "communication contract" to make everything is being built in a compatible way. For example, it's important for a front-end client to know the format and structure of an API response even if the data coming back in those responses may change. Another example is applications that require constant synchronization, like a Chromecast "receiver" app running on your TV that is synced with a Chromecast "sender" app that is running on your phone/tablet/browser. In order to properly display play/pause and other events, the two parts of the app need to understand the rules for communicating with each other.
Will your team be responsible for building it or would another internal team be responsible?
If you decide to build it, that really just means you are deciding not to pay a vendor for it. That doesn't mean your product team will be responsible for building it. It could be another team inside your org, especially if the new thing is a standalone service or something similar. It is important to identify who in your org would be building it from the start. If more than one internal team is involved, then the aforementioned communication overhead may be just as much as if you were working with a vendor. A classic example of this is when an organization has separate front-end and API product teams.
At what point, if ever, will you be responsible for maintaining the codebase?
If you plan to inherit the codebase from someone else, make sure you account for the ongoing documentation and communication overhead involved throughout the external build process. For example, a consultant may build early HTML/CSS prototypes based on designs you provided, but you will ultimately be responsible for populating the prototype with live data and launching the thing into production.
What points of failure would it introduce into your current system?
If new points of failure are introduced, it's important to figure out what happens to your product when one of these new failure occurs. It's just as important is to figure out who owns the point of failure. There is nothing worse then having your entire product break and not being able to explain it to customers because a vendor introduced a point of failure that is unknown to you and your team. For example, when using 3rd party user authentication systems, the error messages make not make sense within the context of your system. You don't know how to troubleshoot it and your users don't know what is going on.
What level of design customization is desired?
Branding and established interaction patterns are important. I've never made it through any build vs. buy scenario without addressing design customization questions early on. One approach that has worked for me is taking an inventory of all the new UI components being introduced, and then make a checklist of what can and can not be customized to match our style guide. I find myself doing this the most systems are being bought and white labeled under our brand, for example, a digital help desk.
Identifying costs & timelines
I believe it is important to have consensus on the strategic objectives and product evaluation criteria before addressing the elephant in the room, costs and timelines. It allows me to deliver a more informed and concise proposal to the people controlling the purse strings. That being said, there are plenty of occasions when product timelines and budgets are not flexible. In those cases, setting the price & timeline constraints earlier on in the evaluation process is an absolute must. Also, just like in any other product management exercise, don't forget the opportunity costs!
How much money is it going to cost upfront?
There will likely be some amount of upfront monetary cost involved, especially when buying. This is the easiest cost to account for, but usually not the most important. Some examples of upfront costs are product licensing fees or a product setup fees.
What will the integration costs be, if any?
If integration is required, there will be likely be some quantifiable cost involved. At the very least, there will be at least one engineer's time spent on the integration.
What are the recurring and future costs associated with it?
A lot of times the recurring and futures costs are directly related to how the contract is structured after you decide to buy. Some common costs are recurring licensing fees, increased operational costs (such as CDN costs), and different cost tiers that kick in once certain usage thresholds are reached. For example, you may buy a user authentication system and have to pay incrementally more for every 500k new users that register via the system. It's always good to map out these costs versus your projections to make sure you aren't caught off guard by the cost increase if the product takes off.
How many engineers/designers would it take to build it?
Usually, everyone in your org is already working on something else whenever a build vs. buy decision comes up. Knowing how many people would have to get pulled off their current projects helps you prioritize the importance of the new thing vs. the other stuff.
Is anyone interested in learning the new skills required to build it?
If you are leaning towards build, then you need someone to build it. If no one has the current skills required, then you will need to set up a training plan or hire from the outside. Either way, there is going to be time and money spent on getting someone up to speed. A frequently occurring example of this is when and organization is figuring out their mobile strategy and needs to figure out whether to hire or to train mobile engineers.
Do new people need to be hired?
Even if you have a full team of people ready to jump at the problem, the new thing could be important enough to justify hiring new people. More time. More money.
How much do your people cost?
People on your team are going to be spending time on this new thing. If possible, do some basic calculations on how much they cost and how much time they plan to spend on the project. People costs also apply when you need to hire someone new.
How long does your team say it would take to build?
Engineering estimates are an imperfect science, so don't put all your eggs in this basket. That being said, you still need to compare the estimates of how long it would take to build vs. how long it would take to integrate a vendor solution and then compare the gap.
How long did it take vendors to build?
Most vendors won't tell you how long it took them to build something. If you are lucky enough to have one tell you, then compare the answer with your own team's estimates.
When do you want (or need) to ship it to customers?
In a perfect world you can set release dates based on readiness, quality, etc...but that is not always the case. Figuring out if your senior management wants to drive towards a hard launch date is extremely important. A crazy date can blow up the rest of your evaluation, so hopefully the date is up for at least some discussion. Regardless, the answer to this question is obviously an important piece of the puzzle.
What would be put on hold if development focus had to shift?
Opportunity cost number one in any product prioritization process. Included here just as a reminder.
What else could the money go towards if not used?
If you've gotten funding for the project, it is a good thought exercise to take a step back and think what else the money could be used for. Sometimes improving a current part of your product results in significantly more user and business benefits than adding something new.
Do your best engineers want to build it?
Asking your best engineers if they want to build it is a good indicator of how fun the project is. If they want to build it, and you can afford to pull them off something else, then it could be a great morale booster and help keep people engaged.
How much time would be spent managing vendors?
Deciding to buy from a vendor is no better than choosing to build if you and the engineering team have to spend an unreasonable amount of your time managing what the vendor is doing. There always will be some management overhead, but you need to carefully monitor it to make sure it doesn't render buy decisions pointless. Proper documentation from both parties involved always minimizes the communication overhead.
Looking at vendor relationships
If you have already decided to build, you can ignore this part. Otherwise, you need to consider the finer aspects of vendor relationships before making the final decisions.
Do any of your current vendors have it built already?
It's usually easier to work with a current vendor, but like anything else there are pros and cons.
- The vendor has knowledge on how to integrate with your current systems
- There is likely favorable pricing opportunities due to bundling and other discount models
- The vendor is in your org's system and a Master Services Agreement (MSA) is already in place, which can shave anywhere from 2-4 weeks off of your timelines
- Depending on how long and successful the relationship has been, the mutual tolerance for risk is higher
- You are familiar with the quality of their support services and their turnaround times on critical issues
- Potential to not get the vendor's "A team" since they already have landed you as a client
- The vendor's product might be of much lower quality than its competitors
Is it necessary to start a new relationship?
It may be necessary to start a new vendor relationship, even if a current vendor is offering you a solution. It's important to look at the new vendor's delivery track record and its relationships with other clients. If possible, connect with one of the vendor's other clients to learn more about what it's like to work with the vendor. Researching who their engineers are is also a good approach. Choosing a new vendor almost always comes with more risk than going with a known option. However, If the vendor is trying to make a name for themselves and you are with a major brand, odds are good that they will put their best people on your first project. Good people always mitigate risk.
Do your senior managers already have a vendor in mind?
In my experience -- albeit limited experience -- Senior managers almost always have friends who are high-level employees at vendor companies. As such, they may ask you to research one of their friend's companies that can provide the service you need. Many times this works out great since there is already be an executive level relationship in place (something that all vendors want). Sometimes you may get drive-by requests to evaluate vendors for a product or service that may seem to not fit with your core strategy. Be sure to follow up on these requests to learn more.
Deciding on a strategy
After you've answered a bunch of these questions, you should be able to pick a strategy based on some of the patterns in your responses. It's not a perfect science, but the good news is that there is usually more than one good strategy for each scenario. The pattern matching just helps minimize the risk of choosing one of the wrong strategies. It's important to know your organization's build vs. buy tendencies going in. The strategies are in no particular order.
Buy a black box
Buying a black box simply means buying a full piece of a software in which you have no direct control over the functionality and don't have access to the source code. I tend to go with this strategy when the following holds true:
- The product is significantly complex and would take a while to build it in a way that would give us a competitive advantage
- There is budget for a big buy
- The technology doesn't match one of our core competencies and there is no desire to build up the necessary core competencies to support or build the product
- I am comfortable with the level of communication overhead involved between myself and the vendor, which is critical when trying to integrate with a black box
- There is a well established vendor that most of the market leaders already use
- The levels of customization required are flexible and the vendor is open to roadmap suggestions
Build the whole thing
Building the whole thing is what, in my experience, most engineering organizations are going to initially lean towards doing. It's usually the most fun of the strategies, but not always the best. In my experience, full build situations usually share some common characteristics.
- There is a determination to discover an offering that delivers significantly more value to customers than competitors
- The technology matches core competencies
- It's a long term strategic play for the business
- There are few pressing deadlines that would result in an inadequate product being shipped
- Your best engineers are passionate about owning the full stack
Buy, with intent to inherit
Buying a product with the intent on inheriting the source code at a later date is a strategy that I've only seen chosen a few times, but it's paid off. Plans to inherit share some characteristics with plans to do a full build, but there are some distinguishing characteristics that lead to a inheritance decision being made over a build decision.
- The product is needed but isn't meant to be a significant competitive differentiator
- The product is simple enough that inheriting the source code won't add a ton of maintenance overhead to the team
- It's cheaper to have someone else do the bulk of the build legwork
Buy to ship early, build later
In some situations you need to buy in order to ship early, but have a long term desire to build. In my experience, these situations share the below characteristics.
- A tactical win is needed resulting in tight launch deadlines
- There is a vision for delivering significant value with the product, but it needs more market testing and is too complex to build quickly
- There is room in the budget for investing in the product in both the short term and the long term
- It is considered both a tactical and a strategic play
Build, with buy fallback
The last of the common strategies I've used, is the one that involves building the product but leaving the buy option on the table in case some unexpected roadblocks appear during build. Sometimes this strategy isn't chosen, it's just thrust upon you due to circumstance. Regardless, it usually shares all of the same characteristics as the "Build the whole thing" strategy, with the following additional unique characteristics:
- While confident that a first-in-class offering can be built, there are established vendors with good offerings already in place
- Build plans running way behind schedule or over budget
- Sometimes a new vendor comes out with an awesome offering mid-way through build process
- A conflicting but more important internal priority shift pulls the engineers from building and the product is still needed