[How Do I] Answer a Product Design Interview Question
If you’ve ever interviewed before, you probably know that it’s much easier to tackle interview questions when you have a strategy. For example, Gayle McDowell outlines a five step process for answering Software Engineering technical questions in her book Cracking the Coding Interview:
- Ask questions to resolve ambiguity
- Design an Algorithm
- Write pseudocode
- Write code
- Test your code
Design is the process of deciding what to build, which is different from actually building something. As a result, design questions should be approached differently than technical questions. The purpose of this blog post is to describe a strategy for answering design interview questions.
- Empathize with the customer
- Align with business strategy
- Define success
- Apply the design process
- Start by validating your assumptions
1. Empathize with the Customer #
There’s a hot trend in design right now called Human-Centered Design (HCD). HCD is a design philosophy that focuses on designing products with the customer in mind (Design). This sounds stupidly obvious, so it’s surprising how easy it is to screw up. For example, one trap that designers often fall into is designing for themselves. It’s particularly contagious among designers that are passionate users of the product they work on. Designers will make decisions in favor of what they want, rather than what’s best for the customer. This is essentially a sampling bias, where designers overrepresent themselves in their mental image of the customer. As a designer, you should have a deep, unbiased understanding of your customer.
So how do you know who your customer is? Your customer is the people (or things) that will be using your product. This information can usually be teased out from the question. Say you’re interviewing at Spotify and the question is: "How would you improve Spotify for people that run?" Then your customer is going to be something like, “people that listen to music when they run.” You should try to pair down this definition as much as possible by asking your interviewer questions: Are these people existing Spotify users? Do they have iPhones or Androids? What type of music do they listen to? Involving your interviewer in a discussion defining the customer will validate that you’re taking steps in the right direction. One patented way to mess up a design interview is to design the right product for the wrong person.
If your company doesn’t know who their customer is, then they’re a startup (Lean). Designing for a startup is essentially the same as designing for an established company, except that there’s two degrees of freedom to work with. A startup has to design a product AND find a customer for that product. Doing so is sometimes called “finding product-market fit”. The approach is to fix one axis, typically the customer, while experimenting along the other. If you’re faced with a startup-y question, your best bet is to double down on steps (1) and (5). This will put your money on choosing which customer to target at the beginning, and validating that you made the right choice at the end.
In later brainstorming phases it will be immensely useful to have a clear and accurate understanding of the customer. One way to achieve this clarity is through the use of customer personas. A customer persona is a fictional character that represents your average customer. It should be concrete and human (one trick is to use real customer quotes), but validated by data. For example: “Rob has been a paying Spotify user for almost three years. He listens to a variety of music, including jazz and rock, mostly at work and on his commute. He’s a consistent runner, and will often log upwards of 10 miles a week, and even more when he’s training for something. He will occasionally listen to Spotify when he runs, but he’s much more likely to leave his phone at home. He….” Although you probably don’t need to create customer personas for something as short as an interview (unless the job is skewed heavily towards UX research), they’re useful to consider. It’s often easier to design for an individual, rather than an abstract. More importantly, it’s often easier to pitch a design in the context of an individual.
Designing is a balancing act. There will be many different customers, each vying to push the product in their own advantageous direction. The designer has to weigh each group carefully, and make the decision that benefits the majority. When compromise can’t be reached, it’s up to the design to stay true to the customer.
Align with Business Strategy #
As a designer, your first priority is to represent the customer. That being said, understanding the business context will enable you to make your designs that much more compelling.
Most companies have some degree of matrix management. This means that employees are responsible to two managers: a functional manager and a product manager. A design functional manager cares about meta problems: how to maintain consistency across different products, or which tools to invest in. Unless you’re interviewing with a design manager, you probably won’t need to consider this perspective (Management).
In all situations it will benefit a designer to understand the product management perspective. A product manager is in charge of some subset of the company’s product. For example, Sarah is the product manager at Spotify responsible for the Mood feature in the Browse tab, where users can find curated music to compliment the mood they’re in. Product management is usually organized hierarchically. This means that a product manager’s manager will be responsible for a superset of all the features managed by the people under them. Sarah’s boss, George, is responsible for the entire Browse tab, where Spotify users go to find music suggestions. George’s boss, Julie, is responsible for user engagement across all of Spotify.
This hierarchy is managed through a system of objectives and strategies, where objectives of the manager map to strategies of the employee. For example, Julie’s objective is to increase the amount of engagement that users have with Spotify. Julie has several strategies that she thinks will help achieve this objective, one of which is to provide music suggestions. This strategy maps directly to George’s objective, which is to help users find the music they want to listen to. George, in turn, has his own strategies for achieving his objective, one of which is to use moods to help suggest music that users might want to listen to. This strategy maps to Sarah’s objective, and so on.
A product manager is trying to validate that their strategies are moving the product towards their objective. By moving towards their objective, they in turn help validate that their manager’s strategies are working to achieve higher level objectives, and so on up the product hierarchy. As a designer, your job has two components. First, to help the product manager devise strategies with a pure connection to the objective. Second, to optimize those strategies so as to capture the strongest signal possible. The best designers are able to merge their technical skills with business awareness to be successful on both points.
You should have a discussion with your interviewer to make sure you understand the product team’s objectives. Ideally, you should be able to trace your product team’s objectives up the hierarchy all the way to the company level objectives. The best way to approach this is by mapping the objectives at each level to the strategies of next level up. Let’s do this for our original question: "How would you improve Spotify for people that run?" Here, the goal of the product is a bit ambiguous. Are we trying to market Spotify to runners as a growth market? Are we trying to build out the premium feature set as a revenue driver? There are a number of different ways we can interpret the objective hiding in this question, and the one we choose will certainly influence the product that we end up designing.
Let’s suppose that this question is being posed by George, who has the hypothesis that Spotify can help people find music by targeting specific activities. This provides us with our business objective: to drive engagement by integrating Spotify into popular activities, like running. With this objective in mind, we can see how the product we’re designing fits into Spotify’s entire business strategy.
Define Success #
Going hand in hand with an objective is a metric to measure success. Objectives and metrics have become so synonymous that some “data-driven” companies will make a point of communicating objectives in the form of metrics. If metrics haven’t yet been discussed, you should bring them up to your interviewer and decide which metric best represents your objective. Translating your objective into a metric will clarify your product’s objective and help confirm that you’re heading in the right direction.
When choosing a metric it’s good practice is to apply the One Metric That Matters (OMTM) philosophy. OMTM says that for any product team there will be one most important metric. A rule of thumb for choosing a metric is to follow the standard progression of a product: first validate the business objective, then fine-tune the product, and finally grow the user base. The validation metric for your team will usually align with your manager’s business objective. If the product has been validated and is now being fine-tuned, then the value metric is usually Time Spent or Revenue. If the product has been validated and fine-tuned, then the growth metric will usually follow Growth Accounting, or New Users - Churned Users + Resurrected Users (Analytics).
At a high level, a good metric should be simple, comparable, and change the way you behave. A simple metric has an obvious connection to the business objective, and uses predictable units at a manageable scale. A comparable metric can be viewed over time to judge improvement. Usually this involves taking derivates of top level metrics. For example, the derivative of total revenue is change in total revenue, which provides insight into whether the company is growing or shrinking.
At many companies new features are evaluated in an A/B testing framework. An A/B test is where you segment out two groups of users, and show the test group a new version of the product, and the control group the old version of the product. Then you measure metrics in both groups and compute if the changes in the test group led to a statistically significant rise in the metrics. In situations where A/B testing is possible (such as our Spotify example), you should expect your feature to go through an A/B testing process.
Good metrics should impact decision making. A powerful trick for staying honest to the data is to draw a line in the sand. Before even starting to design, ask yourself how you expect your new design to impact metrics. If you expect your feature to increase average Time Spent, but after A/B testing it you see Time Spent going down, then you shouldn’t release the feature. Even if you put a lot of work into it.
Let’s apply these principles to find a good metric for our working example: "How would you improve Spotify for people that run?". Remember that our objective was to drive engagement by integrating Spotify into the popular activity of running. Since this is a new feature, we should measure it with a validation metric. In this case, a validation metric should track George’s higher level business objective of helping users find the music they want to listen to. A good choice is average Time Spent listening to music per user per day. To draw a line in the sand we can expect that the addition of this feature will result in a long term statistically significant increase in the aforementioned metric for the target customer of Spotify users that run.
Apply the Design Process #
At this point you should have a customer, a goal, and a metric that defines success. Now you can begin the fun part: the design process. The design process that I’m going to use is called the Double Diamond Design Process (DDDP), thus named because it consists of two stages wherein each stage contains a divergent brainstorming phase and a convergent selection phase (Design).
The purpose of the first two phases, Discover and Define, is to create a strategy. This strategy represents the core assumptions that we will be looking to validate in the fifth and final step. The second two phases, Develop and Deliver, are about implementing the strategy. By the end of the Deliver phase we should have a vision for the finished product.
Let’s walk through the DDDP using our example: Design a feature for runners that use Spotify. Remember, our customer is Spotify users that jog on a regular basis. Our business objective is to drive engagement by integrating Spotify into popular activities. Our validation metric is the average Time Spent listening to music per user per day. With that, we should have enough information to start the DDDP.
Strategy #
The first phase, Discover, is a brainstorming phase: Why aren’t more Spotify users listening to Spotify when they jog? Perhaps the interface isn’t friendly for people in a mobile state. Perhaps the Spotify music catalogue doesn’t contain good jogging music. Perhaps people can’t find good jogging music. Perhaps… In a brainstorming phase, there aren’t any bad ideas. The goal is to get as many possibilities on the table as possible.
One pitfall people often make in the Discover phase is defining problems in terms of their solution. For example, “Perhaps we should create a special pair jogging headphones to help people use Spotify’s interface while running.” Designing a new pair of headphones is just one potential solution to one potential problem. By defining the problem in terms of the solution, we limit our ability to come up with other creative solutions, such as incorporating voice commands.
The second phase, Define, is about narrowing down the ideas generated in the Discover phase to create a strategy. If we were higher up the product hierarchy, we could pass this strategy down to be implemented by an entire team (similar to how George passed down his activity strategy to us). In this case, our strategy will be to help users find good jogging music. In the next phase we’ll try to find the purest implementation of this strategy.
Implementation #
In the third phase, Develop, we begin brainstorming implementations: How do we help users find good jogging music? We could put a special jogging icon next to good jogging songs to make them more visible in the catalogue. We could curate special playlists of good jogging music. There will be many different ways to implement your strategy, and this brainstorming phase is an opportunity to show off your cleverness and creativity.
The fourth phase, Deliver, is about taking the ideas from the Develop phase and crafting them into a vision for the product. When comparing different ideas you should keep in mind two useful concepts: tradeoffs and leverage. Tradeoffs* refer to the fact that most implementations will trade strength in one area for weakness in another. Curating a unique playlist for each person is a good user experience, but is expensive to compute. **Leverage refers to the efficiency of the solution in solving the problem. For example, you could create one jogging playlist and share it with every Spotify user, but doing so would have low leverage (not everyone shares your taste in music). At each tradeoff you should try to get the most leverage for your design decisions.
Once you’ve made your tradeoffs and chosen an implementation with the highest leverage, you should take some extra time to present your vision nicely. Your vision will guide the efforts of everyone on the team. It should be short and easy to digest, but poignant and inspiring. In an interview, a good (and quick) way to do this is by telling a story from the user’s perspective. Let’s suppose that our solution is to provide runners with a special radio station that finds music with a tempo matching the pace of their run. A user story might be:
“As a user, I’m browsing the Discover page on my phone and I see a special radio station with an icon of a runner and the title ‘For Running’. I click into the station and it takes 15 seconds to analyze my phone’s built in step-counter, and find music in my library with a tempo matching my pace. It auto starts playing the first song, which I can play/pause/skip if I choose. It also re-analyzes my pace every couple minutes to refresh the station with appropriate songs.”
Start by Validating Your Assumptions #
Now that you have a vision, you can finally start building something. All hail the engineers!! But what are you going to build? The naive approach is to just start building towards the vision helter-skelter. The problem with this approach is that you’re taking on the accumulated risk of every design decision you’ve made. It’s like only looking at the map once before driving to a new city; the more confusing the route, the more likely you’ll get lost along the way. Instead, you should build towards your vision iteratively, validating your assumptions along the way. The process that we’ll use is called Build-Measure-Learn (Lean).
The Build-Measure-Learn process gives you a structure for systematically testing the assumptions that make up your product vision. By decomposing your product vision into testable chunks, you are essentially building checkups into your roadmap. Remember in Step Three we set our expectations by drawing a line in the sand. If reality ever starts differing from our expectations, then it’s a sign we should take a step back and start revising our assumptions.
You should start by testing higher level assumptions, because higher level assumptions will have a larger impact on the product. Implicitly this means the first assumption you’re helping test is your objective, which was provided from your manager’s strategy. If the results of your work isn’t moving numbers the way your manager expects, the first question to ask is whether you are working towards the right objective. But this won’t be a problem in the interview. Practically speaking, the first assumption you should test is your strategy.
Testing your strategy means building towards your vision in such a way that if your strategy turned out to be wrong it would be discovered quickly. You might hear this iteration of the vision being called the Minimum Viable Product (MVP) because it is the minimum changes required to test the core strategic assumptions. Remember our initial strategy was to help users find good jogging music. Our first build might be something as simple as releasing a curated radio station for jogging. From this we can measure whether or not users are interested in finding good jogging music. With this learning we can decide whether to continue pursuing the vision, or whether we should revisit the assumptions we made in our design process.
Suppose users were not interested in the jogging radio station. Perhaps it’s because users don’t need help finding good jogging music. Then we should consider pivoting our strategy (remember we brainstormed multiple strategies in the Discover phase of Step Four). Perhaps the strategy is sound, but the implementation is faulty, in which case we should consider a one of the other implementations we brainstormed in the Develop phase of Step Four. In this particular case though, users were interested in the jogging playlist, so we continued down the path of making the jogging radio station more engaging and available. The result is Spotify Running.
One of the most powerful aspects of the Build-Measure-Learn loop is its iterative nature; learnings from the n iteration guide the build in the n+1 iteration. This makes Build-Measure-Learn an incredibly optimistic process. It’s not about being right or wrong, but rather about contributing to a process of steady improvement. In the context of an interview, no one expects you to be perfect on the first try. Just give it your best shot and know that in the real world you would use the process to converge towards perfection in the long term.
For Example… #
Let’s walk through the full process using a different example: Design a new drink item for McDonalds (Design). Remember our strategy:
- Empathize with the customer
- Align with business strategy
- Define success
- Apply the design process
- Start by validating your assumptions
Empathize with the customer #
This is a pretty vague question, and McDonalds has a large customer base. We can fish for more information by asking our interviewer if there’s a target market they’re looking to focus on, but let’s suppose the interviewer doesn’t give us any additional information. Instead of getting stuck we can just choose a reasonably promising customer and proceed with the process. The customer that I’ll choose is truckers, whom I assume frequently stop at McDonalds along their routes.
Align with business strategy #
Once again we should fish for information, but let’s suppose we don’t get any. Without more specific instructions, we can just adopt McDonalds general business strategy of being a profitable company.
Define success #
Since our objective is to make McDonalds more profitable, we can define success by measuring profit. Obviously we expect our new drink to increase profit.
Apply the design process #
Strategy #
We can further observe that there are two ways a new drink could increase profit. First it could convince customers that currently don’t buy drinks to buy a drink. Second it could convince customers that currently buy drinks to buy a drink with a higher profit margin. I assume truckers typically do buy drinks, so we can focus our initial strategy on selling truckers a drink with higher profit margin. We can frame this as a question: What would make truckers buy a more profitable drink?
One idea is to take an existing drink and try to make it cheaper. For example, perhaps McDonalds could see re-useable cups to truckers to save on the cost of disposable cups. But the savings would be marginal. Another idea is to sell a drink that takes a long time to consume, such as a frozen drink that has to melt. Since ice takes up more space than liquid, the cost of materials for a frozen drink would be less than for the same quantity of a liquid drink. But once again the savings would be marginal.
I do, however, think that truckers might pay more for a drink that lasts longer. I assume truckers get bored on the road and a drink that lasts longer would be more engaging. Or even if the drink doesn’t take longer to consume, but is just inherently more interesting to consume, it might relieve some of the truckers boredom. Let’s use this as our strategy: create an engaging drink that truckers will pay premium for.
Implementation #
What is an engaging drink that truckers would pay premium for? Perhaps we could create a drink that changes flavor over time (as the ice melts?). Perhaps we could do a bubble tea spinoff by adding chewy chunks of tapioca to a drink. Perhaps we could just add a crazy straw to an existing drink, which would make it more difficult to drink quickly. But that would cause a soda to fizz up. We could add a crazy straw a milkshake, but truckers might find that silly. But we could make a milkshake more difficult to drink by making it thicker, by adding more ice cream instead of milk.
All of these ideas have potential, but only the thick milkshake idea can be incorporated without introducing new materials. This certainly makes the implementation a lot cheaper and easier. On the other hand, a flavor changing drink would lend itself to killer marketing campaign. The growing bubble tea trend shows the validity of the tapioca idea, but McDonalds might have to buy the rights or deal with patents. Without brainstorming more options I think it makes sense to continue with the thick milkshake idea.
We can spend the extra minute to sell the vision of this idea. Imagine you’re a trucker and you’ve been driving for the last three hours. You pull into a McDonalds for your break and you grab a burger and fries and a shake. You have the option of a regular shake, which you know you’ll finish by the time you leave. Or you can buy a new “thick shake” which costs more, but which is much slower to drink. Instead of leaving empty handed, you’ll leave with your “thick shake” and you can slowly drink it while you’re on the road. In terms of cost per time getting enjoyment from your shake you know you’re getting a great value.
Start by validating your assumptions #
The thick shake idea is actually pretty easy to start testing, because it doesn’t require any additional ingredients. As an MVP I would suggest replacing the original milkshake with only thick shakes on a single trucker route, and seeing if that increases the sales of milkshakes. If that test was positive, I would begin testing the thick shake selectively in other markets (family markets, college towns, etc). If the results still come back favorable, I would begin optimizing the recipe and testing different flavors. Finally I would grow the thick milkshake by running a marketing campaign and introducing it globally (in the countries it makes sense for). And I’d change the name to the the Triple Thick Milkshake.
Summary #
- Empathize with the customer
- Human-Centered Design (HCD)
- Customer Personas
- Pain Points
- Startup
- Product-Market Fit
- Align with business strategy
- Matrix Management
- Objectives and Strategies
- Product Hierarchy
- Define success
- One Metric That Matters (OMTM)
- Data-driven
- A/B test
- Draw a Line in the Sand
- Apply the design process
- Double Diamond Design Process (DDDP)
- Brainstorming
- Don’t define problems in terms of solutions
- Use limitations to inspire creativity
- Tradeoffs and Leverage
- Vision and User Stories
- Start by validating your assumptions
- Build-Measure-Learn
- Minimum Viable Products (MVPs)
- Precedents
- Product trajectory