Chris's Blog

The below is the first chapter of a paper I’m writing as part of my Managing Innovation class at Harvard Business School. The paper is a collection of Product Management best practices from tech companies such as Amazon, Google, IDEO, and Microsoft. It also brings in best practices from non-tech companies such as McKinsey and Target that I thought would be useful. This first chapter is specifically on idea generation and screening.

Huge thanks to my classmates Abhinav Agrawal (@agrawal667), Stephanie Christofides (@schristofides), Amit Garg (LI), Kishan Madamala (@kishan7), Minal Mehta (@mili2), and Arjun Ohri (LI) who made this paper possible. Thanks  for putting up with my interviews!

What do I mean by idea generation and screening?

Our first year curriculum introduces the funnel as a metaphor to think about an organization’s innovation system (like any business school, we’re big fans of funnels and 2×2’s):

(Illustration from Professor Roy Shapiro’s TOM wrap-up slides)

In any organization, numerous new ideas are generated and then screened with a select few implemented and shipped as products or services. This section of the paper is about how companies 1) widen the mouth of their funnel (generating more ideas) and 2) how companies screen ideas as they pass through the funnel.

Idea Generation and Screening Tips

Every few months, Facebook hosts an all-night Hackathon, where employees stay on campus and pull an all-nighter building prototypes. Facebook keeps a backlog of promising ideas generated by employees and submitted by users at hackathon-ideas@facebook.com. Project ideas are presented at the beginning of the night, teams (all with fun or goofy team names) are formed to work on each project with the goal of having a prototype that can actually be used and tested in the morning. All employees, even non-engineering related, are encouraged to stay the night. Employees are fueled by an unending supply of Red Bull, burgers, Chinese food, and IHOP. Hackathons have generated an incredible array of Facebook features including its internationalization platform, AJAX-ed wall posts, and type-ahead search functionality.

IDEO’s brainstorming process is fairly well-known and used in many organizations including Google, Microsoft, and McKinsey. The process is broken down into two phases: idea generation and idea screening. The idea generation phase involves a moderator who provides each participant a post-it note pad to write down a short phrase summarizing his or her idea. As participants come up with ideas, the moderator places the post-it notes summarizing each idea on an empty wall or whiteboard. What makes this process successful are the following seven rules:

1. Defer judgment
2. Encourage wild ideas
3. Build on ideas of others
4. Stay focused on topic
5. One conversation at a time
6. Be visual
7. Go for quantity

The idea screening phase is notably separated from the idea generation phase. This reflects IDEO’s belief that idea screening discourages idea generation, and should not be performed concurrently. To screen for ideas, each participant is given a set of small circular stickers (generally three stickers) to place on the post-it notes that he or she believes is most promising. Participants are allowed to put multiple stickers on a given post-it note or spread out their “sticker votes” across many ideas. It’s important to note that both the ideas generated and the sticker votes are unattributed to any participant. The ideas with the most stickers are the ones selected for prototyping. (Rodriguez)

3M and Google both allow engineers to dedicate a set percentage of time to pursue a side project that may be completely unrelated to their primary job. At Google, engineers can save up on their 20% time and take an entire month off to pursue their side project. Many successful products have come from this program including scotch-tape, post-it notes, and Google News.

Google has a mailing list that enables all employees to pitch, discuss, and recruit teams to pursue ideas. Google has a web site that tracks the number of ideas each employee submits, the average rating score, and the “buzz score,” which is based on the amount of discussion generated on the mailing list. These scores serve a good initial screen for ideas, and attempting to achieve a high score serves as a fantastic motivator.

Even in a company that prides itself on “letting its employees run wild”, Google has clear and well-understood screens for the ideas generated through its employees. Google’s idea filtering process involves the following six screens:

1) Rating and buzz scores from idea mailing list and screening site (employees with excellent reputations from previous successes can bypass this)

2) Exec support and funding (funding amounts can be small, often times a five-digit figure)

3) Internal adoption

4) Pre-beta, un-publicized “public experiment”

5) Google Labs: A more formalized, public experiment

6) Beta launch

McKinsey often conducts anonymous surveys to solicit and evaluate a set of ideas. This poll is anonymous both in the poll participant being anonymous, and the source of the idea being anonymous. This poll is quite effective at surfacing tensions in an organization between senior and middle management that may not arise otherwise out of a misplaced sense of loyalty and reluctance to criticize.

McKinsey occasionally brings in experts and clients from other industries who have faced similar functional issues to spur problem solving. Although these individuals do not have domain knowledge, their thoughts and fresh perspective often helps spark creative discussion.

Every season, John Galliano, the head designer at Dior, creates a scrapbook of things that inspire him for the coming season’s clothing line. This book includes building architecture, colors, or clothing seen during his travels. Designers go off independently to generate ideas and concepts using the book as inspiration. The designers meet together where John Galliano serves as the primary screen, selecting which ideas make it to production.

Assemble feature team and relevant experts across the company for a two-hour block of time. Break participants into subgroups of three or four to generate ideas. Have each subgroup present their ideas to the full group. Employ a moderator to write down all ideas that are discussed on a whiteboard and use the majority of the meeting to generate ideas. Dedicate a small portion of time in the latter half of the meeting to discuss criteria for screening ideas. Screening, however, is left to the feature team owners or scrum “pigs” to make the final call on which ideas to implement.

Create an “idea board” where you have a dedicated whiteboard or area placed in a highly trafficked team area. Have the subject of brainstorming clearly written and allow teammates to put up post-it notes of ideas they have during the course of a week or month.

Google and others have incredibly liquid internal labor markets. Employees can quickly and easily move from team to team and project to project.  This acts as an effective screening device for ongoing projects. If employees working on a given idea/project do not believe it will succeed, they are able to quickly abandon it and move elsewhere. This is an interesting means of killing off projects that are unlikely to succeed.

I don’t think there’s a more counter-intuitive math problem than the Monty Hall problem.

It’s based loosely on the TV show Let’s Make a Deal, but was made popular after a Parade magazine columnist was berated, by literally thousands of Ph.D.s and mathematicians, for her correct answer.

Here’s the problem:

Suppose the contestants on a game show are given the choice of three doors: Behind one door is a car; behind the others, goats. After a contestant picks a door, the host, who knows what’s behind all the doors, opens one of the unchosen doors, which reveals a goat. He then says to the contestant, “Do you want to switch to the other unopened door?”

Is it to the contestant’s advantage to make the switch?

The answer and reasoning for it is best described by Wikipedia here. I think the decision tree diagram explains it most clearly. The most mind-boggling aspect of the answer is that it’s a 2-to-1 advantage!

I completely forgot about this problem until I was reading The Drunkard’s Walk: How Randomness Rules Our Lives written by Leonard Mlodinow, who also co-authored A Briefer History of Time with Stephen Hawking. Mlodinow’s new book contains this and other “real-life” probability and statistics problems (How a mathematician won a small fortune by playing roulette; Use of the Prosecutor’s Fallacy in the O.J. Simpson trial; Why firing Merril Lynch’s CEO for losing on subprime investments doesn’t make sense) interspersed with a surprisingly interesting history of famous mathematicians.

I think it’s a fantastic book to read if you’re on a math kick, but a boring one to read if you’re having one of those days where you just don’t feel like calculating your waiter’s tip in your head.

It’s been a month and a half since I read my very first case at school (The case study method is the primary pedagogical tool used here, see link for 10min. video of what it’s like in the classroom). Since then, we’ve gone through about five dozen cases. They’ve been on everything from well-known companies & industries: Toyota, Dell, to companies in obscure industries: iron ore shipping, styrene-based cement coating, and finally to the absolutely bizarre: contact lenses for chickens. For each of them (even the contact lenses for chicken one), I’ve attempted to write down one sentence that summarizes the most interesting takeaway I got from the case discussion. Here are some of them (they’re not necessarily related to the primary lesson of the case) :

• Anytime you hear a statistic such as “1/3 of HBS alumni hold C-level positions”, remember this equally compelling statistic: “1/3 of HBS graduates get fired within the first 10 years of graduating” (statistic derived from HBS class of ’74 data).
• All leaders need to be able to play different roles (coach,cheerleader,disciplinarian), a good leader knows precisely when and where to play them.
• Surprisingly, it is actually possible to apply a linear regression in analyzing and shaping the culture of an organization.
• Accounting is not math, and it certainly is not  black and white. When reviewing any statement keep in mind the parties’ biases. Know that there’s always a trade off between accuracy and precision, and that accounting rules allow for a party’s judgement to come into play. THere’s no right answer, only differences in judgement.
• Always, always, look through the footnotes of a financial statements. There’s a reason why the footnotes are typically more than 30 pages long.
• When competing against a low cost/low quality entrant, consider creating a “fighter brand”, a brand which is completely disassociated from your existing brand. This allows you to maintain the brand equity of your premium brand.
• When creating a marketing strategy, you must take into account your competitors’ likely responses. For example, you may decide not to price low if that puts you at risk of entering a pricing war with a competitor with a significantly stronger balance sheet.
• When segmenting a market, don’t get stuck only segmenting based on the obvious aspects (basing it on existing products on the market, performance, price, etc.). There may be opportunity in targeting an underserved segment defined by dimensions which are not immediately obvious.
• Assumptions drive financial models: Never trust an investment banker’s numbers without understanding his or her assumptions.
• Michael Dell’s true genius is in his financial strategy. Dell is one of the few companies in the world that has managed to create a significantly negative cash conversion cycle (Amazon is another).
• It’s all about the benjamins. It’s better to evaluate companies based on their free cash flow projections, not their net income projections. Net income can hide hideous increases in net working capital and other bad things.

This week, the Bureau of Labor Statistics released its annual American Time Use Survey. This provides incredibly interesting (and somewhat frightening) data about how we spend our time. For example, here’s a direct quote  from the Bureau’s press release:

Conversely, individuals ages 15 to 19 read for an average of 5 minutes per weekend day while spending 1.0 hour playing games or using a computer for leisure.

The same demographic watches more than two hours of television per day.

After doing some quick math from Table 11, this means that:

In a given weekend, an American teenager will spend more time watching TV than he spends reading leisurely in an entire month.

If you’re interested in checking out the data, you can check out the two links above or:

• -Wall Street Journal has a great mini-app that allows you to visualize this year’s survey data.
• -New York Times also created a great mini-app visualizing last year’s survey.

I also threw Table 11, the data about how we spend our leisure time into Excel and played around a bit. Below are two charts I found interesting. What I find most interesting is that the amount of time spent using the computer for leisure is much lower than I would have expected. I’m curious if anyone has a good explanation?

I know that half of US online adults access the internet while watching TV (Forrester 2009), and double counting isn’t allowed in this survey. Or maybe it’s because people are actually spending most of their leisure computer access time during work hours?

This is a follow up blog post to my first post summarizing takeaways on how the best web sites convert first time visitors to new, loyal customers.

Actions are not mutually exclusive

The six actions described in the first post are not mutually exclusive. Often times you may want to present more than one action to a new user on a given page.

RedBeacon

RedBeacon, this year’s TechCrunch50 winner, has recognized that two actions are critical for their users: 1) Learn and 2) Use. They present these two actions, in that order, for all first-time visitors:

ZenDesk

Multiple actions seem particularly prevalent on enterprise sites targeting corporate customers. Trial or Contact actions tend to be present on any first page.

Since many of ZenDesk’s potential customers may be unfamiliar with a SaaS model for helpdesk software, it dedicates considerable screen real estate to Learn, in addition to the Trial that is typical of most enterprise-related sites:

Screen real estate should reflect the importance of the action

RedBeacon

RedBeacon, being a fairly new concept, dedicates roughly 2/3 of their prime screen real estate to Learn, followed by Use for the remaining 1/3.

Dropbox

Perhaps the most extreme example I came across was Dropbox. Most visitors wandering onto Dropbox’s site are a) unlikely to fully understand Dropbox’s value proposition and b) hesitant to download and install any out-of-browser software. To deal with these two issues effectively, Dropbox dedicates more than 80% of their prime screen real estate to the Learn action with a fantastic video. The remaining 20% of the screen real estate is dedicated to the Use action.

Order Matters

It’s important to keep in mind that the order in which you present actions matters. For example, though not always the case, most of the time it makes sense for Learn to be presented first, before Trial or Use.

The order may differ across customer segmentations (e.g. Contact may be optimal for one customer segment, while Trial may be optimal for another). For those targeting international users, recognize that the notion of order depends on your user’s cultural perspective. Presenting certain actions from left-to-right might make sense for English-speaking customers, but for Arabic or Hebrew-speaking customers, it likely won’t, since these users read right-to-left.

Combined Takeaways:

Post #1:

1. There is a fairly simple taxonomy of user actions: Use, Register, Learn, Guide, Trial, and Contact.
2. Getting new users to stick involves two difficult things: a) Knowing which of these actions is “sticky” for your users and b) Incentivizing users to perform those actions.
3. This is highly dependent on the nature of your product and who your users are. For example, “Register” would be ineffective for TheSixtyOne when visitors likely have little patience to register for yet another online radio service.
4. Don’t think about your web site as a series of pages that visitors see. Think of it as a series of actions you want visitors to take.

Post #2:

1. Actions are not mutually exclusive. It often times makes sense to present more than one action on a single page.
2. Screen real estate dedicated to a given action should generally reflect the relative importance you assign to that action.
3. The order in which actions are presented matters a lot. Make sure you understand the optimal order of actions to get users to stick. The optimal order may change based on your customers’ cultural perspective, or differ across customer segmentations.
4. When in doubt, monitor and test. Most of the sites listed above collect and analyze extensive metrics on their user’s behavior. The sites listed above are also likely utilizing some form of an A/B testing framework to methodically increase conversion rates along each step in the funnel.

I have recently analyzed more than two dozen web sites attempting to understand how the best ones convert first time visitors to new, loyal customers. The following two blog posts summarize my takeaways.

These two posts are only about the early portion of the visitor “funnel”: the first few actions that users take on a given web site.

(Image Credit: Modified from an image originally from webdesignseo.com)

The Six Basic Actions

I analyzed what my colleagues and I subjectively consider the best sites for getting new users to stick. I was surprised to find two things:

1. All the possible user actions presented on these sites fit neatly into six categories
2. Most sites tended to focus on only one or two of these actions.

Use

Some sites like Aardvark, Google’s recent acquisition, get users to stick by just getting them to use the full product.

Aardvark

Aardvark simply gets visitors to start using it: No detailed how-to guide. No explanation of how it works. Delayed registration.

It just gets visitors to use it, in this case, by asking a question:

The Sixty One

TheSixtyOne.com is perhaps the most impressive example of a great “use” action. This Pandora-like streaming radio site presents a simple, elegant splash screen:

As you’re reading the three lines of text, it caches the audio stream and the beautiful background image so that when you click ‘ready’ it instantly begins playing music.

No sign up. No instructions. Not even a choice for what type of music you like. It just starts playing music, immediately.

Only after using the service, do you discover features which require registration, and more complex features which require explanation.

Register

For other sites, registration is the most crucial barrier in converting visitors to new customers.

Mint

Mint has done an incredible job at making registration painless and easy. Amazingly, with Mint, it’s possible for a user to link all his financial accounts to Mint in under five minutes.

OkCupid

OkCupid limits each registration page to exactly three questions to keep visitors from feeling overwhelmed. Pre-populating answers is another great way to invite visitors to chug through the registration process. It can reduce the number of clicks necessary for a critical demographic of your visitors, in this case, single women. But it can also be psychologically compelling for other users to correct information about them that is incorrect.

Additionally, OkCupid dedicates half of the screen real estate to keeping users entertained with humorous or interesting facts as they register.

Takeaways:

1. There is a fairly simple taxonomy of user actions: Use, Register, Learn, Guide, Trial, and Contact.
2. Getting new users to stick involves two difficult things: a) Knowing which of these actions is “sticky” for your users and b) Incentivizing users to perform those actions
3. This is highly dependent on the nature of your product and who your users are. For example, “Register” would be ineffective for TheSixtyOne when visitors likely have little patience to register for yet another online radio service.
4. Don’t think about your web site as a series of pages that visitors see. Think of it as a series of actions you want visitors to take.

See Part 2 of this post for more.

I’ve been spending a lot more time doing front end web things, and found the following to be quite helpful:

Avoiding Helvetica and Arial

• Google has a great hosted collection of OpenType Embeddable web fonts and an API available: https://developers.google.com/webfonts/

Visualization

• D3 is fantastic: http://bl.ocks.org/mbostock/1256572
• Rickshaw is a convenient abstraction of D3 for basic graphs: http://code.shutterstock.com/rickshaw/
• Google Charts for something less flexible and pretty, but well baked: https://developers.google.com/chart/

UI Frameworks

• Twitter’s Bootstrap framework is great for layout and basic components: http://twitter.github.com/bootstrap/
•  JQuery (and JQuery UI) are staples for DOM manipulation and AJAX: http://jquery.com/ , http://jqueryui.com/
• LESS for more dynamic CSS: http://lesscss.org/
• TimeAgo, a JQuery plugin for fuzzy dates: http://timeago.yarp.com/
• Google also hosts a bunch of the above for you here: https://developers.google.com/speed/libraries/devguide?hl=fr

First set of course material for the semester:

I saw the below DeBeers ad tagline today and found its message to be confusing: