The Tool that Will Help You Choose Better Product Ideas

When working with PM's we are often tempted to make a decision for our ideas based on weak signals: majority votes, highest-paid person opinion (Hippo), industry trends, etc, but those have been shown time and again to be bad heuristics that are not any better than putting chips on a roulette table (hence the term “Big Bet”).

Instead we could use the ICE score:

Also Intercom uses similar practices like RICE (Reach, Impact, Confidence, Effort)

ICE Scores

ICE score is a prioritization method invented by Sean Ellis, famous for helping grow companies such as DropBox and Eventbrite and for coining the term Growth Hacking. ICE scores were originally intended to prioritize growth experiments, but can also be used for regular project ideas.

You calculate the score, per idea, this way:

ICE score = Impact * Confidence * Ease

• Impact is an estimate of how much the idea will positively affect the key metric you’re trying to improve.
• Ease (of implementation) is an estimation of how much effort and resources will be required to implement this idea. This is typically the inverse of effort (person-week) — lower effort means higher ease.
• Confidence indicates how sure we are about Impact, and to some degree also about ease of implementation. Prioritization by impact and effort is not enough. The short version is that we’re very bad at estimating both and we’re blissfully unaware of this. Confidence values are the antidote — they keep us honest about our assumptions. The only way to increase confidence is by doing Exploratory Research with user interviews (either in person or unmoderated).

Calculating confidence

There is only one way to calculate confidence — looking for supporting evidence. For this purpose you can use this tool shown below. It lists common types of tests and evidence you may have, and what confidence level they provide. When using it consider: what indicators do we have already, how many of them, and what we need to get next to gain more confidence.