Measure Twice, Cut Once - Criteria Definition is Key

June 30, 2016


Adobe Spark-4


Writing this blog post has led me down an interesting path and has reinforced a very valuable lesson: Criteria set up and definition is HUGELY important when setting up a portfolio. It doesn’t matter how many stakeholders I have or how many metrics I attach to my alternatives; if I didn’t set the stage properly, all of the data I collect isn’t going to matter…

With summer officially here, I set out to write about something light hearted, fun, and enjoyable (for those of us 21 years of age or older), BEER! Back yard barbeques, Fourth of July parties, and hot summer days are topped off with a refreshing beer by people all over the world. With so many to choose from, I wanted to gather feedback from my fellow beer-loving coworkers to find the best go-to beer for summer.  Simple enough task, right? Develop a list of criteria (flavor, aroma, appearance, calories, etc.), create a list of beers, put in some data about them and gather feedback. Easy, right up until I was discussing one specific criteria I added: Alcohol by Volume (ABV) – the percentage of alcohol contained in a given volume.

The question was posed to me, “What if someone thinks that a low ABV is more favorable than a high ABV and vice versa?” My head started to spin, the world around me came to a grinding halt; I hadn’t considered that when developing this specific criterion. More importantly, I didn’t take the time to define and put into words what I meant by ABV.

ABV is a known number, a metric, a fact provided by the producer – so what did I want to accomplish by including ABV? I wanted to know how people valued AVB when compared with another criterion: Which is more important, ABV or Calories? ABV or Flavor? I didn’t take into account my stakeholders having different preferences on low or high ABV which then led to a misunderstanding of the criterion and eventually some bad data.

The question I should have asked is, “Is the ABV desirable?” By changing my criteria name and including a clear definition, my stakeholders were better able to provide judgments that accurately reflected their preferences and ultimately gave me a better outcome.

The initial Criteria development in this, and all other portfolio prioritization efforts, is paramount. With the proper set up of my portfolio, I can quickly and easily gather the information I am looking for, do some really interesting data analysis, and provide my stakeholders with a prioritized list of our selections.

GET TO THE BEER ALREADY!!! And the winner is…

You’ll have to check back for my next post where I promise to give you the big reveal plus some really cool and interesting analytics.