[HN Gopher] OKRs vs. KPIs: understanding measurements for your S...
___________________________________________________________________
OKRs vs. KPIs: understanding measurements for your SaaS business
Author : dundalk03
Score : 128 points
Date : 2021-11-06 13:30 UTC (9 hours ago)
(HTM) web link (userpilot.com)
(TXT) w3m dump (userpilot.com)
| jph wrote:
| I teach teams how to implement OKRs and KPIs. AMA.
|
| OKRs = objectives and key results:
| https://github.com/joelparkerhenderson/objectives-and-key-re...
|
| KPIs = key performance indicators:
| https://github.com/joelparkerhenderson/key-performance-indic...
|
| It turns out three templates can help greatly:
|
| OKR for growth = Improve {topic} by {percent} during {timeframe}.
| Measure by {metric}.
|
| OKR for capability = Launch {feature} for {benefit} on {date}.
| Track progress by {metric}.
|
| OKR for process = Accelerate {queue} from {x} to {y} before
| {deadline}. Time by {metric}.
| meekaaku wrote:
| I have read the John Doerr's Measure What Matters, and trying
| to implement some. Do you keep all OKR's visible to everyone in
| the organisation?
| jph wrote:
| Yes all OKRs are visible to everyone in the organization.
| Visibility builds mutual understanding, and also can be a
| great help for inter-team goals, team-of-teams projects,
| cross-functional areas, and roll-ups.
| ineedasername wrote:
| Do you have a goof basic resource you'd recommend for getting
| started in the process? I'm about to redesign a mostly ad-hoc
| operation to be more structured and formalized.
| jph wrote:
| Yes, my post links to GitHub repos where I maintain basic
| resources, examples, collected comments, and the templates
| that I use professionally-- see the "OKR examples by SixArm".
|
| The most important single item for success, IMHO, is to
| invite your whole team to get involved at the start.
| tomnipotent wrote:
| If you don't have at least 100-150 employees, I've found that
| OKR are an absolute nightmare and create more confusion than
| good. There are many better ways to align smaller teams and
| orgs.
| MatthewCampbell wrote:
| Do you have any sense of why? It works best for me on small
| teams and startups, since that's where we need people to
| own and iteratively address full problems. "Understand and
| address why users of type X aren't returning" is not too
| confusing, especially if you have good communication
| processes during the actual work.
| tomnipotent wrote:
| Does your OKR process span multiple teams and management
| layers?
| jms703 wrote:
| Love this. Thank you for posting!
| DougBTX wrote:
| What's the thinking behind this?
|
| > Keeping something static (like keeping a metric at a
| particular number) is not a key result.
|
| I could easily imagine a support team having an objective to
| keep the backlog of queries low. And to be pedantic, any metric
| can be expressed as "keep difference between metric x and
| expected/desired value y close to zero", so there must be
| something deeper here.
| jph wrote:
| IMHO that article quotation is wrong, and you're right.
| Static OKRs can be great for support teams, reliability
| engineering, service queues, recurring maintenance, periodic
| compliance, etc. Static OKRs can be as inspiring as dynamic
| OKRs, especially for highly motivated teams.
|
| A popular example is the objective of 99.999% uptime. This is
| a superb goal, and when you achieve it there's a ton of
| ongoing work to keep it-- it's akin to a fine restaurant that
| earns a Michelin Star, and works very hard to keep it.
| skeeter2020 wrote:
| I like the concise aspects of your OKRs but maybe they're too
| specific? It's hard to tell without seeing the motivating
| problem statements in context though.
| jph wrote:
| Yes you're right about concise and specific. These aspects
| help teams who are new to OKRs and who want to write good
| ones.
|
| The motivating problem is how to get teams to try: "I don't
| want to waste time with project management overhead
| buzzwords" or "I don't understand why we need this because we
| already have task cards".
|
| It turns out when teams see the concise specific templates,
| and write using that format, then the mental model clicks
| into place.
| alexchantavy wrote:
| How do you avoid OKRs for capability that are all-or-nothing?
| I've had the problem picking ones that stay at 0 for a very
| long time.
| underwater wrote:
| OKRs are Objectives AND key results, they're two different
| things. There is no such thing as "an OKR for growth".
|
| I've found it helpful to insert the phrase "as measured by"
| between the two. If I can't do that the there is either a
| tenuous connection or I've blended the two concepts together.
|
| For example instead of "OKR for growth = Improve {topic} by
| {percent} during {timeframe}. Measure by {metric}" You should
| have something like "Improve new user experience to create a
| loyal customers (aspirational objective) as measured by
| Improving onboarding completion by X%, Remove X steps from
| onboarding, Improve NPS for new users by X% (KRs)".
|
| Usually putting both a measure and a deadline is redundant.
| There is an implicit deadline (the OKR cycle length). Likewise
| if you're defining a deliverable and deadline then you may as
| well just use roadmaps. The idea of OKRs is to give teams
| flexibility to learn and change course rather than follow a set
| path.
| le205 wrote:
| I have found that KPIs are a useful addition to an OKR framework.
|
| Objectives are usually big and ambitious, but need to be pinned
| down using Key Results. Key Results are the definition of success
| for an Objective.
|
| However, it's often the case that there are many 'numbers' that
| affect the Key Results. In advance it is very difficult to know
| which combination of such metrics is the right one to target.
|
| For example, if a KR is to increase sales from PS70k to PS100k of
| product X, you can break this down into various other KPIs -
| traffic, inquiries, conversion rate, AOV etc
|
| In many cases these metrics can't be hard coded into a KR, as the
| sweet spot for the business is unknown - and ultimately we don't
| care which combination brings us to hit our KR.
|
| So KPIs allow us to play with the next layer down from KRs,
| setting and revising targets for the constituent numbers that
| ultimately drive a KR.
|
| Note: KRs should not change frequently, but KPIs can - with
| testing, iteration, and new information.
| k__ wrote:
| So OKRs define goals and KPIs measure the progress towards these
| goals?
|
| Seems like the PM equivalent to Op's SLAs, SLOs, and SLIs.
| skeeter2020 wrote:
| OKRs are essentially 2 parts. The "O" is an aspirational goal;
| The "KR"s are the measureable components.
|
| * Start with the problem statement and then make the objective
| to address it.
|
| * If you're always 100% on your OKRs they are not amibtious
| enough. The value is in the progress not "completion".
|
| * Don't make your Key Results a giant checklist of TODOs. It's
| OK to have figuring out the "how are we going to do this?" as
| part of the result to be measured, you just need to predefine
| how it will be measured.
| k__ wrote:
| So it's
|
| O -> SLA
|
| KR -> SLO
|
| KPI -> SLI
| yawz wrote:
| The unfortunate aspect of most (if not all) OKR articles is their
| use of very simple examples. There are a thousand reasons why
| real world is messier. In a changing environment goals give
| people the false sense of comfort to stick with a plan rather
| than review and adapt, not to mention the danger of optimizing
| things for the wrong KPIs. So surprise surprise! It's not as easy
| as it sounds.
| napoleond wrote:
| In terms of actually tracking and charting this data, it's pretty
| useful to have it all in one place and queryable by SQL--this is
| not usually the case by default!
|
| I built https://www.tabbydata.com for SaaS founders who have this
| problem.
| pm90 wrote:
| Apologies in advance for the rant.
|
| As an engineer, I'm really tired of these processes and metrics
| that are supposed to increase focus and enable delivering value
| but in practice often don't. A lot of organizational time is
| spent coming up with OKRs only for them to be either overridden
| due to the reality of running a business (gotta do what clients
| need, gotta do what you can to compete with competitors) or just
| by VPs/Directors deciding that something else requires the teams
| focus. Ultimately it seems like a whole bunch of busy work for
| PMs and TPMs, to justify their existence and hiring more of them.
|
| "But you're not doing OKRs correctly!" Sure, I agree to that. But
| what use is a system or process that's so difficult to get right
| that most organizations fail? I would argue that it is useless.
|
| I've seen this happen with another process: the church of agile.
| The way I've seen agile being used effectively is for orgs to
| agree to use the parts of it that make sense for them, not worry
| about what's "right", and constantly try out new ways of
| improving in response to feedback. That gives the team a sense
| that it's a process that is evolving to meet their specific
| needs, rather than evolving to meet a dogma.
| helge9210 wrote:
| "Plans are useless but planning is indispensable" (c)
| shane_b wrote:
| I'm not really a fan of agile but I like the simplicity of OKRs
| if done well. It's not easy and time consuming like you
| mention. My guess is it rarely gets done correctly.
|
| One of the main benefits of OKRs imo is helping each person
| understand where they fit into the big objective. I don't see
| the value in management using them to set lofty goals or push
| their team.
|
| In threads that complain about work, I see a lot of comments
| about a desire for ownership of their work. OKRs can play a
| role in describing a persons ownership.
| andrewflnr wrote:
| > But what use is a system or process that's so difficult to
| get right that most organizations fail? I would argue that it
| is useless.
|
| This is also what extremely difficult but fundamentally
| important problems look like. Getting clarity about goals
| across a big organization where every single person has a
| different conception of the problems it faces is Very Hard. I
| don't know if OKRs are the right answer, but the right answer
| will definitely reflect the Very Hard nature of the problem
| it's trying to solve. Probably the most any methodology can do
| here is trick you into asking the right questions.
|
| (I think Agile is an entirely different story. But remember
| that agility is only easy if you have solid developers, and
| hiring is in general Very Hard, especially for the kind of
| people who muck up Agile.)
| qqtt wrote:
| I think it's totally fair to see OKRs as silly and useless as
| an engineer. Personally I think they are really only relevant
| at the VP level and above.
|
| All the things you say are true about OKRs, they run up against
| the reality of the business, and at the end of the day are
| fungible, but I would encourage a different way of looking at
| OKRs.
|
| They inspire a conversation about how the company should really
| be spending its time and resources. What are the ultimate goals
| of what the business is doing. What is the real opportunity.
| How do we define success and how can we push success further?
| What's the dream by investing in these various initiatives?
|
| Once you are leading an organization of several people, each of
| whom has their own individual success depending on some of
| these decisions, it behooves you to take this type of planning
| seriously and spend some time trying to answer these questions
| at some cadence (once or twice a year at least) to solidify the
| high level goals - even if there is constantly the risk of
| these OKRs getting trampled by the realities of the business as
| you eloquently put.
|
| The purpose of having these conversations about OKRs at the
| leadership level is a bit different than the pragmatic benefits
| of the agile process and the two are a bit perpendicular in
| terms of purposes. Agile is on the ground execution, dealing
| with and greasing the wheels for the day to day efficiency of
| small teams. OKRs are occasional shocks to the system
| attempting to steer a large barge on course towards a far off
| goal.
| bar_de wrote:
| You are right. I experienced the same cult-like fanatic
| adoration of the process with rather detrimental results for
| the company's IT department similar to the times of "agile will
| solve everything and heal cancer" which bit us like a holy hand
| grenade.
|
| Maybe we computer people are, like anybody else, also in need
| of a Shepard and a holy book?
| 0x4d464d48 wrote:
| "That gives the team a sense that it's a process that is
| evolving to meet their specific needs, rather than evolving to
| meet a dogma."
|
| In my experience the most dogmatic organzations tend to just
| cargo cult their methodology.
|
| There's no real deep understanding for "why" it works for
| whatever MAANG they're trying to mimic. There's just a hollow
| pursuit of vanity metrics and busy work.
|
| There's value to be had with good work methods but it's almost
| always when people craft it to serve the unique value their
| organization creates rather than blindly doing whatever it is
| that Google does.
| version_five wrote:
| Philosophically, it's important to have something concrete to
| work towards.
|
| Like you say, what happens in practice, no matter what the
| system, is that it takes on a life of it's own, and there is a
| management layer discussing, planning, monitoring, etc thr
| KPIs, OKRs, whatever, as if tending the metrics is the goal of
| the business instead of delivering the underlying work. And
| this pisses off the engineers (or whoever the actual "doers"
| are, because it makes management look like a big circle jerk
| instead of a useful function - and a lot of the time, this is
| true.
|
| Personally, I can see emphasis on these metrics and their
| associated processes as an important part of a process driven
| organization, where you want everything to be consistent, and
| are willing to sacrifice efficiency (and job satisfaction),
| while carrying a heavy admin layer. This is probably true for
| most big companies.
|
| Personally, I think the solution is individual responsibility-
| less management and more focus on specifications. Basically
| treating each employee as a subcontractor or tradesman and
| telling them what you want done instead of how to do it, and
| having money or %utilization as a feedback mechanism. But they
| only works with the right mindset, and if you're the government
| or a big Corp, it's probably easier to just spend the time on
| OKRs
| civilized wrote:
| I liked the distinction made here between objectives and key
| results. Objectives are the general direction you want to go, key
| results are how much progress you want to make in how much time.
| In other words, key results set the planned pace in the direction
| of the objectives.
|
| If the business is a vehicle, the business plan is the intended
| velocity vector. The objective is the direction of the vector and
| the key results are the magnitude (speed). And the KPIs measure
| the actual realized progress in the planned direction.
| theptip wrote:
| OKRs are hard. A common error is to couple the O too tightly to
| the KR.
|
| Case in point:
|
| > Objective: Reduce body fat % Key result: Reduce body fat by 5%
| by the end of the year
|
| I really don't like this form of OKR pair. The O is arbitrary
| here. Ideally leadership should convey the O, and the team can
| meaningfully exercise their experience to come up with KRs.
|
| In this case you are just specifying the KR. For example the O
| might really be "Get in shape for the calendar photo shoot", and
| another meaningful KR would be to increase your lean body mass a
| bit. But not if you are training for an ultramarathon!
|
| Anyway, that's perhaps a nit that is orthogonal to the point of
| the article. As to the article itself - I'm not sure I'm
| convinced by using both KPI and OKR. As I learned it, there is no
| reason not to set up KRs that are maintaining instead of
| improving; for example the SRE team might have the O: "keep the
| site up", with one of many KRs: "99.99% API uptime".
|
| I'm a fan of conceptual simplicity and I just don't see a win for
| a third category above "plain metrics" that the team tracks and
| "KR" that the company tracks. The main strength I can see to the
| OKR&KPI approach is it lets you hide your KPIs in some contexts
| and really drill down on the few true KRs you have left. But
| whatever works for your team, go for it.
| sroussey wrote:
| The objective should allude to a why. So why do you want to
| reduce body fat percent?
|
| Let's say you want to fit in your clothes again. Make that the
| objective. The KR is a how for the O why.
| chiefalchemist wrote:
| What do both have in common? K. That is, Key.
|
| It's easy to compile and track minutia. To get sucked into "isn't
| this data and analysis interesting? And fun?" We've all done it.
|
| But time and resources are finite. So it's better to focus, and
| not not try to eat the whole elephant all the time.
|
| The key is Key. Or maybe it's, The Key is key.
| falafelite wrote:
| Solid read! An additional resource for OKRs is a book called
| "Radical Focus" by Christina Wodtke. Here is a link to it on
| goodreads: https://www.goodreads.com/book/show/28951428-radical-
| focus
| vageli wrote:
| Eh, that book was riddled with spelling errors and could have
| been a blog post, in my opinion.
| lowkey_ wrote:
| To agree with the sibling comment, another better alternative
| may be "Measure What Matters" by John Doerr:
| https://www.goodreads.com/book/show/39286958-measure-what-ma...
|
| He has a lot of respect in the tech world, being a leader at
| Kleiner Perkins who invested early in Google, Amazon, and many
| others. He's the person who really popularized OKRs, so the
| book has some great examples.
___________________________________________________________________
(page generated 2021-11-06 23:01 UTC)