https://destraynor.com/writing/2025/02/09/questions-i-always-ask-in-product/

Menu destraynor Go |

  * Writing
  * Speaking
  * Podcast appearances
  * Investments

<< Back to all essays

The questions I'll always ask in a product review

February 9th 2025

I figured some of these might be useful to folks earlier in their
career, certainly I wish I had them earlier, so here goes.

Big picture questions

  * Why did we build this? (Bad answer: because our competitor has
    it. Better: because we are losing deals to xyz who has a great
    version of it)
  * Who, specifically, is it for? (Bad answer: Sales. Better: the
    person in charge of reporting on online revenue every week)
  * What is the most ambitious version of this product? (here I'm
    testing for how well they've thought through the space)
  * How does this version compare to it? (here I'm looking for smart
    scoping decisions, 80/20s etc.)
  * What is the biggest assumption we've made that needs to be true
    for success? How have we tested that? (if there's a bet we're
    making, we should discuss it and see if we can ascertain things
    before going live)
  * What's the estimated impact here if this works out exactly as we
    planned? (numerically what will this feature do: drive sales,
    stop churn, change satisfaction, reduce support queries?)
  * What does this look like in 2 years and what needs to prove true
    for us to continue resourcing it? (here I'm thinking about what
    does 'done' look like, or will this be a feature we'll be
    building for the rest of our lives? e.g. reporting)

AI Questions

  * What's our confidence this feature will actually work for
    ~everyone in real world scenarios? What is needed to improve
    that?
  * How has it performed thus far? What's the worst outcome? What's
    the most amazing thing we've seen? What's the most common?
  * How will we know if it's not working live in production?
  * What couldn't we get working reliably, if anything?
  * What guardrails have we built here?
  * Do we have enough instrumentation to investigate issues in
    production?

Customer Feedback

  * What did customers say when you showed this to them?
  * What was the energy/vibe like - one of relief (e.g., "finally,
    guys!") or excitement (e.g., "holy shit!")?
  * What questions did customers ask? What wasn't obvious? What
    concerns came up immediately?
  * How long did it take for it to click with them? Was there an
    "aha" moment?
  * Were the words we used familiar to them?
  * How did this feature compare to their top 3 feature requests?
  * Did they read the launch notes? Did they understand what we
    claimed to have built before they saw it?

Product Management

  * What do you believe will be the #1 feature request in the days
    following launch? When is that shipping?
  * What's the most important thing we're not shipping on day 1? (not
    necessarily same as above)
  * What's the least important thing that we are shipping?
  * Does this juxtaposition make sense?

Design/UI Questions

  * How long does it take for a user to see value here? Can we
    shorten it?
  * What's the breakdown of tool time vs task time? Have we tried to
    minimize each? (UI improvements, 'quick start' templates)
  * How has the onboarding performed in testing? Are they engaging
    with it, or just clicking "next-next-next"?
  * What's the biggest barrier to adoption?

Engineering

  * What are the known issues on launch day?
  * Have we met our quality bar everywhere?
  * Does this affect performance of any other part of the product?

Analytics

  * What are our success metrics?
  * What are our leading indicators?
  * 24 hours in, what can we tell?
  * Are we running any tests on any part of this?
  * When is our first review where we can learn meaningful insights?

Customer Support

  * Have we given support a heads up?
  * Are we recording feedback on this feature specifically?

Marketing

  * Will this feature be added to our marketing site, sales
    enablement, and docs?
  * Did we consider adding this to our customer lifecycle marketing?
  * Do we need reference customers for this feature?

Back to all essays

---------------------------------------------------------------------
Find me on Twitter LinkedIn Instagram
---------------------------------------------------------------------
Find me on Twitter LinkedIn Instagram