[HN Gopher] User Disengagement
___________________________________________________________________
User Disengagement
Author : skulblock
Score : 155 points
Date : 2022-05-06 10:27 UTC (12 hours ago)
(HTM) web link (zerodha.tech)
(TXT) w3m dump (zerodha.tech)
| plinkplonk wrote:
| Zerodha built a market leading company with conservative tech
| choices, about 30 devs, and no VC funding. Conservatively
| estimated to be worth about $2 billion, last I checked.
|
| Very hard to pull this off, especially in a market like India,
| with a lot of stupid money folks chasing companies with flimsy /
| non existent business models.
| BIackSwan wrote:
| Zerodha has had the good fortune of ending up building a good
| product whilst the market needed it.
|
| If the pandemic had not happened, the growth would not have been
| what it is right now and that is when "build it and the users
| will come" philosophy stops working.
|
| The engagement metrics do work, but just like everything else in
| life, if you overoptimize for any one thing, it is detrimental
| (eg- drinking water in excess is harmful).
| thorn55 wrote:
| This is wrong.
|
| Zerodha was already India's largest broker more than a year
| before the pandemic happened[1]. The stuff mentioned in the
| post isn't something they're just starting to do now; they've
| been doing it since the beginning. They did benefit immensely
| from the pandemic stock frenzy, but they were very successful
| much before it happened.
|
| I've been a user for more than three years, and have never seen
| a single pop-up or ad or basically any kind of nudge from
| Zerodha regarding signing up for any of their products. There
| are third-party add-ons (like Tickertape), but they explicitly
| make people aware that these are separate products with their
| own terms, conditions and fees if any. This is very much
| appreciated by users, and they tend to have fewer complaints
| per user in comparison to the competition[2].
|
| They've grown primarily because they're good at what they do,
| and there is no catch regarding any of their products. Rest of
| the financial industry in India is a cesspool with a massive
| buyer-beware attitude, which is an instant turn-off for most
| people, but there weren't many other options until recently.
|
| The one thing I find wrong about the post is that they aren't
| track-free; I notice Google Analytics being blocked on ublock
| origin. Hard to disagree with anything else.
|
| [1]:
| https://economictimes.indiatimes.com/markets/stocks/news/ris...
|
| [2]:
| https://www.chittorgarh.com/report/broker_complaints_exchang...
| harshal wrote:
| As a happy Zerodha user since 2017, I agree with all of this.
| Even if the pandemic would not have happened, Zerodha would
| still be India's largest, most profitable broker.
| Nextgrid wrote:
| I don't think "engagement metrics" work. I mean, the objective
| of any business should be to make profit, so the metric that
| should matter is ultimately the average revenue per user.
| Engagement may in some cases correlate but why use a proxy when
| you can use the most important metric directly?
|
| "engagement" only comes into play when you don't have a
| business model to begin with (and instead hope to carry on
| raising more VC money or sell to a bigger sucker) so that's the
| "best" you can do because your actual ARPU is negative.
| BIackSwan wrote:
| They do work. You can draw a straight line from "Engagement
| metrics" to a working business model that generates real
| revenue. There are several B2C products that derive their
| value from engagement. And it is frequently a leading
| indicator of how viable a product can be.
|
| But like all metrics, one should not over optimize on any one
| metric - be it engagement, revenue, growth rate etc.
|
| The biggest issue has been a lack of balance between
| engagement and usability which unfortunately has been
| dominated by engagement over usability.
| shishy wrote:
| I think engagement comes into play in both scenarios (having
| and not having a business model) and is really detrimental in
| the latter case.
|
| But if you have a business model then looking at engagement
| metrics can be incredibly useful in informing a lot of
| product decisions and shouldn't be disregarded. Just a
| mistake IMO to optimize for it alone in the absence of other
| things.
| msluyter wrote:
| I don't know anything about the company but I applaud the
| sentiment. But is it obvious to note that a lot of companies
| _start_ with similar ideas... and then any number of forces --
| the market, shareholders, the board, management changes, black
| swans -- cause the screws to tighten? And then the ideals begin
| to erode.
|
| I guess I'm sort of implying (without any empirical proof) that
| there are darwinian reasons for state of things that are
| inherently difficult to avoid.
| Ozzie_osman wrote:
| Business models (aka incentives) are often more important than
| culture or intention.
| Ozzie_osman wrote:
| I love this piece. I philosophically agree with most of it (in
| fact I wrote a previous HN front-pager* called Disrespectful
| Design).
|
| But a bunch of the advice is just not practical because of, well,
| human psychology. For example, there's a lot of research showing
| that in competitive markets, people lazily choose the product
| with the most visible brand (or that is most accessible, ie in
| your face at the shelf on the supermarket--you can draw analogies
| to virtual products).
|
| Or, for products with infrequent use-cases, users won't remember
| to come back and use your product when the time comes. Zillow is
| a good example of this. The real value is when you are buying,
| selling, (or renting) a house, but that happens so infrequently.
| So they've built other ways to keep their app on your phone and
| their brand in your mind, like the Zestimate score (where
| homeowners or potential home buyers will check that score on a
| more frequent basis).
|
| Also, engagement tracking _is_ kind of critical to
| developing/understanding your product.
|
| So it's not so much about what you do and don't do, it's more
| about _how_ you do it. Do you track engagement AND solely
| optimize it?
|
| In any case, loved the article and agree with most of the
| discussion... just would encourage people to take a more
| pragmatic method to avoiding the engagement trap.
|
| *https://somehowmanage.com/2020/09/13/disrespectful-design-us...
| itronitron wrote:
| >> engagement tracking _is_ kind of critical to
| developing/understanding your product
|
| I disagree, engagement tracking is just a lazy stand-in for
| actual user studies and interviews.
| Ozzie_osman wrote:
| I never said it's a stand-in for actual user studies. I said
| it's critical. You need both. User studies will never give
| you the broad, hard data you get from engagement tracking.
| They also won't give you as fast feedback. People have all
| sorts of biases in how they present themselves in user
| studies, and you should get that data and try to sift through
| it, but it's not a substitute for actually seeing how people
| behave.
|
| Also, engagement tracking is actually really hard to do well,
| so it's definitely not lazy. But it definitely is shallow.
| PaulHoule wrote:
| It's pained me to listen to people who have elaborate ideas of
| promoting disengagement with their web sites (e.g. apologizing
| for having ads) in the context that engagement is such a
| precious thing because it hard to get.
|
| If there is an "art of seduction" there is also an art of
| driving people away and the two are closely linked. Soren
| Kierkegaard wrote his notorious _Diary of the Seducer_ to drive
| away his fiance Regine Olsen who never quite fell out of love
| with him despite Kierkegaard demonstrating what an awful person
| he was to come up with that text.
| tareqak wrote:
| This blog post reminds me of older HN posts about in the Silicon
| Valley area schools try to be more low-tech/hands-on in their
| learning whereas schools elsewhere get Chromebooks and ed-tech
| offerings to the brim. Both the "Only do what is truly useful and
| meaningful to end users" and "Don't do unto others, what you
| don't want done unto you" characteristics are relevant because of
| how technology is dropped on the laps of teachers and students
| without necessarily have a clear direction of how to improve
| educational outcomes.
|
| I don't know if this is at all still the case.
|
| https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
| zxienin wrote:
| > We charge users a fee, .. to sign up. This friction causes
| users without a strong intent to do financial investments to drop
| off. .... The mere act of signing up does not create a user.
|
| This is so obvious. It amazes me, that one needs to swim against
| the tide, on something as obvious as this.
|
| this is a good post.
| danuker wrote:
| > Choosing a financial services firm to manage one's personal
| finances is a critical, long term decision.
|
| This is not true for every industry. For example, a lot of
| mobile games are "freemium".
|
| It might be worth offering free services to some people, in
| order to get them hooked. Maybe that way more end up signing
| up.
| salawat wrote:
| Ya know... I'm reminded that the only other industry that
| uses terms like 'users' and 'hooked' is drugs.
|
| When did software and computing become drug dealing?
| danuker wrote:
| If you ask me, it's when smartphones started outnumbering
| PCs. There is almost no software freedom on smartphones.
| lykahb wrote:
| I agree with this. The time spent in app is roughly proportional
| to the number of ads watched and the revenue.
|
| I noticed this shift shortly after when the internet connectivity
| become ubiquitous, in late 2000s. The older software designed to
| work offline may be clunky by the modern standards, but it
| honestly tried to meet the needs. There was an economic incentive
| for that too: a satisfied user is more likely to buy the next
| version or recommend the program to friends.
|
| The connectivity and easy online payments have created new
| sources of revenue that are tied to how an app is used - the
| microtransactions and ads. It is inevitable that the dark
| patterns have followed. It is admirable when a business chooses a
| more ethical business model at the cost of revenue, even if the
| most users may not recognize it. But it will never be widespread
| until the structure of incentives changes. Zerodha has a better
| standing to do this choice because the stock brokers have revenue
| largely independent of the engagement.
| marc_io wrote:
| > No marketing team. $0 marketing spend.
|
| Blogging is marketing too, so this statement in particular is not
| accurate.
|
| In fact, I would say that this very post was quite convincing and
| a good marketing -- and branding -- example.
|
| It made me want to know more about this company and "engage" more
| with it.
| abhayhegde wrote:
| But not written with an intention to market themselves I guess.
| If merely expressing their true intents works in their favor
| while not boasting about it, I would not call it marketing.
|
| Nevertheless, it nearly costs $0 to host the site, so even if
| this _is_ marketing, no spending on it.
| marc_io wrote:
| They are spending time to host the site, to maintain the
| domain and to write and review blog posts. This cleary has a
| cost.
| bibliographer wrote:
| We are working on a B2B application right now and we're seeing 2
| linked patterns we're looking to overcome:
|
| -- one strong champion in the customer organisation that uses us
| actively, then changes jobs / goes on maternity leave / retires
| -> customer churns
|
| -- a champion in the customer organisation does not find the time
| to onboard other users in their company, so the value of what we
| offer is smaller than it could be.
|
| For us, getting more users onboarded and introduced to the value
| offering (we're calling it engagement) seems like a perfectly
| valid challenge to address -- does not feel like a dark pattern
| at all, but is "useful and meaningful to the end users".
|
| We're also tracking "engagement" with different features in the
| platform to help us decide on 1) onboarding gaps, 2) whether the
| features are any good (both, of course, then researched further
| with interviews). Here it also feels like our work qualifies
| under the author's core tenets.
| infogulch wrote:
| I noticed that you scare-quote "engagement", rightly so imo.
| Delivering value to the user is the end, engaging them to
| enable them to extract value is the means. "when means becomes
| an end" is a good description of the toxic dynamic of that
| occurs when engagement metrics are optimized at the expense of
| value delivered which plagues our industry today.
|
| Maybe expanding the term to allude to this dynamic would
| justify dropping the scare quotes; 'valued engagement', 'value
| delivered engagement' or something.
| Belphemur wrote:
| We have the same issue in the B2B company I worked at.
|
| Currently we're implementing strategies to reduce the barrier
| of entry by making the product onboard the new customer. We
| just started and we still have a long way to go but I think
| it's the right solution.
|
| Reducing the cost of onboarding for the customer and making it
| easy to invite other member should help avoid churn.
| jw1224 wrote:
| I'm in exactly the same boat with my B2B SaaS -- particularly
| with the champion finding time to onboard the rest of their
| team.
|
| Our app takes a bit of setup time, and I put a lot of work into
| making a really user-friendly "onboarding guide", basically a
| checklist of things to do to set up the account, accompanied by
| visual call-outs for each step in the UI.
|
| Before we had this, we forced companies to complete the full
| setup before they could use the app... As you might expect,
| many just gave up.
|
| But since adding our onboarding checklist (and changing our
| platform to assume lots more defaults, reducing the number of
| actions required to get started, and allowing it to be used
| "out of the box"), we now have a new problem: more companies
| are using us actively, but many of them never complete the full
| setup, so it isn't fully tailored to suit them.
|
| The checklist UI dismisses itself automatically after all the
| steps are complete (there's only 5, and honestly it takes 2
| minutes, but there we go). I often look at active user accounts
| over 6 months old, and so many of them are using us daily but
| with a big "Getting Started" checklist _still_ following them
| round the site!
|
| I guess that's users for you... Not sure how to improve this,
| practically speaking -- we've already reduced the checklist
| down to the bare essentials -- but open to any ideas...
| XorNot wrote:
| Personally I hate those getting started checklists. They tend
| to be full of a bunch of stuff I observably do _not_ need to
| do to use the product for my job, because I never have to do
| them to do it.
|
| I know in the past I've just ad blocked the UI elements for
| those lists because they contain things which don't matter -
| or worse, which I _can 't_ do because the organization has
| disabled that feature for regular users in the company.
|
| I'd be pretty curious to know what's on your list - I bet
| even with 5 items, your data is probably actually telling you
| you might not need it at all.
| timinou wrote:
| Curious to know whether that checklist is framed in terms of
| things to do or benefits they would get. I would guess they
| would perceive it as less daunting if you present them with
| what they would concretely gain from doing the thing.
|
| No idea what you'd do beyond that though... Maybe I introduce
| some competitive aspect, like "you're already more involved
| than 60% of our users. You can get to 90% with this"? Or
| translate it in terms of real value gained from finishing the
| onboarding? Idk
| maicro wrote:
| I'll reply to you, though there are a couple others in this
| comment chain that it seems relevant (but possibly not...) to
| - Airtable has one of the best API guides I've seen, because
| you can load it up in a table you already have, and it
| creates examples _with your table_.
|
| So no longer do you have docs that say "suppose you want to
| create a new entry for a sedan, which can be either red or
| blue", but instead has content from your actual existing
| table.
|
| It sounds like a small thing, but it greatly reduces that bit
| of cognitive dissonance/overhead ("why are we talking about
| cars, I want to update the contact options for this
| customer") when trying to learn a new system (especially
| critical for less tech-savvy users), so might be applicable
| to other products.
| josefrichter wrote:
| I'd say it's about trying to design it in a way where they get
| maximum utility without any "onboarding". Of course there's a
| fine balance where a tiny bit of "onboarding" is helpful and
| feels natural, but examples of that are very rare.
| josefrichter wrote:
| Great piece. At least the philosophical first half. As a
| designer, I often see raised eyebrows when I say the user should
| get into our app, get stuff done in 5 seconds, and gtfo, ideally
| without even consciously realising they used the product :-)
| pizzaknife wrote:
| this is an excellent read. thank you
| amigogg wrote:
| Engagement metric is a measure, but should it be a goal depends
| on the product. e.g. for google search low engagement is better,
| but for youtube low engagement might mean good or bad depends on
| the users objective
| eimrine wrote:
| > I have come to realise that the term user "engagement" in
| software, more often than not, is a thinly veiled proxy for user
| entrapment, whether intentional on the part of software designers
| or not.
|
| I do not think anything is veiled. Good software does lots of
| work and grab a little attention while bad software needs more
| attention than necessary for the sake of business needs.
| encoderer wrote:
| Another peril with making engagement a north star kpi is that you
| quickly get into a local design maxima. I saw so much of this in
| my time at a well liked consumer tech company. It becomes almost
| impossible to evolve around new business requirements because the
| current thing has been perfectly "optimized" for "engagement".
|
| Here's an example.. imagine a photo carousel with a critical
| button beside it that drives revenue for the business. At some
| point a PM spent a few months "optimizing" the photo carousel.
| Subsequent changes all _tanked_ engagement on that magic button.
| Experiment after experiment all thrown out because they didn't
| result in enough button clicks, despite huge improvements in
| photo experience.
|
| Eventually I took over the team that owned those little revenue
| buttons and even more eventually I figured it out:
|
| A lot of users fly through the photo carousel clicking the right-
| arrow very fast. At the final photo the right arrow disappeared
| and there was a small layout shift and it put a portion of the
| revenue button to where a portion of the ">" button used to be.
| Many many users _accidentally_ clicked that revenue button.
|
| You can't a/b test yourself to PMF and organic growth, even when
| used appropriately it is only accurate but not precise.
| ant6n wrote:
| If u want to maximise revenue, wouldn't it make sense to
| measure the revenue coming from those button clicks as well as
| the button clicks? Perhaps that would be more precise.
| encoderer wrote:
| 2-sided marketplace, revenue not directly attributable
| e3bc54b2 wrote:
| While zerodha is on the good side of the spectrum, I have to call
| out couple of claims from this post.
|
| No marketing is a half-truth at best. Their app often sends
| notifications about blog posts and upcoming trades (IPOs etc that
| user did not initiate). So much so that I had to disable
| notifications for the app.
|
| No ads claim is similarly bull. There are ads for various
| (startups?) firms who supposedly augment zerodha's product right
| on the console (where various reports are shown).
|
| I'd reiterate, compared to various abominations out there,
| zerodha is better. But to claim they don't do marketing or
| advertising is not really true.
| rounakdatta wrote:
| Your idea about marketing might not be entirely correct.
| Sending out notifications on important announcements (could be
| regulation-related changes, acknowledging an outage, confirming
| that a trade was successful) is indeed for keeping the user
| better informed. Sometimes Kite also gives out notifications
| for new Finshots blogs - which again deeplinks to a web URL,
| not the app.
|
| The "ads" you are talking of aren't really ads. They are
| partner startups who solve specific problems around the same
| financial domain. Like Sensibull, Quicko and all. I believe
| it's a nudge to the user to check out some of their partners
| (and often their invested-into startups) who happen to share
| similar philosophy with Zerodha.
| JonChesterfield wrote:
| That sounds exactly like an ad. What's the distinction?
| dr-detroit wrote:
| kkfx wrote:
| Most users do not understand what software is for themselves:
| most do not even understand the difference between local and
| remote, proprietary or FLOSS, perceiving just the UI, locally
| seen, as a thing in their hands...
|
| The sole possible positive engagement is in a community, so
| FLOSS, mandatory by law, a PUBLIC academia who led research so
| most new ideas come from public money, for public interests and
| the private sector just pick what it want but do not led the
| development toward dangerous situations. Something that can't
| happen with enough Citizens comprehension of the issue and it's
| meaning for our society.
|
| In a near future entrapment can mean you are locked out of "your"
| car due to a private third party service issue, you pacemaker can
| cease to function from a remote order or you cant be identified
| because some private-owned "public" ID service does not work
| properly, ... you? ... Oh, yeah, us, all.
| toss1 wrote:
| Excellent approach - I hope it is part of a trend.
|
| I also wonder if after decades of VCs pumping trillions of
| dollars into apps and platforms seeking shallow 'engagement' user
| metrics, and at least the vanguard of the population starting to
| get sick of it, and it becoming a zero-sum-game, the landscape
| has fundamentally changed.
|
| E.g., iirc Google originally used the metric of short user time
| on their site to indicate success - the users who quickly found a
| useful result would navigate away from Google sooner. They grew
| rapidly with this approach because people found G actually
| useful, not engaging, and came back. I wonder (hope) if
| development & funding will more focus on abandoning superficial
| 'engagement' metrics in favor of actual utility metrics - such as
| briefer and repeated usage vs your competitors' products.
| skybrian wrote:
| Google minimizing its own screen time is smart, but I think
| it's harder to encourage this at the ecosystem level. For
| example, a useful but flawed metric for deciding whether a
| search link is good is that the user clicks on the link and
| doesn't come back for a while.
|
| A user bouncing off a website immediately shows that it's
| probably a bad choice for them, but unfortunately _not_
| bouncing off it sometimes means that the website is gaming the
| metrics by wasting people 's time through whatever means.
|
| In particular there are websites that are automatically
| generated in a way so you don't figure that out immediately.
| They look like they might answer your question, but under
| closer inspection it's nonsense.
|
| Videos that look like they might answer your question are
| another example of a time-waster.
|
| In a way, websites where you can see immediately that it's not
| what you're looking for are better than the ones where it's
| unclear. Best would be actually finding an answer. That can't
| be judged by screen time, though.
| usrn wrote:
| People have to stop measuring things and need to start
| justifying things with arguments.
|
| I like to compare "data driven decision making" and "reason
| driven decision making." Animals make data driven decisions,
| you don't want to behave like an animal. Sure it's better
| than being blind but your data _should inform an argument_
| not be the argument.
| toss1 wrote:
| Yes, these are real problems - it is a whole lot easier to
| measure 'engagement' with the proxy of [screen time in our
| app], which is a very good proxy, whereas measuring app
| effectiveness is much harder as you point out.
|
| The problem is that measuring 'engagement' is like the drunk
| looking for his lost key under the lamppost because that's
| where the light is, not because that is where it will be
| found - and almost he entire industry is using the wrong
| metric like that hapless drunk . . .
| Lightbody wrote:
| This is a great article, but like others have said, difficult in
| practice to pull off. But doesn't mean we (the industry)
| shouldn't try.
|
| My own personal experience: I have a calendar assistant startup
| (http://reclaim.ai if you're curious) that does a bunch of
| awesome things. And we designed it to work _where you already
| are_ -- namely in Google Calendar.
|
| But a _lot_ of potential investors, for example, question our
| definition of "active user" when we tell them we consider the
| customer getting value if we blocked out and defended time for
| their lunch as their schedule filled up.
|
| Conventional wisdom is that the user isn't getting value unless
| they are coming back to your UI and getting your notifications
| etc. But that's just not the case for us: we helped our customer
| manage their time better without ever requiring them to come back
| into our app.
|
| Isn't that like... the best possible thing? Solving a problem
| with the least amount of friction? It's working for us and
| fortunately we found investors who agree, but there are a lot of
| people out there that think about "engagement" in fairly
| simplistic terms that I think holds back our industry and also
| hurts the customers.
| duxup wrote:
| I agree / like this article.
|
| I wonder about measuring something related to both engagement and
| disengagement.
|
| Some customers will disengage because they're not going to be a
| customer, never were, and it just wasn't going to happen.
| Recording that type of disengagement and saying "oh no that's
| bad" is a mistake.
|
| I think they do sort of address that with "We charge users a fee,
| albeit nominal, to sign up.". That seems to narrow the field a
| great deal and seems useful to find the folks who are good
| potential customers. Very nice.
|
| I've worked at so many B2B places where they chase big company or
| rando crap company for business and you can just tell that "These
| guys are not a good fit, if they walk that's not bad, it's good."
| But measuring that / understanding it is hard.
|
| Related story. I recently heard about a perspective customer, a
| big company, could be a lot of revenue. But then I found out they
| pretty much ran on spreadsheets emailed to each other, had a
| whole staff whose job it was (as I understand it) was to be a
| human form of "git" to rectify conflicts as they got emailed
| around... and the spreadsheets were never right anyway. That was
| just the tip of the iceburg. A demo call involved the folks at
| this big company arguing over what they even DO now... just the
| most basic things. It was ugly.
|
| The issue is we're a small company and these guys needed a
| consultant to come in, be the bad guy, tell everyone what to do
| (and take all their money) and probably fail at it before their
| management really realizes what is going on ... thankfully they
| passed on us.
___________________________________________________________________
(page generated 2022-05-06 23:01 UTC)