https://themarketplace.guide/dont-overreact-to-weak-signals The Marketplace Guide * Introduction + Comparisons to other books * Management and Planning + Decide if it's a business or a hobby + Don't drink the Koolaid + Building a marketplace is like training for a marathon + Investor value of a marketplace + The tech roadmap should follow the business roadmap + Bug priorities should relate to your ability to collect money + Focus on what if it succeeds + Keep track of why decisions were made and features built + You're not FAANG so don't hire like FAANG + Take hiring seriously + Don't create silos + What it really means to make data-driven decisions + Mistakes to avoid when assigning ownership of metrics + Get good at cohort analysis + Removing things is good + Be careful with customization for a particular client + Convert, grandfather or alienate * Product + Don't half-ass many things + Deciding what not to build is nearly as important + Not everyone's opinion is equal + Observed users personas + You might get customers that you weren't planning on + Two ways of doing design + Don't overreact to weak signals + Your team are your users too + Separate the trackers for admins + Pick role models and copy common features + Product development is process development as well + Think about how new things can affect what you already have + Consider the complementary interaction + Each job should have a custom dashboard + Understand your value to your users + Learn the correct jargon and use it consistently + Simple user permissions are very useful + Build a simple flexible dashboard for viewing stats over time * Technical + Build polite software + You can get quite far with a small tech team + Take security seriously while you're small and no one is likely to exploit you + Take testing seriously as you go + Take performance seriously while it can't cripple you + Use an error tracking tool as early as possible + Implement a background task processing structure + The more documentation you can generate automatically, the better + Consider adding these attributes for every new model + Interface should behave consistently + Generate more data than you think you'll need + Know when to denormalize columns + Maintain a hierarchial structure for settings + Have admin editable platform settings from the start + Implement an audit log + If dealing with money get as close as possible to double entry bookkeeping + Build and use a robust import feature + Build and use a robust export feature + Give admins the ability to comment on everything + Admins need fudgability + Use fewer tools but more completely + Be prepared to roll features out in stages + If the changes you're working on will require downtime, split them into two + Use different keys for different parts of the cache + Users double click everything + Build stats beginning with the rows + Create a separate stat model + Async selects are easy performance wins * Cheatsheet * Further Reading * Contributing + Roadmap * Contributors * Buy the PDF Subscribe to updates --------------------------------------------------------------------- The Marketplace Guide (c) 2021 by Ognjen Regoje [ ] 1. Product 2. Don't overreact to weak signals Don't overreact to weak signals It makes you build poorly thought out and potentially wrong things. Your main job is to look for signals from data Based on these signals, you need to set the direction of the tech and product. But, when you are new to product development, you cannot yet tell when a signal is strong enough. You end up over, or under, reacting. In my experience, over-correcting is more common. Over-reacting to signals that aren't as strong as you might think can be harmful. Over-reacting makes you build the wrong thing Over-reacting happens when you make decisions prematurely. Often, based on incomplete or insufficient data. When you don't have enough data, you can't tell whether what you're seeing is a signal or just variance. Some examples of over-reactions * Considering one instance of user feedback to represent a pattern * Considering a recurrence to indicate a fundamental mistake in architecture * An instance where a feature is abused by disabling it entirely * A slight change in numbers to represent a start of a trend * Connecting the causes of things just because they happen at the same time * Attempting to copy what your immediate competitors are doing * Doing things that aren't directly related to the objective So, you should carefully consider whether a signal is strong enough One way to counteract this is to look at percentages more than raw numbers. Seeing that 10 orders got rejected might make you think something is wrong. But if that's out of 10000, it's not a strong signal. Similarly, if you're A/B testing something, make sure you use a calculator that tells you whether your change in performance is significant enough. In Supplybunny, we had a rule of thumb that there should be at least three instances of the same piece of feedback before we brought it up for discussion. Then we'd look at the full picture to see how prevalent it really is. Two ways of doing design Previous Your team are your users too Next Further reading * Don't scar on the first cut - Signal vs. Noise (by 37signals) 37 signals explaining not making institutional changes from one-off instances Suggest an improvement to this page (me@ognjen.io)