[HN Gopher] Problems with homemade billing systems
___________________________________________________________________
Problems with homemade billing systems
Author : thibo_skabgia
Score : 160 points
Date : 2023-09-18 12:15 UTC (10 hours ago)
(HTM) web link (www.getlago.com)
(TXT) w3m dump (www.getlago.com)
| fmajid wrote:
| I've worked on telco billing systems in the past and the
| generally accepted metric in the industry is 70% of billing
| systems projects fail. Not fail as in "did not completely meet
| requirements or expectations", fail as in "did not deliver
| anything whatsoever".
| 1970-01-01 wrote:
| Creating and maintaining a billing system would be another good
| test for AI.
| Rafsark wrote:
| What's your vision? Can AI handle revenue prediction and
| billing engineering tasks? Both? How?
| 1970-01-01 wrote:
| Resolving the billing edge cases would be a test. There is no
| vision, it's a question: Is your flavor-of-the-week AI
| 'smart' enough to design and code a billing program for my
| startup? How about my medium-sized company with 4 offices in
| 3 different countries and 4 timezones?
| ftxbro wrote:
| This is an ad by a company that sells billing systems. This is
| what ads look like now.
| danenania wrote:
| A lot of useful content gets produced by companies as a form of
| marketing. Just like independent devs are often blogging with
| the goal of getting consulting gigs or advancing their careers.
| There's a lot of garbage too, but if the content is good and
| reasonably objective, what's the problem?
| tylergetsay wrote:
| Same with "Role your own auth"
| dboreham wrote:
| I see what you did there.
| [deleted]
| 542458 wrote:
| Good article, and I generally agree that people are too quick to
| say "Ahh, this looks like a two week project" and ignore broader
| complexity. I've definitely seen a fair amount of that.
|
| My question is, how do I know that the OTS solution would
| actually be much better? For example, OP describes some tricky
| migrations from one type of billing to another, or complex
| grandfathering schemes - how would you have any guarantee that
| your third party solution would be able to support those? At
| least if it's in-house you can implement the needed functionality
| _eventually_ - if it's third-party it might just be flat-out
| impossible.
| ineedasername wrote:
| >how would you have any guarantee that your third party
| solution would be able to support those...
|
| You could say the same thing about literally any 3rd party
| software or service you purchase. "Why use cloud services to do
| anything?" etc.
|
| Sure this thinking will hold for _some_ things, but chances are
| that for something as standard as billing you 're not going to
| be implementing some revolutionary feature _that gets you more
| customers_. On the other hand, if you mess up billing because
| you rolled your own system that doesn 't have many standard
| options that a mature OTS system would offer it _can_ lose you
| potential customers, because for months at a time you 're stuck
| building the most mundane and standard things like an annual
| plan that could be used to better appeal to buyers.
|
| For billing, a one of the most standard things ever for a
| business, you simply take a small amount of time to survey the
| OTS options and compare features. Any you know what? There's an
| excellent chance that buy actually getting a comprehensive look
| at features offered by such options you will learn about the
| types of things you will absolutely want to do in the future.
| Things that, in retrospect, seem like obvious oversights, such
| as an annual plan. You'll see features like that in your survey
| and say to yourself "Oh, yeah, that's something we may
| definitely want in the future, lets put that on our requirement
| list for a final choice."
| scott_w wrote:
| We migrated to Zuora and, while there are limitations, they're
| far less than your own limitations. Honestly, if you can't
| model it in Zuora, what you can do is likely close enough that
| it's not worth the 100x extra effort to build your own system.
| singron wrote:
| Sometimes the fact that you can't change it is a great feature.
| Obviously if it's missing a critical feature, then you are
| screwed, but often these requests for customization are made
| because it seems like they are quick and cheap to make and it
| will be automated and free after a one-time cost. However,
| having a whole team of engineers run your billing department is
| a lot more expensive than hiring someone in finance ops who can
| do a manual refund or spend an extra 5 minutes every quarter
| dealing with anniversary billing. Especially if you are B2B,
| you are going to run into that huge client that wants to be
| billed net 37 on the lunar calendar, and it's better to get
| someone in finance to send invoices if necessary each week than
| to calculate moon phases in your product.
| specialp wrote:
| Well one thing an OTS solution accomplishes is it makes costs
| and capabilities very clear to product and marketing. When you
| have an internal team doing something bespoke to your
| organization it is viewed as much more malleable.
|
| If the third party solution cannot support the billing system
| devised, or will support it for a large cost, companies are
| more likely to choose things that are more in the box.
| cjonas wrote:
| Except with many of these enterprise solutions (SAP,
| Netsuite, Salesforce) most companies will still need to pay
| consultants 300 dollars an hour to configure and maintain the
| application. These costs often exceed the license cost
| itself.
|
| Bottom line, complexity is expense regardless if you build or
| buy. Companies need to factor the systems and operations
| impact of complicated pricing and billing models into the
| decision making process.
| kayodelycaon wrote:
| The absolute cost isn't important here. What's important is
| that cost is now explicitly on the balance sheet were
| someone can see it. If you have an in-house system, that
| team is treated as an infinite supply of features for no
| apparent cost. (Ask me how I know.)
|
| And as a developer, I love being able to hand certain
| people over to vendors who are used to dealing with those
| certain people. Hell, if we picked the right system, those
| vendors will have useful domain knowledge I don't have.
| re-thc wrote:
| > Well one thing an OTS solution accomplishes is it makes
| costs and capabilities very clear to product and marketing.
|
| It gives that illusion. The reality is not so clear cut.
| There are lots of nuances to a capability that's not just a
| box check. It could be half-done, i.e. certain bugs in
| certain scenarios. It could be slow, unreliable or something
| else altogether.
|
| There's also: https://www.businessinsider.com/former-vp-
| claims-salesforce-...
| Spooky23 wrote:
| Sometimes unlimited choice is a vicious thing.
|
| With the power to modify anything, people will make decisions
| and choices with serious consequences and benefits that may or
| may not be tangible, or worth the squeeze.
|
| I've spent most of my career working in and around government,
| and poorly scoped or framed legislation costs billions because
| it drives customization and development. The business and
| engineering impacts of those decisions are often not
| appreciated.
|
| If you work for a corporation, and you have a off the shelf
| billing system that fundamentally does what is needed... you
| have a chance of using ROI or some other metric to save the
| (expensive, hard dollar) customization for when there is real
| value.
|
| The other thing is that sometimes these custom systems create
| customer problems in the name of "helping". I've had many times
| where some legacy SKU or billing model, left in place to make
| my life easier instead made things much much more difficult
| down the line.
| ineedasername wrote:
| >how would you have any guarantee that your third party
| solution would be able to support those...
|
| You could say the same thing about literally any 3rd party
| software or service you purchase. "Why use cloud services to do
| anything?" etc.
|
| Sure this thinking will hold for _some_ things, but chances are
| that for something as standard as billing you 're not going to
| be implementing some revolutionary feature _that gets you more
| customers_. On the other hand, if you mess up billing because
| you rolled your own system that doesn 't have many standard
| options that a mature OTS system would offer it _can_ lose you
| potential customers, because for months at a time you 're stuck
| building the most mundane and standard things like an annual
| plan that could be used to better appeal to buyers.
|
| Survey the main OTS options and see what they can do. It will
| probably give some ideas like "Oh yeah annual plans, we'll
| probably want them at some point". At the very least you do
| this at the outset when you're small. Worry about complex
| billing requirements that won't be part of vanilla options in
| most OTS systems once you're big enough & complex enough to
| require them. Otherwise you're worrying about a Maserati
| problem.
| emilfihlman wrote:
| Indeed. Very often organisations tend towards buying solutions
| without appropriately gauging the situation. One easy example
| from IT is buying servers vs renting them, but fortunately it
| seems that IT people have found the, for some uncomfortable,
| balance of "it always depends", and the analysis might turn
| multifaceted pretty quickly.
|
| I also feel that people often discount the cost of up top
| decisions that people below don't like.
| jasode wrote:
| _> , how do I know that the OTS solution would actually be much
| better? For example, [...] how would you have any guarantee
| that your third party solution would be able to support those?_
|
| The business (employees in the accounting dept and/or
| consultants) do a "fit-gap analysis" when evaluating potential
| software: https://www.google.com/search?q=fit+gap+analysis
|
| For the "gaps" of missing functionality, look at either :
|
| (1) customizations via extra programming or extra add-ons.
|
| (2) eliminate that software as a viable choice and move on to
| the next OTS software for evaluation
|
| _> At least if it's in-house you can implement the needed
| functionality eventually -_
|
| The vast majority of Fortune 1000 businesses replaced their
| "homegrown" accounting and payroll systems they built in the
| 1960s/1970s/1980s -- with COTS ERP systems like Oracle
| Financials and SAP R/3 because their internal IT teams _never
| got around to the "eventually" option_.
|
| There's an anecdote (I think from the book "I'm Feeling Lucky")
| about a Google's early startup years where an employee said
| they "needed to buy SAP" for accounting. The co-founder Sergei
| Brin was dismissive, _" why do we need to buy that when our
| programmers can build it?"_ Well, he was 20-something at the
| time and naive about the complexities of financial accounting
| software for a global business. He must have eventually
| realized the true scope of the problem because instead of
| building in-house accounting software, Google bought Oracle
| Financials. About 20 years later, they migrated from Oracle to
| SAP.
| re-thc wrote:
| > Well, he was 20-something at the time and naive about the
| complexities of financial accounting software for a global
| business. He must have eventually realized the true scope of
| the problem because instead of building in-house accounting
| software, Google bought Oracle Financials.
|
| How do we jump to this conclusion? Did he realize the true
| scope of the problem or was it just left to someone else once
| Google grew in size? It doesn't even explicitly say how
| Oracle Financials came to be at Google.
|
| > because their internal IT teams never got around to the
| "eventually" option.
|
| I doubt that's really the case. Same with how cloud came to
| be. The sales people promised massive savings i.e. bonuses
| for management and here we are.
| p_l wrote:
| Don't forget the money sinks that are any attempts to
| implement something differing with SAP or other big ERP
| systems.
|
| You delivered extra value for customers (making you more
| competetive) thanks to some particular ability of your
| previous (even manual) billing rules? Kiss them bye bye.
| 6510 wrote:
| HAHAHA, my estimate was 2 weeks. This was 6 months ago. It is
| now a marvel of engineering with most of the 100 000 edge cases
| covered. You should read that as: not battle tested on (insane)
| users. (I'm so looking forwards to that) I estimate 2 years to
| completely finish it, give or take a decade or two.
| MightyBuzzard wrote:
| [dead]
| [deleted]
| vidarh wrote:
| I've built several billing systems and managed a team that built
| a much larger one, and it's not _that_ hard, but it does require
| an attitude and attention to detail that a lot of people lack.
|
| E.g. one thing that worked well in the systems I've worked on was
| to break things into _small_ state transitions, _very_ firmly
| document the allowed states and their transitions, and _log every
| transition_ to the database. Wherever possible (anywhere that
| didn 't interact with external API's, and sometimes even when
| they did) we'd aim for state transitions to be idempotent. When
| they weren't, we'd seek to reduce the non-idempotent call to an
| external dependency to _just_ that call as a separate state
| transition.
|
| When something very occasionally went wrong, we could trace
| things in detail, and pinpoint it, and most of the time we could
| safely replay transitions around whatever went wrong until we
| could see what had happened.
|
| It wasn't _difficult_ to build these systems that way, but it was
| _tedious_ , and something where it's easy for people to be
| tempted into shortcuts.
|
| As for scaling, you spit out events for rollups, and can shard
| invoice generation. We handled millions of revenue on a machine
| far less powerful than my laptop today. Scaling really would not
| have been up there for me.
|
| To #3, maintaining old pricing isn't generally _that_ hard. Very
| few people drastically change how they account for usage, which
| tends to be the most complicated billing scenario. They tend to
| change amounts and periods and thresholds, which should be data,
| not code.
|
| With respect to a team, I agree. One of the places I worked on
| billing was Yahoo. At the time it wasn't just one team, but
| several - my team (responsible for Europe) existed largely
| because the European finance and product teams didn't trust the
| US payment services team to take their requirements seriously
| enough and so wouldn't let them near the European premium
| services... Billing isn't something you want to do yourself
| unless it's either a core competency or you're big enough to have
| to deal with that kind of bullshit.
|
| (As for size of teams, I've built a billing system with 2 people,
| but I've also worked on billing systems managed by 50... You
| really want to have a good idea whether you're likely to end up
| towards the former or latter before you go ahead...)
| cvccvroomvroom wrote:
| Exactly. You really need experts in entity relationship data
| modeling (typically in Oracle DBMS), data hygiene, and
| codifying business rules. This isn't something web developers
| should generally take on. Software involving money should
| consider approximating the formality of engineering approaches:
| waterfall development model, formal verification, and extensive
| unit and integration testing. There are hidden costs that edge
| up TCO for DIY. It's possible COTS maybe very expensive or
| incapable of being customized sufficiently to meet the needs of
| the particular application, but DIY should generally be the
| last resort when no other viable solution exists.
| 6510 wrote:
| > #3, maintaining old pricing
|
| I copy the entire product into the order. The thing one buys
| should be the thing on the screen when the decision is made.
| Poor pictures, description with typos, wrong categories,
| old/wrong pricing etc
| wredue wrote:
| This approach is generally okay. It at least keep your
| mandatory data for audit around.
|
| One issue with it is that it often ends up forcing you to
| build out a lot of backward compatibility into your audit
| system as things change.
| vidarh wrote:
| Yeah, that can be a good approach. Including all the data
| defining the old product in an unchanging way was one of the
| sticking points that led to the creation of my team at Yahoo.
| The US team didn't think twice about changing invoice
| templates for examples, while the European finance teams
| broke out in hives at the thought of _anything_ changing on
| already issued orders, much less invoices. I spent three
| years holding the fort to ensure the US team didn 't take
| over invoice processing on our behalf until they could
| guarantee at least the invoices were unchanging...
| swader999 wrote:
| Everything has to be historically available. Think about
| returns and redoing the entire period and so on. Sometimes
| even the old code if you want to be perfect.
| oblib wrote:
| I've been working on an simple invoicing app for small businesses
| for over 20 years. It has become a somewhat complex project and
| my app does not do much of what's cited in this article.
|
| >> Now, when someone asks for advice about their billing system,
| my answer is clear: DO NOT build it yourself.
|
| Moving from using CGI.pm to CouchDB/PouchDB about 10 years ago
| made it much easier to manage users and their data, and implement
| the UI. It also moved most of the workload to the user's web
| browser and made the app much faster for users.
|
| CouchDB was practically designed for this. Early on their
| developers used "invoicing" as an example for using CouchDB.
|
| But... if you have a marketing team that keeps moving the goals
| it wouldn't matter what you built the app with, it's going to
| take time to build and debug it. Could be that CouchDB/PouchDB
| could make that easier, but my experience is developers who've
| only been using SQL may get frustrated with it.
| baq wrote:
| I'm working on an in-house billing system right now. My only
| comment is 'yes'.
| kosolam wrote:
| How about accounting, would anyone recommend a modern open source
| accounting system that would be better than in-house?
| Rafsark wrote:
| Not necessarily open source, but there are a lot of third party
| tools for accounting (netsuite, quickbooks, xero). And I would
| say they scale 100x better than an in house tool
| cmilton wrote:
| Why are we against modifying our processes to align with a
| particular software platform? Maybe it is much easier to follow
| such guidelines during the early stages of a company, but much
| harder later on. Do standards exist in billing systems?
| agentultra wrote:
| In my experience this happens frequently enough that there ought
| to be an article like this for every problem domain that software
| developers and executives under-estimate the complexity of.
|
| I work on card transaction processing and it's absolutely more
| complicated than you can imagine looking at it from the outside.
| It's a vast, distributed peer-to-peer system where trust is built
| into liabilities outside of the network itself. You want to build
| a system that is correct with regards to some specification in
| order to protect peoples' money... but the problem is that there
| is no specification that enumerates every possible transaction
| state because while there are "typical" sequences of messages to
| handle, your system also has to handle unexpected sequences that
| can't be specified: there are no guarantees that systems on the
| other end are emitting properly formatted events let alone
| behaving as expected. Card payment handling systems have to have
| the ability to manually correct and adjust state by humans;
| reconcile with external systems (because yay, payment systems in
| the US aren't based on instantaneous settlement until the roll-
| out of FedNow is complete), etc.
|
| I've watched many businesses walk straight into, _it 's just X,
| how hard can it be?_ Only to watch their ARR shrink and stress
| levels rise when a project they thought would take a month turns
| into a year-long journey of discovery, reflection, and increased
| head-count.
| madjam002 wrote:
| I've found Temporal and Xstate are helpful at building a reliable
| billing system
| Rafsark wrote:
| Could you explain how? I know temporal has not built the
| billing engine themselves, so wondering how the tool can help
| with this..
| hydroid7 wrote:
| I think it depends on the business you run. To be honest, I would
| not mind building a billing system if my company grows to an
| unicorn!
| Rafsark wrote:
| How do you know in advance it will grow into a unicorn? I tend
| to agree with you, but that's a shame to have a team of 30
| engineers working only on billing, even if you are a unicorn..
| I would prefer to have them working on my product
| pmontra wrote:
| I worked for a mobile phone operator many years ago. I was there
| when the company was bootstrapped. We built our own billing
| system because we would be billing for stuff that incumbents
| didn't bill for, because they didn't offer those services: we
| were the first 3G operator of the country and we also had to
| create and sell contents (example: YouTube was not born yet.)
|
| I was not in the billing team but we definitely followed rule #4
| of the post: "You must be prepared to staff an entire team".
|
| And yes, #1 "Pricing changes all the time and billing needs to
| follow".
|
| About #2 "Your billing system needs to scale with your user
| base", we had to go from 0 to 3 million customers in 9 months to
| be viable. We were not typical, we made it.
|
| No idea about #3 "Grandfathering causes headaches" but there were
| probably many headaches, that one and others.
| brianmcc wrote:
| I would instead say - commit properly to building your billing
| systems as a major and ongoing project, don't dismiss it as a 2
| week "fun" side thing.
|
| Not convinced any 3rd party product is going to have the
| flexibility to support ongoing innovation. If anything it's more
| of a case for in-house expertise and resource. Find some folks
| who _want_ to be the billing guys!
|
| Disclaimer: I built a large in-house billing system :-)
| agos wrote:
| see also: e-commerce systems. Hard to build, a lot of work, but
| way better than any 3rd party
| calvinmorrison wrote:
| Interestingly, fastmail just flipped to having an external
| billing system after years of maintaining their own. AND when
| they bought POBOX they got that whole billing stack as well. I
| can imagine the nightmare but also the relief after exporting
| all of that work to another company.
|
| perl billing system: https://github.com/fastmail/Moonpig
| re-thc wrote:
| > Interestingly, fastmail just flipped to having an external
| billing system after years of maintaining their own.
|
| Would have nothing to do with your or someone else's
| decision. And Fastmail may or may not have made the right
| decision. It's hard to say. There are no simulators or time
| machines to gauge the impact anyway.
|
| The most likely answer is preference and available skillset
| in the area. I've seen managers that absolutely want to self
| build and those that never want to maintain anything as much
| as possible (by offloading).
| TomNguy wrote:
| How do you monitor for tax compliance changes, especially for
| international usage?
| Rafsark wrote:
| I think billing, taxes and entitlements are 3 different
| products, with obviously strong relationships. I would rather
| give the advice to rely on a third party tax tool that is
| connected to the billing engine instead of the "all in one" if
| taxes are too complex. Otherwise, just use the basic tax codes
| of Lago's billing engine
| crawsome wrote:
| This is an ad
| lowercased wrote:
| Why do I have the feeling that many people starting projects like
| this get in to situations where functionality is identified and
| they're beat down with "YAGNI!"? I've been in situations where
| 'build v buy' comes up - accounting, inventory, and similar
| domains. Identifying "hey, we need to keep XYZ data to allow for
| future reporting..." has been met with chants of "YAGNI" from
| people blind to the complexities.
|
| It's just one of the things that might lead me to opt for 'buy'
| in 'build v buy', but... it's hard to trust the sales people
| involved in the process as well. And... taking the time to do
| _real_ and _full_ analysis... is time people often don 't want to
| pay for.
|
| And... once you're "in" to a COTS system... you're almost never
| leaving. The companies know that, the salesfolks know that, and
| have a lot of incentive to handwave away concerns, or just
| outright lie. Once you discover the lies... it's likely too late
| to switch.
| hk1337 wrote:
| That situation is why I don't like about the software
| development process. YAGNI gets abused to go with the sloppy
| shortcut that only really saved maybe a day in creation but in
| reality wastes months worth of maintenance.
| wredue wrote:
| >Once you're "in", you're almost never leaving
|
| Yeah. This problem is real. It's so expensive to get out that
| we have had to make business process concessions because the
| COTS ended up not supporting, what I consider, basic
| functionality.
|
| We only found out years later on the basis of changing other
| business processes.
|
| And vendors will bleed you dry if they can. What some vendors
| consider to be "customizations" is insane.
| hrunt wrote:
| Having gone through /exactly/ what this article is about, I know
| the pain. But I will point those that don't read the article to
| the last paragraph:
|
| > We considered implementing an off-the-shelf billing solution
| but there was nothing flexible enough and the switching costs
| were too high. Algolia also tried to migrate to Zuora before
| backing out and rebuilding their billing system for the fourth
| time.
|
| Most (all? I have yet to find one) off-the-shelf systems are not
| geared towards everything businesses want to do with their
| billing. If you take the author's advice and choose one of these
| systems early on, you will have the exact same headaches. Either
| the system simply won't let you do what you want to do or the
| system /will/ let you do what you want to do with custom or hack-
| ish solutions that reproduce the custom-solution problem, but in
| someone else's system. Those that implement broad swathes of
| billing functionality are so complex, they make /everything/,
| even the most basic stuff, hard.
|
| BTW, when I say I know exactly how the author feels, all of what
| they described we encountered and implemented. We even looked at
| a migration to Zuora and came away with the same conclusion
| (also: really freaking expensive). We even had a new product that
| we setup on a third-party billing system, and we ran into the
| flip-side of the problems; we were not able to implement some
| billing functions we needed and we had to migrate off.
| 6510 wrote:
| > there was nothing flexible enough
|
| They would have to solve the same problem in a generic way.
| (add 10 more years)
|
| > we had to migrate off
|
| Adds to the fun doesn't it? My missing features hacked around
| previously exploded into an almost impossible puzzle.
| akitzmiller wrote:
| I came here to pretty much make the same comment. If you have a
| business that can get away with OTS software, more power to
| you. But complicated things are complicated and complex billing
| rules can be make-or-break for a company. Whether you start
| with 3rd party or start home grown, billing systems often need
| humans and there really isn't a substitute.
| jabart wrote:
| When we launched in 2017, we went "in-house" with our billing. We
| found an open-source project and uses that for all billing as it
| had a recurring option and a billing portal. Fast forward a
| number of years, we now sync invoice data in the app and have a
| tiny billing portal there. We can still adjust invoices, keep old
| pricing, setup new pricing, auto-bill, check balances to see if a
| customer hasn't paid yet. Total Lines of Code here are like under
| 2000 for this integration and it's great. Also zero vendor lock
| in as well. Just another way to approach a billing system.
| jasonjayr wrote:
| What open source project did you use?
| jabart wrote:
| InvoiceNinja. The install instructions had a few misses but
| has an API and webhooks so it's great to integrate with.
| freefaler wrote:
| Like with everything, wrap your billing in a layer and when you
| grow enough you put your billing under it if you need it or
| switch to a different provider without changing your codebase.
|
| It's not wise to outsource parts that are essential for your
| business, because: 1. you depend on 3rd party service for a
| getting paid 2. your operations cost may rise or the service may
| change terms/functionality
| bluGill wrote:
| You always need to outsource things. Unless you live on a
| desert island alone some things are out sourced. Most of us out
| source barter to a government run cash system, which we then
| further outsource to various banks.
|
| While out sourcing is a risk, it is also an opportunity to
| offload the hard work. You need to figure out what is worth
| doing yourself; what you outsource completely and ignore; and
| what you out source and audit.
|
| Accounting - including billing, is normally something you
| should out source and audit. Let someone else worry about all
| the hard and weird rules. However you need to audit it because
| if you do not someone corrupt will steal your money and you are
| still required to pay whoever you owe money after that happens
| - this is one way your company can go bankrupt.
| freefaler wrote:
| That's why you need an abstraction layer for each 3rd party
| provider. I've been doing online businesses 20+ years and
| every time we didn't create an abstraction layer our
| switching cost rose.
|
| Many "startups" who are hungry enough to provide good value
| for money switch to "enterprise" pricing when investor money
| run out and your operations costs may rise a lot or you get
| "discontinued". Look what Atlassian did with self-hosted
| versions.
|
| Also for tracking/logging layers it may be beneficial to
| mirror the data you're sending to 3rd party providers to your
| own db/logs, because you won't get data locked when
| switching.
|
| So my point is that you need to outsource smartly and keep in
| mind that the 3rd party dependencies increase operations
| risk.
|
| BTW that's why any user request shouldn't depend on 3rd party
| resources in a blocking matter (e.g. synchronous JS from
| vendors' CDN). You might pay more for your hosting but if you
| factor the service disruption risks you may be better off for
| mission critical parts.
| [deleted]
| swader999 wrote:
| Now take your system and make it multi tenant. Imagine the horror
| of doing that. This is what you often get with "off the shelf".
| There's a lot of broken dreams either way you go.
| siva7 wrote:
| I was that 20-something guy tasked with building a homemade
| billing system. Hint: Don't.
| Rafsark wrote:
| Even if you find that guy, you'll have to find 20 more in the
| next 3 years if the company grows
| marcosdumay wrote:
| If you are not afraid of it, it means you don't know nearly
| enough to be successful.
|
| Maybe there are 20-something people out there with enough world
| knowledge to make one. But it's not a safe bet at all.
| say_it_as_it_is wrote:
| Taxes alone are complicated enough to warrant the existence of
| TaxJar. Billing systems should not be underestimated.
| Rafsark wrote:
| This is completely true. Taxes, entitlements and billing can be
| 3 different products. I don't understand how people think it's
| a 3 week effort. Avoiding this separation can lead to a messy
| setup down the road.
| zer0x4d wrote:
| Billing in general is very hard. Especially subscription billing.
| Just from my experience in building billing and also later on
| using a third party system for another projects, I can mention so
| many edge cases:
|
| 1. Grandfathering
|
| 2. Upgrading across different length memberships (monthly tier 1
| to yearly tier 2)
|
| 3. Crossgrading across different lengths (monthly tier 1 to
| quarterly tier 1)
|
| 4. Offering free trial to tier 2 when user has tier 1.
|
| 5. Promotional offers to non-paying users (half off first month)
|
| 6. Downgrades
|
| 7. Applying coupons to accounts (customer support wants to offer
| a specific user a free month of service when user has a quarterly
| subscription: billing schedule change)
|
| 8. Differences in tax regulations across countries (VAT vs whole
| value tax)
|
| 9. Differences in display prices (in some countries displayed
| prices should include tax)
|
| 10. Handling currency exchange rate changes.
| 1-6 wrote:
| Sounds a lot like Autodesk.
| Rafsark wrote:
| I could add: - metering - relations with accounting/finance
| softwares - timezones (if your customers are spread around the
| globe) - proration - payments and dunnings
| mattboardman wrote:
| It looks like one core issue was designing the system around pre-
| determined bounds (monthly, yearly, etc). This happens in lots of
| systems and isn't specific to billing. A job scheduling system
| will have the same problem (we built it to track seconds and now
| they want us to track calendar dates!).
| Rafsark wrote:
| Navigating this challenge involves the ever-changing boundaries
| of billing. Initially, you build a monthly billing system, only
| to find that your team requires a yearly one, which seems
| relatively straightforward. However, complications arise when
| you introduce usage-based billing, incorporating weighted
| values akin to storage, layered on top of a quarterly plan.
| This complexity is further exacerbated when custom billing
| structures are needed for negotiated contracts, adding another
| layer of intricacy to the mix.
| m3nu wrote:
| I built my billing system on top of Django and it's used in 2
| online services for a few thousand users and hundreds of invoices
| a month.
|
| You have to be mindful of some edge cases, but if the scope is
| kept narrow, it's doable. The added benefit is the flexibility to
| fit it to your use case. And with Python, you get many high-level
| libraries, e.g. for decimal calculations and PDF generation.
| yugarinn wrote:
| My experience is that the scope is never kept narrow, which is
| exactly the same that the author is describing.
| acutesoftware wrote:
| Well, you can - you can have a policy that forces payments
| from a select group. Some users will complain when "No, you
| cant pay with 2 chickens every 3rd full moon", but that is
| too bad.
| dagw wrote:
| Of course you then have to be willing to turn down
| money/customers. As someone who has worked for quite large
| traditional companies, we've ended up not buying from
| certain companies because they couldn't invoice us like we
| wanted to be invoiced.
| 6510 wrote:
| And miss out on all the fun with buyer-created invoices?
| icedchai wrote:
| I built a billing system for a web hosting company in PHP,
| almost 20 years ago. This was for a few hundred users. There
| were a couple of edge cases, but nothing incredibly
| challenging.
| Rafsark wrote:
| Was it a basic subscription model or have you faced
| challenges with taxes, usage-based components, entitlements,
| grand fathering?
| icedchai wrote:
| Basic subscription and it was 20 years ago. There was some
| grandfathering but that was pretty simple to solve with a
| status column on a plan table, etc.
| Rafsark wrote:
| The use case is tiny at the beginning but the complexity
| increases over time to be honest. The scope is never as narrow
| as you think at the beginning of the project. Libraries are
| great, they can help you implement faster, but if the company
| grows, you will need an entire team to build and maintain it
| robaato wrote:
| Worked in mobile Telecoms billing back in the relatively early
| days. Remember discovering that Orange at the time made a good
| feature of competing (and acquired lots of new customers) by
| offering "competitor match" billing options. They made this as a
| business choice, but it cost them a team of over 100 contractors
| developing that system. Other telcos had packages but were
| constrained by what the packages offered as to what their
| marketing could offer. You pays your money and makes your choice!
| p.s. things learned the hard way in those early days - customers
| realised that a call held open for over 24 hours resulted in a
| reset of billing due to counter wrap around!
| jerf wrote:
| Another problem with billing systems I've seen repeatedly is that
| the billing demands flows one way; marketing or product
| management basically scribbles on a napkin how they'd like to
| bill the customer, and that napkin gets passed to the billing
| team as a specification carved into stone. Basically, in practice
| it gets run as Waterfall, actual factual Waterfall, except the
| design phase is skipped. It goes straight from features to
| implementation.
|
| Now, on a business level I acknowledge that frequently the
| monetary costs of the billing system are negligible compared to
| the business advantages of the "correct" billing system. My
| complaint is more that the scribbled note often ignores what the
| rest of the company is doing for billing, e.g, "yes usage based
| billing is great but do you _really_ need to bill on furlongs per
| fortnight when the rest of the company is billing based on seats
| or some basic, easy-to-understand resource allocation? ", ignores
| that maybe there's an easy thing that's _close_ to what you said
| but since it reuses all the existing stuff you can have it in two
| weeks instead of four months, etc. Do you really need to issue a
| discount coupon that takes off a percentage proportional to the
| length of the customer 's first name, unless the customer is in a
| jurisdiction where that's illegal in which case we roll a die,
| unless the customer is in a jurisdiction where _that 's_ illegal
| in which case we give them $10 off their first $100 and take off
| every prime-numbered dollar after that? Because I've got code
| that just takes 10% off their first month right here. Do you
| really need to bill on the first of the month unconditionally
| when the rest of the company bills based on subscription start
| date, or vice versa? Even if you're not too worried about the
| costs of paying 6 months of developers, I bet you _are_ worried
| about those 6 months of opportunity cost.
|
| I can't even count the months of delays caused by treating napkin
| scribbles as carved stone I've personally witnessed, and I'm only
| tangentially involved in billing over all.
|
| And somehow managers who are deeply familiar with costs and
| benefits and tradeoffs and make sensible decisions all the time
| just become the most obstinate customers when it comes to
| billing. No matter how small the tweak proposed and how many
| _months_ forward it may bring your release date to do something
| just _slightly_ different (and consistent with the rest of the
| company 's existing policies), they will go to the wall for their
| billing deviation, even when it was frankly clearly nothing more
| than a whim or a transient thought at some point by somebody
| somewhere of no strategic or marketing consequence.
|
| It isn't just engineers who underestimate the complexity of
| billing until it's too late; it's everyone, really. "Just" print
| an invoice turns out not to be so simple.
| Rafsark wrote:
| Marketing teams can sometimes miss the mark on this one.
| There's a misconception that improving billing is as simple as
| launching a new marketing campaign, but that couldn't be
| further from the truth. Billing is a multifaceted aspect of
| engineering and business operations, and those who solely focus
| on it often prioritize following industry trends over gaining a
| deep understanding of the economic intricacies within the
| company.
|
| It's crucial to bridge this gap in comprehension for more
| effective decision-making and strategy alignment between the
| marketing and billing departments. By doing so, we can ensure
| that both teams are on the same page and working together
| seamlessly to drive the company's success.
| somsak2 wrote:
| Should have a (2022) label
| https://web.archive.org/web/20220715000000*/https://www.getl...
| pantulis wrote:
| Billing is one of those problems that seem deceivingly easy. Just
| talk to someone who has worked in telco billing and learn about
| the 4 horsemen of billpocalypse: metering, mediation, accounting,
| billing.
| trollied wrote:
| Rating can be great fun. US taxes (geocoding, urgh) are an
| absolute nightmare. Reversed charged calls, revenue sharing.
| Roaming is a pain (ASN.1 files). Even document numbering is a
| nightmare in some countries, as are legalities surrounding
| incorrect bills (corrective invoices) and crediting. Did 15
| years in billing (now do finance software), and whenever I
| thought I knew it all there would be another curve ball thrown.
| Rafsark wrote:
| You can add taxes and negotiated contracts to the list tbh
| garganzol wrote:
| I wrote a billing system myself, and my first impression was that
| it couldn't get past 1000 lines of code. However, it had become a
| ~5000 lines monster and keeps on growing. Never underestimate
| details and edge cases.
| mlhpdx wrote:
| The point the article makes to me (as a leader) is "keep the
| billing model simple, honest and stick with it". The cost of
| change is high, and the value isn't to the people getting the
| work done. My peers would likely see this as "naive" and extoll
| the need to "sell to the buyer". I see it as being value focused,
| and keeping the "business needs" rational, and in line.
| Rafsark wrote:
| When you don't understand the complexity of billing (revenue
| ops, top management or marketing), you often end up adding more
| and more complexity, and engineering teams have no choice. They
| have to build new billing features
___________________________________________________________________
(page generated 2023-09-18 23:01 UTC)