[HN Gopher] Twenty five thousand dollars of funny money
___________________________________________________________________
Twenty five thousand dollars of funny money
Author : zdw
Score : 220 points
Date : 2022-12-03 15:24 UTC (7 hours ago)
(HTM) web link (rachelbythebay.com)
(TXT) w3m dump (rachelbythebay.com)
| jacobn wrote:
| Had exactly that bug in production, was using ruby on rails &
| active merchant, and some version change in ActiveMerchant
| switched from cents to dollars for one of the integrations.
|
| Our test harness didn't catch it (weird combination of reasons,
| too long ago for me to remember the details) & it rolled out.
|
| Shortly thereafter I get an anxious customer call that we'd
| charged their debit card $2500.00 instead of $25.00 and they'd
| gotten an overdraft notice. At first I was incredulous ("how is
| that even possible!?"), then I remembered that we'd just version
| bumped ActiveMerchant.
|
| My endocrine response as I realized what must have happened was
| amazing to experience - the sinking feeling in my gut, hairs
| standing up, sweaty palms, dread, pupils dilating, and my
| internal video camera pulling back poltergeist-style in a brief
| out-of-body experience.
|
| Fun times. Live and learn.
| HappySquirrel2 wrote:
| I made an account just to tell you that I was able to almost
| feel & experience what you 're describing just by reading your
| comment. There should be a market for this hah
| mbell wrote:
| > Our test harness didn't catch it (weird combination of
| reasons, too long ago for me to remember the details) & it
| rolled out.
|
| This is why I'm pretty dogmatic about variable comparison in
| tests.
|
| This is dangerous and stuff like this has caused a lot bugs to
| slide though in my experience (and maybe ops):
| expect(account1.balance).to eq account2.balance
|
| This is safe and specifc:
| expect(account1.balance).to eq 2500
| expect(account2.balance).to eq 2500
|
| Unfortunately I've run into a lot of folks that take major
| issue with the later because of 'magic numbers' or some similar
| argument. In tests I want the values being checked to be be as
| specific as possible.
| jiveturkey wrote:
| thanks. i do the same and consistently have to push back on
| reviewers (why don't they learn?) that the hardcoded number
| is there _for a reason_
| hinkley wrote:
| What the complainers often like to do is hoist variable
| declarations out of the local scope. So to them having a
| test suite that uses 1500 in four places is wrong, and up
| to that point they are perfectly right.
|
| The trick with the constants is that if they are declared
| and used in the same test scope, then the data is a black
| box. Nobody else 'sees' it, nobody interacts with it. The
| only time that's not true is when there's false sharing
| between unit tests and _those tests are fundamentally
| broken_. In that case the magic number is not the problem,
| it 's the forcing function that makes you fix your broken
| shit.
| hinkley wrote:
| One of the tenets of unit testing, I don't know if I derived
| this for myself or got it from someone, is that you get a
| lookup on the right side of the expectation or the left, but
| not both.
|
| There are too many yahoos out there writing impure functions
| or breaking pure ones that will mangle your fixture data. And
| the Sahara-DRY chuckleheads who see typing in 2500 twice (but
| somehow are okay with typing result.foo.bar.baz.omg.wtf.bbq
| twice) as some crime against humanity exacerbate things.
| Properly DAMP tests are stupid-simple to fix when the
| requirements change.
| diarrhea wrote:
| I get the intention, but a simple x = 2500 would satisfy
| both worlds.
| majormajor wrote:
| Various things like "manual to change them" that make "magic
| numbers" bad in regular code make them good for testing (or
| at least less bad, a constant for that is still what I'd
| usually use, but a pretty specific constant, sometimes at the
| unit test level - shared ones get dicey).
|
| Agreed on the ease of having problems of using variables on
| both sides.
| hinkley wrote:
| One of the biggest ways that test code is not production
| code is that test code is only read by humans when the
| tests are failing. Whereas any time I'm working on a
| regular feature I am likely to be looking at log(n) lines
| of our codebase due to the logic that exists around the
| code I'm trying to write, and changing loglogn lines of
| existing code to make it work - if the architecture is
| good.
|
| Code that is write once read k < 10 times has very
| different lifecycle expectations than code that is
| constantly being work hardened.
| NortySpock wrote:
| I fend off the "magic numbers" people with a variable with a
| name that describes what the magic number is for.
| var expectedAccountBalance = 2500
| expect(account1.balance).to eq expectedAccountBalance
| expect(account1.balance).to eq account2.balance
| tharkun__ wrote:
| That's a good enough (initial) workaround if you're in an
| environment where you're in the minority (or alone) w/ your
| opinion and you still want to do the right thing
| personally.
|
| I would advise you to try and convince your peers though
| and teach them the better way because I suspect that the
| people that you're fending off would not do what you did
| but rather just go w/ the one
| expect(account1.balance).to eq account2.balance
|
| Now while parts of the code base do the right thing (in a
| slightly long winded way), the rest of the code base
| written by these other people is still using bad tests.
| tuatoru wrote:
| Reasonableness checks are nearly absent these days. Warning the
| user "that's a lot of money/a date far in the future/a large
| quantity.[1] Are you sure?" detracts, I suppose from the UX.
| Except it really doesn't.
|
| 1. "Reasonable" varies according to the particular situation
| (customer's order history, credit rating, the normal range of
| quantites for the product, etc.)
| brundolf wrote:
| I think it's more that it's a game of whack-a-mole. There are
| an infinite number of possible scenarios you could warn
| about, and each one carries a small cost to implement, a
| small cost to maintain, and a risk of false-positives. Which
| ones are worth implementing can be hard to know ahead of time
| (implementing ones that have actually caused problems would
| be one strategy for narrowing it down, but the point remains
| that it's not as simple as "just check for all the
| unreasonable states")
| hinkley wrote:
| Frankly, the ActiveMerchant team broke your trust. There's two
| things you can do with people you don't trust. Either cut them
| out of your life, or start verifying everything they tell you.
| I've dumped libraries with this sort of chaotic 'refactoring'
| (or not picked them because I could see they had made such poor
| initial choices that it was inevitable). I've also written
| unit, integration, or smoke tests for third party code with a
| high Precautionary Principle quotient, sometimes with its own
| separate build pipeline that takes green versions of our code
| to run them against latest of theirs. And for any code we got
| from a customer or business partner? This is practically a
| requirement for my sanity. They think that because money is
| changing hands they can do whatever they want and we just have
| to eat it. We don't have to eat anything.
|
| That doesn't catch the problem the moment it happens, but most
| times that's sufficient to catch it before anything hits
| production.
| dasil003 wrote:
| I also had this problem--with the same stack--but the issue was
| quite subtle. It was due to an upgrade of the Money gem that
| changed the way "cents" were handled.
|
| In our case we had test coverage, including integration tests
| with VCR recordings to the payment gateway. But the problem was
| that the bug only affect Japanese Yen, and we did not cover
| every single currency.
| Eleison23 wrote:
| >internal video camera pulling back poltergeist-style
|
| That cinematic technique is called a "dolly zoom" and it's
| totally dramatic, if used correctly!
| cratermoon wrote:
| Alfred Hitchcock famously first used it in his movie Vertigo,
| after one of his camera operators, Irmin Roberts, came up
| with the effect.
| richard_mcp wrote:
| I'm confused how this was able to give out money before the new
| code was submitted to production. The author claims that both she
| and her coworker tested it before the code was submitted and they
| ended up with the extra $25k. Was this code only executed on the
| front end? Were there no checks in the backend to prevent
| employees from just pulling out whatever money they wanted?
| adamsb6 wrote:
| The way that things work at this particular company is that you
| typically test changes in this codebase on your dev machine,
| but usually the dev machine talks to a prod database.
|
| The prod database is too large to practically have a second
| copy sitting around for testing. Also, if you tested on some
| pristine small test database you're going to end up missing
| bugs that would only manifest with actual prod data.
| namrog84 wrote:
| I get it but that just seems really dangerous. I hope they
| have a lot of guard rails and roll back support or something.
| justinator wrote:
| Local was front end stuff, back end was still talking to
| production.
| rockinghigh wrote:
| As I understand, the frontend and backend code run in a
| development environment but the funds available were stored in
| a production database.
| dmix wrote:
| The way she said it 'crashed' and you could just reload the
| page, and that it was frontend work also points to browser js
| code?
|
| So yeah I'm curious how that worked too.
| hk__2 wrote:
| As I understand it the code gave you $25k credit, not $25k
| actual money.
| tartoran wrote:
| Yeah but you missed the funny part in front of money.
| kleinsch wrote:
| For some systems in that environment, the back-end for dev was
| actual production. Keep in mind that wasn't handing out actual
| money, it was creating ads credit.
| cpursley wrote:
| I feel like we could avoid some of this if we just allowed
| databases to do what they're good at. Looking at you Rails...
| hermanradtke wrote:
| Scalars like this are so dangerous. A New Type would help here,
| but this is much easier to say in hindsight.
| Marazan wrote:
| One of the benefits of learning to program with Ada as the
| teaching language at University was the instinct to sub_type
| all the things. IF you're using a raw int you are probably
| doing it wrong.
| latchkey wrote:
| Where are the unit tests?
| friedman23 wrote:
| People writing unit tests for internal tools? A lot of
| companies don't even having tests for production code.
| latchkey wrote:
| Especially yes. A unit test would have caught this instantly.
|
| It is code that deals with $'s... something you'd really want
| to test, since it'll cost the company money.
|
| Instead, you've got multiple engineers writing code multiple
| times (my euphemism for fixing buggy code), which also costs
| the company money.
| friedman23 wrote:
| I'm just saying people don't do it in practice.
| lovecg wrote:
| Oh I'm sure there are unit tests for new func and old func that
| test them separately just fine. The if statement is probably
| thought of as a temporary switchover kind of thing.
| fbdab103 wrote:
| I could believe there were unit tests, but not at a high enough
| level. new_func input:$25,000, output:$25,000. Test passes.
| cratermoon wrote:
| There are several things happening here worth breaking down.
|
| The first is what I've seen called a "gettier"[1]. The idea of
| "justified true belief" which ends up being true, but not for the
| reason you thought it was true. That's the case of the first of
| the bug fixes: She'd exposed the problem with the first change,
| but it wasn't really the problem.
|
| The second item of note is that one paper found 92% of
| catastrophic system failures come from buggy error-handling
| code.[2] Arguably this doesn't count as catastrophic, but $25K
| adds up.
|
| The third and final item is the failure relating to the use of a
| primitive, number' instead of a domain-relevant type, like Money,
| Dollars, or Pennies. This concept came up as Value Object three
| days ago[3], which Ward Cunningham's CHECKS Pattern Language of
| Information Integrity, published in 1994, called Whole Value[4].
| I've seen (and, as a young code, written) programs that are full
| of strings for everything, because that's how the they are
| represented to users and passed over (some kinds of) network
| services. This "stringly typed" code infests a project I'm
| currently engaged with, simply because the back end depends on a
| bunch of REST/JSON apis and never bothers to deserialize them,
| but passes them throughout large parts of the code completely
| unrelated to the api calls.
|
| 1 https://jsomers.net/blog/gettiers
|
| 2 https://www.eecg.utoronto.ca/~yuan/papers/failure_analysis_o...
|
| 3 https://news.ycombinator.com/item?id=33792874
|
| 4 https://c2.com/ppr/checks.html#1
| mgraczyk wrote:
| I encountered a similar bug at the same company with far worse
| results.
|
| Won't say any specifics about the product impact, but our backend
| passed around two different kinds of user IDs. Each user had two
| different IDs, and the ID spaces overlapped. User Alice could
| have an ID in space 1 that is the same as Bob's ID in space 2.
|
| At some point, at least one function expected a "space 1" ID but
| was being passed a "space 2" ID. This meant that content meant
| for Alice was shown to Bob and vice versa. None of the data was
| private in this case, so there was no legal problem, but it was
| pretty embarrassing. I suggested using strong types for the ID
| spaces instead of `int`, but left the company before implementing
| any of that.
| Quekid5 wrote:
| Oof, I'm happy it was just non-private data!
|
| Regardless, this makes an excellent case for strongly typed
| wrappers as you mention at the end.
|
| Our general approach is to use UUIDv4 for identifiers (so the
| chance of mistaking-one-for-another instantly leads to "not
| found"), but sometimes you don't have a choice. In those cases
| it's super-important to have strongly typed wrappers.
| Twirrim wrote:
| Similar kind of thing, one company I worked for we had two
| smart-card/badges, I can't remember why. It may have been
| nothing more unusual than one was the building card, one was
| the company one.
|
| They had an overlapping ID space. Swipe the wrong one on the
| printer and you got someone else's print job. In my case I
| accidentally caused a Senior Director's print job to be
| printed, and luckily it wasn't anything sensitive.
|
| I had no end of trouble trying to get IT to accept that this
| was actually a problem.
| Akronymus wrote:
| One of the things I really like about f# is thar you can tag even
| raw numbers with types. And even multiply/divide on those
| types.https://learn.microsoft.com/en-us/dotnet/fsharp/language-
| ref...
| aembleton wrote:
| Kotlin has type aliases. This means you can write something
| like: typealias Cent = Int
|
| and then use a type Cent just like you can an Int, with
| multiplication and everything else. More info:
| https://kotlinlang.org/docs/type-aliases.html
| zdragnar wrote:
| This is probably the one feature from F# that I truly miss in
| other languages.
| fbdab103 wrote:
| Nim as well, they even have an example[0] with currencies.
|
| [0] https://nim-lang.org/docs/manual.html#distinct-type-
| modeling...
| cb321 wrote:
| One can, of course, go much further than simply distinct
| number types in Nim: https://github.com/ringabout/awesome-
| nim#science
|
| (Unchained seems maybe the most featureful of those units
| packages.)
| raydiatian wrote:
| Critical things come with critical stickers:
|
| Transformer boxes, Nuclear waste, Highly acidic compounds, High
| energy lasers, choking hazards.
|
| The oldest debate: should there be a money primitive in the type
| system?
|
| I mean, availability breeds use, use breeds awareness, awareness
| breeds or enforces proper use. A currency/money type would be a
| pretty clear label to ward off a whole suite of stupid bugs like
| the one described in this post.
| mxz3000 wrote:
| or just units of measure a more general solution than what
| you're suggesting, implemented by languages like F#
|
| https://learn.microsoft.com/en-us/dotnet/fsharp/language-ref...
| raydiatian wrote:
| Wow that is a cool feature, thanks for sharing.
|
| I think there are some other things like fixed-point rather
| than floating-point decimals for storing/manipulating
| currency. IIRC, the number 0.1 cannot be precisely
| represented with floating point system, but that just may be
| an old wives tale at this point.
| SAI_Peregrinus wrote:
| One of the first programming languages I learned was TI BASIC on
| the TI-89. It had an _excellent_ units (refinement types) system:
| you could define an expression as a "unit" by prefixing a
| variable with an underscore, and it came with a bunch of built-in
| units. A unit expression could thereby automatically convert
| between types as needed, outputing the base unit.
|
| E.g. `5_dollars + 8_cents` would output `508_cents`. There was
| also a conversion operator you could use to change the output
| unit, so `5_dollars + 8_cents-_dollars` would output
| `5.08_dollars`.
|
| It even worked for more complicated derived units, so e.g. `8_m *
| 5_kg / (3_s * 4_s)` returns `3.333_N`.
| ant6n wrote:
| Looks neat. Of course you can do this in c++.
| wtallis wrote:
| I think TI probably copied that from the HP 48, which used more
| or less the same syntax. The only difference is that the
| calculation you show would not have been automatically reduced
| to _N, but you could use the CONVERT or UFACT commands to get
| Newtons from kg*m/s^2.
|
| In practice, HP's UI made it a lot easier to manipulate
| quantities tagged with units: hitting the softkey for a unit
| would multiply the current value by that unit, and using the
| two shift keys you could either divide by that unit or convert
| to that unit. If you made a custom menu to put the handful of
| units relevant to your current problem domain all close at
| hand, you would need very few extra keystrokes compared with
| calculating without using the units system. That ease of use is
| vital; opt-in type safety should be as easy to use as possible,
| so that users aren't tempted to fall back to the simpler, less
| safe method.
| neilv wrote:
| > _This is yet another reason why I say bare numbers can be
| poison in a sufficiently complicated system._
|
| In Racket, I tended to use keyword arguments, and include the
| units in the keyword argument. For example, one library used
| `:velocity-mm/s`. (The `/` character is an identifier
| constituent, not some special operator syntax.)
|
| In Rust, I'm going to try out using language features for static
| checking of some units (with 0 runtime cost).
| stevedonovan wrote:
| Yes, the newtype pattern in Rust works very well, say to wrap a
| string. The somewhat annoying thing is that it is born without
| any properties so you have to teach it to be comparable, etc
| from scratch.
|
| The newtype pattern in Go is easier to use, but isn't strict
| enough - your wrapped string can still appear directly in a
| string concatenation without a cast. Any wrapped integer can
| still be used to index a slice! Considering how careful the
| language is with mixed arithmetic (can't add uint8 to uint16
| without casting) this feels like an oversight.
| chasing wrote:
| Just to say it, for measurements that take a unit I am hardcore
| that you should include the unit in the variable time:
|
| `timeSeconds`
|
| `distanceMeters`
|
| `amountDollars`
|
| Have unit tests and everything, but also write your code so that
| when someone reads it they know as precisely as possible what's
| going on without other documentation.
| sigha887 wrote:
| Would you also add other type-related aspects into variable
| names? `firstNameString`, `ratioFloat`, `colorEnum`? These
| things should be types, not easy-to-ignore type-encodings into
| variable names..
| rtlfe wrote:
| Using types that encode the units, as suggested in OP, is
| strictly better.
| debaserab2 wrote:
| Agreed, but I can't help but think that this is just a poor
| man's typings for languages that don't directly provide them.
| diarrhea wrote:
| At that point, why not have dedicated types and get rid of
| primitive ones? It shifts all that work to the compiler, whose
| job it is to excel at this kind of stuff (adding cents to penny
| types does the right thing automatically etc).
| WrtCdEvrydy wrote:
| I'm gonna be honest, I had never understood the "strict type"
| argument until right now. Seriously, since everything I work in
| is denoted as "cents" from backend to frontend, I personally had
| never understood the need so I'm part of today's lucky 10,000th.
| lxe wrote:
| Strict types have little to do with this. Unless your type
| system validates bounds , this sort of things happens
| regardless of types.
| mrkeen wrote:
| At the very least, strict types:
|
| 1) make you do the check
|
| 2) only require you to do the check once
| onion2k wrote:
| That won't always work unfortunately, particularly if you have
| to work globally. For example the fiscalization requirements
| for cash reporting in Germany specify that all of your
| transactiona have to be done to 6 decimal places (for things
| like discounts.)
| JohnBooty wrote:
| Seriously, since everything I work in is denoted as
| "cents" from backend to frontend, I personally had
| never understood the need
|
| This matches my experience.
|
| Obviously actual strict typing has its benefits, but as far as
| developer ergonomics are concerned, IME you can get about 95%
| of the benefits just by following a convention of including
| unit names in identifier names.
|
| It's easy to see why this Ruby code might fail:
| def launch_rocket(distance) # what units are we
| expecting? end launch_rocket(42)
|
| But this is virtually impossible to screw up, and reduces
| developer cognitive load: def
| launch_rocket(distance_km:) # blahblahblah
| end # not happening unless developer consumes a
| large number # of drugs distance_miles = 42
| launch_rocket(distance_km: distance_miles)
| fr0sty wrote:
| > [follow] a convention of including unit names in identifier
| names.
|
| appending units to identifiers helps, but it relies on a
| developer's eyeballs to spot any errors. It would be
| infinitely preferable if the type system would simply enforce
| this for you and developers not have to expend cycles
| reasoning about this stuff themselves.
| bornfreddy wrote:
| If you stick to just a few units throughout the system
| (ideally, use a consistent base unit everywhere and convert
| any external measurements to it), you can avoid the rest of
| the problems.
|
| Static typing is great, of course, I just don't agree that
| this can't be solved to almost the same level with
| languages with dynamic typing.
| svnpenn wrote:
| here it is with Go: package main
| type km int func (distance km) launch_rocket()
| {} func main() { distance_miles
| := 42 // type int has no field or method
| launch_rocket // distance_miles.launch_rocket()
| // this works, but is obviously wrong:
| km(distance_miles).launch_rocket() }
| Swizec wrote:
| Easy! # not happening unless developer
| consumes a large number # of drugs
| distance_miles = 42 launch_rocket(distance_km:
| distance_miles*1.6)
|
| Aaaand we're off by 0.3924km. Enough to fit 4 football fields
| with a few meters left over. Oopsies
|
| Units are hard. Never under-estimate the ability of
| programmers to think they're converting but get it slightly
| wrong.
| [deleted]
| sgtnoodle wrote:
| It's within a significant digit of the specified distance,
| though. A few football fields of error seems pretty good to
| me for 42 miles. What's the required accuracy and precision
| for this rocket launching system?
| Swizec wrote:
| Well if it's a boom rocket for war, the expected
| precision these days is the size of a car if not smaller.
|
| I think even dumb gravity bombs in ww2 were more precise
| than "a few football fields"
| generationP wrote:
| Nope, this will lead to the same problem that the OP has
| discovered, viz., that a dependency update creates silent
| errors. Unless the argument is a keyword, but everyone is too
| lazy for that.
| soulofmischief wrote:
| Yeah, to me the moral of the story was write more tests.
| tom_ wrote:
| The case you claim requires drugs will happen eventually even
| if everybody is sober. I've seen it many times, and I doubt
| I'm the only one. Somebody is tired, or they get the computer
| to perform some bulk change and get it wrong, or they are
| doing some rote transformation and they miss a case.
|
| If you want to do a bit better, a simple way of handling it
| is to provide a separate type that handles the abstract
| quantity (distance, money, etc. - you'd probably want to be
| cleverer about money if you deal with multiple currencies
| though), and ensure that values of that type can be converted
| to and from numbers only when the units are explicitly
| specified.
|
| So then you end up with functions that consume a distance
| looking something like this: void
| launch_rocket(Distance distance) { call_ancient_f
| ortran_routine(get_miles_from_distance(distance)); }
|
| And functions that create distances looking something like
| this:
| launch_rocket(create_distance_from_km(100));
|
| What you now can't do is create a value that's of one unit,
| and pass it to something that expects another unit - the
| issue doesn't really arise, as Distance values themselves
| don't have specific units. They are a black box that somehow
| encodes a distance, and you specify the units used explicitly
| when initializing and you specify the units desired
| explicitly if retriving an actual number.
|
| (Turning a distance into a number would ideally be a rare
| case, though sometimes you'd need it. You'd provide maths
| functions for all operations required, so you'd hopefully
| rarely need the number for calculation purposes. For
| displaying in any UI, there'd be a function to convert it to
| a string that respects the user's locale and distance unit
| preferences. And so on.)
| denton-scratch wrote:
| You don't need "strict typing" to handle money; in the old
| (COBOL) days, we used BCD to represent monetary amounts with
| arbitrary precision. When they took away BCD, we were stuck, if
| we wanted to build a system that could represent a large sum
| correctly in both dollars and yen.
|
| COBOL was pretty good for dealing with money.
| monocasa wrote:
| BCD doesn't really make it easier. Fixed point can, but
| that's ultimately a typing thing that works just fine in
| binary as well.
| denton-scratch wrote:
| You're right; but COBOL BCD types allowed arbitrary
| precision, and it was super-easy to debug data; the hex
| represention was the same as the decimal representation.
| scatters wrote:
| You still don't know whether something is dollars or cents.
| Some things are conventionally priced in dollars, others in
| cents, and your customers are going to be very unhappy if
| they have to enter or read the price in the "wrong" unit.
| lifeisstillgood wrote:
| Could you expand on BCD? What made it good for multi-currency
| work?
|
| (a quick google did not help, managed to lead to examples of
| COBOL manipulating the first five letters of the alphabet
| ...)
| dahfizz wrote:
| Binary Coded Decimal allows for perfect representation of
| numbers by not restricting you to 4 or 8 bytes. It trades
| speed and memory efficiency for precision and simplicity.
|
| https://en.m.wikipedia.org/wiki/Binary-coded_decimal
| dmurray wrote:
| It allows, like all encodings, a perfect representation
| of some subset of real numbers but not the rest.
|
| In particular it perfectly represents numbers which are
| commonly used in modern commerce, like 19.99 or 1.648
| (the current price per litre of fuel near me). It's not
| great at other numbers like pi or 1/240.
| denton-scratch wrote:
| I appreciated BCD because the in-memory data represented in
| hex (as by an 80's era debugger) was exactly the decimal
| value. As I recall, debuggers of that time only understood
| two datatypes: ASCII characters and hex octets.
|
| On consideration, I think my COBOL compiler's ability to
| define arbitrary-precision fixed-length numerical variables
| wasn't down to the use of BCD; you can do that with other
| binary encodings. But I worked for Burroughs at the time;
| their processors had hardware support for BCD arithmetic,
| so it was fast. The debugging convenience came with no
| great cost.
| fsckboy wrote:
| BCD is simply "work in base ten on a base two digital
| computer". What made it good is that it enforces a
| discipline with the same pattern of rounding errors as base
| ten arithmetic on pencil and paper. This was particularly
| attractive when computers were new and replacing "doing it
| by hand". Bankers were nervous about the new systems
| screwing everything up, and they wanted the new system to
| demonstrate it would produce the exact same results as the
| old system.
|
| To give an illustrative example, what's 2/3 of a dollar? 66
| cents or 67 cents, one or the other, choose the same one
| you would choose with pencil and paper. Now add 33 cents,
| did you "overflow" the cents and need to increment the
| dollars?
|
| Yeah, you can achieve the same thing with binary by
| constantly checking ranges of numbers, but the difference
| is, BCD when you screw up your code produces errors similar
| to adding numbers by hand, errors recognizable by your non
| computer literate accountant; binary screwups will produce
| a different unrecognizable pattern of errors.
|
| the way it worked was pretty straightforward, just like 4
| bits is hex 0-F and 8 bits is 0x00 to 0xFF, a BCD byte is
| 00-99 and you just never have the patterns for A-F. This
| was enforced in hardware, in the CPU/ALU
|
| in terms of multi-currency, same thing, you'll see the same
| familiar rounding problems as traditional pencil and paper
| currency changing systems.
|
| Also the same set of issues extends to fixed point
| implementations of "floating point"/"decimal
| fraction"/"rational number" systems more common in
| engineering. 1/3 is a .33333.... repeating fraction; 1/5 is
| .2, no repeat, because 2x5=10 base 10. In binary, 1/5 is a
| repeating decimal, not good for comparing results,
| rounding, etc. And you can easily see that the same issue
| does apply to currency too (it was my example above with 67
| cents), it's just a bit less visible because it's less
| common to use extended fractional amounts.
| lalopalota wrote:
| BCD = Binary Coded Decimal
| xg15 wrote:
| This bug could easily happen in a typed language though.
| function deduct(int cents) { ... } int dollars = ...
| deduct(dollars);
|
| You need something like (Apps) Hungarian notation [1] as a
| minimum - or even better, subtypes of primitives like in Go to
| represent units in a typesafe way.
|
| [1] https://en.m.wikipedia.org/wiki/Hungarian_notation
| snotrockets wrote:
| I think you're confusing expressiveness of types with
| static/dynamic typing (the latter are also typed!)
|
| An expressive type system would allow you to define both a
| cent and dollar types, s.t. assignments of those types to
| each other without conversion would fail.
|
| In a way, it is a way to have the computer validate apps
| Hungarian rather than trusting the programmer (well, it's
| more, but for this argument).
|
| Go's type system is anachronistic, compared to what modern
| language provides (but then, all of go is anachronistic on
| purpose. The usefulness of this purpose not to be discussed
| here).
| shepherdjerred wrote:
| > The usefulness of this purpose not to be discussed here
|
| This is a fantastic bit to tack on for divisive topics, I'm
| stealing it.
| xg15 wrote:
| Ah, my bad there. That was what I meant with subtypes, but
| you're right, things have moved further there already.
|
| Sorry for the misinfo!
| stavros wrote:
| Yeah, defining "dollars" and "cents" (and over units)
| types are actually a really good solution for this, it
| means that you can never pass a value of one unit when
| the function expects another.
| acdha wrote:
| That's a generic type, and that's why they're not enough.
| Here's a simple Python example: class
| Cents(int): pass def send_money(amount: Cents):
| print(f"Sending ${amount / 100}")
| send_money(42)
|
| That'll immediately fail validation without you needing to
| make sure you have tests which would catch every possible
| problem like that.
|
| expected "Cents" [arg-type] Found 1 error in 1 file (checked
| 1 source file)
|
| In a language which has strict typing, you wouldn't even be
| able to compile it.
| metafunctor wrote:
| You should define separate types for "cents" and "dollars".
| And, probably operations to convert between the types.
| dragonwriter wrote:
| You can have a single type for "money" (or maybe just
| "us_money") with separate cents/dollars/mills factories and
| accessors, and no access to a numeric value except through
| the accessors.
|
| You can do similar things with other dimensions like
| "length".
|
| Or you can go whole hog, and have a single "type" for unit-
| aware values from which you can only successfully extract a
| unitless number by specifying a unit which is dimensionally
| compatible.
|
| But most projects won't do any of these because they will
| start out thinking they don't need it, and by the time they
| realize the value they'll think the cost of converting
| existing code is too high.
| diarrhea wrote:
| In physics, dimensionality makes sense. How would one
| handle money? Can it be represented the same way? A new
| dimension, next to length, time etc? It would allow to
| express money per time, eg dollars per second, for
| example.
| dragonwriter wrote:
| > How would one handle money? Can it be represented the
| same way?
|
| Any _particular_ currency can be modelled simply as a
| single dimension; "money" more generally is more complex.
| You can either use a single currency of account, track
| exchange rates for other currencies with it over time,
| and convert other currencies into it based on the time
| applicable to the event, or you can track each currency
| as a separate domain and convert based on the applicable
| exchange rate for a particular purpose _ad hoc_ based on
| the specific situation. (There's probably other
| approaches that work, but those seem to be, in outline,
| the most obvious.)
| sneak wrote:
| I'm also in the camp that believes that functions or methods
| that take more than 2-3 (positional) arguments should really
| take a structure with named keys to avoid this. Humans are bad
| at lists.
|
| Seeing functions with 5-6 positional arguments makes my skin
| crawl even if they have strong types.
| [deleted]
| disgruntledphd2 wrote:
| Try to avoid looking at any scientific/data science code
| then. Almost every function has 6+ arguments.
| Waterluvian wrote:
| I think I like the duck typing in TypeScript more than I dislike
| it.
|
| But I still wish I could say "this function accepts a type called
| RobotName. It's a string, but so is RobotUuid, and we don't want
| that. So only accept, strictly, objects typed as RobotName."
| malf wrote:
| type robotname = string & {_robotname_marker:true}
|
| or some variation. The marker does not actually need to exist.
| mejutoco wrote:
| In haskell this is done with newtype. Another example would be
| using km and miles, and making sure they are not mixed.
|
| Here there is an interesting article about doing this in
| typescript (not affiliated)
|
| https://kubyshkin.name/posts/newtype-in-typescript/
| akama wrote:
| This is actually one of the nice features about OCaml that
| comes in handy when you have a function that takes two
| arguments of the same type but don't necessarily need to make a
| whole new type for each of them. They are called Labelled
| Arguments [0] and when you call the function, the label has to
| be the same. I've found that using it can clean up code because
| it ensures that variables share the name across the codebase as
| well as making sure arguments don't get mixed up.
|
| [0]: https://ocaml.org/docs/labels
| jzig wrote:
| type RobotName = string;
|
| function foo(r: RobotName) {}
|
| ?
| chpatrick wrote:
| In TypeScript any type that's structurally the same is
| considered equal, type just gives it a different name.
| Waterluvian wrote:
| Yep. But that function will accept
|
| type RobotUuid = string;
|
| const bar: RobotUuid = "abc...";
|
| foo(bar);
|
| This is what duck typing is and specifically my curiosity
| about being able to de-duck on demand.
| roflyear wrote:
| Yeah just means you have to rely on linting and your ide to
| catch those errors. And hopefully the rest of your team
| does the same thing.
|
| Tho I suppose if it is really important you can put an
| assert there but I'm not familiar with that wrt typescript,
| maybe the transpiler would kill that?
|
| I've done the occasional type checking in that way in
| similar languages, it is kind of self documenting too. 90%
| of the time duck typing is what you want.
| tshaddox wrote:
| > Yeah just means you have to rely on linting and your
| ide to catch those errors. And hopefully the rest of your
| team does the same thing.
|
| Yes, and that's roughly what TypeScript is: a linter that
| everyone on your team is running.
| tazard wrote:
| If robot0 name is AAA and uid Is BBB, and robot 1 name is
| BBB and uid AAA, and you call the function
| checkRobot('AAA'), what sort of assert exactly could
| differentiate between a name and a uid?
| chpatrick wrote:
| You can kind of do it: interface RobotName {
| value: string; type: "RobotName"; }
|
| Or if you don't want to make an extra wrapper around every
| object: interface RobotName {
| _phantomType: "RobotName"; } function
| makeRobotName(name: string): RobotName { return name
| as any as RobotName; } function
| getRobotName(robotName: RobotName): string { return
| robotName as any as string; }
|
| Presumably V8 is smart enough to inline the wrapper functions.
| Waterluvian wrote:
| Oh cool. So you basically lie about the runtime structure of
| the type, which works out fine during type checking at
| compilation.
| Waterluvian wrote:
| Discriminators are super power powerful. Love them. But they
| can be just so heavy for things.
| tshaddox wrote:
| The closest way to get nominal typing in TypeScript is with
| type branding. I haven't actually used this small library, but
| it illustrates the idea: https://github.com/kourge/ts-brand
| Waterluvian wrote:
| "nominal typing" thanks for the link and terminology!
| jks wrote:
| Way back in the 1990s I worked in a place where we had to use
| Ada and follow a strict style guide. The guide prohibited using
| raw numeric types (such as "unsigned int" or "double" in C-like
| languages) and required creating a new type (https://en.wikiboo
| ks.org/wiki/Ada_Programming/Type_System#De...) for each
| different quantity, such as temperature or speed. Then you
| couldn't assign a speed value to a temperature variable, or
| compare them to each other or do arithmetic, unless you
| specifically define what the operators mean.
|
| It felt like a lot of busywork, but it did prevent some classes
| of bugs.
| jandrewrogers wrote:
| Many C++ code bases still do this pervasively, it is a common
| practice to improve robustness. I don't know what it is like
| in Ada but C++ metaprogramming makes this not too onerous.
| ThePadawan wrote:
| In the mid-2010s I worked at a company that switched from
| using raw ints to refer to database rows to using (in C#
| terms) "Id<FooTable>".
|
| Across the entire codebase, we discovered an entire class of
| bugs that only never cause any issues because all the
| important rows in all the important tables had Id 1 (e.g.
| Currency 1 was USD and Country 1 was USA) - so in a few
| places where the ints got mixed up, the correct row was still
| accidentally looked up in the wrong table).
| macintux wrote:
| Something I learned long ago, but occasionally disregard to
| my peril: if you see something that looks like a bug, but
| the code/system still works, stop and figure out _why_ it
| works.
|
| It's very easy to mentally shrug and move on, but more
| often than not it comes back to bite you; maybe it's a code
| path that's rarely triggered, e.g.
| russianGuy83829 wrote:
| how did that work out in the end?
| Waterluvian wrote:
| Oh wow. Was there a "stomach sank to the floor" sudden
| feeling upon discovery?
| firloop wrote:
| Worked somewhere with an admin tool that did something similar.
| The code had denominated money in cents, except for the Japanese
| Yen, which has no cents so the number used was just the number of
| yen. After someone refactored some related code, for a short
| period of time Japanese users got refunds 100x the amount they
| were supposed to be.
| MostlyInnocent wrote:
| justinator wrote:
| I understand this whole scenario is simplified for story's sake
| but,
|
| if there's old_func and new_func, and the new_func call in the
| else was added that morning, why would the same error of new_func
| getting the wrong amount of arguments happen weeks before?
| bombcar wrote:
| I suspect the new function was an attempt to fix whatever was
| the backend problem (because before it would work the second
| time, but give the correct $250).
| denton-scratch wrote:
| I wondered that. I thought the bug had been around for a while;
| I suppose the morning checkin was a failed fix for the pre-
| existing bug.
| psim1 wrote:
| This blogger occasionally seems to throw in stories that are
| made up for the sake of making a point; this is likely one of
| them, given the inconsistencies.
| javajosh wrote:
| _> I say bare numbers can be poison in a sufficiently complicated
| system_
|
| Indeed. It's interesting that the _de facto_ solution to this is
| to key value pairs, where sometimes the key can be an array (e.g.
| json, xml, etc), and then one can (kinda) infer units from the
| key. But even this is insufficient, because inevitably the value
| is pulled out and it 's context is lost.
|
| This is, I think, a(nother) powerful argument for immutable
| values, and accreting structure losslessly with pointers. Like in
| Clojure. In general we our runtime should support values that
| have monotonically increasing amounts of metadata added to them
| during runtime, for example units, or more paths, such that any
| user of the value can interrogate that structure and find out
| what it meant to the last people to read or write the value.
| paulirish wrote:
| My primary codebase ingests data with a mix of seconds,
| milliseconds, and microseconds. We've held it at bay for a while,
| but it requires some mental overhead unless we've baked the unit
| into each variable name.
|
| We're likely to normalize on milliseconds, but at the same time,
| the implication of floating point arithmetic on all our ms
| numbers isn't ideal.
| kevan wrote:
| In a similar vein, in the early days of launching Relay[1] I
| ended up spending a couple weeks in total squashing timestamp
| bugs. We were integrating with a few existing systems that used
| epoch seconds or milliseconds for timestamps and they usually
| didn't have a hint in the field name to tell what it was so it
| was really easy to miss in code reviews.
|
| Our problems were caused by a mix of serialization format (JSON
| numbers) and not always converting into the language's
| date/time types at the boundary (sometimes raw epoch
| seconds/millis were passed around layers of code and only
| parsed into a date for display. That created opportunities for
| misinterpretation at every function call.
|
| My general rules for non-performance critical code are
|
| 1. Always Parse into a first-class date/time/duration type at
| the serialization boundary.
|
| 2. Always use an unambiguous format (e.g. ISO-8601) for
| serialization
|
| It's not the most efficient but lets you rely on the type
| system for everything in your code and only deal with
| conversion at one place.
|
| [1] https://relay.amazon.com/
| SideburnsOfDoom wrote:
| Durations aren't ints, so find or make a type suitable for
| storing them.
|
| For a similar issue (cache durations that were expressed as a
| mix of milliseconds, seconds, and minutes) I now insist that
| the framework type `TimeSpan` is used instead for expressing
| these durations - as that's exactly what it was designed for.
| fefe23 wrote:
| Had they had unit tests, this would never have gotten that far.
|
| Have unit tests!!
|
| Could you do other things with those credits, like use them in
| the cafeteria or resell them? Why were they so frantic to close
| the loophole?
| fbdab103 wrote:
| Maybe we are misinterpreting the urgency of the messaging. I
| could just as easily read it as, "Hey guys having a problem
| with the pseudo-money credit. Shutdown until further notice."
| wrigby wrote:
| No, the credits could only be used to run ads. However, the ads
| were run publicly, and competed in ad auctions with campaigns
| from actual customers paying actual money.
|
| If employees had a $25k ad budget, that would mean increasing
| the demand for ad placement by $25k, which could actually
| affect the ad campaigns of real customers - definitely not
| ideal.
| bornfreddy wrote:
| To be pedantic, you would need _integration_ tests to catch
| this. Both of the units (probably) worked just fine. So:
|
| Have unit tests and integration tests!!
|
| :)
| fguerraz wrote:
| I thought it was going to be an article about fiat currencies.
| USD is funny money after all...
| Salgat wrote:
| Reminds me of in C# how you can pass either an integer for time
| (usually in milliseconds) or pass in a TimeSpan object. I always
| use TimeSpan instead of an integer for exactly this reason.
| TillE wrote:
| Similarly, C++11 added a chrono::duration type. It's a massive
| improvement over the mess of C APIs which might expect anything
| from nanoseconds to whole seconds.
| Konohamaru wrote:
| Type systems are to programmers what dimensional analysis is to
| physicists.
| ok123456 wrote:
| You can do dimensional analysis with type systems. It's part of
| the F# standard library, at least for integral dimensions.
| tapvt wrote:
| I put myself in this very position with some cache expiration
| times.
|
| Almost everything in the server-side that is related to times for
| a particular client uses milliseconds. Of course, redis's `SETEX`
| does not. It uses seconds.
|
| The data got quite stale. But, much like this post, other bugs
| were uncovered and usefully fixed.
| aeyes wrote:
| For some time the redis-py library had the increment and member
| argument order of ZINCRBY switched from what standard Redis
| uses. Of course we didn't notice this small difference.
|
| Debugging this was harddd.
|
| They later switched this around, luckily I read the changelog.
| caseysoftware wrote:
| I was working with [top 5 bank in the US] and hit the same issue
| with regards to their interest rates between products.
|
| Some represented interest rates as a simple number "5%" others as
| a decimal "0.05" and others as basis points (500). It caused some
| HUGE problems internally as less clueful loan officers were
| plugging 0.05% interest rates into formulas for customers. The
| naming was just as bad. We had columns and fields called intRate,
| int_rate, interest_rate, and of course iRate.
|
| I asked people if they were irate over the problem. No one
| laughed. :D
___________________________________________________________________
(page generated 2022-12-03 23:01 UTC)