[HN Gopher] Precisely: Better assertions for Python tests
___________________________________________________________________
Precisely: Better assertions for Python tests
Author : trymas
Score : 49 points
Date : 2021-02-18 08:58 UTC (14 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| crazypython wrote:
| See also: https://github.com/Parquery/icontract
| [deleted]
| tom_mellior wrote:
| The original title is "Precisely: better assertions for Python
| tests", and it's far superior to the one submitted here.
| fnord123 wrote:
| Unfortunately that is a misleading headline since these are not
| better assertions for Python tests. These appear to be an
| attempt to infect Python code with pointless verbosity found in
| AssertJ or Hamcrest for Java.
| whalesalad wrote:
| unittest already has a lot of really helpful assertions - I just
| wish they were more accessible outside of a standard class-based
| tests. https://docs.python.org/3/library/unittest.html
|
| I tend to try and lean on the language as much as possible before
| pulling in libraries like this. This is the sort of thing that
| you end up regretting a year or two later when the author
| abandons the project and you can no longer get help for bugs.
| Unlike vanilla language assertions, you won't need to rewrite
| your entire test suite when that situation inevitably comes up.
|
| pytest will give you really awesome feedback with vanilla/basic
| 'assert x in y' statements
| calpaterson wrote:
| It would be great if they could go into more detail on what
| they've improved over pyhamcrest
| (https://github.com/hamcrest/PyHamcrest) which I do use on
| occasion when there is no other easy way to write a good matcher.
| That said though, pytest's assertion rewriting goes a long way
| towards making hamcrest-style assertions redundant. But not
| completely.
| mwilliamson wrote:
| (I'm the author of precisely)
|
| Better error messages was the motivation: PyHamcrest tends to
| put everything onto one line, which makes it hard to discern
| structure. In contrast, precisely tries to produce an error
| message that uses line breaks and indentation to match the
| structure of the matcher, hopefully making it clear exactly
| which part of the matcher has failed.
| neolog wrote:
| That sounds like a feature PyHamcrest could add. Did you try
| adding it there?
| olau wrote:
| pytest assertion rewrite in the standard Python library would
| be really, really nice. I don't think the standard lib gets
| enough love.
| BerislavLopac wrote:
| Do you have an example of the "not completely" case?
| calpaterson wrote:
| It's still not that easy to assert on some of the keys in a
| dict but not others, especially if you only want to assert on
| some of the keys nested inside that key, and so on. There are
| a few other cases that perhaps don't come to mind straight
| away, many to do with having complex rules for what is
| allowed to be in another thing.
| wodenokoto wrote:
| Why choose preceisely over assert all([el in
| result for el in ['A', 'B']])
|
| There is definitely room for more brainfarts in the above, but
| introducing a new library with its own semantics into a team also
| have its costs.
| BerislavLopac wrote:
| This example doesn't test for absence of any other elements, so
| it's not as precise.
|
| That said, I largely agree with you - checking for this exact
| example from the README file works much better (and IMO more
| readable) as: assert set(result) == {"a",
| "b"}
|
| The precisely example doesn't even check if the result is a
| list, so it can be anything.
| oli5679 wrote:
| To avoid repetitions, can you do this?
| assert sorted(result) == sorted(['a','b'])
| BerislavLopac wrote:
| Or even better (since we're testing against an oracle value
| anyway): assert sorted(result) == ['a',
| 'b']
| oli5679 wrote:
| True you can just sort your oracle values once/in your
| head this is a valid point. Reasonable to prefer this
| way!
| wodenokoto wrote:
| I think that only works if `result` is a list
| alexchamberlain wrote:
| Nah, sorted can be used on any iterable
| quietbritishjim wrote:
| Ah that's better than my collections.Counter trick. Yours
| only needs the items to comparible rather than hashable,
| which is usually a looser requirement. (In principle you
| could have an object that's hashable but not comparible I
| can't immediately think of any).
| duckerude wrote:
| The main problem is probably heterogenous collections. If
| you have a collection with both ints and strings then
| hashing will work fine but comparing will fail (in Python
| 3).
|
| (Frozensets are hashable but not sortable but that's
| probably more niche.)
| wodenokoto wrote:
| Thanks for pointing out my brain fart :)
|
| If the author of the library is reading, I think it would be
| a great addition to have a section discussing the benefit vs
| writing your own one line checkers.
| mumblemumble wrote:
| One of the biggest benefits of libraries like this is that
| they tend to explicitly state the test author's intent.
|
| To take your example from up top: If I'm coming to the code
| later, and I notice that it's not asserting for the absence
| of anything, it may be very difficult for me to tell
| whether that behavior is your intent, or just a brain fart.
|
| It's just another layer of confusion in the many layers of
| confusion that lead to test rot.
| quietbritishjim wrote:
| That is better, but doesn't check for incorrectly repeated
| elements: result = ["a", "b", "b"]
| assert set(result) == {"a", "b"}
|
| In fact that is the first main motivating failure case
| discussed in the readme. But actually that's simple enough to
| deal with too: from collections import
| Counter assert Counter(result) = {"a": 1, "b": 1}
|
| But would be trickier for mutable / unhashable types (not
| sure if "precisely" can deal with those).
| mwilliamson wrote:
| Yup, precisely should work fine with mutable / unhashable
| types -- it just relies on equality to check the elements
| in an iterable.
|
| More specifically, writing:
| contains_exactly("a", "b")
|
| is equivalent to:
| contains_exactly(equal_to("a"), equal_to("b"))
| mwilliamson wrote:
| (I'm the author of precisely)
|
| If you were to ask me "Should I use precisely in my tests?", my
| answer would be: it depends. The main benefit is to better
| describe the intent of your test, so that the assertion is
| neither under- nor over-specified, with the intent directly
| stated.
|
| Assuming your assertion is meant to be an alternative to the
| example in the README: assert_that(result,
| contains_exactly("a", "b"))
|
| I'd suggest that the above states the intention of the
| assertion, rather than how you check it. For instance, your
| assertion would allow duplicate elements, whereas the assertion
| as originally written would suggest that this isn't desired. As
| other comments have pointed out, you can do things with sorted,
| Counter or set (depending on exactly what you want to assert),
| but why worry about what trick to use when you could just
| directly state your intention?
|
| The assertion using precisely is also (arguably) easier for a
| reader to know what is (and isn't) being asserted in the test,
| and makes the test less brittle since you're not accidentally
| asserting more than intended (for instance, it's common to
| assert equality with a list, even though you don't actually
| care about order).
|
| Another common case is when you want to make assertions on a
| collection, but equality would check too much. For instance,
| suppose you have a function that fetches users from a database.
| The fetch can return the users in any order, and you just want
| to check the names of the returning users, so you can write
| something like: assert_that(result,
| contains_exactly( has_attr(name="Alice"),
| has_attr(name="Bob"), ))
|
| How would you write something that means the same thing without
| precisely? The order isn't deterministic, so you can't write
| something like: assert result[0].name ==
| "Alice" assert result[1].name == "Bob assert
| len(result) == 2
|
| We could sort the users by name before making the assertion:
| result = sorted(result, key=lambda user: user.name)
| assert result[0].name == "Alice" assert result[1].name
| == "Bob assert len(result) == 2
|
| Personally, I prefer the precisely assertion!
|
| What about something like an equality assertion?
| assert set(result) == { User(id=1, name="Alice",
| email_address="alice@example.com"), User(id=2,
| name="Bob, email_address="bob@example.com"), }
|
| Now we've over-specified our test -- we need to know irrelevant
| details like the ID and e-mail address of the users, which
| might change and break this test even when the functionality we
| care about still works. We'll also break the test if we add any
| more attributes to users.
|
| As you've mentioned, there's a cost to learning precisely. Even
| if you're familiar with the library, then you're still
| potentially writing a more complex assertion (where more can go
| wrong) than just (for instance) an equality assertion. In my
| experience, on many projects, the ability to state the intent
| of assertions precisely has far outweighed the downsides, but
| that's from a position of already being comfortable with the
| library.
| tgb wrote:
| The error messages in the readme look a lot better than what
| you'd get from doing a ad hoc solution. That's worth a fair
| amount.
| shpx wrote:
| You run the asset statement through pytest, which patches the
| generated bytecode to print a good error message
|
| https://docs.pytest.org/en/stable/
| zoozla wrote:
| Many years ago I wrote https://github.com/elifiner/affirm which
| replaces the built-in assert statement and provides similar (?)
| benefits without changing the syntax.
___________________________________________________________________
(page generated 2021-02-18 23:02 UTC)