[HN Gopher] static_assert is all you need (no leaks, no UB)
___________________________________________________________________
static_assert is all you need (no leaks, no UB)
Author : kris-jusiak
Score : 93 points
Date : 2023-04-10 10:44 UTC (12 hours ago)
(HTM) web link (twitter.com)
(TXT) w3m dump (twitter.com)
| dig1 wrote:
| This makes sense for straightforward tests, but static_assert is
| _not_ all you need in general, because some things has be
| executed in runtime, after series of steps or after some
| timeframe. Good luck reproducing or testing these in compile-
| time.
| mr_00ff00 wrote:
| Could you not put a series of steps or something that mocks
| time into constexpr? A comment mentioned above that in C++20,
| almost all features are available at compile time now.
| ovao wrote:
| With few limitations, yes. In C++20, you could for example
| test for constant evaluation[1], using that as a mechanism to
| fall back to a real clock during runtime.
|
| [1]: https://en.cppreference.com/w/cpp/types/is_constant_eval
| uate...
| kris-jusiak wrote:
| With C++20 almost anything can be used in constexpr context
| (vector, unique_ptr, virtual, function, etc.) and as long as it's
| in the scope it can be tested at compile time which guarantees
| memory safatey, no UB, etc. Additionally, since constexpr can be
| executed at run-time and code has been tested at compile-time
| already therefore 'static_assert' is (almost) all you need -
| https://godbolt.org/z/P4cqboGx6.
| mikepurvis wrote:
| From an overall performance point of view, I wonder how the
| timing works out for the compiler to run your unit tests like
| this vs to produce and invoke a binary.
|
| I bet it's mostly a wash, and the ergonomics of conventional
| gtest macros look way better to my eye.
| ynik wrote:
| The idea to let the compiler run the tests only works if you
| constexpr everything, which means putting all code in
| headers. This effectively means giving up on separate
| compilation. Worse, if you use some mixture (most code in
| headers but you still have >1 compilation unit), your compile
| times completely explode as essentially all code is compiled
| repeatedly for each unit.
| cozzyd wrote:
| There are other ways to do this. I made a proof of concept
| of using linker sections to allow you to sprinkle tests
| within the implementation inline once...
| https://github.com/cozzyd/examc (this is obviously not
| production-ready, just serves as a proof of concept).
|
| Basically the idea is that the test code gets written to a
| different linker section that your test runner can iterate
| through, when tests are enabled. This is easy on gcc
| because it generates automatic constants for the beginning
| and end of different linker sections. There may be away to
| do this with clang as well, but I never use clang.
| kris-jusiak wrote:
| Hmm, there are obvisouly trade offs (it depends on the
| compiler how many tests, how are they written, etc.) but for
| apples to apples comparision the gtest binary would have to
| be either compiled with sanitizers (that would be probably
| slower to compile than static_assert tests without
| sanitizers) or run with valgrind or similar (execution would
| be much slower, static_asserts tets don't have to be
| executed, compiles=green).
| gpderetta wrote:
| Pragmatically you can write your code in such a way that you
| can get immediate feedback in your IDE as you write the code
| if a static assert fails as you are implementing your
| function.
|
| You could of course set up your runtime tests in a similar
| way, having the ide run them back to back as you are writing
| code, but it is more complicated, especially if the code is
| in an intermediate state that it is not fully compilable.
|
| So in the end it is not a huge breakthrough, but having
| compile time tests is still quite a nice feature.
| saagarjha wrote:
| Have zero cost abstractions gone too far?
| AshamedCaptain wrote:
| I really don't think it works that way. You can for sure write
| code that will leak and otherwise be UB but run compile-time
| just fine.
| dgellow wrote:
| https://en.cppreference.com/w/cpp/language/constant_expressi.
| ..
|
| Point 8:
|
| > A core constant expression is any expression whose
| evaluation would not evaluate any one of the following:
|
| > 8. an expression whose evaluation leads to any form of core
| language undefined behavior (including signed integer
| overflow, division by zero, pointer arithmetic outside array
| bounds, etc).
| AshamedCaptain wrote:
| This does not really "guarantee" anything. You can still
| have as much UB (incl leaks) as you want, as long as they
| are not evaluated at compile-time. i.e. it's at best
| equivalent to running valgrind.
| dgellow wrote:
| What would be examples of runtime UBs that wouldn't be
| reported when used within a constexpr?
| simiones wrote:
| The point is that the code is only checked for UB with
| the arguments it is given at compile time. There is no
| guarantee that it can't invoke UB with other arguments it
| might receive at runtime.
|
| For example, here is a modification of the original
| program that does invoke UB, but compiles just fine:
|
| https://godbolt.org/z/MaqhcEYqn
| dgellow wrote:
| Oh yes, I see your point. Thanks for clarifying
| layer8 wrote:
| I hope there's also some text in the standard prohibiting
| implementations from allowing any other expressions as a
| constant expression (which they otherwise could as a
| language extension), and thus requires compilation failure
| for such expressions?
| kps wrote:
| Pragmatically, you can't stop extensions; if the fine
| print for `--std=cool++23` says that this mode is not
| actually C++23-compliant, nearly nobody will ever notice
| or care. Pragmatically, if a popular compiler makes
| `--std=cool++23` the default, and requires `--std=C++23
| --iso-eic-jtc1-sc22-wg21 --pompous` to get standard-
| conforming behaviour, nearly nobody will do that; instead
| they will complain that _other_ compilers lack the
| extensions.
| layer8 wrote:
| Extensions can be standard-compliant, in the sense that
| they don't violate any prescription by the standard, and
| thus a program cannot assume their absence. My question
| was whether the standard actually takes care to render
| the acceptance of constexprs-with-UB non-standard-
| compliant. That is, in addition to "must accept X", does
| it also say "must _only_ accept X"?
| tlb wrote:
| Indeed, it fails if you invoke UB. For example, an int
| overflow in a constexpr causes a compile error:
| <source>:5:17: error: static assertion expression is not an
| integral constant expression
| static_assert(1024*1024*1024*3 != 0, "UB");
|
| https://godbolt.org/z/Wc591En7E
| simiones wrote:
| That only works if you know the values at compile-time
| though: constexpr int foo(int x) {
| return 1024*1024*1024*x; } int main()
| { int y; std::cin >> y;
| static_assert(foo(1)); //all good foo(y);
| //oops, UB if user enters 7 }
|
| https://godbolt.org/z/K8h4Kj99s
|
| Edit: small correction so that the numbers are big enough
| to cause problems...
| mort96 wrote:
| Yeah, the tests will only fail if the tests trigger UB.
| It's like all testing, it only detects issues if you
| trigger the issues in the tests. Using static_assert as
| your test system obviously doesn't obviate the need for
| writing good tests.
| tsimionescu wrote:
| Many people in this thread think that if a constexpr
| function can be called at compile time successfully, it
| will also be guaranteed not to have UB at runtime, in
| general.
|
| I was pointing out that this is only true for the cases
| you actually test, not the general case.
|
| Even then, it's not fully true, as a function may have
| different behavior at runtime as opposed to compile time
| (e.g. because of multi-threading), and so it may display
| UB even when called with the same arguments that didn't
| display UB at compile-time.
|
| Overall, this static_assert trick is just a nice way to
| make sure your tests don't accidentally pass while still
| invoking UB, to protect from false negatives.
| mort96 wrote:
| Aha, I think I see. Upon reflection, I do remember that
| some people seem to think that UB is a property of the
| code, not the execution; that a piece of code either
| "contains UB" or does not. I suppose it makes sense then
| that some people may think that code which works under
| constexpr can't "contain UB" and get the wrong idea.
| mgaunard wrote:
| Try forcing the call to happen at compile-time.
|
| constexpr requires that the function be callable at
| compile-time, not that it is so.
| tsimionescu wrote:
| You can't call an expression whose value is IO-dependent
| in a compile-time context, obviously.
|
| The tweet seems to imply that if you can call your
| functions at compile time, they will not present UB at
| runtime either. I am trying to point out that that is not
| the case at all.
| mgaunard wrote:
| That seems unrelated to the sub-thread in question.
| tsimionescu wrote:
| The example program is calling foo(y), where the value of
| y is read from standard input. You suggested to evaluate
| that expression at compile time, which is not possible.
| How is this unrelated to the thread?
| mgaunard wrote:
| By forcing the call to happen at compile-time, he would
| get an error.
|
| The input being potentially known at compile-time is not
| sufficient to make a constexpr function be called at
| compile-time.
| tsimionescu wrote:
| Yes, but that error would simply complain that a variable
| that doesn't have a compile time known value is been used
| in a compile time context.
|
| Conversely, the origin of the thread implies that if a
| function has been successfully called with some argument
| from a static_assert, it will not have UB at runtime even
| if called with some _other arguments_. This subthread was
| showing that this is not the case, and that you can 't
| test all uses of a function using static_assert to
| guarantee that it will not exhibit UB.
|
| In case it's not clear, this piece of code will also fail
| to compile, even though it exhibits no UB:
| constexpr int foo(int y) { return y; }
| int y; std::cin>>y; static_assert(foo(y));
|
| https://godbolt.org/z/xvhP8nq7z
| kris-jusiak wrote:
| constexpr has to checked for leaks and UB so as long as there
| is coverage at compile-time (static_assert + constexpr) I
| would assume there shouldn't be neither leaks nor UB. But the
| context is limitted where that can be applied and actually
| compiles. For example, there is no way to do it with global
| variables but with limited scope that's possible.
| AshamedCaptain wrote:
| At the very minimum, you can simply do
| is_constant_evaluated to change behavior depending on
| runtime vs compile-time...
| the_mitsuhiko wrote:
| I _think_ you cannot conclude from a constexpr not having
| UB at compile time that it won't have UB at runtime.
| DannyBee wrote:
| The standard explicitly disallows compiling of UB for
| constexpr.
|
| Otherwise, what you write is provably correct - UB is not
| statically decidable in all cases (and depending on the
| type of UB, not even in a lot of cases).
| ithkuil wrote:
| Doesn't it depend on the actual values?
|
| If your compile time evaluations don't trigger signed
| integer overflow (or any other UB) does it follow that at
| runtime you couldn't pass a parameter that would trigger
| signed overflow?
|
| I mean it's still useful because at least you know your
| test code is not artificially passing because of some UB
| makes it look like passing
| DannyBee wrote:
| No, it doesn't depend on the values. Yes, it does follow.
| The compiler is not allowed to compile constexpr code
| that could produce UB at runtime. period. :)
|
| That is, conditional constepxr code that depends on
| values and could produce UB is not valid constexpr code,
| and a compiler is not supposed to compile it.
|
| This is very explicit in the standard.
|
| Think of it as a statically decidable set of code.
|
| Now, it wouldn't shock me if compilers don't achieve this
| right now, but the standard is clear that constexpr code
| may not contain operations that could produce undefined
| behavior at runtime.
| simiones wrote:
| That is a complete misunderstanding.
|
| A constexpr function can very well take an input, and it
| can invoke UB based on the runtime value of that input.
|
| What the standard prohibits is compile-time _evaluation_
| of an expression which invokes UB. So, if you actually
| call your function at compile-time with a constexpr value
| that ends up invoking UB in the function (say, an integer
| overflow), THEN the standard mandates that the compiler
| throw an error rather than compiling some random value
| in.
|
| For example: constexpr void foo(int x) {
| std::cout << 1024 * 1024 * 1024 * x; }
| int main() { static_assert(foo(100)); // will
| fail because computing 1024 * 1024 * 100 is signed
| integer overflow, which is UB foo(100); //
| invokes UB at runtime; in practice, will perhaps print
| some overflowed value }
|
| Edit: I should also add that you can very well invoke UB
| in a constexpr expression if it is standard library UB
| and not core language UB (e.g. if you try to pop() from
| an empty std::vector).
| robotresearcher wrote:
| You're both right, but you're talking about different
| things.
|
| If you use the same values at run time as you use in the
| tests, the tests predict the behavior as desired.
|
| If you use different values at run time as you use in the
| tests, the tests do not predict the behavior as desired.
| tsimionescu wrote:
| > You're both right, but you're talking about different
| things.
|
| The GP post is explicitly claiming that the values don't
| matter, and that per the standard a constexpr function
| should be guaranteed by the compiler to be incapable of
| producing UB (or else compilation should fail and it
| should not be allowed to be declared constexpr). This is
| simply false. Here is the GP statement:
|
| >> That is, conditional constepxr code that _depends on
| values_ and _could_ produce UB is not valid constexpr
| code, and a compiler is not supposed to compile it.
| [emphasis mine]
|
| > If you use the same values at run time as you use in
| the tests, the tests predict the behavior as desired.
|
| Even this is not fully guaranteed by the standard, as
| core constant expressions must only be guaranteed not to
| invoke core language UB. They may still be considered
| core constant expressions and evaluated at compile time
| even if they exhibit standard library UB (e.g. if they
| are not respecting some preconditions of an std::vector
| method).
| tialaramex wrote:
| > it's still useful because at least you know your test
| code is not artificially passing because of some UB makes
| it look like passing
|
| Right, that's the extent of what this does. When I saw it
| on r/cpp I thought OK, somebody realised now they can
| make their C++ tests work as a reasonable person would
| expect, or perhaps realised that _without this_ C++ tests
| are almost worthless because they can invoke Undefined
| Behaviour silently.
|
| But increasingly I suspect the OP mistook this for a
| breakthrough in correctness which it isn't, otherwise why
| post it to HN?
|
| On the other hand, the prohibition on UB for constexpr
| doesn't reach up to where IFNDR lives, so I'd guess most
| non-trivial C++ software is technically nonsense with no
| defined meaning as a result of IFNDR regardless of how
| many or few unit tests were written or whether they use
| constexpr to prohibit Undefined Behaviour. A cheerful
| thought.
|
| [Ill-Formed, No Diagnostic Required: A recurring
| statement in the C++ ISO document which basically says if
| you did this then too bad, that's not a well-formed C++
| program, however your compiler may not notice that this
| isn't a C++ program, so, your program might compile, and
| even execute, but what if anything happens when you run
| it isn't specified in this ISO standard, good luck.]
| jenadine wrote:
| Unless you used std::is_constant_evaluated() and the code
| running at runtime was not the one tested at compile time
| kris-jusiak wrote:
| I guess so, can't think of an example now but I'm pretty
| sure there are subtle corner cases (as always) and it
| depends on the testing, coverage and potential
| limitations of checking things at compile-time, though,
| IMHO, the technique is promissing and can help with a lot
| of use cases but defo not everything.
| layer8 wrote:
| (nevermind)
| jvanderbot wrote:
| I think it's worth pointing out that your statement is
| true by definition, perhaps not true by implementation in
| these cases. It's not like UB produces _random_ behavior,
| it's just not specified what compilers _should_ do in
| those cases.
|
| Of course, cannot be relied upon.
| [deleted]
| jcelerier wrote:
| Successful compilation of UB is explicitly disallowed by
| the standard in a constexpr context
| layer8 wrote:
| Thanks, I wasn't aware. Although that seems to be
| restricted to core language UB and not include standard-
| library UB: https://stackoverflow.com/a/72494688/623763
| soulbadguy wrote:
| >constexpr has to checked for leaks and UB so as long as
| there is coverage at compile-time
|
| Source ?I am not sure constexpr give any garanty regarding
| UB and/or leaks
| 3836293648 wrote:
| Constexpr at compile times gives that guarantee, not
| constexpr in general. Hence static assert to force
| comptime evaluation
| soulbadguy wrote:
| > Constexpr at compile times gives that guarantee
|
| Can you point to a source for that ? I am not trying to
| be pendantic, but genuilly curious
| zerr wrote:
| Depends on how you define a class. E.g. you are _not_ using
| std::list in your example.
| fefe23 wrote:
| This is awesome!
|
| I wonder what the compile time cost would be to just have this
| kind of unit testing in all the time.
| kris-jusiak wrote:
| Thanks! Most likely not yet applicable at Google's scale but
| smaller project can defo leverage the approach. Personally, I'm
| writing most of my tests this way and with TDD the red phase is
| always a compilation fail which is quicker than buiding and
| running in my experience. But that's for a medium size project.
| But as always it depends there are trade offs.
| macgyverismo wrote:
| I've been using this technique as well, but I found that
| debugging static_asserts is quite hard. I often fall back to
| calling the failing test at runtime and stepping through. Any
| suggestions for a different workflow?
| kris-jusiak wrote:
| IMHO the best approach is to avoid the problem by applying
| TDD. Then there is very little need to debug anything. But
| otherwise, there is https://github.com/mikael-s-
| persson/templight for compile-time debugging which is
| pretty cool and having something like `expect(auto... args)
| static_asert(args...); assert(args...);` may help with
| being able to debug at run-time and get the coverage
| (though, the code has has to compile aka pass first).
| rphil wrote:
| Keep tests and code separate, and use tdd:
| https://github.com/yellowdragonlabs/TDD
|
| I usually have dozens/hundreds of tests that run faster
| than you can compile #include <vector>.
|
| A shell script executes all tests automatically every
| time I save. It's very nice to watch the output while
| coding, with almost no latency.
| 0xbkt wrote:
| What's going on in this code for someone completely alien to C++?
| tialaramex wrote:
| The C++ code is a simple list type, and also a bunch of tests
| for that type to confirm that it works as intended. The crucial
| trick here is that because static asserts are used, the test
| values are computed during compilation, such computations are
| forbidden (by the standard) from having any leaks or Undefined
| Behaviour. Anything allocated must be freed by the time the
| tests complete, and no language Undefined Behaviour is
| permitted.
|
| The latter is pretty normal for other languages but is a big
| deal in C++ where UB is a constant plague. However many
| languages have either forbid or have strict limits on compile
| time heap allocation - after all that heap isn't going to still
| exist at runtime. Requiring that you free everything allocated
| fixes that hole _and_ means you get free leak detection.
| cubancigar11 wrote:
| It is creating a bunch of objects, modifying them, then
| asserting their value all at compile time. For example the
| first example creates a list and asserts its size is 0. List
| normally allocated on heap, so I am guessing they have made
| changes in thay area in c++20 by making it Constexpr, which is
| a fancy way to say an expression can be known at compile time.
| ksherlock wrote:
| The compiler is doing a bunch of complicated stuff at compile
| time. In fact it's both a C++ compiler and a C++ interpreter.
|
| static_assert is a compile-time check.
|
| [] { ... }(); is an immediately executed lambda function (IIFE
| in javascript parlance).
|
| list<int> list{} is a linked list of integers (double-linked,
| forward and backward). push_back() allocates more memory. pop()
| / clean() deallocates memory.
| leni536 wrote:
| Per standard wording if an evaluation contains an undefined
| operation then it is not a constant expression. It means that
| undefined behavior should result in compilation failure if it
| happens in a context where a constant expression is required,
| like in static_assert.
|
| However the standard only requires this for language-level
| undefined behavior. For undefined behavior happening in the
| standard library it's unspecified whether the expression is a
| constant expression or not. So no, constexpr tests don't cover
| all possible UB.
|
| Also even if in theory language-level undefined behavior should
| be caught in constant expressions, in practice compilers miss a
| number of undefined behaviors. They are generally good at
| catching out-of-range indexing, using objects outside of their
| lifetime, using uninitialized values, signed integer overflow and
| modifying const objects. However there are a number of subtle
| undefined behaviors that they don't catch, like unsequenced
| operations on the same object, invalid values for unscoped enums.
|
| There might be some overlap with runtime tests with
| -fsanitize=undefined,address. For catching uninitialized values
| at runtime though you probably need msan, which is a pain to set
| up, but constexpr tests cover that. On the other hand the
| function you test might not be available at compile-time.
|
| Anyway, constexpr tests are a valuable tool. It's not a silver
| bullet.
| mgaunard wrote:
| New user of C++ discovers C++ has a mechanism for code to be
| evaluated at compile-time, and decides to share his findings on
| twitter.
|
| Why is it a story? Slow day on the Internet?
| gpderetta wrote:
| Someone found ot interesting enough to write about this and
| someone else found it interesting enough to upvote.
|
| In the end it just an excuse to have a discussion on an
| interesting topic.
|
| And yes, it is a slow day.
| JTyQZSnP3cQGa8B wrote:
| (Offtopic but) Kris, I have used your micro unit-test library in
| the past and it was a pleasure to look at your code. You're the
| kind of crazy guy (in a good way) that gives me the motivation to
| learn new stuff. Thanks.
| rphil wrote:
| All you need is tdd https://github.com/yellowdragonlabs/TDD
|
| - Separate tests from code.
|
| - Compiles instantly.
|
| - Generates tests more powerfully than templates.
|
| - Can access private members.
| goldbattle wrote:
| Interesting work. Thanks for open sourcing. It would be nice if
| the "run" script has a bit of documentation, and maybe an
| "examples" folder the the run script can work with. From what I
| can see it seems to search the parent directory? I would want
| for example `./run <dir_with_tests>` to run all tests my
| specified project repo.
| [deleted]
| mrlonglong wrote:
| You need C++ 23 to run this code. I tried it out with C++ 20 and
| got compile errors.
___________________________________________________________________
(page generated 2023-04-10 23:02 UTC)