[HN Gopher] Running C unit tests with Pytest
___________________________________________________________________
Running C unit tests with Pytest
Author : p403n1x87
Score : 112 points
Date : 2022-02-11 14:31 UTC (1 days ago)
(HTM) web link (p403n1x87.github.io)
(TXT) w3m dump (p403n1x87.github.io)
| a-dub wrote:
| this seems cool, but... brittle? especially embedding build steps
| in python.
|
| i've used check in the past and was happy enough with it, it
| didn't take long to learn. all unit test frameworks are basically
| the same at their core.
|
| interop between c and python with ctypes is fun though.
| ereyes01 wrote:
| Really cool. Several jobs ago, I used cffi to create bindings to
| a library that controlled a camera's pan/tilt/zoom motors. Those
| were used to implement a test suite that validates the cameras in
| the manufacturing facility before shipping. The embedded
| developers also found being able to use the cffi bindings in the
| REPL really useful when prototyping changes. Python is a really
| useful tool for these kinds of interfaces.
| masklinn wrote:
| @pytest.fixture def libfact(): yield
| CDLL("./fact.so")
|
| FWIW that's pretty confusing as it gives the impression the
| library is reloaded for every test, but iirc dlopen() will just
| return a handle to the existing one. I don't think ctypes has a
| good way to unload dlls so it should probably be
| `fixture(scope='session')`.
|
| That gets more relevant when on-the-fly compilation is added to
| the mix, spawning a compiler for every test is a complete waste
| of time.
| slowking2 wrote:
| You can use tempfiles and (on OSX) use the private API of
| ctypes to dlclose to close the handle. Different call on
| Windows I think. Something like import _ctypes
| import shutil import tempfile @pytest.fixture
| def libfact(): tmp =
| tempfile.NamedTemporaryFile(delete=True)
| shutil.copy2("./fact.so", tmp.name) lib =
| CDLL(tmp.name) yield lib
| _ctypes.dlclose(lib.handle)
|
| EDIT: fixed error mentioned in reply.
| alblue wrote:
| Did you mean to have the `tmp.name` as the library passed
| into `CDDL`? If not, what's the purpose of the copy?
| slowking2 wrote:
| Yes! Thank you. Fixed it.
| p403n1x87 wrote:
| That's a good observation. Towards the end the fixture is
| dropped in favour of a module-like object, which spaws the
| compiler once per test run. One could enhance this to skip
| compilation in the sandbox process to avoid unnecessary
| compilations.
| stefanhoelzl wrote:
| This similar project could also be interesting
| https://github.com/mrh1997/headlock
|
| It is used at my company to test our firmware written in C with
| python and was developed by a colleague.
| haberman wrote:
| A notable downside that is not mentioned: the Python interpreter
| is far from Valgrind-clean. Valgrind is generally a powerful tool
| for debugging memory errors, but if you wrap your C code in
| Python, Valgrind will be so noisy as to be ineffective. Python
| startup is also very heavyweight; combined with the ~60x slowdown
| from Valgrind this is something you are going to notice.
|
| The Python interpreter does not even use malloc()/free() directly
| by default; it layers its own memory allocator on top called
| PyMalloc (https://docs.python.org/3/c-api/memory.html#pymalloc).
| You can disable this by setting an environment variable
| (PYTHONMALLOC=malloc), but even then you will see many Valgrind
| warnings inside the Python interpreter itself.
| p403n1x87 wrote:
| That's a good argument for keeping these kind of tests
| separate. Indeed Austin has dedicated Valgrind integration
| tests just for this reason.
| rustybolt wrote:
| Shameless self-plug since I recently made a small header-only C
| testing "framework":
|
| https://github.com/rubenvannieuwpoort/c_unit_tests
|
| Feels a bit more in line with the spirit of C (small language,
| few dependencies).
| matheusmoreira wrote:
| This is really nice. I wonder if there is a way to make this
| work without constructors.
| ant6n wrote:
| Nice. Now how could I get this to work on a small embedded
| device compiker (gbdk using lcc), which likely doesn't have
| that constructor attitude.
| ant6n wrote:
| ...after some thinking, maybe it would be possible to do a
| manual preprocess using a simple python script that collects
| all the tests and inserts them in the test main.
| JonChesterfield wrote:
| One approach is to build test traversal machinery out of
| function static variables and have every TEST() macro turn
| into a function that takes a pointer to some state
| controlling which test to run next.
|
| You don't need malloc if using local variables either, which
| is helpful when your target doesn't have malloc or when
| debugging memory problems on targets that do. It's nice to
| know the test framework cannot be leaking or use-after-free
| anything.
| simonw wrote:
| This is brilliant.
|
| pytest is by far the most productive testing framework I've ever
| tried, for any language. It's so good that it switched me from
| treating tests as a necessary chore to actively enjoying writing
| them.
|
| Using ctypes to exercise a C module like this is brilliant -
| especially the mechanisms used here to work around segmentation
| faults.
|
| Reminds me of SQLite, which is written in C but uses TCL for most
| of its test suite.
| drchickensalad wrote:
| How do you add a side effect and still call the original in
| pytest? This is easy in rspec but everywhere I google people
| say there's no good way.
|
| For example, you're calling code which internally does
| Too().bar() and you want to advance your time mock after it's
| called.
| masklinn wrote:
| That... has nothing to do with pytest?
|
| You'd probably use something like `wraps` or `side_effect` in
| `unittest.mock` to delegate to the original with some extra
| behaviours.
|
| And / or use freezegun's tick features, when you're
| specifically dealing with time.
| simonw wrote:
| There are lots of good mocking fixtures available for pytest.
| I've used this one for clock stuff in the past:
| https://github.com/adamchainz/time-machine#pytest-plugin
| mhh__ wrote:
| When I need to test C I just use D unittest blocks.
|
| You can import the C header file directly now so it's literally
| trivial.
| wokaonima wrote:
| How would you go to do this for c++? Specifically for classes and
| templates, is there an easy way to call code without having to
| manually write the demangled names of the functions?
| [deleted]
| timeinput wrote:
| Not that I know of. If I were doing it I'd expose a library /
| interface with pybind11 to test, but really I find Googles unit
| testing framework or boosts to both be pretty effective.
| pkhuong wrote:
| We've had a lot of success combining that approach (cffi instead
| of ctypes) with property-based testing
| (https://github.com/HypothesisWorks/hypothesis) for the query
| engine at backtrace:
| https://engineering.backtrace.io/2020-03-11-how-hard-is-it-t... .
| jaromir_ wrote:
| Have you exposed the C library to Python via ctypes as OP or
| have you taken a different approach?
| pkhuong wrote:
| Just cffi, and keeping the headers free of inline
| implementation noise (a side benefit, in my opinion).
| smitty1e wrote:
| I doubt that it's pytest-specific, but the idea of using a unit
| test tool to explore a new problem space and build up to working
| product is powerful.
|
| Maybe not so much in the UX space, but for ETL and back-end work,
| I'm a fan.
___________________________________________________________________
(page generated 2022-02-12 23:00 UTC)