[HN Gopher] Sys.monitoring: Python Execution event monitoring
       ___________________________________________________________________
        
       Sys.monitoring: Python Execution event monitoring
        
       Author : ingve
       Score  : 44 points
       Date   : 2023-12-28 07:58 UTC (1 days ago)
        
 (HTM) web link (docs.python.org)
 (TXT) w3m dump (docs.python.org)
        
       | ryan-duve wrote:
       | Does anyone know how this would be used in practice? An example
       | would be useful because words like "tool" and "event" in this
       | context are too abstract to make it clear how this is intended to
       | be used.
        
         | mirashii wrote:
         | One example is reimplementing coverage.py with this, as its
         | made to reduce overhead for the types of tools enumerated in
         | the docs.
         | 
         | https://nedbatchelder.com/blog/202312/coveragepy_with_sysmon...
        
         | everforward wrote:
         | One example I thought of is to use the ability to tap into
         | exceptions being handled to audit for places where performance
         | could be improved by swapping exceptions for sentinel values.
         | 
         | Iirc there's a significant performance penalty for raising
         | exceptions. So you could tap into exceptions being handled to
         | see something like "requests that fail validation raise an
         | exception, and this happens often enough that we could get a
         | performance boost by returning a sentinel value instead".
         | 
         | Another idea is that you might be able to use the conditional
         | hooks to write a tool that can suggest re-ordering your if-
         | elif-elif-else kind of statements so that the most common path
         | is first, and prevent evaluating the other possibilities more
         | often.
         | 
         | At a high level, it's just machinery for getting metadata about
         | your code's runtime as part of the interpreter. You could
         | probably write a lot of this as decorators, but because this is
         | part of the interpreter it's decoupled from your
         | frameworks/libraries/etc.
        
         | fabioz wrote:
         | Well, I'm working on reimplementing the pydevd debugger to use
         | it.
         | 
         | The general idea is that pydevd will be able to use that API
         | instead of relying on sys.settrace (which was perceived as slow
         | in general -- pydevd got by because it had a bunch of tricks to
         | just trace the needed contexts but implementing a fast debugger
         | in Python with it is pretty hard).
         | 
         | My initial results are still mixed -- i.e.: on some cases it's
         | definitely faster -- such as when tracking exceptions, but at
         | this point in all other scenarios it's still slower (it's
         | pending a few profiling sessions and I already have some ideas
         | on where to improve), but I'm still not sure it'll ever be as
         | fast as the version that can hook into the python frame eval
         | and change the bytecode for the function to add programatic
         | breakpoints... time will tell (but that approach is also very
         | hard to keep up to date on new python releases, so, I'll
         | probably end up deprecating it as I don't have enough
         | time/resources to keep it up to date).
         | 
         | Anyways, I have most tests already passing, but I have to do a
         | few profiling sessions before the initial release. I guess
         | there's no much point in saying: here's a new version of the
         | debugger using sys.monitoring -- does the same but is slower ;P
        
         | robertlagrant wrote:
         | Yeah - it's a bit opaque:
         | 
         | > Note free_tool_id() will not disable global or local events
         | associated with tool_id, nor will it unregister any callback
         | functions. This function is only intended to be used to notify
         | the VM that the particular tool_id is no longer in use.
         | 
         | Why won't it do those things? Why would I want to notify the VM
         | of this without unregistering?
        
       | jakirooo10 wrote:
       | sys.monitoring is a legendary feature for python Recently, I came
       | across a profiler based on sys.monitoring.
       | 
       | akarsu(https://github.com/furkanonder/akarsu) is new generation
       | profiler based on sys.monitoring. You must try guys!
        
       | woodruffw wrote:
       | Neat, I had no idea this had made it into 3.12. Looks like a
       | nice, modern alternative to `sys.settrace` and its ilk[1].
       | 
       | [1]: https://docs.python.org/3/library/sys.html#sys.settrace
        
       ___________________________________________________________________
       (page generated 2023-12-29 23:00 UTC)