https://microsoft.github.io/verona/pyrona.html

View on GitHub

Project Verona

Research programming language for concurrent ownership

Home Publications FAQ Python

Fearless Concurrency for Python

As part of Project Verona, we have been developing a new ownership
model for Python, called Lungfish. This model is designed to provide
a safe and efficient way to manage memory and concurrency in Python
programs. This page explains our experiments to develop this
approach.

Plan for Python

Quick prototyping in FrankenScript

Modifying a production language is a complex task. As an initial
step, we have been developing a toy language called FrankenScript
that allows us to quickly prototype our ideas for region-based
ownership. FrankenScript is a small language that is designed to be
easy to modify and extend. It is based on the ideas of ownership and
concurrency that we are exploring in Project Verona, but where all
the checks are dynamic.

This prototype has given us confidence in the conceptual ideas behind
our ownership model, and has allowed us to explore the design space
of ownership in a dynamic language. It has also helped us explain our
ideas and the different design decisions to people outside the
Project Verona team.

FrankenScript generates a series of mermaid diagrams that show the
region structure of the program. An example of this is shown below:

mermaidexample

Engaging with the Python community

Over the last two years, we have been engaging with the Faster
CPython team at Microsoft as a sounding board for our ideas. In May
2025, we will be taking our first steps into the boarder Python
community at the Python Language Summit. The presentation that we
gave on the Python language summit can be found here. We are seeking
feedback from the core developers and broader community behind the
language.

Steps to a new ownership model

Building an ownership model for Python is a complex task, and we are
taking a step-by-step approach to ensure that we get it right.

The first step is actually to build a concept of deep immutability
into Python. This can be split into three parts:

  * Deep Immutability: We are starting with a deep immutability
    model, we have been drafting a PEP to describe this model. The
    current prototype is in a PR to a fork of CPython 3.12: https://
    github.com/mjp41/cpython/pull/51

  * Manage cyclic immutable garbage with reference counting and
    atomic reference counting of immutable objects. This will enable
    objects to be moved between sub-interpreters as they will no
    longer be managed by the interpreter local cycle collector. The
    current prototype is on the following branch: https://github.com/
    mjp41/cpython/tree/scc, which is based on top of the PR 51
    mentioned above.

  * Integration with message passing between sub-interpreters (PEP734
    ). This will enable us to get parallel performance even though we
    are technically not losing the GIL.

Deep Immutability is a key part of any ownership model as we need
concurrent threads to be able to share type information without
risking it being mutated under foot. As types in Python are mutable
objects, we need a way to share these objects between threads.

Once the immutability model is in place, then we can start applying
the region-based ownership model to Python to permit safe sharing and
transfer of mutable state. This involves applying the ideas developed
in the FrankenScript prototype to Python. We have made progress on
this in our prototyping ideas on CPython, but nothing is ready to try
yet.

FAQ

Why Python?

Python is the most popular programming language in the world. With
PEP703, aka NoGil or "Free-Threaded" Python, Python is moving to a
fully concurrent model. This means a vast quantity of programmers
will be potentially exposed to concurrency issues. Bringing an
ownership model to Python will help to make it easier to write
concurrent programs, and to avoid the pitfalls of concurrency.

This is a perfect opportunity to influence the future of Python, and
to help make programming experience better for everyone.

Why not threads and locks?

We are initially exploring using message passing between
sub-interpreters. However, we could also apply the region ideas to
locks, and provide a lock that gives temporary access to a region of
memory associated with that lock.

We have sketched out a design for how to integrate with the full PEP
703 work, but have not attempted to implement that yet.

Why not simply use Rust's ownership model?

Rust's ownership model is designed for a statically typed language,
and restricts the type of object graphs that can be used in the
language. There is so much existing code in Python that the
restrictions of Rust's ownership model would be too limiting. We are
designing a new ownership model based on regions that is designed to
work with Python's dynamic typing and existing object graphs.

Our approach draws heavily from the experience of languages with
ownership models like Rust, Cyclone Encore, and Pony, but is based
completely on dynamic checks. This completely alters the kind of
things that can be checked in comparison to a statically typed
approach.

What has Project Verona learnt from this work?

Performing ownership research in a dynamically typed language has
been a massive learning experience for the Project Verona team. It
has challenged every assumption we have made about ownership and
concurrency in a statically typed language. Moreover, it has shown us
certain programming patterns that are really easy to check in a
dynamic region system, but are really hard to check in a statically
typed language. We are currently re-evaluating the balance between
the static and dynamic checks in our ownership model, and how to best
combine them. Hopefully, we will be able to bring some of these ideas
back into the Project Verona research language.

Where can I find out more?

We have a detailed list of publications related to Project Verona on
our publications page. The most relevant paper to this work is
Dynamic Region Ownership for Concurrency Safety.

You can try our toy language, FrankenScript. We also provide a docker
container to test FrankenScript quickly.

We are also working on several forks of Python that implement our
ideas, these are currently on GitHub.

How can I get involved? Who can I talk to?

We have listed a few places where you can find parts of the project
that are available. If you want to find out more about those, raise
issues on the appropriate GitHub repo.

If it is more general, please raise an issue on the Project Verona
Github repo, or you can join our discussions list
lungfish@lists.uu.se.

Project Verona maintained by microsoft

Published with GitHub Pages