https://dlang.org/blog/2024/02/22/dmd-compiler-as-a-library-a-call-to-arms/ The D Blog The official blog for the D Programming Language. Menu Skip to content * D Home * D Forums * Donate * Shop DMD Compiler as a Library: A Call to Arms Leave a reply Digital Mars D logo Having a flexible and powerful compiler library has been one of the stated goals of the D Language Foundation for some time now. This makes sense, as a proper compiler library will channel the efforts of contributors into building developer tools, which in turn, will increase the adoption rate of the language. However, progress on this topic has been slow, mainly due to two aspects: (1) the lack of a clear direction, and (2) the intimidating complexity of the DMD frontend, which requires significant work on the compiler codebase. The good news is that we now have a plan, which I will outline in this blog post. The bad news is that implementing this plan requires significant effort, and we need more contributors. However, the silver lining is that the work, while extensive, mostly involves refactoring the code. This provides an excellent opportunity for contributors to familiarize themselves with the compiler codebase while delivering real value. Before delving into the specifics, let me give you some background. Current Status And How We Got Here To fully understand the work done so far on the compiler-as-a-library project, I highly recommend watching my talk on this subject. In summary: * Several years ago, we began packaging the compiler as a library. * Our goal was to clearly separate compilation phases: lexing, parsing, semantic analysis, optimizations, and code generation. * The parsing and semantic analysis modules were interdependent, necessitating a method for separation. * We opted to template the parser with an ASTFamily template parameter, defining the AST nodes required for parsing. * We created ASTBase (containing AST nodes essential for parsing) and ASTCodegen (containing AST nodes needed for code generation). * ASTBase, as it stands, is code duplicated from ASTCodegen. * We started extracting semantic routines and fields from AST nodes to eliminate ASTBase's code duplication by importing a subset of modules used by ASTCodegen. * Additionally, we began replacing third-party libraries (like libdparse) with the DMD-as-a-library package. For more detailed information on each of these points, I recommend watching the talk I referenced. Recently, I proposed to Walter a modification to the codebase that would significantly enhance the flexibility of the compiler library, allowing any AST node to be overwritten. Walter was hesitant to accept my proposal, concerned about the potential "ugliness" it would introduce to the codebase. He cited the addition of ASTBase and the resulting code duplication as a precedent. He then suggested that if we eliminate ASTBase, he would reconsider my proposal. What You Can Do To Help We are now focused on eliminating the duplication in ASTBase. To achieve this, we need to extract all information related to semantic analysis from the existing AST nodes. The challenge is the sheer number of AST nodes and the multitude of functions associated with each. I have been working on this sporadically over the past few months, and progress is slow due to the nature of the work: it mostly involves moving code, creating visitors, breaking dependencies, etc. While not overly complex, it isn't particularly creative work either. However, for someone interested in understanding a real-life compiler codebase, it's an ideal starting point. If you're willing to support this initiative, I've put together a guide on where to start and what you can do. Feel free to contact me on Slack (razvan.nitu), Discord, or email (razvan.nitu1305@gmail.com) for more details or to request a review of your PR. I see this as an excellent opportunity to onboard new people into compiler development in a way that benefits both the language and the contributor. So, if you have some spare time, please join us in getting this work done! This entry was posted in Community, Compilers & Tools, Core Team, D Foundation on February 22, 2024 by Razvan Nitu. Post navigation - Crafting Self-Evident Code with D Leave a Reply Cancel reply Your email address will not be published. Required fields are marked * [ ] [ ] [ ] [ ] [ ] [ ] [ ] Comment * [ ] Name * [ ] Email * [ ] Website [ ] [Post Comment] [ ] [ ] [ ] [ ] [ ] [ ] [ ] D[ ] This site uses Akismet to reduce spam. Learn how your comment data is processed. Search [ ]Search PAGES * D and C * D in Production * Symmetry Autumn of Code * The GC Series [beerconf-t] Support the D Language Foundation by shopping at the DLang Swag Emporium! [donatenew] Support the D Language Foundation with a direct donation! Proudly powered by WordPress We are using cookies to give you the best experience on our website. You can find out more about which cookies we are using or switch them off in settings. Accept Close GDPR Cookie Settings The D Blog * Privacy Overview * Strictly Necessary Cookies Powered by GDPR Cookie Compliance Privacy Overview This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful. Strictly Necessary Cookies Strictly Necessary Cookie should be enabled at all times so that we can save your preferences for cookie settings. Enable or Disable Cookies [ ] If you disable this cookie, we will not be able to save your preferences. This means that every time you visit this website you will need to enable or disable cookies again. Enable All Save Settings