https://github.com/potamides/DeTikZify Skip to content Navigation Menu Toggle navigation Sign in * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + GitHub Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Enterprise + Enterprise platform AI-powered developer platform Available add-ons + Advanced Security Enterprise-grade security features + GitHub Copilot Enterprise-grade AI features + Premium Support Enterprise-grade 24/7 support * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} potamides / DeTikZify Public * Notifications You must be signed in to change notification settings * Fork 5 * Star 233 * Synthesizing Graphics Programs for Scientific Figures and Sketches with TikZ License Apache-2.0 license 233 stars 5 forks Branches Tags Activity Star Notifications You must be signed in to change notification settings * Code * Issues 2 * Pull requests 0 * Actions * Projects 0 * Security * Insights Additional navigation options * Code * Issues * Pull requests * Actions * Projects * Security * Insights potamides/DeTikZify This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main BranchesTags Go to file Code Folders and files Name Name Last commit message Last commit date Latest commit History 74 Commits detikzify detikzify examples examples .gitattributes .gitattributes .gitignore .gitignore LICENSE LICENSE MANIFEST.in MANIFEST.in README.md README.md pyproject.toml pyproject.toml View all files Repository files navigation * README * Apache-2.0 license DeTikZify [^Synthesizing Graphics Programs for Scientific Figures and Sketches with TikZ] arXiv Hugging Face Colab Creating high-quality scientific figures can be time-consuming and challenging, even though sketching ideas on paper is relatively easy. Furthermore, recreating existing figures that are not stored in formats preserving semantic information is equally complex. To tackle this problem, we introduce DeTikZify, a novel multimodal language model that automatically synthesizes scientific figures as semantics-preserving TikZ graphics programs based on sketches and existing figures. We also introduce an MCTS-based inference algorithm that enables DeTikZify to iteratively refine its outputs without the need for additional training. Showcase.mp4 Installation Tip If you encounter difficulties with installation and inference on your own hardware, consider visiting our Hugging Face Space (if the space is restarting the first run might take 10-15 minutes to download and load the model). Should you experience long queues, you have the option to duplicate it with a paid private GPU runtime for a more seamless experience. Additionally, you can try our demo on Google Colab. However, setting up the environment there might take some time, and the free tier only supports inference for the 1b models. Do not forget to read our usage tips! The Python package of DeTikZify can be easily installed using pip: pip install 'detikzify @ git+https://github.com/potamides/DeTikZify' Or, if your goal is to run the included examples, clone the repository and install it in editable mode like this: git clone https://github.com/potamides/DeTikZify pip install -e DeTikZify In addition, DeTikZify requires a full TeX Live 2023 installation, ghostscript, and, poppler which you have to install through your package manager or via other means. Usage If all required dependencies are installed, the full range of DeTik Zify features such as compiling, rendering, and saving TikZ graphics, and MCTS-based inference can be accessed through its programming interface: from operator import itemgetter from detikzify.model import load from detikzify.infer import DetikzifyPipeline import torch image = "https://w.wiki/A7Cc" pipeline = DetikzifyPipeline(*load( base_model="nllg/detikzify-ds-7b", device_map="auto", torch_dtype=torch.bfloat16, )) # generate a single TikZ program fig = pipeline.sample(image=image) # if it compiles, rasterize it and show it if fig.is_rasterizable: fig.rasterize().show() # run MCTS for 10 minutes and generate multiple TikZ programs figs = set() for score, fig in pipeline.simulate(image=image, timeout=600): figs.add((score, fig)) # save the best TikZ program best = sorted(figs, key=itemgetter(0))[-1][1] best.save("fig.tex") For interactive use and additional usage tips, we recommend checking out our web UI, which can be started from the command line (use --help for a list of all options): python -m detikzify.webui --light More involved examples, for example for evaluation and training, can be found in the examples folder. Model Weights & Datasets We upload all our models and datasets to the Hugging Face Hub. However, please note that for the public release of the DaTikZ[v2] dataset, we had to remove a considerable portion of TikZ drawings originating from arXiv, as the arXiv non-exclusive license does not permit redistribution. We do, however, release our dataset creation scripts and encourage anyone to recreate the full version of DaTikZ [v2] themselves. Acknowledgments The implementation of the model architecture is largely based on LLaVA. Our MCTS implementation takes heavy inspiration from VerMCTS. About Synthesizing Graphics Programs for Scientific Figures and Sketches with TikZ Resources Readme License Apache-2.0 license Activity Stars 233 stars Watchers 4 watching Forks 5 forks Report repository Releases No releases published Packages 0 No packages published Languages * Python 100.0% Footer (c) 2024 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact * Manage cookies * Do not share my personal information You can't perform that action at this time.