[HN Gopher] Show HN: Synthesize TikZ Graphics Programs for Scien...
       ___________________________________________________________________
        
       Show HN: Synthesize TikZ Graphics Programs for Scientific Figures
       and Sketches
        
       Creating high-quality scientific figures can be time-consuming and
       challenging, even though sketching ideas on paper is relatively
       easy. Furthermore, recreating existing figures that are not stored
       in formats preserving semantic information is equally complex. To
       tackle this problem, we introduce DeTikZify, a novel multimodal
       language model that automatically synthesizes scientific figures as
       semantics-preserving TikZ graphics programs based on sketches and
       existing figures. We also introduce a Monte Carlo Tree Search-based
       inference algorithm that enables DeTikZify to iteratively refine
       its outputs without the need for additional training.
        
       Author : potamides
       Score  : 82 points
       Date   : 2024-06-06 12:42 UTC (10 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | nequo wrote:
       | This is brilliant work, OP! A great use case for machine
       | learning.
       | 
       | Why does the arXiv non-exclusive license preclude the inclusion
       | of arXiv figures in the published data set? I don't see how the
       | conditions set by the license imply that:                   I
       | grant arXiv.org a perpetual, non-exclusive license to distribute
       | this article.         I certify that I have the right to grant
       | this license.         I understand that submissions cannot be
       | completely removed once accepted.         I understand that
       | arXiv.org reserves the right to reclassify or reject any
       | submission.
       | 
       | https://arxiv.org/licenses/nonexclusive-distrib/1.0/license....
        
         | wrs wrote:
         | Uploading to arXiv gives distribution rights to arXiv, not the
         | _users_ of arXiv.
        
       | tmaly wrote:
       | I am a little confused by the use cases.
       | 
       | Would someone feed the model a hand drawn scientific figure and
       | it would output the TeX to create the figure digitally?
        
         | potamides wrote:
         | The use cases we see are (i) assisting researchers in creating
         | scientific figures from scratch (sketching them om paper is
         | usually easy but actually developing them can be daunting), and
         | (ii) enabling semantic edits to existing figures stored in
         | lower-level formats by synthesizing high-level graphics
         | programs that generate them.
        
         | dimatura wrote:
         | Yeah, I think that's one of the primary use cases. As a
         | researcher I've looked at using Tikz in the past but it's a
         | nontrivial learning curve. Something like this seems useful to
         | at least get a starting point to tweak.
        
         | Xerox9213 wrote:
         | As a teacher who regularly rips off large expressions from
         | textbooks and test generators with MathPix, this would allow me
         | to also capture diagrams into my exercises.
        
       | wrs wrote:
       | Shout out to an open source model that includes the dataset
       | crawling and training code, not just the weights and inference.
        
       ___________________________________________________________________
       (page generated 2024-06-06 23:01 UTC)