[HN Gopher] Uncurled - running and maintaining Open Source proje...
       ___________________________________________________________________
        
       Uncurled - running and maintaining Open Source projects for three
       decades
        
       Author : adunk
       Score  : 92 points
       Date   : 2022-05-17 13:13 UTC (9 hours ago)
        
 (HTM) web link (un.curl.dev)
 (TXT) w3m dump (un.curl.dev)
        
       | marginalia_nu wrote:
       | So this is a bit tangential.
       | 
       | I've been considering open sourcing my search engine. Search is
       | like a fractal of interesting problems, and pretty much every
       | aspect of the search engine has known areas of improvement, so
       | I'm sure it would be a fun project to collaborate on.
       | 
       | I'm honestly a bit at a loss how to actually go about it, since
       | it's not an application or a library where others are expected to
       | run it, but a fairly bespoke piece of web service that requires
       | specific hardware configurations to do anything useful (as well
       | as extremely unwieldy datasets). The only rolemodel I can find is
       | something like Wikipedia.
       | 
       | I'm curious if anyone knows good "role models"?
        
         | beecafe wrote:
         | Sounds similar to ML research
        
         | tuxie_ wrote:
         | My first question would be: what's your goal? Why do you want
         | to open source it?
         | 
         | Because if the answer is "knowledge sharing" then just document
         | it and open it, that's it, whoever is interested will show up
         | (and if nobody does you don't care, your goal is fulfilled). If
         | instead your goal is to eventually build a community around it
         | then you'll have to put more effort (for example talk in
         | conferences).
         | 
         | You get my point, you will know which road to take as soon as
         | you know where you want to go.
         | 
         | My 2c.
        
         | moron4hire wrote:
         | You have to do a lot of advertising. Not buying ads on
         | Facebook, but talking about the project and trying to convince
         | people to join. Putting a project up on GitHub and then sitting
         | back and waiting for people to show up will result in exactly 0
         | people showing up. You'll need to identify where your target
         | developers are, currently, and go to them. It helps to get
         | interviewed in tech blogs, too.
        
           | marginalia_nu wrote:
           | I don't think finding developers is my first obstacle, I have
           | a bunch of people who have contacted me and shown interest in
           | participating.
           | 
           | The problem is the logistics of an open search project that
           | is a web service with serious hardware requirements. I think
           | the minimum hardware requirements is about 14 Gb of RAM,
           | that's without any real data loaded into the system. Testing
           | is awkward and cumbersome, and the data logistics are a real
           | headache even on the same network as the production instance.
           | 
           | To even run the search engine, you need a few hundred
           | megabytes of language models, as well as a probably few
           | gigabytes of website data to conduct meaningful testing. The
           | production instance has a disk footprint of about half a
           | terabyte.
        
             | closedloop129 wrote:
             | Have you asked that bunch of people how they would like to
             | cooperate?
             | 
             | My ignorant suggestion towards sharing those gigabytes of
             | data: have you considered bittorrents?
        
       ___________________________________________________________________
       (page generated 2022-05-17 23:01 UTC)