[HN Gopher] Replacing Elasticsearch with Rust and SQLite (2017)
       ___________________________________________________________________
        
       Replacing Elasticsearch with Rust and SQLite (2017)
        
       Author : nethunters
       Score  : 81 points
       Date   : 2021-05-16 17:24 UTC (5 hours ago)
        
 (HTM) web link (nickb.dev)
 (TXT) w3m dump (nickb.dev)
        
       | lsb wrote:
       | I wonder how effective this would be with the SQLite with a new
       | virtual file system fetching pages via XHR that someone posted
       | last month. Then you would be able to do full text search on a
       | huge database only sending a few KB per search
       | 
       | https://news.ycombinator.com/item?id=27016630
        
         | [deleted]
        
       | polyrand wrote:
       | I think replacing Elasticsearch with SQLite is a great idea. Even
       | more if you use the full-text search functions that SQLite
       | includes.
       | 
       | I recommend checking out scout[0], which, I think, can be a good
       | replacement for Elasticsearch in some cases. I'm also working on
       | an Elasticsearch replacement built on top of SQLite for my
       | litements[1] project, but it will still take a few weeks to have
       | a working version.
       | 
       | [0] https://github.com/coleifer/scout [1]
       | https://github.com/litements/
        
         | fabbari wrote:
         | It definitely depends on what you're doing with it. In the
         | article the author is using it to parse only nginx logs to
         | report page hits counts - I probably would have not gone with
         | Elasticsearch to begin with. I find it strange that the issue
         | with InfluxDB was the lack of 'Group by'; you can use
         | 'Distinct' with other aggregation functions with the same
         | result - and without having to write and maintain your own
         | stack.
         | 
         | I appreciate - and often indulge - in writing stacks of
         | software that are better implemented other open source, but I
         | wouldn't say that the article describes a solution that is
         | viable as a replacement for Elasticsearch.
        
           | nethunters wrote:
           | I think it's best put as a replacement for ElasticSearch for
           | those who don't really need ElasticSearch.
           | 
           | Edit - i.e. where ES is overkill for the use case, however
           | for an appropriate use case ElasticSearch is amazing
        
             | xbar wrote:
             | I didn't read the article, but I think this is a sizable
             | audience.
        
         | Anonymous4272 wrote:
         | Similar to litements litecache is DiskCache[0]. It is a pure
         | python Redis/Memcached alternative which is faster than them
         | for reads. Under the hood it uses shared Sqlite.
         | 
         | [0] https://github.com/grantjenks/python-diskcache
        
           | polyrand wrote:
           | Yes, I'm aware of DiskCache. Wonderful library and also very
           | useful, I enjoyed reading the source code too. I have taken a
           | few ideas from it that I would like to implement in the
           | future.
        
       | RcouF1uZ4gsC wrote:
       | > This query can't be represented in Diesel's DSL (a sample of
       | the DSL is demonstrated in Diesel's getting started).
       | 
       | I feel like most of the time, SQL DSLs end up being bad. SQL is
       | already a very high level language. There are a ton of examples
       | of how to do stuff with SQL queries. DSLs, even if they have a
       | way to express the SQL, are not nearly as widespread as SQL and
       | then you will have to spend time figuring out how to translate
       | the SQL into the DSL.
       | 
       | In addition, with libraries like sqlx for Rust, you can also get
       | the type safety of a DSL using regular SQL.
       | 
       | I would say that as a developer, the time spent getting familiar
       | with SQL is a good investment that will likely pay off across
       | many projects and programming languages.
        
       | pvillano wrote:
       | also if you can, set up your logging to output JSON/GELF.
        
       | leetrout wrote:
       | I'm always surprised when people haven't heard of Xapian which is
       | also in this space.
       | 
       | I tell people Xapian is the SQLite of search.
       | 
       | https://xapian.org/
        
       ___________________________________________________________________
       (page generated 2021-05-16 23:00 UTC)