Post B1DFqZcT4zcdBibLe4 by alecksgates@podcastindex.social
 (DIR) More posts by alecksgates@podcastindex.social
 (DIR) Post #B1BA4hkWf3gyPpo0Ia by dave@podcastindex.social
       2025-12-12T17:19:50Z
       
       0 likes, 0 repeats
       
       
       
 (DIR) Post #B1BwAKmlkTeqjRbZR2 by alecksgates@podcastindex.social
       2025-12-13T02:18:39Z
       
       0 likes, 0 repeats
       
       @dave hmm shouldn't there be a parse queue in front of the parser?Also how much of a bottleneck is it to generate SQL statements?  In my experience, if we ingest JSON strings as serde structs, it's pretty trivial to use those structs directly in an sqlx insert.
       
 (DIR) Post #B1Cy4Nmc5LgLMDmX2m by dave@podcastindex.social
       2025-12-13T14:14:44Z
       
       0 likes, 0 repeats
       
       @alecksgates 10-4.  Will adjust.
       
 (DIR) Post #B1DEo0oRQ5gehePt3I by dave@podcastindex.social
       2025-12-13T17:22:15Z
       
       0 likes, 0 repeats
       
       @alecksgates Looking at this:  https://github.com/skyaktech/tlq
       
 (DIR) Post #B1DFqZcT4zcdBibLe4 by alecksgates@podcastindex.social
       2025-12-13T17:33:56Z
       
       0 likes, 0 repeats
       
       @dave Interesting.  Do you want persistence?During a planned maintenance window you could just turn off the ingest and wait for jobs to finish and the queue to flush, but that's sort of annoying.In an unexpected restart you would lose queue metadata and have to rebuild from object storage on startup.I feel like redis is the path of least resistance here because of the persistence options and it being very well documented.(Or postgresql unless you surpass a certain # of items per second)
       
 (DIR) Post #B1DFuIeE7bHji4CVxQ by alecksgates@podcastindex.social
       2025-12-13T17:34:36Z
       
       0 likes, 0 repeats
       
       @dave also it's funny how I'm also writing my own rust message queue system 😂