Subj : Re: Mystic 2.0
To   : g00r00
From : Ryan Fantus
Date : Mon Sep 28 2020 09:17 pm

 g0> I don't think there are too many benefits considering the effort it
 g0> would take to switch over to it.  The TLDR version is that if I were
 g0> starting from scratch today I think I'd use SQLite3, but I am not sure
 g0> if the effort required to switch over to it justifies the gains at this
 g0> point.

This is basically where I'm at with Daydream at the moment. On the one hand,
the data files could really use an overhaul, but on the other hand...why? :)

 g0> As far as pros, it would simplify the code once everything is working
 g0> since all of the file locking and disk writing would be left up to
 g0> SQLITE.

Right - and making ontological changes or adding rows here and there would be
pretty straightforward, without some gross migration binary each release.

That shit scares me. Hehe.

 g0> It would make adding new columns into a database schema much easier
 g0> since SQL does all the legwork for you and when I do it with direct
 g0> data, I have to rebuild the file manually.

Yeah, what you said. :P

 g0> It would provide better access to the data outside of Mystic for writing
 g0> utilities.  This is both a pro and a con because its easier for people to
 g0> experiment and mess up their data, but having SQL would be better for
 g0> people to whip up utility functions in whatever script language they
 g0> like.

As Daydream is built in c and opensource, some people have forked it to do
custom things. One such thing was to use as a frontend for glftpd (the
racer/trader communities were into this). The implementation was pretty
screwy and destroyed a lot of existing bbses. User accounts were all hosed
because people didn't pay attention to data structures. Such a shame.

 g0> It'd also make it easier for on-the-fly sorting of data for things like
 g0> message listings and file listings, since Mystic today currently relies
 g0> on sorting that data on disk.  All of the internal paginating could
 g0> probably be moved to SQL although that I've read is another thing that
 g0> SQLite isn't amazing at doing.

I have multiple nets with upwards of 5k messages per base per net. I haven't
hit a perf limit yet as far as I can tell.

 g0> Memory usage would probably increase significantly.  Mystic is very
 g0> lightweight when it comes to memory used when reading data, and SQLITE
 g0> would require much more.  I don't know the amount it would use offhand
 g0> but I do know that if you deprive it of memory it will sometimes slow it
 g0> to a crawl.  On the flip, Mystic today doesn't use more than maybe 8-20
 g0> kilobytes when reading data files.  If there were scaled across a decent
 g0> number of nodes the difference might be pretty massive.

Memory is damn near free these days :) My VPS has 8GB RAM and idles around
400MB. I love efficiency and optimization but also love utility, hehe.

....and justifying an expense here and there..

 g0> SQLite support would probably be best left for a Mystic 2 spin off given
 g0> the amount of work it'd probably take to rewrite all the message base and
 g0> FidoNet/QWK stuff.  It seems like it'd be easier to just build something
 g0> from scratch and that would also allow me to do things like change the
 g0> directory structure, MCI code system, etc, without having to build an
 g0> upgrade path for the data files.

Did The Millionaire fire up your Mystic 2 spirit?!

--- Mystic BBS v1.12 A46 2020/08/06 (Linux/64)
 * Origin: monterey bbs (1:218/820)

.