Self-Hosted Journal with AI?
reddit / selfhosted

----------------------------------------------------------------------
draxter815 · 2023-10-26 01:08

Hey, this is probably a long shot (or an idea for a new app).

Basically I'm looking or a journal app that I could self-host that has
a feature where an AI chatbot "counselor/therapist" then comments on
your journal entries. I thought it would be amazing, as journaling can
be very lonely. I have seen smartphone apps that do something similar,
like Mindsera. But something self-hosted would be even better.

----------------------------------------------------------------------
Aud3o · 2023-10-26 01:27

Sure would. But anyone that could create something like this and has
half a brain is gonna charge money for it.
Creating and maintaining something like what you’re describing is no
easy feat.

----------------------------------------------------------------------
joseflopez · 2023-10-26 01:27

Yes! I would love this one too. I’m currently doing my journal with
Bookstack.

----------------------------------------------------------------------
draxter815 · 2023-10-26 01:33

It could allow users to put their ChatGPT API key, and use that as an
external engine to drive the AI feature. I know, I know, it's a little
bit against self-hosting philosophy, but a reasonable compromise in
this case.

----------------------------------------------------------------------
Mandus_Therion · 2023-10-26 03:51

You would need dGPU with minimum 12 GB vram to run a highly compressed
3B model in-house.

Also, your Ai model wont learn your new input, it will just apply RAG
on the documents you created but even then it won't be good enough
because the model is small.

You could use openai API to get handcrafted comments with specialized
prompts but it will be expensive

----------------------------------------------------------------------
Adam_Meshnet · 2023-10-26 08:49

Contrary to the other comments - you don't need a dGPU and putting
together a PoC for this wouldn't be that hard.

You need two three things:

1. An LLM, you can run it on your server, preferably one wrapped with
an API (I'm running mine on a Ryzen 5 5600G CPU)
2. Either a journaling app with an API (no idea if something like this
exists, I would be surprised if it did) or an app that will send your
journal entries via a POST request to an LLM API and save the
insights. You can even keep it in a conversation, so the LLM would
possibly reference other entries.
3. Journal entries

You can put something like this in a couple of hours if you've got a
fairly basic understanding of  APIs, and HTTP methods as well as a
basic level of a programming language under your belt.

----------------------------------------------------------------------
Aud3o · 2023-10-26 21:29

Absolutely not a reasonable compromise, you'd be training someone
else's AI with your secrets and personal life, no thanks.

And honestly, you'd want a emotionless robot to be your
counselor/therapist? That's a fast road to desperation.

----------------------------------------------------------------------
draxter815 · 2023-10-26 22:43

Fair point about privacy, though there are self-hosted models
available, they do require a decent hardware with a GPU to run at
decent speed. So that's another option.


And, ha. Don't spoil my illusion of AI :D