Retrieval Augmented Generation (RAG) with Free CPU-based LLMs from
Hugging Face
reddit / selfhosted

----------------------------------------------------------------------
Mammoth-Doughnut-160 · 2023-10-29 11:55

Hi everyone, we created a Hugging Face repo with small LLMs (\~3b and
under) that can be run without LLMs for RAG experimentation. Small
LLMs do not perform well without fine-tuning for RAG use cases, so we
fine tuned them for optimized RAG performance while at the same time
not requiring expensive GPUs.
[https://huggingface.co/llmware](https://huggingface.co/llmware)

We also have an open source library for enterprise-grade RAG
implementation that has text parsing, chunking, embedding, and other
great features like source citation and fact checking that also
integrates into LLMs on Hugging Face. [https://github.com/llmware-
ai/llmware](https://github.com/llmware-ai/llmware)

Here are some helpful YT links to get started: [https://www.youtube.co
m/watch?v=JjgqOZ2v5oU](https://www.youtube.com/watch?v=JjgqOZ2v5oU)
and [https://www.youtube.com/watch?v=0naqpH93eEU&t=3s](https://www.you
tube.com/watch?v=0naqpH93eEU&t=3s)

----------------------------------------------------------------------
mcr1974 · 2023-10-30 00:21

remindme! 2 days

----------------------------------------------------------------------
mcr1974 · 2023-10-30 00:22

How does this compare to llamaindex / langchain.

----------------------------------------------------------------------
RemindMeBot · 2023-10-30 00:22

I will be messaging you in 2 days on [**2023-11-01 00:21:53 UTC**](htt
p://www.wolframalpha.com/input/?i=2023-11-
01%2000:21:53%20UTC%20To%20Local%20Time) to remind you of [**this link
**](https://www.reddit.com/r/selfhosted/comments/17j1d6e/retrieval_aug
mented_generation_rag_with_free/k70yz37/?context=3)

[**CLICK THIS LINK**](https://www.reddit.com/message/compose/?to=Remin
dMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2F
selfhosted%2Fcomments%2F17j1d6e%2Fretrieval_augmented_generation_rag_w
ith_free%2Fk70yz37%2F%5D%0A%0ARemindMe%21%202023-11-
01%2000%3A21%3A53%20UTC) to send a PM to also be reminded and to
reduce spam.

^(Parent commenter can ) [^(delete this message to hide from others.)]
(https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete
%20Comment&message=Delete%21%2017j1d6e)

*****

|[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remind
mebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?t
o=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside
%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Yo
ur Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&
subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](ht
tps://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot
%20Feedback)|
|-|-|-|-|

----------------------------------------------------------------------
Dramatic_Bluebird355 · 2023-10-30 00:24

The 2 biggest differences are that it is very easy to get started and
to start using right out of the box and it is enterprise grade (was
closed source and is used in paid SaaS site for legal research for
attorneys).