[HN Gopher] Show HN: OpenCopilot - Build and embed open-source A...
___________________________________________________________________
Show HN: OpenCopilot - Build and embed open-source AI copilots into
your product
Hey HN OpenCopilot is an OSS framework which helps devs to build
open-source AI Copilots that actually work and embed into their
product with ease. Why another LLM framework? Twitter is full of
impressive LLM applications but once you peel off the curtains it's
clear that they are just demos. The reason being because building
an AI Copilot that goes beyond a Twitter demo can be complex, time-
consuming and unreliable. Our team has been in the AI space since
2018 and built numerous LLM apps & copilots. While doing that, we
got approached by many startups saying they'd also like to build a
copilot for their product but they haven't been able to get it
reliable, fast or cost-effective enough for production use. Thus we
built OpenCopilot framework, so devs can intuitively get AI
Copilots running in less than 10 minutes and iterate towards a
useful Copilot in a single day. We believe every product, company
and individual will have their Copilot in the future. Thus, we'd
love your feedback, questions and constructive criticism.
Author : JohannesTk
Score : 93 points
Date : 2023-08-22 15:03 UTC (7 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| bko wrote:
| I'm confused as to what this is? Is it just a wrapper to gpt API
| that parses the response? And it looks like it manages the convo
| so you don't have to send over entire history?
|
| I have a somewhat related project that wraps the gpt API and
| let's you interact in a codebase, where the LLM can request
| access to certain files, propose new files and edits
|
| https://github.com/breeko/j-dev
| nonethewiser wrote:
| Sounds a lot like aider
| squeaky-clean wrote:
| I wouldn't really consider a wrapper around the OpenAI API to be
| an open source copilot. Am I misunderstanding this project?
| bofadeez wrote:
| You understand it fully
| geek_at wrote:
| True, GitHub copilot is also using gpt3.5 at the moment so it's
| actually kind of the same
| JohannesTk wrote:
| You do have a valid point that the LLM is closed aka OpenAI's
| API. We were thorn about it ourselves but decided to release
| with OpenAI because of it's ease of use for devs. We have
| already tested Llama-2 and seeing more and more pull for it,
| especially by enterprise who have can't use OpenAI because of
| privacy and IP issues. Thus, OSS LLMs are next on our roadmap
| :)
|
| Rest of the stack is fully open-source and it's much more than
| a wrapper. Maybe the impression comes from the aspect that
| we've made getting started dead simple with the minimal Copilot
| example. The framework covers a lot of ground so you can build
| your copilot easily: backend, adding a knowledge base & dynamic
| data, front-end, monitoring & evaluation.
|
| This is pretty much v0.1 release. What else would you love to
| see?
| skeptrune wrote:
| What is the point of this if no RAG?
| sq42na wrote:
| RAG comes out of the box - see here:
| https://docs.opencopilot.dev/improve/knowledge-base
|
| We were hoping to make dynamic RAG part of the very first
| release but didn't - we'll be adding that soon.
| boredemployee wrote:
| The amount of cynical comments everywhere always get me.
|
| Anyway, congrats, it will be really useful for building POCs and
| mvps quickly. I'll definetely give it a try and feedback once I
| test it!
| decodebytes wrote:
| Which LLM are you using?
| MourningWood wrote:
| Looks like gpt4
| sq42na wrote:
| Yep, currently gpt-3.5-16k or gpt-4. We wrote the example
| prompts in a relatively Llama-compatible way though (we
| actually started building this onto Llama 1 before switching to
| OpenAI as default), and make few assumptions about the LLM so
| it's easy to switch out. Mostly this is waiting behind us
| adding an option to pass in any LLM, and we're planning to add
| support for this.
|
| Generally, we leave the LLM up to the user -- if OpenAI or
| Google is a no-go, then you probably are anyway in the
| territory of self-hosting or even self-training your LLM, which
| means you're fine setting up your own inference endpoints as
| well.
| michaelmior wrote:
| For the time being, it looks like you can choose between
| gpt-3.5-turbo-16k and gpt-4.
|
| https://github.com/opencopilotdev/opencopilot/blob/d7aa8270d...
| warthog wrote:
| Any plans to add Langchain support? Another thing for OSS LLMs
| could be adding support for the like of Replicate or Baseten as
| well.
| swyx wrote:
| > Our team has been in the AI space since 2018 and built numerous
| LLM apps & copilots.
|
| could you please elaborate?
| sq42na wrote:
| Sure! Taivo here - CTO & co-founder of OpenCopilot.
|
| The most recent LLM app we've built has been the Ready Player
| Me copilot (which you can see linked in the main repo as well,
| and which was an inspiration for actually making something
| open-source based on what we had learned into). The problem it
| solves is onboarding developers onto their platform, and is
| currently live for a subset of their users.
|
| We've built a ton of copilots that we haven't published. Some
| have been for work like founder copilots for ourselves and a
| few friends, sales outreach copilots, etc. Some have been for
| fun: I wanted to see if I could create a copilot that coaches
| me to do things faster a la https://patrickcollison.com/fast -
| so I did. As a PoC I created a copilot that explains the rules
| of an obscure sport I like (floorball).
|
| In addition, with the same team, since the beginning of this
| year we've made a bunch of experiments that I'm sure many
| others have tried: * "chat with your knowledge base" style
| bots: for tech support, for developer docs, for internal HR
| use, and probably some I can't remember * an LLM-based quiz app
| that dynamically makes a quiz based on your Spotify history *
| an automated engineer (a much more basic version of smol-
| ai/developer) * a website builder where you mostly uses a chat
| UI for making the website * making AutoGPT better (higher
| quality, faster, more straightforward to use) * we also briefly
| considered making an OSS LLM before Llama 1 came out and
| kickstarted the OSS LLM wild ride we've seen since then
|
| We killed almost all of these experiments within a few weeks
| after release, because none found early traction; we'd probably
| been wrong about the problem these were solving.
|
| About my AI background: I built a 25-person computer vision
| team from zero at an identity verification unicorn. After that
| tried and failed to build a computer vision labelling tool
| startup. Before that, built robot perception software at
| Starship robots and unsuccessfully tried to make a contribution
| to AI safety with my thesis on active reinforcement learning in
| 2017.
|
| About Johannes (OP) & the rest of the team: in 2018 him and
| part of the current team tried to make Sidekik: something like
| character.ai, but training models from scratch - which
| eventually didn't work out. From 2020 (after a pivot) the same
| team was building Sentinel, a deepfake-detection startup -
| which again did not work out.
|
| FYI: Sidekik's original website:
| https://web.archive.org/web/20181008082142/https://sidekik.a...
| JohannesTk wrote:
| Yeah, we started Sidekik in 2018 where we were enabling
| people to create AI versions of themselves, including famous
| people such as Steve Jobs, Elon Musk, etc: http://web.archive
| .org/web/20191211212908/https://sidekik.ai...
|
| It was way before all of it was cool and back then we didn't
| train LLMs but just LMs with LSTMs lol. Got to around 10K
| users but the technology was too early so we pivoted to
| Sentinel: https://thesentinel.ai/ Scaled that up to the
| largest deepfake detection provider in the world but
| eventually it didn't work out which is a longer story.
| warthog wrote:
| Here is a bigger question: In your background explanations you
| mention AI projects and apps but seems like you guys are CEO
| and CTO of NFTPort. Any reason as to why you are not disclosing
| that?
| zelphirkalt wrote:
| I would still like to have a model, open source at minimum,
| better yet with AGPL or so, that I train locally on my own code
| and also use locally exclusively on my own machine and that does
| not require me to have a Nvidia GPU and can simply be trained on
| CPU.
|
| Does this exist?
| JohannesTk wrote:
| OSS Llama2 based LLMs upcoming which can be also run on a CPU
| so you have full privacy (inference, not train).
|
| Why would you like to train it on your own code?
| corytheboyd wrote:
| But that's not snake oil. Seriously though, these are also my
| table stakes requirements for trying any of this stuff. I'm
| fine waiting a couple years for it, and I'm also fine if it
| never happens. I just want to know if it does :)
| steno132 wrote:
| There are significant safety risks with this approach.
|
| Current foundation AI models have safeguards in place to avoid
| generating content that could be used for disinformation,
| violence, and terrorism.
|
| While not perfect, these safeguards have meant that we've been
| able to use ChatGPT over the last couple months, without a
| major backlash due to the technology being used for destructive
| purposes and it potentially being closed (as has happened many
| times before).
| jazzyjackson wrote:
| Upvoted for at least being honest that gaurdrails are for
| brand safety, ie OpenAI not being embarrased by their bots
| output and having to pull the plug.
| Xeoncross wrote:
| Considering governments murdered literally a million times
| more people than civilizations over the past hundred years,
| it's ironic they alone get the models without safeguards in
| place.
| steno132 wrote:
| The fundamental disconnect is that we as technologists
| believe we are entitled to technology. That technology
| should be open. And, for the most part, it is.
|
| However, if you look at the most sensitive technologies,
| whether it's nuclear weapons or until recently
| cryptography, they're never open. The risk of them falling
| into the wrong hands is too high.
|
| AI seems to us like any other software, as people who may
| understand it and its limitations. However, it's also a
| tool for unfathomable destruction -- not now, but in the
| coming years.
|
| And so restrictions are almost certain to happen, and
| according to many, justified. You're not going to have
| access to "un-censored AI" in the coming years. That's
| unfortunately reality.
| Xeoncross wrote:
| That is the thing though, those who historically pose the
| greatest risk to mankind (the heads of governments) do
| and will continue to be the ones with the greatest access
| and use of the most dangerous technology.
|
| To put it another way, we're all concerned (at least in
| 1st would countries) about terrorists and hackers being
| the greatest threat to people's lives. So we're
| comfortable giving the government sole control of weapons
| - even though statistically that's not an accurate
| representation of the danger graph.
|
| I don't know if it's good marketing or propaganda, but I
| find it ironic that "only government can be trusted".
| neilv wrote:
| What is the relationship to: OpenCopilot - Open source AI copilot
| for your own SaaS product (github.com/openchatai) | 127 points by
| gharbat 1 day ago | 31 comments |
| https://news.ycombinator.com/item?id=37203196
| JohannesTk wrote:
| None, we've been building OpenCopilot for the last few months
| and it seems they beat us with the launch by a few days. I do
| hope they do well cus we believe a world where copilots can be
| built with open-source is a much better one vs built on a
| closed MS, etc. stack.
| CharlesW wrote:
| Related question: Why would you (and they) give your product
| a Microsoft name and open yourself up to a trademark fight
| and probable name change when, with just a little effort and
| imagination (and maybe ChatGPT), you could've called it
| anything else?
| willsmith72 wrote:
| Why a "Microsoft name"? Are you talking about GitHub
| copilot? Because that's far from the first "copilot"
| product
___________________________________________________________________
(page generated 2023-08-22 23:01 UTC)