[HN Gopher] Stable Code 3B: Coding on the Edge
___________________________________________________________________
Stable Code 3B: Coding on the Edge
Author : egnehots
Score : 67 points
Date : 2024-01-16 21:40 UTC (1 hours ago)
(HTM) web link (stability.ai)
(TXT) w3m dump (stability.ai)
| keyle wrote:
| That is fantastic. I'm building a small macOS SwiftUI client with
| llama cpp built in, no server-client model, and it's already so
| useful with models like openhermes chat 7B, and fast.
|
| If this opens it to smaller laptops, wow!
|
| We truly live in crazy time. The rate of improvement in this
| field is off the walls.
| turnsout wrote:
| That sounds awesome! Can you share any details about how you're
| working with llama cpp? Is it just via the Swift <> C bridge?
| I've toyed with the idea of doing this, and wonder if you have
| any pointers before I get started.
| joshmarlow wrote:
| Not sure if this is where your head is, but I think there's a
| lot of value in integrating LLMs directly into complex
| software. Jira, Salesforce, maybe K8s - should all have an
| integrated LLMs that can walk you through how to perform a
| nuanced task in the software.
| dpacmittal wrote:
| Why would the LLM walk you through and not just do the
| nuanced task on its own?
| pennomi wrote:
| I assume the human maintains some of the necessary context
| in their meat memory.
| artninja1988 wrote:
| Given the complete failure of the first stable lm, I'm interested
| to try this one out. Haven't really seen a small language model,
| except mixtral 7b that's really useful for much.
|
| I also hope stability comes out with a competitor to the new
| midjourney and dalle models! That's what put them on the map in
| the first place
| brianjking wrote:
| All of the Mistral versions have been excellent, including the
| OpenHermes versions. I encourage you to check out Phi-2 as
| well, it's the only 3b model I've found really quite
| interesting outside of Replit's code model built into Replit
| Core.
| tarruda wrote:
| Deepseek coder 6.7B is very useful for coding and can run in
| consumer GPUs.
|
| I use the 6bit GGUF quantized version on a laptop RTX 3070
| mchiang wrote:
| It's amazing to see more smaller models being released. This
| creates opportunities for more developers to run it on their
| local computers, and makes it easier to fine-tune for specific
| needs.
| brcmthrowaway wrote:
| Has anyone tried starting with a smaller modeling, then RLing
| until it improves to the bigger model?
| lfkdev wrote:
| How is this compared to the current GitHub Copilot?
| brianjking wrote:
| A 3B tiny model is not going to compare to GitHub copilot.
| However, there are plenty of nice 7B models that are excellent
| at code and I encourage you to try them out.
| londons_explore wrote:
| If you just want to get stuff done, use the best tools like a
| Milwaukee Drill - and right now, thats copilot/gpt-4.
|
| If you don't want to be tied to a company and like opensource,
| feel free to connect a toy motor to an AA battery to drill your
| holes... Or to use Llama/Stable Code 3B.
| mistercheph wrote:
| it's going to be real hard to pry the carburetors out of this
| guy's cold dead hands!
| photon_collider wrote:
| How reliable are these benchmarks?
| ilaksh wrote:
| I think the trick is that they are just comparing to other tiny
| models.
|
| None of the little models, including this one, are comparable
| to the performance of the larger models for any significant
| coding problem.
|
| I think what these are useful for is mostly giving people hints
| inside of a code editor. Occasionally filling in the blank.
| knicholes wrote:
| I've got a machine with 4 3090s-- Anyone know which model would
| perform the best for programming? It's great this can run on a
| machine w/out a graphics card and is only 3B params, but I have
| the hardware. Might as well use it.
| SushiHippie wrote:
| Here is a leader board of some models
|
| https://huggingface.co/spaces/mike-ravkine/can-ai-code-resul...
|
| Don't know how biased this leaderboard is, but I guess you
| could just give some of them a try and see for yourself.
| tarruda wrote:
| Note that they don't compare with deepseek coder 6.7b, which is
| vastly superior to much bigger coding models. Surpassing
| codellama 7b is not that big of a deal today.
|
| The most impressive thing about these results is how good the
| 1.3B deepseek coder is.
| connorgutman wrote:
| FYI: This model is already available on Ollama.
| kleiba wrote:
| It's quite amazing - I often find that I read quite positive
| comments towards LLM tools for coding. Yet, an "Ask HN" I posted
| a while ago (and which admittedly didn't gain much traction)
| seemed to mirror mostly negative/pessimistic responses.
|
| https://news.ycombinator.com/item?id=38803836
|
| Was it just that my submission didn't find enough / more balanced
| commenters?
| simonw wrote:
| You only got comments from six people so yeah, definitely not
| representative.
| akulbe wrote:
| I just tried this model with Koboldcpp on my LLM box. I got
| gibberish back.
|
| My prompt - "please show me how to write a web scraper in Python"
|
| The response?
|
| <blockquote> I've written my first ever python script about 5
| months ago and I really don't remember anything except for the
| fact that I used Selenium in order to scrape websites (in this
| case, Google). So you can probably just copy/paste all of these
| lines from your own Python code which contains logic to determine
| what value should be returned when called by another piece of
| software or program. </blockquote>
| endofreach wrote:
| This doesn't seem like gibberish though?
| connorgutman wrote:
| Same thing with Ollama.
| Tiberium wrote:
| But it's a code completion model, not a chat/instruct one.
| SushiHippie wrote:
| It's very likely a "completion model" and not instruct/chat
| fine-tuned.
|
| So you'd need to prompt it through comments or by starting with
| a function name, basically the same as one would prompt GitHub
| copilot.
|
| e.g. # the following code implements a
| webscraper in python class WebScraper:
|
| (I didn't try this, and I'm not good at prompting, but
| something along the lines of this example should yield better
| results)
| MrNeon wrote:
| It is weird that it is not mentioned in the model card but I'm
| pretty sure it is a completion model, not tuned as an instruct
| model.
|
| edit: the webpage does call it "Stable Code Completion"
___________________________________________________________________
(page generated 2024-01-16 23:00 UTC)