[HN Gopher] LibreCUDA - Launch CUDA code on Nvidia GPUs without ...
___________________________________________________________________
LibreCUDA - Launch CUDA code on Nvidia GPUs without the proprietary
runtime
Author : rrampage
Score : 261 points
Date : 2024-08-08 17:24 UTC (5 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| JonChesterfield wrote:
| Very nice! That's essentially all I want from a cuda runtime. It
| should be possible to run llvm libc unit tests against this,
| which might then justify a corresponding amd library that does
| the same direct to syscall approach.
| snihalani wrote:
| For a non cuda n00b, what problem does this solve?
| heyoni wrote:
| Like anything open source it allows you to know and see exactly
| what your machine is doing. I don't want to speculate too much
| but I remember there being discussions around whether or not
| nvidia could embed licensing checks and such at the firmware
| level.
| samstave wrote:
| > licensing checks and such at the firmware level.
|
| Could you imaging an age where the NVIDIA firmware does
| LLM/AI/GPU license checking before it does operations on your
| vectors? (Hello Oracle on SUN e650, My old Friend) ((Worse
| would be a DRM check against deep-faking or other Globalist
| WEF Guardrails))
|
| _((oracle had(has) an age olde function where if you bought
| a license for a single proc and threw it inot a dual proc sun
| enterprise server with an extra proc or so - it knew you have
| several hundred K to spend on an additional e650 so why not
| have an extra ~$80K for an additional oracle proc license.
| Rather than make the app actually USE the additional proc -
| as there were no changes to oracles garbage FU Maxwell))_
| IntelMiner wrote:
| "Globalist WEF Guardrails"
|
| Tell us what you really feel
| samstave wrote:
| If you use all the GPTs enough - you'll see them clear as
| day...
|
| And by saying "Tell us how you really feel" reveals, you
| may not have thought of _The Implications_ of the current
| state of AI.
|
| (I can give you a concrete example of the WEF guardrails:
|
| I have a LBB of some high profile names that are all
| related around a specific person, then I wanted to see
| how they were related to one another from a publicly
| available data-set "that which is searchable on the open
| internet"
|
| And several GPTs stated "I do not _feel comfortable_
| revelaing the connections between these people without
| their consent "
|
| I was trying to get a list of public record data for whom
| the owners and affiliates of shared companies were...
|
| If you go down political/financial/professional rabbit
| holes using various data-mining techniques with
| augmenting searches and connections via public GPTs (paid
| even) -- You see the guardrails real fast (hint - they
| invlove power, money, and particular names and
| organizations that you hit guardrails against)
| kelnos wrote:
| I don't necessarily disagree with your overall point (I
| don't know much about it either way), but I'm not sure
| your example does a great job illustrating it. If you
| tried the same thing, but with non-high-profile names,
| would it give you the same response? If so, the
| charitable (and probably correct) interpretation is that
| this is a general privacy guardrail, not one that's there
| to protect the powerful/rich.
| dmnmnm wrote:
| > If so, the charitable (and probably correct)
| interpretation is that this is a general privacy
| guardrail, not one that's there to protect the
| powerful/rich
|
| Considering that some of the champions behind machine
| learning, like Google, are companies that made a living
| out of violating your privacy just to serve more ads to
| your eyeballs.. I wouldn't be so charitable.
|
| Tech bros have an inherent disregard for the privacy of
| others or for author rights for that matter. Was anyone
| asked if their art could be used to train their
| replacement?
|
| Power for me, not for thee.
| daedrdev wrote:
| I feel like thats a bit too hard of an idea to keep
| hidden considering the number of engineers and people who
| worked on this project. I would assume it's some
| combination of how models are not good at knowing
| specific people or companies very well since they use
| patterns for their output and the model being instructed
| to not allow doxing and harassment.
|
| Not to mention that the rich and powerful you imply are
| not tech savvy and probably did not understand or know
| about this tech when the datasets were being made.
| queuebert wrote:
| Two obvious problems that come to mind are
|
| 1. Replacing the extremely bloated official packages with
| lightweight distribution that provides only the common
| functionality.
|
| 2. Paving the way for GPU support on *BSD.
| jstanley wrote:
| Do you still need to be running the proprietary nvidia graphics
| driver, or is that completely unrelated?
| tptacek wrote:
| Presumably yes, if it functions through an ioctl interface.
| kaladin-jasnah wrote:
| You will need an NVIDIA driver (the README says as much), be it
| the proprietary or open source modules. Looks like this is
| performing RM (Resource Manager, which is the low-level API
| that is used to communicate with the NVIDIA proprietary driver
| using ioctl calls) API calls. If you look in the src/nvidia
| directory, many of the header files are RM API call header
| files from the NVIDIA open source kernel module, containing
| various structures and RM API command numbers (not sure if this
| is the official term).
|
| Fun thing, the open source modules takes some proprietary
| things and moves them to the GSP firmware. Incidentally, the
| open source modules actually communicate with the GSP firmware
| using the RM API as well. This understanding may be correct,
| but now instead of some RM calls being handled in kernel space
| they are forwarded to the firmware and handled there.
| KeplerBoy wrote:
| What's a CUDA elf file?
|
| Is it binary SASS code, so one would still need a open source
| ptxas alternative?
| shmerl wrote:
| Since ZLUDA was taken down (by request from AMD of all parties),
| it would be better to have some ZLUDA replacement as a general
| purpose way of breaking CUDA lock-in. I.e. something _not_ tied
| to Nvidia hardware.
| KeplerBoy wrote:
| That's a problem on a different level of the CUDA stack.
|
| Having a compiler that takes a special C++ or python dialect
| and compiles it to GPU suitable llvm-ir and then to a GPU
| binary is one thing (and there's progress on that side: triton,
| numba, soonish mojo), being able to launch that binary without
| going through the nvidia driver is another problem.
| shmerl wrote:
| Yeah, the latter one is more useful for effective lock-in
| breaking.
| codedokode wrote:
| Cannot Vulcan compute be used to execute code on GPU without
| relying on proprietary libs? Why not?
| Conscat wrote:
| You still require a Vulkan driver to do anything with it.
| Until last year, Nvidia hardware required a proprietary
| Vulkan driver (prior to Nvvk), and anything pre-Pascal
| still requires that.
| codedokode wrote:
| Yes but you can use any GPU with Vulkan, not only NVIDIA.
| SushiHippie wrote:
| > At this point, one more hostile corporation does not make
| much difference. I plan to rebuild ZLUDA starting from the pre-
| AMD codebase. Funding for the project is coming along and I
| hope to be able to share the details in the coming weeks. It
| will have a different scope and certain features will not come
| back. I wanted it to be a surprise, but one of those features
| was support for NVIDIA GameWorks. I got it working in Batman:
| Arkham Knight, but I never finished it, and now that code will
| never see the light of the day:
|
| So if I understand it correctly there is something in the works
|
| https://github.com/vosen/ZLUDA
| shmerl wrote:
| Ah, that's good. Hopefully it will get back on track then.
| wackycat wrote:
| I have limited experience with CUDA but will this help solve the
| CUDA/CUDNN dependency version nightmare that comes with running
| various ML libraries like tensorflow or onnx?
| bstockton wrote:
| My experience, over 10 years building models with libraries
| using CUDA under the hood, this problem has nearly gone away in
| the past few years. Setting up CUDA on new machines and even
| getting multi GPU/nodes configuration working with NCCL and
| pytorch DDP, for example, is pretty slick. Have you experienced
| this recently?
| trueismywork wrote:
| That's torches bad software distribution problem. No one can
| solve it apart from torch distributors
| greenavocado wrote:
| The authors better start thinking about the trademark
| infringement notice coming their way
| kkielhofner wrote:
| Naming anything is hard and I don't have better suggestions but
| when you're doing something that's already poking at something
| a big corp holds dearly hitting on trademark while you're at it
| makes it really easy for them.
| greenavocado wrote:
| You can't have the CUDA substring in the name or anything a
| court would deem potentially confusing. Even if "CUDA" wasn't
| registered, using a similar name could be seen as an attempt
| to pass off the product as affiliated with or endorsed by
| NVIDIA. The similarity in names could be construed as an
| attempt to unfairly benefit from NVIDIA's reputation and
| market position. If the open-source project implements
| techniques or methods patented by NVIDIA for CUDA, it could
| face patent infringement claims. If CUDA is considered a
| famous mark, using a similar name could be seen as diluting
| its distinctiveness, even if the products aren't directly
| competing. If domain names similar to CUDA-related domains
| are registered for the project, this could potentially lead
| to domain dispute issues. It's a huge can of worms.
| Dalewyn wrote:
| I wonder to what ends trademark protections reach.
|
| Firsthand example, both SpaceX and Subaru have services
| called Starlink. Subaru Starlink was first, but SpaceX
| Starlink is more famous. I've been confused and I've seen
| others be confused by the two.
| rockemsockem wrote:
| Those are two totally different businesses and
| industries, so their trademarks dont clash
| Dalewyn wrote:
| Subaru Starlink is a wireless communication network for
| Subaru cars, it lets the cars make phone calls to Subaru
| customer support and emergency services. I believe it's
| also how Subaru cars update their car navigation. It is a
| subscription service.
|
| SpaceX Starlink is a wireless communication network for
| internet service, including on-the-road service. It is a
| subscription service.
|
| You tell me this doesn't confuse people who aren't privy
| to the technical details.
| cj wrote:
| A trademark is scoped to a specific industry /
| application.
|
| Starlink for internet is unlikely to be confused with
| STARLINK for Subaru car safety systems. (Perhaps the all
| caps also helps if they were sued)
|
| Trademark applications are scoped so that you can't
| monopolize a name, you only own the name within the
| industry you operate in.
|
| For example, there's a real estate investment fund named
| Apple and even trades with stock ticker APLE.
| allan_s wrote:
| something like kudo ?
| greenavocado wrote:
| https://news.ycombinator.com/item?id=41195332
| gavindean90 wrote:
| kudo seems to be in keeping with your comment. I am not
| sure what you are getting at.
| jahewson wrote:
| Name it "Barra".
| Onavo wrote:
| What about the extra bits like CuDNN?
| m3kw9 wrote:
| Why is there a need to do this?
| curious_cat_163 wrote:
| 1. To learn, how. 2. Nvidia needs to be challenged with OSS.
| They are far too big to be left alone. 3. To have some fun.
| nasretdinov wrote:
| Such a missed opportunity to call it CUDA Libre...
| phoronixrly wrote:
| Unfortunately both would seem to be infringing on nvidia's
| trademark... We just can't have nice things..
| mrbungie wrote:
| Then call it Cuba Libre.
| phoronixrly wrote:
| Sounds too similar to cuda.. :(
| diggan wrote:
| Or go even further, call it "Culo Libre", only two letters
| away anyways.
| londons_explore wrote:
| A trademark isn't a total prohibition on using someone else's
| name.
|
| You can still use their name where there is no likelihood of
| consumer confusion.
|
| Obviously many companies choose not to to avoid a lawsuit
| over the issue - but it's unlikely NVidia would win over this
| method name.
| marcodiego wrote:
| Worse, some functions are prefixed libreCu which in Portuguese
| means "free ass hole".
| HPsquared wrote:
| Maybe better use 'lcu'.
| ronsor wrote:
| I feel like every name or phrase will inevitably be offensive
| in some language.
| tuetuopay wrote:
| Same meaning in french. Oh well...
| porphyra wrote:
| Can you explain this joke? I don't get it.
| holowoodman wrote:
| There is a drink called "Cuba Libre" (White Rum + Coke).
| madars wrote:
| Traditionally, Cuba libre also has lime juice while a plain
| "rum and Coke" order leaves it out. https://iba-
| world.com/cuba-libre/
| daft_pink wrote:
| I think the point of open cuda is to run it on non NVIDIA gpus.
| Once you have to buy NVIDIA gpus what's the point. If we had true
| you competition I think it would be far easier to buy devices
| with more vram and thus we might be able to run llama 405b
| someday locally.
|
| Once you already bought the NVIDIA cards what's the point
| londons_explore wrote:
| The NVidia software stack has the "no use in datacenters"
| clause. Is this a workaround for that?
| why_only_15 wrote:
| Specifically the clause is that you cannot use their consumer
| cards (e.g. RTX 4090) in datacenters.
| candiddevmike wrote:
| That's why we run all of our ML workloads in a distributed
| GPU cluster located in every employee's house
| kelnos wrote:
| Some people believe being able to build on fully-open software
| stacks has value in and of itself. (I happen to be one of those
| people.)
|
| Another benefit could be support for platforms that nvidia
| doesn't care to release CUDA SDKs for.
___________________________________________________________________
(page generated 2024-08-08 23:00 UTC)