[HN Gopher] LibreCUDA - Launch CUDA code on Nvidia GPUs without ...
___________________________________________________________________
LibreCUDA - Launch CUDA code on Nvidia GPUs without the proprietary
runtime
Author : rrampage
Score : 618 points
Date : 2024-08-08 17:24 UTC (1 days ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| JonChesterfield wrote:
| Very nice! That's essentially all I want from a cuda runtime. It
| should be possible to run llvm libc unit tests against this,
| which might then justify a corresponding amd library that does
| the same direct to syscall approach.
| snihalani wrote:
| For a non cuda n00b, what problem does this solve?
| heyoni wrote:
| Like anything open source it allows you to know and see exactly
| what your machine is doing. I don't want to speculate too much
| but I remember there being discussions around whether or not
| nvidia could embed licensing checks and such at the firmware
| level.
| samstave wrote:
| > licensing checks and such at the firmware level.
|
| Could you imaging an age where the NVIDIA firmware does
| LLM/AI/GPU license checking before it does operations on your
| vectors? (Hello Oracle on SUN e650, My old Friend) ((Worse
| would be a DRM check against deep-faking or other Globalist
| WEF Guardrails))
|
| _((oracle had(has) an age olde function where if you bought
| a license for a single proc and threw it inot a dual proc sun
| enterprise server with an extra proc or so - it knew you have
| several hundred K to spend on an additional e650 so why not
| have an extra ~$80K for an additional oracle proc license.
| Rather than make the app actually USE the additional proc -
| as there were no changes to oracles garbage FU Maxwell))_
| IntelMiner wrote:
| "Globalist WEF Guardrails"
|
| Tell us what you really feel
| samstave wrote:
| If you use all the GPTs enough - you'll see them clear as
| day...
|
| And by saying "Tell us how you really feel" reveals, you
| may not have thought of _The Implications_ of the current
| state of AI.
|
| (I can give you a concrete example of the WEF guardrails:
|
| I have a LBB of some high profile names that are all
| related around a specific person, then I wanted to see
| how they were related to one another from a publicly
| available data-set "that which is searchable on the open
| internet"
|
| And several GPTs stated "I do not _feel comfortable_
| revelaing the connections between these people without
| their consent "
|
| I was trying to get a list of public record data for whom
| the owners and affiliates of shared companies were...
|
| If you go down political/financial/professional rabbit
| holes using various data-mining techniques with
| augmenting searches and connections via public GPTs (paid
| even) -- You see the guardrails real fast (hint - they
| invlove power, money, and particular names and
| organizations that you hit guardrails against)
| kelnos wrote:
| I don't necessarily disagree with your overall point (I
| don't know much about it either way), but I'm not sure
| your example does a great job illustrating it. If you
| tried the same thing, but with non-high-profile names,
| would it give you the same response? If so, the
| charitable (and probably correct) interpretation is that
| this is a general privacy guardrail, not one that's there
| to protect the powerful/rich.
| dmnmnm wrote:
| > If so, the charitable (and probably correct)
| interpretation is that this is a general privacy
| guardrail, not one that's there to protect the
| powerful/rich
|
| Considering that some of the champions behind machine
| learning, like Google, are companies that made a living
| out of violating your privacy just to serve more ads to
| your eyeballs.. I wouldn't be so charitable.
|
| Tech bros have an inherent disregard for the privacy of
| others or for author rights for that matter. Was anyone
| asked if their art could be used to train their
| replacement?
|
| Power for me, not for thee.
| daedrdev wrote:
| I feel like thats a bit too hard of an idea to keep
| hidden considering the number of engineers and people who
| worked on this project. I would assume it's some
| combination of how models are not good at knowing
| specific people or companies very well since they use
| patterns for their output and the model being instructed
| to not allow doxing and harassment.
|
| Not to mention that the rich and powerful you imply are
| not tech savvy and probably did not understand or know
| about this tech when the datasets were being made.
| samstave wrote:
| Here is the markdown formatting code for your text:
|
| *Warning: Deep Rabbit Hole Info Ahead!* Please ignore if
| the following triggers you in any sense...
|
| > _Take on the archetype of the best corporate counsel
| and behavioral psychologist - as a profiler for the NSA
| regarding cyber security and crypto concerns._ > _With
| this as your discernment lattice - describe Sam Altman in
| your Field 's Dossier given what you understand of the AI
| Climate explain how you're going to structure your
| response, in a way that students of your field but with a
| less sophisticated perception can understand_
|
| ---
|
| >> _Altman 's behavior and leadership style can be
| characterized by the following traits:_ >>- _Strategic
| and Ambitious: He exhibits a strong drive for success,
| often taking calculated risks to achieve his goals._ >>-
| _Manipulative Tendencies: Reports suggest a pattern of
| manipulating situations to his advantage, raising ethical
| concerns._ >>- _Polarizing Figure: Altman 's actions
| elicit strong reactions, with some admiring his
| achievements and others criticizing his ethics._
|
| ---
|
| ### Models and References for Evaluating Sam Altman
|
| #### Five-Factor Model (Big Five Personality Traits)
| _Description:_ This model evaluates personality based on
| five dimensions: openness, conscientiousness,
| extraversion, agreeableness, and neuroticism.
| _Application:_ Used to assess Altman 's personality
| traits and predict potential behaviors and ethical
| considerations. _Reference:_ McCrae, R. R., & John, O.
| P. (1992). "An Introduction to the Five-Factor Model and
| Its Applications." Journal of Personality, 60(2),
| 175-215.
|
| #### Situational Leadership Theory _Description:_ This
| theory suggests that effective leadership varies
| depending on the situation and the leader 's ability to
| adapt. _Application:_ Evaluates Altman 's leadership
| style and effectiveness in different contexts,
| particularly during crises. _Reference:_ Hersey, P., &
| Blanchard, K. H. (1969). "Life Cycle Theory of
| Leadership." Training and Development Journal, 23(5),
| 26-34.
|
| #### Ethical Decision-Making Models _Description:_
| Frameworks that provide structured approaches to making
| ethical decisions, considering factors like stakeholders,
| consequences, and moral principles. _Application:_
| Analyzes Altman 's decision-making processes and ethical
| considerations. _Reference:_ Rest, J. R. (1986). "Moral
| Development: Advances in Research and Theory." Praeger.
|
| #### Corporate Governance Principles _Description:_
| Guidelines and best practices for managing and governing
| a corporation, focusing on transparency, accountability,
| and stakeholder interests. _Application:_ Assesses Altman
| 's alignment with good governance practices and his
| impact on OpenAI's organizational stability. _Reference:_
| Cadbury, A. (1992). "Report of the Committee on the
| Financial Aspects of Corporate Governance." Gee and Co.
| Ltd.
|
| #### Cybersecurity Risk Assessment Frameworks
| _Description:_ Methodologies for identifying, analyzing,
| and mitigating cybersecurity risks, particularly in high-
| tech environments. _Application:_ Evaluates the potential
| cybersecurity risks associated with Altman 's actions and
| OpenAI's technologies. _Reference:_ National Institute of
| Standards and Technology (NIST). (2018). "Framework for
| Improving Critical Infrastructure Cybersecurity."
|
| #### Behavioral Economics _Description:_ Studies the
| effects of psychological, cognitive, and emotional
| factors on economic decisions. _Application:_ Understands
| how Altman 's personal motivations and cognitive biases
| might influence his strategic decisions. _Reference:_
| Kahneman, D., & Tversky, A. (1979). "Prospect Theory: An
| Analysis of Decision under Risk." Econometrica, 47(2),
| 263-291.
|
| ### Supporting Expertise
|
| * *Behavioral Psychology*: Expertise in understanding
| human behavior, personality traits, and decision-making
| processes. * *Corporate Law and Governance*: Knowledge of
| corporate structures, governance frameworks, and ethical
| standards. * *Cybersecurity*: Understanding cybersecurity
| threats and risk management strategies. * *Ethics and
| Compliance*: Proficiency in ethical decision-making and
| compliance standards.
| samstave wrote:
| I dont want to 'pollute' HN with my (its all real and
| well researched and informed) 'conspiracy' theories --
| but I've been keeping receipts folks.
|
| If youre on HN, involved in Tech to any degree of
| seriousness, and dont ask yourself the hard
| alignment/entanglement questions, You're Holding It
| Wrong.
|
| ---
|
| Altman is on the WEF roster, is all in on AI war stuff,
| in bed with Power MIC. If people were _fleeing_ from the
| company and we cant honestly just say they simply are
| Cashin Out, and dismissing all their writings and
| statements and tweets, and podcast appearances, etc...
|
| Check out this guys post on reddit:
|
| https://old.reddit.com/r/OpenAI/comments/1cvtiv2/on_open_
| ai_...
|
| --
|
| After attempting to build a thing with openai AND claude
| (paid) - I am convinced that there is a malevolent AGI -
| and I think there is more than one of them.
| Maken wrote:
| By _malevolent_ you mean the AI systems are designed to
| make profit for its owners and not to benefit humanity or
| whatever?
| samstave wrote:
| Basically that's exactly it.
|
| HN isnt the platform to go deep on it, but im a fairly
| well evolved techno-conspiratist - and I've (as I
| jokingly stated) "forrest Gump'd" around a lot of silicon
| valley history..
|
| And in my use daily of attempting to build what should be
| a simple thing with all the inputs of the GPTs, and paid
| versions - I am convinced that when youre using the tools
| in a meaningful manner which is leading in certain
| directions - there are triggers, and I think even humans,
| that get invloved.
|
| On multiple occassions both on claude (paid) and gpt
| (paid) - ive had them strip out code AS ITS BEING
| GENERATED and tell me that its being stripped out for
| violations/concerns - but it just flashes the message,
| doesnt tell you which code, what violations, etc.
|
| It lies, it ignores direct stements, ignores context in
| uploaded files, violates memory boundaries, and
| maliciously removes previously approved snippets of
| code/features etc.
|
| I have managed exceptional devops teams, developers, PHDs
| even. I know whats up.
|
| These bots are designed to edge, and consume your use of
| them.
|
| THey actively, but very insipidly, thwart certain things.
| Maken wrote:
| But rich and powerful people includes the ones who own
| the companies building those datasets, and they do
| understand the tech.
| sharpshadow wrote:
| I know from this book[1] how deep and far reaching even
| just publicly available data can get you. Furthermore
| with financials even so far that one can predict upcoming
| events from analysing the money flow.
|
| Very powerful people are very good at hiding. It's no
| surprise that they want themselves excluded from various
| searches and are successfully able to do so. Would be
| interesting to know if the data is excluded already from
| the training data or if it's technically inside.
|
| edit: source added 1. https://www.amazon.de/INSIDE-
| CORONA-Pandemie-Netzwerk-Hinter...
| samstave wrote:
| Im fully convinced now that both openai and anthropic
| have agi. mayhaps not in whatever a 'conventional
| definition is' -- in fact, I think its far more
| insidious: I think that its a computing logic/reasoning
| system which, when fully connected to the internet and
| whatever other networks they can give it access to - it
| has Omniscient Capability.
|
| We've known of echelon being a fully capable phone
| surveillance system since the 70s.
|
| We knew of a lot of capability the agencies etc have had
| over the decades.
|
| I wonder when Sam Altman may visit Antarctica?
| Q6T46nT668w6i3m wrote:
| Quadro?
| cyberpunk wrote:
| It was even worse than that. Even if you created a resource
| pool with only the one CPU on a dual system they wanted
| licenses for both as you could "potentially" use both CPUs.
|
| On VMware they extended this to every CPU in the cluster.
|
| A gigantic shower of absolute grifters.
| queuebert wrote:
| Two obvious problems that come to mind are
|
| 1. Replacing the extremely bloated official packages with
| lightweight distribution that provides only the common
| functionality.
|
| 2. Paving the way for GPU support on *BSD.
| jstanley wrote:
| Do you still need to be running the proprietary nvidia graphics
| driver, or is that completely unrelated?
| tptacek wrote:
| Presumably yes, if it functions through an ioctl interface.
| kaladin-jasnah wrote:
| You will need an NVIDIA driver (the README says as much), be it
| the proprietary or open source modules. Looks like this is
| performing RM (Resource Manager, which is the low-level API
| that is used to communicate with the NVIDIA proprietary driver
| using ioctl calls) API calls. If you look in the src/nvidia
| directory, many of the header files are RM API call header
| files from the NVIDIA open source kernel module, containing
| various structures and RM API command numbers (not sure if this
| is the official term).
|
| Fun thing, the open source modules takes some proprietary
| things and moves them to the GSP firmware. Incidentally, the
| open source modules actually communicate with the GSP firmware
| using the RM API as well. This understanding may be correct,
| but now instead of some RM calls being handled in kernel space
| they are forwarded to the firmware and handled there.
| KeplerBoy wrote:
| What's a CUDA elf file?
|
| Is it binary SASS code, so one would still need a open source
| ptxas alternative?
| shmerl wrote:
| Since ZLUDA was taken down (by request from AMD of all parties),
| it would be better to have some ZLUDA replacement as a general
| purpose way of breaking CUDA lock-in. I.e. something _not_ tied
| to Nvidia hardware.
| KeplerBoy wrote:
| That's a problem on a different level of the CUDA stack.
|
| Having a compiler that takes a special C++ or python dialect
| and compiles it to GPU suitable llvm-ir and then to a GPU
| binary is one thing (and there's progress on that side: triton,
| numba, soonish mojo), being able to launch that binary without
| going through the nvidia driver is another problem.
| shmerl wrote:
| Yeah, the latter one is more useful for effective lock-in
| breaking.
| codedokode wrote:
| Cannot Vulcan compute be used to execute code on GPU without
| relying on proprietary libs? Why not?
| Conscat wrote:
| You still require a Vulkan driver to do anything with it.
| Until last year, Nvidia hardware required a proprietary
| Vulkan driver (prior to Nvvk), and anything pre-Pascal
| still requires that.
| codedokode wrote:
| Yes but you can use any GPU with Vulkan, not only NVIDIA.
| cyber_kinetist wrote:
| Vulkan Compute's semantics are limited by SPIR-V and thus
| cannot implement all of the features CUDA provides (ex.
| there is no proper notion of a "pointer")
|
| Also it's much more convenient to use plain C++ rather than
| a custom shading language, especially if you're writing
| complex numerical code or need some heavy templated
| abstractions to do powerful stuff. And the CUDA tooling
| itself is just much easier to use compared to Vulkan, with
| its seamless integration of host / device code.
| nsajko wrote:
| > [...] there's progress [...]
|
| Don't forget about Julia!
| KeplerBoy wrote:
| and jax, tinygrad and halide. God it's such an awesome time
| to be into that stuff.
| SushiHippie wrote:
| > At this point, one more hostile corporation does not make
| much difference. I plan to rebuild ZLUDA starting from the pre-
| AMD codebase. Funding for the project is coming along and I
| hope to be able to share the details in the coming weeks. It
| will have a different scope and certain features will not come
| back. I wanted it to be a surprise, but one of those features
| was support for NVIDIA GameWorks. I got it working in Batman:
| Arkham Knight, but I never finished it, and now that code will
| never see the light of the day:
|
| So if I understand it correctly there is something in the works
|
| https://github.com/vosen/ZLUDA
| shmerl wrote:
| Ah, that's good. Hopefully it will get back on track then.
| wackycat wrote:
| I have limited experience with CUDA but will this help solve the
| CUDA/CUDNN dependency version nightmare that comes with running
| various ML libraries like tensorflow or onnx?
| bstockton wrote:
| My experience, over 10 years building models with libraries
| using CUDA under the hood, this problem has nearly gone away in
| the past few years. Setting up CUDA on new machines and even
| getting multi GPU/nodes configuration working with NCCL and
| pytorch DDP, for example, is pretty slick. Have you experienced
| this recently?
| jokethrowaway wrote:
| yes, especially if you are trying to run various different
| projects you don't control
|
| some will need specific versions of cuda
|
| right now I masked cuda from upgrades in my system and I'm
| stuck on an old version to support some projects
|
| I also had plenty of problems with gpu-operator to deploy on
| k8s: that helm chart is so buggy (or maybe just not great at
| handling some corner cases? no clue) I ended up swapping
| kubernetes distribution a few times (no chance to make it
| work on microk8s, on k3s it almost works) and eventually
| ended up installing drivers + runtime locally and then just
| exposing through containerd config
| trueismywork wrote:
| That's torches bad software distribution problem. No one can
| solve it apart from torch distributors
| amelius wrote:
| By the way, can anyone explain why libcudnn takes on the order
| of gigabytes on my harddrive?
| lldb wrote:
| Primarily because it has specialized functions for various
| matrix sizes which are selected at runtime.
| amelius wrote:
| Ok, so are you saying it contains mostly straightforwardly
| generated code?
| greenavocado wrote:
| The authors better start thinking about the trademark
| infringement notice coming their way
| kkielhofner wrote:
| Naming anything is hard and I don't have better suggestions but
| when you're doing something that's already poking at something
| a big corp holds dearly hitting on trademark while you're at it
| makes it really easy for them.
| greenavocado wrote:
| You can't have the CUDA substring in the name or anything a
| court would deem potentially confusing. Even if "CUDA" wasn't
| registered, using a similar name could be seen as an attempt
| to pass off the product as affiliated with or endorsed by
| NVIDIA. The similarity in names could be construed as an
| attempt to unfairly benefit from NVIDIA's reputation and
| market position. If the open-source project implements
| techniques or methods patented by NVIDIA for CUDA, it could
| face patent infringement claims. If CUDA is considered a
| famous mark, using a similar name could be seen as diluting
| its distinctiveness, even if the products aren't directly
| competing. If domain names similar to CUDA-related domains
| are registered for the project, this could potentially lead
| to domain dispute issues. It's a huge can of worms.
| Dalewyn wrote:
| I wonder to what ends trademark protections reach.
|
| Firsthand example, both SpaceX and Subaru have services
| called Starlink. Subaru Starlink was first, but SpaceX
| Starlink is more famous. I've been confused and I've seen
| others be confused by the two.
| rockemsockem wrote:
| Those are two totally different businesses and
| industries, so their trademarks dont clash
| Dalewyn wrote:
| Subaru Starlink is a wireless communication network for
| Subaru cars, it lets the cars make phone calls to Subaru
| customer support and emergency services. I believe it's
| also how Subaru cars update their car navigation. It is a
| subscription service.
|
| SpaceX Starlink is a wireless communication network for
| internet service, including on-the-road service. It is a
| subscription service.
|
| You tell me this doesn't confuse people who aren't privy
| to the technical details.
| cj wrote:
| A trademark is scoped to a specific industry /
| application.
|
| Starlink for internet is unlikely to be confused with
| STARLINK for Subaru car safety systems. (Perhaps the all
| caps also helps if they were sued)
|
| Trademark applications are scoped so that you can't
| monopolize a name, you only own the name within the
| industry you operate in.
|
| For example, there's a real estate investment fund named
| Apple and even trades with stock ticker APLE.
| actionfromafar wrote:
| It would make total sense for STARLINK to use satellites
| to call for help.
| tbrownaw wrote:
| > _A trademark is scoped to a specific industry /
| application._
|
| Or at least it's supposed to be.
|
| https://www.sportskeeda.com/wwe/wwf
| greenavocado wrote:
| The Sleekcraft test or the Ninth Circuit likelihood of
| confusion test, is used to determine trademark
| infringement in the United States, particularly in the
| Ninth Circuit Court of Appeals. This test evaluates
| several factors to assess whether there is a likelihood
| of confusion between two trademarks. Factors considered:
|
| 1. Strength of the mark
|
| 2. Proximity of the goods
|
| 3. Similarity of the marks
|
| 4. Evidence of actual confusion
|
| 5. Marketing channels used
|
| 6. Type of goods and degree of care likely to be
| exercised by the purchaser
|
| 7. Defendant's intent in selecting the mark
|
| 8. Likelihood of expansion of the product lines
|
| To apply this test, courts examine each factor and weigh
| them collectively to determine if there's a likelihood of
| confusion between the trademarks in question. No single
| factor is determinative, and the importance of each
| factor may vary depending on the specific circumstances
| of the case.
|
| The courts will fudge their reasoning with those eight
| pillars to fit their opinion.
| thayne wrote:
| IANAL, and I don't know how a court would rule, but to me
| the name libreCUDA is self-evidently not affiliated with
| Nvidia, as the libre prefix indicates it is an open source
| alternative.
| allan_s wrote:
| something like kudo ?
| greenavocado wrote:
| https://news.ycombinator.com/item?id=41195332
| gavindean90 wrote:
| kudo seems to be in keeping with your comment. I am not
| sure what you are getting at.
| greenavocado wrote:
| It's far too similar.
| jahewson wrote:
| Name it "Barra".
| Onavo wrote:
| What about the extra bits like CuDNN?
| m3kw9 wrote:
| Why is there a need to do this?
| curious_cat_163 wrote:
| 1. To learn, how. 2. Nvidia needs to be challenged with OSS.
| They are far too big to be left alone. 3. To have some fun.
| nasretdinov wrote:
| Such a missed opportunity to call it CUDA Libre...
| phoronixrly wrote:
| Unfortunately both would seem to be infringing on nvidia's
| trademark... We just can't have nice things..
| mrbungie wrote:
| Then call it Cuba Libre.
| phoronixrly wrote:
| Sounds too similar to cuda.. :(
| chii wrote:
| cuba libre is a drink name, which won't infringe on
| trademark, as it's not really possible to confuse it with
| CUDA.
| diggan wrote:
| Or go even further, call it "Culo Libre", only two letters
| away anyways.
| pezezin wrote:
| If nVidia can release a library called cuLitho, I don't
| see why not.
| londons_explore wrote:
| A trademark isn't a total prohibition on using someone else's
| name.
|
| You can still use their name where there is no likelihood of
| consumer confusion.
|
| Obviously many companies choose not to to avoid a lawsuit
| over the issue - but it's unlikely NVidia would win over this
| method name.
| marcodiego wrote:
| Worse, some functions are prefixed libreCu which in Portuguese
| means "free ass hole".
| HPsquared wrote:
| Maybe better use 'lcu'.
| ronsor wrote:
| I feel like every name or phrase will inevitably be offensive
| in some language.
| tuetuopay wrote:
| Same meaning in french. Oh well...
| candiddevmike wrote:
| I'd love to work on a codebase like that. I think codebases
| need more swearing to fully capture context and intent of the
| developer.
| borsch wrote:
| GoddamnProxyFactory
| vasco wrote:
| Libre isn't a portuguese word so it doesn't mean free
| asshole. Livre does mean free, but these are different words.
| stuaxo wrote:
| Ha, this is great and fits in with that famous Linus speech
| to Nvid
| porphyra wrote:
| Can you explain this joke? I don't get it.
| holowoodman wrote:
| There is a drink called "Cuba Libre" (White Rum + Coke).
| madars wrote:
| Traditionally, Cuba libre also has lime juice while a plain
| "rum and Coke" order leaves it out. https://iba-
| world.com/cuba-libre/
| lmm wrote:
| Wikipedia suggests only a few sources draw that
| distinction and including lime or lime juice in a "rum
| and Coke" is very normal.
| Nursie wrote:
| I've had rum and coke with lime but I've never seen
| something without the lime referred to as Cuba Libre.
|
| That said, I have never been to Cuba, so what do I know?
| mattl wrote:
| Seriously. Such a shame
| daft_pink wrote:
| I think the point of open cuda is to run it on non NVIDIA gpus.
| Once you have to buy NVIDIA gpus what's the point. If we had true
| you competition I think it would be far easier to buy devices
| with more vram and thus we might be able to run llama 405b
| someday locally.
|
| Once you already bought the NVIDIA cards what's the point
| londons_explore wrote:
| The NVidia software stack has the "no use in datacenters"
| clause. Is this a workaround for that?
| why_only_15 wrote:
| Specifically the clause is that you cannot use their consumer
| cards (e.g. RTX 4090) in datacenters.
| candiddevmike wrote:
| That's why we run all of our ML workloads in a distributed
| GPU cluster located in every employee's house
| pplante wrote:
| The bonus is free heating for every employees household!
| rurban wrote:
| Free cooling also. You cannot really run a big GPU with
| external cooling. I needed rather big 15cm isolated
| cooling tubes to get the heat out of the building.
| seniorThrowaway wrote:
| you joke but I've thought about doing this
| bee_rider wrote:
| The employees can also store their desktops in specially
| cooled, centrally located lockers at work if they want.
| And as a perk, we'll buy and administrate these computers
| for them.
| paulmd wrote:
| use the open kernel driver, which is MIT/GPL and thus cannot
| impose usage restrictions.
|
| it's worth noting that "NVIDIA software stack" is an
| imprecise term. the driver is the part that has the
| datacenter usage term, and the open-kernel-driver bypasses
| that. the CUDA stack itself does _not_ have the datacenter
| driver clause, the only caveat is that you can 't run it on
| third-party hardware. So ZLUDA/GpuOcelot is still verboten,
| if you are using the CUDA libraries.
|
| https://docs.nvidia.com/cuda/eula/index.html
| kelnos wrote:
| Some people believe being able to build on fully-open software
| stacks has value in and of itself. (I happen to be one of those
| people.)
|
| Another benefit could be support for platforms that nvidia
| doesn't care to release CUDA SDKs for.
| IgorPartola wrote:
| Hear hear. Yes practically if you need to run a workload on a
| closed source system or if that's your only option to get the
| performance then you have to do what you have to do. But in
| the long run open source wins because once an open source
| alternative exists it is just the better option.
|
| As a bonus, with open source platforms you are much less
| subject to whims of company licensing. If tomorrow Nvidia
| decided to change their licensing strategy and pricing, how
| many here will be affected by it? OSS doesn't do that. And
| even if the project goes in a random direction you don't
| like, someone likely forks it to keep going in the right
| direction (see pfsense/opnsense).
| galkk wrote:
| > But in the long run open source wins because once an open
| source alternative exists it is just the better option.
|
| This is just wishful thinking. Anything close to real
| professional use, not related to IT, and closed source is
| king: office work, CAD, video editing, music production,
| and those domains immediately came to mind. Nowhere there
| open source can seriously challenge commercial, closed
| sourced competitors.
|
| Yes, in any of those domains one can name open source
| products, but they are far from "winning" or "the better
| option".
| funcDropShadow wrote:
| Counter example: blender. It may not be winning in video
| editing, but it has serious market share in 3d rendering.
| Different players are investing money in it and extend it
| with their own stuff.
| galkk wrote:
| Agree, blender is a contender.
| almostgotcaught wrote:
| Have you ever heard the phrase "the exception that proves
| the rule"?
| d4mi3n wrote:
| What if Linux itself? The plethora of open programming
| languages? Tools like OpenSSH?
|
| Commercial, closed source products generally benefit from
| a monopoly within a specific problem domain or some kind
| of regulatory capture. I don't think that means an open
| source alternative isn't desirable or viable, just that
| competing in those contexts is much more difficult
| without some serious investment--be it political,
| monetary, or through many volunteered hours of work.
|
| Another comment mentioned Blender which is a great
| example of a viable competitor in a specific problem
| domain. There are others if you look at things like PCB
| circuit design, audio production/editing, and a
| surprising amount of fantastic computer emulators.
| galkk wrote:
| I specifically mentioned not it related, so that rules
| out "Linux, programming languages, OpenSSh ... fantastic
| computer emulators".
|
| In general you confirmed my point by saying that
| competing in domains is much more difficult. And open
| source isn't a key to a win.
| raxxorraxor wrote:
| It is completely ok to use commercial software in a
| commercial environment. It isn't and shouldn't be the
| goal of open source to provide the best consumer product.
|
| In the grand scheme of things I believe open source at
| least provides serious competition and that commercial
| software has its own work to do.
|
| Also, a lot of not all professional work uses open source
| components. Research is a field where it shines and there
| it matters a lot.
|
| Adobe has to work for its money as well as its
| competitors get more powerful by the day. And everyone
| hates their creative cloud.
| borsch wrote:
| fucking unreal engine 5 is open source, dawg!
| RussianCow wrote:
| Unreal Engine is _source available_. It is definitely
| _not_ open source as you can 't use it without a
| commercial license from Epic.
| borsch wrote:
| It's commercial open source.
|
| Anything else is moving the goalposts.
| borsch wrote:
| lmao, okay, idiots
| horsawlarway wrote:
| I think open source does tend to win, it just does it
| slowly - often when the big commercial name screws up or
| changes ownership.
|
| ex - I think Adobe is in the middle of this swing now,
| Blender is eating marketshare, and Krita is pretty
| incredible.
|
| Unity is also struggling (I've seen a LOT of folks moving
| to Godot, or going back to unreal [which is not open, but
| is source-available - because having access matters]).
|
| CAD hasn't quite tipped yet - but Freecad is getting
| better constantly. I used to default to Fusion360 and
| Solidworks, but I haven't had to break those out for
| personal use in the last 5 years or so (CNC/3d printing
| needs). It's not ready for professional use yet, but it
| now feels like how blender felt in 2010 - usable, if not
| quite up to par.
|
| Office work... is a tough one - to date, Excel still
| remains king for the folks who actually need Excel.
| Everything else has moved to free (although not
| necessarily open source) editors. None of my employers
| have provided word/powerpoint for more than a decade now
| - and I haven't missed not having them.
|
| I would argue that PDFs have gone the opensource route
| though, and that used to be a big name in office work
| (again - Adobe screwed up).
|
| I don't really do any music production or video editing,
| so I can't really comment other than to say that ffmpeg
| is eating the world for commercial solutions under the
| hood, and it is solidly open. And on the streaming side
| of "Video" OBS studio is basically the only real player
| I'm aware of.
|
| So... I don't really think it's wishful thinking. I think
| opensource is genuinely better most times, it just plays
| the long and slow game to getting there.
| dgroshev wrote:
| I'm really sceptical that anything will happen in the CAD
| space bar massive state investment into open source
| infrastructure. Open CASCADE doesn't look to be catching
| up [1], while Solidworks continues to release
| foundational features like G3 continuity constraints, so
| the capability gap is going to widen over time.
|
| I'd be glad to be proven wrong.
|
| [1]: https://git.dev.opencascade.org/gitweb/?p=occt.git
| dahart wrote:
| > I would argue that PDFs have gone the opensource route
| though, and that used to be a big name in office work
| (again - Adobe screwed up).
|
| Naw, just try to find a decent PDF editor. You will have
| a hard time. PDF display is fairly open, but PDF editing
| is not. PDFs are the dominant format for exchange of
| signed documents, still a big name in office work, and
| Adobe still controls the PDF editing app market.
|
| I would love it if open source was winning in the
| imaging, audio or DCC markets, but it's just not even
| close yet. Blender hasn't touched pro market share, it's
| just being used by lots and lots of hobbyists because
| it's free to play with. Just did a survey of the film &
| VFX studios at Siggraph, and they aren't even looking in
| Blender's direction yet, they are good with Houdini,
| Maya, etc. Some of this has to do with fears and lack of
| understanding of open source licensing - studios are
| afraid of the legalities, and Ton has talked about
| needing to help educate them. Some new & small shops use
| Blender, but new & small shops come and go all the time,
| the business is extremely tough.
|
| Office work is moving to Microsoft alternatives like
| Google Office products. That is not open source, not
| source available, and for most medium to large companies
| it's not free either (though being "free" as in beer is
| irrelevant to your point). The company just pays behind
| the scenes and most employees don't know it, or it's
| driven by ad & analytics revenue.
|
| Unix utilities and Linux server software are places where
| open source has some big "wins", but unfortunately when
| it comes to content creation software, it still is
| wishful thinking. It could change in the future, and I
| honestly hope it does, but it's definitely not there yet.
| actionfromafar wrote:
| Yeah like running Linux on a MacBook...
| Q6T46nT668w6i3m wrote:
| CUDA is ubiquitous in science and an open source alternative to
| the CUDA runtime is useful, even if the use is limited to
| verifying expected behavior.
| segmondy wrote:
| Some of us are running llama 405B locally already. All my GPUs
| are ancient Nvidai GPUs. IMO, the point of an open cuda is to
| force Nvidia to stop squeezing us. You get more performance for
| the buck for AMD. If I could run cuda on AMD, I would have
| bought new AMD gpus instead. Have enough people do that and
| Nvidia might take note and stop squeezing us for cash.
| smokel wrote:
| _> the point of an open cuda is to force Nvidia to stop
| squeezing us_
|
| Nobody is forcing you to buy GPUs.
|
| Your logic is flawed in the sense that enough people could
| also simply write alternatives to Torch, which, by the way,
| is already open source.
| funcDropShadow wrote:
| Nobody is forcing you to buy a computer.
|
| Nobody is forcing you to live under a roof.
|
| Nobody is forcing you to eat.
| smokel wrote:
| Sorry for the harsh comment.
|
| I just found it highly unlikely that Nvidia would change
| its ways due to this, and I don't really see how we're
| being "squeezed". Nvidia are delivering amazing products
| (as are AMD), and it is not going to be any cheaper this
| way.
|
| Building this kind of hardware is not something a hacker
| can do over the weekend.
| Wytwwww wrote:
| Nvidia is charging what they are entirely because there
| is no or very little competition. There isn't much else
| to it, if AMD/Intel caught up Nvidia would suddenly start
| selling their GPUs for way less...
| tomooot wrote:
| The squeeze is mostly within the segmentation of VRAM
| between products, it's basically a commodity and this
| week the spot price for 8GB of GDDR6 has varied from
| $1.30 to $3.50 [1].
|
| Yet to get a card with 8GB more than one with comparable
| logical performance, you'd be looking at hundreds (or
| thousands in the case of "machine learning" cards) of
| dollars.
|
| [1] https://www.dramexchange.com/
| oaththrowaway wrote:
| What are you using P100s or something?
| btbuildem wrote:
| > Once you already bought the NVIDIA cards what's the point
|
| Good luck getting a multi-user GPU setup going, for example.
|
| It super sucks when the hardware is capable, but licensing
| doesn't "allow" it.
| jedberg wrote:
| Step 1: Run on NVIDIA gpus until it works just as well as real
| CUDA.
|
| Step 2: Port to other GPUs.
|
| At least I assume that is the plan.
| chii wrote:
| > Step 2: Port to other GPUs.
|
| why not do this first? because the existing closed sourced
| CUDA already runs well on nvidia chips. Replicating it with
| an open stack, while ideologically useful, is going to sap
| resources away from the porting of it to other GPUs (where
| the real value can be had - by stopping the nvidia monopoly
| on ai chips).
| jfim wrote:
| I'm not involved with the project but I'd assume it's
| helpful as a reference implementation. If there's a bug on
| a non-nvidia GPU, the same test case can be run on Nvidia
| GPUs to see if the bug is in common code or specific to
| that GPU vendor.
| kstenerud wrote:
| I think the point of Linux is to run it on non-Intel CPUs. Once
| you have to buy Intel CPUs what's the point.
| lambdaone wrote:
| You have it exactly backwards. The original goal of Linux was
| to create a Unix-like operating system on Linus Torvald's own
| Intel 80386 PC. Once the original Linux had been created, it
| was then ported to other CPUs. The joy of a portable
| operating system is that you can run it on _any_ CPU,
| _including_ Intel CPUs.
| heavyset_go wrote:
| The closed platform is not without its pitfalls.
| lmpdev wrote:
| The point might not necessarily be for consumers
|
| Linus wasn't writing Linux for consumers (arguably the Linux
| kernel team still isn't), he needed a Unix-like kernel on a
| platform which didn't support it
|
| Nvidia is placed with CUDA in a similar way to how Bell was
| with Unix in the late 1980s. I'm not sure if a legal "CUDA
| Wars" is possible in the way the Unix Wars was, but something
| needs to give
|
| Nvidia has a monopoly and many organisations and projects will
| come about to rectify it, I think this is one example
|
| The most interesting thing to see moving forward is where the
| most just place is to draw the line for Nvidia they deserve
| remuneration for CUDA, but the question is how much? The axe of
| the Leviathan (US government) is slowly swinging towards them,
| and I expect Nvidia to pre-emptively open up CUDA just enough
| to keep them (and most of us) happy
|
| After a certain point for a technology so low in the "stack" of
| the global economy, more powerful actors than Nvidia will have
| to step in and clear the IP bottleneck
|
| Tech giants are powerful and influence people more than the
| government, but I think people forget how powerful the
| government can be when push comes to shove over such an
| important piece of technology
|
| ----------
|
| PS my comparison of CUDA to Unix isn't perfect, mostly as
| Nvidia has a hardware monopoly as it stands, but as they don't
| fab it themselves it's just a design/information at the end of
| the day. There's nothing physically preventing other companies
| producing CUDA hardware, just obvious legal and business
| obstacles
|
| Perhaps a better comparison would be Texas Instruments trying
| to monopolise integrated circuits (they never tried). But if
| Fairchild Semiconductors hadn't've independently discovered
| ICs, we might have seen a much slower logistic curve than we
| have had with Moore's law (assuming competition is proportional
| to innovation)
| talldayo wrote:
| > I expect Nvidia to pre-emptively open up CUDA just enough
| to keep them (and most of us) happy
|
| Besides how they've "opened" their drivers by moving all the
| proprietary code on-GPU, I don't expect this to happen at
| all. Nvidia has no incentive to give away their IP, and the
| antitrust cases that people are trying to build against them
| border on nonsense. Nvidia monopolizes CUDA like Amazon
| monopolizes AWS, their "abuse" is the specialization they
| offer to paying customers... which harms the market how?
|
| What really makes me lament the future is the fact that we
| _had_ a chance to kill CUDA. Khronos wanted OpenCL to be a
| serious competitor, and if it wasn 't _specifically for the
| fact_ that Apple and AMD stopped funding it we might have a
| cross-platform GPU compute layer that outperforms CUDA. Today
| 's Nvidia dominance is a result of the rest of the industry
| neglecting their own GPGPU demand.
|
| Nvidia only "wins" because their adversaries would rather
| fight each other than work together to beat a common
| competitor. It's an expensive lesson for the industry about
| adopting open standards when people ask you to, or you suffer
| the consequences of having nothing competitive.
| jokoon wrote:
| I guess this framework was made by amd engineers.
|
| Anyway I wonder why amd never challenged nvidia on that
| market... It smells a bit like amd and nvidia secretly agreed
| to not compete against each other.
|
| Opencl exists but is abandoned.
| snvzz wrote:
| Moving to HiP on LibreCUDA should probably be the first step for
| projects that are dependent on CUDA to gain platform freedom.
| georgehotz wrote:
| Cool to see one of these in C, particularly if it can be binary
| compatible. Why not s/libreCuInit/cuInit?
|
| If you are interested in open source runtimes, tinygrad has them
| in Python for both AMD and NVIDIA, speaking directly to the
| kernel through ioctls and poking the command queues.
|
| https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru...
|
| https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru...
| ZoomerCretin wrote:
| Incredible! Any plans to support SASS instructions for Nvidia
| GPUs, or only PTX?
| georgehotz wrote:
| We'll get there as we push deeper into assemblies. RDNA3
| probably first, since it's documented and a bit simpler.
| JonChesterfield wrote:
| That's interesting. This looks like you've bypassed the rocm
| userspace stack entirely. I've been looking for justification
| to burn libhsa.so out of the dependency graph for running llvm
| compiled kernels on amdgpu for ages now. I didn't expect roct
| to be similarly easy to drop but that's a clear sketch of how
| to build a statically linked freestanding x64 / gcn blob.
| Excellent.
|
| (I want a reference implementation of run-simple-stuff which
| doesn't fall over because of bugs in libhsa so that I _know_
| whatever bug I 'm looking at is in my compiler / the hardware /
| the firmware)
| georgehotz wrote:
| We didn't just bypass all of ROCm, we bypassed HSA!
|
| The HSA parsing MEC firmware running on the GPUs is riddled
| with bugs, fortunately you can bypass 90% of it using PM4,
| which is pretty much direct sets of the GPU registers. That's
| what tinygrad does.
|
| AMD's software is a really sad state. They don't have
| consumer GPUs in CI, they have no fuzz testing, and instead
| of root causing bugs they seem to just twiddle things until
| the application works.
|
| Between our PM4 backend and disabling CWSR, our AMD GPUs are
| now pretty stable.
| mattiasfestin wrote:
| Could this cause legal issues with tech export bans and
| restrictions in Nvidia's driver, specifically if LibreCUDA
| circumvents those restrictions?
| gnulinux wrote:
| Does it make sense to buy Nvidia GPUs as a linux user in 2024
| anyway? I thought Nvidia has abysmal linux support, if you don't
| have Nvidia GPU what's the point of LibreCUDA?
| jfarina wrote:
| It does if you work in data science/machine learning.
___________________________________________________________________
(page generated 2024-08-09 23:02 UTC)