Post AXzanNyZkMF76gm5om by glyph@mastodon.social
(DIR) More posts by glyph@mastodon.social
(DIR) Post #AXzU9ajEJYtVLkEpo8 by simon@fedi.simonwillison.net
2023-07-23T15:55:48Z
0 likes, 0 repeats
Having secound thoughts about adding Pydantic to my LLM project, for a couple of reasons:- The Pydantic 1 to Pydantic 2 breaking change is ongoing now, and I've already had two instances of LLM plugins with dependencies that were incompatible with Pydantic 2- Pydantic 2 now uses Rust code, which is making support for installation via things like Homebrew significantly more complicatedNot sure if I should sweat through it or switch to an alternative (or roll my own features without it)
(DIR) Post #AXzULsc9do0AISmwZU by simon@fedi.simonwillison.net
2023-07-23T15:57:42Z
0 likes, 0 repeats
My llm-replicate plugin had the problem that the replicate Python library used Pydantic 1 and broke in Pydantic 2I handled that by...- Vendoring a fixed version of the replicate library that worked with Pydantic 2: https://github.com/simonw/llm-replicate/commit/0eb0f932591acb12a835d25ccc2f5dd1f0e2f844- Submitting a PR to replicate to fix that: https://github.com/replicate/replicate-python/pull/121- Un-vendoring the library again: https://github.com/simonw/llm-replicate/commit/2f9867a2acb16b58e0e188a415cd10094eee48be
(DIR) Post #AXzXYVjkJIQoyE3Oee by offby1@wandering.shop
2023-07-23T16:33:36Z
0 likes, 0 repeats
@simon The adoption of rust libraries into Python code has definitely made my use of them more fraught. `cryptography` on its own adds over an hour to the docker image build time for images where I am compiling for ARM platforms to run on my Raspberry Pi, because there are no wheels for those.I love what Alex and Paul are doing with that, and I think that Rust-based modules are the right way to go, but we really need to think about how they impact development and low-cost computing.
(DIR) Post #AXzZ6QnxFoOxTQlEMi by simon@fedi.simonwillison.net
2023-07-23T16:50:55Z
0 likes, 0 repeats
@ash unfortunately it matters for things like Homebrew which refuse to install from binary wheels: https://github.com/simonw/homebrew-llm/issues/2
(DIR) Post #AXzZG7MmNcle5BfVwm by simon@fedi.simonwillison.net
2023-07-23T16:52:26Z
0 likes, 0 repeats
@offby1 I'm willing to put up with the tradeoffs for cryptography because I'm totally sold on the need for a memory safe compiler language to implement those primitive, but Pydantic doesn't feel as justified for me
(DIR) Post #AXzZb2D2kUdOAa5YLA by offby1@wandering.shop
2023-07-23T16:56:11Z
0 likes, 0 repeats
@simon That's probably where I'd land too; if I added a dependency for development QoL and suddenly had to wait an hour longer to build my images... I'm not so sure that's a QoL improvement after all.
(DIR) Post #AXzanNyZkMF76gm5om by glyph@mastodon.social
2023-07-23T17:10:03Z
0 likes, 0 repeats
@simon can Homebrew not use binary wheels from upstream in formulae?
(DIR) Post #AXzdAc8nRq7Hbhl928 by ethanrosenthal@recsys.social
2023-07-23T17:36:13Z
0 likes, 0 repeats
@simon if you switch to an alternative, msgspec looks quite nice and lightweight https://jcristharif.com/msgspec/
(DIR) Post #AXzjICShPBWm0U4LOS by simon@fedi.simonwillison.net
2023-07-23T18:45:18Z
0 likes, 0 repeats
@glyph not as far as I can tell, at least not in official packages - I'm still feeling my way around their ecosystem
(DIR) Post #AXzpgMvKl8k982N25Q by glyph@mastodon.social
2023-07-23T16:56:55Z
0 likes, 0 repeats
@offby1 @simon "ARM platforms" meaning 32-bit ARM platforms? cryptography ships aarch64 linux wheels. I think the problem here is mostly that the work on https://www.piwheels.org needs to be folded upstream, not off in its own little parallel universe.
(DIR) Post #AXzpgONNMMlhdIUwQy by offby1@wandering.shop
2023-07-23T16:58:26Z
0 likes, 0 repeats
@glyph @simon yup. My raspberry pis were installed with a 32-bit userspace, to my enduring dismay, but that's what existed at the time. It's a bit much to expect users of a library to reinstall their OS from scratch, hoping that everything can be replaced with up to date versions, just to speed up build time.Which is not to say I won't eventually do that, but it's a heavy lift.
(DIR) Post #AXzpgSgfKaZj0HYP3Y by glyph@mastodon.social
2023-07-23T17:01:02Z
0 likes, 0 repeats
@offby1 @simon You *could* also pin to 40.0.1 — https://www.piwheels.org/project/cryptography/ — although that is understandably pretty unappealing for something so security-sensitive
(DIR) Post #AXzpgXv1sc2HEHJRdg by glyph@mastodon.social
2023-07-23T17:04:39Z
0 likes, 0 repeats
@offby1 @simon anyway, I know this is not the systemic issue you're pointing at, but given that it's always good to get information where it needs to be in the open source ecosystem, I've filed a piwheels bug: https://github.com/piwheels/packages/issues/378
(DIR) Post #AXzpgdagn9KYovqHlg by simon@fedi.simonwillison.net
2023-07-23T19:56:32Z
0 likes, 0 repeats
@glyph @offby1 good news: looks like someone is trying to solve the issue with cryptography and Homebrew right now: https://github.com/Homebrew/homebrew-core/pull/137355
(DIR) Post #AY0oiLEWFMfIdNLswq by SamJSharpe@mastodon.me.uk
2023-07-24T07:20:28Z
0 likes, 0 repeats
@simon @ash the Homebrew argument is that you should be packaging each llm dependency as a Homebrew package, which can then be installed from bottles. I don't agree with that and in my mind if goes against the OSX way of packaging things as self contained blobs with dependencies embedded.You could distribute llm as a bottle, but then you need a build environment for x86_64 and arm64 and potentially multiple versions of MacOS as it's not supported to install bottles on previous MacOS versions.