[HN Gopher] We shrunk our Javascript monorepo git size
       ___________________________________________________________________
        
       We shrunk our Javascript monorepo git size
        
       Author : kwantaz
       Score  : 296 points
       Date   : 2024-10-27 02:35 UTC (20 hours ago)
        
 (HTM) web link (www.jonathancreamer.com)
 (TXT) w3m dump (www.jonathancreamer.com)
        
       | fragmede wrote:
       | > Large blobs happens when someone accidentally checks in some
       | binary, so, not much you can do
       | 
       | > Retroactively, once the file is there though, it's semi stuck
       | in history.
       | 
       | Arguably, the fix for that is to run filter-branch, remove the
       | offending binary, teach and get everyone setup to use git-lfs for
       | binaries, force push, and help everyone get their workstation to
       | a good place.
       | 
       | Far from ideal, but better than having a large not-even-used file
       | in git.
        
         | larusso wrote:
         | The main issue is not a binary file that never changes. It's
         | the small binary file that changes often.
        
         | abound wrote:
         | There's also BFG (https://rtyley.github.io/bfg-repo-cleaner/)
         | for people like me who are scared of filter-branch.
         | 
         | As someone else noted, this is about small, frequently changing
         | files, so you could remove old versions from the history to
         | save space, and use LFS going forward.
        
         | lastdong wrote:
         | It's easier to blame Linus.
        
         | cocok wrote:
         | filter-repo is the recommended way these days:
         | 
         | https://github.com/newren/git-filter-repo
        
       | killingtime74 wrote:
       | Shrank
        
         | Sparkyte wrote:
         | Shrinky dinky
        
           | bubblesnort wrote:
           | Honey, I shrunk the git!
        
         | dougthesnails wrote:
         | I think I prefer shrunked in this context.
        
         | peutetre wrote:
         | I was in the pool!
        
         | tankenmate wrote:
         | Would be correct if it is "We shrank", but from my poor memory
         | of the terminology that is the transitive form, shrunken is the
         | intransitive form. But once again from my poor memory.
        
         | darraghenright wrote:
         | I've spoken English as my native language for almost five
         | decades and I've never seen/heard the word "shranked" before.
         | 
         | This surely cannot be correct. Even the title of the linked
         | article doesn't use "shranked". What?
        
           | forgotpwd16 wrote:
           | Commonly (since ca. 19th century), _shrank_ is used as the
           | past tense of shrink, _shrunk_ as the past particle, and
           | _shrunken_ as an adjective. The title of the linked article
           | uses  "shrunk" as past tense and the submitted title was
           | changed to "shrunked" for some reason. "Shranked" was not
           | mentioned anywhere. (But "shrinked" has had some use in the
           | past.)
        
       | eviks wrote:
       | upd: silly mistake - file _name_ does not include its _full path_
       | 
       | The explanation probably got lost among all the gifs, but the
       | last 16 chars here are different:
       | 
       | > was actually only checking the last 16 characters of a filename
       | > For example, if you changed repo/packages/foo/CHANGELOG.md,
       | when git was getting ready to do the push, it was generating a
       | diff against repo/packages/bar/CHANGELOG.md!
        
         | derriz wrote:
         | I wish they had provided an actual explanation of what exactly
         | was happening and skipped all the "color" in the story. By
         | filename do they mean path? Or is it that git will just pick
         | any file with a matching name to generate a diff? Is there any
         | pattern to the choice of other file to use?
        
           | snthpy wrote:
           | +1
        
         | p4bl0 wrote:
         | I was also bugged by that. I imagine that the meta variables
         | foo and bar are at fault here, and that probably the actual
         | package names had a common suffix like firstPkg and secondPkg.
         | A common suffix of length three is enough in this case to get
         | 16 chars in common as "/CHANGELOG.md" is already 13 chars long.
        
         | daenney wrote:
         | File name doesn't necessarily include the whole path. The last
         | 16 characters of CHANGELOG.md is the full file name.
         | 
         | If we interpret it that way, that also explains why the
         | filepathwalk solution solves the problem.
         | 
         | But if it's really based on the last 16 characters of just the
         | file name, not the whole path, then it feels like this problem
         | should be a lot more common. At least in monorepos.
        
           | floam wrote:
           | It did shrink Chromium's repo quite a bit!
        
           | eviks wrote:
           | yes, this makes sense, thanks for pointing it out, silly
           | confusion on my part
        
         | tux3 wrote:
         | Derrick provides a better explanation in this cover letter:
         | https://lore.kernel.org/git/pull.1785.git.1725890210.gitgitg...
         | 
         | (See also the path-walk API cover letter: https://lore.kernel.o
         | rg/all/pull.1786.git.1725935335.gitgitg...)
         | 
         | The example in the blog post isn't super clear, but Git was
         | essentially taking all the versions of all the files in the
         | repo, putting the last 16 bytes of the path (not filename) in a
         | hash table, and using that to group what they expected to be
         | different versions of the same file together for delta
         | compression.
         | 
         | Indeed in the blog it doesn't work, because foo/CHANGELOG.md
         | and bar/CHANGELOG.md is only 13 chars, but you have to imagine
         | the paths have a longer common suffix. That part is fixed by
         | the --full-name-hash option, now you compare the full path
         | instead of just 16 bytes.
         | 
         | Then they talk about increasing the window size. That's kind of
         | a hack to workaround bad file grouping, but it's not the real
         | fix. You're still giving terrible inputs to the compressor and
         | working around it by consuming huge amounts of memory. So that
         | was a bit confusing to present it as the solution. The path
         | walk API and/or --full-name-hash are the real interesting parts
         | here =)
        
           | lastdong wrote:
           | Thank you! I ended up having to look at the PR to make any
           | sense of the blog post, but your explanation and links makes
           | things much clearer
        
         | js2 wrote:
         | > file name does not include its full path
         | 
         | No, it is the full path that's considered. Look at the commit
         | message on the first commit in the `--full-name-hash` PR:
         | 
         | https://github.com/git-for-windows/git/pull/5157/commits/d5c...
         | 
         | Excerpt: _" /CHANGELOG.json" is 15 characters, and is created
         | by the beachball [1] tool. Only the final character of the
         | parent directory can differntiate different versions of this
         | file, but also only the two most-significant digits. If that
         | character is a letter, then this is always a collision. Similar
         | issues occur with the similar "/CHANGELOG.md" path, though
         | there is more opportunity for differences in the parent
         | directory._
         | 
         | The grouping algorithm puts less weight on each character the
         | further it is from the right-side of the name:
         | hash = (hash >> 2) + (c << 24)
         | 
         | Hash is 32-bits. Each 8-bit char (from the full path) in turn
         | is added to the 8-most significant bits of hash, after shifting
         | any previous hash bits to the right by two bits (which is why
         | only the final 16 chars affect the final hash). Look at what
         | happens in practice:
         | 
         | https://go.dev/play/p/JQpdUGXdQs7
         | 
         | Here I've translated it to Go and compared the final value of
         | "aaa/CHANGELOG.md" to "zzz/CHANGELOG.md". Plug in various
         | values for "aaa" and "zzz" and see how little they influence
         | the final value.
        
       | triyambakam wrote:
       | > we have folks in Europe that can't even clone the repo due to
       | it's size.
       | 
       | What is it about Europe that makes it more difficult? That
       | internet in Europe isn't as good? Actually, I have heard that
       | some primary schools in Europe lack internet. My grandson's
       | elementary school in rural California (population <10k) had
       | internet as far back as 1998.
        
         | RadiozRadioz wrote:
         | At least here in Western Europe, in general the internet is
         | great. Though coverage in rural areas varies by country.
        
         | gnrlst wrote:
         | In most EU countries we have multi-gigabit internet (for cheap
         | too). Current offers are around ~5 GBIT speeds for 20 bucks a
         | month.
        
           | n_ary wrote:
           | Well good for you. On my side of europe, I pay EUR50/- for a
           | cheap 50Mbps(1 month cancellation notice period). I could get
           | a slightly cheaper 100Mbps from a predator for EUR20/- for
           | first 6 month but then it goes up to EUR50/- and they pull bs
           | about not being able to cancel if you even move because your
           | new location is also in their coverage area(over garbage
           | copper) and suffers at least 20 outages per month while there
           | are other providers with much cheaper rates and better
           | service.
           | 
           | Some EU is still suffering from Telekom copper barons.
        
           | jillesvangurp wrote:
           | Sadly, I'm in Germany. Which is a third world country when it
           | comes to decent connectivity. They are rolling out some fiber
           | now in Berlin. Finally. But very slowly and not to my
           | building any time soon. Most of the country is limited to DSL
           | speeds. Mobile coverage is getting better but still non
           | existent outside of cities. Germany has borders with nine
           | countries. Each of those have better connectivity than
           | Germany.
           | 
           | I'm from the Netherlands where over 90% of households now
           | have fiber connections, for example. Here in Berlin it's very
           | hard to get that. They are starting to roll it out in some
           | areas but it's taking very long and each building has to then
           | get connected, which is up to the building owners.
        
             | barrkel wrote:
             | When I travel in Germany I use a Deutsche Telekom pay as
             | you go SIM in a 5G hotspot, and generally get about 200Mbit
             | throughtput, which is far higher than you can expect any
             | place you're staying to provide. It's EUR7 a day (or EUR100
             | a month) but it's worth it to avoid the terrible internet.
        
               | zelphirkalt wrote:
               | Oh, that is an incentive for them not to improve
               | anything. Wouldn't want customers to stop purchasing
               | mobile Internet for 100 Euro a month.
        
             | aniviacat wrote:
             | > Mobile coverage is getting better but still non existent
             | outside of cities.
             | 
             | According to the Bundesnetzagentur over 90% [1] of Germany
             | has 5G coverage (and almost all of the rest has 4G [2]).
             | 
             | [1] https://www.bundesnetzagentur.de/SharedDocs/Pressemitte
             | ilung...
             | 
             | [2] https://gigabitgrundbuch.bund.de/GIGA/DE/MobilfunkMonit
             | oring...
        
               | jillesvangurp wrote:
               | I usually lose connectivity on train journeys across
               | Germany. I'm offline most of the way. Even the in train
               | wifi gets quite bad in remote areas. Because they depend
               | on the same shitty mobile networks. There's a stark
               | difference as soon as you cross the borders with other
               | countries. Suddenly stuff works again. Things stop timing
               | out.
               | 
               | I also deal with commercial customers that have companies
               | in areas with either no or poor mobile connectivity and
               | since we sell mobile apps to them, we always need to
               | double check they actually have a good connection. One of
               | our customers is on the edge of a city with very spotty
               | 4G at best. I recently recommended Star Link to another
               | company that is operating in rural areas. They were
               | asking about offline capabilities of our app. Because
               | they deal with poor connectivity all the time. I made the
               | point that you can get internet anywhere you want now for
               | a fairly reasonable price.
        
               | holowoodman wrote:
               | Those statistics are a half-truth at best.
               | 
               | The "coverage" they are reporting is not by area but by
               | population. So all the villages and fields that the train
               | or autobahn goes by won't have 5G, because they are in
               | the other 10% because of their very low population
               | density.
               | 
               | And the reporting comes out of the mobile phone
               | operators' reports and simulations (they don't have to do
               | actual measurements). Since their license depends on
               | meeting a coverage goal, massive over-reporting is
               | rampant. The biggest provider (Deutsche Telekom) is also
               | partially state-owned, so the regulators don't look as
               | closely...
               | 
               | Edit: accidentially posted this in the wrong comment:
               | Then there is the problem of "5G reception" vs. "5G
               | reception with usable bandwidth". A lot of overbooking
               | goes on, many cells don't have sufficient capacity
               | allocated, so there are reports of 4G actually being
               | faster in many places.
               | 
               | And also, yes, you can get 5G in a lot of actually
               | populated areas. But you certainly will pay through the
               | nose for that, usually you get a low-GB amount of traffic
               | included, so maybe a tenth of the Microsoft monorepo in
               | question. The rest is pay-10Eur-per-GB or something.
        
           | badgersnake wrote:
           | Not in the UK. Still on 80Mbit VDSL here.
        
             | sirsinsalot wrote:
             | I and many I know have Gb fiber in the UK
        
             | _joel wrote:
             | You must be unlucky, according to Openreach "fibre
             | broadband is already available in more than 96.59 per cent
             | of the UK."
        
               | badgersnake wrote:
               | Coming next year apparently. I won't hold my breath.
        
               | mattlondon wrote:
               | Is that "fibre" or "full fibre".
               | 
               | They lied a lot for a good few years saying "OMG fibre
               | broadband!" When in reality is was still copper for the
               | last mile so that "fibre" connection in reality was some
               | ADSL variant and limited to 80/20mpbs.
               | 
               | Actual full fibre all the way from your home to the
               | internet is I think still quite a way behind. Even in
               | London (London! The capital city with high density) there
               | are places where there are no full fibre options.
        
               | _joel wrote:
               | Maybe myself and my friends are lucky as we're all on
               | ftth
        
               | mattlondon wrote:
               | Only a few I know are on ftth. I guess I live in a fairly
               | affluent area in Zone 3 which is lower density than
               | average - zero flats etc, all just individual houses so
               | perhaps not worth their effort rolling out
        
               | Deathmax wrote:
               | According to ThinkBroadband's tracking [1], the headline
               | figures are 85.20% of premises are gigabit capable
               | (FTTP/FTTH/Cable [DOCSIS]) with 71.86% being full fibre.
               | 
               | [1]:
               | https://www.thinkbroadband.com/news/10343-85-gigabit-
               | coverag...
        
         | yashap wrote:
         | They're probably downloading from a server in the states, being
         | much further away makes a big difference with a massive
         | download.
        
         | _kidlike wrote:
         | Let's pretend you didn't write the last 2 sentences...
         | 
         | first of all "internet in Europe" makes close to zero sense to
         | argue about. The article just uses it as a shortcut to not
         | start listing countries.
         | 
         | I live in a country where I have 10Gbps full-duplex and I pay
         | 50$ / month, in "Europe".
         | 
         | The issue is that some countries have telecom lobbies which are
         | still milking their copper networks. Then the "competition
         | committees" in most of these countries are actually working
         | AGAINST the benefit of the public, because they don't allow 1
         | single company to start offering fiber, because that would be a
         | competition advantage. So the whole system is kinda in a
         | deadlock. In order to unblock, at least 2 telecoms have to
         | agree to release fiber deals together. It has happened in some
         | countries.
        
           | 0points wrote:
           | What european countries still dont have fiber?
           | 
           | //Confused swede with 10G fiber all over the place. Writing
           | from literally the countryside next to nowhere.
        
             | SSLy wrote:
             | Germany, GP's situation smells like their policies.
        
             | holowoodman wrote:
             | Germany.
             | 
             | Deutsche Telekom is the former monopoly that was half-
             | privatized around 1995 or something. The state still owns
             | quite a large stake of it.
             | 
             | They milk their ancient copper crap for everything they can
             | while keeping prices high.
             | 
             | They are refusing useful backbone interconnects to
             | monopolize access to their customers (Actually they are not
             | allowed to refuse. They just offer interconnections only in
             | their data centers in the middle of nowhere, where you need
             | to rent their (outrageously priced) rackspace and fibres
             | because there is nothing else. They are refusing for
             | decades to do anything useful at the big exchanges like
             | DECIX).
             | 
             | And if there should ever be a small competitor that on
             | their own tries to lay fibre somewhere, they quickly lay
             | their own fibre into the open ditches (they are allowed to
             | do that) and offer just enough rebates for their former
             | copper customers to switch to their fibre that the
             | competitor cannot recoup the invest and goes bankrupt.
             | Since that dance is now known to everyone, even the
             | announcement of Telekom laying their own fibres kills the
             | competitors' projects there. So after a competitor's
             | announcement of fibre rollout, Telekom does the same,
             | project dead, no fibre rollout at all.
             | 
             | Oh, and since it is a partially-state-owned former
             | monopoly/ministry, the state and competition authorities
             | turn a blind eye to all that, when not actively promoting
             | them...
             | 
             | Then there is the problem of "5G reception" vs. "5G
             | reception with usable bandwidth". A lot of overbooking goes
             | on, many cells don't have sufficient capacity allocated, so
             | there are reports of 4G actually being faster in many
             | places.
             | 
             | And also, yes, you can get 5G in a lot of actually
             | populated areas. But you certainly will pay through the
             | nose for that, usually you get a low-GB amount of traffic
             | included, so maybe a tenth of the Microsoft monorepo in
             | question. The rest is pay-10Eur-per-GB or something.
        
               | ahartmetz wrote:
               | It is almost as bad as you say, except that I recently
               | noticed several instances of competitors offering cheaper
               | fiber than Telekom and surviving. Still, overall fiber
               | buildout is low, like... I looked it up, reportedly 36%
               | now.
        
               | immibis wrote:
               | Wait, I live in that area. Does that mean I'm allowed to
               | lay my own fiber into their open ditches too, or do they
               | have special rights no one else has?
        
               | holowoodman wrote:
               | Afaik the special right is granted to everyone providing
               | fibre services to the public to be informed about any
               | ditches on public ground being dug and getting the
               | opportunity to throw their fibre in before the ditch is
               | closed again.
        
             | zelphirkalt wrote:
             | If you really need it pointed out, take it from a German
             | neighbor: Telekom is running some extortion scheme or so
             | here. Oh we could have gotten fiber to our house already
             | ... if we paid them 800+ Euro! So we rather stick with our
             | 100MBits or so connection that is not fiber but copper. If
             | the German state does not intervene here, or the practices
             | of ISPs and whoever has the power to build fiber changes,
             | we will for the foreseeable future still be on copper.
             | 
             | Then there are villages, which were promised fiber
             | connections, but somehow after switching to the fiber
             | connection made them have unstable Internet and ofter no
             | Internet. Saw some documentary about that, could be fixed
             | by now.
             | 
             | Putting fiber into the ground also requires a whole lot of
             | effort opening up roads and replacing what's there. Those
             | costs they try to push to the consumers with their 800+
             | Euro extortion scheme.
             | 
             | But to be honest, I am also OK with my current connection.
             | All I worry about is it being stable, no package loss, and
             | no ping spikes. A consistently good connection stability is
             | more important than throughout. Sadly, I cannot buy any of
             | those guarantees from any ISP.
        
               | singron wrote:
               | Is 800 euros that bad? In the US, we were quoted $10k a
               | few years back. Even if fiber is already at the road,
               | $800 is probably a fair price just to trench the line
               | from the road to your home and install an entry point. If
               | they provide free installation, then they have to make up
               | the cost by raising your rates.
        
               | zelphirkalt wrote:
               | I think private households paying 800 Euro for what
               | should be public infrastructure, being milked by ISPs is
               | pretty bad.
        
             | ahoka wrote:
             | I pay 42USD for 250Mbit in a larger Swedish city. What is
             | that magic ISP I should be using?
        
         | nyanpasu64 wrote:
         | I've experienced interruptions mid-clone (with no apparent way
         | to resume them) when trying to clone repos on unreliable
         | connections, and perhaps a similar issue is happening with
         | connections between continents.
        
           | joshvm wrote:
           | The only reliable route I've found is to use SSH clone. HTTPS
           | is lousy and as you mention, is not resumable. Works fine in
           | Antarctica even over our slower satellite. Doesn't help if
           | you actually drop, but you can clone to a remote and then
           | rsync everything over time.
        
         | p_l wrote:
         | It's issues cloning super huge repo over crappy protocols
         | across ocean especially when VPNs get included in the problem
        
         | teo_zero wrote:
         | > > we have folks in Europe that can't even clone the repo due
         | to it's size.
         | 
         | > I have heard that some primary schools in Europe lack
         | internet.
         | 
         | Maybe they lack internet but teach their pupils how to write
         | "its".
        
         | nemetroid wrote:
         | The problem is probably that the repo is not hosted in Europe.
        
         | johnisgood wrote:
         | Some countries in Europe (even Poland) definitely offer faster
         | Internet and for cheaper than the US, and without most of the
         | privacy issues that US ISPs have.
        
         | heisenbit wrote:
         | Effective bandwidth can be influenced by roundtrip time. Fewer
         | IP4 numbers means more NAT with more delay and yet another
         | point where occasionally something can go wrong. Last but not
         | least there are some areas in the EU like the Canary Islands
         | where the internet feels like going over a sat.
        
         | mattlondon wrote:
         | I was not sure what this meant either. I know personally I have
         | downloaded and uploaded some very very large files
         | transatlantic (e.g. syncing to cloud storage) with absolutely
         | no issues, so not sure what they are talking about. I guess
         | perhaps there are issues with _git_ cloning such a large amount
         | of data, but that is a problem with git and not the
         | infrastructure.
         | 
         | FWIW every school I've seen (and I recently toured a bunch
         | looking at them for my kids to start at) all had the internet
         | and the kids were using iPads etc for various things.
         | 
         | Anecdotally my secondary school (11-18y in UK) in rural
         | Hertfordshire was online in the 1995 region. It was via I think
         | a 14.4 modem and there actually wasn't that much useful
         | material for kids then to be honest. I remember looking at the
         | "non-professional style" NASA website for instance (the current
         | one is obviously quite fancy in comparison, but it used to be
         | very rustic and at some obscure domain). CD-based encyclopedias
         | we're all the rage instead around that time IIRC - Encarta et
         | al.
        
         | 59nadir wrote:
         | Most european countries have connections with more bandwith and
         | less base latency for cheaper than the US, it's not a
         | connection issue. If there was an issue it's that the repo
         | itself is hosted on the other side of the world, but even so
         | the sidenote itself is odd.
        
           | tom_ wrote:
           | I wouldn't say it's odd at all - it's basically what's
           | justifying actually trying to solve the problem rather than
           | just going "huh... that's weird..." then putting it on the
           | backlog due to it not being a showstopper.
           | 
           | This sort of thing has been a problem on every project I've
           | worked on that's involved people in America. (I'm in the UK.)
           | Throughput is inconsistent, latency is inconsistent, and
           | long-running downloads aren't reliable. Perhaps I'm over-
           | simplifying, but I always figured the problem was fairly
           | obvious: it's a lot of miles from America to Europe, west
           | coast America especially, and a lot of them are underwater,
           | and your're sharing the conduit with everybody else in
           | Europe. Many ways for packets to get lost (or get held up
           | long enough to count), and frankly it's quite surprising more
           | of them don't.
           | 
           | (Usual thing for Perforce is to leave it running
           | overnight/weekend with a retry count of 1 million. I'm not
           | sure what you'd do with Git, though? it seems to do the whole
           | transfer as one big non-retryable lump. There must be
           | something though.)
        
         | o11c wrote:
         | My knowledge is a bit outdated, but we used to say:
         | 
         | * in America, peering between ISPs is great, but the last-mile
         | connection is terrible
         | 
         | * In Europe, the last-mile connection is great, but peering
         | between the ISPs is terrible (ISPs are at war with each other).
         | Often you could massively improve performance by renting a VPS
         | in the correct city and routing your traffic manually.
        
       | yunusabd wrote:
       | > For many reasons, that's just too big, we have folks in Europe
       | that can't even clone the repo due to it's size.
       | 
       | What's up with folks in Europe that they can't clone a big repo,
       | but others can? Also it sounds like they still won't be able to
       | clone, until the change is implemented on the server side?
       | 
       | > This meant we were in many occasions just pushing the entire
       | file again and again, which could be 10s of MBs per file in some
       | cases, and you can imagine in a repo
       | 
       | The sentence seems to be cut off.
       | 
       | Also, the gifs are incredibly distracting while trying to read
       | the article, and they are there even in reader mode.
        
         | anon-3988 wrote:
         | > For many reasons, that's just too big, we have folks in
         | Europe that can't even clone the repo due to it's size.
         | 
         | I read that as an anecdote, a more complete sentence would be
         | "We had a story where someone from Europe couldn't clone the
         | whole repo on his laptop for him to use on a journey across
         | Europe because his disk is full at the time. He has since
         | cleared up the disk and able to clone the repo".
         | 
         | I don't think it points to a larger issue with Europe not being
         | able to handle 180GB files...I surely hope so.
        
           | peebeebee wrote:
           | The European Union doesn't like when a file get too big and
           | powerful. It needs to be broken apart in order to give
           | smaller files a chance of success.
        
             | _joel wrote:
             | People foolishly thought the G in GDPR stood for "general"
             | when it's actually GIANT.
        
             | wizzwizz4 wrote:
             | Ever since they enshrined the Unix Philosophy into law,
             | it's been touch-and-go for monorepotic corporations.
        
         | thrance wrote:
         | The repo is probably hosted on the west coast, meaning it has
         | to cross the Atlantic whenever you clone it from Europe?
        
         | tazjin wrote:
         | > What's up with folks in Europe that they can't clone a big
         | repo, but others can?
         | 
         | They might be in a country with underdeveloped internet
         | infrastructure, e.g. Germany))
        
           | avianlyric wrote:
           | I do t think there's any country in Europe with internet
           | infrastructure as underdeveloped as the US. Most of Europe
           | has fibre-to-the-premise, and all of Europe has consumer
           | internet packages that are faster and cheaper than you're
           | gonna find anywhere in the U.S.
        
             | tazjin wrote:
             | There's (almost) no FTTH in Germany. The US used to be as
             | bad as Germany, but it has improved significantly and is
             | actually pretty decent these days (though connection speed
             | is unevenly distributed).
             | 
             | Both countries are behind e.g. Sweden or Russia, but
             | Germany by a much larger margin.
             | 
             | There's some trickery done in official statistics (e.g. by
             | factoring in private connections that are unavailable to
             | consumers) to make this seem better than it is, but ask
             | anyone who lives there and you'll be surprised.
        
         | acdha wrote:
         | My guess is that "Europe" is being used as a proxy for "high
         | latency, low bandwidth" - especially if the person in question
         | uses a VPN (especially one of those terrible "SSL VPN"
         | kludges). It's still surprisingly common to encounter software
         | with poor latency handling or servers with broken window
         | scaling because most of the people who work on them are
         | relatively close and have high bandwidth connection.
        
           | jerf wrote:
           | And given the way of internal corporate networks, probably
           | also "high failure rate", not because of "the internet", but
           | the pile of corporate infrastructure needed for auditability,
           | logging, security access control, intrusion detection, maxed
           | out internal links... it's amazing any of this ever
           | functions.
        
             | acdha wrote:
             | Or simply how those multiply latency - I've seen enterprise
             | IT dudes try to say 300ms LAN latency is good because
             | nobody wants to troubleshoot their twisted mess of network
             | appliances and it's not technically down if you're not
             | getting an error...
             | 
             | (Bonus game: count the number of annual zero days they're
             | exposed to because each of those vendors still ships
             | 90s-style C code)
        
           | sroussey wrote:
           | Or high packet loss.
           | 
           | Every once in a while, my router used to go crazy with
           | seemingly packet loss (I think a memory issue).
           | 
           | Normal websites would become super slow for any pc or phone
           | in the house.
           | 
           | But git... git would fail to clone anything not really small.
           | 
           | My fix was to unplug the modem and router and plug back in.
           | :)
           | 
           | It took a long time to discover the router was reporting
           | packet loss, and that the slowness the browsers were
           | experiencing has to do with some retries, and that git just
           | crapped out.
           | 
           | Eventually when git started misbehaving I restarted the
           | router to fix.
           | 
           | And now I have a new router. :)
        
           | hinkley wrote:
           | Sounds, based on other responders, like high latency high
           | bandwidth, which is a problem many of us have trouble
           | wrapping our heads around. Maybe complicated by packet loss.
           | 
           | After COVID I had to set up a compressing proxy for
           | Artifactory and file a bug with JFrog about it because some
           | of my coworkers with packet loss were getting request
           | timeouts that npm didn't handle well at all. Npm of that era
           | didn't bother to check bytes received versus content-length
           | and then would cache the wrong answer. One of my many, many
           | complaints about what total garbage npm was prior to ~8 when
           | the refactoring work first started paying dividends.
        
       | bubblesnort wrote:
       | > We work in a very large Javascript monorepo at Microsoft we
       | colloquially call 1JS.
       | 
       | I used to call it office.com.. Teams is the worst offender there.
       | Even a website with a cryptominer on it runs faster than that
       | junk.
        
         | wodenokoto wrote:
         | We were all impressed with google docs, but office.com is way
         | more impressive.
         | 
         | Collaborative editing between a web app, two mobile anpps and a
         | desktop app with 30 years of backwards compatibility and it
         | pretty much just works. No wonder that took a lot of
         | JavaScript!
        
           | tinco wrote:
           | To be fair, we were impressed with Google Docs 15 years ago.
           | Not saying office.com isn't impressive, but Google Docs
           | certainly isn't impressive today. My company still uses
           | GSuite, as I don't like being in Microsoft's ecosystem and we
           | don't need any advanced features of our office suite but
           | Google Docs and the rest of the GSuite seem to be
           | intentionally held back to technology of the early 2010's.
        
             | alexanderchr wrote:
             | Google docs certainly haven't changed much the last 5-10
             | years. I wonder if that's an intentional choice, or if it
             | is because those that built it and understand how it works
             | are long gone to work on other things.
        
               | jakub_g wrote:
               | Actually I did see a few long awaited improvements
               | landing in gdocs lately (e.g. better markdown support,
               | pageless mode).
               | 
               | I think they didn't deliver much new features in early
               | 2020s because they were busy with a big refactoring from
               | DOM to canvas rendering [0].
               | 
               | [0] https://news.ycombinator.com/item?id=27129858
        
               | sexy_seedbox wrote:
               | No more development? Time for Google to kill Google Docs!
        
           | esperent wrote:
           | We use MS Teams at my company. The Word and Excel in the
           | Windows Teams app are so buggy that I can almost never
           | successfully open a file. It just times out and eventually
           | shows a "please try again later" message nearly every time.
           | I've uninstalled and reinstalled the Teams app four or five
           | times trying to fix this.
           | 
           | We've totally given up any kind of collaborative document
           | editing because it's too frustrating, or we use Notion
           | instead, which for all it's fault, at least the basic stuff
           | like loading a bloody file works...
        
             | acdha wrote:
             | This is specific to your company's configuration - likely
             | something related to EDR or firewall policies.
        
               | bubblesnort wrote:
               | Really it's anyone using teams on older or cheaper
               | hardware.
        
               | acdha wrote:
               | So you've tested this with clean installs on unfiltered
               | networks? Just how old is your hardware? It works well
               | on, say, the devices they issue students here so I'm
               | guessing it'd have to be extremely old.
        
               | esperent wrote:
               | I'm the one who set it up. It's a small team of 20
               | people. I've done basically no setup beyond the minimum
               | of following docs to get things running. We've had
               | nonstop problems like this since the very start. Files
               | don't upload, anytime I try to fix it I'm confronted with
               | confusing error messages and cryptic things like people
               | telling me "something related to EDR". What the hell is
               | EDR? I just want to view a Word doc.
               | 
               | I've come to realize that Teams should only be used in
               | large companies who can afford dedicated staff to manage
               | it. But it was certainly sold to us as being easy to use
               | and suitable for a small company.
        
               | acdha wrote:
               | EDR: https://en.wikipedia.org/wiki/Endpoint_detection_and
               | _respons...
               | 
               | I mentioned that because security software blocking
               | things locally or at the network level is such a common
               | source of friction. I don't think Teams is perfect by any
               | means but the core functionality has been quite stable in
               | personal use, both of my wife's schools, and my
               | professional use so I wouldn't conclude that it's
               | hopeless and always like that.
        
           | ezst wrote:
           | That's the thing, though, the compat story is terrible. I
           | can't say much about the backwards one, but Microsoft has
           | started the process of removing features from the native
           | versions just to lower the bar for the web one catching up.
           | Even my most Microsoft-enamoured colleagues are getting
           | annoyed by this (and the state of all-MS things going
           | downhill, but that's another story)
        
             | lostlogin wrote:
             | > That's the thing, though, the compat story is terrible.
             | 
             | It really is. With shared documents you just have to give
             | up. If someone edits them on the web, in Teams, in the
             | actual app or some other way like on iOS, it all goes to
             | hell.
             | 
             | Pages get added or removed, images jump about, fonts change
             | and various other horrors occur.
             | 
             | If you care, you'll get ground into the earth.
        
           | matrss wrote:
           | > [...] and it pretty much just works.
           | 
           | I beg to differ. Last time I had to use PowerPoint (granted,
           | that was ~3 years ago), math on the slides broke when you
           | touched it with a client that wasn't of the same type as the
           | one that initially put it there. So you would need to use
           | either the web app or the desktop app to edit it, but you
           | couldn't switch between them. Since we were working on the
           | slides with multiple people you also never knew what you had
           | to use if someone else wrote that part initially.
        
             | hu3 wrote:
             | could it be a font issue?
        
               | matrss wrote:
               | If I remember correctly I had created the math parts with
               | the windows PowerPoint app and it was shown more or less
               | correctly in the web app, until I double clicked on it
               | and it completely broke; something like it being a
               | singular element that wasn't editable at all when it
               | should have been a longer expression, I don't remember
               | the details. But I am pretty sure it wasn't just a font
               | issue.
        
           | Cthulhu_ wrote:
           | > No wonder that took a lot of JavaScript!
           | 
           | To the point where they quickly found the flaws in JS for
           | large codebases and came up with Typescript. I think. It
           | makes sense that TS came out of the office for web project.
        
           | coliveira wrote:
           | You must be kidding about "just works". There are so many
           | bugs in word and excel that you could spend the rest of your
           | life fixing. And the performance is disastrous.
        
           | fulafel wrote:
           | What's impressive is that MS has such well trained customers
           | that it can get away with extremely buggy and broken web
           | apps. Fundamental brokenness like collaborative editing
           | frequently losing data and thousand cuts of the more mundane
           | bugs.
        
         | inglor wrote:
         | Hey, I worked with Jonathan on 1JS a while ago (on a team,
         | Excel).
         | 
         | Just a note OMR (the office monorepo) is a different (and
         | actually much larger) monorepo than 1JS (which is big on its
         | own)
         | 
         | To be fair I suspect a lot of the bloat in both originates from
         | the amount of home grown tooling.
        
           | IshKebab wrote:
           | I thought Microsoft had one monorepo. Isn't that kind of the
           | point? How many do they have?
        
             | lbriner wrote:
             | The point of a monorepo is that all the dependencies for a
             | suite of related products are all in a single repo, not
             | that everything your company produces is in a single repo.
        
               | cjpearson wrote:
               | Most people use the "suite of related products"
               | definition of monorepo, but some companies like Google
               | and Meta have a single company-wide repository. It's
               | unfortunate that the two distinct strategies have the
               | same name.
        
         | coliveira wrote:
         | Teams is the running version of that repository... It is hard
         | for them even to store on git.
        
       | issung wrote:
       | Having someone in arms reach to help out that knows the inner
       | workings of Git so much must be a lovely perk of working on such
       | projects at companies of this scale.
        
         | jonathanlydall wrote:
         | Certainly being in an org which has close ties to entities like
         | GitHub helps, but any team in any org with that number of
         | developers can justify the cost of bringing in a highly
         | specialized consultant to solve an almost niche problem like
         | this.
        
       | jimjimjim wrote:
       | Did anybody else shudder at "Shrunked"?
        
         | 0points wrote:
         | English is my third language, also yes.
        
         | amsterdorn wrote:
         | Honey, I done shrunked them kids
        
         | tankenmate wrote:
         | Shrunken, shrunked ain't no language I ever heard of.
        
       | snthpy wrote:
       | Thanks for this post. Really interesting and a great win for OSS!
       | 
       | I've been watching all the recent GitMerge talks put up by
       | GitButler and following the monorepo / scaling developments -
       | lots of great things being put out there by Microsoft, Github,
       | and Gitlab.
       | 
       | I'd like to understand this last 16 char vs full path check issue
       | better. How does this fit in with delta compression, pack
       | indexes, multi-pack indexes etc ... ?
        
         | _joel wrote:
         | > Really interesting and a great win for OSS!
         | 
         | Are they going to be opening a merge request to get their
         | custom git command back in git proper then?
        
           | acdha wrote:
           | It appears so: https://lore.kernel.org/git/pull.1785.git.1725
           | 890210.gitgitg...
        
       | tux3 wrote:
       | For those wondering where this new git-survey command is, it's
       | actually not in git.git yet!
       | 
       | The author is using microsoft's git fork, they've added this new
       | command just this summer:
       | https://github.com/microsoft/git/pull/667
        
         | masklinn wrote:
         | I assume full-name-hash and path-walk are also only in the fork
         | as well (or in git HEAD)? Can't see them in the man pages, or
         | in the 2.47 changelog.
        
           | tux3 wrote:
           | Yep. Path-walk is currently pending review here: https://lore
           | .kernel.org/all/pull.1813.git.1728396723.gitgitg...
           | 
           | It more or less replaces the --full-name-hash option (again a
           | very good cover letter that explains the differences and
           | pros/cons of each very well!)
        
         | clktmr wrote:
         | This is your daily Embrace, Extend, Extinguish reminder.
        
           | throwuxiytayq wrote:
           | Can you elaborate how exactly git is at risk here? These
           | posts never do.
        
             | coliveira wrote:
             | They will extend git so that it works extremely well with
             | their proprietary products, and just average with other
             | tools and operating systems. That's always the goal for MS.
        
               | wbl wrote:
               | You know who the maintainer of Git is right?
        
           | PoignardAzur wrote:
           | Oh for crying out loud.
           | 
           | "EEE" isn't a magic incantation, it's the name of an actual
           | policy with actual tangible steps that their executives were
           | implementing back when the CEO thought open source was the
           | greatest threat to their business model.
           | 
           | Microsoft contributing to a project doesn't automatically
           | make it EEE. For one thing, EEE was about adopting open
           | standards in _proprietary_ software. Microsoft during EEE
           | didn 't publish GPL code like this is.
        
             | haolez wrote:
             | Well, most of their extensions to VSCode are proprietary.
             | When their dominance in software development becomes
             | irreversible, it's obvious that they will close things down
             | and create new sources of income. The incentives are clear.
        
               | madeofpalk wrote:
               | What has VS Code got to do with any of this?
        
               | mb7733 wrote:
               | VSCode is _their_ product. It doesn't make sense to say
               | that they are EEEing their own product. EEE is when you
               | take some existing open standard, support it in
               | proprietary a product, and then extend it in proprietary
               | ways, thereby taking over the standard. It doesn't apply
               | for a product that you originally created.
        
           | atombender wrote:
           | Why? You do realize their fork is open source?
           | 
           | The fix described in this post have been submitted as a patch
           | to the official Git project. The fix is improving a
           | legitimate inefficiency in Git, and does nothing towards
           | "embracing", "extending", or "extinguishing" anything.
        
             | dijit wrote:
             | you are in the early extend phase.
             | 
             | it will look good, until the extensions get more and more
             | proprietary- but absurdly useful.
        
               | acdha wrote:
               | The extend phase starts when they make extensions which
               | only work in their proprietary version. Putting extensive
               | work into contributing them back is not the same.
        
               | dijit wrote:
               | Ok. There are a dozen examples of exactly this behaviour,
               | and exactly this argumentation in response over the
               | years.
               | 
               | Right now the most important thing for them is for people
               | to start thinking the microsoft fork is the superior one,
               | even if things are "backported".
        
               | atombender wrote:
               | Which examples?
        
               | dijit wrote:
               | VSCode is a prominent one that is in everyones mind, its
               | starting its journey into extinguish.
               | 
               | For more examples I would consult your local greybeard;
               | since the pattern is broad enough that you can reasonably
               | argue that "this time, its different" which is also what
               | you hear every single time it happens.
        
               | mb7733 wrote:
               | What is being embraced, extended and extinguished by
               | vscode?
        
               | dagw wrote:
               | A lot of new and popular features in VSCode are only
               | available in the official MS version of VSCode. Using any
               | of the forks of VSCode thus becomes a lesser experience.
               | 
               | Microsoft Embraced by making VSCode free and open source.
               | Then they Extended by using their resources to make
               | VSCode the go to open source IDE/Editor for most use
               | cases and languages, killing much of the development
               | momentum for none VSCode based alternatives. Now they're
               | Extinguishing the competition by making it harder and
               | harder to use the ostensibly open source VSCode codebase
               | to build competing tools.
        
               | mb7733 wrote:
               | From the wikipedia definition EEE goes like this:
               | 
               | > Embrace: Development of software substantially
               | compatible with an Open Standard.
               | 
               | > Extend: Addition of features not supported by the Open
               | Standard, creating interoperability problems.
               | 
               | >Extinguish: When extensions become a de facto standard
               | because of their dominant market share, they marginalize
               | competitors who are unable to support the new extensions.
               | 
               | As I see it, there no open standard that Microsoft is
               | rendering proprietary through VSCode. VSCode is their own
               | product.
               | 
               | I see your point that VSCode may have stalled development
               | of other open source editors, and has proprietary
               | extensions... but I don't think really EEE fits. It's
               | just competition.
        
               | opticfluorine wrote:
               | To add to this, there are also official Microsoft
               | extensions to VSCode which add absurdly useful
               | capabilities behind subtle paywalls. For example, the C#
               | extension is actually governed by the Visual Studio
               | license terms and requires a paid VS subscription if your
               | organization does not qualify for Visual Studio Community
               | Edition.
               | 
               | I'm not totally sold on embrace-extemd-extinguish here,
               | but learning about this case was eyebrow raising for me.
        
               | neonsunset wrote:
               | C# extension is MIT, even though vsdbg it ships with is
               | closed-source. There's a fork that replaces it with
               | netcoredbg which is open.
               | 
               | C# _DevKit_ is however based on VS license. It builds on
               | top of base C# extension, the core features like
               | debugger, language server, auto-complete and auto-fixers
               | integration, etc. are in the base extension.
        
               | acdha wrote:
               | I note the conspicuous lack of examples, and it's
               | irrelevant in this case where they are working to get the
               | changes merged upstream exactly the way, say, Red Hat
               | might have something they work on for a while before it
               | merges upstream.
               | 
               | VS Code is the most common example people have, but it's
               | not the same: that's always been their project so while I
               | don't love how some things like Pylance are not open it's
               | not like those were ever promised as such and the core
               | product runs like a normal corporate open source project.
               | It's not like they formed Emacs and started breaking
               | compatibility to prevent people from switching back and
               | forth. One key conceptual difference is that code is
               | inherently portable, unlike office documents 30 years ago
               | - if VSC started charging, you could switch to any of
               | dozens of other editors in seconds with no changes to
               | your source code.
               | 
               | I would recommend thinking about your comment in the
               | context of the boy who cried wolf. If people trot out EEE
               | every time Microsoft does something in open source, it's
               | just lowering their credibility among anyone who didn't
               | say Micro$oft in the 90s and we'll feel that loss when
               | there's a real problem.
        
               | dijit wrote:
               | Ok, examples:
               | 
               | * SMTP
               | 
               | * Kerberos (there was a time you could use KRB4 with
               | Windows because AD is just krb4 with extensions: now you
               | have to use AD).
               | 
               | * HTML (activex etc)
               | 
               | * CALDAV // CARDDAV
               | 
               | * Javas portability breakage
               | 
               | * MSN and AOL compatibility.
               | 
               | "oh, but its not the _same_ ". It never is, which is why
               | I didnt want to give examples and preferred you speak to
               | someone who knows the history more than a tiny internet
               | comment that is unable to convey proper context.
        
               | wbl wrote:
               | You understand in these cases the issue was not
               | contributing back right?
        
               | dijit wrote:
               | Yeah... _that_ was the issue.
        
               | szundi wrote:
               | Part of the game
        
               | nixosbestos wrote:
               | No examples offered. And zero that I know of with respect
               | to Git. This is how all open source development is done
               | with big features - iterated on in a fork and proposed
               | and merged in.
               | 
               | There are so many good things to criticize Microsoft for.
               | When this is what people come with, it serves as a single
               | of emotion-based ignorance and to ignore.
        
             | Alifatisk wrote:
             | It's open-source at start, later it turns into open-core.
        
               | atombender wrote:
               | Is this fix evidence of that?
        
               | Alifatisk wrote:
               | No pure speculation
        
               | atombender wrote:
               | Not relevant, then.
        
             | clktmr wrote:
             | Can you imagine their fork extending git with a feature
             | which is incompatible to mainline git and then forcing
             | user's to switch to their fork via github? I can, and it
             | will give them the power to extinguish mainline git and
             | force everything they want on their users (telemetry,
             | licence agreements, online registration...). That might be
             | the reason they're embracing git right now. The fork being
             | open source doesn't help at all.
             | 
             | I'm not saying this shouldn't be merged, but I think people
             | should be aware and see the early signs.
        
               | atombender wrote:
               | Sure, I can imagine. But this isn't what's happening.
        
           | maccard wrote:
           | No, this is cathedral vs bazaar development
        
           | szundi wrote:
           | This comment is downvoted, however you can be sure that
           | managers in these corporations make these decisions
           | deliberately - like half the time.
           | 
           | I find these insightful reminders. Use the vanilla free
           | versions if the difference is negligeble.
        
       | rettichschnidi wrote:
       | I'm surprised they are actually using Azure DevOps internally.
       | Creating your own hell I guess.
        
         | jonathanlydall wrote:
         | I find the "Boards" part of DevOps doesn't work well for us a
         | small org wanting a less structured backlog, but for components
         | like Pipelines and the Git repositories it's neither here nor
         | there for us.
         | 
         | What aspects of Azure DevOps are hell to you?
        
           | rettichschnidi wrote:
           | Some examples, in no particular order.
           | 
           | Hampering the productivity:
           | 
           | - Review messages get sent out before review is actually
           | finished. It should be sent out only once the reviewer has
           | finished the work.
           | 
           | - Code reviews are implemented in a terrible way compared to
           | GitHub or GitLab.                 - Re-requesting a review
           | once you did implemented proposed changes? Takes a single
           | click on GitHub, but can not be done in Azure DevOps. I need
           | to e.g. send a Slack message to the reviewer or remove and
           | re-add them as reviewer.            - Knowing to what line of
           | code a reviewer was giving feedback to? Not possible after
           | the PR got updated, because the feedback of the reviewer
           | sticks to the original line number, which might now contain
           | something entirely different.
           | 
           | - Reviewing the commit messages in a PR takes way too many
           | clicks. This causes people to not review the commit messages,
           | letting bad commit messages pass and thus making it harder
           | for future developers trying to figure out why something got
           | implemented the way it did. Examples:                 - Too
           | many clicks to review a commit message: PR -> Commits ->
           | Commit -> Details            - Comments on a specific commit
           | does not shown in the commits PR
           | 
           | - Unreliable servers. E.g. "remote: TF401035: The object
           | '<snip>' does not exist.\nfatal: the remote end hung up
           | unexpectedly" happens too often on git fetch. Usually works
           | on a 2nd try.
           | 
           | - Interprets IPv6 addresses in commit messages as emoji. E.g.
           | fc00::6:100:0:0 becomes fc00::60:0.
           | 
           | - Can not cancel a stage before it actually has started
           | (Wasting time, cycles)
           | 
           | - Terrible diffs (can not give a public example)
           | 
           | - Network issues. E.g. checkouts that should take a few
           | seconds take 15+ minutes (can not give a public example)
           | 
           | - Step "checkout": Changes working folder for following steps
           | (shitty docs, shitty behaviour)
           | 
           | - The documentation reads as if their creators get paid by
           | the number of words, but not for actually being useful.
           | Whereas GitHub for example has actually useful documentation.
           | 
           | - PR are always "Show everything", instead of "Active
           | comments" (what I want). Resets itself on every reload.
           | 
           | - Tabs are hardcoded (?) to be displayed as 4 chars - but we
           | want 8 (Zephyr)
           | 
           | - Re-running a pipeline run (manually) does not retain the
           | resources selected in the last run
           | 
           | Security:
           | 
           | - DevOps does not support modern SSH keys, one has to use RSA
           | keys (https://developercommunity.visualstudio.com/t/support-
           | non-rs...). It took them multiple years to allow RSA keys
           | which are not deprecated by OpenSSH due to security concerns
           | (https://devblogs.microsoft.com/devops/ssh-rsa-deprecation/),
           | yet no support for modern algos. This also rules out the
           | usage of hardware tokens, e.g. YubiKeys.
           | 
           | Azure DevOps is dying. Thus, things will not get better:
           | 
           | - New, useful features get implemented by Microsoft for
           | GitHub, but not for DevOps. E.g.
           | https://devblogs.microsoft.com/devops/static-web-app-pr-
           | work...
           | 
           | - "Nearly everyone who works on AzDevOps today became a
           | GitHub employee last year or was hired directly by GitHub
           | since then." (Reddit, https://www.reddit.com/r/azuredevops/co
           | mments/nvyuvp/comment...)
           | 
           | - Looking at Azure DevOps Released Features
           | (https://learn.microsoft.com/en-us/azure/devops/release-
           | notes...) it is quite obvious how much things have slowed
           | down since e.g. 2019.
           | 
           | Lastly - their support is ridiculously bad.
        
         | sshine wrote:
         | > _I 'm surprised they are actually using Azure DevOps
         | internally. Creating your own hell I guess._
         | 
         | Even the hounds of hell may benefit from dogfooding.
        
           | tazjin wrote:
           | houndfooding?
        
             | sshine wrote:
             | Ain't nothing but a hound dog.
        
       | AbuAssar wrote:
       | the gif memes were very distracting...
        
       | jbverschoor wrote:
       | > those branches that only change CHANGELOG.md and
       | CHANGELOG.json, we were fetching 125GB of extra git data?! HOW
       | THO??
       | 
       | Unrecognized 100x programmer somewhere lol
        
       | blumomo wrote:
       | > we have folks in Europe that can't even clone the repo due to
       | it's [sic] size
       | 
       | What's wrong with Europeans? Do they have smaller disks?
        
         | mark_and_sweep wrote:
         | As a German, I assumed he's talking about poor connection
         | speeds.
        
           | blumomo wrote:
           | You Germans have slow internet speed? Why's that?
        
             | btilly wrote:
             | https://youtu.be/W1ZZ-Yni8Fg?si=493ozTdkEsXJnPpB does a
             | good job of explaining it.
        
         | mirekrusin wrote:
         | Size doesn't matter, it's how you use it (no invalid diffs on
         | paths sharing trailing part).
        
         | tom_ wrote:
         | They're not _actually_ smaller. It just looks like it because
         | they 're further away.
        
       | nkmnz wrote:
       | > we have folks in Europe that can't even clone the repo due to
       | it's size
       | 
       | Officer, I'd like to report a murder committed in a side note!
        
       | wodenokoto wrote:
       | Nice to see that Microsoft is dog-fooding Azure DevOps. It seems
       | that more and more Azure services only have native connectors to
       | GitHub so I actually thought it was moving towards abandonware.
        
       | jakub_g wrote:
       | Paraphrasing meat of the article:
       | 
       | - When you have multiple files in the repo which have the same
       | trailing 16 characters in the repo path, git may wrongly
       | calculate deltas, mixing up between those files. In here they had
       | multiple CHANGELOG.md files mixed up.
       | 
       | - So if those files are big and change often, you end up with
       | massive deltas and inflated repo size.
       | 
       | - There's a new git option (in Microsoft git fork for now) and
       | config to use full file path to calculate those deltas, which
       | fixes the issue when pushing, and locally repacking the repo.
       | 
       | ```
       | 
       | git repack -adf --path-walk
       | 
       | git config --global pack.usePathWalk true
       | 
       | ```
       | 
       | - According to a screenshot, Chromium repacked in this way
       | shrinks from 100GB to 22GB.
       | 
       | - However AFAIU until GitHub enables it by default, GitHub clones
       | from such repos will still be inflated.
        
         | kreetx wrote:
         | I don't think GitHub, or any other git host, will have
         | objections to using it once it's part of mainline git?
         | 
         | Also, thank you for the TLDR!
        
           | masklinn wrote:
           | > I don't think GitHub, or any other git host, will have
           | objections to using it once it's part of mainline git?
           | 
           | Fixing an existing repository requires a full repack, and for
           | a repository as big as Chromium it still takes more than half
           | a day (56000 seconds is 15h30), even if that's an improvement
           | over the previous 3 days it's a lot of compute.
           | 
           | From my experience of previous attempts, trying to get Github
           | to run a full repack with harsh settings is extremely
           | difficult (possibly because their infrastructure relies on
           | more loosely packed repositories), I tried to get that for
           | $dayjob's primary repository whose initial checkout had
           | gotten pretty large and got nowhere.
           | 
           | As of right now, said repository is ~9.5GB on disk on initial
           | clone (full, not partial, excluding working copy). Locally
           | running `repack -adf --window 250` brings it down to ~1.5GB,
           | at the cost of a few hours of CPU.
           | 
           | The repository does have some of the attributes described in
           | TFA, so I'm definitely looking forward to trying these
           | changes out.
        
             | leksak wrote:
             | Wouldn't a potential workaround be to create a new
             | barebones repository and push the repacked one there? Sure,
             | people will have to change their remote origin but if it
             | solves the problem that might be worth the hassle?
        
               | masklinn wrote:
               | It breaks the issues, PRs, all the tooling and
               | integration, ...
               | 
               | For now we're getting by with partial clones, and
               | employee machines being imaged with a decently up to date
               | repository.
        
         | jamalaramala wrote:
         | Thank you to the AI that summarised the article. ;-)
        
         | deskr wrote:
         | > in Microsoft git fork for now
         | 
         | Wait, what? Has MS forked git?
        
           | jakub_g wrote:
           | MS has had their fork of git for years, and they contributed
           | many performance features for monorepos since then to the
           | mainline.
        
       | jakub_g wrote:
       | The article mentions Derick Stolee who dig the digging and
       | shipped the necessary changes. If you're interested in git
       | internals, shrinking git clone sizes locally and in CI etc,
       | Derrick wrote some amazing blogs on GitHub blog:
       | 
       | https://github.blog/author/dstolee/
       | 
       | See also his website:
       | 
       | https://stolee.dev/
       | 
       | Kudos to Derrick, I learnt so much from those!
        
       | develatio wrote:
       | Hacking Git sounds fun, but isn't there a way to just not have
       | 2.500 packages in a monorepo?
        
         | Cthulhu_ wrote:
         | Yeah, have 2500 separate Git repos with all the associated
         | overhead.
        
           | develatio wrote:
           | Can't we split the packages into logical groups and maybe
           | have 20 or 30 monorepos of 70-100 packages? I doubt that all
           | the devs involved in that monorepo have to deal with all the
           | 2500 packages. And I doubt that there is a circular
           | dependency that requires all of these packages to be managed
           | in a single monorepo.
        
             | smashedtoatoms wrote:
             | People act like managing lots of git repos is hard, then
             | run into monorepo problems requiring them to fix esoteric
             | bugs in C that have been in git for a decade, all while
             | still arguing monorepos are easy and great and managing
             | multiple repos is complicated and hard.
             | 
             | It's like hammering a nail through your hand, and then
             | buying a different hammer with a softer handle to make it
             | hurt less.
        
               | crazygringo wrote:
               | > _all while still arguing monorepos are easy and great_
               | 
               | I don't know anyone who says monorepos are easy.
               | 
               | To the contrary, the tooling is precisely the hard part.
               | 
               | But the point is that the difficulty of the tooling is a
               | lot less than the difficulty of managing compatibility
               | conflicts between tons of separate repos.
               | 
               | Each esoteric bug in C only needs to be fixed once.
               | Whereas your version compatibility conflict this week is
               | going to be followed by another one next week.
        
               | HdS84 wrote:
               | Thad exactly the problem. At least tooling can solve mono
               | repo problems. But commits , which should span multiple
               | repos, have no tooling at all. Except pain. Lots of pain.
        
               | wavemode wrote:
               | At Amazon, there is no monorepo.
               | 
               | And the tooling to handle this is not even particularly
               | conceptually complicated - a "versionset" is a set of
               | versions - a set of pointers to a particular commit of a
               | repository. When you build and deploy an application,
               | what you're building is a versionset containing the
               | correct versions of all its dependencies. And pull
               | requests can span across multiple repositories.
               | 
               | Working at Amazon had its annoyances, but dependency
               | management across repos was not one of them.
        
           | hinkley wrote:
           | Changing 100 CI pipelines is a giant pain in the ass. The
           | third time I split the work with two other people. The 4th
           | time someone wrote a tool and switched to a config file in
           | the repo. 2500 is nuts. How do you even track red builds?
        
         | lopkeny12ko wrote:
         | This was exactly my first thought as well. This seems like an
         | entirely self-manufactured problem.
        
           | hinkley wrote:
           | When you have hundreds of developers you're going to get
           | millions of lines of code. Thats partly Parkinson's Law but
           | also we have not fully perfected the three way merge,
           | encouraging devs spread out more than intrinsically necessary
           | in order to avoid tripping over each other.
           | 
           | If you really dig down into why we code the way we do, the
           | "best practices" in software development, about half of them
           | are heavily influenced by merge conflict, if not the primary
           | cause.
           | 
           | If I group like functions together in a large file, then I
           | (probably) won't conflict with another person doing an
           | unrelated ticket that touches the same file. But if we both
           | add new functions at the bottom of the file, we'll conflict.
           | As long as one of us does the right thing everything is fine.
        
         | hinkley wrote:
         | Code line count tends to grow exponentially. The bigger the
         | code base, the more unreasonable it is to expect people not to
         | reinvent an existing wheel, due to ignorance of the code or
         | fear of breaking what exists by altering it to handle your use
         | case (ignorance of the uses of the code).
         | 
         | IME it takes less time to go from 100 modules to 200 than it
         | takes to go from 50 to 100.
        
       | mattlondon wrote:
       | I recently had a similar moment of WTF for git in a JavaScript
       | repo.
       | 
       | Much much smaller of course though. A raspberry pi had died and I
       | was trying to recover some projects that had not been pushed to
       | GitHub for a while.
       | 
       | Holy crap. A few small JavaScript projects with perhaps 20 or 30
       | code files, a few thousand lines of code for a couple of 10s of
       | KBs of actual code at most had 10s of gigabytes of data in the
       | .git/ folder. Insane.
       | 
       | In the end I killed the recovery of the entire home dir and had
       | to manually select folders to avoid accidentally trying to
       | recover a .git/ dir as it was taking _forever_ on a poorly SD
       | card that was already in a bad way and I did not want to finally
       | kill it for good by trying to salvage countless gigabytes of
       | trash for git.
        
       | EDEdDNEdDYFaN wrote:
       | better question - does the changelog need to be checked in the
       | first place?
        
         | DeathMetal3000 wrote:
         | They fixed a bug on a tool that is widely used. In what world
         | is questioning why an organization is checking in a file that
         | you have no context on a "better question".
        
       | tazjin wrote:
       | I just tried this on nixpkgs (~5GB when cloned straight from
       | Github).
       | 
       | The first option mentioned in the post (--window 250) reduced the
       | size to 1.7GB. The new --path-walk option from the Microsoft git
       | fork was less effective, resulting in 1.9GB total size.
       | 
       | Both of these are less than half of the initial size. Would be
       | great if there was a way to get Github to run these, and even
       | greater if people started hosting stuff in a way that gives them
       | control over this ...
        
       | dizhn wrote:
       | They call him Linux Torvalds over there?
        
       | nixosbestos wrote:
       | Oh hey I know that name, Stolee. Fellow JSR grad here.
        
       ___________________________________________________________________
       (page generated 2024-10-27 23:02 UTC)