[HN Gopher] Launch HN: Diversion (YC S22) - Cloud-Native Git Alt...
___________________________________________________________________
Launch HN: Diversion (YC S22) - Cloud-Native Git Alternative
Hi Everyone! We're Sasha and Egal, co-founders of Diversion
(https://diversion.dev). We're building a modern, cloud-native
version control. Our first users are game developers, who like its
simplicity and scalability. See a quick demo here:
https://youtu.be/DD0XkL8kDYc Why a new VCS? There is no doubt that
Git vastly improved our lives, and played a significant role in the
advancement of software development over the past 18 years. But -
it was built for a very different world in 2005 (slow networks,
much smaller projects, no cloud), and is not the perfect tool for
everyone today. The biggest drawback of Git is its limited
scalability - both in repository and file sizes, and the number of
concurrent users. This is the reason Google and Meta built their
own version control systems. It's also the reason why other large
companies, most notably in games development, semiconductors and
financial services are still using legacy tools like SVN and
Perforce. Another issue we're trying to fix is Git's famous
complexity. In our previous startup, a data scientist accidentally
destroyed a month's work of his team by using the wrong Git command
(EDIT: we were eventually able to restore from a non-updated repo
clone, after a few hours). As a developer who used CVS and SVN
before Git was created, I often wondered why Git is so difficult to
learn, compared to other tools. On the other hand, Git's branching
and merging abilities are exceptional - this has enabled the modern
software development methodologies that we all take for granted
today (e.g. feature branches, CI/CD), greatly improving developers'
velocity. We were wondering - is it possible to create an easy-to-
use, fast, scalable version control system, with Git's branching
capabilities? And what else can be improved, while we're at it?
One thing available in modern cloud tools is real-time
collaboration (e.g. Google Docs, Figma). While developers don't
necessarily want their work in progress to be visible to everyone,
it may be very useful to easily share it when you want to get
feedback before a commit, to detect and prevent merge conflicts,
and to have visibility into which parts of the codebase are being
changed by others. Diversion is built on top of distributed
storage and databases, accessible via REST API, and runs on
serverless cloud infrastructure. Every repository operation is an
API call (commit, branch, merge etc.). The desktop client
synchronizes all work in progress to the cloud in real time (even
before a commit). Users can work with Diversion using an
interactive CLI, Web UI, or IDE plugins (currently JetBrains, more
coming soon). The Web UI allows to perform most basic operations,
without needing to install a desktop client. Diversion is
compatible with Git, and can synchronize with existing Git
repositories (each new commit in Diversion goes into Git, and vice
versa). We're planning to release it as open source once the code
base matures, and when we implement an open source repositories
directory on our website (naturally, Diversion's code is managed on
Diversion!) We're in open beta, you can try it here
(https://diversion.dev) (click Get Started). It's completely self-
service and there's no need to talk to anyone, and it's free for
small teams (https://diversion.dev/pricing). Building a version
control is hard (as we have learned), and Diversion still has a
long way to go. We are currently working on improving speed, CI
integrations, plugins to IDEs and game engines, and other usability
improvements. We would love to hear your thoughts and feedback on
what we've got so far!
Author : sasham
Score : 246 points
Date : 2024-01-22 12:01 UTC (10 hours ago)
| parasti wrote:
| Focusing on Git seems like completely the wrong pitch. Git is a
| distributed VCS - in all your examples you were clearly trying to
| use Git in a centralized manner with no backups. I suggest
| focusing more on your own product than on Git.
| sasham wrote:
| You're totally right, we were using BitBucket and pushing and
| pulling from there. It's really more of a centralized manner,
| but this is the usual workflow for most teams and companies and
| what they actually need (a single source of truth). Totally
| agree about backups, lesson learned :)
| funcDropShadow wrote:
| Until the day, the company is target by a ransomware group
| and everything is switched off in panic. Or the day the
| network connection of a building goes down. Requiring a REST
| api in the cloud for every VCS is command bytes hard then. We
| had this before with SVN and it wasn't nice.
| solidasparagus wrote:
| I want to add counter-feedback. I think focusing on git's
| weaknesses is really appealing. My background is more ML
| research and data and I viscerally connect with your pitch
| around both git's limited scalability and the
| complex/dangerous nature of git operations (ML researchers +
| git is not great).
| TexasMick wrote:
| Tbh they should really just learn it. It's super simple and
| powerful
| IlliOnato wrote:
| So it sounds to me as you are trying to create a replacement
| for BitBucket\GitHub and their ilk, not Git. This may be a
| worthwhile task. Maybe it makes sense to concentrate on this
| in your pitch.
|
| For BitBucket\GitHub\GitLab and for workflows enabled by them
| Git is just an underlying technology. Some of the
| functionality of these services is implemented using Git
| commands very clumsily. Some Git commands don't make sense or
| are dangerous in such environment\workflow. Yet Git interface
| is fully exposed to the users of these systems.
|
| (Despite your statements and example, Git commands are not
| dangerous in the sense that they can destroy information
| already pushed to the repo. However, as your example
| demonstrated, they are dangerous in the sense that to recover
| from them requires expert knowledge and capabilities.)
|
| Git was designed to for truly distributed development, and it
| is great for that. A lot of projects use it though for a
| centralized development. Git the software is fully capable to
| support such development with proper configuration, but has
| arguably bad defaults for it, and the existing solutions seem
| to be half-assed (to tell the truth, I hate GitHub, but won't
| go into this right now).
|
| For me in my work and personal use the fully distributed
| character of Git is not important, but being able to work
| offline is, crucially. I know it is important issue for many
| developers. With working from home being more and more
| widespread I'd think this issue becomes more important, not
| less. Not being tethered to your good internet connection, or
| being able to work during an outage is really cool :-)
|
| (Before Git I would have 2 VCS applications installed on my
| work laptop, one working against central database that
| required internet connection, one fully local, with separate
| local database. Synchronizing them was a constant chore, a
| significant part of it I was not able to automate and had to
| do manually. Sill, it was worthwhile price for being able to
| work offline.)
| cornholio wrote:
| > In our previous startup, a data scientist accidentally
| destroyed a month's work of his team by using the wrong Git
| command.
|
| While git is indeed a usability clusterbomb and there is massive
| space to improve, the problem above sounds like a devops failure.
| Git gives you all the tools to prevent such a disaster, all you
| have to do not give the root password of your CI server to any
| data scientist.
|
| On the topic of your startup, I would very much like a lighter
| learning slope, where I can introduce regular non-coders to the
| benefits of source control, and still have the advanced
| brancing/merging/rebasing etc. for the wizards.
| sasham wrote:
| True! They didn't actually have a root pwd to anything, the
| repo was hosted on BitBucket which didn't have a branch
| protection feature back then.
|
| Non-coder users are actually an important use case for
| Diversion (like in game development where many/most users are
| artists).
| jsnell wrote:
| And nobody on the team had a reasonably fresh checkout of the
| repo on their local machine?
|
| To put it bluntly, this story does not sound credible. It's
| also one of the first things you say in the pitch, which
| taints everything you write later. I would suggest focusing
| on what you do well, not in making up stories about data loss
| with what you perceive as the competition.
|
| (It's especially odd when Git isn't the obvious competition;
| Perforce is.)
| sasham wrote:
| Swear to god, the story is true! We were able to restore
| most of the work because someone didn't pull the updates.
| (I should have added it in the story, didn't think it was
| important) But it was nerve wracking :')
| dmazzoni wrote:
| But if someone DID pull the update, the old commits would
| have still been in their local repo and they could have
| run "git reflog" to retrieve them.
| xorcist wrote:
| The person, who apparently googled some command sequence
| and happily entered them only to find out it overwrote
| the team's data, could just as easily have used those
| google skills to search for how to undo last git
| operation.
|
| Obviously this person had all the privileges required to
| force push the previous commit in order to save the day.
| The old adage of never fact-checking a good story holds
| true however. Not sure it's a good selling point though,
| by nerd sniping everyone to explain in detail what the
| actual problem was no one will read to the end.
| sgarland wrote:
| I also always wonder in cases like this if the person just
| didn't know about reflog.
|
| You _really_ have to try to fuck up hard enough that everything
| is gone, it just might require even more arcane commands than
| what got you into a mess.
| faitswulff wrote:
| I've used reflog many times, but I'm not sure if this:
|
| > You _really_ have to try to fuck up hard enough that
| everything is gone
|
| ...is still true when using git-lfs, which seems common when
| using large data sets.
| sbergot wrote:
| I believe it works the same. The large files are not
| immediately pruned.
| skymer wrote:
| Is my company's private information e.g. source code kept end-to-
| end encrypted?
| egaldv wrote:
| All data is encrypted in-transit and at-rest, as per the
| industry standard. End-to-end encryption is an interesting
| feature with obvious extended privacy benefits on one hand,
| that could prevent a system from providing other features on
| the other hand. Depending on the requirement for end-to-end
| encryption, in some instances a custom NDA can be signed to
| mitigate a specific concern. It's a feature we haven't seen
| much demand for, but if that changes we may prioritize it
| higher on the road map. Would you mind sharing your use case or
| concern?
| vincnetas wrote:
| I think the question is "can any one outside my company/org
| access my code". Apparently it is because the tool offers
| cloud collaboration using web UI.
| egaldv wrote:
| If you share your repository with a collaborator, they can
| access it of course. Same is true if you share your
| repository with end-to-end encryption, your collaborator
| should be able to decrypt and use it.
| funcDropShadow wrote:
| So, the answer is No.
| sasham wrote:
| Good question! End-to-end encryption would mean that you can't
| run cloud CI or merge conflicts in the cloud and other
| features. To my knowledge none of existing VCS solutions are
| end-to-end encrypted (I might be wrong though!)
|
| We might add end-to-end encryption in the future (disabling
| some capabilities), if there's demand for it.
| throwaway63467 wrote:
| Perforce seems to already fill this niche pretty well, at least
| in the gaming sector which this startup seems to target.
| psantosl wrote:
| Perforce is painful, doesn't have good cloud offering and does
| not evolve
| ryanlitalien wrote:
| We have quite a few cloud options actually, with our Helix
| Cloud option just releasing last week!
| https://www.perforce.com/perforce-and-cloud
| reallyliri wrote:
| One only uses Perforce, well, perforce.
| sasham wrote:
| Totally right, perforce is used by almost all large game
| studios. But it's painful to use for smaller studios and indie
| devs - it requires managing your own server with configuration,
| backups, networking etc. It's also not great for cloud based
| workflows and remote work, and super expensive with rigid
| licensing.
| lemome wrote:
| Most of the time we _need_ to host our own servers for
| licensing issues, you might need to consider a "on premises"
| option if you want to be an viable option for a lot of
| studios.
| spacebanana7 wrote:
| This sounds like a classic innovator's dilemma style
| division in the market.
|
| The incumbents need to cater to existing customers with a
| need to host their own servers for licensing reasons; but
| this forbids them from using cloud native features in the
| core of their products.
|
| There may well be space for a newcomer to make a cloud only
| product targeted at the subset of studios that have the
| legal ability to use the cloud. Instinctively, there may be
| some useful features around large files which are possible
| in the cloud but impractical in an on prem environment.
| withinboredom wrote:
| My understanding from reading much of OPs responses is
| that "the cloud" may be vendor locked to AWS. There are
| serious issues there to hosting your code on servers most
| likely owned by a competitor. E-Commerce, video
| streaming, games, AI, video production, electronics,
| retail, pharmaceuticals, logistics, publishing, etc. Like
| they literally picked the worst vendor to build this on.
| ryanlitalien wrote:
| Our recently released Helix Core Cloud actually doesn't
| require managing your own server, we'll do that for you, as
| it's a SaaS using flexible licensing:
| https://azuremarketplace.microsoft.com/en-
| us/marketplace/app...
| dafelst wrote:
| It does, but perforce is a horrible slow beast, and not as
| reliable as you might expect.
|
| That said, for large game projects, it really is the only
| viable option.
| akamia wrote:
| Do you have more documentation somewhere? If so, I'd suggest
| making it more easily accessible. I couldn't find any on the site
| and the support link just takes me to discord. I'd be more
| inclined to sign up if there was some documentation I could read
| through to get a sense for requirements, setup, branching, CLI
| commands, etc.
| sasham wrote:
| Thanks for flagging that! Just updated the site and forgot the
| link. It should be there now! (You also have an intro video and
| in-app docs, but there's more work we need to do there)
| janjones wrote:
| https://docs.diversion.dev/
| gigatexal wrote:
| Congrats on the HN launch. How does this improve or expand or
| blow git-lfs[1] out of the water because if I needed large blob
| file support it's what I would use instead. It offers pointers to
| the big files to the hosted git instead of pushing around the
| binaries itself -- though I am speculating since I've not used it
| myself just read about it online.
|
| I mean the nice UI and collab features are indeed improvements
| but I'm thinking more core git specific improvements.
|
| [1] https://git-lfs.com/
| sasham wrote:
| Git LFS solves the problem if you need to have occasional large
| files in your repo. It doesn't work great though when you have
| a lot of them or when they're an integral part of your product,
| because it's slow and introduces devops problems (e.g. you can
| bomb your repo by committing from a clone without Git LFS
| installed). Companies that have many large files to manage
| rarely use it because of this.
| gigatexal wrote:
| Okay. That's fair and info I didn't have.
| rajeevk wrote:
| I have not analyzed the full potentials and benefits of Diversion
| but I would not agree with the statements you made about the Git.
| I think you should not focus on Git in your pitch.
|
| >> _it was built for a very different world in 2005 (slow
| networks, much smaller projects, no cloud)_
|
| Slow network: why is this a negative thing? If something is
| designed for a slow network then it should perform well in a fast
| network.
|
| Mush small project: I do not agree. I can say that it was not
| designed for very very large projects initially. But many
| improvements were made later. When Micorosoft adopted Git for
| Windows, they faced this problem and solved it. Please look at
| this https://devblogs.microsoft.com/bharry/the-largest-git-
| repo-o...
|
| No cloud: Again I would not agree. Git is distributed so should
| work perfectly for the cloud. I am not able to understand what is
| the issue of Git in the cloud environment.
|
| >> _In our previous startup, a data scientist accidentally
| destroyed a month's work of his team by using the wrong Git
| command_
|
| This is mostly a configuration issue. I guess this was done by a
| force push command. IFAIK, you can disable force push by
| configuration.
| funcDropShadow wrote:
| Even a force push doesn't destroy the reflog or runs the GC
| server-side. I wonder how you can accidentally loose data with
| Git. I've seen lot's of people not being able to find it, but
| really destroying it is hard.
| cqqxo4zV46cp wrote:
| Destroying it and nobody knowing how to recover, or that it
| can be recovered at all, it are identical.
| sasham wrote:
| He force pushed a diverged branch or something like that, and
| we only found out after a while. We were eventually able to
| recover because someone didn't pull. But it was not a fun
| experience :D
| umanwizard wrote:
| So the work wasn't actually destroyed, and you were able to
| recover it. So all the people pointing out how implausible
| that part of your pitch was were right, and you were in
| fact just lying.
| sasham wrote:
| That's really not the main point of the post, but you're
| right I should have been more precise.
|
| Edit: updated in the top text now!
| ysavir wrote:
| I think the point being made is that you spent a lot of
| your opening post talking Git, and lead with that bit,
| rather than with Diversion. What makes Diversion
| different is added in the end, _after_ you 've spent time
| trying to convince Git fans that their current tooling
| isn't good. Worse, the examples you listed of why Git is
| bad is more reflective of configuration and processes
| than Git itself.
|
| This is ultimately a very weak pitching strategy. The
| first thing you convey to your potential users is
| insecurity--an insecurity that people won't choose your
| product over Git. And it's hard to want to buy something
| from someone that isn't secure enough about their product
| to pitch the product first, and answer questions/make
| comparisons after, as a form of clarification.
|
| Alternatively, instead of doing a comparison to Git, you
| could start with a list of "have you experienced these
| Git issues? <list of problems>. Here's how Diversion
| improves on Git in this regard." In this case you're
| actually solving people's problems, rather than looking
| like you're grasping at straws to complain about Git and
| justify an alternative.
|
| FWIW, I personally have 0 interest in a cloud-first
| version control. I like the cloud as a form of backup and
| syncing with team members, but I ultimately want a
| version control that works as well offline as it does
| online, and prioritizes the local experience.
| umanwizard wrote:
| The main point of your post is how much better you are
| than git. You support this main point by making up lies
| about git. This does not make me personally interested in
| trying your product.
| IlliOnato wrote:
| From my point of view, it's not that much about lying,
| for me the OP demonstrates a degree of incompetence of
| the post writers about Git.
|
| The fact that they don't seem to _fully_ understand
| working of Git (not on the level of Git developers, just
| the level of Git administrators /users) does not inspire
| trust in their competence to create a Git alternative.
| bauruine wrote:
| So multiple people did a git reset --hard origin/master and
| nobody complained or checked what and why this was done?
| That's not "one data scientist with the wrong command" but
| the whole team that fucked up hard IMHO.
| NateEag wrote:
| And the tool made a screwup that hard not only possible,
| but very difficult for the victims to recover from.
|
| Doesn't say a lot for git's usability.
| funcDropShadow wrote:
| They used --force which is usually the flag to say: Here
| there be dragon. Be careful.
| mbreese wrote:
| Yeah, I can't see how use of a ---force flag by people
| who didn't know what they were doing is enough of a
| reason to switch to a different VCS (let alone write
| one). The issue was people using a tool in a way that
| they shouldn't have. Which isn't a technical problem, but
| a training problem. You can't fix people problems with
| technology, so I'm sure there will be other footguns in
| this new system that someone else will figure out how to
| almost lose data.
|
| Git is great in that it is flexible and powerful. But
| that power leaves some tools open to people who don't
| know what they are doing... that's the trade off.
|
| (Now something that better handles non-code assets and
| large data files, I'd be much more willing to listen to
| that pitch.)
| JeremyNT wrote:
| A couple of thoughts about this:
|
| One is that the _possibility_ of overwriting history /
| etc is a really powerful and useful feature, but one that
| should only be used with some consideration, hence being
| gated behind the scary '--force'. The fact that git
| _provides_ one the ability to discard and overwrite
| commits for a ref shouldn 't be an endorsement of doing
| so freely. I'm _glad_ git has this capability though and
| any "git alternative" would be all the worse if it
| didn't provide it, IMO.
|
| Two is that if the concern is git's _usability_ - i.e.
| the "problem" here is that it's too "easy" for users to
| do destructive actions accidentally - well, there are
| ways to solve _that_ other than to reinvent _all_ of git.
| There are plenty of alternative git UIs already, and an
| alternative UI is a great way to be "wire compatible"
| for existing users but still help protect those novice
| users from footguns.
| rusk wrote:
| It doesn't overwrite the commute though. It inserts new
| ones and resets the branch pointer ... doesn't seem like
| you'd need a whole new tool to mitigate this - just an
| automatically generated tag or something when you
| --force-push - would be easy to do if there was demand
| for it ...
| NateEag wrote:
| That all makes sense and mirrors many of my own thoughts.
|
| Though I'll say that "--force" isn't necessarily a
| "scary-sounding" option name unless you're used to Unix
| CLI naming conventions.
|
| Further, the warnings git gives you about this are
| virtually inscrutable if you don't already understand
| what's happening.
|
| A _good_ interface to "blowing away history" would give
| you a brief summary of what will actually be gone, e.g.:
|
| "If you go ahead with this overwrite, the following
| changes will be completely removed from the repo:
|
| a3bf45: Fix bug in arg parsing 22ec04: Add data from
| 2024-01-17 scraper run ...
|
| Are you SURE you want to completely destroy those
| commits? (Y/n)"
|
| and if user says "Y", output should log all removed
| commits and also say:
|
| "These commits can still be recovered until <date>. If
| you realize you want these back before then, run the
| following:
|
| <command to restore commits>"
|
| Generally, I think it's a mistake to put UI improvements
| in a secondary tool.
|
| If there are issues that need fixing, get those changes
| in the canonical project, because layered patches on top
| will always be short of maintainers and behind the main
| project.
| xorcist wrote:
| > Are you SURE you want to completely destroy those
| commits? (Y/n)
|
| While there is a lot of user interfaces that could be
| improved, I believe the above have empirically been shown
| to be inferior to the alternative "re-run this command
| but add scary option to proceed".
|
| Users habitually answer "Y" to questions like the above
| all the time. And certainly after a few times it becomes
| routine for anyone. But having to re-enter the command
| and type some a whole word like "overwrite", "force" or
| "i-know-what-im-doing" is a whole other roadblock. The
| example is especially ill-chosen to have Y as the default
| option.
|
| Any operation in git that destroys so many commits will
| include a list of commits that is destroyed, similar to
| what is suggested here, and trying to push the resulting
| repository will say exactly how many commits will be
| removed, and require rerun with force option (together
| with the necessary privileges). So reality is already not
| far from what you suggest, but with more fail safes.
| lonelyasacloud wrote:
| > but the whole team that fucked up hard IMHO.
|
| Multiple individuals with similar problems would tend to
| imply systematic inadequate training. Or the enterprise
| concerned adopting an inappropriately complex system for
| its intended userbase.
| willy_k wrote:
| Or, git is both very complex and very useful, and a large
| portion of its users have a poor understanding of git but
| enough for it to be a useful tool. If you want to do
| source control (which you do), then you're investing time
| into learning git and/or fixing git, or maybe using a
| project like this.
| keerthiko wrote:
| You literally just said what GP said in different words,
| but prefaced it with "Or" as if it's a disagreement. What
| you said boils down to "inadequate training".
| willy_k wrote:
| We both agree that they didn't know the tool, but GP
| seems to blame them for deciding to use the tool without
| training. I was more or less defending their choice to
| use git, while also acknowledging the potential of a tool
| like Diversion. My interpretation of GP was that it
| doubled down on git, while claiming that anyone using git
| without understanding it is "doing it wrong", which I
| agree with in principle but not in practice, as I argued
| in my initial comment.
| swells34 wrote:
| I think you just sold their pitch with this comment... I,
| like many many people here, have done quite a bit of
| product design. What do you call it when a bunch of
| people use your product, and it breaks for several of
| them? That generally indicates your product is weak, or
| has a very rough UI.
| MrDarcy wrote:
| The pitch simply wasn't true. Data was not destroyed and
| was restored hours later.
| swells34 wrote:
| It's a pitch. The story has obviously been embellished
| and polished and condensed, ready public consumption.
| Being pedantic against it is not productive.
| yjftsjthsd-h wrote:
| If the pitch is dishonest, why would I ever trust them
| with something as vital as my VCS? (And yes,
| "embellished" means dishonest)
| spiderice wrote:
| That's a creative way to defend a dishonest pitch
| MrDarcy wrote:
| Politely disagree. It's productive because hopefully
| future teams who launch on HN ask each other, "Is what
| we're saying true?" during all those polishing and
| condensing sessions. If they don't, the risk is crossing
| a line that damages the reputation of the team and
| undermines months if not years of hard work.
| water-your-self wrote:
| This is not a pedantic criticism.
| lowbloodsugar wrote:
| For many of us, the story rings true. We have ourselves
| had horror stories that we did manage to recover from
| after a few hours of fearfully googling, and we know of
| other, less capable friends and colleagues who were
| unable to recover the data and who just accepted the
| loss.
| TexasMick wrote:
| It's kinda crazy argument, I think data loss is way more
| likely with a centralised system than a decentralised
| system.
| Wytwwww wrote:
| But that seems like pretty much the equivalent to "rm -R
| *"? And also just a permission/configuration issue.
| bauruine wrote:
| Or drop table|database or delete from. To _nearly_ lose
| data it took multiple clueless engineers and not
| detecting the issue for months.
|
| I wonder how Diversion handles operations that possibly
| delete data. Whats their solution?
| sasham wrote:
| To put into perspective, that was in 2014 :D There were
| no branch protections, and git was even harder to use.
| Plus everyone was new at git, obviously (we started in
| 2013 with mercurial, which was still a legit thing to do,
| and switched to git).
| lawgimenez wrote:
| So you were able to recover and did not lost a months work
| of data? Your story just doesn't make sense. Come on.
| egaldv wrote:
| Indeed you're right the work that was erased from
| BitBucket was restored from one of the employees that
| didn't yet pull, the post was edited accordingly.
| umanwizard wrote:
| Wouldn't you still have been able to recover it even if
| everyone did pull, assuming GC had not run on everyone's
| machine?
| sampo wrote:
| > the work that was erased from BitBucket was restored
| from one of the employees that didn't yet pull
|
| Actually those commits that you considered lost, were
| still stored on everyone's personal computer in your
| team. You just didn't know how to use `git reflog` to
| find them.
| sbergot wrote:
| For your information you can use the reflog command to find
| the previous head commit and restore your branch. It takes
| 10 minutes and then you learn to disable force pushing on
| the main branch.
| swells34 wrote:
| I find it funny how many comments in this angry rebuttal
| section actually endorse a Git replacement.
| plagiarist wrote:
| It's an interesting new application of that joke, "when I
| have a question on Linux I use a sock puppet account to
| leave an obviously wrong answer which prompts dozens of
| corrections."
|
| I'm trying to imagine how to generalize this to other
| products. I think if I state the competing product has
| negative feature X, but also intentionally get some
| details confidently incorrect or deliberately feign
| incompetence, you get a group of people confirming X.
| spiderice wrote:
| I find it funny how many comments you've made in this
| thread missing the point. People are reacting against the
| dishonest pitch, not the product.
| renewiltord wrote:
| Just somewhat surprised because if anyone did a `git pull`
| they'd get divergent history and therefore a merge on
| default configuration. It would take a lot of manual work
| to ruin more than one copy of the repo.
| billpg wrote:
| I configured my github to only allow commits with an
| anonymised email address. Time passed and I used another
| machine on which I had already opened that repo before. I
| pulled my recent work successfully, wrote stuff and then
| committed and pushed.
|
| Github rejected my commit as I had the wrong email address. I
| then had to try and work out how I delete a commit but keep
| all my changes so I could commit it all again but with the
| correct email address.
|
| I'm not sure exactly what I did but in my ham-fisted
| experimentation I deleted the commit _and_ restored my local
| copy back to the way it was before my commit, losing all my
| work that day.
| BossingAround wrote:
| If you had already committed, `git reflog` should have
| still found your changes (even after you deleted the commit
| and restored the local working tree) unless you deleted and
| re-cloned the repository.
| fesc wrote:
| Honestly I don't understand why not more people use a GUI
| for git.
|
| What you describe would be 1 Minute of work and maybe 10
| clicks with a very low probability of shooting yourself in
| the foot in Tower.
| noufalibrahim wrote:
| I agree. It's quite hard to actually destroy data in git.
| Even with the so called "destructive" commands, walking
| through the reflogs can usually restore work that was
| accidentally deleted or whatever.
| grumbel wrote:
| > doesn't destroy the reflog or runs the GC server-side.
|
| Git doesn't give you access to the server side reflog either.
| So it's of not much use if you don't control the server.
|
| As for losing data with Git, the easiest way to accomplish
| that is with data that hasn't been committed yet, a simple
| `git checkout` or `git reset --hard` can wipe out all your
| changes and even reflog won't keep record of that.
| bvrmn wrote:
| > As for losing data with Git, the easiest way to
| accomplish that is with data that hasn't been committed yet
|
| Also Git has pretty awful behavior losing changes when one
| doesn't press "Save" in their IDE. Bad, bad Git.
| Spivak wrote:
| Your applications also shouldn't lose work when you don't
| press save, this is the entire impetus for the "recover
| unsaved work" in most document editors. A version of Git
| that shunted uncommitted changes to a special named stash
| whenever you did anything destructive would be a positive
| thing.
|
| It's what I end up doing manually anyway but why make a
| system where the default behavior is destructive and I
| have to remember every.
| xorcist wrote:
| It may be prudent to note that git by default is rather
| kind in that way that it will not change your data unless
| you explicitly force it to with --force or --hard. I
| think git, as hard to learn as it can be, sometimes have
| a bit of an unfair reputation here. It's not all bad.
|
| Not only is it quite careful about not losing data,
| someone actually took the time to make it spit out
| messages that not only describes what just happened, but
| also gives suggestions of what to do next depending on
| how the user wants to proceed. That adds a level of
| discoverability that is usually associated with dialog
| based guis. The quality of these messages can sometimes
| be surprisingly good, far from the Clippy-level
| helpfulness you sometimes see.
|
| There are a few exceptions to the principle of not losing
| local changes, where you explicitly restore an old
| version of a file for example. But saying the default
| behaviour is destructive really gives a false impression.
|
| But yes, you are absolutely right that a system to
| recover unsaved work is a good thing, but I would argue
| that it belongs at the editor level, not in a version
| control system. A user could have a number of files open
| that have local changes. The editor has a much better
| idea in which order changes were made, and which changes
| hasn't even been committed to disk yet.
| whartung wrote:
| I can't say I'm widely traveled, I have no idea how
| desktop Office works, but Apple does this so well.
|
| Using their desktop apps, Pages, Keynote, Numbers,
| TextEdit, Preview, I never hit "Save". I just close the
| apps. When I come back, the windows reopen right where I
| left off.
|
| I wish emacs did this. I honestly don't know what it
| would be like for a code editor to be "constantly
| saving". I guess I would adapt, but there are times when
| I do all sorts of changes and go "Ah, this isn't right"
| and just kill the buffer. The ultimate undo.
|
| But there's a great feeling, to me, when I go to close
| the app (or shutdown the computer) and it just closes. No
| prompts, no warnings, just saves its state, shuts down,
| and comes back later. And with the ever popular "naming
| things" issue of computers, I have a bunch of just
| "Untitled" windows. They're there when I open the app,
| and that's all I need to know.
|
| The nag factor and cognitive load reduction of that is
| just unmatched. "Just deal with it, I'll come back later,
| maybe, and clean it up". One less thing.
| Brian_K_White wrote:
| A month of work for a whole team was never even committed
| or stashed let alone pushed? That is not a git problem.
| xorcist wrote:
| That data not committed to git can not be recovered by git
| should hopefully not surprise anyone.
|
| Neither is it the fault of your version control system, or
| any other system really, if you cannot access your server
| and are without backups.
| Rygian wrote:
| > This is mostly a configuration issue. I guess this was done
| by a force push command. IFAIK, you can disable force push by
| configuration.
|
| If a feature can lead to actual unintended data loss, it should
| come disabled by default. Are there any other "unsafe by
| default" features in Git? What would be a sane general default
| that prevents unwanted data loss, and why is it the case?
| guax wrote:
| --force always imply data loss. You're overriding the remote
| state.
|
| Do people use it in an unsafe manner because they don't
| understand git and there lies a problem that could be
| tackled? yes.
|
| With that, I don't think git has any feature that is unsafe
| by default.
| sasham wrote:
| In that specific case there was some error that the user
| didn't understand, he googled and found a StackOverflow
| answer with --force. And naturally tried it BitBucket
| didn't have branch protection back then, today it's a bit
| better (you can still destroy your work but usually not
| others')
| nullstyle wrote:
| This was Pebcac my dude. git wasn't at fault here, the
| script kiddy that pastes before understanding is the
| fault. Amateurs
| Einenlum wrote:
| I agree that git is very complex (just try reading its
| documentation and how many options or commands you have
| never heard of before). But I think push --force is
| probably one of the easiest git concepts to get. The fact
| that someone in your team copy pasted something from SO
| without understanding it doesn't seem to be related to
| git. Otherwise we could say that the fact some people
| lose their data through "sudo rm -rf /" proves the
| complexity of Unix. I don't think so.
| IshKebab wrote:
| > With that, I don't think git has any feature that is
| unsafe by default.
|
| Well, you just mentioned `--force`. It is unsafe by
| default. Git has a couple of flags to make it safer
| (`--force-with-lease`, `--force-if-includes`) but those
| aren't the default.
| withinboredom wrote:
| If you've ever had to remove private information from
| history before making the repos public (think domains,
| names, configuration, etc) you will appreciate the
| ability to rewrite history (and all the other things
| --force gives you)
| IshKebab wrote:
| I don't get your point. Nobody is saying don't use
| `--force`. Just that the default `--force` flag is the
| most dangerous variant.
| kristjansson wrote:
| The feature is 'git push'. --force is the opt-in to the
| unsafe behavior. It should not be used lightly.
| IshKebab wrote:
| You're missing the point. `--force` is the default of the
| force variants. The other `--force-but-something`
| arguments clearly modify that default. It's the wrong way
| round.
|
| Obviously they've done it for backwards compatibility,
| but the fact that they haven't even added an _option_ to
| make it the default is pretty lame.
| CJefferson wrote:
| My biggest problem with git is branch deletion -- if you
| never do it you end up with far too many, but deleting a
| branch can't be version controlled.
| AdityaSanthosh wrote:
| Just curious, why do you want that to be version
| controlled?
| CJefferson wrote:
| Because I might realize later I made a mistake, or I
| might want to view history.
|
| I'd I never cared about historical state and mistakes, I
| wouldn't need version control at all :)
| couchand wrote:
| You could delete the branches locally while archiving
| them to any another clone of the repo.
| renewiltord wrote:
| It is somewhat version-controlled but not completely. If
| you use the reflog you can find it again and you can find
| how it moved around. But the reflog gets rewritten and
| gc'd so it's not true vc.
| couchand wrote:
| Should a chain saw come with the ability to start the engine
| disabled by default?
| IshKebab wrote:
| Yes. That is a great idea. You could do something like a
| tab that you have to remove that tells you about chainsaw
| safety.
| couchand wrote:
| The problem here is not the tool. The problem is the
| author's colleague's willingness to paste a stackoverflow
| answer into their terminal without taking a moment to
| understand what it does.
|
| If stackoverflow told them to break off the chainsaw
| safety tab there is no chance it would have been read
| first.
| dmazzoni wrote:
| But it doesn't lead to data loss.
|
| The commits that were overwritten by "force" are still there
| on the server. Any admin could recover them pretty easily.
| They're probably still present in the local repo of the
| person who ran "git push --force" too, as well as anyone
| else's machine who has cloned the repo.
|
| The only way you'd actually lose data is if every single
| person who had a clone of the repo ran gc.
|
| Or apparently if nobody knew about "git reflog" and nobody
| bothered to do a Google search for "oops I accidentally force
| pushed in git" to learn how to fix it.
| jsnell wrote:
| > Slow network: why is this a negative thing? If something is
| designed for a slow network then it should perform well in a
| fast network.
|
| Designing for resource-constrained systems usually means you're
| making tradeoffs. If the resource constraint is removed, you're
| no longer getting the benefit of that tradeoff but are paying
| the costs.
|
| For example, TCP was designed for slow and unreliable networks.
| When networks got faster, the design decisions that made sense
| for slow networks (e.g. 32 bit sequence numbers, 16 bit window
| sizes) became untenable, and they had to spend effort on
| retrofitting the protocol to work around these restrictions
| (TCP timestamps, window scaling).
| fourside wrote:
| That makes sense but then the pitch should include something
| about how back in 2005 the design for git had to make a trade
| off because of X limitation, but now that restriction isn't
| applicable which enables features A and B. I don't really see
| what trade offs a faster network enables other than making it
| a requirement that you have a network connection to do work
| (commits are a REST call). I'm not sure that's a trade off
| I'd want in my VCS, but maybe I'm just not the target
| audience for this.
| vintagedave wrote:
| >> a data scientist accidentally destroyed a month's work of
| his team
|
| > This is mostly a configuration issue
|
| git apologism :)
|
| (FWIW I do agree with the rest of your comment, and I hope you
| forgive the slight joke. Product users, for any product are
| fallible humans. That might be fallible in accidentally
| deleting, or it might be fallible in forgetting to turn on the
| safety settings.)
|
| Very seriously, something like this should not be possible in a
| source control system. Data integrity needs to be built in _by
| design._
| devjab wrote:
| The issue with a lot of freedom and unopinionated tools is
| always going to be the multitude of ways to fuck up. On the
| flip-side, you may not like what choices are made if you're
| forced to use it in a certain way.
|
| We enforce a strict pull-request squish commit with four eyes
| approval only. You can't force push, you can't rebase, you
| can't not squish or whatever else you'd want to do. But we
| don't pretend that is the "correct" way to use Git, we think
| it is, but who are we to tell you how to do you?
|
| We take a similar approach to how we use Typescript. We have
| our own library of coding "grammar?" that you have to follow
| if you want to commit TS into our pipelines. Again, we have a
| certain way to do things and you have to follow them, but
| these ways might not work for anyone else, and we do
| sometimes alter them a little if there is a good reason to do
| so.
|
| I don't personally mind strict and opinionated software. I
| too think Git has far too many ways to fuck up, and that is
| far too easy to create a terrible work environment with
| JavaScript. It also takes a lot of initial effort to set
| rules up to make sure everyone works the same way. But again,
| what if the greater community decided that rebase was better
| than squash commit? Then we wouldn't like Git, and I'm sure
| the rebase crowd feels the same way. The result would likely
| leave us with two Gits.
|
| Though I guess with initiatives like the launch here, is two
| Gits. So... well.
| oblio wrote:
| > But again, what if the greater community decided that
| rebase was better than squash commit? Then we wouldn't like
| Git, and I'm sure the rebase crowd feels the same way. The
| result would likely leave us with two Gits.
|
| Meh, this is overrated. We'd end up with 2 Gits, and over
| time just one fork would probably take over, based on
| marketing, PR, dev team activity, etc. The second one would
| probably still be around but used by only a minor part of
| the community.
|
| Just because a thing has on paper many forks, does not mean
| those forks are equal. In fact, a situation with many major
| forks rarely survives the long term. See Jenkins vs Hudson,
| Firefox vs Iceweasel, etc. Most people will congregate
| towards one of the forks and that's it.
| MatthiasPortzel wrote:
| > Data integrity needs to be built in by design
|
| It is built into Git by design. Git keeps commits around for
| 90 days even after they're "deleted." This is why people who
| understand Git were so skeptical of OP's claim. The point
| that Git is confusing still stands, however.
| dmazzoni wrote:
| What if someone pushes something inappropriate? Shouldn't
| there be a way to delete it?
|
| As an example, what if someone pushes:
|
| - A private key or password - Copyrighted content - Illegal
| content
|
| In cases like this, it needs to be possible to remove the bad
| commit from the repository entirely.
| layer8 wrote:
| Yes, but this should be only possible by way of commands
| that make it abundantly clear what you are doing, e.g. `git
| delete <whatever>` with extra confirmation "Do you really
| want to permanently and irrevocably delete <whatever> in
| the master repository?", or a more obvious "recycle bin"
| that presents deleted branches/commits in familiar ways and
| with explicit expiration dates. But the Git architecture
| doesn't lend itself to that level of user-friendlyness.
| sasham wrote:
| Thanks! We're definitely not trying to bash Git, it's done a
| lot of good for software development and for sure is going to
| continue evolving.
|
| Git had much more edge when it was competing vs SVN and other
| centralized VCSs. With 10Mb networks (if you were in office)
| you could feel physical pain when committing stuff ><
|
| Reg how Git is not perfect in the cloud world - check out
| GitHub's blog post here about their cloud dev environment,
| Codespaces https://github.blog/2021-08-11-githubs-engineering-
| team-move...
|
| "The GitHub.com repository is almost 13 GB on disk; simply
| cloning the repository takes 20 minutes."
|
| Moving 13GB inside your own cloud should take seconds at most.
| The problem is the way Git works, it clones your entire
| repository into the container with your cloud environment,
| using a slow network protocol. With Diversion it takes a few
| seconds.
| dartos wrote:
| > We're definitely not trying to bash Git
|
| Using git with bash is the best way to use git (:
| gnarlouse wrote:
| Came here to make a similar joke
| andsoitis wrote:
| > Thanks! We're definitely not trying to bash Git, it's done
| a lot of good for software development and for sure is going
| to continue evolving.
|
| It is not about bashing git; it is about anchoring your
| argument of why Diversion is a better alternative around git.
| You're basically taking your game/arguments to their playing
| field, and thus will have an uphill battle for mindshre.
|
| Instead, consider reframing the playing field and mention git
| less (if at all). Something like "the future of version
| control is blah". Surprise us, talk to us about your vision
| for source control, or better yet, code and multi-discipline
| collaboration (e.g. between eng and design), etc.
| bruh2 wrote:
| I personally would not bother reading any "the future of X"
| if it did not address problems of existing tools. I know
| you're trying to give advice from a marketing pov, and it
| is good, but it's also inherently bulshitty - because its
| purpose is to net more sales rather than actually make a
| good argument
| asimpletune wrote:
| I'm not sure I understand this at all.
|
| > The problem is the way Git works, it clones your entire
| repository into the container with your cloud environment,
| using a slow network protocol.
|
| What about git's network protocol is 'slow'?
|
| I think I can also come up with a pretty simple experiment to
| prove or disprove this: 1. Fill a file with 13Gb of data and
| commit it. 2. Upload that to GitHub or wherever you want 3.
| Time how long it takes to clone and compare that to the real
| GitHub.com
|
| You will find the one we made takes 'seconds' (or minutes,
| depending on your network connection), while the the
| GitHub.com will take some time.
|
| So, same data, two different results? The difference in this
| experiment rules out the 'slow' network protocol as the
| difference maker. The real reason is that the GitHub.com repo
| will have hundreds or thousands of commits.
|
| Basically, the difference is the commit history, because
| that's how git needs to work. Git stores the diffs for the
| entire commit history, not just the literal files at the
| HEAD. I don't know what the network protocol has to do with
| that.
| yjftsjthsd-h wrote:
| It is perhaps worth pointing out that if you don't need the
| history you can just `git clone --depth 1` and save the
| network transfer and disk space.
| TexasMick wrote:
| It reminds of when someone told me git submodules are
| slow.
|
| They just forgot about shallow clones..
| nolist_policy wrote:
| If you use the dumb http protocol, both cases should be
| equally fast.
| asimpletune wrote:
| git clone https://github.com/github/docs.git 123.57s user
| 37.02s system 74% cpu 3:35.73 total
|
| git clone --depth 1 https://github.com/github/docs.git
| 3.37s user 1.83s system 35% cpu 14.521 total
|
| Not a scientific test at all, but the second one was
| literally 15x faster, wall clock time.
| funcDropShadow wrote:
| That article also states that using a standard Git feature,
| shallow clones, you go from 20min to 90s. Most of the
| problems touched upon in the article are about state
| management for local environments, yes that can be tricky.
| And it can take time, but it has nothing to do with Git.
| IshKebab wrote:
| > When Micorosoft adopted Git for Windows, they faced this
| problem and solved it.
|
| _On Windows._ On Linux Git still doesn 't scale well to very
| large repos. Before you say "but Linux uses git!", we're
| talking repos that are much bugger than Linux.
|
| Also the de facto large file "solution" is LFS, which is
| another half baked idea that doesn't really do the job.
|
| You sound like you're offended that Git isn't perfect because
| you like it so much. But OP is 100% right here; these are
| things that Git doesn't do well. It's ok to really like
| something that isn't perfect. You don't have to defend flaws
| that it clearly has.
| graemep wrote:
| How common are repos bigger than Linux?
|
| Linux also has the huge advantage of an ecosystem, tools and
| integrations. It is overkill for small projects and there are
| friendlier alternatives for those - but git wins because it
| is what everyone knows. Something aimed at the small number
| of large projects will suffer the same problem.
| gertop wrote:
| > How common are repos bigger than Linux?
|
| In terms of number of commits, Linux is probably bigger
| than most. In terms of storage size, almost any video game
| project will be significantly bigger.
|
| It's no secret that git is very bad at handling large
| binary files.
| Spivak wrote:
| You don't even need game assets, your company's icon
| library is likely enough to tip the scales into territory
| git doesn't handle well.
| graemep wrote:
| So this is very specifically for things like games with
| large binary assets?
| IshKebab wrote:
| No, large companies using monorepos will have repos much
| bigger than Linux even without large binary assets.
| Apparently Linux has ~10 commits per hour. I probably do
| ~10 commits per week. So a team of ~150 mes produces
| commits at a fast rate than Linux. Very rough estimate
| but it takes less than you'd think.
|
| Also if you vendor a few dependencies that quickly
| increases the size.
| WorldMaker wrote:
| >> When Micorosoft adopted Git for Windows, they faced this
| problem and solved it.
|
| > On Windows. On Linux Git still doesn't scale well to very
| large repos.
|
| All of Microsoft's solutions for git scaling have been cross-
| platform. Even VFS had a FUSE driver if you wanted it, but
| VFS is no longer Microsoft's recommended solution either,
| having moved on to things like sparse "cone" checkouts and
| commit-graphs, almost all of which is in mainline git today.
|
| I also find it funny the complaint that git scales worse on
| Linux than Windows given how many Windows developers I know
| with file operation speed complaints on Windows that Linux
| doesn't have (and is a big reason to move to Windows Dev
| Drive given the chance, because somewhat Linux-like file
| performance).
| IshKebab wrote:
| `fsmonitor` is still only available for Mac and Windows.
|
| https://git-scm.com/docs/git-config#Documentation/git-
| config...
| WorldMaker wrote:
| Fair enough, though there is a hook to provide your own
| on Linux: https://git-
| scm.com/docs/githooks#_fsmonitor_watchman
| Wytwwww wrote:
| > really like something that isn't perfect. You don't have to
| defend flaws that it clearly has.
|
| Certainly true. But it's not clear at all how does the
| product solve these specific problems (they say "Painless
| Scalability" which sounds nice but did they try developing
| any 100+ GB projects with massive numbers of commits/branches
| on it?)
| lifeofguenter wrote:
| Have you ever tried running Git in the cloud? :)
|
| Cloud-native and running things on "EC2" are very different
| things.
| sasham wrote:
| Yep :) Lots of products run Git on EC2/containers, e.g.
| GitPod or GitHub Codespaces. Ironically, Diversion works much
| faster on these than git
|
| https://github.blog/2021-08-11-githubs-engineering-team-
| move...
| Kiro wrote:
| Git is bad for games and they should definitely compare them in
| their pitch if they want to capture that market.
| kfrzcode wrote:
| No, it's not. LFS has improved over the years. Git is
| supported as a first class citizen in Unreal Engine 5 -
| alongside P4.
| Kiro wrote:
| Good for developers using Unreal Engine 5 I guess. Fact
| remains that most game developers struggle with Git.
| Arelius wrote:
| Just because it has integrations, doesn't make it great.
| LFS is still not great. Doesn't have a lot of backends for
| instance. And a real locking system is table-stakes for a
| gamedev VCS
| jasfi wrote:
| The complexity people think they face with Git can often be
| overcome with a good UI and/or tutorials.
| sasham wrote:
| In part yes, e.g. lots of people like SourceTree. Some of the
| complexity is inherent though, e.g. local vs remote branches
| and the various conflicts & errors as a result. Git exists
| for 18 years, and yet the complexity problem wasn't solved
| yet. Other tools like SVN were never considered to be so hard
| to use / easy to screw up.
| aseipp wrote:
| The Windows Git repository is only 300GB, that's basically
| childs' play when people are talking about "large repo
| scalability". Average game developer projects will be multiple
| terabytes per branch, with a very high number of extremely
| large files, _and_ very large histories on top of it. Git
| actually still does handle _large files_ very poorly, not only
| _extremely large repos_ in aggregate. The problem with large
| Git repositories is nowhere near solved, I assure you.
| laeri wrote:
| This includes assets right or some kind of prebuilt data in
| custom formats? Otherwise it would be hard to have this much
| data in source files.
| aseipp wrote:
| Yes, game development studios include their raw art and
| environment assets directly in source control, just like
| source code. That's because the source code and the assets
| for the game must go together and be synchronized. That
| also includes things like "blueprints" or scripting logic.
| Doing anything else (keeping assets desynchronized or using
| a secondary synchronization tool) is often an exercise in
| madness. You want everyone using one tool; most of the
| artists won't be nearly as technical and training them in
| an entirely different set of tools is going to be hard and
| time consuming (especially if they fuck it up.)
|
| But honestly, you can ignore that, because Git doesn't even
| handle small amounts of binary files very well. Ignore
| multi-gigabyte textures and meshes; just the data model
| doesn't really handle binary files well because e.g.
| packfile deltas are often useless for binaries, meaning you
| are practically storing an individual copy of every version
| of a binary file you ever commit. That 10MB PDF is 10MB you
| can never get rid of. You can throw a directory of PDFs and
| PSDs at Git and it will begin slowing down as clones get
| longer, working set repos get bigger, et cetera.
|
| The 300GB size of the Windows repository is mostly a red
| herring, is my point. Compared to most code-only FOSS repos
| that are small, it's crazy large. That kind of thing is
| vastly over-represented here, though. Binary files deserve
| good version control too, at the end of the day.
| Mizza wrote:
| Nice. I make music and use git for my files, but often hit the
| usual space limitations and I hate dealing with git-lfs. I think
| there's a big use case for Diversion and music/video production.
| sasham wrote:
| Nice! We didn't even know about that. Thanks!
| Mizza wrote:
| Yeah, collaboration in music and video production is pretty
| unsolved. It often involves sharing huge files in
| GDrive/Dropbox, lots of
| project_latest_january_version_3_final_modified_updated.zip
| etc
| sasham wrote:
| Final_version_4_really_final, we've all been there :D
| funcDropShadow wrote:
| Did you try git-annex instead of git-lfs?
| sn0opy wrote:
| I assume the space limit your are talking about is rather a
| Github/Gitlab/Bitbucket limit.
|
| May I ask, how you use Git for your music? Since binary diffs
| are not really a thing, the main benefit I could imagine is
| simply having revisions of your files with you having to create
| folders or having files named "track01_test_final2.flac"
| Mizza wrote:
| I literally just `git add .` and let her rip. It's not
| pretty, I just git pretty much everything in my life.
| vintagedave wrote:
| > I often wondered why Git is so difficult to learn, compared to
| other tools.
|
| Yes! This is something many people wonder -- other than those who
| love git :) I used to love Mercurial and I still mourn its
| mostly-loss. So I welcome a new DVCS system that is friendlier
| than git.
|
| > Diversion's code is managed on Diversion!
|
| This is a good sign.
|
| > can synchronize with existing Git repositories (each new commit
| in Diversion goes into Git, and vice versa)
|
| Can you expand on what this means please? Does it simply mirror,
| or is it feature-for-feature compatible? Is it a backup
| capability?
|
| > still using legacy tools like SVN and Perforce
|
| I won't argue re SVN, but Perforce? It's used in the game
| industry primarily for its excellent handling of binaries / large
| binaries. How well does Diversion handle that kind of thing --
| multigigabyte data sets, frequently changing?
|
| The site says large files are fine, but that's too vague for
| games, IMO. Large, frequently changing, binary-not-text hard-to-
| diff files?
|
| Edit: one final question: why cloud only? Why not software that
| can be locally hosted, or hosted by (other) service providers?
| What if I love Diversion and want to run a Diversion setup on my
| own Linux box in a cupboard?
| 4star3star wrote:
| The difficulty of git is that it can do so much in so many
| different ways. What's actually needed is something like a
| linter for git workflow, where an org can enforce an
| opinionated subset of git's capabilities in a prescribed order
| of operations.
| sasham wrote:
| The Git sync feature allows one to import an existing git repo
| (currently GitHub is supported) into a new Diversion repo, and
| keep both in sync: every commit into Git is imported into
| Diversion and vice versa.
|
| This allows a member of a team that works with Git to try
| Diversion, to keep backups, and to use GitHub Actions or other
| CI tools that work with git.
| sasham wrote:
| Perforce is amazingly scalable, but that comes with a price
| (literally and figuratively :)) We still can't handle petabyte
| repos like P4 does, but we'll get there. But we can handle very
| large and frequently changing binaries pretty well.
| sasham wrote:
| Right now we're trying to get it to users who prefer something
| that just works, and don't want to think about hosting. But you
| can actually run Diversion in a container (that looses
| distributed storage and DBs, which means it won't be as
| scalable). Other providers would definitely be great, if we
| succeed in standardizing it like git.
| jacky2wong wrote:
| I'm having a hard time imagining positioning here. Can you
| explain further how Diversion differs from DVC, Git and when
| using Diversion makes sense over other use cases. The GTM is
| slightly confusing to me (also yes - Git is hard - you cannot
| teach data scientists this. It'll take months).
|
| Also agreed git is terrible right now for version-controlling
| workflows in AI (I have a fairly large .gitignore file with
| S3-hosted things ever for my NextJS + FastAPI apps - pain in the
| butt
| egaldv wrote:
| The vast majority of version control system uses are not
| distributed, even if the system itself is (GitHub and BitBucket
| were born to essentially make Git centralized). An example use
| case is game studios having repos with very large histories
| (hundreds of GiBs and more) where the tip is significantly
| smaller. Having the entire repo history on your local machine
| might be infeasible, and usually unnecessary. Being able to get
| just the tip and get the rest via API calls solves this. Having
| things continuously synced has other benefits like preventing
| conflicts at the time they happen on files that are hard for
| conflict resolution like game scene files, graphics etc.
|
| AI workflows are definitely a use case we are looking at in the
| near future. What types of files are you hosting on S3?
| darrenf wrote:
| > _An example use case is game studios having repos with very
| large histories (hundreds of GiBs and more) where the tip is
| significantly smaller. Having the entire repo history on your
| local machine might be infeasible, and usually unnecessary.
| Being able to get just the tip and get the rest via API calls
| solves this._
|
| "Being able to get just the tip" is `git clone --depth 1`,
| isn't it?
| egaldv wrote:
| And then you lose functionality, for example `git blame`
| depends on the history being available locally. If you want
| a working repository with all the source control features
| you need a regular clone. That's where "being able to get
| the rest via API calls..." kicks in :)
| darrenf wrote:
| OK so `git clone --filter=blob:none`, then. That
| downloads the tip _and_ commit history, but no historic
| blobs. `git blame` then works by downloading missing
| blobs on demand, which doesn 't sound too different to
| making an API call.
| fragmede wrote:
| Which, yes, but now we're at the "instead of using
| Dropbox I would just rsync to my Linux server" stage of
| it being a product.
| plesiv wrote:
| > In our previous startup, a data scientist accidentally
| destroyed a month's work of his team by using the wrong Git
| command.
|
| I'd like to hear about how this happened. No one in the team
| heard of reflog?
| meibo wrote:
| Being deceptive about Git's shortcomings is going to raise
| eyebrows with anyone seriously evaluating your solution, which is
| already going to raise eyebrows because it's not free software.
|
| Most studios will try to avoid locking themselves into another
| expensive, annoying VCS that they have no control over. There's
| good attempts at FOSS p4 replacements now, you need to do better
| if you want to stand out.
| maccard wrote:
| > There's good attempts at FOSS p4 replacements now, you need
| to do better if you want to stand out.
|
| There are? Like what?
| Arelius wrote:
| Yes Please. I would switch my studio today if this existed.
| ryanlitalien wrote:
| Hi, why the switch? Purely cost? Or admin overhead?
| maccard wrote:
| Yes, to both. P4 itself is solid, but it's a very chatty
| protocol and is very latency sensitive. Running a master
| in the us, with clients in europe is painful for everyone
| involved. Replicas and edge servers come with other
| tradeoffs too.
|
| As a developer, doing things like "I only want this
| subtree of the stream" is hard. Virtual streams exist,
| but they have a (non-negligible) overhead on the server.
| It has some quirks due to it being 30 years old which
| make it... interesting, to work with sometimes.
| oDot wrote:
| I've been attempting this but have paused the efforts due to
| what seems like lack of interest -- giving me an indication
| that the problem is not that big of a deal.[0][1]
|
| If anyone is interested in my resuming them, let me know at
| contact at weedonandscott dot com
|
| [0] https://www.reddit.com/r/vfx/comments/11s08ne/your_opinio
| n_o...
|
| [1] https://www.reddit.com/r/gamedev/comments/11s5haf/what_do
| _yo...
| oblio wrote:
| > Being deceptive about Git's shortcomings
|
| Are they being deceptive? I'm not sure I see it.
| jayd16 wrote:
| Not deceptive but I would say they're being hand wavy.
| umanwizard wrote:
| The story about irrevocably losing data at least was not true
| (already admitted by the authors).
| Spivak wrote:
| I think I would give them that one, they were only able to
| recover the data because they got lucky, not because the
| system was designed to not destroy data. I would go even
| further coming from my infra background, even if you don't
| truly lose production data, if you have to reach into your
| DR backups that's a failing of the systems in place that
| come before it.
| gosub100 wrote:
| I'd say git is deceptive: "checkout" doesn't mean "delete my
| stuff" any more than checkout a book from the library means
| throw it in a wood chipper.
| lawgimenez wrote:
| > In our previous startup, a data scientist accidentally
| destroyed a month's work of his team by using the wrong Git
| command.
|
| Can someone explain how is this possible? More importantly was
| there any git branching strategy and permissions?
| ryukoposting wrote:
| It _could_ be done with force-pushes, if nobody has a commit
| number for the old tree.
| kaufmae wrote:
| Not sure, even then, doesn't reflog keep it quite a while
| locally on the machine that the force push was sent? Maybe he
| did not commit his changes for a month and sent a git reset
| ---hard
| ryukoposting wrote:
| Your criticisms of git seem off-base, but I like the idea. I
| worked on some indie game dev teams in college where I was the
| _only_ person with any programming experience. Git was difficult
| for my teammates to wrap their heads around, although they did
| figure out Github 's GUI. A more non-tech-person-friendly tool
| would be nice to have in the space.
| gumby wrote:
| It's about time someone revisited & reimagined version control.
| the previous generations each lasted about 15 years: SCCS/RCS ->
| CVS -> Bitbucket/git/mercurial -> ??? so I am glad to see this.
|
| I would start by talking about what is great about Diversion --
| what it lets you do that you couldn't before.
|
| Since you mention gaming and perforce I looked in vain to see if
| it supports binaries (a major limitation of git -- just simply
| not in its design space). "Binaries" can actually mean for some
| people compiled code -- not for me but I understand why people do
| it -- as well as images, data files, Word files etc.
|
| Sounds like the second is scaling but you don't say what you mean
| by that. Git scales pretty well until either the repo & its
| history gets enormous or when there are a _lot_ of people making
| simultaneous changes.
|
| The realtime collab integrated with a version control mentality
| could be interesting -- a major problem with google docs is the
| lack of useful version control (even Word is better).
|
| And why cloud native?
|
| Once you've done that you need only briefly mention "why not just
| use git instead?"
| pas wrote:
| ??? might be https://pijul.org/ with its commutative
| awesomeness
| sasham wrote:
| Thanks for the feedback! Totally agree. Diversion does support
| binaries, should have mentioned it directly (will update the
| website).
|
| The thought behind cloud native is that workloads and devtools
| and data are moving there, and we want Diversion to be the best
| choice for when everything is in the cloud. Besides that cloud
| storage and DBs allow us to build and iterate much faster, and
| worry less about scalability, data distribution & storage etc.
|
| But we can also run Diversion locally / in a container, it just
| won't be as scalable.
| oblio wrote:
| > SCCS/RCS -> CVS -> Bitbucket/git/mercurial
|
| You skipped Subversion/SourceForge. People forget but for about
| 5 if not 10 years, SVN was the biggest SCM in town, next to P4,
| P4 having more of a hold in the game dev world.
| gumby wrote:
| I not only did not forget about subversion, I funded it in
| part. But it was of the CVS generation.
|
| I did leave out the proprietary things like Perforce and Aide
| de Camp, Solidworks PDM and the various in house things.
| Life's too short!
| oblio wrote:
| You also forgot Microsoft Visual SourceSafe :-p
| VyseofArcadia wrote:
| > And why cloud native?
|
| And what even is cloud native? When "cloud native" isn't just
| marketing, it seems to have all sorts of different meanings.
| sasham wrote:
| We're using S3 storage, lambda and ECS compute, and
| serverless DBs. It allows us to build a scalable product much
| faster, and leverages cloud features like multizone backups
| and distribution without having to develop these ourself. It
| also allows fast transfer of data between Diversion and other
| cloud systems.
| gumby wrote:
| Those are things that matter to _you_ not your users.
|
| You did talk a little (in a comment or your post I don't
| remember) that you can use various cloud APIs to integrate
| into other systems.
|
| But at the moment, from what you're telling me "cloud
| native" is as interesting to me as how you format your
| source code.
| withinboredom wrote:
| So basically not self-hostable.
| hasty_pudding wrote:
| why is it about time??
|
| VCS is sort of a solved problem like SQL.
|
| it's like saying it's about time someone revisited those
| Javascript frameworks.
| yencabulator wrote:
| You are not competing with Git, you are competing with Perforce.
|
| (Personally, I would never use a proprietary cloud-only offering
| for version control.)
| oldpersonintx wrote:
| > (Personally, I would never use a proprietary cloud-only
| offering for version control.)
|
| yeah, I have no idea who would want to replace a free, open,
| well-debugged, featureful, ubiquitous vcs with a closed,
| immature SaaS offering...if your sourcecode is valuable, you
| don't let a YC startup gate-keep it (sorry)
|
| git has terrible ergonomics for newcomers, but UI is exactly
| what the various git forges solve
|
| for experienced devs, git issues get internalized like the
| issues with every other tool
|
| a more realistic approach is the jujutsu stuff google is
| working on...works WITH git to create a better workflow
| decide1000 wrote:
| Your core feature is live updates of changes. That is actually a
| nightmare. I don't want live updates. It will be a mess.
|
| If you really want to do cloud. Use git.
| sasham wrote:
| Thanks for the feedback! If you're on a separate branch it
| doesn't update from main automatically. We're also thinking to
| make live update of main optional as well, this is actually one
| of the most debated features internally. Apparently some users
| like it and some really don't.
| leowoo91 wrote:
| how is that different than Dropbox version history?
| sasham wrote:
| Branches is one example! But really Dropbox is not a VCS.
| Although some game studios do use Dropbox/Google Drive for
| versioning graphical assets - but not because it's a good tool
| for the purpose, they just don't have a better one.
| oblio wrote:
| Plus, for a Linux user, you can already build such a system
| yourself quite trivially by getting an FTP account, mounting it
| locally with curlftpfs, and then using SVN or CVS on the
| mounted filesystem. From Windows or Mac, this FTP account could
| be accessed through built-in software.
| jupp0r wrote:
| Having worked with SVN a decade ago and Perforce more recently:
| that part of the market is waiting to be disrupted. I'm a little
| unsure whether it was actual technical reasons (vs cultural) that
| kept git out of those use cases. Many devs were working with git
| locally and using git-svn or git-p4 to interact with the local
| repo. Best of luck!
| sasham wrote:
| Thanks!! Having talked to lots of game studios (and other
| companies with large repos/files) - many of them tried to
| switch to git because devs wanted to, and failed because of
| technical limitations.
| llanowarelves wrote:
| Indeed the choice for me as recent as 1-2 years ago was still
| SVN vs Perforce. Despite only having ever worked with Git
| spacecrafter3d wrote:
| This looks really cool! Well done! Definitely looking into this
| for my startup. Currently we use a mix of Git and Perforce, but
| Perforce is a pain to maintain and very difficult for the non-
| coder on our team to work with for art assets.
|
| When we started I looked at Plastic SCM (owned by Unity)--looks
| like it targets a similar use case to Diversion. I honestly can't
| remember why I moved away from it--I think it was the lack of
| polish and capabilities vs git/GitHub. I'm curious how you
| compare yourselves to them.
|
| I really like the bidirectional git sync you mention so I can try
| it gradually. Pricing seems good.
| sasham wrote:
| Thanks!! What do you work on?
| spacecrafter3d wrote:
| [Skyglass](https://www.skyglass.com), real-time Hollywood VFX
| on mobile.
| buremba wrote:
| Rather than "Better GIT", it would make sense to position it as
| "better GIT backend" in my opinion. I get the desire to replace
| GIT bash but it might be harder to convince developers, who is
| your target audience.
| sasham wrote:
| Yes you might be 100% right. We were debating this a lot in the
| beginning. In the end we decided to build a separate VCS with
| Git sync, and simpler UI (for now at least - might change,
| depends on user feedback)
| yanis_t wrote:
| > a data scientist accidentally destroyed a month's work of his
| team by using the wrong Git command (EDIT: we were eventually
| able to restore from a non-updated repo clone, after a few hours)
|
| This is actually reads like a benefit of using Git. It's really
| really hard to lost something completely in Git, because reflogs,
| because there are multiple people on your team each has the same
| copy of the repo, etc.
| sasham wrote:
| For sure, but that requires a lot of expertise and leaves the
| non-experts free to shoot themselves in the foot...
| fidotron wrote:
| As others have mentioned I believe you underestimate the security
| requirements (often physical) imposed on game studios. Running P4
| (and using the often publisher mandated tape backups) is the
| least of your problems in such an environment.
|
| If diversion were to succeed it would need a huge security team
| because it would represent far too tempting a target. Games stuff
| inspires a level of attack which normal businesses simply do not
| encounter.
| sasham wrote:
| Thanks for the feedback! Totally true, I was surprised by how
| much emphasis on security there is in game studios. They are
| moving to cloud however, and we're hoping that a private cloud
| solution (Diversion running on their cloud account) would
| satisfy the requirements, at least a few years from now as
| cloud usage grows. Security is definitely going to be super
| important for us, in any case.
| fidotron wrote:
| You need to stop worrying about the cloud and focus on being
| agnostic to things like VPNs, on premises and so on. (Also
| SSO mechanisms). Any cloud needs to be optional. If you can
| make it easy to deploy on a private cloud where everyone
| accesses it via corporate Google accounts but also deploy on
| prem (say containerized) with Active Directory integration
| you will cut a lot of noise.
|
| Game artists will increasingly not trust the cloud as it is
| where their data goes to train AIs, but more pressingly a
| large proportion of asset development is done where the
| network (and electricity) is surprisingly flaky.
|
| To be specific I have been involved in several situations
| where we were not allowed to mention things in email due to
| it being cloud hosted, and those restrictions were imposed by
| companies that are themselves cloud providers. That is the
| kind of level being discussed.
| egaldv wrote:
| Thanks for the feedback! We do hear that remote artists
| with flaky network connections is common, and plan to
| address it in several ways like CDC (a method for efficient
| chunking and diffing of binary files to minimize network
| transfer and storage duplication), local network caches or
| peer-to-peer transfers. Private cloud deployment or on-prem
| with SSO will also be available.
| osigurdson wrote:
| >> most notably in games development, semiconductors and
| financial services are still using legacy tools like SVN and
| Perforce
|
| I think this should be your elevator pitch. Don't focus too much
| on "git complexity" as most people already know git so it just
| creates an argument. Scalability, in terms of numbers of users is
| somewhat hard to argue as well (Linux kernel has 1000s of
| contributors). However, it is completely true that git does not
| natively handle large binary assets well. You can even quote
| Linus:
|
| "I really don't know what to do about huge files. We suck at
| them, I know."
| oblio wrote:
| > Don't focus too much on "git complexity" as most people
| already know git so it just creates an argument.
|
| I'd say this phrase is both right and wrong.
|
| It's right in the sense that it creates an argument.
|
| It's wrong in that it creates an argument with the peanut
| gallery of git experts. But guess what, most people using git
| are not experts. They're software developers who don't want to
| learn the intricacies of git (probably most software developers
| out there), they're software development adjacent folks (think
| data scientists, etc) who for sure don't want to learn the
| intricacies of git, etc.
|
| The "common person" using git will most likely resonate on the
| "git complexity" argument.
| osigurdson wrote:
| It is just a tough argument to make: the thing you have been
| using for your entire career and used almost everywhere is
| suddenly too complex.
| oblio wrote:
| Ummm... A lot of people just endure using git. Go to the
| average enterprise software shop, the ones where people
| don't code for fun in their spare time, and ask around.
|
| There are a lot more of those devs than unicorn and FAANG
| devs.
| osigurdson wrote:
| Why aren't these teams choosing something else then? If
| everyone on the team dislikes git, switch to mercurial or
| something else.
| sasham wrote:
| Git might be the best thing there is today (outside of
| very large companies or environments with large
| binaries). It doesn't mean that'll always be the case...
| There were other VCSs before git, and there will be
| after.
| layer8 wrote:
| Because often it's not the team that chooses, but tooling
| is instead standardized across the enterprise. And
| enterprises like to make the "safe" choice of choosing
| what's most popular. And then there's the whole aspect
| that you have to know some basic Git anyway to debug your
| way through the open source code you use (maybe not for
| JavaScript/NPM, I don't know). Git also happens to
| currently be the most interoperable with other kinds of
| tooling, from CI to IDEs, so not using Git makes your
| life harder in ways unrelated to its inherent qualities.
| It's a network effect in multiple dimensions.
| withinboredom wrote:
| Learn your tools or one day you'll lose a finger, or worse,
| your life.
|
| -- high school shop class
| oblio wrote:
| Yet software is not a chainsaw and a huge amount of people
| never learn to use their software tools. Especially since
| they're 100x more complex than hardware tools and nobody
| has time to master everything.
| withinboredom wrote:
| If you are using a chainsaw in a wood shop, you are
| probably doing something wrong. The saying "learn your
| tools" means to spend some time learning your options and
| what is available to you, learning the "gotchas" and why.
| Woodworking tools are rather complex with "gotchas" that
| will kill you in less than a hundred ms.
|
| Using Git isn't much more complex than using a lathe
| (simpler even, as you can get by with no skill and rote
| memorization). Taking a weekend to learn the data
| structures, and how everything fits together is not a
| hard ask. Especially since you literally only have to do
| it once in your entire career.
| oblio wrote:
| > Taking a weekend to learn the data structures, and how
| everything fits together is not a hard ask. Especially
| since you literally only have to do it once in your
| entire career.
|
| Like all simplifications, this is false. If all you do
| for years after is commit, push, merge, you'll forget.
|
| Especially since you'll need those brain cells to learn
| the new CPU/GPU architecture, the new JavaScript
| framework, the new corporate security policy, the docs
| from your internal architecture team, etc.
|
| Nobody's life revolves around intricate VCS details.
| withinboredom wrote:
| You don't need to memorize it for life. Jeez man, don't
| be so hard on yourself.
|
| The point is, you know what is possible. You know what is
| impossible. 12 years later, something happens and you go
| ... hmm, I used to know what is going on. I think I need
| to search for something about git-tree or something?
|
| The point is, you know what to search for, a starting
| point. You don't just reach for git cherry-pick, but
| realize you can use git rebase --onto to copy/paste an
| entire branch. You don't worry about merge conflicts
| because you only have to do it once with rerere. You
| learn git reflog will remind you what branch you were
| working on this morning before you got pulled into some
| shenanigans in prod. You can set up automation with
| global hooks. There's so much you can know to do less
| work and you only need to remember the parts that are
| valuable to you.
|
| After learning git about 5 years ago stuff like ^ is all
| second nature to me, for nearly 10 years since switching
| from SVN. My first 5 years was just like you said.
| Commit, pull, commit, pull. I didn't even know it could
| do anything else and I was worse for it.
| philsnow wrote:
| > peanut gallery of git experts
|
| There are people who use git for its original purpose (kernel
| devs and very few others) and then there is the remaining 99%
| of people who essentially use "github flavored git", using
| only three or four git subcommands and for the most part
| never needing to understand its intricacies.
|
| Unfortunately, although they are using git-the-chainsaw-
| shotgun with all the safeties on, it's nonetheless a chainsaw
| shotgun and sometimes they'll run into issues where they or
| somebody in their company needs to be an expert and figure
| out how to un-scramble an egg, so to speak.
|
| If a new VCS can solve the 99% case and _never_ need users to
| fall back to understanding nitty gritty details, it could
| very well have strong takeup especially among people who
| don't give a crap about what VCS they're using as long as it
| gets out of the way and doesn't make them think (game devs,
| data folks, etc).
| oblio wrote:
| I see 2 huge barriers for a new commercial VCS:
|
| 1. Devs don't like to pay for core tools. And VCSes need
| network effects.
|
| 2. VCSes seem to be really hard.
| gavinhoward wrote:
| As someone who is making one (not Diversion), I agree
| with both of your points.
|
| Do you think that a VCS that solves the large file
| problem and binary file problem could succeed?
| hasty_pudding wrote:
| 1. git is not hard if you learn it
|
| 2. I regularly see people storing multiple gigabyte files in
| git.. I don't understand your issue with large files.
|
| 3. cloud native? why? part of the point of git is to have a repo
| decentralized away from centralized clouds onto dev's machines,
| if your data scientist broke git then that means your main branch
| configuration is off, no one else had that cloned onto their
| machines, and the rest of you don't know how to recover lost data
| which is stored in the reflog.... even ignoring the lack of
| simple CICD.
|
| 4. > Diversion is built on top of distributed storage and
| databases, accessible via REST API, and runs on serverless cloud
| infrastructure
|
| What!?!?!? That is NOT a selling point....all that for a VCS?
|
| You're literally taking everything that makes git great and
| saying that's what's wrong with git.
|
| The only case you could make to sell this is maybe some sort of a
| high-end niche specific media heavy version control for storage
| intense users.
|
| I would think that this is a college project but apparently you
| guys have a team size of 9 according to the website!!!
|
| This is wild!
|
| It kind of feels like an insult to Linus Torvalds.
| igorguerrero wrote:
| This comment is harsh, but the reality.
| dafelst wrote:
| I don't think you understand the scale of large creative
| projects like games. The current (only medium sized) Unreal
| Engine project I'm working on has about 300k files in the head
| alone, and a fresh sync for the stuff only required to
| build/cook the game is about 350GB. If you add the raw content
| from things like Maya, substance Painter, brush, etc, it is
| well in excess of 1TB.
|
| Vanilla git just does not scale to this. Microsoft git and
| Scalar get much closer, but require a bunch more setup and are
| a giant foot gun if someone tries to use the repo with vanilla
| git. Add to that the lack of permissions control and remote
| management, and it is just not a good fit for an industry where
| say 75% of the contributors are non technical.
|
| Git is a fantastic tool and works for the vast majority of
| software projects, but there are non trivial amount of projects
| out there that it just doesn't suit.
|
| Perforce, as horrible as it is, is the game and VFX industry
| default for a reason - think of it this way, given how
| notoriously cost driven and penny pinching games companies are,
| don't you think they'd be using a free/cheaper alternative in
| hosted Git if they could? But they don't. This is why.
|
| Good luck to the Diversion team, more competition in this part
| of the market can only be a good thing.
| hasty_pudding wrote:
| How niche is your use case?
|
| Despite all the questions I have about storing un-versioned
| large media files in Git instead of EC2 and your lack of good
| management about modularizing the code base and lack of build
| tooling....thats the use case I specifically stated in my
| post.
|
| The monstrosity that is Diversion could have great benefits
| for niche use cases like yours. i.e. large media file Version
| Control for teams who aren't knowledgeable about how to
| manage large code bases.
|
| If for some reason you want to pay an additional nine person
| team with a huge infrastructure instead of just using EC2 and
| a build script..feel free.
| Arelius wrote:
| EC2 and a build script? Are you implying that the parents
| 1TB of assets are "un-versioned large media"? If so, I'm
| sorry you are being very dismissive about something you
| don't fully understand.
| hasty_pudding wrote:
| well then modularize your code better!
|
| git can handle large repositories there's no reason you
| should have a one terabyte repository unless youre Google
| or something with the money to write your own VCS.
|
| but if you want to pay for a nine developer team with a
| massive cloud infrastructure for something like version
| control.... go for it!!!
|
| I'm going to check back a year from now and see if
| products still exists
| Arelius wrote:
| _sigh_ So, in GameDev, art assets are as important to the
| final game as code. And is, just as important to be
| version controlled as code.
|
| And sure, you can split it up between many different
| repositories, and call it _modular_ But you 're just
| adding complexity, and not really solving the core
| problem which is that a single version of an art assets
| can possibly be > 1G, and be a poor format to be stored
| delta-encoded, so each revision can possibly be nearly a
| full duplicate. (Yes, we can talk about the file formats
| being a poor fit for VCS, which is true, but GameDevs are
| just trying to make their product) And again, it's
| _really_ _important_ for game assets to be version
| controlled. And maybe Git isn 't the best place for the
| assets to exist in version control, but we are in a
| thread about Diversion and not Git.
|
| Now, don't get me wrong, I'm uncertain that Diversion
| will exist in a year when you check-back, But I can
| guarantee you that Perforce will, and if Unity doesn't
| continue it's trajectory of destroying everything it
| touches, Plastic should continue to exist also.
|
| I don't even completely disagree, Diversion, P4, and
| Plastic appear to be a bad fit for code in comparison to
| git, and even in our studio we've taken a modular
| approach (gasp) and store our code in git, while using a
| better suited VCS for assets. But it doesn't change the
| fact that you are being very dismissive for something
| that you very clearly know nothing about, and clearly
| have far too much ego to even be interested in a
| legitimate dialog about.
| hasty_pudding wrote:
| It's not rocket science to know that if you have a 1 TB
| git repo filled with massive binary files that all need
| to be version controlled, youre doing something VERY
| wrong or you are using the wrong tool.
|
| Literally the very first result on google search says
| almost every major game studio on earth uses perforce due
| to this specific problem because perforce allows you to
| check out binaries on a file to file basis and mark them
| with a change list. This appears to be a solved problem
| with numerous solutions.
|
| Theres also git-annex mentioned in the thread and a
| number of other options.
|
| I love that you're saying I have a big ego, but you're
| doing exactly one of the things I'm suggesting in your
| studio. lol
|
| This Diversion seems doomed if they're competing with
| git, but if they're competing with perforce, that's a
| different story. They're probably still doomed, but at
| least their product fit makes sense.
|
| https://www.reddit.com/r/gamedev/comments/2xc5fx/whats_a_
| com...
|
| https://git-annex.branchable.com/
| acidburnNSA wrote:
| > 1. git is not hard if you learn it
|
| I basically agree. People want to just pick up a tool like git
| and have it just work without investing much time in it. I can
| understand this desire in a complex world with many tools, but
| something as powerful as git is really worth learning. I swear
| if people spent like 1 hour a day for a week in actual git
| training, and then did refreshers 1 hour a month as they were
| using it on a team, this would be less of a problem.
|
| You don't need to know every feature, but knowing how to
| rebase, use the reflog, use the pickaxe, understand the
| concept/value of bisect and how to look up how to use it, etc.
| is so valuable.
|
| The other day a professional developer told me he didn't know
| you could write more than 1 line in a git commit message. WTF.
| hasty_pudding wrote:
| I don't even use that stuff 99.9999% of the time.
|
| You can literally get by with knowing like 5% of git and be
| extremely productive with it.
|
| that other stuff does come in extremely handy for
| troubleshooting or fixing mess ups, but people struggle with
| just the very basics of git.
|
| what you need to be productive with git can be learned in a
| morning.
| gavinhoward wrote:
| Disclaimer: I am designing a Git alternative too.
|
| Maybe "cloud native" will have a pull for game companies, but I
| am not so sure. I think a lot of studios would want to self-host.
|
| "Git compatible" is an interesting phrase; does Diversion use the
| same type of backing store? If so, I am not so sure it will
| handle large files as well as hoped.
|
| I had to solve this problem myself, and I did, but it required a
| different storage design.
|
| Can it handle binary files? Is there a plan for doing so beyond
| "commit the entire file every time" or "use xdelta"?
|
| I think this is how fourth gen version control systems will be
| defined. Game studios have a lot of binary assets, so this will
| be important.
|
| All in all, I could see a product like this succeeding, but I
| think taking VC money was a mistake because there may not be
| enough space in the market for a company that has to keep growing
| to satisfy investors. I am taking zero VC, and that will allow me
| to make money on as few as three clients.
|
| Anyway, I wish you both the best of luck!
| sasham wrote:
| Thanks, good luck to you too! Would love to check it out!
| Diversion has totally different storage, that handles binary
| files with no issues, but same concepts as git - branches,
| commits and tags. It has a Git sync feature that allows to sync
| commits between Git and Diversion repos. Kudos on bootstrapping
| the product, it's definitely not easy! Version control is much
| harder than it seems, you've probably found out already :)
| computershit wrote:
| > Maybe "cloud native" will have a pull for game companies, but
| I am not so sure. I think a lot of studios would want to self-
| host.
|
| Cloud native and self-hosting are not mutually exclusive.
| wokwokwok wrote:
| They are in this case.
|
| ...and, frankly, they are in most cases. Most "cloud native"
| apps are designed to run on cloud services like VendorHere
| cloud storage, cloud functions, cloud containers, etc.
|
| A vanishingly few of them are actually self hostable.
| ryanlitalien wrote:
| Can confirm, a lot of studios want to self-host.
| midnitewarrior wrote:
| > I am taking zero VC, and that will allow me to make money on
| as few as three clients.
|
| A VC would give you legitimacy to help you get the customers
| you need. What large customers are going to back you with their
| business unless they know there's some deep pockets behind you?
| There are few things more valuable than a business's source
| code, it encapsulates all of their business processes and is
| how all of the business's data is accessed.
|
| Why would a company want to entrust you with this? How are you
| protecting against data loss, and the legal liability that
| comes with this responsibility (insurance, legal)?
| gavinhoward wrote:
| You make good points.
|
| My VCS is designed for self-hosting, not cloud.
|
| I will support customer installations, not host customer
| source code.
|
| I am hoping that customers see less of a need for deep
| pockets in that case.
| Spivak wrote:
| This is a tried and true method of bootstrapping a SMB.
| Sell the bits in a box with support until you are big
| enough to sell the service (or don't).
| withinboredom wrote:
| I mean, insurance is cheap. I have professional insurance
| allowing me to personally do several million in damages due
| to a mistake. Or recover your losses if I give bad advice.
| All for the low cost of 50 bucks per month, purchased through
| a local dev-co-op.
|
| Trust isn't built by who backs you though. Does it make it an
| easier sell? Maybe, but as a buyer, if that's what you're
| leaning on to sell to me, I'm going to be put off.
|
| If it comes down to two, I'd trial both and care about how
| well it does before caring about who is backing who. In the
| end, I might choose the smaller company and negotiate access
| to source code to hedge them going under. And that sounds
| like an even better deal than some VC's trying to "monetize
| the fuck out of me" in three years.
| gavinhoward wrote:
| Oh, how might I find that professional insurance? Even
| though I am not going to host, I'd like something like
| that.
| pmontra wrote:
| As this is a startup the competitors are GitHub, Bitbucket,
| GitLab etc. and not the underlying technology. That is a mean to
| a goal, making money or most probably a lucrative exit.
|
| > [git] was built for [...] much smaller projects,
|
| It was built for the Linux kernel. There are larger projects than
| that but how many? Of course if you manage to make them switch to
| Diversion and get paid for that you could be well off even with a
| handful of customers.
|
| About technical matters, as a company I'd be very at unease to
| have all my code only in the cloud on somebody's else servers. At
| least with git there are dozens, hundreds, thousands of copies of
| any repository around the company. If GitHub or AWS close our
| accounts we can keep going on and move somewhere else.
| monsieurbanana wrote:
| I agree that the "project size" argument doesn't make much
| sense, however size as in file size does. There are multiple
| methods to add support for large binary files in git, but none
| are great.
|
| With their focus on game developers, I imagine that's one of
| the primary use cases.
| sasham wrote:
| Thanks for the feedback! We're offering a private cloud option
| for large customers, from many conversations this is OK for
| most of them. And adding a daily backup to any location is
| actually very easy.
|
| > At least with git there are dozens, hundreds, thousands of
| copies of any repository around the company.
|
| This is actually a huge issue for large companies (data
| breach), that doesn't have a good solution with Git.
| mgaunard wrote:
| The cloud is just a crappier version of a datacenter built around
| the notion that you can make users captive to your services and
| make them pay hardware for 3 times its price.
|
| It does not lead to better source versioning systems.
| nojvek wrote:
| Git cli UX made be not great, but the git datastructure of
| representing commits, branches, trees and blobs as immutable
| pointers and merkle trees is a phenomenal invention.
|
| I don't agree with every command needs to hit some REST api. That
| seems like throwing the baby away with bathwater.
|
| The most powerful thing about git is that I can work fully
| offline with a partial clone. And sync the commits when I get
| online.
|
| Git got popular because of it's distributed nature.
|
| I remember when our svn server used to hang and the entire team
| was blocked from even making a commit.
| sasham wrote:
| Yep, exactly! I remember these "happy" SVN days as well. It
| sucked, git was much better. Today you're almost never offline
| though, you even have wifi on airplanes. Git's datastructures
| are a masterpiece, for sure
| jedberg wrote:
| > Today you're almost never offline though, you even have
| wifi on airplanes.
|
| The airplane wifi is good enough to look up an API, but it's
| not good enough to sync a code repo, especially one with
| large assets.
| sasham wrote:
| For assets no, definitely. At least until they get
| Starlinks :D I did commit into our code repo from an
| airplane though.
| igorguerrero wrote:
| Taking a decentralized application like Git back to
| centralization is a step in the wrong direction, it's exactly why
| Git was created.
| egaldv wrote:
| Decentralization is a cool concept, but didn't GitHub and
| BitBucket emerge because a centralized server was in demand?
| Git is a good tool for many uses, but when was the last time
| you pushed directly to a peer's repo on their machine? How many
| firewall and reverse proxy configurations did you have to setup
| to be able to do it?
| igorguerrero wrote:
| It's kinda the standard procedure of securing your machine's
| ssh, I'd recommend trying to setup a bare repo and put it
| online so you see how easy it is to make it work, that's not
| Github but it's more than enough for your org, in fact before
| Github was created that's how we did it, it doesn't really
| take more than 1 afternoon to set it up, multi-user and all
| the other jazz.
|
| But Github and Bitbucket aren't examples to compare Diversion
| to though, Github is just a node in the decentralized network
| of Git repos (your users, or whoever cloned). They even use
| an example of this feature, how they were able to recover
| after a mess-up, then with SVN that couldn't be done and
| they'd have really lose more than a month of work.
|
| There's one of the reasons why Git was created, to make it
| easier to resolve conflicts, if you just want a main /
| children branches structure in one server then use SVN /
| Diversion but then don't complain when you have a branch
| "locked" and you cannot get your job done.
|
| Diversion makes 100% sense if you follow the silly "one repo
| for the whole org" way big companies do, which still doesn't
| make any sense to me.
| funcDropShadow wrote:
| > The biggest drawback of Git is its limited scalability - both
| in repository and file sizes, and the number of concurrent users.
|
| How did you come to the conclusion Git isn't scalable in the
| number of users? There is no limit in the number of users with
| Git. There may be limits in the number of interaction between
| those users, i.e. pull/merge requests, clones, fetches etc. But
| they are mostly limited by humans.
|
| A central system, even a cloud-native system, has far more
| contention points in operations that are not even interactions
| between users, i.e. status, commit, etc.
| nine_k wrote:
| The number of concurrent merges is definitely a problem. With a
| few hundred developers committing to the same repo all day, it
| becomes a chore to make sure that their commits can still be
| put on top of each other, and that one out-of-sync commit is
| not holding back a bunch of others which depend on it.
|
| This all is solvable, both through discipline and tools. But if
| a VCS has a built-in capability to alleviate this, it's a good
| thing.
| funcDropShadow wrote:
| True, that is one those interactions between users that I
| mentioned. But the amount of people having the code checked
| out and poking around its history is only limited by capacity
| to clone the repository. All other operations are just local.
|
| Having hundreds of committers merging into one branch is
| problematic. Perhaps splitting the repository into smaller
| parts does have its advantages. Monorepos often have the
| architectural split of the software while throwing away the
| advantages of the split at the vcs level. But there is also
| other large software, developed by hundreds of people, that
| cannot meaningfully split into separate git repositories
| without messing about versioning and releasing. The Linux
| kernel is such a system. They use a loose network of
| repositories -- mostly a hierarchy -- to pre-aggregate
| merges. Git is well equipped to handle different approaches
| -- separation of mechanism and policy.
|
| I am looking forward to learn how a central but cloud-native
| VCS will improve on that.
| __turbobrew__ wrote:
| The solution to that problem is a merge queue and GitHub
| supports that now. I do agree that it would be nice if the
| VCS solved the problem natively, but for many companies
| GitHub -- and not git -- is their VCS.
| dboreham wrote:
| Quick note to say that discussions like this are asymmetric: the
| git aficionados are very sure about their opinions that it's
| fine. The people who believe git isn't great are unsure if
| perhaps they're holding it wrong or don't understand how to use
| it. So what you see in a thread like this is 90% posts strongly
| asserting that the OP is wrong while the potentially large number
| of people on the other side are scratching their heads, not
| posting.
|
| fwiw, I think they're pretty much right.
| asimpletune wrote:
| I think this is how most git stuff goes as well.
|
| > fwiw, I think they're pretty much right.
|
| Is 'they' the strong posters or the head scratchers?
| ajross wrote:
| > We're planning to release it as open source once the code base
| matures
|
| FWIW, historically this has meant "we'll release it it we fail".
| If the product has actual market value and is attracting
| investment, you'll never win that fight with your backers unless
| you make the decision preemptively before they write their
| checks.
|
| To be clear again: I don't doubt your sincerity here. I'm just
| saying that by the time the "code base matures" it won't be
| solely your decision to make, and that I don't trust the other
| stakeholders.
| krautt wrote:
| any work done in this area is good work.
| krautt wrote:
| i'm thankful there's work being done on improving version
| control. git is great but if you don't respect it, it will punish
| you.
| ho_schi wrote:
| With _Git_ we can work autonomously, fast and distributed. The
| nice thing about Git is that it works on a B747* and the a cabin
| in the woods and on a file-server at MIT.
|
| I will not make myself dependent on a proprietary cloud-service
| which requires an internet connection and reoccurring payments.
|
| * Internet isn't fast there. If it is available. If it is
| affordable. If you want ruin the precious time separated from the
| internet.
| bzmrgonz wrote:
| If competing with git doesn't work out for you... let me tell you
| what the world needs... We need to bring freedom and liberation
| to the .docx nation. Microsoft has held that document format
| hostage for way too long. They keep changing that file format in
| order to keep competitors at bay. It is the most grotesque mote
| in existence in modern times. we need a document creation with
| versioning baked in but that abstracts those aspects and presents
| an intuitive way to switch from creation to publication/print.
| Most lawyers/docx-creators deal with at most 20 or so
| styles/formats. For anything else, there's publisher or adobe. So
| as you develop this project, if the uphill battle becomes too
| unbearable, consider creating a better document platform with
| versioning baked in.
| sasham wrote:
| I think Google Docs are doing a pretty good job there! I
| personally can't stand both Word and Excel so totally agree
| maliker wrote:
| Wow, tons of classic hacker news feedback just like the "dropbox
| won't work" one [0]. And we know how that ended up.
|
| With the trend towards more and more non-devs writing code with
| the help of AI [1], I think you're absolutely correct with your
| assertion that something safer and easier than git is needed. The
| rest of the business case makes perfect sense too (git struggling
| with binaries and large files, massive code bases at Meta/Google
| have required alternate tools).
|
| Love the open source strategy as well. Github showed pretty
| clearly that selling services over an open core can be a winner.
|
| [0] https://news.ycombinator.com/item?id=9224 [1]
| https://twitter.com/amasad/status/1659423752881586176?lang=e...
| sasham wrote:
| Thanks! I agree - there are lots of examples how simplifying
| something (Shopify, Uber, Airbnb) can dramatically increase the
| user pool to people who otherwise wouldn't even use a
| comparable service.
| jjtheblunt wrote:
| Why not just use Perforce?
| casercaramel144 wrote:
| Congrats on the launch! It's always exciting to see more
| competition in the version control space.
|
| One question I have is whether you guys are better than:
|
| https://desktop.github.com/
|
| This seems to do the exact same thing, be free forever, and have
| a more mature GUI that is also easier to use than regular
| terminal git. In my firm, even with people who don't know how to
| code, they can use github desktop (since it babies you through
| the process of committing code.)
| asimpletune wrote:
| They should call it a new concurrent version system, not a git
| alternative.
| DrBenCarson wrote:
| going after git takes COURAGE, so I'll give you that lol
| sasham wrote:
| Haha evidently :D
| jayd16 wrote:
| As a game dev I find the pitch unexciting.
|
| > git is bad we're better
|
| Honestly, a modern git lfs workflow is really smooth. I think it
| handles binaries fine. Show me cumbersome git feature and why
| this works better. You can't just tell me tools I use every day
| are unusable.
|
| I think the main pain of git is if you want to put everything in
| a single repo. Big isn't a problem, getting just what a I need
| (checking out a single large model) is the problem.
|
| From the website I have no idea if this can do partial checkouts.
| I assume yes but its not stated at all.
|
| > cloud native
|
| A lot of studios want on-prem and self-hosted private cloud
| support. Cloud native is touted as a feature but details are left
| out. That has me wondering if some things don't work when I try
| to host on-prem or that its an afterthought.
|
| Can I easily host this on my own k8s cluster? Its not stated.
| Cloud native doesn't mean it's on the internet
|
| Another feature that artists like is file locking. P4 has it and
| git lfs actually has it too. The heavier usage of P4 streams and
| branches makes it hard use locks effectively these days. Merging
| something that was locked but now isn't is sticky
| business...maybe you guys solved that.
|
| > File locking across branches - coming soon!
|
| "coming soon"... so close.
|
| Good luck to you guys but I think the pitch needs work.
| sasham wrote:
| Thanks for the feedback, glad you asked! Partial checkouts are
| supported. K8s not yet, but we do run Diversion in a container
| for testing. Private cloud works as well, it's more a question
| of support manpower though - will be available for large
| clients. Obviously we still don't have every possible VCS
| feature, we're just getting started :) But we are adding
| features pretty quickly (e.g. conflict notifications took a few
| days to implement). Thanks!
| Vermeulen wrote:
| Git LFS is a giant hack ontop of Git. Most game devs I know
| moved away from it over time (back to Perforce or SVN). It
| might seem okay at first - but deep into a project you'll want
| to rearrange/rename folders and keep history logs, and discover
| that Git LFS doesn't actually work like normal Git and your
| file history wasn't kept. Only once you start dealing with
| issues will you find all the weird hacks Git LFS does ontop.
|
| I'd say Git not working well for game dev isn't a pitch that
| Diversion needs to make, because it's already clear to most
| game devs.
| jayd16 wrote:
| Its gotten a lot better over time as far as adding tooling to
| make big changes. Its the same ol' 'its fine if you know what
| you're doing and not so fine if you don't.' Personally I'd
| rather deal with git's issues but p4 has the a lot of built
| in support in engines.
|
| Day to day I think its the industry tooling and the partial
| checkouts that have people pick P4, not esoteric problems you
| face years down the line.
| stevelacy wrote:
| Game dev here, totally agree.
|
| File-locking across branches excited me, support for visual
| diffing of uassets would be insane. The platform screenshots as
| a git/github alternative didn't excite much interest as Plastic
| and Perforce are major players.
| sasham wrote:
| Other game devs mentioned visual diffing as well, we'll try
| to make this happen!
| cebert wrote:
| This sounds like a solution in search of a problem. I've never
| encountered any of the problems suggested with git in this post.
| calderwoodra wrote:
| The world needs a git alternative. Anyone who has used mercurial
| at Google or Facebook knows the tooling could be much better.
|
| As soon as you have 2-3 people committing to the same repo daily,
| git falls apart fast. The biggest difficulties with git are
| merging and branch rebasing. If git could do rebases better, I
| would suspect software development teams to universally move
| about 20% faster.
| hasty_pudding wrote:
| Rebasing is extremely easy with git. Sounds like more of a
| skill problem then a git problem.
| mikepurvis wrote:
| I think the issue comes about more when you have refactoring
| going on at the same time, and suddenly multiple feature
| branch maintainers need to figure out if they're going to
| merge in the main branch, rebase onto the main branch, if it
| makes sense to squash their changes first before attempting
| either path, etc. If they get only partway through either
| one, it's difficult work to pause and resume, so that
| integration work is fundamentally hard to collaborate on or
| even really review.
|
| And never mind maintaining an ongoing integration of multiple
| unmerged feature branches-- perhaps that's more of a "front
| end" issue for GitLab and Github to solve, but the whole
| business of patchsets and the like is very much unsolved, and
| that's painfully obvious when you see Debian storing their
| quilt patches inside of of a git repo, rather than being able
| to leverage git's native capabilities to achieve those
| effects: https://salsa.debian.org/debian/netplan.io/-/tree/de
| bian/0.1...
| TexasMick wrote:
| You can always merge feature branches together to check for
| conflicts
| dclowd9901 wrote:
| I don't understand how it isn't easy?
|
| "git merge origin/master"
|
| Done. Unless you have a conflict, then fix it and commit. It
| couldn't be simpler.
|
| If there's value in this product, it won't be because it's
| somehow simpler than git.
| withinboredom wrote:
| rerere has worked pretty well for me when handling conflicts
| and it's built into git.
|
| I've worked with hundreds of engineers in a single codebase.
| Never had any issues like you're describing.
| dmazzoni wrote:
| I've worked on projects with thousands of developers committing
| to the same repo daily.
|
| Git is far from perfect, but in my experience it's been far
| superior to any of the alternatives I used before it (cvs, svn,
| p4).
| TexasMick wrote:
| 2-3 people daily I would be completely surprised there's any
| issues.
|
| It suggests they're working on the same files and on the same
| lines, why??.
|
| My current project has 40+ developers and I haven't seen a
| merge conflict in a long time.
|
| Why do you think there's issues with merging and revising,
| pretty much all cloud solutions have a button to do both for
| you.
| kaufmae wrote:
| I just left a train, commited several changes to several repos
| and didn't have connecivity. Nope, i dont't want this.
| ta8645 wrote:
| There doesn't seem to be any justification offered on why it's
| not yet open source. Not that one is required, but it is
| suspicious that such a commitment isn't a priority, and that
| maybe there is a desire to keep closed-source as an option.
|
| Personally, I'd not want to check any assets into such a tool
| before it becomes open-source.
| emgeee wrote:
| Open sourcing code can require a lot of resources to do it
| right (manage the community, handle licensing etc). If you
| don't have customers asking for it and you don't need it for
| customer acquisition it might not make sense.
| niwinz wrote:
| Licensing: just choice a good safe default: apache2.0 or
| mpl2.0
|
| Open sourcing does not mean you want or going to accept
| external contribution, just be explicit that you don't accept
| external contributions.
|
| There are several open source projects that are developed
| with no public contribution policy. Nothing new to invent
| here.
| aaomidi wrote:
| > Licensing: just choice a good safe default: apache2.0 or
| mpl2.0
|
| No, this is how you get seriously annoying divergences in
| products like this. GPL2/3 would be 10/10. It would also
| protect the company too.
| myfonj wrote:
| Out of curiosity, are there any VCSs that operate on AST instead
| of plaintext lines? (Or is something like this being developed or
| proven impossible?)
|
| I guess it should be possible to cooperate on shared codebase
| without need for every contributor to check in and out text files
| following exactly the same formatting. Or even naming convention.
| Or even same language, provided all collaborators can transpile
| to and from some agreed-upon shared AST target.
|
| I know it might seem unhinged at first, but think about it: your
| (parseable) code is representation of the tree anyways (with some
| unrelated "whitespace fluff" around). If you follow strict
| formatting rules that you can express programmatically, you can
| recostruct that "fluff" from bare AST. If you can store all your
| violations against your style near the code, you can even sin and
| break it. If you store data about what you need to see
| differently from the shared AST - local renames of variables, for
| example - then you should be able to use your own naming
| convention, formatting and even source language, without
| bothering collaborators with tabs/spaces, hungarian notation or
| the fact that you prefer some different dialect or metalanguage.
| distortedsignal wrote:
| I was thinking about trying this out, but there are some
| reasons why I don't think it's feasible.
|
| Where are your comments stored?
|
| What happens when you need to run out in the middle of a fire
| and you don't have time to make your code compile-able? How do
| you commit "un-compile-able" changes?
|
| I think there are some really compelling reasons to try AST-
| checkin - all your loops can now be changed to functional,
| dialect changes like you mention, etc. - but there are some
| pretty significant downsides as well.
| n42 wrote:
| these are both already solved issues that IDEs deal with with
| red-green trees
| lowbloodsugar wrote:
| Nodes in the AST for comments, block comments and "raw text I
| don't understand" seems like a way to go?
| distortedsignal wrote:
| Honestly yeah. Might have to give this another go.
| Kharacternyk wrote:
| This would enable some advanced merge conflict resolution
| strategies, I suppose. However, it can also be done by building
| the ASTs on demand and still storing plain text.
| ajross wrote:
| > I know it might seem unhinged at first
|
| Not "unhinged". Most kids these days get their first
| introduction to computer programming using of of the many
| "block coding" environments, almost all of which are
| straightforward recapitulations of Javascript under the hood.
| And it works, and it avoids the problem of having to teach them
| how to deal with syntax errors before you teach them imperative
| logic.
|
| The reason people don't do this is that it's just a bad idea.
| The fact that all source code is stored in a universally
| understood data format with pervasive support across decades of
| tools is a feature and not a bug. How do you grep your AST to
| see if it's using some old API that needs to be refactored?
| Surely you'll answer that you use your fancy AST grep tool,
| which is not grep, and thus works differently for every
| environment. Basically every environment now has to have its
| own special editor, grep, diff, merge, etc... Even things like
| documentation generation and source control rely on files being
| text. And you're throwing all that out just to be different.
|
| Also, FWIW: it's optimizing the wrong part of the problem
| anyway. The total cost to an organization that develops and
| deploys software of any form is overwhelmingly dominated by
| tasks like debugging and documentation and integration. The
| time spent actually typing correctly-formatted text into your
| editor is a vanishingly small fraction of software development,
| and really that's all this helps.
| e12e wrote:
| I believe the smalltalk vcs Monticello work on a semantic
| level?
|
| https://eng.libretexts.org/Bookshelves/Computer_Science/Prog...
| nolist_policy wrote:
| You can do most of this in git via custom diff-driver and
| smudge/clean filters.
|
| For example git can already convert line-endings on the fly for
| windows. This is special-cased, but can just as well be
| implemented via smudge/clean.
|
| Oh and git-lfs is done via smudge/clean too.
| itslennysfault wrote:
| Little feedback on the "how is it different from Perforce"
| section of the web page.
|
| There have to be more advantages than "it's in the cloud"
| ...right? Also, for many this is potentially a huge hurdle /
| disadvantage. Who are you, and why should we trust you with our
| precious IP / code? (many game studios are on-prem very
| intentionally)
|
| Personally, I'm currently (being forced to) use Perforce and I've
| learned to tolerate it, but wish we could use git. There have to
| be more things that make your offering better than Perforce and
| you should really highlight them.
| grogenaut wrote:
| For me the absurd size of data in the game I worked on,
| terrabytes of video, mocap data, assets, made cloud hard and
| expensive. Cloud makes me think of all that time I'm blocked on
| network up and my poor data caps for my employees
| belthesar wrote:
| Something of note, is that Perforce Helix does offer a Git
| interface, with Perforce Helix on the backend. It's something
| I'm looking to explore at my studio to better understand the
| tradeoffs there.
| ryanlitalien wrote:
| It's called Helix4Git and you can mirror (and other options)
| your git repo into Helix Core using a "graph" depot type.
| Arelius wrote:
| Ok, I'll bite.
|
| Looks interesting. My first thought, as a gamedev who has been in
| charge of selecting VCS at multiple studios.
|
| 1TB is not a lot, and I would really appreciate clear public
| pricing terms on data before committing to a VCS. Esp at the
| smaller indie scale.
| Arelius wrote:
| Continuing:
|
| > Yes! It can integrate with any CI tool via Git compatibility,
| or using the API. Talk to us for more details.
|
| You really need to provide a comprehensive cli targeted to
| CI/Build. Git really is really going to be a bottleneck for
| games, and most teams aren't going to want to figure out the
| hassle of manually figuring out differential updates and multi-
| part downloads via REST
|
| Secondly, I watched your video, and dug through the website,
| and saw no mention of what your Windows story is. As a gamedev
| I'm going to need some assurance that your Windows support is
| top-notch.
| sasham wrote:
| Windows is supported, as is Mac and Linux - should mention
| that, you're right. Most of our users use Windows, so it's
| definitely 1st priority.
|
| Thanks for the feedback reg CI - I'd love to hear your
| specific needs, as we're working on this right now.
| Arelius wrote:
| I have a lot of thoughts on VCS and CI, feel free to reach
| out: indy@telltale.com
| sasham wrote:
| Hi, sure thing! You can get extra storage at the same price as
| the Free tier (we'll make it clearer on the website).
| Arelius wrote:
| Ahh, I see it... Being in the free tier, it also just looks
| like a feature, I might pull the price/100GB/mo into it's own
| cell to make it more clear. It is a pricing page afterall.
| DanielSlauth wrote:
| Looks awesome!
| mkleczek wrote:
| IMHO the only really interesting alternative to Git currently is
| Pijul (https://pijul.org) as it is not a more-or-less Git clone
| but a different approach to the problem itself.
|
| Pijul allows for very interesting development and ci/cd
| workflows.
| hintymad wrote:
| > Cloud-Native Git Alternative
|
| Not sure if this is a good summary of the product. For one,
| cloud-native is an implementation detail, unless the company
| plans to sell the new VCS as packaged software instead of
| service. For two, I'm not sure how being cloud-native addresses
| any issue with my daily interaction with Git.
|
| > The biggest drawback of Git is its limited scalability
|
| I wonder how many people really has this problem. Millions of
| people have been using github and gitlab. I'm curious about the
| percentage of users who feel that there is a scalability issue
| with _their own_ repositories. Personally, I don 't have any beef
| with git's scalability at all, even though the companies I worked
| for had anywhere between hundreds to tens of thousands of
| engineers. Maybe having a monorepo will lead to scalability
| problem? But monorepo is a debatable topic, to say the least.
|
| > Diversion is built on top of distributed storage and databases,
| accessible via REST API, and runs on serverless cloud
| infrastructure. Every repository operation is an API call
| (commit, branch, merge etc.). The desktop client synchronizes all
| work in progress to the cloud in real time
|
| Again, how does this have to do with me, a user? Why would I care
| about the underlying protocols when I simply use a CLI or a UI?
| sasham wrote:
| Monorepos are indeed causing problems with git, and this is one
| of the main arguments against them (see [1]). Some companies
| are building their own solutions (Google, Meta), and some are
| splitting their monorepos because of these problems. IMO if a
| company wants to run a monorepo for their reasons, they
| shouldn't be limited by their VCS.
|
| The technical details are for the readers who want to know, I
| agree it's not really important for the users (most of them, at
| least).
|
| [1] https://medium.com/@mattklein123/monorepos-please-
| dont-e9a27...
| foofie wrote:
| > Monorepos are indeed causing problems with git, and this is
| one of the main arguments against them (see [1]).
|
| The article is a disappointing read. It spends a lot of time
| talking about monorepos and how they spell all sorts of
| trouble. Yet, the article makes zero mentions of submodules
| as a way to get the best of both worlds.
| kristjansson wrote:
| Submodules are great, but they're hardly an alternative to
| monorepos.
| carlthome wrote:
| Why not? Just want to understand.
| sitzkrieg wrote:
| ive never seen positive feedback for submodules before
| hintymad wrote:
| Having an effective monorepo at scale will also require an
| entire infrastructure to solve all the problems that a poly-
| repo must solve and more. In particular,
|
| - Partial download, as a monorepo will quickly grow too large
| for a single person to download. This is trivial for poly-
| repo but requires dedicated system for monorepo.
|
| - Dependency management. With a decently sized monorepo, one
| can't compile everything and test everything. So, someone
| needs to build a dependency manager to track all the DAGs,
| and build only the DAGs that are impacted by a commit. One
| also has to build a trackign mechanism for deploying
| different build artifacts because a team may deploy all the
| build artifacts in different date and time. We will need more
| sophisticated build tools too.
|
| - Build infrastructure. Even with a perfect dependency-
| tracking system, we may still end up building large-enough
| source code that we need to build the code in parallel.
|
| - Directory-level access control. This is also trivial for
| poly-repo since the granularity is at repo-level, but it
| requires dedicated implementation for a mono-repo.
|
| I'm not sure if the _marginal_ benefit of having a monorepo
| can justify the investment for most of the companies. Google
| created monorepo initially to manage the dependencies of C++
| code, and Perforce already supported partial downloads. But
| with more modern languages that have their own way of
| dependency management? I 'm not so sure about the benefits.
| Making refactoring easier? How many repos are really shared
| at source level across multiple teams in a company?
| Encouraging sharing source and therefore knowledge? Isn't it
| a solved problem? Any decent company allows searching source
| code at semantic level across multiiple repos. If I want to
| see the source code of a particular package in my IDE, it's
| just a click away. Note I'm emphasizing _marginal_ return of
| monorepo. Case in point, Google maintains the very use Guava
| library, which is probably used by millions of engineers.
| Does it lead to pains of incompatibility errors at runtime
| across different releases? Absolutely. Is it worth changing
| my poly-repo to monorepo to solve the problem? I highly doubt
| so. The compatibility issue happens rarely given good testing
| setup. When I do need to migrate my code, the cost is bi-
| modal: either the refactoring is trivial, or it requires
| serious testing and design changes, which a monrepo will not
| help anyway.
|
| Note I'm not saying that monorepo is not useful. Instead, I
| question how many companies will benefit from switching to
| monorepo, which may lead to the discussion on the potential
| market share of Diversion.
| Cthulhu_ wrote:
| Re: scalability, in the very first sentence they mention game
| development, which deals with large quantities of large (and
| growing), nowadays versioned assets like 3D models, textures,
| animations, etc.
| hintymad wrote:
| Ah, I guess this is the curse of ignorance: I saw the
| sentence but didn't register its significance as I'm not
| familiar with what's required in game development.
| gertop wrote:
| Respectfully, your entire comment can be summed up as
| "works for me".
|
| I have first hand experience with storing large assets in
| git, which is the use case they mentioned. Git is just bad
| at it. From performance issues to corruption to bugs, it's
| simply not something the git team cares about. LFS is a
| quirky afterthought.
| lowbloodsugar wrote:
| There's a lot of remarks in this post that git isn't that bad, or
| that "losing data with git" is at best an exaggeration and at
| worst a lie. Other remarks include "People know how to use git",
| "--force tells you here be dragons", and plenty of git internals
| by way of explanation.
|
| The posts remarks about git ring true to me. I, personally, would
| have lost work on _many_ occasions if not for IntelliJ having its
| own "commit log" of _every_ change I made since opening the IDE.
| I, personally, have had terrifying moments where I thought I 'd
| lost large amounts of work, and spent the next hour fearfully
| searching google for how to unf*k myself. And I know several
| friends and colleagues who have had the same experience, and some
| who were unable to unf*k themselves and gave up: resigning
| themselves to just recreating the lost work. The fact that their
| lost work is probably still somewhere on their main repo to this
| day is cold comfort I'm sure. "It works for me" where "me" is an
| advanced level of nerd (myself included) is in the same territory
| as "It works on my machine".
|
| This also seems to have really hit a nerve. There wasn't just one
| post saying "Meh, git is easy to use, you're an idiot" and that
| was enough. There was immediately a great many responses on the
| lines of "well, it's not _that_ easy " or "well, I've lost work",
| and a whole back and forth has ensued. The fact there _has_ been
| a borderline flame-war here tells you that git absolutely has
| room for improvement. Most of the remarks in defense of git feel
| like "Git Gud" [0], a term used to heckle newbies at video
| games, yet rather appropriate here. I don't want my version
| control system to be like Elden Ring: arbitrarily difficult with
| deliberately punishing pitfalls.
|
| So, yeah, I'd love a better version control than git, and so
| would many others. I've had work saved from git by IntelliJ's
| internal commit log (where every edit is saved), so having that
| as a feature would be great. Being able to learn if someone
| decided to rename some class that I'm currently using heavily
| would have been helpful on many occasion (rare, but painful).
| Being able to pull someone else's work before the commit it, for
| review, very helpful. Support for large binary files without
| killing my local machine.
|
| Good luck.
|
| [0] https://knowyourmeme.com/memes/git-gud
| sasham wrote:
| Thanks for the support! Reminded me of https://xkcd.com/1597/
|
| I wonder why git became such a divisive issue, nobody seemed to
| have such strong feelings about SVN :D
| 0xbadcafebee wrote:
| I get that enterprises will buy anything with "Cloud" or "AI" in
| the name, but VCS doesn't have anything to do with the cloud.
| Lots of VCS's have had server-oriented architectures, well before
| Git.
|
| I see a lot about the architecture and design here. This is a
| product smell: focusing more on technology than solving problems.
| Some nerdy people may be interested in it, but it all means
| nothing if the experience of those users isn't good. You want me
| to buy your product? Sell me on why the experience is better. How
| it's going to speed up dev time, reduce errors, make
| collaboration better. None of that will be improved by a REST API
| or distributed storage in your backend app.
|
| I'm also not looking to change my development practice. If most
| of your features are only available in a browser, I'm not going
| to want to use it, even if it were better than what I do now.
| Meet the users where they are.
| e12e wrote:
| How does it compare with bitkeeper? Does it handle big binaries
| (digital negatives, game resources) well?
| sasham wrote:
| Honestly, I never tried bitkeeper. It handles big binaries with
| no problems at all.
| badjuju wrote:
| Can I offer a suggestion - focus on non-code developer use-cases.
| It is still wildly difficult to integrate a VCS system into a
| document-based application, yet this could improve many, many
| engineering and science applications that rely on text files
| (simulation configurations, pre- and post-processing setups,
| semi-manual entry data-backed "dashboards", etc). This is a
| problem that _really_ needs solving and I don 't see anything out
| there that makes it approachable.
|
| Having built this myself (hacked on top of git) for an
| engineering analysis platform in the Energy Space, I'd say a good
| UI (even simpler than GitKraken) and an "just works" integration
| would be a massive win.
|
| Just my 2 bits.
| BlackjackCF wrote:
| I think this is a great approach. Just looking at the comments
| here - I'm not sure how many devs would jump for a git
| replacement.
|
| However, anything that solves version control for all of the
| non-code cases is VERY compelling.
| sasham wrote:
| Thanks for the feedback! We actually encountered a similar
| usecase. SaaS tools that need to version user-generated data
| and configurations. We're offering an API so this can be done
| easily. What you offer can also be done. I'm not sure it's a
| huge market though, meanwhile lots of developers that can't
| work with Git (e.g. games) are looking for solutions, we're
| seeing a real need there. But we might branch out to other
| things, for sure.
| snihalani wrote:
| I don't feel the need for this problem but I'll support anything
| git related. good luck!
| rstat1 wrote:
| I was interested enough to go look at your page. I didn't find
| any ways to download a server package to run locally and
| immediately lost interest.
| anthomtb wrote:
| > cloud-native version control
|
| Just how "cloud-native" are we talking here? One chief selling
| point of git is that you are not reliant on a centralized server.
| You have _remotes_ , yes, but if one of them goes down, you still
| get the entire project history.
|
| I am of the opinion that the software world needs to find
| something better than git. One of my major pain points as a lead
| are the inevitable, yet unpredictable, git questions I will be
| getting from new juniors. But "cloud native" sounds like a poor
| starting point for "better git". Not needing external servers for
| VCS is one of the things git gets right.
| aaomidi wrote:
| > We're planning to release it as open source once the code base
| matures,
|
| If it's not open source, I'm not touching this. The one thing Git
| has over all these alternatives its that GitHub dies, Git is
| still around and is GPL.
|
| If you really want to be taken seriously, please make sure its
| open source with a copyleft license.
| sasham wrote:
| Point taken! Why copyleft though, and not MIT/Apache?
| aaomidi wrote:
| If a competitor to you comes up, and starts building a server
| around this client, they can start diverging from this client
| without upstreaming these changes.
|
| It protects you, future users, and makes this protocol live
| beyond the age of any single company!
|
| Its argubly why git hasn't gone away despite its
| shortcomings. And while it has been commercialized, it hasn't
| significantly diverged.
___________________________________________________________________
(page generated 2024-01-22 23:00 UTC)