[HN Gopher] The Big DevOps Misunderstanding
___________________________________________________________________
The Big DevOps Misunderstanding
Author : WolfOliver
Score : 163 points
Date : 2021-12-19 20:07 UTC (2 hours ago)
(HTM) web link (wolfoliver.medium.com)
(TXT) w3m dump (wolfoliver.medium.com)
| ownagefool wrote:
| Counter point.
|
| How is owning a namespace on k8s with a shared services team
| looking after k8s health any different from using amazons aws
| apis and having shared responsibility with aws?
|
| I don't really understand the railing against complexity just
| because you can see how the sausage is made. What's the deal?
| gobins wrote:
| I have been working in Infra Automation and deployment automation
| for a while now. As much as there are complaints about having a
| dedicated team called "Devops", one thing that this change has
| really done is that CI/CD is almost a non-negotiable of
| application delivery now. I am sure there are a lot of teams and
| orgs still starting out in this transformation but its no longer
| something I have to explain to application teams now.
| brightball wrote:
| He's not wrong. Over complicating DevOps does take the control
| away from developers but I think a lot of the Ops part of DevOps
| is to create an internal PaaS that your devs can deploy to
| themselves.
| ltbarcly3 wrote:
| Except then it's not DevOps. If you have people working on
| 'DevOps' 100% of the time, that means they aren't working on
| Dev, and are only working on Ops.
|
| The idea behind DevOps is that Dev(elopers) do their own
| Op(eration)s, rather than throwing stuff over the wall for a
| separate team to manage.
|
| Not only has the industry done the opposite in many cases, they
| kept the name DevOps, meanwhile the full time operations staff
| have over complicated deployments and builds to the point that
| even they don't understand how they work anymore.
| [deleted]
| excuses_ wrote:
| I agree that the term was kinda stollen and in many places
| Ops became DevOps while still doing Ops stuff 100%.
|
| I think there is a place for both. Going into DevOps does not
| mean you need to get rid of your Ops team.
|
| I like the approach when Ops are handling the backbone
| infrastructure of the organization while individual teams
| work in DevOps methodology meaning they are responsible for
| building necessary application infra, programs, as well as
| bringing it to production and monitoring.
| [deleted]
| jordanbeiber wrote:
| Yes, many enterprise devs I know are completely uninterested in
| infrastructure.
|
| This is also a great way of scaling teams - keeping a path of
| least resistance that is secure and compliant while giving
| observability and conventions.
| thrower123 wrote:
| Infrastructure in enterprises is usually completely
| uninteresting, to be fair. You need like eight different
| people on a call in order to stand up a VM, install a
| service, and configure a database connection.
| jordanbeiber wrote:
| Broken processes and missing self-service, as you point
| out, is why creating a platform team is important for these
| types of places. They might not know it yet, although some
| are trying.
|
| It doesn't make the infra technology more or less
| interesting?
| jbverschoor wrote:
| Or any CS field for that matter.
| manish_gill wrote:
| I had to basically give up my full time role as a Data Engineer
| who is supposed to solve "big data" problems (Storage, Streaming,
| Machine Learning pipelines) etc, and had to solely focus on
| DevOps because there was no other expert in the team.
|
| We (a colleague and I) basically learned Terraform, then
| Kubernetes, and then Helm in an extremely stressful environment.
| I wouldn't recommend that to most people. (Also, I can hear "you
| don't need k8s", but tbh, it makes the difficult things possible,
| and that's what we needed).
|
| I've done Ops in bare-metal environments before, and at least
| there, even if there were new skills to learn, the environment
| just wasn't so hostile for beginners, and the learning curve was
| simpler.
|
| The YAML (or heaven forbid, jsonnet) does *not* help. TBH, I'm
| convinced if the IaC bits were done in a regular Java/Go/Rust SDK
| style instead of mixing and matching YAML glued together via
| labels and tags, I would have had a much easier time.
|
| The positive side of that is that now the IaC bits are mature
| enough that even the Data Scientists in our team write their own
| Terraform code, and I've upskilled myself quite a lot. I'm slowly
| going back to solving Storage/Data/ML issues, which I enjoy a lot
| more.
|
| Maybe I'm just becoming a cranky Software Dev as I enter my 30s.
| I do _not_ enjoy reading YAML.
| ludamad wrote:
| I think the YAML issues center around a simple fact, few are
| taking e.g. 1000+ line YAML (or YAML for X) dev seriously.
| Complex YAML becomes code, and requires as much e.g.
| refactoring, idioms, reuse, static checking. At least it's not
| terribly hard to make a script output it as a format
| commandlinefan wrote:
| > regular Java/Go/Rust SDK style
|
| I thought that until I started working with Gradle (which is a
| build script tool who's syntax is nearly-fully-functioning
| Groovy). I miss Makefiles - they were strictly declarative and,
| while I've seen some hairy Makefiles, you'll never need a
| debugger to figure out what they're actually doing wrong.
| lukevp wrote:
| If you haven't heard about cdktf, check it out! It lets you do
| infrastructure as code but in full programming languages like
| Python, C# and TypeScript. It's by hashicorp, and under the
| covers it runs Terraform, so you have the whole set of
| terraform providers at your disposal.
| manish_gill wrote:
| Ah, CDKTF seems to be the Pulumi equivalent for Terraform. I
| think I remember seeing it.
|
| We're far too invested into Terraform at this point to learn
| yet another tool. Oh well. Hopefully the next time I have to
| implement something like this, K8s ecosystem would also have
| similar tools (or maybe I just missed something)
| chucky_z wrote:
| There's a straight up conversion tool iirc, and you can do
| things like include hcl as well I believe. Cdktf is just
| terraform at the bottom.
| topochico3 wrote:
| I don't understand the appeal of CDK, it's the most difficult
| to understand overly complex part of IaaC I have ever come
| across.
|
| I want simple yaml/hcl that is easy to understand and write.
| I don't need to feel smart writing TF, I need it to be easy
| to understand to reduce MTTR when things go wrong.
| kazen44 wrote:
| the issue with yaml is that is breaks down under real life
| complexities in larger/complexer environments.
| cloudcalvin wrote:
| If you havn't heard, check out Pulumi..
| flas9sd wrote:
| could you give me a hint why you didn't like jsonnet? I dreaded
| k8s yaml when needing to (write) and use helm packages and
| think of indenting levels, but jsonnet via grafanas tanka gave
| me the option to be concise, writing functions and creating
| reusable bits. Of course then you're not far away from using a
| real language "sdk style".
|
| As you so aptly put, k8s learning curve is beginner hostile and
| needs system knowledge _before_ you go at it, so I 'm not sure
| if this was also with being more familiar with the abstractions
| - after having looked at k8s for 2-3 years, jsonnet made me
| finally like interfacing with the k8s apiserver.
| manish_gill wrote:
| Yeah, I think the difference is that I was
|
| a) Learning K8s and Helm Concepts (both different things)
|
| b) Reading/Debugging K8s and Helm YAML (both different
| things)
|
| c) Ending up at Helm Chart Configs (another config)
|
| d) Figuring out how to set all this to helm me deploy the
| solution I needed and customize it acc to my needs
|
| ...all within the span of a few weeks.
|
| So by the time I got to some GitHub repos which were using
| Jsonnet, I was basically done. I'm sure it has its uses and
| given enough time and experience, I can see how to utilize it
| more effectively.
| flas9sd wrote:
| yes, both different things. I guess helm is still the most
| common entry point for people to handle the contents of a
| cluster. I'm checking out other suggestions in this thread
| (dhall), but from my personal experience I can already
| tell, grafanas k8s jsonnet-libs was a second chance to
| kubernetes I'm glad I got to know (thanks coworkers).
| laumars wrote:
| DevOps specialist here and I agree with your complaints about
| IaC meaning JSON and YAML documents. _sigh_
| StreamBright wrote:
| You are on the right track. I have exactly the same story, data
| engineer, working on automation a lot.
|
| YAML, JSON and HCL should be purged from IaC asap. Luckily
| Dhall can be integrated everywhere we need YAML otherwise we
| would go nuts. The bigges failure of k8s is to understand that
| YAML does not scale to teams and beyond 100 lines, especially
| not good to nest it (hello ArgoCD).
|
| A type safe not accidentally Turing complete configuration
| language or for that matter any langue (hello Pulumi) is much
| better than YAML ever will be.
|
| Anyways, if I can give you an advise, drop k8s like its hot. I
| is much more trouble than worth. We have phased it out
| everywhere we could. The companies still using it are in 1 year
| long implementation cycles with it with 10+ consultants. It is
| going very well. For consultants, obviously.
|
| https://dhall-lang.org/
| thamer wrote:
| Dhall or other typed languages like Cue definitely help a
| lot. It seems to me that the only acceptable amount of YAML
| you could have in deployment config is for a setup small
| enough that it doesn't warrant using Kubernetes, so any use
| case where Kubernetes is a good fit quickly becomes a major
| pain to maintain.
|
| That said, you wrote:
|
| > drop k8s like its hot [...] We have phased it out
| everywhere we could.
|
| Could you elaborate on what you've replaced Kubernetes with,
| and for what kind of deployment?
| jjtheblunt wrote:
| "turning complete" : typo?
| StreamBright wrote:
| Yes, thanks.
| manish_gill wrote:
| Dhall looks interesting. Thanks!
|
| I went into the rabbithole of terraform/HCL and came up with
| Terragrunt to do dynamic code generation. Combining it
| together with generous use of `make` keeps me sane, but it's
| not a long term solution.
|
| Even now, there are things within the system where the
| boundaries should be separate, but they aren't.
|
| We literally got exhausted and burned out writing our
| automation, and decided to leave it as it is and then slowly
| build it up over the next few months. :/
|
| PS: You're 100% write re K8s/YAML. It feels like everything
| in the Kubernetes world is optimized for writing the code
| just once. Which, of course, is easy if you already know how
| all the various bits fit together (true in any system).
|
| But Reading/Debugging kubernetes world is a dreadful
| experience.
| StreamBright wrote:
| > But Reading/Debugging kubernetes world is a dreadful
| experience.
|
| Exactly. It is abstraction, over abstraction, over
| abstraction for no obvious benefits.
| benlivengood wrote:
| I've been in two production environments where IaC was done
| with Turing-complete languages (Google and Facebook). It's
| really not better (it just lets the truly
| impossible/unmanageable become possible) because they were both
| dynamically typed.
|
| If anything, Terraform comes the closest to being a good IaC
| language because providers form an API to production resources
| as opposed to unstructured code/config like Helm or
| Jenkins/Puppet/Chef. Production in reality is well-typed, and
| unstructured configuration makes static analysis very difficult
| and makes diffs only a guess at what will actually be changing
| when code changes are applied.
|
| What I'd like to see is Haskell (or similar) typeclasses as
| production API; make it impossible to construct malformed
| configurations in the first place but allow expressive
| functions to replace the limited templating functionality that
| current languages have (why can't helm templatize inside
| values.yaml? Why can't Terraform templatize backends and why
| aren't providers first-class objects?). It's very difficult to
| keep current IaC DRY, and if it's not DRY then it has bugs and
| there are misunderstandings about how environments are actually
| configured.
| manish_gill wrote:
| One can write good and bad code in any language. It just so
| happens that our daily driver - procedural or functional
| languages, happen to be ones where a Software Engineer's
| mental model + Existing tooling is evolved enough to make
| understanding it _easier_ (as opposed to just easy).
|
| The problem with DSLs and Config languages is that someone
| will inevitably try to use them in a way they are not
| designed to be used, and then circumvent the way you're
| "supposed to" do things. Not only do you lose all the
| advances in tooling, but people will end up reinventing
| things that already exist in the other languages.
|
| PS: Of course, I'll pick Static Typing every time.
| girvo wrote:
| In terms of "Haskell type classes as production API", I
| assume you've come across Dhall?
|
| https://dhall-lang.org/
| sofixa wrote:
| > The YAML (or heaven forbid, jsonnet) does _not_ help. TBH, I
| 'm convinced if the IaC bits were done in a regular
| Java/Go/Rust SDK style instead of mixing and matching YAML
| glued together via labels and tags, I would have had a much
| easier time.
|
| I have mixed feelings about this. I've seen some absolutely
| atrocious Python and JavaScript code, and i wouldn't want to
| use either of them, and the flexibility ( to shoot yourself in
| the foot) they provide. Go is better. On the other hand, YAML
| is just terrible.
|
| I think DSLs such as HCL are better than either option, as long
| as they're flexible enough while staying limit in order not to
| get illegible/unmaintainable.
| mac-chaffee wrote:
| > This trend is also supported by Kubernetes as this is a
| technology which is hard to master.
|
| Maybe this is rich coming from a Kubernetes person, but sounds
| like the tide is turning against this viewpoint.
|
| Running Kubernetes itself is definitely hard to master, but you
| can take a <10 hour online course and be able to "helm install"
| your application.
|
| Kubernetes can be the new PaaS (if you have someone else running
| the control plane and maybe a monitoring stack). But there's not
| a PaaS on the planet where you can just skip the step where you
| learn about it. Even Heroku users eventually have to learn about
| how connection handling and limiting works, and the specific
| Heroku commands to upgrade your database.
| midrus wrote:
| Are you comparing the cost of learning kubernetes to the cost
| of learning heroku? Or even AppEngine? Those are totally
| different leagues.
|
| Also, with heroku, AppEngine, etc you have tons of
| documentation (official and unofficial) and resources. It is
| well tested and the exact same service runs thousands of
| businesses.
|
| While kubernetes is like the react of the infrastructure world:
| you won't find two similar implementations. You need thousand
| tools configured around it to make it usable, and of course you
| might have docs about the individual pieces but you never have
| it for the big picture.
| dijit wrote:
| > but you can take a <10 hour online course and be able to
| "helm install" your application.
|
| Genuinely I believe that good QuickStarts are the cause of so
| many expert beginners.
|
| What I mean is: there is a person on an adjacent team who truly
| believes he is very competent in nearly all technologies. He is
| able to deploy a k3s cluster no problem.
|
| He is absolutely unable to debug that cluster, or use it in a
| way which is reliable (local persistent volumes means you are
| dependent on the node never going away), he is unable to
| troubleshoot networking issues that have occurred in that
| cluster and he has struggled many times to admit that it is
| unable.
|
| But heck. He can helm install no problem!
| anticristi wrote:
| > What I mean is: there is a person on an adjacent team who
| truly believes he is very competent in nearly all
| technologies. He is able to deploy a k3s cluster no problem.
|
| Aren't you confusing two roles, Kubernetes application
| developer and Kubernetes administrator? The Kubernetes
| application developer should focus on their application,
| NodeJS, Java, DotNET, whatever. Helm install (actually helm
| update --install) is all they should know/care when it comes
| to Kubernetes.
|
| K3s deploy and local storage providers? Let those be the
| Kubernetes administrator's headaches.
| emptysongglass wrote:
| DevOps was always about de-siloing. In that, it is a culture but
| you need people to "be the change" and that's how I've played a
| DevOps Engineer for the last five years (well now SRE) and how I
| believe DevOps Engineers (and SREs by extension) ought to be
| playing their role today.
|
| Corporate management is always surprised when my first move is to
| go sit with the developers and ask how I can unblock their
| workflow, reduce pain and tighten their development loop. Most
| SREs and professional DevOps people I've met do not do this and
| insist that their function is to tell the developers how they'll
| build their app and in which way AKA the same shit from the 90s.
|
| The portmanteau should be a dead giveaway that the number one
| priority of this job is building cooperation between these
| divisions. And often, it's not a technical solution.
| anticristi wrote:
| Kind of weird to blame Kubernetes.
|
| As long as the application team operates their application and
| the platform team operates their platform, I would argue that you
| do DevOps. It's just that one team consumes services from another
| team. Kubernetes just happens to be a damn good "API" between the
| platform and the application team.
|
| Indeed, the application team is likely to build using code,
| whereas the platform team is likely to use scripts and
| configuration. But each runs the stuff they build.
| Niksko wrote:
| Kubernetes serves to allow a team within an organisation to build
| a PaaS that's customized for your organisation. You do this
| because 1. Baking in your organizational defaults and opinions
| reduces work and cognitive load for your engineers and 2. A PaaS
| is too expensive at scale (compared to IaaS), especially given
| what you get isn't tailored to your org.
| atmosx wrote:
| > You build it, you run it
|
| For any set of non-trivial apps the cognitive load is simply too
| big. Assuming the engineer can handle it, eventually more time
| will be spent in keeping up with the current system(s) than
| writing code/promoting the business logic.
|
| A company could chose to use ready-made tools but all them get
| expensive pretty quickly and make their own assumptions about
| workflows, view-points, etc. The assumptions might fit the use
| case or not. Lock-in then becomes another problem. There's no
| easy way out, it's trade offs all the way :-)
|
| Example: The Haskell lead engineer for App1 has all the necessary
| knowledge to understand how the base-metal NixOs fleet is
| configured because there's deep familiarity with Nix. At some
| point the infra team will start making decisions and assumptions
| to automate _this_ or _that_. The engineer now will have to spent
| time understand what changed since last time, making sure nothing
| is broken while performing changes.
| StreamBright wrote:
| I think k8s is exactly the reason we are moving back to separate
| ops vs dev modus operandi in many cases.
|
| Projects like Terraform and Pulumi moved us towards a more
| vertically integrated development process, bringing the knowledge
| and tooling of SRE people right into the application repository.
| And when k8s turned out to be too complex for this setup we claim
| that the previous way of doing things was better.
|
| I disagree, strongly. If anything, we are going to skip k8s and
| just use the other solutions (AWS, Azure, even on-prem with other
| tools than k8s) and we are going to keep our devops tools,
| integrated teams and the progress we made in the last decade.
| coolgoose wrote:
| I am so tired of devops as a term. It's a stupid bandaid
| solution.
|
| Infrastructure, regardless if you use a paas or not, is a tough
| cookie to crack and deserves a normal sysadmin role.
|
| Monitoring, resource utilization, network management, library
| updates, costs among a few are things you need to master, observe
| and proactively manage, not keep it as an afterthought since it
| magically just works.
| kazen44 wrote:
| troubleshooting infrastructure is hard.
|
| You cannot usually bring down or change something in production
| without extensive research because well, its production. Issue
| usually happen either at seemingly random times, or at moments
| when doing something directly about it is nearly impossible
| because of the impact it will have on the production
| network/system.
|
| Also, in production (in the greater sense of the word, for
| instance, a datacenter with a virtualisation system running
| multiple workloads/applications). that most issues between
| systems start to creep up. Usually, systems themselves have
| been tested enough, its the unintended interaction between them
| that throws things off balance.
| zthrowaway wrote:
| My DevOps team are actually SREs, and our SRE team are just
| webdevs who are on call. It's pretty ridiculous.
| geodel wrote:
| Could be that someone in management understood site reliability
| as website reliability and decided to put web devs on call.
| dvtrn wrote:
| As an SRE, I'm starting to think the misunderstanding of SRE
| tradecraft is probably more pronounced than the
| misunderstandings of Devops tradecraft. The risks definitely
| feel higher.
|
| I'm six months into a new SRE role and it really feels like
| instead of doing any kind of engineering I'm instead just
| getting "overflow pipeline work from the rest of engineering
| that the Devops team was too busy for" and I'm already
| looking for the door.
| Fizzadar wrote:
| Totally agree, I've always insisted myself and/or my team are
| called "infrastructure"[1], who provide the platform for the rest
| of engineering to deploy stuff to (devops). But when hiring for
| this team, have always used devops as the title because it brings
| in the right type of candidates.
|
| [1] platform is probably a better name :)
| GiorgioG wrote:
| Kubernetes is a cancer for 99% of companies. Most companies need
| little more than docker compose/swarm.
|
| Kubernetes is the antithesis to KISS. If you're Google, then you
| need something like K8s. Most orgs using it are pretenders or
| trend followers. Sadly it has become the "standard" way to deploy
| apps.
| throwawayboise wrote:
| Most people just need lxc-create or virsh create, honestly.
| It's the only things I have been able to get to work reliably.
| All the docker and k8s stuff is unnecessary complexity if you
| just need to run a container or VM.
| jedberg wrote:
| Devops is a philosophy not a role. The title DevOps engineer is
| meaningless.
|
| There are software engineers, who write software and then deploy
| it onto self serve infrastructure, and there are operations
| engineers and data engineers who build and maintain the systems
| that the software engineers use to deploy and operate their
| software.
|
| Usually when someone calls themselves a DevOps engineer they are
| an operations engineer, but sometimes they are a software
| engineer at a small company who also has to maintain their own
| infrastructure.
| avip wrote:
| Personally I think it's pretty great that I can write
| Dockerfiles, run them locally, and call it the day because this
| is also what runs in production.
|
| So basically I have no clue what the dude is talking about - we
| have a great setup which is x10 better than what we had before.
|
| No?
| quadrifoliate wrote:
| > Personally I think it's pretty great that I can write
| Dockerfiles, run them locally, and call it the day because this
| is also what runs in production.
|
| It goes far, far beyond Dockerfiles. Are you on-call for
| production? Can you debug a problem in production?
|
| If not, then that's the root of the problem that the dude is
| talking about. If these are true for you (not specifically you,
| any reader of this):
|
| - You don't have a good idea of how to ship a change all the
| way to production after committing it
|
| - You would not know how to debug and fix a failure of your
| code in production
|
| - Your "Dev" and "DevOps" teams are in separate departments
|
| ...then congratulations, you have just reinvented the old-
| school sysadmin/developer dichotomy from the 90s. You just
| happen to be using Dockerfiles instead of Makefiles.
| kazen44 wrote:
| Not to mention the layer that lives behind this.
|
| Can you troubleshoot network level issues that seem to occur
| between your system and someone else's?
|
| What about storage? Can you make sure your data is still
| their if your environment gets royally screwed.
|
| I know these are usually done by seperate companies nowadays,
| but getting an an actual computing environment up and running
| with redundancy in place (down to the cooling and electrical
| level) is no easy feat.
| [deleted]
| pjmlp wrote:
| Somehow it seems System Engineers got renamed into DevOps, from
| the kind of tasks that land on my table when I happen to wear the
| DevOps hat on some consulting gigs.
| KaiserPro wrote:
| I did not entirely agree with the main assertion of the post.
| However having re-read it, I do now broadly agree.
|
| I would say that devops _should_ be a spectrum. I have always
| worked at places that never really had rigid boundaries between
| sysadmin and dev. So I was really quite baffled by the initial
| push to devops (Isn't that what you were doing before? How can
| you be a sysadmin if you can't code.)
|
| Having a department of devops is fine, so long as most of them
| are embedded into dev teams. Assuming you rotate your devops
| around dev teams, its a good way to remove single persons of
| failure, and increase standardisation. Depending on size, there
| might be a chance to have a core systems/platform team.
| safeerm wrote:
| Really great post. When I was at AWS, we did everything with IaC
| but that luxury of time and investment is not avail to many.
| We're working on this problem but ensuring that devs has a) full
| transparency what's going on, b) ability to evolve their arch and
| c) building right on your AWS account. We just moved someone from
| Heroku to AWS handling 1B requests/day at 1/4 of the cost.
|
| have a bunch of AWS execs backing us too; they know this is a
| problem. Happy to chat more if you want to give it a spin:
| tinystacks.com
| qaq wrote:
| cloud is a marketing scam. It is sold to execs as a magic black
| box that scales and you don't have to be concerned with how it
| works internally. In reality it's a set of leaky abstractions
| and you absolutely have to understand how black box works
| internally plus be aware of thousands of bizarre hard/soft
| limits and other gotchas.
| hcrean wrote:
| A sysadmin team leader walks into the room and says, "Everyone,
| you are now DevOps engineers, carry on", and walks out to collect
| a bigger paycheck...
| ltbarcly3 wrote:
| So this, but instead of starting with Sysadmins with tens of
| years of unix experience, start with junior developers with 1
| year of experience using Docker as a developer and have them
| read a blog post on how-to set up Kubernetes.
| geodel wrote:
| Now I feel this is how it is exactly happening at work even
| though I am not part of "devops" team. Any working solution
| at my place is now legacy ready to be dumped and replaced
| with next generation stack. Other day "devops" engineer
| couldn't figure out wrong directory permissions and whole
| deployment failed.
| kazen44 wrote:
| we once had this issue at an old job.
|
| People wanted to replace a monitoring solution for
| something new, because snmp was "bad and outdated".
|
| SNMP is definitely not the holy grail of solutions (to be
| honest, its kind of a mess). But atleast SNMP is widly
| supported, works reasonably well, and gets you the data in
| a dashboard you need. Even for those old Cisco 7200's with
| an uptime of over a decade.
|
| Being new and shiny in operations is usually a sign of
| inmature technology. (with the bugs and issues associated
| with it).
| mschuster91 wrote:
| Can we _please_ get rid of the DevOps hype train?
|
| What gets completely ignored is that there is a _reason_ that
| sysops has been a specialized role for decades. How often do we
| get threads here that people get their AWS accounts hijacked
| because of some credential leak (e.g. access keys in frontend
| builds), extreme expenses due to stupid autoscaling settings
| combined with a DoS attack or content leaks out of insecure s3
| buckets? Or how much money gets wasted because clueless "devops"
| developers think that failsafe redundancy for their tiny PHP app
| is configuring a bunch of x2large instances and putting ELB in
| front of it instead of either going with lambda functions in the
| first place or at the very least use small instances and sensible
| autoscaling.
|
| Or ffs just take gitlab ci pipelines. Nearly every time when
| taking over a project or hopping in to consult I see _utter
| fucking bullshit_ written by people who don 't have any idea what
| the fuck they are doing. Yeah, your build takes 10 minutes
| because you don't use any cache anywhere and you don't use Nexus
| or any other form of repository cache. And ffs no you should not
| push your application image to public Dockerhub because you have
| no idea that Gitlab has its own registry - and no, "I read it on
| some 'getting started' guide for <insert JS framework here>'
| ain't an excuse either. People get barely trained in the
| technology they use, most JS frameworks (and fwiw, Gitlab and
| Github CI themselves!) are young enough that it's impossible to
| get senior people for them, and the stuff that ships for "CI
| templates" there is _atrocious_.
|
| The amount of knowledge required to _securely_ operate any kind
| of hosting or development operation is immense, and acting like
| developers should also shoulder the role of unix sysadmin, IT
| security staff and cloud configurations is extremely ignorant -
| not to mention only half of the developers know more than
| "ls/cd/rm" on the command line, less if they are Windows
| developers.
|
| I do see value in "devops" as in "a person primarily focused on
| operating infrastructure, but with enough capability to
| troubleshoot production issues in applications", but devops as in
| "save money for actually qualified ops staff" is irresponsible
| penny pinching.
| atomicfiredoll wrote:
| This. I think that today the barrier to entry is low enough
| that, especially in small companies, developers with little
| knowledge of the space can get a basic CI system or production
| deployment going. And, the company is more than happy to let
| them instead of hiring somebody specializing in the area. This
| seems to really blow up if it goes unaddressed as the
| team/product grows.
|
| I've watched at least one company burn time on the same issues
| for years because instead of acknowledging the need for and
| empowering people with distinct skillsets or knowledge, they're
| content to let developers bumble through the process of
| learning and reinventing the wheel over and over in the
| margins. That's if they can even get the developers on growing
| and siloed teams to communicate with one another at all. There
| needs to be experienced leadership somewhere.
|
| Developers should have more responsibility over their apps and
| what they're putting in to production. But, it's unfair an
| inefficient to expect all but a few will ever even have time to
| become skilled practitioners in these areas. Especially while
| already keeping up with their normal duties. Now not just tech
| debit, but foundational infrastructure can be pushed aside by
| the drive for new/more features.
|
| For the foreseeable future it seems like the trend of tasking
| devs with so much of this work will result in engineering teams
| continuously, wastefully toiling over fractured infrastructure
| built by novices. (Maybe someday the tools will actually allow
| for developers to take this one with limited
| knowledge/resources, though.)
| g051051 wrote:
| Great to see someone else saying this for a change. "Dev" and
| "Ops" are different skill sets with _some_ potential overlap,
| but not 100%. Organizations I 've been in have never done the
| "throw it over the wall" thing, either. Dev and Ops are
| _partners_ , and treating things any other way is a recipe for
| failure.
| darepublic wrote:
| Access keys on frontend is the fault of the frontend developer
| or culture
| mschuster91 wrote:
| There will always be irresponsible or junior people, the
| problem is that AWS _allows_ extremely unsafe things in the
| first place.
|
| Normal hosters wouldn't even open internal infra like DB
| instances, mail relays or storage up to the Internet, AWS _by
| default_ doesn 't even firewall their internal stuff of and
| thinks, as long as you have the right credentials, you can't
| be wrong. S3 and SES have mind-bogglingly insecure defaults,
| and it is _hard_ to wall off your SES credentials enough that
| they 're useless to an attacker who has managed to exfiltrate
| them (and that one is easy enough, all an attacker needs is a
| vulnerable log4j and the dubious "best practice" of passing
| credentials via environment variables).
|
| It is so incredibly easy to mess up on AWS it _hurts_. AWS is
| a gun shop that not just sells handguns but nuclear armed
| bomber jets to anyone presenting a credit card. No knowledge
| checks, training, _nothing_ required, and you have the
| firepower of all of AWS on your hands.
| jollybean wrote:
| When the cloud was young, devs could indeed push their own code.
| They didn't need the sysops people. It was a major change.
|
| But as the cloud became more complex, invariably, SysOps just
| took on a whole new shape. Instead of buying hardware and keeping
| Ethernet boxes wired and alive, you now have K8s etc..
|
| I suggest we have failed somewhere along the way: it shouldn't be
| necessary for real DevOps up to significant scale.
|
| If our tooling were good enough, it should be mostly fairly
| automated.
|
| 'Containers' seem to be a messy, hacky afterthought when what we
| really probably just needed was 'configs'. Cloud Lambdas, Queing,
| Caching, Logging could be feasibly made very simply configed ...
| on all the Cloud Providers it's fairly complicated, just beyond
| the ability for a regular dev to stay on top of along with the
| rest of dev stuff.
| hodgesrm wrote:
| > This trend is also supported by Kubernetes as this is a
| technology which is hard to master. So people start splitting up
| responsibilities again. One part of the team developing the
| application the other part of the team managing the
| infrastructure.
|
| Kubernetes _administration_ is hard to master. It 's analogous to
| running VMware in production, or any cloud technology for that
| matter. None of them is easy, and they all require relatively
| specialized knowledge.
|
| Kubernetes _development_ on the other hand is relatively
| straightforward. Minikube is my goto environment whenever I have
| to put together an app that consists of more than a single
| process. As long as you pay reasonable attention it is not hard
| to move such applications from dev to staging to production.
|
| Edit: fixed markup
| carlhjerpe wrote:
| I disagree, DevOps people are writing code and "code" (YAML, HCL,
| JSON) that's used to provision infrastructure. When developers
| push to master it'll build and deploy. They're still deploying,
| they're just getting help with running the stuff. Can we really
| expect everyone to understand how solid infrastructure should be
| built? I don't think every JS frontend dev should have this know-
| how, yet their code is deployed on S3 behind Cloudfront (or
| whatever in front of whatever, static file hosting feels like
| kind of a solved problem).
| autophagian wrote:
| I've also become a little disillusioned with the 'DevOps means
| developers create their infra/deploy their applications' mantra
| too, especially when you see several years of accumulated
| infrastructure spun up by developers that don't really care about
| its quality. Then again, having a dedicated ops team can lead to
| siloing and learned helplessness on the part of devs too. It's a
| bit cyclic.
| Mc91 wrote:
| I agree 100%. It's cyclical, the trend goes one way for a
| while, then it goes the other way.
|
| "We don't need a separate devops department, the programmers
| can do it"
|
| "We don't need a separate QA/QE/SDET department, the
| programmers can do it"
|
| "We don't need DBAs designing normalized database schemas for
| ACID databases, we'll have the programmers use
| ORM/NoSQL/whatever"
| Ozzie_osman wrote:
| 100% agree. The industry took the term DevOps and started
| applying it to old-style ops teams (only difference is the ops
| teams use more "modern" tools).
|
| There's still value in having an infrastructure or platform team
| focused on building tools to enable dev teams to more efficiently
| deploy, monitor, and run their code, and to step in with
| expertise when needed (ie help debug things like database
| performance, etc). More "teach a man to fish" type help, but also
| "give them a good boat and fishing net".
| dijit wrote:
| I made a similar ranty blog post about devops.
|
| I was surprised by the amount of history and misconception that
| exists around the term.
|
| The term was originally intended as "Agile Systems
| Administration" but came about at the same time as a talk titled
| 100 deploys a day.
|
| So people conflated the two, and it didn't help that the
| conference was a portmaneau of "Developers, Operations days" to
| encourage it being a conference for everyone. Eitherway. We're
| stuck with it now, and it's the least meaningful job title I've
| ever known.
|
| If you have to ask "what does x mean to you" then it's a failed
| descriptor.
|
| http://blog.dijit.sh/devops-confusion-and-frustration
| g051051 wrote:
| > The term was originally intended as "Agile Systems
| Administration"
|
| That explains why it's such a terrible idea.
| vanusa wrote:
| Exactly. One meaningless definition attempts to justify its
| existence by referencing another.
| streetcat1 wrote:
| I think that the op is missing a side.
|
| The article is about dev -> ops.
|
| The other side is ops->dev. I.e. since infrastructure is
| software, ops become dev (and not just dev become ops).
| kazen44 wrote:
| has ops ever not been dev?
|
| Atleast in my experience (ISP/DC world) most operational people
| where expected to do troubleshooting which at some points,
| results in one having to program to get to the nitty gritty
| efficiently. At a certain scale, issues start to creep up that
| are hard if not impossible to troubleshoot without atleast some
| programming and CS experience.
|
| Are operational people developers? Not in the traditional
| sense, but the development of specific tools is required to
| efficiently keep the lights running.
|
| A good example of this is troubleshooting packet loss. In a
| large network, this can become very difficult to pinpoint
| because more then a few dozen hops exist, and packets drop seem
| to occur randomly.
|
| Without tooling to gather this data from different points in
| the network and doing statistics on it, it becomes nearly
| impossible to get a clear picture and troubleshoot efficiently.
| hiptobecubic wrote:
| There seems to be a lot of people here, and presumably a lot of
| companies, where no one has ever actually seen a successfully run
| devops team and what they provide.
|
| For one, people are arguing past each other about deployment
| environments vs developer tools vs being a sysadmin.
| 1270018080 wrote:
| DevOps Engineer is an oxymoron, but who cares. Prescriptive
| linguistics is a fools errand. DevOps engineers do important
| work, their job is real.
| lmilcin wrote:
| Similar thing happened to Agile.
|
| Agile is a practice of speeding up improvement. Idea being that
| organisations that improve faster will, over time, overtake
| organisations that improve slower. If so, it makes sense to focus
| on the improvement loop itself. You remove management from the
| loop (because management takes too long and slows down
| improvement) and you empower and encourage individual
| contributors within their teams to improve the process because
| they can do it faster, have hands on knowledge of the process and
| can observe results faster.
|
| Additional benefit of Agile is that you now have a tightly knit
| team that tends to be more engaged with their work and more
| focused on thinking through what they are doing -- this tends to
| cause people to be more satisfied with their jobs because
| satisfaction of being able to affect the process positively is a
| great motivator.
|
| Yet, the management thinks they can come with a book and
| institute "Agile process" by telling everybody what they have to
| do, exactly.
|
| Usually, improvement within team structure is then actively
| discouraged with many explanations like "that's not what the book
| says" or "I was doing scrum at a company where it worked and so
| we must do the same" or "we need to standardise processes" or
| some other variation of it.
|
| I have observed this time and time again at various stages --
| always with the same miserable outcome.
|
| At my last company they additionally decided we are introducing
| Scaled Agile Framework meaning, according to them, we will now
| plan not at team structure but at a department level. This
| innovation means all planning and retrospective are combined for
| about 400 people. Because this was too time consuming to do every
| two weeks, they decided we will do this every 5 sprints (10
| weeks). Just to be clear -- every 5 sprints we were supposed to
| select and plan tickets for the next 10 weeks of development.
| This was presented as an efficiency improvement.
| quickthrower2 wrote:
| PaaS is easier and I prefer it for personal projects. I don't
| want to use the terminal or shell at all and have minimal yaml
| files.
|
| For work though kubernetes means portability and not having the
| rug pulled from under you, you can move clouds more easily if
| necessary.
| arielb1 wrote:
| The one system I've seen work is when each medium team (of say,
| 20-36 people) has a "devops team" of say 4-6 people that is
| responsible for all the infrastructure stuff, while the rest of
| the people can focus on product work. It's just that keeping
| track of infrastructure is much more efficient as a full-time job
| rather than a 10%-time job.
|
| It's important to do the org engineering right so that the
| infrastructure people and developers feel responsible for each-
| other and work well together - ideally both teams should be able
| to open PRs against each other's code if needed and get help any
| time.
|
| If the org engineering isn't done right, devs write code that
| pages the ops and the ops don't respond to devs quickly enough
| for things to work well ("traditional ops").
| kristianpaul wrote:
| Could not agree more
| mathfailure wrote:
| There seems to be a quote used as the main argument:
|
| > You build it - you run it.
|
| Well, building != developing, so DevOps both build the code and
| run the code (written by devs), no misunderstanding.
| hnthrowaway0315 wrote:
| One thing I really hate about a dedicated ops team (DevOps that
| is independent of devs) is that they have to use just one hammer
| for all kinds of nails. It's simply impossible for a small team
| to maintain multiple hammers.
|
| So somehow the tools and workflow are exactly same for an app
| that has millions of external users and an internal app that only
| 10 people use. You still have to go through a whole stacks of
| K8S/docker who-knows-wtf-they-are configurations just to publish
| a dashboard from an internal IP. It took more than 8 months for
| my previous team to try to pull it off but it never went through.
| mrweasel wrote:
| As an ops guy: You're absolutely right.
|
| For better or worse I'm constantly annoyed by developers trying
| be "special". Most projects involve stuffing things into a
| database, calling a webservice and reading stuff back. Don't
| try to be clever about it, just stick to one type of database
| and Springboot so we can all go home and sleep with no
| incidents.
|
| But there are projects which really do need to be different,
| and we don't deal with them well, and they don't scale.
| hnthrowaway0315 wrote:
| I don't blame the devops guys though, I think this is a
| corporate thing as teams have to follow procedures and devops
| are not an exception (to waive some rules for us). I just
| hope it's a lot easier if we can just use a VM instead, but
| that would need to go through security certification.
|
| I wonder how other companies deal with this kind of issues,
| eventually my previous team decided to host the app on one of
| the laptops and tell clients to visit a "192.168.0.X:port".
| mrweasel wrote:
| > You build it, you run it
|
| While I do like that definition, it also means that most
| companies/organisation cannot do DevOps, as software development
| is often bought as projects, and operations is a recurring cost.
| throwDec19 wrote:
| We get heavily clamped down on access to production by our
| auditors for Sarbane Oxley reason. Technically we aren't even
| allowed production data in our dev environment. I dont
| understand why any organization would allow it. Seems the tech
| world is still ripe to get sued.
| throwawayboise wrote:
| Organizations did (do?) allow it because it's easy. It's also
| a way to test/debug production problems with actual
| production data instead of spending time trying to make up
| data that you hope covers all the actual use cases.
| 988747 wrote:
| Also, government regulations such as HIPAA explicitily force
| you to have separation of duties, so true DevOps is against
| them. Developer being able to push their code to production can
| earn you some hefty fine at the next audit.
| d0mine wrote:
| Software is never done. It can only be abandoned.
|
| There are projects that change slowly but they do change
| nonetheless unless they are no longer used.
| cortesoft wrote:
| As someone who has lead a devops team, and still has a devops
| team reporting to me, I have also struggled with the idea of
| having a team called devops, since it does go against the devops
| philosophy of devs owning their software in production.
|
| However, in our case (and probably many others), our devops team
| is responsible for creating and maintaining the tools and
| processes that allow the dev teams to manage their systems in
| production. At a larger organization, it doesn't make since for
| each individual dev team to have to create and maintain their own
| deployment and management systems... that would end up with a lot
| of duplicated effort and a wide variety of systems that would
| need to be maintained. Devops creates the tools and procedures,
| and dev teams use them.
|
| You might say this is just sysops, but I think the key difference
| is that the dev teams are the ones who are using the production
| management tools, not the sysops team. Devops is not the ones who
| are paged if a system has an issue, the dev team who owns it is
| paged. Devops is only paged if the tool that the dev team is
| using to handle the issue has issues itself.
| thedougd wrote:
| Would you say it's a platform team?
| kqr wrote:
| This describes what I've heard referred to as an internal
| platform team.
| cortesoft wrote:
| Yeah, that might be a more accurate title at this point.
|
| Part of the reason that the name DevOps was chosen almost 10
| years ago when it was started was to help signal to the wider
| org the cultural shift we were undertaking, and that the
| expectation was going to be for devs to own their systems all
| the way through production.
|
| At this point, we would probably change the name, but the
| team is quite attached to the name after 10 years. We
| actually did try to rename the team, but everyone still
| called it the devops team and we gave up on the change.
| cco wrote:
| Or infrastructure team, or systems administrator, perhaps
| sysadmin for short ;)
| kqr wrote:
| In the places I've been, a sysadmin has been someone who
| operates someone else's product -- not a person operating
| the systems developers use to operate their own product.
| netizen-936824 wrote:
| This is my observation as well. Sysadmins writes scripts
| or small programs for automation but its mostly using
| those tools for the operation of larger systems developed
| elsewhere (including other internal depts)
| dijit wrote:
| A sysadmin wouldn't write payment code.
|
| A sysadmin would write the tooling to ensure the payment
| code could work (failover databases, redundancies) and
| put safeguards in place so that it does not get traffic
| that is not intended for it.
|
| Sysadmins used to write the code that touched thousands
| of machines at once, developers used to write the more
| complex code which were the bread and butter of the
| company.
|
| But where I've worked, it was never a silo. Just a
| different mindset, and everyone worked together to do the
| best they can. That was before "DevOps".
| waynesonfire wrote:
| > sysadmin would write the tooling to ensure the payment
| code could work
|
| Uhh if by tooling you mean a bash script to deploy ssh
| keys to prevent team A from accessing a machine owned by
| team B.
|
| You really have no idea what you're saying.
| dsr_ wrote:
| Dijit is completely correct.
|
| A junior sysadmin executes checklists, writes simple
| scripts, reads logs, and solves common problems, all
| while learning the complex supersystem of their
| workplace. They learn configuration of systems in use and
| how infrastructure is arranged and commonly changed.
|
| A sysadmin helps to create checklists, writes more
| complex code to work in an automation system, and
| generally solves problems with non-obvious causes and/or
| remedies. They know all the routine events in their
| workplace, and add to documentation. They build,
| configure, test and recommend new tools and solutions to
| users, devs and the rest of the ops staff.
|
| A senior sysadmin works on policy, debugs automation,
| solves complex interactions, and has an excellent
| understanding of complex systems and networks, not just
| at their workplace but in interactions and hypothetical
| interactions with the Internet at large.
| INTPenis wrote:
| Being a devops guy these last 2 years I have noticed that
| besides doing what you just mentioned, I am basically end user
| support for developers.
|
| And through this I've noticed that developers can be just as
| bad at communicating issues as any end user I've ever dealt
| with.
|
| Just because they've written the system end users use, does not
| mean they're better at reporting problems to me, because their
| problems are about the pipelines, the kubernetes clusters, the
| containers, anything outside of their programming code
| basically.
|
| So I still need to use all the communication skills I learned
| decades ago when I was doing end user support for people who
| didn't even work in IT.
| jyrkesh wrote:
| It wasn't necessarily the standard or even the majority, but we
| had that with Sysops whenever "the build team" or "the
| deployment team" was building self-serve tools.
|
| And I definitely agree with the article here that if you're
| using PaaS, you don't need a DevOps team. And there's certainly
| a better feedback loop there.
|
| But many will argue that PaaS _still_ isn't there yet for
| highly complex applications. Unless you go to a point where you
| basically need a team managing the eventual complexity that
| PaaS adds back to things for you to achieve complexity.
|
| In any case, this is mostly pedantry. We should all strive for
| simplicity as much as possible, but complexity in apps breeds
| complexity in organizational structures. DevOps was never a
| silver bullet, and there probably won't ever be one.
| OriPekelman wrote:
| Evidently any PaaS has to have _some_ opinions on the kind of
| apps it is going to run. You trade some assumptions against
| magic (and time you won't spend on making decisions that are
| very possibly trivial). You could very probably design some
| pretty complex applications on any PaaS worth its salt.. as
| long as you accept its assumptions, opinions . But when you
| do you might very well end up with the only actually
| achievable form of the original DevOps. Weirdly if Devs
| actually touch Ops it's probably no longer DevOps ... just
| people being cheap and asking other people to have two jobs
| for the price of one...
| stickfigure wrote:
| To adapt Greenspun's tenth rule:
|
| You have a dedicated team building an ad-hoc, informally-
| specified, bug-ridden, slow implementation of half of Google
| App Engine (or Heroku, or Digital Ocean App Platform, etc).
| shrimpx wrote:
| Devops is having an identity crisis, but what you describe is
| what I call devops. It's the subset of engineering that
| supports engineering by defining/inferring workflows and
| building systems and tools to codify them. But I've seen it
| called ops, sysops, internal platform, and even infrastructure
| -- in some companies, devops and infra are the same people.
| cortesoft wrote:
| Yeah, I am not sure how typical this is, but my company has
| DC (data center) ops, NetOps, SysOps, and DevOps...
|
| DC ops is in charge of the physical system, from racking and
| stacking to hardware replacements.
|
| NetOps handles the networking stack, managing the routers and
| switches, vlans, announcements, IP block management, network
| access requests, etc.
|
| SysOps handles provisioning systems, working with dc ops to
| get systems repaired, laying down the OS, managing the
| inventory system, and anything else the systems need to be
| ready to run our software.
|
| Then DevOps is responsible for all the software that our
| company writes, and managing how developers get their
| software onto the machines, how they configure the software,
| and how they interact with their software in production.
|
| The lines are obviously not that clean cut, but that is the
| basic idea.
| kazen44 wrote:
| this seems to be roughly the division i have seen at most
| companies.
|
| Sometimes, automation and tooling are written by seperate
| teams, or are done by teams of different disciplines of
| operations/engineering.
|
| I have almost never seen a network and system groups NOT
| being seperate. Mind you this is in DC/ISP's environment,
| where the complexity of networks is usually the mainstay or
| a heavy dependency of the business.
| jbverschoor wrote:
| It is the J2EE role "Application Deployer"
| sharken wrote:
| The DevOps team can often be the team that handles Compliance,
| Security and Audit for the platform.
|
| Then there is the infrastructure/deployment team and finally
| the developers.
|
| At least in larger organizations i think that is a common way
| of doing things.
| 123pie123 wrote:
| in my experience with in enterprise businesses the DevOps
| people are almost the worst people to handle Governance, not
| too great at end to end Security (I've no idea about Audit)
|
| IMHO I would guess they're too close to the dev people to
| care about the bigger picture
| originalvichy wrote:
| Disclosure: this is my role.
|
| I work as a sort of "dev tooling engineer" or "platform
| engineer" for several companies who are our customers.
|
| After years of consulting in the area of DevOps, the pretty
| smart people I work with figured out that while the developers'
| user story in the process of devops transformation is a
| "revelation", the one thing that usually gets forgotten is the
| maintenance and upkeep of those tools. So we set up a pilot
| project where we started hosting, maintaining and upkeeping a
| set of tools consisting roughly of Jira+Confluence, Jenkins as
| the CI/CD server, Artifactory for binary hosting and
| Bitbucket/Gitlab for source code.
|
| It has worked damn well for the first pilot company (signed on
| 5+ years ago), the second one, and by the third the platform
| started selling itself (getting a public reference from the
| first couple companies helped a lot). The second reason for the
| easy buy-in is to use known software that our customers could
| host themselves if they had to. This practically means that we
| aren't vendor-locking them with our own proprietary tools. We
| just take care of off-the-shelf tools.
|
| If there is a competent tooling team that takes care of the
| intricacies of the tools and maintains their uptimes, the dream
| of "you build it you run it" can still live. There is a giant
| gap between "you build it you run it" and reality -- much how
| there is a gap between a pilot being able to fly a plane and a
| pilot knowing how to maintain an Airbus, the airport and the
| runways.
|
| It says something about the job market that losing a customer
| pretty much never happens. It's so much easier to find
| sysops/devs than it is to find people who specialize in
| tooling.
| Spooky23 wrote:
| The problem with philosophy is that reality has a way of
| getting in the way.
|
| Think of home improvement. If my toilet is clogged, it doesn't
| make sense for most people to call a plumber. That doesn't mean
| that plumbers are bad, just that it's easy for a generalist to
| do basic tasks.
|
| Likewise "pure" devops rarely makes sense because developers
| are assigned to projects that end, and sustainment doesn't
| require the same skills. It's cheaper to pay for 20% of a
| $75/hr ops engineer than 100% of a $150/hr senior dev.
| jordanbeiber wrote:
| Last couple of places we've called it "dev platform" or "dev
| tooling" team.
|
| I use to say that it's like one of the business domain
| development teams, it's just that the domain is "dev". The
| businesses devs are the stakeholders, and the tooling/platform
| team members are usually already domain experts that write
| code.
|
| Having a "devops" team would feel like having a team called
| "agile team" or something - you do "devops", you can't build
| it.
| urthor wrote:
| Exactly. Just call it the platform/tool team.
|
| Realistically though, I'm 100% sure that absolutely nothing
| will stand in the way of the DevOps job title being
| denigrated.
|
| Companies have for a very long time used title inflation to
| bait and switch the unwary software engineer into editing
| enterprise software configs, and will continue to do so for
| all of time..
| recursivedoubts wrote:
| I'm frankly astounded at the complexity around modern development
| due to dev ops and how much worse developer experience is because
| of it. I used to develop in a great IDE with debugging, right
| click re-run failed tests, I could follow the console right there
| in a nice, clean window integrated with my IDE, click on stack
| traces, etc.
|
| Now I'm running my app via multiple docker images, trying to get
| a buggy remote debugger protocol to work (no local variables,
| wtff?) where I have to coordinate the container and the IDE,
| command line wrangling / using print debugging in the container
| to chase down stuff, digging around through multiple log files,
| copying-and-pasting stack traces into my IDE to analyze, which
| kinda works in some cases since the file paths are different, and
| on and on.
|
| DevOps came and took my nice dev environment away, and I am mad.
| excuses_ wrote:
| I second that. I experienced this drive towards overly
| complicated pipelines that require a bunch of external systems
| to work in order to test my application and claims like "it's
| impossible to run it locally".
|
| The answer should be more modular code with clear interfaces
| and contracts so that I can test as much as possible locally.
| stepbeek wrote:
| Given the second principle of devops is amplifying feedback
| loops, I think a more charitable view could be "the common
| misunderstanding of DevOps has lead to a terrible dev
| environment".
|
| Fwiw, I'm fully onboard with you, but really love the core
| message of books like the phoenix project and don't really want
| it to be lost because of overzealous use of docker.
| hnthrowaway0315 wrote:
| OMG, I read the sample chapters from Amazon about the Phoenix
| project and that was a really really good read, thank you!
| mindwok wrote:
| Sounds like what you're really angry at here is a case of
| micro-services taken to the extreme.
| marvinblum wrote:
| Did it? I'm using GoLand and have exactly the experience you
| described. I can debug, rerun tests, terminal integrated, ...
| It's just a decision you took or someone else took for you.
|
| I prefer to be able to fully run my application locally, with
| all databases required, and test everything on my machine. DBs
| and other tools like a simple SMTP server run in Docker, yes,
| but that's just a decision I took and I could easily run these
| tools without containerization.
|
| Maybe I'm just in a lucky position to be able to decide this by
| myself, but I think we can get back there.
| hermanradtke wrote:
| What you just described is not really related to devops. I
| agree that a complicated dev environment is bad. However, that
| isn't a devops thing. You can have devops with a monolith that
| is statically compiled into a single binary. I've built it and
| done it. It is quite nice.
| kazen44 wrote:
| we currently do this at our company.
|
| compiled code gets (autodeployed) with SFTP in some cases
| even.
|
| deployment has become massively complex because of the need
| to keep production and test environments roughly the same.
| Deployment to production should not be complex and basically
| boils down to the following:
|
| - think about method of getting the binaries installed on the
| system in a secure way. - think about sending some state back
| that the binary has been installed succesfully.
|
| - add node/member/app/whathaveyou to the production
| membership pool/loadbalancer/BGP Route reflector/whathaveyou.
| sharken wrote:
| Developers typically care about developing the business side of
| the application and have little interest in the rest.
|
| Actually, it can even be split in three, where developers are
| done when the Pull Request is done.
|
| Deployment, tooling and IaC is then handled by developers who
| specialize in this area.
|
| A third group is then responsible for audit and security on the
| chosen platform, e.g. Azure.
| 988747 wrote:
| >> Developers typically care about developing the business side
| of the application and have little interest in the rest.
|
| Correction: developers care about using cool new programming
| languages and frameworks. "Business side of the application"
| has to be forced on them by business analysts and scrum masters
| :P
___________________________________________________________________
(page generated 2021-12-19 23:00 UTC)