[HN Gopher] Do-nothing scripting: the key to gradual automation
___________________________________________________________________
Do-nothing scripting: the key to gradual automation
Author : jabo
Score : 370 points
Date : 2021-11-02 16:15 UTC (6 hours ago)
(HTM) web link (blog.danslimmon.com)
(TXT) w3m dump (blog.danslimmon.com)
| silvester23 wrote:
| (2019)
|
| Discussed at the time:
| https://news.ycombinator.com/item?id=20495739
| crispyambulance wrote:
| Cool. A nice medium on the way between "seat-of-your-pants"
| manual and completely inscrutable automation!
|
| People like to brag that they never do something manually more
| than once, but in reality it takes time and practice to be able
| to fully automate any non-trivial process. You usually can't do
| it in one go. I really like that this article embraces that.
|
| I especially like how it's broken down such that any particular
| function can be swapped out with full automation once the author
| gets enough confidence to be able to do that.
| lenkite wrote:
| This is nothing but a checklist right ?
| andrewflnr wrote:
| It's an interactive checklist with a gradual upgrade path to
| full automation.
| gorgoiler wrote:
| Nice. A class with a single method can be nicely modelled as a
| closure.
|
| Returning higher order functions and using format's kwargs is an
| even more succinct version of this. def
| explode(**k) def run(): return TheBomb(
| '{user} is!'.format(**k)) return run
| run(user='gorgoiler')
| nerdponx wrote:
| I'd almost always rather have a class than a function returning
| a function, in a language where classes are typical and
| natural. It imposes more structure on the code, provides some
| degree of namespacing, and makes it easier to extend.
| mr-karan wrote:
| The author has written a Golang version of the same as well [1].
|
| 1: https://github.com/danslimmon/donothing
| ImportOllie wrote:
| I really like this, but in the example is there much value
| defaulting to wrapping each function in a class?
| IshKebab wrote:
| Yeah it's not the greatest style. You also don't need to derive
| from `object` anymore. But it gets the idea across and the
| advice is great!
| tusharsadhwani wrote:
| notice the raw_input(), it's written in python 2.
| Jtsummers wrote:
| It gives a uniform interface (the _run_ method) for each step.
| You can also, then, encapsulate more logic around that
| particular step if it needs more than just a single _run_
| function or some other stateful component.
| _dain_ wrote:
| A plain function has a uniform interface too: just calling
| it.
| Jtsummers wrote:
| True, but that doesn't help you with encapsulation or the
| other (potential) benefits of using classes like
| inheritance.
| oehpr wrote:
| Hey I independently came to this as well!
|
| We were performing some migrations to our servers and we weren't
| sure we had nailed the process down. Automating the procedure has
| a problem in that I can't just "reset" the server and start
| again. I could try to make all my operations idempotent or using
| a babushka style "Check - Set - Check" methodology, but that
| would greatly expand the work for a one off procedure that once
| it's done it's done.
|
| I wasn't even sure of the generality of the steps, and they were
| complicated enough that copy and pasting them with all their
| parameters was tedious. So I created a structure just like this!
|
| I called it a living procedure. An excerpt:
| #!/bin/bash source ./procedure-lib.sh question
| "Which client is this?" old_client question "What worker is
| $old_client on?" old_worker question "What is the targets
| name going to be? (it's fine if it's still $old_client)"
| new_client question "What worker is $new_client going to be
| on?" new_worker question "What branch is $new_client going
| to be on?" new_branch question "What should we name the
| migration? (typically something like migraiton-$(date --iso))"
| migration_name #================ Initialize the client
| on the new worker ================ step "Login to
| $new_worker and Initialize $new_client in legacy mode using
| run.sh init_legacy, set their branch to $new_branch"\
| 'eg. ssh '"$new_worker"'.lightship.works "sudo -i bash -c
| \"
|
| These question functions also remembered the answer you put in
| last time (it was based off what the text of the question was, so
| questions that depend on the answer of previous questions did not
| presume anything) so that if you just hit "enter" then it would
| load the last one. This was vital because when a procedure ended
| up incorrect or needed to be amended, you could ctrl+c the
| script, fix it, then spam enter until you reached the point you
| were last time.
|
| I've been thinking about creating a better structure to these
| scripts that allow you to gradually transform these living
| procedures into babushka style "Check Set Check" automations.
| mig39 wrote:
| This is a lot like the checklists a lot of industries use, just
| automated on your shell instead of being on paper.
|
| I'm thinking of a pilot's checklist, or an operating theatre
| checklist.
| mypalmike wrote:
| There's a difference though. I've worked in ops teams where
| there were many such checklists in Word documents, wikis, or
| whatever. Standard procedures involved reading from these
| runbooks and following the steps.
|
| But here, you have code that plays the same role, but which can
| eventually be changed out with code that actually does the work
| for you. It's a seemingly small difference but it's actually
| huge if you have some decent developers who can spend a bit of
| time every once in a while pushing the needle forward in
| automating these manual steps.
|
| Another note: I've found that it's hard to even get teams to
| consider the minor change of using the code version instead of
| the wiki. Inertia is a real problem.
| nicbou wrote:
| I really like this idea. It goes hand in hand with an older post
| of mine: "no script is too simple" [1].
|
| A script is a recipe that your whole team can follow. It's easier
| to update a recipe than to propagate changes across the whole
| team.
|
| It's also similar to a flight check, which is a fancy name for a
| reusable to-do list.
|
| [1] https://nicolasbouliane.com/blog/no-script-is-too-simple
| kaycebasques wrote:
| I also independently stumbled on this idea. My rational was that
| it turns the process into more of a checklist, without all of the
| hassle of handling edge cases in my code. Because usually
| whenever I try to fully automate I re-learn the wisdom of that
| xkcd cartoon about "automation expectations versus reality".
|
| Another wise thing about it is that it's a relatively low-
| friction way to start documenting institutional knowledge. Your
| engineers might hate writing docs, but maybe they won't mind
| writing a bash script like this. Which is essentially
| documentation in disguise. And since it's (presumably) living
| with the rest of your version control system you increase the
| chances that your engineers will explain changes to the process
| over time (via commit logs).
| kristianpaul wrote:
| The lesson for me here is to actually have scripts written where
| the business logic is clear to see and change.
|
| This is a workflow in a python script but could also be a tool
| just by adding argument parsing. But also tools can be called
| from software pipelines and have tools to execute this for us
| (automation).
|
| Look at https://aosabook.org/en/500L/a-continuous-integration-
| system...
| dragontamer wrote:
| How about you just... cut out the middleman and write an expect
| script (or pexpect script, for you Python programmers) ???
| simonw wrote:
| I like Observable https://observablehq.com notebooks for this
| kind of thing - quickly building automated scripts that simply
| accept user input and use it to dynamically construct a copy-and-
| paste output.
|
| You can run and edit them entirely in the browser, they can do
| anything you can do with JavaScript and they're easy to share
| with other people and create forks.
| throwaway20371 wrote:
| This is the way. I wish this were taught in computer science
| class, development bootcamps, operations team onboarding,
| anywhere there is a procedure that is even slightly complicated
| to automate. It is the absolute best solution there is.
|
| * Documentation of the entire procedure is contained in one
| place. No need to go sifting through 20 different sources of
| documentation. This lowers the human emotional barrier to "just
| get it done", as people will always avoid things they aren't
| comfortable/familiar with, or don't have all the steps to. This
| central point of documentation also enables rapidly improving the
| process by letting people see all the steps in one place, which
| makes it easier to fix/collapse/remove steps.
|
| * Automation in small pieces over time avoids the trap of "a
| project" where one or more engineers have to be dedicated to this
| one task for a long period of time. Most things shouldn't be
| automated unless there is demonstrably greater value in the cost
| of automating them than the cost of not doing so. Automating only
| the most valuable/costly pieces first gives immediate gains
| without sinking too much into the entire thing.
|
| * One unified "method" to encapsulate any kind of process means
| your organization can ramp up on processes easier, reducing
| overall organizational cost.
|
| * In the absence of any other similar process, you are
| _guaranteed_ to save time and money.
|
| I would say that the only potential downside is if someone
| decides to "engineer" this method, making it more and more and
| more complicated, until it loses its value. KISS is a requirement
| for it to be sustainable.
| csdvrx wrote:
| > only potential downside is if someone decides to "engineer"
| this method
|
| It can be engineered, if you follow a gradual process.
|
| On servers, I keep a log of what was deployed in a root
| directory following the sequential number _ goal format (ex:
| 00_partitions ... 90_web_server)
|
| It is not fancy, most of the logs are not even scripts: many
| are just ASCII text files, that will only be used as a
| checklist if the same "goal" has to be achieved again. For
| example, 00_partition may be "gdisk /dev/nvme0n1" followed by a
| copy-pasted list of the partitions and some quick description
| about why it was done that way.
|
| But that's on the first iteration only: the next iteration
| turns that into "do-nothing" script, the next iteration into a
| better script with basic checks (supporting both /dev/nvme0n1
| and /dev/sda), then exception handling (if partitions already
| exist, etc), and so on: this gradual complexification process
| avoids the "premature optimization" of creating infrastructure-
| as-code for what you rarely need, while optimizing and fine
| tuning the parts you most often need.
|
| Someone will certainly mention Terraform, or Ansible, or
| something else - yes, they exist and they are nice, but if you
| are doing everything there, you are over-engineering and
| wasting time: not everything needs your equal attention!
|
| If you only install a webserver once in a blue moon, make a
| .txt checklist of the steps you followed.
|
| But if you leave and breathe nginx options and certificate
| deployment, fully automatize all that, including the obscure
| details of what may fail if you use let's encrypt with some
| specific DNS configuration!
|
| And if you don't know yet which is which, start small (a .txt
| checklist will cost you a few minutes) and the next time you
| find yourself doing the same thing, do it better using the
| previous artifact (the .txt file) to create a better one (a
| script, then a better script etc)
| chousuke wrote:
| I would argue that _all_ deployments (no matter how small)
| should have configuration management.
|
| In the simplest case, a deployment consists of: 1. Install
| packages 2. Install configuration files, possibly from
| templates. 3. Configure services to start on boot.
|
| This kind of automation is _trivial_ to do with almost any
| tool, but there 's no reason not to use something like
| Ansible that's designed for infrastructure automation,
| because you get encrypted secrets, templating and idempotency
| with zero effort and the result can be stored in a git
| repository somewhere.
|
| The Ansible playbook is as fast to write as installing and
| configuring things manually, and after some practice, it's
| _faster_ and the resulting system is of higher quality.
|
| Even if you end up using the automation you write only once,
| it still has value because it doubles as a formalized
| description of what you did, and can be stored together with
| additional documentation. Over time you will also accumulate
| a library of bits and pieces that you can copy over to new
| setups, further improving your speed and quality.
| Arch-TK wrote:
| This is assuming you already know Ansible.
| chousuke wrote:
| Sure, but then again, typing shell commands into text
| files is assuming you already know shell commands. You
| have to spend time to learn your tools at some point.
|
| For simple configuration management, Ansible is a
| straight upgrade to most shells because of idempotency
| alone, never mind the fancier features like the more
| advanced modules, multi-node orchestration, or encrypted
| vaults. The YAML syntax is dumb and it has its issues,
| for sure, but it still does even the simple things much
| better than plain old shell.
|
| Anyone who has any familiarity at all with UNIXy systems
| can learn Ansible from zero well enough in a day or two
| for it to start becoming truly useful, and if you don't
| have the foundation for that... why on Earth are you
| setting up a web server? I mean, it's of course fine to
| tinker with things for learning, but I was assuming a
| real deployment scenario.
| csdvrx wrote:
| > I would argue that all deployments (no matter how small)
| should have configuration management.
|
| I would argue that in most cases, they don't need anything
| but some documentation explaining what was installed, and
| why.
|
| I will take a word file with screenshots over a broken
| script in an obscure language every single time.
|
| > but there's no reason not to use something like Ansible
| that's designed for infrastructure automation,
|
| There's a big one: my time isn't free.
|
| If someone is willing to waste money on that, sure, I'll be
| happy to bill them for their extravagant tastes (but only
| after having done my best to explain them it's a waste of
| money)
|
| And still, I will think about the next person that may have
| to maintain or tweak whay I wrote, so I will also leave a
| document full of screenshots in case they don't know
| ansible or whatever new fashionable tool that the client
| may have specifically requested.
|
| > it's faster and the resulting system is of higher
| quality.
|
| Not everything needs to be of high quality.
|
| Forgive me if I'm assuming your gender, but I see a lot of
| black-and-white thinking among male sysadmins/devops: it's
| good or it's bad, it's high quality or it's not.
|
| I prefer to have a "sufficient" degree of quality: if a
| checklist is enough, I will not waste time writing a
| script. If a shell script is enough, I will not waste time
| writing proper code - and so on.
|
| > Over time you will also accumulate a library of bits and
| pieces that you can copy over to new setups, further
| improving your speed and quality
|
| Except you assume a continuous progress, without any change
| of scope or tools, and with the tools themselves never
| evolving. It doesn't work like that: over time, you will
| accumulate a bunch of useless code for old versions.
|
| Even small inconsequential changes (like unbound in debian
| 11 requesting spaces before some options, which wasn't the
| case before) will take some time and effort. Why waste your
| energy one one shots?
|
| The do-nothing approach argues that you should avoid
| premature optimization, which strikes me a good approach in
| software in general.
| throwaway20371 wrote:
| > If you only install a webserver once in a blue moon, make a
| .txt checklist of the steps you followed.
|
| This brings up a very important point about checklists that I
| don't think gets enough attention.
|
| The problem happens when somebody "updates" that web server
| in-place. If they try to record what changes they made in the
| middle of the checklist, eventually when someone tries the
| whole checklist from the beginning, they'll find it's now
| broken; the steps aren't working as expected. This happens to
| me when I try to record changes in my VirtualBox
| configuration after I add a new system package or something;
| later I try to re-deploy my vbox, and it breaks.
|
| So checklists should be considered immutable. Once you create
| them, don't assume they will work again if modified. Instead,
| if you make any change to the checklist, you must follow all
| the steps from beginning to end. This way you catch the
| unexpected problems and confirm the checklist still works for
| the next person.
| csdvrx wrote:
| > The problem happens when somebody "updates" that web
| server in-place.
|
| Imagine this is 28-nginx : I would create another script
| 29-nginx-update only recording the update, even if it:
| "echo apt-get update; apt-get upgrade nginx ; echo "make
| sure to fix variable $foo"
|
| Next time I have to do that, I will integrate that into
| 28-nginx and remove 29-nginx-update
|
| > eventually when someone tries the whole checklist from
| the beginning, they'll find it's now broken; the steps
| aren't working as expected.
|
| Maybe I don't understand the issue, but my scripts or text
| files are simple and meant to be used in sequence. If I
| hack the scripts, I make sure it still works as expected -
| and given my natural laziness, I only ever update scripts
| when deploying to a new server or VM, so I get an immediate
| feedback if they stop working
|
| Still, sometimes something may work as expected (ex: above,
| maybe $foo depends on a context?), but it only means I need
| to generalize the previous solution - and since the script
| update only happen in the context of a new deployment,
| everything is still fresh in my head, so I can do it easily
|
| To help me with that, I also use zfs snapshots at important
| steps, to be able to "observe" what the files looked like
| on the other server at a specific time. The snapshots
| conveniently share the same name (ex etc@28-nginx) so
| comparing the files to create one ot more scripts can be
| easily done with diff -Nur using .zfs/snapshot/ cf https://
| docs.oracle.com/cd/E19253-01/819-5461/gbiqe/index.ht...
|
| Between that + a sqlite database containing the full
| history of commands types (including in which directory,
| and their return code), I rarely have such issues
|
| Shameless plug for that bash history in sqlite:
| https://github.com/csdvrx/bash-timestamping-sqlite
|
| > So checklists should be considered immutable. Once you
| create them, don't assume they will work again if modified.
| Instead, if you make any change to the checklist, you must
| follow all the steps from beginning to end.
|
| I agree: if I don't have time to fix 28-nginx, I write
| 29-nginx-update instead, with the goal next time to
| integrate it. But I don't try to tweak 28-nginx if I know I
| won't have the time to test it.
| throwaway20371 wrote:
| It _can_ work this way (that 's how software patches have
| historically worked) but if you don't test it from the
| beginning, you will still find the odd case where that
| added step is broken, even though it seemed like it
| should have worked. The more you use that method, the
| more chances for breakage.
|
| If you don't want to repeat the steps from the beginning,
| you could make a completely separate checklist to be
| followed on a given system that includes things like
| "make sure X package is installed", "make sure Y
| configuration is applied", so that the new checklist
| accounts for any inconsistencies. This is pretty common
| anyway as checklists are broken up into discrete purposes
| and mixed and matched.
| rambambram wrote:
| Oh, what do I love .txt files. I use them for all kinds of
| simple checklists, logs, and basically everything. I had a
| manager once (not in the software field though) and she asked
| me annoyed why I kept using these strange files, and if I
| wanted to "just use Word and .doc files because that was more
| safe and compatible". I wasn't able to explain that text
| files were there and will be there for a long time. She also
| didn't understand the difference between a text file and a
| document. Not even when I pointed her to the .txt in the
| filename.
| nerdponx wrote:
| The problem I see is that someone will inevitably update the
| procedure (or make a change that unknowingly requires a change
| in the procedure) and not update the script. Either because
| they are pressed for time or because they forgot. Same as any
| other documentation.
|
| The solution ultimately is for PMs to get it into their heads
| that software and infrastructure require maintenance like
| anything else, and consistently refusing to schedule time for
| software/dev-tool maintenance (such as updating documentation)
| has the same effect as refusing to schedule time for physical
| equipment maintenance. Then and only then do engineers have the
| freedom to set up mandatory procedures and checklists for their
| work, the way all engineers should be allowed and encouraged to
| do.
| masukomi wrote:
| > The problem I see is that someone will inevitably update
| the procedure (or make a change that unknowingly requires a
| change in the procedure) and not update the script
|
| why would your procedure be to do anything _other_ than "run
| script foo and do what it says"? If your procedure is not
| that, then your procedure doesn't reflect reality, and thus
| is outdated documentation that needs to be updated.
|
| if the steps of the procedure only exist within the script
| then there's only one place to update it. And yes, this
| suggests the script should be very readable.
| nerdponx wrote:
| > If your procedure is not that, then your procedure
| doesn't reflect reality, and thus is outdated documentation
| that needs to be updated.
|
| Configurations change all the time. There is no
| technological safeguard against someone forgetting to write
| down the change in the playbook script; it has to be
| organizational.
| chousuke wrote:
| Declarative configuration management systems solve this
| by unchanging your configuration after someone messes
| with it manually. :) Hard to forget to change the
| automation when it persistently undoes all your hard
| labour.
|
| You _can_ help solve the problem with technology, you
| just have to make the solution easier than working around
| it.
| csdvrx wrote:
| A script comparing the md5 or the timestamp of the
| configuration files against the md5 or the timestamps of
| the log entry in charge of these files can do that
|
| I mean, if /etc/hosts is more recent than /log-
| directory/03-static-hosts-in-etc or the md5 you have
| recorded for this file, a daemon can easily create a
| ticket / send an email to whoever was logged at the time
| of the change.
| neo2006 wrote:
| If you are going through writing the do-nothing script anyway
| why not do a do-something script and remove the error prone
| human out of the loop, it also can serve as documentation and
| if you are disciplined enough to never make a change other then
| trough the automation you can have the benefit of source
| controlling it and have documentation with historical context
| renewiltord wrote:
| Simple.
|
| Do nothing script: echo "Go to your Google
| Account settings and set a vacation auto responder"
|
| Do something script: import oauth2 # I have
| already lost, there is no salvation in life, without truth or
| peace there is only the void that beckons
| computronus wrote:
| I happen to have just written a do-nothing script, so I can
| answer why I found it helpful vs. a do-something script.
|
| - I am still developing the procedure in the script, so it is
| premature to automate. A do-nothing script still benefits you
| by telling you exactly what to do - in my case, spitting out
| exact commands to run - but you can assess its steps before
| performing them.
|
| - The script still gathers a lot of information and
| associates it together in order to figure out the correct
| commands. That work is valuable all by itself.
|
| - Even though you have to run commands yourself, it's copy-
| and-paste vs. hand-typing, so it's already less error-prone.
|
| - The script documents the procedure even without automation,
| so the benefit is immediate.
|
| Now, I think that it's better for a do-nothing script to
| _evolve_ into a do-something script. But, if that effort is
| delayed or never happens, at least you've got something.
| dragonwriter wrote:
| > If you are going through writing the do-nothing script
| anyway why not do a do-something script and remove the error
| prone human out of the loop
|
| You will, eventually, ideally.
|
| But that's extra up-front cost (especially when do something
| involves a complex integration), the do-nothing script
| crystallizes the definition of the existing manual process
| allowing incremental automation of steps (also, making a to-
| do list for automation.) It is an example of "do the smallest
| useful unit of work".
| mdoms wrote:
| You have comprehensively missed the point. And your questions
| are answered in the article:
|
| > At first glance, it might not be obvious that this script
| provides value. Maybe it looks like all we've done is make
| the instructions harder to read. But the value of a do-
| nothing script is immense:
|
| > * It's now much less likely that you'll lose your place and
| skip a step. This makes it easier to maintain focus and power
| through the slog.
|
| > * Each step of the procedure is now encapsulated in a
| function, which makes it possible to replace the text in any
| given step with code that performs the action automatically.
|
| > * Over time, you'll develop a library of useful steps,
| which will make future automation tasks more efficient.
|
| > A do-nothing script doesn't save your team any manual
| effort. It lowers the activation energy for automating tasks,
| which allows the team to eliminate toil over time.
| cseleborg wrote:
| > It is the absolute best solution there is.
|
| I prefer checklists. With a checklist, I can mark my progress
| as I go through the motions and, more importantly, interrupt
| the work even for several days, before I pick it up again.
| Checklists are much, much easier to adapt. They can also hold
| more information than progress indication, like important
| outputs of the procedure that need to be used somewhere else
| later, etc. They can be archived in case I need to understand
| how I did it on that particular occasion one year ago.
|
| Google Docs added checklist a while ago and I think they are
| very handy. And very KISS.
| csdvrx wrote:
| > Checklists are much, much easier to adapt. They can also
| hold more information than progress indication, like
| important outputs of the procedure that need to be used
| somewhere else later, etc.
|
| echo "## Step 2/10 : preparing the SSH key for xxx"
|
| (...)
|
| KEY=$( cat .ssh/id_rsa.pub )
|
| echo "## Do not forget to use this important output somewhere
| else later: $KEY"
|
| > They can be archived in case I need to understand how I did
| it on that particular occasion one year ago.
|
| (...)
|
| # 20191102
|
| # TODO: 2 years ago, we decide to use 4096 bits key, make
| sure to check the length
|
| # in case I forget, how to do that, the number of bits can be
| forced with -b 4096
|
| # 20201102
|
| # WONTFIX: use ecdsa for the specific host yyyy
|
| So I stand with the author: this is the absolute best
| solution there is: it can be as simple or as thorough as you
| need, while being very low tech and simple to use.
|
| And when you find yourself needing to make say an Ansible
| configuration, you already have most of what you need.
| throwaway20371 wrote:
| What about the do-nothing script do you find is different
| than a checklist? To me the whole thing is already a
| checklist, there just aren't check-boxes.
| magicalhippo wrote:
| I'm primarily a Windows guy, but I'm dabbling more and more
| with Linux lately. Especially after I got a few Raspberry Pi's
| and various clones set up.
|
| What I've ended up doing is to just write Word documents
| containing what I do. I keep my Word documents in a single
| directory on my OneDrive, so I can access them on all my
| machines.
|
| Raspberry Pi iSCSI server? One document for that, containing
| the links to guides used, commands I've run and scripts made. I
| use headers to organize the sections, like one for compiling
| the custom kernel, one for configuring iSCSI on the Pi, one for
| using it on the NAS etc.
|
| Need to create a new Docker SMB mount? Once the right
| incantation has been found, a small script is made and also
| added to the right Word document, ready to be pasted into the
| next machine I might need it on.
|
| It's not terribly pretty and not at all fancy, but I found it's
| low enough barrier that it's easy to do and maintain, and it's
| very helpful to have it all in one place.
|
| Had I been a Linux guy first and foremost I would probably have
| used something else than Word, but it works and I find the
| separation between the regular font for comments and monospace
| for script makes it easy to quickly distinguish. It's also easy
| to add screenshots for clarifications etc.
| lucb1e wrote:
| I think using an executable format like in the article,
| you'll find it's not actually that much harder to read than a
| Word document with proportional fonts. And I'm not sure where
| you need screenshots if it's about doing things on a non-
| Windows system, even if that seems like an odd transition at
| first :)
| magicalhippo wrote:
| Yeah might have to try it out.
|
| The images could be for things like performance graphs,
| interactive menu choices, photos of GPIO wiring etc. Not a
| huge loss to not have them, but given it's dead easy to add
| to a Word document, why not?
| csdvrx wrote:
| > Had I been a Linux guy first and foremost I would probably
| have used something else than Word, but it works and I find
| the separation between the regular font for comments and
| monospace for script makes it easy to quickly distinguish.
| It's also easy to add screenshots for clarifications etc.
|
| As a windows girl, I recommend you give a try to the
| notebooks like RStudio: you can add screenshots (or script
| them with ahk, nircmd etc to automatically screenshot at some
| points of the execution) and execute blocks of commands in
| about anything.
|
| The notebook approach has the additional nice feature of
| stopping execution as soon as a block fails, giving you the
| opportunity to fix it before you continue from that point,
| something often more tedious with Linux scripts (where you
| need to commend the beginning if your script isn't
| idempotent)
|
| Ideally, you would always write idempotent scripts, but
| who've got time for that :)
|
| In practice, if you try to avoid wasting effort, it's often
| the icing on the cake, once everything else has been done. So
| I like notebook environments for this simple feature:
| piecewise execution with verbose output (similar to bash -xe)
|
| That said, a directory per machine (and a subdirectory with
| all the drivers and specific software) on Onedrive with a RTF
| file full of screenshots (because wordpad is everywhere!) is
| how I work most of the time :)
| dradtke wrote:
| Braintree created a tool for doing something similar:
| https://medium.com/braintree-product-technology/https-medium...
| tmerr wrote:
| I do something similar but take it a step further and statefully
| track the progress of the workflow. The code generates a file
| that is like a TODO list, and a separate command runs a single
| step before marking it complete. Great for longer running
| workflows (days).
| Arch-TK wrote:
| I like the article. This is a great idea. I'm going to implement
| it over the next week. I have high hopes.
|
| That being said, I have absolutely no clue what the advantage is
| to the python programming style exemplified in the article. Can
| someone explain why on earth I would ever want to convert:
| def do_something(context): pass ...
| do_something(c)
|
| into class DoSomething(object): def
| run(self, context): pass ...
| DoSomething().run(c)
|
| What is the point here?
| ttymck wrote:
| In that contrived example, there is no point. In the real
| world, there are plenty of reasons to use a class instead of a
| function. Extensibility is the first that comes to mind. What
| answers are you expecting (i.e. performance)?
| bazhova wrote:
| This is a really nice solution. For the 99% who don't work for a
| place with a massive DevOps team, this approach is a nice middle
| ground that actually works. If management doesn't like it they
| don't have to know. These scripts are for getting through the day
| with 10% more energy to spare at the end.
| dctoedt wrote:
| Could crude shell scripts be used for this purpose? For example,
| in zsh [0]: $ echo "Do the first step, then press
| any key."; read -s -k
|
| EDIT: Apparently so: [1]
|
| [0] Modeled on https://stackoverflow.com/questions/5215343/how-
| can-i-pause-...
|
| [1] https://news.ycombinator.com/item?id=20495739
| orf wrote:
| Terraform. My god, just use terraform.
|
| Stop scripting random stuff to create multiple untracked
| resources in different places by hand.
|
| Slap this[1] on top of this[2] and add whatever other services
| you want, and boom. Done.
|
| 1. https://stackoverflow.com/questions/49743220/how-do-i-
| create...
|
| 2.
| https://registry.terraform.io/providers/1Password/onepasswor...
| iso1631 wrote:
| Congratulations, you've just made the situation worse by
| rushing to what you think a solution is, rather than seeing
| that the actual problem is.
| orf wrote:
| The problem is you've got a bunch of things that need to be
| in a particular state (for every employee, for every office,
| for every _foo_ ).
|
| There are a few solutions to this. Documentation with manual
| actions, a bunch of imperative code to create but not manage
| these things, or something that describes what you want
| without the "how I get it".
|
| If you're at the stage where you think writing a bunch of
| random Python code to prompt a user with steps is an
| acceptable solution then I can't see how the situation can
| get much worse.
| avisser wrote:
| I think you're focused on the domain chosen by the author and
| not the more general technique.
|
| Replace the steps with "Releasing a build to my app store" or
| something else.
| scubbo wrote:
| This type of "helpfully-correcting disagreement" (rather than
| pointless-but-satisfying snark) is one reason why I greatly
| prefer Hacker News to almost-every other social network.
| Thank you!
| c7DJTLrn wrote:
| Yes, but I think you've been spoilt if every shop you've worked
| at already knows and uses Terraform. It's a real uphill battle
| to introduce it to a place where nobody has the mindset of
| treating computers as cattle, has never played around with
| declarative tools or config management before.
| sovietmudkipz wrote:
| I did this after seeing this before on HN. There were a few
| processes that were manual that benefitted from the technique in
| the article.
|
| Learn from my folly: I even called them "do nothing scripts,"
| referencing this article. However; I was judged by peers for not
| writing the full automation versions as they didn't appreciate
| the idea of gradual automation (programmer hubris?). Saying "do
| nothing scripts" in meetings did catch the awkward attention of
| leadership.
|
| As a description, "do nothing" communicates a lot. As a brand,
| "do nothing" can use some improvements.
|
| My short prescription of turning "do nothings" into "do some
| things" into "do all the things" didn't help. We had some new
| people join the team and they had fun turning the do nothing
| scripts into a document. * Sigh *
|
| I still build these type of process description scripts still. I
| usually don't advertise them to peers until they do some of the
| things nowadays.
| jjk166 wrote:
| I like the term "scaffold". It intuitively conveys that this is
| something quickly and easily started and which can stand on its
| own, but at the same time is fundamentally incomplete and a
| stepping stone towards something more permanent of greater
| value.
| abalaji wrote:
| "scaffold scripts" that actually has a nice ring to it and is
| alliterative which makes it more memorable
| bredren wrote:
| How about IterScripts. As in, scripts you iterate on.
| dflock wrote:
| It's a Run Book Script.
| bryan_w wrote:
| If you have a ticketing system that has an API, change the
| print() statements into file_ticket_and_wait()
| throwaway20371 wrote:
| You could call it an "E-I Script" for Efficiency Interest
| Script. Over time your costs are gradually lowered as each step
| is automated - like accruing interest in a savings account.
| lumannnn wrote:
| Maybe "<name>.playbook.sh" could give it a better look?
| scrooched_moose wrote:
| Something like "Checklist Script", as that's what they really
| provide?
| andrewflnr wrote:
| "Runnable checklist"
| BariumBlue wrote:
| "Process Automation" would be a good, encompassing term I
| think.
|
| Even if there's no code being run, having all the steps down
| and easy to follow is a way to make the process "automatic".
| The term is extendible to completely automated computer code,
| while including non-computer-code (human) automation.
| simonw wrote:
| How about "scripted playbook"?
| bredren wrote:
| Too easily confused w Ansible
| qwertox wrote:
| I agree with this. It's like a Todo App but a Checklist
| Script.
| colmanhumphrey wrote:
| Maybe "Gradual Scripting" could work
| adrianmonk wrote:
| How about "immediate placeholder scripts"?
|
| "Immediate" is meant to accentuate that they can be deployed
| and start delivering value _right now_ , something that
| (usually) can't be said for the implementation of actual
| automation.
|
| And "placeholder" is meant to convey that you don't intend to
| stop there.
| mamcx wrote:
| Stepwise to-dos
| a9h74j wrote:
| Next time try MVS -- Minimum Viable Script.
|
| Actually, other applicable terms could be: a) programmed
| documentation; b) script as single-source of procedural truth
| mro_name wrote:
| it's a running checklist, isn't it? Awesome.
| [deleted]
| JeremyNT wrote:
| Lengthy previous discussion (2019):
| https://news.ycombinator.com/item?id=20495739
| gregwebs wrote:
| I like to do this with a checklist. In Confluence you can clone
| the checklist page for the onboarding/release, etc and check off
| the cloned page.
|
| Confluence is wiki-style (edit first, ask questions later), but I
| could definitely see git-style (pull request first) as an
| improvement for some situations.
| bionhoward wrote:
| Makefiles can be great for this sort of thing. Over time you
| build up a really nice library of shell commands and it's trivial
| to organize them into a DAG of dependent steps. Lack of makefiles
| is a strong reason to avoid windows dev machines like the
| plague...(IMHO)
| guhidalg wrote:
| Windows has MSBuild that is just as expressive as Makefiles,
| don't use the lack of make as an excuse.
| SeriousM wrote:
| Usually I write instructions in markdown files and test them by
| redo all the steps. This saved me already many times because I
| will forget how to renew this damn certificate two years later,
| or setup the logging engine after 18 months ago. No need for a
| script.
| qwertox wrote:
| I'm so scared of my certificate renewal script. It's really
| cool and sends me a summary via email of the executed tasks and
| runs automatically every month. But when I look at the code I
| don't know what the hell I was thinking when I wrote it.
|
| I mean, it even fetches the certificates prior to renewing them
| and then re-fetches them afterwards in order to make sure that
| the servers are actually using the new ones.
| dnautics wrote:
| I think this is eventually going to be how people use elixir's
| livebook library.
| phaedrus wrote:
| My experience in corporate IT convinced me that even if an entire
| release could be automated down to a single press of a giant red
| button (and even if undoing the release could be another, single
| button), then culturally our organization would still figure out
| a way to turn the pressing of that button into a six hour ordeal
| requiring the participation of twelve people.
| wing-_-nuts wrote:
| This is so true it hurts.
| oscribinn wrote:
| My empirical evidence backs this up, I had deploying new
| releases to all servers completely automated down to just
| creating a tag in GitLab and every time the decision of
| actually running it turned into a series of emails and meetings
| throughout the week.
| catern wrote:
| The idea here can be generalized to a primitive programming
| construct: The magic function "wish", which can do anything you
| want, just give it a string description.
|
| For example: wish('copy these files to this
| host', files, host)
|
| You can augment this further by allowing the program to specify a
| return type from the wish, and magically the wish will produce
| the type you want. So for example: data =
| wish(Path, 'the path of the data files I need')
|
| Of course, the wish is actually implemented by sending a request
| to some human user. In my "wish" library for Python
| http://rsyscall.org/wish/ you can have a stack of handlers for
| wishes, just like exception handlers. If later you want to
| automate some wish, you can specify a handler which intercepts
| certain wishes and forwards the other ones on (by wishing again,
| just like an exception handler can re-raise).
| didip wrote:
| So... I am confused about something, isn't it better to start
| writing some automation scripts and document the missing piece
| for later?
| slim wrote:
| That's implied. Optimisation is not premature when obvious.
| throwaway20371 wrote:
| That's basically what the do-nothing script is. The difference
| is that before you write any automation, you document all the
| steps in the script. Right there - when you've got it all
| written down, and no automation work has been done yet - that
| in itself is a very valuable piece of work. Now you can point
| anyone in the company to that script, and they can all
| accomplish the task without having to figure it out for
| themselves. You can now scale that process N times (N = the
| number of people in your company). Just writing down the steps
| has become a force-multiplier of repeatable work. Then as you
| begin automating each step, people automatically receive the
| benefit of that automation. Because the documented steps and
| the automation are in the exact same place, both will always be
| up-to-date.
| tossaway9000 wrote:
| > Create an SSH key pair for the user.
|
| > Send the user their private key via 1Password.
|
| Why are you generating *private keys* for users, then sharing
| them? Not that this impacts the automation bits but IMHO users
| should known how to generate and maintain a key pair, and send
| you the public key.
| iso1631 wrote:
| Agree, it's painful, and it really takes me out of the larger
| point they are trying to make
|
| A workflow should be be User creates an SSH
| key pair, the private key never leaving the computer
| User sends public key to authoriser Authoriser pushes
| public key to Git (presumably an email + key touple?)
| Wait for the build job to finish (not sure what this does)
| Build process sends the email saying "You can now use your key"
|
| Same with say a wireguard key. Or an SSL certificate.
|
| I think the larger point is that you just have step by step
| instructions, and thus dont need to catch edge cases, but it
| also makes it harder to avoid skipping a step, which seems
| reasonable (I do this myself in some areas)
| renewiltord wrote:
| Did it all the time. Users weren't sophisticated. They want a
| tool they can put a user/pass into and then upload and
| download. They have maybe some conditions from IT (SFTP okay,
| S3 okay, encrypted at rest, whatever).
|
| We did this trivially with S3, our implementation guy gives
| them an access key ID and secret access key, tells them to
| install Cyberduck, and gives them a URL to paste. We're off to
| the races.
|
| Having the user generate the thing will turn going live from
| hours to days.
|
| I've also done this analogously with SFTP. You keep the creds
| so you can help them because they'll type it in wrong, their
| software will fuck it up, whatever.
| DantesKite wrote:
| Why?
| tossaway9000 wrote:
| It's not a "private" key if its shared. Only the end user
| needs to know the private key details.
|
| Consider the usecase of generating a user their initial
| password for a service, this almost always results in the
| user needing to immediately reset their password after
| initial login, this doesn't happen if someone is generating
| both parts of the key pair for you.
|
| My sysadmin doesn't need to know my password, and doesn't
| need to know my private key or passphrase, that would allow
| them to impersonate me.
| ghostly_s wrote:
| well, your sysadmin created your AD account, and probably
| has some means of root access to your box too, so they
| could already impersonate you. *
|
| *this may or may not be true depending on how Enterprise-y
| your workplace is.
| chousuke wrote:
| The difference is in the kind of audit trail it leaves.
| If a sysadmin impersonates a user, that leaves a
| different kind of trail than a user logging in with their
| own key that only they can access.
|
| In principle, the sysadmin should explicitly _avoid_
| knowing any of the user 's secrets, because if they do,
| the user can shift blame onto them: "It wasn't me, it was
| the admin!".
|
| I will never generate a private key for a user and will
| initiate a reset process for any passwords and other
| personal secrets revealed to me; to do anything else
| would be irresponsible.
| nerdponx wrote:
| If nothing else, they need to be able to access your
| files in case you leave.
| lucb1e wrote:
| Unless it's all encrypted with the user's own password:
| sure, you can always invoke the admin powers and override
| the password field in the database or whatever. But the
| default mode should be that an account is only accessed
| by the person it belongs to.
|
| Even as a security firm we have some exceptions in our
| company (disk encryption password for a physical device
| in the office is in a shared vault, for example), or
| support might do a remote control session with the user
| present, but it's definitely not the default setup
| procedure for a new employee to have shared credentials
| for example.
| chousuke wrote:
| I think it should be noted that even with with encrypted
| data you'll generally just encrypt the master key with
| multiple passphrases; one (or more) master keys for admin
| use and one for the user themselves. There's really very
| rarely a _good_ reason to share any secrets.
|
| It's pretty much a necessity for any larger business that
| device data be protected with the user's own password
| _and_ with a master password known to IT so that it can
| be accessed when the user 's password is inevitably lost.
| qwertox wrote:
| This was very enlightening for me.
|
| But I wonder if it wouldn't be better if it were something like a
| ncurses list with checkboxable list items which one would
| progress trough.
|
| Take a checklist for a pilot for example. Wouldn't it be not as
| good if the next item on the list only appeared after an item has
| been checked? It could be beneficial if you could peek a couple
| of items down the line in order to group your actions or get a
| better overview of the overall task.
|
| "Do I put the water for the coffee in the microwave now or do I
| first finish this quick item and let the next long-running item
| start before I go and make my coffee?"
| patleeman wrote:
| One could pretty easily create a tool that takes Github Flavored
| Markdown and interactively displays the text and make checkbox
| items appear one at a time in a manner similar to this. Would be
| much more useful and easier to configure.
| Jtsummers wrote:
| How would it be _easier_ than the proposed Python example?
| Which only relies on bog standard Python (unless your steps are
| more complicated) and can (by virtue of encapsulating steps
| into a class) be wrapped up in nearly any user interface.
| patleeman wrote:
| Fair. In my mind, easier in the sense that past the initial
| script I'm just writing markdown for future checklists. It's
| also compatible with existing markdown documentation.
___________________________________________________________________
(page generated 2021-11-02 23:00 UTC)