[HN Gopher] The Soul of Maintaining a New Machine
       ___________________________________________________________________
        
       The Soul of Maintaining a New Machine
        
       Author : wyndham
       Score  : 258 points
       Date   : 2024-08-06 03:49 UTC (19 hours ago)
        
 (HTM) web link (books.worksinprogress.co)
 (TXT) w3m dump (books.worksinprogress.co)
        
       | ggm wrote:
       | I'd forgotten how close to printing machines the old photocopiers
       | were. You would basically either have a crap one you could
       | operate yourself, or take your stuff to the printery to have
       | professionals (a subset of librarians I think, or the logical
       | join over librarians and computer operations staff) do it for
       | you. Printing machines had a fleet of maintainers, craft unions
       | who walked off the job if you touched a dial.
       | 
       | They were amazing at doing things which really mattered:
       | shrinking an A0 architectural drawing down but maintaining aspect
       | ratio. Adjusting offsets for the print for binding signatures, so
       | the 1st and 16th page was not too far out because of wrapping
       | around the other 8 pairs of pages. Even just working out how to
       | rotate the pages for N-up printing. But the GUI sucked. I think
       | they called ours "the bindery" because it's main gig was doing
       | PHD from soup to nuts, binding included.
       | 
       | The repair techs had the most amazing flight cases, packed with
       | tools which served one specific purpose.Like, A doohickey to
       | adjust the corona wire, without dismantling the imaging and toner
       | roller, with a tonne of equipment hovering over your head on a
       | gas-lift. Screwdrivers with very very carefully chosen lengths.
       | Torque wrenches. It was high tech meets motor racing meets.. IBM.
       | 
       | I am told they were paid better than many computer techs. The IBM
       | guy was paid IBM scale to fix it on IBMs timescales. the xerox
       | guy did more random shit, with more devices, more often.
       | 
       | They had a very corporate look. that amazing briefcase or six.
       | Suit, tie. Very acceptable.
       | 
       | I know a guy who worked for a paper-folding-and-envelope-stuffing
       | company and it was very similar culturally: can-do, fix anything,
       | but working on giant multi-million dollar machines which were
       | used twice a year to do tax mailouts, and election materials, and
       | the rest of the time rented to the original spam merchants for
       | 10c per thousand mailouts. The secondhand value of these machines
       | were like photocopiers: Really significant. He was brought out of
       | retirement to help take one apart into TEU equivalent chunks to
       | be shipped to Singapore from Brisbane. His retirement gig at one
       | point was repairing Espresso machines, he said it made him feel
       | familiar and useful.
       | 
       | The era which was the end of the typing pool was fascinating. All
       | kinds of arcane roles which only make sense in the absence of
       | email and tiny printers everywhere. Some of those jobs had been
       | there from the days of hand-copying, Dickens-era and before.
        
         | Taniwha wrote:
         | Back in my mainframe days (late 70s) we ran a large mainframe,
         | the only one in the local uni - way slower than your phone, a
         | couple of Mb of core, ran payroll and 30 terminals. We had a
         | dedicated local engineer who had his own onsite office. In the
         | south of NZ we were a really long long way away from his home
         | office in the US, he was expected to be able to fix anything,
         | and mostly he could.
         | 
         | But one time the machine started writing crap on random things
         | - screens, printers, worst of all disks - could take a day or
         | to to recover after it scribbled across the equivalent of the
         | root file system (giant head-per-track coffee table sized
         | platters). The poor engineers couldn't figure it out, it
         | happened so in frequently eventually they flew a guy out from
         | head office in the US - he came with a wooden stick - he ran it
         | down a card cagore in the IO process, nothing happened, he
         | tried the next row bang! it crashed, after we were back up he
         | continued with his wooden stick doing a binary search for the
         | source, eventually he pulled a card and 3 little solder balls
         | fell into his hand, they'd been sitting there loose against IC
         | pins since it was installed
         | 
         | As you point out sometimes it just takes having the correct
         | tool and knowing how to use it
        
           | dmd wrote:
           | > way slower than your phone
           | 
           | For the young'uns... I think you're underselling this a bit.
           | A really powerful late 70s mainframe, generously would have
           | been somewhere around 1 to 2 million instructions per second.
           | 
           | Your phone is probably on the order of 5-10 trillion
           | instructions per second. That's a _million_ times faster.
        
             | brk wrote:
             | True, but 90% of the mainframe instructions weren't
             | tracking, telematics, data exfiltration, and user
             | profiling. So it kinda evens out.
        
             | The_Colonel wrote:
             | > Your phone is probably on the order of 5-10 trillion
             | instructions per second. That's a million times faster.
             | 
             | Do you mean from iGPU? The fastest smartphone iGPU I found
             | is MediaTek 9300 which has 2.4 TFLOPS which roughly
             | corresponds to your claim. Cray-1 (1975) had 160 MFLOPS
             | which would make today's high-end smartphone 15 000 times
             | faster than Cray-1.
        
               | Jtsummers wrote:
               | Cray-1 wasn't a mainframe, it was a supercomputer. As
               | such it was designed to push the performance envelope in
               | a way mainframes (of the time, but even today) weren't
               | even trying for. Mainframes were designed for transaction
               | processing and reliability. They were substantially
               | slower than contemporary supercomputers.
        
               | The_Colonel wrote:
               | That's true, but supercomputers are somewhat conceptually
               | similar to GPUs, so it seems like a better comparison to
               | illustrate technological progress. Comparing mainframes
               | and GPUs is comparing apples and oranges.
        
               | zifpanachr23 wrote:
               | This is a good analogy. If you look at a modern mainframe
               | CPU it becomes pretty clear where the differences lie.
               | Fewer, beefier cores with a lot of focus on cache.
               | 
               | Supercomputers tend to use more conventional cores, but
               | way more of them, and connect them in a large fabric.
               | There's a lot more focus on parallelization and
               | horizontal scaling.
               | 
               | Mainframe overall compute is nowhere near a
               | supercomputer, and you probably shouldn't be running a
               | massive physics simulation on a mainframe, but you may
               | get more consistency and reliability for well defined
               | tasks.
        
             | kragen wrote:
             | your phone is 5-10 billion instructions per second, a
             | thousand times faster. it's probably a quad-core in-order
             | arm running at 1-2 gigahertz, not 1-2 terahertz
        
               | dmd wrote:
               | Instructions and hertz have not been equivalent for
               | almost a quarter of a century now, though.
        
               | kragen wrote:
               | not equivalent, but close enough in this case; even my
               | laptop only manages less than 2 ipc per core, and the
               | error in your earlier comment is a factor of 1000
        
         | bruce511 wrote:
         | We still have an old printing machine. It came as part of an
         | acquisition and took a truck to move. We rent a small unit in
         | an industrial complex to house it.
         | 
         | Nobody knows how old it is. It predated the folk who came with
         | it (all of whom were close to retirement.)
         | 
         | It's still running (looked after by one of the retirees on a
         | part time basis.) He does print runs a couple times a week. To
         | seem him with an oil can in his hands is to step back in time.
         | 
         | It's more or less neutral profit-wise, but pays his wage and
         | keeps some old customers happy. When he gives up the machine
         | goes too (likely for scrap I guess).
        
           | actionfromafar wrote:
           | Beautiful little story. I love when such things are allowed
           | to exist.
        
       | Neil44 wrote:
       | Reference - a great read -
       | https://en.wikipedia.org/wiki/The_Soul_of_a_New_Machine
        
         | begueradj wrote:
         | Indeed: (https://news.ycombinator.com/item?id=19133653) and
         | (https://auxiliarymemory.com/2017/01/06/rereading-the-soul-
         | of...) ... and that's not the same book :)
        
         | nonrandomstring wrote:
         | Good share. Haven't read but looking at the wiki entry it's an
         | interesting book, Technology requires a culture to sustain it.
         | 
         | To use a fancy word there's a "noosphere" around that carries
         | knowledge and lore along with it.
         | 
         | Plus the usual kind of Deming wisdom about real knowledge
         | organisation:                   "in many ways opposite of
         | traditional management...innovations are         started at the
         | grassroots level.", and "people will give their best
         | when the work itself is challenging and rewarding."
         | 
         | That atmosphere is still around in engineering teams in smaller
         | companies where there's a good range of advancement within
         | engineering and some stable products so there's a stream of
         | old-timers and apprentices building and exchanging chops.
         | 
         | [0] https://en.wikipedia.org/wiki/Noosphere
         | 
         | [1] https://en.wikipedia.org/wiki/W._Edwards_Deming
        
         | EncomLab wrote:
         | One of my top 5 favorite books of all time - the audiobook
         | version on audible is excellent as well. A lost world - and we
         | are all worse off for it.
        
         | avisser wrote:
         | "a great read" - Almost underselling it. It won the Pulitzer
         | for non-fiction that year.
        
         | f0e4c2f7 wrote:
         | This book is so good. It's about building early computers but
         | it feels just like tech does today in the way it's described.
         | Which make it feel like this bigger context you're reading
         | into. That even before computers were mainstream, there were
         | still those who tinkered.
         | 
         | Two things are especially memorable to me. One is a casual
         | remark in the book that they found the best way to get things
         | done is to pair someone very experienced and cynical with
         | someone very inexperienced and naive. Combined they would get
         | lots done together compared to either alone. I think this is
         | still true today.
         | 
         | The other thing is the intro. It's about the head of the
         | project getting a group together and renting a sailboat on
         | vacation. On the sailboat the get tossed and at times feel like
         | they barely survived and it ends with someone saying "if this
         | was his vacation...what did this man do for fun!?"
        
       | katzenversteher wrote:
       | I while ago (around 2010) I worked at Oce Technologies R&D and at
       | least to me the machines while incredibly complex where quite
       | easy to operate and maintain. In fact every developer was allowed
       | to print their private stuff (non commercial) on the machines
       | under development. I believe this really helped because we
       | basically had to become operators and maintainers and got a feel
       | of their roles. If you print something for yourself, family or
       | friends you also take a better look at the output or sometimes
       | have a very specific use case in mind.
        
       | goffley3 wrote:
       | This goes a long way in explaining the incredibly warm and
       | sociable, yet undeniably peculiar Xerox tech that I would
       | interact with on a regular basis. Despite him not actually
       | working for the company, I would have to call him in constantly
       | to get him to fix one or more of the machines we had in the
       | office.
        
       | neilv wrote:
       | The latter half has a lot of criticism of allegedly boneheaded
       | management (from the perspective of the writer and researchers
       | they quote).
        
         | rini17 wrote:
         | Strictly siloing maintenance away from sales and development is
         | boneheaded clearly, not allegedly.
        
       | EdwardCoffin wrote:
       | This passage particularly struck me: _He noticed when a
       | technician on a call began by examining copies that had been
       | thrown in the trash and deduced from them that the problem with
       | the machine was different from what the customer had reported.
       | "The trashcan is a filter between good copies and bad," one
       | technician explained "Just go to the trashcan to find the bad
       | copies and then... interpret what connects them all."_
       | 
       | On a related note, I'd like to _highly_ recommend Lucy Suchman 's
       | work, mentioned in this article as _Plans and Situated Actions:
       | The Problem of Human-Machine Communication_ , but the updated
       | version now called _Human-Machine Reconfigurations: Plans and
       | Situated Actions_. The new version has several extra chapters and
       | some other revisions. I 've read it several times, and had my
       | mind blown each time.
        
         | indrora wrote:
         | For a long while, I've been helping people diagnose problems
         | with 3D printers and one of the things that stands out to me is
         | how often people throw away bad prints while they're
         | troubleshooting.
         | 
         | One of the first things I ask someone now is to give me their
         | test prints for the last few changes they've made and what they
         | changed, if they remember. Nine times out of ten, just looking
         | at other factors, I've been able to pinpoint what's wrong.
         | 
         | Interestingly, this has also led me to a few "is it turned on"
         | questions, the first of which is "when is the last time you
         | tightened all the bolts that should be tight and loosened the
         | bolts that should be loose?" The simple act of making sure
         | tight things are square and tight has resolved many an
         | unrelated problem because it makes you check so many different
         | parts of the machine.
         | 
         | This also correlates with one of the takeaways from the old
         | Computerworld column called Shark Tank. The most common aspect
         | of the solution was to simply go and observe the failure,
         | because so often it was an unaccounted for externality -- such
         | as a cleaning crew unplugging something at 2AM when a system
         | would "mysteriously" go down.
        
       | gumby wrote:
       | This was such important and transformational work and I remember
       | at the time being quite dismissive of it.
       | 
       | I knew Orr's and Suchman's work (they worked in a physically
       | adjacent area, but completely different group, though we were all
       | under John Seely Brown and because they were nice people).
       | Thankfully I was grown up enough to be polite, but really I was
       | such a techno-determinist that I figured user problems came from
       | ignorance.*
       | 
       | To be fair, I was not the only one: the insights described in
       | this book draft surprised a lot of people, not just how they
       | improved the copiers but how those two even approached the
       | problem (starting with the sociology of the repair workers). It
       | sure surprised Xerox management. But I've heard it said many
       | times that this work led to restructuring the paper path in a way
       | that justified (paid for) everything spent on PARC.
       | 
       | I did grow up of course and now do see my work (machines,
       | chemistry, etc) as a small part of a large social system. A
       | successful company has to base its product plans starting this
       | way.
       | 
       | To choose an example of failure to appreciate the social scope
       | (but not pick on it) the crypto folks spend their time on
       | technology, based on a social model they _want to exist_ rather
       | than the one that currently does. I think it's a big reason why
       | it's barely impacted the world in, what, 15 years? Xerox was the
       | same, and it helped them sell a lot of copiers, but didn't make
       | them as ubiquitous as they could have been. Another example:
       | everybody laughs at Google for launching "products" that go
       | nowhere and are quickly forgotten. We all know it's because of a
       | screwed-up, internally-focused culture. But sometimes a product
       | succeeds without marketing (e.g. gmail, at the time) because it
       | happened to be matched to the actual, external need. It makes
       | this kind of continuous failure even more damning.
       | 
       | * TBH, 40 years later I have not 100% shed this view -- e.g. my
       | attitude towards complaints about git. Maybe this means I'm still
       | a jerk.
        
         | com wrote:
         | Thank you for this very honest comment. It means more to to me
         | than I was expecting it to when I started reading it.
         | 
         | Were there any moments on the journey of growing up that stick
         | in your mind as being turning points in your path from techno-
         | determinism to whatever you describe yourself now?
         | 
         | I'm really interested in how we can intervene earlier in
         | people's journey to provide bigger horizons, rather than just
         | waiting for enough experience to build up...
        
           | gumby wrote:
           | Thanks. I think I mainly developed more empathy.
           | 
           | I married an artist. She knew no physics, no advanced
           | mathematics, but plenty of philosphy and languages as well as
           | the technology and philosophy of representational art. She
           | was a lot smarter than me and we spent a lot of time
           | expanding each others' horizons, not deliberately or
           | didactically but just through living.
           | 
           | Also so many things seem clear when you are young and I was
           | repeatedly humbled by how complicated the real world is
           | compared to academia or research (where I worked on _very_
           | abstruse subjects, e.g. the denotational semantics of
           | reflexive languages).
           | 
           | I think you see this in a lot of people: libertarianism and
           | even Ayn Rand is more popular with young people ("hey I'm
           | smart and responsible; why are there all these annoying and
           | pointless rules getting in my way?"). I was never a
           | libertarian but back then I was sympathetic to Rousseau. But
           | you have fewer grown up libertarians because many learn to
           | realise that sure, things are imperfect, but we live in a
           | world that evolved (sometimes well and sometimes not) to work
           | with fallable systems and lots of opinions, some well thought
           | out and some...not so.
        
         | kragen wrote:
         | > _the crypto folks spend their time on technology, based on a
         | social model they want to exist rather than the one that
         | currently does. I think it's a big reason why it's barely
         | impacted the world in, what, 15 years?_
         | 
         | assuming you mean cryptocurrencies, i think they kept several
         | million people alive throughout the venezuelan civil war,
         | sustained wikileaks through the visa and mastercard blockade,
         | and have made sci-hub and library genesis financially stable;
         | maybe they haven't impacted the people in your direct vicinity,
         | but they've made a huge impact in places like moldova which
         | don't have functional currency
        
           | gumby wrote:
           | Yes, cryptocurrencies not cryptography. Bad of me to use that
           | contraction for anything but the latter.
           | 
           | Have they really made a significant difference? A bit of
           | searching around resulting in a few articles with anecdotes,
           | and if someone's life really was saved this way of course it
           | made a huge difference to them.
           | 
           | But for some macro perspective the best I could find was the
           | "Crypto Adoption Index"
           | (https://www.chainalysis.com/blog/2023-global-crypto-
           | adoption...), where VE, or MD don't even register (AR barely
           | does). In fact US is listed as having one of the highest
           | levels of adoption, and it's basically invisible here.
           | 
           | Countries with nonfunctional currency systems usually do
           | better by adopting an existing stable currency like USD or EU
           | (or even better someplace economically coupled with them).
           | 
           | But I'm no expert and I'd be happy to be proven wrong. On a
           | financial macro perspective cryptocurrencies don't even
           | exist.
        
             | kragen wrote:
             | pretty much the entire currency black market here in
             | argentina used to run on cryptocurrencies, which is more
             | significant than it sounds given how large a fraction of
             | argentina's employment is illegal (something like a third,
             | last i heard). i think us banks have taken some of that
             | business away from cryptocurrencies in recent years, but
             | every illegal moneychanger still accepts them even if most
             | of their business runs over zelle
             | 
             | as for how countries might do better, while it is
             | interesting to discuss what policies policymakers ought to
             | adopt, argentine (and turkish, venezuelan, etc.)
             | policymakers do not want to adopt the policies they ought
             | to adopt; they want to adopt the policies that serve their
             | political interests. the rest of us are left to figure out
             | how to cope with their terrible policies. and that's the
             | sense in which bitcoin matters; it makes it possible for
             | argentines and venezuelans to do things like save money,
             | leave the country, and send money to their families back
             | home when they're working abroad
             | 
             | on a financial macro perspective, there's currently about
             | 1.1 trillion dollars stored in bitcoin, which is a few days
             | of global gdp. other cryptocurrencies together are
             | something like half a trillion. it's entirely plausible
             | that 80%, 90%, 95%, or 99% of bitcoin's number is owned by
             | keys that were lost in disk drive failures back in the cpu
             | mining epoch, but that's less likely for the other
             | cryptocurrencies. so i don't know that i'd agree that they
             | don't exist on a financial macro perspective. certainly
             | they aren't anywhere near the importance of asset classes
             | like real estate, commercial paper, forex, commodities
             | futures, or stocks, and it's questionable how much real
             | liquidity exists in the market--if satoshi were to start
             | selling off his coins now, how much would it tank the
             | market before he was done?
        
       | stonethrowaway wrote:
       | > Though they were doing messy blue-collar work, Xerox required
       | the technicians to act and dress white-collar. They carried their
       | tools in a briefcase.
       | 
       | We don't carry tools in briefcases because it makes us appear
       | white collar, but because the shell is hard and protective, there
       | are many sizes, and the boxy interior can be formed to however
       | you like if you use foam and cut it to fit your tools. Briefcases
       | fit readily into many tight spots for transportation. The photo
       | shows the usual layout of tools that techs use. Companies sell
       | high end equipment in briefcase-like containers because it keeps
       | them safe and waterproof in needed situations.
       | 
       | Not a big fan of the anthropology aspect. It's a job. Techies
       | improvise, it's not a clandestine operation to fix a machine.
        
       | vajrabum wrote:
       | I spent nearly 20 years fixing computers and other electronics.
       | Repair techs are my original work tribe and it was a fun if
       | sometimes stressful way to make a living. I got away from it
       | because the money went away. That said, I never wanted to fix
       | copiers. They were always finicky, messy and dirty, but this is a
       | really great piece and three things stand out for me.
       | 
       | The article claims that PARC paid for itself (1) through the
       | anthropological sociological studies of copier repair technicians
       | which revealed shortcomings in the engineering of the copiers and
       | resulted in changes to the paper path and handling in newer
       | designs and significantly reduced maintenance cost and
       | difficulty. Two, enabling information sharing between repair
       | technicians over radios and technician created and maintained
       | documentation, saved the company 5-8% of service cost and these
       | innovations were resisted by services management which was
       | invested in the idea that copier repair technicians should be
       | cheap, interchangable monkeys. Three, Xerox management likely
       | left significant money on the table because they fundamentally
       | and willfully misunderstood copier repair and copier repair
       | technicians and the value they were creating for the company.
       | Likely, mostly because repair was seen as a cost center which in
       | an ideal world would be eliminated entirely.
       | 
       | 1. It's really astonishing how much and in how many ways PARC
       | paid for itself and yet business literature and likely Xeroxes
       | management often focuses on the money left on the table for
       | others to grab and asserts there was a failure.
        
       ___________________________________________________________________
       (page generated 2024-08-06 23:01 UTC)