[HN Gopher] Failure analysis of the Arecibo 305 meter telescope ...
___________________________________________________________________
Failure analysis of the Arecibo 305 meter telescope collapse
Author : mhb
Score : 173 points
Date : 2024-11-05 13:37 UTC (9 hours ago)
(HTM) web link (nap.nationalacademies.org)
(TXT) w3m dump (nap.nationalacademies.org)
| magicalhippo wrote:
| In the summary they state:
|
| _All the reported experimental zinc electroplasticity (EP) data
| were developed at current densities orders of magnitude higher
| than those possibly present in the Arecibo Telescope but measured
| in laboratory experimental periods that were orders of magnitude
| shorter than the telescope 's socket zinc service._
|
| _There are no reported experimental data concerning low-current,
| long-term EP, which the committee has lumped together under the
| term "LEP", affecting zinc's creep mechanisms over decades._
|
| _The timing and patterns of the Arecibo Telescope 's socket
| failures make the LEP hypothesis the only one that the committee
| could find that could potentially explain the failure patterns
| observed._
|
| Accelerated aging is, as far as I know, pretty much standard in
| the industry. Nobody can wait 20 years to find out if a certain
| material is good enough or not.
|
| However, the real failure seems to be the lack of urgency when
| they signs started to show up:
|
| _Upon reflection, the unusually large and progressive cable
| pullouts of key structural cables that could be seen during
| visual inspection several months and years before the M4N failure
| should have raised the highest alarm level, requiring urgent
| action. The lack of documented concern from the contracted
| engineers about the inconsequentiality of the cable pullouts or
| the safety factors between Hurricane Maria in 2017 and the
| failure is alarming._
| nick3443 wrote:
| Sounds like it was a blind spot for the engineers due to the
| normally bulletproof service history of zinc spelter cable
| terminations in structures.
| potato3732842 wrote:
| >normally bulletproof service history of zinc spelter cable
| terminations in structures.
|
| They're relatively new, they're relatively expensive (which
| saves them from being used in the worst of the applications)
| and like most things inherent to steel cable and whatnot they
| tend to get employed in situations where safety factors are
| generous. Don't count your chickens before they hatch.
| nick3443 wrote:
| I am not an expert, I read this in the report: The long-
| term zinc creep failure of the Arecibo Telescope sockets
| and the subsequent cable pullout has never been documented
| elsewhere despite a century of zinc-filled cable spelter
| sockets use. 17 The type, size, length, and fittings of the
| cables used in the Arecibo Telescope (whether the original
| cables constructed in the 1960s or the auxiliary cables
| installed in the 1990s) were catalog-selected items, not at
| all unusual, with decades of proven performance. Exten-sive
| structural modeling of the Arecibo Telescope, independently
| confirmed by laser cable sag surveys, validated that under
| all static and cyclic loading conditions, the cable loads
| barely exceed half the nominal cable strength. While PLC
| can occur at stresses well below yield, such a creep
| failure has never been reported in spelter socket zinc.
| hinkley wrote:
| I think we might need to send some microbiologists to do
| their own forensics.
|
| You have a mechanical failure that can't be seen in the
| labs and isn't reproducible at other sites. Suggests
| something environmental.
|
| Like when natural fiber army gear starts dissolving in
| the jungles of Vietnam.
|
| Could be direct action, or a side effect of a chemical in
| plant or animal detritus. Like pigeon poop on car paint.
| madaxe_again wrote:
| The paper posits that it's due to low frequency
| electromagnetic induction in the tethers from the
| observatory operations, which is a pretty unique
| environmental factor.
| londons_explore wrote:
| > Exten-sive structural modeling of the Arecibo
| Telescope, independently confirmed by laser cable sag
| surveys, validated that under all static and cyclic
| loading conditions, the cable loads barely exceed half
| the nominal cable strength.
|
| I have a feeling such measurement and modelling might
| have been wrong.
|
| It's easy to miss some oscillation mode, particularly
| high frequency longitudinal wave's in the cables, which
| couldn't be picked up by a laser survey due to the
| kilometers per second these waves travel.
| deskr wrote:
| > The lack of documented concern from the contracted engineers
| about the inconsequentiality of the cable pullouts or the
| safety factors between Hurricane Maria in 2017 and the failure
| is alarming.
|
| This is crazy. Basically cables were pulling out for months and
| years and no one raised the alarm? In many industries that
| could be a criminal or career ending malpractice. Are the
| "contracted engineers" liable?
| magicalhippo wrote:
| In chapter 5 they go into how the ownership was transferred
| to the University of Central Florida (UCF) at the start of
| 2018, after the hurricanes in 2017.
|
| _It seems unlikely that UCF had adequate time and resources
| to review and understand the Arecibo Telescope 's original
| 1963 design, the 1974 upgrade, the structural inspection and
| maintenance records produced for nearly 50 years, [...] and
| the key factors, such as the wire breaks and cable pullout of
| the sockets and their significance on the strength and
| integrity of the structure._
|
| _The measured cable pullout may have appeared "normal" to
| [the UCF staff] and was not on their radar as signs of
| structural distress. The lack of concern may be because a
| small cable pullout was present from the beginning, and no
| one in authority had previously raised an alarm_
|
| But yeah, seems weird no-one of the contractors tried to
| raise an alarm.
| hinkley wrote:
| I bet there's a r/MaliciousCompliance in Spanish out there
| somewhere about how UCF cancelled existing maintenance
| contracts and hired new companies to keep an eye on the
| equipment, and did it so rudely that the old company didn't
| hand over important warnings like this.
|
| Discontinuity could explain problems like this readily.
| renewiltord wrote:
| These are Professional Engineers, not your run of the mill
| fake 'engineer' in software development. They have licenses
| and certifications. They don't just ship bugs. When they say
| that something is fine, it is fine. They have certifications.
| They have licenses. They are professionals. If something
| failed, it must have been management. Professional Engineers
| do not make mistakes. They are real engineers.
| haccount wrote:
| Anyone in the field will likely say that they wouldn't
| trust half of their colleagues or themselves.
|
| It's pretty much how professional prestigious positions
| work. Beneath the gilding there's duct tape repaired with
| super glue.
| renewiltord wrote:
| That's how software works. But professional engineers are
| professionals. They have certifications. They have an
| iron ring. It's real.
| feoren wrote:
| I'd like to live in the world you imagine we live in.
| LaffertyDev wrote:
| > Professional Engineers do not make mistakes. They are
| real engineers.
|
| My _literal_ engineering 101 case studies on "engineering
| failure":
|
| https://en.wikipedia.org/wiki/Tacoma_Narrows_Bridge_(1940)
|
| https://en.wikipedia.org/wiki/Hyatt_Regency_walkway_collaps
| e
|
| https://en.wikipedia.org/wiki/I-35W_Mississippi_River_bridg
| e
|
| Are all failures traced down to the underlying design and
| engineering. Granted, they are famous because they are
| rare. I almost linked the Columbia disaster but that is
| sufficiently murky between engineer and management.
|
| Stop being elitist.
| mNovak wrote:
| I wouldn't overinvest your confidence in these licenses and
| certifications. A "Professional Engineer" is just a
| Bachelors level engineer who took one extra test. It's
| extremely common for MechE and CivE. As far as
| certifications go it's not quite the same as, say, passing
| the bar exam or finishing med school.
| dylan604 wrote:
| But it is different than that cert you receive after
| taking a 2 week bootcamp on Python.
| albrewer wrote:
| > A "Professional Engineer" is just a Bachelors level
| engineer who took one extra test
|
| At least for mechanical, it's 4 years' working
| experience, documentation of sufficiently advanced
| projects, and sign-off from 3 other registered
| professional engineers who are familiar with your work.
|
| A guy I worked with took the mechanical design variant. I
| don't know how the HVAC variant is. The test is over the
| most advanced stuff you learn in your degree; the
| "specialist" portion of the test (what your concentration
| is in, e.g., HVAC, thermal/fluids, or machine design and
| materials) is ~25 questions and you have 4 hours to
| complete it, in addition to the other 4 hour
| comprehensive portion that's the same for all takers.
|
| On top of that, to keep your license, you need to
| complete some amount of classroom education every year.
|
| As far as coursework goes, the coursework for some of my
| upper-level mech E classes was usually identical to a
| graduate-level course, the only difference being graduate
| students do a research paper+experiment for a final exam
| and undergrads do a traditional written exam since they
| had access to funding and we did not.
|
| imo this comment is dismissive and glosses way too
| heavily over the rigor that goes into even the "lowliest"
| of engineering programs.
| wpollock wrote:
| This is how PEs should be certified. Sadly, licensure of
| PEs varies by state.
|
| In Florida, the title of "software engineer" requires a
| license. To obtain the license, you only need to pass the
| national engineering (I forgot the exact name of the
| organization) exam for software engineering. The national
| exam was withdrawn some years ago. I wrote the Florida
| Engineering Board about it, suggesting that they needed
| to update the requirements.
|
| Their response? Florida doesn't distinguish between PE
| specialties; prospective software engineers should just
| get a PE license in another field. They suggested
| electrical engineering.
|
| So in Florida, the PE signing off on your buildings's
| wiring might be a chemist, or the PE supervising a bridge
| construction might be only trained in electrical
| engineering.
| mden wrote:
| I think people are missing that you're being satirical
| (maybe there is a better word for it?). Unless I'm the one
| being wooshed.
| rootusrootus wrote:
| I agree. Maybe on the internet we really do require /s
| just to make sure everyone understands, but it looks to
| me very much sarcastic.
| PhilipRoman wrote:
| It's 100% sarcasm. The capitalization should have made it
| obvious to anyone. The only place I've seen struggling
| with recognizing sarcasm is hackernews.
| rootusrootus wrote:
| Software developers are very sensitive to being attacked
| as 'not real engineers' and it turns off the humor
| detector.
| feoren wrote:
| As someone who works in a company with 10,000+ professional
| engineers*: yes, yes they do make mistakes. Despite what
| you seem to think, they are actually human beings. Good
| processes understand this and have multiple layers to catch
| and correct mistakes. But it's hard to fight limited data.
| Plenty of engineering decisions are still made based on
| numbers in photocopied tables from two or three studies
| from 50 years ago, each having a dozen or two data points.
| It's surprising how limited some of the data is that these
| Professional Engineers are basing their decisions on.
|
| *Edit: okay, I don't know exactly how many are officially
| PEs vs. junior engineers or something, but at least that
| many are "PE-track".
| tiahura wrote:
| I'm not sure by your tone if this is sarcasm, but it has
| been shown by formal proof that if something failed the
| root cause must be an MBA.
| LorenPechtel wrote:
| But that would have required spending money. And why would
| the engineers be liable? They weren't being paid to fix it.
| kreims wrote:
| You go to school and learn that 2+2=4. You get a consulting
| job and learn 2+2= whatever the client says it is.
|
| I wouldn't jump to the conclusion that the consulting
| engineer was incompetent. Sometimes a bureaucrat tells you to
| sharpen your pencils and come back with the answer That fits
| the budget they have if you want to keep your professional
| services contract going.
| londons_explore wrote:
| Since the design had a factor of safety of 2, and no other
| cables exhibited pull out at such high safety factors,
| checking such things might not even have been on the
| checklist.
| bsder wrote:
| > Basically cables were pulling out for months and years and
| no one raised the alarm?
|
| And what if they _did_? Then what? My guess is the
| conversation went something like this:
|
| "Okay, cables are pulling out. Raising this issue will not
| magically make money appear to fix it as nobody wants to fund
| this. Tell me, how much do you like your job? If you flag
| this, you may lose your job directly and if the project gets
| shut down you may lose your job indirectly. So, how about we
| bury this as much as possible, cross our fingers and bank as
| much money as possible in the meantime, eh?"
| mkesper wrote:
| And further:
|
| The reliance by the consultants (before and after the first
| cable failure in 2020) on a perceived allowable pullout of one-
| sixth of the cable diameter, which should only be seen at
| loading at 80 percent of ultimate cable strength, does not
| align with the AASHTO M 277 standard guidance. The committee,
| therefore, disagrees with the suggestion made in the Thornton
| Tomasetti, Inc. (TT) 2022 report, Arecibo Telescope Collapse:
| Forensic Investigation,13 to use the D/6 limit as a threshold
| for slip monitoring.
| Iolaum wrote:
| I would be curious if there are any accounts/reports of people
| working there - especially engineers - that could verify the
| claims of no alarm being raised when the first structural
| failures started showing up. It is often the case that people
| on the ground report such things but when "reports" are written
| for the higher ups, facts get "massaged" according to
| expectations.
| 77pt77 wrote:
| Bearers of bad news get shot.
|
| Everyone working in large corporations knows this.
| lazystar wrote:
| being a bearer of bad news is a sign that you are not
| disagreeing and committing.
| thimkerbell wrote:
| Is this flaw something an LLM could have caught? (Or do I have
| an inflated understanding of their abilities?)
| magicalhippo wrote:
| I think absolutely not, for several reasons. First off, the
| report mentions this failure of the zinc filled sockets never
| been reported elsewhere before.
|
| Secondly, even if it had, the report mentions that written
| documentation of inspections and so on was spotty and
| inconsistent.
|
| That said, if the failure mode had been published before, and
| inspections and maintenance was written down in a consistent
| and detailed manner, then _perhap_ s an LLM-based tool could
| have linked the two. I'm thinking of a RAG-like search thing
| that compared inspection reports with published documents.
| Alternatively just assisting a human in finding relevant
| published reports.
| UniverseHacker wrote:
| As a sailor that uses similar fittings in sailboat rigging...
| the idea of not treating a cable pullout from a terminal as an
| emergency is absurd. If that happened on a sailboat rig I'd
| assume it had a remaining strength of zero and will fall in
| seconds or less- and take the most rapid possible action to
| keep the structure from coming down by deloading the shroud and
| rigging an emergency replacement.
| nonfamous wrote:
| From the summary:
|
| >>> The only hypothesis the committee could develop that provides
| a plausible but unprovable answer to all these questions and the
| observed socket failure pattern is that the socket zinc creep was
| unexpectedly accelerated in the Arecibo Telescope's uniquely
| powerful electromagnetic radiation environment. The Arecibo
| Telescope cables were suspended across the beam of "the most
| powerful radio transmitter on Earth." The other investigations
| failed to note several failure patterns and provided no plausible
| explanation for most of them. To answer these questions with
| empirical evidence instead of only the inferences that can be
| made from the existing data, a more comprehensive and widespread
| forensic analysis of "good" and "bad" socket workmanship and the
| low-current, long-term effect on zinc creep is required.
| shagie wrote:
| (There was a comment that I was replying to about the
| realization that this was a transmitter and not just a receiver
| that was deleted ...)
|
| It was both.
|
| The Arecibo message was transmitted from there -
| https://en.wikipedia.org/wiki/Arecibo_message
|
| > The entire message consisted of 1,679 binary digits,
| approximately 210 bytes, transmitted at a frequency of 2,380
| MHz and modulated by shifting the frequency by 10 Hz, with a
| power of 450 kW.
|
| There's also the field of radar astronomy (not radio astronomy)
| https://en.wikipedia.org/wiki/Radar_astronomy where you bounce
| of a signal off of an object in the solar system (such as an
| asteroid)
|
| > Radar provides the ability to study the shape, size and spin
| state of asteroids and comets from the ground. Radar imaging
| has produced images with up to 7.5-meter resolution. With
| sufficient data, the size, shape, spin and radar albedo of the
| target asteroids can be extracted.
|
| > ...
|
| > In August 2020 the Arecibo Observatory (Arecibo Planetary
| Radar) suffered a structural cable failure, leading to the
| collapse of the main telescope in December of that year.
|
| > There is one remaining radar astronomy facility in regular
| use, the Goldstone Solar System Radar.
| Syonyk wrote:
| Fortunately, more novel research comes out of "Huh, that's
| weird... why did _that_ happen? " than out of things behaving
| as expected. Though this one does seem rather baffling.
| ricksunny wrote:
| I'm interested in hearimg the most speculative hypotheses as to
| why Arecibo Telescope's socket zinc creep proceeded so quickly
| compared to expectation.
| userbinator wrote:
| DC can definitely cause a lot of electrolytic damage, but AC
| usually doesn't. Perhaps something was acting as a diode (these
| are easily formed from natural processes) and rectifying the
| signal.
| whoopdedo wrote:
| The old "Foxhole radio" trick of using a rusty razor blade as
| a diode comes to mind. Some metal oxide coatings are
| semiconducting and under the right conditions will form a
| Schottky barrier.
| Wololooo wrote:
| Even if I am not a radio astronomer, this instrument was unique
| in nature and still in use for very specific observations and
| until the very end of its operation was able to deliver data. It
| is disheartening to learn that it was indeed due to gross
| maintenance negligence (as assumed originally by many) that the
| final fatal failure occurred, when potentially the structure
| could have been salvaged.
| SirMaster wrote:
| What was unique about it compared to this one for example?
|
| https://en.wikipedia.org/wiki/Five-hundred-meter_Aperture_Sp...
| superkuh wrote:
| Arecibo did high power transmit for radar imaging of
| asteroids/comets/the sun/etc and ionosphere studies, among
| other things. FAST is not capable of transmit (it was not
| designed for it and the suspended receiver mass is much
| lower). Also, FAST has a RF-noise emitting theme park built
| just outside of it's grounds.
|
| The uniqueness of Arecibo was this transmit coupled with such
| a large aperture (so small pattern on sky).
| SirMaster wrote:
| Thanks! To a layman it seemed like they were really similar
| devices.
| dylan604 wrote:
| > Arecibo did high power transmit for radar imaging of
|
| the Sun? that's something I never thought about, and now
| that I'm thinking about it, still seems very interesting.
| It seems like there would be so much noise radiating from
| the burning ball of fusion that radar signals would just
| get absorbed/lost. And now I've sat here for 5 minutes
| doing nothing but imagining this.
| axus wrote:
| My uninformed recollection from when the warning signs were
| raised, is there was not even budget to perform emergency
| maintenance, and they were resigned to the inevitable collapse.
| jdhwosnhw wrote:
| I worked and did research atobservatory for my PhD - you are
| exactly correct. It was well known to the scientists and
| onsite engineers that a collapse was imminent, for years.
| Ironically, prior to the collapse the agency responsible for
| funding performed a study to see would need to do to stop
| supplying money to the observatory, and found that due to
| environmental impact, it would actually be more expensive to
| stop funding. After that study we all suspected that the
| agency was just waiting for decay or a natural disaster to do
| the dirty work for them
| trogdor wrote:
| Would you link the study you are referring to?
| WillPostForFood wrote:
| That's not what the report says. There was ~$14 million
| allocated for repairs in 2-18 after Hurricane Maria, but the
| cable sockets weren't identified as needing repairs until the
| first one failed. The problem wasn't the money, it was the
| failure to identify the problem. See page 28/29.
| magicalhippo wrote:
| Losing Arecibo was, as I understand it, a big blow to the
| NANOGrav experiment, which is looking for _very_ low frequency
| gravitational waves by measuring pulsar timing variations[1].
|
| Albeit having limited field of view the Arecibo Telescope was
| very sensitive[2], and so could see pulsars that the other
| telescopes they used could not. And the longer they could
| collect data from a set of pulsars the lower frequency waves
| they could probe.
|
| [1]: https://nanograv.org/news/15yrRelease
|
| [2]: https://pirsa.org/20100068 _Moving Closer to a Detection
| of nHz-frequency Gravitational Waves with NANOGrav_ (Arecibo
| details at around 10:20)
| alexnewman wrote:
| Puerto Rican here. Everything on our island is like that.
| Preventive maintence is ignored and our insurance companies go
| bankrupt every big hurricane
| wileydragonfly wrote:
| Everything in the Caribbean is like that if you want to be
| honest.
| BobaFloutist wrote:
| If we ever make you a state proper hopefully that will offer an
| opportunity for that to change.
| dylan604 wrote:
| Because insurance payouts in Florida are so spectacular?
| Please, don't lead them along with false hope! /s
| ordu wrote:
| Aren't you afraid that US insurance companies will go
| bankrupt?
| ljsprague wrote:
| Statehood? Why not independence?
| leptons wrote:
| How would independence be better?
| BobaFloutist wrote:
| Because they voted for statehood in a referendum.
| bell-cot wrote:
| The NSF's 2022 report on the collapse:
| https://www.nsf.gov/news/special_reports/arecibo/Arecibo-Tel...
| pantulis wrote:
| What an imposing document: the details are overwhelmingly
| detailed, it really does justice to the marvel of engineering
| that the Telescope was.
| sideshowb wrote:
| Analysis of recorded audio contains the phrase "for England,
| James?"
| fijiaarone wrote:
| Finally a cogent post. Kids these days.
| ctippett wrote:
| Practical Engineering on YouTube has an episode covering the
| collapse of this structure[1]. It's very good and worth a watch.
|
| [1] https://youtu.be/3oBCtTv6yOw
___________________________________________________________________
(page generated 2024-11-05 23:00 UTC)