[HN Gopher] Widespread power outage in Spain and Portugal
___________________________________________________________________
Widespread power outage in Spain and Portugal
All of Spain is without energy. All systems have shut down
immediately and are not coming back. Apparently the same has
happened in Portugal.
Author : lleims
Score : 1122 points
Date : 2025-04-28 10:50 UTC (12 hours ago)
(HTM) web link (www.bbc.com)
(TXT) w3m dump (www.bbc.com)
| io84 wrote:
| Source?
| mirekrusin wrote:
| I can confirm, all coworkers from Portugal seem to be affected.
| Big thing.
| MrPapz wrote:
| Yes, it's happening
| T3x wrote:
| I can confirm (I'm from Portugal)
| xavaki wrote:
| barcelona has been out for the past 20 minutes or so
| pablo-120971 wrote:
| Got reports from Toledo and Madrid
| rvz wrote:
| A full nationwide power outage affecting not one but three [0]
| countries.
|
| Sounds like a major infrastructure risk given that it is possible
| for more than one country to experience a full loss of power.
|
| EDIT: Andorra is also affected, so that is three.
|
| [0] https://www.lavanguardia.com/vida/20250428/10624908/caida-
| ge...
| ifwinterco wrote:
| The whole European grid is essentially connected at this point,
| it's a simple trade of resiliency for efficiency. People just
| forget about the resiliency part until something goes wrong
| chippiewill wrote:
| The European grid is a connected synchronous grid. Usually that
| would add stability but it also means if a single country's
| grid blacks out then their neighbours have to respond to that.
| Portugal and Spain's grids will be intimately connected which
| won't have helped.
|
| Not something that's easy to test for.
| derbOac wrote:
| Yeah I've been wondering why this seems so centralized. I'd
| think power would be more decentralized.
| celso wrote:
| Portugal has no electricity as we speak. Funny enough telcos and
| 4G/5G are fine for now, I'm guessing batteries and diesel backups
| kicked in and are doing their job.
| rcarmo wrote:
| Yeah, we just told you that via Signal - that's how we built
| the networks :)
|
| (No relation to the other infamous Signal chat :))
|
| There should be 4-8 hours of battery backup on every site - at
| least.
| ethbr1 wrote:
| Wow! Battery capacity has gotten cheap.
|
| It's always fascinated me during disasters how independent
| telecomm can be. Kudos for all the engineering that went into
| it!
|
| I.e. even when any other conceivable dependency is down, the
| networks keep running.
| p_l wrote:
| Depending on country there are sometimes very strict
| requirements - or just traditions sometimes - around
| building up strong survivability in face of total loss of
| grid power. Including diesel and turbine generators on
| bigger BTSes let alone exchanges. If you drop capacity per
| terminal (so bandwidth) you can cover a lot more range at
| times which helps with mobile network resiliency.
| Scoundreller wrote:
| > Including diesel and turbine generators on bigger BTSes
| let alone exchanges.
|
| Or if you're AT&T, grid natural gas backup, so your CO
| goes down if electrical and natural gas both go out once
| the batteries die. Did I mention how they didn't build in
| roll-up generator connection points and had to emergency
| install those?
|
| https://news.ycombinator.com/item?id=25539586
| myself248 wrote:
| Old-school PSTN folks looked at XKCD 705 and chuckled. Late
| to the party, pal.
|
| The telephone network was designed from the ground up to be
| completely independent of _everything_ except fuel
| deliveries. If grid power is up, that's convenient, but
| it's totally not required.
|
| In many places, that's because telegraph and telephone
| lines got there before the power grid did. Lines running
| along railroads connected communities that had no
| centralized power generation. Delco-light plants at
| individual farms might be the only electric power for
| miles, aside from the communications lines themselves. Even
| if the only phone was at the rail depot, it still had to
| power itself somehow. As those communities sprouted their
| own telephone offices and subscriber lines branching
| throughout town, the office had its own batteries for
| primary power, and eventually generators to recharge them.
| (Telegraph networks largely ran from just batteries,
| recharged chemically rather than with generators, for
| years.)
|
| Fast-forward a century and there was just never a need to
| depend on anything else. As long as the diesel bowser can
| get down the driveway, the office can run indefinitely.
|
| Among old AT&T/Bell/WECo hands, the devotion to reliable
| service goes far beyond fanatical. Many offices built
| during the cold-war have showers in the basement and a room
| of shelf-stable food, though these are no longer
| maintained. The expectation was that whoever was in the
| office when the bombs dropped, would keep things running as
| long as they could. And when they couldn't anymore, well,
| there was probably nobody left to call anyway.
| 7373737373 wrote:
| https://xkcd.com/705/
| rcarmo wrote:
| Batteries lasted almost 8h but they progressively cut down
| services until we barely had voice.
| yreg wrote:
| > we just told you that via Signal
|
| Who are you and what's Signal?
| yreg wrote:
| I genuinely don't understand the sentence "Yeah, we just
| told you that via Signal"
| rcarmo wrote:
| We know each other.
| karohalik wrote:
| There is still no electricity, at least in my neighborhood in
| Lisbon. Less noise, more human voices outside. Time to meet
| some more neighbors I guess.
| karohalik wrote:
| Update: still no electricity, 4G/5G is barely working, city
| is more chaotic than usual but not that dramatic as some
| media say - there are huge queues to the buses and smaller
| shops that are still working, more people are outside.
| miohtama wrote:
| Most base station masts have lead battery backup up to 24h -
| 48h.
| mirekrusin wrote:
| People are not voting this up to top because they're offline.
| This outage is quite massive.
| znq wrote:
| Confirmed by contacts on the ground in Spain and Portugal.
|
| Not the best news source, but it's the only one I've found so
| far. HN moderators, feel free to replace it later with a better
| one.
|
| Supposedly also France is affected (unconfirmed)
| znq wrote:
| This is the first brief article from El Pais, a leading
| newspaper in Spain:
| https://elpais.com/economia/2025-04-28/apagon-electrico-masi...
|
| Translated version: https://elpais-
| com.translate.goog/economia/2025-04-28/apagon...
| EZ-E wrote:
| Looking forward to the postmortem
| rvz wrote:
| _Definitely_ looking forward to this postmortem.
| sph wrote:
| Accidentally dropping the production database doesn't seem that
| big of deal in comparison to killing the electricity in two
| countries.
| Calwestjobs wrote:
| unless it is database of russian oligarchys bank accounts in
| vietnam where they hidden their stolen money. XD
| carlos-menezes wrote:
| Who would be responsible for writing the postmortem? Are they
| required to?
| sph wrote:
| Not required, but engineers tend to enjoy this sort of thing.
| Also, since it affected some 60 million people and EU-wide
| grid interconnects, someone will have to explain what
| happened.
| pjc50 wrote:
| I would expect the Spanish parliament to have some means of
| summoning an explanation, via its local regulator and grid
| authority. https://www.ree.es/en/about-us/regulatory-
| framework
| leomca wrote:
| I think it's ENTSO-E, here's their most recent report into an
| incident on 21 June 2024:
| https://www.entsoe.eu/publications/system-operations-
| reports...
|
| For that incident, an expert panel was set up in July, the
| interim report was published in November, and the final
| report in Feburary 2025: so it'll take a few months.
| pjc50 wrote:
| Unfortunately these things usually take months, being done at
| the speed of bureaucracy.
| mrguyorama wrote:
| It's not the speed of bureaucracy, it's the speed of _doing a
| good job_
|
| Things like Air Crash investigations don't take a year
| because of paperwork FFS. Investigating things takes immense
| time.
| underdeserver wrote:
| How can we follow this story? Who's updating in real time?
| aosaigh wrote:
| BBC https://www.bbc.com/news/live/c9wpq8xrvd9t
| sph wrote:
| The Guardian
| https://www.theguardian.com/world/2025/apr/28/spain-portugal...
| pera wrote:
| Parts of France are also affected according to Sky:
|
| https://news.sky.com/story/large-parts-of-spain-and-portugal...
| FrustratedMonky wrote:
| France and Italy. Aliens? Russia? EMP?
| andy_ppp wrote:
| EMP would kill electronics no?
| Calwestjobs wrote:
| no, most electronics is shielded by default/by law. check
| UL/CE requirements. but it can make circuit breakers "turn
| off". EMP meaning nuclear high above, there exist emp
| devices of size of small family van, but those have range
| few 100 m/ft
| rcarmo wrote:
| Neither. Apparently a local grid overload and a cascading
| failure, but radio and newspapers don't agree on root cause.
| chippiewill wrote:
| There's a connected grid across most of continental europe.
| If Portugal's grid blacked out unexpectedly then Spain's grid
| could have been made unstable too, which then continue
| onwards to South of France and Italy.
|
| Just a typical cascade failure because it means everything's
| now running with lower tolerances.
| addandsubtract wrote:
| So, how do you stop this from cascading further? Did France
| have to cut them off the grid somehow?
| immibis wrote:
| A cascade failure (of any system) would propagate until
| it runs into a part of the system that's resilient enough
| to not fall over.
|
| Several people have said the amount of power transferred
| between France and Spain is very low compared to the
| amount of power consumed inside either France or Spain,
| which seems like as good as reason as any that it didn't
| affect France. The grid in France was presumably able to
| absorb the sudden change in power flow without itself
| breaking.
| raxxorraxor wrote:
| Nobody fed the electricity beetles and they are on strike
| now.
| benob wrote:
| > Parts of France also appear to be affected, according to
| Spanish media reports, which said Seville, Barcelona and
| Valencia were hit by the outage.
|
| The wording in the article makes it look like Seville,
| Barcelona and Valencia are in France.
| verzali wrote:
| The southwest of France seems ok. At least I still I have
| power.
| mrtksn wrote:
| Here's a "real time" map of electricity production and flow
| between countries: https://app.electricitymaps.com/map/72h/hourly
|
| Would be interesting to see if it will register here.
| rcarmo wrote:
| It should take 72 hours to update
| mrtksn wrote:
| I think that must be the scale so you can browse the
| historical data. The resolution on the slider is 1H when on
| 72H scale.
| suslik wrote:
| I'm in Valencia and it is indeed happening here. A street parade
| under my windows continues nonetheless.
| rambambram wrote:
| How are you posting this?
| suslik wrote:
| Mobile internet still works.
| samlinnfer wrote:
| Cell towers have backup generators?
| Tuna-Fish wrote:
| Most of them have batteries. How long they last depends,
| in my country they can typically manage 4 hours.
| sofixa wrote:
| They still need other infrastructure to get anywhere
| though (routers and other networking infrastructure).
| throw0101b wrote:
| Cell towers have batteries and/or generators, with fibre
| connections to data centres, which also generally have
| batteries and/or generators.
| xen2xen1 wrote:
| Always picture someone with a big can of diesel filling
| up a generator. Experience says it could be a genny
| running on natural gas. Had that at work before.
| raxxorraxor wrote:
| I think cell phone towers use surprisingly little energy,
| just a few kW. So even longer term operation should be
| possible withs bobs backyard generator running on vodka.
|
| But I would guess the whole network equipment would draw
| quite a bit, especially a modern infrastructure.
| rdtsc wrote:
| Batteries at least.
| Semaphor wrote:
| We had a (much smaller) regional (Lubeck and Ostholstein)
| power outage here in Northern Germany in I think 2018 or
| 19. No mobile internet until everything was restored ~6
| hours later, and most of the time no other cell service
| either.
| cowfarts wrote:
| "Cell towers have backup generators?"
|
| All essential infrastructure has to. Heck, if you have a
| landline you can probably siphon off some power from the DC
| component.
| codetrotter wrote:
| It works, but only some of the time now. It's been very
| unstable for me since the outage started. I'm in Spain.
| bayindirh wrote:
| Telcos have tons of backup power. They're required to have
| it.
| stavros wrote:
| > A street parade under my windows continues nonetheless.
|
| I mean, what else are you gonna do without power?
| ta1243 wrote:
| Usually?
|
| Sex. At least that's what everyone believes
|
| https://www.snopes.com/fact-check/from-here-to-maternity/
| sybercecurity wrote:
| Well, sex and looting, and maybe inventing new music
| genres.
|
| The legend is that hip hop and sampling started after the
| blackout in NYC. Some electronic and music stores were
| looted and the recording equipment eventually ended up in
| the hands of musicians looking to make a new sound:
| https://www.rollingstone.com/music/music-features/new-
| york-c...
| djuc wrote:
| i had an appointment in Barcelona, but can only guess it's not
| safe on the roads
| sparky_ wrote:
| Traffic lights are out across Iberia apparently, better stay
| home.
| stavros wrote:
| Do people forget how to drive if there are no lights? I can't
| imagine many people would think "well, there are no lights,
| which means GREEEEEEEN!". Over here, a traffic light that
| doesn't work means it's now a stop sign.
| drcongo wrote:
| Having driven in Madrid when lights were working, I think
| the stay home advice is strong.
| mango7283 wrote:
| That's still a recipe for massive gridlock as treating most
| traffic junctions as stop signs doesn't scale unless there
| are traffic cops to manually direct traffic
| cess11 wrote:
| I've never seen a traffic cop but I've driven past maybe
| fifty or a hundred broken or shutoff traffic lights,
| mainly in our largest city and never got stuck. Local law
| states that you give cars coming from the right
| precedence and the crossing self organises around that.
| stavros wrote:
| It won't be _fast_ , but it shouldn't really be
| _dangerous_.
| sph wrote:
| People routinely forget how to drive when the traffic
| lights ARE working.
| Zealotux wrote:
| I just walk through Eixample, traffic is a bit chaotic and
| police has trouble managing it. Stay safe!
| throw0101b wrote:
| As of: Mon 28 Apr 2025 07:22:00 EDT Mon
| 28 Apr 2025 11:22:00 UTC
|
| this is breaking just minutes ago:
|
| * https://www.bbc.com/news/live/c9wpq8xrvd9t (rolling updates)
|
| * http://archive.is/https://www.bloomberg.com/news/articles/20...
|
| * https://www.rte.ie/news/world/2025/0428/1509881-spain-portug...
|
| The Spanish national operator:
|
| * https://www.ree.es/en
|
| * https://en.wikipedia.org/wiki/Red_Electrica_de_Espana
| openplatypus wrote:
| Apparently Italy is also affected.
|
| https://www.elconfidencial.com/espana/2025-04-28/directo-cor...
| amarcheschi wrote:
| Tuscany good so far
|
| Edit
|
| However, I can't get the energy provided outage map to load,
| maybe too many people accessing it
| sph wrote:
| I'm in NW Italy and all seems fine
| g-b-r wrote:
| No
| foul wrote:
| Outage map is down but no blackouts here (NE Italy) atm...
| openplatypus wrote:
| I am getting downvoted for mentioning something from the linked
| article. Fun.
| rcarmo wrote:
| CNN story: https://edition.cnn.com/2025/04/28/europe/spain-
| portugal-pow...
| enopod_ wrote:
| In contrary to the CNN headline, air traffic doesn't seem to be
| affected. (As one would expect, as airports all have backup
| power systems)
| sparky_ wrote:
| Hello from Portugal. Local news is indeed reporting that power is
| out across all of Iberia.
| sgfgross wrote:
| I'm close to Valencia and we've had no electricity for about an
| hour.
| nottorp wrote:
| 4 years ago this almost happened:
|
| https://gridradar.net/en/blog/post/underfrequency_january_20...
|
| https://www.linkedin.com/pulse/day-europes-power-grid-almost...
|
| https://www.acer.europa.eu/news/continental-europe-electrici...
|
| I remember it because power went out in at least 1/3 of Romania
| back then.
| ptsneves wrote:
| It shows that even when the original issues are solved the
| reboot takes time due to power station being spooled down due
| to excessive production. I hope it is all back before telecoms
| start draining the batteries, otherwise things get uglier
| input_sh wrote:
| A similar thing happened last June, with Albania, Bosnia,
| Montenegro, and most of the Croatian coastline losing power
| simultaneously.
|
| Definitely felt surreal to first lose power to the degree that
| even traffic lights were no longer working, and then to hear
| it's also happening across the region just before mobile
| networks also went offline.
|
| https://www.reuters.com/world/europe/power-blackout-hits-mon...
|
| ~90 page report:
| https://eepublicdownloads.blob.core.windows.net/public-cdn-c...
| (beware: PDF)
| soyyo wrote:
| I am currently in my work office in Madrid, main building has
| electricity so I guess they have some backup generators, the
| kitchen however is out of service.
|
| According to local newspapers metro network, airport and traffic
| lights are all down
| Calwestjobs wrote:
| most office buildings have gas(methane) generator backup.
| Scoundreller wrote:
| Dunno about Spain/portugal, but in North America they're
| usually just for elevators/alarms, partial hallway lighting &
| maybe water pumps (for firefighting).
|
| Maybe a tech company with servers will pay extra for full
| backup, but it's not typical.
|
| Without fail, during every grid outage, some will fail to
| start and there will be elevator rescue calls throughout the
| city.
| maipen wrote:
| Yeah we only have telecoms and unreliable at times. If power
| doesnt comeback soon that too will fail.
| rcarmo wrote:
| Apparently a local grid overload near France and a cascading
| failure down the Spanish network, but radio and newspapers don't
| agree on root cause. Of course there is a lot of noise.
|
| For instance, one reporter asked one of the government flunkies
| whether it could be a cyberattack and they turned his
| noncommittal "maybe, we don't know" into "government says
| cyberattack may be ongoing".
|
| Be careful of idiot reporters out there.
|
| Edit: I'm listening to another radio interview where they are
| outlining the plans to bring online Portuguese dams and thermal
| generators over the next few hours, progressively unplugging from
| the Spanish supply (fortunately we have enough of those,
| apparently).
|
| It should take 3-4 hours to get everything balanced with only
| national supplies, and they will restore power from North to
| South.
| rpastuszak wrote:
| I was going to say something similar. I live in Portugal and
| I've heard a lot of panic/fear mongering, mainly from the
| techies in the co-working space I was working on and expats.
|
| (apologies for singling out these specific groups of people -
| my point is that it might be worth to put down news sources
| like xitter, and read AP/translated local Portuguese news)
| divan wrote:
| "xitter" in Portugese would be pronounced as "shee-tehr",
| right?
| amanaplanacanal wrote:
| That's how some have been pronouncing it in English, too.
| martinald wrote:
| Yes, something similar happened in the UK a while back. Full
| info here:
| https://www.ofgem.gov.uk/sites/default/files/docs/2020/01/9_...
|
| Key points that started it were (you can see the chain of
| events in the doc):
|
| 2.4.1. At 16:52:33 on Friday 9 August 2019, a lightning strike
| caused a fault on the Eaton Socon - Wymondley 400kV line. This
| is not unusual and was rectified within 80 milliseconds (ms)
|
| 2.4.2. The fault affected the local distribution networks and
| approximately 150MW of distributed generation disconnected from
| the networks or 'tripped off' due to a safety mechanism known
| as vector shift protection
|
| 2.4.3. The voltage control system at the Hornsea 1 offshore
| wind farm did not respond to the impact of the fault on the
| transmission system as expected and became unstable. Hornsea 1
| rapidly reduced its power generation or 'deloaded' from 799MW
| to 62MW (a reduction of 737MW).
| ethbr1 wrote:
| Curious question for someone familiar with power at grid-
| scale -- How granular is load shedding? And how is this
| measured / tracked?
|
| In my head, I'm thinking of generators/plants, connected by
| some number of lines, to some amount of load, where there are
| limited disconnection points on the lines.
|
| So how do grid operators know what amount of load will be cut
| if they disconnect point A123 (and the demand behind it) vs
| point B456?
|
| Is this done sort-of-blind? Or is there continual
| measurement? (e.g. there's XYZ MW of load behind A123 as of
| 2:36pm)
| martinald wrote:
| They don't really disconnect it like that in these
| circumstances (as in they choose what to disconnect). As
| far as I know generation plants will start disconnect when
| grid frequency is <49Hz or >51Hz (at least in the UK)
| automatically as it's all starting to go very wrong. This
| is what causes this huge cascade effect. Roughly speak less
| frequency means there is more demand than supply and the
| other way round for higher frequencies.
|
| This has changed a lot though, as even home batteries afaik
| will start discharging if they start noticing the frequency
| dropping to provide some support on generation. But if it's
| dropping too fast and too quickly it won't help.
|
| But yes they do have very granular info on all the HV
| sources and how much load is on them.
| pxdm wrote:
| I can speak for the GB case. Low Frequency Demand
| Disconnection (LFDD) occurs automatically and in stages
| when the frequency drops until it stabilises. The
| substations or feeders that are tripped off are not
| currently determined by real-time metering - instead they
| are pre-allocated based on their typical demand. This means
| that the system operator does not really know how much
| demand will be disconnected at any given time. If it's
| sunny, you could easily trip off a lot of solar generation
| connected on the low voltage network, causing the frequency
| to drop further. It is far from optimal!
| pbmonster wrote:
| > The substations or feeders that are tripped off are not
| currently determined by real-time metering - instead they
| are pre-allocated based on their typical demand. This
| means that the system operator does not really know how
| much demand will be disconnected at any given time.
|
| This is wild. From a amateur technical perspective, it
| would only take a cheap hall sensor inside the
| transformer to have a pretty good guess of how much
| current has been flowing to the load.
|
| Hell, put the hall sensor onto a board with a micro
| controller and a LORA transmitter and stick it to the
| outside of the feed line. Seems like an incredibly cheap
| upgrade to get real-time load data from every substation.
| pjc50 wrote:
| The board is not the expensive part. It's the getting
| reliability qualified and then having staff fit it to
| every substation, arrange the data links, and construct
| the dashboard.
|
| I also wonder what the realtime requirement is. Data from
| a minute ago is fine .. except in this kind of situation,
| when things are changing very quickly.
| remus wrote:
| The nice thing about frequency based regulation is it's
| an inherent property of the system, so as long as you're
| connected to the grid you've got the info you need to
| decide when to turn on or off.
|
| If you're monitoring real time power consumption you then
| need a whole extra infrastructure to communicate this
| info back and forth. Of course you then have to consider
| how you're going to keep that extra infra online in the
| event of power issues.
| pyrale wrote:
| Doing that at scale is tricky and requires a lot of
| people to participate in the mechanism, whereas the law
| only forces producers above a given size to participate.
|
| The estimates we get from seasonal studies are usually
| close enough, especially since load shedding isn't a
| finesse exercise.
|
| The situations that require load shedding usually give
| operators only a few minutes to react, where analyzing
| the issue and determining a course of action takes the
| lion's share. Once you're there, you want the actual
| action to be as simple as possible, not factor in many
| details.
| Beretta_Vexee wrote:
| Load shedding is an active measure taken when the grid
| operator knows that it will not be able to balance supply
| and demand. This is what happens in South Africa, where the
| operator preventively disconnects parts of the network at
| predetermined times to ensure network balance. As this is
| programmed, it is possible to rotate the areas under
| blackout.
|
| In this case, we are dealing with a widespread grid
| incident. The various grid protection mechanisms have been
| triggered to prevent interconnection overload. In addition,
| the generators are trying to correct the grid frequency to
| exactly 50Hz. At 49Hz, more power must be generated; at
| 51Hz, less power must be generated. However, if the
| frequency varies too much, there are also protection
| mechanisms to prevent the turbines from overspeeding or
| amplifying frequency variations.
|
| The grid is complex, and normally this type of incident is
| limited to one cell of the electricity distribution grid. A
| blackout is a domino effect, when a minor event triggers a
| chain reaction that disconnects more and more elements from
| the grid.
|
| Th grid operator will have to restart or reconnect the
| power plants one by one, restore power to stations and sub
| stations. All of this must be done in a specific order
| before power can be restored to consumers. All of this
| takes time, requires resources (you need men on the
| ground), and the slightest error can lead to further
| outages.
|
| Some consumers are prioritised, such as hospitals,
| transport infrastructure, telecoms and water networks. Many
| critical pieces of equipment have UPS systems, but these
| are not always designed for such long outages or have not
| been tested for years. There are patients with home
| equipment who will struggle.
|
| This is why rotating load shedding is preferable. The
| outages are not too long and vital infrastructure is not
| affected (or less so).
| pyrale wrote:
| In my area, "buckets" of average load are prepared ahead of
| time in coordination between the transporter (who gives the
| order to shed), the distributer (who implements the
| shedding operation as close to consumers as possible), and
| representatives of the state (who define priority areas
| that can't be load-shed, or that should be load-shed last).
|
| When yhe time comes, you just shed enough "buckets" to
| stabilize. Load shedding is not a precise task, when you're
| at that point, you'd rather load a few more megawatt and be
| safe, than play with the limits and be sorry.
|
| > So how do grid operators know what amount of load will be
| cut if they disconnect point A123
|
| Opening a line within the grid isn't used to shed load, as
| the grid is mainly redundant. It's used 1) to protect the
| line itself (either by letting it trip, or by opening it
| preventively), or 2) to force power to flow differently
| through the network, by modifying its impedance. Opening a
| line from the transport network is _not_ a way to load
| shed.
|
| In this totally random example [1], opening the line A-B
| increases impedance in the right part of the network,
| forcing power to re-route through the left part, and
| reduces the load on another line in the right path that was
| overloaded.
|
| [1]: https://excalidraw.com/#json=5l8OS96Wdke6l9YEClQt8,NJ-
| r2PtiE...
|
| > Or is there continual measurement? (e.g. there's XYZ MW
| of load behind A123 as of 2:36pm)
|
| The network is, indeed, monitored continually, and we do
| have potential network equipments failure simulations every
| few minutes, with contingency plans also simulated to make
| sure that they work in that particular case. This way, when
| shit hits the fan, we at least have a recently tested plan
| to start from.
| dark-star wrote:
| I can only imagine the "fun" in getting those synced back up to
| the European grid once this is all over... That alone will take
| weeks
| phplovesong wrote:
| Local betting site had odds if this was a attack by russia
| nacnud wrote:
| There's a map of realtime load flow here:
| https://gridradar.net/en/wide-area-monitoring-system (currently
| shows Spain and Portugal as 'offline')
| amarcheschi wrote:
| 10 mins ago Malaga was online, now it's offline. It doesn't
| look promising
| sofixa wrote:
| Might just be lag?
|
| In any case, if I recall correctly from a Youtube video I
| can't find (it was either Wendover or Real Engineering), if
| the grid is fully down, it takes quite a lot of effort and
| time to bring it back online because it has to be done in
| small steps to avoid over/under loading/using.
| amarcheschi wrote:
| Idk, it might be updated with some lag
| trollied wrote:
| It was Practical Engineering.
| https://www.youtube.com/watch?v=uOSnQM1Zu4w
|
| Very good video. Very good channel.
| pandemic_region wrote:
| Romania seems offline as well now?
| mrtksn wrote:
| Are you sure "offline" means that? Romania looks offline and
| when I checked their CNN they were reporting live from Spain
| about the blackout without mentioning Romania.
|
| Here: https://tv.garden/ro/F83BfecjsD6BjR
| rcarmo wrote:
| They were probably put offline as the network is rebalanced.
| That just means they (Romania) don't contribute to the
| network.
| nottorp wrote:
| Our government said we're fine with our local generation
| and are even exporting 200 MW (possibly to our eastern
| neighbors?).
|
| I definitely had no problems with electricity all of today
| (on the eastern side). And there was nothing in the news
| about local outages either.
|
| Funny enough, there were news before the Easter holidays
| that they're preparing for _extremely reduced demand_ by
| shutting down facilities.
| nacnud wrote:
| The map seems to be based on monitoring stations in the
| different locations, so yes - it's also possible that a
| station is offline for other reasons (maintenance, etc).
| steanne wrote:
| now mannheim shows offline
| leomca wrote:
| The best source for data seems to be the European grid operator
| themselves: https://transparency.entsoe.eu/dashboard/show
|
| Spain's demand: https://transparency.entsoe.eu/load-
| domain/r2/totalLoadR2/sh...
|
| Spain's generation:
| https://transparency.entsoe.eu/generation/r2/actualGeneratio...
|
| Spain's import/export with France:
| https://transparency.entsoe.eu/transmission-domain/physicalF...
|
| The filters can be used to see similar data for Portugal
| fulafel wrote:
| Interesting that the recovery (edit: of load graph) is going
| at relatively steady 600 MW/hour, it will be a while if the
| pace continues the same way.
| pjc50 wrote:
| Almost certainly being coordinated at that rate by adding
| one plant at a time, then a load region, then checking
| stability is holding, and so on.
| anal_reactor wrote:
| Call me crazy but I'd love to see full power outage at least once
| in my life
| stavros wrote:
| Well, maybe you can see one now, depending on how far from
| Spain you live!
| mango7283 wrote:
| We had one in my country when I was a kid 30 years ago after a
| substation tripped and took down the rest of the grid. We broke
| out the candles and chilled I guess. Power was back up in my
| area that same night.
| sexy_seedbox wrote:
| Has not happened to me yet in Hong Kong, even through 2 Typhoon
| Signal 10s.
| libertine wrote:
| Hopefully there won't be any loss of life due to this event.
| crvdgc wrote:
| In Japan, sometimes earthquakes will cause regional power
| outages, though they are usually recovered quickly.
| flobosg wrote:
| I was living in Chile during its 2010 earthquake
| (https://en.wikipedia.org/wiki/2010_Chile_earthquake). The
| power outage in my hometown lasted approximately two weeks.
| verzali wrote:
| I used to live in an area that regularly had power outages
| during thunderstorms. I was also once in London during a power
| cut that took out most of the city centre. It was a strange
| experience, but not as weird as walking through empty streets
| in the time of covid.
|
| Maybe just my perception, but power outages seem to be getting
| rarer with time, though when they do happen they seem to be far
| larger.
| anonzzzies wrote:
| I've heard from friends in France, Portugal and I am in Spain. I
| saw people are also mentioning parts of Italy.
| brutuscat wrote:
| Yup I confirm. Went to turn on the car to hear the radio. Andorra
| seems to have power from French side, though in Catalunya we do
| not.
|
| Should have paid the extra EUREUR to put the solar panels in
| backup mode...
| solarkraft wrote:
| This is a really good stress test. I'm impressed that
| communications still seem to work somewhat for now.
| ndsipa_pomu wrote:
| It's surprising how vulnerable electric supply can be.
| pandemic_region wrote:
| Reported here
| https://news.ycombinator.com/item?id=43819791#43820127
| nottorp wrote:
| Here too:
|
| https://news.ycombinator.com/item?id=43819914
| dang wrote:
| We eventually merged all the threads.
| codetrotter wrote:
| I'm in Spain. Internet here is very unstable now also.
| Scoundreller wrote:
| How are the routes looking?
| Ronzie wrote:
| Same happened in Murcia. Just thought it was a normal power
| outage!!
| amarcheschi wrote:
| Some parts of Spain are slowly coming back online
| https://x.com/RedElectricaREE/status/1916818043235164267
| libertine wrote:
| The crazy thing is how is Twitter still being used to
| communicate these events.
| dijit wrote:
| Agreed.
|
| It's crazy how momentum can carry a business.
|
| To use a potentially controversial example, Microsoft
| products (Office, Windows) are still extremely entrenched
| despite the overwhelming majority of knowledgable people
| agreeing that they're on a steep downward trajectory and the
| alternatives have long since surpassed them.. leading to
| this[0] recent video from Pewdiepie...
|
| [0]: https://www.youtube.com/watch?v=pVI_smLgTY0
| Ronzie wrote:
| Murcia is also without power. We live in the campo so presumed it
| was a normal outage. Obviously not
| ndsipa_pomu wrote:
| https://www.euronews.com/my-europe/2025/04/28/spain-portugal...
|
| > A fire in the south-west of France, on the Alaric mountain,
| which damaged a high-voltage power line between Perpignan and
| eastern Narbonne, has also been identified as a possible cause.
| ksec wrote:
| I saw it on twitter and had to go to BBC to make sure this isn't
| some joke or hoax. 2025 and we have National level power cuts?
|
| I know it may be rare but I think some day we really need to move
| or mandate every single flat / home / apartment / living places
| to have a 12 - 24 hours backup battery included. Something that
| has 10K+ Cycles, durable and non-flammable. Not only does it make
| sure our modern lives without sudden interruption, it also solves
| the renewable energy problem.
| elric wrote:
| Sounds like an incredibly expensive solution to a problem that
| could be tackled more efficiently at a larger scale.
| jvanderbot wrote:
| Didn't we used to pump steam around for heating and now homes
| have individual furnaces? If the cost for this were about the
| cost of a furnace I don't see why this wouldn't be viable.
|
| I bought a generator for just this situation.
| Loughla wrote:
| There is a very, very appreciable difference between moving
| hot steam around and moving electricity around.
|
| The thing you said doesn't really make sense to me; I'm not
| sure it's an apt analogy.
| tomatocracy wrote:
| Plenty of places still do, and in those places district
| heating still is usually the cheapest source of heat.
| elric wrote:
| That's a great example of why this is a bad idea. District
| heating is a lot more efficient than using individual heat
| sources. Especially so when the heat being used is heat
| that would otherwise go to waste (e.g. waste heat from
| industrial processes).
| jvanderbot wrote:
| I get the efficiency obsession, this being an engineer
| forum, but aren't we talking about resilience?
| elric wrote:
| I think we can have both? I don't think anyone is
| advocating for one giant battery. In my ideal imaginary
| world, there are medium sized battery parks near every
| transformer station. In that same fantasy, the
| foundations of wind turbines are also batteries, so I
| shouldn't pretend that this is hard science.
| matkoniecz wrote:
| In Polish cities there are attempts to move in exact
| opposite way (close individual furnaces) due to rampant
| pollution during winter (1000% of PM 2.5 norm level of
| pollution happening occasionally, over 400% PM2.5 norm
| lasting for weeks etc)
| fulafel wrote:
| First-gen district heating was steam, then moved to hot
| water. It didn't go away [1] but there's regional
| differences. In cold climates there are oil based simpler
| backup plants that can cover for the normal CHP[2] and
| elctric+heat pump based generation. It's a good way to
| buffer and use cheap energy in periods of cheap electricty
| from eg high wind production[3].
|
| [1] https://www.wedistrict.eu/interactive-map-share-of-
| district-...
|
| [2] https://en.wikipedia.org/wiki/Cogeneration
|
| [3] https://en.wikipedia.org/wiki/Thermal_energy_storage
| quickthrowman wrote:
| There are plenty of places that still have district heating
| and cooling.
| alchemist1e9 wrote:
| I've read discussions about preparing for a strong solar
| flare basically simply involves having on hand spare
| equipment stored unconnected and that then would also
| immediately improve repairs and maintenance for regular
| issues. I don't know the details but I believe the US is
| looking at providing funding to the utilities companies to
| acquire the duplicate extra equipment.
| UltraSane wrote:
| Rich countries should have a strategic transformer
| stockpile since they have a long lead time and should last
| forever in storage.
| Loughla wrote:
| Mandate a battery backup. Instead of requiring the power
| suppliers to build and maintain reliable energy.
|
| That is just shifting someone else's mistake into being my
| responsibility.
| littlestymaar wrote:
| "How to waste money and valuable resources" in a nutshell.
|
| Like it or not but we are a species of social animals, you
| cannot live without relying on others. That's just
| delusional.
| leansensei wrote:
| There are degrees of dependence on others, you make it
| sound as if it's a binary choice.
| littlestymaar wrote:
| I don't, the comment I'm replying to does.
| moooo99 wrote:
| People seem to love those expensive solutions though.
|
| I know so many people who invested in overly expensive
| battery storage systems for their solar. Power in Germany is
| expensive, but even with that expensive power many of those
| battery systems will never hit a positive ROI. But they're
| still happy for the feeling of being ,,independent".
|
| I'll turn 26 in a few months. The first time I experienced a
| power outage in my life was two weeks ago when a construction
| worker in our basement drilled into the wrong wall...
| nicoburns wrote:
| It's likely that battery backup solutions won't be
| expensive for long. Battery prices have been falling
| exponentially similar to Solar, and there is no reason to
| think it will slow down anytime soon (there is still room
| for improvement just from economies of scale).
|
| I predict that home batteries will become a "no brainer"
| from a financial perspective (for anyone who has the
| upfront capital to purchase them) within the next 10 years.
| paganel wrote:
| Not all of us live in (semi-)detached homes, and I'm also
| not sure how one would implement this at the building-of-
| appartments level, as in, who's going to pay for it and
| how do you handle the battery storage capacity for
| hundreds of apartments from the same building in a safe
| and cheap-enough manner?
| moooo99 wrote:
| I haven't seen battery price quotes for retail customers
| for a while now, everybody I know who wants one has one.
| But not too long ago a battery bank including
| installation was easily in the range of 800 to 1000
| EUR/kWh of storage.
|
| Battery storage may become a bit of a no brainer from a
| purchasing price point of view, but I don't think it will
| actually be more beneficial for people, especially if BEV
| adaption continues at a similar rate and bi-directional
| charing becomes widespread
| mrguyorama wrote:
| You can put together a simple and fully independent solar
| island with 5kw/h capacity and 7kw nameplate generation
| for $4100. This is a plug and play system, with zero
| wiring and no knowledge needed. For the idea of the kind
| of profit margin that includes for the company, their
| "smart isolation switch" is $1k. Jackery has several
| competitors, and all of their build quality is pretty
| good, and the warranties are acceptable.
|
| 20kw/hr of just storage is about $10k. Every one of these
| companies also has standard DC input ports for bring your
| own solar panels, because their panels are overpriced.
|
| The 8-Bit guy has been using such a system to maintain
| his studio. It is reliable, uses cheap solar panels. This
| stuff is commodity.
| littlestymaar wrote:
| Nitpick: it's kWh not kW/h since Watt is already a unit
| of energy divided by time (1W is 1J per second).
| Finnucane wrote:
| I was living in Brooklyn, NY during the 2003 blackout that
| knocked out much of the northeastern US for a couple of days.
| In those days we weren't quite so dependent on cell phones
| and such things. People walked home, since the trains were
| stopped. Guys grabbed flashlights and directed traffic. We
| listened to the news on the car radio. After that I went to
| J&R and bought a wind-up emergency multi-band radio. In the
| intervening 22 years, I've never used it. Though at the rate
| things are going, maybe that will change.
| joha4270 wrote:
| Most of the time, the grid _is_ reliable. How much investment
| do we actually want to prevent a few hours of downtime once in
| a blue moon?
|
| And even if we do want to invest in large amount of grid
| storage (which we would need to anyway, if we want to
| transition to renewables), I'm not sure pushing this down to
| the individual house is sensible. Its a great way to limit
| economies of scale and make maintenance/inspection harder.
| chii wrote:
| > I'm not sure pushing this down to the individual house is
| sensible.
|
| why not? Having distributed buffers ought to make the system
| more resilient in most cases wont it? Not to mention that
| these home level batteries can be used to smooth out power
| usage and lower peak loads.
|
| In fact, having an EV car act as this same battery would be
| an even more efficient use of resources.
| UltraSane wrote:
| because installing and monitoring 1 million small batteries
| is vastly more work than 100 huge batteries.
| hem777 wrote:
| Not if 1 million people install and monitor them instead
| of 100 people.
| UltraSane wrote:
| 100 huge batteries are going to be monitored and
| maintained far better than 1 million.
| 12_throw_away wrote:
| Given how the average consumer monitors, maintains, and
| secures their home PC ... this is a level of assurance we
| want to put on the power grid?
| megous wrote:
| How many people will turn off AC and will not bake that
| cake, or wash clothes, when they know they're going off
| their own battery vs some electricity distributor's
| battery shared by others?
|
| And if someone is dumb enough to do high load stuff on
| their own battery during a blackout, then it's less of a
| problem for others. Also individual failures will cover
| less homes.
|
| Incentives and consequences will be different and
| differently spread.
|
| And on individual level, you can also chose whether you
| want this or are fine with outage. (I'm against mandating
| this)
| vel0city wrote:
| Why not? Maybe because I'd rather not have to spend that
| much of my own capital on something I'd generally never
| use. I'd rather not spend the maintenance on that thing I'd
| generally never use. And finally, I'd rather not take up
| the space in my home with a large thing I'd generally never
| use.
|
| I'd prefer for the power coming to my house to just not go
| out. The grid operator can install batteries in places
| other than my home. The grid operator can maintain them for
| me. The grid operator can get cheaper loans than I can for
| installing them, they're staffed with supposed experts in
| this stuff, just have them handle it on their premises.
| HeatrayEnjoyer wrote:
| Do you have any reasons besides that?
| vel0city wrote:
| Sure. Having large batteries is an additional fire risk
| in my home for once again something I'd generally never
| need. If these batteries are in my home but managed by
| the grid operator I'd probably have to allow their people
| into my home to service them which I'd rather not have to
| do if it's something I'd generally never use.
|
| Also, this further just makes having stable electricity
| yet another thing in the wealth gap. Only those wealthy
| enough to afford the high upfront capital costs, the
| ongoing maintenance cost, and the space to store it get
| reliable electricity, fuck everyone else! Or we can just
| focus on investing in a stable and clean grid and share
| that cost with everyone, all you need is to just be
| connected.
|
| But hey if I get a massive battery bank I'll have power
| for when the end times come. I won't be able to go get
| groceries anymore and eventually the fiber line and
| radios around me will go quiet but at least I'll be able
| to play videogames. For a few hours at least.
|
| Don't get me wrong, I'm an Eagle Scout, be prepared and
| all that. I've got a big pile of charcoal, a chunk of
| propane, a camping stove, several day's supply of water
| and canned/non-perishable foods, some batteries for
| lanterns, etc. The cars all get topped off when big
| storms seem possible. If the big outage comes this will
| be more worthwhile than being able to turn on the TV for
| a few hours, and cost significantly less than several
| dozen kWh of batteries. And if the power is out for more
| than a week or two I'll have far bigger concerns than
| being able to post on Hacker News.
| Filligree wrote:
| You're not wrong about the fire risk, but just to note,
| house batteries nowadays are almost always LiFePO4 --
| which don't catch fire the same way LiPo batteries like
| to do.
|
| They can still do so for other reasons, like a short
| circuit in the wiring.
| vel0city wrote:
| Fully understood and agreed. That's why I bothered saving
| that for the second comment, because it's not necessarily
| _that_ high of a fire risk but it is still a little bit
| of one. Having that much energy stored in that small of a
| package will always have some kind of risk of "what
| happens when that stored energy gets released in an
| uncontrolled and rapid manner?"
|
| And that said, I do have lots of other somewhat beefy
| batteries around the house. They do a lot of useful
| things for me such as power my tools including my lawn
| mower, string trimmer, hedge trimmer, saws, etc. There is
| a massive one in the car parked in the garage. In these
| cases it is a useful trade off of that slight risk as I'm
| actually getting something normally useful out of it.
| tzs wrote:
| Batteries can be useful for more than just getting
| through power outages. More and more utilities are
| offering time of use based pricing. For people on such
| plans a battery can be useful even if there are never any
| power outages by letting you store power from off peak
| times to use during peak times.
| vel0city wrote:
| Unless it's a really massive swing it'll be a gamble if
| it will actually break even. A useful battery install for
| a home is _at least_ several thousand dollars. I 'll
| install $10k worth of batteries and manage to arbitrage
| $30/mo off my electrical plan, excellent! I'll break even
| in 28 years, wonderful! Will the batteries still have a
| meaningful charge after 28 years of use? Who knows!
| Meanwhile I'll miss out on the opportunity cost of that
| $10k over that same period.
|
| And once again, another point for growing the wealth gap.
| Poor people who can't afford the thousands of dollars for
| installing the batteries get shafted by such things
| instead of us assuming the grid operators will just
| smooth these prices out for us.
|
| Once again in the end I'd just prefer if there were
| actors on the grid running massive batteries able to
| arbitrage the extreme spot prices and sell me electricity
| at a reasonable rate all the time instead of me having to
| actively min/max every dang thing every moment in my
| life.
| vel0city wrote:
| Doing some math on that $10k for an example of lost
| opportunity costs, _today_ you can get a 10yr CD at 5.5%
| APY. Over a decade that 's a hair over $17k, practically
| guaranteed return. So, to properly break even you're
| going to need to not just save $10k worth of electricity,
| you'll need to save something more like $17k. You'll need
| to arbitrage _at least $141.67 /mo on average_ for that
| entire 10-year period to break even compared to what you
| could have had just investing that into pretty safe
| savings.
|
| Personally, I'd rather toss that $10k into my kids'
| education funds and sign up for a fixed rate electricity
| plan. Hopefully that'll be better ROI.
|
| Grid-scale batteries have pretty different economics than
| some batteries in my garage. Their real-estate cost is
| probably significantly lower than the cost for halfway
| decent residential. They're buying a much larger order of
| batteries, so their per-kWh cost is probably a good bit
| lower. Also, their installation cost per-kWh is also a
| good bit cheaper. They're probably completely fine
| buying/selling purely on open spot markets, meanwhile in
| the end I'm going to need to run my pool pump and I'm
| going to want to do dishes and laundry and charge my car
| and all the other things on some reasonable schedule so
| I'd like _some_ amount of price protection on whatever
| time-of-use plan I get (a day of $9 /kWh electricity
| prices will surely wipe out whatever gains you might have
| made). They're probably able to get cheaper loans than
| me, so it's easier for them to arrange the big capital
| investment instead of high interest loans or having to
| invest existing capital.
|
| I'm not arguing the economics of batteries don't make
| sense in today's power markets. Far from it, I've been
| toying with getting into it in North Texas. But buying a
| few kWh of batteries and putting them in my garage
| probably isn't going to break even and almost certainly
| isn't going to make me money.
| vel0city wrote:
| Further thinking about what you'll actually get, it looks
| like a decent average price per kWh of installed home
| battery backup is about $1,000/kWh. So if we got $10k of
| batteries installed, you're looking at 10kWh of energy
| you can store.
|
| Let's say you're in California, which has a pretty decent
| swing in energy prices available to the home consumer.
| You'll see prices swing between $0.27 to $0.65/kWh.
| You've got a 10kWh battery pack, so you can arbitrage
| something like $0.38/kWh * 10 kWh = $3.8/day, assuming
| you get a full charge cycle and ignoring efficiency for
| simplicity's sake. $114/mo. So assuming you didn't have
| any loans and you're ignoring opportunity costs on that
| $10k, your break-even is in ~88 months or ~7.3 years.
|
| After a decade of ~$114/mo, you'll have offset $13,680,
| assuming you _always_ used a full charge and you
| experienced no other maintenance costs and bought it cash
| on hand and that energy prices didn 't change and had a
| perfectly efficient inverter to charge and discharge the
| batteries and the batteries still had 100% charge
| capacity the whole time. The 10-year CD, which you could
| just forget about for a decade, stands at $17,310.
|
| And this is for one of the few markets in the US that
| really offers that big of a time of use plan other than
| "free nights and weekends***" (with pages and pages of
| fine print and other fees) For instance, I just looked at
| a time of use plan available to me here in North Texas.
| 9AM-4PM I'm billed at $0.068 for the generation,
| $0.050029/kWh plus a flat $4.23/mo for delivery. Outside
| of that window I only pay the delivery cost. But that
| means I'm only really able to arbitrage $0.68/day with a
| 10kWh setup. That's 490 months or a hair over _forty
| years_ to break even.
| toast0 wrote:
| The more batteries available for that arbitrage, the less
| effective it is. As a grid user, I'd rather the grid own
| and operate those batteries. Certainly some of my outages
| are due to line issues between me and the substation, but
| I think more of them are due to line issues between the
| substation and its upstream. Having storage at the
| substation could potentially help all the customers and
| centralize the maintenance and safety burdens. They could
| do some energy cost arbitrage (although I live in the PNW
| and we generally don't have time of use cost
| distinctions, because hydro doesn't care)
|
| I do have a 35 kWh propane fired generator onsite though,
| which does provide for outages regardless of where the
| break in the line is.
| joha4270 wrote:
| The peak load of a house is substantially higher than the
| per-house peak load of a city. People don't all turn on
| their ovens/dryers/hairdriers at the same time.
|
| You could distribute this capacity at each house and feed
| it back to the grid during peak times. But is TCO of 1000 *
| 100kWh same price as 100MWh worth of capacity?
|
| If you're going to have the battery anyway(a car) its hard
| to compete with, but once you need more capacity, I'm not
| sure it makes sense to distribute it quite as much.
| trollied wrote:
| In the UK, the grid used to have to manage the load when
| millions of people would all put the kettle on during
| popular TV advertisement breaks.
| Scoundreller wrote:
| With places that have time-of-day rates, I wonder what
| will happen when enough systems get advanced enough to
| precisely kick-off/on (or slow down/speed up) in unison
| at precise times.
|
| I've set this up on my "smart" thermostat to speed up
| just before rates go up and kick off for a while once
| they do (and it was a pain to setup, somehow this
| functionality was not baked in)
| pjc50 wrote:
| _Mandating_ battery storage is nonsense. Making the
| regulatory system better for it is not, though; it would be
| good to make it possible for consumer battery systems to do
| "islanding" properly so they can be true backups.
| Filligree wrote:
| Or at least not making it _illegal_. Regulate it all you
| want, make sure it 's safe, but please let's let homeowners
| install a transfer switch if they're willing to pay for it.
|
| In a longer outage it'd be a massive benefit to have even
| just a few houses here and there that retain power.
| immibis wrote:
| Where is it illegal?
| anonzzzies wrote:
| This is very rare. Not actually worth to make that much fuss
| unless you are a hospital.
| everfrustrated wrote:
| Spot a European at a hundred paces by their reflex that
| anything happens the _government_ must do something.
|
| If electricity is that important to you, buy your _own_
| resilience. Tesla powerwalls and non-tesla equivalents have
| been available for ages.
| arccy wrote:
| unlike americans, people in europe don't just huddle in their
| own fortresses afraid to even take a step outside without
| riding in tanks.
|
| power cuts affect more than just your home.
| Niksko wrote:
| Spot an American at a hundred paces proposing that you either
| have the money to buy batteries yourself, or else you should
| just eat it and suffer
| mr_mitm wrote:
| A Tesla powerwall won't help with trains and traffic lights.
| Those are important to me as well.
| mensetmanusman wrote:
| Something that stores enough energy to bake cookies will always
| somehow be flammable.
| oceanplexian wrote:
| Gravity storage of water. Not flammable, but could definitely
| bake cookies.
| mensetmanusman wrote:
| The things that convert that energy as part of the full
| water gravity battery unit will have flammable bits :)
| andyjohnson0 wrote:
| I have family in Andalusia and they seem to have dropped off the
| net. Mildly concerning.
|
| Also, there are currently four different submissions re this on
| the front page. I'd suggest we dont need any more.
| amval wrote:
| Apparently this spans more countries? Very strange. Possibly a
| cyberattack or sabotage?
|
| Growing up in Spain I've never experienced anything like this
| (not there at the moment, but friends have told me over
| WhatsApp).
| cr3ative wrote:
| Grids are widely interconnected. Problems on one grid can and
| do cascade to another.
| voidUpdate wrote:
| Possibly some kind of rare cascade failure where it might have
| been on the edge for a while, and some small event happened
| that tripped things, similar to the american northeast blackout
| in 2003. High demand, plus a power station going offline meant
| more demand on some interconnects, which shorted on trees and
| were cut off, putting more load on other lines until the entire
| system collapsed
| ksec wrote:
| Continental Europe Synchronous Area [1]
|
| The whole Europe power grid are somewhat interconnected I wont
| be surprise if this knock on effect start knocking out other
| surrounding countries.
|
| [1]
| https://en.wikipedia.org/wiki/Continental_Europe_Synchronous...
| HeatrayEnjoyer wrote:
| Is that a risk with the small amount of interconnect
| capacity?
| londons_explore wrote:
| Europe still has many megawatts of solar with inverters that
| disconnect when detecting any grid disruption.
|
| This is the absolute worst thing to do when there is a shortage
| of power - you immediately make the shortage worse and more grid
| disconnects.
|
| The _real_ fix is a grid with second by second pricing based on
| system frequency, and every individual user allowed to set a
| daily 'spend cap' of euros/dollars, letting them choose how much
| they are willing to pay for reliability.
|
| Such an market has a huge stabilizing effect on demand, meaning a
| major incident would probably only have fairly small impacts on
| system frequency and embedded solar wouldn't disconnect.
| 0xTJ wrote:
| This sounds more like a dystopian novel, where only the wealthy
| can afford stable power and everyone else is left without.
| londons_explore wrote:
| The current system is that everyone pays equally for X amount
| of stability, and for some people they don't get as much
| stability as they desire (eg. These people in Spain), whilst
| others pay more money and than they'd like when they would be
| fine with 5 minutes outages once a year.
|
| That made sense before technology became available for
| everyone to make their own choice - but that is no longer the
| case.
| doix wrote:
| That is basically what happens in South Africa today. There
| are rolling blackouts and rich people with backup generators
| and batteries are unaffected. It's gotten to the point where
| richer cities (like Capetown) have their own power sources to
| mitigate the problem.
| littlestymaar wrote:
| This comment is Dunning-Kruger effect in action...
|
| Believe it or not, but maintaining an electricity grid is a
| massive undertaking, and the people in charge of it knows the
| topic much better than you do.
|
| The problem isn't a _market_ problem, it 's a _physics_
| problem: having a _synchronized_ grid of _AC current_ with many
| producers over a wide area is a real challenge, even when the
| underlying issue is resolved it takes a lot of time to add the
| power plants (or renewable equivalent) to the grid because they
| must be _synchronized_.
| rightbyte wrote:
| Having such interdependent grids seems like a market or
| political problem not a physics problem to me.
| pjc50 wrote:
| Interdependent grids are usually good: they allow you to
| average out the effect of a single power station failure
| over a much larger area, and to amortize prices from a
| wider area of suppliers.
| rightbyte wrote:
| Sure but if you can't cut off failing parts in a sane way
| it seems like a liability.
|
| Like, what can you do, use some 1000 of MW to melt iron
| rods or something to give the power stations time to slow
| down? Free wheels?
| littlestymaar wrote:
| What the heck are you talking about?
|
| Don't you realize that the smaller the grid, the more
| important the instantaneous load variations can be in
| relative term and the harder it is to keep things running
| smoothly? It's not a theoretical concern, it's why
| electric networks on islands are much harder to work with
| and much more prone to collapse than bigger networks.
| rightbyte wrote:
| I think the point I am trying to make is that the risk
| for big area outtakes are lower with more figurative
| islands.
| littlestymaar wrote:
| The reason why we interconnect grids has nothing to do with
| politics and everything to do with physics, politics can
| sometimes lead to disconnections (like how the Baltic
| states disconnected from the Russian grid earlier this
| year) but it comes with great cost and involves careful
| planning (the fact that the Baltic states remained
| connected for almost three years after the start of the
| Russian invasion of Ukraine should give you an hint of how
| challenging it was).
|
| The bigger the grid, the more efficient and resilient it is
| (and managing electric grids on islands is a nightmare),
| but it comes with a significant complexity and means
| restarting from zero is harder.
| japanuspus wrote:
| The fact that the post you reply to includes such technical
| details as frequency-based pricing, indicates that the author
| has an above-average understanding of the technicalities of
| the power-grid.
|
| Also, nobody in the field disagrees that in the more
| distributed grid we are seeing today, more endpoint
| communication and control could lead to more resilience.
| Whether pricing signals are the best path is a more open
| question, but they certainly appear to be a feasible option.
| littlestymaar wrote:
| > The fact that the post you reply to includes such
| technical details as frequency-based pricing, indicates
| that the author has an above-average understanding of the
| technicalities of the power-grid.
|
| No it doesn't. The fact that it's being said in a comment
| full of nonsense tells me that they don't have "above-
| average understanding". They probably have read _something_
| , _once_ , and now thinks they are an expert, that's
| literally what Dunning-Kruger is about.
|
| They seem to believe that the equilibrium of supply and
| demand is all that matters, when it's just one piece of the
| puzzle and among the easiest to manage. Large, nation-
| scale, failures like this one are very unlikely to be
| caused by a lack of supply alone and markets are nowhere
| near fast enough to help preventing these.
| dist-epoch wrote:
| Texas had a pretty free market, with many users on a floating
| price, didn't work that well either a few years ago when their
| grid went dawn.
| Dma54rhs wrote:
| EU electricity market is very similar to it, but
| interconnected and probably more free market style. A lot of
| people pay the spot price depending on county and their
| provider of course.
| xp84 wrote:
| I didn't really find anything problematic from an economic
| standpoint with the Texas power disaster. On the surface, the
| main problem there was just that a bunch of people were on
| spot energy plans, in order to save tremendous amount of
| money as long as the spot price stayed low. This is simply
| gambling. Those people after some period of savings suddenly
| cried foul when their gamble went south as prices spiked. The
| only issue I see here is if they weren't adequately informed
| of that possibility. Obviously, the people who got the $2000
| electricity bills are going to be the decide they weren't
| informed adequately, and I'm sure it was spelled out in a
| document the size of an iTunes license agreement that
| literally no customer ever read. But anyway those people had
| themselves to blame as that plan was not mandatory or
| default.
| ajsnigrutin wrote:
| Oh god.
|
| Let's skip the technical problems in your theory and focus on
| the social.
|
| People need power to survive. You know, food, hot water, light,
| work, internet, mobile phones, entertainment, etc. This
| requires stability, not second by second pricing.
|
| When you put a chicken in an oven, you want to cook that
| chicken and eat it, feed your family. Electricity price rising
| in the next few minutes would mean that you either have to risk
| disease (chicken staying in the dangerous temperatures until
| the electricity price drops) or being hungry and throwing food
| away. This is not how you want society to function.
| acc_297 wrote:
| You require inertia in the grid to maintain frequency and other
| stability stuff.
|
| Solar PV is great but is mostly grid-following so cannot
| operate on it's own. As I understand it you need a minimum
| fraction of power generation to be large spinning turbines.
|
| I think this problem can be mitigated with add-on rotational
| mass style kinetic energy batteries or something like that. I
| don't think variable energy pricing will help if it's an issue
| with over-demand the grid managers can do rolling blackouts to
| manage while fixing the supply problems. The grid is just
| broken at the moment and the solar can't maintain the grid
| alone.
|
| https://en.wikipedia.org/wiki/Inverter-based_resource
| thyristan wrote:
| Large Inverter Based Resources (IBR) such as huge solar
| parcs, grid-scale batteries or high-voltage direct-current
| (HVDC) lines can be programmed to behave like rotating
| generators, or even to smooth out smaller ripples. They also
| don't necessarily need a leading grid frequency but can be
| used to generate their own frequency normal to cold-start or
| resync a grid.
|
| Only "small stuff" IBRs need a leading frequency from the
| grid and disconnect outside their safety corridor because
| those usually aren't controllable from some central grid
| authority. Thus the stupid-but-safe behaviour mandated for
| them.
| eldaisfish wrote:
| While this is plausible, this solution works in the world
| of simulations.
|
| We do not have a solid understanding of how inverter-based
| fast frequency response works with an existing grid that
| uses physics-based inertia.
| eldaisfish wrote:
| This is the exact kind of techbro nonsense that I've come to
| expect from this website. No empathy for your fellow humans and
| dressed with a side of "computers and code plus finance".
|
| I thank the heavens that the people who run the electricity
| system do not share your opinions.
| pcardoso wrote:
| In Portugal here. Lost power about 1 hour ago. Reading the news
| while batteries last.
| netsharc wrote:
| Amazing/curious that all the infrastructure to feed news to
| your phone still works. I can imagine a poorly-built systems
| would have some infrastructure without UPSes, and in all that
| chain of technology, there'll be an element or more without
| power. Not saying Spain/Portugal would necessarily be more
| prone to have poorly-built infrastructure, I assume the whole
| world runs on half-assery.
| postexitus wrote:
| Turns out not everybody in the world are jokers.
| myself248 wrote:
| If their networks are built like the ones I've worked on,
| everything has at least a little battery, but only some of it
| has generators to keep going after that.
|
| So _some or most_ cellular towers will have generators, and
| their fibers will backhaul through repeaters, _some or most_
| of which will have their own generators. When it gets back to
| the MTSO, that will definitely have large diesel turbines on
| site and at least 24 hours of fuel with priority refueling
| contracts.
|
| I'd expect there to be a lot of outages, for instance where
| all the towers in a region end up backhauled through a site
| where the generator fails or was never installed for some
| reason. But there will also be a lot of places that stay up
| in some capacity because, more or less by happenstance, all
| the fuel tank permits got approved and all the equipment
| actually worked.
| eknkc wrote:
| I believe it was 10 years ago or something, we lost power in
| Turkey. I mean the entire country for around 10 hours.
|
| Data, cellular etc everything kept working. But at some point
| I guess the generators and batteries started to fail and
| capacity degraded.
| carlos-menezes wrote:
| This outage does not affect the islands, at least in Portugal.
| Madeira has power.
| enopod_ wrote:
| Spanish islands are also all ok, they have independent networks
| prof-dr-ir wrote:
| The last major European blackout was in 2006:
|
| https://en.wikipedia.org/wiki/2006_European_blackout
| 9dev wrote:
| Anyone read Blackout[1]? So it begins. Both countries completed
| (or almost completed) their smart meter rollout...
|
| [1]: https://en.wikipedia.org/wiki/Blackout_(Elsberg_novel)
| terom wrote:
| It looks like the Iberian peninsula is relatively isolated from
| the rest of the CESA synchronous grid, with only 2% cross-border
| capacity compared to local generation. [1]
|
| There's a map at [2]
|
| > The Spanish electricity system is currently connected to the
| systems of France, Portugal, Andorra and Morocco. The exchange
| capacity of this interconnection is around 3 GW, which represents
| a low level of interconnection for the peninsula. The
| international interconnection level is calculated by comparing
| the electricity exchange capacity with other countries with the
| generation capacity or installed power.
|
| [1] https://www.ree.es/en/ecological-transition/electricity-
| inte...
|
| [2] https://www.entsoe.eu/data/map/
| brohee wrote:
| Edit : I was misreading the confusing Rte site... Entsoe
| actually a lot more readable. France went from importing to
| exporting around the incident.
|
| https://www.rte-france.com/eco2mix/les-echanges-commerciaux-...
|
| https://transparency.entsoe.eu/transmission-domain/physicalF...
| terom wrote:
| That graph doesn't seem to make a very clear distinction
| between historical, real-time and predicted values... I think
| the event happened at 12:30 local time or so.
|
| There seems to be some kind of recurrent daily pattern where
| the French - Spanish interconnect switches from Spain ->
| France imports to France -> Spain exports at around that
| time, and then back again in the late afternoon.
| leomca wrote:
| Yes, France was importing 884MW from Spain at 12:45, 198MW
| at 13:00, no net flow at 13:15, and Spain has been
| importing up to 265MW since:
| https://transparency.entsoe.eu/transmission-
| domain/physicalF...
| luckys wrote:
| I'm in Portugal. Mobile data was slow when it started, now it's
| much better.
|
| The sunny weather is very inviting outside for someone with the
| day off :-)
| pmlnr wrote:
| > "I currently don't have any internet service and just EUR15 in
| my wallet - I can't withdraw any money from the ATM," she added.
|
| from: https://news.sky.com/story/large-parts-of-spain-and-
| portugal...
|
| This is literally the whistleblowers about cashless society have
| been warning everyone about for well over a decade now.
| leereeves wrote:
| I wonder if stores would be able to sell anything without
| power. My supermarket, for example, only has electronic cash
| registers. And no price tags.
| pmlnr wrote:
| Many situations call for pen-and-paper backups. Giving out
| receipts the old way should, in theory, be a possibility,
| then backfill the computer system later.
| kokada wrote:
| Sure, but if we are talking about backup in outages you can
| also get credit card payments when power is down using
| something like a Stripe Terminal.
|
| This is actually exactly the case that I had in one trip to
| Andorra: the power was down for 2 hours while we were
| choosing equipament for skiing. The shop had no issues
| getting our orders done though, because they just manually
| filled the orders with pen-and-paper and did the payment
| with a credit card terminal connected to a smartphone.
| dsr_ wrote:
| If your building has a power outage, that works.
|
| If your city has an extended power outage, the cell nets
| could easily be down as well.
| kokada wrote:
| In my experience, for the 4G/5G network to be down
| something really serious must be happening. I had long
| power outages (more than a few hours, in some cases even
| days) that affected multiple regions in places that I
| lived before that still had working cell network. I
| assume cell networks have backup power and preferential
| usage of the power grid, but I am not a specialist.
|
| And I am not saying that you shouldn't accept money as
| backup, of course you should. But what I am saying is
| that you can still accept credit cards even during most
| power outages.
|
| Same as Software Engineer, it is impossible to have
| perfect, 100% reliability, but it doesn't mean we can't
| improve from 99% to 99.9%, for example, to have a better
| service.
| trollied wrote:
| Used to work for a telco. Cell sites have battery backup.
| Some have generators. Any fibre repeaters also will, as
| do any radio based backhaul sites. The HLR/core network
| etc will run indefinitely due to generators & strict fuel
| supply contracts for said generators.
| XorNot wrote:
| If a city has an extended power outage such that the
| battery backed cell network goes down, then everything
| else will be failing too and payments are the least of
| your problems.
|
| Without electricity the water system depressurized, which
| contaminates it. After about a week the sewage pumping
| stations have backed up so the sewer system is starting
| to fail.
|
| Modern cities cannot operate without electrical power
| given their scale and density.
|
| It is bizarre to think the biggest problem is "how do we
| keep a transaction of value?"
|
| Like, just declare an emergency and let business owners
| be reimbursed by the government.
| mrguyorama wrote:
| Credit card processing existed before widespread
| telecommunications infrastructure. Maybe we should
| require payment cards to have raised numbers like they
| used to so the old carbon copy machines continue to work.
|
| Credit cards and payment networks have always explicitly
| supported "Offline" processing like that.
|
| The kind of fraud that system enables isn't really
| common.
| ptsneves wrote:
| In Lisbon's airport they are temporarily back to just
| stamping the passports without the biometric stuff. There
| are already reports that the checks are being lax.
| kylebenzle wrote:
| It's not just stores, pretty much our entire socity runs on
| cheap power, without it the whole thing falls apart.
| mathgeek wrote:
| Our global society runs on a lot of cheap sources of
| necessary inputs. Power is just one of them.
| corbet wrote:
| Many years ago I worked in a Safeway grocery store. We would
| have occasional power failures that would leave the entire
| store dark; we would all be given flashlights to help
| customers find their way out.
|
| The cash registers, though, had backup power, so the store
| could still take their money.
| mytailorisrich wrote:
| Very good point that nowadays many stores rely on barcodes
| and product IDs to get prices, and don't label individual
| items... So even pen and paper to keep track of takings is no
| use since they can't even figure out prices if the system is
| down!
| pmlnr wrote:
| Valid point, but there must be a price at the location of
| the item in the store.
| throw0101b wrote:
| > _My supermarket, for example, only has electronic cash
| registers. And no price tags._
|
| I know someone who works at a supermarket, and (some of?)
| their point of sale (POS) systems have a small UPS that can
| run for a couple of hours to ride through smaller outages.
| myself248 wrote:
| I used to install those systems, and it's a couple
| _minutes_ at best, if they 've replaced the batteries
| recently, which they never have. It'll give them time to
| finish the transaction they're in the middle of ringing,
| then shut down cleanly, because the server in the back room
| _should_ have a bit more battery to keep the database
| consistent.
|
| PoS systems aren't particularly power-hungry, but store
| owners never want to spend an extra cent, so they go with
| the smallest UPS they can manage. (And arguably if they
| went with a big overkill UPS, its after-outage recharging
| power would be larger so you'd be able to put fewer
| registers on a single circuit, so it's not as simple as
| just dropping in a bigger UPS.)
| DebtDeflation wrote:
| I was on a plane during the Northeast Blackout of 2003.
| Landed, got in my car, and attempted to leave the parking
| garage but with no power the automated EZ-Pass payment wasn't
| working nor were the credit card machines. Most people,
| myself included, had neither sufficient cash nor a checkbook
| on hand. Huge logjam of cars. The workers ended up getting
| the old school handheld metal credit card machines that
| created an imprint of the credit card on carbon paper from
| some long forgotten storeroom and using them.
| bilekas wrote:
| > My supermarket, for example, only has electronic cash
| registers.
|
| That's insane to me, in the EU anyway it's not permitted to
| only accept electronic payments..
|
| > Retailers cannot refuse cash payments unless both parties
| have agreed to use a different means of payment. Displaying a
| label or posters indicating that the retailer refuses
| payments in cash, or payments made in certain banknote
| denominations, is not enough.
| bbddg wrote:
| They have to accept cash in the US as well. The post you're
| replying to is just saying that they can't ring anything up
| or accept any payment without power.
| bilekas wrote:
| Ah okay, that wasn't clear to me, I imagined only being
| able to use electronic payment types.
|
| But in this case, an emergency, I would assume someone
| would still know how to take a manual payment receipt!
| kube-system wrote:
| > They have to accept cash in the US as well.
|
| Only in a handful of cities and states. There is no
| federal law requiring businesses to accept cash for goods
| and services.
| daveoc64 wrote:
| >That's insane to me, in the EU anyway it's not permitted
| to only accept electronic payments..
|
| That's not the case. There are individual laws in each
| country that govern this.
|
| https://fullfact.org/online/UK-not-only-europe-country-
| legal...
| bilekas wrote:
| I guess I was taking the recommendation as a regulation,
| my mistake.
|
| Either way, there should be no reason grocery stores
| don't accept cash imo.
|
| https://www.ecb.europa.eu/euro/cash_strategy/faqs/html/in
| dex...
| richev wrote:
| I worked as a cashier at a large UK supermarket when I was a
| student. In our training we were told that if there was an
| outage with the cash registers we should ask shoppers to
| estimate how much their groceries cost and accept what they
| told us. Payment could be by cash or cheque.
|
| Apparently when this had been done in the past shoppers were
| generally honest & relatively accurate.
| jonathantf2 wrote:
| I was in a Tesco when there was a power outage - the self
| checkouts all died (and rebooted into a custon Debian based
| OS with a Chromium front end by the looks of it), but the
| staffed tills still worked and could accept Chip+PIN but not
| contactless.
| mschuster91 wrote:
| > This is literally the whistleblowers about cashless society
| have been warning everyone about for well over a decade now.
|
| Indeed but it stands to reason that this outage will last maybe
| a few hours until the grid has recovered. A nationwide full
| blackout is a scenario that's on a "once a quarter century"
| level, and the last one in 2006 was resolved after two hours.
| It's Europe, not the US - our grids operate on much, much
| stricter requirements and audits on resiliency, hell since last
| year we got _an active warzone_ in the ENTSO-E grid and it hasn
| 't been too much of an issue!
|
| Not much of value will have been lost in the meantime. The only
| ones who are truly and beyond screwed by such events are large
| smelters and similar factories where any prolonged downtime
| leads to solidification of the products which, in extreme
| cases, require a full reconstruction.
|
| As for "I can't buy eggs in a supermarket now"... lol. People
| need to learn to chill down a bit. You won't die from having to
| wait a few hours to be able to buy the eggs.
| kortilla wrote:
| > our grids operate on much, much stricter requirements and
| audits on resiliency,
|
| None of that changes the difficulty of a black start. If
| there is a full outage, it will take a while to get going.
| raverbashing wrote:
| Yes
|
| But honestly dark starts are the kind of boomer self-made
| problems that'll just have to work around
|
| Whoever built a solar grid inverter without the capacity
| for dark start needs a stern talking to
| mpweiher wrote:
| As in the German government and most of the political
| class...
| mschuster91 wrote:
| Germany has dozens of links to its neighbours. We don't
| need much in terms of "black start" capacity, that's just
| pointless fearmongering by fossil fuel and/or prepper
| propaganda sites.
|
| As long as even a single link to any of our neighbours is
| up and running, it can be used to start the rest of the
| grid - which is exactly what was done in the 2006 outage
| and why that one took barely two hours to be resolved.
| The only truly screwed country at the moment is Portugal
| because all their grid links run through Spain.
| immibis wrote:
| The grid is not just an aggregation of individual sources
| and sinks; it takes active effort to keep them all
| working together in a useful way without just collapsing
| again into another cascading failure. For that reason,
| your solar inverter doesn't come on until the grid
| operator wants the solar inverters to come on in your
| section of the grid.
|
| It's tempting to think of the grid as something grid
| operators _control_ , feeding power from point A to point
| B, but the grid is actually largely uncontrolled - the
| power just flows wherever it wants to - and the only
| controls they have are turning on and off generators,
| adjusting their throttle, disconnecting loads (rolling
| blackouts) and sometimes opening circuit breakers (though
| this is not normally useful). They don't even have
| precise real-time _monitoring_ of the whole grid - only
| specific measurements in specific locations, from which
| the rest is estimated using lots of maths (which is how
| _you_ would design it too, if measurement devices cost
| $100,000 apiece). That 's why it's not a trivial task to
| keep it working.
|
| However, you're able to have your own, private miniature
| grid, on which you can power your own loads from your own
| generators. It's even possible to do this with solar
| inverters! You will need to specifically seek out this
| capability, and get extra hardware installed, which is
| probably why you don't have it. You need a "transfer
| switch" to definitively disconnect your private grid from
| the main grid when you're using your private grid
| capability - it's not allowed (and not safe, and will
| blow up your equipment anyway if you force it) to just
| feed power onto your local unpowered section of the grid.
| mschuster91 wrote:
| > None of that changes the difficulty of a black start.
|
| That's the beauty of the European grid: it is not a black
| start event for Spain, at least as long as even a single
| link to any of the neighbouring countries is available.
| cesarb wrote:
| The difficulty is still the same: for each piece of the
| puzzle (substations, high voltage power lines,
| transformers, generators, loads), they have to wait until
| enough power is available at the connection point, and
| carefully manage the power balance so it's neither too
| much nor too little (and if they get it wrong, things
| trip offline and they have to start all over), while the
| grid is in a degraded state (often meaning no alternative
| paths, so a single fault can set them back several
| steps). The only difference being that the black start
| "generators" are these still working international links
| (which could be very far from the important parts of the
| national grid).
|
| It might be faster to instead black start several
| independent power islands in parallel, and connect them
| together as a final step. At least in my country
| (Brazil), that's how it's done for large-scale blackouts,
| even when some of the country still has power; it was
| done that way for the partial blackout in 2023, and
| there's a written procedure on how to do it (which is
| available on the operator website, if you know where to
| look). In 2023, some areas failed to black start for one
| reason or another, and had to wait for power from the
| outside; other areas managed to black start as expected,
| and were then synchronized with other areas until
| everything came back together.
| ironick09 wrote:
| As long as we're throwing shade at EU vs US, I can't remember
| the last time the US had a nationwide blackout, certainly not
| in my lifetime!
| bombcar wrote:
| The US has three "independent" grids so losing them all
| would be hard. But I believe at times Texas has gotten
| close, and East went pretty dark at some point recently.
|
| CA of course has rolling blackouts for other reasons.
| voidUpdate wrote:
| 2021 was pretty bad in Texas, IIRC
| mrguyorama wrote:
| 2021 would have been a non-event if people in Texas
| weren't propagandized about some nonsensical
| "Independence" bullshit.
|
| A few more interconnects with the rest of the country and
| it wouldn't have even made the news.
|
| this is after decades of Texans bragging about their
| independent power supply. Many Texans still believe
| outright lies about the blackout, like it being "caused"
| by green energy sources, which was false.
|
| It was caused by free market participants not spending
| capital to harden their network. Solar panels and Wind
| Turbines work great in the cold climate of Canada.
|
| The storm that caused such a problem is a once every ten
| years storm. The grid companies all should have foreseen
| this with even minimal investment in planning. They
| didn't, because that's less profitable, and the
| "regulator" in Texas has no ability to punish them for
| pinching pennies on reliability and resilience.
|
| Free Market at work baby!
| throw0101a wrote:
| > _As long as we're throwing shade at EU vs US, I can't
| remember the last time the US had a nationwide blackout,
| certainly not in my lifetime!_
|
| Talking about "national" in the sense Spain (pop. 48M,
| 506,030 km2) is roughly equivalent to a few US states. A
| similarly (population/area) sized outage occurred a couple
| of decades ago:
|
| * https://en.wikipedia.org/wiki/Northeast_blackout_of_2003
|
| North America is organized in regional grids:
|
| * https://en.wikipedia.org/wiki/North_American_power_transm
| iss...
| pmlnr wrote:
| US nationwide would be the whole of Europe category,
| which, I don't believe had ever happened.
|
| Texas, on the other hand, which is easily the size of a
| country...
| anonfordays wrote:
| Just verified: Texas is larger than Spain and Portugal
| combined!
| pjc50 wrote:
| US, no, Texas and California are .. not doing so well.
| immibis wrote:
| There was a very famous one in 2003.
|
| As was pointed out, the USA has three independent grids
| (east, west, and Texas) and EU countries are roughly
| comparable to states (except with less federal power). The
| equivalent of a European nationwide blackout would be a US
| statewide blackout, and those HAVE happened, definitely
| within your lifetime if you're old enough to use Hacker
| News, mostly in Texas.
| gwbas1c wrote:
| It was about 7 hours long:
| https://en.wikipedia.org/wiki/Northeast_blackout_of_2003.
| Where I lived it was restored very quickly.
|
| I had a long blackout as a kid during a hurricane in
| 1985. Once it was safe it was repaired rather quickly.
| razakel wrote:
| There's five different grids in North America (Eastern,
| Western, Texas, Alaska, Quebec) so something would have to
| go very wrong for a nationwide blackout.
| PedroBatista wrote:
| Indeed, and that's the main problem: you can't remember or
| know anything.
|
| It's is a known fact that in general the US power grid is
| orders of magnitude less reliable than in Europe. And the
| excuse of "the weather is more extreme" is just that: a
| lame excuse.
|
| Just count the number of American households that have
| generators and/or batteries vs the Europeans if you really
| have an honest desire to know anything about anything.
| nottorp wrote:
| Except this is Spain and Portugal which combined have 1/3
| of the area of Texas, which had a state wide outage
| recently.
| bilekas wrote:
| > Not much of value will have been lost in the meantime. The
| only ones who are truly and beyond screwed by such events are
| large smelters and similar factories where any prolonged
| downtime leads to solidification of the products which, in
| extreme cases, require a full reconstruction.
|
| I think you've left out a few things, I remember doing on
| site work at a pharma company that required some downtime on
| one of their lines and if we went over the allotted time,
| they would be charging us up to 2 million EUR an hour.
| Hospitals and critical services SHOULD have backup generators
| etc, but depending how long this lasts a lot of things can
| become a major problem.
|
| The majority of the cases will be fine, but when there's mass
| confusion and interruption like this, there's always horrible
| stories that come out.
| Calwestjobs wrote:
| most generators in european cities are connected to gas
| utility so it has essentially unlimited capacity without
| land traffic.
|
| edit: and europe has almmost always atleast half a year of
| whole country supply of natural gas in caverns and other
| storage.
| belter wrote:
| And this is a trial for how a Carrington event will look like.
| pmlnr wrote:
| You mean the part that anything above 130nm technology would
| be fried?
| belter wrote:
| Yes. For the moment I just want to see for how long the AWS
| Spain Region will stay up...
| kortilla wrote:
| Unless this extends to many days, the diesel generators
| will cover this easily.
| Calwestjobs wrote:
| most datacenters do not use diesel, but
| gas/methane/natural gas - utility connection, that way
| you do not need to truck/store anything.
|
| and so being located in middle of city you do not want
| ten of thousands of liters of diesel in tanks there.
|
| same applies for luxury high rises in europe, almost all
| if not all of 20+ story buildings built last 30 years
| have them on roofs.
| HeatrayEnjoyer wrote:
| > and so being located in middle of city you do not want
| ten of thousands of liters of diesel in tanks there.
|
| Why? Every gas station stores that capacity.
| Calwestjobs wrote:
| gas stations, at least in europe, have "safety zone"
| around them.
|
| also depends on data center, some datacenters are
| directly in middle of european city. in most dense part
| of city and dense in europe or asia is something else
| then dense in america.
| kortilla wrote:
| This isn't true in the US and would be a huge dependency
| risk. Natural gas networks require electricity to
| operate.
|
| > and so being located in middle of city you do not want
| ten of thousands of liters of diesel in tanks there.
|
| This is based on FUD. This is the case all over the US
| and they don't cause problems.
|
| It's interesting that Europe has taken such a brittle
| approach to infrastructure.
| HeatrayEnjoyer wrote:
| Why 130nm?
| pjc50 wrote:
| I don't think this is true, and I would like to see an
| explanation of how it might be that quantifies the effect:
| are we talking electric field strength or magnetic field
| strength? Modern parts are generally better at ESD than old
| ones, as well.
| megous wrote:
| I just look to Gaza strip as an example how society continues
| operating under extreme adversity from the western powers,
| incl. total lack of electricity distribution for more than a
| year and a half and full blockade of water, food, and
| medicine.
|
| No need for imaginary scenarios.
| megous wrote:
| Eg. https://www.dropsitenews.com/p/palestinians-gaza-
| create-surv...
| seydor wrote:
| If anything, it shows how little use there is for physical cash
| , only as emergency backup
| pmlnr wrote:
| "emergency backup" is a very strong, rational use, and should
| never be neglected.
| mathverse wrote:
| You can use cash but it mostly does not work in case of big
| chains or stores. They need to have access to their SAP/ERP
| software...It is not about payment for a long time.
| ty6853 wrote:
| Running a generator to power a point of sale machine and
| maybe an offsite server is a lot easier than generators
| to power the whole electronic payment chain.
| lpapez wrote:
| Cash is not really relevant to this particular discussion.
|
| When the power is out one cannot pay with cash either - because
| the cash register is offline.
| kortilla wrote:
| A cash register is not required to make cash transactions
| pmlnr wrote:
| Let me introduce you to the physical receipt block which is
| made of paper and can be filled up with a pen, and what is
| still often used in, for example, Christmas markets.
| seydor wrote:
| Many countries require receipts to be sent to the tax
| services instantly. In many cases a long enough power cut
| (days) would render all transactions illegal
|
| (And in many cases you cannot legally pay large amounts of
| money in cash, it has to be electronic)
| jjice wrote:
| I'm sure they'd make an exception.
| inglor_cz wrote:
| Force majeure should apply.
| mytailorisrich wrote:
| How can this even be a legal requirement?
| cbg0 wrote:
| In some countries whenever you print a receipt, a copy is
| also sent to the IRS equivalent of that country.
| Obviously there are events where that can't happen due to
| technical reasons outside of the store's control.
| mytailorisrich wrote:
| Which countries? And, again, I doubt that this is the
| full picture because there are many cases where people
| simply don't "print a receipt" perfectly legally...
| thyristan wrote:
| Germany for example mandates printing a receipt. The
| receipt must be stored in a certified storage inside the
| cash register and is signed cryptographically, including
| the hash of the previous receipt such that there is a
| hash-chain of printed receipts. Therefore each printed
| receipt that the customer takes home (and maybe at some
| point hands in to the tax office for some reason) can be
| used to check the integrity of the cash register storage
| and all prior receipts in the chain.
|
| https://www.lexware.de/wissen/buchhaltung-finanzen/neue-
| rege... https://www.lexware.de/wissen/buchhaltung-
| finanzen/kassenbon...
|
| Many other EU countries have similar regulations, and in
| some cases had them for a long time.
| mytailorisrich wrote:
| Thanks! European red tape madness strikes again... At
| least in France cash registers are not mandatory (for
| now...) so there is a way around this madness.
| District5524 wrote:
| This is a bit more than just red tape madness, it's a
| strategy to make businesses more transparent. This is
| about trying to reduce non-reported transactions and too
| many people dodging their reporting. Even if the rules
| for cash registries and reporting are detailed, a) that's
| not really expensive for businesses - it's easy to
| automate and there are quite a number of competitors; b)
| compared to accounting and tax rules, they are dead
| simple.
|
| Receipts or invoices are the basis for a firm's whole
| economic activity, including the underpinning of their
| financial reporting, their tax burdens etc. And
| businesses failing to provide receipts erodes not only
| the tax base, but also any rights a consumer may have.
| mytailorisrich wrote:
| The thing with red tape madness is that it is always
| perfectly justified.
| blenderob wrote:
| Which countries exactly? I've traveled in UK and France
| and in both countries when the online cash register was
| down, they opened their physical ledger, made an entry
| and gave me a physical handwritten paper receipt written
| with pen and ink. They said they would make enter the
| same data online when the online cash register comes back
| up.
| pjc50 wrote:
| I believe this is France (NF525), but I don't think it
| says "instantly".
| mytailorisrich wrote:
| My understanding is that this (NF525) only applies to
| computerised cash registers (software must be certified
| NF525 compliant), which there is no obligation to use in
| the first place.
|
| So it is perfectly legal to use pen and paper and a cash
| box.
| guappa wrote:
| Source?
| seydor wrote:
| Despite the EU having a directive to not make electronic
| invoicing mandatory, they now allow exceptions that
| introduce mandatory online invoicing. For example greece
| romania italy: https://sovos.com/blog/vat/greece-
| mandatory-b2b-e-invoicing-...
| immibis wrote:
| Most governments are not as stupid as the anti-government
| wingnuts would have you believe. They will not prosecute
| you for taking transactions non-electronically during a
| power outage.
| Thaxll wrote:
| How do you open the cash register for change when power is
| out?
| edhelas wrote:
| With the physical key, everything doesn't need to be
| electronic you know
| nickjj wrote:
| > How do you open the cash register for change when power
| is out?
|
| I've only seen a few but I believe they have springs on
| the inside and roll on little wheels similar to how desk
| draws roll. Most can be opened with a key to trigger that
| event.
| xlii wrote:
| Depending on the country it's not that easy. Maybe
| something changed in the meantime but where I am those
| blocks are prefilled-numbered and stock needs to be
| controlled.
|
| It's not like you can (could?) keep a block ,,just in case"
| and thus many shopkeepers wouldn't even bother in case of
| outages.
|
| Depending where you live a good old trust can be a
| currency. Humans are great when it comes to adaptation, I
| bet I could just write on paper name, CC number and leave
| it on a paper for shopkeeper and everything would resolve
| just fine..
| xp84 wrote:
| I hear you, and in my second job 20+ years ago we were
| still trained (in a cursory fashion) to use physical charge
| slips in case of a POS system network outage, a full power
| outage would strain my imagination for ability to transact
| business at the majority of stores other than small mom and
| pop operations, and tbh even if such an outage had happened
| to me in 2005, I would not expect my department store to
| remain open and conduct sales.
|
| Most retail workers are GenZ and struggle to understand
| what this would even look like because they've never
| conducted any transaction without POS computers (for
| looking up prices, for tallying them, for figuring tax and
| total, and computing change), so even if a dusty manual in
| the stockroom technically spells out a method of ringing
| sales using nothing but pen and paper and maybe a solar
| calculator, I would be surprised if any of the clerks
| working any given day would have the initiative to initiate
| an offline protocol. Most likely the store manager would
| usher customers out, lock up the store, keep the staff for
| 30 minutes to see if it came back on, and then go home.
| michaelscott wrote:
| Store owners just make out paper receipts in this case, like
| businesses used to do back in the day
| bradly wrote:
| The gas station I worked at used a paper bag when offline.
| Good, not great.
| BozeWolf wrote:
| How did the pumps work when there was no power?
| helf wrote:
| He said "offline" not "no power".
|
| Also, _fuel_ station can probably successfully run it 's
| own backup power;)
| kube-system wrote:
| They don't, at least not here in the US. In areas with
| power outages, gasoline stations do not operate.
| guappa wrote:
| They used to have a crank!
| mistrial9 wrote:
| please be more respectful to the manager of that
| establishment!
| bradly wrote:
| No power is different than no internet, so that was
| different. We were a 24/7 store (7-Eleven) and sometimes
| we would close if it was a prolonged power outage. No one
| has 100% uptime, not even 24 hour gas stations.
| InDubioProRubio wrote:
| Cashier scribbles furiously to add to cash register later
| throw0101a wrote:
| > _When the power is out one cannot pay with cash either -
| because the cash register is offline._
|
| Cash registers can be connected to small UPSes to ride
| through smaller outages. You wouldn't need a larger battery
| if all you want to do is ride through a few-hour outage, or
| even a whole business day (8-12 hours?).
| skc wrote:
| Another reason why itcoin is a pipedream too, particularly for
| third world countries
| PhilippGille wrote:
| If an ATM is down, it's down. With Bitcoin you can still use
| mesh networks. Here's an old article about an example:
| https://bitcoinmagazine.com/culture/samourai-and-gotenna-
| ena...
|
| (Samurai wallet is defunct, but the principle holds up)
| peab wrote:
| You can have physical bitcoins - devices that hold a private
| key in a tamper free way. The private key holds a fixed
| amount of BTC
| _heimdall wrote:
| Without power or an internet connection you can't validate
| that the btc hash is unspent, avoid double spends, or
| validate that your taking ownership of it made it to the
| blockchain and was confirmed for a few blocks.
|
| Sure you can transfer the private key from one device to
| another, but (a) you can't know the other person didn't
| retain a copy of it and (b) you would be limited to
| spending the exact amount you have in an existing
| transaction because you couldn't send a transaction to the
| chain that splits it.
| emporas wrote:
| You've to got a reputation system with identities also
| stored on the blockchain, and you accept the transactions
| only from the most trustworthy people. Counterfeit paper
| currency exists, and counterfeit bitcoin (not BTC)
| transactions might exist, but not every person is equally
| suspect of using counterfeit money.
| red_admiral wrote:
| I agree that bitcoin is a poor solution for anyone but a
| small elite. But as long as you have the mobile network on
| diesel generators (which you do in countries where power cuts
| happen regularly), something like M-PESA still works.
| bradly wrote:
| > This is literally the whistleblowers about cashless society
| have been warning everyone about for well over a decade now.
|
| This is how humans are with all catastrophes-there isn't enough
| money until after something really, really bad happens and
| suddenly there is enough money to fix the issue.
|
| NYC is extremely vulnerable to a 9/11 style attack on the fresh
| water aquaducts. Fuller wrote about this all the way back in
| the 60s in _Operating Manual For Spaceship Earth_ :
|
| _Thus under lethal emergencies vast new magnitudes of wealth
| come mysteriously into effective operation. We don't seem to be
| able to afford to do peacefully the logical things we say we
| ought to be doing to forestall warring-by producing enough to
| satisfy all the world needs. Under pressure we always find that
| we can afford to wage the wars brought about by the vital
| struggle of "have-nots" to share or take over the bounty of the
| "haves." Simply because it had seemed, theretofore, to cost too
| much to provide vital support of those "have- nots." The
| "haves" are thus forced in self-defense suddenly to articulate
| and realize productive wealth capabilities worth many times the
| amounts of monetary units they had known themselves to possess
| and, far more importantly, many times what it would have cost
| to give adequate economic support to the particular "have-nots"
| involved in the warring and, in fact, to all the world's 'have-
| nots."_
| ghc wrote:
| > whistleblowers
|
| Using the term whistleblower in this manner is inappropriate;
| actual whistleblowers are individuals who bring to light
| illicit acts by organizations or governments at great personal
| risk.
| trgn wrote:
| Prophets
| ptsneves wrote:
| Peters
| trgn wrote:
| I'm a dummy, but I don't get it :) can you explain?
| croes wrote:
| There isn't much you can buy with cash either if there is no
| electricity.
| jslakro wrote:
| A fire in the south-west of France, which damaged a high-voltage
| power line has also been identified as a possible cause:
|
| https://www.euronews.com/my-europe/2025/04/28/spain-portugal...
| pjc50 wrote:
| How very California.
| InDubioProRubio wrote:
| Warming world is so hot right now..
| mistrial9 wrote:
| you are right, but the emphasis could use a tune-up. In
| California, home of world-leading tech.. there are sensors
| and information networks, extensive electrical power lines,
| heavy equipment and budgets, a lot of dry and dead tress, a
| history of fire. So you see that California in a way is a
| world-quality testing lab. and the way the information
| travels, and the way the information is applied, could also
| be world-quality .. or, world-theater for government
| imbecility..
| Rygian wrote:
| A power outage in 2021 was ultimately traced to a firefighting
| hydroplane that damaged a high-voltage transmission line in
| southern France.
|
| The similarity between that event and this early-on report is
| striking.
|
| [es language]:
| https://www.lavozdegalicia.es/noticia/espana/2021/07/24/aver...
| tejohnso wrote:
| Surely there is more than 0 redundancy so that one power line
| failure would never result in this level of catastrophe.
| jcranmer wrote:
| The 2003 US Northeast blackout was caused by the failure of
| only a few lines that shorted into trees. These line failures
| created grid instability that resulted, ~5 minutes later, in
| most of the Northeast losing power in a cascading failure.
| pixl97 wrote:
| Redundant systems have failures in one path quite often that
| you never know anything about. We get headlines when the
| failures correlate in the same timeslot.
| matkoniecz wrote:
| > one power line failure would never result in this level of
| catastrophe.
|
| Called N-1 criterion. See https://en.wikipedia.org/wiki/Conti
| ngency_(electrical_grid)#...
|
| And it depends. During
| https://en.wikipedia.org/wiki/2006_European_blackout N-1
| criterion was supposed to be holding, in practice not even
| N-0 was holding and network crashed.
|
| Few years ago nearly entire day European network was sitting
| on N-0 due to multiple issues in Poland, caused by a heat
| wave and deeper root causes. There are many power plants and
| power lines where any further issue would cause Europe-wide
| blackout.
| yaantc wrote:
| From Le Monde live feed, RTE (French electricity network
| manager) declared the issue unrelated to this fire.
|
| "Le gestionnaire francais souligne par ailleurs que cette panne
| n'est pas due a un incendie dans le sud de la France, entre
| Narbonne et Perpignan, contrairement a des informations qui
| circulent."
| oersted wrote:
| I'm seeing some reports saying that a significant frequency
| oscillation happened, which triggered automatic shut-downs,
| which cascaded. Could an event like this have that effect?
|
| I suppose it makes sense that it was an automatic shutdown
| rather than infrastructure failing on such a wide area. And
| then once it's shut down, a black-start is a logistical
| challenge as other comments have explained.
|
| I'm also seeing some reports about it being more likely that
| something happened on the east side, somewhere like the Ebro
| valley or north across the Pyrenees. Catalonia seems to have
| been particularly affected, and it's on the path of important
| lines coming from France. High heat at noon could have caused a
| line to fail and short against a tree, which would be similar
| to the 2003 nation-wide outage in Italy.
| wongarsu wrote:
| The grid is supposed to tolerate any single failure, even
| under full load. Of course sometimes the first failure is a
| fire or equipment malfunction and the second failure is a
| planning failure or someone pressing the wrong button.
| AdamN wrote:
| Grey failures are harder for large systems to handle. If a
| chunk goes hard down that's usually easy. Something like
| voltage oscillations that trigger cascading failures in a
| sequence can lead to negative feedback loops that bring it
| all down.
| alexey-salmin wrote:
| Cascade failures are common. The weakest link fails, load
| gets re-distributed evenly, the second-weakest link fails
| and so on.
|
| In theory [a flawed one] you've had enough spare capacity
| to survive N failures and N+1 failures are statistically
| unlikely because p^(N+1) is close to zero.
|
| On practice [or with a better theory] you can't multiply
| probabilities in a grid system because random variables
| aren't independent. 30% spare capacity can go to -100% in a
| second.
| andy99 wrote:
| Older readers may remeber the Northeastern blackout of 2003 in
| the US and Canada that was caused by cascading overloading, I
| believe originally triggered by high loads due to hot weather and
| poor vegetation maintenance under lines.
|
| I was an an adjacent area at the time and iirc we were saved by
| our nuclear operator releasing some insane amount of steam to
| bring the supply down and avoid more overloading.
|
| https://en.m.wikipedia.org/wiki/Northeast_blackout_of_2003
| ta1243 wrote:
| Older readers would remember the _1965_ blackout. 2003 was only
| 22 years ago
| lwo32k wrote:
| That was a mad few days. We were stuck on campus. Water ran out
| by the next day. And students were using water from the campus
| fountain.
| 7373737373 wrote:
| Good documentary on the ice storm of 1998
|
| https://www.youtube.com/watch?v=0ccTzHBUsYQ
|
| > In January 1998, Montreal and the region surrounding it was
| hit by the most disastrous ice storm ever recorded: more than
| four inches of ice entombed an area larger than the State of
| Florida, causing trees and power lines to collapse on an
| unprecedented scale, leaving millions in the dark without heat
| (some for up to four weeks). 35 people died and damages
| totalled more than $5 billion making it the worst natural
| disaster in Canadian history.
|
| 4 out of 5 power lines to Montreal went down
| RALaBarge wrote:
| I have such great memories of the blackout. I was in college at
| the time, so obviously indestructible.
|
| The first night, we had 2 grills with 10ft high flames roaring
| but I would say in a controlled fashion. The cops quickly asked
| us to not do something dumb like that anymore and we agreed.
|
| Memories of the local party stores selling 40oz's for pennies
| on the dollar, as the outage would last days but the booze
| would not. It didn't really turn into anything bad for us in SE
| Michigan other than a fun story I can tell my internet pals
| about 20 years on.
|
| If I recall correctly, we were also somehow the first house in
| the neighborhood to get power restored...I remember playing a
| Dreamcast with a hacked NES emulator running Rampart 2 player.
| I never made good on it, but I always said I wanted to make a
| shirt that read "I blacked out in the black out of '03"...which
| is probably for the best.
| timewizard wrote:
| And a computer system failure.
|
| Full grid failures are almost certainly never down to a single
| cause.
| pmlnr wrote:
| For those not aware how interconnected the European Grid is:
| https://storpower.co.uk/the-day-europes-power-grid-came-clos...
| paganel wrote:
| > Spain's electricity grid operator Red Electrica has confirmed
| power outages across the country.
|
| That's why you still need a strong diesel/diesel-electric
| locomotive fleet, imagine if Spain had been right in the middle
| military mobilization and military materiel transport, an event
| like this one would have stopped then dead in the tracks had they
| been relying only on electric locomotives.
| fer wrote:
| No, the armed forces have their own fuel locomotives. I imagine
| any sizeable military does the same thing. Plus most (all?)
| cargo trains still use fuel. 100% electric is just passenger
| traffic.
|
| https://www.outono.net/elentir/2021/10/23/the-spanish-army-r...
| paganel wrote:
| Maybe I'm looking at the wrong article, but there's this:
|
| > For journeys outside the base, *the Army uses Renfe
| locomotives*.
|
| which, in my understanding, means that in order to move
| military materiel (to the borders with France, let's say, or
| to the closest sea-ports most probably) and tens to hundreds
| of thousands of mobilised men the Spanish Army does indeed
| rely on Renfe locomotives, i.e not on their own.
| davedx wrote:
| [flagged]
| belter wrote:
| They probably did not patch their firmware in years and have
| their SCADA systems live on Shodan with default
| passwords...allegedly...
| api wrote:
| I remember years ago someone scanned the Internet IPv4 space
| for open unpassworded VNC servers. Many of them looked
| disturbingly like industrial control systems.
| cafeinux wrote:
| Just take a look at https://infosec.exchange/@shodansafari.
| A lot still look like that.
| belter wrote:
| Even the water control system for a big French Dam:
|
| "115 batshit stupid things you can put on the internet in
| as fast as I can go by Dan Tentler" -
| https://youtu.be/hMtu7vV_HmY
| gchamonlive wrote:
| [flagged]
| api wrote:
| It's not Musk fetishism. It's network effects. Most people
| hate X and hated Twitter but people use them because people
| use them.
|
| This should show people just how powerful network effects
| are. They are legitimately a force of nature.
| frank_nitti wrote:
| I strongly agree, and to me this supports the argument for
| heavier regulation (or any really)
| OgsyedIE wrote:
| This complaint needs to be passed to European regulators at
| either BEREC, ACER or DG Connect, since using a loginwalled
| site is a violation of EECC directive 2018/1972. Ideally they
| can just designate 1 approved site that all utilities should
| harmonize on using for their status updates.
| boxed wrote:
| Is this a joke? Because that seems like exactly the wrong
| thing, and such a system would itself be a bottleneck for
| downtime issues.
|
| Using twitter has the huge advantage that spikes in users in
| Spain for checking this stuff is a rounding error in the
| normal traffic so is very unlikely to take down the status
| page.
| matthewdgreen wrote:
| Aside from the login issue, Twitter has outages too. It
| really isn't really that hard to replicate posts across
| multiple services in 2025.
| ABS wrote:
| no one would/should prevent companies from using _also_
| private channels like X, FB, Instagram, etc but enforcing a
| public channel that doesn 't require private citizens to
| register, accept T&S and share their private data with 3rd
| party, unrelated private corporations to be informed of
| critical, public safety information would be helpful.
| teekert wrote:
| They may not be aware. We had Dutch fire depts do the same, it
| used to be possible to see tweets without logging in.
| scarlehoff wrote:
| While I don't particularly like twitter (X), in a situation
| like this it probably has a better reach than the website of
| REE.
|
| That said, twitter should allow for official profiles and
| organizations to have their tweets (xs?) made public.
| deanc wrote:
| You're kidding right? The overwhelming majority of people do
| NOT use X or have accounts there. We are in a bubble. (Not
| making a Musk point here as it was the same before the
| "exodus")
| brohee wrote:
| They did post on BlueSky, but not every update so I assume
| someone is cut and pasting manually...
|
| https://bsky.app/profile/redelectricare.bsky.social
| pcblues wrote:
| A line that disturbed me with just how widespread the outage is.
| (And also people will die from it in all sorts of weird ways)
|
| "In an update, Spanish power grid operator Red Electrica says
| it's beginning to recover power in the NORTH and SOUTH of the
| country."
| megous wrote:
| On the bright side. People die from electricity and from
| operating electricity powered machines, too. So those will be
| spared. :)
| Wobbles42 wrote:
| I've seen the assertion made that we can statistically
| measure how many people died due to (for example) a heat
| wave, but we can't say for sure which ones.
|
| I'd imagine something similar applies here. You'd have some
| number of deaths specifically attributable to lose of power,
| plus countless other deaths caused or prevented in non-
| obvious ways. This might be visible at a high level as a
| statistical outlier in the total number of deaths during the
| time period of an outage.
| carlos-menezes wrote:
| Getting two network bars in Madeira. Seems to be affecting 4G/5G
| now.
| red_admiral wrote:
| I'm going with: never attribute to malice what can be explained
| by ... an incredibly complex system that can fall over even if
| no-one's being stupid. I would want very strong evidence before I
| believe this is an attack.
|
| There is precedent for major power outages, a huge majority of
| which are not malicious:
| https://en.wikipedia.org/wiki/List_of_major_power_outages
|
| I remember the day when the Swiss railway power network went down
| for a day (in 2005) because one power line was down for
| maintenance and someone pressed the wrong button and produced a
| short circuit somewhere else. It's a bit like the incidents in
| planes were one engine has a problem and the crew shut down the
| other one by mistake.
| reitoei wrote:
| a.k.a. Hanlon's Razor
| https://en.wikipedia.org/wiki/Hanlon%27s_razor
| makeitdouble wrote:
| As far as we know stupidity isn't a primary cause, and local
| authorities are also probably competent at scale.
| fbn79 wrote:
| Not many month ago the Italians trains was disrupted by a
| single nail into a cable:
| https://www.reuters.com/world/europe/italian-rail-services-d...
| belter wrote:
| > were one engine has a problem and the crew shut down the
| other one by mistake.
|
| So frequent it even has its own wikipedia page:
| https://en.wikipedia.org/wiki/Category:Airliner_accidents_an...
| JimBlackwood wrote:
| Or that time in 2003 when a tree fell on a power line in
| Switzerland and all of Italy ended up without electricity.
| hersko wrote:
| I seem to recall a tree falling in Ohio knocked out the power
| to NYC.
|
| https://en.wikipedia.org/wiki/Northeast_blackout_of_2003
| jcranmer wrote:
| That wasn't tree falling in Ohio, that was overloaded line
| sagged and shorted into a tree, compounded with several other
| factors that contributed to the grid instability and the
| inability of the grid operator to realize how unstable the
| grid was.
| scoot wrote:
| "Same difference" to use an Irish turn of phrase. If a
| short to ground takes out power to an extensive area does
| it mater what touched what?
| eldaisfish wrote:
| Yes, it absolutely does.
|
| A tree falling can be addressed by vegetation management
| and trimming. A power line sagging because of excess heat
| is operator error.
|
| These are not remotely the same.
| spongebobstoes wrote:
| It seems like both are a monitoring problem and both have
| a maintenance fix. A tree falling is more forgivable
| though.
| mikepurvis wrote:
| I think the point is that neither of those events should
| take out a whole city; the design is such that there is
| considerable redundancy in the system.
| Matthyze wrote:
| Indeed. For instance, power grids ideally operate with
| N-1/N+1 redundancy, i.e., the disablement of any single
| power line should not cause a cascading failure.
| 0_____0 wrote:
| The proximal cause was a race condition in a piece of
| software, so as usual it comes back to you keyboard
| ticklers!
| pavel_lishin wrote:
| > _keyboard ticklers_
|
| I hate this term, and look forward to using it all the
| time.
| conductr wrote:
| Relying on vegetation management along millions(?) of
| miles of lines is the definition of thinking you're in
| control without even knowing what risks exist. I see this
| every year in Texas, they trim back trees thinking they
| have solved the problem - except - they are allowed to
| trim them back I think 6-8 feet from the lines, ignoring
| the fact the trees are 20-40 feet taller than the lines.
| Nobody seems to see the risk that they left there but
| it's quite obvious to me, we're not eliminating risks at
| all. We're minimizing them at best, and even then the
| labor required to trim all the branches around lines
| every year is impossible to keep up with from an economic
| feasibility standpoint.
| applied_heat wrote:
| They trim back what poses the highest risk with the
| budget they have.
|
| As you have identified A wider right of way costs more.
|
| Usually for lines above some voltage, perhaps 200kV, the
| cost of an outage due to a tree strike outweighs the cost
| of additional vegetation management so they will clear
| the right of way wide enough that no tree can fall and
| hit the power line.
|
| Around here for 130kV the right of way is still as narrow
| as it can be and we annually take down the riskiest trees
| as this is the best for our budget, which is not
| unlimited.
| genewitch wrote:
| Concur, they have huge fire breaks they put high tension
| lines in, where the easement is easily 100' to either
| side of the lines. This is in Louisiana. When there's a
| hurricane these lines still break, but not because of
| trees. During heavy storms, smaller trees hit smaller
| (480? Some KV?) Lines that go along highways to
| residences. A high tension line down means a few days
| without power (about 3-4), a tree on a lower voltage line
| is usually fixed within hours.
|
| They go through and remove damaged trees near the
| easements of the highway lines, as well as branches that
| could break into lines.
|
| As an aside we lived on the same section of grid as the
| sheriff, and our power was rock solid for a few years,
| then he left office and now our power is better than
| average (at least better than our neighbors who's power
| line cones from the other direction).
| quickthrowman wrote:
| > During heavy storms, smaller trees hit smaller (480?
| Some KV?)
|
| Residential distribution voltage varies by utility but
| it's usually in the medium voltage range, 5kV to 35kV,
| with 13.8kV being common.
| vel0city wrote:
| Both of these are operator error issues.
| ksenzee wrote:
| Is "same difference" more common in Ireland than in
| Britain? It's certainly common in the US and seems to
| have originated there: https://english.stackexchange.com/
| questions/277142/etymology...
| closewith wrote:
| Definitely not an Irish turn of phrase. An unfortunate
| import.
| Wobbles42 wrote:
| The tree had malicious intent. You can't convince me
| otherwise.
| riskable wrote:
| They've had a grudge against humanity since the fall. Ever
| since--once a year--they send a clear message, "leaf" but
| do the humans listen? No.
| jpmattia wrote:
| Can you blame them? Just look at how they've been treated
| in the last few hundred years.
| HenryBemis wrote:
| I vaguely remember an episode of some thriller series
| when I was too young to be watching it, perhaps Twilight
| Zone (or similar) where someone was hearing screaming,
| they went out a d saw trees being cut.
| Ayesh wrote:
| A monkey took out the whole grid in Sri Lanka this year
|
| https://edition.cnn.com/2025/02/13/asia/sri-lanka-power-
| outa...
| hirako2000 wrote:
| If that's all it took to turn Srilanka off.Even if that
| country lags behind western Europe, what would it take for
| a few humans who intentionally cut off power.
|
| Trying to stay on the facts, this incident is likely
| accidental but some people even the very workers at energy
| companies could send a message for, I don't know. A pay
| raise?
| pjc50 wrote:
| https://en.wikipedia.org/wiki/Winter_of_Discontent
|
| Yes, you can try to hold the country hostage for your
| salary by going on strike, but that's the sort of thing
| that results in very energetic union-busting.
|
| Actually sabotaging the infrastructure would result in
| terrorism charges, or at the very least the JSO
| treatment.
| Ey7NFZ3P0nzAe wrote:
| I mean i'm pretty sure there are laws in place everywhere
| to avoid a single pay raise strike doing nation wide
| catastrophic events. Hospitals for example are usually
| limited in how they can strike.
| genewitch wrote:
| That's assuming the people who want to strike care about
| having a job afterward. I know the state hospitals stop
| loss people when there's a disaster, so I keep abreast of
| predictable disasters and tell my wife and friends "hey
| you're sick, go get a doctor's note for the rest of the
| week" and then they're not stuck at the hospitals for
| days during a disaster.
|
| Not that type of hospital.
| MisterTea wrote:
| > what would it take for a few humans who intentionally
| cut off power.
|
| https://en.wikipedia.org/wiki/Moore_County_substation_att
| ack
|
| Unsolved to this day.
| skrebbel wrote:
| Isn't it amazing? In movies, evil terrorists take out the
| grid all the time (or threaten to), to terrible
| consequences, and they need genius hackers to do so. In
| reality, it's easy peasy and still, people hardly ever do
| it (and nobody (ish) dies if they do). Humans are good!
| htrp wrote:
| > The animal had come into contact with the transformer at
| the station, disrupting supply to the entire country. There
| were no immediate details on whether the monkey survived
| the incident.
| genewitch wrote:
| I'm sure the constituent atoms of the monkey survived.
|
| I saw a video the other day of some human running and
| jumping on a transformer after hopping a fence, dancing
| on the transformer in a distribution site.
|
| It ended as you'd expect, a bright light, a lot of curse
| words from the camera operator who was probably blinded
| temporarily.
|
| Electricity does not care.
| casenmgreen wrote:
| That's a horrible story.
|
| I can't believe and I'm horrified someone actually
| published such a video.
| bobthepanda wrote:
| There was also a race condition in some management software
| that hadn't tripped in over thirty years of usage.
| lenerdenator wrote:
| We live in a society filled with systems that can have "normal
| accidents".
| Keyframe wrote:
| I'm going with AGI awakening in some lab today.
| chaosbolt wrote:
| Sorry I couldn't resist:
| https://www.youtube.com/watch?v=jnDk8BcqoR0
| mattkevan wrote:
| A few months ago I was hit by a blackout literally the second
| I was about to start delivering a company-wide talk on AI.
| Everything went out - Internet, mobile networks, street
| lighting, the lot.
|
| We're a remote business so it seemed like I'd just rudely
| dropped off the call, but as everything was down I couldn't
| let people know what'd happened.
|
| Apparently it was caused by botched maintenance work
| affecting 30,000 houses, but the timing was so perfect I
| can't help thinking it was because our AGI overlords really
| didn't want me to deliver that talk for some reason.
| padjo wrote:
| "induced atmospheric vibration" on 400KV lines is the current
| theory
| CGMthrowaway wrote:
| Probably didn't help that before the outage hit, Spain was
| running its grid with very little dispatchable spinning
| generation, and therefore not much inertia.
|
| Solar PV/thermal + wind: ~78%
|
| Nuclear: 11.5%
|
| Co-generation: 5%
|
| Gas-fired: ~3% (less than 1GW)
| pjc50 wrote:
| > Solar PV/thermal + wind: ~78%
|
| This is (a) incredibly impressive to achieve and (b)
| definitely the point at which the battery infrastructure
| needs to catch up in order to reduce the risk of such
| incidents.
| jandrese wrote:
| I was just thinking that grid scale batteries could be
| extremely useful in a black start situation.
| JumpCrisscross wrote:
| > _very little dispatchable spinning generation_
|
| Makes the case for favouring flywheels over batteries.
| GeoAtreides wrote:
| flywheels are batteries, of a different kind (and
| density) than lithium or lead ones.
| vel0city wrote:
| They're usually almost more capacitators than they are
| batteries from an electrical standpoint. They're there to
| rapidly smooth out the smaller jitters.
| loeg wrote:
| You know what he meant. Respond to the charitable
| interpretation of the comment, etc.
| tonyedgecombe wrote:
| That's part of the UK's plan:
|
| https://www.proactiveinvestors.co.uk/companies/news/10574
| 63/...
| padjo wrote:
| Same all over, apparently we had the biggest in the world
| a few years back, I'd imagine it's been overtaken by now
| https://www.siemens-
| energy.com/global/en/home/stories/irelan...
| JensRantil wrote:
| I would _love_ to understand more about how this atmospheric
| state impacted the vibration of power lines. Sounds exotic.
| williamscales wrote:
| I'm no expert but I think it has to do with this effect:
| https://en.wikipedia.org/wiki/Vortex-induced_vibration
|
| See also: https://en.wikipedia.org/wiki/Aeolian_harp
|
| Edit: might be a completely different kind of oscillation
| than I was thinking of. https://news.sky.com/story/spain-
| portugal-power-outage-lates...
|
| So I would also like to understand how this works :)
| alickz wrote:
| >an incredibly complex system that can fall over even if no-
| one's being stupid
|
| I only have a layman's understanding of power grids, but I
| thought they were incredibly hardened, with backups and
| contingencies in depth
|
| Are the grids at this scale really this brittle? Would there be
| a death toll from this?
|
| I also wouldn't blame malice without corroborating evidence
| slightwinder wrote:
| > but I thought they were incredibly hardened, with backups
| and contingencies in depth
|
| Some are harder than others, and some have random flaws which
| nobody can really predict.
|
| Spain seems in the transition to renewables, so it's possible
| that they have some flaws because they are still in the
| process, or because it's something which never happened
| before and is unknown territory. Also, Spain had some
| economic problems in the last decade, maybe someone build to
| cheap or was even cheating somewhere.
|
| > Are the grids at this scale really this brittle? Would
| there be a death toll from this?
|
| Hospitals should have backup-systems. Traffic should be able
| to stop in time. I guess the most problematic parts are
| people stuck in elevators and other spaces which only open
| electrical, as also the loss of cellular phone-connections
| for calling helpers.
| bonzini wrote:
| Hardening focuses first on not damaging equipment and second
| on providing energy. If things go wrong quickly enough you
| don't have time to react, because after a power plant
| disconnects you get sudden bumps in load that can trigger a
| chain failure near the original point. The last time it
| happened in Europe was in 2003, which isn't too shoddy.
| geophile wrote:
| But there is _so_ much highly motivated malice right now.
| red_admiral wrote:
| Update: BBC news says something something hot weather, EU
| council says it doesn't look like a cyber attack.
| LeonM wrote:
| As confirmed by Antonio Costa (President of the European
| Council) on X:
| https://x.com/eucopresident/status/1916859055546544517
| switch007 wrote:
| Hot weather? Huh? It's spring
| seb1204 wrote:
| Spring is nowadays the best time for the grid and
| renewables. Low consumption as heating period is coming to
| and end, air conditioning is not required yet and the sun
| can be strong already providing lost if solar power output.
| I can imagine with a grid and a huge growth of solar
| generation that there are a myriad tripping stones that can
| upset the balance.
| brewdad wrote:
| For areas utilizing hydro, spring tends to bring the
| strongest annual flows due to snow melt.
| m000 wrote:
| Might as well mean that they suspect the culprit, but they're
| not going to do anything anyway for this specific culprit, so
| let's all pretend it was an accident. Why make yourself look
| like a fool?
|
| They already became a laughing stock once for promising the
| "strongest possible response" for the Nord Stream 2 sabotage
| [1].
|
| [1] https://www.reuters.com/business/energy/eu-sees-sabotage-
| nor...
| guerrilla wrote:
| > I'm going with: never attribute to malice what can be
| explained by ... an incredibly complex system that can fall
| over even if no-one's being stupid. I would want very strong
| evidence before I believe this is an attack.
|
| I think we should prepare for the worst though. It's wrong to
| assume it's _not_ an attack too, and until we can conclude it
| 's not an attack we should be prepared to deal with the
| possible consequences and act accordingly.
| rawgabbit wrote:
| Having lived through the Texas electricity fiasco in 2021, I
| would blame cost cutting, the reckless drive for "efficiency",
| and maximizing shareholder value.
|
| In Texas, the electric providers cut staff and maintenance to
| maximize shareholder value. They will not have redundant
| systems and redundant plants out of the goodness of their
| hearts. The Texas marketplace actually allowed them in the odd
| event of an outage to charge astronomical spot prices thinking
| this will incentivize them to have redundant systems. This was
| a foolish fantasy.
|
| Now in Texas, discussion of how to cost share redundancy have
| taken place. But no one wants to pay for it.
| https://www.texastribune.org/2023/03/01/texas-power-market-p...
| TOMDM wrote:
| Show me the incentive and I'll show you the outcome.
|
| If you want a no fail grid you need to incentivise a no fail
| grid.
| tux3 wrote:
| You can see the crash on the ENTSO-E live data:
| https://transparency.entsoe.eu/generation/r2/actualGeneratio...
|
| Three quarter of the production disconnects from the grid between
| 12:30 and 13:00, with only a bit of solar and onshore wind
| sticking around.
| leomca wrote:
| Spain loses around 15GW of demand at the same time:
| https://transparency.entsoe.eu/load-domain/r2/totalLoadR2/sh...
|
| I don't think we're able to tell from the data if one is the
| cause of the other, are we? Since if production was lost, load
| would have to be shedded to balance the grid, and if load was
| lost (e.g. due to a transmission failure), production would
| have to be disconnected to balance the grid.
| pjc50 wrote:
| Report on a smaller event in the UK: https://www.ofgem.gov.uk
| /publications/investigation-9-august... (see linked pdf)
|
| That started from a combination of a lightning strike and
| generator trip, but turned into a local cascade failure as
| lots of distributed generation noticed that the frequency was
| under 49Hz and disconnected itself. I suspect the Spanish
| situation will be similar - inability to properly contain a
| frequency excursion, resulting in widespread generator trips.
|
| (I suspect this is going to restart a whole bunch of acrimony
| about existing pain points like grid maintenance, renewables,
| domestic solar, and so on, probably with the usual suspects
| popping up to blame renewables)
| scoot wrote:
| Is 50Hz really that important these days - i.e. more
| important than maintaining power? (Honest naive question.)
| pmlnr wrote:
| Yes. https://storpower.co.uk/the-day-europes-power-grid-
| came-clos...
| idiotsecant wrote:
| Frequency is health of the grid. If it's an _entire hz_
| off of where it should be something is massively wrong.
| Most generating equipment will trip on a deviation that
| bad.
| pjc50 wrote:
| Frequency is effectively more important than voltage. Or
| rather it collapses first. Frequency reduction for a
| rotary generator means that more energy is being taken
| out than the rotary shaft energy is being put in, so it
| is effectively an early warning that voltage is about to
| drop.
|
| Frequency coordination is absolutely critical, via phase
| coordination. A large generator must not get
| significantly out of phase. So frequency going out of
| spec triggers the generator to "trip" (disconnect).
| immibis wrote:
| As I understand it, it's not possible for a generator to
| get out of phase with the grid. Sudden changes in the
| grid phase instantly and catastrophically break
| generators, power lines, transformers, and so on, but
| that can only occur when you connect two grids together
| without synchronizing them. Which implies there are two
| separate grids to connect.
|
| I don't know what specific threat is addressed by
| tripping generators offline when the frequency deviates
| by 1 Hz. Are they so mechanically fragile that is already
| damaging to them, or is it a precautionary measure
| because that kind of instability is likely to precede
| sudden frequency or phase jumps that are damaging?
| inglor_cz wrote:
| Absolutely. The frequency needs to be maintained between
| 49.8 and 50.2 Hz, or the situation starts spiraling out
| of control freakishly fast.
| Scoundreller wrote:
| What I wanna know is if I could/should have a plug-in box
| that detects the frequency is drifting and the grid is
| unstable ---> high risk of imminent failure.
|
| Might even give you clues about something big tripping
| offline.
|
| Probably lots of false alarms, but if it an outage is
| particularly bad for you, good to know as soon as the
| system operators do.
|
| Obviously over the internet could work too, but who
| wouldn't love their own box?
| baq wrote:
| no worries, the energy utility does that for you. they
| care, because if they don't, their hardware bursts into
| flames.
| Denvercoder9 wrote:
| > What I wanna know is if I could have a plug-in box that
| detects the frequency is drifting
|
| Yes, it's not that hard. There's smart meters and plugs
| that have frequency measurement built in.
|
| You can even do it with an audio cable:
| https://halcy.de/blog/2025/02/09/measuring-power-network-
| fre...
| 0cf8612b2e1e wrote:
| Old school electric clocks used to (still?) keep time
| with the grid frequency. Guaranteed to always have N
| cycles per day. So if things are gradually failing, you
| might be able to see your clock keeping worse time.
| vel0city wrote:
| > Probably lots of false alarms, but if it an outage is
| particularly bad for you, good to know as soon as the
| system operators do.
|
| When the phase gets pulled down hard like what nearly
| happened in Texas and what probably happened here, it'll
| go from looking like the background noise of phase
| changes to catastrophic in just a few seconds. It isn't
| like you'll get warning an hour ahead of time. You'll
| probably notice your computer monitor going dark before
| your grafana graph refreshes.
| floatrock wrote:
| If you have a switching power supply brick for your tiny
| USB device, not really.
|
| If you have a large spinning inertial mass like a factory
| motor or a power generation turbine, it's extremely
| important. Imagine a manual car transmission, but there's
| no slip-clutch, you need to perfectly align engine with
| the wheels rotating at 300mph, and the inertial mass
| you're up against if it's not perfectly synchronized is a
| freight train.
|
| That's why generators trip offline in a blackout cascade
| if the frequency deviates out of spec. The alternative is
| your turbine turns into a pile of very expensive shiny
| scrap metal.
| baq wrote:
| if a piano is out of tune, you'll hear it, perhaps even
| flinch.
|
| if a high power transformer goes out of tune... it melts
| or blows up (or both); it'll try to shut itself down
| before that. getting it back on becomes a problem if
| other transformers do the same thing, which is apparently
| what happened in the whole country.
| MisterTea wrote:
| Yes. The speed of AC electrical machinery is determined
| by the frequency and number of poles. This means
| generators connected to each other in parallel MUST
| rotate in synchronism. If the frequency changes the
| generator is now producing power out of phase which can
| cause a fault (a short circuit) and possibly damage the
| machinery.
|
| Its like a three legged race. You and your partner have
| to run in synchronism. If either of you slows down or
| speeds up, the other can trip and fall over taking both
| of you out.
| quickthrowman wrote:
| Yes, maintaining frequency is the most important thing,
| provided you want your rotating power generation
| equipment not to explode.
|
| In reality, power generation equipment will disconnect
| itself if the frequency is too low/high to avoid
| catastrophic failure.
| cesarb wrote:
| > probably with the usual suspects popping up to blame
| renewables
|
| Renewables were a factor in the blackout here in Brazil a
| couple of years ago: the models used by the system operator
| did not correspond to reality, many solar and wind power
| plants disconnected on grid disturbances quicker than
| specified. That mismatch led the system operator to allow a
| grid configuration where a single fault could lead to a
| cascade (more power was allowed through a power line than
| could be redistributed safely if that power line shut off
| for any reason), and that single fault happened when a
| protection mechanism misbehaved and disconnected that power
| line. The main fix was to model these solar and wind power
| plants more conservatively (pending a more detailed review
| of their real-life behavior and the corresponding update of
| the models), which allowed them to correctly limit the
| power going through these power lines.
|
| If you want an excruciating level of detail, the final
| 614-page report is at https://www.ons.org.br/AcervoDigitalD
| ocumentosEPublicacoes/R... (in Portuguese; the main page
| for that incident is at
| https://www.ons.org.br/Paginas/Noticias/Ocorr%c3%aancia-
| no-S...).
| xenadu02 wrote:
| This is why solar requires redundant data links to the
| operator in CA now. Mine has it. During instability they
| can command my system to accept larger voltage and
| frequency excursions than would normally be allowed to
| prevent a "pile-on" where an excursion causes solar to make
| the problem worse.
| terom wrote:
| Portugal has an even bigger relative drop in load, from 5852MW
| at 11:00 hours -> 613MW at 13:00 hours - these seem like 1 hour
| averages.
|
| [1] https://transparency.entsoe.eu/load-
| domain/r2/totalLoadR2/sh...
| bombcar wrote:
| Red Electrica - red is Spanish for network, not the color (roja)
| AndrewDucker wrote:
| Regardless of the source of this specific issue, I suspect
| investment in infrastructure security and resilience is about to
| ramp up.
|
| Calling attention to how fragile many of our critical systems are
| is almost certanly a net-positive in the long run.
| smartbit wrote:
| Seems to have briefly expanded into France and even Belgium had
| issues.
|
| https://www.euronews.com/my-europe/2025/04/28/spain-portugal...
|
| Edit: BBC reports it now also
| https://www.bbc.com/news/live/c9wpq8xrvd9t
| parrit wrote:
| Is there a La Liga game on perhaps :)
| rurban wrote:
| There's the Madrid Open tennis tournament affected. No Swiatek
| - Shnaider game until they'll get online. The restaurant is lit
| with candles, but the DJ has power. So there's music. He was
| clever.
| switch007 wrote:
| Context https://news.ycombinator.com/item?id=43157259 /
| https://vercel.com/blog/update-on-spain-and-laliga-blocks-of...
|
| OP I think is joking La Liga got the power turned off to
| protect their revenues.
| mike_hearn wrote:
| This sounds big enough to require a black start. Unfortunately,
| those are slow and difficult.
|
| If an entire nation trips offline then every generator station
| disconnects itself from the grid and the grid itself snaps apart
| into islands. To bring it back you have to disconnect consumer
| loads and then re-energize a small set of plants that have
| dedicated black start capability. Thermal plants require energy
| to start up and renewables require external sources of inertia
| for frequency stabilization, so this usually requires turning on
| a small diesel generator that creates enough power to bootstrap a
| bigger generator and so on up until there's enough electricity to
| start the plant itself. With that back online the power from it
| can be used to re-energize other plants that lack black start
| capability in a chain until you have a series of isolated
| islands. Those islands then have to be synchronized and
| reconnected, whilst simultaneously bringing load online in large
| blocks.
|
| The whole thing is planned for, but you can't really rehearse for
| it. During a black start the grid is highly unstable. If
| something goes wrong then it can trip out again during the
| restart, sending you back to the beginning. It's especially
| likely if the original blackout caused undetected equipment
| damage, or if it was caused by such damage.
|
| In the UK contingency planning assumes a black start could take
| up to 72 hours, although if things go well it would be faster.
| It's one reason it's a good idea to always have some cash at
| home.
|
| _Edit: There 's a press release about a 2016 black start drill
| in Spain/Portugal here: https://www.ree.es/en/press-office/press-
| release/2016/11/spa... _
| leomca wrote:
| It seems Spain lost 15GW of load, but is still running 10GW of
| load: https://transparency.entsoe.eu/load-
| domain/r2/totalLoadR2/sh...
|
| Would this suggest the grid hasn't snapped apart, or is it just
| not possible to tell from the data?
|
| Coal, pumped hydro, and nuclear generation all went to 0 around
| the same time, but presumably that's those sources being
| disconnected from the grid to balance demand?
| https://transparency.entsoe.eu/generation/r2/actualGeneratio...
| mike_hearn wrote:
| They're definitely doing a black start:
|
| https://x.com/RedElectricaREE/status/1916818043235164267
|
| _We are beginning to recover power in the north and south of
| the peninsula, which is key to gradually addressing the
| electricity supply. This process involves the gradual
| energization of the transmission grid as the generating units
| are connected._
|
| I see load dropping to zero on that graph, or rather, load
| data disappears an hour ago.
|
| If the grid frequency goes too far out of range then power
| stations trip automatically, it's not an explicit decision
| anyone takes and it doesn't balance load, quite the opposite.
| A station tripping makes the problem worse as the frequency
| drops even further as the load gets shared between the
| remaining stations, which is why grids experience cascading
| failure. The disconnection into islands is a defense
| mechanism designed to stop equipment being too badly damaged
| and to isolate the outage.
| leomca wrote:
| Interesting, but in terms of load I think think the data
| may just be delayed by ~1 hour. Switching to UTC, to avoid
| timezone confusion, it's currently 13:10:
|
| Last actual load value for Spain at 12:15:
| https://transparency.entsoe.eu/load-
| domain/r2/totalLoadR2/sh...
|
| Last actual load value for France at 12:00:
| https://transparency.entsoe.eu/load-
| domain/r2/totalLoadR2/sh...
| mike_hearn wrote:
| It may also be a measurement artifact. Looking at the
| generation by type page,
|
| https://transparency.entsoe.eu/generation/r2/actualGenera
| tio...
|
| Everything dropped to zero except wind and solar, which
| took huge hits but not to zero. I expect those have been
| disconnected too, as they cannot transmit to the grid
| without enough thermal plant capacity being online, but
| if the measurement at some plants of how much they're
| generating doesn't take into account whether or not they
| were disconnected upstream they may still be reporting
| themselves as generating. You can't easily turn off a
| solar plant after all, just unplug it.
|
| Either that, or they're measuring generation and load
| that's not on the grid at all.
| martinald wrote:
| Probably they are estimates of not grid metered
| generation assets based on wind speed and solar
| production, at least in the UK nearly all solar is
| 'estimated' because it is not measured directly (apart
| from larger sites), at least in real time.
|
| Rooftop solar for example just shows as a reduction in
| demand, not 'generation' per se.
| tialaramex wrote:
| This also true for private wind power. Britain has a
| measurable amount of hill top farms where it just makes
| good economic sense to install a wind turbine and get
| free electricity. But we don't meter it, it shows up in
| charts as an absence - on a windy afternoon maybe Britain
| is seemingly consuming 4GW less electricity than it
| "should be". If the wind drops that load reappears on the
| grid and must be handled by existing infrastructure.
|
| None of this gear is suited to a black start. If you had
| total grid loss for a month you could doubtless rewire it
| to power the farm when it's windy despite no grid, maybe
| even run some battery storage for must-have services like
| a few lights so they keep working on still days but you
| could not start the grid from here.
| leomca wrote:
| BBC reporting the head of Spain's electricity grid saying
| restoring power could take "between six and ten hours": htt
| ps://www.bbc.co.uk/news/live/c9wpq8xrvd9t?post=asset%3A85..
| .
| martinald wrote:
| Interestingly it seems that the black start drill is
| considering a smaller zone of impact than what has happened
| here.
|
| Also I suspect there is far more renewables on the grid now
| than in 2016.
|
| This is potentially the first real black start of a grid with
| high renewable (solar/wind) penetration that I am aware of.
| Black starts with grids like this I imagine are much more
| technically challenging because you have generation coming on
| the grid (or not coming on) that you don't expect and you have
| to hope all the equipment is working correctly on
| "(semi)-distributed" generation assets which probably don't
| have the same level of technical oversight that a major
| gas/coal/nuclear/hydro plant does.
|
| I put in another comment about the 2019 outage which was
| happened because a trip on a 400kV line caused a giant offshore
| wind farm to trip because its voltage regulator detected a
| problem it shouldn't have tripped the entire wind output over.
|
| Eg: if you are doing a black start and then suddenly a bunch of
| smallish ~10MW solar farms start producing and feeding back in
| "automatically", you could then cause another trip because
| there isn't enough load for that. Same with rooftop solar.
| genewitch wrote:
| Grid tied solar won't put power into the grid when the grid
| is down. It's the one reason I didn't grid tie.
|
| Non tied solar won't affect the grid at all. So this is a
| non-issue.
|
| Grid tie requires the grid to tie to, otherwise it can't
| synchronize. So it stays disconnected.
| martinald wrote:
| Youre missing my point. I know that; I mean if you are
| restarting the grid and say you have a segment you think
| has 5MW of load that will come online, but you connect it
| and a minute or two later suddenly 2MW of grid tied solar
| detects the grid and starts exporting and you now have 3MW
| of load it is going to make it much more tricky to balance
| the restart. I'm not sure how much of a problem this is in
| reality but it seems to me restarting a grid is made much
| more tricky when you have millions of generation assets
| with no control over.
| christkv wrote:
| This is why most of the restart of the grid is being done
| as the solar input tappers out I suspect. The grid was
| pretty much down during peak sun and started coming back
| online around 5:30 pm or later.
| sponaugle wrote:
| "Grid tied solar won't put power into the grid when the
| grid is down. It's the one reason I didn't grid tie."
|
| Why would that prevent you from being grid-tie? I have 53
| panels (~21kw) grid tied and pushing to the grid, but in
| the event of grid failure my panels will still operate and
| push into my 42kwh battery array which will power the
| entire house. ( The batteries take over as the 'virtual
| grid source). I can then augment the batteries with
| generator and run fully off grid for an extended amount of
| time ( weeks in my case ).
| Tireings wrote:
| Those small solar panels probably go directly to the next
| consumer.
|
| You need to calculate for it but I don't think this would be
| a problem
| qyckudnefDi5 wrote:
| This guy grids!
| chippiewill wrote:
| The frequency aspect of a black start is presumably a bit
| easier in Europe because there's an interconnected synchronous
| grid so they can bootstrap it from France essentially.
|
| It's far more problematic for the UK because all the
| interconnects are DC.
| padjo wrote:
| I was recently told by an electrical engineering lecturer
| that the black start plan here in Ireland is to use the DC
| interconnectors with the UK to provide startup power to a
| synchronous generator.
| ajmurmann wrote:
| Did he mention the UK's plan in case they have a full
| blackout?
| pjc50 wrote:
| https://www.nationalgrid.com/sites/default/files/document
| s/H... (2017, probably current)
| InsomniacL wrote:
| "it is prudent that the cost of maintaining Black Start
| Capability should demonstrate value to the end consumer"
|
| That sounds like someone explaining why the solution is
| so bad, before describing what the solution is.
| pbhjpbhj wrote:
| To me it sounds like an energy company attempting to
| excuse lack of spending on infrastructure whilst paying
| out millions to C-suite in bonuses and millions more to
| shareholders whilst arguing prices have to rise because
| they can't afford to spend on infrastructure...
| robocat wrote:
| Your incentive "analysis" is wildly incorrect.
|
| Electricity markets and electricity networks are designed
| by the regulator.
|
| Incentives are planned by the regulator so that
| individual stations or companies have the correct
| incentives to have capabilities that the network grid
| needs.
|
| One example is financial incentives to provide black
| start capabilities. Another example is incentives to
| provide power during peak loading (peaker plants). There
| are many more examples of incentives designed so that the
| needs of the whole network are met.
|
| If an operator is incentivised to act selfishly in such a
| way that the grid will fail, then that is a failure of
| the regulator (not the individual operator).
|
| Blaming individual people or companies for systemic
| faults is generally a bad thinking habit to form. There
| are too many examples where I see individuals get blamed.
| Fixing our systems is hard but casting blame in the wrong
| places is not helpful. It's difficult to find a good
| balance between an individual's responsibilities and
| society's responsibilities.
| cyberax wrote:
| > Electricity markets and electricity networks are
| designed by the regulator.
|
| Not quite. They are _influenced_ by the regulators.
|
| And Europe has been incentivizing trash-tier low-quality
| solar and wind power, by making it easy to sell energy
| (purely on a per-Joule basis) on the pan-European market.
|
| Meanwhile, there is no centralized capacity market or
| centralized incentives for black start and grid forming
| functionality.
| htrp wrote:
| Connect with Ireland
| d1sxeyes wrote:
| The UK has pumped storage facilities that we can use to
| black start the national grid.
| beAbU wrote:
| With the new wexford-wales interconnect that went live last
| month, and another one planned from Cork (?) to France
| things might be even easier in the near future I reckon.
| oliwarner wrote:
| Does the source of "truth" matter? You still have to turn
| everything off, bring your generators up, each in sync and
| then allow load back on slowly.
|
| The UK keeping its own time just makes things easier for it
| IMO.
| wongarsu wrote:
| A synchronous interconnect provides not just a source of
| truth but also stabilizes your grid frequency. If you have
| an isolated grid you have to match generation to demand to
| keep the grid frequency stable. If you have a 1GW
| interconnect that means you can mismatch generation and
| demand by up to a gigawatt and still be fine. I imagine
| that makes for a much faster startup procedure
| k7sune wrote:
| If the isolated grids all each synced to a source of
| truth first (GPS?), would it be possible for them to
| immediately connect afterwards?
| wongarsu wrote:
| You can connect two running grids. Earlier this year the
| Baltic countries disconnected from the Russian grid, and
| synchronized and then connected to the European grid.
|
| I imagine you can get close enough by syncing to a shared
| time source like GPS or the DCF77 signal, as long as you
| communicate how the phase is supposed to match up to the
| time source. Or at least you could get close enough that
| you can then quickly sync the islands the traditional
| way.
|
| The question is if it's worth the effort and risk. Cold
| starting a power grid is a once in a lifetime event (at
| least in Europe, I imagine some grids are less stable)
| and Spain seems to plan to have everything back up again
| in 10 hours. Maybe if the entire European grid went down
| we would attempt something like that by having each
| country start up on their own, then synchronize and
| reconnect the European grid over the following week.
|
| https://en.wikipedia.org/wiki/Baltic_states_synchronizati
| on_...
| cyberax wrote:
| Nothing that complicated. You just carefully synchronize
| the state of the grid on both sides of the interconnect,
| and when they are perfectly matched, you throw the big
| switch.
| ars wrote:
| It's a bit difficult in another way: Obviously 50/60hz is
| not such a high speed that it's difficult to synchronize.
|
| The harder part is this: To pump power into the grid you
| lead the cycle ever so slightly, as if you were trying to
| push the cycle to go faster. If instead you lag the phase
| the grid would be pumping power into you.
|
| That lead is very very small, and probably difficult to
| measure and synchronize on. I would imagine that when the
| two grids connect everything jumps just a little as power
| level equalize, it probably generates a lot of torque and
| some heat, I would assume it's hard on the generator.
|
| From a physics point of view, by leading the cycle you
| introduce a tiny voltage difference (squared), divided by
| the tiny resistance of the entire grid. And that's how
| many watts (power) you are putting into the grid.
| tzs wrote:
| Isn't that backwards? When grids interconnect via DC wouldn't
| there no longer be a need for the grids to synchronize
| frequency?
| maxerickson wrote:
| With the DC interconnect, your DC to AC conversion
| equipment would need the capability to provide synchronized
| power to the generator you are trying to start. With the
| synchronized grid tie, your are pulling the generator into
| the running grid.
| celticninja wrote:
| cash is all well and good but if the tills dont work most
| stores wont be able to serve you. corner shops, vape shops and
| barbers will be ok though
| genewitch wrote:
| Stores with tills and freezers etc will have power for the
| tills but the backing network for payments probably won't be
| up. That's the concern with being cashless. They can accept
| cashxl, but no one has any.
| petertodd wrote:
| The tills have keys to manually open them, and you can just
| record transactions on pen-and-paper if needed and enter them
| later. I've seen plenty of businesses do exactly that during
| power outages. Not as fast. But totally doable.
| floatrock wrote:
| Practical Engineering did a really great video a few years ago
| on why black starts are hard, complete with a tabletop demo
| about the physics of synchronizing large spinning generators:
| https://www.youtube.com/watch?v=uOSnQM1Zu4w
| apexalpha wrote:
| Would this be relevant here? Spain is connected to the French
| grid, and probably also Morocco.
|
| The entire EU runs on one synchronised grid so from that
| perspective a single 'province' went offline, not the grid.
| crmd wrote:
| It would be essentially the same thing as a grid black start,
| except that the first breaker to close has the European grid
| on its primary side, instead of a freshly started generator
| under your control.
|
| The complex process of configuring the transmission network
| to bring grid power to each power plant in succession is the
| same.
| lysace wrote:
| The continental part of the EU runs on one synchronized grid.
| The Nordics (except half of Denmark and, uh, Iceland,
| Greenland etc) runs on a separate synchronized grid.
|
| https://en.wikipedia.org/wiki/Continental_Europe_Synchronous.
| ..
| jameshart wrote:
| Re: probably also Morocco: yes, there is an interconnect
| across the Strait of Gibraltar
|
| https://en.wikipedia.org/wiki/Spain-Morocco_interconnection
| dom96 wrote:
| Speaking of Gibraltar, it's interesting that its
| electricity grid isn't connected to Spain/EU's grid. So
| they are completely unaffected.
| mlsu wrote:
| Does solar power make this process easier or harder? I know
| that with thermal plants you have a spinning mass that you have
| to synchronize, and phase shift is used to assess how hard the
| plant is working (and whether to trip a disconnect as we see
| here)
|
| But with solar, how is the synchronization provided? In like a
| giant buck? Or in software somehow? Does the phase shift matter
| as much as in the electromechanical systems?
|
| My intuition is that solar would make the grid harder to keep
| stable (smaller mass spinning in sync) but also may offer more
| knobs to control things (big DC source that you can toggle
| on/off instantly.. as long as sun is out). But I don't actually
| know.
| mike_hearn wrote:
| Most solar and wind plants follow the inertial lead of the
| thermal plants. They can't synchronize without enough thermal
| generation being online. Supposedly there are efforts to
| change that, I don't know enough about grid engineering to
| say how far along that might be in Spain.
| pjc50 wrote:
| Mike_hearn's comment was grey but was correct: phase
| following is indeed done through software in the inverter.
| Phase matching is still required, wherever the phase
| difference is not zero there is a deadweight loss of power as
| heat.
|
| Currently the main driver of battery deployments is not so
| much energy price time arbitrage as "fast frequency
| fresponse": you can get paid for providing battery
| stabilization to the grid.
|
| (for the UK not Spain: https://www.axle.energy/blog/frequency
| )
| mlsu wrote:
| Oh that is fascinating.
|
| So if you have a smarter solar panel, or a smart battery,
| you can stabilize the grid. I'm assuming that all of the
| traditional software complexity things in distributed
| systems apply here: you want something a little bit smart,
| to gain efficiency benefits, but not too smart, to gain
| robustness benefits.
|
| My intuition is that bringing the market into it at small
| timescales probably greatly increases the efficiency
| significantly but at the cost of robustness (California
| learned this "the hard way" with Enron)
|
| > Phase matching is still required, wherever the phase
| difference is not zero there is a deadweight loss of power
| as heat
|
| If the electronic controller is "ahead of" (leading) the
| grid, then that heat would come from the solar plant; if it
| is "behind" (following) then that heat comes from the grid.
| Is that right? And likely, solar plants opted for the
| simplest thing, which is to always follow, that way they
| never need to worry about managing the heat or stability or
| any of it.
|
| I wonder if the simplest thing would be for large solar
| plants to just have a gigantic flywheel on site that could
| be brought up via diesel generators at night...
| sceadu wrote:
| > But with solar, how is the synchronization provided? In
| like a giant buck? Or in software somehow? Does the phase
| shift matter as much as in the electromechanical systems?
|
| If you mean how does solar detect phase and synchronize to
| the grid: https://en.wikipedia.org/wiki/Phase-locked_loop
|
| If you mean how does solar act to reinforce the grid: search
| for terms like "grid forming inverter vs. grid following
| inverter" though not all generators are the same in terms of
| how much resilience they add to the grid, esp. w.r.t. the
| inertia they do or do not add. See e.g.
| https://www.greentechmedia.com/squared/dispatches-from-
| the-g...
| Hydrocarb0n wrote:
| Harder mostly, See the frequency is set by huge rotating
| masses in the form of generators, and when the supply and
| demand is matched the frequency and voltage are stable, when
| demand dramatically increases it pulls the frequency and
| voltage down, which is effectively slowing the generators
| down as load / magnetic drag increases with current drawn.
| Having large inertial masses spinning actually helps smooth
| out frequency changes. whilst large solar farms can and do
| syncronise with the grid, they are reactive and do not add
| the same smoothing effect as humungous spinning masses.
|
| Low Grid frequency & voltage can cause an increase in current
| & heating of transmission lines and conductors and can damage
| the expensive things, this is why these systems trip out
| automatically at low frequency or low voltage, and why load
| shedding is necessary
| bsder wrote:
| > Harder mostly, See the frequency is set by huge rotating
| masses in the form of generators
|
| I'm not saying you're wrong, but this isn't _obviously_
| correct to me.
|
| Since solar going to a grid is completely dependent upon
| electronic DC->AC conversion, I would expect that it could
| follow a lot greater frequency deviation for a lot longer
| than a mechanical system that will literally rip itself
| apart on desync.
| bob1029 wrote:
| > Thermal plants require energy to start up
|
| It's not just about the power. System components cannot be
| brought to operating temperatures, speeds and pressures faster
| than mechanical tolerances allow. If a thermal plant is cold &
| dark, it can take _days_ to ramp it to full production.
| pfdietz wrote:
| That's true of some kinds of thermal generators, but not all.
| Simple cycle gas turbines can come up very quickly (think jet
| engines). Or your car's engine.
| bob1029 wrote:
| > Simple cycle gas turbines can come up very quickly
|
| In some cases yes. Modern combined cycle plants can take as
| little as 30 minutes to ramp to full output. Older designs
| can take upward of 4-6 hours.
|
| If you have steam as an indirection, that's when things
| take a really long time. Natural gas turbines are a more
| direct cycle.
| pfdietz wrote:
| Combined cycle turbines _do_ have a steam component (that
| 's where the word "combined" comes from). The waste heat
| from the combustion turbine front end is used to make
| steam in the back end.
| bob1029 wrote:
| Combined cycle turbines are not primarily powered by
| steam. It is a secondary consequence of their operation.
| This contrasts with nuclear and coal plants where steam
| is the prime mover.
| genewitch wrote:
| That's what they said, though
| pfdietz wrote:
| The steam part of CC systems shows hours and hours aren't
| inherently needed to get a steam power plant in
| operation. For that matter, warships with steam
| propulsion show the same thing, I believe.
| 7952 wrote:
| Can thermal stations be kept running without any laod?
| cyberax wrote:
| Sure. But not cheaply. And ramping up the power still takes
| time.
| Vox_Leone wrote:
| The fewer resources we dedicate to grid resilience and
| modernization, the harder black starts become. And as grids get
| more complex and interdependent, recovering from total failure
| becomes exponentially harder.
|
| A rare but sobering opportunity to reflect on something we
| usually take for granted: electricity.
|
| We live in societies where everything depends on the grid --
| from logistics and healthcare to communications and financial
| systems. And yet, public awareness of the infrastructure behind
| it is shockingly low. We tend to notice the power grid only
| when it breaks.
|
| We've neglected it for decades. In many regions, burying power
| lines is dismissed as "too expensive." But compare that cost to
| the consequences of grid collapse in extreme weather,
| cyberattacks, or even solar storms -- the stakes are
| existential. High-impact, low-frequency events are easy to
| ignore until they're not.
| roughly wrote:
| Yeah, this is the turkey's dilemma - life on a farm is a lot
| better than life in the wild for 51 out of the 52 weeks of
| the year.
|
| Most of our modern economy and systems are built to reduce
| redundancy and buffers - ever since the era of "just in time"
| manufacturing, we've done our best to strip out any "fat"
| from our systems to reduce costs. Consequently, any time we
| face anything but the most idealized conditions, the whole
| system collapses.
|
| The problem is that, culturally, we're extremely short-
| termist- normally I'd take this occasion to dunk on MBAs, and
| they deserve it, but broadly as a people we're bad at
| recognizing just how far down the road you need to kick a can
| so you're not the one who has to deal with it next time and
| we've gotten pretty lazy about actually doing the work
| required to build something durable.
| maxerickson wrote:
| What are some examples of modern system collapse?
|
| We've had substantial disruptions, but they've not been
| particularly irrecoverable or sustained.
| exe34 wrote:
| For the people who died of normally preventable death
| during covid while the health services were overwhelmed,
| the damage is irrecoverable. The chips shortage lasted
| years. Every year we become more, not less, dependent on
| the supply chain working. Every year we become less, not
| more, resilient.
| maxerickson wrote:
| I don't think it's crass to separate the deaths that
| occurred from a novel disease from the impact it had in
| society. In the medium term, it's a blip, never mind the
| long term. There's a huge chunk of society that thinks
| there was a huge overreaction!
|
| The chips shortage has been difficult, but it's also been
| little more than an inconvenience when you look at it in
| terms of goods being available to consumers or whatever.
| tehjoker wrote:
| That chunk is heavily influenced by the propaganda that
| over a million dead people isn't a big deal. The
| propaganda is economically incentivized because slowing
| down the economy is bad for business even if it protects
| human lives.
| Xelbair wrote:
| "Just in time" is a phrase I hate with vehement passion.
| You aren't optimizing the system, you're reducing safety
| marigns - and consequences are usually similar to
| Challanger.
|
| This is a solution that teenager put in management position
| would think of(along with hire more people as solution to
| inefficient processes), not a paid professional.
|
| Systems like electric grid, internal water management
| (anti-flood) shouldn't be lean, they should be antifragile.
|
| What's even more annoying that we have solutions for a lot
| of those problems - in case of electric grids we have
| hydroelectric buffers, we have types of powerplants that
| are easier to shutdown and startup than coal, gas or
| wind/solar(which cannot be used for cold start at all).
|
| The problem is that building any of this takes longer than
| one political term.
| exe34 wrote:
| > You aren't optimizing the system
|
| Of course not, they're optimising shareholder profit.
| Marsymars wrote:
| > Systems like electric grid, internal water management
| (anti-flood) shouldn't be lean, they should be
| antifragile.
|
| How do you make those systems _antifragile_ rather than
| simply highly resilient?
| signatoremo wrote:
| JIT isn't about reducing safety margin. It was pioneered
| by Japanese companies, namely Toyota. They are known for
| risk adverse, safety first.
|
| > This is a solution that teenager put in management
| position would think of(along with hire more people as
| solution to inefficient processes), not a paid
| professional.
|
| What kind of comment is this? Toyota has been using and
| refining it for decades. It wasn't invented yesterday by
| some "teenagers". Such a state of HN's comment section.
|
| JIT is definitely not perfect as exposed during the Covid
| period, but it isn't without merits and its goal isn't
| "reducing safety margin".
|
| Then we have JIT in computing, such as JVM.
| the_duke wrote:
| Just to highlight this: the last significant power outage in
| Western/middle Europe was 2003. [1]
|
| That's 20 years without any significant problems in the grid,
| apart from small localized outages.
|
| It's not hard to start taking things for granted if it works
| perfectly for 20 years.
|
| Many people don't even have cash anymore, either in their
| wallet or at home. In case of a longer power outage a
| significant part of the population might not even be able to
| buy food for days.
|
| [1] https://en.wikipedia.org/wiki/2003_Italy_blackout
| genewitch wrote:
| > a significant part of the population might not even be
| able to buy food for days.
|
| And who's fault is that? Why did europe allow this?
|
| Why will the US allow this, eventually?
| olyjohn wrote:
| Why will the US allow this? Because there's too much
| money to be made by middle men who want their cut of the
| transactions.
| ithkuil wrote:
| I don't know but I find it very practical to not carry
| piles of cash in my pocket and home and know that we're
| less likely to get robbed just because of the cash we
| have.
|
| I don't know how true the relationship between the
| cashless lifestyle and safety actually is, but it works
| and I feel ok; I'm not sure that the prospect of a few
| hours of national blackout once in 20 years will make me
| change my mind significantly.
| layman51 wrote:
| It does seem safer to not carry cash. However, I remember
| around May 2024 that there were some reported incidents
| in Chicago were in the early morning hours, there were
| some groups of robbers who would force victims to do
| something I found especially worrying: they would be
| forced into resetting their phone password and logging
| into their mobile banking app. I'm not sure what ended up
| happening in these cases.
| j1elo wrote:
| Today I was able to walk into a grocery store, pay for
| food, and go home to have a warm lunch (having a gas
| stove also helped tremendously). The matter was having a
| 10EUR note at home. Not what I'd call "piles of cash".
|
| As an added benefit, no bank knows where I bought and
| when, which I find is a great advantage over the
| alternatives. (I also use Gpay; this comes from someone
| who just found a good middle point without forgetting
| about the more reliable, physical and privacy friendly
| option)
| artursapek wrote:
| Cash is extremely easy to conceal. fear of burglary is a
| ridiculous reason not to have some saved for a rainy day
| petertodd wrote:
| > And who's fault is that? Why did europe allow this?
|
| In Spain it's now illegal to pay with cash for
| transactions over 1000EUR. Absurd.
|
| In Norway they recently made it mandatory in most
| circumstances to accept cash for transactions up to
| 20,000kroner (~1700EUR): https://www.norges-
| bank.no/en/topics/notes-and-coins/the-rig...
| emporas wrote:
| We will use better technology for electronic
| transactions. Most of banks worldwide still use COBOL for
| most (all?) their software infrastructure.
|
| You can do as many electronic transactions as you wish
| without internet or electricity, provided you have
| something with charged battery. Problem is the
| transaction cannot be verified without internet, but when
| internet gets restored, all transactions can be applied.
|
| That technology exists for more than a decade, so banks
| will implement it in 20 or 50 years. Most sane people
| will not wait patiently for half a century till some
| software engineer implements electronic transactions with
| COBOL, and we will use some kind of blockchain much
| sooner than that.
| sgt101 wrote:
| > Most of banks worldwide still use COBOL for most (all?)
| their software infrastructure.
|
| Nahhh - some banks have some parts of the infrastructure
| in COBOL. Specifically larger retail banks often have
| their ledgers in COBOL. Most of them want rid and are
| actively getting rid. Most places have had programs to
| root COBOL out since before 2000, but there are residual
| implementations remain. The ledgers are the hardest place
| to deal with because of the business case as well as the
| awkwardness. Basically there's not much of an advantage
| (or at least hasn't been) in modernizing so keeping the
| thing going has been the option. Now people want to have
| more flexible core systems so that they can offer more
| products, although not so sure that customers want this
| or can consume it. Still - it supports the idea of
| modernisation so not many people are keen to challenge.
|
| The most common big implementations I come across are in
| Java.
| emporas wrote:
| They use Go as well, I know someone who writes it for
| banks, but most of their infrastructure is written in
| COBOL. There are some sources [1], and some people have
| told me the same in person, not in exact numbers or
| percentages, but roughly the same.
|
| Anyway point remains, electronic transactions with no
| internet or electricity is a solved problem, and banks
| don't want to solve it or they can't due to incompetency
| or maliciousness.
|
| Currency transactions worth their weight in gold, it is
| of utmost importance for transactions to always be
| published to a central authority right away. If they
| don't have to be published, they should not exist at all.
| Imagine people buying stuff without anyone knowing right
| away! That should never, ever exist, for any reason.
|
| [1] https://thenextweb.com/news/ancient-programming-
| language-cob...
| shallichange wrote:
| I see no problem with COBOL. I'd rather have my credit
| card transaction be processed with 40yo well tested COBOL
| than Java from the newest intern using copilot.
| LunaSea wrote:
| Anti tax evasion and money laundering measures
| rini17 wrote:
| There are now ubiquitous wireless POS terminals for card
| payments that can be recharged from emergency sources of
| electricity(like cars). As long as the mobile internet
| works it's possible. Of course this only little alleviates
| the disruption.
| userbinator wrote:
| _As long as the mobile internet works it 's possible._
|
| If we're talking about a situation where the grid goes
| down, the mobile internet is most definitely _not_
| working.
| jadyoyster wrote:
| Surprisingly mobile networks seemed to stay up in
| Portugal. I'm not sure to what extent and if they lasted
| for the whole duration of the blackout. They definitely
| limited consumer use though.
| citrin_ru wrote:
| > Many people don't even have cash anymore, either in their
| wallet or at home.
|
| Even if you have cash many shops would not sell anything in
| case of a mass outage because registers are just clients
| which depend on a cloud to register a transaction. Not
| reliable but cheap when it works.
| brewdad wrote:
| In a multi-day event like we are talking about here,
| couldn't a shop owner revert to a paper ledger? I mean,
| it would suck and transactions would take much longer but
| if the alternative is people starving or having your
| inventory looted by a desperate mob, a nineteenth century
| solution seems preferable.
| holoduke wrote:
| Well if the situation happens that people can't buy food,
| things will easily become nasty quickly. People will break
| open stores and use violence to get what they need. So cash
| money won't really help a lot in a serious situation.
| cogman10 wrote:
| > Many people don't even have cash anymore, either in their
| wallet or at home. In case of a longer power outage a
| significant part of the population might not even be able
| to buy food for days.
|
| So, what's really interesting is that these sorts of social
| collapses have happened. In fact, they often happen when
| natural disasters strike.
|
| When they do happen, mutual aid networks just sort of
| naturally spring up and capitalism ends up taking a
| backseat. All the sudden worrying about the profits of
| Walmart are far less important than making sure those
| around you don't starve.
|
| As it turns out, most people, even managers of stores,
| aren't so heartless as to let huge portions of the
| population starve. Everyone expects "mad max" but that
| scenario simply hasn't played out in any natural disaster.
| In fact, it mostly only ends up being like that when
| central authority arrives and starts to try and bring
| "order" back.
|
| You can read about this behavior in "A Paradise Built in
| Hell" [1].
|
| [1] https://en.wikipedia.org/wiki/A_Paradise_Built_in_Hell
| jrs235 wrote:
| Honest question, are we better off in the long run, and is it
| a better solution, to decentralize energy generation and make
| more smaller grids rather than linking them all up? This
| isn't to say completely getting rid of the ability to
| transfer between the smaller grids to assist with power
| disruptions but to decouple and make it less likely for
| catastrophic "global" failures like this.
| zanellato19 wrote:
| It's hard to build big generators, so as we already
| struggle with infrastructure I don't think that's feasible,
| but it would be great.
| mhandley wrote:
| With a high fraction of renewables, the reverse is probably
| better in the long run. The larger geographic area you
| connect, the less you're affected by weather systems, and
| the wider area you can draw dependable dispatchable power
| such as hydro from. But that depends on having enough grid
| capacity to move enough power around, which is currently a
| problem.
|
| But I wonder from a reliability (or lack of cascading
| failures) point of view whether synchronous islands
| interconnected with DC interconnects is more robust than a
| large synchronous network?
| pyrale wrote:
| Burying lines is not a panacea, it generates massive
| reactance changes compared to a classic line.
| yibg wrote:
| Maybe a good reason (in parts of the world where this is
| practical) to have some solar + battery storage. Doesn't even
| need to fully replace grid power, just enough to run the
| barebones when the grid goes out.
| dv_dt wrote:
| I would think that renewable infrastructure could be the fix,
| at least if you start installing larger battery capacity to
| meet renewable store and usage shifts, the grid essentially is
| installing the resources that can also be used to respond to &
| contain sudden source losses and prevent cascades.
| oilman wrote:
| In another life I worked as an engineer commissioning oil rigs
| and I've seen how tricky even a small-scale black start can be.
| On a rig, we simulate total power loss and have to hand-crank a
| tiny air compressor just to start a small emergency generator,
| which then powers the compressors needed to fire up the big
| ~7MW main generators. It's a delicate chain reaction -- and
| that's just for one isolated platform.
|
| A full grid black start is orders of magnitude more complex.
| You're not just reviving one machine -- you're trying to bring
| back entire islands of infrastructure, synchronize them
| perfectly, and pray nothing trips out along the way. Watching a
| rig wake up is impressive. Restarting a whole country's grid is
| heroic.
| zdragnar wrote:
| I can appreciate the ability to revert to hand cranking an
| air compressor, yet I can't help but feel that the 99.99% of
| events, you'd be better served with keeping a two stroke gas
| engine ready to go. Air compressors tend to have parts just
| as or more vulnerable to environmental factors, and you get a
| lot more power for less elbow grease out of a two stroke.
| Symbiote wrote:
| A power station can start a decently large generator with
| batteries.
|
| Maybe there are other concerns for an oil rig.
| tonyarkles wrote:
| Batteries are great when they have charge. What happens
| if the generator doesn't want to start the first, second,
| and third time? How many start attempts do you get before
| the batteries are dead?
|
| The hand-pumped air compressor is the tool of last
| resort. You can try an engine start if there's someone
| there who's able to pump it. You don't have to worry
| about how much charge is left in your batteries or
| whether or not the gasoline for the 2-stroke pump engine
| has gone stale. It's the tool that you use as an
| alternative to "well, the batteries are dead too, guess
| we're not going to start the engine tonight... let's call
| the helicopters and abandon ship"
| SoftTalker wrote:
| The data center where I work has large diesel generators
| for power cuts. They are electric (battery) start. There
| is no capability to start them manually. The batteries
| are on maintenance chargers that keep them in good
| condition. The generators are started and tested every
| two weeks.
|
| Could the batteries be dead and the generators not start?
| I guess but it's very unlikely. I get that on an oil rig
| it might be a matter of life and death and you need some
| kind of manual way to bootstrap but there's not much
| that's more reliable than a 12V lead-acid battery and a
| diesel engine in good condition.
| petertodd wrote:
| Also, the data center is probably in a city, surrounded
| by infrastructure that could be used if necessary. An oil
| rig is in the middle of an ocean, and has to rely on
| itself.
| Symbiote wrote:
| It's unrealistic, and if one power station is unable to
| use their batteries to start their emergency generator
| (through the absurd incompetence you describe, or more
| likely through a major fire, flood or assault) the grid
| can be started from a different one.
| Damogran6 wrote:
| Not being at all qualified to comment (though I work for a
| power company), I'd think the hand crank air compressor
| wouldn't suffer from no spark or bad gas.
| 0cf8612b2e1e wrote:
| Stale gas was my guess for the plan. Maintaining an
| emergency system is one of those things that is easy to
| neglect.
| zdragnar wrote:
| If stale gas is a concern, then all of the other steps
| in-between zero power and your full start are also
| screwed.
|
| Air compressors have more valves and gaskets that are
| vulnerable to oxidation, especially in salty
| environments, so I'd have thought the upkeep between the
| two, the two stroke would be easier.
| lostlogin wrote:
| It's quite funny to think about.
|
| Having good, fresh fuel on an oil rig. They need an
| engine that can run on crude.
| baq wrote:
| Diesel will run on mostly anything if it's running...
| including methane in the air intake, so you need to think
| quickly when presented with a generator that keeps
| running after cutting the fuel
| oilman wrote:
| In 99.99% of real-world scenarios, the rig would have other
| options to bootstrap a black start--like fully charged air
| tanks, backup power from a support vessel, or even
| emergency battery systems. The hand-cranked air compressor
| is really a last resort tool. We test it during
| commissioning to prove it could work, but in most cases,
| it's never used again in the rig's working life. It's there
| for the rarest situations--like if a rig was abandoned
| during a hurricane, drifted off station, and someone
| somehow ended up back onboard without normal support. It's
| a true "everything else failed" kind of backup.
| ale42 wrote:
| Nice to see that at least in some places people are
| actually thinking to almost-impossible scenarios and
| taking them into account. I have the feeling that it's
| quite unlike most infrastructure development nowadays,
| unfortunately.
| morning-coffee wrote:
| Based on how difficult it can be to start my chain-saw,
| snow-blower, and motorcycle after they've sat without being
| run for a while, I'd not recommend a gasoline-powered
| engine to be the only thing on stand-by.
| zdragnar wrote:
| Air compressors in adverse environments don't hold up
| that well either, without basic maintenance. I've had
| engines run seasonally for decades. It doesn't take much
| for them to keep working well, though doing nothing at
| all is an easy way to clog up the carburator.
| giantg2 wrote:
| It shouldn't be that bad. A little fogging oil when put
| away and drain all the fuel. Then a little starting fluid
| on the first couple start attempts. Usually they start
| fairly quickly if they're in decent shape. And that's
| just for pull starts. My electric start mower starts
| right up after even 5 months of not running with
| stabilizer in the fuel.
| BenjiWiebe wrote:
| As an ex small engine mechanic, I'd advise against using a
| 2 stroke for something like that. A 4 stroke would be a
| better bet. Better yet would be a natural gas/propane 4
| stroke, since gasoline goes stale and plugs carburetors.
|
| Small diesels could be an option but they're harder to pull
| start for a given size.
| ethbr1 wrote:
| Do you have any small engine mechanic books you'd
| recommend?
|
| They're my kryptonite, but I accept it's mostly my
| ignorance.
| Kon-Peki wrote:
| > Small diesels could be an option but they're harder to
| pull start for a given size.
|
| I once needed to jump-start a small marine diesel, many
| miles from land...
|
| There was a small lever that cuts compression. You have
| to get it spinning really fast before restoring
| compression! It's definitely a lot of work!
|
| EDIT - Here is a cheap modern small marine diesel [1].
| The operation manual suggests that you don't have to do
| anything to get it spinning quickly, you just have to
| crank it 10 times, put away the crank handle, and then
| flip the compression switch. That's progress!
|
| [1] https://www.yanmar.com/marine/product/engines/1gm10-m
| arine-d...
| pomatic wrote:
| Lister diesel generators are much the same - half a dozen
| cranks, restore compression and off they go. The hand
| cranking can easily break your arm if you get it wrong
| though.
| giantg2 wrote:
| Even gas engine pull starts have a compression release
| function built in. That's why you need special cylinder
| pressure tools to check compression on most pull starts.
| dzhiurgis wrote:
| I did that too and crank got stuck on flywheel. To stop
| engine I had to climb over the engine where now-removed
| stairs were since my mate was clueless. Fortunately the
| crank handle stayed on.
|
| Cranks and decompression levers are gone for at least
| 30-40 years now tho.
| stronglikedan wrote:
| I'd rather bet on the simplicity of a glorified bicycle
| pump than the complexity of an engine any day, but then
| again, I'd probably have both!
| cybergibbons25 wrote:
| Black out on a rig or ship is very different to black start
| of a national electricity grid.
|
| Most vessels will experience a blackout periodically and the
| emergency generator start fine, normally on electric or
| stored air start, and then the main generators will come up
| fine. It's really not delicate, complex or tricky - some
| vessels have black outs happen very often, and those that
| don't will test it periodically. There will also be a
| procedure to do it manually should automation fail.
|
| There are air starters on some emergency generators that need
| handling pumping. These will also get tested periodically.
|
| The most complex situation during black out restoration would
| be manual synchronisation of generators but this is nothing
| compared to a black start.
| lenerdenator wrote:
| I remember talking to my ex's dad about his job, which
| involved planning refuels of a large nuclear-powered
| generation station in the Lower Midwest.
|
| The words "it's a miracle it works at all" routinely popped
| up in those conversations, which is... something you don't
| want to hear about any sort of power generation - especially
| not nuclear - but it's true. It's a system basically built to
| produce "common accidents". It's amazing that it doesn't on a
| regular basis.
| tpmoney wrote:
| > The words "it's a miracle it works at all" routinely
| popped up in those conversations, which is... something you
| don't want to hear about any sort of power generation -
| especially not nuclear - but it's true.
|
| Funny thing is, those are the exact words I use when
| talking to people about networking. And realistically
| anytime I dig deep into the underlying details of any big
| enough system I walk away with that impression. At scale, I
| think any system is less "controlled and planned precision"
| and more "harnessed chaos with a lot of resiliency to the
| unpredictability of that chaos"
| r3trohack3r wrote:
| This is one of the key insights in my early SRE career
| that changed how I viewed software engineering at scale.
|
| Components aren't reliable. The whole thing might be duct
| tape and popsicle sticks. But the trick for SRE work is
| to create stability from unreliable components by
| isolating and routing around failures.
|
| It's part of what made chaos engineering so effective.
| From randomly slowing down disk/network speed to
| unplugging server racks to making entire datacenters go
| dark - you intentionally introduce all sorts of crazy
| failure modes to intentionally break things and make sure
| the system remains metastable.
| lisper wrote:
| > those are the exact words I use when talking to people
| about networking
|
| Or the U.S. financial system. Or civilization in general.
| psunavy03 wrote:
| It ultimately comes down to shared norms, shared
| expectations, and trust.
| 0xbadcafebee wrote:
| If people knew how crappy, insecure, and unreliable nuclear
| computer systems were, there'd be a lot more existential
| dread about cyber security
| mikewarot wrote:
| Almost all computers are insecure, not just the systems
| in nuclear stations.
|
| Most operating systems are based on ambient authority,
| which is just a disaster waiting to happen.
| SoftTalker wrote:
| Hand crank? I'd think something like an oil rig would have a
| propane or gasoline or diesel generator with an electric
| start and batteries.
| detaro wrote:
| and if the entire thing depends on it, you'll give that
| generator a handcrank as a backup too instead of assuming
| the batteries ever dying or getting flooded or whatever is
| entirely impossible.
| petertodd wrote:
| The point isn't to make a system that is easy. The point is
| to make a system that is guaranteed to work in any remotely
| realistic circumstance.
|
| In a real black start, the guys might very well grab a
| portable generator and just use that instead. But having
| the option to hand crank something rather than rely on
| batteries that might run flat is good.
| tantalor wrote:
| > hand-crank a tiny air compressor
|
| Is that what Dr. Sattler is doing in this scene from
| _Jurassic Park_?
|
| https://www.youtube.com/watch?v=FoW4vXnkhJw
| petertodd wrote:
| That would be charging up the spring to throw the breaker.
| High voltage breakers need to switch on (or off) very
| quickly, to avoid damage from arcing. It's common for them
| to have some kind of spring or gas piston arrangement that
| you pump up first to give them enough energy to do that
| quickly.
|
| Nice attention to detail by the filmmakers.
| TheJoeMan wrote:
| No, he's winding up a spring to close the circuit breaker
| quicker than a human hand could, which reduces/prevents and
| arc from forming as the electrical contacts close.
| optimalsolver wrote:
| Hold on to your culos.
| Tireings wrote:
| Ah shit now I want that panel for my dream house
| tantalor wrote:
| I found a Siemens WLL2F325 on eBay for only $24k
|
| https://www.ebay.com/itm/115854984950
| marai2 wrote:
| How did you remember this scene?!
| carlhjerpe wrote:
| I love the "analog" handcranked air compressor to 7MW
| generator escalation, it really captures human ingenuity.
|
| I wonder however how being part of the "continental Europe
| synchronous grid" affects this, and how it isolates to
| Portugal and Spain like this.
|
| But yeah there are a lot of capacitors that want juice on
| startup that happily kills any attempt to restore power. My
| father had "a lot" of PA speakers at home and when we tripped
| the 3680w breaker (16A 220v) we had to kill some gear to get
| it back up again. I'm also very sure we had 230v because I
| lived close to the company I worked for and we ran small
| scale DC operations so I could monitor input voltage and
| frequency on SNMP so through work I had "perfect amateur"
| monitoring of our local grid. Just for fun I got
| notifications if the frequency dropped more than .1 and it
| happened, but rarely. Hardly ever above though since that's
| calibrated over time like Google handle NTP leap seconds.
|
| I love infrastructure
| kryogen1c wrote:
| > have to hand-crank a tiny air compressor just to start a
| small emergency generator
|
| Similarly, the US Navy maintains banks of pressurized air
| flasks to air-start emergency diesels. Total Capacity being
| some multiple of the required single-start capacity
| lr1970 wrote:
| bringing islands together requires one to synchronize both --
| frequency and phase. It is super difficult for large
| generators and transmission lines. transient heat dissipation
| can be a real bummer.
| egorfine wrote:
| Ukraine can share some experience with forced black starts.
| petertodd wrote:
| Ukraine is interesting in this context because there are _so_
| many generators. In the richer parts of Odesa I 've even
| found it hard sometimes to tell whether or not the grid power
| is on as literally every single building had sufficient
| backup generators to keep the lights on (also, many big
| businesses seemed to run their generators even when the power
| was on, I presume as a civic-minded way to add generation
| capacity overall and avoid interruptions when the power flips
| on and off).
| gwbas1c wrote:
| Spain is part of the largest interconnected grid in the world:
| https://en.wikipedia.org/wiki/Continental_Europe_Synchronous...
|
| Does this really qualify as "black start" when they can rely on
| the bigger EU grid?
| throwup238 wrote:
| Yes because they have to bring it all back up in phases so
| that they only face the load spike* from one interconnect at
| a time, which can take some time and can fail if there's
| unknown damage like the GP said.
|
| It really depends on the region though because almost all
| large hydroelectric dams are designed to be primary black-
| start sources to restore interconnects and get other power
| plants back up quickly in phase with the dam. i.e. in the US
| 40% of the country has them so it's relatively easy to do.
| The hardest part is usually the messy human coordination bit
| because none of this stuff is automated (or possible even
| automatable).
|
| * the load spike from everyone's motors and compressors
| booting up at the same time
| pbhjpbhj wrote:
| Presumably emergency phone comms still work(?) so they
| could issue instructions to do a phased (heh!) restart to
| avoid every fridge/air conditioner/whatever restarting at
| once. Not sure how successful that would be however.
| throwup238 wrote:
| There's usually two parallel processes going on during a
| black-start: spinning the power plants back up to get
| them synchronized to the grid, and getting power back out
| to all the consumers. Power plants have breakers that
| disconnect them from consumers which they keep
| disconnected until independent connections to other power
| plants allow them to spin up their massive turbines and
| synchronize them to the grid. There's also tons of other
| downstream breakers at substations which will be in
| various states of functionality.
|
| The power plants with direct connections have hard lines
| and black-start procedures that get power out to the most
| important customers like telecom infrastructure, which
| provides the rest of the comms. In a real world full
| restart it's going to mean organizing workers at many
| substations to babysit old infrastructure so cellular is
| pretty much mandatory.
| cyberax wrote:
| And to add more fun, this time they're not dealing with a
| small number of individual power plants that can be
| connected with only some phone-call based coordination.
|
| Instead, there are literally hundreds of smaller
| wind/solar installations. Some of which depend on rapidly
| fading cellular communication to restart. And some might
| need an actual site visit to throw on the physical
| breakers.
| Damogran6 wrote:
| It's a complex enough issue that you can't really 'hand
| wavey' it. The details are specific and thorny.
| JumpCrisscross wrote:
| But _not_ a black start.
| zymhan wrote:
| Presumably not
|
| > A black start is the process of restoring an electric power
| station, a part of an electric grid or an industrial plant,
| to operation _without relying on the external electric power
| transmission network_ to recover from a total or partial
| shutdown.[1]
|
| https://en.wikipedia.org/wiki/Black_start
| throwup238 wrote:
| The key part of the phrase is actually _" electric power
| station, a part of an electric grid or an industrial
| plant."_ Note how the definition doesn't include an entire
| grid.
|
| Only the first power plant in a black-start (like a
| hydroelectric dam or gas plant started by a backup
| generator) is truly "black started." The rest don't fit
| that definition because they _depend_ on an external power
| source to spin up and synchronize frequency before burning
| fuel and supplying any energy to the grid. If they didn 't,
| the second they'd turn on they'd experience catastrophic
| unscheduled disassembly of the (very big) turbines.
|
| Only the first power plant can come online without the
| external transmission network.
| bombcar wrote:
| That's not necessarily 100 true, as you could have
| multiple plants black start and then synchronize later.
|
| In fact, if you're not sure which will start first, you
| might go that way. They're all disconnected from the grid
| at that time anyway.
| Havoc wrote:
| Yup - a true black start would be substantially more
| difficult and time consuming. More like a gradual reconnect.
|
| Then again they might be less prepared precise because of the
| euro grid is available
| mike_hearn wrote:
| Spain is but Portugal is only connected to Spain and they are
| currently doing a full black start.
|
| For Spain the external power and synchronization can come
| from France rather than generators which will help, but the
| process and complexities are still mostly the same. Call it a
| dark start, perhaps.
| larusso wrote:
| I actually have no real clue how and where Spain/Portugal is
| connected to rest of Europe but could they also restart North
| to south with help from the grid from France?
| kennysmoothx wrote:
| This is literally like Factorio.
|
| If your Factory uses too much power, theres not enough energy
| to run the power plants generation, which decreases your power
| production. Death spiraling until theres no power.
|
| You have to disconnect the factory, and independently power
| your power plants back up until you have enough energy
| production to connect your factory up again.
| baq wrote:
| They should allow you to build automatic rate-of-change-of-
| power-generation breakers...
| abdullahkhalids wrote:
| Factorio has logic circuits which allow you to do this.
| eddd-ddde wrote:
| In my server I hooked up a sound alarm to a set of
| capacitors. Too low of a charge indicates higher power
| consumption than production, allowing you to unplug certain
| low priority loads. I also have some emergency coal
| generators ready to go at the flick of a switch if needed.
|
| Man I need to go play some more.
| shagie wrote:
| Capacitor circuit network warning for alarm for "main grid is
| drawing on this bank - bad things may happen if capacity is
| not increased."
|
| Another "trick" is those burner inserters are black start
| capable. They can pick up fuel and feed themselves to keep
| running without an electrical network.
|
| I also tend to put Schmitt triggers in low priority areas.
| They've got a battery on the main grid next to them and if
| the battery drops below 50% power they remain off until it
| goes back above 75% power.
| tux3 wrote:
| Can't wait for a Factorio mod that simulates the grid
| frequency and damages equipment when it drifts too far!
| juancroldan wrote:
| I've been thinking EXACTLY this while listening to FM radio
| for the last 7 hours (we didn't even have phone line)
| somat wrote:
| Same with Satifactory, The larger powerplants need a lot of
| energy for their infrastructure to run and an overload will
| trip breakers and shut the whole grid down, a naively
| designed grid death spirals very easy. My factory was needing
| increasingly complicated black start systems so I started
| putting the powerplant infrastructure on a self contained
| islands. something a factory overload would not trip, it was
| something like one coal power plant can run the machinery
| needed for itself and 8 grid power plants.
| worldsayshi wrote:
| Thinking about this makes me wonder how all real world
| power plants aren't designed to survive a grid outage? Why
| would you ever need to black start a real power plant?
| Like, can't you design it so that it disengages from the
| grid rather than shutting down in a catastrophic outage
| like this? Or make it only take on a fraction of the load
| or something?
| belter wrote:
| One of the Administrators of the REN, the Portuguese electric
| supplier is currently giving a press conference. Confirmed they
| are in scenario of restart from black start.
|
| - Cause of event not known yet.
|
| - They noticed power oscillations from the Spanish grid that
| tripped safety mechanisms in the Portuguese grid. At the time,
| due to the cheaper prices, the Portuguese grid was in a state
| of importing electricity from Spain.
|
| - They are bringing up multiple power systems and the
| Portuguese grid is able to supply 100% of needs if required. It
| was not configured in such a state at the moment of event.
|
| - They had to restart the black start more than once, since
| while starting, noticed instabilities in some sectors that
| forced them to restart the process.
|
| - Time for full recovery unknown at this time, but it will take
| at least 24 hours.
| lastdong wrote:
| Can't upvote enough! Thank you!
| beefnugs wrote:
| And if you are a mere mortal in this world, play Factorio :
| Space age expansion on the planet Aquilo. To learn the precious
| importance of reliable multi stage power bootstrapping
| anovikov wrote:
| Power never went out in a country completely. At the lowest
| point consumption was ~40% of normal for that time of day.
|
| Ukraine went through many black starts in the first winter of
| Russian strikes against energy. I guess they built a skill of
| recovering it quickly enough that it started happening faster
| and easier every next time.
| Fire-Dragon-DoL wrote:
| Oh damn, I thought this was a problem only in Dyson Sphere
| Program, lol
| sgt101 wrote:
| I did a project where we predicted transformer failure (they
| blow up!) from changes in the oil that they have in them (its
| insulative properties suddenly change). This was 30 years ago
| so it's all a bit fuzzy, but the one thing that really stuck
| with me was the story that the SME that we were working with
| had about the UK grid teetering on the edge of failure during
| the "great storm" of 1987. His telling was that they really
| were unsure if they would get it back at all!
| bsder wrote:
| >It's one reason it's a good idea to always have some cash at
| home.
|
| Most places are so dependent upon electricity that they can't
| even take _cash_ during a blackout. And they don 't even have
| the mechanical machines to take a credit card imprint anymore.
| brewdad wrote:
| The last of my raised number credit cards went away last
| month. Those old machines will only get a blank rectangle
| from me. Sad, because I did actually use one of those about 3
| years ago when a rural gas station had power but no network.
| isolli wrote:
| What does synchronization mean in this context, and why is it
| needed?
| seabrookmx wrote:
| The bulk of the power grid is alternating current (AC), and
| the frequency of the resulting sine wave needs to be
| synchronized with the other parts of the grid it is connected
| to.
| Tempest1981 wrote:
| The grid uses AC (not DC), running at 50 Hz (cycles per
| second). So the voltage is going up and down at that
| frequency, in a sine-wave pattern.
|
| If you try to connect another generator to the grid, it needs
| to be at the same point (phase) in the sine-wave cycle, so
| that its power contribution is added, not subtracted.
|
| If it's not in sync, huge currents can flow, causing damage.
| Sort of like connecting jumper cables backwards.
| hnthrowaway0315 wrote:
| I wonder if someone could build a realistic scenario into a
| game -- let's say some sort of smaller scale black start for,
| say, a space station. And throw in a unknown computer
| architecture for the in-game computers so that players need to
| RTFM to figure out how it works.
| mrtz wrote:
| The "Blackout" novel comes to mind:
| https://en.wikipedia.org/wiki/Blackout_(Elsberg_novel)
| carlos-menezes wrote:
| Mobile network is fully gone on the islands now -- it was working
| until ~10 minutes ago.
| bakugo wrote:
| Still working in mainland Portugal but it's getting slow,
| probably severely overloaded.
| carlos-menezes wrote:
| Yeah, it's back and slow. Sidenote: RTP (National Public
| broadcaster) is gone.
| guerrilla wrote:
| RTP came back for me an hour or so ago.
| rurban wrote:
| They just reported on TV it will be on again in 20min earliest
| (15:30 CEST)
| ta1243 wrote:
| Madrid Mayor said
|
| > If emergency calls go unanswered, go to the police and the fire
| stations in person
|
| That's not a statement I expect to see in relation to a developed
| city
| belter wrote:
| As of 2 min ago..( 14:32 WEST ) in many parts of Portugal
| electricity still out, and also Water systems also offline.
| Report is for some it can take up to 10 to 12 hours before power
| back up. Probably many locals not reporting here as they are low
| on battery.
| rich_sasha wrote:
| Grids are tricky, because the electric socket is kind of a leaky
| abstraction for heavy plant machinery in turbines of thermal
| plants, load balancing, synchronisation, and a gazillion other
| things I really don't understand.
|
| Is a grid built on renewables and batteries somehow more
| resilient? Solid state things tend to be less fiddly, hence my
| question.
|
| I remember reading at one point in the past that renewables were
| actually worse for the grid due to less predictable power
| generation or something, but that was a long time ago, certainly
| pre-battery storage.
| Filligree wrote:
| > Is a grid built on renewables and batteries somehow more
| resilient?
|
| So this is a complicated subject in itself, and a full answer
| won't fit inside this textbox. Some bullet points:
|
| - Grid stability is maintained by batteries, but not literally.
| The "batteries" in question are typically rotating generators,
| i.e. turbines, wind, literally anything where you have an
| electrical coupling to a lot of physical inertia. That's what
| keeps the grid running second-to-second; while a power plant
| might pretend it's outputting a constant 4MW, it actually
| shifts noticeably from moment to moment. The kinetic energy of
| the generator helps balance that out.
|
| - Going up from the sub-second range, an overload of the
| generator obviously would cause the shaft to slow down,
| dropping the frequency and causing brownouts. Brownouts are bad
| and can damage the grid, so typically breakers will disconnect
| if it falls below 49Hz; a 2% drop.
|
| - Baseload plants can't cope with this, as they take multiple
| minutes to spool up. Minimum; for something like a coal power
| plant, where you have to shove in additional coal and wait for
| it to catch fire, it's going quite a few minutes. This is what
| defines 'baseload'.
|
| - Peaker power plants can increase (or decrease) their
| mechanical power production in a matter of seconds. These days
| that typically means gas turbines, though hydroelectric power
| is even better, and nuclear power _could_ be used for peaker
| plants -- but isn 't; most nuclear reactor designs outside of
| the navy is a baseload design. France does have some load-
| following designs, and we need more of those.
|
| - Wind turbines can't increase their output, flat out, but they
| can _decrease_ it (by feathering, or by using brakes). This is
| good enough, except this would turn them into 'peaker plants'
| that can't help with peaks. If we had enough wind turbines to
| cover 100% of the load then we'd technically be fine, but
| economically speaking that doesn't work; they'd be at less than
| 10% power most of the time.
|
| - Wind turbines have rotating shafts, but a lot of the time
| they produce DC power, linked through inverters, which removes
| that benefit and makes them act like solar panels in effect.
| However, this is a purely economic issue; they can trivially be
| upgraded to support grid stability if the pricing scheme will
| pay for it.
|
| - Solar panels are worse: They have no inertia! There _is_ no
| rotating shaft there to cover sub-second usage spikes. That 's
| where complaints about 'renewables causing reduction of grid
| stability' come from, along with issues like domestic solar
| needing to backfeed power through distribution lines and
| transformers that aren't necessarily designed for that.
|
| - But batteries can absolutely help. The kinetic energy of a
| rotating turbine isn't actually that big; it's not that
| expensive to pair a solar panel with a battery to build a grid-
| forming system that acts the same way a kinetic power plant
| would.
| p_l wrote:
| For wind turbines, solar, and batteries, the frequency
| stability is doable through controlling frequency from
| inverters.
|
| Some wind turbines are also internally a hybrid design that
| can dynamically adjust the frequency difference angle both
| for minimal losses in production, but also to provide
| frequency shifting and even artificial demand (i.e.
| essentially using wind power as brake)
| padjo wrote:
| The inertia provided by synchronous generators was in
| essence a natural benefit to the grid that we never
| appreciated previously. But now with more renewables it's
| now something we have to consider and design for. Running a
| grid on 100% non synchronous requires a bit more effort!
| immibis wrote:
| Running the grid requires a lot of effort, period!
| nine_k wrote:
| It almost sounds like running a large motor/generator with a
| large flywheel on it, just for the purposes of frequency
| stabilization and flattening load spikes, may be a good idea
| in a large solar installation. The downside is the mass and
| the bulk, and the cost of copper. The upside is zero reaction
| time.
| rich_sasha wrote:
| It is in fact a thing: https://en.m.wikipedia.org/wiki/Flyw
| heel_storage_power_syste...
|
| I believe (but I am 0% expert) these are _mostly_ but not
| exclusively used for adding inertia to the system, rather
| than real energy storage.
| immibis wrote:
| Close, but you're looking for:
| https://en.wikipedia.org/wiki/Synchronous_condenser
| Matthyze wrote:
| I'll give a somewhat simpler answer than Filligree has. The
| problem with renewable energy sources is that they are
| typically both highly variable and not dispatchable (i.e.,
| controllable). The former leads to supply peaks that can exceed
| transmission capacity or supply lows that require compensating
| generation elsewhere. The latter means that energy generation
| can easily be increased or decreased as required, which is of
| course very helpful for grid management. Dispatchable
| generation can be increased if supply requires it, or multiple
| dispatchable generators can be 'redispatched' to relieve
| congestion in a part of the power grid (by rebalancing
| generation). Power from renewable sources can be decreased
| through curtailment, but that wastes the generated energy.
| rich_sasha wrote:
| Yeah, hence I mention batteries specifically in my question.
| I guess a renewable grid will need a lot of batteries. And
| presumably these are highly dispatchable?
| pjc50 wrote:
| Batteries are the ultimate in dispatchable. Provided
| they're charged.
| Havoc wrote:
| Wild that cloudflare was first to report this. At least first I
| saw.
|
| I guess news networks don't have comparable access to live
| metrics
| dredmorbius wrote:
| Wikipedia has an article following this:
|
| <https://en.wikipedia.org/wiki/2025_European_power_outage>
| WJW wrote:
| Well obviously. Computers use electricity, so if the grid goes
| out so does internet traffic. Local data centers might have
| backup power but local homes and everything in between the homes
| and the data center probably won't.
| jonplackett wrote:
| Genuine question: how is it that Ukraine's electricity grid still
| works to any degree despite being bombed for years and no doubt
| intentionally sabotaged / cyber attacked.
|
| But a whole country's grid can go down like this in an instant?
| luke-stanley wrote:
| For offline messaging over Bluetooth/Wi-Fi without internet or
| central infrastructure, I saw these recommended by ChatGPT Deep
| research and they seem current: Bridgefy (Android & iOS) - direct
| Bluetooth mesh within ~100 m, instant one-to-one & group chat, <1
| min install. Berty (Android & iOS) - E2E-encrypted over
| Bluetooth/Wi-Fi; Briar (Android) - E2E-encrypted over
| Bluetooth/Wi-Fi with Tor fallback. Manyverse (Android, iOS,
| Desktop) - offline-first social feed that syncs posts over
| Bluetooth/Wi-Fi when peers meet. Can anyone vouch for these or
| know better?
| luke-stanley wrote:
| Whoever downvoted, a comment explaining why would be nice, when
| the cell tower generators are out, and there is only phone
| battery left, offline communication options could be useful,
| I'm glad I'm not in that position right now.
| luke-stanley wrote:
| Seems like some people are not interested in honest and
| useful feedback because this got downvoted too! I thought it
| made clear points about an important topic. Well, whoever
| sees fit to do this, I hope you get whatever it is you want.
| croes wrote:
| Maybe because you don't have to ask ChatGPT for what could
| have been a simple web search.
| mbgerring wrote:
| I hope somebody building and selling microgrids and energy
| storage systems is ready to take advantage of this situation.
| Mass deployment of batteries at the grid edge is a great idea for
| a lot of reasons, but none more immediately compelling than "the
| power won't go out if the grid goes down."
| spacebanana7 wrote:
| It should be mandatory for critical infrastructure like ATMs to
| have 24hr batteries installed. And some sort of power backup to
| maintain internet access as well - even if it's just a matter
| of everyone being allocated a small amount of starlink data or
| public wifi.
|
| These things have relatively small costs, but make the system
| much more resilient.
| olyjohn wrote:
| What do you do when the ATMs run out of cash? It's not an
| unlimited supply in them.
|
| You should probably just keep some cash on hand. And keep an
| FM radio and a few batteries on hand.
| scotty79 wrote:
| Maybe the grid upgrade should be to DC to get something hopefully
| more stable?
| TriangleEdge wrote:
| Practical Engineering YouTube channel black start video:
| https://youtu.be/uOSnQM1Zu4w
| nunobrito wrote:
| To remember: in the early hours of this morning there was a huge
| purchase/transfer of monero that increased the price +30% (highly
| unusual) and a few hours later before lunch time we see Iberia
| disconnected from the grid.
|
| Monero is the favorite payment coin when ordering real cyber
| attacks.
| stavros wrote:
| How do we know there was a huge purchase? Monero can't be
| tracked.
| nunobrito wrote:
| Because to buy that much monero so fast is only possible with
| bitcoin, which is traceable by governments and interested
| parties.
|
| An estimated 305 million USD of monero were purchased this
| morning before the grid shutdown. That was a +50% increase on
| the monero price, something that never happened before in the
| last 11 years of the coin.
|
| Quite a record. Quite a coincidence.
| refulgentis wrote:
| How do we know there was a huge purchase? Monero can't be
| tracked.
| sva_ wrote:
| ... You can see trading volume on the markets though, if
| it was bought with something like BTC
|
| https://coinmarketcap.com/currencies/monero/
| refulgentis wrote:
| Thank you!
| ivape wrote:
| How do you think this has to do with Spain though?
| nunobrito wrote:
| Related talk in Berlin 5 months ago detailing how to shutdown
| energy grids using only radio waves:
| https://media.ccc.de/v/38c3-blinkencity-radio-controlling-st...
| pantalaimon wrote:
| Not really relevant, but the talk was in Hamburg
| Eduard wrote:
| yes really relevant/related, as that talk also mentions
| that a sub-1 Hz variation of the nominal 50 Hz is already
| enough to cause blackouts - which is also the case for the
| current blackout:
| https://x.com/Nexuist/status/191687508022891747
|
| the talk discusses further aspects of the European energy
| grid which are relevant for the current situation.
| nicoboo wrote:
| I remember the talk and thought about it when I saw the news.
|
| No conspiracy or whatever, just thinking about the complex
| system which is also vulnerable for some parts.
| nerder92 wrote:
| A quick Google seems to point into another direction:
| https://crypto.news/moneros-50-price-pump-likely-tied-to-330...
| WinstonSmith84 wrote:
| yes (of course) .. It's fascinating how always people love to
| jump to conspiration theories .. If it were a cyberattack
| (and so far it isn't proven at all), it would be nothing less
| than a state sponsored cyber attack to be able to get down
| the electrical grid from 2 EU countries - and obviously no
| "reward" such as crypto would be involved behind the scene
| 1970-01-01 wrote:
| Remember folks, there's about 1% chance of a Carrington event
| occurrence every year. We're simply not ready.
| dark-star wrote:
| can't be ready for everything and anything _shrug_
| guerrilla wrote:
| We can be significantly better prepared for most things.
| dark-star wrote:
| no matter how well we're prepared for anything, we can
| still be significantly better prepared for most things...
| guerrilla wrote:
| No, not realistically. There are physical and economic
| constraints. There's no reason to over-prepare either.
| It's absurd to remain as catastrophically vulnerable as
| we are though.
| xyst wrote:
| Would help if fossil fuel industry would stop pumping out anti-
| climate change propaganda.
| whall6 wrote:
| Fossil fuels increase grid stability?
| ttoinou wrote:
| They're hiding Carrington events risks ?
| 7373737373 wrote:
| It's fascinating to see simulations of power grid disturbances
| propagating due to the limited speed of light/causality:
| https://youtu.be/5vAkdXRyKWM?t=713
| blendo wrote:
| AC is overly complex and too reactive.
| PLenz wrote:
| Carrington event?
| GnarfGnarf wrote:
| There would be a broader area affected.
| xyst wrote:
| > The Portuguese electricity operator earlier said the outage was
| caused by a "rare" atmospheric phenomenon, related to variations
| in temperatures
|
| Global warming/climate change strikes again
| yannvon wrote:
| I experienced it first hand in Madrid. This was much scarier than
| I would have imagined.
|
| News travelled extremely slow: phone coverage was just barely
| enough to receive a couple text messages every 15 minutes or so.
| News spread on the street, I even saw a group of 20 people
| hunched around someone owning a hand-held radio in the streets.
|
| Just before power was restored, things started to get worse, as
| the phone coverage went completely out (presumably batteries were
| depleted). People were in between enjoying the work-free day, and
| starting to worry about how tomorrow would look like if power
| didn't come back.
| MisterTea wrote:
| When the northeast blackout hit in 2003 in NYC I dont remember
| any panic. We still had house phones and they still worked in
| the blackout thanks to telcos being legally obligated to give a
| shit about reliability.
|
| I stopped by a friends house and we then went on a walk. Some
| stores were open and cash was accepted. We hung out later that
| night and had a few beers. The sky was amazing as there was
| next to no light pollution. Next day was totally in the dark as
| well and again, no panic. More beers were enjoyed.
|
| The choice to move to electronic everything without having to
| give a shit about reliability is a failure of modern
| government. Move fast and break society for a dollar.
| toast0 wrote:
| > thanks to telcos being legally obligated to give a shit
| about reliability.
|
| Yeah, they don't need to do that anymore. Around me, enough
| towers have battery backups that I can count on 2 hours of
| coverage when utility power goes out (if it goes out late at
| night or early morning, there's usually coverage until 6-7 am
| when people start waking up and use up the rest of the
| power). I don't have a real landline, but the telco DSL would
| drop instantly with utility power so I don't have big hopes
| and I wasn't willing to pay $60/month to find out.
|
| Around when I moved, stores would pull out the credit card
| imprint machines, but those don't work anymore because cards
| are flat. Cash might work, and I've got some, but I don't
| think many people in my community do; people don't have cash
| for the snack shack we run at my kid's sports, so I doubt
| they have it for restaurants and stores either. And we get
| frequent 2-4 hour power outages, at least one, usually two or
| three per year; and ~ 24 hour outages every few years. The
| snack shack runs during summer where electricity is most
| reliable, but I doubt people stock up on cash in the fall and
| use it all up before spring/summer; they probably just don't
| have any.
| lxgr wrote:
| > stores would pull out the credit card imprint machines,
| but those don't work anymore because cards are flat
|
| It's the other way around: Cards are flat because a carbon
| imprint doesn't afford the merchant any payment guarantee
| by the card issuer anymore anyway. (In other words, the
| "floor limit" above which cards require electronic
| authorization is now zero.)
| kccqzy wrote:
| At that time merchants probably still checked the
| signature on the back of your card against your signature
| on a bill. These days nobody bat an eye when I use my
| unsigned card. I guess at that time a matching signature
| would give the merchant enough confidence to process the
| card offline.
| lxgr wrote:
| The merchant's confidence is irrelevant if it's not
| backed by a guarantee of the scheme (effectively forcing
| the issuer) to pay even in case of fraud.
|
| The people operating these imprinters are sales clerks
| and waitstaff, not graphologists or experts in detecting
| altered physical credit cards. The sophistication of
| fraudsters has also advanced, and as a result, a system
| that might have been good enough in a pinch 20+ years ago
| isn't necessarily good enough today.
|
| That said, in my view there's no excuse to not leverage
| the physical chip present on effectively all credit and
| debit cards these days, which _is_ technically capable of
| making limited autonomous spending decisions even with
| both the issuer and terminal offline in scenarios like
| this. It probably won 't happen without regulatory
| pressure, though.
| Jare wrote:
| I started signing my cards with an all caps "ASK FOR
| PHOTO ID" 30 years ago. It raised a few questions when I
| would travel to the US and use them there, but was never
| refused a transaction.
| p0w3n3d wrote:
| Meanwhile (some) politicians in my country (Poland) and EU
| say that we should limit cash handling or eradicate it at
| all
| Kkoala wrote:
| That sounds like the sensible reaction, at the time at least.
|
| It's interesting to think about and realize how much things
| have changed now though, and how reliant people are on
| everything, and especially their tiktoks etc. working all the
| time.
|
| Some of the panic is likely related to the war in Europe too,
| and especially the general talk about war
| MisterTea wrote:
| > Some of the panic is likely related to the war in Europe
| too, and especially the general talk about war
|
| We were just two years removed from 9/11 so terror talk was
| the first thing that happened. We got that news from AM
| radio in our cars. Still no panic.
| rimeice wrote:
| 2003 was over two decades ago. Most people didn't have mobile
| phones let alone smart phones. I've been in Lisbon today and
| it's surreal being 100% cut off from friends and family back
| home and a big relief power is back on, we've become very
| used to and reliant on seamless instant connection. Our
| mindsets and how we live our lives with that instant
| communication is totally different to 22 years ago.
| moonlanders wrote:
| Similar experience in a town in Madrid's metropolitan area.
|
| Electricty went down (something kind of frequent). My UPS kept
| PC up, and alarm system with sim and small UPS mantained wifi
| up for an hour or so.
|
| Scary moments started when people I was in a call with in
| Portugal texted 'Grid is out'. Later no phone signal nor data.
|
| At first, it might seem people running towards supermarkets an
| overreaction on being without TikTok for a couple of hours, but
| you have to live how scary it is to experience this in Europe's
| current political status to know 60 million people (plus
| industry) in three countries are out of the grid.
|
| If you see Snowden's film (this might not be the most
| trustworthy source) it is exactly how CIA's agent describes the
| feasable attack towards these countries. Again, not a valid
| source, but I'd love to understand if that could be feasable.
| sebastiennight wrote:
| I think this is where F-droid and Briar have a (short-lived in
| this instance) chance to shine. Since Briar allows
| communication between phones without access to the Internet,
| and F-droid allows to direct transmit apps between phones as
| well.
|
| I wonder what similar solutions exist in the iOS ecosystem.
| EugeneOZ wrote:
| My son was at the university outside of Barcelona, and I lost
| contact with him for 6 hours. He was traveling home, and I knew
| that the roads had no working semaphores, and there were dozens
| of incidents...
|
| I lost some years today.
|
| My son is fine, thanks to a random person ("the man with a
| rabbit") who just decided to give a lift to my son and his
| friend to the edge of Barcelona.
| tiborsaas wrote:
| You can track the outage with CloudFlare's traffic radar page
| too:
|
| https://radar.cloudflare.com/es?dateRange=1d
|
| https://radar.cloudflare.com/pt?dateRange=1d
|
| Portugal nearly reached zero.
| 42lux wrote:
| Two days ago someone cut a whole overland power pylon in
| Germany...
| guerrilla wrote:
| Link please?
| 42lux wrote:
| In German: https://www.swr.de/swraktuell/rheinland-
| pfalz/strommast-abge...
| perihelions wrote:
| - _" [Portugese electric network Redes Energeticas Nacionais
| (REN)] said: "Due to extreme temperature variations in the
| interior of Spain, there were anomalous oscillations in the very
| high voltage lines (400 kV), a phenomenon known as 'induced
| atmospheric vibration'. These oscillations caused synchronisation
| failures between the electrical systems, leading to successive
| disturbances across the interconnected European network.""_
|
| - _" The risks posed to electrical systems by big variations in
| atmospheric temperatures are well known in the industry, even if
| it is rare for problems to manifest on this scale."_
|
| - _" "Due to the variation of the temperature, the parameters of
| the conductor change slightly," said Taco Engelaar, managing
| director at Neara, a software provider to energy utilities. "It
| creates an imbalance in the frequency.""_
|
| https://www.theguardian.com/business/2025/apr/28/spain-and-p... (
| _" Spain and Portugal power outage: what caused it, and was there
| a cyber-attack?"_)
| Karawebnetwork wrote:
| What caused it? The Portuguese prime minister, Luis Montenegro,
| said that the issue originated in Spain. Portugal's REN said a
| "rare atmospheric phenomenon" had caused a severe imbalance in
| temperatures that led to the widespread shutdowns.
|
| REN said: "Due to extreme temperature variations in the interior
| of Spain, there were anomalous oscillations in the very high
| voltage lines (400 kV), a phenomenon known as 'induced
| atmospheric vibration'. These oscillations caused synchronisation
| failures between the electrical systems, leading to successive
| disturbances across the interconnected European network."
|
| https://www.theguardian.com/business/2025/apr/28/spain-and-p...
| guerrilla wrote:
| So, what you're saying is that literal bad vibes caused it...
| ;)
| Eduard wrote:
| "Conductor gallop" is one of several proposed causes.
|
| https://en.m.wikipedia.org/wiki/Conductor_gallop
| bethekidyouwant wrote:
| It's concerning they still don't know the cause I've certainly
| never experienced a power outage where we didn't know what it
| was
| avree wrote:
| What? Every single major power outage on this list
| https://en.wikipedia.org/wiki/List_of_major_power_outages has
| taken between a day to a week to diagnose. You can read about
| the list of detailed theories if you click through to any of
| them, here's an example:
| https://en.wikipedia.org/wiki/2020_Sri_Lankan_blackouts
|
| Even your minor power outages take 30 minutes to an hour for
| your local power company to determine what failed.
| jmorenoamor wrote:
| They have data and probable causes, but nobody will tell
| until forensic studies take place, which seems a good
| decission. No need to spread speculations.
|
| A cold start from zero generation took 6 to 10 h. No
| casualties, hospitals and crtitical infrastructure kept
| working all day.
|
| Whatever the cause, this has been a great test of the
| emergency protocols.
| MaoSYJ wrote:
| Power back at 20:30 approx in some Madrid province locations at
| this point most of Madrid should have power.
| Mortiffer wrote:
| Power is back on some parts of Lisbon
| ksajadi wrote:
| Apparently Europ network came close to shutdown due to frequency
| drop as the result:
| https://x.com/shellenberger/status/1916914966663434493?s=46
| leumon wrote:
| Except that's not true: an auto shutdown only occurs at 47.5
| Hz, at least in Germany, some countries have the limit even
| lower: https://de.m.wikipedia.org/wiki/Unterfrequenz
| celsoazevedo wrote:
| Is that a reliable source? The guy started blaming this on
| renewables hours ago even though no one seems to know exactly
| what happened.
| geye1234 wrote:
| Relevant:
|
| https://www.pv-magazine.com/2025/04/22/spain-hits-first-week...
| timhigins wrote:
| One guy simulated Australia's electrical grid for 3 years to show
| that the entire grid could run off of >98% renewable energy using
| the installed renewables with just a small amount of storage
| (e.g. 5 hours of average demand, ~10% of demand over the entire
| period).
|
| To me this indicates that in many energy markets where renewables
| are sufficiently built out, the only factor for why we aren't
| using them more is the storage capacity and grid infrastructure
| to handle their variability -- and instead just running stable
| but dirty energy systems.
|
| There are a lot of cool mechanical grid systems like gravity
| batteries (e.g. weights on a pulley system) [2] or compressed or
| liquid air storage systems [3] which provide cheaper storage at
| higher capacities and durations than electric batteries
|
| 1. https://reneweconomy.com.au/a-near-100-per-cent-renewable-
| gr... 2. https://www.youtube.com/watch?v=trA5s2iGj2A 3.
| https://www.youtube.com/watch?v=fjERw-Ol-_s
| worldsayshi wrote:
| It makes intuitive sense to me that a small amount of storage
| would make a big difference. Outside of peak use there has to
| be a huge amount of potential effect that never gets generated
| because lack of demand at that time.
___________________________________________________________________
(page generated 2025-04-28 23:00 UTC)