[HN Gopher] NOAA deploys new generation of AI-driven global weat...
___________________________________________________________________
NOAA deploys new generation of AI-driven global weather models
Author : hnburnsy
Score : 144 points
Date : 2025-12-17 22:32 UTC (3 days ago)
(HTM) web link (www.noaa.gov)
(TXT) w3m dump (www.noaa.gov)
| margalabargala wrote:
| I am dearly hoping that they are using the current "AI" craze to
| talk up the machine learning methods they have presumably been
| using for a decade at this point, and not that they have actually
| integrated an LLM into a weather model.
| username223 wrote:
| Same. I hope this was written by hardened greybeards who have
| dedicated their lives to weather prediction and atmospheric
| modeling, and have "weathered" a few funding cycles.
| akdev1l wrote:
| inb4 it's actually an intern maintaining a 3000+ line
| markdown file
| RHSeeger wrote:
| I can see it now The following snippet
| highlights the algorithm used to determine <thing>
| ```fortran .....
| idontwantthis wrote:
| Hopefully they weren't all forced out this year. The NOAA had
| massive cuts.
| trueismywork wrote:
| NCAR is being dismantled as we speak.
| derbOac wrote:
| I suspect the names of those perpetrating this kind of
| destruction will become synonymous with ignorance and
| intellectual cowardice.
| sigmar wrote:
| Graphcast (the model this is based on) has been validated in
| weather models for a while[1]. It uses transformers, much like
| LLMs. Transformers are really impressive at modeling a variety
| of things and have become very common throughout a lot of ML
| models, there's no reason to besmirch these methods as
| "integrating an LLM into a weather model"
|
| [1] https://github.com/google-deepmind/graphcast
| lynndotpy wrote:
| A lot of shiny new "AI" features being shipped are language
| models being placed where they don't belong. It's reasonable
| to be skeptical here, not just because of the AI label, but
| especially for the troubled history of neural-network based
| ML methods for weather prediction.
|
| Even before LLMs got big, a lot of machine learning research
| being published were models which underperformed SOTA (which
| was the case for weather modeling for a long time!) or models
| which are far far larger than they need to be (e.g. this [1]
| Nature paper using 'deep learning' for aftershock prediction
| being bested by this [2] Nature paper using one neuron.
|
| [1] https://www.nature.com/articles/s41586-018-0438-y
|
| [2] https://www.nature.com/articles/s41586-019-1582-8
| sdenton4 wrote:
| Not all transformers are LLMs.
| lynndotpy wrote:
| Yes, that is not in contention. Not all transformers are
| LLMs, not all neural networks are transformers, not all
| machine learning methods are neural networks, not all
| statistical methods are machine learning.
|
| I'm not saying this is an LLM, margalabargala is not
| saying this is an LLM. They only said they hoped that
| they did not integrate an LLM into the weather model,
| which is a reasonable and informed concern to have.
|
| Sigmar is correctly pointing out that they're using a
| transformer model, and that transformers are effective
| for modeling things other than language. (And,
| implicitly, that this _isn't_ adding a step where they
| ask ChatGPT to vibe check the forecast.)
| brookst wrote:
| "I hope these experts who have worked in the field for
| years didn't do something stupid that I imagine a novice
| would do" is a reasonable concern?
| lynndotpy wrote:
| Yes, it is a very reasonable concern.
|
| The quoted NOAA Administrator, Neil Jacobs, published at
| least one falsified report during the first Trump
| administration to save face for Trump after he claimed
| Hurricane Dorian would hit Alabama.
|
| It's about as stupid as replacing magnetic storage tapes
| with SSDs or HDDs, or using a commercial messaging app
| for war communications and adding a journalist to it.
|
| It's about as stupid as using .unwrap() in production
| software impacting billions, or releasing a buggy and
| poorly-performing UX overhaul, or deploying a kernel-
| level antivirus update to every endpoint at once without
| a rolling release.
|
| But especially, it's about as stupid as putting a
| language model into a keyboard, or an LLM in place of
| search results, or an LLM to mediate deals and sales in a
| storefront, or an LLM in a $700 box that is supported for
| less than a year.
|
| Sometimes, people make stupid decisions even when they
| have fancy titles, and we've seen myriad LLMs inserted
| where they don't belong. Some of these people make
| intentionally malicious decisions.
| Legend2440 wrote:
| It's not an LLM, but it is genAI. It's based on the same idea
| of predict-the-next-thing, but instead of predicting words it
| predicts the next state of the atmosphere from the current
| state.
| adamweld wrote:
| It is in fact one of the least generalized forms of "AI" out
| there. A model focused solely on predicting weather.
| astrange wrote:
| "gen" stands for "generative". If you read the GenCast
| paper they call it a generative AI - IIRC it's an
| autoregressive GNN plus a diffusion model.
|
| Which is surprising to me because I didn't think it would
| work for this; they're bad at estimating uncertainty for
| instance.
| Majromax wrote:
| > Which is surprising to me because I didn't think it
| would work for this; they're bad at estimating
| uncertainty for instance.
|
| FGN (the model that is 'WeatherNext 2'), FourCastNet 3
| (NVIDIA's offering), and AIFS-CRPS (the model from ECMWF)
| have all moved to train on whole ensembles, using a
| cumulative ranked probability score (CRPS) loss function.
| Minimizing the CRPS minimizes the integrated square
| differences of the cumulative density function between
| the prediction and truth, so it's effectively teaching
| the model to have uncertainty proportional to its
| expected error.
|
| GenCast is a more classic diffusion-based model trained
| on a mean-squared-error-type loss function, much like any
| of the image diffusion models. Nonetheless it performed
| well.
| lukeschlather wrote:
| The GraphCast paper says "GraphCast is implemented using GNNs"
| without explaining that the acronym stands for Graph Neural
| Networks. It contrasts GNNs to the " convolutional neural
| network (CNN)" and "graph attention network." (GAN?) It doesn't
| really explain the difference between GAN and a GNN. I think
| LLMs are GANs. So no, it's not an LLM in a weather model, but
| it's very similar to an LLM in terms of how it is trained.
| astrange wrote:
| > I think LLMs are GANs.
|
| They aren't, but both of them are transformer models.
|
| nb GAN usually means something else (Generative Adversarial
| Network).
| lukeschlather wrote:
| I used GAN to mean graph attention network in my comment,
| which is how the GraphCast paper defines transformers.
| https://arxiv.org/pdf/2212.12794
|
| I was looking at this part in particular:
|
| > And while Transformers [48] can also compute arbitrarily
| long-range computations, they do not scale well with very
| large inputs (e.g., the 1 million-plus grid points in
| GraphCast's global inputs) because of the quadratic memory
| complexity induced by computing all-to-all interactions.
| Contemporary extensions of Transformers often sparsify
| possible interactions to reduce the complexity, which in
| effect makes them analogous to GNNs (e.g., graph attention
| networks [49]).
|
| Which kind of makes a soup of the whole thing and suggests
| that LLMs/Graph Attention Networks are "extensions to
| transformers" and not exactly transformers themselves.
| astrange wrote:
| Oh yeah, GNN (graph neural network) is the common term,
| "graph attention network" is pretty confusing because a
| GAN is a totally different architecture.
|
| (Well, not necessarily architecture. Training method?)
| optimalsolver wrote:
| You're absolutely right! That _was_ a category 5. Thanks for
| pointing that out.
| Workaccount2 wrote:
| Interestingly, while this model is based on a Google Deepmind AI
| weather model, it's based on a model from 2023 (GraphCast) rather
| than the WeatherNext 2 model which has grabbed headlines as of
| late. I'd imagine it takes a while to integrate and test
| everything, explaining the gap.
| sigmar wrote:
| I've been assuming that, unlike graphcast, they have no
| intention to make weathernext 2 open source.
| tndl wrote:
| That seems to be the case from what I've heard.
| Majromax wrote:
| Google Research and Google DeepMind also build their models for
| Google's own TPU hardware. It's only natural for them, but
| weather centres can't buy TPUs and can't / don't want to be
| locked to Google's cloud offerings.
|
| For Gencast ('WeatherNext Gen', I believe), the repository
| provides instructions and caveats (https://github.com/google-
| deepmind/graphcast/blob/main/docs/...) for inference on GPU,
| and it's generally slower and more memory intensive. I imagine
| that FGN/WeatherNext 2 would also have similar surprises.
|
| Training is also harder. DeepMind has only open-sourced the
| inference code for its first two models, and getting a working,
| reasonably-performant training loop written is not trivial.
| NOAA hasn't retrained its weights from scratch, but the fine-
| tuning they did re: GFS inputs still requires the full training
| apparatus.
| username223 wrote:
| Whatever it is, it seems like it might be roughly competitive
| with ECMWF, the state of the art when it comes to global weather
| models: https://www.epic.noaa.gov/ai/eagle-verification/
|
| A quick search didn't turn up anything about the model's skill or
| resolution, though I'm sure the data exists.
| ryuuchin wrote:
| They run at 0.25 degree resolution (same as ECMWF AIFS models).
| apawloski wrote:
| I've seen the Microsoft Aurora team make a compelling argument
| that weather is an interesting contradiction of the AI-energy-
| waste narrative. Once deployed at scale, inference with these
| models is actually a sizable energy/compute _improvement_ over
| classical simulation and forecasting methods. Of course it is
| energy intensive to train the model, but the usage itself is more
| energy efficient.
| klysm wrote:
| It's definitely interesting that some neural nets can reduce
| compute requirements, but that's certainly not making a dent on
| the LLM part of the pie.
| lukeschlather wrote:
| Sam Altman has made a lot of grandiose claims about how much
| power he's going to need to scale LLMs, but the evidence
| seems to suggest the amount of power required to train and
| operate LLMs is a lot more modest than he would have you
| believe. (DeepSeek reportedly being trained for just $5M, for
| example.)
| lovich wrote:
| I saw a claim that DeepSeek had piggybacked off of some
| aspect of training that ChatGPT had done, and so that cost
| needed to be included when evaluating DeepSeek.
|
| This training part of LLMs is still mostly Greek to me, so
| if anyone could explain that claim as true or false and the
| reasons why, I'd appreciate it
| TomatoCo wrote:
| So LLMs predict the next token. Basically, you train them
| by taking your training data that's N words long and, for
| X = 1 to N, and optimizing it to predict token X using
| tokens 1 to X-1.
|
| There's no reason you couldn't generate training data for
| a model by getting output from another model. You could
| even get the probability distribution of output tokens
| from the source model and train the target model to
| repeat that probability distribution, instead of a single
| word. That'd be faster, because instead of it learning to
| say "Hello!" and "Hi!" from two different examples, one
| where it says hello and one where it says hi, you'd learn
| to say both from one example that has a probability
| distribution of 50% for each output.
|
| Sometimes DeepSeek said it's name is ChatGPT. This could
| be because they used Q&A pairs from ChatGPT for training
| or because they scraped conversations other people posted
| where they were talking to ChatGPT. Or for unknown
| reasons where the model just decided to respond that way,
| like mixing up some semantics of wanting to say "I'm an
| AI" and all the scraped data referring to AI as ChatGPT.
|
| Short of admission or leaks of DeepSeek training data
| it's hard to tell. Conversely, DeepSeek really went hard
| into an architecture that is cheap to train, using a lot
| of weird techniques to optimize their training process
| for their hardware.
|
| Personally, I think they did. Research shows that a model
| can be greatly improved with a relatively-small set of
| high quality Q&A pairs. But I'm not sure the cost
| evaluation should be influenced that much, because the
| ChatGPT training price was only paid once, it doesn't
| have to be repaid for every new model that cribs its
| answers.
| lukeschlather wrote:
| I think the claim that DeepSeek was trained for $5M is a
| little questionable. But OpenAI is trying to raise $100B
| which is 20,000 times as much as $5M. Though even at $1B
| I think it's probably not that big a deal for Google or
| OpenAI. My feeling is they can profit on the prices they
| are charging for their LLM APIs, and that the dominant
| compute cost is inference, not training. Though obviously
| that's only true if you're selling billions of dollars
| worth of API calls like Google and OpenAI.
|
| OpenAI has had $20B in revenue this year, and it seems
| likely to me they have spent considerably less than that
| on compute for training GPT5. Probably not $5M, but quite
| possibly under $1B.
| throwaway613745 wrote:
| "it's more efficient if you ignore the part where it's not"
| TallGuyShort wrote:
| It's more efficient anyway because the inference is what
| everyone will use for forecasting. Researchers will be using
| huge amounts of compute to develop better models, but that's
| also currently the case, and it isn't the majority of weather
| simulation use.
|
| There's an interesting parallel to Formula One, where there
| are limits on the computational resources teams can use to
| design their cars, and where they can use an aerodynamic
| model that was previously trained to get pretty good outcomes
| with less compute use in the actual design phase.
| apawloski wrote:
| I mean that's cute, but surely you can add up the two parts
| (single training plus globally distributed inference) and
| understand that the net efficiency would be an improvement?
| Majromax wrote:
| > "it's more efficient if you ignore the part where it's not"
|
| Even when you include training, the payoff period is not that
| long. Operational NWP is enormously expensive because high-
| resolution models run under soft real-time deadlines; having
| today's forecast tomorrow won't do you any good.
|
| The bigger problem is that traditional models have decades of
| legacy behind them, and getting them to work on GPUs is
| nontrivial. That means that in a real way, AI model training
| and inference comes at the expense of traditional-NWP
| systems, and weather centres globally are having to strike
| new balances without a lot of certainty.
| brookst wrote:
| I suggest reading up on fixed costs vs variable costs and why
| it is generally preferable to push costs to fixed.
|
| Assuming you're not throwing the whole thing out after one
| forecast, it is probably better to reduce runtime energy
| usage even if it means using more for one-time training.
| threemux wrote:
| This jumped out at me as well - very interesting that it
| actually reduces necessary compute in this instance
| derbOac wrote:
| The press statement is full of stuff like this:
|
| "Area for future improvement: developers continue to improve
| the ensemble's ability to create a range of forecast
| outcomes."
|
| Someone else noted the models are fairly simple.
|
| My question is "what happens if you scale up to attain the
| same levels of accuracy throughout? Will it still be as
| efficient?"
|
| My reading is that these models work well in other regions
| but I reserve a certain skepticism because I think it's
| healthy in science, and also because I think those ultimately
| in charge have yet to prove reliable judges of anything
| scientific.
| Majromax wrote:
| > My question is "what happens if you scale up to attain
| the same levels of accuracy throughout? Will it still be as
| efficient?"
|
| I've done some work in this area, and the answer is
| probably 'more efficient, but not quite as spectacularly
| efficient.'
|
| In a crude, back-of-the-envelope sense, AI-NWP models run
| about three orders of magnitude faster than notionally
| equivalent physics based NWP models. Those three orders of
| magnitude divide approximately evenly between three
| factors:
|
| 1. AI-NWP models produce much sparser outputs compared to
| physics-based models. That means fewer variables and
| levels, but also coarser timesteps. If a model needs to run
| 10x as often to produce an output every 30m rather than
| every 6h, that's an order of magnitude right there.
|
| 2. AI-NWP models are "GPU native," while physics-based
| models emphatically aren't. Hypothetically running physics-
| based models on GPUs would gain most of an order of
| magnitude back.
|
| 3. AI-NWP models have fantastic levels of numerical
| intensity compared to physics-based NWP models since the
| former are "matrix-matrix multiplications all the way
| down." Traditional NWP models perform relatively little
| work per grid point in comparison, which puts them on the
| wrong (badly memory-bandwidth limited) side of the roofline
| plots.
|
| I'd expect a full-throated AI-NWP model to give up most of
| the gains from #1 (to have dense outputs), and dedicated
| work on physics-based NWP might close the gap on #2.
| However, that last point seems much more durable to me.
| AStrangeMorrow wrote:
| Obviously much simpler Neural Nets, but we did have some models
| in my domain whose role was to speed up design evaluation.
|
| Eg you want to find a really good design. Designs are fairly
| easy to generate, but expensive to evaluate and score.
| Understand we can quickly generate millions of designs but
| evaluating one can take 100ms-1s. With simulations that are not
| easy to GPU parallelize. We ended up training models that try
| to predict said score. They don't predict things perfectly, but
| you can be 99% sure that the actual score designs is within a
| certain distance of said score.
|
| So if normally you want to get the 10 best design out of your 1
| million, we can now first have the model predict the best 1000
| and you can be reasonably certain your top 10 is a subset of
| these 1000. So you only need to run your simulation on these
| 1000.
| trillic wrote:
| Heuristical branch-and-bound
| esafak wrote:
| And an LLM can be more energy efficient than a human -- and
| that's precisely when you should use it.
| brewtide wrote:
| That's precisely when, (insert hand wavy motion), we should
| use any of this.
| xphos wrote:
| If its more energy efficient it is doing something different
| there is no guarantee that its more accurate long term.
| Weather is horrible difficult to predict and we are only just
| alright at it. If LLM are guessing at the same rate we are
| calculating but I am doubtful
| xphos wrote:
| Well that was a failed response opps. I am just cautious
| because while transformers get the random guessing right
| you can get the right answer statistically but fail on
| accuracy improvement long term. Clearly this model does
| better than the current model but extending it to be even
| better seems basically intractable besides throw more data
| at it but what if it derived the wrong model you simply
| cannot actually know
| gwern wrote:
| There's also the efficiency argument from new capability: even
| a tiny bit better weather forecast is highly economically
| valuable (and saves a lot of wasted energy) if it means that 1
| city doesn't have to evacuate because of an erroneous hurricane
| forecast, say. But how much would it cost to do that with the
| rivals? I don't know but I would guess quite a lot.
|
| And one of the biggest ironies of AI scaling is that where
| scaling succeeds the most in improving efficiency, we realize
| it the least, because we don't even think of it as an option.
| An example: a Transformer (or RNN) is not the only way to
| predict text. We have scaling laws for n-grams and text
| perplexity (most famously, from Jeff Dean et al at Google back
| in the 2000s), so you can actually ask the question, 'how much
| would I have to scale up n-grams to achieve the necessary
| perplexity for a useful code writer competitive with Claude
| Code, say?' This is a perfectly reasonable, well-defined
| question, as high-order n-grams could in theory write code
| without enough data and big enough lookup tables, and so it can
| be answered. The answer will look something like 'if we turned
| the whole earth into computronium, it still wouldn't be
| remotely enough'. The efficiency ratio is not 10:1 or 100:1 but
| closer to [?]:1. The efficiency gain is _so_ big no one even
| thinks of it as an efficiency gain, because you just couldn 't
| do it before using AI! You would have humans do it, or not do
| it at all.
| kingkawn wrote:
| Now that we've saved infinite energy all carbon tax credit
| markets are unnecessary! Big win for the climate! _pollutes_
| hammock wrote:
| > even a tiny bit better weather forecast is highly
| economically valuable (and saves a lot of wasted energy) if
| it means that 1 city doesn't have to evacuate because of an
| erroneous hurricane forecast
|
| Here is the NOAA on the improvements:
|
| > 8% better predictions for track, and 10% better predictions
| for intensity, especially at longer forecast lead times --
| with overall improvements of four to five days.(1)
|
| I'd love someone to explain what these measurements mean
| though. Does better track mean 8% narrower angle? Something
| else? Compared to what baseline?
|
| And am I reading this right that that improvement is measured
| at the point 4-5 days out from landfall? What's the typical
| lead time for calling an evacuation, more or less than four
| days?
|
| (1)https://www.noaa.gov/news/new-noaa-system-ushers-in-next-
| gen...
| inciampati wrote:
| To have a competitive code writer with ngrams you need more
| than to "scale up the ngrams" you need to have a corpus that
| includes all possible codes that someone would want to write.
| And at that point you'd be better off with a lossless full
| text index like an r-index. But, the lack of any
| generalizability in this approach, coupled with its markovian
| features, will make this kind of model extremely brittle.
| Although, it would be efficient. You just need to somehow
| compute all possible language before hand. tldr; language
| models really are reasoning and generalizing over the domain
| they're trained on.
| ryuuchin wrote:
| These are available on Weatherbell[1] (which requires a
| subscription) now except for the HGEFS ensemble model which I'm
| guessing will probably be added later. AIGFS is on tropical
| tidbits which should be free for some stuff[5]. I believe some of
| the research on this is mentioned in these two[2][3] videos from
| NOAA weather partners site. They also talk about some of the
| other advances in weather model research.
|
| One of the big benefits of both the single run (AIGFS) and
| ensemble (AIGEFS) models is the speed and (less) computation time
| required. Weather modeling is hard and these models should be
| used as complementary to deterministic models as they all have
| their own strengths and weaknesses. They run at the same 0.25
| degree resolution as the ECMWF AIFS models which were introduced
| earlier this year and have been successful[4].
|
| Edit: Spring 2025 forecasting experiment results is available
| here[6].
|
| [1] https://www.weatherbell.com/
|
| [2] https://www.youtube.com/watch?v=47HDk2BQMjU
|
| [3] https://www.youtube.com/watch?v=DCQBgU0pPME
|
| [4] https://www.ecmwf.int/en/forecasts/dataset/aifs-machine-
| lear...
|
| [5] https://www.tropicaltidbits.com/analysis/models/
|
| [6]
| https://repository.library.noaa.gov/view/noaa/71354/noaa_713...
| tndl wrote:
| Really exciting to see NOAA finally make some progress on this
| front, but the AIGFS suite likely won't outperform ECMWF's AIFS
| suite any time soon. The underlying architecture between AIFS
| and GraphCast/AIGFS is pretty similar (both GNNs), so there
| won't likely be a model-level improvement. And most of ECMWF's
| edge lies in its superior 4DVar data assimilation process.
| AIGFS is still being initialized on NOAA's hybrid 4DEnVar
| assimilation process as far as I understand it, which is still
| not as good as straight up 4DVar unfortunately.
| kkylin wrote:
| Came here to say this -- looks like the data assimilation is
| still done the "old fashioned" way. I wonder how long that
| will last?
| Frostlike1417 wrote:
| There are multiple efforts and a good number of VC working
| on AI DA system. DA is fundamentally a hand-crafted
| optimization process just like NN. I once reimplemented an
| EnKF in pytorch and it works amazingly fast. But our
| observations are so dirty and sparse. ECMWF tuned their
| system so well. NOAA definitely has potential being even
| better, but no hope any soon future IMHO.
| padjo wrote:
| What does AI refer to here? Presumably weather models have been
| using all sorts of advanced machine learning for decades now, so
| what's AI about this that wasn't AI previously?
| tomww wrote:
| They're using a graph neural network. From the article - "The
| team leveraged Google DeepMind's GraphCast model as an initial
| foundation and fine-tuned the model using NOAA's own Global
| Data Assimilation System analyses".
|
| > so what's AI about this that wasn't AI previously?
|
| The weather models used today are physics-based numerical
| models. The machine learning models from DeepMind, ECMWF,
| Huawei and others are a big shift from the standard, numerical
| approach used for the last decades.
| padjo wrote:
| Thanks, I guess my assumption that ML was widely used in
| forecasting is wrong.
|
| So are they essentially training a neural net on a bunch of
| weather data and getting a black box model that is expensive
| to train but comparatively cheap to run?
|
| Are there any other benefits? Like is there a reason to
| believe it could be more accurate than a physics model with
| some error bars?
| Majromax wrote:
| > Are there any other benefits? Like is there a reason to
| believe it could be more accurate than a physics model with
| some error bars?
|
| Surprisingly, the leading AI-NWP forecasts are more
| accurate than their traditional counterparts, even at large
| scales and long lead times (i.e. the 5-day forecast).
|
| The reason for this is not at all obvious, to the point I'd
| call it an open question in the literature. Large-scale
| atmospheric dynamics are a well-studied domain, so physics-
| based models essentially have to be getting "the big stuff"
| right. It's reasonable to think that AI-NWP models are
| doing a better job at sub-grid parameterizations and local
| forcings because those are the 'gaps' in traditional NWP,
| but going from "improved modelling of turbulence over urban
| and forest areas" (as a hypothetical example) to
| "improvements in 10,000 km-scale atmospheric circulation 5
| days later" isn't as certain.
| counters wrote:
| Machine learning _has_ been widely used in weather
| forecasting, but in a different way than these models.
| Going back to the 1970's, you never just take the output of
| a numerical weather model and call it a forecast. We know
| that limitations in the models' resolution and
| representation of physical processes lead to huge biases
| and missed details that cause the forecast to disagree with
| real world observations. So a standard technique has been
| to post-process model outputs, calibrating them for station
| observations where available. You don't need super complex
| ML to really dramatically improve the quality or skill of
| the forecast in this manner; typically multiple linear
| regressions with some degree of feature selection and other
| criteria will capture most of the variance, especially when
| you pool observation stations together.
| bee_rider wrote:
| Do these ML models replace the numerical approach completely?
| A lot of numerical methods are iterative. If the ML model can
| produce a good initial guess, it might make convergence of an
| iterative process quite a bit quicker...
| NetMageSCW wrote:
| Reading the article could have helped with this.
| carabiner wrote:
| > Presumably weather models have been using all sorts of
| advanced machine learning for decades now
|
| This isn't actually true, unless you're considering ML to be
| just linear regression, in which case we have been using "AI"
| for >100 years. "Advanced ML" with NN is what's being showcased
| here.
| curt15 wrote:
| AI refers to whatever would have been called "Machine Learning"
| five years ago.
| minimaxir wrote:
| This ambiguity resulted in some very funny drama on Bluesky:
| https://bsky.app/profile/nws.noaa.gov/post/3ma754dbtuj2t
| carabiner wrote:
| Holy shit lmao. Like the wokest tumblr crowd focused into a
| laser for 2025. How do these guys get through life?
| Exhausting existence.
| CalChris wrote:
| Neil Jacobs, Ph.D
|
| This makes me skeptical that it isn't just politicized Trumpian
| nonsense.
| luc_ wrote:
| I wonder if the new models consider land use change and emissions
| from aggressive datacenter development and model training...
| adamredwoods wrote:
| Is there a primer for reading these files?
|
| https://www.nco.ncep.noaa.gov/pmb/products/gens/
|
| https://www.emc.ncep.noaa.gov/emc/pages/numerical_forecast_s...
| jasonmarks_ wrote:
| No, not really. If you are just looking to work with the data
| you want to read about extracting from grib2 format. One of the
| faster ways off the ground is to use the Pywgrib2_s python
| package and iterate against the model files using python to
| extract the fields that are interesting. I have a container on
| docker hub that has pywgrib compiled with all its dependencies
| if you want to tinker.
|
| pywgrib
| https://www.cpc.ncep.noaa.gov/products/people/lxu/cookbook/a...
| containerized
| https://hub.docker.com/repository/docker/jmarks213/container...
| counters wrote:
| It's far, far simpler for users to simply use eccodes[1],
| particularly as implemented in xarray[2].
|
| [1]: https://github.com/ecmwf/eccodes [2]:
| https://docs.xarray.dev/en/stable/index.html
| jasonmarks_ wrote:
| That is another viable path, no more no less.
| jasonmarks_ wrote:
| These look like staging MVP releases with a full rollout planned
| for the future. They are only including a few parameters at every
| 6 hours which is barely interesting to anyone with their feet on
| the ground.
| DaveZale wrote:
| how about working with Weather Underground to validate predicted
| weather at ground level? Here in Southern CO would be a perfect
| place to try this. Weather Underground has thousands of volunteer
| backyard weather stations, including mine.
|
| I understand that aviation safety is certainly a primary concern
| for NWS/NOAA but ground level forecasts are also very important
| for public safety.
| cramcgrab wrote:
| Apparently it seems to be impossible with these files and the
| best AI right now to answer the simple question, will it rain in
| midtown Manhattan tomorrow?
| defrost wrote:
| Take an umbrella if you're concerned.
|
| What is possible is to know with near certainty the rough
| tonnage of water that will fall across a wide area grain region
| in an upcoming week.
|
| Useful for the reliable production of grain (timing seeding,
| harvesting, spraying, etc) in the millions of tonnes.
| ryandrake wrote:
| Protip: Any time you read "AI" in a news article, substitute the
| phrase "faster, more numerous, and confidently incorrect." I
| don't think we need "confidently incorrect" weather models. Who
| is asking for this?
| tndl wrote:
| These models actually outperform traditional methods on many
| fronts, including accuracy a lot of the time. They are
| technically generative AI models, but they're definitely not
| LLMs.
| klysm wrote:
| If it were LLM I'd agree, but that's not the case here.
| Incipient wrote:
| How well do these predict extremes/outliers? Given that I expect
| these are more "ML" type models, these are somewhat limited to
| interpolation, rather than extrapolation?
| gvkhna wrote:
| Working on AI driven weather predictions to make money on
| prediction markets. The accuracy of WeatherNext 2 is astounding.
|
| It may be a fools errand but makes for an extremely interesting
| research project. http://climatesight.app if you're interested in
| climate markets.
| cl42 wrote:
| Have you considered launching your own weather prediction
| market instead?
|
| Parametric insurance, energy traders, etc could be good
| markets.
| gvkhna wrote:
| No I haven't but I think the lack of liquidity as a chicken
| and egg is a huge barrier to entry in these markets
| specifically. They are small right now but there are climate
| derivatives on the Chicago mercantile exchange so this isn't
| a new concept I think.
|
| Could you tell me more? https://discord.gg/HPpN42SKQ
| lisp2240 wrote:
| All these years later and we still don't have the minute-accurate
| forecasts that Dark Sky had before Apple shut it down. Living in
| the future sucks.
| Tempest1981 wrote:
| My friend's kids would ride their bikes to school in the
| morning, and on rainy days, check Dark Sky to find the driest
| time window. It was usually quite accurate.
| vitorgrs wrote:
| If you mean minute-accurate forecast for the next 4-6 years...
| That's called Nowcasting, and yes, it exists. Bing Weather have
| it, ACCU Weather as well. Rain viewer too. I believe Google
| already implemented on Pixel Weather at least.
|
| IMO the best of these are Bing Weather and Rain Viewer, both
| provide rich maps showing where the rain it's going and all
| too. And how much.
| lisp2240 wrote:
| No.
| axismundi wrote:
| Did you mean 4-6 hours?
| NetMageSCW wrote:
| Apple definitely broke something when they incorporated Dark
| Sky into their weather ecosystem and it isn't nearly as good in
| my locality.
| Terretta wrote:
| Apple purports to still have this, but it is indeed less
| reliable than Dark Sky.
|
| However, the author of Carrot for iOS implemented his own
| flavor of this* and it's remarkably decent.
|
| * According to Gruber interview around the same time Carrot
| introduced an entire Broadway musical about the conflict
| between the Carrot AI and her Maker (the dev). Which, while
| made with AI, is rather more listenable than the typical
| weather app.
| matt3210 wrote:
| Does ai mean LLM here or just normal software?
| jmclnx wrote:
| Where I am the last couple of years, the EU model out performed
| the US model. The local stations tend to show both when sever
| weather is on its way to the area.
|
| We know how the current admin views science and with the cuts to
| NOAA done this year, I expect that trend to continue and widen.
| At least where I am, we get to see both.
| JoeDaDude wrote:
| I know someone pursuing a degree in meteorology at well known
| university for the subject and I asked that person if they are
| being taught about these and other AI weather models, about how
| they work, how to evaluate them for effectiveness, etc.
|
| The answer: AI is not even covered, at least at the undergrad
| level. This is just a sample of one, so are any other
| universities educating future meteorologists on this subject?
| NetMageSCW wrote:
| Are meteorologists even the right people to be training on how
| to produce and improve better modeling of weather?
| tylervigen wrote:
| This is big news. For decades, NOAA's model has basically just
| been a huge Fortran physics simulation. Now they are making the
| leap to AI.
|
| I suspect the nail in the coffin was the hurricane season, where
| NOAA's model was basically beat by every major AI model. [0]
|
| The GFS also just had its worst year in predicting hurricane
| paths since 2005. [1] That's not a trend you want to continue.
|
| [0] https://arstechnica.com/science/2025/11/googles-new-
| weather-...
|
| [1] https://www.local10.com/weather/hurricane/2025/11/03/this-
| hu...
___________________________________________________________________
(page generated 2025-12-20 23:01 UTC)