[HN Gopher] Pirate Weather: A free, open, and documented forecas...
___________________________________________________________________
Pirate Weather: A free, open, and documented forecast API
Author : calebegg
Score : 439 points
Date : 2023-01-10 19:15 UTC (3 hours ago)
(HTM) web link (pirateweather.net)
(TXT) w3m dump (pirateweather.net)
| hectormalot wrote:
| I've used https://open-meteo.com/ before and I think it's the
| same type of open data being exposed.
|
| These types of projects are great for stuff like home automation.
| I'm using to to improve my predictions for power generation (PV)
| and consumption (heat pump). Planning to is ergst to optimize
| home battery charging in the future.
|
| (Disclaimer; open sourced a small go library for open meteo, but
| otherwise not affiliated)
| meteo-jeff wrote:
| Hi, creator of open-meteo.com here! I am using a more wide
| range of weather models to better cover Europe, Northern Africa
| and Asia. North America is covered as well with GFS+HRRR and
| even weather models from the Canadian weather service.
|
| In contrast to pirate weather, I am using compressed local
| files to more easily run API nodes, without getting a huge AWS
| bill. Compression is especially important for large historical
| weather datasets like ERA5 or the 10 km version ERA5-Land.
|
| Let me know if you have any questions!
| ushakov wrote:
| The tech diagram on AWS is insane. This is what I call vendor
| lock-in porn
|
| I'm wondering how much your monthly bill is?
| gen220 wrote:
| Which part is insane?
|
| Put another way, how would you implement this without self-
| managing pieces like nginx/apache, rabbitmq/kafka, or mongodb
| on a compute instance?
|
| The pieces of managed infra they're depending on are all pretty
| interchangeable from one cloud provider to another. It's not
| the cheapest way to solve the problem if you discount the cost
| of management, upgrades, etc. But if you factor those costs in,
| it's quite competitive.
| ushakov wrote:
| I would go to cloud providers, which can offer me managed
| rabbitmq, kafka or mongodb
|
| Certainly, I wouldn't get any vendor-specific services like
| Lambda or SNS (in this example)
| aequitas wrote:
| The Lambda functions will just be running your business
| logic code. Only specific part Lambda provides is the glue.
| Which is easy to replace with any other cloud or self
| hosted alternative.
| alexander0042 wrote:
| This is like the 5th iteration of that darn diagram, and I
| still can't quite get it right. The other comments are correct,
| the underlying setup isn't that complicated, and it wouldn't be
| too tricky to build it for another cloud, since all that's
| really happening is a Python script reading a NetCDF file. The
| real perk to building everything using a server-less approach
| was that everything scaled down to 1 user (me), and then has
| scaled up beautifully as more people have signed up.
|
| The monthly bill isn't great, but has been manageable through
| donations so far. I'm handling about 10 million API calls/
| month, so lots of Lambda invocations, but they're all
| thankfully very short with tiny outbound data transfer totals,
| and luckily inbound is free. If the costs ever get out of hand
| is to throttle the request rate down, but there's still a few
| optimizations I can do on the AWS side that should help (ARM
| here I come!).
| shepherdjerred wrote:
| This is a pretty standard looking diagram. It probably isn't
| very expensive to run.
|
| AWS diagrams are pretty intimidating until you've built a few
| things with several AWS services.
|
| The benefit is that a lot of maintenance work is taken care of
| for you, and your costs can be low if you don't need a lot of
| compute.
| jacurtis wrote:
| I am an Infrastructure Architect (aka "Cloud Architect") so I
| design cloud systems like this on the daily. The "vendor lock-
| in" argument always makes me laugh. Its the #1 thing I hear all
| day long.
|
| This diagram is actually pretty simple. It looks worse than it
| is. All it uses are Lambdas (serverless functions), S3 buckets
| (object storage), and SNS (broadcast/push queues). There
| appears to be one traditional server in there with EFS, which
| is just an elastic file system.
|
| All of these systems have equivalents in all the major cloud
| providers. So if the builder of this wanted to move to GCP or
| Azure, they are not really locked to AWS. This can all be built
| in another cloud.
|
| Now, could you do it in a day? No. Assuming they are building
| it with Infrastructure as Code (such as Terraform) then they
| would need to convert the provider and change resource blocks.
| But this akin to refactoring in a codebase. Its work, but its
| not terribly difficult. Then they point it to their new cloud
| and run `terraform apply`.
|
| There is almost no way to entirely remove vendor lock-in. The
| closest you could come is by designing everything yourself on
| bare metal servers and renting those from a cloud provider. So
| instead of using a managed queue system, you run some sort of
| messaging queue on the server. Then you host files on the
| server's filesystem, and you run the "lambdas" as applications
| on the server. But that almost causes more headaches than you
| save or solve for.
|
| I look at Cloud Providers as similar to cell phone providers. I
| know people who live in fear of being locked into a contract
| with Verizon or something. But really, what are you going to
| do? You will always need a cell phone. The only other real
| choice is AT&T or maybe Sprint/TMobile. How often are you
| really going to switch and what are you really gaining by doing
| so? Energy spent worrying about being "locked in" to a cloud
| vendor is energy wasted. Yeah you can move from AWS to Azure or
| GCP. But that's about it. What do you gain by switching?
| Probably almost nothing. They are all pretty comparable at this
| point in reliability, features, and price (GCP is the slight
| laggard here, but not by much). If Google calls your company
| and offers you a huge discount to switch, you could still do
| it. Aside from that, there's minimal incentive to do so.
|
| There are a few weird services that AWS has for example that
| might be considered "lock-in" services. This would be things
| like AWS Snowball or AWS Groundstation. These don't have
| comparable systems on other platforms. In the case of Snowball
| you probably have so much data on AWS that just transferring
| data would take months (or even years) which could be
| considered a form of lock-in.
|
| tl;dr - This is a very tame arch diagram. A few lambdas, s3
| buckets, and messaging queues, all of which have comparable
| services on all major clouds. There isn't significant vendor
| lock-in, this could be rebuilt fairly easily (assuming they
| used IaC) on any major cloud provider.
| ushakov wrote:
| Hello
|
| > This diagram is actually pretty simple
|
| The diagram looks like an ad
|
| > All it uses are Lambdas (serverless functions), S3 buckets
| (object storage), and SNS (broadcast/push queues)
|
| Do you actually _need_ all of this or do you use it because
| Amazon tells you to? I know for instance you cannot use
| Amazon SES without also using S3 and Lambda
|
| > So if the builder of this wanted to move to GCP or Azure,
| they are not really locked to AWS. This can all be built in
| another cloud
|
| You're saying that I cannot move to other cloud provider
| without my existing code becoming useless?
|
| > Assuming they are building it with Infrastructure as Code
| (such as Terraform) then they would need to convert the
| provider and change resource blocks
|
| What about the data pipelines and business logic?
|
| > There is almost no way to entirely remove vendor lock-in
|
| There is: avoiding vendor-specific APIs altogether
|
| > Closest you could come is by designing everything yourself
| on bare metal servers and renting those from a cloud provider
|
| I don't have to. There are things like Railway, Fly.io,
| PlanetScale, Supabase, Upstash, Minio, which can work without
| locking me in
|
| > What do you gain by switching?
|
| Freedom
|
| > There isn't significant vendor lock-in, this could be
| rebuilt fairly easily (assuming they used IaC) on any major
| cloud provider
|
| You are contradicting yourself
| manigandham wrote:
| > _" I know for instance you cannot use Amazon SES without
| also using S3 and Lambda"_
|
| You can absolutely use SES without S3 and Lambda. I've used
| it many times in various projects.
| ushakov wrote:
| Not on the receiving side
|
| https://docs.aws.amazon.com/ses/latest/dg/receiving-
| email-co...
| nicwolff wrote:
| As that page makes clear, SES can hand off incoming mail
| to a Lambda, or to S3 - or to SNS which can deliver it to
| any HTTP endpoint, or e-mail address, or text it to your
| phone for that matter.
| manigandham wrote:
| What do you expect SES to do with your mail after
| receiving it? S3 and Lambda are optional delivery
| locations, amongst other actions.
| jacurtis wrote:
| Exactly this. It receives the email, now what? You need
| to run some code on it and so the way to do that is one
| of the compute services. AWS isn't forcing you to do
| anything here.
|
| 99.9% of SES users I promise are only sending mail
| anyway. You aren't forced to have Lambdas or anything
| else to send mail.
| jacurtis wrote:
| Wow, I struck a nerve. I'm happy to address these points
| however.
|
| > The diagram looks like an ad
|
| Lol, Its an architecture diagram. You could swap the AWS-
| specific icons for generic ones I suppose, and it wouldn't
| change anything. It is fulfilling its purpose of explaining
| how all the services connect together to deliver the
| product. Just because it is an AWS Lambda icon doesn't mean
| you couldn't make it an Azure Function icon instead and
| perform the same goal. You're just too focused on hating
| AWS here to see the forest through the trees.
|
| > Do you actually need all of this or do you use it because
| Amazon tells you to? I know for instance you cannot use
| Amazon SES without also using S3 and Lambda
|
| This is just a standard event-driven architecture. There's
| really nothing exciting to see here. Data comes in, it gets
| stored somewhere (S3), that triggers an event (SNS), a
| compute service (lambda in this case, but could be
| anything, even a standard VM, bare metal or anything else)
| picks up the task and processes it or performs a job on the
| data and stores it, it triggers another event, something
| else picks it up, and so forth. This isn't an AWS design,
| its just an event driven architecture design and this is
| how they work.
|
| SES can be used standalone. It doesn't require Lambda or S3
| like you postulate. There are only a few times AWS requires
| something else and its usually Cloudwatch or S3 and these
| will sometimes be the destinations required for specific
| types of logging or auditing and so forth.
|
| AWS is forcing you to do nothing here. The creator chose
| this stuff. But I assume they chose it to keep it free.
| Most of this will survive under the free tier until the
| project becomes massive. If it inches over the free tier,
| it will still be cheap. That's probably the incentive for a
| lot of this.
|
| > You're saying that I cannot move to other cloud provider
| without my existing code becoming useless?
|
| Correct your code is your code. Think about a lambda. In
| this scenario data comes from NOAA and is put in a bucket.
| You write a serverless function that takes that data out of
| the bucket, reformats the NOAA data into your proprietary
| format and puts it in another bucket. The code that does
| that is written in Go, Python, C++, Java, or whatever you
| want. If written correctly, it accepts data, processes it,
| and outputs it. So if you move to another cloud provider
| your code would still work. It might run in an Azure
| Function instead of an AWS Lambda, but that doesn't matter.
| Your code does the same thing you don't need to throw it
| out.
|
| > What about the data pipelines and business logic?
|
| Hard to say without more information. But its possible BI
| dashboards need to be changed to point to the new service
| and stuff like that. Sure. Again, you're not switching
| clouds in an afternoon. But the point its the next cloud
| system is eerily similar. Its not like you have to rebuild,
| it would be more of a refactor.
|
| > There is: avoiding vendor-specific APIs altogether
|
| Possibly. But if built correctly your code doesn't need to
| be aware of the environment it is in. But there might be
| cases where you interact with the services directly, like
| downloading from S3. This would change with the next
| provider possibly (although most actually have
| S3-compatible APIs). But most of your application will not
| directly interact with the cloud, it will interact with the
| services. So for example you use RDS to host a managed
| Postgres db, but your application is just interacting with
| postgres, not AWS here. But you're right there might be
| some scenarios that use vendor-specific APIs.
|
| > I don't have to. There are things like Railway, Fly.io,
| PlanetScale, Supabase, Upstash, Minio, which can work
| without locking me in
|
| I fail to see how these are any different than tying
| yourself to a product like S3 or Lambda. In many ways,
| these solutions are TRUE vendor lock-in, with all the
| vendor specific APIs that you live in fear of. Fly.io is a
| PaaS, which is going to be way harder to move away from
| than switching from AWS to Azure. PlanetScale, Minio, and
| Upstash are literally no different than equivalent products
| in AWS/GCP/Azure. I guess you could host the instance that
| runs these products ondifferent clouds and it would be the
| same, but you're still tying yourself to something. The
| risk of tying yourself to a startup is higher than tying
| yourself to Amazon/Microsoft/Google. You're trading one
| evil for another, in most ways you are actually losing
| freedom with these not gaining it.
|
| > You are contradicting yourself
|
| My point is that there isn't as much vendor lock-in that
| people fear. Yes it exists, but don't live your life in
| fear of it. Yes you would need to refactor stuff here and
| there. But the same architecture diagram we saw for AWS is
| basically the same one that would exist in Azure or GCP.
| The underlying tools don't change. The marketing names and
| logos change, which clearly bothers you, but the underlying
| system doesn't change.
| kempbellt wrote:
| Also a cloud engineer. I use a similar setup professionally
| and for various personal projects.
|
| For someone who isn't familiar with standing up cloud
| resources the diagram can look overwhelming but once you
| play around with AWS for a bit, most of the resources you
| see are fairly boilerplate.
|
| VPC is essentially just setting up a virtual LAN for your
| resources. S3 is being used as an API mediator to NOAA.
| CloudFront is a CDN for your API Gateway. Lambdas run your
| logic. API Gateway triages api requests to lambdas, and a
| couple other services act as mediators.
|
| There is _some_ vender lock-in in that everything here is
| built on AWS, but all the major cloud providers have
| similar /equivalent services. If you decided to move to GCP
| or Azure you could probably move the entire stack in a few
| days (maybe more depending on how much your lambdas use AWS
| specific internal APIs).
|
| If vendor lock-in is a really big concern for you, you can
| run everything on an EC2 instance running Ubuntu instead.
| That way you can just spin up your service on another
| Ubuntu instance in another datacenter, or locally, or
| whatever.
|
| Soooo, yes. There is some vendor lock-in here, but not
| much.
|
| To answer your cost question. I run a very similar setup
| for personal projects and I rarely exceed AWS's free tier
| for most services. On a high-usage month it's around $85.
| It isn't super optimized. I could make it cheaper and
| nearly free if I put in the work.
|
| That said, cost for a service like this scales very
| proportionally to usage. For example, AWS API Gateway can
| process 1 million requests for free before they start
| asking you for money. If the service becomes super popular
| we'd likely see the "Buy me a coffee" button promoted a
| little more and eventually you may see a paid tier as an
| option, but as it is, it's probably pretty affordable to
| run.
| alexander0042 wrote:
| I'm the dev behind this, and really appreciate all the
| insight from actual cloud professionals! Your guess here
| is spot on, I designed it so that I could more or less
| fit in the free tier with exactly one user, with costs
| scaling pretty linearly afterwards. There are a few more
| optimizations I could do, but it's honestly pretty
| impressive how much traffic (I'm at about 10 million
| calls/ month) can be (somewhat) cheaply handled using AWS
| whalesalad wrote:
| Easy enough to move to other clouds if it was defined in
| something like terraform or even cloudformation. Could even
| move out of the cloud to your own bare metal services pretty
| easily, since the various components are so well defined. AWS
| makes the most sense though as stated in the docs because a lot
| of the data being processed is in S3, and reading data from S3
| inside of most AWS services is free.
| manigandham wrote:
| It's rather simple, just messy to look at. It's a timer that
| triggers a cluster that processes raw data files and uploads
| the JSON to a server with a reliable file system. Then a Lambda
| function processes API requests to read that JSON and serve a
| response.
|
| "Lock-in" is just a trade-off like any other engineering
| decision, same as what programming language or API schema you
| choose. AWS is massive and reliable, and these services are
| cheap and widely available so I don't see much of an issue
| here.
| ourmandave wrote:
| How does this differ from NOAA weather.gov api?
|
| https://www.weather.gov/documentation/services-web-api
|
| Or is this a friendlier overlay of their interface?
| cantaloupe wrote:
| Under "Weather Sources"
|
| > All weather data comes from the AWS open data program
| https://registry.opendata.aws/collab/noaa/. This is a fantastic
| program, since it is more reliable than the NOAA distribution,
| and means there are no data transfer changes!
| alexander0042 wrote:
| Good question! It's the same underlying models, but three key
| differences: 1. Pirate Weather returns data following the Dark
| Sky syntax, as opposed to the custom NWS format. 2. Pirate
| Weather has global coverage, the NWS API is only for the US. 3.
| The NWS one uses human forecasters to come up with their
| forecasts, compared to Pirate Weather's use of raw model
| results.
| earthscienceman wrote:
| Only on HN would the NWS format be described as 'custom'.
| vermilingua wrote:
| Only in the US would you assume your govt's format is
| 'default'.
| hnarn wrote:
| I know literally nothing about weather data
| standardization, but if the format was -- for the sake of
| argument -- unique to the US for example and another more
| popular format was used either internationally or de-facto
| by third parties, I'd say it qualifies as a "custom"
| format.
| byhemechi wrote:
| appears to pretty much US only, unfortunately. it's -5 where i am
| right now, but this claims that it is actually -17 (a very big
| difference)
| DrBenCarson wrote:
| @OP you've exceeded your daily cognito limit
| waynecochran wrote:
| I like having an open API. Every weather site eventually becomes
| trash as it is loaded with obnoxious ads. Maybe the answer it
| just to have numerous weather front ends.
| davchana wrote:
| I usually use forecase.weather.gov They provide a nice textual
| weather page, with English, for next 7 or 8 days [1]. But it is
| tiring to open 3 or 4 bookmarks, so I thought to consume that as
| json & make a single page html app with ajax requests. Its still
| in progress, I spent few minutes yesterday. The surprising thing
| was (I was ready to fetch the html text & parse it, difficult), I
| simply changed one parameter to JSON & I got json [2]. I didn't
| find any easy to find documentation about json endpoint. My plan
| is to have city abbreviations on top in horizontal menu, then
| each click fetches json, puts it into div below.
|
| 1.
| https://forecast.weather.gov/MapClick.php?lat=37.78&lon=-122...
|
| 2.
| https://forecast.weather.gov/MapClick.php?lat=37.98&lon=-120...
| vel0city wrote:
| That is cool the web front end exposes itself as JSON as well,
| good tip.
|
| FWIW, they also have a pretty decent API. Its based around
| zones though, which you'd need to look up. So from that lat and
| lon, you'd get the zone from:
|
| https://api.weather.gov/points/37.7827,-120.38
|
| Using the zone information, you can get to a forecast:
|
| https://api.weather.gov/gridpoints/STO/72,24/forecast
|
| If you're wanting the current observations, you'd pick a
| station for that grid such as MOUC1 and go to its
| observations/latest endpoint:
|
| https://api.weather.gov/stations/MOUC1/observations/latest
|
| The API is in a JSON-LD format so its got a lot of links to
| related topics in the actual JSON payload. Looking at the JSON
| can make it somewhat easy to feel out what you need. The
| documentation is here:
|
| https://www.weather.gov/documentation/services-web-api
| ChrisArchitect wrote:
| Related discussion about Merry Sky, a Dark Sky replacement built
| on this
|
| https://news.ycombinator.com/item?id=34155191
| NoboruWataya wrote:
| > Why "PirateWeather"? I've always thought that the HRRR model
| was pronounced the same way as the classic pirate "ARRR". Also,
| there is one company out there that thinks APIs can be
| copyrighted, which might apply here.
|
| This answers the first question I had upon seeing the name, which
| was, is it free, open, documented _and legal_? Based on the above
| the answer seems to be "probably".
|
| Very commendable effort, and I hope the project can last. It
| seems to be very difficult to maintain a free and reliable
| weather API so hopefully the dev is not biting off more than he
| can chew.
| kxrm wrote:
| > is it free, open, documented _and legal_?
|
| I think it says something that we live in a world where NOAA
| data could be seen as an underground or less than legal means
| of getting the weather.
|
| Perhaps AccuWeather has been successful in their campaign to
| keep free weather data as obscure as possible.
| qup wrote:
| I think it's more about it having "pirate" in the name that
| makes one question its legality.
| alexander0042 wrote:
| I'm the dev behind this, so always great to hear things like
| this! I really struggled to try to come up with a name. My
| first thought was "Bright Ground" (opposite of Dark Sky), but
| that seemed a little too on the nose. Luckily, the legal aspect
| of this (I'm in the clear!) was pretty well settled after the
| final Oracle v. Google case, but at the time was a big enough
| deal that it seemed relevant, and the HRRR model was another
| plus. I should update that README though, since it's now very
| definitely legal!
| calebegg wrote:
| Lol I named my version "Bright Earth" as an opposition to
| dark sky (brightearth.app)
| jacurtis wrote:
| I agree, PirateWeather seems like a misbrand here. When I read
| it, i thought it was stealing weather data or something. On the
| web, the term "pirate" generally doesn't mean good things. This
| name almost implies that it is illegal or something.
|
| I'm imagining designing a software product around this and
| presenting it to a C-Level, explaining that we use
| "PirateWeather" and I think I'm going to get grilled with lots
| of questions and concerns based on the name alone.
|
| This is a good service and should be "branded" with a better
| name. Maybe a play on the whole DarkSky name like LightSky or
| "Sunset" which works exceptionally well since DarkSky was
| sunset by Apple. Maybe StarrySky, LateSky, NewSky.
|
| I am usually someone who says that names don't matter as much
| as people think they do, but PirateWeather just seems like a
| huge hit in the wrong direction. But the product is solid so
| maybe it can survive despite the name.
| holoduke wrote:
| Maybe most of the good products come out of places without
| too many C level types :)
| alexander0042 wrote:
| I'm the dev for this, so can shed some light on this! Weather
| Underground was a pretty out there choice- I guess the kind
| of people who like putting together weather APIs aren't great
| at naming things. I considered "Bright Ground", and still
| have the domain name for it, so maybe it'd be worth spinning
| up another API endpoint/ branding that uses that and has a
| more commercial focus, keeping Pirate Weather as the free and
| open source branding.
| nivenkos wrote:
| Weather Underground would be hilarious.
| ummonk wrote:
| That would actually be a great idea.
| guntherhermann wrote:
| FWIW I love the name Pirate Weather. Thanks for the work!
| jameshart wrote:
| I guess at least it's a step up from the time someone was
| trying to brand a grassroots weather data collection effort
| and decided to name it after a terrorist organization.
|
| I'm a massive fan of - and indeed contribute data to - the
| weather underground project, but the naming has always made
| me a little uncomfortable.
| wendyshu wrote:
| https://pirateweather.net/apis isn't loading for me
| hk1337 wrote:
| I cannot load the apis page right now. Is this just current
| weather data or do you offer historical weather data too?
| pbmango wrote:
| This is great. Can endorse LuckGrib for IoS folks as well. Great
| UI and can pull NOAA and EU models. https://luckgrib.com/. Not
| FOSS but worth the $20 I paid once several years ago. Popular
| among wind sports enthusiasts many of who are hobbyist
| meteorologists.
| jron wrote:
| I was really hoping https://merrysky.net/ had a graphical
| forecast similar to weather underground's 10 day. I've yet to
| find anything else that quickly shows everything you'd want to
| know in a single image. Even the mouseover timeline is
| perfection. It is so good and seemingly exclusive to WU that I
| sometimes wonder if they hold a patent for it.
| chrisweekly wrote:
| Related tangent: Windy (iOS app) does a remarkably good job
| providing multiple data sources and terrific dataviz.
| rsync wrote:
| "... similar to weather underground's 10 day ..."
|
| Came here, like I come to every weather API/tool discussion, to
| ask the same thing ... I would really like a less spammy, less
| bloated 10-day forecast a la WU.
|
| This comes close, however - there's a nice 7-day lookahead ...
| is it possible that WU is just fudging days 8-9-10 and no
| _real_ data is available beyond 7 days ?
| PraetorianGourd wrote:
| The _presentation_ is impressive, but the forecast is often the
| raw output from various models, and thus not entirely reliable.
| For example, I live in a valley where most forecast models
| smooth over giving more snow/cold than we ever get.
| bleomycin wrote:
| I have a similar issue living on a hill with the windy.com
| forecasts. Have you had success finding any forecast
| source/app that's accurate for your microclimate? Everyone
| always points to forecastadvisor.com but that's always
| pulling from some airport 20 miles away for me totally
| irrelevant.
| traceroute66 wrote:
| Try Ventusky (https://www.ventusky.com/)
| xd1936 wrote:
| I agree with you, but I've also found the https://windy.com
| timeline along the bottom to be similarly informative, with
| ECMWF data to boot.
| DotaFan wrote:
| I really like this free Forecase API too https://developer.yr.no/
| shagie wrote:
| One of the things that I've found on yr.no that I haven't seen
| on any other weather app is the classification of cloud level.
|
| For example:
| https://www.yr.no/en/details/table/2-4407066/United%20States...
|
| You will see not only the cloud cover as an overall percentage,
| but also the different levels. A 50 in the middle is very
| different than 50 in low in terms of what you can expect that
| day (for photographs).
| pca2 wrote:
| Can confirm pirateweather.net's forecast API working as a drop-in
| replacement for DarkSky's. I was able to fix a soon to be broken
| DarkSky bitbar script by just replacing the URLs/API keys to
| pirateweather's in under 5 minutes.
| nerdponx wrote:
| I assume there's no recourse for iOS users?
| shagie wrote:
| If you are using the current version of iOS, the new weather
| app _rocks_ in terms of functionality (IMHO better than the
| previous version of Dark Sky).
|
| If you are on an old version of iOS and mourn the loss of
| DarkSky app, I personally like MyRadar - it has some very
| nice features and I used in tandem with DarkSky in the past.
|
| If you are after writing an iOS app, you already have an
| Apple developer account allowing for 500k calls/month.
|
| If you are after writing an iOS app and want to use DarkSky
| rather than WeatherKit, Pirate Weather should also be a drop
| in replacement.
| macintux wrote:
| The new iOS Weather app certainly has lots of
| functionality, lots of data, but I mourn the clean,
| streamlined Dark Sky interface.
|
| Plus the temperature map, unlike the radar map, no longer
| allows you to change to a future date/time to see how the
| temperature changes are going to progress across a larger
| region, which I always found interesting.
|
| Very unfortunate.
| antihero wrote:
| Also the live map just looks weird and interpolated now.
| rootusrootus wrote:
| iOS Weather is pretty nice. But I can't quite love it,
| because it frequently tells my wife and I different values
| for current temperature, all while proclaiming to be giving
| data for the same small city. I can manually program the
| city in and get it to do the same thing. Hard to trust it.
|
| Also, it's almost always wrong with rain predictions. But
| so was Dark Sky, so that's fair.
| dv_dt wrote:
| Yup it's pretty bad - I kept comparing it to the nws
| mobile site and the iOS weather app was frequently wrong
| vs the nws and real outcomes
| aendruk wrote:
| Do you run BitBar on iOS?
| jagged-chisel wrote:
| Any particular reason you couldn't do the same? Just replace
| URLs.
| justusthane wrote:
| I think they're asking about continuing to use the DarkSky
| app. In which case, no. The new Weather app in iOS 16 does
| add some DarkSky features, but I don't think it's as good.
| dzhiurgis wrote:
| > I was able to fix a soon to be broken DarkSky bitbar script
|
| Please fork it for us lazy ones
| earthscienceman wrote:
| It's going to be a tangential comment but I work in science
| research that's adjacent to weather forecasting and I find the
| political/technical jockeying that is happening with forecasting
| to be fascinating. It's a nexus of capitalism, federal government
| spending, politics, and technology that has very real
| implications for individual Americans.
|
| In summary: horrible oversight by the federal govt (read,
| congress) of our technical/scientific forecasting resources means
| that our forecasting ability is extremely fragmented and poorly
| organized. This has lead to a lot of companies being essentially
| resellers of public data. These companies claim to create a lot
| of value added products ('cleaner APIs', 'minutecasts', etc etc)
| that are either scientifically dubious or technically simple and
| then these companies walk away with huge profits based on being a
| portal to government data.
|
| It's so American it is almost laughable, all while the European
| ECMWF eats our lunch in terms of accuracy even for the CONUS.
| I've discussed this on technical internet forums often enough
| that I can practically already write the replies to my own
| comment. "What's the problem with that?" etc et al. But the
| reality of it is that it's emblematic of how politically broken
| the US is, in particular with regards to the agencies in charge
| of scientific products and funding. Not to mention the concrete
| problems with the forecast products themselves.
|
| Anyway. Good luck pirate weather and godspeed. Information was
| meant to be free and open, especially the forecast. It's such a
| laughably simple problem that could/should be so easily solved
| but, alas, there is money to be made!
| jaybo_nomad wrote:
| A thousand times this!
| billiam wrote:
| Have any nerdy Congresspeople wanted to solve this ridiculous
| inefficiency in recent years. Seems like a natural "the
| Europeans kick our ass at forecasting" message would go over
| well and potentially negate the lobbying from Weather Channel
| and others with a vested interest in perpetuating this stupid
| situation.
| counters wrote:
| Yes, Congress passed the Weather Research and Forecasting
| Innovation Act back in 2017 which not only gave massive
| funding injection to model development activities, data
| procurement, and more, but also formalized the new next-
| generation community modeling initiatives which are focused
| on tackling the dominance of the EC models in the 2-5 day
| forecast space.
|
| The diagnosis of the problems of the American forecast
| modeling community here is based on flawed premises. There
| are three major factors which led to the ECMWF leap-frogging
| the US in day-ahead forecasting capability. The first is the
| consolidated investment in supercomputing resources; the
| WRFIA tackles this by earmarking a much larger allocation of
| funding for NOAA's next gen supercomputer, but this still
| pales in comparison to ECMWF investments.
|
| The second factor is the fragmentation of the research and
| operational weather modeling communities due to the divergent
| investment from NOAA and USAF in the 90's and 2000's; USAF in
| conjunction with NCAR sponsored the development of the WRF
| model which was widely adopted by the research community.
| NOAA continued investing the GFS lineage of models. The
| bifurcation of these communities slowed down the ability to
| matriculate advances in model developments to operations, and
| this was exacerbated by an old, closed-off approach by NOAA
| which made it extraordinarily difficult to run and develop
| the GFS on anything other than NOAA hardware.
|
| Finally, the ECMWF went all-in on 4DVAR data assimilation in
| the late 90's, whereas the American community pursued a
| diversity of other approaches ranging from 4DVAR to ensemble
| Kalman filters. 4DVAR necessitates advances to core weather
| model software (e.g. you need to write a model's adjoint or
| its tangent linear in order to actually use 4DVAR) adn the
| US' failure to adopt it led, imo to a "double edged sword"
| effect of (a) failing to provide impetus to greatly improve
| the US modeling software suite and supporting tools, and (b)
| being a worse assimilation technique unless advanced EnsKF
| techniques are employed using very large ensembles of models
| (expensive).
|
| The other problem as others have pointed out is that there is
| no accountability in the US private sector weather market.
| Virtually every player is re-transmitting raw model guidance
| or statistical post-processed forecasts using DiCast, _maybe_
| with a some manual tweaking of forecast fields by humans. But
| this is not transparent, and many companies - if we're being
| charitable, here - are not honest about what they're actually
| doing to produce their forecasts. Put another way - there's a
| lot of BS claims out there, and it seems that investors have
| been more than happy to fund it over the past few years.
| earthscienceman wrote:
| I disagree that my (extremely broad) diagnosis of the
| problems with forecasting in America is based on flawed
| premises. You've provided thorough, correct, and important
| details here but my comment was aimed at the broader HN
| audience, not on writing the central argument for a
| discussion on the historical timeline. I think what I said,
| "Horrible oversight by the federal govt (read, congress) of
| our technical/scientific forecasting resources means that
| our forecasting ability is extremely fragmented and poorly
| organized." is a very concise summary of the things you've
| laid out here.
|
| As for the details in your comments, the only thing I
| disagree with strongly is the comment about investment in
| computing. While sufficient computing resources are central
| to good forecasting, the lack of investment by NOAA in
| computing (I sit at NOAA) is a red herring. ECMWF is
| significantly better than either of the two available
| American forecasts because they are just better at what
| they do, all around. In particular with respect to data
| assimilation. I've sat at meetings with ECMWF forecasters
| who have asked for access to my in-situ data products and
| their pipeline is as simple as "point us to the data
| please". Their data assimilation pipeline is so much more
| sophisticated and thorough that catching up on that alone
| would be a huge huge leap. Mind you, not just '4DVAR' the
| methodology, but literally the way that the community finds
| and integrates observational data.
|
| ECMWF, the organization, is quite literally structured to
| strictly accomplish the goal of 'improve the forecast'.
| Whereas, again broadly, the American institutions are much
| more a congolemerate of associated researchers doing
| individual science projects while small teams work on
| specific segments of the forecast. Yes, we are attempting
| to fix this. No, we haven't fixed it yet.
|
| This is not to say I don't think we should fund computing
| or that computing won't help. But we are quite literally
| 5-10 years of research behind on multiple fronts.
| counters wrote:
| The thing about the computing is that it has impacted the
| culture surrounding NWP model development within the
| American modeling community. At ECMWF, there is capacity
| in the primary systems to support R&D, so the total cost
| to the community to maintain this capability is much
| lower than in the US where everything is fragmented. If
| there was greater capacity for researchers to run GFS on
| the limited number of systems with first-class (or any)
| support, it may have helped consolidate the community.
|
| Totally acknowledge that there are other takes here. And
| I have a bit of skepticism about how much EPIC will
| really achieve and what it can do to resolve these
| issues. But I don't necessarily agree that the science at
| EC is 5-10 years ahead of the American community's.
| What's matriculated R2O is definitely a few years ahead,
| of us, especially for medium-range forecasting. But the
| US likely still maintains a competitive edge in
| mesoscale/rapid refresh forecasting, and even though
| we've lost key developers to the private sector recently,
| the NBM seems (in my admittedly limited experience) to
| perform favorably to similar products out of ECMWF or
| other European national weather services.
|
| Your point about ECMWF being fundamentally structured
| with the singular goal of improving the forecast is super
| important - I 100% agree with that, and the US has yet to
| do much of anything to address this.
| jmbwell wrote:
| 100%.
|
| There is enormous data available at https://www.weather.gov/ at
| no charge, including hourly and weekly forecasts, spot
| forecasts, radar (multiple layers, with/without animation) and
| satellite (multiple layers, with/without animation), plus storm
| watches, hurricane info, historical data, climate data...
|
| I guess it's nice that apps can do things like advise me that
| it might start raining in a few minutes, but often by the time
| I see those alerts, the water on my head has alerted me anyway.
|
| All other weather apps, it seems to me, are for little more
| than tracking my location and serving me relevant ads.
| baq wrote:
| As someone living in continental Europe there's one thing I
| haven't found anything remotely close to NHC quality forecasts.
| Curious what you think of that institution?
| none_to_remain wrote:
| Is there a linguistic shift happening here where this is no
| longer "a service implementing the same API as Dark Sky", rather
| it is "an API"
| nimbius wrote:
| what ive always wanted is a competent API for weather.gov, but
| theres just no impetus to get NOAA to drop their convoluted
| pseudo-XML gobbledygook.
___________________________________________________________________
(page generated 2023-01-10 23:00 UTC)