[HN Gopher] OCR at Edge on Cloudflare Constellation
___________________________________________________________________
OCR at Edge on Cloudflare Constellation
Author : willhackett
Score : 72 points
Date : 2023-07-03 11:01 UTC (12 hours ago)
(HTM) web link (notes.willhackett.com)
(TXT) w3m dump (notes.willhackett.com)
| piotrrojek wrote:
| What's the best for OCR currently that I could deploy for myself?
| Like time I tried Tesseract 5 I wasn't too impressed.
| [deleted]
| robertknight wrote:
| EasyOCR is a popular project if you are in an environment where
| you can use run Python and PyTorch
| (https://github.com/JaidedAI/EasyOCR). Other open source
| projects of note are PaddleOCR
| (https://github.com/PaddlePaddle/PaddleOCR) and docTR
| (https://github.com/mindee/doctr).
| aidenn0 wrote:
| FWIW, I just tried EasyOCR on some sans-serif text and
| Tesseract5 absolutely blew it out of the water. The only
| thing Tesseract got wrong that EasyOCR (sometimes) got right
| was uppercase Is ("I") were recognized pretty much 100% of
| the time as vertical bars ("|"), but since my text of
| interest is extremely unlikely to have any vertical bar
| characters, a simple sed post-processing stage fixed that.
|
| - Tesseract5 *demolished EasyOCR on paragraph detection,
| getting that 100% on the 10 pages I checked. EasyOCR missed
| most of the paragraph breaks.
|
| - Tesseract got most of the punctuation correct, EasyOCR only
| got apostrophes and two double-quotes (out of 14) correct.
| Every single period, comma, exclamation mark, and hyphen was
| missing or wrong, as were most of the double-quotes. Some
| question marks were recognized, but with garbage after them.
|
| - In general EasyOCR seems to just add in square closing
| brackets ("]") where none are
| driscoll42 wrote:
| It depends on your setup and use cases. There's three major
| considerations:
|
| * What language are you trying to OCR? And only language or
| also things like math symbols? * Do you have a GPU or not? *
| Are you trying to OCR handwriting or typed words?
|
| I explored OCRing English documents from the 1960s that were
| primarily typed, though some handwriting. I tried out
| PaddleOCR, TrOCR, Tesseract, EasyOCR, and kerasOCR for FOSS,
| and then Google, Amazon, and Microsoft for paid.
|
| To be clear, the paid solutions beat the FOSs ones handsdown,
| no question. However for FOSS I found that TrOCR was the best
| for both typed and handwritten, however for typed, it was
| closely followed by tesseract, but for handwriting TrOCR was by
| far the best with all the others basically being worthless.
| However, TrOCR took ~200x longer even on GPU than Tesseract on
| CPU (Tesseract if fastttt, even more if you parallalerize it).
| Tesseract isn't the best, but it's the best all around, it's
| the one the Internet Archive uses.
|
| Need to write up a blog on this. And the docTR looks
| interesting, I'm going to check that out.
| jamil7 wrote:
| The "Why Edge Compute?" section compares running OCR on a
| Cloudflare Worker against on-device but the first two advantages
| also apply to running on-device (low latency and low cost). Is
| this a mistake in the article or do I not understand correctly?
| willhackett wrote:
| I apologise, I probably could have been clearer. One key
| advantage is being able to improve the model without needing to
| re-release via the App Store/Play Store. Our app (Blinq) is
| also native so we'd need to prepare a model to natively run on
| both platforms.
| DeathArrow wrote:
| Why are Cloudflare Workers limited to languages that compile to
| Wasm? Can't they use a container of some sort to isolate the
| code? Many languages can be compiled to Wasm but many times is
| quirky.
| uzername wrote:
| The cloudflare workers run as "isolates". I think of them as
| "browser tabs." This post helps explain better"
| https://blog.cloudflare.com/cloud-computing-without-containe...
| re-thc wrote:
| No containers. It's a v8 isolate ie Chrome and Node's JS
| engine. Wasm come included.
| sofixa wrote:
| Containers have isolation problems thus requiring a further
| layer of isolation (microVMs like Firecracker by AWS), which
| slows startup times. CloudFlare Workers are mostly intended for
| Edge-ish scenarios so they need to have fast startup times.
| bencevans wrote:
| Looking forward to GPU support for workers. Cloudflare announced
| they were working on it in 2021 [1], but it doesn't seem
| generally available yet; they still have a signup page for
| notification [2].
|
| I know other companies have struggled with demand, so maybe
| they're doing it on an invite basis.
|
| [1]: https://blog.cloudflare.com/workers-ai/ [2]:
| https://www.cloudflare.com/nvidia-workers/
| ushakov wrote:
| They probably just want to see how many people would sign up if
| they had it. Before they start building something nobody wants
| to pay for.
| babelfish wrote:
| Agreed, but this is a blocker for anyone seriously
| considering using the product. CPU-only inference simply
| isn't good enough for anything besides toy workloads. If
| they're waiting for people to use Constellation before
| investing in GPU support, nobody will use constellation
| because it doesn't have it, so they'll never end up investing
| in it, and so on...
| NicoJuicy wrote:
| They use it themselves to route traffic, which probably is
| a not gpu intensive.
|
| I've ran face detection on cpu and it takes 1 / secondv and
| it's probably a pretty intensive "action".
|
| Plenty of use-cases that don't require a gpu
|
| And plenty that require one too though.
| dknecht wrote:
| I think people will be surprised how far CPU optimization
| will go for specialized inference. An example of the
| progress being made - http://ggml.ai/
| NicoJuicy wrote:
| I was under the assumption that, because of how cloudflare
| works, it has to be globally available.
|
| That would mean support in every dc and a easily to expand
| capacity.
|
| They can't just make it available in one region and then see
| how it goes.
|
| I like how cloudflare works, but this use-case for them ( on
| the edge) seems more difficult to plan. It's not just the tech,
| it's the infrastructure in this case.
|
| Just my 2 cents
| jgrahamc wrote:
| Good things come to those who wait :-)
| facu17y wrote:
| you can do it for free with tesseract.js
| ushakov wrote:
| Isn't constellation just relaying the inference job from the
| worker to a more powerful machine?
| jgrahamc wrote:
| No. It's running on the same machine. Our goal is distributed
| ML/AI in line with our usual mechanism of running everything,
| everywhere. That is powerful because it means anything we build
| scales with our network.
| willhackett wrote:
| This is the key reason we use Cloudflare.
|
| I'm obsessed. Very keen to try PubSub next.
| DarthNebo wrote:
| I did my share of OCR funny business by running the Firebase OCR
| SDK inside an android app acting as a webserver, through nested
| virtualization back in 2018. Nothing at that time beat it's
| accuracy & throughput for something which ran offline once the
| weights were fetched into the container.
| AndrewKemendo wrote:
| Is there a reason you didn't include any non-alphanumeric
| characters in your dictionary?
| tmikaeld wrote:
| I don't get it, how is this useful?
|
| - Takes 1.5 seconds to run, so there goes the "edge" benefit.
|
| - And if it takes that long, it would cost more than having it
| running on a proper instance/vps.
| wongarsu wrote:
| It's "serverless", so
|
| - if you have infrequent access patterns you don't pay for the
| time it isn't used; and
|
| - if you have huge bursts (say your AI project gets on the
| homepage of hacker news) capacity automatically scales with
| demand.
|
| In return each compute-second costs more than a normal vps, so
| there is some threshold where doing it yourself is more cost
| effective.
|
| The benefit of "edge" here is imho that it works together with
| other cloudflare "edge" stuff. Not hugely useful for OCR, but
| imagine your blog on Cloudflare Pages has a contact form (with
| a CF Pages Function) and you want an AI spam filter; or bot
| detection that is updated each time a page is visited, or in
| the auth function you have offloaded to a CF Worker; or if you
| want to enhance your email-triggered worker with AI.
| andersa wrote:
| Using the most expensive service at scale to handle "huge
| bursts" or "spam filters" seems like a fantastic idea!
| [deleted]
| tmikaeld wrote:
| Sure, for traffic spikes it makes sense.
| slig wrote:
| The beta link is broken:
| https://www.cloudflare.com/constellation/
| tmikaeld wrote:
| At least docs exists:
| https://developers.cloudflare.com/constellation/
| jgalt212 wrote:
| I use the word Edge and Microservice in my head interchangeably.
| If something does not make sense for a microservice, then it does
| not make sense for the Edge.
___________________________________________________________________
(page generated 2023-07-03 23:02 UTC)