[HN Gopher] We Built a C++ Rendering Engine for the Web
___________________________________________________________________
We Built a C++ Rendering Engine for the Web
Author : PetrBrzyBrzek
Score : 87 points
Date : 2021-07-08 10:14 UTC (2 days ago)
(HTM) web link (opendesign.dev)
(TXT) w3m dump (opendesign.dev)
| flakiness wrote:
| Parsing proprietary, undocumented formats and turn it a universal
| format, and render it? That sounds like a kind of an ambition
| which is never realized, but they did. And this kind of project
| tends to fail because it's filled with boring tasks with
| unreasonable corner cases. So congrats!
|
| And according to their page [1], they've been doing this for a
| decade. This is such an accomplishment.
|
| [1] https://www.notion.so/The-Story-of-
| Avocode-d96c279b270443f88...
| mwkaufma wrote:
| Isn't that just... A browser?
| ReactiveJelly wrote:
| Title is misleading, it's a PSD renderer for web browser, not a
| web browser's renderer.
| aliasEli wrote:
| I really don't think it was a good idea to write this in C++ for
| security reasons. It accepts external inputs in some complex
| formats, and they also have a web server version. This makes it
| highly vulnerable to all kind of attacks, such as buffer
| overflows.
| nly wrote:
| As a professional C++ programmer I feel we, as a group, are
| constantly under-estimating the complexity of tasks, and over-
| estimating what can get done with the standard library.
|
| C++ is a very powerful, unopinionated language, that gives you
| a lot of freedom to attack your problem domain the way you best
| see fit.
|
| If you're writing a networked application, don't use POSIX
| sockets, which have an API designed for C, go and find a higher
| level library. If you're parsing complex text formats, don't
| iterate over buffers with char*'s, go pick up PEGTL[0]. If
| you're working on graphs, or need to properly index in-memory
| data, go pick up Boost[1][2]. If you need a GUI, go pick up Qt.
|
| It's extremely common in C++, due to the lack of a universal
| package management solution, for people to try and "muddle
| through" and do shit themselves when it's far outside their
| core competency.
|
| At one of my last employers, the core product was parsing JSON
| with std::regex, simply because they couldn't be bothered to
| integrate a JSON library (which can be done header-only).
|
| [0] https://github.com/taocpp/PEGTL
|
| [1] https://www.boost.org/doc/libs/1_76_0/libs/graph/
|
| [2]
| https://www.boost.org/doc/libs/1_76_0/libs/multi_index/doc/i...
| jstimpfle wrote:
| What's so wrong with POSIX sockets? It's maybe not an elegant
| API, but not because "it's designed for C". It's problems are
| for example 1) it's doing too much abstraction (sockets). 2)
| you have to deal with some arcane data structures and ancient
| formats (endian conversions).
|
| But, it does its job well enough: allowing the user to send
| and receive packets from the network.
|
| If you're writing a networked application that works, chances
| are you either don't give a sh*t what API to use as long as
| it lets you send and receive packets (and thus you go with
| the relatively portable POSIX sockets (at least for Linux /
| WinSock2 on Windows)), or you use a _lower_ -level API
| (probably proprietary) to reduce syscall overhead / get more
| control.
|
| If you're parsing text, chances are all you need is fread()
| to read in the next chunk from a file, and from this you'll
| build a "next_byte()" function and then a "next_token()"
| function on top.
|
| (I've done a lot of network code as well as parsing code, and
| the I/O API is among the least of my concerns).
|
| All these fancy bottom-up kitchen sink libraries implementing
| "proper abstractions" or whatever do not provide any value
| past being able to be combined to form barely working and un-
| fixable applications where you will pull your hair out when
| you actually need some control over what's happening.
|
| For something better, you'll need exactly this from external
| libraries: a clean programmatic (function call) interface
| that gives you control at a reasonable level of abstraction.
| pjmlp wrote:
| It does the job so well that in latter releases of Apple,
| Google and Microsoft OSes, the new networking features
| aren't available to classical POSIX sockets.
| jstimpfle wrote:
| Not disagreeing, that's kind of what I said. However it's
| still chugging along just fine and works well enough for
| basic applications (like 99% of applications?).
|
| What features do you allude to if I may ask?
|
| In any case, one can't solve any of these "problems" by
| abstracting over this API ;-)
| pjmlp wrote:
| For example Network.framework in Apple platforms.
|
| https://developer.apple.com/videos/play/wwdc2017/707
|
| https://developer.apple.com/videos/play/wwdc2018/715
|
| See slide 17 on the WWDC 2017 session.
|
| Similarly on the UWP/WinRT based APIs, and on Android the
| NDK doesn't see the network APIs that are only exposed
| via Java APIs.
| nly wrote:
| POSIX sockets aren't portable, period.
|
| Putting aside the super obvious problem that there's no
| common way to use them asynchronously across platforms,
| and that file descriptors are the wrong abstraction for
| TCP connections, they are riddled with more obscure
| issues:
|
| - Linger behavior varies by platform
|
| - Even simple non-blocking behavior varies by platform.
|
| - Common options like enabling TCP keep-alives, or
| setting buffer sizes, varies by platform.
|
| - More often than not, in modern times, you also want
| TLS... and that's not available portability across
| platforms either, and is a whole new awful API to learn
| (if you choose to use OpenSSL directly).
|
| - No RAII, means resource leaks (in C++).
|
| Using the raw BSD sockets APIs as a starting point for
| any portable application in 2021 is fucking insane.
| There's a reason why Python has the 'asyncio' module now
| and Go has the net module and goroutines.
| aliasEli wrote:
| OK, that got down-voted. I thought the basic idea behind HN is
| that we can have some interesting discussions. Simply down-
| voting without adding a reply why you disagree with an opinion
| does not really help.
|
| I have been interested in security for a long time. The number
| of security vulnerabilities that have been caused by insecure
| memory management problems is really huge. Some people will
| probably claim that this is not a problem with modern C++
| because it can remedy this problems. But this assumes that the
| programmers know all the possible pitfalls. With respect to
| security, the problem is that when there is only a single
| weakness in your system it might become a point of attack. With
| a language like C++ there are many possible weaknesses that
| simply do not exist in memory safe languages.
| mrweasel wrote:
| Well, you're not starting of to great by randomly claiming
| that it's a security issue that it's written in C++. It's not
| really productive. A lot of software is, for better or worse,
| written in C or C++, that's not going to change.
|
| For years people have been yelling: "It's broken because it's
| written in C/C++". That same "attacks" was made to promote
| Java 20 years ago.
|
| Sure, maybe they could have picked a memory safe language,
| but they didn't. Perhaps because they know C++ and doing the
| same project in a language they're just learning would result
| in a ton of other bugs. They even write that they hired a few
| brilliant C++ programmers, so chances are that they know how
| to safely handle memory in C++.
| pjmlp wrote:
| From my point of view, better C++ than C, however and not
| speaking from this project rather in general terms,
| adopting best practices for secure coding in C++ seems to
| still be an uphill battle, saying this as C++ aficionado.
|
| https://microblink.com/blog/be-wise-sanitize-keeping-
| your-c-...
| csmpltn wrote:
| > "Simply down-voting without adding a reply why you disagree
| with an opinion does not really help."
|
| This looks like a general criticism of using C++ which has
| nothing to do with the topic of this post. You're free to
| criticize, but this criticism alone brings absolutely nothing
| constructive to the conversation and only serves to incite
| more useless "have you considered writing this in Rust?"
| conversations. You're not even suggesting what you think they
| should've used instead.
|
| > "I have been interested in security for a long time"
|
| Here's a tip for you then: security is not an absolute, and
| things usually aren't as black or white as you might think.
| Take a moment to consider the fact that C++ is one of only a
| small handful of languages with which everything around you
| has been built for the last 30+ years. Do you know something
| all of those other engineers don't already know? Otherwise,
| humility goes a long way.
| kevingadd wrote:
| The article title literally has 'C++' in it.
| jazzyjackson wrote:
| And yet it is more specific.
| secondcoming wrote:
| You're not trying to have a discussion, your post is
| essentially 'why didn't they use Rust'. Sadly predictable.
|
| There is no-one on HN who doesn't know that C++ has historic
| memory management difficulties.
| fortran77 wrote:
| I wonder if he works for "Fronk" [0]
|
| [0] https://news.ycombinator.com/item?id=27782604
| rfrey wrote:
| I disagree with the commenters who are saying your comment
| was just "C++ bad". The point I took from your comment is
| that _this particular class of application_ , which is
| parsing many complex formats from external sources, several
| of which probably have extensive warts and edge cases, and
| rendering the output to a browser, is the type of application
| may be the most vulnerable to the downsides of C++ when it
| comes to security.
|
| I think it's a reasonable point, and a step or two above
| "just use Rust".
| flohofwoe wrote:
| According to the article, the C++ code is compiled via
| Emscripten (presumably to WASM, or maybe to asm.js), so it's
| running sandboxed either in the WASM or JS runtime. Any
| potential memory corruption caused by unsafe C++ code is
| contained within the sandbox (which is the whole point of JS
| and WASM really).
|
| The security implications are exactly the same as writing the
| code in any other language (incuding Javascript or Rust). If
| the sandbox is buggy, then a "safe" language wouldn't help
| either.
| detaro wrote:
| Just because the attack is contained inside the sandbox
| doesn't mean it can't do anything, so no, "it's in a
| sandbox" does not remove all risk automatically.
| hoytech wrote:
| Yes, exactly, otherwise buggy applications wouldn't be a
| big deal because we could run them on their own dedicated
| computers.
|
| Section 2.5 of this paper has a good discussion on this:
| https://cr.yp.to/qmail/qmailsec-20071101.pdf
| azakai wrote:
| You're right that it doesn't remove all risk
| automatically. You can still corrupt data inside the
| sandbox.
|
| However, wasm has a very clear sandboxing boundary. The
| ability of an exploit to escape the sandbox is very small
| if you are careful there.
|
| IIUC the task here is a user that wants to parse their
| own files. For that, I think wasm's sandboxing (if used
| properly) is very useful. Especially since in this case
| it runs on the web and so we also have the browser's
| additional isolation (a sandboxed process).
|
| Memory safety is incredibly important, but there isn't a
| simple answer in the space of tradeoffs, at least not for
| tasks like this. (For things like running an executable
| on bare metal that parses arbitrary inputs, obviously
| things are very different!)
| pjmlp wrote:
| A WASM module basically is like an OS process, from
| security point of view.
|
| So now think what might happen, when not used properly.
|
| Some form of bounds checking should have been part of the
| design, like memory tagging.
| dangerbird2 wrote:
| In a browser environment, all addressable memory accessible
| to WASM is more-or-less just a javascript ArrayBuffer
| object. If you can unintentionally break the browser
| sandbox with buggy C++ code, someone else has almost
| certainly already compromised your system with malicious
| plain ol' javascript.
| dangerbird2 wrote:
| The C++ code is running on webassembly, which is no more at
| risk of buffer overflow attacks than vanilla javascript. Worst-
| case scenario, you could have memory leaks in the C++ code or
| in the wasm-javascript interface (since javascript doesn't
| support finalizers for webassembly objects). But this is a
| usability issue, not a security issue
| zozbot234 wrote:
| > which is no more at risk of buffer overflow attacks than
| vanilla javascript.
|
| This is quite wrong, a Wasm program can overflow internal
| buffers due to a missing bounds check and access unrelated
| data as a result. See HEARTBLEED for a case where this
| created a very real vulnerability. The Wasm safe sandbox only
| protects the boundary with the rest of the system.
| azakai wrote:
| You're right that wasm programs can overflow internal
| buffers in linear memory, which can be dangerous. However,
| wasm does a lot more than only protect the boundary with
| the rest of the system, including
|
| * Safe call stack (opaque / managed by the VM, and so
| uncorruptible).
|
| * Safe control flow (no jumps to unexpected places).
|
| * Safe(r) indirect calls (only methods in the table can be
| called, and the signature is verified).
|
| However, wasm also lacks a few things, like the ability to
| write-protect static data (see "Everything Old is New
| Again: Binary Security of WebAssembly"). Future wasm
| proposals will hopefully address those things.
| rizky05 wrote:
| I thought opendesign was open source, but we need an API Key to
| access the tech. So, it is not open source right ?
| bckr wrote:
| I love reading about projects like this. They are in a class of
| projects that are 100% slog and not very sexy but make things a
| lot better for a lot of people.
|
| Better outputs for more inputs in better time. Beautiful.
|
| Reminds me of Photopea... Oh, I wonder whether they are in
| competition? Probably not since Photopea seems to be more about
| image editing and Avocode is more about design and handoff.
| gfxgirl wrote:
| "the notoriously opaque Photoshop format"
|
| what? it's well been documented for ever. 2nd hit on google
|
| https://www.adobe.com/devnet-apps/photoshop/fileformatashtml...
| nayuki wrote:
| It's also the one with the famous rant, "PSD is not my
| favourite file format":
| https://github.com/gco/xee/blob/7aec0d65f776fa59c58eb6cf163b...
| yrds96 wrote:
| Well, this doc is from 2019, a lot of things can be changed.
|
| As a linux user, there are a plenty software that can render it
| perfectly but cannot edit, and i really don't get it. As a web
| developer i have to use photopea to be able to extract assets
| from psd files. It's really strange that a web application can
| edit psd and a lot of powerful softwares like gimp and Krita
| have the exact problem with layers and masks with modern .psd
| files, so my guess its bcause photopea IT'S designed to be
| compatible with psd files. I suspect it's incompatible by
| design with others softwares and not because it's has a poor
| specification.
| gfxgirl wrote:
| you suspect wrong. I've written Photoshop readers since 1993.
| They all still work. Photoshop has never broken their format
| arcatek wrote:
| If I remember correctly, PSD files have an optional header
| (when files are saved in "Compatibility Mode", which I think
| is the default) in which is encoded the rasterized image.
| It's then "easy" to display the image, but very difficult to
| edit it, as you then need to implement the whole renderer.
___________________________________________________________________
(page generated 2021-07-10 23:01 UTC)