[HN Gopher] So you want to build a browser engine
___________________________________________________________________
So you want to build a browser engine
Author : zmodem
Score : 65 points
Date : 2024-06-09 09:11 UTC (13 hours ago)
(HTM) web link (robert.ocallahan.org)
(TXT) w3m dump (robert.ocallahan.org)
| purple-leafy wrote:
| Great read :)
| ZeroGravitas wrote:
| I was going to suggest:
|
| > initially try just being a faster, lighter or lower-power
| Electron or WebView.
|
| But he mentioned it himself, though maybe someone might want to
| try this with no intention to become a full browser. Can you skip
| any of the tricky security requirements if it'll be bundled into
| an app? Or is that just asking for trouble?
| jraph wrote:
| > Or is that just asking for trouble?
|
| With the interactions an electron-like app might be doing with
| external services and the ton of JS third party library it
| could use, I think it would be indeed risky.
| cxr wrote:
| None of the security mitigations described in the post (nor
| any of those implemented in any browser engine) are aimed at
| protecting developers against themselves when they run an
| agglomeration of third-party modules as a single bundle under
| the same policy.
| jraph wrote:
| CSPs and mechanisms against cross site scripting are such
| protections. They would block a script from calling home or
| executing arbitrary scripts or displaying images that could
| exploit vulnerabilities.
|
| So browser engines definitely protect developers against
| themselves a bit.
|
| Although I agree with you that there's only so much you can
| do for the devs bundling crap themselves, I was wrong on
| this indeed.
|
| Still, I would not be overly confident with web code
| running in a browser where security is not well studied if
| it has any network capacity. Especially if the app displays
| any external content in something like an iframe.
| giancarlostoro wrote:
| So basically what Sciter does?
| giantrobot wrote:
| > Or is that just asking for trouble?
|
| The average Node project pulls in hundreds of dependencies.
| While you'd _hope_ these would have some security vetting
| because of the Many Eyes theory, you have no fucking idea what
| your project is doing. Even a trivial Electron app is running a
| ridiculous amount of unreviewed third party code.
|
| Just one module able to exercise some local exploit in your
| engine because you didn't fix Security Footgun #8176 screws
| over all of your users.
|
| A browser engine that's been developed with a billion dollars
| of person hours runs that same untrusted third party code but
| has security guardrails everywhere.
| ameliaquining wrote:
| Aren't those dependencies trusted anyway? If they want to do
| something evil, they can just do it, they don't need to look
| for a zero-day in the engine they're running on.
| ramon156 wrote:
| Tauri basically?
| IshKebab wrote:
| No. Tauri is not a web browser. It uses the existing platform
| browser.
|
| This would be more like Servo which I believe is focusing on
| embedded use cases. It makes sense because for
| Electron/embedded you don't need it to work for _every_ site
| (really really hard), you only need it to work for _one_
| site. (Or a few hundred /thousand if you count all your
| users.) That is several orders of magnitude easier.
| roca wrote:
| I think sooner or later you're going to want to load lower-
| trust content --- IFRAMEs of third-party Web content, or
| sandboxed extensions, or something like that. Building your
| entire architecture on the assumption you'll never have to do
| that is very risky.
| amelius wrote:
| This makes writing a compiler or writing an OS kernel look like
| child's play.
| Paul-Craft wrote:
| A browser engine _is_ a compiler. Or, more properly, it 's at
| least two compilers (HTML + CSS -- you can outsource JS to V8
| or whatever).
| jsheard wrote:
| The DOM stuff, JS, WASM, WebGL, WebGPU... at least five
| compilers, with JS and WASM needing two distinct frontends
| (baseline/optimizing) and at least two backends (x86/ARM),
| and WebGL and WebGPU needing three backends each
| (D3D/VK/Metal).
| rice7th wrote:
| Isn't webGL's GLSL directly delegated to the driver just
| like normal OpenGL? Also one could easily write a lot of
| frontends and a single massive centralised backend with
| multiple processor targets and optimisation profiles. Think
| about V8 which works for both JavaScript and WebAssembly.
| This would create a much simpler codebase and if you're
| going to use a parser generator it could very well be a
| breeze.
| jsheard wrote:
| > Isn't webGL's GLSL directly delegated to the driver
| just like normal OpenGL?
|
| Perhaps you could in an MVP implementation, but in
| practice no, none of the serious implementations do that
| by default. First because native OpenGL drivers are
| generally a mess, so browsers actually implement WebGL on
| top of DirectX, Vulkan or Metal wherever they can, and
| even when those aren't available the browsers still
| parse, validate and reconstitute the GLSL rather than
| passing it straight to the OpenGL driver as a layer of
| insulation against driver bugs. Chrome and Firefox have
| hidden feature flags which bypass that behavior and call
| the native GL implementation directly, but you probably
| shouldn't enable those unless you're really into
| ShaderToy and want it to compile faster.
| ramesh31 wrote:
| >This makes writing a compiler or writing an OS kernel look
| like child's play.
|
| Indeed. The modern web browser is the single most advanced,
| complex, and highly developed piece of software that 99% of
| people will ever interact with, and we treat it as a given.
| Having a highly performant fully sandboxed VM that runs on
| every system imaginable in billions of devices used by nearly
| every human on earth is the single greatest triumph of software
| engineering ever, perhaps only second to the Linux kernel.
| chrstphrknwtn wrote:
| > used by nearly every human on earth
|
| Roughly 60% of earth's population used the internet in 2023.
| So not quite _nearly every human_.
| xNeil wrote:
| 25% of the Earth's population is younger than 15. Assuming
| 20% of them use the internet, the TAM (not including the
| <15 year olds) is 80% of the world's population.
|
| 3 out of 4, or 75%, seems like nearly ever human, I'd say
| (though it is a matter of opinion I guess.)
| mepian wrote:
| A compiler by itself is surprisingly easy, especially if you
| read Abdulaziz Ghuloum's or Jeremy Siek's tutorials. Making it
| competitive with state-of-the-art compilers like Clang or
| HotSpot is difficult, but this is true for every kind of
| software.
| skilled wrote:
| Who is this post intended for?
|
| Nobody builds their own browser engines. And I mean nobody. I
| didn't get the feeling that he wrote this for pet projects
| either.
|
| Andreas over at Ladybird is probably the only one (and Servo?)
| who is really doing it the way that this post describes.
|
| Still, the last couple of paragraphs made me think that this is
| more of a reflection of his own time over at Mozilla: could have
| / would have.
| jawerty wrote:
| Most people don't need to write their own OS or compiler.
| Knowing the details of how things work is apart of getting to
| gaining expertise
| pvg wrote:
| _And I mean nobody._
|
| One such project has pretty regular HN chonkthreads, including
| last week.
|
| https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
| skilled wrote:
| Yeah that is what I meant. To my knowledge, Andreas is the
| only person crazy enough to do it and also do it in the open.
|
| You have to watch some of his YouTube videos to appreciate
| the insanity that goes into getting things to work, but he at
| least has some help from the OSS community: building
| automated tests, scripts, etc.
|
| As I was reading the post I thought he would at least shout
| him out too!
| pvg wrote:
| Ah that's 'statistically nobody' vs 'nobody'.
| pessimizer wrote:
| The word directly under "Nobody" is "Andreas."
| cjpearson wrote:
| Flow is another example I've seen.
|
| https://www.ekioh.com/flow-browser/
| ramon156 wrote:
| There's a reason they sometimes on interviews ask "what happens
| when I fill in a url and hit enter", because almost no one
| knows. It's good to know how your daily tools work.
| doublerabbit wrote:
| > Nobody builds their own browser engines
|
| Anymore.
|
| https://eylenburg.github.io/browser_engines.htm
| asadotzler wrote:
| As someone who worked with roc for many years at Mozilla, and
| with 25 years of Mozilla and Gecko experience myself, I think
| it serves as a solid warning.
| roca wrote:
| I was inspired to write it by reading about Ladybird, so I
| wrote it for Andreas Kling basically :-). But there is also
| Flow: https://www.ekioh.com/flow-browser/ and Servo. Maybe in
| the future someone else will try. Also I think it's fun to
| think about these things even if you're not building a browser
| engine. It only took about 90 minutes to write.
| phendrenad2 wrote:
| This post starts with a false dichotomy: You're either making a
| toy browser for fun, or you're trying to make the next Chrome.
| There are many points in-between on that spectrum. (Look how long
| Microsoft IE existed, despite being highly inferior.)
| IshKebab wrote:
| That's only because it _was_ the "Chrome" (i.e. the dominant
| browser). Look at what happened when Microsoft eventually tried
| to catch up with Chrome - they gave up.
| wizzwizz4 wrote:
| > _Look at what happened when Microsoft eventually tried to
| catch up with Chrome - they gave up._
|
| That wasn't because they weren't up to the job. It's because
| Google was using Microsoft's playbook against them.
| https://news.ycombinator.com/item?id=18697824
|
| > I very recently worked on the Edge team, and one of the
| reasons we decided to end EdgeHTML was because Google kept
| making changes to its sites that broke other browsers, and we
| couldn't keep up. For example, they recently added a hidden
| empty div over YouTube videos that causes our hardware
| acceleration fast-path to bail (should now be fixed in Win10
| Oct update). Prior to that, our fairly state-of-the-art video
| acceleration put us well ahead of Chrome on video playback
| time on battery, but almost the instant they broke things on
| YouTube, they started advertising Chrome's dominance over
| Edge on video-watching battery life. What makes it so sad, is
| that their claimed dominance was not due to ingenious
| optimization work by Chrome, but due to a failure of YouTube.
| On the whole, they only made the web slower.
|
| (While this may seem like poetic justice, Google's also been
| doing this against everyone else, so we shouldn't cheer.
| Also, unlike with Microsoft, this might not even be
| _deliberate_ on the part of the Google webdevs.)
| mkoubaa wrote:
| I hate that they did it but I have a hard time sympathizing
| with Edge/MS
| kernal wrote:
| What a weak excuse. To blame an empty div for the demise of
| your browser engine just seems desperate.
| dmazzoni wrote:
| Not at all: at the very end it suggests some in-betweens, like
| building an alternative to Electron or embedded WebViews.
|
| There are lots of browser engines like that still around. It'd
| be really interesting to see one that supports the vast
| majority of web standards but without trying to do everything
| else a full browser does.
| atum47 wrote:
| I honestly think (I've been thinking about that for a few years
| now) that eventually a OS will be nothing more than a browser.
| notsylver wrote:
| Isn't that what ChromeOS is?
| fabrice_d wrote:
| that was the idea of FirefoxOS. Come help https://capyloon.org
| if you believe in that!
| shiomiru wrote:
| I'm writing one for fun: https://sr.ht/~bptato/chawan/
|
| "Fixed-width text to a grid" makes things easier (sometimes), but
| I think it still qualifies.
|
| On the article itself; it might be better to start with more...
| basic things than the optimizations it talks about:
|
| * Cascading & layout must be cached and optimized - far from
| trivial. Layout is one of the hardest parts to get right at all,
| to make it fast as well is another level... and I'm just talking
| caching, not even multi-threading.
|
| * The "web platform" contains a very large amount of poorly
| thought out features harmful for user privacy and/or security.
| You can choose between convenience (opt out) and privacy (opt
| in), or try to balance the two as major browsers do. Both is
| often impossible.
|
| * "Serialize the entire application state" sounds like the most
| complex thing you can come up with as a distinguishing feature.
| Much more low hanging fruit exists if you give up on writing a
| Chromium clone. e.g. a fun side project I've had is making my
| browser "protocol-agnostic", or adding a bunch small QOL stuff
| for myself. You can probably find new ideas easily once you start
| out.
|
| * Older browsers are useful for inspiration too. The coolest
| features in mine are often just carbon copies of the same things
| from w3m, just a bit more polished. Reading about what historical
| engines did - even IE! - is often enlightening too. Sometimes
| it's just a smaller scale of what engines do now, and easier to
| implement. Other times it's a whole different approach.
|
| * It's easy to get lost in the details - careful with
| perfectionism. As you start getting more components interacting,
| it will slowly become clear when the naive approach is wrong and
| how to improve it.
|
| Overall, it takes time, but is also fun and rewarding - you will
| learn a lot, even without replacing Chromium.[0] That said, if
| you want to learn how "modern" browsers work, getting involved
| with a "modern" engine itself is probably much more productive.
|
| [0]: To write a Chromium replacement, you may have to reincarnate
| as awesomekling ;)
| roca wrote:
| I guess one of my points is that layout algorithms are not
| really part of the "most basic" decisions anymore. Replacing
| layout algorithms is actually a lot less disruptive to the
| engine architecture than switching to site isolation, say.
| shiomiru wrote:
| Fair; re-reading TFA, now I realize you explicitly instructed
| me to stop reading in the first paragraph :)
|
| Trying to redeem myself with an on-topic question: isn't what
| you want more of a "refactoring of Blink" than "building a
| browser engine"? I would be surprised if a complete rewrite
| was really necessary for the features you want, since "saving
| state" already happens to some extent in all engines (even if
| it's just reloading from the cache) and I've seen reports
| about Gecko integrating multi-core cascade from Servo. What
| makes it hard to incrementally improve upon the current
| engines?
| jwells89 wrote:
| > Much more low hanging fruit exists if you give up on writing
| a Chromium clone.
|
| One idea that's crossed my mind (may never get around to trying
| it) is writing an engine that completely ignores everything
| that's not modern HTML5 and CSS3. That's still a lot but it
| seems like it'd cut down on the scope significantly. This
| engine wouldn't be very useful as a browser but it'd probably
| be enough for displaying bespoke web content in an app or
| something like that.
| djbusby wrote:
| Seems a good place to mention https://sciter.com/
|
| It's been on HN loads of times.
|
| A "browser" engine but very narrow scope. Works a treat for LOB
| type apps.
| Buttons840 wrote:
| I was sad when the creator tried to raise enough funds to open-
| source Sciter and there wasn't much interest. If 1% of the
| people who complain about Electron had pledged something, we
| would have it as a good alternative today.
| crazygringo wrote:
| > _So You Want To Build A Browser Engine_
|
| The only correct answer is, "don't".
|
| I mean, if you want to build a toy browser engine for a CS class
| or fun or something, then sure.
|
| But the idea that "you want to build an engine that's competitive
| with Chromium" is, quite simply, nonsensical.
|
| If you want your own browser engine, you're going to fork
| Chromium or Gecko (Firefox). I mean, even _Microsoft_ gave up on
| maintaining its own independent engine and switched to Chromium.
|
| I literally don't understand who the author thinks this post is
| supposed to be addressed to.
|
| Building an independent browser engine could have made sense in
| 1998. These days it would cost hundreds of millions of dollars in
| dev time to catch up to existing engines... just to duplicate
| something that you already have _two_ open-source versions of?
___________________________________________________________________
(page generated 2024-06-09 23:00 UTC)