https://nolanlawson.com/2022/04/08/the-struggle-of-using-native-emoji-on-the-web/

Read the Tea Leaves Software and other dark arts, by Nolan Lawson

[Search this Blog    ] Search

  * Home
  * Apps
  * Code
  * Talks
  * About

<< Five years of quitting Twitter

8 Apr

The struggle of using native emoji on the web

Posted April 8, 2022 by Nolan Lawson in Web. 9 Comments

Emoji are a standard overseen by the Unicode Consortium. The web is a
standard governed by bodies such as the W3C, WHATWG, and TC39. Both
emoji and the web are ubiquitous.

So you might be forgiven for thinking that, in 2022, it's possible to
plop an emoji on a web page and have it "just work":



If you see a lotus flower above, then congratulations! You're on a
browser or operating system that supports Emoji 14.0, released in
September 2021. If not, you might see something that looks like the
scoreboard on an old 80's arcade game:

Black square with monospace text inside of hexademical encoding

Another apt description would be "robot barf."

Let's try another one. What does this emoji look like to you?



If you see a face with spiral eyes, then wonderful! Your browser can
render Emoji 13.1, released in September 2020. If not, you might see
a puzzling combination of face with crossed-out eyes and a shooting
("dizzy") star:



It's a fun bit of cartoon iconography to know that this combination
means "dizzy face," but for most folks, it doesn't really evoke the
same meaning. It's not much better than the robot barf.

Emoji and browser support

If you're like me, you're a minimalist when it comes to web
development. If I don't have to rebuild something from scratch, then
I'll avoid doing so. I try to "use the platform" as much as possible
and lean on existing web standards and browser capabilities.

When it comes to emoji, there are a lot of potential upsides to using
the platform. You don't need to bring your own heavy emoji font, or
use a spritesheet, or do any manual DOM processing to replace text
with <img>s. But sadly, if you try to avoid these heavy-handed
techniques and just, you know, use emoji on the web, you'll quickly
run into the kinds of problems I describe above.

The first major problem is that, although emoji are released by the
Unicode Consortium at a yearly cadence, OSes don't always update in a
timely manner to add the latest-and-greatest characters. And the
browser, in most cases, is beholden to the OS to render whatever
emoji fonts are provided by the underlying system (e.g. Apple Color
Emoji on iOS, Microsoft Segoe Color Emoji on Windows, etc.).

In the case of major releases (such as Emoji 14.0), a missing
character means the "robot barf" shown above. In the case of minor
releases (such as Emoji 13.1), it can mean that the emoji renders as
a bizarre "double" emoji - some of my favorites include "man with
floating wig of red hair" () for "man with red hair" () and
"bear with snowflake" ([?]) for "polar bear" ([?]).

If I'm trying to convince you that native emoji are worth investing
in for your website, I've probably lost half my audience at this
point. Most chat and social media app developers would prefer to have
a consistent experience across all browsers and devices - not a
broken experience for some users. And even if the latest emoji were
perfectly supported across devices, these developers may still prefer
a uniform look-and-feel, which is why vendors like Twitter, Facebook,
and WhatsApp actually design their own emoji fonts.

Detecting broken emoji

Let's say, though, that you're comfortable with emoji looking
different on different platforms. After all - maybe Apple users would
prefer to see Apple emoji, and Windows users would prefer to see
Windows emoji. And in any case, you'd rather not reinvent what the OS
already provides. What do you have to do in this case?

Well, first you need a way to detect broken emoji. This is actually
much harder than it sounds, and basically boils down to rendering the
emoji to a <canvas>, testing that it has an actual color, and also
testing that it doesn't render as two separate characters. (
is-emoji-supported is a decent JavaScript library that does this.)

This solution has a few downsides. First off, you now need to run
JavaScript before rendering any text - with all the problems therein
for SSR, performance, etc. Second, it doesn't actually solve the
problem - it just tells you that there is a problem. And it might not
even work - I've seen this technique fail in cross-origin iframes in
Firefox, presumably because the <canvas> triggered the browser's
fingerprinting detection.

But again, let's just say that you're comfortable with all this. You
detect broken emoji and perhaps replace them with text saying "emoji
not supported." Or maybe you want a more graceful degradation, so you
include half a megabyte of JSON data describing every emoji ever
created, so that you can actually show some text to describe the
emoji. (Of course, that file is only going to get bigger, and you'll
need to update it every year.)

I know what you're thinking: "I just wanted to show an emoji on my
web page. Why do I have to know everything about emoji?" But just
wait: it gets worse.

Black-and-white older emoji

Okay, so now you're successfully detecting whether an emoji is
supported, so you can hide or replace those newfangled emoji that are
causing problems. But would it occur to you that the oldest emoji
might be problematic too?



This is the classic smiling face emoji. But depending on your
browser, instead of the more familiar full-color version, you might
see a simple black-and-white smiley. In case you don't see it, here
is a comparison, and here's how it looks in Chrome on Windows:

Screenshot showing a black and white smiley face with no font-family
and a color smiley face with system emoji font family

You'll also see this same problem for some other older emoji, such as
red heart ([?]) and heart suit ([?]), which both render as black hearts
rather than red ones.

So how can we render these venerable emoji in glorious Technicolor?
Well, after a lot of trial-and-error, I've landed on this CSS:

div {
  font-family: "Twemoji Mozilla",
               "Apple Color Emoji",
               "Segoe UI Emoji",
               "Segoe UI Symbol",
               "Noto Color Emoji",
               "EmojiOne Color",
               "Android Emoji",
               sans-serif;
}

Basically, what we have to do is point the font-family at a known
list of built-in emoji fonts on various operating systems. This is
similar to the "system font" trick.

If you're wondering what "Twemoji Mozilla" is, well, it turns out
that Firefox is a bit odd in that it actually bundles its own version
of Twitter's Twemoji font on Windows and Linux. This will be
important later, but let's set it aside for now.

What is an emoji, anyway?

At this point, you may be getting pretty tired of this blog post.
"Nolan," you might say, "why don't you just tell me what to do? Just
give me a snippet I can slap onto my website to fix all these dang
emoji problems!" Well I wish it were as simple as just chucking a CSS
font-family onto your body and calling it a day. But if you try that
naive approach, you'll start to see some bizarre characters:

The text "Call me at #555-0123! You might have to hit the * or # on
your smartphone(tm)" with some of the characters thicker and larger and
cartoonier than the others.

As it turns out, characters like the asterisk (*), octothorpe (#),
trademark ((tm)), and even the numbers 0-9 are technically emoji. And
depending on your browser and OS, the system emoji font will either
not render them at all, or it might render them as the
somewhat-cartoony versions you see above.

Maybe to some folks it's acceptable for these characters to be
rendered as emoji, but I would wager that the average person doesn't
consider these numbers and symbols to be "emoji." And it would look
odd to treat them like that.

So all right, some "emoji" are not really emoji. This means we need
to ensure that some characters (like the smiley face) render using
the system emoji font, whereas other kinda-sorta emoji characters
(like * and #) don't. Potentially you could use a JavaScript tool
like emoji-regex or a CSS tool like emoji-unicode-range to manage
this, but in my experience, neither one handles all the various edge
cases (nor have I found an off-the-shelf solution that does). And
either way, it's starting to feel pretty far from "use the platform."

Windows woes

I could stop right here, and hopefully I've made the point that using
native emoji on the web is a painful experience. But I can't help
mentioning one more problem: flag emoji on Windows.

As it turns out, Microsoft's emoji font does not have country flags
on either Windows 10 or Windows 11. So instead of the US flag emoji,
you'll just see the characters "US" (and the equivalent country codes
for other flags). Microsoft might have a good geopolitical reason to
do this (although they'd have to explain why no other emoji vendor
follows suit), but in any case, it makes it hard to talk about sports
matches or national independence days.

Grid of some flag emoji such as the pirate flag and rainbow flag,
followed by many two-letter character codes instead of emoji

Flag emoji in Chrome on Windows. You can have the pirate flag, you
can have the race car flag, but you can't root for Argentina vs
Brazil in a soccer match.

Interestingly, this problem is actually solvable in Firefox, since
they ship their own "Mozilla Twemoji" font (which, furthermore, tends
to stay more up-to-date than the built-in Microsoft font). But the
most popular browser engine on Windows, Chromium, does not ship their
own emoji font and doesn't plan to. There's actually a neat tool
called country-flag-emoji-polyfill that can detect the broken flag
support and patch in a minimal Twemoji font to fix it, but again,
it's a shame that web developers have to jump through so many hoops
to get this working.

(At this point, I should mention that the Unicode Consortium
themselves have come out against flag emoji and won't be minting any
more. I can understand the sentiment behind this - a font consortium
doesn't want to be in the business of adjudicating geopolitical
boundaries. But in my opinion, the cat's already out of the bag. And
it seems bizarre that Wales and Scotland get their own flag, but no
other countries, states, provinces, municipalities, duchies,
earldoms, or holy empires ever will. It seems guaranteed to lead to
an explosion of non-standard vendor-specific flags, which is already
happening according to Emojipedia.)

Conclusion

I could go on. I really could. I could talk about the sad state of
browser support for color fonts, or how to avoid mismatched emoji
fonts in Firefox, or subtle issues with measuring emoji width on
Windows, or how you need to install a separate package for emoji to
work at all in Chrome on Linux.

But in the end, my message is a simple one: I, as a web developer,
would like to use emoji on my web sites. And for a variety of
reasons, I cannot.

Screenshot of a grid of emoji smileys where some emoji are empty
boxes with text inside

I build an emoji picker called emoji-picker-element. This is what it
would look like if I didn't bend over backwards to fix emoji
problems.

At a time when web browsers have gained a staggering array of new
capabilities - including Bluetooth, USB, and access to the filesystem
- it's still a struggle to render a smiley face. It feels a bit odd
to argue in 2022 that "the web should have emoji support," and yet
here I stand, cap in hand, making my case.

You might wonder why browsers have been so slow to fix this problem.
I suspect part of it is that there are ready workarounds, such as
twemoji, which parses the DOM to look for emoji sequences and
replaces them with <img>s. The fact that this technique isn't great
for performance (downloading extra images, processing the DOM and
mutating it, needing to run JavaScript at all) might seem unimportant
when you consider the benefits (a unified look-and-feel across
devices, up-to-date emoji support).

Part of me also wonders if this is one of those cases where the needs
of larger entities have eclipsed the needs of smaller "mom-and-pop"
web shops. A well-funded tech company building a social media app
with a massive user base has the resources to handle these emoji
problems - heck, they might even design their own emoji font! Whereas
your average small-time blogger, agency, or studio would probably
prefer for emoji to "just work" without a lot of heavy lifting. But
for whatever reason, their voices are not being heard.

What do I wish browsers would do? I don't have much of a grand
solution in mind, but I would settle for browsers following the
Firefox model and bundling their own emoji font. If the OS can't keep
its emoji up-to-date, or if it doesn't want to support certain
characters (like country flags), then the browser should fill that
gap. It's not a huge technical hurdle to bundle a font, and it would
help spare web developers a lot of the headaches I listed above.

Another nice feature would be some sensible way to render what are
colloquially known as "emoji" as emoji. So for instance, the "smiley
face" should be rendered as emoji, but the numbers 0-9 and symbols
like * and # should not. If backwards compatibility is a concern,
then maybe we need a new CSS property along the lines of
text-rendering: optimizeForLegibility - something like
emoji-rendering: optimizeForCommonEmoji would be nice.

In any case, even if this blog post has only served to dissuade you
from ever trying to use native emoji on the web, I hope that I've at
least done a decent job of summarizing the current problems and
making the case for browsers to help solve it. Maybe someday, when
browsers everywhere can render a smiley face, I can write something
other than :-) to show my approval.

Related

9 responses to this post.

 1. #

    Posted by Jacky Alcine on April 8, 2022 at 5:26 PM

    I wouldn't have known about this if I didn't see this in my
    reader, and now I'm very glad that I read it through and
    appreciate the work you've spent on getting to this point. It
    does feel like it'll be safer for small shops to lean on what
    Twitter's done thus far. I vaguely remember a community version
    of twemoji that was used for Mastodon and allowed for custom
    emoji. I might be blanking out here though.

    (Originally posted at https://jacky.wtf/2022/4/hg/
    hgRt-1I-5YHFH27n5tftjRn4)

    Reply

 2. #

    Posted by N on April 9, 2022 at 12:10 AM

    Regarding eg # being an emoji, try inverting a Unicode variation
    selector after it. See https://en.wikipedia.org/wiki/Emoticons_
    (Unicode_block) heading "variant forms".

    Reply

      + #

        Posted by KLG on April 9, 2022 at 5:57 AM

        That was my first thought as well when seeing the
        "Black-and-white older emoji" section. Opened the developer
        tools, pasted a VARIATION SELECTOR-16 after that , and got
        the colorful emoji [?]

        OT1H I love learning about all those obscure Unicode
        mechanisms... OTOH as the article points out, it's frustrating
        when one is "forced" to take in all those details to work
        around platform shortcomings.

        Reply

          o #

            Posted by Nolan Lawson on April 9, 2022 at 8:19 AM

            Good tips, thanks! That works when you own the text
            content on the page, although it wouldn't work for a chat
            /social media app, where you can't control if the
            variation selector exists or not. And yeah, it's still a
            lot for the website author to have to know to make these
            characters look "normal."

 3. #

    Posted by Ralph Haygood on April 10, 2022 at 5:01 PM

    "even if this blog post has only served to dissuade you from ever
    trying to use native emoji on the web": You've dissuaded me. I
    was vaguely aware there were problems. Now that I know more about
    how bad they are, I'll steer clear of the whole mess. Please do
    another post if it ever gets better.

    Reply

 4. #

    Posted by Thomas Steiner on April 11, 2022 at 1:05 AM

    May I point out emoji color fonts in the COLRv1 format as a
    new-ish alternative? See this article https://
    developer.chrome.com/blog/colrv1-fonts/#
    no-more-image-replacement-emoji-fonts (deeplink).

    Reply

      + #

        Posted by Nolan Lawson on April 11, 2022 at 8:04 AM

        Yep, I was vaguely aware of this. Unfortunately it looks like
        Chrome Status lists WebKit as "negative" on this spec.

        Having an emoji font that works across all browsers would be
        great, though! It would still require some of the other
        hackery I mention, but it'd be an improvement. :)

        Reply

 5. #

    Posted by Phil Seamore on April 11, 2022 at 6:21 AM

    Unicode ranges?

    @font-face {
    font-family: emojis-fix;
    src: local(".SFNS-Light"), local(".SFNSText-Light"), local
    (".HelveticaNeueDeskInterface-Light"), local(".LucidaGrandeUI"),
    local("Segoe UI Light"), local("Ubuntu Light"), local
    ("Roboto-Light"), local("DroidSans"), local("Tahoma");
    }
    @font-face {
    font-family: emojis-fix;
    src: local("Twemoji Mozilla"), local("Apple Color Emoji"), local
    ("Segoe UI Emoji"), local("Segoe UI Symbol"), local("Noto Color
    Emoji"), local("EmojiOne Color"), local("Android Emoji");
    unicode-range: U+1F300-1F5FF, U+1F600-1F64F, U+1F680-1F6FF,
    U+2600-26FF;
    }
    div {
    font-family: emojis-fix, sans-serif;
    }

    Reply

      + #

        Posted by Nolan Lawson on April 11, 2022 at 8:08 AM

        If I'm not mistaken, this looks similar to
        emoji-unicode-range, which I mentioned. Unfortunately it
        [doesn't handle the "numbers and symbols" problem (demo).
        Presumably there is some range that will do this, although it
        still wouldn't help with polyfilling newer emoji or flag
        emoji. In any case, this is still a lot of work just to get
        emoji to render correctly! :)

        Reply

Leave a Reply Cancel reply

Enter your comment here...
[                    ]

Fill in your details below or click an icon to log in:

  *  
  *  
  *  
  *  

Gravatar
Email (required) (Address never made public)
[                    ]
Name (required)
[                    ]
Website
[                    ]
WordPress.com Logo

You are commenting using your WordPress.com account. ( Log Out / 
Change )

Google photo

You are commenting using your Google account. ( Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. ( Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. ( Log Out /  Change )

Cancel

Connecting to %s

[ ] Notify me of new comments via email.

[ ] Notify me of new posts via email.

[Post Comment] 

 [                                             ] 
 [                                             ] 
 [                                             ] 
 [                                             ] 
 [                                             ] 
 [                                             ] 
 [                                             ] 
D[                                             ] 

This site uses Akismet to reduce spam. Learn how your comment data is
processed.

Recent Posts

  * The struggle of using native emoji on the web
  * Five years of quitting Twitter
  * Memory leaks: the forgotten side of web performance
  * 2021 book review
  * Introducing fuite: a tool for finding memory leaks in web apps

About Me

Nolan Lawson - photo of me flashing a peace sign

Hi, I'm Nolan. I'm a web developer living in Seattle and working for
Salesforce. Opinions expressed in this blog are mine and frequently
wrong.

Archives

  * April 2022 (1)
  * February 2022 (1)
  * January 2022 (1)
  * December 2021 (3)
  * September 2021 (1)
  * August 2021 (6)
  * February 2021 (2)
  * January 2021 (2)
  * December 2020 (1)
  * July 2020 (1)
  * June 2020 (1)
  * May 2020 (2)
  * February 2020 (1)
  * December 2019 (1)
  * November 2019 (1)
  * September 2019 (1)
  * August 2019 (2)
  * June 2019 (4)
  * May 2019 (3)
  * February 2019 (2)
  * January 2019 (1)
  * November 2018 (1)
  * September 2018 (5)
  * August 2018 (1)
  * May 2018 (1)
  * April 2018 (1)
  * March 2018 (1)
  * January 2018 (1)
  * December 2017 (1)
  * November 2017 (2)
  * October 2017 (1)
  * August 2017 (1)
  * May 2017 (1)
  * March 2017 (1)
  * January 2017 (1)
  * October 2016 (1)
  * August 2016 (1)
  * June 2016 (1)
  * April 2016 (1)
  * February 2016 (2)
  * December 2015 (1)
  * October 2015 (1)
  * September 2015 (1)
  * July 2015 (1)
  * June 2015 (2)
  * October 2014 (1)
  * September 2014 (1)
  * April 2014 (1)
  * March 2014 (1)
  * December 2013 (2)
  * November 2013 (3)
  * August 2013 (1)
  * May 2013 (3)
  * January 2013 (1)
  * December 2012 (1)
  * November 2012 (1)
  * October 2012 (1)
  * September 2012 (3)
  * June 2012 (2)
  * March 2012 (3)
  * February 2012 (1)
  * January 2012 (1)
  * November 2011 (1)
  * August 2011 (1)
  * July 2011 (1)
  * June 2011 (3)
  * May 2011 (2)
  * April 2011 (4)
  * March 2011 (1)

Tags

accessibility alogcat android android market apple app tracker blobs
boost bootstrap browsers bug reports catlog chord reader code
contacts continuous integration copyright couch apps couchdb
couchdroid databases developers development grails html5 indexeddb
information retrieval japanese name converter javascript jenkins
keepscore listview localstorage logcat logviewer lucene mobile web
modules nginx nlp node nodejs npm offline-first open source passwords
performance pokedroid pouchdb pouchdroid query expansion relatedness
calculator relatedness coefficient s3 safari satire sectioned
listview security semver social media socket.io software development
solr supersaiyanscrollview synonyms twitter ui design ultimate
crossword w3c webapp webapps web platform web sockets websql web
workers

Links

  * Mastodon
  * GitHub
  * npm
  * Keybase

Blog at WordPress.com.

  * Follow Following
      + [05fe10] Read the Tea Leaves
        Join 1,170 other followers
        [                    ]
        Sign me up
      + Already have a WordPress.com account? Log in now.
  * 
      + [05fe10] Read the Tea Leaves
      + Customize
      + Follow Following
      + Sign up
      + Log in
      + Copy shortlink
      + Report this content
      + View post in Reader
      + Manage subscriptions
      + Collapse this bar

 

  

Loading Comments...
 
Write a Comment... [                    ]
Email (Required) [                    ] Name (Required)
[                    ] Website [                    ]
[Post Comment]

[b]