https://arxiv.org/abs/2202.10631

Skip to main content
Cornell University

In just 5 minutes help us improve arXiv:

Annual Global Survey
We gratefully acknowledge support from the Simons Foundation, member
institutions, and all contributors. Donate
 
arxiv logo > cs > arXiv:2202.10631
[                    ]

Help | Advanced Search

[All fields        ]
Search
arXiv logo
Cornell University Logo
[                    ] GO
quick links

  * Login
  * Help Pages
  * About

Computer Science > Human-Computer Interaction

arXiv:2202.10631 (cs)
[Submitted on 22 Feb 2022]

Title:Hidden bawls, whispers, and yelps: can text be made to sound
more than just its words?

Authors:Calua de Lacerda Pataca, Paula Dornhofer Paro Costa
View a PDF of the paper titled Hidden bawls, whispers, and yelps: can
text be made to sound more than just its words?, by Calu\~a de
Lacerda Pataca and Paula Dornhofer Paro Costa
View PDF

    Abstract:Whether a word was bawled, whispered, or yelped,
    captions will typically represent it in the same way. If they are
    your only way to access what is being said, subjective nuances
    expressed in the voice will be lost. Since so much of
    communication is carried by these nuances, we posit that if
    captions are to be used as an accurate representation of speech,
    embedding visual representations of paralinguistic qualities into
    captions could help readers use them to better understand speech
    beyond its mere textual content. This paper presents a model for
    processing vocal prosody (its loudness, pitch, and duration) and
    mapping it into visual dimensions of typography (respectively,
    font-weight, baseline shift, and letter-spacing), creating a
    visual representation of these lost vocal subtleties that can be
    embedded directly into the typographical form of text. An
    evaluation was carried out where participants were exposed to
    this speech-modulated typography and asked to match it to its
    originating audio, presented between similar alternatives.
    Participants (n=117) were able to correctly identify the original
    audios with an average accuracy of 65%, with no significant
    difference when showing them modulations as animated or static
    text. Additionally, participants' comments showed their mental
    models of speech-modulated typography varied widely.

Comments:          10 pages, 7 figures. This work has been submitted
                   to the IEEE for possible publication
Subjects:          Human-Computer Interaction (cs.HC); Sound (cs.SD);
                   Audio and Speech Processing (eess.AS)
Cite as:           arXiv:2202.10631 [cs.HC]
                   (or arXiv:2202.10631v1 [cs.HC] for this version)
                   https://doi.org/10.48550/arXiv.2202.10631
                   Focus to learn more
                   arXiv-issued DOI via DataCite
Journal reference: IEEE Trans. Affect. Comput. 14 (2023) 6-16
                   https://doi.org/10.1109/TAFFC.2022.3174721
Related DOI:       Focus to learn more
                   DOI(s) linking to related resources

Submission history

From: Calua De Lacerda Pataca [view email]
[v1] Tue, 22 Feb 2022 02:35:25 UTC (1,948 KB)
Full-text links:

Access Paper:

    View a PDF of the paper titled Hidden bawls, whispers, and yelps:
    can text be made to sound more than just its words?, by Calu\~a
    de Lacerda Pataca and Paula Dornhofer Paro Costa
  * View PDF
  * TeX Source

view license
Current browse context:
cs.HC
< prev   |   next >
new | recent | 2022-02
Change to browse by:
cs
cs.SD
eess
eess.AS

References & Citations

  * NASA ADS
  * Google Scholar
  * Semantic Scholar

export BibTeX citation Loading...

BibTeX formatted citation

x
[loading...          ]
Data provided by:

Bookmark

BibSonomy logo Reddit logo
(*) Bibliographic Tools

Bibliographic and Citation Tools

[ ] Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
[ ] Connected Papers Toggle
Connected Papers (What is Connected Papers?)
[ ] Litmaps Toggle
Litmaps (What is Litmaps?)
[ ] scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
( ) Code, Data, Media

Code, Data and Media Associated with this Article

[ ] alphaXiv Toggle
alphaXiv (What is alphaXiv?)
[ ] Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
[ ] DagsHub Toggle
DagsHub (What is DagsHub?)
[ ] GotitPub Toggle
Gotit.pub (What is GotitPub?)
[ ] Huggingface Toggle
Hugging Face (What is Huggingface?)
[ ] Links to Code Toggle
Papers with Code (What is Papers with Code?)
[ ] ScienceCast Toggle
ScienceCast (What is ScienceCast?)
( ) Demos

Demos

[ ] Replicate Toggle
Replicate (What is Replicate?)
[ ] Spaces Toggle
Hugging Face Spaces (What is Spaces?)
[ ] Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
( ) Related Papers

Recommenders and Search Tools

[ ] Link to Influence Flower
Influence Flower (What are Influence Flowers?)
[ ] Core recommender toggle
CORE Recommender (What is CORE?)

  * Author
  * Venue
  * Institution
  * Topic

( ) About arXivLabs

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and
share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have
embraced and accepted our values of openness, community, excellence,
and user data privacy. arXiv is committed to these values and only
works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community?
Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is
MathJax?)

  * About
  * Help

  * Click here to contact arXiv Contact
  * Click here to subscribe Subscribe

  * Copyright
  * Privacy Policy

  * Web Accessibility Assistance
  * arXiv Operational Status