https://openreview.net/forum?id=wK7wUdiM5g0

Toggle navigationOpenReview.net
[                    ]
  * Login

Open Peer Review. Open Publishing. Open Access. Open Discussion. Open
Recommendations. Open Directory. Open API. Open Source.
x

Text Embeddings Reveal (Almost) As Much As TextDownload PDF

Anonymous

20 Jul 2023OpenReview Anonymous Preprint Blind SubmissionReaders: 
Everyone
Keywords: text retrieval, embeddings, inversion, privacy
TL;DR: We propose Vec2Text, a method that can recover 90% of 32-token
embedded inputs exactly
Abstract: How much private information do text embeddings reveal
about the original text? We investigate the problem of embedding \
textit{inversion}, reconstructing the full text represented in dense
text embeddings. We frame the problem as controlled generation:
generating text that, when reembedded, is close to a fixed point in
latent space. We find that although a naive model conditioned on the
embedding performs poorly, a multi-step method that iteratively
corrects and re-embeds text is able to recover 92% of 32-token text
inputs exactly. We train our model to decode text embeddings from two
state-of-the-art embedding models, and also show that our model can
recover important personal information (full names) from a dataset of
clinical notes.
0 Replies
---------------------------------------------------------------------
Loading

  * About OpenReview
  * Hosting a Venue
  * All Venues

  * Contact
  * Feedback
  * Sponsors
  * Join the Team

  * Frequently Asked Questions
  * Terms of Service
  * Privacy Policy

  * About OpenReview
  * Hosting a Venue
  * All Venues
  * Sponsors
  * Join the Team

  * Frequently Asked Questions
  * Contact
  * Feedback
  * Terms of Service
  * Privacy Policy

OpenReview is a long-term project to advance science through improved
peer review, with legal nonprofit status through Code for Science &
Society. We gratefully acknowledge the support of the OpenReview
Sponsors.

x

Send Feedback

Enter your feedback below and we'll get back to you as soon as
possible. To submit a bug report or feature request, you can use the
official OpenReview GitHub repository:
Report an issue

[                    ]
[                    ]
[                    ]
[                    ]
[                    ]
[                    ]
[                    ]
CancelSend
x

BibTeX Record

Click anywhere on the box above to highlight complete record
Done