[HN Gopher] Social network ad targeting can "listen into" conver...
___________________________________________________________________
Social network ad targeting can "listen into" conversations using
"live photos"
Author : benguild
Score : 65 points
Date : 2023-05-18 17:19 UTC (5 hours ago)
(HTM) web link (twitter.com)
(TXT) w3m dump (twitter.com)
| majormajor wrote:
| "Ad targeting is listening more often than you think" is a
| "conspiracy theory" I believe in.
|
| I've had too many "mentioned an idea, then got targeted ads for
| it within a day" incidents to _not_ think some combination of
| things on the Google Home or iPhones or other smart devices is
| spying. Yeah, yeah, familiar with all the "someone else on the
| same wifi probably searched for it" or "Baader-Meinhof" or what
| have you, but I don't live with a lot of people and Instagram ads
| in particular are VERY focused these days in a way that Baader-
| Meinhof doesn't really fit - if I'd seen the ad for [specific
| thing] a couple days before I mentioned it, it would've been
| noticeably weird and out of place in a different way.
|
| I was in ad-tech for a bit in the last decade, and even at a tiny
| company there were some data sources we bought or heard about
| that were pretty spooky, so I would not be shocked if there are
| some wild ones out there today.
| guessmyname wrote:
| My wife uses Instagram and sometimes she gets ads about things
| she talks about with me or her friends.
|
| You would think that _"sometimes"_ makes them fall into the
| realm of coincidences, but they are so specific, it is creepy.
|
| Curiously, we get super excited when it happens because we are
| not fluent in English, so when the spyware manages to
| understand what we've been talking about, despite our heavy
| accent, it makes us feel that our speaking skills are
| improving.
|
| This is inversely proportional to the sentiment we have when we
| inadvertently activate Apple's Siri while talking about random
| stuff. For some reason the chip dedicated to detect "Hey,
| Siri!" thinks one of us used that command and starts listening,
| we look at each other and then burst out laughing.
|
| Maybe there is an idea for an app somewhere in there: an app
| that always listens in the background and randomly tells a joke
| about the topic of conversation.
| jvm___ wrote:
| I mentioned not being flexible enough on a Facebook comment and
| instantly my reels feed is full of flexibility videos.
| camdat wrote:
| It would be trivial to confirm yourself, Wireshark can scan for
| outgoing network packets and audio would require a large,
| continuous, byte stream.
|
| Unless you assume that Facebook et al are running ASR models
| on-device all the time and somehow making it invisible to the
| end-user.
| anon223345 wrote:
| Good call, I'm going to try this actually
| camdat wrote:
| I can save you the effort, it's not happening. This would
| be a monumental national security concern for non-US
| states, not even considering the technical limitations that
| exist for storing or utilizing literally millions of audio
| samples a minute.
|
| People are incredibly predictable using only a handful of
| demographics, there's simply no need to invest the
| astronomical amount that would be necessary to process
| these conversations when there's already many simple ways
| to track/generate user interest.
| antognini wrote:
| The main problem with the ad-companies-are-listening-to-you
| theory is that audio processing is very power hungry. Running
| an ASR model locally would eat up a _ton_ of power. Just
| doing wakeword detection (where you 're only listening for a
| specific phrase like "Hey Siri") generally requires a
| dedicated specialized chip so as to not impact power
| consumption too much.
|
| Same problem if they were surreptitiously streaming audio to
| their servers. You would see it from the outgoing packets and
| streaming that amount of data would also be fairly expensive.
| pcdoodle wrote:
| What if it's not audio processing real time?
|
| Some voice codecs get their rate down to 2-3kbps. Maybe it
| could store and forward with content loads without raising
| flags?
|
| Just spit balling here.
| czx4f4bd wrote:
| And yet there's still zero remotely plausible evidence for it.
|
| Nobody has caught continuous audio feeds being transmitted from
| smart devices to the cloud (which would be noticeable due to
| increased network traffic and bandwidth usage) nor identified
| any secret speech recognition code on the client (which would
| be noticeable due to severely shortened battery life). Nobody
| who's worked in adtech has come forward to blow the whistle or
| admit that they shipped this feature for a big tech company.
|
| I get why it's an appealing conspiracy from a gut instinct
| perspective, but it really makes no sense. When you're
| observing the behavior of billions of people and using machine
| learning algorithms trained to get the best results possible,
| some uncanny shit will naturally result. Look at how effective
| LLMs like ChatGPT have gotten _without_ an obvious route to
| profitability, then think about how much more money has been
| invested into ad targeting algorithms just in the last couple
| decades alone.
| JohnFen wrote:
| > I was in ad-tech for a bit in the last decade
|
| I'm glad that you got out! I tend to be shy about calling
| things "evil", but I really do think the entire ad-tech
| industry qualifies.
| failuser wrote:
| I need to try disabling adblock and say random words. Chariots,
| chariots. Had anyone tested that?
| moomoo11 wrote:
| How do these ad tech engineers at google and meta feel when they
| see posts like this?
|
| "Haha I made that xD. Oh well 600k tc lol"
| Gigachad wrote:
| Some people have no problems developing literal killing
| machines, or even using them. I'm not at all surprised some
| have no problem building advertisement targeting.
| moomoo11 wrote:
| I guess you're right. Even the Death Star engineers were
| probably excited to be working on cutting edge tech.
| version_five wrote:
| There are lots of coherent arguments (whether you agree
| with them or not) about why defence tech could be
| considered valuable. Ad-tech has a lot less going for it.
| jonas21 wrote:
| Live photos are an Apple feature.
| Calvin02 wrote:
| If I were running a social network, I would prioritize using AI
| to describe photos and using that as part of the targeting data
| over the 1-1.5 seconds of audio in Live Photos.
| nullsense wrote:
| I would presume they're already past that point and just want
| all they can get.
| giobox wrote:
| This is actually a wider issue with live photos IMO - people
| often share them not remembering or noticing snippets of private
| conversations can be present. I have live photos enabled by
| default and have to remind myself to check the audio before I
| forward a picture quite often, or remember to turn it back to a
| static image. Sending your mother baby pics while your spouse
| moans about the mother-in-law in background, that kind of
| thing...
|
| I'd actually probably get a lot of use from a feature whereby
| live photos displays a quick transcript of any words it picked up
| in the recording over the top of the image, or in a prompt, any
| time you try to share one with clear button to remove it. Similar
| to the local voicemail transcription feature, then I wouldn't
| have to waste time playing the recording. You could probably add
| the same detected words to the photos search index too.
| benguild wrote:
| agreed
| bobmaxup wrote:
| > your spouse moans about
|
| bemoans?
| giantrobot wrote:
| No, moans.
|
| So that's how it is in their family... /Rooney
| czx4f4bd wrote:
| https://idioms.thefreedictionary.com/moan+about
| ok_dad wrote:
| There's a setting in iPhones which will just send the static
| image when you share images.
| giobox wrote:
| Right, but I don't want to always send static images either.
| hackernewds wrote:
| Live photos do not even load on Android
| kevviiinn wrote:
| What does that have to do with this?
| BLKNSLVR wrote:
| Points out the ecosystem this issue is limited to
| tremon wrote:
| Sorry, I must be from the previous century... what is a "live
| photo" and why does it include audio?
| detaro wrote:
| it's an iOSism for "short video clip"
| umeshunni wrote:
| Is this the 2023 Tin Foil hat version of Instagram is listening
| to your microphone all the time?
| lancesells wrote:
| https://www.businessinsider.com/instagram-camera-on-when-not...
|
| "Caused by a bug"
| june_twenty wrote:
| I'd imagine it would be leaked if they were actually doing
| it. The bug story holds up imo.
| pcdoodle wrote:
| Another thing to consider: A 3d fingerprint of a space can be
| created with live photos. I'm sure object recognition benefits
| from the small perspective changes as well.
| version_five wrote:
| I guess I'm old. Is "live photo" a new name for video? This is
| the first time I've heard the term.
| JohnFen wrote:
| I'd never heard of them either. Seems to be an iPhone thing.
|
| "Basically, Live Photos on iPhone is a regular digital photo
| and a short 2 seconds video recording the few moments before
| the photo. Essentially, the iPhone camera captures 2 media
| files, one photo and one video. When viewing a Live Photo, the
| operating system, iOS plays the video file first and then it
| shows the picture."
|
| The part that makes me feel old is that I have literally no
| idea why this sort of thing would be useful or desirable.
| lilyball wrote:
| It's actually really nice. Not only does it provide a really
| good effect when swiping between photos in your photo album
| (it plays a bit of that video during the swipe, so your photo
| basically animates into place), but it often captures some
| really good stuff. I can't tell you in how many photos of my
| kids I've found utter delight in the live portion.
|
| It also lets you pick a different key frame, so if e.g. you
| get someone blinking, you can pick a different frame from the
| video to use instead.
|
| And if you capture photos back to back such that the live
| portions overlap, you can convert the group of photos into a
| single contiguous video.
| myhf wrote:
| It's similar to the "burst mode" that digital cameras have
| used for decades to capture exact moments during fast action.
|
| https://en.wikipedia.org/wiki/Burst_mode_(photography)
|
| Adding sound can make these bursts more pleasant. But there
| is no easy way to disable or remove the audio.
| majormajor wrote:
| They can be fun, you get a photo plus a short video "for
| free." Much easier than filming constantly and editing things
| down later. You can pick a different key frame to display as
| well, so it can be a good way to fix an awkwardly-timed yawn
| or such.
|
| I don't use them personally, though, but I'm a curmudgeon
| about my camera controls ;)
|
| I think Google added something similar in their Pixel camera
| app.
| JohnFen wrote:
| Eh, I think this is just one of those things I don't "get".
| A 2 second video seems without value to me.
| majormajor wrote:
| From the POV of the user it's much more "a photo" than a
| video. It just gives you a bit of extra context and
| ability to pick your key frame.
| leroy-is-here wrote:
| When on an iPhone and viewing a "live photo", you can hold down
| on the screen and the image turns into a short video leading up
| to the moment of the photo.
| [deleted]
| ben_w wrote:
| Apple thing; they're short clips of about a second, with a few
| options for displaying as loops or long exposures.
|
| I've used them _deliberately_ perhaps twice, and wish I knew
| how to force all the far more numerous _accidental_ uses into
| normal jpgs as I don 't want to waste any of the limited iCloud
| storage space on video I'll never watch.
| orf wrote:
| I just took a Live Photo and a non-raw photo of the same
| thing. The size is identical, likely because Live Photos
| compress really, really well.
| RheingoldRiver wrote:
| Samsung has something like this too, though I'm not sure what
| it's called. My mom was showing me "photos" she'd taken
| recently, and they all had 1-2 seconds of "noise" (motion)
| surrounding them. To me there is no appeal in this as a
| default setting, it just shows off how unsteadily you hold
| your phone when taking a picture. But I can see how an
| experienced photographer might take advantage of it when
| capturing moving targets.
| chatmasta wrote:
| There are a few separate issues here:
|
| 1) photos permission includes access to full file for each photo,
| including geotagged metadata and any audio from "live photo"
|
| This seems desirable and working as expected. You might
| legitimately want to share the live photo, or the photo with
| metadata. However, it would be nice if the photo picker asked you
| whether you wanted to share with metadata. (Note: unlike the
| photo picker in native apps, the Safari file picker, ie the HTML5
| <input type="file" />, _does_ strip EXIF data by default.)
|
| 2) it's easy to take a live photo without realizing it, and
| similarly easy to share it
|
| This could be improved when selecting photos within the photo
| picker. There should be an option to separately select the static
| or full file (maybe with a long press).
|
| 2) granting access to "all photos" doesn't alert you when an app
| accesses a photo without you selecting it for some reason
|
| This used to be much worse, when the default and only option was
| to grant access to all photos, or no photos. Ever since Apple
| added the option to select which photos an app can access, I
| think this is less of a problem. But there is an education issue;
| people don't realize that "all" means the app can truly read all
| your photos, including geotagged metadata and audio of live
| photos, even without you selecting one to "upload" (or "do
| something with," depending on the purpose of the app). Even as a
| relative expert, I never understood this until I saw a demo app
| (to prove the privacy issue) that looped through all your photos
| and displayed your geotagged locations.
|
| But even if people did realize this, there is no indication that
| an app has accessed a photo, like there is with the icon
| indicating the microphone or camera was used recently. Perhaps a
| solution to this could be changing the photo picker to overlay a
| green dot on any photo the app has accessed.
| benguild wrote:
| Would be interesting to see Apple show which apps accessed what
| photo and perhaps why!
| chatmasta wrote:
| Definitely. And it would be a lot more interesting if they
| tracked it for a few months and then rolled out the reporting
| feature retroactively :)
| lilyball wrote:
| FWIW my understanding is you don't get geotagging information
| on photos unless you've requested location access too.
| blakesterz wrote:
| I can see how that apps COULD do this, but is there any evidence
| any apps actually do "listen" to live photos? I don't see any
| evidence presented.
|
| I actually don't doubt it, I'd just like to see some kind of
| evidence it's being done.
| testHNac wrote:
| If they didn't till today, they will start now.
| anamexis wrote:
| Why would random 1 second snippets of audio from when photos
| were being taken be useful to advertisers?
| alephxyz wrote:
| Ultrasonic beacon tracking + being able to tell if someone
| is indoors, in a vehicle, at a concert, sporting event,
| watching something on tv, etc
| SketchySeaBeast wrote:
| I would expect that you could tell if someone was
| indoors, in a vehicle, at a concert, or sporting event
| based upon the content of the photo.
| bastawhiz wrote:
| Most photos have a timestamp and geotag. Knowing whether
| you're in a vehicle, at a concert or sporting event, or
| really doing just about anything can be gathered from
| that information as well as whatever the photos is of.
| One second of audio isn't giving much (additional) useful
| data.
| godelski wrote:
| Training ML models
| hackernewds wrote:
| ??? is this a meme
| godelski wrote:
| Why would this be a meme?
| NotYourLawyer wrote:
| Because there are billions of them.
| bastawhiz wrote:
| All of those individual seconds don't add up to a sum
| greater than their parts. There are trillions
| (quadrillions?) of seconds of reality that those same
| cameras/microphones didn't capture. Capturing a single
| second of each of a billion people's lives isn't really
| all that useful, especially for advertisers.
| SketchySeaBeast wrote:
| "Based upon our spying people spend a large amount of
| time saying '...eese'. Can someone please find out what
| 'eese' is?"
| NotYourLawyer wrote:
| I'm willing to bet that an AI could learn a lot about a
| person by listening to a large number of short audio
| clips, together with the photos themselves.
| bastawhiz wrote:
| One second isn't even long enough to hear the full
| pronunciation of all of most English words. Let's say
| people take ten photos per day. Let's generously say that
| captures ten random spoken words. Ten random words per
| day is hardly enough for a _human_ to learn anything
| about a person, let alone AI. AI cannot magically conjure
| data from noise.
|
| And when you think about what people take pictures of
| (their parking spot, selfies, nudes, landmarks, birthday
| cakes, sunsets, cats), what's heard is likely not even
| relevant to the picture taker's life or interests. If I
| look at all of the photos I've taken in the last two
| weeks, I've got:
|
| - Cat (2) - Building (1) - Stuff in my home (6) - Selfie
| (4)
|
| All thirteen photos were taken in ~silence.
| fauxpause_ wrote:
| There are billions of ants
| soared wrote:
| It's more feasible that an app would analyze the metadata and
| image itself than the audio, and I don't think we've seen that
| happening.
| lancesells wrote:
| Meta analyzes the metadata and image. I would be shocked if
| they aren't analyzing the audio of Live Photos since their
| release.
| soared wrote:
| For ad targeting?
| kristo wrote:
| I have been using social media strictly through mobile Safari for
| a month now and my ads have gotten so insanely irrelevant. It
| made me wonder what data they were getting from the app that they
| don't have access to any more
| Nextgrid wrote:
| A similar risk exists with geotagged photos.
|
| Even if you don't mind it accessing locations from individual
| photos, nothing prevents the app from scanning your entire
| library and getting _all_ the locations from there too.
|
| Same risk with timestamps - the app can get a list of all the
| timestamps and use that to confirm/refute other fuzzy datapoints
| collected elsewhere (web tracking, etc).
___________________________________________________________________
(page generated 2023-05-18 23:00 UTC)