[HN Gopher] Show HN: An AI program to check videos for NSFW content
___________________________________________________________________
Show HN: An AI program to check videos for NSFW content
Author : dynamite-ready
Score : 73 points
Date : 2022-02-08 12:12 UTC (10 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| alexb_ wrote:
| I wonder if it would be possible to not only single scenes in a
| movie out, but also use imdb data to help detect which
| actor/actress is in said scenes for categorization. Could be an
| extremely useful tool.
| dynamite-ready wrote:
| That's an interesting idea. There are off the shelf tools that
| can recognise famous people already (Amazon Rekognition, for
| example). The problem in connection with this hack, is in not
| wanting to call a third party web service.
| hbn wrote:
| Wouldn't it be easier to use facial recognition for that? In
| fact I recall years ago Google Play Movies already did that,
| where you could pause the movie and it would tell you the name
| of the faces on screen at that moment.
|
| Or do you mean if you're looking for NSFW scenes of a
| particular actor?
| bojangleslover wrote:
| Hot dog or not hot dog?
| snek_case wrote:
| Melon or not melon.
| dsr_ wrote:
| If it doesn't understand meaning and context, it can't work.
|
| It will over-report on video of baby's first bath, beach
| volleyball, and high school wrestling; it will under-report on
| many of the kinkier or more subtle forms of porn.
|
| Humans get this stuff wrong all the time simply by having
| different cultural contexts; there's no way the 2022 state of the
| art is up to the challenge.
| nonameiguess wrote:
| It could potentially revolutionize the opposite application,
| those sites where someone manually finds and catalogs all the
| time points at which movies contain nudity so people can see
| their favorite celebs in the buff. False positives are probably
| worth the automation there.
| zffr wrote:
| Even if this can't do a 100% job, isn't it still useful to
| automatically identify the obvious NSFW content?
| FrostKiwi wrote:
| Fully agree - This seems like a project with rather limited in
| scope. And that's ok. The authors even acknowledge it. It's not
| an absolute judge of SFW'ness, neither does it claim to be. I
| can see it as a tool to skim a large set of videos for further
| human review. Besides, beach volleyball very much is NSFW
| depending on the situation.
| rovr138 wrote:
| >depending on the situation.
|
| That's part of the issue.
| Chris2048 wrote:
| > And that's ok. The authors even acknowledge it
|
| That doesn't always stop corps productizing it, and making it
| their official NSFW filter. Good enough is good enough, just
| so long as they can pay less salaries.
| wpietri wrote:
| Exactly. So many products get sold based on handwaving and
| being able to check boxes. Anybody releasing open-source
| software needs to ask themselves, "What uses will this be
| put to?" Computers can be instruments of freedom or
| instruments of control.
| Mtinie wrote:
| While I agree in part, at the rate new content is produced
| and uploaded to the platforms around the globe, could you
| even fill enough seats with reviewers at an economically-
| doable rate to keep up?
| prionassembly wrote:
| I mean, "not safe for work" doesn't mean "immoral". Beach
| volleyball isn't something you should be watching at work,
| at least not in the kind of open office plan where
| everybody sees your screen. There's a lot of talk about
| workplace "lad culture" being hostile to women these days
| -- possibly enough to mean liability; also you probably
| shouldn't be hostile to women in that specific way.
| alasdair_ wrote:
| >Beach volleyball isn't something you should be watching
| at work, at least not in the kind of open office plan
| where everybody sees your screen.
|
| Watching the olympics at lunchtime is commonplace and
| accepted in most workplaces. Being a creep about it is
| not. It's almost always a pretty clear line between the
| two.
|
| It only becomes sexualized by the behavior of the people
| watching - professional athletes competing for their
| country isn't something intrinsically inappropriate.
| andylynch wrote:
| Why? During the Olympics the beach volleyball plays out
| across our floors at work just like any other event. In
| fact they played the olympics on the big screens because
| it's easier on the network.
| tokai wrote:
| I sometimes have mens cycling on while I work. This
| summer I'll properly have the womens tour de france on.
| They are almost as 'scantily' clad as beachvolley
| players. Though their bellies are covered. Is that NSFW
| or hostile to women? Is it okay to have the woman 100
| meters dash on or is it as inappropriate as beach volley
| (they do wear almost as 'revealing' clothing)? Is mens
| sport okay? Maybe marathon and other sports with skinny
| clothed men is SFW, and not diving muscular men in tight
| panties is not?
| causi wrote:
| I can't speak for your office culture but in mine we
| don't watch TV at work, period.
| tonmoy wrote:
| I understand it may be subjective, but I find it hard to
| agree that women cyclists are as 'scantily' clad.
| dynamite-ready wrote:
| It's interesting. I've not tested the model on anything
| too risque, but again, with the well known Baywatch intro
| as a frame of reference, wide shots of the whole cast in
| their swimsuits, are fine. But a close up of any single
| cast member a swimsuit will invariably trigger the model.
| Male or female.
|
| In the blog, I suggest this could be the result of an
| uncultured data set when training the CNN. Or perhaps the
| dataset was fine, and this is pushing the hard limit of
| what ResNet50 CNN architecture can do (the off the shelf
| model I use for this is an extension of ResNet50).
|
| Some of the anomalous results are amusing. One day, I
| uploaded a video of a female violinist in concert, and
| the model flagged every close up of her as NSFW! Just
| those closeups. Wide shots, and closeups of other
| musicians were absolutely fine.
|
| Again some of that might be down to me (clunky code /
| very low NSFW threshold). And I suspect the model I used
| was itself a PoC
| (https://github.com/emiliantolo/pytorch_nsfw_model). But
| it does make you wonder how the bigger labs with critical
| products, like Palantir, handle doubts like this.
| Chris2048 wrote:
| The idea that watching scantily-clad women s.a a Beach
| volleyball tournament is automatically hostile to women
| is just ingrained at this point. Maybe one day we'll
| return to normality.
| some_random wrote:
| NSFW does not literally mean "not safe for work" anymore,
| in almost all usages now it refers to explicit sexual
| material
| some_random wrote:
| We've seen plenty of author acknowledged research grade
| models and datasets get used in production, this is part of
| how black people get labeled as "monkey" every once in a
| while
| [deleted]
| jsharf wrote:
| yeah but this is true for profanity filters as well, and yet
| useful profanity filters still exist. Is it 100% accurate? No.
| But it's probably still useful.
| baobabKoodaa wrote:
| Really? Can you point to a single example of a useful
| profanity filter?
| Cthulhu_ wrote:
| I want to believe it's a matter of adding human labor to the
| Pool of Knowledge in the AI engine(s); this is, this is not, go
| do the learning thing.
|
| But when it comes to online video checking, it's another tool
| in the arsenal; it's the first lines of defense. First check:
| does the signature match a previously known and confirmed video
| marked as porn or otherwise unacceptable. Second check: does
| the more fuzzy AI thing consider it porn with a high
| probability. Third check, which will be things the AI mark as
| 'not sure' or incorrectly marks as 'porn', and things humans
| flag up manually, will be humans checking and judging things.
|
| And of course it'll be flexible, because the definition of porn
| - or what is 'not acceptable' - will vary by country and
| culture, and these companies want to be active everywhere.
|
| except Europe, unless they are given tax breaks and data,
| amirite
| dustymcp wrote:
| Reminds me of hotdog not hotdog from silicon valley,
| soylentcola wrote:
| Hey, it still could be.
| capableweb wrote:
| It seems to acknowledge itself as not being accurate, at least
| from what I gather in this passage:
|
| > it determines the Baywatch intro to be somewhat NSFW at a
| glance
|
| As nothing in the Baywatch intro is NSFW
| (https://www.youtube.com/watch?v=O0nqwgu_Us4), it seems the model
| is failing at large here.
|
| Interesting subject and lots of praise to be had if the model can
| be made accurate, but seems it's not really there yet. I wish you
| luck in getting there!
| hutzlibu wrote:
| "As nothing in the Baywatch intro is NSFW "
|
| I would suppose that depends on the workplace and area(and
| time) where you live. It is definitely "somewhat" sexualised
| content in my opinion (close up of a womens body stripping
| down), even though there is no nipple to see. But sure, lots of
| normal advertisement sells more sex. So is the red line, when a
| nipple is shown? That would classify biological and medical
| content as nsfw.
|
| My point is, human morals are very different. Humans fight
| about, what content is NSFW. A computer model can never be
| "acurate" in that sense.
| hotpotamus wrote:
| Indeed, I worked in a photo lab back in the day when NSFW
| images were still on film and we had to think about these
| kinds of rules, but basically anything was permitted for
| medical purposes - I once scanned slides for a doctor doing a
| lecture on pelvic inflammatory disease. She did not say what
| they were when dropping them off and amusedly asked about it
| when she picked them up. But it was the very opposite of sexy
| and quite clearly for genuine medical purposes.
|
| This reminds me of a question I've long had about this sort
| of thing you reminded me of - in the US at least, we seem to
| draw the line for showing off breasts at the nipple, but only
| female nipples. Setting aside any merit or logic to that, can
| AI actually determine female vs male nipples? I assume not
| through the nipple itself, but the breast it's attached to or
| the face of the person?
| hutzlibu wrote:
| "can AI actually determine female vs male nipples?"
|
| Not with 100% accuracy, but on _average_ yes, as a female
| breast is indeed quite different from a male breast. Shape,
| hair, size, ...
|
| But there are of course exeptions. Muscular women, with
| small breast who did not breast feed yet, are probably
| quite hard to detect.
| amelius wrote:
| > can AI actually determine female vs male nipples?
|
| Yes, through context. A bearded woman holding a chainsaw
| would probably fail the test.
| unfocussed_mike wrote:
| > As nothing in the Baywatch intro is NSFW
|
| Clearly incorrect by any reasonable definition unless you work
| at a swimwear company or a few bits of the media.
|
| I mean, you or I might think Alexandra Paul in a one-piece is
| an image approaching the beauty of classical sculpture, but
| quite a lot of classical sculpture is NSFW in most of the
| world, and the brow is a bit lower where Baywatch is concerned.
| capableweb wrote:
| One (dare I say?) "reasonable" definition of NSFW is nudity,
| which in most contexts I'm guilty of associating it with. I
| know it's an acronym (Not Safe For Work) but I never took it
| that literally, as otherwise most content would be NSFW
| unless you work in media. I wouldn't consider Seinfeld NSFW,
| but the companies I work for wouldn't want to see me watching
| it at work time, even if it's not normally considered NSFW.
| unfocussed_mike wrote:
| Nudity is not even the beginning of NSFW. If you're looking
| at nudity in most places you're way beyond NSFW and into
| gross misconduct.
|
| Very many workplaces have a definition of NSFW that is a
| bit more restrictive than Facebook's content rules.
|
| And rightly so. Go to work, do your work, go home. It's not
| adult daycare.
|
| Watching video footage of people running around in swimwear
| is not really appropriate content unless you work in media
| or swimwear.
|
| And the title sequence to Baywatch is hardly an advert for
| sporting endeavour; Baywatch is titillation. It would be
| entirely reasonable for almost anyone, male or female, to
| complain about that content.
|
| It is far more sensible to define NSFW as _" things I
| shouldn't be looking at just in case someone has an
| unjustifiable complaint to make about me that they are
| looking to bolster with additional offences"._
| globular-toast wrote:
| I would consider the Baywatch intro NSFW. Perhaps you don't
| consider it so because it's Baywatch, but if it was a random
| video of closeups of people in bathing costumes I bet you
| would. One thing I find really interesting with AI/Machine
| Learning is how it can cause you to re-evaluate your own
| biases. I say the AI is right here and you are wrong.
| dynamite-ready wrote:
| Here's a blog post where I explain my motivation for creating
| the program - https://raskie.com/post/practical-ai-
| autodetecting-nsfw. There are some hyperparameters to tune, but
| you're right, it's extremely flaky. Part of that might be down
| to my work (I personally wanted the app to be prudish enough to
| flag the Baywatch intro), but the off the shelf model I used
| for this, does produce some strange results. Kind of wanted to
| use this app as a convo piece to discuss AI with people who
| aren't entirely technical, or with technical people who wonder
| what AI can and can't do.
| heeen2 wrote:
| > nothing in the Baywatch intro is NSFW
|
| a debatable claim. There are several scenes in the intro I
| would not want to have paused fullscreen in a work environment.
| commandlinefan wrote:
| Well, unless you work for NBC that is.
| BeefWellington wrote:
| Are there shows you'd be comfortable having paused fullscreen
| in a work environment?
|
| Just doing so would suggest you aren't actually working (on
| its own a problem in many cultures).
| macksd wrote:
| I probably wouldn't do this in a shared office, but I'm
| pretty open about the fact that I often like to have a show
| in the background while I work. Keeps my mind from
| wandering too far if I have a momentary pause for a big
| download or a long compile or something. And a lot of
| people find moderate background noise helps them focus.
| I've been through every Star Trek multiple times and am
| plenty productive at work. But Bay Watch isn't something I
| would want coworkers to see. Nipples or no nipples, it's
| trying to sexualize / objectify about as much as they could
| get away with in prime time TV. I can't go join a meeting
| about inclusive culture with my female coworkers if they
| just saw me staring at Pamela Anderson's bouncing tits in
| the middle of the workday.
| eru wrote:
| Though Star Trek had its moments as well.
| BeefWellington wrote:
| I agree with pretty much everything you said. My
| statement was not a defense of watching Baywatch but
| rather a comment on watching everything being highly
| dependent on your workplace situation and culture.
|
| In an office setting the background noise can be highly
| disruptive, for example.
| Charlie_26 wrote:
| How can you call yourself Beef Wellington when you aren't
| even familiar with the concept of lunch
| wongarsu wrote:
| NSFW is definitely a sliding (multi-dimensional) scale, and
| different cultures and work places place themselves
| differently on it. At a previous workplace (software
| department in a manufacturing company in Western Europe) a
| women with only the lower half of her bikini laying on a
| beach was fine as a desktop background as long as you
| couldn't see her nipples. At a different company in field
| service exposed frontal breasts were still SFW. Meanwhile in
| many other jobs you would get in trouble for far less.
| meepmorp wrote:
| Many years ago, I worked a printing company, and several of
| our accounts were local porn mags and an adult newspaper
| that was basically escort ads. The account reps loved those
| jobs.
| ct0 wrote:
| High jacking the top comment to include my epiphany that
| "work" is really relative. I used to work at a place that had
| playboys in the bathroom. We should replace NSFW with NSFC -
| Not Safe for Childern.
| commandlinefan wrote:
| And some people actually work at playboy - they have a
| website after all.
| scotty79 wrote:
| > Not Safe for Childern.
|
| Which is also a debatable because I don't think there's
| much resarch indicating that images of naked people are in
| any way harmful for children.
|
| Maybe NSFPP? Not safe for parents and puritans?
| wizzwizz4 wrote:
| "Images of naked people" and "sexualised images of naked
| people" are different things. A six year-old reading
| Playboy is totally different to a six year-old visiting a
| nudist beach.
| easrng wrote:
| Not Safe for Children according to who? And how do you
| define children?
| lambic wrote:
| Or just "contains nudity". I always found it strange that
| people seem more worried about kids seeing naked people
| than they are about kids being exposed to violence.
| binarymax wrote:
| There's no such thing as a perfect model. The acknowledgement
| is welcome.
|
| Per the Baywatch example - there's clearly a grey area here,
| and in my opinion the model is working as intended. False
| positives for NSFW are better than false negatives.
| symisc_devel wrote:
| Shameless plug: At PixLab, we offer similar model available as a
| REST API endpoint: https://pixlab.io/cmd?id=nsfw. The NSFW API
| endpoint which let you detect bloody & adult content. This can
| help the developer automate things such as filtering users' image
| uploads. A tutorial on using such API is available on
| https://dev.to/unqlite_db/filter-image-uploads-according-to-....
|
| The solution can also be deployed on-promises for real-time,
| local video analysis without leaving the deployment environment:
| https://pixlab.io/on-premises.
| SomaticPirate wrote:
| Seems like this might be the use case for OPs project. A overly
| sensitive filter that allows you to send only positive alerts
| to an external API for further validation.
|
| Surprised there is no free tier here but I'm not experienced in
| this space.
| scyzoryk_xyz wrote:
| Ok now can we make this work in the opposite direction - make sfw
| video nsfw please?
| dncornholio wrote:
| What this program seems to do is create some screenshots from an
| mp4 and feed them through 2 .exe files, then display the results
| in a React app
| dynamite-ready wrote:
| Yep. It's classic hack.
|
| If this was for a product, the first thing I would have done,
| was drop half the tools in the stack for their inefficiency!
| For example, the GUI shell is using Node Webkit (which I prefer
| to Electron, but is essentially the same thing). That in itself
| is quite bad, but it's not the worst approach to building a
| desktop app, as Microsoft and Slack have already proven.
|
| But the .exe's you mentioned are also quite large.
|
| The code is very transparent though. I doubt you'd have been
| the only one to note how clunky this all is!
| MayeulC wrote:
| Yep, two random exe, and no license is just what I saw before
| closing the page.
| dynamite-ready wrote:
| Actually, there's a GNU license in both the CLI and GUI
| project. And the source code for the two 'random exe' can be
| found here - https://github.com/dynamite-ready/movie-parser-
| cli
| maleldil wrote:
| There isn't a LICENSE file in the GUI project. The LICENSE
| file in the CLI project seems to be the GPLv3, but the text
| is non-standard.
| dynamite-ready wrote:
| It's in a subfolder for the GUI project. Do you have any
| advice on the non standard text in the license? This is a
| concern I've never had before. Tbh, I'm surprised anyone
| has read it!
| yjftsjthsd-h wrote:
| The first thing I notice is:
|
| > Please link to <https://raskie.com>, if you intend to
| redistribute this .exe.
|
| Now that's _probably_ okay so long as it 's a nice
| request ("please"), and not a requirement; AIUI, GPLv3
| doesn't (generally) allow you to add conditions (ex. a
| license file that says "GPLv3 but no commercial users"
| isn't allowed by the text of GPLv3). But IANAL and I'm
| sure this is more complex than I think it is. Which
| brings us to a reasonable useful point: I would _suggest_
| never putting anything in a license file other than the
| exact standard form of the license; adding anything else,
| _at best_ , makes people have to read it and try to
| understand what you've done rather than being able to say
| "oh, it's GPLv3 and I've cached that that's 100% okay for
| me" and stop thinking about it.
| wiseleo wrote:
| Modifying a single byte in the license file means the
| license is no longer what it purports to be. Don't do it.
|
| The license file needs to be in the root of the repo.
|
| The first thing many developers who have been bitten by
| this in the past do is run all the boilerplate files like
| licenses through diff. Some probably have it hooked to
| every git clone request.
| nottorp wrote:
| NSFW for which culture?
___________________________________________________________________
(page generated 2022-02-08 23:02 UTC)