[HN Gopher] An NSFW filter for Marginalia search
___________________________________________________________________
An NSFW filter for Marginalia search
Author : speckx
Score : 86 points
Date : 2026-03-30 15:58 UTC (7 hours ago)
(HTM) web link (www.marginalia.nu)
(TXT) w3m dump (www.marginalia.nu)
| marginalia_nu wrote:
| This was a very meandering project, and trying to corral it into
| some sort of coherent narrative was a bit of an undertaking on
| its own. Hopefully it makes _some_ sense.
| BrunoBernardino wrote:
| Hi Viktor! Really cool write-up, thanks! Uruky is already using
| the `nsfw` param, but set to `0` or `1`, and I see in your
| example this looks like a new value option (`2`) that's
| "better" than `1`? How "safe" is it to implement it as the
| value to send when someone wants SFW results?
| marginalia_nu wrote:
| 0 disables all filtering
|
| 1 filters 'harmful' sites per the UT1 blacklists
|
| 2 is 1 + the new NSFW filter.
|
| The new filter works pretty good in my assessment. It's not
| infallible, but it gives significantly cleaner results.
|
| And if you do find queries it fails to sanitize, I'd love to
| hear about them.
| BrunoBernardino wrote:
| Thanks, already implemented and tested a couple of queries
| and it does look good!
| IncreasePosts wrote:
| Can you add 3, which _only_ returns content flagged as
| NSFW?
|
| So I can make sure I know what sites to stay away from, of
| course
| VectorLock wrote:
| Or perhaps -2
| marginalia_nu wrote:
| Wouldn't work very well, in that you'd get awful recall.
|
| The way the filter is implemented, it runs after the
| query has been executed. I'd have to run it at document
| processing time, code in a pseudo-keyword for the label,
| and then add that to the query.
|
| It's doable, but I question whether the juice is worth
| the squeeze.
| 8organicbits wrote:
| Have you seen many examples of websites labeling themselves,
| perhaps using rating meta tags (<meta name="rating" ...>)? Self-
| labeling seems valuable in some ways, but I don't think I've seen
| it catch on.
| marginalia_nu wrote:
| Meta tags are almost universally garbage, but the presence of
| '18 USC 2257' (or U.S.C.) is a very strong NSFW signal.
| Wingy wrote:
| Does this comment make this page NSFW on Marginalia?
| ChadNauseam wrote:
| Does marginalia_nu not use embedding models as part of search? I
| guess I assumed it would. If you have embeddings anyway, decision
| trees on the embedding vector (e.g. catboost) tend to work pretty
| well. Fine-tuning modernbert works even better but probably won't
| meet the criteria of "really fast and run well on CPUs". That
| said, the approach described in the article seems to work well
| enough and obviously provides extremely cheap inference
| marginalia_nu wrote:
| It does not use any transformer models right now. I've made
| experiments with BERT-adjacent methods, but not found them fast
| enough to be useful. Basically, whatever approach is used, it
| needs to do inference at ~10us latencies to either make real-
| time result filtering viable, or <1ms not add unreasonable
| overhead to processing-time result labeling.
| VorpalWay wrote:
| Looks like a cool search engine! Hadn't heard about it before.
|
| But the search page says "Simple technology, no AI". With this
| change, that is no longer true though, is it? Of course the
| definition of "AI" is extremely vague. Once upon a time, A-star
| search was considered AI after all.
| marginalia_nu wrote:
| The 'No AI' statement is about gen AI, which is I think what
| most people think of when you say AI.
|
| But sure if I was looking for government or research funding,
| then for sure this would be AI. Not just AI, but the literal
| state of the art AI. Dario wakes in a flop sweat every night,
| terrified of my breakneck advances in single hidden layer
| classifiers that are probably at least 30% sentient. It would
| be so much AI I can't even hold all this AI.
___________________________________________________________________
(page generated 2026-03-30 23:01 UTC)