[HN Gopher] Tag Systems
___________________________________________________________________
Tag Systems
Author : fagnerbrack
Score : 88 points
Date : 2023-04-17 07:59 UTC (1 days ago)
(HTM) web link (buttondown.email)
(TXT) w3m dump (buttondown.email)
| joshu wrote:
| at the very beginning del.icio.us had a hierarchy operator:
| anything tagged "foo/bar" would internally also get the tag "foo"
| as well. i ended up pulling it out.
|
| for finding untagged things, i had "system:untagged"
| automatically added if the items had no tags because mixing inner
| and outer joins was a nightmare in mysql.
| deafpolygon wrote:
| Look, you can be a horse girl if you love horses. You don't need
| to own one to be a horse girl. Why gatekeep, huh?
| egeozcan wrote:
| The best way is, in my experience, let the users tag away. Then
| add the tags to the item descriptions in the full text search
| (while also holding on to the normalized data, because
| refactoring tags is a thing). If your FTS system allows the
| definition of synonyms, even better. Users get the usual
| autocomplete, and suggested tags. Searchers get approximation.
|
| I never implemented this before but the tags can also hold
| references to the tags they are most frequently tagged with to
| further add content to the depleted search results. Never thought
| how the implementation would look like though.
| Night_Thastus wrote:
| I love a good tagging system, if it is done properly. It makes it
| so effortless to find the content you want.
|
| I broadly like this post, but I disagree with "The lesson: don't
| let users encode logical paradoxes.".
|
| If a user says they want (A && !A), just hand them back nothing.
| Let the expressiveness exist regardless. (Ideally, warn the user,
| though)
|
| I want expressive tags. It makes everything so much easier. If I
| can't do (A && B) || !(C or D) then as a user I'm going to get
| frustrated if there's a lot of content to sift through. Grouping
| and negation are critical, IMO.
|
| The post touches on this, but I'd say by far the largest issue is
| the moderation of tags. If only the author is allowed to tag, and
| there's no moderation or oversight from other users or official
| moderators, the tag system is effectively useless. Some users
| will be lazy and not tag effectively. Others will tag incorrectly
| by mistake or intention. You want the system to support new tags
| for a variety of reasons, but this allows typos to create new
| tags.
|
| Generally, when I see that a tag system sucks, it's because it's
| been mis-used, not because the underlying logic is bad.
| bccdee wrote:
| The paradox they were referring to was A := (B), B := !(A).
| That's not a expressiveness issue, that's just a dependency
| cycle--there's no reason to allow it.
| Gehinnn wrote:
| A tag system is actually a deterministic computational model
|
| https://en.m.wikipedia.org/wiki/Tag_system
| hwayne wrote:
| Implementable in vim keystrokes!
| https://buttondown.email/hillelwayne/archive/vim-is-turing-c...
| DontchaKnowit wrote:
| Can anyone tell me if anything like this exists :
|
| A program that runs in the background on your machine and watches
| processes and the file handles they open. Then, keeps a database
| of associations between files and programs. E.g. you open a pdf
| with adobe acrobat, and now it adds an entry with a relation
| between that specific file and acrobat, and between pdf file
| extensions and acrobat.
|
| Then, this information is used to provide an intuitive interface
| into the file system. Searching for that .docx file you made last
| year and cant remember for the life of you where you put it? Just
| search for .docx, or files associated with "Word"
|
| This to me is the holy grail of filesystem search, and it doesn't
| seem _that_ hard to implement, but I 've never found anything
| like it.
| mtVessel wrote:
| I'm not following your use cases. Why do you need an
| association database to look for files named _.docx or_.pdf? I
| could see it possibly being useful for apps like VLC which open
| a number of different file types, but then if I used multiple
| apps to open videos I 'd have to remember all the apps I used
| in order to do that search.
| DontchaKnowit wrote:
| Basically windows search is slpw as all fuck if you search by
| file extension. Itd be a lot faster if you just had a DB of
| associations.
|
| But also in the case that I do not know the extension of a
| file, or want to find all files involed in configuring a
| program, etc, it would allow you to find that info quickly.
| giantrobot wrote:
| Spotlight on the Mac has done this for like 15 years. You can
| search for files by content and metadata attributes[0]. You can
| even create "smart folders" that save the search parameters and
| show an up to date instance of the search when opened. The
| Spotlight search is also exposed on the command line with the
| mdfind command.
|
| Spotlight uses the LaunchServices database of file type
| associations to give friendly names to file types but has a
| hierarchy of type identifiers[1].
|
| [0] https://support.apple.com/guide/mac-help/narrow-search-
| resul...
|
| [1]
| https://developer.apple.com/library/archive/documentation/Fi...
| MisterTea wrote:
| Think about this solution for a second. You want a process that
| spies on other processes to build a database of what files they
| touch so you can remember where you put that file last year.
|
| This is a problem that can be solved simply by better
| organizing your files. No security nightmare Rube Goldberg
| contraption needed. Ive been in that boat before and it just
| boils down to organizing your stuff better.
| Joker_vD wrote:
| > You want a process that spies on other processes to build a
| database of what files they touch
|
| It's basically an extension to the VFS that would maintain
| atime and some additional metadata to it, what's so horrible
| about it?
| DontchaKnowit wrote:
| I mean I would write the damn thing myself, Im not worried
| about security.
|
| I just think windows tools for navigating the file system are
| straight garbage and think it could be done a million times
| better by just indexing interactions between processes and
| files. But who knows maybe im barkin up the wrong tree.
|
| Also- organize your files better : first off easier said than
| done and second off, my shit is already a total mess and I am
| not willing to take off work for 3 weeks to organize
| everythibng. I just want a file viewer UI that is actually
| useful in allowing my to find files.
|
| Imo the folder hierarchy paradigm shouldnt have ever been
| exposed to users as the primary mechanism for accessing
| files. It makes no sense for 99 percent of use cases for the
| average user.
| epilys wrote:
| I haven't come across of any project like that, but in case
| anyone wants to implement this and doesn't know where to start,
| here's a way to do it on a freedesktop-compatible linux:
|
| Make a userspace daemon process that adds eBPF tracepoints[0]
| to open{,_at} etc syscalls which match files of your user
| directories with specific extensions (e.g. .docx).
|
| Associate PIDs that open those files with their .desktop
| entries[1]
|
| Store results in some database like sqlite3.[2]
|
| Search this database with your favorite interface, like a CLI
| script or a GNOME shell search provider[3].
|
| I have seen this Rust project on HN which does something
| similar but with file attribute syscalls, you can use it as
| reference: https://github.com/javierhonduco/sweeper
|
| [0]: https://github.com/iovisor/bpftrace [1]:
| https://www.freedesktop.org/wiki/Specifications/desktop-entr...
| [2]: https://sqlite.org/ [3]:
| https://developer.gnome.org/documentation/tutorials/search-p...
| DontchaKnowit wrote:
| This is wonderful, thank you. I may try this out.
|
| Originally wanted to do it for windows but I eventually
| concluded it would take some prodigious hackery to get it do
| work
| epilys wrote:
| I wish I could help you but I haven't touched windows in
| decades. Perhaps someone else can chime in.
| DontchaKnowit wrote:
| So as far as I can tell the only way a user can access
| this type of info on windows is via procmon. Problem is
| its just a gui application, no programatic access afaik.
| I did find a project where someone reverse engineered it
| to some extent and it allowed you to make calls to it
| programatically. Anyway, it turned out there was a loooot
| of heavy lifting that'd be required to get it to behave
| the way I needed it.
|
| Anywho, I'm happy for the info on linux. I was looking
| for a project that would require me to up my bash skills
| anyway. This sounds perfect
| codemac wrote:
| The worst part about going from G to FB was going from buganizer
| to fbtasks.
|
| The internal tool at G has a strict hierarchy called
| "components", and fbtasks is tags alone.
|
| This means if you want to file a bug against anything at G, you
| could navigate this large hierarchy and find a rough spot for the
| bug. Regardless of that team's documentation, tooling, bug
| tracking helper forms, etc you could always file a bug that would
| get pretty close. Then someone on that team would move it into
| the proper component if you got it wrong.
|
| At FB... you go to file a bug, but you need to know which tags!
| If you miss the right tag, that team may never see it. All teams
| implement their tagging differently, and many implement almost
| nothing at all. This devolves to pretty much all bugs being
| tracked through internal group posts, because at least internal
| groups have names and membership lists that are easily
| searchable. Then teams implement bots & programs to take every
| post filed in a group, and auto generate a task with the right
| tags! And pray that they append the `donotreap` tag...
|
| Bugs need to be filed against specific systems, and those systems
| need to be owned by teams. Those teams are in orgs, those orgs
| are in companies. I cringe whenever I see a bug tracker for a
| large company that is not in a hierarchy.
| didgetmaster wrote:
| Good information that deals with tagging issues that I have been
| thinking about and implementing for several years.
|
| I have been working on a new kind of data management system that
| is designed to replace traditional file systems while also doing
| a bunch of database operations. It is an object store that makes
| extensive use of tagging. Nearly all file metadata comes in the
| form of tags attached to each file. Even its name is a tag, so
| every file can have multiple names (each object's unique ID is a
| 64 bit number).
|
| The system is currently in beta and still needs work, but it can
| do some amazing things so far. It can create 100M+ files and
| attach dozen of tags to each one (there is a limit of 255 tags
| per object). Searching for all photos with certain tags, for
| example, can give you the results extremely fast. Even with
| containers with 100 million objects can give you the 100,000 that
| have a specific tag in just a second or two.
|
| Tags have context. Instead of 'John' as a tag, you attach
| Person.FirstName = 'John' to photos of John or documents written
| by John. The whole system is like a sparse relational database
| table where each row/column intersection can have multiple
| values.
|
| We are looking for more beta testers and anyone can download and
| try the software for free. Here are a couple short videos showing
| a few things it can do.
|
| https://www.youtube.com/watch?v=yLPNLHm9fIk
| https://www.youtube.com/watch?v=dWIo6sia_hw
| runlaszlorun wrote:
| Nice. I've wanted something like this for 15 years.
|
| I'm curious how you implemented them. I've been mulling using a
| tag based system for a completely unrelated project. I figure
| one might get a lot of way there using hash tables and sets
| from any of the high level languages. But it seems you maybe
| used a different approach?
| somat wrote:
| If you think hard enough about tag systems you end up with the
| idea that the unix filesystem is a nearly ideal way to tag data.
|
| My hierarchy of tag systems goes something like
|
| key, key=value, hierarchical key(the unix filesystem),
| hierarchical key=value(oh shit you found ldap, abort, you went to
| far)
| giantrobot wrote:
| Iff you assume the use of symbolic links. A tagging system
| needs the ability to have arbitrary tags _and_ the same item
| referenced by multiple tags.
| zamnos wrote:
| If you're on one file system, hard links work as well. They
| have they added benefit that you can delete an entire tree
| but files that are linked to from elsewhere won't get deleted
| or end up with dangling refs.
| kayo_20211030 wrote:
| From 2001, and still the same.
| https://people.well.com/user/doctorow/metacrap.htm
| dsr_ wrote:
| I am not enamored of tags, because tag systems break down when
| humans get involved.
|
| The same person will choose different sets of tags for the same
| file, when asked on different days; they will end up with
| synonymous tags that should be coalesced, but nobody wants to do
| that drudgework.
|
| The semantics of each tag will alter over time and by person.
|
| An optimal tag system is inferior to a full text search system
| with an indexer that recommends keywords and has a reasonable
| syntax for includes, excludes, and boolean logic.
| twic wrote:
| The way i would frame this is that tag systems have a steep
| return on effort expended. Low-effort tagging is garbage, as
| you have observed. But when users stay on top of it for some
| reason, it can be extremely valuable. Someone else mentioned
| the example of AO3 here.
| mjrpes wrote:
| > I am not enamored of tags, because tag systems break down
| when humans get involved.
|
| One of the things I'm curious to try out with GPT is auto
| tagging content and patching documents that have missing tags.
| You train GPT with your corpus and existing (and incomplete)
| tagging system, and have GPT fill in missing tags and revise
| tags that are synonymous or spelling variants.
|
| Maybe there are already good non-AI system that do this, I
| haven't really looked deeply into other solutions yet.
| joshu wrote:
| tagging wasn't meant as a way to organize information for other
| people. it was meant as a way to augment retrieval.
|
| however, your suggested search system doesn't work - many
| people built these and they weren't right - because they don't
| encode whether or not the object had attention.
| cptcobalt wrote:
| This is exactly what the tag alias system of AO3--as discussed
| on this page--is designed to mitigate. If you've spent any
| amount of time on that website, you'll realize their tag system
| works really, really well.
| dsr_ wrote:
| I've seen it. Their tag system relies on a large group of
| volunteers to reconcile the problems, and all the volunteers
| abiding by a large set of rules:
|
| https://archiveofourown.org/wrangling_guidelines/11
| Joker_vD wrote:
| While I absolutely agree with the downsides you mention in the
| 2nd and 3rd paragraphs, the first paragraph is kinda trivial:
| all systems break down when humans get involved. As for the
| last one... what about images? People tag them too, and no
| "full text search system" exist for them unless you count
| "textual inversion"/"interrogate CLIP" which would still
| requires quite some hefty amount of manual tagging for the
| initial training.
| valenterry wrote:
| That does not really speak against tags in general. You can
| also predefine tags for instance.
| remram wrote:
| I think an important part of tagging systems is that you want
| some way to store what tags have been considered. Like, this
| story has "female-protagonist" but not "teen-protagonist":
| someone clearly looked at the characters and applied tags, but
| does that mean it does not have a teen protagonist? Maybe the tag
| was created after someone reviewed the story in order to apply
| "*-protagonist" tags. Maybe that person was only doing "female-
| protagonist" in bulk.
|
| Having a system that can point out unknowns is useful both for
| search (by including them in results) and for filling in that
| data (by suggesting those stories for review).
| Rygian wrote:
| One very prominent use of Key-value tags is the OpenStreetMap
| database. The datamodel is deceptively simple: nodes, ways, and
| relations to represent geographical locations, and a list of key-
| values attached to any of them to define the semantics.
| oneeyedpigeon wrote:
| > "should users be able to query a specific tag alias?" ie, can I
| search for stories tagged with the alias snarry? There's no
| correct choice here. AO3 went with "no".
|
| This is a very interesting point. My initial thought was: there
| definitely IS a correct choice, which is "no". If the choice were
| "yes", that would imply there's a difference between the tags,
| but they're aliases, so there shouldn't be a difference. An alias
| should be exactly interchangeable.
|
| Then I thought about how practically impossible that is to
| enforce, given the nature of language. The usage choice itself
| implies a difference; there _are_ no true aliases in language.
| zamnos wrote:
| I think it depends on the usage is tags, and how important the
| tagging system actually is, because there's a tagging system in
| wide use that allows for aliases. That system is reactj on
| Slack messages. For those outside the slack ecosystem, that
| feature allows you to an emoji to a message, even or especially
| if you're not the author. Thing is, as tiny pictures, people
| remember what they remember about the emoji, and people often
| remember an alias and may not even know the offical name. Slack
| further allows customization by letting users add their own
| reactj imbuing each slack workspace with its own personality.
|
| If someone wants to type in dumbbell and someone wants to type
| in weight, and it goes to the same tiny picture, is that really
| so broken?
| oneeyedpigeon wrote:
| It always bugs me that the default alias for the fist bump
| emoji is "facepunch"...
|
| I think we're ultimately in agreement -- aliases should be
| 'transparent' and not imbued with any specific meaning. Of
| course, it's much easier for a very limited system like emoji
| to be able to deal with these things than an open-ended
| tagging system, especially one that caters for the entirety
| of Unicode.
| at_a_remove wrote:
| AO3 has many, many decisions around tagging that are quirky, at
| best.
| codr7 wrote:
| I've slowly drifted into preferring tags over log levels. Strict
| hierarchies always turn into dead ends sooner or later.
|
| https://github.com/codr7/cl-redb
| codr7 wrote:
| Edit: https://github.com/codr7/cl-slog
| 2sk21 wrote:
| One thought that has occurred to me when using tagging systems is
| that they ought to have the ability to refactor tags. No matter
| what the tagging scheme is initially, it will surely mutate over
| time. It should be possible to rename tags, join tags and divide
| one tag into multiple tags. And while you are at it, perhaps it
| could use some kind of sentence embeddings to suggest tags that
| mean approximately the same thing.
| flir wrote:
| Stemming would get you 95% of the way there in coalescing tags.
| Would sentence embedding outperform that significantly?
| nerdponx wrote:
| In my opinion this is largely a UI/UX issue. For some reason,
| applications with rich tagging systems have only the most raw
| basic interface for manipulating and searching using tags.
___________________________________________________________________
(page generated 2023-04-18 23:02 UTC)