.\" Text automatically generated by md2man .TH tmww-alts 1 "November 28, 2014" "Linux" "Linux Reference Manual" .SH NAME \fBalts.plugin \fP- query player database for alt/forum name .PP .SH USAGE .TP .B subcommand: char -- character database handler .PP .nf .fam C add id ID char CHAR -- add id/char pair to db; write conflicts to log resolve id ID char CHAR -- same as add + resolve all matched alts in playerdb into accounts grep [ names | ids ] REGEXP -- search known names, output names/names with ids fuzzy [ names | ids ] PATTERN -- case-insensitive levenshtein distance 1 search get [ [ id ] by char CHAR ] -- get CHAR acc_id show [ names | ids | parties ] by id ID -- get all known chars on acc_id show [ names | ids | parties ] by char CHAR -- get all known chars on same account as CHAR dig REGEXP -- grep + show ids by ids from grep matches sanitize -- remove older duplicate entries; write conflicts to log merge FILENAME -- put FILENAME into db; write conflicts to log .fam T .fi .TP .B subcommand: party -- party database handler .PP .nf .fam C add party PARTY char CHAR get [ by char ] CHAR -- get char's party name show [ ids | names | players ] by { party PARTY | char CHAR } -- party members lookup { grep | fuzzy } PATTERN -- grep/approximate grep party name sanitize -- show duplicates in partydb merge FILENAME -- put FILENAME into db; conflicts pushed to db and listed in merge log .fam T .fi .TP .B subcommand: player -- players database handler .PP .nf .fam C ref -- field types quick reference create PLAYER remove PLAYER rename PLAYER to PLAYER add PLAYER FIELD value VALUE add PLAYER FIELD element VALUE -- adding alts will autoresolve charname into account resolve PLAYER -- resolve all player alts into accounts del PLAYER FIELD del PLAYER FIELD element VALUE get { CHAR | by { char CHAR | id ACCID } } -- dereference player entry ids PLAYER -- print all known associated account IDs show [ ids | names | parties ] by { char CHAR | id CHAR | player PLAYER } list with FIELD list with { FIELD [ not ] as VALUE | VALUE [ not ] in FIELD }+ dump PLAYER -- dump JSONline record of PLAYER; tmww player dump veryape record NUMBER -- access players db record by it's order number append STRING -- NOT SAFE append JSON player record of same format as with dump operation to end of dbplayers you should try sanitize operation if you not sure if there are duplicate entries or fields keys PLAYER -- tmww player keys veryape field PLAYER FIELD [FIELD]\.\.\. -- tmww player field veryape name aka search STRING -- simple search in all fields sanitize -- remove keys with 0 length - empty arrays and hashes with null value resolve alts into accounts, report duplicate accounts and alts lregen -- regenerate shortened playerdb version if limiteddb is in use FIXME merge FILENAME -- simple merge player records + sanitize FIXME forcemerge FILENAME -- replace duplicated records with new ones + sanitize .fam T .fi .PP subcommand: arseoscope CHAR -- observe player alias/number of known accounts/alts .PP .SH GLOSSARY .TP .B accid, id account ID seen from game client attached to player accounts. Charnames on same account has same account ID. .TP .B charname character name - name you see in game .TP .B party name of party character is in .TP .B player player alias in dbplayers, which gathers accounts and alts (with yet unknown accid) known to be played by single person under one name .TP .B dbchars \fIdbchars.txt\fP - sorted account ID to charname reference .TP .B dbparty \fIdbparty.txt\fP - sorted partyname to charname reference .TP .B dbplayers \fIdbplayers.jsonl\fP - file describing player aliases - accounts, unresolved alts and extra info; IDs found in dbchars are resolved to account IDs in player record .PP .SH DESCRIPTION WARNING: alts.plugin require jq of version >= 1.4 .PP tmww plugin \fBalts\fP creates and perform all operations/lookup on players alts database and player lists. Database consists of few per server cleartext/jsonl files with format described later. .PP Player database goal is to combine known accounts and belonged chars under common alias. Accounts are automatically collected into chars.txt. Alias for player in players.jsonl contains \fIaccounts\fP mapping with known account IDs and \fIalts\fP mapping with known names but yet unknown account IDs. In case bot which collect account IDs determine missing account ID it'll remove alt name from players.jsonl record and instead add account ID. .PP Most lookup commands expected output of which is single value take only first matching record. Collision detection is specific for each db. .PP Players DB in two words: players.db -> records -> fields -> values. Each record has "player" alias. Character names with unknown account IDs are listed in "alts" field. Every other known char is referenced over "accounts" field. .PP .SH CONFIG .TP .B ALTSPATH : string : ${DIRCONFIG}/alts DB location .TP .B LIMITED : yes/no : no see notes on _Limited access_ .TP .B UPDATELIMITED : yes/no : no see notes on _Limited access_ .PP .SH NOTES NOTE: no \fIalts\fP plugin actions are intended to be called from \fIactions\fP config section. .PP Since whole thing is bunch of shell scripts working with text files, there's improvised lock per server when operation modifies alts db. Timeout is 5 seconds, after which it will skip lock and overwrite temporary files. .PP Another nasty detail about shell is quotation. It's recommended to use single quote to prevent shell variables expansion and so on. When you need to put single quote inside single quote close main quotation and enclose single quote in double quote; do this .PP .nf .fam C echo '"chips'"'n'"'fish"' .fam T .fi .PP to achieve this .PP .nf .fam C "chips'n'fish" .fam T .fi .PP Otherwise use escaping quotes. .PP For conflict/merge'n'add collision log you should try \fIdb_path/servername/char_conflicts.log\fP or similar name for party conflicts. .SS Char operations Most operations requiring char name are case-sensitive. .PP Amount of alts on same account limited on query to 30. .PP By defaul fuzzy search performed from huge slow regexp pattern constructed in script which is case insensitive, allow 1 absent char or 1 missed char. Fuzzy pattern will skip spaces and won't accept lot of special chars. See implementation for details. You can use agrep instead (if you have it). It's not recommended to run fuzzy search with pattern of less than 4 chars. .PP On add operation all duplicate chars will be removed to conflicts log. This operation is safe when character was moved to account with lower id. .PP Default merge strategy will remove all duplicate entries with lower account ids to conflicts log. Default chardb format lacks timestamps to correctly resolve duplicates. .PP Subcommand sanitize will perform same strategy on chardb without additions. .PP Subcommand dig is combination of grep + show ids by id for all grep matches. .SS Party operations Amount of alts in same party limited on query to 15. .PP Subcommand add will move colliding entries to party conflicts log. .PP Party merge will only combine files and remove duplicates. Collisions should be removed by hand. .PP Subcommand sanitize prints partydb duplicate entries. .SS Player operations Player database is JSONlines file with predefined fields with record structure like this: .PP .nf .fam C {"player":"asd","field1":"value","field2":["element1","element2"]} .fam T .fi .PP Most operations on players DB performed using jq json swiss army knife. So if you don't have it or don't want to setup players db fix accsniffer (if you're using it) tmww operation from "resolve" to "add" - it will only add char into chardb and skip playerdb. .PP Records are usually referenced by _player_; there are dedicated commands to rename and delete entry to lessen typo errors. There are 2 general use cases for this DB: automatic alts resolve and additional data storage to be then queried. .PP First case require manual add of elements into \fIalts\fP field, which are char names and get resolved into account IDs on \fIsanitize\fP or \fIresolve\fP commands or in future after matching char resolve. .PP Second case allow storing of associated emails or something like marking of active developers and tmwc members which allow queries like: .PP .nf .fam C tmww -a alts default player list with tmwc as true and code in roles .fam T .fi .PP When you need to store single backslash as field value - it will be added as is. Duplicate backslashes if you want to insert two or more backslashes in row. .PP Adding elements will only check if duplicate was in field, it doesn't touch duplicates in other fields or records. Sanitize won't touch them either, except \fIaccounts\fP and \fIalts\fP fields. .PP As a measure to preserve original ACL group of db files with multiuser access, after operations on db files done they are moved back using "cat"; rsync only preserved permissions but failed to preserve group. .SS Sharing altsdb for multiple users Example setup grants full altsdb access (ALTSPATH) to usergroup simply setting up group and permissions on altsdb files. Shared LOCK is also required (e.g. in shared TMP). .PP IMPORTANT: most probably on fresh run you'll have to touch and chmod db files the way you need them (e.g. to disable/enable world read access) .PP Some altsdb operations assume ACL is set to allow g+w access. .PP .nf .fam C ALTSPATH /share/folder/alts LOCK /shared/folder/.tmp .fam T .fi .PP Sharing limited access to other users should be done with wrapper script above main tmww to enforce "limited" plugin and filter off modifying commands. .SS Limited access It's possible to provide limited access, e.g. for sharing access to limited db over whispers. Reason to remove aliases - throw away GMs and conflicting players + add some information noise, so conflicting users can't freely guess on chars excluded from access to be desired suspected alts. Limited policy users should not be able to write to db and should not see own records for obvious reasons. .PP .nf .fam C tmww -ya alts tmw.org arseoscope jdoe tmww -ya alts limited arseoscope jdoe .fam T .fi .PP Filtered player records are listed in \fIUTILPATH/lregen.players\fP one player alias per line (empty lines and comments starting with "#" allowed). So if jdoe was filtered, arseoscope on core db will show jdoe record and jdoe alt on account, but with limited access will only show jdoe alts on account and no connected accounts. .PP Limited base can be regenerated with "player lregen" command or if UPDATELIMITED config option is set to "yes", limited base will be regenerated on every update of main base. .PP Additional lines of upper example: .PP main config: .PP .nf .fam C UPDATELIMITED yes .fam T .fi .PP limited access config: .PP .nf .fam C LIMITED yes .fam T .fi .SS Using RCS for db archiving For ease of use it's recommended to keep files under RCS with common prefix, e.g. dbchars.txt, dbparty.txt, dbplayers.jsonl. RCS will break file permissions even with ACL enforced; it looks at write permission to check if file locked; if we need files available for group access and use RCS as reserve archiver you'll have to manually chmod 660 files (with git it's done setting up hook). .PP NOTE: no need for chmod operations for single user install .PP .nf .fam C Create rcs archive with no default keyword substitution: $ mkdir RCS && chmod 660 db* && rcs -i -kk -t-'.' -U -M -q db* .fam T .fi .PP .nf .fam C Initial commit/commit new version: $ ci -u -m -q db* && chmod 660 db* .fam T .fi .PP .nf .fam C Show versions: $ rlog dbchars.txt .fam T .fi .PP .nf .fam C Show difference between current version and last committed version: $ rcsdiff dbchars.txt .fam T .fi .PP .nf .fam C Checkout last committed version: $ co -f -q db* && chmod 660 db* .fam T .fi .PP .nf .fam C Checkout particular version (reported from rlog): $ co -f1.2 -q db* && chmod 660 db* .fam T .fi .PP .SH FORMAT .SS dbchars.txt .nf .fam C acc_id __ char_name .fam T .fi .PP Corresponding collision log is by default char_conflicts.log. .SS dbparty.txt .nf .fam C partyname __ char_name .fam T .fi .PP Corresponding collision log is by default party_conflicts.log. .SS dbplayers.jsonl JSONlines consists of self-sufficient json record per line. Each line is called here a record. Every record of player.jsonl consists of mappings with sequences or strings as values. All numbers and bool values should be written as strings. Bool values convention is "true" and "false". No nested structures allowed. .PP Chars allowed for player name are lower/uppercase, digits, space, dash and underscore but recommended convention for player names is only lowercase with digits. Field names are forced as lowercase alphabet only. .PP There's set of predefined field types which is veryfied on "player add" and "player sanitize" operations. Fields not listed here aren't checked. .PP Example dbplayers.jsonl record: .PP .nf .fam C {"player":"jdoe","alts":["alt1"],"accounts":["2112233"],"tmwc":"true"} .fam T .fi .SS Default string fields .TP .B player fixed player alias .TP .B name IRL name .TP .B wiki full wiki link .TP .B trello full trello link .TP .B server own server .TP .B port login server port on own server .TP .B tmwc \fItrue\fP if player is in TMWCommittee .TP .B active \fItrue\fP if player is active (more useful to mark developers and GMs) .TP .B cc country code (reference taken from IANA domains) .PP See recommended fields with "tmww player ref" .SS Default array fields .TP .B forum tmw.org forum names .TP .B aka IRC names, code signatures, whatever .TP .B roles set of prefedined roles recommended values are: "content", "sound", "gm","dev", "map", "pixel", "admin", "host", "wiki", "advisor", "translator", "packager" .TP .B alts associated charnames .TP .B accounts associated account IDs .TP .B links personal blogs, facebook, other traces .TP .B xmpp xmpp .TP .B mail mail .TP .B skype skype .TP .B repo gitorious/github/bitbucket/whatever .TP .B tags random tags; e.g. to to mark scammers .TP .B comments any comment .PP See recommended fields with "tmww player ref" .PP .SH EXAMPLES Next examples demonstrate usage with distributed configs and zsh aliases: .PP .nf .fam C # char ops .fam T .fi .PP .nf .fam C $ tmww -a alts tmw.org char dig nard 2172156 Bernard. 2172156 Nard 2172156 Nardis 2172156 Sidran 2179685 Luxima 2179685 Marguerite 2179685 Nard. 2186035 Cornelius 2186035 CRC-Nard 2186035 .Nard $ tc grep ids '^nar' 2115541 naruto 2121285 Narus 2172156 Nard 2172156 Nardis 2179685 Nard. $ tc fuzzy ids tormanov 2155980 Thormanov $ tcs Grim Grim Grim! $ tcg veryape 2215093 $ tcsi 2215093 Grim Grim! .fam T .fi .PP .nf .fam C # party ops .fam T .fi .PP .nf .fam C $ tgg Nard ☽Amaluna☾ $ tgsp Nard 2214854 Zirry 2186438 johannelaliberte 2088875 mandypinkmind 2214155 rena 2224509 Joseph^Sod 2172156 Nard 2206252 Rill .fam T .fi .PP .nf .fam C # player ops .fam T .fi .PP .nf .fam C # get player alias $ tpg Houston # this will give alias if present and dump all know alts on same acc $ ta Houston # show all known alts with ids for alias $ tps willee # show parties for every char of alias $ tpsp willee .fam T .fi .PP .nf .fam C # dump player record $ tpd bjorn $ tp field Bjorn mail xmpp # example queries $ tpl cc as de and content in role $ tpl code in role and tmwc as true $ tpl code in role or admin in role $ tp search orziffer .fam T .fi .SS Example config directory structure .nf .fam C config/ ├── lists/ │   └── server.themanaworld.org/ │   ├── char_name/ │   │   ├── auto.guild.fixes │   │   ├── auto.guild │   │   └── auto.party │   ├── guilds/ │   │   └── CRC │   ├── auto.gm │   ├── alarm -> friend │   ├── bot │   ├── friend │   └── foe ├── alts/ │   └── server.themanaworld.org/ │   ├── RCS/ │   ├── dbchars.txt │   ├── char_conflicts.log │   ├── dbparty.txt │   ├── party_conflicts.log │   └── dbplayers.jsonl ├── plugins/ │   ├── accsniffer.plugin │   ├── accsniffer.zsh │   └── alts.plugin ├── utils/ │   ├── accsniffer │   └── validjsonl.py ├── default.conf -> tmw.org.conf ├── arseoscope.conf └── tmw.org.conf .fam T .fi .PP .SH NOTES .SS Pretty-print playerdb query .nf .fam C $ tmww player nlist with tmwc as true and active as true | while read line; do tmww player dump $line | jq -r '"\(.player) (\(.name))\(.mail // empty | " <" + .[] + ">" )"'; done .fam T .fi .PP which will print only matches with email, so it results in something like (multiple mails on single record generate multiple lines): .PP .nf .fam C irukard (Krzysztof Daszuta) rotonen (Joni Orponen) wombat (wombat) .fam T .fi .SS How to search forum/charname pairs Searching player by forum name is done using "player search" command. .PP Searching forum name of char (assuming charnames in player records are automagically substituted with accounts) is done by: .IP 1. 4 "player get" + "player field PREV_RESULT forum" .IP 2. 4 "player search" if charname isn't yet in chardb .SS How to browse roles .nf .fam C # print all role tags (recommended and custom) jq -r '.roles[]' dbplayers.jsonl 2>&- | sort | uniq -c | sort -rn # print all players with specified role tp list with pixel in roles .fam T .fi .SS Number of newbies approaching spot with active accsniffer .nf .fam C # usage: charseen [ ] # example: charseen 100; charseen 200 100 charseen() { tail -n "$1" dbchars.txt | head -n "${2:-$1}" | cut -d ' ' -f 1 | uniq | awk 'NR>1{print $1-a}{a=$1}' | sort -n | awk '{a+=$1;b[NR]=$1} END{print NR " uniqs, av. " a/NR ", med. " b[int(NR/2)]}' } .fam T .fi .PP .nf .fam C $ cd $(tmww -g ALTSPATH) && charseen 100 88 uniqs, av. 11.25, med. 8 .fam T .fi .PP .SH BUGS Results on some operations/queries to check if map:[array] contains exact element might be unexpected ("jq contains" will return true if pattern is matched as substring; expression for strict matching was tested where possible). Few commands has substring check on purpose, e.g. "player list". .PP Substring matching is case sensitive e.g. in "player list with Chaos in forum", which will output "axzell", because he has "ChaosCrossAG" forum name, but with "player list with chaos in forum" is will output "chaosava". .PP jq 1.4 added \fB-S\fP key to sort hashes allowing more readable diffs. If you have jq of earlier version just remove \fB-S\fP key from jq calls in \fIplayers.lib.sh\fP . .PP Default recommended fields and roles are hardcoded in 4 places: markdown manual source, plugin, zsh completion and optional validation script. .PP .SH COPYRIGHT This document is part of tmww - The Mana World Watcher scripts. .PP Licensed under terms of GNU General Public License version 3. For full text of license see COPYING file distributed with tmww. .PP .SH AUTHORS willee , 2012-2014 .PP .SH SEE ALSO \fBjq\fP(1), \fBtmww\fP(1), \fBtmww-config\fP(5), \fBtmww-accsniffer\fP(1), \fBtmww-pysniffer\fP(1)