Post ATQMM2Nb7wLbbXYZt2 by null@puddle.town
(DIR) More posts by null@puddle.town
(DIR) Post #ATQMM1cnvuuvGPfCwy by tinker@infosec.exchange
2023-03-08T22:11:37Z
0 likes, 0 repeats
Bash data parsing question....I have two csv files. I need to combine them in a certain way. I can't use the 'join' command, as the specific fields I want to match are not perfectly sorted.Here is an example of the two files:file1.csvrecipe01,chickenrecipe02,beefrecipe03,chickenrecipe04,fishfile2.csvchicken,clucksbeef,moosfish,warblegarbleI need to find a way to create the following file:file3.csvrecipe01,chicken,clucksrecipe02,beef,moosrecipe03,chicken,clucksrecipe04,fish,warblegarbleAnyone know a good command, program, or script that would do that? Go through field 2 of the first file and match it to field 1 of the second file, and add field 2 of the second file to the line?#bash #linux #datascience
(DIR) Post #ATQMM2Nb7wLbbXYZt2 by null@puddle.town
2023-03-08T22:16:38Z
0 likes, 0 repeats
@tinker awk -F',' 'NR==FNR{a[$1]=$2; next} {print $0","a[$2]}' file2.csv file1.csv
(DIR) Post #ATQMM3FpsHjUJkvtQW by tinker@infosec.exchange
2023-03-08T22:12:05Z
0 likes, 0 repeats
Should probably just post this to stackoverflow... But I wanted to post it here as I like it here.
(DIR) Post #ATQMM3L9YVzCaFQ8iO by null@puddle.town
2023-03-08T22:18:02Z
0 likes, 0 repeats
@tinker -F',' specifies the field separator as a commaNR==FNR{a[$1]=$2; next} reads the first file (file2.csv) and stores its second field in an array a indexed by the first field{print $0","a[$2]} reads the second file (file1.csv) and prints each line followed by the corresponding value in the a array indexed by the second field
(DIR) Post #ATQMM3zv6wb0cgUhG4 by tinker@infosec.exchange
2023-03-09T00:16:47Z
0 likes, 0 repeats
@null - See I've been using awk for a while now, but really only for rudimentary uses. I reaaaalllly need and want to dive in fully.Thank you for this example! I'll run with it.
(DIR) Post #ATQMM4YJ366MLKa9r6 by clacke@libranet.de
2023-03-09T01:43:50Z
0 likes, 1 repeats
@tinker @null If you want a calm and patient voice to guide you through everything awk I highly recommend the 16-part series by @perloid and https://hackerpublicradio.org/correspondents.php?hostid=300 with some extra tips by @klaatu :https://hackerpublicradio.org/series.php?id=94If audio is not your thing the show notes are still *chef's kiss*.@hpr
(DIR) Post #ATQMM4w3cn6HWzh7sO by tinker@infosec.exchange
2023-03-08T22:58:20Z
0 likes, 0 repeats
As folks are mentioning, xlookup on Excel does this! I'm trying to find a way to do it within Bash or *nix environment.
(DIR) Post #ATQMM6lqni7lDuw0FE by tinker@infosec.exchange
2023-03-09T00:31:56Z
0 likes, 0 repeats
Alright, but holy shit do folks on Mastodon help out in a pinch! I got so many great answers to my question with so many different ways to go about solving it. Cheers all!
(DIR) Post #ATRBjKa2icDSNMF5UW by jonathanf@toot.thoughtworks.com
2023-03-08T22:53:02Z
1 likes, 1 repeats
@tinker have you tried csvq? https://mithrandie.github.io/csvq/reference/select-query.html
(DIR) Post #ATRBjLU3MN1FB4RonI by tinker@infosec.exchange
2023-03-09T00:14:31Z
0 likes, 0 repeats
@jonathanf - Ok, but wow.
(DIR) Post #ATRBjM3VEZNKx12836 by jonathanf@toot.thoughtworks.com
2023-03-09T07:04:01Z
1 likes, 0 repeats
@tinker I like this approach because is legible and easy to adapt and change to your need. Here you have:csvq --no-header -format csv 'SELECT A.c1, A.c2, B.c2 FROM file1 AS A LEFT JOIN file2 AS B on A.c2=B.c1'