Post AdBKoVPIVXmfrhWD20 by cesare@pl.lain.ch
(DIR) More posts by cesare@pl.lain.ch
(DIR) Post #AdBATKe0KU0krVf8M4 by RustyCrab@clubcyberia.co
2023-12-25T20:33:44.952723Z
1 likes, 2 repeats
does anybody know how I might bypass signed fetch to download post history (I'm making an ebooks bot). I'd rather not set up a whole ass server if I can help it.
(DIR) Post #AdBAaN2BqEQAImKbaK by Inginsub@clubcyberia.co
2023-12-25T20:35:06.360931Z
1 likes, 0 repeats
@RustyCrab use an instance that bypasses signed fetch?
(DIR) Post #AdBAdvN2dVovf9mMd6 by mint@ryona.agency
2023-12-25T20:34:28.260072Z
1 likes, 0 repeats
@RustyCrab Why not download through mastoapi?
(DIR) Post #AdBAueSCml7O7TAd9M by i@declin.eu
2023-12-25T20:38:47.552232Z
0 likes, 0 repeats
@RustyCrab what account? could backfill their profile and get you all their json fresh from the outbox
(DIR) Post #AdBAy0kULlKi8PKAWe by RustyCrab@clubcyberia.co
2023-12-25T20:39:21.600694Z
1 likes, 1 repeats
@mint well I am trying to scrape an account on NCD (nigbot) and they seem to be pretty locked down. I am new to this so I am not sure about the various ways you can get post history. I am using the ebooks outbox query Python script currently. I reached out to Matty but he seems busy.
(DIR) Post #AdBB3523qrxkrpxdVg by RustyCrab@clubcyberia.co
2023-12-25T20:40:17.499572Z
0 likes, 0 repeats
@i @\nice-nigger@nicecrew.digital
(DIR) Post #AdBB6q6jHd8pjmw9dQ by RustyCrab@clubcyberia.co
2023-12-25T20:40:58.839843Z
0 likes, 0 repeats
@zero currently trying to use the ebooks script and I am getting blocked as unauthorized. That's signed fetching isn't it?
(DIR) Post #AdBBF0ds04J6j6KlXc by mint@ryona.agency
2023-12-25T20:41:59.173317Z
1 likes, 0 repeats
@RustyCrab @zero Yes. Shameful display from them.
(DIR) Post #AdBBQIhzAbWjwZdw8m by RustyCrab@clubcyberia.co
2023-12-25T20:44:26.259779Z
1 likes, 0 repeats
@Inginsub I'm looking to scrape nigbot's account. Not continually just one time. I try querying the activititypub outbox and I'm getting "unauthorized", which I think is signed fetching.
(DIR) Post #AdBBcAkEbYe0igM7pQ by RustyCrab@clubcyberia.co
2023-12-25T20:46:35.832112Z
0 likes, 0 repeats
@zero this is probably the right option and I'll just have to manually extract out the post text. PITA but probably required. Nigbot clearly just has a fixed pool of sayings and getting that would be optimal but I guess Matty is busy.
(DIR) Post #AdBBeGxsRwniadjr8q by Inginsub@clubcyberia.co
2023-12-25T20:47:01.553861Z
1 likes, 0 repeats
@RustyCrab why not scrape its local accout, https://clubcyberia.co/users/$ANcyiVnc3H7nkCtR2Gthe posting history will be incomplete, but I don't think you'll lose a lot
(DIR) Post #AdBBlrO1QuypZpaN96 by mint@ryona.agency
2023-12-25T20:44:23.151157Z
0 likes, 0 repeats
@RustyCrab Alex has a readymade library for HTTP signatures, but it's in node and I don't know if anyone made anything similar for Python.https://gitlab.com/soapbox-pub/fedisign
(DIR) Post #AdBBq3AXZGgRHJHHEG by RustyCrab@clubcyberia.co
2023-12-25T20:49:07.240065Z
0 likes, 0 repeats
@Inginsub yes that was my thought and I'm surprised that the ebooks script does not work that way to begin with. I'll need to figure out how to scrape the local instance instead rather than having it query the outbox on the home instance. I'll be honest I thought this was just going to be really simple like it always has been so I wasn't eager to learn a new API/toolset.
(DIR) Post #AdBC4FS0hXXRQKDedM by Inginsub@clubcyberia.co
2023-12-25T20:51:40.679952Z
1 likes, 0 repeats
@RustyCrab I don't know how the ebooks script works, but it probably should be able to pull posts from ANcyiVnc3H7nkCtR2G
(DIR) Post #AdBCTqJMn8akz5NezY by i@declin.eu
2023-12-25T20:56:21.397211Z
3 likes, 0 repeats
@RustyCrab making an ebooks bot off a fortune fille is :niggainsanest:, if getting it off cyberia's timeline doesn't work out, i'll make a dump later
(DIR) Post #AdBCYi4aqgDYsin7tQ by RustyCrab@clubcyberia.co
2023-12-25T20:57:09.035469Z
1 likes, 0 repeats
@Inginsub I will try but that isn't the way it's built. You have to give it an access token to an account which is following the accounts you want to scrape. It then queries outboxes. I will see if I can get it to query the local user instead.
(DIR) Post #AdBE93RmxQVDYFXF1U by RustyCrab@clubcyberia.co
2023-12-25T21:14:57.115719Z
0 likes, 0 repeats
@i yes I am making a bot of a bot but you'll see why when I get it up
(DIR) Post #AdBKoVPIVXmfrhWD20 by cesare@pl.lain.ch
2023-12-25T22:29:45.672374Z
2 likes, 0 repeats
@RustyCrab @Inginsub vantablack coal account
(DIR) Post #AdBkgtzpnbX2cRdx9k by RustyCrab@clubcyberia.co
2023-12-26T03:19:36.411556Z
2 likes, 0 repeats
@Inginsub came back from festivities. Figured out how to use the API to query statuses from the cyberian nigbot. However there seems to be a serverside limit of how many it will return so I am going to have to make a loop to pull them down using the last post id gotten. Always something.
(DIR) Post #AdBlBQBcgJ9f8ti5Cq by Inginsub@clubcyberia.co
2023-12-26T03:25:08.638375Z
2 likes, 0 repeats
@RustyCrab yeah, pleroma only returns 40 and offset does nothing, so you need to fetch posts older than a specified id
(DIR) Post #AdBlTtjZYxeNzlOraa by PunishedD@clubcyberia.co
2023-12-26T03:26:29.859486Z
1 likes, 0 repeats
@RustyCrab @Inginsub This is why the standard response when a woman asks "what are you thinking about" is "nothing". It's only sometimes a lie, but it's always better than explaining.
(DIR) Post #AdBlWvO7ryCXzgoOGG by RustyCrab@clubcyberia.co
2023-12-26T03:29:01.174712Z
1 likes, 0 repeats
@PunishedD @Inginsub not now babe I'm fingering nigbot
(DIR) Post #AdBrutgOSdzhNcAd84 by RustyCrab@clubcyberia.co
2023-12-26T04:40:33.003560Z
2 likes, 0 repeats
@Inginsub I have written an iterator and I have obtained niggaposts. 11,000 niggaposts
(DIR) Post #AdBs2axDydslbwiJ5U by Inginsub@clubcyberia.co
2023-12-26T04:42:01.645795Z
1 likes, 0 repeats
@RustyCrab there’s 20,000 niggaposts total
(DIR) Post #AdBsOAMdWFdDVvsyoq by RustyCrab@clubcyberia.co
2023-12-26T04:45:52.522047Z
1 likes, 0 repeats
@Inginsub that's okay I don't need that many
(DIR) Post #AdByMk7UD1RxYPVJ4a by RustyCrab@clubcyberia.co
2023-12-26T05:52:48.382966Z
2 likes, 0 repeats
@Inginsub the markov chain is very staunchly discriminating against the niggaposts :niggadementia:
(DIR) Post #AdByazkKuGBo4bTIie by cassidyclown@clubcyberia.co
2023-12-26T05:55:28.209123Z
1 likes, 0 repeats
@RustyCrab @Inginsub :nigganigbot:
(DIR) Post #AdBykghyXoP9vJzOym by RustyCrab@clubcyberia.co
2023-12-26T05:57:08.159643Z
1 likes, 0 repeats
@cassidyclown @Inginsub this is actually absurd. The model is being fed niggaposts but it's outright refusing to incorporate them into any outputs.
(DIR) Post #AdBytGyhZttCjFuGuW by cassidyclown@clubcyberia.co
2023-12-26T05:58:43.300236Z
0 likes, 0 repeats
@RustyCrab @Inginsub how
(DIR) Post #AdBz0ZAMLzmjAIpxpo by cassidyclown@clubcyberia.co
2023-12-26T06:00:04.416936Z
1 likes, 0 repeats
@RustyCrab @Inginsub what are you using for it?
(DIR) Post #AdBz318rJMtsVfIubA by cassidyclown@clubcyberia.co
2023-12-26T06:00:31.255003Z
1 likes, 0 repeats
@nice-nigger @Inginsub @RustyCrab eerie
(DIR) Post #AdBz3t6ag1AhsGerYm by RustyCrab@clubcyberia.co
2023-12-26T06:00:32.192153Z
1 likes, 0 repeats
@nice-nigger @Inginsub @cassidyclown see that's what it should be doing
(DIR) Post #AdBz6guYTEvqvKMhxQ by RustyCrab@clubcyberia.co
2023-12-26T06:01:09.515035Z
1 likes, 0 repeats
@cassidyclown @Inginsub it's a surpriseif I can stop hitting brick walls :niggadementia:
(DIR) Post #AdBz7M4DSE1OZACVCy by cassidyclown@clubcyberia.co
2023-12-26T06:01:09.791236Z
0 likes, 0 repeats
@nice-nigger @Inginsub @RustyCrab tell us your secrets
(DIR) Post #AdBzELzc82DQMZNxI0 by cassidyclown@clubcyberia.co
2023-12-26T06:02:34.733427Z
0 likes, 0 repeats
@RustyCrab @Inginsub I mean what are you using to do the markov chain stuff
(DIR) Post #AdBzKEAs6tVFqtS5NA by RustyCrab@clubcyberia.co
2023-12-26T06:03:35.803760Z
1 likes, 0 repeats
@cassidyclown @Inginsub markovify with the pleroma ebooks python script
(DIR) Post #AdBze0QAxCDEzO2cRE by RustyCrab@clubcyberia.co
2023-12-26T06:07:08.875544Z
1 likes, 0 repeats
@cassidyclown @Inginsub I think I'm going to have to play around with the model because this isn't working at all :grinching:
(DIR) Post #AdBzyzta6TlXRKnyOe by RustyCrab@clubcyberia.co
2023-12-26T06:10:55.947292Z
1 likes, 1 repeats
@Inginsub @cassidyclown if I feed the model nothing but niggaposts it actually freezes. What is this ghost in the machine :chiggerniggaseverementaldistress:
(DIR) Post #AdC0JqRcRAg0D2fOpE by RustyCrab@clubcyberia.co
2023-12-26T06:14:42.970552Z
1 likes, 0 repeats
@Inginsub @cassidyclown I think I see what's wrong :transchiggerniggaseverementaldistress:
(DIR) Post #AdC0StzibkHzpFw3MW by diazepam@clubcyberia.co
2023-12-26T06:15:29.976703Z
1 likes, 0 repeats
@RustyCrab @Inginsub @cassidyclown wat did u do
(DIR) Post #AdC0Suvr7anGjZ8Tyq by RustyCrab@clubcyberia.co
2023-12-26T06:16:21.929203Z
1 likes, 0 repeats
@diazepam @Inginsub @cassidyclown formatting thing (probably)
(DIR) Post #AdC0nAK1c0pGFx60OG by Kerosene@bae.st
2023-12-26T06:20:07.988385Z
5 likes, 1 repeats
@RustyCrab @Inginsub @cassidyclown What is this thing man I can't even lmaoimage.png
(DIR) Post #AdC0tl39yHM7u5LoO0 by cassidyclown@clubcyberia.co
2023-12-26T06:21:16.369385Z
2 likes, 0 repeats
@Kerosene @Inginsub @RustyCrab > What is this thingI think you should find the shortcode sufficiently descriptive
(DIR) Post #AdC1fpN3D3vlCjZm7M by FrailLeaf@ryona.agency
2023-12-26T06:29:59.046736Z
1 likes, 0 repeats
@Kerosene @Inginsub @RustyCrab @cassidyclown proper emochi description
(DIR) Post #AdC1wGalllNOGMpcTQ by RustyCrab@clubcyberia.co
2023-12-26T06:32:52.755809Z
1 likes, 0 repeats
@diazepam @Inginsub @cassidyclown so, no, it's not a simple formatting error. I tried limiting the length to short sentences and when fed ebonics the markov generator is not capable of producing anything less than a full paragraph of text. Manmade horrors beyond my comprehension :niggadementia:
(DIR) Post #AdC4IC2DartyPMFmnQ by RustyCrab@clubcyberia.co
2023-12-26T06:59:12.124459Z
1 likes, 0 repeats
@diazepam @Inginsub @cassidyclown I think I need to rethink my strategy for this bot. Markov is not able to make heads or tails of ebonics :niggadementia:
(DIR) Post #AdCFZYLwy1RG2sxnQe by Inginsub@clubcyberia.co
2023-12-26T09:05:39.259119Z
2 likes, 0 repeats
@RustyCrab @cassidyclown @diazepam depending on which ebooks bot you're using it may not actually be based on markov chains. Zero's bot uses natural language processing, for example (not sure about any other)
(DIR) Post #AdCFkoEeG2KyUMuoxE by RustyCrab@clubcyberia.co
2023-12-26T09:07:42.799533Z
2 likes, 0 repeats
@Inginsub @cassidyclown @diazepam no it's definitely markov. I'm in the guts of this thing making some modifications. That said I do want to look into something more sophisticated.
(DIR) Post #AdEnSGzYu0U4vp6z0i by RustyCrab@clubcyberia.co
2023-12-27T14:34:45.429965Z
1 likes, 0 repeats
@zero @mint I think it prevents some of the more blatant kinds of malicious scraping like MIT has been doing lately. Doesn't stop it fully but a lot of these scrapers are lazy and just won't notice if an instance gets missed