Post AhrRCP7L94CjeZhBBI by J12t@social.coop
 (DIR) More posts by J12t@social.coop
 (DIR) Post #AhrRCP7L94CjeZhBBI by J12t@social.coop
       2024-05-02T20:53:52Z
       
       0 likes, 0 repeats
       
       Are there any software #testing frameworks that support testing outcomes other than PASS and FAIL? E.g. “message successfully received but content needed to be lossily reformatted”? Wondering how APIs would look like. #programming
       
 (DIR) Post #AhrRCQ71RjXojsYRKC by philip@mastodon.mallegolhansen.com
       2024-05-02T20:59:16Z
       
       0 likes, 0 repeats
       
       @J12t Realize this isn’t exactly what you’re asking, but the way I would achieve that in a “traditional” testing framework would be: One test confirms response != null, that test is passing, second test confirms that data format is as expected, that test fails.
       
 (DIR) Post #AhrRCRAbVu0I1HEoXw by J12t@social.coop
       2024-05-02T21:08:00Z
       
       0 likes, 0 repeats
       
       @philip For Fediverse testing, I"m looking to produce reports that say something like Pass / Fail / Degraded where Degraded is things like "objects regardless of type have become a Note". Thinking of maybe having two types of AssertionErrors; still has the problem that if they are thrown as exceptions, a test will stop right there.
       
 (DIR) Post #AhrRCS3YDbxKlgwhBw by mat@friendica.exon.name
       2024-05-03T06:13:37Z
       
       0 likes, 0 repeats
       
       The testing mantra is, every test should test just one thing, and I don't see a good reason to depart from that rule here. If there are two things that can go wrong, run the test twice. Efficiency shouldn't be a concern at this point of the pipeline."Degraded" doesn't really say anything actionable to me as a developer or a user. I'd be better off just looking at a straight percentage of failed tests, and diving into the details of the ones that "fail". I'd certainly much rather have a clear yes/no answer to each individual question.I certainly wouldn't choose my testing framework based on the requirement that it doesn't just pass or fail. Best to treat your difficulty in finding such a thing as a message.