[HN Gopher] TDD from the Factorio Team
___________________________________________________________________
TDD from the Factorio Team
Author : sorahn
Score : 423 points
Date : 2021-06-18 10:53 UTC (12 hours ago)
(HTM) web link (factorio.com)
(TXT) w3m dump (factorio.com)
| happyweasel wrote:
| You can TDD as much as you want to once the initial game
| mechanics are in place and a gameprotoype shows enough promise to
| be realized until completion. Because then most of the core
| stuff/ideas/principles won't wildly change and won't be thrown
| away.. The core mechanics are in place. But would TDD help you
| reach that stage? I guess it is simply too much overhead. So
| yeah, this is TDD after the fact ;-). I still love TDD :)
| chii wrote:
| > But would TDD help you reach that stage?
|
| if your game has a lot of interactions, and you want to make
| sure that your changes are not causing unintended interactions,
| tests like these would help a lot during development.
| marcosdumay wrote:
| There is always a comment with that claim on a TDD thread.
| Just to clarify, writing tests is not TDD. There are plenty
| of ways you can have a codebase full of tests, TDD is only
| one.
|
| But anyway, I doubt tests help at all in the prototype phase
| (by any procedure you want to get them). My guess is that
| they are incredibly harmful.
| meheleventyone wrote:
| > You can TDD as much as you want to once the initial game
| mechanics are in place and a gameprotoype shows enough promise
| to be realized until completion. Because then most of the core
| stuff/ideas/principles won't wildly change and won't be thrown
| away.
|
| If only game development worked this way!
| blindmute wrote:
| I'm not sure I understand why they're committing to such a long
| term refactor for a game which has already reached the tail end
| of its sales curve. As far as I know there are no internal
| monetization schemes in Factorio, and I really doubt further
| updates will boost sales anywhere near enough to justify the dev
| salaries.
| trollied wrote:
| They're working on an expansion. See:
| https://factorio.com/blog/post/fff-365
| shepherdjerred wrote:
| They're planning to release a paid expansion.
| colonwqbang wrote:
| I'm also surprised. Factorio feels like a finished game. It's
| more polished than most games I've played.
|
| Maybe the devs haven't found a worthy new project yet?
| naikrovek wrote:
| They're working on an expansion for Factorio, and this
| refactor may have something to do with that.
|
| Maybe they just want to leave their code in good shape so
| they (or someone else) can come back to it at a later time
| and pick it up relatively quickly.
| kllrnohj wrote:
| Expansion packs typically depend on the base game, so
| improving the base engine is likely directly contributing
| to the work on the expansion. The expansion is likely
| changing things about the base game, and they'd want tests
| to assert both with & without the expansion are still
| working as intended.
| robryan wrote:
| As he says, they don't have external shareholders that are
| demanding the everything they do maxmise profit.
| mattmanser wrote:
| Because the worst case scenario is that they get half-way
| through the refactor, the game's a buggy mess, and then they
| all move on.
|
| And that's got a non-trivial chance of happening.
| xyzzyz wrote:
| Sounds like a good deal for customers: either the dev team
| delivers, in which case they get better product, or they
| don't, in which case they can keep using the current
| version.
| fooey wrote:
| I would assume they're working on either an expansion or
| another game using the same engine
|
| If you're building a new game and your current GUI paradigm
| sucks, overhauling it first makes a lot of sense.
| ashtonkem wrote:
| There are some mentions of a DLC in the works, probably that.
| js8 wrote:
| Yes. I don't know what the DLC will be (personally I hope for
| water- and air-borne structures, vehicles and enemies), but I
| am sure I will pay for it.
| ashtonkem wrote:
| I was planning to, but then I saw Kovarex bear his
| metaphorical ass on the subreddit, and now I'm conflicted.
| js8 wrote:
| Oh god, you should really think about what a freedom of
| speech is (suggested reading:
| https://en.wikipedia.org/wiki/Faurisson_affair). People
| should have right not to be involved in political
| discussions, and so they should have the right to tell
| people to take it elsewhere, even harshly.
|
| I am no right-winger (in fact, I am a socialist), but I
| understand why Kovarex said what he said, and I think he
| had a right to do it. Yes, civilized discourse is
| important, but it's also important to respect individual
| freedom not to be pushed into such discourse from random
| Reddit hecklers.
|
| I suppose it offends your sensibilities not because
| Kovarex did anything immoral, but because of his word
| choice. And honestly, I hate this surface level of
| analysis, where we judge the morality of a person by
| looking only at the words they use. It's
| counterproductive to addressing real problems, as there
| are far too many real villains in this world that are
| very nice in person.
| ashtonkem wrote:
| > Oh god, you should really think about what a freedom of
| speech is.
|
| You're free to read my comment history on this subject
| matter rather than just making an assumption and
| patronizing me. You'll probably find that it's more
| persuasive to actually engage with people than treat them
| like an idiot right out of the gate.
|
| > but I understand why Kovarex said what he said, and I
| think he had a right to do it
|
| Who claimed that he didn't have the right to say that?
| This is a weak straw man; nobody is claiming that he
| can't say that, merely that he _shouldn 't_.
|
| > I suppose it offends your sensibilities not because
| Kovarex did anything immoral, but because of his word
| choice.
|
| ... yeah? That's kind of how people work. How you
| communicate matters. Pretending that we should all just
| ignore him telling a fan to "shove it up your ass" is to
| basically ignore how humans actually work and think.
|
| > And honestly, I hate this surface level of analysis,
| where we judge the morality of a person by looking only
| at the words they use.
|
| As compared to what, mind reading? Literally all I have
| is his words online, and they're toxic and combative.
|
| I also really dislike that you're upgrading this to
| "morality" when my original point was about whether I
| wanted to do business with him. I feel like this is a
| neat way to side step my freedom to not do business with
| him.
|
| > It's counterproductive to addressing real problems.
|
| So what, I can't make my own decisions based on someone
| else's behavior because there are other issues? I'm
| obligated to give him money after he behaved like an
| asshole because reasons? How absurd.
| wetmore wrote:
| As an outside observer I find it weird that you two are
| arguing about an event without any reference to what that
| event was. What did Kovarex say?
| ashtonkem wrote:
| A lot, more than I can summarize here. Here's the link to
| where the drama begins, feel free to peruse.
|
| https://www.reddit.com/r/factorio/comments/o2ly6f/friday_
| fac...
| js8 wrote:
| > nobody is claiming that he can't say that, merely that
| he shouldn't
|
| No, there is no practical difference between the "can't"
| and "shouldn't", if your intent is to participate in the
| punishment for what someone does. (Maybe you feel, as an
| individual, you cannot influence it. But in fact you can
| easily become part of the mob that does have the real
| power of punishing somebody.)
|
| > Pretending that we should all just ignore him telling a
| fan to "shove it up your ass"
|
| Yes, of course we should ignore it, it was just a
| heckler. Even a heckler can be a fan. Everybody can be an
| a* sometimes, and deserves to be called out about it.
|
| > As compared to what, mind reading?
|
| Compared to his actions, of course. If you don't have
| additional information, only words, think twice about
| acting upon such information.
|
| > you're upgrading this to "morality" when my original
| point was about whether I wanted to do business with him
|
| It's of course up to you. But why wouldn't you want to do
| business with him anymore for other reason than to uphold
| a moral standard? It isn't like he cheated you, in fact
| we probably agree that Factorio is a great product. The
| morality comes into it from reasoning about your
| response.
|
| In any case, I was speaking in a larger context. People
| do this all the time. For example, there are people who
| think Linus Torvalds is worse than Bill Gates because of
| the harsh words he sometimes uses (in public).
|
| To me, that is a very superficial way of looking at
| things. And same here, I think Kovarex is one of very few
| capitalists who actually deserve all the money they got.
|
| And that's so sad about all this "cancel culture" thing.
| It's boycotting people who are not necessarily the worst,
| just easy targets.
|
| > So what, I can't make my own decisions based on someone
| else's behavior because there are other issues?
|
| You can do whatever you want, all I am saying is you
| "shouldn't".
| ashtonkem wrote:
| > No, there is no practical difference between the
| "can't" and "shouldn't", if your intent is to participate
| in the punishment for what someone does. (Maybe you feel,
| as an individual, you cannot influence it. But in fact
| you can easily become part of the mob that does have the
| real power of punishing somebody.)
|
| Oh the irony of attacking free speech in order to protect
| it. How, prey tell, do you intend to differentiate
| between those who are using their speech legitimately and
| who is part of "the mob"?
| js8 wrote:
| > How, prey tell, do you intend to differentiate
|
| I don't think I need to do it (why would I have to?), but
| it seems pretty straightforward. The online lynching mobs
| cross the line when they cause real world consequences
| for the lynched. (However, it is hard to predict in
| advance that the mob will happen and what the
| consequences will be. That's why people should be
| generally cautious about such participations.)
|
| In other words, the punishment should be proportional to
| the crime. Kovarex got his comment deleted and that
| should close the issue and end the drama.
| ashtonkem wrote:
| Yeah, that's what I expected, a proposal to silence the
| speech of some people to protect others. Maybe you
| haven't thought enough about free speech, not me?
|
| I think Ken White covered this dichotomy best, in this
| case a few years ago over a guy name Pax who got fired
| for his intemperate speech. See below.
|
| > The foundation of "witch hunt" rhetoric is the notion
| that some free speech (say, Pax's) is acceptable, and
| other free speech (say, the speech of people criticizing
| and ridiculing Pax and his employer) is not. You can try
| to find a coherent or principled way to reconcile that,
| but you will fail.
|
| https://www.popehat.com/2013/09/10/speech-and-
| consequences/
| blindmute wrote:
| I don't think anyone in these kinds of discussions is
| trying to make any provision for limiting speech, or your
| ability to not buy a product out of protest. People are
| free to do whatever they want. There is no objective
| criteria that makes your actions "wrong" or his "wrong".
|
| These arguments really just boil down to people
| disagreeing about each other's reactions to things. What
| somehow gets obscured by oblique discussions of ethics
| and politics every time I see this kind of exchange
| online, is that it's simply two sides disagreeing about
| what a reasonable response is. There's nothing really to
| discuss about that. I (and the other guy, probably) think
| your decision to not buy the game because of the
| developer's comment is a dumb decision. You think
| otherwise. There's nothing to argue about here.
| blindmute wrote:
| It's my opinion that your reaction to his comment is, for
| lack of a better word, cringe. Personally, I am more
| likely to purchase the DLC now after seeing that the
| developer has a backbone. I don't understand the reaction
| you're having. He told someone completely unrelated to
| you to "Shove it up your ass," (justifiably, imo) on a
| reddit comment. It is very strange to me that this
| affects your decision to play Factorio.
| EndXA wrote:
| Worth pointing out something that they said in the post
| (emphasis in italics is mine, the full quote is given for
| context)
|
| > Imagine you have a company that goes slower and slower every
| quarter, and then you confront the shareholders with the
| statement, that the way to solve it, is to do absolutely no new
| features for a quarter or two, refactor the code, learn new
| methodologies etc. I doubt that the shareholders would allow
| that. Luckily, we don't have any shareholders, and _we
| understand the vital importance of this investment in the long
| run. Not only in the project, but also in our skill and
| knowledge, so we do better next time._
|
| This isn't necessarily the full explanation, but it's certainly
| something to keep in mind.
| ranger207 wrote:
| Maybe just for fun? This kind of thing is half the game itself
| after all.
| kevmo314 wrote:
| > Which is a big improvement already, as adding and maintaining
| the new logic only requires you to look at one place instead of
| several, and it makes it generally more readable and less prone
| to errors.
|
| It's interesting to think about the other HN thread discussion
| about comments vs one-time-call function abstractions in this
| light: https://news.ycombinator.com/item?id=27546135
|
| I'm a big fan of "put code in one place" too. It was the biggest
| factor that convinced me that JSX was a great idea compared to
| separating the templating logic out.
| adflux wrote:
| Agreed, which is why I love vue components so much
| achairapart wrote:
| Warning: This page almost crashed my browser (FireFox on MacOS)
| and put my CPU on fire.
| DizzyDoo wrote:
| My poor 2015 MacBook Air with FireFox went full 100% CPU on
| this page, I think it's the gifs.
|
| I think Factorio itself actually runs better on this laptop
| than that Factorio blog post does.
| kart23 wrote:
| I have a dual-core 2015 Macbook Pro running FF too. htop
| showed my cpu pinned at 100%, VTDecoderXPCService was taking
| the lions share, there probably is something weird about the
| gifs.
| kllrnohj wrote:
| the "gifs" all appear to be mp4 <video> elements, not
| actual gifs. Possibly FF is doing something weird here
| (such as playing videos that aren't visible), possibly it
| comes down to whether or not the hardware decoding can
| handle it or if it's falling back to CPU decoding.
| diimdeep wrote:
| Same, this page hangs entire Firefox on macOS
| Diggsey wrote:
| Strange. I'm using firefox on windows and didn't notice any
| problems.
| Aachen wrote:
| Firefox @ Linux, also no problems (and a crappy cpu at that).
| I've noticed some of their very-gif-heavy posts slowing down
| this laptop before, but not this post.
| Metacelsus wrote:
| I'm also using Firefox on Mac and had no issues. I'm blocking
| their Javascript though.
| kllrnohj wrote:
| There doesn't seem to be any meaningful JS on the page. A
| single google analytics script and a _tiny_ [0] little toy
| script for doing a silly animation when you click on the
| #rocket element.
|
| Possibly the google analytics is doing something heavy
| (although it doesn't look to be when spot checking with a
| profiler), but there's otherwise nothing JS that runs
| continuously.
|
| 0: https://factorio.com/static/js/factorio.js
| Smaug123 wrote:
| Likewise (Firefox 89.0.1 on macOS 10.14.6) I had to close the
| page once I'd scrolled down to the layout of the various
| building interfaces; ended up just reading the HTML.
| bluGill wrote:
| > Imagine you have a company that goes slower and slower every
| quarter, and then you confront the shareholders with the
| statement, that the way to solve it, is to do absolutely no new
| features for a quarter or two, refactor the code, learn new
| methodologies etc. I doubt that the shareholders would allow
| that.
|
| It is called restructuring and big companies do it all the time.
| Investors allow it, though they are rightly suspicious -
| sometimes it is good, but often it is change for the sake of
| change and not change for better.
| Aardwolf wrote:
| I used to follow friday facts until it stopped being weekly. I'm
| glad that every friday fact now gets posted on hacker news, that
| serves as my notification for new ones :)
|
| I also misread TDD as TTD (related to the trains in factorio)
| first
| depaya wrote:
| The trains are my favorite part of Factorio. I would love "TTD
| but in Factorio"
| ineedasername wrote:
| Every time a Factorio thread makes it to HN I feel like a fully
| recovered meth addict who suddenly has their old dealer knocking
| at their door:
|
| --
|
| Dealer: _" Hey? You there? I got some meth for you."_
|
| Me: "Go away! I don't want any!"
|
| Dealer: _" Oh now don't say that. You remember how good it is? I
| know you do."_
|
| Me: "I can't, I can't afford it, the price is too high."
|
| Dealer: _" What? Come on, it's free! You already paid for it!._
|
| Me: "I'll lose my job, I can't, just go away!"
|
| Dealer: _" Your JOB? This IS your job. Open the damn door, THE
| FACTORY MUST GROW"_
|
| Me: ::cowers in the closet chanting please leave please leave
| please leave::
|
| --
|
| Also it was never _good_. It was more like a mind virus. Like the
| sort of problem or project at work you can 't stop thinking about
| until it's done. Only with Factorio, it's never done. Never.
|
| My best defense against it are other video games I can stop &
| start when needed. Or booting up my VPN connection and picking a
| work task from my back log until the cravings go away.
| LegitGandalf wrote:
| I always say, never start with a zero vacation balance!
| ashtonkem wrote:
| Honestly, I think the factorio team probably now knows more than
| Uncle Bob does, based on their blog posts.
| dgb23 wrote:
| Two interesting takeaways:
|
| > This is the beautiful thing about having a company that isn't
| on the stock market. Imagine you have a company that goes slower
| and slower every quarter, and then you confront the shareholders
| with the statement, that the way to solve it, is to do absolutely
| no new features for a quarter or two, refactor the code, learn
| new methodologies etc. I doubt that the shareholders would allow
| that. Luckily, we don't have any shareholders, and we understand
| the vital importance of this investment in the long run. Not only
| in the project, but also in our skill and knowledge, so we do
| better next time.
|
| This is reassuring the notion of what I think actually matters,
| what the real essence is of developing a product, may that be a
| piece of art and entertainment (like here) or a productivity tool
| etc.
|
| There are creators and there are consumers. We split them up by
| developers, designers, domain experts and so on, but what matters
| is that all the other participants, especially those who can
| exert power traditionally are not part of the essence and if not
| being careful and responsible, can easily add complexity and
| limitations that are entirely accidental and can even be harmful.
|
| This reminds me of the agile manifesto, modern UX approaches and
| other processes that are driven by creators, but are often and
| very unfortunately being bent over backwards to fit into
| hierarchical power structures.
|
| > TDD actually is the constant fast swithing between extending
| the tests and making them pass continously. So as you write
| tests, you write code to satisfy them basically at the same time.
| This allows you to instantly test what you write, and mainly use
| tests as specifiation of what the code should acctually do, which
| guides the thought process to make you think about where you are
| headed to, and to write code that is more structured and testable
| from the very beginning.
|
| The important notion here is that TDD is not about tests and
| correctness, but about development. It continuously checks
| assumptions and explores the surrounding code, state and data
| until a sufficient solution is found.
|
| If we squint a little we can see how closely related TDD with
| REPL Driven Development is. In essence it is the same thing and
| even has similar results, where the tests or REPL code can be
| left as an artifact for further, likely historical understanding.
|
| We know now that neither is sufficient for a high degree of
| correctness, but they are certainly useful for understanding and
| development.
| CraigJPerry wrote:
| For those curious about REPL driven development, this is a good
| example that i ran across recently:
|
| https://gist.github.com/daveray/1441520
|
| Even if you don't follow along and try it, you can probably get
| the gist of how experimental / exploratory you can be.
| adamkl wrote:
| For those who want to see some REPL driven development in
| action:
|
| https://vimeo.com/230220635
| bjornjajayaja wrote:
| I'd love to see a language where you write the tests and then
| the compiler creates the application code.
| dgb23 wrote:
| We typically write _sample_ tests in boolean logic, which
| isn't quite expressive enough for this.
|
| But if you look at logic programming with more expressive
| systems you can have something like what you propose. We
| describe what we expect to have and the system deduces a
| result. Not quite what you want but its closer.
|
| Now there is also an ubiquitous logic system that many use:
| static typing. In a sense you are describing the general
| properties of something and the compiler infers optimizations
| based on your assertions. The concrete program is not a line
| by line translation from your code to machine code, but
| perhaps looked at in its entirety.
|
| I agree that there is a lot of merit in pushing these things
| further and further. Right now we're kind of in a stage of
| patching things together. But I hope and assume that
| programming becomes more holistic in the future. Ironically
| we have to look at the past first, there was a lot of
| momentum in this direction up until the 80's roughly.
| astrange wrote:
| Well, that's how machine learning works.
| mikewarot wrote:
| Machine learning requires a problem that you can have
| partially correct, so that it can climb the gradient to
| optimize on. If you can build tests that have an analog
| instead of pass/fail output, you could, in theory, do it
| with machine learning.
|
| Beware that machine learning in a single pass/fail is more
| like having an infinite number of monkeys trying to write
| the works of Isaac Asimov.
|
| [Edit/Update] All of the tests could be individual values,
| so non-zero (but nowhere near all ones) might help. Thanks
| for making me reconsider this, sdenton4.
| sdenton4 wrote:
| Not all ml is gradient based. Other options exist:
| Bayesian black box optimization (like vizier), or genetic
| algos. And Vizier is actually quite efficient for small
| problems.
| astrange wrote:
| I was actually thinking about decision trees.
| peheje wrote:
| With genetic algorithms you still need to be able to
| calculate a fitness. Usually a test is fail/success.
| There's no fitness in that. I would guess the other
| optimizers also need such a signal?
| sdenton4 wrote:
| Success/failure is just a binary classification signal:
| you can look at how it correlates with your target
| variables, how noisy it is for particular choices of
| variables over multiple trials, etc. The noisier it is,
| the harder it is to learn, but such is life.
| peheje wrote:
| I think it would be impractical for a naive fitness
| function that is 0 for failure and 1 for success.
| Wouldn't the signal be too difficult to find? GA would be
| brute force until you find code that passed a test. I
| don't think tests for factorio are trivial.
|
| Maybe you could move the goal/fitness function along the
| way. So start with something that compiles. Then having
| the desired input output. Etc.
| peheje wrote:
| Addendum. More tests. I see. Would you then lead the
| algorithm along a trajectory. If you can pass this simple
| test you would probably be able to pass this as well then
| this... Babysitting it along the way. Ideally you
| wouldn't need to, but maybe to make it possible..
| kilburn wrote:
| This is an active field of research. Search for "program
| synthesis".
|
| We are advancing, but the current state is... not mind-
| blowing yet (albeit somewhat cool!). See [1] for an example
| interactive demo and [2] for the corresponding presentation.
|
| [1] http://comcom.csail.mit.edu/comcom/#Synquid
|
| [2] https://www.youtube.com/watch?v=HnOix9TFy1A
| hwayne wrote:
| I was in the audience for that talk. The recording doesn't
| capture how much energy there was- we were all gasping and
| cheering throughout.
|
| Still a decade off from production, though. That it doesn't
| just take "test cases": you have to know how to formally
| express the program properties, which is a separate skill
| from both unit testing and implementing.
| sidlls wrote:
| How would this even work? At best you might get a set of
| class/function stubs with some minimal logic. At worst what
| you'd have is a compiler that is actually a complicated,
| truly AGI brain, which could produce some truly awful code.
| In which case you've simply reproduced TDD's normal result:
| truly awful code.
|
| Aside from that, to produce a program that did what is
| actually required would require test cases and functions that
| cover the set of inputs and outputs. This is trivial for
| mathematical functions, but impractical (or impossible) for
| more general applications (e.g. anything dealing with human
| inputs).
| nyberg wrote:
| This sounds a lot lke like minikanren
| (https://github.com/webyrd/Barliman) where you give test
| cases and Idris2 (https://github.com/idris-lang/Idris2) where
| you give type constraints as a tool for building programs.
| phtrivier wrote:
| I would like to see this language handle some lawmaker-
| specified code.
|
| Show examples of French retirement pension computation, and
| watch if a computer can actually commit petit-suicide.
| bregma wrote:
| It's incredibly easy. First, you start with a customer that
| actually knows exactly what they want.
| iamwil wrote:
| This is kinda what logic programming is, like in prolog. You
| tell the computer what you want, and it finds the answer for
| you.
|
| TDD is where you both write what you want, and you do the
| implementation also.
| mikewarot wrote:
| You could just use a fuzzer to generate code, and run tests
| on the output. Each new test would approximately double the
| run time until a new "correct" output was found.
|
| This doesn't become practical until you can do it on a
| quantum computer with millions of cubits.
| oats wrote:
| Is this not almost how logic programming (prolog, etc.)
| works? You tell the language some things which are true, and
| then it'll be able to infer answers to "questions" you ask:
|
| https://wiki.c2.com/?LogicProgramming
| JohnHaugeland wrote:
| No. Not even a little bit.
| oats wrote:
| Could... could you enlighten me please?
| iamwil wrote:
| Yeah. I've often seen TDD where you act both as the logic
| programmer writing the tests and the imperative programmer
| writing the implementation.
| victorNicollet wrote:
| Let's say we work on the "reverse list" function. A few
| tests I could write are that the following clauses are
| true: rev([1], [1]) rev([1, 2,
| 3], [3, 2, 1])
|
| And maybe also that the following clauses are false:
| rev([1, 2], [1, 2]) rev([], [1]) rev([1,
| 1], [1])
|
| Is Prolog able to infer, from the above, that rev([4, 5],
| [5, 4]) ? Or to synthesize the general form rev([H|T],R) :-
| rev(T, RT), concat(RT,[H],R) ?
| ffhhj wrote:
| Sounds like NN training that could be achievable. The
| problem comes with unit testing, because the minimal
| testable unit could be more complex than a function.
| lugged wrote:
| Its almost like Test Driven Development is the practice of
| using tests to drive development.
| dgb23 wrote:
| Much of TDD mantra is less about development but about
| (perceived) correctness, the article in question emphasizes
| the development part, where they describe their AHA moment.
| ffhhj wrote:
| > (perceived) correctness
|
| Right, in TDD there is "always" a range of values against
| which you are not testing. A developer can only create a
| finite amount of tests, and the program will fail with any
| of the missing permutations.
| lugged wrote:
| Also need to echo the above.
|
| The TDD book itself is barely even about testing, it's
| almost entirely about development processes and different
| types of refactoring and how to use tests to get from a to
| b.
|
| Correctness is rarely emphasized if at all.
| berkes wrote:
| > Much of TDD mantra is less about development but about
| (perceived) correctness,
|
| I have (seriously!) never heard it reference like this at
| all. Every introductory article, every book, every
| tutorial, every video about TDD (or BDD) that I consumed,
| emphasised that it is about "workflow", about "a way of
| development" and not about percieved correctness.
|
| Do note that the crucial different is that TDD > automated
| testing. In the sense that with TDD you write automated
| tests (that live in a test suite). But that "writing tests"
| or "developing a testsuite" is not nessecarily TDD.
| hwayne wrote:
| The original TDD book said it was both.
| dgb23 wrote:
| Yes in relation to automated testing it is obviously
| focusing on the development aspect, but as a whole method
| there is an at least implied and sometimes explicitly
| emphasized correctness benefit, that doesn't just come
| from automated tests but from the method of driving
| design through testing.
|
| In my personal opinion this claim goes too far. I think
| we can often explore aspects of design through code
| (including tests) but it can't be the main driver of
| design in any circumstance, nor does it by default or in
| general make our modules and programs more correct.
|
| The key benefit of doing this is that we have to think of
| code in terms of invariants and testability. But that is
| just one of many, sometimes conflicting factors and there
| are certainly more approaches to doing this than writing
| automated (unit-) tests.
| wpietri wrote:
| Yes, you're exactly right about REPL-driven development and TDD
| being the same spirit.
|
| I took to TDD pretty easily because I was already used to doing
| short run-it-and-see-if-it-works cycles. The main difference
| was that instead of checking via eyeball, I talk the computer
| to check it. This is slightly slower early on, but so much
| faster once a program is big enough that manually checking
| everything would take a while.
| truetraveller wrote:
| TDD is basically automated and "named" REPL-driven development.
| Which is very nice!
| infogulch wrote:
| Oh that's an interesting take, and feels right to me. So the
| thing that holds back TDD is ergonomics (compile time/cached
| results) and maybe marketing.
| indigochill wrote:
| Regarding the power structure observation, I've lately been
| weighing the merits of co-ops in the tech industry.
| Intuitively, it seems like a tech product owned by its users
| would lead to more product-centric decision-making. Leaving
| governance to shareholders who aren't necessarily directly
| involved in the product feels weird by comparison.
| dgb23 wrote:
| I think so too. A lot of that free energy is in the FOSS
| movement it seems, and there are plenty freelancers and
| single consultants in tech. This is a strong indicator that
| there is enough developers who primarily want to create,
| while having agency, responsibility and a direct
| communication channel. So why not start businesses that share
| these values and bring consumers and producers closer
| together?
| wpietri wrote:
| I appreciate this, but I still think it's a suboptimal
| approach:
|
| > that the way to solve it, is to do absolutely no new features
| for a quarter or two, refactor the code, learn new
| methodologies
|
| Even in a world without shareholders, we still are building
| things for users. 6 months without improvements has an effect
| on them, too. When possible, I think it's better to spread
| cleanup work out. Even if one spends 80% of the time on cleanup
| and 20% on features, that is much better for relationships than
| going dark for a quarter or two. And in my experience,
| continuing to do productive work during that period makes the
| behind-the-scenes improvements better.
| mikewarot wrote:
| I've both watched all of Uncle Bob's videos (parts more than
| once), and been a long time Reddit/Factorio reader/game
| player. (The factory must grow!)
|
| They did exactly what you suggested! The relationship with
| the users never went dark. They kept up with bug fixes, and
| kept feeding new features at a more than acceptable pace.
| [Edit: Here's the pace of their status updates, as evidence
| at
| https://www.reddit.com/r/factorio/?f=flair_name%3A%22FFF%22 ]
|
| I didn't quite believe that Uncle Bob's lessons worked in the
| real world, but if the Wube team is sold on them, that's
| about as good as an endorsement as I'll ever get.
| wpietri wrote:
| Yay! Glad to hear it.
|
| I keep thinking I should try Factorio. But given amount of
| time friends have spent on it, I'm scared I'll never be
| heard from again. One of my friends has built a whole
| Grafana setup for it.
| mikewarot wrote:
| Factorio is easy to dump a ton of time into, if you are
| the type that likes solving puzzles and optimizing
| systems. You are wise in your caution.
| elevatortrim wrote:
| True, and it is probably worth it. In my late 30s, I
| thought I'd never enjoy a game like I used to in my teen
| years. Well, Factorio was easily as much as if not more
| enjoyable. I do not know if I'll enjoy a game this much
| ever again.
| oarsinsync wrote:
| If you spend your working hours solving problems and
| architecting systems, this game will be fun until you
| cross whatever your personal limits are for immediately
| grasping concepts.
|
| As long as you're immediately understanding, you'll
| continue to have fun.
|
| As soon as you need to stop and research to understand
| what it is that you need to do next, and then start to
| re-architect a bunch of things, you may or may not
| realise this is the same thing you do for a living.
|
| Nothing quite like burning the candle at both ends, for
| fun and profit.
|
| Personally, it's not for me. Others (including some of my
| colleagues) definitely do not share that sentiment.
|
| That said, I enjoyed Mindustry for a similar period too.
| Factorio-lite tower-defence game.
| pault wrote:
| You should take a look at satisfactory[0] if you haven't
| already. It's not (quite) as deep as factorio, but the
| game world is absolutely beautiful and immersive, and
| it's still in early access so new content is constantly
| arriving.I personally had to ban it because I played it
| for 300 hours in less than three weeks. I literally could
| not get myself to stop playing (until I did). :)
|
| [0]
| https://store.steampowered.com/app/526870/Satisfactory/
| bregma wrote:
| Try heroin first. It's easier to quit.
| CreepGin wrote:
| One would think this is a joke. But it's not. =D
| scotty79 wrote:
| In this specific case, Factorio is such beloved product with
| such great development history that 6 months without updates
| is nothing.
| elevatortrim wrote:
| It is not entirely without updates, the team have been
| pushing bug fixes for the most obscure edge cases that
| almost noone is running into.
| rpastuszak wrote:
| > The important notion here is that TDD is not about tests and
| correctness, but about development.
|
| Yup, writing tests helps me sleep at night. TDD helps me manage
| my mental resources and iterate.
|
| (another reason is communication--we code for our colleagues
| first, then for the machine: https://sonnet.io/posts/code-
| sober-debug-drunk/)
|
| IIRC smalltalk had a workflow where you'd debug and write your
| program at the same time. You'd just reach a path that has not
| been implemented yet, break, implement it and continue.
| dgb23 wrote:
| This ,,keeps working while it breaks", does it have a name?
| It comes up in highly dynamic environments often like
| Smalltalk as you mentioned, but also Lisp, Erlang and others.
| dkarp wrote:
| I think this may be "REPL driven development" popular in
| the lisp community
| dgb23 wrote:
| I meant more at the system/vm level. You're going to use
| a REPL (of some sort) for this, but the system needs to
| accommodate failure and just keep going, while giving you
| the opportunity to fix and change things.
|
| This has been implemented to varying degrees of
| reliability, granularity (in terms of preserving state)
| and utility. It's a common concept but I don't know the
| name for it!
|
| Examples:
|
| https://en.wikipedia.org/wiki/Continuation
|
| https://en.wikipedia.org/wiki/Fault_tolerance
|
| Maybe fault tolerance is the right word? But it doesn't
| imply that you actually modify and build the thing while
| it already runs.
| astrange wrote:
| It's not a great idea because failure could mean
| anything, and so you can't necessarily fix it. It's
| better to start over from the beginning. This is what
| reliable systems like Erlang do.
|
| https://en.wikipedia.org/wiki/Crash-only_software
| FigmentEngine wrote:
| this could be many different concepts. fault tolerance is
| the most obvious, and self-healing and auto-rollback
|
| a good read is ROC
| https://en.m.wikipedia.org/wiki/Recovery-oriented_computing
| tobyhinloopen wrote:
| I wonder how common TDD is in game development land, especially
| when you're using things like Unity or Unreal.
|
| I feel like testing your behaviors is pretty hard, and even if
| you unit-test your behaviors, there's still integration tests.
|
| I only write games as a hobby and never use TDD, even if I'd like
| to, since the tooling is just either poorly documented or too
| slow, or both.
|
| Usually this ends with me being frustrated with the slow
| development cycle and pushes me towards more unconventional
| methods of developing games in Javascript using Mocha to run the
| tests directly in the browser.
| mewse wrote:
| In twenty years in game development I have never worked on a
| game which had real unit tests or even integration tests.
|
| I've seen engines which used them, but not games. The rationale
| was that it was just too hard, which always felt like a cop-out
| to me.
|
| I would dearly love to have more automated tests in the game
| I'm working on now, but I've never seen a model of it working
| well which I could copy, and part of me suspects that it'd be a
| huge investment of time to figure it out entirely on my own
| when I'm already vastly overworked as it is.
|
| If anybody has references to indepth case studies of making
| game engines more friendly toward automated tests, I'd be
| _super_ interested to see whether there were lessons I could
| apply toward my own situation!
| tarcon wrote:
| Interesting. I always assumed that to get the balancing feel
| right, a game would have to run huge parameterized test-suits
| to make sure win/lose results to user inputs are in balance.
| beckingz wrote:
| Good luck doing this for a huge modern AAA game.
| kempbellt wrote:
| This is likely why "Game Tester" is an actual job, or at
| least used to be. 14 year old me didn't believe it was a
| real thing...
|
| "Early Access" seems to be the more popular route now.
| People are happy to _pay_ to test a game and feel like
| they participated in its development. Check out Star
| Citizen if you want to be amazed and depressed by this
| truth at the same time.
|
| Sometimes it works well. Sometimes it doesn't. Seems
| largely dependent on how engaged the devs are with their
| community.
| mywittyname wrote:
| I've always assumed they did this for games like LoL.
|
| Bots already exist, so the foundation for automated play
| testing is in place. Take the basic AI and add some
| functionality to track the effectiveness of various
| skills or loadout across plays. Using A/B/n testing to
| choose the most effective character strategy would
| probably highlight overpowered loadouts within a few
| thousand game-test-hours.
|
| They could probably take analytics from real players and
| do what's outlined above and get a reasonable idea of the
| impact a change will have.
| anchpop wrote:
| The issue with that is that LOL bots are laughably bad,
| and making them not bad requires a substantial
| investment. But something like what you describe is
| possible. OpenAI has a paper where they evaluate chess
| variants by training AlphaZero on them and seeing which
| ones generate more balanced games between white vs black,
| and which ones lead to more dynamic games with fewer
| stalemates etc.
| 8note wrote:
| Stalemate a are great fun; it's "the players agreed to a
| draw" and "draw by repetition" that are boring
| wbc wrote:
| I dont think they used bots to test, just rpc endpoints,
| at least as of 2016 blog:
|
| https://technology.riotgames.com/news/automated-testing-
| leag...
| teamonkey wrote:
| It's not exactly what you mean, but automated _gameplay_
| testing is fairly common in the AAA space, although maybe not
| taken seriously enough.
|
| A simple example might be simply starting and closing the
| engine after the automated build & package process to make
| sure the game actually runs, but I've seen things like using
| bots to emulate player behaviour to smoke test gameplay
| functionality. No Man's Sky used automated tools to evaluate
| the procedural-generation algorithms[1]. Here's a more
| comprehensive example of automated gameplay testing[2].
|
| [1] https://youtu.be/sCRzxEEcO2Y?t=3100 [2]
| https://www.youtube.com/watch?v=VVq_hgaX8MQ
| dsego wrote:
| Not TDD but here is a talk about automated testing at Croteam
|
| Continuous integration and testing pipelines in games - case
| studies of The Talos Principle and Serious Sam
| https://www.youtube.com/watch?v=YGIvWT-NBHk
| jayd16 wrote:
| So there's a few reasons tests aren't as ubiquitous in games as
| they are in non-game dev.
|
| You need a large QA team (relative to your team size) to test
| for fun anyway. The game is constantly getting tested and bugs
| will get logged. The marginal benefit of automated tests is
| less than other places because of this.
|
| Games have no specification. You have almost no idea of even
| the genre of game you'll end up with at the end of the dev
| cycle unless you're making a sequel that has to fit into a
| mold. Sure you can write tests as you go along and test that
| enemies with negative health die. The next day someone will
| suggest "what if they stay alive for a period of time and then
| explode!" The definition of correct is constantly changing.
|
| Tests ossify functionality. It makes it harder to change things
| because at the very least you need to also change the test. If
| you're just changing tests whenever you want to suit your new
| desires then its hard to build trust in the tests.
|
| Games don't need to be correct. They just need to be fun. This
| also decreases the marginal benefit of tests compared to other
| industries.
|
| That said, it would be natural to unit test some data structure
| or some well defined system. Also, once your game is done, a la
| Factorio, you can go back and write tests for some refactor
| because you know the full design specs.
| exdsq wrote:
| Anecdotal but I think it's pretty rare - my friend worked as a
| game developer for Epic and didn't know what TDD (or SQL for
| that matter) actually meant.
| tobyhinloopen wrote:
| Given how common it is for bugs to reappear in Fortnite, I'm
| pretty confident their testing suite is either incomplete or
| not present at all
| mschuster91 wrote:
| Testing? In games? There is no such thing on a wide scale, not
| anymore since the cost of distributing patches essentially
| became a small budget line for a CDN.
|
| Modern games are notorious for using the first, most loyal
| customers as beta testers (hello Fallout 76...).
|
| The reason is two-fold... while you absolutely can test _some_
| parts of the engine (e.g. collision detection, networking) you
| can 't really "test" stuff that needs a human eye to see if
| it's working as intended (anything that's rendered) or involves
| randomness (e.g. fire, fog, water, opponent spawning, loot).
| That means you have to hire lots of skilled (!) humans, provide
| them with expensive rigs, and give them time. Which is
| incredibly expensive.
| gmueckl wrote:
| Just to give you some perspective: my current employer's main
| product isn't called a game, but it has an engine at its core
| that is a game engine in all aspect except its name. And we
| test the sh*t out of it. We have thousands of automated and
| very sensitive tests on that stuff. Our test suite goes as
| far as testing for pixel perfect output. And this involves
| stuff that is "random". It took us some effort to be random
| in a perfectly reproduceable way, but we got there.
|
| Game QA is more involved than that, of course. Content needs
| to go through a signoff and QA process that involves humans
| (we do that, too).
| kempbellt wrote:
| This is definitely true for some games, and Early Access is
| increasingly popular, but for others, QA and testing is
| definitely a part of the development process.
|
| I interviewed at a game company a few years ago where one of
| my daily tasks would be to spend an hour just playing the
| game and seeing if I spot any bugs. I didn't end up taking
| the job so I didn't see how involved their actual code
| testing process was, but it was apparent that they actually
| cared a bit about quality control.
| ashtonkem wrote:
| I feel like game dev is probably an area where TDD has some
| value.
|
| My team writes distributed systems, which drastically reduces
| the value of a TDD approach. There's only so far you can take
| the technique with a database backed api before it just becomes
| absurd.
| [deleted]
| Thaxll wrote:
| Video game client don't have tests.
| Danieru wrote:
| Factorio is perhaps the only game I am aware of using TDD. Lots
| of engine teams use extensive automated testing. Only Factorio
| is applying it to game logic of any major game I know.
| mywittyname wrote:
| Factorio seems especially well suited to TDD. The core
| gameplay loop involves automating away manual tasks. So I
| have to imagine that the test cases leverage blueprints a
| lot, i.e., create a blueprint for a feature, pipe resources
| into it via conveyor belt, test output rates on conveyor
| belts.
| Glowbox wrote:
| https://twitter.com/playartifact/status/1051964775658217473?.
| ..
|
| Artifact did it too (which is no longer being worked on
| though).
| CodeGlitch wrote:
| I was in the industry from early 2000 to early 2010. It was
| only towards the end that Unit Testing was a thing. At the
| start we didn't even do code reviews or use a sensible source-
| control system.
|
| Yeah it was a painful experience, but I survived.
| harryf wrote:
| Came here hoping they'd turned Factorio into a tool of creating
| tests in other codebaes. Like literal gamification of work.
| dgb23 wrote:
| You might have looked at Flow Based Programming?
|
| It has certain characteristics that align with a game like
| Factorio or Oxygen Not Included etc. such as visual
| programming, backpressure, common interfaces, local retention
| etc.
|
| I can imagine this being applied to distributed/cloud computing
| as a way to reason about high level interactions and perhaps
| functional/integrated testing.
| ashtonkem wrote:
| I've done a bit in my home automation; it's no replacement
| for a scripting language.
| Keeper93 wrote:
| Kinda sad that they give this guy a platform.
| danielatc wrote:
| I've noticed that as well and got a reply from the creator on
| reddit.
|
| And it's been a rather disappointing one... "Take the cancel
| culture mentaility and shove it up your ass."
|
| https://www.reddit.com/r/factorio/comments/o2ly6f/friday_fac...
| thrower123 wrote:
| It's generally best to not give sealions the time of day
| oauea wrote:
| He's completely right. The behavior of this Uncle Bob person
| on social media (or anywhere outside of the linked videos)
| has absolutely nothing to do with the linked videos. Please
| stop derailing these discussions by trying to insert drama
| where there is none.
| thrwaeasddsaf wrote:
| Tbh I think a rule like "be nice" should also imply "don't
| spread dirt on people."
| Sr_developer wrote:
| Well, tbh you kinda of deserved it. Although uncle Bob's
| methodology can be highly suspect and not worth 1% of its
| hype the article you linked
| (https://techexplained.substack.com/p/tech-bullshit-
| explained...) was not a technical criticism , it was
| something which at best seems written by a tumblerina.
| ashtonkem wrote:
| They "kind of deserved" being told to take their cancel
| culture and "shove it up their ass"? Yikes.
|
| You can disagree about someone's comment without responding
| in such a crude, childish, and reputation damaging way.
| Even if you thought the original comment was bad, Kovarex
| is _way_ out of line here, and doing a lot of damage to his
| reputation with his fans.
| oauea wrote:
| And now the fine folks at reddit are censoring the creator
| of Factorio on the game's own subreddit. This is absolutely
| ridiculous. Cancel culture is real, and Kovarex is right to
| worry about it.
| istjohn wrote:
| No one is allowed to use language like "shove it up your
| ass" here on Hackernews, either. For good reason.
| Respectful discourse is most important precisely when
| debate touches on hot button issues.
| danielatc wrote:
| Fortunately a lot of people within the thread can defend my
| point better than I can.
|
| I prefer to listen rather than to speak up, but in this
| instance and context I felt it was the right thing to speak
| up for once to raise awareness in a circle that otherwise
| might be oblivious to the issue.
|
| And yes the article I linked was certainly not the best. It
| was simply one example out of many that makes a case of him
| being problematic as I couldn't find a more canonical piece
| on the spot.
| iser wrote:
| I think you injected politics into something that is not
| the place for. Yes these issues are important but please
| understand that there are right times and settings for
| these discussions. To be honest, I was appalled at your
| comment and I completely understand Kovarex's reaction.
| He clearly did not word it tactfully, but I understand
| why he was upset.
| danielatc wrote:
| As I tried to state in my initial post: giving a
| problematic person like Uncle Bob a platform is a
| political act. Irrespective of whether it was conscious
| or not. So I wanted to educate as I assumed no bad
| intentions.
|
| And to summarize the point I'm trying to make: promoting
| toxic people will drive away lots of underrepresented
| people from our industry which to me is a massive net
| loss.
| krzyk wrote:
| How is Robert C. Martin toxic? Reading the examples
| pointed above don't make him toxic (and making fun of
| "Uncle" makes the first article completely unreliable,
| not to mention the completely of the hook bit about
| police).
|
| And even if he was toxic - mixing personality with
| articles should not take place ever. If you disagree with
| the articles write so, but don't attack personally.
| carbonguy wrote:
| For what it's worth, and to try and balance the scales
| here, I'm of the opinion that:
|
| 1) you were 100% in the right to point out that Uncle Bob
| is a problematic figure
|
| 2) promoting his work while ignoring 1) is, in fact, a
| political act as you pointed out
|
| 3) you expressed these views very tactfully and clearly
| (to my eye as a native English speaker - I wouldn't have
| known it was a second language for you had you not
| mentioned it)
|
| 4) based on this thread and the post you made on Reddit,
| there was no call whatsoever for you to "shove it up your
| ass" or be called toxic or anything of the sort
|
| From where I sit, you were in the right and you remain in
| the right.
| krzyk wrote:
| > 2) promoting his work while ignoring 1) is, in fact, a
| political act as you pointed out
|
| How can it be a political act? A political act is
| bringing something completely unrelated (like Robin's
| stance on funding police) to a technical discussion about
| TDD.
|
| I to lived in oppressive (communist) regime like creator
| of Factorio, and political party (The Party) forced
| everything to be political. Going to church, not eating
| meat, not going to a rally, ot watching TV.
|
| Now we want to just discuss a topic without dragging
| politics into this.
| js8 wrote:
| I originally didn't want to get pulled into this
| discussion, but I had to respond elsewhere, ergo alea
| iacta est.
|
| > giving a problematic person like Uncle Bob a platform
| is a political act
|
| If we are talking only about platforming UB's technical
| opinions, then it's not a political act. Unless you think
| that everything done in public should be, a priori,
| considered to be a political act.
|
| I was born in the same country as Kovarex, about the same
| time ago, and at the time, we lived in a system where
| everything happening in public sphere was considered a
| political act. Even lack of participation - you didn't
| put up a flag or go to a march at certain state holidays,
| and people would say, he is suspicious, and "politically
| unreliable" (which would block things like promotions at
| your job or if your children could study).
|
| It wasn't a society you would want to live in, I
| guarantee you that. So please, think twice before you
| call something like that a political act.
| randompwd wrote:
| You were the toxic person in that thread. You are the
| toxic person in this thread.
|
| I am of the opinion that you do not have sufficient
| common sense to partake in these discussions.
|
| Nothing but bad intentions on your behalf which you were
| well aware of.
|
| Your behaviour would be categorized as trolling. Nothing
| else.
| oauea wrote:
| If you can't even argue your own point, I'm not sure you
| should be making it...
| danielatc wrote:
| English is not my native language. And these kind of
| debates demand a very careful choice of words that are
| often not easy for me to find.
|
| That's why I'm rather reserved in these types of
| conversations, not because I doubt that my point is
| valid.
| oauea wrote:
| That's pretty weak, easy to make all kinds of claims like
| that, then just be like "whelp sorry not my language!"
| when people respond to your provocation. Good way to stir
| the pot pointlessly.
|
| English is not my native language either, by the way.
| beaconstudios wrote:
| yeah I have to be honest, I read that article and the
| citations to his bad behaviour and while he does seem to
| have a habit of making inappropriate jokes (about women
| "not being allowed" in 70s programming for example) he made
| a public apology about it and clearly realises it's a bad
| habit. People make mistakes, especially if they were raised
| in a different era but we should be grateful that he
| apologised for it.
|
| I don't think "being a boomer" makes somebody a sexist and
| racist.
| carbonguy wrote:
| For anyone like me trying to find the comment that kovarex
| made, apparently it was removed by the moderator of the
| r/factorio subreddit.
|
| EDIT to add link to mod comment: https://www.reddit.com/r/fac
| torio/comments/o2ly6f/friday_fac...
| robocat wrote:
| Also note that kovarex tries to engage with the critics (or
| fans the flames) in the reddit thread, although most of his
| comments have been downvoted hard (hard for me to say
| whether he is being cancelled/brigaded - I don't do
| reddit).
| istjohn wrote:
| Wow. That's very dissapointing. Choosing to look the other
| way when someone uses sexist language is not a principled
| stand. We are all responsible for cultivating a culture in
| software development that is inclusive. That means holding
| prominent people within the field to a high standard of
| professionalism. Not because we want to be thought police,
| but because no one should feel unwelcome in this field
| because of their gender, sexual orientatien, or skin color.
| oauea wrote:
| The US government actively kills many people in other
| countries. Should we start holding every American
| personally responsible? Am I sharing a forum with a bunch
| of horrible murderers? Or can we accept that just because a
| person, organization, or government that you support does
| horrible things, that does not make you yourself a horrible
| person by default?
| birdyrooster wrote:
| 6 of one, half dozen of the other
|
| If we are all horrible by default then we have to hold
| each other accountable in relative terms. I think our
| association with murderers and shitty people makes us all
| horrible and we should accept that and try to become less
| so? It's not that bad being bad, it's just how it is.
| carbonguy wrote:
| Nuance does not seem to be your strong suit, based on
| this and other comments, but I nevertheless feel
| compelled to ask whether you see no actual difference
| between:
|
| a) pointing out that a particular developer has a habit
| of expressing reactionary views (which, to make it extra
| clear for you, is all that OP did), and
|
| b) whatever it is you think happened?
|
| And, to address your questions, because hey why not -
|
| > The US government actively kills many people in other
| countries. Should we start holding every American
| personally responsible?
|
| This depends on whether you believe America is a
| representative democracy or not. Since I do believe this
| is mostly true, then yes, the American voting-age
| population does bear some measure of the responsibility
| for the crimes committed in its name, and I personally
| feel sick at heart and actively motivated to foster
| political change for this specific reason among others.
|
| > Am I sharing a forum with a bunch of horrible
| murderers?
|
| Probably not "a bunch" as I would define it, but I
| suspect that HN is large enough that there might be a
| murderer or two here.
|
| > [C]an we accept that just because a person,
| organization, or government that you support does
| horrible things, that does not make you yourself a
| horrible person by default?
|
| No, I can't accept that. If I hear somebody praising,
| say, Ted Bundy or the CIA to the skies, my takeaway from
| that is "geez, I don't want to be associated with this
| person."
|
| But then, I guess you did throw "horrible" in there, and
| benefit of the doubt being what it is, I wouldn't
| _necessarily_ assume that this hypothetical person was
| "horrible" - I just wouldn't have a shred of interest in
| finding out more about them.
| oauea wrote:
| > Nuance does not seem to be your strong suit, based on
| this and other comments
|
| So you're starting off with personal attacks. Okay.
|
| > but I nevertheless feel compelled to ask whether you
| see no actual difference between:
|
| I am talking about everyone now condemning Kovarex for
| referencing this Uncle Bob person (who I had never heard
| of before, myself).
|
| Since now apparently Kovarex is a bad person for linking
| to some videos and talking about Uncle Bob's teachings,
| and since you said this:
|
| > No, I can't accept that.
|
| That makes you a horrible murderer. You are financing the
| murder of many people by paying your taxes. Or maybe you
| should reconsider that view?
| 1_player wrote:
| No. I will personally do my best to make everyone feel
| _welcome_ whatever their gender, sexual orientation or skin
| colour, but stop forcing people to take a political side,
| to make everything about politics, and that we always need
| to discuss the person instead of the topic at hand.
|
| No, I don't _have to_ "hold prominent people within the
| field to a high standard of professionalism." Don't force
| me to.
|
| There is a time and place to be offended. kovarex wasn't
| endorsing the Uncle Bob persona and everything he stands
| for, guilt by association is definitely not the way to
| build a welcoming environment, when it's really just thin
| facade on a "with us or against us" dictatorship.
| xdennis wrote:
| Everybody makes mistakes, but mobbing every unrelated
| thread isn't the solution to sexism. Calling him racist
| because he's not for defunding the police is even worse.
| [deleted]
| sidlls wrote:
| Yeah, "Clean Code" is pretty terrible, and a number of
| practices promoted within aren't very well substantiated in
| practice. I don't think he should be promoted in software
| circles, either.
| blindmute wrote:
| This comment is out of place. That being said, after reading
| through the reddit thread, the developer Kovarex is extremely
| based.
| truncate wrote:
| > (1) no new features for a quarter or two, refactor the code,
| learn new methodologies etc
|
| > (2) This allows you to instantly test what you write, and
| mainly use tests as specification
|
| > (3) the problem comes when you break something and a lot of
| tests start to fail suddenly
|
| My favorites. Don't expect to give away entire quarter, but at-
| least sometime would definitely be nice. All three so
| fundamental, and often ignored. In my experience, you get these
| right, it makes developer life so much easier. As someone earlier
| mentioned in thread, TDD is kind of like REPL driven development.
|
| I think, one immediate benefit of companies focusing on good code
| is that engineers can aim for much more ambitious projects, and
| they can be more brave with the codebase. Instead we often end up
| with 100 over-engineered components with no well defined/enforced
| contracts, and a set of monolithic tests which runs the entire
| stack to test the most basic case.
| fendy3002 wrote:
| This should be the way. I really like three interation
| approach: research, stable, enforce.
|
| First you develop fast and breaking things with alpha / beta
| versions. Then you make it stable with bug fixes and minor
| enhancements. Finally enforce the code with review, unit tests
| and code coverage, etc.
|
| Theoretically they already have a good game engine (long
| lasting product) and interested in developing it further.
| Without enforcement, any future changes have potential to break
| things. Unit tests (enforcement) reduce that risks and make any
| changes / refactoring closer with specification.
| sidlls wrote:
| On the other hand, with TDD we often end up with code that has
| been butchered in the name of "testability," and which is both
| less efficient and more complex than necessary.
| leprechaun1066 wrote:
| This usually happens when the developers in this situation
| are focusing (or are being forced to focus) on the tests over
| focusing on the solution to the actual problem in the
| product. TDD is just a development methodology which is a
| means to an end, not the goal.
| sidlls wrote:
| Development methodologies exist to solve product
| development problems, not the product problems. In that
| sense, an organization that adopts TDD necessarily focuses
| on the tests (as part of the development process), by
| definition.
|
| The problem is, TDD is a poor development methodology.
| truncate wrote:
| Yes, I think being little extreme either ways is bad. If
| TDD makes it hard to test certain new thing we are
| implementing, maybe do it some other way.
|
| I've found functional style of programming, or just
| decomposing functions into smaller functions, often work
| better with TDD. I'm personally not as much into pure TDD,
| as much as writing code that can be tested quickly and
| easily. In the end, what I want is when I'm writing code, I
| should be able to quickly run that specific piece of code
| and verify the some basic scenarios, instead of waiting a
| minute to push to cluster, another couple minute starting
| whole process and then running actual test.
| fendy3002 wrote:
| Which IMO, a bad practice. Too much interface, abstraction,
| mocking do not reflect real process. I find it often break
| when integrated with services.
| tgtweak wrote:
| Damn I misread TTD and got excited that they were building it
| into factorio... Great article though, was not dissapointed.
| dvgt wrote:
| Exactly the same thing happened to me.
| nanis wrote:
| This is a neat article. I do have comments about testing in
| general though.
|
| IME most developer do not understand each test has four possible
| outcomes:
|
| * Code is good and test passes
|
| * Code is bad and test fails
|
| These are the only two possible outcomes developers focus on:
| When I ask what they should do if a test that used to pass now
| fails, they always tell me stories about how to debug the code
| under test.
|
| There are two additional possibilities in test:
|
| * Code is bad yet test passes (false negative)
|
| * Code is good yet test fails (false positive)
|
| Again, IME, most people do not look at the test again once it
| passes for the first time.
|
| As a result, tests which are themselves code, become the largest
| untested part of the code base. You get these thousands and
| thousands of lines of untested code yet you have 100% code
| coverage.
|
| Some of my blog posts on testing:
|
| * Deception in tests considered harmful
| https://www.nu42.com/2017/02/deception-in-tests-harmful.html
|
| * Know what you are testing: The case of the test for median in
| Boost.Accumulators C++ Library <https://www.nu42.com/2016/12/cpp-
| boost-median-test.html>
|
| * Who is testing the tests? https://www.nu42.com/2015/05/who-is-
| testing-the-tests.html
|
| * Slashing one's feet with tests, or, how to fix 2,950 test
| failures in one fell swoop
| https://www.nu42.com/2015/08/fix-2950-test-failures.html
| IMTDb wrote:
| > Imagine you have a company that goes slower and slower every
| quarter, and then you confront the shareholders with the
| statement, that the way to solve it, is to do absolutely no new
| features for a quarter or two, refactor the code, learn new
| methodologies etc. I doubt that the shareholders would allow that
|
| > now there are 9 programmers
|
| Companies on the stock market don't have "9 programmers". They
| have a lot of teams of 9 programmers. So while it's true that it
| would probably completely be impossible for a stock market
| company to completely freeze for a quarter or two, individual
| teams can still do that.
|
| If the factorio team grows to tens of programmers (it probably
| won't and probably shouldn't), I would be very surprised if they
| find the need - and if they manage to - freeze all teams together
| for a big refactoring round. I am also unsure that it would be
| the right approach. That observation holds wether they go public
| or stay private.
| Aditya_Garg wrote:
| Okay imagine you are a small startup backed with VC money. The
| same situation can still arise.
| meesles wrote:
| Except that as the author of the article says, they don't owe
| investors any explanations. VC money will demand results
| which you cannot just ignore for a quarter.
| ramblerman wrote:
| I'd be curious to hear what kovarex thinks in 2-3 months.
|
| TDD is often sold as a fix-all solution, which is incredibly
| appealing to mgmt and quite fun for most programmers as a new
| paradigm, allowing for quick adoption.
|
| It also has its uses, especially in the enterprise space where
| requirements aren't often clear. But I don't know many good
| programmers that truly stick to the dogma after the honeymoon
| period. It becomes just another tool in your toolset.
|
| Uncle bob is a salesman, not a "craftsman".
| tziki wrote:
| TDD, when normalized for time spent writing tests, has not been
| found to be any better than the 'normal' way of writing tests
| afterwards. It's interesting how much of programming lore falls
| apart when you actually try to measure the impact.
| blacktriangle wrote:
| You say that like its a bad thing. Sure dogmatic TDD can lead
| to issues, but going through a period of dogmatic TDD has, for
| me, resulted in becoming a far better programmer. And as time
| goes on I find myself drifting further back towards the
| dogmatic side of TDD as I know the difference how it feels
| working on tested portions of our code vs the untested portions
| of our code.
| JohnHaugeland wrote:
| > You say that like its a bad thing.
|
| That's because it is
| astrange wrote:
| It is kind of a bad thing. Recently at work there was a
| quality program introduced by some training guy, but by
| quality they seemed to mean write more tests and do TDD, and
| our junior engineers read that as write a ton of unit tests
| (the only kind they'd heard of.)
|
| I pointed out the 10 year old Norvig vs Ron Jeffries fight[1]
| which demonstrates that TDD is useless when you don't already
| know what you're writing, but they just looked confused.
|
| The other problem here is that unit tests never break (since
| you've mocked everything that can break) and therefore aren't
| worth running; it might be more productive to write them for
| TDD and then just not commit them.
|
| [1] https://news.ycombinator.com/item?id=3033446
| blacktriangle wrote:
| I agree with you, which is why I think the GPs post that
| TDD is not a dogma but another tool in your toolbox makes
| sense. Just like the larger development community when
| faced with new tools, individual developers go through the
| pattern of adoption where they start using a technique for
| everything (the honeymoon phase) and then through that
| phase they learn when the tool is appropriate and when the
| tool I not.
|
| As posters in your linked thread point out, the irony is
| that Norvig would have probably been better off using TDD
| since he knew the general shape of his solution already
| where Jeffries likely should have been writing several
| spikes to explore the solution space rather than using TDD
| since he was unfamiliar with the problem and did not yet
| have a good solution.
| bluGill wrote:
| Unit tests do break once in a while. 80% will never break
| once written and should be thrown away, but that other 20%
| will unexpectedly break in the future alerting you to a
| problem that otherwise you wouldn't find until much latter
| in the release cycle. I don't know have any way to know
| which tests are in the 20% and which the 80% to throw away,
| which is why I keep them all.
|
| Of course the 80%/20% figures are estimates. The longer
| your code is maintained the more likely it is some tests
| will break.
| 8note wrote:
| Unit tests that don't break are documentation, both about
| your code, and what you expect dependencies to do.
|
| Gives you something to compare against if the dependency
| changes its behaviour
| solipsism wrote:
| _The other problem here is that unit tests never break
| (since you 've mocked everything that can break) and
| therefore aren't worth running; it might be more productive
| to write them for TDD and then just not commit them._
|
| That's fine for code that never changes, or that has no
| logic in it. As soon as you want to change some code,
| you'll want a portion of the tests to require modification
| and a portion to stay passing.
| [deleted]
| ashtonkem wrote:
| Red/Green is a good technique for fixing bugs and extending
| existing functionality.
| koonsolo wrote:
| What is the chance of a fixed bug getting broken again? As it
| turned out in our analytics over 10 years: very very low.
|
| So the effort you put into writing a test for a bug, has most
| likely a negative return on investment. That time could be
| better spend somewhere else.
| duncan-donuts wrote:
| Maybe for you and your org though. This isn't universal.
| I've worked somewhere that would use the phrase "whack-a-
| mole bugs" because teams would start thrashing and break
| each others' shit over and over. There's a ton of stuff we
| could have done -- communicate (even just a tiny bit), less
| silos, communicate!, write better code, write tests.
|
| When your problems are much more fundamental, like teams
| just flat out don't talk, writing the test is an easier
| investment.
| synthc wrote:
| I disagree: when fixing a bug I usually start with writing
| a failing test that reproduces and exposes the problem, and
| then fix the problem so that the test passes.
|
| Understanding, reproducing and fixing the problem is the
| hard part, once that is done making a test out it hardly
| takes any time.
|
| Writing the test does not only prevent regressions, it also
| helps with confirming you actually fixed the bug.
| RHSeeger wrote:
| You're ignoring various parts of the equation, though.
|
| - Writing a test for a bug is part of understanding the
| bug, much like rubber ducking, it helps you make sure you
| know exactly what causes the bug.
|
| - The better you are at writing tests, the less time it
| takes you to do so.
|
| - Having tests for the code means the code is testable. In
| general (but not always), code being testable means it's
| better code (more readable, less complex, etc).
|
| - Having tests for code means you have to worry less about
| breaking it when making changes. This allows work to
| proceed faster.
|
| So it's not just "did adding this test prevent this one bug
| from re-occurring", it's "did adding tests improve our
| development overall". The later is far more likely to be
| true than the former.
| koonsolo wrote:
| The thing is that all that extra test code also needs to be
| maintained, also contains bugs, etc.
|
| It all comes down to return on investment. For example, I used
| to agree with TDD that for every bug, first write a test that
| fails, and then fix the bug. That way you prevent regression.
|
| So I proposed this to my manager, and he responded: we tracked
| all bugs in our system for the last 10 years, and when you look
| at fixed bugs that get broken again, it only occured very
| rarely. So in the end, doing that was not the best investment
| of effort.
| linspace wrote:
| > The thing is that all that extra test code also needs to be
| maintained, also contains bugs, etc.
|
| I have often observed an evolutionary behavior on tests:
| tests that pass easily survive
| tikhonj wrote:
| When I fix a bug, I have to reproduce it _somehow_ to make
| sure my fix actually works. Adding a regression test is a way
| to save that work as code. Once I have a reasonable test
| infrastructure set up, the majority of the effort is in
| understanding and reproducing the bug; going from that to an
| automated test should not take significantly _more_ effort.
|
| In return, I get to have some extra confidence that a bug
| doesn't return (which would be embarrassing, even if it's
| infrequent!) and I get a more thorough test suite that lets
| me refactor more quickly and aggressively. And if an old bug
| _does_ come up again, the advantage is not only that the test
| will catch it before a release, but also that the person
| fixing the bug won 't have to go through the effort of
| figuring out how to reproduce it from scratch--it's
| reproduced right in the test suite!
|
| So I am not sure that just looking at how often regressions
| actually happen in the existing codebase is sufficient to
| make any real conclusion by itself.
| cbushko wrote:
| > So I proposed this to my manager, and he responded: we
| tracked all bugs in our system for the last 10 years, and
| when you look at fixed bugs that get broken again, it only
| occurred very rarely. So in the end, doing that was not the
| best investment of effort.
|
| That sounds very hand wavy.
|
| It is making the assumption that: - your bug
| tracking system and people are so good that they have found
| all the duplicate tickets. - they don't make mistakes
| and find all duplicates for tickets. - your code is so
| good that it hasn't had any side effects that caused
| regressions. - your boss is so good that he has a full
| grasp on 10 years worth of bugs.
|
| edit: formatting
| drewcoo wrote:
| Brushing _that_ tooth is a bad ROI because it rarely keeps a
| filled cavity from recurring.
| asddubs wrote:
| One point uncle bob makes is that doing this for everything
| allows you to make far-reaching architectural changes with
| confidence that you haven't broken a bunch of things in
| places you don't even realize. so the tests are a tool to
| allow you to do refactorings you would otherwise be scared of
| IggleSniggle wrote:
| This is what good sum types are for, and have the added
| benefit of not just testing what the an output is after the
| fact, but can tell you as you're writing the thing what the
| output must be, making refactoring far faster than waiting
| on a test suite to fail.
| the_gipsy wrote:
| This can also lead to an asphyxiating second-system that
| prevents you making the slightest architectural change.
| Chris_Newton wrote:
| _One point uncle bob makes is that doing this for
| everything allows you to make far-reaching architectural
| changes with confidence that you haven 't broken a bunch of
| things in places you don't even realize._
|
| I was working on a new project recently. Over the first
| month or so I went through four or five significant
| iterations of the architecture before I settled on one that
| seemed to have a healthy mix of power, flexibility and
| simplicity.
|
| Each time, I found the test _cases_ I'd identified during
| earlier iterations helpful. Ultimately they led me to a
| more rigorous analysis to make sure I'd covered all
| required cases in all required places.
|
| However, I hardly ever kept the test _code_ from one
| iteration to the next. Each unit test follows a general
| pattern of arranging whatever scenario I want to test, then
| calling the function under test itself, and then asserting
| some expected result. With a significant architectural
| change, the interfaces for arranging things or to the
| function under test might be changed, leaving much of the
| code in the test obsolete. The responsibility being tested
| might not even be in the same part of the code any more,
| meaning a whole set of test cases now need to be applied to
| a different component in the system.
| koonsolo wrote:
| That is definitely true and I fully agree with this.
|
| The benefit is basically all the tests that you don't have
| to change and give you confidence that your refactorings
| don't break certain things.
|
| But the drawback is all those tests that do need to be
| rewritten because of the refactoring, and so will slow you
| down again.
|
| But in the end for this use-case, I think it's a good ROI.
| Probably the best use-case for TDD.
| buzzwordninja wrote:
| A brittle test suite during refactoring is usually caused
| by too much mocking.
|
| Look at the concept of sociable tests.
|
| https://martinfowler.com/bliki/UnitTest.html
| tarcon wrote:
| Thats a problem of your tests. Go the classicist approach
| and your test don't fail after a significant refactoring.
| ashtonkem wrote:
| Statements like that make me wonder when the last time he
| committed to production code, because that's just laughably
| wrong.
|
| Maybe if you work in a monolith, sure. But most of us work
| in distributed systems with really complex behavior. No TDD
| suite in the world is going to catch a thread pool issue
| that'll open a circuit breaker in your client.
| berkes wrote:
| > No TDD suite in the world is going to catch a thread
| pool issue that'll open a circuit breaker in your client.
|
| I don't know your setup. But this is normally _exactly_
| what system tests do (aka E2E tests, GUI tests and so on;
| the tip of the Testing Pyramid). In distributed systems,
| those often live outside of the various components '
| codebases; maybe in their own project even.
|
| Edit: because it is a) very impractical to check these
| things manually, b) often simply impossible to check them
| before each release and c) requires hoops and tricks to
| get in a state where such events/regressions occur in the
| first place; a state that is near impossible to get to
| manually.
| KronisLV wrote:
| I think that what the majority of the people within the
| industry want to do (microservices and distributed
| systems, much like FAANG) far exceeds their capability to
| deal with the complexities (such as testing distributed
| systems) and doesn't always even fit their needs.
|
| I get the feeling that if more people were okay with
| developing monoliths (albeit modular ones), then a lot of
| things could be easier to do, such as doing TDD properly
| and being able to refactor without fearing the unforseen
| consequences.
|
| Heck, maybe the projects that i work on in my $dayjob
| would even have proper documentation, decent test
| coverage, as well as a set of tools to support the
| development processes, instead of having to waste time
| managing the configuration, deployments, as well as
| system integrations. Maybe it's a matter of there not
| being enough workforce (or the managerial folk not seeing
| much point in investing resources into testing and other
| ops related activities that don't generate business value
| directly, a worrying trend i've noticed), but right now
| i'm implementing Ansible and containerization to at least
| simplify some of this stuff, but it feels like an uphill
| battle, as i'm also supposed to ship new features.
|
| Surely i'm not the only one that kind of sees why the
| person that you're replying to would express the
| viewpoints that they did? It's hard to do elaborate E2E
| tests when everything is figuratively on fire constantly
| and you're struggling with a complicated architecture
| which may or may not be necessary. I'm probably
| projecting here, though, but every single enterprise
| project that i've worked on has been like that.
| peregren wrote:
| Even if it's only rare, the test also shows clearly to a
| reviewer that the bug has been fixed.
|
| In my opinion, as soon as a test suite finds a bug it has
| added a lot of value, even if it's rare.
| dgb23 wrote:
| Regression tests are apparently an effective tool to maintain
| code stability.
| alanfranz wrote:
| My 2c: tdd is great in order to learn to create testable
| designs. You can't tdd nontestable code.
|
| Once you understand how to design testable code, tdd offers
| minimal benefits.
|
| The real value comes from thorough, automated testing suites,
| whatever their origin is.
| Chris_Newton wrote:
| _The real value comes from thorough, automated testing
| suites, whatever their origin is._
|
| Automated testing is useful, for sure. However, TDD purists
| tend to focus on one very specific type of testing: automated
| unit testing, in the small, where you already know the
| expected output for given input and can easily specify that
| output using assertions in code. By its nature, TDD
| emphasises being testable in that specific sense above all
| else. I don't think that is necessarily a good thing, partly
| because that type of universal unit testing may not be a good
| strategy for every software system, and partly because other
| useful properties of the code might be diminished because of
| the changes needed to make it "testable" in the TDD sense.
| alanfranz wrote:
| > where you already know the expected output for given
| input and can easily specify that output using assertions
| in code
|
| Well, "knowing" (or manually generating) the output for
| your code is required to check whether your code is
| working. If you execute your code and copypaste its output
| blindly, you risk testing the wrong thing, don't you.
|
| And if you can't specify your code's output, how do you
| test?
|
| BTW, I agree with the fact that TDD is overly pedantic at
| times.
| Chris_Newton wrote:
| _Well, "knowing" (or manually generating) the output for
| your code is required to check whether your code is
| working._
|
| Not all code is written knowing the answer in advance.
| Sometimes you're implementing a mathematical model to
| predict next week's weather, or the effects of a change
| in economic policy, or the performance of different hull
| shapes for a ship under different conditions at sea.
|
| My second standard challenge to TDD evangelists is to
| write a program to draw a classic multicoloured
| Mandelbrot image, showing an arbitrary rectangular region
| of the complex plane, using only the mathematical
| specification without any reference to existing code or
| output data.
|
| This can be done with a short program that is simple
| enough to verify by eye that it matches the required
| mathematical specification. However, the problem has an
| infinite input space, its fractal nature means predicting
| output values is impossible in the most interesting parts
| of that input space, and you have no idea where those
| interesting parts are anyway.
| sterlind wrote:
| that actually sounds like a fun thing to do in a formally
| verified language like Dafny. you'd write a "ghost
| function" for the Mandelbrot equation, then write your
| code, then prove a theorem that the epsilon between the
| equation and the code was below a threshold at every
| pixel in your rectangle.
|
| I guess that'd still be TDD, but with the first T
| standing for Theorem :p
| snovv_crash wrote:
| Yes, of course there are cases where it doesn't work. But
| what percentage of code is that, really?
| Chris_Newton wrote:
| I don't know what percentage of all code that gets
| written would be better tested in other ways. I doubt
| anyone else does either.
|
| For me, unit testing (in the typical xunit style
| associated with TDD) is most useful for basic data
| processing code with predictable outputs. That might
| include a wide range of code, from little utility
| functions on strings to the business rules for whole CRUD
| applications.
|
| On the other hand, anything with input or output data in
| an awkward format, anything communicating with any
| external equipment or remote API, anything involving
| nondeterminism or heuristics or where the purpose of the
| code is to perform some calculation where you don't know
| the correct answer in advance, these kinds of code don't
| tend to fit well with a TDD approach and that style of
| unit testing as the main test strategy IMHO. Those cover
| a pretty wide range of code as well.
| iudqnolq wrote:
| What's the issue with using the formula and a calculator
| to compute sample points?
| Chris_Newton wrote:
| What you are advocating is duplicating the calculations
| the program does manually for some set of specific cases,
| which can then be used for tests. Let's consider a few
| practical questions about how that might work.
|
| 1. How do you choose which specific cases to calculate
| manually?
|
| Remember, you have no prior knowledge of what the correct
| answer looks like or where the interesting parts of the
| input space are. You have no way to determine whether any
| given set of manual calculations is representative of the
| work your program will be doing or covers the areas with
| the greatest risk of error.
|
| 2. How practical is it to make all those manual
| calculations?
|
| Even in this very simple case, calculating the correct
| answer for a single input point manually might require
| hundreds of complex arithmetic operations to be
| performed. That's going to be slow and error-prone. After
| all, isn't that why we're writing a program to do this
| for us?
|
| Now, how does this idea scale? What if our program isn't
| computing a nice analytical solution to a simple
| arithmetic problem, but instead running a complicated
| numerical method to process many thousands of data points
| in each input? It quickly becomes impractical to rely on
| this strategy for testing.
|
| 3. How will having a set of known outputs you can test
| against _drive_ an implementation from scratch?
|
| I mentioned before that Mandebrot is my second standard
| challenge to TDD evangelists. The first is to write
| add(x,y) driven by tests. After a bit of back and forth,
| this invariably ends up with an implementation that was
| generalised from however many specific cases were given
| to the general case. Invariably, that generalisation is
| the step that actually creates a useful solution to the
| original problem, and invariably it _uses an insight that
| was not driven by the tests_.
|
| Our Mandelbrot scenario is the same situation, just a
| slightly more complicated example. No matter how many
| individual tests you create by choosing sample points in
| the input space and manually calculating the expected
| output, you won't have a systematic way to work back from
| those answers to derive a correct general implementation
| of the Mandelbrot calculation. Your specific cases might
| be useful for verifying an existing implementation, but
| they give you no insight into how to write a good
| implementation from scratch. (If I'm dealing with a
| particularly strident advocate of TDD, this is the point
| where I mention the word "sudoku".)
|
| And again, we have to ask how this process scales. What
| if we had a more challenging problem, say extracting an
| audio track from a video file and running a speech
| recognition process on it to generate subtitles? It might
| actually be easier in that scenario to identify
| individual test cases: just take a known video file as
| input and write down the expected output for it, and now
| you have an end-to-end test. However, it's still true
| that no amount of end-to-end tests will necessarily offer
| any insight into how to structure a good implementation
| of the required functionality in detail, nor will it tell
| us how to implement any specific part or generate useful
| unit tests cases for those parts. End-to-end test cases
| might help us to verify an existing implementation, but
| in general they won't reliably drive a correct one from
| scratch.
| snovv_crash wrote:
| It gives you something that can be verified for when
| someone does some optimization, or ports to a different
| platform. It isn't for today, it is for tomorrow.
| Chris_Newton wrote:
| I agree that having an automated test suite might be
| helpful in those situations, but TDD comes with a lot
| more baggage.
| ashtonkem wrote:
| Good observability standards and fast releases catch more
| bugs than meticulously maintained test suites, imho.
| drenei wrote:
| I actually think if you squint testing is part of
| observability, except it gives you signal during the
| development phase. And observability as a concept is
| fundamental to a reliable system.
| ashtonkem wrote:
| I see where you're going, but I think if we start
| defining tests as part of observability then we'll water
| that term down to meaninglessness.
| snovv_crash wrote:
| If there's a bug and no test suite to try to reproduce
| it, how do you know you actually fixed it?
| pydry wrote:
| Sacrificing at the altar of unit testability doesnt
| necessarily make better code.
|
| Unit tests' inability to handle state is way too often viewed
| as a problem with the code it can't properly test than the
| general crappiness of this form of test.
| pc86 wrote:
| Doesn't the state get handled in integration tests? The
| best environment I've worked in (from a testing
| perspective) had thousands of super fast unit tests. We had
| it run continuously (.NET / Visual Studio) and the entire
| suite probably ran in 3-4 seconds. When you submitted a PR,
| a suite of longer-running integration tests kicked off that
| took a couple minutes but a failure automatically kicked
| the PR back to the dev, and success notified everyone there
| was a new PR ready for review.
| bcrosby95 wrote:
| Most of our tests are "integration" tests. With an in
| memory database which is blazingly fast. Our longer tests
| use a real database, which is just a config away.
|
| Our projects are so database centric, I've almost never
| seen a true unit test fail. The majority of time it's the
| tests that use the in memory RDBMS.
| pydry wrote:
| It does.
|
| For some reason "code testability" is almost exclusively
| used to refer to the overuse of dependency inversion to
| make it easier (although not necessarily more useful) to
| write unit tests, though.
|
| I find that code falls into three camps -
| integration/logical/mixed.
|
| If it's integration code only an integration test is
| really useful and a unit test is a pointless waste of
| time that will do little more than mirror the code you
| wrote and fail when it is changed for any reason at all.
|
| If it's logical code a unit test is most useful and
| integration tests are likely going to be too slow.
|
| If it's mixed, integration tests are most useful although
| drawing out and decoupling the logical from the
| integration code and testing them separately is better in
| the long run.
| naikrovek wrote:
| TDD and OOP as dogmas are very bad, for different reasons.
|
| TDD encourages you to write many times the number of lines of
| code for your tests as the code you're testing, often 10X or
| more. So when you inevitably decide "there's a much better way
| to do this" (which happens to me 100% of the time) then you're
| not only changing your code, you're changing all of those
| tests. That's a lot of weight that you now have to deal with.
|
| Most of the time, that's enough weight that the better design
| simply doesn't happen and becomes yet another chunk of tech
| debt that prevents certain things from happening in the future.
| I've seen that happen, and it's given me a very bad taste in my
| mouth for "TDD" because "TDD inertia" is the reason that better
| designs aren't implemented. If that piece of software lives for
| a while, and grows in scope, someone is going to have to deal
| with that, and eventually make the architectural change anyway.
|
| By all means, test your code, of course. If you can find a way
| to easily generate test cases for an arbitrary code base, by
| all means, do that, because the test cases are no longer
| influencing your decision to change how your application works.
|
| Otherwise, being "test-driven" is bad, IMHO. Software
| development is _never_ as simple as the various dogmas would
| lead you to believe.
| CharlesW wrote:
| > _So when you inevitably decide "there's a much better way
| to do this" (which happens to me 100% of the time) then
| you're not only changing your code, you're changing all of
| those tests._
|
| Isn't the point of TDD that you can change implementation at
| will, safe in the knowledge that the tests will help
| guarantee that you're not inadvertently changing behavior?
| twh270 wrote:
| Usually, yes -- your tests should be decoupled from the
| implementation.
|
| Usually however, what I see (in Java and Spring Boot land)
| is test methods which use mocks that are set up to expect
| certain method calls, e.g.
| when(accountService.getAccount(id)).thenReturn(account).
| Typically there are anywhere from six to 20 lines of this
| stuff per test (and, it's rarely de-duplicated into a
| separate method). So as soon as the contract to
| getAccount() changes, a bunch of tests need to change.
|
| Second, it's common to believe that the "unit" in a "unit
| test" is either a class or (less often) a method.
| Consequently, people write test classes for every single
| class of a group of classes that are acting in
| collaboration, when what they really ought to do is write
| tests against the single class supplying the public
| interface, varying input to that class as necessary to
| achieve good confidence.
| bogdanoff_2 wrote:
| Unless what you're testing has a very clear interface
| boundary (like a pure function whose behavior is unlikely
| changed, or the program's end-to-end behavior), there's no
| clear distinguish between behavior and implementation.
|
| Most of the time you have classes that interact with one
| another. If you test each class individually, and you
| decide to refactor what classes do what, you'll need to
| rewrite a lot of tests, even though the higher level
| behavior should not change.
| cryvate1284 wrote:
| This depends on what you're doing. For example, if you are
| changing an intermediary (abstraction) layer, then the
| (unit) tests (TDD or not) for those will have to change and
| the "guarantee that you're not inadvertently changing
| behaviour" is kinda moot.
|
| If you do not have TDD, I guess the reasoning is that
| either you do not have tests for this (probably bad) and if
| you do, less of them (and so more easily changed).
|
| Not sure it's an argument against TDD, but I guess if
| management/the programmer do not know about sunk cost
| fallacy, it might make them hold on to bad
| abstractions/layers.
| CharlesW wrote:
| Thanks, that's helpful!
| ema wrote:
| Often the better way to do something is achieved by
| changing how two components interface with each other. in
| this case you have to change the tests that test this
| interface.
| imiric wrote:
| > TDD encourages you to write many times the number of lines
| of code for your tests as the code you're testing, often 10X
| or more.
|
| If that happens I'd say you're doing TDD too early. TDD can
| be very useful, but early on in the process of transferring a
| design to code, all your interfaces are highly unstable and
| there's a lot of experimentation, so sticking strictly to TDD
| would naturally be frustrating. I would start a bit later
| when at least some of the API has stabilized and you don't
| have to do major changes.
|
| TDD helps with creating user friendly APIs, since you
| experience it from the user's perspective. And it forces you
| to actually write testable code and not incur technical debt
| that's very costly to remove later (having to refactor code
| to make it testable).
|
| > Most of the time, that's enough weight that the better
| design simply doesn't happen and becomes yet another chunk of
| tech debt that prevents certain things from happening in the
| future.
|
| This reads like you're saying that tests themselves are
| technical debt...? Because you're going to run into this
| issue (or _should_ run into it) regardless if you use TDD or
| not. Eventually you 'll want to refactor parts of your
| codebase and, sure, you'll have to change some of your,
| hopefully mostly, unit tests. So in that sense you can say
| that doing TDD early creates a lot of technical debt that
| needs to be resolved quickly, but like I said above, that
| doesn't have to be the case.
|
| TDD can be a dogma just like any practice (Agile is my
| favorite), but it doesn't mean that it's not useful if used
| correctly.
|
| Kudos to the Factorio team for adopting it, which I think is
| rare in the gaming industry. The idea alone of testing the
| complexities of a video game is mind boggling to me as a web
| developer. Especially in an industry where it's popular to
| hype and sell broken products with the promise of patches and
| DLC.
| naikrovek wrote:
| does TDD not dictate that you write tests _first_ , before
| you write the code under test? then you write only enough
| code to make the test pass, as I recall.
|
| yeah TDD is a mess
| imiric wrote:
| Why does a practice have to "dictate" anything? TDD is a
| workflow suggestion, not gospel that must be strictly
| followed.
|
| Writing and maintaining a test before any experiments
| have been done with the design would indeed be
| frustrating and lead to a lot of rewriting. If you first
| explore the design and allow it to settle into a usable
| interface and then write the test for it as assurance
| that it's stable, you'd have the benefit of being able to
| safely refactor the feature and quickly add more tests
| for it. As long as the deliverable includes tests with
| decent coverage, who cares whether you wrote a test first
| without writing a single line of the implementation?
| kllrnohj wrote:
| I think kovarex is avoiding your TDD concerns by rejecting
| the notion that everything must be covered by minimal-scope
| unit tests. This means that there's much fewer (if any) mocks
| that need updating when the implementation changes, and fewer
| tests should end up needing to be touched as a result as
| well.
|
| See specifically the "Fig. 5 - Test dependencies" section of
| the post.
|
| Personally I think mocks are far over-used in tests, and I
| much prefer the solution kovarex outlines. I think dealing
| with mocks are the bigger source of test-update friction here
| than the tests themselves. I was already layering my tests
| instead of using mocks. As in having no clear line between
| "unit" and "integration" tests, everything just uses the
| "real" implementations of things and tests just get naturally
| more complex the higher up the stack it's testing. The idea
| of sorting by dependency depth is a cool idea I hadn't ever
| considered, though, I'll be borrowing that idea.
| pault wrote:
| I always thought of TDD as a way to exercise your code while
| writing it, not a set in stone spec of your product. I
| consider unit tests disposable and basically noise; they're
| only useful when you're writing code. They are by nature a
| test of your implementation, not your specification. I
| usually throw away a lot of them after I'm done writing, and
| if I refactor I just delete the ones that are failing because
| I'm writing new ones as I go. There are places at boundaries
| where more stable unit tests can go, but even there a change
| to the interface is going to break all the tests anyway.
|
| The better way to test your specification and behavior, IMO,
| is to use comprehensive E2E tests. The traditional test
| pyramid is upside down. If you have an E2E test for every
| user story in your spec, you can develop with confidence that
| your new code will not disrupt your users' activity. Unit
| tests are cattle, E2E tests are pets.
| fendy3002 wrote:
| From my experience, tdd is useful if you already have solid
| specification, the code / part can be tested and you / your
| team understand how to develop unit / integrated tests. The
| problem arises because usually one or some of the points are
| unfulfilled.
|
| And it's not without drawbacks. Increased development time and
| the needs to maintain the unit tests are costly, but rewarding.
|
| Also I don't like too much interface and mocking only for the
| sake of testing. I find it usually breaks when integrated and
| makes code harder to maintain. Maybe I'm just inexperienced.
| berkes wrote:
| > The problem arises because usually one or some of the
| points are unfulfilled.
|
| An important idea of TDD is that it allows you to discover
| those "unfulfilled points" in the tests. When writing code
| (the tests) that _use an API_ , instead of when writing the
| actual API.
|
| When writing code that uses objects, methods, interfaces and
| so on, you are in a mindset of writing "what a user of the
| code would wish there was". This is probably the best place,
| mindset and moment to define those specs in detail.
| skinnyarms wrote:
| Am I missing something, it sounds like they are already doing
| this - not striking off on a new venture.
| wpietri wrote:
| Nobody should stick to any dogma past the honeymoon period. The
| point of a dogma is to get you into the behavior space that you
| will learn how to do something well. It's like using a recipe
| in a cookbook. Once I had enough practice making scrambled
| eggs, I didn't need a recipe anymore.
|
| I learned TDD ~20 years ago from Beck's "TDD by Example" book.
| For me it's far more than "just another tool". I'll certainly
| do exploratory work with throwaway code. And in the early
| stages of something, I might be a bit slack. But the lesson
| over and over for me is that if I don't get to significant test
| coverage soon, I end up wasting a lot of time on bugs. And TDD
| is the easiest way to get to test coverage. Test-first ends up
| feeling like a set of small successes with the red-great-
| refactor loop. With test-after, going back and adding tests
| feels more like drudgery, and I'm less likely to have written
| the code in a testable way.
|
| So TDD is not dogma for me, just the inevitable place I end up
| if I want to maximize the amount of time getting things done
| while working on a non-trivial codebase.
| nightski wrote:
| I had a similar trajectory but opposite experience. I felt
| like after doing heavy TDD and even FP after a while TDD felt
| more and more like a waste of time. I find that I naturally
| think and design in a test driven perspective without
| actually writing the tests due to many years of experience.
|
| 1.) I'd much rather lean on the type system to prove things
| than automated tests if possible. But of course depending on
| the language that often isn't possible.
|
| 2.) I find that only a small fraction of my code really
| benefits from automated testing. This is the
| logic/calculation parts of the code. The rest is slinging IO
| and SQL queries which all ends up being mocked out anyways
| and the tests just become secondary implementations of the
| original code.
| wpietri wrote:
| That said, I totally agree that Martin is a salesman. I knew
| him in the early days of the Agile movement and he did a lot
| of good then. But he's become consistently more strident,
| more dogmatic, more unpleasant. It's sad to see, really.
| cjfd wrote:
| I learned TDD years ago and never looked back. It is true that
| there are some things where it is less suitable and therefore I
| would perhaps decide not to use it. E.g., user interfaces in
| case of changes that are mainly visual. Or code that tightly
| integrates with the system. E.g., file system manipulations.
| For all other code I think TDD is absolutely the best way to
| write anything more complicated than a throw-away script.
| Frankly, I am pretty much at the point where I consider
| anything less than TDD borderline unprofessional. I have come
| to expect code that was not TTD-ed to be buggy or more
| complicated than necessary or both.
| ramblerman wrote:
| > Frankly, I am pretty much at the point where I consider
| anything less than TDD borderline unprofessional.
|
| Would you feel that way about the linux kernel for example?
| cjfd wrote:
| Actually, I am not entirely sure how much of the testing
| there is automated. Presumably there are at least some
| automated test tools floating around here and there for the
| kernel. Major parts of the kernel could fall under the
| 'systems' exceptions where they are very close to the
| hardware. It is possible to do TDD to stuff that is close
| to hardware but it certainly is harder and it is more
| likely that some failures fall through the cracks when
| doing TDD. One thing that is different about the linux
| kernel, as far as I know, is how thorough the reviewing is
| that is going on for every patch. Also, there is the merge
| window followed by a long stabilization period. I would say
| that if you are not doing TDD you are going to need
| processes like that if you want to maintain stability. Also
| note that processes like thorough review by multiple people
| and a stabilization period four or five times longer than
| the merge window sound really expensive. Much more
| expensive than TDD. But also if one does TDD, still
| system/end-to-end tests are desirable. Not as much as
| without TDD but without it things would still break
| somewhat regularly.
| Chris_Newton wrote:
| With systems programming, performance matters, and all
| that extra indirection can be expensive.
|
| Indirection can also make it harder to rigorously analyse
| all possible paths through a piece of code and make sure
| you've handled all required cases.
| dnautics wrote:
| It's very possible to do the indirection at compile-time,
| set a flag only active during tests.
| Chris_Newton wrote:
| It's possible, but typically you then pay a higher price
| on the other point I mentioned.
|
| I used to work on a library that solved a certain type of
| mathematical problem, written mainly in C++. It was built
| separately for many target platforms in production:
| different operating systems, processor architectures,
| compilers. Each supported combination of these might
| require special provisions in the code due to compiler
| bugs, hardware limitations, etc. Then you'd have
| diagnostic code that was only included in the debug
| builds used during development. There are more than
| enough preprocessor shenanigans to go around in that kind
| of environment already, and another layer for test-
| related flags and checks isn't going to break any records
| for most readable code.
| dnautics wrote:
| in the languages I work with this is not a problem. Mox,
| for example, is what I use in Elixir, and even though
| "swapping out a module" is kind of a global change, it
| has facilities to track these mappings in runtime, even
| in concurrent testing, but the module is selected at
| compile-time, and only in test, so it is a "zero-cost
| abstraction" for running in prod.
|
| I think this is just internalized pain of working with
| C++ (I suffered a lot with that 20 years ago). For
| systems-level stuff, I'm getting very excited about Zig,
| I imagine it will be basically "equally easy" to
| instrument these sorts of things at compile-time in Zig,
| as types are first-class values at compile-time. Zig is
| also super-composable; I have an example prime sieve that
| can run 60+ settings (combinations of single threaded,
| multithreaded, bool array, bitmap, hyperthread-awareness)
| by using a compile-time for loop across a series of
| instrumented settings; so long as I stay within the
| standard library the language itself already takes care
| of the polyfill for platform stuff, and I have access to
| those flags, if, say I wanted hyper-optimize for things
| like sse. I don't see any mocking libraries yet, but I
| doubt that instrumenting a mocking namespace in test will
| be any more difficult than it would be, in, say, elixir.
| berkes wrote:
| > One thing that is different about the linux kernel, as
| far as I know, is how thorough the reviewing is that is
| going on for every patch.
|
| This is another underexposed feature of TDD: reviews.
| When I see a PR with solid test coverage, and those are
| green, it greatly speeds up the process. I just need to
| read through tests, see if they make sense, match the
| requirements and test those. And then glance over the
| added code quickly.
|
| It (unfortunately, I might add) is not always possible to
| flat out reject a PR without solid test coverage, so it
| often is _less_ work for me to add some test myself to a
| PR, than to manually poke around the changed code or to
| read through it in detail. This is not TDD, but testing-
| after-the-fact, and those test often are of much lower
| quality, but at least this is less work, for me.
|
| Testing greatly speeds up the work for a reviewer.
| chitowneats wrote:
| Tests != TDD. TDD encourages an architecture with lots of
| indirection so that it can achieve the necessary
| polymorphism for injecting dependencies in test.
|
| I can assure you the linux kernel is not TDD'ed.
| wpietri wrote:
| I think that's typically true, but that's because most
| TDD is done in languages where there's not a lot of cost
| for indirection. I expect we'd see a different outcome if
| something like a kernel were started with a TDD approach.
|
| For example, we might see ways of expressing indirection
| for testing that then get removed when building the final
| product. Or we can look at the great strides made in
| virtualization since the Linux kernel was started.
| Something previously hard to test becomes much easier.
| And I'd expect more so if the testing had co-evolved.
|
| Of course, there still might be hard-to-test areas. But
| given the massive ingenuity applied to the kernel
| development process over the years, I expect that they
| would have advanced what's possible in TDD if they had
| pursued it.
| chitowneats wrote:
| Why do you think that linux kernel developers have not
| changed their approach? TDD is not a new idea.
|
| If indeed they have continued for decades to do it "the
| wrong way", imposing costs onto the project and society
| proportional to the number of preventable bugs creeping
| in, is this not a major scandal for the industry?
|
| I suspect that the core team have reasons that they have
| not developed tooling to enable TDD of the Linux kernel.
| And that those reasons are more substantive than "meh".
| dnautics wrote:
| > TDD encourages an architecture with lots of indirection
| so that it can achieve the necessary polymorphism for
| injecting dependencies in test.
|
| This is a language/library thing. Some language/libraries
| let you do TDD with very little indirection, a table of
| mocked dependencies as compile-time configuration, and
| all of the mocking events happen in code in your tests.
| wpietri wrote:
| If it started today, I certainly would. But I give them a
| pass because a) legacy code, b) legacy process, c)
| distributed process, and d) hardware is hard.
|
| But consider this article on how it is tested:
| https://embeddedbits.org/how-is-the-linux-kernel-tested/
|
| It mentions a lot of automated testing, which is good. And
| if you're doing automated testing anyhow, I think TDD is
| the most effective way to get there. But consider this:
|
| "The kernel development process also proves that there is a
| big focus on testing since around 20% of the development
| process (2 weeks) is reserved for code integration and the
| other 80% (8 to 10 weeks) is focused on testing and bug
| fixes."
|
| 80% is a lot! And that also suggests significant lag
| between writing a bug and having to fix it, which means a
| lot more time figuring out what went wrong, and a lot more
| code written after the bug, meaning really fixing it might
| be expensive.
|
| So I think it's reasonable to ask how much more effective
| kernel development would be if they could safely eliminate
| that 80% of time spent on cleaning up after the 20%.
| Jap2-0 wrote:
| I'm not a kernel developer, but I'm not sure if it's
| entirely accurate to call it an 80/20 split. Sure, Linus
| only merges new features during 20% of the release
| process, but as far as I can tell (again, not a kernel
| developer) most everyone else continues to work on new
| features outside of the merge window, and just submits
| them during that 20%, rather than spending that 80% only
| bugfixing.
| wpietri wrote:
| Oh sure, I'd expect the labor numbers to be different.
| But it's still a pretty obvious constraint, and one that
| causes long feedback loops.
| chitowneats wrote:
| These sentiments are basically copypasta from Bob's books.
| All this indicates to me is that you've drunk the koolaid.
| serverholic wrote:
| Have you considered that maybe TDD is just a really good fit
| for your brain?
|
| I've worked at a TDD-focused company and it always felt like
| I was coding through molasses. Some of us weren't as strict
| about TDD and I didn't notice a difference in our code
| quality.
| wpietri wrote:
| I would be very curious to see the code base there. Some
| places are pretty bonkers about testing, in a way that
| seems almost religious to me. Excess mocking, extremely
| verbose tests, and a lot of cleanup to do in testing-land
| whenever you make significant changes in the production
| code. Maybe you were at one of those?
|
| To me, doing TDD right is about having the minimum amount
| of testing needed to keep bug rates extremely low. It's
| also about keeping the test code as well factored as the
| production code, such that changing or refactoring the
| production code doesn't create disproportionate work in the
| test suite to bring it back in line.
|
| An experiment I've tried repeatedly is to start a new
| project and shoot for zero bugs. I think I'm pretty smart,
| so that at first seems achievable without tests. But pretty
| quickly complexity increases that I at least have to
| manually test. And not long after that, manually testing
| everything gets tedious, so I have to start automating the
| things I am manually checking. If I keep pursuing zero bugs
| and feeling like I'm spending my time optimally, I keep
| ending back up at TDD. Maybe try that for yourself on a
| hobby project? You might find that the TDD-focused company
| was better at talking about TDD than doing it.
| mypalmike wrote:
| I've found more problems with insufficient mocking as
| opposed to excess mocking. If you're not using mocks,
| you're likely using real dependencies. And that's where
| you get molasses testing. One small code change has
| cascading effects on tests, and small feature changes
| that take a few minutes to implement can result in a day
| or more of just fixing the affected tests.
| twh270 wrote:
| Look up the Object Mother pattern. The idea is to have
| 'baked' objects which you can use repeatedly in tests,
| which fulfill a certain scenario. For example a hotel
| reservation system needs to be able to handle a single
| person up to a whole family (maybe with an allergy or
| special needs thrown in for good measure). So you set up
| a bunch of Guest objects using a GuestObjectMother.
|
| When you make a change to Guest, you only need to change
| the GuestObjectMother and any tests that may directly be
| involved in the change to Guest. Tests that simply
| require a Guest to be involved in the test probably won't
| have to change, as they're simply retrieving the
| appropriate Guest object and then handing it off to
| something else.
| serverholic wrote:
| Everything you said is about good testing practice, not
| specifically TDD.
|
| TDD can be nice for small modules when you have a good
| idea of where you want to go with your code. For large
| modules with a lot of unknowns it can be a pain, and in
| those cases I prefer testing after a first draft.
|
| Perhaps you haven't built a module that was very large
| yet? Maybe try writing a large module with lots of
| unknowns. You might find that TDD gets in your way more
| than it helps in the early stages.
| wpietri wrote:
| It's true that TDD and good testing practice are related,
| so I agree on that part of it. I also agree that
| sometimes when I don't know what I'm doing, it's worth
| writing prototype code and throwing it away or bringing
| it up to standards. Other times I'm fine starting with
| TDD as a way of exploring the design space.
|
| As to whether or not I have enough experience with TDD to
| judge it, I'll leave that up to you. But I've been doing
| TDD ~20 years, and I wrote a lot of code before that. Let
| me know if that's sufficient.
| sidlls wrote:
| I take the opposite view. Code developed with TDD tends to be
| overly complicated and inefficient, and the test suites
| themselves are often bizarre labyrinthian hellscapes of
| mocks, facades, and workarounds. "They just did it wrong,
| then," you say? That statement ceases to be meaningful when
| it can applied routinely: it's the norm, and if it's the
| norm, it's not something "they just did it wrong" applies to.
| blacktriangle wrote:
| So the way that TDD advocates point out that if your tests
| are ugly, its a sign your code is ugly. I think this also
| applies to languages and frameworks as well.
|
| For example, I think of the experience doing TDD in Rails
| with all its ORM and callback interconectedness to TDD on
| functional code that is mostly exchanging values between
| systems. All the stub and mock crap goes away since values
| become the boundary of the system. Gary Bernhardt has an
| awesome talk making that exact point.
|
| I'm now of the opinion that if TDD is a struggle, its a
| sign that your environment has serious underlying flaws.
| sidlls wrote:
| "[I]f your tests are ugly, its a sign your code is ugly"
| is a bit tautological, and highly subjective besides.
|
| I consider the 50-line function to be "less ugly" than
| the same function butchered for "testability" into a bog
| of poor abstractions and single-use smaller functions.
| disease wrote:
| For me at least, TDD has driven me to look for more
| functional solutions to problems. It's hard to come up
| with a more testable piece of code than a pure function
| that spits out the same output every time for a given
| input.
|
| Unfortunately this approach tends to be less effective in
| languages that do not have good support for functional
| ideas like Java and C#.
| blacktriangle wrote:
| I agree some TDD advocates are way to big on loc as a
| metric for testability. Loc is pretty much irrelevant for
| both testability and readability.
| carlmr wrote:
| >TDD on functional code that is mostly exchanging values
| between systems. All the stub and mock crap goes away
| since values become the boundary of the system.
|
| That's generally why I think functional programming is
| taking off. It's much easier to reason about input-output
| of stateless functions. They're much easier to test and
| develop. They're way easier to understand for the reader,
| because the reader can understand what's happening in one
| place, instead of having to look through many different
| files, creating a complex model of all the inheritance
| relationships between classes.
| echelon wrote:
| It also seems really ill suited for new systems and places
| where you're experimenting with new languages or frameworks.
| You don't know the shape well enough to be able to test the
| inputs and outputs before writing the thing, and learning via
| TDD seems highly suboptimal.
| pydry wrote:
| Yes, it's awful for spikes.
|
| I have done a variant of TDD where the output is slightly
| uncertain where:
|
| * I build a test with everything filled in except output of
| some kind.
|
| * I write code that generates an output I want to eyeball.
|
| * I run the test in "rewrite mode" where the output is
| generated by the program and saved with the test. I eyeball
| it to check if it's ok and then commit.
|
| * In CI it checks the output against the fixed version.
|
| There's plenty of scenarios where it doesnt work or is
| generally inadvisable but I find it to be a super effective
| technique where it does.
|
| I think it's an approach that would work better for code
| that generates text, images, sounds, than common-o-garden
| TDD.
| rytor718 wrote:
| I think you have a fair point in regards to learning new
| things. YOu don't need tests to explore tools and
| languages. Tests are for development.
|
| Thats just the thing with TDD: used properly (which is
| clearly tricky for many of us), it should help you think
| through what the inputs and outputs of the program you hope
| to write are. It does assume you know what you want to
| build though, even if you're unsure of how to build it.
|
| I think of it as a design tool to be honest. When I'm not
| quite sure exactly how something should work, but I have a
| clear idea of what I want it to do, tests help me work
| through it in a way that produces extensible code.
| user-the-name wrote:
| It's not tools or languages you are exploring, you are
| exploring the problem you are trying to solve, and its
| possible solutions.
|
| If you have a problem that is well understood, well
| documented and that has a straightforward solution, TDD
| is great because you know where you are going.
|
| But a lot of the time, you have none of that. You don't
| know where you are going. You need to experiment, you
| need to iterate, you need to occasionally change
| directions completely. When you're doing that, TDD holds
| you back and gives you nothing.
| berkes wrote:
| I don't entirely agree here, though.
|
| If you are learning a new language, new paradigms or new
| frameworks, then, indeed, often testing is an extra burden
| that adds little at this point. But in this stage, I hope,
| one should be aware that the code you are writing is bad
| anyway. In a year, you'll probably be very ashamed of all
| the things you did there.
|
| But in languages or paradigms that you are already familiar
| with, TDD helps a lot when learning new things. You can
| very easily learn about a new library by writing tests that
| use the lib. Or learn a new pattern by implementing them in
| tests.
|
| For example, write some tests that wrap a Stripe library,
| then change the test to change some setting, call a method,
| or feed it some weird data, and assert some outcome rather
| than manually putting stuff in a cart, filling your
| address, CC details and then see that the setting you
| thought did X actually does Y. And repeat again.
| swiley wrote:
| Factorio convinced me that some people still write good closed
| commercial games. I wish the best for the authors and hope they
| don't stop any time soon.
___________________________________________________________________
(page generated 2021-06-18 23:02 UTC)