[HN Gopher] Documentation Driven Development (2022)
___________________________________________________________________
Documentation Driven Development (2022)
Author : yarapavan
Score : 62 points
Date : 2024-06-21 13:21 UTC (5 days ago)
(HTM) web link (unicorn-utterances.com)
(TXT) w3m dump (unicorn-utterances.com)
| onel wrote:
| I honestly believe documentation is more important than tests in
| some cases. The time spent by a dev every time to understand why
| a method was created that way adds up quickly. The what is more
| evident from the code.
|
| The idea of writing documentation before the code is interesting,
| it's thinking out loud
| kmerroll wrote:
| Wholeheartedly agree, however... It's well accepted that most
| developers would rather chew off their own arms than write
| documentation. The acceptance, sort of, of TDD shows that
| developers would rather code even in pseudo code test languages
| than write narrative prose.
| datavirtue wrote:
| I write a "functional description" that describes the
| functionality of each screen from a user's perspective. It
| also contains technical details to jumpstart devs but the
| main focus is on describing how the software works.
|
| Oddly enough, I write these in Word and keep them on
| SharePoint. This allows the team to add comments, maintaining
| the historic discussion around certain functionality.
| bunderbunder wrote:
| "During the early phase of software development the
| documentation _is_ the specification and _is_ the design... If
| the documentation is bad the design is bad. If the
| documentation does not yet exist there is as yet no design,
| only people thinking and talking about the design which is of
| some value, but not much. "
|
| "During the testing phase, with good documentation the manager
| can concentrate personnel on the mistakes in the program.
| Without good documentation every mistake, large or small, is
| analyzed by one [person] who probably made the mistake in the
| first place because [they are] the only [person] who
| understands the program area."
|
| "Following initial operations, when system improvements are in
| order, good documentation permits effective redesign, updating,
| and retrofitting in the field. If documentation does not exist,
| generally the entire existing framework of operating software
| must be junked, even for relatively modest changes."
|
| All quotes from "Managing the Development of Large Software
| Systems" by Winston Royce, 1970.
|
| All quotes perfectly match my experience working on projects
| that were developed using contemporary low-documentation agile
| development methods.
| alganet wrote:
| This discussion reminds me of literate programming. An old idea
| for a more civilized age.
|
| It also reminds me of clitest[1] (very similar name to the
| author's lib, but it is another thing).
|
| You can write examples with prose in markdown, then let clitest
| verify them for you as if they were tests. Best of both worlds.
|
| [1] https://github.com/aureliojargas/clitest
| adolph wrote:
| The author seems to conflate several different types of
| documentation, like requirements, code comments, and api
| explanations. Additionally TDD isn't intended to work in the
| absence of any documentation, the concept is that a test models
| the requirements.
|
| Maybe a paradigm to consider more is:
|
| _Literate programming (LP) tools are used to obtain two
| representations from a source file: one understandable by a
| compiler or interpreter, the "tangled" code, and another for
| viewing as formatted documentation, which is said to be "woven"
| from the literate source._
|
| https://en.wikipedia.org/wiki/Literate_programming
| smokel wrote:
| Writing out a design before starting to code is often helpful,
| especially when working in teams.
|
| A high-level design is best not written in code, but in a
| separate document. Writing documentation in tickets is just plain
| silly, IMNSHO.
|
| Writing design documentation used to be commonplace. If it helps
| to put a name on this, then that's great.
| bordercases wrote:
| Many of these arguements are moot - because for any top-down
| function definition technique, the How to Design Programs recipe
| covers virtually all the bases.
|
| - https://course.ccs.neu.edu/cs2500f18/design_recipe.html -
| https://cs110.students.cs.ubc.ca/reference/design-recipes.ht... -
| https://docs.racket-lang.org/htdf/index.html
|
| Between a signature, purpose statement and examples, you've
| declared most of what documentation provides short of a longer
| contextual statement of the functions role in a codebase.
|
| For larger modules there is How to Design Worlds.
|
| - https://world.cs.brown.edu/1/htdw-v1.pdf
| threatofrain wrote:
| I wouldn't focus on the fact that the examples are on single
| functions. Libraries and frameworks are about API design and
| not individual atoms, and writing your documentation ahead of
| time is about crystalizing the API vision you have in your
| head.
|
| One may have accidentally glossed over a critical design
| decision and this process can surface that.
| wesselbindt wrote:
| > Making sure tests pass can be challenging to handle with the
| unknowns of implementation detail. After all, if you expect
| parseInt to act one way and it behaves another, you will likely
| have to rewrite all tests that worked off that assumption.
|
| Isn't this kind of the point of TDD? TDD makes it harder to write
| tests based on assumptions about implementation details, making
| it less likely that you'll make such assumptions. You end up with
| less _unnecessary_ coupling between your tests and the
| implementation details of the code they test, which means fewer
| false failures (tests that fail because the structure changes,
| rather than the behavior), and a more robust test suite.
| bunderbunder wrote:
| For designing APIs, my favorite approach is a hybrid of the two.
| I'm not sure if there's an official name, but I've taken to
| calling them walking skeleton tests.
|
| The gist is that you start by writing example client code
| demonstrating small but useful interactions with the API,
| including explanatory comments. They won't work yet, because the
| API hasn't been implemented. But you're already in a position to
| talk them over with your team and start thinking about user
| experience issues. And they give you a nucleus around which you
| can start building high-quality user documentation.
|
| _And_ they 're your automated end-to-end acceptance tests.
|
| Once things look good (not perfect), then you can start
| implementing the API. For this portion I don't necessarily do
| TDD, because I remain unconvinced that writing unit tests first
| is inherently better than writing unit tests afterward. But I
| might. Depends on my mood. But I do keep running the walking
| skeleton tests, which serve as my primary feedback mechanism for
| evaluating implementation progress and success.
|
| The more important thing is that I can start stepping through
| those walking scenario tests and seeing how well the interaction
| actually works in practice. And I can use those observations to
| help inform an ongoing conversation about the API's design and
| functionality. I can also get quick feedback on whether some
| aspect of the high-level design isn't going to work as well as we
| had hoped. Ongoing updates to the design are memorialized and
| communicated by updating the walking skeleton tests.
| hitchstory wrote:
| I built a Python/YAML framework around the ability to build
| these tests and then generate documentation from them. E.g.
|
| https://github.com/hitchdev/hitchstory/blob/master/examples%...
|
| which generates
|
| https://github.com/hitchdev/hitchstory/blob/master/examples%...
|
| When you show this stuff to other people and gather feedback -
| that's essentially what BDD is, too.
| bbojan wrote:
| > For this portion I don't necessarily do TDD, because I remain
| unconvinced that writing unit tests first is inherently better
| than writing unit tests afterward.
|
| I would argue that TDD doesn't imply writing unite tests - it's
| about writing any tests first.
|
| If you have acceptance tests already written out, you're
| already doing TDD.
| p-o wrote:
| I really love this idea in theory, and I believe that for some
| system, specially mature ones, it may work well. I see good
| documentation as a super power; it empowers readers and motivate
| people to understand more about the system without being caught
| in the weeds of reading the source.
|
| The source has baggages, and the intent of every single function
| calls is not always evident. Writing documentation up-front can
| help direct the source, but this is a tug-of-war environment.
| Each affect the other in its own ways.
|
| And for that reason, documentation driven development can be a
| real drag. You start writing documentation with the best
| intentions, everything works great for this first release. But 2
| months down the road you need to modify something and it has a
| ripple effect on many of the things you documented. It's a non-
| negligible cost.
|
| I've been working on this open-source
| tool(https://github.com/pier-oliviert/sequencer) and I've spent a
| lot of time on the documentation. And what I described above
| happened. I wanted to make a not-too-big change, and it required
| me to rewrite 30% of the documentation. I still love the
| documentation aspect of it, but it definitively has a higher cost
| than tests, in my experience.
| lowercased wrote:
| Would be interesting to see what impact some sort of llm system
| might be able to have on this approach. Making some changes in
| one area of the documentation, then having a system indicate
| some other likely areas of impact which you might not have
| considered, seems like it would be useful.
| p-o wrote:
| I never thought of that problem this way, but it sounds very
| cool. I imagine a sort of conversation between the LLM and
| the author.
|
| - LLM: Did you mean to change A to B?
|
| - Author: Yes, but [more nuanced information]
|
| - LLM: Ah ok, here's the documentation updated: [Doc]
| kevindamm wrote:
| If the model can reflect the existing documentation that
| well, w.r.t. changes in code, you may as well use the LLM as
| the documentation by fine tuning it with boilerplate/initial
| docs and have it generate the future documentation based on a
| context of just the code.
|
| My attempts at this (DDD) involve choosing system components
| or protocol/API layer dependencies such that they can
| generate the documentation when they generate the client-lib
| and server interface code. Isn't always possible, and doesn't
| cover everything. Having a comment-doc pipeline also helps
| but sometimes that just adds more trees to occlude the
| forest. This way the documentation doesn't lead the code too
| much (I'd rather see deprecated warnings than Not Yet
| Implemented errors).
|
| Good team communication and a solid codelab for new members
| can go a really long way, too.
| kaycebasques wrote:
| This is basically a holy grail for us technical writers. I
| know quite a few people working on it.
| kaycebasques wrote:
| > it has a ripple effect on many of the things you documented
|
| Welcome to the world of technical writers :D
|
| (In other words, I'm suggesting that this is always the case
| and it's not something peculiar to DDD. I would argue that DDD
| has just made you more sensitive to what we go through every
| day. But yes, if you try to create a final doc and the features
| aren't stable, you're going to have some wasteful churn.)
| javier_e06 wrote:
| I have lived through all:
|
| Rational Rose UML RequistePro. Doors. Just plain prototype
| something through many iterations until is good then document.
| Doxygen
|
| All of the above except the prototyping a waste of resources more
| or less.
|
| If I had my software shop (which I don't) I would expect a
| software module to come with a README.md Which a brief
| explanation of what is is and no more than 3 step instruction for
| a reader to see the software module running and doing something
| useful.
| bunderbunder wrote:
| Given that the article is focusing on designing and developing
| APIs, I'd argue that, done correctly, documentation _is_ a
| prototype. It 's functionally equivalent to paper prototypes
| and low-fi mockups in UI design.
|
| The thing that makes these kinds of artifacts so nice compared
| to just diving straight into cutting code (specifically
| implementation code - see my comment elsewhere on walking
| skeletons) is that they're cheaper to build and easier to throw
| away. Which lets you iterate more rapidly in the earlier stages
| of the project, with less risk of accidentally anchoring
| yourself to decisions that get made before you have sufficient
| information to support a firm decision.
|
| That's also arguably where the various design and documentation
| tools and techniques you listed fail. I haven't used all of
| them in anger, but at least by appearances they're all
| relatively heavyweight and expensive things that are generally
| ill-suited to lightweight early planning in anything short of a
| situation where you're dealing with a CMMI requirement.
| alganet wrote:
| > Just plain prototype something through many iterations until
| is good then document
|
| This is always the best. Iterative prototyping gives you
| continuous hindsight over design and implementation, _nothing_
| beats that.
|
| Unless you're doing something trivial in which hindsight would
| only serve as temptation for yak-shaving. Then it's not so
| good. Unless you don't have deadlines, then it's good again.
| kaycebasques wrote:
| We are trying out some DDD for an upcoming pigweed.dev tutorial.
| I'm docs lead for the site and I'm working with 5 or so
| engineers. It's going decently. As a technical writer, it's hard
| to flesh out exact content until the features are built out.
| Especially in a tutorial, where each section builds off the ones
| that came before it. So we have a lot of placeholders in the
| tutorial. All-in-all I think it's been fairly helpful for me to
| repeatedly say: "This will be our main artifact. This is where
| people will learn of your features. Please make sure your work
| and features are captured in this document, otherwise they kinda
| don't exist (because this doc is where people will most likely
| learn of the new features)." I don't ask them to fully flesh out
| docs themselves; I'm just asking them to dump notes into the
| draft doc and insert commands/code as things become fleshed out.
| I think it's also a useful focusing exercise for the team -
| there's so much we could work on but if it's not going into our
| primary documentation artifact and it's not foundational work
| that is prerequisite to what we want to show in the tutorial then
| that sometimes makes it easier to decide that the thing is lower
| priority. There was also an interesting feedback cycle where the
| team worked on some really important stuff that is usually kinda
| "behind-the-scenes" and low-level, and we asked ourselves "we
| have to work this into the doc and make people aware of it... how
| do we do that?"
| gmd63 wrote:
| Everything is already natural language driven development, but
| management/execs often like to avoid accountability by prompting
| their human LLMs verbally off record.
| liampulles wrote:
| I think a lot of XYZ Driven Development advice can basically be
| boiled down to: write down some kind of plan before you start
| coding.
|
| Certainly one should think about the design before implementing,
| but I think writing acts to force active reflection on the design
| - be it a plan, or documentation, or a test.
|
| At this point my approach is to plan out my current ticket into
| TODOs before I make a branch. I do this in phases until I get
| down to the level of what functions need to be modified/made. If
| I know there is a function that is going to be doing a bunch of
| branching or is otherwise complex, then I know it will need a
| good bit of unit testing, and I might opt for TDD - though its
| rare. I'm used enough to planning my tickets now that it is
| usually sufficient to get to a good design.
|
| I would recommend trying to do some sort of planning of your
| work. It can be hard and time consuming it first but it can be a
| superpower if you get into the habit of it.
| swah wrote:
| Planning upfront like this inevitably leads to better design
| and less rework. I'll see that steps conflict and rewrite the
| plan before rewriting the code.
|
| from https://liampulles.com/jira-tickets.html
| diffxx wrote:
| Writing documentation up front is a good idea, but only if one
| treats it as the first iteration of the program with the
| expectation that the initial documentation may well be thrown
| out. The process of writing the documentation no doubt will help
| you organize the plan for writing the code. But there will also
| be things that the documentation does not capture or problems it
| does not foresee that emerge as the program is being written.
|
| The danger is that either you fail to update the documentation to
| account for the changes in the system that emerge in development
| or you only update parts of the documentation as you go which
| causes the documentation to become inconsistent and unreliable.
|
| A middle ground is write the documentation up front and then
| rewrite it after the system is done. The initial draft helps
| guide the design and the final version captures the full and
| complete essence of the finished program, which is nearly
| impossible to do up front.
| charlie0 wrote:
| This lines up perfectly with my use of the documentation. It
| really helps quite a lot to do it first, but as the
| implementation is underway, new scenarios come up and it's
| often a challenge to go back and update the docs. The new
| things also aren't something that derails the entire design. In
| the end, the docs are usually thrown away or transmuted into
| something useful for other departments.
| bunderbunder wrote:
| Another nice benefit of doing it this way is that it allows the
| two sets of documentation to follow different formats and
| levels of detail. Which can ultimately be less total effort
| than trying to engineer a functional multitool.
| 6510 wrote:
| > Writing documentation up front is a good idea, but only if
| one treats it as the first iteration of the program with the
| expectation that the initial documentation may well be thrown
| out. The process of writing the documentation no doubt will
| help you organize the plan for writing the code. But there will
| also be things that the documentation does not capture or
| problems it does not foresee that emerge as the program is
| being written.
|
| Everyone else is building all kinds of things from technical
| drawings. We could identify and describe what is so different
| about writing code which would make it clear what not to
| document or we can just suck it up, stop pretending our
| problems are special and closely mimic existing methods :)
|
| Making a construction drawing is hard but people are some how
| doing it, they make them for mind blowing size projects. Things
| definitely go wrong but no one says it is just the way things
| are. If changes are made there shall be new drawings.
|
| However, when it comes to renovation or repairs the process is
| often the opposite. Unless it is a boat, there if every brush
| of paint is meticulously documented it adds value in $$$.
|
| A friend told a funny story about construction workers building
| a large building across the street while they were debugging
| their code. First he joked about construction workers doing
| real work, then about how structured everything was and
| eventually about the building being finished while they still
| had countless bugs.
| Swizec wrote:
| > Things definitely go wrong but no one says it is just the
| way things are. If changes are made there shall be new
| drawings.
|
| > then about how structured everything was and eventually
| about the building being finished while they still had
| countless bugs
|
| There is a famous story of the Hyatt Walkway Collapse where a
| change was made, was improperly documented, and caused a
| production bug that led to many deaths. Extreme example but
| production bugs in physical construction are extremely
| common.
|
| https://en.wikipedia.org/wiki/Hyatt_Regency_walkway_collapse
|
| edit:
|
| Here's a whole paper looking into the effects of design
| errors in construction projects. This implies they're common
| enough to study. https://www.researchgate.net/publication/330
| 701193_EFFECTS_O...
|
| And here's some research that showed 10% to 25% of cost in
| construction goes towards fixing errors.
| https://getitright.uk.com/reports/call-to-action
| 4star3star wrote:
| The questionable reliability of documentation is the big
| problem, for sure. It would be nice to be able to collect
| metadata such as referral dates and even comments. If you visit
| the docs, tick a box that says, "I referred to this and it was
| accurate," or, "This needs to be updated," etc.
| delusional wrote:
| >but only if one treats it as the first iteration of the
| program with the expectation that the initial documentation may
| well be thrown out.
|
| This point really hints at there being some other facotr at
| play. I agree that writing documentation first is fine, but in
| the same way writing some bad code is fine.
|
| I'd argue that the most important thing you can do to increase
| your chance of success is getting to the details as fast as
| possible. Do everything you can to get out of nebulous "big
| idea" mode and get into "we have this, what exactly are we
| changing about it" mode.
|
| If you write documentation or code doesn't matter.
| euroderf wrote:
| So why can't I use documentation as a requirements spec and just
| feed it to a LLM for implementation ? With lots of examples in
| the documentation that can serve as test cases for the generated
| code.
| bunderbunder wrote:
| The same reason succeeding with outsourced development is so
| dang hard: it's a monkey's paw.
| euroderf wrote:
| But wouldn't you agree that LLLM's will improve a lot faster
| than outsourcing does ?
| williamdclt wrote:
| that's still to be proven. At the moment it can't do what
| you describe for any meaningful piece of functionality,
| there's no certainty that it will be able to anytime soon
| bunderbunder wrote:
| No.
|
| The problem is fundamental: you need a certain level of
| mind meld to be able to get away with leaving things
| unstated and assuming the other person will know what you
| mean, or can figure out the right thing to do for themself.
| When you don't have that, you have to compensate by making
| your explanation of what you want as specific and precise
| as possible. Which, just like in the story _The Monkey 's
| Paw_, turns out to be infeasible because of the fundamental
| sloppiness of natural language as a communication medium.
|
| The level of mind meld you need to make natural language
| communication behave well comes out of domain expertise and
| a certain amount of time spent working together and getting
| to know each other. Outsource developers and LLMs both lack
| it for reasons that are admittedly different, but
| comparably unfixable.
| datadrivenangel wrote:
| If you're willing to write a very detailed requirements spec
| you can just... write ruby or python code in a framework and
| have a working application...
| siddhantx0 wrote:
| bullshit
| haolez wrote:
| That's mostly how Knuth codes (with a flavor twist in the form of
| literate programming). It's a huge endorsement of the idea
| presented in the article :)
| dimal wrote:
| I think that the usefulness of any "X driven development"
| methodology is so highly variable based on the particular problem
| being solved and the particular brain solving it, that it's
| impossible to make any general recommendation for anything. Like
| everything else in programming, it depends. Documentation driven
| development might be great for one person, but for another person
| it could be a tedious waste of time. Everyone should feel free to
| find their own way. If it works for you, good. If not, let other
| people do what works for them.
___________________________________________________________________
(page generated 2024-06-26 23:01 UTC)