[HN Gopher] Planning and estimating large-scale software projects
___________________________________________________________________
Planning and estimating large-scale software projects
Author : kalleth
Score : 211 points
Date : 2021-07-21 14:38 UTC (8 hours ago)
(HTM) web link (tomrussell.co.uk)
(TXT) w3m dump (tomrussell.co.uk)
| ngrilly wrote:
| > It also tells us how many team-weeks this fictional, idealised
| project would require [...] by adding all the estimates together.
|
| I would be wary with just "adding all the estimates together".
| That's because we tend to estimate the median or the mode of the
| task duration, and not the average. Means can be added together,
| but not medians.
| marcosdumay wrote:
| > Means can be added together
|
| Is the error distribution of task size estimations normally
| distributed? Because I do really expect it to have a fat tail,
| and if it does, you can't add means either.
| jdlshore wrote:
| There's a variety of analyses out there and they very
| consistently show a log-normal distribution for release
| predictions. I've analyzed Star Citizen's publicly available
| data and found the same for their task estimates. It's very
| reliable.
|
| You do see truncated log-normals, though, when the estimates
| are padded.
| ngrilly wrote:
| I think most of us in software engineering assume the
| probability distribution has a fat tail. I've seen some
| authors name this the "blowup factor". For instance, your
| most likely estimation is 10 days, best case is 5 days, and
| worst case is 30 days. I think adding means is still
| meaningful (see central limit theorem and law of large
| numbers).
| marcosdumay wrote:
| It's exactly the central limit theorem that breaks for fat
| tailed distribution.
|
| Some fat tail distributions also break the law of large
| numbers, but I don't think task size estimation is this
| flawed.
| antondd wrote:
| This brings back memories from the days of my early career (an
| ex-PM survivor here). I would be curious to see some data, even
| anecdotal, on the success of this approach. Here's some
| interesting statistics from the industry (not specific to
| software, but you can extrapolate): http://apepm.co.uk/project-
| management-statistics/
|
| In my view, traditional software project management is
| ineffective. I would put it somewhere between the Myers-Briggs
| personality test and modern day astrology.
| kalleth wrote:
| Insightful, thank you! The entire project management industry
| doesn't have that great a "hit rate" -- consider the budget
| overruns for the last few Olympics, or for Crossrail.
|
| I'm just not sure why software projects are "special" -- if you
| can avoid it being a project and instead make it ongoing OpEx
| like, for example, GDS managed for the UK in 2016, then great,
| you've sidestepped that, but until the entire PM industry
| discovers how to improve overall project management techniques,
| I don't see why we'd consider our industry "above" them.
| RogerL wrote:
| It's not special, except that for whatever reason people
| insist on treating random data as gospel.
|
| How long until we cure retinoblastoma? Everyone understands
| there is no way to produce a meaningful timeline. There are
| too many inter-related unknowns - various causes, various
| treatment modalities, varying funding on various fundamental
| and applied research, no real idea if the final answer is
| gene therapy, nano-something, chemo, etc.
|
| I used to develop software for a defense contractor, and it
| was pretty waterfall-y. But we built risks in, to an extent.
| Not by multiplying Sally by 1.3x, Joe by 2.7x or whatever,
| but you'd chart it all out, showing interconnections (x
| depends on Y, which depends on Z and Q, which...). And then
| roughly figure out risks of each of those sub-tasks going
| long.
|
| The idea NOT being you then just multiply by a weight, and
| ta-da, you have an accurate schedule. The idea is that you
| have now identified particularly risky chain of events, and
| now you at least have a chance of managing risk. Every
| day/week you'd have a risk assessment meeting. Where are we
| on X, Y, Z. What can we do to get X back on track? Can't,
| okay, can we we-jigger the dependencies, or is this a hard
| slip? And so on. I've never seen this done on the commercial
| side, and it just seems like people are flying blind as a
| result.
|
| "Waterfall is terrible" you reply. Sure. But when you are
| building an airplane, ya kinda need the 1553B/ARINC bus
| installed before you install and test the avionics. You can't
| attach the engines if the wings haven't shown up. You can't
| redesign the fuselage after the wing builder started building
| the wings (in general). These are hard, unavoidable
| dependencies, and changes are often extremely to
| destructively expensive (hence the endless mind numbing
| meetings arguing about change control).
|
| It is just (IMO) not an unsolved problem, but unsolvable. Too
| many unknowns results in unpredictability. Your only bet is
| to manage the risks, adjust as necessary, and accept some
| things are just unknowable. Agile does that in one way,
| sophisticated waterfall in another.
| some_chap wrote:
| I'd say that software is "special" because it's ephemeral,
| meaning there's incredibly few limitations on the
| possibilities (or changes to requirements mid-project) when
| compared to projects involving physical items. It /can/ be
| managed like a physical engineering project, but the cost and
| time ramp up so severely that it's not practical for most
| situations.
| tonyedgecombe wrote:
| You can't really do that with the Olympics though as there is
| a hard deadline. Once you have a fixed date then either
| quality or cost are going to have to give.
| marcus_holmes wrote:
| > instead make it ongoing OpEx
|
| Every non-tech startup founder who's approached me with "how
| long will it take to build an MVP for my startup idea?", I've
| answered with this. Development is an ongoing cost, a
| process, not a once-off capex cost.
|
| I recommend to them going the other way. Start with "how much
| can you afford to pay a dev team sustainably?" then work out
| how many devs that works out to, then work out how long your
| MVP will take to build based on their estimates (and
| estimates are not deadlines).
|
| Not quite the same as #no_estimates (which I also try to
| argue for whenever possible), but close.
| mumblemumble wrote:
| > you should expect to be held accountable if your estimates are
| too far off the mark because you failed to do your due diligence
| when coming up with them.
|
| In a perfect world, I agree.
|
| In the real world, which has a remarkable knack for failing to
| live up to expectations, what I find is that companies are rarely
| willing to allow the development team adequate time to do their
| due diligence. Answers, in and of themselves, are cheap. I can
| give you those all day. If you want to be able to hold me
| accountable for their accuracy, though, then you need to be
| looking at my _correct_ answer rate sheet.
|
| For me, the magic of #noestimates is the magic of open, honest
| cynicism. If my realistic options are silence and blowing smoke
| up my boss's ass, I'd really prefer it if they would allow me to
| choose silence. That way we can both keep our dignity.
| kalleth wrote:
| Author here. I've been lucky enough in my career to hold some
| senior positions, and I thought I'd give a step-by-step on an
| approach I took with an "enterprise-scale" software project, and
| how I stole some techniques from university project management
| courses to meet with some success. Happy to answer any questions
| :)
| justin_oaks wrote:
| How are deadline dates assigned? Is the deadline exactly the
| same as the estimated completion date?
|
| Realistic estimates aren't padded, but they still have
| significant probability of being inaccurate. After all, they're
| estimates, not information from the future transmitted to the
| past.
| kalleth wrote:
| This is where the ideal meets the annoying reality of The
| Enterprise (tm).
|
| I can't talk in too much detail, but in general, the
| _deadline_ date was fixed through commercial contracts signed
| at a high enough level that engineering didn 't have sight of
| them. The concept and commercial case was sound, but the
| implementation hadn't been worked out yet, when a date was
| set.
|
| My strong preference would be for estimation to come first,
| of course, before a deadline is picked (and even then, only
| picked if it is _really_ a necessary deadline), which is then
| based on reality, and also include some slack for unintended
| discoveries.
| PUSH_AX wrote:
| Our paths crossed on the engineering team of a certain
| letterbox flower company, for a short time at least. Good to
| see you doing well Tom, nice article.
| kalleth wrote:
| Thanks! And I couldn't possibly comment on which company that
| could be...
|
| Hope you're doing well too!
| mzarate06 wrote:
| I'm curious that you mention project management courses from
| _university_. How much benefit have you found them to provide
| in practice?
|
| Asking b/c I've taken two courses in dev process or project
| management in my academic career, and neither provided
| substantial value or benefit to how I've lead projects
| professionally.
| kalleth wrote:
| In the early stages of my career, or in startup life?
| Absolutely not. Very little relevance; XP/SCRUM were both
| covered together in a single 50 minute lecture, the rest of
| the PM aspects were tackling paperwork-generation
| methodologies like the "Rational Unified Process" and
| "Dynamic Systems Development Model", both of which I feel
| like would be _hell_ if I actually had to work within.
|
| However, there _were_ techniques (like critical path
| analysis) that as I 've got more senior, started working at
| larger companies, and started stepping down the senior
| leadership path, I've started to see some applicability to.
| Not direct applications - they still need taking with a
| massive pinch of salt, and modifying for modern learnings in
| industry, but they do start to provide _some_ value, even if
| it 's just learning what the grey-hairs in the exec are used
| to seeing :)
| pphysch wrote:
| What tools and methods do you use for creating, sharing, and
| modifying project charts?
| kalleth wrote:
| Depends how much of a perfectionist I'm feeling. For the
| initial development, a sharpie, index cards, and a whiteboard
| wall - or Lucidchart, because it's basically an in-browser
| whiteboard/drawing tool.
|
| Once the project is ongoing and you'll need to account for
| changes, I've either done it manually (which takes an age) or
| handed it off to PM's to oversee using either MS Project,
| airtable with a custom-authored set of actions/etc, or
| PrimaVera.
| ksec wrote:
| Do you think there is a different in how US and UK tech
| companies in project planning?
| kalleth wrote:
| I've never worked for a US tech company, so I wouldn't know,
| sorry!
| pphysch wrote:
| There are arguably two major phases here, and its only the second
| one that most folks here find controversial.
|
| * Steps 0-2: determining "what" the project "is" (design,
| architecture, ontology)
|
| * Steps 3-6: procuring and allocating resources to complete it
| (economics, management, politics)
|
| It is tempting to decouple the two phases, and as a technologist
| focusing solely on architecture while leaving the economics up to
| leadership. However, social factors (the real people involved
| with the project) are an integral part of actually getting
| anything _done_ , so I agree with the author's premise that the
| whole process should be viewed holistically (and ideally run by
| one technologist).
| kalleth wrote:
| Well phrased, thank you! I didn't think of it in this way, but
| yes, that kind of phasing makes sense.
| lifeisstillgood wrote:
| - Projects only become official and tracked once someone has
| hacked together enough of a prototype to prove it works.
|
| - at this point all project management is pretending it takes 100
| managers to land something one girl / guy got flying.
|
| - stop project managing, stop estimating, and just start treating
| companies as VC firms. Hire good devs, make them care about your
| mission, invest in those that take off. Don't take the control
| away from the original devs
| henning wrote:
| Bullshit. You can make guesses about the future, but since that
| is going to inevitably change, it's a guess and can never be
| anything but a guess. At most companies, the scope is going to be
| radically expanded and looking too far in the future is a
| complete waste of time.
|
| Other people in other fields are held accountable for deadlines
| because their work does not completely change and is not severely
| under-specified. If it is, then they are also just guessing.
| crazy1van wrote:
| > it's a guess and can never be anything but a guess
|
| Of course it's a guess. The question is how to make more
| accurate guesses.
| dragonwriter wrote:
| > The question is how to make more accurate guesses.
|
| I think the question should be about how to maximize the net
| return on time dedicated to development, which may mean
| spending less time on (and investing less reliance on)
| estimates rather than expending unbounded effort improving
| the quality of estimates.
| satyrnein wrote:
| What's the value to Sales/Marketing/etc of knowing what's
| coming when? How do you estimate that? It's estimates all
| the way down!
| NikolaNovak wrote:
| I think author addressed that explicitly as valid and true
| software engineer perspective, right at beginning of the
| article; and explained that such software engineers, and
| products they build, then usually slot into larger company's
| ecosystem (as by strict math, most IT team members will work at
| a large company as opposed to a startup), full of people and
| leaders and departments and teams and project who have plans
| and deadlines and dependencies, which are _also_ valid and
| true.
|
| I've had either luck or misfortune to "flip a switch" from two
| decades of being a techie/architect, to being a mid-manager, on
| basically a specific date as opposed to over the years, due to
| project's needs; and it's like that B&W picture of two faces
| and one vase in middle. Both perspectives are true, even if
| opposing and contradictory.
|
| Good business lead will understand software engineer's
| perspective - even if it's not their primary view of the world,
| they can squint and catch a glimpse as needed. Likewise for
| good software leads.
|
| Obstinate unwillingness to see or ascribe merit to other
| perspectives puts a ceiling on everybody's progress.
| hnbad wrote:
| I have to admit I popped a monocle when I saw the estimate for
| what is described as a rudimentary ecommerce website without even
| basic features like authentication and payment come down to 43
| team-weeks, or in other words 12,040 team-hours. Even at one
| person per team being billed as $20/hr this works out to almost a
| quarter million dollars.
|
| I understand the numbers are given as an example but I think the
| problem is that the scale of the task hardly justifies the
| complexity of the planning and using "teams" and "weeks" as sizes
| instead of "developers" and "hours/days".
| z3t4 wrote:
| The thing with software is that it takes two weeks to get to
| 90% finished, then two years to get from 90% to something 100%
| production ready. Having worked over 5 years with ecommerce
| software I could write a fully working e-commerce site in two
| weeks, but then the marketing teem want to have it all
| integrated with 50 different tracking systems, and support 10
| different payment systems in 30 countries, and have the correct
| VAT handling as well comply to local laws, integrated shipping
| systems, and of course 30 different versions of the site in all
| languages. And integrated with business software. And every
| moth the marketing team want to start a new compaign, buy 3 and
| get 20% off, buy only milk and get 30% off, buy a red shirt and
| get 25% off, discount codes, discount codes that includes
| shipping, etc. Did you know that different products have
| different VAT in different countries and within the same
| country as well? And shipping should use the average VAT from
| the products ordered - depending on country...
| rabuse wrote:
| This has been the case for almost every project I've worked
| on. It starts out as super simple, and then the edge cases
| roll in, and then the feature creep, and then the bugs start
| piling up, and now it's been years on a "just a couple
| months" estimation.
| dboreham wrote:
| Instead, we as an industry prefer to adopt magical thinking
| where we pretend that software projects aren't hard and lengthy
| and risky (even when the domain and tech are well understood),
| then we act surprised when the project "overruns".
| hnbad wrote:
| I'm not saying that this isn't a good estimate for "doing it
| properly". I'm just saying that no client will let me bill
| half a million dollars and spend the better part of a year to
| build them a bare bones ecommerce website from scratch that
| doesn't even have invoicing or an admin interface.
|
| I'm mostly just saying if the author is able to sell what
| amounts to half a broken Woocommerce installation for half a
| million dollars (assuming it's at least a team of two billed
| at $20/hr), I must be in the wrong market.
| mdtusz wrote:
| This is extremely difficult to convey to non-developers
| though because they often look at software as a house and
| think "there's tons of houses (ecommerce sites) built every
| day and it's down to a science - how hard can a house
| (ecommerce site) really be", but miss out on the fact that
| all houses have pretty much the same structure of a
| foundation, framing, wiring and plumbing, and a covering of
| paint and a roof - this is largely kept consistent and we'll
| implemented by building codes.
|
| I can only speak from my own experience, but every single web
| application I've worked on has had a wildly different
| structure than the others and the only consistent thing
| between them has been endpoint routing mechanisms.
| rmah wrote:
| Then this is, IMO, a problem with the software developers
| (in the broadest sense, not just programmers). Most
| e-commerce in fact, DO share the same framing, wiring and
| plumbing as all the others. This is why you can buy an
| e-commerce package off-the-shelf and customize it.
|
| If the business is a relatively generic e-commerce _store_
| , it should usually not be building bespoke e-commerce
| _software_. Unless, of course, there is some technology
| feature that will be your competitive advantage
| /differentiator. But let's be honest, that's pretty damn
| rare in the space of e-commerce stores.
| commandlinefan wrote:
| > I popped a monocle
|
| This is why I don't bother with estimates any more - not
| because I think it's impossible, or even necessarily too hard,
| but because I've observed (consistently over a 30 year career)
| that it's pointless. Even if you could estimate with perfect
| precision exactly how long a software task was going to take,
| they would just push back, say, "that's too long" and argue
| with you until you told them what they wanted to hear.
| marcus_holmes wrote:
| Early in my career I had this. I was given a task and a
| deadline, and told to prepare a plan. I estimated the plan,
| and it came out to longer than the deadline. I presented
| this, only to be told "but that's longer than the deadline!
| go fix it!". So I shortened all the estimates and it fit the
| deadline. When I presented this I was shouted at for
| shortening all the estimates, and told to go back and do it
| properly.
|
| Around this point the penny finally dropped that software
| estimation is a political process, not a technical process.
| NikolaNovak wrote:
| Unfortunate; most of us have had experienced such
| situations separately, it sucks when they come together in
| a "rock and hard place" situation.
|
| Good managers / business leads/ execs / champions CAN be
| reasoned with, as long as you find common language, think
| and understand their priorities, provide alternatives that
| meet their underlying goals (all of which frequently falls
| on the presenters). E.g. in your situation, it _may_ be
| that unspoken expectation was to cut scope or increase
| resource contour or find another way to meet deadline
| rather than just changing estimates; or something
| completely different.
|
| Occasionally though, you're as you say stuck between other
| people's indecipherable politics. I find in such
| situations, I'm most comfortable speaking the most honest
| truth and working hard, openly and explicitly, to
| understand/ask/bring to surface everybody's actual critical
| goals.
| marcus_holmes wrote:
| > understand/ask/bring to surface everybody's actual
| critical goals
|
| This, exactly. Now I'd be working out what the actual
| objective is and how I can achieve that.
| commandlinefan wrote:
| > CAN be reasoned with
|
| Sort of. What you really end up doing is making enemies
| and burning bridges to defend a software estimate that
| ends up being completely unrealistic anyway. Sure you can
| "win" an argument with a "stakeholder" if you fight hard
| enough, but you'll pay for it later. They want to hear
| what they want to hear.
| kalleth wrote:
| That's really tough, and I'm sorry you had to deal with it.
|
| I was lucky in that I was dealing (in both cases where I've
| run similar flows to this) with above-board exec teams who
| wanted the best quality information I could give them -
| even assuming that estimates are just assumptions - even if
| it meant having some tough conversations about scope or
| headcount.
| marcus_holmes wrote:
| Like I said, it was early in my career. I now know how to
| handle this - ask more questions and work out what the
| real objective is. Also refuse to let people shout at me
| ;)
| kalleth wrote:
| I think I would have popped a monocle too, had that been the
| real example and estimate from a team!
|
| I tried to think up an accessible example that didn't require
| too much context on the part of the reader, so obviously, as
| you correctly point out, all the numbers are made up, and I'm
| trying to use it solely to demonstrate the workflow :)
| commandlinefan wrote:
| > all the numbers are made up
|
| Well... have you actually applied this process successfully?
| If so, wouldn't you have some actual numbers to point to from
| a past project? Names and details changed a bit to protect
| the innocent, of course.
| kalleth wrote:
| I have, yes - or I'd feel a bit like a charlatan writing
| about it!
|
| The problem with the real world examples is the business
| domain, which was hyper complex and the specific "pieces"
| of work I described wouldn't have been easy to grasp for
| most not familiar with the esoteric side of fintech that
| the project took place in.
|
| So I went with a simpler, albeit contrived and more
| accessible example.
| commandlinefan wrote:
| Were they monocle-popping results?
| christophergs wrote:
| Great post. A key point you don't bring up is the aftermath, even
| if you do deliver. Especially in non-tech companies there still
| remains the tendency to view these projects as "done" after the
| end of the project/MVP etc., with no understanding that sites
| need ongoing maintenance and improvements. And that this work is
| still considerable.
| noptd wrote:
| Agreed, especially for MVPs and "phase 1" projects.
|
| At my current company, it's reached a point where we flat out
| reject product proposals for features or changes that would
| need to be hacked together for an MVP without a time commitment
| from all necessary stakeholders on how it will properly be
| implemented for phase two (iff phase one is a success). It's
| amazing how quickly "critical" features become irrelevant to
| product when they understand even half the amount of work
| required to properly implement them.
| gonzo41 wrote:
| That reminds me of that scene in the series Chernobyl where the
| main scientist briefs everyone on the cleanup effort and ends
| with something like "The first battle is won and now begins the
| long war" with everyone being suitably cold in their response.
|
| I get the same response sometimes when I talk about the long
| tail of maintenance at work.
| christophergs wrote:
| Haha, love that show. There's definitely a parody in there
| somewhere
|
| > Now that I know what software estimation is, I no longer
| need you.
| galaxyLogic wrote:
| Doesn't this assume you have the "spec" before you start
| implementing it? But then how do you estimate how long it will
| take to come up with that spec?
|
| If you have a very good detailed spec you have already done much
| of to work to make the implementation easy.
|
| If the spec is high-level and "fuzzy" it leaves the work of
| "resolving the spec" to the programmer.
|
| So trying to estimate the time it takes to code a system depends
| on the quality of the spec, and therefore is difficult if there
| is no standard on how detailed the spec is to be.
| kalleth wrote:
| If you don't have the spec, then you likely don't have a
| deadline, or a "project", really -- and something like this
| would be the wrong choice for an approach to follow.
|
| I'd say leaving the overall _roadmap_ (which is all this
| produces, at the end of the day, if you ignore the estimation
| piece) fuzzy and allowing the _team_ to work that out with
| users /subject matter experts is the right approach, imo.
| agentultra wrote:
| This is a great breakdown of your process. I've seen many quite
| like it and have also been asked to make these kinds of estimates
| myself on projects for many of the same reasons: someone made a
| promise to another person, signed a contract, planned a release
| date for something they just made up, etc.
|
| What I find frustrating about the whole situation is that no
| matter what process you use for making these estimates you have a
| roughly 30-some-odd percent chance of being right. It almost
| never has anything to do with the process you used when it does
| go well. If it did estimating software projects would be trivial,
| wouldn't it? Everyone would use this process and we wouldn't have
| 60-some-odd-percent of large enterprise software projects going
| over time and budget.
|
| In reality people have used this very process, I'm sure, and have
| been in the 60-some-odd-precent. People have been studying this
| phenomenon since before I was a nerdy kid hacking on my Amiga.
|
| Having a roadmap or a _plan_ to get from A to B is good. It will
| need to be readjusted as you explore the problem space and
| navigate the waters so to speak. But the only real guarantee we
| can make as engineers is that we 'll make progress in my
| experience. I'm only giving really rough estimates in the
| beginning and those estimates improve as we get closer to our end
| goal. I only start talking about actual release dates when we get
| close to being finished and are mostly polishing out the rough
| corners and have already done a few iterations internally.
|
| If someone makes a promise they can't keep or have no business
| making -- in my books -- that's their mistake and they've made it
| a problem for everyone else.
| AnimalMuppet wrote:
| Yeah. I'm a big fan of _making the person who made the promise
| fix the mess_. You dreamed up something and promised it to the
| customer? _You_ get to go back to the customer and eat the
| crow. Maybe that will teach you not to do it next time.
|
| Anything else is just enabling (or even rewarding) bad
| behavior. If you do, expect to get more of it.
|
| Note well: I have never been a manager, especially not an
| upper-level manager over both sales and engineering. I don't
| know how well my recommendation will fly in the real world.
| (Hey, I guess that makes _me_ the guy who just sold something
| without knowing if it can work...)
| RandomLensman wrote:
| Estimate is but one part. The other is planning for obstacles
| and changes, i.e. have escalation paths and responsibilities in
| place:
|
| Clear procedures to remove impediments, decision makers in the
| loop to ok scope/feature changes, resourcing agreed upfront,
| user engagement for testing locked-down, senior leadership
| aligned and kept informed regularly. Someone running the
| administrative side of the project, keeping people on target,
| etc. (could be a double hat, of course).
|
| Sounds all rather "menial", but high degrees of organization
| really make a difference in delivery on larger projects.
| dpweb wrote:
| I found it really fluctuates based on the team members. We
| definitely had 3x 4x differences in productivity between the
| worst and best on the team. Our estimates had to be good as it
| determined the size of the sales deal. Most everyone on the team
| was there for years, so we knew each other well. We could give a
| solid estimate but with a new team member is more challenging.
| [deleted]
| dls2016 wrote:
| Project management is hard. I wrote and won an SBIR award this
| year. A much different scale than the article, but I spent a lot
| of time writing the plan and budget and sourcing components and
| estimating software tasks. Two months in and a big chunk of that
| is out the window... haha. Finding connectors and other
| components has been a big source of pain, especially if
| purchasing in small quantities. Another example: I bought a
| consumable product which then immediately became unavailable...
| so do I try to make do with what I have on hand or make the
| decision to switch to a replacement?
|
| Stressful, but I try to have fun. And extremely satisfying when
| things work out!
| 28304283409234 wrote:
| I've started estimations by their true name: Assumptions.
| heresie-dabord wrote:
| For those who are new to software development, the following
| terminology may prove helpful:
|
| Software Project Estimumption (or Assumptimation, the
| professional community is split)
|
| Software Requirements GatherWhims
|
| Software Requirements Analysthetics
|
| Unit Test Coveroverage
| ColinHayhurst wrote:
| > Estimates are one of the hardest parts of software development.
|
| And also fundamental. When I was directly estimating big software
| projects the key, for me, was to trust developers recommendations
| but apply a different multiplier for each developer. Multipliers
| ranged from x1 to x3. Those rare devs with x1 were, of course, a
| blessing. And those with x3 were not necessarily bad; they were
| often the ones working on the really hard problems. Of course, it
| meant getting to know those developers and a prior (which we set
| at x2, for new starters).
|
| Individuals were remarkably consistent in terms of their actual
| performance; so a x1 developer would almost always be a x1; a
| x1.5 would almost always be x1.5.
| eikenberry wrote:
| Did you ever come across a person who was < x1, like a x0.5
| (they overestimated)? I know developers generally undershoot,
| which is why we joke about making your best estimate and
| multiplying it by 2-3. But does no one really ever reliably
| overestimates?
|
| If the answer is never or rarely then that lends a reasonable
| amount of credence to the idea of just multiplying your
| estimates. Because you most likely should and the chance of
| overestimating is low.
| ColinHayhurst wrote:
| Not in my experience
| ChrisMarshallNY wrote:
| Good management advice. I was a first-line manager for a
| Japanese company ( _hard_ estimates, with lots of international
| frowny-faces, if the estimate was missed). That was pretty much
| what I did.
|
| We usually came relatively close to our estimates, but would
| sometimes encounter anomalous situations that would force us to
| re-evaluate. I found that my managers were pretty good at
| accepting these, as long as they were not too frequent, and as
| long as I didn't do the "Lucy in the Chocolate Factory" thing.
| They set hard plans, and we were not to deviate from them.
|
| But our customers often did not like the software that we
| delivered on-time, and to-spec. Since we were a hardware
| company, it was considered OK, but we hated it.
|
| So we delivered yesterday's technology; tomorrow. :(
|
| But I also ran a small team of _really_ high-functioning C++
| engineers. They stayed with me for _decades_ , and I got to
| know them quite well.
|
| I can't even imagine running one of these shops with half a
| million inexperienced engineers that flow in and out like
| guppies in a pond. My management experience would be worthless
| for that environment.
|
| Since leaving, I have taken a very different tack with the
| software that I write.
|
| I assume that my estimates are basically fiction. I am going to
| make big changes, to suit changes in the deployment
| environment, customer expectations, competitive landscape,
| platform tech, etc.
|
| I tend to write very flexible software, in layers and modules.
| It allows me to react to these changes fairly quickly, and to
| maintain my very high quality standards, throughout.
|
| I often toss out huge gobs of code, when I hit one of these
| challenges. That actually makes me happy. The code I don't
| write, is the best code of all.
|
| Flexible software is a very double-edged sword. Most quality
| processes advise against it; for good reason.
|
| But I tend to know what I'm doing. I've been at this a long
| time, and have been very humbled by many, many mistakes. I
| would probably have a difficult time trusting developers with
| lesser experience to take the kinds of chances that I take.
|
| So my personal process depends heavily on me being me. I know
| myself, fairly well; warts and all. This means that I can trust
| the Principal engineer on my projects.
|
| I manage scope. My projects are small-batch, artisanal,
| projects. I don't bite off more than I can chew. That said, my
| "small batches" are bigger than a lot of folks' personal
| projects. I often spend months, writing my code, so change
| comes almost invariably, during my projects.
|
| Works for me. YMMV.
| crooked-v wrote:
| I'm reminded of the classic saying: "The first 95% of the work
| takes 95% of the time, and the last 5% of the work takes the
| other 95% of the time."
| ragebol wrote:
| How much learning time did you typically need to arrive at that
| multiplier?
| ColinHayhurst wrote:
| At the end of two completed projects from each new starter
| mason55 wrote:
| One thing I've found to be helpful is to make it clear that _I
| won't be mad about a long estimate_
|
| If that's how long it's going to take then that's how long it's
| going to take. I think there's a group of people who are used
| to getting lots of pushback on their estimates and so they
| estimate low to avoid that conflict. The future conflict of
| things being late is not something they have to deal with right
| now and maybe they actually will get it done.
|
| The key is backing it up continually and really not getting mad
| about estimates that are longer than I'd like. And, for people
| who I do think are sandbagging, asking more specific questions
| about the details of the estimate in a non-combative way.
|
| And of course some people really are bad at estimating. But I
| find if they don't improve with coaching then it's a symptom of
| a bigger problem (not thinking things through all the way)
| which manifests in other ways beyond estimation (eg poor
| design)
| commandlinefan wrote:
| > I won't be mad about a long estimate
|
| So... my experience with this mindset is that you won't be
| _mad_ , but you'll just say "that's too long/expensive" and
| cancel the project entirely. Then the same project will come
| up again in two months with different wording, again and
| again, until I give you the estimate you think you can
| afford. And then the thing will end up taking longer than the
| original "too expensive" estimate (in part because too many
| things were rushed in the beginning to try to meet "the
| date"), but nobody will ever compare the final outcome
| against the original estimate anyway... because estimates are
| never meaningful.
| stingraycharles wrote:
| I think this attitude is too fatalistic. A good approach
| would be to see if there's some way to break up the feature
| in smaller pieces, into smaller milestones, etc.
|
| If you have a manager that is unable to deal with this kind
| of stuff, then your problem is the manager, not the
| estimates. Estimates are extremely useful, and you're doing
| yourself a disfavor if you think that they are never
| meaningful.
| commandlinefan wrote:
| If I was surrounded by people who could put together
| accurate software estimates that made upper management
| happy, and I was the only one who was always got it
| wrong, I'd hang up my hat and see if I could get a job
| selling life insurance. Hell, if 10% of my peers could
| put together accurate software estimates that made upper
| management happy, I'd lobby to have them elevated to
| senior positions and spend all my time begging them to
| explain their secrets.
|
| But they don't. Nobody does. They don't just get the
| timelines wrong, they get the tasks and milestones wrong
| - which makes sense, because the people asking for the
| estimates don't actually know what the goals are, usually
| even after the software is delivered.
|
| The only thing an intelligent software developer can do
| is play along, learn to read subtle cues as to what they
| _want_ you to say, say that, and get on with the actually
| messy business of delivering working software.
| mason55 wrote:
| > _because the people asking for the estimates don 't
| actually know what the goals are_
|
| And this is the real problem. People who know how to
| build & design products should be able to explain the
| goals of what they're asking for. If they don't then
| either you need to move or you need to get them moved.
|
| One of the first things I tell new engineering managers
| is that an easy hack to is to always be asking yourself
| "what am I trying to accomplish?" Whether it's an
| emotional conversation or writing a document, if you
| can't answer that question then you probably shouldn't be
| moving forward with whatever you're about to do. And if
| you're writing a document, make the very first section
| "The goal of this document is..." because as you write
| you can constantly ask yourself "is what I'm writing
| accomplishing my stated goal?"
|
| Communicating your goal also helps your team - if they
| know the goal then they don't act as automatons following
| directions, they can make actual decisions on their own.
|
| If you know the business goal of what you're building
| then you have a better shot at getting the requirements
| and tasks correct, which gives you a better shot at
| giving a good estimate. I've worked with a number of
| product managers who had no real goal behind what they
| were asking for. I refuse to have the team start building
| things until someone can explain why we're doing it and
| what we're trying to achieve.
| majormajor wrote:
| > But they don't. Nobody does. They don't just get the
| timelines wrong, they get the tasks and milestones wrong
| - which makes sense, because the people asking for the
| estimates don't actually know what the goals are, usually
| even after the software is delivered.
|
| I definitely agree that the people asking don't actually
| know what the goals are. They usually have a very fuzzy
| picture there.
|
| But I've always found the estimation and planning process
| to be hugely valuable for revealing the questions that
| reveal to THEM that they don't know all those
| requirements yet. And then we have a discussion about
| them, and we get better specs as a result, and then go on
| from there... which is way better than when we don't
| discover those gaps until we are writing the code for it!
| majormajor wrote:
| I've seen this, but I don't think it was a problem or a bad
| thing.
|
| Let's say the project was first proposed in January 2021.
| Got estimated as taking a year. Didn't get scheduled, since
| that was seen as too long, and there was other stuff people
| wanted done in 2021.
|
| It's pitched again in June 2021. Same story.
|
| Come Feburary 2022, it's pitched again. Now it's estimated
| as 15 months - there's more to do since more code has been
| written in the meantime! - and gets started, and ends up
| taking 19 months (estimates are never perfect!).
|
| You might say "this was stupid, we should've just done it
| in January 2021" but in the cases I've seen, pushing it off
| a few times made perfect sense. The payoff wasn't seen as
| the most valuable thing that could be done, and the effort
| was high. The effort became higher after it was postponed,
| but by that point so was the relative value compared to
| other proposed projects.
|
| On the other hand, if you hadn't come up with that first
| estimate, maybe the assumption is "ok this will take
| several months but we'll still be able to do this other
| stuff in 2021", but instead you work on it throughout 2021
| and ship it in March 2022 or so (so ~15 months, still
| faster than doing it later), but that causes you to _not
| do_ 80% of those other things you thought you could do.
|
| Postponement or cancellation of a project because it's just
| too expensive to be worth it right now is a perfectly valid
| use of an estimate!
| marcosdumay wrote:
| Hum... The way to GP is written looks more like the
| scenario that the dev manager/PM/whatever comes to the
| team with the problem and gets the 1 year estimate, says
| "it's too expensive" and closes the project. A few weeks
| later, somebody states the problem again and pushes the
| manager to get another estimate, that is too large, so
| the project is closed.
|
| Repeat that until the problem gets a singularly
| misleading wording, or some key person is away, and the
| project gets a 6 weeks estimation. It will take 19 months
| anyway, because nothing changed, but for 17 of those you
| will be late.
|
| (Anyway, I have never seen a problem statement to be well
| defined enough for this to be the problem.)
| majormajor wrote:
| > Repeat that until the problem gets a singularly
| misleading wording, or some key person is away, and the
| project gets a 6 weeks estimation.
|
| At that point we're definitely at a point of "severe
| company process problems" and the method outlined in the
| original article probably isn't feasible - I can't really
| imagine that manager accepting the time required to
| create a GOOD estimate - but I've definitely seen it work
| properly at some of my jobs.
| setr wrote:
| My rule of thumb is that if you give me an honest estimate,
| then it's my job to make it work. Negotiate scope, change
| implementation, move resources, increase cost, whatever.
| But I need estimates that at least reasonably correspond to
| reality in order to make my case successfully.
|
| The biggest issue I have is developers making business
| decisions without realizing it or without saying it --
| saying no to something because it would take too long or be
| too expensive, without first asking how much time/cost can
| be consumed
|
| The same occurs with estimating -- they'll give a risky
| estimate instead of a safe one, because they'll
| unilaterally decide the stable one is unacceptable; and not
| bother mentioning this with their estimate, or knowing what
| the rest of the timelines look like.
|
| The other half of the problem is that people tend to only
| want to give one number (and people typically ask for a
| single number), but what you really want for project
| planning is the median +- margin of error timelines. The
| 1x,2x,3x rule is just a hack to work around that
| unwillingness.
| mason55 wrote:
| Yeah, from the product management side this is the way to
| approach it. Make priorities clear and make it clear
| whether the important part is the date or some set of
| features.
|
| If the date is what's important then start the
| conversation with "here's the date, can we get this all
| done by then? If not then what can we get done? If we
| can't do it all, here's the stuff that I think is
| important." And make sure to allow some time for people
| to think about things and give an honest estimate.
|
| If the features are what's important then don't even put
| a real estimate on it because then that just turns into
| features + date which never works well. SWAG it by weeks
| or months and refine as you get further along.
|
| It's basically a law that we always want t do more than
| we have time for, so it's important for the decision
| maker to be clear about which parts really matter.
| mason55 wrote:
| > _my experience with this mindset is that you won 't be
| mad, but you'll just say "that's too long/expensive" and
| cancel the project entirely_
|
| Well, as far as this stuff goes, I kind of have it on
| easymode because I'm a VP Eng who also does a bunch of
| product management work. When I'm doing product design, I
| have the benefit of knowing approximate engineering
| LoE/order of magnitude, so I can make sure that I design
| products where the engineering effort fits into the product
| development cycle.
|
| But it's not unusual for something to be harder than I
| thought because of some detail that I'm just too far from
| to realize, and in those cases it's clear to the team that
| they should be honest with challenges and we'll work around
| them together, vs. some bullshit because they're afraid I
| won't like their estimate. The last part is built with
| trust though and basically never "shooting the messenger"
| when someone tells me that there's a challenge.
| LegitShady wrote:
| The multiplier was the same way I dealt with estimating time
| budget for homework in university. I would estimate the amount
| of time I thought it should take and multiply by 3 and it would
| usually take between 2-3x at the end.
|
| It's a little harder when the work is more nebulous and I don't
| have to do it for others but it's all based on personal
| estimations of productivity as well as understanding of the
| problem. It's not easy to do.
| andruby wrote:
| Do you mind explaining why you say estimating is fundamental?
|
| I don't disagree that it is important, I probably wouldn't
| label it as fundamental, at least in a context of a company
| developing its own software internally.
| x0x0 wrote:
| It's one of the top two or three most important things.
|
| All teams have far more work than they can possibly do. A
| handful of things win; most things lose. Reasonable cost
| tradeoffs to pick which two or three things per quarter get
| worked on require estimates. And even in small startups,
| millions of dollars are spent on salaries per those
| decisions.
| majormajor wrote:
| Even when developing your own stuff internally, you have a
| limited resource: your dev team's capacity to do work.
|
| The most important question for your internal software team
| is still "what should we work on" and estimates are a
| fundamental part of answering that.
|
| It's someone's job to figure out what things that team could
| do to provide the highest value. Usually some combo of
| product/sales/marketing/tech.
|
| But that's only half the equation! You also want to know how
| much it's going to cost, in terms of those scarce dev
| resources. If one project would add X value but take a year,
| and another set of projects would add Y, Z, and A value, but
| each only take four months, you want to compare Y+Z+A vs X to
| see what gets you the most value over the next year. So the
| better the estimates, the more rational you can be about what
| work gets done. If you have _no_ estimates, you might pick a
| project that will go on for way too long, and not get the
| value out of it that you expected.
|
| (However, I doubt anywhere truly uses NO estimates. I've
| never seen a place that didn't at least have an informal
| level like "oh that'll be really hard" or "that'll take us a
| long time.")
| ColinHayhurst wrote:
| Fair enough. It was the word that came to mind and I might
| have used a better one. Important doesn't quite capture it.
| Time is invariably the most precious resource, so perhaps
| "crucial".
| satyrnein wrote:
| Joel Spolsky's team actually built that approach into a
| feature: https://www.joelonsoftware.com/2007/10/26/evidence-
| based-sch...
| killjoywashere wrote:
| A Jira plug-in to use some sort of Bayesian system to adjust
| developer priors at the task and sprint level. That would be
| amazing.
| tel wrote:
| FogBugz used to do something of that nature. I never used it,
| so I'm not sure how well it worked out ultimately.
| msluyter wrote:
| Yeah I've always felt like the missing part of the loop (in
| the context of pointing stories) is looking at point
| estimates and then tracking how accurate they were, _by
| developer_. E.g., I throw 2 points, but it turns out the
| story takes something along the lines of 13 points, then my
| estimation was pretty low and my personal "estimation
| multiplier" is increased. Subsequently, actual point
| estimations could take the estimation multiplier into effect.
|
| Of course, this is mostly a toy idea that's fun to think
| about, but seems pretty untenable given that it requires
| keeping track of time, which most developers hate. That, and
| there'd no doubt be incentive to game the system by padding
| estimates or other shenanigans.
| killjoywashere wrote:
| Just look at completion rates within sprint. How many
| points did they commit to, how many did they finish. 2
| weeks is a fine unit of time.
| sarks_nz wrote:
| How to make developers want to hit their deadlines with quality?
| Startup land.
|
| There was no feedback loop that rewarded developers to meet the
| estimates. Stock options weren't an option, and I didn't want
| them to do a sloppy job just to hit the 'deadline'.
| ed312 wrote:
| Bonus and/or equity grants via review feedback are a blunt but
| effective tool. You could probably reduce this to "My team is
| not highly engaged/motivated, and I think it would better meet
| my company's needs if they were. How can I improve that?"
| skohan wrote:
| Honestly it's not developers' job to hit their estimates.
|
| If we're talking about a long-term estimate, as in "this
| project will be finished in 6 months", it's your job in
| management to find a way to do this. You've got to break the
| goal down into achievable sub-goals, and monitor progress along
| the way.
|
| Long term estimates _will_ be wrong. Software projects
| generally take longer than expected, so it 's up to you as a
| manager to anticipate this and communicate to stakeholders with
| the correct degree of uncertainty.
|
| If it's an external deadline which _must_ be met, firstly you
| should engineer enough extra time into the timeline to handle
| inevitable delays. And if at any point you feel like the
| timeline is unachievable, it 's up to you to renegotiate with
| stakeholders, or adjust the scope to make it achievable.
|
| And if you have the feeling your team is slacking off and not
| getting work done, honestly this sounds like a lack of
| leadership skills. It's up to you to have the kind of
| relationship with your developers so that they are motivated to
| meet the team's collective goals and take responsibility.
| That's basically all that being a manager is.
| sarks_nz wrote:
| Thanks for that. I'm talking about the difference between
| hitting the deadline on Monday versus Friday. How to
| incentivise that? ie, do half an hour more work for a week,
| or skip the table-tennis when someone asks, etc.
|
| As a developer, I was into making sure I hit my goals, and at
| work to work. As a manager I do struggle with how to
| emphasise that ownership of product, quality, time. Why
| should developers care about hitting Monday with effort,
| instead of coasting to Friday?
___________________________________________________________________
(page generated 2021-07-21 23:00 UTC)