[HN Gopher] Fear makes you a worse programmer (2014)
___________________________________________________________________
Fear makes you a worse programmer (2014)
Author : thunderbong
Score : 82 points
Date : 2024-04-06 06:01 UTC (2 days ago)
(HTM) web link (jvns.ca)
(TXT) w3m dump (jvns.ca)
| atomicnumber3 wrote:
| At an org I worked at, we (the whole org) were notorious
| (internally) for just not being able to execute big projects. In
| late 2022 2 of the top-level big priority initiatives got
| "pivoted" (eyeroll) because they just weren't "going". And
| allegedly a big part of the problem was that nobody wanted to
| admit that things weren't going just swell on these big projects,
| so they gussied up their status reports, and then when this was
| done systemically across every person in the entire project, you
| got a project that looked real good until it mysteriously missed
| deadlines, and after enough of that, finally the curtain pulls
| back.
|
| Now, management and leadership were most affronted by the state
| of affairs, I assure you, and many words were spent extolling
| virtues and lambasting vices and sin. But let me assure you, it's
| not that they love eloquent speech, they were simply not allowed
| to say the real reason for any of it because the real reason was
| a trait of the org that the founder thought was a key to their
| success.
|
| They run a meritocracy, you see, and a pay for performance comp
| philosophy. They run performance reviews very tight, and getting
| a passing mark is publicly said to be something to be proud of.
| The natural result being that everyone lives in constant fear,
| and being put on a big risky project is a great way to not get to
| vest your whole equity grant. So everyone plays defensively, the
| smartest people dodge the hard/big projects, and so on and so on.
|
| And they'll never fix it because they are not allowed to. How
| tragic.
| nomel wrote:
| It seems the issue is the metric used for performance is
| incorrect, along with the compensation for the hard/big
| projects.
| bluefirebrand wrote:
| > Even with good design, w/o tests you will fear change and so
| the code will rot. With good tests, there's no fear, so you'll
| clean the code
|
| Has anyone ever actually found this to be true?
|
| I work in a place with 99% test coverage requirements and it's
| honestly still a super brittle system that everyone is afraid to
| make big changes to
|
| In my experience, automated tests don't promote quality
| engineering, they just calcify whatever quality exists
|
| And before you get into it, TDD is not a panacea for this problem
| nickjj wrote:
| > Has anyone ever actually found this to be true?
|
| It depends.
|
| I've written a number of apps where I'm fearless with making
| changes because I trust the tests I wrote. One of the apps has
| been running for 7 years and it's had a lot of big updates,
| refactors, etc.. I'm breaking every rule there is on jinxing
| things but there hasn't been a single bug introduced due to
| those updates and there's ~85% coverage. It's only a ~3k line
| Flask app though, but it does get deployed straight to
| production with no staging environment and gets hundreds of
| requests per day. It writes to a DB, Redis, interacts with
| multiple 3rd party APIs and sends out webhooks so it does quite
| a few "external" things.
|
| I've never been a fan of TDD and personally I think tests that
| really hit your DB, Redis, etc. help a lot more than mocked out
| unit tests or a billion unit tests and nothing else. I tend to
| write more tests that really test things together. Not full
| blown Selenium style tests, but I do really write to the DB and
| other data stores in tests and often use a framework's built-in
| test client for making HTTP calls. Everything can still be
| really fast too (~100 tests in 2 seconds is my usual rough
| guide for having an assortment of "real" tests with Flask).
| solraph wrote:
| > I've never been a fan of TDD and personally I think tests
| that really hit your DB, Redis, etc. help a lot more than
| mocked out unit tests or a billion unit tests and nothing
| else... ...but I do really write to the DB and other data
| stores in tests
|
| I've come to the conclusion that the idea that "unit tests"
| should test functions/objects in isolation with completely
| mocked dependencies is based more on the slow speed of those
| dependencies in the past than what actually makes for good
| tests. Now that we have faster computers and storage devices,
| and easy/fast store creation, we should move past this.
|
| Obviously, this is very dependant (no pun intended) on the
| dependency in question, but as a minimum, anything with SQL
| should have a test that hits a real SQL DB (PG in docker for
| example) at some point.
| vidugavia wrote:
| 99% coverage does not mean you have good tests. Yes, I have
| experienced that good tests, combined with a good understanding
| which parts are critical and which are not, create a great
| comfort about introducing changes, whether it's Friday
| afternoon or Monday morning.
| lukev wrote:
| This is why I'm neutral on the value of unit tests. Sometimes
| they are helpful, other times less so.
|
| I am always in favor of tests around the boundaries between
| system components (regardless of what you call them;
| integration tests, API tests, etc.) The more robust the better.
|
| If I'm working on a system with robust (ideally generative)
| integration tests covering the interface of a component, I feel
| I have near-complete freedom to rewrite any aspect of the
| component I want with high confidence.
| adonovan wrote:
| Unit tests are good when there are clear and durable units,
| but what many people seem to mean by a unit test is one that
| exercises the internal functions of a package, or the API of
| an internal package that evolves as fast as your product.
|
| Stable interfaces that the user cares about (e.g. CLI, public
| API, RPCs) make for the highest-value tests.
| nomel wrote:
| > ideally generative
|
| How have you seen this done well? In my experience, this
| usually ends up being some hello-world type simplicity that
| doesn't really represent the real world use corner cases.
| lukev wrote:
| I have, on multiple occasions. Unfortunately mostly on
| contracting gigs so I can't cite specifics but I've worked
| on a major ecommerce platform's api, and two separate
| databases using this approach.
|
| For public examples, I'd point you to (e.g.) Jepsen.
|
| I'm not going to deny it requires a pretty high level of
| time/money to implement. But done well it's super powerful.
| jasonpeacock wrote:
| I have, and it was amazing.
|
| You could quickly make the changes you want not worry about
| what else breaks, then run the tests. The tests that you expect
| to fail would fail, and you fix them. Then you find other tests
| that you didn't expect also failed; you'd review the impact of
| your changes on those parts of the system and make appropriate
| changes until the tests were happy.
|
| Because the testing was so thorough (and quality), you had high
| confidence that nothing unexpected was broken.
|
| It was "move fast & break [tests], then fix them and deploy" :)
| sealeck wrote:
| I think if you evolve your testing system to feed random
| numbers into your program and ensure that various correctness
| properties hold for all these random inputs you are likely to
| get a pretty robust program. You can make this slightly
| efficient by using a fuzzer (essentially uses feedback from
| running previous inputs to see how running new inputs works).
| See for example https://propertesting.com/ or
| https://en.wikipedia.org/wiki/American_Fuzzy_Lop_(software)
|
| Of course if you need a higher standard of guarantee you can
| try model checking or other formal verification techniques.
| Izkata wrote:
| Test-a-single-function style, what most people default to when
| they hear "unit test", is probably the thing that most leads
| into that situation.
|
| I've been occasionally pushing coworkers to restructure code
| and tests into a higher level style, defining an API boundary
| and calling it from the rest of the code (as if you're writing
| a library except it's inside your codebase instead of
| installed), then writing tests to that API. That does make for
| some easily refactorable code where the tests don't need to
| change at all.
|
| That pseudo-library style is kind of what "unit test"
| originally meant: a business unit, not a code unit. Examples
| simplified it too much and people copied the style instead of
| the substance, and the original meaning was lost.
|
| For another example, some of our recent projects have been on
| updating daemons, those have also been really good candidates
| for this since there are clear entry and exit points.
| solraph wrote:
| > > Even with good design, w/o tests you will fear change and
| so the code will rot. With good tests, there's no fear, so
| you'll clean the code
|
| > Has anyone ever actually found this to be true?
|
| Yes, but you have to have the right kind of test coverage, and
| that's the tricky part.
|
| Yesterday, I refactored a bunch of functions that changed a
| bunch of unit tests. However, since we also have
| integration/system tests on that code, I'm confident that I
| haven't broken the code as a whole. Without those system tests,
| I would not have confidence that the change would be
| successful, and probably would not have refactored.
|
| In another codebase that hadn't been touched for a year, as
| part of a feature change I refactored an SQL statement to what
| I thought was a more optimal design and immediately broke a
| bunch of tests. Based on that, I was able to understand the
| original intent of the SQL, and updated it in line with the
| feature change. I added test scenarios for the new feature, but
| left the existing scenarios as is.
|
| Without those tests, would have broken the system in a subtle
| way.
| furstenheim wrote:
| Take a look at hexagon pattern from Spotify. Once you start
| testing the user contract of your services against real
| databases (testcontainners is a good option), then you can
| change all the internals and be sure that the externals will
| work.
|
| https://engineering.atspotify.com/2018/01/testing-of-microse...
| barfard wrote:
| could still get 99% test coverage with useless tests
| hinkley wrote:
| I've experienced it.
|
| But only at companies that don't give a shit about test
| coverage %. The percent kills the purpose of the tests, which
| is certainty.
| vineyardmike wrote:
| Just to re-iterate another commenter, I had a positive
| experience before.
|
| When I was at BigCompanyA, we had 95% coverage requirements as
| a management fiat. The company generally was very management-
| decree-driven. People would unit test individual methods and
| helper functions and each "unit" of code. If you wanted to
| change an API, you had to dig through a sea of broken tests
| because each little bit of code was individually tested. We
| literally had unit tests that validated one line methods for
| string concatenation of a prefix (in Java). Management said add
| tests so everyone added tests without thinking about what is
| valuable.
|
| At BigCompanyB, our testing was engineering driven, not some
| management metric being tracked. The goal was to test "public
| interfaces" and ensure that these tests captured all the helper
| methods along the way. This helped catch dead or extraneous
| code if you couldn't write a test to exercise a particular
| path. Changing an API didn't require changing a bunch of silly
| tests. We still had equal >95% coverage, and it was very
| useful.
|
| Basically you need to actually think about what makes sense to
| test and write logical tests.
| agumonkey wrote:
| You're both right. TDD is no panacea, and having no test is
| being blind. There's a black art into estimating what angles to
| test and how deep to avoid calcification.
| zelphirkalt wrote:
| At some point you start to fear changing too much, because of
| all the tests you will have to rewrite, which can be tedious,
| especially, when the code mutates things all over the place and
| you have to mock 5 things per unit test.
| elliottkember wrote:
| I think you missed a word. "Good" tests is a crucial part of
| the statement. It's totally possible to write bad tests, the
| same way it's possible to write bad code.
| tibbar wrote:
| Early on in my engineering career, there are a handful of times I
| was assigned to a project where I was set up to fail. I didn't
| understand the situation until far too late. There are a few
| tell-tale symptoms: I didn't quite understand what I was supposed
| to do; I didn't really know how to do it; and I didn't understand
| how the assigned work actually solved a higher-level problem. No
| one else did, either (these situations usually represent a
| failure of management.)
|
| As those projects dragged on and I was unable to make "progress",
| whatever that meant, I felt shame and a mounting dread of
| returning to the console each day. Eventually, fortunately, I was
| able to roll off them (not having accomplished much in the
| preceding month or two) and got back to doing useful things.
|
| These days I can usually recognize such projects in advance, but
| it's still not always possible to avoid them.
| nomel wrote:
| As a senior person, these are some of my favorite types of
| projects, because I feel like it's where I'm the most useful,
| and where I can have the most impact: understand the need, then
| implement the _need_ rather than the request.
|
| When I see people struggling with these, it's usually from a
| lack of information seeking/gathering, where they first sit
| down and code rather than spend the first few weeks talking,
| reviewing understanding, and, most importantly, finding those
| few A-team people that have meaningful input.
|
| Definitely not something someone early in their career should
| be given, but these types of problems usually benefit from
| _devaluing_ manager input, since they have a disconnected
| /warped perception of reality. I take these projects only if
| the understanding is that I'll be solving a problem, taking
| input from all involved, rather than implementing a specific
| solution.
| j45 wrote:
| Getting the need right, in lockstep with the audiences
| realization of it is an important soft and hard skill that
| should be better taught.
| nomel wrote:
| > in lockstep with the audiences realization
|
| Perfect wording! I tell people the hardest problem, in my
| work, is getting people to see there is a problem. The
| crazy workarounds that people will come up with, to avoid
| the root issue, are really _incredible_.
| softfalcon wrote:
| Yeah, these types of projects can turn out to be a real gift.
| They are essentially a "folks here have an initiative they
| want to fund but don't know how" and if you direct the
| conversations well, you can create something incredible out
| of it.
|
| Things being up in the air and vague is often an opportunity
| to step in a tame a wild forest of ideas into a real
| application.
|
| It definitely takes a certain type of mindset to harness that
| energy and herd the cats though.
| ElevenLathe wrote:
| I've rarely had a task assigned to me that was well-defined
| enough to even know if I completed it. Usually there is good
| intentions, but the people above you just don't have time to
| dig into the details of what it is they think they are
| assigning to you. This also means that they can't really check
| up on you too well, either. The trick is to just dig in, do
| some research, do stuff (ideally stuff that genuinely needs
| doing, that you can accomplish, and is related to the "task"
| you have been given), and report on it confidently. Eventually
| people will consider you an expert on that issue/area and will
| defer to you about what needs to be done. Once you hit this
| point, you can make a list of "nice to haves" in this area,
| throw them on the backlog, and declare yourself done. If you're
| recognized as the main expert on that subject, it will be tough
| for people to argue with you.
| tibbar wrote:
| I think the difference here is that a 'bad' project will
| often be very specific. You will have a very particular
| outcome that you must achieve to finish, and you know what it
| is. ("Migrate our database from NoSQL to a relational
| database!" etc.) But you don't actually have the skills to
| perform the task, or the support to even get started in the
| right direction. You don't even know what questions to ask,
| or who to ask them to. You're just ... lost.
|
| It's also possible the project is simply too hard -- maybe
| doing it right would take an experienced engineer two years;
| but since that approach seems like obviously too much work,
| you flail around assuming there must be some alternative.
|
| This is, sadly, pretty common for junior programmers and
| people who are new to a team.
| nestorD wrote:
| > Early on in my engineering career, there are a handful of
| times I was assigned to a project where I was set up to fail. I
| didn't understand the situation until far too late. There are a
| few tell-tale symptoms: I didn't quite understand what I was
| supposed to do; I didn't really know how to do it; and I didn't
| understand how the assigned work actually solved a higher-level
| problem. No one else did, either (these situations usually
| represent a failure of management.)
|
| Yes! I have been in such a situation once (I didn't quite
| understand what I was supposed to do _and_ no one else did,
| either) and, to this day, I remember it as a cautionary tale
| when I think of moving to a new position.
|
| The people who think that they can shine in this situation by
| being proactive underestimate the lack of understanding: it is
| not a blurry task, it is a task where you are told to do X and
| no one knows what X actually means. You can do great things,
| but they will not be able to deliver on the requirements.
| yarg wrote:
| I've been in a situation where I was set to fail despite the
| fact that I was actually performing a vital task for the
| company.
|
| Every damned time I set out to implement changes necessary to
| ensure the maintainability of the project, the boss would bring
| in an architect (who otherwise never even looked at the
| project) and he would pull the rug out from under my feet.
|
| Every damned time. By the time they realised I was actually
| useful and regretted the situation I was checked out and ready
| to walk out the door.
| Attummm wrote:
| In my experience, fear serves as a double-edged sword in software
| development. While it can inhibit us from making necessary
| changes, it also acts as a powerful motivator. The fear of
| introducing new bugs drives us to conduct thorough testing and
| think critically about our code. Moreover, the fear of burdening
| others or causing system outages reinforces the importance of
| careful consideration and caution
|
| The article reminded me of the military's emphasis on maintaining
| a healthy level of fear to prevent complacency.
|
| When working on new projects, there's room for bold moves and
| experimentation. However, in existing large-scale systems, the
| impact of our actions extends beyond ourselves to the entire team
| and company. Let's avoid cowboy coding and prioritize stability
| and reliability.
| kevmo314 wrote:
| > If you're scared of making changes, you can't make something
| dramatically better, or do that big code cleanup.
|
| Funny, reading the headline I thought the article was going to be
| about the polar opposite. Over time, I've found the headline to
| be absolutely true, but I have also found that many invest in
| tooling and overabundant testing out of fear.
|
| eg, "What if we need to recreate our entire stack from absolute
| scratch?"
|
| I mean yeah, from that perspective terraform is totally useful,
| but what's next, automate the entire creation of the company,
| including customer acquisition and hiring? How often do you
| really need to recreate everything from scratch?
|
| As a more junior engineer I used to have the confidence that
| things just wouldn't break, or at least if they broke it wouldn't
| be such a big deal. More often than not I was right. As a senior
| engineer, I find myself just pushing my junior engineer tenets.
| Usually everything will be fine, and even if it does break, we
| can fix it.
|
| Write tests and invest in tooling if they're helpful. If they're
| not, don't be afraid to just write code by ssh'ing into a server.
| kazinator wrote:
| The Rust community, ecosystem and its advocacy and advancement
| depend critically on fear.
| kevingadd wrote:
| A fear not mentioned directly in this post that definitely makes
| you a worse programmer is fear of (perceived) individual failure.
| I've dealt with (usually junior, but not always) programmers who
| shied away from difficult tasks or complex parts of a codebase.
| It seemed like they were afraid of failing and being seen
| failing, and preferred to select tasks they knew they could
| handle. A natural outcome of this is that you end up with big
| important parts of your codebase that are only understood by a
| handful of people, because they're scary. The fact that we had
| tons of automated tests didn't fix this.
|
| You can partly address this by trying to make sure every part of
| your codebase is easy to understand, but sometimes your code is
| just going to be complex. That becomes an education problem, you
| have to work with these developers and coax them into confronting
| difficult chunks of code and help them develop the skills they
| need for understanding.
|
| The same applies for development/debugging techniques. I walked
| one developer of equal seniority through using WinDbg once,
| because javascript he wrote was causing IE6 to crash. It was _my_
| first time doing it, so the role I had to play was the "let's
| just try things and see if we can get anywhere, there's no harm
| in failing" facilitator. Better to try something new than to give
| up. We didn't come away from the exercise with clear answers, but
| we had learned some useful information in the process by
| exploring.
| DustinBrett wrote:
| I've heard it's even the mind killer.
| jt2190 wrote:
| (2014)
| ketanmaheshwari wrote:
| I think 2014 should be added to the title.
___________________________________________________________________
(page generated 2024-04-08 23:00 UTC)