[HN Gopher] The underwhelming impact of software engineering res...
       ___________________________________________________________________
        
       The underwhelming impact of software engineering research
        
       Author : signa11
       Score  : 119 points
       Date   : 2022-04-07 09:17 UTC (2 days ago)
        
 (HTM) web link (iansommerville.com)
 (TXT) w3m dump (iansommerville.com)
        
       | Animats wrote:
       | Is that all he has to say? That reads like a cover sheet.
       | 
       | There's been considerable progress in programming languages and
       | tools. It's now possible to write very large programs with
       | extensive concurrency without having them crash frequently.
       | 
       | There's been progress in project organization tools. There's
       | Github, and Slack, and Stack Overflow. It's now possible for a
       | large number of people to collaborate on something and still get
       | it to work.
       | 
       | That's pretty good, actually.
        
         | ogogmad wrote:
         | > There's been progress in project organization tools. There's
         | Github, and Slack, and Stack Overflow. It's now possible for a
         | large number of people to collaborate on something and still
         | get it to work.
         | 
         | Does this stuff come from academia? That's what's being
         | debated.
        
       | TheAceOfHearts wrote:
       | This is a point of frustration to me. It seems like there's a
       | significant disconnect between research in academia and research
       | in the real world.
       | 
       | Beyond that, a lot of software engineering research doesn't seem
       | to be easily reachable or discovered unless you're in academia.
       | 
       | Real world software engineering has to ship a real working
       | product, while academic software engineering only has to produce
       | a paper. Maybe that's an unfair representation though.
        
         | corpMaverick wrote:
         | I feel that in the past there were more products shipped by
         | Academics. Unix BSD from Berkley, Minix from Tanembaum, Kermit
         | from Columbia, Pascal from USCD; Pascal, Modula and Oberon from
         | N. Wirth, etc.
         | 
         | Building these products would be a great teaching tool and
         | contribution to society. Also there could be a lot of research
         | done on how to build complex software where the product is the
         | software itself and it can be applied to other sciences that
         | badly need software. I for one would love to get a graduate
         | degree while building some product where I get to try different
         | software design approaches.
        
         | sheikheddy wrote:
         | Try reading some of the older, classic papers. I think those
         | have taught me a lot about the fundamentals.
        
           | 0des wrote:
           | There could be the next Knuth among us. Excited to see what
           | they bring.
        
             | BoiledCabbage wrote:
             | Either optimizing ad-clicks on the web. Or a new NFT crypto
             | tie in sham.
        
           | debbiedowner wrote:
           | What's your list of classic papers for language models for
           | code, static analysis, or code summarization?
        
         | hinkley wrote:
         | I told a coworker once that I subscribed to the ACM (which I
         | don't anymore), and he looked at me like I'd just parted the
         | Red Sea. He was amazed I could understand them. I told him I
         | could only understand about half (closer to 3/4) which
         | mollified him a bit.
         | 
         | There are any number of reasons I have stayed engineering track
         | instead of going into management but the biggest is that
         | talking about things and doing them are vastly different, and
         | we've structured management and research around talking. Some
         | days I worry I may know more about project management than I
         | know about cloud or React, but it would kill me to be telling
         | people how to do the job that I'm not also doing, and if I
         | don't like how they did it, have only words available to get it
         | fixed.
         | 
         | In one of my hobbies, time and the pandemic have slowed me down
         | quite a bit and I'm trying to manage more and do less, but
         | unlike work there are plenty of opportunities to lead by doing.
         | Doing 5-10% of the work and then guiding people to do the rest
         | is socially acceptable.
         | 
         | In software we sometimes call that a lead developer, but that
         | definition changes from situation to situation. But the more
         | you do the less influence you tend to have on what you're doing
         | and why. Is that human nature or leftover from feudalism?
        
       | tinco wrote:
       | I think it's not so much the lack of good research being done,
       | but the lack of real world application of that research. And
       | furthermore the real applicable research wasn't really
       | theoretical research but more lessons learned by engineers in the
       | industry.
       | 
       | It's crazy to me that researchers prefer to be published over
       | having world renowned software to their name. You could figure
       | out the fanciest database research, but there was a period where
       | every fancy new NoSQL database was based on BerkeleyDB despite
       | all that, bolstering the reputation of the University of
       | Berkeley.
       | 
       | Despite all the research into garbage collectors nearly every new
       | programming language I've seen uses either BoehmGC (from 1994) or
       | simply gives up on the idea and uses reference counting.
       | 
       | Imagine if university groups would band together to produce
       | really state of the art production quality reference software.
       | They could use them as base for their research, bolster their
       | reputation by having them be great, and even draw funding from
       | big companies looking to hire the PhD's produced by the efforts.
       | 
       | Instead we have got all these half finished demonstration
       | software that no one industry would touch with a ten foot pole.
       | 
       | It took just a few people to develop the first versions of rust,
       | a research based programming language that in my opinion has been
       | the biggest shock to the industry in decades (if you disregard
       | machine learning).
        
         | amelius wrote:
         | People should code in Haskell more, and we would have massive
         | amounts of reusable code. All the problems stem from premature
         | optimization, and rush to market.
        
           | ferdowsi wrote:
           | Academia trying to "Make Haskell Happen" is a clear
           | indication of how disconnected it is from the practice of
           | software engineering. Haskell is not going to happen.
        
             | ParetoOptimal wrote:
             | My current Haskell job and previous Haskell jobs disagree.
             | Also I'm not academia since I don't even have a bachelors
             | degree and am a professional Haskell developer.
        
           | unmole wrote:
           | Yeah. Won't have any software problems if there isn't a
           | software industry to begin with.
        
           | coldtea wrote:
           | > _and we would have massive amounts of reusable code_
           | 
           | Haskell even has problems coming up with a single string /
           | text interface. It's not some magic bullet for code reuse,
           | any more "everybody would write Java and we would have had
           | massive amounts of reusable jar libs".
        
             | amelius wrote:
             | But at least we'd have referential transparency.
        
             | ParetoOptimal wrote:
             | > Haskell even has problems coming up with a single string
             | / text interface.
             | 
             | That's orthogonal to the claim that if everyone used
             | Haskell we'd have better code reuse.
             | 
             | > It's not some magic bullet for code reuse
             | 
             | Not a magic bullet, but makes code reuse much easier than
             | any other lanugage.
             | 
             | That can be attributed to a focus on sound abstractions,
             | referential transparency, and laziness[0] mean that code is
             | much easier to reuse.
             | 
             | 0: Yes, laziness helps composition and code reuse!
        
           | unicornmama wrote:
           | Your problems are my opportunities.
        
             | amelius wrote:
             | You enjoy wading through other people's rushed code?
        
               | unicornmama wrote:
               | I enjoy watching production blow up while I munch on
               | salty popcorn. Then I come along with my master plan and
               | solutions that management just begs to throw resources
               | into. Then I move onto the next blow up.
               | 
               | It's much more lucrative to be a firewoman when there's
               | idiots pouring gasoline and throwing matches everywhere.
        
           | oconnor663 wrote:
           | Are there any prominent examples of a library in Haskell
           | being called from an application that's not written in
           | Haskell?
        
             | ParetoOptimal wrote:
             | I wonder if Pandoc gets used as a shared library much... I
             | think it's usually shelled out to honestly.
        
         | jart wrote:
         | > Despite all the research into garbage collectors nearly every
         | new programming language I've seen uses either BoehmGC (from
         | 1994) or simply gives up on the idea and uses reference
         | counting.
         | 
         | Have you ever seen a garbage collector that uses the NSA
         | instruction?
         | https://github.com/jart/cosmopolitan/tree/master/tool/plinko
        
           | KerrAvon wrote:
           | Not sure Lisp variants count for the purposes of the person
           | you're responding to; it's not a new language and most Lisps
           | actually do their own GC.
        
             | jart wrote:
             | Uhm what?
        
         | lumost wrote:
         | Software collaborations tend to end poorly in Academia from my
         | experience. This is due to most of the day to day boring
         | software work being viewed as "less than" the sexy publishable
         | research.
         | 
         | What this means is that academic software projects become
         | spaghetti code messes that no one wants to touch at warp speed.
        
         | toast0 wrote:
         | > It's crazy to me that researchers prefer to be published over
         | having world renowned software to their name.
         | 
         | If you went into research, you have occupational incentives to
         | publish papers, but not to publish software.
         | 
         | As someone who went into applied software, there's a lot of
         | published research that's so disconnected from reality, that it
         | couldn't possibly be useful for anything but publication stats
         | for the named researchers. On the other hand, every once in a
         | while, there's a gem that's useful.
         | 
         | My personal favorite is network research where everything was
         | done with models of TCP that aren't at all similar to actual
         | TCP stacks, production or research.
        
           | matthewdgreen wrote:
           | If you went into research you did so because you want to
           | solve new problems. With no disrespect to software
           | engineering, most (production) software engineering involves
           | solving old problems over and over again on a daily basis,
           | with genuinely new problems tossed in only from time to time.
           | This is even more the case when you're maintaining a codebase
           | that's widely used.
           | 
           | There is nothing wrong with solving old problems: that
           | describes most useful human work. It's just that finding and
           | solving new problems is a very specialized kind of work, and
           | it's very hard to do that specialist work if you aren't
           | spending most of your time on it. You need to keep up with
           | the literature and fight to stay at the forefront of your
           | field and you can't do that as a 20% time position.
           | 
           | A good analogy here is like the interplay between a general
           | contractor and a licensed electrician. The contractor could
           | do their own wiring in between framing and drywall (and take
           | the necessary tests and certifications to learn the code),
           | but it usually makes more sense to have specialized
           | contractors come in for that work. Unfortunately the problem
           | described by this article is that the software developers
           | (contractors) aren't using the output of the researchers.
           | Insofar as this is true it should be addressed. However the
           | answer probably is not to have the specialized researchers do
           | more general software development, which will just make
           | everyone less efficient.
        
         | glouwbug wrote:
         | Isn't reference counting the obvious implementation? A garbage
         | collector would only free cyclical references
        
         | coldtea wrote:
         | > _It 's crazy to me that researchers prefer to be published
         | over having world renowned software to their name. You could
         | figure out the fanciest database research, but there was a
         | period where every fancy new NoSQL database was based on
         | BerkeleyDB despite all that, bolstering the reputation of the
         | University of Berkeley._
         | 
         | Unless the BerkeleyDB was written based on novel research, it
         | didn't move the state of the art forward, it was just a good,
         | used, embeded database.
         | 
         | In which case, your statement can be rewritten as:
         | 
         | "It's crazy to me that researchers prefer to be known for their
         | research over being known as developers of popular programs".
         | 
         | Doing research is the reason they went to an academic career,
         | if they wanted to be developers, they'd just have gone and be
         | developers.
         | 
         | So, why not strike two birds with one stone, and make high
         | quality widely used software that's ALSO state of the art
         | research based?
         | 
         | Well, because creating "high quality widely used software" is a
         | different process, and needs different resources, dedication,
         | and even skills, than doing research.
         | 
         | To showcase some novel research in practice is much much
         | faster, easier, and cheaper to create "half finished
         | demonstration software", that to create a mature, final,
         | implementation that people can use.
         | 
         | So, either they slow their research considerably, and spent
         | most of the time doing software development work - which
         | besides slowing down the pace of research will also count
         | against them in their academic field, they opt to focus on
         | research and just do simple implementation that are not
         | production ready and aren't meant to be.
         | 
         | Seen in this light, it's not a "crazy" choice, it's an
         | essential tradeoff.
        
           | adminprof wrote:
           | It seems like it's just that the goals are different.
           | Computer science researchers could also make the opposite
           | case, "The underwhelming impact of software development on
           | producing state-of-the-art computer science discoveries."
        
           | eslaught wrote:
           | There are some points in the middle, but it's rare. I work on
           | one of these [1]. We've been building the system for over ten
           | years, and are starting to see some truly killer apps being
           | built on top of it [2, 3].
           | 
           | While it has some great benefits once you arrive, the upfront
           | costs are enormous. You need to find a funding source (or
           | sources) that will pay for this product while you're building
           | it. Also, in order for the research payoff to be worth it,
           | you need both the product itself, and subsequent innovations
           | it enables, to be research-worthy. Not all areas of research
           | can support this. On top of it all, even when you do this,
           | you'll still spend years of effort in activities that are
           | essentially not research. You're basically responsible for
           | all of your own customer support, sales, marketing, etc.---
           | like a startup, but without the financial upside if you
           | succeed. Yes there is recognition and so on, but the payoffs
           | aren't as dramatic. Most people aren't ready to commit to
           | this path.
           | 
           | Keep in mind that you can't build this in 5 years either. So
           | a single generation of PhD students can't get it done. The
           | only reason we were successful is because the key staff on
           | the project stuck around for 5+ years after our PhDs because
           | we all believed in doing the work.
           | 
           | Given all that, I don't hold it against people at all who
           | just want to build prototypes and then move on to the next
           | thing. It's way less risky and higher reward relative to the
           | costs.
           | 
           | [1]: https://legion.stanford.edu/
           | 
           | [2]: https://flexflow.ai/
           | 
           | [3]: https://developer.nvidia.com/cunumeric
        
         | mjw1007 wrote:
         | The history of PostgreSQL's SERIALIZABLE isolation level
         | implementation is one example of this going well.
         | 
         | As far as I remember, the story went something like this, about
         | ten years ago:
         | 
         | - organisation employing a PostgreSQL developer asks "can we
         | have SERIALIZABLE that actually does what it says?"
         | 
         | - developer says "nobody has that, it's a hard research
         | problem"
         | 
         | - developer looks for recent academic research and finds
         | someone with a proposal that might be efficient enough to be
         | practical
         | 
         | - developer contacts the researcher
         | 
         | - developer and researcher implement the proposal in PostgreSQL
         | 
         | - it turns out the design is in fact practical
         | 
         | - PostgreSQL gets a useful feature, researcher gets a paper
         | with the developer as co-author
         | 
         | Looks like this is the paper:
         | https://pmg.csail.mit.edu/pubs/ports12__serial_snaps_isolat_...
        
         | makecheck wrote:
         | Also, how do you reproduce even the demo software from a paper
         | that's been out for a year (an eternity in software maintenance
         | time?). Git repo might be gone, some dependency broken, or
         | maybe they used some system that isn't around anymore.
        
       | hyperpape wrote:
       | I think it's important to understand what the author means by
       | "software engineering research". If I understand it correctly, he
       | means research about the discipline of software engineering, not
       | the output of engineering. So research on analyzing agile and
       | waterfall approaches would be "software engineering research", a
       | programming languages paper would not be. A paper on whether
       | static typing reduces defects is a grey area to me---I think it's
       | SE research, but I'm not sure.
       | 
       | It's clear that there's been lots of progress in technology: in
       | networking, in databases, in compilers, etc. It's just that none
       | of that was substantially influenced by research on the process
       | of software engineering.
        
         | jaimebuelta wrote:
         | The problem is obtaining solid data and not falling on cargo
         | cult. If there's a paper showing that static typing is safer,
         | but is assuming A, B and C, it can be very easy to end up with
         | a lot of people saying that only static typing should be used
         | in every imaginable situation.
         | 
         | Applying the results of research on these kind of practices can
         | be incredibly difficult. Engineering processes are not a hard
         | science...
        
       | adam_arthur wrote:
       | I often read SWE research that is just kind of common sense to
       | somebody in industry. Of course, it's worth trying to take an
       | empirical approach to quantify merits of certain techniques, but
       | the vast majority of the research that's done does not seem that
       | novel.
        
         | sheikheddy wrote:
         | I'm fairly certain that you could probably write and publish a
         | decent quality paper yourself. But why would you, when industry
         | is so much more lucrative? There are many competent people in
         | CS Academia, but their motivations tend to be quite different
         | from the average engineer, and they don't have access to the
         | same resources or expertise that tech giants or startups do
         | (outside of internships and fellowships that are commonplace
         | for visiting researchers who spend a season or two working at a
         | lab)
        
       | nonrandomstring wrote:
       | Sommerville was my graduate software engineering textbook in the
       | late 1980s. Along with Jackson, Yourdon and a few others it was
       | the bedrock of my thinking about how to manage software projects
       | over the past 30 years. It's not as if I got stuck in that mind-
       | set, having been a keen, early adopter (and early critic) of many
       | faddish methodologies over the years - but I've been astounded by
       | the almost total lack of deep traction these philosophies have
       | had in cultivating a mature, long term, responsible engineering
       | attitude in our business.
       | 
       | I am really pleased to hear a heavyweight like Ian Sommerville
       | pitching in on what I've been seeing as a cultural crisis in
       | software spanning almost a couple of decades now. It's
       | disappointing to hear the usual dismissal of research and
       | lionising of "industry" as if a cowboy "move fast and break
       | things" mind-set constituted a single coherent commercial
       | ideology. Can you imagine if the pharmaceutical or aviation
       | industry took a cavalier attitude towards "eggheads with their
       | cell biology theories" or "crazy academics with their fancy
       | aerodynamic equations".
       | 
       | Is software the only engineering discipline where the product
       | gets less efficient each year? Partly I think it's also the
       | public's fault for accepting such shoddy produce, and of
       | governments for not being stricter because of a Faustian bargain
       | to enjoy mass surveillance and cybernetic social control as
       | vicarious side effects of a hands-off approach.
       | 
       | It must be frustrating to have put a lifetime into carefully
       | considered formalisms and see so little uptake.
       | 
       | Anyway, what we've ended up with a bunch of non-interoperable,
       | insecure devices and services built by well paid but mostly
       | unhappy workers who have almost no grasp of the complexity,
       | history or societal import of what they create. Sooner or later
       | the chickens are coming home to roost, as cybersecurity,
       | resilience or interoperability issues.
       | 
       | Says Sommerville:
       | 
       | "As a community, senior members (and in this I include myself)
       | have failed to create an inclusive environment where new ideas
       | from outside of the traditional elite are supported. We do not
       | act together when dealing with policy makers and funding agencies
       | and, with rare exceptions, have not spoken out against the futile
       | short-termism that has hampered the development of our field."
       | 
       | I absolutely salute you Ian. I wrote similarly a year ago in my
       | last book that I feel a great sense of regret as a computer
       | scientist who did not speak out sooner and louder. Perhaps it's
       | grandiose/omnipotent to think we ever _could_ have made a
       | difference, but if not our generation then I really hope a new,
       | radical generation will emerge who have the courage to  "take
       | back tech".
        
       | wrnr wrote:
       | One might just write an article about the underwhelming impact of
       | commenting on the internet.
        
       | lostdog wrote:
       | As a software engineer in the industry, what problems am I hoping
       | academic research could investigate.
       | 
       | Compilers that work like devs do. Almost every time I compile,
       | the code is almost exactly the same, but the compiler forgets
       | everything and throws away details from the last compile.
       | Probably all the optimizer's choices are going to be the same
       | this time as last time, so maybe we could cache these?
       | 
       | Better code merging. When my code diverges from the main branch,
       | I typically have to do a bunch of merging work. Would a merging
       | algorithm that understands code semantics be able to do a better
       | job of understanding what I mean during the merge?
       | 
       | Language safety and ergonomics is huge. I very much appreciate
       | Rust's progress, and it's a shining example of academia brought
       | to industry, but can we do more? Jai's idea of making constraints
       | codeable and incremental is pretty cool, and I bet there's a lot
       | of room to improve on it.
       | 
       | Measures of code complexity and modularity. There's a bunch of
       | silly measurements of the "goodness" of code, but maybe it's time
       | for another! Maybe we could describe how "replaceable" components
       | are automatically, and gives a solid idea of the level of real
       | tech debt in the codebase.
       | 
       | How about some project management focused studies? Have a
       | programmer give an estimate for a task, try to force it to be a
       | deadline, and watch as quality of the results drops. Try applying
       | something like scaled agile framework to a real problem and watch
       | as the meeting load snuffs out meaningful development progress.
       | 
       | Or some systems work. Apache airflow is a pretty cool job running
       | agent, but I have yet to see a good error handling workflow for
       | distributed job agents. Can you come up with the "Result" type of
       | job scheduling that's nicely composable?
       | 
       | There's really so much to do.
        
         | UncleMeat wrote:
         | Cached builds are a thing but actual incremental analysis is a
         | very hard problem. Seemingly unrelated changes a pretty common
         | sources of interesting analyses bailing so you get more spooky
         | action at a distance than you might expect. This gets worse
         | with whole program optimization.
        
           | jaimebuelta wrote:
           | Perhaps a good dependency process would be an interesting
           | research. Knowing what parts change when changing a small bit
           | it's actually very difficult to know, and it seems like a
           | common problem that could improve things in many areas (for
           | example, compilation, but also knowing what tests to run if
           | some piece of the code changes, for example)
        
             | UncleMeat wrote:
             | There are hundreds of papers on incremental analysis. You
             | can find them in PL conferences like POPL, PLDI, and
             | OOPSLA. You can also find this kind of topic analyzed from
             | a less theoretical perspective at conferences like FSE and
             | ICSE. It is a hard problem.
        
       | Barrin92 wrote:
       | I don't think the problem is located in academia or research,
       | it's in the market and user obsession of the software industry.
       | For a very long time we didn't produce software primarily for
       | consumers, we produced software for engineers and researchers.
       | Bell Labs and the corporate research lab as an institution with
       | its significant freedoms is an example of this. A lot of
       | innovation was the result, including practical innovation.
       | 
       | Hardware still works this way today basically. The performance
       | optimizations, practices and goals are engineering centric, not
       | consumer centric. Consumer benefits happen downstream as a side
       | effect. The hardware sector is still run by scientists and
       | engineers, one only needs to look at the people running the
       | companies.
       | 
       | There's no shortage of great software research that could be put
       | into practice, plenty of languages with great features are
       | underused. It's the result of the mentality in software that puts
       | costs, users and marketshare over an engineering culture.
        
         | randcraw wrote:
         | I agree. I think a central reason that software, and the
         | process for creating it, have not improved is that no standards
         | for software quality have been demanded of the industry. The
         | creation of no electrical or mechanical device on which people
         | comparably rely is as underregulated.
         | 
         | Few software developers are required to acquire professional
         | certification, or perform in compliance to standard practices
         | intended to assure due diligence in design, implementation,
         | testing or documentation. Very few software systems must
         | undergo compliance certification. It's unsurprising then that
         | software engineering continues to founder and fail to advance
         | as a discipline.
        
           | closeparen wrote:
           | I am glad we did not install standards to freeze the way
           | software was developed in e.g. the 1990s, and I expect that
           | in 25 years I'll be equally pleased that we did not set
           | standards today.
        
       | debbiedowner wrote:
       | What about deep learning approaches like this one:
       | https://ai.googleblog.com/2022/04/pathways-language-model-pa... ?
       | 
       | Or GitHub copilot?
        
       | [deleted]
        
       | HstryrsrBttn wrote:
        
       | jillesvangurp wrote:
       | I did a PhD in the field. And only then became an actual software
       | engineer. As it turns out, doing software engineering research
       | does not involve a whole lot of engineering software. So, I had a
       | lot to learn.
       | 
       | So, this criticism resonates a bit with me. The problem with
       | software engineering as a scientific discipline is that many
       | engineers probably would not consider it a proper engineering
       | discipline to begin with. Also, many computer scientists look
       | down on it as not really related to that either because it is a
       | bit of a soft science.
       | 
       | So, there's a problem with researchers in this field not really
       | having the right background to do good research:
       | 
       | - they are typically not experienced software engineers. Quite
       | many, like me, start working on a PhD without actually having
       | worked as a software engineer.
       | 
       | - they typically have studied computer science which teaches you
       | a lot of math, algorithms, formal methods, and other typical beta
       | skills but is perhaps a bit light on the type of stuff a
       | sociologist or anthropologist would learn. That's a problem
       | because a lot of software engineering involves the behavior of
       | people in groups and coordinating that behavior to produce
       | software. Studying that behavior to verify theories you might
       | have about how software engineering works or ought to work
       | requires an empirical approach. That's not something a lot of
       | computer scientists learn how to do.
       | 
       | So there's a lot of software engineering research that gets
       | published that you could accuse of being a combination of naive,
       | wrong/misguided, and poorly executed in terms of its methodology.
       | Mathematicians or computer scientists trying to be empirical is
       | not great typically. Sometimes you get some decent papers but
       | there is a lot of pseudo scientific nonsense that gets published.
       | Doing empirical research properly is hard and many computer
       | scientists have no clue how to do it properly nor any
       | appreciation for doing so.
       | 
       | Just looking at ICSE's program last year you can see from the
       | titles a weird mix of empirical studies of various topics and
       | other papers clearly written by more computer science leaning
       | types working on algorithms, machine learning or tools. Looks
       | like there were some interesting papers in there but also a lot
       | eye brow raising titles like "How Java Programmers Test
       | Exceptional Behavior". I actually skimmed through the pdf of that
       | one and it fits the pattern beautifully. It's an actual empirical
       | study that tries to answer the question in the title. That's it.
       | There's no new theories being formulated. No remarkable findings
       | that were reported. It's just a very dry survey that yielded
       | exactly what you'd expect. The answers to the research question
       | are almost beside the point and should be completely unsurprising
       | to any Java programmer. Great example of a well executed emprical
       | research paper that answers a question any Java developer in need
       | of actually testing some code that throws exceptions would answer
       | with a simple Google query in a few minutes.
       | 
       | So, yes underwhelming impact indeed.
        
         | bakuninsbart wrote:
         | - they typically have studied computer science which teaches
         | you a lot of math, algorithms, formal methods, and other
         | typical beta skills but is perhaps a bit light on the type of
         | stuff a sociologist or anthropologist would learn. That's a
         | problem because a lot of software engineering involves the
         | behavior of people in groups and coordinating that behavior to
         | produce software. Studying that behavior to verify theories you
         | might have about how software engineering works or ought to
         | work requires an empirical approach. That's not something a lot
         | of computer scientists learn how to do.
         | 
         | One thing that particularly scratches me lately is the
         | popularity of Actor-Network-Theory in some of the humanities:
         | It is a theory that _might_ benefit quite a lot from CS /Maths
         | input, and I think it _could maybe_ be applied quite well in
         | SWE theory as well.
         | 
         | I have no idea if it is actually the case, but one thing I'm
         | quite sure of at this point is that humanities need more formal
         | systems, and SWE research needs to look more towards the
         | humanities and social sciences in terms of frameworks to
         | explain human and machine interaction.
        
         | nudpiedo wrote:
         | To me the problem sounds like there is way too much pressure on
         | the short term prize and constant publication rather than value
         | density.
         | 
         | Sounds like "capitalism" became the philosophy that drives our
         | research nowadays and researchers optimize for it.
        
         | ford wrote:
         | > That's a problem because a lot of software engineering
         | involves the behavior of people in groups and coordinating that
         | behavior to produce software.
         | 
         | I'd hypothesize that most applicable improvements to a software
         | engineering company are about - collaboration & best practices
         | - than anything highly technical.
        
         | fsloth wrote:
         | "many engineers probably would not consider it a proper
         | engineering discipline to begin with."
         | 
         | Calling software engineering "engineering" is perhaps the most
         | blatant uses of describing a thing by it's exact opposite. Like
         | calling hate love, or war piece. (Which sometimes is done, but
         | mainly for propaganda purposes - for the intent of
         | obfuscation.)
         | 
         | Software engineering is much closer to a craft than anything
         | capable of teaching or predicting anything successfully. The
         | best analogue that comes to my mind are the craftsmen who built
         | cathedrals in medieval Europe. Yes, they built things. But they
         | did not understand with scientific rigor, what and why are they
         | doing it. They did not have proper theory, so they could have
         | built something completely different.
         | 
         | You can _learn_ the activity known as software engineering. For
         | most individuals, the best way is from an informal
         | apprenticeship by working with talented colleagues. Again, this
         | is closer to art or a craft than engineering.
         | 
         | You can learn the basics in structured setting. But based on
         | the stories of long winded recruiting interviews, nobody
         | actually _trusts_ any of that, or at least they don 't trust
         | the accreditation mechanism. Which implies you can pass a
         | degree, but only accidentally learn the skills the industry
         | expects a great candidate to have.
         | 
         | Something is institutionally broken here. Imagine if medicine
         | worked this way. Are doctors whiteboarded over complex
         | surgeries before being hired?
         | 
         | I would consider the work done by NASA's software engineering
         | laboratory one of the things closest to "proper engineering"
         | done in the field of software
         | http://www.cs.umd.edu/projects/SoftEng/ESEG/papers/83.88.pdf
         | 
         | Is there anything similar currently operating?
        
           | bakuninsbart wrote:
           | Well, it seems to be in line with the general absurdity of
           | naming conventions in anglophone computer science, since SWE
           | is one of the few subfields of CS that actually uses the
           | scientific method. However, I'm not sure where you are going
           | with your tangent on engineering; in my mind, engineering
           | mostly differentiates itself from the other types of academic
           | research in the kind of question it asks: "How do we [build
           | this]?" in contrast to "How do we [prove this]?" or "How to
           | we [establish evidence in one direction or the other]?"
        
       | ford wrote:
       | IMO most of the practical "research" I've seen comes in the form
       | of best practices from Big Co. who a) hire very intelligent
       | people and b) have extreme financial incentives to write the best
       | code & have the best processes possible.
       | 
       | In a world where the dependent variable isn't something that can
       | easily be measured in a lab, the next best we can do is either 1\
       | Ask experts or 2\ experiment in an environment where it's
       | financially advisable to experiment & measure (AKA an environment
       | where running an experiment with 100 engineers is <1% of your
       | engineering headcount).
       | 
       | I.e. I don't work for Google, but I do share things like [0, 1]
       | with new engineers who join my team.
       | 
       | [0] https://google.github.io/eng-practices/ [1]
       | https://arxiv.org/ftp/arxiv/papers/1702/1702.01715.pdf
        
       | ta988 wrote:
       | I wish there would be a publication habit for industry SWE, we
       | could all benefit from the reports from the trenches. I know we
       | have blogs and the occasional books but I would really enjoy an
       | in-between.
        
       | matthewdgreen wrote:
       | This article would be a lot more convincing if it illustrated its
       | points with examples. I have no doubt that there is truth here,
       | but the objections here are fairly general and unsubstantial. For
       | example, take the scaling objection: give some examples of useful
       | techniques that won't scale and then talk about what you might do
       | differently.
        
       ___________________________________________________________________
       (page generated 2022-04-09 23:01 UTC)