[HN Gopher] Third High Severity CVE in Log4j Is Published
___________________________________________________________________
Third High Severity CVE in Log4j Is Published
Author : niros_valtos
Score : 414 points
Date : 2021-12-18 12:47 UTC (1 days ago)
(HTM) web link (logging.apache.org)
(TXT) w3m dump (logging.apache.org)
| alanfranz wrote:
| Thanks to all Log4j developers. As users of your free software
| since ~20 years, we're sure that, if this vuln disclosure had
| been better and there were no sign of widespread in-the-wild
| exploitation, you would have done a better job at patching it,
| and a single release would have been enough.
|
| We won't hold a grudge against you; open source means
| collaboration, and you don't blame hard-working people that give
| away their software for free for honest mistakes. Your software
| still did far more good than bad to the software world.
|
| Thanks!
| diob wrote:
| So happy to see this as the top comment. We need way more
| positivity in tech.
| throwuxiytayq wrote:
| I was here just to say this - seeing this comment at the top
| of the thread restores my faith in people.
| vmception wrote:
| I thought it was satire, but I guess that says more about me
| wyager wrote:
| Do we? Or does "positivity" enable insecure software because
| there isn't enough social pressure against making dangerous
| engineering decisions?
| watwut wrote:
| Are you saying there would be more security issues if we
| thanked maintainers more?
| wyager wrote:
| Only if we thank maintainers specifically after they get
| bad publicity for a big security issue.
| jiggawatts wrote:
| I lay any blame squarely at the feet of IT security of large
| organisations that were entirely unprepared to update a widely
| used dependency that wasn't an operating system or a runtime.
| Windows, Linux, Java, and .NET are all commonly updated, but
| "modules" aren't.
|
| This wasn't just a predictable scenario, it was predicted. Or
| more accurately, it has occurred already repeatedly in the NPM
| ecosystem, but for some mysterious reason those incidents were
| simply ignored by security teams world wide.
|
| Instead of chilling them to the bone, they simply shrugged
| their shoulders and said "Well, we don't use NPM... I think.
| Probably?" and went on with their paper-pushing or whatever it
| is CISOs do these days.
|
| If you think Log4j is bad, wait until Rust gets popular and has
| something similar happens with a commonly used crate. How would
| you scan for a vulnerability in compiled code that doesn't even
| have separate module files?
|
| I had to help an organisation with log4j that had on the order
| of 5K distinct executables/applications across 3K servers on
| two clouds and three on-prem networks. At least with log4j it's
| a simple matter of finding JAR files and scanning their
| contents. They're just zip files! With languages that output a
| single binary by default such as Rust and Go, we would have
| been screwed. No way to scan, no way to self-help update.
|
| Introspection and post-shipping updates for security are
| mandatory. Rust and Go like to pretend they aren't, because
| they originate from organisations that are in _total, end-to-
| end control_ of the software on their own network. Google
| famously uses a monorepo and can build everything they run from
| scratch in short order. The shortcuts they can take with their
| post-release management _will never work_ for ordinary
| organisations. Never.
|
| So let this be a lesson: Log4j was made for a language that at
| least allowed us all to find the issue and fix it ourselves. We
| have Sun's forward-thinking and the enterprise-friendliness of
| the Java ecosystem to thank for that.
| nitrogen wrote:
| _With languages that output a single binary by default such
| as Rust and Go, we would have been screwed. No way to scan,
| no way to self-help update._
|
| ELF binaries are just collections of another sort. You can
| scan for symbol names or assembled instructions common to a
| dependency, and you might be able to update with an
| LD_PRELOAD library that patches the symbol table.
|
| Just look at what game modders have accomplished without
| access to source code.
| spockz wrote:
| Besides this, who actually scans the deployed binaries?
| Just check which version of the binary was deployed,
| backtrack to the source, and get the dependency tree from
| there. Problem solved. (Except for shaded dependencies...,
| but scanning for that should happen at build time anyway.)
| lazide wrote:
| Good luck doing that on the custom software some vendor
| provided you 5 years ago and since has gone out of
| business.. and that's a GOOD scenario for a lot of
| enterprise software. Sometimes it's 20 years.
| spockz wrote:
| Okay. Just like any other software that should be
| firewalled in to only connect to what it should connect
| to. Moreover, I think meta sploit already has this
| vulnerability built in so it should be trivial to
| test/fuzz the application.
| Too wrote:
| Lesson learned. Never buy closed source.
|
| Imagine the vulnerability would have been in the vendor
| code instead of in the log4j jar. You'd. still need a way
| to fix and rebuild it.
|
| Note that this doesn't have to mean a public MIT repo on
| GitHub. You can have more restrictive gray box licenses
| between your b2b partners only.
| tormeh wrote:
| If it's a 20 year old closed source application, there
| probably aren't any exploits in the wild. You can bet
| there are exploits for log4j.
| hamilyon2 wrote:
| We need some sort of escrow for source code of closed-
| source product. Whenever there is practical need to have
| access to source code, it should be available.
| junon wrote:
| This isn't how the real world works. Symbol patching is
| only effective if there's a symbol for it, which is often
| not the case (and probably wouldn't be the case here) if
| the binary has been stripped[0].
|
| > Just look at what game modders have accomplished without
| access to source code.
|
| Game modding is usually done on Windows given the target
| market (up until recently of course) and thus usually means
| Windows PE's, which do not have debug symbols attached -
| ever. Windows debug symbol databases are emitted as PDB
| files and are almost always omitted from game releases.
|
| This means modding has to do a sigscan[1] in most cases in
| order to find something interesting - especially in the
| case of ASLR[2]. Then whatever modding framework (or hack)
| can set itself up and hook into the game.
|
| These techniques are NOT what we should be encouraging, and
| are certainly not common, even for vulnerability
| mitigation. Re-compilation and re-deployment should be the
| defacto mode of operation in production, especially for
| systems that have sensitive information. Relying on bin-
| patching is not something many security specialists would
| regard as a "good" mitigation.
|
| [0] https://en.wikipedia.org/wiki/Strip_%28Unix%29
|
| [1] https://wiki.alliedmods.net/Signature_Scanning
|
| [2] https://en.wikipedia.org/wiki/Address_space_layout_rand
| omiza...
| [deleted]
| ncmncm wrote:
| The original claims are garden-variety BS.
|
| Library .so files are not harder to scan than jar files.
| They just need different tools, which everyone has. And
| Java systems are not easier to patch than others.
| junon wrote:
| > Library .so files are not harder to scan than jar
| files.
|
| Of course they are. You can trivially reverse engineer
| Java bytecode back to Java, with almost everything except
| generic annotations due to erasure, and a few other small
| things. Tools have been around for ages that do this[0],
| and while they're not perfect, they can re-generate
| nearly perfectly compilable source code.
|
| .so files are ELF files, which means while they have
| sections, those sections don't typically reveal much
| about the internals of the code. This means you need to
| convert machine code (assembly) back to a higher level
| language - which is _not_ a trivial transformation. There
| are tools that _do_ do this (e.g. hexrays[1]) but they
| are _far_ from perfect, _especially_ for optimized
| machine code.
|
| Java systems are very, _very_ easy to crack, RE and
| patch. There are virtually no protections. JAR injection
| is still a valid means of patching a Java application.
| No, manifest files and integrity checks are not a
| security measure but an error detection measure, and are
| trivially bypassed.
|
| If you asked me to choose between patching this
| vulnerability in a .JAR vs a .SO, I would choose the .JAR
| any day of the week.
|
| [0] http://java-decompiler.github.io/
|
| [1] https://hex-rays.com/
| ncmncm wrote:
| You do not need to decompile any code to scan a .so for
| symbols used.
|
| So, more advocacy BS.
| junon wrote:
| Please re-read my original comment. If the binary is
| stripped, no internal symbols would have been visible or
| even present in the file to be scanned. The jumps between
| subroutines happen directly, without a symbol jump table
| (because a jump table adds runtime overhead and is
| largely unnecessary, especially in statically linked
| executables).
| ncmncm wrote:
| You do not need internal symbols.
|
| All the public entry points are spelled out exactly by
| name, and all the external symbol dependencies.
| junon wrote:
| I addressed the point about symbol tables, and that's
| assuming the vulnerable code was pulled in via an
| external dependency or exported directly (flat
| function/subroutine) from an .SO.
|
| That also assumes that the binary wasn't linked
| statically - in which case, there are no external symbols
| to even look for.
|
| Lastly, please stop calling someone educating you "BS".
| ncmncm wrote:
| You instead wholly failed to address the point about
| symbol tables.
| junon wrote:
| Symbol table patching doesn't do anything if the
| functionality is internal... Do you know how programs are
| linked?
| nitrogen wrote:
| We're all on the same team here, there's no need for this
| hostility back and forth.
|
| Binary patching, JAR patching, disassembly, library
| interception, syscall interception, whatever, these are
| all tools that are on the table. You do what it takes to
| get the job done, knowing and weighing the pros and cons.
| ncmncm wrote:
| Do you? Have you, e.g., written a linker? Educating is an
| entirely different process than you are engaged in.
| junon wrote:
| I've written an operating system.
| ncmncm wrote:
| In Java? Maybe better go do that, then. You are doing
| this poorly.
| derefr wrote:
| > This means you need to convert machine code (assembly)
| back to a higher level language
|
| ...why? You don't need code to be in an HLL
| representation in order to patch it. Where by "patching"
| I mean "automatic insertion of a JMP into the subroutine
| prologue, that branches to your replacement code."
|
| You don't even need to _write_ any assembly in order to
| do that, either; you can write your replacement
| subroutine in C or whatever. The replacement subroutine
| just needs to codegen as a C-ABI subroutine with the same
| register preconditions /postconditions as the subroutine
| it's replacing.
| junon wrote:
| You don't need to, you're right. But people aren't going
| to be writing very complex mitigations by reverse
| engineering the bytecode generated by a compiler,
| especially for OSS. There are many compilers, many
| optimizations, and some of them can even produce non-
| deterministic outputs (not common but it can happen). The
| amount of patching for an _internal subroutine_ , which
| is probably what would have resulted here, would have
| been _completely practically infeasible_.
| wbl wrote:
| Go binaries will contain the commit hash, which contains all
| the information about what it was compiler with.
| fnord77 wrote:
| > With languages that output a single binary by default such
| as Rust and Go, we would have been screwed.
|
| no, best practices include checking in your Cargo lock file
| Perseids wrote:
| > We have Sun's forward-thinking and the enterprise-
| friendliness of the Java ecosystem to thank for that.
|
| I find it quite ironic to praise the forward-thinking of the
| company that has been instrumental to bring us into this mess
| via their vision of loading dependencies at runtime from an
| online repository. Sun imagined that the code and its
| configuration does not have to worry about how to fulfill its
| dependencies, but instead JNDI [1] could magically fetch the
| appropriate objects from _wherever_ , don't worry.
|
| I'd say I would even find it ironic to praise the Java
| ecosystem, which, due to its excessive over-engineering, has
| produced best practice frameworks where fully documented
| behavior has the latent potential to just be catastrophically
| exploitable by accident, as nobody is able to reasonably
| understand how all of it plays together. I have a hard time
| to imagine how you could enable loading of external code in
| Rust or Go _by accident_ using run-of-the-mill logging
| frameworks.
|
| [1] https://en.wikipedia.org/wiki/Java_Naming_and_Directory_I
| nte...
| watersb wrote:
| I'm writing this from memory, with checking any sources.
| IIRC:
|
| Designs like JNDI were widespread in the IT software
| industry at the time.
|
| The academic prototype for all this was CORBA, the Common
| Object Request Broker Architecture. An object-oriented was
| to transmit data between systems running on separate
| processes, CPUs, remote servers.
|
| CORBA was a next-gen RPC (Remote Procedure Call) framework,
| the data format on the wire closely follows the format for
| C structs using Sun-RPC, but was way better because these
| CORBA things were discoverable and described at run-time,
| rather than compile-time libraries.
|
| That's right: RPC was initially a Sun UNIX thing.
|
| Microsoft needed a way to do this, so they had COM, the
| Component Object Model, for run-time discovery and linking
| (late binding).. Microsoft got COM to reach across network
| connections, "DCOM", for Distributed systems.
|
| Meanwhile, corporate IT was still pulling text out of
| databases and shoving it into C structs. It sucked.
|
| Microsoft's object model was better, and Java was better
| than better, because you didn't have to license it to use
| it. Well, you did; it was like an MIT License.
|
| Reason why IBM writes huge Java systems. And why Oracle
| owns Java. For some value of ownership.
|
| Yay objects!
| bboreham wrote:
| In CORBA, RPC, etc., the R stands for Remote and it will
| execute something on the other end of the wire.
|
| In these Java attacks the code is downloaded and run
| locally, with all secrets and data access of the attacked
| program.
| Macha wrote:
| > If you think Log4j is bad, wait until Rust gets popular and
| has something similar happens with a commonly used crate. How
| would you scan for a vulnerability in compiled code that
| doesn't even have separate module files?
|
| Our software scans take 3 forms:
|
| 1. Searching for file names
|
| 2. Running test exploits that have a payload that just logs
| the vulnerability with a central monitoring system
|
| 3. Dependency resolution and checks against a DB (our
| solution _already_ supports Cargo.lock files.
|
| Rust (and Go, C++, etc.) only prevent option 1. Thanks to
| bundlers it also can miss it in JS, or shaded fatjars it can
| also fail kn Java too. So it's not a new challenge
| fulafel wrote:
| You can, and people do, scan native binaries for these
| things. It's called SCA (software composition analysis).
| Spooky23 wrote:
| Perhaps everyone is stupid.
|
| It's funny because when I worked in .gov IT, we were all
| characterized as stupid donkeys by because we weren't able to
| just use NPM, etc to do cool kid stuff.
|
| Good luck getting security people with the power and brass
| balls to do that in any company.
| xyproto wrote:
| You can make dynamically linked executables with both Go and
| Rust. The compilers have no blame in this.
| staticassertion wrote:
| I think you could take this further. There's another "log4j"
| vulnerability out there right now, I guarantee it. It might
| be exploited, it might not be. But there's no patch.
|
| What are you doing about it?
|
| With log4j you have a brutal combination of:
|
| 1. RCE
|
| 2. Exposure
|
| RCE happens frequently but often attackers don't have an easy
| time getting to the exploitable code. With log4j it's trivial
| - every app can be owned.
|
| But here are some questions:
|
| 1. Why do those apps have the ability to make network
| requests?
|
| 2. For the apps that need to make those requests internally,
| why aren't those over mTLS?
|
| 3. For the apps that need to make them externally, why isn't
| that going through egress proxying?
|
| 4. Why are there credentials spewed all over your environment
| variables across your services? Are they short lived?
|
| It's actually not super hard to have a worst-case scenario
| vulnerability like log4j be _not that bad_ for your
| organization. A bit of hardening and even if something like
| this happens you 're in a good position to wait, monitor, and
| patch.
| querulous wrote:
| it's definitely time for a language with a capabilities
| based security model that allows scope to the
| function/method level, at minimum
|
| not just for security either. i can't count the number of
| times i've traced a performance problem or bug to a piece
| of code that was doing some io when it didn't need to
| staticassertion wrote:
| Agreed. In a capabilities system you'd see that, for some
| reason, your logger requires all sorts of absurd
| capabilities for no reason.
| twunde wrote:
| Regarding your questions, they make sense in the context of
| modern software built in the last 5 years. However, most
| large enterprises have a long tail of legacy applications
| built years ago. These may be inherited from a combination
| of acquisitions, from vendors that have since gone under,
| some from. The common factor is that the original
| developers are long gone, and they may be supported using a
| skeleton crew or not at all. Documentation if it ever
| existed is likely lost. What these applications are
| supposed to connect to are likely unknown. This is a major
| problem at enterprises
| to11mtm wrote:
| Egress is a very fair question IMO. I've worked at more
| than one place where Infosec would start integration
| talks at 'What are the static IPs and how do we rotate
| certs' Every layer you go lower (longer certs, domain
| whitelisting vs static IPs, etc.) is a tradeoff of
| maintenance and security.
|
| > What these applications are supposed to connect to are
| likely unknown. This is a major problem at enterprises
|
| Only if they fail to consolidate vendor integrations
| and/or have already insufficient staff. (Which is often
| indeed a problem.)
|
| Simple, if brutal solution; Infosec audits logs and works
| with application teams to confirm existing traffic and
| pre-document new traffic out. With properly staffed teams
| you should run out of surprises within a year.
| cutemonster wrote:
| I suspect that a manager that suggested / insisted on
| such properly staffed teams, would get fired or no longer
| promoted, at most big companies. Since it'd cost a lot,
| and not show any results the nearest time / years. And
| also, they might hire the wrong people, and just burn
| money for nothing
|
| > Simple, if brutal solution ... properly staffed teams
|
| The recruitment part is not simple, maybe bordering to
| impossible? for some larger companies. I say, based on my
| past experiences in how clueless big companies can be
| concerning what technical people they hire.
| staticassertion wrote:
| Yes, some organizations have so much technical debt that
| any security work is going to take extraordinarily large
| amounts of work.
|
| That's a bummer, but I suspect many on HN can in fact
| build software that isn't total garbage.
| aidenn0 wrote:
| Doing it right sometimes pays off in the long run, but
| cutting corners always pays off now.
| ChuckNorris89 wrote:
| Exactly. Managers and execs love pointing out the cost
| savings they made _this year or this quarter_. By the
| time the chicks start to hatch "in the long run", they
| most likely would have already moved to another
| enterprise so someone else has to clean up their mess.
| And by clean up, I mean repeat exactly the same mistakes
| since nobody promotes you for implementing stuff that
| might show its benefits in the long run.
| paledot wrote:
| Also one of the root problems with politics. 2035
| emissions targets are the next government's problem.
| Rehabilitative policing pays off in 20 years, retributive
| buys voters now.
| wynck wrote:
| > _I lay any blame squarely at the feet of IT security of
| large organisations that were entirely unprepared to update a
| widely used dependency that wasn 't an operating system or a
| runtime. This wasn't just a predictable scenario, it was
| predicted. Or more accurately, it has occurred already
| repeatedly in the NPM ecosystem, but for some mysterious
| reason those incidents were simply ignored by security teams
| world wide. Instead of chilling them to the bone, they simply
| shrugged their shoulders and said "Well, we don't use NPM...
| I think. Probably?" and went on with their paper-pushing or
| whatever it is CISOs do these days._
|
| I think this attitude is one of the main issues for this kind
| of incidents in larger organisations. I'm not sure why you
| would be willing to lay blame at your colleagues for this.
| Most of the time, development teams don't tend to like some
| governance over the code and the dependencies they are
| pulling in. Not just because they know of course what they
| are doing, but also because they are under pressure to
| deliver features. That is what matters for business. Also,
| convincing management that budget is needed for correct
| tooling to track all stuff deployed; is also not as straight
| forward as you seem to suggest.
|
| In this case, the problem goes even beyond just the code of
| your own dev teams. This is embedded in countless software
| packages deployed all over your organisation. Same here,
| people want to buy and use whatever they want. And all
| processes to keep some form of control over it, are mostly
| seen as overhead.
|
| And I'm sure they are lots of "security" people who are just
| producing documents and policies which are complete detached
| from reality. But developers who consider security completely
| as somebody else his responsibility, are a problem as well.
|
| The only good think I see coming from this mess, is that
| security teams probably will get the means to try to get more
| control and insight over this. For the coming weeks/months at
| least. After that, everyone in management of dev will be
| forgotten about it. But something tells me the security
| people who are working on this right now, won't.
| pabs3 wrote:
| Bundling/embedding/vendoring/monorepos and other names for
| embedded code copies really need to stop being common
| practice in the tech world.
| spc476 wrote:
| What should be done instead?
| MereInterest wrote:
| When HeartBleed came out, there was a single dynamically
| linked library to be replaced. If you had a vulnerable
| version of libopenssl, then you replace the .so or .dll
| with a patched version, restart any applications that
| were using the old version, and you're set.
|
| This works because the binary is compiled against the
| dynamic library, using the system-provided library. It
| isn't statically compiled, because even though that frees
| the developers from worrying about library versions, it
| makes the sysadmin's job tracking library versions be
| much harder. It isn't vendored, for the same reason.
| c-cube wrote:
| If you still have the source code for your rust code, it
| contains the lock file. Look in there and you'll see all
| dependencies and their version. Why do you want to look at
| the binary instead?
| jiggawatts wrote:
| The source code is not what is _deployed_.
|
| There is typically no link back from a deployed system to
| its source, _even if_ you compiled the binary within your
| organisation.
|
| If it came from an external organisation, things are
| exponentially harder.
|
| Modern deployment systems are largely "one-way", with no
| way to trigger a full recompile from the deployment end of
| things. If you have a VM with "SomeRandomBinary.exe"
| running on it that has a vulnerability in it, you don't
| have many options.
|
| IMHO, typical IT as done _in the wild_ suffers from this
| write-once attitude, where the chain of provenance is too
| easily broken.
|
| Companies like Google have a very non-typical setup that
| can be copied only by very small orgs like startups.
| Typical enterprises look nothing like a startup or a FAANG.
| FridgeSeal wrote:
| > The source code is not what is deployed. There is
| typically no link back from a deployed system to its
| source, even if you compiled the binary within your
| organisation.
|
| Do you not have this? Our docker images are tagged with
| the git hash they were built from, so at any point, for
| any of our envs, I can pull up the lock file of that
| build.
|
| Our deployment config also describes everything that is
| running the relevant env, and nobody has machine access-
| going through the pipeline is the _only_ way to get
| access, so there's nothing deployed that's a mystery.
| jiggawatts wrote:
| This happened last week, for real with ${dayjob}
| customer. Put yourself in their shoes for a second.
|
| Imagine you are Mr SecOps guy, and you've just ran some
| sort of Log4j tool across literally _three thousand
| servers_. Of those, several hundred came back positive.
|
| Those included about a dozen flavours of Linux, a
| smattering of manually built(!) containers, and every
| version of Windows from 2008 R1 to 2022. Most of the code
| was built by third parties, some under support contract,
| some not. Most was built and installed manually, with
| developers using RDP or SSH to edit config files an
| whatnot directly on servers.[1]
|
| So what you have now is literally just a string to a
| path, something like
| "D:\apps\foo\bar\baz\libs\stuff\thingie\log4j-core.jar"
| or the Linux equivalent.
|
| Now what?
|
| No, seriously, now _what do you do_? You 're in SEC OPS.
| Not dev ops. You're certainly not in the dev team with
| access to the Git repo of some random vendor product like
| Tableau, or JIRA, or whatever[2]. You didn't deploy it.
| It got installed by a contractor during a short-term
| project three years ago.
|
| A random hash string is totally useless to you. Even a
| repo URL _and_ the commit hash will more than likely just
| end with an "Access Denied" URL, assuming you even have
| a network route to the Super Secure SCM Server.
|
| There is no way you can figure out who needs to do what
| to make this go away. Not at _this kind of scale_ at any
| rate. Not without first-class automation for literally
| everything. Which you can 't have, because third-party
| software just _doesn 't play nice_ with any one tooling
| you'd like to use.
|
| Containerisation? Bahaha... haha... _snort_. You 're
| dealing with vendors that literally advertise "now with
| 64-bit support" and are unable to comprehend the concept
| of unattended command line installers. Vendors that
| insist on USB dongles for licensing. License that expire.
| Annually. And are tied to CPUID values. And on, and on.
|
| [1] Oh, you think _you_ can dictate release methodologies
| to these people? They 're bureaucrats and they play
| politics _better than you_. Any word of changing their
| workflow in any way will immediately bring their boss,
| their bosses ' boss, and maybe a few more levels up down
| upon your lowly head. You will have people _literally
| screaming at you_ that your fanciful notions of build
| pipelines is "too much" and would "impact the work".
| That's the end of the conversion. I said THE END, and
| _good day sir_.
|
| [2] Just get them to update it under their support
| contract? Ha-ha. Ha. Haaaa... We had vendors straight up
| lie about the vulnerability of their software. Then
| another vendor said that updating the JVM already
| mitigates the issues and hence they're not going to
| release an update. (Narrator: JVM updates aren't
| sufficient.)
| Macha wrote:
| > No, seriously, now what do you do? You're in SEC OPS.
| Not dev ops. You're certainly not in the dev team with
| access to the Git repo of some random vendor product like
| Tableau, or JIRA, or whatever[2]. You didn't deploy it.
| It got installed by a contractor during a short-term
| project three years ago.
|
| Your company has an inventory of hosts to teams which you
| can look up and contact. In addition the package manager
| tracks which package the file belongs to. Therefore you
| raise a ticket with the team from the inventory tracker
| that package XYZ on host ABC contains vulnerable file
| /x/y/z/log4j-xyz.jar (The reverse is also true, for each
| package we can find the corresponding repo and set of
| hosts its deployed on). Fixing it is now that team's
| problem, and secs ops guy's problem is now just to verify
| when they claim to have fixed it.
|
| If the software is JIRA? Doesn't matter, still needs a
| package built before it gets deployed on our machine. "No
| really, the vendor's only supported installation method
| is sudo curl | bash" - into a container it goes, the
| owning team of the container can be identified much the
| same way as the owning team of the host.
|
| Now the problem you find is sometimes the result is that
| the host belongs to team XYZ and you find the team was
| reorged by some exec's great idea in 2019 and only one
| guy who used to be on the team is still in the company,
| but he left the team in 2018 and has no idea what that
| team did since he left, but the application has been
| still runnning and power $millions of real revenue. That
| one is much harder to fix by automation. The business
| reality is the security team can't enforce to the execs
| to not lay anyone off or disband any teams without a
| concrete transition plan for their systems.
| kitd wrote:
| _So what you have now is literally just a string to a
| path, something like
| "D:\apps\foo\bar\baz\libs\stuff\thingie\log4j-core.jar"
| or the Linux equivalent.
|
| Now what?_
|
| The remediation (until you can get an update from the
| vendor) is to remove the JndiLookup.class file from that
| jar. It's been fairly well publicised, as has the way to
| do it.
| jiggawatts wrote:
| That provides incomplete protection even for this Log4j
| vulnerability, and similarly trivial "self help patching"
| may not be available for future vulnerabilities.
| arpa wrote:
| > Now what?
|
| Goodbye and thanks for all the fish. Run!
| rockwotj wrote:
| Yeah our org does the same thing and I'd say this is the
| gold standard (or at least mine) but sadly I'm assuming
| most companies don't do this, not everyone runs on docker
| nor has a build pipeline like this.
| yjftsjthsd-h wrote:
| Sure, but everyone should be able to have their built
| process embed that info into build artifacts, whether
| it's by passing GCC a DEFINE flag that ends up in
| --version output, passing nasm a data structure embedded
| at X bytes into the binary, a version.txt in the output
| zip, an autogenerated comment inlined in each HTML/JS/CSS
| file, whatever... _somehow_ , you can embed build info.
| wbl wrote:
| Do another release. You do have a way to track releases
| right?
| staticassertion wrote:
| This sounds like an ops problem. It shouldn't be hard to
| tell what's running in your production environment and
| tie that back to a specific commit.
| rocqua wrote:
| It is! But a lot of ops departments have problems! OPs
| point is that a lot of deployments out there make it
| really hard to remediate this kind of vuln.
|
| Moreover he is arguing that java, as opposed to fat
| binaries with potentially stripped symbols, made things
| solvable. He fears that many companies would be much
| worse off with a big vulnerability in e.g. a rust crate.
| staticassertion wrote:
| Yes, some places suck. They're going to suck either way.
| If their system is so backwards that they need to
| manually patch services because everything is EOL, and
| compiled code will break their workflow, they have
| _plenty_ of other shit to deal with.
|
| I don't think we should optimize for garbage companies
| with garbage practices.
| rocqua wrote:
| I'd say places lie on a spectrum of sucking. With a lot
| of places being very important and far from perfect.
| Those places really should get good security. That means
| designing for security in the common situation rather
| than the situation of 'best practices'.
|
| I think we should optimize security for say the 80th
| percentile company as far as not following best practices
| (i.e. 80% of companies have better practices than what I
| suggest targeting).
| staticassertion wrote:
| In no way is manually writing a patch and then live
| patching the common situation.
|
| The vast majority of cases, and this has been the case
| for decades, will be that you take responsibility for
| your own dependencies, and you let your vendors take
| responsibility for theirs.
| jethro_tell wrote:
| It shouldn't be hard for a dev to update a dependency
| which should trigger an auto build and roll your
| pipeline. Shouldn't be hard for an upstream vendor to
| patch and have that pulled to kick off a new build.
|
| IDK where this idea that no one can know what's in a
| dependency tree is coming from but is not ops best
| practice for more than a decade.
| bigiain wrote:
| > IDK where this idea that no one can know what's in a
| dependency tree is coming from
|
| I'd guess people running commercial non source available
| binaries in production. And outside startups and
| unicorns, that's almost everybody.
|
| How do you find the dependancy tree for your on prem
| Oracle db, or your self hosted Atlassion stuff, or your
| non cloud ServiceNow or PeopleSoft stuff, or your Huawei
| network management stuff, or or or...
|
| And what can you do for any of your cloud hosted saas
| stuff, beyond telling legal "I dunno, their website
| hasn't said whether they're vulnerable to exploit de jour
| yet"?
| staticassertion wrote:
| Presumably you would go ask Oracle, and then patch it
| when they provide a patch.
| lazide wrote:
| Hope you've got a pretty sweet support contract or you're
| going to be waiting awhile.
|
| Depending on the product, maybe forever.
| jethro_tell wrote:
| Why would you have oracle without a super sweet support
| contract? That's the whole point.
| lazide wrote:
| People don't get Oracle because they want to get stuck
| with a super expensive (and still only ok), or super
| insanely expensive (and maybe sweet) support contract.
| They get it because they usually don't know about these
| things, are trying to do something and BLAM they get
| hooked.
|
| Many people don't even realize how long it can take to
| get a real issue fixed even with a super sweet support
| contract. For instance, I once was a wet behind the ears
| Oracle DBA, and when creating a schema i dumbly used a
| feature in the documentation without checking to see how
| long it had been released. Unfortunately, it had been
| released about 6 months before, as I found out later.
|
| It took _6 months_ for them to figure out that it was the
| cause of our sporadic ORA600 errors (aka core dumps), by
| which point we'd already migrated to MySQL for many of
| the workloads because the database was having at best 95%
| reliability because of it.
|
| Once we migrated the table off with the feature, somehow
| our problems ceased, but that was still well before they
| told us what was going on.
| staticassertion wrote:
| My point is that patching third party binaries is
| generally not something companies take responsibility
| for. They wait for vendors to patch the binaries and ship
| them.
| jiggawatts wrote:
| > generally not something companies take responsibility
| for
|
| I have some bad news for you: viruses and hackers don't
| care about your support contracts and the delays they
| cause.
|
| Actually, I tell a lie: the hackers _love_ them.
| staticassertion wrote:
| That's not relevant? The original premise here is
| "compiled languages will make things worse because
| they're harder to patch" except:
|
| a) It's very rare to patch things manually
|
| b) It's very rare to develop patches yourself (for 3rd
| party software)
|
| I'm sure _some_ companies out there with many decades old
| deployments have to do stupid shit like that, but it 's
| hardly the standard.
| lazide wrote:
| Having been on the receiving end of waiting a long time
| to get a patch for a actively being exploited zero day -
| the point is, even if you want to fix it yourself (which
| I desperately did), you can't.
|
| If it's some important piece of software you also can't
| stop or the business stops - well, you and everyone else
| is just going to take it in the pants.
|
| Many enterprises are in exactly this bind. And support
| contracts only get you so far.
| staticassertion wrote:
| I'm losing your point. So, you wanted to patch it, and
| you couldn't, so how is it bad then that a compiled
| binary would make patching more difficult?
| yawaramin wrote:
| Go 1.18 onwards will solve this problem: https://utcc.uto
| ronto.ca/~cks/space/blog/programming/GoVersi...
|
| > The go command now embeds version control information
| in binaries including the currently checked-out revision
| and a flag indicating whether edited or untracked files
| are present.... Additionally, the go command embeds
| information about the build including build and tool tags
| (set with -tags), compiler, assembler, and linker flags
| (like -gcflags), whether cgo was enabled, and if it was,
| the values of the cgo environment variables (like
| CGO_CFLAGS).
| nefitty wrote:
| My list of languages to practice were Scala and Rust.
| Maybe I'll do Scala and Go considering this single point.
| ncmncm wrote:
| In other words, you are not actually interested in
| programming.
| derefr wrote:
| On Windows, the Microsoft compilers and linkers have
| inserted such toolchain/library version metadata into PE
| executables for decades now (the "RICH" header -- https:/
| /bytepointer.com/articles/the_microsoft_rich_header.h...)
| .
|
| At the time that Microsoft started doing it, paranoiacs
| thought it was some kind of anti-piracy software
| fingerprinting mechanism, embedding the compiling
| computer's MAC address or something. But really, it
| serves exactly the same purpose that Golang's effort does
| here: to let you map compiled-artifact back to inputs.
| xxgreg wrote:
| There's also ongoing work to build a dependency
| vulnerability database. This is proposed to be surfaced
| via a built in `go audit` command.
|
| https://go.googlesource.com/proposal/+/master/design/draf
| t-v...
| pabs3 wrote:
| So if you make a commit to the README.md fixing a typo,
| now your build of the source code has a different
| checksum? I wonder why they don't instead hash all the
| source files and then embed that hash.
| Too wrote:
| That doesn't provide a link back to the source, unless
| you store that hash elsewhere. Giving you the problem of
| maintaining a database also.
| yawaramin wrote:
| > So if you make a commit to the README.md fixing a typo,
| now your build of the source code has a different
| checksum?
|
| Yes...? What's the problem?
|
| > why they don't instead hash all the source files and
| then embed that hash.
|
| This is almost exactly what the git sha is. If you're
| arguing that README.md files shouldn't be included in the
| checksum, that's subjective. Many people would argue
| otherwise. And it doesn't hurt to be more accurate than
| less.
| tatersolid wrote:
| In commercial software the customers don't have the source
| files...
| Perseids wrote:
| Well, we're in a thread that started with
|
| > I lay any blame squarely at the feet of IT security of
| large organisations that were entirely unprepared to
| update a widely used dependency that wasn't an operating
| system or a runtime.
|
| If there is the possibility to plan ahead from a
| sufficiently strong bargaining position, you will not end
| up in the situation you described. The seller should
| either continue to support it with a good SLA, or you get
| the source code with a turn-key build system if the
| seller discontinues its support. Or you switch out the
| software if the SLA ends. Should you not be able to agree
| on such terms, I don't think you would have enough
| influence to tell the seller to write their software in
| such a way that dependencies are easy to hotfix/upgrade.
| Spooky23 wrote:
| ROFL.
|
| Ok, after putting the unicorns herd to bed, an event
| happens and congratulations, you now own a license to
| PeopleSoft v.whatever source code. Oh yeah, you also got
| the management platform for your network provider too.
|
| Now what?
| yjftsjthsd-h wrote:
| It's called code escrow and it's a real thing even if
| it's news to you. Although yes, it's usually an insurance
| plan you want to never ever need because _using_ it will
| suck.
| jacquesc wrote:
| Thanks to Log4J developers for turning me off to Java 20yrs ago
| due to their horrible API design and developer experience. Does
| not shock me that this security vulnerability happened, but it
| did shock me that so many people kept using this awful library.
|
| I thus enjoyed a development career (mostly) free from this
| kind of needless pain and suffering.
| intro-b wrote:
| damn i checked ur profile and ur actually based respect
| dang wrote:
| Please don't cross into personal attack. You can make your
| substantive points without swipes, and that's the spirit we
| want here.
|
| https://news.ycombinator.com/newsguidelines.html
| mml wrote:
| The options at the time were even worse.
| Hallucinaut wrote:
| log4j2 has little to do with the log4j you claim turned you
| off 20 years ago and almost no one works directly to the
| logging library API rather than an slf4j facade.
| jacquesc wrote:
| Fair enough. But to me it was the smell of a toxic Java
| ecosystem, where people had to use crappy software even
| though they knew how bad it was.
|
| It took a simple problem and turned it into a clusterfuck
| of configuration and complexity.
| itronitron wrote:
| In many ways it was a trailblazer for npm and the modern
| javascript 'ecosystem'
| ivan_gammel wrote:
| Java ecosystem is rich and much more developed than
| anything else. There are plenty of better solutions on
| the market, both open source and commercial. There exist
| alternatives to log4j for a while, e.g. logback, which is
| the default option for Spring Boot applications - there
| was absolutely no need to use log4j2. It was always a
| free choice.
|
| It is worth mentioning, that any even very basic solution
| always has a possibility to shoot in the leg, and only
| professionalism of the developers of the client code can
| avoid it, so the real reason why this is happening now on
| a massive scale is that Java world is big and has
| probable hundreds of thousands, if not millions of
| developers, who do not have enough knowledge or
| experience to write better code, understanding all the
| possible side effects and other consequences of their
| decisions.
|
| This was going to happen eventually. It will happen
| again, but maybe next time it will be a Python or
| TypeScript library. Would it be right to call their
| ecosystems toxic? I honestly do not know. But let's at
| least have some respect for people who have built it,
| almost for free.
| stefan_ wrote:
| Of course the ecosystem is toxic. Java is literally the
| only ecosystem in the world where people regularly run
| into classloader issues _because of conflicting logging
| libraries_. And when it happens, that is a _smash the
| fucking computer_ moment for the momentous stupidity that
| lead to it.
|
| Yes, it's free. No, the vast majority of users never
| actively opted into it. No, it's not disrespectful to
| suggest that a log library that interpolates parameters
| was never fit for purpose.
| ivan_gammel wrote:
| I've been writing on Java since v1.0, and had the
| mentioned classloader issue probably once - in mid-2000s
| with something on JavaEE.
|
| This class of problems was back then called DLL hell, and
| guess where the ,,DLL" comes from? Hint: not Java. In any
| case, it is quite rare by now and calling ecosystem
| ,,toxic" because of that?.. Pretty sure you haven't seen
| the others long enough.
| jacquesc wrote:
| Why can't we point out that the "default" choice for most
| java shops is one that is massively overcomplicated, a
| nightmare to use, and subject to security disasters like
| we are seeing? This wasn't "unknowable". Using overly
| complicated, poorly designed software is a direct cause.
|
| Why should I show respect for software I hate using and
| that causes problems for the entire tech community? I
| appreciate that they built it for free. I write open
| source code too, and glady accept criticism if I release
| something that sucks (which I do often)
|
| This saga is a lesson in choosing the right tech, and
| switching off libraries built for use cases 100x more
| complex than what 99.9% users need.
| samhw wrote:
| I'm not a Java developer, but I don't quite understand
| your complaint. You haven't used the library in question,
| you don't have any experience beyond the 'smell of the
| ecosystem' two decades ago, but you're opining at length
| on its 'poor design' and proneness to security disasters.
|
| What knowledge do you have, about this library which
| you've never used, which qualifies you to say that it's
| especially prone to security vulnerabilities - other than
| hindsight bias?
| ivan_gammel wrote:
| Most of the Java software is in fact not affected because
| it is using different logging solutions. It is not
| ,,default" as you say and in many cases it was still a
| reasonable choice made by people with a lot of
| experience, because the library isn't bad. It has plenty
| of useful features, and the fact that once in a decade
| someone found a vulnerability in an obscure part of it
| does not mean they have to switch now or reinvent the
| bicycle with own logging. As for respect... did you do
| the code review and security audit of every dependency
| that you use? Do you do it regularly with all updates and
| patches? What makes you confident, that this is not going
| to happen with your favorite tech of choice?
| kbenson wrote:
| > Why should I show respect for software I hate using and
| that causes problems for the entire tech community?
|
| You aren't expected to. You're expected to show respect
| for the medium with which you are expressing those
| thoughts (HN), and the other people you are communicating
| with on that medium, by keeping the discussion civil and
| doing your part to keep it from devolving I to a flame-
| fest.
|
| Almost nobody cares that you have and express dislike for
| this software. They care that you're doing it in a way
| which encourages additional discourse which isn't useful
| and wastes everyone's time.
|
| At extremes it's the difference between staying you think
| a person is immoral or untruthful with reasons and
| calling someone an asshole. One of those is easier to
| have an actual discussion over than the other.
| jacquesc wrote:
| You're right, it was a bad take based on an old,
| strangely emotional experience I remember. Apologies
| ashtonkem wrote:
| Slf4j is one of the two examples I point to for really
| great java design. It's broken down into one jar for the
| no-op interfaces, and other jars for the various
| implementations. This lets library owners instrument their
| libraries for logging without having to make a final
| decision on the actual logging library and its settings.
| It's great.
|
| The other example for those that are curious is micrometer.
| The basic API and the implementations (prometheus, statsd,
| etc.) are separate jars.
| devit wrote:
| Wouldn't really thank them.
|
| They found a way to make several catastrophic mistakes while
| writing a logging solution (one of the most trivial kinds of
| software) in a memory-safe language.
|
| It's surprising they manage to breathe properly and thus
| survive.
| throwaway81523 wrote:
| Is this still about malicious payloads in serialized Java
| packets? I don't use Java so maybe I'm being unjust, but I'm
| smiling slightly because Python went through this 20 years ago
| with regard to the pickle protocol. It was originally intended to
| be ok for transporting hostile data, and it took a fair amount of
| persuasion and a few security incidents to turn the community's
| thinking around about that. ast.literal_eval was invented partly
| for this reason, but the generally preferred way to serialize now
| is usually json.
| layer8 wrote:
| > Is this still about malicious payloads in serialized Java
| packets?
|
| No, it's not.
| zibzab wrote:
| Hopefully this puts an end to those annoying log4j-LOTR memes.
|
| (Unless some hobbits have 3 breakfasts?)
| mongol wrote:
| I have never understood why logging has needed such complex
| frameworks.
| zibzab wrote:
| Because, this is how we do things in Enterprise.
|
| Okay, more seriously... this is why I am currently staying away
| from Rust. Because sometimes I don't even know if the random
| number generation crate I imported for my banking app came from
| Rust team or some random dude in zanzibar.
|
| (Sorry, I know HN loves Rust, so do I. But I currently that the
| cargo/crate system is a bit too easy to use and misuse).
| dreyfan wrote:
| I'm enjoying the Apache Software Exploit Advent Calendar
| ashtonkem wrote:
| I am not. Although one of my direct's decisions to setup a
| parent pom to manage versions across all our apps is paying
| back dividends this week.
| brabel wrote:
| You still need to update the parent POM version in all sub-
| modules. Or do you use version ranges or something like that
| for the parent POM?
| [deleted]
| jacquesm wrote:
| Smart person!
| MikusR wrote:
| They should open source it so the community can improve and fix
| bugs.
| taftster wrote:
| Great idea! I like where you're thinking.
| brabel wrote:
| I thought the reason for the bugs was that they are not getting
| paid to maintain this library?! /s
| aborsy wrote:
| Does anyone know about the quality of crypto implementations in
| Java Cryptographic Extension (JCE), the standard crypto library
| in Java?
|
| We use a product that uses JCE. The log4j vulnerabilities worry
| me about the security of software in Java ecosystem in general.
| layer8 wrote:
| They are generally high quality and implemented by
| cryptographers. Parts of the implementations are certified by
| NIST.
|
| Log4j is not part of the JDK and is an open source project with
| only few regular contributors. There is really no reason to
| infer any correlation to other parts of the Java ecosystem.
| ncmncm wrote:
| "Certified by NIST" absolves people of responsibility without
| actually indicating any property of the code certified.
| staticassertion wrote:
| I don't know anything about it but I will say that people tend
| to audit crypto libraries and tend not to audit logging
| libraries.
| xigency wrote:
| More evidence that user provided data shouldn't be logged at all.
| samhw wrote:
| There are countless ways that this vulnerability could be an
| issue irrespective of whether user-provided data are being
| logged. Attackers achieving RCE on a machine in your
| infrastructure is likely extremely bad for most realistic
| architectures.
| jacquesm wrote:
| I think the main takeaway for all of these is that tricky and
| possibly unexpected behavior should _always_ be explicitly
| enabled. And it 's not the first time that we are learning
| that particular lesson so there really are no excuses.
| jrochkind1 wrote:
| It is pretty valuable to my debugging and general observability
| to log the URLs requested (including query component), which is
| user-provided data. I'm not saying I couldn't maintain my app
| operations without logging URI's including query strings... but
| it would definitely be more challenging.
|
| Then there's the user-agent header, which also can be pretty
| valuable to debugging and other operations. Although this one
| is easier not to log.
|
| In my current logs, I'm seeing attempts to compromise log4j
| vulnerability in both URL query string and user-agent. (Which
| of course I would not know about if I weren't logging them).
| This particular application has no Java involved, so is not
| vulnerable.
| hn_throwaway_99 wrote:
| It's not difficult to take user-entered data and just dump it
| to a log file. I'm pretty positive none of these
| vulnerabilities exist with System.out.println.
|
| The fundamental problem is that, pretty surprisingly, log4j was
| running the same parsing and logic on _user-entered_ strings as
| they did _format_ strings. That 's essentially the root cause
| of all this - log4j shouldn't be attempting to parse this data
| at all.
|
| So many severe vulnerabilities are due to the complexities of
| parsing potentially malicious user input - just look at all the
| severe iMessage bugs. While many other apps _do_ have a
| fundamental requirement to parse this input, there is no reason
| a logging library needs to do it.
| a-dub wrote:
| > It's not difficult to take user-entered data and just dump
| it to a log file. I'm pretty positive none of these
| vulnerabilities exist with System.out.println.
|
| yes, but there can be (and have been) vulnerabilities in
| terminal emulators, terminal multiplexers, text editors,
| databases, web renderers and all manner of tools that can be
| used to store the contents of or view that file. your
| argument makes sense that the logging library should not be
| attempting to parse the log strings, but think about all the
| other millions of lines of code that could potentially try to
| parse that data as well. patch log4j one day and the next you
| find a bug in some colorizer javascript library or in the
| sixel support for the terminal emulator. people add parsing
| _everywhere_.
|
| i think ideally that all user supplied data should be
| sanitized in the strictest possible manner before anything is
| done with it, including logging.
| NavinF wrote:
| > user supplied data should be sanitized in the strictest
| possible manner before anything is done with it, including
| logging
|
| Oh come on, how would you even do that if you don't know
| how the data is going to be interpreted by parsers that
| nobody knows they're using? WAFs have been trying to do
| this since forever, but it's always been security theater
| that separates gullible companies from their money and
| randomly blocks users that use "special" characters like
| slash or quotation marks.
|
| I guess you could base64 encode everything, but that
| wouldn't save you from the terminal bugs you mentioned. At
| some point you're gonna have to undo the "sanitizing" and
| look at the text.
| hn_throwaway_99 wrote:
| So you never browse the web?
| a-dub wrote:
| sure. i don't think i'm articulating my real point well
| enough.
|
| web browsers and all of their components get audited.
| because they process potentially malicious data as their
| core function, they see A LOT of attention in terms of
| hardening.
|
| the point that i'm trying to make is: the amount of
| software that doesn't immediately and apparently touch
| potentially malicious data is absolutely enormous and the
| number of paths that data can take to touch that software
| is even more enormous. yes, one can put forth good
| principles for "building codes" for software and yes, one
| should... but there's an awful lot of software out there
| that is not up to code (where the craft is so young that
| the idea of a code isn't even close to being finalized)
| and it won't be for a very long time.
|
| that being the actual on the ground situation, what is
| one simple thing that individual developers can do to
| help prevent catastrophes? always sanitize inputs.
| moreover, it's much easier to find and audit every place
| where data is accepted from untrusted sources than it is
| to find every place where data may be parsed.
| a-dub wrote:
| sort of raises an interesting question. rust has memory
| safety, but it's still possible to write code that "isn't
| up to code." i wonder if maybe rust+1 will be the first
| language/environment/ecosystem that has application
| development security principles built in. that would
| mean, like, not just taking the compiler forward in terms
| of memory safety, but also codifying a set of principles,
| enforcing those that can be statically checked in the
| compiler with the compiler, and aggressive curation for a
| singular library archive (similar to how some internet
| library archives require unit tests or style, etc).
| yjftsjthsd-h wrote:
| Up until "singular library archive" I would say that
| exists today in the form of Ada and formal methods (and
| both in the form of SPARK)
| dreamcompiler wrote:
| Ther's nothing wrong with logging user-provided data. What's
| wrong is creating a mechanism where user-provided data can be
| interpreted as a program.
| skwirl wrote:
| User data shouldn't be processed at all. You never know where
| there will be a vulnerability. So you turn off logging so you
| have no idea what pages your customers are now requesting
| (phew!) but you can still be vulnerable to SQL injection
| attacks, buffer overflow attacks, etc. when you are processing
| the user data. Best to just reject it all.
| tekknik wrote:
| The server is happier this way as well, finally people have
| gotten off its lawn!
| staticassertion wrote:
| That sounds like it would make debugging very hard for 0
| benefit to anyone not using log4j.
| jacquesm wrote:
| Logging can easily be a legal requirement.
| pmarreck wrote:
| I'm thinking anyone who has to comply with Sarbanes-Oxley,
| for starters, potentially
| AtNightWeCode wrote:
| Welcome to GDPR and hello to trace log everything. :)
| jacquesm wrote:
| This has absolutely nothing to do with the GDPR. In fact,
| the GDPR and various data retention laws are at odds with
| each other which is why the GDPR has a very explicit
| exemption for this.
| AtNightWeCode wrote:
| Not sure what you are answering. If you for instance
| expose an endpoint by accident and then can't show logs
| for the access to it, you are going to have a problem in
| one of those GDPR hearings. The irony of GDPR is that you
| may end up with more logging of user data, not less.
| jacquesm wrote:
| Explain 'one of those GDPR hearings'. I'm pretty familiar
| with the GDPR and everything that comes with it, I've
| never even heard of the term 'GDPR hearing'.
| Kreotiko wrote:
| If information security people would spend the same amount of
| time they are spending complaining about this vulnerability
| trying to educate their organisations on the importance of
| supporting open source solutions they depend on we wouldn't be in
| this situation. I am sick of reading of very senior people
| complaining about the impact this vulnerability has had on their
| week when their companies don't even contribute a penny to open
| source projects.
| dwaite wrote:
| How do I fund ripping out bad ideas from existing open source
| software?
|
| Log4j2 partially exists because of pushback on adding features
| to v1
| nlitened wrote:
| Fund a new fork
| Too wrote:
| Just leads to the xkcd "there are now 14 competing
| standards."
|
| Honestly the biggest problem of package managers is finding
| a reputable package that one can trust.
|
| The best one can do to aid this situation is reviewing,
| vetting and warning of which packages that can be trusted
| and not. Automatic scanners to find code smells and
| vulnerabilities.
|
| "Security rating: 2/10, This package seems to use JNDI
| loading, are you sure you want to continue (y/n)."
| yjftsjthsd-h wrote:
| Humor me: if log4j had 13 (independently developed)
| competitors, wouldn't the impact of this be much lower,
| because many consumers would use different libraries?
| Like genetic diversity as a hedge against disease.
| rini17 wrote:
| Joke's on you: Then any nontrivial project would end up
| depending on half of them, actually increasing attack
| surface.
| gonzo41 wrote:
| I bet they'd all compete on features and probably
| implement the same exploit trying to maintain market
| share.
|
| Logging as it's understood today should ship with most
| STD libraries. Things like logback an log4j really should
| just eventually be rolled into the std.
| bobbyi wrote:
| Why not logback instead? It's existed for more than 10
| years and its stated goal is "picking up where log4j 1.x
| leaves off"
|
| http://logback.qos.ch/
| symlinkk wrote:
| Giving something away for free and then turning around and
| guilting people into paying for it is wrong.
| ncmncm wrote:
| That seems to be an opinion. Opinions vary. Mine does.
| yjftsjthsd-h wrote:
| Using something for free and expecting support is hilarious.
| akeck wrote:
| I read a post recently that said part of the open source
| funding issue comes down to a financial "structure" mismatch.
| Open source folks are often individuals or loose collections of
| individuals. Sometimes they have things like Patreons, ask to
| receive crypto currencies, etc. Companies don't/can't provide
| funding via these types of structures, so the writer of the
| post ended up incorporating a company, setting up a mercantile
| bank account, commercial payment processing, etc., so companies
| would be comfortable funding his efforts. Fortunately, this
| path is lot easier lately with services like Stripe, etc.
| ldng wrote:
| They must adapt. Period. And don't worry, when they have to,
| they do find solutions. The thing is, with open source, you
| don't have to, worse, the vast majority of enterprises rely
| _because_ you don 't need to make any effort.
| wiether wrote:
| Solving this issue isn't one of the goals of
| https://opencollective.com/ ?
| hotpotamus wrote:
| I have a feeling if a company wrote a large enough check,
| they'd figure out how to cash it.
| ricardobayes wrote:
| Well, wrte it to whom? No legal structure is what the
| comment was partly about.
| themerone wrote:
| My Fortune 500 company has to pay a third party to pay for
| Google Maps, because we can't pay a supplier with a credit
| card.
|
| If we can't find a way to pay Google, there is no hope for
| some random developer in Nebraska.
| macinjosh wrote:
| Just because a dev lives in the Midwest doesn't mean
| they're idiots compared to coastal elites. The
| hate/mockery/minimizing of anyone not from a major coastal
| city is really gross. Screw you.
| [deleted]
| themerone wrote:
| I am a developer in the midwest, and maybe if you hadn't
| completely missed my poin you wouldn't be such a jerk.
|
| Nebraska is a reference to an XKCD Comic
| (https://xkcd.com/2347/). The idea is that there are
| nameless people all over the world contributing to
| critical open source infrastructure.
|
| Back to the point.
|
| Running Google Maps on one of my low traffic sites, would
| generate at most, several dollars a month in charges. It
| would be a rounding error in our budget, and we can't pay
| it. Our purchasing rules are not flexible enough to work
| with google.
|
| Paying Google is hard enough, an independent oss
| developer with no company would be impossible. I would
| get laughed at if I suggested throwing a few bucks at a
| developers Patreon.
| lazide wrote:
| How does your company handle coffee? Surely there is SOME
| mechanism for small incidental charges, or how could your
| company even exist?
| yjftsjthsd-h wrote:
| Coffee company sends a monthly invoice.
| lazide wrote:
| So no Starbucks with a client on sales calls I guess?
| themerone wrote:
| Nebraska is a reference to https://xkcd.com/2347/
| ytdytvhxgydvhh wrote:
| That feels backwards. Little shops are flexible and can
| probably do what needs to be done. Lumbering behemoths have
| (too many) rules and restrict themselves (often)
| unnecessarily.
| solididiot wrote:
| Never understood this. Companies that make billions base their
| operations and products on free open source code. Free as in
| free beer. Then they complain about them not being...perfect.
|
| OTO, people write software for free and licence it in a way
| that allows above companies to use it without giving them a
| single penny. And then they complain about not getting a penny
| whilst somebody else makes billions.
|
| This whole thing is just incomprehensible to me.
| UncleMeat wrote:
| Would more funding have actually uncovered this?
| darkr wrote:
| Not in and of itself.. However if part of that funding was
| used to cover formal (external) audits, investment in
| fuzzing/automated security testing etc.. then it might well
| have done
| svieira wrote:
| "Might have" is the operative word. In a world where
| Project Zero's _This shouldn't have happened_ [1] was also
| published this month (buffer overflow on NSS), I'm less
| certain. Quoted from the article:
|
| ---- This wasn't a process failure, the vendor did
| everything right. Mozilla has a mature, world-class
| security team. They pioneered bug bounties, invest in
| memory safety, fuzzing and test coverage.
|
| NSS was one of the very first projects included with oss-
| fuzz, it was officially supported since at least October
| 2014. Mozilla also fuzz NSS themselves with libFuzzer, and
| have contributed their own mutator collection and distilled
| coverage corpus. There is an extensive testsuite, and
| nightly ASAN builds.
|
| I'm generally skeptical of static analysis, but this seems
| like a simple missing bounds check that should be easy to
| find. Coverity has been monitoring NSS since at least
| December 2008, and also appears to have failed to discover
| this.
|
| Until 2015, Google Chrome used NSS, and maintained their
| own testsuite and fuzzing infrastructure independent of
| Mozilla. Today, Chrome platforms use BoringSSL, but the NSS
| port is still maintained.
|
| Did Mozilla have good test coverage for the vulnerable
| areas? YES.
|
| Did Mozilla/chrome/oss-fuzz have relevant inputs in their
| fuzz corpus? YES.
|
| Is there a mutator capable of extending ASN1_ITEMs? YES.
|
| Is this an intra-object overflow, or other form of
| corruption that ASAN would have difficulty detecting? NO,
| it's a textbook buffer overflow that ASAN can easily
| detect. ----
|
| [1]: https://googleprojectzero.blogspot.com/2021/12/this-
| shouldnt...
| layer8 wrote:
| That's a really big "if" though.
| xyproto wrote:
| No. Funded projects also contains bugs.
| yawaramin wrote:
| Would less funding have uncovered it?
| anchpop wrote:
| probably not but i don't think anyone suggested less
| funding
| yawaramin wrote:
| Well I don't think anyone suggested anything actually
| helpful or relevant, so....
| rileymat2 wrote:
| Would less funding have prevented the feature being
| implemented? Maybe?
| yawaramin wrote:
| There's no way to argue that _less than zero_ funding
| would have prevented the log4j maintainers from _merging
| a patch_ that was sent to them by a third party.
| natded wrote:
| > educate their organisations on the importance of supporting
| open source solutions
|
| Githubs of the world could just gate downloads, pull requests
| etc. behind a payment to see what is the real valuation of open
| source software; I imagine it'd mostly settle around $0
| excluding couple of big projects.
| samhw wrote:
| Up until the semicolon, this was actually a decent point. I
| can see a future for high-quality open-source software hosted
| on an 'app store' which requires a small payment for use - or
| for commercial use, perhaps.
| ahtihn wrote:
| And how are you going to prevent someone from just hosting
| a mirror for free?
| jeffbee wrote:
| I'm baffled by these claims that we should compensate the
| authors and maintainers of what has been conclusively shown to
| be software too dangerous to exist. It has the same problem as
| calls to pay the maintainers of openssl. No amount of money is
| going to impart good taste and best practices onto these
| projects. The uncomfortable truth is these libraries need
| scratch rewrites with better authors.
| RHSeeger wrote:
| Saying things like "I don't like this software because
| <concrete reason>" is reasonable; it can be considered
| constructive criticism, especially if you include a way it
| could be done better. Heck, even just "I think it's too
| complicated to use, and the underlying code base needed to
| support that complexity lends itself to bugs that can be
| exploited" would be better.
|
| Saying things like "No amount of money is going to impart
| good taste and best practices onto these projects" is
| insulting the people who wrote it with no real basis. It's
| not helpful and it makes you (and some other people in the
| thread) sound like a jerk.
| def_true_false wrote:
| If only being nice to people prevented security bugs, eh?
| virtual_void wrote:
| Are you proposing that being rude does?
| [deleted]
| samhw wrote:
| > The uncomfortable truth is these libraries need scratch
| rewrites with better authors.
|
| If that's true, do you think that having money might possibly
| be helpful to that effort?
| ksaj wrote:
| One of the maintainers in question holds two jobs to live.
| Having money might allow him more time to test and retest
| and update the software more expediently.
| dwaite wrote:
| That wouldn't be funding log4j2 though (from the 'better
| authors' part). That would most likely be an internal
| effort (devs you trust) published as a new open source
| logging framework in a sea of existing ones.
| staticassertion wrote:
| Honestly, absolutely not, and I think money would be
| harmful. Lots of software doesn't need a lot of work. It
| can be _simple_. When people get paid to solve problems
| there 's an incentive to add more code, to add new domains,
| etc.
|
| Use the standard java logger and move on, it's that easy.
| tekknik wrote:
| Where's my incentive to work the weekend supporting a
| project after a long week of supporting my production
| code? If we want solid and secure OSS, we gotta pay
| someone!
| staticassertion wrote:
| Presumably it's the same incentive that pushed you to
| write it the first time.
|
| If you don't want to work on it anymore that's OK too.
| Open source libraries are generally not held to
| particularly stringent requirements by the vast majority
| of people - perhaps accepting an obvious patch in a
| critical situation, at most.
|
| You obviously _can_ pay people. They 're called
| employees. The thing about employees is that your money
| translates to business value, which is much broader than
| a single library. This means that when a single library
| is done the developer doesn't have to think "ok but
| what's next for this library" and start adding more code.
| They can stop and move on to other tasks.
|
| Paying for a single library is _nice_. I support multiple
| open source developers because I appreciate their work. I
| recommend everyone does so.
|
| I don't at all believe it's going to help security
| though.
| TingPing wrote:
| OpenSSL at least has significant value, handling logs is
| usually an easy task.
| pclmulqdq wrote:
| The unfortunate thing about scratch rewrites is that they are
| not a good business. Open Source has conditioned people to
| think that libraries should be free. It's not clear to me
| that the alternative where libraries are obscenely expensive
| (like in the hardware world, where a hash table is a $100k
| investment) is any better. Nobody has ever built a successful
| company charging between $0 and $10,000 for a library. That
| is just how people see things, and unfortunately it is
| extremely hard to change.
| tjalfi wrote:
| Some platforms have reasonable markets for libraries;
| Advanced Messaging Systems[0], DevExpress[1] and Aspose[2]
| are three examples of companies that successfully sell
| software libraries for .NET, Java, and Office.
|
| [0] https://www.dimastr.com/redemption/home.htm
|
| [1] https://www.devexpress.com/
|
| [2] https://www.aspose.com/
| kzrdude wrote:
| Maybe we should tell them to engage with open source, not
| "support it". Companies should look for a way to spend work
| hours on open source, not finding someone to send money.
| throwaway19937 wrote:
| If it's an enterprise
| (https://zwischenzugs.com/2018/10/02/why-are-enterprises-
| so-s...) the latter will be a much easier sell to management.
| Paying for support is a given but contributing developer time
| will require many approvals from different departments if it
| can be done at all.
| l5ymep wrote:
| Convincing project managers to tackle tech debt is hard
| enough. Contributing to open source won't even reach the
| middle ear unfortunately.
| tekknik wrote:
| Let's not even get legal involved...
| throwaway19937 wrote:
| > If information security people would spend the same amount of
| time they are spending complaining about this vulnerability
| trying to educate their organisations on the importance of
| supporting open source solutions they depend on we wouldn't be
| in this situation.
|
| Security teams are stuck with securing the tire fire; they
| didn't choose the library or platform. If anyone should be
| advocating for supporting open source, it's the developers who
| benefit by using open source libraries.
| r-w wrote:
| I'm not interested in kicking the can down the lane, I just
| want this stuff to be sustainable.
| alkonaut wrote:
| This isn't a RCE right? Compared to the previous one I wouldn't
| call a denial of service high severity at this point. It might
| have been high severity two weeks ago.
|
| But to be clear: nested/recursive template expansion and
| expansion of user provided strings were never _by design_ ,
| correct, and the removal shouldn't just be by configuration or
| considered breaking - it should simply be corrected (just like
| the jndi should be removed and not even be optionally possible)?
| kafkaIncarnate wrote:
| Might want to read this thread:
|
| https://issues.apache.org/jira/browse/LOG4J2-3230
|
| It's pretty interesting technically, and it's not really
| affecting 2.16.0 much except in weird edge-cases.
| junon wrote:
| I maintain `debug` for Node.js and the Log4J devs are living my
| worst nightmare when it comes to that library. It's used just
| about everywhere and any sort of zero-day would have far reaching
| implications.
|
| Hugs for all of you. This can't be fun.
| tomohawk wrote:
| It's been a minute since I primarily developed in Java, but back
| then we had switched away from log4j in favor of slf4j / logback.
| These were written by the author of log4j. We switched because
| they were significantly simpler than log4j.
|
| Switching over merely involved changing the jar files we used,
| and redoing the config.
| ncmncm wrote:
| The most reliable place to find new bugs and security holes is
| next to others you just found out about.
|
| Bugs come in bunches. If you have fixed just one or two, more
| lurk _right there_. And, anyplace else that coder worked. The
| more you have found, the more remain to be found. Look at other
| places that coder worked that week. Or year.
| post-it wrote:
| Your implication that specific bad coders are responsible for
| bugs is dangerous. Anyone can write a bug, and thinking that
| you're too good to do it is cocky.
| ncmncm wrote:
| Every bug is coded by a specific coder. We know there are
| huge differences in productivity of different coders. We know
| they repeat mistakes. Thus, the best place to look for
| another bug is in other code the same one wrote.
|
| But looking in your own code is probably next best. Ask
| yourself: if there is a bug somewhere in my code, where would
| it be? You may be surprised at how immediately the answer
| comes to mind. Look there.
| kevingadd wrote:
| Yes, but when a coder makes a specific mistake, it's likely
| they made it elsewhere. Similarly, when their mistake made it
| past code review, it likely made it past review elsewhere.
| They're not the only person making that mistake, but their
| diffs are a great place to start looking.
|
| A big subset of mistakes people make when writing software
| are the result of people not having the same understanding of
| a concept, policy or practice.
| extheat wrote:
| It really depends on the subject area of the code in
| specific. "Mistake" is a very broad term, in most cases
| when it's used it's not the implication that the writer
| didn't know what they were doing, rather they wrote
| erroneous code--which ultimately everyone does at some
| point. What it is however indicative of, is a lack of
| thorough testing and review that could have potentially
| caught the error. Obscure errors in places such as parsers,
| rendering logic, complex algorithms, memory management,
| etc. are particularly tough areas because building a good
| exhaustive testing workflow can be a challenge.
|
| Even a good writer will make mistakes, otherwise we
| wouldn't need proofreaders.
| nimrody wrote:
| It's amazing that this library is used by so many large
| enterprises yet none of them made an effort to thoroughly audit
| the code (actually, it looks like Alibaba reported the first one.
| But the point stands that the code was used for so long and by so
| many.)
| [deleted]
| rukuu001 wrote:
| I did this _years_ ago with an open-source and reasonably
| popular non-Apache library. The results were so horrifying my
| boss quickly concluded such audits were outside our
| responsibility and the time I spent on the audit was written
| off.
|
| It was especially difficult for us because we'd shipped so much
| code that used the library, and replacing the library was
| unthinkable.
| mastax wrote:
| Don't ask a question if you won't like the answer.
| skottk wrote:
| I assume at this point that this library has been audited many,
| many times by skilled professionals, none of whom thought to
| abuse the template + JNDI + LDAP chain in this way.
|
| It's just not as simple as "security audit finds all the
| vulnerabilities, then you fix them." You invest X in the
| review, you get the results that X/(hourly rate) finds. This is
| a lot of software with a ton of configurability-- that's a lot
| of variations to review and test.
|
| Now that someone found the first lump of gold and gave it away,
| there are thousands of eyes searching for the next one. These
| recent findings are all abuses of this same chain of
| functionality, just along different sets of settings. In
| another month we might have half-a dozen more of varying
| severity and scope. That _still_ won't prove that the overall
| library is then safe, but we will probably have a a little more
| confidence in this particular bit of crazy template formatting
| flexibility. Maybe not as much as we had had three weeks ago,
| but more than we do now.
| _ph_ wrote:
| Thats the point. Giving back money for the libraries used might
| not fix anything and can be difficult in many cases. But the
| most and important way of "giving back" would be testing and
| review of the libraries used. If especially the large companies
| using this library had used some of the dev hours for testing
| and review, the bug should have been found early on.
| staticassertion wrote:
| Code auditing is extremely expensive. I recently did some
| napkin math and my estimate is that for our relatively small
| project (a company of ~10) it would cost between 2 and 12
| million dollars to have our rust dependencies audited, with
| Rust making up ~60% of our codebase.
|
| And that's a point in time audit. To maintain that value we'd
| have to redo the audit periodically.
|
| It's just not gonna happen.
| draw_down wrote:
| This is getting pretty expensive too, though.
| layer8 wrote:
| Dealing with the current vulnerabilities is very likely cheaper
| than auditing each and every dependency with the thoroughness
| needed to actually catch those vulnerabilities. Also, nobody
| within the by affected enterprises gets blamed for those
| external vulnerabilities, nor are the enterprises liable for
| preventing them, so there is little incentive to prevent them
| proactively.
| hirako2000 wrote:
| Each enterprise doesn't only use this common library, it uses a
| few more hundreds. If the point of pulling libraries is to save
| tremendous time in not having to write them, I can assure you
| managers aren't keen in wasting their gains in looking deeply
| into what's inside. Spotting the flaws, even with careful
| reviewing is still unlikely to happen, and would benefit the
| community at wide over their selfish interests.Enterprises use
| of open source doesn't adhere to the open source etiquette.
| marcus0x62 wrote:
| Not only are they not interested in spending time auditing
| code, your typical enterprise lacks the skillset to do so
| effectively. They couldn't do it if they wanted to, short of
| contracting it out.
| AzzieElbab wrote:
| Omicron variant?
| gundmc wrote:
| At least this one is "only" denial of service and not another
| RCE?
| layer8 wrote:
| Yes, and it's only applicable in very specific and likely quite
| rare circumstances. If not for the initial RCE vulnerability,
| no one would blink an eye.
| [deleted]
| peterkelly wrote:
| Here's what I don't get: What is so complicated about logging
| that you can't implement the necessary functionality in your own
| codebase, and instead have to adopt a third-party library?
|
| The latter option represents a liability, which in this case (as
| with others) has shown can be a tremendous risk. Is the time
| saving really worth it, at the cost of risking disasters like
| this?
|
| I get that the tradeoff it is worth it for complicated things
| (e.g. crypto libraries). But logging, really?
|
| Software development culture today is too quick to adopt a huge
| tree of dependencies of unknown quality, rather than thinking
| about how to minimize dependencies to only those truly necessary.
| The leftpad fiasco was but an extreme example of this, but I see
| it all the time, and it seems probable that there are hundreds
| (maybe even thousands) of similarly severe problems out there in
| widely used dependencies that we just don't know about yet.
| hulitu wrote:
| Why bother to implement something when someone else did it ?
| Why bother thinking to check the quality of the code when
| someone else did it ?
| AtNightWeCode wrote:
| It is not complicated. Structured or semi-structured logging is
| the only way to log things and still be compliant with all the
| demands.
|
| A lot of these libs separate logging into layouts and
| parameters. If you log input parameters as layouts. You might
| break the logging. I have seen this at every company I worked
| with.
|
| The only really surprising thing about log4j is the remote code
| execution.
| fulafel wrote:
| What kinds of demands are you thinking about?
| intro-b wrote:
| bc log4j isn't some random library people found on the street
| it's been used in industry forever lol
|
| yea literally any library/dependency can introduce risk,
| including stuff coded internally for those purposes
| ta988 wrote:
| Problem comes when you have a lot of various parts that want to
| log. Lets say you have a message processing system getting
| messages from http and throwing that in a db and keeping counts
| on an external API. Now every module can use a log library and
| you, final user of the system can decide where the db log goes,
| where the http client debug message go, how much detail you
| want on any of these etc. So in the world of small applications
| and services, you can come up with a simple logging function.
| On large projects it is much more complicated.
| cnorthwood wrote:
| because Log4J is also used by libraries. Which means you can
| configure the log level and capture logs from libraries and
| dependencies you use in a global configuration.
| doikor wrote:
| You really shouldn't use log4j in libraries. Instead use
| slf4j in a library and log4j (or whatever implementation for
| the facade you prefer) in the application using the library.
| isbvhodnvemrwvn wrote:
| Tell that to thousands of existing libraries.
| doikor wrote:
| Main thing is being able to share the logging stuff between
| libraries so something like slf4j and then various actual
| implementations of that facade (which log4j is one of) in jvm
| land.
|
| There is also lots of stuff you might want a logging library to
| do that while not very hard would be annoying to do yourself
| for every project like log file rotation, switching log level
| while the application is running, optimizations (like in
| c/c++/rust you might compile for INFO level and drop all the
| DEBUG and TRACE level logging from the binary), different log
| levels for different parts of the app (including dependencies),
| etc
| ashtonkem wrote:
| > Here's what I don't get: What is so complicated about logging
| that you can't implement the necessary functionality in your
| own codebase, and instead have to adopt a third-party library?
|
| Transitive dependencies for one thing. I rather enjoy having my
| DB driver, connection pool, kafka client, and web framework all
| emit their own logs, which is not possible if I wrote my own
| logging library.
|
| > I get that the tradeoff it is worth it for complicated things
| (e.g. crypto libraries). But logging, really?
|
| Logging at scale is more complicated than people give it credit
| for. Personally I've had my services suffer noticeable
| performance degredations due to poor logging framework config.
| We've ended up with a fairly complicated logging config that
| allows us to carefully balance collecting as much developer
| useful information as possible against the need to drop
| excessive logs during periods of heavy activity. Oh, and it's
| gotta be converted into JSON for aggregation into our
| observability tools, and decorated appropriately so that our
| logs can be correlated with our application traces, because
| that is incredibly useful during incidents.
|
| Oh, and it all needs to work no matter what threading
| configuration you throw at it. That matters a lot too.
|
| Do we _need_ all those things? Arguably no, but then again they
| are helpful tools to have. Without them, the team would have a
| harder time developing on our systems. You have to understand
| the use case that people need to fulfill before dismissing the
| entire thing as unnecessary and pointless.
|
| > Software development culture today is too quick to adopt a
| huge tree of dependencies of unknown quality, rather than
| thinking about how to minimize dependencies to only those truly
| necessary. The leftpad fiasco was but an extreme example of
| this, but I see it all the time, and it seems probable that
| there are hundreds (maybe even thousands) of similarly severe
| problems out there in widely used dependencies that we just
| don't know about yet.
|
| I see this argument a lot, and I think it's a great example of
| "grass is greener" syndrome. People don't think through the
| practical consequences of drastically reducing their library
| usage. I personally worked on a project that had basically no
| external libraries due to the language choice, and we wasted
| inordinate amounts of time trying to chase down bugs we wrote
| in things like our database library, localization library, and
| yes, our logging library. Looking back I can say with very
| strong confidence that what we were doing was a massive waste
| of the company's resources, and rewriting it into Java or C#
| would have been the right call, even with the complexities
| those ecosystems bring to the table.
|
| Yes, using external libraries comes with tradeoffs. Fixing bugs
| like this is one unpleasant possibility. But let's be clear
| that the alternative is a massive drop in developer
| productivity to write basic functionality that already is
| implemented elsewhere. Having been on both sides of this
| equation, I can confidently say that you'll spend way less time
| fixing security issues as they arise than you will reinventing
| the metaphorical wheel.
|
| (I do agree that leftpad is an example of going way too far
| with libraries, but I think the library culture for NPM and the
| Java ecosystem are quite different).
| nostrademons wrote:
| It's because your program probably depends upon other
| libraries, which all need to log their own info, and your life
| will be significantly easier if you can view, save, and control
| logging from one centralized place at the top of your program.
| If every library rolled their own logging, configuration would
| be a huge mess, and your program would need to specify log
| files/formats/levels with each API call (or at least on API
| initialization).
|
| Logging facades like slf4j help a lot, but then you still need
| to implement the required interfaces. For something like the
| specific Log4J CVE, things would be significantly _worse_ if it
| weren 't centralized in a library. They have a bug in
| deserialization when talking to LDAP servers, allowing a
| compromised LDAP server to execute code remotely. It's likely
| that a subset of libraries would likely have rolled this
| functionality independently, and then you have many libraries
| that all need to be patched and updated instead of one.
| pimlottc wrote:
| It's worth noting that log4j does a lot more than, say,
| JavaScript's console.log function. It's dynamically
| configurable, supports a wide range of output formats and
| destinations, easily filterable, and highly optimized to reduce
| performance impact (in part through delayed evaluation and
| multi-threading).
|
| Rolling your own logging library to reimplement it is not
| trivial, even if you ignore the more esoteric output formats.
|
| Sure, in retrospect they went too far with some of the dynamic
| logging features in log4j2, but it is no exaggeration to say
| that log4j revolutionized logging in the Java world when it
| first came out, and heavily shaped later solutions like logback
| and java.util.logging.
| marcosdumay wrote:
| Yeah, doing so much more is a bug, not a feature.
|
| > It's dynamically configurable
|
| It's a logging library. You don't reconfigure logging, you
| just make another logger and use it. And by the way, you take
| the configurations from your global configuration system,
| "dynamic configurable" is a feature of the configuration
| system, not of the logging.
|
| > supports a wide range of output formats and destinations
|
| Yeah, like network aware templates. (But the destinations are
| a feature, multiple destinations are one of the few things a
| logging library should support.)
|
| Despite what people claim, the Java culture of loving
| complexity never did go away.
| ashtonkem wrote:
| > It's a logging library. You don't reconfigure logging,
| you just make another logger and use it.
|
| Prior company had a shared plugin that let you turn up the
| logging for a given logger remotely for 30 minutes (it self
| reset). Extremely helpful for incident debugging.
|
| Now we can have an interesting discussion about whether
| that capability is worth the complexity, but that's a very
| different discussion than unilaterally declaring that other
| people's use cases just don't exist.
|
| > Despite what people claim, the Java culture of loving
| complexity never did go away.
|
| Do you develop in Java? Because I do, and your takes don't
| match my daily experience one bit. My experience going from
| Spring 3 to Spring Boot 2 is in fact one of reducing
| complexity (goodbye XML bean configuration!) and to more
| "it just works" situations than before.
| marcosdumay wrote:
| > Prior company had a shared plugin that let you turn up
| the logging for a given logger remotely for 30 minutes
| (it self reset).
|
| And you really believe this should be a feature of you
| logging library, and not of you configuration system?
| lazide wrote:
| How is your logging system going to get that new
| configuration without restarting if it doesn't support
| dynamic reconfiguration?
| tekknik wrote:
| You log and collect everything, then use your log
| aggregator to filter the appropriate content.
| isbvhodnvemrwvn wrote:
| This turns gigabytes per hour of logs to hundreds of
| terabytes per hour during normal operation, in many cases
| it's an inferior solution for problems which pop up in a
| fraction of infrastructure several times a month.
| marcosdumay wrote:
| Why is there any problem in restarting your logging
| system? There is nothing in a logging system that could
| be harmed by a restart. Restarting it is exactly how you
| should handle changes, because it deals with plenty of
| stuff that isn't reconfigurable on a lower level.
|
| The only way to make a "reconfigurable" logging system is
| by having a stable facade that restarts everything behind
| it. But that extra layer does not belong on a logging
| system, there are plenty of other things you need to make
| "reconfigurable", and now you are adding an extra layer
| into each of them, instead of having only one on your
| main entry point.
|
| Pushing that feature down into the logging system is the
| choice that maximizes complexity and minimizes
| functionality.
| ashtonkem wrote:
| You act as if "restarting" the logging system is easy and
| doesn't require the same facade that you decry for a
| reconfigurable one. With dependency injection, there are
| loggers everywhere in the system, possibly in sections of
| the code you didn't even rewrite. Restarting such a
| system while continuing to run everything else is just as
| complex as reconfiguring, possibly more complex.
|
| Also, I noticed you didn't answer my question about
| whether you're a Java developer.
| marcosdumay wrote:
| > about whether you're a Java developer
|
| I was once, but haven't used it professionally for a
| while. It's on my "I'd rather not" list, but not strongly
| so.
|
| Oh, and now that you described it with so many details,
| the attachment to complexity does look more like
| addiction than love.
| lazide wrote:
| If you turn on debug logging to see why somewhere weird
| is happening, and the only way to do that is to
| completely restart the entire system - good luck figuring
| out why a large class of weird things happen!
| TedDoesntTalk wrote:
| > wide range of output formats
|
| Not to mention a wide range of destinations, too. Want to log
| to a database table? You're covered. What about log messages
| to a chat server (xmpp, slack) or email but only fatal
| errors? You're covered.
|
| Now roll your own logging system that supports those
| destinations safely and get back to me to do pen testing. I
| bet we'll find some vulns in your code.
| tekknik wrote:
| This is the job of a log collector, not a log
| producer/library. The logging library itself should have no
| mechanism for shipping files anywhere except a file /
| stdout / stderr.
| alextheparrot wrote:
| Consistency of configuration is a big thing -- I can configure
| my Spark, internal framework, and application logs in the same
| file, for example. That's actually one thing I liked about
| Java, how easy configuring logging via Log4j was and how widely
| adopted it was by most dependencies.
|
| In terms of features, one that comes to mind log rotation.
|
| I dunno, I suppose all together I'd answer your third paragraph
| to the affirmative -- I'm more productive and the libraries
| improve significantly within crucibles like this.
| scrame wrote:
| The issue is that java did not initially ship with logging
| capabilities, and by the time it was part of the java.util
| package, a handful of logging systems had already been
| introduced by library vendors. JBoss, Jakarta Commons, Log4j
| and ultimately slf4j all were introduced to address these
| shortcomings.
|
| The larger issue is in javas dependency system are built with
| dependencies on these libraries, so if there is no commons
| logging (or bridge, ala slf4j), then the program won't run.
|
| There are a few other techinical traps with the jvm, like using
| system.out logging can cause issues with threaded code, or
| having one giant static logger being imported all over the
| place conveniently (this is before inversion-of-control became
| the dominant paradigm) had a few shortcomings, especially as
| programs grew. Ultimately, though, if you were going to use
| some third party libraries or app servers, you were almost
| certainly going to get pulled into the commons-logging vortex,
| which means you're now configuring that AND your homespun
| logger.
|
| There is also the case for things like SOLR, where it might be
| just being used out of the box, but the distribution includes
| the affected JARs, even if you're just posting and retrieving
| documents with a python script doing print-logging.
|
| Honestly, I didn't even know log4j2 was a thing, I've moved as
| much over to slf4j/logback as I could ages ago because of the
| madness of the JDK logging ecosystem.
| tomohawk wrote:
| It's worse than that.
|
| When Java finally added a logging library, they added one
| that no-one had ever heard of before, and which had
| fundamental flaws. That's why no-one uses it. At the time,
| log4j existed and was in wide use, but was not represented on
| the committee.
|
| The author of log4j eventually decided to go a different way
| and created slf4j / logback. This offering is compatible with
| log4j and considerably simpler in many ways.
|
| Having not been in the Java world for many years, I was
| surprised log4j was still in widespread use. But maybe I
| shouldn't be surprised - whenever we have to interact with
| Java teams, we always have to tamp down on the tendency
| towards complexity.
| yjftsjthsd-h wrote:
| > Having not been in the Java world for many years, I was
| surprised log4j was still in widespread use.
|
| It's Java; the ecosystem is, politely, rather conservative
| and slow moving.
| hirako2000 wrote:
| Very well explained. About log4j2, I think it came to exist
| because log4j fell far behind logback, and many veteran dev
| didn't like the inherent push for slf4j over logback. While
| logback can be used standalone, snobs would yell at you. And
| since figuring out how to config slf4j over it can easily be
| less than half a day of work, many simply drooped in log4j2
| and told snobs to go away.
|
| Note: In case someone is tempted to say that it is trivial to
| plug slf4j over logback, it is for greenfield work. Replacing
| an existing logging library, or the use of multiple logging
| libraries in favor of slf4j over one lib is a confusing
| matter, mostly because the error thrown at runtime are
| misleading.
| dakra wrote:
| I found this a good overview of the history/current status of
| Java logger libraries
| https://lambdaisland.com/blog/2020-06-12-logging-in-
| clojure-...
| dragonwriter wrote:
| > What is so complicated about logging that you can't implement
| the necessary functionality in your own codebase, and instead
| have to adopt a third-party library?
|
| (1) Coordinating with dependencies requires at a minimum a
| common API (which, in a statically typed language, means a
| library providing at least the interface).
|
| (2) Configurability to log to different destinations and with
| different formats, which you eventually tend to want on major
| or long-lived projects, can be fiddly, as can the specifics of
| different formats and targets. Each of the bits is simple, but
| added up it's a lot over time, and reinventing the wheel, the
| axle, the tire, and the air pump for it just doesn't make a lot
| of sense. Especially given the need for a library for #1, it's
| best to get the core functionality together with it, and the
| peripheral pieces either together or as plugins, and only
| solving unique problems, if you have them.
| tomohawk wrote:
| We asked the same thing about log4j over 10 years ago. We spent
| an hour or two and switched to slf4j / logback. It's written by
| the original author of log4j, and has a compatible api. Much
| simpler and much more understandable.
| linuxhansl wrote:
| I think the mere fact that use of log4j is so wide-spread
| answers your first question. It's not that simple to do it
| (well) yourself.
|
| That said, I do agree that there is a problem with large
| dependency trees. It seems building a simple hello-world app
| with Maven ends up downloading about 1/2 the internet. :)
| johnisgood wrote:
| Whenever I write C, the goal is to have no or very few
| dependencies. When I see projects written in C, one of the
| features is that it has no dependencies. This is a good thing.
| I do not know why we are steering away from this.
| sahila wrote:
| It's pretty easy to understand really; it allows engineers to
| focus on other things and do more. It's not nefarious or
| because people are always lazy, but the choice between
| spending time on a business feature or a logging library when
| one exists that is widely used is hard to sell.
| [deleted]
| ahtihn wrote:
| Dependency management in C is absolutely awful, that's why
| "no dependencies" is a "feature"
| johnisgood wrote:
| I have a different experience. I use my Linux
| distribution's (Arch Linux) package manager.
| WFHRenaissance wrote:
| > implement the necessary functionality in your own codebase
|
| Sir, you might have just extended all of our careers by at
| least a decade. We salute you.
|
| I agree that developers are too quick to put themselves at the
| mercy of 3rd parties, but it's a risk-dollar trade off we'll
| continue to make because reinventing the wheel each time would
| sap our margins.
| whartung wrote:
| > Software development culture today is too quick to adopt a
| huge tree of dependencies of unknown quality,
|
| Log4J has been around 20 years. Log4J inspired and was lifted
| into the JDK. JDK logging is essentially an inspired copy of
| Log4J. The Sun coders were no better or worse than the Apache
| coders. I use JDK logging personally simply because it's one
| less dependency. JDK logging hasn't change since JDK 1.4, and
| is weaker than, well, pretty much anything else. But, with a
| simple wrapper I've used for 15 years, it does most everything
| I want from a logger.
|
| That said.
|
| Log4J has been, and is still, a boon to the Java community.
| Arguably, Log4J is the root of a tree of vast array of logging
| frameworks, across languages. Java server developers
| essentially live and die by their logs. It's routine for
| developers in dark rooms with screen lit faces to pouring
| through logs with endless stack traces. Thank heavens for Java
| stack traces.
|
| Logging is part and parcel to the Java server side experience,
| and even the client side, and we can place much of that on the
| shoulders of giants like Log4J because it made logging easy and
| set the stage. It's so helpful, so useful, so flexible
| (obviously, perhaps, a bit too flexible), and so powerful.
|
| Because Log4J inspired the other loggers like the JDK Logger,
| we have logging shims. Shims like Commons Logging, that act as
| intermediaries that can have adapters written so that we can
| use other logging frameworks. Log4J was one of the first, was,
| and is still, dominant in the community, but it's not alone,
| and thus a bad choice for things like libraries. Instead, those
| choose the shims like Commons Logging that developers can use
| to configure to route through Log4J or JDK logging or any of
| the others. That said, even programs that use Log4J directly
| can be routed to other loggers.
|
| This is all entrenched. It's part of the flavor of server side
| Java, configuring the different logging shims to write to your
| logger of choice on your system. Just the way it is. But,
| that's what happens when you don't live in a mono-culture.
| Feature, not a bug, and it helps empower the vast array of
| software that millions of developers and applications rely on
| everyday. Nobody designed it this way. It didn't start this
| way, it just evolved this way. It's a very "Java" thing.
|
| I watch Stack Overflow questions and 90+% of the time when
| someone asks "How do I" what they mean is "What library do I
| need", not "How can I write this". This is the sign of the
| times, and Java is not alone. The beauty of Java is that it
| made this kind of sharing REALLY easy. REALLY REALLY easy.
|
| Apache has a solid reputation for good projects and good code
| and good stewardship. It's not a back alley transaction to grab
| an Apache Java jar file and shove it in your project. Is it all
| perfect? No, but what is?
| jmfldn wrote:
| You of course have a point here, the massive dependency trees
| of modern apps carries a certain risk.
|
| The problem with rolling your own is that, for a non trivial
| technology like a producionised logging framework or some
| encryption library, you're highly likely to write a massive
| security hole yourself. The great thing about battle-tested
| open source software, used by 1000s, is that years of real
| world use, development, bug fixes and general scrutiny from a
| large number of people makes for a high level of reliability
| normally.
|
| Obviously here we have a problem with this approach but
| avoiding third party dependencies carries its own risks and
| obvious costs too. Let's focus on how reliable so many of these
| libraries seem to be and how rare problems of this scale are.
|
| I think a bigger problem here is the fact that so many people
| are using so few libraries. Maybe we need more open source
| libraries to counter single points of failure. As with how we
| deploy our code on the same clouds and so on, single points of
| failure abound these days. Things fail however well they're
| engineered and therefore we need diversification of everything.
| BrandoElFollito wrote:
| I see plenty of comments about security teams that should be
| doing this or that.
|
| I do not know about your security teams, but my security teams
| are now sitting 9 days straight trying to stop that shitshow.
| With teams complaining that they have to patch instead of doing
| Great Things.
|
| After that these security teams will be back to invisible work
| and forgotten again. Until the next issue.
|
| If your company has a truly different approach to security, tell
| it here - you may get really talented people knocking in because
| they are fed up with the security theater at their current job.
| efitz wrote:
| On behalf of all information security people everywhere, I want
| to thank Apache for all the wonderful Christmas presents this
| year.
|
| I hope you step on a Lego.
| klyrs wrote:
| Having worked with security-minded institutions in the past,
| I've endured and participated in year(s)-long audits of three
| different open source software projects. The thing that amazes
| me is that _nobody_ using this software ever did their due
| diligence. You literally get what you pay for. Kindly step on
| your own legos and be happy this didn 't happen in early April.
| bob1029 wrote:
| > The thing that amazes me is that nobody using this software
| ever did their due diligence.
|
| How could they be expected to? Reviewing all of the source
| code for log4j is not a walk in the park. Even if you
| thoroughly reviewed all the source code, would this specific
| exploit have crossed your mind?
|
| The game theory and assumptions around a "logging" library
| probably led a lot of organizations to not even consider it a
| potential threat and focus their efforts elsewhere.
|
| There is also the transitive dependency angle which turns
| this into more of a plague than a blame assignment session.
| jjav wrote:
| > How could they be expected to? Reviewing all of the
| source code for log4j is not a walk in the park. Even if
| you thoroughly reviewed all the source code, would this
| specific exploit have crossed your mind?
|
| Back a long time ago, companies were extremely wary of
| including any open source library (let alone thousands!)
| into their product. It took a very long time with audits,
| lawyers, etc to get a single library approved.
|
| Yes, it was kind of a pain.
|
| So the world has swung into the other extreme these days.
| Just pull in a couple thousand libraries from the internet,
| nobody knows what they even are or what's in them. All in
| the name of speed. So it has its benefits.
|
| But the fundamental concerns of long ago are still there.
| Bringing in unreviewed third party code is a vector for
| vulnerabilities, supply chain attacks and all that joy.
| klyrs wrote:
| I don't expect _every_ user to audit code. But one of the
| audits I was privvy to was a part of the federal government
| and had really stringent requirements for the code that
| they run -- which, of course, includes tracking down all
| the dependencies, and, yes, is expensive. The fact that
| this extremely widely-used project never hit such an audit
| is a curiosity. And a bit of a tragedy of the commons --
| many millions of dollars of commerce has depended on this
| project, and now folks are up in arms because every
| individual decided that it wasn 't their problem to
| investigate.
| jacquesm wrote:
| Indeed. Every Node.js/npm installation, ever.
| TedDoesntTalk wrote:
| Even with an audit and 100% unit test coverage, and
| static code analysis, vulnerabilities slip through.
| Software is not perfect and you do not have a silver
| bullet.
| klyrs wrote:
| I'd agree if this was the first CVE. But that got some
| attention to log4j, which _quickly_ exposed two more
| CVEs. There 's no such thing as perfect, but these
| problems almost certainly would have been caught by a
| competent security audit.
| TedDoesntTalk wrote:
| Has any of your software been through a security audit?
| Mine has. It's 99.9% OWASP coverage with some automated
| fuzzing with almost no imagination applied by the
| auditors. You're assigning far too much ability to
| auditors.
| klyrs wrote:
| Upthread, I mentioned that I've been party to it three
| times. And yes, auditor skill and methods vary.
| tekknik wrote:
| > Reviewing all of the source code for log4j is not a walk
| in the park.
|
| Traditionally if something is too large to audit then you
| shouldn't use it, which few exceptions (like Linux). Maybe
| this should be a red flag in the future, especially for a
| logging library that can almost be replaced with a call to
| stdout and a log collector.
| humanistbot wrote:
| On behalf of open source software developers, go pay for
| commercial software yourself.
| renewiltord wrote:
| Surely it has to be a joke. An infosec guy who only tells you
| something is insecure after you know from the news is kind of
| useless.
|
| The point is to keep you secure, not be a slow news relay. In
| that regard, GP is just really bad at his job.
| alanfranz wrote:
| This attitude should stop. They're volunteers. It's open
| source. Users are required to use such piece of software at
| their own will and risk. Log4j didn't ever get a single penny
| from most (all?) of their users, and now they should take all
| the blame. Why?
|
| The disclosure process was terrible, they clearly had no time
| to perform a thorough validation of the hotfixes, and this is
| the result. Why didn't anybody else step up to help them? Did
| any of the large companies using log4j pour some thousand
| dollars into the Apache foundation to let a couple of security-
| oriented engineers take a complete review of the patches?
|
| But then, it's log4j developers and Apache that should be
| blamed. Bah. That's SO pretentious.
| dylan604 wrote:
| >But then, it's log4j developers and Apache that should be
| blamed. Bah. That's SO pretentious.
|
| Um, yeah. They wrote the idiotic code that allowed this to
| happen. Who should get the blame then, the users? Duh, no.
| Should the hackers taking advantage of the exploit? Maybe,
| some, but they wouldn't be able to do it if the code wasn't
| so badly wrritten.
|
| >Why didn't anybody else step up to help them?
|
| To me, this is the pretentious part. I didn't write the code,
| I don't use the code, so why in the world should I be
| expected to fix the code? Do you use the code? Why didn't
| _YOU_ fix it?
| alanfranz wrote:
| If you don't use the code, you're not vulnerable
|
| > who should get the blame? The users?
|
| If you leverage a ton of open source deps, shit is going to
| happen. You should not blame anyone.
|
| But if you must, yes: blame the users that choose such
| piss-poor software.
| dylan604 wrote:
| Just because someone releases code with AS IS/NO
| WARRANTY/BLAH BLAH BLAH doesn't mean they should not get
| called out for introducing such a vuln. Yes, you chose
| the code, failed to evaluate it's security, etc, but at
| the end of they day, the Log4* devs made a very very bad
| decision. Sadly, because of it's deep use in so many
| projects, it has kind of become too big to fail. I doubt
| %1 of users will swap out the logging package for
| something else. They will all wait for the Log4 devs to
| keep releasing updates.
|
| People keep singing the praises of FOSS, but yet whenever
| this kind of shit happens, it's always the user's fault
| for choosing poorly which packages are being used. I'm
| saying that the original devs are criminals, but they did
| the original sin here. The fault ultimately lies at their
| feet. Why is this such a unrespected point of view?
| 5e92cb50239222b wrote:
| Would it be any better if this library was a proprietary
| product, the vulnerability would have been discovered by
| NSA and exploited until the end of time?
|
| Come to think of it, they wouldn't even have to search
| for any exploits. Just ask devs nicely to put the
| backdoor in and be done with it.
| kafkaIncarnate wrote:
| Who said anything about 'nicely'? Or 'ask'?
| TedDoesntTalk wrote:
| > I'm saying that the original devs are criminals
|
| That's an unbelievable stretch. I wonder how your mind
| works.
| dylan604 wrote:
| Sorry, I absolutely meant to include NOT in that
| sentence. No, I do not believe they are criminals. I am
| guilty of not double checking my post to make sure it
| says what I meant to say.
| alanfranz wrote:
| Discussion is ok. Disrespect is not.
|
| What original sin? What fault? You're basically saying
| that log4j developers lured you into temptation,
| otherwise you would have coded your own logging library,
| which, for sure, would have been better.
|
| I haven't seen 1/100 of this hatred for Apple iMessage
| vuln that led to NSO zero-click exploits, and I don't get
| why.
| filmgirlcw wrote:
| > I haven't seen 1/100 of this hatred for Apple iMessage
| vuln that led to NSO zero-click exploits, and I don't get
| why.
|
| I think there are a few reasons for this, and most of
| them aren't based on logic, but emotion.
|
| First, as bad as the NSO zero-click exploits are, they
| aren't things that developers were potentially injecting
| into their own projects. So there is an easier force to
| "blame" -- Apple (for making the mistakes in the first
| place -- and to be clear, all software can have bugs) and
| NSO Group (for being pieces of shit) -- whereas if you
| either chose Log4j for your own project, or more likely,
| chose a project/product that chose to use Log4j, the "who
| to blame" question becomes a lot less comfortable to
| answer.
|
| At least with the NSO thing, there is a face of who the
| bad guy is. So it's easy to sort of not hate Apple,
| provided Apple patched exploits as soon as they found
| them. That doesn't mean that Apple, a company that has
| used its "better" security relative to its competitors
| one of its marketing messages, gets off easy. It's a
| black mark for them for sure and will make it harder for
| the company to argue that it is so much more secure than
| x, y, z service or platform.
|
| With Log4j, similar to OpenSSL, people have to grapple
| with the reality that they don't really know a lot about
| a lot of the code they run -- and even worse, acknowledge
| that they aren't capable of understanding or auditing
| that code themselves. I certainly don't understand a lot
| of the code and libraries I rely on for things I build.
| And if I really stop to think about that, that's scary.
| Fortunately, my own projects are personal and aren't
| taking data from others. But facing that reality can be a
| difficult pill to swallow. And Log4j is even worse than
| OpenSSL in a way, because one could be forgiven for
| having trust in a well-respected cryptographic library,
| and for not having a deep understanding of how that
| library works. But having an insecure logging library?
| That almost feels avoidable -- even if it isn't.
|
| So people want to blame someone. And when it looks like
| the code -- made by volunteers or not -- wasn't the best
| written, that's an easy thing to go to. Rather than self-
| reflecting about all the libraries we use every day that
| we don't think about the security or code quality of.
|
| It doesn't help that the disclosure, for a host of
| reasons, wasn't good. That isn't me blaming the Log4j
| team, it just is what it is. Disclosure wasn't great and
| the fury to patch made some mistakes and now that there
| is outsized attention on the library, even more
| vulnerabilities have been found. Which is all
| understandable and indicative of what often happens in
| high stress situations.
|
| Ultimately, we should all accept that mistakes happen and
| that this wasn't done out of malice. We all have some
| level of culpability for what we choose to use in our
| projects. But mistakes happen.
|
| And as you say, discussion is OK but disrespect isn't.
| But I think we see lashing out because people want to
| find someone to blame.
| jjav wrote:
| It's actually fairly similar to the Apple bug, in that it
| was also a library with unnecessarily complex
| functionality that wasn't needed for the use case but
| just came along for the ride and ended up exploitable.
| dylan604 wrote:
| Disrespect for calling a spade a spade? Where have we
| found ourselves?
|
| No, I would never use a logging library, so there was no
| luring. If you want to call those that did use it lured,
| then you're already coming at it that the original
| software devs were coniving.
| grafs50 wrote:
| I don't understand what the problem is if you don't use the
| code. Why does it matter to you if it gets fixed?
| thowaway202107 wrote:
| Not approving of what OP said, I'd point out that
| everyone has an interest in it being fixed, because it's
| a wide spread and severe CVE that impacts lots of
| software that we all probably use daily without knowing
| it.
| dylan604 wrote:
| I use services that do use that code though, so I am
| affected when it goes down
| yawaramin wrote:
| Then you should blame the people who provide those
| services for going down. Not the open source libraries
| that they _chose_ to use.
| thowaway202107 wrote:
| > Um, yeah. They wrote the idiotic code that allowed this
| to happen.
|
| And released it with NO WARRANTY. And no mega corp thought
| it'd be a good idea to try and buy one.
|
| > To me, this is the pretentious part. I didn't write the
| code, I don't use the code, so why in the world should I be
| expected to fix the code? Do you use the code? Why didn't
| YOU fix it?
|
| If you don't use it, then noone's expecting _you_ to fix
| anything. But those out there using it for free,
| complaining that it's broken, and not doing anything to
| help? They should hush their hush holes. Or open pull
| requests. But mostly hush their hush holes.
| samhw wrote:
| This comment isn't even consistent with itself. First you
| say:
|
| > Who should get the blame then, the users? Duh, no.
|
| And then you say:
|
| > I don't use the code, so why in the world should I be
| expected to fix the code? Do you use the code? Why didn't
| YOU fix it?
|
| Just take a step back, stop trying to throw shade at anyone
| and everyone, and look at the situation analytically.
|
| I don't really see the point of these comments. The people
| who make a contribution to the world are the people who see
| a problem and take practical steps to fix it and improve
| the situation, rather than the people who see a problem and
| immediately descend into finger-pointing and shouting.
| wbl wrote:
| Log4J is bigger than all of V7 Unix. What do you need all
| that for?
| kabdib wrote:
| Because the only unsolved problem in logging libraries is:
| "When do I stop adding features?"
|
| Decent logging is surprisingly complicated, but it's not
| "We need an _arrrchitecturreee!_ for infinitely
| configurable logging or we are utterly doomed "
| complicated.
| laumars wrote:
| > The disclosure process was terrible, they clearly had no
| time to perform a thorough validation of the hotfixes, and
| this is the result.
|
| Even with proper disclosure, patching can be harder than it
| seems at face value. Take shellshock (the bash RCE from 2014)
| for example, the first patch was released a week after
| discovery and publication happened after the patch was
| released. Yet there were still a flurry of further CVEs found
| and further emergency patches required as a result.
|
| The problem is once a major vulnerability is discovered, you
| then have a the worldwide attention focused on that specific
| vulnerability looking for other ways to break it. So it's not
| really a surprise that you find multiple CVEs for most major
| vulnerabilities.
| raverbashing wrote:
| So, again, why is log4j "evaluating" stuff in log strings? This
| is the root cause that people should be addressing
|
| Why are people still using JDNI? It is frail enough to have logs
| depend on a dns lookup, I can't imagine depending on a LDAP
| lookup. This is not such a fundamental issue as the above but it
| is pretty dangerous.
|
| Oh yes and of course one of the worse and least pythonic package
| in the Python standard library was "inspired" by the log4j
| monstrosity
| wiether wrote:
| Because a few years back it was all the hype and the Java
| community doesn't like deprecating stuff.
| cesarb wrote:
| > Why are people still using JDNI?
|
| Because "java:comp/env" is the way for a J2EE web application
| to get values or objects from the configuration outside its
| container. Yes, nobody actually uses the remote parts of JNDI
| anymore, but the _local_ variant (within the same process) is
| still in use.
|
| And that's what led to this vulnerability: someone wanted to be
| able to get these local values within the logging
| configuration, which is a valid use case. Not realizing that
| this opened the door to _remote_ values and objects.
___________________________________________________________________
(page generated 2021-12-19 23:02 UTC)