[HN Gopher] Log4j 2.15.0 - Previously suggested mitigations may ...
       ___________________________________________________________________
        
       Log4j 2.15.0 - Previously suggested mitigations may not be enough
        
       Author : News-Dog
       Score  : 252 points
       Date   : 2021-12-16 07:26 UTC (15 hours ago)
        
 (HTM) web link (isc.sans.edu)
 (TXT) w3m dump (isc.sans.edu)
        
       | charcircuit wrote:
       | We already had a submission about this
        
       | usrbinbash wrote:
       | So code written in a language that encourages building a
       | gazillion indirections instead of just doing things procedurally
       | is hard to fix?
       | 
       | I'm shocked ;-)
        
         | _old_dude_ wrote:
         | More realistically, this code is 20 years old, several
         | generations of maintainers worked on it, accumulating patches
         | over patches over patches and nobody wanted to be the guy that
         | say No / we should remove that feature.
         | 
         | You can see the discussions on HN about Gnome removing features
         | as a good example about how hard it is.
        
           | jsjohnst wrote:
           | > More realistically, this code is 20 years old
           | 
           | Actually, it's not. Log4j 2 is a ground up rewrite of Log4j
           | and is less than 9 years old from when the rewrite started, 7
           | years for the GA release. Log4j 1.x isn't vulnerable as it
           | doesn't have this "feature".
           | 
           | Imho, this isn't a "patches over patches" situation, it's a
           | very poorly thought out "feature" that almost nobody needed
           | and shouldn't have been added in the rewrite. The concept of
           | "separation of concerns" exists for a reason. Your logging
           | library should _just_ be concerned with logging.
        
           | simion314 wrote:
           | >You can see the discussions on HN about Gnome removing
           | features as a good example about how hard it is.
           | 
           | The issue with GNOME is the insincerity of feature removal,
           | things like "nobody uses that", "nobody needs that", "you are
           | doing it wrong" instead of just be sincere , "file picker is
           | a usefull feature but we can't or don't care to implement it"
           | , "optional server side decoration is a good feature but our
           | devs want to work on cool shit not backwards compatibility"
           | ... if you are an open source dev, be sincere and people will
           | not be disgusted by your community and you don't get the go
           | to example for bad developers/community.
        
             | wcoenen wrote:
             | I'm not familiar with the GNOME story, but removing
             | features takes courage and is actually a form of sincerity.
             | 
             | A developer team can be insincere and avoid outrage by just
             | leaving half-broken features in the software, stop testing
             | them, and ignore all complaints about it. Low quality and
             | inactivity doesn't make for as good a story on social
             | media.
        
               | simion314 wrote:
               | >but removing features takes courage and is actually a
               | form of sincerity
               | 
               | That is false, say I remove the feature X because reason
               | A but I tell everyone that is for reason B. So I had the
               | courage to remove X but I did not the courage to be
               | sincere, maybe I am not sincere with myself and the real
               | reason A is that the code was written by somebody else in
               | the past and I want to implement new cool shit and not
               | read somebody else code, but I can't admit that I don't
               | care about feedback because I might be replaced as a
               | project leader so I invent bullshit excuses like blame
               | the previous developers that wrote bad code, blame the
               | user that are using it wrong or invent some bullshit UX
               | story with no real user tests to push my egotistic vision
               | on everyone else because I am really an Apple fan but I
               | could only get a job to work for RedHat
        
               | BrazzVuvuzela wrote:
               | > _I 'm not familiar with the GNOME story, but removing
               | features takes courage_
               | 
               | Courage is not intrinsically admirable. Whether it's
               | admirable or abhorrent depends on the context of what
               | you're doing. Ditto for sincerity.
        
               | silon42 wrote:
               | I wish they didn't add SELECTION back... https://wiki.gno
               | me.org/Initiatives/Wayland/PrimarySelection
               | 
               | But I'll never accept client side decorations.
        
           | unbanned wrote:
           | No it isn't 20yo. This feature is about 5
        
             | mol4711 wrote:
             | This JNDI "feature" is 8 years old. It was merged in 2013
             | from what I have seen, and has been defended by some as an
             | "useful" feature. It has been known since at least BlackHat
             | 2016 presentation that this is an incredibly dangerous and
             | stupid thing to keep.
             | 
             |  _cricket noises_ nobody reacted until now when minecraft
             | servers got hacked.
        
               | selimnairb wrote:
               | This whole fiasco is an argument to keep things simpler
               | whenever possible.
        
               | josefx wrote:
               | > and has been defended by some as an "useful" feature.
               | 
               | I would say the feature is useful but the implementation
               | is insane. I would expect a SQL prepared statement
               | approach, only contents of the statement / format string
               | are resolved. Interpreting variables in user provided
               | input just asks for SQL injection attacks and needs to be
               | either prohibited or extremely restricted.
        
               | abhishekjha wrote:
               | I guess you are referring to this[0]. I am surprised if
               | it was known that early then why did it take so long to
               | find the issue? Or were exploits already run earlier and
               | nobody reported?
               | 
               | [0]https://www.blackhat.com/docs/us-16/materials/us-16-Mu
               | noz-A-...
        
               | js8 wrote:
               | > nobody reacted until now when minecraft servers got
               | hacked
               | 
               | That shows which part of the IT industry is really
               | needed.. :-)
        
               | bopbeepboop wrote:
               | On average, humans spend five days of their life watching
               | Minecraft related content.
               | 
               | Google recently had a banner announcing a trillion hours
               | of Minecraft content watched; there are roughly 8 billion
               | humans. So on average, a human has watched ~125 hours of
               | Minecraft content.
               | 
               | I think it's just InfoSec has been too busy playing
               | secret squirrel to secure our infrastructure:
               | 
               | - SolarWinds
               | 
               | - MS Exchange
               | 
               | - log4j
               | 
               | - Minecraft
        
               | bombcar wrote:
               | More importantly Minecraft servers are a target of
               | convenience that both could and would be attacked by
               | script kiddies.
               | 
               | The open question is did anyone exploit this quietly and
               | targeted before?
        
               | jethro_tell wrote:
               | Of coarse. Especially if it was known since black hat
               | 2016.
        
             | wiseowise wrote:
             | They didn't say anything about the feature?
        
       | archi42 wrote:
       | In other news: If I can intercept the log messages (unencrypted
       | remote syslog, or log file public accessible, log messages also
       | shown to the user) I can still just inject stuff like
       | "${env:DATABASE_PASSWORD:-xxx}". Lots of people are only focusing
       | on the aspect how this can be abused to inject arbitrary code,
       | while it is actually a similar class like SQL injections: Blindly
       | trusting user input instead of doing proper sanitization.
       | 
       | (The whole discussion _why_ a logging mechanism should support
       | the whole JDNI stuff is still perfectly valid).
        
         | tie_ wrote:
         | Please stop suggesting that SQL Injections are a sanitization
         | problem. They are a problem of escaping in the application
         | constructing the query, not a problem with whatever the user
         | has typed in.
         | 
         | The log4j vulnerability shares some common traits, but it's
         | completely different at the core. It's not a problem of
         | sanitization, and I'd argue it's not a problem of escaping
         | either (though escaping _could_ fix it). It 's a problem with
         | an obscure feature that is left with insecure defaults, and is
         | (was?) unknown to the vast majority of developers who integrate
         | the library in their apps.
        
           | blueplanet200 wrote:
           | > They are a problem of escaping in the application
           | constructing the query
           | 
           | This is a form of sanitization.
        
           | jeltz wrote:
           | Yeah, people should stop talking about sanitation in relation
           | to SQL, because that is how we end up with data like
           | "OConnor". The two correct solutions are escaping and
           | parametrization/prepared statements (i.e. sending the
           | parameters out of band).
        
             | archi42 wrote:
             | Huh? I understand "sanitation" to mean "a transformation of
             | data that makes the data safe for use in the subsequent
             | program". Escaping is one way of doing that transformation,
             | and it's good because it's not lossy.
             | 
             | Another example: A "transformation" that's a "sanitation"
             | but not "escaping" would be replacing all occurrences of
             | "<" with "&lt;" (among others!). It surely doesn't add
             | escape characters (e.g. \\), but instead replaces the
             | problematic substring with a replacement string that makes
             | the string safe to display on a website. Of course you'll
             | want to replace user-supplied "&lt;" with "&amp;lt;".
             | 
             | (btw, thanks for that it's "sanitation" and not
             | "sanitization" ^^).
        
               | jeltz wrote:
               | Not sure I agree, for me sanitation has a strong
               | connotation with removing things. And sending data out of
               | band from code on the other hand cannot be seen as a form
               | of sanitation. And is my preferred method of solving this
               | issue.
        
               | archi42 wrote:
               | You're essentially saying "sanitation" equals
               | "filtering". Looking at CWE-707
               | (https://cwe.mitre.org/data/definitions/707.html) I'd
               | rather say that "sanitation" is what MITRE calls [begin
               | quote]transformation of the input/output to be "safe"
               | using techniques such as filtering, encoding/decoding,
               | escaping/unescaping, quoting/unquoting, or
               | canonicalization[end quote] (well, I'm repeating myself
               | here).
               | 
               | Now a Google search finds instances where
               | sanitization/sanitation also includes techniques beyond
               | filtering: https://www.webopedia.com/definitions/input-
               | sanitization/ https://hack.technoherder.com/input-
               | sanitization/ https://developer.wordpress.org/plugins/sec
               | urity/securing-in...
               | https://stackoverflow.com/questions/129677/how-can-i-
               | sanitiz...
               | 
               | But there are also results where it isn't really clear,
               | or where the only sanitation technique considered is
               | filtering. So I'd say "yeah, it's unclear and poorly
               | defined".
               | 
               | Buuuuut: I still like my definition more, as I have a
               | word for "all techniques that aim to make an input safe
               | for processing" (sanitation/sanitization) while I can
               | still refer to "destructive elimination of substrings" as
               | just "filtering", which is a again different from
               | outright "rejection of input" by using an "allow list" or
               | "deny list". :P
               | 
               | I agree that splitting data and code is the way to go, if
               | that's an option. But I didn't talk about that in the
               | post you're answering to, so I'll ignore that ;-)
        
               | hermitdev wrote:
               | I think, generally, sanitization means protecting against
               | potentially malicious input. Whether it takes form via
               | escaping or removal or some other remediation is beside
               | the point.
        
               | Dylan16807 wrote:
               | I'd say sanitization implies that there's something wrong
               | with the data and you need to clean/fix it. But the
               | former isn't true and the latter sounds like removing or
               | irreversibly remapping characters.
        
               | chiyc wrote:
               | I've seen "sanitization" used to often mean filtering or
               | removal of some of the input
        
           | Sniffnoy wrote:
           | I would normally assume "sanitization" means "escaping"... as
           | far as I'm concerned, anything other than escaping is
           | incorrect sanitization!
        
           | [deleted]
        
           | twic wrote:
           | Part of the problem is (AIUI) that expansion of ${} type
           | constructs is carried out _after_ parameters are inserted
           | into the log message 's format string. I don't believe the
           | use case requires that, i think it was just easier to
           | implement. If that expansion was done on the format string,
           | before parameters were inserted, this attack would not be
           | possible. Injecting dodgy strings into log messages as
           | archi42 suggests would also not be a problem if the insertion
           | was done safely.
           | 
           | Agreed that this is not a sanitisation attack, but i think
           | that means it is a format string attack:
           | 
           | https://owasp.org/www-community/attacks/Format_string_attack
           | 
           | It requires a couple of other loopholes, related to JNDI, to
           | work, but that is the first step which goes wrong.
        
             | archi42 wrote:
             | Hah, linking to OWASP is a good point. I have now put some
             | more thought into "how would I EXACTLY classify this, if I
             | had to list it as a finding for a customer".
             | 
             | Now, I think format string is not entirely right, either.
             | That's more related to `printf`, which splits code (the
             | format string) and data (the varargs). But log4j actually
             | mixes code and data into one string. In the OWASP frame
             | work, I think the more generic variant of format string
             | attacks would be https://owasp.org/www-
             | community/attacks/Code_Injection (that's actually linked
             | under "related" for the format string attacks).
             | 
             | OWASP lives mostly in the web world, but links to CWE-77
             | (Command Injection,
             | https://cwe.mitre.org/data/definitions/77.html), which is
             | pretty generic. And log4shell matches the description just
             | nice: 1. Data from untrusted source: yes; 2. data is part
             | of a string that's executed: interpreted, which I think
             | qualifies as a yes; 3. the execution gives capabilities the
             | attacker would not have other wise: oh yes!
             | 
             | So it's probably safe to claim that this could fall under
             | CWE-77 (too bad CVEs rarely use CWEs).
             | 
             | Now, CWE-77 (Command Injection) is a child of "CWE-74:
             | Improper Neutralization of Special Elements in Output Used
             | by a Downstream Component ('Injection')"
             | https://cwe.mitre.org/data/definitions/74.html. Which
             | states "The most classic instantiations of this category of
             | weakness are SQL injection and format string
             | vulnerabilities." - that's probably why the both of us
             | thought of the the two of these! :) And even if CWE-77 is
             | to specialized (this could be argued) CWE-74 should be a
             | good match, quoting CWE-74: "The software constructs all or
             | part of a command, data structure, or record using
             | externally-influenced input from an upstream component, but
             | it does not neutralize or incorrectly neutralizes special
             | elements that could modify how it is parsed or interpreted
             | when it is sent to a downstream component" [end quote].
             | 
             | Funnily, CWE-74 is a child of
             | https://cwe.mitre.org/data/definitions/707.html - "Improper
             | Neutralization". Which says neutralization can be done by
             | (among others): "[...] transformation of the input/output
             | to be "safe" using techniques such as filtering,
             | encoding/decoding, escaping/unescaping, quoting/unquoting,
             | or canonicalization [...]"
             | 
             | :) Thanks for triggering me on this.
        
           | omegalulw wrote:
           | > Please stop suggesting that SQL Injections are a
           | sanitization problem. They are a problem of escaping in the
           | application constructing the query, not a problem with
           | whatever the user has typed in.
           | 
           | How are they different?
        
           | archi42 wrote:
           | I disagree with you: IMHO "escaping user input in the
           | application constructing the query" is one possible form of
           | sanitization. Why do you think it is not?
           | 
           | Also I said "similar class", not "strictly equivalent". Both
           | instances are a case of "user input handled as code, not as
           | data". Now with SQLi the problem is widely known (but sadly
           | still happens) and we have techniques like parameter binding
           | to outright avoid the problem. The same techniques are among
           | the <edit>theoretical</edit> possibilities to avoid the
           | problem with log4j.
           | 
           | Though I agree, the problem are the insecure defaults (JNDI
           | enabled) and that seemingly every java dev used log4j but
           | wasn't aware of the "${}"-expressions. I just pointed out
           | that this "trend of neglect" [sry, non-native speaker
           | here<edit>, but I hope you get what I mean?</edit>] seems to
           | continue: Everyone and their CISO is now aware that attackers
           | can abuse JNDI and the whole mitigation recommendations
           | (<edit>e.g.</edit> remove JDNI classes) so far was primarily
           | focused on that. Still, even if JNDI wasn't a thing, the
           | "${}"-expressions can still be abused (it's just more
           | specialized now and not a huge pwn fest), requiring
           | mitigations beyond disabling JNDI. Just look at the reporting
           | for more examples of ignorance: When this started to explode,
           | there were some who thought blocking the string "${jdni" in
           | their WAF would be enough (and maybe it helped against script
           | kiddies), but a smart attacker just used/uses a string like
           | "${${env:randomstring:-j}dni[...]" defeating the simple
           | filters...
        
         | avgcorrection wrote:
         | > Blindly trusting user input instead of doing proper
         | sanitization.
         | 
         | More like blindly trusting that a logging library will just
         | _log_ the strings that you pass to it.
        
         | NtGuy25 wrote:
         | This is actually literally a problem call SSTI, which is REALLY
         | common. It's better to use the correct term instead of SQL
         | injection.
        
           | michaelt wrote:
           | Maybe - but if you don't know what QWER is and I explain that
           | "QWER is just like ASDF" but you don't know what ASDF is
           | either, I haven't really clarified anything for you.
        
         | tgflynn wrote:
         | Are you saying that log4j is also used for analyzing logs that
         | have already been generated ? My understanding was that it was
         | intended for use inside an application for generating log
         | messages and processing them before they are sent out to a file
         | or syslog.
        
           | regularfry wrote:
           | In addition to sibling comments, there's also a SocketServer
           | component in the library you can instantiate and fire log
           | messages at if you want to.
           | 
           | Which, if you're still on 1.2.x, also has an RCE against it.
        
           | oasisbob wrote:
           | I had the same thought. However, I think it's a more generic
           | attack. If you can persist or inject the jdni strings,
           | there's a chance they will be emitted by another component
           | which analyzes the logs and has a log4j dependency elsewhere
           | in the stack.
           | 
           | I've heard people talk about this problem with analyzing
           | windows event logs using Java-based software.
        
             | tgflynn wrote:
             | That's interesting. It seems a bit weird that a log
             | analyzer would need to do additional logging, but maybe
             | that's the case in larger systems. I wonder if using log4j
             | as a tool for log analysis as opposed to generation might
             | be a shadow use case of the library. You could have an app
             | that parses log files, converts them to Event objects and
             | then processes that event stream through log4j. I have no
             | idea whether that would be terribly useful and is actually
             | being done by anyone but I guess that's one scenario where
             | these CVE's could cause delayed effects.
        
           | retzkek wrote:
           | Logstash is a hugely popular application for
           | collecting/processing/shipping logs, mainly associated with
           | elasticsearch ("ELK") but has lots of output plugins. It uses
           | log4j, so if one of the log events it was processing
           | contained an exploit string, and also caused logstash to
           | encounter an error processing or shipping the message*, that
           | message would get logged to logstash's own log, thus
           | triggering the exploit deep in someone's infrastructure.
           | 
           | * probably not too hard to come up with a way to break one of
           | the many "grok" pattern regexes https://github.com/logstash-
           | plugins/logstash-patterns-core/b...
        
         | jabiko wrote:
         | > I can still just inject stuff like
         | "${env:DATABASE_PASSWORD:-xxx}
         | 
         | In 2.16.0 message lookups have been completely removed:
         | https://github.com/apache/logging-log4j2/pull/623
         | 
         | Lookups now only work in configured patterns. Thats IMHO the
         | way it should have been in the first place.
        
           | terom wrote:
           | They still work in the CVE-2021-45046 context lookup vector.
        
             | Macha wrote:
             | My understanding is this is true if you've simply disabled
             | them with the config flag to 2.15, but is not true in 2.16?
        
               | terom wrote:
               | That applies to the <2.15 mitigation for disabling
               | message lookups. That mitigation for message lookups does
               | not affect context lookups. Nor does the 2.15 fix for
               | disabling, nor the 2.16 fix for removing message lookups.
               | 
               | But 2.16 disables JNDI lookups entirely, so that they
               | cannot be triggered via any lookups, including context
               | lookups. But context lookups can still trigger other non-
               | JNDI lookups.
        
           | dekmetzi wrote:
           | Agree. Too easy for folks to create vulnerabilities by
           | evaluating those arguments as if they were trusted.
        
         | Sebb767 wrote:
         | This is somewhat intended, though. Also, for intercepting the
         | logs you need _a lot_ more access than for an unauthenticated
         | RCE and that database might not actually be accessible from the
         | outside.
         | 
         | It's not cool, but far less severe than the original issue.
        
           | archi42 wrote:
           | Yes, absolutely. log4shell is a pwn fest that makes the
           | recent Exchange debacle look rather innocent (and that was
           | already quite bad). But I can envision scenarios like
           | unencrypted rsyslog on a LAN being relevant during pentests,
           | or misconfigured (read: attacker readable) log files.
        
         | xorcist wrote:
         | People think they are safe because they don't allow for
         | outgoing ldap, but combined with a way to do dns lookups, these
         | kind of things can get really scary.
         | 
         | The java ecosystem is full of these abstractions, that are not
         | outright security problems in themselves, but give rise to
         | complex interactions that makes it hard to judge what the
         | implications are.
         | 
         | This particular logging system has been used by millions of
         | developers for the better part of a decade before any one
         | single person realized the actual implications.
        
           | ethanbond wrote:
           | Just a minor clarification that's worth being explicit about:
           | 
           | It has been used by millions of developers for the better
           | part of a decade _before anyone disclosed_ the implications.
        
             | tfigment wrote:
             | What i discovered was all of my applications that use log4j
             | do not use version 2. They have out of date dependencies or
             | use slf4j. I'm not sure how i feel but mostly happy im not
             | scrambling. I wonder what adoption rate v2 actually has.
        
               | yawaramin wrote:
               | I think adoption has mostly gone towards slf4j+logback
               | now.
        
               | mrclark411 wrote:
               | logback is *gpl, I think that scares folks away.
        
               | gunnarmorling wrote:
               | Logback is dual-licensed under EPL v1.0 and the LGPL 2.1,
               | as per the licensee's choosing [1]. In particular, it is
               | _not_ licensed under the GPL.
               | 
               | [1] http://logback.qos.ch/license.html
        
       | stickfigure wrote:
       | I must confess to feeling that the reach of this CVE is a bit
       | oversold. Yes, it's a terrible RCE and a major catastrophe for
       | anyone using log4j. But log4j hasn't been the popular choice for
       | logging implementations in a long time. Libraries typically use
       | facades like slf4j or commons-logging, so the risk of pulling in
       | an active log4j implementation transitively is pretty small. Your
       | application/appserver either logs with log4j or not, and your
       | developers are probably aware of this when they configured
       | logging. You shouldn't need to pour over code to figure it out.
       | 
       | It would be far worse if this exploit existed in slf4j or
       | commons-logging.
        
         | raesene9 wrote:
         | The major risk here probably isn't in-house developed code but
         | versions of log4j used in a vast array of packaged software and
         | systems.
         | 
         | https://github.com/cisagov/log4j-affected-db <-- that's a
         | (probably incomplete) list of possibly affected vendors and
         | applications, and in many cases the vendor can't even say yet
         | whether they're vulnerable.
        
           | anyonecancode wrote:
           | Yup. Here's a concrete example -- my team owns a couple of
           | java services. They're spring boot with slf4j as our logging
           | wrapper, using the default logging implementation of logback
           | -- we didn't opt for log4j. So we're good, right? Well, let's
           | do a `gradle dependencies` and see if we're pulling it in
           | transitively -- still good. See some `log4j-api`, but no
           | `log4j-core`. We're not actually doing any logging with log4j
           | -- the "api" is from spring to allow you to use log4j if you
           | want. Again, we're not, so we're fine.
           | 
           | Oh, but it does turn out that we do have the elastic apm
           | agent, and it has log4j as a shaded dependency. So not as
           | free and clear as we thought, and damn hard to detect that.
        
             | wing-_-nuts wrote:
             | You're now making me feel better about our decision to
             | exclude the log4j-api from our spring build entirely.
        
               | freeqaz wrote:
               | log4j-api isn't actually a problem (directly). It's the
               | usage of log4j-core that has the JndiManager class that's
               | the injection point for RCE.
               | 
               | In the case of the transient dependency, it's further
               | complicated because we've seen that some packages
               | repackage the JARs they depend on... that means that
               | statically analyzing for log4j is very difficult because
               | you can't use hashes (even if you unzip the jar and hash
               | class files directly).
               | 
               | I've been working on a scanner for this stuff on
               | GitHub[0], and it's a real pain in the neck lol.
               | Especially for Vendor software that you don't control.
               | 
               | 0: https://github.com/lunasec-
               | io/lunasec/tree/master/tools/log4...
        
             | ishjoh wrote:
             | Shaded dependencies are a nightmare, and it's a particular
             | bone I have to pick with using Elastic products that they
             | think this practice is ok.
             | 
             | I prefer to package all my projects into a single fat jar,
             | since it makes distribution so easy and clean, but if I was
             | creating jar files with the expectation that other projects
             | would use them I would make sure to package everything
             | separately.
             | 
             | In this case using fat jars made things really easy to
             | check for the effected classes and luckily for me I also
             | didn't have any log4j-core classes in my fat jars.
        
         | zootm wrote:
         | Using logging facades means that libraries don't need to update
         | -- which is great -- but libraries were never directly
         | vulnerable anyway. log4j is, to my knowledge, still by far the
         | most common actual implementation of logging in the Java
         | ecosystem. The assertion that it's not popular for logging only
         | holds if you assume that logging facades are logging
         | implementations, which they are not.
         | 
         | My completely-unverified guess would be that there are more
         | people immune to this issue because they never migrated past
         | log4j 1.x than there are who are immune because they picked up
         | Logback or something similar.
         | 
         | > You shouldn't need to pour over code to figure it out.
         | 
         | This is true but as sibling comments have pointed out, a lot of
         | other software you might be deploying without having written or
         | configured the logging for are written in Java.
        
           | regularfry wrote:
           | That's my observation. My favourites are the projects where
           | the last commit was 5 years ago with a copy-pasted log4j
           | stanza that must have been 4 years old at that point, which
           | now can't practically be upgraded _at all_ because of the
           | bitrot and loss of organisational knowledge. I 've seen one
           | that needed somewhat special measures just to regain access
           | to the source code...
        
           | jillesvangurp wrote:
           | Most people using e.g. spring boot or quarkus would end up
           | using the defaults that come with those frameworks. For
           | spring boot, the default is actually logback. However, you
           | can switch it to log4j2.
           | https://spring.io/blog/2021/12/10/log4j2-vulnerability-
           | and-s...
           | 
           | Log4j2 never quite got the same status that v1 had. V1 should
           | be considered a bit obsolete at this point. It still works of
           | course but it has some performance issues that both log4j2
           | and logback try to address.
           | 
           | The issue with high profile vulnerabilities like this is that
           | there are a lot of projects where dependencies are rarely
           | updated.
           | 
           | I update aggressively on my own projects to stay on top of
           | changes and keep the effort related to mitigating
           | compatibility issues at a minimum. A nice side effect is that
           | you get all the latest security, performance, and other
           | fixes. In my experience, updates get harder the further you
           | fall behind. So, the longer you wait, the more likely you
           | will have a lot of fallout from updates and the more likely
           | it is that you will be exposed to pretty serious bugs that
           | have since been addressed upstream.
           | 
           | If you are like me, I can recommend the excellent
           | refreshVersions plugin for gradle. It makes staying on top of
           | dependency updates a breeze. I run it every few weeks to
           | spend a few minutes updating misc libraries, and verifying
           | everything still works. Run the command, update to the
           | suggested versions, create a pull request and merge when it
           | works.
           | 
           | Occasionally there are issues with specific libraries but 95%
           | of the updates are completely painless and the remainder are
           | usually pretty easy to deal with. And if there are show
           | stopper issues, I want to know about them and document them
           | why we can't update.
           | 
           | I would recommend doing the same for packaged software. I
           | work with a lot of customers running ancient versions of
           | whatever for no other reason than that they seem a
           | combination of fearful, ignorant, and indifferent about what
           | will break because they can't be bothered to even try. Mostly
           | updating them to more recent versions isn't that big of a
           | deal and it tends to address a multitude of performance,
           | security, and other issues.
        
             | regularfry wrote:
             | "A bit obsolete" is underselling a 9-year-old release
             | that's been EOL for 6...
        
             | zootm wrote:
             | > For spring boot, the default is actually logback.
             | 
             | I did not know this, thanks for letting me know!
             | 
             | > I update aggressively on my own projects to stay on top
             | of changes and keep the effort related to mitigating
             | compatibility issues at a minimum. A nice side effect is
             | that you get all the latest security, performance, and
             | other fixes. In my experience, updates get harder the
             | further you fall behind.
             | 
             | This is absolutely a best practice, though I think people
             | struggle with it for all sorts of reasons. In general one
             | of the downsides of maintaining a diverse codebase is that
             | this constant update cycle becomes more and more difficult,
             | and it's one of the things that I find drives towards more
             | consistent tooling within a team.
             | 
             | > I work with a lot of customers running ancient versions
             | of whatever for no other reason than that they seem a
             | combination of fearful, ignorant, and indifferent about
             | what will break because they can't be bothered to even try.
             | 
             | While I agree this is something people need to get over, we
             | have to take some blame for this as an industry. A lot of
             | people have bad experiences with upstream Shiny Object
             | Syndrome.
        
         | whattimeisnow wrote:
         | Oh sweet summer child
        
         | oblio wrote:
         | He he, an optimist.
         | 
         | log4j is in widespread use, directly or indirectly.
         | 
         | Most people don't give a crap about the exact Java logging
         | library, the only places that actually care are those that bike
         | shed, since functionally Java logging stuff is generally quite
         | robust and full featured.
         | 
         | The worst part about my previous phrase is that the places that
         | don't care about the Java logging library frequently don't do
         | security audits, so prepare for a ton of exploits and data
         | leaks in the following months and years.
        
         | theevilsharpie wrote:
         | > You shouldn't need to pour over code to figure it out.
         | 
         | It's not just your code that is vulnerable.
         | 
         | Our in-house software doesn't use log4j, but a number of third-
         | party components we use do and needed to be patched.
        
         | xorcist wrote:
         | My initial reaction was also tentative skepticism. Not that the
         | package isn't popular, but that it must be hard to find real
         | world interactions where you can successfully get a payload in,
         | and a lot fewer still where you can actually execute it.
         | 
         | Yet we have seen a large number of high profile hacks already,
         | and they keep coming. Turns out that not only are there many
         | ancient java versions in use, the aren't very compartmentalized
         | either.
        
           | twic wrote:
           | Is there a list of successful hacks anywhere? Most of what
           | i've seen on Twitter is someone posting an attack string to a
           | website, then showing a record of a DNS lookup, which is some
           | distance from actually being a successful exploit.
        
         | MarkSweep wrote:
         | The fact that putting these strings in the name of an iPhone or
         | a Tesla was vulnerable suggests to me that there are plenty of
         | vulnerable systems out there.
         | 
         | https://www.theverge.com/2021/12/13/22832552/iphone-tesla-sm...
        
         | smarx007 wrote:
         | For the record, both SLF4J and commons-logging as logging
         | frontends allow log4j to be used as a logging backend.
        
         | alenmilk wrote:
         | I have ported a Spring Boot application from log4j to logback
         | and it was pretty easy since we used slf4j. The code was the
         | same, the only thing I had to do is to convert the logging
         | configuration. The features mapped one to one and it to some
         | fidgeting to make it work, but all in all it was pretty
         | straight forward. Would recommend.
        
           | shashashasha___ wrote:
           | > The features mapped one to one
           | 
           | did they also map the jndi feature ;)
        
             | alenmilk wrote:
             | Haha, no that was missing. I mean all the features/loggers
             | that were used in the project mapped one to one.
        
         | alexbakker wrote:
         | I can't say I'm feeling the same. Still lots of people testing
         | over at https://log4shell.tools almost a week after this
         | vulnerability became widely known. Plenty of people still
         | discovering they're vulnerable as well. I think it's likely
         | that these are just the people who _know_ they 're using log4j.
         | If you're running a black box product from some vendor you'll
         | have no clue you're vulnerable until it's too late.
        
       | archo5 wrote:
       | Why is nobody still concerned with the fact that log4j-core is
       | nearly 50 KLOC big and massively overengineered? A bit much for a
       | logging library, isn't it?
       | 
       | How many users of this library even know 10% of what's going on
       | in there?
       | 
       | For a comparison, the entire original Doom was only 40 KLOC.
       | https://www.reddit.com/r/Doom/comments/g7wbxr/number_of_line...
        
         | broodbucket wrote:
         | You can apply the same logic to most of the stack of most of
         | everything.
         | 
         | Not to downplay the issue, but it's hardly an isolated case.
         | Noone cares until they have to care, and by that point it's too
         | late.
        
       | rq1 wrote:
       | Did someone think about tricking google bots into following URLs
       | with the malicious bits?
       | 
       | It can be "fun". :)
        
       | krylon wrote:
       | When I was shopping for groceries today, I saw something strange.
       | The buses have displays to show what Line they are, today all of
       | the buses I saw had blank displays. On my way home, I could see
       | the display on the bus driver's seat, which only displayed an
       | otherwise blank Windows Embedded CE 6.0 start screen.
       | 
       | Oh, and the thingy you put your ticket in get the stamp was out
       | of order on all the buses.
       | 
       | Made me wonder if that was due to some component involved was
       | using log4j? I don't think it's very likely - why would the bus
       | display or the ticket stamping machine be affected in a way that
       | requires turning them off? But who knows, maybe somebody
       | overreacted, or maybe these depend on some backoffice
       | infrastructure that is temporarily offline due to log4j.
       | 
       | It was weird, though. This is the first time I remember a
       | security vulnerability made the national news (in Germany, if it
       | matters). I don't think even Heartbleed got that much attention
       | outside IT news outlets.
        
       | guidedlight wrote:
       | This vulnerability will likely turn out to follow the trajectory
       | of the Meltdown and Spectre vulnerabilities from...(checks
       | google)... 2018.
       | 
       | I speculate will be a cluster of newly discovered vulnerabilities
       | that follow a similar pattern, possibility across a number of
       | familiar Java libraries.
        
         | chromanoid wrote:
         | Mmh, I don't think so. Beside logging most other libraries will
         | already sanitize user input since it is a more commonly known
         | attack vector for those kind of libraries. I would compare the
         | vulnerability to https://github.com/frohoff/ysoserial.
        
         | sumtechguy wrote:
         | What I have been pondering again in the past few days is
         | dependency management. For many projects this is a open pom
         | file pop in the new version, recompile, deploy and bobs your
         | uncle you are good.
         | 
         | But some projects are little more interesting in that they
         | include libs like this, or include libs that include it. It is
         | part of their foundation/framework. So you can still put in
         | override and ignore what the lib wants.
         | 
         | But now when it comes time to upgrade that top lib maybe there
         | are 6 newer versions of the one problematic lib you were
         | concerned about today, but now it is a year later and you
         | forgot about it. What is the mental context to say 'oh yeah go
         | back and check that'. You may have got used to ignoring the
         | 'hey you are overriding that jar file'. Are you going to do
         | that for all of the libs/jars this top level framework drags
         | in?
         | 
         | It is not just java that has this sort of issue. NPM (though
         | you can float the versions at least), nuget, rust, C++
         | containers, etc. A lot of projects out there have taken on this
         | maven central style building. Which is a huge productivity
         | boon. But in many ways has not helped us with dependency hell,
         | and in some cases has made it wildly worse. I see _simple_
         | projects pulling in 150+ items in some cases. What is the
         | mental context of actually managing that? And have real fun if
         | your project depends on a  'dead' project where there are
         | little to no updates.
         | 
         | This one is at least high profile enough that many people are
         | revisiting old build chains that have long ago been forgotten
         | and updating them. That will kill out many other vulins that
         | have been lurking. But many will just do that one jar set and
         | call it a day. But you are probably right we are going to get
         | see a few of these style of attacks.
        
           | thereddaikon wrote:
           | There isn't one. This method of development inherently leads
           | to these kinds of problems. Software engineering is still in
           | its infancy and has yet to truly develop a universal best
           | practice development cycle. Eventually these things will be
           | codified.
        
             | giobox wrote:
             | > Eventually these things will be codified.
             | 
             | Is there actually evidence for this? The 30 year arc of the
             | industry to date that I am familiar with so far shows
             | little sign of engineering standards in software codifying
             | around anything.
             | 
             | If we are in the infancy stage in 2020, we are one very
             | large infant. More likely to me is that in the future,
             | software will change so much again that what we do today is
             | unrecognisable; not because of codification but because of
             | the inevitable tech churn between now and then.
        
               | Buttons840 wrote:
               | The closest things we get to standards are automated
               | vulnerability checkers. I'm currently dealing with one of
               | these vulnerability scanners which claims our code base
               | has some XSS vulnerabilities, which I know is wrong
               | because the app isn't a web app. SMH
        
         | kzrdude wrote:
         | What they have in common with spectre is that: Suddenly it was
         | realized that the design was all wrong. What was thought of as
         | a feature, was a giant vulnerability.
         | 
         | However, log4j seems to be adjusting their design rather
         | quickly, so they should be out of it rather fast.
        
       | John23832 wrote:
       | This seems to be what we've always known: Interpreters with
       | public input are just vulnerabilities waiting to happen.
        
       | jet390 wrote:
       | The recent log4j vulnerability really piqued my interest, and
       | I've spent the last few evenings working on a proof of concept
       | Java agent that could mitigate similar vulnerabilities in the
       | future, for applications that are able to completely forego
       | platform features like JNDI, serialization or native process
       | execution.
       | 
       | Link to the project: https://github.com/gredler/aegis4j
       | 
       | It's not a lot of code, but it uses parts of the platform that I
       | think are a bit unusual for most devs, so it was quite
       | interesting to implement. Happy to discuss details, ideas, and
       | concerns.
       | 
       | One idea for a possible improvement is to make the feature block
       | list adaptive, i.e. watch what the application uses in the first
       | few minutes of execution, and then shut down all unused
       | "dangerous" features for the remaining lifetime of the VM. Not
       | sure how reliable this would be though, especially for services
       | which have background jobs that might only run once a day.
        
       | freeqaz wrote:
       | Thread on this from yesterday:
       | https://news.ycombinator.com/item?id=29561532
        
       | tinus_hn wrote:
       | Why is there no environment variable to turn off parsing the
       | messages completely?
        
         | Macha wrote:
         | There was. It wasn't checked in all contexts. Hence the second
         | CVE
        
       | diegocg wrote:
       | This is starting to remind me of the coronavirus
        
         | sschueller wrote:
         | https://twitter.com/rickhanlonii/status/1470074730279546895
        
         | bartread wrote:
         | If you mean it's turning into a grinding and repetitive
         | groundhog day, I concur.
         | 
         | We don't even use log4j and yet are fielding a steady stream of
         | requests to reassure clients that we've mitigated against it,
         | which hasn't been an issue because we don't use it (although we
         | do use log4net with Stackify and it's certainly a possibility
         | that may be compromised in future), and of course are having to
         | audit all our own suppliers.
         | 
         | I encourage companies to post information on their websites
         | about this so we don't all have to keep emailing you (I'm
         | trying to get some info up on ours detailing mitigations, for
         | similar reasons; once it's there I can send some all company
         | comms and hopefully never see another email in my inbox with
         | "log4j" in the subject line).
         | 
         | Of course, this whole issue is crucially important to deal
         | with, but it's also frustrating because it's an entirely non-
         | value-adding time-vampire.
        
       ___________________________________________________________________
       (page generated 2021-12-16 23:02 UTC)