[HN Gopher] Bank of England to crack down on 'secretive' cloud c...
       ___________________________________________________________________
        
       Bank of England to crack down on 'secretive' cloud computing
       services
        
       Author : adrian_mrd
       Score  : 105 points
       Date   : 2021-07-14 06:56 UTC (16 hours ago)
        
 (HTM) web link (www.itnews.com.au)
 (TXT) w3m dump (www.itnews.com.au)
        
       | MrBuddyCasino wrote:
       | > But big providers could dictate terms and conditions - as well
       | as prices - to key financial firms.
       | 
       | What exactly is the concern here? Cloud compute is becoming
       | cheaper over time due to market forces. Its not like Amazon is
       | cornering the market for CPUs.
        
         | pmlnr wrote:
         | > What exactly is the concern here?
         | 
         | Top of the article:
         | 
         | > Concentration of compute could threaten financial stability.
         | 
         | If BigCloud goes down, so does banking, and banking doesn't
         | seem to like that - and I'm with them.
        
           | cm2187 wrote:
           | Also changing IT in a bank is slow, dead slow. A bank
           | wouldn't be able to cope with being booted overnight from its
           | infrastructure Parler-style.
        
             | nindalf wrote:
             | That FUD is unwarranted. No bank is going to be kicked off
             | it's infrastructure. Bringing up Parler seems completely
             | irrelevant.
        
               | gizdan wrote:
               | While Parler is somewhat irrelevant and the likelihood of
               | any bank being taken down for similar reasons as Parler,
               | it isn't completely incomprehensible.
               | 
               | There have been many horror stories of businesses being
               | banned, blocked, or messed around by Google and Amazon.
               | Their policy does not protect anyone but themselves. It
               | is within the realms of reality for bank to be taken down
               | by them in a matter of days.
        
               | jon-wood wrote:
               | These things do happen, but I'd be willing to place money
               | on it not happening to a major financial institution. The
               | day AWS and friends summarily terminate the account of a
               | major UK bank, causing people to lose the ability to
               | access their money, would also be the day that every
               | company in the country immediately pulls out the business
               | continuity plan and digs into the section on dealing with
               | having to migrate to a new provider.
        
               | cm2187 wrote:
               | I doubt the payment systems would run on a 3rd party
               | cloud provider. But there are many other systems used by
               | a bank, to calculate their risks, run their accounting,
               | do their regulatory reporting, all the network drives
               | that are used across the bank, emails, that could be
               | killed in such a scenario. That alone is enough to
               | threaten the stability of the banking system.
        
               | andylynch wrote:
               | Any provider offering this kind of service is
               | exceptionally unlikely to offboard any financial services
               | firm without serious forethought- any outsourcing of
               | critical processes like this would need pre-approval from
               | the FCA and for a bank the PRA would take a hard look
               | too.
        
               | pmlnr wrote:
               | I have my own horror story, and that's not even about the
               | cloud, just how centralizing a service magnifies issues.
               | 
               | I work for near FAANG company who still sends important
               | email information out to people. One day, many years ago,
               | a new gmail feature landed: automatic smart tabs - and
               | ALL of our email started landing in promotions. And those
               | mails were send from IP addresses which were dedicated
               | for these and only these kind of email, so we started to
               | panic.
               | 
               | We started going through the correct channels for
               | reporting this as a problem, at which point we got a
               | "cheers, we'll get back to you in 2 weeks". At that point
               | we were certain we were loosing money in the millions
               | soon, so we walked over to marketing - Google PPC (pay
               | per click) to be specific, and asked them to get hold of
               | someone high enough at Google, we don't care how, or over
               | what channel.
               | 
               | Within an hour, the change in gmail to put everything in
               | promotions that was "noreply@" was rolled back, and our
               | arses were saved.
               | 
               | If we were still in the early 2000s with countless small
               | email servers and providers, if one of them done this,
               | their customers would be angry at them. But since people
               | believe Gmail is their saviour - and because nobody can
               | get hold of anyone at google to report a problem - now it
               | was our fault, despite the fact that we had absolutely no
               | idea what went wrong, and there were no changes on our
               | side.
               | 
               | My summary? Fuck the cloud and the centralized services;
               | I want the small, loosely connected internet services
               | back.
        
               | wdb wrote:
               | Guess, as Bank you just need to make sure you have Amazon
               | and Google as customer :)
        
               | pjc50 wrote:
               | Kick a UK bank off cloud infrastructure for no good
               | reason and you _will_ get rung up by the PM, who will
               | make it clear that you will reinstate them or be forever
               | banned from government procurement.
        
               | csomar wrote:
               | The issue is that the Cloud provider is not UK-based. If
               | they were to ban a bank, it's probably due to pressure
               | from their own government (US) who could be having some
               | problems with their client's government (UK). It's not
               | like it is not happening right now.
        
               | gizdan wrote:
               | Sure, just like all the other people who have been called
               | to parliament and suffered the consequences. Ah wait..
               | 
               | Our current leaders are useless in this regard. They'll
               | be given a stern warning letter and nothing will come of
               | it.
        
               | sokoloff wrote:
               | I have little doubt that the major cloud providers would
               | be willing to contractually bind themselves to a notice
               | and appeals period that would suffice for "because we
               | don't want your business anymore" reasons.
               | 
               | The cross-bank correlated technical outages and general
               | "fear of the new (15 years now, but continually being
               | enhanced and changed)" is harder to get beyond.
        
               | gizdan wrote:
               | Notice and appeals period isn't good enough. It takes UK
               | banks millions to migrate to a cloud provider. Being
               | given a notice period just means they have less than x
               | amount of time to potentially spend equally as much as it
               | took for them to get onto the cloud provider in the first
               | place.
        
               | h_anna_h wrote:
               | A lot of people hate banks and want them to go down, just
               | like how a lot of people hate Parler.
        
           | MrBuddyCasino wrote:
           | This is obvious, concentration of risk is a legitimate
           | concern. I was specifically wondering about the "dictate
           | terms and conditions - as well as prices" line, which sounds
           | bullshit to me.
        
           | lbriner wrote:
           | How is that different than the banks own systems? In the UK,
           | we have seen about 4 or 5 really bad system failures in banks
           | leading to unpaid salaries/bills etc. and I believe all of
           | them were on-prem.
           | 
           | Sure, hedge your bets but I would hope that most people using
           | cloud are smart enough to at least get it 95% cloud-agnostic
           | so if price gouging occurs, they can leave.
        
           | JackFr wrote:
           | If the ATM stops working because of an AWS outage, I will
           | very much be disappointed in my banking regulator.
        
       | kaydub wrote:
       | If you're _that_ size why not split the workload between a couple
       | or all 3 of the big providers. Get them to work against each
       | other.
       | 
       | All the concerns about vendor lock-in are ridiculous right now.
       | Prices have consistently lowered for all cloud providers.
        
         | [deleted]
        
       | ExposeThem wrote:
       | BoE wants all to use the big cloud providers, no on premise
       | things or boutique clouds, because they want to monitor
       | EVERYTHING.
        
       | dsign wrote:
       | The risk Bank of England doesn't like comes with the territory,
       | and everybody who uses public cloud is subject to it. The only
       | difference is that smaller companies don't have a big megaphone
       | or the clout that they have, and must either accept the terms and
       | shut up or find an alternate solution. If Bank of England wants
       | to use their power constructively, they can literally go shopping
       | for data center and IT companies to serve their needs.
        
         | PeterisP wrote:
         | What BoE can say is that certain risks are unacceptable for UK
         | banks, period. It's not BoE shopping for data centers, it would
         | be BoE setting the rules for every bank shopping for data
         | centers.
         | 
         | If BoE does not like some risk that really does come with the
         | territory and everybody who uses public cloud is subject to
         | that risk, oh well, then that would imply that UK banks have to
         | keep out of that territory and can't use public cloud for their
         | key services - it is a bit drastic, but certainly something
         | that BoE has the right to rule if it really wanted.
         | 
         | On the other hand, if there's a way to use public cloud and
         | meet the requirements, but current major vendors are simply not
         | offering it, then BoE can make up arbitrary requirements,
         | followed by the major UK banks going to the major cloud vendors
         | with a proposal "meet these requirements or we won't use your
         | services, because we'd be prohibited to".
        
         | pjc50 wrote:
         | > risk Bank of England doesn't like comes with the territory
         | 
         | The BoE is the central bank; to a great extent they get to
         | define the territory. Especially with regard to risk. Basel III
         | and that kind of thing.
         | 
         | > they can literally go shopping for data center and IT
         | companies to serve their needs.
         | 
         | No, it's not about what they themselves buy, it's about what
         | all the banks in the UK buy. A situation where all the banks
         | are using the same cloud provider(s) is unacceptable because of
         | the high risk of correlated failure. It's bad enough when one
         | bank goes down and its ATMs stop working; if EU-west-1 goes
         | down and takes out _all_ the banks, that 's a disaster, and the
         | BoE is rightly taking steps to prevent it.
        
           | dsign wrote:
           | Ahh, I stand corrected.
        
         | cameronh90 wrote:
         | The risk that they're concerned with is if every bank is using
         | AWS, then AWS goes down, the entire UK economy crashes until
         | it's back up.
         | 
         | Even if AWS itself is more reliable than every bank's on-prem
         | solution, that's no good if it goes down for everyone
         | simultaneously and I can't just use a backup credit card.
        
           | thedougd wrote:
           | Exactly, they're concerned with the systemic risk. As an
           | example, there's no visibility or processes in place to
           | ensure that all these banks aren't sharing the same
           | availability zone.
           | 
           | They're likely only looking for ways to mitigate these risks,
           | not necessarily a complete return to self-hosting.
        
       | asaikali wrote:
       | I think a big risk is a cpu level security issue similar to
       | meltdown or spectre that ends up weakening the hardware isolation
       | between tenants to the point where it can be exploited on mass on
       | the cloud providers to wreak havoc. The probability of something
       | like this happening is very low but not zero, I would say same
       | level of probability as datacenter fire or earthquake banks
       | should be planning for how to handle this type of event.
        
         | fulafel wrote:
         | We can be almost certain that there are sw and hw
         | vulnerabilities that can be so exploited, given the rate of
         | discovery and knowing what now-public hypervisor and cpu vulns
         | a time traveler from today could exploit eg 5 or 10 years in
         | the past.
        
         | lbriner wrote:
         | I don't see that this risk is any different than a similar
         | apocolyptic failure happening to your on-prem equipment.
         | There's not much you can do about it differently than the cloud
         | just add some extra controls and hope for the best.
         | 
         | I very much doubt that anyone would not use the cloud because
         | of a theoretical de-isolation bug.
         | 
         | Also, by the time you found out, it would probably already be
         | too late anyway if you were a victim. If not, you just switch
         | it off.
        
           | asaikali wrote:
           | I am not saying that they should not use cloud just that it
           | is important to have a plan in place to deal with a unlikely
           | but high impact security event affecting a cloud provider.
           | Just like companies have business continuity plans in case a
           | data center disaster they need to have plans for evacuate a
           | cloud provider should they need too.
           | 
           | I put on my seat belt when I drive on the highways even
           | though a nasty crash at 120 kph would likely kill me. Not
           | using a seat belt because you will be severely injured anyway
           | is not wise.
           | 
           | Given the amount of profit banks make what is the Downside of
           | having them be resilient against public cloud failures?
        
         | davidgerard wrote:
         | Separate hardware for your stuff is a standard AWS product, for
         | example - you can just buy this.
        
           | asaikali wrote:
           | That's one way of dealing with hardware isolation risk but
           | not every bank is doing this on public cloud.
        
       | nobodyandproud wrote:
       | This is a good conversation to have. There are no meaningful
       | guarantees that cloud providers won't scrape data or ideas.
       | 
       | Then there's the cost: Cloud is extremely expensive over the long
       | run; more-so than the equivalent on-prem.
       | 
       | The long term solution here is to add better Internet in all
       | regions, similar to our highway system today.
       | 
       | Then any company of a large enough size can build out their own
       | DCs on the cheap.
       | 
       | It would also inject much needed dollars into the non-coastal
       | areas.
        
         | nobodyandproud wrote:
         | Some ups and downs, but cloud's value isn't really for its
         | costs.
        
       | bob1029 wrote:
       | We do business in the US financial sector, and the sentiment we
       | are getting with regard to cloud vs on-prem seems to be growing
       | into a bimodal distribution.
       | 
       | I would say it's nearly a 50/50 split until we have conversations
       | about how our product actually works and the incredibly sticky
       | problem that is PII...
       | 
       | Once the risks are reviewed in open and honest ways, we find that
       | virtually all of our clients would prefer to keep our solution
       | on-prem. Only those who have already made a full step into cloud
       | compute have to continue to take exception for obvious
       | practicality reasons.
        
         | pwarner wrote:
         | How is the problem of PII better solved on premises?
        
           | oliwarner wrote:
           | What's the worst that can happen in a on-premises attack, Vs
           | the worst that could happen if AWS was hacked?
           | 
           | The amount of financial data that could be exploited _at
           | once_ is magnitudes larger in a popular cloud. I don 't think
           | it's strange that a regulator might look at that failure
           | point with some trepidation.
        
             | lbriner wrote:
             | On the other hand, at least AWS, Azure etc. all have a
             | vested interest in doing things well and securely. At a
             | bank, most employees know nothing and care nothing about
             | the HW and SW systems, they just use and abuse them.
        
               | oliwarner wrote:
               | Why do you think banks don't have vested interest?
               | 
               | And I don't expect my cleaner to service my car. Banks
               | have DBAs, network and physical security experts on tap
               | (just like cloud providers do).
        
           | xeromal wrote:
           | Smaller surface area to guard for one.
        
           | latch wrote:
           | The surface area of vulnerabilities like spectre and meltdown
           | is much lower
        
           | someguydave wrote:
           | because you can control physical access to the hardware
        
           | LinuxBender wrote:
           | Just my observations, but some business entities run into
           | legal challenges that vary greatly by industry. B2B customers
           | have a contractual relationship with you, not your hosting
           | provider. If your hosting provider has an "oops we leaked
           | your data" they only breached the contract with their vendor,
           | not themselves. The contract can specify how data is managed.
           | Adding to this, some companies/banks legal controls are
           | compatible with SOC2 controls of a 3rd party data processor
           | being audited and some companies are not. Some companies can
           | cite the 3rd parties audited certifications and some can not
           | based on preexisting legal contractual agreements with their
           | own customers. I am not a lawyer, but had to sit in many
           | meetings with lawyers and businesses and this is a real issue
           | they have to address. I have also worked for a large bank.
           | Rules, regulations and contracts around 3rd party data
           | processors and financial institutions can get very
           | complicated. There are a myriad of additional complicating
           | variables that go beyond regulations. Legal obligations also
           | vary by relationship. If a bank has preexisting relationships
           | with other banks, they may be obligated to get approval from
           | the other banks to change how their data is managed. Amending
           | contracts is non-trivial. This rabbit hole can get very deep.
        
             | elliekelly wrote:
             | I'm a banking attorney and I think you've hit the nail on
             | the head. Vendor management is popular with regulators,
             | particularly with respect to info sec and data privacy, and
             | the navigating the contractual responsibilities can quickly
             | become burdensome.
             | 
             | Getting the lawyers on both sides to agree to language for
             | the ongoing certification(s) that will also satisfy the
             | internal auditors, the outside auditors, the regulators...
             | suddenly something that sounded simple when the contract
             | was signed takes several meetings and a lot of back and
             | forth.
        
           | bob1029 wrote:
           | Think about it more abstractly from the perspective of trust
           | and # of actors involved.
           | 
           | If you run 100% of your IT workload on-prem, the ability to
           | control the flow of data can be boiled down into a physical
           | exercise of following fiber channel cables in your own
           | datacenter. Having a unified set of firewall rules that
           | define your entire public interface also helps a lot.
           | 
           | You can actually make deterministic guarantees to your
           | customers that not only your own systems are secure, but also
           | that the systems of your vendors and other 3rd parties are as
           | well. The moment you start configuring site-to-site VPNs with
           | 3rd parties across which you intend to transact sensitive
           | business knowledge, you are surrendering an entire mountain
           | of security constraints.
           | 
           | If we are being honest with ourselves, a lot of shops that
           | are 100% on-prem probably have worse security practices than
           | AWS, et. al. Perhaps the biggest hazard is really the hybrid
           | model. If some fintech went 100% into the cloud without even
           | an HSM on-prem to worry about, then you could probably have a
           | solid argument on the other side of the spectrum. Also,
           | remember that multi-cloud might seem like a resiliency
           | measure, but it also adds another target to your back.
           | 
           | The middle ground is where all the pain seems to be. Hybrid
           | cloud usually means more required trust than most
           | organizations ever wanted to enter into. I frequently find
           | myself as the harbinger of bad news when I get into deep-dive
           | technical calls with some of our customers. Turns out a lot
           | of the other vendors we work with like to bend the truth in
           | order to make a quick buck. Many perverse incentives are
           | pulling these massive organizations into hilariously-
           | contorted IT stances, and some of us are starting to see a
           | consulting opportunity.
        
             | marcosdumay wrote:
             | > If we are being honest with ourselves, a lot of shops
             | that are 100% on-prem probably have worse security
             | practices than AWS, et. al.
             | 
             | Does it matter? You have the same freedom to fuck security
             | up setting your AWS infrastructure as you have setting your
             | on-prem infrastructure. All the very competent AWS staff is
             | able to do is add less risk, they can't save you from
             | anything.
        
             | wolverine876 wrote:
             | > You can actually make deterministic guarantees to your
             | customers that not only your own systems are secure, but
             | also that the systems of your vendors and other 3rd parties
             | are as well.
             | 
             | You can make a "deterministic" guarantee, whatever that is,
             | that your systems are secure? That's seems pretty bold and
             | probably dangerous, no?
        
               | bob1029 wrote:
               | > That's seems pretty bold and probably dangerous, no?
               | 
               | Its not dangerous in my experience. The more dangerous
               | angle for me is this belief that it is impossible (or
               | hopelessly difficult) to build a secure system.
               | 
               | The reality is that it is only possible if you are
               | willing to take total ownership of the entire vertical.
               | If you control every single byte that enters and exits
               | your enterprise, you _can_ prove that things are secure.
               | Is it practical to do this in all cases? No. Is it
               | feasible in theory and in certain cases? Absolutely.
               | 
               | If you buy into the 3rd party hosting game, you instantly
               | lose control over the critical variables you would need
               | to in order to create the _opportunity_ for these sorts
               | of guarantees to exist in the first place. You (and your
               | customers) will be stuck wondering about side channel
               | damage and human factors that you have no direct control
               | over. When you own the hardware and the real estate it is
               | parked on top of, you can start to reel these things back
               | in really quickly with powerful policy frameworks
               | (2-person rules for critical changes, mandatory
               | checklists, etc). These sorts of policies seem to work
               | really well for very tricky areas like keeping our
               | nuclear weapons from doing inappropriate things.
        
             | whatshisface wrote:
             | > _the ability to control the flow of data can be boiled
             | down into a physical exercise of following fiber channel
             | cables in your own datacenter_
             | 
             | I suppose the infamous Equifax breach was due to a secret
             | fiber optic cable running out of their datacenter?
        
               | bob1029 wrote:
               | No it was due to the officially-endorsed fiber optic
               | cables sitting in plain sight and the fact that they do
               | business with so many other parties.
               | 
               | I work with some intermediate vendors in this space (they
               | have direct access to the credit bureau data), and their
               | security mechanisms are of concern. I am under some very
               | strict NDA constraints, but I can say that there are
               | serious problems and I am not surprised that breaches
               | occur with regular frequency.
               | 
               | You can barely trust your own in-house developers to get
               | these things right. How can you possibly hope to trust
               | many other additional parties to get it right
               | simultaneously as well?
        
               | tablespoon wrote:
               | > I suppose the infamous Equifax breach was due to a
               | secret fiber optic cable running out of their datacenter?
               | 
               | No, of course not. But when you're dealing with physical
               | infrastructure you can actually touch, it's much clearer
               | and more certain what you're dealing with.
        
               | lbriner wrote:
               | I don't think that is as true as you make it sound. I
               | have managed on-prem and cloud infrastructure and am much
               | more confident that my cloud servers are secure because a
               | whole lot of stuff is done by the provider who know a lot
               | more than me between them.
               | 
               | Even on a really simple on-prem scenario, you have
               | switches to configure, vlans to setup, hardware drivers,
               | a gazillion updates to make all the time and a tonne of
               | employees making it all very difficult. The fact I can
               | see it physically doesn't realy help me that much.
        
         | matheusmoreira wrote:
         | > Once the risks are reviewed in open and honest ways, we find
         | that virtually all of our clients would prefer to keep our
         | solution on-prem.
         | 
         | I hope this kind of risk assessment becomes more common. I'm
         | used to people not caring until things blow up on their faces.
        
           | jiriknesl wrote:
           | Well, I guess you are speaking about companies who don't know
           | what they do. But this article speaks about banks, who have
           | hundreds of servers, ability to recover anything, full-time
           | employed ops teams, monitoring & automation in place for 20
           | years already. Moving to cloud provides very little advantage
           | (definitely not financially) to such companies. They are not
           | SaaS who might need to double their infrastructure overnight.
           | Lots of advantages provided by cloud has been in place years
           | before clouds because of VMWare. Another thing is existing
           | bank apps are often monoliths (when you are happy, in Java
           | with Spring, C# .NET, when not, in Visual Basic, Cobol,
           | PowerBuilder, SAP) or huge interconnected services, huge
           | databases with thousands of stored procedures, SOAP APIs in
           | place. This is not a place where you would leverage
           | ElasticBeanstalk or AWS Lambda.
        
             | garethmcc wrote:
             | Its not as simple as that. Just because its the case of "we
             | have lots of old stuff" doesn't mean you need to ignore the
             | new stuff. Building solutions with traditional data centers
             | and staff, even if you have a lot of it, is often a lot
             | slower. You can spin up entire fleets of servers (or even
             | use services such as AWS Lambda, API Gateway and DynamoDB
             | so you never use servers) and get a solution out in much
             | less the time. Its not just about responding to load but
             | also to be fast enough to get new stuff out there.
             | Traditional financial services organisations are
             | notoriously slow to adapt to changes in the market. Using
             | cloud resources alongside the legacy infrastructure is one
             | way to try and remain competitive.
        
               | bob1029 wrote:
               | > Traditional financial services organisations are
               | notoriously slow to adapt to changes in the market. Using
               | cloud resources alongside the legacy infrastructure is
               | one way to try and remain competitive.
               | 
               | Or, you could have a single executive action revamp IT
               | resource acquisition policies. Simply mandate that
               | internalized self-service resource provisioning
               | capabilities be developed.
               | 
               | It is not rocket science to put a web dashboard around
               | vmware or some other virtualization solution. Most of
               | them already sell something like this as part of their
               | feature set.
               | 
               | You could set something like this up in a week if you had
               | enough buy-in. There are no excuses if you want to win at
               | this kind of game. The bad guys are way more patient in
               | aggregate.
               | 
               | A 2nd perspective - One of our customers has a
               | "traditional" process for setting up new IT workloads,
               | and we were still able to get 3 servers provisioned
               | within 8 hours along with 3 new publicly-routable IPv4
               | addresses and matching DNS+TLS certs. Anything less than
               | this in 2021 for _any_ organization is indicative of
               | sheer incompetence IMO. We do have some customers that
               | are really slow, but they are also really small. I don 't
               | think any F500 is taking weeks to provision SQL Server
               | anymore.
        
             | garethmcc wrote:
             | Not to mention that that same infrastructure you say is in
             | abundance is usually pretty heavily utilised already so
             | there isn't just spare capacity lying around. Procuring new
             | hardware can take months and if there is no rack space you
             | are looking at more months to get that deployed.
             | 
             | The people that need to plan, coordinate and install all of
             | this are also pretty heavily overworked at the moment
             | because, believe it or not, there is a very large shortage
             | of skilled sys admins in the world.
             | 
             | Using the cloud doesn't solve those problems, but it does
             | help reduce their impact.
        
             | JackFr wrote:
             | > Moving to cloud provides very little advantage
             | (definitely not financially) to such companies.
             | 
             | As someone who has worked as a software developer for big
             | NY banks for the past 25 years, that's simply not true.
             | 
             | The answer is, it's complicated. JPMorganChase for instance
             | has a $12 billion annual IT spend. They do a LOT of
             | different things. Admittedly certain things are best left
             | on prem for regulatory audit points (more with respect to
             | resilience/business continuity rather than security.) But a
             | substantial portion of it could be moved to the cloud at
             | some cost savings.
             | 
             | Additionally the brittleness of the service and database
             | infrastructure is a pathology of the on-prem environment
             | rather than an argument for it. Cohorts of SA's and DBA's
             | are wasting their time doing work which in a modern
             | environment would be scripted and more flexible.
        
               | [deleted]
        
         | twic wrote:
         | Does "on prem" ever include (a) on your own hardware, but in a
         | third-party colo and (b) on rented hardware, where only you
         | have root? Where do those sit between full cloud and
         | traditional servers-in-the-basement?
         | 
         | To look at it another way, how much of the risk is about the
         | physical location of machines, and how much is about who
         | operates them?
        
       | 908B64B197 wrote:
       | Something to keep in mind here: The employees of the Bank of
       | England are ultimately government employees.
       | 
       | They are subjected to the pay scales dictated by bureaucrats and
       | politicians. Of course, they can't attract the caliber of
       | engineers that works for large commercial cloud providers, so
       | they have to settle for programmers that didn't make the cut. And
       | the leadership is non-technical and from the financial and
       | government worlds (you know how these "elites" see programmers
       | and software engineers in the UK...).
       | 
       | The existing bureaucrats running the bank probably see the Cloud
       | as something that will reduce their headcount (why would they
       | have sysadmins and datacenter employees on their payroll when
       | they can simply buy it from a reliable provider), thus making
       | them look less important to other managers. The existing
       | unionized employees see it as a threat to their stable jobs (they
       | are now competing with engineers that are 10x their caliber).
       | That's a pretty bad thing for both.
        
       | jollybean wrote:
       | We need a better 'Open Stack' for cloud stuff.
       | 
       | So that you can just run a bunch of your own servers, lay over
       | the 'Stack' and then get nice Lambdas, provisioning, and other
       | things.
       | 
       | Imagine if you could just buy some hardware, and have some
       | regular IT guys reproduce most of Amazon without the Amazon?
       | 
       | That would be a 'reverse revolution'.
       | 
       | National Regulators could also support strategic investment in
       | the sector, i.e. 'financial ops have to be backed-up in-nation by
       | a local provider' i.e. some kind of forced local diversity in the
       | system by regulatory fiat. As an idea.
        
       | game_the0ry wrote:
       | I worked in a brand-name bank, so I am familiar with the goings
       | in the tech side of finance.
       | 
       | Call me crazy, but I would trust aws, microsoft, and google with
       | my PII and finances before I would trust Wells, BofA, JPMC,
       | Goldman, et al.
       | 
       | The cloud giants pay their engineers more and technologists are
       | second-class citizens at the financial institutions - infer what
       | you want from that.
        
         | tristor wrote:
         | Definitely agree. Having worked extensively with some large
         | banks I've found that they are absolutely cutting edge in
         | technology in one area and one area only: credit card fraud
         | analytics. That's because credit card fraud creates a material
         | financial risk for the bank that they can't defer to someone
         | else. Most other risks banks deal with are managed through
         | complex financial hedging, government backstops, or insurance
         | schemes and so they invest very little in doing anything
         | "correctly". Rather, almost everything about banking compliance
         | is driven by checkbox tickers and not by experts.
        
           | lbriner wrote:
           | I think it is unfair to think that banks don't want to do
           | things correctly but they suffer along with all other
           | corporates of too many levels of management to be agile,
           | systems that are far to risky to change/replace, legacy
           | systems and a high turnover of staff. If I ran a bank, I
           | suspect I would run it exactly the same way because I have
           | to.
        
             | tristor wrote:
             | Perhaps I'm too close to the issue as I have worked with
             | these businesses, but I disagree. This is intentional on
             | their part, because banks first and foremost are about
             | identifying and managing risk. If they can find a way to
             | mitigate that risk or externalize it, then they no longer
             | need to deal with it head-on. Most of the risks related to
             | IT should be dealt with head-on because they grow over time
             | as you stay still and technology moves further away from
             | you, and it's impossible for anyone to accurately predict
             | the future and therefore future risks.
             | 
             | Many banks and other financial institutions are finding
             | this out now in their desperate bid to find competent COBOL
             | programmers to continuing maintaining legacy critical
             | applications running on mainframes when most COBOL
             | programmers are retired or dead, and more are headed that
             | way every day. It wouldn't shock me to find out that the
             | effects of COVID being weighted towards worse outcomes for
             | older people had a material effect on the human resource
             | risk of using COBOL-based critical systems.
             | 
             | Banks will absolutely hold on to anything to avoid an
             | unknown risk as long as they think they can hedge or
             | mitigate known risks, and utterly fail to acknowledge the
             | truth of unknown unknowns. This will ultimately be their
             | downfall if governments ever let them follow standard
             | business outcomes, otherwise they'll eventually absorb some
             | upstart to keep hedging forever.
        
         | onlyrealcuzzo wrote:
         | My understanding is that at least Chase, Blackrock, and Visa
         | are rapidly raising their pay for new engineering / analyst
         | hires. Is this not true?
        
           | ZeroCool2u wrote:
           | I know for a fact that base pay for a first year SWE with a
           | BS+MS in CS at Goldman just broke $100k a couple years ago.
           | That probably sounds like a lot to most people, but if you're
           | hiring staff to live and work in NYC, especially when there's
           | an absolutely massive Google campus 10 minutes North of the
           | GS HQ, that's pretty much table stakes. Never mind the
           | glaring cultural differences.
        
             | elliekelly wrote:
             | Does that take bonuses into account? Goldman (and others in
             | the industry) tend to have lower base pay with a much
             | higher bonus. It's typical for low level employees to get a
             | year-end bonus equal to about 50-60% of their salary. As
             | they advance to mid-level their bonus will equal their
             | salary and, if they're in a senior position, it will
             | greatly exceed their salary.
        
               | ZeroCool2u wrote:
               | This is strictly base pay, but the bonuses are simply not
               | that good for the average SWE at the banks, because as
               | others have said, technologists are seen as a supporting
               | role. There are exceptions, but a much more senior
               | coworker of mine that worked at a number of HF's and
               | banks once told me a good rule of thumb is that the
               | further away you are from executing a trade, the lower
               | your comp. Of course, management doesn't really follow
               | that rule.
               | 
               | This is all with worse work/life balance, low
               | flexibility, (GS was the first big bank to recall
               | everyone and enforce working on site I believe.), a
               | 'conservative' tech stack if you're lucky or a simply
               | unpleasant one if you're not, and generally not being
               | valued the same way as you would be at a FAANG or typical
               | tech company.
        
               | onlyrealcuzzo wrote:
               | Do banks not offer engineers RSUs?
               | 
               | If so, $100k plus a 15% bonus is laughably low. Even
               | shitty startups pay new grads more than that.
               | 
               | My understanding is that GS's interview is actually
               | harder to pass than a lot of FAANG companies. So you have
               | to wonder why anyone would even interview there if the
               | pay is that low...
        
               | tsycho wrote:
               | "Back office" (which includes most programmers, with the
               | exception of algo traders) bonuses are much much lower
               | than "front office" (traders, ibankers, sales) bonuses.
               | 
               | Google/FB devs total comp (including RSUs) will easily
               | beat Goldman devs by 150-200%, depending upon your
               | performance.
        
               | game_the0ry wrote:
               | Bonuses are not that high at Goldman and their comp still
               | does not compare to FAANG, it never will.
        
               | Tostino wrote:
               | Keep in mind it easily could at the stroke of a pen
               | though. It's entirely their choice to not compensate
               | competitively.
        
               | onlyrealcuzzo wrote:
               | https://tipalti.com/profit-per-employee/
               | 
               | Visa regularly makes double the profit per employee of
               | FAANG - so, yes, they could easily pay more.
        
               | Tostino wrote:
               | Thanks for the source to backup that feeling.
        
             | nobodyandproud wrote:
             | Goldman's payday is through bonuses, not the base salary.
        
               | feu wrote:
               | Only if you're in a front office role, which the vast
               | majority of SWEs aren't.
        
               | nobodyandproud wrote:
               | No? Even at entry level, you're still getting $150k for
               | base+bonus. You can expect at least 25% to be bonus if
               | not more.
               | 
               | Maybe you and I have a different concept of payday.
               | 
               | What makes GS less than FAANG is the annual stock grant
               | FAANGs throw at software engineers, and of course it
               | Finance.
        
           | game_the0ry wrote:
           | I can speak for the bank I worked at - this is true, but the
           | base pay level was very low to begin with and the increase is
           | nowhere near competitive for the most talented engineers.
           | Also, the experienced engineers at those companies are salty
           | about new hires making more then they are.
           | 
           | Again, this stems from the culture of financial institutions
           | where tech workers are viewed as support staff, not
           | strategically critical, despite what financial institutions
           | say publicly.
        
         | ZeroCool2u wrote:
         | Definitely agree. In my experience, the financial industry
         | tends to lean heavily on checklists and bureaucracy to enforce
         | security. This requires additional headcount and prevents
         | automation.
         | 
         | Technology companies lean the other way and enforce security
         | through comprehensive automation of the controls they must
         | follow.
        
           | game_the0ry wrote:
           | Yes, agreed. The choking bureaucracy has the unintended
           | consequence of lowering risk - if its very time consuming to
           | build thing, you build less things that break over time, and
           | rely on old things that have worked for a long time.
        
             | Notanothertoo wrote:
             | Intended*?
        
               | game_the0ry wrote:
               | ...or intended, yes
        
       | surfingdino wrote:
       | One secretive group used to dictating their own terms unhappy
       | about another secretive group used to dictating their own
       | terms...
        
         | wolverine876 wrote:
         | The Bank of England is under the democratic control of the
         | people of England.
        
           | lbriner wrote:
           | How exactly do you figure that? Like any other quango, they
           | are almost entirely unaccountable to anyone except their own
           | staff. A Politician might be able to cause a stink but that's
           | assuming they knew enough about what was going on and they
           | probably don't.
        
             | wolverine876 wrote:
             | The elected representatives of the English people could
             | shut down the Bank tomorrow, or today.
        
           | andrewzah wrote:
           | *In theory. However, in practice...
        
       | DrBazza wrote:
       | One of the (legal ?) requirements we had to meet back in the 00s
       | was that financial software (quant code, options trading) was
       | "reproducible" for some length of time (7 years?). That meant:
       | version control (SCCS, CVS, svn), archiving OS versions (SunOS,
       | Solaris), and kit (Sun workstations and servers).
       | 
       | I have no idea if the last 2 ("OS", hardware) is even achievable
       | in 2021. I can always get my source code from git, and possibly
       | the 'configuration', but can I expect to run cloudy code in 7
       | years time?
        
         | WJW wrote:
         | I'd expect that if the cloud providers wish to acquire
         | customers that have to comply with those regulations, the cloud
         | providers will have clause in the contract that they are
         | contractually obligated to keep the same services running for
         | that period of time.
        
         | selfhoster11 wrote:
         | OS is reproducible with artifact repositories for OS and
         | library packages. Hardware perhaps less so, but I guess a
         | custom deal could be worked out with the right supplier. That,
         | or go with emulation.
        
         | throwaway_2047 wrote:
         | What about having regulations updated instead of bending the
         | industry backward to fit the outdated regulations?
        
           | DrBazza wrote:
           | I can't testify to my statement being 100% true without a lot
           | of tedious searching on the UK FSA/SFA/FCA/whatever they are
           | now's website. But that's what I have a vague recollection
           | of.
           | 
           | It's similar to HMRC's "we can come after you for unpaid tax
           | for 7 years", but you can only go after the HMRC for 5 years
           | for overpaid tax.
           | 
           | The rules are ultimately for the benefit of long term
           | investigations such as the LIBOR rigging, and Guinness
           | trials.
           | 
           | https://www.theguardian.com/business/2016/jul/04/libor-
           | riggi...
           | 
           | https://en.wikipedia.org/wiki/Guinness_share-trading_fraud
        
           | handrous wrote:
           | What's outdated about it?
        
       | ChrisArchitect wrote:
       | original/source article on Reuters:
       | https://www.reuters.com/business/retail-consumer/bank-englan...
       | (https://news.ycombinator.com/item?id=27819016)
       | 
       | any other more detailed breakdown of this somewhere?
        
         | ChrisArchitect wrote:
         | _How reliant are banks and insurers on cloud outsourcing?_
         | https://www.bankofengland.co.uk/bank-overground/2020/how-rel...
         | (January 2020)
        
       | beermonster wrote:
       | Does anyone know if CSPs can be managed via supplier agreements?
       | I've always assumed it's a commodity service and as such there
       | are standard T&Cs, TOS etc. Certainly those in the financial
       | sector wish to manage other types of supplier risk in that way
       | where possible.
        
         | lbriner wrote:
         | Isn't that the million dollar question (or probably billion
         | now).
         | 
         | Theoretically, the Ts & Cs cover everything but clearly you
         | cannot reasonably mitigate risk on the basis that the "supplier
         | told me that it wouldn't happen".
         | 
         | You have a (sometimes legal) requirement to do due diligence
         | and at least decide what additional controls you can have to
         | help with compliance and what your BC/DR process is if
         | everything really goes south.
         | 
         | Realistically, I don't really see AWS or Azure etc. being able
         | to promise that their systems can never be hacked/broken. On
         | the other hand, the assumption that doing things on-prem
         | removes this risk is naiive since we can make just as many
         | cock-ups even if we work for the company. Anyone ever forgotten
         | to setup a firewall or vpn properly?
         | 
         | True story: When I worked at a security company, the sales team
         | wanted to demo a digital camera-over-IP system and the IT
         | Manager told them not to use our internal network. They ignored
         | him and plugged it in, flooding the network with packets that
         | took down the computers and phones for about 30 minutes until
         | we worked out what had happened. That was inconvenient but
         | imagine if they had plugged in something much worse "because
         | sales"
        
       | allyant wrote:
       | Seems a bit like a nothing-article - doesn't make any real point.
       | The FCA/FSA have already been noticing a lot of the financial
       | providers placing 'all their eggs in one basket' and have been
       | putting guidelines in place for banks to have a multi-'cloud'
       | strategy.
       | 
       | [0] https://www.fca.org.uk/publication/finalised-
       | guidance/fg16-5...
        
         | lbriner wrote:
         | That's what I thought but also, when a statement is made by a
         | senior executive, I usually assume they don't really know what
         | they are talking about anyway. Loads of "keynote" speeches are
         | very hand-wavy and have a feel of importance with little of
         | real value.
        
       | politician wrote:
       | TLDR "Secretive" means "big providers could dictate terms and
       | conditions - as well as prices - to key financial firms."
        
         | DecoPerson wrote:
         | Thanks, I only skimmed this headline and assumed "secretive"
         | meant groups of developers set up their own AWS/GCP/Azure
         | accounts because they were hitting walls with the company's IT
         | dept.
        
           | daniellarusso wrote:
           | That is what I thought the article was about, too.
        
       | peteretep wrote:
       | Cloud computing feels like one area we have fantastic competition
       | in, even if there are only 3 major players (with one or two
       | smaller ones). Feature and price competition between AWS and
       | Azure is fierce, and if you're dumb enough to have any trust at
       | all in Google as a supplier then there's even GCP.
        
       ___________________________________________________________________
       (page generated 2021-07-14 23:02 UTC)