[HN Gopher] Deutsche Bank's "dysfunctional" IT division (2018)
___________________________________________________________________
Deutsche Bank's "dysfunctional" IT division (2018)
Author : Cwizard
Score : 168 points
Date : 2023-07-23 10:01 UTC (13 hours ago)
(HTM) web link (www.efinancialcareers.co.uk)
(TXT) w3m dump (www.efinancialcareers.co.uk)
| tragomaskhalos wrote:
| Don't think this is atypical of many organisations that have
| grown their IT over a long period. When I did some work for
| Network Rail (here in the UK) there were some absolutely hair-
| raising diagrams of umpteen systems all with lines between them,
| doubtless many going back to BR days.
| xmcqdpt2 wrote:
| This makes me excited for what current microservices
| architecture systems will look like in fifty years!
| Nextgrid wrote:
| Microservices just means you get that kind of mess in months
| instead of years/decades, and that mess is actually the
| objective.
| mxschumacher wrote:
| maintaining web frontends will also be an infinite source of
| employment
| tragomaskhalos wrote:
| True, but at least those lines (you'd hope) will represent
| some json going over http, whereas with these legacy systems
| the lines could be more or less anything!
| throwaway_db wrote:
| This is 5 year old scuttlebutt and even what's publicly known
| about DB's systems has superseded most of the details here.
| disgruntledphd2 wrote:
| Cool, can you share some of the publicly available information
| then?
|
| I interviewed at DB around 2018, and from that experience
| (what's the worst thing about working here, I asked my
| interviewer. The politics, she replied) plus that of all my
| friends who spent time there, this article rings pretty true.
|
| But I'd love to get some more up to date info, so would be
| happy if you could share.
| throwaway_db wrote:
| Much fresher scuttlebutt:
| https://www.efinancialcareers.com/news/2022/05/eisler-
| capita...
|
| Kannon isn't even mentioned in the older article.
|
| Note that I am not claiming that Deutsche's IT department is
| no longer dysfunctional! Just nothing in TFA provides
| convincing evidence about the current state of affairs.
| arianvanp wrote:
| Yeh you notice it as a customer.
|
| I can't choose a password for my account; instead they assigned
| me a 6-digit PIN (Not kidding. A PIN).
|
| When I registered my account at a local branch I gave them my
| dutch phone number. Their website's transaction page wouldn't
| load due to a JavaScript parsing error over the dutch phone
| number format. I called customer support and they basically
| reacted as if water was burning. They would promise to get back
| to me. 3 weeks later I still couldn't access my online bank
| account's transaction page. I ended up eventually manually
| editing the HTML in inspector in order to be able to navigate to
| the page where I could update my phone number to a German one and
| I never heard back from support.
|
| I tried opening a depot account for trading stocks. It just says
| "Internal server error. your account id is incompatible. Please
| open an additional account with Deutsche Bank". I then tried
| this; had to go to KYC all over again; but I can't because I
| can't register two accounts on the 2FA app that is mandatory to
| have on my phone. Customer support adviced to get a second phone
| (??).
|
| and I'm just a normal customer. It's impossible to bank with this
| bank. Every time I have to interact with them it feels like i'm
| in some weird fever dream.
| vstollen wrote:
| Similar experience with Comdirect. They also have a 6-digit
| pin.
|
| When I asked support why I have to pay for my depot (even
| though its free if I have a checking account) they said they
| signed me up under two separate customer IDs, but they cannot
| simply consolidate them. I have to open a new depot and
| transfer my stocks.
| meibo wrote:
| That just sounds like a normal day of interacting with basic
| German IT infrastructure. It's all the same.
| FirmwareBurner wrote:
| Germany, IT superpower by 2030. /s
| lobsterthief wrote:
| To be honest, average American IT infrastructure is the same;
| the larger the business, the worse it seems to be. Large tech
| companies like Google are outliers.
| lopatin wrote:
| I bank with Chase and the app is really good. Are there
| major US banks with crappy apps?
| paradox242 wrote:
| I was pretty impressed with the internal IT situation
| working at JPMC. It's not perfect, but reading this about
| Deutsche Bank I can see what the alternative is and it's
| rightfully horrifying.
| cebert wrote:
| PNC Bank's mobile app and website are pretty bad. The
| only reason I haven't switched is due to friction and
| because I have my mortgage with them. The website won't
| work unless I allow 3rd party scripts from Adobe to run.
| The UX is terrible too.
| [deleted]
| drewda wrote:
| Last I heard, Google staff use Concur to handle travel...
| there's no escaping SAP (and similarly frustrating
| enterprise software) in large corporations, even FAANG
| ones.
| FirmwareBurner wrote:
| _> Google staff use Concur to handle travel_
|
| I mean what else should they use?
| dacryn wrote:
| Zoho? Airbase? Rydoo?
| FirmwareBurner wrote:
| Nobody got fired for using Concur
| dmvdoug wrote:
| Is SAP good at anything aside from locking customers in?
| I ask this seriously because I am an outsider on this,
| but I have never heard a single good thing about SAP. NOT
| ONE.
| housemusicfan wrote:
| Rather, the large tech companies are some of the worst
| offenders, but it's coupled with egregious arrogance and a
| self-held belief that they're better and smarter than
| everyone else. Google is in an even more unique position as
| they can bully their way around the internet by breaking
| and influencing standards at their whims. Ever tried to get
| support from Google? Go ahead, I'll wait.
| jiggawatts wrote:
| That's a conspiracy theory.
|
| Reality is much mundane and boring: the person writing
| the web app has no vested interest in the app actually
| providing a good service for customers. His or her boss
| doesn't care either.
|
| It's simple disinterest: the self-interests of the coders
| are _too far removed_ from the interests of the
| customers.
|
| If you're a solo dev working on a startup and every lost
| customer is money gone from your pocket, _you will_ pay
| attention to every little detail.
|
| If you're a cog in a giant soulless machine, you do the
| bare minimum and go home.
| dmvdoug wrote:
| Not just business. My dad worked for the FBI in IT for
| about 15 years after he retired from the military. The
| stories he can tell....
| cscurmudgeon wrote:
| Have you used Bank of America's app? It is quite good.
| andersa wrote:
| My personal favorite is the website for reporting large
| international transactions, which can only be using during
| regular working hours on weekdays.
| scanny wrote:
| What exactly is that? Is it for moving savings from a non-
| eu country into Germany ?
| andersa wrote:
| This thingy: https://extranet.bundesbank.de/ams
|
| How they describe it:
| https://www.bundesbank.de/en/service/reporting-
| systems/exter...
|
| Any single transaction exceeding 12500EUR entering (or
| leaving) Germany has to be manually reported here. So for
| example if you're living in Germany and working remotely
| as a contractor for a US based company you'll probably
| have to report your income every month.
|
| There are many exceptions that don't need manual
| reporting though, like if you're buying physical goods
| from overseas. I guess those are already automatically
| tracked through other ways.
|
| For fun, I just tried to login at ~9:30 PM local time on
| a Sunday, and it said:
|
| "Access Denied"
|
| "You cannot access the resource you have requested at
| this time."
|
| "The resource you have requested is protected by a policy
| restricting access to specific time periods. Your request
| has occured outside of those permitted time periods."
| pimeys wrote:
| For some of us, using a middle company such as Deel might
| be a good idea. They take some money for that, but you
| get a real German contract and don't have to care dealing
| with the local bureaucracy.
| paulddraper wrote:
| I would imagine a water fire to be a very interesting
| phenomenon, no?
| LaurensBER wrote:
| German speaking financial institutions are a special kind of IT
| hell. Perhaps it's a cultural love for rules in combination
| with compliance that evolves in a hellish bureaucracy, perhaps
| it's something in the local beer. We can only speculate but
| I've never worked in an environment as dysfunctional as the
| ones I've seen at big German companies.
|
| To give a few examples:
|
| - Running all services under a single Linux account, then
| requiring vendors to support password encryption in the startup
| scripts (everyone can access our production passwords!), then
| proceeding to store the production password in plain text in
| GIT and Confluence where everyone could read it.
|
| - Everyone (including contractors) had access to the production
| environment through the testing environments, including
| unrestricted access to the production database (without
| requiring as much as a password). This was not deemed nearly as
| important as the passwords in the point above.
|
| - I received an official warning for disclosing to AWS/GCloud
| that we did not have CI/CD setup but would be interested in
| getting one up and running at some point. This was considered
| to be "sensitive" information.
|
| - Ansible deployments taking 12 hours (on a 6 server
| environment) because as it turns out, some random "security"
| settings slowed down SSH, incompetent people not really
| understanding what they we're doing did the rest.
|
| - Ansible being blocked because someone in the security
| department read that it only required HTTPs access, turned out
| that they were reading the documentation for the WebUI.
|
| - Requiring manual deployment of PR's to test environments and
| manual testing, the "evidence" + logs of doing this was to be
| documented in a Word document and attached to the PR.
|
| - Automation and DevOps was banned because "The law does not
| allow us to do this"
|
| - No rsync due to security reasons, great fun when you had to
| sync 200k small files :)
|
| - PRs were kept around for months, so "senior" developers could
| estimate the risk. This included fixes that were hotfixed in
| production so that it was guaranteed to break again at the next
| 12 hour deployment (thus giving the same developers even more
| reasons to be even more "cautious" with all future PRs)
|
| - Three different database schema's, three different database
| technologies, microservices, a React frontend, 50 outsourced
| developers for an internal CRUD application with 2000 entities,
| to be used by 10-20 users
|
| - All .exes not on a whitelist were banned, so in order to run
| your application you just had to rename it to word.exe :)
|
| - At one point external developers were hired, we requested a
| desktop for them that would allow them to "write" code. They
| got a locked down VM with notepad installed and nothing else
| Ingaz wrote:
| I worked for a Swiss firm and never noticed anything strange.
|
| Both founders are German speaking. Good guys.
|
| Although that was a small firm not a giant
| [deleted]
| [deleted]
| LaurensBER wrote:
| I work for a small Swiss firm at the moment as well, the
| smaller firms are a lot better especially in Zurich where
| Google also has a big office (and Meta and Nvidia have
| smaller offices) so there's lots of cross pollination.
|
| The firm in my post above is very much a Swiss firm though,
| you've probably heard of them, they're not a bank.
| luplex wrote:
| Most likely it's a lack of talent, and lack of political will
| to actually take IT seriously.
| FirmwareBurner wrote:
| _> Most likely it's a lack of talent_
|
| Lack of talent is not a national issue in Germany. German
| unis and schools produces great devs, along with the slef-
| taught "hackers" and tinkerers which are quite numerous
| there. The whole "talent shortage" in Germany is propaganda
| from shitty companies engaging in wage suppression, and
| actually a shortage of pay that keeps decent devs away from
| such companies.
|
| It's more the lack of good pay/equity to motivate people,
| but the main issues are the immovable and outdated
| bureaucracy without any wish to address it, and the heavy
| top - down approach where top management is clueless and
| dysfunctional, scaring away the top talent over time and
| ensuring that only the equally clueless yes-men stay, and
| the result is what wee can see.
|
| These large German "ships" are only good at bureaucracy,
| maintaining the jobs and status quo for those at the top,
| not innovation, efficiency(despite the stereotype) or
| customer satisfaction.
|
| My favorite about German companies is project management
| done in Excel, and bosses who don't want to use Jira and
| demand their seniors send them emails explaining the
| current work status. You can't make this up.
| progbits wrote:
| Developer hoping for management to use Jira? Now I've
| seen it all.
|
| Just kidding, nightmarish as it is, beats emails and
| excel for sure.
| monetus wrote:
| For some reason, I think forcing the use of a word document
| in a PR is the funniest part of these anecdotes. Thanks for
| sharing.
| intelVISA wrote:
| > - PRs were kept around for months, so "senior" developers
| could estimate the risk. This included fixes that were
| hotfixed in production so that it was guaranteed to break
| again at the next 12 hour deployment (thus giving the same
| developers even more reasons to be even more "cautious" with
| all future PRs)
|
| Classic strategy :D, those aren't seniors: they're Gods.
| nunez wrote:
| > DevOps was banned because "The law does not allow us to do
| this"
|
| I want that on a shirt lol
| ano-ther wrote:
| They sent me a very phishing looking email.
|
| From a domain like db-service.com, only text "there is a
| problem with your account, please call xxx", with xxx a number
| that was nowhere to be found on their website.
|
| Turned out to be a legit inquiry from some audit department.
| DrBazza wrote:
| Even though this is from 2018, it was and still is completely
| applicable to every other investment bank out there.
|
| Investment banking organizational structures are set up such that
| bureaucracy encourages 'silos' where teams can deliver quicker if
| they don't depend on other teams, and performance related pay
| dominates the managers (managing directors) thinking.
|
| Having worked at Megabank, it ran at least one system on SunOS (
| _not_ Solaris), well into the late 2010s. It had Windows XP, long
| after end-of-life.
|
| Nothing about that story is peculiar to DB.
| steveBK123 wrote:
| DB was running LotusNotes instead of Outlook well into the
| 2010s.
| galacticdessert wrote:
| So did McKinsey ;)
| steveBK123 wrote:
| OMG
| rcbdev wrote:
| RBI did so until 2-3 years ago.
| steveBK123 wrote:
| Kings
| DrBazza wrote:
| This feels like crap-Investment-Bank poker.
|
| I'll raise you a company-wide ticketing system written in
| COBOL, running on an IBM mainframe.
| steveBK123 wrote:
| That sounds kind of awesome though.
| bilekas wrote:
| Deutsche Bank has made 5 acquisitions Norisbank 1954, Bonn
|
| Carron Energy 2004, Cardiff
|
| Better Payment 2014, Berlin
|
| United Financial Group 1981, Garden City
|
| Quantiguous 2014, Mumbai
|
| Given all of these needing to integrate together to some degree,
| and the bureaucracy of banks in general, add that to their
| resistence to change / improve something related to the tech if
| it still works, none of this surprises me.
|
| You see these kind of problems in all conglomerates which have to
| absorb other companies into their ecosystem.
|
| Edit :
|
| Slightly unrelated but I have just seen that she has passed away
| last year from cancer.
|
| https://www.fnlondon.com/articles/former-deutsche-bank-chief...
|
| Edit 2 : my mistake of details
| jiggawatts wrote:
| My current consulting gig is at a government "superdepartment"
| that was formed by mashing together a few dozen small agencies
| that were previously already combined into large agencies.
|
| Their IT was as diverse as it gets, and then they were told to
| move into one building and "make it work".
|
| It's as messy and as broken as you imagine it is.
| raverbashing wrote:
| I wonder if maybe AI could help with such an ungrateful job.
|
| Yes because one division is sending the data in XML and a 3
| letter county code and values in thousands of monetary unit
| and another system does things in a completely different way
| Roark66 wrote:
| >I wonder if maybe AI could help with such an ungrateful
| job.
|
| It's not necessarily ungrateful. It can even be
| interesting, but it all depends how everyone approaches it
| and the most important thing ever is: realistic timeliness
| and outcomes.
| raverbashing wrote:
| Yeah but it's hard not to feel it's ungrateful when you
| see the 5th "creative" way of doing something which is
| pretty much like running with a spare tire and thinking
| it's all good
| dsr_ wrote:
| No, because the field that in one system is called "cash"
| is called "settled" in another; system A uses BCD for
| money, system B uses 32 bit unsigned ints reckoned in
| millicents, and system C uses a cash object which can
| define original currency, calculated currency value,
| guaranteed conversion preference, contract reference
| number, and precision.
|
| Understanding semantics is required, not just form.
| raverbashing wrote:
| Yes, you got it. But I still think AI can at least make
| it easier
| rvba wrote:
| IT guys are paid a lot not to be lazy.
|
| Changing data types in an interface is a boring job
| (didnt Office Space joke at that?) but doing this via
| some broken AI will probably introduce even more
| spaghetti.
|
| Reality is that those systems should be killed one by one
| and finally when there is one system, it should be
| upgraded to something better. What is very difficult.
| shapefrog wrote:
| 20mil for the new system, 80mil + 5 years of implementing
| for the translation layers to communicate with the legacy
| systems.
|
| > Reality is that those systems should be killed one by
| one and finally when there is one system
|
| It take so long to replace each of the systems, by the
| time you get to the "one true solution" 40 years have
| passed and you are back where you started.
| simonh wrote:
| If you mean deep learning type systems, you need to train
| them on vast training sets of similar data so they can
| learn very general patterns, similarities and
| connections. Like giving them 10,000 pictures of dogs, so
| they can learn to recognise pictures of dogs.
|
| These banking systems are all massively complex one off
| systems, often with very different sets of functions or
| even definitions of the problem, and the implementation
| details are different in almost every way possible.
| They're the exact opposite of the sorts of problems
| modern trained AI systems are good at dealing with.
| osclarto wrote:
| well said
| ta988 wrote:
| If you can write one in COBOL and it can finish
| processing the whole set of transactions in less than 24h
| you may have a chance. /s
| objclxt wrote:
| > I have just seen that she has passed away last year at 35
| years old from cancer.
|
| You've misread the article, she was diagnosed with cancer on
| 2003 aged 35, and died this year aged 55.
| bilekas wrote:
| You're absolutely right! I'll fix that up so! Thanks
| sys_64738 wrote:
| This is part of the problem with banks with acquisitions and
| why they end up being so unwieldy. In the old days we used to
| create gateways to each by "joining" them in some short-term
| strategy to merge in the long-run. But as we know nothing in
| the long-run generally happens due to changing priorities and
| differing goals of teams.
| bluepuma77 wrote:
| Norisbank was bought 2006/2012.
|
| https://en.m.wikipedia.org/wiki/Norisbank
| thilog wrote:
| One major acquisition that is missing from your list was the
| aquisition of Postbank in 2018.
| BayesianDice wrote:
| Note the article is from 2018.
| evrimoztamur wrote:
| Deutsche Bank looks like a gigantic organisation rampant with
| misaligned incentives and barely any oversight. Looking at their
| list of controversies
| (https://en.m.wikipedia.org/wiki/Deutsche_Bank#Controversies), it
| is absolutely unsurprising that their IT infrastructure is a
| disorganised and mismanaged mess. It looks like it's beyond
| saving, with its depressing history over the last decade and a
| half.
| rnd33 wrote:
| This is what I think a lot people don't understand when it
| comes to unruly IT systems in large organizations. Deutsche
| Bank does not have an IT or software engineering problem, they
| have a culture and corporate politics problem. Even if you have
| the world's best engineered systems, as soon as you have 30
| different systems you'll have a giant mess, and the reason you
| have 30 different systems is because you have groups of people
| with different incentives not interested in using someone
| else's system.
| throwaway_db wrote:
| This is really a non sequitur. It's perfectly easy to imagine a
| large organization which has a disorganized and mismanaged mess
| for their IT infrastructure, but a great compliance and
| regulatory culture, or vice versa.
|
| DB happens to have both a broken engineering culture and (at
| least historically) an even more broken culture of mis-selling
| and fraud, but there's nothing to say that one always engenders
| the other.
| marginalia_nu wrote:
| I very briefly worked in a similar organization a decade ago, and
| it checks out with how I remember things.
|
| Though in that case half the mess were from mergers leaving the
| bank with multiple instances of similar yet incompatible IT
| systems. Another source was the wave of regulation and reporting
| requirements crashing onto finance after the subprime crisis.
| Roark66 wrote:
| I've worked as a contractor in a large bank leading a project to
| migrate from a plethora of backup systems into one newly built
| system. 30% of my time was spent on filling "paperwork".
| Implementation plans, risk assessments, fail back plans and so
| on. There was a lot of this, but people in general had positive
| attitude (no big difference between contractors and employees) so
| I think of the time as an interesting experience rather than a
| negative. We too had around 30 OSes to deal with, but in itself
| it wasn't a problem.
|
| I suspect the problem in DB is not it's aging, disjointed IT
| infrastructure, but the culture.
| deadbunny wrote:
| I did a short stint at BP and it was the same. The final straw
| for me was the 8 week process to get an internally signed TLS
| cert which required signoffs by like 6 VPs across every region
| (US, EMEA, etc.).
|
| It paid well but fuck that noise.
| zxcvbn4038 wrote:
| Deutsche Bank was one of the first places I worked at, closer to
| its heyday, I absolutely loved it. But it has been in decline for
| years, and has a lot of management issues, and fiefdoms.
|
| One of my favorite stories to retell is my wife had a job there
| also and she had a medical issue from a car wreck. Her boss said
| the bank couldn't function if she was doing physical therapy two
| hours a week, basically fired her, all the paperwork was drawn
| up, sent to a manager in London to sign. The manager in London
| was "too important" to sign things so he just leaves it for
| someone else to rubber stamp in his name the next day. Meanwhile
| my wife goes across the street (literally), gets a doctor to sign
| her disability paperwork, walks it into HR - and instead of doing
| physical therapy two hours a week she was on paid medical leave
| for almost a year. Her termination paperwork did get signed
| eventually but because she was on a protected leave they had to
| throw it out.
| FirmwareBurner wrote:
| Is getting disability for a year so easy? Afaik in Austria your
| GP is not enough for that. It needs to get approved by
| supervisors of the national insurance to get it approved to
| prevent cheaters with "friendly" getting fake disability
| claims.
| rcbdev wrote:
| Yeah, in AT you'll get referred to an Amtsarzt from the OGK
| before anything - Also pretty sure the employer is off the
| hook and you get the money straight from either the OGK
| (federal health insurance) or the AMS (unemployment office)
| for any sick / disability leave longer than 6 weeks.
| nuancebydefault wrote:
| The branch of the company I used to work for got bought around
| 2013 by Lear corp. Our European headquarters shifted to a city
| near Munich. It took 6 months for us to get at laptop that was
| allowed to logon to the corporate network. After less then a
| year, suddenly our builds became very slow. After some
| investigations i sent some screendumps from task manager to one
| of our managers, showing that 90 percent of the cpu cycles was
| eaten by some security service scanning our running scripts each
| and every time they were invoked. It took several months before
| they bothered to fix the resource hog. Truly the most work-
| hampering IT I could imagine.
| [deleted]
| ahartmetz wrote:
| They acquired Postbank (where I have an account) some years ago.
| Postbank had a reputation for good IT. Now they are consolidating
| their online banking system... Of course, on the bad system of
| Deutsche Bank. It's a huge mess. I need to log into two different
| systems for credit card and bank account things. They mailed me
| account statements for the timeframe of one day once and for a
| few days another time. I never get printed statements otherwise.
| The switchover is also months behind schedule.
|
| So, Deutsche Bank is still crap at IT.
| johngladtj wrote:
| I've worked in IT (system Integration) for Postbank a number of
| years ago... Their IT was never good when you can see how the
| sausage is made
| rft wrote:
| I have been in the process of being migrated from Postbank to
| Deutsche Bank since about a year now (no, not a typo). They
| occasionally send letters, emails, pop-ups etc. that at some
| weekend they will do a migration. There is always some online
| banking downtime for that and for some reason the CC migration
| seems to have broken something with my account. I just got a
| different CC and called it good, I need a reliable way to pay
| one-off things.
|
| It has gotten that bad that the regulator is starting to get
| interested [1, German]. Luckily I was not hit by the more
| serious problems of missed payments. Really "funny" to see that
| nothing has changed since 2018, I would have thought this is a
| current article if it weren't for the year in the title...
|
| [1]
| https://www.tagesschau.de/wirtschaft/verbraucher/umstellung-...
|
| P.S. for the Germans looking for a privacy.com alternative (not
| available in Germany): Revolut took less than 30min to setup
| from scratch to first CC payment, not affiliated or anything,
| just really happy with a financial service not stuck in the
| past
| ant6n wrote:
| Uh oh, I was considering opening an account with them...
| actually they couldnt in July, because of their IT transition.
| Maybe I'll pick a different one. Any recommendations for a good
| Business Account in Germany?
| V__ wrote:
| Sadly, business accounts are kinda all over the place. I use
| Kontist, but it's pretty basic. You could also look into your
| local Sparkasse, it should have everything you need.
| konschubert wrote:
| Revolut?
| V__ wrote:
| It's a UK bank with a Lithuanian subsidiary. Which is
| fine, but means you don't have a German account number,
| nor an easy way to sue/contact if legal problems arise.
| The latter is rather unlikely to be a concern, but the
| former might be a pain to deal with, depending on your
| customers. A lot of companies have rules which require
| double-checking every transfer to non-German accounts.
| Suddenly, your payments take a few weeks longer to
| arrive.
| tough wrote:
| N26 is German fwiw similar to Revolut
| V__ wrote:
| N26 has a history of problems. I would recommend staying
| as far away as possible.
| nyreed wrote:
| Could you elaborate on that? I've not been aware of any.
| V__ wrote:
| I'm just gonna link a few German articles. They seem to
| have reestablished their phone hotline by now, but I
| don't think they have changed their "move fast and break
| things" attitute.
|
| [1] https://www.anwalt.de/rechtstipps/erneute-probleme-
| bei-n26-b... [2] https://www.businessinsider.de/wirtschaf
| t/einem-n26-kunden-w... [3] https://www.businessinsider.d
| e/gruenderszene/business/n26-er...
| tough wrote:
| Well, since we were just talking about how dysfunctional
| IT traditional banks are in germany, maybe in this
| context a -move fast and break things- attitude is
| appreciated?
| V__ wrote:
| I think the difference is, as a customer I don't care how
| dysfunctional their internal IT is, as long as the
| customer facing IT and support "works".
| ant6n wrote:
| N26 is not for corporations (GmbH, UG) so cannot be used
| for most startups. they have poor ratings.
|
| I tried Finom, but the experience was abysmal. They
| behave like some sort of shady porn subscription, not a
| place I would want to entrust my money with.
|
| I'm not sure about giving money to these Neobanks.
| They're not really cheaper, and have so many downsides.
| konschubert wrote:
| I'd say that's an issue with those companies but of
| course that doesn't help the situation
| rad_gruchalski wrote:
| I use Volksbank and I'm really happy.
| ahartmetz wrote:
| Volksbank, Raiffeisenbank, and Sparkasse are not the
| cheapest but very solid. They are typically too small and
| local to be host to the sort of high-powered CEO that
| optimizes shareholder value and then takes the golden
| parachute when it crashes. (Actually, some of them are
| legally cooperatives, too lazy to look up which is which.)
| fxtentacle wrote:
| I've been with Deutsche Bank for 14 years and it has been
| mostly smooth sailing. Some things require calling my account
| manager and/or driving to their local office with my ID card
| and their contract forms are ridiculously lengthy, but
| overall I can't remember any major issue. I use the Banking4
| app for online access and import into Agenda bookkeeping. For
| Sepa deductions from my customers, I can import CSV files
| through Banking4 and then submit them with HBCI. Oh and don't
| be afraid to ask for rebates. Good behaviour in the past may
| allow you to get lower fees, because then you're lower risk
| for them.
| valzam wrote:
| I can recommend Comdirect. Have been with them for a number of
| years. It's not a perfect all digital experience but largely a
| good offering. moved to a non-european country and was able to
| keep my account and stock portfolio with them (just have to
| tell them that you aren't a tax resident anymore so they don't
| deduct captial gains tax automatically). I am always a bit
| worried about using neo-anything for the core of my savings
| since I would like to not have to touch then for a long time.
| Having to sell or move all your assets because a startup shuts
| down isn't fun.
| locallost wrote:
| People build, sorry architect, incredibly complex systems and
| then try to stop them from being complex. I for one have learned
| to stop worrying and love the complexity. Maybe a naive question:
| why is it important that a company with thousands of people in
| the IT department all run on the same system? I mean I understand
| the many theoretical benefits of it, but in reality software is
| there to do something, and if you spend millions and millions and
| years and years on operating systems, what are you even doing for
| your customers? I am actually a Deutsche Bank customer and I do
| not care about operating systems they run. I anticipate people
| will tell me that surely this helps eventually fulfill business
| needs -- but does it really? In my experience and as others have
| pointed out, practically all other places are in the exact same
| mess, dreaming of holy grails of standardized infrastructure,
| clean code and engineering that will make other engineers pat you
| on the back. And some get it done and some can't, so basically at
| the end of the day you need people that will sort through the
| mess and get things done. It's never easy.
|
| Once they are done with all this standardization, they will
| realize they just have new roadblocks, and will spend a half
| decade trying to fix those too.
| nvm0n1 wrote:
| It does lead to real customer facing problems. A jungle of
| different systems won't have transactionality between them at
| all, it's all a mess of rec scripts, so customers will see a
| lot of weirdness like the bank being unable to decide what
| their balance actually is.
| Nextgrid wrote:
| This is not surprising at all, and is not specific to Deutsche
| Bank or even banking in general.
|
| This is a property of large, bloated legacy companies in any
| field. IT is still seen as a cost center and a secondary concern
| rather than the enabler of their business. As a result,
| pay/resources and "political capital" (for the lack of a better
| word) are allocated accordingly.
|
| IT folks there aren't given the pay nor recognition they deserve,
| so no good talent joins or stays for long enough. Junior talent
| that joins ends up just learning from the mess and has no chance
| of actually becoming "good", so the problem continues.
|
| Furthermore, the messy and unefficient IT systems benefit many
| people there, from lower-level menial positions whose jobs would
| be obsoleted by good IT to managerial positions who have a large
| list of reports to manage which gives them prestige and justifies
| their salary. Third-party suppliers also benefit as a bad IT
| system requires constant attention while a good system would
| require less attention (and a competent in-house team can attend
| do it, requiring no third-party involvement). Bad IT can also
| serve as cover - problems can be blamed on it instead of
| incompetence.
|
| Fixing it incrementally from inside is politically impossible as
| people who rely on the status-quo will fight you on every step of
| the way. The only potential way is the organizational equivalent
| of a "full rewrite" - set up a subsidiary, give it unlimited
| money and task it with building a competing product. Operate it
| like a startup with the appropriate culture (especially regarding
| tech). Once the product is competitive, migrate customers onto it
| over time. This should be feasible at least for retail banking as
| UK fintech startups proved it's not actually impossible to create
| a bank from scratch. Rinse and repeat for every vertical of the
| business.
| danieldk wrote:
| _This is not surprising at all, and is not specific to Deutsche
| Bank or even banking in general. This is a property of large,
| bloated legacy companies in any field._
|
| There is also a large cultural component. Having lived both in
| The Netherlands and Germany, dealing with banks or government
| institutions is a stark difference. The large Dutch banks (eg.
| ING) are efficient, you can arrange everything online and they
| have modern apps. With German banks for every small
| administrative change we had to go to a physical office and the
| apps/website looked as if they were literally from 1999 (and
| weird security practices like using a pin to log in). Similar
| with government institutions, in NL almost everything can be
| arranged online with a single authentication method (everything
| from requesting a parking permit to filing for taxes). In
| Germany it's a lot of literal paperwork, going to offices,
| waiting in line, and no integration between administration of
| different public institutions.
|
| My wife (who is German) says that one factor is German history
| (nazis and the stasi), which makes people distrust any kind of
| central (digital) administration. So all databases are
| uncoupled from eachother, people prefer cash, etc. Another
| thing that is really stifling in Germany is the hierarchy,
| there is so much formalistic bullshit around
| company/government/... hierarchies. I experienced this a lot
| when working with the university administration (I was a
| researcher/lecturer at a German university). No one dares to
| make choices that can have some impact, afraid that higher-ups,
| the court, or whatever will punish them for making the wrong
| choice. So they hide behind rules and necessary decisions are
| not made/postponed. Almost no one stands up to their
| leadership, so even they have wrong ideas, there is rarely any
| counterbalance.
| aleph_minus_one wrote:
| > My wife (who is German) says that one factor is German
| history (nazis and the stasi), which makes people distrust
| any kind of central (digital) administration. So all
| databases are uncoupled from eachother, people prefer cash,
| etc.
|
| True.
|
| > Another thing that is really stifling in Germany is the
| hierarchy, there is so much formalistic bullshit around
| company/government/... hierarchies. I experienced this a lot
| when working with the university administration (I was a
| researcher/lecturer at a German university). No one dares to
| make choices that can have some impact, afraid that higher-
| ups, the court, or whatever will punish them for making the
| wrong choice. So they hide behind rules and necessary
| decisions are not made/postponed. Almost no one stands up to
| their leadership, so even they have wrong ideas, there is
| rarely any counterbalance.
|
| This depends a lot on the company, and is something that you,
| in my experience, mostly experience at big companies,
| government-run companies and government agencies (this
| includes university administrations, even though at many
| faculties, things are completely fine (even many professors
| prefer to keep away from the university administration as far
| as it is possible in their job)). So, if you really don't
| like this, attempt to look for a smaller company as your
| employer and/or find a job in the private sector.
| readlikeasloth wrote:
| ... but then you have two competing organizations: OLD versus
| NEW. This is how it plays out: NEW starts as your shiny, agile,
| start-up-ish org with all the young people with the latest
| ideas. A company nearby got rid of all their freelancers?
| Great, let's hire all of them. Think big. Great meetings and
| people seem to really make progress happen. We're getting rid
| of all the REST APIs and introduce Apache Kafka as a message
| bus for the company. Only then progress stalls and OLD slowly
| begins to crawl back in. Legal department audits the agile
| process and unfortunately German laws are quite tough on bogus
| self-employment. All the freelancers have to go and NEW loses
| all their expertise. Other employees follow as this whole thing
| does not pay that great and looks more and more like any other
| 9/5 job. Fast forward, 5 years later: the startup spirit is
| long gone. Corporate culture at NEW mirrors OLD. And not only
| culture wise. The NEW, "-tech" company is not seen as an
| independent company any more but supposed to be merged with the
| parent company. Because why have two of the same, right?
| Meanwhile OLD made some progress and introduced some reform
| projects. So developers do not have to fill out a printed paper
| application to get new servers any more. Only now you have two
| Kafka clusters with completely different setups as OLD also
| started their one one at some point.
|
| My learning here: new is not new if the culture stays the same.
| Also: never underestimate the power of old. People always talk
| about the new, shiny stuff. But old was there first. And is
| much more resilient than it seems.
| Nextgrid wrote:
| Paying good salaries (2x market rate) to 300 people will
| still be much cheaper than paying market rates to thousands
| of people (which is what legacy orgs currently do) and then
| the problem goes away. Disguised self-employment is not the
| only way to run a successful engineering team.
| lIIllIIllIIllII wrote:
| there are a whole array of "neobanks" in Australia, a few of
| which have been bought out by large incumbents and their IT
| platforms subsumed into the colossus or whatever.
|
| I can hazard a guess why this hasn't happened in Germany but I
| don't want to make assumptions based on stereotypes, lol.
| jamil7 wrote:
| No this definitely happened in Germany too, Solaris has been
| around for a long time and a whole slew of German neobanks
| and trading apps have been built on top of it. You can open
| an account with most of them by downloading the app and doing
| a video identification process with your id.
|
| Culturally though, older Germans mistrust then and want to
| use the older, established banks with physical branches.
| Tainnor wrote:
| The problem is that some of these neobanks suck too, e.g.
| N26 is a bit notorious for bad security (at least in the
| past).
| valzam wrote:
| Or they were initially started by the big banks anyway. It's
| funny, when we moved to Australia we opened a joint NAB
| account. My wife still had a Citibank account. Shortly
| thereafter NAB bought Citibank, now we had two NAB accounts.
| Then we opened a Ubank account for better interest and turns
| out that one is also owned by NAB. Can't escape them...
| myth2018 wrote:
| These days I had big trouble to get a refund from an airline. I
| had their confirmation, via email, that the ticket would be
| reimbursed. But then, after some days, they alleged, absurdly,
| that I had flown the ticket and hence I wouldn't get the refund.
|
| It turns out that they were mixing up that ticket with another
| one I bought, and I only managed to find that out after almost 1
| month of trial and error talking with their staff, which pretty
| much resembled a chat-bot experience.
|
| We put ourselves in a sort of quicksand. Technology's promises
| are tempting, but there are hidden costs. We didn't know that and
| now we're finding out what happens when complexity is not tamed.
| Lessons are being hopefully learned, but I'm afraid that what we
| currently see is just the beginning of the nightmare
| nexus7556 wrote:
| You should name and shame the airline
| nvm0n1 wrote:
| Not the same poster but as airlines have come up, this German
| IT dysfunction appears there too. At some point Lufthansa
| bought Swiss. Swiss previously had a pretty competent
| website. It worked fine, so of course, they ported it to the
| Lufthansa system. What a disaster. It regressed about 15
| years overnight. Incredibly buggy, primitive ugly clunky UI.
| Ordering tickets was easy and never failed on the old system,
| on the new one there would often be just random failures
| halfway through the process and you'd get stuck unable to
| proceed.
|
| Did NOT leave me with a good impression of German IT.
| nexus7556 wrote:
| Definitely agree. Lufthansa's website and app are a
| complete mess.
| danielovichdk wrote:
| I have so many good stories from developing software inside
| banking.
|
| Every management team I have ever worked for in banking - I have
| worked 4 stints in 4 different banks as a contractor - has been
| so old school that its been impossible to apply any up to date
| constructs from the professional software world.
|
| I once had a manager who had been with the bank for 40 years. He
| simply drawed 3 boxes on a board, drawed a few lines between them
| and asked "how long will this take ?"
|
| Hmm...what is in the first box?
|
| Old school, grey and stale. That's banking for you in Europe at
| least.
| baz00 wrote:
| I have to run integrations with things those sort of people
| designed. We get EDI streams and error codes straight from some
| COBOL thing from the 70s which has been carefully ported
| through to whatever crap they're running today verbatim.
|
| Recently there was an issue where a field overflowed which
| broke literally everything. Then they added a zero to the start
| as part of the specification which broke it even more because
| half the companies on the planet were treating it as a string
| and not a number.
|
| If there's anything I've ever learned about working in the
| financial sector it is that the only thing that is protecting
| you is the legal framework around it, not the banks or
| financial companies which are universally a fucking shit show.
| sgt wrote:
| Most banks are still running COBOL at the core, I think.
| That's at least the case here in South Africa. Each bank then
| has large IT teams building interfaces and pretty things
| around the core, using languages such as C# and Java.
| fnordpiglet wrote:
| It varies vastly depending on the bank and what part. I know
| from my career some of the most advanced computing platforms
| I've ever seen across any industry were in leading banks with
| sophisticated quant trading divisions - but only in the trading
| front office. Goldman in the late 1990's through 2010 or so was
| bar none extraordinary with some of the most brilliant human
| beings alive developing systems that are unmatched today. On
| the other hand in the same bank some of the crustiest most
| process laden software management was crushing the life out of
| human souls.
| rkwz wrote:
| > Now that Cryan's gone, however, it's becoming apparent that his
| authority over some senior managers at Deutsche Bank was weak. He
| and Hammonds' ability to challenge Deutsche's most powerful
| tribal leaders appears to have been correspondingly limited.
| "It's going to be a far more uphill task to cut systems in
| future," says one senior insider. "The people who control these
| systems don't want them cut. It's all about politics."
|
| > "Hammonds was innovative, but she couldn't do what she set out
| to do," says one DB insider. "So we're just sitting here, waiting
| to see what happens next."
|
| Yikes! Sounds like an impossible uphill task.
|
| Is there any examples of such turnarounds in the recent years?
| kwant_kiddo wrote:
| Worked in a similar organisation also. The culture in european
| banks is even more dull than I expected before working there.
|
| The view on technology and infrastructure is very primitive... as
| you can also read from just the header "IT division". I initially
| thought that they were competent but greedy turns out the banks
| are really just greedy, naive and sometimes... plain stupid.
|
| I don't know if it's unique to Europe, but I am never working for
| a non tech-first company again.
| eru wrote:
| > The culture in european banks is even more dull than I
| expected before working there.
|
| > I don't know if it's unique to Europe, but I am never working
| for a non tech-first company again.
|
| For what it's worth, Goldman Sachs is reasonably competent in
| tech, and used to be even more so (at least in relative terms).
|
| Bloomberg is about half-way between the banks and the more
| competent tech companies.
|
| (I worked at both, and a few other financial companies and also
| at Google, Facebook etc.)
| LaurensBER wrote:
| Working on the buy side, I can tell you that GS is definitely
| not competent anymore, with all the cost cutting going on.
| Getting a bunch of CSV files on a SFTP server seems to be
| extremally challenging for them. Getting the numbers to match
| with what we're getting over email/on the website is
| impossible.
|
| It's so bad that they're losing serious business on this
| despite being competitive on fees and service.
|
| Bloomberg is great if you're big and bring a lot of money to
| the table. For any small/mid-size firm (think less than 5M
| ARR on Bloomberg licenses) the answer for even basic requests
| is often "we'll file an enhancement request" only for things
| to disappear and never get adjusted. To give an example, it's
| still not possible to get an hourly export of the trade
| history in CSV or XML on a SFTP server. There's workarounds
| but it's definitely not half-way competent.
| kwant_kiddo wrote:
| I almost wrote a non-US company. In general I must say that
| working for tech in European firms vs US firms is night and
| day, but that is another discussion :)
| zeisss wrote:
| Willing to elaborate a bit on this?
| mardifoufs wrote:
| Not the person you asked but in my limited experience,
| tech is actually valued in the US. Sure, there is also a
| culture of MBAs and business types taking decisions, but
| they at least understand that tech is essential
| (regardless of the reason).
|
| In Europe, tech is seen as "just IT" especially in non
| tech corporations. It's much less important than in the
| US.
|
| It's a pretty minor thing but personally I think just the
| fact that I rarely see the distinction between "software
| engineering/development" and "IT" in Europe is pretty
| telling.
|
| The salaries are also another indicator.
| rnd33 wrote:
| Same experience, putting everything related to tech (printers,
| programming, Wifi, email) into the "IT department", which is
| seen as a support service to the core business of banking,
| instead of seeing automation as business critical in reducing
| overhead and managing risk.
|
| This is why fintech (although a very overused word by now) is
| such a breath of fresh air in the banking industry, and
| desperately needed.
| drooopy wrote:
| Perfectly sums up my experience dealing with the mess that is the
| IT of major German company in the automotive industry that I work
| for. Tons of mini-fiefdoms, each with its own strong-minded lord
| who has held that position for literal decades and who wants to
| expand their influence on other fiefdoms, an infrastructure where
| every single one of its components is incompatible and was never
| designed to work with anything else and that is held together
| with figurative duct tape and tons of depressed and overworked
| colleagues who would jump to another opportunity in a heartbeat
| but the money is actually good so we all just have to deal with
| it.
| rft wrote:
| Yepp, we once spent about 1.5 hours in a meeting walking the
| org chart of such a company to figure out which 4 emails to 4
| people (plus copious amounts of CCs just to add to the
| "urgency" and inbox bloat) I had to write to unblock the
| project after 2 or 3 unproductive weekly meetings. And I did
| that as an external supplier, I would have thought it was in
| their best interested to just collect what we need and get the
| ball rolling internally. After all, they are paying us to
| deliver something.
| drooopy wrote:
| If you guys had 3 different town halls in the past couple of
| years where 3 completely different strategies and visions for
| IT were discussed, then we are probably colleagues lol
| Tommstein wrote:
| I was working for another bank a few years ago, and a recruiter
| for Deutsche hit me up about a role that sounded interesting,
| something pretty hardcore whose details I forget other than it
| involved C++. We finally get to the part about what they're
| looking to pay. Massive pay cut over what I was already making.
| The recruiter assured me they'd see if they could maybe possibly
| get me a slightly less massive pay cut. Never heard back. I
| assume they either never filled that role, or eventually found
| someone to take it who couldn't get a job anywhere else and was
| about to be homeless. Hearing that they're a shitshow internally
| does not surprise. Sometimes you get what you pay for.
| pmarreck wrote:
| 45 different operating systems??
|
| I had no idea 45 different operating systems still existed that
| were functional
| coreDNA123 wrote:
| Think of all OSs, hypervisors, mainframes and their own
| platform, iterations, versions, flavors, builds, dependencies
| etc... I personally once serviced what looked like a 2003
| windows server in a DB office where it has a weird logo to it,
| just like it was a custom/special flavor built for DB, I
| beleive at the time 2003 was already EOL but they were probably
| paying MSFT millions to keep supporting it.
| [deleted]
| throwaway_db wrote:
| It's an article written by someone who doesn't know what an
| operating system is.
| curiousgal wrote:
| That website is nothing but glogspam and straight up AI
| generated material. I wouldn't trust anything they post.
| munchler wrote:
| I had the same reaction, but I don't think they mean "operating
| system" in the computer science sense. (E.g. Windows, Linux,
| etc.) I think they're actually talking about 45 different
| systems they developed internally to run their business.
| bennyelv wrote:
| I think you're correct - I worked there for about 5 years in
| the infrastructure engineering department. We had very
| rigorous processes around the underlying platform and keeping
| the operating systems and hardware up to date.
|
| They're talking about the "banking" operating systems.
|
| Also note that Deutsche Bank (like many global financial
| institutions) is not really a single company, but more like
| 10 different companies all operating under one umbrella.
|
| It's all very well pointing at a company like Goldman and
| saying "look, they do IT well" when they don't have the same
| range of commercial and consumer businesses as part of their
| company. If you're just an investment bank, life is a bit
| simpler.
|
| On a more general note I feel like in our industry there's a
| tendency towards "perfection worship" where people think
| there's some kind of immortal "perfect solution" that a
| company needs and anything less than that is terrible and
| must be chastised.
|
| This is egoistical and forgets that IT and computing exists
| to enable companies and not the other way round. The world
| does not do computing for computing's sake. Company change
| can be almost constant, changing systems to keep up is hard
| and expensive, being able to live with imperfection is
| critical to commercial success.
| smcin wrote:
| > [DB is] more like 10 different companies all operating
| under one umbrella.
|
| A standard part of an acquisition is integrating and
| migrating IT over a defined time period, so you don't keep
| accruing and perpetuating technical debt. Who were the CIOs
| at DB?
| [deleted]
| Havoc wrote:
| I've yet to hear (in person) a single positive story about
| Deutsche. No doubt there are positives out there, but always
| found it striking how one-sided the chatter is in finance
| circles.
| Hendrikto wrote:
| I see lots of people abbreviating Deutsche Bank as DB. Germans
| think Deutsche Bahn (train company) when you say DB.
|
| Just FYI.
| diarrhea wrote:
| Complaints in this thread seem just as applicable!
| hardlianotion wrote:
| Multiplicity of systems was a feature of nearly all banks before
| 2008 and was a contributing factor to some of the inability to
| perceive and react to risk. After 2008, and forced to by
| regulators, most banks focussed a lot of resources and attention
| to consolidate and give themselves a strategic view. Deutsche
| seems to have found this task far too difficult.
| pmarreck wrote:
| As a firstborn American nerd with 100% German ancestors, it's
| basically been frustrating my entire life to watch Germany whiff
| on basically any computer- or Internet-related technology. I
| can't even attribute it to the language barrier since they're all
| taught English at an early age. Perhaps it's that old German
| pride/hubris/NIH? "Stolz"... They're a very smart people, not
| sure what gives. Perhaps a lack of respect for "nerd" aesthetic
|
| If you ever visit Germany, prepare to not be impressed by their
| Internet access (but possibly impressed by everything else)
| the_monocle wrote:
| One thing to consider: Germany has a rapidly aging population
| so it naturally gets more unlikely to fund and get success with
| new technology. Imagine trying to develop some shiny new app in
| a country where over a quarter of the population is 60+ and
| most people refuse to pay cashless let alone accept any other
| new tech.
| febed wrote:
| Personal opinion as an outsider: Germany seems to be a very
| risk averse society. They love to plan everything a year in
| advance and hate surprises. Their lives are so ordered and
| predictable that they have an unconscious resistance to change,
| compared to other cultures.
| FirmwareBurner wrote:
| _> They love to plan everything a year in advance_
|
| Apart from energy independence and defense.
| mqus wrote:
| I'm not saying you're completely wrong from a business
| perspective, but the open source hobbyist community on the
| other hand is pretty big here. Just the structures haven't kept
| up.
| nvm0n1 wrote:
| It's not that Germany is uniquely bad at software. It's that
| the USA is uniquely good.
|
| A culture in which software people are considered to be
| _really_ low status is pretty much the global default and is
| still common today, even in the US. I worked for American
| finance types before. They couldn 't understand why programmers
| earned so much and why their software team couldn't be
| completely outsourced to India. They took perverse pride in
| having absolutely no idea what their own software stack
| actually did. Last time I checked they had got rid of the only
| people who were any good and then outsourced the rest (to
| India, of course), and apparently lost the ability to ship new
| versions of their software in the process. Even in the US many
| investment banks have totally dysfunctional IT, with the
| possible exception of Goldman which is famous for actually
| being good at it (although from reading the comments in this
| thread maybe that has changed?).
|
| Why is there such a thing as a "tech" firm when all firms use
| tech? It's because tech firm is really meaning a firm created
| and run by programmers, as that's the only environment in which
| they can get respect and a productive setup. If it weren't the
| case companies like Amazon could never have existed because
| they'd have just been crushed by other better established
| retailers doing the internet well.
| MrDresden wrote:
| Having experienced working in a legacy financial institution for
| half a decade, the main takeaway for me was to never work for a
| legacy financial institution again.
|
| They are horrible places if you take your craft seriously.
| sharas- wrote:
| banks are particularly low imagination, high control places. The
| "resources" they hire are particularly generic, safe and
| fungible/expendable. By definition, such "resources" are playing
| with themselves, creating "frameworks" and "platforms". The low
| imagination management of banks cannot understand that they got
| "resources" they filtered for and results accordingly.
| https://bitslap.it/blog/posts/playground-it.html
| [deleted]
___________________________________________________________________
(page generated 2023-07-23 23:02 UTC)