At about 9pm on Monday December 4, Westpac bank customers either couldn’t log into their accounts, or if they could, noticed they had no accounts or balances, leading many to wonder they or Westpac had been hacked. It’s all fixed now – but what are the lessons yet to be learned?

So, as Westpac customers know, and as many would have heard by now, Westpac went down last night, due to a software update that went so badly wrong, it took down the Westpac network in a spectacular way.

Here’s my chat with ABC Mornings host James Vyver on ABC Radio Canberra, on Dec 5 2023 at 9.20am AEDT – the article continues below, please listen and read on!

Customers couldn’t log in, but even if they could – no bank accounts showed, not savings accounts, mortgages, credit cards or anything else. People couldn’t pay for items with credit cards, they couldn’t transfer money between accounts, their account was frozen – and not in a way that fans of Disney movies would appreciate.

Westpac has acknowledge this morning that the problem took too long to fix, as you’ll see in the X/Twitter post below, and we still don’t know whether the “updates went wrong” story is the truth, although on face value, the fact it was fixed in hours rather than days would suggest it wasn’t a hack attack or ransomware but a genuine update SNAFU that modern systems haven’t yet figured out how to truly prevent from happening, or roll back in minutes rather than hours or overnight.

Here’s Westpac’s latest Tweet:

Obviously this raises questions – should you have more than one bank account with more than one bank? It now seems obvious you should have cash at home, and in your purse and wallet, so when the next inevitable bank outage occurs, you’re not left without a way to pay for fuel, food or anything else.

It’s also a sign that, as the Optus outage showed, you should have a second SIM in your existing (or a spare phone) handy in case there’s another telco outage.

Of course, the move to the cashless society is being questioned, once again – if everything goes digital and there’s no paper based backup, like physical cash – we really are at the mercy of a cyber attack from a nation state, or a solar flare that takes out satellite networks and fries global electricity grids (as happened in the late 1800s)…

There’s also the topic of the big tech companies making sure they have way to test updates in a part of the network that, should something fail, won’t take the rest of the network down with it – as well as the ability to quickly roll back to a previous state should a change manage to screw things up.

Of course it’s hard with banks – if there were a stack of transactions that took place after an update occurred, if you reverse things back to where they were 5 or 10 minutes ago, how many transactions are then lost?

And can AI technology help to make these updates more reliable?

Clearly a lot of work needs to be done, and with both Optus and Westpac going down within a couple of weeks of each other, every other company, large and small, needs to urgently check their systems to do all they can to ensure a similar meltdown can’t occur, and if it does, that there’s a way to quickly restore things back to normal.

After all, to err is human, but to stuff up really takes a computer – and one, very shortly, with AI help!