Mr Bailey said earlier interest rate rises could have damaged the UK’s economic recovery following the pandemic.
It came after claims from politicians, including Attorney General Suella Braverman, that the Bank was asleep at the wheel and allowed inflation to get out of control.
On Thursday, officials at the Bank warned that households will face a 15-month recession and inflation of more than 13% later this year.
In response, it raised interest rates from 1.25% to 1.75%, the biggest increase for 27 years, in a move which will put further pressure on mortgage holders.
Mr Bailey told BBC Radio 4’s Today programme said he does not believe the Bank acted too slowly and that earlier action could have brought forward a recession.
“We don’t make policy with the benefit of hindsight,” he added.
“I’d challenge anyone sitting here a year, two years ago, to say there will be war on Ukraine and it will have this effect on inflation.”
The Bank had warned that Britons face two years of tumbling household incomes due to the pressure from inflation.
The runaway costs are largely due to the soaring price of energy, as the fallout from Russian President Vladimir Putin’s war against Ukraine forced gas prices skywards.
However, criticism of the Bank’s response to building inflation last year has prompted Conservative leadership candidate Liz Truss to pledge to re-examine the Bank’s mandate to make sure it has a “tight enough focus on the money supply and on inflation”.
Business Secretary Kwasi Kwarteng also called for the Bank’s targets and operations to be reassessed.
“We need to look again at what the mandate is and how best they can actually fulfil that mandate,” he told Sky News on Friday.
“That means that if your target is 2% and you’re predicting 13%, something has gone wrong and you’ve got to look at how the Bank is organised and what the targets are.
“It’s a hugely missed target.”
The Governor defended the independence of the Bank but said he would remain committed to his position even if the next prime minister seeks to make changes to the Bank’s mandate.
“I made a commitment to see the tenure out – it’s an eight-year term,” he said, after being asked whether or not he could quit in response to a shake-up.
“It’s part of the fabric of the independence of the Bank of England that doesn’t change with changes of Government and changes in views.
“We will discuss with the Government a number of issues, some of them monetary policy, some in the new financial regulatory regime. But it’s very important for the Bank that it’s clear there is this stability in the term of the governor.”
Mr Bailey also reiterated concerns over the potential impact of higher pay despite rocketing household bills.
He also stressed worries over firms seeking to pass bigger pay rises onto consumers.
“If everybody tries to beat inflation, it doesn’t come down, it gets worse, that’s the problem,” he said.
“I put this in terms of high pay rises and high price increases, because in that world it’s the people who are least well off who are worst affected, because they don’t have the bargaining power.
“I think that is something that broadly we all have to be very conscious of.”