The Chancellor, in a statement to the Commons, said the investigation by the Financial Services Authority (FSA) had uncovered "systemic failures" at the heart of the financial system.
The FSA is now in talks with the Serious Fraud Office about how to proceed as it considers "every avenue open to them", Mr Osborne said.
Amid calls for a criminal investigation, he added that the FSA is not currently able to impose criminal sanctions for rate manipulation but suggested action against directors for criminal negligence may be possible.
"It is clear that what happened in Barclays, and potentially other banks, was completely unacceptable, was symptomatic of a financial system that elevated greed above all other concerns and brought our economy to its knees," he said.
Shares in the bank plunged up to 17% on Thursday as calls for its chief executive Bob Diamond to quit over the row continued to escalate.
In the Commons, Mr Osborne said the bank boss had "some very serious questions to answer".
He asked: "What did he know and when did he know it? Who in the Barclays management was involved and who, therefore, should pay the price?"
The Treasury Select Committee has already summoned Mr Diamond to appear and Mr Osborne said: "We all want to hear his answers."
Investigations into practices at more than 20 banks are continuing after Barclays was caught and fined £291m for trying to manipulate inter-bank interest rates.
Emails revealed in the FSA report showed traders promised each other champagne to fix the rates, which affect millions of businesses, homeowners and savers.
Mr Osborne said the cosy messages between traders and rate submitters "read like an epitaph to the age of irresponsibility".
He vowed that the Financial Services Bill, which is currently going through Parliament, will create a tougher regime and said the Government would consider strengthening criminal sanctions for City abuse.
He also revealed that he is considering changing the rules on fines imposed on banks so that they go to the taxpayer. Officials are looking at whether this could be backdated to cover the Barclays fine.
Mr Osborne said: "The story of irresponsibility is not over yet. Our financial services should be a source of economic strength and national pride for this country.
"But failures in our banks and financial system have cost the country billions and put thousands out of work. Those responsible should be held responsible."
Earlier, Labour leader Ed Miliband called for a criminal investigation, insisting in a speech to the Unite union that "this cannot be about a slap on the wrist".
"When ordinary people break the law, they face charges, prosecution and punishment. We need to know who knew what when, and criminal prosecutions should follow against those who broke the law," he said.
On a visit to West Yorkshire, David Cameron stopped short of calling on Mr Diamond to resign but said managers at Barclays had to account for themselves
"I think the whole management team have got some serious questions to answer. Let them answer those questions first," he said. "Who was responsible? Who was going to take responsibility? How are they being held accountable?"
Former Barclays chief executive Sir Martin Taylor said someone high up at the bank will "certainly have known what was happening".
"I can't believe Barclays haven't identified who that is. They've been investigating for years, so have the FSA, and no doubt they will take appropriate action.
"It's really for the board to decide whether Bob Diamond, who has amazing leadership qualities and huge personal following in the organisation, can be the person to turn the page on this or whether he's part of the problem."
Lord Oakeshott, appearing on Sky News, said: "If Mr Diamond had any shred of shame, he would resign, and if the Barclays board had an inch of backbone, then they would get rid of him and get a proper professional banker to sort it all out."
Mr Diamond, who was in charge of Barclays Capital at the time the breaches happened between 2005 and 2009, has already waived his annual bonus for 2012 following the fine and apologised.
Tracey McDermott, the FSA's acting director of enforcement and financial crime, says investigations are ongoing - opening to door to fines on other institutions worldwide.