Online sales put in their worst monthly performance in nearly five years in August as people chose to watch the Olympics on television rather than shop on the internet.
Non-store retailing sales volumes, which also includes mail order, fell by 6.7% between July and August, the biggest drop since December 2007, according to the Office for National Statistics (ONS).
The London 2012 Games had a negligible overall effect, as UK retail sales excluding fuel fell 0.3% month-on-month in August.
Samuel Tombs, UK economist at Capital Economics, said the figures "add to the evidence that the Olympics did not provide the hoped-for boost to spending on the high street".
He also said: "With consumer confidence still very weak and inflation set to outpace earnings growth for another six months or so, we would not be surprised to see further falls in retail sales in the coming months."
There was some positive impact from the London 2012 Games at sporting goods stores, which helped sales volumes in the "other stores" category jump 1.5% month on month.
But this was offset by a 0.7% fall in department stores and a 2.7% fell in household goods stores between July and August.
Food stores saw a slight 0.2% rise in sales volumes in the period, which some retailers put down to improved weather and the Olympics.
Suit specialist Moss Bros is among retailers that have blamed Britain's summer of major events for slow business.
The retailer said it had missed out on nearly £2m of hire business due to events such as the Queen's diamond jubilee and the Olympic Games as more weddings were deferred.
Hopes that the squeeze on consumer spending will ease have faded as economists warn that inflation will fall more slowly than previously expected.
The rate of inflation edged down to 2.5% in August from 2.6% in July, while average earnings rose 1.4% in the year to July.
Rising petrol prices, utility bill hikes and higher tuition fees will pile pressure on the cost of living in the months ahead.