The British pair had looked on course for victory a set and a break up against the Croatian pair only to be broken back and lose the second set in a relatively one-sided tiebreak.
In the subsequent tiebreak they fell way behind, briefly rallied before Cilic and Dodig rounded off the win for a place in the semi-finals.
Murray, who had pulled out of the singles on medical advice, had been targeting a third Olympic gold in Tokyo having won the singles at both London 2012 and again four years later.
And it looks likely to be his last shot at the Games although he hinted Paris could yet be a possibility in three years’ time depending on how his ailing body holds up.
Murray said: “I’ve loved every minute of playing in the Olympics. I wished that today could have gone differently. We were so close, that’s what disappointing. I’d like to have done stuff in the match to try to help out more. But yeah very disappointed.
“It’s hard. I hate losing. I wanted to try to win a medal with Joe. It’s difficult to take, it’s disappointing. You have regrets, think about points, things you should have done differently, I’ve always loved team sports, I love being part of the Olympics. I just wish we could have done better.”