Niall Currie’s men, who sit second in the Championship, hadn’t beaten the top-flight Crues in any of their last 15 meetings, but find themselves in the quarter-final draw following a memorable evening at Shamrock Park.
Kane handed the hosts an 11th minute lead but their advantage didn’t last for long as Mikhail Kennedy netted his first senior goal for Crusaders since joining from Dergview this summer.
Fyfe put Portadown back ahead in the 74th minute and that ensured they would be the only Championship team progressing to the last-eight.
Dungannon Swifts also caused a huge shock by dumping out league high-flyers Cliftonville with Matthew Lusty’s second-half effort proving enough to secure a 1-0 triumph at Stangmore Park.
Coleraine’s quest to reach a fourth consecutive League Cup final goes on after they defeated neighbours Ballymena United 18-17 in a marathon shootout that ended with Sky Blues goalkeeper Sam Johnston missing the 44th spot-kick.
Elsewhere, Larne required extra-time to navigate their way past Championship strugglers Ards, eventually progressing 5-3.
Callum Dougan opened the scoring for the hosts before Sean Graham and Andy Ryan fired the defending Premiership champions into a 2-1 lead at the break.
Ross Hunter brought Ards back on level terms and Paul O’Neill then put Larne ahead once more, but Matthew Tipton’s men drew level again through Connor Maxwell’s strike.
Lee Bonis struck in the 101st minute and O’Neill added a late second to spur any potential Larne blushes.
Meanwhile, there was further penalty shootout drama at Taylor’s Avenue.
Newry City took the lead against Carrick Rangers with Ciaran O’Connor’s first-half penalty – after Carrick striker Danny Purkis had an earlier spot-kick saved by Tom Murphy – but Stuart King’s side responded through Kyle Cherry.
Reece Glendinning handed them the lead in extra-time before Adam Salley levelled for Newry, who then won the resulting shootout 3-1.
Limavady United’s tie with Glenavon was postponed due to floodlight failure.