Third division Cambridge were kicking themselves from missing a glorious chance to knock out first division opposition and make some money out of the Carling Cup.
After outplaying Gillingham for virtually all of the first half, they looked favourites when it seemed the game was certain to go into extra time after the visitors' late equaliser.
Gillingham had Tommy Johnson sent off for a foul on David Kitson in the 84th minute and were virtually a nine-man side with player-manager Andy Hessenthaler struggling with a pulled hamstring.
But four minutes from time Gillingham's John Hills lobbed a ball to the far post when Nyron Nosworthy banged in a shock winner.
Justin Walker finished off a fine four-man move in the 28th minute and Cambridge could have been more comfortably in the lead before Hills curled the ball under the crossbar from a 73rd minute corner.
Hessenthaler slammed his side for not showing enough passion. He said: "Cambridge were the better side in the first half and at the end I felt sorry for them.
"We scraped through we did not put enough in until right at the end and we were very lucky."
Cambridge boss John Taylor added: "I don't think I have ever seen us play better than in the first half we outplayed a very solid first division team.
"We couldn't quite match it in the second but it was a bit of a freak win for Gillingham and we have to remember the quality of our play when we get back to the league again on Saturday. We are certainly not downhearted."
Cambridge Utd: Marshall, Tann, Angus, Venus, Bimson, Tudor, Guttridge, Walker, Murray, Kitson, Chillingworth. Subs: Brennan, Duncan, Turner, Revell, Nacca.
Gillingham: Brown, Ashby, Hope, Cox, Southall, Saunders, Shaw, Spiller, Hills, Nosworthy, Sidibe. Subs: Bartram, Hessenthaler, Tommy Johnson, Rose, Perpetuini.
Referee: Mike Thorpe (Suffolk).
Attendance: 3,044