Neil Warnock's FA Cup semi finalists had their minds on the Premiership as goals from Steve Kabba and Paul Peschisolido powered the Blades a step closer to the play-offs.
But is was Andy Hessenthaler's unfashionable side, bouncing back from a 6-0 thrashing against Wolves at the weekend, who almost stole the show.
United keeper Paddy Kenny foiled Gillingham in the 39th minute after defender John Curtis blundered in the United area.
The United full-back had time to clear as the lively Mamady Sidibe steamed in to slam a curling shot towards the top corner as Kenny, stranded and back-peddling, stuck out a hand to tip the ball over the bar.
Kabba struck a minute before the interval after Michael Tonge's corner had been headed into the goalmouth by Michael Brown, the former Crystal Palace striker sticking out a foot to steer the ball over the line from four yards.
Gillingham were back on level terms in the 53rd minute. Nayron Nosworthy's cross from the right flank was met by Simon Osborn and the midfielder finished in clinical fashion, lashing the ball into the top corner from just inside the area.
Dean Windass was foiled by the woodwork minutes later, the former Middlesbrough defender beating his marker only to crack his header against the bar from six yards.
Sidibe should have grabbed a second in the 61st minute but spooned the ball high over the bar before Paul Shaw struck in the 70th minute.
Kenny could only push away Nicky Southall's stinging shot and Shaw fired past the helpless keeper from six yards.
Peschisolido snatched the equaliser nine minutes from time. Brown touched a short corner to Tonge and the Canadian striker, with his back to goal, twisted to tuck the ball low into the left-hand corner.