Heffernan came on as a second-half substitute to strike in the 84th and 89th minutes - just as extra-time was looming.
Victory was no more than Doncaster deserved but it was heartbreaking for Gillingham who played their part in a cup tie played at a tremendous tempo.
The Gills worked their socks off to close Rovers down and compete in midfield but the home side always had the edge.
Rovers played some neat attacking football and stamped their authority on the game at the start with James Coppinger twice testing visiting keeper Tony Bullock in the first 16 minutes.
Simon Marples had a magnificent cross cleared off the line by John Wallis in the 19th minute when it only needed a touch to turn the ball into the net.
But the Gills weathered the storm and began to create chances of their own with Ian Cox having a goalbound shot deflected in the 27th minute before Steve Hislop hammered a great chance wide six minutes later.
Gillingham had keeper Bullock to thank for them going in level at the interval when he produced a brilliant instinctive save to keep out a darting header from Lewis Guy which looked bound for the bottom corner.
The second half followed a similar pattern with Rovers forcing the pace but failing to find a cutting edge to their slick approach work.
Michael McIndoe missed a good chance in the 50th minute and Neil Roberts had a header saved three minutes later before Paul Green slammed a super shot inches wide in the 64th minute.
The game looked set for extra-time, but Heffernan had other ideas and he broke the deadlock with a splendid glancing header from Marples' cross in the 84th minute.
Matthew Jarvis went close to equalising in the 87th minute, but it was all over when Heffernan pounced again with a clinical finish from Guy's incisive pass a couple of minutes later.