Hayter, who missed a glut of chances, showed composure to net the equaliser in the 65th minute to cancel out Chris Dickson's sixth goal of the season.
Sean O'Driscoll's men had begun the game brimming with confidence but saw Gareth Roberts, Hayter and Jason Price all waste early chances and were punished in the 18th minute by a wonderful individual goal.
There seemed little on when Dickson received Nicky Southall's hopeful long ball with his back to goal, but the on-loan striker turned Rovers skipper Adam Lockwood inside out before beating Neil Sullivan with a tidy finish at his near post from ten yards.
The visitors rallied and Hayter prodded a glorious chance wide in the 33rd minute, moments before Lewis Guy saw another close-range effort chalked off for offside.
It appeared they would be punished once more when Sullivan tripped Dickson in the area on 36 minutes, but Delroy Facey belted the spot-kick against the bar and away to safety.
Rovers continued to pull forward after the break and Simon Royce had to be at his best to tip over Lockwoods close-range header.
But the keeper was helpless when, on 64 minutes, Barry Cogan pulled back Richie Wellens in the area and Hayter calmly placed the resulting penalty into the bottom right-hand corner.
And Wellens should have put his side in front moments later, but failed to convert from six yards out after Price had found him with a deft cross.
The Gills' new boss Stimson threw on Dave Graham in the 76th minute and the striker had an immediate impact, firing a curling effort from 20 yards that Sullivan brilliantly tipped past the post.
Both sides went in search of a winner and only a fine save by Royce five minutes from time to deny substitute James Coppinger prevented Rovers from snatching the win.