He scrambled home a stoppage-time corner by back-heeling the ball over the line despite appeals from team-mate Mark McCammon that he had got the final touch.
Darlington appeared to have salvaged a point from Rob Purdie's 78th minute penalty, but they slipped to their first defeat of the season.
Simeon Jackson gave Gillingham a 20th minute lead against the run of play.
Darlington appeared to have their minds on a missed chance when Purdie fired wide.
From the goal-kick taken by Simon Royce, Jackson burst through the static Darlington defence to fire low past Simon Brown for his first goal of the season.
Darlington made little inroads into the Gills defence for the rest of the half and, shortly after the break, brought on Liechtenstein international Franz Burgmeier for his debut.
Ten minutes later, Darlington fullback Neil Austin moved forward to send in a 25-yard chip which Gills keeper Royce saved at full stretch.
With 15 minutes remaining, Darlington brought on teenage striker Curtis Main and, with his first touch, he brought another smart save out of Royce.
It was frustrating for the home side who had dominated for much of the game.
But once Darlington equalised, Gillingham took charge with Jackson bringing a good save out of Brown before Richards struck the winner from Nicky Southall's corner.
And they thought they had got their reward when Burgmeier was brought down by John Nutter with Purdy driving the penalty high into the net.