Leeds will now play Carlisle in the play-off semi-finals, but they will need to improve on this showing.
The crowd of 38,256 was the biggest in League One this season, but much changed Leeds were booed off at half-time.
Chairman Ken Bates claimed beforehand that it would be "poetic justice" if a Leeds win confirmed Gillingham's relegation after Gills chief Paul Scally backed the Elland Road club's 15 points deduction.
Leeds brought in Scott Gardner, 20, at right-back for his second start of the season and rested Dougie Freedman, Neil Kilkenny and Frazer Richardson, while Alan Sheehan was suspended.
Adam Miller, available for the Gills after suspension, had to be content with a place on the bench.
Gillingham got the start they wanted but it took a blunder by Leeds defender Paul Huntington to gift-wrap a 20th minute goal for striker Jackson.
Huntington lost possession to Jackson who drifted into the goalmouth, avoiding challenges by Lubo Michalik and goalkeeper Casper Ankergren, before scoring from 12 yards.
Huntington went some way towards atoning for his mistake with an excellent block in the goalmouth from Stuart Lewis' 20-yard drive as the Gills attempted to turn the screw.
Leeds' team changes left them short of attacking menace on the flanks, while Freedman's link-up play was badly missed and Gillingham keeper Derek Stillie was largely unemployed in he first half.
Leeds improved after Seb Carole left the bench on the hour and nine minutes later the Frenchman crossed from the right for Johnson to equalise with a crisp shot from 20 yards.
The winner arrived in the 88th minute when Andrew Hughes' long through ball was dispatched into the net by Kandol.