The Reds were frustrated by Marco Silvas side for 90 minutes until Divock Origi took advantage of a horrible blunder from Jordan Pickford to score the only goal of the game.
Liverpools win means they are now unbeaten in 16 matches with Everton and remain just two points behind defending champions Manchester City.
Klopps side were far from at their best on Sunday and the Merseyside derby appeared to be drifting towards a goalless draw.
But England goalkeeper Pickford failed to catch Virgil van Dijks miss-hit shot – which appeared to be going over the bar – before Origi headed home into an empty net.
Klopp, who is known for his emotional reactions, immediately sprinted onto the pitch and celebrated with Reds goalkeeper Alisson.
Wow!!! Jurgen Klopp charges onto the pitch to celebrate with Alisson!
Liverpool score a bizzare LATE goal to win the Merseyside derby 1-0. Simply incredible! pic.twitter.com/7NM4P0pltV
— Soccer AM (@SoccerAM) December 2, 2018
Speaking to Sky Sports after the dramatic win, Klopp said: I have to apologise because I didnt want to be disrespectful but I couldnt stop myself. It just happened!
The plan was not to run [onto the pitch to celebrate]!
When I chose to stop I was close to Alisson. He was surprised! It was the last situation of the game.
Both teams delivered an unbelievable fight. Both teams tried really to play football. No harsh things. The game was really good.