Having taken an early 2-0 lead through Mohamed Salah and Naby Keita, Klopp’s side were pegged back by an Antoine Griezmann double before half-time.
Griezmann was sent off after the break to give Liverpool a numerical advantage and Salah’s penalty put them ahead with 12 minutes to go.
But it still required a reversal of his decision to award Atletico a spot-kick by referee Daniel Siebert after reviewing the pitchside monitor for the visitors to make it three wins from three in Group B.
“We still had to defend with all we had because even with 10 men they were quite intense to play,” said Klopp.
“But the dirty three points are very often the most important, and they were dirty tonight of course.
“It was not our best football but we got them and that’s a big step.
“When these two teams face each other then some drama I would say is guaranteed. Both teams are proper fighting units, are used to fighting for the things they want to reach.
“The story of the game is a special one because you don’t very often go 2-0 up against Atletico after 13 minutes.
“The human brain can be your biggest enemy. We misunderstood that situation completely: we wanted to control the game in the wrong way, we played in the wrong spaces and obviously gave two cheap goals away.
“I told the boys at half-time, it’s really positive because we know much more now than we knew before the game about them, we just have to use the right spaces.
“That’s what we tried to do, but then obviously the game got even more dramatic because of the decisions and the things that happened.”
Victory not only gave Liverpool a five-point lead in their group, with them able to secure qualification next time out at home to Atletico, but extended their unbeaten run to 21 matches in all competitions, their longest sequence without defeat since a run of 24 from January to May 1989.
Salah’s two goals took him to 31 in the Champions League for the club, passing Steven Gerrard’s record to become the Reds’ leading scorer in Europe’s premier competition.
By taking his tally to 12 in 11 matches the Egypt international also set another record, becoming the first Liverpool player to score in nine consecutive matches.
Griezmann’s sending-off and the overturned penalty were the final straw for Atleti boss Diego Simeone who ran down the tunnel at the end without shaking hands – much to Klopp’s initial annoyance.
However, by the time he arrived at his post-match press conference the German had a more rounded view of the incident.
“I wanted to shake his and, he was running off. I could have turned around but I was as emotional as he was,” said Klopp.
“I don’t think he did anything wrong and I am not overly happy with my reaction but that’s not too bad as well.
“We didn’t speak during the game but we were calm in most situations in a very intense game.
“But next time we see each other, we will definitely shake hands.”
Simeone saw enough in the performance to take some positives from the game but that did not take away the pain of defeat.
“I’m not happy, because I’m happy when I win, even if I play badly,” he said.
“We had clear chances against Liverpool. I don’t know if Liverpool have defended like this before.”
On the games contentious incidents which got him so animated on the touchline, he added: “On the field I can get angry, complain, but I accept the decisions.
“I don’t think it (Griezmann’s red card) hurt us. The referee did very well.”