The BVB started the match brightly and dominated possession but failed to create clear chances until Julian Brandt scored shortly before half time.
Dortmund conceded an equalizer early in the second half before pushing for a winner which eventually came in the 87th minute through striker Serhou Guirassy.
"It was a very intense game from both teams today," said Kovac at the post-match press conference.
"We dominated the first half and VfL dominated the second. All in all, we played really well in the first half, kept the pressure on our opponents and didn't allow them many chances, but perhaps we should have gone into the break with a second goal.
"After that, we did exactly what we didn't want to do. We slowed the game down, we lost possession too easily, and our opponents switched gears immediately after winning the ball. This gave VfL a lot of great chances."
The Croatian mentor did praise his side for their ability to get all three points in the match while reserving special praise for the players who created the winner.
"My team then showed their mentality and still managed to win the game thanks to their quality and a bit of luck," added Kovak.
"What Felix Nmecha, Fabio Silva and Serhou Guirassy did at the end was truly spectacular."
Dortmund will turn their attention to next Friday's next Bundesliga clash against Mainz at Signal Iduna Park.