Junior Zindoga scored the only goal of the game in the 18th minute at the Seisa Ramabodu Stadium.
Usuthu appeared to have equalised in the second half but he goal wasn't given as the ball was adjudged to have not crossed the line.
Vilakazi's side then had a penalty appeal waved away after Rowan Human was upended inside the box.
Speaking after the game, the coach said on SuperSport: "Firstly, I'm very disappointed with the referee today.
"Everything was going against us, but we are very proud of Arthur [Zwane] for the effort of the players. They were working very hard, it was just not meant to be, but the effort was there."
Vilakazi added: "You can check maybe their goal, maybe that was not our best structures defensively and that's why we suffered, but other than that, they worked very hard and created chances which needed to be converted.
"I'm very much disappointed in the referees and their decisions apart from the goal, we are not a team that always complains about referees but this game was very bad and we suffered. That's disappointing for our football in South Africa."
Marumo Gallants coach Alexandre Lafitte says the 4-0 defeat to Golden Arrows in the Betway Premiership last weekend is motivation to reach the Carling Knockout final.
Golden Arrows coach Manqoba Mngqithi says he is expecting a 'different' Carling Knockout Cup semi-final match against Marumo Gallants at King Zwelithini Stadium on Sunday.
AmaZulu coach Arthur Zwane admits it was disappointing to lose at home against Polokwane City in their midweek Betway Premiership clash.
Kaizer Chiefs forward Wandile Duba is refusing to let a lack of playing time derail his ambitions, despite facing one of the most competitive attacking line-ups in the Premier Soccer League this season.
