Bulelani Nikani scored the only goal at the Princess Magogo Stadium after just 15 minutes.
The three points lifted his side up to fifth place on the standings with 19 points from 12 games - and left Usuthu in sixth with 18 points - although from one game less.
It was a disappointing result after a run of three games without defeat and following a 3-0 hammering of Chippa United last weekend.
Speaking after the game, Zwane admitted that they wasted an opportunity to move up in the standings.
He said on IOL: "Last season, we had five opportunities to move into fifth position, and we didn't take any of them.
"It was easy to get into position five. Whenever there was a game for us to move higher up, we dropped points. That's the mentality I need to fix.
"We don't need that. We need to take our chances and ensure that, by hook or by crook, we see ourselves in a top position this time around. But it's a work in progress. I'm just disappointed that we lost the game."
Zwane also revealed his concern at a growing injury list after defender Keagan Allan was forced off against Polokwane.
"I'm still going to meet with the medical team, and they'll give their feedback to the technical team," explained the coach.
"However, it's a worrying factor because we've lost a few important players from the starting lineup. That also breaks the rhythm of the team. It's worrying the way we're losing players, especially the core of the team."
Golden Arrows coach Manqoba Mngqithi says he is expecting a 'different' Carling Knockout Cup semi-final match against Marumo Gallants at King Zwelithini Stadium on Sunday.
Siwelele head coach Lehlohonolo Seema has singled out midfielder Grant Margeman for praise following the 2-0 Betway Premiership victory over Stellenbosch in midweek.
Miguel Cardoso says Mamelodi Sundowns need to build on their 1-0 midweek win over TS Galaxy to make up for a poor start to the Betway Premiership campaign.
Mamelodi Sundowns coach Miguel Cardoso has praised Sphelele Mkhulise's influence and professionalism, describing the midfielder as a player who represents the very identity of the club.
