Some Amakhosi players and club officials were involved in a melee that required police intervention that forced the team to abandon their pre-match warm-up.
The start of the contest was then delayed for 45 minutes due to traffic congestion outside the stadium, and Ben Youssef was not impressed following the 1-1 draw.
"I think, before I start to discuss the game, I have to send a message – I'm not from South Africa but I know how many people watch the game, how many were waiting and how many ask where they can watch the game," said Ben Youssef.
"And I think what happened before the game is something we can't accept. We are sending bad picture about our football, people who are waiting to see football but what we saw before the game we can't accept.
"I think if you want to win the league, if you want to beat the teams, you have to win with football, not like that. It's about safety, we spoke about safety, but people from outside are coming and fighting with our player in our warm-up, in our half.
"I think this kind of thing is a shame and we can't accept this kind of situation and, like I said, in football, if you want to win the league, win it and deserve to win it with football, and people will say you deserve to win the league, you can't win the league like that."
The Tunisian mentor did not want to go into detail about what exactly happened on the pitch before kick-off when pressed by reporters.
"You didn't see? I was also far, I just saw a fight. People were calling me from outside, asking me if I'm okay," he said.
"People from another country were calling me to ask if I'm safe or not, and you are here in the pitch but didn't see?"