Bafana suffered a disappointing 2-0 defeat to co-hosts Mexico in their opening Group A encounter at the Azteca Stadium, a result that sparked widespread criticism back home. Much of the backlash focused on Broos' cautious tactical approach, while television pundits and former players also questioned the team's overall display.
Speaking from the team's camp ahead of Thursday's crucial clash against Czechia in Atlanta, Goss admitted he was disappointed by the negativity directed at both the players and coaching staff.
"It's just disappointing that there are people who have been in the game and the comments that they give," said the goalkeeper.
"For some, they are people who you played with and you don't expect people who you were with to speak so much negative stuff."
The Siwelele shot-stopper suggested some of the criticism may stem from the desire many former players had to experience football's biggest stage themselves.
"It is everyone's dream to be here at the World Cup, maybe in a way they also wanted to be here. But we are a positive bunch."
Goss was equally vocal in defending Broos, insisting the Belgian coach deserves far more recognition for what he has achieved with the national team since taking charge.
"It's unfortunate that we aren't always realistic as South Africans," Goss said.
"When you look at the disparity against a Mexican team ranked 15th or 16th in the world, the difference is clear. The criticism directed at the coach is unfair."
The goalkeeper pointed to Broos' achievements, including guiding South Africa back to the World Cup and leading the team to a bronze medal at the 2023 Africa Cup of Nations.
"If you look at the coach's tenure, I think his progress has been phenomenal - something we haven't seen in a while. He's helped us qualify for the World Cup and secured a bronze medal. The criticism he has been receiving is simply unjust."
Despite the setback against Mexico, Goss believes Bafana can draw confidence from their AFCON campaign in Ivory Coast, where they recovered from a 2-0 opening defeat to Mali before finishing third.
"We need to have that mentality that we can bounce back," he said. "If we do that, I think we can get out of the group."
The goalkeeper also stressed that the squad remains united following the defeat, which included an error in the build-up to Mexico's opening goal.
"We support each other, we don't blame anyone and we are a team," Goss explained.
"You can see the team vibe is back after the two days we had after the game and we are happy and excited to go again."
With matches against Czechia and Spain still to come, Goss insists Bafana remain confident of reaching the knockout stages and proving their doubters wrong.
"The way we started the tournament was not according to plan, but we still have two games to get out of the group," he said.
Bafana have now shifted their focus to Atlanta, where they hope a strong response against Czechia can reignite their World Cup hopes and reward the faith they continue to place in Broos and his methods.