The five-time world champions looked in danger of suffering a shock defeat after Japan took a 1-0 lead into half-time in Houston. However, Brazil responded after the break, with Casemiro equalising before substitute Gabriel Martinelli scored a dramatic winner in the 95th minute.
The late goal became the latest winning strike in normal time during a World Cup knockout match since records began in 1966.
Ancelotti admitted Brazil had been made to work hard but insisted he never lost faith that his players would find a way through.
"Suffering is normal, as is relief," the Brazil coach said. "At half-time I told them not to lose patience because we were going to score sooner or later."
The Italian, who was taking charge at his first World Cup as a national team coach, was pleased with how his players reacted after a difficult first half.
"No one is perfect, but you can handle how to get out of mistakes, how to think and look ahead," Ancelotti explained. "The team did very well in the second half. No one thought that this team wouldn't score."
Brazil's record goalscorer Neymar remained on the bench despite his team searching for a winner. Ancelotti revealed he had planned to introduce the forward if the equaliser did not arrive.
"I spoke to Neymar and told him that if we hadn't equalised by the 60th minute, I would bring him on," he said. "We were considering sending him on during extra time because he is fit and ready. However, since we scored, Neymar didn't play."
Casemiro was another key figure for Brazil, scoring the equaliser with a header after facing criticism from supporters. Ancelotti defended the experienced midfielder and highlighted his leadership qualities.
"Casemiro is a leader," Ancelotti said. "He knows his position very well and how to play his position. No one is teaching him how he has to play in his role. That's very important."
Brazil will now prepare for their last-16 match against either Ivory Coast or Norway at the New York New Jersey Stadium on 5 July as Ancelotti continues his mission to guide the nation to their first World Cup triumph since 2002.