Chelsea head into the return fixture at the Emirates Stadium trailing 3-2 on aggregate after a narrow defeat in the first leg at Stamford Bridge.
Despite his side's recent upturn in form, Rosenior expects Arsenal to finish the job and book their place in March's final, where either Manchester City or Newcastle await.
Rosenior, who has won six of his seven matches since taking over as Chelsea's new head coach, acknowledged that overturning the deficit represents a difficult task.
Chelsea are widely considered to be in good form, but Rosenior feels momentum in the tie still lies with Arsenal, particularly given the venue and the Gunners' league position.
He also suggested that the pressure will be firmly on Arsenal to protect their advantage, noting that defending a slim aggregate lead can bring its own challenges.
Arsenal come into the match on the back of a convincing 4-0 win over Leeds on Saturday, a result that helped arrest a mid-season dip in form. Prior to that victory, Mikel Arteta's side had drawn with Nottingham Forest and lost to Manchester United in January.
"Players are showing me that they're engaged," Rosenior said of his team's strong second-half showings.
"The players who are so important and frustrated not to start games are all coming on and having an impact. The reality is that Arsenal are favourites for the tie. They're a goal up and they're at home, so they'll expect to go through.
"We need to take it as far as we can and as deep as we can and then the second half hopefully will be massive in terms of turning the tie around.
"It's going to be a really physical game on Tuesday and preparing the team as best I can for that game."
Everton have confirmed the loan signing of Chelsea winger Tyrique George late on Monday night, completing the deal more than two hours after the transfer window had officially closed.
Arsenal manager Mikel Arteta has said his side still had "a lot to do" despite holding a 3-2 lead ahead of Tuesday's League Cup semi-final second leg against an in-form Chelsea at the Emirates Stadium.
Arsenal have confirmed that midfielder Mikel Merino has suffered a bone injury to his right foot and will undergo surgery in the coming days.
Chelsea's season took another dramatic turn at Stamford Bridge on Saturday as a booed and bewildered side somehow emerged with a 3-2 victory over West Ham, powered by a second-half transformation sparked by Liam Rosenior's bold use of his bench.
