Rossiter, who featured in the Young Lions' 4-1 win over Holland earlier this month, was named in the starting line-up by Reds boss Brendan Rodgers.
He had a dream start opening the scoring to put Liverpool ahead, shooting past Boro's on-loan Chelsea 'keeper and recent Under-21s call-up, Jamal Blackman.
Another Young Lion, former U20 international Adam Reach, headed the Championship side deservedly level in the second half to take the game to extra time before Liverpool substitute Suso put the hosts back in front and seemingly into the next round.
But England Under-21s striker Pat Bamford, also on loan with Boro from Chelsea, had other ideas as he calmly netted from the spot in added time to take the tie to spot-kicks.
And the two sides couldn't be separated via a shoot-out either, with a second round of penalties required before Liverpool eventually won 14-13 after Albert Adomah lost his nerve to fire high and wide.
Rossiter is mobbed by team-mates after his strike
Elsewhere in Tuesday's Third Round ties, England U19 international Isaac Hayden played for Arsenal alongside senior internationals Calum Chambers and Jack Wilshere, as they bowed out of the competition to Southampton at the Emirates Stadium.
Despite the introduction of Alex Oxlade-Chamberlain and England U20s striker Chuba Akpom in the second half, the Gunners were unable to overcome an impressive Saints side that included Fraser Forster in their goal.
Alexis Sanchez gave the hosts the lead, but strikes from Dusan Tadic and Nathaniel Clyne sealed Southampton’s spot in the next round.
Dele Alli played the whole game for MK Dons as they defeated Bradford City 2-0 at home thanks to a brace from another former Young Lion Benik Afobe, currently on loan at stadiummk from Arsenal.
And England Under-21 left-back Luke Garbutt featured for Everton at Swansea City, but it won’t be a night that he will want to remember for long as the Toffees went down 3-0 in South Wales.
Fellow Under-21 man Will Hughes and U20s forward Jordon Ibe were also in action as they helped Derby County to the next round with a 2-0 win against Reading.
