The champions were favourites to claim the points prior to kick-off, but the Potters registered their first ever Premier League victory in Manchester as Mame Biram Diouf's fine solo effort left the hosts stunned.
The Senegal international raced 70 yards before slotting under Hart to claim a surprise 1-0 win.
Manchester United, with new Three Lions captain Wayne Rooney in their attack, remain winless this season as they were held 0-0 by Burnley in the lunch time kick-off at Turf Moor.
Taylor Blackett, who was this week called up into the Under-21 squad for the first time, lined up in a back three that included senior international Phil Jones – and they helped to shut out Clarets forward Danny Ings, another member of Gareth Southgate’s team.
Danny Welbeck appeared off the bench for United – who had £59.7m British record buy Angel Di Maria included in their attack – but neither could find a breakthrough.
Wilfried Zaha celebrates his last-ditch equaliser
England international Wilfried Zaha ensured Neil Warnock began his second stint in charge of Crystal Palace with a point by earning the club's first point of the season with a 3-3 draw at Newcastle United.
Palace led inside 30 seconds through Dwight Gayle, then again after Jason Puncheon's 48th-minute strike.
Daryl Janmaat, then Rolando Aarons, included in Aidy Boothroyd’s U20 squad this week, equalised before Mike Williamson's 88th-minute goal looked to have secured victory for the Magpies.
But Zaha, on his second debut after returning on loan from Manchester United, equalised in stoppage time.
The home side also had Jack Colback, called-up for Roy Hodgson’s seniors for the first time on Thursday, in their midfield.
Elsewhere, Three Lions goalkeeper Ben Foster had a day to forget as his West Brom side were gunned down by three goals to nil against Swansea City.
And in League Two, Dele Ali, who will join up with the England U19s on Sunday, scored the first goal for MK Dons as they defeated Crawley Town 2-0. Benik Afobe, an U21 international, notched the second.
In the afternoon's late kick-off, Leighton Baines and Phil Jagielka were in action against Gary Cahill's Chelsea - but it is not a day the defenders will look back fondly on. After a frantic 90 minutes, Jose Mourinho's Blues triumphed 6-3 on Merseyside.
