Yet England rallied rather than wilted in stoppage time as Kane, wearing the captain's armband for the first time, met a super cross from Raheem Sterling with an even better finish to grab a 2-2 draw.
"We're in stoppage time, 2-1 down - to get that goal was special".
"Now you can't take your eye off the immediate goal of being able to get there, but we also can't wait for qualifying and wait until next March before we start working towards things".
"He is in between me and Charlie (Mulgrew) and it is a good height that maybe Craigy could have come out but all three of us have maybe got to look at ourselves, that we could have done a bit better, we were all a bit uncertain".
England hardly sparkled against an industrious and spirited Scotland side in the Group F qualifier but Southgate believes they showed they have the right personality to get results in testing environments.
England team: Hart, Walker, Cahill, Smalling, Bertrand, Dier, Livermore, Lallana, Alli, Rashford, Kane.
"It is annoying for them -- I can't be annoyed".
Southgate added his primary goal is to reach the 2018 World Cup in Russian Federation but also said the development of the side is of upmost importance.
"They were with us all the way and they helped us get over the line or get the point".
"It's hard to sit here and say it's a good point given the manner in which the game went in the last minutes, but we'll take the point against England". "Butland was in pole position before he got that injury in Germany a year ago".
"Because the difference of where the players play and the standard we were playing against, the gulf was probably the biggest gap I have seen for a wee while in big games which I have won". I know what I want to do and this team has great ambition.
"When I first came into the England squad, Wazza was great with me", said Kane. Suddenly it looked like a bad day at the office for us.
"I am disappointed but it's important we have that never-say-die attitude. We knew it was going to be tough and credit to them for standing there and getting the two goals, but it was nice to obviously not lose the game". These guys are giants, they play at a different level to most of us, and they kept coming back.
Julian Brandt made it six 18 minutes from time before Wagner completed his hat-trick in the 85th as Germany continued its ideal start to qualifying, a feat matched only by the Swiss in Group B.