There were conflicting reports about casualties in the blast.
Citing Taluka Hospital Medical Superintendent Moinuddin Siddiqui, Dawn reported that at least 30 bodies and more than 100 injured were brought to the hospital.
The bomber entered the shrine as crowds massed on Thursday, a statement from the Sindh police spokesman said.
The blast hit as Sufi Muslims were gathering to perform the dhamaal ritual.
"This is an attack on entire Pakistan", Sharif said in a statement adding that the country was facing a hard time in the last few days but "the nation stands united against terrorists and will win this war".
It was one of the worst attacks in the past few years on the nuclear-armed country, which has suffered a string of ISIS- or Taliban-based attacks since Monday, splintering a period of increased security. "We stand for our nation", he said in a statement tweeted out by DG ISPR Major General Asif Ghafoor. Without providing any further details, he said "we shall defend and respond". "No more restraint for anyone", the army chief was quoted as saying.
Murad Ali Shaf, chief minister of Sindh Province, told NBC News that hundreds of people were at the service when the attacker detonated the bomb. The incident happened when a large of devotees were gathered on a Thursday night, killing at least 40 people and injuring 170 others.
No group has claimed responsibility for the attack, but the government in Islamabad has blamed the Taliban-linked militant groups for such attacks in the past. The first explosion took place in the basement reserved for ablution and the second in front of the shrine's main gate.
On Wednesday, seven people were killed when suicide bombers targeted a van carrying judges in Peshawar and a government compound in the restive Mohmand region. The driver of the van was killed and five people, including the judge, were injured.
On Tuesday, two police officers were killed while trying to defuse a bomb in Quetta.