Ardern, who was elected previous year and announced her pregnancy in January, plans to take six weeks of parental leave.
"Thank you so much for your best wishes and your kindness".
Prime Minster Jacinda Ardern has gone into hospital to give birth, her office says.
Deputy Prime Minister Winston Peters has now taken over as acting prime minister.
The Prime Minister of New Zealand announced she had given birth in a post on Instagram, which began: "Welcome to our village wee one".
Ms Ardern's due date was June 17.
In an interesting twist of fate, the baby girl's birthday of June 21 means she shares it with former Pakistani Prime Minister Benazir Bhutto, the only other female leader of a nation to have given birth while serving in office.
The public has generally been supportive of the popular prime minister.
The politician is only the second elected leader in recent history to give birth in office.
Bhutto, who was assassinated in 2007, gave birth to her daughter Bakhtawar in January 1990, and also gave birth while out on the campaign trail in 1988. Ardern plans to take six weeks of leave before returning to work.
Prime Minister Jacinda Ardern is the first prime minister to give birth while still in office since Benazir Bhutto did it nearly 30 years ago.
However, the Prime Minister will still be consulted on all major decisions.
When announcing the pregnancy, Ardern said she and Gayford were surprised they were able to conceive naturally. "We already talk about significant issues, that will just continue, we'll just be in different roles", she said in a interview with Fairfax media just before going on maternity leave.
Ardern became the leader of New Zealand's Labour Party in September and was elected prime minister a few weeks later.
Italy's Licia Ronziulli was snapped with her six-week old child in the European parliament in 2010, and has regularly been accompanied by her little person since then - to the joy of photographers.