Paraguay Foreign Minister Luis Alberto Castiglioni said at a press conference that the move to Jerusalem was "unilateral, visceral and without justification", according to the Associated Press.
The step by Paraguay - announced by its foreign minister on Wednesday - marked a reversal of a previous decision made in May by former President Horacio Cartes to follow the U.S. and Guatemala in establishing embassies in Israel's contested capital city.
Then-President Horacio Cartes, who left office in mid-August when the government of Mario Abdo Benitez took over, announced in late April during an event in Asuncion marking Israel's 70th Independence Day that he planned to move the embassy before the end of his term.
The Palestinians said the latest "diplomatic achievement" was the fruit of lobbying efforts by their officials.
In the statement, the foreign ministry said that Paraguay had pledged two weeks ago to return the embassy to Tel Aviv during a visit by Foreign Minister Riad al-Malki to the country.
Paraguay's foreign minister called Israel's reaction "disproportionate".
The Palestinian Authority (PA) presidency welcomed the "courageous" decision taken by Paraguay and attributed it to "the wise and vigorous Palestinian diplomatic efforts to show the danger of the transfer of embassies to Jerusalem".
Cartes had traveled to Jerusalem to inaugurate the new embassy in May.
The statement said Malki "expresses his appreciation and gratitude to the Paraguayan president and his foreign minister for moving the embassy back and for their commitment to global law".
Benitez, the grandson of a Lebanese immigrant, said that he was not consulted about the move.
The nation of Paraguay has chose to move its embassy back to Tel Aviv after initially being one of a small handful of nations that had joined the United States in opening embassies in Jerusalem. Netanyahu also holds the foreign minister's portfolio.
The United States and Guatemala also moved their embassies to Jerusalem in May and Netanyahu has attempted to persuade other countries to follow their lead. "Only respecting global law and United Nations resolutions will help to achieve a just and lasting peace", he added.
Not to be outdone, Paraguay announced on May 21 that it would be moving its embassy to Jerusalem.
Most countries do not recognise Israeli sovereignty over the entire city and say its final status should be set in peace talks. In retaliation, Netanyahu recalled the Israeli ambassador from Paraguay.