The hardening stance follows reports in The Times that young sex workers were hired by Oxfam's senior staff in Haiti after the 2010 natural disaster which devastated the island and left up to 300,000 people dead, and that the UK-based charity tried to cover up the scandal at the time.
She told the BBC's Andrew Marr Show: I am affording them the opportunity to tell me in person what they did after these events and I'm going to be looking to see if they are displaying the moral leadership that I think they need to now.
The majority of Oxfam's 230 staff members working in Haiti at the time are not accused of doing anything improper, but a small group of male aid workers living in Delmas, near Port-au-Prince, allegedly turned a rented guesthouse into what a source told the Times the men called "the whorehouse".
She said she would meet the charity on Monday to discuss the case, adding: "If the moral leadership at the top of the organisation is not there then we cannot have you as a partner".
The charity added it will soon deliver a "safeguarding training course for faith organisations, especially churches, so they can understand, recognise and respond to safeguarding issues, and develop a safer culture".
Women believed to be sex workers were reportedly invited to the Oxfam team house on a number of occasions.
"In the 21st century, it is utterly despicable that sexual exploitation and abuse continues to exist in the aid sector", she sad.
In a statement on Friday, the charity said: "The behaviour of some members of Oxfam staff uncovered in Haiti in 2011 was totally unacceptable, contrary to our values and the high standards we expect of our staff".
'Our approach to this matter would have been different had the full details that have been reported been disclosed to us at the time, ' it said in a statement.
Ms Mordaunt said it was her "absolute priority" to keep the world's most vulnerable people safe from harm.
Marr said: 'That was a lie, wasn't it?'
Ms Mordaunt replied: "Well, quite".
However, the minister said Oxfam did "absolutely the wrong thing" by not reporting the detail of the incidents to the government.
The charity said it publicly announced an investigation into the allegations when they surfaced and kept the industry regulator informed.
The charity said it investigated the allegations in 2011 and perpetrators were all sacked or resigned.
Caroline Thomson, chair of Oxfam's trustees, announced a package of measures to show the organisation was committed to changing.
The former Secretary of State said that "the reason why Oxfam has landed in this position is because they have not been fully open and transparent about what happened".
She added that Oxfam staff had come forward with concerns about the recruitment and vetting of workers involved in the scandal.
The Charity Commission said on Saturday that it had written to Oxfam "as a matter of urgency" to request further information.