The two men, Siraj Wahhaj and Lucas Morten, were arrested at the scene, the three women were taken into custody for questioning and later released, and the 11 children were placed in custody of the New Mexico Children, Youth and Families Department.
The children's age ranged from 1-year-old all the way up to age 15-years-old.
Three women, believed to be the children's mothers, were also briefly detained, but not charged, police said.
"The only food we saw were a few potatoes and a box of rice in the filthy trailer", said Hogrefe.
Authorities discovered the children while searching for a missing 3-year-old boy, who still hasn't been located.
Morton was arrested on the charge of suspicion of harboring a fugitive.
However, the Taos County Sheriff could not secure a warrant for probable cause to investigate the location until a Georgia detective investigating the missing child received a message sent to a third party from inside the compound saying, "We are starving and need food and water".
FBI investigators had reason to believe that Wahhaj was occupying the compound with Morten and others, but they didn't feel there was enough probable cause to obtain a search warrant.
Mr Hogrefe told ABC it appeared the women and children "were brainwashed and feel great intimidation from the men that were in control of this facility".
They were turned over to state child welfare workers for their own health and safety, police said.
"I absolutely knew we couldn't wait on another agency to step up and we had to go check this out as soon as possible", Hogrefe said, who added his agency believed the occupants were likely heavily armed and of an extremist Muslim belief.
Tarps flapped across covered spaces behind the wall of tires stacked in a half circle around the property's perimeter, which contains a partially buried travel trailer, an earthen berm and other smaller structures.
On Thursday, Taos County Sheriff Jerry Hogrefe approved a search warrant for the compound because he believed that a child abduction suspect and the abducted child were at the makeshift residence.
The search began nine months earlier and 1,500 miles east - in Jonesboro, Ga., where a mother told police that her husband had taken little Abdul-Ghani Wahhaj to a park and never returned.
Hogrefe said in an interview with The New Mexican late Saturday that the women would not speak with investigators to provide information about their relationships with the suspects or other details in the case.
After the accident, the group was picked up in a 2006 Ford truck with DE license CL085217.
When Wahhaj was captured, police say he was heavily armed with an AR15 rifle, five loaded 30-round magazines and four loaded pistols.
Once both men were in custody, deputies tried to comfort the children, who Hogrefe said had no shoes and were wearing "rags" for clothing.
Wahhaj was booked on his warrant out of Clayton County.
After hearing the message, Hogrefe decided the Sheriff's Office couldn't wait on the Federal Bureau of Investigation or any other law enforcement agency; he wanted to move on the compound as soon as possible.
Throughout the day on Sunday, other area residents drove past the property as a New Mexico State Police helicopter cut the air overhead.