Out of the talks, a meeting has been arranged for July 12 at the Korean demilitarized zone to discuss with North Korean officials the return of U.S. service members remains, according to Pompeo, who said that the date could move "by one day or two".
The tough statement comes hours after Pompeo said progress with North Korea had been made "on nearly all of the central issues" including denuclearisation, although he insisted that work still needed to be done.
The North Korean spokesman said a "shortcut" to a nuclear-free Korean peninsula was through a step-by-step approach under which both sides took steps at the same time.
A former top US diplomat for Asia, Daniel Russel, said the setback was to be expected and warned Trump he is engaged in a long negotiation that would not produce easy quick, made for television results that the president likely wants.
Pompeo said North Korea had confirmed it meant to destroy a missile-engine testing facility and the two sides discussed the "modalities" of what that would look like. Mr Pompeo said he "slept just fine".
Pompeo, speaking at a press conference with his Japanese and South Korean counterparts, said he had "lengthy discussions" about complete denuclearisation with North Korean officials.
The secretary has come under increasing pressure to produce results, with Trump having touted the summit as a game-changing moment that eliminated North Korea's nuclear threat.
However, in another twist of approach towards Kim - who Trump once called "Little Rocket Man" - the USA administration renewed its sanctions on North Korea for another year.
That term has been repeatedly used by Pompeo and other US officials to describe Washington's aims in its talks with Pyongyang, though the State Department recently began using another term in its official statements: final, fully verified denuclearization.
The North's statement, coming so soon after Pompeo's trip, was sure to fuel growing skepticism in the US over how serious Kim is about giving up his nuclear arsenal.
The United States may not ultimately agree to North Korea's goals, but Wit and others said that talks should continue.
"The North Koreans are in the game to get, not to give", said Daniel Russel, the top USA diplomat for East Asia until past year.
He had said progress was made during his two-day visit to Pyongyang.
But Pompeo said Saturday that North Korea remained "equally committed" to denuclearization.
The task of establishing the disarmament programme now falls to Pompeo, who is seeking a formal declaration by the North of the size of its nuclear programme as well as an eventual timetable for it to be ended under global verification and inspection.
CVID stands for complete, verifiable and irreversible dismantlement of the North's nuclear program, an oft-cited demand by Washington for Pyongyang.
"The North Koreans are in the game to get, not to give", said Daniel Russel, the top USA diplomat for East Asia until a year ago. He stressed that "there's still more work to be done" in others, which will be handled by the working groups.
Trump raised the issue when he met with Kim in Singapore and Pompeo also talked about it during his latest visit to Pyongyang.
Mike Pompeo and Kim Yong-chol say goodbye as the United States envoy departs for Japan.