A mosque official said the attacks were triggered when several people, including some Buddhist monks, demanded a search of the main building after soldiers had inspected a 43-hectare lake nearby.
Tensions have been running high in the Buddhist-majority Indian Ocean island nation since the attacks by seven suicide bombers who struck two Catholic and one Protestant church and three luxury hotels.
But they came back, and this time there were about 1,300 people.
Abdul Bari, 48, told Reuters his small brick shop had been burned down with a petrol bomb. "He was ducking, he was ducking", Dilara said.
It cost the business $113,100 (20 million Sri Lankan rupees) in goods that night.
Further attacks were reported close to the capital Colombo, as mosques where attacked in Chilaw town.
Since then, Muslim groups say they have received dozens of complaints about people being harassed.
The man was hacked to death, confirmed Rauff Hakeem, a Cabinet minister and and leader of the Sri Lanka Muslim Congress. "Mobs had attacked him with sharp weapons at his carpentry workshop", a police official told AFP. Video footage shows some parts of the factory still on fire, with most of it reduced to smouldering rubble.
Police arrested six people they believed to be behind the mob attacks.
Others blamed the police for failing to disperse the crowd.
CPA also expressed concern over reports indicating inaction and/or delays in response to this violence by the security authorities, "an unfortunate trend which has repeatedly been witnessed in the past".
Social media platforms remain blocked after a government decision, and several Sri Lankans also complained of mobile data being interrupted. "Security forces are working tirelessly to apprehend terrorists and ensure the security of the country, but each time there is civil unrest, we increase their burden and hamper ongoing investigations", he said.
The Special Advisers acknowledged and welcomed the swift response of the Government, including by deploying the security forces to protect affected communities and addressing the spread of false information and incitement to violence.
The government's warning against panic caused on social media: Sri Lanka's defence secretary General (Rtd) Shantha Kottegoda urged the public to approach the authorities with security threats instead of spreading it online, stating that it "would only cause panic and unrest".
One of the victims described the moment the family home was attacked. "They smashed the windows of our new house where we live with our parents", he said.
There have already been clashes between Christians and Muslims in Negombo, the town north of Colombo that was one of the targets for the suicide attackers.
Some, including former president Mahinda Rajapaksa, fear a similar fate to the Tamils during the pogrom of 1983.
Police said there were incidents of mobs pelting stones and torching motorcycles and cars owned by Muslims.
Muslims said this week's violence was more widespread.