One employee at the scene who declined to be named estimated that around 40 workers were in the building when it collapsed, but he was not sure about the number of customers.
President John Dramani Mahama suspended his electoral campaign ahead of December 7 polls in the west African nation and offered prayers for those trapped.
"My son, my son, my son! My son is going to die! Get him for me! He is my only son!" one woman could be heard screaming.
Assistant Police Superintendent Freeman Tettey told journalists that three people have so far been confirmed dead and six rescued.
Family members could be seen at the site attempting to call relatives who may be trapped, while scores of rescue workers at the scene sought to clear debris and get oxygen masks to victims.
An AFP correspondent witnessed four people being brought out of the rubble alive.
A man who saw his brother being put in an ambulance on a stretcher yelled out in agony: "He can't move! He can't move!"
The cause of the collapse of the Melcom shopping mall in the centre of the capital was not immediately clear.
Mahama issued a brief statement on his Twitter account about the disaster, saying: "My prayers are with the workers, shoppers and others who are trapped in the rubble of the collapsed Achimota Melcom building."
Another tweet from his account said he was suspending his election campaign ahead of the presidential and parliamentary polls next month because of the disaster.
"President John Mahama suspends his campaign tour of the Northern Region in the wake of the Achimota Melcom disaster," it said.
Police rushed to the scene and were still seeking to confirm details as a large crowd gathered at the site.
Soldiers arrived to help maintain order.
"The issue is still fresh," Assistant Superintendent Juliana Obeng told AFP. "They are still working to get to the root of the building. Meanwhile we have sent ambulances to the scene."
Witness Ama Okyere told AFP: "I was very close to the mall because I was going to buy something only for me to see the building coming down.
"I had to run for my life. I was so terrified. I believe there are lots of people trapped under this because this is a heavily patronised shopping mall in the area."