The demolition, which began late Saturday, continued overnight and bulldozers were busy as dawn broke in Abbottabad's suburban Bilal Town neighbourhood.
Witnesses said troops blocked access to the compound, brought in heavy machinery and barred journalists from taking pictures or coming close to the site.
Around 500 police were deployed, some of them manning an outer cordon while soldiers were positioned at the inner perimeter around the compound.
"The demolition work by security forces, including troops, continued overnight," a police official on the spot told AFP, and officials said the work would continue during Sunday.
The compound has been under the security forces' control since bin Laden was killed by US Navy SEALs.
Residents said they heard the noise of machines and thuds of debris throughout the night, and some perched on surrounding rooftops to watch the process.
"The provincial government had requested us to raze this compound as it might create problems for them in the future," the official said.
Copyright AFP (Agence France-Presse), 2012