The state government has changed planning controls so about 3000 apartments and buildings up to 24 storeys high can be built on a parcel of land in North Parramatta.

A collection of state heritage buildings are spread across the 30-hectare site, including the 1818 Female Factory which was the destination for all unassigned convict women sent to NSW.

The buildings are largely empty, dilapidated and inaccessible to the public. The government says funds from the residential development will go towards their restoration, which is expected to cost $100 million.

And while heritage advocates have rallied against the development since it was first proposed, the government appears to have the royal seal of approval.

Planning Minister Rob Stokes said on Friday that he had discussed the project with Prince Charles, during his recent visit to Sydney.

The Prince's Foundation for Building Community has since agreed to be involved in the design of buildings for the site.

"If we had not have done anything this site would have continued to deteriorate, it would have continued to decay and its future would have been put at risk," Mr Stokes said.

"Here we have an opportunity to re-imagine it, to get the very best results right in the heart of Parramatta, to ensure that this crucial heritage precinct is protected and sustained into the future."

The approved plans are a scaled back version from those originally proposed by the government's property development arm Urban Growth, which triggered an outcry by community groups and heritage advocates.

A petition with 10,000 signatories was handed to NSW Parliament in August, and the CFMEU has also launched a Green Ban on the site.

Minister Stokes said the total dwelling yield had been slashed 30 per cent and most of the buildings had been reduced in height, with the tallest moved further away from the core heritage buildings.

But opponents say the proposed development will ruin the site, which is currently being considered by the Commonwealth government for national heritage listing.

North Parramatta Residents Action Group president Suzette Meade said the group was seeking advice as to whether it could take legal action against the government.

"The NSW government has declared war on the people of Parramatta. It begins now. We were just warning up before this," Ms Meade said.

"I'm disappointed that the western suburbs has to sell their public land to fund heritage restoration when the eastern suburbs and northern suburbs just get government funding."

North Parramatta precinct<\/p>\n",colour:"green", title:"Title", maxWidth:200, open:0}] ); } if (!window.gmapsLoaders) window.gmapsLoaders = []; window.gmapsLoaders.push(CreateGMapgmap2015102014424); window.gmapsAutoload=true; /*]]>*/ ]]>

Mr Stokes said key archaeological investigations at the site would start immediately, followed by the repair and conservation of heritage buildings.

There has been no decision yet on how restored heritage buildings will be used.

The sale of the first sites is expected to take place in 2017, and although the land has been rezoned, development applications will still need approval from the NSW Heritage Office and Parramatta Council.