Builders uncovered the item during renovations on the house in Atchison Street, Crows Nest.

It was a smoke grenade from the 1970s, a type of ordinance used in the Vietnam War, a police spokeswoman said.

She said it was not known how the device got there but such discoveries were quite common when old properties were being dug up or knocked down around Sydney.

The military were called in and deemed the device to be live so it was taken away for safe disposal.

"They also did a search around the area to see if there were any others," the spokeswoman said.

An exclusion zone surrounding the property was removed at 2.40pm on Friday.