As authorities on Thursday faced mounting pressure to explain how Man Haron Monis, a man with a long criminal history and links to extremism, fell off watch lists and had access to a gun, the families slowly made their way around the memorial site and the enormous floral tribute.

Family members of Ms Dawson were visibly upset as they read cards and embraced each other at the shrine on Thursday morning.

Ms Dawson, 38, a barrister and mother-of-three, along with Lindt cafe manager Johnson, 34, were pronounced dead after the dramatic 16-hour stand-off that came to an end in the early hours on Tuesday.

The Dawson family visited Martin Place as members of the public continued to lay bunches of flowers and leave messages of remembrance.

On Wednesday afternoon, the Johnson family shook the hands of strangers during their emotional visit.

"This love comes for free," said Ken Johnson, Tori's father.

"We want to thank everybody for supporting us through this terrible time."

NSW Premier Mike Baird says a permanent memorial will be erected in Martin Place.

"This is something that will be with us forever. We need to recognise that and pay tribute to the inspiring Australians that we have lost, and what they have given us," he said.

Standing in front of the sea of flowers, Mr Baird said the city and state was coming together like never before.

And as the public outpouring of grief continued, Prime Minister Tony Abbott said the urgent and wide-ranging review would also examine how Monis was granted asylum, citizenship and welfare benefits.

"And we do need to know why he seems to have fallen off our security agencies' watch list, back in about 2009," he said.

Monis was on bail over a raft of violent charges, including being an accessory to the murder of his ex-wife, when he took 17 people hostage inside a popular CBD cafe on Monday morning.

The 50-year-old had been on an ASIO watch list in 2008 when he sent offensive letters to the families of dead Australian soldiers.

"It was extraordinary he was on our streets," Mr Abbott said.

Meanwhile, Federal police admit it gave inaccurate advice to Mr Abbott about Monis holding a gun licence.

During a press conference on Wednesday, Mr Abbott questioned how Monis, who died with two of his hostages in a dramatic end to the 16-hour siege, could have obtained a gun licence. NSW Police was quick to contradict that statement, saying it had no record of Monis ever holding a licence.

Mr Abbott later clarified his information was based on a briefing provided by authorities, ordering the Australian Federal Police to double-check its advice.

The AFP on Thursday admitted Monis was not a registered firearms licensee. AAP