The 20-time grand slam champion is fighting the cancellation of his visa ahead of the Australian Open.

The Serbian superstar had claimed he had an exemption against vaccination allowing him to enter Australia, but it appears it was only granted for the tournament and not entry into the country.

He was taken to the Park Hotel in Carlton on Thursday - lodgings better known for housing detainees flown from Manus Island and Nauru.

Outside the hotel on Friday, about 100 protesters unfurled a large banner featuring his portrait, and waved Serbian flags.

Online material promoting the rally called for the renewal of the No.1's visa, an end to vaccine passports and what was described as medical "segregation".

Djokovic is known to have contracted COVID-19 in the past, and has spoken out publicly against vaccination.

Victoria's Acting Premier Jacinta Allan again distanced her government from the controversy surrounding the star player.

She said the Victorian medical panel which provided Djokovic with the exemption to play in the Australian Open was an entirely separate process to the exemption on which his visa was apparently based.

"How players and support staff enter the country to participate in the Australian Open ... is a matter for the commonwealth government and Tennis Australia," Ms Allan said on Friday.

"The role of the (Victorian) panel was to assess the medical documentation, not to assess their eligibility for a visa."

Meanwhile, refugee advocates have condemned Djokovic, and called for 36 refugees detained in the same hotel to be released, saying the tennis star is a risk to their health.

"The fact that all eyes are on the Park Hotel because a wealthy tennis player is in there is outrageous," activist Omar Hassan said.

"The whole world should have their eyes on this hotel prison in the middle of Melbourne's CBD because of the refugees who are indefinitely locked up and tortured in there."

Australian Associated Press