Gleniffer Brae should be a stand-alone function centre earning income for the cash-strapped Wollongong City Council, Lord Mayor Gordon Bradbery said.
Councillor Bradbery made controversial comments about the historic site on Sunday, saying the Wollongong Conservatorium of Music was tying up most of the manor house due to a "ridiculous" long-term lease agreement.
"Personally, I think the conservatorium shouldn't be in the manor house and it should be separated out as a function centre which is returning funds to the council," he said.
"At the moment, the conservatorium has got that place for cheap rent, it's an asset we have poured a lot of money in to maintain, it's a treasure of the city and all we've got out of it is the front two rooms."
The future of the historic house has been in limbo for the past 18 months, after councillors voted to reject a land rezoning proposal to allow the University of Wollongong to build a $40 million creative arts precinct at the site.
After a protracted debate during 2012, Cr Bradbery cast the deciding vote to reject the university's plan in January 2013.
However, he remains unhappy with this decision.
"I'm not happy with the way it all turned out, and it's going to be an impost on the people of Wollongong because we have failed to capitalise on that asset," he said.
Council staff are working with stakeholders on what to do with the historic home, with a report due in August.
However, Cr Bradbery said the options for its future were likely extremely limited because of the conservatorium's lease, which was put in place by previous councils.
Under the agreement, Cr Bradbery said, the conservatorium had the right to use about 60 per cent of Gleniffer Brae as well as the ageing former school classrooms at the site.
It is also able take out two five-year lease extensions, the first of which it is exercising now. Cr Bradbery said this meant the important Wollongong landmark would be tied up for use by the conservatorium for "about another seven or eight years".
"This is a ridiculous situation, that should never have been allowed to happen and, as a result, the city is missing out on the benefits of that building," he said.
He said the arrangement was made 15 to 20 years ago and the council was stuck with it.