This week James Packer came a step closer to realising his dream of "creating and developing Sydney's first six star luxury Hotel Resort with iconic stature and landmark status" at Barangaroo, on the western harbourside edge of the CBD.

After months of delays and behind-the-scenes arm wrestling, the Department of Planning published its recommendation that Crown's hotel, casino and apartment complex should go ahead.

At the same time it released the report of the Design Assessment Panel, a three-person panel of eminent architects, who were appointed by the Premier, to provide independent advice on the architectural merit, particularly on the impact on the public domain of Barangaroo.

The panel was scathing about several aspects of the project, the main being the size and bulk of the six-storey, 38-metre-high giant podium, which tilts outwards from the site over public space and houses the gaming floors.

"It's just a giant leagues club," said one close watcher of the project. "The tower's elegant, and to give Packer due credit, he's been prepared to spend more than is commercially sensible."

The design panel also had concerns about the way in which the 275m Crown tower relates to the podium, the way the restaurants and cafes located in the podium impinged on the public walkway, and the way public space around the building had effectively been defined as private and inaccessible.

Also of concern were the proposed giant neon Crown logos planned for the top of the tower, and the entry foyer.

Mr Packer has publicly blamed the government for delays but the release of documents this week reveals a major argument over the Department of Planning's draft assessment, known as B3, which attempted to take up the concerns of the design panel.

It suggested that the podium should be reduced in size by 12 metres at its northern end, so that a person standing in Hickson road and the park located behind the Crown tower would have views to the water. It also wanted eight metres shaved off the southern end of the podium so that the walkway – only 12 metres wide at this point – would be more generous.

These suggestions drew a furious response from the Packer camp, which threatened to abandon the project.

"Reducing the southern portion of the podium by 20 metres will result in a loss of approximately 5500 square metres GFA [gross floorspace area], or 14.4 per cent of the total podium GFA," Crown told the government.

"However, of greater significance ... is the specific impact that removing a 20-metre segment of the southern half of the podium would have on the gaming floors," Crown wrote in its response to the design panel.

"In total, the 40 per cent reduction in the area of the gaming facilities will result in a 48 per cent reduction in the number of gaming tables at Levels 1 and 2," Crown said, adding that it could not be readily put elsewhere.

"The reduction in capacity for gaming will adversely impact the ability of Crown to fund and operate key tourism assets, amenities and social initiatives such as meeting/convention space; world class food and beverage; public art and social programs. More importantly, Crown Sydney Property Pty Ltd will be unable to meet its existing obligations with the NSW government."

In other words, the whole project was in doubt. For the Baird government, which prides itself on being "open for business" such an outcome would have been a political disaster.

In theory, there is no political input into the planning department's assessment of projects. It is done on the merits. But government policy is a legitimate consideration.

The ministerial diaries reveal that Crown executives met the Premier, Mike Baird, within days of receiving the design report in November. The diaries beyond December have not yet been released, but sources say there has been an intensive lobbying effort by Crown to convince the government and the planning department to drop the demand for a smaller podium.

This week the planning department released its final recommendation and dropped its demand for a smaller podium, in favour of other changes that were proposed by Crown.

Whether these are sufficient remains to be seen. The next – and last – step in the approvals process is for a three-member Planning Assessment Commission panel to weigh up the project. It will have before it the competing views of the design panel and the department, but is bound by neither.

It will also hold a public meeting in late April and take submissions from Crown, Lend Lease, the City of Sydney and the public.

First the good news on the changes proposed by Crown and accepted by the department.

Crown has adopted the design panel's recommendation that there be an observation deck atop the tower, which after Centrepoint, will be the second-tallest structure in Sydney. It will afford stunning views over the western harbour.

"The observation deck at the top of Crown Sydney will be an iconic experience for Sydneysiders and an additional tourism asset to attract interstate and international tourists," a Crown spokesman told the Herald.

But when it comes to the size of the podium, Crown has proposed other solutions, rather than shrinking it. These have been accepted by Planning, even though some of these arguably come at a further cost to the public and government coffers.

First, Crown has agreed to place indents in the podium facade so it now appears as three pieces on its western waterfront side, instead of a wall of glass and concrete. The southern tip, which was to be clad in stone, will now be glass, to give it "a lighter feel". The department agreed that the changes were sufficient to address concerns about size and gave it "a more human scale".

The issue of the narrow walkway at the southern end has been solved by reshaping Waterman's Cove, one of two small bays that punctuate the foreshore. (A spokesman said the actual design of the Cove has not yet been finalised.) The walkway around the base of the Crown tower will now be 18.5 metres wide – as wide as its narrowest point at South Barangaroo, in front of the white building known as the Cloud.

The issue of the views of the water from Hickson Road and the park have been resolved by shaving a slice off one of the buildings at Central Barangaroo, known as Block 5, still owned by the government, via the Barangaroo Delivery Authority.

This will have implications for the value of the site, and possibly heights of buildings, which might get higher to maintain the return to the government. There are already plans for more floorspace at Central Barangaroo to pay for the railway station and some observers fear the result will be a loss of views from the Rocks to the harbour.

"The BDA supports the proposed amendments to the Barangaroo Concept Plan as part of Mod 8, including those affecting Block 5, on the basis that they are consistent with the BDA's master plan for Barangaroo", a spokesman said.

The second major issue was public space around the podium. Cafes and restaurants in the casino will have nine metres of the 30-metre public walkway on the western side for outdoor seating in licensed premises.

The design panel had argued strongly that these seating areas should be discernable as public space and be accessible 24 hours a day. Several public submitters also argued that the use of public space for restaurant areas amounted to a further privatisation of the site, as drawings showed the areas fenced with planter boxes, screens and glass panels.

The planning department appears to have sided with Crown. It has said it considers Crown's designs "acceptable" because the canopy and other protection over the restaurant areas will be transparent and lightweight, the elevated terraces for seating will have numerous entry points and the soft landscaping (planter boxes) adding to the broader landscaping.

Planning also sided with Crown over the relationship of the podium and the two towers that make up the entire structure.

The design excellence panel has said it did not believe the three elements were sufficiently coherent, and that it lacks "the simplicity, clarity and proportional slenderness promised by the concept sketches".

The department disagreed and said it is "satisfied that the three building components are appropriately related and the overall building is appropriately proportioned."

Crown also had a win over its plans for seven-metre-high neon Crown logos atop the tower. The design panel said it was inappropriate given the iconic status of the building. The planning department sided with Crown, who argued that other companies, including Westfield and HSBC, had been permitted to display their logos in neon on the Sydney skyline.

The three-person Planning Assessment Commission, headed by former public service commissioner Lynelle Briggs, will have the final say.

The design assessment panel said in its report: "Any assessment of built form and public domain must place at its centre the notion of public benefit – that for a development to be viable it must provide a net improvement in the lives of the people it effects.

"Barangaroo will impact Sydney and its identity in numerous ways. Its legacy will be enduring and it is imperative that the abiding public benefit arising for the project is embodied in the built form and public domain".

Crown and the Department of Planning will argue that Crown has done enough to meet the design panel's concerns about the public benefit.

If history is any guide, the PAC has generally followed the recommendations of the department. But there have been exceptions. For example, the department backed the controversial Drayton South coal mine in the Hunter, in 2014, but the PAC ruled against it.

But by the time the PAC came to deal with the mine, a formidable array of forces were marshalling against the project, including a sagging coal price, the Darley stud, owned by the Dubai royal family, and Alan Jones, the Sydney broadcaster.

This time the influential forces will be arraigned Packer's way.

A Crown spokesman said the company has worked with its architects to ensure the planning department's conditions are addressed. "Our updated designs, including significant changes to the hotel podium, will help ensure Crown Sydney is embraced by the local community and visitors as one of the world's best hotels and architectural icons."

The April meeting of the PAC promises to be lively.