Workers and commuters heard multiple explosions as the old Gold Fields House building went up in flames at the corner of Pitt Street and Alfred Street shortly before 9am on Tuesday.

Police are investigating the blaze, which was ignited by a powerful blow torch and sparked the evacuation of several nearby office buildings and Circular Quay train station.

Emergency services sent 16 fire crews and more than 60 fire fighters to battle the fire after reports of a building fire about 8.50am, a Fire and Rescue NSW spokeswoman said.

Fire and Rescue NSW Superintendent Andrew Ticehurst said numerous gas cylinders exploded during the blaze, and fire crews removed other cylinders affected by the heat to prevent further explosions.

"There were some significant dangers to the firefighters with the gas cylinders exploding as well as it being a building that was under demolition," he said.

"Sometimes you can get some collapse of scaffolding, thankfully today the fire fighters were on scene very quickly ... which prevented any significant risk of that scaffolding collapsing."

The first crews at the scene reported outside sheeting and scaffolding at the site were alight, and the site had been evacuated.

Superintendent Ticehurst said the fire created "significant smoke", but most travelled upwards and was low-toxicity.

"We believe the fire started on one of the lower levels of the building, then spread predominantly on the outside through that screening that was around the outside of the building," he said.

He said the screening used was a plastic-type product that could be "quite flammable".

Police remained at the site late Tuesday afternoon and said in a statement: "It is not known if there are any suspicious circumstances; inquiries are continuing."

About 24 workers were on site at the time and NSW Ambulance Inspector Giles Buchanan said paramedics treated 13 workers at the site for potential respiratory problems. No-one was taken to hospital.

"We were quite fortunate that none were affected by any explosions or structural collapse."

The FRNSW spokesman said there were reports of a forklift on the site causing small explosions, but there was no indication that was the cause of the fire.

CFMEU NSW assistant secretary Rob Kera said use of the mesh screening on the outside of the building was standard practice but he would ask an engineer to inspect the material.

"It is a concern. [The fire] accelerated pretty expeditiously so our question would be why it spread so quickly."

SafeWork NSW and engineers were expected to visit the site to examine scaffolding and a tower crane that could have sustained fire damage.

Mr Kera said the union had not been aware of any particular dangers at the site.

He said several members had been treated for smoke inhalation.

"Our primary focus this morning in attending was to make sure our workers on site were receiving the best possible treatment by the first responders."

Circular Quay train station was evacuated as sirens sounded and commuters were told to leave the station as quickly as possible due to "an emergency situation".

Commuters were told to avoid the Alfred Street side of the station and any smoke.

Train services to Circular Quay were temporarily halted and buses were were diverting away from Pitt Street. Ferries were also stopped temporarily but public transport services later started again.

Nearby office buildings were evacuated and workers spilled out onto Loftus Street and gathered outside Customs House.

Ali Bennett, who works at a nearby coffee shop which was evacuated shortly after 9am, said she noticed flames coming from the top of the building.

"It was a pretty big fire. We were working and one of our customers came by and said it exploded twice."

Her colleague Seif Ajam said he heard two explosions.

"I was just worried there might be someone inside the site."

One woman, who worked Gateway building in Macquarie Place, saw flames coming from the mesh.

"One of our office managers got off the train at 8.30 and there was nothing.

"I got in at 8.45 and there was smoke and fire.

"I got upstairs and the guys said there were four or five fire trucks there, so it was quick."

When her colleague walked out of the building at 9am it had been "pitch black".

Hugh Fraser was walking up from Circular Quay when he saw the fire start, and said the blaze appeared to be in the webbing that surrounded the scaffolding.

The fire was at the site of the old Gold Fields House building.

The 1960s highrise was bought for a reported $425 million by the Chinese Wanda Group in 2014, and is being demolished to make way for planned $2 billion mixed-use development One Circular Quay that includes a 59-storey apartment building.

Demolition works commenced in late March 2017 by Delta, and are due to finish next month. Construction of One Circular Quay is then scheduled to commence and is due to be completed in late 2021.

In January, Wanda announced it had sold some of its Australian assets, including the Circular Quay property and another apartment project on the Gold Coast.

It emerged the 23-year-old son of controversial political donor Huang Xiangmo, Jimmy Huang, was the mystery buyer, snapping up the properties once valued at $1 billion each for a bargain $312 million in cash and $807.1 million in debt repayment.

The purchaser was AWH Investment Group, which is half-owned by Jimmy Huang. The private company is also an associate company of Yuhu Group Australia.

A spokesman for Yuhu Group said the property is still being managed by Wanda.

"The Circular Quay site [remains] under the management of Wanda as the transfer process continues, but as a developer we expect the highest standards of workplace safety."

With Kirsty Needham and Carolyn Cummins.