And then there are all the indulgent things the 64-year-old is talking of doing in his retirement. Travel, read, learn how to cook properly – there are blue skies on the other side.

But for now, there's still plenty left to do. In the past month the gallery has opened a large and luminous new exhibition of works by Arthur Boyd, unveiled a major acquisition of works by the Australian artist Dale Frank, announced a daringly hip summer blockbuster – a retrospective by the US artist James Turrell – and is about to open a new gallery by the lake, of contemporary Australian art: a precursor to the next big building project somewhere on the horizon – a wing devoted entirely to Australian art.

"I'm just quite delighted that I'm going out on not only Australian art but contemporary Australian art by living Australian artists – I rather like that," he says.

Of course in the midst of all this has been the scandal surrounding of the NGA's purchase of an Indian artefact that looks, after all, to have been looted from a temple and sold to the gallery with false papers – a controversy that has been felt all the way to the Prime Minister's office.

And yet, on this light, spring morning, in his office high up in the gallery's admin wing, overlooking the lake, Ron Radford insists the scandal has not dented his pride in the gallery's triumphs.

"You've got to be positive," he says, as he reels off a list of gallery achievements in the past 12 months: the second largest attendance last year with three back-to-back blockbusters; the successful 100 Works for 100 Years campaign for Canberra's centenary that raised $21 million for acquisitions; and a growing international exhibition program including the Australia show at the Royal Academy in London – a project instigated by Radford himself. The national collection is scattered across the world, with Sidney Nolans showing in Dublin, Ballet Russes costumes in Tokyo, Polynesian deities in St Louis, and a soon-to-open Lichtenstein show in Taiwan. All, he says, achieved in the midst of dealing with the legal quagmire of the Dancing Shiva.

Damning international headlines have dogged the NGA's acquisition of the exquisite work which, until recently, had pride of place downstairs in the Asian gallery, one of the first things visitors saw as they entered the realm of the mystical and spiritual. It's now back in India, couriered by Prime Minister Tony Abbott himself in an act of the kind of soft diplomacy that the gallery has often been called on to provide, what with collections from Indonesia and elsewhere in the region.

For Radford, the public interest – he would call it media obsession – with the Shiva is just par for the course. The decisions of those in the top job in the top gallery have always been subject to an extraordinary level of scrutiny – especially compared with their state counterparts.

Radford can say this because he knows what it's like to be a director of a state gallery (the Art Gallery of South Australia, 1991-2004), and a director of a regional gallery (Ballarat Fine Art Gallery, 1973-1980). Going national has always been a different ball game.

"We were by far the richest gallery and that therefore that's always in for a dig – that's before all the efficiency dividends came in," he says. "But we wouldn't be the best-funded now per population, but we certainly were initially. We had to be, to start from scratch."

In fact, Radford never wanted the job in the first place. When previous director Brian Kennedy left the gallery in 2004 the council scrambled to find a replacement. Radford, warmly ensconced in Adelaide where he had been director of the Art Gallery of South Australia for 13 years, didn't apply. And when the council approached him directly, he still wasn't interested.

"The gallery wasn't at its best moment, but that wasn't why I didn't apply," he said, referring to the years-long saga involving the NGA's faulty air-conditioning, one that left a cloud over Kennedy.

"It was mainly that I was loving Adelaide so much. I'm from Victoria, but I loved Adelaide, I had a fabulous house there, I still own the house there, and I just loved being part of the community … Everything was perfect, my football club and so on."

(It's Port Adelaide, in case you're wondering – "and not doing so badly", he points out, giving one of his famous high hoots of laughter.)

"So coming to Canberra, which I didn't know as well and thought it would be too cold and too this and too that, I just didn't want to do it."

The NGA council, having advertised the job and interviewed several candidates, pursued Radford solidly for the next nine months. Radford said they made the offer more and more attractive, until he finally said yes.

He recalls he didn't even have a proper interview for the job, instead flying secretly to Melbourne airport, where he met council members in a room in the Chairman's Lounge. To this day, he says, he still walks past that room at the airport with a shudder.

"Amongst the many people who said I should go was Neil McGregor, who's now the director of the British Museum," Radford recalls. "He was doing that 'You need to do it for your country' and all that, and I'm thinking, that's a bit high-minded and cheesy."

"But others were saying to me, and that struck a note as well, that you're getting too comfortable in Adelaide. And I really was … It was just such a good lifestyle, but you need to be challenged, for god's sake, or you're going to just keep on doing what you're doing."

He knew the job would be difficult and it was, right from the start. When the announcement came through he was on an island in Bali, and when he returned he did just one day of interviews in Adelaide. And so began his famously shaky relationship with the media.

"I think the media knew that because of the bad media that the gallery was having all the time, I was putting a gap between it. It looked like the media was running the gallery and I made a conscious decision."

His ambivalence was justified, in a way, on his very first day on the job. Listening to the radio just before he set out for work, he heard himself mentioned on the ABC.

"Before I left, I heard the ABC news saying, 'Today is the first day of the new director Ron Radford at the National Gallery, and it is expected that, like in Adelaide, he would buy British art and de-accession the American collection'."

Radford is still astonished as he recounts this. While it's true that the Adelaide gallery had been notable for its extensive collection of British art and Old Masters, and Radford had published a weighty book on the subject, the bit about American art baffled him.

"What was interesting about all this is, what you do in one institution is not what you do in another institution, and people expect the same thing," he says.

"I got a bit of insight about why there's speculation in the media, and how destructive it is. It doesn't relate much to the truth, and the high profile of Canberra. So I was reinforced not to speak to the media and all these sorts of things."

Nearly 10 years later, there are many things he is proud of, not least the realisation of the massive Stage 1 building project, which involved a grand new entrance and a new wing devoted to indigenous Australian art.

It's no secret that the NGA is housed in a famously difficult building, designed in the Brutalist tradition by a famously difficult architect, Colin Madigan, who wanted in when it came to making amendments. It was a fraught process begun long before Radford came on the scene, and involved 37 separate consultations with Madigan, the last seven of which involved Radford. Still, Madigan claimed repeatedly that he had not been consulted about the new additions to the building, leading to questions in Senate Estimates. But four years on from the grand opening, Radford is sanguine about the stoush.

"I didn't see this initially of course, but he was never, ever going to agree to any extensions, ever. That was a cross I had to bear right at the beginning," he says.

But he's relieved Madigan had approved of the process of restoring the integrity of the building which had, in the intervening years, been modified to cope with its difficult nature.

"At some stage, we'd clad all the building in white plaster, and it was far better backing for works of art, but it turned the building from a bit of a concrete bunker into sort of a cardboard box. So the collection looked better and the building looked worse," he says.

"The trick was to restore the building to its integrity but not compromise the collection, and that was done with lighting, it was done with different shades of greys, it was done with different textures on the wall, it was done in a way that didn't fight with the architecture and didn't fight with the gallery."

Of course, he says, in an ideal world, a major art collection shouldn't be fighting with a relatively new building at all.

"To be fair, and I wish Colin would say this, he had to design it when we didn't really have a collection … Some architects would say that's a dream, that's fantastic, you can do an abstract building – and this is what we want to do with Stage 2, very much design the building around the art, not try and fit Australian art into a new building. It means that the art comes first and it dictates the architecture."

There is more than a hint of glee in Radford's voice that Stage 2 won't be his problem anymore. Nor will fronting up to Senate Estimates twice a year, or, most of all, fundraising. It has often been said of Radford that he is particularly good at persuading rich people to part with their money. But he is relieved to be giving it up.

"All my career I've fundraised, with good training at Ballarat – big collection, poor gallery. I've done it all my life. It never becomes easier," he says.

"You say, oh well, people give to the National Gallery – it's not true. Fundraising is hard work, it's not pleasant work. You get lots of knockbacks, as well as successes. I've had successes but lots of knockbacks. It is never-ending, it's relentless, and I believe every modern gallery director or any museum director has had to put so much time and energy into it. It never ceases. So I will be quite grateful when I retire that I won't be fundraising again."

Still, despite his looming and longed-for retirement – he had only ever planned to stay five years, but has extended his contract repeatedly over the years – he says it has been hard to think of the future when there is so much to do. The gallery is acquiring works, building on existing collections, and moving around exhibits constantly, and he will miss the satisfaction of seeing the right art honoured in the right places. The gallery's Australian art collection is now the largest and most balanced in the country – three times bigger than Melbourne, which has the second largest. And it will, sometime in the near future, have a proper home rather than the poky attic status it was afforded when the building first opened.

"We're the national collection, we should be showing indigenous art and non-indigenous Australian art in the greatest depth in the national capital," he says.

"Only we can do it, with our collection, and the fact that we don't at the moment is not very good. Having Australian art in the attic is really an act of cultural cringe. Because the national galleries in Washington, in Canada, in the National Gallery in London, their national art is prominently display and we should do the same thing."

Another thing that is often noted about Radford is his devotion to Asian art, but in fact, he says, he has only ever done what the gallery set out to achieve in its founding document, the 1966 Lindsay report. This has involved adding to the South Asia and South-East Asia collections and acquiring more works from the Pacific.

In a 43-year career, he has overseen around 47,000 acquisitions,

staged some 500 exhibitions, and has brought back Betty Churcher's beloved blockbusters after they were scaled back in Kennedy's time for reasons of cost.

"They keep the institutions lively, they complement the collections, they keep people coming to see them – I've been proud of our blockbuster shows," he said.

The standout here, of course, is the 2010 Masterpieces from Paris show, which knocked all Australian records for an art show off the perch. Incredibly, he actually had some hard work convincing the board to agree to staging the show. In the end, with nearly half a million visitors, it was the first blockbuster ever to turn a profit.

At this point in the interview, with all this talk of Paris and the Musee d'Orsay, talk inevitably wafts back to the treats that lie ahead. Where will he go when he leaves Canberra? Asia, perhaps?

"That would be nearest and warmest, but also I love Italy, I love France, so it might be a little bit there, but also Southeast Asia. And to live – not just tour but actually live in a foreign culture," he says.

"In other words, savouring the food, savouring the people will be so much fun. I'm getting excited already."

The fact is, he says, to finish up, it's been a hard slog.

"I have to say, I have actually worked extremely hard in 43 years, and people often say you've got the best job in Canberra, the best job in Ballarat or the best job in Adelaide, but it's not the best job," he says.

"Betty was seven years, Brian was seven years, James Mollison was much, much longer, but it's long enough, and I just want to do something else, or do nothing, probably."