Can I have the key to the castle please?'' Without batting an eyelid, the owner of Cafe Ilios reaches under the counter and produces a large, medieval-looking key. I place it carefully in my pocket and our group embarks on an excitingly illicit Famous Five-style adventure.
After following a winding alleyway up to the foot of a rocky outcrop, we climb a steep flight of whitewashed stairs that leads to a small wooden door. The rusted lock clicks open and after passing through a narrow passageway, we emerge into the remnants of a 13th-century Venetian castle.
Behind us are arid, featureless hills whose barren lower slopes have been partially tamed by stone terracing. In front, a jumble of flat roofs and church domes eventually disappears into the endless blue of the Aegean Sea. Aside from the five of us and a scattering of goats on the hillsides, there's not another soul in sight.
This time yesterday we were battling through the crowded streets of Santorini, the blue-domed poster child of the Greek Cyclades. Today, we're less than 100 kilometres away on the archipelago's easternmost atoll and the contrast couldn't be starker.
Amorgos is the Greece I'd hoped to find, with its bleached white houses, azure wooden shutters and tumbling blooms of magenta bougainvillea. Of course, Santorini and Mykonos also have these things, but in summer you'll feel like you're sharing them with every other tourist on the planet.
As we stroll through Chora, Amorgos' photogenic medieval capital (population 300), we encounter just a trickle of other tourists. The town's meandering thoroughfare delivers us to a succession of idyllic cafe-lined squares where patrons sip homemade lemonade under the cool shade of eucalyptus trees.
These elements alone would make Amorgos a captivating destination but the island is also home to one of the Cyclades' most impressive medieval churches. Improbably wedged into a vertiginous cliff 300 metres above the sea, the eight-storey, 11th-century Hozoviotissa monastery was once home to 100 monks. Now just three reside here, but if you make it to the top of the steep, faith-testing approach path, they'll offer you a restorative glass of psimeni raki (a homemade liquor infused with honey and cloves) and allow you inside the small wooden chapel to see the silver-plated icon of the Virgin Mary, to which the church is dedicated.
Amorgos has managed to retain its sleepy appeal despite having a starring role in Luc Besson's 1988 free-diving classic The Big Blue. Beach-lovers will appreciate its sandy coves lapped by gin-clear water while hikers can romp through its rugged interior on well-marked trails.
Our visit is part of an eight-day, island-hopping Peregrine cruise around the Cyclades. There are just 35 of us on the 48-metre-long Galileo and it's a compelling advert for the benefits of small-ship travel. Amorgos doesn't have the infrastructure to support the larger ships that frequent Santorini and Mykonos and there's understandable reluctance to introduce it.
For lunch I choose Estiatorio Birzolaki, a small taverna away from the main square and order patatato, a local speciality of tender, slow-cooked goat in a rich, garlic-infused tomato stew. ''Good isn't it?'' says the waiter, bringing me more bread to mop up the sauce. ''The recipe is a family secret - the cook learnt it from her grandmother.''
- Rob McFarland was a guest of Peregrine