The passengers from Singapore were perturbed. The revelation that we were leaving Antarctica earlier than planned sent them into a huddle to discuss options, as though they had any.

I caught snippets of their conversation; the tone seemed to lie between conspiratorial and mutinous. Finally, one raised a hand and tried to catch the attention of the Aurora Expeditions’ staff member who was explaining why we had to leave the White Continent and head north-east to South Georgia.

Antarctica was the reason everyone on board had booked, a place that offers bragging rights like no other, but now bad weather was forcing us away. Were the Singaporeans being cheated? Conned? Their spokesperson struggled to make herself heard over a louder, more aggressive Australian group close to the front of the ship’s lecture theatre.

Seeing their distress, I leaned over and said as reassuringly as I could: “Don’t worry, South Georgia is better.” I’m not sure they believed me entirely, but I wasn’t simply trying to placate them — I happened to be telling the truth.

Named by Captain Cook in 1775, South Georgia and the South Sandwich Islands is a British Overseas Territory. There is no permanent human population, just two bases staffed by British scientists, conservationists and officials, fewer than 20 people in total. The government is based in Stanley, in the Falkland Islands, about 900 miles to the west. From there, or from Antarctica, South Georgia is around a three-day sail in a modern cruise ship. Owing to its inclement topography, it has no airstrip, and because of the extra distances and time at rough sea, it is not included on most Antarctic itineraries. This is good news for those who make the effort, and doubly so for anyone troubled by the volume of ships now cruising the southern seas.

I first visited in 2015 aboard Aurora Expedition’s plucky little ship the Polar Pioneer, then again at the start of 2020 on the vessel they had built to replace it, the Greg Mortimer. In many ways, the ships were irrelevant — I was there for the archipelago, its dizzying menagerie, and its improbable, magnificent history.

The Akademik Sergey Vavilov sails into Fortuna Bay, South Georgia

When I told the Singaporean ladies that it’s better than Antarctica, what I really meant was that it is more diverse — it has several things the continent proper does not, most notably the colour green. From afar, tussock grass gives the South Georgian foothills an alpine appearance and, along with 25 other native plants (plus many more introduced besides) this comparative lushness allows for species of birds to thrive here which would perish further south. Among them are the South Georgian pipit, the world’s southernmost songbird, and the South Georgian pintail, the world’s only meat-eating duck.

These endemic curiosities are of great interest to twitchers, but they represent just a handful of the 30 million or so birds that have made South Georgia home. They are ruled by the ridiculous regents of the south, the king penguins, which number around a half a million here. They may not be as tall as their emperor cousins in Antarctica, but they are more colourful, more numerous, and far more accessible.

Both times I’ve been to South Georgia I made it to one of their strongholds, Gold Harbour, a notoriously difficult landing for helmsmen, but one unanimously beloved by passengers. The long beach is stuffed with penguins and fur seals; behind them rise mountains wearing glaciers like heavy capes. Above, giant petrels and vicious southern skuas patrol the sky, looking for abandoned eggs, or young penguins, though the youngest seals aren’t entirely safe, either.

At the southern end of the beach, the king penguins are packed as tightly as a Tokyo train station at rush hour. The continual jostling of black and white and gold is occasionally interrupted by the hulking brown mass of adolescent elephant seals, rearing up to butt and barge each other, playfighting now before the gladiatorial battles of adulthood.

The area is famous for elephant seals, and its large number of penguins and fur seals

While I stood watching this bedlam last year, expedition staff member Isabelle Howells came up and grabbed the thick cuff of my polar jacket, her eyes glittering. Though she had worked several Arctic seasons as a whale expert, this was her first time in the south and she found it impossible to feign professional indifference. Days later, when we spoke in the ship’s bar, she found the words to describe what she had felt: “South Georgia was a much more intense experience than I had been ready for,” she said. “A few years ago, I watched some BBC documentaries, so knew that places like it existed, but I didn’t expect I’d ever realistically be there. Gold Harbour was the first time we had that real density of animals and I was running around taking photos . . . I had to pinch myself to remember: ‘Oh yeah I need to do some work, too.’”

It’s easy to be undone by South Georgia, by the ludicrous drama of Drygalski Fjord, at the head of which icebergs gather like a spectral fleet. Or by huge pods of fin whales, so numerous their spouts appear like fleeting forests, now returned to these rich waters in great numbers since the last of South Georgia’s whaling stations were abandoned in the 1960s.

In the early 20th century, South Georgia was the most important location for shore-based whaling in the southern hemisphere. Grytviken, the first station, was established in 1904 and only shut in 1965, when the whales had become so scarce it was unviable. Many of the old buildings still stand there, macabre remnants of an industry that saw more than 175,000 cetaceans butchered, an almost incalculable volume of gore.

Today, these grim facilities are hollow and silent, rusted the colour of ground cinnamon. Near the town limits the Norwegian Lutheran church is looked after a handful of enthusiasts from the South Georgia Heritage Trust, who come in summer to run a little museum in Grytviken and what must be one of the world’s most remote gift shops.

The museum covers the whaling industry as well as more recent history, with an understandably large section devoted to the Falklands war, which started here when Argentinians seized the old whaling station at Leith Harbour, then fought a two-day skirmish with the SAS in the bay around Grytviken.

If the South Georgia preamble to the Falklands war will come as a surprise to many, there is also a more famous history that continues to draw and enthral visitors.

Sir Ernest Shackleton died leading his third Antarctic mission in 1922, not in the frozen hellscapes of the continent, but here in Grytviken Bay, most likely of a heart attack, while on his way south. He is now buried in a small cemetery at the far end of the settlement, under a monolithic headstone with a simple dedication: Explorer.

There are other tales of incredible Antarctic survival — Sir Douglas Mawson’s excruciating month-long trek in 1913, completed alone and starving, and Swedish explorer-scientist Otto Nordenskjöld’s rescue a decade earlier. Yet above all the mighty figures of the “heroic age of Antarctic exploration”, Shackleton surely stands alone.

In 1915, after months stuck in pack ice and drifting, his expedition watched as their ship, Endurance, sank. They escaped, making it in lifeboats to Elephant Island, just north of the Antarctic peninsula, and in April 1916, Shackleton chose five men with whom to sail to South Georgia to try to raise the alarm.

Their 16-day journey saw freezing salt water flay the skin from their hands and their rank reindeer sleeping bags go rotten in the boat. They were sleep-deprived and, with minimal rations of fresh water, suffered from a thirst which, Shackleton later wrote, grew “quickly to a burning pain”. When they reached South Georgia, they were almost dashed on the rocks by a storm but, after more than 36 hours of toiling, they finally made it ashore. It was then they realised they had 20 miles of trekking to do in their raggedy, sodden clothes, over uncharted mountains and glaciers, to reach the Norwegian whaling station at Stromness.

With three of the men already near-dead with exhaustion, Shackleton chose Frank Worsley and Tom Crean to follow him across the island, an unimaginable test of their will to live. Miraculously they made it — arriving, in Worsley’s words, like “a terrible trio of scarecrows” — and began the complex rescue of the constituent parts of the expedition. All the crew of the Endurance survived. “We had pierced the veneer of outside things,” Shackleton wrote. “We had suffered, starved and triumphed, grovelled down yet grasped at glory, grown bigger in the bigness of the whole.”

Hunter S Thompson used to write out pages of F Scott Fitzgerald’s The Great Gatsby so he could “feel” what it was like to be a great writer. In some ways, coming to South Georgia is like that with Shackleton. It offers the chance to see land he saw, unchanged by development; to imagine just a little of what it was like to be an explorer of old. On my last trip, we followed in his footsteps over the final four miles of his epic trek to salvation.

The weather was kind, so we disembarked the Greg Mortimer in Fortuna Bay, where we tiptoed past seals then climbed steeply through tussock grass, before transitioning on to shale, then over a wide, bare pass. After an hour or two, the scenery opened up and we eventually reached a promontory overlooking a yawning U-shaped valley. At its head, we could see Stromness. Close to this spot on May 20, 1916, Shackleton and his men huddled together and listened closely for a whistle coming from the whaling station. When they heard it, they knew they were saved.

The drama for our group was most related to more cantankerous fur seals close to shore, many of which flopped angrily around the old whale oil drums like the last protesters of a long forgotten cause. When we boarded our Zodiacs, most passengers did so with thoughts of Shackleton, but also a sense of relief.

The following morning, there were more seal confrontations on the walk to Salisbury Plain, though as this was through a nursery they were mostly babies, full of vim and vigour but little menace. We were there to see one of the archipelago’s vast penguin colonies, where more than 100,000 breeding king penguins cover the hillsides like snowfall. I wondered where else such a glut of animal life might be visible — during Africa’s Great Migration, perhaps, if you timed it just right.

King penguins at Salisbury Plain; there are about a half a million of the species on South Georgia

As we made a final retreat to the ship, I fell in stride with Heidi Krajewsky, one of the guides. I’d met her on my previous trip and by chance we had ended up sailing together again. We brought up the rear as our group marched back to the ship for a final time. This was our last landing and, as well as the reek of elephant seals, melancholy carried on the air. Walking across the grey sand, we were challenged by two adolescent fur seals, awkward teenagers seized by the bravado of the adults, but not the chutzpah to commit to violence. Heidi addressed them with the weariness of a nanny: “Come on guys, we both know you aren’t going to do anything.”

The seals didn’t know what she was saying, but they understood her message and shuffled aside. I mentioned how sad I was to be leaving again and felt a slight creep of anxiety — what if this was the last time? “If you love it so much, why don’t you apply to become expedition staff?” asked Heidi, a veteran of 12 Antarctic seasons. I thought about it and realised I had no good answer, so when we got back to the Argentinian port of Ushuaia a few days later, that’s exactly what I did.

Jamie Lafferty is an award-winning travel writer based in Glasgow; his application to become an Antarctic guide has been delayed by the pandemic but is ongoing

Follow on Twitter to find out about our latest stories first