On a sunny spring afternoon in the Musée des Beaux Arts in Calais, Jane Birkin is talking about the great love of her youth, the great French singer-songwriter Serge Gainsbourg. Photos of the happy couple are strewn around the walls: Jane and Serge kissing in the back seat of a car on a rainy day in Paris; Jane and Serge beaming into the camera on a lazy summer’s day, after a long, languid lunch. Jane looks beautiful, an English rose, the epitome of a carefree era when anything seemed possible. Serge looks incroyable, like a gargoyle or a goblin. No wonder the French press called him l’homme à tête de chou (the man with the head of a cabbage).

From 1968 to 1980, Birkin and Gainsbourg were Europe’s most glamorous creative couple. They could scarcely have been less alike. She was young and lovely, health and innocence personified. He was seedy and dishevelled, yet he oozed Gallic élan. They appeared together in movies and fashion shoots, but the thing that made them famous was the song they sang together, “Je T’Aime… Moi Non Plus” (I love you… neither do I).

Gasinbourg wrote “Je T’Aime” for Brigitte Bardot, with whom he had a brief romance in 1967. In 1969, he released the song with his new love, Jane Birkin. It was a Succès de Scandale, on account of Birkin’s contribution, in which she simulated the sounds of female arousal (20 years before When Harry Met Sally – quelle horreur!). If you’ve never heard it, it sounds awful – but, remarkably, it was a masterpiece. A conversation between two passionate, nihilistic lovers, it contained poetic pearls like “You are the wave – I am the island”, and Birkin’s heavy breathing, of course.

“Je T’Aime” was condemned by the Vatican and banned from the radio in Britain and several Continental countries. France was a bit more enlightened. The song was played on the radio, but only after 11pm, and you had to be over 21 to buy the record. Even then, it was sold in a plain wrapper, like a porn mag or a blue movie. Naturally, this furore was wonderful exposure. A broadcast ban is fantastic for sales, giving a hit single a rebellious cachet that no amount of advertising can achieve. All these years later, “Je T’Aime” still stands proud in the pantheon of brilliant records banned by the BBC, from “God Save the Queen” by the Sex Pistols to “Relax” by Frankie Goes to Hollywood.  

The record reached number one in Britain, and all across the Continent. Serge told Jane that the Pope was their greatest PR man. “He loved it!” she says, remembering Gainsbourg’s reaction to the controversy. He had a terrific appetite for mischief, and he knew no publicity was bad publicity.

That was nearly 50 years ago, but the memories of that hedonistic era still burn bright, and now Birkin is revisiting that heady époque in a new show called Jane & Serge here in Calais. It’s an exhibition of photos of the couple by her brother, Andrew Birkin, taken from the late Sixties to the late Seventies. Andrew is an eminent filmmaker (his biggest hit was his haunting film of Ian McEwan’s novel, The Cement Garden) and he has a director’s eye for drama. Andrew has always been close to Jane and he became close to Gainsbourg too. Consequently, these photos are incredibly intimate. Each frame preserves a particular mood, a unique atmosphere. They not only capture the essence of their relationship but the spirit of their age.

Andrew is here too, for the opening of the exhibition, and he takes time out to show me and a few other hacks around. “He was a provocateur,” he says, of Serge. “He was always on.” This isn’t a criticism, merely a statement of fact. Twenty-seven years since Gainsbourg died, of a heart attack, aged just 62, Andrew’s deep affection for the legendary chanteur endures. “That streak of melancholy was one of his most attractive facets.”

Half a century later, Jane Birkin is still striking – the same fine features half hidden beneath a cascade of tousled brown hair. The laugh lines around her almond eyes merely add to her allure, and her rich voice is full of joie de vivre. Although she left Gainsbourg in 1980, on account of his escalating alcoholism, they remained friends thereafter, united by the child they had together, the actress and singer Charlotte Gainsbourg, and she speaks of him with fondness, and admiration.

“Andrew was around on all our holidays – and all the rather bad films I did!” she recalls, looking at the photos of her younger self that surround her, reducing her to a pale replica of the image on the wall. “It’s a bit weird. You feel as if you’re dead.” Everyone laughs, but it’s not so funny. Her youthful fame has eclipsed her, leaving her public persona stranded in a time that’s long since passed.

She still spends most of her time in France, and today she seems more French than English. Watching her being interviewed by French reporters, speaking fluent French, she seems more natural, more at ease. It looks like an admirable life, from a distance, but she’s had her share of sadness. Kate Barry, her eldest daughter, from her first marriage to the composer John Barry, died in Paris in 2013 after falling from a fourth-floor apartment.

Jane left Serge for the French director Jacques Doillon, with whom she had another daughter, the actress and singer-songwriter Lou Doillon. She still sings, and last year she released an album of songs written for her by Serge – he wrote some of his finest songs about her after they’d parted. “Which, for me, are the most beautiful,” she says. “Everything on Baby Alone in Babylone, which was perhaps the best record I ever made.”

That was two years after they’d split, and she’d already given birth to Lou, but the aura of their romance lived on in Serge’s songs. “I liked the ones after I left more, perhaps, because they were really about what he was feeling, and I felt it was deeper.” It felt emotional, singing them in front of him after a parting which caused them both such pain.

What made him so significant as an artiste, I ask her. “He really rediscovered the French language,” she tells me. “That’s why I think every writer these days is bound to mention his name.” She likens him to Picasso, passing through so many styles. “He never repeated himself.”

Like a true poet, he loved ambiguity. “There were always two meanings, and sometimes a third.” Yet he far preferred to simply call himself a songwriter. “He didn’t want to say he was a poet. He thought that was for Baudelaire and Apollinaire. The people he really appreciated were classical musicians.” He said that songwriting was a minor occupation. She found such modesty refreshing.

Time’s up. Jane has another appointment. A French TV crew are waiting. I return to Andrew and his photos. “The problem with photography is that it starts to overwrite your memories of what you didn’t photograph,” he says. I think I know what he means. The more you look at these photos of his sister, the more unreal the real person becomes.

However Andrew has a very clear memory of his final meeting with Serge Gainsbourg, about six months before he died. “I bought him a 1910 bottle of Absinthe from Sotheby’s.” Andrew went to see him in his home in Paris, and they watched a film together. “He was asleep within five minutes of putting on the video, so I got up, crept out of the black house and into the black street and that was the last time I saw him.”

In the end, it was the same old, sad old story. “The main problem with Serge is just that he became an alcoholic, and had no particular interest in stopping himself.” So what was the connection between his sister and the chanteur? I ask him. “Sex,” he says, and everyone laughs, but he then stops and thinks about it. “It was the attraction of opposites, in a way.”

Jane & Serge is at the Musée des Beaux Arts in Calais (calais.fr) until 4 November