It should have been a sure-fire hit. In 1939, a jazz-infused musical of Shakespeare’s glorious A Midsummer Night’s Dream opened on Broadway, packed with popular jazz tunes and star performers — Count Basie, Benny Goodman, Maxine Sullivan and even the great Louis Armstrong among them — and set in New Orleans. It launched the mellow “Darn that Dream”, which was later covered by artists such as Billie Holiday, Miles Davis and Petula Clark.

And yet. Swingin’ the Dream ran for just 13 days. It lost its investors an eye-watering $100,000 (nearly $2m in today’s money) and then vanished, so comprehensively that even the artistic director of the Royal Shakespeare Company, Gregory Doran, had never heard of it. To say he was startled when he discovered its existence would be an understatement.

“I came across this reference: Midsummer Night’s Dream, 1939, Broadway, with Louis Armstrong as Bottom,” says Doran. “And I went, ‘What? I never heard of that!’”

Doran was hooked and began digging to find out more. “We couldn’t find any recordings, any script anywhere,” he says. “There was no record of it.”

Eventually, they turned up three pages: the comical Pyramus and Thisbe section of the play, annotated down the side with the names of jazz standards such as “Ain’t Misbehavin’” and “Melancholy Baby”. That discovery, he says, was “gold dust”: “They did Pyramus and Thisbe as a jazz opera medley. And that made me realise we could reconstruct it.”

The RSC has joined forces with New York’s Theatre for a New Audience and London’s Young Vic as co-producers, with a view to staging a new version. That project is very much in its infancy, but this Saturday you can see a taster concert of the music, performed by eight actors and a jazz band at the Royal Shakespeare Theatre in Stratford-upon-Avon and presented by Kwame Kwei-Armah, artistic director of the Young Vic. The hour-long concert will tell the story of this ambitious musical, share some of that foot-tapping music and be streamed live, globally, as a work in progress, says Doran.

Exactly why the original failed to catch on remains a mystery. The show seemed, as Doran says, “to have everything going for it”. It featured scenery based on Walt Disney cartoons, signed up Agnes de Mille as choreographer and boasted a raft of great African American performers. And it came in the wake of several productions that successfully updated classical work or fielded integrated casts.

“The Gershwins had had a huge hit with Porgy and Bess in 1935,” says Doran. “A year later, up in Harlem, Orson Welles had produced his all-black Macbeth which was a great box office sensation. Then in 1938 Rodgers and Hart did The Boys from Syracuse, based on The Comedy of Errors: it ran for 235 performances and was a huge hit.”

Swingin’ the Dream also featured an integrated cast at a moment of potential change and hope: Benny Goodman had recently conducted his groundbreaking concert at Carnegie Hall. So what went wrong? Was it ahead of its time? Or was something else awry?

“I read some of the reviews, which basically said: ‘Too much Shakespeare not enough jitterbug’,” says Doran. “And there were 100 people in the cast — maybe it just didn’t have enough rehearsal. It may have been the wrong theatre: [the Center Theatre] was a huge theatre that was later turned into an ice rink. It could have been that somehow the thing just didn’t gel properly. It could have been all of those things.”

There’s also the uneasy fact that the show’s creators, Erik Charell and Gilbert Seldes, were white and that white actors played the governing class while the parts of the fairies and “rude mechanicals” were taken by black performers. Might an African American co-author have found a deeper, more eloquent fusion between jazz and Shakespeare? The new project will have an African American writer and be directed by Kwei-Armah. It is likely, too, to weave in that historical context and to address the contemporary resonances of the issues raised, such as representation and cultural ownership.

“It feels like it has found its moment,” says Doran. “It is speaking to who has the right to tell this story? Whose story is it? How do we tell it? Who is Shakespeare for? All those things. We can’t just do a period piece: it has to be about now.”

Quite what final form the show might take has yet to be decided and alongside the streaming of the concert (available on demand for seven days) will be an invitation to audiences to offer their opinions. Doran is relaxed about the possibility of any negative feedback.

“What’s important is that it’s a conversation,” he says. “And what I don’t want to lose is the joy of that music. When I listen to it, it just lifts my heart. I’m hoping it will lift everybody’s spirits.”

Like all theatre companies, the RSC has been severely impacted by the pandemic. It hasn’t yet been able to welcome audiences into its theatres, has had to cancel several tours, postpone a West End transfer and is currently assessing plans for a spring 2021 season. The impact on Stratford-upon-Avon, a town synonymous worldwide with Shakespeare, has been bleak, says Doran: “Every £1 spent in the theatre is £3 in the hotels, restaurants and bars in the rest of Stratford. So it’s affected the whole ecology of the place. It’s a tragedy.”

The company has been active, however, performing outdoors during the summer, working with partner companies and supporting schools and young people throughout. Costume staff made PPE. Saturday’s concert is one of a range of online offerings and the RSC is a lead partner in the government-funded Audience of the Future programme, exploring how live performance can use emerging technologies.

Doran suggests that one glimmer of optimism during the pandemic has been the take-up of new forms and ideas and the forging of new links with communities: “We’ve been able to redefine what our relationship with our audience is and how we think of ourselves as a multi-platform company.”

The experience has also shed fresh light for the director on the works of Shakespeare himself and the way plague impacted on the playwright's thinking (he explores this in a BBC Radio 4 programme on January 17). “I think it changed the course of his writing,” he says. “I had never fully realised it before. I can see how from the sunny uplands of As You Like It in 1599 you slide down through Hamlet, Troilus and Cressida and Measure for Measure and then his attitude plunges. It seems to just fall into the abyss of Macbeth and King Lear and those tragedies.”

But it’s also important to note, he adds, that Shakespeare then moved on to his final great works of hope and reconciliation and that a key moment in Lear is the scene where Edgar encourages his blind father to keep going.

“It’s a bit like the tramps in Waiting for Godot,” says Doran. “The fact that they continue to wait validates their existence. It may be bleak, but somehow the fact that Shakespeare is articulating that experience is hopeful in itself.”

Keeping going, trying to lift our spirits: that’s something all too familiar now. For those in theatre, the going looks extremely tough for the next few months. The RSC, says Doran, is looking at “recalibration not restoration”.

“We won’t get back to where we were. But the need for the arts — the need for connection, congregation, communion — is testament to how much we need and want them back. However we get there, I’m sure we’ll get there.”

‘Swingin' the Dream’, January 9, 7pm UK time, available on demand for seven days, rsc.org.uk