How to record remotely and stream a concert online

If you’ve ever attempted to sing ‘Happy Birthday’ with a group of friends on a Zoom or FaceTime call, you’ll know that the results can be awkwardly, if hilariously, off-kilter. The presence of latency, or lag, makes any synchronisation next to impossible, and most video-conferencing platforms allow only one person to speak or sing at once. For those who make music for a living, however, a number of online tools and ultra-fast networks are being harnessed to try and address a basic challenge: is it truly possible to hold a real-time, online performance with musicians all in separate locations? Can latency be held to an acceptable level and audio quality improved?

• Online orchestras and musical activities to take part in during the second lockdown
• Music lessons and workshops for kids during lockdown

‘Before lockdown, this had been an esoteric practice,’ says Chris Chafe, director of Stanford University’s Center for Computer Research in Music and Acoustics. ‘Music is fun to play together in the same room. Why would you want to play it in different places? Well, here we are.’

Chafe is the creator of JackTrip, an online collaboration platform that launched in 2004 as a ‘modestly sleepy open-source project’ geared for ‘a lot of jamming and teaching.’ It uses uncompressed audio and a ‘hub-and-spoke’ setup of local network servers in an effort to reduce the latency bottlenecks between users. When the coronavirus lockdowns took effect in March, the platform was refreshed and used by groups of up to a dozen musicians performing original works and improvising together. Chafe is seeking to host a large choir or orchestra in the near future.

‘We learned that this hub model is very, very well suited to regional ensembles getting connected, particularly in chamber music and jazz combos,’ he said. ‘It may be particularly interesting to the schools who are going to reopen their music departments in the autumn.’

JackTrip is one of at least half-a-dozen free or low-cost apps that aim to connect remote musicians, along with Jammr, NINJAM, Doozzoo, Soundjack and Jamulus. Most have all the visual appeal of a Wikipedia page, but together have seen a surge in interest. Downloads of Jamulus, created by the California-based developer Sourceforge, increased more than 1,000 per cent over the same period last year, with more than 12,000 copies per week accessed in late March and early April.

How the Concordia Quartet made music remotely

In April, the homebound members of the Concordia Quartet, an ensemble in Singapore, downloaded Jamulus in an effort to rehearse together online. They found it was nothing like jumping on a simple Zoom call. A delay between the players’ audio feeds was so palpable that synchronisation proved impossible and the sound was marred by feedback. ‘Our first violinist was playing one note faster than what he was supposed in order to stay together with everyone else,’ said Kim Kyu Ri, the quartet’s second violinist. ‘We couldn’t work on anything regarding balance or intonation because the audio was so poor.’

But the quartet’s gear-savvy manager Mervin Beng stepped in with software adjustments, four sets of professional headphones, USB audio interfaces and ethernet cables (Wi-Fi is less stable than plugging in). ‘As a young group you just cannot afford not to keep rehearsing,’ he said. After various patches and tweaks, the ensemble began to coalesce, the once yawning latency reduced to a manageable 35 milliseconds. That’s the equivalent of spacing musicians 10 metres apart. In June the Concordia Quartet integrated their Jamulus setup with a Zoom session and broadcast a live recital over YouTube.

JamKazam

In the thick of global lockdowns, interest peaked in other remote collaboration platforms. JamKazam, founded in Austin, Texas in 2014, saw the number of sessions on its service jump from 5,000 a month to 235,000 in early April. JamKazam’s chief selling-point is video integration, a difference from most other tools, including Jamulus, which are audio only. JamKazam claims to reduce latency by streamlining audio processing, meaning the way one’s instrument or voice is turned into a digital signal and sent to collaborators and vice versa.

Musicians’ latency data is visible to other members on JamKazam, so it’s possible to gauge whether someone will be a drag on your session. Some performers favour loose arrangements and slower tempos to help mask synchronisation problems, but at best, sessions can have the precision of Barton Strings, a string quartet that performed Pachelbel’s Canon from separate homes in Austin. In fact, physical proximity was part of the group’s key to reducing lag.

‘If you are at 20 milliseconds or less of total latency, you can have a super-tight, amazing session,’ says JamKazam co-founder David Wilson. ‘Thirty milliseconds or less is good, and you can play pretty much anything there. If you get in the 30 to 40 range, it’s kind of acceptable but you are feeling it more. Forty to 50 and you are at the edge of what most music will tolerate.’

Internet2

The latency obsession is not exclusive to music. Microsoft and Google are building data centers around the globe to serve the speed demands of cloud gaming, in which users play a library of on-demand video games streamed to their devices. Sports betting, which requires fans to make split-second wagers during games, is also counting on faster networks, notably 5G.

For classical musicians, another frontier involves Internet2, a network that connects more than 480 universities and organisations in the US, and Géant, a consortium that links 40 national and research networks in Europe. Established in 1996, Internet2 can transmit huge quantities of data, enabling multi-city ‘networked performances’ and masterclasses. In a simple but effective example posted on YouTube in 2014, the Chiara String Quartet, at the University of Nebraska-Lincoln, performed Mendelssohn’s Octet with the Avalon Quartet, 450 miles away at Northern Illinois University in. The latency was a mere 19 milliseconds.

‘As long as you are inside a 750-800 mile range, you get the sort of delay that happens inside a room, inside a performance hall,’ said Howard Herring, president and CEO of the New World Symphony (NWS), the Miami-based training academy that uses Internet2 for remote coaching sessions and link-ups with composers. In May 2018, I watched NWS artistic director Michael Tilson Thomas lead a rehearsal of Stravinsky’s Firebird with the Atlanta Symphony Youth Orchestra. Apart from a momentary video drop-out, the responses to Tilson Thomas showed little or no evidence of lag and the audio came through with notable clarity.

For the past seven years the NWS has collaborated with two other Internet2 adopters – the Manhattan School of Music and the Cleveland Institute of Music – on a Global Audition Training Programme. Every six months, students are evaluated by the faculty of other schools in a ‘mock audition’ setting (other participants include London’s Royal College of Music, the Royal Danish Academy of Music and the Shanghai Conservatory).

Global collaboration

Some of the more inventive – and public facing – uses of research networks involve cross-disciplinary art. In 2015 the Royal College of Music produced The Infinite Bridge, a music theatre piece featuring live link-ups between dancers at Barcelona’s MACBA contemporary arts museum, folk musicians in Helsinki, a horn player in Copenhagen and dancers, actors and orchestra in London. Dancers from Barcelona and London appeared to be sharing a stage along with other video projections.

As international travel loses its ease and allure, cross-cultural networked performances like this may gain new appeal. Musicians and producers who currently spend hours editing pre-recorded ‘mosaic’ videos of Ravel’s Bolero could devote that time to live, synchronous performances from multiple locations. That in turn may bring some new-found spontaneity to our screens.

‘We’ve learned a lot during lockdown – about our daily work and the need to communicate,’ said Chafe. ‘I’ve always had this adage that whatever can go online will go online. But this has been drastically experienced, for better or worse. That’s changing a lot of things in music.’

This article appeared in the September 2020 issue of BBC Music Magazine. Words by Brian Wise.

• Will we run out of new music?
• Which came first, music or language?