An Immersive Conversation with Jim Anderson and Ulrike Schwarz

On the interplay between music and technology, discussions generally emphasize two areas. The first is the emergence of new instruments, from the electric guitar to the synthesizer to the turntable. The second couches itself in terms of formatting; how streaming overcame physical media or the arguable resurgence of vinyl. But technology’s influences go deeper. Another area shaped by new developments is the quality of audio recordings. Throughout most of recording history, a listener could hear sounds from only certain directions. Even with surround sound, sound is limited to two dimensions. It may come from sources in front of or behind the listener or to their left or right. But listeners do not experience live music in only two dimensions. In a live performance, sounds can bounce off the ceiling or floor of the concert hall; really, sound can come from infinite directions. Enter immersive or 3D audio. The abstract concept of immersive audio – sound coming from all directions to present a more realistic environment- is hardly new. Eighty years ago, Disney experimented with a precursor to the technology for some select screenings of Fantasia (Walt Disney Pictures, 1940). But in a more practical sense, its use for at-home listening is a more recent phenomenon. Over the last few years, an increasing number of audio engineers have incorporated immersive audio into their output. Two such engineers are the team of Jim Anderson and Ulrike Schwarz. 

One of Anderson and Schwarz’s more recent projects is Patricia Barber’s Clique (Impex, 2021). Their work on the pianist-singer’s latest album is among five nominees for the 2021 Grammy Award for Best Immersive Audio. Part of what gives their recordings a unique quality comes from the diversity of Anderson and Schwarz’s backgrounds. While the former has significant recognition in the jazz community, the latter is known primarily for her classical works. While Anderson is American, Schwarz brings a European sentimentality to their work. By combining their forces, the duo produces something out of the ordinary. Further adding to the uncommonness of their vision are the contributions of their collaborator Bob Ludwig, an engineer rooted in classical music but well known for his works with Nirvana, Metallica, Queen, and David Bowie.

We sat down with Anderson and Schwarz to discuss immersive audio and the benefits and detriments of technology. 

PostGenre: How did you both get into immersive audio, and what do you like most about it?

Ulrike Schwarz: I’ll go first. The national broadcasting agency, Bavarian Radio, that I was working for in Germany began broadcasting their weekly symphony orchestra concerts in 2001. I first became involved in these transmissions in 2005 and did them until 2015. With those broadcasts, we would mix the music live in stereo and 5.0 and simulcast in 5.0 from the same board. I did my first 9.1, or 3D, recording in 2010 with a Robert Schumann oratorio, “Paradise and the Peri,” conducted by Sir Simon Rattle.

The concept of immersive audio is great. With immersive, you can build the concert hall itself through sound, or you can construct a whole new scenery out of the orchestra. Immersive audio gives you more space to work with and is a lot more fun to do.

Jim Anderson: My entry into immersive audio isn’t as lofty as Schumann. It came primarily from two artists I was working with, Patricia Barber and Jane Ira Bloom. 

With Jane, we did an album called Chasing Paint (Arabesque, 2003) which was her interpretation of paintings by Jackson Pollock. Even back then, Jane was thinking about how 3D audio could bring the album to a new level. But there was no real distribution model at the time which would allow for much higher quality audio released commercially. While we agreed that releasing the recording in 3D audio was a great idea, it was not feasible. A few years earlier, Patricia and I were working on her album Modern Cool (Premonition, 1998) and went in and did some mixes. Again, no real way to release them, but we did experiment with them at least. Later, I did some work for a film, Chico & Rita (GKids, 2010), which we had mixed in 5.1 as well. By 2012, Bluray was just starting to come into fashion as a medium for 5.1 distribution and the emerging technology gave us the idea to remix Modern Cool and distribute it in 5.1 Bluray. That was the first commercially released recording I had done in 5.1 and it ended up winning a Grammy, so I am pretty proud of it. 

PG: Much of the ability to present music in immersive audio is seemingly tied to the technology available at the time. Simultaneously, technology also pushes in the opposite direction, specifically in how the rise of things like MP3s came at the cost of audio resolution. Does technology harm audio quality as much as it helps?

JA: That’s kind of a loaded question. The rise of digital platforms makes things more democratic in the sense that, now, more people will hear an album than before. 

I think ultimately there are two groups of listeners. The first is people who care greatly about audio quality. These people will always seek something beyond a standard MP3-ish kind of sound. The other group seems to care more about convenience and accessibility and is less concerned about recording quality. But, in some ways, such a division is not new. With recorded music, you always have a couple of different masters to serve. Going back, there were the people who loved cassettes, for example. The cassette is kind of like an MP3 in the sense that it is much easier to access but does not provide the best audio quality available. And, at the same time, you had people who loved vinyl – less convenient but better overall quality. 

While that conflict between quality and convenience exists, we have also found that the work and quality that we capture in such high-resolution eventually trickles down to other formats as well. So, ultimately, exploring higher resolution audio makes everything sound so much better regardless of the specific format type. 

US: I would also add that when the Bluray for Modern Cool was pressed, or when some of my broadcasts were released in SACD, the record labels always had to make an initial investment to release the recording in higher resolution. Because labels can only forecast a certain amount of sales on each release, they would often try to cut costs by just omitting the 5.1 audio and making the recording less immersive. They often determined that they could not justify the additional expense of an immersive recording based on the then projected sales. But with a system where the focus is more on downloads than physical media, often your other costs – for example, to manufacture a CD or its packaging – drop in the process. And that decreased expense often provides additional funds to produce and release higher resolution audio. The barrier to getting a good mix and selling it has come down.

We are on some platforms, such as NativeDSD.com, that have ultra HD and they are not cheap. But the amount of albums sold on those platforms is considerable. With Clique we not only SACD and sold more SACD’s than thehas for any other SACD release. We also pressed an MQA CD and the sales have been quite encouraging. As far as streaming services, it seems like they are making efforts to improve their audio quality as well. Clique is on Tidal, Qobuz, and Spotify in high resolution. So, for example, Apple Music has been working on rolling out Dolby Atmos, which is a higher resolution. We are not on the Apple Music platform in Atmos, yet, but have some clients who are interested in having their music there. If we put music on a platform like that, we would like to think it would be an opportunity for listeners to check out the music, but if listeners enjoy it, we would encourage them to buy it in higher resolution.

JA: We’re not trying to discourage people from going into a high-resolution format. Ultimately, if you don’t make music available to people in any resolution, they won’t buy it. 

US: If, ultimately, people are getting only low-resolution versions of music, regardless of whether bought or streamed, what is their incentive to buy it? But, on the other hand, giving access to a lower resolution version and then selling at a higher resolution gives people more choice in terms of whether they want to buy or stream the music. Ultimately, you have to embrace technology for what it is; it’s a part of the PR for an album. And it is cheaper than mailing out thousands of CDs to get the news out on an album, as you once did. 

JA: In the past, we had to send 500 to 800 discs of an album to radio stations. Those days are gone. At some point, radio stations decided they no longer wanted to deal with physical products. That is kind of a shame. But, at the same time, much better for us.

US: And people who want to buy vinyl are not necessarily impressed by streaming. But streaming can allow them to essentially audition the recording online then go out and see if they would like to buy the vinyl. It seems much better than going out and buying an album, unheard, that they may not even like. Streaming allows them to find out beforehand whether they would enjoy an album. There are so many different segments that you can reach with streaming that you couldn’t reach before, because of the flexibility that streaming provides.

PG: It is also interesting that Clique relies on new recording technology but the songs themselves are older standards. What are your thoughts on the interplay between technology and tradition?

JA: Well, the whole thing about Clique is that it was sort of like musical comfort food. The songs were all ones that we in the studio knew and loved and the band had coalesced around. I think that familiarity is what makes this record work. 

US: In terms of bringing stuff out of the archives and doing very good transfers of old recordings, technology is great. But the problem is who does it and what they do with it. Back when I was working in radio, the station’s whole archive was analog. In 2015, you could use a machine that would easily play a tape from back in 1946. That flexibility was great. You cannot do that in the same way today. With digital audio, you can present the music to a much wider audience, but so many factors shape the end product. You always strive to transfer music at the highest quality you can, but there are a lot of different directions you can follow. 

JA:  Also, sometimes, when you look for a tune on streaming services, you will see only the most recent mix.  But maybe you are not looking for the 2021 mix. Maybe you are looking for the original version from 1962. Sometimes those older mixes are buried, and you just can’t find them on streaming. That is a danger we have with recordings that are frequently remastered and the streaming platforms; it creates confusion as to what is the “original” version. I think the music industry needs to be a little more careful about how they release the different mixes on streaming services. It would be great to pull up all versions of an album. We tried looking for the Beatles’ Let it Be (Apple, 1970) recently and we kept getting only newer remasters. But maybe we wanted to listen to the Phil Spector version we couldn’t find, not the newer ones. And we can’t necessarily do that on streaming, not easily anyway. 

US:  And, going back to the analog and digital traditions, I can fairly easily play a tape from 1946 with analog. But with digital, I need to dig deeper to see who has a DAT machine or an F1 machine, or whatever it is. And that is more difficult to do than using analog. Because of this, it is often easier to go back fifty years than it is thirty years. There is something wrong with digital technology. [Laughing].

JA: And the other thing is that an analog tape will play but a digital one may or may not. 

US: Yes, it’s all or nothing at all. 

PG: You were both musicians before you went into sound engineering, Ulrike on piano and Jim on French Horn. How do you feel that background as a musician gives you a different insight into recording compared to someone who did not have that experience?

JA: The ability to read a score is also incredibly helpful. Once, I was recording with [pianist] Jim McNeeley conducting the Swiss Jazz Orchestra, a big band, in Zurich, Switzerland. We were recording our first piece and he said to me that something wasn’t working, so I suggested that we go back to a specific bar number and pick it up from there. He looked at me shocked and suddenly remembered that I knew how to read a score. Apparently, he had been working with a lot of engineers who did not know how to read music or a score. 

US: It’s also helpful to have the experience of being a recorded artist and seeing that side of the process. In Berlin, I was involved in some sessions where I was the one recorded. That type of experience helps you figure out how to treat artists on the stage, how to better handle the recording process, and how to work with talent in general. It is very unlikely any of us are going to tell more established artists something about the score that they don’t already know. But they do need you to keep track and record. On the other side, some artists aren’t quite at that level and may need some training in navigating the process. Being a musician is quite important for the whole process of recording. 

JA: And even with the more established artists, there are times they may ask for your opinion on the music. And when that happens, you just have to rise to the occasion. You can’t just be a wallflower sitting in the back and hitting the record button. 

US: Yes, you do need to have an opinion on the music, but best not to give it if it’s not wanted. Only give it when you are asked. [Laughing].

JA: Yeah, you can certainly get in trouble if you start offering too many opinions. It all depends on who is writing the check and what your role is. 

PG: Back to Clique, the album was mixed at Skywalker Sound. Jim, you have used Skywalker before. What do you like most about working there?

JA: Yes, we used Skywalker for a couple of reasons. First, we like going out there and working with the Skywalker team. But we also knew we were doing Clique in both stereo and 5.1. All of our previous work in 5.1 was done at Skywalker because we have found we can work well and fast there. We have never walked out of there with a mix that hasn’t translated well. We think it has one of the best control rooms that we work in.

PG: In addition to your working relationship, you are also married to one another. Do you feel like that personal relationship gives your professional one a different dynamic?

JA: I will let my wife answer that. 

US: [Laughing]. Smart man. [Laughing]. The dynamic may be different, in that you spend many more hours together but I think we can separate our personal and professional relationships quite well. 

JA: Ulrike has expertise in areas that I don’t. And there are other areas that I have expertise in that she does not. I think we complement each other well. Working with high-resolution material has been her area while I keep doing what I have always done, but am recording it better. 

US: I also think our collaboration brings out possibilities that would not come out if it were just one of us. For instance, in terms of microphone techniques, they may come a little more from the classical side, where I come from, but some have morphed into jazz versions, which is more aligned with Jim’s background. 

JA: I also think our existing relationship made it easier to navigate the pandemic. We did the orchestral score for the Oscar-nominated film Judas and Black Messiah (Warner Bros., 2021) during the pandemic. There was a point when I could not leave the control room. So, Ulrike was on the floor, placing the microphones and doing all of the things that I would normally do but couldn’t because we were in lockdown. The trust and understanding of one another that comes from our relationship give us flexibility compared to another team. 

PG: And, now you are both nominated for Best Immersive Audio Album, along with Bob Ludwig, who has a significant number of rock recordings to his credit.

US: Yes, though actually,  Bob is a classical trumpeter by training. He has a Master’s degree from Eastman [School of Music at Rochester University] in classical trumpet. Once he had us come to the studio to listen to fifty minutes of Stockhausen in quadraphonic. Sometimes, we discuss some very obscure scores or recordings with him. We’ve taken three classical symphonies to him for mastering. Between classical recordings and rock, he has almost two careers. Actually, he goes by different names for each type of album; Bob Ludwig for rock and Robert C. Ludwig for classical. 

JA: The first album we brought to Bob was a recording of Brahms’ Second Symphony. When we were putting together the credits for that album, I said to Ulrike that we should send a note to Bob and ask if he would prefer to be credited as Bob Ludwig or Robert C. Ludwig since all of his classical recordings had been under Robert C. He greatly appreciated that we understood that distinction. 

US: It is a lot of fun working with Robert C on classical things and also to see when he works on the jazz side of things. It is funny because, in the jazz world where transitions between tunes happen a lot faster, I very much defer to him in terms of timings and those types of things. It is an unparalleled experience to work with a mastering engineer of Bob’s quality.

Click here for a more in-depth conversation with Jim Anderson on a few other recordings in his discography, including those by Miles Davis, Ron Carter, the Lounge Lizards, and John Zorn.

More information on Anderson and Schwarz can be found on the Anderson Audio New York website.

Patricia Barber’s Clique can be purchased in various formats here.