Arcus Pluvius: A Conversation with Steve Lehman on Big Bands, Artificial Intelligence, and ‘Ex Machina’

Artificial Intelligence (AI) is going to kill us all. Or, at least, that is what some theorize. Many commentators portray the emergence of AI as inevitably leading to some technological dystopia where a master computer controls everyone and everything. In the words of Neil deGrasse Tyson, now is the “Time to behave, so when Artificial Intelligence becomes our overlord, we’ve reduced the reasons for it to exterminate us all.” Or, as Elon Musk put it, “With artificial intelligence, we are summoning the demon.” Steve Lehman’s Ex Machina (Pi, 2023) forces listeners to look beyond the histrionics of some Max Headroom-ian fever dream and see that the reality of AI is far more nuanced.

Ex Machina finds the saxophonist furthering his explorations of spectral harmony – composition based on the acoustic properties of sound – in conjunction with the acclaimed French large ensemble Orchestre National de Jazz (ONJ) under the direction of Frédéric Maurin. Expansion of Lehman’s ideas to a setting beyond even his octet would, with nothing further, already be a worthwhile listen. But what truly sets Ex Machina apart is its incorporation of artificial intelligence in the form of live electronics developed at the renowned IRCAM (Institut de Coordination Acoustique Musique). These electronics interact and, to some extent, think for itself how best to create music. 

To Lehman, such artificial intelligence is like any other tool for which the primary distinguishing line between good and evil is the intent of its user. In our conversation, Lehman likens AI to fire. Perhaps an even better analog, however, can be found in dynamite. Although many used Nobel’s invention as a tool of war, its use also saved the lives of miners otherwise confined to applying unstable compounds. Fittingly, like the subterranean laborers, Ex Machina bores new tunnels by explosively blasting through perceptively tough walls – perceived divisions between human and machine expression and the distinction between improvisation and precomposition. 

Even as alien as things sound at times, there is something distinctly familiar – human – in the music. The presence of a large ensemble itself draws out even the computer’s human traits. For almost a century, the big band form has moved people emotionally, whether Benny Goodman instilling fervent dance or Gil Evans invoking deep reflection. The ONJ is thoroughly modern but nevertheless built upon that lineage, and its ancestral DNA continues in its music. On “Ode to akLaff”, named after the legendary drummer, electronic murmurs and whirs evoke some extraterrestrial exaltation. When the ONJ joins with rich chords evoking a quizzical scene, it inexplicably adds a sense of normalness to the unusual surroundings until live band and virtual imagination become nearly indistinguishable.

Although spectral music is generally outside the musical mainstream, its concepts also add humanity to Ex Machina. A computer can analyze sound spectra better than a human, but it cannot truly hear the shimmering effects produced by music using those inputs. Where machines can be more technically precise, they cannot perceive the emotional aspects inherent in sound. Hence, some of the album’s finest moments are when the electronics seemingly learn in real-time. As the computer programs interact more closely with human musicians, the gap between natural and artificial closes even further, with each seeking a middle ground. One can find a superb example in “Chimera”, where Chris Dingman’s vibes find a distorted non-human parallel before the two converge. 

There is something inherently futuristic about the highly listenable Ex Machina, even as the work reflects contemporary technological reality. Unusual sonorities present an exotic tomorrow just within arm’s reach. But the recording is not experimental for the mere sake of being different. Instead, Lehman, the ONJ, and IRCAM are presenting new methods to express the continued human condition. More than a mere highlight among this year’s recorded output, one cannot help but sense that Ex Machina is setting a new standard for electro-acoustic improvisation. We sat down with Lehman to discuss the fascinating project. 

PG: How did you start working with the Orchestre National de Jazz (ONJ)?

SL: Right. ONJ is a state-funded ensemble that has a long history at this point. The French government founded ONJ in the ’80s under then-French Cultural Minister Jack Lang. The ONJ’s Director changes every five years or so, which is cool because it means the group’s stylistic orientation and aesthetics evolve. Frédéric Maurin reached out to me after he took over as Director in 2018. We met in Paris briefly, and he asked if I would be open to writing a piece for the group and collaborating with him on a program of music. We had to pause for a while because of the pandemic and other things, but we ultimately did a bunch of concerts in Europe and the US and then went into the studio to record earlier this year.

PG: Do you have a sense as to why Maurin sought you out?

SL: Fred and I have a lot of overlap in our areas of interest, particularly as composers. He had been aware of my work. While Fred’s not particularly well known in the States, he has a long history – about twenty years- of being a very admired leader of a big band, or jazz orchestra, in France called Ping Machine. I think his significant experience in the trenches with large jazz ensembles made him the logical choice to serve as Director of the ONJ. 

PG: What interested you in working with the ONJ instead of augmenting your octet with additional musicians?

SL: That’s a great question. To be perfectly honest, I probably would have hand-picked additional musicians to add to the octet had the opportunity with the ONJ not come up. But it also takes a massive amount of financial resources to put together a large ensemble. I honestly don’t think I would ever have had the opportunity to do a project with this many musicians had Fred not reached out and proposed it to me. I wouldn’t even be sure how to begin to get it off the ground.  And, even if I had, it probably would have been under much less favorable conditions. Many of the people in the group – to my surprise and delight – were familiar with my music, very excited about the project, and well-equipped to execute it at an incredibly high level.

PG: You also brought in Jonathan Finlayson and Chris Dingman, both of whom you have worked with for many years, including in your octet.

SL: Yes, because this was a collaborative project, we brought in people Fred wanted to be involved, as well as people I wanted. That’s what led to our ultimate configuration: a mix of long-time collaborators of mine and his. 

PG: The music is built around spectral harmony. Was it more difficult making spectral music for a larger group compared to a smaller one, as you have done in the past?

SL: In terms of my work with spectral harmony, I think it is easier in some ways with a larger group. It’s a little easier to realize spectral harmonies that are very detailed and are a composite of several components coming together to create a specific color. Having more instrumental voices can make things a little more straightforward. I think writing improvised music that prioritizes and foregrounds improvisation with a bunch of musicians that, for the most part, I hadn’t met before was a new challenge. I’m used to existing collaborative relationships being a really important of my compositional process. But we had a lot of rehearsal time, and by the time the piece came together, I was pretty familiar with a lot of the key soloists. Things were certainly more efficient because I had more voices – more colors – to work with. Plus all the electronics.

PG: If you go back eighty or ninety years, big bands were at the core of popular culture. Their music was essentially the pop music of that era. Today you are working with a similar format to explore different ways of composing and the most recent technological developments. Is there something special about the large ensemble form itself that has carried through even as technology keeps changing?

SL: I think there are two sides to it. There is something about the instrumentation of a big band or large jazz ensemble that is timeless. The idea of a reed section of about five woodwinds, a brass section, and a rhythm section produces a magical, enduring combination of colors. People keep coming back to that form and finding new resources within it.

I was recently listening to Darcy James’ Argue’s new album, which is brilliant. It has a totally different aesthetic orientation, more traditional in some ways, than my work with the ONJ. But he still uses incredible writing and a personal approach to that instrumentation. Jason Moran also has a beautiful- albeit smaller with maybe ten or eleven people – group with his James Reese Europe Project that amplifies who he is and what he does as a pianist and a composer.

But you also cannot ignore the economic realities of operating a large ensemble. As you mentioned, right before World War II, those bands made the most popular music in the country. At that time, their music was more oriented around dance music. As we know, during World War II and post-war, the economics entailed in keeping a big band functioning exploded everything and began to define the history of big bands. To me, it’s hard for me to separate those economic factors from the history of the idiom.

PG: By the 1950s, even [Duke] Ellington purportedly played half-empty skating rinks.

SL: Yeah. If even the king of the form had to make those kinds of compromises, that tells you where things were financially. Obviously, big bands have continued since then, and I have done other things with them. I did a little work with Oliver Lake’s big band. And, of course, with [Anthony] Braxton. Also, with Frank Lacy and a little rehearsing with Andrew Hill’s big band. Even back in school, I played Frank Foster arrangements and stuff like that, though Thad Jones’ work is the beginning of where I get really excited about this stuff. But it’s not an area of my expertise as it is someone like Darcy or John Hollenbeck. 

PG: In terms of incorporating the electronics into the album, was it difficult adding them to the group in a way that made sense? 

SL: I’ve been doing stuff at IRCAM (Institut de Recherche et Coordination Acoustique/Musique) with interactive electronics for a long time now, since at least 2011. So, the challenge of incorporating electronics is something with which I’m familiar. With this project, the electronics were another layer that made it challenging to write, in the sense that we did not just want them to serve as some decorative layer on top of the music. It was important to us that the electronics be central to the piece. And, of course, that approach has its own structural implications. We had to consider how the integration of electronics would make us play differently or make the music sound different than if the electronics were solely an afterthought or as a subtle embellishment. Integrating the electronics fully was a big part of the puzzle to figure out how to best utilize the resources of this world-famous electronic music center. 

PG: Did you encounter problems getting the musicians to work well with the electronics?

SL: No. We worked to ensure the two were integrated well, but everyone was, thankfully, super open-minded and very hip to the concept. All the soloists were asked to interact with electronics or have the electronics intelligently respond to them. Everybody was very enthusiastic and had the perfect mindset, which was lovely.

 PG: Could you provide more color on how the electronics work? 

SL: It’s difficult to describe distinctly how the electronics operate because they function differently on each piece. 

On a piece like “Alchimie”, I created all the harmonies as I wrote the piece. We then created a database of electronic sounds that I thought would mesh well with the acoustic part. Then, we developed software to analyze and sketch what I had written for the acoustic instruments. The software proposed sounds from the database that connected well with the rest of the music. It found connections to the acoustic sounds that I wouldn’t normally find myself, but that are super interesting and hip. 

As a result, on the finished project, we intentionally left things unclear as to what sounds are created by electronic means and which are created by acoustic means. The acoustic and electronic parts are integrated and synthesized in a way where you end up with an otherworldly sonority. Minimizing opportunities to parse out what’s electronic and what’s acoustic underscores that the distinction isn’t that important. What matters is whether the result is something that resonates with you, moves you, interests you, or that you love.

PG: You suggested that other tracks on the album use AI in ways different from this database approach. 

SL: Right. Other pieces use artificial intelligence as an improvisor. So, for example, “Chimera” starts with Chris’ vibraphone solo, but it interacts with a distorted alien-sounding vibraphone sound that artificial intelligence is creating in response to him. And ultimately, Chris and the computer keep responding to one another.

To do so, we use a collection of sounds that I created. We then developed a program to respond intelligently to what Chris was playing. It considers what notes Chris played, how much space he left, and how loud or soft he is played. The computer then determined how it wants to imitate or contrast what he did. There is a very rich collection of things that go into musical interaction, and we tried to approach it in a way that’s interesting to listen to. And all of that is happening in real-time, in the moment. The fact the computer is improvising in that setting is what sets it apart from the process I described with “Alchimie.”

PG: Although the electronics sounds fully automated, the album materials suggest that one or two people may have been operating them. 

SL: Jérôme Nika is credited with generative electronics creation and artistic collaboration. It was his job to help us develop the programs. The other name is Dionysios Papanikolaou. His role was similar to someone doing live sound; he monitored the program and ensured it was not doing anything stupid; not making any boneheaded decisions. Many times, he just made sure the program was loud enough. If the program behaved in a way that it shouldn’t, he adjusted back to where it should be. He was able to change what was prioritized in terms of how the computer listened.

PG: So there’s still a human artistic element in the electronic component, whether from the samples you created for your databases or someone monitoring and adjusting things. 

SL: Yes, that’s fair to say. I think that’s 100% right.

I think that way of working with artificial intelligence and computer music is something I inherited from George Lewis. He was the person who first really opened my eyes to the fact that you can program a computer, create a patch, or create a way of working with electronics or a version of artificial intelligence that reflects and represents your own ideas, priorities, and musical aesthetics.

Even beyond that, sort of to your point, it’s almost impossible to create a computer program for music or anything else that doesn’t demonstrate some bias in terms of your own ideas. In this case, those ideas are on the purpose of music and how it’s supposed to function. I think that element – being informed or inspired by human conditions- is always a part of AI. The cool thing is that if you are aware of that element, you can take advantage of it.

PG: Do you think the human element will continue to always be part of AI or is it just a function of current technology with continued advancements in machine learning possibly reducing or eliminating the human element?

SL: Well, I don’t know if I’m smart enough to imagine a scenario without the human element. Right now, there are models of AI that draw from preexisting collections of data, whether musical information, audio files, books, or screenplays, and make intelligent decisions based on them. I think, in the best-case scenario, AI comes up with ideas and solutions that are new and surprising to us as humans. That should be a good thing. The scenario where a computer makes music not based on any preexisting data or input from a human is hard for me to imagine, but I suppose it’s possible. I don’t know, but the idea is certainly interesting.

My wife is a filmmaker, writer, and director. Many people in Los Angeles are up in arms about the possibility of a computer taking all the best moments from all the best scripts and creating its own scripts. If it could do that, AI would largely get rid of screenwriting as a career. I can understand the concern, but I also think that in that scenario it is AI doing the same thing writers already do but at a much faster pace. You have to look at things from a larger perspective and focus on what humans bring to the table that’s unique to us. I think there’s no shortage of answers to that [inquiry].

PG: One could say something similar about music.

SL: Of course. And that’s another thing I learned from George Lewis: when working with computers, you should abandon the tendency to think in terms of a man versus machine binary. Instead, look at working with computers and artificial intelligence as a way to share what makes us human and what’s special about who we are as humans. This program is meant to be an example of that. And this book of music hopefully highlights, as you said, those human contributions. Making the electronics blend so well into the ensemble writing and soloing, which we’re very proud of, requires you to think deeply about how human musicians interact, what we like to listen to, and what works.

PG: Lewis also worked with IRCAM on his project, Rainbow Family. 

SL: That’s right. He was an artist in residence at IRCAM in 1984 and did that piece as part of it. I believe Rainbow Family was the last time that IRCAM commissioned a composition centered around improvisation. IRCAM has had other people put in research projects related to improvisation, but I think Rainbow People was the first time they commissioned a piece of music that focused on improvisation. I’m very aware of that history. At one point, I even wanted to give [Ex Machina] a name that referenced that history and used the word “rainbow” in Latin or something else to reference George’s legacy.

PG: Do you see Ex Machina as a continuation of Lewis’ work from almost 40 years ago?

SL: George is so brilliant, man. I can’t even begin. I’m definitely standing on his shoulders. This piece is possible because of his work and the trails he blazed forty years ago and continues to blaze today. I owe him a lot. I’m very inspired by his work with computer music. Maybe that’s another way of saying I’m continuing his tradition, but he looms so large that I hesitate to put myself in the same breath as him in that way. 

PG: In terms of stepping away from the man versus machine narrative, was it difficult for you to move beyond the media hype of AI as solely some dangerous force that will wipe out humanity instead of another tool that humans can use for self-expression?

SL: It’s not difficult for me because, in terms of creativity, I’m always looking at new technologies as new tools I can use. I don’t have anxiety about what I do in terms of more widespread implications.

AI, in general, can be a concern. You’d have to be pretty naive not to think about the potential to use these incredibly powerful resources for some nefarious purposes, which is already happening. The social media universe is a good example of people purposely spreading disinformation. But ultimately, AI is a resource. It’s an incredibly powerful tool that some people may compare to fire, something that can be used for great good but can be very damaging in the wrong hands.

But I’ve long been in a place where I’ve been comfortable looking at new technologies, and I put artificial intelligence into that larger umbrella of new technologies as a resource tool. It’s opened up possibilities for me.

PG: Do you see yourself continuing to work with these types of electronics on future projects?  

SL: Definitely. The main researcher I worked with, Jérôme Nika, and I have worked together since 2011. We’re already 12 years deep at this point. Before Ex Machina, those were all projects with smaller ensembles. We did a duo piece called “Silver Lake Studies” at a conference that IRCAM put together in Greece. We’ll definitely keep doing stuff. We’re talking about continuing our duo and looking for ways to integrate this artificial intelligence into more work with Sélébéyone. 

PG: Integrating this AI into Sélébéyone could be a pretty fascinating project. 

SL: Yeah, we could do some amazing things. A program could analyze an a Capella track a rapper lays down and, then, instantly draw from a library of audio files and have it line up with the vocal accents. It could find the rapper’s inflection and have it echoed in samples. Certain aspects of that can happen instantaneously now, which is exciting. I’m looking forward to seeing where we go with this technology. 

‘Ex Machina’ will be released on September 15, 2023 on Pi Recordings. It can be purchased on Bandcamp. More information about Steve Lehman can be found on his website.