m-onz
being an audio visual artist
15.09.24
I'm working on a new AV set and I wanted to share this replicate model: minecraft-textures-sdxl that can be used to easily generate images to texture 3D objects in GEM. I also want to explain why I chose to use pure data and program my own system instead of buying off the shelf software.
Programming forces you to be an architect, designer, debugger etc and it's different to being a consumer of a product. A consumer of a product expects support or some kind of interaction with a company. As a programmer you are on your own and you have to make your own decisions. Making decisions is probably the single most difficult thing to do. This is why it's always attractive to outsource decision making by relying on a company. The problem is that in the new medium of art and technology the system is essentially the art. So if your system is in fact a hodge podge of commercial tools its not cool.
I would say that being an "audio visual" or AV artists is difficult for many reasons. Being a visual artist exclusively is also difficult too. The most annoying aspect of it is that no-one really knows what a VJ or AV person is. Most people think that AV is installing projectors in conference rooms. There are no famous AV or VJ people in the world wierdly only DJ'ing is understood. Music is easier to wrap your head around and the concept of a DJ is well understood.
Its not just difficult to make meaningful work as an AV/VJ (whatever) artist that cuts through the noise. It's difficult to sell to people who don't really have a frame of reference. In my experience creating visuals and especially audio visuals is extremely difficult. It takes much longer to create visuals and audio. Making them together compounds the challenges. You will face technological choices, cultural and artistic choices. The extent of the complexity and rabbit holes really explains why there are no famous AV artists save DJ Yoda or that guy who did projection mapping when that was a thing: Amon Tobin?
I feel like I'm on the brink of actually being an audio visual artist and I'll explain how I do it. I'm saying this is the only way to do it but I'm just sharing my perspective. I develop and exclusively use just one tool that is open source and can do audio visuals. I love pure data and I've invested a lot of time learning it. If I had invested time in commercial tools like max/msp I would be at the mercy of any decision those companies make. I personally find max/msp unusable, to me its like using a submarine or space ship control panel on acid. Inititially mesmerising but then annoying when you try to get actual work done when the rug gets pulled from under you.
Pure data is open source, has a steeper learning curve but honestly it has everything I need. Its a real time system that I can patch in real time. Its open source and stable, maintained by a global community of nice people. It doesn't change and have loads of commercial features etc. Choosing the right tool is an incredibly important decision in my opinion. Having tried lots of approaches to AV and visuals all the commercial ones fail because they essentially shiny toys that look cool but don't have the substance we need.
As an audio visual artist you are creating both sound and visuals, so you need a real time environment in which to do this that is flexible and open source. You need a tool that is a programming language not a product because you are trying to build a system of your own making. If you just "use" commercial tools etc your output will look boring and tacky.
Visual perception is different to audio and most people have an acute sense of repetition. If you just download loads of video loops and play them for ages your audience will fall asleep. The good VJ's in my opinion build systems with tonnes of content and tonnes of controls so that each set is dynamic, constantly moving and non-repititious. People notice repetition more with visuals and most people in this world have an extremely short attention span etc. Most VJ's underestimate just how content they will need to keep people engaged. Unless your doing wallpaper visuals. In which case... do you really need to be there?
Developing a system is a time consuming and individual task, having Pd and sticking with it is my only personal win. I've always failed when I try shiny new toys that are about style over substance. The ultimate truth is, you have to make the thing. You can't cheat really. You have to build a system that does something and program is otherwise your just a button pusher on someone elses toolkit. You can get a bit of satisfaction but it will dry up.
To be a VJ or AV artist you need to think about designing a system that you can control live that does what you want and makes interesting output. This is something you can develop over time and doesn't need to be code/geeky or really nerdy or whatver. You could curate loads of nice video's and have MIDI controllers etc. However I would just warn you about the difference between video jockey style vs live coded / algorithmic or generative style.
I don't have a huge hard drive full of video's and I don't intent to create one. I feel its an anti pattern instead if you build a flexible enough system capable of dynamic output you will have a small corpus of media to use and play around with. Content generation and VJ'ing are related but I feel the difference is that content can be played anywhere like a film at a cinema and VJ'ing is a performance using a system live.
Video, content and playing video's is different to having an AV or VJ system that you are using like an instrument and have programmed with your intent fully. A static video is both large and will become stale whereas a dynamic patch or system can produce unique output each time. At my event fake[dac~] I encourage this style of work (a bit like live coding in a way) because its fresh. I don't really book or intend to book someone with resolume or commercial VJ software playing "content". Content is boring and not art, especially in today's hyper tech, media saturated world. Everyone is drowning in "content" but starving for real art.
This new "art" I feel is a subtle combination of tech, process and intention. Its interesting to see "how" something is done as much as what is shown. This will play out hopefully in the fake[dac~] and algorave scenes hopefully as people recognise this format. The formula is basically building a system that encodes some cultural perspective and presenting it. I think that the concept of a live coder or audio visual artist will break through into the collective conciousness.
Process
Commercial apps suffer from what I call "vendor lock in"... which means. You invest loads of time and energy into making something that becomes fragile and eventually unusable for some arbitrary reason. Building your own patches in pure data circumvents this problem because we have control of our own patches at all times. We are programmers programming a system not consumers using a product and that is a massive difference. You get a blank/clean slate with Pd and code... you have to build what you want. In my experience the limiting factors are experience and imagination, not money.
Process output
Given enough time the output of a process will become predictable or repetitious somehow. That's why its much harder to make an original sounding piece of music copying music producers on youtube and using the same instruments everyone else uses. The benefit of the algorithmic approach is that you are in your own universe and you will have to create your own system from scratch. Its really difficult at first but when you get the hang of it the output you make is way more interesting.
I have noticed that a process, algorithm or approach always generates output in a range that will become predictable. So if your process or methodology is easily replicated it will likely not be interesting. If you study the output for long enough you will eventually exhaust all tidbits of intrique and it will become repetition. This repetition might happen at such a scale that no-one notices... so you content only repeats every year but its still a cycle. A process of any artistic worth must be dynamic and capable of producing a huge range of output. As an artist if someone else can easily do it or it looks or sounds similar it actually undermines your reputation. Philip glass is who he is because he has carved out that minimalism protocol. Steve Reich sounds similar but not too similar enough to cause a minimalism flame war. I can type in "make some minamilsm classical music" into udio.com right now and generate Philip glass music.
Using genAI we can create works that are original but are clearly derivitive, close enough to sound like what we are aiming for but different enough not to get sued for copyright infringement. This is an incredibly fine line I believe genAI is walking, this is the of creative ownership or the idea that you can turn art into a product. Its both scary and mind bendingly wierd that I can generate a Philip Glass-esque album with no skill within minutes.
Makes you wonder how Philip Glass feels having had his "style" and tendencies assimilated by a machine capable of synthesizing and creating cheap copies at whim. Once a style or thing can be modelled it can be stolen, i wonder if this will make ever more ingenius and complex artistic process and real world stimula a more engaging component of modern culture to come.