Mechanical Arm

Biographical Information

About Jesse Harlin

Jesse Harlin

As a musician, Jesse Harlin has been composing music for over fifteen years. With experience ranging from vocal work in an award winning electro-acoustic ensemble to performing bassoon in symhonic orchestras (and everything in between), jess eHarlin (stage name : the Simian) is informed by a wide range of influences. Currently, Jesse Harlin aspires to write music for electronic media, such as video games as well as continue to perform in live experimental electronic ensembles on a bizzare performance medium he has invented called the Cube. He is also currently experimenting with robotics that make music as well as with algorithmic compositon.

Harlin creates sound with self-made tools, hybrid compotitions of many electric and electronic devices. In his studio, The Cube and The Motherbrain are made up of many components including computers, nintendo game systems, synthesizers and of course lazers.

As an artist and designer, Jesse Harlin therefore seeks to fuse his musical and sound design training with a healthy dose fo video editing, robotics, programming and other things. Already he has designed new ways of making music and visual art.

About My Work

This paper entitled "My Work" is a biographical work composed in the fall of 2007.

Click the links below to quickly navigate the paper.

My Art: Embracing New Mediums for Human Expression

-Introduction-

I remember when I was younger; I had a conversation with an older musician. He was a pianist, and definitely of the “Old School” way of thought.

“Something went wrong”, he said. “At the beginning of the century we took a wrong turn. We destroyed tonality. Suddenly the physicists decided to make music. And what does that music sound like? There’s no emotion. Everything is a theory exercise.”

I thought about what he had pointed out. There was a lot of validity to his statements. Writing tonal music was blasé in the eyes of many theorists, and the work of some, like Schoenberg works to break down the barriers of tonality. Indeed, the consummate romantic hero musician had collapse under the weight of its own chromaticism.

I replied. “I think physicists have emotions too. I also don’t think all music is about emotion.”

“Well you’re one of THEM he retorted. What then? Will computers make music? How can a calculator express anything? Where is the invention if a computer makes music?”

It was then I had the good sense to think quickly. “Some guy with emotions made that computer. You cannot escape invention. A quill pen is also a technology. Why isn’t a computer now a more complex quill pen? It is all an extension of our nervous system, and as a species our extended nervous system has ascended. Why then shall we not embrace that?”

Intro: A Parking Lot Sales Pitch

Perhaps the hardest thing for an artist to do is to quickly and concisely articulate his work. That is precisely what the thesis of this paper should be: an all encompassing, succinct and explanatory burst of information that explains everything about me in a short time frame.

And here it is:

My art utilizes a multitude of new and interesting media. The medium is the message as well as the content of the medium is the message, involving a combination of music, sound art, video editing, electronics, computer programming, robotics and various other materials for creative output. Subject matter often involves demystification of religion and the supernatural, new interfaces for art generation, creative approaches to performance art, artistic refinement of user interfaces and the exploration of humans as social animals in an emergent system. I embrace the new computational extended nervous system of humans and pull influence heavily from the sciences. To me, innovation leads to a more profound degree of artistic expression.

-Getting Here-

Getting Here: Early Influences

I knew from my conversation with the aging pianist that I had picked a difficult path with my art. I knew I was becoming the kind of creative person that many other creative people love to hate. While they were generating mysticism and voodoo to add mystique to their work, I was trying to dispel mysticism. The creative process was transparent. I wanted people to understand everything about my work, and more importantly the world they lived in. While they were insistent of using age old practices and techniques, I was thirsty for new ones, to explore new artistic ground. I couldn’t even understand why they so succinctly and neatly organized established techniques such as painting, printmaking, photography and digital image manipulation into different categories. Wasn’t this all the same stuff fundamentally?

While many saw little value in using computers to not only implement, but also guide the artistic process, I saw great value and great potential. Was an acoustic piano not a piece of technology? Was not a paintbrush? Was the pipe organ not the first synthesizer, played by J.S. Bach? Did the first percussionist receive such criticism for moving from rocks to stretched membranes?

At the beginning of my creative career I found myself drawn to the work of J.S Bach and Arnold Schoenberg. These were musicians who employed an almost scientific study to their creative output. While their work varies greatly, both were methodical workers who valued ideas of balance and beauty. I have never considered the output of either artist to be devoid of emotion, yet they defile the stereotype of the passionate “rockstar” musician (I’m looking at you Lizst, Mozart, and Kurt Cobain), whom can no better articulate himself on his art than restrain his mysterious passion he claims fuels the artistic process. These two were, in fact, quite sane. They worked much like scientists. I found this to be beautiful.

Soon after, I encountered the work of Iannis Xenakis and John Cage. These are two artists that seem to change the way one even thinks music should be composed. Suddenly, for the first time, I was dipping my toes outside the paradigm of music and it was very refreshing. Iannis was an architect, who did not begin composition until later in life, nearly 30. John Cage redefined what music even was, or should be. He invented new instruments, like the prepared piano, and wrote pieces, like 4:33, and sonatas and interludes that pushed the boundaries of what makes a composer a composer. I began to wonder what made, for example, a composer different from a computer programmer. I saw actually very little difference between both fields.

It was my desire to be like these people that initiated my first attempts at creative material. I saw Bach as the maestro of the first additive synthesizer, the pipe organ, which he maintained and composed for. Schoenberg had frequently built his own instruments, as well as a new tonal system. John cage gave us a new instrument, the prepared piano. It would seem my heroes tried to innovate their craft, as well as their artistic expression.

In an early musical work, The Lunar Savant, I was so inspired by John Cage’s prepared piano piece and my newfound exploration of sound synthesis that I wrote a piece of music for prepared piano and the Access Virus C Subtractive synthesizer.

 

Iannis Xenakis had combined his knowledge of building and architecture with tonal systems to produce interesting new music. My heroes, indeed, were more than composers, they were inventors, engineers, scientists and generally multitalented creative people. For a first step, I knew it was time to build my own instruments so that I might innovate as they did.<

 

Getting Here: Building Excalibur

It is difficult to discuss my emergence to my current state in a linear fashion. Like any developing creative person, a multitude of influences happens simultaneously. While I was stretching my capabilities as a traditional composer, I was also developing a taste for computer music.

I worked for nearly ten years to build and educate myself in the art of studio engineering. I wanted a studio that was interesting and different and tailored to my specific abilities. At the time, I wanted a music generation machine as well as a performance machine. In time I built the Shadow Observatory, my personal studio from the ground up.

The Shadow Observatory Studio

I worked hard to become an expert level studio engineer. My studio has custom build touch screen interfaces, personalized performance interfaces and a hacked Nintendo that is used like a synthesizer. I use this studio to produce music and other media arts.

 

On another front, I was developing as a performer. I started as a humble bassoonist while in college, but eventually began to build my own instruments to perform with. (I assure anyone that playing the bassoon will inspire you to want to redesign your instrument into something a bit less clunky.) It was not long before I completed The Cube, a portable performance apparatus that allows me to play a guitar with MIDI capabilities, process vocals, run a synthesizer, control fog, run sound for the other players, record and shoot lasers( Yes. Lasers). I am still working on new and interesting musical instruments that performers can use.

The CubeThe Cube Interior

The Cube was an early foray into building new musical interfaces. It can digitally interface with my studio.

 

Jesse Harlin playing the Cube

Me performing with The Cube and another performer, whose saxophone we hacked and ran into The Cube’s effects matrix.

 

An advantage of completing the studio and making all my performance devices integrated with the studio was a high degree of customizability and also the ability to expand very easily. A number of applications that do more than record and produce music are currently present, notably applications that allow for conversion of serial information, computer programming, web development and video editing. This hub of information is not totally unlike a painter’s palette or a sculptor’s clay mound. What is interesting is that its customizability is somewhat art in itself, as it is my creation, built for me to create more art. That is art that makes art. The entire apparatus is therefore a machine that really somehow transcends paint or clay. It is a servant that I built with my own hands that I collaborate with on nearly every project.

After the completion of my studio I had most, if not all of, the materials I needed to make the kind of media I wanted. Soon I found I needed even more interaction with the user. I was not content to make music and video the entire time. I wanted complete immersion. The last bit of material acquisition involved the incorporation of electromechanical circuits into my work. I began study of electromechanical circuits and build a robotics lab. It is still under construction, but has served my very well in a variety of projects thus far by increasing user interactivity and art automation in my work.

Jesse Harlin's robotic Workshop

This is a very clean robotics lab. It is usually quite messy, with capacitors, resistors IC’s and a variety of other components strewn about during a project.

 

Getting Here: Later Influences

After expanding beyond the realm of music, namely by expanding the medium of creation I use, I found myself looking to a larger set of influences in art, engineering, and philosophy.

In engineering and mathematics, I was impressed by the minds of Nicola Tesla and Thomas Edison. Also, Issac Newton, Stephen Hawking, Leonhard Euler, Max Planck, Alan Turing, Douglass Engelbart and Don Buchla have been great influences, each for different reasons. I am aware that the mediums I use are very influenced by these people, and in some very basic sense I, like the rest of humans, am standing on the shoulders of giants. I never forget that when I begin my own work.

In philosophy I found myself drawn to Stephen Pinker, Daniel Dennett, Carl Sagan and Richard Dawkins. I should also mention Penn and Teller, as they are performers and illusionists that are dedicated to debunking myths and demystification of magic. All the above are Atheists, and much like Bach was inspired by the Martin Luther of his time, I am inspired by the Atheist leaders of my time. My work frequently employs the concepts of memes and mind viruses and that is influenced by Richard Dawkins’ ideas. Daniel Dennett and Stephen Pinker’s written works additionally give me insight into the idea of a mind virus. A number of works, especially my mobile media pieces, deal with that topic directly.

In art and literature I have acquired some new influences such as Neil Stephenson and Arthur c. Clarke, two science fiction authors, and Tim Hawkinson, an artist who works with robotics and complex signal chains. Hawkins especially influences my work. He is renowned for creating complex sculptural systems through surprisingly simple means. Often his work involved musical elements, and notational elements.

Even other forms of new media like video games have influenced me greatly. Savvy game designers like Warren Spector and Ken Levine are responsible for inventing the “intelligent first person” genre, with philosophically driven open ended storylines. Their visual elements employ a lot of cyber punk imagery and their narrative themes have echoes of philosophers such a Locke, Hobbes and Voltaire, as well as new ideas in science such as those from Richard Dawkins and Stephen Hawkings. They also employed new forms of artificial intelligence in their works, which proved that computer code can be used aesthetically in itself, and not just as an engine that drives aesthetically enriched material.

Musically, my tastes were expanding as well. As I was beginning to experiment with computer music, the academic works of Milton Babbitt, Stockhausen and various minimalist composers, such as Phillip Glass began to make their way into my CD player. Most importantly I began to listen to very complex electronic music, such as Aphex Twin, Venetian Snares, Wisp, The Flashbulb, Squarepusher and Autechre. This music would inform the style I wished to write in, just as much, if not more, than most of my former classical experience and training.

-My Work-

My Work: Early Work

My early work as a creative person consisted almost entirely of composed music. Naturally, my undergraduate career consisted primarily of music school. The earliest works were therefore written in order to work on craft. These were not works necessarily designed to change the world, but they refined my technique and help instill discipline in my work ethic.

At first, like many young artists or musicians, my work consisted of emulating styles and learning already established techniques. At this time my output was very high, and I covered a lot of musical ground. In the academic realm, I wrote two full orchestra pieces, five pieces using strict Baroque counterpoint (including two fugues), two string quartets, a brass ensemble piece, three wind ensemble pieces, four percussion pieces, and four solo instrumental pieces. At this time I was also gigging with an industrial band and studying popular music styles independently.

Soon I began to start writing music in an electronic medium. I wrote seventeen pieces, of varying length and complexity. I also began setting music to film and scoring video games. It was not long before I had scored two games for amateur developers and five short films (between two and twenty minutes of music each).

At this point I then began to experiment with programming and video editing myself.

My Work: Not-So Early Work

My first attempt at an algorithmic composition was a piece entitled “Color Spray” in which players rolled dice to determine pitch and form. Every performance was different, and the piece altogether was successful.

Color spray by Jesse Harlin

This is a screenshot from a page of Color Spray. All information in the piece was determined by chance, and then execute by performers. This was my first step towards using programming and computers to make music, and eventually other forms of art.

 

The next step was the adoption of a program called Max/Msp as well as the introduction of video and image manipulation into my work.

Some early video works included some simple music video- typed works. I created two pieces based on the science of Cymatics. This is the study of wave phenomena as is creates physical patterns in various materials. This is an appropriate topic as it coincided with my personal elision from working with sound only to working with a visual medium as well. I soon realized that sound and light both are just energy. I was working in applications that could easily convert one to the other, and allow them to interact in interesting ways.

Cymatics, a video work by Jesse HarlinCymatics, a video work by Jesse HarlinCymatics, a video work by Jesse HarlinCymatics, a video work by Jesse Harlin

These are some screenshot from Cymatics a multimedia work that explored the relationship between sound energy and physical form.

 

In Cymatics, my first work was to take the audio from the Cymatics and reform them into a musical collage that corresponded with the images. The result was two video pieces totaling about 11 minutes.

After the completion of Cymatics, I wanted to expand my capabilities as a video editor and my technique as a musician working with visual media. I had also finished reading The Ancestor’s Tale by Richard Dawkins, The Selfish Gene by Dawkins and Sperm Wars by Robin Baker. Rather than make another work that commented on the medium itself, I wanted to use my newfound techniques to explore an idea unrelated to the technology I was using. The result was an interesting found footage work that depicted the human animal courtship ritual. The footage was taken, interestingly enough, from an early OU “how-to” courtship video by a psychologist called “Are You Popular”. Interspliced are other bits of found footage, mostly from nature footage, microbe footage, and era advertisements. The result was a multimedia work entitled Walley likes Sally and was incorporated into a live performance given at the Upgrade! International Art Convention.

Walley Likes Salley, a video work by Jesse HarlinWalley Likes Salley, a video work by Jesse HarlinWalley Likes Salley, a video work by Jesse Harlin

Wally likes Sally is a video work that can be performed in front of a live audience. It has found footage and triggered event that coincide with musical moments and can be controlled by The Cube, my live performance device.

 

I also wanted to make video works that explored my newfound interest in meme theories and emergent systems. One of my next video pieces involved the creation of a series of mobile media pieces meant to exist on a cellphone, PDA, iPhone, or any other piece of multimedia. The first works compared information to a lambda bacteriophage virus.

Virus Information for Mobile Media, a video work by Jesse HarlinVirus Information for Mobile Media, a video work by Jesse HarlinVirus Information for Mobile Media, a video work by Jesse HarlinVirus Information for Mobile Media, a video work by Jesse Harlin

This was an early work that combined video editing, compositing and rendering in order to express Daniel Dennet’s idea of mind virus and Dawkins’ memetic theory as it applies to mass communication in humans.

This was an important work from a technical perspective because it allowed me to learn how to manipulate images beyond simple cutting. I was able to render material completely from scratch, and my skill as an editor increased greatly.

My Work: Recent Work

In recent work, I am finally synthesizing all of the elements I have developed independently into singular pieces. A program that is helping me do so is called Max/Msp and is an object oriented programming environment with extensive video, MIDI, and data converting capabilities. In one piece entitled, Growth, the user walks into a room and sees a hanging video game controller. The user then manipulates the video game controller that controls an algorithmically generated piece of music that plays as computer graphic leaves are generated on a wall. As the music becomes more intense, the leaves grow faster and thicker.

Growth, an interactive installation by Jesse Harlin

This is the artist’s statement that corresponds with the installation. The leaves in the background are the same leaves the user controls.

 

There are glowing projected leaves as well as leaves coming out of the screen in to the physical spaces. Likewise, the music comes out of a large PA system. It is as if the user is completely immersed in the growth.

Growth, an interactive installation by Jesse HarlinGrowth, an interactive installation by Jesse Harlin

The piece has been executed twice thus far. It is a synthesis of computer graphics rendering, algorithmic computer music, computer programming, installation sculpture and a user interface.

 

After growth, I wanted to further develop my robotic skills in a way that resulted in another new musical interface. It was then that I created Harpbot, a fully functional robot with two accelerometers, eighteen LEDs, two microprocessors and eight strings. This was also my first attmpt to program a microprocessor.

Harpbot by Jesse Harlin Harpbot by Jesse Harlin Harpbot by Jesse Harlin

Harpbot can be played by manipulating the two accelerometers. It is quite intuitive, and resulted in an interesting and eerie sound palette.

 

turned out pretty well, and was an interesting new way to control a vibrating string. Virtually no musical experience was necessary to control Harpbot, and the accelerometers added a fun and kinesthetic way to make music.

Harpbot by Jesse Harlin

In this image, Harpbot is on display at the IAO gallery in Oklahoma City.

 

Another recent work that synthesizes visual media, music, programming and an interesting user interface is The Modern Art Machine. This device used a joystick to create a Mondrian-esque painting, which can then be saved and printed by the user. During the generation process, FM synthesis adds music to the user’s movements resulting in a musical interplay of sight and sound.

Screenshot of The modern Art Generator by jesse Harlin

This is a screenshot of the program. Sound design and image programming are controlled by a joystick the user wields.

 

The patch was built entirely in the Max/ Msp programming environment, and pulled from many aspects of my training. A joystick was hacked, and its serial information was sent into a custom built image processor. The information was also routed to a custom built FM synthesizer that generates a new sonic palette each time the user decides to make a new work.

The work has yet to be officially premiered, but I did post 310 images generated from the machine throughout the art building one day. The effect of the machine is best felt when a large quantity of its generated works is on display simultaneously.

 

Modern Art Modern Art Modern Art
Modern Art Modern Art Modern Art
Modern Art Modern Art Modern Art

Over 500 unique works of art, such as the nine above, have been created by The Modern Art Machine. It is intuitive to use, as well as musical.

 

My Work: Future Work

I am excited about the future of my work. I have already started a few projects, that will combine many facets of my skill set to synthesize interesting art that is, in some sense, alive.

The first project I am working on involves the reworking of a fish tank to essential treat the fish like a microprocessor. The Fish is in a tank with a hard light cast upon it. The light then kits photoresistive cells that detect the shadows of the fish.

fishtank sketch

This is a basic concept drawing of the fish microprocessor.

The circuit has already been built, and I am currently applying it to the fish tank. There will be over 300 photoresistors and 3000 red LED’s.

 

 

circuit for fishtank circuit for fishtankcircuit for fishtank circuit for fishtank

The circuit detects shadows, thus enabling the fish to make “decisions” in the circuit.

The work will be something of a sculpture. Later, I am working a way to have a camera pointed at the LED output matrix, which will then generate sound. The sound will then be fed back into the Beta’s aquarium (as well as the art space). The result will be a sort of feedback loop, as the fishes respond the results of their actions. This is a short term project and is expect to be completed within the month.

Another project, a slightly longer term project, involves the use of Markov sets to generate tonal music based on sensory input. For this project, the craft is very important, because it involves an interesting mathematical approach to generative music. The result; however, should be immune to any cleverness of craft. The goal is to generate music the sounds inherently more natural, by means of a good mathematical approach.

Markov Sets

Here is a general pictorial depiction of how a Markov set works. A Markov set is a stochastic process, or in other words a way of doing things randomly. You essentially have an array of numbers, and each number has a probability assigned to the next number it can go to. Furthermore, there are hierarchies of orders that enable how far back the system “remembers”. The result isn’t entirely random, and isn’t entirely scripted. I believe that this model, that has been used in a number of other applications, such as the stock market, energy fluctuation models, remote protein homologies, and artificial intelligence, is also well suited for tonal systems.

The primary element comes from sensor systems. First, the goal is to simply make a closed Markov model that makes music that, to my composer’s ears, is aesthetically pleasing. That is the first way in which artistry is introduced into application of the mathematical model. Afterwards I can introduce the aforementioned sensor systems. I am still debating exactly what I want to sense, but my first impressions are things like temperature and room occupancy. The result would be ambient music in a space that actually changes as the environment changes. The unit could be moved to any environment and would generate new music. My job as the composer would be to decide how the environment affects the music. This is a long term project that I have considered devoting to graduate thesis level material.

The payoff would be great. This would be a great step in learning how to implement low-level artificial intelligence into the artistic process. I think that composers and artists absolutely have a lot of intellectual material at their disposal. I hope the craft of the work does not overshadow my aesthetic decisions with the craft. Perhaps, someday, artists who use this sort of medium will be as commonplace as an artist working on a piece of canvas, and likewise just as readily understood. I hope that the entire process is largely demystified to the viewer. I do not intend to hide the clockwork of the Markov set piece in a metaphorical “black box”, but rather a “transparent box of understanding”.

I have a few other ideas on the table as well. I am currently selecting a microprocessor for a MIDI hamster cage, I am researching and working on a program that will help visualize fractal geometry, and I am working on a short piece that uses a piezo microphone to amplify the sound of an ant colony. The other projects exist somewhere between the Markov set project and the LED augmented fishtank in respect to how long they will take to complete.

-Looking Ahead-

I am excited about my future projects. As I look back over the progress it took to arrive at my current research, I can see that I have traversed a variety of mediums, a variety of approaches to artistic aesthetic, and a multitude of technical challenges. I am happy that my work has turned this direction, and I hope to follow it where it leads me enthusiastically. I see a future where the lines between practitioners of various crafts are blurred and the lines between practitioners of various fields of study such as art, engineering, and science are likewise just as homogenous. I also feel like my work fits well in this future. Essentially that is my art’s place in society.

As far as my art’s purpose, I feel like a trend emerges as one looks back at my progression to this point. As humans we no longer think linearly about out work and media. The manuscript is becoming hypertext and the movie is becoming a nonlinear videogame. Media is becoming ever more interactive and nonlinear in its experience. The possibilities are still relatively unexplored, and as we progress technologically the artist is able to express so much more, and in such a different manner. I want to prototype new ways of working with the artistic medium, as well as new ways of experiencing the artistic medium.