Preface: This week I wanted to begin working towards my final project. I'm still unsure exactly what that will be, but I am pretty certain that it will involve some translation between sounds and colors. I just ordered a color sensor, so if I manage to get it in time, I will try to go back and begin using its color samples as inputs to generate sound. For now though, I've decided to use still images as a proxi and figure that it will be easier to figure out what's going on with my script because an image is more stable than light moving in an envirnoment. My goal was to use Processing to come up with a way to trandlate a picture into a song. I wanted to be able to feed in and experiment with various images(and later substitute moving images from life).

TOOLS / SKILLS: Processing

The basic idea.

I used a ton of different images while debugging and playing around--these were some of my favorites. Some are just images I grabbed online, but I made the gradients specifically for this because they helped me calibrate the translation between color/sound.

Interface breakdown.

Working!

The script loads an initial image (any image 1200 x 800 pixels) and immediately begins scanning across it with a moving 50 x 50 px cell sampler. The sampler moves from left to right, up to down and will loop back to the top of the image each time it reaches the bottom. The 2,500 pixels within it are sampled to produce a single tone (sine wave), where the average BRIGHTNESS of the sample determines the OCTAVE, the average HUE of the sample determines the letter of the NOTE, and the average SATURATION of the sample determines whether the note is b / REGULAR / #.

The user can alter the image in 2 ways: STAMP and COLOR. In STAMP MODE the user samples from one portion of the image by typing S. When the mouse is held down and dragged, the pixels around it will be colored over (Alpha=150) by pixels from the sample location. In COLOR MODE, the user sam- ples a single color from one portion of the image by typing “c.” Now when the mouse is held down and dragged, the pixels around it will be colored over (Alpha=150) with that sample color.

Ultimately, editing the image alters the sound and editing the sound alters the image. The user has the ability to transform both image and sound simultaneously and in the process, produce unexpected/inter- esting new things.

I didn't get around to integrating my color sensor in any way for this week, but I had a great time and feel like I have some ideas for my final project now!