Introduction

Since my background is in computer science, I decided to take this week's assignment as an opportunity to improve my skills in a programming interface and style I am currently less familiar with. I wanted to practice my visual programming skills using input/output nodes and wanted the project to do with video input and sound output as that might come in handy for my final project.

Max MSP

Max MSP was the perfect platform for my programming needs this week. The software is primarily used for video and sound processing and manipulation which is exactly what I was looking to work with. Additionally, they have built-in nodes that support packet sending and serial commands.

Concept

This project was designed to be imaginative, generative and fun. I hoped to create a design that allows users to learn and create together. Drawing inspiration from Prof. Hiroshi Ishii's Clearboard and from many other musical interactives, I set out to write software for a camera/projector telepresence interface that allows for users to paint and generate music together. For children, this could be a tool to help them make spatial associations and to learn more about concepts concerning color, sound and interaction.

The user at one platform loads a blank sheet of semi-transparent paper. As they draw on the page, their platform starts playing back sound based on their composition. If there is another user present and drawing, their drawings and sounds will be projected onto their partners page. Users can save their compositions by storing the physical art piece and replay their content by placing it on the platform. The i/o coincidence of this project is such that the two platforms would share and output the same design space. Additionally, by using physical paintbrush and paint, the setup signifies to the user that they are meant to paint with this device.

Software Interface and Programming

Figure 2: Software Design and Implementation

The video above walks through the Max patch I created. The biggest challenge for me this week was figuring out the vocabulary of Max. I feel that once you know the terminology, this language is extremely intuitive and filled with nifty built-in features. Compared to a game engine, like Unity or Unreal, it can't do everything. What it is great for however, is sound design and 2D video processing which is exactly what I needed my programming environment to do!