For the entire documentation of the Coordinated Camera Rails we built, please check out this [readme](https://gitlab.cba.mit.edu/classes/863.23/CBA/machine).
The overall project had 6 modules.
1. Designing the camera mount
2. Assembling the Camera Rails
3. Camera (Enclosure for the camera and networking)
4. Control Panel to interface with the camera and view camera streams
5. Pi Server to collect raw videos from camera
5. Post processing to generate a final scene.
For this week, my main focus was on the post-processing the videos we received from the cameras mounted on the rails, specifically the AI style transfer pipeline. Here is the software flowchart:
Since my background is in software and AI, I was more familiar with this area of work and it also fit well with my interest in exploring video-video AI based models.
In addition, I also helped out with hardware assembly of the first and second camera rail, using [Jake's belt axis kit](https://gitlab.cba.mit.edu/jakeread/machineweek-2023/-/blob/main/CAD/belt_axis/belt_axis_kit.stl)
and worked with Chance to interface the servos motors with the Modular-Things.
## Video to Video Style Transfer of videos using AI:
Since we wanted to transfer low resolution videos obtained from rs pi camera. The quickest way to do this without using any GPUs was to use models that could be run locally. I had never worked with video to video models so I decided to
try using a some older GAN (Generative Adversarial Network) based models because they are light weight and do not require high processing power. Then moved to working with stable diffusion based Runway ML model to generate cool videos of Alan and Lucy obtained from our camera rails setup.
Since this is a lot of videos - For my full exploration with video to video style transfer that originated with machine building see documentation [here](https://tome.app/genai-0e9/my-experiments-with-generative-ai-and-video-style-transfer-clq16susy15fcof7h55yjfouz)
## Camera Rails
Here is us on the first day of setting up our camera rails and dividing up the tasks -
We looked at CAD designs shared by Jake below to replicate the belt-axis using the servos motors. We realized we were missing some parts of the mounts and 3D printed these parts, for example the [feet](https://gitlab.cba.mit.edu/jakeread/machineweek-2023/-/blob/main/CAD/parts/foot.stl) of the belt axis.
Here is the final video we obtained from our camera rails that we demoed in class:
[Placeholder]