As a group, I participated in brainstorming sessions, as a software/UX lead hold meetings to set some goals and timelines, and I also participated in video filming + recording voice!
We all had wild ideas and aspirations of what our machine will be, and the entire diverging and converging process was very fun with the media lab + ACT people!
We were very happy with the final product and storytelling we came up with around the project. Everyone worked hard, and we all actively tried to make the machine building project successful in less than 10 days!
Individual Assignment
For our group, I was a software engineeer / UX Design group lead. As a lead, my role was to understand the needs of software, making sure everything works, but also integrating into larger system.
To do so, I focused on communicating the needs with other group leads, identifying what are the immediate needs or roadblocks that are present in order to execute the next steps.
Other than the responsibilities as the lead, I also actively worked on two different components of the software. I first worked on network communication with the machine, where I can receive information (such as the image from the camera, or the axes of the machine).
After setting network endpoints that will retrieve and send over information, I then moved on to work on the computer vision algorithm that will successfully classify front and back of the coin.
I initially tried to train a CNN model that will identify the similar pattern of the coin back/front, but I later changed into identifying rgb pixel average after using Hough's circle detection.
The decision choice mainly was because this would be a much faster and less load for the training with multiple tasks and communication happening between machine and computer as well as to solve the delay of the videos being real-time streamed
of the coin flip and decision making. Another reason was because during conversation with the mechanical engieneering team, we learned that their way of using solenoids to flip the coin required much lighter coin, therefore we end up 3D printing the coins,
and it was difficult to make the coin prints in time. So we decided to create easy to see,flat and light coin with diffrerent color back and front, which will be used for flipping the coin.
Some small challenges, or important part to notice was being robust to change of lighting as well as the size of the coin. Therefore, I developed a side algorithm that will automatically find the size range of circles to detect as well as being able to monitor the control of light, so that
rgb values can be successfully detected. If not set the circle range, it can end up detecting the circle of either top or the bottom of the cone shaped container we are flipping the coins inside.
As a future consideration, I would try to make the program to be multi-threaded, so that we can have multiple processes going without and task delays and lock, and I will also try to use different 3D print other than red and blue, since it could not be good for people with some color blind challenges.
Moreover, I participated in the UX design part of the software, where I designed how the UI of the website should look like, as well as programming so that the website can livestream both videos (coin flipping on the left panel and crossing out machine on the right panel real time), along with some real-time decision making choices
being shown on the bottom. I initially have it show the decision making process (how the algorithm is deciding) on the bottom, but for simplicity, we decided to focus on showing what are the ones that are being deleted and so on.
