<![CDATA[
<topbar style="display:none;">
<item><a href="../index.html">Home</a></item>
<item><a href="../about_me/about.html">About</a></item>
<item><a href="../week_x/index.html">Final Project</a></item>
</topbar>
<!-- This is a comment, it is ignored by the compiler/interpreter -->
<style>
h0 {
font-family:;
font-size: 30px;
color: #414040;
margin-top: 50px;
margin-bottom: 6px;
word-spacing: 5px;
}
a {
color: #e479f1;
}
</style>
##Final Project
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
###Embody Paint
This is a project about using your body movement with wearables such as shoe or glove to 'paint' in the space.
<iframe width="1000" height="600" src="https://www.youtube.com/embed/7HHD5OSNyz0" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
###Project Overview
01 / Overview
<img src="./system.jpg" alt="strandbeest" width="1000"/>
02 / Calibration
<img src="./shoe-2.JPG" alt="strandbeest" width="1000"/>
03 / Scenario
<img src="./shoe-3.JPG" alt="strandbeest" width="500"/> <img src="./shoe-4.JPG" alt="strandbeest" width="500"/>
###Weekly progress
<a href="http://fab.cba.mit.edu/classes/MAS.863/Harvard/people/rosalielin/week_1/index.html" >week1: Initial Ideas</a>
<a href="http://fab.cba.mit.edu/classes/MAS.863/Harvard/people/rosalielin/week_8/index.html" >week8: Molding and Casting</a>
<a href="http://fab.cba.mit.edu/classes/MAS.863/Harvard/people/rosalielin/week_9/index.html" >week9: Input Devices</a>
<a href="http://fab.cba.mit.edu/classes/MAS.863/Harvard/people/rosalielin/week_11/index.html" >week11: Networking and Communications</a>
<a href="http://fab.cba.mit.edu/classes/MAS.863/Harvard/people/rosalielin/week_12/index.html" >week12: Interface and Application Programming</a>
###Electronics: ESP32 + BNO055
01 / Sketch the schematic
<img src="./01_Electronics/sketch.JPG" alt="strandbeest" width="1000"/>
02 / Draw schematic in Eagle
<img src="./01_Electronics/Eagle-1.jpg" alt="strandbeest" width="1000"/>
03 / PCB fabrication files
<img src="./01_Electronics/Eagle-2.jpg" alt="strandbeest" width="338"/> <img src="./01_Electronics/10k-trace.png" alt="strandbeest" width="330"/> <img src="./01_Electronics/10k-outline.png" alt="strandbeest" width="330"/>
Download PCB files here: <a href="./01_Electronics/10k-trace.png" download>Trace</a> ,and <a href="./01_Electronics/10k-outline.png" download>Outline</a>
04 / PCB milling
<img src="./01_Electronics/Milling.jpg" alt="strandbeest" width="1000"/>
<img src="./01_Electronics/sand.JPG" alt="strandbeest" width="1000"/>
<img src="./01_Electronics/allboard.jpg" alt="strandbeest" width="1000"/>
05 / ESP32 + BNO055 + Battery
<img src="./01_Electronics/result-1.JPG" alt="strandbeest" width="1000"/>
<img src="./01_Electronics/result-2.JPG" alt="strandbeest" width="500"/> <img src="./01_Electronics/result-3.JPG" alt="strandbeest" width="500"/>
06 / Program testing
<img src="./01_Electronics/programm_fail.JPG" alt="strandbeest" width="500"/> <img src="./01_Electronics/programm_success.JPG" alt="strandbeest" width="500"/>
07 / Enclosure design
<img src="./01_Electronics/Enclosure.jpg" alt="strandbeest" width="1000"/>
<img src="./01_Electronics/EN1.JPG" alt="strandbeest" width="500"/> <img src="./01_Electronics/EN3.jpg" alt="strandbeest" width="500"/>
Download enclosure files here: <a href="./01_Electronics/Enclosure-solidoworks.SLDPRT" download>.SLDPRT (for Solidworks)</a>, <a href="./01_Electronics/Enclosure-Rhino.STL" download>.STEP (for Rhino)</a>
###Fabrication: Sole Design
<img src="./03_Sole/Sketch1.jpg" alt="strandbeest" width="500"/> <img src="./03_Sole/Sketch2.jpg" alt="strandbeest" width="500"/>
<img src="./03_Sole/09.png" alt="strandbeest" width="500"/> <img src="./03_Sole/04.JPG" alt="strandbeest" width="500"/>
<img src="./03_Sole/03.JPG" alt="strandbeest" width="500"/> <img src="./03_Sole/05.JPG" alt="strandbeest" width="500"/>
<img src="./03_Sole/07.JPG" alt="strandbeest" width="500"/> <img src="./03_Sole/08.JPG" alt="strandbeest" width="500"/>
### Programming: Websocket to P5.js & AR
-
#### 01 / Data from BNO-055 serial input
There are many data that BNO-055 can read, acceleration, orientation, magnetic, gyroscope,....While not being familiar with the relationship between X,Y,and Z,
<a href='http://fab.cba.mit.edu/classes/MAS.863/Harvard/people/reuben-fischer-baum/index.html'> Reuben Fischer-Baum</a>, who is also investigating in BNO055 for his final project,
gave me a general review of how different dataset works, and what matters to the shoe application.
<img src="./05_Final/read-all-data.jpg" alt="strandbeest" width="500"/>
Download Arduino IDE file here: <a href="./05_Final/BNO055-read-all-data" download>BNO055 read-all-data </a>
#### 02 / Websocket framework
-
#### Server-Client-Websocket Workshop
##### Iulian black boarding time!
<img src="./05_Final/server_client_arduinomix.jpg" alt="strandbeest" width="1000"/>
##### Networking
<img src="./05_Final/router.jpg" alt="strandbeest" width="1000"/>
##### Server-Client
<img src="./05_Final/client-server.jpg" alt="strandbeest" width="1000"/>
##### Websocket-ChromeClient-P5
<img src="./05_Final/websocket-chrome-p5.jpg" alt="strandbeest" width="1000"/>
##### Merge code in Arduino IDE
<img src="./05_Final/merge_arduino.jpg" alt="strandbeest" width="1000"/>
#### LED to Websocket
One of the key features of my project so to make two devices(shoe and web browser) talk wirelessly.
I chose ESP32 and wi-fi communication to achieve so.
The first step to learn websocket is try to control the LED switch through wi-fi.
By following along a handy tutorial <a href="https://randomnerdtutorials.com/esp32-web-server-arduino-ide/">ESP32 Web Server-Arduino IDE</a>,
I was able to build a LED switch that can be controlled from the web browser wirelessly.
<img src="./w12/ws-1.jpg" alt="strandbeest" width="1000"/>
<img src="./w12/ws-2.jpg" alt="strandbeest" width="500"/> <img src="./w12/ws-3.JPG" alt="strandbeest" width="500"/>
<iframe width="1000" height="600" src="https://www.youtube.com/embed/BCZdSpHwOno" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
-
#### BNO055 to Websocket
Then the next step is to modify this websocket code to send accelerometer data instead of LED.
<a href="https://chrome.google.com/webstore/detail/chrome-client/efknfeccddnbjfamncbmieaijdpklphn?hl=en-US" >Chrome Client</a>,
a chrome plug-in, and <a href="https://editor.p5js.org/hsinjul/sketches/U-7FUO0Md" >P5.js</a>, a web-based code visualization tool.
<a href='https://www.gse.harvard.edu/faculty/iulian-radu'>Iulian</a>, our generous TA who offered a websocket workshop to teach me the whole websocket concept, from serve, client,to websocket.
He also gave me a very clear instruction of how to transfer from example code(LED) to my purpose by the following steps:
1. On the ESP use the example from RandomNerd, which sends the LED status: 0/1.
2. On the PC first test using the <a href="https://chrome.google.com/webstore/detail/chrome-client/efknfeccddnbjfamncbmieaijdpklphn?hl=en-US" >Chrome Websocket Client</a> that you can connect to the ESP websocket to read 0/1.
3. On the PC, make <a href="https://editor.p5js.org/hsinjul/sketches/U-7FUO0Md" >P5.js</a> connect to the ESP websocket, so you can see 0/1 in <a href="https://editor.p5js.org/hsinjul/sketches/U-7FUO0Md" >P5.js</a>.
4. Then together we can change the ESP program, so it sends accelerometer instead of 0/1 to <a href="https://editor.p5js.org/hsinjul/sketches/U-7FUO0Md" >P5.js</a>.
<img src="./05_Final/Arduino.jpg" alt="strandbeest" width="500"/> <img src="./05_Final/p5.jpg" alt="strandbeest" width="500"/>
<img src="./05_Final/chromeclient.jpg" alt="strandbeest" width="1000"/>
Download code for websocket here:
1) <a href="./05_Final/BNO055-Arduino-Websocket" download>installed in Arduino IDE (mix Websocket with BNO055 code) </a>
2) <a href="./05_Final/p5%20visualization" download>installed in P5.js </a>
More useful tutorials here:
1) <a href="https://editor.p5js.org/tigoe/sketches/rJDfv8N0"> ws.Client in P5.js</a>
2) <a href="https://randomnerdtutorials.com/esp32-web-server-arduino-ide/">ESP32 Web Server-Arduino IDE</a>
3) <a href="https://chrome.google.com/webstore/detail/chrome-client/efknfeccddnbjfamncbmieaijdpklphn?hl=en-US" >Chrome Client plug-in</a>
-
#### 03 / P5.js websocket
I used P5.js websocket to read 9 data from Arduino IDE, and tried to map the motion data to understandable visualization in P5.
I'm particularly interested in the following datasets: orientation XYZ, linearAccelerationXYZ, magnetic XYZ, which are more relevant to the movement of the shoes or gloves.
The orientation represents the angle between the normal vector of your feet plane to the horizontal, which can be explained by <a href="https://danceswithcode.net/engineeringnotes/rotations_in_3d/images/fig01.png">Euler angles</a>.
The linear acceleration data is to sense how fast each X,Y,Z direction the foot is moving. The magnetic is as compass, indicate where you are in the earth context.
<img src="./05_Final/p5-1.jpg" alt="strandbeest" width="1000"/>
<img src="./05_Final/p5-2.jpg" alt="strandbeest" width="1000"/>
-
#### 04 / AR - Unity websocket
Inspired by <a href='https://www.gse.harvard.edu/faculty/iulian-radu'>Iulian</a>'s AR workshop, I started to think about the possibility of combining shoe movement to augmented reality.
By consulting with Iulian, he also agreed it would be an interesting exploration, so the collaboration formed! We actually got to kick off this collaboration just two weeks before the final.
<img src="./05_Final/I1.JPG" alt="strandbeest" width="500"/> <img src="./05_Final/RosalieAR.jpg" alt="strandbeest" width="500"/>
<img src="./05_Final/AR.jpg" alt="strandbeest" width="1000"/>
<iframe width="1000" height="600" src="https://www.youtube.com/embed/y5xQO2g1gm0" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
-
### Final Design
<img src="./05_Final/R8.jpg" alt="strandbeest" width="1000"/>
<img src="./05_Final/R2.jpg" alt="strandbeest" width="1000"/>
<img src="./05_Final/R7.jpg" alt="strandbeest" width="1000"/>
<img src="./05_Final/R4.jpg" alt="strandbeest" width="330"/> <img src="./05_Final/R3.jpg" alt="strandbeest" width="330"/> <img src="./05_Final/R5.jpg" alt="strandbeest" width="330"/>
<iframe width="1000" height="600" src="https://www.youtube.com/embed/7HHD5OSNyz0" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
]]>