Application Programming

This week, we write an application that interfaces with our input/output board. I decided to take sensor data from my custom-made board and visualize the result first in processing, and then in p5.js, a javascript adaptation of processing for the browser. I was able to get the graphing working properly in processing, but encountered some troubleshooting problems with the p5serial server not reading the serial output from the arduino for the second method (notes on this second approach below).

The Application Approach

I decided to try various attempts at visualizing both the X, Y, Z accelerometer data and the X, Y, Z magnetometer data. My idea for using the former is to detect free-fall conditions in the accelerometer data and depict a free-falling ball on the screen when these conditions are met (i.e. acceleration only due to gravity). For both sensors, I will also do a straightforward plot of X, Y and Z values over time. For the magnetomer, I'm envisioning a data viz of the earth and simulated magnetic field rings, where the apparent thickness of the rings get stronger or weaker depending on the Gauss readings. Eventually, I am looking for my sensors to measure the magnetic field strength of the earth from LEO, and these values should change slightly over the course of an orbit. I will need to procure special sensors for this, as the sensitivity of detecting changes in Earth's magnetic field from LEO requires magnetometers with a different baseline sensitivity. Below is an example concept drawing of the magnetic field, and the idea would be to dynamically vary the appearance of the field lines:

Using Processing

Processing is a great tool for getting started with visualizing sensor data. The basic approach is as follows:

• Plug in your sensor board to Arduino, open serial window to ensure sensor is writing data over the serial port (at the expected baud rate), and close serial window after verifying (or else processing won't be able to read the data in when the port is busy).
• Download processing, and open or write a script using the processing Serial library (availabe for input directly from the processing file menus)to read data from the serial port on your computer (you can check which port your computer is using by checking the bottom right corner of the arduino gui, or by running a USB serial list command in your terminal/command prompt of choice.
• Use the traditional processing draw() methods to visualize the data that the script is reading in from your board.

Digging into the code

Here's my Arduino script (chunked by function) that is reading the accelerometer data and formatting my board's sensor data for transmission over the serial port. A few notes on what it's accomplishing:

• I have intentionally formatted the serial stream to use parameters, for later robustness for dropped packets or a processing script that is a little smarter in what elements of data it wants to pull from the sensor in the future. By chunking the serial stream in segments preceded by these parameters (1,2,3, etc) I can lay a foundation for a smarter communication protocol. I also chose to send ASCII data (normal characters), but could have transmitted the data in raw byte format as well (which is often better for conserving memory and requiring less processing).
• Other than the Serial.print() lines, most of this code is boilerplate from the Examples menu that is included with the Arduino IDE once you have loaded the LSM303dlhc library.
• Use the serial monitor to read the SensorDetails section and check for any errors. This is how you'll know whether the sensor is a) outputting any data at all and b) whether it is outputting reasonably reliable, consistent data (or not).

Arduino libraries, global variables and displaySensorDetails function:

Arduino void Setup() loop, with baud rate setting and sensor detection:

Arduino main loop, writing sensor data from the board to the computer's serial port:

Here's my Processing script below that is receiving data over the serial port and graphing it. A few notes on what it's accomplishing:

• Expecting specific parameters to aid in parsing the serial string
• Using the ControlP5 library for a live-rendering gui of the sensor data (allows incremental "pushing" of data)

Processing libraries, global variables and setup() loop for serial connection and canvas creation:

Processing draw() loop, checking for data on the serial connection before trying to parse and push data to a chart:

Processing parseInput() function, testing for comm protocl parameter variables and assigning variables to store the serial stream values of interest:

Processing initiatlizeGraph() function, setting graphing parameters particular to the ControlP5 library and loading data for the chart:

Output from processing script

Below, you can see the visualized graph, overlaying the X, Y and Z coordinates of the accelerometer data over time. The two subsequent photos show physical flips of the board where the green (), blue (), and red() lines swap positions as the board is turned over.

A few notes on this week's assignment

• Beware trying to reprogram a chip that is running a constant-send serial output. Because the chip is programmed over the tx and rx pins, you may catch the chip while in a loop that's using those tx and rx pins for the previously-programmed serial output.
• To make this processing script more robust, I will eventually modify it to request only the data it needs from the sensor (via parameter selection), rather than accepting all data that is sent
• Instead of setting separate global variables to capture the various serial inputs, I should create an event data structure (like events.accel.X, etc)
• In the future, I could improve this code by considering edge cases (such as when it doesn't find an /n character to end the line, what should happen so it doesn't get stuck in that test loop)
• May need to check the sensor initialization values for proper calibration later (some values were not making sense and might be an overflow range issue)

Attempt at using p5.js for the sensor visualization

Trying a second method, now that the Processing method was successful--> First, I'll need to download some p5 libraries and examples to work off of. Because this application will be built for the browser, I don't need a special application or programming environment--rather, just an editor and the ability to pin samples up to Chrome or Firefox. Below, you can see the rich list of examples from p5, the p5 serial library (required for being able to read from the serial port that my board is sending data over) and a read-me for this library. You can find these materials here

Below, you can see some getting started code, for reading from the serial connection and drawing an output based on this data. I am still working on troubleshooting the proper connectivity in p5, as the p5serialserver utility that is part of the install instructions does not appear to be successfully relaying the serial data from arduino to the browser application (browsers, obviously, cannot read data over a computer's serial port, so this extra node server step is required)