Interface and Application Programming
This week we need to design an application to interface a user to an input or output device we made previously. This is about as far from my comfort zone as we can get, so I decided to scope the assignment for something rather simple and straightforward. Neil linked a 2021 Fab Academy page as the example assignment for this week, and one of the projects there could be easily adapted for my situation. Let's dive in.
My goal is to visually represent the incoming serial data from my theremin to a basic software interface. On Nadieh's 2021 Fab Academy page, she shows that serial data can be graphically displayed in a retro stereo-mixer program using node.js and D3. Below are examples of her work from a phototransistor.
Since my theremin is making sounds, this seems like a nice example to follow along and have a simple interface for the device. The SAMD21 in my theremin prints serial data in the loop that is measured from the native QTouch inputs. Connecting this serial data to my app and graphically displaying it will be the main objectives for this work. My final project wont require this interface, so this week's assignment is an isolated case. The main hurdle is going to be JavaScript. I have no experience with JS and I have no experience with any of the tools associated with it. This is a steep learning curve for me.
While I suppose it's rather trivial, a lot of effort went into the below figure. Just getting a JS 'Hello World' is a victory for me. I downloaded Node, a JavaScript environment, and called that in both cmd and my VSCode workspace to test it out (this sentence holds about a day's worth of troubleshooting and feeling like a disappointment to my bloodline). But here is my small victory!
From there it was a matter of following along with the example on Nadieh's page (along with a lot of googling). Her example notes that D3 also has a very steep learning curve, so my spiral development will first be to get her example working, as is, with my theremin serial data, then redesign the interface a bit to familiarize myself with D3. Below are Nadieh's codes that I'm using to create the interface, calling on D3 and SerialPort (the JS/Serial communication package). The server side code is what runs the data acquisition from the serial connection and parses it to be used in the script side code that generates the evolving bar graph.
Server Code
/////// Express & Socket.io ///////
//Keep in order
const path = require('path');
const express = require('express');
const app = express();
const http = require('http');
const server = http.createServer(app);
const port_num = 3000;
const { Server } = require("socket.io");
const io = new Server(server);
app.use(express.static(path.join(__dirname, '/'))); //Had to add this otherwise it wouldn't work
app.get('/', (req, res) => {
res.sendFile(__dirname + '/index.html');
})
io.on('connection', (socket) => {
console.log('connection found');
socket.on('disconnect', function(){
console.log('disconnected');
});
})//io.on
server.listen(port_num, () => {
console.log(`listening on *:${port_num}`);
})
/////// Serial port ///////
const SerialPort = require('serialport');
const port = new SerialPort('COM7', {
baudRate: 9600
})
const ReadLine = require('@serialport/parser-readline')
const parser = new ReadLine({delimiter: '\r\n'});
port.pipe(parser);
parser.on('data', readSerialData);
function readSerialData(data) {
//console.log(data);
io.emit('parsed-data', data);
}//function readSerialData
The main thing I needed to change here was the address to the serial port for my device. I couldn't exactly find a straightforward answer to how the SerialPort package wants to take that path, both the documentation and online forums were vague, but I did manage to find an example that worked using the tpyical COM specification like in the Arduino IDE. This seems like a reasonable place to start for me.
For the script side code, the only thing I need to change upfront is the scaling. The phototransistor Nadieh used maxed at 1023, but my application should be capped closer to 60. While large jumps can happen in the QTouch data when the user accidentaly touches the leads, the main bounds for the data is 0-50. Setting an upper bound of 75 seems like a reasonable starting point that I can work from.
Script Code
let socket = io()
//////////////////// Set-up the Visual ////////////////////
//10-bit value that can come in from the phototransistor
const max_value = 75
//Create SVG
const width = 500
const height = 850
const svg = d3.select("#chart").append("svg")
.attr("width", width)
.attr("height", height)
.attr("class", "svg-chart")
//Create scales
const scale_color_wb = d3.scaleLinear()
.domain([500, 950])
.range(["white", "#0d0f1d"])
.clamp(true)
const scale_color_bw = d3.scaleLinear()
.domain([500, 950])
.range(["#0d0f1d", "white"])
.clamp(true)
const colors = ["#66489f", "#ff33a5", "#efb605"].reverse()
const scale_color_bars = d3.scaleLinear()
.domain(d3.range(colors.length).map(d => { return d/(colors.length-1) * max_value}))
.range(colors)
.clamp(true)
///////////////////// Background color ////////////////////
const rect_background = svg.append("rect")
.attr("class", "background")
.attr("x", 0)
.attr("y", 0)
.attr("width", width)
.attr("height", height)
.style("opacity", 0.7)
function updateBackground(data) {
rect_background.style("fill", scale_color_bars(data))
}//function updateBackground
////////////////// Text on the background /////////////////
const text_value = svg.append("text")
.attr("class", "background-text")
.attr("transform", `translate(${width/2}, ${height/2})`)
.style("text-anchor", "middle")
.style("font-size", "170px")
function updateText(data) {
text_value
.style("fill", "white") //scale_color_wb(data))
.text(data)
}//function updateText
//////////////////////// Bar chart ////////////////////////
const bar_group = svg.append("g")
.attr("class", "bar-group")
.attr("transform", `translate(${width/2}, ${0})`)
const bar_width = 25
const bar_height = 6
const bar_corner = bar_height/2
function updateBars(data) {
let num_bars = Math.round(data/10)
//Append new data to the group
let bars = bar_group.selectAll(".bar")
.data(d3.range(num_bars), d => d)
//EXIT
bars.exit().remove()
//ENTER - Group
let enter_bars = bars.enter()
.append("rect")
.attr("class", "bar")
.attr("id", d => `bar-${d}`)
.attr("x", -bar_width/2)
.attr("width", bar_width)
.attr("height", bar_height)
.attr("rx", bar_corner)
.style("fill", d => scale_color_bars(d * 10))
//MERGE
bars = enter_bars.merge(bars)
/////////////////////// ENTER + UPDATE //////////////////////
bars
.attr("y", d => height - d * (bar_height + 2))
}//function updateBars
///////////////////////// Get Data ////////////////////////
//Get the data from the serial port and update the visual
socket.on('connect', function () {
socket.on('parsed-data', function (data) {
//console.log(data)
//Update the visual
updateBackground(data)
updateText(data)
updateBars(data)
})//socket.on
})//socket.on
This is where I started running into problems. As I called on node to execute the scripts I would recieve no errors but also no actions from the terminal. In some basic tests to see if I was able to stream any serial data from the SAMD I got nothing as well. When I tried to have SerialPort list the available serial ports and their addresses I got nothing. This was rather frustrating, and I can tell I am close to my goal, but that I am likely missing a simple but crucial aspect of this code. Debugging this, in combination with the holiday, ate up my time for the week, and means that this is where I leave it for now. I will need to come back to this and get some further help with JavaScript. I am still happy with my 'Hello World' since that was a big accomplishment for me, but a few extra days (and some help from the TAs) will yeild the results I am looking for. It's getting to be crunch time, so I'll need to budget extra time for revisiting this task while also ensuring my final project progresses as planned.