Week 12 - User Interface Programming

Week 12 - User Interface Programming

2018, Oct 31    

Notes

  • Websocket & Pyserial
  • webgl -> three.js
  • Unity & Unreal

Plan

This week is Thanksgiving holiday! I hope I could enjoy the holiday and do something interesting~

There’re bunch of TODO-list:

  • Write js interface to read multiple touch
  • Read the potentiometer from Arduino -> AVR
  • Control the dynamixel by computer -> Arduino -> AVR
  • Control SG90 Servo motor by Arduino -> AVR

Make

There’re some multi-touch html, but has been adapted to all the browser yet. So I think I could try other tasks first.

Read from the potentiometer

I plan to use the potentiometer as an angle sensor in the final project. So I bought the potentiometer from DigiKey. The most convenient way to test it is with the arduino.

The potentiometer is ranging from 0-10k Ohm. To protect the circuit, I add another 10k resistor in series connection. (It will be better to use a bridge to acquire higher precision in the future).

With the following code, it can read the rotation of the potentiometer.

int analogPin = 3;     // potentiometer wiper (middle terminal) connected to analog pin 3
                       // outside leads to ground and +5V
int val = 0;           // variable to store the value read

void setup()
{
  Serial.begin(9600);              //  setup serial
}

void loop()
{
  val = analogRead(analogPin);     // read the input pin
  Serial.println(val);             // debug value
}

3D print the cap for the potentiometer

To rotate the potentiometer more easily, I plan to 3D print a cap for it. Based on the datasheet, I drew the cap for it. The key parameter is the clearance for the radius. After some iterations, the cap fit the potentiometer perfectly. The final clearance for the radius is 0.2 mm.

Control the Dynamixel MX28T motor with python

The Dynamixel has the SDK for python & C++. Using the library, the computer can control the motor.

Write the user interface

Solving problems:

Javascript talking to Python

By socket, specific with flask.sockio

Real-time control

Using a queue to store command. Push each command into the queue, and start a new thread to execute the last command.

Because the motorMove command has some delay during communication with the motor, directly using the list will cause a stacked delay.

To solve this problem, instead pop the queue after each excution which will cause atom excution problem. I changed the queue into circular queue, and use the head and tail to represent current queue. This change guarantee the last command will be executed and, at the same time, can work in real-time.

This will consume the CPU resource when waiting for the command.

Try multithread Event. The event works perfectly! The subprocess can wait for the event is ready. Here’re some example:

from threading import Event, Thread
import time

class ChildProgram:
    def __init__(self, ready=None):
        self.ready = ready

    def connect(self):
        while True:
            ready.wait()
            ready.clear()
            print("connected")

ready = Event()
program = ChildProgram(ready)

# configure & start thread
thread = Thread(target=program.connect)
thread.start()

while True:
    time.sleep(1)
    ready.set()

The subprocess will wait until the ready event is set. The CPU resouce is close to 0 while the original while True loop will consume 100% CPU.

Demo

Control the openhand with web client.

Control the hand with potentiometers.

Code

flask server

from flask import Flask, url_for, render_template, request, jsonify
from flask_socketio import SocketIO
import openhand as OH
import time
from threading import Thread, Lock, Event

app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app)

BUFFER_SIZE = 100
NUM_MOTOR = 2
q = [[0 for i in range(BUFFER_SIZE)] for _ in range(NUM_MOTOR)]
head = [0 for _ in range(NUM_MOTOR)]
tail = [0 for _ in range(NUM_MOTOR)]

T = OH.Model_T42("/dev/tty.usbserial-AI03QEJD", 0, 1, "MX", 0.2, 0.23)
lock = Lock()

ready = Event()

@app.route("/", methods=['POST', 'GET'])
def hello():
   print(request.form)
   return render_template('base.html', message="hello world", get_flashed_messages=get_flashed_messages)

def get_flashed_messages():
   return ["1","2","3"]

def motor_listener():
    global q, head, tail, ready
    while(1):
        # print(q)
        ready.wait()
        ready.clear()
        for i in range(NUM_MOTOR):
            if head[i] < tail[i] - 1:
                head[i] = tail[i] - 1
                # if i == 0:
                T.moveMotor(i, q[i][head[i] % BUFFER_SIZE])
                # if i == 1:
                    # T.moveMotor(i, q[i][head[i] % BUFFER_SIZE])

@socketio.on('message')
def handle_message(message):
    global q, head, tail
    print('received message: ', message)

    for i in range(NUM_MOTOR):
        k = str(i)
        if k in message:
            value = int(message[k]) / 10000.0
            print('servo', k, "value", value)

            q[i][tail[i] % BUFFER_SIZE] = value
            tail[i] += 1
            ready.set()



if __name__ == '__main__':

    t = Thread(target=motor_listener,)
    t.start()

    # app.run()
    socketio.run(app, host='0.0.0.0')

flask front-page:

<!doctype html>
<title>Flaskr</title>
<link rel="stylesheet" href="">

<style>
.slidecontainer {
    width: 100%;
}

.slider {
    -webkit-appearance: none;
    width: 100%;
    height: 25px;
    background: #d3d3d3;
    outline: none;
    opacity: 0.7;
    -webkit-transition: .2s;
    transition: opacity .2s;
}

.slider:hover {
    opacity: 1;
}

.slider::-webkit-slider-thumb {
    -webkit-appearance: none;
    appearance: none;
    width: 100px;
    height: 100px;
    background: #4CAF50;
    cursor: pointer;
}

.slider::-moz-range-thumb {
    width: 25px;
    height: 25px;
    background: #4CAF50;
    cursor: pointer;
}
</style>

<section class="content">

  <div class="slidecontainer" align="center">
    <br/><br/><br/><br/><br/><br/><br/><br/><br/><br/>
    <input type="range" min="1" max="10000" value="1" class="slider" id="servo0" style="width: 50%; height: 100px; orient: vertical" name='slide'>
    <p id="value0"></p>
    <br/><br/><br/><br/><br/><br/><br/><br/><br/><br/>
    <input type="range" min="1" max="10000" value="1" class="slider" id="servo1" style="width: 50%; height: 100px" name='slide'>
    <p id="value1"></p>
  </div>


  <script type="text/javascript">
    var slider0 = document.getElementById("servo0");
    var slider1 = document.getElementById("servo1");
    var output0 = document.getElementById("value0");
    var output1 = document.getElementById("value1");
    output0.innerHTML = slider0.value; // Display the default slider value
    output1.innerHTML = slider1.value; // Display the default slider value

    // Update the current slider value (each time you drag the slider handle)
    slider0.oninput = function() {
        output0.innerHTML = this.value;
        socket.emit('message', {'0': this.value})
    }
    slider1.oninput = function() {
        output1.innerHTML = this.value;
        socket.emit('message', {'1': this.value})
    }
  </script>


  <script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/socket.io/1.3.6/socket.io.min.js"></script>
  <script type="text/javascript" charset="utf-8">
      var socket = io.connect('http://' + document.domain + ':' + location.port);
      socket.on('connect', function() {
          socket.emit('message', {data: '1', status: 'connected'});
      });
  </script>

</section>

Read angle from potentiometer:

from socketio_client import SocketIO, LoggingNamespace
import serial
from threading import Thread, Lock, Event
import openhand as OH
import numpy

ser = serial.Serial('/dev/tty.usbmodem14441', 115200)
socket = SocketIO('localhost', 5000, LoggingNamespace)

NUM_MOTOR = 2


BUFFER_SIZE = 100
q = [[0 for i in range(BUFFER_SIZE)] for _ in range(NUM_MOTOR)]
head = [0 for _ in range(NUM_MOTOR)]
tail = [0 for _ in range(NUM_MOTOR)]

ready = Event()

x_min = 800.0
x_max = 8000.0
thresh = 1
last = [0, 0]
while True:
    line = ser.readline()   # read a '\n' terminated line
    elem = str(line)[2:-5].split(' ')
    print(elem)

    for i in range(NUM_MOTOR):
        try:
            value = eval(elem[i])
        except:
            continue
        if abs(value - last[i]) < thresh:
            continue
        last[i] = value
        x = (value - x_min) / (x_max - x_min)
        x = max(min(x, 1.0), 0.0) 
        if i == 1:
            x = 1 - x
        print(x)

        socket.emit('message', {str(i): x * 10000.0})
        # # socket.wait(seconds=0.005)
Twitter Facebook Google+
# UI