0. What I want to achieve this week

Build and experiment basic module sets for a PnP machine controlled by HTML5/JS.
For PnP control software, there is an open source project - Open PnP which is active and mainly written in Java. I have not decided yet whether I use that or build by myself for the final project.
I wanted to test basic building modules with Java Script and C++ this week.

  1. Computer vision 101
  2. OpenCV programming
  3. WebRTC experiment
  4. Eagle .brd format

1. Computer Vision 101

Read resources online: tutorials, blog postings mainly on OpenCV/ C++.

Template Matching — OpenCV documentation : http://docs.opencv.org/doc/tutorials/imgproc/histograms/template_matching/template_matching.html

Multi-scale Template Matching using Python and OpenCV - PyImageSearch : http://www.pyimagesearch.com/2015/01/26/multi-scale-template-matching-using-python-opencv/

2. OpenCV programming

OpenCV(Open Source Computer Vision) is a library mainly for real-time computer vision developed by Intel Russia research center. OpenCV provides handy functions for each step described in the above section.

2.1 Set up OpenCV environment (for C++ development)

I already have OpenCV installed.

$ port installed | grep opencv
opencv @2.4.10_0 (active)

2.2 Webcam test 

I compiled a sample program written in C++ and run to test the webcam.

#include <stdio.h>
#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/nonfree/nonfree.hpp"

int main(void){
int key;
CvCapture *capture;
IplImage *frameImage;
char windowNameCapture[] = "capture";

capture = cvCaptureFromCAM(0);
printf("there is no camera");
return 0;

cvNamedWindow(windowNameCapture, CV_WINDOW_AUTOSIZE);

frameImage = cvQueryFrame(capture);
cvShowImage(windowNameCapture, frameImage);
key = cvWaitKey(1);
if(key == 'q'){

return 0;
source: http://www.yasutomo57jp.com/2010/09/01/linux%E3%81%A7opencv%E3%82%92%E5%85%A5%E3%82%8C%E3%81%A6%E9%81%A9%E5%BD%93%E3%81%AA%E3%82%B3%E3%83%BC%E3%83%89%E3%82%92%E3%82%B3%E3%83%B3%E3%83%91%E3%82%A4%E3%83%AB%E3%81%99%E3%82%8B%E3%81%BE%E3%81%A7/

$ g++ test.cpp `pkg-config --cflags opencv` `pkg-config --libs opencv`
$ ./a.out

Below commands list up necessary library files. Backqoutes (``) will hand these as arguments to g++ compiler.
$ pkg-config —cflags opencv
$ pkg-config —libs opencv

2.3 SURF test (video and several FPS)

I could run SURF detections several frames per second.

$ g++ surf-test.cpp `pkg-config --cflags opencv` `pkg-config --libs opencv`
$ ./a.out

#include <cv.h>
#include <highgui.h>
#include <iostream>

using namespace std;

int main(int argc, char** argv) {
CvCapture* capture;

// カメラを初期化
if ((capture = cvCreateCameraCapture(0)) == NULL) {
cerr << "cannot find camera" << endl;
return -1;

// ウィンドウを生成

IplImage* captureImage = cvQueryFrame(capture);
while (true) {
CvMemStorage* storage = cvCreateMemStorage(0);
CvSeq* imageKeypoints = 0;
CvSeq* imageDescriptors = 0;
CvSURFParams params = cvSURFParams(500, 1);

captureImage = cvQueryFrame(capture);

// グレースケールに変換
IplImage* grayImage = cvCreateImage(cvGetSize(captureImage), 8, 1);
cvCvtColor(captureImage, grayImage, CV_BGR2GRAY);

// フレーム画像からSURFを取得
cvExtractSURF(grayImage, 0, &imageKeypoints, &imageDescriptors, storage, params);

// 画像にキーポイントを描画
for (int i = 0; i < imageKeypoints->total; i++) {
CvSURFPoint* point = (CvSURFPoint*)cvGetSeqElem(imageKeypoints, i); // i番目のキーポイント
CvPoint center; // キーポイントの中心座標
center.x = cvRound(point->pt.x);
center.y = cvRound(point->pt.y);
cvCircle(captureImage, center, 2, cvScalar(0,255,255), CV_FILLED);
cvShowImage("SURF", captureImage);

// ループ内で作成したオブジェクトは始末

// ESCキーが押されたらループを抜ける
int key = cvWaitKey(30);
if (key == 27) {

// 後始末

return 0;
source: http://aidiary.hatenablog.com/entry/20091030/1256905218

2.4 SURF test (matching between video stream and a template image)

Real-time object detection in OpenCV using SURF | Computer Vision, Robotics and Arduino : http://robocv.blogspot.com/2012/02/real-time-object-detection-in-opencv.html

Downloaded main.cpp.
Added “#include <opencv2/nonfree/features2d.hpp>” for an error occurred.

c++ - OpenCV - undefined reference: SurfFeatureDetector and BruteForceMatcher - Stack Overflow : http://stackoverflow.com/questions/9968993/opencv-undefined-reference-surffeaturedetector-and-bruteforcematcher
If you're using opencv 2.4, SURF and SIFT interfaces are changed to nonfree folder."

$ g++ main.cpp `pkg-config --cflags opencv` `pkg-config --libs opencv`
$ ./a.out

Other refs:
[OpenCV2.3.1] SURFでの画像・ビデオ間のマッチング | Sweden::erasmusLife : https://shinen1.wordpress.com/2012/07/12/opencv2-3-1-surf%E3%81%A7%E3%81%AE%E7%94%BB%E5%83%8F%E3%83%BB%E3%83%93%E3%83%87%E3%82%AA%E9%96%93%E3%81%AE%E3%83%9E%E3%83%83%E3%83%81%E3%83%B3%E3%82%B0/

www.tateisi-f.org/html/reports/h24reports/2021012.pdf : http://www.tateisi-f.org/html/reports/h24reports/2021012.pdf

エッジ画像を用いた照明変化に堅牢なテンプレートマッチング - OhtaLabWiki : http://www.ail.cs.gunma-u.ac.jp/ailwiki/index.php?%E3%82%A8%E3%83%83%E3%82%B8%E7%94%BB%E5%83%8F%E3%82%92%E7%94%A8%E3%81%84%E3%81%9F%E7%85%A7%E6%98%8E%E5%A4%89%E5%8C%96%E3%81%AB%E5%A0%85%E7%89%A2%E3%81%AA%E3%83%86%E3%83%B3%E3%83%97%E3%83%AC%E3%83%BC%E3%83%88%E3%83%9E%E3%83%83%E3%83%81%E3%83%B3%E3%82%B0

2.4 SURF test (matching PCB images)

OpenCV2でSIFT, SURFによる画像の対応付け - Kesin's diary : http://kesin.hatenablog.com/entry/20120810/1344582180

$ g++ test_image_surf_sift.cpp `pkg-config --cflags opencv` `pkg-config --libs opencv`
$ ./a.out

2.5 Template based approach

I used OpenCV tutorial: 

$ g++ template.cpp `pkg-config --cflags opencv` `pkg-config --libs opencv`
$ ./a.out

4. WebRTC experiment

Final plan is to establish WebRTC connection between Pi and a laptop.
As a start, I used peerJS to establish peer-to-peer connection using laptops. It worked fine in MIT network. I also used this for a demo function for edible bytes (city farm) project to monitor plants remotely. It worked fine with below steps and codes using node.js/ yeoman web app generator. 

1. On the page I want to put the video on, I inserted:
<script src="viewer.js"></script>
<video id="video" muted="muted" autoplay="autoplay"></video>

2. Implemented a route to camera.html for the camera side laptop

3. Opened camera.html on the browser of the camera side laptop
* Web browsers should be the latest ver. of chrome, firefox or opera. For android, Chrome 29 or later might work (though I've not tested)

4. Could see the live streaming on the page I put the tags on step 1 and 2.

        <title>fixed point camera - camera</title>
        <script src="http://cdn.peerjs.com/0.3/peer.js" type="text/javascript"></script>
        <script src="camera.js"></script>

// Prepare a camera
navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia;
    video: true, // Video only
    audio: false
}, function(cameraStream){
}, function(){
    alert('You cannot use webrtc on this browser.');
function connectPeer (cameraStream) {
    var peer = new Peer(
        "camera3333", {
        key:   “APIKEY", // PeerJS API key
        debug: 3
    peer.on('connection', function(dataConnection) {
        var mediaConnection = peer.call(dataConnection.peer, cameraStream);
    $(window).on('beforeunload', function(){
        if (! peer.destroyed) {

var peer = new Peer({
    key: “APIKEY", // peerJS API key
    debug: 3
peer.on('call', function(call) {
    // Answer to the camera
    // Bind the stream from the camera to video tag
    call.on('stream', function(stream) {
        $('#video').attr('src', URL.createObjectURL(stream));
// Connect to the camera
$(window).on('beforeunload', function(){
    if (! peer.destroyed) {

5. Integration Plan