Build and experiment basic module sets for a PnP machine controlled by HTML5/JS.

For PnP control software, there is an open source project - Open PnP which is active and mainly written in Java. I have not decided yet whether I use that or build by myself for the final project.

I wanted to test basic building modules with Java Script and C++ this week.

Read resources online: tutorials, blog postings mainly on OpenCV/ C++.

Template Matching — OpenCV 2.4.11.0 documentation : http://docs.opencv.org/doc/tutorials/imgproc/histograms/template_matching/template_matching.html

OpenCV(Open Source Computer Vision) is a library mainly for real-time computer vision developed by Intel Russia research center. OpenCV provides handy functions for each step described in the above section.

I already have OpenCV installed.

$ port installed | grep opencv

opencv @2.4.10_0 (active)

I compiled a sample program written in C++ and run to test the webcam.

---

#include <stdio.h>

#include "opencv2/core/core.hpp"

#include "opencv2/highgui/highgui.hpp"

#include "opencv2/imgproc/imgproc.hpp"

#include "opencv2/nonfree/nonfree.hpp"

int main(void){

int key;

CvCapture *capture;

IplImage *frameImage;

char windowNameCapture[] = "capture";

capture = cvCaptureFromCAM(0);

if(!capture){

printf("there is no camera");

return 0;

}

cvNamedWindow(windowNameCapture, CV_WINDOW_AUTOSIZE);

while(1){

frameImage = cvQueryFrame(capture);

cvShowImage(windowNameCapture, frameImage);

key = cvWaitKey(1);

if(key == 'q'){

break;

}

}

cvReleaseCapture(&capture);

cvDestroyWindow(windowNameCapture);

return 0;

}

---

$ g++ test.cpp `pkg-config --cflags opencv` `pkg-config --libs opencv`

$ ./a.out

Below commands list up necessary library files. Backqoutes (``) will hand these as arguments to g++ compiler.

$ pkg-config —cflags opencv

$ pkg-config —libs opencv

I could run SURF detections several frames per second.

---

#include <cv.h>

#include <highgui.h>

#include <iostream>

using namespace std;

int main(int argc, char** argv) {

CvCapture* capture;

// カメラを初期化

if ((capture = cvCreateCameraCapture(0)) == NULL) {

cerr << "cannot find camera" << endl;

return -1;

}

// ウィンドウを生成

cvNamedWindow("SURF");

IplImage* captureImage = cvQueryFrame(capture);

while (true) {

CvMemStorage* storage = cvCreateMemStorage(0);

CvSeq* imageKeypoints = 0;

CvSeq* imageDescriptors = 0;

CvSURFParams params = cvSURFParams(500, 1);

captureImage = cvQueryFrame(capture);

// グレースケールに変換

IplImage* grayImage = cvCreateImage(cvGetSize(captureImage), 8, 1);

cvCvtColor(captureImage, grayImage, CV_BGR2GRAY);

// フレーム画像からSURFを取得

cvExtractSURF(grayImage, 0, &imageKeypoints, &imageDescriptors, storage, params);

// 画像にキーポイントを描画

for (int i = 0; i < imageKeypoints->total; i++) {

CvSURFPoint* point = (CvSURFPoint*)cvGetSeqElem(imageKeypoints, i); // i番目のキーポイント

CvPoint center; // キーポイントの中心座標

center.x = cvRound(point->pt.x);

center.y = cvRound(point->pt.y);

cvCircle(captureImage, center, 2, cvScalar(0,255,255), CV_FILLED);

}

cvShowImage("SURF", captureImage);

// ループ内で作成したオブジェクトは始末

cvReleaseImage(&grayImage);

cvClearSeq(imageKeypoints);

cvClearSeq(imageDescriptors);

cvReleaseMemStorage(&storage);

// ESCキーが押されたらループを抜ける

int key = cvWaitKey(30);

if (key == 27) {

break;

}

}

// 後始末

cvReleaseCapture(&capture);

cvDestroyAllWindows();

return 0;

}

---

source: http://aidiary.hatenablog.com/entry/20091030/1256905218

Real-time object detection in OpenCV using SURF | Computer Vision, Robotics and Arduino : http://robocv.blogspot.com/2012/02/real-time-object-detection-in-opencv.html

Downloaded main.cpp.

Added “#include <opencv2/nonfree/features2d.hpp>” for an error occurred.

c++ - OpenCV - undefined reference: SurfFeatureDetector and BruteForceMatcher - Stack Overflow : http://stackoverflow.com/questions/9968993/opencv-undefined-reference-surffeaturedetector-and-bruteforcematcher

$ g++ main.cpp `pkg-config --cflags opencv` `pkg-config --libs opencv`

Other refs:

[OpenCV2.3.1] SURFでの画像・ビデオ間のマッチング | Sweden::erasmusLife : https://shinen1.wordpress.com/2012/07/12/opencv2-3-1-surf%E3%81%A7%E3%81%AE%E7%94%BB%E5%83%8F%E3%83%BB%E3%83%93%E3%83%87%E3%82%AA%E9%96%93%E3%81%AE%E3%83%9E%E3%83%83%E3%83%81%E3%83%B3%E3%82%B0/

www.tateisi-f.org/html/reports/h24reports/2021012.pdf : http://www.tateisi-f.org/html/reports/h24reports/2021012.pdf

エッジ画像を用いた照明変化に堅牢なテンプレートマッチング - OhtaLabWiki : http://www.ail.cs.gunma-u.ac.jp/ailwiki/index.php?%E3%82%A8%E3%83%83%E3%82%B8%E7%94%BB%E5%83%8F%E3%82%92%E7%94%A8%E3%81%84%E3%81%9F%E7%85%A7%E6%98%8E%E5%A4%89%E5%8C%96%E3%81%AB%E5%A0%85%E7%89%A2%E3%81%AA%E3%83%86%E3%83%B3%E3%83%97%E3%83%AC%E3%83%BC%E3%83%88%E3%83%9E%E3%83%83%E3%83%81%E3%83%B3%E3%82%B0

OpenCV2でSIFT, SURFによる画像の対応付け - Kesin's diary : http://kesin.hatenablog.com/entry/20120810/1344582180

$ g++ test_image_surf_sift.cpp `pkg-config --cflags opencv` `pkg-config --libs opencv`

$ ./a.out

I used OpenCV tutorial: 

$ ./a.out

Final plan is to establish WebRTC connection between Pi and a laptop.

As a start, I used peerJS to establish peer-to-peer connection using laptops. It worked fine in MIT network. I also used this for a demo function for edible bytes (city farm) project to monitor plants remotely. It worked fine with below steps and codes using node.js/ yeoman web app generator.