AsTeRICS

# XFacetrackerLK

# Component Type: Sensor (Subcategory: Computer Vision)

OS: Windows (x86, x64), Linux (x86, x64, RPi3), Mac OSX

This component provides a cross-platform face tracking computer vision algorithm which puts out estimated movement of a user’s nose and chin in x and y coordinates. The underlying mechanism builds upon the JavaCV library for frame grabbing and computer vision processing. In particular a trained cascade of haar-like features is used in combination with a Lukas Kanade optical flow algorithm to track a face and its movement. The x- and y- coordinates can be used in camera-mouse configurations or to enable selection or control tasks. The values are only provided if a face can be tracked by the algorithm. Note that the x- and y- values represent relative movement in pixels and have to be accumulated (e.g. via the integrate component) to generate e.g. absolute mouse positions.

# Requirements

A camera has to be available (this can be any webcam or a camera which is available as image acquisition device via the operating system). The picture below shows the Logitech Webcam 9000 Pro. Also the camera device of a Kinect sensor or PS3Eye should work. Furthermore, a video stream of an IP camera can be used for tracking.

Logitech Webcam 9000 Pro
Logitech Webcam 9000 Pro

# Note for Raspberry Pi

The plugin can be used on a Raspberry Pi 3 (tested with Raspbian Jessie) with USB cameras, the Kinect camera or the Raspberry Pi camera module.

In order for the raspi cam to work, the following steps are necessary:

  1. Enable the camera with

    sudo raspi-config

  2. Add the v4l driver for the camera, to make it accessible as /dev/videoXX device:

    sudo modprobe bcm2835-v4l2 max_video_width=2592 max_video_height=1944

# Output Port Description

  • noseX [integer]: This value specifies the relative change in the x coordinate of the user’s nose with respect to the previous image frame.
  • noseY [integer]: This value specifies the relative change in the y coordinate of the user’s nose with respect to the previous image frame.
  • chinX [integer]: This value specifies the relative change in the x coordinate of the user’s chin with respect to the previous image frame.
  • chinY [integer]: This value specifies the relative change in the y coordinate of the user’s chin with respect to the previous image frame.

# Event Listener Description

  • init: if this event is triggered, the face recognition procedure is initiated. This can be useful if the correct face position has been lost due to drifting of the LK algorithm.
  • showCameraSettings an incoming event displays the settings window for the camera device (only on Windows), where parameters like image brightness or contrast can be adjusted.

# Properties

  • **frameGrabber[string, combobox selection]😗*Name of FrameGrabber to use (Default, VideoInput, OpenCV, FFmpeg, OpenKinect, PS3Eye, IPCamera).
  • **frameGrabberFormat[string]😗*Format for grabber, e.g. FFmpeg: ‘dshow’ (default), ‘vfwcap’, ‘gdigrab’.
  • **deviceList [string, combobox selection]😗*List of available devices, if supported by grabber
  • cameraSelection [string]: The camera device to be used - use camdIdx e.g. ‘0’ (VideoInput, OpenCV, OpenKinect, PS3Eye), or camName e.g. ‘video=Integrated Camera’ or ‘desktop’ (FFmpeg), stream-url for IPCamera.
  • cameraResolution [string, combobox selection]: This selection box provides several standard camera resolutions. Changing the resolution affects accuracy and performance (CPU load of the runtime system). Provided selections include �160x120�, �320x240�, �640x480�, �800x600�, �1024x768� and �1600x1200�.
  • **frameRate [integer]😗*The frame rate to use for frame grabbing. In case of 0 or a negative value, the maximum frame rate will be used.
  • **titleVideoFrameWindow [string]😗*The title of the window showing the video frame.
  • **displayGUI [boolean]😗*if selected, the GUI window will be shown.
  • **enableOverlaySettings [boolean]😗*if selected, the video frame shows the current frame rate and device name as overlayed text.

# Example Configuraitons

# Default

Selects the best frame grabber for the platform the ARE is running on. (Linux: FFmpeg, Mac OSX: OpenCV, Windows: VideoInput)

Configuration

  • frameGrabber:Default
  • cameraSelection:0

# OpenCV

  • frameGrabber:OpenCV
  • cameraSelection:0

# VideoInput (Windows only)

  • frameGrabber:VideoInput
  • cameraSelection:0

# FFmpeg (Linux only)

  • frameGrabber:FFmpeg
  • frameGrabberFormat (optional): e.g. vfwcap or dshow
  • cameraSelection (device name or device number): e.g. /dev/video0 or 0 (will be mapped to /dev/video0)

# MJPEG stream of SmartPhone camera

Install the following Android app on your smartphone and start streaming the front camera: Camera Stream - WiFi IP Webcam (Web Host LLC)

Enter the http url displayed at your smartphone and add /video to the url

e.g.

http://192.168.1.100:8080/video

Set the cameraSelection property of the XFacetrackerLK plugin to this url.

Configuration

  • frameGrabber:IPCamera
  • cameraSelection:http://192.168.1.100:8080/video

←