Kissenklang Demovideo
Demo of Kissenklang generating sounds according to the pressure value of the blood pressure cuff.
# Gaming, Toys & Music
AsTeRICS can be used to control computer games, music programs or infrared-controlled lights and toys. The keyboard input of a game or a music program can be substituted by another input modality (e.g. head movement, eye gaze, switch press,…) which emulates a key press. Similarily, a toy can be controlled by adapting an input modality to an RC or infrared command.
# Head Movement to Smiley Tutorial
In a first tutorial, you will learn how to map up/down head movements to a discrete number between 1 and 4. The number will than be mapped to one of 4 image paths and the selected image will be shown in the ImageBox widget depending on the head movement.
Later this tutorial can be used as a basis for other use cases as well, simply by exchanging the input signal (e.g. Use an air pressure or sip/puff value instead of head movements) or by exchanging the action (e.g. Send cursor keys UP/DOWN or send a certain Infrared command instead of showing an image).
# Model Description
Below you can see the model used for this tutorial. You can download (opens new window) and open it in the ACS program or directly open it in the WebACS (opens new window) web application.
- The XFacetrackerLK plugin opens the first camera device with a resolution of 320x240 pixels, tracks the head movements and sends the relative movement in y-direction to the Integrate plugin.
- The Integrate plugin integrates the relative movements with minimum and maximum values set to 0 and 240 respectively (same as camera resolution).
- The SignalTranslation plugin translates the values between 0 and 240 to values between 1.00 and 4.00.
- The Quantizer plugin creates a quantized (integer) number by rounding to the nearest integer number.
- The StringDispatcher plugin has predefined Strings (paths to image files) which are sent to the output port depending on the input number (1-4).
- The TextDisplay widget shows the quantized number in a text box for debugging purposes only.
- The ImageBox widget shows the image which path was received at the input port.
# Gaming
Many games can be controlled by keyboard shortcuts e.g. a,s,d,f or left,right,up,down. You can now use the Head Movement to Smiley Tutorial and simply map the head up/down movement to a keyboard key by using they Keyboard plugin.
Simply exchange the slot Strings in the StringDispatcher plugin by the key strings or special key code strings (e.g. {UP} for a cursor up key press emulation).
# Sound and Voice
Instead of showing smiley images you can use the MidiPlayer to create midi tones. In this case directly send the output value of the Quantizer plugin to the input port pitch.
If you want to generate voice, set the slot strings of the StringDispatcher plugin to the words and sentences you would like to synthesize. You must then connect one of the voice generation plugins (e.g. SpeechProcessor, SyntheticVoice, PicoTTS to it.
The project kissenklang uses a blood pressure cuff to create sounds using the MidiPlayer plugin.
# Light
If you would like to control an infrared-controlled light bulb, you can use the IRTrans plugin and set the slot strings of the StringDispatcher plugin to the action strings of the respective infrared commands (e.g. @IRTRANS: snd luminea-bulb,red).
Likewise you could use the PhilipsHue plugin to control a Hue bulb depending on the head movement by setting the respective action string (e.g. {"hue": 0, "sat": 254}).
The project kissenklang uses a blood pressure cuff to control a light bulb via infrared commands sent by the IRTrans plugin.
# Toys
The project kissenklang uses a blood pressure cuff to control a toy robot via infrared commands sent by the IRTrans plugin.
The following tutorial documents a toy helicopter controlled with muscle signals (opens new window).
# Example Projects
# Blood Pressure Cuff Sound and Toy Control
The project Kissenklang (opens new window) used a consumer blood pressure cuff and the button interface FABI (opens new window) to create sounds according to the pressure value of the cuff or to control various items like toy robots, soap bubble machine or infrared lights.