Flick Large gesture control

Radu_Stancu

Hello all,

I have tried to use the flick Large sensor together with my Raspberry Pi 2 to control the interface of the MagicMirror, as I have made some pages and wanted to scroll through them.

What I have managed to do is to control MMM-pages, MMM-page-indicator and the default news module via my MMM-flick module.

The controls are as follows:

Swipe Left - Decrement Page
Swipe Right - Increment Page
Swipe Up - Show news description
Swipe Up again - Open News page
Swipe Up as the news page is opened - Scroll down the news page
Swipe Down - Close News Page
Swipe Down again - Close News Description

I`m currently working on implementing the touch functionality. I want to also have mouse control.

The code is not perfect, I`m working it.

Here is the code (note that I have changed some libraries for it to work with the ASUS Tinker Board, as far as I know I`m the only one who has working flicklibs on the TinkerBoard. If someone is interested in getting the flick sensor to work with the TinkerBoard pm me ):

MMM-flick.py - used to read sensor data

 #!/usr/bin/env python
 
 import sys
 
 import json
 
 import time
 
 import signal
 
 import flicklib
 
 import RPi.GPIO as GPIO
 
 #import autopy
 
 
 
 GPIO.setmode(GPIO.BOARD)
 
 GPIO.setup(15, GPIO.OUT) 
 
 GPIO.setup(7, GPIO.OUT)
 
 
 
 #Turn on both LEDs for orange color in Stand By
 
 #GPIO.output(7, True) # Turn on RED LED
 
 #GPIO.output(15, True)# Turn on GREEN LED
 
 
 
 
 
 #Airwheel data
 
 
 
 some_value = 5000
 

 
 last_airwheel = 0
 
 delay = 5000
 
 
 
 
 
 #Get display size
 
 
 
 #width, height = autopy.screen.get_size()
 
 
 
 def to_node(type, message):
 
 	# convert to json and print (node helper will read from stdout)
 
 	try:
 
 		print(json.dumps({type: message}))
 
 	except Exception:
 
 		pass
 
 	# stdout has to be flushed manually to prevent delays in the node helper communication
 
 	sys.stdout.flush()
 
 
 
 to_node("status", 'Flick has started...')
 
 
 
 @flicklib.flick()
 
 def flick(start,finish):
 
   
 
 	#Slide down the newsfeed (DOWN GESTURE)
 
 	if(start == "north" and finish == "south"):
 
 		to_node("gesture", "ARTICLE_LESS_DETAILS")
 
 		GPIO.output(7, True)
 
 		time.sleep(0.5)
 
 		GPIO.output(7, False)
 
 		
 
 	#Slide up the newsfeed (UP GESTURE)
 
 	elif(start == "south" and finish == "north"):
 
 		to_node("gesture", "ARTICLE_MORE_DETAILS")
 
 		GPIO.output(15, True)
 
 		time.sleep(0.5)
 
 		GPIO.output(15, False)
 
 		
 
 	#Next page (RIGHT GESTURE)
 
 	elif(start == "west" and finish == "east"):
 
 		to_node("gesture", "PAGE_DECREMENT")
 
 		GPIO.output(7, True)
 
 		time.sleep(0.5)
 
 		GPIO.output(7, False)
 
 		
 
   
 
 	#Previous page (LEFT GESTURE)
 
 	elif(start == "east" and finish == "west"):
 
 		to_node("gesture", "PAGE_INCREMENT")
 
 		GPIO.output(15, True)
 
 		time.sleep(0.5)
 
 		GPIO.output(15, False)
 
    
 
 @flicklib.airwheel()
 
 def spinny(delta):
 
 	global some_value
 
 	global last_airwheel
 
 	global delay
 
 	some_value += delta
 
 	if some_value < 0:
 
 		some_value = 0
 
 	if some_value > 10000:
 
 		some_value = 10000
 
 	now = int(round(time.time() * 1000))
 
 	if(now - last_airwheel > delay):
 
 		#to_node()
 
 		last_airwheel = now
 
 
 
 #Mouse control via flick board
 
 
 
 #@flicklib.move()
 
 #def move(x,y,z):
 
 #	x = (x) * width
 
 #	y = (y) * height
 
 
 
 #	x = int(x)
 
 #	y = height - int (y)
 
 	
 
 #	if( y > 799 ):
 
 #		y = 799
 
 
 
 	#autopy.mouse.move(x, y)
 
 
 
 #Double tap gesture 
 
 
 
 #@flicklib.double_tap()
 
 #def doubletap(position):
 
 	#
 
 
 
 #Tap gesture
 
 
 
 @flicklib.tap()
 
 def tap(position):
 
 	if position == 'center':
 
 		GPIO.output(15, True)# Turn on GREEN LED
 
 		time.sleep(0.5)
 
 		GPIO.output(15, False)# Turn off GREEN LED
 
 
 
 #Touch gesture
 
 
 
 #@flicklib.touch()
 
 #def touch(position):
 
 	#
 
 
 
 
 
 
 
 signal.pause()

node_helper.js

 'use strict';
 
 
 
 const NodeHelper = require('node_helper');
 
 const {PythonShell} = require('python-shell');
 
 var pythonStarted = false
 
 
 
 module.exports = NodeHelper.create({
 
   
 
 	python_start: function () {
 
 		const self = this;
 
 		const pyshell = new PythonShell('modules/' + this.name + '/MMM-flick.py', { mode: 'json', args: [JSON.stringify(this.config)]});
 
 		
 
 		pyshell.on('message', function (message) {
 
       
 
 			if (message.hasOwnProperty('status')){
 
 			console.log("node_helper_[" + self.name + "]" + message.status);
 
 			}
 
 			
 
 			if (message.hasOwnProperty('gesture')){
 
 			console.log("node_helper_[" + self.name + "] " + message.gesture);
 
 			self.sendSocketNotification("gesture_observed", message.gesture);
 
 			}
 
 		});
 
 		
 
 		 pyshell.end(function (err) {
 
 			if (err) throw err;
 
 			console.log("node_helper_[" + self.name + "] " + 'finished running...');
 
 		});
 
 	},
 
 
 
 	// Subclass socketNotificationReceived received.
 
 	socketNotificationReceived: function(notification, payload) {
 
 		if(notification === 'CONFIG') {
 
 		  this.config = payload
 
 		  if(!pythonStarted) {
 
 			pythonStarted = true;
 
 			this.python_start();
 
 		  };
 
 		};
 
 	}
 
 	
 
 });

MMM-flick.js

Module.register("MMM-flick",{	

		gesture_up: 0,

		gesture_right: 0,	

	// Override socket notification handler.

	socketNotificationReceived: function(notification, payload) {

		if (notification === "gesture_observed"){

			const self = this;

			self.sendNotification(payload);

			if (payload === "up"){

					MM.getModules().withClass(this.config.defaultClass).exceptWithClass(this.config.everyoneClass).enumerate(function(module) {

						module.hide(1000, function() {

							Log.log(module.name + ' is hidden.');

						});

					});

			

					MM.getModules().withClass("class_up_1_show").enumerate(function(module) {

						module.show(1000, function() {

							Log.log(module.name + ' is shown.');

						});

					});

			}

			else if (payload === "down") {

				MM.getModules().withClass("class_up_1_show").enumerate(function(module) {

						module.hide(1000, function() {

							Log.log(module.name + ' is hidden by gesture.');

						});

					});

			}

		}

	},

	

	start: function() {

		this.current_user = null;

		this.sendSocketNotification('CONFIG', this.config);

		Log.info('Starting module: ' + this.name);

	}

	

});

Radu_Stancu

Nvm, I got it working perfectly! xD

macnice

Hello, is it possible to place it behind the mirror? Could you show us a picture of your integration into MagicMirror?
Thank you in advance.

Radu_Stancu

That is exactly what I’m planning to do. In theory it works behind the mirror, but you will lose the touch gestures more than likely. Unfortunately I do not have the mirror yet (it has not been delivered yet), but I’ll post a picture/results when I get it.

So far I wrote the code this way:

Left/right swipes go through MMM-pages & MMM-pages_indicator

Up/down swipes open/close newsfeed details

Each gesture also triggers the LED on the sensor board.

I’m still thinking what other things I can do with the remaining gestures.

I’ll update the correct code after I clean it up a bit.

macnice

This post is deleted!

macnice

@radu_stancu Thank you for your answer I look forward to your feedback, because I would also like to use this solution. 🙂

SteffenSchmidt

Could you describe, if you were successfull with the implementation of the board? Best Regards.

Radu_Stancu

@steffenschmidt, I was successful to some extent. I have managed to use the gestures from the board to control the interface. Up, down, left and right are working and more will follow when I know what I want them to do. The only problem is that the flick sensor is NOT WORKING behind the mirror, probably because of the aluminum/silver layer that makes a mirror a mirror.

I will work on the project in the winter holidays and post some more details.

Radu_Stancu

Updated the code, and also support for the Asus Tinker Board.

Frek

This is exactly what Im looking for. Im trying to get your module to run but Im first of all not a programmer (can do simple things in python an such)

I altered your python script to work with my skywriter board, easy enough. But i wonder how i set up the config file? Just load the module? If I do that I get a error that says "cannot find module “python-shell”

I was hoping the sensor would work behind the mirror to but it wont. I will put it on the inside of the frame insted…