Flick Large gesture control

[Radu_Stancu]

Hello all,

I have tried to use the flick Large sensor together with my Raspberry Pi 2 to control the interface of the MagicMirror, as I have made some pages and wanted to scroll through them.

What I have managed to do is to control MMM-pages, MMM-page-indicator and the default news module via my MMM-flick module.

The controls are as follows:

Swipe Left - Decrement Page
Swipe Right - Increment Page
Swipe Up - Show news description
Swipe Up again - Open News page
Swipe Up as the news page is opened - Scroll down the news page
Swipe Down - Close News Page
Swipe Down again - Close News Description

I`m currently working on implementing the touch functionality. I want to also have mouse control.

The code is not perfect, I`m working it.

Here is the code (note that I have changed some libraries for it to work with the ASUS Tinker Board, as far as I know I`m the only one who has working flicklibs on the TinkerBoard. If someone is interested in getting the flick sensor to work with the TinkerBoard pm me ):

MMM-flick.py - used to read sensor data

 #!/usr/bin/env python
 
 import sys
 
 import json
 
 import time
 
 import signal
 
 import flicklib
 
 import RPi.GPIO as GPIO
 
 #import autopy
 
 
 
 GPIO.setmode(GPIO.BOARD)
 
 GPIO.setup(15, GPIO.OUT) 
 
 GPIO.setup(7, GPIO.OUT)
 
 
 
 #Turn on both LEDs for orange color in Stand By
 
 #GPIO.output(7, True) # Turn on RED LED
 
 #GPIO.output(15, True)# Turn on GREEN LED
 
 
 
 
 
 #Airwheel data
 
 
 
 some_value = 5000
 

 
 last_airwheel = 0
 
 delay = 5000
 
 
 
 
 
 #Get display size
 
 
 
 #width, height = autopy.screen.get_size()
 
 
 
 def to_node(type, message):
 
 	# convert to json and print (node helper will read from stdout)
 
 	try:
 
 		print(json.dumps({type: message}))
 
 	except Exception:
 
 		pass
 
 	# stdout has to be flushed manually to prevent delays in the node helper communication
 
 	sys.stdout.flush()
 
 
 
 to_node("status", 'Flick has started...')
 
 
 
 @flicklib.flick()
 
 def flick(start,finish):
 
   
 
 	#Slide down the newsfeed (DOWN GESTURE)
 
 	if(start == "north" and finish == "south"):
 
 		to_node("gesture", "ARTICLE_LESS_DETAILS")
 
 		GPIO.output(7, True)
 
 		time.sleep(0.5)
 
 		GPIO.output(7, False)
 
 		
 
 	#Slide up the newsfeed (UP GESTURE)
 
 	elif(start == "south" and finish == "north"):
 
 		to_node("gesture", "ARTICLE_MORE_DETAILS")
 
 		GPIO.output(15, True)
 
 		time.sleep(0.5)
 
 		GPIO.output(15, False)
 
 		
 
 	#Next page (RIGHT GESTURE)
 
 	elif(start == "west" and finish == "east"):
 
 		to_node("gesture", "PAGE_DECREMENT")
 
 		GPIO.output(7, True)
 
 		time.sleep(0.5)
 
 		GPIO.output(7, False)
 
 		
 
   
 
 	#Previous page (LEFT GESTURE)
 
 	elif(start == "east" and finish == "west"):
 
 		to_node("gesture", "PAGE_INCREMENT")
 
 		GPIO.output(15, True)
 
 		time.sleep(0.5)
 
 		GPIO.output(15, False)
 
    
 
 @flicklib.airwheel()
 
 def spinny(delta):
 
 	global some_value
 
 	global last_airwheel
 
 	global delay
 
 	some_value += delta
 
 	if some_value < 0:
 
 		some_value = 0
 
 	if some_value > 10000:
 
 		some_value = 10000
 
 	now = int(round(time.time() * 1000))
 
 	if(now - last_airwheel > delay):
 
 		#to_node()
 
 		last_airwheel = now
 
 
 
 #Mouse control via flick board
 
 
 
 #@flicklib.move()
 
 #def move(x,y,z):
 
 #	x = (x) * width
 
 #	y = (y) * height
 
 
 
 #	x = int(x)
 
 #	y = height - int (y)
 
 	
 
 #	if( y > 799 ):
 
 #		y = 799
 
 
 
 	#autopy.mouse.move(x, y)
 
 
 
 #Double tap gesture 
 
 
 
 #@flicklib.double_tap()
 
 #def doubletap(position):
 
 	#
 
 
 
 #Tap gesture
 
 
 
 @flicklib.tap()
 
 def tap(position):
 
 	if position == 'center':
 
 		GPIO.output(15, True)# Turn on GREEN LED
 
 		time.sleep(0.5)
 
 		GPIO.output(15, False)# Turn off GREEN LED
 
 
 
 #Touch gesture
 
 
 
 #@flicklib.touch()
 
 #def touch(position):
 
 	#
 
 
 
 
 
 
 
 signal.pause() 

node_helper.js

 'use strict';
 
 
 
 const NodeHelper = require('node_helper');
 
 const {PythonShell} = require('python-shell');
 
 var pythonStarted = false
 
 
 
 module.exports = NodeHelper.create({
 
   
 
 	python_start: function () {
 
 		const self = this;
 
 		const pyshell = new PythonShell('modules/' + this.name + '/MMM-flick.py', { mode: 'json', args: [JSON.stringify(this.config)]});
 
 		
 
 		pyshell.on('message', function (message) {
 
       
 
 			if (message.hasOwnProperty('status')){
 
 			console.log("node_helper_[" + self.name + "]" + message.status);
 
 			}
 
 			
 
 			if (message.hasOwnProperty('gesture')){
 
 			console.log("node_helper_[" + self.name + "] " + message.gesture);
 
 			self.sendSocketNotification("gesture_observed", message.gesture);
 
 			}
 
 		});
 
 		
 
 		 pyshell.end(function (err) {
 
 			if (err) throw err;
 
 			console.log("node_helper_[" + self.name + "] " + 'finished running...');
 
 		});
 
 	},
 
 
 
 	// Subclass socketNotificationReceived received.
 
 	socketNotificationReceived: function(notification, payload) {
 
 		if(notification === 'CONFIG') {
 
 		  this.config = payload
 
 		  if(!pythonStarted) {
 
 			pythonStarted = true;
 
 			this.python_start();
 
 		  };
 
 		};
 
 	}
 
 	
 
 });

MMM-flick.js

Module.register("MMM-flick",{	

		gesture_up: 0,

		gesture_right: 0,	

	// Override socket notification handler.

	socketNotificationReceived: function(notification, payload) {

		if (notification === "gesture_observed"){

			const self = this;

			self.sendNotification(payload);

			if (payload === "up"){

					MM.getModules().withClass(this.config.defaultClass).exceptWithClass(this.config.everyoneClass).enumerate(function(module) {

						module.hide(1000, function() {

							Log.log(module.name + ' is hidden.');

						});

					});

			

					MM.getModules().withClass("class_up_1_show").enumerate(function(module) {

						module.show(1000, function() {

							Log.log(module.name + ' is shown.');

						});

					});

			}

			else if (payload === "down") {

				MM.getModules().withClass("class_up_1_show").enumerate(function(module) {

						module.hide(1000, function() {

							Log.log(module.name + ' is hidden by gesture.');

						});

					});

			}

		}

	},

	

	start: function() {

		this.current_user = null;

		this.sendSocketNotification('CONFIG', this.config);

		Log.info('Starting module: ' + this.name);

	}

	

});

[Radu_Stancu]

Nvm, I got it working perfectly! xD

[macnice]

Hello, is it possible to place it behind the mirror? Could you show us a picture of your integration into MagicMirror?
Thank you in advance.

[Radu_Stancu]

That is exactly what I’m planning to do. In theory it works behind the mirror, but you will lose the touch gestures more than likely. Unfortunately I do not have the mirror yet (it has not been delivered yet), but I’ll post a picture/results when I get it.

So far I wrote the code this way:

Left/right swipes go through MMM-pages & MMM-pages_indicator

Up/down swipes open/close newsfeed details

Each gesture also triggers the LED on the sensor board.

I’m still thinking what other things I can do with the remaining gestures.

I’ll update the correct code after I clean it up a bit.

[macnice]

This post is deleted!

[macnice]

@radu_stancu Thank you for your answer I look forward to your feedback, because I would also like to use this solution. 🙂

[SteffenSchmidt]

Could you describe, if you were successfull with the implementation of the board? Best Regards.

[Radu_Stancu]

@steffenschmidt, I was successful to some extent. I have managed to use the gestures from the board to control the interface. Up, down, left and right are working and more will follow when I know what I want them to do. The only problem is that the flick sensor is NOT WORKING behind the mirror, probably because of the aluminum/silver layer that makes a mirror a mirror.

I will work on the project in the winter holidays and post some more details.

[Radu_Stancu]

Updated the code, and also support for the Asus Tinker Board.

[Frek]

This is exactly what Im looking for. Im trying to get your module to run but Im first of all not a programmer (can do simple things in python an such)

I altered your python script to work with my skywriter board, easy enough. But i wonder how i set up the config file? Just load the module? If I do that I get a error that says "cannot find module “python-shell”

I was hoping the sensor would work behind the mirror to but it wont. I will put it on the inside of the frame insted…

[Frek]

I sorted it out. Didn’t have python-shell installed :)

[Radu_Stancu]

So you did you get it to work? I have an updated version if you are interested.

[bolish]

@Radu_Stancu
Hi,

Did you considered using (and maybe tweaking a bit) the MMM-GroveGestures module and “only” replace the Grove Gesture Sensor (PAJ7620u2) by the Flick one?

It’s maybe a bit “basic thinking” from me but the MMM-GroveGestures plugin is really working great and both sensor seems to be based on the same working method / supplier / …
So maybe worth to consider it, don’t you think?

(would eb great for me at least, as I’m using MMM-GroveGestures and I’m interested into switching to the flick one).

Regards

[Radu_Stancu]

Hi,

Sorry for my really late reply, I have been working on the module for a while, getting the mouse control to work.

As for your request, I did not know of the MMM-GroveGestures module. I had a look at it and the basic commands are basically the same although the code is written a bit differently. The gestures sequence though is a nice feature and I will look into that and try to do it the following days, but I am not really sure what actions should I bind to those new gestures.

Best regards,

Radu

[xne0n]

@Radu_Stancu
Hi,

Did you make some progress in your work, and so can you share it, I’d be glad to help with the module

Thanks

[Radu_Stancu]

@xne0n, I have made some progress, it runs perfectly smooth on the Asus Tinker board, I have managed to get mouse input working decently. It works by putting your finger close to the flick, the orange led turns on, and then the mouse follows your finger. It is not mouse pad like precision, but it gets the job done. I will update the source code, or maybe add it as a module on github after I finish writing my thesis.

[xne0n]

@Radu_Stancu very interesting ! ok thank you

[kayakbabe]

@Radu_Stancu I would really like to have a copy of your module. I’ve been working with the code you posted at the beginning of this topic thread and I’ve gotten tap added. But the wheel just crashes my flick. I’m building a mirror for my Dad and I’d like to surprise him with one he can control with the flick. It would really impress him.

I hope you are still checking in on this magicmirror builders forum.