MagicMirror² v2.5.0 is available! For more information about this release, check out this topic.

Flick Large gesture control



  • Hello all,

    I am trying to build a smart mirror controlled by a Flick large sensor for RaspBerry Pi. I want to integrate this sensor in the MMM-Gestures module, so no infrared sensors and without the Arduino. I am new to coding in python and I`m not really sure how to do it. Any help would be greatly apreciated.

    So far I have started to understand how the demo python script works. Showing the gestures and positions. Currently I get no error but not working either…

    What I have so far:

    MMM-flick.py - used to read sensor data

     #!/usr/bin/env python
    
     import sys
    
     import json
    
     import time
    
     import signal
    
     import flick
    
     some_value = 5000
    
     last_airwheel = 0
     delay = 5000
    
     def to_node(type, message):
    
    try:
    print(json.dumps({type: message}))
    except Exception:
    	pass
    
    sys.stdout.flush()
    
    to_node("status", 'Flick has started...')
    
    @flicklib.flick()
     def flick(start,finish):
    
    #Down gesture
    if(start == "north" and finish == "south"):
    	to_node("gesture", "down")
    
    #Up gesture
    elif(start == "south" and finish == "north"):
    	to_node("gesture", "up")
    
    #Right gesture
    elif(start == "west" and finish == "east"):
    	to_node("gesture", "next")
    
    #Left Gesture
    elif(start == "east" and finish == "west"):
    	to_node("gesture", "previous")
    
       @flicklib.airwheel()
      def spinny(delta):
    global some_value
    global last_airwheel
    global delay
    some_value += delta
    if some_value < 0:
    	some_value = 0
    if some_value > 10000:
    	some_value = 10000
    now = int(round(time.time() * 1000))
    if(now - last_airwheel > delay):
    	#command
    	last_airwheel = now
    
       signal.pause()
    

    MMM-flick.js:

     Module.register("MMM-flick",{
    
    	defaults: {
    	serverIp: 'localhost', // 192.168.178.29
    },	
    
    // Override socket notification handler.
    socketNotificationReceived: function(notification, payload) {
    	if (notification === "gesture_observed")
    	{
    		const self = this;
    		
    		console.log ("socketnotificationreceived.");
    	
    		self.sendNotification(notification, payload);
    		
    		// interact with newsfeed module upon UP, DOWN
    		var newsfeedModules = MM.getModules().withClass('newsfeed');
    		
    		if(newsfeedModules)
    		{
    		
    			var notification = "UNKNOWN";
    	
    			} else if(payload == 'down'){
    				notification = "ARTICLE_LESS_DETAILS";
    			} else if(payload == 'up'){
    				notification = "ARTICLE_MORE_DETAILS";
    			}
    			
    			// forward gesture to other modules
    			Log.info('Sending notification: ' + notification + '.');
    			self.sendNotification(notification, payload);
    		}
    		// interact with newsfeed module upon UP, DOWN
    		var newsfeedModules = MM.getModules().withClass('newsfeed');
    		
    		if(newsfeedModules)
    		{
    		
    			var notification = "UNKNOWN";
    	
    			} else if(payload == 'next'){
    				notification = "ARTICLE_LESS_DETAILS";
    			} else if(payload == 'previous'){
    				notification = "ARTICLE_MORE_DETAILS";
    			} else {
    				Log.info('No handling received gesture in this module directly:');
    				Log.info(payload);					
    			}
    			
    			// forward gesture to other modules
    			Log.info('Sending notification: ' + notification + '.');
    			this.sendNotification(notification, payload);
    		}
    	}
    },
    
    notificationReceived: function(notification, payload, sender) {
    	if (notification === 'DOM_OBJECTS_CREATED') {
    		MM.getModules().exceptWithClass("default").enumerate(function(module) {
    			module.hide(1000, function() {
    				Log.log('Module is hidden.');
    			});
    		});
    	}
    },
    
    start: function() {
    	this.current_user = null;
    	this.sendSocketNotification('CONFIG', this.config);
    	Log.info('Starting module: ' + this.name);
    }
    
     });
    

    node_helper.js:

    'use strict';
    
      // communication between web view and this node.js application happens via 
     websockets
    var ws = require("nodejs-websocket");
    
     // by default assuming monitor is on
      var hdmiOn = true;
    
    // handler for timeout function, used to clear timer when display goes off
    var turnOffTimer;
    
    // put monitor to sleep after 5 minutes without gesture or distance events
    var WAIT_UNTIL_SLEEP = 5*60*1000;
    
    const NodeHelper = require('node_helper');
    var PythonShell = require('python-shell');
    var pythonStarted = false
    
    // setup websocket server
    var server = ws.createServer(function (conn) {
    console.log("New connection from client established")
    conn.on("text", function (str) {
        console.log("Received "+str)
        // echo received text to sender
        conn.sendText(str.toUpperCase()+"!!!")
    })
    conn.on("close", function (code, reason) {
        console.log("Connection closed")
    })
     }).listen(8004);
    
    // broadcast text messages to all subscribers (open web views)
    function broadcast(str) {
    server.connections.forEach(function (connection) {
    	connection.sendText(str);
     });
    }
    
    // turn display on or off
      function saveEnergy(person) {
    
    // deactivate timeout handler if present
    if(turnOffTimer){
    	clearTimeout(turnOffTimer);
    }
    
    // turn on display if off and person is present in front of mirror
    if(person == "PRESENT" && !hdmiOn){
    
    	// make system call to power on display
    	var exec = require('child_process').exec;
    	exec('tvservice -p', function(error, stdout, stderr) {
    		if (error !== null) {
    			console.log(new Date() + ': exec error: ' + error);
    		} else {
    			process.stdout.write(new Date() + ': Turned monitor on.\n');
    			hdmiOn = true;
    		}
    	});
    
    }
    // activate timer to turn off display if display is on and person is away for a while
    else if(person == "AWAY" && hdmiOn) {
    	
    	// activate time to turn off display
    	turnOffTimer = setTimeout(function(){
    	
    		// make system call to turn off display
    		var exec = require('child_process').exec;
    		exec('tvservice -o', function(error, stdout, stderr) {
    			if (error !== null) {
    				console.log(new Date() + ': exec error: ' + error);
    			} else {
    				process.stdout.write(new Date() + ': Turned monitor off.\n');
    				hdmiOn = false;
    			}
    		});
    	
    	}, WAIT_UNTIL_SLEEP);
    
    }
    	
    };
    
    module.exports = NodeHelper.create({
    
    python_start: function () {
    	const self = this;
    	const pyshell = new PythonShell('modules/MMM-flick/MMM-flick.py', { mode: 'json', args: [JSON.stringify(this.config)]});
    	
    	pyshell.on('message', function (message) {
      
    		if (message.hasOwnProperty('status')){
    		console.log("node_helper_[" + self.name + "]" + message.status);
    		}
    		
    		if (message.hasOwnProperty('gesture')){
    		console.log("node_helper_[" + self.name + "] " + message.gesture);
    		self.sendSocketNotification("gesture_observed", message.gesture);
    		}
    	});
    	
    	 pyshell.end(function (err) {
    		if (err) throw err;
    		console.log("node_helper_[" + self.name + "] " + 'finished running...');
    	});
    },
    
    // Subclass socketNotificationReceived received.
    socketNotificationReceived: function(notification, payload) {
    	if(notification === 'CONFIG') {
    	  this.config = payload
    	  if(!pythonStarted) {
    		pythonStarted = true;
    		this.python_start();
    	  };
    	};
    }
    
    });
    

    Thanks.



  • Nvm, I got it working perfectly! xD



  • Hello, is it possible to place it behind the mirror? Could you show us a picture of your integration into MagicMirror?
    Thank you in advance.



  • That is exactly what I’m planning to do. In theory it works behind the mirror, but you will lose the touch gestures more than likely. Unfortunately I do not have the mirror yet (it has not been delivered yet), but I’ll post a picture/results when I get it.

    So far I wrote the code this way:

    Left/right swipes go through MMM-pages & MMM-pages_indicator

    Up/down swipes open/close newsfeed details

    Each gesture also triggers the LED on the sensor board.

    I’m still thinking what other things I can do with the remaining gestures.

    I’ll update the correct code after I clean it up a bit.



  • This post is deleted!


  • @radu_stancu Thank you for your answer I look forward to your feedback, because I would also like to use this solution. 🙂



  • Could you describe, if you were successfull with the implementation of the board? Best Regards.