MMM-Face-Reco-DNN - All new Face Recognition

nischi

Hi all

Because our “old” MMM-Facial-Recognition-OCV3 module is not working fine for me and is some days old i decided to create a new module with OpenCV 4.1 and Deep Neural Network to recognize faces. Thanks to normyx for this great module and inspiration for my own new one.

Over all it is compatible with the module from normyx, you can use it over the same classes as he used. The training of the images are a little bit different but much easier as his version (for my point of view).

To be honest the development status are still beta, i finished the module yesterday evening and tested it on my macbook and not yet on my mirror. But this tests on my local machine was successfully.

So if you want try it out and let me know what you think about and what i can extend / change or whatever.

Happy to hear your experiences with the module and how it works with your mirror.

Please read carefully the readme of the module, i hope i don’t forget something, otherwise, also please let me know and i can change it or extend it. Happy to help where i can.

If you find a bug, so let me know with a ticket on GitHub or feel free to do a Pull Request, i will check it as fast as i can.

[card:nischi/MMM-Face-Reco-DNN]

Thanks for your Feedback.

nischi

Module are now Final and tested on real MagicMirror

Ivanov_d

I will give it a go. I am getting mixed results with OCV3 facial recognition.

Ivanov_d

After few days of installing (it takes a lot of time to build OpenCV 4.1 and dlib), trial and errors I finally installed and configured the module and oh boy is it good? Yes it is. It is much faster and accurate than Facial_Recognition_OCV3 that I used before. It was tottaly worth the time.

Thanks a lot to nischi for responding so quickly to the issues I posted on GitHub and to the enhansments he made based on my feedback.

nischi

@Ivanov_d Thanks a lot, glad to hear that.

ChrizZz

as a Noob I have a few questions regarding the installation process - I passed now OpenCV:

At first for me it looks like this is the newest face recognition module and therefore I should go for installing this one instead of the other 2 options, right?

Now I finished the OpenCV installation by using the linked guide from pyimagesearch. There OpenCV was installed in an virtual environment. Is this important? Do I have to install all the libs like dlib or face recognition also in this virtual environment?

nischi

@ChrizZz

Yes its the newest version of the face reco 😊

No its not that important that you use a virtual environment, bit with this you can also install different versions on the same machine. but if you have a virtual environment you need to install all dependencies ther, dlib and face reco too.

Ivanov_d

@ChrizZz I followed the tutorial and installed OpenCV and all dependencies in a virtual environment, but that did not go very well afterwards, you have to reconfigure the module with the correct paths and that can be cumbersome, therefore I started on a clean state and installed OpenCV and the dependencies without a virtual envioronment - e.g. follow the same tutorial without the part related to virtual enviornment. That is the most appropriate for our use case.

ChrizZz

damn - 2nd try without a virtual environment.

@nischi: Is there a chance that you merge your module also with MMM-MotionDetector? Currently I use the camera to activate and deactive my screen and Google told me that it isn’t possible that 2 processes use the same camera. The suggested solution was to use a 2nd camera, merge both processes or stream the camera picture.

Ivanov_d

@ChrizZz I have used camera motion detection and I gave it up, because:

It is not very reliable (maybe it was just my cheap camera that I used for testing back then)
it is resource hungry

Because of that I use an external wireless PIR sensor which is part of my smart home system to turn on/off my SmartMirror and it works great. Since the motion processing happens on my smart home system and the Mirror is turned on/off via SSH command it takes up no resources at all from the MagicMirror which is already at 60% load when using Facial Recognition.

nischi

@ChrizZz
You can use my other Module https://github.com/nischi/MMM-MotionControl to control TV with my Face Reco Module. It communicate with Notifications and is not using an own Camera Stream.

ChrizZz

oh, I didn’t know that you have a modul for it. But unfortunately it isn’t working on my Pi. Your modul doesn’t recognize a motion.

Do I have to uninstall the MMM-MotionDetector first?
Or do I need to run Face-Reco-DNN or the OCV3 version?

//edit: I think I know why it isn’t working. Maybe my display isn’t HDMI-CEC ready. I changed the command of your node_helper.js.
Default:

turnOn: function() {
		var self = this;
		exec('echo "on 0" | cec-client ' + this.config.comport + ' -s -d 1', function (error, stdout, stderr) {
			self.status = 'on';

My version:

turnOn: function() {
		var self = this;
		exec('vcgencmd display_power 1', function (error, stdout, stderr) {
			self.status = 'on';

Instead of sending a note to my display for standby I switch off the display_power. But I can’t wake up the display again. I think I have to use Face-Reco-DNN or Facial-Recognition-OCV3. Currently I don’t use them because I have to do the setup for the DNN version.

nischi

@ChrizZz
Yes you need to use one of the Face Reco Module that this works. Better my new Module ;) it’s much more accurate.

ChrizZz

@nischi said in MMM-Face-Reco-DNN - All new Face Recognition:

@ChrizZz
Yes you need to use one of the Face Reco Module that this works. Better my new Module ;) it’s much more accurate.

Thanks, I will use your new module.

During the setup I have another issue. It’s when I want to start encoding I get this error message:

Traceback (most recent call last):
  File "encode.py", line 5, in <module>
    from imutils import paths
  File "/usr/local/lib/python3.7/dist-packages/imutils/__init__.py", line 8, in <module>
    from .convenience import translate
  File "/usr/local/lib/python3.7/dist-packages/imutils/convenience.py", line 6, in <module>
    import cv2
ModuleNotFoundError: No module named 'cv2'

I think it’s a problem of my OpenCV installation where I skipped the virtualenvs. In step 6 of the tutorial I should do this:

$ cd ~/.virtualenvs/cv/lib/python3.5/site-packages/
$ ln -s /usr/local/python/cv2/python-3.5/cv2.cpython-35m-arm-linux-gnueabihf.so cv2.so
$ cd ~

I skipped that because I have no idea where I have to create the link when I install this without virtualenvs. But it seems that I have to link it…

//edit: Key to success:

cd ~/opencv-4.0.0/build/lib/python3
sudo cp cv2.cpython-35m-arm-linux-gnueabihf.so /usr/local/lib/python3.5/dist-packages/cv2.so

nischi

@ChrizZz
Thanks for your solution. Usually it’s because the link for cv2 is to the wrong python Version.
Happy to hear that it works.

ChrizZz

is it common that it takes 40sec between face recognition and turning the TV on?

I see it happen after a freshly restarted pi, that it takes 40/45sec. The scenario is that I restart my pi, mm is autostarting and the TV is turned off by the module. Between the restart and the 1st face recognition is ~3min time.

nischi

@ChrizZz
Sound a little bit long to me, for me it works faster. But it took a while until they are startet. Do you use default settings?

ChrizZz

@nischi said in MMM-Face-Reco-DNN - All new Face Recognition:

@ChrizZz
Sound a little bit long to me, for me it works faster. But it took a while until they are startet. Do you use default settings?

My logs file is also a bit weird:

[MMM-Face-Reco-DNN] Users ChrizZz logged in.
CECControl received (current, new): off on
CECControl received (current, new): on off
CECControl received (current, new): off on
CECControl received (current, new): on off
CECControl received (current, new): off on
CECControl received (current, new): on off
[MMM-Face-Reco-DNN] Users ChrizZz logged out.
CECControl received (current, new): off on
CECControl received (current, new): on off
CECControl received (current, new): off on
CECControl received (current, new): on off
[MMM-Face-Reco-DNN] Users ChrizZz logged in.
CECControl received (current, new): off on
CECControl received (current, new): on off
[MMM-Face-Reco-DNN] Users ChrizZz logged out.
[MMM-Face-Reco-DNN] Users ChrizZz logged in.
CECControl received (current, new): off on
CECControl received (current, new): on off
CECControl received (current, new): off on
CECControl received (current, new): on off
[MMM-Face-Reco-DNN] Users ChrizZz logged out.
CECControl received (current, new): off on
CECControl received (current, new): on off
CECControl received (current, new): off on
CECControl received (current, new): on off

This is a 3min part of the logs and the 1st time I was in front of the mirror and recognized by the module this evening.

My settings are:

		{
			module: 'MMM-CECControl',
    				config: {
      					  // Comport of your Raspberry Pi
      					  comport: 'RPI',
      					  // Turn the TV off if the Mirror start
       					 offOnStartup: true
    					}
		},
		{
    			module: 'MMM-MotionControl',
   				 config: {
     					   // Delay to turn the TV off
       						 delay: 60000,
       					 // Interval to check modules
 					       interval: 5000,
 				       // Use the module MMM-Facial-Recognition-OCV3
  				      useFacialRecognitionOCV3: false,
   				     // Use the module MMM-Face-Reco-DNN
   				     useMMMFaceRecoDNN: true,
   				     // Array where tv should be on
				        ontime: []
					    }
		},
		{
			module: 'MMM-Face-Reco-DNN',
    				config: {
      					// Logout 15 seconds after user was not detecte anymore, if they will be detected between this 15 Seconds, they delay will start again
      					logoutDelay: 30000,
      					// How many time the recognition starts, with a RasPi 3+ it would be good every 2 seconds
      					checkInterval: 2000,
      					// Module set used for strangers and if no user is detected
      					defaultClass: 'default',
      					// Set of modules which should be shown for every user
      					everyoneClass: 'everyone',
      					// XML to recognize with haarcascae
      					cascade: 'modules/MMM-Face-Reco-DNN/tools/haarcascade_frontalface_default.xml',
      					// Pre encoded pickle with the faces
      					encodings: 'modules/MMM-Face-Reco-DNN/tools/encodings.pickle',
      					// You wanna use pi camera or usb / builtin (1 = raspi camera, 0 = other camera)
      					usePiCamera: 1,
      					// Method of face detection (dnn = deep neural network, haar = haarcascade)
      					method: 'dnn',
      					// Which face detection model to use. "hog" is less accurate but faster on CPUs. "cnn" is a more accurate deep-learning model which is GPU/CUDA accelerated (if available).
      					detectionMethod: 'hog',
      					// How fast in ms should the modules hide and show (face effect)
      					animationSpeed: 0,
      					// Path to Python to run the face recognition (null / '' means default path)
      					pythonPath: null,
      					// Should shown welcome message over alert module from MagicMirror
      					welcomeMessage: true
    					}
		},

dragontattoo

Is it working fine??

ChrizZz

@dragontattoo said in MMM-Face-Reco-DNN - All new Face Recognition:

Is it working fine??

Depends on how you define working fine…The face recognition is working quite good, setting it up is easy with the advice you get in this thread. For me it’s just the “slow” reaction to activate the display which is an issue. But I’m optimistic that there is a way to solve it and I have a special environment with a display which hasn’t CEC control.