Facial Recognition - customize your mirror for every user!

paviro

Sorry guys :D Has been a while since I last coded MM modules… It’s fixed now :)
@benkej Could you provide me a zip containing your complete MM directory?

benkej

@paviro sure, thank you!
here are my MMM-Facial-Recognition-Tools and MMM-Facial-Recognition folders in tar.bz2 format. My config is included in the second archive.

PointPubMedia

@paviro You think it would be possible to display somewhere in MM2 the output from the camera at the same time as your module with a “debug” option?

It would be easier to place the camera at the right location while working on MM!

s1084836

Hello,

Thanks for building this great module. When I run the “npm start” I keep getting the following message:

[MMM-Facial-Recognition] Webcam Found...
[MMM-Facial-Recognition] User undefined with confidence null logged in.
[MMM-Facial-Recognition] User undefined logged out.
[MMM-Facial-Recognition] User undefined with confidence null logged in.
[MMM-Facial-Recognition] User undefined logged out.
[MMM-Facial-Recognition] User undefined with confidence null logged in.

Sometimes it regoonizes my face, but it is like a 1/1000 change. I’m wondering why, when I run the test script my it recognizes my face really fast and always. I uploaded like 350 photo’s. I’ve tryed diffrend back grounds, with enough light.

I just dont understand why it won’t see my face while the test script is 95% of the time right.

My config File:

`/* Magic Mirror Config Sample
 *
 * By Michael Teeuw http://michaelteeuw.nl
 * MIT Licensed.
 */

var config = {
	port: 8080,

	language: 'en',
	timeFormat: 24,
	units: 'metric',

	modules: [

		{
			module: 'clock',
			position: 'top_left',
			classes: 'default',
		},

		
		{
		    module: 'MMM-Facial-Recognition',
		    classes: 'default',
			config: {
		        // 1=LBPH | 2=Fisher | 3=Eigen
		        recognitionAlgorithm: 1,
		        // Threshold for the confidence of a recognized face before it's considered a
		        // positive match.  Confidence values below this threshold will be considered
		        // a positive match because the lower the confidence value, or distance, the
		        // more confident the algorithm is that the face was correctly detected.
		        lbphThreshold: 50,
		        fisherThreshold: 250,
		        eigenThreshold: 3000,
		        // force the use of a usb webcam on raspberry pi (on other platforms this is always true automatically)
		        useUSBCam: true,
		        // Path to your training xml
		        trainingFile: '/home/pi/Desktop/MagicMirror/modules/MMM-Facial-Recognition/training.xml',
		        // recognition intervall in seconds (smaller number = faster but CPU intens!)
		        interval: 1,
		        // Logout delay after last recognition so that a user does not get instantly logged out if he turns away from the mirror for a few seconds
		        logoutDelay: 5,
		        // Array with usernames (copy and paste from training script)
		        users: ['test', 'tom', 'Tom'],
		        //Module set used for strangers and if no user is detected
		        defaultClass: "default",
		        //Set of modules which should be shown for every user
		        everyoneClass: "everyone"
			
		    }
		},
		

		{
		        module: 'helloworld',
		        position: 'lower_third', // This can be any of the regions.
			classes: 'default',
		        config: {
		            // See 'Configuration options' for more information.
		            text: 'OMG, look at you!'
		     }
  		},


		{
			module: 'compliments',
			position: 'upper_third',
			classes: 'Tom',
		},


			{
			module: 'newsfeed',
			position: 'bottom_left',
			classes: 'Tom',
			config: {
				feeds: [
					{
						title: "Nu Algemeen nieuws!",
						url: "http://www.nu.nl/rss/Algemeen"
					}
				],
				showSourceTitle: true,
				showPublishDate: true
			
			}
		},
		
		
	]

};

/*************** DO NOT EDIT THE LINE BELOW ***************/
if (typeof module !== 'undefined') {module.exports = config;}

benkej

@s1084836 like my mirror. I have the same problem

s1084836

@benkej Oh, its really strange. Did you manage to work around it?

benkej

@s1084836 unfortunately not. I switched off the facial recognition

yakko2001

Hey,

my issue is, i cannt select my USB Cam.
I set “useUSBCam” to “true”, but still get piCam as the choosen one. (PiCam ausgewählt)

MMM-Facial-Recognition] ALGORITHM: LBPH
[MMM-Facial-Recognition] Training data loaded!
[MMM-Facial-Recognition] --------------------
[MMM-Facial-Recognition] PiCam ausgewählt...
Whoops! There was an uncaught exception...
{ Error: mmal: mmal_vc_component_create: failed to create component 'vc.ril.camera' (1:ENOMEM)

{
		    module: 'MMM-Facial-Recognition',
		    config: {
		        // 1=LBPH | 2=Fisher | 3=Eigen
		        recognitionAlgorithm: 1,
		        // Threshold for the confidence of a recognized face before it's considered a
		        // positive match.  Confidence values below this threshold will be considered
		        // a positive match because the lower the confidence value, or distance, the
		        // more confident the algorithm is that the face was correctly detected.
		        lbphThreshold: 50,
		        fisherThreshold: 250,
		        eigenThreshold: 3000,
		        useUSBCam: true,
		        // Path to your training xml
		        trainingFile: 'modules/MMM-Facial-Recognition/training.xml',
		        // recognition intervall in seconds (smaller number = faster but CPU intens!)
		        interval: 2,
		        // Logout delay after last recognition so that a user does not get instantly logged out if he turns away from the mirror for a few seconds
		        logoutDelay: 15,
		        // Array with usernames (copy and paste from training script)
		        users: ['Yakko'],
		        //Module set used for strangers and if no user is detected
		        defaultClass: "default",
		        //Set of modules which should be shown for every user
		        everyoneClass: "everyone",
		        // Boolean to toggle welcomeMessage
		        welcomeMessage: true
		    }
		},

Any Ideas how to force it to take my USB cam?
Thanks,
Yakko

piedaniel

Hi everyone. I try to make work the face recognition but always tell me that I´m strange, but when I do test with the facialreconition.py test alway recognize me. Someone know why I have this problem? How can I solve the problem? Thanks

eliddell

I have not loaded this on the mirror yet but did you change the face users and algorithm on the mirror and it’s config?

I’m having an issue where if I train it to recognize two different people the test script recognizes both as the same person :(

strawberry 3.141

@eliddell you must be twins :D

eliddell

Actually I found a problem in the config.py

If I comment out the line i= min(0, i-1) and return i-1 in the user_label method it works… Now just want to make sure we don’t get an index out of bounds…

Not terribley familiar with python. So what does that min(int, int) do exactly

b00002561

Hi, I seem to be having an issue when trying to implement the face recognition tester. When I run the tester on the file I am getting an error, I cannot seem to resolve it.

This is what displays in terminal:

Loading training data...

ALGORITHM: LBPH

Training data loaded!

Picam selected...

Traceback (most recent call last):

File "facerecognition.py", line 61, in 

label, confidence = model.predict(crop)

TypeError: 'int' object is not iterable

My picamera stays active but I can see that it has stopped on my face in the background, Do you have any idea how to resolve this issue, as I am at a loss here.

j.e.f.f

Hello

This module looks freaking amazing!! I want to give this a try, but I have a few questions that I haven’t been able to find answers for.

Do you recommend I used the NoIR camera, or just the regular Pi Camera? The mirror will be in my front hall by the doorway, under what should be adequate light whenever someone will want to use it. No expectation that this should work in the dark
Where is the best place to mount the camera? My mirror will be based on a 32" TV mounted such that the top of the frame would be slightly higher than my eye level (I’m 188 cm tall), and my wife’s eye level would be at about the halfway point of the mirror. I also have two young boys, aged 8 and 6 and I’d want the camera to recognize them as well. I’m thinking of either mounting the camera on the frame at the top of the mirror, aimed slightly downward, or on one side of the frame at about the vertical center of the mirror. I’ve also read about people mounting the camera behind the mirror, but with mixed results. If that is an option, I could theoretically mount the camera right in the center of the mirror. What’s the best way to route the cable in this case? I don’t plan on having anything in the middle of the screen from the top down to about the center, so I could run the cable from the camera straight up and over the top of the screen. Suppose I would need to colour the cable black or purchase a black cable. Anyone have insight here?
Given that the Pi Camera has a relatively narrow field of view (approx 35 mm) would you suggest I use one of those phone camera wide angle attachments so that the cam can pick up a wider image and have a better chance of getting the face in the image, or do I just increase the likelihood that several faces will be in the picture, defeating the facial recognition routine?

Thanks!
-Jeff

Advokaten

Hey! I managed to ‘complete the training’ and copied training.xml into the modules folder, it recognized my face during testing and everything, however, I get this error when doing npm start and I have no clue what it means:

Starting MagicMirror: v2.1.1
Loading config ...
Loading module helpers ...
Initializing new module helper ...
Module helper loaded: MMM-Facial-Recognition
No helper found for module: clock.
No helper found for module: compliments.
All module helpers loaded.
Starting server on port 8080 ... 
Starting server op port 8080 ... 
Server started ...
Connecting socket for: MMM-Facial-Recognition
Staring module helper: MMM-Facial-Recognition
Sockets connected & modules started ...
Launching application.
[MMM-Facial-Recognition] Facerecognition started...
[MMM-Facial-Recognition] Loading training data...
[MMM-Facial-Recognition] ALGORITHM: LBPH
[MMM-Facial-Recognition] Training data loaded!
[MMM-Facial-Recognition] --------------------
[MMM-Facial-Recognition] Webcam ausgewählt...
Whoops! There was an uncaught exception...
{ Error: TypeError: Required argument 'device' (pos 1) not found
    at PythonShell.parseError (/home/pi/MagicMirror/modules/MMM-Facial-Recognition/node_modules/python-shell/index.js:183:17)
    at terminateIfNeeded (/home/pi/MagicMirror/modules/MMM-Facial-Recognition/node_modules/python-shell/index.js:98:28)
    at ChildProcess. (/home/pi/MagicMirror/modules/MMM-Facial-Recognition/node_modules/python-shell/index.js:88:9)
    at emitTwo (events.js:106:13)
    at ChildProcess.emit (events.js:191:7)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:215:12)
    ----- Python Traceback -----
    File "modules/MMM-Facial-Recognition/facerecognition/facerecognition.py", line 63, in 
      camera = config.get_camera()
    File "/home/pi/MagicMirror/modules/MMM-Facial-Recognition/facerecognition/config.py", line 57, in get_camera
      return webcam.OpenCVCapture(device_id=0)
    File "/home/pi/MagicMirror/modules/MMM-Facial-Recognition/facerecognition/webcam.py", line 31, in __init__
      self._camera.open()
  traceback: 'Traceback (most recent call last):\n  File "modules/MMM-Facial-Recognition/facerecognition/facerecognition.py", line 63, in \n    camera = config.get_camera()\n  File "/home/pi/MagicMirror/modules/MMM-Facial-Recognition/facerecognition/config.py", line 57, in get_camera\n    return webcam.OpenCVCapture(device_id=0)\n  File "/home/pi/MagicMirror/modules/MMM-Facial-Recognition/facerecognition/webcam.py", line 31, in __init__\n    self._camera.open()\nTypeError: Required argument \'device\' (pos 1) not found\n',
  executable: 'python',
  options: null,
  script: 'modules/MMM-Facial-Recognition/facerecognition/facerecognition.py',
  args: [ '{"recognitionAlgorithm":1,"lbphThreshold":50,"fisherThreshold":250,"eigenThreshold":3000,"useUSBCam":true,"trainingFile":"modules/MMM-Facial-Recognition/training.xml","interval":2,"logoutDelay":15,"users":["Alex"],"defaultClass":"default","everyoneClass":"everyone","welcomeMessage":true}' ],
  exitCode: 1 }
MagicMirror will not quit, but it might be a good idea to check why this happened. Maybe no internet connection?
If you think this really is an issue, please open an issue on GitHub: https://github.com/MichMich/MagicMirror/issues

bhepler

@j.e.f.f - I played around with this module months ago. I wasn’t able to get it working satisfactorily for my purposes, but I think what I learned can help.

I don’t think the NOIR camera is necessary for your intended setup. Especially if you aren’t going to mount the camera behind the mirror itself. I do not recommend mounting the camera behind the mirror glass. I tried it and without a strong light source, I think you will just be frustrated in your results.

Given that you’re using a 32" TV, I’m going to assume you’re locked into a portrait orientation. With little kids in the house, I suspect mounting the camera either off to one side or on the top of the frame is your best bet.

The magic mirror concept works best when you’re fairly close to it. The best effect after all is that the mirror shows you information while you’re looking at your own reflection. This is helpful as it encourages people to come close to the mirror to marvel at it, which aids in the facial recognition. The camera has a decent field of view already. Sufficient to get a good look at a person standing in front of the mirror from arms-length to about two meters away. Beyond that, the facial recognition starts to get spotty because there isn’t enough detail in your face, but most people won’t be looking at the mirror from two meters away, right?

I would skip the wide-angle attachments. I don’t think it’s going to be necessary.

j.e.f.f

@bhepler Thanks for your insight! This helps a lot.

FuX

Hey

When i run npm start in “/home/pi/MagicMirror” i am struck at below message:

pi@raspberrypi:~/MagicMirror $ npm start

> magicmirror@2.1.1 start /home/pi/MagicMirror
> sh run-start.sh

Starting MagicMirror: v2.1.1
Loading config ...
Loading module helpers ...
Initializing new module helper ...
Module helper loaded: MMM-Facial-Recognition
No helper found for module: alert.
Initializing new module helper ...
Module helper loaded: updatenotification
No helper found for module: clock.
No helper found for module: compliments.
No helper found for module: currentweather.
No helper found for module: weatherforecast.
Initializing new module helper ...
Module helper loaded: newsfeed
All module helpers loaded.
Starting server on port 8080 ...
Starting server op port 8080 ...
Server started ...
Connecting socket for: MMM-Facial-Recognition
Staring module helper: MMM-Facial-Recognition
Connecting socket for: updatenotification
Connecting socket for: newsfeed
Starting module: newsfeed
Sockets connected & modules started ...
Launching application.
Create new news fetcher for url: http://www.20min.ch/rss/rss.tmpl?type=channel&get=4 - Interval: 300000
[MMM-Facial-Recognition] Facerecognition started...
[MMM-Facial-Recognition] Loading training data...
[MMM-Facial-Recognition] ALGORITHM: LBPH
[MMM-Facial-Recognition] Training data loaded!
[MMM-Facial-Recognition] --------------------
[MMM-Facial-Recognition] PiCam ausgewählt...

/* Magic Mirror Config Sample
 *
 * By Michael Teeuw http://michaelteeuw.nl
 * MIT Licensed.
 */

var config = {
	port: 8080,
	ipWhitelist: ["127.0.0.1", "::ffff:127.0.0.1", "::1"], // Set [] to allow all IP addresses.

	language: "en",
	timeFormat: 24,
	units: "metric",

	modules: [
		{
			module: "MMM-Facial-Recognition",
			classes: "default everyone",
			config: {
				recognitionAlgorithm: 1,
				lbphThreshold: 50,
				fisherThreshold: 250,
				eigenThreshold: 3000,
				useUSBCam: false,
				trainingFile: "modules/MMM-Facial-Recognition/training.xml",
				interval: 2,
				logoutDelay: 15,
				users: ["Stefan"],
				defaultClass: "default",
				everyoneClass: "everyone",
				welcomeMessage: true
			}
		},
	]

};

/*************** DO NOT EDIT THE LINE BELOW ***************/
if (typeof module !== "undefined") {module.exports = config;}

python facerecongnition.py also not work:

pi@raspberrypi:~/MagicMirror/modules/MMM-Facial-Recognition-Tools $ python facerecognition.py
['Stefan']
Using FACE_ALGORITM: 1
Loading training data...
Training data loaded!
Strong:Stefan
No windowing system, writing face.jpg image
Traceback (most recent call last):
  File "facerecognition.py", line 153, in 
    clean_shutdown()
TypeError: clean_shutdown() takes exactly 2 arguments (0 given)

j.e.f.f

So I’ve managed to get this working with a Logitech C170 USB webcam, and I’m pretty stoked!

I have a few questions:

Facial recognition is generally pretty slow… I need to stand in front of the camera for about 10 seconds before I am recognized. Is this normal or a bi-product of the fact that this is a fairly low quality camera… e.g.: if I were to upgrade say to the C920 instead, should I expect better performance? I’ve left the polling setting in config.js at the default 2 seconds – if the recognition were to happen this fast I’d be pretty happy!
There is a reference earlier in the thread that the capture resolution can be increased by modifying the module code. Where do I modify this? The c170 can capture natively at 640x480, so I’d like to experiment with capturing both the training images and the real-time facial recognition at this resolution, in an effort to see if I can get better confidence numbers. I’m currently seeing high 40s to mid 50s confidence. At the default threshold of 50, I am often not recognized, but it works more frequently if I increase the threshold to 60. (Training is based on about 50 images all captured from the c170 in representative lighting.) EDIT: one day later and I’m now often seeing confidence in the high 60’s so I needed to increase the threshold to 70.
I’d like to experiment with the other recognition algorithms, but I can only get LBPH to work. If I try to use Fisher or Eigen I get an error that the training data has not yet been computed. Training.xml was generated using the appropriate algorithm, and config.js was set to the matching algorithm.
With MMM-Facial-Recognition-Tools module, I’m having the same issue as the previous post where I get an error when trying to run tests using facerecognition.py (same error as above). So I need to do my tests with the actual MMM-Facial-Recognition module running on MagicMirror, which as you can imagine is fairly time consuming. Anyone have any insight on getting the test script working?

For context, I’m opting for a webcam instead of the picam because webcams generally look nicer. I’ve spent HOURS scouring the web for a picam enclosure that didn’t look like a high school engineering project, without finding a single one. The closest I could find is this one by Adafruit but it has two problems:

I don’t like that the cover is clear – I don’t want to see the electronics
I don’t like how it routes the ribbon cable

This first point is easily remedied with a can of spray paint, but the second is a little harder to deal with. If anyone has found a nice sleek enclosure for the picam, please share a link!

Secondly, I like that webcams generally also have a microphone built-in, so that I can also use it for voice control. I REALLY wish I could use this one but I contacted their support and they told me it wouldn’t work at all with the Raspberry Pi. Too bad… was hoping the 4-microphone array would really with not having to yell at my mirror to get it to hear me and I like how sleek it is. Would look quite unassuming sitting on top of my mirror.

Thanks for the help
-Jeff

j.e.f.f

Also one more thought: I’m thinking of turning the camera 90 degrees so that it takes a vertical image – better chance of capturing faces of people of different heights that – thinking myself as a tall adult vs my kids, 9 and 6 years old. I’m thinking a 2:3 image (or 9:16 in the case of an HD cam) would really work well here to turn a wide FoV to a “tall” FoV – perfect for people standing right in front of the mirror… If I just turn the camera 90 degrees, and use it to train and capture in that orientation, will the facial recognition algorithms work as-is (i.e.: Can they recognize a face turned sideways) or would the module need to be changed to rotate the capture first before feeding into the algorithm?