Facial Recognition - customize your mirror for every user!

j.e.f.f

Hello

This module looks freaking amazing!! I want to give this a try, but I have a few questions that I haven’t been able to find answers for.

Do you recommend I used the NoIR camera, or just the regular Pi Camera? The mirror will be in my front hall by the doorway, under what should be adequate light whenever someone will want to use it. No expectation that this should work in the dark
Where is the best place to mount the camera? My mirror will be based on a 32" TV mounted such that the top of the frame would be slightly higher than my eye level (I’m 188 cm tall), and my wife’s eye level would be at about the halfway point of the mirror. I also have two young boys, aged 8 and 6 and I’d want the camera to recognize them as well. I’m thinking of either mounting the camera on the frame at the top of the mirror, aimed slightly downward, or on one side of the frame at about the vertical center of the mirror. I’ve also read about people mounting the camera behind the mirror, but with mixed results. If that is an option, I could theoretically mount the camera right in the center of the mirror. What’s the best way to route the cable in this case? I don’t plan on having anything in the middle of the screen from the top down to about the center, so I could run the cable from the camera straight up and over the top of the screen. Suppose I would need to colour the cable black or purchase a black cable. Anyone have insight here?
Given that the Pi Camera has a relatively narrow field of view (approx 35 mm) would you suggest I use one of those phone camera wide angle attachments so that the cam can pick up a wider image and have a better chance of getting the face in the image, or do I just increase the likelihood that several faces will be in the picture, defeating the facial recognition routine?

Thanks!
-Jeff

Advokaten

Hey! I managed to ‘complete the training’ and copied training.xml into the modules folder, it recognized my face during testing and everything, however, I get this error when doing npm start and I have no clue what it means:

Starting MagicMirror: v2.1.1
Loading config ...
Loading module helpers ...
Initializing new module helper ...
Module helper loaded: MMM-Facial-Recognition
No helper found for module: clock.
No helper found for module: compliments.
All module helpers loaded.
Starting server on port 8080 ... 
Starting server op port 8080 ... 
Server started ...
Connecting socket for: MMM-Facial-Recognition
Staring module helper: MMM-Facial-Recognition
Sockets connected & modules started ...
Launching application.
[MMM-Facial-Recognition] Facerecognition started...
[MMM-Facial-Recognition] Loading training data...
[MMM-Facial-Recognition] ALGORITHM: LBPH
[MMM-Facial-Recognition] Training data loaded!
[MMM-Facial-Recognition] --------------------
[MMM-Facial-Recognition] Webcam ausgewählt...
Whoops! There was an uncaught exception...
{ Error: TypeError: Required argument 'device' (pos 1) not found
    at PythonShell.parseError (/home/pi/MagicMirror/modules/MMM-Facial-Recognition/node_modules/python-shell/index.js:183:17)
    at terminateIfNeeded (/home/pi/MagicMirror/modules/MMM-Facial-Recognition/node_modules/python-shell/index.js:98:28)
    at ChildProcess. (/home/pi/MagicMirror/modules/MMM-Facial-Recognition/node_modules/python-shell/index.js:88:9)
    at emitTwo (events.js:106:13)
    at ChildProcess.emit (events.js:191:7)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:215:12)
    ----- Python Traceback -----
    File "modules/MMM-Facial-Recognition/facerecognition/facerecognition.py", line 63, in 
      camera = config.get_camera()
    File "/home/pi/MagicMirror/modules/MMM-Facial-Recognition/facerecognition/config.py", line 57, in get_camera
      return webcam.OpenCVCapture(device_id=0)
    File "/home/pi/MagicMirror/modules/MMM-Facial-Recognition/facerecognition/webcam.py", line 31, in __init__
      self._camera.open()
  traceback: 'Traceback (most recent call last):\n  File "modules/MMM-Facial-Recognition/facerecognition/facerecognition.py", line 63, in \n    camera = config.get_camera()\n  File "/home/pi/MagicMirror/modules/MMM-Facial-Recognition/facerecognition/config.py", line 57, in get_camera\n    return webcam.OpenCVCapture(device_id=0)\n  File "/home/pi/MagicMirror/modules/MMM-Facial-Recognition/facerecognition/webcam.py", line 31, in __init__\n    self._camera.open()\nTypeError: Required argument \'device\' (pos 1) not found\n',
  executable: 'python',
  options: null,
  script: 'modules/MMM-Facial-Recognition/facerecognition/facerecognition.py',
  args: [ '{"recognitionAlgorithm":1,"lbphThreshold":50,"fisherThreshold":250,"eigenThreshold":3000,"useUSBCam":true,"trainingFile":"modules/MMM-Facial-Recognition/training.xml","interval":2,"logoutDelay":15,"users":["Alex"],"defaultClass":"default","everyoneClass":"everyone","welcomeMessage":true}' ],
  exitCode: 1 }
MagicMirror will not quit, but it might be a good idea to check why this happened. Maybe no internet connection?
If you think this really is an issue, please open an issue on GitHub: https://github.com/MichMich/MagicMirror/issues

bhepler

@j.e.f.f - I played around with this module months ago. I wasn’t able to get it working satisfactorily for my purposes, but I think what I learned can help.

I don’t think the NOIR camera is necessary for your intended setup. Especially if you aren’t going to mount the camera behind the mirror itself. I do not recommend mounting the camera behind the mirror glass. I tried it and without a strong light source, I think you will just be frustrated in your results.

Given that you’re using a 32" TV, I’m going to assume you’re locked into a portrait orientation. With little kids in the house, I suspect mounting the camera either off to one side or on the top of the frame is your best bet.

The magic mirror concept works best when you’re fairly close to it. The best effect after all is that the mirror shows you information while you’re looking at your own reflection. This is helpful as it encourages people to come close to the mirror to marvel at it, which aids in the facial recognition. The camera has a decent field of view already. Sufficient to get a good look at a person standing in front of the mirror from arms-length to about two meters away. Beyond that, the facial recognition starts to get spotty because there isn’t enough detail in your face, but most people won’t be looking at the mirror from two meters away, right?

I would skip the wide-angle attachments. I don’t think it’s going to be necessary.

j.e.f.f

@bhepler Thanks for your insight! This helps a lot.

FuX

Hey

When i run npm start in “/home/pi/MagicMirror” i am struck at below message:

pi@raspberrypi:~/MagicMirror $ npm start

> magicmirror@2.1.1 start /home/pi/MagicMirror
> sh run-start.sh

Starting MagicMirror: v2.1.1
Loading config ...
Loading module helpers ...
Initializing new module helper ...
Module helper loaded: MMM-Facial-Recognition
No helper found for module: alert.
Initializing new module helper ...
Module helper loaded: updatenotification
No helper found for module: clock.
No helper found for module: compliments.
No helper found for module: currentweather.
No helper found for module: weatherforecast.
Initializing new module helper ...
Module helper loaded: newsfeed
All module helpers loaded.
Starting server on port 8080 ...
Starting server op port 8080 ...
Server started ...
Connecting socket for: MMM-Facial-Recognition
Staring module helper: MMM-Facial-Recognition
Connecting socket for: updatenotification
Connecting socket for: newsfeed
Starting module: newsfeed
Sockets connected & modules started ...
Launching application.
Create new news fetcher for url: http://www.20min.ch/rss/rss.tmpl?type=channel&get=4 - Interval: 300000
[MMM-Facial-Recognition] Facerecognition started...
[MMM-Facial-Recognition] Loading training data...
[MMM-Facial-Recognition] ALGORITHM: LBPH
[MMM-Facial-Recognition] Training data loaded!
[MMM-Facial-Recognition] --------------------
[MMM-Facial-Recognition] PiCam ausgewählt...

/* Magic Mirror Config Sample
 *
 * By Michael Teeuw http://michaelteeuw.nl
 * MIT Licensed.
 */

var config = {
	port: 8080,
	ipWhitelist: ["127.0.0.1", "::ffff:127.0.0.1", "::1"], // Set [] to allow all IP addresses.

	language: "en",
	timeFormat: 24,
	units: "metric",

	modules: [
		{
			module: "MMM-Facial-Recognition",
			classes: "default everyone",
			config: {
				recognitionAlgorithm: 1,
				lbphThreshold: 50,
				fisherThreshold: 250,
				eigenThreshold: 3000,
				useUSBCam: false,
				trainingFile: "modules/MMM-Facial-Recognition/training.xml",
				interval: 2,
				logoutDelay: 15,
				users: ["Stefan"],
				defaultClass: "default",
				everyoneClass: "everyone",
				welcomeMessage: true
			}
		},
	]

};

/*************** DO NOT EDIT THE LINE BELOW ***************/
if (typeof module !== "undefined") {module.exports = config;}

python facerecongnition.py also not work:

pi@raspberrypi:~/MagicMirror/modules/MMM-Facial-Recognition-Tools $ python facerecognition.py
['Stefan']
Using FACE_ALGORITM: 1
Loading training data...
Training data loaded!
Strong:Stefan
No windowing system, writing face.jpg image
Traceback (most recent call last):
  File "facerecognition.py", line 153, in 
    clean_shutdown()
TypeError: clean_shutdown() takes exactly 2 arguments (0 given)

j.e.f.f

So I’ve managed to get this working with a Logitech C170 USB webcam, and I’m pretty stoked!

I have a few questions:

Facial recognition is generally pretty slow… I need to stand in front of the camera for about 10 seconds before I am recognized. Is this normal or a bi-product of the fact that this is a fairly low quality camera… e.g.: if I were to upgrade say to the C920 instead, should I expect better performance? I’ve left the polling setting in config.js at the default 2 seconds – if the recognition were to happen this fast I’d be pretty happy!
There is a reference earlier in the thread that the capture resolution can be increased by modifying the module code. Where do I modify this? The c170 can capture natively at 640x480, so I’d like to experiment with capturing both the training images and the real-time facial recognition at this resolution, in an effort to see if I can get better confidence numbers. I’m currently seeing high 40s to mid 50s confidence. At the default threshold of 50, I am often not recognized, but it works more frequently if I increase the threshold to 60. (Training is based on about 50 images all captured from the c170 in representative lighting.) EDIT: one day later and I’m now often seeing confidence in the high 60’s so I needed to increase the threshold to 70.
I’d like to experiment with the other recognition algorithms, but I can only get LBPH to work. If I try to use Fisher or Eigen I get an error that the training data has not yet been computed. Training.xml was generated using the appropriate algorithm, and config.js was set to the matching algorithm.
With MMM-Facial-Recognition-Tools module, I’m having the same issue as the previous post where I get an error when trying to run tests using facerecognition.py (same error as above). So I need to do my tests with the actual MMM-Facial-Recognition module running on MagicMirror, which as you can imagine is fairly time consuming. Anyone have any insight on getting the test script working?

For context, I’m opting for a webcam instead of the picam because webcams generally look nicer. I’ve spent HOURS scouring the web for a picam enclosure that didn’t look like a high school engineering project, without finding a single one. The closest I could find is this one by Adafruit but it has two problems:

I don’t like that the cover is clear – I don’t want to see the electronics
I don’t like how it routes the ribbon cable

This first point is easily remedied with a can of spray paint, but the second is a little harder to deal with. If anyone has found a nice sleek enclosure for the picam, please share a link!

Secondly, I like that webcams generally also have a microphone built-in, so that I can also use it for voice control. I REALLY wish I could use this one but I contacted their support and they told me it wouldn’t work at all with the Raspberry Pi. Too bad… was hoping the 4-microphone array would really with not having to yell at my mirror to get it to hear me and I like how sleek it is. Would look quite unassuming sitting on top of my mirror.

Thanks for the help
-Jeff

j.e.f.f

Also one more thought: I’m thinking of turning the camera 90 degrees so that it takes a vertical image – better chance of capturing faces of people of different heights that – thinking myself as a tall adult vs my kids, 9 and 6 years old. I’m thinking a 2:3 image (or 9:16 in the case of an HD cam) would really work well here to turn a wide FoV to a “tall” FoV – perfect for people standing right in front of the mirror… If I just turn the camera 90 degrees, and use it to train and capture in that orientation, will the facial recognition algorithms work as-is (i.e.: Can they recognize a face turned sideways) or would the module need to be changed to rotate the capture first before feeding into the algorithm?

reaper81

I have read all posts in this thread. Is there realy no answer why the facerecognation in the testtool is so much better than in the MM-Module?

j.e.f.f

@paviro I had a hard time getting this to play nicely with modules that show and hide themselves. I created a pull-request that addresses this:

https://github.com/paviro/MMM-Facial-Recognition/pull/48

Basically I added a lockString config option to each show and hide command, and when combined with another module that also uses a lockString the particular module is only visible when both have removed their respective lock strings.

See here for details about visibility locking:
https://github.com/MichMich/MagicMirror/tree/develop/modules

cheers!

reaper81

hello everyone,

after update to the new version of the FR-Module i get this error-Massage while starting the MM.

Starting MagicMirror: v2.1.1
Loading config …
Loading module helpers …
No helper found for module: alert.
WARNING! Could not load config file. Starting with default configuration. Error found: Error: Cannot find module ‘node_helper’
Loading module helpers …
No helper found for module: alert.
App threw an error during load
Error: Cannot find module ‘node_helper’
at Module._resolveFilename (module.js:470:15)
at Function.Module._resolveFilename (/home/pi/MagicMirror/node_modules/electron/dist/resources/electron.asar/common/reset-search-paths.js:35:12)
at Function.Module._load (module.js:418:25)
at Module.require (module.js:498:17)
at require (internal/module.js:20:19)
at Object. (/home/pi/MagicMirror/modules/default/updatenotification/node_helper.js:6:18)
at Object. (/home/pi/MagicMirror/modules/default/updatenotification/node_helper.js:89:3)
at Module._compile (module.js:571:32)
at Object.Module._extensions…js (module.js:580:10)
at Module.load (module.js:488:32)
…

Can anybody help me?