MMM-Facial-Recognition and MMM-Assistant don't seem to be working together

A Former User · Nov 16, 2017, 4:01 PM

For MMM-Assistant;

How about not together? Without MMM-Facial-Recognition, does MMM-Assistant work?
Maybe With other Mic-or-speaker-using modules or applications, MMM-Assistant might not work correctly. Sorry, that is out of my ability.
Show me your configuration.

dk7988 · Nov 17, 2017, 2:00 AM

@Sean
1.) The only two places I’ve tried running it alone has been at home and at the school. At home, both ‘MIRROR’ and ‘ASSISTANT’ functions seem to be working but ding.wav and dong.wav don’t play. At the school, only ‘ASSISTANT’ functions seem to be working and the .wav files still don’t play either. If i asked it “what was the score of monday night’s fooball game?” it came back with answers but if i said “smart mirror, list all commands” or “list all modules” it wouldn’t do anything and the symbol (lower left corner) was stuck on the “#” symbol. So, I’m not quite sure how to answer that… i’m going to say kinda?

2.) No worries, thanks for making it! It’s a pretty sweet module! I wish I understood it more or the MM platform in general better I have some ideas I’d like to add to it. I’ll try it with another mic and see what that does. Could you recommend some mic.s you know work?

3.)
Admins, can I get some help with these, please?
Oops I goofed can you delete this one -->[0_1510878576377_MMM-Config.txt](Uploading 100%)
[0_1510879991664_MMM-Config.txt](Uploading 100%)
[0_1510878940046_MMM-Config2.txt](Uploading 100%)
and thank you for the help last time too!

Config for MMM-Facial-Recognition and MMM-Assistant plus all other modules:

/* Magic Mirror Config Sample
*

By Michael Teeuw http://michaelteeuw.nl
MIT Licensed.
For more information how you can configurate this file
See https://github.com/MichMich/MagicMirror#configuration

*/

var config = {
address: “localhost”, // Address to listen on, can be:
// - “localhost”, “127.0.0.1”, “::1” to listen on loopback interface
// - another specific IPv4/6 to listen on a specific interface
// - “”, “0.0.0.0”, “::” to listen on any interface
// Default, when address config is left out, is “localhost”
port: 8080,
ipWhitelist: [“127.0.0.1”, “::ffff:127.0.0.1”, “::1”], // Set [] to allow all IP addresses
// or add a specific IPv4 of 192.168.1.5 :
// [“127.0.0.1”, “::ffff:127.0.0.1”, “::1”, “::ffff:192.168.1.5”],
// or IPv4 range of 192.168.3.0 --> 192.168.3.15 use CIDR format :
// [“127.0.0.1”, “::ffff:127.0.0.1”, “::1”, “::ffff:192.168.3.0/28”],

language: “en”,
timeFormat: 12,
units: “imperial”,

modules: [
{
module: “MMM-Facial-Recognition”,
classes: “default1”,
config: {
// 1=LBPH | 2=Fisher | 3=Eigen
recognitionAlgorithm: 1,
// Threshold for the confidence of a recognized face before it’s considered a
// positive match. Confidence values below this threshold will be considered
// a positive match because the lower the confidence value, or distance, the
// more confident tdefault1he algorithm is that the face was correctly detected.
lbphThreshold: 60,
fisherThreshold: 250,
eigenThreshold: 3000,
// force the use of a usb webcam on raspberry pi (on other platforms this is always true automatically)
useUSBCam: false,
// Path to your training xml
trainingFile: ‘/home/pi/MagicMirror/modules/MMM-Facial-Recognition/training.xml’,
// recognition intervall in seconds (smaller number = faster but CPU intens!)
interval: 0.5,
// Logout delay after last recognition so that a user does not get instantly logged out if he turns away from the mirror for a few seconds
logoutDelay: 20,
// Array with usernames (copy and paste from training script)
users: [‘Ethan’,‘Nick’,‘dakota’, ‘Michelle’,‘Tom’,
‘Rachel’],
//Module set used for strangers and if no user is detected
defaultClass: “default1”,
//Set of modules which should be shown for every user
everyoneClass: “everyone”,
// Boolean to toggle welcomeMessage
welcomeMessage: true
}
},

{
module: ‘MMM-Assistant’,
position: ‘bottom_left’,
classes: “default1”,
config: {
assistant: {
auth: {
keyFilePath: “secret.json”, //When you want to change the location of this file, set this.
savedTokensPath: “resources/tokens.js” //When you want to change the location of this file, set this.
},
audio: {
encodingIn: “LINEAR16”, //Ignore this. I think you don’t need to change this.
sampleRateOut: 16000 //Ignore this. I think you don’t need to change this.
}
},
snowboy: {
models: [
{
file: “resources/smart_mirror.umdl”,
sensitivity: 0.6,
hotwords : “MIRROR” //One of models should be “MIRROR”
},
{
file: “resources/snowboy.umdl”,
sensitivity: 0.6,
hotwords : “ASSISTANT” //One of models should be “ASSISTANT”
}
]
},
record: {
threshold: 0, //Ignore this. I think you don’t need to change this.
verbose:false, //true for checking recording status.
recordProgram: ‘arecord’, //You can use ‘rec’, ‘sox’. But I recommend use ‘arecord’.
silence: 2.0 //Ignore this. I think you don’t need to change this.
},
stt: {
auth: [{
projectId: ‘’, //ProjectId from Google Console
keyFilename: ‘’
}], //You can use multi accounts for saving money.
request: {
encoding: ‘LINEAR16’, //Ignore this. I think you don’t need to change this.
sampleRateHertz: 16000, //Ignore this. I think you don’t need to change this.
languageCode: ‘en-US’ //See https://cloud.google.com/speech/docs/languages
},
},
speak: {
useAlert: true, //If you want to show the text of speech, set this true. But It could be ignored by command of modules directly.
language: ‘en-US’, //If you want to set the default language of speech of command result, set this.
},
alias: [
{
“help :command” : [“teach me :command”, “what is :command”]
}
] // You can use aliases for difficult pronunciation or easy using.
}
},

// {
// module: “everyone”,
// position: “bottom_bar”,
// //Set your classes here seperated by a space.
// //Shown for all users
// classes: “everyone”
// },
// {
// module: “default1”,
// position: “top_bar”,
// //Only shown for default1
// classes: “default1”
// },
//-------------------------------------------------------------------
{
module: “clock”,
position: “top_left”,
classes: “default1”,
},
{
module: “compliments”,
position: “lower_third”,
classes: “default1”,
},
{
module: “currentweather”,
position: “top_right”,
classes: “default1”,
config: {
location: “My City”,
locationID: “My city codeId”, //ID from http://www.openweathermap.org/help/city_list.txt
appid: “My id”
}
},
{
module: “weatherforecast”,
position: “top_right”,
header: “Weather Forecast”,
classes: “default1”,
config: {
location: “My city”,
locationID: “My city codeId”, //ID from http://www.openweathermap.org/help/city_list.txt
appid: “My 2nd Id”
}
},
//-------------------------------------------------------------------
{
module: ‘MMM-Assistant’,
position: ‘bottom_left’,
classes: “Nick”,
config: {
assistant: {
auth: {
keyFilePath: “secret.json”, //When you want to change the location of this file, set this.
savedTokensPath: “resources/tokens.js” //When you want to change the location of this file, set this.
},
audio: {
encodingIn: “LINEAR16”, //Ignore this. I think you don’t need to change this.
sampleRateOut: 16000 //Ignore this. I think you don’t need to change this.
}
},
snowboy: {
models: [
{
file: “resources/hey mirror.pmdl”,
sensitivity: 0.55,
hotwords : “MIRROR” //One of models should be “MIRROR”
},
{
file: “resources/hey google.pmdl”,
sensitivity: 0.55,
hotwords : “ASSISTANT” //One of models should be “ASSISTANT”
}
]
},
record: {
threshold: 0, //Ignore this. I think you don’t need to change this.
verbose:false, //true for checking recording status.
recordProgram: ‘arecord’, //You can use ‘rec’, ‘sox’. But I recommend use ‘arecord’.
silence: 2.0 //Ignore this. I think you don’t need to change this.
},
stt: {
auth: [{
projectId: ‘’, //ProjectId from Google Console
keyFilename: ‘’
}], //You can use multi accounts for saving money.
request: {
encoding: ‘LINEAR16’, //Ignore this. I think you don’t need to change this.
sampleRateHertz: 16000, //Ignore this. I think you don’t need to change this.
languageCode: ‘en-US’ //See https://cloud.google.com/speech/docs/languages
},
},
speak: {
useAlert: true, //If you want to show the text of speech, set this true. But It could be ignored by command of modules directly.
language: ‘en-US’, //If you want to set the default language of speech of command result, set this.
},
alias: [
{
“help :command” : [“teach me :command”, “what is :command”]
}
] // You can use aliases for difficult pronunciation or easy using.
}
},
{
module: “Nick”,
position: “lower_third”,
//Only shown for Nick
classes: “Nick”
},
{
module: “alert”,
classes: “Nick”,
},
{
module: “updatenotification”,
position: “top_bar”,
classes: “Nick”,
},
{
module: “clock”,
position: “top_left”,
classes: “Nick”,
},
{
module: “calendar”,
header: “US Holidays”,
position: “top_left”,
classes: “Nick”,
config: {
calendars: [
{
symbol: "calendar-check-o ",
url: “webcal://www.calendarlabs.com/templates/ical/US-Holidays.ics”
}
]
}
},
{
module: “currentweather”,
position: “top_right”,
classes: “Nick”,
config: {
location: “My City”,
locationID: “My CityId”, //ID from http://www.openweathermap.org/help/city_list.txt
appid: “My id”
}
},
{
module: “weatherforecast”,
position: “top_right”,
header: “Weather Forecast”,
classes: “Nick”,
config: {
location: “My City”,
locationID: “My CityId”, //ID from http://www.openweathermap.org/help/city_list.txt
appid: “My id”
}
},
{
module: “newsfeed”,
position: “bottom_bar”,
classes: “Nick”,
config: {
feeds: [
{
title: “Google News”,
url: “https://news.google.com/output=rss”
},
{
title: “CNN”,
url: “http://rss.cnn.com/rss/cnn_topstories.rss”
},
{
title: “Yahoo News”,
url: “http://news.yahoo.com/rss”
},
{
title: “ESPN”,
url: “http://www.espn.com/espn/rss/news”
},
{
title: “New York Times”,
url: “http://www.nytimes.com/services/xml/rss/nyt/HomePage.xml”
}
],
showSourceTitle: true,
showPublishDate: true
}
},
//-------------------------------------------------------------------
{
module: “clock”,
position: “top_left”,
classes: “Michelle”,
},
{
module: “calendar”,
header: “US Holidays”,
position: “top_left”,
classes: “Michelle”,
config: {
calendars: [
{
symbol: "calendar-check-o ",
url: “webcal://www.calendarlabs.com/templates/ical/US-Holidays.ics”
}
]
}
},
{
module: “currentweather”,
position: “top_right”,
classes: “Michelle”,
config: {
location: “My City”,
locationID: “My CityId”, //ID from http://www.openweathermap.org/help/city_list.txt
appid: “My id”
}
},
{
module: “weatherforecast”,
position: “top_right”,
header: “Weather Forecast”,
classes: “Michelle”,
config: {
location: “My City”,
locationID: “My CityId”, //ID from http://www.openweathermap.org/help/city_list.txt
appid: “My id”
}
},
//-------------------------------------------------------------------
{
module: “clock”,
position: “top_left”,
classes: “Rachel”,
},
{
module: “calendar”,
header: “US Holidays”,
position: “top_left”,
classes: “Rachel”,
config: {
calendars: [
{
symbol: "calendar-check-o ",
url: “webcal://www.calendarlabs.com/templates/ical/US-Holidays.ics”
}
]
}
},
{
module: “currentweather”,
position: “top_right”,
classes: “Rachel”,
config: {
location: “My City”,
locationID: “My CityId”, //ID from http://www.openweathermap.org/help/city_list.txt
appid: “My id”
}
},
{
module: “weatherforecast”,
position: “top_right”,
header: “Weather Forecast”,
classes: “Rachel”,
config: {
location: “My City”,
locationID: “My CityId”, //ID from http://www.openweathermap.org/help/city_list.txt
appid: “My id”
}
},
//-------------------------------------------------------------------

module: “dakota”,
position: “top_bar”,
//Only shown for dakota
classes: “dakota”
},
]

};

/*************** DO NOT EDIT THE LINE BELOW ***************/
if (typeof module !== “undefined”) {module.exports = config;}

Config for MMM-Assistant :

/* Magic Mirror Config Sample
*

By Michael Teeuw http://michaelteeuw.nl
MIT Licensed.
For more information how you can configurate this file
See https://github.com/MichMich/MagicMirror#configuration

*/

var config = {
address: “localhost”, // Address to listen on, can be:
// - “localhost”, “127.0.0.1”, “::1” to listen on loopback interface
// - another specific IPv4/6 to listen on a specific interface
// - “”, “0.0.0.0”, “::” to listen on any interface
// Default, when address config is left out, is “localhost”
port: 8080,
ipWhitelist: [“127.0.0.1”, “::ffff:127.0.0.1”, “::1”], // Set [] to allow all IP addresses
// or add a specific IPv4 of 192.168.1.5 :
// [“127.0.0.1”, “::ffff:127.0.0.1”, “::1”, “::ffff:192.168.1.5”],
// or IPv4 range of 192.168.3.0 --> 192.168.3.15 use CIDR format :
// [“127.0.0.1”, “::ffff:127.0.0.1”, “::1”, “::ffff:192.168.3.0/28”],

language: “en”,
timeFormat: 12,
units: “imperial”,

modules: [
{
module: ‘MMM-Assistant’,
position: ‘bottom_left’,
config: {
assistant: {
auth: {
keyFilePath: “secret.json”, //When you want to change the location of this file, set this.
savedTokensPath: “resources/tokens.js” //When you want to change the location of this file, set this.
},
audio: {
encodingIn: “LINEAR16”, //Ignore this. I think you don’t need to change this.
sampleRateOut: 16000 //Ignore this. I think you don’t need to change this.
}
},
snowboy: {
models: [
{
file: “resources/smart_mirror.umdl”,
sensitivity: 0.6,
hotwords : “MIRROR” //One of models should be “MIRROR”
},
{
file: “resources/snowboy.umdl”,
sensitivity: 0.6,
hotwords : “ASSISTANT” //One of models should be “ASSISTANT”
}
]
},
record: {
threshold: 0, //Ignore this. I think you don’t need to change this.
verbose:false, //true for checking recording status.
recordProgram: ‘arecord’, //You can use ‘rec’, ‘sox’. But I recommend use ‘arecord’.
silence: 2.0 //Ignore this. I think you don’t need to change this.
},
stt: {
auth: [{
projectId: ‘’, //ProjectId from Google Console
keyFilename: ‘’
}], //You can use multi accounts for saving money.
request: {
encoding: ‘LINEAR16’, //Ignore this. I think you don’t need to change this.
sampleRateHertz: 16000, //Ignore this. I think you don’t need to change this.
languageCode: ‘en-US’ //See https://cloud.google.com/speech/docs/languages
},
},
speak: {
useAlert: true, //If you want to show the text of speech, set this true. But It could be ignored by command of modules directly.
language: ‘en-US’, //If you want to set the default language of speech of command result, set this.
},
alias: [
{
“help :command” : [“teach me :command”, “what is :command”]
}
] // You can use aliases for difficult pronunciation or easy using.
}
},

]

};

/*************** DO NOT EDIT THE LINE BELOW ***************/
if (typeof module !== “undefined”) {module.exports = config;}

4.) Off topic, but as I was going through the files for MMM-Assistant and noticed a file called ISS something (can’t remember the full name) so i clicked on it and started looking at some of the code… is the stuff in that folder for the assistant or something else?

A Former User · Nov 17, 2017, 2:00 AM

@dk7988
I cannot figure out some ISSblah things.

Ok. Lets check somthings.

Have you tried npm start dev? It can tell you something wrong or not in frontend client side (You can also see termianl log for server side). Maybe some error might be found, or even any helpful messages.
There is some possibility of ‘node-aplay’ was not installed properly. Remove it and reinstall by npm install node-aplay in your MMM-Assistant directory.
inactivated ‘command mode’ could also have some dependency problem. I think you have already tried reinstalling whole module many times, but check it again.

dk7988 · Nov 18, 2017, 10:44 PM

@Sean
Aww… i see that sucks (talking about the ISS blah things)

Yay! I have news! haha
Good first: I took it home turned it on and didn’t really change anything (except I added some more classes to some of the modules in the config file) and everything started working both fac.recon and the assistant (both snowboy and smart mirror were working).

The Bad: right now i’m at the school trying the same thing that worked at home and fac.recon and half of the assistant is working (smart mirror still will not work, snowboy works fine) still getting the same error. I also tried running just the assistant and got the same thing.

After reading the error again “Check your iat and exp values and use a clock with skew to account for clock differences between systems.” is popping out to me. Do you know how i might be able to check my iat and exp values? actually what are “iat” and “exp” values? and do you know how i might be able to use a “clock with skew to account for clock differences between systems”?

1.) I ran npm start dev and the only thing i could see that was wrong was in the ‘console’ the last line it said [ASSTNT] Error: RECOGNIZESTREAM VM93 MMM-Assistant.js:412 am I looking in the right place? I tried researching to figure out how to find the terminal log but i had no luck. Do you have any ideas how i can find it?

2.) I tried running npm install node-aplay in the assistant folder and got this
MMM-TelegramBot@1.0.0 /home/pi/MagicMirror/modules/MMM-Assistant
└── node-aplay@1.0.3

npm WARN MMM-TelegramBot@1.0.0 No repository field.
npm WARN MMM-TelegramBot@1.0.0 No license field.

I wasn’t sure what the WARN was trying to tell me (I assumed it wasn’t good) so i also ran sudo npm install node-aplay and got the same thing. Either way ding and dong still don’t play.

3.) I think some how it’s the schools internet cause last night and Thursday night everything (including the clock) worked perfectly at home. Now the clock is back to being 12+ hours off and smart mirror won’t work. Any thoughts on the clock and smart mirror problem being related?

Thanks again for all your help!

dk7988 · Nov 18, 2017, 11:06 PM

@dk7988
Found this,this, and this on stackoverflow. I’m still going through it to see if/how it applies to me but though it might be worth throwing up here.

A Former User · Nov 19, 2017, 8:14 PM

Well, your system clock could cause that kind of error, because api might carry the time information for auth between two systems.
But I’m not so expert to declare it. :) but at least, auth token is related with time for expiry of life.

All I can suggest is fixing your clock first. Maybe it might not be a real reason, but you could take one more thing works fine.

May the luck with you.

dk7988 · Nov 19, 2017, 8:14 PM

@Sean
haha thanks! I tried it first thing when i got home from school last night and everything worked perfectly, the clock was spot on! I’m going to call the fix to my problem being the school’s internet. At my school they have something on all the computers that freezes everything on the back to a specific time so i’m thinking that functionally maybe part of my problem… I don’t know but it works at home and that’s mostly what I care about. When I turn in the project I’ll just put in my write up that the school’s internet is messing with the security portion of the smart mirror and hope my teacher doesn’t ding me to much :/