Creating Custom Voice Commands for Hello-Lucy...?

sdetweil

@Doctor0ctoroc its builds the library from the sentences and words files…
then calls lmtool to generate the lm & dic files

Doctor0ctoroc

@sdetweil So are the .lm and .dic files generated in real time? Like, does whatever is added to the words.json and sentences.json files propagate into the .lm and .dic files or are you saying that both the words and sentences files are the basis for generating the .lm and .dic files through the sphinx tool?

sdetweil

@Doctor0ctoroc on module startup

sdetweil

@Doctor0ctoroc module sends a message to node_helper “START”
and then u can read the code in node_helper

Doctor0ctoroc

@sdetweil Ah, that’s fantastic. That would explain why when I changed “Hello Lucy” to “Hey Jarvis” in the Hello-Lucy.js and config file, it was added to the .dic and .lm file…I thought it was included in there from the get go (assuming the code was written to include an alternative, ‘familiar’ AI name that users might want) but all this time, it was my change of the code that put it in there - and it totally works when I say “Hey Jarvis” instead of “Hello Lucy”!

So there’s no need to even edit the .lm or .dic files directly then?

sdetweil

@Doctor0ctoroc no, they are generated each time the module starts

Mykle1

@Doctor0ctoroc

I think, for your purposes, you can simply do this, although I have not tested it:

In sentences.js file add your command(s). “CHEER ME UP” and “THANKS I FEEL BETTER” (MUST ALL BE CAPS)./
In checkCommands.json Sample -->

["SHOW","COMMAND","","","true","MMM-ModuleName",""],
["HIDE","COMMAND","","","false","MMM-ModuleName",""],

Changed to

["CHEER","ME","UP","","true","compliments",""],
["THANKS","I","FEEL","BETTER","false","compliments",""],

This should now make sense to you. I used the hide/show pairs mostly because I found the success rate higher when using shorter commands. I’ve “learned” how to talk to Lucy so my success rate is pretty darn good. I know others that have struggled for success.

Using what @sdetweil has told you and this simple format above, you should be on your way.

Mykle1

Ok, I just tested this and it works. You may find some commands work better than others. You’ll have to do some experimenting.

Mykle1

@Doctor0ctoroc said in Creating Custom Voice Commands for Hello-Lucy...?:

when I changed “Hello Lucy” to “Hey Jarvis” in the Hello-Lucy.js

You would only need to change it in the config entry

Doctor0ctoroc

@Mykle1 said in Creating Custom Voice Commands for Hello-Lucy...?:

@Doctor0ctoroc said in Creating Custom Voice Commands for Hello-Lucy...?:

when I changed “Hello Lucy” to “Hey Jarvis” in the Hello-Lucy.js

You would only need to change it in the config entry

Ah, good to know so I’m not being redundant. And thanks for the direction on the coding, that helps a lot! Do I need to add the extra words to the words.json file as well?

Mykle1

@Doctor0ctoroc said in Creating Custom Voice Commands for Hello-Lucy...?:

Do I need to add the extra words to the words.json file as well?

No sir. Re-read the instructions in my post above

Doctor0ctoroc

@Mykle1 Okay, got it to work! Although it seems when I say “cheer me up” it will 25% of the time pull up the help menu haha. Is there a function that pulls up help if it can’t ‘understand’ the input? I can’t imagine it’s reading my voice command as “show help”!

Mykle1

@Doctor0ctoroc

‘False Positives’ do occur. That’s why I told you that’ some commands work better than others, you’ll have to experiment and Lucy will never be as accurate as an Amazon Echo or Google Home.’ You wanted custom commands. The longer they are (4 words) the higher the risk of false positives. Also, the Pi is hard put to handle Lucy. That’s the reason I moved on to laptop boards. Fantastic response time with non-usb microphone (3.5mm) and higher accuracy.

Doctor0ctoroc

@Mykle1 Fair enough! Out of curiosity, if I were to remove all the commands that I know I won’t use, would that improve the performance by limiting the number of possible commands or is the bulk of the delay just a result of the Pi’s limited processing power? For my mic, I’ve actually been using the Respeaker 2-mic Pi Hat. Not sure how much more responsive that is than a USB mic but it definitely picks up my voice better than the $6 USB mic most people get on Amazon. With a breakout ribbon cable I can place it anywhere I want up to a foot or so from the Pi itself so it doesn’t have to seat directly on top!

sdetweil

@Doctor0ctoroc no, the sphinx lib is just not that good in my opinion. i stopped using as i am spoiled by google speech reco

i warned about this back at the beginning

Mykle1

@Doctor0ctoroc said in Creating Custom Voice Commands for Hello-Lucy...?:

I’ve actually been using the Respeaker 2-mic Pi Hat

I’ve never used one but I can tell you that USB microphones slow Lucy response times considerably, even on a quad core i7. I’ve tested this. USB microphones (several) introduced too much lag for my liking. My guess is how the audio is processed through USB. The same machine with a common 3.5mm jack microphone (several) makes Lucy fly.

I am curious. What sort of response time do you get with the Pi Hat microphone, from command to response?

Doctor0ctoroc

@sdetweil I’ve made multiple attempts to incorporate the Google API (both with voice rec and photos) but it seems they made a number of changes to the way things work earlier this year so all of the existing tutorials from as recent as late 2019 are harder to follow and I haven’t been able to sort it out with my limited knowledge (and my limited I mean I have never attempted anything using the API before so I am guessing half the time). I would prefer Google Assistant but I’ll settle for module control with some delay as I don’t think my girlfriend who I’m building this for will use it for much aside from switching between preset pages and I can customize it to her liking so she doesn’t need to interact with it much. She’ll definitely get a kick out of calling it JARVIS though! I’m not going to lie, I probably would have been better off just getting a Chromecast so she can put whatever she wants up on it from her phone (and the ‘screensaver’ for the 'cast has time and weather) but I wanted to be able to customize all the colors, layout, etc so I went with the Pi and MM2. It’s decision I’m only half regretting…

@Mykle1 It’s not great. What’s more is it varies from reboot to reboot of the Pi. Sometimes it seems to take a second at most, other times it takes the amount of time it takes me to feel the need to give the command again, about 3 or 4 seconds. Not sure why the discrepancy but if I can boot it to work on the faster end at any point then I’m fine with it since it’ll just be left on the wall at that point. Would be nice to figure out why it goes slower sometimes and faster others though. Unfortunately, there is limited documentation for the hat I’m using. I originally got it because it has a GPIO through port that’s Grove compatible and I picked up a Grove PIR sensor but I’m struggling to get that to work and there is no documentation on using these components together so as with everything else in this build, I’m just guessing half the time.

sdetweil

@Doctor0ctoroc the Google apis are none too easy thats for sure.

sdetweil

@sdetweil there IS a new MMM-GoogleAssistant module, which uses the google voice reco, which supports user defined commands (change page)

Doctor0ctoroc

@sdetweil I may have to look into that. For now I’ll have to settle since I’m on a time crunch to finish the mirror by the 18th because it just wouldn’t be the same without the mirror starting up with a ‘happy birthday’ message in the compliments module on her actual birthday. Part of the gift (that keeps on giving) is that I’ll have all the time in the world to add to the mirror down the road.