Yet Another Voice Module (Sphinx + Alexa)

whyjustin

Hey all,

I’m happy to be making my first post to this community. I discovered it recently when embarking upon making a Magic Mirror and am impressed by the active support and discussion that occurs. Not sure if this post belongs in Development or Showcase as the module is mostly done, but I just had a daughter and may not be able to provide initial support for it so wanted to do a soft release here first.

I know there are existing voice modules for MM2 but I found my use case was to use Alexa alongside Sphinx, providing all the voice recognition functionality within one module. This prevents different modules competing for resources (microphone) and having to synchronize the allocation and deallocation of these resources. The hope is that non-audio related functionality will be handled by other modules, there is a not yet documented command with action “sendNotification” that will allow configuring certain voice commands to invoke a process in a different module.

Another goal for this project was to eliminate the need for lengthy configuration and setup. The module prefers convention to configuration, and most of the nerdy voice recognition and Alexa setup is bootstrapped for the user during the load. This allows the user to have a simple json configuration and the module will generate the appropriate ssh keys and language models/dictionaries required for the module to work.

Enough talk, here is the module. Pull requests warmly welcome, around documentation or functionality. Thanks.

https://github.com/whyjustin/magic-mirror-voice

cowboysdude

Sounds great… what is the response time like?

whyjustin

Sounds great… what is the response time like?

Response time of what? Loading time is pretty slow given the bootstrapping, but that should only happen once. The Sphinx client actively listens for commands as set in the JSON sphinx:commands dictionary and processes speech every time you pause. Alexa will record until you are quiet for over 2 seconds and then send it off the Amazon Voice Services. Guessing this response is similar to an Echo but I’ve never owned one.

cowboysdude

I own and Echo and it’s pretty quick :)

d3r

That’s pretty cool.
I have amazon alexa running alongside the MM2 on my pi too. Mine is just a simple work around and I have to click(I use a button) start listening each time I want to interact with it.

So it its always listening in your version?

whyjustin

So it its always listening in your version?

Yup, Sphinx is always listening (as long as Alexa is not listening). You can configure the command that invokes Alexa in the sphinx:commands dictionary. Any key in the object will be recognized by Sphinx and perform the action as configured by the value. For example:

"mirror mirror": {
  action: 'alexa'
}

Saying “mirror mirror” will trigger Alexa.

Note from admin: Please use Markdown on code snippets for easier reading!

d3r

I’ll give this a try when I get some free time this weekend.

cowboysdude

sed -i ‘s/“electron-prebuilt”: “latest”/“electron-prebuilt”: “1.2.8”/g’ package.json
sed: can’t read package.json: No such file or directory

whyjustin

sed -i ‘s/“electron-prebuilt”: “latest”/“electron-prebuilt”: “1.2.8”/g’ package.json
sed: can’t read package.json: No such file or directory

Answered in the GitHub issue https://github.com/whyjustin/magic-mirror-voice/issues/1

d3r

If I already have the alexa voice service set up on the pi is there an ‘easier’ way of setting this up or would it be better to start from scratch?
I’ll of course back up my sd card before playing with this.

whyjustin

If I already have the alexa voice service set up on the pi is there an ‘easier’ way of setting this up or would it be better to start from scratch?
I’ll of course back up my sd card before playing with this.

If you already have Alexa set up, then you should have all the required bits. Just update the configuration with your productId, dsn, clientId, clientSecret, alpnVersion, vlcPath, vlcPluginPath (all of which you acquired when setting up Alexa). For first run, I’d try to do serveronly mode so you don’t have to mess with the electron node version disparity.

node serveronly

then open a browser on the pi @ http://localhost:8080. If this works, then follow the Installation Notes.

d3r

@whyjustin said in Yet Another Voice Module (Sphinx + Alexa):

If I already have the alexa voice service set up on the pi is there an ‘easier’ way of setting this up or would it be better to start from scratch?
I’ll of course back up my sd card before playing with this.

If you already have Alexa set up, then you should have all the required bits. Just update the configuration with your productId, dsn, clientId, clientSecret, alpnVersion, vlcPath, vlcPluginPath (all of which you acquired when setting up Alexa). For first run, I’d try to do serveronly mode so you don’t have to mess with the electron node version disparity.

node serveronly

then open a browser on the pi @ http://localhost:8080. If this works, then follow the Installation Notes.
Thank you, I got till that part and the modules are currently loading. Your module and 2 others haven’t loaded yet. How long should it usually take?
I am using the firefox iceweasel browser.
Edit: I reloaded the page and not all my modules are loaded except the voice one. Which makes sense because I haven’t run the npm install from the module directory yet. Will do that and post back.

d3r

@whyjustin
I was able to set up the module but I think I did a few things wrong. I think I have the vlcPluginpath wrong in my configuration. I am not sure where it is located. I believe it is here "vlcPluginPath: ‘/usr/lib/vlc/plugins’ " but I am not sure what exacly I should be looking for. When the module loads, there is currently nothing under the module. If I reload the page it switches to “loading…” and is stuck on “loading…”
Also , once I run node serveronly and open up the localhhost page the voice module does start to load but I get the following error:
`
Listening for transport dt_socket at address: 5005
Unable to run Alexa Java Client: Error: ENOENT: no such file or directory, open './magic-mirror-voice/alexa-certificate-generator/certs/server/node.key

I went to that directory and the files do exist. I used the script to generate them. My understanding is sphinx is working but there’s something wrong with the alexa part. I’m not really sure what I should do next. I’ll keep playing with it, hopefully I’ll get it to work.

Full log here:

whyjustin

I went to that directory and the files do exist. I used the script to generate them. My understanding is sphinx is working but there’s something wrong with the alexa part.

From the logs this seems to be true. Thanks for the log, it makes debugging much easier. As mentioned in OP, my life is pretty hectic right now. I’m hoping to have some chance to look into this over the next week but may not be able to provide the optimal feedback cycle for you. Thanks for the patience.

whyjustin

@d3r said in Yet Another Voice Module (Sphinx + Alexa):

Unable to run Alexa Java Client: Error: ENOENT: no such file or directory, open './magic-mirror-voice/alexa-certificate-generator/certs/server/node.key

Do you have the optional sslKey entry in the config? If you are autogenerating all the keys, I would remove all the //optional values from the ssl configuration. May be a red herring but all I can think of w/o my computer.

d3r

@whyjustin Thank you. I understand you’re pretty busy with your familly so no rush.

You were correct about the optional ssl. I commented it out and was able to get to the alexa registering part
Register Alexa Java Client by navigating to https://localhost:5555/provision/143d3e3b67c6577291228ff9
Now the problem is that the module is not registering. I cannot connect to that link if I put it in a browser. Also, I think it should be localhost:3000/provision/... because that’s how the alexa standalone app registration works but I could be wrong.
I played around with the config.
port: 5555,
serviceUrl: ‘https://localhost:5555’, // if I change this to port 3000 I get an error /
redirectUrl: ‘https://localhost:3000/authresponse’, // changing this port still returns https://localhost:5555/provision/143d3e3b67c6577291228ff

Again no rush, this can wait.

whyjustin

Now the problem is that the module is not registering. I cannot connect to that link if I put it in a browser.

This is a long shot since you’ve setup Alexa already, but are you using a browser on the pi to connect to the site? If not you would have to replace localhost with the pi’s IP/Hostname

Also, I think it should be localhost:3000/provision/… because that’s how the alexa standalone app registration works but I could be wrong.

The oauth service can run on any port, I believe that 3000 may been taken by another process on my raspberry pi (either by Magic Mirror or something else) which is I why I used 5555. I think there is some redundancy in the three options (port/serviceUrl/redirectUrl) but it is required to keep the same port and hostname on all three. If you choose to keep 5555, you will have to update the Alexa settings that you previously set up to indicate the proper port:

https://github.com/alexa/alexa-avs-raspberry-pi/blob/master/assets/avs-web-settings-filled.png

d3r

@whyjustin said in Yet Another Voice Module (Sphinx + Alexa):

The oauth service can run on any port, I believe that 3000 may been taken by another process on my raspberry pi (either by Magic Mirror or something else) which is I why I used 5555. I think there is some redundancy in the three options (port/serviceUrl/redirectUrl) but it is required to keep the same port and hostname on all three. If you choose to keep 5555, you will have to update the Alexa settings that you previously set up to indicate the proper port:
https://github.com/alexa/alexa-avs-raspberry-pi/blob/master/assets/avs-web-settings-filled.png

Thank you.
Changed the port on the amazon website and I got it to work.
Your module is awesome! This is really good.

whyjustin

@d3r said in Yet Another Voice Module (Sphinx + Alexa):

Your module is awesome! This is really good.

Thanks, good to hear it’s working. Let me know if you have any other difficulties or feature requests. As mentioned to @cowboysdude, it is easiest for me to track these through GitHub Issues. Cheers.

clumsy_ninja

@whyjustin What kind of visual feedback does it display on the mirror?