MMM-AssistantMk2

jomwells

@sean Ah I see, its not essential at all! one minor bug I seem to have, that I have noticed does not occur for you in your demonstration video is that now, having finished the audio reply, there is often a very short sound, it sounds like the last half a second of text is repeated, sometimes it is more of a ‘buzz’, I attempted changing the encodingOut setting to one of the other options (MP3 or OPUS_IN_OGG) but both of these just give white noise (like an old tv with no signal) in place of any audio, is this something you have come across at all, and is there anything you can suggest to fix it? maybe getting the alternative encoding to work? thanks

A Former User

@jomwells
I’m not sure.
Voice output is just buffered stream, so just piping it to the speaker is all of mechanism. So at the moment I have no idea why last data is duplicated(?) in your environment.
Voice output stream comes chunked, so there is some possibility of the time gap between incoming stream chunks.

Maybe instead piping Linear16 to speaker directly, writing response stream to mp3(or ogg) file then playing that file could be one of solutions.(sorry, I didn’t implement that)

A Former User

@jomwells
For the Speaker, research this.
https://github.com/TooTallNate/node-speaker

Sun

@jomwells Mine is a little bit different as I use the hdmi on tv as speaker as well. the default device fall on card 0 with alsa. For microphone is usb on card 1. I don’t have problem with microphone but I did have problem with sound via hdmi. Sometime sound just come out right through the hdmi but with lag sound, sometime no sound at all and say device busy. Also, to avoid hdmi device busy, I use usb sound card instead of hdmi sound also very lag.

pcm.!default {
type asym
playback.pcm
{
type hw
card 0
device 1
}
capture.pcm
{
type hw
card 1
device 0
}
}

Sun

@sean How to change the piping Linear 16 to ogg. Can I just put ogg in the

encodingOut: “OPUS_IN_OGG”,

A Former User

@sun
No, mp3 and Ogg should be written to file or piping to proper player, frankly I didn’t implement those function, because I thought nobody needs.
I’ll try those till next update.

jomwells

Thank you, I will look into piping the audio, but its unlikely I will get very far :)

Question: is it possible to run terminal commands at the different stages of MMM-AssistantMk2, such as running a command to turn on a GPIO pin (using the built in ‘GPIO write X 1’ ) when the module begins listening, and turning it off again when it is understanding, or speaking perhaps? the use case for which would be a status LED on the mirror. Thanks again.

A Former User

@jomwells
You can fork my repository to yours and modify it.
When you want to relay status to other modules, add .sendNotification() in ’socketNotificationReceived()’ of AssistantMk2.js where proper event is located.
When you want to do something directly, add your job in ’Conversation.on(EVENT)’ of node_helper.js.

Sun

What else do I need to do for install for mmm-assistantmk2 to play youtube.
I try the same command say Celine Dion The power of love. Then pop up below detail about Celine Dion and the youtube video automatically pop up but cannot play anything.
My magic mirror is vertical mode, rotate = 1 . I try to adjust from your height 480 width 854 of display but it also show in vertical mode height 480 with 640 also height 640 width 480. No different.

A Former User

@sun

Some youtube videos are not allowed for embedded player.
some videos are not caught, it will be fixed.
some videos are not only video but playlist, so it is not caught, but it will be fixed also.
sometime Google gives no information about youtube. I guess smart tv is using not only YouTube but also other dedicated apps for music and video, yogether

JRCelis

@jomwells

I get the same issue, I hear a buzz at the beginning of the answered sentences in my speaker and clear voice after two seconds of the sentences start.

A Former User

@jrcelis
What kind of speaker do you use? Analog? HDMI output? USB? Bluetooth?
Except for Bluetooth speaker, All others could have buzz sound.

JRCelis

@sean
Hello @Sean and thanks for your help. I’m using the speaker that is coming with Google Voice Kit V1 and is working fine when I’m testing the sound, no buzz or distorted sound and the beginning of the sentences, the issue starts when I click the speaker icon when MMM-assistantk2 is running, the assistants answer start whit a big distorted noise o buzz and then will be fine.

A Former User

@jrcelis
Unfortunately, I’m not an expert in sound configuration.
I use node module speaker for output. The buffered voice output from Google Assistant Server is directly piped to speaker module. And there is narrowly little margin to control speaker module.

My guess is, burden of system could make some delay causing distortion. but just guessing.

Anyway, I’m considering to convert voice output to mp3 file then play it. but it needs somewhat computing power, so I’m not sure whether it will be really good for RPI.
Or I will search other solution for better result.

mariecbilodeau

Hello! I got Hotword working but assistant triggers error. I tried to rebuild electron as explained in the installation guide but it’s not working. I have the google voice kit installed on my raspberry… do you think that is the problem?

A Former User

@mariecbilodeau more details are needed to answer. Error logs or screenshots could be helpful.

mariecbilodeau

this is when I load my mirror with just hotword and assistant installed. I’ll try to rebuild electron again to get the log

> magicmirror@2.4.1 start /home/pi/MagicMirror
> sh run-start.sh

Starting MagicMirror: v2.4.1
Loading config ...
Loading module helpers ...
Initializing new module helper ...
Module helper loaded: MMM-Hotword
WARNING! Could not load config file. Starting with default configuration. Error found: Error: Failed to load gRPC binary module because it was not installed for the current system
Expected directory: electron-v2.0-linux-arm-glibc
Found: [node-v59-linux-arm-glibc]
This problem can often be fixed by running "npm rebuild" on the current system
Original error: Cannot find module '/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/node/extension_binary/electron-v2.0-linux-arm-glibc/grpc_node.node'
Loading module helpers ...
Initializing new module helper ...
Module helper loaded: MMM-Hotword
App threw an error during load
Error: Failed to load gRPC binary module because it was not installed for the current system
Expected directory: electron-v2.0-linux-arm-glibc
Found: [node-v59-linux-arm-glibc]
This problem can often be fixed by running "npm rebuild" on the current system
Original error: Cannot find module '/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/node/extension_binary/electron-v2.0-linux-arm-glibc/grpc_node.node'
    at Object.<anonymous> (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/grpc_extension.js:53:17)
    at Object.<anonymous> (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/grpc_extension.js:63:3)
    at Module._compile (module.js:642:30)
    at Object.Module._extensions..js (module.js:653:10)
    at Module.load (module.js:561:32)
    at tryModuleLoad (module.js:504:12)
    at Function.Module._load (module.js:496:3)
    at Module.require (module.js:586:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client_interceptors.js:145:12)
Whoops! There was an uncaught exception...
{ Error: Failed to load gRPC binary module because it was not installed for the current system
Expected directory: electron-v2.0-linux-arm-glibc
Found: [node-v59-linux-arm-glibc]
This problem can often be fixed by running "npm rebuild" on the current system
Original error: Cannot find module '/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/node/extension_binary/electron-v2.0-linux-arm-glibc/grpc_node.node'
    at Object.<anonymous> (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/grpc_extension.js:53:17)
    at Object.<anonymous> (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/grpc_extension.js:63:3)
    at Module._compile (module.js:642:30)
    at Object.Module._extensions..js (module.js:653:10)
    at Module.load (module.js:561:32)
    at tryModuleLoad (module.js:504:12)
    at Function.Module._load (module.js:496:3)
    at Module.require (module.js:586:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client_interceptors.js:145:12) code: 'MODULE_NOT_FOUND' }
MagicMirror will not quit, but it might be a good idea to check why this happened. Maybe no internet connection?
If you think this really is an issue, please open an issue on GitHub: https://github.com/MichMich/MagicMirror/issues
Launching application.
pi@raspberrypi:~/MagicMirror $

mariecbilodeau

this is part of my rebuild error

pi@raspberrypi:~/MagicMirror/modules/MMM-AssistantMk2 $ npm install --save-dev electron-rebuild
npm WARN MMM-AssistantMk2@1.0.1 No repository field.

+ electron-rebuild@1.8.2
updated 1 package in 18.167s
pi@raspberrypi:~/MagicMirror/modules/MMM-AssistantMk2 $ ./node_modules/.bin/electron-rebuild
✖ Rebuild Failed

An unhandled error occurred inside electron-rebuild
gyp info it worked if it ends with ok
gyp info using node-gyp@3.8.0
gyp info using node@9.11.2 | linux | arm
gyp info spawn /usr/bin/python2
gyp info spawn args [ '/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/pi/.electron-gyp/.node-gyp/iojs-2.0.0/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/home/pi/.electron-gyp/.node-gyp/iojs-2.0.0',
gyp info spawn args   '-Dnode_gyp_dir=/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/home/pi/.electron-gyp/.node-gyp/iojs-2.0.0/<(target_arch)/iojs.lib',
gyp info spawn args   '-Dmodule_root_dir=/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.' ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make : on entre dans le répertoire « /home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/build »
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/surface/init.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/avl/avl.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/backoff/backoff.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/channel/channel_args.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/channel/channel_stack.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/channel/channel_stack_builder.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/channel/channel_trace.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/channel/channelz.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/channel/channelz_registry.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/channel/connected_channel.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/channel/handshaker.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/channel/handshaker_factory.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/channel/handshaker_registry.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/channel/status_util.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/compression/compression.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/compression/compression_internal.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/compression/message_compress.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/compression/stream_compression.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/compression/stream_compression_gzip.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/compression/stream_compression_identity.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/debug/stats.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/debug/stats_data.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/http/format_request.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/http/httpcli.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/http/parser.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/call_combiner.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/combiner.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/endpoint.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/endpoint_pair_posix.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/endpoint_pair_uv.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/endpoint_pair_windows.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/error.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/ev_epoll1_linux.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/ev_epollex_linux.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/ev_epollsig_linux.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/ev_poll_posix.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/ev_posix.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/ev_windows.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/exec_ctx.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/executor.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/fork_posix.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/fork_windows.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/gethostname_fallback.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/gethostname_host_name_max.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/gethostname_sysconf.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/iocp_windows.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/iomgr.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/iomgr_custom.o
  CXX(target) Release/obj.target/grpc/deps/grpc/src/core/lib/iomgr/iomgr_internal.o

A Former User

@mariecbilodeau
“gyp ERR …” lines are needed. Your log is missing those infos.

themc

is there any way to play local stored mp3 files on the raspbery pi using the play … command

currently when saying prhases like play mp3 it says oke and stops.

im wondering if it is possible to play music mp3s from the music folder using the built in player of vlc player by sying the google command of a gaction

would be a nice addition to the plugin