MMM-AssistantMk2

A Former User

@Cr4z33
Well, now it is on development and not ready to release so details could be changed.
But it might have these features;

No need to wait after hotword. You could say “mirror, blahblah” instead “mirror (pause and wait activation) blahblah.”
Some annoying dependency would be removed. HTML5 default audio output would be used, so you don’t be bothered by sound output configuration. If your web browser can play some audio, it would be enough
Installer will be provided.
Recorded wav file by user or other module could be used for your query. (So User can query via 3 ways - text, real-time vocal query, pre-recorded vocal query)
Annoying center region issue would be gone.
ScreenOutput would be more controllable (duration time, size, activation inner links, suggested commands, …)
- The core informations of screenOutput would be extracted for later using. (e.g: external link to open a broswer for getting additional information. it could be transferable to other module for later using.)
Creating and managing custom action would be easier.
- So I’m considering to remove transcriptionHook trick to use action instead.
- I served transcriptionHook for making simple voice command, but that is not a natural way to use Google Assistant. In many cases, Ppl tried to make commands which are not related with google assistant directly. Rather those kinds of commands could be achieved by just simple voice detection without AI. So I now add that feature to MMM-Hotword and being condering to drop out from MMM-AssistantMk2
YouTube features will be removed. (And Spotify also)
- Instead, YouTube, Spotify, Web Browsing, Map Direcions, Image search would be separated as independent module. those modules will use the info which is extracted from result of query. Of course, each module could be used standalone without AMK2. I’ll make it one-by-one. (Or whoever has interests could)

Cr4z33

@Sean wow that’s a BIG change in the module! 😲

Can’t wait to start trying it out mate to use all the good stuff you are adding in it.

Thanks for letting us know and have a nice Sunday. 😁

saljoke

Wonderful. I want to try these changes!!!

smarthome

Hello
How can I stop Assistant’s response and make request again like Google Home Speaker can ?

smarthome

This is my config, still don’t work yet

{
    module: "MMM-AssistantMk2",
    .....

      notifications: {
    ...
        ASSISTANT_UNDERSTOOD: "HOTWORD_RESUME",
      },
    }
  },
  {
  module: "MMM-Hotword",
  ...
    models: [
     ...
      {
        hotwords    : "stop",
        file        : "stop.pmdl",
        sensitivity : "0.5",
      },
    ],
    command: {
     ...
      "stop":{
      notificationExec: {
        notification: "ASSISTANT_CLEAR",
        payload: (detected, afterRecord) => {
          return {profile:"default"}
        }
      },
      afterRecordLimit:0,
      restart:false,
    },
  },
},

A Former User

@smarthome Sorry, at this moment, there is no way to interrupt response. Just wait ends.

smarthome

@Sean said in MMM-AssistantMk2:

@smarthome Sorry, at this moment, there is no way to interrupt response. Just wait ends.

Tks for your quick answer! Can we make a command exec to mpg21 to stop blayback google response?

smarthome

And other question is, if I ask a command that Google can not find the suitable answer, there’s no response while google home speaker give a promise answer? Is there any configure to do that?

A Former User

@smarthome

Can we make a command exec to mpg21 to stop blayback google response?

Ideally, might be possible. But not only playing sound is the issue related. So, currently not so easy.

And other question is, if I ask a command that Google can not find the suitable answer, there’s no response while google home speaker give a promise answer? Is there any configure to do that?

Currently, No.

Anyway, I’m building a new updates. It might have possibility of solving your issue. I’m planning to remove dependencies about sound out. It might become more controllable about voice response.
But due to my vacation, and new RPI4, Raspiban Buster and other many things which I should concern besides make its release somewhat lately. sorry.

smarthome

This post is deleted!

smarthome

@Sean
How can I remove “Repeat after me” whenever ask ASSISTANT _SAY to speech text?

A Former User

@smarthome Impossible. ASSISTANT_SAY is not perfect solution to make mirror saying something. At any time it could be changed by Google how it works. When I implement that feature at first time, there is no additional “Repeat after me” spoken phrase, but Google changed it’s working way. So I didn’t recommend using it. Use real TTS solution to make your mirror say something.

smarthome

@Sean
I’ also active billing account to enable Google Cloud text to Speech API to use your MMM Google-TTS Module
But, this can not speak my local language (Vietnamese) like ASSISTANT_SAY can
How to do that?

A Former User

@smarthome
Sorry, Vietnamese is not supported (https://cloud.google.com/text-to-speech/docs/voices)

smarthome

@Sean
After your answer, I’m looking for google cloud amd find out the way to make it support, hi
Not only Vietnamese language, a wide range of local language, like assistant language
That’ now MMM Google TTS work ok with vietnamese language
Tks!

t.architecture99

How can google TTS respond to the answers I have prepared? eg :

“smart mirror, I want to drink something.” - Me
“This is your drink.” - Smart mirror answers.
So how to do that?
I have successfully installed the api and modules, but i dont know how…
Sorry i am newbie

A Former User

@t-architecture99

Recommending beverage is not the job of this module.
But if you have MMM-RecommendDrinks (Of course, currently it exists not), You can make an order of I want to drink something to MMM-AssistantMk2 then, AMK2 would transfer your order to that RD module, and RD can RESPONSE with voice via TTS module.
So, that feature is for developers not for end-users directly.

t.architecture99

@Sean i mean i want to set default question and answers with voice response (using google TTS. I dont know how to command to create . :(

A Former User

@t-architecture99
By default, If the response could be handled by Assistant itself, it can response with voice. (e.g : “How’s the weather today?” - “It’s rainy blah blah…”)
So what is your default question? hmmm… It’s better to tell me “YOUR REAL INTENTION or SCENARIO” - what you want.

In case that you need some extended response from MM & modules
By example,

if you want to get a response “Yes, sir!” when you order “Shut down now” (by your custom command),
- Use custom action instead transcriptionHook on making a command. custom action can handle a simple static voice response.(around 150 words)
But more complex dynamic conversation like - “Read me first article of news module” then “New york times said blah blah…”
- You should modify news module to speak somethings, and at that time TTS solution be needed.
- And the role of AMK2 is just sending “Read first article” order to news module. That is the way how it works.

EstoyMejor

Hello guys, i hope someone can help me here. I found several people with the same problem but all got fixed by the Troubleshooting in the README, mine doesnt (or maybe i did something wrong?!)

My problem is this error:

Error: Failed to load gRPC binary module because it was not installed for the current system
Expected directory: electron-v3.1-linux-arm-glibc
Found: [node-v72-linux-arm-glibc]

Full start sequence log:

pi@InfoDisplayPi:~/MagicMirror $ npm start

> magicmirror@2.8.0 start /home/pi/MagicMirror
> sh run-start.sh

Starting MagicMirror: v2.8.0
Loading config ...
Loading module helpers ...
No helper found for module: alert.
Initializing new module helper ...
Module helper loaded: updatenotification
No helper found for module: clock.
Initializing new module helper ...
Module helper loaded: calendar
No helper found for module: compliments.
No helper found for module: currentweather.
No helper found for module: weatherforecast.
Initializing new module helper ...
Module helper loaded: newsfeed
WARNING! Could not load config file. Starting with default configuration. Error found: Error: Failed to load gRPC binary module because it was not installed for the current system
Expected directory: electron-v3.1-linux-arm-glibc
Found: [node-v72-linux-arm-glibc]
This problem can often be fixed by running "npm rebuild" on the current system
Original error: Cannot find module '/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/node/extension_binary/electron-v3.1-linux-arm-glibc/grpc_node.node'
Loading module helpers ...
No helper found for module: alert.
Initializing new module helper ...
Module helper loaded: updatenotification
No helper found for module: clock.
Initializing new module helper ...
Module helper loaded: calendar
No helper found for module: compliments.
No helper found for module: currentweather.
No helper found for module: weatherforecast.
Initializing new module helper ...
Module helper loaded: newsfeed
App threw an error during load
Error: Failed to load gRPC binary module because it was not installed for the current system
Expected directory: electron-v3.1-linux-arm-glibc
Found: [node-v72-linux-arm-glibc]
This problem can often be fixed by running "npm rebuild" on the current system
Original error: Cannot find module '/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/node/extension_binary/electron-v3.1-linux-arm-glibc/grpc_node.node'
    at Object.<anonymous> (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/grpc_extension.js:53:17)
    at Object.<anonymous> (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/grpc_extension.js:64:3)
    at Module._compile (internal/modules/cjs/loader.js:711:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:722:10)
    at Module.load (internal/modules/cjs/loader.js:620:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:559:12)
    at Function.Module._load (internal/modules/cjs/loader.js:551:3)
    at Module.require (internal/modules/cjs/loader.js:658:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at Object.<anonymous> (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client_interceptors.js:144:12)
Whoops! There was an uncaught exception...
{ Error: Failed to load gRPC binary module because it was not installed for the current system
Expected directory: electron-v3.1-linux-arm-glibc
Found: [node-v72-linux-arm-glibc]
This problem can often be fixed by running "npm rebuild" on the current system
Original error: Cannot find module '/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/node/extension_binary/electron-v3.1-linux-arm-glibc/grpc_node.node'
    at Object.<anonymous> (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/grpc_extension.js:53:17)
    at Object.<anonymous> (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/grpc_extension.js:64:3)
    at Module._compile (internal/modules/cjs/loader.js:711:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:722:10)
    at Module.load (internal/modules/cjs/loader.js:620:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:559:12)
    at Function.Module._load (internal/modules/cjs/loader.js:551:3)
    at Module.require (internal/modules/cjs/loader.js:658:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at Object.<anonymous> (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client_interceptors.js:144:12) code: 'MODULE_NOT_FOUND' }
MagicMirror will not quit, but it might be a good idea to check why this happened. Maybe no internet connection?
If you think this really is an issue, please open an issue on GitHub: https://github.com/MichMich/MagicMirror/issues
Launching application.
Shutting down server...
Stopping module helper: updatenotification
Stopping module helper: calendar
Stopping module helper: newsfeed
Stopping module helper: updatenotification
Stopping module helper: calendar
Stopping module helper: newsfeed

I Feel like i just oversee something really easy here…