MMM-AssistantMk2

[costascontis]

@Sean at last got MMM-AssistantMk2 up and running with MMM-Hotword.I have only one little problem,when i ask something i lose the first word of the answer.It is like hdmi sound channel is down and needs 1 sec to wake so i cant hear the first word of the answer.I add to /boot/config.txt the line “force_hdmi_open=1” but didnt help.At my standalone raspberry google assistant installation i had the same problem and this line solved it,but not here…

[A Former User]

@costascontis
Sorry, I have no idea about it. Maybe my module would close HDMI when it doesn’t be needed. (Standalone version could open HDMI continuously by force, because there is no need to close for other’s occupation)

[costascontis]

@sean it is very annoying not to listen the first word or missing the half of it,do you have an idea to keep hdmi on all the time?
edit: if i use the .asoundrc file of my other gassistant i have no delay but the voice is choppy.

pcm.dsnooper {
    type dsnoop
    ipc_key 816357492
    ipc_key_add_uid 0
    ipc_perm 0666
    slave {
        pcm "hw:1,0"
        channels 1
    }
}

pcm.!default {
        type asym
        playback.pcm {
                type plug
                slave.pcm "hw:0,1"
        }
        capture.pcm {
                type plug
                slave.pcm "dsnooper"
        }
}

if i use the default .asoundrc the voice is clear but with a start delay

pcm.!default {
	type hw
	card 0
}

ctl.!default {
	type hw
	card 0
}

with the first .asoundrc delayless file all other sounds at the Rpi are playback just fine ,only the MMM-AssistantMk2 voice is grainy and choppy… : (

[A Former User]

@costascontis
But in that case, Other modules cannot use speaker-out. I rather recommend to use other speaker instead HDMI.

[costascontis]

yes i understand,i just think that is a MMM-AssistantMk2 's problem because when i trigger MMM-Hotword it makes a ding sound to show that its listening and that sound i can always hear it so hdmi is awake ,then i make a request to MMM-AssistantMk2 and when it answers i cant hear the first word or the first letters of it anyway…

[Lowis]

Hello

how can MMM-AssistantMk2 start a webradio stream ? i tell the Assistant to play radio 1live. Respone is “OK” but nothing will come up.
I say "show me a picture from a Dog . Respone “OK” but nothing comes up.

BTW : Great Module !

[A Former User]

@lowis
Those features are allowed to REAL GOOGLE HOME devices at this moment. I hope somedays it could be allowed to Assistant SDK.

[Lowis]

But a something like your youtube search and playing Should be possible for a webradio

[A Former User]

@lowis Youtube features are added by myself with Youtube API.

[Lowis]

@Sean Yes i know. Great work ! Maybe it could expand with a webradio directory like shoutcast.com

[Jmh474]

I’m guessing that this question has been asked quite a few times so I’m sorry for the repeat.

I’ve got my mirror up and running using latest resabian, MagicMirror2, modules MMM-AssistantMk2 and MMM-HOTWORD.
Adding more modules soon just taking a step at a time as I’m a total beginner.

Seems to be running OK apart from the Google Assistant, its running so slow when asking questions and waiting for replies, and I don’t get voice response back just like a humming sound (I’m guessing this is when Google is answering), I know sound and mic are working as I can her myself talk when testing the mic.

Also what is the best setup for mic and speakers in using a USB sound card
Mic is just standard 3.5mm jack type
Speakers are running through a separate amp after the sound card.

One more this is there a way to add an Led for when the Google Assistant is listening??

Sorry for the long post I’m just so close

[A Former User]

@jmh474
For slow speed, I have no idea about it. Maybe it is related with your network status. Usually others have no problem about speed.
I think, it is not about speed. You just couldn’t get the response with some reason.
Test like this;

• Stop the pm2 (If you are using)
• Execute MM with npm start dev on MagicMirror directory.
• See what happens on terminal shell.
  There could be errors or some useful messages to examine.

[Licorini]

Hello … I am a beginner, I am trying to install Mk2 following the instructions, however it presents errors as below:

pi@raspberrypi:~/MagicMirror $ npm start

magicmirror@2.6.0 start /home/pi/MagicMirror
sh run-start.sh

Starting MagicMirror: v2.6.0
Loading config …
Loading module helpers …
No helper found for module: alert.
Initializing new module helper …
Module helper loaded: updatenotification
WARNING! Could not load config file. Starting with default configuration. Error found: Error: Failed to load gRPC binary module because it was not installed for the current system
Expected directory: electron-v2.0-linux-arm-glibc
Found: [node-v64-linux-arm-glibc]
This problem can often be fixed by running “npm rebuild” on the current system
Original error: Cannot find module ‘/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/node/extension_binary/electron-v2.0-linux-arm-glibc/grpc_node.node’
Loading module helpers …
No helper found for module: alert.
Initializing new module helper …
Module helper loaded: updatenotification
App threw an error during load
Error: Failed to load gRPC binary module because it was not installed for the current system
Expected directory: electron-v2.0-linux-arm-glibc
Found: [node-v64-linux-arm-glibc]
This problem can often be fixed by running “npm rebuild” on the current system
Original error: Cannot find module ‘/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/node/extension_binary/electron-v2.0-linux-arm-glibc/grpc_node.node’
at Object. (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/grpc_extension.js:53:17)
at Object. (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/grpc_extension.js:63:3)
at Module._compile (module.js:642:30)
at Object.Module._extensions…js (module.js:653:10)
at Module.load (module.js:561:32)
at tryModuleLoad (module.js:504:12)
at Function.Module._load (module.js:496:3)
at Module.require (module.js:586:17)
at require (internal/module.js:11:18)
at Object. (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client_interceptors.js:144:12)
Whoops! There was an uncaught exception…
{ Error: Failed to load gRPC binary module because it was not installed for the current system
Expected directory: electron-v2.0-linux-arm-glibc
Found: [node-v64-linux-arm-glibc]
This problem can often be fixed by running “npm rebuild” on the current system
Original error: Cannot find module ‘/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/node/extension_binary/electron-v2.0-linux-arm-glibc/grpc_node.node’
at Object. (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/grpc_extension.js:53:17)
at Object. (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/grpc_extension.js:63:3)
at Module._compile (module.js:642:30)
at Object.Module._extensions…js (module.js:653:10)
at Module.load (module.js:561:32)
at tryModuleLoad (module.js:504:12)
at Function.Module._load (module.js:496:3)
at Module.require (module.js:586:17)
at require (internal/module.js:11:18)
at Object. (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client_interceptors.js:144:12) code: ‘MODULE_NOT_FOUND’ }
MagicMirror will not quit, but it might be a good idea to check why this happened. Maybe no internet connection?
If you think this really is an issue, please open an issue on GitHub: https://github.com/MichMich/MagicMirror/issues
Launching application.

Can someone help me ??

[Lowis]

@sean but how can i setup the notification to start play MMM_Webradio ?

MMM_Webradio has this notifications :

WEBRADIO_ONOFF Starts / Pauses playback
WEBRADIO_VOLUP Volume up by 5%
WEBRADIO_VOLDOWN Volume down by 5%
WEBRADIO_NEXT Next station
WEBRADIO_PREV Previous station

[A Former User]

@licorini
You’ve missed or failed electron-rebuild step. Do it with instruction on README

[A Former User]

@lowis
There are two options to make custom commands - transcriptionHook and gAction.

I’ll explain with transcriptionHook because it is easier (but limited)
first. define hook phrase.

transcriptionHook: {
  "RADIO": {
    pattern: "my radio",  // When you say "my radio",
    command: "RADIO_ONOFF" // command "RADIO_ONOFF" will be executed
  },
  ... // if you need more hooks, add here.
}

Then, Let’s make command RADIO_ONOFF to emit notification when your voice is hooked.

command: {
  "RADIO_ONOFF": {
    notificationExec: {
      notification: "WEBRADIO_ONOFF",
    },
  },
  ... // if you need more commands, add here.
},

Now, when you say “my radio” in your voice order, “RADIO_ONOFF” command will be executed to send notification “WEBRADIO_ONOFF”.

But, I think there could be a problem to use webradio. Because your radio sound will interrupt HOTWORD or AMK2 occasionally, it could make unintended misbehaviours. Anyway, I wish you good luck.

[Lowis]

@sean thank you very much for the detailed information . Webradio is running !

[teitlebot]

@Sean
Thanks again for all the work and help
So how would I get assistant module to disable and enable the google photo module?
I would like to speak “Hide Photos” and the Google Photos will stop displaying.
I know I need transition hook and command
I tried learning how to do this using your example of hide clock except after I added the code nothing happened. When I say “hide clock” it searches google for hide clock instead of turning off module. So I must be doing something wrong
BTW the hide all command works fine.

[A Former User]

@teitlebot
I don’t know how you do your code. I have no second sight. Show your work to examine.

[Egnos]

Hi @Sean.
I have some questions to ask you:

• I installed AssistantMK2 and Hotword successfully but the only way to wake up the assistant is to say “snowboy”. The hot word “smartmirror” and “jarvis” do not work. Can you help me? (my config.js below…)

• When I ask for a song on youtube I can not stop the song anymore. The assistant does not listen to me. To talk to the assistant I have to wait for the end of the song. How do I solve?

• When the assistant begins to speak, he cuts the first words, is it possible to solve?

• The videos on youtube are reproduced in full screen. Is it possible to reproduce them in a small box (example: bottom right …)?

Thanks you for your great work and patience! :winking_face:

/* Magic Mirror Config Sample
 *
 * By Michael Teeuw http://michaelteeuw.nl
 * MIT Licensed.
 *
 * For more information how you can configurate this file
 * See https://github.com/MichMich/MagicMirror#configuration
 *
 */

var config = {
	address: "localhost", // Address to listen on, can be:
	                      // - "localhost", "127.0.0.1", "::1" to listen on loopback interface
	                      // - another specific IPv4/6 to listen on a specific interface
	                      // - "", "0.0.0.0", "::" to listen on any interface
	                      // Default, when address config is left out, is "localhost"
	port: 8080,
	ipWhitelist: ["127.0.0.1", "::ffff:127.0.0.1", "::1"], // Set [] to allow all IP addresses
	                                                       // or add a specific IPv4 of 192.168.1.5 :
	                                                       // ["127.0.0.1", "::ffff:127.0.0.1", "::1", "::ffff:192.168.1.5"],
	                                                       // or IPv4 range of 192.168.3.0 --> 192.168.3.15 use CIDR format :
	                                                       // ["127.0.0.1", "::ffff:127.0.0.1", "::1", "::ffff:192.168.3.0/28"],

	language: "en",
	timeFormat: 24,
	units: "metric",

	modules: [
		{
		  module: "MMM-AssistantMk2",
		  position: "top_right",
		  config: {
		    /*useWelcomeMessage: "brief today",*/
		    deviceLocation: {
					coordinates: { // set the latitude and longitude of the device to get localized information like weather or time. (ref. mygeoposition.com)
					latitude: 45.200049, // -90.0 - +90.0
					longitude: 7.568917, // -180.0 - +180.0					
          }
        },
			profiles: {
				"default" : { // profile name.
				profileFile: "default.json", // profile file name.
				lang: "it-IT",
				//currently available (estimation, not all tested):
				//  de-DE, en-AU, en-CA, en-GB, en-US, en-IN
				// fr-CA, fr-FR, it-IT, ja-JP, es-ES, es-MX, ko-KR, pt-BR
				// https://developers.google.com/assistant/sdk/reference/rpc/languages
				}
        },
			record: { // Full values are in `FOR EXPERTS` section.
				recordProgram: "arecord",  // Defaults to "arecord" - also supports "rec" and "sox"
				device: "plughw:1"        // recording device (e.g.: "plughw:1")
      },
			play: { // Full values are in `FOR EXPERTS` section.
				playProgram: "mpg321", // recommended.
			},
			notifications: {
        ASSISTANT_ACTIVATED: "HOTWORD_PAUSE",
        ASSISTANT_DEACTIVATED: "HOTWORD_RESUME",
      },        
     
				}
    },
		{
    module: "MMM-Hotword",
    config: {
      record: {
        recordProgram : "arecord",  
        device        : "plughw:1",
      },
      autostart:true,
      onDetected: {
        notification: function (payload) {
          return "ASSISTANT_ACTIVATE"
        },
        payload: function (payload){
          return {
            profile: payload.hotword
          }
        }
      },
    },
  },	
	
{
  module: 'MMM-Hotword',
  config: {
    snowboy: [
  {
    hotwords: "smartmirror", //this will be sent to other module for distinguishing which hotword is detected.
    file: "resources/models/smart_mirror.umdl",
    sensitivity: '0.5',
  },
  {
    hotwords: "snowboy",
    file: "resources/models/snowboy.umdl",
    sensitivity: '0.5',
  },
  {
    file: 'resources/models/jarvis.umdl',
    sensitivity: '0.8,0.80',
    hotwords: ['jarvis','jarvis'] //Kitt.ai changed their Jarvis UMDL, it has 2 models in one file. So weird.
    //anyway, you can give different name for each. ['jarvis_1', 'jarvis_2']. Even though I think this is useless.
  }
    ],
    record: {
      sampleRate    : 16000,      // audio sample rate
      threshold     : 0.5,        // silence threshold (rec only)
      thresholdStart: null,       // silence threshold to start recording, overrides threshold (rec only)
      thresholdEnd  : null,       // silence threshold to end recording, overrides threshold (rec only)
      silence       : 1.0,        // seconds of silence before ending
      verbose       : true,      // log info to the console. Use this when you want to check mic working or not.
      recordProgram : 'arecord',  // Defaults to 'arecord' - also supports 'rec' and 'sox'
      device        : 'plughw:1'        // recording device (e.g.: 'plughw:1')
    },
    autostart: true,              // if 'false', this module will wait for 'HOTWORD_RESUME' notification to start hotwords detection at the beginning.
    autorestart: false,          // You can set this 'true' when you want this module to go back to listening mode automatically again after hotword is detected. But use this carefully when your other modules are using microphone or speaker.

    // customizable notification trigger
    notifications: {
      PAUSE: "HOTWORD_PAUSE",
      RESUME: "HOTWORD_RESUME",
      LISTENING : "HOTWORD_LISTENING",
      SLEEPING : "HOTWORD_SLEEPING",
      ERROR : "HOTWORD_ERROR",
    },
    onDetected: {
      notification: (payload) => {
        return "HOTWORD_DETECTED"
      },
      payload: (payload) => {
        return payload
      }
    },
  }
},

/*		{
        module: 'MMM-pages',
        config: {
                modules:
                    [[ "weatherforecast", "newsfeed"],
                     [ "calendar", "compliments" ]],
                excludes: ["clock", "currentweather", "MMM-page-indicator", "MMM-AssistantMk2"],
				}
		},
		{
        module: 'MMM-page-indicator',
        position: 'bottom_bar',
        config: {
            pages: 3,
			}
		},
*/		{
			module: "alert",
		},
		{
			module: "updatenotification",
			position: "top_bar"
		},
		{
			module: "clock",
			position: "top_left"
		},
		{
			module: "calendar",
			header: "US Holidays",
			position: "top_left",
			config: {
				calendars: [
					{
						symbol: "calendar-check-o ",
						url: "webcal://www.calendarlabs.com/templates/ical/US-Holidays.ics"
					}
				]
			}
		},
		{
			module: "compliments",
			position: "lower_third"
		},
		{
			module: "currentweather",
			position: "top_right",
			config: {
				location: "New York",
				locationID: "",  //ID from http://bulk.openweathermap.org/sample/; unzip the gz file and find your city
				appid: "YOUR_OPENWEATHER_API_KEY"
			}
		},
		{
			module: "weatherforecast",
			position: "top_right",
			header: "Weather Forecast",
			config: {
				location: "New York",
				locationID: "5128581",  //ID from http://www.openweathermap.org/help/city_list.txt
				appid: "YOUR_OPENWEATHER_API_KEY"
			}
		},
		{
			module: "newsfeed",
			position: "bottom_bar",
			config: {
				feeds: [
					{
						title: "La Stampa",
						url: "https://www.lastampa.it/rss.xml"
					}
				],
				showSourceTitle: true,
				showPublishDate: true
			}
		},
	]

};

/*************** DO NOT EDIT THE LINE BELOW ***************/
if (typeof module !== "undefined") {module.exports = config;}