MMM-AssistantMk2

[A Former User]

@costascontis
But in that case, Other modules cannot use speaker-out. I rather recommend to use other speaker instead HDMI.

[costascontis]

yes i understand,i just think that is a MMM-AssistantMk2 's problem because when i trigger MMM-Hotword it makes a ding sound to show that its listening and that sound i can always hear it so hdmi is awake ,then i make a request to MMM-AssistantMk2 and when it answers i cant hear the first word or the first letters of it anyway…

[Lowis]

Hello

how can MMM-AssistantMk2 start a webradio stream ? i tell the Assistant to play radio 1live. Respone is “OK” but nothing will come up.
I say "show me a picture from a Dog . Respone “OK” but nothing comes up.

BTW : Great Module !

[A Former User]

@lowis
Those features are allowed to REAL GOOGLE HOME devices at this moment. I hope somedays it could be allowed to Assistant SDK.

[Lowis]

But a something like your youtube search and playing Should be possible for a webradio

[A Former User]

@lowis Youtube features are added by myself with Youtube API.

[Lowis]

@Sean Yes i know. Great work ! Maybe it could expand with a webradio directory like shoutcast.com

[Jmh474]

I’m guessing that this question has been asked quite a few times so I’m sorry for the repeat.

I’ve got my mirror up and running using latest resabian, MagicMirror2, modules MMM-AssistantMk2 and MMM-HOTWORD.
Adding more modules soon just taking a step at a time as I’m a total beginner.

Seems to be running OK apart from the Google Assistant, its running so slow when asking questions and waiting for replies, and I don’t get voice response back just like a humming sound (I’m guessing this is when Google is answering), I know sound and mic are working as I can her myself talk when testing the mic.

Also what is the best setup for mic and speakers in using a USB sound card
Mic is just standard 3.5mm jack type
Speakers are running through a separate amp after the sound card.

One more this is there a way to add an Led for when the Google Assistant is listening??

Sorry for the long post I’m just so close

[A Former User]

@jmh474
For slow speed, I have no idea about it. Maybe it is related with your network status. Usually others have no problem about speed.
I think, it is not about speed. You just couldn’t get the response with some reason.
Test like this;

• Stop the pm2 (If you are using)
• Execute MM with npm start dev on MagicMirror directory.
• See what happens on terminal shell.
  There could be errors or some useful messages to examine.

[Licorini]

Hello … I am a beginner, I am trying to install Mk2 following the instructions, however it presents errors as below:

pi@raspberrypi:~/MagicMirror $ npm start

magicmirror@2.6.0 start /home/pi/MagicMirror
sh run-start.sh

Starting MagicMirror: v2.6.0
Loading config …
Loading module helpers …
No helper found for module: alert.
Initializing new module helper …
Module helper loaded: updatenotification
WARNING! Could not load config file. Starting with default configuration. Error found: Error: Failed to load gRPC binary module because it was not installed for the current system
Expected directory: electron-v2.0-linux-arm-glibc
Found: [node-v64-linux-arm-glibc]
This problem can often be fixed by running “npm rebuild” on the current system
Original error: Cannot find module ‘/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/node/extension_binary/electron-v2.0-linux-arm-glibc/grpc_node.node’
Loading module helpers …
No helper found for module: alert.
Initializing new module helper …
Module helper loaded: updatenotification
App threw an error during load
Error: Failed to load gRPC binary module because it was not installed for the current system
Expected directory: electron-v2.0-linux-arm-glibc
Found: [node-v64-linux-arm-glibc]
This problem can often be fixed by running “npm rebuild” on the current system
Original error: Cannot find module ‘/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/node/extension_binary/electron-v2.0-linux-arm-glibc/grpc_node.node’
at Object. (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/grpc_extension.js:53:17)
at Object. (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/grpc_extension.js:63:3)
at Module._compile (module.js:642:30)
at Object.Module._extensions…js (module.js:653:10)
at Module.load (module.js:561:32)
at tryModuleLoad (module.js:504:12)
at Function.Module._load (module.js:496:3)
at Module.require (module.js:586:17)
at require (internal/module.js:11:18)
at Object. (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client_interceptors.js:144:12)
Whoops! There was an uncaught exception…
{ Error: Failed to load gRPC binary module because it was not installed for the current system
Expected directory: electron-v2.0-linux-arm-glibc
Found: [node-v64-linux-arm-glibc]
This problem can often be fixed by running “npm rebuild” on the current system
Original error: Cannot find module ‘/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/node/extension_binary/electron-v2.0-linux-arm-glibc/grpc_node.node’
at Object. (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/grpc_extension.js:53:17)
at Object. (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/grpc_extension.js:63:3)
at Module._compile (module.js:642:30)
at Object.Module._extensions…js (module.js:653:10)
at Module.load (module.js:561:32)
at tryModuleLoad (module.js:504:12)
at Function.Module._load (module.js:496:3)
at Module.require (module.js:586:17)
at require (internal/module.js:11:18)
at Object. (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client_interceptors.js:144:12) code: ‘MODULE_NOT_FOUND’ }
MagicMirror will not quit, but it might be a good idea to check why this happened. Maybe no internet connection?
If you think this really is an issue, please open an issue on GitHub: https://github.com/MichMich/MagicMirror/issues
Launching application.

Can someone help me ??

[Lowis]

@sean but how can i setup the notification to start play MMM_Webradio ?

MMM_Webradio has this notifications :

WEBRADIO_ONOFF Starts / Pauses playback
WEBRADIO_VOLUP Volume up by 5%
WEBRADIO_VOLDOWN Volume down by 5%
WEBRADIO_NEXT Next station
WEBRADIO_PREV Previous station

[A Former User]

@licorini
You’ve missed or failed electron-rebuild step. Do it with instruction on README

[A Former User]

@lowis
There are two options to make custom commands - transcriptionHook and gAction.

I’ll explain with transcriptionHook because it is easier (but limited)
first. define hook phrase.

transcriptionHook: {
  "RADIO": {
    pattern: "my radio",  // When you say "my radio",
    command: "RADIO_ONOFF" // command "RADIO_ONOFF" will be executed
  },
  ... // if you need more hooks, add here.
}

Then, Let’s make command RADIO_ONOFF to emit notification when your voice is hooked.

command: {
  "RADIO_ONOFF": {
    notificationExec: {
      notification: "WEBRADIO_ONOFF",
    },
  },
  ... // if you need more commands, add here.
},

Now, when you say “my radio” in your voice order, “RADIO_ONOFF” command will be executed to send notification “WEBRADIO_ONOFF”.

But, I think there could be a problem to use webradio. Because your radio sound will interrupt HOTWORD or AMK2 occasionally, it could make unintended misbehaviours. Anyway, I wish you good luck.

[Lowis]

@sean thank you very much for the detailed information . Webradio is running !

[teitlebot]

@Sean
Thanks again for all the work and help
So how would I get assistant module to disable and enable the google photo module?
I would like to speak “Hide Photos” and the Google Photos will stop displaying.
I know I need transition hook and command
I tried learning how to do this using your example of hide clock except after I added the code nothing happened. When I say “hide clock” it searches google for hide clock instead of turning off module. So I must be doing something wrong
BTW the hide all command works fine.

[A Former User]

@teitlebot
I don’t know how you do your code. I have no second sight. Show your work to examine.

[Egnos]

Hi @Sean.
I have some questions to ask you:

• I installed AssistantMK2 and Hotword successfully but the only way to wake up the assistant is to say “snowboy”. The hot word “smartmirror” and “jarvis” do not work. Can you help me? (my config.js below…)

• When I ask for a song on youtube I can not stop the song anymore. The assistant does not listen to me. To talk to the assistant I have to wait for the end of the song. How do I solve?

• When the assistant begins to speak, he cuts the first words, is it possible to solve?

• The videos on youtube are reproduced in full screen. Is it possible to reproduce them in a small box (example: bottom right …)?

Thanks you for your great work and patience! :winking_face:

/* Magic Mirror Config Sample
 *
 * By Michael Teeuw http://michaelteeuw.nl
 * MIT Licensed.
 *
 * For more information how you can configurate this file
 * See https://github.com/MichMich/MagicMirror#configuration
 *
 */

var config = {
	address: "localhost", // Address to listen on, can be:
	                      // - "localhost", "127.0.0.1", "::1" to listen on loopback interface
	                      // - another specific IPv4/6 to listen on a specific interface
	                      // - "", "0.0.0.0", "::" to listen on any interface
	                      // Default, when address config is left out, is "localhost"
	port: 8080,
	ipWhitelist: ["127.0.0.1", "::ffff:127.0.0.1", "::1"], // Set [] to allow all IP addresses
	                                                       // or add a specific IPv4 of 192.168.1.5 :
	                                                       // ["127.0.0.1", "::ffff:127.0.0.1", "::1", "::ffff:192.168.1.5"],
	                                                       // or IPv4 range of 192.168.3.0 --> 192.168.3.15 use CIDR format :
	                                                       // ["127.0.0.1", "::ffff:127.0.0.1", "::1", "::ffff:192.168.3.0/28"],

	language: "en",
	timeFormat: 24,
	units: "metric",

	modules: [
		{
		  module: "MMM-AssistantMk2",
		  position: "top_right",
		  config: {
		    /*useWelcomeMessage: "brief today",*/
		    deviceLocation: {
					coordinates: { // set the latitude and longitude of the device to get localized information like weather or time. (ref. mygeoposition.com)
					latitude: 45.200049, // -90.0 - +90.0
					longitude: 7.568917, // -180.0 - +180.0					
          }
        },
			profiles: {
				"default" : { // profile name.
				profileFile: "default.json", // profile file name.
				lang: "it-IT",
				//currently available (estimation, not all tested):
				//  de-DE, en-AU, en-CA, en-GB, en-US, en-IN
				// fr-CA, fr-FR, it-IT, ja-JP, es-ES, es-MX, ko-KR, pt-BR
				// https://developers.google.com/assistant/sdk/reference/rpc/languages
				}
        },
			record: { // Full values are in `FOR EXPERTS` section.
				recordProgram: "arecord",  // Defaults to "arecord" - also supports "rec" and "sox"
				device: "plughw:1"        // recording device (e.g.: "plughw:1")
      },
			play: { // Full values are in `FOR EXPERTS` section.
				playProgram: "mpg321", // recommended.
			},
			notifications: {
        ASSISTANT_ACTIVATED: "HOTWORD_PAUSE",
        ASSISTANT_DEACTIVATED: "HOTWORD_RESUME",
      },        
     
				}
    },
		{
    module: "MMM-Hotword",
    config: {
      record: {
        recordProgram : "arecord",  
        device        : "plughw:1",
      },
      autostart:true,
      onDetected: {
        notification: function (payload) {
          return "ASSISTANT_ACTIVATE"
        },
        payload: function (payload){
          return {
            profile: payload.hotword
          }
        }
      },
    },
  },	
	
{
  module: 'MMM-Hotword',
  config: {
    snowboy: [
  {
    hotwords: "smartmirror", //this will be sent to other module for distinguishing which hotword is detected.
    file: "resources/models/smart_mirror.umdl",
    sensitivity: '0.5',
  },
  {
    hotwords: "snowboy",
    file: "resources/models/snowboy.umdl",
    sensitivity: '0.5',
  },
  {
    file: 'resources/models/jarvis.umdl',
    sensitivity: '0.8,0.80',
    hotwords: ['jarvis','jarvis'] //Kitt.ai changed their Jarvis UMDL, it has 2 models in one file. So weird.
    //anyway, you can give different name for each. ['jarvis_1', 'jarvis_2']. Even though I think this is useless.
  }
    ],
    record: {
      sampleRate    : 16000,      // audio sample rate
      threshold     : 0.5,        // silence threshold (rec only)
      thresholdStart: null,       // silence threshold to start recording, overrides threshold (rec only)
      thresholdEnd  : null,       // silence threshold to end recording, overrides threshold (rec only)
      silence       : 1.0,        // seconds of silence before ending
      verbose       : true,      // log info to the console. Use this when you want to check mic working or not.
      recordProgram : 'arecord',  // Defaults to 'arecord' - also supports 'rec' and 'sox'
      device        : 'plughw:1'        // recording device (e.g.: 'plughw:1')
    },
    autostart: true,              // if 'false', this module will wait for 'HOTWORD_RESUME' notification to start hotwords detection at the beginning.
    autorestart: false,          // You can set this 'true' when you want this module to go back to listening mode automatically again after hotword is detected. But use this carefully when your other modules are using microphone or speaker.

    // customizable notification trigger
    notifications: {
      PAUSE: "HOTWORD_PAUSE",
      RESUME: "HOTWORD_RESUME",
      LISTENING : "HOTWORD_LISTENING",
      SLEEPING : "HOTWORD_SLEEPING",
      ERROR : "HOTWORD_ERROR",
    },
    onDetected: {
      notification: (payload) => {
        return "HOTWORD_DETECTED"
      },
      payload: (payload) => {
        return payload
      }
    },
  }
},

/*		{
        module: 'MMM-pages',
        config: {
                modules:
                    [[ "weatherforecast", "newsfeed"],
                     [ "calendar", "compliments" ]],
                excludes: ["clock", "currentweather", "MMM-page-indicator", "MMM-AssistantMk2"],
				}
		},
		{
        module: 'MMM-page-indicator',
        position: 'bottom_bar',
        config: {
            pages: 3,
			}
		},
*/		{
			module: "alert",
		},
		{
			module: "updatenotification",
			position: "top_bar"
		},
		{
			module: "clock",
			position: "top_left"
		},
		{
			module: "calendar",
			header: "US Holidays",
			position: "top_left",
			config: {
				calendars: [
					{
						symbol: "calendar-check-o ",
						url: "webcal://www.calendarlabs.com/templates/ical/US-Holidays.ics"
					}
				]
			}
		},
		{
			module: "compliments",
			position: "lower_third"
		},
		{
			module: "currentweather",
			position: "top_right",
			config: {
				location: "New York",
				locationID: "",  //ID from http://bulk.openweathermap.org/sample/; unzip the gz file and find your city
				appid: "YOUR_OPENWEATHER_API_KEY"
			}
		},
		{
			module: "weatherforecast",
			position: "top_right",
			header: "Weather Forecast",
			config: {
				location: "New York",
				locationID: "5128581",  //ID from http://www.openweathermap.org/help/city_list.txt
				appid: "YOUR_OPENWEATHER_API_KEY"
			}
		},
		{
			module: "newsfeed",
			position: "bottom_bar",
			config: {
				feeds: [
					{
						title: "La Stampa",
						url: "https://www.lastampa.it/rss.xml"
					}
				],
				showSourceTitle: true,
				showPublishDate: true
			}
		},
	]

};

/*************** DO NOT EDIT THE LINE BELOW ***************/
if (typeof module !== "undefined") {module.exports = config;}

[A Former User]

@egnos

I installed AssistantMK2 and Hotword successfully but the only way to wake up the assistant is to say “snowboy”. The hot word “smartmirror” and “jarvis” do not work. Can you help me? (my config.js below…)

Adjust sensitivity.

When I ask for a song on youtube I can not stop the song anymore. The assistant does not listen to me. To talk to the assistant I have to wait for the end of the song. How do I solve?

set pauseOnYoutube:false. You can activate hotword during videoplaying and it will stop the video. Anyway, there is a risk that your video sounds could also activate hotword.

When the assistant begins to speak, he cuts the first words, is it possible to solve?

I think you are using HDMI as output. HDMI could have that symptom. Use other output.

The videos on youtube are reproduced in full screen. Is it possible to reproduce them in a small box (example: bottom right …)?

Redefine CSS selector #ASSISTANT_YOUTUBE in css/custom.css.

[Egnos]

@sean said in MMM-AssistantMk2:

Adjust sensitivity.

The sensitivity of the microphone?
If yes, how do you do it?

@sean said in MMM-AssistantMk2:

Redefine CSS selector #ASSISTANT_YOUTUBE in css/custom.css.

Where should I make this modify?

Thanks a lot!

[A Former User]

@egnos

{
    hotwords: "smartmirror", 
    file: "resources/models/smart_mirror.umdl",
    sensitivity: '0.5', // < - SENSITIVITY
  },

Where should I make this modify?

css/custom.css