Playing Youtube video with MMM-AssistantMk2 not working

vschinazi

Hello. First, I want to say thank you @Sean for the absolutely amazing module. I love the “MMM-Assistant Mk2” module. I was able to install it and it is working perfectly. The only thing that I am struggling with is the ability to ask Google assistant to play a Youtube video in a small part of the screen. Whenever I ask Google Assistant for something that would trigger a Youtube video (e.g., “Andrew Bird Night Sky”) I get a reply that the Youtube video was found but then the screen freezes.

I feel like the module is trying to call on the Youtube API (maybe to open a frame) but it never manages to do it. I find this behaviour strange because I was able to play Youtube videos from other modules in my Magic mirror. More interesting, if I install the “MMM-YouTube-API”, module, add it to my config file and then say “Andrew Bird Night Sky” then the Youtbe video begins to play but for the full screen (and on top of the other video from the MMM-YouTube-API".

Am I missing a dependency or a line of code in my config file that maybe activates the Youtube play for a section of the screen?

Can you help me with this?

Here is a copy of my config file:

{
  module: "MMM-AssistantMk2",
  position: "top_right",
  config: {

    // --- ESSENTIALS / modifying for your environment might be needed.


    deviceLocation: {
      coordinates: { // set the latitude and longitude of the device to get localized information like weather or time. (ref. mygeoposition.com)
        latitude: 47.358720, // -90.0 - +90.0
        longitude: 8.525780, // -180.0 - +180.0
      },
    },

    defaultProfile: "default", // If you have several profiles and want to set one of them as default profile, describe here.

    profiles: {
      "default" : { // profile name.
        profileFile: "default.json", // profile file name.
        lang: "en-GB"
        //currently available (estimation, not all tested):
        //  de-DE, en-AU, en-CA, en-GB, en-US, en-IN
        // fr-CA, fr-FR, it-IT, ja-JP, es-ES, es-MX, ko-KR, pt-BR
        // https://developers.google.com/assistant/sdk/reference/rpc/languages
      },
      /* Add your other profiles here, if exists.
      "other_profile" : {
        profileFile: "other.json",
        lang: "en-GB"
      }
      */
    },

    record: { // Full values are in `FOR EXPERTS` section.
      recordProgram: "arecord",  // Defaults to "arecord" - also supports "rec" and "sox"
      device: null        // recording device (e.g.: "plughw:1")
    },

    play: { // Full values are in `FOR EXPERTS` section.
      playProgram: "mpg321", // recommended.
    },


    // --- OPTIONAL / not important but customizable for your usage


    responseVoice: true, // If available, Assistant will response with her voice.
    responseScreen: true, // If available, Assistant will response with some rendered HTML
    responseAlert: true, // If available, Assistant will response with Alert module of MM
    // Sometimes, any response might not be returned. responseAlert is useful for displaying error.

    screenZoom: "80%", // Adjust responseScreen to your mirror size.
    screenDuration: 4000, // milliseconds. How long responseScreen will be shown after speech.
    //If you set 0, Screen Output will be closed after Response speech finishes ASAP.

    youtubeAutoplay: true, //If set as true, found Youtube video will be played automatically.
    pauseOnYoutube:true, //If set as true, You cannot activate Assistant during youtube playing. Recommended for the performance (Because permanent hotword detecting might make performance lower)

    useWelcomeMessage: "", //Try "brief today" as this value. You can use this value to check module working when MM is starting.

    onIdle: {
      timer: 1000*60*30, // if you don't want to use this feature, just set timer as `0` or command as ""
      command: "HIDEMODULES"
    },

    onActivate: {
      timer: 0,
      command: "SHOWMODULES"
    },


    // --- FOR EXPERTS / For development, debug or more


    verbose:false, // You can get error or some logs when this value is set as true.

    startChime: "connection.mp3", // you can use `mp3` to play chime when your mic is ready. It should be playable with your `play.playProgram`
    noChimeOnSay: false, // When using the `ASSISTANT_SAY` trigger, you can prevent the chime from being played before your words

    auth: { // I believe you don't need to change this.
      keyFilePath: "./credentials.json"
    },

    record:  { // Full version
      sampleRate    : 16000,      // audio sample rate
      threshold     : 0.5,        // silence threshold (rec only)
      thresholdStart: null,       // silence threshold to start recording, overrides threshold (rec only)
      thresholdEnd  : null,       // silence threshold to end recording, overrides threshold (rec only)
      silence       : 1.0,        // seconds of silence before ending
      verbose       : false,      // log info to the console
      recordProgram : "arecord",  // Defaults to "arecord" - also supports "rec" and "sox"
      device        : null        // recording device (e.g.: "plughw:1")
    },

    play: { // Full version
      encodingOut: "MP3", //'MP3' or 'WAV' is available, but you might not need to modify this.
      sampleRateOut: 24000,
      playProgram: "mpg321", //Your prefer sound play program. By example, if you are running this on OSX, `afplay` could be available.
      playOption: [],
      // If you need additional options to use playProgram, describe here. (except filename)
      // e.g: ["-d", "", "-t", "100"]
    },

    useGactionCLI: false, // If set as true, you can update your gAction when MM is rebooted.
    projectId: "", // Google Assistant ProjectId (Required only when you use gAction.)
    deviceModelId: "", // It should be described in your config.json. In most of case, you don't need to this.
    deviceInstanceId: "", // It should be described in your config.json. In most of case, you don't need to this.

    action:{}, // You can catch your gAction command.

    transcriptionHook: {}, // You can catch transcription hook and be able to make your own `COMMAND` with this.
    //See the `transcriptionHook` section.

    command: {}, // You can make your own MM command for gAction and transcriptionHook
    //See the `command` section.
    record: {
        recordProgram : "arecord",  
        device        : "plughw:1",
      },

      notifications: {
        ASSISTANT_ACTIVATED: "HOTWORD_PAUSE",
        ASSISTANT_DEACTIVATED: "HOTWORD_RESUME",
      },
	
    notifications: { // You can redefine these notifications to communicate with specific modules.
      ASSISTANT_ACTIVATE: "ASSISTANT_ACTIVATE",
      ASSISTANT_DEACTIVATE: "ASSISTANT_CLEAR",
      ASSISTANT_ACTIVATED: "ASSISTANT_ACTIVATED",
      ASSISTANT_DEACTIVATED: "ASSISTANT_DEACTIVATED",
      ASSISTANT_ACTION: "ASSISTANT_ACTION",
      ASSISTANT_ACTIVATED: "HOTWORD_PAUSE",     
      ASSISTANT_DEACTIVATED: "HOTWORD_RESUME",      
      DEFAULT_HOOK_NOTIFICATION: "ASSISTANT_HOOK",
      TEXT_QUERY: "ASSISTANT_QUERY",
      SAY_TEXT: "ASSISTANT_SAY",
    }
  }
},
{
  module: "MMM-Hotword",
  position: "top_right",
  config: {
    chimeOnFinish:null,
    mic: {
      recordProgram : "arecord",  
      device        : "plughw:1",
    },
    models: [
      {
        hotwords    : "smart_mirror",
        file        : "smart_mirror.umdl",
        sensitivity : "0.5",
      },
    ],
    defaultCommand: {
      notificationExec: {
        notification: "ASSISTANT_ACTIVATE",
        payload: (detected, afterRecord) => {
          return {profile:"default"}
        }
      },
      afterRecordLimit:0,
      restart:false,
    },
  },
},
{
  module: "MMM-NotificationTrigger",
  config: {
    triggers:[
      {
        trigger: "ASSISTANT_ACTION",
        triggerSenderFilter: (sender) => {
          console.log(sender)
          if (sender.name == 'MMM-AssistantMk2') {
            return true
          } else {
            return false
          }
        },
        triggerPayloadFilter: (payload) => {
          console.log(payload)
          return true
        },
        fires: [
          {
            fire:"SHOW_ALERT",
            payload: (payload) => {
              return {
                type: "notification",
                title: payload.type,
                message: payload.command
              }
            },
          },
        ],
      },
    ]
  }

},

@Sean, I sent you a chat message before but realised this issues might be useful for other so I posted here.

Looking forward to you reply.

Thank you!

A Former User

@vschinazi
Maybe you might have some front-errors(from mine or other modules). You can check it on dev-console by npm start dev or Ctrl+i(Or Shift)
So some errors would prevent loading of Youtube API. Check it first.

vschinazi

@Sean thank you. I will run it this evening and send it to you. I really appreciate all your help.