MagicMirror Forum
    • Recent
    • Tags
    • Unsolved
    • Solved
    • MagicMirror² Repository
    • Documentation
    • 3rd-Party-Modules
    • Donate
    • Discord
    • Register
    • Login
    A New Chapter for MagicMirror: The Community Takes the Lead
    Read the statement by Michael Teeuw here.

    MMM-OpenAI

    Scheduled Pinned Locked Moved Productivity
    24 Posts 4 Posters 12.1k Views 4 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S Offline
      sdmydbr9 @MMRIZE
      last edited by

      @MMRIZE a similar idea has crossed my mind. Using a Hotword detector like the one used in google assignment module to activate a listener for voice and ask for voice input and then using the input provided, get the transcript of the audio using the whisper api or something similar to get the transcript… pass on the transcript as a content in the OPENAI_REQUEST notification and then SHOW_ALERT or just read it out loud using the MMM-GoogleTTS. Just a rough sketch. I’m sure someone out here will come up with more convincing and better ideas.

      S M 2 Replies Last reply Reply Quote -1
      • S Offline
        sdetweil @sdmydbr9
        last edited by

        @sdmydbr9 use a GoogleAssistant recipe

        see it’s recipies folder

        Sam

        How to add modules

        learning how to use browser developers window for css changes

        S 1 Reply Last reply Reply Quote 0
        • M Offline
          MMRIZE @sdmydbr9
          last edited by MMRIZE

          @sdmydbr9
          I have a few already, like MMM-Hotword or MMM-Clap, and some…

          Anyway new OpenAI’s Whisper looks so interesting, So I’ll make it as a STT module in a few days, hope so.

          S 1 Reply Last reply Reply Quote 0
          • S Offline
            sdmydbr9 @sdetweil
            last edited by

            @sdetweil I just meant it as a similar approach, I don’t actually want to implement the whole assistant module.

            1 Reply Last reply Reply Quote 0
            • S Offline
              sdmydbr9 @MMRIZE
              last edited by

              @MMRIZE I think the Hotword module is not Available anymore and the clap module, might work but it will not look as asthetic as activating with the hot word…

              1 Reply Last reply Reply Quote 0
              • R Offline
                Rags @MMRIZE
                last edited by

                @MMRIZE said in MMM-OpenAI:

                @Rags
                Speech example: Conjunction with MMM-GoogleTTS, now it can speak.

                https://youtu.be/daWuSfQc5Ns

                postProcessing: (helper, resp) => {
                  let choice = resp.response?.choices[0]
                  if (choice) helper.sendNotification('TTS_SAY', choice?.message?.content ?? choice?.text)
                }
                

                I am getting a syntax error in these lines can you please check.
                Thanks

                R 1 Reply Last reply Reply Quote 0
                • R Offline
                  Rags @Rags
                  last edited by Rags

                  @MMRIZE
                  It’s possible that the error is caused by using optional chaining in a JavaScript version that doesn’t support it.

                  I modified the code as below, rewriting the code without using optional chaining.
                  postProcessing: function(helper, resp) {
                  let choice = resp.response ? resp.response.choices[0] : undefined;
                  if (choice) {
                  let message = choice.message ? choice.message.content.choice.text : undefined;
                  if (message) {
                  helper.sendNotification(‘TTS_SAY’, message);
                  }
                  }

                  I am not getting any syntax error now. However, there is some issue with Google TTS (unrelated I think) , as it is sending a notification as TTS_Say_Error. Google TTS API hopefully is configured correctly as no errors are thrown up during starting MM. How should i go about resolving it.

                  M 2 Replies Last reply Reply Quote 0
                  • M Offline
                    MMRIZE @Rags
                    last edited by

                    @Rags
                    Give me the front and backend log on error happens

                    1 Reply Last reply Reply Quote 0
                    • M Offline
                      MMRIZE @Rags
                      last edited by

                      @Rags
                      TTS_SAY notification could have more complex payload like this.

                      .sendNotification("TTS_SAY", {
                        content: "Live long and prosper", //text to be spoken
                        //belows are optional
                        type: "text", // "text" or "ssml"
                        voiceName: "en-US-Standard-B",
                        languageCode: "en-US",
                        ssmlGender: "FEMALE",
                        callback: (error)=> {
                          console.log(error) // <= What you need.
                        }
                      })
                      
                      1 Reply Last reply Reply Quote 0
                      • R Offline
                        Rags @Rags
                        last edited by

                        @Rags said in MMM-OpenAI:

                        @MMRIZE @MMRIZE
                        Thanks for the update. Yes now it works fine. I am not getting "something is wrong’ messages, but prompts about the token lengths whenever they exceed.
                        Thanks. I hope there will be voice integration soon.

                        So once again some issue has cropped up. The Openai integration with Telegrambot was working fine up to now. However, now the /chatai messages are not getting displayed on the MM whereas they are getting displayed on the Telegrambot app on the phone. The /txtai and /imgai have no issues. Please help in getting it to work again.

                        R M 2 Replies Last reply Reply Quote 0
                        • 1
                        • 2
                        • 3
                        • 2 / 3
                        • First post
                          Last post
                        Enjoying MagicMirror? Please consider a donation!
                        MagicMirror created by Michael Teeuw.
                        Forum managed by Sam, technical setup by Karsten.
                        This forum is using NodeBB as its core | Contributors
                        Contact | Privacy Policy