ChatGpt intergration

SILLEN 0 · Jan 8, 2023, 1:22 PM

@sdetweil yea i got no idea about coding but coudnt you make it so that you use a existing voice to text module and then put the text results as text in the api?

sdetweil · Jan 8, 2023, 1:31 PM

@SILLEN-0 sure, there are 2.

the pocketSphinx Carnegie Mellon all local , which for me is about 60% accurate, see MMM-Voice

and then there is the hosted language model
goggle and others, with some filter front end to NOT analyze EVERY sound, like
snowboy(deprecated)

this latter is really important, as mm is intended to be keyboard/mouse less.

basically the Ga/Alexa hotword model

this would be easy with the smart-mirror mirror platform which I support too. as it is voice oriented, and does all the work to present the text to the plugin… SM uses snowboy.

sdetweil · Jan 8, 2023, 1:34 PM

@SILLEN-0 AND presentation is in the browser while the API is in the helper, so you have to design and implement the exchange process and the UI side… a module is just one little component.

mm is a user compostable information panel, w independent providers (modules).

SILLEN 0 · Jan 8, 2023, 1:37 PM

@sdetweil ok but if i wanted to start with just a preconfigured text without voice input. how would i do that the simplest way. i have little too no coding and i could probobally make a script for displaying an answer in a command line but i have no idea on how to display anything on the magicmirror.

sdetweil · Jan 8, 2023, 1:49 PM

@SILLEN-0 simple is in the eye of the viewer…

you will have to write some code, some script or both

script could prompt, call code, and display,
code takes input from prompt and calls API, returns result

or do it all in code…

SILLEN 0 · Jan 8, 2023, 2:02 PM

@sdetweil if for a start i use this super simple code:
import openai

openai.Completion.create(
engine=“davinci”,
prompt=“Make a list of astronomical observatories:”
)

how would i make the answer show up on the magicmirror

sdetweil · Jan 8, 2023, 2:26 PM

@SILLEN-0 see my samplemodule

it sends text from helper to browser side and calls the mm functions to get content presented

for MagicMirror, a module creates a small segment of html content (in a div), and gives that to mm when it asks (getDom() function) and mm places the content in the module’s defined position in the predefined web page (see index.html and main.css for the structure)

MMRIZE · Jan 9, 2023, 11:55 AM

There are several TTS modules, so output would not be a big issue.(And text-only-displaying would not be the issue at all.)

The real issue might be how to ask/input.
One idea is using telegramBot. Hmmm… interesting.

SILLEN 0 · Jan 13, 2023, 3:05 PM

as i have said before i am very bad at making custom code i have been stuggling for the past 2 days and i have a code that i thought would work but when i restart the magicmirror it just shows this:

its just stuck on “loading response…”
this is all my code:

chatgpt.js:

Module.register("chatgpt", {
    // Default module config.
    defaults: {
        question: "What is the weather like today?"
    },

    // Define start sequence.
    start: function() {
        Log.info("Starting module: " + this.name);
        this.sendSocketNotification("QUESTION", this.config.question);
    },

    // Override socket notification handler.
    socketNotificationReceived: function(notification, payload) {
        if (notification === "RESPONSE") {
            this.response = payload;
            this.updateDom();
        }
    },

    // Override dom generator.
    getDom: function() {
        var wrapper = document.createElement("div");
        if (this.response) {
            wrapper.innerHTML = this.response;
        } else {
            wrapper.innerHTML = "Loading response...";
        }
        return wrapper;
    }
});

and here is my node_helper.js script:

const NodeHelper = require("node_helper");
const openai = require("openai");

module.exports = NodeHelper.create({
    start: function() {
        console.log("Starting node helper for: " + this.name);
    },

    // Send a message to the chatGPT API and receive a response
    getResponse: function(question) {
        openai.apiKey = "XXXXXXXXXXXXXXXXXXXXXXX";
        openai.Completion.create({
            prompt: question,
            temperature: 0.7
        }, (error, response) => {
            if (error) {
                console.log(error);
            } else {
                return response.choices[0].text;
            }
        });
    },

    // Handle socket notifications
    socketNotificationReceived: function(notification, payload) {
        if (notification === "CONNECT") {
            var response = this.getResponse(payload.question);
            this.sendSocketNotification("ANSWER", response);
        }
    },
});

and finally this is the config file code:

{
    module: "chatgpt",
    position: "bottom_bar",
classes: "always",
    config: {
        question: "What is the weather like today?",
    },
    node_helper: "node_helper.js",
},

i dont understand why it doesnt show up if anyone has any idea why it would be appreciated.

this is my logs:


[13.01.2023 15:44.15.423] [LOG]   Initializing new module helper ...
[13.01.2023 15:44.15.424] [LOG]   Module helper loaded: Gateway
[13.01.2023 15:44.15.820] [LOG]   Initializing new module helper ...
[13.01.2023 15:44.15.820] [LOG]   Module helper loaded: MMM-GoogleAssistant
[13.01.2023 15:44.15.824] [LOG]   Initializing new module helper ...
[13.01.2023 15:44.15.824] [LOG]   Module helper loaded: EXT-Detector
[13.01.2023 15:44.16.275] [LOG]   Initializing new module helper ...
[13.01.2023 15:44.16.276] [LOG]   Module helper loaded: EXT-Spotify
[13.01.2023 15:44.16.519] [LOG]   Initializing new module helper ...
[13.01.2023 15:44.16.520] [LOG]   Module helper loaded: EXT-YouTubeCast
[13.01.2023 15:44.16.773] [LOG]   Initializing new module helper ...
[13.01.2023 15:44.16.773] [LOG]   Module helper loaded: EXT-Librespot
[13.01.2023 15:44.16.776] [LOG]   Initializing new module helper ...
[13.01.2023 15:44.16.777] [LOG]   Module helper loaded: EXT-Browser
[13.01.2023 15:44.16.777] [LOG]   All module helpers loaded.
[13.01.2023 15:44.16.790] [LOG]   Starting server on port 8080 ... 
[13.01.2023 15:44.16.793] [WARN]  You're using a full whitelist configuration to allow for all IPs
[13.01.2023 15:44.16.809] [LOG]   Server started ...
[13.01.2023 15:44.16.810] [LOG]   Connecting socket for: chatgpt
[13.01.2023 15:44.16.811] [LOG]   Starting node helper for: chatgpt
[13.01.2023 15:44.16.826] [LOG]   Connecting socket for: MMM-Remote-Control
[13.01.2023 15:44.16.827] [LOG]   Starting node helper for: MMM-Remote-Control
[13.01.2023 15:44.16.833] [LOG]   Connecting socket for: MMM-PushBulletNotifications
[13.01.2023 15:44.16.834] [LOG]   Connecting socket for: mmpm
[13.01.2023 15:44.16.835] [LOG]   Starting module helper: mmpm
[13.01.2023 15:44.16.835] [LOG]   Connecting socket for: MMM-Face-Reco-DNN
[13.01.2023 15:44.16.836] [LOG]   Starting module helper: MMM-Face-Reco-DNN
[13.01.2023 15:44.16.836] [LOG]   Connecting socket for: MMM-Skolmaten
[13.01.2023 15:44.16.837] [LOG]   Connecting socket for: updatenotification
[13.01.2023 15:44.16.838] [LOG]   Starting module helper: updatenotification
[13.01.2023 15:44.16.839] [LOG]   Connecting socket for: Gateway
[13.01.2023 15:44.16.841] [LOG]   Connecting socket for: MMM-GoogleAssistant
[13.01.2023 15:44.16.842] [LOG]   [GA] Read config.js and check ConfigDeepMerge...
[13.01.2023 15:44.16.843] [LOG]   [GA] Perfect ConfigDeepMerge activated!
[13.01.2023 15:44.16.844] [LOG]   Connecting socket for: EXT-Detector
[13.01.2023 15:44.16.845] [LOG]   [DETECTOR] EXT-Detector Version: 1.0.1 rev: 220315
[13.01.2023 15:44.16.846] [LOG]   Connecting socket for: EXT-Spotify
[13.01.2023 15:44.16.847] [LOG]   Connecting socket for: EXT-YouTubeCast
[13.01.2023 15:44.16.848] [LOG]   Starting module helper: EXT-YouTubeCast
[13.01.2023 15:44.16.848] [LOG]   Connecting socket for: EXT-Librespot
[13.01.2023 15:44.16.850] [LOG]   Connecting socket for: EXT-Browser
[13.01.2023 15:44.16.850] [LOG]   Starting module helper: EXT-Browser
[13.01.2023 15:44.16.851] [LOG]   Sockets connected & modules started ...
[13.01.2023 15:44.17.083] [LOG]   Launching application.
[13.01.2023 15:44.19.211] [ERROR] ERROR! Could not validate main module js file.
[13.01.2023 15:44.19.218] [ERROR] ReferenceError: Log is not defined
    at Object.<anonymous> (/home/pi/MagicMirror/modules/MMM-RAIN-MAP/MMM-RAIN-MAP.js:28:151853)
    at Module._compile (node:internal/modules/cjs/loader:1141:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1196:10)
    at Module.load (node:internal/modules/cjs/loader:1011:32)
    at Module._load (node:internal/modules/cjs/loader:846:12)
    at f._load (node:electron/js2c/asar_bundle:2:13328)
    at Module.require (node:internal/modules/cjs/loader:1035:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Class.loadModuleDefaultConfig (/home/pi/MagicMirror/modules/MMM-Remote-Control/node_helper.js:313:30)
    at /home/pi/MagicMirror/modules/MMM-Remote-Control/node_helper.js:267:26
[13.01.2023 15:44.25.545] [LOG]   [MMM-PushBulletNotifications][Info] 15:44:25.456 - START received
[13.01.2023 15:44.25.712] [INFO]  Checking git for module: chatgpt
[13.01.2023 15:44.25.724] [LOG]   [GATEWAY] Gateway Version: 2.2.4 rev: 230105
[13.01.2023 15:44.25.750] [LOG]   [GA] MMM-GoogleAssistant Version: 4.0.8 rev: 221210
[13.01.2023 15:44.25.756] [LOG]   [GA] Platform: 'raspberry-pi'; attempting to use 'arecord' to access microphone ...
[13.01.2023 15:44.25.766] [LOG]   [GA] RECIPE_LOADED: ../../EXT-Spotify/recipe/EXT-Spotify.js
[13.01.2023 15:44.25.770] [LOG]   [GA] RECIPE_LOADED: ../../MMM-NewsFeed/recipe/MMM-NewsFeed.js
[13.01.2023 15:44.25.778] [LOG]   [GA] RECIPE_LOADED: grej.js
[13.01.2023 15:44.25.782] [LOG]   [GA] RECIPE_LOADED: iss.js
[13.01.2023 15:44.25.785] [LOG]   [GA] Google Assistant is initialized.
[13.01.2023 15:44.25.789] [LOG]   [SPOTIFY] EXT-Spotify Version: 2.0.5 rev: 221127
[13.01.2023 15:44.25.827] [LOG]   [CAST] EXT-YouTubeCast Version: 1.0.4 rev: 220805
[13.01.2023 15:44.25.841] [LOG]   [CAST] {
  debug: true,
  fullscreen: true,
  alwaysDisplayed: false,
  width: '30vw',
  height: '30vh',
  castName: 'MagiskaSpegeln',
  port: 8569
}
[13.01.2023 15:44.25.847] [LOG]   [CAST] Starting Cast module...
[13.01.2023 15:44.25.872] [LOG]   [LIBRESPOT] EXT-Librespot Version: 1.0.3 rev: 220815
[13.01.2023 15:44.25.874] [LOG]   [LIBRESPOT] Launch Librespot...
[13.01.2023 15:44.25.896] [LOG]   [LINKS] EXT-Browser Version: 1.0.0 rev: 220228
[13.01.2023 15:44.25.908] [LOG]   [CAST] MagiskaSpegeln is listening on port 8569
[13.01.2023 15:44.26.317] [INFO]  Checking git for module: MMM-Remote-Control
[13.01.2023 15:44.26.420] [LOG]   [DETECTOR] All needed @bugsounet library loaded !
[13.01.2023 15:44.26.424] [LOG]   [DETECTOR] Platform: 'raspberry-pi'; attempting to use 'arecord' to access microphone ...
[13.01.2023 15:44.26.432] [LOG]   [DETECTOR] Snowboy is initialized with 1 Models: jarvis
[13.01.2023 15:44.26.631] [LOG]   [GATEWAY] Loaded: node-pty
[13.01.2023 15:44.27.247] [LOG]   [GATEWAY] Loaded: ./tools/tools.js
[13.01.2023 15:44.27.364] [LOG]   [DETECTOR] Starts listening. 1 Models
[13.01.2023 15:44.27.372] [LOG]   [GATEWAY] Start app...
[13.01.2023 15:44.27.373] [LOG]   [GATEWAY] EXT plugins in database: 34
[13.01.2023 15:44.27.373] [WARN]  [GATEWAY] WARN: You are using default username or default password
[13.01.2023 15:44.27.374] [WARN]  [GATEWAY] WARN: Don't forget to change it!
[13.01.2023 15:44.27.410] [LOG]   [GATEWAY] Find 10 installed plugins in MagicMirror
[13.01.2023 15:44.27.411] [LOG]   [GATEWAY] Find 6 configured plugins in config file
[13.01.2023 15:44.27.415] [LOG]   [GATEWAY] Find MMM-GoogleAssistant v4.0.8
[13.01.2023 15:44.27.415] [LOG]   [GATEWAY] Find MMM-GoogleAssistant configured in MagicMirror
[13.01.2023 15:44.27.416] [LOG]   [GATEWAY] webviewTag Configured: true
[13.01.2023 15:44.27.417] [LOG]   [GATEWAY] Language set undefined
[13.01.2023 15:44.27.443] [LOG]   [GATEWAY] Create all needed routes...
[13.01.2023 15:44.28.541] [LOG]   [HyperWatch] Logger is enabled
[13.01.2023 15:44.28.547] [LOG]   [GATEWAY] Start listening on http://192.168.1.126:8081
[13.01.2023 15:44.28.825] [INFO]  Checking git for module: MMM-PushBulletNotifications
[13.01.2023 15:44.29.174] [LOG]   [CAST] PONG YouTube {
  name: 'YouTube',
  state: 'stopped',
  allowStop: true,
  pid: null,
  launch: [Function: launch]
}
[13.01.2023 15:44.29.687] [INFO]  Checking git for module: MMM-Face-Multi-User-Recognition-SMAI
[13.01.2023 15:44.29.820] [LOG]   [LIBRESPOT] Connected!
[13.01.2023 15:44.30.069] [INFO]  Checking git for module: mmpm
[13.01.2023 15:44.30.071] [INFO]  Checking git for module: MMM-Face-Reco-DNN
[13.01.2023 15:44.30.258] [LOG]   [LIBRESPOT] Librespot already launched
[13.01.2023 15:44.30.414] [INFO]  Checking git for module: MMM-RAIN-MAP
[13.01.2023 15:44.30.548] [INFO]  Checking git for module: MMM-Weather-SMHI
[13.01.2023 15:44.30.670] [INFO]  Checking git for module: MMM-Skolmaten
[13.01.2023 15:44.30.806] [INFO]  Checking git for module: Gateway
[13.01.2023 15:44.30.981] [INFO]  Checking git for module: MMM-GoogleAssistant
[13.01.2023 15:44.31.077] [LOG]   [MMM-PushBulletNotifications][Info] 15:44:31.75 - PushBullet connected
[13.01.2023 15:44.31.087] [INFO]  Checking git for module: EXT-Detector
[13.01.2023 15:44.31.183] [INFO]  Checking git for module: EXT-Spotify
[13.01.2023 15:44.31.297] [INFO]  Checking git for module: EXT-YouTubeCast
[13.01.2023 15:44.31.407] [INFO]  Checking git for module: EXT-Librespot
[13.01.2023 15:44.31.582] [INFO]  Checking git for module: EXT-Browser
[13.01.2023 15:44.31.697] [INFO]  Checking git for module: default
[13.01.2023 15:44.34.493] [LOG]   [MMM-Face-Reco-DNN] loading encodings + face detector...
[13.01.2023 15:44.34.551] [LOG]   [MMM-Face-Reco-DNN] starting video stream...

sdetweil · Jan 13, 2023, 4:41 PM

@SILLEN-0 Log is not defined in Rainmap, nothing after that runs

only in browser side

best when doing your dev is to only use your module

add console.log stmts to your node_helper so u can see it’s progress in the console output