Read the statement by Michael Teeuw here.
ChatGpt intergration
-
@SILLEN-0 see my samplemodule
it sends text from helper to browser side and calls the mm functions to get content presented
for MagicMirror, a module creates a small segment of html content (in a div), and gives that to mm when it asks (getDom() function) and mm places the content in the module’s defined position in the predefined web page (see index.html and main.css for the structure)
-
There are several TTS modules, so output would not be a big issue.(And text-only-displaying would not be the issue at all.)
The real issue might be how to ask/input.
One idea is using telegramBot. Hmmm… interesting. -
as i have said before i am very bad at making custom code i have been stuggling for the past 2 days and i have a code that i thought would work but when i restart the magicmirror it just shows this:
its just stuck on “loading response…”
this is all my code:chatgpt.js:
Module.register("chatgpt", { // Default module config. defaults: { question: "What is the weather like today?" }, // Define start sequence. start: function() { Log.info("Starting module: " + this.name); this.sendSocketNotification("QUESTION", this.config.question); }, // Override socket notification handler. socketNotificationReceived: function(notification, payload) { if (notification === "RESPONSE") { this.response = payload; this.updateDom(); } }, // Override dom generator. getDom: function() { var wrapper = document.createElement("div"); if (this.response) { wrapper.innerHTML = this.response; } else { wrapper.innerHTML = "Loading response..."; } return wrapper; } });and here is my node_helper.js script:
const NodeHelper = require("node_helper"); const openai = require("openai"); module.exports = NodeHelper.create({ start: function() { console.log("Starting node helper for: " + this.name); }, // Send a message to the chatGPT API and receive a response getResponse: function(question) { openai.apiKey = "XXXXXXXXXXXXXXXXXXXXXXX"; openai.Completion.create({ prompt: question, temperature: 0.7 }, (error, response) => { if (error) { console.log(error); } else { return response.choices[0].text; } }); }, // Handle socket notifications socketNotificationReceived: function(notification, payload) { if (notification === "CONNECT") { var response = this.getResponse(payload.question); this.sendSocketNotification("ANSWER", response); } }, });and finally this is the config file code:
{ module: "chatgpt", position: "bottom_bar", classes: "always", config: { question: "What is the weather like today?", }, node_helper: "node_helper.js", },i dont understand why it doesnt show up if anyone has any idea why it would be appreciated.
this is my logs:
[13.01.2023 15:44.15.423] [LOG] Initializing new module helper ... [13.01.2023 15:44.15.424] [LOG] Module helper loaded: Gateway [13.01.2023 15:44.15.820] [LOG] Initializing new module helper ... [13.01.2023 15:44.15.820] [LOG] Module helper loaded: MMM-GoogleAssistant [13.01.2023 15:44.15.824] [LOG] Initializing new module helper ... [13.01.2023 15:44.15.824] [LOG] Module helper loaded: EXT-Detector [13.01.2023 15:44.16.275] [LOG] Initializing new module helper ... [13.01.2023 15:44.16.276] [LOG] Module helper loaded: EXT-Spotify [13.01.2023 15:44.16.519] [LOG] Initializing new module helper ... [13.01.2023 15:44.16.520] [LOG] Module helper loaded: EXT-YouTubeCast [13.01.2023 15:44.16.773] [LOG] Initializing new module helper ... [13.01.2023 15:44.16.773] [LOG] Module helper loaded: EXT-Librespot [13.01.2023 15:44.16.776] [LOG] Initializing new module helper ... [13.01.2023 15:44.16.777] [LOG] Module helper loaded: EXT-Browser [13.01.2023 15:44.16.777] [LOG] All module helpers loaded. [13.01.2023 15:44.16.790] [LOG] Starting server on port 8080 ... [13.01.2023 15:44.16.793] [WARN] You're using a full whitelist configuration to allow for all IPs [13.01.2023 15:44.16.809] [LOG] Server started ... [13.01.2023 15:44.16.810] [LOG] Connecting socket for: chatgpt [13.01.2023 15:44.16.811] [LOG] Starting node helper for: chatgpt [13.01.2023 15:44.16.826] [LOG] Connecting socket for: MMM-Remote-Control [13.01.2023 15:44.16.827] [LOG] Starting node helper for: MMM-Remote-Control [13.01.2023 15:44.16.833] [LOG] Connecting socket for: MMM-PushBulletNotifications [13.01.2023 15:44.16.834] [LOG] Connecting socket for: mmpm [13.01.2023 15:44.16.835] [LOG] Starting module helper: mmpm [13.01.2023 15:44.16.835] [LOG] Connecting socket for: MMM-Face-Reco-DNN [13.01.2023 15:44.16.836] [LOG] Starting module helper: MMM-Face-Reco-DNN [13.01.2023 15:44.16.836] [LOG] Connecting socket for: MMM-Skolmaten [13.01.2023 15:44.16.837] [LOG] Connecting socket for: updatenotification [13.01.2023 15:44.16.838] [LOG] Starting module helper: updatenotification [13.01.2023 15:44.16.839] [LOG] Connecting socket for: Gateway [13.01.2023 15:44.16.841] [LOG] Connecting socket for: MMM-GoogleAssistant [13.01.2023 15:44.16.842] [LOG] [GA] Read config.js and check ConfigDeepMerge... [13.01.2023 15:44.16.843] [LOG] [GA] Perfect ConfigDeepMerge activated! [13.01.2023 15:44.16.844] [LOG] Connecting socket for: EXT-Detector [13.01.2023 15:44.16.845] [LOG] [DETECTOR] EXT-Detector Version: 1.0.1 rev: 220315 [13.01.2023 15:44.16.846] [LOG] Connecting socket for: EXT-Spotify [13.01.2023 15:44.16.847] [LOG] Connecting socket for: EXT-YouTubeCast [13.01.2023 15:44.16.848] [LOG] Starting module helper: EXT-YouTubeCast [13.01.2023 15:44.16.848] [LOG] Connecting socket for: EXT-Librespot [13.01.2023 15:44.16.850] [LOG] Connecting socket for: EXT-Browser [13.01.2023 15:44.16.850] [LOG] Starting module helper: EXT-Browser [13.01.2023 15:44.16.851] [LOG] Sockets connected & modules started ... [13.01.2023 15:44.17.083] [LOG] Launching application. [13.01.2023 15:44.19.211] [ERROR] ERROR! Could not validate main module js file. [13.01.2023 15:44.19.218] [ERROR] ReferenceError: Log is not defined at Object.<anonymous> (/home/pi/MagicMirror/modules/MMM-RAIN-MAP/MMM-RAIN-MAP.js:28:151853) at Module._compile (node:internal/modules/cjs/loader:1141:14) at Module._extensions..js (node:internal/modules/cjs/loader:1196:10) at Module.load (node:internal/modules/cjs/loader:1011:32) at Module._load (node:internal/modules/cjs/loader:846:12) at f._load (node:electron/js2c/asar_bundle:2:13328) at Module.require (node:internal/modules/cjs/loader:1035:19) at require (node:internal/modules/cjs/helpers:102:18) at Class.loadModuleDefaultConfig (/home/pi/MagicMirror/modules/MMM-Remote-Control/node_helper.js:313:30) at /home/pi/MagicMirror/modules/MMM-Remote-Control/node_helper.js:267:26 [13.01.2023 15:44.25.545] [LOG] [MMM-PushBulletNotifications][Info] 15:44:25.456 - START received [13.01.2023 15:44.25.712] [INFO] Checking git for module: chatgpt [13.01.2023 15:44.25.724] [LOG] [GATEWAY] Gateway Version: 2.2.4 rev: 230105 [13.01.2023 15:44.25.750] [LOG] [GA] MMM-GoogleAssistant Version: 4.0.8 rev: 221210 [13.01.2023 15:44.25.756] [LOG] [GA] Platform: 'raspberry-pi'; attempting to use 'arecord' to access microphone ... [13.01.2023 15:44.25.766] [LOG] [GA] RECIPE_LOADED: ../../EXT-Spotify/recipe/EXT-Spotify.js [13.01.2023 15:44.25.770] [LOG] [GA] RECIPE_LOADED: ../../MMM-NewsFeed/recipe/MMM-NewsFeed.js [13.01.2023 15:44.25.778] [LOG] [GA] RECIPE_LOADED: grej.js [13.01.2023 15:44.25.782] [LOG] [GA] RECIPE_LOADED: iss.js [13.01.2023 15:44.25.785] [LOG] [GA] Google Assistant is initialized. [13.01.2023 15:44.25.789] [LOG] [SPOTIFY] EXT-Spotify Version: 2.0.5 rev: 221127 [13.01.2023 15:44.25.827] [LOG] [CAST] EXT-YouTubeCast Version: 1.0.4 rev: 220805 [13.01.2023 15:44.25.841] [LOG] [CAST] { debug: true, fullscreen: true, alwaysDisplayed: false, width: '30vw', height: '30vh', castName: 'MagiskaSpegeln', port: 8569 } [13.01.2023 15:44.25.847] [LOG] [CAST] Starting Cast module... [13.01.2023 15:44.25.872] [LOG] [LIBRESPOT] EXT-Librespot Version: 1.0.3 rev: 220815 [13.01.2023 15:44.25.874] [LOG] [LIBRESPOT] Launch Librespot... [13.01.2023 15:44.25.896] [LOG] [LINKS] EXT-Browser Version: 1.0.0 rev: 220228 [13.01.2023 15:44.25.908] [LOG] [CAST] MagiskaSpegeln is listening on port 8569 [13.01.2023 15:44.26.317] [INFO] Checking git for module: MMM-Remote-Control [13.01.2023 15:44.26.420] [LOG] [DETECTOR] All needed @bugsounet library loaded ! [13.01.2023 15:44.26.424] [LOG] [DETECTOR] Platform: 'raspberry-pi'; attempting to use 'arecord' to access microphone ... [13.01.2023 15:44.26.432] [LOG] [DETECTOR] Snowboy is initialized with 1 Models: jarvis [13.01.2023 15:44.26.631] [LOG] [GATEWAY] Loaded: node-pty [13.01.2023 15:44.27.247] [LOG] [GATEWAY] Loaded: ./tools/tools.js [13.01.2023 15:44.27.364] [LOG] [DETECTOR] Starts listening. 1 Models [13.01.2023 15:44.27.372] [LOG] [GATEWAY] Start app... [13.01.2023 15:44.27.373] [LOG] [GATEWAY] EXT plugins in database: 34 [13.01.2023 15:44.27.373] [WARN] [GATEWAY] WARN: You are using default username or default password [13.01.2023 15:44.27.374] [WARN] [GATEWAY] WARN: Don't forget to change it! [13.01.2023 15:44.27.410] [LOG] [GATEWAY] Find 10 installed plugins in MagicMirror [13.01.2023 15:44.27.411] [LOG] [GATEWAY] Find 6 configured plugins in config file [13.01.2023 15:44.27.415] [LOG] [GATEWAY] Find MMM-GoogleAssistant v4.0.8 [13.01.2023 15:44.27.415] [LOG] [GATEWAY] Find MMM-GoogleAssistant configured in MagicMirror [13.01.2023 15:44.27.416] [LOG] [GATEWAY] webviewTag Configured: true [13.01.2023 15:44.27.417] [LOG] [GATEWAY] Language set undefined [13.01.2023 15:44.27.443] [LOG] [GATEWAY] Create all needed routes... [13.01.2023 15:44.28.541] [LOG] [HyperWatch] Logger is enabled [13.01.2023 15:44.28.547] [LOG] [GATEWAY] Start listening on http://192.168.1.126:8081 [13.01.2023 15:44.28.825] [INFO] Checking git for module: MMM-PushBulletNotifications [13.01.2023 15:44.29.174] [LOG] [CAST] PONG YouTube { name: 'YouTube', state: 'stopped', allowStop: true, pid: null, launch: [Function: launch] } [13.01.2023 15:44.29.687] [INFO] Checking git for module: MMM-Face-Multi-User-Recognition-SMAI [13.01.2023 15:44.29.820] [LOG] [LIBRESPOT] Connected! [13.01.2023 15:44.30.069] [INFO] Checking git for module: mmpm [13.01.2023 15:44.30.071] [INFO] Checking git for module: MMM-Face-Reco-DNN [13.01.2023 15:44.30.258] [LOG] [LIBRESPOT] Librespot already launched [13.01.2023 15:44.30.414] [INFO] Checking git for module: MMM-RAIN-MAP [13.01.2023 15:44.30.548] [INFO] Checking git for module: MMM-Weather-SMHI [13.01.2023 15:44.30.670] [INFO] Checking git for module: MMM-Skolmaten [13.01.2023 15:44.30.806] [INFO] Checking git for module: Gateway [13.01.2023 15:44.30.981] [INFO] Checking git for module: MMM-GoogleAssistant [13.01.2023 15:44.31.077] [LOG] [MMM-PushBulletNotifications][Info] 15:44:31.75 - PushBullet connected [13.01.2023 15:44.31.087] [INFO] Checking git for module: EXT-Detector [13.01.2023 15:44.31.183] [INFO] Checking git for module: EXT-Spotify [13.01.2023 15:44.31.297] [INFO] Checking git for module: EXT-YouTubeCast [13.01.2023 15:44.31.407] [INFO] Checking git for module: EXT-Librespot [13.01.2023 15:44.31.582] [INFO] Checking git for module: EXT-Browser [13.01.2023 15:44.31.697] [INFO] Checking git for module: default [13.01.2023 15:44.34.493] [LOG] [MMM-Face-Reco-DNN] loading encodings + face detector... [13.01.2023 15:44.34.551] [LOG] [MMM-Face-Reco-DNN] starting video stream... -
@SILLEN-0 Log is not defined in Rainmap, nothing after that runs
only in browser side
best when doing your dev is to only use your module
add console.log stmts to your node_helper so u can see it’s progress in the console output
-
yea i fixed that problem but now with the same script it is just stuck at loading response
-
@SILLEN-0 look at the logs… loading means data did not arrive to replace the dummy message
-
@SILLEN-0 did the node helper get a response from lib?
-
i think the problem is with node helper not actually sending a api request because if i look in usage tab on the openai api page it should show activity but it says that it has not recieved any api requests:
but it should work! im going too look into it now and see if i can fix it. -
@SILLEN-0 as I said add console.log statements to the node helper so you can see the flow in the console output…
-
yea i just did that and i get this error in the console:
0|MagicMirror | [14.01.2023 13:01.03.714] [LOG] Received QUESTION notification with payload: What is the weather like today? 0|MagicMirror | [14.01.2023 13:01.03.724] [LOG] Error in getResponse: TypeError: Cannot set properties of undefined (setting 'apiKey') 0|MagicMirror | at /home/pi/MagicMirror/modules/chatgpt/node_helper.js:12:27 0|MagicMirror | at new Promise (<anonymous>) 0|MagicMirror | at Class.getResponse (/home/pi/MagicMirror/modules/chatgpt/node_helper.js:11:16) 0|MagicMirror | at Class.socketNotificationReceived (/home/pi/MagicMirror/modules/chatgpt/node_helper.js:31:18) 0|MagicMirror | at Socket.<anonymous> (/home/pi/MagicMirror/js/node_helper.js:108:11) 0|MagicMirror | at Socket.emit (node:events:513:28) 0|MagicMirror | at Socket.emitUntyped (/home/pi/MagicMirror/node_modules/socket.io/dist/typed-events.js:69:22) 0|MagicMirror | at /home/pi/MagicMirror/node_modules/socket.io/dist/socket.js:614:39 0|MagicMirror | at process.processTicksAndRejections (node:internal/process/task_queues:78:11)and this is my node_helper file:
const NodeHelper = require("node_helper"); const openai = require("openai").default; console.log(openai); module.exports = NodeHelper.create({ start: function() { console.log("Starting node helper for: " + this.name); }, // Send a message to the chatGPT API and receive a response getResponse: function(question) { return new Promise((resolve, reject) => { openai.apiKey = "XXXXXXXXXXXXXX"; openai.Completion.create({ prompt: question, temperature: 0.7 }, (error, response) => { if (error) { reject(error); } else { console.log("Received response from API: ", response); resolve(response.choices[0].text); } }); }); }, // Handle socket notifications socketNotificationReceived: function(notification, payload) { if (notification === "QUESTION") { console.log("Received QUESTION notification with payload: ", payload); this.getResponse(payload) .then((response) => { console.log("Sending RESPONSE notification with payload: ", response); this.sendSocketNotification("RESPONSE", response); }) .catch((error) => { console.log("Error in getResponse: ", error); }); } }, }); -
@SILLEN-0 was openainin context before the
return new Promise inside getResponse() -
@sdetweil if im honest i have no idea what you mean by that. as i said i am very bad at any sort of coding this whole thing is held up with ducktape could you try to explain more what you mean by that?
-
@SILLEN-0 the error says that at the time of trying to to assign the apiKey to the openai object, the openai object is null (0)
TypeError: Cannot set properties of undefined (setting 'apiKey') 0|MagicMirror | at /home/pi/MagicMirror/modules/chatgpt/node_helper.js:12:27‘undefined’ here is openai,
so, I see you print the object just after the require() at the top
const openai = require("openai").default; console.log(openai);and ‘assume’ that you looked at the output of npm start (where console.log messages go) and are satisfied that the require(‘openai’) worked
so now to check again later
so I would add and another console.log(openai)
after the getResponse:getResponse: function(question) { // here .. is the openai object good (not null) here? return new Promise((resolve, reject) => { openai.apiKey = "XXXXXXXXXXXXXX"; // this is the statement that failed -
HOLY SHIT I GOT IT TOO WORK!. ok i need to calm down i am so happy right now. so turn out i was just trying too call the everything with openai.apiKey instead of just apiKey and then i had too do somthing else i dont even remember what i did but now i got it too work and it displays on the magicmirror. but now comes the hard part. if you look in the code there is a variabele named question that the prompt uses and send too the api. and i am no expert but can you have the variable question be defined with some kind of speech too text thing?
-
@SILLEN-0 said in ChatGpt intergration:
and i am no expert but can you have the variable question be defined with some kind of speech too text thing?
i don’t know what you mean…
unless u want to capture speech and convert that to the text for openai
welcome to the problem I have highlighted since the beginning… there is no GOOD speech capture library, and nothing built in.
this is why the MMM-GoogleAssistant provides mechanisms (recipe) to use the captured text for non- google uses.
MMM-Voice uses the pocketsphinx lib, from carnegie mellon, I think this one is terrible for me, <60% accurate and I have to keep saying it over and over.
GA and most others use cloud based services (none free)
but put a filter (hotword, like ok google, or alexa) in front to keep the cost down, not convert everything.the other mirror platform I support , smart-mirror , is voice based, so your ‘plugin’ can get the text from the speech. not compatible with MM.
-
i have MMM-googleassistant and i have used the recipes for some other stuff but could you get that too work with chatgpt. saying somthing like "ok-google chatgpt followed by the question you want too ask and then it takes the question makes it a variable or somthing that the prompt can read and then refresh the answer on the magicmirror. also what do you mean by (none free) MMM-google assistant is free. but i think it wouldnt be a big issue having too pay for chat gpt as in all of my testing and making api calls i have only paid 0.09 usd
-
@SILLEN-0 so to use ga recipe it would send a notification and your module side would get that and send that text to node_helper… rest is the same
-
@SILLEN-0 meaning of life. I assume service has blocked killer questions
-
@sdetweil any idea on how to do that? Also i fixed it i was just stupid i changed the default question if I had nothing else in the config.js but in there i had the question still set too what’s the weather like and when i changed that everything worked
-
@SILLEN-0 how to do what?
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login