read csv-data and put it in an array

Perlchamp

hallo sam :-)
i have now this in my node_helper.js:

var NodeHelper = require("node_helper");

// add require of other javascripot components here
// var xxx = require('yyy'); here
const csv = require("csvtojson");
const csvFilePath = this.path + 'birthdaylist.csv';


module.exports = NodeHelper.create({

	init(){
		console.log("init module helper perlchamp");
	},

	start() {
		console.log("Starting module helper: ${this.name}");
		console.log("dieser Pfad: ${this.path}");
		
		csv()
		.fromFile(csvFilePath)
		.then((jsonObj)=>{
			console.log(jsonObj);
		})

	},

/*
	stop(){
		console.log(`Stopping module helper: ${this.name}`);
	},
*/
	// handle messages from our module// each notification indicates a different messages
	// payload is a data structure that is different per message.. up to you to design this
	socketNotificationReceived(notification, payload) {
		console.log(this.name + " received a socket notification: " + notification + " - Payload: " + payload);
		// if config message from module
		if (notification === "CONFIG") {
			// save payload config info
			this.config=payload
			// wait 15 seconds, send a message back to module
			setTimeout(()=> { this.sendSocketNotification("message_from_helper"," this is a test_message")}, 15000)
		}
		else if(notification === "????2") {
		}

	},

});

terminal says:

Unhandled rejection Error: File does not exist. Check to make sure the file path to your csv is correct.
    at /home/dirk/MagicMirror/node_modules/csvtojson/v2/Converter.js:81:37
    at suppressedCallback (fs.js:209:5)
    at FSReqCallback.oncomplete (fs.js:153:23)

it’s from your SampleModule …

how can i see the console.log statements in the browser, starting MM ?
did i enter the code in the right place?
if I enter the following into the terminal window:
node console.log (‘path: $ {this.path}’)
i got an error message:
bash: Syntaxerror unexpected word `(’
what am I doing wrong ?

many thanks in advance

sdetweil

@Perlchamp said in read csv-data and put it in an array:

how can i see the console.log statements in the browser, starting MM ?
cannot see node_helper messages in browser, must look in the terminal window where u did npm start
you can redirect those messages to a file
npm start >somefile
and then look at somefile live with editor
did i enter the code in the right place?
const csvFilePath = this.path + ‘birthdaylist.csv’;
u have this.path outside the NodeHelper.create, at that time ‘this’ does not exist
‘this’ means ‘instance of’, and u are outside the thing that creates the instance
if I enter the following into the terminal window:

because u are not running in magic mirror , ‘this’ does not exist
node console.log (‘path: $ {this.path}’)

i got an error message:
bash: Syntaxerror unexpected word `(’
what am I doing wrong…
bash would not see any ( unless u put it on the line with node

node (something)
that is a command to bash (the terminal window program, he has his own rules about parens, and other special characters (* means ALL filenames in this folder for example!)

I never ever use $ in node_helper (or anywhere else)
I would do this
console.log (‘path: '+ this.path)

Perlchamp

thank you. ok, i will test …

only for your information:

start() {
		console.log(`Starting module helper: ${this.name}`);
	},

	stop(){
		console.log(`Stopping module helper: ${this.name}`);
	},

this is from your SampleModule. i’m not here to reprimand you. it should only be information. so please do not misunderstand.

sdetweil

@Perlchamp thanks… will fix… never copy from another without review!!

Perlchamp

always my pleasure

sdetweil

@Perlchamp fixed

sdetweil

if u redirect the output of npm start to a file, this is what you will see in the file


> magicmirror@2.11.0 start /home/sam/MagicMirror
> DISPLAY="${DISPLAY:=:0}" ./node_modules/.bin/electron js/electron.js

[2020-04-30 11:15:53.521] [LOG]    Starting MagicMirror: v2.11.0
[2020-04-30 11:15:53.524] [LOG]    Loading config ...
[2020-04-30 11:15:53.526] [LOG]    Loading module helpers ...
[2020-04-30 11:15:53.528] [LOG]    Initializing new module helper ...
[2020-04-30 11:15:53.528] [LOG]    Module helper loaded: MMM-Logging
[2020-04-30 11:15:53.529] [LOG]    No helper found for module: alert.
[2020-04-30 11:15:53.529] [LOG]    No helper found for module: clock.
[2020-04-30 11:15:53.663] [LOG]    Initializing new module helper ...
[2020-04-30 11:15:53.663] [LOG]    Module helper loaded: MyCovid19
[2020-04-30 11:15:53.784] [LOG]    Initializing new module helper ...
[2020-04-30 11:15:53.785] [LOG]    Module helper loaded: MMM-AlexaControl
[2020-04-30 11:15:53.786] [LOG]    handler helper in init
[2020-04-30 11:15:53.786] [LOG]    Module helper loaded: MMM-SleepWake
[2020-04-30 11:15:53.786] [LOG]    All module helpers loaded.
[2020-04-30 11:15:53.820] [LOG]    Starting server on port 8090 ... 
[2020-04-30 11:15:53.821] [INFO]   [33mYou're using a full whitelist configuration to allow for all IPs[39m
[2020-04-30 11:15:53.823] [LOG]    Server started ...
[2020-04-30 11:15:53.823] [LOG]    Connecting socket for: MMM-Logging
[2020-04-30 11:15:53.824] [LOG]    Module helper started for MMM-Logging
[2020-04-30 11:15:53.824] [LOG]    Connecting socket for: MyCovid19
[2020-04-30 11:15:53.824] [LOG]    Starting node_helper for module: MyCovid19
[2020-04-30 11:15:53.824] [LOG]    path=/home/sam/MagicMirror/modules/MyCovid19
[2020-04-30 11:15:53.824] [LOG]    Connecting socket for: MMM-AlexaControl
[2020-04-30 11:15:53.824] [LOG]    MMM-AlexaControl helper, started...
[2020-04-30 11:15:53.824] [LOG]    Connecting socket for: MMM-SleepWake
[2020-04-30 11:15:53.825] [LOG]    handler helper in start
[2020-04-30 11:15:53.825] [LOG]    Sockets connected & modules started ...
[2020-04-30 11:15:53.906] [LOG]    Launching application.
[2020-04-30 11:15:53.928] [LOG]    setup for latent showing  // < --- from my module node_helper
[2020-04-30 11:15:54.487] [LOG]    requested file exists=modules/MyCovid19/states-rawdata-04-30-2020 sending back to module_13_MyCovid19_5731  // < --- from my module node_helper
[2020-04-30 11:15:54.490] [LOG]    requested file exists=modules/MyCovid19/countries-rawdata-04-30-2020 sending back to module_16_MyCovid19_7951  // < --- from my module node_helper
[2020-04-30 11:15:54.492] [LOG]    requested file exists=modules/MyCovid19/countries-rawdata-04-30-2020 sending back to module_14_MyCovid19_1721  // < --- from my module node_helper
[2020-04-30 11:15:54.493] [LOG]    requested file exists=modules/MyCovid19/countries-rawdata-04-30-2020 sending back to module_15_MyCovid19_91  // < --- from my module node_helper
[2020-04-30 11:15:54.498] [LOG]    requested file exists=modules/MyCovid19/countries-rawdata-04-30-2020 sending back to module_17_MyCovid19_2191  // < --- from my module node_helper
etc

Perlchamp

ok, it’s slowly dawning on me, see little light at the end of the tunnel

sdetweil

@Perlchamp if u install the logging module I suggested, then it captures all the messages from the modulename.js in the browser and prints them to the terminal window log, which would be saved to the file as well.

sdetweil

@sdetweil a node_helper has two built in functions (I have never useed init() or stop() for anything, but they are available)

start: when the helper is loaded
socketNotificationReceived, when the modulename.js sends a message to the helper

the helper has one useful function

sendSocketNotification , to send a message back to the modulename.js

everything else is all up to you

note that the modulename.js gets the config.js config:{} info for this module, NOT the node_helper… so if the node_helper needs to know (filenames, timings, ???) , the modulename.js has to send a message with the this.config info down to the helper, and the helper has to save it to use later

Perlchamp

@sdetweil said in read csv-data and put it in an array:

if u install the logging module I suggested, then it captures all the messages from the modulename.js in the browser and prints them to the terminal window log, which would be saved to the file as well.

so i also can do (?):
npm start dev
in the window on the right klicking on “console”. there i see a lot of massages coming from console/moduls
and then i can use your module to lock out other modules (?) …

sdetweil

@Perlchamp

so i also can do (?):
npm start dev

yes

can use your module to lock out other modules (

no… it collects all messages. so I add my module name to them…

on windows the notepad++ editor allows you to find all, and gives a new window with the lines
from where the find was done and allows u to click on the find results and navigate to the original file…
very helpful when there is a big log and maybe only 1 or 2 specific messages u are looking for

Perlchamp

ok, understand

Perlchamp

i have another information for you (github - MMM-Logging):

You can find more detailed information on debugging your MagicMirror here: Module Debugging. If you also want the Electron rederer (web browser) console logs to be printed to the standard console (or PM2 logs), change the following line to the very top of ~/MagicMirror/run-start.sh <<

this file does not exist. i can’t find it in the file-manager (i see all hidden files) …

sdetweil

@Perlchamp yeh, that shell script was removed in 2.11… again I copied someone elses module and improved it for me.

you can do that

export ELECTRON_ENABLE_LOGGING=true

in the terminal window where you will do npm start

but with my changes, I don’t think u need to do that instruction anymore…

Perlchamp

ok, that’s fine

Perlchamp

ok, now i can see the output of my csvtojson object. now the question about handle this all:
is it better (csv-file) to call the rows ‘month of birth’, ‘birthday’, ‘year of birth’, ‘name’
=> mai,23,1964,stan smith
or ‘birth’,name
=> 05.23.1964,stan smith

how said:
i want to display the birthdays of the current month, then the name and the age. if people should have birthday at the same day, they should listed, in which the daynumber only displayed once (=> two cells).
what should happen, if the day of birth is over, i don’t know yet. there will be two options, i think, between the user can choose:

delete the names on the list (with daynumber)
dimm these entries.
finally it should display like this:

sdetweil

@Perlchamp welcome to the fun

its really month and day… year you would only want to know for big event things… 18, 21, 30, 40,50, …

so, I would reorg the data as a hash (object) by date, and list [array] of people [‘Stan Smith’, ‘Joe Anyone’]

birthdays = {}
u can use the moment() object to get the month/day, or had code to extract the part of the date month/day,
birthdays[birth_date][‘names’] = [‘Stan Smith’, ‘Joe Anyone’]
then send the birthdays object up to modulename.js

modulename.js in getDom()
it uses moment().format(‘MMDD’) or (‘DDMM’) (same as node_helper did)
then
if( this.birthdays[date] !=undefined) // if there is something for today
then can add the names in a for loop to the html to display

and if you use moment() in modulename, u can add/subtract to look ahead, and it will format the right month/day,
and u can get the day name

today,
tomorrow
saturday
sunday,
next week…

Perlchamp

uff, now first i have to read some papers for understanding that stuff …
in the moment i only can see the output of the json-object via terminal. i don’t have them saved in an array or string. i wanted to use a async-method (by await), but it seems that this is the wrong way (error-messages) …
i think to put the data first in an array couldn’t be wrong so far ?

 {
    Geburtsmonat: 'März',
    Geburtstag: '7',
    Geburtsjahr: '1972',
    Name: 'Julia Ützglütz'
  },
{
...
}

could it ?

sdetweil

@Perlchamp should already be an array of row(of column name/value pairs)…

the JSON class has stringify(object) to convert json to text,

console.log(JSON.stringify(results of convert))

also has parse (from text form of json to object)