read csv-data and put it in an array

[sdetweil]

@Perlchamp should already be an array of row(of column name/value pairs)…

the JSON class has stringify(object) to convert json to text,

console.log(JSON.stringify(results of convert))

also has parse (from text form of json to object)

[Perlchamp]

ok, now i have a dictionary (is that called though?) - always 4 columns (i will change that, if i read how to put it all in an array. in the moment i just have the output via console …)

[{"Geburtsmonat":"Januar","Geburtstag":"3","Geburtsjahr":"1940","Name":"Renate Stögger"},{...},{...}]

but i’m on the right way, thanks again :-)

[sdetweil]

@Perlchamp

[{"

array [ of objects {}, {}. {} ]
the names of the object elements are the names of the columns (row 1 in the csv)

[Perlchamp]

@sdetweil said in read csv-data and put it in an array:

the names of the object elements are the names of the columns (row 1 in the csv)

yes i know.
i thought it’s an dictionairy and not an array. but it’s ok …

[sdetweil]

@Perlchamp the [ (open square bracket) means array in javascript

the array can be numbers, strings or objects (open squiggly bracket)

[Perlchamp]

status:
i wanted to “play” with the jsonArray, but things like jsonArray[“birth”][0] or jsonArray.birth[3] will not be accepted, although it is listed in many websites … i try and try and read and read, but in the moment no solution. the parameter “length” count the number of letters and numbers ?
then I will have to deal with moment () first …

[sdetweil]

@Perlchamp u did not show the code, so I will call the output of the csvtojson() birthday_info

example of how to navigate the json data

in module there would be no ‘var’ cause of the way the modulename and helper are packaged inside object

// if not using numbers but month names 
// make indexOf return 1-12, not 0-11 (dummy thing never matched 1st)
// fix this with your language abbreviations
// usually in lower case, then always convert data to lowercase for compare to prevent having users remember to get the case right
var months= ["0", "jan","feb","mar"...."dec"]

var today_day_month = moment().format("DDMM")
// today would be "3004"
// tomorrow will be "0105"
// loop thru the array of birthday info from file, one 'row' per birthday

for(var birthday of birthday_info) {
     // get the day/month string in numbers to match output of moment()
     // get the last 2 chars of the day and month with a 0 on front.. 01-09, 12, ---31, or 01-09,10/11/12
     if( ("0"+birthday.Geburtstag).subString(-2)+("0"+months.indexOf(birthday.Geburtsmonat.toLowerCase())).subString(-2) 
                         == today_month_day) {
         // this birthday is for today
            condole.log(" birthday on "+ today_day _month+" is for "+birthday.Name);
     } 
}

``

[Perlchamp]

i didn’t post the code, because it only was a status-info …
but here is the code. i think i will first try alone. if there any questions i will post them.
here is my array:

[{"birth":"12.09.1967", "name":"Otto Billing"},{...}]

many thanks in advance

[sdetweil]

@Perlchamp ok, went to numbers, SO much easier!!

var today_day_month = moment().format("DD.MM")
// today would be "30.04"
// tomorrow will be "01.05"
// loop thru the array of birthday info from file, one 'row' per birthday

for(var birthday of birthday_info) {
     if( (birthday.birth.startsWith(today_day_month)) {
         // this birthday is for today
            console.log(" birthday on "+ today_day_month+" is for "+birthday.name);
     } 
}

[Perlchamp]

@sdetweil said in read csv-data and put it in an array:

ok, went to numbers, SO much easier!!

ok, i will try. therefor my array don’t match, i think … that’s right ?

[sdetweil]

@Perlchamp the new data matches my new code

[Perlchamp]

thanks, but now i’m a little bit confused, because my master problem will be, to reorg the array, to combine people with the same date of birth in as list in one row with the day of birth. but let me try …

[sdetweil]

@Perlchamp ok, this was just to give you ideas on how to access the data… in a useful way

[Perlchamp]

… and I’m very grateful to you for that. I learn from your messages every time. I think it’s great that there are people like you. that also costs time (apart from health, the most important thing in life). Thanks again for that!

[Perlchamp]

hi sam,
i have a major problem, that i don’t understand. this is my code in node_helper.js:

var NodeHelper = require("node_helper");
var moment = require("moment");

// add require of other javascripot components here
// var xxx = require('yyy'); here
const csv = require("csvtojson");
var birthdaysArray = [];


module.exports = NodeHelper.create({

	init(){
		console.log("init module helper perlchamp");
	},

	start() {
		console.log("Starting module helper: " + this.name);
		//	console.log("Pfad zur csv-Datei: " + this.path + "/data/birthdays.csv");
		
		const csvFilePath = this.path + '/data/birthdays.csv';
		csv()
		.fromFile(csvFilePath)
		.then((jsonObj)=>{      
			birthdaysArray = JSON.stringify(jsonObj);
			console.log("birthdaysArray: " + birthdaysArray);

			var today_day_month = moment().format("DD.MM");
			var today_month = moment().format("MM");
		})
		
		console.log("birthdaysArray #2: " + birthdaysArray);
		
		// moment.js
		moment.locale(config.language);  // set locale
		console.log("aktuelle Zeit: " + moment().format('Do MMMM YYYY, hh:mm:ss'));
		console.log("aktueller Monat(Zahl): " + moment().format('MM'));
		console.log("aktueller Monat(Text): " + moment().format('MMM'));
		console.log("aktueller Tag(Zahl mit führender Null): " + moment().format('DD'));
		console.log("aktueller Tag(Text): " + moment().format('dddd'));
		console.log("aktueller Tag(Abkürzung): " + moment().format('dd'));
		console.log("aktuelles Jahr(Zahl): " + moment().format('YYYY'));
	},

	stop(){
		console.log("Stopping module helper: " + this.name);
	},

	// handle messages from our module// each notification indicates a different messages
	// payload is a data structure that is different per message.. up to you to design this
	socketNotificationReceived(notification, payload) {
		//	console.log(this.name + " received a socket notification: " + notification + " - Payload: " + payload);
		console.log(this.name + " received a socket notification: " + birthdays);
		// if config message from module
		if (notification === "CONFIG") {
			// save payload config info
			this.config=payload
			// wait 15 seconds, send a message back to module
			setTimeout(()=> { this.sendSocketNotification("message_from_helper"," this is a test_message")}, 15000)
		}
		else if(notification === "????2") {
		}

	},
});

why returns

console.log("birthdaysArray #2: " + birthdaysArray);

nothing.

many thanks in advance

[Perlchamp]

console.log(this.name + " received a socket notification: " + birthdays);

should be

console.log(this.name + " received a socket notification: " + birthdaysArray);

of course. couldn’t edit it, because of spam-message from board …

[sdetweil]

@Perlchamp said in read csv-data and put it in an array:

> 	const csvFilePath = this.path + '/data/birthdays.csv';
> 		csv()
> 		.fromFile(csvFilePath)
> 		.then((jsonObj)=>{      
> 			birthdaysArray = JSON.stringify(jsonObj);
> 			console.log("birthdaysArray: " + birthdaysArray);
> 
> 			var today_day_month = moment().format("DD.MM");
> 			var today_month = moment().format("MM");
> 		})
> 		
> 		console.log("birthdaysArray #2: " + birthdaysArray);

welcome to programming with a asyncronous libraries

csv().fromFile(csvFilePath) starts a ‘long’ operation,
going out to disk, reading in the data, and converting it to json from spreadsheet… (long is milliseconds)

the processor should not wait, so it does not, and the next instruction AFTER the start of the fromFile()
is console.log("birthdaysArray #2: " + birthdaysArray);

	// moment.js
	moment.locale(config.language);  // set locale
	console.log("aktuelle Zeit: " + moment().format('Do MMMM YYYY, hh:mm:ss'));
	console.log("aktueller Monat(Zahl): " + moment().format('MM'));
	console.log("aktueller Monat(Text): " + moment().format('MMM'));
	console.log("aktueller Tag(Zahl mit führender Null): " + moment().format('DD'));
	console.log("aktueller Tag(Text): " + moment().format('dddd'));
	console.log("aktueller Tag(Abkürzung): " + moment().format('dd'));
	console.log("aktuelles Jahr(Zahl): " + moment().format('YYYY'));

and the start function runs out of things to do and returns back to MM…

sometime later , the fromFile() will complete, and call the .then() routine with the data

(jsonObj)=>{      
			birthdaysArray = JSON.stringify(jsonObj);
			console.log("birthdaysArray: " + birthdaysArray);

			var today_day_month = moment().format("DD.MM");
			var today_month = moment().format("MM");
		}

so, if you need to wait for the data(to prints is content) , then the operations would be inside the .then() handler.

[Perlchamp]

ok, thanks. to move the code is not heavy ;-) … i’ve found a snippte to sort my birthdaysArray, but that doesn’t match :

		const csvFilePath = this.path + '/data/birthdays.csv';
		csv()
		.fromFile(csvFilePath)
		.then((jsonObj)=>{      
			birthdaysArray = JSON.stringify(jsonObj);
			console.log("birthdaysArray: " + birthdaysArray);
			
		var result = Object.entries(birthdaysArray.reduce((a, {birth, name}) => {
			const day = +birth.split('.')[0];
			a[day] = [...(a[day] || []), name];
			return a
		}, {})).map(([day, name]) => ({day, name})).sort((a, b) => +a.day - b.day)
		console.log("sorted birthdays : " + result);

		var today_day_month = moment().format("DD.MM");
		var today_month = moment().format("MM");
		})

now i’m looking for another one. is the birthdaysArray a json-Array or a javascript-array or … someone told me, it’s an object-Array and that this has nothing to do witch json … but: csvtojson, so …

[Perlchamp]

so i’m bloody happy :

var NodeHelper = require("node_helper");
var moment = require("moment");

// add require of other javascripot components here
// var xxx = require('yyy'); here
const csv = require("csvtojson");
var birthdaysArray = [];


module.exports = NodeHelper.create({

	init(){
		console.log("init module helper perlchamp");
	},

	start() {
		console.log("Starting module helper: " + this.name);
		console.log("Pfad zur csv-Datei: ", this.path + "/data/birthdays.csv");
		
		const csvFilePath = this.path + '/data/birthdays.csv';
		csv()
		.fromFile(csvFilePath)
		.then((jsonObj)=>{      
			birthdaysArray = JSON.stringify(jsonObj);
			console.log("birthdaysArray: ", birthdaysArray);
			
		var result = Object.entries(jsonObj.reduce((a, {birth, name}) => {
			const day = +birth.split('.')[0];
			a[day] = [...(a[day] || []), name];
			return a
		}, {})).map(([day, name]) => ({day, name})).sort((a, b) => +a.day - b.day)
		console.log("sorted birthdays : ", result);

		var today_day_month = moment().format("DD.MM");
		var today_month = moment().format("MM");
		})
		
	},

	stop(){
		console.log("Stopping module helper: " + this.name);
	},

	// handle messages from our module// each notification indicates a different messages
	// payload is a data structure that is different per message.. up to you to design this
	socketNotificationReceived(notification, payload) {
		//	console.log(this.name + " received a socket notification: " + notification + " - Payload: " + payload);
		console.log(this.name + " received a socket notification: " + birthdaysArray);
		// if config message from module
		if (notification === "CONFIG") {
			// save payload config info
			this.config=payload
			// wait 15 seconds, send a message back to module
			setTimeout(()=> { this.sendSocketNotification("message_from_helper"," this is a test_message")}, 15000)
		}
		else if(notification === "????2") {
		}

	},
});

so now I just have to manage to filter out the people from the complete csv-file who have their birthday in the current month …

[Perlchamp]

if i run :

		for(var birthday of birthdaysArray) {
			if(birthday.birth.startsWith(today_day_month)) {
				// this birthday is for today
				console.log(" birthday on "+ today_day_month+" is for "+birthday.name);
			} 
		}

than i get the following error-messages:

 Unhandled rejection TypeError: Cannot read property 'startsWith' of undefined
    at /home/dirk/MagicMirror/modules/perlchamp/node_helper.js:33:22
    at Object.onfulfilled (/home/dirk/MagicMirror/node_modules/csvtojson/v2/Converter.js:112:33)
    at Result.endProcess (/home/dirk/MagicMirror/node_modules/csvtojson/v2/Result.js:83:50)
    at Converter.processEnd (/home/dirk/MagicMirror/node_modules/csvtojson/v2/Converter.js:179:21)
    at /home/dirk/MagicMirror/node_modules/csvtojson/v2/Converter.js:172:19
    at tryCatcher (/home/dirk/MagicMirror/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/home/dirk/MagicMirror/node_modules/bluebird/js/release/promise.js:547:31)
    at Promise._settlePromise (/home/dirk/MagicMirror/node_modules/bluebird/js/release/promise.js:604:18)
    at Promise._settlePromise0 (/home/dirk/MagicMirror/node_modules/bluebird/js/release/promise.js:649:10)
    at Promise._settlePromises (/home/dirk/MagicMirror/node_modules/bluebird/js/release/promise.js:729:18)
    at _drainQueueStep (/home/dirk/MagicMirror/node_modules/bluebird/js/release/async.js:93:12)
    at _drainQueue (/home/dirk/MagicMirror/node_modules/bluebird/js/release/async.js:86:9)
    at Async._drainQueues (/home/dirk/MagicMirror/node_modules/bluebird/js/release/async.js:102:5)
    at Immediate.Async.drainQueues [as _onImmediate] (/home/dirk/MagicMirror/node_modules/bluebird/js/release/async.js:15:14)
    at processImmediate (internal/timers.js:439:21)

I was probably happy too early ;-( … shit happens.