MMM-Calendar not seem to load .ics

[sdetweil]

@Dennis-010 said in MMM-Calendar not seem to load .ics:

https://www.xxx.xx/ical TypeError: terminated

this seems to indicate a problem with the server not sending the required info. ( and terminating the connection before the complete transmission was received) . the new internal fetch is enforcing the spec, which the prior node-fetch did not.

can u identify what

https://www.xxx.xx

this is exactly?

[Dennis-010]

@sdetweil

This is the url of my Kerio mailserver. It is my own dedicated server running in NL.
You asked to xxx out the url right?

If the connection is terminated before completion of the transmission, why is it that if I just paste the url in the default browser on raspberry, I just get to download the .ics file? What am I missing here?

[sdetweil]

@Dennis-010 the browser has error recovery code that we don’t, probably. they’ve had fetch for a long time. years…

this is one of the.problens w trying to keep up with changes and general supportability .

[Dennis-010]

@sdetweil
Point is, it has worked before… like a charm for over 2 years.
Recently we moved to another house and we have not been using the mirror for 8-9 months, now I reinstalled the OS and modules and now the calendar stopped working.

Is there any way I can try and sort out what the source of this issue could be?

[sdetweil]

@Dennis-010 I hear you.

none of the code that needs to be debugged is ours(mm) it’s all lower layers.

let me see if there is a hacky way to help you out.

[Dennis-010]

@sdetweil

I’'l be happy to test and try out things or methods to fix this issue.
Thanks in advance

[sdetweil]

@Dennis-010

ok, lets try this

cd ~/MagicMirror/modules/default/calendar
cp calendarfetcher.js  savefetcher.js
nano calendarfetcher.js 

then insert this line

const Log = require("logger");
const fetchit = require('node-fetch');   //<----- add this line
const NodeHelper = require("node_helper");

then add this line

		const nodeVersion = Number(process.version.match(/^v(\d+\.\d+)/)[1]);
		let fetcher = null //<------ add this line
		let httpsAgent = null;

then later change this

		fetch(url, { headers: headers, agent: httpsAgent })

to this

		if (fetcher === null) {
			fetcher = fetchit(url, { headers: headers, agent: httpsAgent });
		}
		fetcher

then save, (ctrl-o)
and exit nano (ctrl-x)

then

cd ~/MagicMirror
npm install node-fetch@2

then npm start
calendar should come up
mine works…

[Dennis-010]

@sdetweil
This seem to work like a charm, thank you for fixing this.

Now…when I want to add 2 more url’s I get a new error:
For each of the new url. Both from the same domain.
is it possible to have a hacky line for that as well?

[ERROR] Calendar Error. Could not fetch calendar:  https://www.xxxxxxxx/ical/**url**/Calendar.ics RangeError: Invalid time value

[23.10.2023 22:56.15.687] [LOG]   Create new calendarfetcher for url: https://**url**/ical/**url**/Calendar.ics - Interval: 3600000
[23.10.2023 22:56.15.728] [LOG]   Create new calendarfetcher for url: https://**url**/ical/**url**/Calendar.ics - Interval: 3600000
[23.10.2023 22:56.15.733] [LOG]   Create new calendarfetcher for url: https://**url**/ical/**url**/Calendar.ics - Interval: 3600000

[sdetweil]

@Dennis-010 this sounds like an ics data format problem

change this line in config.js

logLevel: ["INFO", "LOG", "WARN", "ERROR"], 

to

logLevel: ["INFO", "LOG", "WARN", "ERROR","DEBUG"], 

then start MM like this

pm2 stop all
cd ~/MagicMirror
npm start > somefile.txt 2>&1

wait 15 seconds
then in the same window as where you did npm start do
ctrl-c
and then examine the somefile.txt to see where that error is…

i’d like to see the ics file entry for that event
npm start

[Dennis-010]

@sdetweil

Is this what you are looking for?

[23.10.2023 23:18.36.721] [ERROR] Calendar Error. Could not fetch calendar:  https://**url**/ical/**url**/Calendar.ics RangeError: Invalid time value
    at Date.toISOString (<anonymous>)
    at /home/dennis/MagicMirror/modules/default/calendar/calendarfetcherutils.js:312:28
    at Array.forEach (<anonymous>)
    at Object.filterEvents (/home/dennis/MagicMirror/modules/default/calendar/calendarfetcherutils.js:140:24)
    at /home/dennis/MagicMirror/modules/default/calendar/calendarfetcher.js:72:36
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[23.10.2023 23:18.38.049] [DEBUG] parsed data={"ecaf4d70-3f05-470f-a8f3-2566b4f2f1b5":{"type":"VTIMEZONE","params":[],"tzid":"Europe/Amsterdam","487c4058-cf12-4ad3-9d19-abf7c43101a2":{>
[23.10.2023 23:18.38.053] [DEBUG] There are 327 calendar entries.
[23.10.2023 23:18.38.055] [DEBUG] Processing entry...
[23.10.2023 23:18.38.058] [DEBUG] Processing entry...
[23.10.2023 23:18.38.060] [DEBUG] Event:

[23.10.2023 23:18.38.153] [ERROR] Calendar Error. Could not fetch calendar:  https://**url**/ical/**url**/Calendar.ics RangeError: Invalid time value
    at Date.toISOString (<anonymous>)
    at /home/dennis/MagicMirror/modules/default/calendar/calendarfetcherutils.js:312:28
    at Array.forEach (<anonymous>)
    at Object.filterEvents (/home/dennis/MagicMirror/modules/default/calendar/calendarfetcherutils.js:140:24)
    at /home/dennis/MagicMirror/modules/default/calendar/calendarfetcher.js:72:36
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[23.10.2023 23:18.39.055] [DEBUG] parsed data={"634747":{"type":"VEVENT","params":[],"dtstamp":"2018-03-19T19:38:00.000Z","uid":"634747","description":"Datum: 04/24/2018\nTijd: 19:30\n>
[23.10.2023 23:18.39.060] [DEBUG] There are 434 calendar entries.
[23.10.2023 23:18.39.061] [DEBUG] Processing entry...
[23.10.2023 23:18.39.061] [DEBUG] Event:
{"type":"VEVENT","params":[],"dtstamp":"2018-03-19T19:38:00.000Z","uid":"634747","description":"Datum: 04/24/2018\nTijd: 19:30\n\nTot dan!\n\nP.S.: Gebruik je GMAIL? Controleer of de t>
[23.10.2023 23:18.39.062] [DEBUG] start: Tue Apr 24 2018 19:30:00 GMT+0200 (Midden-Europese zomertijd)

[sdetweil]

@Dennis-010 thx… I meant download the ics file and find that entry

like this from my calendar

BEGIN:VEVENT
DTSTART:20160729T233000Z
DTEND:20160730T010000Z
DTSTAMP:20231023T212812Z
UID:2prgqt2idcgs5jjf0bvd557v6k@google.com
CREATED:20160912T121808Z
LAST-MODIFIED:20160912T121808Z
LOCATION:georgetown
SEQUENCE:0
STATUS:CONFIRMED
SUMMARY:Dinner at Monument Cafe
TRANSP:OPAQUE
END:VEVENT

[Dennis-010]

@sdetweil
I don’t think I understand what you’re asking.
What event you want me to find?

[sdetweil]

@Dennis-010 the one that throws the error in the somefile.txt, then look it up in the ICS file

[Dennis-010]

@sdetweil
Think I found it, will check back tomorrow.

BEGIN:VEVENT
DTSTAMP:20210413T203456Z
UID:E689AEB8C02C4E2CADD8C7D3D303CEAD0
DTSTART;TZID="Amsterdam, Belgrade, Berlin, Brussels, Budapest, Madrid, Paris, Prague, Stockholm":20210415T190000
DTEND;TZID="Amsterdam, Belgrade, Berlin, Brussels, Budapest, Madrid, Paris, Prague, Stockholm":20210415T210000
CLASS:PUBLIC
LOCATION:albert heijn
SUMMARY:xxx xxxx
SEQUENCE:10
RRULE:FREQ=DAILY;UNTIL=20210418T170000Z
EXDATE;TZID="Amsterdam, Belgrade, Berlin, Brussels, Budapest, Madrid, Paris, Prague, Stockholm":20210417T190000
EXDATE;TZID="Amsterdam, Belgrade, Berlin, Brussels, Budapest, Madrid, Paris, Prague, Stockholm":20210416T190000
EXDATE;TZID="Amsterdam, Belgrade, Berlin, Brussels, Budapest, Madrid, Paris, Prague, Stockholm":20210415T190000
BEGIN:VALARM
ACTION:DISPLAY
TRIGGER;RELATED=START:-PT15M
END:VALARM
END:VEVENT

{"type":"VEVENT","params":[],"uid":"E689AEB8C02C4E2CADD8C7D3D303CEAD0","start":"2021-04-15T17:00:00.000Z","datetype":"date-time","end":"2021-04-15T19:00:00.000Z","a94d8d2a-3>
[24.10.2023 00:16.22.001] [DEBUG] start: Thu Apr 15 2021 19:00:00 GMT+0200 (Midden-Europese zomertijd)
[24.10.2023 00:16.22.001] [DEBUG] end:: Thu Apr 15 2021 21:00:00 GMT+0200 (Midden-Europese zomertijd)
[24.10.2023 00:16.22.002] [DEBUG] duration: 7200000
[24.10.2023 00:16.22.002] [DEBUG] title: xxx xxxx
[24.10.2023 00:16.22.002] [DEBUG] Search for recurring events between: Tue Oct 24 2023 00:16:22 GMT+0200 (Midden-Europese zomertijd) and Tue Oct 22 2024 23:59:59 GMT+0200 (M>
[24.10.2023 00:16.22.051] [DEBUG] Title: xxx xxxx, with dates: [null,null,null,null]
[24.10.2023 00:16.22.051] [DEBUG] event.recurrences: [object Object]
[24.10.2023 00:16.22.055] [ERROR] Calendar Error. Could not fetch calendar:  https://url/ical/url/Calendar.ics RangeError: Invalid time value
    at Date.toISOString (<anonymous>)
    at /home/dennis/MagicMirror/modules/default/calendar/calendarfetcherutils.js:312:28
    at Array.forEach (<anonymous>)
    at Object.filterEvents (/home/dennis/MagicMirror/modules/default/calendar/calendarfetcherutils.js:140:24)
    at /home/dennis/MagicMirror/modules/default/calendar/calendarfetcher.js:72:36
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

[sdetweil]

@Dennis-010 awesome, will need you to run it again with one debugging change

in calendarfetchutils.js
change this

					Log.debug(`event.recurrences: ${event.recurrences}`);

to this

			Log.debug("event.recurrences:" +JSON.stringify(event.recurrences));

but I really think its THIS

DTSTART;TZID="Amsterdam, Belgrade, Berlin, Brussels, Budapest, Madrid, Paris, Prague, Stockholm":20210415T190000
DTEND;TZID="Amsterdam, Belgrade, Berlin, Brussels, Budapest, Madrid, Paris, Prague, Stockholm":20210415T210000

the until is outside the event window start/end (04/15, this is 04/18) 
RRULE:FREQ=DAILY;UNTIL=20210418T170000Z  

AND the event is in the past 2021
but the code tried to find recurrences THIS year

Tue Oct 24 2023 00:16:22 GMT+0200 (Midden-Europese zomertijd) and Tue Oct 22 2024 23:59:59 GMT+0200 (M>
[24.10.2023 00:16.22.051] [DEBUG] Title: xxx xxxx, with dates: [null,null,null,null]  <----- and returned junk.. those are bad dates. 

[sdetweil]

@Dennis-010 try this

cd ~/MagicMirror/modules/default/calendar
cp calendarfetcherutils.js  utils.js
nano calendarfetcherutils.js 

insert this line

						futureLocal = futureMoment.toDate(); // future
					}
					if(moment(endDate)>=moment(pastLocal)){ //<--- this line (approx line 283) 

						Log.debug(`Search for recurring events between: ${pastLocal} and ${futureLocal}`);
						const dates = rule.between(pastLocal, futureLocal, true, limitFunction);

and this line

							i
									description: description
								});
							}
						} // < ----- this line (approx line 454)
					}

this adds a check if the event end is after the calculated start (for include past events thats a year back) , then process the event, else skip it…

let me know

[Dennis-010]

@sdetweil
This did the trick, thank you very much.

You are a genius!!

[sdetweil]

@Dennis-010 said in MMM-Calendar not seem to load .ics:

You are a genius!!

well, i don’t know about that… I’ve just spent the last few years fixing bugs in there