@sdetweil https://github.com/MagicMirrorOrg/MagicMirror/issues/3384 raised.

Thanks Sam.

Matt

Hello,
I can’t see that this topic has been raised before, so I thought I’d start.

The Met Office (UK weather) has issued notice of a gateway migration project they’re doing. I think there will be changes needed to both the module code and account changes by magicmirror users.

The new gateway is now available (as of Jan 2024). Migration by 30th April 2024 is the deadline.

As I understand so far, the following (high-level) changes are required:
User

• In Met Office Datahub subscriptions, user will need to migrate their ‘legacy’ application to the new gateway
• user will then get new API key

Module
“You will need to update your client code to use our new APIs”. Changes include:

• base URL edit - this is set in the config, so may just be a documentation update
• request headers
• ensure you are using URL encoding

Docs for reference: https://datahub.metoffice.gov.uk/support/api-gateway-migration

I’m not a developer so I may be well off in my understanding. I’ve not read too far in to the documentation, either. Happy to close this topic if it’s already under discussion/development somewhere.

Cheers,
Matt

@sdetweil sorry that will be my bad use of language… will update

For anyone following…

Issue still persists. https://github.com/MagicMirrorOrg/MagicMirror/issues/3380 has been opened.

Symptoms:
Calendar events that would be expected to be displayed on the calendar module are not. pm2 restart and full OS reboot does not resolve. I am using a baikal caldav server, FYI, which other devices have no trouble.

Troubleshooting:
We can see in the magicmirror logs that the events from the ics file are being received by the calendarfetcher but they are not being shown in events to process in the UI. I cannot see any pattern as to which events are affected.

I’ve tried this with a fresh install of magicmirror and I’ve also tried my main magicmirror with a fresh nextcloud calendar. Same behavior.

I’ll update if we find a solution.
Cheers.

@sdetweil hi Sam. I can’t see the chat message with the utils file. Just the parser file. Would you send again please?

@sdetweil Hi Sam. Thank you. Still no joy.
Here is what is displayed on my mirror:

Yet here is what is in my calendar (as shown on my iphone):

You can see multiple events missing, but I can’t figure out any pattern as to what shows and what does not.
I added in your calendarfetcherutils.js and restarted. Same behavior. The logs show the mirror is seeing the ‘house viewing PB’ events, for example, but it’s not showing up at all in dev tools or on the display.

[23.01.2024 09:17.06.900] [DEBUG] Event:
{"type":"VEVENT","params":[],"created":"2024-01-19T11:31:14.000Z","end":"2024-01-26T18:00:00.000Z","dtstamp":"2024-01-22T15:11:06.000Z","start":"2024-01-26T17:30:00.000Z","datetype":"date-time","lastmodified":"2024-01-22T15:11:05.000Z","sequence":"0","summary":"House viewing PB","transparency":"OPAQUE","uid":"92DD34B0-0226-4928-B476-EF1A5AC18471","url":{"params":{"VALUE":"URI"},"val":""},"APPLE-CREATOR-IDENTITY":"com.apple.mobilecal","APPLE-CREATOR-TEAM-IDENTITY":"0000000000"}
[23.01.2024 09:17.06.901] [DEBUG] start: Fri Jan 26 2024 17:30:00 GMT+0000 (Greenwich Mean Time)
[23.01.2024 09:17.06.901] [DEBUG] end:: Fri Jan 26 2024 18:00:00 GMT+0000 (Greenwich Mean Time)
[23.01.2024 09:17.06.901] [DEBUG] duration: 1800000
[23.01.2024 09:17.06.901] [DEBUG] title: House viewing PB
[23.01.2024 09:17.06.902] [DEBUG] full day event =false
[23.01.2024 09:17.06.902] [DEBUG] initial tz=Europe/London
[23.01.2024 09:17.06.902] [DEBUG] corrected tz=Europe/London
[23.01.2024 09:17.06.902] [DEBUG] start date/time=Fri Jan 26 2024 17:30:00 GMT+0000 (Greenwich Mean Time)
[23.01.2024 09:17.06.903] [DEBUG] start offset=0
[23.01.2024 09:17.06.903] [DEBUG] start date/time w tz =Fri Jan 26 2024 17:30:00 GMT+0000 (Greenwich Mean Time)
[23.01.2024 09:17.06.903] [DEBUG] event date=Fri Jan 26 2024 17:30:00 GMT+0000 (Greenwich Mean Time)
[23.01.2024 09:17.06.904] [DEBUG] event offset=0 hour=17 event date=Fri Jan 26 2024 17:30:00 GMT+0000 (Greenwich Mean Time)
[23.01.2024 09:17.06.904] [DEBUG] adjustHours=0
[23.01.2024 09:17.06.904] [DEBUG] saving event: House viewing PB
[23.01.2024 09:17.06.904] [DEBUG] Processing entry...

Dev tools:

before filter events=[{"title":"Danielle clean","startDate":"1706004000000","endDate":"1706014800000","fullDayEvent":false,"recurringEvent":true,"class":"","firstYear":2023,"location":false,"geo":false,"description":false},{"title":"Matt Hoxton monthly social?","startDate":"1706198400000","endDate":"1706209200000","fullDayEvent":false,"class":"","location":false,"geo":false,"description":false},

@sdetweil said in Calendar not updating:

npm install node-ical@0.16.1

pi@magicmirror:~/MagicMirror $ npm install node-ical@0.16.1

up to date in 11s

196 packages are looking for funding
  run `npm fund` for details

Done. Same result - event which should display is seen in the pm2 logs, but not in the UI events.

@sdetweil Hi Sam.
For a test I created an event in my calendar called test111 and verified server sync across several devices. This event starts in a few hours time so should be at the top of the events displayed on the mirror. I then stopped and started pm2.

Here are the magicmirror debug logs for the event:

[19.01.2024 08:41.23.286] [DEBUG] Event:
{"type":"VEVENT","params":[],"created":"2024-01-19T08:39:52.000Z","lastmodified":"2024-01-19T08:40:05.000Z","dtstamp":"2024-01-19T08:40:05.000Z","uid":"7383cead-3ce7-4a50-80d0-cf214e98caac","summary":"test111","start":"2024-01-19T11:00:00.000Z","datetype":"date-time","end":"2024-01-19T12:00:00.000Z","transparency":"OPAQUE"}
[19.01.2024 08:41.23.287] [DEBUG] start: Fri Jan 19 2024 11:00:00 GMT+0000 (Greenwich Mean Time)
[19.01.2024 08:41.23.287] [DEBUG] end:: Fri Jan 19 2024 12:00:00 GMT+0000 (Greenwich Mean Time)
[19.01.2024 08:41.23.287] [DEBUG] duration: 3600000
[19.01.2024 08:41.23.287] [DEBUG] title: test111
[19.01.2024 08:41.23.288] [DEBUG] initial tz=Europe/London
[19.01.2024 08:41.23.288] [DEBUG] corrected tz=Europe/London
[19.01.2024 08:41.23.288] [DEBUG] start date/time=Fri Jan 19 2024 11:00:00 GMT+0000 (Greenwich Mean Time)
[19.01.2024 08:41.23.288] [DEBUG] start offset=0
[19.01.2024 08:41.23.289] [DEBUG] start date/time w tz =Fri Jan 19 2024 11:00:00 GMT+0000 (Greenwich Mean Time)
[19.01.2024 08:41.23.289] [DEBUG] event date=Fri Jan 19 2024 11:00:00 GMT+0000 (Greenwich Mean Time)
[19.01.2024 08:41.23.289] [DEBUG] event offset=0 hour=11 event date=Fri Jan 19 2024 11:00:00 GMT+0000 (Greenwich Mean Time)
[19.01.2024 08:41.23.289] [DEBUG] adjustHours=0
[19.01.2024 08:41.23.290] [DEBUG] Processing entry...
[19.01.2024 08:41.23.297] [INFO]  Calendar-Fetcher: Broadcasting 621 events from https://my.cal.url

But in dev tools the event does not appear in the before filter events or events to process console sections which, i think, would indicate it’s not being parsed through to the UI…?

For comparison, if it helps, an event that is being parsed through looks like this in the logs (with a couple of bits redacted for privacy):

[19.01.2024 08:41.45.109] [DEBUG] Event:
{"type":"VEVENT","params":[],"created":"2024-01-05T14:17:41.000Z","end":"2024-01-20T21:00:00.000Z","dtstamp":"2024-01-16T18:42:50.000Z","start":"2024-01-20T17:00:00.000Z","datetype":"date-time","lastmodified":"2024-01-16T18:42:49.000Z","location":"redacted","sequence":"2","summary":"Jarv birthday beers","transparency":"OPAQUE","uid":"7DF90CA7-D3BA-4519-96F7-944D027F79DE","url":{"params":{"VALUE":"URI"},"val":""},"APPLE-CREATOR-IDENTITY":"com.apple.mobilecal","APPLE-CREATOR-TEAM-IDENTITY":"0000000000","APPLE-STRUCTURED-LOCATION":{"params":{"VALUE":"URI","X-ADDRESS":"redacted","X-APPLE-MAPKIT-HANDLE":"redacted","X-APPLE-RADIUS":141.1751152233582,"X-APPLE-REFERENCEFRAME":1,"X-TITLE":"redacted"},"val":"geo:redacted"}}
[19.01.2024 08:41.45.109] [DEBUG] start: Sat Jan 20 2024 17:00:00 GMT+0000 (Greenwich Mean Time)
[19.01.2024 08:41.45.110] [DEBUG] end:: Sat Jan 20 2024 21:00:00 GMT+0000 (Greenwich Mean Time)
[19.01.2024 08:41.45.110] [DEBUG] duration: 14400000
[19.01.2024 08:41.45.110] [DEBUG] title: Jarv birthday beers
[19.01.2024 08:41.45.110] [DEBUG] initial tz=Europe/London
[19.01.2024 08:41.45.110] [DEBUG] corrected tz=Europe/London
[19.01.2024 08:41.45.111] [DEBUG] start date/time=Sat Jan 20 2024 17:00:00 GMT+0000 (Greenwich Mean Time)
[19.01.2024 08:41.45.111] [DEBUG] start offset=0
[19.01.2024 08:41.45.111] [DEBUG] start date/time w tz =Sat Jan 20 2024 17:00:00 GMT+0000 (Greenwich Mean Time)
[19.01.2024 08:41.45.112] [DEBUG] event date=Sat Jan 20 2024 17:00:00 GMT+0000 (Greenwich Mean Time)
[19.01.2024 08:41.45.112] [DEBUG] event offset=0 hour=17 event date=Sat Jan 20 2024 17:00:00 GMT+0000 (Greenwich Mean Time)
[19.01.2024 08:41.45.112] [DEBUG] adjustHours=0
[19.01.2024 08:41.45.113] [DEBUG] Processing entry...

Thanks again.

@sdetweil said in Calendar not updating:

node-ical@0.16.1

Done. Different events shown on the mirror - events over the next few days rather than 10 days away! - but still events not displaying.

Browser dev tools indicate they’re sitll not getting parsed.
Server pm2 logs do show the events are being picked up from the calendar server e.g.

0|MagicMirror  | {"type":"VEVENT","params":[],"created":"2024-01-16T10:33:48.000Z","end":"2024-01-19T15:00:00.000Z","dtstamp":"2024-01-16T10:33:49.000Z","start":"2024-01-19T14:00:00.000Z","datetype":"date-time","lastmodified":"2024-01-16T10:33:48.000Z","sequence":"0","summary":"Alex bike","transparency":"OPAQUE","uid":"66D38BB9-70EA-486B-A297-58C2E0F3642C","url":{"params":{"VALUE":"URI"},"val":""},"APPLE-CREATOR-IDENTITY":"com.apple.mobilecal","APPLE-CREATOR-TEAM-IDENTITY":"0000000000"}

is not showing on the display and is not showing in browser devtools.

@sdetweil Hi Sam.
The before filter section is not showing the events that are missing. So not sent from the parser…?
The events to process section reflects what is seen in the before filter section.

Over the weekend the calendar looked good, but come Monday morning lots of events are missing. The mirror is currently showing events 10 days in the future even though there are plenty of events before then.

I have had a play with module config maximumEntries and limitDays to see if anything improves, but with no luck.

Thanks again,
Matt

@sdetweil good tip, thanks.

Little update - I put maximumEntries back to 5 and all my events showed up! But then I added another event, pm2 restart and they disappeared again :weary_face:

We’re getting closer! I think you’re on to something with the maximumEntries effect. And also handling between back and front end.

I’ll have a go with editing calendar.js this weekend and report back.

Thanks

@sdetweil Thanks Sam. Will have a go and report back.

@sdetweil Updated to 20. This now shows calendar entries all the way to early May but the test event that starts at 20:00 this evening does not show. There are actually lots of events over the next few days that are not showing.

@sdetweil

{
			module: "calendar",
			header: "--- Calendar testing ---",
			position: "top_left",
			config: {
				fetchInterval: 120000,
				timeFormat: 'absolute',
				maximumEntries: 5,
				broadcastPastEvents: false,
				calendars: [
					{
						url: 'https://my.cal.url/dav.php/calendars/family/default?export',
						auth: 	{
							user: 'family',
							pass: 'pass',
							method: 'basic'
							}
					}
					]
				}
		},

With the maximumEntries I currently have events showing several days in the future. When I am adding events to test this, I add the event to start just a few hours in the future so that it should show within the 5 events.

@sdetweil yes, the broadcasting event number increases by 1 when I add my new event, but the event does not show on the display.

@sdetweil I wonder if I could run a nightly cron job to purge the calendar module or something similar…? Not sure how the calendar fetcher works.
It still feels like the calendar will load an ics link ok the first time, but is not displaying events created since that first load.
Nightly purge and re-connect would be acceptable, I think. Any advice there?

OK interesting. I created a brand new calendar server. Empty. Added one event to it and configured the mirror calendar module to use this server. It shows the one single event ok. I added a second new event to the calendar and this also shows on the mirror. Seems to work as expected - new events are showing!

Then I import my whole famlly calendar and run the same test. I now have 5 family events showing on the calendar. But if I add a new event that should show within my 5 events, it does not show. :(

Debug logs >somefile.txt show the mirror is picking up the new event from the ics file, it’s just not showing on the display.

Initial testing with the apache version of the baikal container looks promising (I’m currently having problems with the nginx version). Will continue testing along these lines…

@sdetweil this is from the curl’d ics file, showing one of the events that will not display on the mirror display:

BEGIN:VEVENT
CREATED:20240109T161646Z
DTEND;TZID=Europe/London:20240110T170000
DTSTAMP:20240109T161647Z
DTSTART;TZID=Europe/London:20240110T160000
LAST-MODIFIED:20240109T161646Z
SEQUENCE:0
SUMMARY:test54321
TRANSP:OPAQUE
UID:156ED8F1-9BDA-43D2-BE7C-DDEFE18E95EF
URL;VALUE=URI:
X-APPLE-CREATOR-IDENTITY:com.apple.mobilecal
X-APPLE-CREATOR-TEAM-IDENTITY:0000000000
END:VEVENT

I’d be happy to send over the contents of the ics file before events start, if that would be helpful? Far too much personal family info to send the whole thing, Im afraid (again, no offence intended)!

@sdetweil so I’ve just spun up a radicale calendar server and added a few test events, then some more - works perfectly!

Looks like a problem with baikal integration