MMM-CalendarExt3 not syncing automatically with icloud

lightfixer2000

Hi, relative newbie to world of Pi and Magic Mirror. I have installed MM 2.26 with several modules on a Pi4, these all seem to be working. I have installed the MMM-CalendarExt3 module which i am using with a shared icloud calendar. The calendar on first install synced and pulled events through after following instructions on installing Vdirsyncer from the fantastic instructions given by Beh.
However, changing event information does not update on Calendar-Ext3 after the next refresh interval. I see the Calendar fade and re-appear as it refreshes but with same info.
Checking the ics file in Modules, Calendars properties I see that the file has not been updated since the initial sync.
I can however force CalendarExt3 to Sync and Update using the Vdirsyncer Sync command.
I am getting two errors logged in the error file pm2 creates.

[3010:0324/105027.892355:ERROR:gbm_wrapper.cc(75)] Failed to get fd for plane.: No such file or directory (2)
[3010:0324/105027.897460:ERROR:gbm_wrapper.cc(258)] Failed to export buffer to dma_buf: No such file or directory (2)

I am unsure what these errors are telling me? gbm_wrapper? And if they are related to my issue.
Here is my config ,in case there is something amiss with this.

let config = {
	address: "localhost",	// Address to listen on, can be:
							// - "localhost", "127.0.0.1", "::1" to listen on loopback interface
							// - another specific IPv4/6 to listen on a specific interface
							// - "0.0.0.0", "::" to listen on any interface
							// Default, when address config is left out or empty, is "localhost"
	port: 8080,
	basePath: "/",			// The URL path where MagicMirror² is hosted. If you are using a Reverse proxy
					  		// you must set the sub path here. basePath must end with a /
	ipWhitelist: ["127.0.0.1", "::ffff:127.0.0.1", "::1"],	// Set [] to allow all IP addresses
															// or add a specific IPv4 of 192.168.1.5 :
															// ["127.0.0.1", "::ffff:127.0.0.1", "::1", "::ffff:192.168.1.5"],
															// or IPv4 range of 192.168.3.0 --> 192.168.3.15 use CIDR format :
															// ["127.0.0.1", "::ffff:127.0.0.1", "::1", "::ffff:192.168.3.0/28"],

	useHttps: false, 		// Support HTTPS or not, default "false" will use HTTP
	httpsPrivateKey: "", 	// HTTPS private key path, only require when useHttps is true
	httpsCertificate: "", 	// HTTPS Certificate path, only require when useHttps is true

	language: "en",
	locale: "en-GB",
	logLevel: ["INFO", "LOG", "WARN", "ERROR"], // Add "DEBUG" for even more logging
	timeFormat: 24,
	units: "metric",

	modules: [
		{
			module: "alert",
		},
		{
			module: "calendar",
			//position: "",
			config: {
				maximumEntries: 20,
				broadcastPastEvents: true,
				calendars: [
					{
						fetchInterval: 10 * 60 * 1000, //10mins
						symbol: "calendar-check",
						url: "http://localhost:8080/modules/calendars/*****.ics"					
						}
				]
			}
		},

		{
			module: "updatenotification",
			position: "top_bar",
		},
		{
			module: "clock",
			position: "top_left",
		},
		{
			module: 'MMM-GoogleMapsTraffic',
            		position: 'top_left',
            		config: {
                		key: '******',
                		lat: *******,
                		lng: *******,
                		height: '550px',
                		width: '550px',
				zoom: 13,
                		styledMapType: "standard",
                		disableDefaultUI: true,
                		backgroundColor: 'hsla(0, 0%, 0%, 0)',
            			},
		},
		
		{
			module: "MMM-CalendarExt3",
			position: "lower_third",
            		config: {
			firstDayOfWeek: 0,
			locale: 'en-GB',
			},
		},
		{
			module: "weather",
			position: "top_right",
			config: {
				weatherProvider: "ukmetoffice",
				type: "current",
				location: "********",
				locationID: "******", //ID
				apiKey: "*******"
			}
		},
		{
			module: "weather",
			position: "top_right",
			header: "Weather Forecast",
			config: {
				weatherProvider: "ukmetoffice",
				type: "forecast",
				location: "*******",
				locationID: "*******", //ID
				apiKey: "********"
			}
		},
		{
			module: "newsfeed",
			position: "bottom_bar",
			config: {
				feeds: [
					{
						title: "BBC News",
						url: "https://feeds.bbci.co.uk/news/rss.xml#"
					}
				],
				showSourceTitle: true,
				showPublishDate: true,
				broadcastNewsFeeds: true,
				broadcastNewsUpdates: true
			}
		},
	]
};

I know that there are quite a few posts on this subject and I have tried various answers offered from within these posts without any success.
Is this an issue with Vdirsyncer fetching?
Is there anything else I can try? Or maybe not doing correctly?
Thanks in advance.

MMRIZE

@lightfixer2000
At this moment, I’m unsure the exact symptom what you said.
There is an undocumented options for more detail configuration.
Try expand: false, in your module config.

{
  module: "MMM-CalDAV", 
  config: {
    servers: [
      { 
        envPrefix: "ICLOUD_", // I assumed.
        serverUrl: "https://caldav.icloud.com", 
        expand: false,  // <== This.
        targets: [
        ...

I’m not sure whether this will help yours or not. I just suspect.

PS. (more detail).
This option is related to expanding repeated events to a series of single independent events. As far as I remember, in some prior calendars, there were some issues handling repeated events by RRULE. So, I made this module to forcefully avoid that issue by default. I think recently things might be changed, so hidden option expand: true might cause your symptom.

MMRIZE

@lightfixer2000
Nothing seems wrong. It seems just a sync timing issue. I don’t think it is the kind of issue to fix, but…
Your vdirsyncer may sync calendars at some point, and the calendar module fetches the change at another point (repeatedly in 10 minutes). CX3 will refresh itself at the other point (by default, every 10 minutes).
So when you change an event at some point, vdirsyncer may not have synced that event yet, or even if vdirsyncer synced successfully, the calendar module may not have fetched the event yet.
In the worst case, I think at least 20 minutes may be needed to reflect the change of event. It takes 2 cycles of CX3’s defaults.

Should it be the problem? I don’t think so. Calendar events are usually very stable and not frequently changed. Just waiting 20 ~ 30 minutes is an acceptable time for daily usage.

However, if you want a more urgent reflection of the change, shorten the period of sync/fetch/refresh of the 3 related programs(vdirsync, calendar module, CX3). But 99% of the process would be wasted with unchanged, same events,

lightfixer2000

@MMRIZE
Hi, Thanks for your response.
The actual MM works well and the CalendarExt3 display’s ok. I will leave the calendar for a longer time to allow further syncs to take place.
Any thoughts on the errors that are being reported in the log?
Thanks

sdetweil

@lightfixer2000 said in MMM-CalendarExt3 not syncing automatically with icloud:

[3010:0324/105027.892355:ERROR:gbm_wrapper.cc(75)] Failed to get fd for plane

this is an electron reported ‘error’, but is really a warning. you can ignore it. we can’t hide it

greedyvegan

even though the module loads, I see this error all the time; even after a refresh.
Should I ignore it?

Screenshot 2024-03-24 at 3.25.01 PM.png

MMRIZE

@greedyvegan That is not an error, just warning you can ignore it.

lightfixer2000

@MMRIZE I’ve left the MMM Calendar running for a far longer period and sync has never occured.
I did some digging/reading on Vdirsyncer and located the journal, here’s what it has been recording over and over.

Mar 26 00:11:18 raspberrypi systemd[923]: vdirsyncer.service: Scheduled restart job, restart counter is at 2.
Mar 26 00:11:18 raspberrypi systemd[923]: Stopped vdirsyncer.service - Synchronize calendars and contacts.
Mar 26 00:11:18 raspberrypi systemd[923]: vdirsyncer.service: Start request repeated too quickly.
Mar 26 00:11:18 raspberrypi systemd[923]: vdirsyncer.service: Failed with result 'exit-code'.
Mar 26 00:11:18 raspberrypi systemd[923]: Failed to start vdirsyncer.service - Synchronize calendars and contacts.
Mar 26 00:26:18 raspberrypi systemd[923]: Started vdirsyncer.service - Synchronize calendars and contacts.
Mar 26 00:26:18 raspberrypi (irsyncer)[4142]: vdirsyncer.service: Failed to locate executable /usr/bin/vdirsyncer: No such file or directory
Mar 26 00:26:18 raspberrypi (irsyncer)[4142]: vdirsyncer.service: Failed at step EXEC spawning /usr/bin/vdirsyncer: No such file or directory
Mar 26 00:26:18 raspberrypi systemd[923]: vdirsyncer.service: Main process exited, code=exited, status=203/EXEC
Mar 26 00:26:18 raspberrypi systemd[923]: vdirsyncer.service: Failed with result 'exit-code'.
Mar 26 00:26:18 raspberrypi systemd[923]: vdirsyncer.service: Scheduled restart job, restart counter is at 1.
Mar 26 00:26:18 raspberrypi systemd[923]: Stopped vdirsyncer.service - Synchronize calendars and contacts.
Mar 26 00:26:18 raspberrypi systemd[923]: Started vdirsyncer.service - Synchronize calendars and contacts.
Mar 26 00:26:18 raspberrypi (irsyncer)[4143]: vdirsyncer.service: Failed to locate executable /usr/bin/vdirsyncer: No such file or directory
Mar 26 00:26:18 raspberrypi (irsyncer)[4143]: vdirsyncer.service: Failed at step EXEC spawning /usr/bin/vdirsyncer: No such file or directory
Mar 26 00:26:18 raspberrypi systemd[923]: vdirsyncer.service: Main process exited, code=exited, status=203/EXEC
Mar 26 00:26:18 raspberrypi systemd[923]: vdirsyncer.service: Failed with result 'exit-code'.
Mar 26 00:26:18 raspberrypi systemd[923]: vdirsyncer.service: Scheduled restart job, restart counter is at 2.
Mar 26 00:26:18 raspberrypi systemd[923]: Stopped vdirsyncer.service - Synchronize calendars and contacts.
Mar 26 00:26:18 raspberrypi systemd[923]: vdirsyncer.service: Start request repeated too quickly.
Mar 26 00:26:18 raspberrypi systemd[923]: vdirsyncer.service: Failed with result 'exit-code'.
Mar 26 00:26:18 raspberrypi systemd[923]: Failed to start vdirsyncer.service - Synchronize calendars and contacts.

It states it can’t find the vdirsyncer file which isn’t really a file but a symlink as per Beh instructions.

Here are my vdirsyncer service and timer files in case I’m missing something?

[Unit]
Description=Synchronize calendars and contacts
Documentation=https://vdirsyncer.readthedocs.org/
StartLimitBurst=2

[Service]
ExecStart=/usr/bin/vdirsyncer sync
RuntimeMaxSec=3m
Restart=on-failure

[Unit]
Description=Synchronize vdirs

[Timer]
OnBootSec=2m
OnUnitActiveSec=10m
AccuracySec=5m

[Install]
WantedBy=timers.target

I did the Clean but Hard Way install of vdirsyncer and it created a vdirsyncer_env folder which according to instructions keeps Python contained. Could this be the issue?
Thanks in advance.

sdetweil

@lightfixer2000 many python scripts require a virtual environment. using the venv command to load from from the folder the installer creates before executing any python script

sdetweil

@lightfixer2000 I try not to use services anymore as pm2 can launch anything, like my installer sets up for MagicMirror.

see the pm2 --help

lightfixer2000

@sdetweil Thanks. I have pm2 installed and it runs MMM at boot up. According to the timer file the first fetch is after 2 mins of program start. I still don’t see any update to the ics file.
I will read pm2 help is a config or something else with pm2 I should be using?

sdetweil

@lightfixer2000 pm2 start xxx

xxx can be a bash script
or an executable

if you the bash script the you can ha e multile commands and logic

pm2’s job is to start and run things and keep them running

MMRIZE

@lightfixer2000
I’m not using vdirsyncer to get iCloud or Google private/family calendars as iCal. So I have no idea.
Instead I use MMM-CalDAV. Sorry.

lightfixer2000

@MMRIZE Hi thanks for the heads up on CalDAV. Is this also a program which can deal privately with iCloud calendars? Rather than Vdirsyncer? Thanks.

lightfixer2000

Update. So I have removed Vdirsyncer and installed MMM-CalDAV.
I have two calendars running, the standard calendar which I have set to UK Holidays and our shared icloud family calendar.
CalDav is accessing my shared family icloud calendar. I see timed single events ok and even events spread over a couple of days. But, it is still not showing repeating weekly events. You see the initial start date/event and that is all.
Having read quite a few posts on missing events I have tried several suggestions to no effect. Checking the family ics file I see the data there. However, there is no RRULE listing associated with these events.
Also should there be further data on the URL VALUE? I have not deleted anything, but its blank. Is this because CalDAV is handling the URL?
Here is part of the shared ics file. I’ve deleted UID in case this unique to me.

BEGIN:VEVENT
DTSTAMP:20240411T163851Z
CREATED:20240410T140919Z
DTSTAMP:20240411T112142Z
LAST-MODIFIED:20240411T112141Z
SEQUENCE:3
SUMMARY:Testing….
UID:******
URL;VALUE=URI:
X-APPLE-CREATOR-IDENTITY:com.apple.mobilecal
X-APPLE-CREATOR-TEAM-IDENTITY:0000000000
TRANSP:OPAQUE
X-RECURRENCE-MASTER:True
DTSTART;TZID=Europe/London:20240412T130000
DTEND;TZID=Europe/London:20240412T140000
RECURRENCE-ID:20240412T120000Z
END:VEVENT
BEGIN:VEVENT
DTSTAMP:20240411T163851Z
CREATED:20240410T140919Z
DTSTAMP:20240411T112142Z
LAST-MODIFIED:20240411T112141Z
SEQUENCE:3
SUMMARY:Testing….
UID:*****
URL;VALUE=URI:
X-APPLE-CREATOR-IDENTITY:com.apple.mobilecal
X-APPLE-CREATOR-TEAM-IDENTITY:0000000000
TRANSP:OPAQUE
DTSTART;TZID=Europe/London:20240419T130000
DTEND;TZID=Europe/London:20240419T140000
RECURRENCE-ID:20240419T120000Z
END:VEVENT
BEGIN:VEVENT
DTSTAMP:20240411T163851Z
CREATED:20240410T140919Z
DTSTAMP:20240411T112142Z
LAST-MODIFIED:20240411T112141Z
SEQUENCE:3
SUMMARY:Testing….
UID:******
URL;VALUE=URI:
X-APPLE-CREATOR-IDENTITY:com.apple.mobilecal
X-APPLE-CREATOR-TEAM-IDENTITY:0000000000
TRANSP:OPAQUE
DTSTART;TZID=Europe/London:20240426T130000
DTEND;TZID=Europe/London:20240426T140000
RECURRENCE-ID:20240426T120000Z
END:VEVENT
BEGIN:VEVENT
DTSTAMP:20240411T163851Z
CREATED:20240410T140919Z
DTSTAMP:20240411T112142Z
LAST-MODIFIED:20240411T112141Z
SEQUENCE:3
SUMMARY:Testing….
UID:******
URL;VALUE=URI:
X-APPLE-CREATOR-IDENTITY:com.apple.mobilecal
X-APPLE-CREATOR-TEAM-IDENTITY:0000000000
TRANSP:OPAQUE
DTSTART;TZID=Europe/London:20240503T130000
DTEND;TZID=Europe/London:20240503T140000
RECURRENCE-ID:20240503T120000Z
END:VEVENT
BEGIN:VEVENT
DTSTAMP:20240411T163851Z
CREATED:20240410T140919Z
DTSTAMP:20240411T112142Z
LAST-MODIFIED:20240411T112141Z
SEQUENCE:3
SUMMARY:Testing….
UID:******
URL;VALUE=URI:
X-APPLE-CREATOR-IDENTITY:com.apple.mobilecal
X-APPLE-CREATOR-TEAM-IDENTITY:0000000000
TRANSP:OPAQUE
DTSTART;TZID=Europe/London:20240510T130000
DTEND;TZID=Europe/London:20240510T140000
RECURRENCE-ID:20240510T120000Z
END:VEVENT
BEGIN:VEVENT
DTSTAMP:20240411T163851Z
CREATED:20240410T140919Z
DTSTAMP:20240411T112142Z
LAST-MODIFIED:20240411T112141Z
SEQUENCE:3
SUMMARY:Testing….
UID:******
URL;VALUE=URI:
X-APPLE-CREATOR-IDENTITY:com.apple.mobilecal
X-APPLE-CREATOR-TEAM-IDENTITY:0000000000
TRANSP:OPAQUE
DTSTART;TZID=Europe/London:20240517T130000
DTEND;TZID=Europe/London:20240517T140000
RECURRENCE-ID:20240517T120000Z
END:VEVENT

Reading the posts I gather there is on an ongoing issue with regard to RRULE. Is this the cause of the problem?
I have tried returning node-ical back to its previous version 16 also with no effect.
Am I missing something really obvious?
Any further suggestions or help greatly received. Thanks.

sdetweil

@lightfixer2000 MMM-CalDav is creating the ics file

the node-ical problem is processing the rrule, if any

i have not seen reports of calendar troubles with the latest mm release on April 1

lightfixer2000

@sdetweil
A couple of questions for my understanding.
Does RRULE make calendar display repeat events?
Is CalDAV responsible for dropping the RRULE?
Is it possible to show repeat weekly events?

MMRIZE

@lightfixer2000
At this moment, I’m unsure the exact symptom what you said.
There is an undocumented options for more detail configuration.
Try expand: false, in your module config.

{
  module: "MMM-CalDAV", 
  config: {
    servers: [
      { 
        envPrefix: "ICLOUD_", // I assumed.
        serverUrl: "https://caldav.icloud.com", 
        expand: false,  // <== This.
        targets: [
        ...

I’m not sure whether this will help yours or not. I just suspect.

PS. (more detail).
This option is related to expanding repeated events to a series of single independent events. As far as I remember, in some prior calendars, there were some issues handling repeated events by RRULE. So, I made this module to forcefully avoid that issue by default. I think recently things might be changed, so hidden option expand: true might cause your symptom.

sdetweil

@lightfixer2000 RRULE stands for Repeating Rule
it handles ALL the rules that make this event happen again (on some cycle)
it is optional… as some (most) events don’t repeat

MMM-CalDav is processing the caldav calendar and creating the ICS file we process, using the ICAL standards.

if there is no RRULE, then it is not repeating.
the default calendar does not look at the text of the ICS file, we download it, then send it to the node-ical module to convert the text into workable objects.
it replicates the RRULE into an object in the event entry, and then we check for the RRULE object… if present we ask the RRULE library for the events that match the rules in the next year.

lightfixer2000

@sdetweil & @MMRIZE
Every day is a school day!!
So adding “expand” into my config has fixed my issue. The ics is now including RRULE and the calendar is displaying repeating events.
Thank you for the explanation.
Happy Days.

MMRIZE

@lightfixer2000
Happy to hear. I’ll reveal that option in the readme. Recently, someone else reported me the same issue, and the solution was the same. I’ll change the default value soon.