electron-rebuild problems with MagicMirror 2.22.0

wishmaster270

Today i started a new approach and removed the “node_modules” folder of MagicMirror before the update to make sure no other electron-rebuild packages are installed.

As i do have a second electron app running on my mirror (TeleFrame) is stopped this application and removed the “/home/pi/.electron-gyp” folder to make sure everything is clean before the update.

I then did the update with your script and the “npm install” step of my MMM-GPIO-Notifications module fails.

I cloned your “MMM-Buttons” module and tried to install only this module but i get the same compile error. I enabled the DEBUG flag. Maybe you see something i do not see but it looks like the correct electron-rebuild version is used:

pi@mirror-eg:~/MagicMirror/modules/MMM-Buttons $ npm install

> Magic-Mirror-Module-Buttons@1.0.0 postinstall
> ./postinstall

/home/pi/MagicMirror/modules/MMM-Buttons
⠋ Searching dependency tree  electron-rebuild rebuilding with args: [Arguments] {
  '0': {
    buildPath: '/home/pi/MagicMirror/modules/MMM-Buttons',
    electronVersion: '22.0.0',
    arch: 'arm',
    extraModules: [],
    onlyModules: null,
    force: undefined,
    headerURL: undefined,
    types: [ 'prod', 'optional' ],
    mode: undefined,
    debug: undefined,
    prebuildTagPrefix: 'v',
    forceABI: undefined,
    useElectronClang: false,
    disablePreGypCopy: false,
    projectRootPath: '/home/pi/MagicMirror/modules/MMM-Buttons'
  }
} +0ms
  electron-rebuild rebuilding with args: /home/pi/MagicMirror/modules/MMM-Buttons 22.0.0 arm Set(0) {} false https://www.electronjs.org/headers [ 'prod', 'optional' ] false +8ms
  electron-rebuild exploring /home/pi/MagicMirror/modules/MMM-Buttons/node_modules/onoff +0ms
  electron-rebuild exploring /home/pi/MagicMirror/modules/MMM-Buttons/node_modules/epoll +4ms
  electron-rebuild exploring /home/pi/MagicMirror/modules/MMM-Buttons/node_modules/lodash.debounce +1ms
  electron-rebuild exploring /home/pi/MagicMirror/modules/MMM-Buttons/node_modules/bindings +2ms
  electron-rebuild exploring /home/pi/MagicMirror/modules/MMM-Buttons/node_modules/nan +1ms
  electron-rebuild exploring /home/pi/MagicMirror/modules/MMM-Buttons/node_modules/file-uri-to-path +1ms
  electron-rebuild identified prod deps: Set(0) {
  onoff: true,
  epoll: true,
  'lodash.debounce': true,
  bindings: true,
  nan: true,
  'file-uri-to-path': true
} +1ms
  electron-rebuild scanning: /home/pi/MagicMirror/modules/MMM-Buttons/node_modules +5ms
  electron-rebuild rebuilding epoll with args [
  'node',
  'node-gyp',
  'rebuild',
  '--runtime=electron',
  '--target=22.0.0',
  '--arch=arm',
  '--dist-url=https://www.electronjs.org/headers',
  '--build-from-source',
  '--verbose'
] +0ms
gyp verb command rebuild []
gyp verb command clean []
gyp verb clean removing "build" directory
⠙ Building module: epoll, Completed: 0gyp verb command configure []
gyp verb download using dist-url https://www.electronjs.org/headers
gyp verb find Python Python is not set from command line or npm configuration
gyp verb find Python Python is not set from environment variable PYTHON
gyp verb find Python checking if "python3" can be used
gyp verb find Python - executing "python3" to get executable path
⠹ Building module: epoll, Completed: 0gyp verb find Python - executable path is "/usr/bin/python3"
gyp verb find Python - executing "/usr/bin/python3" to get version
⠸ Building module: epoll, Completed: 0gyp verb find Python - version is "3.7.3"
gyp info find Python using Python version 3.7.3 found at "/usr/bin/python3"
gyp verb get node dir compiling against --target node version: 22.0.0
gyp verb command install [ '22.0.0' ]
gyp verb download using dist-url https://www.electronjs.org/headers
gyp verb install input version string "22.0.0"
gyp verb install installing version: 22.0.0
gyp verb install --ensure was passed, so won't reinstall if already installed
⠼ Building module: epoll, Completed: 0gyp verb install version is already installed, need to check "installVersion"
gyp verb got "installVersion" 9
gyp verb needs "installVersion" 9
gyp verb install version is good
gyp verb get node dir target node version installed: 22.0.0
gyp verb build dir attempting to create "build" dir: /home/pi/MagicMirror/modules/MMM-Buttons/node_modules/epoll/build
gyp verb build dir "build" dir needed to be created? Yes
gyp verb python symlink creating symlink to "/usr/bin/python3" at "/home/pi/MagicMirror/modules/MMM-Buttons/node_modules/epoll/build/node_gyp_bins/python3"
gyp verb build/config.gypi creating config file
gyp verb build/config.gypi writing out config file: /home/pi/MagicMirror/modules/MMM-Buttons/node_modules/epoll/build/config.gypi
gyp verb config.gypi checking for gypi file: /home/pi/MagicMirror/modules/MMM-Buttons/node_modules/epoll/config.gypi
gyp verb common.gypi checking for gypi file: /home/pi/MagicMirror/modules/MMM-Buttons/node_modules/epoll/common.gypi
gyp verb gyp gyp format was not specified; forcing "make"
gyp info spawn /usr/bin/python3
gyp info spawn args [
gyp info spawn args   '/home/pi/MagicMirror/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/home/pi/MagicMirror/modules/MMM-Buttons/node_modules/epoll/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/pi/MagicMirror/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/pi/.electron-gyp/22.0.0/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/home/pi/.electron-gyp/22.0.0',
gyp info spawn args   '-Dnode_gyp_dir=/home/pi/MagicMirror/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/home/pi/.electron-gyp/22.0.0/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/home/pi/MagicMirror/modules/MMM-Buttons/node_modules/epoll',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
⠙ Building module: epoll, Completed: 0gyp verb command build []
gyp verb build type Release
gyp verb architecture armv7l
gyp verb node dev dir /home/pi/.electron-gyp/22.0.0
gyp verb `which` succeeded for `make` /usr/bin/make
gyp verb bin symlinks adding symlinks (such as Python), at "/home/pi/MagicMirror/modules/MMM-Buttons/node_modules/epoll/build/node_gyp_bins", to PATH
gyp info spawn make
gyp info spawn args [ 'V=1', 'BUILDTYPE=Release', '-C', 'build' ]
make: Verzeichnis „/home/pi/MagicMirror/modules/MMM-Buttons/node_modules/epoll/build“ wird betreten
  g++ -o Release/obj.target/epoll/src/epoll.o ../src/epoll.cc '-DNODE_GYP_MODULE_NAME=epoll' '-DUSING_UV_SHARED=1' '-DUSING_V8_SHARED=1' '-DV8_DEPRECATION_WARNINGS=1' '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-D_GLIBCXX_USE_CXX11_ABI=1' '-DELECTRON_ENSURE_CONFIG_GYPI' '-D_LARGEFILE_SOURCE' '-D_FILE_OFFSET_BITS=64' '-DUSING_ELECTRON_CONFIG_GYPI' '-DV8_COMPRESS_POINTERS' '-DV8_COMPRESS_POINTERS_IN_SHARED_CAGE' '-DV8_ENABLE_SANDBOX' '-DV8_31BIT_SMIS_ON_64BIT_ARCH' '-D__STDC_FORMAT_MACROS' '-DOPENSSL_NO_PINSHARED' '-DOPENSSL_THREADS' '-DOPENSSL_NO_ASM' '-DBUILDING_NODE_EXTENSION' -I/home/pi/.electron-gyp/22.0.0/include/node -I/home/pi/.electron-gyp/22.0.0/src -I/home/pi/.electron-gyp/22.0.0/deps/openssl/config -I/home/pi/.electron-gyp/22.0.0/deps/openssl/openssl/include -I/home/pi/.electron-gyp/22.0.0/deps/uv/include -I/home/pi/.electron-gyp/22.0.0/deps/zlib -I/home/pi/.electron-gyp/22.0.0/deps/v8/include -I../../nan  -fPIC -pthread -Wall -Wextra -Wno-unused-parameter -Wno-deprecated-declarations -Wno-cast-function-type -O3 -fno-omit-frame-pointer -fno-rtti -fno-exceptions -std=gnu++17 -MMD -MF ./Release/.deps/Release/obj.target/epoll/src/epoll.o.d.raw  -std=c++17 -c
⠴ Building module: epoll, Completed: 0In file included from /home/pi/.electron-gyp/22.0.0/include/node/v8-local-handle.h:12,
                 from /home/pi/.electron-gyp/22.0.0/include/node/v8-array-buffer.h:12,
                 from /home/pi/.electron-gyp/22.0.0/include/node/v8.h:24,
                 from ../src/epoll.cc:11:
/home/pi/.electron-gyp/22.0.0/include/node/v8-internal.h:124:62: warning: left shift count >= width of type [-Wshift-count-overflow]
 constexpr size_t kPtrComprCageReservationSize = size_t{1} << 32;
                                                              ^~
/home/pi/.electron-gyp/22.0.0/include/node/v8-internal.h:124:59: error: right operand of shift expression ‘(1 << 32)’ is >= than the precision of the left operand [-fpermissive]
 constexpr size_t kPtrComprCageReservationSize = size_t{1} << 32;
                                                 ~~~~~~~~~~^~~~~
/home/pi/.electron-gyp/22.0.0/include/node/v8-internal.h:125:60: warning: left shift count >= width of type [-Wshift-count-overflow]
 constexpr size_t kPtrComprCageBaseAlignment = size_t{1} << 32;
                                                            ^~
/home/pi/.electron-gyp/22.0.0/include/node/v8-internal.h:125:57: error: right operand of shift expression ‘(1 << 32)’ is >= than the precision of the left operand [-fpermissive]
 constexpr size_t kPtrComprCageBaseAlignment = size_t{1} << 32;
                                               ~~~~~~~~~~^~~~~
/home/pi/.electron-gyp/22.0.0/include/node/v8-internal.h:128:27: error: static assertion failed: Pointer compression can be enabled only for 64-bit architectures
     kApiSystemPointerSize == kApiInt64Size,
     ~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~
/home/pi/.electron-gyp/22.0.0/include/node/v8-internal.h:186:38: warning: conversion from ‘long long unsigned int’ to ‘size_t’ {aka ‘unsigned int’} changes value from ‘1099511627776’ to ‘0’ [-Woverflow]
 constexpr size_t kSandboxSize = 1ULL << kSandboxSizeLog2;
                                 ~~~~~^~~~~~~~~~~~~~~~~~~
/home/pi/.electron-gyp/22.0.0/include/node/v8-internal.h:204:50: warning: conversion from ‘long long unsigned int’ to ‘size_t’ {aka ‘unsigned int’} changes value from ‘34359738368’ to ‘0’ [-Woverflow]
 constexpr size_t kSandboxGuardRegionSize = 32ULL * GB;
                                            ~~~~~~^~~~
/home/pi/.electron-gyp/22.0.0/include/node/v8-internal.h:220:56: warning: conversion from ‘long long unsigned int’ to ‘size_t’ {aka ‘unsigned int’} changes value from ‘8589934592’ to ‘0’ [-Woverflow]
 constexpr size_t kSandboxMinimumReservationSize = 8ULL * GB;
                                                   ~~~~~^~~~
/home/pi/.electron-gyp/22.0.0/include/node/v8-internal.h:230:60: warning: conversion from ‘long long unsigned int’ to ‘size_t’ {aka ‘unsigned int’} changes value from ‘34359738367’ to ‘4294967295’ [-Woverflow]
 constexpr size_t kMaxSafeBufferSizeForSandbox = 32ULL * GB - 1;
                                                 ~~~~~~~~~~~^~~
/home/pi/.electron-gyp/22.0.0/include/node/v8-internal.h:231:44: error: static assertion failed: The maximum allowed buffer size must not be larger than the sandbox's guard regions
 static_assert(kMaxSafeBufferSizeForSandbox <= kSandboxGuardRegionSize,
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~
/home/pi/.electron-gyp/22.0.0/include/node/v8-internal.h:236:48: error: static assertion failed: The maximum size of a BoundedSize must be synchronized with the kMaxSafeBufferSizeForSandbox
 static_assert(1ULL << (64 - kBoundedSizeShift) ==
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~
                   kMaxSafeBufferSizeForSandbox + 1,
                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/pi/.electron-gyp/22.0.0/include/node/v8-internal.h:260:56: error: static assertion failed: kExternalPointerTableReservationSize and kExternalPointerIndexShift don't match
 static_assert((1 << (32 - kExternalPointerIndexShift)) == kMaxExternalPointers,
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
⠴ Building module: epoll, Completed: 0make: *** [epoll.target.mk:129: Release/obj.target/epoll/src/epoll.o] Fehler 1
make: Verzeichnis „/home/pi/MagicMirror/modules/MMM-Buttons/node_modules/epoll/build“ wird verlassen
✖ Rebuild Failed

An unhandled error occurred inside electron-rebuild
node-gyp failed to rebuild '/home/pi/MagicMirror/modules/MMM-Buttons/node_modules/epoll'.
For more information, rerun with the DEBUG environment variable set to "electron-rebuild".

Error: `make` failed with exit code: 2



Error: node-gyp failed to rebuild '/home/pi/MagicMirror/modules/MMM-Buttons/node_modules/epoll'.
For more information, rerun with the DEBUG environment variable set to "electron-rebuild".

Error: `make` failed with exit code: 2


    at NodeGyp.rebuildModule (/home/pi/MagicMirror/node_modules/electron-rebuild/lib/src/module-type/node-gyp.js:120:19)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async ModuleRebuilder.rebuildNodeGypModule (/home/pi/MagicMirror/node_modules/electron-rebuild/lib/src/module-rebuilder.js:98:9)
    at async ModuleRebuilder.rebuild (/home/pi/MagicMirror/node_modules/electron-rebuild/lib/src/module-rebuilder.js:128:14)
    at async Rebuilder.rebuildModuleAt (/home/pi/MagicMirror/node_modules/electron-rebuild/lib/src/rebuild.js:149:13)
    at async Rebuilder.rebuild (/home/pi/MagicMirror/node_modules/electron-rebuild/lib/src/rebuild.js:112:17)
    at async /home/pi/MagicMirror/node_modules/electron-rebuild/lib/src/cli.js:158:9
npm ERR! code 255
npm ERR! path /home/pi/MagicMirror/modules/MMM-Buttons
npm ERR! command failed
npm ERR! command sh -c ./postinstall

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/pi/.npm/_logs/2023-01-07T11_55_30_111Z-debug-0.log

Edit: I will try to re-setup my mirror today to make sure no old things are messing up the installation but it will take a moment.

wishmaster270

@sdetweil
New installation, new luck!
Everything works well now and i think if found the reason for the problems.

As i wrote earlier i am running TeleFrame on the same Pi with MagicMirror. This time i installed TeleFrame manually instead of with the installer as i saw that the automatic installer of Teleframe installs a global version of Electron and adds architecture settings to the “.profile” file of the user.

sdetweil

@wishmaster270 cool. thx for feedback

don’t know Teleframe.

KristjanESPERANTO

The problems on my machine are obviously of a different nature. I’ll investigate that later.

For the original issue it should be worth using @electron/rebuild instead of magicmirror-rebuild or electron-rebuild. I tested it in a virtual machine and it works great. Here is my solution which is less invasive:
https://github.com/Tom-Hirschberger/MMM-GPIO-Notifications/pull/7

What do you think?

sdetweil

@KristjanESPERANTO my postinstall script uses electron-rebuild

his development branch

KristjanESPERANTO

Yes, but @electron/rebuild is the successor of electron-rebuild. And the “trick” in the postinstall script seems not necessary at all.

wishmaster270

@KristjanESPERANTO

Hi and thank you for contributing. I confirm that changing to magicmirror-rebuild did not make things batter but worse. I now followed the suggestion of @sdetweil and implemented his postinstall script. Things are working perfectly now and should work in future releases, too.

I merged the new changes to the master a few minutes ago and released version 0.0.7 of MMM-GPIO-Notifications. I will implement this approach in my other modules, too.

KristjanESPERANTO

@wishmaster270 OK, you’re welcome.

Since the thread is also for other developers: I recommend considering my approach, as I consider the other approach (installing to a higher-level directory) to be more error-prone. But it’s a creative workaround.

wishmaster270

@KristjanESPERANTO
The problem with installing it in the module directory is that we need to compile the library against the electron version installed in the MagicMirror project.
If electron-rebuild is installed in the module directory it might fail to detect the right electron version (which was the cause of the rebuild problems in the past).
So the right way is to install electron-rebuild in the same project as electron and call it from there.

It might be worth to think about adding it as a dependency in the main project.

KristjanESPERANTO

@wishmaster270 Okay, thanks for the explanation! I haven’t been able to read that anywhere in such clear words and I haven’t dealt with the past problems. Then I withdraw my recommendation because I lack the overview.

sdetweil

@KristjanESPERANTO ah I see

haven’t heard of @electron/rebuild

karsten13

@wishmaster270 said in electron-rebuild problems with MagicMirror 2.22.0:

So the right way is to install electron-rebuild in the same project as electron and call it from there.

installing module stuff in the node_modules folder of mm is a hack and will not work out of the box with a docker setup because this folder is only in the container (you have to map it on the host to get changes there persistent)

It might be worth to think about adding it as a dependency in the main project.

in this case may the best solution

sdetweil

@karsten13 yes, that’s what I think we come to but doesn’t help any existing broken modules

sdetweil

@karsten13 docker is a pita with this stuff. too much/not enough in the container. which is supposed to be immutable.

KristjanESPERANTO

@sdetweil As a soft step, the broken modules could switch to @electron/rebuild. This should also work for docker.

wishmaster270

@KristjanESPERANTO

I have a docker installation running for development here on my notebook.
Everytime a module needs “npm install” I exec to the container and run the command in the container cause there will be called the correct npm and node version in the container and not the ones I currently use on my notebook.
In this case the directory structure is the same as with a locally installed MagicMirror and the postinstall script will work as excepted.

karsten13

@wishmaster270 said in electron-rebuild problems with MagicMirror 2.22.0:

and the postinstall script will work as excepted.

yes, will work, but all the changes in the node_modules folder of mm are gone with the next restart of the container …

wishmaster270

@karsten13
In case of electron-rebuild this is no problem as it is only required during compile but not at runtime.

karsten13

@wishmaster270

o.k., but where are the compile results stored?

wishmaster270

@karsten13
They are stored in the node_modules folder of the module which should be persistent

Edit: Do not misunderstand me. In my opinion the best way to solve the electron-rebuild problems of a lot of modules would be to put “@electron/rebuild” as a dependency to “package.json” of MagicMirror.