MagicMirror Forum
    • Recent
    • Tags
    • Unsolved
    • Solved
    • MagicMirror² Repository
    • Documentation
    • 3rd-Party-Modules
    • Donate
    • Discord
    • Register
    • Login
    A New Chapter for MagicMirror: The Community Takes the Lead
    Read the statement by Michael Teeuw here.

    A few days after upgrading to 2.4.1, my disk filled up and now my mirror won't run. Sad face!

    Scheduled Pinned Locked Moved Solved Troubleshooting
    4 Posts 2 Posters 1.4k Views 2 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • P Offline
      Preschool girl
      last edited by

      Upgraded. Did all the config changes. Worked great. Went out of town. Came back to a blank mirror. Was confused to find the screen and RPi still working fine. SSH’ed in, and every pm2 command I tried gave me

      [PM2] Spawning PM2 daemon with pm2_home=/home/pi/.pm2
      

      Much googling ensued. Tried changing permission on a couple of files. Didn’t work. Finally decided it must mean my disk is full.

      pi@raspberrypi:~ $ df -h
      Filesystem      Size  Used Avail Use% Mounted on
      /dev/root        13G   13G     0 100% /
      devtmpfs        333M     0  333M   0% /dev
      tmpfs           462M     0  462M   0% /dev/shm
      tmpfs           462M   12M  450M   3% /run
      tmpfs           5.0M  4.0K  5.0M   1% /run/lock
      tmpfs           462M     0  462M   0% /sys/fs/cgroup
      /dev/mmcblk0p6   68M   21M   47M  31% /boot
      tmpfs            93M     0   93M   0% /run/user/1000
      

      So suspicions confirmed I guess?

      I used ncdu to try to figure out if something ballooned out of control and that’s when I realized I’m a complete beginner and have no idea what to look for. Here’s what I saw:

      --- / ------------------------------------------------------------------------------------------------------------------------------
      .   7.4 GiB [##########] /var                                                                                                       
          2.8 GiB [###       ] /usr
          1.0 GiB [#         ] /home
        945.6 MiB [#         ] /opt
        182.2 MiB [          ] /lib
         21.0 MiB [          ] /boot
      .  11.9 MiB [          ] /run
          7.3 MiB [          ] /sbin
          6.8 MiB [          ] /bin
      .   5.7 MiB [          ] /etc
      . 228.0 KiB [          ] /tmp
      !  16.0 KiB [          ] /lost+found
      .  12.0 KiB [          ] /media
      e   4.0 KiB [          ] /srv
      !   4.0 KiB [          ] /root
      e   4.0 KiB [          ] /mnt
      .   0.0   B [          ] /sys
      .   0.0   B [          ] /proc
          0.0   B [          ] /dev
      

      Selecting that top directory (assuming the culprit is somewhere in there), I see this:

      --- /var ---------------------------------------------------------------------------------------------------------------------------
                               /..
      .   7.1 GiB [##########] /log                                                                                                       
      . 145.6 MiB [          ] /cache
      . 115.3 MiB [          ] /lib
        100.0 MiB [          ]  swap
        296.0 KiB [          ] /backups
      .  68.0 KiB [          ] /tmp
      .  28.0 KiB [          ] /spool
      e   4.0 KiB [          ] /opt
      e   4.0 KiB [          ] /mail
      e   4.0 KiB [          ] /local
      @   0.0   B [          ]  lock
      @   0.0   B [          ]  run  
      

      Hm. Let’s try that top one again. Here’s what we get (truncated to the directories that take up any substantial space):

      --- /var/log -----------------------------------------------------------------------------------------------------------------------
                               /..                                                                                                        
          2.4 GiB [##########]  kern.log
          1.8 GiB [#######   ]  messages
          1.4 GiB [#####     ]  messages.1
        534.7 MiB [##        ]  syslog.1
        533.4 MiB [##        ]  kern.log.1
        287.9 MiB [#         ]  syslog
         24.9 MiB [          ]  syslog.3.gz
      
      

      And that’s where I’m stuck. Is my culprit in there? Can I delete it? How can I prevent this from happening in the future?

      1 Reply Last reply Reply Quote 0
      • M Offline
        MadScientist
        last edited by

        I think you should be able to delete everything from /var/log as these are just log files. But the question remains why the logs are inflated so much. Something gets triggered to write something in the logs. I wouldn’t know how to find out what it is as I am a total n00b myself (even after years of using Linux).

        1 Reply Last reply Reply Quote 0
        • P Offline
          Preschool girl
          last edited by

          Thanks for the response!

          Based on that, I went ahead and deleted those guys. I did this:

          pi@raspberrypi:~ $ sudo rm /var/log/kern* &>/dev/null
          pi@raspberrypi:~ $ sudo rm /var/log/messages* &>/dev/null
          

          …then rebooted and everything came back to life.

          I’ve now got a little breathing room on my disk:

          pi@raspberrypi:~ $ df -h
          Filesystem      Size  Used Avail Use% Mounted on
          /dev/root        13G   11G  1.4G  89% /
          devtmpfs        333M     0  333M   0% /dev
          tmpfs           462M     0  462M   0% /dev/shm
          tmpfs           462M   12M  450M   3% /run
          tmpfs           5.0M  4.0K  5.0M   1% /run/lock
          tmpfs           462M     0  462M   0% /sys/fs/cgroup
          /dev/mmcblk0p6   68M   21M   47M  31% /boot
          tmpfs            93M     0   93M   0% /run/user/1000
          

          And I guess I need to implement some kind of solution to “rotate my logs” if that’s the correct nomenclature?

          The question remains: why did this happen? I know one of my modules is kind of noisy in the logs – like it updates once a minute and even though it’s working it sends an error message to the logs every time. But it’s hard to imagine that that one-line error would take up multiple gigs in a couple months. I dunno.

          Maybe “log rotation” should be addressed in the config instructions?

          1 Reply Last reply Reply Quote 0
          • P Offline
            Preschool girl
            last edited by

            For future generations looking to solve this problem:

            I believe one of my modules (MMM-CTA for the record) poops out a ton of log entries. Those fill up my SD card after a few weeks.

            I ended up installing a pm2 module called pm2-logrotate. https://www.npmjs.com/package/pm2-logrotate

            I’m setting it to rotate my var/log file before it gets too big. Hopefully this will solve future problems.

            pi@raspberrypi:~ $ pm2 install pm2-logrotate
            pi@raspberrypi:~ $ pm2 set pm2-logrotate:max_size 2G
            
            1 Reply Last reply Reply Quote 0
            • 1 / 1
            • First post
              Last post
            Enjoying MagicMirror? Please consider a donation!
            MagicMirror created by Michael Teeuw.
            Forum managed by Sam, technical setup by Karsten.
            This forum is using NodeBB as its core | Contributors
            Contact | Privacy Policy