/dev/sda2 full

on a server with SLES 12, noticed that all of a sudden, the system would not go into the GUI, and complains that there is ‘no space left on device’. It does take me to the prompt, where doing a ‘df -kP’ command i can see that /dev/sda2 is 100%.
i have tried to truncate system log files, but only gained small space in Kbytes. I then planned on removing orphaned RPM packages, but observed that removal of RPM packages also complains that there is no space on device, therefore this plan of action did not work.
Please provide input in order to gain space so that i can boot properly into the GUI.

Hi
Sounds like you haven’t configured snapper with btrfs as your
filesystem…
https://www.suse.com/documentation/sles11/stor_admin/data/trbl_btrfs_volfull.html

I also look at the /etc/snapper/configs/root file and set the number
snapshot numbers down.

Also if doing peicemeal installs of this and that package the number
snapshots can grow quick in a day, so manually running the
suse.de-snapper cronjob can help as well.


Cheers Malcolm °¿° LFCS, SUSE Knowledge Partner (Linux Counter #276890)
SUSE Linux Enterprise Desktop 12 | GNOME 3.10.1 | 3.12.51-52.31-default
If you find this post helpful and are logged into the web interface,
please show your appreciation and click on the star below… Thanks!

ok , let me try your suggestions and will let you know of the outcome

It has been a tough battle and here is how it went. So i change the number of snapshots for TIMELINE_LIMIT_HOURLY,DAILY and MONTHLY to 5, it was originally 10. in /.snapshots i can see a lot of 2 and 3 digits folders, which relate to the snapshot taken. I try deleting the snapshot by using ‘snapper -c root delete #’ and manage to delete some snapshots but am unable to delete snapshot number 99. As i am in the process of deleting, suddenly the GUI appears and voila, i find my self inside the GUI (although the screen is on a low resolution). i check the usage using ‘df -kP’ and still have 100% usage on /dev/sda2 with around 8000 bytes free. I again try deleting snapshot 99, try to change permissions, but to no avail. I decide to reboot the machine, and i end up at the ‘grub rescue>’ prompt.
Then i loose 3-4 hours trying to atleast have a bootable machine with a valid grub. I manage to go from a ‘grub rescue>’ prompt to ‘grub>’ prompt by loading the normal.mod file, which in my case was not inside (hd0,msdos2)/boot/grub2 folder but i had to navigate to the snapshot folder and restore it from there. Then i restored grub.cfg from (hd0,msdos2)/usr/lib/grub folder, and manage to get to the GUI, alas!
once in the gui, i try to remove some software installation that are not used any more. I notice that whenever i uninstall something the log file size increases. I then truncate the log files in /var/log.
At this stage i have remove snapper, so that it stops taking snapshots every few minutes. Also i have shut the system, until i can figure out what is causing the high space usage on /dev/sda2.

Hi
You need to run the balance job to help clean things out, it may also
help with the stuck snapshot. Boot and shutdown the GUI and drop to
runlevel 3 and work from the console VT (press ctrl+alt+F1);


Cheers Malcolm °¿° LFCS, SUSE Knowledge Partner (Linux Counter #276890)
SUSE Linux Enterprise Desktop 12 | GNOME 3.10.1 | 3.12.51-52.31-default
If you find this post helpful and are logged into the web interface,
please show your appreciation and click on the star below… Thanks!

can you tell me how to run the balance job? how do I initiate this?

Hi
From the link :wink:

btrfs fi usage
btrfs fi balance start /dev/sda2 -dusage=5

That should help clean it up, it may take some time so be patient.

it says ERROR: can’t access /dev/sda2

alright, so i mounted /dev/sda2 as /mnt:

mount -t btrfs /dev/sda2 /mnt

then did:
btrfs fi balance start /mnt -dusage=5

and now i get the following message:
Done, had to relocate 0 out of 54 chunks

Hi
You might need to run it a few times, also try increasing the dusage number.

i increased the dusage number to 100 and it says:

ERROR : error during balancing /mnt no space left on device