Hi,
I’m having disk-space problems and discovered, that a program I use produces quite large files, which then seemingly use up space, although they are “deleted”.
So, how does this happen? And is it something I, as admin can do anything about, or is this something code-specific? Could I for example write a script, that deletes all the deleted files every now and then?
Thanks, Sven
Hi Sven,
in unixoid environments, files are removed from disk when there are no more references to it.
Typical “references” are directory entries - those that you remove via “rm”… if no more directory entries point to a file, it is considered “deleted” (but not necessarily removed!)
Another type of reference are open file handles.
So a file can be “deleted”, but still remain in the file system, if there are programs that have open file handles to the deleted file. A typical case is “syslogd”: You may remove /var/log/messages, but the space will not be cleared until syslogd closes that file… that’s why there is a “restart syslogd” on log file rotation with some versions of syslogd. The sames goes for other programs.
You as the admin can only check which processes hold open file handles and decide to stop/restart these programs, if no other option is available to make the program close the file handle.
Regards,
Jens
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I’ve seen ‘(deleted)’ in lsof output a bunch of times, often with
Java-based applications which go out and build a JAR, use it, and then
delete it but still seem to show it.
Can you delete it again? No… the file is not there (go look) so there
is nothing by which you can refer to it. You could perhaps find its
inode and write zeroes to it, but to what point and purpose?
If it really is the deleted files solely causing the problem you could
restart the application from time to time, or maybe you could grow the
disk. There may be a way to configure the application to release the
files more-quickly, but I’m not familiar enough with the application
(what is it?) or why lsof still shows things as ‘deleted’. Also, how
are you seeing disk space and is it accurate?
If you want to find out what is in those files (if you do not already
know) this may be an interesting to see what data are in them:
http://www.serverwatch.com/tutorials/article.php/3822816/Recovering-Deleted-Files-With-lsof.htm
Good luck.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/
iQIcBAEBAgAGBQJQoSX8AAoJEF+XTK08PnB5izMP/1WhO+zHJAM5TKVomY1xP9G7
AfY4cK5t3GP2nLcCqDkfA8bi8TTb4NsIfuXwX2ghZWdLNdX4q1vViZHKhOfYLesi
l8PjxjPxT2bUl+VQtaqEyb8Dso5UKtwfKIdmEcr73E/qF9l/lpHAa9lWGHp4sAGK
8toXnsb5C3TIZO85KnRqQl9LklJ2Bk9VlQEOePMEIPR+Ia/vuwMO7rY8UNHVbvlM
WB1CNemXNLmOWStS1rWia/kKtQojkvphzEDEO2GFqEw4p9rahdao6MmytxeGsF0g
PSV/e3DQmGBu6vbfJQ8lbnyljQHEO8/d6T9Nc/3DqMHKkK/ADi6n1o/bxcJZBS2A
RY9H+ZlItC4Vddz1YxZXx/Q+5e1N+16lpFdco0vdxfkmSB3T/GhRiR6One+4ckAT
BlLrOvQ368pREZhXOIUelmWyGGT0iDsxfuCSr6DqHEMwcTHXeSznCNXun61qlwe9
BOQVB8Df5bnsckObqAd8uyglIGwzifS2oY5MRuP+J0v+dfapYZPnwy52luES5cQf
p1rMNnYOKCQxfHLmmXpLJsgTrLmnWdtdkeho4Kxw6KCdqCVlU4KczsCiC9SP49hk
fC3FljUHBpF1PX3ZEtvM+iHkt17x5v+gw3mZfOl9Zbvsb3m8Jb9AK/ADuqoXvG9i
F6ynnxA4O7oZtaeIAd2w
=hZr0
-----END PGP SIGNATURE-----
Ok, thanks to both of you.
(We work with the program, so I have to let it run. Maybe I’ll write the authors, think the last version of it didn’t do this (as much at least - the files sum up to around 100 GB, depending on the jobs, probably), so it may be some kind of bug. Have to test it, but wanted to make sure it’s not me having set some linux- or filesystem-options wrong. And depending on df and du the disk is full or rather empty, Guest, but as other jobs cancel with a “not enough disk space”-error…)