We have been running Rancher 1.0.x for a while now in production sine its GA release.
However we have just noticed that Rancher server creates cattle .tmp logs files which seems to be never removed/cleaned up …so they eat up lot of space on the server.
This is the current situation:
we are on rancher 1.0.1. See the current undesired situation with huge .tmp files that are never deleted…I expect rancher to delete them itself. our server crashed because of this.
root@2fbdf877da7a:/var/lib/cattle/logs# ls -lh
total 15G
-rw-r--r--. 1 root root 3616623 Apr 11 02:00 cattle-debug.2016-04-10.log.gz
-rw-r--r--. 1 root root 10080577 Apr 12 02:00 cattle-debug.2016-04-11.log.gz
-rw-r--r--. 1 root root 26416665 Apr 13 02:00 cattle-debug.2016-04-12.log.gz
-rw-r--r--. 1 root root 26188300 Apr 14 02:00 cattle-debug.2016-04-13.log.gz
-rw-r--r--. 1 root root 16296914 Apr 15 02:00 cattle-debug.2016-04-14.log.gz
-rw-r--r--. 1 root root 13780663 Apr 16 02:00 cattle-debug.2016-04-15.log.gz
-rw-r--r--. 1 root root 13442126 Apr 17 02:00 cattle-debug.2016-04-16.log.gz
-rw-r--r--. 1 root root 13371954 Apr 18 02:00 cattle-debug.2016-04-17.log.gz
-rw-r--r--. 1 root root 13785917 Apr 19 02:00 cattle-debug.2016-04-18.log.gz
-rw-r--r--. 1 root root 14068817 Apr 20 02:00 cattle-debug.2016-04-19.log.gz
-rw-r--r--. 1 root root 13769198 Apr 21 02:00 cattle-debug.2016-04-20.log.gz
-rw-r--r--. 1 root root 14020577 Apr 22 02:00 cattle-debug.2016-04-21.log.gz
-rw-r--r--. 1 root root 4521191 Apr 23 02:00 cattle-debug.2016-04-22.log.gz
-rw-r--r--. 1 root root 5256146 Apr 24 02:00 cattle-debug.2016-04-23.log.gz
-rw-r--r--. 1 root root 4349392 Apr 25 02:00 cattle-debug.2016-04-24.log.gz
-rw-r--r--. 1 root root 5170880 Apr 26 02:00 cattle-debug.2016-04-25.log.gz
-rw-r--r--. 1 root root 5229840 Apr 27 02:00 cattle-debug.2016-04-26.log.gz
-rw-r--r--. 1 root root 4426412 Apr 28 02:00 cattle-debug.2016-04-27.log.gz
-rw-r--r--. 1 root root 7373921 Apr 29 02:00 cattle-debug.2016-04-28.log.gz
-rw-r--r--. 1 root root 5240734 Apr 30 02:00 cattle-debug.2016-04-29.log.gz
-rw-r--r--. 1 root root 6186057 May 1 02:00 cattle-debug.2016-04-30.log.gz
-rw-r--r--. 1 root root 5590913 May 2 02:00 cattle-debug.2016-05-01.log.gz
-rw-r--r--. 1 root root 7320029 May 3 02:00 cattle-debug.2016-05-02.log.gz
-rw-r--r--. 1 root root 34909908 May 4 02:00 cattle-debug.2016-05-03.log.gz
-rw-r--r--. 1 root root 46470978 May 5 02:00 cattle-debug.2016-05-04.log.gz
-rw-r--r--. 1 root root 41325638 May 5 11:29 cattle-debug.2016-05-05.log.gz
-rw-r--r--. 1 root root 44818120 May 6 11:02 cattle-debug.2016-05-06.log.gz
-rw-r--r--. 1 root root 64912972 May 7 21:53 cattle-debug.2016-05-07.log.gz
-rw-r--r--. 1 root root 37947657 May 8 08:53 cattle-debug.2016-05-08.log.gz
-rw-r--r--. 1 root root 54382757 May 9 12:30 cattle-debug.2016-05-09.log.gz
-rw-r--r--. 1 root root 22176866 May 10 05:28 cattle-debug.2016-05-10.log.gz
-rw-r--r--. 1 root root 339335267 May 10 10:00 cattle-debug.log
-rw-r--r--. 1 root root 1073742821 May 5 21:36 cattle-debug.log123279789857127.tmp
-rw-r--r--. 1 root root 454023239 May 6 01:59 cattle-debug.log139069242902314.tmp
-rw-r--r--. 1 root root 1073744868 May 6 13:26 cattle-debug.log180252244661795.tmp
-rw-r--r--. 1 root root 770165852 May 7 01:59 cattle-debug.log225467195719777.tmp
-rw-r--r--. 1 root root 223657321 May 8 01:59 cattle-debug.log311865602972644.tmp
-rw-r--r--. 1 root root 1073745429 May 8 11:04 cattle-debug.log344541221251161.tmp
-rw-r--r--. 1 root root 1073743036 May 8 13:15 cattle-debug.log352423666911846.tmp
-rw-r--r--. 1 root root 1073744440 May 8 15:25 cattle-debug.log360202136775544.tmp
-rw-r--r--. 1 root root 1073745901 May 8 17:35 cattle-debug.log367969405601005.tmp
-rw-r--r--. 1 root root 718147786 May 9 01:59 cattle-debug.log398266002053867.tmp
-rw-r--r--. 1 root root 1073746549 May 9 16:53 cattle-debug.log451905266953631.tmp
-rw-r--r--. 1 root root 1073742101 May 9 20:14 cattle-debug.log463910235216070.tmp
-rw-r--r--. 1 root root 1073743960 May 9 23:21 cattle-debug.log475134628399076.tmp
-rw-r--r--. 1 root root 837150726 May 10 01:59 cattle-debug.log484668479728887.tmp
-rw-r--r--. 1 root root 1073744104 May 5 12:11 cattle-debug.log89370040680529.tmp
-rw-r--r--. 1 root root 379182 Apr 22 06:39 cattle-error.2016-04-21.log.gz
-rw-r--r--. 1 root root 7565 Apr 23 03:30 cattle-error.2016-04-22.log.gz
-rw-r--r--. 1 root root 10616 Apr 24 02:53 cattle-error.2016-04-23.log.gz
-rw-r--r--. 1 root root 5876 Apr 25 02:30 cattle-error.2016-04-24.log.gz
-rw-r--r--. 1 root root 13331 Apr 26 04:35 cattle-error.2016-04-25.log.gz
-rw-r--r--. 1 root root 17338 Apr 27 15:21 cattle-error.2016-04-26.log.gz
-rw-r--r--. 1 root root 5355 Apr 28 06:56 cattle-error.2016-04-27.log.gz
-rw-r--r--. 1 root root 55698 Apr 29 06:31 cattle-error.2016-04-28.log.gz
-rw-r--r--. 1 root root 5433 Apr 30 15:32 cattle-error.2016-04-29.log.gz
-rw-r--r--. 1 root root 29225 May 1 03:12 cattle-error.2016-04-30.log.gz
-rw-r--r--. 1 root root 12309 May 2 02:07 cattle-error.2016-05-01.log.gz
-rw-r--r--. 1 root root 41745 May 3 02:10 cattle-error.2016-05-02.log.gz
-rw-r--r--. 1 root root 705821 May 4 02:01 cattle-error.2016-05-03.log.gz
-rw-r--r--. 1 root root 804258 May 5 02:00 cattle-error.2016-05-04.log.gz
-rw-r--r--. 1 root root 3248960 May 6 02:00 cattle-error.2016-05-05.log.gz
-rw-r--r--. 1 root root 5771468 May 7 02:00 cattle-error.2016-05-06.log.gz
-rw-r--r--. 1 root root 4494922 May 8 02:00 cattle-error.2016-05-07.log.gz
-rw-r--r--. 1 root root 2402093 May 9 02:00 cattle-error.2016-05-08.log.gz
-rw-r--r--. 1 root root 3511086 May 10 02:00 cattle-error.2016-05-09.log.gz
-rw-r--r--. 1 root root 9262076 May 11 02:13 cattle-error.2016-05-10.log.gz
-rw-r--r--. 1 root root 7236942 May 12 02:05 cattle-error.2016-05-11.log.gz
-rw-r--r--. 1 root root 6040423 May 13 02:04 cattle-error.2016-05-12.log.gz
-rw-r--r--. 1 root root 7342225 May 14 02:04 cattle-error.2016-05-13.log.gz
-rw-r--r--. 1 root root 8766257 May 15 02:04 cattle-error.2016-05-14.log.gz
-rw-r--r--. 1 root root 10843837 May 16 02:07 cattle-error.2016-05-15.log.gz
-rw-r--r--. 1 root root 9029674 May 17 02:03 cattle-error.2016-05-16.log.gz
-rw-r--r--. 1 root root 15014986 May 18 01:24 cattle-error.2016-05-17.log.gz
-rw-r--r--. 1 root root 10486872 May 19 02:03 cattle-error.2016-05-18.log.gz
-rw-r--r--. 1 root root 14540955 May 20 02:07 cattle-error.2016-05-19.log.gz
-rw-r--r--. 1 root root 10061248 May 21 02:06 cattle-error.2016-05-20.log.gz
-rw-r--r--. 1 root root 257947779 May 21 12:33 cattle-error.log
-rw-r--r--. 1 root root 984259307 May 18 01:57 cattle-error.log456382747586183.tmp
I have also posted a comment here
What shall we do in the mean time to avoid server crashes?
recreate the rancher server container (we have a distinct one for the mysql data) every week? What is Rancher proper planned solution to this?