What would you guys recommend as the best filesystem to use
for a squid proxy on SLES 11 SP3.
Bare in mind squid writes tons of small short-lived files to the filesystem.
Today we’re using ext3, but we’re seeing issues where it seems to be
to slow, thinking ext3 might not be the most optimal fs for this.
There’s reiserfs, but giving it’s future I don’t see this to be the best
supported way to go.
Anyone have any tips?
Thanks
Hi jdoeconsulting,
I know this is pretty old, but I’ve stumbled across an interesting comparison wrt small parallel writes: http://marc.info/?l=linux-fsdevel&m=107619075328201 The source of the test program is attached and fairly small, it could be compiled and run to test various scenarios (different file systems, different configurations)…
Running that test on a modern disk and ext3 (with data=ordered) definitely doesn’t show that decrease as reported in that message - unless I run two tests in parallel, making write & read jobs run at the same time.
Single test run:
Writing 256Mb of data to 1 files in parallel
Transferred 256Mb of data in 3.38 seconds (75.78Mb/s)
Writing 128Mb of data to 2 files in parallel
Transferred 256Mb of data in 3.43 seconds (74.65Mb/s)
Writing 4Mb of data to 64 files in parallel
Transferred 256Mb of data in 4.69 seconds (54.53Mb/s)
Two runs in parallel:
[CODE]
Writing 256Mb of data to 1 files in parallel
Transferred 256Mb of data in 3.78 seconds (67.64Mb/s)
Writing 128Mb of data to 2 files in parallel
Transferred 256Mb of data in 6.94 seconds (36.89Mb/s)
Writing 4Mb of data to 64 files in parallel
Transferred 256Mb of data in 29.03 seconds (8.82Mb/s)[/CODE]
But there’s a lot to take into consideration with this simplified test, especially caching strategies. And that’s also a pointer where you might be able to get a significant performance improvement, without changing the file system: By fine-tuning the system-level cache and choosing/configuring the best i/o scheduler for your device.
Speaking of the device: Have you actually monitored the file system and disk behavior (disk i/o, waits, memory consumption) or are your statements based on an application-level experience of slowness?
Regards,
Jens
Thanks for the reply Jens!
No the statements are based on application-level experience.
I’ve asked to do an actual monitoring of file and disk system behavior
in order to be able to decide what would be best.
Anyway I wanted to throw the question out just to hear what others
are using. I mean choosing the right file system isn’t only based on
speed, there are so many other factors to take in account as well.
Thanks!