copy and tar problem

Hi:

I found there is some problem with SLE problem: each time I am
copying large files into removable disk or I am tar some big files, the
whole system became very slow: almost all the program and windows stuck
there… No matter how well (I get a 24 CPU (3.6 GHZ), 32 GB memory
workstation) is my CPU or memory is, the problem always happen. I don’t
install beagle and compiz in my system.

How to solve this problem ?

THX


albumns

albumns’s Profile: http://forums.novell.com/member.php?userid=48883
View this thread: http://forums.novell.com/showthread.php?t=444967

On Tue, 20 Sep 2011 06:46:04 GMT
albumns albumns@no-mx.forums.novell.com wrote:
[color=blue]

Hi:

I found there is some problem with SLE problem: each time I am
copying large files into removable disk or I am tar some big files,
the whole system became very slow: almost all the program and windows
stuck there… No matter how well (I get a 24 CPU (3.6 GHZ), 32 GB
memory workstation) is my CPU or memory is, the problem always
happen. I don’t install beagle and compiz in my system.

How to solve this problem ?

THX

[/color]
Hi
Sounds like I/O rather than CPU/RAM. How are you transferring to
removable disk, USB eSATA? How fast are the hard drives in your system,
10K RPM or SSD?


Cheers Malcolm °¿° (Linux Counter #276890)
openSUSE 11.4 (x86_64) Kernel 2.6.37.6-0.7-desktop
up 7 days 14:19, 3 users, load average: 0.08, 0.16, 0.19
GPU GeForce 8600 GTS Silent - Driver Version: 280.13

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Exactly. The main bottleneck of almost every system in the world is
I/O. When your system is busy using the disk, especially when it is
reading from it (to get the files) and writing to it (to create the tar)
at the same time your system will seem slow because of that bottleneck
no matter how much RAM or how many CPUs you have.

When you’re sitting at a stoplight (waiting on something else) it
doesn’t matter how fast your car goes or how much your pickup can carry;
the stop light is going to keep you from getting anything done.

Good luck.


Want to yell at me in person?
Come to BrainShare 2011 in October: http://tinyurl.com/brainshare2011
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.15 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJOeKDSAAoJEF+XTK08PnB5U38QAMLQFsIVYaCASpg/ZbNPDGmL
8kNCyW5Z3RRkTrBXm4VD4nu1jjDE5UV4/AAwagE6X4uEIFIOPu0IPlgW/5jIWY7f
pVJLcxVUxq2FxgFvE7wcwImhYOBx8EecEdz+fl5bH4vEQErsrs6pKKABXXivKhoM
SHzLuODunOLHJhbrhjkIWbESlgMS8PfQtjLTccqaYQRsZ+5naQ12jiuqYS1wXPNu
IhwhpI+2rVNbTSnx530oDFRkcPhAjvzoK85lfd07ZEPzNJLpWEu9DWwWJ2ci0UNO
oVVIkjA1OXV/24zWUcJWeAoTw+iLkuu0niOfq0U/ODXCvaIRUyRclwA+UWa2KeIx
ZFwIhFcs6oSQaR71dPHKm0zwFoSsjToFgcMAEFXds6E3ntSn7TAdyLuHwxyBzgVc
gqqFgBSKNus1NsUQjA7+awmAPxtOnTEqb1qPLi+9Q6C3+1h59ef+88tQApiAvWDU
hrIgWAvPLZ//8l/ZEj+HXG/pepsiiKNF0fOlAmjrztS8N0+bKW5at6a/CCiyqLkP
gPFSaGW8Vplo0XlguoA7OosFq5OgjQZ8Fn5XZheLEaUOI4uI65/MZlxAD3YhakAf
0q2wfCd+5S7735VVSty0z9Qtcf9CiDr6wf7OXMOPZQtNYdFOJUkzvt+0a9WIqSRT
ccpIq8TMcELKObkGwXJv
=8NEc
-----END PGP SIGNATURE-----

malcolmlewis;2138888 Wrote:[color=blue]

On Tue, 20 Sep 2011 06:46:04 GMT
albumns albumns@no-mx.forums.novell.com wrote:
[color=green]

Hi:

I found there is some problem with SLE problem: each time I am
copying large files into removable disk or I am tar some big files,
the whole system became very slow: almost all the program and windows
stuck there… No matter how well (I get a 24 CPU (3.6 GHZ), 32 GB
memory workstation) is my CPU or memory is, the problem always
happen. I don’t install beagle and compiz in my system.

How to solve this problem ?

THX

[/color]
Hi
Sounds like I/O rather than CPU/RAM. How are you transferring to
removable disk, USB eSATA? How fast are the hard drives in your system,
10K RPM or SSD?


Cheers Malcolm °¿° (Linux Counter #276890)
openSUSE 11.4 (x86_64) Kernel 2.6.37.6-0.7-desktop
up 7 days 14:19, 3 users, load average: 0.08, 0.16, 0.19
GPU GeForce 8600 GTS Silent - Driver Version: 280.13[/color]

my disk is 7200 rpm. I use USB removable disk for copying files. But I
don’t understand even I tar the files or untar files in local machine,
the whole system becomes very slow. Is there any methods to solve this
problem?

THX


albumns

albumns’s Profile: http://forums.novell.com/member.php?userid=48883
View this thread: http://forums.novell.com/showthread.php?t=444967

ab;2138966 Wrote:[color=blue]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Exactly. The main bottleneck of almost every system in the world is
I/O. When your system is busy using the disk, especially when it is
reading from it (to get the files) and writing to it (to create the
tar)
at the same time your system will seem slow because of that bottleneck
no matter how much RAM or how many CPUs you have.

When you’re sitting at a stoplight (waiting on something else) it
doesn’t matter how fast your car goes or how much your pickup can
carry;
the stop light is going to keep you from getting anything done.

Good luck.


Want to yell at me in person?
Come to BrainShare 2011 in October: http://tinyurl.com/brainshare2011
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.15 (GNU/Linux)
Comment: Using GnuPG with Mozilla - ‘Enigmail: A simple interface for
OpenPGP email security’ (http://enigmail.mozdev.org/)

iQIcBAEBAgAGBQJOeKDSAAoJEF+XTK08PnB5U38QAMLQFsIVYaCASpg/ZbNPDGmL
8kNCyW5Z3RRkTrBXm4VD4nu1jjDE5UV4/AAwagE6X4uEIFIOPu0IPlgW/5jIWY7f
pVJLcxVUxq2FxgFvE7wcwImhYOBx8EecEdz+fl5bH4vEQErsrs6pKKABXXivKhoM
SHzLuODunOLHJhbrhjkIWbESlgMS8PfQtjLTccqaYQRsZ+5naQ12jiuqYS1wXPNu
IhwhpI+2rVNbTSnx530oDFRkcPhAjvzoK85lfd07ZEPzNJLpWEu9DWwWJ2ci0UNO
oVVIkjA1OXV/24zWUcJWeAoTw+iLkuu0niOfq0U/ODXCvaIRUyRclwA+UWa2KeIx
ZFwIhFcs6oSQaR71dPHKm0zwFoSsjToFgcMAEFXds6E3ntSn7TAdyLuHwxyBzgVc
gqqFgBSKNus1NsUQjA7+awmAPxtOnTEqb1qPLi+9Q6C3+1h59ef+88tQApiAvWDU
hrIgWAvPLZ//8l/ZEj+HXG/pepsiiKNF0fOlAmjrztS8N0+bKW5at6a/CCiyqLkP
gPFSaGW8Vplo0XlguoA7OosFq5OgjQZ8Fn5XZheLEaUOI4uI65/MZlxAD3YhakAf
0q2wfCd+5S7735VVSty0z9Qtcf9CiDr6wf7OXMOPZQtNYdFOJUkzvt+0a9WIqSRT
ccpIq8TMcELKObkGwXJv
=8NEc
-----END PGP SIGNATURE-----[/color]

well, but why this doesn’t happen in windows…?


albumns

albumns’s Profile: http://forums.novell.com/member.php?userid=48883
View this thread: http://forums.novell.com/showthread.php?t=444967

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

It does, but you don’t notice it because windows is slow all of the
time. :wink:

If you really think it doesn’t happen in windows you’re really not
looking. I/O is a constant in the computing world… it’s inherent with
all kinds of things and especially disks with moving (spinning) parts.

Good luck.


Want to yell at me in person?
Come to BrainShare 2011 in October: http://tinyurl.com/brainshare2011
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.15 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJOeNI7AAoJEF+XTK08PnB5Uw8QALC9Nrh4dOgxDjn3+gbafk57
zGvFrKmu5QlkAnoLF8akpEcx/tfBoBOUXjfVkQ+LmGlw5MbR9RNmp+5TVEmfdz+A
hA0zCIeb+BSSB545E+DeFAGERdtEZXNlaB3lgeHK5AF5u2foc3K743dVLHaFY+28
MLNxyIBPf3r80MAa6jjILb8FThZzP5NxFmZWSAV+ZntApW7KuSUxB93uCAhUKi/P
VoeEVyrDkUPlrHVoLZB/Yv4MZJbv6iTkOO+ULiXjhcGOTdRWrFrEWCQQGgisKZB0
CoeeA4BlZcRoMeiJsypHIeAYjc00pHUJUFhOpU9e1KoO5peRg2+rV2exaPkfdAhz
4LbCazfKrdqAzsFm994oOCt5LRqpfy4RH8nukcqKEZUrxkXqEkSUdmniaYw60wLx
yTJRvF3KA8DkiFXdteJRcxOjCvpvEmpCynCQqnFnMHPlFwXgHgBRqG0KQYQrsZq+
cbHpNR0SG30/GMUcpTY+GexBjH/5OR40YH1yVHG+urvzu66ZWGNAe2eHV1GIbsBj
+D2M4xB59oNADo4B7HVXakaog9KEDLdv+aVtb5g22uObbjNYVZZflO/UymDCM7WN
S080MjsO9a4gtdskCVuwJ2lHOlFKzZilc4jLJ9JSiPW8G4Su/4o1MgcLbyloUDQG
2RUN4hsZmqa2s01PyyWk
=ekw/
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

How are you objectively measuring the slowness of the system
before/during/after the tar operation?

Good luck.


Want to yell at me in person?
Come to BrainShare 2011 in October: http://tinyurl.com/brainshare2011
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.15 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJOeNJ2AAoJEF+XTK08PnB5i3oQAIU59QCGeb78Ytuphp0jzqHP
jye1lgAHK+kSW5TYBDucOVBHnaTFlarp2rI5LDlo8JNNZoGOAtoI1JYs27BJk0KQ
DKtVOl93llCgRxaGS/X4VgGd+02V/R8pyJXijRgbZLH4L/ufqgo2xRyzSPNqX3D/
Yf7mHly3bcVvMtUdplM6vImYdKsUuioft8xopCBp0oKnp4NtbNYxwmObQIBGOHqZ
u3enqWqml37fHpJ+ChcrYHUdwNMhGKZV35+SUktq52pwF6a131vu4wEI1N2jD78s
jwJTTgKEN9M564yikmFYrf10PGHSuq5oSNc833v/pr1oSwpnrciCM2Y5LieIAitr
zl6h3tM+Ml+O7NWQRy5U5w+uB+i//4HZaRjTVGe9oVrgsz9UOA+GsktaUx5xJEeT
hQxZiCFjED+t/Kio9f6g5KJpv0GZqdyDhCdhGlvSp7Gun8Z5ETQ5rtZycjqavfee
RGKxMfwXVfi+e1ekR9859AiJYJbx94AS+SFwroXk4BsC750/C9RBF4qphhrnqicN
tSxmEymDHfIvwRE6Rz5+1Jw57e1s/tczW1sOcr7FjMuURLeuQB3LKwZJocskyiji
iu6NLmbRq9+BbHmQLqPDhXG5x7CXst2DGZaXRA9whiYXZlp0Y1yYdCsx3yk0/A7T
mhTFfFD2M7TuMAGls+Y4
=hx1z
-----END PGP SIGNATURE-----

On Tue, 20 Sep 2011 17:16:01 GMT
albumns albumns@no-mx.forums.novell.com wrote:
[color=blue]

ab;2138966 Wrote:[color=green]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Exactly. The main bottleneck of almost every system in the world is
I/O. When your system is busy using the disk, especially when it is
reading from it (to get the files) and writing to it (to create the
tar)
at the same time your system will seem slow because of that
bottleneck no matter how much RAM or how many CPUs you have.

When you’re sitting at a stoplight (waiting on something else) it
doesn’t matter how fast your car goes or how much your pickup can
carry;
the stop light is going to keep you from getting anything done.

Good luck.


Want to yell at me in person?
Come to BrainShare 2011 in October:
http://tinyurl.com/brainshare2011 -----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.15 (GNU/Linux)
Comment: Using GnuPG with Mozilla - ‘Enigmail: A simple interface
for OpenPGP email security’ (http://enigmail.mozdev.org/)

iQIcBAEBAgAGBQJOeKDSAAoJEF+XTK08PnB5U38QAMLQFsIVYaCASpg/ZbNPDGmL
8kNCyW5Z3RRkTrBXm4VD4nu1jjDE5UV4/AAwagE6X4uEIFIOPu0IPlgW/5jIWY7f
pVJLcxVUxq2FxgFvE7wcwImhYOBx8EecEdz+fl5bH4vEQErsrs6pKKABXXivKhoM
SHzLuODunOLHJhbrhjkIWbESlgMS8PfQtjLTccqaYQRsZ+5naQ12jiuqYS1wXPNu
IhwhpI+2rVNbTSnx530oDFRkcPhAjvzoK85lfd07ZEPzNJLpWEu9DWwWJ2ci0UNO
oVVIkjA1OXV/24zWUcJWeAoTw+iLkuu0niOfq0U/ODXCvaIRUyRclwA+UWa2KeIx
ZFwIhFcs6oSQaR71dPHKm0zwFoSsjToFgcMAEFXds6E3ntSn7TAdyLuHwxyBzgVc
gqqFgBSKNus1NsUQjA7+awmAPxtOnTEqb1qPLi+9Q6C3+1h59ef+88tQApiAvWDU
hrIgWAvPLZ//8l/ZEj+HXG/pepsiiKNF0fOlAmjrztS8N0+bKW5at6a/CCiyqLkP
gPFSaGW8Vplo0XlguoA7OosFq5OgjQZ8Fn5XZheLEaUOI4uI65/MZlxAD3YhakAf
0q2wfCd+5S7735VVSty0z9Qtcf9CiDr6wf7OXMOPZQtNYdFOJUkzvt+0a9WIqSRT
ccpIq8TMcELKObkGwXJv
=8NEc
-----END PGP SIGNATURE-----[/color]

well, but why this doesn’t happen in windows…?

[/color]
Hi
Maybe they change the io scheduling in the background, have no idea as
it’s all subjective.

Anyway, maybe using ionice to reschedule the activity, watching the
output of the command vmstat 2 may show info.

What is the exact command your running? Are you running tar as one
operation then copying, or running tar straight to the device?

With the device plugged in can you post the output from it’s dev
number eg;

hdparm -t /dev/sda

Are you using tar on windows? Remember tar is i/o bound it does a read
or a write then waits.

You need to look at something that will use all that processing
power… pbzip2 for example.
http://compression.ca/pbzip2/
http://software.opensuse.org/search?q=pbzip2&baseproject=SUSE%3ASLE-11%3ASP1&lang=en&exclude_debug=true


Cheers Malcolm °¿° (Linux Counter #276890)
openSUSE 11.4 (x86_64) Kernel 2.6.37.6-0.7-desktop
up 7 days 21:09, 6 users, load average: 0.00, 0.09, 0.17
GPU GeForce 8600 GTS Silent - Driver Version: 280.13

ab;2139108 Wrote:[color=blue]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

How are you objectively measuring the slowness of the system
before/during/after the tar operation?

[/color]

before the tar operations, everything goes well: I can do any
operations I want such as edit document. However, when I tar/untar
something big, my mouse stuck there and software windows become
white… I can’t do any operations anymore until the tar/untar
finished.

THX


albumns

albumns’s Profile: http://forums.novell.com/member.php?userid=48883
View this thread: http://forums.novell.com/showthread.php?t=444967

The only time I’ve seen behaviour like you describe was about five years
ago no a machine that was old enough to have a PATA harddisk and DMA had
somehow been disabled. The system was hopelessly unresponsive when there
was any significant disk access going on. Given the specs you mentioned
I very much doubt you have a PATA harddisk though.

What does this give you?

Code:

$ for i in {1…5};do hdparm -t /dev/sda;done

I get figures of around 80 MB/sec.

This sounds like an issue that could be very difficult to diagnose.
Have you considered raising a Service Request with Novell?

Hey look you can edit posts now!


mikewillis

mikewillis’s Profile: http://forums.novell.com/member.php?userid=7510
View this thread: http://forums.novell.com/showthread.php?t=444967

Honey, I don’t tar or untar files all the time in my computer. Of course
I can edit posts.

Here is the output of the command:

/dev/sda:
Timing buffered disk reads: 304 MB in 3.00 seconds = 101.28 MB/sec

/dev/sda:
Timing buffered disk reads: 250 MB in 3.00 seconds = 83.32 MB/sec

/dev/sda:
Timing buffered disk reads: 306 MB in 3.01 seconds = 101.80 MB/sec

/dev/sda:
Timing buffered disk reads: 306 MB in 3.01 seconds = 101.81 MB/sec

/dev/sda:
Timing buffered disk reads: 242 MB in 3.00 seconds = 80.63 MB/sec


albumns

albumns’s Profile: http://forums.novell.com/member.php?userid=48883
View this thread: http://forums.novell.com/showthread.php?t=444967

Hi,
i red all replies and have some idea :wink:

Are you using tar operation on removable or on physical disk? With file
system are you using?

Do you anytime check the value “%wa = waiting operation” in the top
during tar or coping? If is bigger then “%id” there is possible mistake
in the disk partitioning.
I had these problem with 4096-Byte Sector Hard Drives, and it sound
very similar. :wink: Witch OS create the partitions? If windows, that can
be problem. :wink:


Honzik

friedl_jan’s Profile: http://forums.novell.com/member.php?userid=56174
View this thread: http://forums.novell.com/showthread.php?t=444967

friedl_jan;2139335 Wrote:[color=blue]

Hi,
i red all replies and have some idea :wink:

Are you using tar operation on removable or on physical disk? With file
system are you using?

Do you anytime check the value “%wa = waiting operation” in the top
during tar or coping? If is bigger then “%id” there is possible mistake
in the disk partitioning.
I had these problem with 4096-Byte Sector Hard Drives, and it sound
very similar. :wink: Witch OS create the partitions? If windows, that can
be problem. ;-)[/color]

Thank you very much for kind reply. I do the tar operation in physical
disk and I use ext3 which is the same with removable disk. someone
suggest me to use this command to check something, but I don’t know
what’s that:

sudo /sbin/hdparm -i /dev/sda
/dev/sda:

Model=ST9500420AS, FwRev=D005SDM1, SerialNo=5VJ5J20R
Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4
BuffType=unknown, BuffSize=16384kB, MaxMultSect=16, MultSect=16
CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=976773168
IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: mdma0 mdma1 mdma2
UDMA modes: udma0 udma1 udma2 udma3 udma4 udma5 *udma6
AdvancedPM=yes: unknown setting WriteCache=enabled
Drive conforms to: unknown: ATA/ATAPI-4,5,6,7

  • signifies the current active mode


albumns

albumns’s Profile: http://forums.novell.com/member.php?userid=48883
View this thread: http://forums.novell.com/showthread.php?t=444967

here is the output for extracting files:

/dev/sda:
Timing buffered disk reads: 124 MB in 3.07 seconds = 40.43 MB/sec


albumns

albumns’s Profile: http://forums.novell.com/member.php?userid=48883
View this thread: http://forums.novell.com/showthread.php?t=444967

here is the out put for command ''for i in {1…5};do hdparm -t
/dev/sda;done" when extracting something

/dev/sda:
Timing buffered disk reads: 130 MB in 3.02 seconds = 43.01 MB/sec

/dev/sda:
Timing buffered disk reads: 98 MB in 3.04 seconds = 32.28 MB/sec

/dev/sda:
Timing buffered disk reads: 64 MB in 3.03 seconds = 21.09 MB/sec

/dev/sda:
Timing buffered disk reads: 106 MB in 3.05 seconds = 34.71 MB/sec

/dev/sda:
Timing buffered disk reads: 62 MB in 3.00 seconds = 20.67 MB/sec


albumns

albumns’s Profile: http://forums.novell.com/member.php?userid=48883
View this thread: http://forums.novell.com/showthread.php?t=444967