Slow gigabit networking!?! (6-10%)

[Nathanael]

This is my connection scheme:

When copying to my PC from Server I get the following results:

The server PC is Windows 2003 Enterprise Server R2 x64, my PC is Vista Ultimate x64. I've tried copying Vista to Vista and XP to XP with the same results. All the connections are Cat5e cables.

NIC in Server: Realtek 8169S on PCI

NIC in My PC: Marvell Yukon 88E8053 on PCI-E

Am I missing something? ...the performance shouldn't be as bad.

[Paul]

What kind of cable are you using? I might be wrong but as far as I know CAT5 and CAT5e isn't good enough for gigabit networking. It has to be CAT6. Both use RJ45 connectors.

I could be wrong however.

[Colin-uk]

Yeah, thats what I thought too, you may need Cat6 Cable..

[Simon-]

CATe is fine for Gigabit at short distances, it's more to do with your system bandwidth when it comes to gigabit performance. I'm leaning towards the Realtek 8169S on the PCI Bus being the bottleneck in this network

[Nathanael]

What kind of cable are you using? I might be wrong but as far as I know CAT5 and CAT5e isn't good enough for gigabit networking. It has to be CAT6. Both use RJ45 connectors.I could be wrong however.

http://www.windowsnetworking.com/articles_...t-Ethernet.html

Currently, the only twisted pair cables that support gigabit Ethernet are CAT5E and CAT6.

Doesn't seem to be the case.

CATe is fine for Gigabit at short distances, it's more to do with your system bandwidth when it comes to gigabit performance. I'm leaning towards the Realtek 8169S on the PCI Bus being the bottleneck in this network

Tried copying from Broadcom NetXtreme PCI-E gigabit adapter in a HP nx9420 notebook with same results.

[+StevoFC]

It's been said already...but use cat 6. That will help a lot i'm sure because it is less likely to receive interference.

[Nathanael]

It's been said already...but use cat 6. That will help a lot i'm sure because it is less likely to receive interference.

Surely cat5e can't run 10-times slower than it should?

[Simon-]

CAT6 or not, I doubt that you will get much better than about 15% utilisation

[Nathanael]

CAT6 or not, I doubt that you will get much better than about 15% utilisation

The gbit networks max out at ~200mbits generally than?

[betasp]

Cat5e should be fine for gigabit. I have a whole building to prove it. Are you sure your ports on the switch are sensing gigabit? Are you using Jumbo Frames and is it supported by your switch? What is the internal bandwidth of the switch?

[bobbba]

what results do you expect? The pc and the server are not going to generate much traffic as their other components will affect the amount of throughput. Get more PC's and more servers and you will see the utilisation go up.

If you are getting 6% of 1000Mb, that would be 60Mb per sec. I don't think that you've got a network problem here. Just performance limited by other bottle necks like hdd storage.

[Nathanael]

Cat5e should be fine for gigabit. I have a whole building to prove it. Are you sure your ports on the switch are sensing gigabit? Are you using Jumbo Frames and is it supported by your switch? What is the internal bandwidth of the switch?

Gigabit is detected...

Jumbo frames? Probably not - where can I read up on it?

Internal switching speed, while not mentioned should be sufficient:

Five (5) 10/100/1000BASE-T Gigabit Ethernet ports

Cable Diagnostic function at Switch boot up

Supports Auto-Negotiation for 10/100/1000Mbps and duplex mode

Supports Auto-MDI/MDIX for each port

Supports Full/Half duplex transfer mode for 10 and 100Mbps

Supports Full-duplex transfer mode for 1000Mbps Full wire speed reception and transmission Store-and-Forward switching method

Supports 4K absolute MAC addresses

Supports 1M Bits RAM for data buffering

Extensive front-panel diagnostic LEDs

IEEE 802.3x flow control for full duplex

Back pressure flow control for half duplex

[The_Decryptor]

Yeah, 15 MBps throughput seems a bit low, 100Mbps ethernet tops out at 12.5MBps (edit: you'll never hit that though).

Edit: try unplugging the other devices from the switch and try it again.

[Nathanael]

Yeah, 15 MBps throughput seems a bit low, 100Mbps ethernet tops out at 12.5MBps (edit: you'll never hit that though).

Edit: try unplugging the other devices from the switch and try it again.

I'll try the ad-hoc setup later today...

Could the DHCP on the router be setting the thing up wrong?

[The_Decryptor]

nah, DHCP won't effect throughput.

A small packet size could slow it down though, that's where jumbo packets come in.

[dragon2611]

Do you have antivirus software running on one/both boxes?

Try turning the resident sheild off for a moment see if it makes a difference.

I found that the resident shield sometimes slows down a network transfer as the virusscanner scans everything as its being read/written to the disk

[Nathanael]

Do you have antivirus software running on one/both boxes?

Try turning the resident sheild off for a moment see if it makes a difference.

I found that the resident shield sometimes slows down a network transfer as the virusscanner scans everything as its being read/written to the disk

NOD32 - no change

[bobbba]

NOD32 - no change

Really? :rolleyes:

[gertin]

What kind of hdd do you have in your computer? Is the load on the hdd very high when copying? Is the drive heavily fragmented?

[+BudMan]

I can assure you cat6 is NOT need to run gig. Period end of story!

Cat5e is more than capable of providing you the bandwidth you would ever be able to generate.

Jumbo frames could give you a boost.. That switch does support it. But you could have performance issues when talking to machines that are not running them, etc. Jumbo frames can produce a reduction in overhead - but is not always worth setting up. Depends on what exactly your wanting to accomplish.

15Mbps does seems a bit slow - why don't you grab Iperf an check your wire speed.. http://dast.nlanr.net/Projects/Iperf/

For example - here is a test between 2 machines connected at gig - home type equipment! I also have a dqs-105, the nics in this test are ga-311 on a p4-2.4ghz an a builtin intel Pro 1000 MT p4-2.8ghz -- very moderate equipment in this day an age. Hard drives are nothing special. The one is running a raid0 off of 2 old maxtor 7200rpm 40gb drives I picked up for like 30 dollars each I do believe. And the 2.8 is running a single 7200rpm 200gb drive I picked up on sale for like $50 quite some time ago.. Hardware is clearly nothing to brag about.. But performance is not bad at all!

C:\iperf>iperf -c 192.168.1.4 -w 256k

------------------------------------------------------------

Client connecting to 192.168.1.4, TCP port 5001

TCP window size: 256 KByte

------------------------------------------------------------

[1920] local 192.168.1.100 port 3747 connected with 192.168.1.4 port 5001

[ ID] Interval Transfer Bandwidth

[1920] 0.0-10.0 sec 857 MBytes 718 Mbits/sec

Or here;

C:\iperf>iperf -c 192.168.1.4 -w 256k -fM

------------------------------------------------------------

Client connecting to 192.168.1.4, TCP port 5001

TCP window size: 0.25 MByte

------------------------------------------------------------

[1920] local 192.168.1.100 port 3812 connected with 192.168.1.4 port 5001

[ ID] Interval Transfer Bandwidth

[1920] 0.0-10.0 sec 861 MBytes 86.0 MBytes/sec

Now lets bump that up to 4k jumbo -- only common size the cards support :(

C:\iperf>iperf -c 192.168.1.4 -w 256k

------------------------------------------------------------

Client connecting to 192.168.1.4, TCP port 5001

TCP window size: 256 KByte

------------------------------------------------------------

[1920] local 192.168.1.100 port 4012 connected with 192.168.1.4 port 5001

[ ID] Interval Transfer Bandwidth

[1920] 0.0-10.0 sec 891 MBytes 746 Mbits/sec

C:\iperf>iperf -c 192.168.1.4 -w 256k -fM

------------------------------------------------------------

Client connecting to 192.168.1.4, TCP port 5001

TCP window size: 0.25 MByte

------------------------------------------------------------

[1920] local 192.168.1.100 port 4025 connected with 192.168.1.4 port 5001

[ ID] Interval Transfer Bandwidth

[1920] 0.0-10.0 sec 887 MBytes 88.6 MBytes/sec

As you can see there is a bit of a boost..

Iperf will tell you want your cpu an wire is capable of of. But that rarely is your bottle neck -- do a simple file copy.

As you can see from here Cat5e with common nics and hard drives is more than capable of well above 15Mbps

C:\test>robocopy \\p4-24g\c$\test c:\test ubuntu-7.04-server-i386.iso /NP

-------------------------------------------------------------------------------

ROBOCOPY :: Robust File Copy for Windows :: Version XP010

-------------------------------------------------------------------------------

Started : Sat Jul 14 21:04:01 2007

Source : \\p4-24g\c$\test\

Dest : c:\test\

Files : ubuntu-7.04-server-i386.iso

Options : /COPY:DAT /NP /R:1000000 /W:30

------------------------------------------------------------------------------

1 \\p4-24g\c$\test\

New File 492.4 m ubuntu-7.04-server-i386.iso

------------------------------------------------------------------------------

Total Copied Skipped Mismatch FAILED Extras

Dirs : 1 0 1 0 0 0

Files : 1 1 0 0 0 0

Bytes : 492.41 m 492.41 m 0 0 0 0

Times : 0:00:17 0:00:17 0:00:00 0:00:00

Speed : 29270726 Bytes/sec.

Speed : 1674.884 MegaBytes/min.

Ended : Sat Jul 14 21:04:18 2007

Give us some more details of how your testing - other than looking at taskmanagers performance graphs :rolleyes: An I am sure you can find some help in figuring out if your at the limit of your hardware - or if there is something you can do to boost its performance!

[tao muon]

didn't see anyone mention IPv6.

[+BudMan]

didn't see anyone mention IPv6.

An what exactly would IPv6 have to do with bandwidth between 2 machines connected to the same switch?

[Nathanael]

With iPerf I get a speed of 133 mbits.

Do you have only gigabit devices connected to your switch?

[+BudMan]

No I have 100mbit devices also connected directly to that 105 switch, another gig switch off of that with 100mbit devices connected to it as well.

Where did you get the idea that having other devices connected to the switch that were not gig would cause you issues?? Either you misunderstood what you were reading - or whoever said that does not know WTF they are talking about.

Now if the switches backplane was being fully used by other traffic - then yeah you could get ****ty performance.. But the backplane of that switch is;

http://www.dlink.com/products/resource.asp...=1717&sec=0

Switch Fabric •10Gbps Forwarding Capacity

Forwarding Rate •10Mbps- 14,880pps/100Mbps- 148,809pps/1000Mbps -1,488,095pps

If your only getting 133mbps second using iperf - then you clearly have something out of wack..

Give us something to work with -- what kind of machines are you working with, what nics? What drivers.. Have you messed with any tcpip settings in the registry? ie some tweak(s) you read some where? Are you doing anything with 802.1p? The switch does support qos -- could there be other traffic that is higher priority than between these machines?

Is there other devices talking to one of the gig machines at the same time your doing your testing? Are you running P2P? for example?

When there is not suppose to be any traffic -- are you seeing your activity lights blink on your switch??

edit: Ah I see you did give the nic info, I just missed it - what specific drivers are you using??? The realtek based latest driver is 5.671.601.2007 direct from them I do believe - which is the one Im using. the ga-311 is a RTL8169 based nic as well.

edit2: for a test to take the OS out of it.. why don't you boot your 2 machines from some liveCD, ie damnsmall linux or something - an do a iperf test then.. And yeah for testing -- could not hurt to remove other connections to the switch, just so you know there is no other traffic other than between the 2 machines in test, etc. I might do this - just to verify that the numbers match up compared to xp to 2k3, etc..

Edited July 15, 2007 by BudMan

[+BudMan]

Perfect example of how the driver can make a big different in performance of the same hardware.. I just booted the 2 machines used for the numbers I posted to DSL 3.4 -- found the nics on both machines, got IPs, etc..

But the best I could do was about 600mbps..

[/tmp]# ./iperf -c 192.168.1.192

------------------------------------------------------------

Client connecting to 192.168.1.192, TCP port 5001

TCP window size: 16.0 KByte (default)

------------------------------------------------------------

[ 5] local 192.168.1.191 port 32771 connected with 192.168.1.192 port 5001

[ ID] Interval Transfer Bandwidth

[ 5] 0.0-10.0 sec 685 MBytes 575 Mbits/sec

But this is way closer to best numbers I have seen on the hardware, then what your seeing.

Tests to a crappy ole dual p2-400 running ubuntu still give numbers in the mid 300 range.. Something is not right if your only seeing 133.

[Nathanael]

I'll test all that info tomorrow - got to go now.

Driver for realtek already latest - Jumbo frame setting set to 7kB MTU.

One thing:

DGS-1005D (http://images.google.si/images?svnum=100&um=1&hl=en&client=firefox-a&rls=org.mozilla%3Aen-US%3Aofficial&q=DGS-105D&btnG=Search+Images)

is not the same as DGS-105 :-/

The product can only be found on the european site - the german one seems to have it:

http://www.dlink.de/?go=jN7uAYLx/oIJaWVUDL...M7k3qDlLUsTseLc

I've got some translating to do - my german is in it's infancy, so I'll check if there's anything there.

Seems to have jumbo frames.

God - the D-link site is a mess:

From the german version:

- jumbo frames up 9,6k

From the UK site:

10gbs switching speed

It's probably a repackaged EU version of the US DGS-105.

Edited July 15, 2007 by Nathanael