Jump to content



Photo

FAT32 4GB file size limit?


  • Please log in to reply
9 replies to this topic

#1 moeburn

moeburn

    tracer bullet

  • Joined: 10-March 04
  • Location: Toronto, Ontario

Posted 30 August 2013 - 14:53

So I've set up a torrent client on my WNDR3700v2 router running Gargoyle and Transmission, and I needed an external harddrive for that.  Well I had an old WD 80GB that was still working, and a powered SATA-USB enclosure for it.  I figured the best file system for it would be FAT32, because the router (Linux) can read it just fine, and if I ever need to unplug the drive and use it direct, so can a Windows computer.  So I fired up GParted (because Windows refused to format such a large drive to FAT32)

 

But I just discovered a major flaw with my plan:  Transmission won't let me add any torrents with single files greater than 4GB, because FAT32 won't let it.  And many of the torrents I'll be downloading have single files greater than 4GB.

 

So my other options are NTFS, and EXT2/3/4.  ExFat is out of the question, as it is proprietary licensed, and isn't supported by Linux.  EXT4 is the fastest option, but it takes extra effort to get it working on the Windows computers.  NTFS is the most compatible option, but it is apparently slower than EXT4.

 

My question is, which should I choose?  The oldest OS in the house that might need to access this drive is running Windows Vista.  If all I have to do is go around to all the Windows computers one time, and install EXT4 drivers on them, then that's what I'll do.  But will those drivers still work through the SATA->USB enclosure?  Or if the slowdown caused by NTFS isn't significant enough for me to ever encounter, then I'll choose NTFS.  I don't know if the bottleneck is the drive, or the USB, but it reads and writes at about 25mB/s.  But I'll never need more than 10mB/s of that, because thats roughly the limit of my 100mbit network.  So if NTFS only brings that 25mB/s down to 15mB/s, I'll never notice, and I'll stick with NTFS.

 

What would you suggest?




#2 Max Norris

Max Norris

    Neowinian Senior

  • Tech Issues Solved: 10
  • Joined: 20-February 11
  • OS: Windows, *Nix

Posted 30 August 2013 - 15:03

Personally, NTFS. EXT4 is doable under Windows but (last I checked a year or two ago) the performance was pretty bad, never mind the hassle of getting it going. You're dealing with very large files (video I'm guessing), the drive is a USB drive, that's your biggest bottleneck.. you won't notice a speed difference.. EXT4 is faster on read/write performance benchmarks (and that's synthetic, real world, at least as far as I've noticed, they're much closer in performance), but for this... you won't see it. Plus the convenience you'll get from using NTFS and having it accessible on any system is super handy,zero effort involved. That's assuming the *Nix system is up to par, if I recall FUSE and NTFS can cause some heavy CPU hits on old systems... but it's been a long time since I've done anything on a dinosaur, take that with a grain of salt.

#3 sc302

sc302

    Neowinian Senior

  • Tech Issues Solved: 21
  • Joined: 12-July 05
  • Location: NJ, USA

Posted 30 August 2013 - 15:06

NTFS.



#4 Hurricane Andrew

Hurricane Andrew

    Neowinian

  • Joined: 03-October 06
  • Location: Delaware
  • Phone: Galaxy S4

Posted 30 August 2013 - 15:06

^^^ What Max said...



#5 Dubstep Nixon

Dubstep Nixon

    Neowinian Senior

  • Tech Issues Solved: 1
  • Joined: 08-August 05
  • Location: USA

Posted 30 August 2013 - 15:09

NTFS is not slow, no reason not to use it and it's by far the best choice.



#6 Nick H.

Nick H.

    Neowinian Senior

  • Tech Issues Solved: 7
  • Joined: 28-June 04
  • Location: Switzerland

Posted 30 August 2013 - 15:11

NTFS. I seem to remember downloading something for Ubuntu so that it could understand NTFS, so that could be a solution for your router as well.

#7 OP moeburn

moeburn

    tracer bullet

  • Joined: 10-March 04
  • Location: Toronto, Ontario

Posted 30 August 2013 - 15:22

Hmm, how much do you suppose the performance hit would be on a 680mhz MIPS CPU?  Because Transmission uses about 30% of the CPU (according to the top command) when downloading at full speed, and SAMBA uses 50% of the CPU when copying a large file from the attached USB drive over the wired network at 100mbit.  Using both together at once, there's just barely enough CPU left over for the other clients to watch Netflix and such.  The last thing I need is for the NTFS file system driver to also use 30% of the CPU.



#8 Max Norris

Max Norris

    Neowinian Senior

  • Tech Issues Solved: 10
  • Joined: 20-February 11
  • OS: Windows, *Nix

Posted 30 August 2013 - 15:37

Hmm, how much do you suppose the performance hit would be on a 680mhz MIPS CPU?

I wouldn't have a guess here, the absolute slowest crappiest computer in this house (that's still in one piece) is a dual processor 800MHz P3 Proliant and I never really noticed anything dramatic out of it, but it rarely sees a non-*Nix file system either. Best bet, plug the sucker in, shut down the background stuff and copy a big file, see what's what... that said, I doubt you're going to see any difference at all if it was previously formatted with FAT32, should be about the same.

#9 cybertimber2008

cybertimber2008

    Neowinian Senior

  • Joined: 02-December 08

Posted 30 August 2013 - 16:07

Ok I'll be the one black sheep.

 

I have a Pogoplug P21 set up with Arch linux ARM, and with NTFS drives copy speeds are extremely slow because of the slower ARM processor and the overhead of NTFS. Changed to EXT4 and it was much faster (10x). I only needed samba, never needed to plug the drive into a Windows system, so that worked for me.

 

I would format it in NTFS and see what your copy speeds are like Max said. If all else fails and you have to use EXT4, rather than making it work with Windows, you could set up Samba and share the drive like you would in Windows. YMMV

 

Edit: From the Arch Linux ARM forums: http://archlinuxarm....opic.php?t=2946

 

 

've got on average 7-8 MB/sec copy speed of large files with TeraCopy over Samba on Gigabit Ethernet to a DockStar connected USB 2.0 drive with the Plug CPU load as per Htop at 97-100% mostly to handle NTFS writes to the disk. Compare to 28-30 MB/sec copy to EXT4 with 82-97 CPU load, and 29-30 MB/sec copy to XFS with 88-92 CPU load, again mostly to handle writes to disk.


#10 OP moeburn

moeburn

    tracer bullet

  • Joined: 10-March 04
  • Location: Toronto, Ontario

Posted 30 August 2013 - 18:36

Ok I'll be the one black sheep.

 

I have a Pogoplug P21 set up with Arch linux ARM, and with NTFS drives copy speeds are extremely slow because of the slower ARM processor and the overhead of NTFS. Changed to EXT4 and it was much faster (10x). I only needed samba, never needed to plug the drive into a Windows system, so that worked for me.

 

I would format it in NTFS and see what your copy speeds are like Max said. If all else fails and you have to use EXT4, rather than making it work with Windows, you could set up Samba and share the drive like you would in Windows. YMMV

 

Edit: From the Arch Linux ARM forums: http://archlinuxarm....opic.php?t=2946

 

Alright, looks like I'll be choosing EXT4.  This drive is shared via Samba over the network using the router.  But it would have given me a nice sense of security to know that, if for some reason the router fails/reboots/is being worked on, I can just unplug the drive and plug it into a TV/computer, and my friends/family can watch the movies downloaded onto it, without having to wait for the router to reboot/get fixed by me.

 

But that isn't likely to happen very often.  And I have another Linux computer that I can use to temporarily share the drive via Samba, should the router be temporarily out of service for a few hours.





Click here to login or here to register to remove this ad, it's free!