[FAQ] Linux File System Overview

[markwolfe Veteran]

"This Linux file system is crazy..."

This FAQ is a quick overview that I have written to summarize how the files are generally organized into their directories. I have often described this, in the past, as looking at a vertical vs. horizontal method of determining where different files should be placed. :blink: However, that doesn't really clear things up for 99% of the people I have tried to describe this to, so I will attempt to better explain this here. I will cover the file & directory organization in very broad brush strokes. For more detailed information, there are thousands of in-depth resources to be found with a brief google. Most Linux books cover this as well (but are often skipped as they aren't readily understood by most people who haven't worked with Linux for a while).

Half Full Glasses and Photos

The first thing (or one of the first things, anyhow) that a Windows user who has moved to Linux (or any Unix-like OS) must shed is the belief of How Things Should Be. Just because one OS does things a certain way, doesn't mean that any other way is 'wrong'. Like whether you consider a glass to be half-empty or half-full, it is a matter of perspective.

Consider this to be like your digital family photos on your computer. You may keep your files organized by year (2003, 2004, 2005, etc.) and keep the sets of photos belonging to the separate events (Michael's Birthday or Joshua's Football Game) in their own folders inside each year. It is a convenient way to see all the events of each year, plainly organized in front of you when you open that 2004 folder.

But, that isn't the only way that photos can be organized. If you want to go to all of the "Summer Vacation" photos and reminisce about the all family vacation fun you have had, you will find them scattered into all the various year folders. This brings up another way photos can be organized: by subject. This can be done on your family photos, so that all the Birthdays that Timothy has had through the years can be found in their own specific folder. Modeling agencies would almost have to sort by model name, not by year of photo.

Windows and Linux Directories

What that talk about photos have to do with Linux file organization? Quite a bit, actually.

Windows organizes installed applications by their own folder in "My Programs". Each app creates a folder for everything to go into. This may sound extremely neat, but Windows really isn't as neatly organized as the that made it sound. It scatters .dll files around to different folders and modifies the registry, too. Try copying an app by moving the "Program Files" folder to a new computer, and it won't function.

Linux, on the other hand, puts all "executables" (or "binaries") into a bin/ directory. You won't find the man pages (help files) in there, as those are placed in the man/ directory. Any source code you have on your system will be in a src/ directory. It's a different way of seeing how a file system should be organized, and quite often it seems counter-intuitive to a Windows user.

Ok, Why so Many?

While the concept of bin/ may be easy enough to understand, Windows users will next run into the many different bin/ directories (and sbin/, for system binaries that are usually reserved for users with elevated privileges). Often this will be the next big stumbling block. After all, it just seemed to make sense - then they find that there is no "one place" for these binaries.

This is understood when you really accept that Unix has been a multi-user system for quite a long time (longer than Windows has been around!), and its filesystem reflects this. Different conventions exist, and the following is not a fact for all systems, but is meant to illustrate by example:

• The basic OS installs what it needs in /bin/. Everyone needs to have access to these basic functions.
  
• The distro will often install and additional apps it uses into /usr/bin/. Again, all users may have access to this.
  
• Add-ons may be installed into /usr/local/bin/, as these files are local to the machine you are working on, and may not exist on another PC, even if installed with the same version.
  
• Users without write/execute permissions outside their home directory may use /home/username/bin/, so they can have their own executables that are unique to them (perhaps they are learning C and making test programs).
  

To see what your particular system is set to look for any binaries you ask it to run, perform an

echo $PATH

to see what directories it uses.

File Extensions...

This is another piece of baggage from the days of DOS, and even Windows attaches great significance to a file extension. In particular, the whole .exe thing is thrown away in Linux. Not only a compiled binary file, but any file may be made "executable" by setting the proper file attribute (either via chmod at the command line, or by mouse click on Properties using the more advanced file managers available with Gnome or KDE). A text file can be made executable, which is then interpreted like a DOS .bat "batch file".

Hidden Files

Files in Unix don't have a "hidden" attribute. A file (or directory) is considered hidden when you start the filename with a dot. For example, the file .bash_history (which contains the last several hundred commands you have issued at the bash prompt) in your home folder is not shown by a plain ls command. If you wish to see the hidden files, you can use ls -a to display all files.

Almost Everything is a File!

Or, if not a file, then it is represented in the Linux file system. Your devices are all in the /dev/ folder. Various system and kernel information is represented (and can be set!) in the /proc/ directory. An interesting task is to do a cat /proc/cpuinfo and look what Linux tells you about your CPU.

The Forward Slash: The 'root' of all Goodness

Unlike what a DOS/Windows user is used to, drives (or, more accurately, partitions are not set up each as separate "tops" of the file tree. There is no C:\filename.ext with no file system above. All partitions that are mounted are assigned to a folder (typically in the /mnt/ directory). Unix and all variants use the forward slash '/' as the starting point for all file references, regardless of which drive, network attachment, or device you are referring to. There is no need to care whether /home/, /boot/, /mnt/cdrom or /mnt/windows_c are on your "Linux drive", or if they are elsewhere. Even devices such as scanners are found in the file tree. It can be a strange concept when you first migrate to Linux, but it makes sense when you become used to it.

More awaits you...

Alas, I will end this here. This little FAQ just touched on the filesystem that Linux (and other Unix variants) use to keep things organized. It may seem totally foreign at first, but once you get used to thinking the *nix way, things do get easier.

In closing, I would like to invite anyone else that wants to add to this with some additional detail or information that I did not cover, please post this information as an "addendum" to this first post. Or, if you feel particularly energetic, write your own short HOWTO or FAQ for the Linux section.

As always, those of you who are reading this and notice an error in what I have said, please let me know so that it can be corrected.

EDIT: I forgot to include a couple of links to some more detailed references. :blush:

http://www.comptechdoc.org/os/linux/comman...x_crfilest.html

and

http://web.archive.org/web/20050310142547/...p?articles_id=6

may be of interest...

Also: Filesystem Hierarchy Standard:

http://www.pathname.com/fhs/

Edited June 27, 2007 by markjensen

[Hurmoth]

Awesome... thanks for the info (Y) Not a big Linux person, yet, so this really helps!

Edited December 22, 2004 by Hurmoth

[Rix]

Wow, thanks for that info :D

[rezza Veteran]

/bin : system-wide binaries essential to the running of the system

/boot : often separated from the root filesystem (ie on another partition) and contains the actual kernel as well as bootloader config files

/dev : a directory full of devices, populated either by the "devfs" filesystem, or more recently "udev". Really old systems had to have this directory filled with device nodes manually.

/etc : contains configuration files for the system, as well as stuff which doesn't fit into any other directory

/home : contains users' home directories

/lib : system-wide libraries essential to the running of the system

/lost + found : the directory to find things after your drive has been fscked (fsck = file system check)

/media : removable drives (such as CDs, DVDs, floppies) should be mounted to a sub-directory of this dir

/mnt : static filesystems (such as windows drives, etc) should be mounted to a sub-directory of this dir

/opt : used for add-on apps and some other software packages

/proc : a virtual filesystem which contains information about your system

/root : the root user's home directory

/sbin : system binaries, where things that are essential to system maintenance but not to normal users go

/sys : another virtual filesystem, which gives information on the running kernel, and can be used to pass options to the kernel during operation

/tmp : system-wide temporary files

/usr : this directory normally contains a whole new filesystem heirarchy, with its own bin, include, lib, and sbin subdirectories. This is where most installed software should put its binaries and libraries, as well as default option files (normally in /usr/share/<appname>). The /usr directory helps to keep files essential to the running of the system/OS separate from user-installed applications.

/var : contains variable data files such as mailboxes, news directories, spooling files, and logs.

[enklus]

Thank you very much! (Y)

[macssuck]

File Extensions...

This is another piece of baggage from the days of DOS, and even Windows attaches great significance to a file extension.? In particular, the whole .exe thing is thrown away in Linux.? Not only a compiled binary file, but any file may be made "executable" by setting the proper file attribute (either via chmod at the command line, or by mouse click on Properties using the more advanced file managers available with Gnome or KDE).?  A text file can be made executable, which is then interpreted like a DOS .bat "batch file".

The primary reason file extensions are considered a bad idea are because they contain important meta-data about a file (what it is and to a lesser extent what to do with it) in a place that's easy to edit/change or corrupt. Why should it matter what your files are named when you have a billions of CPU cycles that could be used to figure out what they are and what to do with them? UNIX-like operating systems follow this mantra as do other operating systems like BeOS and Mac OSDiscovering Specific File-type Information Without Extensions Using a CLI/b>

Tfile/i>[1] is used for finding content-type information about a file. The commafile somefile.xyz/i> will look at somefile.xyz and determine what kind of information is stored inside by performing a series of tests. The exact nature of the tests and the file-type information is controlled by tmagic/i>[2] file is typically found /usr/share/file/magic/i>. It is possible to add tests and associated file types to the magic file but there is little need because most distributions ship with a fairly comprehensive magic file.

An example of the file command's out put is:

$ file /bin/ls

/bin/ls: ELF 32-bit LSB executable, Intel 80386, version 1, for OpenBSD, dynamically linked (uses shared libs), stripped

$ file /usr/share/pf/faq-example1

/usr/share/pf/faq-example1: ASCII text

$ file /sys

sys: broken symbolic link to `usr/src/sys'

$ file /dev/tty01

/dev/tty01: character special (8/1)

[/code[b]Generic File Information With the ls Command[/b]/b>

If you want more generic information about a file like whether it is a directory, block device, ascii file, link, socket, etc. then you can use t[i]ls[/i]/i>[3] command along with either t[i]-G[/i]/i> [i]--color[/i]/i> arguments (variable depending on your OS).BSD tends to use -G to colors output, Linux tends to respond to --color or -G, and some *NIX operating systems won't color output at all so you will need to install GNU/ls from the GNU File Utilities[4] to get color output[i]ls -G[/i]/i> [i]ls --color[/i]/i> will show directories in a dark blue, named pipes in yellow, symbolic links in purple, executable files in green, etc.

If you have trouble seeing color or you are using a terminal device that does not support ANSI colors then you can use t[i]-F[/i]/i> argument wi[i]ls[/i]/i> to display special characters after the file names instead of or in addition to color information. symbolic links have an "@" character after their name, executables are marked with a "*", and directories with a "/", etc. More information can be found in t[i]man page for ls[/i]/i>[3].

The colors used for differentiating files are controlled by the LS_COLORS environmental variable which can be configured using t[i]dircolors[/i]/i>[5] command. The specifics of colorizi[i]ls[/i]/i> output are worth of their own FAQ.

It is also perfectly acceptable to use file extensions if you want to. You are free to continue naming your JPEG images with .jpg extensions, doing so may save you a lot of hassle if you ever need to access those files on Windows (as an example). *NIX doesn't force you to use file extensions a certain way, but it doesn't penalize you either.

--Edit 1--

[1] Die.net: file(1): Determine File Type Information - Linux Man Page

[2] Die.net: magic(5): File Commands Numbers File - Linux Man Page

[3] Huntsville Macintosh User Group: Mac OS X / Darwin Man Pages

[4] GNU Operating System - Free Software Foundation: Fileutils

[5] Rt.com: Unix Man Pages: dircolors(1)

Edited December 24, 2004 by macssuck

[anir]

Nice overview, but I thought this topic would be about file systems like ext2, ext3, XFS, ReiserFS, etc...

And there are no "folders" in a file system, only directories.

Edited December 24, 2004 by anir

[Knight']

Nice overview, but I thought this topic would be about file systems like ext2, ext3, XFS, ReiserFS, etc...

And there are no "folders" in a file system, only directories.

585159776[/snapback]

That's interesting, what's the difference between a Folder and a Directory?

[+John Teacake MVC]

And there are no "folders" in a file system, only directories.

585159776[/snapback]

Arent they the same thing :huh:

[markwolfe Veteran]

Arent they the same thing  :huh:

585159980[/snapback]

Normally, in the *nix world they are referred to properly as 'directories'. A 'folder' is a GUI representation that sort of symbolizes the concept visually.

[tiagosilva29]

Very useful guide, thanks (Y) :D

[SaguratuS]

(Y)(Y)(Y)

Another good one Mark :)

Perhaps adding the differences between FS types might be beneficial? I see a lot of users pick ext3 by default since they are unaware of the differences. But I guess I'm just a die-hard gentoo / reiserfs fan

[markwolfe Veteran]

(Y)(Y)(Y)

Another good one Mark :)

Perhaps adding the differences between FS types might be beneficial? I see a lot of users pick ext3 by default since they are unaware of the differences.  But I guess I'm just a die-hard gentoo / reiserfs fan

585169257[/snapback]

Thanks! But I don't know much about the different filesystem types, except what I have read. Maybe you could write something up, and explain the differences (and advantages/disadvantages)?

[Rablet]

Thanks for a very good guide!

[cork1958]

Whoa! Awesome!! Double thanks for the info. Tracking this topic for future reference while trying to figure out this OS.

[blik]

(Y) Nice one, helped me understand alot more about how things are organised :)

[amcgeetb]

Thank you, I've been weaning myself from m$ to Linux for a wee while now and this is probably one of the most helpful threads. Informative and very pertinent. Thank you.

[SaguratuS]

Thanks!  But I don't know much about the different filesystem types, except what I have read.  Maybe you could write something up, and explain the differences (and advantages/disadvantages)?

585170191[/snapback]

Haha, I'll try, I know you can fill in the rest :happy:

I'll make this as simplistic as possible, and I know there are many more, but these would be the most common choices.

Ext2: Commonly used for /boot to keep everything nice and simple, but is by no means the way to go for your main filesystem. This FS does NOT have journaling, meaning if you crash, you can very easily lose data.

Pros:

Clean, safe to use for /boot

Well rounded when it comes to speed

Wide range of support, easily readable and writable from within windows

Cons:

Non-Journaling -- Data Loss is probable in a crash

Ext3: The typical choice most users make, however the ReiserFS has surpassed this FS. Ext3 is well rounded for speed and reliability.

Pros:

Very well rounded FS

Wide range of support, easily readable from within windows

Cons:

Potential problems when handling large numbers of files

Inferior to ReiserFS

Reiser3: Most likely the best choice for users, this FS is extremely fast, very reliable, and can handle the extremes in a filesystem very well due to the BTree algorithm it uses. From massive 100+ gig files to 1+ million files in a directory, reiser can take it, and handle it very efficiently.

Pros:

Extremely fast and reliable in almost any situation

Stable / Bug Free, code is no longer altered unless mandatory

More space efficient than Ext3

Does not fragment over time

Cons:

Limited readability from within windows

Reiser4: The new and improved Reiser3, however support is not readily available in most distros, and not yet widely supported for 3rd party programs. You will NOT be able to read this FS in windows.

Pros:

Far exceeds Reiser3 in nearly every aspect

Cons:

Lack of support - it's relatively new, you typically won't find distros / kernels supporting it without patches

Actively developed, bugs may show up

NO readability from within windows

XFS: This FS is very efficient when it comes to handling very large files, however it lacks greatly in other areas. I'd recommend staying away from this FS unless you have a special use with large files.

Pros:

Efficient handling of large files

Cons:

Normally a "specialized" application filesystem

Performance drops like a brick when it comes to handling multiple small files

Readability in windows (?)

JFS: I'm not really going to go into this one, I'd recommend staying away from it altogether. It's similar to Ext3, but lacks when it comes to performance.

There's a quick overview, I **KNOW** things may be missing and may also be inaccurate, please correct me on anything you find. I'm unable to bring up the FS lists at the moment, so this is entirely off the top of my head. I'll try and fix / add whatever I can tomorrow, when I can bring up my spec sheet for the filesystems.

[rezza Veteran]

Haha, I'll try, I know you can fill in the rest :happy:

585171299[/snapback]

OK, I agree with most of your points here, but I'd like to jump in with another viewpoint:

ReiserFS - Yes, its good. Yes, its better than ext3. Yes, its possible to read from within windows. Yes, most people should probably be using it. But no, most people won't notice a performance difference between Ext3 and ReiserFS. Ext3 is still journalled, so crashes and hard reboots etc won't screw up your filesystem, and Ext3 is an absolutely rock solid filesystem, and has been for a long time - longer than ReiserFS.

Reiser4 - Reiser4 is the newest version of the Reiser filesystem, and aims to improve on ReiserFS (which is what all previous versions of the Reiser filesystem were known as) in every area. A lot of money is being spent on developing Reiser4 by some large corporations, and Hans Reiser seems to have excelled himself. This filesystem seems to be ahead of every other filesystem available today, for any platform whatsoever. Unfortunately, this includes crashes and instability - this filesystem is so new that it hasn't been accepted as stable enough to be included in the kernel by Linus and Andrew yet. Having said that, I know many people who happily use it, and despite some teething issues, it is useable on a day-to-day basis, as long as you're not averse to having to search around for some solutions to small issues.

[Elmo]

awesome thread thanks (Y)

[Hurmoth]

Reiser4 - Reiser4 is the newest version of the Reiser filesystem, and aims to improve on ReiserFS (which is what all previous versions of the Reiser filesystem were known as) in every area. A lot of money is being spent on developing Reiser4 by some large corporations, and Hans Reiser seems to have excelled himself. This filesystem seems to be ahead of every other filesystem available today, for any platform whatsoever. Unfortunately, this includes crashes and instability - this filesystem is so new that it hasn't been accepted as stable enough to be included in the kernel by Linus and Andrew yet. Having said that, I know many people who happily use it, and despite some teething issues, it is useable on a day-to-day basis, as long as you're not averse to having to search around for some solutions to small issues.

585183279[/snapback]

I can't wait until Reiser4 is stable enough to become mainstream...

How come Red Hat (Fedora) doesn't use ReiserFS?

Edited January 3, 2005 by Hurmoth

[markwolfe Veteran]

How come Red Hat (Fedora) doesn't use ReiserFS?

585218698[/snapback]

I was googling to see if I could find out why Fedora didn't include ReiserFS, when I found this:

http://www.fedorafaq.org/#reiserjfs

Apparently, even though Fedora doesn't officially support Reiser, they include it, and all you have to do is type linux reiserfs at the initial install prompt. :o

[Hurmoth]

I was googling to see if I could find out why Fedora didn't include ReiserFS, when I found this:

http://www.fedorafaq.org/#reiserjfs

Apparently, even though Fedora doesn't officially support Reiser, they include it, and all you have to do is type linux reiserfs at the initial install prompt. :o

585219152[/snapback]

Well, damn, I just upgraded to FC3... well whenever FC4 comes out, I'll have to do this. Thanks for the info :D

[gflores]

Two questions from a Linux noob.

Installing programs: Where should I install my programs? I installed Firefox in my home directory because that was the default place in the installer. Should I have pointed it to /usr directory? Where exactly? Also, when I install tar.gz files, like for XMMS, where should I put that directory? I don't really want it in my home directory. Also, how do I know where RPM files are installed?

File System: I'm not sure what file system I installed for my Linux distribution (Mandrake 10.1). Is there a way to check? Also, what file systems in Linux can be seen from Windows, because that will be a major plus. On a related note, is there a way that Linux can view my Windows partitions. It'd be cool if I could listen to my music in Linux. They're currently in NTFS.

Thanks for any help. Man, I have so many questions... but I'm learning :)

[markwolfe Veteran]

Installing programs: Where should I install my programs? I installed Firefox in my home directory because that was the default place in the installer. Should I have pointed it to /usr directory? Where exactly? Also, when I install tar.gz files, like for XMMS, where should I put that directory? I don't really want it in my home directory.

585231579[/snapback]

Depends mostly on which users will need access to the program, and also on how organized you want your directories.

If you are the only user that will be using that program, and you know you aren't going to have other users using it, then your home is ok. A better place for a system-wide binary would probably be /usr/local/bin. Ultimately, it doesn't really matter too much for a home user (at least I have seen no reason it would). If you have an automatic backup script, you will want all your manually added apps to be in a controlled number of locations, to make backup easier.

Also, how do I know where RPM files are installed?

585231579[/snapback]

The different parts go to their correct parts (i.e. man pages to the man/ directory and executables to the bin/ directory). If you want to find out where your executable went (continuing from the first part I quoted), you can do a which firefox to find out. In my case, the result of that yields /usr/bin/firefox, so I know the executable is in /usr/bin/.

File System: I'm not sure what file system I installed for my Linux distribution (Mandrake 10.1). Is there a way to check? Also, what file systems in Linux can be seen from Windows, because that will be a major plus. On a related note, is there a way that Linux can view my Windows partitions. They're currently in NTFS.

585231579[/snapback]

Your answers here are:

• Use the mount command to find out what is currrently mounted (accessable).
  
• I believe that ext2/3 can be read from Windows with a third-party app for Windows...
  
• Yes. I think Mandrake includes this ability by default. DiskDrake or something like that may help. (it may also tell you what filesystem format your drives use, like the mount command does)
  

Thanks for any help.  Man, I have so many questions... but I'm learning :)

585231579[/snapback]

LOL That was a bit more than the "two questions" you initially stated you had. :shifty: But we are pretty friendly folk here, and are happy to help. :yes: