How to use more than a gig of RAM on 32-bit Linux

Nowadays, many machines are running with 2-4 gigabytes of RAM, and their owners are discovering a problem: When they run 32-bit GNU/Linux distributions, their extra RAM is not being used. Fortunately, correcting the problem is only a matter of installing or building a kernel with a few specific parameters enabled or disabled.

The problem exists because 32-bit Linux kernels are designed to access only 1GB of RAM by default. The workaround for this limitation is vaguely reminiscent of the virtual memory solution once used by DOS, with a high memory area of virtual memory being constantly mapped to physical addresses. This high memory can be enabled for up to 4GB by one kernel parameter, or up to 64GB on a Pentium Pro or higher processor with another parameter. However, since these parameters have not been needed on most machines until recently, the standard kernels in many distributions have not enabled them.

View: Full Story
News source: linux.com

Report a problem with article
Previous Story

Inventors Protest US Patent Bill

Next Story

Microsoft Tips New HD DVD Drive for Xbox 360, Retracts Tip

58 Comments

Commenting is disabled on this article.

I remember having to recompile my kernel to address more than 1GB with Slackware 9.2 but I was under the assumption that all (or most) of the current distributions came with this option enabled by default, or at least Ubuntu recognizes 4GB w/o a recompile.

Fortunately, correcting the problem is only a matter of installing or building a kernel with a few specific parameters enabled or disabled.

And their wonder why the general public prefer Windows

Fortunately, correcting the problem is only a matter of installing or building a kernel with a few specific parameters enabled or disabled.

Only a matter of... :rolleyes:

I've used Linux for many-many years now. It is a good OS for server side stuff, and is fun to tinker with. But I've found myself fighting problems that were "only a matter of..." that ended up taking countless hours to get to the bottom of.

although accessing more than 4GB of address space is impossible for a basic 32-bit CPU, There is a thing called Physical Address Extension. Both windows and linux CAN do this. In windows, your application will have to be compiled to support it as well (not sure about in linux ). HOWEVER, using more than 4GB on a 32-bit CPU is a trick since the CPU can only compute 32-bit numbers. In the end, there will be a performance hit by using more than 4GB, but if your application needs the space, its probably less of a hit than going into swap. So... Yes you can do it, but you should probably just go to 64-bit (if you can live with the lack of certain 32-bit apps in linux...)

Yes, indeed for a 32 bits computer addressing more that 4gb is possible but inefficient and limited (not by size but in use).

I checked. Their description of it seemed confusing to me. I was using the smp kernel before but for some reason after upgrading it put me back to generic. I was also hoping that if this isnt going to make a difference if someone might know why it made very little noticeable difference

Would this explain why when I upgraded from 1 to 2gb I noticed very little increase in speed and repsonsiveness even using Fluxbox?

The way to tell is posted in the linked-to article.

To check whether your kernel is configured to use all your RAM, enter the command free -m. This command gives you the total amount of unused RAM on your system, as well as the size of your swap file, in megabytes. If the total memory is 885, then no high memory is enabled on your system (the rest of the first gigabyte is reserved by the kernel for its own purposes). Similarly, if the result shows over 1 gigabyte but less than 4GB when you know you have more, then the 4GB parameter is enabled, but not the 64GB one. In either case, you will need to add a new kernel to take full advantage of your RAM.

If this is the case in your install, you need to probably install the smp kernel. Do you use Ubuntu? It should appear in your package manager.

Linux does not have this issue. For many years now Linux has supported the full 4GB addressable by 32-bits. In fact, Linux will support up to 64GB on 32-bit systems using the HighMem/LowMem swapping technique. There is no need to enable, change or install anything.

Windows, on the other hand, only supports up to 3.25GB(for 0.75 virtual) on 32-bit.

3.25GB vs 64GB -- who here has the problem?
Windows

so he just wrote the articel for fun???

p.s. it is theoretically impossible to adress 64 gig in 32 bit as zivan56 said above

In 32 bit, 2^32 = 4294967296

this is how much address space 32 bit can address, it is IMPOSSIBLE to get more with 32bit

whocares78 said,
so he just wrote the articel for fun???
No. It seems the article exists to point out some older distros may have elected the "safe" route for compatibility and used a kernel that would work on lower-end CPU systems. And some people may have mistakenly elected to accept the default kernel type, rather than selecting a "bigmem" or "smp" kernel.

I guess the errors are occurring by those who mistakenly presume that no 32-bit Linux install can address over 1GB of RAM. Doesn't stop them from posting without understanding.

As for me, I did state in my first post here that this may be useful for those who have an older distro, or have otherwise made a mistake. This isn't an "OMG!!! Teh Lunix can't access more that a GiggleBite without recompiling UR kernel!" article.

markjensen said,
No. It seems the article exists to point out some older distros may have elected the "safe" route for compatibility and used a kernel that would work on lower-end CPU systems. And some people may have mistakenly elected to accept the default kernel type, rather than selecting a "bigmem" or "smp" kernel.

I guess the errors are occurring by those who mistakenly presume that no 32-bit Linux install can address over 1GB of RAM. Doesn't stop them from posting without understanding.

As for me, I did state in my first post here that this may be useful for those who have an older distro, or have otherwise made a mistake. This isn't an "OMG!!! Teh Lunix can't access more that a GiggleBite without recompiling UR kernel!" article.

so it's really just a waste of space, that got everyone all worried about nothing

whocares78 said,
so he just wrote the articel for fun???

p.s. it is theoretically impossible to adress 64 gig in 32 bit as zivan56 said above

In 32 bit, 2^32 = 4294967296

this is how much address space 32 bit can address, it is IMPOSSIBLE to get more with 32bit


Correct.
But let's get our terminology straight here, people.
It is actually possible to address an unlimited amount of memory on any computer (practical limitations lile slowness ow swapfiles vs. RAM asside, that is) if you built the OS to handle vast amounts of virtual memory, but the actual RAM your 32-bit computer can use is limited by the 2^32 issue. The reason Linux seems to be able to see more than 4GB on some 32-bit distros is apparently because some users don't know how to tell RAM from physical memory.

DOS belongs to Microsoft, Caldera and IBM (probably a few name changes in there with Caldera, plus there were other players in the DOS arena).

Not sure what wrong branch of the conversation you are replying to, since your post seems to be a reply, but wasn't done right and ended up top-level.

markjensen said,
DOS belongs to Microsoft, Caldera and IBM (probably a few name changes in there with Caldera, plus there were other players in the DOS arena).

Not sure what wrong branch of the conversation you are replying to, since your post seems to be a reply, but wasn't done right and ended up top-level.

From TFA:

The workaround for this limitation is vaguely reminiscent of the virtual memory solution once used by DOS, with a high memory area of virtual memory being constantly mapped to physical addresses.

Parent seems a reasonable top-level response to me. No room for nostalgia around here?

Citrusleak said,
The easy option is to go 64 bit or use Windows.

The easy option is to use a Linux distribution that came out after 1996, which pretty much rules out Debian. :P

Linux doesn't exactly have the same smooth 32 bt support in their 64 bit releases as windows do. as I see it using 64 bit linux distros isn't an option untill they can offer similar solutions

"...correcting the problem is only a matter of installing or building a kernel with a few specific parameters enabled or disabled."


Yeah, a walk in the park I say! Compiling kernels is liek SOOOO easy.

Or, get this, use the current version of most distros (Ubuntu and Fedora were both specificially mentioned), then you do nothing because it is already there.

If you are using an older distro, select the "bigmem" or "smp" kernel when you install.

Or, if you do both of those wrong, do an apt-get install kernel-smp to install the correct kernel for large memory. And if you are using an old distro and didn't pick the right distro, and are afraid of the command line, open up synaptic (or other GUI package manager), and click the smp kernel and click "install".

I think you skipped over the "matter of installing" part of what you quoted. I don't know many that would rebuild or compile their kernel. I never have, and certainly wouldn't ever want to do that (in that respect, I agree with you).

markjensen said,
Or, if you do both of those wrong, do an apt-get install kernel-smp

Sadly, apt is being phased out by a bunch of distros in favour of a package called yum. Even more sadly, it'll take another 6 months for my chosen OS to switch to yum.

seamer said,
Sadly, apt is being phased out by a bunch of distros in favour of a package called yum. Even more sadly, it'll take another 6 months for my chosen OS to switch to yum.
Ummm... Say what?!?!

apt will be used by Debian as far into the future as I can see. Yum used to be the YellowDog (PPC) updater, with YellowDog derived from Red Hat. Fedora uses yum, and that is the only distro I know that uses it. Yum is nothing more than a front-end manager for RPMs. Apt is the manager for DEBs.

Not sure what you are talking about. The only distros that I can imagine would even consider yum would be other RPM-based distros (SUSE, Mandriva), not Ubuntu/Debian. Not Slack. Not Gentoo.

Angel Blue01 said,
But on my system openSUSE 10.2 detects all 2GB of my RAM. Its does feel slow though.

Remove ZMD and Beagle. That should pretty much remove the "slowness" in the system.

I wish I knew how to remove ZMD, I get lots of dependency errors when I tried. I use Beagle extensively.

At least this bug isn't present in most distros

well, this sounds like a royal pain to do.. much like so many other things in Linux. Why can't the OS detect what I have and automagically configure itself? bah!

gadean said,
well, this sounds like a royal pain to do.. much like so many other things in Linux. Why can't the OS detect what I have and automagically configure itself? bah!

Read the other comments above to find out...

gadean said,
well, this sounds like a royal pain to do.. much like so many other things in Linux. Why can't the OS detect what I have and automagically configure itself? bah!

Why doesn't Windows detect more than 3.25GB on a HP DL 380G5 and why does Linux utilitize the full 4GB on an "out of the box" SMP kernel?

Man! Couldn't agree more. Athough most things can be readily looked up and actually done by most joe blows, you simply shouldn't have to, in this days and age.

daPhoenix said,

Why doesn't Windows detect more than 3.25GB on a HP DL 380G5 and why does Linux utilitize the full 4GB on an "out of the box" SMP kernel?

Because Linux is lying to you. It still has to reserve memory space for PCI devices, just like windows has to. Sure, it can "see" 4 GB of ram, but you sure as hell can't use all 4 GB to run programs on 32-bit Linux.
Go ahead: Try an get an app under Linux to actualy address 4 GB of RAM (and NOT the not swap-file) all at once. Good luck with that. You probably won't get much further than about 3 GB, just like on Windows.

This is not a OS-specific limitation, people. It's all about the hardware. 2^32 is 2^32, all day, everyday; it doesn't matter if it's Linux or Windows that's doing the math, it's still gonna come out the same.

Croquant said,
This is not a OS-specific limitation, people. It's all about the hardware. 2^32 is 2^32, all day, everyday; it doesn't matter if it's Linux or Windows that's doing the math, it's still gonna come out the same.

Not really - you can use the PAE enabled (BigSMP in most distros) kernels - yes you'll be bound to 4GB PER application but you can use up to 64GB in the machine.

So yes, I can fully utilize 4GB _per_ application.

daPhoenix said,

Not really - you can use the PAE enabled (BigSMP in most distros) kernels - yes you'll be bound to 4GB PER application but you can use up to 64GB in the machine.

So yes, I can fully utilize 4GB _per_ application.

The memory reserved for the PCI device would still be reserved though

daPhoenix said,

Not really - you can use the PAE enabled (BigSMP in most distros) kernels - yes you'll be bound to 4GB PER application but you can use up to 64GB in the machine.

So yes, I can fully utilize 4GB _per_ application.

And how do you plan on getting the system to see more than 4GB of RAM on a 32-bit machine? PAE has nothing to do with the 4GB limit that all 32-bit machines suffer from. PAE is just there so you can use more than 4GB of total physical memory (physical memory doesn't mean RAM.... it's RAM plus virtual memory). IN other words, if you have to shift memory out of RAM but still keep it in memory, PAE lets you do it by using the swap file.

Hmmm... Red Hat had included "bigmem" kernels up to the point that they split off to Fedora (at which point the smp kernels included this support).

Some of this goes back to distros supporting all the way down to 386 or 486 CPUs, where the large memory size is just not supported on those CPUs at all.

This news items is probably a suprise to many (it was to me, but I only have 512MB RAM, so it is a non-issue). Looks like current editions of distros (Ubuntu 6.10 and Fedora 7 were specifically mentioned) have this enabled by default.

neufuse said,
wow didn't know this was a problem in linux... kinda shocking to me

It's not a problem. It's there for compatibility reasons and the person who configures the kernel in most distros apparently would rather like it to run on a 386 SX rather than use all available memory.

FYI, Windows can only use up to 4GB of "total memory," compared to 64GB on Linux with a Pentium Pro or higher...

zivan56 said,

It's not a problem. It's there for compatibility reasons and the person who configures the kernel in most distros apparently would rather like it to run on a 386 SX rather than use all available memory.

FYI, Windows can only use up to 4GB of "total memory," compared to 64GB on Linux with a Pentium Pro or higher...

Windows can use more then 4GB with PAE enabled... it's been like that since PAE was added into NT... now days though with Linux you would assume that distros would out of box allow more then 1GB and make it for the people who want to run it on older hardware use a different config... personally I think it is stuff like this that helps keep linux from going big time in the public market (the average consumer doesn't want to deal with this)

Slimy said,
Actually Vista Home Premium maxes out at 16GB while Enterprise/Business/Ultimate max out at 128GB.

From: http://msdn2.microsoft.com/en-gb/library/aa366778.aspx
Physical Memory Limits: Windows Vista

The following table specifies the limits on physical memory for Windows Vista.
Version Limit in 32-bit Windows
Windows Vista Ultimate 4 GB
Windows Vista Enterprise 4 GB
Windows Vista Business 4 GB
Windows Vista Home Premium 4 GB
Windows Vista Home Basic 4 GB
Windows Vista Starter 1 GB

You are talking about 64-bit...

Windows can use more then 4GB with PAE enabled... it's been like that since PAE was added into NT...

Only the Windows Server kernel can use more, no consumer x86 version can...

neufuse said,
wow didn't know this was a problem in linux... kinda shocking to me

Because it's not.

Every distribution under the sun ships with 4GB enabled kernels and distributions such as SuSE ship with "BigSMP" flag enabled by default which allows 36 bit registers, 64GB of memory and 128 processors.

The original post on linux.com seems awfully uninformed regarding the current state of most distributions.

zivan56 said,

From: http://msdn2.microsoft.com/en-gb/library/aa366778.aspx
Physical Memory Limits: Windows Vista

The following table specifies the limits on physical memory for Windows Vista.
Version Limit in 32-bit Windows
Windows Vista Ultimate 4 GB
Windows Vista Enterprise 4 GB
Windows Vista Business 4 GB
Windows Vista Home Premium 4 GB
Windows Vista Home Basic 4 GB
Windows Vista Starter 1 GB

You are talking about 64-bit...

And yet if you install 4gb, you still can only use 3gb of it, where as if you install 3gb you can use it all. Pretty much a false economy, stupid windows coding.

randomnut said,

And yet if you install 4gb, you still can only use 3gb of it, where as if you install 3gb you can use it all. Pretty much a false economy, stupid windows coding.

Stupid windows coding? I thought the 3GB limit was a hardware limit because of memory reserved for the pci express bus.. not windows once you went over 3GB you only got 3.4GB to the OS and the reset reserved for the hardware.... It's called PCI Express Configuration Space Additional PCI Device Memory and every OS is prone to this, MS is just reporting it correctly as the OS can not use the last part of RAM... any OS that reports it can is lieing to you, its a motherboard specification... more info on it is on Intel's website

Slimy said,
Actually Vista Home Premium maxes out at 16GB while Enterprise/Business/Ultimate max out at 128GB.

Thats the 64bit version. a 64bit processor can theoretically address around 128 Terabytes, not gigabytes.

neufuse said,
Stupid windows coding? I thought the 3GB limit was a hardware limit because of memory reserved for the pci express bus.. not windows once you went over 3GB you only got 3.4GB to the OS and the reset reserved for the hardware....

No, the limit is there because of the limits of basic direct addressing of memory. In 32 bit, 2^32 = 4294967296 is the maximum directly addressable memory, each of which addresses a byte. However, there are a number of ways to avoid this after the Pentium Pro came out. My guess is that MS purposely put the limit there on the consumer editions so that you can't take full advantage of it (say use it as a high volume server, or in intensive workstation applications), as server version do not have this issue (but use the same basic kernel). Also, applications need to be compiled to support PAE (or whatever windows calls it), so that is another issue...

randomnut said,

And yet if you install 4gb, you still can only use 3gb of it, where as if you install 3gb you can use it all. Pretty much a false economy, stupid windows coding.

Yes but then you would lose Dual Channel on a capable system...

zivan56 said,

It's not a problem. It's there for compatibility reasons and the person who configures the kernel in most distros apparently would rather like it to run on a 386 SX rather than use all available memory.

FYI, Windows can only use up to 4GB of "total memory," compared to 64GB on Linux with a Pentium Pro or higher...

FYI actually most 32 bit windows will only see 3 gig

whocares78 said,
FYI actually most 32 bit windows will only see 3 gig

No, Windows will see the 4gb. However, it will not be able to address it. Hence the reason I put "total memory" in brackets (which includes other devices' mapped memory, i.e vga)

zivan56 said,

No, the limit is there because of the limits of basic direct addressing of memory. In 32 bit, 2^32 = 4294967296 is the maximum directly addressable memory, each of which addresses a byte. However, there are a number of ways to avoid this after the Pentium Pro came out. My guess is that MS purposely put the limit there on the consumer editions so that you can't take full advantage of it (say use it as a high volume server, or in intensive workstation applications), as server version do not have this issue (but use the same basic kernel). Also, applications need to be compiled to support PAE (or whatever windows calls it), so that is another issue...


There is no way to get more than 4GB of actual RAM on 32-bit computer. More than 4 GB of memory, yes, but then you're talking about counting the Swap File as memory in addition to the RAM.

In theory, one could design a logic board that has two different memory registers on two different memory buses each with it's own memory controller and it's own CPU, with each CPU being directed by and reporting to a "traffic cop" IC, but no such hardware design exists. The reason it doesn't exist is that it's a very inelegant design: It may sound good on paper, but it's not very practical at all.

The only real way to go above 4 GB of RAM is to use a 64-bit computer with a 64-bit OS.

zivan56 said,

No, Windows will see the 4gb. However, it will not be able to address it. Hence the reason I put "total memory" in brackets (which includes other devices' mapped memory, i.e vga)

i realise that i was just trying to not confuse poeple, most people have no idea the differnece between seeing it and addressing it

Everyone's mentioning Vista in here though. What about XP Professional 32-bit? 4GB max, correct?

Just curious as I'll be be configuring a new machine soon.

Dakkaroth said,
Everyone's mentioning Vista in here though. What about XP Professional 32-bit? 4GB max, correct?

Just curious as I'll be be configuring a new machine soon.

Win2K3 + "workstation hack" FTW

Anyway, yeah XP has a 4GB limit, but you will only see ~3GB.

Croquant said,
There is no way to get more than 4GB of actual RAM on 32-bit computer. More than 4 GB of memory, yes, but then you're talking about counting the Swap File as memory in addition to the RAM.

In theory, one could design a logic board that has two different memory registers on two different memory buses each with it's own memory controller and it's own CPU, with each CPU being directed by and reporting to a "traffic cop" IC, but no such hardware design exists. The reason it doesn't exist is that it's a very inelegant design: It may sound good on paper, but it's not very practical at all.

The only real way to go above 4 GB of RAM is to use a 64-bit computer with a 64-bit OS.

Not really. A processor that supports PAE has a 36-bit address bus, which allows it to address up to 64 GB of memory.