Memory usage beyond 32Bit limitations of Windows OS

[JunkMail]

Brandon Live has a far better knowledge than me, you and all the people in the thread combined when it comes to Windows. I believe that whatever he says, is the best method, after all, who's knows Windows better than an actual Microsoft software engineer?

Microsoft didn't do it, so others had to do it (Intel guys must be idiots as they put PAE). I never knew gaining access to more memory through PAE was unsupported and called HACK? again Intel guys are idiots as they put PAE since??? how long did wiki say??? Pention PROs???

I won't take word of any software engineer unless Mark Russinovich pops here out of no where :)

[ToneKnee]

This is an issue to me because when i get my new system built i plan to use 32bit xp for a while until i can become familiar with vista/7 64bit and ensure everything is going to work correctly (i'll be doing my work on this pc) i do plan to upgrade but i need to spend more time on it make sure nothing is going to die when i need it :p

I have no issues about you using Windows XP mate :) But why not use the x64 version of Windows XP if you want to stay with XP a bit longer?

[hdood]

Microsoft didn't do it, so others had to do it (Intel guys must be idiots as they put PAE). I never knew gaining access to more memory through PAE was unsupported and called HACK? again Intel guys are idiots as they put PAE since??? how long did wiki say??? Pention PROs???

The hack isn't PAE, it's that your driver goes behind Windows' back and gets access to it in an unsupported manner. Don't get hung up on the word "hack", it doesn't have to be negative.

[ToneKnee]

Well, first of all, that is a fallacy. Working on part of Windows in no way makes you an expert on the whole OS. Second, all he said was "don't use hacks that rely on anything undocumented." Fine, I know it really annoys Microsoft that programs do (especially when they make changes to the OS which then end up introducing random instability because third-party software was doing something naughty), but this is still something that actually works, and is what Microsoft gets for not having a legitimate way for users to do what they want.

Microsoft didn't do it, so others had to do it (Intel guys must be idiots as they put PAE). I never knew gaining access to more memory through PAE was unsupported and called HACK? again Intel guys are idiots as they put PAE since??? how long did wiki say??? Pention PROs???

I don't take any software engineer unless its Mark Russinovich pops here out of no where :)

[DDStriker]

I have no issues about you using Windows XP mate :) But why not use the x64 version of Windows XP if you want to stay with XP a bit longer?

I'd like to but i've always been put down when i try to get it major compatability issues with existing software etc but recently i heard it wasn't that bad so i would like to get my hands on it and try it out....but i've also got the issue of not having an 64bit hardware available to test it on :p

i'll give it a shot when i get my new pc :D

you seem to know a fair bit about windows whats your opinion on x64 xp?

(sorry to go off topic op :p)

[ToneKnee]

I'd like to but i've always been put down when i try to get it major compatability issues with existing software etc but recently i heard it wasn't that bad so i would like to get my hands on it and try it out....but i've also got the issue of not having an 64bit hardware available to test it on :p

i'll give it a shot when i get my new pc :D

you seem to know a fair bit about windows whats your opinion on x64 xp?

(sorry to go off topic op :p)

Not too sure about XP x64 to be honest, when I first when x64, it was with Vista. The best bet is what you said and just try it and see what works for you mate. :)

[JunkMail]

I have no issues about you using Windows XP mate :) But why not use the x64 version of Windows XP if you want to stay with XP a bit longer?

I will use x64 when I feel comfortable leaving x86 mate... (and eventually, when there is plenty of softwares, not many x86 users, when microsoft completely removes x86 from OS, or when hardwares won't come pre-installed with x86 anymore) thats the time!

I used XPx64 and it was almost same as XPx86 just that it occupies more space (double system files with WoW64) and 16Bit support is OFF (can't run any DOS stuff etc, but who does anyways)

[hdood]

Why don't you feel comfortable "leaving" it? Just what is it you run that won't run on x64? It must be rather obscure. Clinging to the past is a bit silly.

[ToneKnee]

I will use x64 when I feel comfortable leaving x86 mate... (and eventually, when there is plenty of softwares, not many x86 users, when microsoft completely removes x86 from OS, or when hardwares won't come pre-installed with x86 anymore) thats the time!

That seems like now. How about you try it first before going against it? Windows Vista/7's memory management will do far more than what you hope to do.

[Brandon Live Veteran]

Well, first of all, that is a fallacy. Working on part of Windows in no way makes you an expert on the whole OS. Second, all he said was "don't use hacks that rely on anything undocumented." Fine, I know it really annoys Microsoft that programs do (especially when they make changes to the OS which then end up introducing random instability because third-party software was doing something naughty), but this is still something that actually works, and is what Microsoft gets for not having a legitimate way for users to do what they want.

It wasn't just that it's undocumented. It's that it simply doesn't work. I was told this by two kernel developers who were alarmed at this practice. On Server SKUs this may work normally. But in client SKUs the API they're calling doesn't actually work. It's not implemented, or not implemented completely. Basically, someone found an API that happens to work sometimes but isn't guaranteed to and often enough will just result in memory corruption. I don't recall the exact details, but when a kernel developer is alarmed by something and says "Holy smokes! There's no way they can do that reliably!" I tend to believe them.

[JunkMail]

That seems like now. How about you try it first before going against it? Windows Vista/7's memory management will do far more than what you hope to do.

OK, let win7 come out properly... (I'm obviously going to get it :) ) the reviews are quite cool when it comes to TaskBar but even after I install it, I will keep dual boot configuration (so when I'm completely confident, to hell with x86) hehe

But then if I'm not comfortable, its the other way around :D

[ToneKnee]

It wasn't just that it's undocumented. It's that it simply doesn't work. I was told this by two kernel developers who were alarmed at this practice. On Server SKUs this may work normally. But in client SKUs the API they're calling doesn't actually work. It's not implemented, or not implemented completely. Basically, someone found an API that happens to work sometimes but isn't guaranteed to and often enough will just result in memory corruption. I don't recall the exact details, but when a kernel developer is alarmed by something and says "Holy smokes! There's no way they can do that reliably!" I tend to believe them.

Now all we need is for JunkMail to understand why the Page File system exists and why it shouldn't be in the memory, and this thread can be locked.

[JunkMail]

It wasn't just that it's undocumented. It's that it simply doesn't work. I was told this by two kernel developers who were alarmed at this practice. On Server SKUs this may work normally. But in client SKUs the API they're calling doesn't actually work. It's not implemented, or not implemented completely. Basically, someone found an API that happens to work sometimes but isn't guaranteed to and often enough will just result in memory corruption. I don't recall the exact details, but when a kernel developer is alarmed by something and says "Holy smokes! There's no way they can do that reliably!" I tend to believe them.

Kernel developers? Is there a name Mark Russinovich anywhere? I Lu that dude :D got some emails from him too... (lucky me)

Now all we need is for JunkMail to understand why the Page File system exists and why it shouldn't be in the memory, and this thread can be locked.

Hehe at the moment, as i'm on both x86 + x64, that might not happen...

[DDStriker]

Not too sure about XP x64 to be honest, when I first when x64, it was with Vista. The best bet is what you said and just try it and see what works for you mate. :)

yeah will have to give it a shot only issue is lack of dx10 support (i have a dx10 card now but since i'm on xp its not being utilized...but as of yet i havn't seen anything from dx10 being used so i don't think i'm missing much...unless it was dx10 that could use the geometry shaders :p)

[ToneKnee]

OK, let win7 come out properly... (I'm obviously going to get it :) ) the reviews are quite cool when it comes to TaskBar but even after I install it, I will keep dual boot configuration (so when I'm completely confident, to hell with x86) hehe

But then if I'm not comfortable, its the other way around :D

Do whatever you want, but don't be afraid of x64, it's came on leap and bounds since it was first made public with Windows XP (at least the client side of things). I won't promise that you won't be disappointed, but I think you might just fall in love with it. :p

Also, don't take it to heart when Windows 7 isn't really that fast at boot times when it's first installed. Let the Superfetch learn over a period of a week or two and it's brilliant! :) As soon as I type in my password at the login, it fades straight into windows and my programs load straight away.

And don't think I don't know what you are trying to do with the page file, because this has been tempted over the years, more so in the Windows 98 days when RAMdisks were quite popular, but it's been tried and it's not a good solution. As someone has already said, it's okay if you want to store temp files into the memory like a cache or something, but the page file was designed for hard disks and for memory redundancy. You are basically not letting it do it's job! :)

[leslieboy]

i personally own 3 computers, 1 netbook and 2 desktops. so basically, i'm working with 3 different processors, an Atom(limited to 32-bit), a Dual Core Celeron(supports 64-bit) and a Core 2 Duo (64-bit). I installed Windows 7 RC x86(32-bit) on my Atom netbook and on the Celeron machine. Both have 2GB of RAM. i decided to go for the ReadyBoost solution which is imho, gave me a good boost especially when copying files.

for my core 2 duo machine, i tested installing both 32-bit and 64-bit versions of Windows 7 RC. the 32-bit advantage is that it ate less hard disk space than the 64-bit version. but since i have a large enough hard drive, the space eaten away by the 64-bit version is negligible. i also tried to have PAE enabled on the 32-bit version...the 4GB RAM is still not usable...as reported by the OS. tried the ramdisk, there's very little performance boost.

going 64-bit isn't a bad thing, whether you're a home user or not. i'm a home user and had no problems using 64-bit. all may applications which are 32-bit run without any problems. I tried running both on the core 2 duo with the 32-bit and 64-bit versions...there's hardly any difference. running 32-bit apps on a 64-bit OS had no performance impacts. if there was one...would a few seconds really matter?

i would recommend going 64-bit if you have more than 3GB of ram. it's a more logical solution since the 64-bit OS would make use of all the available ram. if you are forced to use a 32-bit OS (just like my Atom), then stick with it, but don't expect to use of all the ram in the system(given it's more than 3GB). maybe forget about getting that 4GB ram solution.

[ToneKnee]

yeah will have to give it a shot only issue is lack of dx10 support (i have a dx10 card now but since i'm on xp its not being utilized...but as of yet i havn't seen anything from dx10 being used so i don't think i'm missing much...unless it was dx10 that could use the geometry shaders :p)

Have you given Windows 7 ago yet?

[Brandon Live Veteran]

Microsoft didn't do it, so others had to do it (Intel guys must be idiots as they put PAE). I never knew gaining access to more memory through PAE was unsupported and called HACK? again Intel guys are idiots as they put PAE since??? how long did wiki say??? Pention PROs???

I won't take word of any software engineer unless Mark Russinovich pops here out of no where :)

PAE isn't magic. It requires work from the OS and device drivers. In this case you have a RAMdisk driver that's trying to control the switching of page tables in order to write to memory that the OS isn't using. Someone thought they could do this by calling an undocumented API, and it works some of the time. But the OS isn't expecting anyone to be using this API in this way, so it may switch out the page table mapping out from underneath the driver - or it may map an address table in an unexpected or unpredictable way, causing the driver to overwrite memory that it thinks is above the 4GB limit but really isn't.

Whatever the specific problem was, it wasn't a theoretical one. It was a "that is *definitely* going to go wrong" kind of thing.

[hdood]

It wasn't just that it's undocumented. It's that it simply doesn't work. I was told this by two kernel developers who were alarmed at this practice.

Fair enough. I have tried it, and it appears to work, although I didn't try it over a longer period so there could still have been corruption issues. I believe you. Regardless, I'd still give it a damn good go if I was forced to use 32-bit Windows and had hardware just sitting there unused, although I wouldn't recommend it to anyone else if it was documented to not work right.

[Brandon Live Veteran]

Now all we need is for JunkMail to understand why the Page File system exists and why it shouldn't be in the memory, and this thread can be locked.

Having the page file in physical memory makes no sense. The entire purpose of the page file is for when you need to swap pages out of physical memory. If you put your pagefile on a RAM disk, you're taking physical memory away and then giving it back in a more costly way.

Say you have 4GB of RAM and a 2GB page file on your hard drive. You load up 3GB of data. Where is it? It's in physical memory.

Now let's say you take out 2GB of that RAM and repeat. Now roughly 2GB of your data is in the physical RAM, while the rest is in the page file on the disk.

Now put back the 2GB of RAM but instead of using it normally, allocate a 2GB page file on it. Now repeat the exercise and allocate 3GB of data. What happens?

First, as the memory is allocated, the first 2GB are loaded into the free physical RAM. As soon as you go over the available physical memory, the oldest pages in the physical RAM are swapped with free pages from the page file. So you copy data from one chunk of RAM (non-RAM disk) into another chunk of RAM (the RAM disk). Now you have a free page in the available "regular" RAM, so the new data is loaded there. Then you get to the next page of data, and you repeat the process.

You probably end up with the 2nd and 3rd GB of data in the normal physical RAM. The 1st GB had been loaded there, but was then copied to the pagefile RAMdisk to make room for the 3rd GB. Then when you go to access that 1st GB of data, it gets copied back from the RAM disk memory into the normal RAM. But since there's no room, the oldest data from there needs to be swapped to the page file. So 1GB gets copied to the RAMdisk to make room for the data coming from it.

All for no damn reason.

Why anyone would consider this a good idea is beyond my imaginative capacity.

[hdood]

In all fairness, I don't think anyone was suggesting that it made sense to put the page file in memory that was already usable by the system, only memory that wasn't -- through the use of hacks like this. They're kind of different things. If you were just going to put it in regular memory, you might as well just turn the page file off and just run out of memory and lose crash dumps.

[ToneKnee]

Fair enough. I have tried it, and it appears to work, although I didn't try it over a longer period so there could still have been corruption issues. I believe you. Regardless, I'd still give it a damn good go if I was forced to use 32-bit Windows and had hardware just sitting there unused, although I wouldn't recommend it to anyone else if it was documented to not work right.

There we go! It's not something you would recommend to others and this thread has now been defeated, so I'll say my final words so I can go to bed.

There is nothing wrong with JunkMail offering the community a guide to something, I think it's great that he has contributed, but unfortunately, it's not something you want to do, and if people did it and crashed their systems when they were doing something vital (worse case issue), then people will only come looking for him.

As Brandon Live said, it's undocumented, and like others said, it's tailored more towards server environments. If you want to try it yourself, then do it, because learning how a computer works is not just by reading material, it's also about living through the experience of a system crash and looking at why it happened and so on. This feature I wouldn't recommend to any one in the community.

I hope I haven't been too harsh on you both, but I really get fired up (and so did a couple of other people) when I've had the experience and witnessed such problems happen and it's not a good experience.

Windows Vista and Windows 7 have come on leap and bounds, and like someone else said, there is a feature called 'ReadyBoost' that is quite efficient to making a computer faster. Microsoft have worked hard on addressing speed issues and they've come a long way. Their methods are more safer and efficient than the ones you are suggesting. If you won't take advice off any one else on the forum, at least take it from someone who works for Microsoft. They might be strong about their views about Microsoft (as in they don't like people saying crap about them) but it doesn't make them wrong, and if anything, they know how Windows work better than anyone on this place.

[DDStriker]

Have you given Windows 7 ago yet?

Yes i've followed it through its beta releases it installs very quickly is a nice step up from vista in terms of performance

the problem is i don't like the interface and pointless changes they've made the documents section doesn't feel as comfortable as it does in xp

(not the whole "my" thing even though it was pointless to change that since now i'm always looking for the "my" prefix and since i don't see it on vista/7 it takes me that bit longer to find what i'm after)

may not make sense but the my documents area just seems dead i use that place all the time for my programs source code and generally everything

whenever i want to backup my computer i usually only need to take My Documents + Desktop and thats all my gear

on vista/7 it seems harder to get to i also don't like the look of the taskbar i know these may seem like minor issues but the look and feel of vista/7 to me is offputting and since i'm using it all the time i have to be comfortable with what i'm using

theres just alot of little issues like the above that put me off that and how everything is laid out and how stupid some functionality is like the network interface it seems the only way i can get to a certain section is by repeating an action until windows gives me an option to try something else but i can never find that same option again directly

also setting up wireless devices on vista is a lot harder then xp i had to physically bring the router right next to the pc and shoved it in its "virtual face" saying HERE ITS RIGHT HERE CAN'T YOU SEE IT? before it could connect to it then i could take the router away and it would still be able to see it but on the initial connection it failed (connection was still strong btw around 70%~)

all i want from the new os is the background cleanup but would like the front end to remain the same but i guess as they develop more os they will cripple it in someway where some people will like the changes but most will hate it until they are forced to accept it due to some limitation (compatability; max memory etc)

as of this moment i have no reason to upgrade there is no incentive except using more then 4GB of memory and dx10 capability thats it...so its a big sacrifice for me to have to refamiliarize myself with a new platform filled with needless changes for more memory support; dx10+(till whenever they want to force another update); x64 instruction set

i'm tempted to stick with 3GB of ram more may be better but the cost just doesn't seem worth it sometimes

i would like a complete shell replacement something i could customize and maintain it the way i want and when a new OS comes out i can just use my new shell so i get the advantages of the new OS back end updates but maintain the front end i'm so familiar with some say change is a good thing but if it makes the situation worse then to hell with change :p

[hdood]

There we go! It's not something you would recommend to others and this thread has now been defeated, so I'll say my final words so I can go to bed.

With the qualifier that I had some documentation to show that it did not work reliably most of the time. Don't get too excited though, nothing of this is your doing, and the various crap and falsehoods (Windows can't run without a page file, Windows creates secret page files, Windows will expand fixed page files, Windows reuses your page file after a reboot, etc) you've come up with throughout the thread is amazing and isn't canceled out just because someone else gave me some insightful information.

If you won't take advice off any one else on the forum, at least take it from someone who works for Microsoft. They might be strong about their views about Microsoft (as in they don't like people saying crap about them) but it doesn't make them wrong, and if anything, they know how Windows work better than anyone on this place.

I see no reason to blindly accept everything a person says just because they happen to work for Microsoft (or anyone else for that matter). Why should anyone? It's all about what they say and the details they provide. Same as for anyone else.

[notuptome2004]

the new Taskbar is damn simple to use and only takes maby a day to get used to it. how hard is it for you to Pin a program to the taskbar and clik on the program to open it? also if you wan to open a new instance of it ya just click the middle of the icon with your scroll wheel now i am referring to the default view witch of course the taskbar can be reconfigured but if you want to see other instances of a program open like webpages or things of the same application ya hover over the icon to get a live preview of it witch can be clicked and as ya hover over everything else on screen goes transparent.

Now how Freaking hard is it to get used to so simple to run and use. on XP or vista with a billions things open it got to be a big mess for alot of users who have many things open daily and now with windows 7s re-designed taskbar it makes switching and finding your applications so damn easy. so microsoft cleaned up alot of the UI in windows 7 while adding tones of new functionality to windows

as for the my documents things it is easy to get it just to go start menu then Documents how hard is that ?

Before you Bash something like window s7 witch your doing in a small way you should learn to play around with it some more then 5mins and go (OH F This Shat )

now for everyone else haveing the hard time of going 64bit well install windows 7 64bit and i doubt you will find a single issue with it all your shat will work just fine on 7 unless you Refuse to get rid of a Early windows XP/ late windows 98 program cause i am sure ether is a alternative out by now or a newer versions