Subj : Re: virtual addresses To : comp.programming From : Randy Howard Date : Wed Sep 28 2005 06:42 am Brian wrote (in article ): >>> Why haven't we outgrown page files / swap files? It was an early >>> trick and kludge was it not? > >> Because applications constantly want more memory for themselves >> than is available. Plus, as I indicated above, there are some >> very good reasons for the existence of virtual addressing, /even >> if/ you have sufficient physical memory. > > Separating address space and paging are key here. Are we talking > apples and apples? I am not saying that you should be paging. I am saying that you should have a system /capable/ of paging should the need arise. > I agree that a flat address space per process is a key feature > that is very much needed. Page tables are needed. I don't believe > cycling pages to disk is strictly necessary. Unless you have a shortage of memory for one or more applications at a given time. >>> My first XP install was on a PIII with 256MB of memory. I suppose >>> the page file was an additional 256MB. The computer had a practical >>> upper memory limit of 512MB. > >> All you had to do was use a larger swap file. In fact, if you >> left XP alone to manage it's own virtual memory, it would have. > > Generally the swap/pf is the same size as > the physical memory. This is a rule of thumb on several OSes. It varies, and the exact default ratios chosen by each don't really matter to this discussion. The key thing is, are you allowed to use more memory total across all processes than the physical memory in the machine or not. Also, if you need more than those values, you are free to increase the size of the swap file(s) or partition(s) up to very large values if needed. >>> I have the same version of XP running on a P4 with 2GB of memory >>> today. 2GB is approximately 4x my original system's TOTAL available >>> memory including it's page file. > >> Irrelevant. If you try to load up applications totalling more >> than 2GB of memory consumption (including OS), then you still >> need a swap file, or you run the risk of crash-o-rama (ahem). > > This is ignoring the entire fact that memory limitations vary > between 128MB and 4GB for the same exact Windows 32-bit system. I don't follow what "entire fact" I am supposed to be ignoring here. Some 32-bit Windows server systems support much more than 4GB of physical memory, but you must turn on /PAE in boot.ini to get there, and no single process can use more than 2GB total when in that mode. There is also an options to have a max of 3GB address space per process, but IIRC, that precludes you from also using /PAE. If for some reason that doesn't work for you, feel free to install Win64 instead. > Certainly the practical virtual memory space of the first system > would fit conveniently into the second /without/ paging. What is that supposed to mean? > I'd suggest most users would be hard pressed to tell the difference > between a 512MB WinXP system and a 2GB WinXP system. So again, > plenty of memory. It all depends upon what you are doing. If you open up 40 20MB ..tiff files in photoshop, you'll be less than happy with 512MB of RAM. Your "plenty" is somebody else's "not enough". The OS is designed to run for a lot of different user models, not just one. If you want no page file, you can turn it off. After a few months, you should have a good idea of whether or not you really needed it or not, because the presence of absence of crashes will be an indicator. > Java developers frequently avoid filling system memory > for fears of garbage collection pauses. Ok. Java developers aren't particularly interesting to me, so I won't argue either way. > In fact, I sat in a > seminar recently where the system admins learned to their chagrin > that the extra 14GB in their BEA servers was to go unused. They could always sell it on ebay, or revel in the knowledge that buffer cache won't go hungry, ever. :-) Linux is much more aggressive about using extra RAM for file system cache, but they both do it. If you have enough disk I/O in flight for enough hours in a row (at a rate in excess of physical disk throughput), top will actually show you almost out of memory, as it continues to buffer more and more until you stop it, after which it will eventually get freed. > The 2GB's tuned for BEA WebLogic was about enough to prevent > pauses. So the extra memory is now a space heater. The OS designer is not responsible for an overzealous purchasing department with no clue about what they really need. >>> In other words, I have plenty o' memory without the disk. So why >>> is the page file still there? > >> Because you don't have enough memory. You *think* you have >> enough, you can't predict how much memory you will need. On >> some systems, 16GB of RAM is insufficient. > > I offer that I use XP with the same OS version, "nearly" the same > software, the same OO paradigm that's been around for the past 10 > years or more. Microsoft's stuff has been Win32 and MFC (wrapper) > for ages, hasn't it? For a single instance, you may. If you can guarantee that you'll never run more applications in parallel than you ever have in the past, that you have never needed to swap, that you'll never install a new application or update one that requires more memory, never open a 2500 page word document and run spell-check on it, etc., etc. then go for it. Turn off the paging and relax. You're the exception, not the rule. > Same libraries. Same OS. How does this add up to bigger apps > exactly? Office still fits on the same ISO cd it did in 1995. Irrelevant. Binary size != memory consumption, even if it did matter. Try opening 500 word documents of a few MB in size each all at once. Good luck. No, that's not normal use, but I've seen people accidentally launch hundreds of files in explorer by accident just clicking the wrong button or hitting at the wrong time. > But I'm seeing 2GB being used quite commonly. Memory has actually > grown faster than Moore's law. My memory stays constant in size most of the time, unless I buy more of it. :-) I have worked on 64GB servers quite a bit, and you have to work to use it all, but it can be done, from the command line, with a surprisingly short C program. Real application developers will start sucking down memory in even bigger gulps once the 64-bit transition is farther along, you can count on it. > Exponential memory growth. Constant or linear application > growth. You can see why I don't buy the argument. Buy what argument? Make a convincing argument that operating systems shouldn't have virtual memory if that's your position. So far you haven't done so. The debate isn't whether or not your personal computer has enough RAM or not. >> Either way, virtual memory provides some advantages even in >> cases where you have more physical memory than your peak usage >> (which you can't ever guarantee). > > So your saying that it's always better to map virtual memory > to magnetic media versus system memory? I can't see how that > would ever be true. I can't see how you came to that conclusion. It's in fact the last thing I would have expected you to take away from the discussion. > The entire question boils down to this. What type of memory > is faster? Is there enough of it? We definitely are concerned about two different questions, because I don't think that's even a question worth asking. The answer has been back on that one for decades. > And I think that, perhaps on a tunable level, system memory is > plentiful enough to do away with the hard disk. Hardly. > The problem is, > whenever the swap file is eliminated, performance does indeed > suffer. Source? > The only reasonable explanation is that the virtual memory > manager degrades at the boundary case - similar to zip compression > creating a bigger archive file than the raw input. I don't follow that. If performance /does indeed suffer/ when the swap file is eliminated, why is an explanation for your hypothesis that the vmm is having problems? > That's the part that hasn't been explained to me. I'll dive into > Linux or OpenSolaris one day and figure it out -- assuming swap > files aren't eliminated in the meantime. You do that. -- Randy Howard (2reply remove FOOBAR) .