Virtual, Private Bytes and Working Set

I was reading this blog post : http://blogs.msdn.com/ricom/archive/2005/08/01/446329.aspx which says in the beginning that private bytes represent the space allocated in the swap file to eventualy hold the contents of the private memory if swapped out.

Windows memory managment has way too many concepts, and explained in a way too unclear in almost every page - official or not.

Still, you can find some good pages describing some of these concepts:
1) http://www.piclist.com/tecHREF/os/win/api/win32/sys/src/memman.htm

2) http://www.tenouk.com/WinVirtualAddressSpace.html

You should also download the windows tool called "process explorer" developed by sysinternals, if I am not mistaken. It will help you clear out, in a visual way, some of the doubts.

You want to track the memory used by one of your processes, namely you wanna check how much ram it is consuming. You can use the task manager for that. Ctrl Alt Del, task manager, search for your process, that's it.  The task manager is showing you the working set.

Officially, Microsoft says that the working set is a set of pages that have recently been accessed by your process. Unofficially, you can open the process explorer, click on a process you wish to monitor, go into to the performance tab, and you will see that the "Working Set" statistics is listed under the Physical Memory section of the performance counters.


As for the private memory of a program, I think that I am not mistaken if I say that is whatever "user code" or "user data" that your application has committed in "user virtual address space".

When your program starts up the Virtual address space is 4 GB for a 32 bits system.
Slash 2 GB out which are reserved for kernel code and buffers, and you are left with 2 GB of virtual address space.

Most of that address space the average application does not use. A fraction of those 2 GB user address space are reserved.

I believe that the Reserved memory of your application is equal to what you call: Virtual Size.
A subset of this reserved memory is actually being used by your application. And that fraction of address spaces - keep in mind it is still a subset of the reserved memory - is what you call "committed memory" or if your prefer "private memory". I believe that the commited memory does not include any memory inside the Kernel Address Space. I may be wrong here. If it is not: Committed Memory <=> Private Memory.

Now, committed memory could be anything. It is implicit from what I said be fore that committed memory is: the set address that contain application code, it is the set of addresses that contain your application buffers, it is the set of addresses that contain your dll code of your application has loaded, etc...
it is everything that you are application depends on to run (with the exception of all the addresses that belong to the kernel addres space).

So, rather often, when you check the statistics of a running process you may see a Huge virtual Size (2GB tops).
That could because, for example, a Database System knows that its buffer manager cache can have a size varying from 10 MB to 200 MB. You could reserve 200MB and commit 10. For instance.

More importantly, the Virtual Memory size of your process does not tell you an awful lot of the resources it consumes.

Then you have the private bytes of your process. Now this matters, all those address in the private bytes probably have to have entries in the Windows Page Table. If you rever to any address of your private address space, windows has got to be able to track there location. Either on swap, or if you are lucky, directly on ram.

And Finally, an amount of memory, typically smaller than your private size, constitutes your Working Memory. i believe that your working memory could - hypotethically - be bigger than your private size, assuming that your application is using lots of Kernel Code, for doing whatever.

Another Interesting thing, you can tune your Operating System and disable Windows Swap File. Which is something I like to do in any machine with more than 1 GB of ram. Because windows may swap out pages even when a lot of Ram/physical memory is free.
Even so, even if you tune your system to not use a Page File, you will see that a lot of cache misses an disk I/O continues to occur. This is because, even with a page file, a lot of resources that comprise an application can be "swaped out". You application Code, that does not change, your Program Constants, etc... Pages containing these types of data can, at any time, be removed from your physical memory (they are not written to disk - because what they are in disk is just fine).

But well, this last paragraph is just ranting.

Anyway, I don't know if i managed to Help.

Best Regards.


Edited by 99sono - 01 November 2009 at 8:58pm

Ah, thank you for the correction.

Pages sharing is not related to their disposition. In general, any page can be swapped.

Shared pages are just those, which can be used in memory sharing.

UP!