Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Look up keyword
Like this
7Activity
0 of .
Results for:
No results containing your search query
P. 1
Virtual Memory Constraints in 32bit Windows

Virtual Memory Constraints in 32bit Windows

Ratings: (0)|Views: 500|Likes:
Published by djkaushal

More info:

Published by: djkaushal on Dec 11, 2009
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

12/20/2012

pdf

text

original

 
Virtual memory constraints in 32-bit Windows
Mark B. Friedman
Demand Technology1020 Eighth Avenue South, Suite 6Naples, FL USA 34102markf@demandtech.com
Abstract.
 Many server workloads can exhaust the 32-bit virtual address space in the Windows server operating systems. Machinesconfigured with 2 GB or more of RAM installed are particularly vulnerable to this condition. This paper discusses thesigns that indicate a machine is suffering from a virtual memory constraint. It also discusses options to keep this fromhappening, including (1) changing the way 32-bit virtual address spaces are partitioned into private and shared ranges,(2) settings that govern the size of system memory pools, (3) hardware that supports 36-bit addressing. Ultimately,running Windows on 64-bit processors is the safest and surest way to relieve the virtual memory constraints associated with 32-bit Windows.
Introduction.
The Microsoft Windows Server 2003 operating systemcreates a separate and independent virtual address spacefor each individual process that is launched. On 32-bitprocessors, each process virtual address space can be aslarge as 4 GB. (On 64-bit processors process virtualaddress spaces can be as large as 16 TB in WindowsServer 2003.) There are many server workloads that canexhaust the 32-bit virtual address space associated withcurrent versions of Windows Server 2003. Machinesconfigured with 2 GB or more of RAM installed appear tobe particularly vulnerable to these virtual memory con-straints. When Windows Server 2003 workloads exhausttheir 32-bit virtual address space, the consequences areusually catastrophic. This paper discusses the signs andsymptoms that indicate there is a serious virtual memoryconstraint.This paper also discusses the features and options thatsystem administrators can employ to forestall runningshort of virtual memory. The Windows Server 2003operating system offers many forms of relief for virtualmemory constraints that arise on 32-bit machines. Theseinclude (1) options to change the manner in which 32-bitprocess virtual address spaces are partitioned into privateaddresses and shared system addresses, (2) settings thatgovern the size of key system memory pools, and (3)hardware options that permit 36-bit addressing. Byselecting the right options, system administrators canavoid many situations where virtual memory constraintsimpact system availability and performance. Nevertheless,these virtual memory addressing constraints arise inevita-bly as the size of RAM on these servers grows. The mosteffective way to deal with these constraints in the long runis to move to processors that can access 64-bit virtualaddresses, running the 64-bit version Windows Server2003.
Virtual addressing.
Virtual memory is a feature sup-ported by most advanced processors. Hardware support forvirtual memory includes a hardware mechanism to map fromlogical (i.e., virtual) memory addresses that applicationprograms reference to physical (or real) memory hardwareaddresses. When an executable program's image file is firstloaded into memory, the logical memory address range of theapplication is divided into fixed size chunks called pages.These logical pages are then mapped to similar-sized physicalpages that are resident in real memory. This mapping isdynamic so that logical addresses that are frequently refer-enced tend to reside in physical memory, while infrequentlyreferenced pages are relegated to paging files on secondarydisk storage. The active subset of virtual memory pagesassociated with a single process address space that arecurrently resident in RAM is known as the process's workingset because those are the active pages the program referencesas it executes.Most performance-oriented treatises on virtual memorysystems concern the problems that can arise when realmemory is over-committed. Virtual memory systems tend towork well because executing programs seldom require alltheir pages to be resident in physical memory concurrently inorder to run. With virtual memory, only the active pagesassociated with a program's current working set remainresident in real memory. On the other hand, virtual memorysystems can run very poorly when the working sets of activeprocesses greatly exceed the amount of RAM that thecomputer contains. A real memory constraint is then trans-formed into an I/O bottleneck due to excessive amounts of activity to the paging disk (or disks).Performance and capacity problems associated withvirtual memory architectural constraints tend to receive farless scrutiny. These problems arise out of a hardwarelimitation, namely, the number of bits associated with amemory address. The number of bits associated with ahardware memory address determines the memory addressing
 
range. In the case of the 32-bit Intel-compatible processorsthat run Microsoft Windows, address registers are 32 bitswide, allowing for addressability of 0-4,294,967,295 bytes,which is conventionally denoted as a 4 GB range. This 4 GBrange can be an architectural constraint, especially withworkloads that need 2 GB or more of RAM to perform well.Virtual memory constraints tend to appear during periodsof transition from one processor architecture to another. Overthe course of Intel's processor evolution, there was a period of transition from 16-bit addressing that was a feature of theoriginal 8086 and 8088 processors that launched the PCrevolution to the 24-bit segmented addressing mode of the80286 to the current flat 32-bit flat addressing model that isimplemented across all Intel IA-32 processors. Currently, theIA-32 architecture is in a state of transition. As 64-bitmachines start to become more commonplace, they willdefinitively relieve the virtual memory constraints that arebecoming apparent on the 32-bit platform.This aspect of the evolution of Intel processing hardwareand software bears an uncanny resemblance to other proces-sor families that similarly evolved from 16 or 24-bit machinesto 32-bit addresses, and ultimately to today's 64-bit machines.In the IBM mainframe world, there was a prolonged focus onVirtual Storage Constraint Relief (VSCR) in the progressionof the popular 24-bit OS/360 hardware and software inalmost every subsequent release of new hardware andsoftware that IBM produced from 1980 to the present day.The popular book The Soul of a New Machine [1],chronicled the development of Data General's 32-bit addressmachines to keep pace with the Digital EquipmentCorporation's 32-bit Virtual Address Extension (VAX) of itsoriginal 16-bit minicomputer architecture. Even thoughtoday's hardware and software engineers are hardly ignorantof the relevant past history, they are still condemned to repeatthe cycle of delivering stopgap solutions that provide amodicum of virtual memory constraint relief until the nextmajor technological advance is ready.
Process virtual address spaces.
The Windows operatingsystem constructs a separate virtual memory address space onbehalf of each running process, potentially addressing up to 4GB of virtual memory on 32-bit machines. Each 32-bitprocess virtual address space is divided into two equal parts,as depicted in Figure 1. The lower 2 GB of each processaddress space consists of private addresses associated withthat specific process only. This 2 GB range of addressesrefers to pages that can only be accessed by threads runningin that process address space context. Each per process virtualaddress space can range from 0x0001 0000 to address 0x7fff ffff, spanning 2 GBs, potentially. (The first 64K addresses areprotected from being accessed - it is possible to trap manycommon programming errors that way.) Each process gets itsown unique set of user addresses in this range. Furthermore,no thread running in one process can access virtual memoryaddresses in the private range that is associated with anotherprocess.Since the operating system builds a unique addressspace for every process, Figure 2 is perhaps a betterpicture of what User virtual address spaces look like.Notice that the System portion of each process addressspace is identical. One set of System Page Table Entries(PTEs) maps the System portion of the process virtualaddress space for every process. Because System ad-dresses are common to all processes, they offer aconvenient way for processes to communicate with eachother, when necessary
Shared system addresses.
The upper half of each perprocess address space in the range of '0x8000 0000' to '0xffff ffff' consists of system addresses common to all virtualaddress spaces. All running processes have access to thesame set of addresses in the system range. This feat is
F
IGURE
2. U
SER
 
PROCESSES
 
SHARE
 
THE
S
YSTEM
 
PORTION
 
OF
 
THE
4 GB
VIRTUAL
 
 ADDRESS
 
SPACE
.
 
accomplished by combining the system's page tables witheach unique per process set of page tables.User mode threads running inside a process cannotdirectly address memory locations in the system rangebecause system virtual addresses are allocated using Privi-leged mode. This restricts memory access to kernel threadsrunning in privileged mode. This is a form of security thatrestricts access to system memory to authorized kernelthreads. When an application execution thread calls a systemfunction, it transfers control to an associated kernel modethread, and, in the process, routinely changes the executionstate from User mode to Privileged. It is in this fashion that anapplication thread gains access to system virtual memoryaddresses.Commonly addressable system virtual memory locationsplay an important role in interprocess communication, or IPC.Win32 API functions can be used to allocate portions of commonly addressable system areas to share data betweentwo or more distinct processes. For example, the mechanismWindows Server 2003 uses that allows multiple processaddress spaces to access common modules known asDynamically Linked Libraries (DLLs) utilizes this form of shared memory addressing. (DLLs are library modules thatcontain subroutines and functions which can be calleddynamically at run-time, instead of being linked statically tothe programs that utilize them.)
Virtual memory addressingconstraints
The 32-bit addresses that can be used on IA-32-compat-ible Intel servers are a serious architectural constraint. Thereare several ways in which this architectural constraint canmanifest itself. The first problem is running up against theCommit Limit, an upper limit on the total number of virtualmemory pages the operating system will allocate. This is astraightforward problem that is easy to monitor and easy toaddress - up to a point.The second problem occurs when a User process exhauststhe 2 GB range of private addresses that are available for itsexclusive use. The processes that are most susceptible torunning out of addressable private area are database applica-tions that rely on memory-resident caches to reduce theamount of I/O operations they perform. Windows Server2003 supports a boot option that allows you to specify how a4 GB virtual address space is divided between User privatearea virtual addresses and shared system virtual addresses.This boot option extends the User private area to 3 GB andreduces the system range of virtual memory addresses to 1GB.If you specify a User private area address range that islarger than 2 GB, you must also shrink the range of systemvirtual addresses that are available by a correspondingamount. This can easily lead to the third type of virtualmemory limitation, which is when the range of system virtualaddresses that are available is exhausted. Since the systemrange of addresses is subdivided into several different pools,it is also possible to run out virtual addresses in one of thesepools long before the full range of system virtual addresses isexhausted.In all three circumstances described above, running out of virtual addresses is often something that happens suddenly,the result of a program with a memory leak. Memory leaksare program bugs where a process allocates virtual memoryrepeatedly, but then neglects to free it when it is finishedusing it. Program bugs where a process leaks memory inlarge quantities over a short period of time are usually prettyeasy to spot. The more sinister problems arise when aprogram leaks memory slowly, or only under certain circum-stances, so that the problem only manifests itself at erraticintervals or only after a long period of continuous executiontime.However, not all virtual memory constraints are the resultof specific program flaws. Virtual memory creep due to slow,but inexorable workload growth can also exhaust the virtualmemory address range that is available. Virtual memorycreep can be detected by continuous performance monitoringprocedures, allowing you to intervene in advance to avoidotherwise catastrophic application and system failures.Unfortunately, many Windows server administrators do nothave effective proactive performance monitoring proceduresin place and only resort to using performance tools after acatastrophic problem that might have been avoided hasoccurred. This paper discusses the performance monitoringprocedures, as well as the use of other helpful diagnostic toolsfor Windows servers, that you should employ to detect virtualmemory constraints.
Virtual memory Commit Limit.
The operating systembuilds page tables on behalf of each process that is created. Aprocess's page tables get built on demand as virtual memorylocations are allocated, potentially mapping the entire virtualprocess address space range. The Win32 VirtualAlloc APIcall provides both for reserving contiguous virtual addressranges and committing specific virtual memory addresses.Reserving virtual memory does not trigger building pagetable entries because you are not yet using the virtual memoryaddress range to store data. Reserving a range of virtualmemory addresses is something your application might wantto do in advance for a data file intended to be mapped intovirtual storage. Only later when the file is being accessed arethose virtual memory pages actually allocated (or commit-ted).In contrast, committing virtual memory addresses causesthe Virtual Memory Manager to fill out a page table entry(PTE) to map the address into RAM. Alternatively, a PTEcontains the address of the page on one or more paging filesthat are defined that allow virtual memory pages to spill overonto disk. Any unreserved and unallocated process virtualmemory addresses are considered free.

Activity (7)

You've already reviewed this. Edit your review.
1 hundred reads
1 thousand reads
Tomasz Wójcik liked this
lwolfs4337 liked this
pn360 liked this
YetAnotherCoder liked this
jineshsurya liked this

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->