Ideally what is desired is an infinitely large, inexpensive, fast memory that is also nonvolatile. Unfortunately technology does not provide for such. The memory hierarchy is based on storage capacity, speed, and cost. Lower in the hierarchy, storage capacities have lesser speed and cost. Consequently, most computers have a memory hierarchy, with a small amount of very fast, inexpensive, volatile cache memory, some number of megabytes of medium-speed, medium-price volatile main memory (RAM) and hundreds or thousands of megabytes of slow, cheap, nonvolatile disk storage. It is the work of the operating systems to co-ordinate how these memories are used.

Figure 17 – Memory hierarchy The part of the operating system that manages the memory hierarchy is called the memory manager. Its role is to keep track of which parts of memory are in use and which parts are not in use, to allocate memory to processes when they need it and de-allocate it when they are done. It will also manage swapping between main memory and disk when main memory is not enough to hold all the processes. Memory management Memory management is one of the most important services provided by the operating system. An operating system has five major responsibilities for managing memory:

Process isolation – The operating system should prevent independent processes from interfering with the data and code segments of each other. Automatic allocation and management - Programs should be dynamically allocated across the memory depending on the availability and this may or may not be contiguous. It is not necessary that the programmer perceive this allocation. Modular programming support - The programmers should be able to define program modules and also be able to dynamically create/destroy/alter the size of modules. Protection and access control - Different programs should be able to co-operate in sharing some memory space. It is important to ensure that such sharing is controlled and processes should not be able to indiscriminately access the memory allocated to other processes.






Long-term storage - Users and applications may require means for storing information for extended periods of time. This is implemented using a file system.


3.2 Basic memory management
Memory management systems can be divided into two classes: those that move processes back and forth between main memory and disk during execution (swapping and paging) and those that do not. Swapping and paging are caused by lack of sufficient main memory to hold all the programs at once. 3.2.1 Mono-programming without swapping or paging The simplest possible memory management scheme is to run one program at a time, sharing the memory between the program and the operating system. In these scheme, there are three variations: The operating system may be at the bottom of memory in RAM, or it may be at the top in ROM, or device drivers may at the top in ROM and the rest of the system in RAM. The later model is common with small MS-DOS systems. On IBM PCs, the portion of the system in ROM is called the BIOS (Basic Input Output System).

Figure 18 When the system is organized like this, only one process can be running. As soon as the user types a command, the operating system copies the requested program from disk to memory and executes it. When the operating system receives another command, it loads a new program into memory and overwrites the previous one. 3.2.2 Multiprogramming with fixed partitions Mono programming is common with small computers with simple operating systems. It is desirable to have multiple processes running at once. On timesharing systems, having multiple processes in memory at the same time means that when one process is blocked waiting for I/O to finish, another one can use the CPU. One way of achieving multiprogramming is to divide memory into unequal partitions. Partitioning can be done manually when the system is started up. When a job is started, it is put in the input queue for the smallest partition that is large enough to hold it. Since partitions are fixed, any unused space in the partition is lost.


Fig 19(a) fixed memory partitions with separate queues( b) fixed memory partitions with single queues The disadvantage with of sorting incoming jobs into separate queues becomes apparent when the queue for a large partition is empty but the queue for a small partition is empty. For example, in Fig 17(a), partitions 1 and 3. An alternative is to maintain a single queue as shown in fig (b). Whenever a partition becomes free, the job closest to the front of the queue that fits in it could be loaded into the empty partition and run. Since it is undesirable to waste a large partition on a small job, a different strategy is to search the whole input queue whenever a partition becomes free and pick the largest job that fits it. Note that this algorithm discriminates against small jobs as being unworthy of having a whole partition, whereas, it is desirable to give small jobs the best service and not the worst. A solution is to have at least one small partition around. Another approach is to have a rule stating that a job that is eligible to run may not be skipped over more than k times. Each time it is skipped over it is given one point. When it has gained k points, it may not be skipped again. Relocation and protection Multiprogramming introduces relocation and protection. Different jobs will now have to be run at different addresses. When a program is linked (main program, user-written procedures and library procedures are combined into a single address space), the linker must know at what address the program will start in memory. Suppose the first instruction is a call to a procedure at absolute address 100 within the binary file produced by the linker. If this program is loaded in partition 1, that instruction will jump to absolute address 100, which is inside the operating system. What is needed is a call to 100K + 100. If the program is loaded into partition 2, it must be carried out as call to 200K + 100. This problem is known as the relocation problem. One possible solution is to actually modify the instructions as the program is loaded into memory. Programs loaded into partition 1 have 100K added to each address; programs loaded into partition 2 have 200K added to addresses. Relocation during loading does not solve the protection problem. A malicious program can always construct a new instruction and jump to it. In multiuser systems, it is undesirable to let processes read and write memory belonging to other users.


One solution is to divide memory into blocks of 2K bytes and assign a 4-bit protection code to each block. The PSW contained a 4-bit key. The hardware traps any attempt by a running process to access memory whose protection code differed from the PSW key. It is only the operating system that was allowed to change this key and not user processes. Another solution to both the relocation and protection problems is to equip the machine with two special hardware registers, called the base and the limit registers. When a process is loaded the base and limit registers are loaded with address of the start of its partition and length of the partition respectively. Every memory address generated automatically has the base register contents added to it before being sent to memory. Addresses are also checked against the limit registers to make sure that they do not attempt to address memory outside the current partition. Hardware protects the base and limit registers from user programs.

3.3 Swapping
With batch systems, organizing memory into fixed partitions is simple and effective. Each job is loaded into a partition when it gets to the head of the queue. It stays in memory until it is finished. With timesharing systems, the situation is different. Sometimes there is not enough main memory to hold all the currently active processes, so excess processes must be kept on disk and brought in to run dynamically. Two general approaches to memory management can be used depending on the available hardware. The simplest strategy, called swapping, consists of bringing each process in its entirety, running it for a while then putting it back to disk. The other strategy, called virtual memory, allows programs to run even when they are only partially in main memory. The operation of swapping is illustrated in the diagram below. Initially, only process A is in memory. Then process B and C are created or swapped in from disk. In fig (d) A terminates or is swapped out to disk. Then D comes in and B goes out. Finally E comes in.

Figure 20 – Memory allocation with variable partitions The main difference between the fixed partitions in the previous section and the variable partitions is that the number, location and size of partitions vary dynamically in the latter as processes come and go. The flexibility of not being tied to a fixed number of partitions that may


be too large or too small improves memory utilization, but complicates memory allocation and de-allocation as well as keeping track of it. When swapping creates multiple holes in memory, it is possible to combine them all into one big one by moving all processes downward as far as possible. This is known as memory compaction. It is usually not a common practice because it takes a lot of CPU time. How much memory should we allocate to a process when it is created or swapped in? If processes are created with a fixed size that never changes, then the allocation is simple: allocate what is needed, no more and no less. However, if a process’ data segments can grow, then there is a problem. If a hole is adjacent to the process, it can be allocated. On the other hand if the process is adjacent to another process, the growing process will have will either have to be moved to a hole big enough for it, or one or more processes will have to be swapped out to create a large enough hole. If a process cannot grow in memory and the swap area on the disk is full, the process will have to wait or be killed. If it is expected that most processes will grow as they run, it is better to allocate a little extra memory whenever a process is swapped in or moved, to reduce the reallocation overhead. However, when swapping processes to disk, only memory actually in use should be swapped.

Figure 21 – Process growth in memory In the figure above, we see a memory configuration in which space for growth has been allocated to two processes. Processes can have two growing segments, for example the data and stack segments. Each process has a stack at the top of its allocated memory that is growing downwards, and a data segment beyond the program text that is growing upward. The memory between them can be used for either segment. If it runs out, either the process will have to be moved to a hole with large enough space, swapped out of memory until a large enough hole can be created, or killed. 3.3.1 Memory Management with Bit Maps


When memory is allocated dynamically, the operating system must manage it. There are two ways to keep track of memory usage: bit maps and free lists. With bit maps, memory is divided into allocation units. Corresponding to each allocation unit is a bit in the bit map, which is 0 if the unit is free and 1 if it is occupied.

Figure 22 – Memory management with bit maps


The size of the allocation unit is an important design issue. The smaller the allocation unit the larger the bit maps. If the allocation unit is large, the bit map will be smaller, but appreciable memory will be wasted in the last unit if the process size is not exact multiple of the allocation unit. A bit map provides a simple way to keep track of memory words in a fixed amount of memory because the size of the bit map depends on the size of memory and size of allocation unit. The main problem with it is that when it has been decided to bring a k unit process into memory, the memory manager must search the bit map to find a run of k consecutive 0 bits in the map. This is a slow operation. 3.3.2 Memory Management with Linked Lists Another way of keeping track of memory is to maintain a linked list of allocated and free memory segments, where a segment is either a process or whole between two processes. The memory of fig 22(a) is represented in fig 22(c) as linked list of segments. Each entry in the list specifies a hole (H) or a process (P), the address at which it starts, the length and a pointer to the next entry. In this example, the segment list is kept sorted by address. When the processes and holes are kept on a list sorted by address, several algorithms can be used to allocate memory for a newly created or swapped in process. We assume that the memory manager knows how much memory to allocate. The simplest algorithm is first fit. The memory manager scans along the list of segments until it finds a hole that is big enough. The whole is then broken up into two pieces, one for the process and one for unused memory, except for the unlikely case of a perfect fit. Another variation of first fit is the next fit. It works the same way as first fit, except that it keeps track of where it is whenever it finds a suitable whole. The next time it is called to find a hole, it starts searching the list from the place where it left off last time, instead of the beginning as with the previous algorithm. The best-fit algorithm searches the entire list and takes the smallest hole that is adequate. Rather than breaking up a big hole that may be needed later, best fit, tries to find a hole that is close to the actual size needed. Best fit is slower than first fit and also results in more wasted memory because it tends to fill memory with tiny useless holes. To get around the problem of breaking up nearly exact matches into a process and a tiny hole, one could think about worst fit, i.e. take the largest available hole, so that the hole broken off will be big enough to be useful.

3.4 Virtual Memory
In certain situations you can have programs that don’t fit in the available memory. The solution is to split the program into units called overlays. The first overlay would start running and when it was done it would call another overlay. Some overlay systems are highly complex, allowing multiple overlays in memory at once. The overlays were kept on disk and swapped in and out of memory by the operating system dynamically on demand. In the early days, although the actual work of swapping overlays in and out of memory was done by the operating system, the work of splitting the program into modular units had to be done by the programmer. This was not easy. Later, people found ways of handing over this job to the system. The method that was devised came to be known as virtual memory.

The basic idea behind virtual memory is that the combined size of the program, data and stack may exceed the amount of physical memory available for it. The operating system keeps those parts of the program currently in use in main memory and the rest on disk. For example, a 16MB program can run on a 4MB machine by carefully choosing which 4MB to keep in memory at each instant, with pieces of the program being swapped between disk and memory as needed. Virtual memory can also work in a multiprogramming system, with bits and pieces of many programs in memory at once. While a program is waiting for part of itself to be brought in (waiting for I/O operation), the CPU can be given to another process, the same as in other multiprogramming systems. 3.4.1 Paging

Most virtual memory systems use a technique called paging. On any computer, there exists a set of addresses that programs can produce. When a program uses an instruction like: MOVE REG, 1000 It is copying, the contents of memory address 1000 to REG or vice versa. Addresses can be generated using indexing, base registers and other ways. These program-generated addresses are called virtual addresses and form the virtual address space. On computers without virtual memory, the virtual address is put directly onto memory bus and causes the physical memory word with the same address to be read or written. When virtual memory is used, the virtual addresses do not go directly to memory bus. Instead they go to a Memory Management Unit (MMU). MMU maps the virtual addresses onto physical memory addresses.

Figure 23 –position of MMU


An example of how this mapping works is shown in the figure below. In this example, we have a computer that can generate 16-bit addresses, from 0 up to 64K. These are virtual addresses. The computer however has only 32K of physical memory. A 64K program cannot be loaded entirely in memory. A complete copy of the program’s core image, up to 64K must be present on disk, however so that pieces can be brought in as needed.

Figure 24 – relation between virtual address and physical memory addresses The virtual address space is divided into units called pages. The corresponding units in the physical memory are called page frames. The pages and page frames are usually exactly the same size. In this example they are 4K. With 64K of virtual address space and 32K of physical memory, we have 16 virtual pages and 8 page frames. Transfers between memory and disk are always in units of a page. When a program tries to access address 0, for example, using the instruction MOVE REG, 0, the virtual address 0 is sent to the MMU. The MMU sees that this virtual address falls in page 0 (0 to 4095), which according to its mapping is page frame 2 (8192 to 12287). The memory knows nothing about the MMU and just sees a request for reading or writing address 8192, which it honors. Similarly, instruction MOVE REG, 8192 is effectively transformed into MOVE REG, 24576 The virtual address 8192 is on virtual page 2 and this page is mapped onto physical page frame 6 (physical address 24576 to 28671). In another example, virtual address 20500 is 20 bytes from the start of virtual page 5 (virtual address 20480 to 24575) and maps onto physical address 12888 + 20 = 12308. The fact that we can map 16 virtual pages onto any 8-page frames does not solve the problem that the virtual address space is larger than physical memory. Only 8 of the virtual pages are mapped


onto physical memory. In the hardware, a present/absent bit keeps track of whether a page is mapped or not. If a program tries to use an unmapped page, the MMU notices that the page is unmapped and causes the CPU to trap to the operating system. The trap is called a page fault. The operating system then picks one page frame and writes its content back to the disk. It then fetches the page just referenced into the page frame just freed, changes the map and restarts the trapped instruction. For example, using the instruction MOVE REG, 32780 which is 12 bytes within virtual page 8. The operating system may decide to evict page frame 1, then load virtual page 8 at physical address 4K and make two changes to the MMU map. First, it would mark page frame 1 entry’s as unmapped to trap any future access to virtual address between 4K and 8K. Then it would replace the cross in virtual page 8’s entry with a 1, so that when the trapped instruction is re-executed, it will map to virtual address 32780 onto physical address 4108. Lets see how the MMU works. In the figure below, we see an example of a virtual address, 8196 (0010000000000100 in binary), being mapped using the MMU map of fig 24. The incoming 16bit virtual address is split into a 4-bit page number and 12-bit offset. With 4 bits for page number we can represent 16 pages, and with 12 bits for the offset we can address all 4096 bytes within a page.


The page number is used as an index to the page table, yielding the number of the page frame corresponding to that virtual page. If the present/absent bit is 0, a trap to the operating system is caused. If the bit is 1, the page frame number is found in the page table is copied to the high-order 3 bits of the output register along with the 12-bit offset, which is unmodified. The 15-bit output register is then put onto the memory bus as the physical memory address.


3.4.2 Page Tables When mapping virtual addresses into physical addresses, the virtual address is split into a virtual page number and an offset. The virtual page number is used as an index into the page table to find the entry for that virtual page. From the page table entry, the page frame number (if any) is found. The page frame number is attached to the high-order end of the offset, replacing the virtual page number, to form a physical address that can be sent to the memory. The purpose of the page table is to map virtual pages onto page frames. There are two major issues to be considered:   The page table can be extremely large The mapping must be fast

Modern computers use virtual addresses of at least 32 bits. With a 4K-page size, a 32-bit address space has 1 million pages. With 1 million pages in virtual address space, the page table must have 1 million entries. The second point is a consequence of the fact that the virtual-to-physical mapping must be done on every memory reference. A typical instruction has an instruction word and often a memory operand as well. Consequently, it is necessary to make 2 or more page references per instruction. The need for large, fast page mapping is a significant constraint on the way computers are built. The simplest page table design is to have a single page table consisting of an array of fast hardware registers, with one entry for each virtual page, indexed by virtual page number. When a process is started, the operating system loads the registers with the process’ page table, taken from main memory. During execution no more memory references are needed for the page table. The advantage of this approach is that it is straightforward and requires no memory references during mapping. A disadvantage is that it is potentially expensive if the page table is large. Having to load the page table at every context switch can also hurt performance. At the other extreme, the page table can be entirely in main memory and a single register that points to the start of the page table is used. This design allows the memory map to be changed at context switching by reloading one register. The disadvantage is that it requires one or more memory references to read page table entries during execution of each instruction. 3.5 Page Replacement Algorithms When a page fault occurs, the operating system has to choose which page to remove from the memory to make room for the page that has to be brought in. If the page to be removed has been modified while in memory, it must be rewritten to the disk to bring the disk copy up to date. If the page has not been modified there is no need to rewrite it. The page to be read in just overwrites the page being evicted. It is possible to choose the page to be removed randomly, though system performance is much better if a page that is not heavily used is chosen. If a heavily used page is removed it will probably be brought back in quickly, resulting in extra overhead. 3.5.1 Optimal Page Replacement Algorithm


At the moment that a page fault occurs, one of the pages in memory will be referenced on the next instruction. Other pages may not be referenced until later on. Each page can be labeled with the number of instructions that will be executed before the page is first referenced. This algorithm says that the page with the highest label should be removed. The problem with this algorithm is that it is not realizable. At the point of the page fault the operating system has no idea of when each of the pages will be referenced next. By running a program on a simulator and keeping track of all page references, it is possible to implement optimal page replacement algorithm on the second run by using the page reference information collected during the first run. This method is not used in practical systems. 3.5.2 Not recently Used (NRU) Page Replacement Algorithm

Most computers with virtual memory have two status bits associated with each page for collecting useful statistics. R is set when the page is referenced (read) and M when the page is written to (modified). The bits are contained in each page table entry. It is essential that once a bit has been set to 1, it will stay until the operating system resets it to 0. When a process is started up, both page bits for all its pages are set to 0 by the operating system. Periodically, the R bit is cleared to distinguish pages that have been referenced recently, from those that have been. When a page fault occurs, the operating system inspects all the pages and divides them into four categories based on the current values of the R and M bits. Class 0: not referenced, not modified. Class 1: not referenced, modified Class 2: referenced, not modified Class 3: referenced, modified The NRU algorithm removes a page at random from the lowest numbered non-empty class. NRU is easy to understand, efficient to implement and gives performance that, while not optimal is often adequate. 3.5.3 The First-In, First-Out (FIFO) Page Replacement Algorithm

Consider a supermarket that has enough shelves to display exactly k different products. One day, some company introduces a new product that is an instant success. The supermarket has to get rid of one old product in order to stock it. One possibility is to find the product that the supermarket has been stocking longest and get rid of it on the grounds that no one is interested in it any more. This way, the supermarket maintains a list of all products it is currently selling in the order they were introduced. The same idea is applicable as a page replacement algorithm. The operating system maintains a list of all pages currently in memory, with the page at the head of the list the oldest one and the page at the tail the most recent arrival. On a page fault, the page at the head is removed and the new page added to the tail of the list. This is not a good algorithm because there are chances of removing a heavily used page. 3.5.4 The second Chance Page Replacement Algorithm.


A simple modification to FIFO that avoids the problem of throwing out a heavily used page is to inspect the R bit of the oldest page. If it is 0, the page is both old and unused, so it is replaced. If the R bit is 1, the bit is cleared and the page is put at the end of the list of pages, and its load time is updated as though it has just arrived in memory. The search continues. The algorithm can be illustrated using the following example. Pages A through H are kept on a linked list sorted by the time they arrived in memory. Suppose that a page fault occurs at time 20. The oldest page is A, which arrived at time 0, when the process started. If A has the R bit cleared, it is evicted from memory. On the other hand if the R bit is set, A is put on to the end of the list and its load time reset to current time (20). The R bit is also cleared and the search for a suitable page continues with B.

Figure 25 The second chance looks for an old page that has not been referenced in the previous clock interval. If all have been referenced then second chance degenerates into a pure FIFO. Suppose that all the pages have their R bits set. One by one the operating system moves the pages to the end of the list and clears their R bit. Eventually, it comes back to page A, which now has its R bit cleared. At this point A is evicted. Thus the algorithm always terminates. 3.5.5 The Least recently Used (LRU) Page Replacement Algorithm

A good approximation to the optimal algorithm is based on the observation that pages that have been heavily used in the last few instructions will probably be heavily used again in the next few instructions. Conversely, pages that have not been used for ages will probably remain unused for a longer time. This idea suggests a realizable algorithm: when a page fault occurs, throws out the pages that have been unused for the longest time. This strategy is called LRU (Least Recently Used) paging. To implement LRU, it is necessary to maintain a linked list of all pages in memory, with the most recently used page at the front and the least recently used page at the rear. The problem is that the list must be updated on every memory reference. Finding a page in the list, deleting and then moving it from to the front is a very time consuming operation.