Chapter 4 (File System)

Chapter 4
File Systems
File Systems
• Many important applications need to store more

information then have in virtual address space of a
process
• The information must survive the termination of the
process using it.
• Multiple processes must be able to access the
information concurrently.
Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall, Inc. All rights reserved. 0-13-6006639
File Systems
• Disks are used to store files

• Information is stored in blocks on the disks
• Can read and write blocks
File Systems
• Files are logical units of information created by

processes.
• A file is a named collection of related information that is
recorded on secondary storage
• Files must be persistent
• Managed by the OS
The part of OS dealing with files is known as file system
File Systems
• OS structures them, names them, protects them

• Two ways of looking at file system
• User-how do we name a file, protect it, organize the
files
• Implementation-how are they organized on a disk
• Start with user, then go to implementer
File Systems
• File Naming
• File Structure
• Directories
File Naming
● One to 8 letters in all current OS’s

● Many OS support 255 characters
● Unix, MS-DOS (Fat16) file systems discussed
● Fat (16 and 32) were used in first Windows systems
● Latest Window systems use Native File System
● All OS’s use suffix as part of name
● Unix does not always enforce a meaning for the suffixes
● DOS does enforce a meaning
File Naming
● Many OS support two part file names, two part separated

by a period.
● The part following the period is called file extension.
Suffix Examples
.
File Structure
File can be structured in three possible ways:
● Byte sequences
● Record sequences
● Tree
File Structure
Byte sequences:
● Unstructured sequence of bytes

● Provide maximum flexibility
Record sequences:
● Sequence of fixed length records with some internal

structure.
● For each file read and write operation evaluate one
record.
File Structure
Tree:
● File consist of a tree of record with different length.

● Key field in fixed position.
● Key is used for rapid searching.
File Structure
Three kinds of files. (a) Byte sequence.

(b) Record sequence. (c) Tree.
File Types
Regular- contains user information

Directories- system files for maintaining the structure of the
file system.
Character special files- related to input/output and used to
model serial (e.g. printers) I/O devices
Block special files-model disks
Regular Files
Regular files are the most common files. Another name for
regular files is ordinary files. Regular files contain data.
• ASCII
• Binary
ASCII:
● Contain information readable by the user.
● User can display and print
● Can be edited with any text editor
● Each line is terminated by a carriage return character. In
some system the line feed character is used.
Regular Files
Binary File:
● Contain information readable by the computer.

● Two Unix examples
○ Executable (magic field identifies file as being
executable)
○ Archive-compiled, not linked library procedures
● Every OS must recognize its own executable
Binary File Types (Early Unix)
(b)An archive-library
procedures compiled but
not linked
(a) An executable file (magic # identifiies it as an

Tanenbaum, Modern Operating Systems 3 e, ©
executable file)
File Access
● Sequential access
● Random access
File Attributes (hypothetical OS)
File Operations
● Create
● Delete
● Open
● Close
● Read
● Write
● Append
● Seek
● Get Attributes
● Set Attributes
● Rename
Directories
•Files which are used to organize a collection of

files
•Also called folders in some OS’s
Single Level Directory Systems
A single-level directory system containing four files.
Hierarchical Directory Systems
Path names
• Absolute /usr/carl/cs310/miderm/answers
• Relative cs310/midterm/answers
• . Refers to current (working) directory
• .. Refers to parent of current directory
Path Names
A UNIX directory tree.

Unix cp commands involving dots
Cp ../lib/dictionary .
• .. says go to parent (usr)

• . says that target of the copy is current directory
• cp /usr/lib/dictionary dictionary works
• cp /usr/lib/dictionary /usr/ast/dictionary also
works
Directory Operations
•Create
•Delete
•Opendir
•Closedir
•Readdir
•Rename
•Link
•Unlink
Directory Operations
System calls for managing directories
(from Unix)
• Readdir-reads next entry in open directory
• Rename
• Link-links file to path. File can appear in multiple
directories!
• Unlink-what it sounds like. Only unlinks from
pathname specified in call
File Implementation
File System Layout
• Files stored on disks. Disks broken up into one or more

partitions, with separate fs on each partition
• Sector 0 of disk is the Master Boot Record
• Used to boot the computer
• End of MBR has partition table. Has starting and ending
addresses of each partition.
• One of the partitions is marked active in the master boot
table
File Implementation
• Boot computer => BIOS reads/executes MBR

• MBR finds active partition and reads in first block
(boot block)
• Program in boot block locates the OS for that
partition and reads it in
• All partitions start with a boot block
A Possible File System Layout
Allocating Blocks to files
Implementing File
• Contiguous allocation
• Linked list allocation
• Linked list using table
• I-nodes
Contiguous Allocation
(a) Contiguous allocation of disk space for 7 files.

(b) The state of the disk after files D and F have been removed.
Contiguous Allocation
The good
• Easy to implement
• Read performance is great. Only need one seek to
locate the first block in the file. The rest is easy.
The bad-disk becomes fragmented over time
• CD-ROM’s use contiguous allocation because the fs size

is known in advance
• Reusing the space requires maintaining a list of holes.
Linked List Allocation
Storing a file as a linked list of disk blocks.
Linked Lists
The good
• Gets rid of fragmentation
• Only store the disk address of the first block.
The bad
• Random access is slow. Need to chase pointers to get to
a block
• The amount of data storage is no longer power of two.
Linked lists using a table in memory
• Put pointers in table in memory

• File Allocation Table (FAT)
The Solution-Linked List Allocation Using a
Table in Memory
Figure 4-12. Linked list allocation using a file allocation table

in main memory.
Linked lists using a table in memory
Advantage:
● Random access is much easier without making any disk
reference.
Disadvantage:
● Entire table must be in memory all the time to make it
work.
I-nodes
• Keep data structure in memory only for active files

• Data structure lists disk addresses of the blocks and
attributes of the files
• K active files, N blocks per file => k*n blocks max!!
• Solves the growth problem
I-nodes
• How big is N?
• Solution: Last entry in table points to disk block which
contains pointers to other disk blocks
I-nodes
Figure 4-13. An example i-node.
Implementing Directories
▪ Open file, path name used to locate directory

▪ Directory specifies block addresses by providing
▪ Address of first block (contiguous)
▪ Number of first block (linked)
▪ Number of i-node
(a) fixed-size entries with the disk addresses and attributes (DOS)
(b) each entry refers to an i-node. Directory entry contains
attributes. (Unix)
▪ How do we deal with variable length names?

▪ Problem is that names have gotten very long
Simplest approach:
set a limit on file name length
Advantage:
Simple
Disadvantage:
wastes a great deal of directory space
Alternative Approaches:
● In line
● In a Heap
In line Disadvantage:
● Removing a file introduced a variable-sized gap
● Reading a file causing page fault
In a Heap Disadvantage:
● Heap must be managed and page faults can still occur
Two ways of handling long file names in a directory. (a) In-line. (b)
In a heap.
Page Fault
P1 1 KB
P2
P1 1 KB
P3
P2
P4
CPU P3 P5
P6
P4
P7
P7
P5
P8
Not present in
main memory so
page fault
Main Memory
Secondary Memory
Searching in Directories:
● Linear Search
● Hash table
A different way to speed up searching large directories is to

cache the results of searches.
Hash Table
o Various way of making hash function
o Table size=n
name is hashed between (0 to n-1)
name/n = remainder
o Summation of number of words in a file name / n = remainder.
Advantage:
o Much faster lookup
Disadvantage:
o More complex administration.

Hash Table
The keys 1,3,12,4,25,6,18,20,8 are inserted into empty hash table
of length 10. Using open addressing with hash function H(i) =
𝑖 2 mod 10 and linear probing. What is the resultant hash table?
20 0
1 1
8 2
3
12 4
25 5
4 6 6
6 7
18 8
3 9
Shared Files
File system containing a shared file. File systems is a directed

acyclic tree (DAG)
Shared files
▪ If B or C adds new blocks, how does other owner find

out?
▪ Use special i-node for shared files-indicates that file
is shared
▪ Use symbolic link - a special file put in B’s directory if
C is the owner. Contains the path name of the file to
which it is linked
I-node problem
▪ If C removes file, B’s directory still points to i-node for

shared file
▪ If i-node is re-used for another file, B’s entry points to
wrong i-node
▪ Solution is to leave i-node and reduce number of owners
I node problem and solution
(a) Situation prior to linking. (b) After the link is created. (c) After
the original owner removes the file.
Symbolic links
▪ Symbolic link solves problem

▪ Can have too many symbolic links and they take time to
follow
▪ Big advantage-can point to files on other machines
Log Structured File System
CPU’s faster, disks and memories bigger (much) but
disk seek time has not decreased
• Caches bigger-can do reads from cache
• Want to optimize writes because disk needs to be updated
• Structure disk as a log-collect writes and periodically send
them to a segment in the disk. Writes tend to be very small
• Segment has summary of contents (i-nodes, directories….).
• Keep i-node map on disk and cache it in memory to locate i-
nodes
Log Structured File System
• Cleaner thread compacts log. Scans segment for current
i-nodes, discarding ones not in use and sending current
ones to memory.
• Writer thread writes current ones out into new segment.
• Works well in Unix. Not compatible with most file systems
• Not used
• Free space management in 2 ways: (Threading,Copying)
Journaling File Systems
Want to guard against lost files when there are
crashes. Consider what happens when a file has
to be removed.
• Remove the file from its directory.
• Release the i-node to the pool of free i-nodes.
• Return all the disk blocks to the pool of free disk blocks
• If there is a crash somewhere in this process, have a

mess.
Journaling File Systems
o Keep a journal (i,.e. list) of actions before you take them,
write journal to disk, then perform actions. Can recover
from a crash!
o Need to make operations idempotent. Must arrange data
structures to do so.
o Mark block n as free is an idempotent operation.
o Adding freed blocks to the end of a list is not
idempotent
o NTFS (Windows) and Linux use journaling
Virtual File Systems
o Have multiple fs on same machine
o Windows specifies fs (drives)
o Unix integrates into VFS
o Vfs calls from user
o Lower calls to actual fs
o Supports Network File System-file can be on a remote
machine
Virtual File Systems (1)
VFS-how it works
o File system registers with VFS (e.g. at boot time)
o At registration time, fs provides list of addresses of
function calls the vfs wants
o Vfs gets info from the new fs i-node and puts it in a v-node
o Makes entry in fd table for process
o When process issues a call (e.g. read), function pointers
point to concrete function calls
Virtual File Systems
. A simplified view of the data structures and code used by

the VFS and concrete file system to do a read.
File System Management
o Disk space management

o File System Backups
o File System Consistency
Disk Space Management
o Disk space management-use fixed size blocks which don’t
have to be adjacent
o If stored as consecutive bytes and file grows it will
have to be moved
o What is optimum (good) block size? Need information on
file size distribution.
o Don’t have it when building fs.
Disk Space Management Block Size (1)
Figure 4-20. Percentage of files smaller than a given size

(in bytes).
Disk Space Management Block Size (2)
Figure 4-21. The solid curve (left-hand scale) gives the data rate
of a disk. The dashed curve (right-hand scale) gives the disk
space efficiency. All files are 4 KB.
Disk Space Management
o Bigger block size results in better space utilization, but
worse transfer utilization
o If allocation unit is too large- waste space
o If allocation unit is too small- waste time
o trade-off is space vs data rate
o There is no good answer (Nature wins this time)
o Might as well use large (64 KB) block size as disks are
getting much bigger and stop thinking about it
Keeping Track of Free Blocks (1)
Figure 4-22. (a) Storing the free list on a linked list. (b) A bitmap.
How to keep track of free blocks
o Links-need ~1.9 million blocks
o Bit-map~need 60,000 blocks
o If free blocks come in runs, could keep track of beginning
and block and run length. Need nature to cooperate for
this to work.
o Only need one block of pointers in main memory at a time.
Fills up=>go get another
Keeping Track of Free Blocks (2)
Figure 4-23. (a) An almost-full block of pointers to free disk blocks

in memory and three blocks of pointers on disk. (b) Result of
freeing a three-block file. (c) An alternative strategy for
handling the three free blocks. The shaded entries represent
pointers to free disk blocks.
Disk Quotas-you can’t have it all
o Entry in open file table points to quota table

o One entry for each open file
o Places limits (soft, hard) on users disk quota
o Illustration on next slide
Disk Quotas
Figure 4-24. Quotas are kept track of on a per-user basis

in a quota table.
File System Backups
Backups to tape are generally made to handle one

of two potential problems:
• Recover from disaster

• Recover from stupidity
Backup takes a long time and occupies a large amount of

spaces.
File System Backups
Entire file system should backup/ part of it-
o Can get binaries from manufacturer’s CD’s
o Temporary files don’t need to be backed up
o Special files (I/O) don’t need it
Desirable to backup specific directories and files

rather than entire file system.
File System Backups
● No need to backup files that have not changed
since the previous backup - Incremental dumps.
o Complete dump weekly/monthly and daily

dump of modified files.
o Dump only those files changed since the
previous backup
● Compress the data before backup. The decision
to compress the backup stream must be carefully
considered.
File System Backups
● Problem-want to compress data before dumping
it, but if part of the tape is bad…..
● It is difficult to perform a backup on an active file
system. Snapshot algorithms are available.
● Introduces non technical problems for a
organization.
● Backup tapes should be kept off-site, but that also
have more security issue.
File System Backups
Two strategies can be used for dumping a disk to
tape:
● Physical dump
● Logical dump
Dumping strategies
o Physical-dump the whole thing.

o The good –it is simple to implement
o Don’t want to dump
o unused blocks => program needs access
to the unused block list and must write
block number for used blocks on tape
o bad blocks. Disk controller must detect and
replace them or the program must know
where they are (they are kept in a bad
block area by the OS)
Good vs Bad
o The good-easy to implement.

o Simplicity.
o The bad
o Can’t skip a particular directory
o Can’t make incremental dumps
o Can’t restore individual files
o Not used
o Use logical dumps
Logical Dumps
● Starts at one or more specified directory and

recursively dumps all files/directories below it
which have changed since a given time.
● Dump tapes gets carefully identified directories

and files to make easy to restore a specific file
or directory.
Logical Dumps
● Examine standard algorithm used by Unix. Dumps

files/directories that lie on path to modified
file/directory because,
o Can restore path on different computer

o Have the ability to restore a single file
File System Backups
Figure 4-25. A file system to be dumped. Squares are directories,

circles are files. Shaded items have been modified since last
dump. Each directory and file is labeled by its i-node number.
Logical Dump Algorithm
o Uses bitmap indexed by i-node

o 4 phases
o Phase 1-starts at root and marks bits for
modified files and all directories (a)
o Phase 2-walks the tree, unmarks directories
without modified files in/under them (b)
o Phase 3-go through i-nodes and dump
marked directories (c)
o Phase 4-dump files (d)
File System Backups
Bitmaps used by the logical dumping algorithm.
File System Consistency
o Crash before blocks written out results in an
inconsistent state=>
o Need utility program to check for consistency
in blocks and files
(fsck in Unix, scandisk in Windows)(blocks/files)
o For block uses two tables
o How many times is block present in a file
o How many times block is present in free
list
o Device then reads all of the i-nodes,
increments counters
File System Consistency
Figure 4-27. File system states. (a) Consistent. (b) Missing block.
(c) Duplicate block in free list. (d) Duplicate data block.
Solutions
o Missing block (b)-put on free list
o Block occurs twice on free list (c)-re-build free
list
o Block occurs twice on blocks in use (d)- notify
user. (If one file is removed, then block
appears on both lists)
File Consistency
o Look at files instead of blocks

o Use table of counters, one per file
o Start at root directory, descend, increment
counter each time file shows up in a directory
o Compares counts with link counts from the i-
nodes. Have to be the same to be consistent

Chapter 4 (File System)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 4 (File System)

Uploaded by

Copyright:

Available Formats

Chapter 4

• Many important applications need to store more

• Disks are used to store files

• Files are logical units of information created by

The part of OS dealing with files is known as file system

• OS structures them, names them, protects them

● One to 8 letters in all current OS’s

● Many OS support two part file names, two part separated

File can be structured in three possible ways:

● Unstructured sequence of bytes

● Sequence of fixed length records with some internal

● File consist of a tree of record with different length.

Three kinds of files. (a) Byte sequence.

Regular- contains user information

● Contain information readable by the computer.

● Every OS must recognize its own executable

(a) An executable file (magic # identifiies it as an

•Files which are used to organize a collection of

A single-level directory system containing four files.

A UNIX directory tree.

• .. says go to parent (usr)

• Files stored on disks. Disks broken up into one or more

• Boot computer => BIOS reads/executes MBR

(a) Contiguous allocation of disk space for 7 files.

The bad-disk becomes fragmented over time

• CD-ROM’s use contiguous allocation because the fs size

Storing a file as a linked list of disk blocks.

• The amount of data storage is no longer power of two.

• Put pointers in table in memory

Figure 4-12. Linked list allocation using a file allocation table

• Keep data structure in memory only for active files

Figure 4-13. An example i-node.

▪ Open file, path name used to locate directory

▪ How do we deal with variable length names?

A different way to speed up searching large directories is to

o Summation of number of words in a file name / n = remainder.

o Much faster lookup

o More complex administration.

File system containing a shared file. File systems is a directed

▪ If B or C adds new blocks, how does other owner find

▪ If C removes file, B’s directory still points to i-node for

▪ Symbolic link solves problem

• If there is a crash somewhere in this process, have a

o NTFS (Windows) and Linux use journaling

. A simplified view of the data structures and code used by

o Disk space management

Figure 4-20. Percentage of files smaller than a given size

Figure 4-23. (a) An almost-full block of pointers to free disk blocks

o Entry in open file table points to quota table

Figure 4-24. Quotas are kept track of on a per-user basis

Backups to tape are generally made to handle one

• Recover from disaster

Backup takes a long time and occupies a large amount of

Desirable to backup specific directories and files

o Complete dump weekly/monthly and daily

o Physical-dump the whole thing.

o The good-easy to implement.

● Starts at one or more specified directory and

● Dump tapes gets carefully identified directories

● Examine standard algorithm used by Unix. Dumps

o Can restore path on different computer

Figure 4-25. A file system to be dumped. Squares are directories,

o Uses bitmap indexed by i-node