Linux Journaling File System: ext3

Shangyou zeng
Physics & Astronomy Dept., Ohio University
Athens, OH, 45701

Abstract
In Red Hat Linux 7.2, Red Hat provides the first officially supported journaling file
system: ext3. The ext3 file system has incremental enhancements to the ext2 file system.
It has several advantages: availability, data integrity, speed and easy transition. This
paper summarizes some of those advantages, explain the detail of journaling file system,
and discuss the methods that transfer ext2 file system to ext3 file system.

1.Introduction

ext3 is a Journalizing file system for Linux. It was written by Dr Stephen C. Tweedie for
2.2 kernels. Peter Braam, Andreas Dilger and Andrew Morton ported the file system to
2.4 kernels with much valuable assistance from Stephen Tweedie Hurray. Now ext3 is
merged into Linus kernel code from Kernel 2.4.15 onwards. The ext3 file system is a
journaling file system that is 100% compatible with all of the utilities created for
creating, managing, and fine-tuning the ext2 file system, which is the default file system
used by Linux systems for the last few years.

2.Advantages of Ext3

Why do we want to transfer the file system ext2 to the file system ext3? What are
advantages of ext3 compared to ext2? There are four main reasons: availability, data
integrity, speed, and easy transition.

A. Availability

After an unclean system shutdown (unexpected power failure, system crash), each ext2
file system needs to check the consistency by the e2fsck program. Only passing the
consistency check, ext2 file system then can be mounted. The amount of time that the
e2fsck program takes is mainly determined by the size of the file system. Today, the size
of the file system is becoming larger and larger, then this takes a long time. The bigger
the file system is, the longer the consistency check takes. File systems that are several
hundreds of gigabytes in size may take an hour or more to check.

Ext3 writes the data into disk in such a way that the file system is always consistent.
Then, ext3 does not need a file system consistency check after an unclean system
shutdown, except for certain rare hardware failure cases. The time to recover an ext3 file
system after an unclean system shutdown does not depend on the size of the file system,
but, it depends on the size of the “journal” used to maintain the file system consistency.
The default journal size is very small, only takes about a second to recover.
B. Data Integrity

The ext3 file system can provide stronger guarantees about data integrity in case of an
unclean system shutdown. We can choose the type and level to protect the data
received.Essentially, you can choose either to keep the file system consistent but allow
for damage to data on the file system in the case of unclean system shutdown (for a
modest speed up under some but not all circumstances) or to ensure that the data is
consistent with the state of the file system (which means that you will never see garbage
data in recently-written files after a crash.) Ext3 has the same on-disk format as ext2 it
can use the very well tested and reliable e2fsck to ensure file system integrity and recover
from errors. The safe choice keeps the data consistent with the state of the file system,
and it is default.

C. Speed

Because ext3’s journaling optimizes the hard drive head motion, ext3 is often faster than
ext2, despite writing some data more than once. There are three journaling modes to
optimize speed, choosing to trade off some data integrity.

The first mode is data = writeback. This mode limits the data integrity guarantees, allow
old data to show up in files after a crash, for a potential increase in speed under some
circumstances. This mode is the default journaling mode for most journaling file systems.
This mode provides more limited data integrity guarantees of the ext2 file system, and
avoid the long file system check at the boot time.

The second mode is data = ordered. This mode is the default mode. This mode
guarantees that the data is consistent with the file system; recently-written files will never
show up with garbage contents after a crash.

The last mode is data=journal. This mode requires a larger journal for reasonable speed
in most cases and therefore takes longer to recover in case of unclean shutdown, but is
sometimes faster for certain database operations, NFS, and synchronous MTA (mail
server) operations.

The default mode is recommended for all general-purpose computing needs. To change
the mode, add the data = something option to the mount options for that file system in
the /etc/fstab file.

D. Easy Transition

It is easy to transfer ext2 to ext3 and gain the benefits of a robust journaling file system,
without reformatting. There is no need to do a long, tedious and error-prone backup-
reformat-restore operation in order to experience the advantages of ext3. The tune2fs
program can add a journal to an existing ext2 file system. If the file system is already
mounted while it is being transitioned, the journal will be visible as the file .journal in the
root directory of the file system. If the file system is not mounted, the journal will be
hidden and will not appear in the file system. Just run tune2fs –j /dev/hda1.

3.What is Journaling?

In order to minimize the file system inconsistencies and minimize system restart time
after an unclean system shutdown, before the changes are actually made to the file
system, journaling file system will keep track of the changes that will be made to the file
system. The records of journaling file system changes are stored in a separate part of the
file system, and the records are usually known as the “journal” or “log”. Once these
journal records are safely written, the journaling file system applies these changes to the
file system and then purges those records from the journaling record.

Because journaling records are written before the file system changes are made, and
because the file system keeps these file system change records until they have been safely
and completely applied to the file system, journaling file systems maximize file system
consistency and minimize system restart time after an unclean system shutdown. When a
computer using journaling file system is rebooted, the mount program will check the
journal record. If the journal records have some changes that are not marked as being
done, then the changes will be applied to the file systems. The mount program then can
guarantee the consistency of the file systems. In most cases, the system does not have to
check the file system consistency, meaning that computers using journaling file systems
are available almost instantly after rebooting them. The chances of losing data due to file
system consistency problems are similarly reduced.

There are many journaling file systems available for Linux. The best known is XFS. XFS
is originally developed by Silicon Graphics but now released as open source. The
ReiserFS is a journaling file system developed especially for Linux. JFS is a journaling
file system originally developed by IBM but now released as open source. The ext3 file
system is developed by Dr. Stephen Tweedie at Red Hat and a host of other contributors.

4. How does journaling work?

The action of most serious database engines is a transaction. A transaction composes of a

set of single operation that satisfy several properties. The most important property of
transaction is the so-called ACID. ACID is the abbreviation of Atomicity, Consistency,
Isolation and Durability. The most important feature related to the journaling file system
is the Atomicity. This property implies that all operations belonging to a single
transaction are completed without errors or cancelled, producing no changes. Combined
with the Isolation, this property makes the transaction look as if they were atomic
operation that can not be broken into parts. While exploiting concurrency in database, the
transaction property solves the problem related to keeping consistency. In order to
implement this property, database records every single operation within one transaction
into a log file. In the log file, the operation name and operation argument’s content before
the operation’s execution are logged. After every single transaction, the log buffer is
written into disks, which is so called commit operation. Therefore, if there is a system
crash, we could trace the log back to the first commit statement, writing the argument’s
previous content back to its position in the disk.

Journal file system uses the same technique above to log the file system operations, then
the file system can be recoverable in a small period time after a system crash.

There are some differences between databases and file system journaling. One major
difference is that databases log users and control data, but file systems only log metadata.
Metadata are the control structures inside a file system: I-nodes, free block allocation
maps, I-nodes maps, etc.

5. Multiple Journaling Modes in the ext3 file system

Ext3 file system is compatible with ext2 file system, and it is easy to convert ext2 file
system to ext3 file system. The ext3 file system also offers several different types of
journaling. A classic issue in journaling file systems is whether they only log changes to
file system metadata or log changes to all file system data, including changes to files
themselves. The ext3 file system supports three different journaling modes, which you
can activate in the /etc/fstab entry for an ext3 file system. The three journaling modes are
the following:

Journal: This mode records all file system data and metadata changes. It is the
slowest mode of the three ext3 journaling modes. This journaling mode minimizes
the chance of losing the changes that have been made to any file in an ext3 file
system.

Ordered: This mode only records changes to file system metadata, but flushed file
data updates to disk before making changes to associated file system metadata.
This mode is the default ext3 journaling mode.

Writeback: This mode only records changes to file system metadata. This mode
relies on the standard file system write process to write file data changes to disk.
This is the fastest ext3 journaling mode.

The differences between these journaling modes are both subtle and profound. When the
journal mode is used, it is necessary that ext3 file system writes every change to a file
system twice, once to the journal, and then to the file system again. This can reduce the
overall performance of the file system. This mode is the favorite mode of the users. This
mode records both metadata and data updates in ext3 journal, which can be replayed in a
system reboot, so it can minimize the chances of losing changes to the file.

When the order mode is used, only file system metadata changes are logged. Then it
reduces redundancy between writing to the file system and to the journal, so it is faster
than the journaling mode. This mode does not record the changes to file data. In order to
guarantee that files in the file system will never be out of synchronization with any
related changes to the file system metadata, the changes to file data must be done before
associated file system metadata changes are made by the ext3 journaling daemon, which
can reduce the performance of the file system.

The writeback mode is faster than the other two ext3 journaling modes. This mode only
records change to file system metadata and does not wait for associated changes to file
data to be written before updating things like file size and directory information. Because
updates to file data are not done synchronously to recorded changes to the file system
metadata, files in the file system may exhibit metadata inconsistencies to which updated
data was not written when the system went down.

The ordered mode is the default mode of ext3 file system, but the different journaling
mode can be specified by updating the file system option potion of an /etc/fstab entry. For
example, if in order to specify the journal mode, an /etc/fstab entry would look like the
following:
/dev/hda5 /opt ext3 data=journaling 10

6. Installing ext3

The ext3 file system is directly developed on the basis of its ancestor, ext2. Because ext3
file system is just the ext2 file system with journaling, it is convenient to transfer from
ext2 file system to ext3 file system, and vice versa. The obvious shortcoming of ext3 file
system is that it does not implement any modern file system features to increase data
manipulation speed and packing.

Ext3 comes as a patch of 2.2.19 kernel of linux. So at first, we need get a linux-2.2.19
kernel from ftp.kernel.org or from one of its mirrors. The patch is available at
ftp.linux.org.uk/pub/linux/sct/fs/jfs or ftp.kernel.org/pub/linux/kernel/people/sct/ext3 or
from one mirror of this site. In the above address, we need get the following files,

• ext3-0.0.7a.tar.bz2: the kernel patch.

• e2fsprogs-1.21-WIP-0601.tar.bz2: the e2fsprogs suite with ext3 support.

Then we need to copy Linux kernel linux-2.2.19.tar.bz2 and ext3-0.0.7a.tar.bz2 files to

/usr/src directory and extract them:
mv linux linux-old
tar -Ixvf linux-2.2.19.tar.bz2
tar -Ixvf ext3-0.0.7a.tar.bz2
cd linux
cat ../ext3-0.0.7a/linux-2.2.19.kdb.diff | patch -sp1
cat ../ext3-0.0.7a/linux-2.2.19.ext3.diff | patch -sp1

After the kernel is compiled and installed, we need to make and install the e2fsprogs:
tar -Ixvf e2fsprogs-1.21-WIP-0601.tar.bz2
cd e2fsprogs-1.21
./configure
make
make check
make install

The next step is to make an ext3 file system in a partition. After rebooting with the new
kernel, we have two options: make a new journaling file system or journal an existing
one.

In order to make a new ext3 file system, we just need use the mke2fs from the installed
e2fsprogs, and use the “-j” option when running mke2fs. The command is following:
mke2fs -j /dev/xxx

where /dev/xxx is the device where you would create the ext3 file system. The "-j" flag
tells mke2fs to create an ext3 file system with a hidden journal. You could control the
size of the journal using the optional flag -Jsize=<n> (n is the preferred size of the
journal in Mb).

7. Converting from Ext2 to Ext3

The conversion procedure from ext2 to ext3 is simple enough. Suppose /dev/hda10 is
mounted as /test, then the procedure would be as follows:

• Log in as root
• Make sure /etc/fstab has /dev/hda10 mounted to /test as ext2, read
write
• unmount /dev/hda10
o If /dev/hda10 can not be unmounted, then we need remount it as read
only mode. The command is:
mount -o remount,rw /dev/hda10
• tune2fs -j /dev/hda10
• Edit the file of /etc/fstab, and change ext2 to ext3 for the enrty
/dev/hda10.
• mount /dev/hda10
• /sbin/shutdown -h now
• mount | grep /dev/hda10
o If it's not shown as ext3, reboot again, if still not shown as ext3, then need
troubleshoot to fix the conversion.
o If it is shown as ext3, then the conversion is finished.

The tunefs command creates the journal file. The journal file is kept in a special inode on
the device by default. You then need change the /etc/fstab entry from ext2 to ext3
to reflect it's a journaling file system, and then mount it again.
8. Conclusion

The ext3 journaling file systems provide significant across the whole spectrum of Linux
users. The file systems minimize delays when rebooting a Linux system after a computer
crash, and improve the consistency of file systems. The ext3 file system is compatible
with the ext2 file system, and it is easy to transfer ext2 file system to ext3 file system, and
vice versa. This compatibility also extends the usability of all of the utilities that have
already been developed for working with the ext2 file system. The ext3 file system is a
true successful solution for improving the availability and consistency of Linux systems
everywhere.

Bibliography

1. Bill Von Hagen http://www.linuxplanet.com/linuxplanet/reports/4136/1/

2. http://www.symonds.net/~rajesh/howto/ext3/ext3-1.html
3. Rajesh Fowkar http://home.arcor.de/m.heider/ext3.html
4. Steve Litt http://www.troubleshooters.com/linux/ext2toext3.htm
5. Matteo Dell'Omodarme http://www.linuxgazette.com/issue68/dellomodarme.html
6. Michael K. Johnson http://linux.bryanconsulting.com/stories/storyReader$121
7. Juan I. Santos Florido http://www.linuxgazette.com/issue55/florido.html
8. Moshe Bar http://www.linux-mag.com/2000-08/journaling_04.html