Introduction to XFS

HEPSYSMAN
5th December 2005

Jonathan Wheeler
E-Science Centre, RAL
 What exactly is a filesystem ?

Organization:
‡ Large disk space to store small data items
‡ Use files and directories

In UNIX everything is a stream of bytes:

‡ More complex file definition in other OS¶s
‡ Devices are also presented in the filesystem
‡ Linux uses virtual filesystems, e.g. /proc
Jonathan Wheeler
e-Science Centre, RAL
 A little bit of background

Three generation of UNIX filesystems:

a) v7 / sysv / coherent / minix
b) ffs / ext2
c) vxfs / ext3 / jfs / xfs / reiserfs

Jonathan Wheeler
e-Science Centre, RAL
 Features of 3rd generation UNIX
filesystems

a) Intent logging / journaling

b) Flexible metadata structures
c) Dynamic inode allocations
d) Extents

Jonathan Wheeler
e-Science Centre, RAL
 History of XFS

Developed by Silicon Graphics (now SGI)

for use on IRIX:
‡ 1993 - original design
‡ 1994 - available for IRIX v 5.3
‡ 2000 - released as Open Source
‡ 2001/2002 ± some Linux distributions with
native XFS support
‡ 2003 ± XFS support in 2.4 kernel
Jonathan Wheeler
e-Science Centre, RAL
 XFS features (1)

Journaling:
‡ but only metadata updates, not data
updates.
Fast Transactions:
‡ uses efficient B+ tree structures for fast
searches and rapid space allocation

Jonathan Wheeler
e-Science Centre, RAL
 XFS Features (2)

64-bit file system:

a) 64bit variables for global counters and disk addresses

b) Large theoretical maximum filesystem size

i. For Linux 2.4: 2 Terabytes.
ii. For Linux 2.6 (when using 64 bit addressing on a
64 bit platform): 9 Petabytes (or the device limits)
iii. For Linux 2.6 (on 32 bit platforms): 16Terabytes
(even with 64 bit addressing)

c) Maximum file size is same as filesystem size

Jonathan Wheeler
e-Science Centre, RAL
 XFS Features (3)

Partitioned into Allocation Groups:

a) each AG manages its own free space
and inodes
b) provides scalability and parallelism within
the file system
c) limits the size of the structures needed to
track this information

Jonathan Wheeler
e-Science Centre, RAL
 XFS Features (3, continued)

d) allows many internal pointers to be 32-

bits
e) AGs typically range in size from 0.5 to
4GB
f) files and directories are not limited to a
single AG.

Jonathan Wheeler
e-Science Centre, RAL
 XFS Features (4)

Sophisticated support utilities:

a) fast mkfs (make a file system)
b) dump and restore utilities for backup
c) xfsrepair to fix corrupt filesystem
d) xfs_fsr (XFS defragmenter)
e) xfsdb (XFS debug)
f) xfscheck (XFS check)
g) xfs_growfs (enlarges filesystems online)
Jonathan Wheeler
e-Science Centre, RAL
 Why you might want to use XFS

a) Stable, mature code base (oldest

journaling filesystem available on Unix)
b) Very good performance for large reads
or writes
c) Designed for large systems
d) Supports user and group quotas
e) Support for Access Control Lists and
Extended Attributes
Jonathan Wheeler
e-Science Centre, RAL
 Extended Attributes

An extended attribute is a name/value pair

associated with a file. Attributes can be
attached to all types of inodes: regular
files, directories, symbolic links, device
nodes, and so forth. Attribute values can
contain up to 64KB of arbitrary binary
data.
See man attr for more details
Jonathan Wheeler
e-Science Centre, RAL
 Extended attribute namespaces

XFS implements three attribute namespaces:

a) user namespace available to all users,
protected by the normal file permissions
b) system namespace, accessible only to
privileged users and used for protected
filesystem meta-data such as access
control lists (ACLs) and hierarchical storage
manager (HSM) file migration status
c) security namespace used by security
modules (SELinux)
Jonathan Wheeler
e-Science Centre, RAL
 Why you might not want to use XFS

a) No data journaling
b) tar, cpio do not understand Extended
Attributes

Jonathan Wheeler
e-Science Centre, RAL
 Resources

XFS/Linux homepage:
http://oss.sgi.com/projects/xfs/ contains
pointers to whitepapers, books, articles,
etc.
Public mailing list: linux-xfs@oss.sgi.com
with searchable list archive
SGI Bugzilla database to report any bugs in
XFS for Linux
Jonathan Wheeler
e-Science Centre, RAL
 Comparison tests - 1

Tests were performed:

‡ on the same filesystem on the same
hardware (AMD Athlon 1666 Mhz, 500Mb)
‡ using 3 different operating systems:
i. Redhat 7.3 (2.4.20-28.7)
ii. Scientific Linux version 3 (2.4.21-20.EL)
iii. Scientific Linux version 4
(2.6.9-22.0.1.EL and 2.6.9.-11.EL.XFS)
Jonathan Wheeler
e-Science Centre, RAL
 Comparison tests - 2

Tests used different filesystem types:

‡ RH 7.3 ± ext2, ext3 (xfs not available)
‡ SL 3 ± ext2, ext3 (attempts to create xfs
filesystem caused system crash requiring
hardware reset)
‡ SL 4 ± ext2, ext3, xfs (default), xfs
(custom: mkfs.xfs ±d agsize=4g ±i
version=1 ±I size=512 <device>)
Jonathan Wheeler
e-Science Centre, RAL
 Comparison tests - 3

Tests were performed:

‡ using bonnie++ test program (see
http://www.coker.com.au/bonnie++)
‡ using the same parameters for each test:
/usr/local/sbin/bonnie++ -d /data \
-s 4096M:1024k \
-n 192:1048576:128:1024 -m csfnfs30 \
-u root 1>> outputfile 2>&1
Jonathan Wheeler
e-Science Centre, RAL
 Comparison tests - 4

12 tests in total, each reporting filesystem

performance and CPU use
The first 6 tests measure I/O intensive work:
‡ Sequential output for character, blocked
and rewrites
‡ Sequential input for character and blocked
‡ Random seeks
Jonathan Wheeler
e-Science Centre, RAL
 Comparison tests - 6

The other 6 tests measure performance for

many small files using:
‡ Sequential create, read and delete
‡ Random create, read and delete

Jonathan Wheeler
e-Science Centre, RAL
 Comparsion of filesystems (Sequential output)

)
c
e
/s
b
K
(
Serie
r
te
c
a
r
a
h
c
r
e
P

et et l et l et l et l et l l ecial

Filesystem type

Jonathan Wheeler
e-Science Centre, RAL
 Comparison of filesystems (Sequential output)

)
c
e
s
/
b
K
Serie
(
k
c
o
l
b
r
e
P

et et l et l et l et l et l l ecial

Filesystem type

Jonathan Wheeler
e-Science Centre, RAL
 Comparison of filesystems (Sequential output

)
c
e
s/
b Serie
K
(
et
ri
w
e
R

et et l et l et l et l et l l ecial

Filesystem type

Jonathan Wheeler
e-Science Centre, RAL
 Comparsion of filesystems (Sequential input)

)
c
e
s
/
b
K
( Serie
r
e
t
c
a
r
a
h
c
r
e
P

et et l et l et l et l et l l ecial

Filesystem type

Jonathan Wheeler
e-Science Centre, RAL
 Comparison of filesystems (Sequential input)

)
c
e
s
/
b
K Serie
(
k
c
o
l
b
r
e
P

et et l et l et l et l et l l ecial
Filesystem type

Jonathan Wheeler
e-Science Centre, RAL
 Compariosn of filesystems (Random seeks)

c
e
s
r/ Serie
e
b
m
u
N

et et l et l et l et l et l l ecial

Filesystem type

Jonathan Wheeler
e-Science Centre, RAL
 Comparison of filesystems (Sequential create)

)
c
e
s
/
m Serie
u
N
(
e
t
a
e
r
C

et et l et l et l et l et l l ecial

Filesystem type

Jonathan Wheeler
e-Science Centre, RAL
 Comparison of filesystems (Sequential create

)
c
e
s
r/
e Serie
b
m
u
n(
d
a
e
R

et et l et l et l et l et l l ecial

Filesystem type

Jonathan Wheeler
e-Science Centre, RAL
 Comparison of filesystems

)
c
e
s
r/
e Serie
b
m
u
n(
et
el
e
D

et et l et l et l et l et l l ecial

Filesystem type

Jonathan Wheeler
e-Science Centre, RAL
 Comparison of filesystems (Random create)

)
c
e
s
r/
e Serie
b
m
u
(N
e t
a
re
C

et et l et l et l et l et l l ecial

Filesystem type

Jonathan Wheeler
e-Science Centre, RAL
 Comparison of filesystems (Random create)

)
c
e
s
r/
e Serie
b
m
u
N
(
d
a
e
R

et et l et l et l et l et l l ecial

Filesystem type

Jonathan Wheeler
e-Science Centre, RAL
 Comparison of filesystems (Random create)

)
c
e
s
r/
e Serie
b
m
u
(N
e t
e l
e
D

et et l et l et l et l et l l ecial

Filesystem type

Jonathan Wheeler
e-Science Centre, RAL