Professional Documents
Culture Documents
The Server Windows of The Storage Now
The Server Windows of The Storage Now
Filesystem - method of storing and organizing computer files and the data they
contain to make it easy to find and access this information. A file system also maintains
the physical location of the files so that you can find and access the files in the future
A Filesystem provides:
- a way to store data about these files — for example, their filenames,
permissions, and other attributes.
- an index — a list of the files on the drive and where they’re located on the
drive, so the operating system can see what’s on the drive in one place rather
than combing through the entire drive to find a file.
- has a Master file table(index) contains all information about the file: size,
allocation, name etc
- examples of filesystem: NTFS local filesystem for Windows, EXT3 local for
Linux, network filesystem NFS
1. File based storage - a file is typically structured in a file system, which is nothing
more than a hierarchical way of organizing files so that an individual file can be
located by describing the path to that file. We know that certain attributes --
information that might describe a file and its contents, such as its owner, who can
access the file, and its size -- are conveniently stored as metadata in a file
system.
2. Block storage - a block is a chunk of data, and when appropriate blocks are
combined, it creates a file. A block has an address, and the application retrieves
a block by making a SCSI call to that address. It is a very microscopic way of
controlling storage. Unlike in the case of NAS, the application decides where to
place the data and how to organize the storage. How the blocks are combined or
accessed is left up to the application.
With block storage, files are split into evenly sized blocks of data, each with its
own address but with no additional information (metadata) to provide more
context for what that block of data is.
Under the right circumstances, granting this level of granular control to the
application allows it to extract the best performance from a given storage array.
This is the reason why block storage has been king of the hill for performance-
centric applications, mostly transactional and database-oriented
1
3. Object based storage - is a storage architecture that manages data as objects,
as opposed to other storage architectures like file systems which manage data
as a file hierarchy and block storage which manages data as blocks within
sectors and tracks.
- abstracts file and block and stores and has a flat namespace of objects
1. objects are kept inside a single repository, and are not nested as files inside a
folder inside other folders
2. object storage keeps the blocks of data that make up a file together and adds
all of its associated metadata to that file.
3. the totality of the data, be it a document, audio or video file, image or photo,
or other unstructured data, is stored as a single object
Instead of organizing files in a directory hierarchy, object storage systems store files in a
flat organization of containers (called "buckets" in Amazon S3) and use unique IDs
(called "keys" in S3) to retrieve them.
2
Extra references:
https://cloudian.com/blog/object-storage-vs-file-storage/
3
Physical types of storage:
The main interfaces and protocols that are used for direct-attached storage connections
are Serial ATA (SATA), external Serial Advanced Technology Attachment (eSATA),
serial attached SCSI (SAS), and Fibre Channel.
4
Network-attached storage - devices connect storage to a network where they are
accessed through file shares. Unlike direct-attached storage, they are not directly
attached to the computer.
Network-attached storage devices support Ethernet connection, and they typically allow
an administrator to manage disk space, set disk quotas, provide security, and utilize
snapshot technologies. Network-attached storage devices support multiple protocols.
These include network attached file systems, Common Internet File Systems (CIFS),
and Server Message Block (SMB).
Storage Area Network (SAN) - is a dedicated network that allows you to share storage.
A SAN consists of a storage device, the interconnecting network infrastructure (switches
and so on), and servers that are connected to this network. SAN devices provide
continuous and fast access to large amounts of data. The communication and data
transfer mechanism for a given deployment is commonly known as a storage fabric.
A SAN uses a separate network, and it is generally not accessible by other devices
through the local area network.
A SAN does not provide file abstraction, only block-level operations. The most common SAN
protocols used are iSCSI, Fiber Channel, and Fiber Channel over Ethernet (FCOE).
5
HBA (Host Bus Adapter) - is most often used to refer to a Fibre Channel interface card
and iSCSI HBA for iSCSI SAN (offload TCP/IP and iSCSI processing from a server's CPU)
Master Boot Record (MBR) - is legacy partitioning scheme that is used by most
operating systems:
1. the MBR itself resides entirely on the first sector (512 bytes) of a hard disk.
2. Of that space, the first 440 bytes of the MBR are devoted to the boot loader.
6
3. the BIOS reads this code and executes it when the computer boots.
4. followed by data space used to store information about the four Primary
partitions
Extended Partition is just a defined area where logical drives reside. In order to create
more than 4 partitions, extended partition is introduced. Logical Drive is the hard disk
partition created in extended partition.
GUID Partition Table (GPT) - is a new standard for the layout of the partition table:
9. uses the 64 bit disk pointers, which allow for a maximum disk partition size of
9.4 Zeta bytes, or 9.4 billion Tera Bytes.
10. from Windows Server 2008 onwards, only x64 versions ** of the Microsoft OS
support GPT disks as the boot disk in EFI (Extensible Firmware Interface)
boot mode
11. GPT disks are not supported as the boot disk in BIOS Mode when booting to
Windows OSes.
7
8
You can convert a disk between MBR and GPT as long as the disk is completely empty.
Unfortunately, once you've created any partitions or volumes on the disk, you can no
longer convert between the two types.
9
To list partitions from a disk:
10
1. you need to physically install or add a new virtual disk if machine is virtual
2. Windows automatically** recognizes the new hardware and makes it
available(not ready for use yet, you just can see it as an offline disk):
o If the disk is a basic disk that is already partitioned and formatted, you
can use it without initializing, but it will initially appear "offline" in Disk
Management.
o If it's a brand-new disk that has never been partitioned or formatted,
you need to initialize it first:
11
3. If disk is new, initialize the disk and choose MBR or GPT partition table as
necessary:
4. Create partitions/volumes, choosing the size of it, the mount point drive letter,
the filesystem (most cases is NTFS, the other ones are obsolete and not
used in enterprise environments anymore) and Quick format:
12
5. Once the formatting finished the new partition should be available in Windows
Explorer to upload data on it.
13
SAN disks (iSCSI storage) in Windows
iSCSI is an IP based storage networking standard for linking storage devices with
servers.
By carrying SCSI commands over TCP/IP networks, iSCSI can facilitate data transfers
over local area networks (LANs), wide area networks (WANs), or the Internet. Through
iSCSI, the space on storage server will be regarded as local disks by client's operation
system. In reality, all data transferred to the disk are actually transferred over network to
the storage server.
- the only interconnectivity required between servers and the disks on storage
subsystems is an IP network (this allows many servers to share the same storage
subsystem)
- is block based storage and the storage vendors offer SAN with iSCSI technologies
(EMC VNXe, Netapp iSCSI SAN etc)
- there are NAS systems with iSCSI support as well
- starting with Win 2008 R2 *** iSCSI server support was introduced (you can create
an iSCSI server/target with Win 2008 R2 )
*** Win 2008 R2 is not natively installed the iSCSI target software, needs to be
downloaded from Microsoft and installed
iSCSI target server - is the server where iSCSI target service is running.
Microsoft Internet iSCSI Initiator - enables you to connect a host computer that is
running Windows Server® 2008 R2 or higher to an external iSCSI-based storage array
through an Ethernet network adapter (any kind of iSCSI storage not only Microsoft).
iSCSI initiator service runs on the client and used to make a connection to the iSCSI
Target by logging on to a Target server.
iSCSI virtual disks - are created on iSCSI target server and associated to the iSCSI
target. iSCSI virtual disk represents an iSCSI LUN, which are connect to the clients
using iSCSI initiator.
IQN: It is a unique identifier of the Target or Initiator. The Target IQN is shown when it is
created on the Server. The initiator IQN can be found by typing a simple “iscsicli” cmd in
the command window.
Logical unit numbers (usually referred to as LUNs) allow a SAN to break its storage
down into manageable pieces, which are then assigned to one or more servers in the
14
SAN. It is a logical reference that can comprise a disk, a section of a disk, a whole disk
array, or a section of a disk array.
15
2. Add virtual disk to iSCSI server and specify the size of the virtual disk that
you want:
3. Create a New iSCSI Target, give it a name and description and select the
iSCSI initiators *** (clients) that will have access to this disk (this will allow the
Target to be discovered by defined list of initiators):
*** Initiator has to be started on the client side and have an iqn
16
4. Optional CHAP authentication can be configured for this disk
5. On the client side, open iSCSI initiator, provide the iSCSI Target server IP or
host name to the initiator, and the initiator will be able to do a discovery of the
iSCSI Target:
17
18
6. Once the connection is established, the iSCSI virtual disk will be presented to
the initiator as a disk. By default, this disk will be offline.
7. The rest of the steps are the same as for a local disk
- for iSCSI disks in Windows, you cannot use teamed network adapters, because they
are not supported with iSCSI (because you have multipath solution based on
Microsoft Multipath I/O (MPIO) which Windows Server includes one or more DSMs
as part of the operating system and also your hardware vendor will usually supply
an MPIO device-specific module (DSM) for your hardware)
- Jumbo frames of 9000 bytes are recommended to be activated end to end on the
iSCSI network
Extra references
https://blogs.technet.microsoft.com/filecab/2012/05/21/introduction-of-iscsi-target-in-
windows-server-2012/
Resizing partitions/volumes
You can add space to a volume without having to back up or reboot, if the volume is a
simple volume and there is free space at the end of the volume (including the boot
volume)!!!
19
2. choose the amount to extend with:
20
Check disk tool: chkdsk
- scans the volumes for filesystem errors and physical errors like bad sectors
- depending on the size of your volume it can take from 5 min to couple of
hours
- can be started from GUI or cmd:
A simple chkdsk Drive_letter: will only check the error but won’t do any corrections. If
chkdsk /f Drive_letter is used the volume will be dismounted (if is the OS partition will
21
request you to schedule at next reboot as the OS partition cannot be dismounted with
disrupting the OS) and errors corrected.
There is an error event logged in System event log when a volume needs a chkdsk,
otherwise the filesystem may become corrupted:
Memory dump
- Page file has to be set up on the boot volume and has to be the size of the
physical memory installed on the server, otherwise the Kernel Dump file
might not be generated when a crash happens
22
Page file
Windows memory management is based on Virtual Memory, where each process has its
own private virtual address space. Windows will move the least used memory pages to a
hidden file called the page file, when approaching a low memory condition.
The Page file is a special file used by Windows to store modified pages, and the process
of moving pages from RAM to the Page file is called "Paging".
Extra references
Understanding Crash Dump Files:
https://blogs.technet.microsoft.com/askperf/2008/01/08/understanding-crash-dump-files/
24
Looking at Effective NTFS Permissions of a user/group for a file/folder:
25
The owner of a file/folder controls how permissions are set on the object and to whom
permissions are granted. If for some reason you have been denied access to a file or
folder, you need to reset the permissions by taking ownership of the file or folder and
modifying the permissions.
Encrypting File System (EFS) is a core file encryption technology used to store
encrypted files on NTFS file system volumes. Encrypted files cannot be used unless a
user has access to the keys required to decrypt the information.
After a file has been encrypted, you do not have to manually decrypt that file before you
can use it. Rather, once you encrypt a file or folder, you can work with that file or folder
just as you would with any other file or folder.
EFS can be used to encrypt individual folders or files and BitLocker can be used to
encrypt entire volumes.
Sharing files/folders – for files to be access over network they need to be Shared and
assigned Share permissions along with NTFS permissions.
CIFS and SMB are Windows file sharing protocols (CIFS being the latest version of
SMB).
NFS is traditionally a Unix file sharing protocol but now Windows Server supports it
natively.
SMB/CIFS uses Windows-style access control lists, whereas NFS uses Unix-style file
permissions (User ID owner, Group ID owner, and read/write/execute permissions).
26
Do not grant Everyone Share permissions as it can turn into a vulnerability, but use
specific users/groups to grant Share permissions!!!
27
How to map a shared folder from a remote server/client, once Share and NTFS
permissions have been granted:
28
To list mapped network drives from cmd:
You have to have Administrator rights on the remote server to be able to access the $
shares:
30
Delete files/folders
Volume Shadow Copies - allows to restore previous states of the entire volume, you can’t
restore previous states of single files and/or folders.
31
32
Microsoft suggests to use a dedicated drive to store Volume Shadow Copies in case of high-
load.
33
To restore a previous snapshot just select it and click Revert:
34
Once Shadow Copies enabled for a drive, contents of folders and subfolders can be restored as
well to a previous version:
35
Enable Quota on a Volume
36
Enable Quota on file shares:
Soft quotas are useful for monitoring disk space use. Quotas are commonly applied using
quota templates, which are a mechanism for easily applying the same quota settings to one or
more folders.
37
Extra references:
Configure folders quota and templates
https://www.red-gate.com/simple-talk/sysadmin/exchange/implementing-windows-server-2008-
file-system-quotas/
38