College of Technology - Staff & Faculty File Server Project

Mark Stanislav College of Technology Eastern Michigan University July 2007
Abstract This document represents all relevant information regarding the College of Technology’s Staff & Faculty file server as well as the back-up/monitoring server. This is not an exhaustive documentation of every last aspect of the servers involved in this project, but it is comprehensive. The College of Technology in late 2006 under the guidance of Markus Shane Davis found the need to replace a mixture of under-maintained and mostly insecure Windows servers with a single UNIX-like system. The project was to examine the problems with the current implementation and greatly improve the technology currently being used. Notably, this project actually utilizes two servers. One server is the actual file server that is being accessed day-to-day by the employees. The other server is tasked with providing back-up storage as well as system monitoring for the production server using software such as Nagios & Cacti. You can certainly run only the file server and still have a functioning situation, but it is not recommended as part of the strength of this configuration is file remote redundancy and ensured uptime. Portions of the document may need to be updated in regards to distribution versioning, package installation methods, or network addressing. Such updates should be done as soon as possible, with a log to show changes with attribution to the person making the amendment.





General Server Configurations
File Server Dell PowerEdge 6650 FreeBSD 6.2-Stable 2 x 2.0GHz Intel Xeon 3.5GB 340GB Hardware 2xGbit Backup Server Dell Dimension FreeBSD 6.2-Stable 1 x 3.0GHz P4 2.0GB 300GB None Configured 1x100Mbit

Machine Operating System Processors Memory Hard Drive Capacity RAID Configuration Networking Hostname IP Address


File Server Details

This machine was previously a Windows-based file server for the College of Technology that we re-commissioned for the purpose of this project. This machine works well as a file server because of its hardware RAID array accepting up to eight drives, each with a capacity of 73.4GB. Currently, seven of these drive bays are in use. The additional bay could be used in the future for a hot-swap drive that sits offline until needed. From this, we have generated two separate RAID arrays. This was done for logical simplicity, as well as filesystem management and integrity. RAID Array #1 • Uses two drives, totaling approximately 70GB • Configured as RAID-1 • Contains OS installation and a 30GB slice for on-server backups (/backup) RAID Array #2 • Uses five drives, totaling approximately 270GB • Configured as RAID-5 • Contains entire space for user file storage (/storage) The extra Gbit ethernet card in the server has been left available for future expansion and networking. No IP address has been allocated for it, so be sure to acquire one from the proper channels if you plan to use a non-internal IP address.



Backup Server Details

This machine is a classical tower-style machine that overly suffices the needs of a back-up and monitoring server. Nothing extremely unique as far as the configuration, other than that there are two individual hard drives creating the overall filesystem structure. This could potentially lead to an issue as there is no symmetry in the drives and no RAID is configured. However, for the purpose of what this box does, we are confident in the implementation as is. A hardware failure will have to be addressed at that time.


Software Overview

The following is a quick overview of software installed on each server with its respective purpose mentioned. This list is in no way exhaustive, but does give a fair showing of what was originally installed. The major version number may be shown, only to reference what tree is being followed at the time of writing. Please be advised that any future tree changes (such as with Apache) needs to be carefully tested in a similar configuration as it may affect some of the core services.


File Server

The major portion of this software relates to the web-based file manager. Gollem is a project built in the Horde Framework, and officially released from the same developers. It has been customized for simplicity and design. In order for this software to function (which is mandatory), we must have Apache installed with PHP 5. Pure-FTPD and OpenLDAP also need to be installed. Software Apache Web Server (2.0.x) Megarc Tripwire Gollem Pure-FTPD OpenLDAP rSSH SSHD CVSup OpenVPN SNMPD RSync Purpose Allows for our web-based file manager to run A utility to interact with the RAID array on the server File integrity monitoring and reporting Web-based file manager for data accesss Our method to authenticate Gollem Support for LDAP authentication Provides SCP and SFTP abilities to users, without shell access Remote system access Synchronizes system software and ports tree Securely connect between the file server and backup server Allows our backup server to monitor the file server’s metrics File synching for backups



Backup Server

The backup server is very similar to the actual file server in that it needs Apache to function appropriately. The main services on this server are Nagios and Cacti. Both are web based projects, so there is reliance on Apache’s uptime. Also however, Cacti uses MySQL for a backend and must be present and functioning. Software Apache Web Server (2.0.x) MySQL (5.0.x) Cacti Nagios SSHD CVSup OpenVPN RSync Purpose Allows for Cacti and Nagios to run Provides the database backend for Cacti Network and system metric graphing for monitoring Network service and system uptime monitoring Remote system access Synchronizes system software and ports tree Securely connect between the file server and backup server File synching for backups


Information Security

With faculty and staff data requiring much privacy and protection from tampering, it is essential that the file server is as secure as possible without interfering in the operation of the needed software. Many security features have been implemented through the FreeBSD kernel, system software, and also the network. As mentioned previously, the backup server also acts as a monitoring server for the file server. Each subsection of this section could potentially lead to problems if re-configured improperly. Anyone attempting to rulesets of any security feature should produce a testing box of a similar configuration, or better yet, an image of this machine. It becomes easy to lock yourself or others out of this machine by being too hasty with changes. Changes listed will be outlined in greater detail later on in this document.


Basic Security Enhancements

The following portions will be changes in the configuration of very basic functionality. While most of these changes may first appear picky, they actually have a lot of power to change the available exploitations of a server like this.



/var/tmp & /tmp Modifications

On a standard FreeBSD installation, /var/tmp and /tmp exist. Since we have fine-grained control over mounting options of our separate filesystem partition /tmp, we will symlink /var/tmp to that and create one large temporary file repository. File-system mounting options will be discussed later in this section. A key part of our /tmp security is that we enabled the ’t’ bit on our /tmp folder. This allows us to prevent users from deleting other users files. By default, in a world-readable situation, anyone could delete anyone else’s files. 3.2.2 Login Class Modifications - /etc/login.conf

One quick note is that we have changed the default password type from MD5 to Blowfish for user accounts. This change is also reflected in /etc/auth.conf. Be sure to update both if you change one or the other. We have created two new login classes: restricted and admin. All LDAP authenticated users (which is our entire user-base sans administrators) should be part of this restricted class. Any student or staff administrators of this machine should be part of the admin login class. The restricted class receives a lower priority on their processes than any administrator or system account by defining the ’priority’ parameter equal to 2. Also, the users in this class will be allowed to be idle on the system for no more than 30 minutes, with an overall session time of 60 minutes. Finally, users have a umask of 0017. This makes files readable and writable by the owner and members of the same group. Of course, users are capable within the web-based file manager to change file permissions to make certain files private from even co-workers in their department. 3.2.3 RSSH for Restricted-Class Users

At the time of writing, users in the restricted class should have no access to an actual command-line interface. However, we do want to offer them access to SCP and SFTP services. To do this easily, we use the RSSH as their login shell. This will prevent them from logging into an actual shell account, but grants them access to use SCP and SFTP. At a later time, an administrator could grant person-by-person access to the server through changing simply their shell to bash or otherwise. This is a great way to provide fine-grained system access without implementing overly complicated software. 6


Mounted Filesystems

One genuinely easy way to protect the system from unauthorized attacks is to limit the abilities of files on a mounted filesystem. Because our installation has been created on multiple partitions, we are allowed to configured major portions of our installation differently. On our installation, we have limited the file system in places that should not normally interfere with any functionality. The partitions for /backup, /home, /tmp, /var, and /storage are all configured mostly the same way. They each do not allow for executables to be ran, or set-uid files to exist on their respective partitions. In addition to those primary mounting options, the partitions of /backup, /home, and /storage also have the ability to utilize multi-label Mandatory Access Controls. At the time of writing, no labeling is currently really being implemented, but the ability should the need arise is already enabled. Editing /etc/fstab is the appropriate way to make edits to these configurations. Any changes you make will be reflected on a reboot. It is not advised to try and unmount filesystems while the system is production-live; better to reboot than corrupt data. 3.2.5 Encrypted Swap Space

One rarely implemented, but easily configured feature of FreeBSD is encrypted swap space. Once physical memory is exhausted, FreeBSD will start using part of our hard drive to store information. When this happens, sensitive contents can be kept on the hard drive and vulnerable to stealing. We’d rather not have our passwords readably cached. Once enabled, the swap partition will be encrypted with AES-256-CBC. 3.2.6 Simplistic File Security - /usr/local/sbin/

This very simple shell script executes a long list of changes to files and directories that are mostly associated with the base installation. Primary objectives of this script are to prevent global reading access and snooping of the filesystem, as well as removal of set-uid bits that are not needed. While the former task could be considered security through obscurity, this layer does a great job of frustrating a would-be attacker from spending their time trying to find interesting files. While the script is in no way exhaustive, it does do a great job. This script should execute regularly within crontab.



User dot-file Security

The .bashrc, .bash profile, and .bash logout files all have the system immutable flag enabled. This means that the user cannot do anything at all to these files. A very notable change is that we set the user’s .bash history file to append only. This allows them to not wipe their bash history file to hide any possible wrong doing or otherwise. It does however allow the file to function as normal. Note that this does not prevent them from simply switching shells, but few people committing such acts are very aware of such rare system changes until it is too late.


Kernel Security Enhancements

Changes with the kernel have either been compiled directly into the kernel, or have been altered in some way using sysctl values. Viewing /etc/sysctl.conf will shows relevant boottime changes. 3.3.1 Basic Networking Features

• We drop all packets on closed TCP and UDP ports without sending anything back to our sender. This saves on CPU time and can help aid in the making DDoS attacks less effective. (net.inet.tcp.blackhole=2 and net.inet.udp.blackhole=1) • By allowing a large amount of sockets to be allowed open, we can prevent simple DDoS attacks to our system. SYN attacks will have a harder time succeeding. (kern.ipc.somaxconn=32768) • We will drop and log ICMP redirects to help prevent and monitor possible attacks. (net.inet.icmp.drop redirect=1 and net.inet.icmp.log redirect=1) • Protect ourselves from SMURF attacks and other broadcast attacks. (net.inet.icmp.bmcastecho=0 and net.inet.icmp.maskrepl=0) • Prevent information gathering attacks by someone tracking our IP IDs leaving a router. (net.inet.ip.random id=1) • We utilize IPFW2 for our firewall needs. In addition, we also enable Dummynet for bandwidth shaping, as it is part of the IPFW facilities. We will discuss the firewall configuration later in this document. No Dummynet ruleset is used, but the feature is enabled if needed. (options IPFIREWALL and options DUMMYNET)



Miscellenous System Features

• Our Process IDs (PID) are randomly generated with a defined modulus. This is an obscurity type feature, but that doesn’t hurt anything. (kern.randompid=modulus value) • We don’t allow for core dumps as they could be used for an attacks on the system to work on exploiting software easier. (kern.coredump=0) • Prevent users from seeing other users processes, sockets, and the alike. Great to limit users from snooping on other people’s activities. (security.bsd.see other uids=0 and security.bsd.see other gids=0) • We do not allow unprivileged processes from reading the system console message buffer or to invoke system debugging primitives. (security.bsd.unprivileged read msgbuf=0 and security.bsd.unprivileged proc debug=0) • Quota support is compiled into the kernel, but not currently implemented. (options QUOTA) 3.3.3 Port ACLs

With the integration of part of the TrustedBSD framework, FreeBSD has the ability to limit to whom a port can be bound by. In example, we limit the MySQL user (uid 88) to only bind port 3306 (MySQL’s default port). By configuring our Port ACL sysctl values appropriately, we actually limit the ability of every single port on the machine to be bound by anyone except those authoritatively specified. By issuing security.mac.portacl.port high=65535 we force the administrator to alter the Port ACL ruleset to allow for any new ports that should be getting bound. All necessary ports have already been issued appropriately in the ruleset, found in /etc/sysctl.conf (security.mac.portacl.rules). 3.3.4 Security Event Audit Daemon

This is an implementation of Sun’s BSM API and file format for a security audit system. This system provides very invasive monitoring of users. The control file can allow for extremely fine-grained control of what events are logged, and for which users. Files for editing how the system works are stored in /etc/security, while the actual audit logs are stored in /var/audit. You can use the praudit utility on a logfile to view its contents.



File System Firewall

Another part of the TrustedBSD framework integration is the bsdextended module. This module provides for a file system firewall of sorts. The syntax is very similar to that of IPFW, with a very different purpose. The advantage to this is that since is part of the Mandatory Access Control side of FreeBSD, users aren’t capable of editing these permissions. The positive behind this is that even if they were to make all of their files world readable, it wouldn’t actually matter. The default ruleset that we use can be found in /etc/rc.bsdextended. Please note that this file should be edited very carefully. As is, a script does most of the work, but you can add additional rules for refined security if needed. 3.3.6 Process Accounting

Through the process accounting facility, we are able to easily monitor who is issuing what commands to our system, and what resources those processes took up. The process accounting database is saved as /var/accounting/acct and can be viewed using lastcomm and sa. It is advisable to read the man pages for both commands to get the best out of this data. Process accounting is a great way to track a user’s activity or to help find where an attack may have originated. The databases are automatically rotated, so space shouldn’t be an issue. It would be wise to occasionally monitor what commands a local user is executing if they are not a known administrator. 3.3.7 MAC Paritioning, MLS, and Biba Modules

For a full Mandatory Access Control configuration, FreeBSD provides three modules to handle almost anyone’s paranoid needs. These are all available to the system but are disabled due to the apparent overkill factor for this server. While we value the privacy of every person’s files, the privacy of each file doesn’t warrant such overbearing restrictions from what we already have. If our system had more of a logical hierarchy to the staffing, with obvious lines in power for multiple people or groups, the usefulness of these would be greatly increased. Be advised if you plan on implementing these that you can easily lock yourself out of the system very easily. Also, make sure you have a real purpose to configure all of the settings, as it can get quite involved depending on your userbase.



Security/Monitoring Software
Tripwire File Integrity Monitoring

Tripwire is a software package that allows you to create a list of files that you want to monitor for changes. This is helpful to make sure critical system files aren’t being altered accidently or by another person. Reports can be generated and the output is easily readable to see what is occurring on the system. By default, a report is generated and e-mail nightly to the person listed in the file /usr/local/etc/tripwire/twcfg.txt under the parameter ’GLOBALEMAIL’. Anytime major changes occur to this file and the report shows a lot of changes, the policy database should be updated so that you can easily parse through future changes. 3.4.2 Cacti

Cacti provides us with constant monitoring of our file server for a number of metrics. Each metric is graphed multiple times an hour to provide an accurate picture of what is happening with it. As of now, we are currently monitoring each hard drive partition individually, load average, logged in users, ping latency, processes, and network traffic. Obviously each one of these metrics serves a different usefulness for monitoring the security of our server. For example, if all of the sudden we see three users logged in when there are only supposed to be two allowed accounts, we may want to see if there is a dupe login or a rogue user. Network traffic is always good to measure, especially for a file server. 3.4.3 Nagios

This services provides us another web interface to help monitor the availability of our file server. Nagios will allow us to get alerts via e-mail or otherwise if there is an unexpected downtime to our file server. More so, we can monitor individual services to make sure they are available at all times. This is especially needed for our web server as it is the crux of the file server for end users. 3.4.4 OpenVPN

To establish a cryptographically secure connection between our file server and backup machines, we will use OpenVPN. This allows us to transmit anything between the servers and have guaranteed security while doing so. This point-to-point link is established as a local private network as far as IP addressing is concerned. 11


File Server Re-Installation Guide

The following is a very comprehensive set of steps to give an idea of how the server was initially configured. This isn’t a 100% step-by-step guide, but it is pretty close. If at anytime a re-installation is needed, follow the guide as closely as possible, fixing errors where they arise and document them. As FreeBSD and its package set changes, so will the types of problems that can occur during what used to be a quick installation. IMPORTANT: You will note in a number of steps that files are copied from /files/. As of this writing, there should be an attached tar file with this documentation, most likely on a CD-ROM. If you do not have these files, the installation will be much less smooth. If you do re-install, please backup the files, if not more. Less configuration can be the difference between a 2-hour setup and a 20-hour setup.


Installation of FreeBSD

Obviously if you are following this document, you have some idea how to install FreeBSD on the computer already. This part of the documentation will merely list a number of notes for the initial installation to make sure things are in decent order on first boot. These may not necessarily be in the correct order, so read them over before beginning your installation. 4.1.1 Initial Setup Questions

• Do not install a boot loader since we only are booting one OS on this server • Install a full set of sources to ensure less downloading time to sync for re-building world, use your best discretion which set to take • Select ”YES” when prompted if we want the ports system installed, but do not actually install any ports until after we reboot • Enable the SSH daemon, do NOT enable the INET deamon • Do NOT act as a network gateway • Select ”YES” when prompted if we want Linux Compatability • Create the ’admins’ group • Create a user (not your my.emich account) and add it into the admins and wheel groups



File System Partitioning

That should get you through almost everything except partitioning of the file system. This is a very important part since it relates directly to our system security as well as overall functionality. Sizes are approximate, so when creating your last partition (should be /storage), just give it all of the rest of the space available. Take time to make sure you edit the proper of the two arrays for each step. I *highly* recommend you keep the array structure the same or similar to what was established earlier in the document. Of course, any changes to the logical RAID arrays would have had to of been done prior to installing FreeBSD. Partition / SWAP /backup /home /tmp /usr /var /storage 4.1.3 Size 5GB 2GB 30GB 3GB 2GB 15GB 10GB 245GB Reason Most files will be inside of other partitions, not much is needed Most likely overkill for our physical RAM, but worth having Available if the need arises for local backups just-in-case Majority of accounts will be in /storage, only for local admin accounts Overkill again most likely, but we have the space Bulk of our installations of software will be here, extra space is good A lot of logging and e-mail storage is needed Our file server’s primary data storage space

Network Configuration

Choose to setup the network information manually, as you need to configure a static IP address. Use the assigned IP address for the server at the time you are configuring it. Be sure to setup the fully-qualified domain name (FQDN) to ensure smooth setups of network services. Also be sure to use a current DNS server when configuring the name server in the initial setup.



User Shells: Installation and Setup

First we need to install both Bash and RSSH as they are the two choices for accounts in this system. Remember, all admins should run bash, all file server users should use rssh unless there is a specific need for shell access. # cd /usr/ports/shells/bash && make install clean # cd /usr/ports/shells/rssh && make install clean # cp -f /files/rssh.conf /usr/local/etc/ Next we will copy over a bunch of bash & vim dot-files. # cp -f /files/dot.bash history /root/.bash history # cp -f /files/dot.bashrc /root/.bashrc # cp -f /files/dot.bash profile /root/.bash profile # cp -f /files/dot.bash logout /root/.bash logout # cp -f /files/dot.vimrc /root/.vimrc # # # # # cp cp cp cp cp -f -f -f -f -f /files/dot.bash history /home/myUser/.bash history /files/dot.bashrc /home/myUser/.bashrc /files/dot.bash profile /home/myUser/.bash profile /files/dot.bash logout /home/myUser/.bash logout /files/dot.vimrc /home/myUser/.vimrc

After the files are copied, we will set appropriate permissions to insecure their security. # chown myUser:admins /home/myUser/.bash* # chown root:wheel /root/.bash* # # # # chflags chflags chflags chflags sappnd /root/.bash history /home/myUser/.bash history schg /root/.bash profile /home/myUser/.bash profile schg /root/.bashrc /home/myUser/.bashrc schg /root/.bash logout /home/myUser/.bash logout

Lastly, we will change our root and myUser accounts to use the bash shell. Once we have done this, we will reboot. Rebooting in this process is helpful because it’s a good way to make sure you don’t get too far with critical mistakes. # pw usermod myUser -s /usr/local/bin/bash # pw usermod root -s /usr/local/bin/bash # reboot



Quick Housecleaning

Below we will clean out some undesired files, create a useful symlink for updating our file location database and delete the toor user. Of importance also, we are going to symlink /var/tmp to /tmp and add our /tmp file security feature that was mentioned previously. NOTE: Be sure to alter these commands to fit your current situation. Change ’myUser’ to the unprivileged user account that you created initially. # cd /root && rm -f .cshrc .login* .mail* .profile .rhosts .k5login # cd /home/myUser && rm -f .cshrc .login* .mail* .profile .rhosts # rm -f /.cshrc /.profile # ln -s /usr/libexec/locate.updatedb /usr/sbin/updatedb # /usr/sbin/updatedb # pw userdel toor # # # # mv /var/tmp/* /tmp/ rm -rf /var/tmp ln -s /tmp /var/tmp chmod +t /tmp


Adduser Configuration

For our addition of users to go more smoothly, we pre-configure a few portions of our system. We will add the ’restricted’ user group into /etc/group, copy our adapted adduser.conf to /etc/, as well as priming /etc/skel with user files and permissions. # echo “restricted:*:1002:” >>/etc/group # cp -f /files/adduser.conf /etc/ # cp -f /files/dot.* /etc/skel/ # # # # chflags chflags chflags chflags sappnd /etc/skel/dot.bash history schg /etc/skel/dot.bash profile schg /etc/skel/dot.bashrc schg /etc/skel/dot.bash logout



Logging Configuration

We have some custom logging demands for our server, as well as files that need to be created to do so. Also, there are some log files that are not needed and we will delete them in the interest of directory cleanliness. # cp -f /files/newsyslog.conf /etc/ # cp -f /files/syslog.conf /etc/ # touch /var/log/ipfw /var/log/console.log # rm -f /var/log/pflog /var/log/amd.log /var/log/all.log # rm -f /var/log/lpd-errs /var/log/slip.log /var/log/ppp.log # rm -f /var/log/kerberos.log /var/log/


Configuring /etc/fstab

Since your /etc/fstab may have different partitions then what we originally setup, it is better to explain what changes should be made to your /etc/fstab file and why. Adapt these changes to your partitions and be sure to type them properly so that you don’t lock yourself out. These changes are primarily for security, but a couple are just functionality related (like quota and proc support). Mountpoint / /backup /home /tmp /usr /var /storage Options rw rw,noexec,nosuid rw,noexec,nosuid rw,noexec,nosuid rw rw,noexec,nosuid rw,noatime,noexec,nosuid,userquota,groupquota

Also though, we want to add two lines for proc and swap. For the encrypted swap partition, at the end of the device name, add ’.eli’ (e.g. /dev/amrd0s1b.eli). For the proc filesystem add a line like: proc /proc procfs rw 0 0



User Classes & Password Security

One of the important portions of our user configuration is to separate each user into a different login class. Also, we configure users to have passwords with the Blowfish cipher. Regarding the user base we will mostly have, they never have local passwords stored, so this is only for administrators. We have to reset our local account passwords in order for this change to be accepted. # cp -f /files/auth.conf /etc/ # cp -f /files/login.conf /etc/login.conf # cap mkdb /etc/login.conf # passwd myUser # passwd root


Local Mail Configuration

For our cron job e-mails and system reports to be sent to an appropriate e-mail address, we are going to forward all e-mail bound for the root user to an external e-mail address. Also, we want to make sure mail is properly handled for our host. Edit /etc/mail/aliases and where you find “root: ”, set it to an e-mail address (e.g. root: After doing that, you must rebuild the e-mail alias database. # cd /etc/mail && make Lastly, edit /etc/mail/local-host-names (the file doesn’t exist by default) and type each hostname that the server should handle e-mail for on a new line.


IPFW Configuration

IPFW is an important configuration step because it grants us a lot of ability to filter out unwanted traffic to and from our server). With a properly configured firewall you can minimize a lot of basic security threats and add another layer of security to the overall model. The included IPFW ruleset should be perfect for what the file server is designed to do, but you will of course want to verify IPs and rules before launching it live. # cp -f /files/ipfw.rules /etc/



Quota Support

In addition to our kernel and /etc/fstab changes to support user and group quotas, we also want to run a few commands. To administrate quotas easier, we will install the setquota package from ports. # cd /usr/ports/sysutils/setquota/ && make install clean # quotacheck -a # quotaon -a


Miscellaneous Configuration Files

Since not every configuration file needs its own step, we will lump a lot of them together. Please go ahead and read over each one if you have any questions as to why. Also, check for consistency in things like IP addresses and hostnames. Also, we want to quickly remove some default sample templates for applications. You may want to run this second step again after all of your software installations. # cp -f /files/rc.conf /etc/ # cp -f /files/make.conf /etc/ # cp -f /files/sysctl.conf /etc/ # cp -f /files/motd /etc/ # cp -f /files/banner /etc/ # cp -f /files/ttys /etc/ # cp -f /files/crontab /etc/ # cp -f /files/rc.bsdextended /etc/ # cp -f /files/hosts /etc/ # cp -f /files/shells /etc/ # cp -f /files/loader.conf /boot/ # # # # rm rm rm rm -f -f -f -f /usr/local/etc/*.sample /usr/local/etc/*-dist /usr/local/etc/*-recommended /usr/local/etc/*.default



Software Installations

Obviously we have already done some software installations, and we will do others later. However, we will try and organize things some by separating out the majority of our software installations, with some basic notes. 4.12.1 OpenSSL & OpenSSH Portable

The portable versions of OpenSSL and OpenSSH are easier to maintain than the built-in versions that ship with FreeBSD. The installation for these is really quick. # cd /usr/ports/security/openssh-portable # make -DOPENSSH OVERWRITE BASE install clean # cp -f /files/sshd config /etc/ssh/sshd config # /etc/rc.d/sshd restart # cd /usr/ports/security/openssl # make -DOPENSSL OVERWRITE BASE install clean



We are using OpenVPN for our secure communications across the unsecured network. The configuration must account for both our servers in order to have a purpose. The backup server will act as the server for the VPN, and the file server will act as the client. # cd /usr/ports/security/openvpn && make install # mkdir /usr/local/etc/openvpn



For our web file manager Gollem, we are going to authenticate users through Pure-FTPD. Not only can Pure-FTPD give us LDAP authentication as a mechanism, but it also gives us some great security features. It is possible to run Gollem via LDAP directly, but after testing, I would recommend using Pure-FTPD. # cd /usr/ports/ftp/pure-ftpd/ && make install clean # cp -f /files/pure-ftpd.conf /usr/local/etc/



LDAP Authentication

LDAP is the authentication method of choice for all staff and faculty accessing the server. We are authenticating them via their my.emich username and password. Each account utilizing this type of authentication must have a basic shell template already established, otherwise they can’t login. # cd /usr/ports/net/openldap23-client/ && make install clean # cd /usr/ports/net/nss ldap/ && make install clean # cd /usr/ports/security/pam ldap/ && make install clean # cp -f /files/pam.d/* /etc/pam.d/ # cp -R /files/openldap /usr/local/etc/ # cp -f /files/ldap.conf /usr/local/etc/ # cp -f /files/nss ldap.conf /usr/local/etc/



CVSup is an easy way to synchronize your system and port sources very easily. We will install the CVSup port, create a directory for the application to use, and copy our config files into that directory. # cd /usr/ports/net/cvsup-without-gui && make install clean # mkdir /var/db/sup # cp -f /files/cvs-supfile /var/db/sup/ # cp -f /files/refuse /var/db/sup/



For our backup server to monitor information about our server, we want to run SNMP to provide a standardized way for that collection to occur. SNMP will allow for many metrics to pass their information on request to our backup server to use as desired by our configuration settings. # cd /usr/ports/net-mgmt/net-snmp && make install clean # cp -f /files/snmpd.conf /usr/local/share/snmp/



Apache 2.0.x & SSL

As of this writing, there have been issues running Horde & Gollem on anything but Apache 2.0.x. Be sure to test on another server if you plan to upgrade Apache, or even Gollem for that matter. # cd /usr/ports/www/apache20/ && make install clean # cp -f /files/httpd.conf /usr/local/etc/apache/ # cp -f /files/ssl.conf /usr/local/etc/apache/ # cp -f /files/cotdrive-cert.pem /etc/ssl/crt/ # cp -f /files/cotdrive-key.pem /etc/ssl/key/ # rm -f /usr/local/www/data/index.* # rm -f /usr/local/www/data/apache pb*


PHP 5 & Horde/Gollem

Horde is a PHP framework, and PHP5 is the appropriate choice as of this writing. These steps will get PHP installed, copy over our php.ini into the correct directory, install PEAR::DB, and give you a selection of PHP 5 extensions to install. Use some judgement in what you will need. Things like Session support, MySQL support, SSL support, and the alike are definitely needed. # cd /usr/ports/lang/php5/ && make install clean # cp -f /usr/local/etc/php.ini-recommended /usr/local/etc/php.ini # cp -f /files/php.ini /usr/local/etc/php.ini # cd /usr/ports/databases/pear-DB && make install clean # pear channel-update # pear upgrade-all # cd /usr/ports/lang/php5-extensions && make install clean # cp -Rf /files/horde /usr/local/www/




Tripwire will monitor system files that should not be changed, or that we want to known when changes have been made. It is a great way to monitor things daily on a server for changes. # cd /usr/ports/security/tripwire/ && make install clean # cp -f /files/twcfg.txt /usr/local/etc/tripwire/ # cp -f /files/twpol.txt /usr/local/etc/tripwire/ # cp -f /files/site.key /usr/local/etc/tripwire/ # twadmin –create-cfgfile –site-keyfile site.key twcfg.txt # twadmin –create-polfile twpol.txt # tripwire –init # tripwire –update-policy twpol.txt



In regards to monitoring services and system availability, we use Nagios. The setup to get the process going is quick, but configuration can be time-consuming. This is again only meant for the backup server to run. Once installation is done, create htaccess and htpasswd files for authentication. Also verify Nagios is enabled inside of /etc/rc.conf. # cd /usr/ports/net-mgmt/nagios && make install clean # cp /files/nagios/* /usr/local/etc/nagios/ # chmod 644 /usr/local/etc/nagios/* # cp /files/httpd-nagios.conf /usr/local/etc/apache2/Includes/ # chmod 644 /usr/local/etc/apache2/Includes/*




On our backup server we need to setup Cacti for our graphing of the file server’s system metrics like memory usage, network usage, and system load. This requires MySQL to be used, so make sure that is setup on the backup server first. After the installation is done, the username and password is admin/admin. Login, change the password and set a font path. # cd /usr/ports/net/rrdtool && make install clean # cd /usr/ports/net/cacti && make install clean # cd /usr/ports/net/cactid && make install clean # mysqladmin –user=root -p create cacti # echo “GRANT ALL PRIVILEGES ON cacti.* TO cacti@localhost IDENTIFIED BY ’a password’;” — mysql -uroot -p # mysql -ucacti -p cacti ¡ /usr/local/share/cacti/cacti.sql # cp /files/db-settings.php /usr/local/share/cacti/include/ # cp /files/httpd-cacti.conf /usr/local/etc/apache2/Includes/ # chmod -R 655 /usr/local/share/cacti/ We need a cronjob task added for Cacti to properly process data. Add to your /etc/crontab: */5 * * * * cacti php /usr/local/share/cacti/poller.php >/dev/null 2>&1


Archiving Utilities

These are a list of ports to install for archiving/unarchiving utilities that are good to have handy. # cd /usr/ports/archivers/bzip2/ && make install clean # cd /usr/ports/archivers/rar && make install clean # cd /usr/ports/archivers/unrar && make install clean # cd /usr/ports/archivers/unzip && make install clean # cd /usr/ports/archivers/zip && make install clean # cd /usr/ports/archivers/untar && make install clean



Miscellaneous Software

These are a list of ports to install for archiving/unarchiving utilities that are good to have handy. # cd /usr/ports/editors/vim-lite && make install clean # cd /usr/ports/editors/nano && make install clean # cd /usr/ports/www/lynx-ssl && make install clean # cd /usr/ports/ftp/wget && make install clean # # # # cd cd cd cd /usr/ports/sysutils/megarc && make install clean /usr/ports/sysutils/lsof && make install clean /usr/ports/sysutils/screen && make install clean /usr/ports/devel/strace && make install clean

# cd /usr/ports/net/netcat && make install clean # cd /usr/ports/net/rsync && make install clean



How to Maintain the Server

After you have configured your system appropriately, before setting it for production use, you will want to recompile the base system as well as upgrading all port software. More so, you will want to execute these tasks on an interval of your choosing. It would be ideal if you could schedule downtime every three months to keep the system adequately safe (under general circumstances) from vulnerability. Of course, there may be a time when a security issue presents its self and an immediate upgrade may be appropriate. It is worth mentioning that you could synchronize your sources daily if you really wanted. That would minimize the amount that you would have to download in case of an emergency recompile of the system. Just because you download the sources does not mean you need to compile the system.


CVS Synchronization

The initial step in keeping your system software up to date is to synchronize all of your sources. When using CVSup you will gather the source updates for the system tree that you are following. Look over /var/db/sup/cvs-supfile for the options you have in configuring this task. NOTE: If you receive a rejection note from the CVS server, try editing the cvs-supfile with a different number in the server name. # cvsup -g -L 2 /var/db/sup/cvs-supfile


Building World

In FreeBSD, the task of recompiling all of your base system software is called ’Building World’. Once your sources have been updated, this task will run through all of your sources, compiling where appropriate and preparing the contents of your updated system. At this point, nothing is being overwritten. NOTE: If you receive any errors during this process, do not just ignore them. Building world should be an error-free process. # cd /usr/src # make buildworld



Compiling a New Kernel

Your kernel is the cornerstone of the entire Operating System. If your kernel is improperly configured, your system will most likely fail on the next reboot. The included configuration file for the kernel will be copied over to the appropriate spot in the first step. Depending on what version of FreeBSD is out when you are compiling, you may want to read up on any changes between FreeBSD 6.2 and today’s version. # cp -f /files/ /usr/src/sys/i386/conf/ # cd /usr/src # make buildkernel KERNCONF=HOSTNAME # make installkernel KERNCONF=HOSTNAME


Install World

Pending your kernel compiled and installed properly, the next step before you boot into your freshly compiled system is to install all of the software we previously compiled. NOTE: When you install world, you must make sure that /tmp is configured without noexec & nosuid mounting options. If you have to, change the options for the mount point in /etc/fstab, and reboot. # cd /usr/src # make installworld


Final Steps to Updating the System

If you have completed each of these steps, you can now reboot into the new system. Hopefully everything is working properly and you can keep the process moving. Your next step is to verify all of your software is up and running appropriately. Once you do, move on to updating the files in /etc. 5.5.1 Mergemaster the /etc directory

When updating our system, we have to integrate changes to the /etc directory. While using mergemaster, carefully select to either install the newly downloaded file (press ’i’) or delete the new file and keep our existing file (press ’d’). If you pay attention and go slow, the process should be simple and not ruin anything. # mergemaster -cv -w 120



Remove Build Leftovers

In the interest of saving space and keeping our compilation directories clean for the next compilation of world, we will go ahead and remove old files that are no longer needed. Be sure to type these commands probably as you could potentially ruin something if you misplaced them ’rm -rf *’ command. # cd /usr/obj # chflags -R noschg * # rm -rf *


Port Upgrades

In relation to us updating all of our sources on the system, we also received updates to the ports tree. Because of this, we are able to update the port installed software on our system. If this is your first run through, be sure to actually install the portupgrade utility. Else, just follow the steps for updating the ports database and executing portupgrade. NOTE: The portsdb command can take a while to execute, so be patient! After the ports database updates, go ahead and start updating software using the portupgrade command. # cd /usr/ports/ports-mgmt/portupgrade && make install clean # portsdb -Uu # portupgrade -aiRr




This section is dedicated to some notes on a few tasks that may need a small bit of explanation. These notes are possibly not fully complete, but are a great starting point for completing the task. Use them loosely to get an idea of what to do, and adapt as needed. It wouldn’t hurt if this document was updated regularly with new How-Tos so that the usefulness of this section continues to grow.


Add/Remove LDAP Users

Every staff or faculty member that we want to authenticate via LDAP, we need them to have a template of an account present in our password database. Even though we don’t store a password locally, we still have to have them created as a user. Without this step, LDAP users will not be able to login or do anything else. To add a user for LDAP usage 1. Type: adduser 2. Type the my.emich username of the account you want added 3. Type the full name of the user you are adding (as best as you know it) 4. Leave the UID empty (just press enter) 5. The default login group for any staff or faculty is just staff, so press enter 6. If you have groups created and know the user is supposed to be added, type the group name you want to add the user to 7. The login class for all staff and faculty is restricted, so just press enter 8. The default shell for every staff and faculty is rssh, so just press enter 9. The directory that you are given is correct, so just press enter 10. We want password-based authentication, so just press enter 11. We do want an empty password. This is necessary for LDAP authentication to be used. Just press enter 12. We do not want to lock out the account after creation, so just press enter 13. Verify all of the information is correct and then type ’yes’ 14. If you want to add another user, type ’yes’. Else, type ’no’ 15. Verify that the user was created successfully by checking /storage/ for their account 28

To remove a user 1. Be sure any files that the user has that may be needed by someone else in the department or on campus is backed up securely 2. Type: chflags noschg /storage/user name/.bash* 3. Type: chflags nosappnd /storage/user name/.bash* 4. Type: rmuser user name 5. Verify that the record you are deleting is the correct one 6. Type ’yes’ if you want to remove that record for sure 7. Type ’yes’ to delete the user’s home directory (remember, verify you don’t need their files first) 8. Edit /etc/group with your favorite text editor and remove the username you just deleted from any groups for cleanliness


Add/Remove User Groups

For each user account that we add, we want to have at least one user group to add them into. By default, every user is going to be placed into the ’staff’ user group as it gives us a standard place to have people. When editing user groups, you won’t actually see users in the staff group, but they are. User groups are found in /etc/group and are easily edited with any text editor. To add a user group 1. Open /etc/group in a text editor of your choice 2. Go to the bottom of the list of groups and find the last user group number in the list (e.g. 1008) 3. Go to the next empty line and copy & paste the previous group template and change the name of the group (the first parameter) and increment the user group number (third parameter) 4. Verify that your new user group is not a dupe of another group, not a dupe of a group number, and has the correct syntax 5. Add users you want in that group after the last colon, and add each additional one with a comma after the previous 6. Save /etc/group


To delete a user group 1. Open /etc/group in a text editor of your choice 2. Go to the line of the file for the user group you want to delete 3. Delete the line, end-to-end 4. Save /etc/group


Add/Remove Department Share

Since our server hosts multiple departments at the College of Technology, we want to be able to share files between members of each area. To do so, we have group shares that are based on the department that the users are in. Be careful removing a department share as it could contain a lot of valuable information to people other than the one who requested you delete it. More often than not, a department share should never be removed, unless the department goes away or the name of the department changes. Either way, exercise great caution with this step. To add a departmental group share 1. Type: ls -la /storage/shares and verify that your desired share doesn’t already exist 2. Type: mkdir /storage/shares/desired share 3. Type: chmod 770 /storage/shares/desired share 4. Type: chown root:department user group /storage/shares/desired share 5. You now have to create a symlink to each user that needs the department share access 6. Type: ln -s /storage/shares/desired share /storage/user account/ 7. Type: chown user account:staff /storage/user account/desired share 8. Type: chmod 770 /storage/user account/desired share To delete a departmental group share 1. Verify that you backup any files in the directory share you want to delete 2. Type: rm -rf /storage/shares/desired share/ (Be very careful) 3. Type: rm -f /storage/*/desired share (Be very careful)



Enabling Multi-Label Security on Your Partitions

If you decide to use MLS as part of your security plan on the new server, you will need to enable it on each partition that you want to use it on. For our purposes, you should only be enabling MLS on /, /storage, /home, and /backup. 1. Edit /etc/fstab and set / to ro 2. Reboot the machine 3. Type 4 at the menu screen (Single User Mode) 4. Type: /sbin/tunefs -l enable / 5. Reboot again 6. # Type: mount -urw / 7. Edit /etc/fstab and set / to rw 8. # Type: shutdown now 9. # Type: umount /storage && tunefs -l enable /storage 10. # Type: umount /home && tunefs -l enable /home 11. # Type: umount /backup && tunefs -l enable /backup 12. # Reboot a final time


Setting a “Secure Level”

While it would be ideal to user the FreeBSD secure levels, we would have to change some of how we do things on the system. As of this time, the secure level is at your standard Insecure Mode. This is because we use the system immutable and append-only flags on our system with user accounts. Unfortunately, in order to delete a user properly, we have to remove these flags from their bash files. Secure Level Alterations 1. Verify that either you are fine with some user files not being able to get delete or you are not settings schg or appnd flags on user bash files 2. Type: sysctl kern.securelevel=your number (Recommended: 3) 3. Add a line into /etc/sysctl.conf for this setting



Create an SSL Certificate For Apache

You must be running SSL on Apache when using Gollem. Do not skip this step as people’s usernames and passwords will be easily vulnerable to being sniffed. If for some reason you have to create a new SSL certificate for the server, do so via this method. # cp -f /files/openssl.cnf /etc/ssl/ # mkdir -p /root/ssl cert/certs /root/ssl cert/private /root/ssl cert/newcerts # touch /root/ssl cert/index.txt # openssl req -new -nodes -out cotdrive-req.pem -keyout private/cotdrivekey.pem -config /etc/ssl/openssl.cnf # openssl req -new -x509 -days 3650 -extensions v3 ca -keyout private/cakey.pem -out cacert.pem -config /etc/ssl/openssl.cnf # openssl ca -config /etc/ssl/openssl.cnf -out cotdrive-cert.pem -days 3650 -infiles cotdrive-req.pem # mkdir /etc/ssl/crt # mkdir /etc/ssl/key # cp -f /root/ssl cert/cotdrive-cert.pem /etc/ssl/crt # cp -f /root/ssl cert/private/cotdrive-key.pem /etc/ssl/key


Master your semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master your semester with Scribd & The New York Times

Cancel anytime.