RedHatLinuxNetworking

FundamentalNetworkConfigurationandTroubleshooting
PeteNesbitt
March2006

TheinformationpresentedshouldactasaguidetoRedHatLinuxnetworking.Itisintendedtobe
accompaniedwithtrainingandselfstudy.Toaccessmostoftheseitemsyouwillneedtohaveroot
access,eitherdirectlyorthrough'sudosu',orviasudoexploitswewon'tdiscuss:)Someitemsare
leftforyoutoreviewwhileothersarecoveredinmoredetailonthefollowingpages.
Thedescriptionsarebriefandaremeanttobestartingpoints.Use'man<commandorfilename>'for
helppages.Beawarethatmanpagesexpectacertainleveloftechnicalknowledgeandmayrequire
severalvisitstograsptheinformation.Makinguseofmultiplewindowswhenworkingonasystem,be
itlocalorremote,allowsyoutoviewamanpagewhiletryingthecommandoreditingthefileina
differentwindow.
Beforethisinformationcanbeuseful,youshouldbeabletousethefollowingtools:
vi
fileeditorcommonanallLinuxandUnixsystems
cat
concatenateandviewfiles
less
filepagerwithmanyfeatures
grep
extractinformationbasedonkeywords
man
use'mank<word>'tosearchmanpagesforakeyword,ortry'manman'
alwayslookatthe'seealso'sectionnearthebottom
basiccommandlinenavigation,commandexecution,andoutputmanipulation
understandenvironmentalvariables
itisvaluabletounderstandsystemvariables,althoughtheymaynotcomeintoplayoften
bashshellscripting
shouldbeabletoatleastfollowtheflowandprocessofabashscript
basicnetworkserviceclientssuchassshandftp
usetotesttolocalhost,forconfirmingifitisaserviceproblemoranetworkproblem
tee
usetosendoutputtothemonitoraswellasafileforlaterreview

1of11

RedHatLinuxNetworking

FundamentalNetworkConfigurationandTroubleshooting
PeteNesbitt
March2006

ConfigurationFiles:
/etc/hosts
usedprimarilytodefinestaticIPtohostnamemappings.
/etc/resolv.conf
definesnameserversforDNSresolution.
/etc/host.conf
definesparameters,themostcommonbeingsearchorder,fornameresolution.Theorder
keywordmayincludehosts,bind,andnis.
/etc/sysconfig/network
definesnetworksettingscommontoallinterfacessuchashostnameanddefaultgateway
/etc/sysconfig/networkscripts/
ifcfgloifcfgethn
definetheinterfaceparameterssuchasIPaddressandnetworkmask
/etc/services
definesknownportnumberstoservicenames.Commands,services,andutilitiesoften
displaytheservicenameintheiroutput.Portsundefinedin/etc/servicesshouldalwaysbe
definedbyportnumber.
/etc/rc.local
commandstorunaftertheinitscripts.Youmaywantastaticroutingentryinhereafterthe
networkissetup.
/procpseudofilesystem
allowsyoutoreadandinsomecaseswritekernelvariables.
lookin/proc/sys/net/forstarters.Filesshowzerosizebutarepopulatedwhenaccessed.
/etc/sysctl.conf
fileyoucanusetocontrol/procfileentries.Thisallowschangestosurviveareboot.
/etc/sysconfig/hwconf
informationonsystemhardwareusedforautodetectionofnewhardware
canedittoforceredetectioniftroubleshootinganetworkcard.Fileisrarelyofinterest.
/etc/modprobe.conf
nicdriversandoptionsaredefinedhere
AccessControlFiles:
2of11

RedHatLinuxNetworking

FundamentalNetworkConfigurationandTroubleshooting
PeteNesbitt
March2006

/etc/hosts.allow/etc/hosts.deny
controlsTCPwrappersutility(seetcpdbelow)
viewmanpagewith'man5hosts_access'[notethe5]
/etc/init.d/iptables(aliasof/etc/rc.d/init.d/iptables)
controlandconfigurationofIptables/NetFilterLinuxFirewall

LogFiles:
Logscanprovidevaluabletroubleshootinginformationbutaretoooftenoverlooked.
dmesgfallsintothelogscategoryandshouldbereviewed,particularlyiftherearelocalproblemsas
opposedtoremotehostproblems.
lookin/var/log/
toviewstartupmessagesuse'dmesg|less'
CommandsandUtilities:
arp
displayorsetarptableentries
dig
querynameservers
dmesg
displayringbufferinformationfromsystemstartup
ethereal
notinstalledbydefault
ethtool
host
querynameservers
hostname
displayorsethostname
changesdonotsurviveareboot
ifup
bringaninterfaceup
ifdown
shutdownaninterface
ifconfig
3of11

RedHatLinuxNetworking

FundamentalNetworkConfigurationandTroubleshooting
PeteNesbitt
March2006

vieworsetinterfaceparameters
changesdonotsurviveareboot

ip
iptables
userspacetoolstomanipulatenetfilterkernelbasedfirewall
iptraf
displaysnetworkstatistics
notinstalledbydefault
netstat
usetodisplayrouting,open(listening)ports,etc
ping
sendanicmppackettoanIPaddressorhostname
route
displayormanipulateroutingtables
changesdonotsurviveareboot
service
usetostart,stoporrestartservicesviasupportedinitscripts
tcpd(tcpwrappers)
accesscontrolforsupportedservices
tcpdump
viewnetworktraffic
telinit(andrunlevel)
usetochangerunlevels,userunleveltoviewcurrentandpreviousrunlevel
notusuallyinvolvedinnetworking
traceroute
displaystherouteandstatisticsforUDPorICMPpacketsastheytraverseanetwork.

NetworkServices:
Startingandstoppingofnetworkservicescanbeautomatedthrough:
initscripts
symlinkstovariousscriptsusedtomanagemostsystemdaemons
/etc/rc.localfile
commandstorunaftertheinitscripts.Youmaywantastaticroutingentryinhereafterthe

4of11

RedHatLinuxNetworking

FundamentalNetworkConfigurationandTroubleshooting
PeteNesbitt
March2006

networkissetup.
xinetdsystem
see/etc/xinetd.confandfilesin/etc/xinetd.d/
replacesinetdandincorporatesTCPwrapers

EnvironmentalVariables:
Thesecommandsareforthebashshell,thedefaultshellformostLinuxdistributionsincludingRed
Hat.Othershellsmayusedifferentmethods.
Viewcurrentsettings
displaywith'env'forallor'echo$<varname>'
Defineneworchangeexistingvariables
setvariableswith'<varname>=<value>'
VirtualNetworkInterfaces:
AsingleinterfacecaneasilybeconfiguredtoanswermultipleIPaddresses.VirtualInterfaces
aredefinedasethn:masaretheconfigurationfiles.(0:0isthephysicalnic,0:1andhigherare
virtual,anyuniquenumbercanbeused.
Createaconfigurationfile/etc/sysconfig/networkscripts/ifcfgethn:mthenrestartthenetwork
oruseifconfigtobringupthenewinterface.
Whenworkingwithiptablesbeawareitdoesnotdistinguishbetweenrealandvirtualinterfaces
AVirtualInterfaceconfigurationfilemaylooklike:
DEVICE=eth0:1
BOOTPROTO=static
BROADCAST=192.168.1.255
IPADDR=192.168.1.41
NETWORK=192.168.1.0
ONBOOT=yes
TroubleshootingSteps:
Theseareverygeneralstepsusedforisolatingbasicconnectivityproblems.
ping127.0.0.1
iffailhere,networkservicemaynotberunning.Startnetwork,checklogsandrunlevel
ping<localhostipaddr>
iffails,confirmIPaddressandinterfacestatewithifconfigcommand
5of11

RedHatLinuxNetworking

FundamentalNetworkConfigurationandTroubleshooting
PeteNesbitt
March2006

ping<defaultgwip>
iffails,tryotherlocalhostsandconfirmphysicalcabling,checkarptable
ping<defaultgwhostname>
iffirstfailurehere,lookatnameresolution.Trydigorhostandpingthenameserver.
ping<destinationhostname>
tryanotherremotehostname,checknameresolution,trypingingIPaddress
traceroute<destinationhostname>
thiswillshoweachstepofthetripandindicatewherealongthepathitfails
tcpdump
vieworsavenetworktraffic.
Learntousethefilterssuchas'tcpdumpieth1host192.168.1.254andnotportssh'

CommonTasks:
changethesystemshostname
use'hostname<newname.domain.com>'tosetthehostnamefortherunningsystem.
Tochangethehostnamepermanently,edit/etc/sysconfig/networkandrestartthenetwork
(orcombinethetwomethods)
changeaninterfaceIPaddress
temporarilysettheIPaddressviachkconfigoripcommands.
Forapermanentchange,editetc/sysconfig/networkscripts/ifcfgethnandrestartthe
network(orcombinethetwomethods)
MoreDetailonCommandsandFiles:
ifcfgethnfiles
Thedirectory/etc/sysconfig/networkscriptscontains,amongotherrelatedfiles,the
configurationfilesforeachphysicalandvirtualinterface.
Theconfigurationentriesareprettyselfexplanatory.
Typicalentriesareshownbelow(notallentriesarealwaysrequired):
DHCP:
DEVICE=eth0
BOOTPROTO=dhcp
HWADDR=00:11:D8:44:12:19
ONBOOT=yes
6of11

RedHatLinuxNetworking

FundamentalNetworkConfigurationandTroubleshooting
PeteNesbitt
March2006

TYPE=Ethernet

StaticIP:
DEVICE=eth0
BOOTPROTO=static
NETWORK=192.168.1.0
BROADCAST=192.168.1.255
IPADDR=192.168.1.40
HWADDR=00:11:D8:44:12:19
ONBOOT=yes
TYPE=Ethernet
Ifyouwanttodisableorsaveacopyofanifcfgfile,youshouldremoveitfromthe
directorysinceasimpleappendingof_orig(orsomethingsimilar)willstillleaveitasa
targetforthenetworkservicestouse.Dependingonotherdetailsthismayharmlesslyfail
ormaycausemuchdisruptionwhenthenetworkservicesare(re)started.
ifconfigcommand
Thiscommandwillallowyoutoviewortemporarilymodifymanyaspectsofthenetwork
settings.Withnoarguments'ifconfig'displaysinformationonallactiveinterfaces.
InformationincludesIP,broadcast,networkmask,andthroughputinformationamongst
otherthings.
Youcanchangemuchofthenetworksetupwiththiscommand.Itistypicallyusedtoeither
changesomethingtemporarily(e.g.fortesting)ortobringasingleinterfaceupordown,
whichcanbedonewithifupandifdownusinglesstyping:).Onebenefitofusingifconfigto
changeanIPaddressisthatyoudonotneedtostartandstopthenetwork.Youcaneven
changetheMACaddressusingthiscommand.
ipcommand
Verysimilartoifconfigthiscommandcandisplayormanipulatenetworksettings.Although
someinformationoverlapsthesetwocommands,'ip'dealswithIP,networkaddress,routing
andARPinformation.
Thedifferentoutputformatscanworkforyouwhencreatingscriptstoreportormanipulate
anetwork.AprimeexampleishowtheIPaddressandNetworkMaskaredisplayed:
#ifconfigeth0|grep"ineta"
7of11

RedHatLinuxNetworking

FundamentalNetworkConfigurationandTroubleshooting
PeteNesbitt
March2006

inetaddr:192.168.1.40Bcast:192.168.1.255Mask:255.255.255.0
#ipaddrshowdeveth0|grep"inet"
inet192.168.1.40/24brd192.168.1.255scopeglobaleth0
iptablesutility
TheLinuxkernelbasedFirewall.WedonottypicallyusetheLinuxFirewallbutyou
shouldknowhowtocheckit.ThefirewalliscomprisedofNetFilter,theheartofthesystem
whichispartoftheLinuxkernel,andIPtableswhichistheuserspacetoolsand
configurationsyouusetocontrolthefirewall.Thefirewallisalwaysrunningandis
controlledbyrules(orchains)theuserdefines.Inwhatisreferredtoasstoppedthe
firewallisactuallyrunningbuthasnoblockingrules.Youcandisplaythecurrentstateof
thefirewallwiththecommand'iptablesL'.Youcanflushallrulesandinessence'stop'a
firewallwiththecommand'serviceiptablesstop'(seeservicecommanddetailedbelow).
Anonblockingfirewallwilldisplaythefollowing:

tcpdumpcommand
Displayspacketheadersfromaninterface.Thiscommandoffersmanywaystomanipulate
thedisplay.Themostcommonistoaddsimplefilterstothecommandline.
Atypicalremotemonitorofwebtrafficmightlooklikethis(wrappedlinesareindented):
[root@d20721614287~]#tcpdumpieth1porthttp
tcpdump:verboseoutputsuppressed,usevorvvforfullprotocoldecode
listeningoneth1,linktypeEN10MB(Ethernet),capturesize96bytes
16:35:38.976614IPd20721614287.bchsia.telus.net.50436>nfl.bodog.com.http:S
1104743985:1104743985(0)win5840<mss1460,sackOK,timestamp307804218
0,nop,wscale2>
8of11

RedHatLinuxNetworking

FundamentalNetworkConfigurationandTroubleshooting
PeteNesbitt
March2006

16:35:38.986239IPnfl.bodog.com.http>d20721614287.bchsia.telus.net.50436:S
78647379:78647379(0)ack1104743986win5792<mss1380,sackOK,timestamp
744177594307804218,nop,wscale0>
16:35:38.986447IPd20721614287.bchsia.telus.net.50436>nfl.bodog.com.http:.ack1
win1460<nop,nop,timestamp307804221744177594>
16:35:38.989257IPd20721614287.bchsia.telus.net.50436>nfl.bodog.com.http:P
1:462(461)ack1win1460<nop,nop,timestamp307804222744177594>
16:35:39.004945IPnfl.bodog.com.http>d20721614287.bchsia.telus.net.50436:.ack
462win6432<nop,nop,timestamp744177596307804222>
Whenloggedintoaremotesystemviasshandwanttoruntcpdumponthesameremote
interface,alwaysincludea'notssh'filteroreverybitoftcpdumpinfosenttoyourdisplay
willbecaughtbytcpdumpcausingamassivesshloop.
Youshouldbeawarethattcpdumpcapturespacketsattheinterfacelevelandiptables
handlesarepassedthroughtothenetworkIPstack,soincomingtrafficiscapturedby
tcpdumppriortoiptablesrulesbeingappliedbutoutboundtrafficispassedthroughiptables
rulesonit'swaytotheinterface.
networkfile
Thefile/etc/sysconfig/networkdefinesnetworksettingscommontoallinterfaces.
Atypicalfilemaylooklikethis:
NETWORKING=yes
HOSTNAME=nova3.linux1.ca
GATEWAY=192.168.101.1
NOZEROCONF=yes
The'NOZEROCONF=yes'removesanunnecessaryroutingentry.
chkconfigcommand
Thiscommandisusedtosimplifythemanagementofservicedaemons.Itworksin
conjunctionwiththe'service'command.Itsetswhatrunlevelsaserviceswillbestartedand
stoppedat.
Anyinitscriptthatcontainstherequiredlinescanbemanagedbychkconfig.
Thefollowingtwokeywordbasedentriesallowaninitscripttofallunderthecontrolofthe
chkconfigcommand:
#chkconfig:23450892
#description:Automatesapacketfilteringfirewallwithiptables.
9of11

RedHatLinuxNetworking

FundamentalNetworkConfigurationandTroubleshooting
PeteNesbitt
March2006

Use'manchkconfig'fordetailsabouttheinitscriptentriesaswellashowtoadd,delete,or
modifyaservicesbehavior.
Themostcommonuse,'chkconfiglist'(twodashes),istodisplaywhatserviceswillbe
runningatrunlevel3and5.
Youcannotchangethecurrentstateofadaemonwiththiscommand.Useserviceinstead.
servicecommand
Wherechkconfigisusedtoreportorsettheplanedstatesofdaemons,'service'isusedto
checkorchangethestateofaservicedaemon.
Forservicesthatusethe'case'statement,theservicecommandsisusedtostart,stop,restart
orgetthestatusofadaemon.Ifyoudroptheactionargumentthevalidchoicesare
displayed.
Typicalusage,saytotroubleshootflakysshconnection,couldlooksomethinglikethis:

SeethecommentunderNotesandTipsregardingrestartingaremotenetwork.
pscommand
Althoughthisisaverybasiccommand,Iwanttotouchonit'simportance.When
troubleshootingnetworkservices,youneedtoconfirmthedaemonisactuallyrunning.Itis
possiblefortheservicecommandtoreportadaemonisrunningwheninfactithascrashed
andleftbehindPIDandlockfiles.
Thiscommandhasmanyoptionsbutfornetworktroubleshootingyoudon'tneedtoknow
themall,howeveritwouldbeprudenttospendsometimeandgetcomfortablewithsomeof
theoutputformats.
Asimplechecktoseeifjavaisrunningcouldbe'pse|grepjava'
SeetheitemunderNotesandTipsforsomefurtheractions

10of11

RedHatLinuxNetworking

FundamentalNetworkConfigurationandTroubleshooting
PeteNesbitt
March2006

NoteandTips:
Whenworkingonaremotemachine,alwaysuse'servicenetworkrestart'nevertryandstop
thenstartthenetworkintwostepsoryouwillbedisconnected.Samethingwithssh.
Whenusingtcpdumpremotely,besuretofilterout'ssh'packets.Ifyouwonderwhy,tryssh'ing
intoaremotesystemandrun'tcpdump'withnoarguments.
IfIPmovesfromonesystemtoanotheritwillconfusesomelayer4switches,pingoutfrom
newmachinetotellswitcheswhereyouare.Otherwise,thenewsystemwillnotbeavailable
untiltheswitchflushesit'stables.
Ifpsdoesnotshowaprocessrunningor'netstatl'doesnotshowalisteningport,butservice
reportsitisrunningorreportsarelatederror,youmayneedtoremoveoldPIDorlockfiles.
Thesearelocatedin/var/run/and/var/lock/subsys/respectably.
IfpsreportsadaemonhasaPIDbutitisnotfunctioningorwillnotstoporstart,youmayneed
tokillthedefunctprocess.Todothis,seethekillandkillallmanpages.

EOF

11of11