Professional Documents
Culture Documents
Purpose:
The purpose of this document is to provide an overview of the ViaSat LinkStar system as a whole and provide
troubleshooting aides for use on the ViaSat LinkStar system.
Scope:
The comments in this guideline are for reference only in order to aide an operator in basic RCST and LinkStar
hub troubleshooting. The comments neither are stated policy nor replace any comments set forth in ViaSat
LinkStar manuals, technical notes, release notes or procedures. Those documents, which might be referenced
below, can be found on the ViaSat Customer Extranet website at http://extranet.viasat.com/vsat.
References:
ViaSat LinkStar manuals, technical notes, release notes or procedures, which can be found on the ViaSat
Customer Extranet website at http://extranet.viasat.com/vsat.
Contact Information:
ViaSat Network Operations Center http://extranet.viasat.com/vsat/service/noc/
noc-carlsbad@viasat.com (888) 272-7232 or +1-760-476-2600
ViaSat Proprietary 1038741_000 (July 28, 2005
1
LinkStar LinkStar IP LinkStar Troubleshooting Guide
6) Problem symptom: TCP PEP not running in CPU Real Time and with CPU Priority............................................. 14
TDU/ITDU Specific Issues ................................................................................................................................... 16
1) Problem symptom: Intermittent ITDU and/or GCU alarms, network failures ......................................................... 16
2) Problem symptom: Solid TDU alarm, LinkStar NMS Console indicates Uplink Chain Failure ............................. 16
3) Problem symptom: Solid ITDU alarm, LinkStar NMS Console indicates Uplink Chain Failure ............................ 16
4) Problem symptom: ITDU ASI Rate change, LinkStar Hub is unstable ................................................................... 17
IPE Specific Issues................................................................................................................................................ 17
1) Problem symptom: IPE Manager Data Rate change, LinkStar Hub is unstable ...................................................... 17
2) Problem symptom: After SMR Software upgrade, LinkStar Hub is unstable.......................................................... 17
Modulator Specific Issues ..................................................................................................................................... 17
1) Problem symptom: Modulator Symbol and/or FEC Rate change, LinkStar Hub is unstable................................... 17
RCST/GCU Specific Issues .................................................................................................................................. 17
1) Problem symptom: RCSTs on a specific GCU are down......................................................................................... 17
LinkStar Process Specific issues ........................................................................................................................... 18
1) Problem symptom: Linkstar Processes do not recover after power failure .............................................................. 18
2) Problem symptom: After an RNCC restart, only a portion of the network recovers ............................................... 19
3) Problem symptom: Linkstar LDAP process not recover after power failure ........................................................... 19
Cisco Switches
The LinkStar hub employs the Cisco 3500 XL and 3550 series of switches. The 3500XL Layer 2 switch has
since been phased out by Cisco and was replaced with the 3550 Layer 3 switch. The switches are the main
components that provide the connectivity to all of the LinkStar hub devices.
Cisco 3550 TX Rack Switch
The Cisco 3550 TX (Transmit) Rack Switch provides connectivity to the entire LinkStar hub and it’s
components. Please view Appendix C for the default configuration of the Cisco 3550 TX Rack Switch.
RF Equipment
GPS
The LinkStar Hub incorporates timing using a high stability GPS (Global Positioning Satellite)-based clock
reference. The GPS receiver supplies a 10MHz clock to the LinkStar Intelligent Timing Distribution Unit
(ITDU). The GPS receiver also provides the clock to the DVB modulator to
ensure stability of the DVB signal.
ITDU
The ITDU generates a Network Clock Reference (NCR) using the Program Clock Reference (PCR) mechanism
and inserts it into the MPEG stream. Under DVB-RCS (Digital Video Broadcast-Return Channel Satellite), the
ITDU assumes an additional responsibility of inserting DVB-RCS tables in the outbound. The RNCC instructs
the ITDU to transmit the tables in the outbound. The ITDU multiplexes tables and the NCR with its DVB-ASI
(DVB-Asynchronous Serial Interface). Optionally, the tables and NCR generated by the ITDU can be
multiplexed with other traffic externally using a DVB multiplexer. In typical configurations, the IP
Encapsulator DVB ASI Out is connected the DVB ASI In of the ITDU.
GCU
The GCU (Gateway Channel Unit) receives and demodulates the inbound TDMA signal, extracting the IP
packets. The ITDU is connected to the GCUs to synchronize the TDMA frames with the NCR.
DVB Modulator
The DVB modulator receives the aggregate DVB stream from the ITDU, modulates it to the 70/140 MHz
Intermediate Frequency (IF), and sends it to the uplink RF chain.
IP Encapsulation
IPE
The PBR has a connection to the outbound link via the IP Encapsulator (IPE). Normal LinkStar hubs are
configured with the SkyStream SMR (Source Media Router) as its IPE device. The IPE receives outbound IP
packets, inserts them into MPEG2-TS packets after performing Multi Protocol Encapsulation (MPE), and
creates an outbound IP over DVB stream. The output of the IPE is provided to the DVB ASI input of the ITDU.
1) To prevent the SNMP Subagent from flapping, ensure that the SNMP
Master Agent (/etc/init.d/init.snmpdx) is configured correctly
a. Connect to the NCC/RNCC/TCP PEP, and at the CLI (Command Line
Interface) prompt, log in as super user
b. Edit the /etc/init.d/init.snmpdx file with the vi editor
c. Ensure that the /etc/init.d/init.snmpdx file appears as below
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/linkstar/snmp/lib
export LD_LIBRARY_PATH
case "$1" in
'start')
if [ -f /home/linkstar/snmp/conf/snmpdx.rsrc -a -x /home/linkstar/snmp/b
in/snmpdx ]; then
/home/linkstar/snmp/bin/snmpdx -y -c /home/linkstar/snmp/conf -a
/home/linkstar/snmp/conf/snmpdx.acl -o /home/linkstar/snmp/conf/enterprises.oid
fi
d. Once the changes have been made, save the file and exit out of vi editor
mode
e. Remain logged in as super user mode for the next step
2) Restart the SNMP Master Agent after the parameters in the
/etc/init.d/init.snmpdx file have been modified/verified
a. /etc/init.d/init.snmpdx start
i) The following message appears after the SNMP Master Agent has
been restarted:
REG_AGENT_IS_ALIVE
3) Log out of super user mode
a. At the telnet (#) prompt, type exit
4) While logged in as linkstar, restart the SNMP Subagent by killing the
process on the NCC/RNCC/TCP PEP
a. Issue an lstar_stat on the NCC/RNCC (or tcpspoof_stat on the TCP
PEP) and note the PID for the SNMP Subagent process
PROCESS STATUS PID
------- ------ ---
NCC process UP 14189
Java Adapter for NCC UP 14249
SNMP subagent for NCC UP 14365
NMS Server UP 14316
Map Server UP 14244
X Server UP 14233
LDAP Server UP 14200
b. Issue the following command “kill -9 pid_of_snmp_subagent_process",
replacing pid_of_snmp_subagent_process with the actual PID noted in
step 3) a. above, in this case 14365
c. Wait several moments for cron to respond to restart the SNMP
Subagent
5) SNMP will now function properly
6) Repeat the above steps on every NCC/RNCC/TCP PEP server
a. The devices do not have the LinkStar processes stopped or started in
order to implement the change
7) After any NCC/RNCC/TCP PEP reboot, ensure that the SNMP Subagent
process is operating
a. At a telnet prompt, issue ps –eaf | grep snmp
b. An example output from the NCC would be as follows:
[ncc1:~] ps -eaf | grep snmp
4) Problem symptom: Other processes on the NCC are running in CPU Real Time mode
Component: NCC
Category: Software
Software Version: 1.0.8.16
Description: NCCs should be running in CPU Real Time mode
Problem Resolution: In order to reduce problems with the NCC services, only the NCC should be
running in CPU Real Time mode, as opposed to other processes running in
CPU Real Time mode
1) To view the CPU priority status of the NCC, issue prstat -a, and in the
output make sure the PRI (priority) for the NCC process is shown as 100,
which indicates that the NCC is running in CPU Real Time mode
a. The expectant results of the prstat –a command is as follows:
PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP
12145 linkstar 244M 223M sleep 26 11 0:00.04 9.6% java/165
12023 linkstar 512M 507M sleep 52 2 0:02.04 1.9% ns-slapd/46
12012 linkstar 13M 12M sleep 100 - 3:32.33 0.3% lstar-2.0.8.ncc/1
7187 linkstar 1944K 1408K run 53 2 0:00.01 0.1% fping/1
b. If other processes are running in CPU Real Time mode according to the
outputs given above, modifications will need to be made to the
/viasat/tcpspoof_prio_cronjob file, while logged in as root
i) Connect to the NCC, and at the CLI prompt, log in as super user
ii) Edit the /viasat/bin/ncc_prio_cronjob file with the vi editor
iii) Make the following changes to the file. The “ORIGINAL” lines
shown below are for reference purposes, and the updates are
highlighted in red:
ORIGINAL:
PROCESSPID=`$PS $PSOPTIONS | grep "lstar-.*ncc " | \
egrep -v '(grep|nccexec|is_ncc_active)' | \
awk '{print $1}'`
MODIFIED:
PROCESSPID=`$PS $PSOPTIONS | grep "lstar-.*ncc " | \
egrep -v '(grep|nccexec|is_ncc_active|update_ncc_dbs)' | \
awk '{print $1}'`
iv) Once the changes have been made, save the file and exit out of vi
editor mode
v) Log out of super user mode
vi) To verify that the changes made have been implemented, wait
several moments for cron to respond, and then follow the steps 1) a.
c. Repeat the above steps on the other NCC as well
5) Problem symptom: RNCC not running in CPU Real Time mode
Component: RNCCs
Category: Software
Software Version: All
Description: RNCC should be running in CPU Real Time mode however is not
Problem Resolution: In order to reduce problems seen with Late BTP segments on the GCUs, ensure
that the RNCCs are running in CPU Real Time mode. Starting in LinkStar
software version 1.0.8.15, both the NCC and RNCC are automatically
configured to run in CPU Real Time mode. To configure the RNCC manually
to run in CPU Real Time mode, perform the following steps:
1) Issue an lstar_stat on the active RNCC and note the PID for the RNCC
process
PROCESS STATUS PID
------- ------ ---
RNCC process for Region 0x1e UP 26186
Java Adapter for RNCC for Region 0x1e UP 26222
SNMP subagent for Region 0x1e UP 26689
IPE Manager for Region 0x1e UP 8037
Java Adapter for IPE Manager for Region 0x1e UP 8059
PEP Manager for Region 0x1e UP 26679
Java Adapter for PEP Manager for Region 0x1e UP 26255
Trap Manager for Region 0x1e UP 26684
Java Adapter for TRAP Manager for Region 0x1e UP 26272
NMS Server UP 26533
2) Log in as root super user using the "su" command
3) Issue the following command "priocntl -s -c RT -i pid pid_of_rncc",
replacing pid_of_rncc with the actual PID noted in step 1 above, in this
case 26186
4) Exit out of the super user mode
5) Type prstat -a, and in the output make sure the PRI for the RNCC process
is shown as 100
a. This means that the RNCC process is running in CPU Real Time mode.
Type the letter q or use control c (^c) to stop the prstat output
b. Below is an example of the RNCC process running in CPU Real Time
mode, as seen via the prtstat –a command:
PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP
3061 linkstar 12M 12M sleep 100 - 1:21.36 3.6% lstar-1.0.8.11.r/1
11953 linkstar 3752K 3360K run 23 2 0:00.00 0.5% ps/1
20177 linkstar 34M 19M run 53 2 0:00.00 0.2% java/12
25971 linkstar 1088K 752K sleep 0 2 0:48.54 0.1% nccexec/1
Note: If there is a RNCC switch or restart, the above process needs to be repeated on the active RNCC. You
can also perform the same actions on the offline (standby) RNCC, so in the event of a RNCC handover, the
LinkStar RNCC process will already be running in CPU Real Time mode
6) Problem symptom: TCP PEP not running in CPU Real Time and with CPU Priority
Component: TCP PEP
Category: Software
Software Version: 1.0.8.14
Description: TCP PEP is not running in CPU Real Time mode with CPU Priority
Problem Resolution: In order to reduce problems with jitter (large variations between the minimum
and maximum RTT of packets), the TCP PEP should be running in CPU Real
Time mode with priority
1) To view the CPU priority status of the active TCP PEP, issue the following
command:
a. [tcpacc1:~] tcpspoof_stat
b. The expectant result is to have the RTPRI status indicate 50. The
number 50 equals a high CPU priority for the TCP PEP process
PROCESS STATUS PID
------- ------ ---
TCP PEP Process UP 21343
TCP PEP SNMP Subagent UP 21354
REAL TIME PROCESSES:
PID RTPRI TQNTM
21343 50 100
2) To view the CPU Real Time mode/priority status of the active TCP PEP is
to issue prstat -a, and in the output make sure the PRI (priority) for the TCP
PEP process is shown as 150, with the results of the prstat command
a. The expectant results of the prstat –a command is as follows:
PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP
21343 linkstar 38M 37M sleep 150 - 75:35.47 8.9% lstar-1.0.8.16./1
26781 linkstar 1536K 1192K cpu0 58 0 0:00.00 0.2% prstat/1
26703 linkstar 1472K 1176K sleep 51 0 0:00.00 0.1% csh/1
26712 linkstar 2424K 1680K sleep 48 0 0:00.00 0.0% bash/1
227 root 2800K 2024K sleep 58 0 0:00.00 0.0% nscd/15
3) If other processes are running in CPU Real Time mode according to the
outputs given above, modifications will need to be made to the
/viasat/tcpspoof_prio_cronjob file, while logged in as root
a. Connect to the TCP PEP, and at the CLI (Command Line Interface)
prompt, log in as super user
b. Edit the /viasat/tcpspoof_prio_cronjob file with the vi editor
c. Make the following changes to the file. The “ORIGINAL” lines shown
below are for reference purposes, and the updates are highlighted in
red:
ORIGINAL:
SPOOFERPID=`$PS $PSOPTIONS | grep "lstar-.*tcpacc" | \
egrep -v '(grep|tcpspoofexec|is_tcpspoofer_active)' | \
awk '{print $1}'`
MODIFIED:
SPOOFERPID=`$PS $PSOPTIONS | grep "lstar-.*tcpacc$ " | \
egrep -v
'(grep|tcpspoofexec|is_tcpspoofer_active|update_tcpspoof_dbs )' | \
awk '{print $1}'`
Description: The RNCC is assigning the Bursts to the GCU; A Spectrum Analyzer trace
indicates the RCSTs are transmitting bursts and are being received at the output
of the L-Band splitter at the correct power level and frequency
Problem Resolution: This could be RF related
1) Replace the RF Cable from a known good working GCU
2) Check to see if the RCSTs on that GCU have been restored to service
a. If the RCSTs do not restore to service, connect to the GCU and ensure
that the bootconf (pconf) parameters are correct
i) Ensure that the population ID is correct
ii) Ensure that the RXAtten field is configured appropriately for this
LinkStar network
3) It is possible that there are other issues that might cause these symptoms.
Please contact the ViaSat Network Operations Center for further assistance
LinkStar Process Specific issues
1) Problem symptom: Linkstar Processes do not recover after power failure
Component: NCC/RNCC
Category: Hardware
Software Version: All
Description: /home/linkstar/nccexec.ncc.ncc1or2.log or
/home/linkstar/nccexec.rncc_1e.rncc1 or rncc2.log indicates processes up
except Tomcat server, LDAP keeps restarting every 2 minutes. lstar_stat
indicates all processes in Standby except down for SNMP
Problem Resolution: Enable logging on the mounted partitions to ensure a safe recovery of data in
the event of a severe power loss issue
1) Issue lstar_stop on the affected device
2) Ensure that the /etc/vfstab has “logging” enabled for the partitions, to
enable consistency of the partitions
3) While logged in as root, or super user, edit the /etc/vfstab file. Some of the
lines look like:
#device device mount FS fsck mount mount
#to mount to fsck point type pass at boot options
#
# Some lines that have been removed...
#
/dev/dsk/c1t1d0s0 /dev/rdsk/c1t1d0s0 / ufs 1 no -
/dev/dsk/c1t1d0s3 /dev/rdsk/c1t1d0s3 /home ufs 1 yes -
4) In reading the lines, the output equals:
Device to mount = /dev/dsk/c1t1d1s0
Device to fsck = /dev/rdsk/c1t1d0s0
Mount point = /
FS type = ufs
fsck pass = 1
Mount at boot = no
Mount option = (none, specified by the hyphen)
5) To enable logging:
The option is called 'logging' and should be listed under the 'Mount option' column. Note that all it means is that
the hyphens in the last column of the lines that begin with '/dev/dsk/...' need to be replaced with the word
'logging'. So, the file will now look like:
# Some lines that I have not shown here...
/dev/dsk/c1t1d0s0 /dev/rdsk/c1t1d0s0 / ufs 1 no logging
/dev/dsk/c1t1d0s3 /dev/rdsk/c1t1d0s3 /home ufs 1 yes logging
6) After this, type the 'sync' command and then type 'reboot' to restart the
machine. The next time the machine comes up, the logging feature will be
enabled
7) After logging has been enabled and the server has been rebooted, restart the
LinkStar processes
Note: Make sure that the spelling of 'logging' is correct. If /etc/vfstab is damaged, unpredictable behavior may
occur
2) Problem symptom: After an RNCC restart, only a portion of the network recovers
Component: RNCCs
Category: Hardware
Software Version: All
Description: Some RCST records that were on the NMS are no longer present
Problem Resolution: Remove temporary LinkStar database files from the RNCC
1) From rttelnet issue the “showhealth” command
2) If the numbers of GCUs and/or RCSTs from the showhealth report do not
match what is configured on the NMS, check the
home/linkstar/etc.<hubname>_rncc1e/etc/db for .tdb files
3) The RNCC will use these before pulling the information from LDAP
during a network restart
4) Delete all of the .tdb files except RnccConf.tdb and almAdapterInfo.tdb
files, and then restart the RNCC
5) The RnccConf.tdb file should contain the following entries ONLY and
nothing else
6) If there are other entries in that file, edit the file with the vi editor and copy
and paste the below entries into that file
Name : "CustomerRNCC1e"
regionid : 0x1e
ipaddr : 192.168.2.1
AdminStatus : 0
;;
7) After the file has been edited with the correct information, stop the RNCC
(lstar_stop) and then restart the RNCC (lstar_start)
3) Problem symptom: Linkstar LDAP process not recover after power failure
Component: NCC/RNCC
Category: Software
Software Version: All
Description: /home/linkstar/nccexec.ncc.ncc1or2.log indicates processes up except Tomcat
server, LDAP keeps restarting every 2 minutes. The lstar_stat output indicates
all processes in Standby except down for SNMP
Problem Resolution: LDAP corruption occurred due to abrupt power failure on the NCCs. LDAP is
attempting to recover, and could take 1+ hours
1) Create an LDIF on both NCCs for safety and for import
2) De-install/re-install LDAP (might have to be done on both NCCs)
3) Import the LDIF on the last known running NCC
4) Re-initialize LDAP replication, verify processes
Problem Resolution: GCU specific static routes are no longer supported on LinkStar software
versions 1.0.8.14 and above. The static routes must now be applied to the hub
level, instead of independently on the GCUs
1) Prior to upgrading the LinkStar hub to 1.0.8.14 or above, delete the GCU
specific static routes and add them to the hub level static routes on the
NMS
2) The hub level static routes can be found here:
a. LinkStar NMS Console expand RNCC 0x<RegionID> or LinkStar
RNCC 0x<RegionID> Select Hub 0x<RegionID> Configuration
IP Static Route Configuration
b. Add the static routes as appropriate according to the configurations that
are necessary to permit the intended traffic to reach it’s destination
10) Problem symptom: Ability to limit users on the LinkStar NMS Console
Component: NMS
Category: Software
Software Version: All
Description: The need might arise to limit the visibility to certain LinkStar NMS Console
users to view only portions of the LinkStar NMS Console
Problem Resolution: Write access can be restricted to the User Group Set, Carrier Sets, Flow Profile
Set, IP QoS Queue Set and Hub Set. View access can be restricted to
population views
1) On the LinkStar NMS Console, select Network Administration
OperatorGroup Set
2) Select Configuration Add, and create the authorized Operator Group
a. Add in the required data for the Group Name: and Description: fields
b. Select Submit
3) On the LinkStar NMS Console, select Network Administration
Operator Set
4) Select Configuration Add, and create the authorized Operator
a. Add in the required data for the User Id:, User Name:, Description:,
Password:, Re-enter Password:, IP Address:, IP Mask:, and Telephone:
fields
b. In the "Unassigned Groups:" field, select the newly created
OperatorGroup and move it to the "Assigned Groups:" field
c. Select Submit
5) Select the appropriate permissions for this new user/group in both the NCC
and RNCC
a. Select NCC Utility Permissions
b. For the group that needs to be restricted, select Read Only
c. For all other groups, select Inherited
d. Select RNCC 0x1e or LinkStar RNCC 0x1e Utility Permissions
e. For the group that needs to be restricted, select Read Only
f. For all other groups, select Inherited
6) Select the appropriate permissions for this new user/group for the
population(s) that the group needs access to or needs to be restricted
a. Select the Population(s) Utility Permissions
b. For the group that needs to be restricted, select Read Only or No
Access, as appropriate
c. For all other groups, select Inherited
7) Select the appropriate permissions for this new user/group for the Software
Download Set
a. Select the Software Download Set Utility Permissions
Description: Bursts remain assigned to the RCSTs even though the RCST is not transmitting
data
Problem Resolution: There are parameters that can be changed to allow for quicker de-allocation of
bandwidth from RCSTs on a LinkStar Region level
1) Change BWFactors from 0x3c02 (60%/02%) to something like 0x3c14
(60%/20%)
a. This will de-allocate CIR/BoD bursts more quickly
b. Please take care when adjusting the values, as this could be detrimental
to the operations of the LinkStar network
2) Decrease the DeallocDelay from its default of 10 seconds to a shorter value
of 6 seconds
a. Do not set the value to anything under 6 seconds, as this could be
detrimental to the operations of the LinkStar network
3) Contact the ViaSat Network Operations Center for assistance in making the
necessary changes, and to properly configure these changes into the
LinkStar database
TCP PEP Specific Issues
1) Problem symptom: TCP PEP Failures due to Multicast traffic
Component: TCP PEP, PBR, TX Switch
Category: Software
Software Version: 1.0.8.14 and above
Description: Multicast traffic that is allowed to flow towards the TCP PEP will cause the
TCP PEP to crash
Problem Resolution: Ensure that the LinkStar hub is properly configured to prevent Multicast traffic
from flooding the TCP PEP
1) Ensure via a VLAN/PBR Policy that no multicast traffic is passed to the
PEP
a. One way of achieving this is by not allowing VLAN 1 on the trunk port
between the hme0 interface and PBR interface Fa0/1 (Multicast Traffic
in LinkStar software release 1.0.8.12 and above can only flow on
Management VLAN 1)
b. This also restricts TCP traffic flowing on the Management VLAN (1)
from passing through to the TCP PEP
i) In order to avoid this, the native VLAN on this trunk port should be
defined to a number other than VLAN 1
ii) Using a VLAN ID such as VLAN 999 for the native VLAN
prevents this issue from occurring
iii) It is imperative to note that VLAN 999 should not be configured as
a user VLAN by the customer and no normal traffic should be
passed on this VLAN. If for whatever reason the there is a need to
use VLAN 999 as a traffic VLAN identifier, then the native VLAN
ID of 999 should be replaced in the TX Switch and PBR
configuration by some other number not used in the network
iv) It is also imperative to note that VLAN 3 should not be configured
as a user VLAN by the customer and no normal traffic should be
passed on this VLAN. VLAN 3 is used for the connection between
the PBR Fa1/0 interface and interface hme1 of the TCP PEP. It
should not be used to pass normal traffic as a configured VLAN in
the customer network
v) Use the show vlan or show vlan brief command on the TX Switch
(Cisco 3550 or 3750) to verify which VLANs are configured and
their status
ViaSat Proprietary 1038741_000 (July 28, 2005
25
LinkStar LinkStar IP LinkStar Troubleshooting Guide
*** Any VLANs used in the TX switch configurations to achieve traffic separation should not be used by
the customer as configured VLANs in their network ***
2) Problem symptom: TCP PEP communication failures
Component: NMS
Category: Software
Software Version: All
Description: TCP PEP communication failures occur. SNMP Trap messages are not passed
to the LinkStar NMS Console
Problem Resolution: Ensure that the LinkStar NMS Console TCP PEP Manager Configuration
screen has the correct community string configured
1) On the LinkStar NMS Console, select RNCC 0x<RegionID> or LinkStar
RNCC 0x<RegionID> Configuration TCP PEP Manager
Configuration
2) Leave that window open, since it will be needed in the next steps to copy
the data
3) On the LinkStar NMS Console, open the TCP PEP Manager Configuration
screen again
a. Select RNCC 0x<RegionID> or LinkStar RNCC 0x<RegionID>
Configuration TCP PEP Manager Configuration
4) On the second screen, in the “Community String for SNMP Read/Write
Access:” field, type in the word “public”, without the “” marks
5) On the second screen, disable the TCP PEP Manager
a. Deselect the check mark in the “Enable TCP PEP Manager:” box
b. Select the Submit button at the bottom of the page
6) After the screen refreshes, enter in the IP Address data from screen one
onto the second screen
7) Select the Submit button at the bottom of the page
8) After the screen refreshes, the SNMP Community screen is now configured
to public
a. TCP PEP SNMP Trap communications should now function
3) Problem symptom: RCSTs are passing intermittent TCP traffic
Component: RNCCs/RCSTs
Category: Software
Software Version: All
Description: Network is up however TCP traffic is passing intermittently or there is a saw
tooth effect as seen via the SkyStream SkyAlarm
Problem Resolution: One of the reasons why this might happen is if the TCP window size is not set
correctly
1) To calculate the window size, use the following formula:
Maximum TCP throughput per connection (in bps) =
Window size in bytes / RTT value in seconds
a. For example, if the window size is set to 512 Kbytes (as seen via the
LinkStar NMS Console, TCP PEP Manager Configuration) and the
average satellite RTT delay is 650 ms, the maximum throughput per
connection is (512* 8)/ (650 /1000) = ~6.302 Mbits/s.
b. In a different example, if the window size is set to 64 Kbytes and the
average satellite RTT delay is 650 ms, the maximum throughput per
connection is (64* 8)/ (650 /1000) = ~788 kbits/s
2) This will help resolve the window size issue. If the problem continues,
then troubleshoot with the PEP further per the PEP Troubleshooting Guide
1) The Allot NetEnforcer needs to be placed between the PBR 0/1 interface
and hme0 of TCP PEP
a. This is needed if the customer wants to use the Allot to manage
bandwidth using VLANs
2) There should not be any equipment other than VIASAT provided
equipment within the 192.168.150.x subnet
a. Issues have been noticed that when Cisco routers are placed within the
192.168.150.x/24 subnet, some VLAN traffic ceased functioning
b. For reasons unknown, the newly introduced Cisco router (not the PBR)
responds with a message to the GCU with hme1 interface ip address
not found, or it proxy-arps for that address, causing the GCU:
i) Either to assume that the route is unreachable or,
ii) To cause the GCU to send the packets to that device
c. If there is a router present which cannot be removed (customer
requirements) try performing the following:
i) no cdp enable
ii) no ip-proxy arp
3) After the router is configured with VLAN/VPN configurations, and VRF
routes are defined in the router, the following command will have to be
issued in order to run ping tests:
a. ping vrf <vpn name> <ip address>
i) If VRF routes are not defined, use the following command to ping:
ping <vpn name> <ip address>
4) Ping commands should be used to test basic connectivity in order to test if
the VPN is connected properly
a. The first ping should be done from the RCST VPN IP Address to the
PEP VPN IP Address using the command:
i) ping -v <vpn id in decimal> -i <interface id obtained from prt ifmgr
command> <TCP PEP VPN IP Address>
(1) This will check out the path till from the RCST to the PEP
b. The second ping should be from the TCP PEP to the PBR for a VPN
connection
i) ping -v <vpn id in decimal> <PBR IP Address for this VPN trunk>
c. The third ping should be one from the RCST to the PBR
i) ping -v <vpn id in decimal> -i < interface id obtained from prt
ifmgr command > <PBR VPN IP Address>
2) Problem symptom: A newly added VLAN(s) fails to pass traffic
Component: TX Switch
Category: Software
Software Version: 1.0.8.14 and above
Description: When utilizing the VLAN/VPN feature on LinkStar software version 1.0.8.14
and above, configurations on the TX Switch need to be implemented in order to
make the newly added VLAN(s) active
Problem Resolution: Follow the below steps to configure the TX Switch to enable and view VLANs
Beginning in privileged EXEC mode, follow these steps to add an Ethernet
VLAN:
1) Enter VLAN configuration mode
a. LinkStar_TX_Switch# configure terminal
b. LinkStar_TX_Switch(config)# vlan database
2) Add a VLAN by assigning a number to it
a. If no name is entered for the VLAN, the default is to append the vlan-id
to the word VLAN. For example, VLAN0004 could be a default
VLAN name for VLAN 4
ViaSat Proprietary 1038741_000 (July 28, 2005
29
LinkStar LinkStar IP LinkStar Troubleshooting Guide
1) The RCST sends out all Management (native) VLAN packets untagged on
its Ethernet side
2) The LinkStar network is designed this way in order to support non-VLAN
remote sites, with no trunk port configured to the switch connected to the
RCST
3) If the switch connected to the RCST has the Management (native) VLAN
configured to something other than VLAN 1, the switch will receive these
packets untagged and will not be able to forward them
a. Configure the switch behind the RCST in “native” mode with a non-1
VLAN ID
8) Problem symptom: Unable to configure the Max Data Rate on a specific RCST
Component: NMS
Category: Software
Software Version: All
Description: On an already configured RCST, the Max Data Rate: on the IP Forward
Channel configuration screen does not get passed to the SkyStream SMR
Problem Resolution: Manually configuring the Max Data Rate: in the IP Forward Channel
configuration screen on the NMS for a specific RCST that already has an IP
Address configured causes the IPE Manager to crash and restart
1) In order to utilize the Max Data Rate: IP Forward Channel configuration
screen, the IP Forward Channel rate must be configured first, prior to
configuring the RCST IP Address information
a. With an already configured RCST, configure the RCST IP Address and
Sub Net Mask fields to 0.0.0.0/0.0.0.0
b. This will remove any entry on the SkyStream SMR for that RCST
c. Enter in the Max Data Rate: in the IP Forward Channel configuration
screen
2) Once the IP Forward Channel configuration has been submitted on a
specific RCST, enter in the RCST IP Address information
3) The IP Forward Channel rate will then be properly forwarded to the
SkyStream SMR, and the rate will be limited according to the rate specified
on the IP Forward Channel configuration screen
4) This has been fixed in LinkStar software version 1.0.8.17
b. This means that at least the frequency and symbol rate are correct
6) From the online RNCC command prompt, connect to rttelnet <RegionID>,
as in rttelnet 1e
7) Issue the following command to reset the root user password for all RCSTs
on LinkStar population 0x011e0001
a. senduterm 0x011e0001 passwd -u root -p <pass>
i) Replace <pass> with the actual password that is to be used for the
root user, without the <> symbols
8) Issue the following command to reset the guest user password for all
RCSTs on LinkStar population 0x011e0001
a. senduterm 0x011e0001 passwd -u guest -p <guest>
i) Replace <guest> with the actual password that is to be used for the
guest user, without the <> symbols
3) Problem symptom: RCST Is in TX Sync however not passing traffic
Component: RCSTs
Category: Software
Software Version: All
Description: The LinkStar NMS Console indicates that the RCST is up (Green) however the
user is unable to pass any data
Problem Resolution: Verify the following:
1) The RCST has an IP address assigned in the NMS
2) Check to see if the IPE is working properly and the RCST has the proper
string in the command file of the IPE
3) Assign a channel of CIR to the RCST
4) Check Internet connectivity to the hub
5) Check RCST TXPower level to ensure that it isn't configured too high
(placing the BUC in saturation)
6) Check RCST and GCU BER to ensure proper link budget is maintained
4) Problem symptom: RCST Is in TX Sync however passing very little to no traffic
Component: RCSTs
Category: Software Configuration
Software Version: All
Description: The user complains of DNS timeouts or unable to browse. The RCST is
assigned a public IP address and not a private (RFC 1918 address) IP address
Problem Resolution: It is possible that the RCST in question might be a subject of a Dos or DDoS
(Denial of Service or Distributed Denial of Service) attack from the OUTSIDE
1) Verify connections to/from the RCST via Allot box (if it is inline between
the PBR and the TX Switch)
a. Utilize the acstat -i | grep command and grep for the RCST IP subnet or
portion thereof
2) Block the destination port at the gateway router
3) Use ACLs (Access Control Lists) to block offending traffic prior to
reaching the DVB and the RCSTs
5) Problem symptom: High packet loss on a specific RCST
Component: RCSTs
Category: Software Configuration
Software Version: All
Description: Terminal TX/RX BERs were checked and both were good. Terminal was
assigned CIR, problem persisted. Receiving GCU indicates CRC errors on
return channel
Problem Resolution: There is a possibility that the RCST has too high of a transmit power
configured
1) Increasing the RCST TXPower worsens the problem
2) Decreasing the RCST TXPower clears up the problem
a. Terminal was transmitting too hot and clipping signal
3) Verify antenna pointing and for proper BUC polarity alignment
6) Problem symptom: RCST IP Accounting records not being collected
Component: RCSTs
Category: Software
Software Version: All
Description: A new GCU added to the network after region (RNCC) was started. The RCST
is running on that new GCU
Problem Resolution: TTP port not opened on newly added GCUs after the region has started.
Handover the RNCCs to restore
7) Problem symptom: RCST unable to gain TX Sync while in Maritime mode
Component: RCSTs
Category: Software
Software Version: 1.0.8 and higher
Description: All RCST bootconf parameters are correct, but SAT led remains in fast blink
and terminal fails to acquire network
Problem Resolution: Verify Maritime Mode operation
1) Telnet to the RCST via the static RCST IP address previously assigned per
the Maritime Mode operations as described in the LinkStar 1.0.8 Release
notes, ViaSat document 1003074_001
2) Perform a gps_getpos
3) Verify that the Latitude and Longitude updates are getting reported
correctly
4) If Latitude = 0, Longitude = 0 appears, check the Ethernet connection to
GPS
5) If the Maritime Mode functions do not work after viewing the "Notes On
Troubleshooting Maritime" in the 1.0.8 Release notes, disable the Maritime
Mode function via save -m 0 and call the LinkStar operator with updated
GPS coordinates to update the NMS configuration for that RCST site
8) Problem symptom: ss and prt ipqos RCST commands have different output displays
Component: RCSTs
Category: Software
Software Version: All
Description: Comparison of the output display of the two commands is reversed
Problem Resolution: This is how the outputs of the ss vs. prt ipqos line up next to each other in
regards to precedence:
Usr 0 = Expedited Forwarding
Usr 1 = AC 1
Usr 2 = AC 2
Usr 3 = AC 3
Usr 4 = AC 4
Usr 5 = Best Effort
The outputs of each are reversed in regards to the display order because of
where they lie in the signal flow
9) Problem symptom: VPN Membership information remains after RCST is deleted
Component: NMS
Category: Software
ViaSat Proprietary 1038741_000 (July 28, 2005
35
LinkStar LinkStar IP LinkStar Troubleshooting Guide
(1) This dynamically adjusts the MTU size of the RCST to 1500
for non-VLAN traffic and 1476 for VLAN traffic
configurations
b. Configure end devices with a MTU setting less than 1476
c. Configure a policy statement on a gateway router and the end routing
devices to clear the DF bit, in order to allow fragmentation
d. To configure a policy statement on a gateway router, enter in privileged
EXEC mode
i) Router# configure terminal
e. Create a route-map statement with a name of clear-df matching ACL
101
i) Router(config)# route-map clear-df permit 10
ii) Router(route-map)# match ip address 101
iii) Router(route-map)# set ip df 0
f. Apply the route-map statement to the Router’s interface
i) Router(route-map)# interface ethernet 0 (as an example)
ii) Router(config-if)# ip policy route-map clear-df
g. Return to privileged EXEC mode and write the configuration to
memory
i) Router(config-if)# exit
ii) Router# write memory
h. (Optional) Save your entries in the configuration file
i) Router# copy running-config startup-config
6) Contact the ViaSat Network Operations Center to obtain the appropriate
RCST software patch
11) Problem symptom: IP Supernetting is not supported on RCSTs
Component: RCST/NMS
Category: Hardware
Software Version: All
Description: The IP Address of an RCST is configured with a supernet address, however
traffic fails
Problem Resolution: IP Supernetting at the RCST level is not supported on the LinkStar system.
RCSTs can be assigned CIDR (Classless Inter-Domain Routing) networks,
however not in the supernet category
1) For example, 10.10.10.1 with a subnet mask of 255.255.254.0 is not
supported, however 10.10.10.1 with a subnet mask of 255.255.255.0 or
255.255.255.224 or 255.255.255.248 is supported
Useful Commands
Cisco Switch troubleshooting commands
show int status
The show int status command lists the overall port status of the TX and/or RX Switches. This is a good
command to use to view how all of the ports are configured and their status. Ports that show a Duplex value of
a-full and a Speed value of a-100 means that the port is configured as Auto-Duplex, Auto-Speed and has
negotiated with the connected device (Say, a Sun Server) to a Full-Duplex, 100 Mbits/s Ethernet link.
show int Fa0/X
The show int command lists all of the particulars of a specific port. The items to look for are any of the error
fields, such as: input errors, CRC, frame, overrun, ignored, output errors, collisions, interface resets, babbles,
late collision, deferred lost carrier, no carrier, output buffer failures, output buffers swapped out and any input or
output queue drops. Any of those errors indicate specific problems on the Ethernet link and should be
troubleshot. Reference Cisco’s Ethernet troubleshooting guide at
http://www.cisco.com/warp/public/473/46.html
show vlan or show vlan brief
show vlan or show vlan brief gives information about the VLANs and the ports that belong to a
particular VLAN
show ver
The show ver command will provide you an overview of the switch’s software and memory configurations and
any failure codes that are present from prior software/hardware failures.
show controllers ethernet-controller fastEthernet
The show controllers ethernet-controller fastEthernet 0/1, 0/2 etc command lists in detail the statistics for the
Ethernet Controllers for the selected port.
no service pad
service timestamps debug uptime
service timestamps log uptime
service password-encryption
!
hostname LinkStar_TX_Switch
!
enable password 7 <removed>
!
ip subnet-zero
!
ip igmp snooping vlan 1 static 0100.5e40.0101 interface Fa0/1
ip igmp snooping vlan 1 static 0100.5e40.0101 interface Fa0/2
ip igmp snooping vlan 1 static 0100.5e40.0101 interface Fa0/3
ip igmp snooping vlan 1 static 0100.5e40.0101 interface Fa0/4
ip igmp snooping vlan 1 static 0100.5e40.0103 interface Fa0/3
ip igmp snooping vlan 1 static 0100.5e40.0103 in terface Fa0/4
ip igmp snooping vlan 1 static 0100.5e40.0103 interface Fa0/24
ip igmp snooping vlan 1 static 0100.5e40.0104 interface Fa0/3
ip igmp snooping vlan 1 static 0100.5e40.0104 interface Fa0/4
ip igmp snooping vlan 1 static 0100.5e40.0104 interface Fa0/24
!
!
spanning-tree mode pvst
spanning-tree extend system-id
no spanning-tree vlan 1
no spanning-tree vlan 2
no spanning-tree vlan 3
!
!
interface FastEthernet0/1
description Connected to NCC1
switchport mode dynamic desirable
!
interface FastEthernet0/2
description Connected to NCC2
switchport mode dynamic desirable
!
interface FastEthernet0/3
description Connected to RNCC1
switchport mode dynamic desirable
!
interface FastEthernet0/4
description Connected to RNCC2
switchport mode dynamic desirable
!
interface FastEthernet0/5
description Connected to TCPACC1 eri0
switchport mode dynamic desirable
!
interface FastEthernet0/6
description Connected to TCPACC2 eri0
switchport mode dynamic desirable
!
interface FastEthernet0/7
description Connected to NMS PC
switchport mode dynamic desirable
!
interface FastEthernet0/8
description Empty Port
switchport mode dynamic desirable
!
interface FastEthernet0/9
description Connected to SMR 1 LAN 1
switchport mode dynamic desirable
!
interface FastEthernet0/10
ViaSat Proprietary 1038741_000 (July 28, 2005
45
LinkStar LinkStar IP LinkStar Troubleshooting Guide
speed 10
!
interface FastEthernet0/4
description Connected to GCU 4
switchport mode dynamic desirable
duplex half
speed 10
!
interface FastEthernet0/5
description Connected to GCU 5
switchport mode dynamic desirable
duplex half
speed 10
!
interface FastEthernet0/6
description Connected to GCU 6
switchport mode dynamic desirable
duplex half
speed 10
!
interface FastEthernet0/7
description Connected to GCU 7
switchport mode dynamic desirable
duplex half
speed 10
!
interface FastEthernet0/8
description Connected to GCU 8
switchport mode dynamic desirable
duplex half
speed 10
!
interface FastEthernet0/9
description Connected to GCU 9
switchport mode dynamic desirable
duplex half
speed 10
!
interface FastEthernet0/10
description Connected to GCU 10
switchport mode dynamic desirable
duplex half
speed 10
!
interface FastEthernet0/11
description Connected to GCU 11
switchport mode dynamic desirable
duplex half
speed 10
!
interface FastEthernet0/12
description Connected to GCU 12
switchport mode dynamic desirable
duplex half
speed 10
!
interface FastEthernet0/13
description Connected to GCU 13
switchport mode dynamic desirable
duplex half
speed 10
!
interface FastEthernet0/14
description Connected to GCU 14
switchport mode dynamic desirable
duplex half
speed 10
!
interface FastEthernet0/15
description Connected to GCU 15
switchport mode dynamic desirable
duplex half
speed 10
!
interface FastEthernet0/16
description Connected to GCU 16
switchport mode dynamic desirable
duplex half
speed 10
!
interface FastEthernet0/17
description Empty Port
switchport mode dynamic desirable
!
interface FastEthernet0/18
description Empty Port
switchport mode dynamic desirable
!
interface FastEthernet0/19
description Empty Port
switchport mode dynamic desirable
!
interface FastEthernet0/20
description Empty Port
switchport mode dynamic desirable
!
interface FastEthernet0/21
description Empty Port
switchport mode dynamic desirable
!
interface FastEthernet0/22
description Empty Port
switchport mode dynamic desirable
!
interface FastEthernet0/23
description Empty Port
switchport mode dynamic desirable
!
interface FastEthernet0/24
description Connected to Linkstar_Tx_Switch
switchport mode dynamic desirable
duplex full
speed 100
!
interface GigabitEthernet0/1
switchport mode dynamic desirable
!
interface GigabitEthernet0/2
switchport mode dynamic desirable
!
interface Vlan1
ip address 192.168.150.53 255.255.255.0
no ip route-cache
!
ip default-gateway 192.168.150.51
ip classless
ip http server
!
!
line con 0
line vty 0 4
password 7 <removed>
login
line vty 5 15
password 7 <removed>
login
!
!
end
If you have any questions about this bulletin, please contact our 24/7 Product Customer Support line at
+1-760-476-2600 (outside the U.S.), +1-888-272-7232 (in the U.S.), or send an e-mail to noc-
support@viasat.com.
Publication Information
ViaSat, Inc.
Clarksburg, MD
22300 Comsat Drive
Clarksburg, MD 20871
Phone: (301) 428-4500
Fax: (301) 428-4700
Norcross, GA
4356 Communications Drive
Norcross, GA 30093-2999
Phone: (678) 924-2400
Fax: (678) 924-2480