|
4701 Willowtree Lane
Clayton, NC 27520
United States of America
(919) 553-TBOS [553-8267]
fax: (919) 553-9095
tsa@tbos.net
This page last updated 12 December 1998.
This report describes tests to verify that TBOS monitoring system elements are operating correctly, and provides troubleshooting hints if abnormal operation is observed.
A TBOS test device (such as a TBOS Tester from Telephony Software Associates, Inc.) is not absolutely essential but will expedite matters considerably. As summarized at contents, three major sections include:
This report has been formatted for on screen viewing. A printable, PDF version may be downloaded by clicking here. General information about TBOS and definitions of its specialized terms can be found at TBOS Serial E-telemetry: An Overview.
This document was last updated 12 December, 1998. Your questions, suggestions and comments are welcome. If you encounter a TBOS related problem not described here please feel welcome to call TSA on (919) 553-TBOS [553-8256] or contact us by e-mail. We would hope to assist with your immediate situation and to expand this page for future visitors.
1. Basic TBOS tests:
2.a Common problems:
1.1 Verify monitored equipment is responding
The most common test of a TBOS installation is verifying that a monitored network element is responding to scan requests (alarm polls). This should be done as part of equipment installation to make sure the TBOS port is functioning and the display address is correctly set.
If you have a TBOS Tester , connect your PC via the tester's RS232/422 converter directly to the monitored equipment TBOS interface and use the Display Mode test. Ideally you should test one NE at a time, rather than multiple NE's daisy chained together. Poll only the display address(es) for which the NE is supposed to respond.
Once active polling is in evidence with no timeouts, create a non service affecting alarm at the NE that should be reflected in its TBOS output. Observe that the appropriate alarm point is activated, and is deactivated when the alarm condition is cleared. Should timeouts or other irregularities be observed, refer to "Problems and Symptoms" for troubleshooting strategies.
If you are troubleshooting an existing installation the alarm remote will be available to generate alarm polls. Use the tester's Monitor Mode to detect the polls and verify responses from the monitored equipment, rather than actively generating polls from the tester.
Without a TBOS Tester , the alarm remote itself must be used to generate alarm polls. You will probably not be able to limit alarm polling solely to the display address of interest. This creates the possibility that display addressing problems will not be obvious, even if the NE is responding. The alarm center should report links down to all display addresses except the one(s) on which the NE is responding. Create a non service affecting alarm that should be reflected on the TBOS output. Check with the alarm center to make sure that the correct alarm was reported and that it occurred on the correct display address.
1.2 Verify monitored equipment accepts commands
This test should be conducted only after an alarm polling test has been successfully completed. Note that the NE is not obligated to process serial commands simply because it reports alarms via TBOS.
If you have a TBOS Tester , connect your PC via the tester's RS232/422 converter directly to the monitored equipment TBOS interface and use the Command Mode test. Choose a command that will result in a non-service affecting action and that will be detectable by observing the NE (e.g. LED indicator changes, CID messages, office alarms, etc.). The command should be acknowledged on screen and confirmed by observation of the NE.
If a TBOS Tester is not available, request that the command be sent from the alarm center. The command should be acknowledged on screen and confirmed by observation of the NE.
Should timeouts or other irregularities be observed, refer to "Problems and Symptoms" for troubleshooting strategies.
1.3 Verify alarm remote is polling and accepting responses
This test checks the operation and programming of the alarm remote, rather than the monitored equipment. Each port of the remote should be tested. The remote should be issuing alarm polls to each of the populated display addresses on the port and should not be polling vacant addresses.
If you have a TBOS Tester , it will generate responses to the remote's alarm polls when operated in Respond Mode. This allows the remote to be tested independently of the monitored equipment (or even prior to monitored equipment installation). Connect your PC via the tester's RS232/422 converter directly to the remote port under test, and set the tester to respond to all eight displays. The on screen scan indicator will reveal which displays are being polled by the remote. To verify responses are being received by the remote, set an alarm point active on the tester and check with the alarm center to verify the alarm is accurately reported. The alarm center can also test remote command capability by sending a test command at this time. The command should be acknowledged at the alarm center and reported on the tester's screen.
The tester can also emulate monitored equipment on selected displays if a port is partially populated and being expanded. Example: displays 1 through 4 are dedicated to existing NE's. Two more NE's are being added that will utilize displays 5 and 6; remote programming to poll the two additional displays needs to be verified. To do this, set the tester to respond on displays 5 and 6 only. Add the tester in parallel to the NE's on displays 1 through 4. A Respond Mode test will show, as before, all displays being polled but the tester will confine its responses to the new addresses.
If the monitored equipment has been installed and is capable of generating responses to alarm polls, another approach is to use Monitor Mode to observe the TBOS data activity. The monitored equipment responses are confirmed by the absence of timeouts and the display of any active alarms. Displays being polled by the remote are indicated on screen, and should not include any vacant addresses. If timeouts, data errors, or other irregularities are observed, refer to "Problems and Symptoms" for troubleshooting strategies.
If a TBOS Tester is not available the monitored equipment must reply to alarm polls from the remote. Verify one by one that polls are being issued to each NE on the port by creating a non service affecting alarm that should be reflected on the monitored equipment TBOS output. Check with the alarm center to make sure that the correct alarm was reported and that it occurred on the correct display address. Timeouts or link status alarms (Point 64) at the alarm center may indicate polls are being issued to non-existent NE's, that one or more NE's is failing to respond, or that a display address is duplicated such that two NE's are responding to the same alarm poll.
1.4 Verify alarm center database is correct
TBOS exchanges do not include explicit identification of monitored equipment elements nor explicit alarm messages. Equipment is identified only by port and display address assignments; alarms are reported only as high or low bits. This results in an efficient protocol but creates the possibility for confusion if addressing is misassigned. An important test, therefore, is to make sure the alarm center receives alarm indications on the appropriate address, and that the alarm indications at the center accurately reflect the alarms the monitored equipment conveys.
If a TBOS Tester is available, Respond Mode can be used to emulate the monitored equipment and generate test alarms. This eliminates the need to pull circuit packs or use other methods to generate test alarms from the monitored equipment, and takes the guesswork out of which alarm points may be activated by such actions.
By far the most common problem with TBOS installations is miswiring. TBOS uses a 4-wire interface (RS-422 or RS-485) which includes a transmit pair and a receive pair. Each pair has a plus (tip or non-inverted) lead and a minus (ring or inverted) lead. It is fairly evident that the transmit pair of the remote must be connected to the receive pair of the monitored equipment and vice versa. Be aware, however, that there is no standardization regarding pair nomenclature. While the pair marked "transmit" generally refers to that element's transmitter, occasionally it means the opposing element's transmitted signal should be connected to that pair.
Generally speaking connections should be made plus-to-plus and minus-to-minus. Again, there is no standardization. Some monitored equipment has been observed to require +/+ connection on one pair and +/- connection on the other pair.
If your network has a limited munber of combinations of remotes and monitored equipment it may be sufficient to solve this "connection puzzle" for each combination and keep good notes. If several types of monitored equipment and remotes are in use, determining how each connects to a standard element (such as a TBOS Tester) may be more effective. Once this is known, the appropriate direct connections between any two elements can be inferred easily.
2.a.2 Display address incorrect
Each monitored equipment element connected to a given remote port must be uniquely addressed. The remote must poll that address, and the alarm center computer must expect responses to be on the display address the monitored equipment is actually using. A common source of confusion is use of display address numbering zero through seven on one element and one through eight on other elements.
Timeouts and/or link down indications typically indicate an absent or non-responding address, or that the address is not being scanned by the remote. Mismatches and other data errors are indicative of a duplicated address, that is, two NE's on the same remote port being programmed with the same display address. Each attempts to respond to the same alarm poll, and data corruption results.
If you have a TBOS Tester it is easy to to determine the address to which monitored equipment is programmed, to check which addresses the remote is polling, and to verify that the alarm center database has the monitored equipment assigned to the appropriate display.
TBOS specifies an RS-422 interface, a basic four wire scheme that supports relatively long distances (up to 4,000 feet) and is fairly noise tolerant. Unfortunately, RS-422 is not optimal for multidrop applications, including TBOS, though the protocol's relatively low data rate (2400 baud) generally permits adequate operation.
In a TBOS application as many as eight monitored equipment elements are daisy chained in parallel to a single alarm remote port (see Figure One). This places as many as eight transmitter outputs on the same wire pair, in contention for one receiver. Because the transmit idle state in RS-422 is identical to transmit data zero (the + lead at a negative voltage relative to the - lead), the active transmitter must supply energy sufficient to overcome the other seven transmitters in order to be detected by the receiver.
This scheme can be sensitive to the termination impedance of each pair. Often a 120 resistor must be placed across each pair at the end of the daisy chain furthest removed from the transmitter(s), that is, across the alarm remote receive pair and across the receive pair of the monitored equipment element furtherst from the remote. The resistor at the remote is more critical, as it is that pair which hosts multiple transmitters.
The exact value of the resistor is application dependent, and affected by a variety of factors including the length of the daisy chain, the number of transmitters and their characteristics, noise in the environment, and the characteristics of the receiver.
A termination impedance problem may be indicated if the monitored equipment appears to be responding to polls but the remote is unable to detect them (timeouts or link down indications are observed). This condition is readily revealed by the datascope window of a TBOS Tester operated in Monitor Mode. Often the monitored equipment responses will be detectable when that equipment is connected individually to the remote, but problems will ensue when multiple monitored equipments are paralleled.
If a TBOS Tester is not available, use a scope to examine the waveform on the receive pair at the remote. (Note: you should measure the signal differentially. Use a dual channel scope with one channel inverted and the channels added. Don't clip the scope probe ground lead to any of the TBOS leads.) If the signal does not cross zero volts by at least 200mV, the termination resistor should be adjusted. Ideally the RS422 signal is centered around zero volts, with a magnitude of 4 to 12 volts (±2 to ±6 volts).
RS-485 is a virtually identical 4-wire standard to RS-422, but defines an "active" and a "passive" state for the transmitter. When the transmitter is active (enabled by a request to send indication or by transmitted data from the transmitting element) it operates similarly to an RS-422 transmitter (a low impedance voltage source). In the passive state, however, it becomes a relatively high impedance "unit load". This arrangement is much more reliable for multidrop applications because only one transmitter is active even though multiple transmitters may share the same pair. Transmitter contention problems are thus effectively eliminated.
2.a.4 Remote not programmed correctly
The alarm remote is programmed or otherwise configured to poll certain display addresses for each port. If this is not done properly, alarm polls will not be generated for the monitored equipment and it will not respond. Use a TBOS Tester in Respond Mode or in Monitor Mode to detect alarm polls from the remote. Other devices such as scopes or LED indicators can reveal whether a remote port is issuing polls but cannot inform as to which addresses may be being scanned. A protocol analyzer can provide this insight but the binary or hexadecimal value of alarm polls for each display address must be recognized, as TBOS messages are not verbose (as are TL1 or MML, for example).
2.a.5 Dead TBOS interface on monitored equipment
If the remote is generating alarm polls normally but the monitored equipment is not responding, the possibility of a bad TBOS interface on the monitored equipment cannot be discounted. In addition to eliminating the more common problems, it is generally reassuring to try testing known good equipment in order to eliminate the possibility of test equipment or test procedure problems.
However, do not dismiss the unlikely as impossible. In one case reported to us
the TBOS interface on every piece of equipment in the CO had been destroyed by a
ground fault on the remote, but a trip to another CO was required to regain confidence that
what the TBOS test equipment and procedures were indicating was indeed true.
Monitored equipment has up to 200 mSec to respond to an alarm poll or command request, although response generally occur in 10 to 20 mSec. If the 200 mSec standard is violated a timeout results and the remote moves on to the next display address. Consecutive errors on the same display result in a link down indication to the central.
Intermittent timeouts may result if the monitored equipment is preoccupied and/or if it receives polls too rapidly. For example, running TBOS test software on a Pentium fast enough to cope with the additional overhead of recent operating systems may bombard monitored equipment with so many polls that it cannot handle other processes. Depending on how the TBOS functions are implemented in the monitored equipment firmware, rapid polling rates may interfere with those functions or more likely the TBOS processing may be dropped on occasion.
In the case of test software, the solution is to slow the polling rate if that option is available (this variable is called "poll delay" in TBOS Tester). Remotes generally poll multiple network elements and multiple ports, so rapid polling is rarely a problem in "the real world".
Slow overall polling rates are more likely to be a problem, especially with remotes monitoring many network elements. Revisitation intervals of more than a few seconds are generally frowned upon as they imply a delay in reporting alarms. One typical cause of excessive polling overhead is the polling of display addresses from which the monitored equipment has been removed. The remote should declare a link down and revisit these addresses only occasionally, but if the revisitation interval is very short or if the address is polled each poll cycle, a 200 mSec delay per defunct address results. Use a TBOS test device to identify these instances, then reprogram the remote to skip the unused addresses.
In addition to timeouts, three other error types can affect TBOS transactions. These are parity errors, framing errors, and mismatches. The most likely causes are reversed polarity of one of the pairs, multiple NE's assigned to the same display address, a noisy electrical environment, or a termination problem.
The alarm remote may not indicate data errors as such but simply declare a link down condition. For troubleshooting it is useful to know whether polls and responses are being generated and corrupted, or are absent altogether. It can also be helpful to know if errors are occasional, bursty, regularly occurring, or confined to one or some displays. A TBOS test device should quickly reveal these conditions.
The TBOS protocol includes provision for sending commands to monitored equipment as well as scanning it for alarms. A three byte sequence is employed that on first analysis would seem to make the possibility of a false command vanishingly small. Yet the requirements are not so strenuous as they might appear. Essentially three bytes must occur in sequence with correct parity and framing, the first beginning with 01, the second with 10, and the third being 11001100 (see Figure Two). If data errors are occuring frequently enough and are not completely random, the probability of hitting this combination increases substantially. On at least one occasion we have received reports of monitored equipment "receiving" commands to take maintenance actions (such as protection switching) from remotes allegedly incapable of generating TBOS commands.
Capturing such random events takes time, but it can be done by monitoring the TBOS transactions or by using a TBOS Tester in Respond Mode to emulate the monitored equipment. The logging function captures and time stamps each remote command received, so the argument between the remote manufacturer and the monitored equipment manufacturer can be resolved. If a false command is executed but not logged, the monitored equipment is at fault. If a serial command is detected and logged where none should exist, the culprit is the remote.
This is a list of most commonly observed symptoms associated with TBOS problems, and their most likely causes.
2.c.2 Regularly occurring, intermittent timeouts
2.c.6 Rx Data always on (no polling)
2.c.7 NE's respond individually but not when daisy chained
2.c.8 Non-sequential polling and rapidly changing alarms (monitor mode test)
3. How can I test to determine...
3.1 ...if my TBOS Tester is working
Use loopback tests. Set hardware loopback switch on converters so equipped, or connect Tx+/Rx+ and Tx-/Rx- and start polling. Self test pattern (responses=polls) should be observed. If this fails, check for software setup problems using soft loopback (<F10>) option. Check serial port by shorting pins 2 and 3 on RS232 port. Also, test known good TBOS port.
3.2 ...what display address the NE is set for
In Display Mode, poll all eight displays and see which one is not timing out. View the master display and note the display(s) that are rapidly polled or showing alarms, or check log (window or disk log) and see which displays are reporting timeout errors and which are not. Modify polling parameters to poll only the "live" displays". Timeouts should not occur.
In Monitor Mode, view the master display and note the display(s) that are rapidly polled or showing alarms, or check log (window or disk log) and see which displays are reporting timeout errors and which are not.
3.3 ...what displays are being polled
Use Monitor Mode or Respond Mode and examine the scan indicator at the bottom of the screen.
3.4 ...if the NE is responding
If timeouts are occurring, responses may be absent or may simply not be detected. Use the datascope (<Alt-D>) in Display or Monitor modes and look for response bytes (blue or non-shaded background).
3.5 ...what wiring pattern to use
If the wiring pattern to use has not been previously documented, employ the "educated trial and error" method. For monitored equipment, use Display Mode and for Remotes use Respond Mode. To start with, connect the NE's transmit pair to the test device receive pair (TBOS Testerchannel 2). Connect the NE's receive pair to the test device transmit pair (TBOS Tester channel 1). In each case connect + to +/tip/data normal and - to -/ring/inverted data. Run a test. If results are normal, the wiring pattern is probably correct (for single elements and depending on the make and model of equipment, reversals between Tx and Rx of the same polarity can be tolerated for single connections). Write it down in the test equipment manual or documentation for the NE.
If timeouts are observed, perform a self test on the TBOS test device.
If the self test works or if rapid data errors were observed with the first connecttion pattern, try other wiring combinations. Make careful notes so that each possible wiring pattern is exercised and not duplicated. Note a continually Rx Data LED generally indicates a channel 2 polarity reversal. The recommended order to proceed is:
3.6 ...if the NE's TBOS port is working
Use Display Mode to poll the network element and note responses. Once polling error free, create a non service affecting alarm at the NE that should be reflected in its TBOS output. Observe that the appropriate alarm point is activated, and is deactivated when the alarm is cleared.
Alternatively, use Monitor Mode to observe that the alarm remote is issuing polls to the monitored equipment and responses are being generated.
3.7 ...if the alarm center computer is reporting all alarm points
Use Respond Mode to simulate the monitored equipment. Activate the "All Ones" option to turn on all alarms at once, or use the auto-respond feature to turn each point on and off in sequence.
3.8 ...if the polling rate is too fast
Intermittent timeouts may be occurring during Display Mode tests. Use the poll delay option to add 5 to 10 milliseconds of delay between polls. (Normally polls are issued as quickly as possible). If the timeouts stop occurring you may optionally find the delay threshold. Reduce the poll delay value by half and repeat the test. When the timeouts reoccur, the threshold has been crossed. Increase the delay value until the timeouts no longer occur.
# # #
Contents
. . .Feedback. . .Get PDF
. . .TBOS Overview
TBOS Tester. . .About TSA. . .TBOS.net Home
|
|
4701 Willowtree Lane
Clayton, NC 27520
United States of America
(919) 553-TBOS [553-8267]
fax: (919) 553-9095
tsa@tbos.net
This page last updated 12 December 1998.