Best Practices for IP Infrastructure Monitoring
Article ID: 55203
Updated On:
Products
Issue/Introduction
The IP Resource Monitor regularly checks on IP resources, samples and stores their performance data, and sends alerts when they violate performance triggers.
The IP Resource Monitor also provides a single screen from which you can enter diagnostic and performance commands against a resource. Supported resources:
| z/OS IP Stacks
| OSA | Cisco Channel Cards | 2216 routers |
| Enterprise Extender | VIPA | CSM (Communication Storage Manager) | Address Spaces |
| (physical infrastructure)
(logical infrastructure) | |||
Step 1 Delete less important resources
When you first set up a NetMaster for TCP/IP region, Express Setup automatically discovers the IP resources that are present and active at the time.
After this initial setup, they are not dynamically updated - you need to update them.
Express Setup is pretty good when finding the big stuff - stacks, OSAs, CIPs and so on. It also finds every address space that was active when it ran. It is highly unlikely you want to monitor all of them - delete all but the most important ones.
As with the IP Node Monitor, don't leave everything that Express Setup finds on the IP Resource Monitor. Don't monitor things just because you can - this wastes space and CPU, and produces unwanted alerts.
To delete an IP resource from the IP Resource Monitor
Use /IPMON (M.I) then enter DEL next to the unwanted resource name
Unlike IP Nodes, IP Resource Monitor resources of all types are defined individually, not as members of groups.
Stacks and Network Interfaces
Step 2 Add z/OS IP Stacks
To add a stack to the IP Resource Monitor
Use /IPMON (M.I) then PF4=Add then select a class of STACK
More information:
See the section "How to Define A Stack" in Chapter 10, "Setting Up Monitoring", in the Administrator Guide .
What's the fuss about?
Seeing a STACK resource as a single line on the IP Resource Monitor does not look too exciting. But behind the scenes, stack monitoring enables many useful things.
- When you monitor a stack, the NetMaster Packet Analyzer can examine all IP connections to/from the stack, producing throughput and other statistics.
- Stack performance monitoring can include data about:
- Connection Workload (from Packet analyzer)
- FTP and Telnet Workload (from SMF)
- Stack network interfaces (from Packet analyzer and SNMP)
- Stack Address Space and Ports (from Packet analyzer)
- IP, TCP and UDP activity (from SNMP IP, TCP, UDP MIBS)
- Any other stack SNMP MIB attributes (from any IBM MVS MIB)
Not all of the numerous /IPMON Stack commands are equally useful. Try these first. - Connection Workload (from Packet analyzer)
- TRS (Traffic Statistics) shows real-time byte/packet throughput grouped by:
- Application
- Local TCP Port
- Stack Home Address
- Stack Interface
- Protocol
- Protocol by Stack Home Address
- Protocol by Stack Interface
- Remote Network
This info comes from the Packet analyzer, and can be shown over the last 5 minutes (1-minute intervals), or the last hour (5-minute intervals) - Application
- PT starts a packet trace, PTV views it, PTI stops it
- ICF lists IP connections, ITF lists Telnet connections
- S shows a snapshot of all IP activity, similar to /IPSUM
- IP, WI, WC, WF, T and H show different kinds of stack performance data.
Step 3 Activate Stack IP, TCP, and UDP SNMP Monitoring
To add IP, TCP, and UDP monitoring to a stack
Use /IPMON (M.I) then UM next to the stack
- Activate/Select IP, TCP, and UDP Monitoring
- Use F4=Add to add additional attributes
- Select an attribute, to set optional alerting conditions
Monitor each attribute for at least a few weeks, before setting any alert conditions. This lets history data build up, and gives you an idea of typical hourly values.
Recommended stack IP, TCP, and UDP performance attributes
| IP network | TCP protocol | UDP protocol |
| ipDelivers ipReceives ipDiscards ipReasmRequired ipOutRequests ipOutDiscards ipFragOk | tcpActiveOpens tcpPassiveOpens tcpFailedOpens tcpCurrentConns tcpSegmentsRecvd tcpSegmentsSent tcpResetConnects tcpSegmentsInErr tcpSegmentsRST tcpSegmentsRxmit | udpDgrmsInError udpDgrmsNoPort udpDgrmsReceived udpDgrmsSent
|
If you are running Enterprise Extender , this uses UDP, so make sure you monitor the UDP attributes.
Step 4 Activate Stack Network Interface Monitoring
To add network interface monitoring to a stack
Use /IPMON (M.I) then UM next to the stack
- Activate/Select Network Interface Monitoring
- Use F4=Add to add additional attributes
- Select an attribute, to set optional alerting conditions
Monitor each attribute for at least a few weeks, before setting any alert conditions. This lets history data build up, and gives you an idea of typical hourly values.
Recommended stack network interface attributes
| ifInPkts ifInPktsDiscard ifInPktsError ifStatusOper | ifOutPkts ifOutPktsDiscard ifOutPktsError |
By default, ifStatusOper will raise an alert if the value is not UP.
OSA (Open Systems Adaptor)
Step 5 Add OSAs
To add an OSA to the IP Resource Monitor
Use /IPMON (M.I) then PF4=Add then select a class of OSA
To update the monitored attributes & alert conditions of an OSA
Use /IPMON (M.I) then UM then F10=EditLst
Use F4=Add to add an attribute
Select an attribute to set alert conditions
MVS command D M=CHP will display information about OSA channel types, and their status.
More information:
See the section "Define OSA Resources" in Chapter 10, "Setting Up Monitoring", in the Administrator Guide .
OSA Performance Attributes
Most OSA performance attributes are only applicable to specific OSA models, so no specific attribute list will suit all OSAs. However, for your model of OSA, monitor the following aspects:
- TotalOSAErrors
- DeviceStatus
- PCI Bus utilization
- Processor Utilization
- Read and Write rates
- Counts of significant error types
- Packets discarded, sent and received in error
- Attributes that may affect performance, related to congestion or queue sizes
For complete OSA monitoring, NetMaster for TCP/IP needs to access one of:
- OSA/SF
- The OSA-Express Direct SNMP subagent ( IOBSNMP )
If you don't have OSA/SF or IOBSNMP , you can use the RMF* attributes, which come from RMF.
If you do have OSA/SF or IOBSNMP, use the CHP* attributes from OSA/SF or IOBSNMP, in preference to the RMF* ones.
Cisco CIP host
Step 6 Add Cisco CIP hosts
To add a CIP host to the IP Resource Monitor
Use /IPMON (M.I) then PF4=Add then select a class of CIP
To update the monitored attributes & alert conditions of an CIP host
Use /IPMON (M.I) then UM then F10=EditLst
Use F4=Add to add an attribute
Select an attribute to set alert conditions
CIP Performance Attributes
Monitor attributes related to:
- Bandwidth utilization
- Packets received and sent in error
- Total packets
- Packets discarded
- Device availability
More information:
See the sections "Define Cisco Channel Cards" and "Collect Data from Cisco Routers" in Chapter 10, "Setting Up Monitoring", in the Administrator Guide.
Enterprise Extender
Step 7 Add Enterprise Extender
To enable EE monitoring by the IP Resource Monitor
Use /IPMON (M.I) then PF4=Add then select a class of EE
To update the monitored attributes & alert conditions of EE on this system:
Use /IPMON (M.I) then UM then F10=EditLst
Use F4=Add to add an attribute
Select an attribute to set alert conditions
High, medium, and low priority traffic correspond to VTAM COS (Class of Service) priorities. EE uses a separate UDP port for each priority of traffic.
You should monitor all available EE performance attributes.
EE can only have one definition on a system (because there is only one VTAM).
VIPA
Step 8 Add VIPAs
NetMaster for TCP/IP VIPA resources are used to monitor dynamic VIPAs.
The procedure for defining a VIPA is slightly different to other resources. This is because dynamic VIPAs can move from one stack and system to another.
To add a VIPA to the IP Resource Monitor, use Chapter 10, "Setting Up Monitoring" in the Administrator Guide . See the section, " Define VIPA Resources ".
You should monitor all available VIPA performance attributes.
Performance data alerts
Performance data attributes generally fall into a few main categories.
| Category | What is measured? |
| Availability, Stability, Reliability | Is this resource active, reachable and responsive? |
| Throughput | How many bytes, packets, datagrams, segments went through, in an interval? |
| Errors | How many errors occurred in an interval? |
| Response time | What is the actual response time? What was the lowest, highest, average, in an interval? |
| Workload | How much work did application end-users do, in an interval? |
| Capacity, Utilization | How much of its own internal resources was a device using? |
| Resource Usage | How much of the external system resources was a device using? |
| Configuration | Device and environmental specific configuration |
Alerts can be raised when performance data attribute samples:
- Fall above or below an absolute numeric limit
- Fall above or below a percentage difference from a moving average
- Equal or do not equal a particular character value
More information:
Technical document TEC417964, NetMaster for TCP/IP: Performance Data Reference , available from supportconnect.ca.com
Monitoring Maps
You may not want to get performance data alerts at all times. For example, you need to be alerted if a key application has lower than normal throughput during its peak processing hours - but you don't care if this happens during off-peak times.
Monitoring Maps specify the days and hours that a resource will be monitored. Alternatively, you can continue to monitor a resource, but just disable alerting. One Monitoring Map can be attached to many IP resources, so they are all on the same 'schedule'.
More information:
"Monitoring Maps" in Chapter 10, "Setting Up Monitoring" in the Administrator Guide .
See the results
After you do all these steps to define your critical IP network and infrastructure resources, where do you see the results?
The IP Resource Monitor
To access the IP Resource Monitor
3270 Use /IPMON (M.I)
WebCenter Menu option Monitoring, IP Resources
IP Resource Monitor Display
The IP Node Monitor only includes one type of resource, so it has columns specific to IP Nodes, such PINGRTT.
In contrast, the IP Resource Monitor display includes very different kinds of resources, so its columns have to be more general, about monitoring statuses and so on. It is more like a CA SOLVE:Operations monitor.
Things that determine the status and color of an IP resource on the display include:
- Whether the resource is active
- Whether the resource can be sampled, or if there are environment errors preventing it being sampled
(SNMP errors, Packet analyzer not up, attribute is not supported by the device type, like trying to get 10G OSA attributes from older OSAs) - Whether or not any of its sampled attributes show abnormal values - you set what you want to be regarded as 'abnormal'.
The colors indicate the status of each IP resource, and are the same as /IPNODE
| Status of IP Resource | Color of Line |
| status OK, no alerts | Green |
| Unknown or NoAttr error status | White |
| Timeout, Error or SNMPerror status | Turquoise |
| outstanding severity 1 alert or serious error during data sampling | Red |
| outstanding severity 2 alert | Yellow |
| outstanding severity 3 alert or actual state is INACTIVE | Pink |
| outstanding severity 4 alert | Blue |
While its main general display is not as simple or informative to look at as the IP Node Monitor, the IP Resource Monitor's power is 'hidden' behind it. It is a central control point from where you can enter commands against any resource.
IP Resource Monitor Commands
Type ? next to any /IPMON resource, to see the available commands
- Turquoise commands are specific to the type of resource.
The available commands depend on the resource type. Some resource types have very few commands - others (like Stacks) have dozens. - Green commands are general, common monitor functions.
- You can run Packet Traces from the IP Resource Monitor.
SmartTrace commands ( PT, PTV, PTI and PTD ) are supported from the IP Resource Monitor for STACK , ASMON and EE resources.
If the color and/or Alert columns indicate there are current alerts open for this resource, use the AL command to access the Alert Monitor and see them.
The Alert Monitor
If any IP resources have performance-related alerts, these alerts will appear on the Alert Monitor, and be sent to any forwarding destinations.
The Graphical Monitor
The Graphical Monitor (3270-only) is fed by the IP Resource Monitor, and can be a more suitable way to display resource statuses than the IP Resource Monitor.
See The CA NetMaster for TCP/IP Handbook to learn about exploring the Graphical Monitor.
Environment
Component: NMTIP
Feedback
