Understanding the JSON Export for Edge SWG Diagnostic Reporting

search cancel

Understanding the JSON Export for Edge SWG Diagnostic Reporting

book

Article ID: 408717

calendar_today

Updated On:

Products

ProxySG Software - SGOS

Issue/Introduction

In SGOS 7.4.11.1, the diagnostic reporting feature was introduced to diagnose issues related to sessions. You must have a minimum of SGOS 7.4.11.1 and SGAC 2.2.9 to use diagnostic reporting. You can export diagnostic reporting results to a JSON file to analyze the results offline. This article describes the contents of the JSON file.

Cause

Resolution

The JSON file contains arrays of objects and statistic objects on the following system components:

Sessions
HTTP requests
Sockets

See the following for an example of what the JSON file looks like:

 {"Sockets": [
{"socket_id":2082,"ip_version":2,"protocol":1,"remote_address":"192.0.2.255","remote_port":443,"local_address":"192.0.2.1",
   "local_port":6394,"complete":1}, 
{"socket_id":2083,"ip_version":2,"protocol":1,"remote_address":"192.0.2.255","remote_port":59137,"local_address":"192.0.2.1",
   "local_port":8081,"complete":1}, 
]
,"Socket Statistics": [
{"stat_id":8313,"timestamp":70676174511590,"socket_id":2082,"action":10,"interface":0,"state":0,"inputs":0,"outputs":0,
   "segments_in":0,"segments_out":0,"bytes_in":0,"bytes_out":0,"reassembly_queue_length":0,"sent_unacknowledged":0,    
   "receive_next":0,"receive_window":0,"send_window":0,"congestion_events":0,"congestion_window":1073725440,
   "round_trip_time":0,"round_trip_variance":12000,"packets_retransmitted":0,"out_of_order":0,"retransmit_threshold":3}, 
{"stat_id":8314,"timestamp":70676174512907,"socket_id":2082,"action":13,"interface":0,"state":0,"inputs":0,"outputs":0,
   "segments_in":0,"segments_out":0,"bytes_in":0,"bytes_out":0,"reassembly_queue_length":0,"sent_unacknowledged":0,
   "receive_next":0,"receive_window":0,"send_window":0,"congestion_events":0,"congestion_window":1073725440,
   "round_trip_time":0,"round_trip_variance":12000,"packets_retransmitted":0,"out_of_order":0,"retransmit_threshold":3}, 
{"stat_id":8315,"timestamp":70676198174538,"socket_id":2082,"action":13,"interface":0,"state":4,"inputs":1,"outputs":2,
   "segments_in":1,"segments_out":2,"bytes_in":0,"bytes_out":0,"reassembly_queue_length":0,"sent_unacknowledged":3931602534,
   "receive_next":1974056662,"receive_window":263408,"send_window":65535,"congestion_events":0,"congestion_window":5216,
   "round_trip_time":768,"round_trip_variance":192,"packets_retransmitted":0,"out_of_order":0,"retransmit_threshold":3}, 
{"stat_id":8316,"timestamp":70676198175425,"socket_id":2082,"action":12,"interface":0,"state":0,"inputs":0,"outputs":0,
   "segments_in":0,"segments_out":0,"bytes_in":0,"bytes_out":0,"reassembly_queue_length":0,"sent_unacknowledged":0,
   "receive_next":0,"receive_window":0,"send_window":0,"congestion_events":0,"congestion_window":0,
   "round_trip_time":0,"round_trip_variance":0,"packets_retransmitted":0,"out_of_order":0,"retransmit_threshold":0}, 
{"stat_id":8324,"timestamp":70680052988059,"socket_id":2083,"action":10,"interface":2,"state":4,"inputs":0,"outputs":1,
   "segments_in":1,"segments_out":1,"bytes_in":0,"bytes_out":0,"reassembly_queue_length":0,"sent_unacknowledged":3709354379,
   "receive_next":2205647001,"receive_window":263408,"send_window":262144,"congestion_events":0,"congestion_window":21137,
   "round_trip_time":1752,"round_trip_variance":217,"packets_retransmitted":0,"out_of_order":0,"retransmit_threshold":3}, 
{"stat_id":8325,"timestamp":70680053014159,"socket_id":2083,"action":13,"interface":2,"state":4,"inputs":0,"outputs":1,
   "segments_in":1,"segments_out":1,"bytes_in":0,"bytes_out":0,"reassembly_queue_length":0,"sent_unacknowledged":3709354379,
   "receive_next":2205647001,"receive_window":263408,"send_window":262144,"congestion_events":0,"congestion_window":21137,
   "round_trip_time":1752,"round_trip_variance":217,"packets_retransmitted":0,"out_of_order":0,"retransmit_threshold":3}, 
{"stat_id":8326,"timestamp":70680065407892,"socket_id":2083,"action":13,"interface":2,"state":4,"inputs":1,"outputs":3,
   "segments_in":2,"segments_out":3,"bytes_in":540,"bytes_out":507,"reassembly_queue_length":0,"sent_unacknowledged":3709354379,
   "receive_next":2205647541,"receive_window":263408,"send_window":262144,"congestion_events":0,"congestion_window":21137,
   "round_trip_time":1752,"round_trip_variance":217,"packets_retransmitted":0,"out_of_order":0,"retransmit_threshold":3}, 
{"stat_id":8327,"timestamp":70680065408709,"socket_id":2083,"action":12,"interface":0,"state":0,"inputs":0,"outputs":0,
   "segments_in":0,"segments_out":0,"bytes_in":0,"bytes_out":0,"reassembly_queue_length":0,"sent_unacknowledged":0,
   "receive_next":0,"receive_window":0,"send_window":0,"congestion_events":0,"congestion_window":0,
   "round_trip_time":0,"round_trip_variance":0,"packets_retransmitted":0,"out_of_order":0,"retransmit_threshold":0}, 
]
,"Sessions": [
{"session_id":10,"type":31,"complete":1}, 
{"session_id":11,"type":31,"complete":1}
]
,"Session Statistics": [
{"stat_id":40,"timestamp":70676054148251,"session_id":10,"action":10,"cpu_cycles":0,"memory_bytes":0}, 
{"stat_id":41,"timestamp":70676054206789,"session_id":10,"action":13,"cpu_cycles":0,"memory_bytes":24}, 
{"stat_id":42,"timestamp":70676054207760,"session_id":10,"action":13,"cpu_cycles":0,"memory_bytes":64}, 
{"stat_id":43,"timestamp":70676054790023,"session_id":10,"action":12,"cpu_cycles":0,"memory_bytes":0}, 
{"stat_id":44,"timestamp":70686121468706,"session_id":11,"action":10,"cpu_cycles":0,"memory_bytes":0}, 
{"stat_id":45,"timestamp":70686121523560,"session_id":11,"action":13,"cpu_cycles":0,"memory_bytes":24}, 
{"stat_id":46,"timestamp":70686121524490,"session_id":11,"action":13,"cpu_cycles":0,"memory_bytes":64}, 
{"stat_id":47,"timestamp":70686122475492,"session_id":11,"action":12,"cpu_cycles":0,"memory_bytes":0}
]
,"HTTP Requests": [{"request_id":1,"session_id":10,"request_method":"GET","request_scheme":"http","request_hostname":"192.168.0.100",
   "request_port":"80","request_path":"/open/missing","request_query":"?param=1&arg=2","response_code":404,"complete":1}
]
,"HTTP Request Statistics": [
{"stat_id":1,"request_id":1,"timestamp":70659151076307,"action":13,"state":10}, 
{"stat_id":2,"request_id":1,"timestamp":70659151263035,"action":13,"state":12}, 
{"stat_id":3,"request_id":1,"timestamp":70659151263980,"action":13,"state":13}, 
{"stat_id":4,"request_id":1,"timestamp":70659151400315,"action":13,"state":14}, 
{"stat_id":5,"request_id":1,"timestamp":70659152470985,"action":13,"state":16}, 
{"stat_id":6,"request_id":1,"timestamp":70659152531380,"action":13,"state":18}, 
{"stat_id":7,"request_id":1,"timestamp":70659152615609,"action":13,"state":19}, 
{"stat_id":8,"request_id":1,"timestamp":70659152642820,"action":12,"state":20}
]
,"Session Sockets": [
{"socket_id":2079,"session_id":10,"socket_type":12,"complete":1}, 
{"socket_id":2185,"session_id":11,"socket_type":12,"complete":1}
]
}

The following diagram shows the relationship between the system components, how they interact with each other, and how the statistics relate to the components:

Sessions:
- A single session can capture multiple HTTP requests and sockets M:N relationship). Session-Socket tables manage the relationships between the various sessions and sockets.
- A single session can have multiple statistical objects (1:N relationship).
HTTP Requests:
- A single HTTP request can only belong to one session, but that session may have multiple HTTP requests (1:N relationship).
- A single HTTP request can have multiple statistical objects (1:N relationship).
Sockets:
- A session may have multiple sockets (1:N relationship). Session-Socket tables manage the relationships between the various sessions and sockets.
- A single socket can have multiple statistical objects (1:N relationship).

Sessions Objects and Statistics

Session-Sockets Statistic Objects

Object Name	Value Type	Description	Example
`downstream_socket`	Integer	Unique identifier for the session the socket belongs to	`"socket_id":2079`
`upstream_socket`	Integer	Unique identifier for the socket	`"session_id":10`
`internal _socket`	Integer	Type of the socket (represented by a code): 10: Downstream socket 11: Upstream socket 12: Internal socket	`socket_type":12`
`complete`	Integer	Indicates if the socket is complete (1) or active (0)	`"complete":1`

Session Objects

Object Name	Value Type	Description	Example
`session_id`	Integer	Unique identifier for the session	`"session_id":10`
`type`	Integer	Type of session (represented by a code): 10: FTP_PROXY 11: HTTP_PROXY 12: SSL_PROXY 19: UNKNOWN_PROXY 20: TCP_TUNNEL 21: UDP_TUNNEL 29: UNKNOWN_TUNNEL 30: ACCESSLOG 31: DNS 32: HTTP 33: OPP 34: SSH 49: UNKNOWN	`"type":31`
`complete`	Integer	Indicates if the session is complete (1) or active (0)	`"complete":1`

Session Statistic Objects

Object Name	Value Type	Description	Example
`stat_id`	Integer	Unique identifier for the statistics record	`"stat_id":40`
`timestamp`	Integer	Timestamp of the statistics in nanoseconds. Timestamps are recorded as the nanoseconds since the Edge SWG appliance last booted up.	`"timestamp":70676054148251`
`session_id`	Integer	ID of the session that the statistics relate to	`"session_id":10`
`action`	Integer	Action that triggered the statistics (represented by a code): 10: Create session 11: Timer update 12: Destroy session 13: Manual update	`"action":10`
`cpu_cycles`	Integer	Number of CPU samples recorded while the session was processed	`"cpu_cycles":0`
`memory_bytes`	Integer	Amount of memory (in bytes) that the session used	`"memory_bytes":0`

HTTP Requests and Statistics

HTTP Request Objects

Object Name	Value Type	Description	Example
`request_id`	Integer	Unique identifier for the HTTP request	`"request_id":1`
`session_id`	Integer	ID of the session that the statistics relate to	`"session_id":10`
`request_method`	String	HTTP request method (such as “GET” or “POST”)	`"request_method":"GET"`
`request_scheme`	String	URL scheme of the HTTP request (such as “http” or “https”)	`"request_scheme":"http"`
`request_hostname`	String	Hostname of the HTTP request URL	`"request_hostname":"192.168.0.100"`
`request_port`	String	Port of the HTTP request URL	`"request_port":"80"`
`request_path`	String	Path of the HTTP request URL	`"request_path":"/open/missing"`
`request_query`	String	Query string of the HTTP request URL	`"request_query":"?param=1&arg=2"`
`response_code`	Integer	Status code of the HTTP response	`"response_code":404`
`complete`	Integer	Indicates if the request is complete (1) or active (0)	`"complete":1`

HTTP Request Statistic Objects

Object Name	Value Type	Description	Example
`stat_id`	Integer	Unique identifier for the statistics record	`"stat_id":1`
`request_id`	Integer	ID of the HTTP request that the statistics relate to	`"request_id":1`
`timestamp`	Integer	Timestamp of the statistics in nanoseconds. Timestamps are recorded as the nanoseconds since the Edge SWG appliance last booted up.	`"timestamp":70659151076307`
`action`	Integer	Action that triggered the statistics (represented by a code): 10: Create session 11: Timer update 12: Destroy session 13: Manual update	`"action":13`
`state`	Integer	State of the HTTP request (represented by a code): 10: New connection 11: Client in SSL 12: Client in 13: Cache query 14: Server out 15: Server in SSL 16: Server in 17: Cache hit 18: Client out 19: Log access 20: Destroyed	`"state":10`

Socket Objects and Statistics

Socket Objects

Object Name	Value Type	Description	Example
`socket_id`	Integer	Unique identifier for the socket	`"socket_id":2082`
`ip_version`	Integer	IP version that the socket uses: 2: IPv4 10: IPv6	`"ip_version":2`
`protocol`	Integer	Protocol that the socket uses: 1: TCP 2: UDP	`"protocol":1`
`remote_address`	String	Remote IP address (such as “192.0.2.255”)	`"remote_address":"192.0.2.255"`
`remote_port`	Integer	Remote port number	`"remote_port":443`
`local_address`	String	Local IP address (such as “192.0.2.1”)	`"local_address":"192.0.2.1"`
`local_port`	Integer	Local port number	`"local_port":6394`
`complete`	Integer	Indicates if the socket is complete (1) or active (0)	`"complete":1`

Socket Statistic Objects

Object Name	Value Type	Description	Example
`stat_id`	Integer	Unique identifier for the statistics record	`"stat_id":8313`
`timestamp`	Integer	Timestamp of the statistics in nanoseconds. Timestamps are recorded as the nanoseconds since the Edge SWG appliance last booted up.	`"timestamp":70676174511590`
`socket_id`	Integer	ID of the Edge SWG socket that the statistics relate to	`"socket_id":2082`
`action`	Integer	Action that triggered the statistics (represented by a code): 10: Create session 11: Timer update 12: Destroy session 13: Manual update	`"action":10`
`interface`	Integer	Identifier for the network interface of the Edge SWG socket	`"interface":0`
`state`	Integer	State of the Edge SWG socket (represented by a code): 0: TCPS closed 1: TCPS listen 2: TCPS SYN sent 3: TCPS SYN received 4: TCPS established 5: TCPS close wait 6: TCPS FIN wait 1 7: TCPS closing 8: TCPS last ACK 9: TCPS FIN wait 2 10: TCPS time wait	`"state":0`
`inputs`	Integer	Number of input events for the Edge SWG socket	`"inputs":0`
`outputs`	Integer	Number of output events for the Edge SWG socket	`"outputs":0`
`segments_in`	Integer	Number of segments the Edge SWG socket received	`"segments_in":0`
`segments_out`	Integer	Number of segments the Edge SWG socket sent	`"segments_out":0`
`bytes_in`	Integer	Number of bytes the Edge SWG socket received	`"bytes_in":0`
`bytes_out`	Integer	Number of bytes the Edge SWG socket sent	`"bytes_out":0`
`reassembly_queue_length`	Integer	Length of the reassembly queue for TCP packets at the Edge SWG socket	`"reassembly_queue_length":0`
`sent_unacknowledged`	Integer	Number of TCP bytes that the Edge SWG socket did not acknowledge	`"sent_unacknowledged":0`
`receive_next`	Integer	TCP sequence number that the Edge SWG socket next expected to receive	`"receive_next":0`
`receive_window`	Integer	Window size for the number of TCP bytes the Edge SWG socket received	`"receive_window":0`
`send_window`	Integer	Window size for the number of TCP bytes the Edge SWG socket sent	`"send_window":0`
`congestion_events`	Integer	Number of TCP congestion events (such as duplicate ACKs, ECN notifications, or receiving timeouts) on the Edge SWG socket	`"congestion_events":0`
`congestion_window`	Integer	Window size for TCP congestion events on the Edge SWG socket	`"congestion_window":1073725440`
`round_trip_time`	Integer	Amount of time (in milliseconds) TCP traffic took to complete a round trip	`"round_trip_time":0`
`round_trip_variance`	Integer	Amount of time (in milliseconds) that TCP round-trips varied by. The Round-trip time variation (RTTVAR) is calculated using RFC6298.	`"round_trip_variance":12000`
`packets_retransmitted`	Integer	Number of TCP packets the Edge SWG socket retransmitted	`"packets_retransmitted":0`
`out_of_order`	Integer	Number of TCP packets that arrived at the Edge SWG socket out of order	`"out_of_order":0`
`retransmit_threshold`	Integer	Maximum number of times the Edge SWG socket could attempt to retransmit TCP packets	`"retransmit_threshold":3`

Feedback

thumb_up Yes

thumb_down No