GPCC agent PANIC with GPCC 6.9.1 running on GPDB 6.27.1

Products

VMware Tanzu Data Suite VMware Tanzu Greenplum VMware Tanzu Greenplum / Gemfire VMware Tanzu Data Suite

Issue/Introduction

you might notice GPCC agents not starting, getting PANIC and data not populating within GPCC 6.9.1.

Here are few common errors seen in the webserver.log and cli.logs:

webserver.log

2024-07-31 16:43:54 [WARNING]AgentManager: agent pkt version incompatible, SupportedPktVersion: 8 - 14, agent pkt versions on hosts:

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

cli.logs

agent.log



panic: runtime error: slice bounds out of range [:13191] with length 64

goroutine 184 [running]:
github.com/pivotal/gp-command-center/backend/mcollector/packets.(*Initer).GetVarNameData(0xc0002ae000?, 0x0?, 0x0?)
    /tmp/build/ee3e6707/gopath/src/gpcc/backend/mcollector/packets/packet_initer.go:151 +0xe5
github.com/pivotal/gp-command-center/backend/mcollector/packets.(*MQlog).Init(0xc0002ae000, 0xc00013c4d8?, 0x0)
    /tmp/build/ee3e6707/gopath/src/gpcc/backend/mcollector/packets/udp_qlog.go:93 +0x2cb
github.com/pivotal/gp-command-center/backend/mcollector/agent/querypit.contentUnmarshallerForPkt(0xc0003020d8, 0xf, 0xdbfece?, 0xc0002ae000, 0xc00059dc50?, 0xe5?)
    /tmp/build/ee3e6707/gopath/src/gpcc/backend/mcollector/agent/querypit/unmarshal_functions.go:210 +0x126
github.com/pivotal/gp-command-center/backend/mcollector/agent/querypit.handleOneMsg({0xc00072c000, 0xe7e, 0x20000}, 0xe7e, {0xed8900, 0xc00055a000}, 0x1?)
    /tmp/build/ee3e6707/gopath/src/gpcc/backend/mcollector/agent/querypit/unmarshal_functions.go:131 +0xaba
github.com/pivotal/gp-command-center/backend/mcollector/agent/querypit.RecvAndUnmarshal({0xed0400, 0xc0003d4020}, {0xed8900, 0xc00055a000})
    /tmp/build/ee3e6707/gopath/src/gpcc/backend/mcollector/agent/querypit/unmarshal_functions.go:187 +0x75
github.com/pivotal/gp-command-center/backend/mcollector/agent/querypit.DomainSocketSource.Run({{0xedc2a0?, 0xc0003d4020?}}, {0xc00012d770, 0xba615a?, 0xdcc279?})
    /tmp/build/ee3e6707/gopath/src/gpcc/backend/mcollector/agent/querypit/domain_socket_source.go:26 +0x122
github.com/pivotal/gp-command-center/backend/mcollector/agent/querypit.newUnixDomainConn({0xed8900?, 0xc00055a000}, {0xedc2a0, 0xc0003d4020}, 0xe04e58)
    /tmp/build/ee3e6707/gopath/src/gpcc/backend/mcollector/agent/querypit/domain_socket_source.go:33 +0x14c
created by github.com/pivotal/gp-command-center/backend/mcollector/agent/querypit.StartUnixDomainSocketServer.func1
    /tmp/build/ee3e6707/gopath/src/gpcc/backend/mcollector/agent/querypit/domain_socket_helper_functions_linux.go:103 +0x28f

Environment

GPDB: 6.27.1

GPCC: 6.9.1

Cause

GPCC 6.9.1 was released long time ago and it cannot be used with GPDB new version 6.27.1 because the metrics collector version of GPDB 6.27.1 is incompatible with GPCC 6.9.1. This may cause unexpected errors, including missing data. The incompatible version of the metrics collector caused agent panic.

Resolution

Upgrading GPCC 6.9.1 to 6.11.1 or higher will resolve issues with agent not starting, PANICs and data not populating