Kubernetes agent showing error messages in agent log

book

Article ID: 77019

calendar_today

Updated On:

Products

CA Application Performance Management SaaS

Issue/Introduction

Customer has enabled the Kubernetes monitoring agent for APM Saas environment. The Kubernetes monitoring agent is running ok, and agent is also reporting data to APM Saas environment. But the agent is encountering many error messages which is showing in Kubernetes agent log.

[INFO] [IntroscopeAgent.OpenShiftMonitor] Trying to connect https://xxxx.xxxx.xxxx.azure.com using API Token
io.fabric8.kubernetes.client.KubernetesClientException: An error has occurred.
    at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:57)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getRootPaths(BaseOperation.java:190)
    at io.fabric8.kubernetes.client.BaseClient.rootPaths(BaseClient.java:150)
    at io.fabric8.openshift.client.OpenshiftAdapterSupport.isOpenShift(OpenshiftAdapterSupport.java:51)
    at io.fabric8.openshift.client.OpenshiftAdapterSupport.isAdaptable(OpenshiftAdapterSupport.java:35)
    at io.fabric8.kubernetes.client.AutoAdaptableKubernetesClient.adapt(AutoAdaptableKubernetesClient.java:107)
    at io.fabric8.kubernetes.client.AutoAdaptableKubernetesClient.<init>(AutoAdaptableKubernetesClient.java:96)
    at com.ca.apm.openshift.OpenShiftMonitorService.IAgentService_startService(OpenShiftMonitorService.java:156)
    at com.wily.introscope.agent.service.ServiceAdministrator$ServiceInitializer.startService(ServiceAdministrator.java:604)
    at com.wily.introscope.agent.service.ServiceAdministrator$ServiceInitializer.startServiceSafe(ServiceAdministrator.java:618)
    at com.wily.introscope.agent.service.ServiceAdministrator$ServiceInitializer.IExecutableItem_execute(ServiceAdministrator.java:482)
    at com.wily.util.task.AsynchExecutionQueue$WrapperItem.IExecutableItem_execute(AsynchExecutionQueue.java:414)
    at com.wily.util.task.AsynchExecutionQueue.doTask(AsynchExecutionQueue.java:205)
    at com.wily.util.task.ATask$CoreTask.run(ATask.java:132)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.UnknownHostException: xxxx.xxxx.xxxx.azure.com
    at java.net.InetAddress.getAllByName0(InetAddress.java:1280)
    at java.net.InetAddress.getAllByName(InetAddress.java:1192)
    at java.net.InetAddress.getAllByName(InetAddress.java:1126)
    at okhttp3.Dns$1.lookup(Dns.java:39)
    at okhttp3.internal.connection.RouteSelector.resetNextInetSocketAddress(RouteSelector.java:172)
    at okhttp3.internal.connection.RouteSelector.nextProxy(RouteSelector.java:138)
    at okhttp3.internal.connection.RouteSelector.next(RouteSelector.java:80)
    at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:178)
    at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:129)
    at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:98)
    at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
    at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:109)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
    at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
    at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:124)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
    at io.fabric8.kubernetes.client.utils.HttpClientUtils$3.intercept(HttpClientUtils.java:102)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
    at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:170)
    at okhttp3.RealCall.execute(RealCall.java:60)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:235)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:230)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getRootPaths(BaseOperation.java:183)
    ... 13 more

[ERROR] [IntroscopeAgent.OpenShiftMonitor] Failed to connect to https://xxxx.xxxx.xxxx.azure.com

[ERROR] [IntroscopeAgent.InfrastructureAgent Agent] Failed to re-register IAgentRemoteConfigurationService on Agent name change event.

 

Cause

It looks like host name is not being resolved properly. 
This can be because of the way Kubernetes network is setup from the container..

Environment

APM Kubernetes monitoring agent 10.7
APM Saas

Resolution

We suggest you to use the IP address for host xxxx.xxxx.xxxx.azure.com  which is showing up in above error message.