Pulse Hosted on a Web Application Server When GemFire Integrated Security Is Enabled

Products

• VMware Tanzu Gemfire

Issue/Introduction

Symptoms:
When Pulse is Hosted on a Web Application Server and GemFire Integrated Security has been enabled on the cluster, the application fails to connect and the following error is shown in PULSE logs:

2018-10-29 16:50:00,099 FATAL o.a.g.t.p.i.d.JMXDataUpdater [http-nio-8080-exec-1] null
java.lang.NullPointerException: null
	at java.util.Hashtable.put(Hashtable.java:460) ~[?:1.8.0_181]
	at java.util.Properties.setProperty(Properties.java:166) ~[?:1.8.0_181]
	at org.apache.geode.internal.security.shiro.JMXShiroAuthenticator.authenticate(JMXShiroAuthenticator.java:56) ~[?:?]
	at javax.management.remote.rmi.RMIServerImpl.doNewClient(RMIServerImpl.java:232) ~[?:1.8.0_181]
	at javax.management.remote.rmi.RMIServerImpl.newClient(RMIServerImpl.java:199) ~[?:1.8.0_181]
	at sun.reflect.GeneratedMethodAccessor155.invoke(Unknown Source) ~[?:?]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_181]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_181]
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357) ~[?:1.8.0_181]
	at sun.rmi.transport.Transport$1.run(Transport.java:200) ~[?:1.8.0_181]
	at sun.rmi.transport.Transport$1.run(Transport.java:197) ~[?:1.8.0_181]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_181]
	at sun.rmi.transport.Transport.serviceCall(Transport.java:196) ~[?:1.8.0_181]
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573) ~[?:1.8.0_181]
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834) ~[?:1.8.0_181]
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688) ~[?:1.8.0_181]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_181]
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687) ~[?:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_181]
	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_181]
	at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:283) ~[?:1.8.0_181]
	at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:260) ~[?:1.8.0_181]
	at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161) ~[?:1.8.0_181]
	at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source) ~[?:1.8.0_181]
	at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2430) ~[?:1.8.0_181]
	at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308) ~[?:1.8.0_181]
	at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) ~[?:1.8.0_181]
	at org.apache.geode.tools.pulse.internal.data.JMXDataUpdater.connect(JMXDataUpdater.java:226) [classes/:?]
	at org.apache.geode.tools.pulse.internal.data.Cluster.connectToGemFire(Cluster.java:2789) [classes/:?]
	at org.apache.geode.tools.pulse.internal.data.Repository.getCluster(Repository.java:139) [classes/:?]
	at org.apache.geode.tools.pulse.internal.data.Repository.getCluster(Repository.java:128) [classes/:?]
	at org.apache.geode.tools.pulse.internal.service.ClusterDiskThroughputService.execute(ClusterDiskThroughputService.java:48) [classes/:?]
	at org.apache.geode.tools.pulse.internal.controllers.PulseController.getPulseUpdate(PulseController.java:106) [classes/:?]
	at sun.reflect.GeneratedMethodAccessor47.invoke(Unknown Source) ~[?:?]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_181]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_181]
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) [spring-web-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) [spring-web-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) [spring-webmvc-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) [spring-webmvc-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) [spring-webmvc-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) [spring-webmvc-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967) [spring-webmvc-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901) [spring-webmvc-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) [spring-webmvc-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872) [spring-webmvc-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:660) [servlet-api.jar:?]
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) [spring-webmvc-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) [servlet-api.jar:?]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [catalina.jar:9.0.12]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:9.0.12]
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) [tomcat-websocket.jar:9.0.12]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:9.0.12]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:9.0.12]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:66) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177) [spring-security-web-4.2.7.RELEASE.jar:4.2.7.RELEASE]
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347) [spring-web-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263) [spring-web-4.3.18.RELEASE.jar:4.3.18.RELEASE]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [catalina.jar:9.0.12]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [catalina.jar:9.0.12]
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) [catalina.jar:9.0.12]
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [catalina.jar:9.0.12]
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:490) [catalina.jar:9.0.12]
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) [catalina.jar:9.0.12]
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [catalina.jar:9.0.12]
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:668) [catalina.jar:9.0.12]
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) [catalina.jar:9.0.12]
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) [catalina.jar:9.0.12]
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408) [tomcat-coyote.jar:9.0.12]
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-coyote.jar:9.0.12]
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:770) [tomcat-coyote.jar:9.0.12]
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1415) [tomcat-coyote.jar:9.0.12]
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-coyote.jar:9.0.12]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-util.jar:9.0.12]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]

Environment

Cause

Out of the box, PULSE has three different authentication-manager implementations:

<!-- Default user authentication -->
<beans:beans profile="pulse.authentication.default">
	<authentication-manager>
		<authentication-provider>
			<user-service>
				<user name="admin" password="admin" authorities="ROLE_CLUSTER:READ,ROLE_DATA:READ"/>
			</user-service>
		</authentication-provider>
	</authentication-manager>
</beans:beans>
	
<!-- User Authentication based on GemFire integrated security -->
<beans:beans profile="pulse.authentication.gemfire">
	<authentication-manager alias="authenticationManager">
		<authentication-provider ref="gemAuthenticationProvider"/>
	</authentication-manager>
</beans:beans>

<!-- Custom user authentication specified externally -->
<beans:beans profile="pulse.authentication.custom">
	<beans:import resource="classpath:pulse-authentication-custom.xml" />
</beans:beans>

The default one (pulse.authentication.default) is used when there is no security manager installed for the cluster, and the second one (pulse.authentication.gemfire)is used when the integrated-security is turned on.

When pulse is deployed as an embedded service inside the locator, it has the knowledge about the cluster it needs to connect, so it knows which authentication-manager to use. But if PULSE is hosted on a Web Application Server, it does not know what cluster it will be used to connect and whether or not that cluster has security turned on, hence it needs some extra configuration steps.

Resolution

Set the spring.profiles.active environment variable as pulse.authentication.gemfire.
As an example, if Apache Tomcat  is the Web Application Server on which PULSE is deployed, the following steps need to be executed:

1. Create the file "setenv.sh" under the bin directory, within the Apache Tomcat installation folder.
2. Add JAVA_OPTS="$JAVA_OPTS -Dspring.profiles.active=pulse.authentication.gemfire" to the recently created file.