Inquiry Regarding AWS Probe Configuration for EKS
Article ID: 384637
Updated On:
Products
Issue/Introduction
We are currently setting up an account in the aws probe, but we noticed that we are not receiving metrics for the EKS services in AWS.
We have assigned the permissions mentioned in the following KB (https://knowledge.broadcom.com/external/article/217820/uim-aws-probe-ports-connection-permis.html) .
Are these permissions sufficient to retrieve metrics for the EKS service? If not, could you please specify which permission is required for this service?
Additionally, we observed the following error message in the log, <user> is not authorized to perform: cloudwatch: ListMetric with an explicit deny in a service control policy
So it seems it might be related to a missing permission:
Status Code: 403; Erroc code: Access Denied
Could you help us identify the specific permission that might be causing this issue?
Environment
- DX UIM 23.4
Cause
- user access
Resolution
Search for ‘EKS’ and review the available metrics info here in the DX UIM techdocs:
Most of what is needed to monitor EKS is on the AWS side of things.
A "user is not authorized to perform cloudwatch:ListMetrics with an explicit deny in a service control policy 403 Access Denied" error means that your IAM user may be explicitly blocked from executing the "cloudwatch:ListMetrics" action by a service control policy (SCP) in your AWS organization, preventing them from listing CloudWatch metrics, resulting in a "403 Access Denied" error message.
Service Control Policy (SCP):
An SCP is a policy set at the organization level in AWS, which applies to all accounts within that organization and can override individual IAM user policies.
Explicit Deny:
If an SCP contains a "Deny" statement for a specific action like "cloudwatch:ListMetrics", it will take precedence over any "Allow" statements in the user's IAM policy, effectively blocking the action.
Check SCPs:
Access your AWS Organizations console and review the SCPs attached to your organization.
Look for any deny statements that specifically target the "cloudwatch:ListMetrics" action.
Modify SCP:
If necessary, update the SCP to remove the deny statement for "cloudwatch:ListMetrics" or add conditions to allow the action for specific users or scenarios.
Important Considerations:
Policy Evaluation Order:
When evaluating permissions, AWS checks policies in the following order: IAM user policies, IAM role policies, permissions boundaries, and then SCPs.
Review User Policies:
Even if you modify the SCP, ensure that your user's IAM policy also allows the "cloudwatch:ListMetrics" action if needed.
How can I resolve access denied issues caused by AWS service - permissions boundaries?
https://repost.aws/knowledge-center/iam-access-denied-permissions-boundary
Grant IAM users access to Kubernetes with EKS access entries
https://docs.aws.amazon.com/eks/latest/userguide/access-entries.html
For EKS access, it seems you have to assign policy to your Cluster roles for the user.
To grant users permissions to monitor Amazon EKS, you can use AWS Identity and Access Management (IAM) to create access entries and associate policies with them:
- Open the EKS console in AWS
- Select the cluster to grant access to
- Click Access under Cluster
- Click Create access entry
- Search for users in IAM to grant access to the cluster
- Add the user with cluster admin permissions
- Click Add
A deep dive into simplified Amazon EKS access management ...
You can also use the EKS API, AWS Command Line Interface, AWS SDKs, AWS CloudFormation, or AWS Management Console to manage access to the cluster.
Here are some best practices for managing access to Amazon EKS clusters:
- Create clusters with a dedicated IAM role
- Use IAM Roles when multiple users need identical access to the cluster
- Use least privileged access when creating RoleBindings and ClusterRoleBindings
- Regularly audit access to the cluster
Feedback
