Dumping Data to Another Account

Constraints

• You can create a maximum of 20 data dump tasks per account.
• Data dump is available only for whitelisted customers.

Prerequisites

• Before adding a data dump task, you need to create two agencies and grant them fine-grained DMS and IAM permissions. The agencies are as follows:
  • DMS resource query agency

  The delegating party creates a delegator account for the delegated party. This account is used to obtain the project list, DMS instance list, and topic list from the delegating party for a data dump task on the Cloud Eye console.

  • Cloud Eye account agency
  The delegating party creates an agency for the op_svc_ces account. This agency is used to dump metric data of the delegated party to the delegating DMS instance.

  

  A delegating party is an account that owns DMS resources, and a delegated party is an account that has the metric data to be dumped.

• If the delegated account is a member account, you need to configure the permissions for data dump.

Creating a Custom Policy for DMS

1. Log in to the IAM console using a delegating account.
2. On the IAM console, choose Permissions > Policies/Roles in the navigation pane, and click Create Custom Policy in the upper right corner.
3. Enter a policy name.
4. Select Visual editor for Policy View.
5. Configure a policy in Policy Content.
   1. Select Allow.
   2. Select Cloud Service, enter Distributed Message Service or DMS in the search box, and click Distributed Message Service (DMS).
   3. Select Actions. In the search box, enter dms:instance:get and dms:instance:list and select them.
   4. Click OK. The custom policy is created.

Creating the DMS Resource Query Agency and Assigning Permissions

1. Log in to the IAM console using a delegating account.
2. On the IAM console, choose Agencies in the navigation pane, and click Create Agency in the upper right corner.
3. Enter an agency name.
4. Specify the agency type as Account, and enter the name of a delegated account.
5. Select a validity period and enter a description.
6. Click OK.
7. In the displayed dialog box, click Authorize.
8. Select the custom policy created in Creating a Custom Policy for DMS, click Next, and set the authorization scope.
9. Click OK. The agency is created.

Creating the Cloud Eye Account Agency and Assigning Permissions

1. Log in to the Cloud Eye console using the delegator account.
2. Create a data dump task by referring to Dumping Data to the Current Account. The Cloud Eye account agency is automatically created and authorized.

Required Permissions for the Delegated Party

If the delegated account is a master account, no permission needs to be configured. If the delegated account is a member account, contact the account administrator to grant data dump permissions to the member account.

1. Log in to the IAM console using the administrator account of the delegated party.
2. Create project-level and global-level custom policies in the JSON view.

   For details about how to create project-level and global-level permissions, see Creating a Custom Policy.

3. After the custom policies are created, grant permissions to the delegated account by referring to Assigning Permissions to an IAM User.

{
  "Version": "1.1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ces:quotas:get",
        "ces:dataShareJob:get",
        "ces:dataShareTask:delete",
        "ces:dataShareJob:action",
        "ces:dataShareTask:list",
        "ces:namespaces:list",
        "ces:sysEventsNames:list",
        "ces:dataShareTask:get",
        "ces:dataShareTask:action",
        "ces:dataShareJob:list",
        "ces:dataShareTask:put",
        "ces:dataShareTask:create",
        "ces:dataShareJob:action",
        "ces:dataShareJob:delete",
        "ces:dataShareJob:create",
        "dms:instance:list",
        "dms:instance:get",
        "ces:dataShareJob:listDmsInstancesByAgency",
        "ces:dataShareJob:listAgencyProjects",
        "ces:dataShareJob:listDmsTopicsByAgency",
        "ces:agency:get",
        "ces:agency:post",
        "ces:namespacesDimensions:list",
        "mqs:instance:list",
        "mqs:instance:get",
        "ces:i18n:list"
      ]
    }
  ]
}

{
  "Version": "1.1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "iam:agencies:assume",
        "iam:agencies:createAgency",
        "iam:agencies:listAgencies",
        "iam:permissions:grantRoleToAgency",
        "iam:permissions:grantRoleToAgencyOnProject",
        "iam:permissions:listRolesForAgency",
        "iam:permissions:listRolesForAgencyOnDomain",
        "iam:permissions:listRolesForAgencyOnProject",
        "iam:permissions:revokeRoleFromAgency",
        "iam:roles:createRole",
        "iam:roles:listRoles",
        "iam:roles:updateRole"
      ]
    }
  ]
}

Configuring a Data Dump Task by the Delegated Party

After the agency and fine-grained authorization are complete, you can configure a data dump task for the delegated party. The procedure is as follows:

1. Log in to the Cloud Eye console using the delegated account.
2. In the navigation pane, choose Data Dump.
3. Click Add Dump Task.
4. On the displayed page, set parameters.

   Table 1 Parameters for configuring a dump task

   Parameter	Description
   Name	Dump task name. Only letters, digits, underscores (_), and hyphens (-) are allowed. You can enter 1 to 64 characters. Example value: dataShareJob-ECSMetric
   Resource Type	Type of resources monitored by Cloud Eye. Example value: Elastic Cloud Server
   Dimension	Dimension of the monitored object. For details about the dimensions of monitored objects of each service, see Cloud Product Metrics. If All is selected, all metrics of the selected service will be dumped to DMS for Kafka. If ECSs is selected, ECS metrics will be dumped to DMS for Kafka. Example value: All
   Monitoring Scope	The scope can only be All resources, indicating that all metrics of the specified monitored object will be dumped to DMS for Kafka.
   Resource Type	Resource type. Only Distributed Message Service for Kafka is supported.
   Destination	Destination location for saving dumped data. You can select Other account.
   Delegator Account	Delegator account ID. For details, see Appendix.
   Agency Name	Agency name of the delegated account. For details, see Appendix.
   Project Name	Project of the resource.
   Kafka	Name of the Kafka instance where metric data is dumped to. If no Kafka instance is available, see Buying a Kafka Instance.
   Topic	Name of the topic to which data is dumped. If no topic is available, see Creating a Topic.

   Figure 1 Configuring a dump task
   
5. After setting the parameters, click Add to create a dump task.
   

   You can query the dumped data in DMS for Kafka using the delegator account. For details, see Viewing Kafka Messages.

Appendix

To search for an agency, perform the following steps:

1. Log in to the Cloud Eye console using the delegator account.
2. Hover your mouse over the username in the upper right corner and choose Identity and Access Management.

   

3. In the navigation pane, choose Agencies.
   Figure 2 Viewing agencies
   

To search for an account ID, perform the following steps:

1. Log in to the Cloud Eye console using the delegator account.
2. In the upper right corner, hover the mouse over the username and click My Credentials.

   

3. On the My Credentials page, obtain the Account ID.
   Figure 3 My credentials
   

Helpful Links

What Should I Do When There Is an Abnormal Dump Destination?