Skip to content

fix: How do I create Organization API Key with Organization Billing Admin permission and Project Read Only for projects #1369

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Aug 11, 2023
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .github/workflows/acceptance-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,11 +58,13 @@ jobs:
project:
- 'mongodbatlas/data_source_mongodbatlas_project_invitation*.go'
- 'mongodbatlas/data_source_mongodbatlas_project_ip_access_list*.go'
- 'mongodbatlas/data_source_mongodbatlas_project*.go'
- 'mongodbatlas/data_source_mongodbatlas_project.go'
- 'mongodbatlas/data_source_mongodbatlas_projects.go'
- 'mongodbatlas/resource_mongodbatlas_access_list_api_key*.go'
- 'mongodbatlas/resource_mongodbatlas_project_invitation*.go'
- 'mongodbatlas/resource_mongodbatlas_project_ip_access_list*.go'
- 'mongodbatlas/resource_mongodbatlas_project*.go'
- 'mongodbatlas/resource_mongodbatlas_project.go'
- 'mongodbatlas/resource_mongodbatlas_project_test.go'
serverless:
- 'mongodbatlas/**_serverless**.go'
network:
Expand Down
2 changes: 1 addition & 1 deletion mongodbatlas/data_source_mongodbatlas_project_api_key.go
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ func dataSourceMongoDBAtlasProjectAPIKeyRead(ctx context.Context, d *schema.Reso
return diag.FromErr(fmt.Errorf("error setting `private_key`: %s", err))
}

if projectAssignments, err := newProjectAssignment(ctx, conn, apiKeyID); err == nil {
if projectAssignments, err := newProjectAssignment(ctx, conn, projectID, apiKeyID); err == nil {
if err := d.Set("project_assignment", projectAssignments); err != nil {
return diag.Errorf(errorProjectSetting, `project_assignment`, projectID, err)
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ func flattenProjectAPIKeys(ctx context.Context, conn *matlas.Client, projectID s
"role_names": flattenProjectAPIKeyRoles(projectID, apiKey.Roles),
}

projectAssignment, err := newProjectAssignment(ctx, conn, apiKey.ID)
projectAssignment, err := newProjectAssignment(ctx, conn, projectID, apiKey.ID)
if err != nil {
return nil, err
}
Expand Down
57 changes: 46 additions & 11 deletions mongodbatlas/resource_mongodbatlas_project_api_key.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,14 @@ import (
matlas "go.mongodb.org/atlas/mongodbatlas"
)

const (
orgRolePrefix = "ORG_"
projectRolePrefix = "GROUP_"
orgReadOnlyRole = "ORG_READ_ONLY"
)

var orgRoleProvided = false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a risk in having this global variable being used in separate operations? Set during create, and then used during read and delete.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, I don't this is an issue but I renamed the var to make it more specific to the project API key resource

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My only doubt is that this might work during acceptance tests as the test framework maintains one plugin gRPC server for the duration of each test case, while in normal Terraform operations the plugin server starts and stops (reference).
Approving the PR, I would just try a local build to be sure org roles are supported with no issue.

Copy link
Collaborator Author

@andreaangiolillo andreaangiolillo Aug 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a risk in having this global variable being used in separate operations? Set during create, and then used during read and delete.

I think I misread this comment the first time. The reason why I defined this as a global var was to use it in the create and read operations as I need to know if the user has provided any ORG roles in the config:

The main reason for this is that the CREATE endpoint creates an org API key with ORG_READ_ONLY role if the user did not provide any ORG_ROLES as input. In this scenario, the READ will return the ORG_READ_ONLY role (in addition to the GROUP_* roles) even if the user provided only GROUP roles

while in normal Terraform operations the plugin server starts and stops (reference).

In this case, the READ operation is called as last step of the CREATE so we do not hit this issue

Copy link
Collaborator Author

@andreaangiolillo andreaangiolillo Aug 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested the changes with the Terraform configurations provided in the description of the PR ( ORG_READ_ONLY is there) and in this comment #1369 (comment). Let me know if I should test locally another scenario

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested with the following example:

resource "mongodbatlas_project_api_key" "api_1" {
  description = "test api_key multii"
  project_id  = "<project_id>"

  project_assignment {
    project_id = "<project_id>"
    role_names = ["ORG_READ_ONLY", "GROUP_READ_ONLY"]
  }
}
  • terraform apply creates the api key successfully.
  • then running terraform plan outputs incorrect diff showing that "ORG_READ_ONLY" will be added.

I believe the projectAPIKeyOrgRoleProvided is not preserving its value during the create and following read operation done in terraform plan, so it filters out the org role.

Let me know if you are able to reproduce the same, it is indeed an edge case but might be worth addressing.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not able to reproduce the issue

apply

tfa                                                                                                  6s  1.20.33.1.116.18.0 bazel 5.4.0 ﴃ cloud-local
mongodbatlas_project.atlas-project: Refreshing state... [id=64d4acf6bf469a0c1549c869]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # mongodbatlas_project_api_key.api_1 will be created
  + resource "mongodbatlas_project_api_key" "api_1" {
      + api_key_id  = (known after apply)
      + description = "test api_key multii"
      + id          = (known after apply)
      + private_key = (sensitive value)
      + project_id  = "64d4acf6bf469a0c1549c869"
      + public_key  = (known after apply)

      + project_assignment {
          + project_id = "64d4acf6bf469a0c1549c869"
          + role_names = [
              + "GROUP_READ_ONLY",
              + "ORG_READ_ONLY",
            ]
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.
mongodbatlas_project_api_key.api_1: Creating...
mongodbatlas_project_api_key.api_1: Creation complete after 1s [id=YXBpX2tleV9pZA==:NjRkNGFkMWJlODA3YjAzNWZjMjlmNTY2-cHJvamVjdF9pZA==:NjRkNGFjZjZiZjQ2OWEwYzE1NDljODY5]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

plan:

tf plan                                                                                              3s  1.20.33.1.116.18.0 bazel 5.4.0 ﴃ cloud-local
mongodbatlas_project.atlas-project: Refreshing state... [id=64d4acf6bf469a0c1549c869]
mongodbatlas_project_api_key.api_1: Refreshing state... [id=YXBpX2tleV9pZA==:NjRkNGFkMWJlODA3YjAzNWZjMjlmNTY2-cHJvamVjdF9pZA==:NjRkNGFjZjZiZjQ2OWEwYzE1NDljODY5]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was able to reproduce the issue: the reason why it did not "fail" during the plan is that I was using the provider in debug mode. Thanks for the discussion here. I will see if I find a way around

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, I have not found a way to allow the use of ORG_READ_ONLY since it is automatically added when ORG roles are not provided. As a result, I updated the logic to now allow the ORG_READ_ONLY for project assignments and update the documentation to explain why. This should not be a breaking change since customers were not able to use this resource with org roles.


func resourceMongoDBAtlasProjectAPIKey() *schema.Resource {
return &schema.Resource{
CreateContext: resourceMongoDBAtlasProjectAPIKeyCreate,
Expand Down Expand Up @@ -78,7 +86,7 @@ func resourceMongoDBAtlasProjectAPIKey() *schema.Resource {
}

type APIProjectAssignmentKeyInput struct {
ProjectID string `json:"desc,omitempty"`
ProjectID string `json:"projectId,omitempty"`
RoleNames []string `json:"roles,omitempty"`
}

Expand All @@ -97,6 +105,7 @@ func resourceMongoDBAtlasProjectAPIKeyCreate(ctx context.Context, d *schema.Reso
for _, apiKeyList := range projectAssignmentList {
if apiKeyList.ProjectID == projectID {
createRequest.Roles = apiKeyList.RoleNames
orgRoleProvided = apiKeyList.findOrgRole()
apiKey, resp, err = conn.ProjectAPIKeys.Create(ctx, projectID, createRequest)
if err != nil {
if resp != nil && resp.StatusCode == http.StatusNotFound {
Expand All @@ -123,7 +132,6 @@ func resourceMongoDBAtlasProjectAPIKeyCreate(ctx context.Context, d *schema.Reso
}
} else {
createRequest.Roles = expandStringList(d.Get("role_names").(*schema.Set).List())

apiKey, resp, err = conn.ProjectAPIKeys.Create(ctx, projectID, createRequest)
if err != nil {
if resp != nil && resp.StatusCode == http.StatusNotFound {
Expand Down Expand Up @@ -186,7 +194,7 @@ func resourceMongoDBAtlasProjectAPIKeyRead(ctx context.Context, d *schema.Resour
if err := d.Set("role_names", nil); err != nil {
return diag.FromErr(fmt.Errorf("error setting `roles`: %s", err))
}
if projectAssignments, err := newProjectAssignment(ctx, conn, apiKeyID); err == nil {
if projectAssignments, err := newProjectAssignment(ctx, conn, projectID, apiKeyID); err == nil {
if err := d.Set("project_assignment", projectAssignments); err != nil {
return diag.Errorf(errorProjectSetting, `created`, projectID, err)
}
Expand Down Expand Up @@ -317,7 +325,7 @@ func resourceMongoDBAtlasProjectAPIKeyDelete(ctx context.Context, d *schema.Reso
return diag.FromErr(fmt.Errorf("error getting api key information: %s", err))
}

projectAssignments, err := getAPIProjectAssignments(ctx, conn, apiKeyOrgList, apiKeyID)
projectAssignments, err := getAPIProjectAssignments(ctx, conn, projectID, apiKeyOrgList, apiKeyID)
if err != nil {
return diag.FromErr(fmt.Errorf("error getting api key information: %s", err))
}
Expand Down Expand Up @@ -386,7 +394,7 @@ func flattenProjectAPIKeyRoles(projectID string, apiKeyRoles []matlas.AtlasRole)
flattenedOrgRoles := []string{}

for _, role := range apiKeyRoles {
if strings.HasPrefix(role.RoleName, "GROUP_") && role.GroupID == projectID {
if role.GroupID == projectID {
flattenedOrgRoles = append(flattenedOrgRoles, role.RoleName)
}
}
Expand All @@ -408,13 +416,13 @@ func ExpandProjectAssignmentSet(projectAssignments *schema.Set) []*APIProjectAss
return res
}

func newProjectAssignment(ctx context.Context, conn *matlas.Client, apiKeyID string) ([]map[string]interface{}, error) {
func newProjectAssignment(ctx context.Context, conn *matlas.Client, projectID, apiKeyID string) ([]map[string]interface{}, error) {
apiKeyOrgList, _, err := conn.Root.List(ctx, nil)
if err != nil {
return nil, fmt.Errorf("error getting api key information: %s", err)
}

projectAssignments, err := getAPIProjectAssignments(ctx, conn, apiKeyOrgList, apiKeyID)
projectAssignments, err := getAPIProjectAssignments(ctx, conn, projectID, apiKeyOrgList, apiKeyID)
if err != nil {
return nil, fmt.Errorf("error getting api key information: %s", err)
}
Expand Down Expand Up @@ -467,25 +475,42 @@ func getStateProjectAssignmentAPIKeys(d *schema.ResourceData) (newAPIKeys, chang
return
}

func getAPIProjectAssignments(ctx context.Context, conn *matlas.Client, apiKeyOrgList *matlas.Root, apiKeyID string) ([]APIProjectAssignmentKeyInput, error) {
func getAPIProjectAssignments(ctx context.Context, conn *matlas.Client, projectIDUsedToCreateAPIKeys string, apiKeyOrgList *matlas.Root, apiKeyID string) ([]APIProjectAssignmentKeyInput, error) {
projectAssignments := []APIProjectAssignmentKeyInput{}
for idx, role := range apiKeyOrgList.APIKey.Roles {
if strings.HasPrefix(role.RoleName, "ORG_") {
orgKeys, _, err := conn.APIKeys.List(ctx, apiKeyOrgList.APIKey.Roles[idx].OrgID, nil)
if err != nil {
return nil, fmt.Errorf("error getting api key information: %s", err)
}

for _, val := range orgKeys {
if val.ID == apiKeyID {
for _, r := range val.Roles {
temp := new(APIProjectAssignmentKeyInput)
if strings.HasPrefix(r.RoleName, "GROUP_") {
roles := map[string]string{}
if strings.HasPrefix(r.RoleName, projectRolePrefix) {
temp.ProjectID = r.GroupID
for _, l := range val.Roles {
if l.GroupID == temp.ProjectID {
temp.RoleNames = append(temp.RoleNames, l.RoleName)
if l.GroupID == temp.ProjectID || (l.GroupID == "" && temp.ProjectID == projectIDUsedToCreateAPIKeys) {
roles[l.RoleName] = l.RoleName
}
}

tempRoleList := make([]string, 0, len(roles))
for k := range roles {
if !orgRoleProvided && k == orgReadOnlyRole {
// When the user does not provide org roles
// the API key POST endpoing creates an org api key with
// the role ORG_READ_ONLY. We want to remove this from the state
// since the user did not provided it
continue
}

tempRoleList = append(tempRoleList, k)
}

temp.RoleNames = tempRoleList
projectAssignments = append(projectAssignments, *temp)
}
}
Expand All @@ -496,3 +521,13 @@ func getAPIProjectAssignments(ctx context.Context, conn *matlas.Client, apiKeyOr
}
return projectAssignments, nil
}

func (apiKey *APIProjectAssignmentKeyInput) findOrgRole() bool {
for _, r := range apiKey.RoleNames {
if strings.HasPrefix(r, orgRolePrefix) {
return true
}
}

return false
}
70 changes: 70 additions & 0 deletions mongodbatlas/resource_mongodbatlas_project_api_key_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,42 @@ func TestAccConfigRSProjectAPIKey_RecreateWhenDeletedExternally(t *testing.T) {
})
}

func TestAccConfigRSProjectAPIKey_OrgRoles(t *testing.T) {
var (
resourceName = "mongodbatlas_project_api_key.test"
dataSourceName = "data.mongodbatlas_project_api_key.test"
dataSourcesName = "data.mongodbatlas_project_api_keys.test"
orgID = os.Getenv("MONGODB_ATLAS_ORG_ID")
firstProjectName = acctest.RandomWithPrefix("test-acc")
secondProjectName = acctest.RandomWithPrefix("test-acc")
description = fmt.Sprintf("test-acc-project-api_key-%s", acctest.RandString(5))
)

resource.Test(t, resource.TestCase{
PreCheck: func() { testAccPreCheckBasic(t) },
ProviderFactories: testAccProviderFactories,
CheckDestroy: testAccCheckMongoDBAtlasProjectAPIKeyDestroy,
Steps: []resource.TestStep{
{
Config: testAccMongoDBAtlasProjectAPIKeyConfigOrgRoles(orgID, firstProjectName, secondProjectName, description),
Check: resource.ComposeTestCheckFunc(
resource.TestCheckResourceAttrSet(resourceName, "project_id"),
resource.TestCheckResourceAttrSet(resourceName, "description"),
resource.TestCheckResourceAttr(resourceName, "description", description),
resource.TestCheckResourceAttrSet(resourceName, "project_assignment.0.project_id"),
resource.TestCheckResourceAttrSet(resourceName, "project_assignment.0.role_names.0"),
resource.TestCheckResourceAttrSet(dataSourceName, "project_assignment.0.project_id"),
resource.TestCheckResourceAttrSet(dataSourceName, "project_assignment.0.role_names.0"),
resource.TestCheckResourceAttrSet(dataSourceName, "project_id"),
resource.TestCheckResourceAttrSet(dataSourceName, "description"),
resource.TestCheckResourceAttrSet(dataSourcesName, "results.0.project_assignment.0.project_id"),
resource.TestCheckResourceAttrSet(dataSourcesName, "results.0.project_assignment.0.role_names.0"),
),
},
},
})
}

func deleteAPIKeyManually(orgID, descriptionPrefix string) error {
conn := testAccProvider.Meta().(*MongoDBClient).Atlas
list, _, err := conn.APIKeys.List(context.Background(), orgID, &matlas.ListOptions{})
Expand Down Expand Up @@ -268,3 +304,37 @@ func testAccMongoDBAtlasProjectAPIKeyConfigMultiple(orgID, projectName, descript

`, orgID, projectName, description, roleNames)
}

func testAccMongoDBAtlasProjectAPIKeyConfigOrgRoles(orgID, firstProjectName, secondProjectName, description string) string {
return fmt.Sprintf(`
resource "mongodbatlas_project" "test" {
name = %[2]q
org_id = %[1]q
}

resource "mongodbatlas_project" "testProject" {
name = %[3]q
org_id = %[1]q
}

resource "mongodbatlas_project_api_key" "test" {
project_id = mongodbatlas_project.test.id
description = %[4]q
project_assignment {
project_id = mongodbatlas_project.test.id
role_names = ["ORG_READ_ONLY", "ORG_BILLING_ADMIN", "GROUP_READ_ONLY"]
}

}

data "mongodbatlas_project_api_key" "test" {
project_id = mongodbatlas_project.test.id
api_key_id = mongodbatlas_project_api_key.test.api_key_id
}

data "mongodbatlas_project_api_keys" "test" {
project_id = mongodbatlas_project.test.id
}

`, orgID, firstProjectName, secondProjectName, description)
}
33 changes: 32 additions & 1 deletion website/docs/r/project_api_key.html.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -35,13 +35,43 @@ resource "mongodbatlas_project_api_key" "test" {
}

project_assignment {
project_id = "74259ee860c43338194b0f8e"
project_id = "64229ee820c42228194b0f4a"
role_names = ["GROUP_READ_ONLY"]
}

}
```

## Example Usage - Create Org PAK and Assign it to Multiple Projects

```terraform
resource "mongodbatlas_project" "atlas-project" {
name = "ProjectTest"
org_id = "60ddf55c27a5a20955a707d7"
}

resource "mongodbatlas_project_api_key" "api_1" {
description = "test api_key multi"
project_id = mongodbatlas_project.atlas-project.id

// NOTE: The `project_id` of the first `project_assignment` element must be the same as the `project_id` of the resource.
project_assignment {
project_id = mongodbatlas_project.atlas-project.id
role_names = ["ORG_READ_ONLY", "ORG_BILLING_ADMIN", "GROUP_READ_ONLY"]
}

project_assignment {
project_id = "63dcfc256af00a5934e60924"
role_names = ["GROUP_READ_ONLY"]
}

project_assignment {
project_id = "64c23af6f133166c39176cbf"
role_names = ["GROUP_OWNER"]
}
}
```

## Argument Reference

* `project_id` -Unique 24-hexadecimal digit string that identifies your project.
Expand All @@ -56,6 +86,7 @@ List of Project roles that the Programmatic API key needs to have. `project_assi
* `project_id` - (Required) Project ID to assign to Access Key
* `role_names` - (Required) List of Project roles that the Programmatic API key needs to have. Ensure you provide: at least one role and ensure all roles are valid for the Project. You must specify an array even if you are only associating a single role with the Programmatic API key. The [MongoDB Documentation](https://www.mongodb.com/docs/atlas/reference/user-roles/#project-roles) describes the valid roles that can be assigned.

~> **NOTE:** The `project_id` of the first `project_assignment` element must be the same as the `project_id` of the resource.
## Attributes Reference

In addition to all arguments above, the following attributes are exported:
Expand Down