How can the data engineer set up access to meet these requirements?
Create an S3 bucket for each dataset. Create an ACL for each S3 bucket. For each S3 bucket that contains a sensitive dataset, set the ACL to allow access only from the Finance department user group. Allow all three department user groups to access each S3 bucket that contains a non-sensitive dataset.
Create an S3 bucket for each dataset. For each S3 bucket that contains a sensitive dataset, set the bucket policy to allow access only from the Finance department user group. Allow all three department user groups to access each S3 bucket that contains a non-sensitive dataset.
Create a single S3 bucket that includes two folders to separate the sensitive datasets from the non-sensitive datasets. For the Finance department user group, attach an IAM policy that provides access to both folders. For the Marketing and Human Resources department user groups, attach an IAM policy that provides access to only the folder that contains the non-sensitive datasets.
Create a single S3 bucket that includes two folders to separate the sensitive datasets from the non-sensitive datasets. Set the policy for the S3 bucket to allow only the Finance department user group to access the folder that contains the sensitive datasets. Allow all three department user groups to access the folder that contains the non-sensitive datasets.
Explanations:
ACLs are not ideal for managing permissions at the group level, and managing them for each bucket can be cumbersome. Using IAM policies or bucket policies is a more scalable solution for controlling access.
While bucket policies can be used for controlling access, it’s not optimal to create a separate bucket for each dataset. A single bucket with folder-level policies is more efficient.
This option uses IAM policies to control access at the folder level within a single S3 bucket. The Finance department has access to both folders, while Marketing and HR only have access to the non-sensitive folder. This is efficient and scalable.
While the folder separation is a good approach, setting a policy on the bucket itself to manage access is less flexible than using IAM policies attached to user groups. Policies at the user or group level are preferred for fine-grained control.
I would say the answer is:
Create a single S3 bucket that includes two folders to separate the sensitive datasets from the non-sensitive datasets. For the Finance department user group, attach an IAM policy that provides access to both folders. For the Marketing and Human Resources department user groups, attach an IAM policy that provides access to only the folder that contains the non-sensitive datasets.