S3 fundamentals are;
- object storage
- accessible by http
- highly available & durable
- 0-5TB object size, max 5GB PUT size and we can use multipart put for larger objects
- multiple storage classes and encryption options
- versioning and lifecycle management
- access control
- tight integration
Security and access
- Bucket & Object ACL applied to S3resources
- Bucket Policy applied to S3 resources (AWS recommends)
- IAM Policy for applied to user and roles (AWS recommends)
Customer Requirement: restrict access to a resource except a group
We have a bucket with sensitive data and want to guarantee that only EC2 instances in a specific ASG can reach them. We can have a Deny except rule in bucket policy. Then either we should give access to specific asg with bucket policy or IAM.
A. An explicit DENY for resources arn:aws:s3:::my_bucket & my_bucket/* except aws:userId [the group that should access] B. 1. An Allow for arn:aws:iam:123456789:role/MyRole to resources arn:aws:s3:::my_bucket & my_bucket/* B. 2. Allow rule in IAM
Customer requirement: store sensitive data and guarantee that data is encrypted in transit and at rest
Http is unencrypted, https is encrypted. By using bucket policies we can be sure that http is unused with explicit DENY to aws:SecureTransport: false conditional. Anything is denied in this bucket if the secureTransport is false.
Encryption at rest can be applied with KMS. We can use default AWS managed key (AES256) or customer managed keys (aws:kms) . Two ways to enforce encryption at rest is through bucket policies & default encryption. In Bucket policy
Do an explicit DENY on s3:PutObject for conditional s3:x-amz-server-side-encryption: true is not sent.
Alternatively, any object that is not unencrypted will be encrypted through either AES256, or KMS( asw/s3, ca-key or Custom KMS ARN). The two options are not compatible. If we specify that the user should choose a key in policy, default setting does not work. User should not specify the encryption method, and default encryption would work.
Cost of S3 are due to
- storage
- requests
- data transfer
Traffic from S3 to CloudFront is free. It is cheaper to request from CloudFront that access S3 directly.
S3 Consistency model
Amazon S3 provides read-after-write consistency for PUTs of new objects in your S3 bucket in all Regions, with one caveat. The caveat is that if you make a HEAD or GET request to a key name before the object is created, then create the object shortly after that, a subsequent GET might not return the object due to eventual consistency.
For existing objects, Amazon S3 offers eventual consistency for overwrite PUTS and DELETES in all Regions.
S3 replication
Amazon S3 features asynchronous replication to another bucket – optionally in another account or region. This is good if
- safely sharing data
- backup purposes
- log aggregation
- latency reduction
S3 replication uses an IAM role that needs read access to the source bucket and write access to the target bucket (and KMS keys, when used) Versioning needs to be enabled in source and destination buckets. The entire source bucket or a selection of objects (based on prefix or tags) can be replicated.
Encrypted objects are not replicated by default (you have to check it). During replication the storage class can be changed. Optionally the permissions to the replicated objects can be extended to the target bucket owner.
Default, when an IAM user or role writes an object, the object owner is the IAM user, so objects are owned by the users that put them. The bucket may still be owned by another IAM, but the object is owned by who put them. This can be soled by
- When user B uploads the file, he can set –acl bucket-owner-full-control full permissions for user A. the object is still owned by user B, but userA has full control.
- User B assumes IAM user or role of A, and uploads the file with the assumed role, so the object is owned by user A
This can be enforced by bucket policies. If this user tries to put object without “bucket-owner-full-control”, he will not be allowed.
"Id": "Policy154691" "Version": "2012-10-17" "Statement": - "Sid": "Stmt154275" "Action": - "s3:PutObject" - "s3:PutObjectAcl" "Effect": "Allow" "Resource": "arn:aws:s3::awsexamplebucket/*" "Condition": "StringEquals": "s3:x-amz-acl": "bucket-owner-full-control" "Principal": "AWS": - "arn:aws:iam::111333:user/ExampleUser"
aws s3 cp dummyFileToCopy.dmy s3://other-users-bucket/ –acl bucket-owner-full-control
To look further
- website hosting
- cross origin resource sharing
- s3 access points
- s3 badge operations
- s3 wans
Well architected framework / AWS s3 faq / AWS s3 documentation