AWS Transfer: SFTP rename Fails with Permission Denied

Just hit this little fun thing. Using the rename command for SFTP seems to use the s3:CopyObject action, which attempts to copy any object tags as well as the actual object itself. The IAM Permission associated with the SFTP user must allow s3:GetObjectTagging and s3:PutObjectTagging or a rename command will get an Access Denied message… Continue reading AWS Transfer: SFTP rename Fails with Permission Denied

Using AWS Transfer with a Custom Lambda Identity Provider

This week I got a chance to work on implementing AWS Transfer as an SFTP server backed by a set of S3 buckets. Authentication in this new system is handled by another, self-serve SFTP application. Users can create an manage SFTP users there and AWS Transfer uses their usernames, passswords, and public keys to authenticate… Continue reading Using AWS Transfer with a Custom Lambda Identity Provider

Posted in AWS

How Access an AWS Container Repository from Another Account

Like many things AWS all this information can be found in the AWS docs themselves, but scattered everywhere. This article on service policies (or resource policies) vs IAM permissions provides some background for what we'll do here. There are two pieces here: 1. The Elastic Container Repository (ECR) in one AWS account (account ID 1111111111… Continue reading How Access an AWS Container Repository from Another Account

Use Many S3 Buckets

use many s3 buckets

Should applications use one S3 bucket, putting different categories of files under different directories?1 Or should an application use multiple S3 buckets -- one for each different category of files? As the title implies, the answer is many S3 buckets. While some things, like lifecycle rules, can be configured based on directories (key prefixes, really),… Continue reading Use Many S3 Buckets

Posted in S3

Building an Upload System Backed by S3 and Client-Side Uploads

Client-Side S3 Uploads

One of the things I try to do when building applications is keep servers stateless. This makes those servers easy to throw away -- a piece of infrastructure failing (which it always will) is not a big deal with stateless servers. Just spin up a new one. When the requirement came down the pipeline to… Continue reading Building an Upload System Backed by S3 and Client-Side Uploads

One Way to Manage Secrets in Dockerized Applications

Dockerized Application Secrets

Putting configuration in the environment is a fairly well acknowledged best practice now. That configuration often includes secrets. But environment variables in container images -- like the docker ENV stanza -- are not really secure. They are built as part of the image, after all, so anyone with access to the image can get at… Continue reading One Way to Manage Secrets in Dockerized Applications

AWS Service Policies vs IAM Permissions

AWS is, at best, complicated. Part of that complexity is that there are often seems to be more than one way to accomplish a goal. Service policies and IAM permissions are a great example of this. Both seem to accomplish the same thing (permissions), but serve different purposes. Service policies is probably not an official… Continue reading AWS Service Policies vs IAM Permissions

A Recommended VPC Subnet Configuration

This is a recommendation on how to organize subnets inside an AWS VPC. Before continuing, it's important to understand the difference between a public and private subnet. There are three broad classes of subnets to run inside your VPC: Public Private Internal Internal subnets aren't really a thing, but it's a convenient way to talk… Continue reading A Recommended VPC Subnet Configuration

What’s the Difference Between a Public and Private Subnet in an AWS VPC?

AWS Public vs Private Subnet

This is a brief intro to quite a few of the AWS VPC components and how they fit together with the goal of describing the difference between a public and private subnet. Amazon has this line in some VPC docs that hints at the difference: The instances in the public subnet can send outbound traffic… Continue reading What’s the Difference Between a Public and Private Subnet in an AWS VPC?