Integrate AWS Transfer for SFTP With A Custom Identity Provider

Feb 6, 2019

|

Volodymyr Rudyi

AWS Transfers for SFTP is a fully managed service that allows to easily upload/download data to/from AWS S3 using the SFTP protocol. In the previous blog post, we created a managed SFTP endpoint using the public key authentication. Sometimes, a username/password authentication may be required, e.g. if the access should be provided to existing users, e.g. from the AWS Cognito User Pool. In this blog post we are giving step-by-step instructions on how to implement a custom authentication for AWS Transfers for SFTP.

Check out our new blog post about AWS SFTP custom identity provider for Active Directory. It also contains a very handy CloudFormation template that can be fully customized for your needs!

Deployment

We'll be using the Serverless framework to create corresponding infrastructure. We assume the AWS Cognito Userpool already exists to simulate a real-world scenario. The resulting CloudFormation stack contains:

  • API Gateway
  • AWS Lambda function which validates username/password supplied to the SFTP endpoint
  • A custom resource for the AWS Transfers for SFTP since at the moment the blog post was written, it was not present in the CloudFormation.

Here is a diagram with the resulting infrastructure:

Integration Infrastructure

Custom resource definition is provided below:

   SftpServer:
     Type: "Custom::SFTPServer"
     Version: "1.0"
     DependsOn:
       - CustomSftpServerManagerLambdaFunction
       - TransferIdentityProviderRole
       - TransferLoggingRole
       - ApiGatewayRestApi
       - SftpBucket
     Properties:
       ServiceToken:
         Fn::GetAtt: [ "CustomSftpServerManagerLambdaFunction", "Arn" ]
       InvocationRole:
         Fn::GetAtt: [ "TransferIdentityProviderRole", "Arn" ]
       LoggingRole:
          Fn::GetAtt: [ "TransferLoggingRole", "Arn" ]
       AuthorizeUrl:
         Fn::Join:
           - ""
           -
             - https://
             - Ref: ApiGatewayRestApi
             - .execute-api.
             - Ref: 'AWS::Region'
             - .amazonaws.com/dev/

IAM Roles and Permissions

The Lambda execution role is rather standard:

   LambdaExecutionRole:
     Type: AWS::IAM::Role
     Properties:
       Path: /sftp/
       AssumeRolePolicyDocument:
         Version: '2012-10-17'
         Statement:
           - Effect: Allow
             Principal:
               Service:
                 - lambda.amazonaws.com
             Action: "sts:AssumeRole"
       ManagedPolicyArns:
         - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

For our SFTP endpoint we need to create two roles: TransferIdentityProviderRole(for API Gateway invocation) and TransferLoggingRole(for logging):

   TransferIdentityProviderRole:
     Type: AWS::IAM::Role
     Properties:
       Path: /sftp/
       AssumeRolePolicyDocument:
         Version: '2012-10-17'
         Statement:
           - Effect: Allow
             Principal:
               Service:
                 - transfer.amazonaws.com
             Action: sts:AssumeRole
       Policies:
         - PolicyName: TransferCanInvokeAuthorizeApi
           PolicyDocument:
             Version: '2012-10-17'
             Statement:
               - Effect: Allow
                 Action:
                   - execute-api:Invoke
                 Resource:
                   Fn::Join:
                     - ":"
                     -
                       - "arn"
                       - Ref: 'AWS::Partition'
                       - execute-api
                       - Ref: 'AWS::Region'
                       - Ref: 'AWS::AccountId'
                       - Fn::Join:
                           - ""
                           -
                             - Ref: ApiGatewayRestApi
                             - /${self:provider.stage}/GET/*
         - PolicyName: TransferCanReadAuthorizeApi
           PolicyDocument:
             Version: '2012-10-17'
             Statement:
               - Effect: Allow
                 Action:
                   - apigateway:GET
                 Resource: "*"
   TransferLoggingRole:
     Type: AWS::IAM::Role
     Properties:
       Path: /sftp/
       AssumeRolePolicyDocument:
         Version: '2012-10-17'
         Statement:
           - Effect: Allow
             Principal:
               Service:
                 - transfer.amazonaws.com
             Action: sts:AssumeRole
       ManagedPolicyArns:
         - arn:aws:iam::aws:policy/service-role/AWSTransferLoggingAccess

Another required role is the common SFTP user role. ARN of this role in the response from Lambda will indicate a successful authorization:

   SftpUserRole:
     Type: AWS::IAM::Role
     Properties:
       Path: /sftp/
       AssumeRolePolicyDocument:
         Version: '2012-10-17'
         Statement:
           - Effect: Allow
             Principal:
               Service:
                 - transfer.amazonaws.com
             Action: "sts:AssumeRole"
       Policies:
         - PolicyName: SftpBucketAccessPolicy
           PolicyDocument:
             Version: '2012-10-17'
             Statement:
               - Effect: Allow
                 Action:
                   - s3:*
                 Resource:
                   Fn::Join:
                     - ":"
                     -
                       - "arn"
                       - Ref: 'AWS::Partition'
                       - "s3::"
                       - ${env:TRANSFER_BUCKET_NAME}*

Additionally, if you would like to use custom "SFTPServer" resource, you should define a role that allows managing the AWS Transfer for SFTP and also write logs to CloudWatch:

   LambdaSftpManageRole:
     Type: AWS::IAM::Role
     Properties:
       Path: /sftp/
       AssumeRolePolicyDocument:
         Version: '2012-10-17'
         Statement:
           - Effect: Allow
             Principal:
               Service:
                 - lambda.amazonaws.com
             Action: "sts:AssumeRole"
       Policies:
         - PolicyName: LambdaSftpManageRole
           PolicyDocument:
             Version: '2012-10-17'
             Statement:
               - Effect: Allow
                 Action:
                   - transfer:CreateServer
                   - transfer:UpdateServer
                   - transfer:DeleteServer
                   - logs:CreateLogGroup
                   - logs:CreateLogStream
                   - logs:PutLogEvents
                   - iam:PassRole
                   - apigateway:GET
                 Resource: "*"

IMPORTANT: iam:PassRole and apigateway:GET permissions are required, otherwise you will get an error if you try to create an SFTP Server.

Lambdas declaration

First, lets take a look at "SftpAuthorizer" lambda configuration:

 SftpAuthorizer:
   handler: src/handler.authorize
   role: LambdaExecutionRole
   environment:
     COGNITO_USER_POOL_ID: ${env:COGNITO_USER_POOL_ID}
     COGNITO_CLIENT_ID: ${env:COGNITO_CLIENT_ID}
     SERVER_ID: { "Fn::GetAtt": ["SftpServer", "ServerId" ] }
     BUCKET_ARN: { "Fn::GetAtt": ["SftpBucket", "Arn" ] }
     ROLE_ARN: { "Fn::GetAtt": ["SftpUserRole", "Arn" ] }
   events:
     - http:
         path: /servers/{serverId}/users/{user}/config
         method: GET
         authorizer: aws_iam

This lambda is being triggered by a request to AWS API Gateway and is used for authorization event handling. You should provide the following environment variables:

  • COGNITOUSER_POOL_ID and COGNITO_CLIENT_ID – AWS Cognito IDs
  • ROLE_ARN – an ARN of a common role for your SFTP users
  • SERVER_ID – a server id of our SFTP server (we will get it from the custom resource)
  • BUCKET_ARN – an ARN of an S3 bucket which you would like to serve to your users

IMPORTANT NOTE: Please note that you must specify /servers/{serverId}/users/{user}/config URL because it’s a path which AWS Transfers for SFTP uses by convention.

Also, you need to define the "SFTPServer" resource management lambda:

 CustomSftpServerManager:
   handler: src/handler.manageSftpServer
   role: LambdaSftpManageRole
   timeout: 180

Lambdas implementation

Now when we are done with the configuration, the last step is creating handlers for configured lambdas. First, let’s create the “authorize” function. This function will authorize users in the AWS Cognito using amazon-cognito-identity-js library. If the user was successfully authorized it should return the user's role ARN. Optionally, you can also provide Scope-Down Policy which allows restricting access for users based on HomeBucket and HomeDirectory variables, e.g. to define directories that will be available to the SFTP user. For this blog post, we created a policy which allows access only for bucket prefix that equals a username. If te user is unauthorized, the function should return a response with an empty body. An example of an authorized response code:

 const response = {
   headers: {
     "Access-Control-Allow-Origin": "*",
     "Content-Type": "application/json"
   },
   body: JSON.stringify({
     Role: process.env.ROLE_ARN,
     Policy: `{
       "Version": "2012-10-17",
       "Statement": [
           {
               "Sid": "AllowListingOfUserFolder",
               "Action": [
                   "s3:ListBucket"
               ],
               "Effect": "Allow",
               "Resource": [` +
                   '"arn:aws:s3:::${transfer:HomeBucket}"' +
               `],
               "Condition": {
                   "StringLike": {
                       "s3:prefix": [` +
                           '"${transfer:UserName}/*",' +
                           '"${transfer:UserName}"' +
                       `]
                   }
               }
           },
           {
               "Sid": "AWSTransferRequirements",
               "Effect": "Allow",
               "Action": [
                   "s3:ListAllMyBuckets",
                   "s3:GetBucketLocation"
               ],
               "Resource": "*"
           },
           {
               "Sid": "HomeDirObjectAccess",
               "Effect": "Allow",
               "Action": [
                   "s3:PutObject",
                   "s3:GetObject",
                   "s3:DeleteObjectVersion",
                   "s3:DeleteObject",
                   "s3:GetObjectVersion"
               ],` +
               '"Resource": "arn:aws:s3:::${transfer:HomeDirectory}*"' +
            `}
       ]
     }`,    
     HomeDirectory: `/${process.env.BUCKET_ARN.substring("arn:aws:s3:::".length)}/${username}/`,
     HomeBucket: process.env.BUCKET_ARN.substring("arn:aws:s3:::".length)
   }),
   statusCode: 200
 };

IMPORTANT NOTE: Policy parameter must be a string, it doesn’t work with an object type.

The second "manageSftpServer" handler is used for custom SFTPServer resource management. It handles create, update and delete events. There is also a utility "sendResponse" function which sends a response from the custom resource to the CloudFormation.

Summary

As you may noticed, AWS Transfer for SFTP with a custom identity provider configuration is not just "plug-and-play", but it's very flexible and can work with any identity provider.

Additionally, you can specify more conditions for the authorization, for example, grant access only for particular groups or for users, that have certain attribute values.

If you don't want to do all of this configuration by yourself, you can download our project from the GitHub using the link below, provide some environment variables and simply deploy your SFTP server with the "sls deploy" command.

Useful Links

Related tags:

/

Don't miss these posts