Skip to main content

Create an AWS Service Account with S3 Permissions

1. Create an IAM User (Service Account)​

  1. Sign in to the AWS Management Console.
  2. Go to IAM service (search "IAM" in the search bar).
  3. In the left navigation pane, click Users, then click Add users.
  4. Set user details:
    • User name: e.g., s3-service-account
  5. Click Next.
  6. Set permissions:
    • Permissions options: select Attach policies directly
    • Permissions policies:
      • AmazonS3FullAccess: need to create, list bucket, uploading files, ...
      • AmazonAthenaFullAccess (Optional): need for Athena operations.
      • AWSGlueServiceRole (Optional): need for Glue crawler operations to create table in Athena.
  7. Click Next and Create user.

If Athena will be used as query engine, please make sure include all permissions policies.

2. Use Athena as Query Engine (Optional)​

If option Use Athena as Query Engine is enabled, Elton will use AWS Glue Crawler to setup Athena table.

Create Glue Crawler Role​

  1. Go to IAM service (search "IAM" in the search bar).
  2. In the left navigation pane, click Roles, then click Create role.
  3. Selected trusted entity:
    • Trusted entity type: AWS service.
    • Use case: Glue.
  4. Add permissions:
    • AWSGlueServiceRole: need for Glue crawler operations to create table in Athena.
  5. Click Next, input for Role name e.g AWSGlueServiceRole-S3Crawler then click Create role. This name will be used later in Destination configuration in Elton.
  6. After creation, click to role detail, under Permissions tab, click Add permissions then Create inline policy.
    • Policy editor: click JSON and input the following value 
      {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "s3:GetObject",
                    "s3:PutObject"
                ],
                "Resource": [
                    "*"
                ]
            }
        ]
      }
  • Click Next, input policy name e.g AWSGlueServiceRole-S3CrawlerInlinePolicy then click Create policy.
  1. Go back to IAM > Users, click to user (created above) detail, under Permissions policies tab, click Add permissions then Create inline policy.

    • Policy editor: click JSON and input the following value

      {
        "Version": "2012-10-17",
        "Statement": [
          {
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": "your-role"
          }
        ]
      }

      Replace your-role with the ARN name from the role created above e.g. arn:aws:iam::726580148243:role/AWSGlueServiceRole-S3Crawler

  • Click Next, input policy name e.g AWSGlueServiceRole-S3Crawler then click Create policy.

Configuration in Elton Data

Elton's default setting is to write data solely to S3. If you wish to leverage Athena and AWS Glue Crawler for table configuration, you must enable the Use Athena as Query Engine option. Please note that using Glue Crawler will incur additional costs.

Elton tries to reduce costs for AWS Glue Crawlers by applying the following practices:

  • All crawlers created by Elton are scheduled to run on-demand and will only be triggered by Elton when needed.
  • Elton only triggers a crawler in the following scenarios:
    • The first-time run of a connection.
    • When a user clicks the Reload button.
    • When data types change in the data source.
Last updated on