S3 Import
Import data from S3-compatible storage services.
Properties
| Name | Type | |
|---|---|---|
| engine | String=s3:read | |
| credentials | Object | |
| bucket | String | |
| region | String | |
| path | String | |
| endpoint | String | |
| use_ssl | Boolean | |
| decoder | String | |
| schema | Object | |
engine
Must be s3:read.
- Required
- Yes
credentials
The AWS credentials.
- Required
- Yes
Properties
| Name | Type | |
|---|---|---|
| access_key_id | String | |
| secret_access_key | String | |
credentials.access_key_id
Access key ID to authenticate with.
- Required
- Yes
credentials.secret_access_key
Secret access key to authenticate with.
- Required
- Yes
bucket
Bucket name to store file in. Creates the bucket if it doesn’t exist.
- Required
- Yes
region
The region of the S3 bucket.
- Required
- Yes
- Default
- us-east-1
path
The path from which to get the file on S3.
- Required
- Yes
endpoint
Defaults to s3.amazonaws.com.
- Required
- No
- Default
- s3.amazonaws.com
use_ssl
Defaults to true.
- Required
- No
- Default
- true
decoder
The format of the data. See Parsing Data for details.
- Required
- Yes
- Enum
-
csvjsonndjsonmsgpackprotobufparquethtmlxmlregex
schema
Define a schema for the data.
- Required
- Yes