s3

Usage:

include s3

Hiera

Install packages

The package name must be exact (phalcon-3.0.3-1.x86_64), without the extension, otherwise puppet will try to install it at each run

s3::packages:
  phalcon-3.0.3-1.x86_64:
    source: 'bucket_s3/phalcon-letgo-3.0.3-1.x86_64.rpm'
    provider: 'rpm'
  librdkafka-0.9.2-1.el6.remi.x86_64:
    source: 'bucket_s3/librdkafka-0.9.2-1.el6.remi.x86_64.rpm'
    provider: 'rpm'

Copy single file

Make sure you have the appropriate permissions on the role or user of AWS, otherwise the execution of puppet will fail

s3::manage:
  file1:
    command: 'cpbl'
    destination: '/opt/app/libs/'
    bucket: 'bucket_s3/app/'
    source: 'file.jar'

  file2:
    command: 'cpbl'
    destination: '/opt/app/docs/'
    bucket: 'bucket_s3/app/docs/'
    source: 'doc.jar'

Copy a directory files (multiple files), in this case it is not necessary specify the source. We will only configure the parameter refreshonly to false when we want the folder to be synchronized in each puppet execution, otherwise we will not be able to do anything (by default this is true), in this case the entire contents of the folder will be copied once unless there are changes in the content

s3::manage:
  app:
    command: 'sybl'
    destination: '/opt/app'
    bucket: 'bucket_s3/app/'
    refreshonly: false

Parameters

s3

s3::packages:

ensure: By default = present

provider: By default = rpm

s3_domain: By default = s3.amazonaws.com

Supports all parameters of the puppet package resource

More information in https://docs.puppet.com/puppet/latest/types/package.html

s3::manage:

destination: By default = ''

source: By default = ''

command: By default = 'cpbl'

flag: By default = ''

refreshonly: By default = true

More information in http://docs.aws.amazon.com/cli/latest/reference/s3/

Command:

cpbl: copy from bucket to local

cplb: copy from local to bucket

cpbb: copy from bucket to bucket

mvbl: move from bucket to local

mvlb: move from local to bucket

mvbb: move from bucket to bucket

sybl: sync from bucket to local

sylb: sync from local to bucket

sylb: sync bucket to bucket

rm: delete files from bucket

Flags:

--dryrun (boolean) Displays the operations that would be performed using the specified command without actually running them.

--quiet (boolean) Does not display the operations performed from the specified command.

--include (string) Don't exclude files or objects in the command that match the specified pattern. See Use of Exclude and Include Filters for details.

--exclude (string) Exclude all files or objects from the command that matches the specified pattern.

--acl (string) Sets the ACL for the object when the command is performed. If you use this parameter you must have the "s3:PutObjectAcl" permission included in the list of actions for your IAM policy. Only accepts values of private, public-read, public-read-write, authenticated-read, aws-exec-read, bucket-owner-read, bucket-owner-full-control and log-delivery-write. See Canned ACL for details

--follow-symlinks | --no-follow-symlinks (boolean) Symbolic links are followed only when uploading to S3 from the local filesystem. Note that S3 does not support symbolic links, so the contents of the link target are uploaded under the name of the link. When neither --follow-symlinks nor --no-follow-symlinks is specifed, the default is to follow symlinks.

--no-guess-mime-type (boolean) Do not try to guess the mime type for uploaded files. By default the mime type of a file is guessed when it is uploaded.

--sse (string) Specifies server-side encryption of the object in S3. Valid values are AES256 and aws:kms. If the parameter is specified but no value is provided, AES256 is used.

--sse-c (string) Specifies server-side encryption using customer provided keys of the the object in S3. AES256 is the only valid value. If the parameter is specified but no value is provided, AES256 is used. If you provide this value, --sse-c-key must be specfied as well.

--sse-c-key (string) The customer-provided encryption key to use to server-side encrypt the object in S3. If you provide this value, --sse-c must be specfied as well. The key provided should not be base64 encoded.

--sse-kms-key-id (string) The AWS KMS key ID that should be used to server-side encrypt the object in S3. Note that you should only provide this parameter if KMS key ID is different the default S3 master KMS key.

--sse-c-copy-source (string) This parameter should only be specified when copying an S3 object that was encrypted server-side with a customer-provided key. It specifies the algorithm to use when decrypting the source object. AES256 is the only valid value. If the parameter is specified but no value is provided, AES256 is used. If you provide this value, --sse-c-copy-source-key must be specfied as well.

--sse-c-copy-source-key (string) This parameter should only be specified when copying an S3 object that was encrypted server-side with a customer-provided key. Specifies the customer-provided encryption key for Amazon S3 to use to decrypt the source object. The encryption key provided must be one that was used when the source object was created. If you provide this value, --sse-c-copy-source be specfied as well. The key provided should not be base64 encoded.

--storage-class (string) The type of storage to use for the object. Valid choices are: STANDARD | REDUCED_REDUNDANCY | STANDARD_IA. Defaults to 'STANDARD'

--website-redirect (string) If the bucket is configured as a website, redirects requests for this object to another object in the same bucket or to an external URL. Amazon S3 stores the value of this header in the object metadata.

--content-type (string) Specify an explicit content type for this operation. This value overrides any guessed mime types.

--cache-control (string) Specifies caching behavior along the request/reply chain.

--content-disposition (string) Specifies presentational information for the object.

--content-encoding (string) Specifies what content encodings have been applied to the object and thus what decoding mechanisms must be applied to obtain the media-type referenced by the Content-Type header field.

--content-language (string) The language the content is in.

--expires (string) The date and time at which the object is no longer cacheable.

--source-region (string) When transferring objects from an s3 bucket to an s3 bucket, this specifies the region of the source bucket. Note the region specified by --region or through configuration of the CLI refers to the region of the destination bucket. If --source-region is not specified the region of the source will be the same as the region of the destination bucket.

--only-show-errors (boolean) Only errors and warnings are displayed. All other output is suppressed.

--page-size (integer) The number of results to return in each response to a list operation. The default value is 1000 (the maximum allowed). Using a lower value may help if an operation times out.

--ignore-glacier-warnings (boolean) Turns off glacier warnings. Warnings about an operation that cannot be performed because it involves copying, downloading, or moving a glacier object will no longer be printed to standard error and will no longer cause the return code of the command to be 2.

--force-glacier-transfer (boolean) Forces a transfer request on all Glacier objects in a sync or recursive copy.

--metadata (map) A map of metadata to store with the objects in S3. This will be applied to every object which is part of this request. In a sync, this means that files which haven't changed won't receive the new metadata. When copying between two s3 locations, the metadata-directive argument will default to 'REPLACE' unless otherwise specified.

--metadata-directive (string) Specifies whether the metadata is copied from the source object or replaced with metadata provided when copying S3 objects. Note that if the object is copied over in parts, the source object's metadata will not be copied over, no matter the value for --metadata-directive, and instead the desired metadata values must be specified as parameters on the command line. Valid values are COPY and REPLACE. If this parameter is not specified, COPY will be used by default. If REPLACE is used, the copied object will only have the metadata values that were specified by the CLI command. Note that if you are using any of the following parameters: --content-type, content-language, --content-encoding, --content-disposition, --cache-control, or --expires, you will need to specify --metadata-directive REPLACE for non-multipart copies if you want the copied objects to have the specified metadata values.

--expected-size (string) This argument specifies the expected size of a stream in terms of bytes. Note that this argument is needed only when a stream is being uploaded to s3 and the size is larger than 5GB. Failure to include this argument under these conditions may result in a failed upload due to too many parts in upload.

--recursive (boolean) Command is performed on all files or objects under the specified directory or prefix.