YAS3 - Yet Another S3 Client
Yas3 is a simple client for managing buckets/storage in S3 (and S3 compatible endpoints) for people who don't want the complexity of learning boto3.
Uploading files
You can upload from file paths:
with yas3.Connection(access_key=ACCESS_KEY, secret_key=SECRET_KEY, endpoint=ENDPOINT) as conn:
conn.upload('test.txt', 'file.txt', bucket='bucket')Or as bytes:
with yas3.Connection(access_key=ACCESS_KEY, secret_key=SECRET_KEY, endpoint=ENDPOINT) as conn:
conn.upload(file_bytes, 'file.txt', bucket='bucket', type='application/octet-stream')The file type is guessed using mimetypes, but it can be easily specified using mimetype='text/json'.
Downloading files
You can download the file locally:
with yas3.Connection(access_key=ACCESS_KEY, secret_key=SECRET_KEY, endpoint=ENDPOINT) as conn:
conn.download('file.txt', 'test.txt', bucket='bucket')Or you can get the response directly:
with yas3.Connection(access_key=ACCESS_KEY, secret_key=SECRET_KEY, endpoint=ENDPOINT) as conn:
bucket_data = conn.get('file.txt', bucket='bucket')
print(bucket_data)File Operations
Moving Files:
with yas3.Connection(access_key=ACCESS_KEY, secret_key=SECRET_KEY, endpoint=ENDPOINT) as conn:
conn.move('file.txt', 'new_file.txt', source_bucket='bucket', target_bucket='bucket')Copying Files:
with yas3.Connection(access_key=ACCESS_KEY, secret_key=SECRET_KEY, endpoint=ENDPOINT) as conn:
conn.copy('file.txt', 'new_file.txt', source_bucket='bucket', target_bucket='bucket')Deleting Files:
with yas3.Connection(access_key=ACCESS_KEY, secret_key=SECRET_KEY, endpoint=ENDPOINT) as conn:
conn.delete('file.txt', bucket='bucket')Listing files in a bucket
with yas3.Connection(access_key=ACCESS_KEY, secret_key=SECRET_KEY, endpoint=ENDPOINT) as conn:
conn.list(bucket='bucket', prefix=None)Update File Metadata:
with yas3.Connection(access_key=ACCESS_KEY, secret_key=SECRET_KEY, endpoint=ENDPOINT) as conn:
conn.update_metadata('file.txt', metadata={'x-amz-storage-class': 'REDUCED_REDUNDANCY'}, bucket='bucket')Update File Permissions:
with yas3.Connection(access_key=ACCESS_KEY, secret_key=SECRET_KEY, endpoint=ENDPOINT) as conn:
conn.update_metadata('file.txt', metadata={'x-amz-storage-class': 'REDUCED_REDUNDANCY'}, private=False, bucket='bucket')Using it as an object
If you want to create an object, feel free:
conn = yas3.Connection(...)
...
conn.close()Multithreading
If you want to use multithreading, we support that too:
conn = yas3.Connection(..., threads=10)
futures = []
for obj, obj_path in convenient_list_of_objects:
futures.append(conn.async_upload(obj, obj_path, bucket='bucket', type='application/octet-stream'))
# Wait for a file to be done
futures[-1].wait()
# Or upload synchronously (will wait for the data to finish uploading before continuing)
conn.upload(obj, obj_path, bucket='bucket', type='application/octet-stream')
# Wait for everything to finish
conn.wait_all()Note: If you kill the program without waiting for the futures, there's no guarantee that every operation will be finished. That's on you. Be careful.
Full Docs
See the full API documentation here.
