boto3 paginated object deletion against ONTAP S3 leaves objects in bucket
Applies to
- ONTAP 9
- boto3 library based S3 client
- more than 1000 objects in bucket are going to be deleted at once
Issue
- Running the python script below, leaves objects in bucket when object count is greater than 1000 objects
- When commenting out the
obj.delete()line however, all objects are listed
#!/usr/bin/python
import boto3
import logging
import sys
import botocore
logging.getLogger('boto3').setLevel(logging.DEBUG)
logging.getLogger('botocore').setLevel(logging.DEBUG)
logging.basicConfig(level=logging.DEBUG, stream=sys.stdout)
s3 = boto3.resource('s3', endpoint_url='https://svm-s3.local:443', aws_access_key_id='user',aws_secret_access_key='secret')
bucket = s3.Bucket('user-bucket-1')
for obj in bucket.objects.all():
print("Will delete:", obj)
print("response:", obj.delete())