Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to delete S3 folder based on filter criteria using policy?

Tags:

amazon-s3

How to delete the objects in S3 bucket based on specified criteria using bucket policy?

  1. Total object counts in the bucket should be more than 5 everytime.
  2. Object should have a common prefix.
  3. Object age should be more than X(1,2,3,4...n) days.

Bucket Structure -

myprefixtest1234bucket //bucket 
        |- prefixtestobject1 //object
        |- prefixtestobject2
        |- prefixtestobject3
        |- prefixtestobject4
        |- prefixtestobject5
        |- prefixtestobject6
        |- prefixtestobject7
        |- testobject8
        |- testobject9

I am trying to delete the object in a bucket which has prefix "prefixtext" and is older than X days and this bucket should keep Y numbers of object all the time even if it is older than X days. Means keeping Y number of object should be given precedence over age (older than X days).

I tried below policy but didn't work and not sure how to add the age and object count logic.

{
"Id": "Policy123456",
  "Version":"2012-10-17",
  "Statement":[
    {
      "Sid":"TestBucketObjectDeletion",
      "Effect":"Allow",
      "Principal": {"AWS": ["arn:aws:iam::123456789:root"]},
      "Action":["s3:DeleteObject"],
      "Resource":["arn:aws:s3:::myprefixtest1234/*"],
      "Condition":{"StringEquals":{"myprefixtest1234"}}
    }
  ]
}
like image 656
VIPIN KUMAR Avatar asked Dec 21 '25 08:12

VIPIN KUMAR


1 Answers

Currently, there is no way to specify to keep number(1,2,3....n) of objects in the bucket and delete rest of them based on common prefix but we can set the expiration of the bucket objects based on age using bucket Life Cycle.

{
    "Rules": [
        {
            "Filter": {
                "Prefix": "documents/"
            },
            "Status": "Enabled",
            "Transitions": [
                {
                    "Days": 365,
                    "StorageClass": "GLACIER"
                }
            ],
            "Expiration": {
                "Days": 3650
            },
            "ID": "ExampleRule"
        }
    ]
}

And put it using s3api like this -

aws s3api put-bucket-lifecycle-configuration  \
--bucket bucketname  \
--lifecycle-configuration file://lifecycle.json

More info available here - Here and here.

like image 86
VIPIN KUMAR Avatar answered Dec 23 '25 20:12

VIPIN KUMAR



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!