How to filter an s3 data event by object key suffix on AWS EventBridge

Question

I've created a rule on AWS EventBridge that trigger a Sagemaker Pipeline execution. To do so, I have the following event pattern:

{
  "source": ["aws.s3"],
  "detail-type": ["AWS API Call via CloudTrail"],
  "detail": {
    "eventSource": ["s3.amazonaws.com"],
    "eventName": ["PutObject", "CopyObject", "CompleteMultipartUpload"],
    "requestParameters": {
      "bucketName": ["my-bucket-name"],
      "key": [{
        "prefix": "folder/inside/my/bucket/"
      }]
    }
  }
}

I have enabled CloudTrail to log my S3 Data Events and the rule are triggering my Sagemaker Pipeline execution correctly.

The problem here is: A pipeline execution are being triggered for all put/copy of any object in my prefix. Then, I would like to trigger my pipeline execution only when a specific object is uploaded in the bucket, by I don't know its entire name.

For instance, possible object name I will have is, where this date is builded dynamically:

my-bucket-name/folder/inside/my/bucket/2021-07-28/_SUCESS

I would like to write an event pattern with something like this:

"prefix": "folder/inside/my/bucket/{current_date}/_SUCCESS"

or

"key": [{
  "prefix": "folder/inside/my/bucket/"
}, {
  "suffix": "_SUCCESS"
}]

I think that Event Pattern on AWS do not support suffix filtering. In the documentation, isn't clear the behavior. I have configured a S3 Event Notification using a suffix and sent the filtered notification to a SQS Queue, but now I don't know what to do with this queue in order to invoke my EventBridge rule to trigger a Sagemaker Pipeline execution.

Answer 1

I was looking at a similar functionality.

Unfortunately, based on the docs from AWS, it looks like it only supports the following patterns:

Comparison	Example	Rule syntax
Null	UserID is null	"UserID": [ null ]
Empty	LastName is empty	"LastName": [""]
Equals	Name is "Alice"	"Name": [ "Alice" ]
And	Location is "New York" and Day is "Monday"	"Location": [ "New York" ], "Day": ["Monday"]
Or	PaymentType is "Credit" or "Debit"	"PaymentType": [ "Credit", "Debit"]
Not	Weather is anything but "Raining"	"Weather": [ { "anything-but": [ "Raining" ] } ]
Numeric (equals)	Price is 100	"Price": [ { "numeric": [ "=", 100 ] } ]
Numeric (range)	Price is more than 10, and less than or equal to 20	"Price": [ { "numeric": [ ">", 10, "<=", 20 ] } ]
Exists	ProductName exists	"ProductName": [ { "exists": true } ]
Does not exist	ProductName does not exist	"ProductName": [ { "exists": false } ]
Begins with	Region is in the US	"Region": [ {"prefix": "us-" } ]

Answer 2

Looks like EventBridge has added this capability. Suffix matching should do the trick..

{
  "detail-type": ["Object Created"],
  "source": ["aws.s3"],
  "detail": {
    "bucket": {
      "name": ["MY BUCKET"]
    },
    "object": {
      "key": [{
        "suffix": ".pdf"
      }]
    }
  }
}

For reference, here is the structure of events from S3.