AmazonS3 client org.xml.sax.SAXParseException when trying to listObjects

Question

I'm having some drama when running the listObjects(..) method of AmazonS3. I'm certain that my credentials are set up correctly as I am able to download individual files using s3Client.getObject(..). The logs read::

com.amazonaws.SdkClientException: Failed to parse XML document with handler class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$ListObjectsV2Handler Caused by: org.xml.sax.SAXParseException: Premature end of file.

I understand that listObjects(..) does include in it's response some xml containing meta data. The code to reproduce the error is very simple. I can't see anything wrong here :(

ListObjectsRequest listObjectsRequest = new ListObjectsRequest() .withBucketName(ENV.getProperty("cloud.aws.s3.bucket"));

ObjectListing objectListing = amazonS3Client.listObjects(listObjectsRequest);

Here is the version of spring-cloud-aws-context I am using:: <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-aws-context</artifactId <version>1.2.1.RELEASE</version> </dependency>

Does anybody have any insight? Or know away around this issue?

Thanks in advance :)

Answer 1

I encountered the exact exception *Failed to parse XML document with handler class * and the failure is not truly descriptive. But my problem was not permissions but rather trying to list the bucket subfolder directly.

I was trying to listObjects from /bucketName/subFolder/subFolder2 instead of just /bucketName and prefix.

This results in exception above (In Scala):

val path = "/bucketName/myFolder/subFolder"
val results = s3Client.listObjectsV2(path)

I needed to separate the bucket name and the prefix and then use ListObjectRequestV2

val path = "/bucketName/myFolder/subFolder"
val bucketName = "bucketName"
val prefix = "myFolder/subFolder"
val listObjectsRequest = new 
val ListObjectsV2Request().withBucketName(bucketName).withPrefix(prefix) 
val results = s3Client.listObjectsV2(path)

Answer 2

Ok, I found the solution! The problem was with the permissions, apparently listObjects(..) requires it's own set of permissions. Specifically the 'ListBucket' action must be enabled.

{
    "Version": "2012-10-17",
    "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:ListBucket"],
      "Resource": ["arn:aws:s3:::test"]
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject"
      ],
      "Resource": ["arn:aws:s3:::test/*"]
    }
  ]
}

Another issue was with how the name that I gave to my bucket. I used mybucket/some/prefix/before/files. I corrected this to mybucket. The prefix is only used with getObject(..) like this:

GetObjectRequest getObjectRequest = new 
GetObjectRequest("mybucket/some/prefix/before/files", key);
S3Object s3Object = amazonS3Client.getObject(getObjectRequest);