Amazon S3 Client setReadLimit

Question

While uploading file to S3 , we are getting this random error msg for one single case

"If the request involves an input stream, the maximum stream buffer size can be configured via request.getRequestClientOptions().setReadLimit(int)"

source being : https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/AmazonS3Client.java

As per AWS SDK for Java 1.8.10 We can set maximum stream buffer size to be configured per request via request.getRequestClientOptions().setReadLimit(int)

We are using com.amazonaws.services.s3.AmazonS3 object to upload data.

Can anyone suggest how we can set ReadLimit() via com.amazonaws.services.s3.AmazonS3

https://aws.amazon.com/releasenotes/0167195602185387

Answer 1

It sounds like you're uploading data from an InputStream, but some sort of transient error is interrupting the upload. The SDK isn't able to retry the request because InputStreams are mark/resetable by default. The error message is trying to give guidance on buffer sizes, but for large data, you probably don't want to load it all into memory anyway.

If you're able to upload from a File source, then you shouldn't see this error again. Because Files are resettable, the SDK is able to retry your request if it encounters an error during the first attempt.