Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why I get NoSuchKey in DigitalOcean Buckets?

I am using DigitalOcean's spaces which in turn is S3 compatible. I created some buckets, I can WITHOUT any problem upload files with Python/aioboto3 using credentials, BUT, I just can't get a list of files contained in a bucket... here's the code in python:

async def list_files(bucket_name):
    session = aioboto3.Session()
    async with session.client(
        's3',
        region_name=config("S3REGION"),
        endpoint_url=config("S3URL"),
        aws_access_key_id=config("S3AK"),
        aws_secret_access_key=config("S3PK"),
        config=Config(signature_version='s3v4')
    ) as s3_client:
        try:
            response = await s3_client.list_objects(Bucket=bucket_name)
            return response.get("Contents", [])
        except Exception as e:
            return "500 - " + str(e) + " // " + bucket_name

forget the return, the problem arises in the line: response = await s3_client.list_objects(Bucket=bucket_name)

which gives the error: "500 - An error occurred (NoSuchKey) when calling the ListObjects operation: Unknown // mybucketname"

TRUST that credentials are ok since the same are using for fileu ploading and they work. I tryed in aioboto3 docs, the whole internet and DigitalOcean's community which has an UNANSWERED same exact issue since 2021.

I'm new in the cloud and hope someone can give me an insight on what might be wrong.

Thanks is advance.

I've tryed uploading a file which works I've tryed to retrieve the buckets which gives an empty array I verifyed in DigitalOcean's spaces manages that all the names are correct I verifyed that credentials work I wrestled with chatgpt which proved useless I looked to the poorly implemented aioboto3 documentation I searched the new with no success IDK what else to do

like image 956
Lychmaster Avatar asked Jan 19 '26 00:01

Lychmaster


1 Answers

Ok, my problem was that I was including my bucket's name in the URL that I was using. As you guys pointed out, I modified my code as follows:

async def list_files(bucket_name, prefix):
    session = aioboto3.Session()
    async with session.client(
        's3',
        region_name=config("S3REGION"),
        endpoint_url=config("S3URLCLEAN"), #This is the url WITHOUT my bucket
        aws_access_key_id=config("S3AK"),
        aws_secret_access_key=config("S3PK"),
        config=Config(signature_version='s3v4')
    ) as s3_client:
        try:
            response = await s3_client.list_objects(Bucket=bucket_name, Prefix=prefix) #the prefix is 'myfoldername/'
            return response.get("Contents", [])
        except Exception as e:
            return "500 - " + str(e) + " // " + bucket_name

Obviously I will work the right error handlers and stuff, but for testing purposes, this works. Now I receive a JSON with all the files in my folder contained in the bucket!!!

Thanks a lot for the real quick help guys!!!

like image 175
Lychmaster Avatar answered Jan 20 '26 14:01

Lychmaster



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!