Google Cloud Storage gives 'insufficient permissions'

Question

I'm using this endpoint:

 get_media(bucket=*, object=*, ifGenerationNotMatch=None, generation=None, ifMetagenerationMatch=None, ifGenerationMatch=None, ifMetagenerationNotMatch=None, projection=None) 

which gives me the error:

apiclient.errors.HttpError: <HttpError 403 when requesting https://www.googleapis.com/storage/v1/b/my-logs/o/clickstream.1413365729497.log?alt=json returned "Insufficient Permission">

The code:

service = build('storage', 'v1')
contents = service.objects().get(bucket=item['bucket'], object=item['name']).execute(http=http)

The item is from a previous (successful) buckets.list call. Which is why the permission error for the service account is so strange.

This is currently on my localhost, and my authentication is:

with open(FILE_KEY, 'rb') as f:
    key = f.read()

credentials = SignedJwtAssertionCredentials(
    SERVICE_EMAIL_ADDRESS,
    key,
    scope='https://www.googleapis.com/auth/devstorage.full_control',
)

http = httplib2.Http()
http = credentials.authorize(http)

The objects (and new test objects I uploaded afterwards), are created with the default acl permissions. How come this doesn't work for a service account to retrieve the file contents?

Answer 1

Two things that may be interfering here:

1. Listing the objects in a bucket requires READ permission on the bucket, whereas getting an object, even just its metadata, requires READ permission on the object. It's possible your service account has the first permission and not the second.
2. Service accounts, by default, are not part of the project Owners, Editors or Viewers groups, so they do not show up on either bucket default ACLs or object default ACLs.

Does your bucket ACL have the service account on it explicitly? Does the object? Is the service account listed in the bucket's default object ACL, so new objects written that don't otherwise override that ACL allow the service account to read it?

See also Access Control.