Creating new collection and document with Firestore REST API returning HTTP 400

Question

Hello stackoverflow community,

I am using Firestore and not Realtime database for a test project I am working on.

• As a starter, I am trying to create a new document inside an existing collection and project which I manually created through the web interface. My project id is newproject-30f72 and the already created collection name is testcollection.
• I have enabled the Email/Password authentication method, and I have registered a new email-id/password user. The rule I have for auth enabled write is as follows.

service cloud.firestore {
  match /databases/{database}/documents {
    match /{document=**} {
      allow read, write: if request.auth.uid != null;
    }
  }
}

At this point, I am not sure, a barer token alone passed in the header is enough to validate the request.auth.uid != null, so most probably my rule could also be wrong, as I was using the simulator to auto generate the rule, but I verified that the rule is not causing the problem I am facing at the moment by disabling the rule completely.

• I have enabled rules so that only authenticated users can write to firestore. (and by authentication, I mean the idToken for a specific email/password based user and not the apiKey for a web app. I don't know if this is possible though.)
• I have enabled the Add Firebase to your web app / register app and that gave me a json object named firebaseConfig with the following details.

   var firebaseConfig = {
      apiKey: "some_api_key",
      authDomain: "newproject-30f72.firebaseapp.com",
      databaseURL: "https://newproject-30f72.firebaseio.com",
      projectId: "newproject-30f72",
      storageBucket: "newproject-30f72.appspot.com",
      messagingSenderId: "111122223333",
      appId: "1:1111:web:2222"
   };  

I am using the apiKey from the above json object to get an idToken valid for some time, to write to the firestore database. The code for that is as below in python3.

import json
from urllib import request, error
from collections import defaultdict

firebase_apikey = 'some_api_key'
auth_request_url = "https://www.googleapis.com/identitytoolkit/v3/relyingparty/verifyPassword?key={}".format(firebase_apikey)

class auth:
    def __init__(self, email: str, password: str):
        self.email = email
        self.password = password
        self.post_data = defaultdict()
        self.post_data['email'] = self.email
        self.post_data['password'] = self.password
        self.post_data['returnSecureToken'] = True
        self.headers = {"Content-Type": "application/json"}
        # print("POST DATA IS:: " + json.dumps(self.post_data))
        self.r = request.Request(auth_request_url,
                                 data=json.dumps(self.post_data).encode(),
                                 headers=self.headers)
        # self.url_open

    def get_auth_token(self):
        try:
            self.url_open = request.urlopen(self.r)
        except Exception as e:
            return e
        try:
            return json.loads(self.url_open.read())
        except Exception as e:
            return e


s = auth("[email protected]", "somepassword")
response = s.get_auth_token()
id_token = response['idToken']
expires_in = response['expiresIn']

I got the idToken which is a 924 character long string.

Now I am trying to write to firestore with the idToken that I received and valid for 3600(seconds I presume) using the Authorization': 'Bearer header as follows.

firestore_project_url = "https://firestore.googleapis.com/v1beta1/projects/{}/databases/(default)/documents:write".format(
    'newproject-30f72')

headers = {
    'Content-type': 'application/json',
    'Authorization': 'Bearer %s' % id_token,
}

test_data = '''
{
    "writes": [{
        "updateMask": {
            "fieldPaths": ["name"]
        },
        "update": {
            "name": "projects/newproject-30f72/databases/(default)/documents/testcollection/testdoc/",
            "fields": {
                "name": {
                    "stringValue": "test"
                }
            }
        }
    }]
}
'''

test_data_json_bytes = json.dumps(test_data).encode("utf-8")
req = request.Request(url=firestore_project_url,
                      data=test_data_json_bytes,
                      headers=headers,
                      method='POST')
print(headers)
f = request.urlopen(req)

I can see the headers as

{'Content-type': 'application/json', 'Authorization': 'Bearer eyJhbGciOiJSUzI1NiIsI........<snip>'}

but I am getting a urllib.error.HTTPError: HTTP Error 400: Bad Request error.

I was referring to Firebase Firestore REST example to check it using curl with custom headers to add the bearer token and to see if there is any possible verbosity, and I am seeing

*   Trying 74.125.68.95...
* TCP_NODELAY set
* Connected to firestore.googleapis.com (74.125.68.95) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Mountain View; O=Google LLC; CN=*.googleapis.com
*  start date: May 14 13:35:00 2019 GMT
*  expire date: Aug  6 13:20:00 2019 GMT
*  subjectAltName: host "firestore.googleapis.com" matched cert's "*.googleapis.com"
*  issuer: C=US; O=Google Trust Services; CN=Google Internet Authority G3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x560e5884af30)
> POST /v1beta1/projects/newproject-30f72/databases/(default)/documents/testcollection3 HTTP/2
> Host: firestore.googleapis.com
> User-Agent: curl/7.64.1
> Accept: */*
> {Content-type: application/json, Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6........<snip>}
> Content-Type: application/json
> Content-Length: 258
> 
* We are completely uploaded and fine
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 400 
< content-type: text/html; charset=UTF-8
< referrer-policy: no-referrer
< content-length: 1555
< date: Fri, 31 May 2019 14:31:36 GMT
< 
* HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)
* stopped the pause stream!
* Connection #0 to host firestore.googleapis.com left intact
curl: (92) HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)
* Closing connection 0

I was referring to the following links for references.

https://groups.google.com/forum/#!topic/google-cloud-firestore-discuss/4Le2RskC3cg https://firebase.google.com/docs/firestore/reference/rest/v1beta1/projects.databases.documents/commit

What I am trying to achieve is

• use the email/password auth to get authenticated to firestore
• create collection and write docs to it.

Any help is greatly appreciated.

Thank you very much.

Answer 1

SO this is what worked for me (I used requests module but net effect is the same)

First the Auth, exactly as you do.

url = "https://www.googleapis.com/identitytoolkit/v3/relyingparty/verifyPassword?key=" + API_KEY
payload = {"email": email, "password": password, "returnSecureToken": True}
rsp = requests.post(url, data=payload)
id_token = rsp.json().get("idToken")

# User idToken from your auth for later requests
headers = {
    'Content-type': 'application/json',
    'Authorization': "Bearer %s" % id_token
}

Now create a document in a collection

# Create a doc in "testcoll"
url = "https://firestore.googleapis.com/v1beta1/projects/" + PROJECT + "/databases/(default)/documents/testcoll"
payload = {
  "fields": {
    "meaningOfLife": {"integerValue": 42}
  }
}
# Create Doc
rsp = requests.post(url, headers=headers, data = json.dumps(payload))
assert(rsp.status_code == 200)
docref = rsp.json().get("name")
print(rsp.json())

Now update the same document

# Update doc
url = "https://firestore.googleapis.com/v1beta1/" + docref
payload = {
  "fields": {
    "meaningOfLife": {"stringValue": '43'}
  }
}
rsp = requests.patch(url, headers=headers, data = json.dumps(payload))
assert(rsp.status_code == 200)
print(rsp.json())