Extract depthmap from Google Street View

Question

Many posts such as this one use the REST API of Google Street View like this:

http://maps.google.com/cbk?output=xml&ll=40.7625000,-73.9741670&dm=1

However, this API does not work sometimes.

Google Street View seems does not rely on that REST API, and it may store the depthmap in this meta-file:

https://www.google.com/maps/photometa/v1?authuser=0&hl=en&gl=us&pb=!1m4!1smaps_sv.tactile!11m2!2m1!1b1!2m2!1sen!2sus!3m3!1m2!1e2!2sG2__WfQLZTzrsr7FP1wUyQ!4m57!1e1!1e2!1e3!1e4!1e5!1e6!1e8!1e12!2m1!1e1!4m1!1i48!5m1!1e1!5m1!1e2!6m1!1e1!6m1!1e2!9m36!1m3!1e2!2b1!3e2!1m3!1e2!2b0!3e3!1m3!1e3!2b1!3e2!1m3!1e3!2b0!3e3!1m3!1e8!2b0!3e3!1m3!1e1!2b0!3e3!1m3!1e4!2b0!3e3!1m3!1e10!2b1!3e2!1m3!1e10!2b0!3e3

But I do not know how to extract depthmap from it. The method using Base64 in this post of Convert depth map (base64) of google street view to image does not work because the encoded string are very different.

Could anyone can get me some ideas? Thanks very much!

Here are some Python code lines to get the meta-file from Google Street View:

def getPanoJS_frm_lonlat(lon, lat):
        url = "https://maps.googleapis.com/maps/api/js/GeoPhotoService.SingleImageSearch?pb=!1m5!1sapiv3!5sUS!11m2!1m1!1b0!2m4!1m2!3d{0:}!4d{1:}!2d50!3m10!2m2!1sen!2sGB!9m1!1e2!11m4!1m3!1e2!2b1!3e2!4m10!1e1!1e2!1e3!1e4!1e8!1e6!5m1!1e2!6m1!1e2&callback=_xdc_._v2mub5"
        url = url.format(lat, lon)
        return requests.get(url, proxies=None)

lon = -74.2249174
lat = 40.7040015
resp = getPanoJS_frm_lonlat(lon, lat)

line = resp.text.replace("/**/_xdc_._v2mub5 && _xdc_._v2mub5( ", "")[:-2]
# print(line)
jdata = json.loads(line)

print("This part may be the depthmap:")
print(jdata[1][5][0][5][3][2])

A part of the encoded depthmap in the metafile(not quite sure, and the entire string is too long to post here, please using the code above the download the meta-file):

Q0RrQUFBSUFBUWdCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUURUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFFQkFRRUJBUUVCQVFRlZyZDFsalYwWmtVVGxoUlUxQmJYazRlaTFKYVdoMlZHYzJhbWhzZVVwNFJrNVNaRWRMYjNObFdVNWhVVk4zUXkxblJtTTBhM2hLYjFWTllYSTBjRFJtTmxOUFp6TnhRWEJ4VkRkSVVtaHVZMGN0T1Uxa1ptMXNXR2MzZDB4MVVGSkdVVVpOVEdRNGNIcFZSMWxSUkRoM0FBQUFBQUFBQUlBaWVzbTlzWFVnd1M3QzBUeXpzeHRCa0FHaE84WThtRUVxaENTX0lCRDN3Y1RYV3I3UkthTEJuZElYd0h0MkpzSmVZd0ZBZjM5Q3dzZjhjc0F2dGt2Q2VzV3d2X1dkY01JOTNRVEJsVGxBd2lKNlJzTEk4empDZVpKdHdvcXNPTUtvOXV6QnJzVTV3cDZNaXNMVDR6akNramtmd2h0U09jTFZaR2pDeUJBalF0cUZKOFBhS3pyQ01TM0N3dHp5SlVMUjF0WENmdklsUWl4WE1jT3I4ampDUGVjVXd1djJIMExfMk11OXJYN2dRUXJ6UHNKT2pBRkRES0dUdjRLdm0wTElEWnFfZG5HR1FnbGM1OEgwRUI1Q1BHU2V3WU1aSFVLV0xxSEI0eWdJd3lPakZjTEIwUUZEa2xGZHYzY1ZzRUl1RmtlX1RLRmhRc21tenJ3QjZUUkQxNThDdjdXZ3cwSkc5NEUtZXhUb1F1c1QzVUdUYmdKRGtlMzJRV2gtSDBJUjc2VkJUOFVlUWg3RkZFS3ViUUpERFlzalFvYzJJRUpTUTB0Q1RCd2dRbEhJY2tMV3RSOUNNVktiUVRGcVBNS1dIZVpCb2ZZOXdpQmFRRUowNTBIQ0Jud1pRcDV6UU1MNml0Q18wWWFjd3V6RG5FSTNFd25EZUNZc1FlODhQTUl1dThDX2tFLUt3dTF4RDhETVBzUEM0aGhCd241Z0lrSThfSG9fOTRRMFFwOExKa0ZoZUI1QzE3OTN3T3VORkVMSEQ1akJRZnM2d2c=

Answer 1

I recently came across this problem too and I decided to dig into it with the help of a friend.

I found the GSVPanoDepth.js library that hadn't been updated in a while but still had the fundamentals of decoding the data and parsing it. It was all pretty much the same except the url has changed and the structure of the data a bit too. I forked the repo and updated it to work with the new url and structure - https://github.com/sidequestlegend/GSVPanoDepth.js

 // This url no longer works
    // url = "http://maps.google.com/cbk?output=json&cb_client=maps_sv&v=4&dm=1&pm=1&ph=1&hl=en&panoid=" + panoId;

    url = "https://www.google.com/maps/photometa/v1?authuser=0&hl=en&gl=uk&pb=!1m4!1smaps_sv.tactile!11m2!2m1!1b1!2m2!1sen!2suk!3m3!1m2!1e2!2s" +
        panoId +
        "!4m57!1e1!1e2!1e3!1e4!1e5!1e6!1e8!1e12!2m1!1e1!4m1!1i48!5m1!1e1!5m1!1e2!6m1!1e1!6m1!1e2!9m36!1m3!1e2!2b1!3e2!1m3!1e2!2b0!3e3!1m3!1e3!2b1!3e2!1m3!1e3!2b0!3e3!1m3!1e8!2b0!3e3!1m3!1e1!2b0!3e3!1m3!1e4!2b0!3e3!1m3!1e10!2b1!3e2!1m3!1e10!2b0!3e3"

    fetch(url)
        .then(res=>res.text())
        .then(res=>JSON.parse(res.substr(4)))
        .then(res=>res[1][0][5][0][5][1][2])
        .then(res=>{
            console.log(res);
            return res;
        })
        .then(dm => {
            var decoded, depthMap;
            try {
                decoded = self.decode(dm);
                depthMap = self.parse(decoded);
            } catch(e) {
                console.error("Error loading depth map for pano " + panoId + "\n" + e.message + "\nAt " + e.filename + "(" + e.lineNumber + ")");
                depthMap = self.createEmptyDepthMap();
            }
            if(self.onDepthLoad) {
                self.depthMap = depthMap;
                self.onDepthLoad();
            }
        })
        .catch(e=>{
            console.error("Request failed: " + url + "\n" + e);
            var depthMap = self.createEmptyDepthMap();
            if(self.onDepthLoad) {
                self.depthMap = depthMap;
                self.onDepthLoad();
            }
        });