how to fetch all records (more than 100) from the airtable using axios?

Question

I am trying to fetch all the records present in the airtable but it's only getting me 100 records only but i need to fetch more than 100 records.

loadListings(){
  var self = this;
  var app_id = "**********";
  var app_key = "**********";
  axios.get(
      "https://api.airtable.com/v0/"+app_id+"/Weekly%20Report?view=Main%20View",
      {
          headers: { Authorization: "Bearer "+app_key }
      }
  ).then(function(response){
    console.log(response.data.records);
    self.listings = response.data.records;
  }).catch(function(error){
    console.log(error)
  });
}

Answer 1

Airtable API returns 100 records at most. You can't change that. It has pagination included by default.

You need to make multiple calls to get all your data. The response will include a property called offset. You need to retrieve it and use it in your next call.

https://api.airtable.com/v0/"+app_id+"/Weekly%20Report?view=Main%20View&offset=previously_saved_offset

The next call will return a new offset and so one until you reach the end of your records.

Answer 2

TLDR:
You could try using a recursive function that executes when an offset exists in the http response (json).

Acknowledgment:
This solution took 7 hours of research, troubleshooting, and advice from the great, and powerful Doug. The project uses Alamofire to perform http requests, and SwiftyJson to access the JSON.

Cause:
In the Airtable documentation, they state their rate limit is 100 items per request. If a request has more than 100 items, the request will include an offset.

They give an instruction to include the offset in your next request to get the next 100 results. But, they don't explain or demonstrate how to do it.

Solution:
It's been tested to retrieve 2,565 items from 25 http requests. Written in Swift, the logic is simple:

Write a recursive function with an argument for an optional offset. Call the function without the offset. Check the http response (json) for the offset. If the offset exists, store the http request (json) in an array outside the function. Then, call that same function from inside itself - this time with the offset.

Extended code here.

func requestAirtableRecords(forTable table: String, withTableView tableView: String, withOffset offset: String?, completion: @escaping ([JSON]) -> ()) {
    let parameters: [String: Any] = offset != nil ? ["view": tableView, "offset": offset!] : ["view": tableView]
    do {
        let url: URLRequest = try self.requestRecordsURL(table: table, method: HttpRequest.get, parameters: parameters)!
        Alamofire.request(url).responseJSON { (response) in
            switch response.result {
            case .success(_):
                let json = JSON(response.result.value!)
                self.jsonArray.append(json)
                let nextOffset = json["offset"]
                if nextOffset.exists() {
                    self.requestAirtableRecords(forTable: table, withTableView: tableView, withOffset: nextOffset.stringValue, completion: { _ in
                        completion(self.jsonArray)
                    })
                } else {
                    completion(self.jsonArray)
                }
            case .failure(let error):
                print(error)
            }
        }
    } catch {
        print("Error: Unable to request records from Airtable.")
    }
}