Resumable file upload in python

Question

I am trying to upload the file in python and i want to upload the file in resumable mode i.e when the internet connection resume , the file upload resume from the previous stage.

Is there any specific protocol that supports resumable file upload.

Thanks in advance

Answer 1

So what you need is to seek the file and send a REST command to tell the server to download from the right location.

Here is code that would try untill it finishes to upload with resume, also debug is on so you can follow:

#!/usr/bin/env python3
import ftplib
import os
import sys
import time
import socket

class FtpUploadTracker:
    sizeWritten = 0
    totalSize = 0.0
    lastShownPercent = 0

    def __init__(self, totalSize):
        self.totalSize = totalSize

    def handle(self, block):
        self.sizeWritten += 1024
        percentComplete = round((self.sizeWritten / self.totalSize) * 100)

        if (self.lastShownPercent != percentComplete):
            self.lastShownPercent = percentComplete
            print(str(percentComplete) + "% complete ramaing: " + str(self.totalSize - self.sizeWritten), flush=True)



if __name__ == "__main__":
    Server="servername.com"
    Username="username"
    Password="secret password"
    filename = "/path/to/folder"
    Directory="/path/on/server"

    tries = 0
    done = False

    print("Uploading " + str(filename) + " to " + str(Directory), flush=True)

    while tries < 50 and not done:
        try:
            tries += 1
            with ftplib.FTP(Server) as ftp:
                ftp.set_debuglevel(2)
                print("login", flush=True)
                ftp.login(Username, Password)
                # ftp.set_pasv(False)
                ftp.cwd(Directory)
                with open(filename, 'rb') as f:
                    totalSize = os.path.getsize(filename)
                    print('Total file size : ' + str(round(totalSize / 1024 / 1024 ,1)) + ' Mb', flush=True)
                    uploadTracker = FtpUploadTracker(int(totalSize))

                    # Get file size if exists
                    files_list = ftp.nlst()
                    print(files_list, flush=True)
                    if os.path.basename(filename) in files_list:
                        print("Resuming", flush=True)
                        ftp.voidcmd('TYPE I')
                        rest_pos = ftp.size(os.path.basename(filename))
                        f.seek(rest_pos, 0)
                        print("seek to " + str(rest_pos))
                        uploadTracker.sizeWritten = rest_pos
                        print(ftp.storbinary('STOR ' + os.path.basename(filename), f, blocksize=1024, callback=uploadTracker.handle, rest=rest_pos), flush=True)
                    else:
                        print(ftp.storbinary('STOR ' + os.path.basename(filename), f, 1024, uploadTracker.handle), flush=True)
                        done = True

        except (BrokenPipeError, ftplib.error_temp, socket.gaierror) as e:
            print(str(type(e)) + ": " + str(e))
            print("connection died, trying again")
            time.sleep(30)


    print("Done")

The magic line is:

print(ftp.storbinary('STOR ' + os.path.basename(filename), f, blocksize=1024, callback=uploadTracker.handle, rest=rest_pos), flush=True)

Which has rest=rest_pos.

If optional rest is given, a REST command is sent to the server, passing rest as an argument. rest is usually a byte offset into the requested file, telling the server to restart sending the file’s bytes at the requested offset, skipping over the initial bytes. Note however that RFC 959 requires only that rest be a string containing characters in the printable range from ASCII code 33 to ASCII code 126. The transfercmd() method, therefore, converts rest to a string, but no check is performed on the string’s contents. If the server does not recognize the REST command, an error_reply exception will be raised. If this happens, simply call transfercmd() without a rest argument

Source
Also some code taken from here