POST file (as binary stream) using java.net.HttpURLConnection as a param file=<file binary stream>

Question

I am trying to upload (POST) a file to an endpoint using java.net.HttpURLConnection but I keep getting http code 400 (bad request).

I refered to Send File And Parameters To Server With HttpURLConnection in android API 23

but problem is that I need to send this file as request body param (file=).

Note: The files will be of small size only (4-5mb) so I am reading it entirely in memory.

Corresponding curl request is:

curl -X POST "API" -H "Content-Type: multipart/form-data" -F "file="

---

Excerpts of Code that I am using:

    Proxy webproxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(" 
                                <proxy host>", <proxy_port>));
    HttpURLConnection http_conn = (HttpURLConnection) 
                                     url.openConnection(webproxy);
    String authorization = getAuthorization(access_token);
    http_conn.setRequestMethod("POST");
    http_conn.setRequestProperty("Accept-Charset", "UTF-8");        
    http_conn.setRequestProperty("Authorization", authorization);
    http_conn.setRequestProperty("Connection", "Keep-Alive");
    http_conn.setRequestProperty("Content-Type", "multipart/form-data);  
    http_conn.setDoOutput(true);
    http_conn.setDoInput(true);
    DataOutputStream outputStream;
    outputStream = new DataOutputStream(http_conn.getOutputStream());
    File file_obj = new File(this.file);                                
    byte[] allBytes = new byte[(int) file_obj.length()];
    FileInputStream fileInputStream = new FileInputStream(file_obj);                
    outputStream.write("file=".getBytes("UTF-8")); <---Trying to add file param here
    fileInputStream.read(allBytes);
    outputStream.write(allBytes);

Post that I just read response using below piece of code (works fine for different GET requests):

    InputStream inputStream = http_conn.getInputStream();            
        BufferedReader bufferedReader = new BufferedReader(new 
    InputStreamReader(inputStream));
    String line = "";            
    while ((line = bufferedReader.readLine()) != null) {
        data = data + line;                
    }

Note: I use java rarely an am not very familiar with it so please be descriptive in your response.

Answer 1

When looking at your curl command line, it shows that the file needs to be send as a multipart/form-data request. This is actually a complex way of formatting your data when it is requires.

An example of the format you need to send is:

Headers:

Content-Type: multipart/form-data; boundary=AaB03x

Body:

--AaB03x
Content-Disposition: form-data; name="files"; filename="file1.txt"
Content-Type: text/plain

... contents of file1.txt ...
--AaB03x--

At the moment, your code is sending the file as a POST/GET formatted request, and this doesn't work as the backend isn't expecting that.

To solve this problem, we need to format the source files into the format required by the backend, and once you know that the "boundary" header option is just a randomly generated value, it becomes more easy to send the request.

String boundary = "MY_AWESOME_BOUNDARY"
http_conn.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary);  

try(DataOutputStream outputStream = new DataOutputStream(http_conn.getOutputStream())) {
    File file_obj = new File(this.file);                      

    // Write form-data header   
    outputStream.write(("--" + boundary + "\r\n").getBytes("UTF-8"));
    outputStream.write(("Content-Disposition: form-data; name=\"file\"; filename=\"file1.txt\"\r\n").getBytes("UTF-8"));
    outputStream.write(("Content-Type: text/plain\r\n").getBytes("UTF-8"));
    outputStream.write(("\r\n").getBytes("UTF-8"));
    // Write form-data body
    Files.copy(file_obj.toPath(), outputStream)
    // Write form-data "end"
    outputStream.write(("--" + boundary + "--\r\n").getBytes("UTF-8"));
}
// Read backend response here
try(InputStream inputStream = http_conn.getInputStream()) {           
    BufferedReader bufferedReader = new BufferedReader(new 
    InputStreamReader(inputStream));
    StringBuilder lines = new StringBuilder(); // StringBuilder is faster for concatination than appending strings           
    while ((line = bufferedReader.readLine()) != null) {
        lines.append(line);                
    }
    System.out.println(lines);
}

Note that I used "try-with-resource" blocks, these blocks make sure that any external resources are closed and disposed when you are done using them, generally the open resource limit of the OS is very low, compared to the amount of memory your program has, so what happens is that your program could give weird errors that only happens after some time of running or when the user executes certain actions inside your application

Answer 2

The above didnt worked for me so I switched to different package (okhttp3), here is what worked for me:

    File file_obj = new File(this.file); 
    String authorization = "my authorization string";
    Proxy webproxy = new Proxy(Proxy.Type.HTTP, new 
          InetSocketAddress("proxy", <port>));

    OkHttpClient client = new OkHttpClient.Builder().proxy(webproxy).build();      

    RequestBody requestBody = new MultipartBody.Builder().setType(MultipartBody.FORM).addFormDataPart("file", "filename",
        RequestBody.create(MediaType.parse("application/octet-stream"), file_obj)).build();

    Request request = new Request.Builder().header("Authorization", authorization).url(this.url).post(requestBody).build();

    try (Response response = client.newCall(request).execute()){
        if(!response.isSuccessful()) return "NA";
        return (response.body().string());
    }