Net::SFTP Errors

Question

I have been trying to download a file using Net::SFTP and it keeps getting an error.

The file is partially downloaded, and is only 2.1 MB, so it's not a huge file. I removed the loop over the files and even tried just downloading the one file and got the same error:

yml = YAML.load_file Rails.root.join('config', 'ftp.yml')
Net::SFTP.start(yml["url"], yml["username"], password: yml["password"]) do |sftp|
  sftp.dir.glob(File.join('users', 'import'), '*.csv').each do |f|
    sftp.download!(File.join('users', 'import', f.name), Rails.root.join('processing_files', 'download_files', f.name), read_size: 1024)
  end
end

NoMethodError: undefined method `close' for #<Pathname:0x007fc8fdb50ea0>
from /[my_working_ap_dir]/gems/net-sftp-2.1.2/lib/net/sftp/operations/download.rb:331:in `on_read'

I have prayed to Google all I can and am not getting anywhere with it.

Answer 1

Rails.root returns a Pathname object, but it looks like the sftp code doesn't check to see whether it got a Pathname or a File handle, it just runs with it. When it runs into entry.sink.close it crashes because Pathnames don't implement close.

Pathnames are great for manipulating paths to files and directories, but they're not substitutes for file handles. You could probably tack on to_s which would return a string.

Here's a summary of the download call from the documentation that hints that the expected parameters should be a String:

To download a single file from the remote server, simply specify both the
remote and local paths:

  downloader = sftp.download("/path/to/remote.txt", "/path/to/local.txt")

I suspect that if I dig into the code it will check to see whether the parameters are strings, and, if not, assumes that they are IO handles.

See ri Net::SFTP::Operations::Download for more info.

---

Here's an excerpt from the current download! code, and you can see how the problem occurred:

def download!(remote, local=nil, options={}, &block)
  require 'stringio' unless defined?(StringIO)
  destination = local || StringIO.new
  result = download(remote, destination, options, &block).wait
  local ? result : destination.string
end

local was passed in as a Pathname. The code checks to see if there's something passed in, but not what that is. If nothing is passed in it assumes it's something with IO-like features, which is what StringIO provides for the in-memory caching.

Answer 2

Apparently you can't use Rails.root.join, which was causing the problem. It is really stupid though because it would download part of the file.

Changed:

sftp.download!(File.join('users', 'import', f.name), Rails.root.join('processing_files', 'download_files', f.name))

To:

sftp.download!(File.join('users', 'import', f.name), File.join('processing_files', 'download_files', f.name))