How to include a pipe executing a command with FileUtils#sh?

Question

I know about enough Ruby to be dangerous, so I'm writing a bunch of build automation scripts as Rake tasks mostly by working from examples in the API documentation.

I'm scripting a bunch of build commands with code like this:

results = {}
schemes.each { |scheme|
    command = "xcodebuild", "-scheme", scheme
    if options.configuration
        command.push "-configuration", options.configuration
    end
    command.push "archive"

    sh *command do |ok, res|
        results[scheme] = ok
    end
}

This works fine, but dumps a lot of output to the console that kind of gets in the way. So I'd love to pipe this stuff through xcpretty to reformat it.

If I actually run the command from my terminal, it works fine:

xcodebuild -scheme Foo -configuration Release archive | xcpretty

But if I modify the Rake task to just append those parts to the end of the command array, sh thinks they're arguments to xcodebuild.

    command.push "archive", "|", "xcpretty"

I've seen a number of examples using sh with pipes in its string form, but not the array form. Because of the nature of the commands I'm building—optional arguments, arguments with spaces and other characters—the array form is much cleaner to deal with. But is there a way to include output piping when using it?

Answer 1

A good solution would be to use Open3, a standard part of Ruby which has lots of utilities for opening processes and arranging their pipes.

Here is an example that worked for me:

require 'open3'
Open3.pipeline(['ls', '-l'], ['grep', 'z'])

When you run this, the standard output pipe of the last command in the pipeline gets connected to the pre-existing standard output pipe of the Ruby process. This means that you will be able to see any output immediately as it is produced instead of waiting for the processes to end. Another benefit of this method is that it does not spawn a shell; it just spawns the processes directly.

Another option is to simply use Array#join to join the arguments together with spaces before passing them to sh. You just have to watch out for the possibility that your arguments might have spaces in them, and if so then you have to quote or escape them properly. This method would spawn a shell to process your command, and it should also provide immediate output for long-running processes.