Using argparse.REMAINDER at beginning of parser / sub parser

Question

I want to implement an arg parser that allows me to run unittests as one of the sub commands, blindly passing the arguments on to unittest.main(). e.g.,

$ foo.py unittest [args to pass to unittest.main()]

along with other sub commands:

$ foo.py foo ...
$ foo.py bar ...

Following argparse's example, this works:

#!/usr/bin/python                                                               
import argparse                                                                 

p = argparse.ArgumentParser(prog='PROG')                                        
p.add_argument('-v', '--verbose', action='store_true')                          
sub = p.add_subparsers(dest='cmd')                                              
foo = sub.add_parser('foo')                                                     
bar = sub.add_parser('bar')                                                     
unittest = sub.add_parser('unittest')                                           
unittest.add_argument('command') # Need to add this to make it work.                                              
unittest.add_argument('args', nargs=argparse.REMAINDER)                         

print(p.parse_args('unittest command -blah blah'.split()))       

Output:

Namespace(args=['-blah', 'blah'], cmd='unittest', command='command', verbose=False)

But this doesn't. It seems to require a "normal" argument first:

#!/usr/bin/python                                                               
import argparse                                                                 

p = argparse.ArgumentParser(prog='PROG')                                        
p.add_argument('-v', '--verbose', action='store_true')                          
sub = p.add_subparsers(dest='cmd')                                              
foo = sub.add_parser('foo')                                                     
bar = sub.add_parser('bar')                                                     
unittest = sub.add_parser('unittest')                                           
unittest.add_argument('args', nargs=argparse.REMAINDER)                         

print(p.parse_args('unittest -blah blah'.split()))             

Output:

$ /tmp/foo.py    
usage: PROG [-h] [-v] {foo,bar,unittest} ...
PROG: error: unrecognized arguments: -blah

I can do print(p.parse_args('unittest -- -f -g'.split())), but requiring -- kind of defeats the purpose of argparse.REMAINDER.

Is there a way to get argparse to do what I want? Or do I just need to hand parse this case?

Python 2.7.5

Answer 1

As noted, the existing behavior is bad. One workaround is to implement a simple ArgumentParser subclass and use that for your subparser:

class SubcommandParser(argparse.ArgumentParser):
    """This subparser puts all remaining arguments in args attribute of namespace"""
    def parse_known_args(self, args=None, namespace=None):
        if namespace is None:
            namespace = argparse.Namespace()
        setattr(namespace, 'args', args)
        return namespace, []

...

p.add_subparsers(dest='cmd', parser_class=SubcommandParser) 

Answer 2

Looks like the same issue discussed in http://bugs.python.org/issue17050, argparse.REMAINDER doesn't work as first argument

My deduction from 4 years ago still holds - the -blah is being classed as an optional's flag even before REMAINDER has a chance to act. '--' is parsed earlier, but ... is, in a sense just a generalization of '*'. And not a widely used one. For what it's worth the 'subparsers' Action has a nargs='+...' value (argparse.PARSER) - it's like REMAINDER except it requires at least one string, the 'cmd'.

The possible fix in http://bugs.python.org/issue9334 has not been acted on. So you either need to handle the '-blah' by itself, or use '--'. parse_known_args might also work in your case.