creating Linux i386 a.out executable shorter than 4097 bytes

Question

I'm trying to create a Linux i386 a.out executable shorter than 4097 bytes, but all my efforts have failed so far.

I'm compiling it with:

$ nasm -O0 -f bin -o prog prog.nasm && chmod +x prog

I'm testing it in a Ubuntu 10.04 i386 VM running Linux 2.6.32 with:

$ sudo modprobe binfmt_aout
$ sudo sysctl vm.mmap_min_addr=4096
$ ./prog; echo $?
Hello, World!
0

This is the source code of the 4097-byte executable which works:

; prog.nasm
        bits 32
        cpu 386
        org 0x1000  ; Linux i386 a.out QMAGIC file format has this.

SECTION_text:
a_out_header:
        dw 0xcc  ; magic=QMAGIC; Demand-paged executable with the header in the text. The first page (0x1000 bytes) is unmapped to help trap NULL pointer references.
        db 0x64  ; type=M_386
        db 0  ; flags=0
        dd SECTION_data - SECTION_text  ; a_text=0x1000 (byte size of .text; mapped as r-x)
        dd SECTION_end - SECTION_data  ; a_data=0x1000 (byte size of .data; mapped as rwx, not just rw-)
        dd 0  ; a_bss=0 (byte size of .bss)
        dd 0  ; a_syms=0 (byte size of symbol table data)
        dd _start  ; a_entry=0x1020 (in-memory address of _start == file offset of _start + 0x1000)
        dd 0  ; a_trsize=0 (byte size of relocation info or .text)
        dd 0  ; a_drsize=0 (byte size of relocation info or .data)

_start:     mov eax, 4              ; __NR_write
        mov ebx, 1              ; argument: STDOUT_FILENO
        mov ecx, msg            ; argument: address of string to output
        mov edx, msg_end - msg  ; argument: number of bytes
        int 0x80                ; syscall

        mov eax, 1              ; __NR_exit
        xor ebx, ebx            ; argument: EXIT_SUCCESS == 0.
        int 0x80                ; syscall

msg:        db 'Hello, World!', 10
msg_end:

        times ($$ - $) & 0xfff db 0  ; padding to multiple of 0x1000  ; !! is this needed?
SECTION_data:   db 0
;       times ($$ - $) & 0xfff db 0  ; padding to multiple of 0x1000  ; !! is this needed?
SECTION_end:

How can I make the executable file smaller? (Clarification: I still want a Linux i386 a.out executable. I know that that it's possible to create a smaller Linux i386 ELF executable.) There is several thousands bytes of padding at the end of the file, which seems to be required.

So far I've discovered the following rules:

• If a_text or a_data is 0, Linux doesn't run the program. (See relevant Linux source block 1 and 2.)
• If a_text is not a multiple of 0x1000 (4096), Linux doesn't run the program. (See relevant Linux source block 1 and 2.)
• If the file is shorter than a_text + a_data bytes, Linux doesn't run the program. (See relevant Linux source code location.)

Thus file_size >= a_text + a_data >= 0x1000 + 1 == 4097 bytes.

The combinations nasm -f aout + ld -s -m i386linux and nasm -f elf + ld -s -m i386linux and as -32 + ld -s -m i386linux produce an executable of 4100 bytes, which doesn't even work (because its a_data is 0), and by adding a single byte to section .data makes the executable file 8196 bytes long, and it will work. Thus this path doesn't lead to less than 4097 bytes.

Did I miss something?

Answer 1

TL;DR It doesn't work.

It is impossible to make a Linux i386 a.out QMAGIC executable shorter than 4097 bytes work on Linux 2.6.32, based on evidence in the Linux kernel source code of the binfmt_aout module.

Details:

• If a_text is 0, Linux doesn't run the program. (Evidence for this check: a_text is passed as the length argument to mmap(2) here.)

• If a_data is 0, Linux doesn't run the program. (Evidence for this check: a_data is passed as the length argument to mmap(2) here.)

• If a_text is not a multiple of 0x1000 (4096), Linux doesn't run the program. (Evidence for this check: fd_offset + ex.a_text is passed as the offset argument to mmap(2) here. For QMAGIC, fd_offset is 0.)

• If the file is shorter than a_text + a_data bytes, Linux doesn't run the program. (Evidence for this check: file sizes is compared to a_text + a_data + a_syms + ... here.)

• Thus file_size >= a_text + a_data >= 0x1000 + 1 == 4097 bytes.

I've also tried OMAGIC, ZMAGIC and NMAGIC, but none of them worked. Details:

• For OMAGIC, read(2) is used instead of mmap(2) within here, thus it can work. However, Linux tries to load the code to virtual memory address 0 (N_TXTADDR is 0), and this causes SIGKILL (if non-root and vm.mmap_min_addr is larger than 0) or SIGILL (otherwise), thus it doesn't work. Maybe the reason for SIGILL is that the page allocated by set_brk is not executable (but that should be indicated by SIGSEGV), this could be investigated further.

• For ZMAGIC and NMAGIC, read(2) instead of mmap(2) within here if fd_offset is not a multiple of the page size (0x1000). fd_offset is 32 for NMAGIC, and 1024 for ZMAGIC, so good. However, it doesn't work for the same reason (load to virtual memory address 0).

• I wonder if it's possible to run OMAGIC, ZMAGIC or NMAGIC executables at all on Linux 2.6.32 or later.