libvirt on Apple Silicon with qemu-system-aarch64

Question

Compiled qemu from this manual

Try to run this on Apple Silicon with qemu-system-aarch64, but got an error:

Error: internal error: Failed to start QEMU binary /usr/local/bin/qemu-system-aarch64 for probing: qemu-system-aarch64: invalid accelerator kvm Could not allocate dynamic translator buffer

This is the XML:

<domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
    <name>ubuntu</name>
    <uuid>2005CB24-522A-4485-9B9A-E60A61D9F8CF</uuid>
    <memory unit='GB'>2</memory>
    <cpu mode='custom'>
        <model>Westmere</model>
    </cpu>
    <vcpu>2</vcpu>
    <features>
        <acpi/>
        <apic/>
    </features>
    <os>
        <type arch='aarch64' machine='cortex-a57'>hvf</type>
        <bootmenu enable='yes'/>
    </os>
    <clock offset='localtime'/>
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>destroy</on_crash>
    <pm>
        <suspend-to-mem enabled='no'/>
        <suspend-to-disk enabled='no'/>
    </pm>
    <devices>
        <emulator>/usr/local/bin/qemu-system-aarch64</emulator>
        <controller type='usb' model='ehci'/>
        <disk type='file' device='disk'>
            <driver name='qemu' type='qcow2'/>
            <source file='/Users/matthias/VM/Ubuntu_20.04-LTS/disk.qcow2'/>
            <target dev='vda' bus='virtio'/>
        </disk>
        <!--disk type='file' device='cdrom'>
            <source file='/Users/matthias/VM/Ubuntu_20.04-LTS/ubuntu-20.04.2-live-server-arm64.iso'/>
            <target dev='sdb' bus='sata'/>
        </disk-->
        <console type='pty'>
            <target type='serial'/>
        </console>
        <input type='tablet' bus='usb'/>
        <input type='keyboard' bus='usb'/>
        <graphics type='vnc' port='5900' listen='127.0.0.1'/>
        <video>
            <model type='virtio' vram='16384'/>
        </video>
    </devices>
    <seclabel type='none'/>
    <qemu:commandline>
        <!--qemu:arg value='-machine'/>
        <qemu:arg value='type=q35,accel=hvf'/>
        <qemu:arg value='-netdev'/>
        <qemu:arg value='user,id=n1,hostfwd=tcp::2222-:22'/>
        <qemu:arg value='-device'/>
        <qemu:arg value='virtio-net-pci,netdev=n1,bus=pcie.0,addr=0x19'/-->
    <qemu:arg value='-accel hvf -m 2048 -cpu cortex-a57 -M virt,highmem=off'/>
    <qemu:arg value='-drive file=/usr/local/share/qemu/edk2-aarch64-code.fd,if=pflash,format=raw,readonly=on'/>
    <qemu:arg value='-drive file=ovmf_vars.fd,if=pflash,format=raw'/>
    <qemu:arg value='-serial telnet::4444,server,nowait'/>
    <qemu:arg value='-device virtio-blk-device,drive=hd0,serial="dummyserial"'/>
    <qemu:arg value='-device virtio-net-device,netdev=net0'/>
    <qemu:arg value='-netdev user,id=net0,hostfwd=tcp:127.0.0.1:2222-0.0.0.0:22'/>
    <qemu:arg value='-vga none -device ramfb'/>
    <qemu:arg value='-device usb-ehci -device usb-kbd -device usb-mouse -usb'/>
    <qemu:arg value='-nographic -serial mon:stdio'/>
    </qemu:commandline>
</domain>

Thx for any feedback!

Answer 1

You need to pass -machine accel=hvf,highmem=off as a QEMU argument. I've got a fully working Libvirt + QEMU solution in https://github.com/ihsakashi/VM . Libvirt couldn't identify the aarch64 architecture using uname since macOS gave something different. That's been patched.

Answer 2

Instead of using XML and virsh, I found it is much easier to use shell to call QEMU directly. Here is the script I am using:

qemu-system-aarch64 \
         -machine virt,accel=hvf,highmem=off \
         -cpu cortex-a72 -smp 2 -m 4G \
         -device intel-hda -device hda-output \
         -device qemu-xhci \
         -device virtio-gpu-gl-pci \
         -device usb-kbd \
         -device virtio-mouse-pci \
         -display cocoa,gl=es \
             -device e1000,netdev=net0 \
             -netdev user,id=net0 \
         -drive "if=pflash,format=raw,file=#{firmware_path}/edk2-aarch64-code.fd,readonly=on" \
         -drive "if=pflash,format=raw,file=#{firmware_path}/edk2-arm-vars.fd,discard=on" \
         -drive "if=virtio,format=qcow2,file=#{disk_file_location},discard=on" \
             -chardev qemu-vdagent,id=spice,name=vdagent,clipboard=on \
             -device virtio-serial-pci \
             -device virtserialport,chardev=spice,name=com.redhat.spice.0

If you happen to use vagrant, you may find my plugin useful: https://github.com/billyan2018/vagrant-qemu