How to get a Python console to access the vim module

Question

Recently I have been looking into vim plugin development, and I found I missed the ability to use a Python REPL (ipython/bpython for eg) to inspect the vim module, and generally the environment (current open document, line number, selection etc).

This is in principle - not very advanced and something I've done from other applications that embed Python...

Typically you could do this:

import code
code.interact(local=locals())

Or with IPython:

import IPython
IPython.embed()

However when vim embeds Python, it replaces sys.stdin/stdout/stderr, I managed to temp restore these and it very nearly works but there is still some problems with scrambled line endings and stdin isnt reading properly.

eg,

std_back = sys.stderr, sys.stdin, sys.stdout
sys.stderr = sys.__stderr__
sys.stdout = sys.__stdout__
sys.stdin = sys.__stdin__

import IPython
IPython.embed()

sys.stderr, sys.stdin, sys.stdout = std_back

I tried with both vim and gvim on linux, and the stdin/stdout was not working right. (hard to explain, but only every second key input was accepted and newlines were not printed, text wrapping).

So my final attempt was to use idle, which bypasses the terminal and opens up a tk interface with a command line.

import idlelib
import idlelib.PyShell
idlelib.PyShell.main()

This loads the console, but in a sub-process so no access to vim module, I checked on the idlelib source code, and found you can disable subprocess use by faking a command line argument which idle would normally access from being launched from the command line directly.

import sys
sys.argv.append("-n")

import idlelib
import idlelib.PyShell
idlelib.PyShell.main()

Ok, so this works, but, Python devs are going to remove idlelib's option to run inside the process (currently its deprecated). Do any other devs know a way of using a Python REPL within vim called from pyfile / py3file ?

Answer 1

This is a simple interpreter shell that listens on a network socket:

import socket
import sys
import code
from threading import Thread


def remote_shell(local, host='localhost', port=5555):
    serv = socket.socket()
    serv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    serv.bind((host, port))
    serv.listen(1)
    streams = sys.stdout, sys.stderr
    print('listening on %s:%d' % (host, port))
    try:
        sock, addr = serv.accept()
        f = sock.makefile('rw')
        sys.stdout = sys.stderr = f

        def read(prompt):
                f.write(prompt)
                f.flush()
                return f.readline().rstrip('\n')

        code.interact(readfunc=read, local=local)
    finally:
        sys.stdout, sys.stderr = streams
        serv.close()
        sock.close()


if __name__ == '__main__':
    remote_shell(local=locals())
    # or if it should run in background:
    # Thread(target=remote_shell, kwargs={'local': locals()}).start()

To connect, simply use: nc localhost 5555 - but it's probably too restricted for advanced use. With a tool like rlwrap at least it would be possible to have history, but the more advanced readline features like tab completion won't work. But at least it's something...