Jupyter notebook show and capture output at the same time

Question

I'm trying to figure out a way to "save" the variables in a jupyter notebook but also capture the rich output at the same time. Mainly for reasons of ssh disconnecting, etc. Just assigning to variables doens't work since some of the libraries print information like progress bars.

So far the most promising way is to use %%capture however seems in recent versions that just captures the output and doesn't output while the browser session is active. Ideally I'm looking for a solution that does both, if the browser session is active print out the output, but if for whatever reason it gets interrupted, I can dump the captured output later.

Answer 1

We can create a custom magic with register_cell_magic, based on capture code from IPython, I could do this function that simulates Linux's tee command (hence the name). The only thing I did is extract it from the class and calling the io thing at the end.

from IPython import get_ipython
from IPython.core import magic_arguments
from IPython.core.magic import register_cell_magic
from IPython.utils.capture import capture_output

@magic_arguments.magic_arguments()
@magic_arguments.argument('output', type=str, default='', nargs='?',
    help="""The name of the variable in which to store output.
    This is a utils.io.CapturedIO object with stdout/err attributes
    for the text of the captured output.
    CapturedOutput also has a show() method for displaying the output,
    and __call__ as well, so you can use that to quickly display the
    output.
    If unspecified, captured output is discarded.
    """
)
@magic_arguments.argument('--no-stderr', action="store_true",
    help="""Don't capture stderr."""
)
@magic_arguments.argument('--no-stdout', action="store_true",
    help="""Don't capture stdout."""
)
@magic_arguments.argument('--no-display', action="store_true",
    help="""Don't capture IPython's rich display."""
)
@register_cell_magic
def tee(line, cell):
    args = magic_arguments.parse_argstring(tee, line)
    out = not args.no_stdout
    err = not args.no_stderr
    disp = not args.no_display
    with capture_output(out, err, disp) as io:
        get_ipython().run_cell(cell)
    if args.output:
        get_ipython().user_ns[args.output] = io
    
    io()

As you can see in the following image, It is working correctly