Cannot get STD handle to AllocConsole

Question

I've been trying to get the output handle to my console, but it doesn't seem to work. I got it to set the color of my text, but it's not changing.

HANDLE hOut = GetStdHandle(STD_OUTPUT_HANDLE);
SetConsoleTextAttribute(hOut, 0x0A)

I tried to debug it and I think I the handle isn't right. Is there any other way to do this, and is it normal that it doesn't work? Any fixes?

Thanks!

EDIT: Let me clarify, the handle that I'm trying to get is invalid. I've no idea how to fix it. I guess I'll start looking for alternatives; maybe something is wrong with my code.

Answer 1

The standard handlers are initialized during process creation, if you call AllocConsole the new console is created far later. AllocConsole can change the standard handles, but it's far too late for them to be used by startup code, such as the C runtime library initialization.

The best thing to do in this case is CreateFileW(L"CONOUT$", ...), which gets a console handle no matter whether you are attached to parent process's console, the OS created one for you because your PE header is /SUBSYSTEM:CONSOLE, or you called AllocConsole. And it gets the console handle even when standard handles are redirected.

And if you think you may call FreeConsole, you should be sure to close any handles returned by CreateFile first. In the general case where the console remains active until process exit, you can let the OS close the handle for you during process cleanup.

Answer 2

Since you specify hOut is INVALID_HANDLE_VALUE (or potentially NULL), try calling GetLastError to find out why. It is likely you don't have console session established.

Is this a win32 Console Application or is it a Windows SubSystem application (does it have WinMain?)

You could try AttachConsole(ATTACH_PARENT_PROCESS) instead of AllocConsole before GetStdHandle.

In either case, AllocConsole and AttachConsole return a BOOL which, if FALSE, indicates you can call GetLastError to find out why.

Make sure you aren't calling hOut = GetStdHandle(STD_OUTPUT_HANDLE) followed by CloseHandle(hOut) prior to the lines listed above. Unlike AllocConsole and FreeConsole, closing the std handle is not a good idea.

Finally, try:

#define _WIN32_WINNT 0x0501 before #include <windows.h>