When to create header file only classes

Question

I'm learning c++ (not my choice) and faced some classes with no .cpp file. I was told that classes should be implemented in 2 files : header file (.h) and source file (.cpp) but it seems it's not always like this. My question is : When and why it is preferred to implement class methods in header file (.h) and when it's not?

Answer 1

Implementations of templated classes practically have to go into headers (though see alternatives described in the link below).
Non-templated classes are indeed recommended to be split as you describe.

See Why can templates only be implemented in the header file?

Some libraries (header-only libs) consist only of headers and that is a special design decision. They have pros and cons, see https://en.wikipedia.org/wiki/Header-only
I think that is probably not what you are asking about, but the contribution (in a comment on this answer, by user super) is worth mentioning.