How can I convert a char into an int within the range of 0 and 255 in C++

Question

I am trying to make my own file compressor and some of the chars in the file that I am trying to compress are '�'.

I tried:

#include <iostream>



int main(){
    std :: cout << (int)'�';
    return 0;
}

But it returned an error saying

test.cpp:6:25: error: character too large for enclosing character literal type
    6 |     std :: cout << (int)'�';
      |                         ^
1 error generated.

And even if I go straight in the file with fstream and try to convert this char into a number, I get a negative number but if I use unsigned int, I get a number way higher than the range of 0 and 255

Something I Tried to do is

#include <iostream>
#include <sstream>
#include <fstream>

using namespace std;

vector<int> something;

stringstream ssom;

int main(){
    fstream file12("Compresser");
    ssom << file12.rdbuf();
    string buffer = ssom.str();
    for (char i : buffer){
        something.push_back((int)i);
        
    }
    cout << something.size() << "\n";
    cout << something[0];
    return 0;
}

I printed the size of it and the int value associated with the '�' char and it outputted 50696 and -54 where 50696 is the size and -54 is the number associated with '�' but I am trying to keep it in the range of 0 and 255 so I can use this is python and most other programming languages as bytes.

Answer 1

When you’re writing a file compressor, you shouldn’t worry about text encodings at all. Text encodings like UTF-8 or ASCII are just ways to interpret sequences of bytes as readable characters for humans, but files themselves are always represented and stored as raw bytes, where you can think of a byte as an integer type of a specific size, 8-bits. You only need to care about encodings if you actually want to display or process the file as text; for compression, you just operate directly on the bytes, no matter what they “mean.”

A character (like 'A', 'é', or '�') is an abstract concept that can take up one or more bytes in a particular encoding. But a byte is always just a value between 0 and 255. If you read a file in C++ using char, you might see negative numbers for bytes with values above 127, because char is usually signed. This can also cause display issues or weird symbols if you try to print those bytes as characters, especially if your terminal expects UTF-8 or ASCII.

For a file compressor, just open your file in binary mode and read the bytes into a std::vector<uint8_t> (or unsigned char). That way, every element will always be a value between 0 and 255, with no risk of negative numbers or accidental character conversions. For example:

#include <fstream>
#include <vector>
#include <cstdint>

int main() {
    std::ifstream file("Compresser", std::ios::binary);
    std::vector<uint8_t> data(
        std::istreambuf_iterator<char>(file),
        std::istreambuf_iterator<char>());
    // Now you can compress data however you like, and each value is a true byte.
}

You only run into issues like seeing '�' or negative numbers if you try to treat the file as text or use a signed type. For compression, you can ignore encodings completely and work with unsigned 8-bit integers.

Answer 2

Plain ASCII is actually seven bits. "Extended" ASCII variants are eight bits. Its implementation defined if a char is signed or unsigned. And multi-byte encodings (like e.g. UTF-8) can't fit in an eight-bit type. If you want to "compress" bytes, don't compress "characters", compress a stream of unknown bytes instead. And if you want to compress raw data, don't open files in text-mode, that will mess things up on Windows. Also, I recommend you use a vector of std::uint8_t elements instead of a string. With std::uint8_t you won't have your negative value problem to begin with. The lesson to be learned: If you want to only deal with unsigned values, use unsigned types. Credits to: @Someprogrammerdude for answering this question in the comment section!