does polars support columns selection using boolean type?

Question

I want to select those columns whose all rows value is null. In pandas, I can use:

nan_columns = specific_data.columns[specific_data.isna().all()]

but I found its logic may be a little hard for polars. I found it:

 nan_columns = specific_data[:, specific_data.select(pl.all().is_null().all()).row(0)].columns[0]

Am I wrong? Or is there better solution for it?

Answer 1

Note. Please take a look at this polars discussion page to better understand the intricacies of data-dependant column selection and implications on lazy/eager evaluation.

Lets consider a dataframe defined as

import polars as pl

df = pl.DataFrame({
    "a": [1, 2, 3],
    "b": [None, None, None],
    "c": [True, False, True],
})

which looks as follows

shape: (3, 3)
┌─────┬──────┬───────┐
│ a   ┆ b    ┆ c     │
│ --- ┆ ---  ┆ ---   │
│ i64 ┆ f32  ┆ bool  │
╞═════╪══════╪═══════╡
│ 1   ┆ null ┆ true  │
│ 2   ┆ null ┆ false │
│ 3   ┆ null ┆ true  │
└─────┴──────┴───────┘

You could now use

df.select(col for col in df if col.is_null().all())

or

df.select(col.name for col in df.select(pl.all().is_null().all()) if col.item())

The latter will ensure the inner conditions are computed in parallel.