Loading enum succeeds even if number of variants doesn't match saved type

[watsaig]

I'm not sure if this is an issue or expected behavior, but I'm surprised that this doesn't cause an error.

If I have a dataset with an enum type, and attempt to read it as a different enum, reading succeeds but results in incorrect values.

Concretely:

use hdf5_metno::{File, H5Type};
use ndarray::Ix1;

#[derive(H5Type, Debug)]
#[repr(u8)]
#[allow(dead_code)]
enum First {
    A = 0,
    B = 1,
    C = 2,
}

#[derive(H5Type, Debug)]
#[repr(u8)]
enum Second {
    A = 0,
    B = 1,
    C = 2,
    D = 3,
    E = 4,
}

fn main() -> anyhow::Result<()> {
    let file = File::create("test.h5")?;
    let to_write = [Second::A, Second::B, Second::C, Second::D, Second::E];
    let dataset = file.new_dataset::<Second>().shape([5]).create("second")?;
    dataset.write(&to_write)?;
    file.close()?;

    let file = File::open("test.h5")?;
    let dataset = file.dataset("second")?;
    let data_in_as_second = dataset.read::<Second, Ix1>()?;
    println!("data read as `Second`: {:?}", data_in_as_second);
    let data_in_as_first = dataset.read::<First, Ix1>()?;
    println!("data read as `First`: {:?}", data_in_as_first);

    Ok(())
}

Outputs:

data read as `Second`: [A, B, C, D, E], shape=[5], strides=[1], layout=CFcf (0xf), const ndim=1
data read as `First`: [A, B, C, A, A], shape=[5], strides=[1], layout=CFcf (0xf), const ndim=1

Cargo.toml

[package]
name = "hdf5_enums"
version = "0.1.0"
edition = "2024"

[dependencies]
anyhow = "1.0.100"
hdf5-metno = "0.11.0"
ndarray = "0.17.1"

Should read return an error in this kind of situation?

[mulimoen]

There does indeed seem to be an issue in the type conversion where we disregard the enum names or a mismatch in enum values. There is some logic in hdf5 that maps invalid/unknown enum values to zero, as seen if we modify your example a little bit:

use hdf5_metno::{File, H5Type};
use ndarray::Ix1;

#[derive(H5Type, Debug)]
#[repr(u8)]
#[allow(dead_code)]
enum First {
    UNDEF = 0,
    A = 255,
    B = 1,
    C = 2,
}

#[derive(H5Type, Debug)]
#[repr(u8)]
#[allow(dead_code)]
enum Second {
    UNDEF = 0,
    A = 255,
    B = 1,
    C = 2,
    D = 3,
    E = 4,
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let file = File::create("test.h5")?;
    let to_write = [Second::A, Second::B, Second::C, Second::D, Second::E];
    let dataset = file.new_dataset::<Second>().shape([5]).create("second")?;
    dataset.write(&to_write)?;
    file.close()?;

    let file = File::open("test.h5")?;
    let dataset = file.dataset("second")?;
    let data_in_as_second = dataset.read::<Second, Ix1>()?;
    println!("data read as `Second`: {:?}", data_in_as_second.as_slice().unwrap());
    let data_in_as_first = dataset.read::<First, Ix1>()?;
    println!("data read as `First`: {:?}", data_in_as_first.as_slice().unwrap());
    file.close()?;

    Ok(())
}

which prints

data read as `Second`: [A, B, C, D, E]
data read as `First`: [A, B, C, UNDEF, UNDEF]

I see two ways of solving this:

1. Require enum types to be equal
2. Require value 0 to be mapped to an enum value

[watsaig]

Thanks for the extra investigation. To me, the first option feels more intuitive because it avoids any unexpected value conversion. But for some it may be overly strict. For example, one might add some enum variants to some reading code and still want to support older files which were saved with fewer variants (which is more or less what I did to encounter this behavior).
Maybe requiring the enum variants in the file to be a strict subset of the enum variants in code would work?

If it's any help I'm happy to PR this, just let me know what approach you prefer.