Check if downloading sample data is stable

[bouweandela]

Description

Checklist

Please confirm that this pull request has done the following:

• Tests added
• Documentation added (where applicable)
• Changelog item added to changelog/

[bouweandela]

@lewisjared When I run the download script on my laptop the NetCDF files are shown as changed including the hashes in registry.txt (my hashes look the same as what is shown here in CI). However, I couldn't find any difference when comparing the NetCDF files using ncdump. Do you have any idea what is causing this?

[lewisjared]

I wonder if there are subtle difference depending on which data node the files are coming from... I'm assuming my closest data node will be different to you. I'll try deleting my existing data and see if that resolves things.

[lewisjared]

I'm also on a mac which might play a role in this. I wonder if we need to make the CI the source of truth for the outputs.

My local versions of netCDF

import netCDF4
print(f"netcdf: {netCDF4.__version__}")
print(f"hdf5: {netCDF4.__hdf5libversion__}")
print(f"libnetcdf: {netCDF4.__netcdf4libversion__}")

netcdf: 1.7.2
hdf5: 1.14.4
libnetcdf: 4.9.2

[lewisjared]

/regenerate

[lewisjared]

/regenerate

[lewisjared]

/regenerate

[lewisjared]

/regenerate

[lewisjared]

/regenerate

[lewisjared]

/regenerate

[lewisjared]

/regenerate

[lewisjared]

/regenerate

[github-actions]

PR comment handling
Something went wrong!

Details: https://github.com/CMIP-REF/ref-sample-data/actions/runs/12879272828

[lewisjared]

/regenerate

[github-actions]

PR comment handling
The regenerate task is done!

You can find the workflow here:
https://github.com/CMIP-REF/ref-sample-data/actions/runs/12879629529

[lewisjared]

@bouweandela Sorry for the spam, but I was testing the regeneration functionality on a forked branch. I'll close this and make a clean PR

[lewisjared]

Closing in preference for #8

[bouweandela]

I appear to have slightly different versions of the software:

11:42 $ git log | head
commit ab99667c9f38bd08885d332464b846679d76ee84
Author: Bouwe Andela <b.andela@esciencecenter.nl>
Date:   Thu Jan 16 20:57:59 2025 +0100

    Check if downloading sample data is stable

commit a28ef6978922a77e9d7de13194dfc3c3ba21cca7
Author: lewisjared <ci-runner@cmip-ref.invalid>
Date:   Fri Jan 10 03:27:20 2025 +0000

(.venv) ✔ ~/src/cmip-ref/ref-sample-data [test-fetch ↓·14|⚑ 1] 
11:42 $ uv sync
Resolved 95 packages in 0.63ms
Audited 89 packages in 0.02ms
(.venv) ✔ ~/src/cmip-ref/ref-sample-data [test-fetch ↓·14|⚑ 1] 
11:44 $ python 
Python 3.12.8 (main, Dec  6 2024, 19:59:28) [Clang 18.1.8 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import netCDF4
>>> print(f"netcdf: {netCDF4.__version__}")
netcdf: 1.7.2
>>> print(f"hdf5: {netCDF4.__hdf5libversion__}")
hdf5: 1.14.2
>>> print(f"libnetcdf: {netCDF4.__netcdf4libversion__}")
libnetcdf: 4.9.4-development
>>> ^D

Did I do the environment synchronization correctly? Maybe it is indeed because I'm Linux and you're on Mac.

[lewisjared]

Yes. The python dependencies are the same. I'm not sure where the netCDF and hdf libraries are bundled in from. Your values match ther results from the CI which is a good sign.