Potential Bug Accessing Masked Data

[nmiles2718]

I'm not sure if this is the intended behavior or not, but it seems like it could be a bug. When accessing data from a netCDF variable, there are two methods that give different answers depending on whether or not the data is masked at the given index.

• If the data is masked, variable[i].data != variable[:].data[i]
  1. variable[i].data = 0, holds true even if fill value is set
  2. variable[:].data[i] = FillValue

I think both methods should return the same value for consistency and performance. If a user wants to ensure the fill value is always returned when accessing masked data, they will have to take performance hit because the the second method requires reading the entire masked array, then index it for the desired value.

MWE

import netCDF4 as nc
import numpy as np
print(f'numpy version: {np.__version__}')
print(f'netCDF4 version: {nc.__version__}')
data = np.ones(10)
data[1::2] = -9999
dset = nc.Dataset('test.nc', 'w')
dim = dset.createDimension(
    'dim',
    size=data.shape[0]
)
variable = dset.createVariable(
    'test',
    'f4',
    dimensions=('dim',),
    fill_value=-9999
)
variable[:] = data
print(f'Variable: {variable}\n{"-"*30}')
print(f'Variable data: {variable[:]}')
for i in range(data.shape[0]):
    print(
        f'idx {i:0d}\n'
        f'  variable[i].data={variable[i].data}\n'
        f'  variable[:].data[i]={variable[:].data[i]}\n'
    )

Output

Python 3.13.0 | packaged by conda-forge | (main, Oct  8 2024, 20:04:32) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import netCDF4 as nc
... import numpy as np
... print(f'numpy version: {np.__version__}')
... print(f'netCDF4 version: {nc.__version__}')
... data = np.ones(10)
... data[1::2] = -9999
... dset = nc.Dataset('test.nc', 'w')
... dim = dset.createDimension(
...     'dim',
...     size=data.shape[0]
... )
... variable = dset.createVariable(
...     'test',
...     'f4',
...     dimensions=('dim',),
...     fill_value=-9999
... )
... variable[:] = data
... print(f'Variable: {variable}\n{"-"*30}')
... print(f'Variable data: {variable[:]}')
... for i in range(data.shape[0]):
...     print(
...         f'idx {i:0d}\n'
...         f'  variable[i].data={variable[i].data}\n'
...         f'  variable[:].data[i]={variable[:].data[i]}\n'
...     )
...
numpy version: 2.1.3
netCDF4 version: 1.7.4
Variable: <class 'netCDF4.Variable'>
float32 test(dim)
    _FillValue: -9999.0
unlimited dimensions:
current shape = (10,)
filling on
------------------------------
Variable data: [1.0 -- 1.0 -- 1.0 -- 1.0 -- 1.0 --]
idx 0
  variable[i].data=1.0
  variable[:].data[i]=1.0

idx 1
  variable[i].data=0.0
  variable[:].data[i]=-9999.0

idx 2
  variable[i].data=1.0
  variable[:].data[i]=1.0

idx 3
  variable[i].data=0.0
  variable[:].data[i]=-9999.0

idx 4
  variable[i].data=1.0
  variable[:].data[i]=1.0

idx 5
  variable[i].data=0.0
  variable[:].data[i]=-9999.0

idx 6
  variable[i].data=1.0
  variable[:].data[i]=1.0

idx 7
  variable[i].data=0.0
  variable[:].data[i]=-9999.0

idx 8
  variable[i].data=1.0
  variable[:].data[i]=1.0

idx 9
  variable[i].data=0.0
  variable[:].data[i]=-9999.0