Equalise cubes

[pp-mo]

Closes #6248

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.83%. Comparing base (0fdedb4) to head (1265c26).
Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6257      +/-   ##
==========================================
- Coverage   89.83%   89.83%   -0.01%     
==========================================
  Files          88       88              
  Lines       23315    23380      +65     
  Branches     4338     4356      +18     
==========================================
+ Hits        20945    21003      +58     
- Misses       1644     1649       +5     
- Partials      726      728       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pp-mo]

Status update

I think this is now good enough to consider as it is, though I'm still anticipating other useful features could be added.
It's worth considering what's included here, and why, and also what is anticipated to be added in future

"Grouping" implementation

As noted here in the original issue - section "Grouping of input cubes ?",
it was realised that in order to usefully apply "equalisation" operations like equalise_attributes to all cubes in a file load
(as we eventually hope to - see here, in section "Embedding in extended "combine_cubes" operation" ),
- they must be applied over "groups" of input cubes, not all cubes from the whole file.

The notes there describe the problem of trying to rationalise the different ways in which merge and concatenate do this "grouping"
.. however .. the current implementation drastically simplifies this, by grouping based on cube.metadata only.
I think this will make "adequate" distinction between the input cube groups over which "equalisation" operations are applied;
and obviously ..

• it works the same for both merge + concatenate
• it is simple enough to clearly + fully document

Currently included options :

• unify_time_units because
  • it already exists + has proved useful
• equalise_attributes because ...
  • it already exists + has proved useful
  • it has a particular relation to cube metadata, and interaction with input 'grouping'
    --both to affect it + to be affected by it-- so this is an opportunity to sort out how that needs to function
• unify_names because ...
  • this operation may be needed to enable netcdf data to be concatenated,
    which is more likely to be wanted now we've made that available on loading
  • it has a particular relation to cube metadata, and interaction with input 'grouping',
    so it's worth resolving + documenting how that works in the initial version of the function

Other possible, future options

See also list in original issue

• unify selected (compatible) units, e.g.
  • unify_compatible_units=['m', 'Pa']
• make approximately-equal coordinates equal
• remove aux-coords, cell-methods, cell-measures etc, e.g.
  • remove_ancils=True
  • remove_cell_measures="a_cell"
• apply 'new-axis' to ensure promote certain scalar coords to dims, with selected additional components (see Code solutions for time-dependent hybrid height #6165 for conceptual background)
  • make_axis="time"
  • make_axis={"time": "surface_altitude"}
• remove_coord_bounds

We can also, in future, usefully include this in the "combine_cubes" operation and "COMBINE_POLICY" / "LOAD_POLICY" settings.
I anticipate that we can add a single "equalisation_kwargs" keyword which is set to a dictionary arg. This enables a single 'equalisation phase' to occur just once before the merge/concat operation.

[stephenworsley]

Looking good, just a couple things, mostly about documentation. This will also want a whatsnew describing the new function.

[stephenworsley]

It looks like the default behaviour, when all kwargs are False, does nothing to the cubes. Would it be helpful to raise a warning in the case where all of these are False? I can imagine a user having an expectation that this might do something without having to alter any of the kwargs.

[stephenworsley]

This should probably be renamed cube_grouping_dicts. This is a list of dictionarys rather than a list of keys.

[stephenworsley]

As noted above, the name cube_group_key is misleading as I believe this object should be a dictionary rather than a key.

[stephenworsley]

I think this can be edited a little so that it doesn't suggest that the cubes might be modified in place in a different order. Something like "..., and are possibly returned in a different order".

[stephenworsley]

Is unify_names the correct name for this operation? Compared to unify_time_units, this operation works quite differently. Only removing information rather than editing it. unify_names might suggest that equivalent standard_names are made the same for example. This may also cause us problems if we ever wanted to add that sort of functionality in the future. A better name might be standardise_names or trim_names.

[stephenworsley]

Could [0:] not be replaced by [:]?