Insufficient keys, especially for variant

[rowlesmr]

Hi again.

I've been play around with database construction from dictionaries, and have come across some issues, mainly with variant. I know that this is probably known, but just flagging it explicitly

Variant is used in many categories, but they don't have enough keys to properly refer to a particular variant, as variant is keyed on variant, diffrn_id, and entry_id.

The following are all category keys that are linked to _variant.variant, but can't find the right one, as we're missing diffrn_id and entry_id.

  !  FK: '_array_data.variant'                -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_array_element_size.variant'        -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_array_intensities.variant'         -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_array_structure.variant'           -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_array_structure_list.variant'      -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_array_structure_list_axis.variant' -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_axis.variant'                      -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_diffrn_data_frame.variant'         -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_diffrn_detector.variant'           -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_diffrn_detector_axis.variant'      -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_diffrn_detector_element.variant'   -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_diffrn_measurement.variant'        -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_diffrn_measurement_axis.variant'   -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_diffrn_radiation.variant'          -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_diffrn_refln.variant'              -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_diffrn_scan.variant'               -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_diffrn_scan_axis.variant'          -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_diffrn_scan_collection.variant'    -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_diffrn_scan_frame.variant'         -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_diffrn_scan_frame_axis.variant'    -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_diffrn_scan_frame_monitor.variant' -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_map.variant'                       -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']
  !  FK: '_map_segment.variant'               -> '_variant.variant': partial FK to 'variant' — covers ['variant'] of PKs=['variant', 'diffrn_id', 'entry_id']

additionally, _entry.id is used in MAP, and VARIANT, but is never defined?

There are other underspecified keys:

  !  FK: '_array_element_size.index'                   -> '_array_structure_list.index'     : partial FK to 'array_structure_list'         — covers ['index'] of PKs=['array_id', 'index', 'variant']
  !  FK: '_array_intensities.binary_id'                -> '_array_data.binary_id'           : partial FK to 'array_data'                   — covers ['binary_id'] of PKs=['array_id', 'binary_id', 'variant']
  !  FK: '_array_structure_list_axis.axis_id'          -> '_axis.id'                        : partial FK to 'axis'                         — covers ['id'] of PKs=['id', 'equipment', 'variant']
  !  FK: '_array_structure_list_section.index'         -> '_array_structure_list.index'     : partial FK to 'array_structure_list'         — covers ['index'] of PKs=['array_id', 'index', 'variant']
  !  FK: '_axis.depends_on'                            -> '_axis.id'                        : partial FK to 'axis'                         — covers ['id'] of PKs=['id', 'equipment', 'variant']
  !  FK: '_diffrn_data_frame.binary_id'                -> '_array_data.binary_id'           : partial FK to 'array_data'                   — covers ['binary_id'] of PKs=['array_id', 'binary_id', 'variant']
  !  FK: '_diffrn_data_frame.array_section_id'         -> '_array_structure_list_section.id': partial FK to 'array_structure_list_section' — covers ['id'] of PKs=['id', 'array_id', 'index', 'variant']
  !  FK: '_diffrn_data_frame.detector_element_id'      -> '_diffrn_detector_element.id'     : partial FK to 'diffrn_detector_element'      — covers ['id'] of PKs=['id', 'detector_id', 'variant']
  !  FK: '_diffrn_detector_axis.axis_id'               -> '_axis.id'                        : partial FK to 'axis'                         — covers ['id'] of PKs=['id', 'equipment', 'variant']
  !  FK: '_diffrn_detector_axis.detector_id'           -> '_diffrn_detector.id'             : partial FK to 'diffrn_detector'              — covers ['id'] of PKs=['diffrn_id', 'id', 'variant']
  !  FK: '_diffrn_measurement_axis.axis_id'            -> '_axis.id'                        : partial FK to 'axis'                         — covers ['id'] of PKs=['id', 'equipment', 'variant']
  !  FK: '_diffrn_measurement_axis.measurement_device' -> '_diffrn_measurement.device'      : partial FK to 'diffrn_measurement'           — covers ['device_class', 'id'] of PKs=['device_class', 'diffrn_id', 'id', 'variant']
  !  FK: '_diffrn_measurement_axis.measurement_id'     -> '_diffrn_measurement.id'          : partial FK to 'diffrn_measurement'           — covers ['device_class', 'id'] of PKs=['device_class', 'diffrn_id', 'id', 'variant']
  !  FK: '_diffrn_refln.frame_id'                      -> '_diffrn_data_frame.id'           : partial FK to 'diffrn_data_frame'            — covers ['id'] of PKs=['id', 'detector_element_id', 'variant']
  !  FK: '_diffrn_scan_axis.axis_id'                   -> '_axis.id'                        : partial FK to 'axis'                         — covers ['id'] of PKs=['id', 'equipment', 'variant']
  !  FK: '_diffrn_scan_frame.frame_id'                 -> '_diffrn_data_frame.id'           : partial FK to 'diffrn_data_frame'            — covers ['id'] of PKs=['id', 'detector_element_id', 'variant']
  !  FK: '_diffrn_scan_frame_axis.axis_id'             -> '_axis.id'                        : partial FK to 'axis'                         — covers ['id'] of PKs=['id', 'equipment', 'variant']
  !  FK: '_diffrn_scan_frame_axis.frame_id'            -> '_diffrn_data_frame.id'           : partial FK to 'diffrn_data_frame'            — covers ['id'] of PKs=['id', 'detector_element_id', 'variant']
  !  FK: '_diffrn_scan_frame_monitor.frame_id'         -> '_diffrn_data_frame.id'           : partial FK to 'diffrn_data_frame'            — covers ['id'] of PKs=['id', 'detector_element_id', 'variant']
  !  FK: '_diffrn_scan_frame_monitor.detector_id'      -> '_diffrn_detector.id'             : partial FK to 'diffrn_detector'              — covers ['id'] of PKs=['diffrn_id', 'id', 'variant']
  !  FK: '_map_segment.array_section_id'               -> '_array_structure_list_section.id': partial FK to 'array_structure_list_section' — covers ['id'] of PKs=['id', 'array_id', 'index', 'variant']
  !  FK: '_map_segment.id'                             -> '_map.id'                         : partial FK to 'map'                          — covers ['id'] of PKs=['id', 'diffrn_id', 'entry_id', 'variant']

and I don't know if this one matters: a part of the primary key of array_structure_list_axis, which is also a foriegn key to array_structure_list, doesn't point to the primary key of that category.

  !  FK: '_array_structure_list_axis.axis_set_id'      -> '_array_structure_list.axis_set_id': target column 'axis_set_id' is not a PK of 'array_structure_list' (PKs=['array_id', 'index', 'variant'])

[jamesrhester]

Ah yes, variant. variant was added to DDL2 imgCIF sometime after the first edition of Vol G, and there seems to be little record of consultation, although it's possible that all happened within the PDB. I've never seen variant used in the wild.

Clearly the function of variant is to index into an alternative description of the same underlying data. Unfortunately, as is clear from the above, by including _diffrn.id in the key, every single category (all of them) that uses variant has to include a pointer to _diffrn.id. This includes something like AXIS which then forces all axes to be restated for every _diffrn.id, even when the same equipment is used.

_entry.id is how mmCIF/PDBx creates "one per block" items, because _entry.id is restricted to a single value per PDB entry. The other thing DDL2 does is to list all child data names within the definition of the parent, so _diffrn.id and _entry.id have a long list of children (which do not include any of the imgCIF categories). In theory, every single PDB category should have a child pointer to _entry.id, as a PDB entry obviously includes everything, but in mmCIF/PDBx only categories that should have a single row are explicitly given child pointers. So the inclusion of _entry.id in the variant category seems to be saying that there can only be one variant, which doesn't seem logical.

Anyway, a very simple solution is to remove _diffrn.id and _entry.id from the keys of the category, and then _variant.variant can just be a UUID.

What I think would be a concise DDLm approach in addition to the above would be to decide that variant is a Set category, and therefore variants would require a separate data block. What's more, we'd create the equivalent of the multiblock dictionary for adding the variant category and variant.variant key data names and remove them from the normal DDLm imgCIF dictionary.

I will try and work out where the active imgCIF DDL2 repository is currently located, and raise an issue regarding dropping those data names from the key of variant.

[jamesrhester]

! FK: '_array_structure_list_axis.axis_set_id' -> '_array_structure_list.axis_set_id': target column 'axis_set_id' is not a PK of 'array_structure_list' (PKs=['array_id', 'index', 'variant'])

I think this is an error in imgCIF. The FK should point the other way, that is, _array_structure_list.axis_set_id -> _array_structure_list_axis.axis_set_id, because the logic is that an axis set is constructed from axes, and then referenced from array_structure_list to explain which axis set corresponds to which dimension of the array. That is, each dimension of the array should correspond to a single axis set, but an axis set may not correspond to any dimension of the array (e.g. is defined just in case). So the functional arrow can only go in the direction _array_structure_list.axis_set_id to _array_structure_list_axis.axis_set_id.

@yayahjb , @phyy-nx, any thoughts?

[jamesrhester]

I think this is an error in imgCIF. The FK should point the other way

Upon consideration, I think this is more subtle and not an error in imgCIF. array_structure_list_axis (hereafter asla) defines the axes that collaborate in producing the axis direction labelled by asla.axis_set_id, and then that axis_set_id value is indeed assigned to an array direction in array_structure_list. However, asla.axis_set_id is not the key of asla, as it is repeated when there are multiple axes for one axis set. So where is the category for which axis_set_id is the sole key data name? Only array_structure_list is available. We can't invert the keys in this category so that array_structure_list.axis_set_id is the sole key, because this same axis direction could be reused in a different array (or at least that appears to be the intention). So it is like there is an invisible category just listing the axis_set_ids that both asl.axis_set_id and asla.axis_set_id are children of.

So I conclude that this, at least, is not something we need to fix.