Skip to content

[Fixes #14155] Fixes parameter order for mdict to set precedence from settings.PYCSW to pycsw_local defaults#14161

Closed
cmotadev wants to merge 54 commits intoGeoNode:masterfrom
nds-cprm:5.0.x
Closed

[Fixes #14155] Fixes parameter order for mdict to set precedence from settings.PYCSW to pycsw_local defaults#14161
cmotadev wants to merge 54 commits intoGeoNode:masterfrom
nds-cprm:5.0.x

Conversation

@cmotadev
Copy link
Copy Markdown
Contributor

Now it's possible to overwrite PyCSW parameterrs from settings and view mods in pyCSW GetCapabilites

Checklist

Reviewing is a process done by project maintainers, mostly on a volunteer basis. We try to keep the overhead as small as possible and appreciate if you help us to do so by completing the following items. Feel free to ask in a comment if you have troubles with any of them.

For all pull requests:

  • [ x] Confirm you have read the contribution guidelines
  • [ x ] You have sent a Contribution Licence Agreement (CLA) as necessary (not required for small changes, e.g., fixing typos in the documentation)
  • [ x ] Make sure the first PR targets the master branch, eventual backports will be managed later. This can be ignored if the PR is fixing an issue that only happens in a specific branch, but not in newer ones.

The following are required only for core and extension modules (they are welcomed, but not required, for contrib modules):

  • [ x ] There is a ticket in https://github.com/GeoNode/geonode/issues describing the issue/improvement/feature (a notable exemption is, changes not visible to end-users)
  • The issue connected to the PR must have Labels and Milestone assigned
  • [ x ] PR for bug fixes and small new features are presented as a single commit
  • [ x ] PR title must be in the form "[Fixes #<issue_number>] Title of the PR"
  • New unit tests have been added covering the changes, unless there is an explanation on why the tests are not necessary/implemented

Submitting the PR does not require you to check all items, but by the time it gets merged, they should be either satisfied or inapplicable.

giohappy and others added 30 commits November 14, 2025 18:09
(cherry picked from commit a492be2)
(cherry picked from commit 1ddae09)
(cherry picked from commit 281e984)
(cherry picked from commit 36654b9)

Co-authored-by: Mattia <mattia.giupponi@gmail.com>
* [Fixes #13719] Upsert with authority different that 4326 raise error
* [Fixes #13719] fix field creation on dynamic model for upsert/replace
* [Fixes #13719] Cannot upsert a dataset after replace
* [Fixes #13719] Make all geom to be promoted to multi

(cherry picked from commit e817494)
…13736) (#13746)

(cherry picked from commit f395969)

Co-authored-by: Sijan Dhungana <sumoseason35@gmail.com>
* Some cleanup to settings
---------



(cherry picked from commit 5726abc)

Co-authored-by: Giovanni Allegri <giohappy@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Mattia <mattia.giupponi@gmail.com>
* Adding missing init.py file in metadata

(cherry picked from commit 214969c)

Co-authored-by: mattiagiupponi <51856725+mattiagiupponi@users.noreply.github.com>
* build(deps): bump django from 5.2.7 to 5.2.8

Bumps [django](https://github.com/django/django) from 5.2.7 to 5.2.8.
- [Commits](django/django@5.2.7...5.2.8)

---
updated-dependencies:
- dependency-name: django
  dependency-version: 5.2.8
  dependency-type: direct:production
...



* align setup.cfg

---------




(cherry picked from commit 04d440e)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mattiagiupponi <51856725+mattiagiupponi@users.noreply.github.com>
* feat: Add project-specific fixture loading configuration

* fix: code linting

* Simplify error message formatting for fixture loading

---------


(cherry picked from commit 6953eb9)

Co-authored-by: Niraj Adhikari <41701707+nrjadkry@users.noreply.github.com>
Co-authored-by: Giovanni Allegri <giohappy@gmail.com>
)

* adding Beat in a separate process

* avoid restarting

* defining a restart policy to the celery container

* avoid PID conflicts which make Celery container to enter in a restart loop

* removing starting policy from the Celery service itself

(cherry picked from commit 5e1f441)

Co-authored-by: George Petrakis <gkpetrak@gmail.com>
…moves user_messages (#13772)

* Fixes #13565: add app metadata in B/R procedure

* Remove 'upload' from fixtures apps and dumps

(cherry picked from commit 140fa43)

Co-authored-by: mattiagiupponi <51856725+mattiagiupponi@users.noreply.github.com>
(cherry picked from commit 6385c7a)

Co-authored-by: allyoucanmap <stefano.bovio@geosolutionsgroup.com>
…13803) (#13804)

* [Fixes #13802] sanitize the layer name during cloning operation

* [Fixes #13802] sanitize the name also in raster during cloning

(cherry picked from commit f12421f)

Co-authored-by: Sijan Dhungana <sumoseason35@gmail.com>
(cherry picked from commit cc94699)

Co-authored-by: G.Allegri <giohappy@gmail.com>
(cherry picked from commit baa10c0)

Co-authored-by: Giovanni Allegri <giohappy@gmail.com>
* [Fixes #1381] Improve regenerate_xml

* [Fixes #1381] Improve regenerate_xml - improvs

(cherry picked from commit bde9c9d)

Co-authored-by: Emanuele Tajariol <etj@geo-solutions.it>
(cherry picked from commit 105ea13)
* Update Django version to 5.2.10

* Update Django version to 5.2.10
* Upgrade to Geoserver 2.27.4

* revert change to Dockerfile
) (#13941)

Ensure GeoJSON files containing non-ASCII (e.g. Chinese) attribute values
are correctly detected and validated by explicitly decoding input as UTF-8
in GeoJsonFileHandler.

This prevents false negatives in handler detection and avoids incorrectly
reporting valid GeoJSON files as invalid due to implicit or system-dependent
encoding assumptions.

(cherry picked from commit d93a30d)

Co-authored-by: F0rtiter <40894236+F0rtiter@users.noreply.github.com>
Gpetrak and others added 24 commits February 17, 2026 19:22
… the remote WMS service (#13967) (#13972)

* handling the title mismatch with GeoNode and the remote WMS service

(cherry picked from commit b94cb9a)

Co-authored-by: George Petrakis <gkpetrak@gmail.com>
(cherry picked from commit 249762f)

Co-authored-by: George Petrakis <gkpetrak@gmail.com>
… normal (#14003) (#14022)

* [Fixes #13996] Permissions cache reset when changing from Readonly to normal

(cherry picked from commit 4324ae1)

Co-authored-by: Sijan Dhungana <sumoseason35@gmail.com>
* [Fixes #14019] Fixes clone of CSV dataset

(cherry picked from commit cc33602)

Co-authored-by: Sijan Dhungana <sumoseason35@gmail.com>
…udes column names with special characters or spaces (#13993) (#14024)

* [Fixes #13992] Handle non-spatial data when an uploaded CSV file includes column names with special characters or spaces

(cherry picked from commit f0a9bba)

Co-authored-by: Sijan Dhungana <sumoseason35@gmail.com>
* Update Django version to 5.2.12
…isible resources

* [Fixes #13994] fix fecet count coherent with the visible resources

* [Fixes #13994] simplified the logic

* [Fixes #13994] merge 5.0.x and update a fix
#13991) (#14056)

* [Fixes #13990] Basic auth is not fully propagated into the WMS Service

* Update geonode/services/serviceprocessors/wms.py



* Update geonode/services/views.py



* Update geonode/services/views.py



* [Fixes #13990] Basic auth is not fully propagated into the WMS Service

* [Fixes #13990] Basic auth is not fully propagated into the WMS Service

---------


(cherry picked from commit ca8e960)

Co-authored-by: mattiagiupponi <51856725+mattiagiupponi@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
#14064)

(cherry picked from commit 712ffd9)

Co-authored-by: Mattia Giupponi <mattia.giupponi@gmail.com>
…pload (#14058) (#14065)

* disable thumbnail generation in the remote documents upload

(cherry picked from commit 4a852cf)

Co-authored-by: George Petrakis <gkpetrak@gmail.com>
(cherry picked from commit 9b91659)

Co-authored-by: mattiagiupponi <51856725+mattiagiupponi@users.noreply.github.com>
…NABLE_DYN_MODELS=False (#14084) (#14085)

* [Fixes #14083] fixes Cloning error for older datasets with IMPORTER_ENABLE_DYN_MODELS=False

* [Fixes #14083] updated review suggestion

(cherry picked from commit fae6656)

Co-authored-by: Sijan Dhungana <sumoseason35@gmail.com>
…String (#14110) (#14114)

(cherry picked from commit e7a2e66)

Co-authored-by: Sijan Dhungana <sumoseason35@gmail.com>
* Add resourcehandlerinfo creation management command

(cherry picked from commit 2deaccd)

Co-authored-by: mattiagiupponi <51856725+mattiagiupponi@users.noreply.github.com>
… settings.PYCSW to pycsw_local defaults. Now it's possible to overwrite PyCSW parameterrs from settings
@cla-bot
Copy link
Copy Markdown

cla-bot Bot commented Apr 23, 2026

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have the users @github-actions[bot] on file. In order for us to review and merge your code, please contact the project maintainers to get yourself added.

@cmotadev cmotadev changed the title Fixes parameter order for mdict to set precedence from settings.PYCSW to pycsw_local defaults [Fixes #14155] Fixes parameter order for mdict to set precedence from settings.PYCSW to pycsw_local defaults Apr 23, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces extensive updates to the GeoNode ecosystem, including dependency upgrades for Django, Pillow, and urllib3, alongside updating Geoserver images to version 2.27.4. Significant security enhancements were made to the upload process by transitioning ogr2ogr command execution to a list-based Popen implementation with proper shell quoting. The PR also refactors settings management, improves permissions cache invalidation, and adds a new AdvertisedFilter for resource visibility. Review feedback identifies a critical dictionary iteration bug in the vector handler, suggests using BigIntegerField for Integer64 to avoid precision loss, recommends simplifying the advertised filter logic, and advises replacing a deprecated Pillow method in EXIF metadata extraction.

feature_to_save = []
for feature in data_chunk:
feature_as_dict = feature.items()
feature_as_dict = {self.fixup_name(key): value for key, value in feature.items().items()}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

feature.items() returns a dictionary. Calling .items() on it again is incorrect and will raise an AttributeError. You should iterate directly over the dictionary returned by feature.items().

Suggested change
feature_as_dict = {self.fixup_name(key): value for key, value in feature.items().items()}
feature_as_dict = {self.fixup_name(key): value for key, value in feature.items()}


STANDARD_TYPE_MAPPING = {
"Integer64": "django.db.models.IntegerField",
"Integer64": "django.db.models.FloatField",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Mapping Integer64 to FloatField can lead to loss of precision for large integer values. It would be more appropriate to map it to django.db.models.BigIntegerField to ensure large integers are stored correctly without precision loss.

Suggested change
"Integer64": "django.db.models.FloatField",
"Integer64": "django.db.models.BigIntegerField",

Comment on lines +149 to +161
except Exception:
advertised = None

if advertised == "all":
pass
elif advertised is not None:
queryset = queryset.filter(advertised=advertised)
else:
is_admin = user.is_superuser if user and user.is_authenticated else False
if not is_admin and user and not user.is_anonymous:
queryset = (queryset.filter(advertised=True) | queryset.filter(owner=user)).distinct()
elif not user or user.is_anonymous:
queryset = queryset.filter(advertised=True)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The logic for filtering based on user type can be simplified for better readability. Also, it's better to catch a more specific ValueError from strtobool instead of a generic Exception.

Suggested change
except Exception:
advertised = None
if advertised == "all":
pass
elif advertised is not None:
queryset = queryset.filter(advertised=advertised)
else:
is_admin = user.is_superuser if user and user.is_authenticated else False
if not is_admin and user and not user.is_anonymous:
queryset = (queryset.filter(advertised=True) | queryset.filter(owner=user)).distinct()
elif not user or user.is_anonymous:
queryset = queryset.filter(advertised=True)
except ValueError:
advertised = None
if advertised == "all":
pass
elif advertised is not None:
queryset = queryset.filter(advertised=advertised)
else:
if user.is_superuser:
# Superusers can see all resources, so no additional filter is applied.
pass
elif user.is_authenticated:
queryset = (queryset.filter(advertised=True) | queryset.filter(owner=user)).distinct()
else: # Anonymous user
queryset = queryset.filter(advertised=True)


img = Image.open(doc.doc_file.path)
img = Image.open(file_path)
exif_data = {ExifTags.TAGS[k]: v for k, v in img._getexif().items() if k in ExifTags.TAGS}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The _getexif() method is considered internal and is deprecated in Pillow. It's better to use the public getexif() method, which is available in recent versions of Pillow and returns an Exif object that behaves like a dictionary.

Suggested change
exif_data = {ExifTags.TAGS[k]: v for k, v in img._getexif().items() if k in ExifTags.TAGS}
exif_data = {ExifTags.TAGS[k]: v for k, v in img.getexif().items() if k in ExifTags.TAGS}

@cmotadev cmotadev closed this Apr 23, 2026
@cmotadev
Copy link
Copy Markdown
Contributor Author

My apologies, I will merge to master first

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants