Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
114 changes: 76 additions & 38 deletions base_import_match/README.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
.. image:: https://odoo-community.org/readme-banner-image
:target: https://odoo-community.org/get-involved?utm_source=readme
:alt: Odoo Community Association

=================
Base Import Match
=================
Expand All @@ -17,7 +13,7 @@ Base Import Match
.. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png
:target: https://odoo-community.org/page/development-status
:alt: Beta
.. |badge2| image:: https://img.shields.io/badge/license-AGPL--3-blue.png
.. |badge2| image:: https://img.shields.io/badge/licence-AGPL--3-blue.png
:target: http://www.gnu.org/licenses/agpl-3.0-standalone.html
:alt: License: AGPL-3
.. |badge3| image:: https://img.shields.io/badge/github-OCA%2Fserver--backend-lightgray.png?logo=github
Expand All @@ -35,8 +31,8 @@ Base Import Match
By default, when importing data (like CSV import) with the
``base_import`` module, Odoo follows this rule:

- If you import the XMLID of a record, make an **update**.
- If you do not, **create** a new record.
- If you import the XMLID of a record, make an **update**.
- If you do not, **create** a new record.

This module allows you to set additional rules to match if a given
import is an update or a new record.
Expand All @@ -45,41 +41,65 @@ This is useful when you need to sync heterogeneous databases, and the
field you use to match records in those databases with Odoo's is not the
XMLID but the name, VAT, email, etc.

After installing this module, the import logic will be changed to:
After installing this module, the import preview UI adds a **Match**
column. Checking the box on a field means:

- Use it to **find existing records** (match key).
- **Do not write** its value during the actual import.

This lets you update records by matching on fields like name, email or
VAT without needing an XMLID in your file.

Pre-configured **Import Match** rules act as templates: their fields are
pre-checked in the import preview. You can toggle any field on or off
before importing, regardless of whether a rule exists.

When match-only fields are selected in the UI, matching uses all
selected fields together to search for a single existing record. If any
row fails to match exactly one record (zero or multiple matches), the
entire import is blocked and errors are shown for the affected rows.
This prevents accidental creation of duplicate records.

- If you import the XMLID of a record, make an **update**.
- If you do not:
Note that conditional rules (fields with a required imported value) are
only applied during programmatic imports; the UI-driven matching uses
the selected fields unconditionally.

- If there are import match rules for the model you are importing:
For programmatic imports (without UI), the configured rules are used
instead. The import logic in that case is:

- Discard the rules that require fields you are not importing.
- Traverse the remaining rules one by one in order to find a match
in the database.
- If you import the XMLID of a record, make an **update**.
- If you do not:

- Skip the rule if it requires a special condition that is not
satisfied.
- If one match is found:
- If there are import match rules for the model you are importing:

- Stop traversing the rest of valid rules.
- **Update** that record.
- Discard the rules that require fields you are not importing.
- Traverse the remaining rules one by one in order to find a
match in the database.

- If zero or multiple matches are found:
- Skip the rule if it requires a special condition that is not
satisfied.
- If one match is found:

- Continue with the next rule.
- Stop traversing the rest of valid rules.
- **Update** that record.

- If all rules are exhausted and no single match is found:
- If zero or multiple matches are found:

- **Create** a new record.
- Continue with the next rule.

- If there are no match rules for your model:
- If all rules are exhausted and no single match is found:

- **Create** a new record.
- **Create** a new record.

- If there are no match rules for your model:

- **Create** a new record.

By default 2 rules are installed for production instances:

- One rule that will allow you to update companies based on their VAT,
when ``is_company`` is ``True``.
- One rule that will allow you to update users based on their login.
- One rule that will allow you to update companies based on their VAT,
when ``is_company`` is ``True``.
- One rule that will allow you to update users based on their login.

In demo instances there are more examples.

Expand All @@ -91,7 +111,8 @@ In demo instances there are more examples.
Configuration
=============

To configure this module, you need to:
Import Match rules are optional templates that pre-check the **Match**
column in the import preview. To configure them:

1. Go to *Settings > Technical > Database Structure > Import Match*.
2. *Create*.
Expand All @@ -114,15 +135,28 @@ Usage

To use this module, you need to:

1. Follow steps in **Configuration** section above.
2. Go to any list view.
3. Press *Import* and follow the import procedure as usual.
1. Go to any list or kanban view.
2. Go to *Action > Import records* and upload your file.
3. In the import preview, check **Match** on the fields you want to use
as matching keys (e.g. name, email, VAT). These fields will be used
to find existing records but will not be written.
4. Proceed with the import as usual. If any row cannot be matched to
exactly one existing record, the import will be blocked and error
messages will indicate which rows had zero or multiple matches.

If Import Match rules are configured for the model, their fields will be
pre-checked automatically.

Known issues / Roadmap
======================

- Add a setting to throw an error when multiple matches are found,
instead of falling back to creation of new record.
- Add a setting to throw an error when multiple matches are found
during programmatic imports, instead of falling back to creation of
new record. (UI-driven imports already block on zero or multiple
matches.)
- Support matching on child record fields (one2many subfields) in the
import preview. Currently, matching only works for direct fields of
the imported model.

Bug Tracker
===========
Expand All @@ -145,11 +179,15 @@ Authors
Contributors
------------

- `Tecnativa <https://www.tecnativa.com>`__:
- `Tecnativa <https://www.tecnativa.com>`__:

- Jairo Llopis
- Vicent Cubells
- Ernesto Tejeda

- `Quartile <https://www.quartile.co>`__:

- Jairo Llopis
- Vicent Cubells
- Ernesto Tejeda
- Yoshi Tashiro

Maintainers
-----------
Expand Down
6 changes: 6 additions & 0 deletions base_import_match/__manifest__.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,10 @@
"views/base_import_match_view.xml",
],
"demo": ["demo/base_import_match.xml"],
"assets": {
"web.assets_backend": [
"base_import_match/static/src/**/*.js",
"base_import_match/static/src/**/*.xml",
],
},
}
75 changes: 72 additions & 3 deletions base_import_match/models/base.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,56 @@
# Copyright 2017 Jairo Llopis <jairo.llopis@tecnativa.com>
# Copyright 2026 Quartile (https://www.quartile.co)
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
from odoo import api, models
from odoo import _, api, models


class Base(models.AbstractModel):
_inherit = "base"

@api.model
def _match_by_fields(self, match_fields, converted_row, imported_row):
"""Find existing records matching on the given fields.

Return a ``(recordset, count)`` tuple. When exactly one record matches,
*recordset* is that record; otherwise it is an empty record.
"""
domain = []
for fname in match_fields & converted_row.keys():
value = converted_row[fname]
# For x2many fields
if isinstance(value, list) and value and isinstance(value[0], tuple):
for ref in imported_row.get(fname, "").split(","):
ref = ref.strip()
if ref:
domain.append((fname, "=", ref))
else:
domain.append((fname, "=", value))
if not domain:
return self, 0
match = self.search(domain, limit=2)
if len(match) == 1:
return match, 1
return self, len(match)

@api.model
def _match_error(self, match_fields, row, record_index, count):
"""Build an import error dict for a failed match attempt."""
criteria = ", ".join(f"{f}={row.get(f, '')}" for f in sorted(match_fields))
if count == 0:
msg = _("No matching record found for: %(criteria)s", criteria=criteria)
else:
msg = _(
"Multiple matching records found (expected 1) for: %(criteria)s",
criteria=criteria,
)
return {
"type": "error",
"message": msg,
"rows": {"from": record_index, "to": record_index},
"record": record_index,
"field": False,
}

@api.model
def load(self, fields, data):
"""Try to identify rows by other pseudo-unique keys.
Expand All @@ -15,9 +60,16 @@ def load(self, fields, data):
XMLID in place, Odoo will understand that it must *update* the
record instead of *creating* a new one.
"""
# We only need to patch this call if there are usable rules for it
if self.env["base_import.match"]._usable_rules(self._name, fields):
# UI-selected match fields prevail; configured rules are only used when the
# context key is absent (e.g. programmatic imports).
ctx_match_only = self.env.context.get("import_match_only_fields")
match_only_fields = set(ctx_match_only or []) & set(fields)
has_rules = ctx_match_only is None and bool(
self.env["base_import.match"]._usable_rules(self._name, fields)
)
if match_only_fields or has_rules:
newdata = list()
match_errors = []
# Change .id (dbid) by id (xmlid)
if ".id" in fields:
column = fields.index(".id")
Expand Down Expand Up @@ -48,6 +100,15 @@ def load(self, fields, data):
elif dbid:
# Find the xmlid for this dbid
match = self.browse(dbid)
elif match_only_fields:
# Match using user-selected fields from the UI
match, count = self._match_by_fields(match_only_fields, record, row)
if count != 1:
match_errors.append(
self._match_error(
match_only_fields, row, info["record"], count
)
)
else:
# Store records that match a combination
match = self.env["base_import.match"]._match_find(self, record, row)
Expand All @@ -58,7 +119,15 @@ def load(self, fields, data):
row["id"] = ext_id[match.id] if match else row.get("id", "")
# Store the modified row, in the same order as fields
newdata.append(tuple(row[f] for f in clean_fields))
if match_errors:
return {"ids": False, "messages": match_errors, "nextrow": False}
# We will import the patched data to get updates on matches
data = newdata
# Rebuild fields/data without match-only columns.
if match_only_fields:
drop_set = {fields.index(f) for f in match_only_fields}
keep_indexes = [i for i in range(len(fields)) if i not in drop_set]
fields[:] = [fields[i] for i in keep_indexes]
data = [tuple(row[i] for i in keep_indexes) for row in data]
# Normal method handles the rest of the job
return super().load(fields, data)
20 changes: 20 additions & 0 deletions base_import_match/models/base_import.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# Copyright 2016 Grupo ESOC Ingeniería de Servicios, S.L.U. - Jairo Llopis
# Copyright 2016 Tecnativa - Vicent Cubells
# Copyright 2026 Quartile (https://www.quartile.co)
# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl.html).
import logging

Expand Down Expand Up @@ -177,3 +178,22 @@ def _compute_display_name(self):
def _onchange_match_id_name(self):
"""Update match name."""
self.mapped("match_id")._compute_name()


class BaseImportImport(models.TransientModel):
_inherit = "base_import.import"

def parse_preview(self, options, count=10):
res = super().parse_preview(options, count=count)
if not res.get("error"):
rules = self.env["base_import.match"].search(
[("model_name", "=", self.res_model)]
)
res["match_fields"] = list(set(rules.mapped("field_ids.name")))
return res

def execute_import(self, fields, columns, options, dryrun=False):
match_only = options.pop("import_match_only_fields", None)
if match_only is not None:
self = self.with_context(import_match_only_fields=match_only)
return super().execute_import(fields, columns, options, dryrun=dryrun)
3 changes: 2 additions & 1 deletion base_import_match/readme/CONFIGURE.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
To configure this module, you need to:
Import Match rules are optional templates that pre-check the **Match** column in the
import preview. To configure them:

1. Go to *Settings \> Technical \> Database Structure \> Import Match*.
2. *Create*.
Expand Down
2 changes: 2 additions & 0 deletions base_import_match/readme/CONTRIBUTORS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,5 @@
- Jairo Llopis
- Vicent Cubells
- Ernesto Tejeda
- [Quartile](https://www.quartile.co):
- Yoshi Tashiro
25 changes: 24 additions & 1 deletion base_import_match/readme/DESCRIPTION.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,30 @@ This is useful when you need to sync heterogeneous databases, and the
field you use to match records in those databases with Odoo's is not the
XMLID but the name, VAT, email, etc.

After installing this module, the import logic will be changed to:
After installing this module, the import preview UI adds a **Match** column. Checking
the box on a field means:

- Use it to **find existing records** (match key).
- **Do not write** its value during the actual import.

This lets you update records by matching on fields like name, email or VAT without
needing an XMLID in your file.

Pre-configured **Import Match** rules act as templates: their fields are pre-checked in
the import preview. You can toggle any field on or off before importing, regardless of
whether a rule exists.

When match-only fields are selected in the UI, matching uses all selected fields
together to search for a single existing record. If any row fails to match exactly one
record (zero or multiple matches), the entire import is blocked and errors are shown for
the affected rows. This prevents accidental creation of duplicate records.

Note that conditional rules (fields with a required imported value) are only applied
during programmatic imports; the UI-driven matching uses the selected fields
unconditionally.

For programmatic imports (without UI), the configured rules are used instead. The import
logic in that case is:

- If you import the XMLID of a record, make an **update**.
- If you do not:
Expand Down
7 changes: 5 additions & 2 deletions base_import_match/readme/ROADMAP.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,5 @@
- Add a setting to throw an error when multiple matches are found,
instead of falling back to creation of new record.
- Add a setting to throw an error when multiple matches are found during
programmatic imports, instead of falling back to creation of new record.
(UI-driven imports already block on zero or multiple matches.)
- Support matching on child record fields (one2many subfields) in the import preview.
Currently, matching only works for direct fields of the imported model.
14 changes: 11 additions & 3 deletions base_import_match/readme/USAGE.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
To use this module, you need to:

1. Follow steps in **Configuration** section above.
2. Go to any list view.
3. Press *Import* and follow the import procedure as usual.
1. Go to any list or kanban view.
2. Go to *Action \> Import records* and upload your file.
3. In the import preview, check **Match** on the fields you want to use as matching
keys (e.g. name, email, VAT). These fields will be used to find existing records
but will not be written.
4. Proceed with the import as usual. If any row cannot be matched to exactly one
existing record, the import will be blocked and error messages will indicate
which rows had zero or multiple matches.

If Import Match rules are configured for the model, their fields will be pre-checked
automatically.
Loading