Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@
/config.toml
/pidfile
/tests_output
config-suite.toml
29 changes: 29 additions & 0 deletions config-production-firefox.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,32 @@ sentry_dsn = ""

[clones]
directory = "/clones"

###########
# FIREFOX #
###########

[[tracked_repositories]]
name = "firefox"
url = "https://github.com/mozilla-firefox/firefox.git_NOT_YET_IN_USE"


#
# MOZILLA-UNIFIED
#
# We don't sync to this repository, but we put it here first to fetch all
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we could build that into a separate config like refs_fetch_source instead of shoe-horning it into branch_mappings?

Copy link
Copy Markdown
Member Author

@shtrom shtrom Mar 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good idea! I'll file a bug.

https://bugzilla.mozilla.org/show_bug.cgi?id=2022092

Though, thinking more about it, maybe we don't need it so much, now that we have bundles setup for those repos, too.

# references early, with the benefit of bundles.
#
[[branch_mappings]]
source_url = "https://github.com/mozilla-firefox/firefox.git_NOT_YET_IN_USE"
branch_pattern = "THIS_SHOULD_MATCH_NOTHING"
destination_url = "https://hg.mozilla.org/mozilla-unified/"
destination_branch = "NOT_A_VALID_BRANCH"

[[tag_mappings]]
source_url = "https://github.com/mozilla-firefox/firefox.git_NOT_YET_IN_USE"
tag_pattern = "THIS_SHOULD_MATCH_NOTHING"
destination_url = "https://hg.mozilla.org/mozilla-unified/"
tags_destination_branch = "NOT_A_VALID_BRANCH"
# Default
#tag_message_suffix = "a=tagging CLOSED TREE DONTBUILD"
78 changes: 78 additions & 0 deletions config-production-thunderbird.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
[pulse]
# Those parameters can be overriden by PULSE_* environment variables
host = "pulse.mozilla.org"
port = 5671
ssl = true
userid = "SET THIS IN PULSE_USERID ENV VARIABLE"
password = "SET THIS IN PULSE_PASSWORD ENV VARIABLE"
# The exchange is declared by the Producer.
exchange = "exchange/landoprod/pushes"
routing_key = "gitpushes"
# The Consumer declares the queue and binds it to the exchange.
heartbeat = 30
queue = "queue/githgsyncprod/pushes"

[sentry]
sentry_dsn = ""

[clones]
directory = "/clones"

#############################
# THUNDERBIRD-INFRA-TESTING #
#############################

[[tracked_repositories]]
name = "thunderbird-infra-testing"
url = "https://github.com/thunderbird/infra-testing.git"

#
# COMM-UNIFIED
#
# We don't sync to this repository, but we put it here first to fetch all
# references early, with the benefit of bundles.
#
[[branch_mappings]]
source_url = "https://github.com/thunderbird/infra-testing.git"
branch_pattern = "THIS_SHOULD_MATCH_NOTHING"
destination_url = "https://hg.mozilla.org/comm-unified/"
destination_branch = "NOT_A_VALID_BRANCH"

[[branch_mappings]]
source_url = "https://github.com/thunderbird/infra-testing.git"
branch_pattern = "main"
destination_url = "ssh://hg.mozilla.org/conduit-testing/comm-infra-testing/"
destination_branch = "default"

[[tag_mappings]]
source_url = "https://github.com/thunderbird/infra-testing.git"

# comm-central:
# tag_pattern = ".+"
# destination_url = "ssh://hg.mozilla.org/comm-central"
# tags_destination_branch = "tags-testing"

# comm-beta:
# tag_pattern = "(^THUNDERBIRD|NIGHTLY|BETA|RELEASE)_.*$"
# destination_url = "ssh://hg.mozilla.org/releases/comm-beta/"
# tags_destination_branch = "tags-unified"

# comm-release:
# tag_pattern = "(^THUNDERBIRD|NIGHTLY|BETA|RELEASE)_.*$"
# destination_url = "ssh://hg.mozilla.org/releases/comm-release/"
# tags_destination_branch = "tags-unified"

# comm-esrXXX
# tag_pattern = "^(THUNDERBIRD)_(\\d+)(_\\d+)+esr_(BUILD\\d+|RELEASE)$"
# destination_url = "ssh://hg.mozilla.org/releases/comm-esr\\2/"
Copy link
Copy Markdown

@coreycb coreycb Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should there be a tag_pattern for comm-central?

Do the tag_patterns only include tags from the corresponding git branch?
I think the following would cover tags, assuming only tags from the corresponding git branches are included:

https://hg-edge.mozilla.org/comm-central/tags

(NIGHTLY)_(\d+)_(BASE|END)
(BETA)_(\d+)_(BASE|END)

https://hg-edge.mozilla.org/releases/comm-beta/tags

(NIGHTLY)_(\d+)_(BASE|END)
(BETA)_(\d+)_(BASE|END)
(RELEASE)_(\d+)_(BASE|END)
^(THUNDERBIRD)_(\\d+)_0b(\\d+)_(BUILD\\d+|RELEASE)$  # beta tags

https://hg-edge.mozilla.org/releases/comm-release/tags

(NIGHTLY)_(\d+)_(BASE|END)
(BETA)_(\d+)_(BASE|END)
(RELEASE)_(\d+)_(BASE|END)
^(THUNDERBIRD)_(\\d+)_0b(\\d+)_(BUILD\\d+|RELEASE)$  # beta tags
^(THUNDERBIRD)_(\\d+)(_\\d+)+_(BUILD\\d+|RELEASE)$  # release tags

https://hg-edge.mozilla.org/releases/comm-esr140/tags

(NIGHTLY)_(\d+)_(BASE|END)
(BETA)_(\d+)_(BASE|END)
(RELEASE)_(\d+)_(BASE|END)
(ESR)_(\d+)_(BASE|END)
^(THUNDERBIRD)_(\\d+)_0b(\\d+)_(BUILD\\d+|RELEASE)$  # beta tags
^(THUNDERBIRD)_(\\d+)(_\\d+)+esr_(BUILD\\d+|RELEASE)$  # esr tags

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should there be a tag_pattern for comm-central?

It will get .+ to get all tags.

Do the tag_patterns only include tags from the corresponding git branch?

The logic is less smart^W^Wmore flexible: we need to explicitly list all the tag patterns that we want to go to a given repo. This is regardless of whether the commit is on a matching branch for that repo. In case it isn't (because we didn't write this config file correctly), Hg will ignore the tag, and it will simply not be created on the target repo.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@coreycb Thanks for the updated mapping. I've pushed an updated config (note: it's only comments for now, as it doesn't apply to infra-testing, but it's good to capture and discuss this early nonetheless).

I have put in simpler (less specific) regexps, but I think they should achieve the same. Two things to note:

  • most repos have some ^(NIGHTLY|BETA|RELEASE) tags, but they are inherited from merges or forks (e.g, esr140 doesn't have the most recent ones)
  • I can't include them all for the ESR, as we use the tags to determine the target repo. For Firefox, only the esr versions are used, though I might had to add support for ESR_ here, too. I'm not completely sure how well this will work.

-> Is the ESR_nnn_BASE tag used in any downstream automation?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-> Is the ESR_nnn_BASE tag used in any downstream automation?

It doesn't look like it. @dandarnell can you check too?

# tags_destination_branch = "tags-unified"
# ...
# tag_pattern = "^ESR_(\\d+)_BASE$"
# destination_url = "ssh://hg.mozilla.org/releases/comm-esr\\1/"
# tags_destination_branch = "tags-unified"

tag_pattern = ".+"
destination_url = "ssh://hg.mozilla.org/conduit-testing/comm-infra-testing/"
tags_destination_branch = "tags-testing"
# Default
#tag_message_suffix = "a=tagging CLOSED TREE DONTBUILD"
Empty file removed config-suite.toml
Empty file.
25 changes: 12 additions & 13 deletions tests/test_config_files.py
Original file line number Diff line number Diff line change
@@ -1,27 +1,26 @@
from pathlib import Path

import pytest
from pydantic import ValidationError

from git_hg_sync.config import Config

HERE = Path(__file__).parent
BASEDIR = Path(__file__).parent.parent


@pytest.mark.parametrize(
"config_file",
[
"config-development.toml",
"config-docker.toml",
"config-production.toml",
"config-staging.toml",
],
list(BASEDIR.glob("config-*.toml"))
)
def test_config_files(config_file: str) -> None:
config = Config.from_file(HERE / ".." / config_file)
def test_config_files(config_file: Path) -> None:
try:
config = Config.from_file(config_file)
except ValidationError as exc:
raise AssertionError(f"Syntax in {config_file}") from exc

# We just do a shallow verification. What we really care is that the file could be
# loaded correctly.
assert config.pulse
assert config.tracked_repositories
assert config.branch_mappings
assert config.tag_mappings
assert config.pulse, f"`pulse` section missing in {config_file}"
assert config.tracked_repositories, f"`tracked_repositories` section missing in {config_file}"
assert config.branch_mappings, f"`branch_mappings` section missing in {config_file}"
assert config.tag_mappings, f"`tag_mappings` section missing in {config_file}"