-
-
Notifications
You must be signed in to change notification settings - Fork 410
[Feat] Networks: private networking via WireGuard, custom, and provider-synced VPCs #1209
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
RichardAnderson
wants to merge
32
commits into
vitodeploy:4.x
Choose a base branch
from
RichardAnderson:feat/networks
base: 4.x
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+14,244
−80
Open
Changes from all commits
Commits
Show all changes
32 commits
Select commit
Hold shift + click to select a range
98cf83d
feat: wireguard & provider networking
RichardAnderson 53c6fd3
fixes
RichardAnderson c49a387
Merge remote-tracking branch 'origin/4.x' into feat/networks
RichardAnderson edd84e4
Merge remote-tracking branch 'origin/4.x' into feat/networks
RichardAnderson a1d20e8
fix: ordering of network servers
RichardAnderson d2bea42
Merge remote-tracking branch 'origin/4.x' into feat/networks
RichardAnderson 6ffc2a4
feat: server network firewall rules
RichardAnderson 25c473b
feat: peers work
RichardAnderson 2794080
feat: user interface and logs
RichardAnderson ff299c5
merge
RichardAnderson 6c26b18
feat: ui updates and various fixes
RichardAnderson 6cc0d5d
feat: provider private network sync
RichardAnderson 3e59592
feat: various fixes + docs
RichardAnderson 7d01090
fix: peer dialog persistant config
RichardAnderson b36ca70
fix: ensure tests pass + copilot fixes
RichardAnderson 22df400
fix: various fixes
RichardAnderson 65b627f
fix: styling
RichardAnderson 09b1bc9
fix: docblock update
RichardAnderson 5b4793d
fix: coderabbit findings
RichardAnderson 71f42cf
fix: review fixes
RichardAnderson c76b75a
fix: coderabbit findings
RichardAnderson ac1a724
fix: additional findings
RichardAnderson 13d3b12
fix: linting
RichardAnderson a9f5917
fix: out of change fixes, but worthwhile
RichardAnderson 2432c05
Merge branch '4.x' into pr/1209/feat/networks
saeedvaziry 611c8e1
feat: support ipv6 fully
RichardAnderson f866cd6
fix: fixes
RichardAnderson 3c20cae
fixes: code review
RichardAnderson c246ff9
fix: linting
RichardAnderson e1f8334
fix: code-review fixes round 2
RichardAnderson c018414
fix: refactors and reviews, updated unit tests
RichardAnderson edd807c
fix: linting
RichardAnderson File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,188 @@ | ||
| <?php | ||
|
|
||
| namespace App\Actions\Network; | ||
|
|
||
| use App\Enums\IpAddressType; | ||
| use App\Enums\NetworkServerStatus; | ||
| use App\Enums\NetworkType; | ||
| use App\Models\Network; | ||
| use App\Models\Project; | ||
| use App\Models\Server; | ||
| use App\ValidationRules\WithinCidrRule; | ||
| use Illuminate\Support\Facades\DB; | ||
| use Illuminate\Support\Facades\Validator; | ||
| use Illuminate\Validation\Rule; | ||
| use Illuminate\Validation\ValidationException; | ||
|
|
||
| class AddServersToNetwork | ||
| { | ||
| public function __construct( | ||
| private AllocateWireGuardPort $ports, | ||
| private CreateWireGuardMembers $members, | ||
| private DispatchNetworkServerSync $sync, | ||
| private RecomputeNetworkStatus $recompute, | ||
| private ApplyNetworkFirewall $firewall, | ||
| ) {} | ||
|
|
||
| /** | ||
| * @param array<string, mixed> $input | ||
| * @return ?int the port the network moved to, when an incoming server forced it off its own | ||
| */ | ||
| public function add(Network $network, array $input): ?int | ||
| { | ||
| if ($network->type === NetworkType::PROVIDER) { | ||
| throw ValidationException::withMessages([ | ||
| 'servers' => __('Members of a provider-managed network are synced from the provider.'), | ||
| ]); | ||
| } | ||
|
|
||
| $this->validate($network, $input); | ||
|
|
||
| $portBefore = $network->port; | ||
|
|
||
| $newMemberIds = DB::transaction(function () use ($network, $input): array { | ||
| return $network->type === NetworkType::WIREGUARD | ||
| ? $this->addWireGuard($network, $input) | ||
| : $this->addCustom($network, $input); | ||
| }); | ||
|
|
||
| if ($network->type === NetworkType::WIREGUARD) { | ||
| $network->load('servers.server'); | ||
| foreach ($network->servers as $member) { | ||
| if (in_array($member->id, $newMemberIds, true) | ||
| || in_array($member->status, [NetworkServerStatus::ACTIVE, NetworkServerStatus::UPDATING], true)) { | ||
| $this->sync->toPresent($member); | ||
| } | ||
| } | ||
| } else { | ||
| $this->firewall->handle($network); | ||
| } | ||
|
|
||
| $this->recompute->handle($network); | ||
|
|
||
| return $network->port !== $portBefore ? $network->port : null; | ||
| } | ||
|
|
||
| /** | ||
| * @param array<string, mixed> $input | ||
| * @return array<int, int> | ||
| */ | ||
| private function addWireGuard(Network $network, array $input): array | ||
| { | ||
| Project::query()->whereKey($network->project_id)->lockForUpdate()->first(); | ||
| Network::query()->whereKey($network->id)->lockForUpdate()->first(); | ||
|
|
||
| $this->resolvePortConflict($network, $input['servers']); | ||
|
|
||
| $used = $network->servers()->lockForUpdate()->pluck('ip') | ||
| ->concat($network->peers()->lockForUpdate()->pluck('ip')) | ||
| ->filter() | ||
| ->values() | ||
| ->all(); | ||
|
|
||
| $servers = Server::query() | ||
| ->where('project_id', $network->project_id) | ||
| ->whereIn('id', $input['servers']) | ||
| ->get(); | ||
|
|
||
| return $this->members->create($network, $servers, $used); | ||
| } | ||
|
|
||
| /** | ||
| * @param array<string, mixed> $input | ||
| * @return array<int, int> | ||
| */ | ||
| private function addCustom(Network $network, array $input): array | ||
| { | ||
| $ids = []; | ||
| foreach ($input['servers'] as $serverId) { | ||
| $member = $network->servers()->create([ | ||
| 'server_id' => $serverId, | ||
| 'server_ip_address_id' => $input['ip_addresses'][$serverId], | ||
| 'status' => NetworkServerStatus::ACTIVE, | ||
| ]); | ||
| $ids[] = $member->id; | ||
| } | ||
|
|
||
| return $ids; | ||
| } | ||
|
|
||
| /** | ||
| * @param array<string, mixed> $input | ||
| */ | ||
| private function validate(Network $network, array $input): void | ||
| { | ||
| $rules = [ | ||
| 'servers' => ['required', 'array', 'min:1'], | ||
| 'servers.*' => [ | ||
| 'integer', | ||
| 'distinct', | ||
| Rule::exists('servers', 'id')->where('project_id', $network->project_id), | ||
| Rule::unique('network_servers', 'server_id')->where('network_id', $network->id), | ||
| ], | ||
| ]; | ||
|
|
||
| if ($network->type === NetworkType::CUSTOM) { | ||
| $rules['ip_addresses'] = ['required', 'array']; | ||
| } | ||
|
|
||
| Validator::make($input, $rules)->validate(); | ||
|
|
||
| if ($network->type === NetworkType::CUSTOM) { | ||
| $this->validateMemberIps($network, $input); | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Runs only once `servers` is known to be a list of integers — building these rules from | ||
| * unvalidated input would interpolate an array into a rule key and fail with a 500. | ||
| * | ||
| * @param array<string, mixed> $input | ||
| */ | ||
| private function validateMemberIps(Network $network, array $input): void | ||
| { | ||
| $rules = []; | ||
|
|
||
| foreach ($input['servers'] as $serverId) { | ||
| $rules["ip_addresses.$serverId"] = [ | ||
| 'required', | ||
| Rule::exists('server_ip_addresses', 'id') | ||
| ->where('server_id', $serverId) | ||
| ->where('type', IpAddressType::PRIVATE->value), | ||
| Rule::unique('network_servers', 'server_ip_address_id'), | ||
| new WithinCidrRule($network->cidr), | ||
| ]; | ||
| } | ||
|
|
||
| Validator::make($input, $rules)->validate(); | ||
| } | ||
|
|
||
| /** | ||
| * An incoming server may already run this network's port for a different network, which the | ||
| * two would then fight over on that host. The network moves to a free port instead of | ||
| * refusing the server — every healthy member is resynced by the caller, so they follow it. | ||
| * | ||
| * Peers do not follow: their endpoint port is baked into the config at download time, so an | ||
| * already-imported config keeps the old port. The caller warns when peers exist. | ||
| * | ||
| * @param array<int, int> $serverIds | ||
| */ | ||
| private function resolvePortConflict(Network $network, array $serverIds): void | ||
| { | ||
| $serverIds = array_merge($network->servers()->pluck('server_id')->all(), $serverIds); | ||
|
|
||
| $port = $this->ports->allocate( | ||
| $network->project_id, | ||
| $serverIds, | ||
| $network->port ?? 51820, | ||
| $network->id, | ||
| ); | ||
|
|
||
| if ($port === $network->port) { | ||
| return; | ||
| } | ||
|
|
||
| $network->port = $port; | ||
| $network->save(); | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,134 @@ | ||
| <?php | ||
|
|
||
| namespace App\Actions\Network; | ||
|
|
||
| use App\Enums\NetworkAddressingPool; | ||
| use App\Models\Server; | ||
| use App\Models\ServerIpAddress; | ||
| use App\Support\Cidr; | ||
| use Illuminate\Support\Collection; | ||
| use Illuminate\Validation\ValidationException; | ||
|
|
||
| class AllocateNetworkBlock | ||
| { | ||
| /** | ||
| * Provider/Docker/AWS/Linode ranges to avoid on the RFC1918 opt-in path. | ||
| * The whole 172.16.0.0/12 (Docker/AWS default) is excluded by not listing | ||
| * it as an RFC1918 supernet below. | ||
| * | ||
| * @var array<int, string> | ||
| */ | ||
| private const BLOCKLIST = [ | ||
| '10.244.0.0/16', | ||
| '10.245.0.0/16', | ||
| '10.246.0.0/24', | ||
| '10.229.0.0/16', | ||
| '192.168.128.0/17', | ||
| ]; | ||
|
|
||
| /** | ||
| * @return array<int, string> | ||
| */ | ||
| private function supernets(NetworkAddressingPool $pool): array | ||
| { | ||
| return match ($pool) { | ||
| NetworkAddressingPool::CGNAT => ['100.64.0.0/10'], | ||
| NetworkAddressingPool::RFC1918 => ['10.0.0.0/8', '192.168.0.0/16'], | ||
| }; | ||
| } | ||
|
|
||
| /** | ||
| * Carve the next free canonical block from the pool, avoiding overlap with | ||
| * existing project networks and with any member server's observed subnets | ||
| * (of any type — a CGNAT-WAN interface is stored PUBLIC). | ||
| * | ||
| * @param Collection<int, string> $existingCidrs | ||
| * @param Collection<int, Server> $memberServers | ||
| */ | ||
| public function allocate( | ||
| NetworkAddressingPool $pool, | ||
| int $blockPrefix, | ||
| Collection $existingCidrs, | ||
| Collection $memberServers | ||
| ): string { | ||
| $existing = $existingCidrs->filter()->values()->all(); | ||
| $memberSubnets = $this->memberSubnets($memberServers); | ||
| $blocklist = $pool === NetworkAddressingPool::RFC1918 ? self::BLOCKLIST : []; | ||
|
|
||
| foreach ($this->supernets($pool) as $supernet) { | ||
| $candidate = $this->scan($supernet, $blockPrefix, $existing, $memberSubnets, $blocklist); | ||
| if ($candidate !== null) { | ||
| return $candidate; | ||
| } | ||
| } | ||
|
|
||
| throw ValidationException::withMessages([ | ||
| 'servers' => __('No free address block is available in the selected pool. Choose a smaller block size or the RFC1918 pool.'), | ||
| ]); | ||
| } | ||
|
|
||
| /** | ||
| * @param array<int, string> $existing | ||
| * @param array<int, string> $memberSubnets | ||
| * @param array<int, string> $blocklist | ||
| */ | ||
| private function scan( | ||
| string $supernet, | ||
| int $blockPrefix, | ||
| array $existing, | ||
| array $memberSubnets, | ||
| array $blocklist | ||
| ): ?string { | ||
| $supernetPrefix = Cidr::prefix($supernet); | ||
| if ($blockPrefix < $supernetPrefix) { | ||
| return null; | ||
| } | ||
|
|
||
| $base = Cidr::toLong(Cidr::network($supernet)); | ||
| $blockSize = Cidr::size($blockPrefix); | ||
| $count = 2 ** ($blockPrefix - $supernetPrefix); | ||
|
|
||
| for ($i = 0; $i < $count; $i++) { | ||
| $candidate = long2ip($base + ($i * $blockSize)).'/'.$blockPrefix; | ||
|
|
||
| if ($this->conflicts($candidate, $existing) || $this->conflicts($candidate, $blocklist) | ||
| || $this->conflicts($candidate, $memberSubnets)) { | ||
| continue; | ||
| } | ||
|
|
||
| return $candidate; | ||
| } | ||
|
|
||
| return null; | ||
| } | ||
|
|
||
| /** | ||
| * @param array<int, string> $others | ||
| */ | ||
| private function conflicts(string $candidate, array $others): bool | ||
| { | ||
| foreach ($others as $other) { | ||
| if (Cidr::overlaps($candidate, $other)) { | ||
| return true; | ||
| } | ||
| } | ||
|
|
||
| return false; | ||
| } | ||
|
|
||
| /** | ||
| * @param Collection<int, Server> $memberServers | ||
| * @return array<int, string> | ||
| */ | ||
| private function memberSubnets(Collection $memberServers): array | ||
| { | ||
| return ServerIpAddress::query() | ||
| ->whereIn('server_id', $memberServers->pluck('id')->all()) | ||
| ->get() | ||
| ->filter(fn (ServerIpAddress $address): bool => filter_var($address->ip, FILTER_VALIDATE_IP, FILTER_FLAG_IPV4) !== false) | ||
| ->map(fn (ServerIpAddress $address): string => Cidr::canonical($address->ip.'/'.$address->prefix_length)) | ||
| ->unique() | ||
| ->values() | ||
| ->all(); | ||
| } | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.