Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .Jules/palette.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,7 @@
## 2024-05-24 - Fail Fast & Friendly
**Learning:** In CLI tools involving APIs, cascade failures (hundreds of "Failed to X") caused by basic auth issues (401/403) are overwhelming and confusing. A dedicated "Pre-flight Check" that validates credentials *before* attempting the main workload allows for specific, actionable error messages (e.g. "Check your token at [URL]") instead of generic HTTP errors.
**Action:** Implement a `check_api_access()` step at the start of any CLI workflow to validate permissions and provide human-readable guidance on failure.

## 2024-05-25 - Interactive Input Validation
**Learning:** In CLI interactive modes, validating input immediately (looping until valid) prevents user frustration from waiting for a script to start only to crash later. It turns a "Run -> Error -> Restart" cycle into a "Prompt -> Typo -> Correction -> Success" flow.
**Action:** Always implement `get_validated_input` loops for critical configuration prompts in interactive CLI sessions.
64 changes: 52 additions & 12 deletions main.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,10 @@
import concurrent.futures
import ipaddress
import socket
import getpass
from functools import lru_cache
from urllib.parse import urlparse
from typing import Dict, List, Optional, Any, Set, Sequence
from typing import Dict, List, Optional, Any, Set, Sequence, Callable

import httpx
from dotenv import load_dotenv
Expand Down Expand Up @@ -140,6 +141,25 @@
return v


def get_validated_input(
prompt: str,

Check warning

Code scanning / Pylint (reported by Codacy)

Wrong hanging indentation before block (add 4 spaces). Warning

Wrong hanging indentation before block (add 4 spaces).
validator: Callable[[str], bool],

Check warning

Code scanning / Pylint (reported by Codacy)

Wrong hanging indentation before block (add 4 spaces). Warning

Wrong hanging indentation before block (add 4 spaces).
error_msg: str,

Check warning

Code scanning / Pylint (reported by Codacy)

Wrong hanging indentation before block (add 4 spaces). Warning

Wrong hanging indentation before block (add 4 spaces).
is_password: bool = False

Check warning

Code scanning / Pylint (reported by Codacy)

Wrong hanging indentation before block (add 4 spaces). Warning

Wrong hanging indentation before block (add 4 spaces).
) -> str:
"""Prompts for input until the validator returns True."""
while True:
if is_password:
value = getpass.getpass(prompt).strip()
else:
value = input(prompt).strip()

Check notice

Code scanning / Bandit (reported by Codacy)

The input method in Python 2 will read from standard input, evaluate and run the resulting string as python source code. This is similar, though in many ways worse, then using eval. On Python 2, use raw_input instead, input is safe in Python 3. Note

The input method in Python 2 will read from standard input, evaluate and run the resulting string as python source code. This is similar, though in many ways worse, then using eval. On Python 2, use raw_input instead, input is safe in Python 3.

if validator(value):
return value

print(f"{Colors.FAIL}❌ {error_msg}{Colors.ENDC}")

Comment on lines +144 to +161
Copy link

Copilot AI Jan 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new get_validated_input function and is_valid_profile_id_format function lack test coverage. While the test file (test_main.py) includes tests for other functions like check_api_access, push_rules, and interactive prompts, there are no tests for these new validation functions. Since the codebase has comprehensive test coverage for similar functions, these new functions should also be tested to ensure they work correctly, especially for edge cases like empty input, whitespace-only input, invalid characters, and length boundaries.

Copilot uses AI. Check for mistakes.

TOKEN = _clean_env_kv(os.getenv("TOKEN"), "TOKEN")

# Default folder sources
Expand Down Expand Up @@ -189,7 +209,6 @@
)

_gh = httpx.Client(timeout=30)
MAX_RESPONSE_SIZE = 10 * 1024 * 1024 # 10 MB limit for external resources

# --------------------------------------------------------------------------- #
# 3. Helpers
Expand Down Expand Up @@ -228,7 +247,7 @@
try:
# Resolve hostname to IPs (IPv4 and IPv6)
# We filter for AF_INET/AF_INET6 to ensure we get IP addresses
addr_info = socket.getaddrinfo(hostname, None, proto=socket.IPPROTO_TCP)

Check warning

Code scanning / Pylint (reported by Codacy)

Missing function docstring Warning

Missing function docstring

Check warning

Code scanning / Pylintpython3 (reported by Codacy)

Missing function or method docstring Warning

Missing function or method docstring
for res in addr_info:
# res is (family, type, proto, canonname, sockaddr)
# sockaddr is (address, port) for AF_INET/AF_INET6
Expand All @@ -247,12 +266,19 @@

return True

def validate_profile_id(profile_id: str) -> bool:
def is_valid_profile_id_format(profile_id: str) -> bool:

Check warning

Code scanning / Pylint (reported by Codacy)

Missing function docstring Warning

Missing function docstring
if not re.match(r"^[a-zA-Z0-9_-]+$", profile_id):
log.error("Invalid profile ID format (contains unsafe characters)")
return False
if len(profile_id) > 64:
log.error("Invalid profile ID length (max 64 chars)")
return False
return True
Comment on lines +269 to +274
Copy link

Copilot AI Jan 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new is_valid_profile_id_format function should also lack test coverage. While the test file (test_main.py) includes tests for other functions like check_api_access, push_rules, and interactive prompts, there are no tests for this new validation function. Since the codebase has comprehensive test coverage for similar functions, this function should also be tested to ensure it works correctly, especially for edge cases like empty input, whitespace-only input, invalid characters, and length boundaries.

Copilot uses AI. Check for mistakes.

def validate_profile_id(profile_id: str) -> bool:
if not is_valid_profile_id_format(profile_id):
if not re.match(r"^[a-zA-Z0-9_-]+$", profile_id):
log.error("Invalid profile ID format (contains unsafe characters)")
elif len(profile_id) > 64:
log.error("Invalid profile ID length (max 64 chars)")
return False
return True
Comment on lines +276 to 283
Copy link

Copilot AI Jan 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The validate_profile_id function contains redundant validation logic. After calling is_valid_profile_id_format (line 277), the function re-checks the same conditions (regex match on line 278 and length check on line 280) that were already validated inside is_valid_profile_id_format. This duplicates the validation logic and makes the code harder to maintain. Consider simplifying this function to only call is_valid_profile_id_format and provide appropriate error messages based on which specific check failed, or refactor is_valid_profile_id_format to return more detailed information about why validation failed.

Copilot uses AI. Check for mistakes.

Expand Down Expand Up @@ -376,9 +402,9 @@
log.critical(f"{Colors.FAIL}❌ Authentication Failed: The API Token is invalid.{Colors.ENDC}")
log.critical(f"{Colors.FAIL} Please check your token at: https://controld.com/account/manage-account{Colors.ENDC}")
elif code == 403:
log.critical(f"{Colors.FAIL}🚫 Access Denied: Token lacks permission for Profile {profile_id}.{Colors.ENDC}")

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

This expression logs
sensitive data (password)
as clear text.
elif code == 404:
log.critical(f"{Colors.FAIL}🔍 Profile Not Found: The ID '{profile_id}' does not exist.{Colors.ENDC}")

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

This expression logs
sensitive data (password)
as clear text.
log.critical(f"{Colors.FAIL} Please verify the Profile ID from your Control D Dashboard URL.{Colors.ENDC}")
else:
log.error(f"API Access Check Failed ({code}): {e}")
Expand Down Expand Up @@ -854,7 +880,7 @@
return success_count == len(folder_data_list)

except Exception as e:
log.error(f"Unexpected error during sync for profile {profile_id}: {sanitize_for_log(e)}")

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

This expression logs
sensitive data (password)
as clear text.
return False

# --------------------------------------------------------------------------- #
Expand All @@ -881,17 +907,31 @@
if not profile_ids:
print(f"{Colors.CYAN}ℹ Profile ID is missing.{Colors.ENDC}")
print(f"{Colors.CYAN} You can find this in the URL of your profile in the Control D Dashboard.{Colors.ENDC}")
p_input = input(f"{Colors.BOLD}Enter Control D Profile ID:{Colors.ENDC} ").strip()
if p_input:
profile_ids = [p.strip() for p in p_input.split(",") if p.strip()]

def validate_profile_input(value: str) -> bool:

Check warning

Code scanning / Pylint (reported by Codacy)

Missing function docstring Warning

Missing function docstring
if not value:
return False
ids = [p.strip() for p in value.split(",") if p.strip()]
return bool(ids) and all(is_valid_profile_id_format(pid) for pid in ids)

p_input = get_validated_input(
f"{Colors.BOLD}Enter Control D Profile ID:{Colors.ENDC} ",
validate_profile_input,
"Invalid ID(s). Must be alphanumeric (including - and _), max 64 chars. Comma-separate for multiple."

Check warning

Code scanning / Pylintpython3 (reported by Codacy)

Line too long (117/100) Warning

Line too long (117/100)

Check warning

Code scanning / Pylint (reported by Codacy)

Line too long (117/100) Warning

Line too long (117/100)
)
profile_ids = [p.strip() for p in p_input.split(",") if p.strip()]

if not TOKEN:
print(f"{Colors.CYAN}ℹ API Token is missing.{Colors.ENDC}")
print(f"{Colors.CYAN} You can generate one at: https://controld.com/account/manage-account{Colors.ENDC}")
import getpass
t_input = getpass.getpass(f"{Colors.BOLD}Enter Control D API Token:{Colors.ENDC} ").strip()
if t_input:
TOKEN = t_input

t_input = get_validated_input(
f"{Colors.BOLD}Enter Control D API Token:{Colors.ENDC} ",
lambda x: bool(x),

Check warning

Code scanning / Prospector (reported by Codacy)

Lambda may not be necessary (unnecessary-lambda) Warning

Lambda may not be necessary (unnecessary-lambda)

Check notice

Code scanning / Pylintpython3 (reported by Codacy)

Lambda may not be necessary Note

Lambda may not be necessary

Check notice

Code scanning / Pylint (reported by Codacy)

Lambda may not be necessary Note

Lambda may not be necessary
Copy link

Copilot AI Jan 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This 'lambda' is just a simple wrapper around a callable object. Use that object directly.

Suggested change
lambda x: bool(x),
bool,

Copilot uses AI. Check for mistakes.
"Token cannot be empty.",
is_password=True
)
TOKEN = t_input

if not profile_ids and not args.dry_run:
log.error("PROFILE missing and --dry-run not set. Provide --profiles or set PROFILE env.")
Expand Down Expand Up @@ -925,7 +965,7 @@
})
continue

log.info("Starting sync for profile %s", profile_id)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

This expression logs
sensitive data (password)
as clear text.
status = sync_profile(
profile_id,
folder_urls,
Expand Down Expand Up @@ -1015,11 +1055,11 @@
status_color = Colors.GREEN if res['success'] else Colors.FAIL

print(
f"{res['profile']:<{profile_col_width}} | "
f"{res['folders']:>10} | "
f"{res['rules']:>10,} | "
f"{res['duration']:>9.1f}s | "
f"{status_color}{res['status_label']:<15}{Colors.ENDC}"

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

This expression logs
sensitive data (password)
as clear text.
)
total_folders += res['folders']
total_rules += res['rules']
Expand All @@ -1045,12 +1085,12 @@
total_status_color = Colors.GREEN if all_success else Colors.FAIL

print(
f"{Colors.BOLD}"
f"{'TOTAL':<{profile_col_width}} | "
f"{total_folders:>10} | "
f"{total_rules:>10,} | "
f"{total_duration:>9.1f}s | "
f"{total_status_color}{total_status_text:<15}{Colors.ENDC}"

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

This expression logs
sensitive data (password)
as clear text.
)
print("=" * table_width + "\n")

Expand Down
Loading