Fixes #365 - Add tail command to show recent service logs

[jakduch]

Adds a new foremanctl tail command as an alternative to the foreman-tail tool from foreman-installer based installations.

Uses journalctl to collect and display recent logs from all Foreman services. Only services that are actually running on the system are included in the output.

Usage:

• foremanctl tail — show last 50 lines from all services
• foremanctl tail --lines 200 — show last 200 lines
• foremanctl tail --service foreman --service pulp-api — filter specific services

Services covered: foreman, dynflow-sidekiq (orchestrator, worker, worker-hosts-queue), pulp-api, pulp-content, candlepin, httpd, redis, postgresql

[pablomh]

Does it support -f?

[jakduch]

Not yet — follow mode (-f) is tricky through Ansible since journalctl --follow is a blocking command and Ansible buffers output rather than streaming it. For now the command shows the last N lines (default 50).

We could add -f support in a follow-up by having foremanctl exec journalctl --follow directly (bypassing Ansible) when follow mode is requested. Would that approach work, or is there an established pattern in foremanctl/obsah for streaming output?

[ehelms]

There is a trend of listing services. Something to think about is whether this means we should put them in centralized location.

[ehelms]

Doesn't line 5 handle that?

[ehelms]

We could add -f support in a follow-up by having foremanctl exec journalctl --follow directly (bypassing Ansible) when follow mode is requested. Would that approach work, or is there an established pattern in foremanctl/obsah for streaming output?

No pattern for streaming output. As mentioned, this is not something Ansible does. Can you paste example output of the command as is?

This begs the question is foremanctl the right tool for the job? One of our design paradigms is to embrace the core system libraries for handling things (e.g. systemd, journald). Can we define configuration on the system that makes interacting with journald for our services easier?

[jakduch]

You're right - I was trying to avoid changing the underlying infrastructure and just wrapped the existing services in an Ansible playbook. But that goes against the design principle of embracing systemd/journald.

Since all Foreman services are already registered under foreman.target via PartOf=, users can already do:

journalctl -u foreman.target -n 50
journalctl -u foreman.target -f

This gives both recent logs and follow mode for free, without foremanctl needing to manage it. The only thing missing is discoverability - users might not know about foreman.target.

We could address that by documenting it (e.g. in foremanctl --help or a man page), or by making foremanctl tail a thin passthrough to journalctl -u foreman.target with the same arguments. But at that point it's questionable whether the wrapper adds enough value.

Happy to close this PR if you think the native journald approach is sufficient. Or should I rework it here to use the foreman.target approach instead? Also open to a separate PR if you'd prefer to compare both solutions side by side.