feat: streaming#182
Merged
Merged
Conversation
Move video decode work off the render loop by adding a prioritized SDL worker thread for video decoding and refactor decode logic. Introduce run_video_decode_thread, decode_video_decode_unit, receive/publish helpers, packet buffering and frame-versioning to avoid unnecessary copies and only upload textures when a new frame or size change occurs. Expose submit_video_decode_unit as a thin wrapper and mark pull-renderer capability; add proper start/stop/join handling for the decoder thread. Add controllerMutex and a separate input polling thread to safely read/send controller snapshots and detect stream-exit combo (with a fallback poll when the thread can't start). Misc: SDL texture/format handling, resource cleanup updates, and various safety/validation checks and logging.
Move/rename the existing .github/copilot-instructions.md file to AGENTS.md. File contents are unchanged (100% similarity); this is purely a rename to relocate or rebrand the instructions file.
Update streaming defaults and improve video decode handling for lower-latency playback.
Key changes:
- Change default stream settings: framerate moved from 20 -> 30 and bitrate from 1500kbps -> 1000kbps (client_state, settings_storage, session constants and tests updated).
- Reduce exposed stream resolution presets and favor low-latency small presets (startup/video_mode.cpp) and simplify default selection to pick low-latency NTSC/PAL presets. Unit tests updated accordingly.
- Add decode-queue dropping support: new drop_queued_video_decode_units, has_unrendered_video_frame, droppedDecodeUnitCount counter, and logging for dropped frames (ffmpeg_stream_backend.{cpp,h}). The video decode loop now drops queued decode units before decoding and requests IDR when needed.
- Improve FFmpeg codec behavior by setting low-delay / output-corrupt flags, fast/show-all flags, and stricter error recognition; zero dropped count on reset and include dropped counts in overlay status.
- Tweak render loop timing: faster present polling when decoded video exists and performance overlay is disabled; adjust default packet size to 1392.
Files touched include src/app/*, src/startup/video_mode.cpp, src/streaming/ffmpeg_stream_backend.* and src/streaming/session.cpp, plus tests reflecting the new defaults and presets.
Introduce a mutex to logging::Logger and guard getters, setters and log paths (added should_log_unlocked) to make logging safe for concurrent use; add a unit test to verify bounded retained buffer under concurrent logging. Add NXDK-specific direct framebuffer presentation: new helpers (build_unscaled_destination, rectangles_match, xbox framebuffer format/bytes helpers), a render_latest_video_frame_to_framebuffer path, presentScaleContext and bookkeeping fields in VideoState, and attempt direct presentation when allowed. Also filter/normalize noisy connection logs in session (high-volume message suppression and connection_log_level) and update render_latest_video_frame API to accept an allowDirectFramebuffer flag with a default. Misc: free presentScaleContext on reset and minor includes/headers updates.
Add a new playAudioOnXbox setting with UI/menu toggle and persistence (default: false). Wire the new setting through client state, settings storage, main startup, and unit tests. Enhance FFmpeg/streaming behavior to support disabling Xbox-side audio decode/playback: add FfmpegStreamBackend::set_audio_playback_enabled, short-circuit Opus decode when disabled, expose capability flags, and avoid unnecessary audio work. Improve audio buffering by reusing an output buffer vector and tracking queued bytes. Make several decoder and runtime improvements to reduce latency and robustness: lower FFmpeg log level and ignore a noisy message, add codec skip flags, yield briefly in the decoder loop, track per-frame decode timing/queue/age and expose it in the overlay, request IDR frames after idle video periods, adjust thread priorities (scoped RAII helper), and change default packet size. Also: expand stream bitrate options and introduce DEFAULT_STREAM_BITRATE_KBPS constant; minor UI text tweak and updated unit tests to account for the new setting.
Switch video timestamping from SDL ticks to platform microseconds (PltGetMicroseconds): rename lastDecodedFrameTicks -> lastDecodedFrameUs, change milliseconds_since_last_decoded_video_frame to accept microseconds, and persist microsecond timestamps. Tighten decode logic: avoid decoding non-IDR frames after dropped frames, mark need for IDR, and append "waiting_for_idr" to drop logs. Remove unused build_unscaled_destination and use build_letterboxed_destination for rendering. Add presentation helpers: select_stream_presentation_video_mode and ScopedStreamVideoMode (NXDK-only) to optionally switch output mode for stream presentation and use the active mode when initializing UI resources and configuring stream launch parameters. Minor reorder of idle/IDR checks to use the new time API.
Switch stream-resolution settings from fixed presets to detected Xbox video modes and add encoder-aware filtering. Introduces filter_stream_video_modes_for_encoder_settings() to hide SD wide-width modes unless VIDEO_WIDESCREEN is set, and overloads choose_default_stream_video_mode() to prefer a detected mode (or smallest available) with a canonical fallback. initialize_stream_video_mode_settings() now accepts encoder settings and handles empty mode lists, and select_stream_presentation_video_mode() is simplified to use the chosen stream mode when available. Updated comments/labels and adjusted unit tests to match new mode list, defaults, and behavior.
Add explanatory comments for several compatibility macros in ffmpeg_compat.h and share.h (ENOSYS, O_BINARY, F_SETFD, FD_CLOEXEC, CP_ACP, CP_UTF8, MB_ERR_INVALID_CHARS, WC_ERR_INVALID_CHARS, SH_DENYNO) to clarify their purpose for FFmpeg/nxdk builds. Also add an extern "C" declaration (with doc comment) for PltGetMicroseconds in ffmpeg_stream_backend.cpp and session.cpp so the platform monotonic clock is visible to moonlight-common-c. These are documentation and forward-declaration changes to improve maintainability and cross-module compatibility; no behavioral changes intended.
Rename cmake variables to explicit *wrapper* names and set ffmpeg_cc_shell_path / ffmpeg_cxx_shell_path to invoke the wrapper via `sh`. Update ffmpeg-nxdk-cc.sh and ffmpeg-nxdk-cxx.sh to correctly handle `-o` and `-o<file>` forms, create output directories when needed, touch the output file, and try to mark it executable. These changes ensure the nxdk ffmpeg wrappers are invoked reliably and produce expected output files during build.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #182 +/- ##
==========================================
+ Coverage 62.41% 63.05% +0.64%
==========================================
Files 32 32
Lines 4898 5235 +337
Branches 2185 2385 +200
==========================================
+ Hits 3057 3301 +244
- Misses 910 931 +21
- Partials 931 1003 +72
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report in Codecov by Sentry.
|
Modularize and clean up streaming and session code: extract helper structs and functions in session.cpp (StreamStartAttemptContext, ActiveStreamLoopContext, TextLineLayout), add assign_status_message, and split stream startup and active-loop logic into run_stream_start_attempt, run_stream_start_with_rtsp_fallback, run_active_stream_loop and related helpers. Improve controller handling (open/close helpers, fallback polling), implement RTSP session URL fallback retry, and centralize input-thread lifecycle and rendering/polling behavior.
Also applied numerous style and safety improvements across the codebase: use if-with-init and compact conditionals, add NOSONAR annotations to silence specific static analysis warnings in ffmpeg_stream_backend, make several const-correctness and type simplifications (remove unnecessary casts), refactor render_text_line to accept a layout struct, and minor cleanups in main.cpp, host_pairing.cpp, ffmpeg_stream_backend.{cpp,h}, and ui/shell_screen.cpp. These changes are intended to improve readability, maintainability, and static-analysis cleanliness without changing external behavior.
Unset the MSYS2_ARG_CONV_EXCL environment variable in the cmake nxdk ffmpeg preparation and in both ffmpeg-nxdk-cc.sh and ffmpeg-nxdk-cxx.sh scripts. This prevents MSYS2 argument/path conversion from interfering with nxdk/FFmpeg build steps and helps ensure consistent behavior in the build environment.
Update session.cpp to accept std::string_view in assign_status_message and use string::assign to copy the view, improving clarity and avoiding implicit conversions. Add copy_controller_device_event to safely copy SDL controller device events and use it in pump_stream_events when opening/closing controllers. Simplify StreamStartAttemptContext usage by moving its initialization into an if-with-initializer. In tests, add expect_unique_id_request, expect_unauthenticated_unique_id_request, and expect_authenticated_unique_id_request helpers and refactor repetitive EXPECT_* checks to use these helpers for clearer, less duplicated assertions.
Modularize and harden the FFmpeg-for-Xbox build logic: extract validation, revision/signature computation, shell path conversion, configure-args composition, Windows MSYS2 command runner, and the rebuild routine into helper functions. The main moonlight_prepare_xbox_ffmpeg now validates inputs, computes a rebuild signature from sources and wrapper files, and only rebuilds when needed. Also make MSYS2 configure invocation more robust in GetOpenSSL by concatenating the configure script, and tighten CMake formatting. Update ffmpeg wrapper scripts to set CDPATH='' when resolving script_dir to avoid unexpected path resolution behavior on some shells.
Replace direct WIN32 checks with CMAKE_HOST_WIN32 across CMake scripts and centralize MSYS2 shell lookup. Calls to the hardcoded C:/msys64/msys2_shell.cmd were removed in favor of _moonlight_get_windows_msys2_shell, and several nxdk-related functions now use CMAKE_HOST_WIN32 to detect a Windows host. Modified files: cmake/moonlight-dependencies.cmake, cmake/msys2.cmake, cmake/nxdk.cmake. This makes host detection and MSYS2 resolution more robust (e.g. when cross-compiling) and avoids embedding a fixed MSYS2 path.
Add an end-of-stream performance summary display and update related settings/labels. This change introduces rendering helpers (render_text_lines, render_stream_end_statistics_frame, show_stream_end_statistics) plus a polling loop to wait for user input before leaving the summary screen. The in-stream overlay handling was simplified (removed per-frame overlay blend) and stream-frame rendering calls were updated to the new signature that uses StreamUiResources. A new poll interval constant (STREAM_END_STATS_POLL_MILLISECONDS) was added. Settings text and persistence comment for show_performance_stats were updated to reflect the new end-of-stream semantics, and playAudioOnXbox default was changed to true; unit tests adjusted accordingly. Misc docs/comments were tweaked to match the new behavior.
Add tests to increase coverage for settings UI, storage defaults, and stats formatting: - tests/unit/app/client_state_test.cpp: add DisplaySettingsCanToggleXboxAudioAndEndStreamStats to verify toggling "Play Audio on Xbox" and "Show End Stream Stats" updates state, status messages, and persistence flags. - tests/unit/app/settings_storage_test.cpp: include playAudioOnXbox in the saved defaults and assert loaded value is false. - tests/unit/streaming/stats_overlay_test.cpp: add FormatsPartiallyAvailableMetricGroups to validate overlay line formatting when some metrics are missing. These tests ensure toggles, default persistence, and stats overlay formatting behave correctly with partial data.
|
Introduce 60 FPS streaming support and related behavior changes. Added DEFAULT_STREAM_FRAMERATE (30) and extended STREAM_FRAMERATE_OPTIONS to include 60; initialize settings.streamFramerate from the default constant and make cycle_stream_framerate fall back to the default when the current value is invalid. Bumped MAX_STREAM_FPS to 60 in the streaming session constants. Updated unit tests to expect the default 30 FPS and added a test that verifies cycling includes 60 FPS and that invalid values reset to the default.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds the initial streaming.
TODO:
Screenshot
Issues Fixed or Closed
Roadmap Issues
Type of Change
Checklist
AI Usage