video: required video capability (0xfa) + 0xBEDE RTP extension so sent video renders#3
Open
AstraOnlineWeb wants to merge 2 commits into
Open
Conversation
… path When answering an inbound video call, the preaccept/accept were built as if it were audio: the audio capability blob (byte 6 0xbb) and no <video> child. A real client flips that byte to 0xfa and advertises <video> in both the preaccept and the accept; without it the callee never brings video up (which is why enabling the audio capability on a video call was failing). - add CapabilityVideoOffer (…e4 fa 13) alongside the audio CapabilityOffer - preaccept: advertise <video dec=H264> + the video capability when the offer is video - accept: advertise <video enc=h264> when the offer is video - tests for the capability byte and the preaccept node
Every H.264 packet from the official client carries an RFC 5285 one-byte header extension (profile 0xBEDE): abs-send-time (id 3), video-orientation/CVO (id 5) and transport-wide-cc seq (id 6). The peer drops video from a sender that omits it; audio is tolerated without it. - RtpHeader gains a generic extension (ExtensionProfile + ExtensionData) next to the existing audio 0xdebe word; EncodeRtpHeader emits it - BuildVideoRtpExtension builds the abs-send-time + CVO + transport-cc block - videoSender attaches it per packet and carries a per-packet transport-cc seq - tests for the extension layout and header encoding
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
While testing the video send path against the official WhatsApp clients (Android/iOS), the peer never rendered my video — even though audio and the relay bridge were fine. Comparing my answer-path stanzas and outbound RTP against a capture of a real video call surfaced two missing pieces. With both fixed, the sent camera shows up on the peer.
1. The answer path advertised audio, not video
On an inbound video call, the
preaccept/acceptwere built as if it were an audio call:01 05 f7 09 e4 bb 13. A real client flips byte 6 to0xfaon a video call (01 05 f7 09 e4 fa 13);<video>child in thepreaccept(and none in theacceptunless explicitly requested).Without the
0xfacapability + the<video>advertisement the callee doesn't bring video up. This also explains the existingsendAcceptnote that "capability+both-rates fails" — with the audio capability on a video call, it does.This PR:
CapabilityVideoOffer(…e4 fa 13) next to the audioCapabilityOffer;preaccept: advertises<video dec="H264" …>and uses the video capability when the offer is video;accept: advertises<video enc="h264">when the offer is video (via the existingAcceptParams.Video).2. Every sent video packet needs the RFC 5285
0xBEDEheader extensionThe official client carries a one-byte header extension on every H.264 RTP packet (profile
0xBEDE):abs-send-time(id 3) + video-orientation/CVO (id 5) + transport-wide-cc seq (id 6). The peer drops video from a sender that omits it; audio is tolerated without it.videoSenderwas sending none.This PR:
RtpHeadera generic extension (ExtensionProfile+ExtensionData) alongside the existing audio0xdebeword, and hasEncodeRtpHeaderemit it;BuildVideoRtpExtension(tccSeq, cvo)for the abs-send-time + CVO + transport-cc block;videoSenderattach it per packet with a per-packet transport-cc sequence.Notes
0xfacapability byte and the0xBEDEextension layout are taken from real video-call captures. The exact stanza wiring mirrors a working sender (WaCalls).<video>node, the extension byte layout and the header encoding;go test ./...is green andgofmtclean.