feat(video): build video generation -> ingest -> publish (flag-OFF)#721
Merged
Conversation
Builds the missing generation half of the video pipeline; the Meta Graph publish branches already existed gated behind ARIES_VIDEO_PUBLISH_ENABLED. Provider is Grok/xAI via Hermes (Aries sends a high-level request and never names the tool). Ships dormant: the flag stays OFF and IG video is inert until a Meta IG Business account is linked to the Page (#691). - schema: width_px/height_px/duration_seconds on creative_assets/posts/ scheduled_posts (+ migration 20260623120000); thread mediaMetadata from scheduled_posts -> worker -> scheduled-dispatch -> publishToMetaGraph (direct + composio seam), built only for a video surface with all 3 dims. - ingest: branch on type=generated_video / media_type=video -> write media_type, real aspect_ratio, and dims; logo-composite skips video. - synthesize: copy the linked asset's dims onto the posts row. - outbound: VIDEO_EXECUTION_CONTRACT + per-clip 9:16 brief, gated on the flag + videoRenderCount>0; byte-identical contextBlock when off. - publish: Composio FB feed-video (FACEBOOK_CREATE_VIDEO_POST, new publish_video op) + IG REELS/STORIES two-step container+publish; this also fixes the previously broken single-call IG branch. A pre-publish validateMediaForSurface throw is wrapped as ComposioToolError so it is classified never-posted (a raw MetaPublishError was mis-classified outcome-unknown). IG feed-video validated against reel 9:16 constraints. - dashboard: video assets render as <video> (shared MediaPreview already supported it); manual-schedule route threads dims. - contract spec for the Hermes side: docs/plans/2026-06-23-video-generation.md verify + validate:social-content + validate:execution-provider all green; composio-publisher 22/22. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Jq6YvGxzP23enohEiMpmD
Contributor
There was a problem hiding this comment.
Pull request overview
This PR builds out the generation → ingest → synthesize → schedule/dispatch → publish wiring needed for short-form vertical video in the weekly social-content pipeline, while keeping production behavior dormant behind ARIES_VIDEO_PUBLISH_ENABLED.
Changes:
- Adds
width_px/height_px/duration_secondscolumns tocreative_assets,posts, andscheduled_posts, and threads them through scheduling + internal publish dispatch so Meta video validation can run fail-closed before any publish call. - Extends ingest/synthesize/workspace mapping to treat generated videos as first-class creative assets (including dashboard rendering support via
posterUrlplumbing). - Adds/updates Composio publishing for video (FB feed video + IG two-step container→publish) and adds extensive regression tests for the new wiring.
Reviewed changes
Copilot reviewed 34 out of 34 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/video-synthesize-dims.test.ts | Verifies synthesize step copies video dims from creative_assets into posts inserts. |
| tests/video-publish-dispatch-wiring.test.ts | End-to-end tests for mediaMetadata threading + DDL assertions for new columns. |
| tests/social-content/video-production-context.test.ts | Golden tests ensuring contextBlock is byte-identical when flag is off; verifies video brief emission when enabled. |
| tests/social-content-schedule-dims.test.ts | Ensures manual schedule route threads dims into scheduled_posts upsert. |
| tests/marketing/workspace-video-asset-mapping.test.ts | Ensures workspace view maps video assets to video/mp4 and correct titles. |
| tests/marketing/video-ingest-dims.test.ts | Verifies ingest binds media_type, aspect ratio, and dims for video assets. |
| tests/marketing/ingest-production-assets.test.ts | Updates ingest SQL-shape assertions for param-bound media fields + dims. |
| tests/marketing-auto-schedule.test.ts | Updates auto-schedule tests for widened post row shape (dims included). |
| tests/composio-publisher.test.ts | Adds/updates coverage for IG two-step publish + FB video publish + validation behavior. |
| scripts/init-db.js | Adds/backfills new video metadata columns in init schema and backfill ALTERs. |
| scripts/automations/scheduled-posts-worker.mjs | Threads scheduled_post dims into scheduled-dispatch POST payload. |
| migrations/20260623120000_video_media_metadata.sql | Adds new nullable dims/duration columns to key publish tables (idempotent). |
| lib/api/marketing.ts | Adds optional posterUrl to creative asset review payload. |
| lib/api/aries-v1.ts | Adds optional posterUrl to runtime review items. |
| frontend/aries-v1/review-item.tsx | Passes poster through to media preview for review items. |
| frontend/aries-v1/post-workspace.tsx | Passes poster through to media preview in post workspace. |
| docs/plans/2026-06-23-video-generation.md | Documents Hermes return contract and Aries-side wiring/rollout gates. |
| docker-compose.yml | Adds Composio env vars for FB video publish + IG account info lookup. |
| backend/social-content/workflow-request.ts | Adds gated video prompt construction + return schema injection into contextBlock. |
| backend/social-content/scheduled-posts.ts | Extends scheduled_posts upsert to persist dims/duration fields. |
| backend/marketing/workspace-views.ts | Widens creative_assets select and maps video rows to video content types/titles. |
| backend/marketing/synthesize-publish-posts.ts | Selects asset dims and threads them into synthesized posts inserts. |
| backend/marketing/ports/hermes.ts | Adds exported VIDEO_EXECUTION_CONTRACT to production-stage instructions. |
| backend/marketing/ingest-production-assets.ts | Adds video-aware ingest (media_type/aspect/dims) and skips framing for video. |
| backend/marketing/hermes-callbacks.ts | Widens posts query + threads dims into auto-schedule input rows. |
| backend/marketing/auto-schedule.ts | Propagates dims through auto-scheduling into scheduled_posts upsert. |
| backend/integrations/publish-dispatch.ts | Forwards mediaMetadata through dispatchPublish request plumbing. |
| backend/integrations/providers/types.ts | Adds mediaMetadata to PublishPostInput for provider-agnostic validation. |
| backend/integrations/direct/direct-meta-provider.ts | Threads mediaMetadata into direct Meta publish calls. |
| backend/integrations/composio/composio-publisher-provider.ts | Implements FB video publish + IG two-step publish + pre-publish validation wrapping. |
| backend/integrations/composio/composio-config.ts | Adds new Composio operation type publish_video. |
| app/api/social-content/jobs/[jobId]/posts/[postId]/schedule/route.ts | Threads dims from posts into scheduled_posts upsert in manual scheduling route. |
| app/api/internal/publishing/scheduled-dispatch/route.ts | Builds mediaMetadata from body dims only when complete; forwards to dispatchPublish. |
| .env.example | Adds documented env vars for new Composio actions/operations. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+338
to
+342
| // Derive media type: 'video' when the entry carries video markers. | ||
| const isVideo = | ||
| asset.type === 'generated_video' || | ||
| (typeof asset.media_type === 'string' && asset.media_type.trim().toLowerCase() === 'video'); | ||
| const mediaType: 'image' | 'video' = isVideo ? 'video' : 'image'; |
Comment on lines
+338
to
+343
| toolArgs = { | ||
| url: input.mediaUrls[0], | ||
| message: input.content, | ||
| page_id: pageId, | ||
| ...(input.scheduledFor ? { scheduled_publish_time: input.scheduledFor } : {}), | ||
| }; |
Comment on lines
+345
to
+351
| // Text-only post: FACEBOOK_CREATE_POST via the `publish_post` op slot. | ||
| slug = this.requireSlug(input.platform, 'publish_post', 'publish posts'); | ||
| toolArgs = { | ||
| message: input.content, | ||
| page_id: pageId, | ||
| ...(input.scheduledFor ? { scheduled_publish_time: input.scheduledFor } : {}), | ||
| }; |
Comment on lines
+5
to
+23
| -- duration_seconds is INTEGER to match insights_posts.duration_seconds (INT). | ||
| -- | ||
| -- Additive + idempotent (ADD COLUMN IF NOT EXISTS, all nullable). Population of | ||
| -- these columns (ingest -> creative_assets, synthesize -> posts) is a separate | ||
| -- deferred follow-up; this migration only creates the storage. | ||
| -- Applied via scripts/init-db.js on fresh DBs; run manually on existing ones. | ||
| -- Reverse: ALTER TABLE <t> DROP COLUMN duration_seconds, DROP COLUMN height_px, DROP COLUMN width_px; | ||
|
|
||
| ALTER TABLE creative_assets ADD COLUMN IF NOT EXISTS width_px INTEGER; | ||
| ALTER TABLE creative_assets ADD COLUMN IF NOT EXISTS height_px INTEGER; | ||
| ALTER TABLE creative_assets ADD COLUMN IF NOT EXISTS duration_seconds INTEGER; | ||
|
|
||
| ALTER TABLE posts ADD COLUMN IF NOT EXISTS width_px INTEGER; | ||
| ALTER TABLE posts ADD COLUMN IF NOT EXISTS height_px INTEGER; | ||
| ALTER TABLE posts ADD COLUMN IF NOT EXISTS duration_seconds INTEGER; | ||
|
|
||
| ALTER TABLE scheduled_posts ADD COLUMN IF NOT EXISTS width_px INTEGER; | ||
| ALTER TABLE scheduled_posts ADD COLUMN IF NOT EXISTS height_px INTEGER; | ||
| ALTER TABLE scheduled_posts ADD COLUMN IF NOT EXISTS duration_seconds INTEGER; |
Comment on lines
289
to
+293
| variant_batch_id TEXT, | ||
| variant_index INTEGER, | ||
| width_px INTEGER, | ||
| height_px INTEGER, | ||
| duration_seconds INTEGER, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Builds the generation half of the video pipeline so Aries can produce and publish short-form vertical video (Reels / Stories / feed video). The Meta Graph publish branches already existed gated behind
ARIES_VIDEO_PUBLISH_ENABLED; this wires up generation → ingest → synthesize → dashboard + the publish-metadata plumbing, and adds the Composio video publish branch.Ships dormant —
ARIES_VIDEO_PUBLISH_ENABLEDstays OFF, and IG video is inert until a Meta IG Business account is linked to the Page (#691). No prod behavior changes until the flag is flipped.Provider is Grok / xAI via Hermes — Aries sends a high-level "generate a 9:16 video" request and never names the tool. The Hermes-side return contract Aries was built against is documented in
docs/plans/2026-06-23-video-generation.md.Slices
width_px/height_px/duration_seconds(INTEGER, nullable) oncreative_assets/posts/scheduled_posts(+ migration20260623120000).mediaMetadatanow threadsscheduled_posts → worker → scheduled-dispatch → dispatchPublish → publishToMetaGraph(direct + Composio seam), built only for a video surface with all three dims present — never fabricated.type=generated_video/media_type=video: writesmedia_type='video', a realaspect_ratio, and dims; logo-composite skips video.postsrow.VIDEO_EXECUTION_CONTRACT+ per-clip 9:16 brief, gated on the flag +videoRenderCount>0; byte-identicalcontextBlockwhen off (golden-tested).FACEBOOK_CREATE_VIDEO_POST, newpublish_videoop) + IG REELS/STORIES two-step container+publish. This also fixes the previously broken single-call IG branch (its params matched no real Composio IG action — IG had never published). A pre-publishvalidateMediaForSurfacethrow is wrapped asComposioToolErrorso it's classified never-posted (a rawMetaPublishErrorwould have been mis-classified outcome-unknown). IG feed-video is validated against reel 9:16 constraints since it publishes as aREELScontainer.<video>(sharedMediaPreview/VideoPreviewalready supported it); manual-schedule route threads dims.Composio coverage (honest)
IG feed-video / Reel / Story + FB feed-video are real. FB Reel / Story are not distinct Composio actions (both collapse to a Page video) — out of scope; a FB video target is published as a Page feed video.
Tests
npm run verify,validate:social-content,validate:execution-providerall green;composio-publisher22/22. New fail-before/pass-after coverage for ingest dims, synthesize dim-copy, the 9:16 video brief + byte-identical-when-off golden guards, workspace video mapping, schedule dim threading, the IG two-step + FB video + never-posted classification, and the IG feed-video-as-reel validation.To flip the flag (follow-up, deferred)
media_type:'video'+ per-asset width/height/duration + a localized mp4 basename — not an expiring CDN URL).ARIES_VIDEO_PUBLISH_ENABLED=1+ the Composio action env keys in both.envand the composeenvironment:block.<video>post.🤖 Generated with Claude Code
https://claude.ai/code/session_018Jq6YvGxzP23enohEiMpmD