Implement social media data collection, LLM-powered Rust analysis backend, and GitHub Actions scheduling

[Copilot]

Greenfield implementation of the KashCompiler platform: scheduled social media ingestion via pluggable TypeScript adapters, Prisma ORM persistence, and a Rust analysis backend with pluggable LLM-driven algorithms.

TypeScript collector (packages/collector/)

• Schemas — TypeScript-first types (SocialPost, SocialUser, CollectionRun, AnalysisResult) mirrored in Prisma ORM (SQLite dev / Postgres prod)
• Adapter framework — BaseAdapter abstract class + AdapterRegistry for runtime-pluggable sources
  • GrokAdapter — xAI Grok search API
  • TwitterAdapter — Twitter/X v2 recent search (******
  • RedditAdapter — Reddit public hot posts (no auth required)
• ORM helpers — upsert-safe savePosts, createCollectionRun, finalizeCollectionRun
• CLI runner (runners/collect.ts) — fully env-var-driven; consumed directly by GitHub Actions

// Custom adapter in 10 lines
export class MyAdapter extends BaseAdapter {
  readonly name = "my-source";
  readonly requiredConfigKeys = ["apiKey"];
  async collect(config: AdapterConfig): Promise<CollectionResult> { ... }
}
defaultRegistry.register(new MyAdapter());

Rust analyzer (crates/analyzer/)

• Pluggable LlmProvider trait — GrokProvider (xAI) and OpenAiProvider (OpenAI-compatible); swap or add providers without touching pipeline code
• Pluggable Algorithm trait — SentimentAlgorithm (keyword heuristic with optional LLM override) and TrendAlgorithm (tag aggregation + LLM summary)
• CLI binary — reads posts JSON, runs algorithm pipeline, writes AnalysisResult JSON; --llm none falls back to offline heuristics

GitHub Actions

• collect.yml — scheduled every 6 h; collect → persist → export JSON → analyze → upload artifacts. Manual dispatch with sources, topics, max_posts overrides
• ci.yml — TypeScript build/test + Rust fmt/clippy/test on push and PR

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• checkpoint.prisma.io
  • Triggering command: /opt/hostedtoolcache/node/24.14.1/x64/bin/node /opt/hostedtoolcache/node/24.14.1/x64/bin/node /home/REDACTED/work/KashCompiler/KashCompiler/node_modules/prisma/build/child {"product":"prisma","version":"5.22.0","cli_install_type":"local","information":"","local_timestamp":"2026-04-07T01:17:42Z","project_hash":"35c0b3fa","cli_path":"/home/REDACTED/work/KashCompiler/KashCompiler/node_modules/prisma/build/index.js","cli_path_has (dns block)
  • Triggering command: /opt/hostedtoolcache/node/24.14.1/x64/bin/node /opt/hostedtoolcache/node/24.14.1/x64/bin/node /home/REDACTED/work/KashCompiler/KashCompiler/node_modules/prisma/build/child {"product":"prisma","version":"5.22.0","cli_install_type":"local","information":"","local_timestamp":"2026-04-07T01:17:49Z","project_hash":"35c0b3fa","cli_path":"/home/REDACTED/work/KashCompiler/KashCompiler/node_modules/.bin/prisma","cli_path_hash":"7caad4 (dns block)
  • Triggering command: /opt/hostedtoolcache/node/24.14.1/x64/bin/node /opt/hostedtoolcache/node/24.14.1/x64/bin/node /home/REDACTED/work/KashCompiler/KashCompiler/node_modules/prisma/build/child {"product":"prisma","version":"5.22.0","cli_install_type":"local","information":"","local_timestamp":"2026-04-07T01:17:55Z","project_hash":"35c0b3fa","cli_path":"/home/REDACTED/work/KashCompiler/KashCompiler/node_modules/.bin/prisma","cli_path_hash":"7caad4 (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

We will create a site which gathers data from social media websites, which are freely accessible, but via api keys such as grok, to gather data in a schema purely defined in typescript -> ORM to process later. Ideally, these background and scheduled processes, as part of the github infrastructure, can gather our data off of these adapter based (think chrontab but not) sources, what we need later for algorithmic decsions. We will utilize a rust backend for efficient analysis, with algorithms plugged into a variety of LLM sources, to consume the data, and make decisions, with pluggable algorithms designed by these LLMs