Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 29 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,28 +7,27 @@
[<img src="https://img.shields.io/pypi/dm/fluxative" alt="PyPI - Downloads">](https://pypi.org/project/fluxative/)
[<img src="https://img.shields.io/github/stars/JakePIXL/fluxative" alt="GitHub Repo stars">](https://github.com/JakePIXL/fluxative/stargazers)

Fluxative streamlines the conversion of Git repositories into standardized context files optimized for LLM consumption. The project architecture consists of three core components working together:

A tool to convert Git repositories into standardized context files for LLM consumption. Consists of three main components:

- `converter.py`: Converts GitIngest output to llms.txt and llms-full.txt formats
- `expander.py`: Expands llms.txt files with actual file content from GitIngest
- `llmgentool.py`: Integrates both modules for an end-to-end solution
- `converter.py`: Transforms GitIngest output into structured llms.txt and llms-full.txt formats
- `expander.py`: Enhances llms.txt files by embedding actual file content from GitIngest
- `fluxative.py`: Integrates both modules for a seamless end-to-end solution

## Features

- Generate LLM-friendly context files from Git repositories or GitHub URLs
- Creates five output files:
- Create a comprehensive set of output files:
- `repo-raw.txt`: Complete original GitIngest output with Summary, Tree, and File Contents
- `repo-llms.txt`: Basic repository summary with original structure preserved
- `repo-llms-full.txt`: Comprehensive repository summary with original structure preserved
- `repo-llms-ctx.txt`: Basic summary with file contents
- `repo-llms-full-ctx.txt`: Comprehensive summary with file contents
- Preserves the full structure (Summary, Tree, and Content) from GitIngest
- Automatically organizes output files in a directory named after the repository
- `repo-llms-ctx.txt`: Basic summary with embedded file contents
- `repo-llms-full-ctx.txt`: Comprehensive summary with embedded file contents
- Preserve the full structure (Summary, Tree, and Content) from GitIngest
- Automatically organize output files in a structured directory named after the repository

## Installation

### Using uv
### Using uv (Recommended)

```bash
uv install fluxative
Expand All @@ -37,16 +36,16 @@ uv install fluxative
### From source

```bash
git clone https://github.com/JakePIXL/Fluxative.git
cd Fluxative
git clone https://github.com/JakePIXL/fluxative.git
cd fluxative
pip install -e .
```

### For development

```bash
git clone https://github.com/JakePIXL/Fluxative.git
cd Fluxative
git clone https://github.com/JakePIXL/fluxative.git
cd fluxative
pip install -e ".[dev]"
```

Expand All @@ -61,33 +60,38 @@ fluxative /path/to/repo
# Process a GitHub URL
fluxative https://github.com/username/repo

# Specify an output directory
# Specify a custom output directory
fluxative /path/to/repo --output-dir /custom/output/path
```

### With uvx

If you have [uv](https://docs.astral.sh/uv) installed:
If you have [uv](https://docs.astral.sh/uv) installed, you can run Fluxative directly without installation:

```bash
# Process a repository
uvx fluxative /path/to/repo

# With output directory
# With custom output directory
uvx fluxative /path/to/repo -o /custom/output/path
```

## Output

The tool creates a directory named `<repo-name>-docs` containing:
Fluxative creates a directory named `<repo-name>-docs` containing different files based on the arguments used:

- `<repo-name>-raw.txt`: Complete original GitIngest output with Summary, Tree structure, and File Contents
- `<repo-name>-llms.txt`: Basic overview of the repository including original structure
- `<repo-name>-llms-full.txt`: Comprehensive overview with all files including original structure
- `<repo-name>-llms-ctx.txt`: Basic overview with embedded file contents
### Default Output (Always Generated)
- `<repo-name>-llms.txt`: Basic overview of the repository preserving original structure
- `<repo-name>-llms-ctx.txt`: Basic overview with embedded file contents for quick reference

### With `--full-context` Flag
- `<repo-name>-llms-full.txt`: Comprehensive overview including all files with original structure
- `<repo-name>-llms-full-ctx.txt`: Comprehensive overview with all embedded file contents

Each file preserves the original structure from GitIngest, ensuring you have access to:
### With `--dump-raw` Flag
- `<repo-name>-raw.txt`: Complete original GitIngest output with Summary, Tree structure, and File Contents

Each output file maintains the original structure from GitIngest, providing you with:
- Repository summary (name, URL, branch, commit)
- Complete directory tree structure
- File contents organized by category
Expand All @@ -96,6 +100,7 @@ Each file preserves the original structure from GitIngest, ensuring you have acc

- Python 3.10+
- GitIngest 0.1.4 or higher
- Typer 0.15.2 or higher

## License

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ readme = "README.md"
requires-python = ">=3.10"
authors = [{ name = 'JakePIXL' }]
license = { text = "MIT" }
dependencies = ["gitingest>=0.1.4"]
dependencies = ["gitingest>=0.1.4", "typer>=0.15.2"]

[project.urls]
Homepage = "https://github.com/JakePIXL/fluxative"
Expand Down
76 changes: 44 additions & 32 deletions src/llmgentool.py → src/fluxative.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,29 +5,35 @@
This script integrates converter.py and expander.py to generate standardized context files
for LLMs from either a local repository or a GitHub URL.

It generates:
- llms.txt: Basic repository summary
- llms-full.txt: Comprehensive repository summary
- llms-ctx.txt: Basic summary with file contents
- llms-ctx-full.txt: Comprehensive summary with file contents
It can generate the following:
- llms.txt: Basic repository summary
- llms-ctx.txt: Basic summary with file contents

with --full-context enabled:
- llms-full.txt: Comprehensive repository summary
- llms-ctx-full.txt: Comprehensive summary with file contents

Usage:
python llmgentool.py <repo_path_or_url> [output_directory]
python fluxative.py <repo_path_or_url> [output_directory]
"""

import sys
import os
import shutil
import tempfile
import argparse
from typing import Optional
from typing_extensions import Annotated
from urllib.parse import urlparse

# Import from local modules
from typing import Dict
from src.converter import parse_gitingest_output, generate_llms_txt
from src.expander import generate_ctx_file

import typer

app = typer.Typer(context_settings={"help_option_names": ["-h", "--help"]})

try:
from gitingest import ingest
except ImportError:
Expand Down Expand Up @@ -70,7 +76,12 @@ def get_repo_name(repo_path_or_url: str) -> str:
return os.path.basename(path)


def process_repository(repo_path_or_url: str, output_dir: Optional[str] = None) -> tuple[str, Dict]:
def process_repository(
repo_path_or_url: str,
output_dir: Optional[str] = None,
full_context: bool = False,
dump_raw: bool = False,
) -> tuple[str, Dict]:
"""
Process a repository to generate LLM context files.

Expand Down Expand Up @@ -134,14 +145,16 @@ def process_repository(repo_path_or_url: str, output_dir: Optional[str] = None)
llms_full_txt_path = os.path.join(temp_dir, f"{repo_name}-llms-full.txt")

generate_llms_txt(repo_info, llms_txt_path, full_version=False)
generate_llms_txt(repo_info, llms_full_txt_path, full_version=True)
if full_context:
generate_llms_txt(repo_info, llms_full_txt_path, full_version=True)

# Generate context files
ctx_output_path = os.path.join(temp_dir, f"{repo_name}-llms-ctx.txt")
ctx_full_output_path = os.path.join(temp_dir, f"{repo_name}-llms-full-ctx.txt")

generate_ctx_file(llms_txt_path, gitingest_file, ctx_output_path)
generate_ctx_file(llms_full_txt_path, gitingest_file, ctx_full_output_path)
if full_context:
generate_ctx_file(llms_full_txt_path, gitingest_file, ctx_full_output_path)

# Copy files to output directory
for file in [
Expand All @@ -164,29 +177,28 @@ def process_repository(repo_path_or_url: str, output_dir: Optional[str] = None)
sys.exit(1)


def main():
"""Main function"""
parser = argparse.ArgumentParser(
description="Generate LLM context files from a repository or GitHub URL"
)
parser.add_argument("repo_path_or_url", help="Local path or URL to a git repository")
parser.add_argument(
"--output-dir",
"-o",
help="Directory to save output files (default: current directory)",
default=None,
)

# Support for command-line usage within a package like 'uvx'
if len(sys.argv) > 1 and sys.argv[1] == "fluxative":
# Handle the case where this is called as 'uvx fluxative <args>'
args = parser.parse_args(sys.argv[2:])
else:
# Normal command-line usage
args = parser.parse_args()
@app.command(no_args_is_help=True)
def main(
repo_path_or_url: Annotated[str, typer.Argument(help="Local path or URL to a git repository")],
output_dir: Annotated[
Optional[str],
typer.Option(
"--output-dir", "-o", help="Directory to save output files (default: current directory)"
),
] = None,
dump_raw: Annotated[
bool, typer.Option("--dump-raw", "-d", help="Dump raw GitIngest output")
] = False,
full_context: Annotated[
bool, typer.Option("--full-context", "-f", help="Generate full context files")
] = False,
):
"""
Generate LLM context files from a repository.
"""

# Process the repository
output_path, repo_info = process_repository(args.repo_path_or_url, args.output_dir)
output_path, repo_info = process_repository(repo_path_or_url, output_dir)

repo_name = repo_info["name"]
print(f"Files generated in: {output_path}")
Expand All @@ -198,4 +210,4 @@ def main():


if __name__ == "__main__":
main()
app()
4 changes: 2 additions & 2 deletions tests/test_import.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ def test_import():
"""Test that modules can be imported directly."""
import src.converter as converter
import src.expander as expander
import src.llmgentool as llmgentool
import src.fluxative as fluxative

assert converter is not None
assert expander is not None
assert llmgentool is not None
assert fluxative is not None
Loading