Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .github/workflows/validate.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: Validate Plots

on:
push:
pull_request:

jobs:
validate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- run: python -m pip install --upgrade pip
- run: python -m pip install -r requirements.txt
- run: python build_all.py
- run: python scripts/generate_homepage.py
- run: python scripts/generate_readme_links.py
- run: python scripts/generate_sitemap.py
- run: python scripts/validate_all.py
- run: python scripts/check_links.py
- run: python scripts/check_accessibility_static.py
221 changes: 125 additions & 96 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,169 +1,198 @@
# Plots
# Exponential Progress Atlas

Interactive timelines of exponential tech progress – showing growth, compression, and scaling laws enabling modern AI.
Interactive timelines showing how compute, energy, coordination, memory, and adoption compound into civilizational acceleration.

## Quick Links
The root inventory is [`plots_manifest.json`](plots_manifest.json). Homepage cards, README links, build ordering, dashboard lanes, and validation all read from that manifest.

- [AI Compute Timeline (Interactive)](ai-compute-timeline/output/ai_compute_timeline_interactive.html)
- [Adoption Timeline (Interactive)](adoption-timeline/output/adoption_timeline_interactive.html)
- [Energetic Scaling (Interactive)](energetic-scaling/output/energetic_scaling_interactive.html)
- [Civilization Scaling (Interactive)](civilization-scaling/output/civilization_scaling_interactive.html)
- [Energy Leverage (Interactive)](energy-leverage-per-person/output/energy_leverage_interactive.html)
## Published Inventory (9)

- [AI Compute Timeline](ai-compute-timeline/output/ai_compute_timeline_interactive.html)
- [Adoption Timeline](adoption-timeline/output/adoption_timeline_interactive.html)
- [Energetic Scaling](energetic-scaling/output/energetic_scaling_interactive.html)
- [Civilization Scaling](civilization-scaling/output/civilization_scaling_interactive.html)
- [Energy Leverage](energy-leverage-per-person/output/energy_leverage_interactive.html)
- [Model Sizes](model-sizes/output/model_sizes_interactive.html)
- [AI Benchmark Progress](ai-benchmark-progress/output/benchmark_progress_interactive.html)
- [Cost to Train](cost-to-train/output/cost_to_train_interactive.html)
- [Unified Dashboard](dashboard/index.html)

---

## 1. AI Compute Timeline (1900–2026)
## 1. AI Compute Timeline

Training compute from early electronic computing to frontier AI, with proxies and speculative projections labeled separately.

Training FLOPs milestones for AI history – from vacuum tubes to frontier models.
Hero stat: **10^27+ FLOPs**. Data confidence: **mixed**.

- **Interactive**: [View Full Interactive Version (Plotly – Hover/Zoom)](ai-compute-timeline/output/ai_compute_timeline_interactive.html)
- **Interactive**: [AI Compute Timeline](ai-compute-timeline/output/ai_compute_timeline_interactive.html)
- **Static**: [PNG](ai-compute-timeline/output/ai_compute_timeline_highres.png) | [SVG](ai-compute-timeline/output/ai_compute_timeline.svg)
- **Data**: [ai-compute-timeline/data/ai_milestones.csv](ai-compute-timeline/data/ai_milestones.csv)
- **Metadata**: [ai-compute-timeline/data/meta.json](ai-compute-timeline/data/meta.json)
- **Details**: [ai-compute-timeline/](ai-compute-timeline/)

![AI Compute Timeline](ai-compute-timeline/output/ai_compute_timeline_highres.png)

*Zoom recommended for 2010+ frontier cluster (10²⁴–10²⁷+ range).*

---

## 2. Accelerating Paradigms in Computing & Connectivity (1957–2030+)
## 2. Adoption Timeline

Time to ~50M users adoption – shows exponential compression from years to days.
Time-to-scale proxies across computing, connectivity, mobile, cloud, and AI paradigms.

- **Interactive**: [View Full Interactive Version (Plotly – Hover/Zoom)](adoption-timeline/output/adoption_timeline_interactive.html)
Hero stat: **60x faster**. Data confidence: **mixed**.

- **Interactive**: [Adoption Timeline](adoption-timeline/output/adoption_timeline_interactive.html)
- **Static**: [PNG](adoption-timeline/output/adoption_timeline_highres.png) | [SVG](adoption-timeline/output/adoption_timeline.svg)
- **Data**: [adoption-timeline/data/tech_adoption.csv](adoption-timeline/data/tech_adoption.csv)
- **Metadata**: [adoption-timeline/data/meta.json](adoption-timeline/data/meta.json)
- **Details**: [adoption-timeline/](adoption-timeline/)

![Adoption Timeline](adoption-timeline/output/adoption_timeline_highres.png)

*From ~10 years (1957) to ~60 days (2022) – a 60x+ acceleration, with AI pushing toward near-instant scaling.*

---

## 3. Energetic Scaling: Biology vs. Technology
## 3. Energetic Scaling

Neural efficiency vs. body size (Biology) and compute efficiency vs. time (Tech) – both reveal power laws.
Biology, hardware efficiency, AI training compute, and foraging energetics compared with clean source datasets.

- **Interactive**: [View Full Interactive Version (Plotly – Hover/Zoom)](energetic-scaling/output/energetic_scaling_interactive.html)
Hero stat: **10^6x+ efficiency**. Data confidence: **mixed**.

- **Interactive**: [Energetic Scaling](energetic-scaling/output/energetic_scaling_interactive.html)
- **Static**: [PNG](energetic-scaling/output/energetic_scaling_highres.png) | [SVG](energetic-scaling/output/energetic_scaling.svg)
- **Data**: [energetic-scaling/data/scaling_data.csv](energetic-scaling/data/scaling_data.csv)
- **Metadata**: [energetic-scaling/data/meta.json](energetic-scaling/data/meta.json)
- **Details**: [energetic-scaling/](energetic-scaling/)

![Energetic Scaling](energetic-scaling/output/energetic_scaling_highres.png)

*Humans are the biological outlier (EQ~7); AI is the technological outlier (75 quadrillion-fold compute/$ since 1939).*

---

## 4. Scaling Civilization: Energy, Coordination, Memory, Replication
## 4. Civilization Scaling

Multi-lane log-time timeline showing how four fundamental metrics have scaled from 1M years ago to 2030+.
Five civilizational lanes: energy, coordination, memory, replication, and latency over log-time.

- **Interactive**: [View Full Interactive Version (Plotly – Hover/Zoom)](civilization-scaling/output/civilization_scaling_interactive.html)
Hero stat: **5 lanes**. Data confidence: **mixed**.

- **Interactive**: [Civilization Scaling](civilization-scaling/output/civilization_scaling_interactive.html)
- **Static**: [PNG](civilization-scaling/output/civilization_scaling_highres.png) | [SVG](civilization-scaling/output/civilization_scaling.svg)
- **Data**: [civilization-scaling/data/civilization_metrics.csv](civilization-scaling/data/civilization_metrics.csv)
- **Metadata**: [civilization-scaling/data/meta.json](civilization-scaling/data/meta.json)
- **Details**: [civilization-scaling/](civilization-scaling/)

![Civilization Scaling](civilization-scaling/output/civilization_scaling_highres.png)

*Log-time compresses ~99% of human existence (pre-writing) into the left side, expanding modern acceleration on the right. Phase flips (Fire → Agriculture → Writing → Printing → Internet → AI) stack to enable exponential progress.*

---

## 5. Energy Leverage per Person (NEW)
## 5. Energy Leverage

How much total energy does an average human command compared to the metabolic baseline (~114 W)?
Per-person energy command relative to the metabolic baseline, with period anchors labeled explicitly.

- **Interactive**: [View Full Interactive Version (Plotly – Hover/Zoom)](energy-leverage-per-person/output/energy_leverage_interactive.html)
Hero stat: **17x body energy**. Data confidence: **high**.

- **Interactive**: [Energy Leverage](energy-leverage-per-person/output/energy_leverage_interactive.html)
- **Static**: [PNG](energy-leverage-per-person/output/energy_leverage_highres.png) | [SVG](energy-leverage-per-person/output/energy_leverage.svg)
- **Data**: [energy-leverage-per-person/data/energy_leverage_datapoints.csv](energy-leverage-per-person/data/energy_leverage_datapoints.csv)
- **Metadata**: [energy-leverage-per-person/data/meta.json](energy-leverage-per-person/data/meta.json)
- **Details**: [energy-leverage-per-person/](energy-leverage-per-person/)

![Energy Leverage](energy-leverage-per-person/output/energy_leverage_highres.png)
---

## 6. Model Sizes

*Humans went from ~1–2× body energy (foragers) to ~17× body energy (modern). The post-1750 coal/steam and post-1950 oil/electricity jumps dominate the visual story.*
Language model parameter counts over time, separating disclosed counts from estimates and unreleased projections.

Hero stat: **1.5B -> 5T params**. Data confidence: **speculative**.

- **Interactive**: [Model Sizes](model-sizes/output/model_sizes_interactive.html)
- **Static**: [PNG](model-sizes/output/model_sizes_highres.png) | [SVG](model-sizes/output/model_sizes.svg)
- **Data**: [model-sizes/data/llm_model_sizes.csv](model-sizes/data/llm_model_sizes.csv)
- **Metadata**: [model-sizes/data/meta.json](model-sizes/data/meta.json)
- **Details**: [model-sizes/](model-sizes/)

---

## Why These Plots?
## 7. AI Benchmark Progress

| Timeline | Shows | Trend |
|----------|-------|-------|
| **Compute** | Training FLOPs (10⁰ → 10²⁷) | Exponential **growth** ↑ |
| **Adoption** | Time to 50M users | Exponential **compression** ↓ |
| **Energetic** | Neurons/kg & cps/$ | **Power laws** (log-log linear) |
| **Civilization** | Energy/Coord/Memory/Repl | **Stacking infrastructure** layers |
| **Energy Leverage** | Watts/person vs metabolic | **~17× body energy** (modern) |
Benchmark progress against human baselines across knowledge, coding, software engineering, and reasoning tasks.

Together they illustrate:
- **Compute**: Raw exponential growth enabling AI scale
- **Adoption**: Ecosystem acceleration compressing timelines
- **Energetic**: Fundamental scaling rules – humans and AI as outliers
- **Civilization**: How infrastructure layers (fire → writing → internet → AI) enable each successive leap
- **Energy Leverage**: Per-person energy command from foragers (~2×) to modern (~17×)
Hero stat: **4 benchmark lanes**. Data confidence: **mixed**.

Inspired by Kurzweil, [Epoch AI](https://epochai.org/), [Statista](https://www.statista.com/), [Asymco](http://www.asymco.com/), Herculano-Houzel (neuronal scaling), Kleiber (metabolic 0.75), Kaplan/Charnov (LHT/OFT).
- **Interactive**: [AI Benchmark Progress](ai-benchmark-progress/output/benchmark_progress_interactive.html)
- **Static**: [PNG](ai-benchmark-progress/output/benchmark_progress_highres.png) | [SVG](ai-benchmark-progress/output/benchmark_progress.svg)
- **Data**: [ai-benchmark-progress/data/benchmark_data.csv](ai-benchmark-progress/data/benchmark_data.csv)
- **Metadata**: [ai-benchmark-progress/data/meta.json](ai-benchmark-progress/data/meta.json)
- **Details**: [ai-benchmark-progress/](ai-benchmark-progress/)

---

## Repository Structure
## 8. Cost to Train

Each plot follows a standardized structure:
Training cost, FLOPs, and capability over time, showing the efficiency paradox at the frontier.

```
<plot-name>/
├── index.html # Interactive page (uses shared/site.css)
├── data/
│ ├── <slug>.csv # Source data
│ └── meta.json # Metadata: title, description, fields, sources
├── output/
│ ├── *_interactive.html # Plotly interactive chart
│ ├── *_highres.png # High-res PNG export
│ └── *.svg # SVG vector export
├── src/
│ ├── *.py # Matplotlib static generator
│ └── *_plotly.py # Plotly interactive generator
└── README.md
```

### Shared Assets
Hero stat: **$/FLOP collapse**. Data confidence: **mixed**.

- `shared/site.css` – Common styles for all pages
- `shared/site.js` – Navigation bar injection
- `scripts/validate_all.py` – Validate all plots (run: `python scripts/validate_all.py`)
- **Interactive**: [Cost to Train](cost-to-train/output/cost_to_train_interactive.html)
- **Static**: [PNG](cost-to-train/output/cost_to_train_highres.png) | [SVG](cost-to-train/output/cost_to_train.svg)
- **Data**: [cost-to-train/data/training_costs.csv](cost-to-train/data/training_costs.csv)
- **Metadata**: [cost-to-train/data/meta.json](cost-to-train/data/meta.json)
- **Details**: [cost-to-train/](cost-to-train/)

---

## Development
## 9. Unified Dashboard

### Install dependencies
A synchronized overview of the atlas inventory using the same manifest as the homepage, README, build, and validator.

```bash
pip install -r requirements.txt
```
Hero stat: **9 atlas entries**. Data confidence: **mixed**.

### Regenerate all plots
- **Interactive**: [Unified Dashboard](dashboard/index.html)
- **Data**: [plots_manifest.json](plots_manifest.json)
- **Metadata**: [plots_manifest.json](plots_manifest.json)
- **Details**: [dashboard/](dashboard/)

```bash
python build_all.py
```

### Regenerate a specific plot
---

## Data Contracts

```bash
cd <plot-name>/src
python <slug>.py # Static PNG/SVG
python <slug>_plotly.py # Interactive HTML
- `ai-compute-timeline/data/ai_milestones.csv` uses normalized fields: `year,event,category,value_numeric,value_low,value_high,value_unit,estimate_status,source_id,confidence,display_label,notes`.
- `adoption-timeline/data/tech_adoption.csv` includes `adoption_metric_type`, `comparability_level`, `source_id`, `confidence`, and notes so unlike adoption proxies are not treated as perfectly comparable.
- Energetic Scaling keeps comparison-level data in `scaling_data.csv` and splits clean source contracts into `biology_neural_scaling.csv`, `hardware_efficiency.csv`, `ai_training_flops.csv`, and `foraging_lht.csv`.

## Repository Structure

Each plot should follow this structure:

```text
<plot-name>/
├── index.html
├── data/
│ ├── <slug>.csv
│ └── meta.json
├── output/
│ ├── *_interactive.html
│ ├── *_highres.png
│ └── *.svg
├── src/
│ ├── *.py
│ └── *_plotly.py
└── README.md
```

### Validate all plots
## Development

```bash
python -m pip install -r requirements.txt
python build_all.py
python scripts/generate_homepage.py
python scripts/generate_readme_links.py
python scripts/validate_all.py
python scripts/check_links.py
python scripts/check_accessibility_static.py
```

---
## Adding a New Plot

1. Create the standard plot directory structure.
2. Add data, metadata, generator scripts, output paths, and README.
3. Add the entry to `plots_manifest.json` with `status: "draft"` until outputs and sources pass validation.
4. Run the build, generators, validators, link checker, and accessibility checker.
5. Change `status` to `"published"` only when the plot should appear on the homepage and dashboard.

## Contributing
## Deployment

Ideas, new milestones, or bug reports welcome! See [CONTRIBUTING.md](CONTRIBUTING.md).
GitHub Pages deploys should run the same validation commands in CI before publishing. A failed build, broken relative link, missing alt text, stale output, or manifest mismatch should block deployment.

## License

Expand Down
29 changes: 23 additions & 6 deletions adoption-timeline/data/meta.json
Original file line number Diff line number Diff line change
@@ -1,34 +1,51 @@
{
"title": "Adoption Timeline",
"description": "Time to ~50M users adoption, showing exponential compression from years to days (1957-2026).",
"description": "Time-to-scale proxies across technology paradigms. The chart labels metric type and comparability so FORTRAN, ARPANET, iPhone, ChatGPT, and agentic tools are not treated as perfectly equivalent.",
"fields": {
"Year": "Launch year of technology/product",
"Year": "Launch year of technology or product",
"Event": "Name of technology or product",
"Category": "Tech category (Hardware, Software, Internet/Web, Mobile, Social/Apps, Cloud, AI/Agentic)",
"Days_to_Adoption": "Days to reach ~50M users or equivalent adoption milestone",
"Impact": "Historical impact level (Transformative, High, Medium, Speculative)"
"Category": "Technology category",
"Days_to_Adoption": "Days to reach the stated proxy scale",
"Impact": "Historical impact level",
"adoption_metric_type": "Metric type: users, developers, devices, accounts, installs, or organizations",
"comparability_level": "strict_proxy, comparable_proxy, rough_analogy, or projection",
"source_id": "Source identifier for auditable rows",
"confidence": "Qualitative confidence",
"comparability_notes": "Why the proxy is or is not comparable with other rows",
"notes": "Additional audit notes"
},
"sources": [
{
"id": "statista",
"name": "Statista",
"url": "https://www.statista.com/",
"accessed": "2026-01",
"notes": "User growth and adoption data"
},
{
"id": "asymco",
"name": "Asymco",
"url": "http://www.asymco.com/",
"accessed": "2026-01",
"notes": "Historical technology adoption curves"
},
{
"id": "epoch",
"name": "Epoch AI",
"url": "https://epochai.org/",
"accessed": "2026-01",
"notes": "AI adoption metrics"
},
{
"id": "source_review_needed",
"name": "Source review needed",
"url": "https://github.com/mschwar/plots",
"accessed": "2026-04",
"notes": "Projection retained for chart continuity but flagged for source review"
}
],
"transformations": "Log10(days) for y-axis. Midpoints used for date ranges. Pre-1990 estimates approximate.",
"transformations": "Log10(days) for y-axis. Metric types and comparability levels control marker shape. Trend line is a visual guide, not a causal model.",
"created": "2026-01",
"last_updated": "2026-04-24",
"author": "mschwar"
}
Loading
Loading