Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
162 changes: 87 additions & 75 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,36 +1,79 @@
# SQL Converter

SQL Converter is a browser-side tool for converting MariaDB/MySQL `.sql` dump files into SQLite-compatible SQL.
SQL Converter converts MariaDB/MySQL `.sql` dump files into SQLite-compatible SQL in the browser.

Live app: https://dominosaurs.github.io/sql-converter/

## What It Does
## ✨ Why This Exists

- Converts MariaDB/MySQL dump syntax into SQLite-oriented SQL.
- Streams large input files directly from the browser.
- Writes the converted output to a local file without uploading database data.
- Handles common dump features such as data type conversion, quoted identifiers, auto-increment primary keys, indexes, foreign keys, JSON checks, dump directives, and MySQL string escapes.
MariaDB and MySQL dumps often contain syntax that SQLite cannot import directly: backtick identifiers, `AUTO_INCREMENT`, engine options, charset/collation options, MariaDB dump directives, and MySQL-style escaped strings.

## Browser Requirement
SQL Converter rewrites the common parts of those dumps so the output can be imported into SQLite with fewer manual edits.

Large-file conversion uses the File System Access API so the app can stream output directly to disk.
## 🔒 Privacy

Use a browser that supports `showSaveFilePicker`, such as:
Conversion runs locally in your browser.

- The input dump is read from your disk.
- The converted file is written back to your disk.
- The app does not upload your database dump to a server.

## 🌐 Browser Requirement

Large-file conversion uses the File System Access API so output can be streamed directly to disk.

Recommended browsers:

- Chrome
- Edge

Firefox and Safari may not support the required direct-to-disk streaming flow.
Firefox and Safari may not support the required `showSaveFilePicker` API for large-file direct-to-disk conversion.

## 🚀 Usage

1. Open https://dominosaurs.github.io/sql-converter/
2. Select a MariaDB/MySQL `.sql` dump file.
3. Choose where to save the converted SQLite SQL file.
4. Wait for conversion to finish.
5. Import the generated `.sqlite.sql` file with your SQLite client.

## 🔁 Example

MariaDB/MySQL dump input:

```sql
CREATE TABLE `users` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(255) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

INSERT INTO `users` VALUES
(1,'O\'Connor');
```

## Supported Conversion Scope
SQLite output:

The converter is designed for common `mysqldump` / MariaDB dump files.
```sql
CREATE TABLE "users" (
"id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
"name" TEXT NOT NULL
);

INSERT INTO "users" VALUES
(1,'O''Connor');
```

## ✅ Supported Conversion Scope

The converter is designed for common `mysqldump`, MariaDB dump, and phpMyAdmin-style SQL exports.

Currently handled:

- `CREATE TABLE`
- `DROP TABLE`
- `INSERT`
- Backtick identifiers to SQLite double-quoted identifiers
- MariaDB/MySQL integer, decimal, text, blob, date/time, enum/set, and JSON-like column types
- `AUTO_INCREMENT` to SQLite `INTEGER PRIMARY KEY AUTOINCREMENT`
- Table-level primary keys
Expand All @@ -42,87 +85,56 @@ Currently handled:
- MySQL escaped string values in inserts
- MySQL hex literals like `0xDEADBEEF`

Known limits:
Generated string values use standard SQLite single-quoted strings:

- This is not a full SQL parser.
- Stored procedures, triggers, views, generated columns, and complex vendor-specific SQL may need more handling.
- Memory use is bounded by chunks and completed statements, but a single extremely large SQL statement can still be expensive.
```sql
'O''Connor'
```

## Usage
Multi-row inserts are preserved for performance.

1. Open https://dominosaurs.github.io/sql-converter/
2. Select a MariaDB/MySQL `.sql` dump file.
3. Choose the output file location when prompted.
4. Wait for conversion to finish.
5. Import the generated `.sqlite.sql` file with your SQLite client.
## ⚠️ Known Limits

## Local Development
This is a practical dump converter, not a full SQL parser or validator.

This project uses Bun.
Known limits:

```bash
bun install
bun run dev
```
- Stored procedures are not supported.
- Triggers and views may need manual review.
- Generated columns may need manual review.
- Vendor-specific functions may need manual review.
- A single extremely large SQL statement can still be expensive, even though file reading/writing is streamed.
- SQLite client/importer behavior varies. Some GUI tools may have script execution bugs even when the generated SQL is valid SQLite.

Run checks:
## 📥 Import Notes

```bash
bun run lint
bun test
bun run build
```
If your SQLite GUI client fails to import a generated file, test the failing statement with another importer before assuming the SQL is invalid.

Preview production build:
Recommended SQLite CLI import:

```bash
bun run build
bun run preview
sqlite3 output.sqlite < converted.sqlite.sql
```

## Scripts
Or inside SQLite:

- `bun run dev` starts the Vite dev server.
- `bun run build` type-checks and builds the app.
- `bun run lint` runs Knip, Biome, and TypeScript checks.
- `bun run format` applies Biome formatting/fixes.
- `bun test` runs converter tests.
- `bun run preview` previews the production build.
```sql
.read converted.sqlite.sql
```

## Deployment
## 🛠️ Local Development

The app is configured for GitHub Pages at:
This project uses Bun.

```text
https://dominosaurs.github.io/sql-converter/
```bash
bun install
bun run dev
```

Vite uses:
Run checks:

```ts
base: '/sql-converter/'
```bash
bun run lint
bun test
bun run build
```

GitHub Actions workflows:

- `.github/workflows/ci.yml` runs lint, tests, and build.
- `.github/workflows/deploy-pages.yml` builds and deploys `dist` to GitHub Pages.

In GitHub repository settings, configure Pages source as **GitHub Actions**.

## Reporting Issues

If a dump fails to import after conversion, open an issue:

https://github.com/dominosaurs/sql-converter/issues

Include:

- The SQLite error message.
- The source SQL statement that caused it.
- The converted output statement.
- Whether the dump came from MariaDB, MySQL, phpMyAdmin, or another tool.

## License

MIT
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -43,5 +43,5 @@
"lint": "knip && biome check --write . && tsc --noEmit"
},
"type": "module",
"version": "0.0.2"
"version": "0.0.3"
}
107 changes: 92 additions & 15 deletions src/app.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -36,22 +36,35 @@ type ConversionState =
| 'error'
| 'cancelled'

const DEFAULT_STATUS =
'Choose a MariaDB .sql dump, then convert it to a SQLite import file.'
const UNSUPPORTED_BROWSER_STATUS =
'This browser cannot stream directly to disk. Use Chrome or Edge before selecting a large SQL file.'

export default function App() {
const inputFileRef = useRef<null | HTMLInputElement>(null)
const abortControllerRef = useRef<AbortController | null>(null)
const isBrowserSupported = supportsSaveFilePicker()
const [warnings, setWarnings] = useState<ConversionWarning[]>([])
const [progress, setProgress] = useState<ConversionProgress | null>(null)
const [selectedFileName, setSelectedFileName] = useState('')
const [selectedFileSize, setSelectedFileSize] = useState(0)
const [outputFileName, setOutputFileName] = useState('')
const [status, setStatus] = useState(
'Choose a MariaDB .sql dump, then convert it to a SQLite import file.',
const [status, setStatus] = useState(() =>
isBrowserSupported ? DEFAULT_STATUS : UNSUPPORTED_BROWSER_STATUS,
)
const [conversionState, setConversionState] = useState<ConversionState>(
() => (isBrowserSupported ? 'idle' : 'error'),
)
const [conversionState, setConversionState] =
useState<ConversionState>('idle')
const [isConverting, setIsConverting] = useState(false)

const handleConvertClick = () => {
if (!isBrowserSupported) {
setStatus(UNSUPPORTED_BROWSER_STATUS)
setConversionState('error')
return
}

const file = inputFileRef.current?.files?.[0]

if (!file) {
Expand All @@ -63,6 +76,12 @@ export default function App() {
}

const handleFileChange = (file: File | undefined) => {
if (!isBrowserSupported) {
setStatus(UNSUPPORTED_BROWSER_STATUS)
setConversionState('error')
return
}

setSelectedFileName(file?.name ?? '')
setSelectedFileSize(file?.size ?? 0)

Expand All @@ -78,9 +97,7 @@ export default function App() {
.showSaveFilePicker

if (!saveFilePicker) {
setStatus(
'This browser cannot stream directly to disk. Use Chrome or Edge for large 1GB files.',
)
setStatus(UNSUPPORTED_BROWSER_STATUS)
setConversionState('error')
return
}
Expand Down Expand Up @@ -168,9 +185,9 @@ export default function App() {
setSelectedFileSize(0)
setOutputFileName('')
setStatus(
'Choose a MariaDB .sql dump, then convert it to a SQLite import file.',
isBrowserSupported ? DEFAULT_STATUS : UNSUPPORTED_BROWSER_STATUS,
)
setConversionState('idle')
setConversionState(isBrowserSupported ? 'idle' : 'error')
}

const isFinished =
Expand Down Expand Up @@ -212,10 +229,12 @@ export default function App() {
</p>
</div>

<label className="file-drop">
<label
className={`file-drop ${isBrowserSupported ? '' : 'file-drop-disabled'}`}
>
<input
accept=".sql"
disabled={isConverting}
disabled={isConverting || !isBrowserSupported}
onChange={event =>
handleFileChange(event.currentTarget.files?.[0])
}
Expand All @@ -231,13 +250,14 @@ export default function App() {
<div className="action-row">
<button
className="primary-action"
disabled={isConverting}
disabled={isConverting || !isBrowserSupported}
onClick={handleConvertClick}
type="button"
>
{isConverting
? 'Converting...'
: 'Convert to SQLite SQL'}
{getPrimaryActionLabel(
isBrowserSupported,
isConverting,
)}
</button>

{isConverting ? (
Expand Down Expand Up @@ -355,6 +375,46 @@ export default function App() {
</article>
</section>

<section aria-labelledby="sample-title" className="sample-section">
<div className="sample-heading">
<p className="card-kicker">Example conversion</p>
<h2 id="sample-title">MariaDB dump SQL into SQLite SQL</h2>
<p>
The converter keeps your data and rewrites common dump
syntax so SQLite can import it.
</p>
</div>

<div className="sample-grid">
<article>
<h3>MariaDB / MySQL input</h3>
<pre>
<code>{`CREATE TABLE \`users\` (
\`id\` bigint unsigned NOT NULL AUTO_INCREMENT,
\`name\` varchar(255) NOT NULL,
PRIMARY KEY (\`id\`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

INSERT INTO \`users\` VALUES
(1,'O\\'Connor');`}</code>
</pre>
</article>

<article>
<h3>SQLite output</h3>
<pre>
<code>{`CREATE TABLE "users" (
"id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
"name" TEXT NOT NULL
);

INSERT INTO "users" VALUES
(1,'O''Connor');`}</code>
</pre>
</article>
</div>
</section>

{warnings.length > 0 ? (
<section
aria-labelledby="warnings-title"
Expand Down Expand Up @@ -420,6 +480,23 @@ function formatBytes(bytes: number): string {
return `${(bytes / 1024 ** 3).toFixed(2)} GB`
}

function supportsSaveFilePicker(): boolean {
return (
typeof window !== 'undefined' &&
typeof (window as WindowWithSaveFilePicker).showSaveFilePicker ===
'function'
)
}

function getPrimaryActionLabel(
isBrowserSupported: boolean,
isConverting: boolean,
): string {
if (!isBrowserSupported) return 'Browser not supported'
if (isConverting) return 'Converting...'
return 'Convert to SQLite SQL'
}

function getProgressPercent(bytesRead: number, totalBytes: number): number {
if (totalBytes <= 0) return 0
return Math.min(100, Math.round((bytesRead / totalBytes) * 100))
Expand Down
Loading