Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@ contact_links:
url: https://github.com/facebook/zstd
about: Check the official zstd documentation before opening new issues.
- name: Plugin Documentation
url: https://github.com/landamessenger/zstandard/wiki
url: https://github.com/vypdev/zstandard/wiki
about: Check the plugin/cli documentation before opening new issues.
2 changes: 1 addition & 1 deletion .github/workflows/push_checks_android.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
- '!develop'
- '!release/**'
- '!hotfix/**'
# - '!master'
- '!master'
# paths:
# - "zstandard_android/**"

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/push_checks_cli.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
- '!develop'
- '!release/**'
- '!hotfix/**'
# - '!master'
- '!master'
#paths:
# - "zstandard_cli/**"

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/push_checks_ios.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
- '!develop'
- '!release/**'
- '!hotfix/**'
# - '!master'
- '!master'
#paths:
# - "zstandard_ios/**"

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/push_checks_linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
- '!develop'
- '!release/**'
- '!hotfix/**'
# - '!master'
- '!master'
#paths:
# - "zstandard_linux/**"

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/push_checks_macos.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
- '!develop'
- '!release/**'
- '!hotfix/**'
# - '!master'
- '!master'
#paths:
# - "zstandard_macos/**"

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/push_checks_platform_interface.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
- '!develop'
- '!release/**'
- '!hotfix/**'
# - '!master'
- '!master'
#paths:
# - "zstandard_platform_interface/**"

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/push_checks_web.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
- '!develop'
- '!release/**'
- '!hotfix/**'
# - '!master'
- '!master'
#paths:
# - "zstandard_web/**"

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/push_checks_windows.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
- '!develop'
- '!release/**'
- '!hotfix/**'
# - '!master'
- '!master'
#paths:
# - "zstandard_windows/**"

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/push_checks_zstandard.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
- '!develop'
- '!release/**'
- '!hotfix/**'
# - '!master'
- '!master'
#paths:
# - "zstandard/**"

Expand Down
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
## 1.4.0 - Zstd Update

- Zstd lib updated
- Multiple symbol resolution bugs fixed
- Unified source of zstd (not one per platform)
- Web de/compression improved

## 1.3.32 - Test Release

- Test release
Expand Down
26 changes: 13 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[![pub package](https://img.shields.io/pub/v/zstandard.svg)](https://pub.dev/packages/zstandard)
[![pub package](https://img.shields.io/pub/v/zstandard_cli.svg)](https://pub.dev/packages/zstandard_cli)
[![codecov](https://codecov.io/gh/landamessenger/zstandard/graph/badge.svg)](https://codecov.io/gh/landamessenger/zstandard)
[![codecov](https://codecov.io/gh/vypdev/zstandard/graph/badge.svg)](https://codecov.io/gh/vypdev/zstandard)

# Zstandard

Expand Down Expand Up @@ -123,27 +123,27 @@ void main() async {

#### macOS

<p align="center"><img width="90%" vspace="10" src="https://github.com/landamessenger/zstandard/raw/master/zstandard_cli/images/macos_compression_sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/landamessenger/zstandard/raw/master/zstandard_cli/images/macos_decompression_sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/vypdev/zstandard/raw/master/zstandard_cli/images/macos_compression_sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/vypdev/zstandard/raw/master/zstandard_cli/images/macos_decompression_sample.png"></p>

#### Windows

<p align="center"><img width="90%" vspace="10" src="https://github.com/landamessenger/zstandard/raw/master/zstandard_cli/images/windows_compression_sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/landamessenger/zstandard/raw/master/zstandard_cli/images/windows_decompression_sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/vypdev/zstandard/raw/master/zstandard_cli/images/windows_compression_sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/vypdev/zstandard/raw/master/zstandard_cli/images/windows_decompression_sample.png"></p>

#### Linux

<p align="center"><img width="90%" vspace="10" src="https://github.com/landamessenger/zstandard/raw/master/zstandard_cli/images/linux_compression_sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/landamessenger/zstandard/raw/master/zstandard_cli/images/linux_decompression_sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/vypdev/zstandard/raw/master/zstandard_cli/images/linux_compression_sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/vypdev/zstandard/raw/master/zstandard_cli/images/linux_decompression_sample.png"></p>

#### Flutter Plugin (Android, iOS, macOS, Web, Windows, Linux)

<p align="center"><img width="50%" vspace="10" src="https://github.com/landamessenger/zstandard/raw/master/zstandard_android/images/sample.png"></p>
<p align="center"><img width="50%" vspace="10" src="https://github.com/landamessenger/zstandard/raw/master/zstandard_ios/images/sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/landamessenger/zstandard/raw/master/zstandard_macos/images/sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/landamessenger/zstandard/raw/master/zstandard_web/images/sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/landamessenger/zstandard/raw/master/zstandard_windows/images/sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/landamessenger/zstandard/raw/master/zstandard_linux/images/sample.png"></p>
<p align="center"><img width="50%" vspace="10" src="https://github.com/vypdev/zstandard/raw/master/zstandard_android/images/sample.png"></p>
<p align="center"><img width="50%" vspace="10" src="https://github.com/vypdev/zstandard/raw/master/zstandard_ios/images/sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/vypdev/zstandard/raw/master/zstandard_macos/images/sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/vypdev/zstandard/raw/master/zstandard_web/images/sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/vypdev/zstandard/raw/master/zstandard_windows/images/sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/vypdev/zstandard/raw/master/zstandard_linux/images/sample.png"></p>

---

Expand Down
2 changes: 1 addition & 1 deletion docs/architecture/web-implementation.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ zstd is built with Emscripten. High-level steps:
3. Run `emcc` on the zstd sources with flags for WASM, exported functions (`ZSTD_compress`, `ZSTD_decompress`, `ZSTD_compressBound`, `ZSTD_getFrameContentSize`, `malloc`, `free`), and output name.
4. Add the wrapper functions `compressData` and `decompressData` in `zstd.js` (or a separate script) that allocate buffers, call the exported C functions, and return the result or null.

Detailed commands and wrapper code are in the [zstandard_web README](https://github.com/landamessenger/zstandard/tree/master/zstandard_web) and in [Platforms — Web](../platforms/web.md).
Detailed commands and wrapper code are in the [zstandard_web README](https://github.com/vypdev/zstandard/tree/master/zstandard_web) and in [Platforms — Web](../platforms/web.md).

## Small Data Handling

Expand Down
4 changes: 2 additions & 2 deletions docs/development/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ When changing the platform interface, ensure all platform implementations are up

## Questions

- Open a [GitHub Issue](https://github.com/landamessenger/zstandard/issues) for bugs or feature requests.
- Use [GitHub Discussions](https://github.com/landamessenger/zstandard/discussions) for questions and ideas.
- Open a [GitHub Issue](https://github.com/vypdev/zstandard/issues) for bugs or feature requests.
- Use [GitHub Discussions](https://github.com/vypdev/zstandard/discussions) for questions and ideas.

Thank you for contributing.
8 changes: 4 additions & 4 deletions docs/development/ci-cd.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ The repository uses **GitHub Actions** for:
- **Release workflow**: Version bumping, building precompiled CLI libraries, tagging, and publishing to pub.dev.
- **Hotfix workflow**: Expedited fixes and releases when needed.

Workflows are in [`.github/workflows/`](https://github.com/landamessenger/zstandard/tree/master/.github/workflows).
Workflows are in [`.github/workflows/`](https://github.com/vypdev/zstandard/tree/master/.github/workflows).

## Push check workflows

Expand Down Expand Up @@ -40,7 +40,7 @@ Each package has a dedicated workflow that runs on push to non-protected branche

## Release workflow

**File**: [`.github/workflows/release_workflow.yml`](https://github.com/landamessenger/zstandard/blob/master/.github/workflows/release_workflow.yml)
**File**: [`.github/workflows/release_workflow.yml`](https://github.com/vypdev/zstandard/blob/master/.github/workflows/release_workflow.yml)

Triggered manually (**workflow_dispatch**) with inputs such as version, title, changelog, and optional issue reference.

Expand All @@ -58,7 +58,7 @@ The workflow uses **self-hosted** runners for macOS, Linux, and Windows to build

## Hotfix workflow

**File**: [`.github/workflows/hotfix_workflow.yml`](https://github.com/landamessenger/zstandard/blob/master/.github/workflows/hotfix_workflow.yml)
**File**: [`.github/workflows/hotfix_workflow.yml`](https://github.com/vypdev/zstandard/blob/master/.github/workflows/hotfix_workflow.yml)

Used for expedited fixes (e.g. security or critical bugs). Typically triggered manually and may skip some steps or use a shorter path to release. See the workflow file and team docs for details.

Expand All @@ -73,7 +73,7 @@ To mimic CI locally:

## Build automation scripts

Scripts under [**scripts/**](https://github.com/landamessenger/zstandard/tree/master/scripts) help build native libraries and run tests locally:
Scripts under [**scripts/**](https://github.com/vypdev/zstandard/tree/master/scripts) help build native libraries and run tests locally:

- `build_macos.sh`, `build_linux.sh`, `build_windows.bat`: Build precompiled zstd libraries for the CLI.
- `build_android.sh`, `build_ios.sh`: Build or prepare the Android/iOS plugin.
Expand Down
4 changes: 2 additions & 2 deletions docs/development/setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,14 @@ Optional, for native work:
## Clone the Repository

```bash
git clone https://github.com/landamessenger/zstandard.git
git clone https://github.com/vypdev/zstandard.git
cd zstandard
```

If you use a fork:

```bash
git remote add upstream https://github.com/landamessenger/zstandard.git
git remote add upstream https://github.com/vypdev/zstandard.git
```

## Get Dependencies
Expand Down
2 changes: 1 addition & 1 deletion docs/guides/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ No extra steps are required for **Android, iOS, macOS, Windows, or Linux**; the

For **Flutter web**, you must add two assets and include a script:

1. Copy **zstd.js** and **zstd.wasm** from the [zstandard_web](https://github.com/landamessenger/zstandard/tree/master/zstandard_web) package (e.g. from its `blob/` or as documented there) into your app’s **web/** directory (e.g. `web/zstd.js`, `web/zstd.wasm`).
1. Copy **zstd.js** and **zstd.wasm** from the [zstandard_web](https://github.com/vypdev/zstandard/tree/master/zstandard_web) package (e.g. from its `blob/` or as documented there) into your app’s **web/** directory (e.g. `web/zstd.js`, `web/zstd.wasm`).

2. In **web/index.html**, include the script before your app loads:

Expand Down
2 changes: 1 addition & 1 deletion docs/platforms/web.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ If you need to rebuild the WebAssembly artifacts:
- Exported functions: `ZSTD_compress`, `ZSTD_decompress`, `ZSTD_compressBound`, `ZSTD_getFrameContentSize`, `malloc`, `free`
4. Add the wrapper functions `compressData` and `decompressData` in `zstd.js` (or a separate script) that allocate buffers, call the C functions, and return the result or null.

Detailed commands and wrapper code are in the [zstandard_web README](https://github.com/landamessenger/zstandard/tree/master/zstandard_web).
Detailed commands and wrapper code are in the [zstandard_web README](https://github.com/vypdev/zstandard/tree/master/zstandard_web).

## Small Data Behavior

Expand Down
49 changes: 40 additions & 9 deletions scripts/build_web_wasm.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
ZSTD_ROOT="$ROOT/zstd"
OUT_BLOB="$ROOT/zstandard_web/blob"
OUT_EXAMPLE_WEB="$ROOT/zstandard_web/example/web"
OUT_ZSTANDARD_EXAMPLE_WEB="$ROOT/zstandard/example/web"

if [[ ! -d "$ZSTD_ROOT" || ! -f "$ZSTD_ROOT/zstd.h" ]]; then
echo "Error: Canonical zstd source not found at $ZSTD_ROOT (expected zstd.h)."
Expand Down Expand Up @@ -51,6 +52,9 @@ emcc -O3 \
-s EXPORT_NAME="zstdWasmModule" \
-s EXPORTED_FUNCTIONS="['_ZSTD_compress','_ZSTD_decompress','_malloc','_free','_ZSTD_getFrameContentSize','_ZSTD_compressBound']" \
-s EXPORTED_RUNTIME_METHODS="['HEAPU8']" \
-s INITIAL_MEMORY=134217728 \
-s ALLOW_MEMORY_GROWTH=1 \
-s MAXIMUM_MEMORY=2147483648 \
-o zstd_generated.js

if [[ ! -f zstd_generated.js || ! -f zstd_generated.wasm ]]; then
Expand All @@ -61,20 +65,37 @@ fi
# Append the compressData/decompressData wrappers required by the web plugin (see zstandard_web/README.md).
cat >> zstd_generated.js << 'WRAPPER_JS'

function compressData(inputData, compressionLevel) {
// Promise that resolves when the module is ready
let moduleReady = new Promise((resolve) => {
if (typeof Module !== 'undefined' && Module.calledRun) {
// Module already initialized
resolve();
} else {
// Wait for module initialization
const originalOnRuntimeInitialized = Module.onRuntimeInitialized || function() {};
Module.onRuntimeInitialized = function() {
originalOnRuntimeInitialized();
resolve();
};
}
});

async function compressData(inputData, compressionLevel) {
await moduleReady;

let inputPtr = Module._malloc(inputData.length);
Module.HEAPU8.set(inputData, inputPtr);

let outputBufferSize = Module._ZSTD_compressBound(inputData.length);
let outputBufferSize = Number(Module._ZSTD_compressBound(inputData.length));
let outputPtr = Module._malloc(outputBufferSize);

let compressedSize = Module._ZSTD_compress(
let compressedSize = Number(Module._ZSTD_compress(
outputPtr,
outputBufferSize,
inputPtr,
inputData.length,
compressionLevel
);
));

if (compressedSize < 0) {
console.error('Compression error, error code: ', compressedSize);
Expand All @@ -90,11 +111,13 @@ function compressData(inputData, compressionLevel) {
}
}

function decompressData(compressedData) {
async function decompressData(compressedData) {
await moduleReady;

let compressedPtr = Module._malloc(compressedData.length);
Module.HEAPU8.set(compressedData, compressedPtr);

let decompressedSize = Module._ZSTD_getFrameContentSize(compressedPtr, compressedData.length);
let decompressedSize = Number(Module._ZSTD_getFrameContentSize(compressedPtr, compressedData.length));
if (decompressedSize === -1 || decompressedSize === -2) {
console.error('Error in obtaining the original size of the data');
Module._free(compressedPtr);
Expand All @@ -103,12 +126,12 @@ function decompressData(compressedData) {

let decompressedPtr = Module._malloc(decompressedSize);

let resultSize = Module._ZSTD_decompress(
let resultSize = Number(Module._ZSTD_decompress(
decompressedPtr,
decompressedSize,
compressedPtr,
compressedData.length
);
));

if (resultSize < 0) {
console.error('Decompression error, error code: ', resultSize);
Expand All @@ -125,14 +148,22 @@ function decompressData(compressedData) {
}
WRAPPER_JS

mkdir -p "$OUT_BLOB" "$OUT_EXAMPLE_WEB"
mkdir -p "$OUT_BLOB" "$OUT_EXAMPLE_WEB" "$OUT_ZSTANDARD_EXAMPLE_WEB"

# Replace the wasm filename in the generated JS to match what we'll copy
sed -i.bak 's/zstd_generated\.wasm/zstd.wasm/g' zstd_generated.js
rm -f zstd_generated.js.bak

cp zstd_generated.wasm "$OUT_BLOB/zstd.wasm"
cp zstd_generated.wasm "$OUT_EXAMPLE_WEB/zstd.wasm"
cp zstd_generated.wasm "$OUT_ZSTANDARD_EXAMPLE_WEB/zstd.wasm"
cp zstd_generated.js "$OUT_BLOB/zstd.js"
cp zstd_generated.js "$OUT_EXAMPLE_WEB/zstd.js"
cp zstd_generated.js "$OUT_ZSTANDARD_EXAMPLE_WEB/zstd.js"
rm -f "$ZSTD_ROOT/zstd_generated.js" "$ZSTD_ROOT/zstd_generated.wasm"

echo "Done. zstd.js and zstd.wasm have been written to:"
echo " - $OUT_BLOB/"
echo " - $OUT_EXAMPLE_WEB/"
echo " - $OUT_ZSTANDARD_EXAMPLE_WEB/"
echo "Built from the same zstd/ source used by Android, iOS, macOS, Windows, Linux, and CLI."
7 changes: 7 additions & 0 deletions zstandard/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
## 1.4.0 - Zstd Update

- Zstd lib updated
- Multiple symbol resolution bugs fixed
- Unified source of zstd (not one per platform)
- Web de/compression improved

## 1.3.32 - Test Release

- Test release
Expand Down
12 changes: 6 additions & 6 deletions zstandard/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,14 +44,14 @@ void act() async {

Below are examples of the plugin in action across different platforms.

<p align="center"><img width="50%" vspace="10" src="https://github.com/landamessenger/zstandard/raw/master/zstandard_android/images/sample.png"></p>
<p align="center"><img width="50%" vspace="10" src="https://github.com/vypdev/zstandard/raw/master/zstandard_android/images/sample.png"></p>

<p align="center"><img width="50%" vspace="10" src="https://github.com/landamessenger/zstandard/raw/master/zstandard_ios/images/sample.png"></p>
<p align="center"><img width="50%" vspace="10" src="https://github.com/vypdev/zstandard/raw/master/zstandard_ios/images/sample.png"></p>

<p align="center"><img width="90%" vspace="10" src="https://github.com/landamessenger/zstandard/raw/master/zstandard_macos/images/sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/vypdev/zstandard/raw/master/zstandard_macos/images/sample.png"></p>

<p align="center"><img width="90%" vspace="10" src="https://github.com/landamessenger/zstandard/raw/master/zstandard_web/images/sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/vypdev/zstandard/raw/master/zstandard_web/images/sample.png"></p>

<p align="center"><img width="90%" vspace="10" src="https://github.com/landamessenger/zstandard/raw/master/zstandard_windows/images/sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/vypdev/zstandard/raw/master/zstandard_windows/images/sample.png"></p>

<p align="center"><img width="90%" vspace="10" src="https://github.com/landamessenger/zstandard/raw/master/zstandard_linux/images/sample.png"></p>
<p align="center"><img width="90%" vspace="10" src="https://github.com/vypdev/zstandard/raw/master/zstandard_linux/images/sample.png"></p>
Loading
Loading