Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ jobs:
fetch-depth: 0

- name: Install uv (official Astral action)
uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
with:
# Update this as needed:
version: "0.10.2"
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
fetch-depth: 0

- name: Install uv (official Astral action)
uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
with:
version: "0.10.2"
enable-cache: true
Expand Down Expand Up @@ -52,7 +52,7 @@ jobs:
# Pinned to commit SHA because this is a single-maintainer third-party action
# that runs alongside a publish step with write permissions. Update the SHA
# together with the version comment when bumping.
uses: softprops/action-gh-release@3bb12739c298aeb8a4eeaf626c5b8d85266b0e65 # v2.6.2
uses: softprops/action-gh-release@b4309332981a82ec1c5618f44dd2e27cc8bfbfda # v3.0.0
with:
generate_release_notes: true
prerelease: ${{ contains(github.ref_name, '-') }}
47 changes: 43 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,16 @@ It is simply a few functions and tricks that have repeatedly shown value in vari
projects. The goal is not to give a comprehensive suite of utilities but simply to
complement the standard libraries and fill in a few gaps.

✨ **NEW:** **Version 3.0** is out and has additions and updates for Python 3.10-3.13! ✨
✨ **NEW:** **Version 3.1** adds `atomic_write_text()`/`atomic_write_bytes()`, exposes
`__version__`, and supports Python 3.10-3.14. ✨

## Key Features

- **Atomic file operations** with handling of parent directories and backups.
This is essential for thread safety and good hygiene so partial or corrupt outputs are
never present in final file locations, even in case a program crashes.
See `atomic_output_file()`, `copyfile_atomic()`.
See `atomic_output_file()`, `atomic_write_text()`, `atomic_write_bytes()`,
`copyfile_atomic()`.

- **Abbreviate and quote strings**, which is useful for logging a clean way.
See `abbrev_str()`, `single_line()`, `quote_if_needed()`.
Expand Down Expand Up @@ -43,6 +45,27 @@ The libs are all small so see pydoc strings or code for full docs.
> that has some extra functions for pretty, human-readable outputs for objects, sizes,
> times and dates, etc.

## Using strif with LLM Agents

Strif is handy for code that generates files, which is increasingly often AI agent code.

- **Atomic writes for streamed or generated output.** If a generation is interrupted or
crashes mid-write, you never leave a truncated or corrupt file in its final location.
`atomic_write_text("out.md", content)` is a one-liner for the common case.

- **Content hashing for caching and dedup.** Use `hash_file()` or `hash_string()` to key
a cache on file contents, or `file_mtime_hash()` for a fast (content-free) cache key.

- **Sortable, readable run ids.** `new_timestamped_uid()` gives ids that sort by creation
time, which is convenient for logs and scratch directories.

```python
from strif import atomic_write_text

# Safe even if the process dies partway through writing:
atomic_write_text("some-dir/output.md", generated_text, make_parents=True)
```

## Installation

```sh
Expand Down Expand Up @@ -178,6 +201,13 @@ pip install strif
Moves a file to a new location, automatically creating parent directories and
optionally keeping a backup of the destination if it already exists.

- **`atomic_write_text(dest_path, text, make_parents=False, backup_suffix=None,
encoding='utf-8')`** and **`atomic_write_bytes(dest_path, data, make_parents=False,
backup_suffix=None)`**

Convenience wrappers around `atomic_output_file()` for the common case of writing a
whole string or bytes value atomically in a single call.

For example, it is generally a good idea to wrap an `open()` call with
`atomic_output_file()`:

Expand All @@ -187,6 +217,12 @@ with atomic_output_file("some-dir/my-final-output.txt") as temp_target:
f.write("some contents")
```

Or, for the common whole-value case, just:

```python
atomic_write_text("some-dir/my-final-output.txt", "some contents")
```

And this can (and in most cases should) be used in place of `shutil.copyfile`:

```python
Expand All @@ -204,7 +240,7 @@ There are also some handy additional options:
with atomic_output_file("some-dir/my-final-output.txt",
make_parents=True, backup_suffix=".old.{timestamp}") as temp_target:
with open(temp_target, "w") as f:
sf.write("some contents")
f.write("some contents")
```

This creates parent folders as needed (a major convenience).
Expand Down Expand Up @@ -310,14 +346,17 @@ Examples:

## Multiple String Replacements

`Insertion` and `Replacement` are `NamedTuple`s, so you can use named fields
(`Insertion(offset, text)`, `Replacement(start, end, text)`) or plain positional tuples.

- **`insert_multiple(text: str, insertions: list[Insertion]) -> str`**

Insert multiple strings into `text` at the given offsets, at once.

- **`replace_multiple(text: str, replacements: list[Replacement]) -> str`**

Replace multiple substrings in `text` with new strings, simultaneously.
The replacements are a list of tuples (start_offset, end_offset, new_string).
Each `Replacement` is `(start_offset, end_offset, new_string)`.

## FAQ

Expand Down
4 changes: 4 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,10 @@ bump = true
# The source location for the package.
packages = ["src/strif"]

[tool.hatch.build.targets.sdist]
# Keep agent/tooling state out of the published source distribution.
exclude = [".claude", ".tbd", ".github", ".copier-answers.yml", "attic"]


# ---- Settings ----

Expand Down
20 changes: 16 additions & 4 deletions src/strif/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
from importlib.metadata import PackageNotFoundError, version

__all__ = ( # noqa: F405
"__version__",
# atomic_var.py
"AtomicVar",
# strif.py
Expand All @@ -11,6 +14,7 @@
"clean_alphanum_hash",
"file_mtime_hash",
"base36_encode",
"HashAlgorithm",
"Hash",
"hash_string",
"hash_file",
Expand All @@ -25,6 +29,8 @@
"move_file",
"make_parent_dirs",
"atomic_output_file",
"atomic_write_text",
"atomic_write_bytes",
"temp_output_file",
"temp_output_dir",
"copyfile_atomic",
Expand All @@ -41,7 +47,13 @@
"StringTemplate",
)

from .atomic_var import * # noqa: F403
from .strif import * # noqa: F403
from .string_replace import * # noqa: F403
from .string_template import * # noqa: F403
try:
__version__ = version("strif")
except PackageNotFoundError:
# Running from a source tree that isn't installed.
__version__ = "0.0.0.dev0"

from .atomic_var import * # noqa: F403, E402
from .strif import * # noqa: F403, E402
from .string_replace import * # noqa: F403, E402
from .string_template import * # noqa: F403, E402
56 changes: 43 additions & 13 deletions src/strif/strif.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
from typing import Any, Literal

__all__ = (
"DEV_NULL",
Expand All @@ -30,6 +30,7 @@
"clean_alphanum_hash",
"file_mtime_hash",
"base36_encode",
"HashAlgorithm",
"Hash",
"hash_string",
"hash_file",
Expand All @@ -44,6 +45,8 @@
"move_file",
"make_parent_dirs",
"atomic_output_file",
"atomic_write_text",
"atomic_write_bytes",
"temp_output_file",
"temp_output_dir",
"copyfile_atomic",
Expand Down Expand Up @@ -183,6 +186,10 @@ def base36_encode(n: int) -> str:
return encoded


HashAlgorithm = Literal["sha1", "sha256", "sha384", "sha512", "md5", "blake2b", "blake2s"]
"""Common hash algorithms, for autocompletion. Any name `hashlib` accepts also works."""


@dataclass(frozen=True)
class Hash:
"""
Expand Down Expand Up @@ -222,7 +229,7 @@ def with_prefix(self) -> str:
return f"{self.algorithm}:{self.hex}"


def hash_string(string: str, algorithm: str = "sha1") -> Hash:
def hash_string(string: str, algorithm: HashAlgorithm | str = "sha1") -> Hash:
"""
Flexible hash of a string.
"""
Expand All @@ -231,13 +238,10 @@ def hash_string(string: str, algorithm: str = "sha1") -> Hash:
return Hash(algorithm, hasher.digest())


def hash_file(file_path: str | Path, algorithm: str = "sha1") -> Hash:
def hash_file(file_path: str | Path, algorithm: HashAlgorithm | str = "sha1") -> Hash:
"""
Hash the content of a file.
"""
if algorithm not in hashlib.algorithms_available:
raise ValueError(f"Unsupported hash algorithm: {algorithm}")

hasher = hashlib.new(algorithm)
file_path = Path(file_path)
with file_path.open("rb") as file:
Expand Down Expand Up @@ -286,13 +290,6 @@ def abbrev_list(
return joiner.join(shortened)


abbreviate_str = abbrev_str
"""Deprecated. Use `abbrev_str()` instead."""

abbreviate_list = abbrev_list
"""Deprecated. Use `abbrev_list()` instead."""


def single_line(text: str) -> str:
"""
Convert newlines and other whitespace to spaces.
Expand Down Expand Up @@ -569,6 +566,39 @@ def atomic_output_file(
tmp_path.replace(dest_path)


def atomic_write_text(
dest_path: str | Path,
text: str,
make_parents: bool = False,
backup_suffix: str | None = None,
encoding: str = "utf-8",
) -> None:
"""
Atomically write a string to a file, so a partial or corrupt file never appears
at `dest_path`. Convenience wrapper around `atomic_output_file()`.
"""
with atomic_output_file(
dest_path, make_parents=make_parents, backup_suffix=backup_suffix
) as tmp_path:
tmp_path.write_text(text, encoding=encoding)


def atomic_write_bytes(
dest_path: str | Path,
data: bytes,
make_parents: bool = False,
backup_suffix: str | None = None,
) -> None:
"""
Atomically write bytes to a file, so a partial or corrupt file never appears
at `dest_path`. Convenience wrapper around `atomic_output_file()`.
"""
with atomic_output_file(
dest_path, make_parents=make_parents, backup_suffix=backup_suffix
) as tmp_path:
tmp_path.write_bytes(data)


@contextmanager
def temp_output_file(
prefix: str = "tmp",
Expand Down
14 changes: 11 additions & 3 deletions src/strif/string_replace.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
from typing import TypeAlias
from __future__ import annotations

from typing import NamedTuple

__all__ = ["Insertion", "insert_multiple", "Replacement", "replace_multiple"]

Insertion = tuple[int, str]

class Insertion(NamedTuple):
offset: int
text: str


def insert_multiple(text: str, insertions: list[Insertion]) -> str:
Expand All @@ -19,7 +24,10 @@ def insert_multiple(text: str, insertions: list[Insertion]) -> str:
return "".join(chunks)


Replacement: TypeAlias = tuple[int, int, str]
class Replacement(NamedTuple):
start: int
end: int
text: str


def replace_multiple(text: str, replacements: list[Replacement]) -> str:
Expand Down
Loading