Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
18172d6
Adapted to Asphalt 5
agronholm Feb 4, 2024
cf9b204
Removed unused test dependency
agronholm Feb 4, 2024
97fd3ec
Fixed tests on PyPy3
agronholm Feb 4, 2024
13d2ba3
Enabled the tests to run on Trio where possible
agronholm Feb 4, 2024
90afc1f
Adapted code to the latest 5.0 core changes
agronholm Feb 5, 2024
aa56e76
Adapted examples to Asphalt 5.0
agronholm Feb 5, 2024
c88bdc1
Use a component reference consistent with the other projects
agronholm Feb 7, 2024
78ea3d1
Updated docstrings and MyPy settings
agronholm Feb 8, 2024
893f3c6
Updated documentation
agronholm Feb 8, 2024
1711c7e
Updated for the latest Asphalt 5 changes
agronholm Mar 5, 2024
38c8b63
Updated pre-commit modules and ruff configuration
agronholm Mar 5, 2024
6697bd3
Added some finishing touches/updates
agronholm Mar 5, 2024
7955450
Updated GitHub actions and fixed Mypy errors
agronholm Mar 7, 2024
47e9509
Downgraded the minimum Asphalt requirement for testing
agronholm Mar 7, 2024
6cd4a24
Updated code, docs and tests to match the new resource API
agronholm Mar 26, 2024
8b6ca10
Updated pre-commit modules
agronholm Mar 26, 2024
c2d8b9c
Updated use of run_application() in examples to match the latest changes
agronholm Apr 18, 2024
f0a4b26
Removed uses of GeneratedResource
agronholm May 5, 2024
af9b4ef
Updated code to work with the latest changes in the core
agronholm Jun 10, 2024
3cb8d0a
Asphalt5 update
agronholm Dec 14, 2024
a9910d5
Don't test on pypy
agronholm Dec 14, 2024
c8b0275
Updated configs, docs and some code for Asphalt 5
agronholm Dec 30, 2024
3c54df0
Updated pre-commit modules
agronholm Dec 30, 2024
3cf0350
Merge branch 'master' into asphalt5
agronholm Dec 30, 2024
d3af4d1
Updated the changelog
agronholm Dec 30, 2024
b29faf5
Fixed mypy errors
agronholm Dec 30, 2024
4d9f1c7
Added mypy_path
agronholm Dec 30, 2024
a5bafe2
Updated the examples for Asphalt 5
agronholm Dec 30, 2024
cb88971
Removed the existing DB in prepare() rather than start()
agronholm Dec 30, 2024
3ff52f2
Fixed config oopsie and increased test coverage
agronholm Dec 31, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
- name: Create packages
run: python -m build
- name: Archive packages
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: dist
path: dist
Expand All @@ -36,7 +36,7 @@ jobs:
id-token: write
steps:
- name: Retrieve packages
uses: actions/download-artifact@v3
uses: actions/download-artifact@v4
- name: Upload packages
uses: pypa/gh-action-pypi-publish@release/v1

Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: test suite

on:
push:
branches: [master]
branches: [master, "asphalt5"]
pull_request:

jobs:
Expand All @@ -11,7 +11,7 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest]
python-version: ["3.8", "3.10", "3.11", "3.12", "pypy-3.9"]
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
Expand All @@ -22,9 +22,9 @@ jobs:
cache: pip
cache-dependency-path: pyproject.toml
- name: Start external services
run: docker-compose up -d
run: docker compose up -d
- name: Install dependencies
run: pip install .[test]
run: pip install -e .[test]
- name: Test with pytest
run: coverage run -m pytest -v
- name: Generate coverage report
Expand Down
11 changes: 7 additions & 4 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
# * Run "pre-commit install".
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0
rev: v5.0.0
hooks:
- id: check-toml
- id: check-yaml
Expand All @@ -16,17 +16,20 @@ repos:
- id: trailing-whitespace

- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.5.2
rev: v0.8.4
hooks:
- id: ruff
args: [--fix, --show-fixes]
- id: ruff-format

- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.10.1
rev: v1.14.0
hooks:
- id: mypy
additional_dependencies: ["pytest"]
additional_dependencies:
- asphalt@git+https://github.com/asphalt-framework/asphalt
- pytest
- sqlalchemy

- repo: https://github.com/pre-commit/pygrep-hooks
rev: v1.10.0
Expand Down
2 changes: 1 addition & 1 deletion .readthedocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ version: 2
build:
os: ubuntu-22.04
tools:
python: "3.8"
python: "3.9"

sphinx:
configuration: docs/conf.py
Expand Down
12 changes: 5 additions & 7 deletions docs/api.rst
Original file line number Diff line number Diff line change
@@ -1,17 +1,15 @@
API reference
=============

Components
----------
.. py:currentmodule:: asphalt.sqlalchemy

.. py:currentmodule:: asphalt.sqlalchemy.component
Component
---------

.. autoclass:: SQLAlchemyComponent

Utilities
---------

.. py:currentmodule:: asphalt.sqlalchemy.utils
Utility functions
-----------------

.. autofunction:: clear_database
.. autofunction:: clear_async_database
Expand Down
3 changes: 3 additions & 0 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
"sphinx.ext.autodoc",
"sphinx.ext.intersphinx",
"sphinx_autodoc_typehints",
"sphinx_rtd_theme",
"sphinx_tabs.tabs",
]

Expand All @@ -25,6 +26,8 @@

exclude_patterns = ["_build"]
pygments_style = "sphinx"
autodoc_default_options = {"members": True, "show-inheritance": True}
autodoc_inherit_docstrings = False
highlight_language = "python3"
todo_include_todos = False

Expand Down
35 changes: 8 additions & 27 deletions docs/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,16 @@ Configuration

A typical SQLAlchemy configuration consists of a single database.
At minimum, you only need a connection URL (see the
`SQLAlchemy documentation`_ for how to construct one). Such a configuration would look
something like this::
:doc:`SQLAlchemy documentation <sqlalchemy:tutorial/engine>` on how to construct one).
Such a configuration would look something like this::

components:
sqlalchemy:
url: postgresql+asyncpg://user:password@10.0.0.8/mydatabase
url: postgresql+psycopg://user:password@10.0.0.8/mydatabase

This will add two static resources and one resource factory, as follows:

* engine (type: :class:`sqlalchemy.future.engine.Engine` or
* engine (type: :class:`sqlalchemy.engine.Engine` or
:class:`sqlalchemy.ext.asyncio.AsyncEngine`)
* sessionmaker (type: :class:`sqlalchemy.orm.sessionmaker`)
* session factory (generates :class:`~sqlalchemy.orm.Session` or
Expand All @@ -26,23 +26,22 @@ sharing features)::
components:
sqlalchemy:
url:
drivername: postgresql+asyncpg
drivername: postgresql+psycopg
username: user
password: password
host: 10.0.0.8
database: mydatabase

.. seealso::
* :class:`sqlalchemy.engine.URL`
* :class:`asphalt.sqlalchemy.component.SQLAlchemyComponent`

.. _SQLAlchemy documentation: https://docs.sqlalchemy.org/en/14/core/engines.html
* :class:`asphalt.sqlalchemy.SQLAlchemyComponent`

Setting engine or session options
---------------------------------

If you need to adjust the options used for creating new sessions, or pass extra
arguments to the engine, you can do so by specifying them in the ``session`` option::
arguments to the engine, you can do so by specifying them in the ``session_args`` and
``engine_args``, respectively::

components:
sqlalchemy:
Expand All @@ -52,21 +51,3 @@ arguments to the engine, you can do so by specifying them in the ``session`` opt
session_args:
info:
hello: world

Multiple databases
------------------

If you need to work with multiple databases, you will need to use multiple instances
of the ``sqlalchemy`` component::

components:
sqlalchemy:
resource_name: db1
url: postgresql+asyncpg:///mydatabase
sqlalchemy2:
type: sqlalchemy
resource_name: db2
url: sqlite+aiosqlite:///mydb.sqlite

This will make the appropriate resources available using their respective namespaces
(``db1`` or ``db2`` instead of ``default``).
4 changes: 2 additions & 2 deletions docs/events.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,15 +33,15 @@ a synchronous ``sessionmaker`` resource is provided by this component even for a
engines. To add listeners, simply use this session maker as the target for the
listeners::

from asphalt.core import NoCurrentContext, get_resource, inject, resource
from asphalt.core import NoCurrentContext, get_resource_nowait, inject, resource
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import sessionmaker
from sqlalchemy import event


def precommit_hook(session):
try:
async_session = get_resource(AsyncSession)
async_session = get_resource_nowait(AsyncSession)
except NoCurrentContext:
return

Expand Down
27 changes: 5 additions & 22 deletions docs/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -46,31 +46,15 @@ you can skip this section. For those having to use engines for which no async
counterpart is available, read on.

Synchronous database connections in an asynchronous application pose a problem because
running operations against a database will block the event loop, and thus needs to be
running operations against a database will block the event loop, and thus need to be
wrapped in worker threads. Another thing to watch out for is lazy loading of
relationships and deferred columns which triggers implicit queries when those attributes
are accessed.

On Python 3.9 and above, you would do::

from asyncio import to_thread

from asphalt.core import inject, resource
from sqlalchemy.orm import Session
from sqlalchemy.future import select

from yourapp.model import Person


@inject
async def handler(*, dbsession: Session = resource()) -> None:
people = await to_thread(dbsession.scalars, select(Person))


On earlier Python versions::

from asyncio import get_running_loop
Here's an example of how you could use worker threads to run a blocking database
operation::

from anyio import to_thread
from asphalt.core import inject, resource
from sqlalchemy.orm import Session
from sqlalchemy.future import select
Expand All @@ -80,8 +64,7 @@ On earlier Python versions::

@inject
async def handler(*, dbsession: Session = resource()) -> None:
loop = get_running_loop()
people = await loop.run_in_executor(None, dbsession.scalars, select(Person))
people = await to_thread.run_sync(dbsession.scalars, select(Person))

Releasing database resources during a long operation
----------------------------------------------------
Expand Down
7 changes: 7 additions & 0 deletions docs/versionhistory.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,13 @@ Version history

This library adheres to `Semantic Versioning 2.0 <http://semver.org/>`_.

**UNRELEASED**

- Dropped support for Python 3.8
- **BACKWARD INCOMPATIBLE** Bumped minimum Asphalt version to 5.0
- **BACKWARD INCOMPATIBLE** Dropped the ``resource_name`` option in favor of Asphalt's
native alternate resource name syntax

**5.1.0** (2024-01-16)

- Dropped support for Python 3.7
Expand Down
29 changes: 15 additions & 14 deletions examples/csvimport.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@
import logging
from pathlib import Path

from anyio import to_thread
from asphalt.core import (
CLIApplicationComponent,
Context,
inject,
resource,
run_application,
Expand All @@ -37,23 +37,21 @@ class CSVImporterComponent(CLIApplicationComponent):
def __init__(self) -> None:
super().__init__()
self.csv_path = Path(__file__).with_name("people.csv")

async def start(self, ctx: Context) -> None:
# Remove the db file if it exists
db_path = self.csv_path.with_name("people.db")
if db_path.exists():
db_path.unlink()

self.db_path = self.csv_path.with_name("people.db")
self.add_component(
"sqlalchemy",
url=f"sqlite:///{db_path}",
url=f"sqlite:///{self.db_path}",
ready_callback=lambda bind, factory: metadata.create_all(bind),
)
await super().start(ctx)

async def prepare(self) -> None:
# Remove the db file if it exists
if self.db_path.exists():
self.db_path.unlink()

@inject
async def run(self, ctx: Context, *, dbsession: Session = resource()) -> None:
async with ctx.threadpool():
async def run(self, *, dbsession: Session = resource()) -> None:
def insert_rows_in_thread() -> int:
num_rows = 0
with self.csv_path.open() as csvfile:
reader = csv.reader(csvfile, delimiter="|")
Expand All @@ -65,7 +63,10 @@ async def run(self, ctx: Context, *, dbsession: Session = resource()) -> None:
)
)

logger.info("Imported %d rows of data", num_rows)
return num_rows

inserted_rows = await to_thread.run_sync(insert_rows_in_thread)
logger.info("Imported %d rows of data", inserted_rows)


run_application(CSVImporterComponent(), logging=logging.DEBUG)
run_application(CSVImporterComponent)
28 changes: 14 additions & 14 deletions examples/csvimport_orm.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@
import logging
from pathlib import Path

from anyio import to_thread
from asphalt.core import (
CLIApplicationComponent,
Context,
inject,
resource,
run_application,
Expand Down Expand Up @@ -39,25 +39,23 @@ class CSVImporterComponent(CLIApplicationComponent):
def __init__(self) -> None:
super().__init__()
self.csv_path = Path(__file__).with_name("people.csv")

async def start(self, ctx: Context) -> None:
# Remove the db file if it exists
db_path = self.csv_path.with_name("people.db")
if db_path.exists():
db_path.unlink()

self.db_path = self.csv_path.with_name("people.db")
self.add_component(
"sqlalchemy",
url=f"sqlite:///{db_path}",
url=f"sqlite:///{self.db_path}",
ready_callback=lambda bind, factory: DeclarativeBase.metadata.create_all(
bind
),
)
await super().start(ctx)

async def prepare(self) -> None:
# Remove the db file if it exists
if self.db_path.exists():
self.db_path.unlink()

@inject
async def run(self, ctx: Context, *, dbsession: Session = resource()) -> None:
async with ctx.threadpool():
async def run(self, *, dbsession: Session = resource()) -> None:
def insert_rows_in_thread() -> int:
num_rows = 0
with self.csv_path.open() as csvfile:
reader = csv.reader(csvfile, delimiter="|")
Expand All @@ -70,8 +68,10 @@ async def run(self, ctx: Context, *, dbsession: Session = resource()) -> None:
# Emit pending INSERTs (though this would happen when the context ends
# anyway)
dbsession.flush()
return num_rows

logger.info("Imported %d rows of data", num_rows)
inserted_rows = await to_thread.run_sync(insert_rows_in_thread)
logger.info("Imported %d rows of data", inserted_rows)


run_application(CSVImporterComponent(), logging=logging.DEBUG)
run_application(CSVImporterComponent)
Loading