Skip to content

Add unittest suite and GitHub Actions CI workflow#10

Merged
wannaphong merged 3 commits into
mainfrom
copilot/create-unittest-and-github-action
Apr 7, 2026
Merged

Add unittest suite and GitHub Actions CI workflow#10
wannaphong merged 3 commits into
mainfrom
copilot/create-unittest-and-github-action

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 7, 2026

No test infrastructure existed for the LEKCut library. This adds a full unittest suite covering every supported model and a CI workflow to run it automatically.

Tests (tests/test_lekcut.py)

25 tests across all seven tokenizer models (deepcut, attacut-sc, attacut-c, oskut, sefr-ws1000, sefr-tnhc, sefr-best), each covering:

  • Basic tokenization — output is a non-empty list whose concatenation equals the input
  • Known output assertions (deepcut)
  • Empty string → []
  • NotImplementedError raised for unknown model names
from lekcut import word_tokenize

result = word_tokenize("ทดสอบการตัดคำ", model="deepcut")
assert result == ["ทดสอบ", "การ", "ตัด", "คำ"]

result = word_tokenize("", model="deepcut")
assert result == []

CI (.github/workflows/tests.yml)

Runs python -m unittest discover -s tests -v on push/PR to main across Python 3.9–3.12, with permissions: contents: read.

Bug fixes

deepcut.py and attacut.py lacked the empty-string guard already present in oskut.py, causing an ONNX runtime crash on empty input. Added if not text: return [] to their tokenize methods.

Copilot AI linked an issue Apr 7, 2026 that may be closed by this pull request
Copilot AI and others added 2 commits April 7, 2026 08:46
Agent-Logs-Url: https://github.com/PyThaiNLP/LEKCut/sessions/6266cef0-f9f2-420d-abda-2c42afcb0a4a

Co-authored-by: wannaphong <8536487+wannaphong@users.noreply.github.com>
…sefr empty-string tests, add workflow permissions

Agent-Logs-Url: https://github.com/PyThaiNLP/LEKCut/sessions/6266cef0-f9f2-420d-abda-2c42afcb0a4a

Co-authored-by: wannaphong <8536487+wannaphong@users.noreply.github.com>
Copilot AI changed the title [WIP] Add unittest and GitHub Action for testing Add unittest suite and GitHub Actions CI workflow Apr 7, 2026
Copilot AI requested a review from wannaphong April 7, 2026 08:54
@wannaphong wannaphong marked this pull request as ready for review April 7, 2026 08:58
@wannaphong wannaphong merged commit c847be9 into main Apr 7, 2026
8 checks passed
@wannaphong wannaphong deleted the copilot/create-unittest-and-github-action branch April 7, 2026 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create unittest and github action for testing

2 participants