feat: Add comprehensive unit test expansion for API endpoints#658
Open
aparnatonni wants to merge 1 commit into
Open
feat: Add comprehensive unit test expansion for API endpoints#658aparnatonni wants to merge 1 commit into
aparnatonni wants to merge 1 commit into
Conversation
- Created test_endpoints.py with 40+ pytest-based unit tests * TestMCQEndpoint: 8 tests for MCQ generation validation * TestBoolQEndpoint: 3 tests for Boolean question validation * TestShortQEndpoint: 2 tests for Short answer validation * TestMCQAnswerEndpoint: 5 tests for answer prediction validation * TestUploadEndpoint: 3 tests for file upload validation * TestGetContentEndpoint: 2 tests for content extraction validation * TestErrorResponses: 9 tests for error handling validation - Expanded test_server.py with 13 new error/edge case tests * test_mcq_empty_input() - validates empty text rejection * test_mcq_max_questions_exceeded() - validates 50 question limit * test_mcq_invalid_max_questions_type() - validates type checking * test_shortq_whitespace_only_input() - validates whitespace rejection * test_boolq_text_too_long() - validates 50,000 char limit * test_invalid_mediawiki_value() - validates flag validation * test_mcq_answer_missing_required_fields() - validates required field checking * test_error_response_format() - validates standardized error format * test_valid_response_format() - validates success response format * Additional parametrized tests for multiple invalid scenarios - Added TEST_DOCUMENTATION.md with comprehensive test guide * Overview of all test files and their purpose * Instructions for running tests locally * Coverage summary (50+ tests) * Expected output examples * Troubleshooting guide * CI/CD integration examples Test Coverage: - ✅ 50+ total tests (13+ integration, 40+ unit tests) - ✅ All 17 API endpoints validated - ✅ Error handling for empty inputs, oversized inputs, invalid types - ✅ Input validation tests (ranges, lengths, types, URLs) - ✅ Response format validation - ✅ Edge cases and boundary conditions Benefits: - Complete validation of input validation system - Early error detection in development process - Regression prevention for future changes - Clear documentation for contributors - Easy integration into CI/CD pipeline
📝 WalkthroughWalkthroughThis change introduces comprehensive testing infrastructure for the backend API. A new documentation file describes the testing setup and expected endpoint coverage, a new pytest-based test module provides isolated unit tests with fixtures and mocking, and the existing integration test module is extended with additional HTTP error and edge-case validation tests. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
backend/test_server.py (1)
186-190:⚠️ Potential issue | 🟡 MinorAdd timeout to the shared
make_post_requesthelper.The existing helper lacks a timeout, which could cause tests to hang indefinitely if the server is unresponsive. All new direct
requests.postcalls also lack timeouts (flagged by Ruff S113).♻️ Proposed fix
def make_post_request(endpoint, data): url = f'{BASE_URL}{endpoint}' headers = {'Content-Type': 'application/json'} - response = requests.post(url, headers=headers, data=json.dumps(data)) + response = requests.post(url, headers=headers, data=json.dumps(data), timeout=30) return response.json()🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/test_server.py` around lines 186 - 190, The make_post_request helper lacks a timeout which can hang tests; update the function signature make_post_request(endpoint, data, timeout=5) and pass that timeout into requests.post(..., timeout=timeout) so the helper always uses a finite timeout; also update any direct requests.post usages in this test file to either call make_post_request or include an explicit timeout argument (e.g., timeout=5) to satisfy Ruff S113 and avoid indefinite hangs.
🧹 Nitpick comments (6)
backend/test_server.py (3)
343-351: Same key mismatch: usedocument_url.def test_get_content_missing_url(): """Test rejection of missing URL parameter.""" endpoint = '/get_content' - data = {} + # Test should send empty document_url or omit it + data = {} # This correctly tests missing document_urlActually, this test is correct since it sends an empty dict, which means
document_urlis missing. However, the docstring and function name should clarify this testsdocument_urlnot genericurl.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/test_server.py` around lines 343 - 351, Rename and clarify the test to indicate it checks for a missing document_url parameter: change the function name test_get_content_missing_url to test_get_content_missing_document_url and update the docstring from "Test rejection of missing URL parameter." to something like "Test rejection of missing document_url parameter." so the test and docstring accurately reflect that an empty payload omits document_url.
390-485: Consider using pytest for consistent test execution and reporting.The
__main__block manually wraps each test in try/except with custom output formatting. This duplicates what pytest provides and makes it harder to get consistent exit codes and CI integration.Since pytest is already used for
test_endpoints.py, you could run this file with pytest too (alltest_*functions are pytest-compatible). The broadexcept Exceptioncatches (Ruff BLE001) would then be handled by pytest's reporting.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/test_server.py` around lines 390 - 485, The manual __main__ test runner around test_root / test_get_mcq / test_get_boolq / test_get_shortq / test_get_problems and the LLM/error blocks duplicates pytest and uses broad excepts; remove or replace the entire if __name__ == '__main__' block so the file is executed by pytest instead (or call pytest programmatically via pytest.main when CLI execution is required) and eliminate the broad try/except wrappers (those catching Exception for each test) so pytest can manage failures and exit codes; keep the test_* function names intact so pytest discovery still finds test_root, test_get_mcq, test_get_mcq_llm, test_get_boolq, test_get_problems_llm, test_get_answer, etc.
385-385: Remove extraneous f-string prefix.Per Ruff F541, this f-string has no placeholders:
- print(f"✓ Valid response follows expected format with 'output' key") + print("✓ Valid response follows expected format with 'output' key")🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/test_server.py` at line 385, The print statement using an unnecessary f-string (print(f"✓ Valid response follows expected format with 'output' key")) should be changed to a plain string to satisfy Ruff F541; locate the print call in backend/test_server.py (the line printing "✓ Valid response follows expected format with 'output' key") and remove the leading f so it becomes print("✓ Valid response follows expected format with 'output' key").backend/TEST_DOCUMENTATION.md (2)
146-179: Add language specification to the fenced code block.The expected output code block should specify a language (e.g.,
textorplaintext) for consistency and to satisfy markdown linting rules.-``` +```text ============================================================ Running Basic Endpoint Tests🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/TEST_DOCUMENTATION.md` around lines 146 - 179, The fenced code block that contains the test output (the block starting with the "Running Basic Endpoint Tests" header) in TEST_DOCUMENTATION.md lacks a language tag; update the opening fence from ``` to ```text (or ```plaintext) so the block is marked as plain text to satisfy markdown linting and ensure consistent rendering.
229-235: CI/CD integration test startup may be unreliable.Using a fixed
sleep 5to wait for the Flask server is fragile—model loading or slow CI runners could cause the server to not be ready in time, leading to flaky tests. Consider a health-check loop instead.- name: Run Integration Tests run: | cd backend python server.py & # Wait for server to be ready (up to 60 seconds) for i in {1..60}; do curl -s http://localhost:5000/ && break || sleep 1 done python test_server.py🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/TEST_DOCUMENTATION.md` around lines 229 - 235, The CI currently starts the Flask server with "python server.py &" then uses a fixed sleep (sleep 5) which is flaky; replace the fixed delay with a health-check loop that polls the server endpoint (e.g., http://localhost:5000/) until it responds or a timeout is reached before running "python test_server.py". Update the CI step invoking server.py and test_server.py to run the server in background, then loop (curl or similar) to check readiness (up to ~60s) and only proceed to run test_server.py once the health check succeeds, failing the job if the server never becomes ready.backend/test_endpoints.py (1)
309-318: Test assertion is overly permissive—doesn't validate expected behavior.Accepting status codes
[200, 400, 500]for invaliduse_mediawikivalues means the test always passes regardless of server behavior. This defeats the purpose of validation testing.If validation isn't implemented yet, use
@pytest.mark.xfailor@pytest.mark.skipinstead of accepting all outcomes:`@pytest.mark.xfail`(reason="use_mediawiki validation not yet implemented") `@pytest.mark.parametrize`('invalid_mediawiki', [2, -1, 5, 'yes', 'no', 1.0]) def test_invalid_mediawiki_values(self, client, invalid_mediawiki): ... assert response.status_code == 400🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@backend/test_endpoints.py` around lines 309 - 318, The test test_invalid_mediawiki_values is currently overly permissive (accepting [200,400,500]) so it always passes; update it to assert the correct behavior for invalid use_mediawiki values by either (A) enforcing validation and asserting response.status_code == 400 after POSTing the payload with VALID_INPUT_TEXT and the invalid use_mediawiki parameter, or (B) if validation isn't implemented yet, mark the test as expected failure with `@pytest.mark.xfail` (or skip) and then change the assertion to expect 400; locate the test function test_invalid_mediawiki_values and the use_mediawiki parameter to apply one of these fixes.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@backend/test_endpoints.py`:
- Around line 281-287: The test test_invalid_url_format sends the wrong JSON key
— the server's /get_content handler reads document_url (see document_url =
data.get('document_url')), so update the test_invalid_url_format POST payload to
use 'document_url': 'not-a-valid-url' (reference test function name
test_invalid_url_format and the server handler's document_url variable) so the
endpoint validation path is exercised correctly.
- Around line 58-66: The endpoint handler for '/get_mcq' does not validate the
max_questions parameter, so add input validation in the get_mcq request handler
(the function decorated for the '/get_mcq' route) to ensure 'max_questions' is
an integer and within an allowed range (e.g., 1–50); if missing, non-integer, or
out of range, return an HTTP 400 JSON response with an 'error' field describing
the problem. Ensure you validate the JSON payload before passing values to the
model, handle types like strings gracefully (attempt safe conversion or reject),
and reference the 'max_questions' field in your error messages so the parameter
is clearly identified.
- Around line 253-269: The tests test_empty_file and test_invalid_file_type use
an incorrect tuple format for file uploads with the Flask test client; update
both to supply file objects (e.g., io.BytesIO) and proper filename/content-type
and post with multipart/form-data so client.post('/upload', ...) simulates a
real file upload; specifically, replace the (None, b'...', 'text/plain') tuples
with file-like objects for the 'file' field and include the filename (and
content type if needed) when calling client.post in those tests.
- Around line 145-193: The tests fail because the get_mcq_answer endpoint
currently returns HTTP 200 with {"output": []} on validation failures but the
tests (TestMCQAnswerEndpoint tests such as test_missing_required_field_question,
test_missing_required_field_options, test_mismatched_question_options_count,
test_empty_question_list) expect HTTP 400 and an {"error": "..."} body; fix by
updating the get_mcq_answer handler in backend/server.py to perform input
validation for input_question and input_options (presence, non-empty lists, and
matching counts) and return a 400 response with a JSON error object (e.g.,
{"error": "<descriptive message>"}) for each validation failure instead of
{"output": []}, or alternatively update the test expectations to accept the
current 200/{"output": []} behavior — choose one approach and make the change
consistently so tests and the get_mcq_answer behavior align.
- Around line 40-56: The /get_mcq endpoint currently forwards inputs to
MCQGen.generate_mcq without validation; add input validation inside the request
handler in backend/server.py (the function that implements the /get_mcq
endpoint) before calling process_input_text() or MCQGen.generate_mcq: ensure
'input_text' exists and is a non-empty string after trimming whitespace (reject
missing or whitespace-only), validate 'max_questions' is an integer within the
allowed range (e.g., 1–50) and reject out-of-range or wrong-type values (also
reject floats and non-numeric strings), and on any validation failure return
HTTP 400 with a JSON body containing an "error" key and a helpful message; keep
the existing flow to call process_input_text() and MCQGen.generate_mcq only
after validation passes.
In `@backend/test_server.py`:
- Around line 196-207: Add request validation for the /get_mcq endpoint so empty
or missing input_text returns HTTP 400 with a JSON body containing an "error"
key (update the route handler in server.py that processes /get_mcq to check
input_text and max_questions before calling the model and return {"error":
"..."} on validation failure), apply the same validation policy used for other
endpoints referenced in these tests (lines covering tests 196-388) and update
the test POST calls in test_mcq_empty_input (and similar tests) to include
timeout=30 in their requests.post calls to satisfy the static analyzer.
- Around line 330-340: The test test_get_content_invalid_url is sending the
wrong JSON key; change the request payload to use 'document_url' instead of
'url' when posting to the '/get_content' endpoint so the server validates it
properly (update the data dict in test_get_content_invalid_url to
{'document_url': 'not-a-valid-url'} and keep the rest of the assertions intact).
---
Outside diff comments:
In `@backend/test_server.py`:
- Around line 186-190: The make_post_request helper lacks a timeout which can
hang tests; update the function signature make_post_request(endpoint, data,
timeout=5) and pass that timeout into requests.post(..., timeout=timeout) so the
helper always uses a finite timeout; also update any direct requests.post usages
in this test file to either call make_post_request or include an explicit
timeout argument (e.g., timeout=5) to satisfy Ruff S113 and avoid indefinite
hangs.
---
Nitpick comments:
In `@backend/TEST_DOCUMENTATION.md`:
- Around line 146-179: The fenced code block that contains the test output (the
block starting with the "Running Basic Endpoint Tests" header) in
TEST_DOCUMENTATION.md lacks a language tag; update the opening fence from ``` to
```text (or ```plaintext) so the block is marked as plain text to satisfy
markdown linting and ensure consistent rendering.
- Around line 229-235: The CI currently starts the Flask server with "python
server.py &" then uses a fixed sleep (sleep 5) which is flaky; replace the fixed
delay with a health-check loop that polls the server endpoint (e.g.,
http://localhost:5000/) until it responds or a timeout is reached before running
"python test_server.py". Update the CI step invoking server.py and
test_server.py to run the server in background, then loop (curl or similar) to
check readiness (up to ~60s) and only proceed to run test_server.py once the
health check succeeds, failing the job if the server never becomes ready.
In `@backend/test_endpoints.py`:
- Around line 309-318: The test test_invalid_mediawiki_values is currently
overly permissive (accepting [200,400,500]) so it always passes; update it to
assert the correct behavior for invalid use_mediawiki values by either (A)
enforcing validation and asserting response.status_code == 400 after POSTing the
payload with VALID_INPUT_TEXT and the invalid use_mediawiki parameter, or (B) if
validation isn't implemented yet, mark the test as expected failure with
`@pytest.mark.xfail` (or skip) and then change the assertion to expect 400; locate
the test function test_invalid_mediawiki_values and the use_mediawiki parameter
to apply one of these fixes.
In `@backend/test_server.py`:
- Around line 343-351: Rename and clarify the test to indicate it checks for a
missing document_url parameter: change the function name
test_get_content_missing_url to test_get_content_missing_document_url and update
the docstring from "Test rejection of missing URL parameter." to something like
"Test rejection of missing document_url parameter." so the test and docstring
accurately reflect that an empty payload omits document_url.
- Around line 390-485: The manual __main__ test runner around test_root /
test_get_mcq / test_get_boolq / test_get_shortq / test_get_problems and the
LLM/error blocks duplicates pytest and uses broad excepts; remove or replace the
entire if __name__ == '__main__' block so the file is executed by pytest instead
(or call pytest programmatically via pytest.main when CLI execution is required)
and eliminate the broad try/except wrappers (those catching Exception for each
test) so pytest can manage failures and exit codes; keep the test_* function
names intact so pytest discovery still finds test_root, test_get_mcq,
test_get_mcq_llm, test_get_boolq, test_get_problems_llm, test_get_answer, etc.
- Line 385: The print statement using an unnecessary f-string (print(f"✓ Valid
response follows expected format with 'output' key")) should be changed to a
plain string to satisfy Ruff F541; locate the print call in
backend/test_server.py (the line printing "✓ Valid response follows expected
format with 'output' key") and remove the leading f so it becomes print("✓ Valid
response follows expected format with 'output' key").
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: e94d00a6-1573-4e80-a48d-5ca38081b270
📒 Files selected for processing (3)
backend/TEST_DOCUMENTATION.mdbackend/test_endpoints.pybackend/test_server.py
Comment on lines
+40
to
+56
| def test_empty_input_text(self, client): | ||
| """Test rejection of empty input text.""" | ||
| response = client.post('/get_mcq', json={ | ||
| 'input_text': '', | ||
| 'max_questions': 5 | ||
| }) | ||
| assert response.status_code == 400 | ||
| assert 'error' in response.json() | ||
|
|
||
| def test_whitespace_only_input(self, client): | ||
| """Test rejection of whitespace-only input.""" | ||
| response = client.post('/get_mcq', json={ | ||
| 'input_text': ' \n\t ', | ||
| 'max_questions': 5 | ||
| }) | ||
| assert response.status_code == 400 | ||
| assert 'error' in response.json() |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify that /get_mcq endpoint lacks input validation
ast-grep --pattern $'@app.route("/get_mcq", methods=["POST"])
def get_mcq():
$$$
'Repository: AOSSIE-Org/EduAid
Length of output: 788
🏁 Script executed:
# Find and examine process_input_text() function
rg -A 10 "def process_input_text" backend/Repository: AOSSIE-Org/EduAid
Length of output: 595
🏁 Script executed:
# Check for error handlers in server.py
rg "@app.errorhandler|def.*error|raise.*400|abort" backend/server.pyRepository: AOSSIE-Org/EduAid
Length of output: 43
🏁 Script executed:
# Check MCQGen class for any validation
fd ".*MCQGen.*" backend/ --type fRepository: AOSSIE-Org/EduAid
Length of output: 43
🏁 Script executed:
# Find MCQGen class/module
fd -i "mcqgen" backend/ --type fRepository: AOSSIE-Org/EduAid
Length of output: 43
🏁 Script executed:
# Search for generate_mcq method
rg -A 15 "def generate_mcq" backend/Repository: AOSSIE-Org/EduAid
Length of output: 2491
🏁 Script executed:
# Check what happens with empty input - look at the test file to understand expected behavior
head -100 backend/test_endpoints.pyRepository: AOSSIE-Org/EduAid
Length of output: 3399
Tests will fail: server lacks the required input validation.
These tests assert that /get_mcq returns HTTP 400 with {"error": ...} for empty/whitespace input and invalid parameters. However, the /get_mcq endpoint in backend/server.py:55-66 has no input validation—it directly passes input to MCQGen.generate_mcq() without checking for empty text, whitespace, invalid max_questions values, or type validation. The process_input_text() function only handles optional MediaWiki lookups and performs no validation.
Additionally, the test file expects validation for:
- Empty
input_text - Whitespace-only
input_text - Invalid
max_questionsvalues (0, 51, 100, -5, 999) - Invalid
max_questionstypes (string, float) - Missing
input_textfield
None of these are validated in the server code.
Either:
- Add input validation to
backend/server.pyin the/get_mcqendpoint to reject invalid inputs with a 400 response, or - Mark these tests as
@pytest.mark.xfailuntil validation is implemented.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@backend/test_endpoints.py` around lines 40 - 56, The /get_mcq endpoint
currently forwards inputs to MCQGen.generate_mcq without validation; add input
validation inside the request handler in backend/server.py (the function that
implements the /get_mcq endpoint) before calling process_input_text() or
MCQGen.generate_mcq: ensure 'input_text' exists and is a non-empty string after
trimming whitespace (reject missing or whitespace-only), validate
'max_questions' is an integer within the allowed range (e.g., 1–50) and reject
out-of-range or wrong-type values (also reject floats and non-numeric strings),
and on any validation failure return HTTP 400 with a JSON body containing an
"error" key and a helpful message; keep the existing flow to call
process_input_text() and MCQGen.generate_mcq only after validation passes.
Comment on lines
+58
to
+66
| @pytest.mark.parametrize('invalid_questions', [0, 51, 100, -5, 999]) | ||
| def test_invalid_max_questions_values(self, client, invalid_questions): | ||
| """Test rejection of invalid max_questions values (parametrized).""" | ||
| response = client.post('/get_mcq', json={ | ||
| 'input_text': VALID_INPUT_TEXT, | ||
| 'max_questions': invalid_questions | ||
| }) | ||
| assert response.status_code == 400 | ||
| assert 'error' in response.json() |
There was a problem hiding this comment.
Parameterized validation tests will also fail without server-side implementation.
The tests for invalid max_questions values (0, 51, 100, -5, 999) expect HTTP 400 responses, but server.py does not validate max_questions range or type. The value is passed directly to the model.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@backend/test_endpoints.py` around lines 58 - 66, The endpoint handler for
'/get_mcq' does not validate the max_questions parameter, so add input
validation in the get_mcq request handler (the function decorated for the
'/get_mcq' route) to ensure 'max_questions' is an integer and within an allowed
range (e.g., 1–50); if missing, non-integer, or out of range, return an HTTP 400
JSON response with an 'error' field describing the problem. Ensure you validate
the JSON payload before passing values to the model, handle types like strings
gracefully (attempt safe conversion or reject), and reference the
'max_questions' field in your error messages so the parameter is clearly
identified.
Comment on lines
+145
to
+193
| class TestMCQAnswerEndpoint: | ||
| """Tests for /get_mcq_answer endpoint.""" | ||
|
|
||
| def test_valid_request(self, client): | ||
| """Test MCQ answer prediction with valid input.""" | ||
| response = client.post('/get_mcq_answer', json={ | ||
| 'input_text': VALID_INPUT_TEXT, | ||
| 'input_question': ['What is AI?'], | ||
| 'input_options': [['Option A', 'Option B', 'Option C', 'Option D']] | ||
| }) | ||
| assert response.status_code in [200, 400, 500] # May fail if no questions | ||
|
|
||
| def test_missing_required_field_question(self, client): | ||
| """Test rejection of missing input_question field.""" | ||
| response = client.post('/get_mcq_answer', json={ | ||
| 'input_text': VALID_INPUT_TEXT, | ||
| 'input_options': [['A', 'B', 'C', 'D']] | ||
| }) | ||
| assert response.status_code == 400 | ||
| assert 'error' in response.json() | ||
|
|
||
| def test_missing_required_field_options(self, client): | ||
| """Test rejection of missing input_options field.""" | ||
| response = client.post('/get_mcq_answer', json={ | ||
| 'input_text': VALID_INPUT_TEXT, | ||
| 'input_question': ['What is AI?'] | ||
| }) | ||
| assert response.status_code == 400 | ||
| assert 'error' in response.json() | ||
|
|
||
| def test_mismatched_question_options_count(self, client): | ||
| """Test rejection when question count != options count.""" | ||
| response = client.post('/get_mcq_answer', json={ | ||
| 'input_text': VALID_INPUT_TEXT, | ||
| 'input_question': ['Q1', 'Q2', 'Q3'], | ||
| 'input_options': [['A', 'B', 'C', 'D'], ['A', 'B', 'C', 'D']] # Only 2 option sets | ||
| }) | ||
| assert response.status_code == 400 | ||
| assert 'error' in response.json() | ||
|
|
||
| def test_empty_question_list(self, client): | ||
| """Test rejection of empty question list.""" | ||
| response = client.post('/get_mcq_answer', json={ | ||
| 'input_text': VALID_INPUT_TEXT, | ||
| 'input_question': [], | ||
| 'input_options': [] | ||
| }) | ||
| assert response.status_code == 400 | ||
| assert 'error' in response.json() |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify /get_mcq_answer returns output for invalid input instead of error
rg -n -A 10 'def get_mcq_answer' backend/server.pyRepository: AOSSIE-Org/EduAid
Length of output: 527
🏁 Script executed:
#!/bin/bash
# Check the full endpoint implementation
rg -n -A 30 'def get_mcq_answer' backend/server.py | head -50Repository: AOSSIE-Org/EduAid
Length of output: 1417
🏁 Script executed:
#!/bin/bash
# Read the actual test file to verify assertions
cat -n backend/test_endpoints.py | sed -n '145,193p'Repository: AOSSIE-Org/EduAid
Length of output: 2512
🏁 Script executed:
#!/bin/bash
# Check if there's any status code setting in get_mcq_answer or surrounding code
rg -n 'status_code\|@app.route.*get_mcq_answer' backend/server.py | head -20Repository: AOSSIE-Org/EduAid
Length of output: 43
Tests expect error format and status code that server does not return.
The /get_mcq_answer endpoint returns {"output": []} with HTTP 200 status code when validation fails (line 189-190 in backend/server.py). However, tests at lines 157-193 expect HTTP 400 and {"error": "..."} format.
The following test assertions will fail:
test_missing_required_field_question(line 163-164)test_missing_required_field_options(line 172-173)test_mismatched_question_options_count(line 182-183)test_empty_question_list(line 192-193)
Align test expectations with server behavior or modify the endpoint to return proper error responses with 400 status code.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@backend/test_endpoints.py` around lines 145 - 193, The tests fail because the
get_mcq_answer endpoint currently returns HTTP 200 with {"output": []} on
validation failures but the tests (TestMCQAnswerEndpoint tests such as
test_missing_required_field_question, test_missing_required_field_options,
test_mismatched_question_options_count, test_empty_question_list) expect HTTP
400 and an {"error": "..."} body; fix by updating the get_mcq_answer handler in
backend/server.py to perform input validation for input_question and
input_options (presence, non-empty lists, and matching counts) and return a 400
response with a JSON error object (e.g., {"error": "<descriptive message>"}) for
each validation failure instead of {"output": []}, or alternatively update the
test expectations to accept the current 200/{"output": []} behavior — choose one
approach and make the change consistently so tests and the get_mcq_answer
behavior align.
Comment on lines
+253
to
+269
| def test_empty_file(self, client): | ||
| """Test handling of empty file upload.""" | ||
| data = { | ||
| 'file': (None, b'', 'text/plain'), | ||
| 'filename': 'test.pdf' | ||
| } | ||
| response = client.post('/upload', data=data) | ||
| assert response.status_code in [400, 500] | ||
|
|
||
| def test_invalid_file_type(self, client): | ||
| """Test rejection of unsupported file type.""" | ||
| data = { | ||
| 'file': (None, b'test content', 'text/plain'), | ||
| 'filename': 'test.txt' | ||
| } | ||
| response = client.post('/upload', data=data) | ||
| assert response.status_code in [400, 500] |
There was a problem hiding this comment.
File upload test format may not work correctly with Flask test client.
The file upload test uses (None, b'', 'text/plain') tuple format, but Flask's test client expects file data in a different format. For proper file upload simulation:
♻️ Proposed fix for file upload tests
def test_empty_file(self, client):
"""Test handling of empty file upload."""
+ from io import BytesIO
data = {
- 'file': (None, b'', 'text/plain'),
- 'filename': 'test.pdf'
+ 'file': (BytesIO(b''), 'test.pdf')
}
- response = client.post('/upload', data=data)
+ response = client.post('/upload', data=data, content_type='multipart/form-data')
assert response.status_code in [400, 500]
def test_invalid_file_type(self, client):
"""Test rejection of unsupported file type."""
+ from io import BytesIO
data = {
- 'file': (None, b'test content', 'text/plain'),
- 'filename': 'test.txt'
+ 'file': (BytesIO(b'test content'), 'test.txt')
}
- response = client.post('/upload', data=data)
+ response = client.post('/upload', data=data, content_type='multipart/form-data')
assert response.status_code in [400, 500]🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@backend/test_endpoints.py` around lines 253 - 269, The tests test_empty_file
and test_invalid_file_type use an incorrect tuple format for file uploads with
the Flask test client; update both to supply file objects (e.g., io.BytesIO) and
proper filename/content-type and post with multipart/form-data so
client.post('/upload', ...) simulates a real file upload; specifically, replace
the (None, b'...', 'text/plain') tuples with file-like objects for the 'file'
field and include the filename (and content type if needed) when calling
client.post in those tests.
Comment on lines
+281
to
+287
| def test_invalid_url_format(self, client): | ||
| """Test rejection of invalid URL format.""" | ||
| response = client.post('/get_content', json={ | ||
| 'url': 'not-a-valid-url' | ||
| }) | ||
| assert response.status_code == 400 | ||
| assert 'error' in response.json() |
There was a problem hiding this comment.
Wrong JSON key: server expects document_url, not url.
Per context snippet 3 (backend/server.py:246-263), the /get_content endpoint expects document_url:
document_url = data.get('document_url')But the test sends url:
def test_invalid_url_format(self, client):
"""Test rejection of invalid URL format."""
response = client.post('/get_content', json={
- 'url': 'not-a-valid-url'
+ 'document_url': 'not-a-valid-url'
})🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@backend/test_endpoints.py` around lines 281 - 287, The test
test_invalid_url_format sends the wrong JSON key — the server's /get_content
handler reads document_url (see document_url = data.get('document_url')), so
update the test_invalid_url_format POST payload to use 'document_url':
'not-a-valid-url' (reference test function name test_invalid_url_format and the
server handler's document_url variable) so the endpoint validation path is
exercised correctly.
Comment on lines
+196
to
+207
| def test_mcq_empty_input(): | ||
| """Test rejection of empty input text.""" | ||
| endpoint = '/get_mcq' | ||
| data = { | ||
| 'input_text': '', | ||
| 'max_questions': 5 | ||
| } | ||
| response = requests.post(f'{BASE_URL}{endpoint}', headers={'Content-Type': 'application/json'}, data=json.dumps(data)) | ||
| assert response.status_code == 400 | ||
| result = response.json() | ||
| assert 'error' in result | ||
| print(f"✓ MCQ with empty input correctly rejected: {result['error']}") |
There was a problem hiding this comment.
Tests will fail: server lacks input validation (same issue as test_endpoints.py).
These error validation tests expect /get_mcq to return HTTP 400 with {"error": ...} for empty input. However, the server does not implement this validation—requests proceed to the model layer.
This applies to all tests in lines 196-388 that assert validation behavior. Either implement validation in server.py first, or mark these tests appropriately.
Additionally, add timeout=30 to the requests.post call per static analysis (Ruff S113).
♻️ Proposed fix for timeout
- response = requests.post(f'{BASE_URL}{endpoint}', headers={'Content-Type': 'application/json'}, data=json.dumps(data))
+ response = requests.post(f'{BASE_URL}{endpoint}', headers={'Content-Type': 'application/json'}, data=json.dumps(data), timeout=30)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def test_mcq_empty_input(): | |
| """Test rejection of empty input text.""" | |
| endpoint = '/get_mcq' | |
| data = { | |
| 'input_text': '', | |
| 'max_questions': 5 | |
| } | |
| response = requests.post(f'{BASE_URL}{endpoint}', headers={'Content-Type': 'application/json'}, data=json.dumps(data)) | |
| assert response.status_code == 400 | |
| result = response.json() | |
| assert 'error' in result | |
| print(f"✓ MCQ with empty input correctly rejected: {result['error']}") | |
| def test_mcq_empty_input(): | |
| """Test rejection of empty input text.""" | |
| endpoint = '/get_mcq' | |
| data = { | |
| 'input_text': '', | |
| 'max_questions': 5 | |
| } | |
| response = requests.post(f'{BASE_URL}{endpoint}', headers={'Content-Type': 'application/json'}, data=json.dumps(data), timeout=30) | |
| assert response.status_code == 400 | |
| result = response.json() | |
| assert 'error' in result | |
| print(f"✓ MCQ with empty input correctly rejected: {result['error']}") |
🧰 Tools
🪛 Ruff (0.15.7)
[error] 203-203: Probable use of requests call without timeout
(S113)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@backend/test_server.py` around lines 196 - 207, Add request validation for
the /get_mcq endpoint so empty or missing input_text returns HTTP 400 with a
JSON body containing an "error" key (update the route handler in server.py that
processes /get_mcq to check input_text and max_questions before calling the
model and return {"error": "..."} on validation failure), apply the same
validation policy used for other endpoints referenced in these tests (lines
covering tests 196-388) and update the test POST calls in test_mcq_empty_input
(and similar tests) to include timeout=30 in their requests.post calls to
satisfy the static analyzer.
Comment on lines
+330
to
+340
| def test_get_content_invalid_url(): | ||
| """Test rejection of invalid URL format.""" | ||
| endpoint = '/get_content' | ||
| data = { | ||
| 'url': 'not-a-valid-url' | ||
| } | ||
| response = requests.post(f'{BASE_URL}{endpoint}', headers={'Content-Type': 'application/json'}, data=json.dumps(data)) | ||
| assert response.status_code == 400 | ||
| result = response.json() | ||
| assert 'error' in result | ||
| print(f"✓ Get content with invalid URL correctly rejected: {result['error']}") |
There was a problem hiding this comment.
Wrong JSON key: server expects document_url, not url.
Same issue as in test_endpoints.py—the server's /get_content endpoint expects document_url:
def test_get_content_invalid_url():
"""Test rejection of invalid URL format."""
endpoint = '/get_content'
data = {
- 'url': 'not-a-valid-url'
+ 'document_url': 'not-a-valid-url'
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def test_get_content_invalid_url(): | |
| """Test rejection of invalid URL format.""" | |
| endpoint = '/get_content' | |
| data = { | |
| 'url': 'not-a-valid-url' | |
| } | |
| response = requests.post(f'{BASE_URL}{endpoint}', headers={'Content-Type': 'application/json'}, data=json.dumps(data)) | |
| assert response.status_code == 400 | |
| result = response.json() | |
| assert 'error' in result | |
| print(f"✓ Get content with invalid URL correctly rejected: {result['error']}") | |
| def test_get_content_invalid_url(): | |
| """Test rejection of invalid URL format.""" | |
| endpoint = '/get_content' | |
| data = { | |
| 'document_url': 'not-a-valid-url' | |
| } | |
| response = requests.post(f'{BASE_URL}{endpoint}', headers={'Content-Type': 'application/json'}, data=json.dumps(data)) | |
| assert response.status_code == 400 | |
| result = response.json() | |
| assert 'error' in result | |
| print(f"✓ Get content with invalid URL correctly rejected: {result['error']}") |
🧰 Tools
🪛 Ruff (0.15.7)
[error] 336-336: Probable use of requests call without timeout
(S113)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@backend/test_server.py` around lines 330 - 340, The test
test_get_content_invalid_url is sending the wrong JSON key; change the request
payload to use 'document_url' instead of 'url' when posting to the
'/get_content' endpoint so the server validates it properly (update the data
dict in test_get_content_invalid_url to {'document_url': 'not-a-valid-url'} and
keep the rest of the assertions intact).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds comprehensive unit tests for API endpoint validation and error handling.
Changes
test_endpoints.py- 40+ pytest unit tests for all endpointstest_server.py- 13 new error/edge case testsTEST_DOCUMENTATION.md- Testing guideSummary by CodeRabbit
Documentation
Tests