Understanding Test Reports
After running tests, Aethrix generates detailed reports at every level: project, suite, test case, and individual step. Reports are viewable in the IDE and exportable for CI/CD.
Accessing Reports
- Automatic: the Report panel opens by itself when you run a project from the Studio IDE.
- IDE (toolbar): Click description Show Report in the Test Automation toolbar after a run.
- IDE (project menu): Right-click the project node and choose Show Report.
- Report Viewer tool: Open any saved report file from disk.
Both entry points open the Project Report view, which shows a Test Suites tree on the left and a detail panel on the right, with a breadcrumb bar (Project → Suite → Case → Step) for navigation.
Report Levels
Project Report
| Metric | Description |
|---|---|
| Total Tests | Number of test cases executed |
| Passrate | Percentage of passing tests |
| Duration | Wall-clock execution time |
| Start Time | When the run began |
| Test Suites | Number of suites in the run |
Test Suite Report
Per-suite summary with:
- Suite-level pass/fail/skipped counts
- Execution time per test case
- Data-driven iteration counts
- Error summary for failed cases
Test Case Report
Detailed per-case view:
- Step-by-step status with color coding (green = pass, red = fail, gray = skipped)
- Per-step timing breakdown
- Request and response details for each Send Request step
- Variable values at each step
- Assertion results with expected vs actual values
Each step row carries a status symbol:
| Symbol | Meaning |
|---|---|
| ✅ | Step passed |
| ❌ | Step failed (assertion or error) |
| ⚠️ | Step passed with warnings |
| ⏭️ | Step skipped (disabled or Continue on Failure triggered) |
Step Report
| Step Type | Reported Details |
|---|---|
| Send Request | Full request (URL, headers, body) and response (status, headers, body, timing). The response HTTP status code is stored on the step report even after the run, so downstream tools (e.g. the smart-stop classifier, the AI's get_response) can read it later. |
| Assert | Assertion type, expected value, actual value, pass/fail reason |
| C# Script | Console output, exception details |
| SQL Query | Query text, result count, mapped variables |
| All Others | Configuration, inputs, outputs, execution time |
Security Test Case Report
When you run a Security Test Case, the case report additionally shows the aggregated security scan summary:
- Total attack instances and verdict counts (Secure / Vulnerable / NeedsReview / BaselineFailed / NotRun)
- A findings list for every Vulnerable / NeedsReview instance (step, category, mutation target, payload, severity, diagnostic, request URI)
- Per-category and per-severity rollups of the findings
The summary is exported with the report (JSON / JUnit XML / HTML) so CI can fail or gate on Vulnerable > 0.
Saving and Opening Reports
Within the Project Report view:
- Save Report (
) writes the report to disk. - Open Report () loads a previously saved report file for review.
Export Formats
| Format | CLI Flag | Use Case |
|---|---|---|
| JSON | (default) | Machine-readable, custom dashboards |
| JUnit XML | --junit | CI/CD integration (Azure DevOps, Jenkins, GitLab, etc.) |
| HTML | --html | Human-readable, shareable reports |
| Language | --language <code> | Localize the HTML report (e.g. fr, de, ja, zh-Hans) |
CLI Report Generation
aeth run MyProject.aprj --env staging --output ./reports --junit --html
This generates:
./reports/report.json— native Aethrix report./reports/report.junit.xml— JUnit XML for CI/CD./reports/report.html— standalone HTML report