Ruud Welling
11/12/2025, 8:20 AMBoris
11/13/2025, 12:40 AMBoris
11/13/2025, 12:41 AMRuud Welling
11/13/2025, 10:25 AMMatt (pactflow.io / pact-js / pact-go)
Matt (pactflow.io / pact-js / pact-go)
I need a COMPLETE, DETAILED analysis. Do NOT summarize. Output EXACTLY this format:
## Coverage Summary
| Metric | Value |
|--------|-------|
| Coverage Percentage | [calculate]% |
| Total API Scenarios | [count] |
| Total Pact Interactions | [count] |
| Pact Test File Status | ✅ Has Tests / ❌ Empty |
## Coverage Details
| API Resource | HTTP Method | Path | Scenario | Status |
|------------------|-------------|---------------|----------|--------|
| getAllProducts | GET | /products | 200 | ✅/❌ |
| getAllProducts | GET | /products | 401 | ✅/❌ |
| someOperation | POST | /op/{id} | 404 | ✅/❌ |
## 🚨 Coverage Gaps & Recommendations
- List each missing test case
- Provide specific implementation recommendations
ANALYZE THESE FILES:
=== API CLIENT (xyz.ts) ===
...
=== OPENAPI SPEC (xyz.yml) ===
...
=== PACT FILE (abc-def.json)===
...
=== PACT TESTS (xyz.pact.spec.ts)===
...
REQUIREMENTS:
1. Fill in ALL table rows with actual data
2. Calculate coverage percentage as: (covered scenarios / total API scenarios) * 100
3. List every missing test scenario
4. Do NOT summarize - provide complete detailed output
5. Follow the exact table format shown above
**Analysis Rules:**
- Only consider methods that exist in the API client - ignore OpenAPI-only endpoints
- Check for coverage of common (or as documented in the OpenAPI spec) HTTP status codes: 200 (success), 400 (bad request), 401 (unauthorized), 404 (not found)
- Mark as covered if ANY Pact test exists for that method/endpoint and status code combination
- Calculate coverage percentage as: (covered scenarios / total API scenarios) * 100
- **Total API Scenarios = sum of all unique combinations of (HTTP method + path + status code) that should be tested based on the API client methods and OpenAPI spec**
- **Each row in the Coverage Details table represents ONE scenario (one method + path + status code combination)**
**CRITICAL VALIDATION STEPS:**
1. **Before creating the Coverage Details table**: List out all scenarios you identified above
2. **After creating the Coverage Details table**: Count the actual rows in your table
3. **Final Check**: Verify the row count matches your "Total API Scenarios" in the Coverage Summary
4. **If they don't match**: Re-examine your analysis and fix the discrepancy before finalizing
START YOUR RESPONSE WITH "## Coverage Summary"Matt (pactflow.io / pact-js / pact-go)
## Coverage Summary
| Metric | Value |
|--------|-------|
| Coverage Percentage | 10% |
| Total API Scenarios | 10 |
| Total Pact Interactions | 1 |
| Pact Test File Status | ✅ Has Tests |
## Coverage Details
| API Resource | HTTP Method | Path | Scenario | Status |
|------------------|-------------|---------------|----------|--------|
| getAllProducts | GET | /products | 200 | ✅ |
| getAllProducts | GET | /products | 401 | ❌ |
| getProduct | GET | /product/{id} | 200 | ❌ |
| getProduct | GET | /product/{id} | 400 | ❌ |
| getProduct | GET | /product/{id} | 401 | ❌ |
| getProduct | GET | /product/{id} | 404 | ❌ |
| deleteProduct | DELETE | /product/{id} | 200 | ❌ |
| deleteProduct | DELETE | /product/{id} | 400 | ❌ |
| deleteProduct | DELETE | /product/{id} | 401 | ❌ |
| deleteProduct | DELETE | /product/{id} | 404 | ❌ |
## 🚨 Coverage Gaps & Recommendations
- **Missing Test Case:** `GET /products` – 401 Unauthorized
**Recommendation:** Add a Pact interaction with status `401` and validate that the client throws or handles the error appropriately.
- **Missing Test Case:** `GET /product/{id}` – 200 OK
**Recommendation:** Create a Pact test that returns a single product and ensure `API.getProduct(id)` maps the response to a `Product` instance correctly.
- **Missing Test Case:** `GET /product/{id}` – 400 Bad Request
**Recommendation:** Write a Pact test where an invalid ID is supplied, expecting a `400` response, and confirm that the client handles it gracefully.
- **Missing Test Case:** `GET /product/{id}` – 401 Unauthorized
**Recommendation:** Add a Pact test for `401` status and verify that authentication failures are surfaced properly.
- **Missing Test Case:** `GET /product/{id}` – 404 Not Found
**Recommendation:** Include a Pact test that simulates a non‑existent product, ensuring the client throws a `404` error or returns `null`.
- **Missing Test Case:** `DELETE /product/{id}` – 200 OK
**Recommendation:** Implement a Pact interaction that deletes a product and returns `200`; confirm the client's `deleteProduct` method completes without error.
- **Missing Test Case:** `DELETE /product/{id}` – 400 Bad Request
**Recommendation:** Test how the client reacts to a malformed request with a `400` response.
- **Missing Test Case:** `DELETE /product/{id}` – 401 Unauthorized
**Recommendation:** Add a Pact test for `401` to ensure proper handling of authentication failures during deletion.
- **Missing Test Case:** `DELETE /product/{id}` – 404 Not Found
**Recommendation:** Create a Pact scenario where the product to delete does not exist, verifying that the client handles the `404` status correctly.
So you could potentially get away with free tools in GH Actions. If you have claude, openapi or copilot you could use those too (e.g. opencode + copilot)