feat: Import 35+ skills, merge duplicates, add openclaw installer
Major updates: - Added 35+ new skills from awesome-opencode-skills and antigravity repos - Merged SEO skills into seo-master - Merged architecture skills into architecture - Merged security skills into security-auditor and security-coder - Merged testing skills into testing-master and testing-patterns - Merged pentesting skills into pentesting - Renamed website-creator to thai-frontend-dev - Replaced skill-creator with github version - Removed Chutes references (use MiniMax API instead) - Added install-openclaw-skills.sh for cross-platform installation - Updated .env.example with MiniMax API credentials
This commit is contained in:
313
skills/testing-master/SKILL.md
Normal file
313
skills/testing-master/SKILL.md
Normal file
@@ -0,0 +1,313 @@
|
||||
---
|
||||
name: testing-master
|
||||
description: |
|
||||
Master testing skill combining TDD, E2E testing, Playwright, and webapp testing.
|
||||
Use when writing tests, setting up testing infrastructure, or fixing failing tests.
|
||||
---
|
||||
|
||||
# Testing Master
|
||||
|
||||
Comprehensive testing skill combining: TDD, E2E testing, Playwright, webapp testing, and testing patterns.
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Task | Use Section |
|
||||
|------|-------------|
|
||||
| Write tests first | **TDD** |
|
||||
| E2E browser tests | **E2E Testing** |
|
||||
| Playwright automation | **Playwright** |
|
||||
| Fix failing tests | **Test Fixing** |
|
||||
| Generate unit tests | **Unit Testing** |
|
||||
| Web app testing | **Webapp Testing** |
|
||||
|
||||
---
|
||||
|
||||
## Test-Driven Development (TDD)
|
||||
|
||||
### Core Principle
|
||||
> "Write the test first. Watch it fail. Write minimal code to pass."
|
||||
|
||||
### The Iron Law
|
||||
```
|
||||
NO PRODUCTION CODE WITHOUT A FAILING TEST FIRST
|
||||
```
|
||||
|
||||
### TDD Cycle (Red-Green-Refactor)
|
||||
1. **RED** - Write a failing test
|
||||
2. **GREEN** - Write minimal code to pass
|
||||
3. **REFACTOR** - Improve code while keeping tests passing
|
||||
|
||||
### When to Use TDD
|
||||
- ✅ New features
|
||||
- ✅ Bug fixes
|
||||
- ✅ Refactoring
|
||||
- ✅ Behavior changes
|
||||
- ❌ Throwaway prototypes
|
||||
- ❌ Generated code
|
||||
- ❌ Configuration files
|
||||
|
||||
### Example: TDD Cycle
|
||||
```javascript
|
||||
// 1. RED - Write failing test
|
||||
describe('Calculator', () => {
|
||||
it('should add two numbers', () => {
|
||||
const calc = new Calculator();
|
||||
expect(calc.add(2, 3)).toBe(5);
|
||||
});
|
||||
});
|
||||
|
||||
// 2. GREEN - Minimal implementation
|
||||
class Calculator {
|
||||
add(a, b) {
|
||||
return a + b;
|
||||
}
|
||||
}
|
||||
|
||||
// 3. REFACTOR - Improve (if needed)
|
||||
```
|
||||
|
||||
### Test Naming Conventions
|
||||
```javascript
|
||||
describe('UserService', () => {
|
||||
it('should return null for non-existent user', () => {});
|
||||
it('should hash password before storing', () => {});
|
||||
it('should throw ValidationError for invalid email', () => {});
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## E2E Testing
|
||||
|
||||
### Playwright Setup
|
||||
```bash
|
||||
npm init playwright@latest
|
||||
npx playwright install chromium
|
||||
```
|
||||
|
||||
### Basic Test Structure
|
||||
```javascript
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test.describe('Login Flow', () => {
|
||||
test.beforeEach(async ({ page }) => {
|
||||
await page.goto('/login');
|
||||
});
|
||||
|
||||
test('should login with valid credentials', async ({ page }) => {
|
||||
await page.fill('[name="email"]', 'user@example.com');
|
||||
await page.fill('[name="password"]', 'password123');
|
||||
await page.click('[type="submit"]');
|
||||
await expect(page).toHaveURL('/dashboard');
|
||||
});
|
||||
|
||||
test('should show error for invalid credentials', async ({ page }) => {
|
||||
await page.fill('[name="email"]', 'invalid@example.com');
|
||||
await page.fill('[name="password"]', 'wrong');
|
||||
await page.click('[type="submit"]');
|
||||
await expect(page.locator('.error')).toContainText('Invalid credentials');
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### Page Object Model
|
||||
```javascript
|
||||
// pages/LoginPage.js
|
||||
class LoginPage {
|
||||
constructor(page) {
|
||||
this.page = page;
|
||||
this.emailInput = page.locator('[name="email"]');
|
||||
this.passwordInput = page.locator('[name="password"]');
|
||||
this.submitButton = page.locator('[type="submit"]');
|
||||
}
|
||||
|
||||
async login(email, password) {
|
||||
await this.emailInput.fill(email);
|
||||
await this.passwordInput.fill(password);
|
||||
await this.submitButton.click();
|
||||
}
|
||||
}
|
||||
|
||||
// test.js
|
||||
test('login', async ({ page }) => {
|
||||
const loginPage = new LoginPage(page);
|
||||
await loginPage.login('user@example.com', 'password123');
|
||||
});
|
||||
```
|
||||
|
||||
### Visual Regression Testing
|
||||
```javascript
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test('homepage visual regression', async ({ page }) => {
|
||||
await page.goto('/');
|
||||
const screenshot = await page.screenshot();
|
||||
expect(screenshot).toMatchSnapshot('homepage.png');
|
||||
});
|
||||
```
|
||||
|
||||
### Cross-Browser Testing
|
||||
```javascript
|
||||
test.describe('Browser Compatibility', () => {
|
||||
test('works on Chrome', async ({ browserName }) => {
|
||||
test.skip(browserName !== 'chromium', 'Skip on non-Chrome');
|
||||
// Chrome-specific test
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Playwright Testing
|
||||
|
||||
### Locators (Priority Order)
|
||||
1. **role** - `getByRole('button', { name: 'Submit' })`
|
||||
2. **label** - `getByLabel('Email')`
|
||||
3. **placeholder** - `getByPlaceholder('Enter email')`
|
||||
4. **text** - `getByText('Sign in')`
|
||||
5. **testid** - `getByTestId('submit-btn')`
|
||||
6. **CSS** - `locator('.submit')`
|
||||
7. **XPath** - `locator('//button[@type="submit"]')`
|
||||
|
||||
### Common Actions
|
||||
```javascript
|
||||
// Click
|
||||
await page.click('button');
|
||||
|
||||
// Fill input
|
||||
await page.fill('input[name="email"]', 'test@example.com');
|
||||
|
||||
// Select
|
||||
await page.selectOption('select', 'Option 1');
|
||||
|
||||
// Checkbox
|
||||
await page.check('input[type="checkbox"]');
|
||||
|
||||
// Hover
|
||||
await page.hover('.dropdown');
|
||||
|
||||
// Drag and drop
|
||||
await page.dragAndDrop('.source', '.target');
|
||||
```
|
||||
|
||||
### Assertions
|
||||
```javascript
|
||||
expect(locator).toBeVisible()
|
||||
expect(locator).toBeHidden()
|
||||
expect(locator).toBeEnabled()
|
||||
expect(locator).toBeDisabled()
|
||||
expect(locator).toHaveText('expected')
|
||||
expect(locator).toContainText('partial')
|
||||
expect(locator).toHaveValue('value')
|
||||
expect(locator).toHaveCount(5)
|
||||
expect(page).toHaveURL('**/dashboard')
|
||||
expect(page).toHaveTitle('Dashboard')
|
||||
```
|
||||
|
||||
### Network Interception
|
||||
```javascript
|
||||
await page.route('**/api/**', route => {
|
||||
route.fulfill({
|
||||
status: 200,
|
||||
body: JSON.stringify({ data: 'mocked' }),
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### File Upload
|
||||
```javascript
|
||||
await page.setInputFiles('input[type="file"]', 'path/to/file.pdf');
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Webapp Testing
|
||||
|
||||
### Test Coverage Checklist
|
||||
- [ ] Homepage loads
|
||||
- [ ] Navigation works
|
||||
- [ ] Forms submit correctly
|
||||
- [ ] Validation messages appear
|
||||
- [ ] Error states handled
|
||||
- [ ] Loading states work
|
||||
- [ ] Authentication flows
|
||||
- [ ] Responsive design
|
||||
- [ ] Accessibility (a11y)
|
||||
- [ ] Performance
|
||||
|
||||
### Accessibility Testing
|
||||
```javascript
|
||||
test('accessibility check', async ({ page }) => {
|
||||
await page.goto('/');
|
||||
|
||||
// Check for accessibility violations
|
||||
const violations = await new AxeBuilder({ page }).analyze();
|
||||
expect(violations.length).toBe(0);
|
||||
});
|
||||
```
|
||||
|
||||
### Mobile Testing
|
||||
```javascript
|
||||
test('mobile responsive', async ({ page }) => {
|
||||
await page.setViewportSize({ width: 375, height: 667 });
|
||||
await page.goto('/');
|
||||
// Mobile-specific assertions
|
||||
});
|
||||
```
|
||||
|
||||
### API Testing
|
||||
```javascript
|
||||
test('API integration', async ({ request }) => {
|
||||
const response = await request.get('https://api.example.com/users');
|
||||
expect(response.status()).toBe(200);
|
||||
expect(response.ok()).toBeTruthy();
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Test Fixing
|
||||
|
||||
### Debugging Failing Tests
|
||||
1. **Read the error** - What is it saying?
|
||||
2. **Check the test** - Is it testing the right thing?
|
||||
3. **Check the code** - Is the implementation correct?
|
||||
4. **Check the mocks** - Are they set up correctly?
|
||||
5. **Check the data** - Is the test data valid?
|
||||
|
||||
### Common Issues
|
||||
| Issue | Solution |
|
||||
|-------|----------|
|
||||
| Flaky tests | Add retries, stabilize timing |
|
||||
| Race conditions | Add waits, use explicit conditions |
|
||||
| Environment differences | Use Docker, consistent setup |
|
||||
| Data dependencies | Use fixtures, clean state |
|
||||
|
||||
### Test Isolation
|
||||
```javascript
|
||||
// Bad - shared state
|
||||
let user;
|
||||
test('creates user', () => { user = createUser(); });
|
||||
test('modifies user', () => { modifyUser(user.id); });
|
||||
|
||||
// Good - isolated
|
||||
test('creates and modifies user', () => {
|
||||
const user = createUser();
|
||||
modifyUser(user.id);
|
||||
// assertions
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **AAA Pattern** - Arrange, Act, Assert
|
||||
2. **One Assertion** - Per test when possible
|
||||
3. **Descriptive Names** - `it('should return 404 for missing resource')`
|
||||
4. **Fast Tests** - Mock external dependencies
|
||||
5. **Independent Tests** - No order dependency
|
||||
6. **Real Data** - Don't over-mock
|
||||
7. **Coverage** - Aim for meaningful coverage, not 100%
|
||||
614
skills/testing-master/resources/implementation-playbook.md
Normal file
614
skills/testing-master/resources/implementation-playbook.md
Normal file
@@ -0,0 +1,614 @@
|
||||
# Bats Testing Patterns Implementation Playbook
|
||||
|
||||
This file contains detailed patterns, checklists, and code samples referenced by the skill.
|
||||
|
||||
## Bats Fundamentals
|
||||
|
||||
### What is Bats?
|
||||
|
||||
Bats (Bash Automated Testing System) is a TAP (Test Anything Protocol) compliant testing framework for shell scripts that provides:
|
||||
- Simple, natural test syntax
|
||||
- TAP output format compatible with CI systems
|
||||
- Fixtures and setup/teardown support
|
||||
- Assertion helpers
|
||||
- Parallel test execution
|
||||
|
||||
### Installation
|
||||
|
||||
```bash
|
||||
# macOS with Homebrew
|
||||
brew install bats-core
|
||||
|
||||
# Ubuntu/Debian
|
||||
git clone https://github.com/bats-core/bats-core.git
|
||||
cd bats-core
|
||||
./install.sh /usr/local
|
||||
|
||||
# From npm (Node.js)
|
||||
npm install --global bats
|
||||
|
||||
# Verify installation
|
||||
bats --version
|
||||
```
|
||||
|
||||
### File Structure
|
||||
|
||||
```
|
||||
project/
|
||||
├── bin/
|
||||
│ ├── script.sh
|
||||
│ └── helper.sh
|
||||
├── tests/
|
||||
│ ├── test_script.bats
|
||||
│ ├── test_helper.sh
|
||||
│ ├── fixtures/
|
||||
│ │ ├── input.txt
|
||||
│ │ └── expected_output.txt
|
||||
│ └── helpers/
|
||||
│ └── mocks.bash
|
||||
└── README.md
|
||||
```
|
||||
|
||||
## Basic Test Structure
|
||||
|
||||
### Simple Test File
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bats
|
||||
|
||||
# Load test helper if present
|
||||
load test_helper
|
||||
|
||||
# Setup runs before each test
|
||||
setup() {
|
||||
export TMPDIR=$(mktemp -d)
|
||||
}
|
||||
|
||||
# Teardown runs after each test
|
||||
teardown() {
|
||||
rm -rf "$TMPDIR"
|
||||
}
|
||||
|
||||
# Test: simple assertion
|
||||
@test "Function returns 0 on success" {
|
||||
run my_function "input"
|
||||
[ "$status" -eq 0 ]
|
||||
}
|
||||
|
||||
# Test: output verification
|
||||
@test "Function outputs correct result" {
|
||||
run my_function "test"
|
||||
[ "$output" = "expected output" ]
|
||||
}
|
||||
|
||||
# Test: error handling
|
||||
@test "Function returns 1 on missing argument" {
|
||||
run my_function
|
||||
[ "$status" -eq 1 ]
|
||||
}
|
||||
```
|
||||
|
||||
## Assertion Patterns
|
||||
|
||||
### Exit Code Assertions
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bats
|
||||
|
||||
@test "Command succeeds" {
|
||||
run true
|
||||
[ "$status" -eq 0 ]
|
||||
}
|
||||
|
||||
@test "Command fails as expected" {
|
||||
run false
|
||||
[ "$status" -ne 0 ]
|
||||
}
|
||||
|
||||
@test "Command returns specific exit code" {
|
||||
run my_function --invalid
|
||||
[ "$status" -eq 127 ]
|
||||
}
|
||||
|
||||
@test "Can capture command result" {
|
||||
run echo "hello"
|
||||
[ $status -eq 0 ]
|
||||
[ "$output" = "hello" ]
|
||||
}
|
||||
```
|
||||
|
||||
### Output Assertions
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bats
|
||||
|
||||
@test "Output matches string" {
|
||||
result=$(echo "hello world")
|
||||
[ "$result" = "hello world" ]
|
||||
}
|
||||
|
||||
@test "Output contains substring" {
|
||||
result=$(echo "hello world")
|
||||
[[ "$result" == *"world"* ]]
|
||||
}
|
||||
|
||||
@test "Output matches pattern" {
|
||||
result=$(date +%Y)
|
||||
[[ "$result" =~ ^[0-9]{4}$ ]]
|
||||
}
|
||||
|
||||
@test "Multi-line output" {
|
||||
run printf "line1\nline2\nline3"
|
||||
[ "$output" = "line1
|
||||
line2
|
||||
line3" ]
|
||||
}
|
||||
|
||||
@test "Lines variable contains output" {
|
||||
run printf "line1\nline2\nline3"
|
||||
[ "${lines[0]}" = "line1" ]
|
||||
[ "${lines[1]}" = "line2" ]
|
||||
[ "${lines[2]}" = "line3" ]
|
||||
}
|
||||
```
|
||||
|
||||
### File Assertions
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bats
|
||||
|
||||
@test "File is created" {
|
||||
[ ! -f "$TMPDIR/output.txt" ]
|
||||
my_function > "$TMPDIR/output.txt"
|
||||
[ -f "$TMPDIR/output.txt" ]
|
||||
}
|
||||
|
||||
@test "File contents match expected" {
|
||||
my_function > "$TMPDIR/output.txt"
|
||||
[ "$(cat "$TMPDIR/output.txt")" = "expected content" ]
|
||||
}
|
||||
|
||||
@test "File is readable" {
|
||||
touch "$TMPDIR/test.txt"
|
||||
[ -r "$TMPDIR/test.txt" ]
|
||||
}
|
||||
|
||||
@test "File has correct permissions" {
|
||||
touch "$TMPDIR/test.txt"
|
||||
chmod 644 "$TMPDIR/test.txt"
|
||||
[ "$(stat -f %OLp "$TMPDIR/test.txt")" = "644" ]
|
||||
}
|
||||
|
||||
@test "File size is correct" {
|
||||
echo -n "12345" > "$TMPDIR/test.txt"
|
||||
[ "$(wc -c < "$TMPDIR/test.txt")" -eq 5 ]
|
||||
}
|
||||
```
|
||||
|
||||
## Setup and Teardown Patterns
|
||||
|
||||
### Basic Setup and Teardown
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bats
|
||||
|
||||
setup() {
|
||||
# Create test directory
|
||||
TEST_DIR=$(mktemp -d)
|
||||
export TEST_DIR
|
||||
|
||||
# Source script under test
|
||||
source "${BATS_TEST_DIRNAME}/../bin/script.sh"
|
||||
}
|
||||
|
||||
teardown() {
|
||||
# Clean up temporary directory
|
||||
rm -rf "$TEST_DIR"
|
||||
}
|
||||
|
||||
@test "Test using TEST_DIR" {
|
||||
touch "$TEST_DIR/file.txt"
|
||||
[ -f "$TEST_DIR/file.txt" ]
|
||||
}
|
||||
```
|
||||
|
||||
### Setup with Resources
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bats
|
||||
|
||||
setup() {
|
||||
# Create directory structure
|
||||
mkdir -p "$TMPDIR/data/input"
|
||||
mkdir -p "$TMPDIR/data/output"
|
||||
|
||||
# Create test fixtures
|
||||
echo "line1" > "$TMPDIR/data/input/file1.txt"
|
||||
echo "line2" > "$TMPDIR/data/input/file2.txt"
|
||||
|
||||
# Initialize environment
|
||||
export DATA_DIR="$TMPDIR/data"
|
||||
export INPUT_DIR="$DATA_DIR/input"
|
||||
export OUTPUT_DIR="$DATA_DIR/output"
|
||||
}
|
||||
|
||||
teardown() {
|
||||
rm -rf "$TMPDIR/data"
|
||||
}
|
||||
|
||||
@test "Processes input files" {
|
||||
run my_process_script "$INPUT_DIR" "$OUTPUT_DIR"
|
||||
[ "$status" -eq 0 ]
|
||||
[ -f "$OUTPUT_DIR/file1.txt" ]
|
||||
}
|
||||
```
|
||||
|
||||
### Global Setup/Teardown
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bats
|
||||
|
||||
# Load shared setup from test_helper.sh
|
||||
load test_helper
|
||||
|
||||
# setup_file runs once before all tests
|
||||
setup_file() {
|
||||
export SHARED_RESOURCE=$(mktemp -d)
|
||||
echo "Expensive setup" > "$SHARED_RESOURCE/data.txt"
|
||||
}
|
||||
|
||||
# teardown_file runs once after all tests
|
||||
teardown_file() {
|
||||
rm -rf "$SHARED_RESOURCE"
|
||||
}
|
||||
|
||||
@test "First test uses shared resource" {
|
||||
[ -f "$SHARED_RESOURCE/data.txt" ]
|
||||
}
|
||||
|
||||
@test "Second test uses shared resource" {
|
||||
[ -d "$SHARED_RESOURCE" ]
|
||||
}
|
||||
```
|
||||
|
||||
## Mocking and Stubbing Patterns
|
||||
|
||||
### Function Mocking
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bats
|
||||
|
||||
# Mock external command
|
||||
my_external_tool() {
|
||||
echo "mocked output"
|
||||
return 0
|
||||
}
|
||||
|
||||
@test "Function uses mocked tool" {
|
||||
export -f my_external_tool
|
||||
run my_function
|
||||
[[ "$output" == *"mocked output"* ]]
|
||||
}
|
||||
```
|
||||
|
||||
### Command Stubbing
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bats
|
||||
|
||||
setup() {
|
||||
# Create stub directory
|
||||
STUBS_DIR="$TMPDIR/stubs"
|
||||
mkdir -p "$STUBS_DIR"
|
||||
|
||||
# Add to PATH
|
||||
export PATH="$STUBS_DIR:$PATH"
|
||||
}
|
||||
|
||||
create_stub() {
|
||||
local cmd="$1"
|
||||
local output="$2"
|
||||
local code="${3:-0}"
|
||||
|
||||
cat > "$STUBS_DIR/$cmd" <<EOF
|
||||
#!/bin/bash
|
||||
echo "$output"
|
||||
exit $code
|
||||
EOF
|
||||
chmod +x "$STUBS_DIR/$cmd"
|
||||
}
|
||||
|
||||
@test "Function works with stubbed curl" {
|
||||
create_stub curl "{ \"status\": \"ok\" }" 0
|
||||
run my_api_function
|
||||
[ "$status" -eq 0 ]
|
||||
}
|
||||
```
|
||||
|
||||
### Variable Stubbing
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bats
|
||||
|
||||
@test "Function handles environment override" {
|
||||
export MY_SETTING="override_value"
|
||||
run my_function
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" == *"override_value"* ]]
|
||||
}
|
||||
|
||||
@test "Function uses default when var unset" {
|
||||
unset MY_SETTING
|
||||
run my_function
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" == *"default"* ]]
|
||||
}
|
||||
```
|
||||
|
||||
## Fixture Management
|
||||
|
||||
### Using Fixture Files
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bats
|
||||
|
||||
# Fixture directory: tests/fixtures/
|
||||
|
||||
setup() {
|
||||
FIXTURES_DIR="${BATS_TEST_DIRNAME}/fixtures"
|
||||
WORK_DIR=$(mktemp -d)
|
||||
export WORK_DIR
|
||||
}
|
||||
|
||||
teardown() {
|
||||
rm -rf "$WORK_DIR"
|
||||
}
|
||||
|
||||
@test "Process fixture file" {
|
||||
# Copy fixture to work directory
|
||||
cp "$FIXTURES_DIR/input.txt" "$WORK_DIR/input.txt"
|
||||
|
||||
# Run function
|
||||
run my_process_function "$WORK_DIR/input.txt"
|
||||
|
||||
# Compare output
|
||||
diff "$WORK_DIR/output.txt" "$FIXTURES_DIR/expected_output.txt"
|
||||
}
|
||||
```
|
||||
|
||||
### Dynamic Fixture Generation
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bats
|
||||
|
||||
generate_fixture() {
|
||||
local lines="$1"
|
||||
local file="$2"
|
||||
|
||||
for i in $(seq 1 "$lines"); do
|
||||
echo "Line $i content" >> "$file"
|
||||
done
|
||||
}
|
||||
|
||||
@test "Handle large input file" {
|
||||
generate_fixture 1000 "$TMPDIR/large.txt"
|
||||
run my_function "$TMPDIR/large.txt"
|
||||
[ "$status" -eq 0 ]
|
||||
[ "$(wc -l < "$TMPDIR/large.txt")" -eq 1000 ]
|
||||
}
|
||||
```
|
||||
|
||||
## Advanced Patterns
|
||||
|
||||
### Testing Error Conditions
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bats
|
||||
|
||||
@test "Function fails with missing file" {
|
||||
run my_function "/nonexistent/file.txt"
|
||||
[ "$status" -ne 0 ]
|
||||
[[ "$output" == *"not found"* ]]
|
||||
}
|
||||
|
||||
@test "Function fails with invalid input" {
|
||||
run my_function ""
|
||||
[ "$status" -ne 0 ]
|
||||
}
|
||||
|
||||
@test "Function fails with permission denied" {
|
||||
touch "$TMPDIR/readonly.txt"
|
||||
chmod 000 "$TMPDIR/readonly.txt"
|
||||
run my_function "$TMPDIR/readonly.txt"
|
||||
[ "$status" -ne 0 ]
|
||||
chmod 644 "$TMPDIR/readonly.txt" # Cleanup
|
||||
}
|
||||
|
||||
@test "Function provides helpful error message" {
|
||||
run my_function --invalid-option
|
||||
[ "$status" -ne 0 ]
|
||||
[[ "$output" == *"Usage:"* ]]
|
||||
}
|
||||
```
|
||||
|
||||
### Testing with Dependencies
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bats
|
||||
|
||||
setup() {
|
||||
# Check for required tools
|
||||
if ! command -v jq &>/dev/null; then
|
||||
skip "jq is not installed"
|
||||
fi
|
||||
|
||||
export SCRIPT="${BATS_TEST_DIRNAME}/../bin/script.sh"
|
||||
}
|
||||
|
||||
@test "JSON parsing works" {
|
||||
skip_if ! command -v jq &>/dev/null
|
||||
run my_json_parser '{"key": "value"}'
|
||||
[ "$status" -eq 0 ]
|
||||
}
|
||||
```
|
||||
|
||||
### Testing Shell Compatibility
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bats
|
||||
|
||||
@test "Script works in bash" {
|
||||
bash "${BATS_TEST_DIRNAME}/../bin/script.sh" arg1
|
||||
}
|
||||
|
||||
@test "Script works in sh (POSIX)" {
|
||||
sh "${BATS_TEST_DIRNAME}/../bin/script.sh" arg1
|
||||
}
|
||||
|
||||
@test "Script works in dash" {
|
||||
if command -v dash &>/dev/null; then
|
||||
dash "${BATS_TEST_DIRNAME}/../bin/script.sh" arg1
|
||||
else
|
||||
skip "dash not installed"
|
||||
fi
|
||||
}
|
||||
```
|
||||
|
||||
### Parallel Execution
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bats
|
||||
|
||||
@test "Multiple independent operations" {
|
||||
run bash -c 'for i in {1..10}; do
|
||||
my_operation "$i" &
|
||||
done
|
||||
wait'
|
||||
[ "$status" -eq 0 ]
|
||||
}
|
||||
|
||||
@test "Concurrent file operations" {
|
||||
for i in {1..5}; do
|
||||
my_function "$TMPDIR/file$i" &
|
||||
done
|
||||
wait
|
||||
[ -f "$TMPDIR/file1" ]
|
||||
[ -f "$TMPDIR/file5" ]
|
||||
}
|
||||
```
|
||||
|
||||
## Test Helper Pattern
|
||||
|
||||
### test_helper.sh
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bash
|
||||
|
||||
# Source script under test
|
||||
export SCRIPT_DIR="${BATS_TEST_DIRNAME%/*}/bin"
|
||||
|
||||
# Common test utilities
|
||||
assert_file_exists() {
|
||||
if [ ! -f "$1" ]; then
|
||||
echo "Expected file to exist: $1"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
assert_file_equals() {
|
||||
local file="$1"
|
||||
local expected="$2"
|
||||
|
||||
if [ ! -f "$file" ]; then
|
||||
echo "File does not exist: $file"
|
||||
return 1
|
||||
fi
|
||||
|
||||
local actual=$(cat "$file")
|
||||
if [ "$actual" != "$expected" ]; then
|
||||
echo "File contents do not match"
|
||||
echo "Expected: $expected"
|
||||
echo "Actual: $actual"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Create temporary test directory
|
||||
setup_test_dir() {
|
||||
export TEST_DIR=$(mktemp -d)
|
||||
}
|
||||
|
||||
cleanup_test_dir() {
|
||||
rm -rf "$TEST_DIR"
|
||||
}
|
||||
```
|
||||
|
||||
## Integration with CI/CD
|
||||
|
||||
### GitHub Actions Workflow
|
||||
|
||||
```yaml
|
||||
name: Tests
|
||||
|
||||
on: [push, pull_request]
|
||||
|
||||
jobs:
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
|
||||
- name: Install Bats
|
||||
run: |
|
||||
npm install --global bats
|
||||
|
||||
- name: Run Tests
|
||||
run: |
|
||||
bats tests/*.bats
|
||||
|
||||
- name: Run Tests with Tap Reporter
|
||||
run: |
|
||||
bats tests/*.bats --tap | tee test_output.tap
|
||||
```
|
||||
|
||||
### Makefile Integration
|
||||
|
||||
```makefile
|
||||
.PHONY: test test-verbose test-tap
|
||||
|
||||
test:
|
||||
bats tests/*.bats
|
||||
|
||||
test-verbose:
|
||||
bats tests/*.bats --verbose
|
||||
|
||||
test-tap:
|
||||
bats tests/*.bats --tap
|
||||
|
||||
test-parallel:
|
||||
bats tests/*.bats --parallel 4
|
||||
|
||||
coverage: test
|
||||
# Optional: Generate coverage reports
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Test one thing per test** - Single responsibility principle
|
||||
2. **Use descriptive test names** - Clearly states what is being tested
|
||||
3. **Clean up after tests** - Always remove temporary files in teardown
|
||||
4. **Test both success and failure paths** - Don't just test happy path
|
||||
5. **Mock external dependencies** - Isolate unit under test
|
||||
6. **Use fixtures for complex data** - Makes tests more readable
|
||||
7. **Run tests in CI/CD** - Catch regressions early
|
||||
8. **Test across shell dialects** - Ensure portability
|
||||
9. **Keep tests fast** - Run in parallel when possible
|
||||
10. **Document complex test setup** - Explain unusual patterns
|
||||
|
||||
## Resources
|
||||
|
||||
- **Bats GitHub**: https://github.com/bats-core/bats-core
|
||||
- **Bats Documentation**: https://bats-core.readthedocs.io/
|
||||
- **TAP Protocol**: https://testanything.org/
|
||||
- **Test-Driven Development**: https://en.wikipedia.org/wiki/Test-driven_development
|
||||
299
skills/testing-master/testing-anti-patterns.md
Normal file
299
skills/testing-master/testing-anti-patterns.md
Normal file
@@ -0,0 +1,299 @@
|
||||
# Testing Anti-Patterns
|
||||
|
||||
**Load this reference when:** writing or changing tests, adding mocks, or tempted to add test-only methods to production code.
|
||||
|
||||
## Overview
|
||||
|
||||
Tests must verify real behavior, not mock behavior. Mocks are a means to isolate, not the thing being tested.
|
||||
|
||||
**Core principle:** Test what the code does, not what the mocks do.
|
||||
|
||||
**Following strict TDD prevents these anti-patterns.**
|
||||
|
||||
## The Iron Laws
|
||||
|
||||
```
|
||||
1. NEVER test mock behavior
|
||||
2. NEVER add test-only methods to production classes
|
||||
3. NEVER mock without understanding dependencies
|
||||
```
|
||||
|
||||
## Anti-Pattern 1: Testing Mock Behavior
|
||||
|
||||
**The violation:**
|
||||
```typescript
|
||||
// ❌ BAD: Testing that the mock exists
|
||||
test('renders sidebar', () => {
|
||||
render(<Page />);
|
||||
expect(screen.getByTestId('sidebar-mock')).toBeInTheDocument();
|
||||
});
|
||||
```
|
||||
|
||||
**Why this is wrong:**
|
||||
- You're verifying the mock works, not that the component works
|
||||
- Test passes when mock is present, fails when it's not
|
||||
- Tells you nothing about real behavior
|
||||
|
||||
**your human partner's correction:** "Are we testing the behavior of a mock?"
|
||||
|
||||
**The fix:**
|
||||
```typescript
|
||||
// ✅ GOOD: Test real component or don't mock it
|
||||
test('renders sidebar', () => {
|
||||
render(<Page />); // Don't mock sidebar
|
||||
expect(screen.getByRole('navigation')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
// OR if sidebar must be mocked for isolation:
|
||||
// Don't assert on the mock - test Page's behavior with sidebar present
|
||||
```
|
||||
|
||||
### Gate Function
|
||||
|
||||
```
|
||||
BEFORE asserting on any mock element:
|
||||
Ask: "Am I testing real component behavior or just mock existence?"
|
||||
|
||||
IF testing mock existence:
|
||||
STOP - Delete the assertion or unmock the component
|
||||
|
||||
Test real behavior instead
|
||||
```
|
||||
|
||||
## Anti-Pattern 2: Test-Only Methods in Production
|
||||
|
||||
**The violation:**
|
||||
```typescript
|
||||
// ❌ BAD: destroy() only used in tests
|
||||
class Session {
|
||||
async destroy() { // Looks like production API!
|
||||
await this._workspaceManager?.destroyWorkspace(this.id);
|
||||
// ... cleanup
|
||||
}
|
||||
}
|
||||
|
||||
// In tests
|
||||
afterEach(() => session.destroy());
|
||||
```
|
||||
|
||||
**Why this is wrong:**
|
||||
- Production class polluted with test-only code
|
||||
- Dangerous if accidentally called in production
|
||||
- Violates YAGNI and separation of concerns
|
||||
- Confuses object lifecycle with entity lifecycle
|
||||
|
||||
**The fix:**
|
||||
```typescript
|
||||
// ✅ GOOD: Test utilities handle test cleanup
|
||||
// Session has no destroy() - it's stateless in production
|
||||
|
||||
// In test-utils/
|
||||
export async function cleanupSession(session: Session) {
|
||||
const workspace = session.getWorkspaceInfo();
|
||||
if (workspace) {
|
||||
await workspaceManager.destroyWorkspace(workspace.id);
|
||||
}
|
||||
}
|
||||
|
||||
// In tests
|
||||
afterEach(() => cleanupSession(session));
|
||||
```
|
||||
|
||||
### Gate Function
|
||||
|
||||
```
|
||||
BEFORE adding any method to production class:
|
||||
Ask: "Is this only used by tests?"
|
||||
|
||||
IF yes:
|
||||
STOP - Don't add it
|
||||
Put it in test utilities instead
|
||||
|
||||
Ask: "Does this class own this resource's lifecycle?"
|
||||
|
||||
IF no:
|
||||
STOP - Wrong class for this method
|
||||
```
|
||||
|
||||
## Anti-Pattern 3: Mocking Without Understanding
|
||||
|
||||
**The violation:**
|
||||
```typescript
|
||||
// ❌ BAD: Mock breaks test logic
|
||||
test('detects duplicate server', () => {
|
||||
// Mock prevents config write that test depends on!
|
||||
vi.mock('ToolCatalog', () => ({
|
||||
discoverAndCacheTools: vi.fn().mockResolvedValue(undefined)
|
||||
}));
|
||||
|
||||
await addServer(config);
|
||||
await addServer(config); // Should throw - but won't!
|
||||
});
|
||||
```
|
||||
|
||||
**Why this is wrong:**
|
||||
- Mocked method had side effect test depended on (writing config)
|
||||
- Over-mocking to "be safe" breaks actual behavior
|
||||
- Test passes for wrong reason or fails mysteriously
|
||||
|
||||
**The fix:**
|
||||
```typescript
|
||||
// ✅ GOOD: Mock at correct level
|
||||
test('detects duplicate server', () => {
|
||||
// Mock the slow part, preserve behavior test needs
|
||||
vi.mock('MCPServerManager'); // Just mock slow server startup
|
||||
|
||||
await addServer(config); // Config written
|
||||
await addServer(config); // Duplicate detected ✓
|
||||
});
|
||||
```
|
||||
|
||||
### Gate Function
|
||||
|
||||
```
|
||||
BEFORE mocking any method:
|
||||
STOP - Don't mock yet
|
||||
|
||||
1. Ask: "What side effects does the real method have?"
|
||||
2. Ask: "Does this test depend on any of those side effects?"
|
||||
3. Ask: "Do I fully understand what this test needs?"
|
||||
|
||||
IF depends on side effects:
|
||||
Mock at lower level (the actual slow/external operation)
|
||||
OR use test doubles that preserve necessary behavior
|
||||
NOT the high-level method the test depends on
|
||||
|
||||
IF unsure what test depends on:
|
||||
Run test with real implementation FIRST
|
||||
Observe what actually needs to happen
|
||||
THEN add minimal mocking at the right level
|
||||
|
||||
Red flags:
|
||||
- "I'll mock this to be safe"
|
||||
- "This might be slow, better mock it"
|
||||
- Mocking without understanding the dependency chain
|
||||
```
|
||||
|
||||
## Anti-Pattern 4: Incomplete Mocks
|
||||
|
||||
**The violation:**
|
||||
```typescript
|
||||
// ❌ BAD: Partial mock - only fields you think you need
|
||||
const mockResponse = {
|
||||
status: 'success',
|
||||
data: { userId: '123', name: 'Alice' }
|
||||
// Missing: metadata that downstream code uses
|
||||
};
|
||||
|
||||
// Later: breaks when code accesses response.metadata.requestId
|
||||
```
|
||||
|
||||
**Why this is wrong:**
|
||||
- **Partial mocks hide structural assumptions** - You only mocked fields you know about
|
||||
- **Downstream code may depend on fields you didn't include** - Silent failures
|
||||
- **Tests pass but integration fails** - Mock incomplete, real API complete
|
||||
- **False confidence** - Test proves nothing about real behavior
|
||||
|
||||
**The Iron Rule:** Mock the COMPLETE data structure as it exists in reality, not just fields your immediate test uses.
|
||||
|
||||
**The fix:**
|
||||
```typescript
|
||||
// ✅ GOOD: Mirror real API completeness
|
||||
const mockResponse = {
|
||||
status: 'success',
|
||||
data: { userId: '123', name: 'Alice' },
|
||||
metadata: { requestId: 'req-789', timestamp: 1234567890 }
|
||||
// All fields real API returns
|
||||
};
|
||||
```
|
||||
|
||||
### Gate Function
|
||||
|
||||
```
|
||||
BEFORE creating mock responses:
|
||||
Check: "What fields does the real API response contain?"
|
||||
|
||||
Actions:
|
||||
1. Examine actual API response from docs/examples
|
||||
2. Include ALL fields system might consume downstream
|
||||
3. Verify mock matches real response schema completely
|
||||
|
||||
Critical:
|
||||
If you're creating a mock, you must understand the ENTIRE structure
|
||||
Partial mocks fail silently when code depends on omitted fields
|
||||
|
||||
If uncertain: Include all documented fields
|
||||
```
|
||||
|
||||
## Anti-Pattern 5: Integration Tests as Afterthought
|
||||
|
||||
**The violation:**
|
||||
```
|
||||
✅ Implementation complete
|
||||
❌ No tests written
|
||||
"Ready for testing"
|
||||
```
|
||||
|
||||
**Why this is wrong:**
|
||||
- Testing is part of implementation, not optional follow-up
|
||||
- TDD would have caught this
|
||||
- Can't claim complete without tests
|
||||
|
||||
**The fix:**
|
||||
```
|
||||
TDD cycle:
|
||||
1. Write failing test
|
||||
2. Implement to pass
|
||||
3. Refactor
|
||||
4. THEN claim complete
|
||||
```
|
||||
|
||||
## When Mocks Become Too Complex
|
||||
|
||||
**Warning signs:**
|
||||
- Mock setup longer than test logic
|
||||
- Mocking everything to make test pass
|
||||
- Mocks missing methods real components have
|
||||
- Test breaks when mock changes
|
||||
|
||||
**your human partner's question:** "Do we need to be using a mock here?"
|
||||
|
||||
**Consider:** Integration tests with real components often simpler than complex mocks
|
||||
|
||||
## TDD Prevents These Anti-Patterns
|
||||
|
||||
**Why TDD helps:**
|
||||
1. **Write test first** → Forces you to think about what you're actually testing
|
||||
2. **Watch it fail** → Confirms test tests real behavior, not mocks
|
||||
3. **Minimal implementation** → No test-only methods creep in
|
||||
4. **Real dependencies** → You see what the test actually needs before mocking
|
||||
|
||||
**If you're testing mock behavior, you violated TDD** - you added mocks without watching test fail against real code first.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Anti-Pattern | Fix |
|
||||
|--------------|-----|
|
||||
| Assert on mock elements | Test real component or unmock it |
|
||||
| Test-only methods in production | Move to test utilities |
|
||||
| Mock without understanding | Understand dependencies first, mock minimally |
|
||||
| Incomplete mocks | Mirror real API completely |
|
||||
| Tests as afterthought | TDD - tests first |
|
||||
| Over-complex mocks | Consider integration tests |
|
||||
|
||||
## Red Flags
|
||||
|
||||
- Assertion checks for `*-mock` test IDs
|
||||
- Methods only called in test files
|
||||
- Mock setup is >50% of test
|
||||
- Test fails when you remove mock
|
||||
- Can't explain why mock is needed
|
||||
- Mocking "just to be safe"
|
||||
|
||||
## The Bottom Line
|
||||
|
||||
**Mocks are tools to isolate, not things to test.**
|
||||
|
||||
If TDD reveals you're testing mock behavior, you've gone wrong.
|
||||
|
||||
Fix: Test real behavior or question why you're mocking at all.
|
||||
Reference in New Issue
Block a user