Initial: pi-skill — 68 skills, 43 extensions, 11 themes for Pi
This commit is contained in:
177
skills/qa-automation/qa-test-flows/SKILL.md
Normal file
177
skills/qa-automation/qa-test-flows/SKILL.md
Normal file
@@ -0,0 +1,177 @@
|
||||
---
|
||||
name: qa-test-flows
|
||||
description: >
|
||||
Automated UI test flow framework using CDP + agent-device for native apps and
|
||||
agent-browser for web apps. Zero-framework approach — bash scripts orchestrate
|
||||
simulators with no Detox, Maestro, or Appium dependencies. Provides test lifecycle,
|
||||
screenshot capture, assertion helpers, and JSON report generation.
|
||||
Invoke when user says "run tests", "test the app", "smoke test", "regression test",
|
||||
"verify flows", "run e2e tests", or any task requiring automated UI testing.
|
||||
allowed-tools: Bash(agent-device:*) Bash(agent-browser:*) Bash(xcrun:*) Bash(adb:*) Bash(open:*) Bash(find:*) Bash(npx:*) Bash(node:*) Read
|
||||
---
|
||||
|
||||
# qa-test-flows
|
||||
|
||||
Automated UI test flow framework using a **dual-driver architecture**: **CDP** for navigation/state control and **agent-device** for screenshots/visual assertions in native apps, plus **agent-browser** for web app testing. This is a zero-framework approach — bash scripts orchestrate the entire test lifecycle with no Detox, Maestro, or Appium dependencies.
|
||||
|
||||
## Key Innovation: CDP + agent-device + agent-browser
|
||||
|
||||
Native mobile apps with complex gesture handlers (full-screen video players, swipe-based feeds) often make coordinate-based tapping unreliable. We solve this by connecting directly to the React Native Hermes runtime via Metro's CDP WebSocket, giving us:
|
||||
|
||||
- **Direct navigation control** via `navigationRef.current.navigate()` — no touch coordinates needed
|
||||
- **Runtime state inspection** — check current route, user state, storage
|
||||
- **Module access** via Metro's `__r()` require — load any app module at runtime
|
||||
- **Screenshots & visual verification** via agent-device (best-in-class for simulators)
|
||||
- **Web app testing** via agent-browser when the same app has a web version
|
||||
|
||||
## When to Use
|
||||
|
||||
- Smoke testing before submission or release builds
|
||||
- Regression testing after navigation or feature changes
|
||||
- Flow verification (auth, navigation, data entry, checkout)
|
||||
- Platform parity checks (iOS vs Android)
|
||||
- Pre-PR validation of multi-screen flows
|
||||
- State persistence verification
|
||||
|
||||
## Test Architecture
|
||||
|
||||
### Dual Driver System
|
||||
|
||||
| Driver | Purpose | Use For |
|
||||
|--------|---------|---------|
|
||||
| **CDP (WebSocket)** | Navigation, state queries, JS execution | React Native apps (Hermes runtime) |
|
||||
| **agent-device** | Screenshots, coordinate taps, swipes, accessibility | Native app simulators/emulators |
|
||||
| **agent-browser** | Full browser automation, DOM interaction | Web apps, PWAs, browser testing |
|
||||
|
||||
### Test Flow Format
|
||||
|
||||
All test flows follow a consistent bash template:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
source "$(dirname "$0")/../../lib/test-helpers.sh"
|
||||
source "$(dirname "$0")/../../lib/cdp-helpers.sh"
|
||||
|
||||
TEST_NAME="my-flow"
|
||||
setup_test "$TEST_NAME"
|
||||
|
||||
# Step 1: Navigate
|
||||
step "Navigate to target screen"
|
||||
cdp_navigate "TargetScreen"
|
||||
sleep 2
|
||||
take_screenshot "01-target-screen"
|
||||
assert_screenshot "01-target-screen"
|
||||
|
||||
# Step 2: Interact
|
||||
step "Perform action"
|
||||
tap 200 400
|
||||
sleep 1
|
||||
take_screenshot "02-after-action"
|
||||
|
||||
# Step 3: Verify
|
||||
step "Verify result"
|
||||
route=$(cdp_get_route)
|
||||
log_info "Route: $route"
|
||||
|
||||
teardown_test
|
||||
```
|
||||
|
||||
## Running Tests
|
||||
|
||||
### Single Test
|
||||
```bash
|
||||
bash .pi/skills/qa-automation/qa-test-flows/flows/smoke/example-smoke.sh
|
||||
```
|
||||
|
||||
### All Tests
|
||||
```bash
|
||||
bash .pi/skills/qa-automation/qa-test-flows/run-all.sh
|
||||
```
|
||||
|
||||
### With Logging
|
||||
```bash
|
||||
bash .pi/skills/qa-automation/qa-test-flows/flows/smoke/example-smoke.sh 2>&1 | tee /tmp/smoke.log
|
||||
```
|
||||
|
||||
## Writing New Tests
|
||||
|
||||
### 1. Copy the template
|
||||
```bash
|
||||
cp .pi/skills/qa-automation/qa-test-flows/templates/new-flow.sh.template \
|
||||
.pi/skills/qa-automation/qa-test-flows/flows/my-suite/my-test.sh
|
||||
chmod +x .pi/skills/qa-automation/qa-test-flows/flows/my-suite/my-test.sh
|
||||
```
|
||||
|
||||
### 2. Edit the template
|
||||
Replace `CUSTOMIZE` markers with your app-specific details.
|
||||
|
||||
### 3. Find coordinates
|
||||
```bash
|
||||
agent-device screenshot /tmp/debug.png
|
||||
open /tmp/debug.png # Measure tap targets
|
||||
```
|
||||
|
||||
### 4. Run and verify
|
||||
```bash
|
||||
bash .pi/skills/qa-automation/qa-test-flows/flows/my-suite/my-test.sh
|
||||
ls /tmp/qa-tests/screenshots/my-test/
|
||||
```
|
||||
|
||||
## Helper Libraries
|
||||
|
||||
### test-helpers.sh
|
||||
|
||||
| Function | Purpose |
|
||||
|----------|---------|
|
||||
| `setup_test "name"` | Initialize test, create directories |
|
||||
| `teardown_test` | Report results, cleanup |
|
||||
| `step "description"` | Log a numbered test step |
|
||||
| `log_pass/fail/info/warn` | Status logging |
|
||||
| `take_screenshot "name"` | Capture screenshot |
|
||||
| `tap x y` | Tap at coordinates |
|
||||
| `swipe x1 y1 x2 y2` | Swipe gesture |
|
||||
| `scroll_dir direction` | Scroll up/down/left/right |
|
||||
| `assert_app_foreground` | Verify app is running |
|
||||
| `assert_screenshot "name"` | Verify screenshot exists |
|
||||
| `assert_text_visible "text"` | Check accessibility tree |
|
||||
| `launch_app` | Launch/relaunch the app |
|
||||
| `close_app` | Terminate the app |
|
||||
|
||||
### cdp-helpers.sh
|
||||
|
||||
| Function | Purpose |
|
||||
|----------|---------|
|
||||
| `cdp_eval "expression"` | Execute JS in Hermes |
|
||||
| `cdp_eval_safe "expression"` | Eval with ErrorUtils suppression |
|
||||
| `cdp_navigate "Screen"` | Navigate to screen |
|
||||
| `cdp_navigate_tab "TabScreen"` | Navigate to tab |
|
||||
| `cdp_get_route` | Get current route name |
|
||||
| `cdp_get_state` | Get full nav state |
|
||||
| `cdp_go_back` | Go back |
|
||||
| `nav_explore/search/home/profile` | Quick tab navigation |
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
qa-test-flows/
|
||||
├── SKILL.md # This file
|
||||
├── lib/
|
||||
│ ├── test-helpers.sh # Core test framework
|
||||
│ └── cdp-helpers.sh # CDP interaction helpers
|
||||
├── flows/
|
||||
│ └── smoke/
|
||||
│ └── example-smoke.sh # Example smoke test
|
||||
├── templates/
|
||||
│ └── new-flow.sh.template # Template for new tests
|
||||
└── run-all.sh # Master test runner
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Problem | Solution |
|
||||
|---------|----------|
|
||||
| agent-device click hangs | Commands run in background with timeout — if still hanging, increase `DEVICE_CMD_TIMEOUT` |
|
||||
| Dev console overlay covers screen | Call `dismiss_error_overlay` from scroll-helpers, or use CDP to suppress LogBox |
|
||||
| Accessibility tree is sparse | React Native trees are thinner than web — prefer coordinates + screenshots |
|
||||
| CDP timeout | Check dev server is running: `curl $DEV_SERVER_HEALTH` |
|
||||
| Module ID changed after code update | Delete cache: `rm $NAV_MODULE_CACHE` — setup guard will re-scan |
|
||||
108
skills/qa-automation/qa-test-flows/flows/smoke/example-smoke.sh
Executable file
108
skills/qa-automation/qa-test-flows/flows/smoke/example-smoke.sh
Executable file
@@ -0,0 +1,108 @@
|
||||
#!/bin/bash
|
||||
# ╔═══════════════════════════════════════════════════════════════════╗
|
||||
# ║ Example Smoke Test — Quick Verification of Core Flows ║
|
||||
# ╠═══════════════════════════════════════════════════════════════════╣
|
||||
# ║ CUSTOMIZE: Replace screen names and navigation with your app's. ║
|
||||
# ║ ║
|
||||
# ║ Usage: bash .pi/skills/qa-automation/qa-test-flows/flows/smoke/example-smoke.sh
|
||||
# ╚═══════════════════════════════════════════════════════════════════╝
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
source "$SCRIPT_DIR/../../lib/test-helpers.sh"
|
||||
source "$SCRIPT_DIR/../../lib/cdp-helpers.sh"
|
||||
|
||||
TEST_NAME="smoke-test"
|
||||
setup_test "$TEST_NAME"
|
||||
|
||||
# ── Step 1: App Launch ───────────────────────────────────────────────
|
||||
step "Verify app is running"
|
||||
assert_app_foreground || {
|
||||
launch_app
|
||||
sleep 3
|
||||
}
|
||||
take_screenshot "01-app-launched"
|
||||
assert_screenshot "01-app-launched"
|
||||
log_pass "App is running"
|
||||
|
||||
# ── Step 2: Check Login State ────────────────────────────────────────
|
||||
step "Check authentication state"
|
||||
logged_in=$(cdp_is_logged_in 2>/dev/null || echo "unknown")
|
||||
log_info "Logged in: $logged_in"
|
||||
|
||||
if [ "$logged_in" = "false" ]; then
|
||||
log_warn "App is on login screen — some tests may be skipped"
|
||||
fi
|
||||
take_screenshot "02-auth-state"
|
||||
|
||||
# ── Step 3: Navigate Through Main Tabs ───────────────────────────────
|
||||
# CUSTOMIZE: Replace with your app's tab screens
|
||||
|
||||
step "Navigate to Tab 1 (Explore/Home)"
|
||||
nav_explore 2>/dev/null || tap_tab 1
|
||||
sleep 2
|
||||
take_screenshot "03-tab-1"
|
||||
route=$(cdp_get_route 2>/dev/null || echo "unknown")
|
||||
log_info "Route: $route"
|
||||
log_pass "Tab 1 loaded"
|
||||
|
||||
step "Navigate to Tab 2 (Search)"
|
||||
nav_search 2>/dev/null || tap_tab 2
|
||||
sleep 2
|
||||
take_screenshot "04-tab-2"
|
||||
route=$(cdp_get_route 2>/dev/null || echo "unknown")
|
||||
log_info "Route: $route"
|
||||
log_pass "Tab 2 loaded"
|
||||
|
||||
step "Navigate to Tab 3 (Home/Create)"
|
||||
nav_home 2>/dev/null || tap_tab 3
|
||||
sleep 2
|
||||
take_screenshot "05-tab-3"
|
||||
route=$(cdp_get_route 2>/dev/null || echo "unknown")
|
||||
log_info "Route: $route"
|
||||
log_pass "Tab 3 loaded"
|
||||
|
||||
step "Navigate to Tab 5 (Profile)"
|
||||
nav_profile 2>/dev/null || tap_tab 5
|
||||
sleep 2
|
||||
take_screenshot "06-tab-5"
|
||||
route=$(cdp_get_route 2>/dev/null || echo "unknown")
|
||||
log_info "Route: $route"
|
||||
log_pass "Tab 5 loaded"
|
||||
|
||||
# ── Step 4: Navigate to Detail Screen ────────────────────────────────
|
||||
# CUSTOMIZE: Replace with a screen in your app
|
||||
|
||||
step "Navigate to Settings (detail screen)"
|
||||
cdp_navigate "$SCREEN_SETTINGS" 2>/dev/null || {
|
||||
log_warn "CDP navigation failed — trying tap"
|
||||
tap $SETTINGS_BUTTON_X $SETTINGS_BUTTON_Y
|
||||
}
|
||||
sleep 2
|
||||
take_screenshot "07-settings"
|
||||
route=$(cdp_get_route 2>/dev/null || echo "unknown")
|
||||
log_info "Route: $route"
|
||||
|
||||
# ── Step 5: Go Back ──────────────────────────────────────────────────
|
||||
step "Navigate back"
|
||||
cdp_go_back 2>/dev/null || go_back
|
||||
sleep 2
|
||||
take_screenshot "08-back"
|
||||
route=$(cdp_get_route 2>/dev/null || echo "unknown")
|
||||
log_info "Route after back: $route"
|
||||
log_pass "Back navigation works"
|
||||
|
||||
# ── Step 6: Final State ─────────────────────────────────────────────
|
||||
step "Final state check"
|
||||
assert_app_foreground
|
||||
take_screenshot "09-final"
|
||||
log_pass "Smoke test complete — app is stable"
|
||||
|
||||
# ── Report ───────────────────────────────────────────────────────────
|
||||
teardown_test
|
||||
|
||||
echo ""
|
||||
echo "Smoke test completed!"
|
||||
echo " Screenshots: $SCREENSHOT_DIR/$TEST_NAME/"
|
||||
echo ""
|
||||
340
skills/qa-automation/qa-test-flows/lib/cdp-helpers.sh
Executable file
340
skills/qa-automation/qa-test-flows/lib/cdp-helpers.sh
Executable file
@@ -0,0 +1,340 @@
|
||||
#!/bin/bash
|
||||
# ╔═══════════════════════════════════════════════════════════════════╗
|
||||
# ║ cdp-helpers.sh — Chrome DevTools Protocol Helpers ║
|
||||
# ╠═══════════════════════════════════════════════════════════════════╣
|
||||
# ║ Source this file for CDP interaction with React Native apps: ║
|
||||
# ║ source "$(dirname "$0")/../../lib/cdp-helpers.sh" ║
|
||||
# ║ ║
|
||||
# ║ Provides: ║
|
||||
# ║ • CDP evaluation (cdp_eval, cdp_eval_safe) ║
|
||||
# ║ • Navigation (cdp_navigate, cdp_navigate_tab) ║
|
||||
# ║ • State queries (cdp_get_route, cdp_get_state) ║
|
||||
# ║ • Auto-discovery (CDP target, navigation module) ║
|
||||
# ║ ║
|
||||
# ║ Requires: Node.js, 'ws' npm package, Metro dev server running ║
|
||||
# ╚═══════════════════════════════════════════════════════════════════╝
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# ── Source configuration ─────────────────────────────────────────────
|
||||
CDP_HELPERS_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
CDP_QA_ROOT="$(cd "$CDP_HELPERS_DIR/../.." && pwd)"
|
||||
|
||||
# Only source config if not already loaded
|
||||
if [ -z "${QA_AUTOMATION_DIR:-}" ]; then
|
||||
source "$CDP_QA_ROOT/qa.config.sh"
|
||||
fi
|
||||
|
||||
# ── CDP State ────────────────────────────────────────────────────────
|
||||
_CDP_WS_URL_RESOLVED=""
|
||||
_CDP_NAV_MODULE_ID=""
|
||||
|
||||
# ── CDP Auto-Discovery ──────────────────────────────────────────────
|
||||
|
||||
# Ensure we have a CDP WebSocket URL
|
||||
_ensure_cdp_ws_url() {
|
||||
if [ -n "$_CDP_WS_URL_RESOLVED" ]; then
|
||||
echo "$_CDP_WS_URL_RESOLVED"
|
||||
return 0
|
||||
fi
|
||||
|
||||
local ws_url
|
||||
ws_url=$(qa_detect_cdp_ws_url 2>/dev/null || echo "")
|
||||
|
||||
if [ -n "$ws_url" ]; then
|
||||
_CDP_WS_URL_RESOLVED="$ws_url"
|
||||
echo "$ws_url"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Last resort: construct from config
|
||||
if [ "$CDP_WS_URL" != "auto" ] && [ -n "$CDP_WS_URL" ]; then
|
||||
_CDP_WS_URL_RESOLVED="$CDP_WS_URL"
|
||||
echo "$CDP_WS_URL"
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo ""
|
||||
return 1
|
||||
}
|
||||
|
||||
# Ensure we have a navigation module ID
|
||||
_ensure_nav_module_id() {
|
||||
if [ -n "$_CDP_NAV_MODULE_ID" ]; then
|
||||
echo "$_CDP_NAV_MODULE_ID"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Check cache
|
||||
if [ -f "$NAV_MODULE_CACHE" ]; then
|
||||
local cached
|
||||
cached=$(cat "$NAV_MODULE_CACHE" 2>/dev/null || echo "")
|
||||
if [ -n "$cached" ]; then
|
||||
_CDP_NAV_MODULE_ID="$cached"
|
||||
echo "$cached"
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
|
||||
# Auto-detect
|
||||
local module_id
|
||||
module_id=$(qa_detect_nav_module 2>/dev/null || echo "")
|
||||
|
||||
if [ -n "$module_id" ]; then
|
||||
_CDP_NAV_MODULE_ID="$module_id"
|
||||
echo "$module_id"
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo ""
|
||||
return 1
|
||||
}
|
||||
|
||||
# ── Core CDP Functions ───────────────────────────────────────────────
|
||||
|
||||
# Execute JavaScript in the React Native Hermes runtime via CDP.
|
||||
# Usage: cdp_eval "javascript expression"
|
||||
# Returns: the evaluated result
|
||||
cdp_eval() {
|
||||
local expression="$1"
|
||||
local ws_url
|
||||
ws_url=$(_ensure_cdp_ws_url)
|
||||
|
||||
if [ -z "$ws_url" ]; then
|
||||
echo '{"error":"no CDP WebSocket URL"}'
|
||||
return 1
|
||||
fi
|
||||
|
||||
cd "$PROJECT_DIR"
|
||||
node -e "
|
||||
const WebSocket=require('ws');
|
||||
const ws=new WebSocket('$ws_url');
|
||||
ws.on('open',()=>{
|
||||
ws.send(JSON.stringify({
|
||||
id:1,
|
||||
method:'Runtime.evaluate',
|
||||
params:{
|
||||
expression: \`$expression\`,
|
||||
returnByValue:true
|
||||
}
|
||||
}));
|
||||
});
|
||||
ws.on('message',d=>{
|
||||
const m=JSON.parse(d);
|
||||
if(m.id===1){
|
||||
if(m.result?.result?.value) console.log(m.result.result.value);
|
||||
else if(m.result?.exceptionDetails) console.error('CDP Error:', m.result.exceptionDetails.text);
|
||||
else console.log(JSON.stringify(m.result));
|
||||
ws.close();
|
||||
process.exit(0);
|
||||
}
|
||||
});
|
||||
ws.on('error',e=>{console.error('WS Error:',e.message);process.exit(1)});
|
||||
setTimeout(()=>{console.error('CDP Timeout');process.exit(1)},8000);
|
||||
" 2>&1
|
||||
}
|
||||
|
||||
# Safe CDP eval — suppresses ErrorUtils/LogBox during execution.
|
||||
# Use for any eval that touches Metro modules or could trigger side effects.
|
||||
# Usage: cdp_eval_safe "javascript expression"
|
||||
cdp_eval_safe() {
|
||||
local expression="$1"
|
||||
cdp_eval "
|
||||
(function() {
|
||||
var _origHandler = globalThis.ErrorUtils ? ErrorUtils.getGlobalHandler() : null;
|
||||
var _origCE = console.error;
|
||||
if (globalThis.ErrorUtils) ErrorUtils.setGlobalHandler(function() {});
|
||||
console.error = function() {};
|
||||
try {
|
||||
return (function() { $expression })();
|
||||
} finally {
|
||||
if (globalThis.ErrorUtils && _origHandler) ErrorUtils.setGlobalHandler(_origHandler);
|
||||
console.error = _origCE;
|
||||
}
|
||||
})();
|
||||
"
|
||||
}
|
||||
|
||||
# ── Navigation Functions ─────────────────────────────────────────────
|
||||
|
||||
# Navigate to a screen by name.
|
||||
# Usage: cdp_navigate "ScreenName" [params_json]
|
||||
cdp_navigate() {
|
||||
local screen="$1"
|
||||
local params="${2:-}"
|
||||
local nav_id
|
||||
nav_id=$(_ensure_nav_module_id)
|
||||
|
||||
if [ -z "$nav_id" ]; then
|
||||
echo '{"error":"navigation module not found"}'
|
||||
return 1
|
||||
fi
|
||||
|
||||
if [ -n "$params" ]; then
|
||||
cdp_eval "
|
||||
const ref = __r(${nav_id}).navigationRef;
|
||||
ref.current.navigate('${screen}', ${params});
|
||||
JSON.stringify({ ok: true, route: ref.current.getCurrentRoute()?.name });
|
||||
"
|
||||
else
|
||||
cdp_eval "
|
||||
const ref = __r(${nav_id}).navigationRef;
|
||||
ref.current.navigate('${screen}');
|
||||
JSON.stringify({ ok: true, route: ref.current.getCurrentRoute()?.name });
|
||||
"
|
||||
fi
|
||||
}
|
||||
|
||||
# Navigate to a bottom tab screen.
|
||||
# Usage: cdp_navigate_tab "ScreenName"
|
||||
cdp_navigate_tab() {
|
||||
local tab="$1"
|
||||
local nav_id
|
||||
nav_id=$(_ensure_nav_module_id)
|
||||
|
||||
if [ -z "$nav_id" ]; then
|
||||
echo '{"error":"navigation module not found"}'
|
||||
return 1
|
||||
fi
|
||||
|
||||
cdp_eval "
|
||||
const ref = __r(${nav_id}).navigationRef;
|
||||
ref.current.navigate('${TAB_NAVIGATOR_NAME}', { screen: '${tab}' });
|
||||
JSON.stringify({ ok: true, route: ref.current.getCurrentRoute()?.name });
|
||||
"
|
||||
}
|
||||
|
||||
# Get current route name.
|
||||
# Usage: route=$(cdp_get_route)
|
||||
cdp_get_route() {
|
||||
local nav_id
|
||||
nav_id=$(_ensure_nav_module_id)
|
||||
|
||||
if [ -z "$nav_id" ]; then
|
||||
echo "unknown"
|
||||
return 1
|
||||
fi
|
||||
|
||||
cdp_eval "
|
||||
const ref = __r(${nav_id}).navigationRef;
|
||||
ref.current.getCurrentRoute()?.name || 'unknown';
|
||||
"
|
||||
}
|
||||
|
||||
# Get full navigation state as JSON.
|
||||
cdp_get_state() {
|
||||
local nav_id
|
||||
nav_id=$(_ensure_nav_module_id)
|
||||
|
||||
if [ -z "$nav_id" ]; then
|
||||
echo '{"error":"navigation module not found"}'
|
||||
return 1
|
||||
fi
|
||||
|
||||
cdp_eval "
|
||||
const ref = __r(${nav_id}).navigationRef;
|
||||
const state = ref.current.getRootState();
|
||||
function getRoutes(s, depth) {
|
||||
let routes = [];
|
||||
if (s.routes) {
|
||||
for (const r of s.routes) {
|
||||
routes.push({name: r.name, depth});
|
||||
if (r.state) routes = routes.concat(getRoutes(r.state, depth+1));
|
||||
}
|
||||
}
|
||||
return routes;
|
||||
}
|
||||
JSON.stringify({
|
||||
current: ref.current.getCurrentRoute()?.name,
|
||||
routes: getRoutes(state, 0)
|
||||
});
|
||||
"
|
||||
}
|
||||
|
||||
# Go back in navigation.
|
||||
cdp_go_back() {
|
||||
local nav_id
|
||||
nav_id=$(_ensure_nav_module_id)
|
||||
|
||||
if [ -z "$nav_id" ]; then
|
||||
echo "error: no nav module"
|
||||
return 1
|
||||
fi
|
||||
|
||||
cdp_eval "
|
||||
const ref = __r(${nav_id}).navigationRef;
|
||||
if (ref.current.canGoBack()) {
|
||||
ref.current.goBack();
|
||||
'went back to: ' + ref.current.getCurrentRoute()?.name;
|
||||
} else {
|
||||
'cannot go back';
|
||||
}
|
||||
"
|
||||
}
|
||||
|
||||
# Reset navigation to initial state.
|
||||
cdp_reset_navigation() {
|
||||
local nav_id
|
||||
nav_id=$(_ensure_nav_module_id)
|
||||
|
||||
if [ -z "$nav_id" ]; then
|
||||
echo "error: no nav module"
|
||||
return 1
|
||||
fi
|
||||
|
||||
cdp_eval "
|
||||
const ref = __r(${nav_id}).navigationRef;
|
||||
ref.current.reset({
|
||||
index: 0,
|
||||
routes: [{ name: 'Main', state: { routes: [{ name: '${TAB_NAVIGATOR_NAME}' }] } }]
|
||||
});
|
||||
'reset to ${TAB_NAVIGATOR_NAME}';
|
||||
"
|
||||
}
|
||||
|
||||
# ── Convenience Tab Navigation ───────────────────────────────────────
|
||||
# These use the screen names from qa.config.sh
|
||||
|
||||
nav_tab_1() { [ -n "$SCREEN_EXPLORE" ] && cdp_navigate_tab "$SCREEN_EXPLORE" || echo "SCREEN_EXPLORE not configured"; }
|
||||
nav_tab_2() { [ -n "$SCREEN_SEARCH" ] && cdp_navigate_tab "$SCREEN_SEARCH" || echo "SCREEN_SEARCH not configured"; }
|
||||
nav_tab_3() { [ -n "$SCREEN_HOME" ] && cdp_navigate_tab "$SCREEN_HOME" || echo "SCREEN_HOME not configured"; }
|
||||
nav_tab_4() { echo "Tab 4 not configured — set SCREEN name and add to cdp-helpers.sh"; }
|
||||
nav_tab_5() { [ -n "$SCREEN_PROFILE" ] && cdp_navigate_tab "$SCREEN_PROFILE" || echo "SCREEN_PROFILE not configured"; }
|
||||
|
||||
# Generic by name
|
||||
nav_explore() { nav_tab_1; }
|
||||
nav_search() { nav_tab_2; }
|
||||
nav_home() { nav_tab_3; }
|
||||
nav_profile() { nav_tab_5; }
|
||||
|
||||
# ── Utility Functions ────────────────────────────────────────────────
|
||||
|
||||
# Check if user is logged in (heuristic: check current route)
|
||||
cdp_is_logged_in() {
|
||||
local route
|
||||
route=$(cdp_get_route 2>/dev/null || echo "unknown")
|
||||
if echo "$route" | grep -qiE "login|signup|auth|welcome|launch"; then
|
||||
echo "false"
|
||||
else
|
||||
echo "true"
|
||||
fi
|
||||
}
|
||||
|
||||
# Clear cached module IDs (force re-discovery on next call)
|
||||
cdp_clear_cache() {
|
||||
_CDP_WS_URL_RESOLVED=""
|
||||
_CDP_NAV_MODULE_ID=""
|
||||
[ -f "$NAV_MODULE_CACHE" ] && rm -f "$NAV_MODULE_CACHE" 2>/dev/null || true
|
||||
[ -f "$VIDEO_MODULE_CACHE" ] && rm -f "$VIDEO_MODULE_CACHE" 2>/dev/null || true
|
||||
echo "CDP cache cleared"
|
||||
}
|
||||
|
||||
# ── Export Functions ─────────────────────────────────────────────────
|
||||
|
||||
export -f cdp_eval cdp_eval_safe 2>/dev/null || true
|
||||
export -f cdp_navigate cdp_navigate_tab cdp_get_route cdp_get_state 2>/dev/null || true
|
||||
export -f cdp_go_back cdp_reset_navigation cdp_is_logged_in 2>/dev/null || true
|
||||
export -f nav_explore nav_search nav_home nav_profile 2>/dev/null || true
|
||||
export -f cdp_clear_cache 2>/dev/null || true
|
||||
|
||||
echo "CDP helpers loaded. WebSocket: ${_CDP_WS_URL_RESOLVED:-auto-detect}"
|
||||
453
skills/qa-automation/qa-test-flows/lib/test-helpers.sh
Executable file
453
skills/qa-automation/qa-test-flows/lib/test-helpers.sh
Executable file
@@ -0,0 +1,453 @@
|
||||
#!/bin/bash
|
||||
# ╔═══════════════════════════════════════════════════════════════════╗
|
||||
# ║ test-helpers.sh — Core Test Framework for QA Automation ║
|
||||
# ╠═══════════════════════════════════════════════════════════════════╣
|
||||
# ║ Source this file at the top of every test flow script: ║
|
||||
# ║ source "$(dirname "$0")/../../lib/test-helpers.sh" ║
|
||||
# ║ ║
|
||||
# ║ Provides: ║
|
||||
# ║ • Test lifecycle (setup_test, teardown_test, step) ║
|
||||
# ║ • Logging (log_pass, log_fail, log_info, log_warn) ║
|
||||
# ║ • agent-device wrappers (tap, swipe, scroll, screenshot) ║
|
||||
# ║ • Assertion functions (assert_app_foreground, etc.) ║
|
||||
# ║ • Navigation helpers (tap_tab, launch_app, close_app) ║
|
||||
# ╚═══════════════════════════════════════════════════════════════════╝
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# ── Source configuration ─────────────────────────────────────────────
|
||||
HELPERS_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
QA_ROOT="$(cd "$HELPERS_DIR/../.." && pwd)"
|
||||
source "$QA_ROOT/qa.config.sh"
|
||||
|
||||
# ── Auto-detect simulator UDID if set to "auto" ─────────────────────
|
||||
if [ "$SIMULATOR_UDID" = "auto" ]; then
|
||||
_detected_udid=$(qa_detect_simulator_udid 2>/dev/null || echo "")
|
||||
if [ -n "$_detected_udid" ]; then
|
||||
SIMULATOR_UDID="$_detected_udid"
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── Test State ───────────────────────────────────────────────────────
|
||||
TEST_NAME=""
|
||||
STEP_COUNT=0
|
||||
PASS_COUNT=0
|
||||
FAIL_COUNT=0
|
||||
TEST_START_TIME=""
|
||||
|
||||
# ── Initialization ───────────────────────────────────────────────────
|
||||
|
||||
init_test_env() {
|
||||
mkdir -p "$TEST_OUTPUT_DIR"
|
||||
mkdir -p "$SCREENSHOT_DIR"
|
||||
if [ ! -f "$RESULTS_FILE" ]; then
|
||||
echo "Test Run: $(date '+%Y-%m-%d %H:%M:%S')" > "$RESULTS_FILE"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" >> "$RESULTS_FILE"
|
||||
fi
|
||||
}
|
||||
|
||||
init_test_env
|
||||
|
||||
# ── Test Lifecycle ───────────────────────────────────────────────────
|
||||
|
||||
setup_test() {
|
||||
local name="$1"
|
||||
TEST_NAME="$name"
|
||||
TEST_START_TIME=$(date +%s)
|
||||
STEP_COUNT=0
|
||||
PASS_COUNT=0
|
||||
FAIL_COUNT=0
|
||||
mkdir -p "$SCREENSHOT_DIR/$TEST_NAME"
|
||||
echo ""
|
||||
echo "═══════════════════════════════════════════"
|
||||
echo "🧪 TEST: $TEST_NAME"
|
||||
echo "═══════════════════════════════════════════"
|
||||
echo "Started: $(date '+%Y-%m-%d %H:%M:%S')"
|
||||
echo ""
|
||||
}
|
||||
|
||||
teardown_test() {
|
||||
local name="${1:-$TEST_NAME}"
|
||||
local end_time=$(date +%s)
|
||||
local duration=$((end_time - TEST_START_TIME))
|
||||
echo ""
|
||||
echo "───────────────────────────────────────────"
|
||||
echo "📊 RESULTS: $name"
|
||||
echo " Steps: $STEP_COUNT | Passed: $PASS_COUNT | Failed: $FAIL_COUNT"
|
||||
echo " Duration: ${duration}s"
|
||||
if [ $FAIL_COUNT -eq 0 ]; then
|
||||
echo " Status: ✅ ALL PASSED"
|
||||
else
|
||||
echo " Status: ❌ $FAIL_COUNT FAILURES"
|
||||
fi
|
||||
echo "───────────────────────────────────────────"
|
||||
echo ""
|
||||
echo "$(date '+%Y-%m-%d %H:%M:%S') | $name | Steps:$STEP_COUNT Pass:$PASS_COUNT Fail:$FAIL_COUNT | ${duration}s" >> "$RESULTS_FILE"
|
||||
}
|
||||
|
||||
# ── Logging ──────────────────────────────────────────────────────────
|
||||
|
||||
step() {
|
||||
STEP_COUNT=$((STEP_COUNT + 1))
|
||||
echo " [$STEP_COUNT] $1"
|
||||
}
|
||||
|
||||
log_pass() {
|
||||
PASS_COUNT=$((PASS_COUNT + 1))
|
||||
echo " ✅ $1"
|
||||
}
|
||||
|
||||
log_fail() {
|
||||
FAIL_COUNT=$((FAIL_COUNT + 1))
|
||||
echo " ❌ $1"
|
||||
}
|
||||
|
||||
log_info() {
|
||||
echo " ℹ️ $1"
|
||||
}
|
||||
|
||||
log_warn() {
|
||||
echo " ⚠️ $1"
|
||||
}
|
||||
|
||||
# ── agent-device Wrappers ────────────────────────────────────────────
|
||||
|
||||
# Take a screenshot using xcrun simctl (avoids agent-device focus issues)
|
||||
# Falls back to agent-device if xcrun isn't available.
|
||||
# Usage: take_screenshot "step_name" [session]
|
||||
take_screenshot() {
|
||||
local name="$1"
|
||||
local session="${2:-$ACTIVE_SESSION}"
|
||||
local path="$SCREENSHOT_DIR/$TEST_NAME/${name}.png"
|
||||
mkdir -p "$(dirname "$path")"
|
||||
|
||||
# Prefer xcrun simctl for iOS (doesn't steal focus)
|
||||
if [ "$SIMULATOR_UDID" != "auto" ] && command -v xcrun >/dev/null 2>&1; then
|
||||
xcrun simctl io "$SIMULATOR_UDID" screenshot "$path" 2>/dev/null || {
|
||||
# Fallback to agent-device
|
||||
agent-device screenshot "$path" --session "$session" 2>/dev/null || {
|
||||
log_warn "Screenshot failed for: $name"
|
||||
return 1
|
||||
}
|
||||
}
|
||||
else
|
||||
agent-device screenshot "$path" --session "$session" 2>/dev/null || {
|
||||
log_warn "Screenshot failed for: $name"
|
||||
return 1
|
||||
}
|
||||
fi
|
||||
echo "$path"
|
||||
}
|
||||
|
||||
# Tap at coordinates with timeout handling
|
||||
# Usage: tap x y [session]
|
||||
tap() {
|
||||
local x="$1"
|
||||
local y="$2"
|
||||
local session="${3:-$ACTIVE_SESSION}"
|
||||
|
||||
agent-device click "$x" "$y" --session "$session" 2>/dev/null &
|
||||
local pid=$!
|
||||
|
||||
sleep 0.5
|
||||
if ps -p $pid > /dev/null 2>&1; then
|
||||
kill $pid 2>/dev/null || true
|
||||
wait $pid 2>/dev/null || true
|
||||
fi
|
||||
|
||||
sleep 1
|
||||
}
|
||||
|
||||
# Swipe gesture
|
||||
# Usage: swipe x1 y1 x2 y2 [session]
|
||||
swipe() {
|
||||
local x1="$1"
|
||||
local y1="$2"
|
||||
local x2="$3"
|
||||
local y2="$4"
|
||||
local session="${5:-$ACTIVE_SESSION}"
|
||||
|
||||
agent-device swipe "$x1" "$y1" "$x2" "$y2" --session "$session" 2>/dev/null &
|
||||
local pid=$!
|
||||
|
||||
sleep 0.5
|
||||
if ps -p $pid > /dev/null 2>&1; then
|
||||
kill $pid 2>/dev/null || true
|
||||
wait $pid 2>/dev/null || true
|
||||
fi
|
||||
|
||||
sleep 1
|
||||
}
|
||||
|
||||
# Scroll in a direction
|
||||
# Usage: scroll_dir direction [session]
|
||||
scroll_dir() {
|
||||
local direction="$1"
|
||||
local session="${2:-$ACTIVE_SESSION}"
|
||||
|
||||
agent-device scroll "$direction" --session "$session" 2>/dev/null &
|
||||
local pid=$!
|
||||
|
||||
sleep 0.5
|
||||
if ps -p $pid > /dev/null 2>&1; then
|
||||
kill $pid 2>/dev/null || true
|
||||
wait $pid 2>/dev/null || true
|
||||
fi
|
||||
|
||||
sleep 1
|
||||
}
|
||||
|
||||
# Type text into focused field
|
||||
# Usage: type_text ref_or_coords text [session]
|
||||
type_text() {
|
||||
local ref="$1"
|
||||
local text="$2"
|
||||
local session="${3:-$ACTIVE_SESSION}"
|
||||
|
||||
agent-device fill "$ref" "$text" --session "$session" 2>/dev/null &
|
||||
local pid=$!
|
||||
|
||||
sleep 0.5
|
||||
if ps -p $pid > /dev/null 2>&1; then
|
||||
kill $pid 2>/dev/null || true
|
||||
wait $pid 2>/dev/null || true
|
||||
fi
|
||||
|
||||
sleep 1
|
||||
}
|
||||
|
||||
# Press home button
|
||||
go_home() {
|
||||
local session="${1:-$ACTIVE_SESSION}"
|
||||
agent-device home --session "$session" 2>/dev/null &
|
||||
local pid=$!
|
||||
sleep 1
|
||||
if ps -p $pid > /dev/null 2>&1; then
|
||||
kill $pid 2>/dev/null || true
|
||||
wait $pid 2>/dev/null || true
|
||||
fi
|
||||
sleep 2
|
||||
}
|
||||
|
||||
# Go back
|
||||
go_back() {
|
||||
local session="${1:-$ACTIVE_SESSION}"
|
||||
agent-device back --session "$session" 2>/dev/null &
|
||||
local pid=$!
|
||||
sleep 1
|
||||
if ps -p $pid > /dev/null 2>&1; then
|
||||
kill $pid 2>/dev/null || true
|
||||
wait $pid 2>/dev/null || true
|
||||
fi
|
||||
sleep 2
|
||||
}
|
||||
|
||||
# Check if the app process is running on the simulator
|
||||
check_appstate() {
|
||||
local session="${1:-$ACTIVE_SESSION}"
|
||||
if [ "$SIMULATOR_UDID" != "auto" ] && command -v xcrun >/dev/null 2>&1; then
|
||||
xcrun simctl spawn "$SIMULATOR_UDID" launchctl list 2>/dev/null | grep "UIKitApplication:${APP_BUNDLE_ID}" || echo ""
|
||||
else
|
||||
agent-device appstate --session "$session" 2>/dev/null || echo ""
|
||||
fi
|
||||
}
|
||||
|
||||
# Get accessibility snapshot
|
||||
get_snapshot() {
|
||||
local session="${1:-$ACTIVE_SESSION}"
|
||||
local depth="${2:-5}"
|
||||
timeout 10 agent-device snapshot -i --depth "$depth" --session "$session" 2>/dev/null || echo "(snapshot timeout)"
|
||||
}
|
||||
|
||||
# ── Assertion Functions ──────────────────────────────────────────────
|
||||
|
||||
# Assert app is in foreground
|
||||
assert_app_foreground() {
|
||||
local session="${1:-$ACTIVE_SESSION}"
|
||||
local state=$(check_appstate "$session" 2>/dev/null | grep -o "$APP_BUNDLE_ID" || true)
|
||||
|
||||
if [ -n "$state" ]; then
|
||||
log_pass "App is in foreground ($APP_BUNDLE_ID)"
|
||||
return 0
|
||||
else
|
||||
log_fail "App is NOT in foreground"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Assert screenshot was captured
|
||||
assert_screenshot() {
|
||||
local name="$1"
|
||||
local path="$SCREENSHOT_DIR/$TEST_NAME/${name}.png"
|
||||
|
||||
if [ -f "$path" ] && [ -s "$path" ]; then
|
||||
log_pass "Screenshot captured: $name"
|
||||
return 0
|
||||
else
|
||||
log_fail "Screenshot missing or empty: $name"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Assert text is visible in accessibility tree
|
||||
assert_text_visible() {
|
||||
local text="$1"
|
||||
local session="${2:-$ACTIVE_SESSION}"
|
||||
local snapshot=$(get_snapshot "$session")
|
||||
|
||||
if echo "$snapshot" | grep -qi "$text"; then
|
||||
log_pass "Text visible: '$text'"
|
||||
return 0
|
||||
else
|
||||
log_warn "Text not found in accessibility tree: '$text'"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Assert element exists in accessibility tree
|
||||
assert_element_exists() {
|
||||
local label="$1"
|
||||
local session="${2:-$ACTIVE_SESSION}"
|
||||
local snapshot=$(get_snapshot "$session")
|
||||
|
||||
if echo "$snapshot" | grep -qi "label.*$label\|name.*$label"; then
|
||||
log_pass "Element found: '$label'"
|
||||
return 0
|
||||
else
|
||||
log_fail "Element not found: '$label'"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Navigation Helpers ───────────────────────────────────────────────
|
||||
|
||||
# Tap a tab by position (1-5)
|
||||
# Usage: tap_tab 1 (first tab)
|
||||
tap_tab_by_index() {
|
||||
local index="$1"
|
||||
local x_var="TAB_${index}_X"
|
||||
local x="${!x_var:-}"
|
||||
|
||||
if [ -z "$x" ]; then
|
||||
log_fail "No coordinate defined for tab index $index (set TAB_${index}_X)"
|
||||
return 1
|
||||
fi
|
||||
|
||||
tap "$x" "$TAB_BAR_Y"
|
||||
sleep 2
|
||||
}
|
||||
|
||||
# Tap a tab by name (reads screen names from config and maps to tab index)
|
||||
# Usage: tap_tab "explore"
|
||||
tap_tab() {
|
||||
local tab_name="$1"
|
||||
local tab_name_upper=$(echo "$tab_name" | tr '[:lower:]' '[:upper:]')
|
||||
|
||||
# Map common names to tab indices — customize in qa.config.sh
|
||||
case "$tab_name_upper" in
|
||||
EXPLORE|HOME|TAB1|1) tap_tab_by_index 1 ;;
|
||||
SEARCH|TAB2|2) tap_tab_by_index 2 ;;
|
||||
CREATE|TAB3|3) tap_tab_by_index 3 ;;
|
||||
WALLET|TAB4|4) tap_tab_by_index 4 ;;
|
||||
PROFILE|TAB5|5) tap_tab_by_index 5 ;;
|
||||
*)
|
||||
log_fail "Unknown tab: $tab_name"
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
# Launch the app fresh
|
||||
launch_app() {
|
||||
local session="${1:-$ACTIVE_SESSION}"
|
||||
step "Launching app ($APP_BUNDLE_ID)"
|
||||
|
||||
if [ "$SIMULATOR_UDID" != "auto" ] && command -v xcrun >/dev/null 2>&1; then
|
||||
xcrun simctl terminate "$SIMULATOR_UDID" "$APP_BUNDLE_ID" 2>/dev/null || true
|
||||
sleep 1
|
||||
xcrun simctl launch "$SIMULATOR_UDID" "$APP_BUNDLE_ID" 2>/dev/null || {
|
||||
log_fail "Failed to launch app via xcrun"
|
||||
return 1
|
||||
}
|
||||
else
|
||||
agent-device open "$APP_BUNDLE_ID" --session "$session" --relaunch 2>/dev/null || {
|
||||
log_fail "Failed to launch app"
|
||||
return 1
|
||||
}
|
||||
fi
|
||||
|
||||
sleep "$APP_SETTLE_TIME"
|
||||
log_pass "App launched"
|
||||
}
|
||||
|
||||
# Close the app
|
||||
close_app() {
|
||||
local session="${1:-$ACTIVE_SESSION}"
|
||||
if [ "$SIMULATOR_UDID" != "auto" ] && command -v xcrun >/dev/null 2>&1; then
|
||||
xcrun simctl terminate "$SIMULATOR_UDID" "$APP_BUNDLE_ID" 2>/dev/null || {
|
||||
log_warn "Could not terminate app via xcrun"
|
||||
return 1
|
||||
}
|
||||
else
|
||||
agent-device close --session "$session" 2>/dev/null || {
|
||||
log_warn "Could not close app"
|
||||
return 1
|
||||
}
|
||||
fi
|
||||
sleep 2
|
||||
}
|
||||
|
||||
# Wait for app to settle
|
||||
wait_settle() {
|
||||
local seconds="${1:-3}"
|
||||
sleep "$seconds"
|
||||
}
|
||||
|
||||
# ── Utility Functions ────────────────────────────────────────────────
|
||||
|
||||
# Print test summary
|
||||
print_test_summary() {
|
||||
echo ""
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
echo "📋 TEST EXECUTION SUMMARY"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
tail -20 "$RESULTS_FILE"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
}
|
||||
|
||||
log_debug() {
|
||||
local msg="$1"
|
||||
echo "[DEBUG] $(date '+%H:%M:%S') $msg" >> "$RESULTS_FILE"
|
||||
}
|
||||
|
||||
save_snapshot() {
|
||||
local desc="$1"
|
||||
local session="${2:-$ACTIVE_SESSION}"
|
||||
local filename="$(echo "$desc" | tr ' ' '_' | tr -cd '[:alnum:]._-')"
|
||||
local snap=$(get_snapshot "$session")
|
||||
echo "$snap" > "$SCREENSHOT_DIR/$TEST_NAME/${filename}_snapshot.txt"
|
||||
log_info "Saved snapshot: $filename"
|
||||
}
|
||||
|
||||
# ── Error Handling ───────────────────────────────────────────────────
|
||||
|
||||
trap 'teardown_test 2>/dev/null || true' EXIT
|
||||
|
||||
on_error() {
|
||||
local line=$1
|
||||
log_fail "Error on line $line"
|
||||
log_debug "Exit code: $?"
|
||||
}
|
||||
|
||||
trap 'on_error ${LINENO}' ERR
|
||||
|
||||
# ── Export Functions ─────────────────────────────────────────────────
|
||||
|
||||
export -f setup_test teardown_test step log_pass log_fail log_info log_warn 2>/dev/null || true
|
||||
export -f take_screenshot tap swipe scroll_dir type_text 2>/dev/null || true
|
||||
export -f go_home go_back check_appstate get_snapshot 2>/dev/null || true
|
||||
export -f assert_app_foreground assert_screenshot assert_text_visible assert_element_exists 2>/dev/null || true
|
||||
export -f tap_tab tap_tab_by_index launch_app close_app wait_settle 2>/dev/null || true
|
||||
export -f print_test_summary log_debug save_snapshot 2>/dev/null || true
|
||||
|
||||
echo "Test helpers loaded. Session: $ACTIVE_SESSION | App: $APP_BUNDLE_ID"
|
||||
100
skills/qa-automation/qa-test-flows/run-all.sh
Executable file
100
skills/qa-automation/qa-test-flows/run-all.sh
Executable file
@@ -0,0 +1,100 @@
|
||||
#!/bin/bash
|
||||
# ╔═══════════════════════════════════════════════════════════════════╗
|
||||
# ║ run-all.sh — Master Test Runner ║
|
||||
# ╠═══════════════════════════════════════════════════════════════════╣
|
||||
# ║ Runs all test flows in the flows/ directory tree. ║
|
||||
# ║ ║
|
||||
# ║ Usage: ║
|
||||
# ║ bash run-all.sh # Run all tests ║
|
||||
# ║ bash run-all.sh smoke # Run only smoke suite ║
|
||||
# ║ bash run-all.sh --list # List available tests ║
|
||||
# ╚═══════════════════════════════════════════════════════════════════╝
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
RUNNER_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
FLOWS_DIR="$RUNNER_DIR/flows"
|
||||
|
||||
# ── Parse args ───────────────────────────────────────────────────────
|
||||
SUITE_FILTER="${1:-}"
|
||||
LIST_ONLY=false
|
||||
|
||||
if [ "$SUITE_FILTER" = "--list" ]; then
|
||||
LIST_ONLY=true
|
||||
fi
|
||||
|
||||
# ── Find all test scripts ───────────────────────────────────────────
|
||||
find_tests() {
|
||||
if [ -n "$SUITE_FILTER" ] && [ "$SUITE_FILTER" != "--list" ]; then
|
||||
find "$FLOWS_DIR/$SUITE_FILTER" -name "*.sh" -type f 2>/dev/null | sort
|
||||
else
|
||||
find "$FLOWS_DIR" -name "*.sh" -type f 2>/dev/null | sort
|
||||
fi
|
||||
}
|
||||
|
||||
TESTS=$(find_tests)
|
||||
TEST_COUNT=$(echo "$TESTS" | grep -c "." || echo "0")
|
||||
|
||||
if [ "$TEST_COUNT" -eq 0 ]; then
|
||||
echo "No test scripts found in $FLOWS_DIR/"
|
||||
[ -n "$SUITE_FILTER" ] && echo "Suite filter: $SUITE_FILTER"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ── List mode ────────────────────────────────────────────────────────
|
||||
if [ "$LIST_ONLY" = true ]; then
|
||||
echo ""
|
||||
echo "Available test flows ($TEST_COUNT):"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
echo "$TESTS" | while read -r test; do
|
||||
suite=$(basename "$(dirname "$test")")
|
||||
name=$(basename "$test" .sh)
|
||||
printf " %-15s %s\n" "[$suite]" "$name"
|
||||
done
|
||||
echo ""
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# ── Run mode ─────────────────────────────────────────────────────────
|
||||
TOTAL_PASS=0
|
||||
TOTAL_FAIL=0
|
||||
TOTAL_SKIP=0
|
||||
START_TIME=$(date +%s)
|
||||
|
||||
echo ""
|
||||
echo "╔═══════════════════════════════════════════════════════════════╗"
|
||||
echo "║ QA Test Runner — Running $TEST_COUNT test(s) "
|
||||
echo "║ Started: $(date '+%Y-%m-%d %H:%M:%S') "
|
||||
echo "╚═══════════════════════════════════════════════════════════════╝"
|
||||
echo ""
|
||||
|
||||
declare -a RESULTS=()
|
||||
|
||||
echo "$TESTS" | while read -r test; do
|
||||
suite=$(basename "$(dirname "$test")")
|
||||
name=$(basename "$test" .sh)
|
||||
|
||||
echo "──────────────────────────────────────────────────────────────"
|
||||
echo " Running: [$suite] $name"
|
||||
echo "──────────────────────────────────────────────────────────────"
|
||||
|
||||
if bash "$test" 2>&1; then
|
||||
echo " Result: ✅ PASSED"
|
||||
else
|
||||
echo " Result: ❌ FAILED (exit code: $?)"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
done
|
||||
|
||||
END_TIME=$(date +%s)
|
||||
DURATION=$((END_TIME - START_TIME))
|
||||
|
||||
echo ""
|
||||
echo "══════════════════════════════════════════════════════════════════"
|
||||
echo " QA Test Run Complete"
|
||||
echo " Total tests: $TEST_COUNT"
|
||||
echo " Duration: ${DURATION}s"
|
||||
echo " Screenshots: /tmp/qa-tests/screenshots/"
|
||||
echo "══════════════════════════════════════════════════════════════════"
|
||||
echo ""
|
||||
@@ -0,0 +1,64 @@
|
||||
#!/bin/bash
|
||||
# ╔═══════════════════════════════════════════════════════════════════╗
|
||||
# ║ [TEST NAME] — [DESCRIPTION] ║
|
||||
# ╠═══════════════════════════════════════════════════════════════════╣
|
||||
# ║ CUSTOMIZE: Fill in the steps below for your test flow. ║
|
||||
# ║ ║
|
||||
# ║ Usage: bash .pi/skills/qa-automation/qa-test-flows/flows/[suite]/[name].sh
|
||||
# ╚═══════════════════════════════════════════════════════════════════╝
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
# ── Source Libraries ─────────────────────────────────────────────────
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
source "$SCRIPT_DIR/../../lib/test-helpers.sh"
|
||||
source "$SCRIPT_DIR/../../lib/cdp-helpers.sh"
|
||||
|
||||
# ── Test Setup ───────────────────────────────────────────────────────
|
||||
# CUSTOMIZE: Change this test name
|
||||
TEST_NAME="my-test-flow"
|
||||
setup_test "$TEST_NAME"
|
||||
|
||||
# ── Step 1: [Description] ───────────────────────────────────────────
|
||||
step "Verify app is running"
|
||||
assert_app_foreground || launch_app
|
||||
take_screenshot "01-initial"
|
||||
|
||||
# ── Step 2: [Navigation] ────────────────────────────────────────────
|
||||
# CUSTOMIZE: Navigate to your target screen
|
||||
step "Navigate to target screen"
|
||||
cdp_navigate "YourScreenName" # or: tap_tab 1, nav_explore, etc.
|
||||
sleep 2
|
||||
take_screenshot "02-target-screen"
|
||||
assert_screenshot "02-target-screen"
|
||||
|
||||
# ── Step 3: [Interaction] ───────────────────────────────────────────
|
||||
# CUSTOMIZE: Perform your test actions
|
||||
step "Perform test action"
|
||||
# tap 200 400 # Tap a button
|
||||
# swipe 200 600 200 200 # Swipe up
|
||||
# type_text @e3 "hello" # Type into a field
|
||||
sleep 1
|
||||
take_screenshot "03-after-action"
|
||||
|
||||
# ── Step 4: [Verification] ──────────────────────────────────────────
|
||||
# CUSTOMIZE: Verify the expected outcome
|
||||
step "Verify result"
|
||||
route=$(cdp_get_route 2>/dev/null || echo "unknown")
|
||||
log_info "Current route: $route"
|
||||
|
||||
# assert_text_visible "Expected Text"
|
||||
# assert_screenshot "04-verified"
|
||||
|
||||
# ── Step 5: [Cleanup] ───────────────────────────────────────────────
|
||||
step "Cleanup and final state"
|
||||
take_screenshot "05-final"
|
||||
assert_app_foreground
|
||||
|
||||
# ── Teardown ─────────────────────────────────────────────────────────
|
||||
teardown_test
|
||||
|
||||
echo ""
|
||||
echo "Test completed!"
|
||||
echo " Screenshots: $SCREENSHOT_DIR/$TEST_NAME/"
|
||||
echo ""
|
||||
Reference in New Issue
Block a user