Initial: pi-skill — 68 skills, 43 extensions, 11 themes for Pi

This commit is contained in:
Kunthawat Greethong
2026-05-25 16:38:02 +07:00
commit 69f7d8bdda
1689 changed files with 342427 additions and 0 deletions

View File

@@ -0,0 +1,177 @@
---
name: qa-test-flows
description: >
Automated UI test flow framework using CDP + agent-device for native apps and
agent-browser for web apps. Zero-framework approach — bash scripts orchestrate
simulators with no Detox, Maestro, or Appium dependencies. Provides test lifecycle,
screenshot capture, assertion helpers, and JSON report generation.
Invoke when user says "run tests", "test the app", "smoke test", "regression test",
"verify flows", "run e2e tests", or any task requiring automated UI testing.
allowed-tools: Bash(agent-device:*) Bash(agent-browser:*) Bash(xcrun:*) Bash(adb:*) Bash(open:*) Bash(find:*) Bash(npx:*) Bash(node:*) Read
---
# qa-test-flows
Automated UI test flow framework using a **dual-driver architecture**: **CDP** for navigation/state control and **agent-device** for screenshots/visual assertions in native apps, plus **agent-browser** for web app testing. This is a zero-framework approach — bash scripts orchestrate the entire test lifecycle with no Detox, Maestro, or Appium dependencies.
## Key Innovation: CDP + agent-device + agent-browser
Native mobile apps with complex gesture handlers (full-screen video players, swipe-based feeds) often make coordinate-based tapping unreliable. We solve this by connecting directly to the React Native Hermes runtime via Metro's CDP WebSocket, giving us:
- **Direct navigation control** via `navigationRef.current.navigate()` — no touch coordinates needed
- **Runtime state inspection** — check current route, user state, storage
- **Module access** via Metro's `__r()` require — load any app module at runtime
- **Screenshots & visual verification** via agent-device (best-in-class for simulators)
- **Web app testing** via agent-browser when the same app has a web version
## When to Use
- Smoke testing before submission or release builds
- Regression testing after navigation or feature changes
- Flow verification (auth, navigation, data entry, checkout)
- Platform parity checks (iOS vs Android)
- Pre-PR validation of multi-screen flows
- State persistence verification
## Test Architecture
### Dual Driver System
| Driver | Purpose | Use For |
|--------|---------|---------|
| **CDP (WebSocket)** | Navigation, state queries, JS execution | React Native apps (Hermes runtime) |
| **agent-device** | Screenshots, coordinate taps, swipes, accessibility | Native app simulators/emulators |
| **agent-browser** | Full browser automation, DOM interaction | Web apps, PWAs, browser testing |
### Test Flow Format
All test flows follow a consistent bash template:
```bash
#!/bin/bash
source "$(dirname "$0")/../../lib/test-helpers.sh"
source "$(dirname "$0")/../../lib/cdp-helpers.sh"
TEST_NAME="my-flow"
setup_test "$TEST_NAME"
# Step 1: Navigate
step "Navigate to target screen"
cdp_navigate "TargetScreen"
sleep 2
take_screenshot "01-target-screen"
assert_screenshot "01-target-screen"
# Step 2: Interact
step "Perform action"
tap 200 400
sleep 1
take_screenshot "02-after-action"
# Step 3: Verify
step "Verify result"
route=$(cdp_get_route)
log_info "Route: $route"
teardown_test
```
## Running Tests
### Single Test
```bash
bash .pi/skills/qa-automation/qa-test-flows/flows/smoke/example-smoke.sh
```
### All Tests
```bash
bash .pi/skills/qa-automation/qa-test-flows/run-all.sh
```
### With Logging
```bash
bash .pi/skills/qa-automation/qa-test-flows/flows/smoke/example-smoke.sh 2>&1 | tee /tmp/smoke.log
```
## Writing New Tests
### 1. Copy the template
```bash
cp .pi/skills/qa-automation/qa-test-flows/templates/new-flow.sh.template \
.pi/skills/qa-automation/qa-test-flows/flows/my-suite/my-test.sh
chmod +x .pi/skills/qa-automation/qa-test-flows/flows/my-suite/my-test.sh
```
### 2. Edit the template
Replace `CUSTOMIZE` markers with your app-specific details.
### 3. Find coordinates
```bash
agent-device screenshot /tmp/debug.png
open /tmp/debug.png # Measure tap targets
```
### 4. Run and verify
```bash
bash .pi/skills/qa-automation/qa-test-flows/flows/my-suite/my-test.sh
ls /tmp/qa-tests/screenshots/my-test/
```
## Helper Libraries
### test-helpers.sh
| Function | Purpose |
|----------|---------|
| `setup_test "name"` | Initialize test, create directories |
| `teardown_test` | Report results, cleanup |
| `step "description"` | Log a numbered test step |
| `log_pass/fail/info/warn` | Status logging |
| `take_screenshot "name"` | Capture screenshot |
| `tap x y` | Tap at coordinates |
| `swipe x1 y1 x2 y2` | Swipe gesture |
| `scroll_dir direction` | Scroll up/down/left/right |
| `assert_app_foreground` | Verify app is running |
| `assert_screenshot "name"` | Verify screenshot exists |
| `assert_text_visible "text"` | Check accessibility tree |
| `launch_app` | Launch/relaunch the app |
| `close_app` | Terminate the app |
### cdp-helpers.sh
| Function | Purpose |
|----------|---------|
| `cdp_eval "expression"` | Execute JS in Hermes |
| `cdp_eval_safe "expression"` | Eval with ErrorUtils suppression |
| `cdp_navigate "Screen"` | Navigate to screen |
| `cdp_navigate_tab "TabScreen"` | Navigate to tab |
| `cdp_get_route` | Get current route name |
| `cdp_get_state` | Get full nav state |
| `cdp_go_back` | Go back |
| `nav_explore/search/home/profile` | Quick tab navigation |
## File Structure
```
qa-test-flows/
├── SKILL.md # This file
├── lib/
│ ├── test-helpers.sh # Core test framework
│ └── cdp-helpers.sh # CDP interaction helpers
├── flows/
│ └── smoke/
│ └── example-smoke.sh # Example smoke test
├── templates/
│ └── new-flow.sh.template # Template for new tests
└── run-all.sh # Master test runner
```
## Troubleshooting
| Problem | Solution |
|---------|----------|
| agent-device click hangs | Commands run in background with timeout — if still hanging, increase `DEVICE_CMD_TIMEOUT` |
| Dev console overlay covers screen | Call `dismiss_error_overlay` from scroll-helpers, or use CDP to suppress LogBox |
| Accessibility tree is sparse | React Native trees are thinner than web — prefer coordinates + screenshots |
| CDP timeout | Check dev server is running: `curl $DEV_SERVER_HEALTH` |
| Module ID changed after code update | Delete cache: `rm $NAV_MODULE_CACHE` — setup guard will re-scan |

View File

@@ -0,0 +1,108 @@
#!/bin/bash
# ╔═══════════════════════════════════════════════════════════════════╗
# ║ Example Smoke Test — Quick Verification of Core Flows ║
# ╠═══════════════════════════════════════════════════════════════════╣
# ║ CUSTOMIZE: Replace screen names and navigation with your app's. ║
# ║ ║
# ║ Usage: bash .pi/skills/qa-automation/qa-test-flows/flows/smoke/example-smoke.sh
# ╚═══════════════════════════════════════════════════════════════════╝
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
source "$SCRIPT_DIR/../../lib/test-helpers.sh"
source "$SCRIPT_DIR/../../lib/cdp-helpers.sh"
TEST_NAME="smoke-test"
setup_test "$TEST_NAME"
# ── Step 1: App Launch ───────────────────────────────────────────────
step "Verify app is running"
assert_app_foreground || {
launch_app
sleep 3
}
take_screenshot "01-app-launched"
assert_screenshot "01-app-launched"
log_pass "App is running"
# ── Step 2: Check Login State ────────────────────────────────────────
step "Check authentication state"
logged_in=$(cdp_is_logged_in 2>/dev/null || echo "unknown")
log_info "Logged in: $logged_in"
if [ "$logged_in" = "false" ]; then
log_warn "App is on login screen — some tests may be skipped"
fi
take_screenshot "02-auth-state"
# ── Step 3: Navigate Through Main Tabs ───────────────────────────────
# CUSTOMIZE: Replace with your app's tab screens
step "Navigate to Tab 1 (Explore/Home)"
nav_explore 2>/dev/null || tap_tab 1
sleep 2
take_screenshot "03-tab-1"
route=$(cdp_get_route 2>/dev/null || echo "unknown")
log_info "Route: $route"
log_pass "Tab 1 loaded"
step "Navigate to Tab 2 (Search)"
nav_search 2>/dev/null || tap_tab 2
sleep 2
take_screenshot "04-tab-2"
route=$(cdp_get_route 2>/dev/null || echo "unknown")
log_info "Route: $route"
log_pass "Tab 2 loaded"
step "Navigate to Tab 3 (Home/Create)"
nav_home 2>/dev/null || tap_tab 3
sleep 2
take_screenshot "05-tab-3"
route=$(cdp_get_route 2>/dev/null || echo "unknown")
log_info "Route: $route"
log_pass "Tab 3 loaded"
step "Navigate to Tab 5 (Profile)"
nav_profile 2>/dev/null || tap_tab 5
sleep 2
take_screenshot "06-tab-5"
route=$(cdp_get_route 2>/dev/null || echo "unknown")
log_info "Route: $route"
log_pass "Tab 5 loaded"
# ── Step 4: Navigate to Detail Screen ────────────────────────────────
# CUSTOMIZE: Replace with a screen in your app
step "Navigate to Settings (detail screen)"
cdp_navigate "$SCREEN_SETTINGS" 2>/dev/null || {
log_warn "CDP navigation failed — trying tap"
tap $SETTINGS_BUTTON_X $SETTINGS_BUTTON_Y
}
sleep 2
take_screenshot "07-settings"
route=$(cdp_get_route 2>/dev/null || echo "unknown")
log_info "Route: $route"
# ── Step 5: Go Back ──────────────────────────────────────────────────
step "Navigate back"
cdp_go_back 2>/dev/null || go_back
sleep 2
take_screenshot "08-back"
route=$(cdp_get_route 2>/dev/null || echo "unknown")
log_info "Route after back: $route"
log_pass "Back navigation works"
# ── Step 6: Final State ─────────────────────────────────────────────
step "Final state check"
assert_app_foreground
take_screenshot "09-final"
log_pass "Smoke test complete — app is stable"
# ── Report ───────────────────────────────────────────────────────────
teardown_test
echo ""
echo "Smoke test completed!"
echo " Screenshots: $SCREENSHOT_DIR/$TEST_NAME/"
echo ""

View File

@@ -0,0 +1,340 @@
#!/bin/bash
# ╔═══════════════════════════════════════════════════════════════════╗
# ║ cdp-helpers.sh — Chrome DevTools Protocol Helpers ║
# ╠═══════════════════════════════════════════════════════════════════╣
# ║ Source this file for CDP interaction with React Native apps: ║
# ║ source "$(dirname "$0")/../../lib/cdp-helpers.sh" ║
# ║ ║
# ║ Provides: ║
# ║ • CDP evaluation (cdp_eval, cdp_eval_safe) ║
# ║ • Navigation (cdp_navigate, cdp_navigate_tab) ║
# ║ • State queries (cdp_get_route, cdp_get_state) ║
# ║ • Auto-discovery (CDP target, navigation module) ║
# ║ ║
# ║ Requires: Node.js, 'ws' npm package, Metro dev server running ║
# ╚═══════════════════════════════════════════════════════════════════╝
set -euo pipefail
# ── Source configuration ─────────────────────────────────────────────
CDP_HELPERS_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
CDP_QA_ROOT="$(cd "$CDP_HELPERS_DIR/../.." && pwd)"
# Only source config if not already loaded
if [ -z "${QA_AUTOMATION_DIR:-}" ]; then
source "$CDP_QA_ROOT/qa.config.sh"
fi
# ── CDP State ────────────────────────────────────────────────────────
_CDP_WS_URL_RESOLVED=""
_CDP_NAV_MODULE_ID=""
# ── CDP Auto-Discovery ──────────────────────────────────────────────
# Ensure we have a CDP WebSocket URL
_ensure_cdp_ws_url() {
if [ -n "$_CDP_WS_URL_RESOLVED" ]; then
echo "$_CDP_WS_URL_RESOLVED"
return 0
fi
local ws_url
ws_url=$(qa_detect_cdp_ws_url 2>/dev/null || echo "")
if [ -n "$ws_url" ]; then
_CDP_WS_URL_RESOLVED="$ws_url"
echo "$ws_url"
return 0
fi
# Last resort: construct from config
if [ "$CDP_WS_URL" != "auto" ] && [ -n "$CDP_WS_URL" ]; then
_CDP_WS_URL_RESOLVED="$CDP_WS_URL"
echo "$CDP_WS_URL"
return 0
fi
echo ""
return 1
}
# Ensure we have a navigation module ID
_ensure_nav_module_id() {
if [ -n "$_CDP_NAV_MODULE_ID" ]; then
echo "$_CDP_NAV_MODULE_ID"
return 0
fi
# Check cache
if [ -f "$NAV_MODULE_CACHE" ]; then
local cached
cached=$(cat "$NAV_MODULE_CACHE" 2>/dev/null || echo "")
if [ -n "$cached" ]; then
_CDP_NAV_MODULE_ID="$cached"
echo "$cached"
return 0
fi
fi
# Auto-detect
local module_id
module_id=$(qa_detect_nav_module 2>/dev/null || echo "")
if [ -n "$module_id" ]; then
_CDP_NAV_MODULE_ID="$module_id"
echo "$module_id"
return 0
fi
echo ""
return 1
}
# ── Core CDP Functions ───────────────────────────────────────────────
# Execute JavaScript in the React Native Hermes runtime via CDP.
# Usage: cdp_eval "javascript expression"
# Returns: the evaluated result
cdp_eval() {
local expression="$1"
local ws_url
ws_url=$(_ensure_cdp_ws_url)
if [ -z "$ws_url" ]; then
echo '{"error":"no CDP WebSocket URL"}'
return 1
fi
cd "$PROJECT_DIR"
node -e "
const WebSocket=require('ws');
const ws=new WebSocket('$ws_url');
ws.on('open',()=>{
ws.send(JSON.stringify({
id:1,
method:'Runtime.evaluate',
params:{
expression: \`$expression\`,
returnByValue:true
}
}));
});
ws.on('message',d=>{
const m=JSON.parse(d);
if(m.id===1){
if(m.result?.result?.value) console.log(m.result.result.value);
else if(m.result?.exceptionDetails) console.error('CDP Error:', m.result.exceptionDetails.text);
else console.log(JSON.stringify(m.result));
ws.close();
process.exit(0);
}
});
ws.on('error',e=>{console.error('WS Error:',e.message);process.exit(1)});
setTimeout(()=>{console.error('CDP Timeout');process.exit(1)},8000);
" 2>&1
}
# Safe CDP eval — suppresses ErrorUtils/LogBox during execution.
# Use for any eval that touches Metro modules or could trigger side effects.
# Usage: cdp_eval_safe "javascript expression"
cdp_eval_safe() {
local expression="$1"
cdp_eval "
(function() {
var _origHandler = globalThis.ErrorUtils ? ErrorUtils.getGlobalHandler() : null;
var _origCE = console.error;
if (globalThis.ErrorUtils) ErrorUtils.setGlobalHandler(function() {});
console.error = function() {};
try {
return (function() { $expression })();
} finally {
if (globalThis.ErrorUtils && _origHandler) ErrorUtils.setGlobalHandler(_origHandler);
console.error = _origCE;
}
})();
"
}
# ── Navigation Functions ─────────────────────────────────────────────
# Navigate to a screen by name.
# Usage: cdp_navigate "ScreenName" [params_json]
cdp_navigate() {
local screen="$1"
local params="${2:-}"
local nav_id
nav_id=$(_ensure_nav_module_id)
if [ -z "$nav_id" ]; then
echo '{"error":"navigation module not found"}'
return 1
fi
if [ -n "$params" ]; then
cdp_eval "
const ref = __r(${nav_id}).navigationRef;
ref.current.navigate('${screen}', ${params});
JSON.stringify({ ok: true, route: ref.current.getCurrentRoute()?.name });
"
else
cdp_eval "
const ref = __r(${nav_id}).navigationRef;
ref.current.navigate('${screen}');
JSON.stringify({ ok: true, route: ref.current.getCurrentRoute()?.name });
"
fi
}
# Navigate to a bottom tab screen.
# Usage: cdp_navigate_tab "ScreenName"
cdp_navigate_tab() {
local tab="$1"
local nav_id
nav_id=$(_ensure_nav_module_id)
if [ -z "$nav_id" ]; then
echo '{"error":"navigation module not found"}'
return 1
fi
cdp_eval "
const ref = __r(${nav_id}).navigationRef;
ref.current.navigate('${TAB_NAVIGATOR_NAME}', { screen: '${tab}' });
JSON.stringify({ ok: true, route: ref.current.getCurrentRoute()?.name });
"
}
# Get current route name.
# Usage: route=$(cdp_get_route)
cdp_get_route() {
local nav_id
nav_id=$(_ensure_nav_module_id)
if [ -z "$nav_id" ]; then
echo "unknown"
return 1
fi
cdp_eval "
const ref = __r(${nav_id}).navigationRef;
ref.current.getCurrentRoute()?.name || 'unknown';
"
}
# Get full navigation state as JSON.
cdp_get_state() {
local nav_id
nav_id=$(_ensure_nav_module_id)
if [ -z "$nav_id" ]; then
echo '{"error":"navigation module not found"}'
return 1
fi
cdp_eval "
const ref = __r(${nav_id}).navigationRef;
const state = ref.current.getRootState();
function getRoutes(s, depth) {
let routes = [];
if (s.routes) {
for (const r of s.routes) {
routes.push({name: r.name, depth});
if (r.state) routes = routes.concat(getRoutes(r.state, depth+1));
}
}
return routes;
}
JSON.stringify({
current: ref.current.getCurrentRoute()?.name,
routes: getRoutes(state, 0)
});
"
}
# Go back in navigation.
cdp_go_back() {
local nav_id
nav_id=$(_ensure_nav_module_id)
if [ -z "$nav_id" ]; then
echo "error: no nav module"
return 1
fi
cdp_eval "
const ref = __r(${nav_id}).navigationRef;
if (ref.current.canGoBack()) {
ref.current.goBack();
'went back to: ' + ref.current.getCurrentRoute()?.name;
} else {
'cannot go back';
}
"
}
# Reset navigation to initial state.
cdp_reset_navigation() {
local nav_id
nav_id=$(_ensure_nav_module_id)
if [ -z "$nav_id" ]; then
echo "error: no nav module"
return 1
fi
cdp_eval "
const ref = __r(${nav_id}).navigationRef;
ref.current.reset({
index: 0,
routes: [{ name: 'Main', state: { routes: [{ name: '${TAB_NAVIGATOR_NAME}' }] } }]
});
'reset to ${TAB_NAVIGATOR_NAME}';
"
}
# ── Convenience Tab Navigation ───────────────────────────────────────
# These use the screen names from qa.config.sh
nav_tab_1() { [ -n "$SCREEN_EXPLORE" ] && cdp_navigate_tab "$SCREEN_EXPLORE" || echo "SCREEN_EXPLORE not configured"; }
nav_tab_2() { [ -n "$SCREEN_SEARCH" ] && cdp_navigate_tab "$SCREEN_SEARCH" || echo "SCREEN_SEARCH not configured"; }
nav_tab_3() { [ -n "$SCREEN_HOME" ] && cdp_navigate_tab "$SCREEN_HOME" || echo "SCREEN_HOME not configured"; }
nav_tab_4() { echo "Tab 4 not configured — set SCREEN name and add to cdp-helpers.sh"; }
nav_tab_5() { [ -n "$SCREEN_PROFILE" ] && cdp_navigate_tab "$SCREEN_PROFILE" || echo "SCREEN_PROFILE not configured"; }
# Generic by name
nav_explore() { nav_tab_1; }
nav_search() { nav_tab_2; }
nav_home() { nav_tab_3; }
nav_profile() { nav_tab_5; }
# ── Utility Functions ────────────────────────────────────────────────
# Check if user is logged in (heuristic: check current route)
cdp_is_logged_in() {
local route
route=$(cdp_get_route 2>/dev/null || echo "unknown")
if echo "$route" | grep -qiE "login|signup|auth|welcome|launch"; then
echo "false"
else
echo "true"
fi
}
# Clear cached module IDs (force re-discovery on next call)
cdp_clear_cache() {
_CDP_WS_URL_RESOLVED=""
_CDP_NAV_MODULE_ID=""
[ -f "$NAV_MODULE_CACHE" ] && rm -f "$NAV_MODULE_CACHE" 2>/dev/null || true
[ -f "$VIDEO_MODULE_CACHE" ] && rm -f "$VIDEO_MODULE_CACHE" 2>/dev/null || true
echo "CDP cache cleared"
}
# ── Export Functions ─────────────────────────────────────────────────
export -f cdp_eval cdp_eval_safe 2>/dev/null || true
export -f cdp_navigate cdp_navigate_tab cdp_get_route cdp_get_state 2>/dev/null || true
export -f cdp_go_back cdp_reset_navigation cdp_is_logged_in 2>/dev/null || true
export -f nav_explore nav_search nav_home nav_profile 2>/dev/null || true
export -f cdp_clear_cache 2>/dev/null || true
echo "CDP helpers loaded. WebSocket: ${_CDP_WS_URL_RESOLVED:-auto-detect}"

View File

@@ -0,0 +1,453 @@
#!/bin/bash
# ╔═══════════════════════════════════════════════════════════════════╗
# ║ test-helpers.sh — Core Test Framework for QA Automation ║
# ╠═══════════════════════════════════════════════════════════════════╣
# ║ Source this file at the top of every test flow script: ║
# ║ source "$(dirname "$0")/../../lib/test-helpers.sh" ║
# ║ ║
# ║ Provides: ║
# ║ • Test lifecycle (setup_test, teardown_test, step) ║
# ║ • Logging (log_pass, log_fail, log_info, log_warn) ║
# ║ • agent-device wrappers (tap, swipe, scroll, screenshot) ║
# ║ • Assertion functions (assert_app_foreground, etc.) ║
# ║ • Navigation helpers (tap_tab, launch_app, close_app) ║
# ╚═══════════════════════════════════════════════════════════════════╝
set -euo pipefail
# ── Source configuration ─────────────────────────────────────────────
HELPERS_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
QA_ROOT="$(cd "$HELPERS_DIR/../.." && pwd)"
source "$QA_ROOT/qa.config.sh"
# ── Auto-detect simulator UDID if set to "auto" ─────────────────────
if [ "$SIMULATOR_UDID" = "auto" ]; then
_detected_udid=$(qa_detect_simulator_udid 2>/dev/null || echo "")
if [ -n "$_detected_udid" ]; then
SIMULATOR_UDID="$_detected_udid"
fi
fi
# ── Test State ───────────────────────────────────────────────────────
TEST_NAME=""
STEP_COUNT=0
PASS_COUNT=0
FAIL_COUNT=0
TEST_START_TIME=""
# ── Initialization ───────────────────────────────────────────────────
init_test_env() {
mkdir -p "$TEST_OUTPUT_DIR"
mkdir -p "$SCREENSHOT_DIR"
if [ ! -f "$RESULTS_FILE" ]; then
echo "Test Run: $(date '+%Y-%m-%d %H:%M:%S')" > "$RESULTS_FILE"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" >> "$RESULTS_FILE"
fi
}
init_test_env
# ── Test Lifecycle ───────────────────────────────────────────────────
setup_test() {
local name="$1"
TEST_NAME="$name"
TEST_START_TIME=$(date +%s)
STEP_COUNT=0
PASS_COUNT=0
FAIL_COUNT=0
mkdir -p "$SCREENSHOT_DIR/$TEST_NAME"
echo ""
echo "═══════════════════════════════════════════"
echo "🧪 TEST: $TEST_NAME"
echo "═══════════════════════════════════════════"
echo "Started: $(date '+%Y-%m-%d %H:%M:%S')"
echo ""
}
teardown_test() {
local name="${1:-$TEST_NAME}"
local end_time=$(date +%s)
local duration=$((end_time - TEST_START_TIME))
echo ""
echo "───────────────────────────────────────────"
echo "📊 RESULTS: $name"
echo " Steps: $STEP_COUNT | Passed: $PASS_COUNT | Failed: $FAIL_COUNT"
echo " Duration: ${duration}s"
if [ $FAIL_COUNT -eq 0 ]; then
echo " Status: ✅ ALL PASSED"
else
echo " Status: ❌ $FAIL_COUNT FAILURES"
fi
echo "───────────────────────────────────────────"
echo ""
echo "$(date '+%Y-%m-%d %H:%M:%S') | $name | Steps:$STEP_COUNT Pass:$PASS_COUNT Fail:$FAIL_COUNT | ${duration}s" >> "$RESULTS_FILE"
}
# ── Logging ──────────────────────────────────────────────────────────
step() {
STEP_COUNT=$((STEP_COUNT + 1))
echo " [$STEP_COUNT] $1"
}
log_pass() {
PASS_COUNT=$((PASS_COUNT + 1))
echo "$1"
}
log_fail() {
FAIL_COUNT=$((FAIL_COUNT + 1))
echo "$1"
}
log_info() {
echo " $1"
}
log_warn() {
echo " ⚠️ $1"
}
# ── agent-device Wrappers ────────────────────────────────────────────
# Take a screenshot using xcrun simctl (avoids agent-device focus issues)
# Falls back to agent-device if xcrun isn't available.
# Usage: take_screenshot "step_name" [session]
take_screenshot() {
local name="$1"
local session="${2:-$ACTIVE_SESSION}"
local path="$SCREENSHOT_DIR/$TEST_NAME/${name}.png"
mkdir -p "$(dirname "$path")"
# Prefer xcrun simctl for iOS (doesn't steal focus)
if [ "$SIMULATOR_UDID" != "auto" ] && command -v xcrun >/dev/null 2>&1; then
xcrun simctl io "$SIMULATOR_UDID" screenshot "$path" 2>/dev/null || {
# Fallback to agent-device
agent-device screenshot "$path" --session "$session" 2>/dev/null || {
log_warn "Screenshot failed for: $name"
return 1
}
}
else
agent-device screenshot "$path" --session "$session" 2>/dev/null || {
log_warn "Screenshot failed for: $name"
return 1
}
fi
echo "$path"
}
# Tap at coordinates with timeout handling
# Usage: tap x y [session]
tap() {
local x="$1"
local y="$2"
local session="${3:-$ACTIVE_SESSION}"
agent-device click "$x" "$y" --session "$session" 2>/dev/null &
local pid=$!
sleep 0.5
if ps -p $pid > /dev/null 2>&1; then
kill $pid 2>/dev/null || true
wait $pid 2>/dev/null || true
fi
sleep 1
}
# Swipe gesture
# Usage: swipe x1 y1 x2 y2 [session]
swipe() {
local x1="$1"
local y1="$2"
local x2="$3"
local y2="$4"
local session="${5:-$ACTIVE_SESSION}"
agent-device swipe "$x1" "$y1" "$x2" "$y2" --session "$session" 2>/dev/null &
local pid=$!
sleep 0.5
if ps -p $pid > /dev/null 2>&1; then
kill $pid 2>/dev/null || true
wait $pid 2>/dev/null || true
fi
sleep 1
}
# Scroll in a direction
# Usage: scroll_dir direction [session]
scroll_dir() {
local direction="$1"
local session="${2:-$ACTIVE_SESSION}"
agent-device scroll "$direction" --session "$session" 2>/dev/null &
local pid=$!
sleep 0.5
if ps -p $pid > /dev/null 2>&1; then
kill $pid 2>/dev/null || true
wait $pid 2>/dev/null || true
fi
sleep 1
}
# Type text into focused field
# Usage: type_text ref_or_coords text [session]
type_text() {
local ref="$1"
local text="$2"
local session="${3:-$ACTIVE_SESSION}"
agent-device fill "$ref" "$text" --session "$session" 2>/dev/null &
local pid=$!
sleep 0.5
if ps -p $pid > /dev/null 2>&1; then
kill $pid 2>/dev/null || true
wait $pid 2>/dev/null || true
fi
sleep 1
}
# Press home button
go_home() {
local session="${1:-$ACTIVE_SESSION}"
agent-device home --session "$session" 2>/dev/null &
local pid=$!
sleep 1
if ps -p $pid > /dev/null 2>&1; then
kill $pid 2>/dev/null || true
wait $pid 2>/dev/null || true
fi
sleep 2
}
# Go back
go_back() {
local session="${1:-$ACTIVE_SESSION}"
agent-device back --session "$session" 2>/dev/null &
local pid=$!
sleep 1
if ps -p $pid > /dev/null 2>&1; then
kill $pid 2>/dev/null || true
wait $pid 2>/dev/null || true
fi
sleep 2
}
# Check if the app process is running on the simulator
check_appstate() {
local session="${1:-$ACTIVE_SESSION}"
if [ "$SIMULATOR_UDID" != "auto" ] && command -v xcrun >/dev/null 2>&1; then
xcrun simctl spawn "$SIMULATOR_UDID" launchctl list 2>/dev/null | grep "UIKitApplication:${APP_BUNDLE_ID}" || echo ""
else
agent-device appstate --session "$session" 2>/dev/null || echo ""
fi
}
# Get accessibility snapshot
get_snapshot() {
local session="${1:-$ACTIVE_SESSION}"
local depth="${2:-5}"
timeout 10 agent-device snapshot -i --depth "$depth" --session "$session" 2>/dev/null || echo "(snapshot timeout)"
}
# ── Assertion Functions ──────────────────────────────────────────────
# Assert app is in foreground
assert_app_foreground() {
local session="${1:-$ACTIVE_SESSION}"
local state=$(check_appstate "$session" 2>/dev/null | grep -o "$APP_BUNDLE_ID" || true)
if [ -n "$state" ]; then
log_pass "App is in foreground ($APP_BUNDLE_ID)"
return 0
else
log_fail "App is NOT in foreground"
return 1
fi
}
# Assert screenshot was captured
assert_screenshot() {
local name="$1"
local path="$SCREENSHOT_DIR/$TEST_NAME/${name}.png"
if [ -f "$path" ] && [ -s "$path" ]; then
log_pass "Screenshot captured: $name"
return 0
else
log_fail "Screenshot missing or empty: $name"
return 1
fi
}
# Assert text is visible in accessibility tree
assert_text_visible() {
local text="$1"
local session="${2:-$ACTIVE_SESSION}"
local snapshot=$(get_snapshot "$session")
if echo "$snapshot" | grep -qi "$text"; then
log_pass "Text visible: '$text'"
return 0
else
log_warn "Text not found in accessibility tree: '$text'"
return 1
fi
}
# Assert element exists in accessibility tree
assert_element_exists() {
local label="$1"
local session="${2:-$ACTIVE_SESSION}"
local snapshot=$(get_snapshot "$session")
if echo "$snapshot" | grep -qi "label.*$label\|name.*$label"; then
log_pass "Element found: '$label'"
return 0
else
log_fail "Element not found: '$label'"
return 1
fi
}
# ── Navigation Helpers ───────────────────────────────────────────────
# Tap a tab by position (1-5)
# Usage: tap_tab 1 (first tab)
tap_tab_by_index() {
local index="$1"
local x_var="TAB_${index}_X"
local x="${!x_var:-}"
if [ -z "$x" ]; then
log_fail "No coordinate defined for tab index $index (set TAB_${index}_X)"
return 1
fi
tap "$x" "$TAB_BAR_Y"
sleep 2
}
# Tap a tab by name (reads screen names from config and maps to tab index)
# Usage: tap_tab "explore"
tap_tab() {
local tab_name="$1"
local tab_name_upper=$(echo "$tab_name" | tr '[:lower:]' '[:upper:]')
# Map common names to tab indices — customize in qa.config.sh
case "$tab_name_upper" in
EXPLORE|HOME|TAB1|1) tap_tab_by_index 1 ;;
SEARCH|TAB2|2) tap_tab_by_index 2 ;;
CREATE|TAB3|3) tap_tab_by_index 3 ;;
WALLET|TAB4|4) tap_tab_by_index 4 ;;
PROFILE|TAB5|5) tap_tab_by_index 5 ;;
*)
log_fail "Unknown tab: $tab_name"
return 1
;;
esac
}
# Launch the app fresh
launch_app() {
local session="${1:-$ACTIVE_SESSION}"
step "Launching app ($APP_BUNDLE_ID)"
if [ "$SIMULATOR_UDID" != "auto" ] && command -v xcrun >/dev/null 2>&1; then
xcrun simctl terminate "$SIMULATOR_UDID" "$APP_BUNDLE_ID" 2>/dev/null || true
sleep 1
xcrun simctl launch "$SIMULATOR_UDID" "$APP_BUNDLE_ID" 2>/dev/null || {
log_fail "Failed to launch app via xcrun"
return 1
}
else
agent-device open "$APP_BUNDLE_ID" --session "$session" --relaunch 2>/dev/null || {
log_fail "Failed to launch app"
return 1
}
fi
sleep "$APP_SETTLE_TIME"
log_pass "App launched"
}
# Close the app
close_app() {
local session="${1:-$ACTIVE_SESSION}"
if [ "$SIMULATOR_UDID" != "auto" ] && command -v xcrun >/dev/null 2>&1; then
xcrun simctl terminate "$SIMULATOR_UDID" "$APP_BUNDLE_ID" 2>/dev/null || {
log_warn "Could not terminate app via xcrun"
return 1
}
else
agent-device close --session "$session" 2>/dev/null || {
log_warn "Could not close app"
return 1
}
fi
sleep 2
}
# Wait for app to settle
wait_settle() {
local seconds="${1:-3}"
sleep "$seconds"
}
# ── Utility Functions ────────────────────────────────────────────────
# Print test summary
print_test_summary() {
echo ""
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "📋 TEST EXECUTION SUMMARY"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
tail -20 "$RESULTS_FILE"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
}
log_debug() {
local msg="$1"
echo "[DEBUG] $(date '+%H:%M:%S') $msg" >> "$RESULTS_FILE"
}
save_snapshot() {
local desc="$1"
local session="${2:-$ACTIVE_SESSION}"
local filename="$(echo "$desc" | tr ' ' '_' | tr -cd '[:alnum:]._-')"
local snap=$(get_snapshot "$session")
echo "$snap" > "$SCREENSHOT_DIR/$TEST_NAME/${filename}_snapshot.txt"
log_info "Saved snapshot: $filename"
}
# ── Error Handling ───────────────────────────────────────────────────
trap 'teardown_test 2>/dev/null || true' EXIT
on_error() {
local line=$1
log_fail "Error on line $line"
log_debug "Exit code: $?"
}
trap 'on_error ${LINENO}' ERR
# ── Export Functions ─────────────────────────────────────────────────
export -f setup_test teardown_test step log_pass log_fail log_info log_warn 2>/dev/null || true
export -f take_screenshot tap swipe scroll_dir type_text 2>/dev/null || true
export -f go_home go_back check_appstate get_snapshot 2>/dev/null || true
export -f assert_app_foreground assert_screenshot assert_text_visible assert_element_exists 2>/dev/null || true
export -f tap_tab tap_tab_by_index launch_app close_app wait_settle 2>/dev/null || true
export -f print_test_summary log_debug save_snapshot 2>/dev/null || true
echo "Test helpers loaded. Session: $ACTIVE_SESSION | App: $APP_BUNDLE_ID"

View File

@@ -0,0 +1,100 @@
#!/bin/bash
# ╔═══════════════════════════════════════════════════════════════════╗
# ║ run-all.sh — Master Test Runner ║
# ╠═══════════════════════════════════════════════════════════════════╣
# ║ Runs all test flows in the flows/ directory tree. ║
# ║ ║
# ║ Usage: ║
# ║ bash run-all.sh # Run all tests ║
# ║ bash run-all.sh smoke # Run only smoke suite ║
# ║ bash run-all.sh --list # List available tests ║
# ╚═══════════════════════════════════════════════════════════════════╝
set -euo pipefail
RUNNER_DIR="$(cd "$(dirname "$0")" && pwd)"
FLOWS_DIR="$RUNNER_DIR/flows"
# ── Parse args ───────────────────────────────────────────────────────
SUITE_FILTER="${1:-}"
LIST_ONLY=false
if [ "$SUITE_FILTER" = "--list" ]; then
LIST_ONLY=true
fi
# ── Find all test scripts ───────────────────────────────────────────
find_tests() {
if [ -n "$SUITE_FILTER" ] && [ "$SUITE_FILTER" != "--list" ]; then
find "$FLOWS_DIR/$SUITE_FILTER" -name "*.sh" -type f 2>/dev/null | sort
else
find "$FLOWS_DIR" -name "*.sh" -type f 2>/dev/null | sort
fi
}
TESTS=$(find_tests)
TEST_COUNT=$(echo "$TESTS" | grep -c "." || echo "0")
if [ "$TEST_COUNT" -eq 0 ]; then
echo "No test scripts found in $FLOWS_DIR/"
[ -n "$SUITE_FILTER" ] && echo "Suite filter: $SUITE_FILTER"
exit 1
fi
# ── List mode ────────────────────────────────────────────────────────
if [ "$LIST_ONLY" = true ]; then
echo ""
echo "Available test flows ($TEST_COUNT):"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "$TESTS" | while read -r test; do
suite=$(basename "$(dirname "$test")")
name=$(basename "$test" .sh)
printf " %-15s %s\n" "[$suite]" "$name"
done
echo ""
exit 0
fi
# ── Run mode ─────────────────────────────────────────────────────────
TOTAL_PASS=0
TOTAL_FAIL=0
TOTAL_SKIP=0
START_TIME=$(date +%s)
echo ""
echo "╔═══════════════════════════════════════════════════════════════╗"
echo "║ QA Test Runner — Running $TEST_COUNT test(s) "
echo "║ Started: $(date '+%Y-%m-%d %H:%M:%S') "
echo "╚═══════════════════════════════════════════════════════════════╝"
echo ""
declare -a RESULTS=()
echo "$TESTS" | while read -r test; do
suite=$(basename "$(dirname "$test")")
name=$(basename "$test" .sh)
echo "──────────────────────────────────────────────────────────────"
echo " Running: [$suite] $name"
echo "──────────────────────────────────────────────────────────────"
if bash "$test" 2>&1; then
echo " Result: ✅ PASSED"
else
echo " Result: ❌ FAILED (exit code: $?)"
fi
echo ""
done
END_TIME=$(date +%s)
DURATION=$((END_TIME - START_TIME))
echo ""
echo "══════════════════════════════════════════════════════════════════"
echo " QA Test Run Complete"
echo " Total tests: $TEST_COUNT"
echo " Duration: ${DURATION}s"
echo " Screenshots: /tmp/qa-tests/screenshots/"
echo "══════════════════════════════════════════════════════════════════"
echo ""

View File

@@ -0,0 +1,64 @@
#!/bin/bash
# ╔═══════════════════════════════════════════════════════════════════╗
# ║ [TEST NAME] — [DESCRIPTION] ║
# ╠═══════════════════════════════════════════════════════════════════╣
# ║ CUSTOMIZE: Fill in the steps below for your test flow. ║
# ║ ║
# ║ Usage: bash .pi/skills/qa-automation/qa-test-flows/flows/[suite]/[name].sh
# ╚═══════════════════════════════════════════════════════════════════╝
set -euo pipefail
# ── Source Libraries ─────────────────────────────────────────────────
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
source "$SCRIPT_DIR/../../lib/test-helpers.sh"
source "$SCRIPT_DIR/../../lib/cdp-helpers.sh"
# ── Test Setup ───────────────────────────────────────────────────────
# CUSTOMIZE: Change this test name
TEST_NAME="my-test-flow"
setup_test "$TEST_NAME"
# ── Step 1: [Description] ───────────────────────────────────────────
step "Verify app is running"
assert_app_foreground || launch_app
take_screenshot "01-initial"
# ── Step 2: [Navigation] ────────────────────────────────────────────
# CUSTOMIZE: Navigate to your target screen
step "Navigate to target screen"
cdp_navigate "YourScreenName" # or: tap_tab 1, nav_explore, etc.
sleep 2
take_screenshot "02-target-screen"
assert_screenshot "02-target-screen"
# ── Step 3: [Interaction] ───────────────────────────────────────────
# CUSTOMIZE: Perform your test actions
step "Perform test action"
# tap 200 400 # Tap a button
# swipe 200 600 200 200 # Swipe up
# type_text @e3 "hello" # Type into a field
sleep 1
take_screenshot "03-after-action"
# ── Step 4: [Verification] ──────────────────────────────────────────
# CUSTOMIZE: Verify the expected outcome
step "Verify result"
route=$(cdp_get_route 2>/dev/null || echo "unknown")
log_info "Current route: $route"
# assert_text_visible "Expected Text"
# assert_screenshot "04-verified"
# ── Step 5: [Cleanup] ───────────────────────────────────────────────
step "Cleanup and final state"
take_screenshot "05-final"
assert_app_foreground
# ── Teardown ─────────────────────────────────────────────────────────
teardown_test
echo ""
echo "Test completed!"
echo " Screenshots: $SCREENSHOT_DIR/$TEST_NAME/"
echo ""