Leave GitHub comment with playwright results (#1965)

<!-- CURSOR_SUMMARY -->
> [!NOTE]
> Posts a per-OS Playwright test summary as a PR comment by adding a
JSON reporter and a CI step to generate and publish the results.
> 
> - **CI / Reporting**
> - Add permissions and a GitHub Script step in
`.github/workflows/ci.yml` to generate and post/update a Playwright test
summary comment after merging shard reports.
>   - Upload merged HTML report artifact and link to full run.
> - **Playwright config**
> - Update `merge.config.ts` to add JSON reporter output to
`playwright-report/results.json` alongside HTML.
> - **New Script**
> - Add `scripts/generate-playwright-summary.js` to parse Playwright
JSON, compute per-OS (macOS/Windows) pass/fail/flaky/skipped counts,
list top failures/flaky tests, and write both PR comment and job
summary.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
d5ca6987f65e9a7063533960382516af89e67391. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->





<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Posts Playwright test results as a PR comment with a per-OS summary,
flaky test counts, and top failures, improving CI visibility across
macOS and Windows. Adds a JSON reporter and a GitHub Script step that
links to the full report.

- **New Features**
  - Adds JSON reporter output to playwright-report/results.json.
- Adds a script to parse results, include flaky tests, post/update the
PR comment, and write the job summary.
- Updates CI workflow permissions and runs the summary script after
report merge.

<sup>Written for commit d5ca6987f65e9a7063533960382516af89e67391.
Summary will update automatically on new commits.</sup>

<!-- End of auto-generated description by cubic. -->
This commit is contained in:
Will Chen
2025-12-15 23:24:08 -08:00
committed by GitHub
parent 91cf1e97c3
commit 47992f48dd
3 changed files with 296 additions and 1 deletions

View File

@@ -1,4 +1,7 @@
export default {
testDir: "e2e-tests",
reporter: [["html", { open: "never" }]],
reporter: [
["html", { open: "never" }],
["json", { outputFile: "playwright-report/results.json" }],
],
};