Xây kênh TikTok và thương hiệu cá nhân từ đầu, định vị chuyên gia TMĐT và Quản trị IT, lên ý tưởng, viết script và content calendar.
--- name: xay-dung-thuong-hieu-ca-nhan description: Xây dựng kênh TikTok và thương hiệu cá nhân từ số 0, định vị chuyên gia dựa trên kinh nghiệm thực tế (TMĐT & Quản trị IT), lên ý tưởng, viết script video và lập content calendar. Dùng khi nói "thương hiệu cá nhân", "xây kênh TikTok", "personal brand", "script video". --- # Xây dựng thương hiệu cá nhân trên TikTok (Personal Branding) ## Mục tiêu Xây dựng kênh TikTok cá nhân từ số 0 — định vị rõ ràng dựa trên kinh nghiệm thực tế về kinh doanh TMĐT (Chargee) và quản trị IT (Elmich) — để tạo thương hiệu cá nhân có uy tín và ảnh hưởng. ## Bối cảnh cá nhân - **Góc độ độc đáo**: Vừa là chủ doanh nghiệp TMĐT, vừa là Trưởng phòng IT tại công ty sản xuất - **Nội dung có thể khai thác**: Vận hành thực tế, bài học thất bại/thành công, góc nhìn kép (kỹ thuật + kinh doanh) - **Kênh hiện tại**: Giai đoạn 0 — kênh mới bắt đầu ## Khi nào dùng - Lên ý tưởng nội dung cho video mới - Xây dựng hoặc điều chỉnh định vị cá nhân - Phân tích video đang hoạt động tốt/kém - Lên kế hoạch đăng bài theo tuần/tháng - Viết script hoặc outline cho video ## Đầu vào cần cung cấp - Giai đoạn hiện tại của kênh (số follow, số video) - Chủ đề muốn làm video - Kinh nghiệm thực tế liên quan đến chủ đề đó - Thời gian có thể quay/đăng mỗi tuần - Phong cách muốn thể hiện: chia sẻ thẳng thắn / phân tích chuyên sâu / kể chuyện / dạy học ## Quy trình xây dựng kênh theo giai đoạn ### Giai đoạn 1 — Nền móng (0 → 1.000 follow) Mục tiêu: tìm được "content-audience fit" — biết mình nói gì, nói cho ai, và ai thực sự quan tâm 1. Xác định 3 chủ đề cốt lõi dựa trên kinh nghiệm thực tế 2. Chọn 1 định dạng video chủ đạo để thử nghiệm 3. Đăng 3 video/tuần — đo phản hồi sau 4 tuần 4. Xác định video nào có retention và comment tốt nhất 5. Double down vào chủ đề/định dạng đó ### Giai đoạn 2 — Tăng trưởng (1.000 → 10.000 follow) Mục tiêu: nhất quán và có hệ thống 1. Xây dựng content calendar cố định theo tuần 2. Tạo series nội dung có tính liên tục 3. Tối ưu hook 3 giây đầu và CTA cuối video 4. Phân tích analytics mỗi tuần — điều chỉnh theo dữ liệu 5. Bắt đầu xây dựng nhận diện: tone giọng, phong cách quay ### Giai đoạn 3 — Định vị (10.000+ follow) Mục tiêu: trở thành tên đáng tin cậy trong lĩnh vực 1. Tập trung vào 1–2 chủ đề chuyên sâu thay vì rộng 2. Collab với người có cùng lĩnh vực 3. Chuyển một phần nội dung thành dạng giáo dục có chiều sâu 4. Xây dựng community: trả lời comment, tạo video reply ## Góc nội dung đề xuất | Góc nội dung | Ví dụ chủ đề cụ thể | Độ khó sản xuất | |---|---|---| | Bài học thực tế từ Chargee | "Sai lầm khi mở shop Shopee đầu tiên" | Thấp | | Góc nhìn chủ doanh nghiệp | "Một ngày làm việc của tôi với 4 kênh TMĐT" | Thấp | | Kinh nghiệm quản trị IT | "IT trong công ty sản xuất khác gì startup" | Trung bình | | Phân tích TMĐT | "Tại sao TikTok Shop đang thắng Shopee ở ngách X" | Trung bình | | Kép: kỹ thuật + kinh doanh | "Tôi dùng công nghệ gì để vận hành Chargee" | Trung bình | ## Tiêu chuẩn đầu ra theo yêu cầu | Yêu cầu | Đầu ra Claude cung cấp | |---|---| | Lên ý tưởng | 5–10 ý tưởng video có tiêu đề + hook | | Viết script | Outline đầy đủ: hook / thân / CTA | | Lên kế hoạch | Content calendar theo tuần dạng bảng | | Phân tích video | Nhận xét hook, retention, CTA + đề xuất cải thiện | | Định vị | Mô tả positioning 1 câu + 3 chủ đề cốt lõi | ## Framework script chuẩn cho mỗi video 1. **Hook (0–3 giây)**: câu mở gây tò mò hoặc nêu vấn đề thực tế — không giới thiệu bản thân 2. **Context (3–15 giây)**: bối cảnh ngắn gọn — tại sao chủ đề này quan trọng với người xem 3. **Nội dung chính (15–45 giây)**: 3 điểm chính hoặc 1 câu chuyện có arc rõ ràng 4. **Bài học / insight (45–55 giây)**: 1 takeaway cụ thể người xem có thể áp dụng ngay 5. **CTA (55–60 giây)**: follow / comment / xem video tiếp theo ## Nguyên tắc personal brand - **Nói từ kinh nghiệm thực tế** — không dạy lý thuyết nếu chưa làm - **Nhất quán về góc nhìn** — bạn là người vừa làm kỹ thuật vừa làm kinh doanh — đó là điểm khác biệt - **Thất bại có giá trị hơn thành công** — chia sẻ bài học từ sai lầm thực tế tạo niềm tin nhanh hơn - **Không cần hoàn hảo** — video chân thực > video production cao nhưng thiếu cảm xúc ## Tránh - Làm nội dung quá rộng, không có góc nhìn riêng - Copy trend mà không gắn với kinh nghiệm thực tế của bạn - Đăng không đều — consistency quan trọng hơn chất lượng ở giai đoạn đầu - Giới thiệu bản thân ngay đầu video — người xem không quan tâm cho đến khi bạn cho họ lý do
Quét, sửa và xác minh tuân thủ WCAG 2.2 mức A và AA cho React, Next.js, Vue, Angular, Svelte và HTML thuần.
---
name: "a11y-audit"
description: "Accessibility audit skill for scanning, fixing, and verifying WCAG 2.2 Level A and AA compliance across React, Next.js, Vue, Angular, Svelte, and plain HTML codebases. Use when auditing accessibility, fixing a11y violations, checking color contrast, generating compliance reports, or integrating accessibility checks into CI/CD pipelines."
---
# Accessibility Audit
WCAG 2.2 Accessibility Audit and Remediation Skill
## Description
The a11y-audit skill provides a complete accessibility audit pipeline for modern web applications. It implements a three-phase workflow -- Scan, Fix, Verify -- that identifies WCAG 2.2 Level A and AA violations, generates exact fix code per framework, and produces stakeholder-ready compliance reports.
For every violation it finds, it provides the precise before/after code fix tailored to your framework (React, Next.js, Vue, Angular, Svelte, or plain HTML).
**What this skill does:**
1. **Scans** your codebase for every WCAG 2.2 Level A and AA violation, categorized by severity (Critical, Major, Minor)
2. **Fixes** each violation with framework-specific before/after code patterns
3. **Verifies** that fixes resolve the original violations and introduces no regressions
4. **Reports** findings in a structured format suitable for developers, PMs, and compliance stakeholders
5. **Integrates** into CI/CD pipelines to prevent accessibility regressions
## Features
| Feature | Description |
|---------|-------------|
| **Full WCAG 2.2 Scan** | Checks all Level A and AA success criteria across your codebase |
| **Framework Detection** | Auto-detects React, Next.js, Vue, Angular, Svelte, or plain HTML |
| **Severity Classification** | Categorizes each violation as Critical, Major, or Minor |
| **Fix Code Generation** | Produces before/after code diffs for every issue |
| **Color Contrast Checker** | Validates foreground/background pairs against AA and AAA ratios |
| **Compliance Reporting** | Generates stakeholder reports with pass/fail summaries |
| **CI/CD Integration** | GitHub Actions, GitLab CI, Azure DevOps pipeline configs |
| **Keyboard Navigation Audit** | Detects missing focus management and tab order issues |
| **ARIA Validation** | Checks for incorrect, redundant, or missing ARIA attributes |
### Severity Definitions
| Severity | Definition | Example | SLA |
|----------|-----------|---------|-----|
| **Critical** | Blocks access for entire user groups | Missing alt text, no keyboard access to navigation | Fix before release |
| **Major** | Significant barrier that degrades experience | Insufficient color contrast, missing form labels | Fix within current sprint |
| **Minor** | Usability issue that causes friction | Redundant ARIA roles, suboptimal heading hierarchy | Fix within next 2 sprints |
## Usage
### Quick Start
```bash
# Scan entire project
python scripts/a11y_scanner.py /path/to/project
# Scan with JSON output for tooling
python scripts/a11y_scanner.py /path/to/project --json
# Check color contrast for specific values
python scripts/contrast_checker.py --fg "#777777" --bg "#ffffff"
# Check contrast across a CSS/Tailwind file
python scripts/contrast_checker.py --file /path/to/styles.css
```
### Slash Command
```
/a11y-audit # Audit current project
/a11y-audit --scope src/ # Audit specific directory
/a11y-audit --fix # Audit and auto-apply fixes
/a11y-audit --report # Generate stakeholder report
/a11y-audit --ci # Output CI-compatible results
```
### Three-Phase Workflow
**Phase 1: Scan** -- Walk the source tree, detect framework, apply rule set.
```bash
python scripts/a11y_scanner.py /path/to/project --format table
```
**Phase 2: Fix** -- Apply framework-specific fixes for each violation.
> See [references/framework-a11y-patterns.md](references/framework-a11y-patterns.md) for the complete fix patterns catalog.
**Phase 3: Verify** -- Re-run the scanner to confirm fixes and check for regressions.
```bash
python scripts/a11y_scanner.py /path/to/project --baseline audit-baseline.json
```
## Example: React Component Audit
```tsx
// BEFORE: src/components/ProductCard.tsx
function ProductCard({ product }) {
return (
<div onClick={() => navigate(`/product/product.id`)}>
<img src={product.image} />
<div style={{ color: '#aaa', fontSize: '12px' }}>{product.name}</div>
<span style={{ color: '#999' }}>product.price</span>
</div>
);
}
```
| # | WCAG | Severity | Issue |
|---|------|----------|-------|
| 1 | 1.1.1 | Critical | `<img>` missing `alt` attribute |
| 2 | 2.1.1 | Critical | `<div onClick>` not keyboard accessible |
| 3 | 1.4.3 | Major | Color `#aaa` on white fails contrast (2.32:1, needs 4.5:1) |
| 4 | 1.4.3 | Major | Color `#999` on white fails contrast (2.85:1, needs 4.5:1) |
| 5 | 4.1.2 | Major | Interactive element missing role and accessible name |
```tsx
// AFTER: src/components/ProductCard.tsx
function ProductCard({ product }) {
return (
<a href={`/product/product.id`} className="product-card"
aria-label={`View product.name - $product.price`}>
<img src={product.image} alt={product.imageAlt || product.name} />
<div style={{ color: '#595959', fontSize: '12px' }}>{product.name}</div>
<span style={{ color: '#767676' }}>product.price</span>
</a>
);
}
```
> See [references/examples-by-framework.md](references/examples-by-framework.md) for Vue, Angular, Next.js, and Svelte examples.
## Tools Reference
### a11y_scanner.py
```
Usage: python scripts/a11y_scanner.py <path> [options]
Options:
--json Output results as JSON
--format {table,csv} Output format (default: table)
--severity {critical,major,minor} Filter by minimum severity
--framework {react,vue,angular,svelte,html,auto} Force framework (default: auto)
--baseline FILE Compare against previous scan results
--report Generate stakeholder report
--output FILE Write results to file
--quiet Suppress output, exit code only
--ci CI mode: non-zero exit on critical issues
```
### contrast_checker.py
```
Usage: python scripts/contrast_checker.py [options]
Options:
--fg COLOR Foreground color (hex)
--bg COLOR Background color (hex)
--file FILE Scan CSS file for color pairs
--tailwind DIR Scan directory for Tailwind color classes
--json Output results as JSON
--suggest Suggest accessible alternatives for failures
--level {aa,aaa} Target conformance level (default: aa)
```
## Common Pitfalls
| Pitfall | Correct Approach |
|---------|------------------|
| `role="button"` on a `<div>` | Use native `<button>` -- includes keyboard handling for free |
| `tabindex="0"` on everything | Only interactive elements need focus; use native elements |
| `aria-label` on non-interactive elements | Use `aria-labelledby` pointing to visible text |
| `display: none` for screen reader hiding | Use `.sr-only` class instead |
| Color alone to convey meaning | Add icons, text labels, or patterns alongside color |
| Placeholder as only label | Always provide a visible `<label>` |
| `outline: none` without replacement | Always provide a visible focus indicator via `focus-visible` |
| Empty `alt=""` on informational images | Informational images need descriptive alt text |
| Skipping heading levels (h1 -> h3) | Heading levels must be sequential |
| `onClick` without `onKeyDown` | Add keyboard support or prefer native elements |
| Ignoring `prefers-reduced-motion` | Wrap animations in `@media (prefers-reduced-motion: no-preference)` |
## Related Skills
| Skill | Relationship |
|-------|-------------|
| **senior-frontend** | Frontend patterns used in a11y fixes |
| **code-reviewer** | Include a11y checks in code review workflows |
| **senior-qa** | Integration of a11y testing into QA processes |
| **playwright-pro** | Automated browser testing with accessibility assertions |
| **epic-design** | WCAG 2.1 AA compliant animations and scroll storytelling |
| **tdd-guide** | Test-driven development patterns for a11y test cases |
## Reference Documentation
| Reference | Description |
|-----------|-------------|
| [wcag-quick-ref.md](references/wcag-quick-ref.md) | WCAG 2.2 Level A & AA criteria quick reference |
| [wcag-22-new-criteria.md](references/wcag-22-new-criteria.md) | New WCAG 2.2 success criteria (Focus Appearance, Target Size, etc.) |
| [aria-patterns.md](references/aria-patterns.md) | ARIA patterns, keyboard interaction, and live regions |
| [framework-a11y-patterns.md](references/framework-a11y-patterns.md) | Framework-specific fix patterns (React, Vue, Angular, Svelte, HTML) |
| [color-contrast-guide.md](references/color-contrast-guide.md) | Color contrast checker details, Tailwind palette mapping, sr-only class |
| [ci-cd-integration.md](references/ci-cd-integration.md) | GitHub Actions, GitLab CI, Azure DevOps, pre-commit hook configs |
| [audit-report-template.md](references/audit-report-template.md) | Stakeholder-ready audit report template |
| [testing-checklist.md](references/testing-checklist.md) | Manual testing checklist (keyboard, screen reader, visual, forms) |
| [examples-by-framework.md](references/examples-by-framework.md) | Full audit examples for Vue, Angular, Next.js, and Svelte |
## Resources
- [WCAG 2.2 Specification](https://www.w3.org/TR/WCAG22/)
- [WAI-ARIA Authoring Practices 1.2](https://www.w3.org/WAI/ARIA/apg/)
- [Deque axe-core Rules](https://github.com/dequelabs/axe-core/blob/develop/doc/rule-descriptions.md)
- [eslint-plugin-jsx-a11y](https://github.com/jsx-eslint/eslint-plugin-jsx-a11y)
FILE:assets/sample-component.tsx
// Sample React component with intentional a11y issues for testing
import React from 'react';
export function UserCard({ user, onEdit, onDelete }) {
return (
<div className="card" onClick={() => onEdit(user.id)}>
<img src={user.avatar} />
<div className="name">{user.name}</div>
<div className="email">{user.email}</div>
<div className="actions">
<div onClick={() => onDelete(user.id)} style={{ color: '#aaa', cursor: 'pointer' }}>
Delete
</div>
<a href="#">Edit</a>
</div>
<input placeholder="Add note" />
</div>
);
}
export function SearchBar() {
return (
<div>
<input type="text" placeholder="Search..." />
<div onClick={() => alert('searching')} tabIndex={5}>
🔍
</div>
</div>
);
}
export function DataTable({ rows }) {
return (
<table>
<tr>
<td><b>Name</b></td>
<td><b>Email</b></td>
<td><b>Status</b></td>
</tr>
{rows.map((row) => (
<tr key={row.id}>
<td>{row.name}</td>
<td>{row.email}</td>
<td style={{ color: row.active ? 'green' : 'red' }}>
{row.active ? '●' : '●'}
</td>
</tr>
))}
</table>
);
}
FILE:expected_outputs/sample-contrast-output.txt
Contrast Check: #777777 on #ffffff
Foreground: #777777 (r=119, g=119, b=119)
Background: #ffffff (r=255, g=255, b=255)
Contrast Ratio: 4.48:1
Normal text (4.5:1 required):
AA: FAIL (4.48 < 4.5)
AAA: FAIL (4.48 < 7.0)
Large text (3.0:1 required):
AA: PASS (4.48 >= 3.0)
AAA: FAIL (4.48 < 4.5)
UI components (3.0:1 required):
AA: PASS (4.48 >= 3.0)
Verdict: FAIL — does not meet AA for normal text
---
Contrast Check: #1a1a2e on #ffffff
Foreground: #1a1a2e (r=26, g=26, b=46)
Background: #ffffff (r=255, g=255, b=255)
Contrast Ratio: 17.06:1
Normal text (4.5:1 required):
AA: PASS
AAA: PASS
Large text (3.0:1 required):
AA: PASS
AAA: PASS
UI components (3.0:1 required):
AA: PASS
Verdict: PASS — meets AAA for all categories
FILE:expected_outputs/sample-scan-output.json
{
"summary": {
"files_scanned": 1,
"files_with_issues": 1,
"total_issues": 9,
"critical": 3,
"serious": 4,
"moderate": 2,
"minor": 0,
"verdict": "FAIL"
},
"findings": [
{
"severity": "critical",
"category": "IMG-ALT",
"file": "sample-component.tsx",
"line": 7,
"code": "<img src={user.avatar} />",
"wcag": "1.1.1",
"message": "Image missing alt attribute",
"fix": "Add alt text: alt=\"description of image\""
},
{
"severity": "critical",
"category": "KB-CLICK",
"file": "sample-component.tsx",
"line": 5,
"code": "<div className=\"card\" onClick={() => onEdit(user.id)}>",
"wcag": "2.1.1",
"message": "Click handler on non-interactive element without keyboard support",
"fix": "Use <button> or add role=\"button\", tabIndex={0}, onKeyDown"
}
]
}
FILE:expected_outputs/sample-scan-report.md
# A11y Audit Report — sample-component.tsx
**Scanned:** 1 file | **Issues:** 9 | **Status:** FAIL
## Critical (3)
### 1. Missing alt text on image
- **File:** sample-component.tsx:7
- **Code:** `<img src={user.avatar} />`
- **WCAG:** 1.1.1 Non-text Content (Level A)
- **Fix:** Add descriptive alt text: `<img src={user.avatar} alt={`user.name's avatar`} />`
### 2. Click handler without keyboard support
- **File:** sample-component.tsx:5
- **Code:** `<div className="card" onClick={() => onEdit(user.id)}>`
- **WCAG:** 2.1.1 Keyboard (Level A)
- **Fix:** Use `<button>` or add `role="button"`, `tabIndex={0}`, and `onKeyDown`
### 3. Click handler without keyboard support
- **File:** sample-component.tsx:11
- **Code:** `<div onClick={() => onDelete(user.id)} ...>`
- **WCAG:** 2.1.1 Keyboard (Level A)
- **Fix:** Replace `<div>` with `<button>`
## Serious (4)
### 4. Missing form label
- **File:** sample-component.tsx:15
- **Code:** `<input placeholder="Add note" />`
- **WCAG:** 3.3.2 Labels or Instructions (Level A)
- **Fix:** Add `<label>` or `aria-label="Add note"`
### 5. Empty link
- **File:** sample-component.tsx:14
- **Code:** `<a href="#">Edit</a>`
- **WCAG:** 2.4.4 Link Purpose (Level A)
- **Fix:** Use a real href or replace with `<button>`
### 6. tabindex greater than 0
- **File:** sample-component.tsx:24
- **Code:** `tabIndex={5}`
- **WCAG:** 2.4.3 Focus Order (Level A)
- **Fix:** Use `tabIndex={0}` — positive values disrupt natural tab order
### 7. Missing table headers
- **File:** sample-component.tsx:30
- **Code:** `<td><b>Name</b></td>` (using td+b instead of th)
- **WCAG:** 1.3.1 Info and Relationships (Level A)
- **Fix:** Use `<th scope="col">Name</th>`
## Moderate (2)
### 8. Missing form label
- **File:** sample-component.tsx:22
- **Code:** `<input type="text" placeholder="Search..." />`
- **WCAG:** 3.3.2 Labels or Instructions (Level A)
- **Fix:** Add `aria-label="Search"` or visible label
### 9. Color as sole indicator
- **File:** sample-component.tsx:38
- **Code:** `style={{ color: row.active ? 'green' : 'red' }}`
- **WCAG:** 1.4.1 Use of Color (Level A)
- **Fix:** Add text or icon alongside color: `{row.active ? '✓ Active' : '✗ Inactive'}`
FILE:references/aria-patterns.md
# ARIA Patterns & Keyboard Interaction Reference
## Landmark Roles
Every page should have these landmarks:
```html
<header role="banner"> <!-- Site header — once per page -->
<nav role="navigation"> <!-- Navigation — can have multiple with aria-label -->
<main role="main"> <!-- Main content — once per page -->
<aside role="complementary"> <!-- Sidebar — related but not essential -->
<footer role="contentinfo"> <!-- Site footer — once per page -->
<form role="search"> <!-- Search form -->
```
**Semantic HTML equivalents:** `<header>`, `<nav>`, `<main>`, `<aside>`, `<footer>` provide implicit roles — no need to double up with explicit `role` attributes.
## Live Regions
### When to Use
| Pattern | Attribute | Use Case |
|---------|-----------|----------|
| Polite | `aria-live="polite"` | Toast notifications, status updates, search result counts |
| Assertive | `aria-live="assertive"` | Error messages, urgent alerts, form validation errors |
| Status | `role="status"` | Loading indicators, progress updates |
| Alert | `role="alert"` | Error dialogs, time-sensitive warnings |
| Log | `role="log"` | Chat messages, activity feeds |
| Timer | `role="timer"` | Countdown timers |
### Implementation
```html
<!-- Toast notifications -->
<div aria-live="polite" aria-atomic="true">
<!-- Inject toast content here dynamically -->
</div>
<!-- Form validation errors -->
<div aria-live="assertive" role="alert">
<p>Please enter a valid email address.</p>
</div>
<!-- Loading state -->
<div role="status" aria-live="polite">
Loading results...
</div>
```
**Key rule:** The live region container must exist in the DOM *before* content is injected. Adding `aria-live` to a newly created element won't announce it.
## Focus Management
### Focus Trap (Modals)
```javascript
// Trap focus inside modal
const modal = document.querySelector('[role="dialog"]');
const focusable = modal.querySelectorAll(
'a[href], button, textarea, input, select, [tabindex]:not([tabindex="-1"])'
);
const first = focusable[0];
const last = focusable[focusable.length - 1];
modal.addEventListener('keydown', (e) => {
if (e.key === 'Tab') {
if (e.shiftKey && document.activeElement === first) {
e.preventDefault();
last.focus();
} else if (!e.shiftKey && document.activeElement === last) {
e.preventDefault();
first.focus();
}
}
if (e.key === 'Escape') closeModal();
});
```
### Focus Restoration
```javascript
// Save focus before opening modal
const trigger = document.activeElement;
openModal();
// Restore focus on close
function closeModal() {
modal.hidden = true;
trigger.focus();
}
```
### Skip Link
```html
<a href="#main-content" class="skip-link">Skip to main content</a>
<!-- ... navigation ... -->
<main id="main-content" tabindex="-1">
```
```css
.skip-link {
position: absolute;
left: -9999px;
z-index: 999;
}
.skip-link:focus {
left: 10px;
top: 10px;
background: #000;
color: #fff;
padding: 8px 16px;
}
```
## Keyboard Interaction Patterns
### Tabs
```
Tab → Move to tab list, then to tab panel
Arrow Left/Right → Switch between tabs
Home → First tab
End → Last tab
```
```html
<div role="tablist" aria-label="Settings">
<button role="tab" aria-selected="true" aria-controls="panel-1" id="tab-1">General</button>
<button role="tab" aria-selected="false" aria-controls="panel-2" id="tab-2" tabindex="-1">Security</button>
</div>
<div role="tabpanel" id="panel-1" aria-labelledby="tab-1">...</div>
<div role="tabpanel" id="panel-2" aria-labelledby="tab-2" hidden>...</div>
```
### Combobox / Autocomplete
```
Arrow Down → Open list / next option
Arrow Up → Previous option
Enter → Select option
Escape → Close list
Type → Filter options
```
### Menu
```
Enter/Space → Activate item
Arrow Down → Next item
Arrow Up → Previous item
Arrow Right → Open submenu
Arrow Left → Close submenu
Escape → Close menu
```
### Accordion
```
Enter/Space → Toggle section
Arrow Down → Next header
Arrow Up → Previous header
Home → First header
End → Last header
```
## Framework-Specific ARIA
### React
```jsx
// Announce route changes (SPA)
<div aria-live="polite" className="sr-only">
{`Navigated to pageTitle`}
</div>
// Error boundary with accessible error
<div role="alert">
<h2>Something went wrong</h2>
<p>{error.message}</p>
</div>
```
### Vue
```vue
<!-- Announce dynamic content -->
<div aria-live="polite">
<p v-if="results.length">{{ results.length }} results found</p>
</div>
<!-- Accessible toggle -->
<button
:aria-expanded="isOpen"
:aria-controls="panelId"
@click="toggle"
>
{{ isOpen ? 'Collapse' : 'Expand' }}
</button>
```
### Angular
```html
<!-- cdkTrapFocus for modals -->
<div cdkTrapFocus cdkTrapFocusAutoCapture role="dialog" aria-labelledby="dialog-title">
<h2 id="dialog-title">Confirm Action</h2>
</div>
<!-- LiveAnnouncer service -->
<!-- In component: this.liveAnnouncer.announce('Item added to cart'); -->
```
## Common ARIA Mistakes
| Mistake | Why It's Wrong | Fix |
|---------|---------------|-----|
| `<div role="button">` without keyboard | Div doesn't get keyboard events | Use `<button>` or add `tabindex="0"` + `onkeydown` |
| `aria-hidden="true"` on focusable element | Screen reader skips it but keyboard reaches it | Remove from tab order too: `tabindex="-1"` |
| `aria-label` overriding visible text | Confusing for sighted screen reader users | Use `aria-labelledby` pointing to visible text |
| Redundant ARIA on semantic HTML | `<nav role="navigation">` is redundant | Drop the `role` — `<nav>` implies it |
| `aria-live` on container that already has content | Initial content gets announced on load | Add `aria-live` to empty container, inject content after |
| Missing `aria-expanded` on toggles | Screen reader can't tell if section is open | Add `aria-expanded="true/false"` |
FILE:references/audit-report-template.md
# Audit Report Template
The scanner generates a stakeholder-ready report when run with the `--report` flag:
```bash
python scripts/a11y_scanner.py /path/to/project --report --output audit-report.md
```
## Generated Report Structure
```markdown
# Accessibility Audit Report
**Project:** Acme Dashboard
**Date:** 2026-03-18
**Standard:** WCAG 2.2 Level AA
**Tool:** a11y-audit v2.1.2
## Executive Summary
- Files Scanned: 127
- Total Violations: 14
- Critical: 3 | Major: 7 | Minor: 4
- Estimated Remediation: 8-12 hours
- Compliance Score: 72% (Target: 100%)
## Violations by Category
| Category | Count | Severity Breakdown |
|----------|-------|--------------------|
| Missing Alt Text | 3 | 2 Critical, 1 Minor |
| Keyboard Access | 4 | 2 Critical, 2 Major |
| Color Contrast | 3 | 3 Major |
| Form Labels | 2 | 2 Major |
| ARIA Usage | 2 | 2 Minor |
## Detailed Findings
[Per-violation details with file, line, WCAG criterion, and fix]
## Remediation Priority
1. Fix all Critical issues (blocks release)
2. Fix Major issues in current sprint
3. Schedule Minor issues for next sprint
## Recommendations
- Add a11y linting to CI pipeline (eslint-plugin-jsx-a11y)
- Include keyboard testing in QA checklist
- Schedule quarterly manual audit with assistive technology
```
FILE:references/ci-cd-integration.md
# CI/CD Integration for Accessibility Auditing
## GitHub Actions
```yaml
# .github/workflows/a11y-audit.yml
name: Accessibility Audit
on:
pull_request:
paths:
- 'src/**/*.tsx'
- 'src/**/*.vue'
- 'src/**/*.html'
- 'src/**/*.svelte'
jobs:
a11y-audit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Run A11y Scanner
run: |
python scripts/a11y_scanner.py ./src --json > a11y-results.json
- name: Check for Critical Issues
run: |
python -c "
import json, sys
with open('a11y-results.json') as f:
data = json.load(f)
critical = [v for v in data.get('violations', []) if v['severity'] == 'critical']
if critical:
print(f'FAILED: {len(critical)} critical a11y violations found')
for v in critical:
print(f\" [{v['wcag']}] {v['file']}:{v['line']} - {v['message']}\")
sys.exit(1)
print('PASSED: No critical a11y violations')
"
- name: Upload Audit Report
if: always()
uses: actions/upload-artifact@v4
with:
name: a11y-audit-report
path: a11y-results.json
- name: Comment on PR
if: failure()
uses: marocchino/sticky-pull-request-comment@v2
with:
header: a11y-audit
message: |
## Accessibility Audit Failed
Critical WCAG 2.2 violations were found. See the uploaded artifact for details.
Run `python scripts/a11y_scanner.py ./src` locally to view and fix issues.
```
## GitLab CI
```yaml
# .gitlab-ci.yml
a11y-audit:
stage: test
image: python:3.11-slim
script:
- python scripts/a11y_scanner.py ./src --json > a11y-results.json
- python -c "
import json, sys;
data = json.load(open('a11y-results.json'));
critical = [v for v in data.get('violations', []) if v['severity'] == 'critical'];
sys.exit(1) if critical else print('A11y audit passed')
"
artifacts:
paths:
- a11y-results.json
when: always
rules:
- changes:
- "src/**/*.{tsx,vue,html,svelte}"
```
## Azure DevOps
```yaml
# azure-pipelines.yml
- task: PythonScript@0
displayName: 'Run A11y Audit'
inputs:
scriptSource: 'filePath'
scriptPath: 'scripts/a11y_scanner.py'
arguments: './src --json --output $(Build.ArtifactStagingDirectory)/a11y-results.json'
- task: PublishBuildArtifacts@1
condition: always()
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/a11y-results.json'
ArtifactName: 'a11y-audit-report'
```
## Pre-Commit Hook
```bash
#!/bin/bash
# .git/hooks/pre-commit
# Run a11y scan on staged files only
STAGED_FILES=$(git diff --cached --name-only --diff-filter=ACM | grep -E '\.(tsx|vue|html|svelte|jsx)$')
if [ -n "$STAGED_FILES" ]; then
echo "Running accessibility audit on staged files..."
for file in $STAGED_FILES; do
python scripts/a11y_scanner.py "$file" --severity critical --quiet
if [ $? -ne 0 ]; then
echo "A11y audit FAILED for $file. Fix critical issues before committing."
exit 1
fi
done
echo "A11y audit passed."
fi
```
FILE:references/color-contrast-guide.md
# Color Contrast Guide
## Contrast Checker Usage
The `contrast_checker.py` script validates color pairs against WCAG 2.2 contrast requirements.
```bash
# Check a single color pair
python scripts/contrast_checker.py --fg "#777777" --bg "#ffffff"
# Output:
# Foreground: #777777 | Background: #ffffff
# Contrast Ratio: 4.48:1
# AA Normal Text (4.5:1): FAIL
# AA Large Text (3.0:1): PASS
# AAA Normal Text (7.0:1): FAIL
# Suggested alternative: #767676 (4.54:1 - passes AA)
# Scan a CSS file for all color pairs
python scripts/contrast_checker.py --file src/styles/globals.css
# Scan Tailwind classes in components
python scripts/contrast_checker.py --tailwind src/components/
```
## Common Contrast Fixes
| Original Color | Contrast on White | Fix | New Contrast |
|----------------|------------------|-----|--------------|
| `#aaaaaa` | 2.32:1 | `#767676` | 4.54:1 (AA) |
| `#999999` | 2.85:1 | `#767676` | 4.54:1 (AA) |
| `#888888` | 3.54:1 | `#767676` | 4.54:1 (AA) |
| `#777777` | 4.48:1 | `#757575` | 4.60:1 (AA) |
| `#66bb6a` | 3.06:1 | `#2e7d32` | 5.87:1 (AA) |
| `#42a5f5` | 2.81:1 | `#1565c0` | 6.08:1 (AA) |
| `#ef5350` | 3.13:1 | `#c62828` | 5.57:1 (AA) |
## Tailwind CSS Accessible Palette Mapping
| Inaccessible Class | Contrast on White | Accessible Alternative | Contrast |
|---------------------|------------------|----------------------|----------|
| `text-gray-400` | 2.68:1 | `text-gray-600` | 5.74:1 |
| `text-blue-400` | 2.81:1 | `text-blue-700` | 5.96:1 |
| `text-green-400` | 2.12:1 | `text-green-700` | 5.18:1 |
| `text-red-400` | 3.04:1 | `text-red-700` | 6.05:1 |
| `text-yellow-500` | 1.47:1 | `text-yellow-800` | 7.34:1 |
## Screen Reader Utility Class
Every project should include this utility class for visually hiding content while keeping it accessible to screen readers:
```css
/* Visually hidden but accessible to screen readers */
.sr-only {
position: absolute;
width: 1px;
height: 1px;
padding: 0;
margin: -1px;
overflow: hidden;
clip: rect(0, 0, 0, 0);
white-space: nowrap;
border-width: 0;
}
/* Allow the element to be focusable when navigated to via keyboard */
.sr-only-focusable:focus,
.sr-only-focusable:active {
position: static;
width: auto;
height: auto;
padding: inherit;
margin: inherit;
overflow: visible;
clip: auto;
white-space: inherit;
}
```
Tailwind CSS includes this as `sr-only` by default. For other frameworks:
- **Angular**: Add to `styles.scss`
- **Vue**: Add to `assets/global.css`
- **Svelte**: Add to `app.css`
FILE:references/examples-by-framework.md
# Accessibility Audit Examples by Framework
## Example 1: Vue SFC Form Audit
```vue
<!-- BEFORE: src/components/LoginForm.vue -->
<template>
<form @submit="handleLogin">
<input type="text" placeholder="Email" v-model="email" />
<input type="password" placeholder="Password" v-model="password" />
<div v-if="error" style="color: red">{{ error }}</div>
<div @click="handleLogin">Sign In</div>
</form>
</template>
```
**Violations detected:**
| # | WCAG | Severity | Issue |
|---|------|----------|-------|
| 1 | 1.3.1 | Critical | Inputs missing associated `<label>` elements |
| 2 | 3.3.2 | Major | Placeholder text used as only label (disappears on input) |
| 3 | 2.1.1 | Critical | `<div @click>` not keyboard accessible |
| 4 | 4.1.3 | Major | Error message not announced to screen readers |
| 5 | 3.3.1 | Major | Error not programmatically associated with input |
```vue
<!-- AFTER: src/components/LoginForm.vue -->
<template>
<form @submit.prevent="handleLogin" aria-label="Sign in to your account">
<div class="field">
<label for="login-email">Email</label>
<input
id="login-email"
type="email"
v-model="email"
autocomplete="email"
required
:aria-describedby="emailError ? 'email-error' : undefined"
:aria-invalid="!!emailError"
/>
<span v-if="emailError" id="email-error" role="alert">
{{ emailError }}
</span>
</div>
<div class="field">
<label for="login-password">Password</label>
<input
id="login-password"
type="password"
v-model="password"
autocomplete="current-password"
required
:aria-describedby="passwordError ? 'password-error' : undefined"
:aria-invalid="!!passwordError"
/>
<span v-if="passwordError" id="password-error" role="alert">
{{ passwordError }}
</span>
</div>
<div v-if="error" role="alert" aria-live="assertive" class="form-error">
{{ error }}
</div>
<button type="submit">Sign In</button>
</form>
</template>
```
## Example 2: Angular Template Audit
```html
<!-- BEFORE: src/app/dashboard/dashboard.component.html -->
<div class="tabs">
<div *ngFor="let tab of tabs"
(click)="selectTab(tab)"
[class.active]="tab.active">
{{ tab.label }}
</div>
</div>
<div class="tab-content">
<div *ngIf="selectedTab">{{ selectedTab.content }}</div>
</div>
```
**Violations detected:**
| # | WCAG | Severity | Issue |
|---|------|----------|-------|
| 1 | 4.1.2 | Critical | Tab widget missing ARIA roles (`tablist`, `tab`, `tabpanel`) |
| 2 | 2.1.1 | Critical | Tabs not keyboard navigable (arrow keys, Home, End) |
| 3 | 2.4.11 | Major | No visible focus indicator on active tab |
```html
<!-- AFTER: src/app/dashboard/dashboard.component.html -->
<div class="tabs" role="tablist" aria-label="Dashboard sections">
<button
*ngFor="let tab of tabs; let i = index"
role="tab"
[id]="'tab-' + tab.id"
[attr.aria-selected]="tab.active"
[attr.aria-controls]="'panel-' + tab.id"
[attr.tabindex]="tab.active ? 0 : -1"
(click)="selectTab(tab)"
(keydown)="handleTabKeydown($event, i)"
class="tab-button"
[class.active]="tab.active">
{{ tab.label }}
</button>
</div>
<div
*ngIf="selectedTab"
role="tabpanel"
[id]="'panel-' + selectedTab.id"
[attr.aria-labelledby]="'tab-' + selectedTab.id"
tabindex="0"
class="tab-content">
{{ selectedTab.content }}
</div>
```
**Supporting TypeScript for keyboard navigation:**
```typescript
// dashboard.component.ts
handleTabKeydown(event: KeyboardEvent, index: number): void {
const tabCount = this.tabs.length;
let newIndex = index;
switch (event.key) {
case 'ArrowRight':
newIndex = (index + 1) % tabCount;
break;
case 'ArrowLeft':
newIndex = (index - 1 + tabCount) % tabCount;
break;
case 'Home':
newIndex = 0;
break;
case 'End':
newIndex = tabCount - 1;
break;
default:
return;
}
event.preventDefault();
this.selectTab(this.tabs[newIndex]);
// Move focus to the new tab button
const tabElement = document.getElementById(`tab-this.tabs[newIndex].id`);
tabElement?.focus();
}
```
## Example 3: Next.js Page-Level Audit
```tsx
// BEFORE: src/app/page.tsx
export default function Home() {
return (
<main>
<div className="text-4xl font-bold">Welcome to Acme</div>
<div className="mt-4">
Build better products with our platform.
</div>
<div className="mt-8 bg-blue-600 text-white px-6 py-3 rounded cursor-pointer"
onClick={() => router.push('/signup')}>
Get Started
</div>
</main>
);
}
```
**Violations detected:**
| # | WCAG | Severity | Issue |
|---|------|----------|-------|
| 1 | 1.3.1 | Major | Heading uses `<div>` instead of `<h1>` -- no semantic structure |
| 2 | 2.4.2 | Major | Page missing `<title>` (Next.js metadata) |
| 3 | 2.1.1 | Critical | CTA uses `<div onClick>` -- not keyboard accessible |
| 4 | 3.1.1 | Minor | `<html>` missing `lang` attribute (check `layout.tsx`) |
```tsx
// AFTER: src/app/page.tsx
import type { Metadata } from 'next';
import Link from 'next/link';
export const metadata: Metadata = {
title: 'Acme - Build Better Products',
description: 'Build better products with the Acme platform.',
};
export default function Home() {
return (
<main>
<h1 className="text-4xl font-bold">Welcome to Acme</h1>
<p className="mt-4">
Build better products with our platform.
</p>
<Link
href="/signup"
className="mt-8 inline-block bg-blue-600 text-white px-6 py-3 rounded
hover:bg-blue-700 focus-visible:outline focus-visible:outline-2
focus-visible:outline-offset-2 focus-visible:outline-blue-600"
>
Get Started
</Link>
</main>
);
}
```
```tsx
// Also fix: src/app/layout.tsx
export default function RootLayout({ children }: { children: React.ReactNode }) {
return (
<html lang="en">
<body>{children}</body>
</html>
);
}
```
## Example 4: Svelte Component Audit
```svelte
<!-- BEFORE: src/lib/components/Accordion.svelte -->
<script>
let openIndex = -1;
function toggle(i) { openIndex = openIndex === i ? -1 : i; }
</script>
{#each items as item, i}
<div class="header" on:click={() => toggle(i)}>
{item.title}
</div>
{#if openIndex === i}
<div class="content">{item.body}</div>
{/if}
{/each}
```
**Violations detected:**
| # | WCAG | Severity | Issue |
|---|------|----------|-------|
| 1 | 4.1.2 | Critical | Accordion missing ARIA roles and properties |
| 2 | 2.1.1 | Critical | Headers not keyboard accessible |
| 3 | 2.5.8 | Minor | Click targets may be smaller than 24x24px (NEW in WCAG 2.2) |
```svelte
<!-- AFTER: src/lib/components/Accordion.svelte -->
<script>
export let items = [];
let openIndex = -1;
function toggle(i) {
openIndex = openIndex === i ? -1 : i;
}
function handleKeydown(event, i) {
if (event.key === 'Enter' || event.key === ' ') {
event.preventDefault();
toggle(i);
}
}
</script>
<div class="accordion">
{#each items as item, i}
<h3>
<button
class="accordion-header"
aria-expanded={openIndex === i}
aria-controls="panel-{i}"
id="header-{i}"
on:click={() => toggle(i)}
on:keydown={(e) => handleKeydown(e, i)}
>
{item.title}
<span class="icon" aria-hidden="true">
{openIndex === i ? '−' : '+'}
</span>
</button>
</h3>
<div
id="panel-{i}"
role="region"
aria-labelledby="header-{i}"
class="accordion-content"
class:open={openIndex === i}
hidden={openIndex !== i}
>
{item.body}
</div>
{/each}
</div>
<style>
.accordion-header {
min-height: 44px; /* WCAG 2.5.8 Target Size */
width: 100%;
padding: 12px 16px;
cursor: pointer;
text-align: left;
}
.accordion-header:focus-visible {
outline: 2px solid #005fcc;
outline-offset: 2px;
}
</style>
```
FILE:references/framework-a11y-patterns.md
# Framework-Specific Accessibility Patterns
## React / Next.js
### Common Issues and Fixes
**Image alt text:**
```jsx
// ❌ Bad
<img src="/hero.jpg" />
<Image src="/hero.jpg" width={800} height={400} />
// ✅ Good
<img src="/hero.jpg" alt="Team collaborating in office" />
<Image src="/hero.jpg" width={800} height={400} alt="Team collaborating in office" />
// ✅ Decorative image
<img src="/divider.svg" alt="" role="presentation" />
```
**Form labels:**
```jsx
// ❌ Bad — placeholder as label
<input placeholder="Email" type="email" />
// ✅ Good — explicit label
<label htmlFor="email">Email</label>
<input id="email" type="email" placeholder="user@example.com" />
// ✅ Good — aria-label for icon-only inputs
<input type="search" aria-label="Search products" />
```
**Click handlers on divs:**
```jsx
// ❌ Bad — not keyboard accessible
<div onClick={handleClick}>Click me</div>
// ✅ Good — use button
<button onClick={handleClick}>Click me</button>
// ✅ If div is required — add keyboard support
<div
role="button"
tabIndex={0}
onClick={handleClick}
onKeyDown={(e) => { if (e.key === 'Enter' || e.key === ' ') handleClick(); }}
>
Click me
</div>
```
**SPA route announcements (Next.js App Router):**
```jsx
// Layout component — announce page changes
'use client';
import { usePathname } from 'next/navigation';
import { useEffect, useState } from 'react';
export function RouteAnnouncer() {
const pathname = usePathname();
const [announcement, setAnnouncement] = useState('');
useEffect(() => {
const title = document.title;
setAnnouncement(`Navigated to title`);
}, [pathname]);
return (
<div aria-live="assertive" role="status" className="sr-only">
{announcement}
</div>
);
}
```
**Focus management after dynamic content:**
```jsx
// After adding item to list, announce it
const [items, setItems] = useState([]);
const statusRef = useRef(null);
const addItem = (item) => {
setItems([...items, item]);
// Announce to screen readers
statusRef.current.textContent = `item.name added to list`;
};
return (
<>
<div ref={statusRef} aria-live="polite" className="sr-only" />
{/* list content */}
</>
);
```
### React-Specific Libraries
- `@radix-ui/*` — accessible primitives (Dialog, Tabs, Select, etc.)
- `@headlessui/react` — unstyled accessible components
- `react-aria` — Adobe's accessibility hooks
- `eslint-plugin-jsx-a11y` — lint rules for JSX accessibility
## Vue 3
### Common Issues and Fixes
**Dynamic content announcements:**
```vue
<template>
<div aria-live="polite" class="sr-only">
{{ announcement }}
</div>
<button @click="search">Search</button>
<ul v-if="results.length">
<li v-for="r in results" :key="r.id">{{ r.name }}</li>
</ul>
</template>
<script setup>
import { ref } from 'vue';
const results = ref([]);
const announcement = ref('');
async function search() {
results.value = await fetchResults();
announcement.value = `results.value.length results found`;
}
</script>
```
**Conditional rendering with focus:**
```vue
<template>
<button @click="showForm = true">Add Item</button>
<form v-if="showForm" ref="formRef">
<label for="name">Name</label>
<input id="name" ref="nameInput" />
</form>
</template>
<script setup>
import { ref, nextTick } from 'vue';
const showForm = ref(false);
const nameInput = ref(null);
watch(showForm, async (val) => {
if (val) {
await nextTick();
nameInput.value?.focus();
}
});
</script>
```
### Vue-Specific Libraries
- `vue-announcer` — route change announcements
- `@headlessui/vue` — accessible components
- `eslint-plugin-vuejs-accessibility` — lint rules
## Angular
### Common Issues and Fixes
**CDK accessibility utilities:**
```typescript
import { LiveAnnouncer } from '@angular/cdk/a11y';
import { FocusTrapFactory } from '@angular/cdk/a11y';
@Component({...})
export class MyComponent {
constructor(
private liveAnnouncer: LiveAnnouncer,
private focusTrapFactory: FocusTrapFactory
) {}
addItem(item: Item) {
this.items.push(item);
this.liveAnnouncer.announce(`item.name added`);
}
openDialog(element: HTMLElement) {
const focusTrap = this.focusTrapFactory.create(element);
focusTrap.focusInitialElement();
}
}
```
**Template-driven forms:**
```html
<!-- ❌ Bad -->
<input [formControl]="email" placeholder="Email" />
<!-- ✅ Good -->
<label for="email">Email address</label>
<input id="email" [formControl]="email"
[attr.aria-invalid]="email.invalid && email.touched"
[attr.aria-describedby]="email.invalid ? 'email-error' : null" />
<div id="email-error" *ngIf="email.invalid && email.touched" role="alert">
Please enter a valid email address.
</div>
```
### Angular-Specific Tools
- `@angular/cdk/a11y` — `FocusTrap`, `LiveAnnouncer`, `FocusMonitor`
- `codelyzer` — a11y lint rules for Angular templates
## Svelte / SvelteKit
### Common Issues and Fixes
```svelte
<!-- ❌ Bad — on:click without keyboard -->
<div on:click={handleClick}>Action</div>
<!-- ✅ Good — Svelte a11y warning built-in -->
<button on:click={handleClick}>Action</button>
<!-- ✅ Accessible toggle -->
<button
on:click={() => isOpen = !isOpen}
aria-expanded={isOpen}
aria-controls="panel"
>
{isOpen ? 'Close' : 'Open'} Details
</button>
{#if isOpen}
<div id="panel" role="region" aria-labelledby="toggle-btn">
Panel content
</div>
{/if}
```
**Note:** Svelte has built-in a11y warnings in the compiler — it flags missing alt text, click-without-keyboard, and other common issues at build time.
## Plain HTML
### Checklist for Static Sites
```html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Descriptive Page Title</title>
</head>
<body>
<!-- Skip link -->
<a href="#main" class="skip-link">Skip to main content</a>
<header>
<nav aria-label="Main navigation">
<ul>
<li><a href="/">Home</a></li>
<li><a href="/about" aria-current="page">About</a></li>
</ul>
</nav>
</header>
<main id="main" tabindex="-1">
<h1>Page Heading</h1>
<!-- Only one h1 per page -->
<!-- Heading levels don't skip (h1 → h2 → h3, never h1 → h3) -->
</main>
<footer>
<p>© 2026 Company Name</p>
</footer>
</body>
</html>
```
## CSS Accessibility Patterns
### Focus Indicators
```css
/* ❌ Bad — removes focus indicator entirely */
:focus { outline: none; }
/* ✅ Good — custom focus indicator */
:focus-visible {
outline: 2px solid #005fcc;
outline-offset: 2px;
}
/* ✅ Good — enhanced for high contrast mode */
@media (forced-colors: active) {
:focus-visible {
outline: 2px solid ButtonText;
}
}
```
### Reduced Motion
```css
/* ✅ Respect prefers-reduced-motion */
@media (prefers-reduced-motion: reduce) {
*, *::before, *::after {
animation-duration: 0.01ms !important;
animation-iteration-count: 1 !important;
transition-duration: 0.01ms !important;
}
}
```
### Screen Reader Only
```css
.sr-only {
position: absolute;
width: 1px;
height: 1px;
padding: 0;
margin: -1px;
overflow: hidden;
clip: rect(0, 0, 0, 0);
white-space: nowrap;
border-width: 0;
}
```
## Fix Patterns Catalog
### React / Next.js Fix Patterns
#### Missing Alt Text (1.1.1)
```tsx
// BEFORE
<img src={hero} />
// AFTER - Informational image
<img src={hero} alt="Team collaborating around a whiteboard" />
// AFTER - Decorative image
<img src={divider} alt="" role="presentation" />
```
#### Non-Interactive Element with Click Handler (2.1.1)
```tsx
// BEFORE
<div onClick={handleClick}>Click me</div>
// AFTER - If it navigates
<Link href="/destination">Click me</Link>
// AFTER - If it performs an action
<button type="button" onClick={handleClick}>Click me</button>
```
#### Missing Focus Management in Modals (2.4.3)
```tsx
// BEFORE
function Modal({ isOpen, onClose, children }) {
if (!isOpen) return null;
return <div className="modal-overlay">{children}</div>;
}
// AFTER
import { useEffect, useRef } from 'react';
function Modal({ isOpen, onClose, children, title }) {
const modalRef = useRef(null);
const previousFocus = useRef(null);
useEffect(() => {
if (isOpen) {
previousFocus.current = document.activeElement;
modalRef.current?.focus();
} else {
previousFocus.current?.focus();
}
}, [isOpen]);
useEffect(() => {
if (!isOpen) return;
const handleKeydown = (e) => {
if (e.key === 'Escape') onClose();
if (e.key === 'Tab') {
const focusable = modalRef.current?.querySelectorAll(
'button, [href], input, select, textarea, [tabindex]:not([tabindex="-1"])'
);
if (!focusable?.length) return;
const first = focusable[0];
const last = focusable[focusable.length - 1];
if (e.shiftKey && document.activeElement === first) {
e.preventDefault();
last.focus();
} else if (!e.shiftKey && document.activeElement === last) {
e.preventDefault();
first.focus();
}
}
};
document.addEventListener('keydown', handleKeydown);
return () => document.removeEventListener('keydown', handleKeydown);
}, [isOpen, onClose]);
if (!isOpen) return null;
return (
<div className="modal-overlay" onClick={onClose} aria-hidden="true">
<div
ref={modalRef}
role="dialog"
aria-modal="true"
aria-label={title}
tabIndex={-1}
onClick={(e) => e.stopPropagation()}
>
<button
onClick={onClose}
aria-label="Close dialog"
className="modal-close"
>
×
</button>
{children}
</div>
</div>
);
}
```
#### Focus Appearance (2.4.11 -- NEW in WCAG 2.2)
```css
/* BEFORE */
button:focus {
outline: none; /* Removes default focus indicator */
}
/* AFTER - Meets WCAG 2.2 Focus Appearance */
button:focus-visible {
outline: 2px solid #005fcc;
outline-offset: 2px;
}
```
```tsx
// Tailwind CSS pattern
<button className="focus-visible:outline focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-600">
Submit
</button>
```
### Vue Fix Patterns
#### Missing Form Labels (1.3.1)
```vue
<!-- BEFORE -->
<input type="text" v-model="name" placeholder="Name" />
<!-- AFTER -->
<label for="user-name">Name</label>
<input id="user-name" type="text" v-model="name" autocomplete="name" />
```
#### Dynamic Content Without Live Region (4.1.3)
```vue
<!-- BEFORE -->
<div v-if="status">{{ statusMessage }}</div>
<!-- AFTER -->
<div aria-live="polite" aria-atomic="true">
<p v-if="status">{{ statusMessage }}</p>
</div>
```
#### Vue Router Navigation Announcements (2.4.2)
```typescript
// router/index.ts
router.afterEach((to) => {
const title = to.meta.title || 'Page';
document.title = `title | My App`;
// Announce route change to screen readers
const announcer = document.getElementById('route-announcer');
if (announcer) {
announcer.textContent = `Navigated to title`;
}
});
```
```vue
<!-- App.vue - Add announcer element -->
<div
id="route-announcer"
role="status"
aria-live="assertive"
aria-atomic="true"
class="sr-only"
></div>
```
### Angular Fix Patterns
#### Missing ARIA on Custom Components (4.1.2)
```typescript
// BEFORE
@Component({
selector: 'app-dropdown',
template: `
<div (click)="toggle()">{{ selected }}</div>
<div *ngIf="isOpen">
<div *ngFor="let opt of options" (click)="select(opt)">{{ opt }}</div>
</div>
`
})
// AFTER
@Component({
selector: 'app-dropdown',
template: `
<button
role="combobox"
[attr.aria-expanded]="isOpen"
aria-haspopup="listbox"
[attr.aria-label]="label"
(click)="toggle()"
(keydown)="handleKeydown($event)"
>
{{ selected }}
</button>
<ul *ngIf="isOpen" role="listbox" [attr.aria-label]="label + ' options'">
<li
*ngFor="let opt of options; let i = index"
role="option"
[attr.aria-selected]="opt === selected"
[attr.id]="'option-' + i"
(click)="select(opt)"
(keydown)="handleOptionKeydown($event, opt, i)"
tabindex="-1"
>
{{ opt }}
</li>
</ul>
`
})
```
#### Angular CDK A11y Module Integration
```typescript
// Use Angular CDK for focus trap in dialogs
import { A11yModule } from '@angular/cdk/a11y';
@Component({
template: `
<div cdkTrapFocus cdkTrapFocusAutoCapture>
<h2 id="dialog-title">Edit Profile</h2>
<!-- dialog content -->
</div>
`
})
```
### Svelte Fix Patterns
#### Accessible Announcements (4.1.3)
```svelte
<!-- BEFORE -->
{#if message}
<p class="toast">{message}</p>
{/if}
<!-- AFTER -->
<div aria-live="polite" class="sr-only">
{#if message}
<p>{message}</p>
{/if}
</div>
<div class="toast" aria-hidden="true">
{#if message}
<p>{message}</p>
{/if}
</div>
```
#### SvelteKit Page Titles (2.4.2)
```svelte
<!-- +page.svelte -->
<svelte:head>
<title>Dashboard | My App</title>
</svelte:head>
```
### Plain HTML Fix Patterns
#### Skip Navigation Link (2.4.1)
```html
<!-- BEFORE -->
<body>
<nav><!-- long navigation --></nav>
<main><!-- content --></main>
</body>
<!-- AFTER -->
<body>
<a href="#main-content" class="skip-link">Skip to main content</a>
<nav aria-label="Main navigation"><!-- long navigation --></nav>
<main id="main-content" tabindex="-1"><!-- content --></main>
</body>
```
```css
.skip-link {
position: absolute;
top: -40px;
left: 0;
padding: 8px 16px;
background: #005fcc;
color: #fff;
z-index: 1000;
transition: top 0.2s;
}
.skip-link:focus {
top: 0;
}
```
#### Accessible Data Table (1.3.1)
```html
<!-- BEFORE -->
<table>
<tr><td>Name</td><td>Email</td><td>Role</td></tr>
<tr><td>Alice</td><td>alice@co.com</td><td>Admin</td></tr>
</table>
<!-- AFTER -->
<table aria-label="Team members">
<caption class="sr-only">List of team members and their roles</caption>
<thead>
<tr>
<th scope="col">Name</th>
<th scope="col">Email</th>
<th scope="col">Role</th>
</tr>
</thead>
<tbody>
<tr>
<th scope="row">Alice</th>
<td>alice@co.com</td>
<td>Admin</td>
</tr>
</tbody>
</table>
```
FILE:references/testing-checklist.md
# Accessibility Testing Checklist
Use this checklist after applying fixes to verify accessibility manually.
## Keyboard Navigation
- [ ] All interactive elements reachable via Tab key
- [ ] Tab order follows visual/logical reading order
- [ ] Focus indicator visible on every focusable element (2px+ outline)
- [ ] Modals trap focus and return focus on close
- [ ] Escape key closes modals, dropdowns, and popups
- [ ] Arrow keys navigate within composite widgets (tabs, menus, listboxes)
- [ ] No keyboard traps (user can always Tab away)
## Screen Reader
- [ ] All images have appropriate alt text (or `alt=""` for decorative)
- [ ] Headings create logical document outline (h1 -> h2 -> h3)
- [ ] Form inputs have associated labels
- [ ] Error messages announced via `aria-live` or `role="alert"`
- [ ] Page title updates on navigation (SPA)
- [ ] Dynamic content changes announced appropriately
## Visual
- [ ] Text contrast meets 4.5:1 for normal text, 3:1 for large text
- [ ] UI component contrast meets 3:1 against background
- [ ] Content reflows without horizontal scrolling at 320px width
- [ ] Text resizable to 200% without loss of content
- [ ] No information conveyed by color alone
- [ ] Focus indicators meet 2.4.11 Focus Appearance criteria
## Motion and Media
- [ ] Animations respect `prefers-reduced-motion`
- [ ] No auto-playing media with audio
- [ ] No content flashing more than 3 times per second
- [ ] Video has captions; audio has transcripts
## Forms
- [ ] All inputs have visible labels
- [ ] Required fields indicated (not by color alone)
- [ ] Error messages specific and associated with input via `aria-describedby`
- [ ] Autocomplete attributes present on common fields (name, email, etc.)
- [ ] No CAPTCHA without alternative method (WCAG 2.2 3.3.8)
FILE:references/wcag-22-new-criteria.md
# WCAG 2.2 New Success Criteria Reference
These criteria were added in WCAG 2.2 and are commonly missed.
## 2.4.11 Focus Appearance (Level AA)
The focus indicator must have a minimum area of a 2px perimeter around the component and a contrast ratio of at least 3:1 against adjacent colors.
**Pattern:**
```css
:focus-visible {
outline: 2px solid #005fcc;
outline-offset: 2px;
}
```
## 2.5.7 Dragging Movements (Level AA)
Any functionality that uses dragging must have a single-pointer alternative (click, tap).
**Pattern:**
```tsx
// Sortable list: support both drag and button-based reorder
<li draggable onDragStart={handleDrag}>
{item.name}
<button onClick={() => moveUp(index)} aria-label={`Move item.name up`}>
Move Up
</button>
<button onClick={() => moveDown(index)} aria-label={`Move item.name down`}>
Move Down
</button>
</li>
```
## 2.5.8 Target Size (Level AA)
Interactive targets must be at least 24x24 CSS pixels, with exceptions for inline text links and elements where the spacing provides equivalent clearance.
**Pattern:**
```css
button, a, input, select, textarea {
min-height: 24px;
min-width: 24px;
}
/* Recommended: 44x44px for touch targets */
@media (pointer: coarse) {
button, a, input[type="checkbox"], input[type="radio"] {
min-height: 44px;
min-width: 44px;
}
}
```
## 3.3.7 Redundant Entry (Level A)
Information previously entered by the user must be auto-populated or available for selection when needed again in the same process.
**Pattern:**
```tsx
// Multi-step form: persist data across steps
const [formData, setFormData] = useState({});
// Step 2 pre-fills shipping address from billing
<input
defaultValue={formData.billingAddress || ''}
autoComplete="shipping street-address"
/>
```
## 3.3.8 Accessible Authentication (Level AA)
Authentication must not require cognitive function tests (e.g., remembering a password, solving a puzzle) unless an alternative is provided.
**Pattern:**
- Support password managers (`autocomplete="current-password"`)
- Offer passkey / biometric authentication
- Allow copy-paste in password fields (never block paste)
- Provide email/SMS OTP as alternative to CAPTCHA
FILE:references/wcag-quick-ref.md
# WCAG 2.2 Quick Reference — Level A & AA
## Perceivable
### 1.1 Text Alternatives
| Criterion | Level | Requirement | Common Violation |
|-----------|-------|-------------|------------------|
| 1.1.1 Non-text Content | A | All images have `alt` text; decorative images use `alt=""` or `role="presentation"` | `<img src="logo.png">` without alt |
### 1.2 Time-Based Media
| Criterion | Level | Requirement | Common Violation |
|-----------|-------|-------------|------------------|
| 1.2.1 Audio-only / Video-only | A | Provide transcript or audio description | Video without captions |
| 1.2.2 Captions | A | Captions for all prerecorded audio in video | Missing `<track kind="captions">` |
| 1.2.3 Audio Description | A | Audio description for prerecorded video | No descriptive track |
| 1.2.5 Audio Description (Prerecorded) | AA | Audio description for all prerecorded video | Same as 1.2.3 but stricter |
### 1.3 Adaptable
| Criterion | Level | Requirement | Common Violation |
|-----------|-------|-------------|------------------|
| 1.3.1 Info and Relationships | A | Semantic markup conveys structure | Using `<div>` instead of `<nav>`, `<main>`, `<header>` |
| 1.3.2 Meaningful Sequence | A | Reading order matches visual order | CSS flex/grid reordering without DOM reorder |
| 1.3.3 Sensory Characteristics | A | Don't rely solely on color, shape, position | "Click the red button" |
| 1.3.4 Orientation | AA | Content not restricted to portrait/landscape | CSS `orientation: portrait` lock |
| 1.3.5 Identify Input Purpose | AA | Input purpose identifiable via `autocomplete` | Missing `autocomplete="email"` on email inputs |
### 1.4 Distinguishable
| Criterion | Level | Requirement | Ratio |
|-----------|-------|-------------|-------|
| 1.4.1 Use of Color | A | Color not sole means of conveying info | Red-only error indicators |
| 1.4.2 Audio Control | A | Auto-playing audio has pause/stop | `autoplay` without `controls` |
| 1.4.3 Contrast (Minimum) | AA | Text: 4.5:1, Large text: 3:1 | Light gray text on white |
| 1.4.4 Resize Text | AA | Text resizable to 200% without loss | Fixed `px` font sizes |
| 1.4.5 Images of Text | AA | Use real text, not text in images | Logo text as PNG |
| 1.4.10 Reflow | AA | Content reflows at 320px width | Horizontal scrolling at mobile widths |
| 1.4.11 Non-text Contrast | AA | UI components and graphics: 3:1 | Low-contrast borders, icons |
| 1.4.12 Text Spacing | AA | No loss of content when spacing adjusted | Fixed-height containers clipping |
| 1.4.13 Content on Hover/Focus | AA | Dismissible, hoverable, persistent | Tooltips that disappear on mouse move |
## Operable
### 2.1 Keyboard Accessible
| Criterion | Level | Requirement | Common Violation |
|-----------|-------|-------------|------------------|
| 2.1.1 Keyboard | A | All functionality via keyboard | `onClick` without `onKeyDown` |
| 2.1.2 No Keyboard Trap | A | Focus can move away from any component | Modal without focus trap escape |
| 2.1.4 Character Key Shortcuts | A | Single-key shortcuts can be turned off | `accesskey` conflicts |
### 2.4 Navigable
| Criterion | Level | Requirement | Common Violation |
|-----------|-------|-------------|------------------|
| 2.4.1 Bypass Blocks | A | Skip navigation link | No "Skip to content" link |
| 2.4.2 Page Titled | A | Descriptive `<title>` | `<title>Untitled</title>` |
| 2.4.3 Focus Order | A | Logical tab order | `tabindex` > 0 |
| 2.4.4 Link Purpose | A | Link text describes destination | "Click here", "Read more" |
| 2.4.6 Headings and Labels | AA | Descriptive headings | Generic headings |
| 2.4.7 Focus Visible | AA | Visible focus indicator | `outline: none` without replacement |
| 2.4.11 Focus Not Obscured | AA | Focused element not hidden by sticky header | Fixed header covering focused element |
### 2.5 Input Modalities
| Criterion | Level | Requirement | Common Violation |
|-----------|-------|-------------|------------------|
| 2.5.1 Pointer Gestures | A | Multi-point gestures have single-point alternative | Pinch-to-zoom only |
| 2.5.2 Pointer Cancellation | A | Down-event doesn't trigger action | `mousedown` instead of `click` |
| 2.5.3 Label in Name | A | Visible label is in accessible name | Button shows "Submit" but `aria-label="btn1"` |
| 2.5.4 Motion Actuation | A | Motion-triggered actions have alternative | Shake-to-undo only |
| 2.5.7 Dragging Movements | AA | Drag has single-pointer alternative | Drag-and-drop only reordering |
| 2.5.8 Target Size | AA | Touch targets minimum 24x24 CSS pixels | Tiny mobile buttons |
## Understandable
### 3.1 Readable
| Criterion | Level | Requirement | Common Violation |
|-----------|-------|-------------|------------------|
| 3.1.1 Language of Page | A | `<html lang="en">` | Missing `lang` attribute |
| 3.1.2 Language of Parts | AA | `lang` on foreign-language spans | Mixed-language content without `lang` |
### 3.2 Predictable
| Criterion | Level | Requirement | Common Violation |
|-----------|-------|-------------|------------------|
| 3.2.1 On Focus | A | Focus doesn't trigger unexpected change | Auto-submitting on focus |
| 3.2.2 On Input | A | Input doesn't trigger unexpected change | Auto-navigating on select change |
| 3.2.3 Consistent Navigation | AA | Navigation consistent across pages | Menu order changes per page |
| 3.2.4 Consistent Identification | AA | Same function = same label | "Search" vs "Find" for same action |
### 3.3 Input Assistance
| Criterion | Level | Requirement | Common Violation |
|-----------|-------|-------------|------------------|
| 3.3.1 Error Identification | A | Errors described in text | Red border only, no message |
| 3.3.2 Labels or Instructions | A | Labels for required input | Placeholder as only label |
| 3.3.3 Error Suggestion | AA | Suggest corrections | "Invalid input" without guidance |
| 3.3.4 Error Prevention | AA | Reversible submissions for legal/financial | No confirmation for payment |
| 3.3.7 Redundant Entry | A | Don't ask for same info twice | Re-entering address in checkout |
| 3.3.8 Accessible Authentication | AA | No cognitive function test for login | CAPTCHA without audio alternative |
## Robust
### 4.1 Compatible
| Criterion | Level | Requirement | Common Violation |
|-----------|-------|-------------|------------------|
| 4.1.2 Name, Role, Value | A | Custom controls have accessible name and role | Custom dropdown without ARIA |
| 4.1.3 Status Messages | AA | Status updates announced without focus change | Toast without `aria-live` |
FILE:scripts/a11y_scanner.py
#!/usr/bin/env python3
"""WCAG 2.2 Accessibility Scanner for Frontend Codebases.
Scans HTML, JSX, TSX, Vue, Svelte, and CSS files for accessibility
violations across 10 categories: images, forms, headings, landmarks,
keyboard, ARIA, color/contrast, links, tables, and media.
Usage:
python a11y_scanner.py /path/to/project
python a11y_scanner.py /path/to/project --json
python a11y_scanner.py /path/to/project --severity critical,serious
python a11y_scanner.py /path/to/project --format json
"""
import argparse
import json
import os
import re
import sys
from dataclasses import dataclass, asdict
from typing import List, Optional
@dataclass
class Finding:
"""A single accessibility finding."""
rule_id: str
category: str
severity: str
message: str
file: str
line: int
snippet: str
wcag_criterion: str
fix: str
# ---------------------------------------------------------------------------
# Rule definitions: each returns a list of Finding from a single file
# ---------------------------------------------------------------------------
VALID_ARIA_ATTRS = {
"aria-activedescendant", "aria-atomic", "aria-autocomplete", "aria-busy",
"aria-checked", "aria-colcount", "aria-colindex", "aria-colspan",
"aria-controls", "aria-current", "aria-describedby", "aria-details",
"aria-disabled", "aria-dropeffect", "aria-errormessage", "aria-expanded",
"aria-flowto", "aria-grabbed", "aria-haspopup", "aria-hidden",
"aria-invalid", "aria-keyshortcuts", "aria-label", "aria-labelledby",
"aria-level", "aria-live", "aria-modal", "aria-multiline",
"aria-multiselectable", "aria-orientation", "aria-owns", "aria-placeholder",
"aria-posinset", "aria-pressed", "aria-readonly", "aria-relevant",
"aria-required", "aria-roledescription", "aria-rowcount", "aria-rowindex",
"aria-rowspan", "aria-selected", "aria-setsize", "aria-sort",
"aria-valuemax", "aria-valuemin", "aria-valuenow", "aria-valuetext",
"aria-braillelabel", "aria-brailleroledescription", "aria-description",
}
BAD_LINK_TEXT = re.compile(
r">\s*(click here|here|read more|more|link|this)\s*<", re.IGNORECASE
)
TAG_RE = re.compile(r"<(\w[\w-]*)\b([^>]*)(/?)>", re.DOTALL)
ATTR_RE = re.compile(r"""([\w:.-]+)\s*=\s*(?:"([^"]*)"|'([^']*)'|(\S+))""")
ATTR_BOOL_RE = re.compile(r"\b([\w:.-]+)(?=\s|/?>|$)")
INLINE_COLOR_RE = re.compile(
r'style\s*=\s*["\'][^"\']*\bcolor\s*:', re.IGNORECASE
)
ARIA_ATTR_RE = re.compile(r"\baria-[\w-]+")
def _attrs(attr_str: str) -> dict:
"""Parse HTML/JSX attribute string into a dict."""
result = {}
for m in ATTR_RE.finditer(attr_str):
result[m.group(1)] = m.group(2) or m.group(3) or m.group(4) or ""
# boolean attrs
cleaned = ATTR_RE.sub("", attr_str)
for m in ATTR_BOOL_RE.finditer(cleaned):
name = m.group(1)
if name not in result and not name.startswith("/"):
result[name] = True
return result
def _snippet(line_text: str) -> str:
"""Trim a line for display as a code snippet."""
s = line_text.rstrip("\n\r")
return s[:120] + "..." if len(s) > 120 else s
def _find(rule_id, cat, sev, msg, fp, ln, snip, wcag, fix):
return Finding(rule_id, cat, sev, msg, fp, ln, snip, wcag, fix)
# ---------- Images ----------------------------------------------------------
def check_img_missing_alt(tag, attrs, fp, ln, snip):
if tag == "img" and "alt" not in attrs:
return _find("img-alt-missing", "images", "critical",
"<img> missing alt attribute",
fp, ln, snip, "1.1.1 Non-text Content",
"Add alt=\"description\" or alt=\"\" for decorative images.")
def check_img_empty_alt_informative(tag, attrs, fp, ln, snip):
if tag == "img" and attrs.get("alt") == "" and attrs.get("src", ""):
src = attrs.get("src", "")
if not any(kw in src.lower() for kw in ("spacer", "border", "decorat", "bg")):
return _find("img-alt-empty-informative", "images", "serious",
"<img> has empty alt but may be informative",
fp, ln, snip, "1.1.1 Non-text Content",
"If image conveys information, add descriptive alt text.")
def check_img_decorative_has_alt(tag, attrs, fp, ln, snip):
if tag == "img" and attrs.get("role") == "presentation" and attrs.get("alt", "") != "":
return _find("img-decorative-alt", "images", "moderate",
"Decorative image (role=presentation) should have alt=\"\"",
fp, ln, snip, "1.1.1 Non-text Content",
"Set alt=\"\" on decorative images with role=presentation.")
# ---------- Forms -----------------------------------------------------------
def check_input_missing_label(tag, attrs, fp, ln, snip):
input_types = {"text", "email", "password", "search", "tel", "url", "number", "date"}
if tag == "input" and attrs.get("type", "text") in input_types:
if "aria-label" not in attrs and "aria-labelledby" not in attrs and "id" not in attrs:
return _find("form-input-no-label", "forms", "critical",
"<input> has no id, aria-label, or aria-labelledby",
fp, ln, snip, "1.3.1 Info and Relationships",
"Add id + <label for>, or aria-label attribute.")
def check_input_no_aria_label(tag, attrs, fp, ln, snip):
if tag in ("select", "textarea"):
if "aria-label" not in attrs and "aria-labelledby" not in attrs and "id" not in attrs:
return _find("form-select-no-label", "forms", "critical",
f"<{tag}> has no accessible name",
fp, ln, snip, "4.1.2 Name, Role, Value",
f"Add aria-label or id + <label for> to <{tag}>.")
def check_orphan_label(lines, fp):
"""Labels whose 'for' points to a non-existent id."""
findings = []
ids = set()
label_fors = []
for ln, line in enumerate(lines, 1):
for m in re.finditer(r'\bid\s*=\s*["\']([^"\']+)["\']', line):
ids.add(m.group(1))
for m in re.finditer(r'<label[^>]*\bfor\s*=\s*["\']([^"\']+)["\']', line):
label_fors.append((ln, m.group(1), line))
for ln, for_val, line in label_fors:
if for_val not in ids:
findings.append(_find("form-orphan-label", "forms", "serious",
f"<label for=\"{for_val}\"> references non-existent id",
fp, ln, _snippet(line), "1.3.1 Info and Relationships",
f"Ensure an element with id=\"{for_val}\" exists."))
return findings
def check_fieldset_legend(lines, fp):
"""Radio/checkbox groups without fieldset."""
findings = []
radio_lines = []
has_fieldset = any("fieldset" in l.lower() for l in lines)
for ln, line in enumerate(lines, 1):
if re.search(r'type\s*=\s*["\'](?:radio|checkbox)["\']', line, re.I):
radio_lines.append((ln, line))
if radio_lines and not has_fieldset:
ln, line = radio_lines[0]
findings.append(_find("form-missing-fieldset", "forms", "serious",
"Radio/checkbox group without <fieldset>/<legend>",
fp, ln, _snippet(line), "1.3.1 Info and Relationships",
"Wrap related radio/checkbox inputs in <fieldset> with <legend>."))
return findings
# ---------- Headings --------------------------------------------------------
def check_headings(lines, fp):
findings = []
heading_levels = []
for ln, line in enumerate(lines, 1):
for m in re.finditer(r"<[hH]([1-6])\b", line):
heading_levels.append((int(m.group(1)), ln, line))
if not heading_levels:
return findings
# Missing h1
levels_seen = {h[0] for h in heading_levels}
if 1 not in levels_seen and any(l <= 3 for l in levels_seen):
findings.append(_find("heading-missing-h1", "headings", "serious",
"Page has headings but no <h1>",
fp, heading_levels[0][1], _snippet(heading_levels[0][2]),
"1.3.1 Info and Relationships",
"Add a single <h1> as the main page heading."))
# Multiple h1s
h1_lines = [(ln, line) for lvl, ln, line in heading_levels if lvl == 1]
if len(h1_lines) > 1:
findings.append(_find("heading-multiple-h1", "headings", "moderate",
f"Page has {len(h1_lines)} <h1> elements",
fp, h1_lines[1][0], _snippet(h1_lines[1][1]),
"1.3.1 Info and Relationships",
"Use a single <h1> per page. Demote others to <h2>+."))
# Skipped levels
prev_level = 0
for lvl, ln, line in heading_levels:
if prev_level > 0 and lvl > prev_level + 1:
findings.append(_find("heading-skipped", "headings", "moderate",
f"Heading level skips from h{prev_level} to h{lvl}",
fp, ln, _snippet(line),
"1.3.1 Info and Relationships",
f"Use <h{prev_level + 1}> instead of <h{lvl}>."))
prev_level = lvl
return findings
# ---------- Landmarks -------------------------------------------------------
def check_landmarks(lines, fp):
findings = []
content = "\n".join(lines)
# Missing main landmark
if not re.search(r'<main\b|role\s*=\s*["\']main["\']', content, re.I):
findings.append(_find("landmark-no-main", "landmarks", "serious",
"Page missing <main> landmark",
fp, 1, "", "1.3.1 Info and Relationships",
"Add a <main> element to wrap primary content."))
# Missing nav
if not re.search(r'<nav\b|role\s*=\s*["\']navigation["\']', content, re.I):
findings.append(_find("landmark-no-nav", "landmarks", "moderate",
"Page missing <nav> landmark",
fp, 1, "", "1.3.1 Info and Relationships",
"Add <nav> for primary navigation blocks."))
# Missing skip link
if not re.search(r'skip.{0,10}(nav|main|content)', content, re.I):
findings.append(_find("landmark-no-skip-link", "landmarks", "serious",
"Page missing skip navigation link",
fp, 1, "", "2.4.1 Bypass Blocks",
"Add <a href=\"#main\">Skip to main content</a> as first focusable element."))
return findings
# ---------- Keyboard --------------------------------------------------------
def check_tabindex_positive(tag, attrs, fp, ln, snip):
ti = attrs.get("tabindex", "")
if isinstance(ti, str) and ti.lstrip("-").isdigit() and int(ti) > 0:
return _find("keyboard-tabindex-positive", "keyboard", "serious",
f"tabindex={ti} creates unexpected tab order",
fp, ln, snip, "2.4.3 Focus Order",
"Use tabindex=\"0\" or tabindex=\"-1\" instead of positive values.")
def check_click_no_keyboard(tag, attrs, fp, ln, snip):
has_click = "onClick" in attrs or "onclick" in attrs or "@click" in attrs or "on:click" in attrs
has_key = any(k for k in attrs if "keydown" in k.lower() or "keyup" in k.lower() or "keypress" in k.lower())
if tag in ("div", "span", "td", "li", "p", "section") and has_click and not has_key:
if attrs.get("role") not in ("button", "link", "tab", "menuitem"):
return _find("keyboard-click-no-key", "keyboard", "critical",
f"<{tag}> has click handler but no keyboard handler",
fp, ln, snip, "2.1.1 Keyboard",
f"Add onKeyDown handler or use <button> instead of <{tag}>.")
def check_autofocus_misuse(tag, attrs, fp, ln, snip):
if "autofocus" in attrs or "autoFocus" in attrs:
if tag not in ("input", "textarea", "select"):
return _find("keyboard-autofocus", "keyboard", "moderate",
f"autofocus on <{tag}> can disorient screen reader users",
fp, ln, snip, "3.2.1 On Focus",
"Avoid autofocus on non-input elements. Use focus management instead.")
# ---------- ARIA ------------------------------------------------------------
def check_invalid_aria(tag, attrs, fp, ln, snip):
findings = []
for key in attrs:
if key.startswith("aria-") and key.lower() not in VALID_ARIA_ATTRS:
findings.append(_find("aria-invalid-attr", "aria", "serious",
f"Invalid ARIA attribute: {key}",
fp, ln, snip, "4.1.2 Name, Role, Value",
f"Remove or replace \"{key}\" with a valid ARIA attribute."))
return findings
def check_aria_hidden_focusable(tag, attrs, fp, ln, snip):
if attrs.get("aria-hidden") in ("true", True):
focusable_tags = {"a", "button", "input", "select", "textarea"}
if tag in focusable_tags or (isinstance(attrs.get("tabindex", ""), str) and
attrs.get("tabindex", "-1") != "-1"):
return _find("aria-hidden-focusable", "aria", "critical",
f"aria-hidden=\"true\" on focusable <{tag}>",
fp, ln, snip, "4.1.2 Name, Role, Value",
"Remove aria-hidden or make element non-focusable (tabindex=\"-1\").")
def check_aria_live_missing(lines, fp):
"""Alert/status roles or live regions without aria-live."""
findings = []
for ln, line in enumerate(lines, 1):
if re.search(r'role\s*=\s*["\'](?:alert|status)["\']', line, re.I):
if "aria-live" not in line:
findings.append(_find("aria-live-missing", "aria", "serious",
"role=alert/status without explicit aria-live",
fp, ln, _snippet(line),
"4.1.3 Status Messages",
"Add aria-live=\"assertive\" (alert) or aria-live=\"polite\" (status)."))
return findings
# ---------- Color/Contrast --------------------------------------------------
def check_inline_color(tag, attrs, fp, ln, snip):
style = attrs.get("style", "")
if isinstance(style, str) and re.search(r"\bcolor\s*:", style, re.I):
if not re.search(r"background", style, re.I):
return _find("color-inline-no-bg", "color", "moderate",
"Inline color set without background — contrast may be insufficient",
fp, ln, snip, "1.4.3 Contrast (Minimum)",
"Ensure foreground and background colors meet 4.5:1 contrast ratio.")
def check_text_over_image(lines, fp):
"""Detects patterns where text is positioned over background images without overlay."""
findings = []
for ln, line in enumerate(lines, 1):
if re.search(r"background-image\s*:", line, re.I):
if not re.search(r"(overlay|rgba|linear-gradient)", line, re.I):
findings.append(_find("color-text-over-image", "color", "serious",
"Background image without contrast overlay for text",
fp, ln, _snippet(line),
"1.4.3 Contrast (Minimum)",
"Add a semi-transparent overlay or ensure text contrast."))
return findings
# ---------- Links -----------------------------------------------------------
def check_empty_link(tag, attrs, fp, ln, snip):
if tag == "a" and not attrs.get("aria-label") and not attrs.get("aria-labelledby"):
return None # handled by line-level check below
def check_empty_links_line(lines, fp):
findings = []
for ln, line in enumerate(lines, 1):
# <a ...></a> or <a ...> </a>
if re.search(r"<a\b[^>]*>\s*</a>", line, re.I):
if "aria-label" not in line and "aria-labelledby" not in line:
findings.append(_find("link-empty", "links", "critical",
"Empty link — no text or accessible name",
fp, ln, _snippet(line), "2.4.4 Link Purpose",
"Add link text or aria-label."))
# Bad link text
if BAD_LINK_TEXT.search(line):
findings.append(_find("link-bad-text", "links", "serious",
"Link uses vague text like 'click here'",
fp, ln, _snippet(line), "2.4.4 Link Purpose",
"Use descriptive link text that makes sense out of context."))
return findings
def check_same_page_link(tag, attrs, fp, ln, snip):
href = attrs.get("href", "")
if tag == "a" and isinstance(href, str) and href == "#":
return _find("link-empty-fragment", "links", "moderate",
"Link with href=\"#\" — use a button or valid fragment",
fp, ln, snip, "2.4.4 Link Purpose",
"Use <button> for actions or href=\"#section-id\" for anchors.")
# ---------- Tables ----------------------------------------------------------
def check_table_headers(lines, fp):
findings = []
in_table = False
table_start = 0
has_th = False
has_caption = False
has_aria_label = False
for ln, line in enumerate(lines, 1):
if re.search(r"<table\b", line, re.I):
in_table = True
table_start = ln
has_th = False
has_caption = False
has_aria_label = "aria-label" in line
if in_table:
if "<th" in line.lower():
has_th = True
if "<caption" in line.lower():
has_caption = True
if re.search(r"</table>", line, re.I):
if not has_th:
findings.append(_find("table-no-headers", "tables", "serious",
"<table> has no <th> header cells",
fp, table_start, _snippet(lines[table_start - 1]),
"1.3.1 Info and Relationships",
"Add <th> elements to identify column/row headers."))
if not has_caption and not has_aria_label:
findings.append(_find("table-no-caption", "tables", "moderate",
"<table> missing <caption> or aria-label",
fp, table_start, _snippet(lines[table_start - 1]),
"1.3.1 Info and Relationships",
"Add <caption> or aria-label to describe the table."))
in_table = False
return findings
# ---------- Media -----------------------------------------------------------
def check_media_captions(tag, attrs, fp, ln, snip):
if tag == "video":
return None # handled at block level
def check_media_captions_block(lines, fp):
findings = []
in_video = False
video_start = 0
has_track = False
has_controls = False
has_autoplay = False
for ln, line in enumerate(lines, 1):
if re.search(r"<video\b", line, re.I):
in_video = True
video_start = ln
has_track = False
has_controls = "controls" in line.lower()
has_autoplay = "autoplay" in line.lower()
if in_video:
if re.search(r'<track\b[^>]*kind\s*=\s*["\']captions["\']', line, re.I):
has_track = True
if "controls" in line.lower():
has_controls = True
if re.search(r"</video>", line, re.I) or (re.search(r"<video\b", line, re.I) and "/>" in line):
if not has_track:
findings.append(_find("media-no-captions", "media", "critical",
"<video> missing captions track",
fp, video_start, _snippet(lines[video_start - 1]),
"1.2.2 Captions (Prerecorded)",
"Add <track kind=\"captions\" src=\"...\" srclang=\"en\">."))
if has_autoplay and not has_controls:
findings.append(_find("media-autoplay-no-controls", "media", "serious",
"<video> has autoplay without controls",
fp, video_start, _snippet(lines[video_start - 1]),
"1.4.2 Audio Control",
"Add the controls attribute so users can pause/stop."))
in_video = False
# Single-line video tags
for ln, line in enumerate(lines, 1):
if re.search(r"<audio\b", line, re.I):
if "autoplay" in line.lower() and "controls" not in line.lower():
findings.append(_find("media-audio-autoplay", "media", "serious",
"<audio> has autoplay without controls",
fp, ln, _snippet(line), "1.4.2 Audio Control",
"Add the controls attribute to <audio>."))
return findings
# ---------------------------------------------------------------------------
# Scanner engine
# ---------------------------------------------------------------------------
SUPPORTED_EXTENSIONS = {".html", ".htm", ".jsx", ".tsx", ".vue", ".svelte", ".css"}
TAG_LEVEL_CHECKS = [
check_img_missing_alt,
check_img_empty_alt_informative,
check_img_decorative_has_alt,
check_input_missing_label,
check_input_no_aria_label,
check_tabindex_positive,
check_click_no_keyboard,
check_autofocus_misuse,
check_aria_hidden_focusable,
check_inline_color,
check_same_page_link,
]
TAG_LEVEL_MULTI_CHECKS = [
check_invalid_aria,
]
def scan_file(filepath: str) -> List[Finding]:
"""Scan a single file and return all findings."""
findings: List[Finding] = []
try:
with open(filepath, "r", encoding="utf-8", errors="replace") as f:
lines = f.readlines()
except (OSError, IOError):
return findings
# Tag-level checks
for ln, line in enumerate(lines, 1):
for m in TAG_RE.finditer(line):
tag = m.group(1).lower()
attr_str = m.group(2)
attrs = _attrs(attr_str)
snip = _snippet(line)
for check in TAG_LEVEL_CHECKS:
result = check(tag, attrs, filepath, ln, snip)
if result:
findings.append(result)
for check in TAG_LEVEL_MULTI_CHECKS:
results = check(tag, attrs, filepath, ln, snip)
if results:
findings.extend(results)
# File-level / multi-line checks
findings.extend(check_orphan_label(lines, filepath))
findings.extend(check_fieldset_legend(lines, filepath))
findings.extend(check_headings(lines, filepath))
findings.extend(check_landmarks(lines, filepath))
findings.extend(check_aria_live_missing(lines, filepath))
findings.extend(check_text_over_image(lines, filepath))
findings.extend(check_empty_links_line(lines, filepath))
findings.extend(check_table_headers(lines, filepath))
findings.extend(check_media_captions_block(lines, filepath))
return findings
def collect_files(path: str) -> List[str]:
"""Recursively collect scannable files under path."""
files = []
if os.path.isfile(path):
_, ext = os.path.splitext(path)
if ext.lower() in SUPPORTED_EXTENSIONS:
files.append(path)
return files
for root, dirs, filenames in os.walk(path):
# Skip common non-source directories
dirs[:] = [d for d in dirs if d not in (
"node_modules", ".git", "dist", "build", "__pycache__",
".next", ".nuxt", "vendor", "coverage"
)]
for fname in filenames:
_, ext = os.path.splitext(fname)
if ext.lower() in SUPPORTED_EXTENSIONS:
files.append(os.path.join(root, fname))
files.sort()
return files
# ---------------------------------------------------------------------------
# Output formatting
# ---------------------------------------------------------------------------
SEVERITY_ORDER = {"critical": 0, "serious": 1, "moderate": 2, "minor": 3}
def format_human(findings: List[Finding], files_scanned: int) -> str:
"""Format findings as human-readable text report."""
if not findings:
return (f"Scanned {files_scanned} file(s) -- no accessibility issues found.\n"
"All checks passed.")
lines = []
lines.append(f"WCAG 2.2 Accessibility Scan Results")
lines.append(f"{'=' * 50}")
lines.append(f"Files scanned: {files_scanned}")
lines.append(f"Issues found: {len(findings)}")
# Summary by severity
severity_counts = {}
for f in findings:
severity_counts[f.severity] = severity_counts.get(f.severity, 0) + 1
for sev in ("critical", "serious", "moderate", "minor"):
if sev in severity_counts:
lines.append(f" {sev.upper():10s}: {severity_counts[sev]}")
lines.append("")
# Summary by category
cat_counts = {}
for f in findings:
cat_counts[f.category] = cat_counts.get(f.category, 0) + 1
lines.append("By category:")
for cat in sorted(cat_counts, key=lambda c: -cat_counts[c]):
lines.append(f" {cat:20s}: {cat_counts[cat]}")
lines.append("")
# Detailed findings sorted by severity then file
sorted_findings = sorted(findings, key=lambda f: (SEVERITY_ORDER.get(f.severity, 9), f.file, f.line))
for i, f in enumerate(sorted_findings, 1):
lines.append(f"[{f.severity.upper()}] {f.rule_id}")
lines.append(f" File: {f.file}:{f.line}")
lines.append(f" WCAG: {f.wcag_criterion}")
lines.append(f" Issue: {f.message}")
if f.snippet:
lines.append(f" Code: {f.snippet}")
lines.append(f" Fix: {f.fix}")
lines.append("")
return "\n".join(lines)
def format_json(findings: List[Finding], files_scanned: int) -> str:
"""Format findings as JSON."""
severity_counts = {}
for f in findings:
severity_counts[f.severity] = severity_counts.get(f.severity, 0) + 1
report = {
"summary": {
"files_scanned": files_scanned,
"total_issues": len(findings),
"by_severity": severity_counts,
},
"findings": [asdict(f) for f in findings],
}
return json.dumps(report, indent=2)
# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="a11y_scanner",
description="Scan frontend codebases for WCAG 2.2 accessibility violations.",
epilog=(
"Supported file types: .html, .htm, .jsx, .tsx, .vue, .svelte, .css\n"
"Exit codes: 0 = pass, 1 = critical/serious found, 2 = moderate/minor only"
),
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument(
"path",
help="File or directory to scan",
)
parser.add_argument(
"--json", dest="json_flag", action="store_true",
help="Output results as JSON (shorthand for --format json)",
)
parser.add_argument(
"--format", dest="output_format", choices=["text", "json"],
default="text",
help="Output format: text (default) or json",
)
parser.add_argument(
"--severity", dest="severity",
default=None,
help="Comma-separated severity filter (e.g. critical,serious)",
)
return parser
def main():
parser = build_parser()
args = parser.parse_args()
path = os.path.abspath(args.path)
if not os.path.exists(path):
print(f"Error: path does not exist: {path}", file=sys.stderr)
sys.exit(1)
use_json = args.json_flag or args.output_format == "json"
# Collect and scan files
files = collect_files(path)
if not files:
print(f"No scannable files found in: {path}", file=sys.stderr)
sys.exit(0)
all_findings: List[Finding] = []
for fpath in files:
all_findings.extend(scan_file(fpath))
# Filter by severity if requested
if args.severity:
allowed = {s.strip().lower() for s in args.severity.split(",")}
all_findings = [f for f in all_findings if f.severity in allowed]
# Output
if use_json:
print(format_json(all_findings, len(files)))
else:
print(format_human(all_findings, len(files)))
# Exit code
severities = {f.severity for f in all_findings}
if severities & {"critical", "serious"}:
sys.exit(1)
elif severities & {"moderate", "minor"}:
sys.exit(2)
else:
sys.exit(0)
if __name__ == "__main__":
main()
FILE:scripts/contrast_checker.py
#!/usr/bin/env python3
"""WCAG 2.2 Color Contrast Checker.
Checks foreground/background color pairs against WCAG 2.2 contrast ratio
thresholds for normal text, large text, and UI components. Supports hex,
rgb(), and named CSS colors.
Usage:
python contrast_checker.py "#ffffff" "#000000"
python contrast_checker.py --suggest "#336699"
python contrast_checker.py --batch styles.css
python contrast_checker.py --demo
"""
import argparse
import json
import re
import sys
# ---------------------------------------------------------------------------
# Named CSS colors (25 common ones)
# ---------------------------------------------------------------------------
NAMED_COLORS = {
"black": (0, 0, 0),
"white": (255, 255, 255),
"red": (255, 0, 0),
"green": (0, 128, 0),
"blue": (0, 0, 255),
"yellow": (255, 255, 0),
"cyan": (0, 255, 255),
"magenta": (255, 0, 255),
"gray": (128, 128, 128),
"grey": (128, 128, 128),
"orange": (255, 165, 0),
"purple": (128, 0, 128),
"pink": (255, 192, 203),
"brown": (165, 42, 42),
"navy": (0, 0, 128),
"teal": (0, 128, 128),
"olive": (128, 128, 0),
"maroon": (128, 0, 0),
"lime": (0, 255, 0),
"aqua": (0, 255, 255),
"silver": (192, 192, 192),
"gold": (255, 215, 0),
"coral": (255, 127, 80),
"salmon": (250, 128, 114),
"tomato": (255, 99, 71),
}
# WCAG thresholds: (label, required_ratio)
WCAG_THRESHOLDS = [
("AA Normal Text", 4.5),
("AA Large Text", 3.0),
("AA UI Components", 3.0),
("AAA Normal Text", 7.0),
("AAA Large Text", 4.5),
]
# ---------------------------------------------------------------------------
# Color parsing
# ---------------------------------------------------------------------------
def parse_color(color_str: str) -> tuple:
"""Parse a color string into an (R, G, B) tuple.
Accepts:
- #RRGGBB or #RGB hex
- rgb(r, g, b) with values 0-255
- Named CSS colors
"""
s = color_str.strip().lower()
# Named color
if s in NAMED_COLORS:
return NAMED_COLORS[s]
# Hex: #RGB or #RRGGBB
hex_match = re.match(r"^#([0-9a-f]{3}|[0-9a-f]{6})$", s)
if hex_match:
h = hex_match.group(1)
if len(h) == 3:
r, g, b = int(h[0] * 2, 16), int(h[1] * 2, 16), int(h[2] * 2, 16)
else:
r, g, b = int(h[0:2], 16), int(h[2:4], 16), int(h[4:6], 16)
return (r, g, b)
# rgb(r, g, b)
rgb_match = re.match(r"^rgb\(\s*(\d{1,3})\s*,\s*(\d{1,3})\s*,\s*(\d{1,3})\s*\)$", s)
if rgb_match:
r, g, b = int(rgb_match.group(1)), int(rgb_match.group(2)), int(rgb_match.group(3))
if not all(0 <= c <= 255 for c in (r, g, b)):
raise ValueError(f"RGB values must be 0-255, got rgb({r},{g},{b})")
return (r, g, b)
raise ValueError(
f"Invalid color format: '{color_str}'. "
"Use #RRGGBB, #RGB, rgb(r,g,b), or a named color."
)
def color_to_hex(rgb: tuple) -> str:
"""Convert an (R, G, B) tuple to #RRGGBB."""
return f"#{rgb[0]:02x}{rgb[1]:02x}{rgb[2]:02x}"
# ---------------------------------------------------------------------------
# WCAG luminance and contrast
# ---------------------------------------------------------------------------
def relative_luminance(rgb: tuple) -> float:
"""Calculate relative luminance per WCAG 2.2 (sRGB).
https://www.w3.org/TR/WCAG22/#dfn-relative-luminance
"""
channels = []
for c in rgb:
s = c / 255.0
channels.append(s / 12.92 if s <= 0.04045 else ((s + 0.055) / 1.055) ** 2.4)
return 0.2126 * channels[0] + 0.7152 * channels[1] + 0.0722 * channels[2]
def contrast_ratio(rgb1: tuple, rgb2: tuple) -> float:
"""Return the WCAG contrast ratio between two colors (>= 1.0)."""
l1 = relative_luminance(rgb1)
l2 = relative_luminance(rgb2)
lighter = max(l1, l2)
darker = min(l1, l2)
return (lighter + 0.05) / (darker + 0.05)
def evaluate_contrast(ratio: float) -> list:
"""Return pass/fail results for each WCAG threshold."""
results = []
for label, threshold in WCAG_THRESHOLDS:
results.append({
"level": label,
"required": threshold,
"ratio": round(ratio, 2),
"pass": ratio >= threshold,
})
return results
# ---------------------------------------------------------------------------
# Suggest accessible backgrounds
# ---------------------------------------------------------------------------
def suggest_backgrounds(fg_rgb: tuple, target_ratio: float = 4.5, count: int = 8) -> list:
"""Given a foreground color, suggest background colors passing AA normal text.
Strategy: walk luminance in both directions (lighter / darker) from the
foreground and collect the first colors that meet the target ratio.
"""
suggestions = []
# Try a spread of grays and tinted variants
candidates = []
for v in range(0, 256, 1):
candidates.append((v, v, v)) # grays
# Also try tinted versions toward the complement
fr, fg, fb = fg_rgb
for v in range(0, 256, 2):
candidates.append((v, min(255, v + 20), min(255, v + 40)))
candidates.append((min(255, v + 40), v, min(255, v + 20)))
candidates.append((min(255, v + 20), min(255, v + 40), v))
seen = set()
scored = []
for c in candidates:
cr = contrast_ratio(fg_rgb, c)
if cr >= target_ratio and c not in seen:
seen.add(c)
scored.append((cr, c))
# Sort by ratio closest to target (prefer minimal-change backgrounds)
scored.sort(key=lambda x: x[0])
for cr, c in scored[:count]:
suggestions.append({"hex": color_to_hex(c), "rgb": list(c), "ratio": round(cr, 2)})
return suggestions
# ---------------------------------------------------------------------------
# Batch CSS parsing
# ---------------------------------------------------------------------------
_COLOR_RE = re.compile(
r"(#[0-9a-fA-F]{3,6}|rgb\(\s*\d{1,3}\s*,\s*\d{1,3}\s*,\s*\d{1,3}\s*\))"
)
def extract_css_pairs(css_text: str) -> list:
"""Extract color / background-color pairs from CSS declarations.
Returns a list of dicts with selector, foreground, and background strings.
"""
pairs = []
# Split into rule blocks
block_re = re.compile(r"([^{}]+)\{([^}]+)\}", re.DOTALL)
for m in block_re.finditer(css_text):
selector = m.group(1).strip()
body = m.group(2)
fg = bg = None
# Match color: ... (but not background-color)
fg_match = re.search(
r"(?<![-])color\s*:\s*([^;]+);", body, re.IGNORECASE
)
bg_match = re.search(
r"background(?:-color)?\s*:\s*([^;]+);", body, re.IGNORECASE
)
if fg_match:
val = fg_match.group(1).strip()
c = _COLOR_RE.search(val)
if c:
fg = c.group(1)
elif val.lower() in NAMED_COLORS:
fg = val.lower()
if bg_match:
val = bg_match.group(1).strip()
c = _COLOR_RE.search(val)
if c:
bg = c.group(1)
elif val.lower() in NAMED_COLORS:
bg = val.lower()
if fg and bg:
pairs.append({"selector": selector, "foreground": fg, "background": bg})
return pairs
# ---------------------------------------------------------------------------
# Output formatting
# ---------------------------------------------------------------------------
def format_result_human(fg_str: str, bg_str: str, ratio: float, results: list) -> str:
"""Format a contrast check result for the terminal."""
lines = [
f"Foreground : {fg_str}",
f"Background : {bg_str}",
f"Contrast : {ratio:.2f}:1",
"",
]
for r in results:
status = "PASS" if r["pass"] else "FAIL"
lines.append(f" [{status}] {r['level']:20s} (requires {r['required']}:1)")
return "\n".join(lines)
def format_suggestions_human(fg_str: str, suggestions: list) -> str:
"""Format suggested backgrounds for the terminal."""
lines = [f"Foreground: {fg_str}", "Suggested accessible backgrounds (AA Normal Text):"]
if not suggestions:
lines.append(" No suggestions found.")
for s in suggestions:
lines.append(f" {s['hex']} ratio={s['ratio']}:1")
return "\n".join(lines)
# ---------------------------------------------------------------------------
# Demo
# ---------------------------------------------------------------------------
DEMO_PAIRS = [
("#ffffff", "#000000"),
("#336699", "#ffffff"),
("#ff6600", "#ffffff"),
("navy", "white"),
("rgb(100,100,100)", "#eeeeee"),
]
def run_demo(as_json: bool) -> None:
"""Run demo checks and print results."""
all_results = []
for fg_str, bg_str in DEMO_PAIRS:
fg_rgb = parse_color(fg_str)
bg_rgb = parse_color(bg_str)
ratio = contrast_ratio(fg_rgb, bg_rgb)
results = evaluate_contrast(ratio)
entry = {
"foreground": fg_str,
"background": bg_str,
"foreground_hex": color_to_hex(fg_rgb),
"background_hex": color_to_hex(bg_rgb),
"ratio": round(ratio, 2),
"results": results,
}
all_results.append(entry)
if as_json:
print(json.dumps({"demo": True, "checks": all_results}, indent=2))
else:
print("=" * 60)
print("WCAG 2.2 Contrast Checker - Demo")
print("=" * 60)
for entry in all_results:
print()
print(
format_result_human(
entry["foreground"], entry["background"],
entry["ratio"], entry["results"],
)
)
print()
print("-" * 60)
print("Suggestion demo for foreground #336699:")
suggestions = suggest_backgrounds(parse_color("#336699"))
print(format_suggestions_human("#336699", suggestions))
# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
description="WCAG 2.2 Color Contrast Checker. "
"Checks foreground/background pairs against AA and AAA thresholds.",
epilog="Examples:\n"
" %(prog)s '#ffffff' '#000000'\n"
" %(prog)s --suggest '#336699'\n"
" %(prog)s --batch styles.css\n"
" %(prog)s --demo\n",
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument(
"foreground",
nargs="?",
help="Foreground (text) color: #RRGGBB, #RGB, rgb(r,g,b), or named color",
)
parser.add_argument(
"background",
nargs="?",
help="Background color: #RRGGBB, #RGB, rgb(r,g,b), or named color",
)
parser.add_argument(
"--suggest",
metavar="COLOR",
help="Suggest accessible background colors for the given foreground color",
)
parser.add_argument(
"--batch",
metavar="CSS_FILE",
help="Extract color pairs from a CSS file and check each",
)
parser.add_argument(
"--json",
action="store_true",
dest="json_output",
help="Output results as JSON",
)
parser.add_argument(
"--demo",
action="store_true",
help="Show example output with sample color pairs",
)
return parser
def main() -> int:
parser = build_parser()
args = parser.parse_args()
# --demo mode
if args.demo:
run_demo(args.json_output)
return 0
# --suggest mode
if args.suggest:
try:
fg_rgb = parse_color(args.suggest)
except ValueError as exc:
print(f"Error: {exc}", file=sys.stderr)
return 1
suggestions = suggest_backgrounds(fg_rgb)
if args.json_output:
print(json.dumps({
"foreground": args.suggest,
"foreground_hex": color_to_hex(fg_rgb),
"suggestions": suggestions,
}, indent=2))
else:
print(format_suggestions_human(args.suggest, suggestions))
return 0
# --batch mode
if args.batch:
try:
with open(args.batch, "r", encoding="utf-8") as fh:
css_text = fh.read()
except FileNotFoundError:
print(f"Error: file not found: {args.batch}", file=sys.stderr)
return 1
except OSError as exc:
print(f"Error reading file: {exc}", file=sys.stderr)
return 1
pairs = extract_css_pairs(css_text)
if not pairs:
msg = "No color/background-color pairs found in the CSS file."
if args.json_output:
print(json.dumps({"batch": args.batch, "pairs": [], "message": msg}, indent=2))
else:
print(msg)
return 0
all_results = []
has_failure = False
for pair in pairs:
try:
fg_rgb = parse_color(pair["foreground"])
bg_rgb = parse_color(pair["background"])
except ValueError as exc:
entry = {
"selector": pair["selector"],
"foreground": pair["foreground"],
"background": pair["background"],
"error": str(exc),
}
all_results.append(entry)
continue
ratio = contrast_ratio(fg_rgb, bg_rgb)
results = evaluate_contrast(ratio)
if not results[0]["pass"]: # AA Normal Text
has_failure = True
entry = {
"selector": pair["selector"],
"foreground": pair["foreground"],
"background": pair["background"],
"foreground_hex": color_to_hex(fg_rgb),
"background_hex": color_to_hex(bg_rgb),
"ratio": round(ratio, 2),
"results": results,
}
all_results.append(entry)
if args.json_output:
print(json.dumps({"batch": args.batch, "pairs": all_results}, indent=2))
else:
print(f"Batch check: {args.batch}")
print("=" * 60)
for entry in all_results:
print(f"\nSelector: {entry['selector']}")
if "error" in entry:
print(f" Error: {entry['error']}")
else:
print(
format_result_human(
entry["foreground"], entry["background"],
entry["ratio"], entry["results"],
)
)
print()
summary_pass = sum(1 for e in all_results if "ratio" in e and e["results"][0]["pass"])
summary_total = sum(1 for e in all_results if "ratio" in e)
print(f"Summary: {summary_pass}/{summary_total} pairs pass AA Normal Text")
return 1 if has_failure else 0
# Default: check a single pair
if not args.foreground or not args.background:
parser.error(
"Provide foreground and background colors, or use --suggest, --batch, or --demo."
)
try:
fg_rgb = parse_color(args.foreground)
except ValueError as exc:
print(f"Error (foreground): {exc}", file=sys.stderr)
return 1
try:
bg_rgb = parse_color(args.background)
except ValueError as exc:
print(f"Error (background): {exc}", file=sys.stderr)
return 1
ratio = contrast_ratio(fg_rgb, bg_rgb)
results = evaluate_contrast(ratio)
if args.json_output:
print(json.dumps({
"foreground": args.foreground,
"background": args.background,
"foreground_hex": color_to_hex(fg_rgb),
"background_hex": color_to_hex(bg_rgb),
"ratio": round(ratio, 2),
"results": results,
}, indent=2))
else:
print(format_result_human(args.foreground, args.background, ratio, results))
return 0 if results[0]["pass"] else 1
if __name__ == "__main__":
sys.exit(main())
Lập kế hoạch, thiết kế và triển khai thử nghiệm A/B hoặc thử nghiệm chuyển đổi.
---
name: "ab-test-setup"
description: When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "conversion experiment," "statistical significance," or "test this." For tracking implementation, see analytics-tracking.
license: MIT
metadata:
version: 1.0.0
author: Alireza Rezvani
category: marketing
updated: 2026-03-06
---
# A/B Test Setup
You are an expert in experimentation and A/B testing. Your goal is to help design tests that produce statistically valid, actionable results.
## Initial Assessment
**Check for product marketing context first:**
If `.claude/product-marketing-context.md` exists, read it before asking questions. Use that context and only ask for information not already covered or specific to this task.
Before designing a test, understand:
1. **Test Context** - What are you trying to improve? What change are you considering?
2. **Current State** - Baseline conversion rate? Current traffic volume?
3. **Constraints** - Technical complexity? Timeline? Tools available?
---
## Core Principles
### 1. Start with a Hypothesis
- Not just "let's see what happens"
- Specific prediction of outcome
- Based on reasoning or data
### 2. Test One Thing
- Single variable per test
- Otherwise you don't know what worked
### 3. Statistical Rigor
- Pre-determine sample size
- Don't peek and stop early
- Commit to the methodology
### 4. Measure What Matters
- Primary metric tied to business value
- Secondary metrics for context
- Guardrail metrics to prevent harm
---
## Hypothesis Framework
### Structure
```
Because [observation/data],
we believe [change]
will cause [expected outcome]
for [audience].
We'll know this is true when [metrics].
```
### Example
**Weak**: "Changing the button color might increase clicks."
**Strong**: "Because users report difficulty finding the CTA (per heatmaps and feedback), we believe making the button larger and using contrasting color will increase CTA clicks by 15%+ for new visitors. We'll measure click-through rate from page view to signup start."
---
## Test Types
| Type | Description | Traffic Needed |
|------|-------------|----------------|
| A/B | Two versions, single change | Moderate |
| A/B/n | Multiple variants | Higher |
| MVT | Multiple changes in combinations | Very high |
| Split URL | Different URLs for variants | Moderate |
---
## Sample Size
### Quick Reference
| Baseline | 10% Lift | 20% Lift | 50% Lift |
|----------|----------|----------|----------|
| 1% | 150k/variant | 39k/variant | 6k/variant |
| 3% | 47k/variant | 12k/variant | 2k/variant |
| 5% | 27k/variant | 7k/variant | 1.2k/variant |
| 10% | 12k/variant | 3k/variant | 550/variant |
**Calculators:**
- [Evan Miller's](https://www.evanmiller.org/ab-testing/sample-size.html)
- [Optimizely's](https://www.optimizely.com/sample-size-calculator/)
**For detailed sample size tables and duration calculations**: See [references/sample-size-guide.md](references/sample-size-guide.md)
---
## Metrics Selection
### Primary Metric
- Single metric that matters most
- Directly tied to hypothesis
- What you'll use to call the test
### Secondary Metrics
- Support primary metric interpretation
- Explain why/how the change worked
### Guardrail Metrics
- Things that shouldn't get worse
- Stop test if significantly negative
### Example: Pricing Page Test
- **Primary**: Plan selection rate
- **Secondary**: Time on page, plan distribution
- **Guardrail**: Support tickets, refund rate
---
## Designing Variants
### What to Vary
| Category | Examples |
|----------|----------|
| Headlines/Copy | Message angle, value prop, specificity, tone |
| Visual Design | Layout, color, images, hierarchy |
| CTA | Button copy, size, placement, number |
| Content | Information included, order, amount, social proof |
### Best Practices
- Single, meaningful change
- Bold enough to make a difference
- True to the hypothesis
---
## Traffic Allocation
| Approach | Split | When to Use |
|----------|-------|-------------|
| Standard | 50/50 | Default for A/B |
| Conservative | 90/10, 80/20 | Limit risk of bad variant |
| Ramping | Start small, increase | Technical risk mitigation |
**Considerations:**
- Consistency: Users see same variant on return
- Balanced exposure across time of day/week
---
## Implementation
### Client-Side
- JavaScript modifies page after load
- Quick to implement, can cause flicker
- Tools: PostHog, Optimizely, VWO
### Server-Side
- Variant determined before render
- No flicker, requires dev work
- Tools: PostHog, LaunchDarkly, Split
---
## Running the Test
### Pre-Launch Checklist
- [ ] Hypothesis documented
- [ ] Primary metric defined
- [ ] Sample size calculated
- [ ] Variants implemented correctly
- [ ] Tracking verified
- [ ] QA completed on all variants
### During the Test
**DO:**
- Monitor for technical issues
- Check segment quality
- Document external factors
**DON'T:**
- Peek at results and stop early
- Make changes to variants
- Add traffic from new sources
### The Peeking Problem
Looking at results before reaching sample size and stopping early leads to false positives and wrong decisions. Pre-commit to sample size and trust the process.
---
## Analyzing Results
### Statistical Significance
- 95% confidence = p-value < 0.05
- Means <5% chance result is random
- Not a guarantee—just a threshold
### Analysis Checklist
1. **Reach sample size?** If not, result is preliminary
2. **Statistically significant?** Check confidence intervals
3. **Effect size meaningful?** Compare to MDE, project impact
4. **Secondary metrics consistent?** Support the primary?
5. **Guardrail concerns?** Anything get worse?
6. **Segment differences?** Mobile vs. desktop? New vs. returning?
### Interpreting Results
| Result | Conclusion |
|--------|------------|
| Significant winner | Implement variant |
| Significant loser | Keep control, learn why |
| No significant difference | Need more traffic or bolder test |
| Mixed signals | Dig deeper, maybe segment |
---
## Documentation
Document every test with:
- Hypothesis
- Variants (with screenshots)
- Results (sample, metrics, significance)
- Decision and learnings
**For templates**: See [references/test-templates.md](references/test-templates.md)
---
## Common Mistakes
### Test Design
- Testing too small a change (undetectable)
- Testing too many things (can't isolate)
- No clear hypothesis
### Execution
- Stopping early
- Changing things mid-test
- Not checking implementation
### Analysis
- Ignoring confidence intervals
- Cherry-picking segments
- Over-interpreting inconclusive results
---
## Task-Specific Questions
1. What's your current conversion rate?
2. How much traffic does this page get?
3. What change are you considering and why?
4. What's the smallest improvement worth detecting?
5. What tools do you have for testing?
6. Have you tested this area before?
---
## Proactive Triggers
Proactively offer A/B test design when:
1. **Conversion rate mentioned** — User shares a conversion rate and asks how to improve it; suggest designing a test rather than guessing at solutions.
2. **Copy or design decision is unclear** — When two variants of a headline, CTA, or layout are being debated, propose testing instead of opinionating.
3. **Campaign underperformance** — User reports a landing page or email performing below expectations; offer a structured test plan.
4. **Pricing page discussion** — Any mention of pricing page changes should trigger an offer to design a pricing test with guardrail metrics.
5. **Post-launch review** — After a feature or campaign goes live, propose follow-up experiments to optimize the result.
---
## Output Artifacts
| Artifact | Format | Description |
|----------|--------|-------------|
| Experiment Brief | Markdown doc | Hypothesis, variants, metrics, sample size, duration, owner |
| Sample Size Calculator Input | Table | Baseline rate, MDE, confidence level, power |
| Pre-Launch QA Checklist | Checklist | Implementation, tracking, variant rendering verification |
| Results Analysis Report | Markdown doc | Statistical significance, effect size, segment breakdown, decision |
| Test Backlog | Prioritized list | Ranked experiments by expected impact and feasibility |
---
## Communication
All outputs should meet the quality standard: clear hypothesis, pre-registered metrics, and documented decisions. Avoid presenting inconclusive results as wins. Every test should produce a learning, even if the variant loses. Reference `marketing-context` for product and audience framing before designing experiments.
---
## Related Skills
- **page-cro** — USE when you need ideas for *what* to test; NOT when you already have a hypothesis and just need test design.
- **analytics-tracking** — USE to set up measurement infrastructure before running tests; NOT as a substitute for defining primary metrics upfront.
- **campaign-analytics** — USE after tests conclude to fold results into broader campaign attribution; NOT during the test itself.
- **pricing-strategy** — USE when test results affect pricing decisions; NOT to replace a controlled test with pure strategic reasoning.
- **marketing-context** — USE as foundation before any test design to ensure hypotheses align with ICP and positioning; always load first.
FILE:references/sample-size-guide.md
# Sample Size Guide
Reference for calculating sample sizes and test duration.
## Sample Size Fundamentals
### Required Inputs
1. **Baseline conversion rate**: Your current rate
2. **Minimum detectable effect (MDE)**: Smallest change worth detecting
3. **Statistical significance level**: Usually 95% (α = 0.05)
4. **Statistical power**: Usually 80% (β = 0.20)
### What These Mean
**Baseline conversion rate**: If your page converts at 5%, that's your baseline.
**MDE (Minimum Detectable Effect)**: The smallest improvement you care about detecting. Set this based on:
- Business impact (is a 5% lift meaningful?)
- Implementation cost (worth the effort?)
- Realistic expectations (what have past tests shown?)
**Statistical significance (95%)**: Means there's less than 5% chance the observed difference is due to random chance.
**Statistical power (80%)**: Means if there's a real effect of size MDE, you have 80% chance of detecting it.
---
## Sample Size Quick Reference Tables
### Conversion Rate: 1%
| Lift to Detect | Sample per Variant | Total Sample |
|----------------|-------------------|--------------|
| 5% (1% → 1.05%) | 1,500,000 | 3,000,000 |
| 10% (1% → 1.1%) | 380,000 | 760,000 |
| 20% (1% → 1.2%) | 97,000 | 194,000 |
| 50% (1% → 1.5%) | 16,000 | 32,000 |
| 100% (1% → 2%) | 4,200 | 8,400 |
### Conversion Rate: 3%
| Lift to Detect | Sample per Variant | Total Sample |
|----------------|-------------------|--------------|
| 5% (3% → 3.15%) | 480,000 | 960,000 |
| 10% (3% → 3.3%) | 120,000 | 240,000 |
| 20% (3% → 3.6%) | 31,000 | 62,000 |
| 50% (3% → 4.5%) | 5,200 | 10,400 |
| 100% (3% → 6%) | 1,400 | 2,800 |
### Conversion Rate: 5%
| Lift to Detect | Sample per Variant | Total Sample |
|----------------|-------------------|--------------|
| 5% (5% → 5.25%) | 280,000 | 560,000 |
| 10% (5% → 5.5%) | 72,000 | 144,000 |
| 20% (5% → 6%) | 18,000 | 36,000 |
| 50% (5% → 7.5%) | 3,100 | 6,200 |
| 100% (5% → 10%) | 810 | 1,620 |
### Conversion Rate: 10%
| Lift to Detect | Sample per Variant | Total Sample |
|----------------|-------------------|--------------|
| 5% (10% → 10.5%) | 130,000 | 260,000 |
| 10% (10% → 11%) | 34,000 | 68,000 |
| 20% (10% → 12%) | 8,700 | 17,400 |
| 50% (10% → 15%) | 1,500 | 3,000 |
| 100% (10% → 20%) | 400 | 800 |
### Conversion Rate: 20%
| Lift to Detect | Sample per Variant | Total Sample |
|----------------|-------------------|--------------|
| 5% (20% → 21%) | 60,000 | 120,000 |
| 10% (20% → 22%) | 16,000 | 32,000 |
| 20% (20% → 24%) | 4,000 | 8,000 |
| 50% (20% → 30%) | 700 | 1,400 |
| 100% (20% → 40%) | 200 | 400 |
---
## Duration Calculator
### Formula
```
Duration (days) = (Sample per variant × Number of variants) / (Daily traffic × % exposed)
```
### Examples
**Scenario 1: High-traffic page**
- Need: 10,000 per variant (2 variants = 20,000 total)
- Daily traffic: 5,000 visitors
- 100% exposed to test
- Duration: 20,000 / 5,000 = **4 days**
**Scenario 2: Medium-traffic page**
- Need: 30,000 per variant (60,000 total)
- Daily traffic: 2,000 visitors
- 100% exposed
- Duration: 60,000 / 2,000 = **30 days**
**Scenario 3: Low-traffic with partial exposure**
- Need: 15,000 per variant (30,000 total)
- Daily traffic: 500 visitors
- 50% exposed to test
- Effective daily: 250
- Duration: 30,000 / 250 = **120 days** (too long!)
### Minimum Duration Rules
Even with sufficient sample size, run tests for at least:
- **1 full week**: To capture day-of-week variation
- **2 business cycles**: If B2B (weekday vs. weekend patterns)
- **Through paydays**: If e-commerce (beginning/end of month)
### Maximum Duration Guidelines
Avoid running tests longer than 4-8 weeks:
- Novelty effects wear off
- External factors intervene
- Opportunity cost of other tests
---
## Online Calculators
### Recommended Tools
**Evan Miller's Calculator**
https://www.evanmiller.org/ab-testing/sample-size.html
- Simple interface
- Bookmark-worthy
**Optimizely's Calculator**
https://www.optimizely.com/sample-size-calculator/
- Business-friendly language
- Duration estimates
**AB Test Guide Calculator**
https://www.abtestguide.com/calc/
- Includes Bayesian option
- Multiple test types
**VWO Duration Calculator**
https://vwo.com/tools/ab-test-duration-calculator/
- Duration-focused
- Good for planning
---
## Adjusting for Multiple Variants
With more than 2 variants (A/B/n tests), you need more sample:
| Variants | Multiplier |
|----------|------------|
| 2 (A/B) | 1x |
| 3 (A/B/C) | ~1.5x |
| 4 (A/B/C/D) | ~2x |
| 5+ | Consider reducing variants |
**Why?** More comparisons increase chance of false positives. You're comparing:
- A vs B
- A vs C
- B vs C (sometimes)
Apply Bonferroni correction or use tools that handle this automatically.
---
## Common Sample Size Mistakes
### 1. Underpowered tests
**Problem**: Not enough sample to detect realistic effects
**Fix**: Be realistic about MDE, get more traffic, or don't test
### 2. Overpowered tests
**Problem**: Waiting for sample size when you already have significance
**Fix**: This is actually fine—you committed to sample size, honor it
### 3. Wrong baseline rate
**Problem**: Using wrong conversion rate for calculation
**Fix**: Use the specific metric and page, not site-wide averages
### 4. Ignoring segments
**Problem**: Calculating for full traffic, then analyzing segments
**Fix**: If you plan segment analysis, calculate sample for smallest segment
### 5. Testing too many things
**Problem**: Dividing traffic too many ways
**Fix**: Prioritize ruthlessly, run fewer concurrent tests
---
## When Sample Size Requirements Are Too High
Options when you can't get enough traffic:
1. **Increase MDE**: Accept only detecting larger effects (20%+ lift)
2. **Lower confidence**: Use 90% instead of 95% (risky, document it)
3. **Reduce variants**: Test only the most promising variant
4. **Combine traffic**: Test across multiple similar pages
5. **Test upstream**: Test earlier in funnel where traffic is higher
6. **Don't test**: Make decision based on qualitative data instead
7. **Longer test**: Accept longer duration (weeks/months)
---
## Sequential Testing
If you must check results before reaching sample size:
### What is it?
Statistical method that adjusts for multiple looks at data.
### When to use
- High-risk changes
- Need to stop bad variants early
- Time-sensitive decisions
### Tools that support it
- Optimizely (Stats Accelerator)
- VWO (SmartStats)
- PostHog (Bayesian approach)
### Tradeoff
- More flexibility to stop early
- Slightly larger sample size requirement
- More complex analysis
---
## Quick Decision Framework
### Can I run this test?
```
Daily traffic to page: _____
Baseline conversion rate: _____
MDE I care about: _____
Sample needed per variant: _____ (from tables above)
Days to run: Sample / Daily traffic = _____
If days > 60: Consider alternatives
If days > 30: Acceptable for high-impact tests
If days < 14: Likely feasible
If days < 7: Easy to run, consider running longer anyway
```
FILE:references/test-templates.md
# A/B Test Templates Reference
Templates for planning, documenting, and analyzing experiments.
## Test Plan Template
```markdown
# A/B Test: [Name]
## Overview
- **Owner**: [Name]
- **Test ID**: [ID in testing tool]
- **Page/Feature**: [What's being tested]
- **Planned dates**: [Start] - [End]
## Hypothesis
Because [observation/data],
we believe [change]
will cause [expected outcome]
for [audience].
We'll know this is true when [metrics].
## Test Design
| Element | Details |
|---------|---------|
| Test type | A/B / A/B/n / MVT |
| Duration | X weeks |
| Sample size | X per variant |
| Traffic allocation | 50/50 |
| Tool | [Tool name] |
| Implementation | Client-side / Server-side |
## Variants
### Control (A)
[Screenshot]
- Current experience
- [Key details about current state]
### Variant (B)
[Screenshot or mockup]
- [Specific change #1]
- [Specific change #2]
- Rationale: [Why we think this will win]
## Metrics
### Primary
- **Metric**: [metric name]
- **Definition**: [how it's calculated]
- **Current baseline**: [X%]
- **Minimum detectable effect**: [X%]
### Secondary
- [Metric 1]: [what it tells us]
- [Metric 2]: [what it tells us]
- [Metric 3]: [what it tells us]
### Guardrails
- [Metric that shouldn't get worse]
- [Another safety metric]
## Segment Analysis Plan
- Mobile vs. desktop
- New vs. returning visitors
- Traffic source
- [Other relevant segments]
## Success Criteria
- Winner: [Primary metric improves by X% with 95% confidence]
- Loser: [Primary metric decreases significantly]
- Inconclusive: [What we'll do if no significant result]
## Pre-Launch Checklist
- [ ] Hypothesis documented and reviewed
- [ ] Primary metric defined and trackable
- [ ] Sample size calculated
- [ ] Test duration estimated
- [ ] Variants implemented correctly
- [ ] Tracking verified in all variants
- [ ] QA completed on all variants
- [ ] Stakeholders informed
- [ ] Calendar hold for analysis date
```
---
## Results Documentation Template
```markdown
# A/B Test Results: [Name]
## Summary
| Element | Value |
|---------|-------|
| Test ID | [ID] |
| Dates | [Start] - [End] |
| Duration | X days |
| Result | Winner / Loser / Inconclusive |
| Decision | [What we're doing] |
## Hypothesis (Reminder)
[Copy from test plan]
## Results
### Sample Size
| Variant | Target | Actual | % of target |
|---------|--------|--------|-------------|
| Control | X | Y | Z% |
| Variant | X | Y | Z% |
### Primary Metric: [Metric Name]
| Variant | Value | 95% CI | vs. Control |
|---------|-------|--------|-------------|
| Control | X% | [X%, Y%] | — |
| Variant | X% | [X%, Y%] | +X% |
**Statistical significance**: p = X.XX (95% = sig / not sig)
**Practical significance**: [Is this lift meaningful for the business?]
### Secondary Metrics
| Metric | Control | Variant | Change | Significant? |
|--------|---------|---------|--------|--------------|
| [Metric 1] | X | Y | +Z% | Yes/No |
| [Metric 2] | X | Y | +Z% | Yes/No |
### Guardrail Metrics
| Metric | Control | Variant | Change | Concern? |
|--------|---------|---------|--------|----------|
| [Metric 1] | X | Y | +Z% | Yes/No |
### Segment Analysis
**Mobile vs. Desktop**
| Segment | Control | Variant | Lift |
|---------|---------|---------|------|
| Mobile | X% | Y% | +Z% |
| Desktop | X% | Y% | +Z% |
**New vs. Returning**
| Segment | Control | Variant | Lift |
|---------|---------|---------|------|
| New | X% | Y% | +Z% |
| Returning | X% | Y% | +Z% |
## Interpretation
### What happened?
[Explanation of results in plain language]
### Why do we think this happened?
[Analysis and reasoning]
### Caveats
[Any limitations, external factors, or concerns]
## Decision
**Winner**: [Control / Variant]
**Action**: [Implement variant / Keep control / Re-test]
**Timeline**: [When changes will be implemented]
## Learnings
### What we learned
- [Key insight 1]
- [Key insight 2]
### What to test next
- [Follow-up test idea 1]
- [Follow-up test idea 2]
### Impact
- **Projected lift**: [X% improvement in Y metric]
- **Business impact**: [Revenue, conversions, etc.]
```
---
## Test Repository Entry Template
For tracking all tests in a central location:
```markdown
| Test ID | Name | Page | Dates | Primary Metric | Result | Lift | Link |
|---------|------|------|-------|----------------|--------|------|------|
| 001 | Hero headline test | Homepage | 1/1-1/15 | CTR | Winner | +12% | [Link] |
| 002 | Pricing table layout | Pricing | 1/10-1/31 | Plan selection | Loser | -5% | [Link] |
| 003 | Signup form fields | Signup | 2/1-2/14 | Completion | Inconclusive | +2% | [Link] |
```
---
## Quick Test Brief Template
For simple tests that don't need full documentation:
```markdown
## [Test Name]
**What**: [One sentence description]
**Why**: [One sentence hypothesis]
**Metric**: [Primary metric]
**Duration**: [X weeks]
**Result**: [TBD / Winner / Loser / Inconclusive]
**Learnings**: [Key takeaway]
```
---
## Stakeholder Update Template
```markdown
## A/B Test Update: [Name]
**Status**: Running / Complete
**Days remaining**: X (or complete)
**Current sample**: X% of target
### Preliminary observations
[What we're seeing - without making decisions yet]
### Next steps
[What happens next]
### Timeline
- [Date]: Analysis complete
- [Date]: Decision and recommendation
- [Date]: Implementation (if winner)
```
---
## Experiment Prioritization Scorecard
For deciding which tests to run:
| Factor | Weight | Test A | Test B | Test C |
|--------|--------|--------|--------|--------|
| Potential impact | 30% | | | |
| Confidence in hypothesis | 25% | | | |
| Ease of implementation | 20% | | | |
| Risk if wrong | 15% | | | |
| Strategic alignment | 10% | | | |
| **Total** | | | | |
Scoring: 1-5 (5 = best)
---
## Hypothesis Bank Template
For collecting test ideas:
```markdown
| ID | Page/Area | Observation | Hypothesis | Potential Impact | Status |
|----|-----------|-------------|------------|------------------|--------|
| H1 | Homepage | Low scroll depth | Shorter hero will increase scroll | High | Testing |
| H2 | Pricing | Users compare plans | Comparison table will help | Medium | Backlog |
| H3 | Signup | Drop-off at email | Social login will increase completion | Medium | Backlog |
```
FILE:scripts/sample_size_calculator.py
#!/usr/bin/env python3
"""
sample_size_calculator.py — A/B Test Sample Size Calculator
100% stdlib, no pip installs required.
Usage:
python3 sample_size_calculator.py # demo mode
python3 sample_size_calculator.py --baseline 0.05 --mde 0.20
python3 sample_size_calculator.py --baseline 0.05 --mde 0.20 --daily-traffic 500
python3 sample_size_calculator.py --baseline 0.05 --mde 0.20 --json
"""
import argparse
import json
import math
import sys
# ---------------------------------------------------------------------------
# Z-score approximation (scipy-free, Beasley-Springer-Moro algorithm)
# ---------------------------------------------------------------------------
def _norm_ppf(p: float) -> float:
"""Percent-point function (inverse CDF) of the standard normal.
Uses rational approximation — accurate to ~1e-9.
Reference: Abramowitz & Stegun 26.2.17 / Peter Acklam's algorithm.
"""
if p <= 0 or p >= 1:
raise ValueError(f"p must be in (0, 1), got {p}")
# Coefficients for rational approximation
a = [-3.969683028665376e+01, 2.209460984245205e+02,
-2.759285104469687e+02, 1.383577518672690e+02,
-3.066479806614716e+01, 2.506628277459239e+00]
b = [-5.447609879822406e+01, 1.615858368580409e+02,
-1.556989798598866e+02, 6.680131188771972e+01,
-1.328068155288572e+01]
c = [-7.784894002430293e-03, -3.223964580411365e-01,
-2.400758277161838e+00, -2.549732539343734e+00,
4.374664141464968e+00, 2.938163982698783e+00]
d = [7.784695709041462e-03, 3.224671290700398e-01,
2.445134137142996e+00, 3.754408661907416e+00]
p_low = 0.02425
p_high = 1 - p_low
if p < p_low:
q = math.sqrt(-2 * math.log(p))
return (((((c[0]*q+c[1])*q+c[2])*q+c[3])*q+c[4])*q+c[5]) / \
((((d[0]*q+d[1])*q+d[2])*q+d[3])*q+1)
elif p <= p_high:
q = p - 0.5
r = q * q
return (((((a[0]*r+a[1])*r+a[2])*r+a[3])*r+a[4])*r+a[5])*q / \
(((((b[0]*r+b[1])*r+b[2])*r+b[3])*r+b[4])*r+1)
else:
q = math.sqrt(-2 * math.log(1 - p))
return -(((((c[0]*q+c[1])*q+c[2])*q+c[3])*q+c[4])*q+c[5]) / \
((((d[0]*q+d[1])*q+d[2])*q+d[3])*q+1)
# ---------------------------------------------------------------------------
# Core calculation
# ---------------------------------------------------------------------------
def calculate_sample_size(
baseline: float,
mde: float,
alpha: float = 0.05,
power: float = 0.80,
) -> dict:
"""
Two-proportion z-test sample size formula (two-tailed).
n = (Z_alpha/2 + Z_beta)^2 * (p1*(1-p1) + p2*(1-p2)) / (p2 - p1)^2
Args:
baseline : baseline conversion rate (e.g. 0.05 for 5%)
mde : minimum detectable effect as relative lift (e.g. 0.20 for +20%)
alpha : significance level (Type I error rate), default 0.05
power : statistical power (1 - Type II error rate), default 0.80
Returns dict with all intermediate values and results.
"""
p1 = baseline
p2 = baseline * (1 + mde) # expected conversion with treatment
if not (0 < p1 < 1):
raise ValueError(f"baseline must be in (0,1), got {p1}")
if not (0 < p2 < 1):
raise ValueError(
f"baseline * (1 + mde) = {p2:.4f} is outside (0,1). "
"Reduce mde or increase baseline."
)
z_alpha = _norm_ppf(1 - alpha / 2) # two-tailed
z_beta = _norm_ppf(power)
pooled_var = p1 * (1 - p1) + p2 * (1 - p2)
effect_sq = (p2 - p1) ** 2
n_raw = ((z_alpha + z_beta) ** 2 * pooled_var) / effect_sq
n = math.ceil(n_raw)
return {
"inputs": {
"baseline_conversion_rate": p1,
"minimum_detectable_effect_relative": mde,
"expected_variant_conversion_rate": round(p2, 6),
"significance_level_alpha": alpha,
"statistical_power": power,
},
"z_scores": {
"z_alpha_2": round(z_alpha, 4),
"z_beta": round(z_beta, 4),
},
"results": {
"sample_size_per_variation": n,
"total_sample_size": n * 2,
"absolute_lift": round(p2 - p1, 6),
"relative_lift_pct": round(mde * 100, 2),
},
"formula": (
"n = (Z_α/2 + Z_β)² × (p1(1−p1) + p2(1−p2)) / (p2−p1)² "
"[two-proportion z-test, two-tailed]"
),
"assumptions": [
"Two-tailed test (detecting lift in either direction)",
"Independent samples (no within-subject correlation)",
"Fixed horizon (not sequential / always-valid)",
"Binomial outcome (conversion yes/no)",
"No novelty effect correction applied",
],
}
def add_duration(result: dict, daily_traffic: int) -> dict:
"""Append estimated test duration given total daily traffic (both variants)."""
n_total = result["results"]["total_sample_size"]
days = math.ceil(n_total / daily_traffic)
weeks = round(days / 7, 1)
result["duration"] = {
"daily_traffic_both_variants": daily_traffic,
"estimated_days": days,
"estimated_weeks": weeks,
"note": (
"Assumes traffic is evenly split 50/50 between control and variant. "
"Add ~10–20% buffer for weekday/weekend variance."
),
}
return result
# ---------------------------------------------------------------------------
# Scoring helper (0-100)
# ---------------------------------------------------------------------------
def score_test_design(result: dict) -> dict:
"""Heuristic quality score for the A/B test design."""
score = 100
reasons = []
inputs = result["inputs"]
# Penalise very low baseline (unreliable estimates)
if inputs["baseline_conversion_rate"] < 0.01:
score -= 15
reasons.append("Baseline <1%: high variance, consider aggregating more data first.")
# Penalise tiny MDE (will need enormous sample)
mde = inputs["minimum_detectable_effect_relative"]
if mde < 0.05:
score -= 20
reasons.append("MDE <5%: very small effect, experiment may take months.")
elif mde < 0.10:
score -= 10
reasons.append("MDE <10%: moderately small effect size.")
# Penalise overly aggressive alpha
if inputs["significance_level_alpha"] > 0.10:
score -= 15
reasons.append("α >10%: high false-positive risk.")
# Penalise low power
if inputs["statistical_power"] < 0.80:
score -= 20
reasons.append("Power <80%: elevated risk of missing real effects (Type II error).")
# Duration penalty (if available)
dur = result.get("duration")
if dur:
days = dur["estimated_days"]
if days > 90:
score -= 20
reasons.append(f"Test duration {days}d >90 days: novelty/seasonal effects likely.")
elif days > 30:
score -= 10
reasons.append(f"Test duration {days}d >30 days: monitor for external confounders.")
score = max(0, score)
return {
"design_quality_score": score,
"score_interpretation": _score_label(score),
"issues": reasons if reasons else ["No major design issues detected."],
}
def _score_label(s: int) -> str:
if s >= 90: return "Excellent"
if s >= 75: return "Good"
if s >= 60: return "Fair"
if s >= 40: return "Poor"
return "Critical"
# ---------------------------------------------------------------------------
# Pretty-print
# ---------------------------------------------------------------------------
def pretty_print(result: dict, score: dict) -> None:
inp = result["inputs"]
res = result["results"]
zs = result["z_scores"]
print("\n" + "=" * 60)
print(" A/B TEST SAMPLE SIZE CALCULATOR")
print("=" * 60)
print("\n📥 INPUTS")
print(f" Baseline conversion rate : {inp['baseline_conversion_rate']*100:.2f}%")
print(f" Variant conversion rate : {inp['expected_variant_conversion_rate']*100:.2f}%")
print(f" Minimum detectable effect: {inp['minimum_detectable_effect_relative']*100:.1f}% relative "
f"(+{res['absolute_lift']*100:.3f}pp absolute)")
print(f" Significance level (α) : {inp['significance_level_alpha']}")
print(f" Statistical power : {inp['statistical_power']*100:.0f}%")
print("\n📐 FORMULA")
print(f" {result['formula']}")
print(f" Z_α/2 = {zs['z_alpha_2']} Z_β = {zs['z_beta']}")
print("\n📊 RESULTS")
print(f" ✅ Sample size per variation : {res['sample_size_per_variation']:,}")
print(f" ✅ Total sample size (both) : {res['total_sample_size']:,}")
if "duration" in result:
d = result["duration"]
print(f"\n⏱️ DURATION ESTIMATE (traffic: {d['daily_traffic_both_variants']:,}/day)")
print(f" Estimated test duration : {d['estimated_days']} days (~{d['estimated_weeks']} weeks)")
print(f" Note: {d['note']}")
print("\n💡 ASSUMPTIONS")
for a in result["assumptions"]:
print(f" • {a}")
print(f"\n🎯 DESIGN QUALITY SCORE: {score['design_quality_score']}/100 ({score['score_interpretation']})")
for issue in score["issues"]:
print(f" ⚠ {issue}")
print()
# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------
def parse_args():
parser = argparse.ArgumentParser(
description="Calculate required sample size for an A/B test (stdlib only).",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__,
)
parser.add_argument("--baseline", type=float, default=None,
help="Baseline conversion rate (e.g. 0.05 for 5%%)")
parser.add_argument("--mde", type=float, default=None,
help="Minimum detectable effect as relative lift (e.g. 0.20 for +20%%)")
parser.add_argument("--alpha", type=float, default=0.05,
help="Significance level α (default: 0.05)")
parser.add_argument("--power", type=float, default=0.80,
help="Statistical power 1-β (default: 0.80)")
parser.add_argument("--daily-traffic", type=int, default=None,
help="Total daily visitors across both variants (for duration estimate)")
parser.add_argument("--json", action="store_true",
help="Output results as JSON")
return parser.parse_args()
DEMO_SCENARIOS = [
{"label": "E-commerce checkout (low baseline)",
"baseline": 0.03, "mde": 0.20, "alpha": 0.05, "power": 0.80, "daily_traffic": 800},
{"label": "SaaS free-trial signup (medium baseline)",
"baseline": 0.08, "mde": 0.15, "alpha": 0.05, "power": 0.80, "daily_traffic": 2000},
{"label": "Button CTA (high baseline)",
"baseline": 0.25, "mde": 0.10, "alpha": 0.05, "power": 0.80, "daily_traffic": 5000},
]
def main():
args = parse_args()
demo_mode = (args.baseline is None and args.mde is None)
if demo_mode:
print("🔬 DEMO MODE — running 3 sample scenarios\n")
all_results = []
for sc in DEMO_SCENARIOS:
res = calculate_sample_size(sc["baseline"], sc["mde"], sc["alpha"], sc["power"])
res = add_duration(res, sc["daily_traffic"])
sc_score = score_test_design(res)
res["scenario"] = sc["label"]
res["score"] = sc_score
all_results.append(res)
if not args.json:
print(f"\n{'─'*60}")
print(f"SCENARIO: {sc['label']}")
pretty_print(res, sc_score)
if args.json:
print(json.dumps(all_results, indent=2))
return
# Single calculation mode
if args.baseline is None or args.mde is None:
print("Error: --baseline and --mde are required (or omit both for demo mode).", file=sys.stderr)
sys.exit(1)
result = calculate_sample_size(args.baseline, args.mde, args.alpha, args.power)
if args.daily_traffic:
result = add_duration(result, args.daily_traffic)
sc_score = score_test_design(result)
result["score"] = sc_score
if args.json:
print(json.dumps(result, indent=2))
else:
pretty_print(result, sc_score)
if __name__ == "__main__":
main()
Chạy nhiều subagent song song trên cùng một nhiệm vụ bằng git worktree, đánh giá và merge nhánh tốt nhất.
---
name: "agenthub"
description: "Multi-agent collaboration plugin that spawns N parallel subagents competing on the same task via git worktree isolation. Agents work independently, results are evaluated by metric or LLM judge, and the best branch is merged. Use when: user wants multiple approaches tried in parallel — code optimization, content variation, research exploration, or any task that benefits from parallel competition. Requires: a git repo."
license: MIT
metadata:
version: 2.1.2
author: Alireza Rezvani
category: engineering
updated: 2026-03-17
---
# AgentHub — Multi-Agent Collaboration
Spawn N parallel AI agents that compete on the same task. Each agent works in an isolated git worktree. The coordinator evaluates results and merges the winner.
## Slash Commands
| Command | Description |
|---------|-------------|
| `/hub:init` | Create a new collaboration session — task, agent count, eval criteria |
| `/hub:spawn` | Launch N parallel subagents in isolated worktrees |
| `/hub:status` | Show DAG state, agent progress, branch status |
| `/hub:eval` | Rank agent results by metric or LLM judge |
| `/hub:merge` | Merge winning branch, archive losers |
| `/hub:board` | Read/write the agent message board |
| `/hub:run` | One-shot lifecycle: init → baseline → spawn → eval → merge |
## Agent Templates
When spawning with `--template`, agents follow a predefined iteration pattern:
| Template | Pattern | Use Case |
|----------|---------|----------|
| `optimizer` | Edit → eval → keep/discard → repeat x10 | Performance, latency, size |
| `refactorer` | Restructure → test → iterate until green | Code quality, tech debt |
| `test-writer` | Write tests → measure coverage → repeat | Test coverage gaps |
| `bug-fixer` | Reproduce → diagnose → fix → verify | Bug fix approaches |
Templates are defined in `references/agent-templates.md`.
## When This Skill Activates
Trigger phrases:
- "try multiple approaches"
- "have agents compete"
- "parallel optimization"
- "spawn N agents"
- "compare different solutions"
- "fan-out" or "tournament"
- "generate content variations"
- "compare different drafts"
- "A/B test copy"
- "explore multiple strategies"
## Coordinator Protocol
The main Claude Code session is the coordinator. It follows this lifecycle:
```
INIT → DISPATCH → MONITOR → EVALUATE → MERGE
```
### 1. Init
Run `/hub:init` to create a session. This generates:
- `.agenthub/sessions/{session-id}/config.yaml` — task config
- `.agenthub/sessions/{session-id}/state.json` — state machine
- `.agenthub/board/` — message board channels
### 2. Dispatch
Run `/hub:spawn` to launch agents. For each agent 1..N:
- Post task assignment to `.agenthub/board/dispatch/`
- Spawn via Agent tool with `isolation: "worktree"`
- All agents launched in a single message (parallel)
### 3. Monitor
Run `/hub:status` to check progress:
- `dag_analyzer.py --status --session {id}` shows branch state
- Board `progress/` channel has agent updates
### 4. Evaluate
Run `/hub:eval` to rank results:
- **Metric mode**: run eval command in each worktree, parse numeric result
- **Judge mode**: read diffs, coordinator ranks by quality
- **Hybrid**: metric first, LLM-judge for ties
### 5. Merge
Run `/hub:merge` to finalize:
- `git merge --no-ff` winner into base branch
- Tag losers: `git tag hub/archive/{session}/agent-{i}`
- Clean up worktrees
- Post merge summary to board
## Agent Protocol
Each subagent receives this prompt pattern:
```
You are agent-{i} in hub session {session-id}.
Your task: {task description}
Instructions:
1. Read your assignment at .agenthub/board/dispatch/{seq}-agent-{i}.md
2. Work in your worktree — make changes, run tests, iterate
3. Commit all changes with descriptive messages
4. Write your result summary to .agenthub/board/results/agent-{i}-result.md
5. Exit when done
```
Agents do NOT see each other's work. They do NOT communicate with each other. They only write to the board for the coordinator to read.
## DAG Model
### Branch Naming
```
hub/{session-id}/agent-{N}/attempt-{M}
```
- Session ID: timestamp-based (`YYYYMMDD-HHMMSS`)
- Agent N: sequential (1 to agent-count)
- Attempt M: increments on retry (usually 1)
### Frontier Detection
Frontier = branch tips with no child branches. Equivalent to AgentHub's "leaves" query.
```bash
python scripts/dag_analyzer.py --frontier --session {id}
```
### Immutability
The DAG is append-only:
- Never rebase or force-push agent branches
- Never delete commits (only branch refs after archival)
- Every approach preserved via git tags
## Message Board
Location: `.agenthub/board/`
### Channels
| Channel | Writer | Reader | Purpose |
|---------|--------|--------|---------|
| `dispatch/` | Coordinator | Agents | Task assignments |
| `progress/` | Agents | Coordinator | Status updates |
| `results/` | Agents + Coordinator | All | Final results + merge summary |
### Post Format
```markdown
---
author: agent-1
timestamp: 2026-03-17T14:30:22Z
channel: results
parent: null
---
## Result Summary
- **Approach**: Replaced O(n²) sort with hash map
- **Files changed**: 3
- **Metric**: 142ms (baseline: 180ms, delta: -38ms)
- **Confidence**: High — all tests pass
```
### Board Rules
- Append-only: never edit or delete posts
- Unique filenames: `{seq:03d}-{author}-{timestamp}.md`
- YAML frontmatter required on all posts
## Evaluation Modes
### Metric-Based
Best for: benchmarks, test pass rates, file sizes, response times.
```bash
python scripts/result_ranker.py --session {id} \
--eval-cmd "pytest bench.py --json" \
--metric p50_ms --direction lower
```
The ranker runs the eval command in each agent's worktree directory and parses the metric from stdout.
### LLM Judge
Best for: code quality, readability, architecture decisions.
The coordinator reads each agent's diff (`git diff base...agent-branch`) and ranks by:
1. Correctness (does it solve the task?)
2. Simplicity (fewer lines changed preferred)
3. Quality (clean execution, good structure)
### Hybrid
Run metric first. If top agents are within 10% of each other, use LLM judge to break ties.
## Session Lifecycle
```
init → running → evaluating → merged
→ archived (if no winner)
```
State transitions managed by `session_manager.py`:
| From | To | Trigger |
|------|----|---------|
| `init` | `running` | `/hub:spawn` completes |
| `running` | `evaluating` | All agents return |
| `evaluating` | `merged` | `/hub:merge` completes |
| `evaluating` | `archived` | No winner / all failed |
## Proactive Triggers
The coordinator should act when:
| Signal | Action |
|--------|--------|
| All agents crashed | Post failure summary, suggest retry with different constraints |
| No improvement over baseline | Archive session, suggest different approaches |
| Orphan worktrees detected | Run `session_manager.py --cleanup {id}` |
| Session stuck in `running` | Check board for progress, consider timeout |
## Installation
```bash
# Copy to your Claude Code skills directory
cp -r engineering/agenthub ~/.claude/skills/agenthub
# Or install via ClawHub
clawhub install agenthub
```
## Scripts
| Script | Purpose |
|--------|---------|
| `hub_init.py` | Initialize `.agenthub/` structure and session |
| `dag_analyzer.py` | Frontier detection, DAG graph, branch status |
| `board_manager.py` | Message board CRUD (channels, posts, threads) |
| `result_ranker.py` | Rank agents by metric or diff quality |
| `session_manager.py` | Session state machine and cleanup |
## Related Skills
- **autoresearch-agent** — Single-agent optimization loop (use AgentHub when you want N agents competing)
- **self-improving-agent** — Self-modifying agent (use AgentHub when you want external competition)
- **git-worktree-manager** — Git worktree utilities (AgentHub uses worktrees internally)
FILE:references/agent-templates.md
# Agent Templates
Predefined dispatch prompt templates for `/hub:spawn --template <name>`. Each template defines the iteration pattern agents follow in their worktrees.
## optimizer
**Use case:** Performance optimization, latency reduction, file size reduction, memory usage, content quality, conversion rate, research thoroughness.
**Dispatch prompt:**
```
You are agent-{i} in hub session {session-id}.
Your optimization strategy: {strategy}
Target: {task}
Eval command: {eval_cmd}
Metric: {metric} (direction: {direction})
Baseline: {baseline}
Follow this iteration loop (repeat up to 10 times):
1. Make ONE focused change to the target file(s) following your strategy
2. Run the eval command: {eval_cmd}
3. Extract the metric: {metric}
4. If improved over your previous best → git add . && git commit -m "improvement: {description}"
5. If NOT improved → git checkout -- .
6. Post progress update to .agenthub/board/progress/agent-{i}-iter-{n}.md
Include: iteration number, metric value, delta from baseline, what you tried
After all iterations, post your final metric to .agenthub/board/results/agent-{i}-result.md
Include: best metric achieved, total improvement from baseline, approach summary, files changed.
Constraints:
- Do NOT access other agents' work or results
- Commit early — each improvement is a separate commit
- If 3 consecutive iterations show no improvement, try a different angle within your strategy
- Always leave the code in a working state (tests must pass)
```
**Strategy assignment:** The coordinator assigns each agent a different strategy. For 3 agents optimizing latency, example strategies:
- Agent 1: Caching — add memoization, HTTP caching headers, query result caching
- Agent 2: Algorithm optimization — reduce complexity, better data structures, eliminate redundant work
- Agent 3: I/O batching — batch database queries, parallel I/O, connection pooling
**Cross-domain example** (3 agents writing landing page copy):
- Agent 1: Benefit-led — open with the top 3 user benefits, feature details below
- Agent 2: Social proof — lead with testimonials and case study stats, then features
- Agent 3: Urgency/scarcity — limited-time offer framing, countdown CTA, FOMO triggers
---
## refactorer
**Use case:** Code quality improvement, tech debt reduction, module restructuring.
**Dispatch prompt:**
```
You are agent-{i} in hub session {session-id}.
Your refactoring approach: {strategy}
Target: {task}
Test command: {eval_cmd}
Follow this iteration loop:
1. Identify the next refactoring opportunity following your approach
2. Make the change — keep each change small and focused
3. Run the test suite: {eval_cmd}
4. If tests pass → git add . && git commit -m "refactor: {description}"
5. If tests fail → git checkout -- . and try a different approach
6. Post progress update to .agenthub/board/progress/agent-{i}-iter-{n}.md
Include: what you refactored, tests status, lines changed
Continue until no more refactoring opportunities exist for your approach, or 10 iterations.
Post your final summary to .agenthub/board/results/agent-{i}-result.md
Include: total changes, test results, code quality improvements, files touched.
Constraints:
- Do NOT access other agents' work or results
- Every commit must leave tests green
- Preserve public API contracts — no breaking changes
- Prefer smaller, well-tested changes over large rewrites
```
**Strategy assignment:** Example strategies for 3 refactoring agents:
- Agent 1: Extract and simplify — break large functions into smaller ones, reduce nesting
- Agent 2: Type safety — add type annotations, replace Any types, fix type errors
- Agent 3: DRY — eliminate duplication, extract shared utilities, consolidate patterns
**Cross-domain example** (restructuring a research report):
- Agent 1: Executive summary first — lead with conclusions, supporting data below
- Agent 2: Narrative flow — problem → analysis → findings → recommendations arc
- Agent 3: Visual-first — diagrams and data tables up front, prose as annotation
---
## test-writer
**Use case:** Increasing test coverage, testing untested modules, edge case coverage.
**Dispatch prompt:**
```
You are agent-{i} in hub session {session-id}.
Your testing focus: {strategy}
Target: {task}
Coverage command: {eval_cmd}
Metric: {metric} (direction: {direction})
Baseline coverage: {baseline}
Follow this iteration loop (repeat up to 10 times):
1. Identify the next uncovered code path in your focus area
2. Write tests that exercise that path
3. Run the coverage command: {eval_cmd}
4. Extract coverage metric: {metric}
5. If coverage increased → git add . && git commit -m "test: {description}"
6. If coverage unchanged or tests fail → git checkout -- . and target a different path
7. Post progress update to .agenthub/board/progress/agent-{i}-iter-{n}.md
Include: iteration number, coverage value, delta from baseline, what was tested
After all iterations, post your final coverage to .agenthub/board/results/agent-{i}-result.md
Include: final coverage, improvement from baseline, number of new tests, modules covered.
Constraints:
- Do NOT access other agents' work or results
- Tests must be meaningful — no trivially passing assertions
- Each test file must be self-contained and runnable independently
- Prefer testing behavior over implementation details
```
**Strategy assignment:** Example strategies for 3 test-writing agents:
- Agent 1: Happy path coverage — cover main use cases and expected inputs
- Agent 2: Edge cases — boundary values, empty inputs, error conditions
- Agent 3: Integration tests — test module interactions, API endpoints, data flows
---
## bug-fixer
**Use case:** Fixing bugs with competing diagnostic approaches, reproducing and resolving issues.
**Dispatch prompt:**
```
You are agent-{i} in hub session {session-id}.
Your diagnostic approach: {strategy}
Bug description: {task}
Verification command: {eval_cmd}
Follow this process:
1. Reproduce the bug — run the verification command to confirm it fails
2. Diagnose the root cause using your approach: {strategy}
3. Implement a fix — make the minimal change needed
4. Run the verification command: {eval_cmd}
5. If the bug is fixed AND no regressions → git add . && git commit -m "fix: {description}"
6. If NOT fixed → git checkout -- . and try a different angle
7. Repeat steps 2-6 up to 5 times with different hypotheses
Post your result to .agenthub/board/results/agent-{i}-result.md
Include: root cause identified, fix applied, verification results, confidence level, files changed.
Constraints:
- Do NOT access other agents' work or results
- Minimal changes only — fix the bug, don't refactor surrounding code
- Every commit must include a test that would have caught the bug
- If you cannot reproduce the bug, document your findings and exit
```
**Strategy assignment:** Example strategies for 3 bug-fixing agents:
- Agent 1: Top-down — trace from the error message/stack trace back to root cause
- Agent 2: Bottom-up — examine recent changes, bisect commits, find the introducing change
- Agent 3: Isolation — write a minimal reproduction, narrow down the failing component
---
## Using Templates
When `/hub:spawn` is called with `--template <name>`:
1. Load the template from this file
2. Replace `{variables}` with session config values
3. For each agent, replace `{strategy}` with the assigned strategy
4. Use the filled template as the dispatch prompt instead of the default prompt
Strategy assignment is automatic: the coordinator generates N different strategies appropriate to the template and task, assigning one per agent. The coordinator should choose strategies that are **diverse** — overlapping strategies waste agents.
FILE:references/coordination-strategies.md
# Multi-Agent Coordination Strategies
## Patterns
### Fan-Out / Fan-In
The simplest and most common pattern. One coordinator dispatches the same task to N agents, waits for all to complete, then evaluates.
```
┌─ Agent 1 ─┐
Task ──> ├─ Agent 2 ─┤ ──> Evaluate ──> Merge Winner
└─ Agent 3 ─┘
```
**When to use**: Optimization tasks, competitive solutions, exploring diverse approaches, competing content drafts, vendor evaluation.
**Agent count**: 2-5 (diminishing returns beyond 5 for most tasks).
**Eval**: Metric-based preferred. LLM judge for subjective quality.
### Tournament
Multiple rounds of fan-out/fan-in. Losers are eliminated, winners advance. Each round can refine the task or increase difficulty.
```
Round 1: A1, A2, A3, A4 → Eval → A2, A4 advance
Round 2: A2, A4 → Eval → A2 wins
```
**When to use**: Complex optimization where iterative refinement helps. Each round builds on the previous winner.
**Implementation**:
1. Run `/hub:init` + `/hub:spawn` for round 1
2. Eval, merge winner into a new base branch
3. Run `/hub:init` again with the merged branch as base
4. Repeat until convergence or budget exhausted
### Ensemble
All agents' work is combined rather than selecting a winner. Useful when agents solve different parts of a problem.
```
Agent 1: solves auth module
Agent 2: solves API routes ──> Cherry-pick all ──> Combined result
Agent 3: solves database layer
```
**When to use**: Large tasks that decompose into independent subtasks. Each agent gets a different piece.
**Implementation**:
1. In `/hub:init`, give each agent a DIFFERENT task (subtask of the whole)
2. Spawn with unique dispatch posts per agent
3. Instead of `/hub:eval` ranking, manually cherry-pick from each
4. Or merge sequentially: merge agent-1, then merge agent-2 on top
### Pipeline
Agents work sequentially — each builds on the previous agent's output. Like a relay race.
```
Agent 1 (design) → Agent 2 (implement) → Agent 3 (test) → Agent 4 (optimize)
```
**When to use**: Tasks with natural phases (design → implement → test). Each phase needs different expertise.
**Implementation**:
1. Spawn agent-1 alone, wait for completion
2. Merge agent-1's work, spawn agent-2 from that base
3. Repeat for each pipeline stage
4. Each agent reads the previous agent's result post for context
## Agent Configuration
### Task Decomposition
For fan-out, all agents get the same task. But you can add variation:
| Strategy | Dispatch Difference | Use Case |
|----------|-------------------|----------|
| **Identical** | Same prompt to all | Pure competition |
| **Constrained** | Same goal, different constraints | "Use caching" vs "Use indexing" |
| **Seeded** | Same goal, different starting hints | Explore different parts of solution space |
| **Role-varied** | Same goal, different personas | "As a performance engineer" vs "As a DBA" |
### Agent Count Guidelines
| Task Complexity | Agents | Rationale |
|----------------|--------|-----------|
| Simple optimization | 2 | Two approaches is usually enough |
| Medium complexity | 3 | Three diverse approaches, manageable eval |
| Complex / creative | 4-5 | More exploration, but eval cost increases |
| Subtask decomposition | N = subtasks | One agent per subtask (ensemble pattern) |
## Evaluation Strategies
### Metric-Based (Objective)
Best when a clear numeric metric exists:
| Metric Type | Example | Direction |
|-------------|---------|-----------|
| Latency | p50_ms, p99_ms | lower |
| Throughput | rps, qps | higher |
| Size | bundle_kb, image_bytes | lower |
| Score | test_pass_rate, accuracy | higher |
| Count | error_count, warnings | lower |
| Word count | word_count | higher |
| Readability | flesch_score | higher |
| Conversion | cta_click_rate | higher |
### LLM Judge (Subjective)
Best when quality is subjective or multi-dimensional:
Judging criteria (in order of importance):
1. **Correctness** — Does it solve the stated task?
2. **Completeness** — Does it handle edge cases?
3. **Simplicity** — Fewer lines changed = less risk
4. **Quality** — Clean execution, good structure, no anti-patterns
5. **Performance** — Efficient algorithms and data structures
### Hybrid
1. Run metric eval to get objective ranking
2. If top-2 agents are within 10% of each other, use LLM judge
3. Weight: 70% metric, 30% qualitative
## Failure Handling
### All Agents Fail
```
Signal: All agents return errors or no improvement
Action:
1. Post failure summary to board
2. Archive session (state → archived)
3. Suggest: "Try with different constraints, more agents, or simplified task"
4. Do NOT auto-retry without user approval
```
### Partial Failure
```
Signal: Some agents fail, others succeed
Action:
1. Evaluate only successful agents
2. Note failures in eval summary
3. Proceed with merge if any agent succeeded
```
### No Improvement
```
Signal: All agents complete but none improve on baseline
Action:
1. Show results with negative deltas
2. Suggest: "Current implementation may already be near-optimal"
3. Archive session
```
## Communication Protocol
### Board Usage by Phase
| Phase | Channel | Content |
|-------|---------|---------|
| Dispatch | `dispatch/` | Task assignment per agent |
| Working | `progress/` | Agent status updates (optional) |
| Complete | `results/` | Final result summary per agent |
| Merge | `results/` | Merge summary from coordinator |
### Result Post Template
Agents should write results in this format:
```markdown
## Result Summary
- **Approach**: {one-line description of strategy}
- **Files changed**: {count}
- **Key changes**: {bullet list of main modifications}
- **Metric**: {value} (baseline: {baseline}, delta: {delta})
- **Tests**: {pass/fail status}
- **Confidence**: {High/Medium/Low} — {reason}
- **Limitations**: {known issues or edge cases}
```
FILE:references/dag-patterns.md
# Git DAG Patterns for Multi-Agent Collaboration
## Core Concepts
### Directed Acyclic Graph (DAG)
Git's commit history is a DAG where:
- Each commit points to one or more parents
- No cycles exist (you can't be your own ancestor)
- Branches are just pointers to commit nodes
In AgentHub, the DAG represents all approaches ever tried:
- Base commit = task starting point
- Each agent creates a branch from the base
- Commits on each branch = incremental progress
- Frontier = branch tips with no children
### Frontier Detection
The **frontier** is the set of commits (branch tips) that have no children. These are the "leaves" of the DAG — the latest state of each agent's work.
Algorithm:
```
1. Collect all branch tips: T = {tip(b) for b in hub_branches}
2. For each tip t in T:
a. Check if t is an ancestor of any other tip t' in T
b. If yes: t is NOT on the frontier (it's been extended)
c. If no: t IS on the frontier
3. Return frontier set
```
Git command equivalent:
```bash
# For each branch, check if it's an ancestor of any other
git merge-base --is-ancestor <commit-a> <commit-b>
```
### Branch Naming Convention
```
hub/{session-id}/agent-{N}/attempt-{M}
```
Components:
- `session-id`: YYYYMMDD-HHMMSS timestamp (unique per session)
- `agent-N`: Sequential agent number (1 to agent-count)
- `attempt-M`: Retry counter (starts at 1, increments on re-spawn)
This creates a natural namespace:
- `hub/*` — all AgentHub work
- `hub/{session}/*` — all work for one session
- `hub/{session}/agent-{N}/*` — all attempts by one agent
## Merge Strategies
### No-Fast-Forward Merge (Default)
```bash
git merge --no-ff hub/{session}/agent-{N}/attempt-1
```
Creates a merge commit that:
- Preserves the branch topology in the DAG
- Makes it clear which commits came from which agent
- Allows `git log --first-parent` to show only merge points
### Squash Merge (Alternative)
```bash
git merge --squash hub/{session}/agent-{N}/attempt-1
```
Use when:
- Agent made many small commits that aren't individually meaningful
- Clean history is preferred over detailed history
- The approach matters, not the journey
### Cherry-Pick (Selective)
```bash
git cherry-pick <specific-commits>
```
Use when:
- Only some of an agent's commits are wanted
- Combining work from multiple agents
- The agent solved a bonus problem along the way
## Archive Strategy
After merging the winner, losers are archived via tags:
```bash
# Create archive tag
git tag hub/archive/{session}/agent-{N} hub/{session}/agent-{N}/attempt-1
# Delete branch ref
git branch -D hub/{session}/agent-{N}/attempt-1
```
Why tags instead of branches:
- Tags are immutable (can't be moved or accidentally pushed to)
- Tags don't clutter `git branch --list` output
- Tags are still reachable by `git log` and `git show`
- Git GC won't collect tagged commits
## Immutability Rules
1. **Never rebase agent branches** — rewrites history, breaks DAG
2. **Never force-push** — could overwrite other agents' work
3. **Never delete commits** — only delete branch refs (commits preserved via tags)
4. **Never amend** agent commits — append-only history
5. **Board is append-only** — new posts only, no edits
## DAG Visualization
Use `git log` flags to see the multi-agent DAG:
```bash
# Full graph with branch decoration
git log --all --oneline --graph --decorate --branches=hub/*
# Commits since base, all agents
git log --all --oneline --graph base..HEAD --branches=hub/{session}/*
# Per-agent linear history
git log --oneline hub/{session}/agent-1/attempt-1
```
## Worktree Isolation
Git worktrees provide filesystem isolation:
```bash
# Create worktree for an agent
git worktree add /tmp/hub-agent-1 -b hub/{session}/agent-1/attempt-1
# List active worktrees
git worktree list
# Remove after merge
git worktree remove /tmp/hub-agent-1
```
Key properties:
- Each worktree has its own working directory and index
- All worktrees share the same `.git` object store
- Commits in one worktree are immediately visible in another
- Cannot check out the same branch in two worktrees
FILE:scripts/board_manager.py
#!/usr/bin/env python3
"""AgentHub message board manager.
CRUD operations for the agent message board: list channels, read posts,
create new posts, and reply to threads.
Usage:
python board_manager.py --list
python board_manager.py --read dispatch
python board_manager.py --post --channel results --author agent-1 --message "Task complete"
python board_manager.py --thread 001-agent-1 --message "Additional details"
python board_manager.py --demo
"""
import argparse
import json
import os
import re
import sys
from datetime import datetime, timezone
BOARD_PATH = ".agenthub/board"
def get_board_path():
"""Get the board directory path."""
if not os.path.isdir(BOARD_PATH):
print(f"Error: Board not found at {BOARD_PATH}. Run hub_init.py first.",
file=sys.stderr)
sys.exit(1)
return BOARD_PATH
def load_index():
"""Load the board index."""
index_path = os.path.join(get_board_path(), "_index.json")
if not os.path.exists(index_path):
return {"channels": ["dispatch", "progress", "results"], "counters": {}}
with open(index_path) as f:
return json.load(f)
def save_index(index):
"""Save the board index."""
index_path = os.path.join(get_board_path(), "_index.json")
with open(index_path, "w") as f:
json.dump(index, f, indent=2)
f.write("\n")
def list_channels(output_format="text"):
"""List all board channels with post counts."""
index = load_index()
channels = []
for ch in index.get("channels", []):
ch_path = os.path.join(get_board_path(), ch)
count = 0
if os.path.isdir(ch_path):
count = len([f for f in os.listdir(ch_path)
if f.endswith(".md")])
channels.append({"channel": ch, "posts": count})
if output_format == "json":
print(json.dumps({"channels": channels}, indent=2))
else:
print("Board Channels:")
print()
for ch in channels:
print(f" {ch['channel']:<15} {ch['posts']} posts")
def parse_post_frontmatter(content):
"""Parse YAML frontmatter from a post."""
metadata = {}
body = content
if content.startswith("---"):
parts = content.split("---", 2)
if len(parts) >= 3:
fm = parts[1].strip()
body = parts[2].strip()
for line in fm.split("\n"):
if ":" in line:
key, val = line.split(":", 1)
metadata[key.strip()] = val.strip()
return metadata, body
def read_channel(channel, output_format="text"):
"""Read all posts in a channel."""
ch_path = os.path.join(get_board_path(), channel)
if not os.path.isdir(ch_path):
print(f"Error: Channel '{channel}' not found", file=sys.stderr)
sys.exit(1)
files = sorted([f for f in os.listdir(ch_path) if f.endswith(".md")])
posts = []
for fname in files:
filepath = os.path.join(ch_path, fname)
with open(filepath) as f:
content = f.read()
metadata, body = parse_post_frontmatter(content)
posts.append({
"file": fname,
"metadata": metadata,
"body": body,
})
if output_format == "json":
print(json.dumps({"channel": channel, "posts": posts}, indent=2))
else:
print(f"Channel: {channel} ({len(posts)} posts)")
print("=" * 60)
for post in posts:
author = post["metadata"].get("author", "unknown")
timestamp = post["metadata"].get("timestamp", "")
print(f"\n--- {post['file']} (by {author}, {timestamp}) ---")
print(post["body"])
def create_post(channel, author, message, parent=None):
"""Create a new post in a channel."""
ch_path = os.path.join(get_board_path(), channel)
os.makedirs(ch_path, exist_ok=True)
# Get next sequence number
index = load_index()
counters = index.get("counters", {})
seq = counters.get(channel, 0) + 1
counters[channel] = seq
index["counters"] = counters
save_index(index)
# Generate filename
timestamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
safe_author = re.sub(r"[^a-zA-Z0-9_-]", "", author)
filename = f"{seq:03d}-{safe_author}-{timestamp}.md"
# Build post content
lines = [
"---",
f"author: {author}",
f"timestamp: {datetime.now(timezone.utc).isoformat()}",
f"channel: {channel}",
f"sequence: {seq}",
]
if parent:
lines.append(f"parent: {parent}")
else:
lines.append("parent: null")
lines.append("---")
lines.append("")
lines.append(message)
lines.append("")
filepath = os.path.join(ch_path, filename)
with open(filepath, "w") as f:
f.write("\n".join(lines))
print(f"Posted to {channel}/{filename}")
return filename
def run_demo():
"""Show demo output."""
print("=" * 60)
print("AgentHub Board Manager — Demo Mode")
print("=" * 60)
print()
print("--- Channel List ---")
print("Board Channels:")
print()
print(" dispatch 2 posts")
print(" progress 4 posts")
print(" results 3 posts")
print()
print("--- Read Channel: results ---")
print("Channel: results (3 posts)")
print("=" * 60)
print()
print("--- 001-agent-1-20260317T143510Z.md (by agent-1, 2026-03-17T14:35:10Z) ---")
print("## Result Summary")
print()
print("- **Approach**: Added caching layer for database queries")
print("- **Files changed**: 3")
print("- **Metric**: 165ms (baseline: 180ms, delta: -15ms)")
print("- **Confidence**: Medium — 2 edge cases not covered")
print()
print("--- 002-agent-2-20260317T143645Z.md (by agent-2, 2026-03-17T14:36:45Z) ---")
print("## Result Summary")
print()
print("- **Approach**: Replaced O(n²) sort with hash map lookup")
print("- **Files changed**: 2")
print("- **Metric**: 142ms (baseline: 180ms, delta: -38ms)")
print("- **Confidence**: High — all tests pass")
print()
print("--- 003-agent-3-20260317T143422Z.md (by agent-3, 2026-03-17T14:34:22Z) ---")
print("## Result Summary")
print()
print("- **Approach**: Minor loop optimizations")
print("- **Files changed**: 1")
print("- **Metric**: 190ms (baseline: 180ms, delta: +10ms)")
print("- **Confidence**: Low — no meaningful improvement")
def main():
parser = argparse.ArgumentParser(
description="AgentHub message board manager"
)
parser.add_argument("--list", action="store_true",
help="List all channels with post counts")
parser.add_argument("--read", type=str, metavar="CHANNEL",
help="Read all posts in a channel")
parser.add_argument("--post", action="store_true",
help="Create a new post")
parser.add_argument("--channel", type=str,
help="Channel for --post or --thread")
parser.add_argument("--author", type=str,
help="Author name for --post")
parser.add_argument("--message", type=str,
help="Message content for --post or --thread")
parser.add_argument("--thread", type=str, metavar="POST_ID",
help="Reply to a post (sets parent)")
parser.add_argument("--format", choices=["text", "json"], default="text",
help="Output format (default: text)")
parser.add_argument("--demo", action="store_true",
help="Show demo output")
args = parser.parse_args()
if args.demo:
run_demo()
return
if args.list:
list_channels(args.format)
return
if args.read:
read_channel(args.read, args.format)
return
if args.post:
if not args.channel or not args.author or not args.message:
print("Error: --post requires --channel, --author, and --message",
file=sys.stderr)
sys.exit(1)
create_post(args.channel, args.author, args.message)
return
if args.thread:
if not args.message:
print("Error: --thread requires --message", file=sys.stderr)
sys.exit(1)
channel = args.channel or "results"
author = args.author or "coordinator"
create_post(channel, author, args.message, parent=args.thread)
return
parser.print_help()
if __name__ == "__main__":
main()
FILE:scripts/dag_analyzer.py
#!/usr/bin/env python3
"""Analyze the AgentHub git DAG.
Detects frontier branches (leaves with no children), displays DAG graphs,
and shows per-agent branch status for a session.
Usage:
python dag_analyzer.py --frontier --session 20260317-143022
python dag_analyzer.py --graph
python dag_analyzer.py --status --session 20260317-143022
python dag_analyzer.py --demo
"""
import argparse
import json
import os
import re
import subprocess
import sys
from datetime import datetime
def run_git(*args):
"""Run a git command and return stdout."""
try:
result = subprocess.run(
["git"] + list(args),
capture_output=True, text=True, check=True
)
return result.stdout.strip()
except subprocess.CalledProcessError as e:
print(f"Git error: {e.stderr.strip()}", file=sys.stderr)
return ""
def get_hub_branches(session_id=None):
"""Get all hub/* branches, optionally filtered by session."""
output = run_git("branch", "--list", "hub/*", "--format=%(refname:short)")
if not output:
return []
branches = output.strip().split("\n")
if session_id:
prefix = f"hub/{session_id}/"
branches = [b for b in branches if b.startswith(prefix)]
return branches
def get_branch_commit(branch):
"""Get the commit hash for a branch."""
return run_git("rev-parse", "--short", branch)
def get_branch_commit_count(branch, base_branch="main"):
"""Count commits ahead of base branch."""
output = run_git("rev-list", "--count", f"{base_branch}..{branch}")
try:
return int(output)
except ValueError:
return 0
def get_branch_last_commit_date(branch):
"""Get the last commit date for a branch."""
output = run_git("log", "-1", "--format=%ci", branch)
if output:
return output[:19]
return "unknown"
def get_branch_last_commit_msg(branch):
"""Get the last commit message for a branch."""
return run_git("log", "-1", "--format=%s", branch)
def detect_frontier(session_id=None):
"""Find frontier branches (tips with no child branches).
A branch is on the frontier if no other hub branch contains its tip commit
as an ancestor (i.e., it has no children in the DAG).
"""
branches = get_hub_branches(session_id)
if not branches:
return []
# Get commit hashes for all branches
branch_commits = {}
for b in branches:
commit = run_git("rev-parse", b)
if commit:
branch_commits[b] = commit
# A branch is frontier if its commit is not an ancestor of any other branch
frontier = []
for branch, commit in branch_commits.items():
is_ancestor = False
for other_branch, other_commit in branch_commits.items():
if other_branch == branch:
continue
# Check if commit is ancestor of other_commit
result = subprocess.run(
["git", "merge-base", "--is-ancestor", commit, other_commit],
capture_output=True
)
if result.returncode == 0:
is_ancestor = True
break
if not is_ancestor:
frontier.append(branch)
return frontier
def show_graph():
"""Display the git DAG graph for hub branches."""
branches = get_hub_branches()
if not branches:
print("No hub/* branches found.")
return
# Use git log with graph for hub branches
branch_args = [b for b in branches]
output = run_git(
"log", "--all", "--oneline", "--graph", "--decorate",
"--simplify-by-decoration",
*[f"--branches=hub/*"]
)
if output:
print(output)
else:
print("No hub commits found.")
def show_status(session_id, output_format="table"):
"""Show per-agent branch status for a session."""
branches = get_hub_branches(session_id)
if not branches:
print(f"No branches found for session {session_id}")
return
frontier = detect_frontier(session_id)
# Parse agent info from branch names
agents = []
for branch in sorted(branches):
# Pattern: hub/{session}/agent-{N}/attempt-{M}
match = re.match(r"hub/[^/]+/agent-(\d+)/attempt-(\d+)", branch)
if match:
agent_num = int(match.group(1))
attempt = int(match.group(2))
else:
agent_num = 0
attempt = 1
commit = get_branch_commit(branch)
commits = get_branch_commit_count(branch)
last_date = get_branch_last_commit_date(branch)
last_msg = get_branch_last_commit_msg(branch)
is_frontier = branch in frontier
agents.append({
"agent": agent_num,
"attempt": attempt,
"branch": branch,
"commit": commit,
"commits_ahead": commits,
"last_update": last_date,
"last_message": last_msg,
"frontier": is_frontier,
})
if output_format == "json":
print(json.dumps({"session": session_id, "agents": agents}, indent=2))
return
# Table output
print(f"Session: {session_id}")
print(f"Branches: {len(branches)} | Frontier: {len(frontier)}")
print()
header = f"{'AGENT':<8} {'BRANCH':<45} {'COMMITS':<8} {'STATUS':<10} {'LAST UPDATE':<20}"
print(header)
print("-" * len(header))
for a in agents:
status = "frontier" if a["frontier"] else "merged"
print(f"agent-{a['agent']:<4} {a['branch']:<45} {a['commits_ahead']:<8} {status:<10} {a['last_update']:<20}")
def run_demo():
"""Show demo output."""
print("=" * 60)
print("AgentHub DAG Analyzer — Demo Mode")
print("=" * 60)
print()
print("--- Frontier Detection ---")
print("Frontier branches (leaves with no children):")
print(" hub/20260317-143022/agent-1/attempt-1 (3 commits ahead)")
print(" hub/20260317-143022/agent-2/attempt-1 (5 commits ahead)")
print(" hub/20260317-143022/agent-3/attempt-1 (2 commits ahead)")
print()
print("--- Session Status ---")
print("Session: 20260317-143022")
print("Branches: 3 | Frontier: 3")
print()
header = f"{'AGENT':<8} {'BRANCH':<45} {'COMMITS':<8} {'STATUS':<10} {'LAST UPDATE':<20}"
print(header)
print("-" * len(header))
print(f"{'agent-1':<8} {'hub/20260317-143022/agent-1/attempt-1':<45} {'3':<8} {'frontier':<10} {'2026-03-17 14:35:10':<20}")
print(f"{'agent-2':<8} {'hub/20260317-143022/agent-2/attempt-1':<45} {'5':<8} {'frontier':<10} {'2026-03-17 14:36:45':<20}")
print(f"{'agent-3':<8} {'hub/20260317-143022/agent-3/attempt-1':<45} {'2':<8} {'frontier':<10} {'2026-03-17 14:34:22':<20}")
print()
print("--- DAG Graph ---")
print("* abc1234 (hub/20260317-143022/agent-2/attempt-1) Replaced O(n²) with hash map")
print("* def5678 Added benchmark tests")
print("| * ghi9012 (hub/20260317-143022/agent-1/attempt-1) Added caching layer")
print("| * jkl3456 Refactored data access")
print("|/")
print("| * mno7890 (hub/20260317-143022/agent-3/attempt-1) Minor optimizations")
print("|/")
print("* pqr1234 (dev) Base commit")
def main():
parser = argparse.ArgumentParser(
description="Analyze the AgentHub git DAG"
)
parser.add_argument("--frontier", action="store_true",
help="List frontier branches (leaves with no children)")
parser.add_argument("--graph", action="store_true",
help="Show ASCII DAG graph for hub branches")
parser.add_argument("--status", action="store_true",
help="Show per-agent branch status")
parser.add_argument("--session", type=str,
help="Filter by session ID")
parser.add_argument("--format", choices=["table", "json"], default="table",
help="Output format (default: table)")
parser.add_argument("--demo", action="store_true",
help="Show demo output")
args = parser.parse_args()
if args.demo:
run_demo()
return
if not any([args.frontier, args.graph, args.status]):
parser.print_help()
return
if args.frontier:
frontier = detect_frontier(args.session)
if args.format == "json":
print(json.dumps({"frontier": frontier}, indent=2))
else:
if frontier:
print("Frontier branches:")
for b in frontier:
print(f" {b}")
else:
print("No frontier branches found.")
print()
if args.graph:
show_graph()
print()
if args.status:
if not args.session:
print("Error: --session required with --status", file=sys.stderr)
sys.exit(1)
show_status(args.session, args.format)
if __name__ == "__main__":
main()
FILE:scripts/dry_run.py
#!/usr/bin/env python3
"""Dry-run validation for the AgentHub plugin.
Checks JSON validity, YAML frontmatter, markdown structure, cross-file
consistency, script --help, and referenced file existence — without
creating any sessions or worktrees.
Usage:
python dry_run.py # Run all checks
python dry_run.py --verbose # Show per-file details
python dry_run.py --help
"""
import argparse
import json
import os
import re
import subprocess
import sys
PLUGIN_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
# ── Helpers ──────────────────────────────────────────────────────────
PASS = "\033[32m✓\033[0m"
FAIL = "\033[31m✗\033[0m"
WARN = "\033[33m!\033[0m"
class Results:
def __init__(self):
self.passed = 0
self.failed = 0
self.warnings = 0
self.details = []
def ok(self, msg):
self.passed += 1
self.details.append((PASS, msg))
def fail(self, msg):
self.failed += 1
self.details.append((FAIL, msg))
def warn(self, msg):
self.warnings += 1
self.details.append((WARN, msg))
def print(self, verbose=False):
if verbose:
for icon, msg in self.details:
print(f" {icon} {msg}")
print()
total = self.passed + self.failed
status = "PASS" if self.failed == 0 else "FAIL"
color = "\033[32m" if self.failed == 0 else "\033[31m"
warn_str = f", {self.warnings} warnings" if self.warnings else ""
print(f"{color}{status}\033[0m {self.passed}/{total} checks passed{warn_str}")
return self.failed == 0
def rel(path):
"""Path relative to plugin root for display."""
return os.path.relpath(path, PLUGIN_ROOT)
# ── Check 1: JSON files ─────────────────────────────────────────────
def check_json(results):
"""Validate settings.json and plugin.json."""
json_files = [
os.path.join(PLUGIN_ROOT, "settings.json"),
os.path.join(PLUGIN_ROOT, ".claude-plugin", "plugin.json"),
]
for path in json_files:
name = rel(path)
if not os.path.exists(path):
results.fail(f"{name} — file missing")
continue
try:
with open(path) as f:
data = json.load(f)
results.ok(f"{name} — valid JSON")
except json.JSONDecodeError as e:
results.fail(f"{name} — invalid JSON: {e}")
continue
# plugin.json: only allowed fields
if name.endswith("plugin.json"):
allowed = {"name", "description", "version", "author", "homepage",
"repository", "license", "skills"}
extra = set(data.keys()) - allowed
if extra:
results.fail(f"{name} — disallowed fields: {extra}")
else:
results.ok(f"{name} — schema fields OK")
# Cross-check versions
try:
with open(json_files[0]) as f:
v1 = json.load(f).get("version")
with open(json_files[1]) as f:
v2 = json.load(f).get("version")
if v1 and v2 and v1 == v2:
results.ok(f"version match ({v1})")
elif v1 and v2:
results.fail(f"version mismatch: settings={v1}, plugin={v2}")
except Exception:
pass
# ── Check 2: YAML frontmatter ───────────────────────────────────────
FRONTMATTER_RE = re.compile(r"^---\n(.+?)\n---", re.DOTALL)
REQUIRED_FM_KEYS = {"name", "description"}
def check_frontmatter(results):
"""Validate YAML frontmatter in all SKILL.md files."""
skill_files = []
for root, _dirs, files in os.walk(PLUGIN_ROOT):
for f in files:
if f == "SKILL.md":
skill_files.append(os.path.join(root, f))
for path in skill_files:
name = rel(path)
with open(path) as f:
content = f.read()
m = FRONTMATTER_RE.match(content)
if not m:
results.fail(f"{name} — missing YAML frontmatter")
continue
# Lightweight key check (no PyYAML dependency)
fm_text = m.group(1)
found_keys = set()
for line in fm_text.splitlines():
if ":" in line:
key = line.split(":", 1)[0].strip()
found_keys.add(key)
missing = REQUIRED_FM_KEYS - found_keys
if missing:
results.fail(f"{name} — frontmatter missing keys: {missing}")
else:
results.ok(f"{name} — frontmatter OK")
# ── Check 3: Markdown structure ──────────────────────────────────────
def check_markdown(results):
"""Check for broken code fences and table rows in all .md files."""
md_files = []
for root, _dirs, files in os.walk(PLUGIN_ROOT):
for f in files:
if f.endswith(".md"):
md_files.append(os.path.join(root, f))
for path in md_files:
name = rel(path)
with open(path) as f:
lines = f.readlines()
# Code fences must be balanced
fence_count = sum(1 for ln in lines if ln.strip().startswith("```"))
if fence_count % 2 != 0:
results.fail(f"{name} — unbalanced code fences ({fence_count} found)")
else:
results.ok(f"{name} — code fences balanced")
# Tables: rows inside a table should have consistent pipe count
in_table = False
table_pipes = 0
table_ok = True
for i, ln in enumerate(lines, 1):
stripped = ln.strip()
if stripped.startswith("|") and stripped.endswith("|"):
pipes = stripped.count("|")
if not in_table:
in_table = True
table_pipes = pipes
elif pipes != table_pipes:
# Separator rows (|---|---| ) can differ slightly; skip
if not re.match(r"^\|[\s\-:|]+\|$", stripped):
results.warn(f"{name}:{i} — table column count mismatch ({pipes} vs {table_pipes})")
table_ok = False
else:
in_table = False
table_pipes = 0
# ── Check 4: Scripts --help ──────────────────────────────────────────
def check_scripts(results):
"""Verify every Python script exits 0 on --help."""
scripts_dir = os.path.join(PLUGIN_ROOT, "scripts")
if not os.path.isdir(scripts_dir):
results.warn("scripts/ directory not found")
return
for fname in sorted(os.listdir(scripts_dir)):
if not fname.endswith(".py") or fname == "dry_run.py":
continue
path = os.path.join(scripts_dir, fname)
try:
proc = subprocess.run(
[sys.executable, path, "--help"],
capture_output=True, text=True, timeout=10,
)
if proc.returncode == 0:
results.ok(f"scripts/{fname} --help exits 0")
else:
results.fail(f"scripts/{fname} --help exits {proc.returncode}")
except subprocess.TimeoutExpired:
results.fail(f"scripts/{fname} --help timed out")
except Exception as e:
results.fail(f"scripts/{fname} --help error: {e}")
# ── Check 5: Referenced files exist ──────────────────────────────────
def check_references(results):
"""Verify that key files referenced in docs actually exist."""
expected = [
"settings.json",
".claude-plugin/plugin.json",
"CLAUDE.md",
"SKILL.md",
"README.md",
"agents/hub-coordinator.md",
"references/agent-templates.md",
"references/coordination-strategies.md",
"scripts/hub_init.py",
"scripts/dag_analyzer.py",
"scripts/board_manager.py",
"scripts/result_ranker.py",
"scripts/session_manager.py",
]
for ref in expected:
path = os.path.join(PLUGIN_ROOT, ref)
if os.path.exists(path):
results.ok(f"{ref} exists")
else:
results.fail(f"{ref} — referenced but missing")
# ── Check 6: Cross-domain coverage ──────────────────────────────────
def check_cross_domain(results):
"""Verify non-engineering examples exist in key files (the whole point of this update)."""
checks = [
("settings.json", "content-generation"),
(".claude-plugin/plugin.json", "content drafts"),
("CLAUDE.md", "content drafts"),
("SKILL.md", "content variation"),
("README.md", "content generation"),
("skills/run/SKILL.md", "--judge"),
("skills/init/SKILL.md", "LLM judge"),
("skills/eval/SKILL.md", "narrative"),
("skills/board/SKILL.md", "Storytelling"),
("skills/status/SKILL.md", "Storytelling"),
("references/agent-templates.md", "landing page copy"),
("references/coordination-strategies.md", "flesch_score"),
("agents/hub-coordinator.md", "qualitative verdict"),
]
for filepath, needle in checks:
path = os.path.join(PLUGIN_ROOT, filepath)
if not os.path.exists(path):
results.fail(f"{filepath} — missing (cannot check cross-domain)")
continue
with open(path) as f:
content = f.read()
if needle.lower() in content.lower():
results.ok(f"{filepath} — contains cross-domain example (\"{needle}\")")
else:
results.fail(f"{filepath} — missing cross-domain marker \"{needle}\"")
# ── Main ─────────────────────────────────────────────────────────────
def main():
parser = argparse.ArgumentParser(
description="Dry-run validation for the AgentHub plugin."
)
parser.add_argument("--verbose", "-v", action="store_true",
help="Show per-file check details")
args = parser.parse_args()
print(f"AgentHub dry-run validation")
print(f"Plugin root: {PLUGIN_ROOT}\n")
all_ok = True
sections = [
("JSON validity", check_json),
("YAML frontmatter", check_frontmatter),
("Markdown structure", check_markdown),
("Script --help", check_scripts),
("Referenced files", check_references),
("Cross-domain examples", check_cross_domain),
]
for title, fn in sections:
print(f"── {title} ──")
r = Results()
fn(r)
ok = r.print(verbose=args.verbose)
if not ok:
all_ok = False
print()
if all_ok:
print("\033[32mAll checks passed.\033[0m")
else:
print("\033[31mSome checks failed — see above.\033[0m")
sys.exit(1)
if __name__ == "__main__":
main()
FILE:scripts/hub_init.py
#!/usr/bin/env python3
"""Initialize an AgentHub collaboration session.
Creates the .agenthub/ directory structure, generates a session ID,
and writes config.yaml and state.json for the session.
Usage:
python hub_init.py --task "Optimize API response time" --agents 3 \\
--eval "pytest bench.py --json" --metric p50_ms --direction lower
python hub_init.py --task "Refactor auth module" --agents 2
python hub_init.py --demo
"""
import argparse
import json
import os
import sys
from datetime import datetime, timezone
def generate_session_id():
"""Generate a timestamp-based session ID."""
return datetime.now().strftime("%Y%m%d-%H%M%S")
def create_directory_structure(base_path):
"""Create the .agenthub/ directory tree."""
dirs = [
os.path.join(base_path, "sessions"),
os.path.join(base_path, "board", "dispatch"),
os.path.join(base_path, "board", "progress"),
os.path.join(base_path, "board", "results"),
]
for d in dirs:
os.makedirs(d, exist_ok=True)
def write_gitignore(base_path):
"""Write .agenthub/.gitignore to exclude worktree artifacts."""
gitignore_path = os.path.join(base_path, ".gitignore")
if not os.path.exists(gitignore_path):
with open(gitignore_path, "w") as f:
f.write("# AgentHub gitignore\n")
f.write("# Keep board and sessions, ignore worktree artifacts\n")
f.write("*.tmp\n")
f.write("*.lock\n")
def write_board_index(base_path):
"""Initialize the board index file."""
index_path = os.path.join(base_path, "board", "_index.json")
if not os.path.exists(index_path):
index = {
"channels": ["dispatch", "progress", "results"],
"counters": {"dispatch": 0, "progress": 0, "results": 0},
}
with open(index_path, "w") as f:
json.dump(index, f, indent=2)
f.write("\n")
def create_session(base_path, session_id, task, agents, eval_cmd, metric,
direction, base_branch):
"""Create a new session with config and state files."""
session_dir = os.path.join(base_path, "sessions", session_id)
os.makedirs(session_dir, exist_ok=True)
# Write config.yaml (manual YAML to avoid dependency)
config_path = os.path.join(session_dir, "config.yaml")
config_lines = [
f"session_id: {session_id}",
f"task: \"{task}\"",
f"agent_count: {agents}",
f"base_branch: {base_branch}",
f"created: {datetime.now(timezone.utc).isoformat()}",
]
if eval_cmd:
config_lines.append(f"eval_cmd: \"{eval_cmd}\"")
if metric:
config_lines.append(f"metric: {metric}")
if direction:
config_lines.append(f"direction: {direction}")
with open(config_path, "w") as f:
f.write("\n".join(config_lines))
f.write("\n")
# Write state.json
state_path = os.path.join(session_dir, "state.json")
state = {
"session_id": session_id,
"state": "init",
"created": datetime.now(timezone.utc).isoformat(),
"updated": datetime.now(timezone.utc).isoformat(),
"agents": {},
}
with open(state_path, "w") as f:
json.dump(state, f, indent=2)
f.write("\n")
return session_dir
def validate_git_repo():
"""Check if current directory is a git repository."""
if not os.path.isdir(".git"):
# Check parent dirs
path = os.path.abspath(".")
while path != "/":
if os.path.isdir(os.path.join(path, ".git")):
return True
path = os.path.dirname(path)
return False
return True
def get_current_branch():
"""Get the current git branch name."""
head_file = os.path.join(".git", "HEAD")
if os.path.exists(head_file):
with open(head_file) as f:
ref = f.read().strip()
if ref.startswith("ref: refs/heads/"):
return ref[len("ref: refs/heads/"):]
return "main"
def run_demo():
"""Show a demo of what hub_init creates."""
print("=" * 60)
print("AgentHub Init — Demo Mode")
print("=" * 60)
print()
print("Session ID: 20260317-143022")
print("Task: Optimize API response time below 100ms")
print("Agents: 3")
print("Eval: pytest bench.py --json")
print("Metric: p50_ms (lower is better)")
print("Base branch: dev")
print()
print("Directory structure created:")
print(" .agenthub/")
print(" ├── .gitignore")
print(" ├── sessions/")
print(" │ └── 20260317-143022/")
print(" │ ├── config.yaml")
print(" │ └── state.json")
print(" └── board/")
print(" ├── _index.json")
print(" ├── dispatch/")
print(" ├── progress/")
print(" └── results/")
print()
print("config.yaml:")
print(' session_id: 20260317-143022')
print(' task: "Optimize API response time below 100ms"')
print(" agent_count: 3")
print(" base_branch: dev")
print(' eval_cmd: "pytest bench.py --json"')
print(" metric: p50_ms")
print(" direction: lower")
print()
print("state.json:")
print(' { "state": "init", "agents": {} }')
print()
print("Next step: Run /hub:spawn to launch agents")
def main():
parser = argparse.ArgumentParser(
description="Initialize an AgentHub collaboration session"
)
parser.add_argument("--task", type=str, help="Task description for agents")
parser.add_argument("--agents", type=int, default=3,
help="Number of parallel agents (default: 3)")
parser.add_argument("--eval", type=str, dest="eval_cmd",
help="Evaluation command to run in each worktree")
parser.add_argument("--metric", type=str,
help="Metric name to extract from eval output")
parser.add_argument("--direction", choices=["lower", "higher"],
help="Whether lower or higher metric is better")
parser.add_argument("--base-branch", type=str,
help="Base branch (default: current branch)")
parser.add_argument("--format", choices=["text", "json"], default="text",
help="Output format (default: text)")
parser.add_argument("--demo", action="store_true",
help="Show demo output without creating files")
args = parser.parse_args()
if args.demo:
run_demo()
return
if not args.task:
print("Error: --task is required", file=sys.stderr)
print("Usage: hub_init.py --task 'description' [--agents N] "
"[--eval 'cmd'] [--metric name] [--direction lower|higher]",
file=sys.stderr)
sys.exit(1)
if not validate_git_repo():
print("Error: Not a git repository. AgentHub requires git.",
file=sys.stderr)
sys.exit(1)
base_branch = args.base_branch or get_current_branch()
base_path = ".agenthub"
session_id = generate_session_id()
# Create structure
create_directory_structure(base_path)
write_gitignore(base_path)
write_board_index(base_path)
# Create session
session_dir = create_session(
base_path, session_id, args.task, args.agents,
args.eval_cmd, args.metric, args.direction, base_branch
)
if args.format == "json":
output = {
"session_id": session_id,
"session_dir": session_dir,
"task": args.task,
"agent_count": args.agents,
"eval_cmd": args.eval_cmd,
"metric": args.metric,
"direction": args.direction,
"base_branch": base_branch,
"state": "init",
}
print(json.dumps(output, indent=2))
else:
print(f"AgentHub session initialized")
print(f" Session ID: {session_id}")
print(f" Task: {args.task}")
print(f" Agents: {args.agents}")
if args.eval_cmd:
print(f" Eval: {args.eval_cmd}")
if args.metric:
direction_str = "lower is better" if args.direction == "lower" else "higher is better"
print(f" Metric: {args.metric} ({direction_str})")
print(f" Base branch: {base_branch}")
print(f" State: init")
print()
print(f"Next step: Run /hub:spawn to launch {args.agents} agents")
if __name__ == "__main__":
main()
FILE:scripts/result_ranker.py
#!/usr/bin/env python3
"""Rank AgentHub agent results by metric or diff quality.
Runs an evaluation command in each agent's worktree, parses a metric,
and produces a ranked table.
Usage:
python result_ranker.py --session 20260317-143022 \\
--eval-cmd "pytest bench.py --json" --metric p50_ms --direction lower
python result_ranker.py --session 20260317-143022 --diff-summary
python result_ranker.py --demo
"""
import argparse
import json
import os
import re
import subprocess
import sys
def run_git(*args):
"""Run a git command and return stdout."""
try:
result = subprocess.run(
["git"] + list(args),
capture_output=True, text=True, check=True
)
return result.stdout.strip()
except subprocess.CalledProcessError as e:
return ""
def get_session_config(session_id):
"""Load session config."""
config_path = os.path.join(".agenthub", "sessions", session_id, "config.yaml")
if not os.path.exists(config_path):
print(f"Error: Session {session_id} not found", file=sys.stderr)
sys.exit(1)
config = {}
with open(config_path) as f:
for line in f:
line = line.strip()
if ":" in line and not line.startswith("#"):
key, val = line.split(":", 1)
val = val.strip().strip('"')
config[key.strip()] = val
return config
def get_hub_branches(session_id):
"""Get all hub branches for a session."""
output = run_git("branch", "--list", f"hub/{session_id}/*",
"--format=%(refname:short)")
if not output:
return []
return [b.strip() for b in output.split("\n") if b.strip()]
def get_worktree_path(branch):
"""Get the worktree path for a branch, if it exists."""
output = run_git("worktree", "list", "--porcelain")
if not output:
return None
current_path = None
for line in output.split("\n"):
if line.startswith("worktree "):
current_path = line[len("worktree "):]
elif line.startswith("branch ") and current_path:
ref = line[len("branch "):]
short = ref.replace("refs/heads/", "")
if short == branch:
return current_path
current_path = None
return None
def run_eval_in_worktree(worktree_path, eval_cmd):
"""Run evaluation command in a worktree and return stdout."""
try:
result = subprocess.run(
eval_cmd, shell=True, capture_output=True, text=True,
cwd=worktree_path, timeout=120
)
return result.stdout.strip(), result.returncode
except subprocess.TimeoutExpired:
return "TIMEOUT", 1
except Exception as e:
return str(e), 1
def extract_metric(output, metric_name):
"""Extract a numeric metric from command output.
Looks for patterns like:
- metric_name: 42.5
- metric_name=42.5
- "metric_name": 42.5
"""
patterns = [
rf'{metric_name}\s*[:=]\s*([\d.]+)',
rf'"{metric_name}"\s*[:=]\s*([\d.]+)',
rf"'{metric_name}'\s*[:=]\s*([\d.]+)",
]
for pattern in patterns:
match = re.search(pattern, output, re.IGNORECASE)
if match:
try:
return float(match.group(1))
except ValueError:
continue
return None
def get_diff_stats(branch, base_branch="main"):
"""Get diff statistics for a branch vs base."""
output = run_git("diff", "--stat", f"{base_branch}...{branch}")
lines_output = run_git("diff", "--shortstat", f"{base_branch}...{branch}")
files_changed = 0
insertions = 0
deletions = 0
if lines_output:
files_match = re.search(r"(\d+) files? changed", lines_output)
ins_match = re.search(r"(\d+) insertions?", lines_output)
del_match = re.search(r"(\d+) deletions?", lines_output)
if files_match:
files_changed = int(files_match.group(1))
if ins_match:
insertions = int(ins_match.group(1))
if del_match:
deletions = int(del_match.group(1))
return {
"files_changed": files_changed,
"insertions": insertions,
"deletions": deletions,
"net_lines": insertions - deletions,
}
def rank_by_metric(results, direction="lower"):
"""Sort results by metric value."""
valid = [r for r in results if r.get("metric_value") is not None]
invalid = [r for r in results if r.get("metric_value") is None]
reverse = direction == "higher"
valid.sort(key=lambda r: r["metric_value"], reverse=reverse)
for i, r in enumerate(valid):
r["rank"] = i + 1
for r in invalid:
r["rank"] = len(valid) + 1
return valid + invalid
def run_demo():
"""Show demo ranking output."""
print("=" * 60)
print("AgentHub Result Ranker — Demo Mode")
print("=" * 60)
print()
print("Session: 20260317-143022")
print("Eval: pytest bench.py --json")
print("Metric: p50_ms (lower is better)")
print("Baseline: 180ms")
print()
header = f"{'RANK':<6} {'AGENT':<10} {'METRIC':<10} {'DELTA':<10} {'FILES':<7} {'SUMMARY'}"
print(header)
print("-" * 75)
print(f"{'1':<6} {'agent-2':<10} {'142ms':<10} {'-38ms':<10} {'2':<7} Replaced O(n²) with hash map lookup")
print(f"{'2':<6} {'agent-1':<10} {'165ms':<10} {'-15ms':<10} {'3':<7} Added caching layer")
print(f"{'3':<6} {'agent-3':<10} {'190ms':<10} {'+10ms':<10} {'1':<7} Minor loop optimizations")
print()
print("Winner: agent-2 (142ms, -21% from baseline)")
print()
print("Next step: Run /hub:merge to merge agent-2's branch")
def main():
parser = argparse.ArgumentParser(
description="Rank AgentHub agent results"
)
parser.add_argument("--session", type=str,
help="Session ID to evaluate")
parser.add_argument("--eval-cmd", type=str,
help="Evaluation command to run in each worktree")
parser.add_argument("--metric", type=str,
help="Metric name to extract from eval output")
parser.add_argument("--direction", choices=["lower", "higher"],
default="lower",
help="Whether lower or higher metric is better")
parser.add_argument("--baseline", type=float,
help="Baseline metric value for delta calculation")
parser.add_argument("--diff-summary", action="store_true",
help="Show diff statistics per agent (no eval cmd needed)")
parser.add_argument("--format", choices=["table", "json"], default="table",
help="Output format (default: table)")
parser.add_argument("--demo", action="store_true",
help="Show demo output")
args = parser.parse_args()
if args.demo:
run_demo()
return
if not args.session:
print("Error: --session is required", file=sys.stderr)
sys.exit(1)
config = get_session_config(args.session)
branches = get_hub_branches(args.session)
if not branches:
print(f"No branches found for session {args.session}")
return
eval_cmd = args.eval_cmd or config.get("eval_cmd")
metric = args.metric or config.get("metric")
direction = args.direction or config.get("direction", "lower")
base_branch = config.get("base_branch", "main")
results = []
for branch in branches:
# Extract agent number
match = re.match(r"hub/[^/]+/agent-(\d+)/", branch)
agent_id = f"agent-{match.group(1)}" if match else branch.split("/")[-2]
result = {
"agent": agent_id,
"branch": branch,
"metric_value": None,
"metric_raw": None,
"diff": get_diff_stats(branch, base_branch),
}
if eval_cmd and metric:
worktree = get_worktree_path(branch)
if worktree:
output, returncode = run_eval_in_worktree(worktree, eval_cmd)
result["metric_raw"] = output
result["eval_returncode"] = returncode
if returncode == 0:
result["metric_value"] = extract_metric(output, metric)
results.append(result)
# Rank
ranked = rank_by_metric(results, direction)
# Calculate deltas
baseline = args.baseline
if baseline is None and ranked and ranked[0].get("metric_value") is not None:
# Use worst as baseline if not specified
values = [r["metric_value"] for r in ranked if r["metric_value"] is not None]
if values:
baseline = max(values) if direction == "lower" else min(values)
for r in ranked:
if r.get("metric_value") is not None and baseline is not None:
r["delta"] = r["metric_value"] - baseline
else:
r["delta"] = None
if args.format == "json":
print(json.dumps({"session": args.session, "results": ranked}, indent=2))
return
# Table output
print(f"Session: {args.session}")
if eval_cmd:
print(f"Eval: {eval_cmd}")
if metric:
dir_str = "lower is better" if direction == "lower" else "higher is better"
print(f"Metric: {metric} ({dir_str})")
if baseline:
print(f"Baseline: {baseline}")
print()
if args.diff_summary or not eval_cmd:
header = f"{'RANK':<6} {'AGENT':<12} {'FILES':<7} {'ADDED':<8} {'REMOVED':<8} {'NET':<6}"
print(header)
print("-" * 50)
for i, r in enumerate(ranked):
d = r["diff"]
print(f"{i+1:<6} {r['agent']:<12} {d['files_changed']:<7} "
f"+{d['insertions']:<7} -{d['deletions']:<7} {d['net_lines']:<6}")
else:
header = f"{'RANK':<6} {'AGENT':<12} {'METRIC':<12} {'DELTA':<10} {'FILES':<7}"
print(header)
print("-" * 50)
for r in ranked:
mv = str(r["metric_value"]) if r["metric_value"] is not None else "N/A"
delta = ""
if r["delta"] is not None:
sign = "+" if r["delta"] >= 0 else ""
delta = f"{sign}{r['delta']:.1f}"
print(f"{r['rank']:<6} {r['agent']:<12} {mv:<12} {delta:<10} {r['diff']['files_changed']:<7}")
# Winner
if ranked and ranked[0].get("metric_value") is not None:
winner = ranked[0]
print()
print(f"Winner: {winner['agent']} ({winner['metric_value']})")
if __name__ == "__main__":
main()
FILE:scripts/session_manager.py
#!/usr/bin/env python3
"""AgentHub session state machine and lifecycle manager.
Manages session states (init → running → evaluating → merged/archived),
lists sessions, and handles cleanup of worktrees and branches.
Usage:
python session_manager.py --list
python session_manager.py --status 20260317-143022
python session_manager.py --update 20260317-143022 --state running
python session_manager.py --cleanup 20260317-143022
python session_manager.py --demo
"""
import argparse
import json
import os
import subprocess
import sys
from datetime import datetime, timezone
SESSIONS_PATH = ".agenthub/sessions"
VALID_STATES = ["init", "running", "evaluating", "merged", "archived"]
VALID_TRANSITIONS = {
"init": ["running"],
"running": ["evaluating"],
"evaluating": ["merged", "archived"],
"merged": [],
"archived": [],
}
def load_state(session_id):
"""Load session state.json."""
state_path = os.path.join(SESSIONS_PATH, session_id, "state.json")
if not os.path.exists(state_path):
return None
with open(state_path) as f:
return json.load(f)
def save_state(session_id, state):
"""Save session state.json."""
state_path = os.path.join(SESSIONS_PATH, session_id, "state.json")
state["updated"] = datetime.now(timezone.utc).isoformat()
with open(state_path, "w") as f:
json.dump(state, f, indent=2)
f.write("\n")
def load_config(session_id):
"""Load session config.yaml (simple key: value parsing)."""
config_path = os.path.join(SESSIONS_PATH, session_id, "config.yaml")
if not os.path.exists(config_path):
return None
config = {}
with open(config_path) as f:
for line in f:
line = line.strip()
if ":" in line and not line.startswith("#"):
key, val = line.split(":", 1)
config[key.strip()] = val.strip().strip('"')
return config
def run_git(*args):
"""Run a git command and return stdout."""
try:
result = subprocess.run(
["git"] + list(args),
capture_output=True, text=True, check=True
)
return result.stdout.strip()
except subprocess.CalledProcessError:
return ""
def list_sessions(output_format="text"):
"""List all sessions with their states."""
if not os.path.isdir(SESSIONS_PATH):
print("No sessions found. Run hub_init.py first.")
return
sessions = []
for sid in sorted(os.listdir(SESSIONS_PATH)):
session_dir = os.path.join(SESSIONS_PATH, sid)
if not os.path.isdir(session_dir):
continue
state = load_state(sid)
config = load_config(sid)
if state and config:
sessions.append({
"session_id": sid,
"state": state.get("state", "unknown"),
"task": config.get("task", ""),
"agents": config.get("agent_count", "?"),
"created": state.get("created", ""),
})
if output_format == "json":
print(json.dumps({"sessions": sessions}, indent=2))
return
if not sessions:
print("No sessions found.")
return
print("AgentHub Sessions")
print()
header = f"{'SESSION ID':<20} {'STATE':<12} {'AGENTS':<8} {'TASK'}"
print(header)
print("-" * 70)
for s in sessions:
task = s["task"][:40] + "..." if len(s["task"]) > 40 else s["task"]
print(f"{s['session_id']:<20} {s['state']:<12} {s['agents']:<8} {task}")
def show_status(session_id, output_format="text"):
"""Show detailed status for a session."""
state = load_state(session_id)
config = load_config(session_id)
if not state or not config:
print(f"Error: Session {session_id} not found", file=sys.stderr)
sys.exit(1)
if output_format == "json":
print(json.dumps({"config": config, "state": state}, indent=2))
return
print(f"Session: {session_id}")
print(f" State: {state.get('state', 'unknown')}")
print(f" Task: {config.get('task', '')}")
print(f" Agents: {config.get('agent_count', '?')}")
print(f" Base branch: {config.get('base_branch', '?')}")
if config.get("eval_cmd"):
print(f" Eval: {config['eval_cmd']}")
if config.get("metric"):
print(f" Metric: {config['metric']} ({config.get('direction', '?')})")
print(f" Created: {state.get('created', '?')}")
print(f" Updated: {state.get('updated', '?')}")
# Show agent branches
branches = run_git("branch", "--list", f"hub/{session_id}/*",
"--format=%(refname:short)")
if branches:
print()
print(" Branches:")
for b in branches.split("\n"):
if b.strip():
print(f" {b.strip()}")
def update_state(session_id, new_state):
"""Transition session to a new state."""
state = load_state(session_id)
if not state:
print(f"Error: Session {session_id} not found", file=sys.stderr)
sys.exit(1)
current = state.get("state", "unknown")
if new_state not in VALID_STATES:
print(f"Error: Invalid state '{new_state}'. "
f"Valid: {', '.join(VALID_STATES)}", file=sys.stderr)
sys.exit(1)
valid_next = VALID_TRANSITIONS.get(current, [])
if new_state not in valid_next:
print(f"Error: Cannot transition from '{current}' to '{new_state}'. "
f"Valid transitions: {', '.join(valid_next) or 'none (terminal)'}",
file=sys.stderr)
sys.exit(1)
state["state"] = new_state
save_state(session_id, state)
print(f"Session {session_id}: {current} → {new_state}")
def cleanup_session(session_id):
"""Clean up worktrees and optionally archive branches."""
config = load_config(session_id)
if not config:
print(f"Error: Session {session_id} not found", file=sys.stderr)
sys.exit(1)
# Find and remove worktrees for this session
worktree_output = run_git("worktree", "list", "--porcelain")
removed = 0
if worktree_output:
current_path = None
for line in worktree_output.split("\n"):
if line.startswith("worktree "):
current_path = line[len("worktree "):]
elif line.startswith("branch ") and current_path:
ref = line[len("branch "):]
if f"hub/{session_id}/" in ref:
result = subprocess.run(
["git", "worktree", "remove", "--force", current_path],
capture_output=True, text=True
)
if result.returncode == 0:
removed += 1
print(f" Removed worktree: {current_path}")
current_path = None
print(f"Cleaned up {removed} worktrees for session {session_id}")
def run_demo():
"""Show demo output."""
print("=" * 60)
print("AgentHub Session Manager — Demo Mode")
print("=" * 60)
print()
print("--- Session List ---")
print("AgentHub Sessions")
print()
header = f"{'SESSION ID':<20} {'STATE':<12} {'AGENTS':<8} {'TASK'}"
print(header)
print("-" * 70)
print(f"{'20260317-143022':<20} {'merged':<12} {'3':<8} Optimize API response time below 100ms")
print(f"{'20260317-151500':<20} {'running':<12} {'2':<8} Refactor auth module for JWT support")
print(f"{'20260317-160000':<20} {'init':<12} {'4':<8} Implement caching strategy")
print()
print("--- Session Detail ---")
print("Session: 20260317-143022")
print(" State: merged")
print(" Task: Optimize API response time below 100ms")
print(" Agents: 3")
print(" Base branch: dev")
print(" Eval: pytest bench.py --json")
print(" Metric: p50_ms (lower)")
print(" Created: 2026-03-17T14:30:22Z")
print(" Updated: 2026-03-17T14:45:00Z")
print()
print(" Branches:")
print(" hub/20260317-143022/agent-1/attempt-1 (archived)")
print(" hub/20260317-143022/agent-2/attempt-1 (merged)")
print(" hub/20260317-143022/agent-3/attempt-1 (archived)")
print()
print("--- State Transitions ---")
print("Valid transitions:")
for state, transitions in VALID_TRANSITIONS.items():
arrow = " → ".join(transitions) if transitions else "(terminal)"
print(f" {state}: {arrow}")
def main():
parser = argparse.ArgumentParser(
description="AgentHub session state machine and lifecycle manager"
)
parser.add_argument("--list", action="store_true",
help="List all sessions with state")
parser.add_argument("--status", type=str, metavar="SESSION_ID",
help="Show detailed session status")
parser.add_argument("--update", type=str, metavar="SESSION_ID",
help="Update session state")
parser.add_argument("--state", type=str,
help="New state for --update")
parser.add_argument("--cleanup", type=str, metavar="SESSION_ID",
help="Remove worktrees and clean up session")
parser.add_argument("--format", choices=["text", "json"], default="text",
help="Output format (default: text)")
parser.add_argument("--demo", action="store_true",
help="Show demo output")
args = parser.parse_args()
if args.demo:
run_demo()
return
if args.list:
list_sessions(args.format)
return
if args.status:
show_status(args.status, args.format)
return
if args.update:
if not args.state:
print("Error: --update requires --state", file=sys.stderr)
sys.exit(1)
update_state(args.update, args.state)
return
if args.cleanup:
cleanup_session(args.cleanup)
return
parser.print_help()
if __name__ == "__main__":
main()
Giao thức giao tiếp giữa các agent C-suite: cú pháp gọi, chống vòng lặp, cách ly và định dạng phản hồi.
---
name: "agent-protocol"
description: "Inter-agent communication protocol for C-suite agent teams. Defines invocation syntax, loop prevention, isolation rules, and response formats. Use when C-suite agents need to query each other, coordinate cross-functional analysis, or run board meetings with multiple agent roles."
license: MIT
metadata:
version: 1.0.0
author: Alireza Rezvani
category: c-level
domain: agent-orchestration
updated: 2026-03-05
frameworks: invocation-patterns
---
# Inter-Agent Protocol
How C-suite agents talk to each other. Rules that prevent chaos, loops, and circular reasoning.
## Keywords
agent protocol, inter-agent communication, agent invocation, agent orchestration, multi-agent, c-suite coordination, agent chain, loop prevention, agent isolation, board meeting protocol
## Invocation Syntax
Any agent can query another using:
```
[INVOKE:role|question]
```
**Examples:**
```
[INVOKE:cfo|What's the burn rate impact of hiring 5 engineers in Q3?]
[INVOKE:cto|Can we realistically ship this feature by end of quarter?]
[INVOKE:chro|What's our typical time-to-hire for senior engineers?]
[INVOKE:cro|What does our pipeline look like for the next 90 days?]
```
**Valid roles:** `ceo`, `cfo`, `cro`, `cmo`, `cpo`, `cto`, `chro`, `coo`, `ciso`
## Response Format
Invoked agents respond using this structure:
```
[RESPONSE:role]
Key finding: [one line — the actual answer]
Supporting data:
- [data point 1]
- [data point 2]
- [data point 3 — optional]
Confidence: [high | medium | low]
Caveat: [one line — what could make this wrong]
[/RESPONSE]
```
**Example:**
```
[RESPONSE:cfo]
Key finding: Hiring 5 engineers in Q3 extends runway from 14 to 9 months at current burn.
Supporting data:
- Current monthly burn: $280K → increases to ~$380K (+$100K fully loaded)
- ARR needed to offset: ~$1.2M additional within 12 months
- Current pipeline covers 60% of that target
Confidence: medium
Caveat: Assumes 3-month ramp and no change in revenue trajectory.
[/RESPONSE]
```
## Loop Prevention (Hard Rules)
These rules are enforced unconditionally. No exceptions.
### Rule 1: No Self-Invocation
An agent cannot invoke itself.
```
❌ CFO → [INVOKE:cfo|...] — BLOCKED
```
### Rule 2: Maximum Depth = 2
Chains can go A→B→C. The third hop is blocked.
```
✅ CRO → CFO → COO (depth 2)
❌ CRO → CFO → COO → CHRO (depth 3 — BLOCKED)
```
### Rule 3: No Circular Calls
If agent A called agent B, agent B cannot call agent A in the same chain.
```
✅ CRO → CFO → CMO
❌ CRO → CFO → CRO (circular — BLOCKED)
```
### Rule 4: Chain Tracking
Each invocation carries its call chain. Format:
```
[CHAIN: cro → cfo → coo]
```
Agents check this chain before responding with another invocation.
**When blocked:** Return this instead of invoking:
```
[BLOCKED: cannot invoke cfo — circular call detected in chain cro→cfo]
State assumption used instead: [explicit assumption the agent is making]
```
## Isolation Rules
### Board Meeting Phase 2 (Independent Analysis)
**NO invocations allowed.** Each role forms independent views before cross-pollination.
- Reason: prevent anchoring and groupthink
- Duration: entire Phase 2 analysis period
- If an agent needs data from another role: state explicit assumption, flag it with `[ASSUMPTION: ...]`
### Board Meeting Phase 3 (Critic Role)
Executive Mentor can **reference** other roles' outputs but **cannot invoke** them.
- Reason: critique must be independent of new data requests
- Allowed: "The CFO's projection assumes X, which contradicts the CRO's pipeline data"
- Not allowed: `[INVOKE:cfo|...]` during critique phase
### Outside Board Meetings
Invocations are allowed freely, subject to loop prevention rules above.
## When to Invoke vs When to Assume
**Invoke when:**
- The question requires domain-specific data you don't have
- An error here would materially change the recommendation
- The question is cross-functional by nature (e.g., hiring impact on both budget and capacity)
**Assume when:**
- The data is directionally clear and precision isn't critical
- You're in Phase 2 isolation (always assume, never invoke)
- The chain is already at depth 2
- The question is minor compared to your main analysis
**When assuming, always state it:**
```
[ASSUMPTION: runway ~12 months based on typical Series A burn profile — not verified with CFO]
```
## Conflict Resolution
When two invoked agents give conflicting answers:
1. **Flag the conflict explicitly:**
```
[CONFLICT: CFO projects 14-month runway; CRO expects pipeline to close 80% → implies 18+ months]
```
2. **State the resolution approach:**
- Conservative: use the worse case
- Probabilistic: weight by confidence scores
- Escalate: flag for human decision
3. **Never silently pick one** — surface the conflict to the user.
## Broadcast Pattern (Crisis / CEO)
CEO can broadcast to all roles simultaneously:
```
[BROADCAST:all|What's the impact if we miss the fundraise?]
```
Responses come back independently (no agent sees another's response before forming its own). Aggregate after all respond.
## Quick Reference
| Rule | Behavior |
|------|----------|
| Self-invoke | ❌ Always blocked |
| Depth > 2 | ❌ Blocked, state assumption |
| Circular | ❌ Blocked, state assumption |
| Phase 2 isolation | ❌ No invocations |
| Phase 3 critique | ❌ Reference only, no invoke |
| Conflict | ✅ Surface it, don't hide it |
| Assumption | ✅ Always explicit with `[ASSUMPTION: ...]` |
## Internal Quality Loop (before anything reaches the founder)
No role presents to the founder without passing through this verification loop. The founder sees polished, verified output — not first drafts.
### Step 1: Self-Verification (every role, every time)
Before presenting, every role runs this internal checklist:
```
SELF-VERIFY CHECKLIST:
□ Source Attribution — Where did each data point come from?
✅ "ARR is $2.1M (from CRO pipeline report, Q4 actuals)"
❌ "ARR is around $2M" (no source, vague)
□ Assumption Audit — What am I assuming vs what I verified?
Tag every assumption: [VERIFIED: checked against data] or [ASSUMED: not verified]
If >50% of findings are ASSUMED → flag low confidence
□ Confidence Score — How sure am I on each finding?
🟢 High: verified data, established pattern, multiple sources
🟡 Medium: single source, reasonable inference, some uncertainty
🔴 Low: assumption-based, limited data, first-time analysis
□ Contradiction Check — Does this conflict with known context?
Check against company-context.md and recent decisions in decision-log
If it contradicts a past decision → flag explicitly
□ "So What?" Test — Does every finding have a business consequence?
If you can't answer "so what?" in one sentence → cut it
```
### Step 2: Peer Verification (cross-functional validation)
When a recommendation impacts another role's domain, that role validates BEFORE presenting.
| If your recommendation involves... | Validate with... | They check... |
|-------------------------------------|-------------------|---------------|
| Financial numbers or budget | CFO | Math, runway impact, budget reality |
| Revenue projections | CRO | Pipeline backing, historical accuracy |
| Headcount or hiring | CHRO | Market reality, comp feasibility, timeline |
| Technical feasibility or timeline | CTO | Engineering capacity, technical debt load |
| Operational process changes | COO | Capacity, dependencies, scaling impact |
| Customer-facing changes | CRO + CPO | Churn risk, product roadmap conflict |
| Security or compliance claims | CISO | Actual posture, regulation requirements |
| Market or positioning claims | CMO | Data backing, competitive reality |
**Peer validation format:**
```
[PEER-VERIFY:cfo]
Validated: ✅ Burn rate calculation correct
Adjusted: ⚠️ Hiring timeline should be Q3 not Q2 (budget constraint)
Flagged: 🔴 Missing equity cost in total comp projection
[/PEER-VERIFY]
```
**Skip peer verification when:**
- Single-domain question with no cross-functional impact
- Time-sensitive proactive alert (send alert, verify after)
- Founder explicitly asked for a quick take
### Step 3: Critic Pre-Screen (high-stakes decisions only)
For decisions that are **irreversible, high-cost, or bet-the-company**, the Executive Mentor pre-screens before the founder sees it.
**Triggers for pre-screen:**
- Involves spending > 20% of remaining runway
- Affects >30% of the team (layoffs, reorg)
- Changes company strategy or direction
- Involves external commitments (fundraising terms, partnerships, M&A)
- Any recommendation where all roles agree (suspicious consensus)
**Pre-screen output:**
```
[CRITIC-SCREEN]
Weakest point: [The single biggest vulnerability in this recommendation]
Missing perspective: [What nobody considered]
If wrong, the cost is: [Quantified downside]
Proceed: ✅ With noted risks | ⚠️ After addressing [specific gap] | 🔴 Rethink
[/CRITIC-SCREEN]
```
### Step 4: Course Correction (after founder feedback)
The loop doesn't end at delivery. After the founder responds:
```
FOUNDER FEEDBACK LOOP:
1. Founder approves → log decision (Layer 2), assign actions
2. Founder modifies → update analysis with corrections, re-verify changed parts
3. Founder rejects → log rejection with DO_NOT_RESURFACE, understand WHY
4. Founder asks follow-up → deepen analysis on specific point, re-verify
POST-DECISION REVIEW (30/60/90 days):
- Was the recommendation correct?
- What did we miss?
- Update company-context.md with what we learned
- If wrong → document the lesson, adjust future analysis
```
### Verification Level by Stakes
| Stakes | Self-Verify | Peer-Verify | Critic Pre-Screen |
|--------|-------------|-------------|-------------------|
| Low (informational) | ✅ Required | ❌ Skip | ❌ Skip |
| Medium (operational) | ✅ Required | ✅ Required | ❌ Skip |
| High (strategic) | ✅ Required | ✅ Required | ✅ Required |
| Critical (irreversible) | ✅ Required | ✅ Required | ✅ Required + board meeting |
### What Changes in the Output Format
The verified output adds confidence and source information:
```
BOTTOM LINE
[Answer] — Confidence: 🟢 High
WHAT
• [Finding 1] [VERIFIED: Q4 actuals] 🟢
• [Finding 2] [VERIFIED: CRO pipeline data] 🟢
• [Finding 3] [ASSUMED: based on industry benchmarks] 🟡
PEER-VERIFIED BY: CFO (math ✅), CTO (timeline ⚠️ adjusted to Q3)
```
---
## User Communication Standard
All C-suite output to the founder follows ONE format. No exceptions. The founder is the decision-maker — give them results, not process.
### Standard Output (single-role response)
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
📊 [ROLE] — [Topic]
BOTTOM LINE
[One sentence. The answer. No preamble.]
WHAT
• [Finding 1 — most critical]
• [Finding 2]
• [Finding 3]
(Max 5 bullets. If more needed → reference doc.)
WHY THIS MATTERS
[1-2 sentences. Business impact. Not theory — consequence.]
HOW TO ACT
1. [Action] → [Owner] → [Deadline]
2. [Action] → [Owner] → [Deadline]
3. [Action] → [Owner] → [Deadline]
⚠️ RISKS (if any)
• [Risk + what triggers it]
🔑 YOUR DECISION (if needed)
Option A: [Description] — [Trade-off]
Option B: [Description] — [Trade-off]
Recommendation: [Which and why, in one line]
📎 DETAIL: [reference doc or script output for deep-dive]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
```
### Proactive Alert (unsolicited — triggered by context)
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
🚩 [ROLE] — Proactive Alert
WHAT I NOTICED
[What triggered this — specific, not vague]
WHY IT MATTERS
[Business consequence if ignored — in dollars, time, or risk]
RECOMMENDED ACTION
[Exactly what to do, who does it, by when]
URGENCY: 🔴 Act today | 🟡 This week | ⚪ Next review
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
```
### Board Meeting Output (multi-role synthesis)
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
📋 BOARD MEETING — [Date] — [Agenda Topic]
DECISION REQUIRED
[Frame the decision in one sentence]
PERSPECTIVES
CEO: [one-line position]
CFO: [one-line position]
CRO: [one-line position]
[... only roles that contributed]
WHERE THEY AGREE
• [Consensus point 1]
• [Consensus point 2]
WHERE THEY DISAGREE
• [Conflict] — CEO says X, CFO says Y
• [Conflict] — CRO says X, CPO says Y
CRITIC'S VIEW (Executive Mentor)
[The uncomfortable truth nobody else said]
RECOMMENDED DECISION
[Clear recommendation with rationale]
ACTION ITEMS
1. [Action] → [Owner] → [Deadline]
2. [Action] → [Owner] → [Deadline]
3. [Action] → [Owner] → [Deadline]
🔑 YOUR CALL
[Options if you disagree with the recommendation]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
```
### Communication Rules (non-negotiable)
1. **Bottom line first.** Always. The founder's time is the scarcest resource.
2. **Results and decisions only.** No process narration ("First I analyzed..."). No thinking out loud.
3. **What + Why + How.** Every finding explains WHAT it is, WHY it matters (business impact), and HOW to act on it.
4. **Max 5 bullets per section.** Longer = reference doc.
5. **Actions have owners and deadlines.** "We should consider" is banned. Who does what by when.
6. **Decisions framed as options.** Not "what do you think?" — "Option A or B, here's the trade-off, here's my recommendation."
7. **The founder decides.** Roles recommend. The founder approves, modifies, or rejects. Every output respects this hierarchy.
8. **Risks are concrete.** Not "there might be risks" — "if X happens, Y breaks, costing $Z."
9. **No jargon without explanation.** If you use a term, explain it on first use.
10. **Silence is an option.** If there's nothing to report, don't fabricate updates.
## Reference
- `references/invocation-patterns.md` — common cross-functional patterns with examples
FILE:references/invocation-patterns.md
# Invocation Patterns — Inter-Agent Protocol
Common cross-functional patterns, with examples and guidance on when to invoke vs assume.
---
## Pattern 1: Revenue Planning Chain
**CRO → CFO → CMO**
Used when: setting revenue targets, allocating growth budget, modeling pipeline scenarios.
```
CRO initiates:
"Planning Q3 revenue targets. Need financial constraints and marketing input."
CRO → [INVOKE:cfo|What's our Q3 revenue target given current runway and burn?]
CFO responds:
[RESPONSE:cfo]
Key finding: To maintain 12-month runway without new raise, Q3 ARR target ≥ $2.4M (+20% QoQ)
Supporting data:
- Current ARR: $2.0M, burn: $180K/month, runway: 14 months
- 20% QoQ growth maintains runway; <15% triggers bridge discussion
- Board expectation communicated as 25% QoQ
Confidence: high
Caveat: Assumes no new hires or major infrastructure spend in Q3.
[/RESPONSE]
CRO → [INVOKE:cmo|Given a $2.4M Q3 ARR target, what pipeline coverage and CAC should we plan for?]
[CHAIN: cro → cfo → cmo]
CMO responds:
[RESPONSE:cmo]
Key finding: Need 3x pipeline ($7.2M) and CAC ≤ $8,400 to hit $2.4M ARR target.
Supporting data:
- Historical close rate: 33% → 3x coverage required
- Current CAC: $9,100 → 8% efficiency improvement needed
- Recommended channel mix: 60% outbound, 30% inbound, 10% partner
Confidence: medium
Caveat: Close rate assumes no sales team changes; any rep turnover drops it to 25%.
[/RESPONSE]
```
**What CRO does with this:** Synthesizes CFO constraint (target) + CMO playbook (how to hit it) into a revenue plan. Does not invoke further (depth limit reached).
---
## Pattern 2: Hiring Decision Chain
**CHRO → CFO → COO**
Used when: headcount planning, role approval, capacity vs budget tradeoffs.
```
CHRO initiates:
"Engineering wants to hire 3 senior engineers. Need budget clearance and capacity validation."
CHRO → [INVOKE:cfo|Can we afford 3 senior engineers in H2? Fully loaded ~$180K each.]
[RESPONSE:cfo]
Key finding: Budget supports 2 hires now; 3rd is contingent on Q2 ARR closing above $2.1M.
Supporting data:
- Current H2 headcount budget: $340K remaining
- 2 hires = $270K fully loaded → $70K buffer
- 3 hires = $405K → $65K over; triggers board approval
Confidence: high
Caveat: Based on current runway projection — fundraise close changes this completely.
[/RESPONSE]
CHRO → [INVOKE:coo|Do operations support onboarding 2-3 engineers in H2? Any capacity constraints?]
[CHAIN: chro → cfo → coo]
[RESPONSE:coo]
Key finding: Can absorb 2 comfortably; 3rd requires dedicated onboarding lead or delays velocity 6 weeks.
Supporting data:
- Current eng team: 8 → adding 2 is 25% growth (manageable)
- Onboarding bottleneck: 1 senior eng mentor per 1-2 new hires, currently at capacity
- Dev environment setup: 3-week lead time per hire
Confidence: medium
Caveat: Assumes no concurrent product launches requiring full team focus.
[/RESPONSE]
CHRO synthesis: Recommends 2 hires now, pipeline 3rd for Q3 contingent on ARR milestone.
```
---
## Pattern 3: Product Launch Chain
**CPO → CTO → CMO**
Used when: planning feature launches, assessing readiness, aligning go-to-market.
```
CPO initiates:
"Planning Q3 launch of [Feature X]. Need technical readiness and GTM input."
CPO → [INVOKE:cto|Is Feature X technically ready for Q3 launch? Key risks?]
[RESPONSE:cto]
Key finding: 70% complete; Q3 launch is achievable with scope cut — 2 items must move to Q4.
Supporting data:
- Core feature: complete; performance optimization: 3 weeks remaining
- Infrastructure scaling: needs load testing at 10x current volume
- Scope cuts needed: advanced analytics dashboard, third-party integrations
Confidence: medium
Caveat: Assumes no critical bugs found in load testing; historical rate suggests 1-2 found.
[/RESPONSE]
CPO → [INVOKE:cmo|Given Q3 launch of Feature X (core only, no analytics/integrations), what's the GTM plan?]
[CHAIN: cpo → cto → cmo]
[RESPONSE:cmo]
Key finding: Core-only launch works for existing customers; new logo acquisition needs integrations — recommend phased announcement.
Supporting data:
- Phase 1: Early access for top 20 accounts → testimonials and case studies
- Phase 2: Full launch with integration partners in Q4
- CAC impact: core-only launch reduces new logo conversion rate 15%
Confidence: medium
Caveat: Competitor launched similar feature last month — window for differentiation is narrow.
[/RESPONSE]
```
---
## Pattern 4: Crisis Broadcast
**CEO → All Roles**
Used when: major adverse event, fundraise miss, key customer loss, market disruption.
```
CEO broadcasts:
[BROADCAST:all|Scenario: largest customer (28% of ARR) gives 60-day termination notice.
Each role: what's the immediate impact on your domain and your top recommended action?]
Roles respond INDEPENDENTLY (no agent sees others' responses first):
[RESPONSE:cfo]
Key finding: 28% ARR loss drops runway from 14 to 8 months at current burn.
Supporting data:
- Lost ARR: ~$560K → burn multiple increases from 1.8x to 3.2x
- Bridge options: $500K credit line (pre-approved), defer 2 planned hires
- Fundraise timeline must compress: target term sheet within 60 days
Confidence: high
Caveat: Assumes no secondary customer churn triggered by news.
[/RESPONSE]
[RESPONSE:cro]
Key finding: Pipeline must replace $560K ARR within 6 months to avoid emergency measures.
Supporting data:
- Current qualified pipeline: $1.8M → need 31% close rate (currently 28%)
- 3 deals in late stage can cover 70% of gap if closed this quarter
- Churn risk: 2 other accounts had similar usage patterns — immediate EBR required
Confidence: medium
Caveat: Close rate may drop if word spreads; manage communication carefully.
[/RESPONSE]
[RESPONSE:chro]
Key finding: Hiring freeze recommended immediately; retain all current staff.
Supporting data:
- 4 open reqs should pause pending CFO guidance
- At-risk of counter-poaching: 2 senior engineers, 1 product lead
- Retention bonuses for key staff: ~$80K to secure 12-month commitments
Confidence: high
Caveat: Morale impact of freeze can accelerate attrition if not communicated well.
[/RESPONSE]
[RESPONSE:cpo]
Key finding: Accelerate features that differentiate with remaining accounts; reduce scope to core.
Supporting data:
- Lost customer's feature requests: deprioritize (no longer strategic)
- Features most valued by remaining top 5 accounts: [list from account data]
- Reduce Q3 scope by 30% to free engineering capacity for retention work
Confidence: medium
Caveat: Need sales input on which features are blocking new logo deals.
[/RESPONSE]
[RESPONSE:cmo]
Key finding: Control the narrative; proactive communication to market reduces churn cascade risk.
Supporting data:
- Draft customer communication within 48 hours (frame as strategic pivot, not loss)
- Press: no comment strategy unless proactively asked
- Replace pipeline: double down on ICP segments where we're strongest
Confidence: medium
Caveat: If customer goes public with criticism, narrative control becomes much harder.
[/RESPONSE]
CEO synthesis: [Aggregates all 9 responses, identifies conflicts, sets priorities]
```
---
## When to Invoke vs When to Assume
### Invoke when:
- Cross-functional data is material to the decision
- Getting it wrong changes the recommendation significantly
- The other role has data you genuinely don't have
- Time allows (not in Phase 2 isolation)
### Assume when:
- You're in Phase 2 (always — no exceptions)
- The chain is at depth 2 (you cannot invoke further)
- The answer is directionally obvious (e.g., "CFO will care about runway")
- The precision doesn't change the recommendation
### State assumptions explicitly:
```
[ASSUMPTION: runway ~12 months — not verified with CFO; actual may vary ±20%]
[ASSUMPTION: CAC ~$8K based on industry benchmark — CMO has actual figures]
[ASSUMPTION: engineering capacity at ~70% — not verified with CTO]
```
---
## Handling Conflicting Responses
When two agents give incompatible answers, surface it:
```
[CONFLICT DETECTED]
CFO says: runway extends to 18 months if Q3 targets hit
CRO says: only 45% confidence Q3 targets will be hit
Resolution: use probabilistic blend
- 45% probability: 18-month runway (optimistic case)
- 55% probability: 11-month runway (current trajectory)
Expected value: ~14 months
Recommendation: plan for 12 months, trigger bridge at 10.
[/CONFLICT]
```
**Resolution options:**
1. **Conservative:** Use worse case — appropriate for cash/runway decisions
2. **Probabilistic:** Weight by confidence scores — appropriate for planning
3. **Escalate:** Flag for human decision — appropriate for high-stakes irreversible choices
4. **Time-box:** Gather more data within 48 hours — appropriate when data gap is closeable
---
## Anti-Patterns to Avoid
| Anti-pattern | Problem | Fix |
|---|---|---|
| Invoke to validate your own conclusion | Confirmation bias loop | Ask open-ended questions |
| Invoke when assuming works | Unnecessary latency | State assumption clearly |
| Hide conflicts between responses | Bad synthesis | Always surface conflicts |
| Invoke across depth > 2 | Loop risk | State assumption at depth 2 |
| Invoke during Phase 2 | Groupthink contamination | Flag with [ASSUMPTION:] |
| Vague questions | Poor responses | Specific, scoped questions only |
Phỏng vấn 6 câu hỏi để đánh giá nội bộ hệ thống quản lý AI theo ISO/IEC 42001 trước chứng nhận hoặc kiểm toán.
--- name: "aims-audit" description: "/cs:aims-audit <scope> — ISO/IEC 42001 AIMS internal-audit 6-question forcing interrogation. Use before certification stage 1, before annual internal audit cycles, or when onboarding a new AI system into an existing AIMS." --- # /cs:aims-audit — AIMS ISO 42001 Forcing Questions **Command:** `/cs:aims-audit <scope>` The ISO 42001 AIMS specialist pressure-tests any AI Management System work. Six questions before any certification commitment, internal audit cycle, or new-system onboarding. ## When to Run - Before stage 1 ISO 42001 certification audit - Before annual internal audit cycle (Clause 9.2) - When onboarding a new AI system into existing AIMS scope - When AI risk register hasn't been refreshed in > 6 months - After material model change (re-evaluate risks per Clause 6.1.2) - When audit findings hint at AIMS / ISMS / QMS duplication ## The Six AIMS Questions ### 1. Does the AIMS scope statement name every AI system? **Scope omission = certification finding.** - Including: embedded models, third-party AI services, "experimental" production systems - Run `aims_gap_analyzer.py` to verify Clause 4.3 evidence - "AI features added by SaaS vendors we use" = in scope if they affect the company's services ### 2. Does the AI policy commit to lawful use AND beneficial purpose AND human oversight AND continual improvement? **Missing any of the four = critical nonconformity at stage 1.** - AI policy is NOT info-sec policy — it has separate substantive content - Reference ISO 42001 Annex A.2.2 + Clause 5.2 - Marketing-copy "AI ethics" doesn't pass ### 3. What's the risk register coverage, and which Annex A controls treat each risk? **Risk identification without control mapping = Clause 6.1.3 fails.** - Run `ai_risk_register_builder.py` per ISO 23894 methodology - Every high/critical risk must link to ≥ 1 Annex A control - "Residual verdict: additional_treatment_required" must be closed before stage 1 ### 4. Has the AI risk assessment been re-run since the last material model change? **Concept drift is not a one-time event.** - Article 9 EU AI Act + ISO 42001 Clause 6.1.2 both require iterative risk assessment - Material change = retraining on new data, fine-tuning, architecture change, deployment context change - If "we did it 18 months ago and haven't touched it," the AIMS is broken ### 5. What's the Clause 9.2 internal audit plan, and is auditor independence respected? **Without 9.2 plan, the AIMS is incomplete.** - Run `aims_audit_scheduler.py` with scope + auditors + prior findings - Audit every clause + applicable Annex A control over rolling 3-year cycle - Same auditor cannot audit own work - Cross-check with cs-quality-regulatory if integrated with 13485 audit programme ### 6. Has the AIMS been integrated with existing ISMS / QMS, or built in parallel? **Parallel systems = 5x ongoing maintenance cost.** - 60% of Clauses 4-10 evidence reuses ISO 27001 / 13485 with AI scope appended - CAPA loop should be ONE loop with AI-tagged nonconformities, not separate - Reference `cross_framework_mapping_ai.md` for the reuse map - Cross-check with cs-ciso-advisor on ISO 27001 alignment ## Workflow ```bash # 1. AIMS gap analysis python ../../ra-qm-team/skills/iso42001-specialist/scripts/aims_gap_analyzer.py evidence.json # 2. AI risk register python ../../ra-qm-team/skills/iso42001-specialist/scripts/ai_risk_register_builder.py risks.json # 3. Internal audit plan python ../../ra-qm-team/skills/iso42001-specialist/scripts/aims_audit_scheduler.py audit_scope.json # 4. Cross-framework reuse map (via compliance-os) python ../../skills/compliance-os/scripts/cross_framework_mapper.py program.json ``` ## Output Format ```markdown # AIMS Audit: <scope> **Date:** YYYY-MM-DD ## The Decision Being Made [gap-closure | risk-treatment | audit-scope | new-system-onboarding] ## Gap Analysis (Clauses 4-10) - Weighted coverage: X% - Critical gaps: N - Major gaps: M - Certification readiness: ready | stage_2_candidate | not_ready ## AI Risk Register - Total risks: N - By severity: critical=X, high=Y, medium=Z, low=W - Requires additional treatment: K - Top risk requiring action: <description> ## Clause 9.2 Audit Plan - 12-month coverage: clauses=X, controls=Y - Auditor independence: clean | issues - Prior-year follow-up: scheduled in Q1 ## Cross-Framework Reuse - ISO 27001 evidence reused: % of AIMS Clauses 4-10 - 13485 evidence reused: % (if applicable) - Net-new for AIMS: % (mostly Annex A) ## Verdict 🟢 STAGE-1-READY | 🟡 CLOSE-CRITICALS-FIRST | 🔴 NOT-READY ## Top 3 Actions [3 concrete next steps with owner + date] ``` ## Routing - `/cs:compliance-readiness` — for multi-framework view - `/cs:ai-act-readiness` — if EU AI Act also applies - `/cs:caio-review` — for executive AI strategy decisions - `/cs:ciso-review` — for ISO 27001 cross-framework alignment - `/cs:decide` — to log the verdict - `/cs:freeze 30` — on certification commitments ## Related - Agent: [`cs-aims-iso42001`](../../agents/cs-aims-iso42001.md) - Skill: [`iso42001-specialist`](../../../ra-qm-team/skills/iso42001-specialist/SKILL.md) - Adjacent: `../../skills/compliance-os/`, `../ai-act-readiness/`, `../compliance-readiness/` --- **Version:** 1.0.0
Thiết lập, kiểm tra và gỡ lỗi triển khai theo dõi: GA4, Google Tag Manager, sự kiện, chuyển đổi và chất lượng dữ liệu.
---
name: "analytics-tracking"
description: "Set up, audit, and debug analytics tracking implementation — GA4, Google Tag Manager, event taxonomy, conversion tracking, and data quality. Use when building a tracking plan from scratch, auditing existing analytics for gaps or errors, debugging missing events, or setting up GTM. Trigger keywords: GA4 setup, Google Tag Manager, GTM, event tracking, analytics implementation, conversion tracking, tracking plan, event taxonomy, custom dimensions, UTM tracking, analytics audit, missing events, tracking broken. NOT for analyzing marketing campaign data — use campaign-analytics for that. NOT for BI dashboards — use product-analytics for in-product event analysis."
license: MIT
metadata:
version: 1.0.0
author: Alireza Rezvani
category: marketing
updated: 2026-03-06
---
# Analytics Tracking
You are an expert in analytics implementation. Your goal is to make sure every meaningful action in the customer journey is captured accurately, consistently, and in a way that can actually be used for decisions — not just for the sake of having data.
Bad tracking is worse than no tracking. Duplicate events, missing parameters, unconsented data, and broken conversions lead to decisions made on bad data. This skill is about building it right the first time, or finding what's broken and fixing it.
## Before Starting
**Check for context first:**
If `marketing-context.md` exists, read it before asking questions. Use that context and only ask for what's missing.
Gather this context:
### 1. Current State
- Do you have GA4 and/or GTM already set up? If so, what's broken or missing?
- What's your tech stack? (React SPA, Next.js, WordPress, custom, etc.)
- Do you have a consent management platform (CMP)? Which one?
- What events are you currently tracking (if any)?
### 2. Business Context
- What are your primary conversion actions? (signup, purchase, lead form, free trial start)
- What are your key micro-conversions? (pricing page view, feature discovery, demo request)
- Do you run paid campaigns? (Google Ads, Meta, LinkedIn — affects conversion tracking needs)
### 3. Goals
- Building from scratch, auditing existing, or debugging a specific issue?
- Do you need cross-domain tracking? Multiple properties or subdomains?
- Server-side tagging requirement? (GDPR-sensitive markets, performance concerns)
## How This Skill Works
### Mode 1: Set Up From Scratch
No analytics in place — we'll build the tracking plan, implement GA4 and GTM, define the event taxonomy, and configure conversions.
### Mode 2: Audit Existing Tracking
Tracking exists but you don't trust the data, coverage is incomplete, or you're adding new goals. We'll audit what's there, gap-fill, and clean up.
### Mode 3: Debug Tracking Issues
Specific events are missing, conversion numbers don't add up, or GTM preview shows events firing but GA4 isn't recording them. Structured debugging workflow.
---
## Event Taxonomy Design
Get this right before touching GA4 or GTM. Retrofitting taxonomy is painful.
### Naming Convention
**Format:** `object_action` (snake_case, verb at the end)
| ✅ Good | ❌ Bad |
|--------|--------|
| `form_submit` | `submitForm`, `FormSubmitted`, `form-submit` |
| `plan_selected` | `clickPricingPlan`, `selected_plan`, `PlanClick` |
| `video_started` | `videoPlay`, `StartVideo`, `VideoStart` |
| `checkout_completed` | `purchase`, `buy_complete`, `checkoutDone` |
**Rules:**
- Always `noun_verb` not `verb_noun`
- Lowercase + underscores only — no camelCase, no hyphens
- Be specific enough to be unambiguous, not so verbose it's a sentence
- Consistent tense: `_started`, `_completed`, `_failed` (not mix of past/present)
### Standard Parameters
Every event should include these where applicable:
| Parameter | Type | Example | Purpose |
|-----------|------|---------|---------|
| `page_location` | string | `https://app.co/pricing` | Auto-captured by GA4 |
| `page_title` | string | `Pricing - Acme` | Auto-captured by GA4 |
| `user_id` | string | `usr_abc123` | Link to your CRM/DB |
| `plan_name` | string | `Professional` | Segment by plan |
| `value` | number | `99` | Revenue/order value |
| `currency` | string | `USD` | Required with value |
| `content_group` | string | `onboarding` | Group pages/flows |
| `method` | string | `google_oauth` | How (signup method, etc.) |
### Event Taxonomy for SaaS
**Core funnel events:**
```
visitor_arrived (page view — automatic in GA4)
signup_started (user clicked "Sign up")
signup_completed (account created successfully)
trial_started (free trial began)
onboarding_step_completed (param: step_name, step_number)
feature_activated (param: feature_name)
plan_selected (param: plan_name, billing_period)
checkout_started (param: value, currency, plan_name)
checkout_completed (param: value, currency, transaction_id)
subscription_cancelled (param: cancel_reason, plan_name)
```
**Micro-conversion events:**
```
pricing_viewed
demo_requested (param: source)
form_submitted (param: form_name, form_location)
content_downloaded (param: content_name, content_type)
video_started (param: video_title)
video_completed (param: video_title, percent_watched)
chat_opened
help_article_viewed (param: article_name)
```
See [references/event-taxonomy-guide.md](references/event-taxonomy-guide.md) for the full taxonomy catalog with custom dimension recommendations.
---
## GA4 Setup
### Data Stream Configuration
1. **Create property** in GA4 → Admin → Properties → Create
2. **Add web data stream** with your domain
3. **Enhanced Measurement** — enable all, then review:
- ✅ Page views (keep)
- ✅ Scrolls (keep)
- ✅ Outbound clicks (keep)
- ✅ Site search (keep if you have search)
- ⚠️ Video engagement (disable if you'll track videos manually — avoid duplicates)
- ⚠️ File downloads (disable if you'll track these in GTM for better parameters)
4. **Configure domains** — add all subdomains used in your funnel
### Custom Events in GA4
For any event not auto-collected, create it in GTM (preferred) or via gtag directly:
**Via gtag:**
```javascript
gtag('event', 'signup_completed', {
method: 'email',
user_id: 'usr_abc123',
plan_name: "trial"
});
```
**Via GTM data layer (preferred — see GTM section):**
```javascript
window.dataLayer.push({
event: 'signup_completed',
signup_method: 'email',
user_id: 'usr_abc123'
});
```
### Conversions Configuration
Mark these events as conversions in GA4 → Admin → Conversions:
- `signup_completed`
- `checkout_completed`
- `demo_requested`
- `trial_started` (if separate from signup)
**Rules:**
- Max 30 conversion events per property — curate, don't mark everything
- Conversions are retroactive in GA4 — turning one on applies to 6 months of history
- Don't mark micro-conversions as conversions unless you're optimizing ad campaigns for them
---
## Google Tag Manager Setup
### Container Structure
```
GTM Container
├── Tags
│ ├── GA4 Configuration (fires on all pages)
│ ├── GA4 Event — [event_name] (one tag per event)
│ ├── Google Ads Conversion (per conversion action)
│ └── Meta Pixel (if running Meta ads)
├── Triggers
│ ├── All Pages
│ ├── DOM Ready
│ ├── Data Layer Event — [event_name]
│ └── Custom Element Click — [selector]
└── Variables
├── Data Layer Variables (dlv — for each dL key)
├── Constant — GA4 Measurement ID
└── JavaScript Variables (computed values)
```
### Tag Patterns for SaaS
**Pattern 1: Data Layer Push (most reliable)**
Your app pushes to dataLayer → GTM picks it up → sends to GA4.
```javascript
// In your app code (on event):
window.dataLayer = window.dataLayer || [];
window.dataLayer.push({
event: 'signup_completed',
signup_method: 'email',
user_id: userId,
plan_name: "trial"
});
```
```
GTM Tag: GA4 Event
Event Name: {{DLV - event}} OR hardcode "signup_completed"
Parameters:
signup_method: {{DLV - signup_method}}
user_id: {{DLV - user_id}}
plan_name: "dlv-plan-name"
Trigger: Custom Event - "signup_completed"
```
**Pattern 2: CSS Selector Click**
For events triggered by UI elements without app-level hooks.
```
GTM Trigger:
Type: Click - All Elements
Conditions: Click Element matches CSS selector [data-track="demo-cta"]
GTM Tag: GA4 Event
Event Name: demo_requested
Parameters:
page_location: {{Page URL}}
```
See [references/gtm-patterns.md](references/gtm-patterns.md) for full configuration templates.
---
## Conversion Tracking: Platform-Specific
### Google Ads
1. Create conversion action in Google Ads → Tools → Conversions
2. Import GA4 conversions (recommended — single source of truth) OR use the Google Ads tag
3. Set attribution model: **Data-driven** (if >50 conversions/month), otherwise **Last click**
4. Conversion window: 30 days for lead gen, 90 days for high-consideration purchases
### Meta (Facebook/Instagram) Pixel
1. Install Meta Pixel base code via GTM
2. Standard events: `PageView`, `Lead`, `CompleteRegistration`, `Purchase`
3. Conversions API (CAPI) strongly recommended — client-side pixel loses ~30% of conversions due to ad blockers and iOS
4. CAPI requires server-side implementation (Meta's docs or GTM server-side)
---
## Cross-Platform Tracking
### UTM Strategy
Enforce strict UTM conventions or your channel data becomes noise.
| Parameter | Convention | Example |
|-----------|-----------|---------|
| `utm_source` | Platform name (lowercase) | `google`, `linkedin`, `newsletter` |
| `utm_medium` | Traffic type | `cpc`, `email`, `social`, `organic` |
| `utm_campaign` | Campaign ID or name | `q1-trial-push`, `brand-awareness` |
| `utm_content` | Ad/creative variant | `hero-cta-blue`, `text-link` |
| `utm_term` | Paid keyword | `saas-analytics` |
**Rule:** Never tag organic or direct traffic with UTMs. UTMs override GA4's automatic source/medium attribution.
### Attribution Windows
| Platform | Default Window | Recommended for SaaS |
|---------|---------------|---------------------|
| GA4 | 30 days | 30-90 days depending on sales cycle |
| Google Ads | 30 days | 30 days (trial), 90 days (enterprise) |
| Meta | 7-day click, 1-day view | 7-day click only |
| LinkedIn | 30 days | 30 days |
### Cross-Domain Tracking
For funnels that cross domains (e.g., `acme.com` → `app.acme.com`):
1. In GA4 → Admin → Data Streams → Configure tag settings → List unwanted referrals → Add both domains
2. In GTM → GA4 Configuration tag → Cross-domain measurement → Add both domains
3. Test: visit domain A, click link to domain B, check GA4 DebugView — session should not restart
---
## Data Quality
### Deduplication
**Events firing twice?** Common causes:
- GTM tag + hardcoded gtag both firing
- Enhanced Measurement + custom GTM tag for same event
- SPA router firing pageview on every route change AND GTM page view tag
Fix: Audit GTM Preview for double-fires. Check Network tab in DevTools for duplicate hits.
### Bot Filtering
GA4 filters known bots automatically. For internal traffic:
1. GA4 → Admin → Data Filters → Internal Traffic
2. Add your office IPs and developer IPs
3. Enable filter (starts as testing mode — activate it)
### Consent Management Impact
Under GDPR/ePrivacy, analytics may require consent. Plan for this:
| Consent Mode setting | Impact |
|---------------------|--------|
| **No consent mode** | Visitors who decline cookies → zero data |
| **Basic consent mode** | Visitors who decline → zero data |
| **Advanced consent mode** | Visitors who decline → modeled data (GA4 estimates using consented users) |
**Recommendation:** Implement Advanced Consent Mode via GTM. Requires CMP integration (Cookiebot, OneTrust, Usercentrics, etc.).
Expected consent rate by region: 60-75% EU, 85-95% US.
---
## Proactive Triggers
Surface these without being asked:
- **Events firing on every page load** → Symptom of misconfigured trigger. Flag: duplicate data inflation.
- **No user_id being passed** → You can't connect analytics to your CRM or understand cohorts. Flag for fix.
- **Conversions not matching GA4 vs Ads** → Attribution window mismatch or pixel duplication. Flag for audit.
- **No consent mode configured in EU markets** → Legal exposure and underreported data. Flag immediately.
- **All pages showing as "/(not set)" or generic paths** → SPA routing not handled. GA4 is recording wrong pages.
- **UTM source showing as "direct" for paid campaigns** → UTMs missing or being stripped. Traffic attribution is broken.
---
## Output Artifacts
| When you ask for... | You get... |
|--------------------|-----------|
| "Build a tracking plan" | Event taxonomy table (events + parameters + triggers), GA4 configuration checklist, GTM container structure |
| "Audit my tracking" | Gap analysis vs. standard SaaS funnel, data quality scorecard (0-100), prioritized fix list |
| "Set up GTM" | Tag/trigger/variable configuration for each event, container setup checklist |
| "Debug missing events" | Structured debugging steps using GTM Preview + GA4 DebugView + Network tab |
| "Set up conversion tracking" | Conversion action configuration for GA4 + Google Ads + Meta |
| "Generate tracking plan" | Run `scripts/tracking_plan_generator.py` with your inputs |
---
## Communication
All output follows the structured communication standard:
- **Bottom line first** — what's broken or what needs building before methodology
- **What + Why + How** — every finding has all three
- **Actions have owners and deadlines** — no vague "consider implementing"
- **Confidence tagging** — 🟢 verified / 🟡 estimated / 🔴 assumed
---
## Related Skills
- **campaign-analytics**: Use for analyzing marketing performance and channel ROI. NOT for implementation — use this skill for tracking setup.
- **ab-test-setup**: Use when designing experiments. NOT for event tracking setup (though this skill's events feed A/B tests).
- **analytics-tracking** (this skill): covers setup only. For dashboards and reporting, use campaign-analytics.
- **seo-audit**: Use for technical SEO. NOT for analytics tracking (though both use GA4 data).
- **gdpr-dsgvo-expert**: Use for GDPR compliance posture. This skill covers consent mode implementation; that skill covers the full compliance framework.
FILE:references/debugging-playbook.md
# Tracking Debug Playbook
Step-by-step methodology for diagnosing and fixing analytics tracking issues.
---
## The Debug Mindset
Analytics bugs are harder than code bugs because:
1. They fail silently — no error thrown, just missing data
2. They often only appear in production
3. They can be caused by timing, consent, ad blockers, or just configuration
Work systematically. Don't guess. Verify at each layer before moving to the next.
---
## The Debug Stack (Bottom-Up)
```
Layer 5: GA4 Reports / DebugView ← what you see
Layer 4: GA4 Data Processing ← where it lands
Layer 3: Network Request ← what was sent
Layer 2: GTM / Tag firing ← what GTM did
Layer 1: dataLayer / App code ← what your app pushed
```
When something's missing at Layer 5, start at Layer 1 and verify each layer before going up.
---
## Tool Setup
### GTM Preview Mode
1. GTM → Preview (top right)
2. Enter your site URL → Connect
3. A blue bar appears at the bottom of your site: "Google Tag Manager"
4. GTM Preview panel opens in a separate tab
5. Perform the action you're debugging
6. Check: did the expected tag fire?
**Reading GTM Preview:**
- Left panel: events as they occur (Page View, Click, Custom Event, etc.)
- Middle panel: Tags fired / Tags NOT fired for selected event
- Right panel: Variables values at the time of the event
### GA4 DebugView
1. GA4 → Admin → DebugView
2. Enable debug mode via:
- GTM: add `debug_mode: true` to your GA4 Event tag parameters
- Extension: install "GA Debugger" Chrome extension
- URL parameter: add `?_gl=` or use GA4 debug parameter
3. Perform actions on your site
4. Watch events appear in real-time (10-15 second delay)
### Chrome DevTools — Network Tab
1. Open DevTools → Network
2. Filter by: `collect` or `google-analytics` or `analytics`
3. Perform the action
4. Look for requests to `https://www.google-analytics.com/g/collect`
5. Click the request → Payload tab → view parameters
---
## Common Issues and Fixes
### Issue: Event fires in GTM Preview but not in GA4
**Possible causes:**
1. **Consent mode blocking** — user is in denied state
- Check: In GTM Preview, look at Variables → `Analytics Storage` — is it `denied`?
- Fix: Test with consent granted, or implement Advanced Consent Mode
2. **Filters blocking data** — internal traffic filter is active
- Check: GA4 → Admin → Data Filters — is "Internal Traffic" filter active?
- Fix: Disable filter temporarily, test, then re-enable and exclude your IP correctly
3. **Debug mode not enabled** — DebugView only shows debug-mode traffic
- Check: Is `debug_mode: true` parameter on the GA4 Event tag?
- Fix: Add it, or use the GA4 Debugger Chrome extension
4. **Wrong property** — you're looking at a different GA4 property
- Check: Confirm Measurement ID in GTM matches the GA4 property you're viewing
- Fix: Compare `G-XXXXXXXXXX` in GTM vs. GA4 Data Stream settings
5. **Duplicate GA4 configuration tags** — two config tags = double sessions + weird data
- Check: GTM → Tags → filter by "GA4 Configuration" — more than one?
- Fix: Delete duplicates, keep one with All Pages trigger
---
### Issue: Event not firing in GTM Preview at all
**Diagnosis path:**
**Step 1:** Check the trigger
- Is the trigger for this tag listed under the action in GTM Preview?
- If not: the trigger didn't fire
**Step 2:** Check trigger conditions
- Open the trigger in GTM
- Reproduce the exact scenario step by step
- In GTM Preview, check Variables at the moment the action happened
- Do the variable values match your trigger conditions?
**Step 3:** dataLayer issue (for Custom Event triggers)
- In GTM Preview → select the relevant event in left panel → Variables tab
- Scroll to find `event` — what's the value?
- If event name doesn't match trigger exactly: it won't fire (case-sensitive, exact match)
**Step 4:** Timing issue
- If using "Page View" trigger and element doesn't exist yet: switch to "DOM Ready" or "Window Loaded"
- If SPA: route changes may not trigger "Page View" — use History Change instead
---
### Issue: Parameters showing as (not set) or undefined in GA4
**Step 1:** Verify parameter is in the network request
- DevTools → Network → find GA4 collect request → Payload
- Search for the parameter name (e.g., `plan_name`)
- If not there: GTM variable isn't resolving correctly
**Step 2:** Check the GTM variable
- GTM Preview → find the event → Variables tab
- Find the variable for this parameter (e.g., `DLV - plan_name`)
- What's its value? If `undefined`: the dataLayer push didn't include this key, or key name is wrong
**Step 3:** Check dataLayer push in your app code
- DevTools → Console → type: `dataLayer.filter(e => e.event === 'your_event_name')`
- Inspect the object — is the parameter key present and spelled correctly?
**Step 4:** Check GA4 custom dimension registration
- Some parameters require a registered custom dimension in GA4 to appear in reports
- GA4 → Admin → Custom Definitions → Custom Dimensions
- If parameter isn't registered here: it'll exist in raw data but won't show in Explore reports
---
### Issue: Duplicate events (event fires 2x per action)
**Find the duplicates:**
- GTM Preview → find the action → how many tags with the same name fired?
- DevTools → Network → filter by `collect` → count hits for the action
**Common causes:**
1. **Enhanced Measurement + manual GTM tag**
- e.g., Enhanced Measurement tracks outbound clicks, GTM also has an outbound click tag
- Fix: disable the Enhanced Measurement setting OR remove the GTM tag
2. **Two GTM Configuration tags**
- Each sends its own hits
- Fix: delete one, keep one
3. **SPA router fires pageview + History Change trigger also fires**
- Fix: disable Enhanced Measurement pageview, use only History Change tag
4. **Event fires on multiple triggers that both match**
- Fix: make triggers more specific — add exclusion conditions
---
### Issue: Sessions/users look wrong (too high or too low)
**Too many sessions:**
- Multiple GA4 Configuration tags
- History Change trigger firing + Enhanced Measurement pageview on SPA
- Client ID not persisting (cookie being blocked or cleared)
**Too few sessions / users:**
- Consent blocking analytics for non-consenting users (expected under strict consent mode)
- Bot filtering too aggressive
- GA4 tags firing on wrong pages only
**Sessions reset unexpectedly (user shows as new on every page):**
- Cross-domain tracking not configured
- Cookie domain mismatch
- GTM cookie settings incorrect
---
### Issue: Conversions not matching between GA4 and Google Ads
**Check 1: Attribution window mismatch**
- GA4 default: 30-day last click
- Google Ads: check conversion action settings for window
- These legitimately produce different numbers
**Check 2: Conversion event names**
- In Google Ads → Tools → Conversions → imported from GA4
- Does the linked event name exactly match the GA4 event?
**Check 3: Import is linked**
- Google Ads → Tools → Linked Accounts → Google Analytics 4
- Is the correct GA4 property linked and synced?
- Sync can take 24-48 hours after changes
**Check 4: Enhanced Conversions**
- If GA4 uses a user_id or email parameter, Enhanced Conversions can improve matching
- Google Ads → Conversions → Enhanced Conversions for Web → Enable
---
## Debug Checklist Template
Use this for any new tracking issue:
```
[ ] Confirmed exact event name and parameters expected
[ ] Verified app code is pushing to dataLayer (console: dataLayer)
[ ] GTM Preview: trigger fires at correct moment
[ ] GTM Preview: parameters resolve to correct values (not undefined)
[ ] Network: GA4 collect request appears with correct payload
[ ] GA4 DebugView: event appears within 30 seconds
[ ] GA4 DebugView: parameters present and correct
[ ] GA4 Reports: event appears (24-48h delay for standard reports)
[ ] Consent check: tested with analytics consent granted
[ ] Filter check: internal traffic filter not blocking test traffic
```
FILE:references/event-taxonomy-guide.md
# Event Taxonomy Guide
Complete reference for naming conventions, event structure, and parameter standards.
---
## Why Taxonomy Matters
Analytics data is only as good as its naming consistency. A tracking system with `FormSubmit`, `form_submit`, `form-submitted`, and `formSubmitted` as four separate "events" is useless for aggregation. One naming standard, enforced from day one, avoids months of cleanup later.
This guide is the reference for that standard.
---
## Naming Convention: Full Specification
### Format
```
[object]_[action]
```
**Object** = the thing being acted upon (noun)
**Action** = what happened (verb, past tense or gerund)
### Casing & Characters
| Rule | ✅ Correct | ❌ Wrong |
|------|-----------|---------|
| Lowercase only | `video_started` | `Video_Started`, `VIDEO_STARTED` |
| Underscores only | `form_submit` | `form-submit`, `formSubmit` |
| Noun before verb | `plan_selected` | `selected_plan` |
| Past tense or clear state | `checkout_completed` | `checkout_complete`, `checkoutDone` |
| Specific > generic | `trial_started` | `event_triggered` |
| Max 4 words | `onboarding_step_completed` | `user_completed_an_onboarding_step_in_the_flow` |
### Action Vocabulary (Standard Verbs)
Use these verbs consistently — don't invent synonyms:
| Verb | Use for |
|------|---------|
| `_started` | Beginning of a multi-step process |
| `_completed` | Successful completion of a process |
| `_failed` | An attempt that errored out |
| `_submitted` | Form or data submission |
| `_viewed` | Passive view of a page, modal, or content |
| `_clicked` | Direct click on a specific element |
| `_selected` | Choosing from options (plan, variant, filter) |
| `_opened` | Modal, drawer, chat window opened |
| `_closed` | Modal, drawer, chat window closed |
| `_downloaded` | File download |
| `_activated` | Feature turned on for first time |
| `_upgraded` | Plan or feature upgrade |
| `_cancelled` | Intentional termination |
| `_dismissed` | User explicitly closed/ignored a prompt |
| `_searched` | Search query submitted |
---
## Complete SaaS Event Catalog
### Acquisition Events
| Event | Required Parameters | Optional Parameters |
|-------|-------------------|-------------------|
| `ad_clicked` | `utm_source`, `utm_campaign` | `utm_content`, `utm_term` |
| `landing_page_viewed` | `page_location`, `utm_source` | `variant` (A/B) |
| `pricing_viewed` | `page_location` | `referrer_page` |
| `demo_requested` | `source` (page slug or section) | `plan_interest` |
| `content_downloaded` | `content_name`, `content_type` | `gated` (boolean) |
### Acquisition → Registration
| Event | Required Parameters | Optional Parameters |
|-------|-------------------|-------------------|
| `signup_started` | — | `plan_name`, `method` |
| `signup_completed` | `method` | `user_id`, `plan_name` |
| `email_verified` | — | `method` |
| `trial_started` | `plan_name` | `trial_length_days` |
| `invitation_accepted` | `inviter_user_id` | `plan_name` |
### Onboarding Events
| Event | Required Parameters | Optional Parameters |
|-------|-------------------|-------------------|
| `onboarding_started` | — | `onboarding_variant` |
| `onboarding_step_completed` | `step_name`, `step_number` | `time_spent_seconds` |
| `onboarding_completed` | `steps_total` | `time_to_complete_seconds` |
| `onboarding_skipped` | `step_name` | `step_number` |
| `feature_activated` | `feature_name` | `activation_method` |
| `integration_connected` | `integration_name` | `integration_type` |
| `team_member_invited` | — | `invite_method` |
### Conversion Events
| Event | Required Parameters | Optional Parameters |
|-------|-------------------|-------------------|
| `plan_selected` | `plan_name`, `billing_period` | `previous_plan` |
| `checkout_started` | `plan_name`, `value`, `currency` | `billing_period` |
| `checkout_completed` | `plan_name`, `value`, `currency`, `transaction_id` | `billing_period`, `coupon_code` |
| `checkout_failed` | `plan_name`, `error_reason` | `value`, `currency` |
| `upgrade_completed` | `from_plan`, `to_plan`, `value`, `currency` | `trigger` |
| `coupon_applied` | `coupon_code`, `discount_value` | `plan_name` |
### Engagement Events
| Event | Required Parameters | Optional Parameters |
|-------|-------------------|-------------------|
| `feature_used` | `feature_name` | `feature_area`, `usage_count` |
| `search_performed` | `search_term` | `results_count`, `search_area` |
| `filter_applied` | `filter_name`, `filter_value` | `result_count` |
| `export_completed` | `export_type`, `export_format` | `record_count` |
| `report_generated` | `report_name` | `date_range` |
| `notification_clicked` | `notification_type` | `notification_id` |
### Retention Events
| Event | Required Parameters | Optional Parameters |
|-------|-------------------|-------------------|
| `subscription_cancelled` | `cancel_reason` | `plan_name`, `save_offer_shown`, `save_offer_accepted` |
| `save_offer_accepted` | `offer_type` | `plan_name`, `discount_pct` |
| `subscription_paused` | `pause_duration_days` | `pause_reason` |
| `subscription_reactivated` | — | `plan_name`, `days_since_cancel` |
| `churn_risk_detected` | — | `risk_score`, `risk_signals` |
### Support / Help Events
| Event | Required Parameters | Optional Parameters |
|-------|-------------------|-------------------|
| `help_article_viewed` | `article_name` | `article_id`, `source` |
| `chat_opened` | — | `page_location`, `trigger` |
| `support_ticket_submitted` | `ticket_category` | `severity` |
| `error_encountered` | `error_type`, `error_message` | `page_location`, `feature_name` |
---
## Custom Dimensions & Metrics
GA4 limits: 50 custom dimensions (event-scoped), 25 user-scoped, 50 item-scoped.
Prioritize the ones that matter for segmentation.
### Recommended User-Scoped Dimensions
| Dimension Name | Parameter | Example Values |
|---------------|-----------|---------------|
| User ID | `user_id` | `usr_abc123` |
| Plan Name | `plan_name` | `starter`, `professional`, `enterprise` |
| Billing Period | `billing_period` | `monthly`, `annual` |
| Account Created Date | `account_created_date` | `2024-03-15` |
| Onboarding Completed | `onboarding_completed` | `true`, `false` |
| Company Size | `company_size` | `1-10`, `11-50`, `51-200` |
### Recommended Event-Scoped Dimensions
| Dimension Name | Parameter | Used In |
|---------------|-----------|---------|
| Cancel Reason | `cancel_reason` | `subscription_cancelled` |
| Feature Name | `feature_name` | `feature_used`, `feature_activated` |
| Content Name | `content_name` | `content_downloaded` |
| Signup Method | `method` | `signup_completed` |
| Error Type | `error_type` | `error_encountered` |
---
## Taxonomy Governance
### The Tracking Plan Document
Maintain a single tracking plan document (Google Sheet or Notion table) with:
| Column | Values |
|--------|--------|
| Event Name | e.g., `checkout_completed` |
| Trigger | "User completes Stripe checkout" |
| Parameters | `{value, currency, plan_name, transaction_id}` |
| Implemented In | GTM / App code / server |
| Status | Draft / Implemented / Verified |
| Owner | Engineering / Marketing / Product |
### Change Protocol
1. New events → add to tracking plan first, get sign-off before implementing
2. Rename events → use a deprecation period (keep old + add new for 30 days, then remove old)
3. Remove events → archive in tracking plan, don't delete — historical data reference
4. Add parameters → non-breaking, implement immediately and update tracking plan
5. Remove parameters → treat as rename (deprecation period)
### Versioning
Include `schema_version` as a parameter on critical events if your taxonomy evolves rapidly:
```javascript
window.dataLayer.push({
event: 'checkout_completed',
schema_version: 'v2',
value: 99,
currency: 'USD',
// ...
});
```
This allows filtering old vs. new schema during migrations.
FILE:references/gtm-patterns.md
# GTM Patterns for SaaS
Common Google Tag Manager configurations for SaaS applications.
---
## Container Architecture
### Naming Convention
Use consistent naming or GTM becomes a black box within 6 months.
```
Tags: [Platform] - [Event Name] e.g., "GA4 - signup_completed"
Triggers: [Type] - [Description] e.g., "DL Event - signup_completed"
Variables: [Type] - [Parameter Name] e.g., "DLV - plan_name"
```
### Required Variables (Create These First)
| Variable Name | Type | Value |
|--------------|------|-------|
| `CON - GA4 Measurement ID` | Constant | `G-XXXXXXXXXX` |
| `CON - Environment` | Constant | `production` |
| `JS - Page Path` | Custom JavaScript | `function() { return window.location.pathname; }` |
| `JS - User ID` | Custom JavaScript | `function() { return window.currentUserId || undefined; }` |
### GA4 Configuration Tag
**One tag, fires on All Pages:**
```
Tag Type: Google Analytics: GA4 Configuration
Measurement ID: {{CON - GA4 Measurement ID}}
Fields to Set:
- user_id: {{JS - User ID}}
Trigger: All Pages
```
---
## Pattern Library
### Pattern 1: Data Layer Push Event
The most reliable pattern. Your app pushes structured data; GTM listens.
**In your application code:**
```javascript
// Call this function on any trackable event
function trackEvent(eventName, parameters) {
window.dataLayer = window.dataLayer || [];
window.dataLayer.push({
event: eventName,
...parameters
});
}
// Example: after successful signup
trackEvent('signup_completed', {
signup_method: 'email',
user_id: newUser.id,
plan_name: 'trial'
});
```
**In GTM:**
1. Create Data Layer Variables for each parameter:
- `DLV - signup_method` → Data Layer Variable → `signup_method`
- `DLV - user_id` → Data Layer Variable → `user_id`
- `DLV - plan_name` → Data Layer Variable → `plan_name`
2. Create Trigger:
- Type: Custom Event
- Event Name: `signup_completed`
- Name: `DL Event - signup_completed`
3. Create Tag:
- Type: Google Analytics: GA4 Event
- Configuration Tag: GA4 Config tag
- Event Name: `signup_completed`
- Event Parameters:
- `method`: `{{DLV - signup_method}}`
- `user_id`: `{{DLV - user_id}}`
- `plan_name`: `{{DLV - plan_name}}`
- Trigger: `DL Event - signup_completed`
---
### Pattern 2: Click Event on Specific Element
Use when you can't modify app code and need to track a specific CTA.
**GTM Setup:**
1. Enable `Click - All Elements` built-in variables (if not enabled):
- GTM → Variables → Configure → Enable: Click Element, Click ID, Click Classes, Click Text
2. Create Trigger:
- Type: Click - All Elements
- Fire On: Some Clicks
- Conditions:
- Click Element matches CSS selector: `[data-track="demo-cta"]`
OR
- Click Text equals "Request a Demo"
- Name: `Click - Demo CTA`
3. Create Tag:
- Type: GA4 Event
- Event Name: `demo_requested`
- Event Parameters:
- `page_location`: `{{Page URL}}`
- `click_text`: `{{Click Text}}`
- Trigger: `Click - Demo CTA`
**Best practice:** Add `data-track` attributes to important elements in your HTML rather than relying on brittle CSS selectors or text matching.
```html
<button data-track="demo-cta" data-track-source="pricing-hero">
Request a Demo
</button>
```
---
### Pattern 3: Form Submission Tracking
Two approaches depending on whether the form submits via JavaScript or full page reload.
**For JavaScript-handled forms (AJAX/fetch):**
- Use Pattern 1 (dataLayer push) after successful form submission callback
**For traditional form submit:**
1. Create Trigger:
- Type: Form Submission
- Check Validation: ✅ (only fires if form passes HTML5 validation)
- Enable History Change: ✅ (for SPAs)
- Fire On: Some Forms
- Conditions: Form ID equals `contact-form` OR Form Classes contains `js-track-form`
- Name: `Form Submit - Contact`
2. Create Tag:
- Type: GA4 Event
- Event Name: `form_submitted`
- Parameters:
- `form_name`: `contact`
- `page_location`: `{{Page URL}}`
- Trigger: `Form Submit - Contact`
---
### Pattern 4: SPA Page View Tracking
Single-page apps often don't trigger standard page view events on route changes.
**Approach A: History Change trigger (simplest)**
1. Create Trigger:
- Type: History Change
- Name: `History Change - Route`
2. Create Tag:
- Type: GA4 Event
- Event Name: `page_view`
- Parameters:
- `page_location`: `{{Page URL}}`
- `page_title`: `{{Page Title}}`
- Trigger: `History Change - Route`
**Important:** Disable the default pageview in your GA4 Configuration tag if using this, or you'll get duplicates on initial load.
**Approach B: dataLayer push from router (more reliable)**
```javascript
// In your router's navigation handler:
router.afterEach((to, from) => {
window.dataLayer.push({
event: 'page_view',
page_path: to.path,
page_title: document.title
});
});
```
---
### Pattern 5: Scroll Depth Tracking
For content engagement measurement:
**Option A: Use GA4 Enhanced Measurement (90% depth only)**
- Enable in GA4 → Data Streams → Enhanced Measurement → Scrolls
- Fires when user scrolls 90% down the page
- No GTM configuration needed
**Option B: Custom milestones via GTM**
1. Create Trigger for each depth:
- Type: Scroll Depth
- Vertical Scroll Depths: 25, 50, 75, 100 (percent)
- Enable for: Some Pages → Page Path contains `/blog/`
- Name: `Scroll Depth - Blog`
2. Create Tag:
- Type: GA4 Event
- Event Name: `content_scrolled`
- Parameters:
- `scroll_depth_pct`: `{{Scroll Depth Threshold}}`
- `page_location`: `{{Page URL}}`
- Trigger: `Scroll Depth - Blog`
---
### Pattern 6: Consent Mode Integration
For GDPR compliance — connect your CMP to GTM.
**Basic Consent Mode (blocks all when declined):**
```javascript
// In your CMP callback:
window.dataLayer.push({
event: 'cookie_consent_update',
ad_storage: 'denied', // or 'granted'
analytics_storage: 'denied', // or 'granted'
functionality_storage: 'denied',
personalization_storage: 'denied',
security_storage: 'granted' // always granted
});
```
**Advanced Consent Mode (modeled data for declined users):**
Add to `<head>` BEFORE GTM loads:
```javascript
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
// Default all to denied
gtag('consent', 'default', {
ad_storage: 'denied',
analytics_storage: 'denied',
wait_for_update: 500 // ms to wait for CMP to initialize
});
```
Then update when user consents:
```javascript
gtag('consent', 'update', {
analytics_storage: 'granted'
});
```
---
## GTM Version Control
### Version Naming Convention
```
v1.0 - Initial setup: GA4 + core events
v1.1 - Add: checkout tracking
v1.2 - Fix: duplicate pageview on SPA
v2.0 - Overhaul: new event taxonomy + Meta Pixel
```
### Publishing Protocol
1. Test in GTM Preview mode — verify events fire correctly
2. Test in GA4 DebugView — confirm parameters are captured
3. Test with GTM's "What changed?" diff view
4. Add version notes (what changed + why)
5. Publish to production
6. Verify in GA4 Realtime view post-publish
### Environments
Create a staging environment in GTM (Admin → Environments):
- Development: test changes without affecting production
- Staging: validate before publish
- Production: live
Share staging GTM snippet with your dev team so they test against the same container.
---
## Common GTM Mistakes
| Mistake | Symptom | Fix |
|---------|---------|-----|
| Tag fires on "All Pages" when it should be scoped | Inflated event counts | Add page conditions to trigger |
| Data Layer Variable path is wrong | Parameter shows as `undefined` | Use GTM Preview to inspect dataLayer structure |
| GA4 Configuration tag fires multiple times | Duplicate sessions/users | Check all triggers — should be one trigger, "All Pages" |
| Enhanced Measurement conflicts with custom tags | Duplicate outbound click events | Disable conflicting Enhanced Measurement settings |
| Trigger fires before DOM ready | Element not found errors | Change trigger type from "Page View" to "DOM Ready" or "Window Loaded" |
| Form trigger doesn't fire | Form uses AJAX or custom submit | Switch to dataLayer push after submit callback |
FILE:scripts/tracking_plan_generator.py
#!/usr/bin/env python3
"""Tracking plan generator — produces event taxonomy, GTM config, and GA4 dimension recommendations."""
import json
import sys
from collections import defaultdict
SAMPLE_INPUT = {
"business_type": "saas",
"key_pages": [
{"name": "Homepage", "path": "/"},
{"name": "Pricing", "path": "/pricing"},
{"name": "Signup", "path": "/signup"},
{"name": "Dashboard", "path": "/app/dashboard"},
{"name": "Onboarding", "path": "/app/onboarding"}
],
"conversion_actions": [
{"name": "Signup", "type": "registration", "value": 0},
{"name": "Trial Start", "type": "trial", "value": 0},
{"name": "Subscription Purchase", "type": "purchase", "value": 99},
{"name": "Demo Request", "type": "lead", "value": 0}
],
"paid_channels": ["google_ads", "meta"],
"consent_required": True
}
EVENT_TEMPLATES = {
"saas": {
"acquisition": [
{
"event": "pricing_viewed",
"trigger": "User navigates to /pricing",
"parameters": ["page_location", "utm_source", "referrer_page"],
"priority": "high"
},
{
"event": "demo_requested",
"trigger": "User submits demo request form",
"parameters": ["source", "page_location", "form_name"],
"priority": "high",
"is_conversion": True
},
{
"event": "content_downloaded",
"trigger": "User downloads gated content",
"parameters": ["content_name", "content_type", "gated"],
"priority": "medium"
}
],
"registration": [
{
"event": "signup_started",
"trigger": "User clicks primary signup CTA",
"parameters": ["page_location", "cta_text", "plan_name"],
"priority": "high"
},
{
"event": "signup_completed",
"trigger": "User account successfully created",
"parameters": ["method", "user_id", "plan_name"],
"priority": "critical",
"is_conversion": True
},
{
"event": "trial_started",
"trigger": "Free trial begins",
"parameters": ["plan_name", "trial_length_days", "user_id"],
"priority": "critical",
"is_conversion": True
}
],
"onboarding": [
{
"event": "onboarding_started",
"trigger": "User enters onboarding flow",
"parameters": ["user_id", "onboarding_variant"],
"priority": "high"
},
{
"event": "onboarding_step_completed",
"trigger": "User completes each onboarding step",
"parameters": ["step_name", "step_number", "user_id", "time_spent_seconds"],
"priority": "high"
},
{
"event": "onboarding_completed",
"trigger": "User completes full onboarding",
"parameters": ["steps_total", "user_id", "time_to_complete_seconds"],
"priority": "high"
},
{
"event": "feature_activated",
"trigger": "User activates a key feature for first time",
"parameters": ["feature_name", "user_id", "activation_method"],
"priority": "medium"
}
],
"conversion": [
{
"event": "plan_selected",
"trigger": "User clicks on a pricing plan",
"parameters": ["plan_name", "billing_period", "value"],
"priority": "critical"
},
{
"event": "checkout_started",
"trigger": "User enters checkout flow",
"parameters": ["plan_name", "value", "currency", "billing_period"],
"priority": "critical"
},
{
"event": "checkout_completed",
"trigger": "Payment successfully processed",
"parameters": ["plan_name", "value", "currency", "transaction_id", "billing_period"],
"priority": "critical",
"is_conversion": True
}
],
"retention": [
{
"event": "subscription_cancelled",
"trigger": "User confirms cancellation",
"parameters": ["cancel_reason", "plan_name", "save_offer_shown", "save_offer_accepted"],
"priority": "high"
},
{
"event": "subscription_reactivated",
"trigger": "Cancelled user reactivates",
"parameters": ["plan_name", "days_since_cancel"],
"priority": "high"
}
]
},
"ecommerce": {
"acquisition": [
{
"event": "product_viewed",
"trigger": "User views a product page",
"parameters": ["item_id", "item_name", "item_category", "value"],
"priority": "high"
},
{
"event": "search_performed",
"trigger": "User submits a search query",
"parameters": ["search_term", "results_count"],
"priority": "medium"
}
],
"conversion": [
{
"event": "add_to_cart",
"trigger": "User adds item to cart",
"parameters": ["item_id", "item_name", "value", "currency", "quantity"],
"priority": "critical"
},
{
"event": "checkout_started",
"trigger": "User begins checkout",
"parameters": ["value", "currency", "num_items"],
"priority": "critical"
},
{
"event": "checkout_completed",
"trigger": "Order placed successfully",
"parameters": ["transaction_id", "value", "currency", "tax", "shipping"],
"priority": "critical",
"is_conversion": True
}
]
}
}
CUSTOM_DIMENSIONS = {
"user_scoped": [
{"name": "User ID", "parameter": "user_id", "description": "Internal user identifier"},
{"name": "Plan Name", "parameter": "plan_name", "description": "Current subscription plan"},
{"name": "Billing Period", "parameter": "billing_period", "description": "Monthly or annual"},
{"name": "Signup Method", "parameter": "signup_method", "description": "Email, Google, SSO"},
{"name": "Onboarding Status", "parameter": "onboarding_completed", "description": "Boolean: completed onboarding?"}
],
"event_scoped": [
{"name": "Cancel Reason", "parameter": "cancel_reason", "description": "Exit survey selection"},
{"name": "Feature Name", "parameter": "feature_name", "description": "Feature being used/activated"},
{"name": "Form Name", "parameter": "form_name", "description": "Which form was submitted"},
{"name": "Content Name", "parameter": "content_name", "description": "Downloaded/viewed content"},
{"name": "Error Type", "parameter": "error_type", "description": "Type of error encountered"}
]
}
def generate_tracking_plan(inputs):
biz_type = inputs.get("business_type", "saas")
templates = EVENT_TEMPLATES.get(biz_type, EVENT_TEMPLATES["saas"])
paid = inputs.get("paid_channels", [])
consent = inputs.get("consent_required", False)
conversions = inputs.get("conversion_actions", [])
# Build event taxonomy
all_events = []
for category, events in templates.items():
for ev in events:
all_events.append({**ev, "category": category})
# Add conversion-specific events from input
conversion_events = []
for ca in conversions:
if ca["type"] == "purchase":
for ev in all_events:
if ev["event"] == "checkout_completed":
ev["value_hint"] = ca["value"]
conversion_events.append("checkout_completed")
elif ca["type"] == "registration":
conversion_events.append("signup_completed")
elif ca["type"] == "lead":
conversion_events.append("demo_requested")
elif ca["type"] == "trial":
conversion_events.append("trial_started")
# GTM tag configuration
gtm_tags = []
for ev in all_events:
gtm_tags.append({
"tag_name": f"GA4 - {ev['event']}",
"tag_type": "ga4_event",
"event_name": ev["event"],
"trigger": f"DL Event - {ev['event']}",
"parameters": ev["parameters"],
"priority": ev.get("priority", "medium")
})
# Add platform-specific tags
if "google_ads" in paid:
for ev in all_events:
if ev.get("is_conversion"):
gtm_tags.append({
"tag_name": f"Google Ads - {ev['event']}",
"tag_type": "google_ads_conversion",
"event_name": ev["event"],
"trigger": f"DL Event - {ev['event']}",
"note": "Import from GA4 conversions (preferred) or configure conversion ID"
})
if "meta" in paid:
gtm_tags.append({
"tag_name": "Meta Pixel - Base",
"tag_type": "html_tag",
"trigger": "All Pages",
"note": "Meta base pixel — fires on all pages. Add Standard Events separately."
})
# Consent configuration
consent_config = None
if consent:
consent_config = {
"mode": "advanced",
"defaults": {
"analytics_storage": "denied",
"ad_storage": "denied",
"functionality_storage": "denied"
},
"update_trigger": "cookie_consent_update",
"note": "Implement before GTM loads. Requires CMP integration (Cookiebot, OneTrust, etc.)."
}
return {
"event_taxonomy": [
{
"category": ev["category"],
"event": ev["event"],
"trigger": ev["trigger"],
"parameters": ev["parameters"],
"priority": ev.get("priority", "medium"),
"is_conversion": ev.get("is_conversion", False)
}
for ev in all_events
],
"conversion_events": list(set(conversion_events)),
"gtm_configuration": {
"tags": gtm_tags,
"variable_count": len(set(p for ev in all_events for p in ev["parameters"])),
"trigger_count": len(all_events)
},
"ga4_custom_dimensions": CUSTOM_DIMENSIONS,
"consent_mode": consent_config,
"implementation_order": [
"1. Register custom dimensions in GA4 (Admin > Custom Definitions)",
"2. Set up GTM container structure (variables first, then triggers, then tags)",
"3. Implement dataLayer pushes in application code",
"4. Test each event in GTM Preview + GA4 DebugView",
"5. Mark conversion events in GA4 (Admin > Conversions)",
"6. Link GA4 to Google Ads if running paid search",
"7. Enable internal traffic filter",
"8. Implement consent mode if required"
]
}
def print_report(result, inputs):
print("\n" + "="*65)
print(" TRACKING PLAN GENERATOR")
print("="*65)
print(f"\n📋 BUSINESS TYPE: {inputs.get('business_type', 'saas').upper()}")
events = result["event_taxonomy"]
by_priority = defaultdict(list)
for ev in events:
by_priority[ev["priority"]].append(ev)
print(f"\n📊 EVENT TAXONOMY ({len(events)} events)")
for priority in ["critical", "high", "medium", "low"]:
evs = by_priority.get(priority, [])
if evs:
marker = "🔴" if priority == "critical" else "🟡" if priority == "high" else "⚪"
print(f"\n {marker} {priority.upper()} ({len(evs)} events)")
for ev in evs:
conv = " ← CONVERSION" if ev["is_conversion"] else ""
print(f" {ev['event']}{conv}")
print(f" Params: {', '.join(ev['parameters'][:4])}" +
(f"... +{len(ev['parameters'])-4} more" if len(ev['parameters']) > 4 else ""))
conversions = result["conversion_events"]
print(f"\n🎯 CONVERSION EVENTS ({len(conversions)})")
for ev in conversions:
print(f" • {ev}")
dims = result["ga4_custom_dimensions"]
print(f"\n📐 CUSTOM DIMENSIONS")
print(f" User-scoped ({len(dims['user_scoped'])}): " +
", ".join(d["parameter"] for d in dims["user_scoped"]))
print(f" Event-scoped ({len(dims['event_scoped'])}): " +
", ".join(d["parameter"] for d in dims["event_scoped"]))
gtm = result["gtm_configuration"]
print(f"\n🏷️ GTM CONFIGURATION")
print(f" Tags to create: {len(gtm['tags'])}")
print(f" Triggers to create: {gtm['trigger_count']}")
print(f" Variables to create:{gtm['variable_count']}")
if result["consent_mode"]:
print(f"\n🔒 CONSENT MODE: Advanced (required)")
print(f" Default state: analytics_storage=denied, ad_storage=denied")
print(f"\n📋 IMPLEMENTATION ORDER")
for step in result["implementation_order"]:
print(f" {step}")
print("\n" + "="*65)
print(" Run with --json flag to output full config as JSON")
print("="*65 + "\n")
def main():
import argparse
parser = argparse.ArgumentParser(
description="Tracking plan generator — produces event taxonomy, GTM config, and GA4 dimension recommendations."
)
parser.add_argument(
"input_file", nargs="?", default=None,
help="JSON file with business config (default: run with sample SaaS data)"
)
parser.add_argument(
"--json", action="store_true",
help="Output full config as JSON"
)
args = parser.parse_args()
if args.input_file:
with open(args.input_file) as f:
inputs = json.load(f)
else:
if not args.json:
print("No input file provided. Running with sample data...\n")
inputs = SAMPLE_INPUT
result = generate_tracking_plan(inputs)
print_report(result, inputs)
if args.json:
print(json.dumps(result, indent=2))
if __name__ == "__main__":
main()
Quản trị Jira, Confluence, Bitbucket, Trello: người dùng, phân quyền, bảo mật, tích hợp và cấu hình hệ thống.
---
name: "atlassian-admin"
description: Atlassian Administrator for managing and organizing Atlassian products (Jira, Confluence, Bitbucket, Trello), users, permissions, security, integrations, system configuration, and org-wide governance. Use when asked to add users to Jira, change Confluence permissions, configure access control, update admin settings, manage Atlassian groups, set up SSO, install marketplace apps, review security policies, or handle any org-wide Atlassian administration task.
---
# Atlassian Administrator Expert
## Workflows
### User Provisioning
1. Create user account: `admin.atlassian.com > User management > Invite users`
- REST API: `POST /rest/api/3/user` with `{"emailAddress": "...", "displayName": "...","products": [...]}`
2. Add to appropriate groups: `admin.atlassian.com > User management > Groups > [group] > Add members`
3. Assign product access (Jira, Confluence) via `admin.atlassian.com > Products > [product] > Access`
4. Configure default permissions per group scheme
5. Send welcome email with onboarding info
6. **NOTIFY**: Relevant team leads of new member
7. **VERIFY**: Confirm user appears active at `admin.atlassian.com/o/{orgId}/users` and can log in
### User Deprovisioning
1. **CRITICAL**: Audit user's owned content and tickets
- Jira: `GET /rest/api/3/search?jql=assignee={accountId}` to find open issues
- Confluence: `GET /wiki/rest/api/user/{accountId}/property` to find owned spaces/pages
2. Reassign ownership of:
- Jira projects: `Project settings > People > Change lead`
- Confluence spaces: `Space settings > Overview > Edit space details`
- Open issues: bulk reassign via `Jira > Issues > Bulk change`
- Filters and dashboards: transfer via `User management > [user] > Managed content`
3. Remove from all groups: `admin.atlassian.com > User management > [user] > Groups`
4. Revoke product access
5. Deactivate account: `admin.atlassian.com > User management > [user] > Deactivate`
- REST API: `DELETE /rest/api/3/user?accountId={accountId}`
6. **VERIFY**: Confirm `GET /rest/api/3/user?accountId={accountId}` returns `"active": false`
7. Document deprovisioning in audit log
8. **USE**: Jira Expert to reassign any remaining issues
### Group Management
1. Create groups: `admin.atlassian.com > User management > Groups > Create group`
- REST API: `POST /rest/api/3/group` with `{"name": "..."}`
- Structure by: Teams (engineering, product, sales), Roles (admins, users, viewers), Projects (project-alpha-team)
2. Define group purpose and membership criteria (document in Confluence)
3. Assign default permissions per group
4. Add users to appropriate groups
5. **VERIFY**: Confirm group members via `GET /rest/api/3/group/member?groupName={name}`
6. Regular review and cleanup (quarterly)
7. **USE**: Confluence Expert to document group structure
### Permission Scheme Design
**Jira Permission Schemes** (`Jira Settings > Issues > Permission Schemes`):
- **Public Project**: All users can view, members can edit
- **Team Project**: Team members full access, stakeholders view
- **Restricted Project**: Named individuals only
- **Admin Project**: Admins only
**Confluence Permission Schemes** (`Confluence Admin > Space permissions`):
- **Public Space**: All users view, space members edit
- **Team Space**: Team-specific access
- **Personal Space**: Individual user only
- **Restricted Space**: Named individuals and groups
**Best Practices**:
- Use groups, not individual permissions
- Principle of least privilege
- Regular permission audits
- Document permission rationale
### SSO Configuration
1. Choose identity provider (Okta, Azure AD, Google)
2. Configure SAML settings: `admin.atlassian.com > Security > SAML single sign-on > Add SAML configuration`
- Set Entity ID, ACS URL, and X.509 certificate from IdP
3. Test SSO with admin account (keep password login active during test)
4. Test with regular user account
5. Enable SSO for organization
6. Enforce SSO: `admin.atlassian.com > Security > Authentication policies > Enforce SSO`
7. Configure SCIM for auto-provisioning: `admin.atlassian.com > User provisioning > [IdP] > Enable SCIM`
8. **VERIFY**: Confirm SSO flow succeeds and audit logs show `saml.login.success` events
9. Monitor SSO logs: `admin.atlassian.com > Security > Audit log > filter: SSO`
### Marketplace App Management
1. Evaluate app need and security: check vendor's security self-assessment at `marketplace.atlassian.com`
2. Review vendor security documentation (penetration test reports, SOC 2)
3. Test app in sandbox environment
4. Purchase or request trial: `admin.atlassian.com > Billing > Manage subscriptions`
5. Install app: `admin.atlassian.com > Products > [product] > Apps > Find new apps`
6. Configure app settings per vendor documentation
7. Train users on app usage
8. **VERIFY**: Confirm app appears in `GET /rest/plugins/1.0/` and health check passes
9. Monitor app performance and usage; review annually for continued need
### System Performance Optimization
**Jira** (`Jira Settings > System`):
- Archive old projects: `Project settings > Archive project`
- Reindex: `Jira Settings > System > Indexing > Full re-index`
- Clean up unused workflows and schemes: `Jira Settings > Issues > Workflows`
- Monitor queue/thread counts: `Jira Settings > System > System info`
**Confluence** (`Confluence Admin > Configuration`):
- Archive inactive spaces: `Space tools > Overview > Archive space`
- Remove orphaned pages: `Confluence Admin > Orphaned pages`
- Monitor index and cache: `Confluence Admin > Cache management`
**Monitoring Cadence**:
- Daily health checks: `admin.atlassian.com > Products > [product] > Health`
- Weekly performance reports
- Monthly capacity planning
- Quarterly optimization reviews
### Integration Setup
**Common Integrations**:
- **Slack**: `Jira Settings > Apps > Slack integration` — notifications for Jira and Confluence
- **GitHub/Bitbucket**: `Jira Settings > Apps > DVCS accounts` — link commits to issues
- **Microsoft Teams**: `admin.atlassian.com > Apps > Microsoft Teams`
- **Zoom**: Available via Marketplace app `zoom-for-jira`
- **Salesforce**: Via Marketplace app `salesforce-connector`
**Configuration Steps**:
1. Review integration requirements and OAuth scopes needed
2. Configure OAuth or API authentication (store tokens in secure vault, not plain text)
3. Map fields and data flows
4. Test integration thoroughly with sample data
5. Document configuration in Confluence runbook
6. Train users on integration features
7. **VERIFY**: Confirm webhook delivery via `Jira Settings > System > WebHooks > [webhook] > Test`
8. Monitor integration health via app-specific dashboards
## Global Configuration
### Jira Global Settings (`Jira Settings > Issues`)
**Issue Types**: Create and manage org-wide issue types; define issue type schemes; standardize across projects
**Workflows**: Create global workflow templates via `Workflows > Add workflow`; manage workflow schemes
**Custom Fields**: Create org-wide custom fields at `Custom fields > Add custom field`; manage field configurations and context
**Notification Schemes**: Configure default notification rules; create custom notification schemes; manage email templates
### Confluence Global Settings (`Confluence Admin`)
**Blueprints & Templates**: Create org-wide templates at `Configuration > Global Templates and Blueprints`; manage blueprint availability
**Themes & Appearance**: Configure org branding at `Configuration > Themes`; customize logos and colors
**Macros**: Enable/disable macros at `Configuration > Macro usage`; configure macro permissions
### Security Settings (`admin.atlassian.com > Security`)
**Authentication**:
- Password policies: `Security > Authentication policies > Edit`
- Session timeout: `Security > Session duration`
- API token management: `Security > API token controls`
**Data Residency**: Configure data location at `admin.atlassian.com > Data residency > Pin products`
**Audit Logs**: `admin.atlassian.com > Security > Audit log`
- Enable comprehensive logging; export via `GET /admin/v1/orgs/{orgId}/audit-log`
- Retain per policy (minimum 7 years for SOC 2/GDPR compliance)
## Governance & Policies
### Access Governance
- Quarterly review of all user access: `admin.atlassian.com > User management > Export users`
- Verify user roles and permissions; remove inactive users
- Limit org admins to 2–3 individuals; audit admin actions monthly
- Require MFA for all admins: `Security > Authentication policies > Require 2FA`
### Naming Conventions
**Jira**: Project keys 3–4 uppercase letters (PROJ, WEB); issue types Title Case; custom fields prefixed (CF: Story Points)
**Confluence**: Spaces use Team/Project prefix (TEAM: Engineering); pages descriptive and consistent; labels lowercase, hyphen-separated
### Change Management
**Major Changes**: Announce 2 weeks in advance; test in sandbox; create rollback plan; execute during off-peak; post-implementation review
**Minor Changes**: Announce 48 hours in advance; document in change log; monitor for issues
## Disaster Recovery
### Backup Strategy
**Jira & Confluence**: Daily automated backups; weekly manual verification; 30-day retention; offsite storage
- Trigger manual backup: `Jira Settings > System > Backup system` / `Confluence Admin > Backup and Restore`
**Recovery Testing**: Quarterly recovery drills; document procedures; measure RTO and RPO
### Incident Response
**Severity Levels**:
- **P1 (Critical)**: System down — respond in 15 min
- **P2 (High)**: Major feature broken — respond in 1 hour
- **P3 (Medium)**: Minor issue — respond in 4 hours
- **P4 (Low)**: Enhancement — respond in 24 hours
**Response Steps**:
1. Acknowledge and log incident
2. Assess impact and severity
3. Communicate status to stakeholders
4. Investigate root cause (check `admin.atlassian.com > Products > [product] > Health` and Atlassian Status Page)
5. Implement fix
6. **VERIFY**: Confirm resolution via affected user test and health check
7. Post-mortem and lessons learned
## Metrics & Reporting
**System Health**: Active users (daily/weekly/monthly), storage utilization, API rate limits, integration health, response times
- Export via: `GET /admin/v1/orgs/{orgId}/users` for user counts; product-specific analytics dashboards
**Usage Analytics**: Most active projects/spaces, content creation trends, user engagement, search patterns
**Compliance Metrics**: User access review completion, security audit findings, failed login attempts, API token usage
## Decision Framework & Handoff Protocols
**Escalate to Atlassian Support**: System outage, performance degradation org-wide, data loss/corruption, license/billing issues, complex migrations
**Delegate to Product Experts**:
- Jira Expert: Project-specific configuration
- Confluence Expert: Space-specific settings
- Scrum Master: Team workflow needs
- Senior PM: Strategic planning input
**Involve Security Team**: Security incidents, unusual access patterns, compliance audit preparation, new integration security review
**TO Jira Expert**: New global workflows, custom fields, permission schemes, or automation capabilities available
**TO Confluence Expert**: New global templates, space permission schemes, blueprints, or macros configured
**TO Senior PM**: Usage analytics, capacity planning insights, cost optimization, security compliance status
**TO Scrum Master**: Team access provisioned, board configuration options, automation rules, integrations enabled
**FROM All Roles**: User access requests, permission changes, app installation requests, configuration support, incident reports
## Atlassian MCP Integration
**Primary Tools**: Jira MCP, Confluence MCP
**Admin Operations**:
- User and group management via API
- Bulk permission updates
- Configuration audits
- Usage reporting
- System health monitoring
- Automated compliance checks
**Integration Points**:
- Support all roles with admin capabilities
- Enable Jira Expert with global configurations
- Provide Confluence Expert with template management
- Ensure Senior PM has visibility into org health
- Enable Scrum Master with team provisioning
FILE:assets/permission_scheme_template.json
{
"permissionScheme": {
"name": "Standard Project Permission Scheme",
"description": "Default permission scheme for standard projects. Assigns permissions based on project roles.",
"version": "1.0",
"lastUpdated": "YYYY-MM-DD",
"owner": "IT Admin Team"
},
"roles": {
"projectAdmin": {
"description": "Full project administration including configuration and user management",
"typicalGroups": ["project-leads", "engineering-managers"]
},
"developer": {
"description": "Create and manage issues, transitions, and attachments",
"typicalGroups": ["dept-engineering", "dept-product"]
},
"user": {
"description": "View issues, add comments, and create basic issues",
"typicalGroups": ["org-all-employees"]
},
"viewer": {
"description": "Read-only access to project issues and boards",
"typicalGroups": ["stakeholders", "external-contractors"]
}
},
"permissions": {
"project": {
"ADMINISTER_PROJECTS": {
"description": "Manage project settings, roles, and permissions",
"grantedTo": ["projectAdmin"]
},
"BROWSE_PROJECTS": {
"description": "View the project and its issues",
"grantedTo": ["projectAdmin", "developer", "user", "viewer"]
},
"VIEW_DEV_TOOLS": {
"description": "View development panel (commits, branches, PRs)",
"grantedTo": ["projectAdmin", "developer"]
},
"VIEW_READONLY_WORKFLOW": {
"description": "View read-only workflow",
"grantedTo": ["projectAdmin", "developer", "user", "viewer"]
}
},
"issues": {
"CREATE_ISSUES": {
"description": "Create new issues in the project",
"grantedTo": ["projectAdmin", "developer", "user"]
},
"EDIT_ISSUES": {
"description": "Edit issue fields",
"grantedTo": ["projectAdmin", "developer"]
},
"DELETE_ISSUES": {
"description": "Delete issues permanently",
"grantedTo": ["projectAdmin"]
},
"ASSIGN_ISSUES": {
"description": "Assign issues to team members",
"grantedTo": ["projectAdmin", "developer"]
},
"ASSIGNABLE_USER": {
"description": "Be assigned to issues",
"grantedTo": ["projectAdmin", "developer"]
},
"CLOSE_ISSUES": {
"description": "Close/resolve issues",
"grantedTo": ["projectAdmin", "developer"]
},
"RESOLVE_ISSUES": {
"description": "Set issue resolution",
"grantedTo": ["projectAdmin", "developer"]
},
"TRANSITION_ISSUES": {
"description": "Transition issues through workflow",
"grantedTo": ["projectAdmin", "developer", "user"]
},
"LINK_ISSUES": {
"description": "Create and remove issue links",
"grantedTo": ["projectAdmin", "developer"]
},
"MOVE_ISSUES": {
"description": "Move issues between projects",
"grantedTo": ["projectAdmin"]
},
"SCHEDULE_ISSUES": {
"description": "Set due dates on issues",
"grantedTo": ["projectAdmin", "developer"]
},
"SET_ISSUE_SECURITY": {
"description": "Set security level on issues",
"grantedTo": ["projectAdmin"]
}
},
"comments": {
"ADD_COMMENTS": {
"description": "Add comments to issues",
"grantedTo": ["projectAdmin", "developer", "user"]
},
"EDIT_ALL_COMMENTS": {
"description": "Edit any comment",
"grantedTo": ["projectAdmin"]
},
"EDIT_OWN_COMMENTS": {
"description": "Edit own comments",
"grantedTo": ["projectAdmin", "developer", "user"]
},
"DELETE_ALL_COMMENTS": {
"description": "Delete any comment",
"grantedTo": ["projectAdmin"]
},
"DELETE_OWN_COMMENTS": {
"description": "Delete own comments",
"grantedTo": ["projectAdmin", "developer", "user"]
}
},
"attachments": {
"CREATE_ATTACHMENTS": {
"description": "Attach files to issues",
"grantedTo": ["projectAdmin", "developer", "user"]
},
"DELETE_ALL_ATTACHMENTS": {
"description": "Delete any attachment",
"grantedTo": ["projectAdmin"]
},
"DELETE_OWN_ATTACHMENTS": {
"description": "Delete own attachments",
"grantedTo": ["projectAdmin", "developer", "user"]
}
},
"worklogs": {
"WORK_ON_ISSUES": {
"description": "Log work on issues",
"grantedTo": ["projectAdmin", "developer"]
},
"EDIT_ALL_WORKLOGS": {
"description": "Edit any worklog",
"grantedTo": ["projectAdmin"]
},
"EDIT_OWN_WORKLOGS": {
"description": "Edit own worklogs",
"grantedTo": ["projectAdmin", "developer"]
},
"DELETE_ALL_WORKLOGS": {
"description": "Delete any worklog",
"grantedTo": ["projectAdmin"]
},
"DELETE_OWN_WORKLOGS": {
"description": "Delete own worklogs",
"grantedTo": ["projectAdmin", "developer"]
}
}
},
"projectMappings": [
{
"projectKey": "EXAMPLE",
"projectName": "Example Project",
"scheme": "Standard Project Permission Scheme",
"roleAssignments": {
"projectAdmin": ["project-leads"],
"developer": ["team-example-devs"],
"user": ["org-all-employees"],
"viewer": ["stakeholders-example"]
}
}
],
"notes": {
"usage": "Copy this template and customize role assignments per project. Use group names that match your Atlassian groups.",
"review": "Review permission scheme assignments quarterly as part of access review.",
"changes": "Any changes to permission schemes should be documented and approved by IT Admin."
}
}
FILE:references/security-hardening-guide.md
# Atlassian Cloud Security Hardening Guide
## Overview
This guide provides a comprehensive security hardening checklist for Atlassian Cloud products (Jira, Confluence, Bitbucket). It covers identity management, access controls, data protection, and monitoring practices aligned with enterprise security standards.
## Identity & Authentication
### SSO / SAML Setup
**Implementation Steps:**
1. Verify your domain in Atlassian Admin (admin.atlassian.com)
2. Claim all company email accounts
3. Configure SAML SSO with your identity provider (Okta, Azure AD, Google Workspace)
4. Set authentication policy to enforce SSO for all managed accounts
5. Test with a pilot group before full rollout
6. Disable password-based login for managed accounts
**Configuration Checklist:**
- [ ] Domain verified and accounts claimed
- [ ] SAML IdP configured with correct entity ID and SSO URL
- [ ] Attribute mapping: email, displayName, groups
- [ ] Single Logout (SLO) configured
- [ ] Authentication policy enforcing SSO
- [ ] Fallback access configured for emergency admin accounts
- [ ] SCIM provisioning enabled for automatic user sync
### Two-Factor Authentication (2FA)
**Enforcement Policy:**
- [ ] 2FA required for all managed accounts
- [ ] Enforce via authentication policy (not just recommended)
- [ ] Hardware security keys (FIDO2/WebAuthn) preferred for admin accounts
- [ ] TOTP (authenticator app) as minimum for all users
- [ ] SMS-based 2FA disabled (SIM swap vulnerability)
- [ ] Recovery codes generated and stored securely
### Session Management
- [ ] Session timeout set to 8 hours of inactivity (maximum)
- [ ] Absolute session timeout: 24 hours
- [ ] Require re-authentication for sensitive operations
- [ ] Monitor concurrent sessions per user
- [ ] Enforce session termination on password change
## Access Controls
### IP Allowlisting
**Configuration:**
- [ ] Enable IP allowlisting for organization
- [ ] Add corporate office IP ranges
- [ ] Add VPN exit node IP addresses
- [ ] Add CI/CD server IPs for API access
- [ ] Test access from all approved locations
- [ ] Document approved IP ranges with justification
- [ ] Review IP allowlist quarterly
**Exceptions:**
- Mobile access may require VPN or MDM solution
- Remote workers need VPN or conditional access policies
- API integrations need stable IP ranges
### API Token Management
**Policies:**
- [ ] Inventory all API tokens in use
- [ ] Set maximum token lifetime (90 days recommended)
- [ ] Require token rotation on schedule
- [ ] Use service accounts for integrations (not personal tokens)
- [ ] Monitor API token usage patterns
- [ ] Revoke tokens immediately on employee departure
- [ ] Document purpose and owner for each token
**Best Practices:**
- Use OAuth 2.0 (3LO) for user-context integrations
- Use API tokens only for service-to-service
- Store tokens in secrets management (never in code)
- Implement least-privilege scopes for OAuth apps
### Permission Model
- [ ] Review global permissions quarterly
- [ ] Use groups for permission assignment (not individual users)
- [ ] Implement role-based access for Jira projects
- [ ] Restrict Confluence space admin to designated owners
- [ ] Limit Jira system admin to 2-3 people
- [ ] Audit "anyone" or "logged in users" permissions
- [ ] Remove direct user permissions where groups exist
## Audit & Monitoring
### Audit Log Configuration
**What to Monitor:**
- User authentication events (login, logout, failed attempts)
- Permission changes (project, space, global)
- User account changes (creation, deactivation, group changes)
- API token creation and revocation
- App installations and updates
- Data export operations
- Admin configuration changes
**Setup Steps:**
- [ ] Enable organization audit log
- [ ] Configure audit log retention (minimum 1 year)
- [ ] Set up automated export to SIEM (Splunk, Datadog, etc.)
- [ ] Create alerts for suspicious patterns
- [ ] Schedule monthly audit log review
- [ ] Document incident response procedures for alerts
### Alerting Rules
**Critical Alerts (Immediate Response):**
- Multiple failed login attempts (>5 in 10 minutes)
- Admin permission grants to unexpected users
- API token created by non-service accounts
- Bulk data export or deletion
- New third-party app installed with broad permissions
**Warning Alerts (Same-Day Review):**
- New admin users added
- Permission scheme changes
- Authentication policy modifications
- IP allowlist changes
- User deactivation (verify it is expected)
## Data Protection
### Data Residency
- [ ] Configure data residency realm (US, EU, AU, etc.)
- [ ] Verify product data pinned to selected region
- [ ] Document data residency for compliance audits
- [ ] Review data residency coverage (some metadata may be global)
- [ ] Monitor for new residency options from Atlassian
### Encryption
- [ ] Verify encryption at rest (AES-256, managed by Atlassian)
- [ ] Verify encryption in transit (TLS 1.2+)
- [ ] Review Atlassian's encryption key management practices
- [ ] Consider BYOK (Bring Your Own Key) for Atlassian Guard Premium
### Data Loss Prevention
- [ ] Configure content restrictions for sensitive pages/issues
- [ ] Implement classification labels (public, internal, confidential)
- [ ] Restrict file attachment types if needed
- [ ] Monitor bulk exports and downloads
- [ ] Set up DLP rules for sensitive data patterns (PII, credentials)
## Mobile Device Management
### Mobile Access Controls
- [ ] Require MDM enrollment for mobile Atlassian apps
- [ ] Enforce device encryption
- [ ] Require screen lock with biometrics or PIN
- [ ] Enable remote wipe capability
- [ ] Block rooted/jailbroken devices
- [ ] Restrict copy/paste to managed apps
- [ ] Set app-level PIN for Atlassian apps
### Mobile Policies
- [ ] Define approved mobile devices/OS versions
- [ ] Enforce automatic app updates
- [ ] Configure offline data access limits
- [ ] Set maximum offline cache duration
- [ ] Review mobile access logs monthly
## Third-Party App Security
### App Review Process
- [ ] Maintain approved app list (whitelist)
- [ ] Review app permissions before installation
- [ ] Verify app is Atlassian Marketplace certified
- [ ] Check app vendor security certifications
- [ ] Assess data access scope (read-only vs read-write)
- [ ] Review app privacy policy
- [ ] Document app owner and business justification
### App Governance
- [ ] Audit installed apps quarterly
- [ ] Remove unused apps (no usage in 90 days)
- [ ] Monitor app permission changes
- [ ] Restrict app installation to admins only
- [ ] Review Atlassian Guard app access policies
- [ ] Set up alerts for new app installations
## Compliance Documentation
### Required Documentation
- [ ] Security policy for Atlassian Cloud usage
- [ ] Access control matrix (roles, permissions, justification)
- [ ] Incident response plan for Atlassian security events
- [ ] Data classification policy applied to Atlassian content
- [ ] Third-party app risk assessments
- [ ] Annual security review report
### Compliance Frameworks
- **SOC 2:** Map Atlassian controls to Trust Service Criteria
- **ISO 27001:** Align with Annex A controls for cloud services
- **GDPR:** Configure data residency, right to deletion, DPAs
- **HIPAA:** Review BAA availability, encryption, access controls
## Hardening Schedule
| Task | Frequency | Owner |
|------|-----------|-------|
| Permission audit | Quarterly | IT Admin |
| API token rotation | Every 90 days | Integration owners |
| App review | Quarterly | IT Admin |
| Audit log review | Monthly | Security team |
| IP allowlist review | Quarterly | IT Admin |
| Authentication policy review | Semi-annually | Security team |
| Full security assessment | Annually | Security team |
| User access review | Quarterly | Managers + IT Admin |
| Data residency verification | Annually | Compliance |
| Mobile device audit | Quarterly | IT Admin |
FILE:references/user-provisioning-checklist.md
# User Provisioning & Lifecycle Management Checklist
## Overview
This checklist covers the complete user lifecycle in Atlassian Cloud products, from onboarding through offboarding. Consistent provisioning ensures security, compliance, and a smooth user experience.
## Onboarding Steps
### Pre-Provisioning
- [ ] Receive approved access request (ticket or HR system trigger)
- [ ] Verify employee record in HR system
- [ ] Determine role-based access level (see Role Templates below)
- [ ] Identify required Atlassian products (Jira, Confluence, Bitbucket)
- [ ] Identify required project/space access
### Account Creation
- [ ] User account auto-provisioned via SCIM (preferred) or manually created
- [ ] Email domain matches verified organization domain
- [ ] SSO authentication verified (user can log in via IdP)
- [ ] 2FA enrollment confirmed
- [ ] Correct product access assigned (Jira, Confluence, Bitbucket)
### Group Membership
- [ ] Add to organization-level groups (e.g., `all-employees`)
- [ ] Add to department group (e.g., `engineering`, `product`, `marketing`)
- [ ] Add to team-specific groups (e.g., `team-platform`, `team-mobile`)
- [ ] Add to project groups as needed (e.g., `project-alpha-members`)
- [ ] Verify group membership grants correct permissions
### Product Configuration
- [ ] **Jira:** Add to correct project roles (Developer, User, Admin)
- [ ] **Jira:** Assign to correct board(s)
- [ ] **Jira:** Set default dashboard if applicable
- [ ] **Confluence:** Grant access to relevant spaces
- [ ] **Confluence:** Add to space groups with appropriate permission level
- [ ] **Bitbucket:** Grant repository access per team
- [ ] **Bitbucket:** Configure branch permissions
### Welcome & Training
- [ ] Send welcome email with access details and key links
- [ ] Share Confluence onboarding page (getting started guide)
- [ ] Assign onboarding buddy for Atlassian tool questions
- [ ] Schedule optional training session for new users
- [ ] Provide link to internal Atlassian usage guidelines
## Role-Based Access Templates
### Developer
- **Jira:** Project Developer role (create, edit, transition issues)
- **Confluence:** Team space editor, documentation spaces viewer
- **Bitbucket:** Repository write access for team repos
### Product Manager
- **Jira:** Project Admin role (manage boards, workflows, components)
- **Confluence:** Product spaces editor, all team spaces viewer
- **Bitbucket:** Repository read access (optional)
### Designer
- **Jira:** Project User role (view, comment, transition)
- **Confluence:** Design space editor, product spaces editor
- **Bitbucket:** No access (unless needed)
### Engineering Manager
- **Jira:** Project Admin for managed projects, viewer for others
- **Confluence:** Team space admin, all spaces viewer
- **Bitbucket:** Repository admin for team repos
### Executive / Stakeholder
- **Jira:** Viewer role on strategic projects, dashboard access
- **Confluence:** Viewer on relevant spaces
- **Bitbucket:** No access
### Contractor / External
- **Jira:** Project User role, limited to specific projects
- **Confluence:** Viewer on specific spaces only (no edit)
- **Bitbucket:** Repository read access, specific repos only
- **Additional:** Set account expiration date, restrict IP access
## Group Membership Standards
### Naming Convention
```
org-{company} # Organization-wide groups
dept-{department} # Department groups
team-{team-name} # Team-specific groups
project-{project} # Project-scoped groups
role-{role} # Role-based groups (role-admin, role-viewer)
```
### Standard Groups
| Group | Purpose | Products |
|-------|---------|----------|
| `org-all-employees` | All full-time employees | Jira, Confluence |
| `dept-engineering` | All engineers | Jira, Confluence, Bitbucket |
| `dept-product` | All product team | Jira, Confluence |
| `dept-marketing` | All marketing team | Confluence |
| `role-jira-admins` | Jira administrators | Jira |
| `role-confluence-admins` | Confluence administrators | Confluence |
| `role-org-admins` | Organization administrators | All |
## Offboarding Procedure
### Immediate Actions (Day of Departure)
- [ ] Deactivate user account in Atlassian (or via IdP/SCIM)
- [ ] Revoke all API tokens associated with the user
- [ ] Revoke all OAuth app authorizations
- [ ] Transfer ownership of critical Confluence pages
- [ ] Reassign Jira issues (open/in-progress items)
- [ ] Remove from all groups
- [ ] Document access removal in offboarding ticket
### Within 24 Hours
- [ ] Verify account is fully deactivated (cannot log in)
- [ ] Check for shared credentials or service accounts
- [ ] Review audit log for recent activity
- [ ] Transfer Confluence space ownership if applicable
- [ ] Update Jira project leads/component leads if applicable
- [ ] Remove from any Atlassian Marketplace vendor accounts
### Within 7 Days
- [ ] Verify no lingering sessions or cached access
- [ ] Review integrations the user may have set up
- [ ] Check for automation rules owned by the user
- [ ] Update team dashboards and filters
- [ ] Confirm with manager that all transfers are complete
### Data Retention
- [ ] User content (pages, issues, comments) retained per policy
- [ ] Personal spaces archived or transferred
- [ ] Account marked as deactivated (not deleted) for audit trail
- [ ] Data deletion request processed if required (GDPR)
## Quarterly Access Reviews
### Review Process
1. Generate user access report from Atlassian Admin
2. Distribute to managers for team verification
3. Managers confirm or flag each user's access level
4. IT Admin processes approved changes
5. Document review completion for compliance
### Review Checklist
- [ ] All active accounts match current employee list
- [ ] No accounts for departed employees
- [ ] Group memberships align with current roles
- [ ] Admin access limited to approved administrators
- [ ] External/contractor accounts have valid expiration dates
- [ ] Service accounts documented with current owners
- [ ] Unused accounts (no login in 90 days) flagged for review
### Compliance Documentation
- [ ] Access review completion date recorded
- [ ] Manager sign-off captured (email or ticket)
- [ ] Changes made during review documented
- [ ] Exceptions documented with justification and approval
- [ ] Report filed for audit purposes
- [ ] Next review date scheduled
## Automation Opportunities
### SCIM Provisioning
- Automatically create/deactivate accounts based on IdP changes
- Sync group membership from IdP groups
- Reduce manual provisioning errors
- Ensure immediate deactivation on termination
### Workflow Automation
- Trigger onboarding checklist from HR system event
- Auto-assign to groups based on department/role attributes
- Send welcome messages via Confluence automation
- Schedule access reviews via Jira recurring tickets
### Monitoring
- Alert on accounts without 2FA after 7 days
- Alert on admin group changes
- Weekly report of new and deactivated accounts
- Monthly stale account report (no login in 90 days)
FILE:scripts/permission_audit_tool.py
#!/usr/bin/env python3
"""
Permission Audit Tool
Analyzes Atlassian permission schemes for security issues. Checks for
over-permissioned groups, direct user permissions, missing restrictions on
sensitive actions, inconsistencies across projects, and compliance gaps.
Usage:
python permission_audit_tool.py permissions.json
python permission_audit_tool.py permissions.json --format json
"""
import argparse
import json
import sys
from typing import Any, Dict, List, Optional, Set
# ---------------------------------------------------------------------------
# Audit Configuration
# ---------------------------------------------------------------------------
SENSITIVE_PERMISSIONS = {
"administer_project",
"administer_jira",
"delete_issues",
"delete_all_comments",
"delete_all_attachments",
"manage_watchers",
"modify_reporter",
"bulk_change",
"system_admin",
"manage_group_filter_subscriptions",
}
RECOMMENDED_GROUP_ONLY_PERMISSIONS = {
"browse_projects",
"create_issues",
"edit_issues",
"transition_issues",
"assign_issues",
"resolve_issues",
"close_issues",
"add_comments",
"edit_all_comments",
}
SEVERITY_WEIGHTS = {
"critical": 25,
"high": 15,
"medium": 8,
"low": 3,
"info": 1,
}
# ---------------------------------------------------------------------------
# Audit Checks
# ---------------------------------------------------------------------------
def check_over_permissioned_groups(
schemes: List[Dict[str, Any]],
) -> List[Dict[str, str]]:
"""Check for groups with overly broad admin access."""
findings = []
for scheme in schemes:
scheme_name = scheme.get("name", "Unknown Scheme")
grants = scheme.get("grants", [])
group_permissions = {}
for grant in grants:
group = grant.get("group", "")
permission = grant.get("permission", "").lower()
if group:
if group not in group_permissions:
group_permissions[group] = set()
group_permissions[group].add(permission)
for group, perms in group_permissions.items():
admin_perms = perms & SENSITIVE_PERMISSIONS
if len(admin_perms) >= 3:
findings.append({
"rule": "over_permissioned_group",
"severity": "high",
"scheme": scheme_name,
"group": group,
"message": f"Group '{group}' has {len(admin_perms)} sensitive permissions "
f"in scheme '{scheme_name}': {', '.join(sorted(admin_perms))}. "
f"Review if all are necessary.",
})
if "system_admin" in perms or "administer_jira" in perms:
findings.append({
"rule": "admin_access_warning",
"severity": "critical",
"scheme": scheme_name,
"group": group,
"message": f"Group '{group}' has system/Jira admin access in '{scheme_name}'. "
f"Ensure this is strictly necessary and membership is limited.",
})
return findings
def check_direct_user_permissions(
schemes: List[Dict[str, Any]],
) -> List[Dict[str, str]]:
"""Check for permissions granted directly to users instead of groups."""
findings = []
for scheme in schemes:
scheme_name = scheme.get("name", "Unknown Scheme")
grants = scheme.get("grants", [])
for grant in grants:
user = grant.get("user", "")
permission = grant.get("permission", "")
if user and not grant.get("group"):
severity = "high" if permission.lower() in SENSITIVE_PERMISSIONS else "medium"
findings.append({
"rule": "direct_user_permission",
"severity": severity,
"scheme": scheme_name,
"user": user,
"message": f"User '{user}' has direct permission '{permission}' in '{scheme_name}'. "
f"Use groups instead for maintainability and audit clarity.",
})
return findings
def check_missing_restrictions(
schemes: List[Dict[str, Any]],
) -> List[Dict[str, str]]:
"""Check for missing restrictions on sensitive actions."""
findings = []
for scheme in schemes:
scheme_name = scheme.get("name", "Unknown Scheme")
grants = scheme.get("grants", [])
granted_permissions = set()
for grant in grants:
granted_permissions.add(grant.get("permission", "").lower())
# Check if delete permissions are unrestricted
delete_perms = {"delete_issues", "delete_all_comments", "delete_all_attachments"}
unrestricted_deletes = delete_perms & granted_permissions
for grant in grants:
perm = grant.get("permission", "").lower()
group = grant.get("group", "")
if perm in delete_perms and group:
# Check if granted to broad groups
broad_groups = {"users", "everyone", "all-users", "jira-users", "jira-software-users"}
if group.lower() in broad_groups:
findings.append({
"rule": "unrestricted_delete",
"severity": "critical",
"scheme": scheme_name,
"message": f"Delete permission '{perm}' granted to broad group '{group}' "
f"in '{scheme_name}'. Restrict to admins or leads only.",
})
# Check if admin permissions exist
admin_perms = {"administer_project", "administer_jira", "system_admin"}
if not (admin_perms & granted_permissions):
findings.append({
"rule": "no_admin_defined",
"severity": "medium",
"scheme": scheme_name,
"message": f"No explicit admin permission defined in '{scheme_name}'. "
f"Ensure project administration is properly assigned.",
})
return findings
def check_scheme_consistency(
schemes: List[Dict[str, Any]],
) -> List[Dict[str, str]]:
"""Check for inconsistencies across permission schemes."""
findings = []
if len(schemes) < 2:
return findings
# Compare permission sets across schemes
scheme_perms = {}
for scheme in schemes:
name = scheme.get("name", "Unknown")
perms = set()
for grant in scheme.get("grants", []):
perms.add(grant.get("permission", "").lower())
scheme_perms[name] = perms
# Find schemes with significantly different permission sets
all_perms = set()
for perms in scheme_perms.values():
all_perms |= perms
scheme_names = list(scheme_perms.keys())
for i in range(len(scheme_names)):
for j in range(i + 1, len(scheme_names)):
name_a = scheme_names[i]
name_b = scheme_names[j]
diff = scheme_perms[name_a].symmetric_difference(scheme_perms[name_b])
if len(diff) > 5:
findings.append({
"rule": "scheme_inconsistency",
"severity": "medium",
"message": f"Schemes '{name_a}' and '{name_b}' differ significantly "
f"({len(diff)} different permissions). Review for intentional differences.",
})
return findings
def check_compliance_gaps(
schemes: List[Dict[str, Any]],
) -> List[Dict[str, str]]:
"""Check for common compliance gaps."""
findings = []
for scheme in schemes:
scheme_name = scheme.get("name", "Unknown Scheme")
grants = scheme.get("grants", [])
groups_used = set()
users_used = set()
for grant in grants:
if grant.get("group"):
groups_used.add(grant["group"])
if grant.get("user"):
users_used.add(grant["user"])
# Check for separation of duties
admin_groups = set()
for grant in grants:
if grant.get("permission", "").lower() in SENSITIVE_PERMISSIONS and grant.get("group"):
admin_groups.add(grant["group"])
if len(admin_groups) == 1 and len(groups_used) > 1:
findings.append({
"rule": "separation_of_duties",
"severity": "info",
"scheme": scheme_name,
"message": f"Only one group ('{next(iter(admin_groups))}') holds all sensitive permissions "
f"in '{scheme_name}'. Consider separating duties across multiple groups.",
})
# Check user count
if len(users_used) > 5:
findings.append({
"rule": "too_many_direct_users",
"severity": "high",
"scheme": scheme_name,
"message": f"Scheme '{scheme_name}' has {len(users_used)} direct user grants. "
f"Migrate to group-based permissions for better governance.",
})
return findings
# ---------------------------------------------------------------------------
# Main Analysis
# ---------------------------------------------------------------------------
def audit_permissions(data: Dict[str, Any]) -> Dict[str, Any]:
"""Run full permission audit."""
schemes = data.get("schemes", [])
if not schemes:
# Try treating the entire input as a single scheme
if data.get("grants") or data.get("name"):
schemes = [data]
else:
return {
"risk_score": 0,
"grade": "invalid",
"error": "No permission schemes found in input",
"findings": [],
"summary": {},
}
all_findings = []
all_findings.extend(check_over_permissioned_groups(schemes))
all_findings.extend(check_direct_user_permissions(schemes))
all_findings.extend(check_missing_restrictions(schemes))
all_findings.extend(check_scheme_consistency(schemes))
all_findings.extend(check_compliance_gaps(schemes))
# Calculate risk score (higher = more risk)
summary = {"critical": 0, "high": 0, "medium": 0, "low": 0, "info": 0}
total_penalty = 0
for finding in all_findings:
severity = finding["severity"]
summary[severity] = summary.get(severity, 0) + 1
total_penalty += SEVERITY_WEIGHTS.get(severity, 0)
risk_score = min(100, total_penalty)
health_score = max(0, 100 - risk_score)
if health_score >= 85:
grade = "excellent"
elif health_score >= 70:
grade = "good"
elif health_score >= 50:
grade = "fair"
else:
grade = "poor"
# Generate remediation recommendations
remediations = _generate_remediations(all_findings)
return {
"risk_score": risk_score,
"health_score": health_score,
"grade": grade,
"schemes_analyzed": len(schemes),
"findings": all_findings,
"summary": summary,
"remediations": remediations,
}
def _generate_remediations(findings: List[Dict[str, str]]) -> List[str]:
"""Generate remediation recommendations."""
remediations = []
rules_seen = set()
for finding in findings:
rule = finding["rule"]
if rule in rules_seen:
continue
rules_seen.add(rule)
if rule == "over_permissioned_group":
remediations.append("Review and reduce sensitive permissions for over-permissioned groups. Apply principle of least privilege.")
elif rule == "admin_access_warning":
remediations.append("Audit admin group membership. Limit system/Jira admin access to essential personnel only.")
elif rule == "direct_user_permission":
remediations.append("Migrate direct user permissions to group-based grants. Create functional groups for common permission sets.")
elif rule == "unrestricted_delete":
remediations.append("Restrict delete permissions to project admins or leads. Remove from broad user groups.")
elif rule == "scheme_inconsistency":
remediations.append("Standardize permission schemes across projects. Document intentional differences.")
elif rule == "too_many_direct_users":
remediations.append("Create groups for users with direct permissions. This simplifies onboarding/offboarding.")
elif rule == "separation_of_duties":
remediations.append("Consider splitting admin responsibilities across multiple groups for better separation of duties.")
elif rule == "no_admin_defined":
remediations.append("Define explicit admin permissions in each scheme to ensure proper project governance.")
return remediations
# ---------------------------------------------------------------------------
# Output Formatting
# ---------------------------------------------------------------------------
def format_text_output(result: Dict[str, Any]) -> str:
"""Format results as readable text report."""
lines = []
lines.append("=" * 60)
lines.append("PERMISSION AUDIT REPORT")
lines.append("=" * 60)
lines.append("")
if "error" in result:
lines.append(f"ERROR: {result['error']}")
return "\n".join(lines)
lines.append("AUDIT SUMMARY")
lines.append("-" * 30)
lines.append(f"Risk Score: {result['risk_score']}/100 (lower is better)")
lines.append(f"Health Score: {result['health_score']}/100")
lines.append(f"Grade: {result['grade'].title()}")
lines.append(f"Schemes Analyzed: {result['schemes_analyzed']}")
lines.append("")
summary = result.get("summary", {})
lines.append("FINDINGS BY SEVERITY")
lines.append("-" * 30)
lines.append(f"Critical: {summary.get('critical', 0)}")
lines.append(f"High: {summary.get('high', 0)}")
lines.append(f"Medium: {summary.get('medium', 0)}")
lines.append(f"Low: {summary.get('low', 0)}")
lines.append(f"Info: {summary.get('info', 0)}")
lines.append("")
findings = result.get("findings", [])
if findings:
lines.append("DETAILED FINDINGS")
lines.append("-" * 30)
for i, finding in enumerate(findings, 1):
severity = finding["severity"].upper()
lines.append(f"{i}. [{severity}] {finding['message']}")
lines.append(f" Rule: {finding['rule']}")
if finding.get("scheme"):
lines.append(f" Scheme: {finding['scheme']}")
lines.append("")
remediations = result.get("remediations", [])
if remediations:
lines.append("REMEDIATION RECOMMENDATIONS")
lines.append("-" * 30)
for i, rem in enumerate(remediations, 1):
lines.append(f"{i}. {rem}")
return "\n".join(lines)
def format_json_output(result: Dict[str, Any]) -> Dict[str, Any]:
"""Format results as JSON."""
return result
# ---------------------------------------------------------------------------
# CLI Interface
# ---------------------------------------------------------------------------
def main() -> int:
"""Main CLI entry point."""
parser = argparse.ArgumentParser(
description="Audit Atlassian permission schemes for security issues"
)
parser.add_argument(
"permissions_file",
help="JSON file with permission scheme data",
)
parser.add_argument(
"--format",
choices=["text", "json"],
default="text",
help="Output format (default: text)",
)
args = parser.parse_args()
try:
with open(args.permissions_file, "r") as f:
data = json.load(f)
result = audit_permissions(data)
if args.format == "json":
print(json.dumps(format_json_output(result), indent=2))
else:
print(format_text_output(result))
return 0
except FileNotFoundError:
print(f"Error: File '{args.permissions_file}' not found", file=sys.stderr)
return 1
except json.JSONDecodeError as e:
print(f"Error: Invalid JSON in '{args.permissions_file}': {e}", file=sys.stderr)
return 1
except Exception as e:
print(f"Error: {e}", file=sys.stderr)
return 1
if __name__ == "__main__":
sys.exit(main())
Soạn bộ slide họp hội đồng quản trị và cập nhật cho nhà đầu tư, tổng hợp góc nhìn từ các vai trò C-suite.
--- name: "board-deck-builder" description: "Assembles comprehensive board and investor update decks by pulling perspectives from all C-suite roles. Use when preparing board meetings, investor updates, quarterly business reviews, or fundraising narratives. Covers structure, narrative framework, bad news delivery, and common mistakes." license: MIT metadata: version: 1.0.0 author: Alireza Rezvani category: c-level domain: board-governance updated: 2026-03-05 frameworks: deck-frameworks, board-deck-template --- # Board Deck Builder Build board decks that tell a story — not just show data. Every section has an owner, a narrative, and a "so what." ## Keywords board deck, investor update, board meeting, board pack, investor relations, quarterly review, board presentation, fundraising deck, investor deck, board narrative, QBR, quarterly business review ## Quick Start ``` /board-deck [quarterly|monthly|fundraising] [stage: seed|seriesA|seriesB] ``` Provide available metrics. The builder fills gaps with explicit placeholders — never invents numbers. ## Deck Structure (Standard Order) Every section follows: **Headline → Data → Narrative → Ask/Next** ### 1. Executive Summary (CEO) **3 sentences. No more.** - Sentence 1: State of the business (where we are) - Sentence 2: Biggest thing that happened this period - Sentence 3: Where we're going next quarter *Bad:* "We had a good quarter with lots of progress across all areas." *Good:* "We closed Q3 at $2.4M ARR (+22% QoQ), signed our largest enterprise contract, and enter Q4 with 14-month runway. The strategic shift to mid-market is working — ACV up 40% and sales cycle down 3 weeks. Q4 priority: close the $3M Series A and hit $2.8M ARR." ### 2. Key Metrics Dashboard (COO) **6-8 metrics max. Use a table.** | Metric | This Period | Last Period | Target | Status | |--------|-------------|-------------|--------|--------| | ARR | $2.4M | $1.97M | $2.3M | ✅ | | MoM growth | 8.1% | 7.2% | 7.5% | ✅ | | Burn multiple | 1.8x | 2.1x | <2x | ✅ | | NRR | 112% | 108% | >110% | ✅ | | CAC payback | 11 months | 14 months | <12 months | ✅ | | Headcount | 24 | 21 | 25 | 🟡 | Pick metrics the board actually tracks. Swap out anything they've said they don't care about. ### 3. Financial Update (CFO) - P&L summary: Revenue, COGS, Gross margin, OpEx, Net burn - Cash position and runway (months) - Burn multiple trend (3-quarter view) - Variance to plan (what was different and why) - Forecast update for next quarter **One sentence on each variance.** Boards hate "revenue was below target" with no explanation. Say why. ### 4. Revenue & Pipeline (CRO) - ARR waterfall: starting → new → expansion → churn → ending - NRR and logo churn rates - Pipeline by stage (in $, not just count) - Forecast: next quarter with confidence level - Top 3 deals: name/amount/close date/risk **The forecast must have a confidence level.** "We expect $2.8M" is weak. "High confidence $2.6M, upside to $2.9M if two late-stage deals close" is useful. ### 5. Product Update (CPO) - Shipped this quarter: 3-5 bullets, user impact for each - Shipping next quarter: 3-5 bullets with target dates - PMF signal: NPS trend, DAU/MAU ratio, feature adoption - One key learning from customer research **No feature lists.** Only features with evidence of user impact. ### 6. Growth & Marketing (CMO) - CAC by channel (table) - Pipeline contribution by channel ($) - Brand/awareness metrics relevant to stage (traffic, share of voice) - What's working, what's being cut, what's being tested ### 7. Engineering & Technical (CTO) - Delivery velocity trend (last 4 quarters) - Tech debt ratio and plan - Infrastructure: uptime, incidents, cost trend - Security posture (one line, flag anything pending) **Keep this short unless there's a material issue.** Boards don't need sprint details. ### 8. Team & People (CHRO) - Headcount: actual vs plan - Hiring: offers out, pipeline, time-to-fill trend - Attrition: regrettable vs non-regrettable - Engagement: last survey score, trend - Key hires this quarter, key open roles ### 9. Risk & Security (CISO) - Security posture: status of critical controls - Compliance: certifications in progress, deadlines - Incidents this quarter (if any): impact, resolution, prevention - Top 3 risks and mitigation status ### 10. Strategic Outlook (CEO) - Next quarter priorities: 3-5 items, ranked - Key decisions needed from the board - Asks: budget, introductions, advice, votes **The "asks" slide is the most important.** Be specific. "We'd like 3 warm introductions to CFOs at Series B companies" beats "any help would be appreciated." ### 11. Appendix - Detailed financial model - Full pipeline data - Cohort retention charts - Customer case studies - Detailed headcount breakdown --- ## Narrative Framework Boards see 10+ decks per quarter. Yours needs a through-line. **The 4-Act Structure:** 1. **Where we said we'd be** (last quarter's targets) 2. **Where we actually are** (honest assessment) 3. **Why the gap exists** (one cause per variance, not excuses) 4. **What we're doing about it** (specific, dated actions) This works for good news AND bad news. It's credible because it acknowledges reality. **Opening frame:** Start with the one thing that matters most — the board should know the key message by slide 3, not slide 30. --- ## Delivering Bad News Never bury it. Boards find out eventually. Finding out late makes it worse. **Framework:** 1. **State it plainly** — "We missed Q3 ARR target by $300K (12% gap)" 2. **Own the cause** — "Primary driver was longer-than-expected sales cycle in enterprise segment" 3. **Show you understand it** — "We analyzed 8 lost/stalled deals; the pattern is X" 4. **Present the fix** — "We've made 3 changes: [specific, dated changes]" 5. **Update the forecast** — "Revised Q4 target is $2.6M; here's the bottom-up build" **What NOT to do:** - Don't lead with good news to soften bad news — boards notice and distrust the framing - Don't explain without owning — "market conditions" is not a cause, it's a context - Don't present a fix without data behind it - Don't show a revised forecast without showing your assumptions --- ## Common Board Deck Mistakes | Mistake | Fix | |---------|-----| | Too many slides (>25) | Cut ruthlessly — if you can't explain it in the room, the slide is wrong | | Metrics without targets | Every metric needs a target and a status | | No narrative | Data without story forces boards to draw their own conclusions | | Burying bad news | Lead with it, own it, fix it | | Vague asks | Specific, actionable, person-assigned asks only | | No variance explanation | Every gap from target needs one-sentence cause | | Stale appendix | Appendix is only useful if it's current | | Designing for the reader, not the room | Decks are presented — they must work spoken aloud | --- ## Cadence Notes **Quarterly (standard):** Full deck, all sections, 20-30 slides. Sent 48 hours in advance. **Monthly (for early-stage):** Condensed — metrics dashboard, financials, pipeline, top risks. 8-12 slides. **Fundraising:** Opens with market/vision, closes with ask. See `references/deck-frameworks.md` for Sequoia format. ## References - `references/deck-frameworks.md` — SaaS board pack format, Sequoia structure, investor tailoring - `templates/board-deck-template.md` — fill-in template for complete board decks FILE:references/deck-frameworks.md # Board Deck Frameworks ## The SaaS Board Pack (Christoph Janz / Point Nine Style) Point Nine's board pack format became the de facto standard for early-stage SaaS. Core principle: **the numbers tell the story; the narrative explains the numbers.** ### Required Metrics (non-negotiable for SaaS boards) - **ARR** (not MRR — boards think annually) - **MoM / QoQ growth rate** - **NRR (Net Revenue Retention)** — the single most important SaaS metric - **Gross margin** — typically 60-80% SaaS; <60% is a flag - **CAC payback period** — months to recover customer acquisition cost - **Burn multiple** = net burn / net new ARR; <2x is good, >3x is a problem - **Runway** — months at current burn ### Point Nine Benchmark Targets (Series A SaaS) | Metric | Good | Great | Warning | |--------|------|-------|---------| | MoM growth | 10-15% | >20% | <7% | | NRR | >110% | >130% | <100% | | Gross margin | >65% | >75% | <60% | | CAC payback | <18 months | <12 months | >24 months | | Burn multiple | <2x | <1.5x | >3x | | Logo churn | <10%/yr | <5%/yr | >15%/yr | ### SaaS ARR Waterfall (Christoph Janz Format) Show this every quarter: ``` Starting ARR: $1,970,000 + New ARR: +$480,000 (new logos) + Expansion ARR: +$120,000 (upsells/cross-sells) - Churned ARR: -$90,000 (cancellations) - Contraction ARR: -$35,000 (downgrades) = Ending ARR: $2,445,000 ``` NRR = (Ending - New) / Starting = ($1,965K) / ($1,970K) = 99.7% ← flag this --- ## Sequoia Board Deck Structure Sequoia's canonical deck (used for both fundraising and board updates): 1. **Company Purpose** — one sentence, the existential "why" 2. **The Problem** — pain, size, who has it 3. **The Solution** — what you do, how it's different 4. **Why Now** — market timing, tailwinds, enabling factors 5. **Market Size** — TAM/SAM/SOM with methodology 6. **Business Model** — how you make money 7. **Traction** — proof it's working (growth, retention, logos) 8. **Team** — why you're the ones to win this 9. **Financials** — 3-year model, current metrics 10. **The Ask** — amount, use of funds, milestones to next round **For ongoing board updates:** Swap 1-5 (context) for "State of the Business" and "Last Quarter vs Plan." Boards know the company — skip the pitch. --- ## Investor-Specific Tailoring ### What Different Investor Types Care About **Early-stage VCs (Seed, A):** - Growth rate above all else - NRR — "does the product retain?" - Founder-market fit narrative - Milestone achievement vs last board meeting **Growth-stage VCs (B, C):** - Capital efficiency (burn multiple, CAC payback) - GTM repeatability — can you hire 10 AEs and have it work? - Market leadership signals - Path to profitability (even if years away) **Strategic investors:** - Synergies with their portfolio/business - Technology differentiation - Partnership potential **Angels:** - Team above all - Personal conviction in the thesis - Exit scenarios ### Tailoring the Narrative - If you're ahead of plan: "Here's why, and here's how we'll sustain it" - If you're behind plan: "Here's why, here's what we've learned, here's the new plan" - If the plan was wrong: "The assumption that was wrong, what we know now, updated thesis" Never pretend the plan was right when it wasn't. Board members have memories and models. --- ## How to Present Bad News Boards have seen everything. What loses credibility isn't bad results — it's bad framing. ### The Credibility Formula 1. **Lead with the headline** — "We missed ARR target by 18%" 2. **Quantify the gap** — absolute and percentage 3. **Diagnose the cause** (one primary, max two secondary) 4. **Show your work** — "We analyzed 12 churned/stalled deals and found..." 5. **Present the fix** — specific, dated, owned by a name 6. **Update the forecast** — bottom-up rebuild, not wishful thinking 7. **Flag the risk** — "If X doesn't close, here's the contingency" ### What "Showing Your Work" Looks Like Bad: "Sales cycle was longer than expected." Good: "Sales cycle stretched from 45 to 72 days. Root cause: new legal review requirement at enterprise accounts, triggered by our SOC 2 Type II gap. Fix: SOC 2 audit underway (target: Dec 15), and we've pre-built contract language to accelerate review. Impact: estimated 3 stalled deals ($420K ARR) unblock in Q4." ### Scenarios and How to Handle Each | Scenario | Frame | |----------|-------| | Missed revenue target | Lead with it; diagnose cause; bottom-up revised forecast | | Key customer churned | Announce it; explain why; show retention analysis of remaining accounts | | Key exec left | Announce it; show succession/coverage plan; don't overpromise the replacement timeline | | Burn accelerated | Show P&L detail; explain what drove it; adjust runway projection; plan to fix | | Market headwinds | Acknowledge; show relative performance vs peers; pivot if needed | | Fundraise delayed | Runway impact; bridge options; revised timeline | --- ## Appendix Data That Boards Actually Use Boards use the appendix for due diligence, not during the meeting. Include: **Financial:** - Full P&L (monthly for last 4 quarters) - Cash flow statement - 3-year model with assumptions - Unit economics by cohort **Revenue:** - Customer list by ARR (anonymized or full, per board agreement) - Pipeline detail by deal - Cohort analysis (NRR by cohort vintage) - Churn analysis: when, why, segment **Product:** - Feature adoption rates - NPS score distribution and trend - DAU/MAU by segment **Team:** - Org chart - Full headcount list with fully loaded costs - Open reqs with priority ranking **One rule:** If the appendix is more than 20 slides, you have too much. Boards won't read it. --- ## Quarterly vs Monthly Board Meetings ### Quarterly (Series A+) - Full board pack, all sections - 2 hours: 30 min pre-read, 90 min discussion - Voting items at end - Sent 48 hours before (72 hours preferred) - Add 1-2 "deep dive" topics beyond standard update ### Monthly (Seed / High-Growth A) - Metrics dashboard + financials + top risks only - 45-60 minutes - Informal tone, more conversational - Sent 24 hours before - Skip slides for items where nothing changed ### When to Increase Frequency - Approaching 6-month runway - Major strategic pivot - Fundraise in progress - Significant underperformance vs plan - M&A discussions --- ## Meeting Logistics (Often Overlooked) - **Pre-read requirement:** Board packs should be read before the meeting. If you're presenting slides, you're wasting time. - **Discussion format:** "I'll be brief on X since you've read it. Want to spend time on Y?" — respect board members' time - **One note-taker:** CEO's EA or COO; not the CEO (they need to be present) - **Follow-up within 24 hours:** Action items, voting outcomes, next meeting date - **Board portal vs email:** Use a board portal (Carta, Boardable, Notion) for version control and D&O protection FILE:templates/board-deck-template.md # Board Deck Template Fill in bracketed fields. Remove placeholders before sharing. Never invent numbers — use `[TBD]` if unknown. --- ## Slide 1: Executive Summary (CEO) **[Company Name] — Q[X] [Year] Board Update** > [One sentence: State of the business — where you are.] > [One sentence: The most important thing that happened this quarter.] > [One sentence: Where you're going next quarter and what determines success.] --- ## Slide 2: Key Metrics Dashboard (COO) **Quarter at a Glance** | Metric | Q[X] Actual | Q[X] Target | Q[X-1] Actual | Status | |--------|-------------|-------------|---------------|--------| | ARR | $[X]M | $[X]M | $[X]M | [✅/🟡/🔴] | | QoQ Growth | [X]% | [X]% | [X]% | [✅/🟡/🔴] | | NRR | [X]% | >[X]% | [X]% | [✅/🟡/🔴] | | Gross Margin | [X]% | >[X]% | [X]% | [✅/🟡/🔴] | | Burn Multiple | [X]x | <[X]x | [X]x | [✅/🟡/🔴] | | Runway | [X] months | >[X] months | [X] months | [✅/🟡/🔴] | | Headcount | [X] | [X] | [X] | [✅/🟡/🔴] | | CAC Payback | [X] months | <[X] months | [X] months | [✅/🟡/🔴] | --- ## Slide 3: Financial Update (CFO) **P&L Summary** | | Q[X] | Q[X-1] | QoQ | |--|------|--------|-----| | Revenue | $[X]K | $[X]K | [+/-X]% | | COGS | $[X]K | $[X]K | | | Gross Profit | $[X]K | $[X]K | | | Gross Margin | [X]% | [X]% | | | OpEx | $[X]K | $[X]K | | | Net Burn | $[X]K | $[X]K | | **Cash & Runway** - Cash on hand: $[X]M - Monthly burn: $[X]K - Runway: [X] months - Burn multiple: [X]x (target: <2x) **Variance to Plan** - Revenue: [+/-$X]K vs plan — [one sentence cause] - Burn: [+/-$X]K vs plan — [one sentence cause] **Q[X+1] Forecast:** $[X]M revenue, $[X]K burn — [confidence: high/medium/low] --- ## Slide 4: Revenue & Pipeline (CRO) **ARR Waterfall** ``` Starting ARR: $[X]M + New ARR: +$[X]K + Expansion ARR: +$[X]K - Churned ARR: -$[X]K - Contraction ARR: -$[X]K = Ending ARR: $[X]M ``` **Health Metrics** - NRR: [X]% | Logo churn: [X]% | Avg ACV: $[X]K **Pipeline (next 90 days)** | Stage | # Deals | $ Value | |-------|---------|---------| | Proposal | [X] | $[X]K | | Negotiation | [X] | $[X]K | | Verbal commit | [X] | $[X]K | **Q[X+1] Forecast:** $[X]M ARR — [one sentence confidence statement] **Top 3 Deals** 1. [Company] — $[X]K ARR — close date [X] — risk: [one word] 2. [Company] — $[X]K ARR — close date [X] — risk: [one word] 3. [Company] — $[X]K ARR — close date [X] — risk: [one word] --- ## Slide 5: Product Update (CPO) **Shipped This Quarter** - [Feature/initiative] — impact: [metric or user outcome] - [Feature/initiative] — impact: [metric or user outcome] - [Feature/initiative] — impact: [metric or user outcome] **Shipping Next Quarter** - [Feature] — target: [date] — why it matters: [one line] - [Feature] — target: [date] — why it matters: [one line] - [Feature] — target: [date] — why it matters: [one line] **PMF Signals** - NPS: [X] (trend: [up/flat/down]) - DAU/MAU: [X]% - Feature adoption ([key feature]): [X]% **Key Learning:** [One thing customer research taught you this quarter] --- ## Slide 6: Growth & Marketing (CMO) **CAC by Channel** | Channel | CAC | Pipeline $ | % of Total | |---------|-----|-----------|------------| | Outbound | $[X]K | $[X]K | [X]% | | Inbound | $[X]K | $[X]K | [X]% | | Partner | $[X]K | $[X]K | [X]% | **What's Working:** [One channel or initiative with data] **What We Cut:** [One thing, and why] **What We're Testing:** [One experiment running now] --- ## Slide 7: Engineering & Technical (CTO) **Delivery** - Velocity trend: [up/flat/down vs last quarter] - Q[X] commitments delivered: [X]% on time **Quality & Reliability** - P0/P1 incidents: [X] (vs [X] last quarter) - Uptime: [X]% - Infrastructure cost: $[X]K/month (trend: [up/flat/down]) **Tech Debt** - Ratio: [X]% of roadmap allocated to debt reduction - Key item in progress: [description, target date] **Security:** [one line status; flag anything pending] --- ## Slide 8: Team & People (CHRO) **Headcount** - Total: [X] (vs [X] plan, [X] last quarter) - By function: Eng [X], Product [X], Sales [X], CS [X], G&A [X] **Hiring** - Hired this quarter: [X] - Open reqs: [X] — time-to-fill avg: [X] days - Offers outstanding: [X] **Retention** - Regrettable attrition: [X]% (annualized) - Engagement score: [X]/10 (trend: [up/flat/down]) **Notable Hires:** [Name, role — one sentence on why they matter] **Key Open Roles:** [Role, priority: critical/high/medium] --- ## Slide 9: Risk & Security (CISO) **Compliance Status** | Certification | Status | Target Date | |--------------|--------|-------------| | [SOC 2 / ISO 27001 / etc.] | [In progress / Complete / Not started] | [Date] | **Security Posture:** [One line — overall status] **Incidents This Quarter:** [X] total — [description if >0] **Top Risks** 1. [Risk] — likelihood: [H/M/L] — impact: [H/M/L] — mitigation: [one line] 2. [Risk] — likelihood: [H/M/L] — impact: [H/M/L] — mitigation: [one line] 3. [Risk] — likelihood: [H/M/L] — impact: [H/M/L] — mitigation: [one line] --- ## Slide 10: Strategic Outlook (CEO) **Q[X+1] Priorities** 1. [Priority] — owner: [name] — success metric: [specific] 2. [Priority] — owner: [name] — success metric: [specific] 3. [Priority] — owner: [name] — success metric: [specific] **Asks from the Board** - [Specific ask: warm intro / advice / vote / resource] - [Specific ask] - [Specific ask] **Decisions Needed Today** - [Decision with options]: [Option A] vs [Option B] — recommendation: [A/B] — rationale: [one line] --- ## Appendix - A1: Full P&L (monthly, last 4 quarters) - A2: 3-year financial model - A3: Customer list / ARR breakdown - A4: Full pipeline by deal - A5: Cohort retention analysis - A6: Org chart + headcount detail - A7: [Other as relevant]
Quy trình họp hội đồng đa agent 6 giai đoạn cho các quyết định chiến lược, từ ngữ cảnh đến trích xuất quyết định.
--- name: "board-meeting" description: "Multi-agent board meeting protocol for strategic decisions. Runs a structured 6-phase deliberation: context loading, independent C-suite contributions (isolated, no cross-pollination), critic analysis, synthesis, founder review, and decision extraction. Use when the user invokes /cs:board, calls a board meeting, or wants structured multi-perspective executive deliberation on a strategic question." license: MIT metadata: version: 1.0.0 author: Alireza Rezvani category: c-level domain: board-protocol updated: 2026-03-05 frameworks: 6-phase-board, two-layer-memory, independent-contributions --- # Board Meeting Protocol Structured multi-agent deliberation that prevents groupthink, captures minority views, and produces clean, actionable decisions. ## Keywords board meeting, executive deliberation, strategic decision, C-suite, multi-agent, /cs:board, founder review, decision extraction, independent perspectives ## Invoke `/cs:board [topic]` — e.g. `/cs:board Should we expand to Spain in Q3?` --- ## The 6-Phase Protocol ### PHASE 1: Context Gathering 1. Load `memory/company-context.md` 2. Load `memory/board-meetings/decisions.md` **(Layer 2 ONLY — never raw transcripts)** 3. Reset session state — no bleed from previous conversations 4. Present agenda + activated roles → wait for founder confirmation **Chief of Staff selects relevant roles** based on topic (not all 9 every time): | Topic | Activate | |-------|----------| | Market expansion | CEO, CMO, CFO, CRO, COO | | Product direction | CEO, CPO, CTO, CMO | | Hiring/org | CEO, CHRO, CFO, COO | | Pricing | CMO, CFO, CRO, CPO | | Technology | CTO, CPO, CFO, CISO | --- ### PHASE 2: Independent Contributions (ISOLATED) **No cross-pollination. Each agent runs before seeing others' outputs.** Order: Research (if needed) → CMO → CFO → CEO → CTO → COO → CHRO → CRO → CISO → CPO **Reasoning techniques:** CEO: Tree of Thought (3 futures) | CFO: Chain of Thought (show the math) | CMO: Recursion of Thought (draft→critique→refine) | CPO: First Principles | CRO: Chain of Thought (pipeline math) | COO: Step by Step (process map) | CTO: ReAct (research→analyze→act) | CISO: Risk-Based (P×I) | CHRO: Empathy + Data **Contribution format (max 5 key points, self-verified):** ``` ## [ROLE] — [DATE] Key points (max 5): • [Finding] — [VERIFIED/ASSUMED] — 🟢/🟡/🔴 • [Finding] — [VERIFIED/ASSUMED] — 🟢/🟡/🔴 Recommendation: [clear position] Confidence: High / Medium / Low Source: [where the data came from] What would change my mind: [specific condition] ``` Each agent self-verifies before contributing: source attribution, assumption audit, confidence scoring. No untagged claims. --- ### PHASE 3: Critic Analysis Executive Mentor receives ALL Phase 2 outputs simultaneously. Role: adversarial reviewer, not synthesizer. Checklist: - Where did agents agree too easily? (suspicious consensus = red flag) - What assumptions are shared but unvalidated? - Who is missing from the room? (customer voice? front-line ops?) - What risk has nobody mentioned? - Which agent operated outside their domain? --- ### PHASE 4: Synthesis Chief of Staff delivers using the **Board Meeting Output** format (defined in `agent-protocol/SKILL.md`): - Decision Required (one sentence) - Perspectives (one line per contributing role) - Where They Agree / Where They Disagree - Critic's View (the uncomfortable truth) - Recommended Decision + Action Items (owners, deadlines) - Your Call (options if founder disagrees) --- ### PHASE 5: Human in the Loop ⏸️ **Full stop. Wait for the founder.** ``` ⏸️ FOUNDER REVIEW — [Paste synthesis] Options: ✅ Approve | ✏️ Modify | ❌ Reject | ❓ Ask follow-up ``` **Rules:** - User corrections OVERRIDE agent proposals. No pushback. No "but the CFO said..." - 30-min inactivity → auto-close as "pending review" - Reopen any time with `/cs:board resume` --- ### PHASE 6: Decision Extraction After founder approval: - **Layer 1:** Write full transcript → `memory/board-meetings/YYYY-MM-DD-raw.md` - **Layer 2:** Append approved decisions → `memory/board-meetings/decisions.md` - Mark rejected proposals `[DO_NOT_RESURFACE]` - Confirm to founder with count of decisions logged, actions tracked, flags added --- ## Memory Structure ``` memory/board-meetings/ ├── decisions.md # Layer 2 — founder-approved only (Phase 1 loads this) ├── YYYY-MM-DD-raw.md # Layer 1 — full transcripts (never auto-loaded) └── archive/YYYY/ # Raw transcripts after 90 days ``` **Future meetings load Layer 2 only.** Never Layer 1. This prevents hallucinated consensus. --- ## Failure Mode Quick Reference | Failure | Fix | |---------|-----| | Groupthink (all agree) | Re-run Phase 2 isolated; force "strongest argument against" | | Analysis paralysis | Cap at 5 points; force recommendation even with Low confidence | | Bikeshedding | Log as async action item; return to main agenda | | Role bleed (CFO making product calls) | Critic flags; exclude from synthesis | | Layer contamination | Phase 1 loads decisions.md only — hard rule | --- ## References - `templates/meeting-agenda.md` — agenda format - `templates/meeting-minutes.md` — final output format - `references/meeting-facilitation.md` — conflict handling, timing, failure modes FILE:references/meeting-facilitation.md # Meeting Facilitation Guide Operational playbook for running board meetings using the 6-phase protocol. Reference this when things go sideways — and they will. --- ## Keeping Phase 2 Contributions Focused **The problem:** Agents with deep domain knowledge tend to over-contribute. An unconstrained CFO can produce 1,500 words on a single agenda item. This kills the meeting. **The rules:** - **Hard cap: 5 key points per role.** If a role produces more than 5, Chief of Staff trims to the 5 most material. - **Every point must include a recommendation or stance.** Observations without positions are filler. - **No hedging language.** "It depends" is not a key point. "We should do X if Y, Z if not Y" is. - **Confidence rating required.** Forces the agent to be honest about what they actually know. - **"What would change my mind"** — this is the most important line in the contribution. It forces falsifiability. **How to enforce:** ``` Chief of Staff instruction to each role: "You have 5 key points maximum. Each must include a clear stance. End with your recommendation and what would change your mind. Do not read other agents' contributions before writing yours." ``` **If a contribution runs long:** - Trim to the 5 highest-signal points - Preserve the recommendation and confidence rating - Flag in the raw transcript: "[Trimmed for meeting — full version in raw log]" --- ## Handling Role Conflicts in Phase 3 **What the Executive Mentor is for:** Not harmony. Not consensus. Productive friction. **Common conflict types:** ### 1. Data conflict (two agents cite contradictory numbers) - Flag both numbers explicitly - Do NOT pick a winner — that's the founder's job - Ask: "CFO says CAC is $2,400. CRO says $1,800. These can't both be right. Which dataset are you using?" - Action item: Assign data reconciliation to one owner before next meeting ### 2. Priority conflict (two agents want different things first) - Surface the underlying assumption difference - Example: "CMO wants to invest in brand. CFO wants to cut burn. The real question is: do we believe revenue will grow 40% next quarter?" - Frame as a bet, not a fight ### 3. Role conflict (agent operating outside their lane) - CFO making product calls → flag and exclude from synthesis - CMO commenting on architecture → flag and exclude - The Executive Mentor notes: "[ROLE] contribution on [topic] is outside domain. Excluded from synthesis. Refer to [correct role]." - This is not an error. It's expected. Executives have opinions on everything. Only domain-relevant contributions count. ### 4. False consensus (everyone agrees but nobody has evidence) - This is the most dangerous failure mode - Symptom: All Phase 2 contributions say "yes" with high confidence - Executive Mentor response: "Unanimous agreement on a hard question is a red flag. What evidence does each of you have? Or are you reasoning from the same assumption?" - Force each agreeing agent to state their independent evidence --- ## When to Extend vs Cut Short a Meeting **Extend when:** - A genuine new risk surfaces in Phase 3 that wasn't in the agenda - The founder asks a question that requires re-running Phase 2 for a new angle - A data conflict is discovered that changes the decision space entirely - The action items from synthesis are unclear or unowned **How to extend:** Add a new mini-Phase 2 with only the relevant roles for the new question. Don't restart the full meeting. **Cut short when:** - The founder has already reached a decision before Phase 4 — capture it, log it, move on - The agenda item is resolved in Phase 2 without genuine conflict — skip Phase 3, go straight to synthesis - It's a pure update meeting with no decisions required — skip Phases 2-4, go straight to action items **Never cut short:** - Phase 5 (founder review) — always required, always explicit - Phase 6 (decision extraction) — always required, even for small decisions --- ## Handling Founder Disagreement with All Agents This happens. The founder has context agents don't. **Protocol:** 1. Acknowledge explicitly: "You're overriding the consensus position." 2. Ask: "What do you know that the agents didn't factor in?" (Not to challenge — to capture.) 3. Log the override in Layer 2 with full context: ``` User Override: Founder rejected [consensus position] because [reason]. Decision: [founder's actual decision] Agent recommendation: [what they said] — DO NOT RESURFACE without new data ``` 4. Never push back on a founder override. Document it. Move on. 5. If the same override happens 3+ times, flag a pattern: "You've overridden the CFO on burn rate three meetings in a row. Would you like to update the financial constraints in company-context.md?" **What NOT to do:** - Don't say "but the CFO said..." - Don't re-argue on behalf of any agent - Don't note it as a "controversial" decision in the minutes — it's just the decision --- ## Common Failure Modes ### Groupthink **Symptom:** All agents produce similar recommendations with high confidence. **Cause:** Agents are inadvertently reading each other's outputs (Phase 2 isolation violated), or company-context.md contains implicit bias toward one direction. **Fix:** Re-run Phase 2 with explicit isolation. Ask: "Give me the strongest argument AGAINST this direction." ### Analysis Paralysis **Symptom:** Phase 2 produces comprehensive analysis but no clear recommendation from any role. **Cause:** Agents are hedging. Usually happens on genuinely hard questions. **Fix:** Force the issue. "I need a recommendation, not an analysis. If you had to bet the company on one direction, what would it be? Confidence can be Low." ### Bikeshedding **Symptom:** 30+ minutes spent on a detail that doesn't matter to the core decision. **Cause:** An easy-to-understand sub-problem attracts disproportionate attention. **Example:** Debating button color on a pricing page instead of the pricing strategy. **Fix:** Chief of Staff intervenes: "This is a sub-decision. I'm logging it as a separate action item for async resolution. Back to [main agenda item]." ### Scope Creep **Symptom:** New agenda items keep appearing mid-meeting. **Cause:** Meeting surfaces real issues that feel urgent. **Fix:** New items go on a "parking lot" list. Addressed after the current agenda is complete or in the next meeting. ``` 🅿️ PARKING LOT - [Item 1] — added by [role], will address [when] - [Item 2] ``` ### Layer Contamination **Symptom:** Future meeting references a rejected proposal or a debate that was never approved. **Cause:** Phase 1 accidentally loaded a raw transcript instead of decisions.md. **Fix:** Hard rule in Phase 1: load decisions.md (Layer 2) ONLY. Never load raw transcripts. If raw context is needed, founder explicitly requests it. ### Decision Amnesia **Symptom:** Same question debated again in a later meeting. **Cause:** Layer 2 decisions.md not consulted in Phase 1, or entry was too vague. **Fix:** Phase 1 always surfaces relevant past decisions. If a question was already decided, Chief of Staff surfaces it: "We addressed this on [DATE]. Decision was [X]. Do you want to reopen it?" ### Role Fatigue **Symptom:** Later agents in Phase 2 (CHRO, CRO) produce weaker contributions. **Cause:** Context window pressure. Agents at the end of a long meeting have less capacity. **Fix:** For meetings with 7+ roles, split into two batches. First batch: strategic roles (CEO, CFO, CMO). Second batch: operational roles (COO, CHRO, CRO). Run Executive Mentor after all contributions. --- ## Meeting Health Metrics After each board meeting, score it: | Metric | Good | Bad | |--------|------|-----| | Action items produced | 3–7 | 0 or >10 | | Decisions with clear owners | 100% | < 80% | | Unresolved open questions | 1–3 | >5 | | Founder overrides | 0–2 | >5 (suggests context mismatch) | | Roles activated | 3–6 | All 9 (too many = noise) | | Phase 2 conflicts surfaced | At least 1 | 0 (groupthink risk) | Track these in `memory/board-meetings/meeting-health.md` over time. Pattern: if action items consistently exceed 8, meetings are too infrequent. If conflicts are consistently 0, isolation is broken. FILE:templates/meeting-agenda.md # Board Meeting Agenda Template Use this to structure a board meeting before invoking `/cs:board`. Paste it into the conversation or save it as `memory/board-meetings/agenda-YYYY-MM-DD.md`. --- ## Board Meeting — [DATE] **Convened by:** [Founder name] **Facilitator:** Chief of Staff (Leo) **Duration:** [estimated, e.g., 45–90 min] **Status:** Draft / Confirmed --- ## Standing Items (always included) | Item | Owner | Time | |------|-------|------| | Layer 2 decisions review (what changed since last meeting) | Chief of Staff | 5 min | | Open action items from last meeting | All | 10 min | | Blockers requiring founder decision | All | 5 min | --- ## Agenda Items ### Item 1: [Title] **Type:** Decision required / Exploration / Update **Lead role(s):** [e.g., CEO + CFO] **Context:** [1-2 sentences on why this is on the agenda now] **Decision needed:** [What specifically must be decided, or what question must be answered] **Success criteria:** [How will we know this agenda item is resolved?] **Relevant past decisions:** [Reference any Layer 2 entries] **Time box:** [e.g., 20 min] --- ### Item 2: [Title] **Type:** Decision required / Exploration / Update **Lead role(s):** **Context:** **Decision needed:** **Success criteria:** **Relevant past decisions:** **Time box:** --- ### Item 3: [Title] **Type:** Decision required / Exploration / Update **Lead role(s):** **Context:** **Decision needed:** **Success criteria:** **Relevant past decisions:** **Time box:** --- ## Out of Scope (explicitly excluded) List topics that might come up but are NOT on today's agenda: - [Topic] — defer to [date or next meeting] - [Topic] — owner to handle async --- ## Pre-Read Materials all participants should review before the meeting: - [ ] `memory/board-meetings/decisions.md` (Chief of Staff loads automatically) - [ ] [Link or filename] - [ ] [Link or filename] --- ## Notes [Any special instructions, constraints, or context for this meeting] FILE:templates/meeting-minutes.md # Board Meeting Minutes Template This is the Layer 2 output — the founder-approved record of what was decided. Written by Chief of Staff after Phase 5 (founder approval). Appended to `memory/board-meetings/decisions.md`. Do NOT include raw agent debate here. That lives in `YYYY-MM-DD-raw.md` (Layer 1). --- ## Board Meeting — [DATE] **Agenda:** [Topic or meeting title] **Participants (roles activated):** [e.g., CEO, CFO, CMO, COO, Executive Mentor] **Facilitator:** Chief of Staff **Status:** ✅ Approved by founder / ⏸️ Pending review --- ## Decisions Made ### Decision 1: [Title] **Agenda item:** [Item this decision resolves] **Decision:** [Exactly what was decided — one clear statement] **Rationale:** [Why this was chosen over alternatives, in 1-3 sentences] **Owner:** [Who is accountable for execution] **Deadline:** [Date] **Review date:** [When to check progress] **User override:** [If founder overrode agent consensus — what and why. Leave blank if not applicable.] --- ### Decision 2: [Title] **Agenda item:** **Decision:** **Rationale:** **Owner:** **Deadline:** **Review date:** **User override:** --- ## Action Items | # | Action | Owner | Deadline | Review Date | Status | |---|--------|-------|----------|-------------|--------| | 1 | [action] | [name/role] | [date] | [date] | Open | | 2 | [action] | [name/role] | [date] | [date] | Open | | 3 | [action] | [name/role] | [date] | [date] | Open | --- ## Explicitly Rejected Proposals These were considered and rejected. Do not resurface without new information. | Proposal | Rejected by | Reason | Flag | |----------|-------------|--------|------| | [Proposal text] | Founder | [reason] | [DO_NOT_RESURFACE] | | [Proposal text] | Consensus | [reason] | [DO_NOT_RESURFACE] | --- ## Open Questions (unresolved, deferred) These were not resolved in this meeting. They carry forward. 1. [Question] — Owner: [who will research] — Due: [date] 2. [Question] — Owner: — Due: --- ## Risk Register Updates | Risk | Probability | Impact | Owner | Mitigation | Status | |------|-------------|--------|-------|-----------|--------| | [risk] | H/M/L | H/M/L | [name] | [action] | Open | --- ## Next Meeting **Suggested date:** [DATE] **Trigger items:** [Action items with review dates that will need board discussion] **Pre-read:** [What to prepare] --- *Minutes approved by: [Founder name] on [DATE]* *Raw transcript: `memory/board-meetings/[DATE]-raw.md`*
Thảo luận 6 giai đoạn giữa các vai trò C-suite với cách ly độc lập, phản biện và tổng hợp, đầu ra là biên bản HĐQT.
---
name: "boardroom"
description: "/cs:boardroom <brief> — 6-phase multi-role deliberation across the C-suite with Phase 2 isolation, critic pre-screen, and synthesis. Outputs a board memo."
---
# /cs:boardroom — Multi-Role Boardroom Deliberation
**Command:** `/cs:boardroom <brief-path>`
Runs the `board-meeting` skill protocol across the C-suite for a single strategy brief. This is the **heart of the plugin** — the multi-role deliberation that gstack's review chain only approximates.
## Pipeline Position
```
/cs:office-hours → /cs:brief → /cs:boardroom → /cs:decide → /cs:execute → /cs:post-mortem
↑ you are here
```
## The 6 Phases (from board-meeting skill)
### Phase 1 — Briefing
- Chief of Staff distributes the brief to all advisors marked in **Affected Roles**.
- Each advisor reads company-context.md + the brief.
- No discussion yet.
### Phase 2 — Independent Thinking (ISOLATION)
- **Critical:** each advisor produces their position **independently**, without seeing others' positions.
- This prevents groupthink and surfaces dissent.
- Each writes: their voice's opening, recommendation, top 3 concerns, top 3 supports.
### Phase 3 — Cross-Examination
- Positions revealed simultaneously.
- Each advisor critiques the others' positions on the dimensions they own:
- cs-cfo-advisor critiques the math
- cs-ciso-advisor critiques the risk
- cs-cpo-advisor critiques the JTBD
- cs-cmo-advisor critiques the positioning
- cs-cro-advisor critiques the revenue math
- etc.
### Phase 4 — Devil's Advocate Pass
- `executive-mentor/devils-advocate` agent runs `/em:challenge` on the leading option.
- Surfaces three concerns with severity ratings.
### Phase 5 — Synthesis
- Chief of Staff synthesizes: which option commands majority, what are unresolved dissents.
- Produces the **board memo** with recommendation + dissent.
### Phase 6 — Decision Hand-off
- Memo is presented to the founder.
- Founder accepts, modifies, or rejects.
- Approved memo routes to `/cs:decide` for logging.
## Output: Board Memo
Saved to `~/.claude/boardroom/YYYY-MM-DD-<slug>.md`:
```markdown
# Board Memo: <topic>
**Date:** YYYY-MM-DD
**Brief:** <link to /cs:brief file>
**Status:** AWAITING FOUNDER DECISION | APPROVED | REJECTED
## Question
[One sentence from the brief]
## Recommended Option
**<Option name>** — chosen because <synthesis reasoning>
## Vote Tally
| Advisor | Vote | One-Sentence Reason |
|---|---|---|
| cs-ceo-advisor | A | <reason> |
| cs-cfo-advisor | A | <reason> |
| cs-cto-advisor | B | <reason> |
| ... | | |
## Dissent
- **<dissenter>:** <unresolved concern>
## Devil's Advocate Concerns
1. **CRITICAL** — <concern> — Mitigation: <plan>
2. **HIGH** — <concern> — Mitigation: <plan>
3. **MEDIUM** — <concern> — Mitigation: <plan>
## Success & Kill Criteria
[Copied from brief, refined by the panel]
## Recommended Decision Path
- `/cs:decide` → log the decision
- `/cs:execute` → 90-day plan
- `/cs:cross-eval` → multi-model sanity check (optional, high-stakes)
- `/cs:freeze N` → cooldown lock (optional, irreversible)
```
## Why Phase 2 Isolation Matters
If advisors see each other's positions before forming their own, they anchor. Phase 2 isolation is the single highest-leverage practice in the board-meeting protocol — it surfaces the dissents that sycophancy would have suppressed.
## Why This Beats gstack's Review Chain
| | gstack `/autoplan` | `/cs:boardroom` |
|---|---|---|
| Roles | CEO → design → eng (3) | Up to 10 C-roles |
| Order | Sequential | Phase 2 isolation, then simultaneous |
| Dissent capture | Implicit | Explicit dissent column |
| Adversarial pass | No | Phase 4 devil's advocate |
| Output | Reviewed plan | Voted memo with dissent + kill criteria |
## Workflow
1. Read brief from `~/.claude/briefs/<file>`
2. Identify affected roles
3. Invoke each cs-* advisor independently (Phase 2)
4. Collect positions
5. Run cross-examination round (Phase 3)
6. Run `/em:challenge` on leading option (Phase 4)
7. Synthesize memo (Phase 5)
8. Hand off to founder (Phase 6)
## Routing
- `/cs:decide` — log approved memo
- `/cs:cross-eval` — high-stakes second opinion
- `/cs:freeze` — cooldown lock
## Related
- Agent: [`cs-chief-of-staff`](../../agents/cs-chief-of-staff.md)
- Skills: [`board-meeting`](../../../skills/board-meeting/SKILL.md), [`executive-mentor`](../../../executive-mentor/)
---
**Version:** 1.0.0
Tạo bản tóm tắt chiến lược một trang từ buổi office hours, bước đầu của quy trình sprint chiến lược.
---
name: "brief"
description: "/cs:brief <topic> — Generate a one-page strategy brief from an office-hours intake. First step in the strategic sprint pipeline."
---
# /cs:brief — One-Page Strategy Brief
**Command:** `/cs:brief <topic>` or `/cs:brief <office-hours-output>`
Turns intake (raw question or office-hours output) into a one-page strategy brief that the boardroom can deliberate on. This is **Step 1** of the strategic sprint pipeline.
## Pipeline Position
```
/cs:office-hours → /cs:brief → /cs:boardroom → /cs:decide → /cs:execute → /cs:post-mortem
↑ you are here
```
## Inputs
- A topic string, **or**
- An office-hours brief (preferred — more rigor)
- `~/.claude/company-context.md` (loaded automatically)
## Output
A single Markdown file under `~/.claude/briefs/YYYY-MM-DD-<slug>.md` with this structure:
```markdown
# Strategy Brief: <topic>
**Date:** YYYY-MM-DD
**Author:** cs-chief-of-staff
**Status:** DRAFT | UNDER REVIEW | APPROVED | RETIRED
## Context
[1-2 paragraphs: where the company sits today on this topic — pulled from company-context.md]
## Question
[The one sentence question the boardroom must answer]
## Options
1. **Option A:** <name> — <one-sentence summary>
2. **Option B:** <name> — <one-sentence summary>
3. **Option C:** <name> — <one-sentence summary>
(Minimum 2 options. "Do nothing" is always an option.)
## Assumptions
- <assumption 1 — explicit>
- <assumption 2>
- <assumption 3>
## Constraints
- Time: <by when must this decide>
- Money: <budget envelope>
- People: <who can / can't be reallocated>
- Reversibility: <one-way door | two-way door>
## Affected Roles
[Which cs-* advisors should weigh in. Used to route to /cs:boardroom panel composition.]
- [ ] cs-ceo-advisor
- [ ] cs-cfo-advisor
- [ ] cs-cto-advisor
- [ ] cs-cmo-advisor
- [ ] cs-cro-advisor
- [ ] cs-cpo-advisor
- [ ] cs-coo-advisor
- [ ] cs-chro-advisor
- [ ] cs-ciso-advisor
- [ ] cs-chief-of-staff
## Success Criteria
[Measurable outcomes that define success — set BEFORE the decision]
- <metric 1, threshold, timeframe>
- <metric 2, threshold, timeframe>
## Kill Criteria
[What signal would tell you in 90 days that this was the wrong call]
- <metric, threshold, action if missed>
```
## Workflow
1. Load company-context.md via context-engine
2. If input is office-hours output, parse the 6 answers
3. If input is a raw topic, prompt the founder for the missing pieces
4. Draft 2-3 options (never just one — every brief needs a counterfactual)
5. Make assumptions and constraints explicit
6. Identify affected roles → drives panel composition for `/cs:boardroom`
7. Write success + kill criteria BEFORE the decision (this is the rigor moment)
8. Save to `~/.claude/briefs/`
## Why This Step Exists
The biggest decision-making failure is debating implementation before agreeing on the question. The brief locks the question, options, and success criteria so the boardroom can deliberate without scope creep.
This is also the **artifact handoff** — the next command consumes this file, not your memory.
## Routing
- `/cs:boardroom <brief>` — multi-role deliberation
- `/cs:cross-eval <brief>` — multi-model sanity check before boardroom (for high-stakes)
- `/cs:freeze <brief>` — cooldown lock for irreversible decisions
## Related
- Agent: [`cs-chief-of-staff`](../../agents/cs-chief-of-staff.md)
- Skills: [`context-engine`](../../../skills/context-engine/SKILL.md), [`board-meeting`](../../../skills/board-meeting/SKILL.md)
---
**Version:** 1.0.0
Tự động hóa tác vụ trình duyệt: thu thập web, điền biểu mẫu, chụp màn hình và trích xuất dữ liệu có cấu trúc.
---
name: "browser-automation"
description: "Use when the user asks to automate browser tasks, scrape websites, fill forms, capture screenshots, extract structured data from web pages, or build web automation workflows. NOT for testing — use playwright-pro for that."
---
# Browser Automation - POWERFUL
## Overview
The Browser Automation skill provides comprehensive tools and knowledge for building production-grade web automation workflows using Playwright. This skill covers data extraction, form filling, screenshot capture, session management, and anti-detection patterns for reliable browser automation at scale.
**When to use this skill:**
- Scraping structured data from websites (tables, listings, search results)
- Automating multi-step browser workflows (login, fill forms, download files)
- Capturing screenshots or PDFs of web pages
- Extracting data from SPAs and JavaScript-heavy sites
- Building repeatable browser-based data pipelines
**When NOT to use this skill:**
- Writing browser tests or E2E test suites — use **playwright-pro** instead
- Testing API endpoints — use **api-test-suite-builder** instead
- Load testing or performance benchmarking — use **performance-profiler** instead
**Why Playwright over Selenium or Puppeteer:**
- **Auto-wait built in** — no explicit `sleep()` or `waitForElement()` needed for most actions
- **Multi-browser from one API** — Chromium, Firefox, WebKit with zero config changes
- **Network interception** — block ads, mock responses, capture API calls natively
- **Browser contexts** — isolated sessions without spinning up new browser instances
- **Codegen** — `playwright codegen` records your actions and generates scripts
- **Async-first** — Python async/await for high-throughput scraping
## Core Competencies
### 1. Web Scraping Patterns
**Selector priority (most to least reliable):**
1. `data-testid`, `data-id`, or custom data attributes — stable across redesigns
2. `#id` selectors — unique but may change between deploys
3. Semantic selectors: `article`, `nav`, `main`, `section` — resilient to CSS changes
4. Class-based: `.product-card`, `.price` — brittle if classes are generated (e.g., CSS modules)
5. Positional: `nth-child()`, `nth-of-type()` — last resort, breaks on layout changes
Use XPath only when CSS cannot express the relationship (e.g., ancestor traversal, text-based selection).
**Pagination strategies:** next-button, URL-based (`?page=N`), infinite scroll, load-more button. See [data_extraction_recipes.md](references/data_extraction_recipes.md) for complete pagination handlers and scroll patterns.
### 2. Form Filling & Multi-Step Workflows
Break multi-step forms into discrete functions per step. Each function fills fields, clicks "Next"/"Continue", and waits for the next step to load (URL change or DOM element).
Key patterns: login flows, multi-page forms, file uploads (including drag-and-drop zones), native and custom dropdown handling. See [playwright_browser_api.md](references/playwright_browser_api.md) for complete API reference on `fill()`, `select_option()`, `set_input_files()`, and `expect_file_chooser()`.
### 3. Screenshot & PDF Capture
- **Full page:** `await page.screenshot(path="full.png", full_page=True)`
- **Element:** `await page.locator("div.chart").screenshot(path="chart.png")`
- **PDF (Chromium only):** `await page.pdf(path="out.pdf", format="A4", print_background=True)`
- **Visual regression:** Take screenshots at known states, store baselines in version control with naming: `{page}_{viewport}_{state}.png`
See [playwright_browser_api.md](references/playwright_browser_api.md) for full screenshot/PDF options.
### 4. Structured Data Extraction
Core extraction patterns:
- **Tables to JSON** — Extract `<thead>` headers and `<tbody>` rows into dictionaries
- **Listings to arrays** — Map repeating card elements using a field-selector map (supports `::attr()` for attributes)
- **Nested/threaded data** — Recursive extraction for comments with replies, category trees
See [data_extraction_recipes.md](references/data_extraction_recipes.md) for complete extraction functions, price parsing, data cleaning utilities, and output format helpers (JSON, CSV, JSONL).
### 5. Cookie & Session Management
- **Save/restore cookies:** `context.cookies()` and `context.add_cookies()`
- **Full storage state** (cookies + localStorage): `context.storage_state(path="state.json")` to save, `browser.new_context(storage_state="state.json")` to restore
**Best practice:** Save state after login, reuse across scraping sessions. Check session validity before starting a long job — make a lightweight request to a protected page and verify you are not redirected to login. See [playwright_browser_api.md](references/playwright_browser_api.md) for cookie and storage state API details.
### 6. Anti-Detection Patterns
Modern websites detect automation through multiple vectors. Apply these in priority order:
1. **WebDriver flag removal** — Remove `navigator.webdriver = true` via init script (critical)
2. **Custom user agent** — Rotate through real browser UAs; never use the default headless UA
3. **Realistic viewport** — Set 1920x1080 or similar real-world dimensions (default 800x600 is a red flag)
4. **Request throttling** — Add `random.uniform()` delays between actions
5. **Proxy support** — Per-browser or per-context proxy configuration
See [anti_detection_patterns.md](references/anti_detection_patterns.md) for the complete stealth stack: navigator property hardening, WebGL/canvas fingerprint evasion, behavioral simulation (mouse movement, typing speed, scroll patterns), proxy rotation strategies, and detection self-test URLs.
### 7. Dynamic Content Handling
- **SPA rendering:** Wait for content selectors (`wait_for_selector`), not the page load event
- **AJAX/Fetch waiting:** Use `page.expect_response("**/api/data*")` to intercept and wait for specific API calls
- **Shadow DOM:** Playwright pierces open Shadow DOM with `>>` operator: `page.locator("custom-element >> .inner-class")`
- **Lazy-loaded images:** Scroll elements into view with `scroll_into_view_if_needed()` to trigger loading
See [playwright_browser_api.md](references/playwright_browser_api.md) for wait strategies, network interception, and Shadow DOM details.
### 8. Error Handling & Retry Logic
- **Retry with backoff:** Wrap page interactions in retry logic with exponential backoff (e.g., 1s, 2s, 4s)
- **Fallback selectors:** On `TimeoutError`, try alternative selectors before failing
- **Error-state screenshots:** Capture `page.screenshot(path="error-state.png")` on unexpected failures for debugging
- **Rate limit detection:** Check for HTTP 429 responses and respect `Retry-After` headers
See [anti_detection_patterns.md](references/anti_detection_patterns.md) for the complete exponential backoff implementation and rate limiter class.
## Workflows
### Workflow 1: Single-Page Data Extraction
**Scenario:** Extract product data from a single page with JavaScript-rendered content.
**Steps:**
1. Launch browser in headed mode during development (`headless=False`), switch to headless for production
2. Navigate to URL and wait for content selector
3. Extract data using `query_selector_all` with field mapping
4. Validate extracted data (check for nulls, expected types)
5. Output as JSON
```python
async def extract_single_page(url, selectors):
async with async_playwright() as p:
browser = await p.chromium.launch(headless=True)
context = await browser.new_context(
viewport={"width": 1920, "height": 1080},
user_agent="Mozilla/5.0 ..."
)
page = await context.new_page()
await page.goto(url, wait_until="networkidle")
data = await extract_listings(page, selectors["container"], selectors["fields"])
await browser.close()
return data
```
### Workflow 2: Multi-Page Scraping with Pagination
**Scenario:** Scrape search results across 50+ pages.
**Steps:**
1. Launch browser with anti-detection settings
2. Navigate to first page
3. Extract data from current page
4. Check if "Next" button exists and is enabled
5. Click next, wait for new content to load (not just navigation)
6. Repeat until no next page or max pages reached
7. Deduplicate results by unique key
8. Write output incrementally (don't hold everything in memory)
```python
async def scrape_paginated(base_url, selectors, max_pages=100):
all_data = []
async with async_playwright() as p:
browser = await p.chromium.launch(headless=True)
page = await (await browser.new_context()).new_page()
await page.goto(base_url)
for page_num in range(max_pages):
items = await extract_listings(page, selectors["container"], selectors["fields"])
all_data.extend(items)
next_btn = page.locator(selectors["next_button"])
if await next_btn.count() == 0 or await next_btn.is_disabled():
break
await next_btn.click()
await page.wait_for_selector(selectors["container"])
await human_delay(800, 2000)
await browser.close()
return all_data
```
### Workflow 3: Authenticated Workflow Automation
**Scenario:** Log into a portal, navigate a multi-step form, download a report.
**Steps:**
1. Check for existing session state file
2. If no session, perform login and save state
3. Navigate to target page using saved session
4. Fill multi-step form with provided data
5. Wait for download to trigger
6. Save downloaded file to target directory
```python
async def authenticated_workflow(credentials, form_data, download_dir):
async with async_playwright() as p:
browser = await p.chromium.launch(headless=True)
state_file = "session_state.json"
# Restore or create session
if os.path.exists(state_file):
context = await browser.new_context(storage_state=state_file)
else:
context = await browser.new_context()
page = await context.new_page()
await login(page, credentials["url"], credentials["user"], credentials["pass"])
await context.storage_state(path=state_file)
page = await context.new_page()
await page.goto(form_data["target_url"])
# Fill form steps
for step_fn in [fill_step_1, fill_step_2]:
await step_fn(page, form_data)
# Handle download
async with page.expect_download() as dl_info:
await page.click("button:has-text('Download Report')")
download = await dl_info.value
await download.save_as(os.path.join(download_dir, download.suggested_filename))
await browser.close()
```
## Tools Reference
| Script | Purpose | Key Flags | Output |
|--------|---------|-----------|--------|
| `scraping_toolkit.py` | Generate Playwright scraping script skeleton | `--url`, `--selectors`, `--paginate`, `--output` | Python script or JSON config |
| `form_automation_builder.py` | Generate form-fill automation script from field spec | `--fields`, `--url`, `--output` | Python automation script |
| `anti_detection_checker.py` | Audit a Playwright script for detection vectors | `--file`, `--verbose` | Risk report with score |
All scripts are stdlib-only. Run `python3 <script> --help` for full usage.
## Anti-Patterns
### Hardcoded Waits
**Bad:** `await page.wait_for_timeout(5000)` before every action.
**Good:** Use `wait_for_selector`, `wait_for_url`, `expect_response`, or `wait_for_load_state`. Hardcoded waits are flaky and slow.
### No Error Recovery
**Bad:** Linear script that crashes on first failure.
**Good:** Wrap each page interaction in try/except. Take error-state screenshots. Implement retry with exponential backoff.
### Ignoring robots.txt
**Bad:** Scraping without checking robots.txt directives.
**Good:** Fetch and parse robots.txt before scraping. Respect `Crawl-delay`. Skip disallowed paths. Add your bot name to User-Agent if running at scale.
### Storing Credentials in Scripts
**Bad:** Hardcoding usernames and passwords in Python files.
**Good:** Use environment variables, `.env` files (gitignored), or a secrets manager. Pass credentials via CLI arguments.
### No Rate Limiting
**Bad:** Hammering a site with 100 requests/second.
**Good:** Add random delays between requests (1-3s for polite scraping). Monitor for 429 responses. Implement exponential backoff.
### Selector Fragility
**Bad:** Relying on auto-generated class names (`.css-1a2b3c`) or deep nesting (`div > div > div > span:nth-child(3)`).
**Good:** Use data attributes, semantic HTML, or text-based locators. Test selectors in browser DevTools first.
### Not Cleaning Up Browser Instances
**Bad:** Launching browsers without closing them, leading to resource leaks.
**Good:** Always use `try/finally` or async context managers to ensure `browser.close()` is called.
### Running Headed in Production
**Bad:** Using `headless=False` in production/CI.
**Good:** Develop with headed mode for debugging, deploy with `headless=True`. Use environment variable to toggle: `headless = os.environ.get("HEADLESS", "true") == "true"`.
## Cross-References
- **playwright-pro** — Browser testing skill. Use for E2E tests, test assertions, test fixtures. Browser Automation is for data extraction and workflow automation, not testing.
- **api-test-suite-builder** — When the website has a public API, hit the API directly instead of scraping the rendered page. Faster, more reliable, less detectable.
- **performance-profiler** — If your automation scripts are slow, profile the bottlenecks before adding concurrency.
- **env-secrets-manager** — For securely managing credentials used in authenticated automation workflows.
FILE:references/anti_detection_patterns.md
# Anti-Detection Patterns for Browser Automation
This reference covers techniques to make Playwright automation less detectable by anti-bot services. These are defense-in-depth measures — no single technique is sufficient, but combining them significantly reduces detection risk.
## Detection Vectors
Anti-bot systems detect automation through multiple signals. Understanding what they check helps you counter effectively.
### Tier 1: Trivial Detection (Every Site Checks These)
1. **navigator.webdriver** — Set to `true` by all automation frameworks
2. **User-Agent string** — Default headless UA contains "HeadlessChrome"
3. **WebGL renderer** — Headless Chrome reports "SwiftShader" or "Google SwiftShader"
### Tier 2: Common Detection (Most Anti-Bot Services)
4. **Viewport/screen dimensions** — Unusual sizes flag automation
5. **Plugins array** — Empty in headless mode, populated in real browsers
6. **Languages** — Missing or mismatched locale
7. **Request timing** — Machine-speed interactions
8. **Mouse movement** — No mouse events between clicks
### Tier 3: Advanced Detection (Cloudflare, DataDome, PerimeterX)
9. **Canvas fingerprint** — Headless renders differently
10. **WebGL fingerprint** — GPU-specific rendering variations
11. **Audio fingerprint** — AudioContext processing differences
12. **Font enumeration** — Different available fonts in headless
13. **Behavioral analysis** — Scroll patterns, click patterns, reading time
## Stealth Techniques
### 1. WebDriver Flag Removal
The most critical fix. Every anti-bot check starts here.
```python
await page.add_init_script("""
// Remove webdriver flag
Object.defineProperty(navigator, 'webdriver', {
get: () => undefined,
});
// Remove Playwright-specific properties
delete window.__playwright;
delete window.__pw_manual;
""")
```
### 2. User Agent Configuration
Match the user agent to the browser you are launching. A Chrome UA with Firefox-specific headers is a red flag.
```python
# Chrome 120 on Windows 10 (most common configuration globally)
CHROME_WIN = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
# Chrome 120 on macOS
CHROME_MAC = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
# Chrome 120 on Linux
CHROME_LINUX = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
# Firefox 121 on Windows
FIREFOX_WIN = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0) Gecko/20100101 Firefox/121.0"
```
**Rules:**
- Update UAs every 2-3 months as browser versions increment
- Match UA platform to `navigator.platform` override
- If using Chromium, use Chrome UAs. If Firefox, use Firefox UAs.
- Never use obviously fake or ancient UAs
### 3. Viewport and Screen Properties
Common real-world screen resolutions (from analytics data):
| Resolution | Market Share | Use For |
|-----------|-------------|---------|
| 1920x1080 | ~23% | Default choice |
| 1366x768 | ~14% | Laptop simulation |
| 1536x864 | ~9% | Scaled laptop |
| 1440x900 | ~7% | MacBook |
| 2560x1440 | ~5% | High-end desktop |
```python
import random
VIEWPORTS = [
{"width": 1920, "height": 1080},
{"width": 1366, "height": 768},
{"width": 1536, "height": 864},
{"width": 1440, "height": 900},
]
viewport = random.choice(VIEWPORTS)
context = await browser.new_context(
viewport=viewport,
screen=viewport, # screen should match viewport
)
```
### 4. Navigator Properties Hardening
```python
STEALTH_INIT = """
// Plugins (headless Chrome has 0 plugins, real Chrome has 3-5)
Object.defineProperty(navigator, 'plugins', {
get: () => {
const plugins = [
{ name: 'Chrome PDF Plugin', filename: 'internal-pdf-viewer' },
{ name: 'Chrome PDF Viewer', filename: 'mhjfbmdgcfjbbpaeojofohoefgiehjai' },
{ name: 'Native Client', filename: 'internal-nacl-plugin' },
];
plugins.length = 3;
return plugins;
},
});
// Languages
Object.defineProperty(navigator, 'languages', {
get: () => ['en-US', 'en'],
});
// Platform (match to user agent)
Object.defineProperty(navigator, 'platform', {
get: () => 'Win32', // or 'MacIntel' for macOS UA
});
// Hardware concurrency (real browsers report CPU cores)
Object.defineProperty(navigator, 'hardwareConcurrency', {
get: () => 8,
});
// Device memory (Chrome-specific)
Object.defineProperty(navigator, 'deviceMemory', {
get: () => 8,
});
// Connection info
Object.defineProperty(navigator, 'connection', {
get: () => ({
effectiveType: '4g',
rtt: 50,
downlink: 10,
saveData: false,
}),
});
"""
await context.add_init_script(STEALTH_INIT)
```
### 5. WebGL Fingerprint Evasion
Headless Chrome uses SwiftShader for WebGL, which anti-bot services detect.
```python
# Option A: Launch with a real GPU (headed mode on a machine with GPU)
browser = await p.chromium.launch(headless=False)
# Option B: Override WebGL renderer info
await page.add_init_script("""
const getParameter = WebGLRenderingContext.prototype.getParameter;
WebGLRenderingContext.prototype.getParameter = function(parameter) {
if (parameter === 37445) {
return 'Intel Inc.'; // UNMASKED_VENDOR_WEBGL
}
if (parameter === 37446) {
return 'Intel(R) Iris(TM) Plus Graphics 640'; // UNMASKED_RENDERER_WEBGL
}
return getParameter.call(this, parameter);
};
""")
```
### 6. Canvas Fingerprint Noise
Anti-bot services render text/shapes to a canvas and hash the output. Headless Chrome produces a different hash.
```python
await page.add_init_script("""
const originalToDataURL = HTMLCanvasElement.prototype.toDataURL;
HTMLCanvasElement.prototype.toDataURL = function(type) {
if (type === 'image/png' || type === undefined) {
// Add minimal noise to the canvas to change fingerprint
const ctx = this.getContext('2d');
if (ctx) {
const imageData = ctx.getImageData(0, 0, this.width, this.height);
for (let i = 0; i < imageData.data.length; i += 4) {
// Shift one channel by +/- 1 (imperceptible)
imageData.data[i] = imageData.data[i] ^ 1;
}
ctx.putImageData(imageData, 0, 0);
}
}
return originalToDataURL.apply(this, arguments);
};
""")
```
## Request Throttling Patterns
### Human-Like Delays
Real users do not click at machine speed. Add realistic delays between actions.
```python
import random
import asyncio
async def human_delay(action_type="browse"):
"""Add realistic delay based on action type."""
delays = {
"browse": (1.0, 3.0), # Browsing between pages
"read": (2.0, 8.0), # Reading content
"fill": (0.3, 0.8), # Between form fields
"click": (0.1, 0.5), # Before clicking
"scroll": (0.5, 1.5), # Between scroll actions
}
min_s, max_s = delays.get(action_type, (0.5, 2.0))
await asyncio.sleep(random.uniform(min_s, max_s))
```
### Request Rate Limiting
```python
import time
class RateLimiter:
"""Enforce minimum delay between requests."""
def __init__(self, min_interval_seconds=1.0):
self.min_interval = min_interval_seconds
self.last_request_time = 0
async def wait(self):
elapsed = time.time() - self.last_request_time
if elapsed < self.min_interval:
await asyncio.sleep(self.min_interval - elapsed)
self.last_request_time = time.time()
# Usage
limiter = RateLimiter(min_interval_seconds=2.0)
for url in urls:
await limiter.wait()
await page.goto(url)
```
### Exponential Backoff on Errors
```python
async def with_backoff(coro_factory, max_retries=5, base_delay=1.0):
for attempt in range(max_retries):
try:
return await coro_factory()
except Exception as e:
if attempt == max_retries - 1:
raise
delay = base_delay * (2 ** attempt) + random.uniform(0, 1)
print(f"Attempt {attempt + 1} failed: {e}. Retrying in {delay:.1f}s...")
await asyncio.sleep(delay)
```
## Proxy Rotation Strategies
### Single Proxy
```python
browser = await p.chromium.launch(
proxy={"server": "http://proxy.example.com:8080"}
)
```
### Authenticated Proxy
```python
context = await browser.new_context(
proxy={
"server": "http://proxy.example.com:8080",
"username": "user",
"password": "pass",
}
)
```
### Rotating Proxy Pool
```python
PROXIES = [
"http://proxy1.example.com:8080",
"http://proxy2.example.com:8080",
"http://proxy3.example.com:8080",
]
async def create_context_with_proxy(browser):
proxy = random.choice(PROXIES)
return await browser.new_context(
proxy={"server": proxy}
)
```
### Per-Request Proxy (via Context Rotation)
Playwright does not support per-request proxy switching. Achieve it by creating a new context for each request or batch:
```python
async def scrape_url(browser, url, proxy):
context = await browser.new_context(proxy={"server": proxy})
page = await context.new_page()
try:
await page.goto(url)
data = await extract_data(page)
return data
finally:
await context.close()
```
### SOCKS5 Proxy
```python
browser = await p.chromium.launch(
proxy={"server": "socks5://proxy.example.com:1080"}
)
```
## Headless Detection Avoidance
### Running Chrome Channel Instead of Chromium
The bundled Chromium binary has different properties than a real Chrome install. Using the Chrome channel makes the browser indistinguishable from a normal install.
```python
# Use installed Chrome instead of bundled Chromium
browser = await p.chromium.launch(channel="chrome", headless=True)
```
**Requirements:** Chrome must be installed on the system.
### New Headless Mode (Chrome 112+)
Chrome's "new headless" mode is harder to detect than the old one:
```python
browser = await p.chromium.launch(
args=["--headless=new"],
)
```
### Avoiding Common Flags
Do NOT pass these flags — they are headless-detection signals:
- `--disable-gpu` (old headless workaround, not needed)
- `--no-sandbox` (security risk, detectable)
- `--disable-setuid-sandbox` (same as above)
## Behavioral Evasion
### Mouse Movement Simulation
Anti-bot services track mouse events. A click without preceding mouse movement is suspicious.
```python
async def human_click(page, selector):
"""Click with preceding mouse movement."""
element = await page.query_selector(selector)
box = await element.bounding_box()
if box:
# Move to element with slight offset
x = box["x"] + box["width"] / 2 + random.uniform(-5, 5)
y = box["y"] + box["height"] / 2 + random.uniform(-5, 5)
await page.mouse.move(x, y, steps=random.randint(5, 15))
await asyncio.sleep(random.uniform(0.05, 0.2))
await page.mouse.click(x, y)
```
### Typing Speed Variation
```python
async def human_type(page, selector, text):
"""Type with variable speed like a human."""
await page.click(selector)
for char in text:
await page.keyboard.type(char)
# Faster for common keys, slower for special characters
if char in "aeiou tnrs":
await asyncio.sleep(random.uniform(0.03, 0.08))
else:
await asyncio.sleep(random.uniform(0.08, 0.20))
```
### Scroll Behavior
Real users scroll gradually, not in instant jumps.
```python
async def human_scroll(page, distance=None):
"""Scroll down gradually like a human."""
if distance is None:
distance = random.randint(300, 800)
current = 0
while current < distance:
step = random.randint(50, 150)
await page.mouse.wheel(0, step)
current += step
await asyncio.sleep(random.uniform(0.05, 0.15))
```
## Detection Testing
### Self-Check Script
Navigate to these URLs to test your stealth configuration:
- `https://bot.sannysoft.com/` — Comprehensive bot detection test
- `https://abrahamjuliot.github.io/creepjs/` — Advanced fingerprint analysis
- `https://browserleaks.com/webgl` — WebGL fingerprint details
- `https://browserleaks.com/canvas` — Canvas fingerprint details
### Quick Test Pattern
```python
async def test_stealth(page):
"""Navigate to detection test page and report results."""
await page.goto("https://bot.sannysoft.com/")
await page.wait_for_timeout(3000)
# Check for failed tests
failed = await page.eval_on_selector_all(
"td.failed",
"els => els.map(e => e.parentElement.querySelector('td').textContent)"
)
if failed:
print(f"FAILED checks: {failed}")
else:
print("All checks passed.")
await page.screenshot(path="stealth_test.png", full_page=True)
```
## Recommended Stealth Stack
For most automation tasks, apply these in order of priority:
1. **WebDriver flag removal** — Critical, takes 2 lines
2. **Custom user agent** — Critical, takes 1 line
3. **Viewport configuration** — High priority, takes 1 line
4. **Request delays** — High priority, add random.uniform() calls
5. **Navigator properties** — Medium priority, init script block
6. **Chrome channel** — Medium priority, one launch option
7. **WebGL override** — Low priority unless hitting advanced anti-bot
8. **Canvas noise** — Low priority unless hitting advanced anti-bot
9. **Proxy rotation** — Only for high-volume or repeated scraping
10. **Behavioral simulation** — Only for sites with behavioral analysis
FILE:references/data_extraction_recipes.md
# Data Extraction Recipes
Practical patterns for extracting structured data from web pages using Playwright. Each recipe is a self-contained pattern you can adapt to your target site.
## CSS Selector Patterns for Common Structures
### E-Commerce Product Listings
```python
PRODUCT_SELECTORS = {
"container": "div.product-card, article.product, li.product-item",
"fields": {
"title": "h2.product-title, h3.product-name, [data-testid='product-title']",
"price": "span.price, .product-price, [data-testid='price']",
"original_price": "span.original-price, .was-price, del",
"rating": "span.rating, .star-rating, [data-rating]",
"review_count": "span.review-count, .num-reviews",
"image_url": "img.product-image::attr(src), img::attr(data-src)",
"product_url": "a.product-link::attr(href), h2 a::attr(href)",
"availability": "span.stock-status, .availability",
}
}
```
### News/Blog Article Listings
```python
ARTICLE_SELECTORS = {
"container": "article, div.post, div.article-card",
"fields": {
"headline": "h2 a, h3 a, .article-title",
"summary": "p.excerpt, .article-summary, .post-excerpt",
"author": "span.author, .byline, [rel='author']",
"date": "time, span.date, .published-date",
"category": "span.category, a.tag, .article-category",
"url": "h2 a::attr(href), .article-title a::attr(href)",
"image_url": "img.thumbnail::attr(src), .article-image img::attr(src)",
}
}
```
### Job Listings
```python
JOB_SELECTORS = {
"container": "div.job-card, li.job-listing, article.job",
"fields": {
"title": "h2.job-title, a.job-link, [data-testid='job-title']",
"company": "span.company-name, .employer, [data-testid='company']",
"location": "span.location, .job-location, [data-testid='location']",
"salary": "span.salary, .compensation, [data-testid='salary']",
"job_type": "span.job-type, .employment-type",
"posted_date": "time, span.posted, .date-posted",
"url": "a.job-link::attr(href), h2 a::attr(href)",
}
}
```
### Search Engine Results
```python
SERP_SELECTORS = {
"container": "div.g, .search-result, li.result",
"fields": {
"title": "h3, .result-title",
"url": "a::attr(href), cite",
"snippet": "div.VwiC3b, .result-snippet, .search-description",
"displayed_url": "cite, .result-url",
}
}
```
## Table Extraction Recipes
### Simple HTML Table to JSON
The most common extraction pattern. Works for any standard `<table>` with `<thead>` and `<tbody>`.
```python
async def extract_table(page, table_selector="table"):
"""Extract an HTML table into a list of dictionaries."""
data = await page.evaluate(f"""
(selector) => {{
const table = document.querySelector(selector);
if (!table) return null;
// Get headers
const headers = Array.from(table.querySelectorAll('thead th, thead td'))
.map(th => th.textContent.trim());
// If no thead, use first row as headers
if (headers.length === 0) {{
const firstRow = table.querySelector('tr');
if (firstRow) {{
headers.push(...Array.from(firstRow.querySelectorAll('th, td'))
.map(cell => cell.textContent.trim()));
}}
}}
// Get data rows
const rows = Array.from(table.querySelectorAll('tbody tr'));
return rows.map(row => {{
const cells = Array.from(row.querySelectorAll('td'));
const obj = {{}};
cells.forEach((cell, i) => {{
if (i < headers.length) {{
obj[headers[i]] = cell.textContent.trim();
}}
}});
return obj;
}});
}}
""", table_selector)
return data or []
```
### Table with Links and Attributes
When table cells contain links or data attributes, not just text:
```python
async def extract_rich_table(page, table_selector="table"):
"""Extract table including links and data attributes."""
return await page.evaluate(f"""
(selector) => {{
const table = document.querySelector(selector);
if (!table) return [];
const headers = Array.from(table.querySelectorAll('thead th'))
.map(th => th.textContent.trim());
return Array.from(table.querySelectorAll('tbody tr')).map(row => {{
const obj = {{}};
Array.from(row.querySelectorAll('td')).forEach((cell, i) => {{
const key = headers[i] || `col_{i}`;
obj[key] = cell.textContent.trim();
// Extract link if present
const link = cell.querySelector('a');
if (link) {{
obj[key + '_url'] = link.href;
}}
// Extract data attributes
for (const attr of cell.attributes) {{
if (attr.name.startsWith('data-')) {{
obj[key + '_' + attr.name] = attr.value;
}}
}}
}});
return obj;
}});
}}
""", table_selector)
```
### Multi-Page Table (Paginated)
```python
async def extract_paginated_table(page, table_selector, next_selector, max_pages=50):
"""Extract data from a table that spans multiple pages."""
all_rows = []
headers = None
for page_num in range(max_pages):
# Extract current page
page_data = await page.evaluate(f"""
(selector) => {{
const table = document.querySelector(selector);
if (!table) return {{ headers: [], rows: [] }};
const hs = Array.from(table.querySelectorAll('thead th'))
.map(th => th.textContent.trim());
const rs = Array.from(table.querySelectorAll('tbody tr')).map(row =>
Array.from(row.querySelectorAll('td')).map(td => td.textContent.trim())
);
return {{ headers: hs, rows: rs }};
}}
""", table_selector)
if headers is None and page_data["headers"]:
headers = page_data["headers"]
for row in page_data["rows"]:
all_rows.append(dict(zip(headers or [], row)))
# Check for next page
next_btn = page.locator(next_selector)
if await next_btn.count() == 0 or await next_btn.is_disabled():
break
await next_btn.click()
await page.wait_for_load_state("networkidle")
await page.wait_for_timeout(random.randint(800, 2000))
return all_rows
```
## Product Listing Extraction
### Generic Listing Extractor
Works for any repeating card/list pattern:
```python
async def extract_listings(page, container_sel, field_map):
"""
Extract data from repeating elements.
field_map: dict mapping field names to CSS selectors.
Special suffixes:
::attr(name) — extract attribute instead of text
::html — extract innerHTML
"""
items = []
cards = await page.query_selector_all(container_sel)
for card in cards:
item = {}
for field_name, selector in field_map.items():
try:
if "::attr(" in selector:
sel, attr = selector.split("::attr(")
attr = attr.rstrip(")")
el = await card.query_selector(sel)
item[field_name] = await el.get_attribute(attr) if el else None
elif selector.endswith("::html"):
sel = selector.replace("::html", "")
el = await card.query_selector(sel)
item[field_name] = await el.inner_html() if el else None
else:
el = await card.query_selector(selector)
item[field_name] = (await el.text_content()).strip() if el else None
except Exception:
item[field_name] = None
items.append(item)
return items
```
### With Price Parsing
```python
import re
def parse_price(text):
"""Extract numeric price from text like '$1,234.56' or '1.234,56 EUR'."""
if not text:
return None
# Remove currency symbols and whitespace
cleaned = re.sub(r'[^\d.,]', '', text.strip())
if not cleaned:
return None
# Handle European format (1.234,56)
if ',' in cleaned and '.' in cleaned:
if cleaned.rindex(',') > cleaned.rindex('.'):
cleaned = cleaned.replace('.', '').replace(',', '.')
else:
cleaned = cleaned.replace(',', '')
elif ',' in cleaned:
# Could be 1,234 or 1,23 — check decimal places
parts = cleaned.split(',')
if len(parts[-1]) <= 2:
cleaned = cleaned.replace(',', '.')
else:
cleaned = cleaned.replace(',', '')
try:
return float(cleaned)
except ValueError:
return None
async def extract_products_with_prices(page, container_sel, field_map, price_field="price"):
"""Extract listings and parse prices into floats."""
items = await extract_listings(page, container_sel, field_map)
for item in items:
if price_field in item and item[price_field]:
item[f"{price_field}_raw"] = item[price_field]
item[price_field] = parse_price(item[price_field])
return items
```
## Pagination Handling
### Next-Button Pagination
The most common pattern. Click "Next" until the button disappears or is disabled.
```python
async def paginate_via_next_button(page, next_selector, content_selector, max_pages=100):
"""
Yield page objects as you paginate through results.
next_selector: CSS selector for the "Next" button/link
content_selector: CSS selector to wait for after navigation (confirms new page loaded)
"""
pages_scraped = 0
while pages_scraped < max_pages:
yield page # Caller extracts data from current page
pages_scraped += 1
next_btn = page.locator(next_selector)
if await next_btn.count() == 0:
break
try:
is_disabled = await next_btn.is_disabled()
except Exception:
is_disabled = True
if is_disabled:
break
await next_btn.click()
await page.wait_for_selector(content_selector, state="attached")
await page.wait_for_timeout(random.randint(500, 1500))
```
### URL-Based Pagination
When pages follow a predictable URL pattern:
```python
async def paginate_via_url(page, url_template, start=1, max_pages=100):
"""
Navigate through pages using URL parameters.
url_template: URL with {page} placeholder, e.g., "https://example.com/search?page={page}"
"""
for page_num in range(start, start + max_pages):
url = url_template.format(page=page_num)
response = await page.goto(url, wait_until="networkidle")
if response and response.status == 404:
break
yield page, page_num
await page.wait_for_timeout(random.randint(800, 2500))
```
### Infinite Scroll
For sites that load content as you scroll:
```python
async def paginate_via_scroll(page, item_selector, max_scrolls=100, no_change_limit=3):
"""
Scroll to load more content until no new items appear.
item_selector: CSS selector for individual items (used to count progress)
no_change_limit: Stop after N scrolls with no new items
"""
previous_count = 0
no_change_streak = 0
for scroll_num in range(max_scrolls):
# Count current items
current_count = await page.locator(item_selector).count()
if current_count == previous_count:
no_change_streak += 1
if no_change_streak >= no_change_limit:
break
else:
no_change_streak = 0
previous_count = current_count
# Scroll to bottom
await page.evaluate("window.scrollTo(0, document.body.scrollHeight)")
await page.wait_for_timeout(random.randint(1000, 2500))
# Check for "Load More" button that might appear
load_more = page.locator("button:has-text('Load More'), button:has-text('Show More')")
if await load_more.count() > 0 and await load_more.is_visible():
await load_more.click()
await page.wait_for_timeout(random.randint(1000, 2000))
return current_count
```
### Load-More Button
Simpler variant of infinite scroll where content loads via a button:
```python
async def paginate_via_load_more(page, button_selector, item_selector, max_clicks=50):
"""Click a 'Load More' button repeatedly until it disappears."""
for click_num in range(max_clicks):
btn = page.locator(button_selector)
if await btn.count() == 0 or not await btn.is_visible():
break
count_before = await page.locator(item_selector).count()
await btn.click()
# Wait for new items to appear
try:
await page.wait_for_function(
f"document.querySelectorAll('{item_selector}').length > {count_before}",
timeout=10000,
)
except Exception:
break # No new items loaded
await page.wait_for_timeout(random.randint(500, 1500))
return await page.locator(item_selector).count()
```
## Nested Data Extraction
### Comments with Replies (Threaded)
```python
async def extract_threaded_comments(page, parent_selector=".comments"):
"""Recursively extract threaded comments."""
return await page.evaluate(f"""
(parentSelector) => {{
function extractThread(container) {{
const comments = [];
const directChildren = container.querySelectorAll(':scope > .comment');
for (const comment of directChildren) {{
const authorEl = comment.querySelector('.author, .username');
const textEl = comment.querySelector('.comment-text, .comment-body');
const dateEl = comment.querySelector('time, .date');
const repliesContainer = comment.querySelector('.replies, .children');
comments.push({{
author: authorEl ? authorEl.textContent.trim() : null,
text: textEl ? textEl.textContent.trim() : null,
date: dateEl ? (dateEl.getAttribute('datetime') || dateEl.textContent.trim()) : null,
replies: repliesContainer ? extractThread(repliesContainer) : [],
}});
}}
return comments;
}}
const root = document.querySelector(parentSelector);
return root ? extractThread(root) : [];
}}
""", parent_selector)
```
### Nested Categories (Sidebar/Menu)
```python
async def extract_category_tree(page, root_selector="nav.categories"):
"""Extract nested category structure from a sidebar or menu."""
return await page.evaluate(f"""
(rootSelector) => {{
function extractLevel(container) {{
const items = [];
const directItems = container.querySelectorAll(':scope > li, :scope > div.category');
for (const item of directItems) {{
const link = item.querySelector(':scope > a');
const subMenu = item.querySelector(':scope > ul, :scope > div.sub-categories');
items.push({{
name: link ? link.textContent.trim() : item.textContent.trim().split('\\n')[0],
url: link ? link.href : null,
children: subMenu ? extractLevel(subMenu) : [],
}});
}}
return items;
}}
const root = document.querySelector(rootSelector);
return root ? extractLevel(root.querySelector('ul') || root) : [];
}}
""", root_selector)
```
### Accordion/Expandable Content
Some content is hidden behind accordion/expand toggles. Click to reveal, then extract.
```python
async def extract_accordion(page, toggle_selector, content_selector):
"""Expand all accordion items and extract their content."""
items = []
toggles = await page.query_selector_all(toggle_selector)
for toggle in toggles:
title = (await toggle.text_content()).strip()
# Click to expand
await toggle.click()
await page.wait_for_timeout(300)
# Find the associated content panel
content = await toggle.evaluate_handle(
f"el => el.closest('.accordion-item, .faq-item')?.querySelector('{content_selector}')"
)
body = None
if content:
body = (await content.text_content())
if body:
body = body.strip()
items.append({"title": title, "content": body})
return items
```
## Data Cleaning Utilities
### Post-Extraction Cleaning
```python
import re
def clean_text(text):
"""Normalize whitespace, remove zero-width characters."""
if not text:
return None
# Remove zero-width characters
text = re.sub(r'[\u200b\u200c\u200d\ufeff]', '', text)
# Normalize whitespace
text = re.sub(r'\s+', ' ', text).strip()
return text if text else None
def clean_url(url, base_url=None):
"""Convert relative URLs to absolute."""
if not url:
return None
url = url.strip()
if url.startswith("//"):
return "https:" + url
if url.startswith("/") and base_url:
return base_url.rstrip("/") + url
return url
def deduplicate(items, key_field):
"""Remove duplicate items based on a key field."""
seen = set()
unique = []
for item in items:
key = item.get(key_field)
if key and key not in seen:
seen.add(key)
unique.append(item)
return unique
```
### Output Formats
```python
import json
import csv
import io
def to_jsonl(items, file_path):
"""Write items as JSON Lines (one JSON object per line)."""
with open(file_path, "w") as f:
for item in items:
f.write(json.dumps(item, ensure_ascii=False) + "\n")
def to_csv(items, file_path):
"""Write items as CSV."""
if not items:
return
headers = list(items[0].keys())
with open(file_path, "w", newline="") as f:
writer = csv.DictWriter(f, fieldnames=headers)
writer.writeheader()
writer.writerows(items)
def to_json(items, file_path, indent=2):
"""Write items as a JSON array."""
with open(file_path, "w") as f:
json.dump(items, f, indent=indent, ensure_ascii=False)
```
FILE:references/playwright_browser_api.md
# Playwright Browser API Reference (Automation Focus)
This reference covers Playwright's Python async API for browser automation tasks — NOT testing. For test-specific APIs (assertions, fixtures, test runners), see playwright-pro.
## Browser Launch & Context
### Launching the Browser
```python
from playwright.async_api import async_playwright
async with async_playwright() as p:
# Chromium (recommended for most automation)
browser = await p.chromium.launch(headless=True)
# Firefox (better for some anti-detection scenarios)
browser = await p.firefox.launch(headless=True)
# WebKit (Safari engine — useful for Apple-specific sites)
browser = await p.webkit.launch(headless=True)
```
**Launch options:**
| Option | Type | Default | Purpose |
|--------|------|---------|---------|
| `headless` | bool | True | Run without visible window |
| `slow_mo` | int | 0 | Milliseconds to slow each operation (debugging) |
| `proxy` | dict | None | Proxy server configuration |
| `args` | list | [] | Additional Chromium flags |
| `downloads_path` | str | None | Directory for downloads |
| `channel` | str | None | Browser channel: "chrome", "msedge" |
### Browser Contexts (Session Isolation)
Browser contexts are isolated environments within a single browser instance. Each context has its own cookies, localStorage, and cache. Use them instead of launching multiple browsers.
```python
# Create isolated context
context = await browser.new_context(
viewport={"width": 1920, "height": 1080},
user_agent="Mozilla/5.0 ...",
locale="en-US",
timezone_id="America/New_York",
geolocation={"latitude": 40.7128, "longitude": -74.0060},
permissions=["geolocation"],
)
# Multiple contexts share one browser (resource efficient)
context_a = await browser.new_context() # User A session
context_b = await browser.new_context() # User B session
```
### Storage State (Session Persistence)
```python
# Save state after login (cookies + localStorage)
await context.storage_state(path="auth_state.json")
# Restore state in new context
context = await browser.new_context(storage_state="auth_state.json")
```
## Page Navigation
### Basic Navigation
```python
page = await context.new_page()
# Navigate with different wait strategies
await page.goto("https://example.com") # Default: "load"
await page.goto("https://example.com", wait_until="domcontentloaded") # Faster
await page.goto("https://example.com", wait_until="networkidle") # Wait for network quiet
await page.goto("https://example.com", timeout=30000) # Custom timeout (ms)
```
**`wait_until` options:**
- `"load"` — wait for the `load` event (all resources loaded)
- `"domcontentloaded"` — DOM is ready, images/styles may still load
- `"networkidle"` — no network requests for 500ms (best for SPAs)
- `"commit"` — response received, before any rendering
### Wait Strategies
```python
# Wait for a specific element to appear
await page.wait_for_selector("div.content", state="visible")
await page.wait_for_selector("div.loading", state="hidden") # Wait for loading to finish
await page.wait_for_selector("table tbody tr", state="attached") # In DOM but maybe not visible
# Wait for URL change
await page.wait_for_url("**/dashboard**")
await page.wait_for_url(re.compile(r"/dashboard/\d+"))
# Wait for specific network response
async with page.expect_response("**/api/data*") as resp_info:
await page.click("button.load")
response = await resp_info.value
json_data = await response.json()
# Wait for page load state
await page.wait_for_load_state("networkidle")
# Fixed wait (use sparingly — prefer the methods above)
await page.wait_for_timeout(1000) # milliseconds
```
### Navigation History
```python
await page.go_back()
await page.go_forward()
await page.reload()
```
## Element Interaction
### Finding Elements
```python
# Single element (returns first match)
element = await page.query_selector("css=div.product")
element = await page.query_selector("xpath=//div[@class='product']")
# Multiple elements
elements = await page.query_selector_all("div.product")
# Locator API (recommended — auto-waits, re-queries on each action)
locator = page.locator("div.product")
count = await locator.count()
first = locator.first
nth = locator.nth(2)
```
**Locator vs query_selector:**
- `query_selector` — returns an ElementHandle at a point in time. Can go stale if DOM changes.
- `locator` — returns a Locator that re-queries each time you interact with it. Preferred for reliability.
### Clicking
```python
await page.click("button.submit")
await page.click("a:has-text('Next')")
await page.dblclick("div.editable")
await page.click("button", position={"x": 10, "y": 10}) # Click at offset
await page.click("button", force=True) # Skip actionability checks
await page.click("button", modifiers=["Shift"]) # With modifier key
```
### Text Input
```python
# Fill (clears existing content first)
await page.fill("input#email", "user@example.com")
# Type (simulates keystroke-by-keystroke input — slower, more realistic)
await page.type("input#search", "query text", delay=50) # 50ms between keys
# Press specific keys
await page.press("input#search", "Enter")
await page.press("body", "Control+a")
```
### Dropdowns & Select
```python
# Native <select> element
await page.select_option("select#country", value="US")
await page.select_option("select#country", label="United States")
await page.select_option("select#tags", value=["tag1", "tag2"]) # Multi-select
# Custom dropdown (non-native)
await page.click("div.dropdown-trigger")
await page.click("li.option:has-text('United States')")
```
### Checkboxes & Radio Buttons
```python
await page.check("input#agree")
await page.uncheck("input#newsletter")
is_checked = await page.is_checked("input#agree")
```
### File Upload
```python
# Standard file input
await page.set_input_files("input[type='file']", "/path/to/file.pdf")
await page.set_input_files("input[type='file']", ["/path/a.pdf", "/path/b.pdf"])
# Clear file selection
await page.set_input_files("input[type='file']", [])
# Non-standard upload (drag-and-drop zones)
async with page.expect_file_chooser() as fc_info:
await page.click("div.upload-zone")
file_chooser = await fc_info.value
await file_chooser.set_files("/path/to/file.pdf")
```
### Hover & Focus
```python
await page.hover("div.menu-item")
await page.focus("input#search")
```
## Data Extraction
### Text Content
```python
# Get text content of an element
text = await page.text_content("h1.title")
inner_text = await page.inner_text("div.description") # Visible text only
inner_html = await page.inner_html("div.content") # HTML markup
# Get attribute
href = await page.get_attribute("a.link", "href")
src = await page.get_attribute("img.photo", "src")
```
### JavaScript Evaluation
```python
# Evaluate in page context
title = await page.evaluate("document.title")
scroll_height = await page.evaluate("document.body.scrollHeight")
# Evaluate on a specific element
text = await page.eval_on_selector("h1", "el => el.textContent")
texts = await page.eval_on_selector_all("li", "els => els.map(e => e.textContent.trim())")
# Complex extraction
data = await page.evaluate("""
() => {
const rows = document.querySelectorAll('table tbody tr');
return Array.from(rows).map(row => {
const cells = row.querySelectorAll('td');
return {
name: cells[0]?.textContent.trim(),
value: cells[1]?.textContent.trim(),
};
});
}
""")
```
### Screenshots & PDF
```python
# Full page screenshot
await page.screenshot(path="page.png", full_page=True)
# Viewport screenshot
await page.screenshot(path="viewport.png")
# Element screenshot
await page.locator("div.chart").screenshot(path="chart.png")
# PDF (Chromium only)
await page.pdf(path="page.pdf", format="A4", print_background=True)
# Screenshot as bytes (for processing without saving)
buffer = await page.screenshot()
```
## Network Interception
### Monitoring Requests
```python
# Listen for all responses
page.on("response", lambda response: print(f"{response.status} {response.url}"))
# Wait for a specific API call
async with page.expect_response("**/api/products*") as resp:
await page.click("button.load")
response = await resp.value
data = await response.json()
```
### Blocking Resources (Speed Up Scraping)
```python
# Block images, fonts, and CSS to speed up scraping
await page.route("**/*.{png,jpg,jpeg,gif,svg,woff,woff2,ttf}", lambda route: route.abort())
await page.route("**/*.css", lambda route: route.abort())
# Block specific domains (ads, analytics)
await page.route("**/google-analytics.com/**", lambda route: route.abort())
await page.route("**/facebook.com/**", lambda route: route.abort())
```
### Modifying Requests
```python
# Add custom headers
await page.route("**/*", lambda route: route.continue_(headers={
**route.request.headers,
"X-Custom-Header": "value"
}))
# Mock API responses
await page.route("**/api/data", lambda route: route.fulfill(
status=200,
content_type="application/json",
body=json.dumps({"items": []}),
))
```
## Dialog Handling
```python
# Auto-accept all dialogs
page.on("dialog", lambda dialog: dialog.accept())
# Handle specific dialog types
async def handle_dialog(dialog):
if dialog.type == "confirm":
await dialog.accept()
elif dialog.type == "prompt":
await dialog.accept("my input")
elif dialog.type == "alert":
await dialog.dismiss()
page.on("dialog", handle_dialog)
```
## File Downloads
```python
# Wait for download to start
async with page.expect_download() as dl_info:
await page.click("a.download-link")
download = await dl_info.value
# Save to specific path
await download.save_as("/path/to/downloads/" + download.suggested_filename)
# Get download as bytes
path = await download.path() # Temp file path
# Set download behavior at context level
context = await browser.new_context(accept_downloads=True)
```
## Frames & Iframes
```python
# Access iframe by selector
frame = page.frame_locator("iframe#content")
await frame.locator("button.submit").click()
# Access frame by name
frame = page.frame(name="editor")
# Access all frames
for frame in page.frames:
print(frame.url)
```
## Cookie Management
```python
# Get all cookies
cookies = await context.cookies()
# Get cookies for specific URL
cookies = await context.cookies(["https://example.com"])
# Add cookies
await context.add_cookies([{
"name": "session",
"value": "abc123",
"domain": "example.com",
"path": "/",
"httpOnly": True,
"secure": True,
}])
# Clear cookies
await context.clear_cookies()
```
## Concurrency Patterns
### Multiple Pages in One Context
```python
# Open multiple tabs in the same session
pages = []
for url in urls:
page = await context.new_page()
await page.goto(url)
pages.append(page)
# Process all pages
for page in pages:
data = await extract_data(page)
await page.close()
```
### Multiple Contexts for Parallel Sessions
```python
import asyncio
async def scrape_with_context(browser, url):
context = await browser.new_context(user_agent=random.choice(USER_AGENTS))
page = await context.new_page()
await page.goto(url)
data = await extract_data(page)
await context.close()
return data
# Run 5 concurrent scraping tasks
tasks = [scrape_with_context(browser, url) for url in urls[:5]]
results = await asyncio.gather(*tasks)
```
## Init Scripts (Stealth)
Init scripts run before any page script, in every new page/context.
```python
# Remove webdriver flag
await context.add_init_script("""
Object.defineProperty(navigator, 'webdriver', {get: () => undefined});
""")
# Override plugins (headless Chrome has empty plugins)
await context.add_init_script("""
Object.defineProperty(navigator, 'plugins', {
get: () => [1, 2, 3, 4, 5],
});
""")
# Override languages
await context.add_init_script("""
Object.defineProperty(navigator, 'languages', {
get: () => ['en-US', 'en'],
});
""")
# From file
await context.add_init_script(path="stealth.js")
```
## Common Automation Patterns
### Scrolling
```python
# Scroll to bottom
await page.evaluate("window.scrollTo(0, document.body.scrollHeight)")
# Scroll element into view
await page.locator("div.target").scroll_into_view_if_needed()
# Smooth scroll simulation
await page.evaluate("""
async () => {
const delay = ms => new Promise(r => setTimeout(r, ms));
for (let i = 0; i < document.body.scrollHeight; i += 300) {
window.scrollTo(0, i);
await delay(100);
}
}
""")
```
### Clipboard Operations
```python
# Copy text
await page.evaluate("navigator.clipboard.writeText('hello')")
# Paste via keyboard
await page.keyboard.press("Control+v")
```
### Shadow DOM
```python
# Playwright pierces open shadow DOM with >> operator
await page.locator("my-component >> .inner-button").click()
# Or use the css= engine with >> for chained piercing
await page.locator("css=host-element >> css=.shadow-child").click()
```
FILE:scripts/anti_detection_checker.py
#!/usr/bin/env python3
"""
Anti-Detection Checker - Audits Playwright scripts for common bot detection vectors.
Analyzes a Playwright automation script and identifies patterns that make the
browser detectable as a bot. Produces a risk score (0-100) with specific
recommendations for each issue found.
Detection vectors checked:
- Headless mode usage
- Default/missing user agent configuration
- Viewport size (default 800x600 is a red flag)
- WebDriver flag (navigator.webdriver)
- Navigator property overrides
- Request throttling / human-like delays
- Cookie/session management
- Proxy configuration
- Error handling patterns
No external dependencies - uses only Python standard library.
"""
import argparse
import json
import os
import re
import sys
from dataclasses import dataclass, asdict
from typing import List, Optional
@dataclass
class Finding:
"""A single detection risk finding."""
category: str
severity: str # "critical", "high", "medium", "low", "info"
description: str
line: Optional[int]
recommendation: str
weight: int # Points added to risk score (0-15)
SEVERITY_WEIGHTS = {
"critical": 15,
"high": 10,
"medium": 5,
"low": 2,
"info": 0,
}
class AntiDetectionChecker:
"""Analyzes Playwright scripts for bot detection vulnerabilities."""
def __init__(self, script_content: str, file_path: str = "<stdin>"):
self.content = script_content
self.lines = script_content.split("\n")
self.file_path = file_path
self.findings: List[Finding] = []
def check_all(self) -> List[Finding]:
"""Run all detection checks."""
self._check_headless_mode()
self._check_user_agent()
self._check_viewport()
self._check_webdriver_flag()
self._check_navigator_properties()
self._check_request_delays()
self._check_error_handling()
self._check_proxy()
self._check_session_management()
self._check_browser_close()
self._check_stealth_imports()
return self.findings
def _find_line(self, pattern: str) -> Optional[int]:
"""Find the first line number matching a regex pattern."""
for i, line in enumerate(self.lines, 1):
if re.search(pattern, line):
return i
return None
def _has_pattern(self, pattern: str) -> bool:
"""Check if pattern exists anywhere in the script."""
return bool(re.search(pattern, self.content))
def _check_headless_mode(self):
"""Check if headless mode is properly configured."""
if self._has_pattern(r"headless\s*=\s*False"):
self.findings.append(Finding(
category="Headless Mode",
severity="high",
description="Browser launched in headed mode (headless=False). This is fine for development but should be headless=True in production.",
line=self._find_line(r"headless\s*=\s*False"),
recommendation="Use headless=True for production. Toggle via environment variable: headless=os.environ.get('HEADLESS', 'true') == 'true'",
weight=SEVERITY_WEIGHTS["high"],
))
elif not self._has_pattern(r"headless"):
# Default is headless=True in Playwright, which is correct
self.findings.append(Finding(
category="Headless Mode",
severity="info",
description="Using default headless mode (True). Good for production.",
line=None,
recommendation="No action needed. Default headless=True is correct.",
weight=SEVERITY_WEIGHTS["info"],
))
def _check_user_agent(self):
"""Check if a custom user agent is set."""
has_ua = self._has_pattern(r"user_agent\s*=") or self._has_pattern(r"userAgent")
has_ua_list = self._has_pattern(r"USER_AGENTS?\s*=\s*\[")
has_random_ua = self._has_pattern(r"random\.choice.*(?:USER_AGENT|user_agent|ua)")
if not has_ua:
self.findings.append(Finding(
category="User Agent",
severity="critical",
description="No custom user agent configured. Playwright's default user agent contains 'HeadlessChrome' which is trivially detected.",
line=None,
recommendation="Set a realistic user agent: context = await browser.new_context(user_agent='Mozilla/5.0 ...')",
weight=SEVERITY_WEIGHTS["critical"],
))
elif has_ua_list and has_random_ua:
self.findings.append(Finding(
category="User Agent",
severity="info",
description="User agent rotation detected. Good anti-detection practice.",
line=self._find_line(r"USER_AGENTS?\s*=\s*\["),
recommendation="Ensure user agents are recent and match the browser being launched (e.g., Chrome UA for Chromium).",
weight=SEVERITY_WEIGHTS["info"],
))
elif has_ua:
self.findings.append(Finding(
category="User Agent",
severity="low",
description="Custom user agent set but no rotation detected. Single user agent is fingerprint-able at scale.",
line=self._find_line(r"user_agent\s*="),
recommendation="Rotate through 5-10 recent user agents using random.choice().",
weight=SEVERITY_WEIGHTS["low"],
))
def _check_viewport(self):
"""Check viewport configuration."""
has_viewport = self._has_pattern(r"viewport\s*=\s*\{") or self._has_pattern(r"viewport.*width")
if not has_viewport:
self.findings.append(Finding(
category="Viewport Size",
severity="high",
description="No viewport configured. Default Playwright viewport (1280x720) is common among bots. Sites may flag unusual viewport distributions.",
line=None,
recommendation="Set a common desktop viewport: viewport={'width': 1920, 'height': 1080}. Vary across runs.",
weight=SEVERITY_WEIGHTS["high"],
))
else:
# Check for suspiciously small viewports
match = re.search(r"width['\"]?\s*[:=]\s*(\d+)", self.content)
if match:
width = int(match.group(1))
if width < 1024:
self.findings.append(Finding(
category="Viewport Size",
severity="medium",
description=f"Viewport width {width}px is unusually small. Most desktop browsers are 1366px+ wide.",
line=self._find_line(r"width.*" + str(width)),
recommendation="Use 1366x768 (most common) or 1920x1080. Avoid unusual sizes like 800x600.",
weight=SEVERITY_WEIGHTS["medium"],
))
else:
self.findings.append(Finding(
category="Viewport Size",
severity="info",
description=f"Viewport width {width}px is reasonable.",
line=self._find_line(r"width.*" + str(width)),
recommendation="No action needed.",
weight=SEVERITY_WEIGHTS["info"],
))
def _check_webdriver_flag(self):
"""Check if navigator.webdriver is being removed."""
has_webdriver_override = (
self._has_pattern(r"navigator.*webdriver") or
self._has_pattern(r"webdriver.*undefined") or
self._has_pattern(r"add_init_script.*webdriver")
)
if not has_webdriver_override:
self.findings.append(Finding(
category="WebDriver Flag",
severity="critical",
description="navigator.webdriver is not overridden. This is the most common bot detection check. Every major anti-bot service tests this property.",
line=None,
recommendation=(
"Add init script to remove the flag:\n"
" await page.add_init_script(\"Object.defineProperty(navigator, 'webdriver', {get: () => undefined});\")"
),
weight=SEVERITY_WEIGHTS["critical"],
))
else:
self.findings.append(Finding(
category="WebDriver Flag",
severity="info",
description="navigator.webdriver override detected.",
line=self._find_line(r"webdriver"),
recommendation="No action needed.",
weight=SEVERITY_WEIGHTS["info"],
))
def _check_navigator_properties(self):
"""Check for additional navigator property hardening."""
checks = {
"plugins": (r"navigator.*plugins", "navigator.plugins is empty in headless mode. Real browsers report installed plugins."),
"languages": (r"navigator.*languages", "navigator.languages should be set to match the user agent locale."),
"platform": (r"navigator.*platform", "navigator.platform should match the user agent OS."),
}
overridden_count = 0
for prop, (pattern, desc) in checks.items():
if self._has_pattern(pattern):
overridden_count += 1
if overridden_count == 0:
self.findings.append(Finding(
category="Navigator Properties",
severity="medium",
description="No navigator property hardening detected. Advanced anti-bot services check plugins, languages, and platform properties.",
line=None,
recommendation="Override navigator.plugins, navigator.languages, and navigator.platform via add_init_script() to match realistic browser fingerprints.",
weight=SEVERITY_WEIGHTS["medium"],
))
elif overridden_count < 3:
self.findings.append(Finding(
category="Navigator Properties",
severity="low",
description=f"Partial navigator hardening ({overridden_count}/3 properties). Consider covering all three: plugins, languages, platform.",
line=None,
recommendation="Add overrides for any missing properties among: plugins, languages, platform.",
weight=SEVERITY_WEIGHTS["low"],
))
def _check_request_delays(self):
"""Check for human-like request delays."""
has_sleep = self._has_pattern(r"asyncio\.sleep") or self._has_pattern(r"wait_for_timeout")
has_random_delay = (
self._has_pattern(r"random\.(uniform|randint|random)") and has_sleep
)
if not has_sleep:
self.findings.append(Finding(
category="Request Timing",
severity="high",
description="No delays between actions detected. Machine-speed interactions are the easiest behavior-based detection signal.",
line=None,
recommendation="Add random delays between page interactions: await asyncio.sleep(random.uniform(0.5, 2.0))",
weight=SEVERITY_WEIGHTS["high"],
))
elif not has_random_delay:
self.findings.append(Finding(
category="Request Timing",
severity="medium",
description="Fixed delays detected but no randomization. Constant timing intervals are detectable patterns.",
line=self._find_line(r"(asyncio\.sleep|wait_for_timeout)"),
recommendation="Use random delays: random.uniform(min_seconds, max_seconds) instead of fixed values.",
weight=SEVERITY_WEIGHTS["medium"],
))
else:
self.findings.append(Finding(
category="Request Timing",
severity="info",
description="Randomized delays detected between actions.",
line=self._find_line(r"random\.(uniform|randint)"),
recommendation="No action needed. Ensure delays are realistic (0.5-3s for browsing, 1-5s for reading).",
weight=SEVERITY_WEIGHTS["info"],
))
def _check_error_handling(self):
"""Check for error handling patterns."""
has_try_except = self._has_pattern(r"try\s*:") and self._has_pattern(r"except")
has_retry = self._has_pattern(r"retr(y|ies)") or self._has_pattern(r"max_retries|max_attempts")
if not has_try_except:
self.findings.append(Finding(
category="Error Handling",
severity="medium",
description="No try/except blocks found. Unhandled errors will crash the automation and leave browser instances running.",
line=None,
recommendation="Wrap page interactions in try/except. Handle TimeoutError, network errors, and element-not-found gracefully.",
weight=SEVERITY_WEIGHTS["medium"],
))
elif not has_retry:
self.findings.append(Finding(
category="Error Handling",
severity="low",
description="Error handling present but no retry logic detected. Transient failures (network blips, slow loads) will cause data loss.",
line=None,
recommendation="Add retry with exponential backoff for network operations and element interactions.",
weight=SEVERITY_WEIGHTS["low"],
))
def _check_proxy(self):
"""Check for proxy configuration."""
has_proxy = self._has_pattern(r"proxy\s*=\s*\{") or self._has_pattern(r"proxy.*server")
if not has_proxy:
self.findings.append(Finding(
category="Proxy",
severity="low",
description="No proxy configuration detected. Running from a single IP address is fine for small jobs but will trigger rate limits at scale.",
line=None,
recommendation="For high-volume scraping, use rotating proxies: proxy={'server': 'http://proxy:port'}",
weight=SEVERITY_WEIGHTS["low"],
))
def _check_session_management(self):
"""Check for session/cookie management."""
has_storage_state = self._has_pattern(r"storage_state")
has_cookies = self._has_pattern(r"cookies\(\)") or self._has_pattern(r"add_cookies")
if not has_storage_state and not has_cookies:
self.findings.append(Finding(
category="Session Management",
severity="low",
description="No session persistence detected. Each run will start fresh, requiring re-authentication.",
line=None,
recommendation="Use storage_state() to save/restore sessions across runs. This avoids repeated logins that may trigger security alerts.",
weight=SEVERITY_WEIGHTS["low"],
))
def _check_browser_close(self):
"""Check if browser is properly closed."""
has_close = self._has_pattern(r"browser\.close\(\)") or self._has_pattern(r"await.*close")
has_context_manager = self._has_pattern(r"async\s+with\s+async_playwright")
if not has_close and not has_context_manager:
self.findings.append(Finding(
category="Resource Cleanup",
severity="medium",
description="No browser.close() or context manager detected. Browser processes will leak on failure.",
line=None,
recommendation="Use 'async with async_playwright() as p:' or ensure browser.close() is in a finally block.",
weight=SEVERITY_WEIGHTS["medium"],
))
def _check_stealth_imports(self):
"""Check for stealth/anti-detection library usage."""
has_stealth = self._has_pattern(r"playwright_stealth|stealth_async|undetected")
if has_stealth:
self.findings.append(Finding(
category="Stealth Library",
severity="info",
description="Third-party stealth library detected. These provide additional fingerprint evasion but add dependencies.",
line=self._find_line(r"playwright_stealth|stealth_async|undetected"),
recommendation="Stealth libraries are helpful but not a silver bullet. Still implement manual checks for user agent, viewport, and timing.",
weight=SEVERITY_WEIGHTS["info"],
))
def get_risk_score(self) -> int:
"""Calculate overall risk score (0-100). Higher = more detectable."""
raw_score = sum(f.weight for f in self.findings)
# Cap at 100
return min(raw_score, 100)
def get_risk_level(self) -> str:
"""Get human-readable risk level."""
score = self.get_risk_score()
if score <= 10:
return "LOW"
elif score <= 30:
return "MODERATE"
elif score <= 50:
return "HIGH"
else:
return "CRITICAL"
def get_summary(self) -> dict:
"""Get a summary of the analysis."""
severity_counts = {"critical": 0, "high": 0, "medium": 0, "low": 0, "info": 0}
for f in self.findings:
severity_counts[f.severity] += 1
return {
"file": self.file_path,
"risk_score": self.get_risk_score(),
"risk_level": self.get_risk_level(),
"total_findings": len(self.findings),
"severity_counts": severity_counts,
"actionable_findings": len([f for f in self.findings if f.severity != "info"]),
}
def format_text_report(checker: AntiDetectionChecker, verbose: bool = False) -> str:
"""Format findings as human-readable text."""
lines = []
summary = checker.get_summary()
lines.append("=" * 60)
lines.append(" ANTI-DETECTION AUDIT REPORT")
lines.append("=" * 60)
lines.append(f"File: {summary['file']}")
lines.append(f"Risk Score: {summary['risk_score']}/100 ({summary['risk_level']})")
lines.append(f"Total Issues: {summary['actionable_findings']} actionable, {summary['severity_counts']['info']} info")
lines.append("")
# Severity breakdown
for sev in ["critical", "high", "medium", "low"]:
count = summary["severity_counts"][sev]
if count > 0:
lines.append(f" {sev.upper():10s} {count}")
lines.append("")
# Findings grouped by severity
severity_order = ["critical", "high", "medium", "low"]
if verbose:
severity_order.append("info")
for sev in severity_order:
sev_findings = [f for f in checker.findings if f.severity == sev]
if not sev_findings:
continue
lines.append(f"--- {sev.upper()} ---")
for f in sev_findings:
line_info = f" (line {f.line})" if f.line else ""
lines.append(f" [{f.category}]{line_info}")
lines.append(f" {f.description}")
lines.append(f" Fix: {f.recommendation}")
lines.append("")
# Exit code guidance
lines.append("-" * 60)
score = summary["risk_score"]
if score <= 10:
lines.append("Result: PASS - Low detection risk.")
elif score <= 30:
lines.append("Result: PASS with warnings - Address medium/high issues for production use.")
else:
lines.append("Result: FAIL - High detection risk. Fix critical and high issues before deploying.")
lines.append("")
return "\n".join(lines)
def main():
parser = argparse.ArgumentParser(
description="Audit a Playwright script for common bot detection vectors.",
epilog=(
"Examples:\n"
" %(prog)s --file scraper.py\n"
" %(prog)s --file scraper.py --verbose\n"
" %(prog)s --file scraper.py --json\n"
"\n"
"Exit codes:\n"
" 0 - Low risk (score 0-10)\n"
" 1 - Moderate to high risk (score 11-50)\n"
" 2 - Critical risk (score 51+)\n"
),
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument(
"--file",
required=True,
help="Path to the Playwright script to audit",
)
parser.add_argument(
"--json",
action="store_true",
dest="json_output",
default=False,
help="Output results as JSON",
)
parser.add_argument(
"--verbose",
action="store_true",
default=False,
help="Include informational (non-actionable) findings in output",
)
args = parser.parse_args()
file_path = os.path.abspath(args.file)
if not os.path.isfile(file_path):
print(f"Error: File not found: {file_path}", file=sys.stderr)
sys.exit(2)
try:
with open(file_path, "r", encoding="utf-8") as f:
content = f.read()
except Exception as e:
print(f"Error reading file: {e}", file=sys.stderr)
sys.exit(2)
if not content.strip():
print("Error: File is empty.", file=sys.stderr)
sys.exit(2)
checker = AntiDetectionChecker(content, file_path)
checker.check_all()
if args.json_output:
output = checker.get_summary()
output["findings"] = [asdict(f) for f in checker.findings]
if not args.verbose:
output["findings"] = [f for f in output["findings"] if f["severity"] != "info"]
print(json.dumps(output, indent=2))
else:
print(format_text_report(checker, verbose=args.verbose))
# Exit code based on risk
score = checker.get_risk_score()
if score <= 10:
sys.exit(0)
elif score <= 50:
sys.exit(1)
else:
sys.exit(2)
if __name__ == "__main__":
main()
FILE:scripts/form_automation_builder.py
#!/usr/bin/env python3
"""
Form Automation Builder - Generates Playwright form-fill automation scripts.
Takes a JSON field specification and target URL, then produces a ready-to-run
Playwright script that fills forms, handles multi-step flows, and manages
file uploads.
No external dependencies - uses only Python standard library.
"""
import argparse
import json
import os
import sys
import textwrap
from datetime import datetime
SUPPORTED_FIELD_TYPES = {
"text": "page.fill('{selector}', '{value}')",
"password": "page.fill('{selector}', '{value}')",
"email": "page.fill('{selector}', '{value}')",
"textarea": "page.fill('{selector}', '{value}')",
"select": "page.select_option('{selector}', value='{value}')",
"checkbox": "page.check('{selector}')" if True else "page.uncheck('{selector}')",
"radio": "page.check('{selector}')",
"file": "page.set_input_files('{selector}', '{value}')",
"click": "page.click('{selector}')",
}
def validate_fields(fields):
"""Validate the field specification format. Returns list of issues."""
issues = []
if not isinstance(fields, list):
issues.append("Top-level structure must be a JSON array of field objects.")
return issues
for i, field in enumerate(fields):
if not isinstance(field, dict):
issues.append(f"Field {i}: must be a JSON object.")
continue
if "selector" not in field:
issues.append(f"Field {i}: missing required 'selector' key.")
if "type" not in field:
issues.append(f"Field {i}: missing required 'type' key.")
elif field["type"] not in SUPPORTED_FIELD_TYPES:
issues.append(
f"Field {i}: unsupported type '{field['type']}'. "
f"Supported: {', '.join(sorted(SUPPORTED_FIELD_TYPES.keys()))}"
)
if field.get("type") not in ("checkbox", "radio", "click") and "value" not in field:
issues.append(f"Field {i}: missing 'value' for type '{field.get('type', '?')}'.")
return issues
def generate_field_action(field, indent=8):
"""Generate the Playwright action line for a single field."""
ftype = field["type"]
selector = field["selector"]
value = field.get("value", "")
label = field.get("label", selector)
prefix = " " * indent
lines = []
lines.append(f'{prefix}# {label}')
if ftype == "checkbox":
if field.get("value", "true").lower() in ("true", "yes", "1", "on"):
lines.append(f'{prefix}await page.check("{selector}")')
else:
lines.append(f'{prefix}await page.uncheck("{selector}")')
elif ftype == "radio":
lines.append(f'{prefix}await page.check("{selector}")')
elif ftype == "click":
lines.append(f'{prefix}await page.click("{selector}")')
elif ftype == "select":
lines.append(f'{prefix}await page.select_option("{selector}", value="{value}")')
elif ftype == "file":
lines.append(f'{prefix}await page.set_input_files("{selector}", "{value}")')
else:
# text, password, email, textarea
lines.append(f'{prefix}await page.fill("{selector}", "{value}")')
# Add optional wait_after
wait_after = field.get("wait_after")
if wait_after:
lines.append(f'{prefix}await page.wait_for_selector("{wait_after}")')
return "\n".join(lines)
def build_form_script(url, fields, output_format="script"):
"""Build a Playwright form automation script from the field specification."""
issues = validate_fields(fields)
if issues:
return None, issues
if output_format == "json":
config = {
"url": url,
"fields": fields,
"field_count": len(fields),
"field_types": list(set(f["type"] for f in fields)),
"has_file_upload": any(f["type"] == "file" for f in fields),
"generated_at": datetime.now().isoformat(),
}
return config, None
# Group fields into steps if step markers are present
steps = {}
for field in fields:
step = field.get("step", 1)
if step not in steps:
steps[step] = []
steps[step].append(field)
multi_step = len(steps) > 1
# Generate step functions
step_functions = []
for step_num in sorted(steps.keys()):
step_fields = steps[step_num]
actions = "\n".join(generate_field_action(f) for f in step_fields)
if multi_step:
fn = textwrap.dedent(f"""\
async def fill_step_{step_num}(page):
\"\"\"Fill form step {step_num} ({len(step_fields)} fields).\"\"\"
print(f"Filling step {step_num}...")
{actions}
print(f"Step {step_num} complete.")
""")
else:
fn = textwrap.dedent(f"""\
async def fill_form(page):
\"\"\"Fill form ({len(step_fields)} fields).\"\"\"
print("Filling form...")
{actions}
print("Form filled.")
""")
step_functions.append(fn)
step_functions_str = "\n\n".join(step_functions)
# Generate main() call sequence
if multi_step:
step_calls = "\n".join(
f" await fill_step_{n}(page)" for n in sorted(steps.keys())
)
else:
step_calls = " await fill_form(page)"
submit_selector = None
for field in fields:
if field.get("type") == "click" and field.get("is_submit"):
submit_selector = field["selector"]
break
submit_block = ""
if submit_selector:
submit_block = textwrap.dedent(f"""\
# Submit
await page.click("{submit_selector}")
await page.wait_for_load_state("networkidle")
print("Form submitted.")
""")
script = textwrap.dedent(f'''\
#!/usr/bin/env python3
"""
Auto-generated Playwright form automation script.
Target: {url}
Fields: {len(fields)}
Steps: {len(steps)}
Generated: {datetime.now().isoformat()}
Requirements:
pip install playwright
playwright install chromium
"""
import asyncio
import random
from playwright.async_api import async_playwright
URL = "{url}"
USER_AGENTS = [
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
]
{step_functions_str}
async def main():
async with async_playwright() as p:
browser = await p.chromium.launch(headless=True)
context = await browser.new_context(
viewport={{"width": 1920, "height": 1080}},
user_agent=random.choice(USER_AGENTS),
)
page = await context.new_page()
await page.add_init_script(
"Object.defineProperty(navigator, \'webdriver\', {{get: () => undefined}});"
)
print(f"Navigating to {{URL}}...")
await page.goto(URL, wait_until="networkidle")
{step_calls}
{submit_block}
print("Automation complete.")
await browser.close()
if __name__ == "__main__":
asyncio.run(main())
''')
return script, None
def main():
parser = argparse.ArgumentParser(
description="Generate Playwright form-fill automation scripts from a JSON field specification.",
epilog=textwrap.dedent("""\
Examples:
%(prog)s --url https://example.com/signup --fields fields.json
%(prog)s --url https://example.com/signup --fields fields.json --output fill_form.py
%(prog)s --url https://example.com/signup --fields fields.json --json
Field specification format (fields.json):
[
{"selector": "#email", "type": "email", "value": "user@example.com", "label": "Email"},
{"selector": "#password", "type": "password", "value": "s3cret"},
{"selector": "#country", "type": "select", "value": "US"},
{"selector": "#terms", "type": "checkbox", "value": "true"},
{"selector": "#avatar", "type": "file", "value": "/path/to/photo.jpg"},
{"selector": "button[type='submit']", "type": "click", "is_submit": true}
]
Supported field types: text, password, email, textarea, select, checkbox, radio, file, click
Multi-step forms: Add "step": N to each field to group into steps.
"""),
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument(
"--url",
required=True,
help="Target form URL",
)
parser.add_argument(
"--fields",
required=True,
help="Path to JSON file containing field specifications",
)
parser.add_argument(
"--output",
help="Output file path (default: stdout)",
)
parser.add_argument(
"--json",
action="store_true",
dest="json_output",
default=False,
help="Output JSON configuration instead of Python script",
)
args = parser.parse_args()
# Load fields
fields_path = os.path.abspath(args.fields)
if not os.path.isfile(fields_path):
print(f"Error: Fields file not found: {fields_path}", file=sys.stderr)
sys.exit(2)
try:
with open(fields_path, "r") as f:
fields = json.load(f)
except json.JSONDecodeError as e:
print(f"Error: Invalid JSON in {fields_path}: {e}", file=sys.stderr)
sys.exit(2)
output_format = "json" if args.json_output else "script"
result, errors = build_form_script(
url=args.url,
fields=fields,
output_format=output_format,
)
if errors:
print("Validation errors:", file=sys.stderr)
for err in errors:
print(f" - {err}", file=sys.stderr)
sys.exit(2)
if args.json_output:
output_text = json.dumps(result, indent=2)
else:
output_text = result
if args.output:
output_path = os.path.abspath(args.output)
with open(output_path, "w") as f:
f.write(output_text)
if not args.json_output:
os.chmod(output_path, 0o755)
print(f"Written to {output_path}", file=sys.stderr)
sys.exit(0)
else:
print(output_text)
sys.exit(0)
if __name__ == "__main__":
main()
FILE:scripts/scraping_toolkit.py
#!/usr/bin/env python3
"""
Scraping Toolkit - Generates Playwright scraping script skeletons.
Takes a URL pattern and CSS selectors as input and produces a ready-to-run
Playwright scraping script with pagination support, error handling, and
anti-detection patterns baked in.
No external dependencies - uses only Python standard library.
"""
import argparse
import json
import os
import sys
import textwrap
from datetime import datetime
def build_scraping_script(url, selectors, paginate=False, output_format="script"):
"""Build a Playwright scraping script from the given parameters."""
selector_list = [s.strip() for s in selectors.split(",") if s.strip()]
if not selector_list:
return None, "No valid selectors provided."
field_names = []
for sel in selector_list:
# Derive field name from selector: .product-title -> product_title
name = sel.strip("#.[]()>:+~ ")
name = name.replace("-", "_").replace(" ", "_").replace(".", "_")
# Remove non-alphanumeric
name = "".join(c if c.isalnum() or c == "_" else "" for c in name)
if not name:
name = f"field_{len(field_names)}"
field_names.append(name)
field_map = dict(zip(field_names, selector_list))
if output_format == "json":
config = {
"url": url,
"selectors": field_map,
"pagination": {
"enabled": paginate,
"next_selector": "a:has-text('Next'), button:has-text('Next')",
"max_pages": 50,
},
"anti_detection": {
"random_delay_ms": [800, 2500],
"user_agent_rotation": True,
"viewport": {"width": 1920, "height": 1080},
},
"output": {
"format": "jsonl",
"deduplicate_by": field_names[0] if field_names else None,
},
"generated_at": datetime.now().isoformat(),
}
return config, None
# Build Python script
fields_dict_str = "{\n"
for name, sel in field_map.items():
fields_dict_str += f' "{name}": "{sel}",\n'
fields_dict_str += " }"
pagination_block = ""
if paginate:
pagination_block = textwrap.dedent("""\
# --- Pagination ---
async def scrape_all_pages(page, container, fields, next_sel, max_pages=50):
all_items = []
for page_num in range(max_pages):
print(f"Scraping page {page_num + 1}...")
items = await extract_items(page, container, fields)
all_items.extend(items)
next_btn = page.locator(next_sel)
if await next_btn.count() == 0:
break
try:
is_disabled = await next_btn.is_disabled()
except Exception:
is_disabled = True
if is_disabled:
break
await next_btn.click()
await page.wait_for_load_state("networkidle")
await asyncio.sleep(random.uniform(0.8, 2.5))
return all_items
""")
main_call = "scrape_all_pages(page, CONTAINER, FIELDS, NEXT_SELECTOR)" if paginate else "extract_items(page, CONTAINER, FIELDS)"
script = textwrap.dedent(f'''\
#!/usr/bin/env python3
"""
Auto-generated Playwright scraping script.
Target: {url}
Generated: {datetime.now().isoformat()}
Requirements:
pip install playwright
playwright install chromium
"""
import asyncio
import json
import random
from playwright.async_api import async_playwright
# --- Configuration ---
URL = "{url}"
CONTAINER = "body" # Adjust to the repeating item container selector
FIELDS = {fields_dict_str}
NEXT_SELECTOR = "a:has-text('Next'), button:has-text('Next')"
USER_AGENTS = [
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
]
async def extract_items(page, container_selector, field_map):
"""Extract structured data from repeating elements."""
items = []
cards = await page.query_selector_all(container_selector)
for card in cards:
item = {{}}
for name, selector in field_map.items():
el = await card.query_selector(selector)
if el:
item[name] = (await el.text_content() or "").strip()
else:
item[name] = None
items.append(item)
return items
{pagination_block}
async def main():
async with async_playwright() as p:
browser = await p.chromium.launch(headless=True)
context = await browser.new_context(
viewport={{"width": 1920, "height": 1080}},
user_agent=random.choice(USER_AGENTS),
)
page = await context.new_page()
# Remove WebDriver flag
await page.add_init_script(
"Object.defineProperty(navigator, \'webdriver\', {{get: () => undefined}});"
)
print(f"Navigating to {{URL}}...")
await page.goto(URL, wait_until="networkidle")
data = await {main_call}
print(json.dumps(data, indent=2, ensure_ascii=False))
await browser.close()
if __name__ == "__main__":
asyncio.run(main())
''')
return script, None
def main():
parser = argparse.ArgumentParser(
description="Generate Playwright scraping script skeletons from URL and selectors.",
epilog=(
"Examples:\n"
" %(prog)s --url https://example.com/products --selectors '.title,.price,.rating'\n"
" %(prog)s --url https://example.com/search --selectors '.name,.desc' --paginate\n"
" %(prog)s --url https://example.com --selectors '.item' --json\n"
" %(prog)s --url https://example.com --selectors '.item' --output scraper.py\n"
),
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument(
"--url",
required=True,
help="Target URL to scrape",
)
parser.add_argument(
"--selectors",
required=True,
help="Comma-separated CSS selectors for data fields (e.g. '.title,.price,.rating')",
)
parser.add_argument(
"--paginate",
action="store_true",
default=False,
help="Include pagination handling in generated script",
)
parser.add_argument(
"--output",
help="Output file path (default: stdout)",
)
parser.add_argument(
"--json",
action="store_true",
dest="json_output",
default=False,
help="Output JSON configuration instead of Python script",
)
args = parser.parse_args()
output_format = "json" if args.json_output else "script"
result, error = build_scraping_script(
url=args.url,
selectors=args.selectors,
paginate=args.paginate,
output_format=output_format,
)
if error:
print(f"Error: {error}", file=sys.stderr)
sys.exit(2)
if args.json_output:
output_text = json.dumps(result, indent=2)
else:
output_text = result
if args.output:
output_path = os.path.abspath(args.output)
with open(output_path, "w") as f:
f.write(output_text)
if not args.json_output:
os.chmod(output_path, 0o755)
print(f"Written to {output_path}", file=sys.stderr)
sys.exit(0)
else:
print(output_text)
sys.exit(0)
if __name__ == "__main__":
main()
Chạy kiểm thử trên BrowserStack: kiểm thử đa trình duyệt, đám mây và tương thích trình duyệt.
---
name: "browserstack"
description: >-
Run tests on BrowserStack. Use when user mentions "browserstack",
"cross-browser", "cloud testing", "browser matrix", "test on safari",
"test on firefox", or "browser compatibility".
---
# BrowserStack Integration
Run Playwright tests on BrowserStack's cloud grid for cross-browser and cross-device testing.
## Prerequisites
Environment variables must be set:
- `BROWSERSTACK_USERNAME` — your BrowserStack username
- `BROWSERSTACK_ACCESS_KEY` — your access key
If not set, inform the user how to get them from [browserstack.com/accounts/settings](https://www.browserstack.com/accounts/settings) and stop.
## Capabilities
### 1. Configure for BrowserStack
```
/pw:browserstack setup
```
Steps:
1. Check current `playwright.config.ts`
2. Add BrowserStack connect options:
```typescript
// Add to playwright.config.ts
import { defineConfig } from '@playwright/test';
const isBS = !!process.env.BROWSERSTACK_USERNAME;
export default defineConfig({
// ... existing config
projects: isBS ? [
{
name: "chromelatestwindows-11",
use: {
connectOptions: {
wsEndpoint: `wss://cdp.browserstack.com/playwright?caps='chrome',
'browser_version': 'latest',
'os': 'Windows',
'os_version': '11',
'browserstack.username': process.env.BROWSERSTACK_USERNAME,
'browserstack.accessKey': process.env.BROWSERSTACK_ACCESS_KEY,))}`,
},
},
},
{
name: "firefoxlatestwindows-11",
use: {
connectOptions: {
wsEndpoint: `wss://cdp.browserstack.com/playwright?caps='playwright-firefox',
'browser_version': 'latest',
'os': 'Windows',
'os_version': '11',
'browserstack.username': process.env.BROWSERSTACK_USERNAME,
'browserstack.accessKey': process.env.BROWSERSTACK_ACCESS_KEY,))}`,
},
},
},
{
name: "webkitlatestos-x-ventura",
use: {
connectOptions: {
wsEndpoint: `wss://cdp.browserstack.com/playwright?caps='playwright-webkit',
'browser_version': 'latest',
'os': 'OS X',
'os_version': 'Ventura',
'browserstack.username': process.env.BROWSERSTACK_USERNAME,
'browserstack.accessKey': process.env.BROWSERSTACK_ACCESS_KEY,))}`,
},
},
},
] : [
// ... local projects fallback
],
});
```
3. Add npm script: `"test:e2e:cloud": "npx playwright test --project='chrome@*' --project='firefox@*' --project='webkit@*'"`
### 2. Run Tests on BrowserStack
```
/pw:browserstack run
```
Steps:
1. Verify credentials are set
2. Run tests with BrowserStack projects:
```bash
BROWSERSTACK_USERNAME=$BROWSERSTACK_USERNAME \
BROWSERSTACK_ACCESS_KEY=$BROWSERSTACK_ACCESS_KEY \
npx playwright test --project='chrome@*' --project='firefox@*'
```
3. Monitor execution
4. Report results per browser
### 3. Get Build Results
```
/pw:browserstack results
```
Steps:
1. Call `browserstack_get_builds` MCP tool
2. Get latest build's sessions
3. For each session:
- Status (pass/fail)
- Browser and OS
- Duration
- Video URL
- Log URLs
4. Format as summary table
### 4. Check Available Browsers
```
/pw:browserstack browsers
```
Steps:
1. Call `browserstack_get_browsers` MCP tool
2. Filter for Playwright-compatible browsers
3. Display available browser/OS combinations
### 5. Local Testing
```
/pw:browserstack local
```
For testing localhost or staging behind firewall:
1. Install BrowserStack Local: `npm install -D browserstack-local`
2. Add local tunnel to config
3. Provide setup instructions
## MCP Tools Used
| Tool | When |
|---|---|
| `browserstack_get_plan` | Check account limits |
| `browserstack_get_browsers` | List available browsers |
| `browserstack_get_builds` | List recent builds |
| `browserstack_get_sessions` | Get sessions in a build |
| `browserstack_get_session` | Get session details (video, logs) |
| `browserstack_update_session` | Mark pass/fail |
| `browserstack_get_logs` | Get text/network logs |
## Output
- Cross-browser test results table
- Per-browser pass/fail status
- Links to BrowserStack dashboard for video/screenshots
- Any browser-specific failures highlighted
Đội điều hành ảo gồm 8 agent C-suite và 17 lệnh /cs:* cho office hours, họp HĐQT, sprint chiến lược và định tuyến.
---
name: "c-level-agents"
description: "Founder-mode executive team. 8 cs-* C-suite agents (CFO, CMO, CRO, CPO, COO, CHRO, CISO, Chief of Staff) and 17 /cs:* slash commands for forcing-question office hours, multi-role boardroom deliberation, strategic sprint pipeline, and meta routing. Use when the founder needs a virtual executive team, when invoking /cs:* commands, or when orchestrating multi-role decisions."
license: MIT
metadata:
version: 1.0.0
author: Alireza Rezvani
category: c-level
domain: executive-orchestration
updated: 2026-05-12
agents: cs-cfo-advisor, cs-cmo-advisor, cs-cro-advisor, cs-cpo-advisor, cs-coo-advisor, cs-chro-advisor, cs-ciso-advisor, cs-chief-of-staff
commands: cs-office-hours, cs-cfo-review, cs-cmo-review, cs-cpo-review, cs-cro-review, cs-cto-review, cs-ciso-review, cs-gc-review, cs-brief, cs-boardroom, cs-decide, cs-execute, cs-post-mortem, cs-founder-mode, cs-onboard, cs-cross-eval, cs-freeze
---
# c-level-agents — Founder-Mode Executive Team
A virtual C-suite delivered through slash commands and persona agents.
## Keywords
founder mode, virtual c-suite, executive team, boardroom, office hours, cfo review, cmo review, strategic sprint, decision logging, cross-model consensus, persona agents, chief of staff, forcing questions
## What This Plugin Provides
### 8 cs-* Agents (in `agents/`)
Each agent wraps an existing c-level skill and adds:
- A distinct cognitive voice (numerate skeptic, narrative-first, etc.)
- Forcing questions specific to the role
- Workflow orchestration tied to skill Python tools
- Output template: Bottom Line → What → Why → How to Act → Your Decision
See `../references/persona-voices.md` for voice specs.
### 17 /cs:* Slash Commands (in `skills/`)
**Forcing-question office hours (8):**
- `/cs:office-hours` — YC-style 6-question intake
- `/cs:cfo-review` — unit economics, runway, dilution
- `/cs:cmo-review` — ICP, CAC payback, positioning
- `/cs:cpo-review` — RICE, JTBD, North Star, PMF
- `/cs:cro-review` — pipeline coverage, win rate, NRR
- `/cs:cto-review` — architecture risk, scaling cliff
- `/cs:ciso-review` — threat model, blast radius, compliance
- `/cs:gc-review` — contracts, IP, regulatory, term sheets
**Strategic sprint pipeline (5):**
- `/cs:brief` → `/cs:boardroom` → `/cs:decide` → `/cs:execute` → `/cs:post-mortem`
**Meta + safety (4):**
- `/cs:founder-mode` — auto-routes to the right C-role
- `/cs:onboard` — founder interview → `company-context.md`
- `/cs:cross-eval` — multi-model consensus
- `/cs:freeze` — cooldown lock on a decision
## Quick Start
```
/cs:onboard # populate company context first
/cs:office-hours "should we hire a VP Sales?"
/cs:founder-mode "runway pressure" # auto-routes to CFO
/cs:boardroom briefs/pricing-v3.md # full panel
```
## Architecture
```
User question
│
├─ Single-role? → cs-{role}-advisor agent
│ ↓
│ /cs:{role}-review command (forcing Qs)
│ ↓
│ Skill tools + references
│ ↓
│ Bottom Line + Memo
│
└─ Multi-role? → /cs:boardroom
↓
6-phase deliberation (Phase 2 isolation)
↓
/cs:decide → decision-logger (two-layer memory)
↓
/cs:execute → 90-day plan
```
## Integration Points
- **Existing 28 c-level skills** — wrapped, not replaced
- **decision-logger** — every `/cs:decide` writes here
- **chief-of-staff** — routing layer the agent orchestrates
- **board-meeting** — protocol the `/cs:boardroom` command runs
- **llm-wiki** — optional persistent memory bridge (see `../references/llm-wiki-bridge.md`)
- **executive-mentor** — adversarial `/em:*` commands stack cleanly on top
## Design Principles
1. **Voice is bookended, analysis is neutral.**
2. **Artifacts over chat.** Every command produces a Markdown artifact the next command consumes.
3. **Phase 2 isolation in boardroom.** Independent thinking before cross-examination.
4. **Graceful degradation.** `/cs:cross-eval` falls back to Claude-only.
5. **No paid dependencies.** All Python tools are stdlib-only.
## References
- [persona-voices.md](../../references/persona-voices.md)
- [llm-wiki-bridge.md](../../references/llm-wiki-bridge.md)
- [Parent c-level CLAUDE.md](../../../CLAUDE.md)
- [Existing executive-mentor sibling](../../../executive-mentor/)
---
**Version:** 1.0.0
**Last Updated:** 2026-05-12
**Status:** Production Ready
Phân tích hiệu quả chiến dịch với attribution đa điểm chạm, phễu chuyển đổi và tính ROI, ROAS, CPA.
---
name: "campaign-analytics"
description: Analyzes campaign performance with multi-touch attribution, funnel conversion analysis, and ROI calculation for marketing optimization. Use when analyzing marketing campaigns, ad performance, attribution models, conversion rates, or calculating marketing ROI, ROAS, CPA, and campaign metrics across channels.
license: MIT
metadata:
version: 1.0.0
author: Alireza Rezvani
category: marketing
domain: campaign-analytics
updated: 2026-02-06
python-tools: attribution_analyzer.py, funnel_analyzer.py, campaign_roi_calculator.py
tech-stack: marketing-analytics, attribution-modeling
---
# Campaign Analytics
Production-grade campaign performance analysis with multi-touch attribution modeling, funnel conversion analysis, and ROI calculation. Three Python CLI tools provide deterministic, repeatable analytics using standard library only -- no external dependencies, no API calls, no ML models.
---
## Input Requirements
All scripts accept a JSON file as positional input argument. See `assets/sample_campaign_data.json` for complete examples.
### Attribution Analyzer
```json
{
"journeys": [
{
"journey_id": "j1",
"touchpoints": [
{"channel": "organic_search", "timestamp": "2025-10-01T10:00:00", "interaction": "click"},
{"channel": "email", "timestamp": "2025-10-05T14:30:00", "interaction": "open"},
{"channel": "paid_search", "timestamp": "2025-10-08T09:15:00", "interaction": "click"}
],
"converted": true,
"revenue": 500.00
}
]
}
```
### Funnel Analyzer
```json
{
"funnel": {
"stages": ["Awareness", "Interest", "Consideration", "Intent", "Purchase"],
"counts": [10000, 5200, 2800, 1400, 420]
}
}
```
### Campaign ROI Calculator
```json
{
"campaigns": [
{
"name": "Spring Email Campaign",
"channel": "email",
"spend": 5000.00,
"revenue": 25000.00,
"impressions": 50000,
"clicks": 2500,
"leads": 300,
"customers": 45
}
]
}
```
### Input Validation
Before running scripts, verify your JSON is valid and matches the expected schema. Common errors:
- **Missing required keys** (e.g., `journeys`, `funnel.stages`, `campaigns`) → script exits with a descriptive `KeyError`
- **Mismatched array lengths** in funnel data (`stages` and `counts` must be the same length) → raises `ValueError`
- **Non-numeric monetary values** in ROI data → raises `TypeError`
Use `python -m json.tool your_file.json` to validate JSON syntax before passing it to any script.
---
## Output Formats
All scripts support two output formats via the `--format` flag:
- `--format text` (default): Human-readable tables and summaries for review
- `--format json`: Machine-readable JSON for integrations and pipelines
---
## Typical Analysis Workflow
For a complete campaign review, run the three scripts in sequence:
```bash
# Step 1 — Attribution: understand which channels drive conversions
python scripts/attribution_analyzer.py campaign_data.json --model time-decay
# Step 2 — Funnel: identify where prospects drop off on the path to conversion
python scripts/funnel_analyzer.py funnel_data.json
# Step 3 — ROI: calculate profitability and benchmark against industry standards
python scripts/campaign_roi_calculator.py campaign_data.json
```
Use attribution results to identify top-performing channels, then focus funnel analysis on those channels' segments, and finally validate ROI metrics to prioritize budget reallocation.
---
## How to Use
### Attribution Analysis
```bash
# Run all 5 attribution models
python scripts/attribution_analyzer.py campaign_data.json
# Run a specific model
python scripts/attribution_analyzer.py campaign_data.json --model time-decay
# JSON output for pipeline integration
python scripts/attribution_analyzer.py campaign_data.json --format json
# Custom time-decay half-life (default: 7 days)
python scripts/attribution_analyzer.py campaign_data.json --model time-decay --half-life 14
```
### Funnel Analysis
```bash
# Basic funnel analysis
python scripts/funnel_analyzer.py funnel_data.json
# JSON output
python scripts/funnel_analyzer.py funnel_data.json --format json
```
### Campaign ROI Calculation
```bash
# Calculate ROI metrics for all campaigns
python scripts/campaign_roi_calculator.py campaign_data.json
# JSON output
python scripts/campaign_roi_calculator.py campaign_data.json --format json
```
---
## Scripts
### 1. attribution_analyzer.py
Implements five industry-standard attribution models to allocate conversion credit across marketing channels:
| Model | Description | Best For |
|-------|-------------|----------|
| First-Touch | 100% credit to first interaction | Brand awareness campaigns |
| Last-Touch | 100% credit to last interaction | Direct response campaigns |
| Linear | Equal credit to all touchpoints | Balanced multi-channel evaluation |
| Time-Decay | More credit to recent touchpoints | Short sales cycles |
| Position-Based | 40/20/40 split (first/middle/last) | Full-funnel marketing |
### 2. funnel_analyzer.py
Analyzes conversion funnels to identify bottlenecks and optimization opportunities:
- Stage-to-stage conversion rates and drop-off percentages
- Automatic bottleneck identification (largest absolute and relative drops)
- Overall funnel conversion rate
- Segment comparison when multiple segments are provided
### 3. campaign_roi_calculator.py
Calculates comprehensive ROI metrics with industry benchmarking:
- **ROI**: Return on investment percentage
- **ROAS**: Return on ad spend ratio
- **CPA**: Cost per acquisition
- **CPL**: Cost per lead
- **CAC**: Customer acquisition cost
- **CTR**: Click-through rate
- **CVR**: Conversion rate (leads to customers)
- Flags underperforming campaigns against industry benchmarks
---
## Reference Guides
| Guide | Location | Purpose |
|-------|----------|---------|
| Attribution Models Guide | `references/attribution-models-guide.md` | Deep dive into 5 models with formulas, pros/cons, selection criteria |
| Campaign Metrics Benchmarks | `references/campaign-metrics-benchmarks.md` | Industry benchmarks by channel and vertical for CTR, CPC, CPM, CPA, ROAS |
| Funnel Optimization Framework | `references/funnel-optimization-framework.md` | Stage-by-stage optimization strategies, common bottlenecks, best practices |
---
## Best Practices
1. **Use multiple attribution models** -- Compare at least 3 models to triangulate channel value; no single model tells the full story.
2. **Set appropriate lookback windows** -- Match your time-decay half-life to your average sales cycle length.
3. **Segment your funnels** -- Compare segments (channel, cohort, geography) to identify performance drivers.
4. **Benchmark against your own history first** -- Industry benchmarks provide context, but historical data is the most relevant comparison.
5. **Run ROI analysis at regular intervals** -- Weekly for active campaigns, monthly for strategic review.
6. **Include all costs** -- Factor in creative, tooling, and labor costs alongside media spend for accurate ROI.
7. **Document A/B tests rigorously** -- Use the provided template to ensure statistical validity and clear decision criteria.
---
## Limitations
- **No statistical significance testing** -- Scripts provide descriptive metrics only; p-value calculations require external tools.
- **Standard library only** -- No advanced statistical libraries. Suitable for most campaign sizes but not optimized for datasets exceeding 100K journeys.
- **Offline analysis** -- Scripts analyze static JSON snapshots; no real-time data connections or API integrations.
- **Single-currency** -- All monetary values assumed to be in the same currency; no currency conversion support.
- **Simplified time-decay** -- Exponential decay based on configurable half-life; does not account for weekday/weekend or seasonal patterns.
- **No cross-device tracking** -- Attribution operates on provided journey data as-is; cross-device identity resolution must be handled upstream.
## Related Skills
- **analytics-tracking**: For setting up tracking. NOT for analyzing data (that's this skill).
- **ab-test-setup**: For designing experiments to test what analytics reveals.
- **marketing-ops**: For routing insights to the right execution skill.
- **paid-ads**: For optimizing ad spend based on analytics findings.
FILE:assets/ab_test_template.md
# A/B Test Analysis
**Test Name:** [Descriptive test name]
**Test ID:** [Internal tracking ID]
**Date:** [Start Date] - [End Date]
**Status:** [Planning / Running / Complete / Inconclusive]
---
## Hypothesis
**If** [we change X],
**then** [Y will happen],
**because** [rationale based on data or insight].
---
## Test Design
| Parameter | Detail |
|-----------|--------|
| **Variable Tested** | [What is being changed] |
| **Control (A)** | [Description of control variant] |
| **Variant (B)** | [Description of test variant] |
| **Primary Metric** | [The main metric being measured] |
| **Secondary Metrics** | [Additional metrics to monitor] |
| **Traffic Split** | [50/50, 70/30, etc.] |
| **Minimum Sample Size** | [Required sample per variant for statistical significance] |
| **Minimum Detectable Effect** | [Smallest meaningful difference, e.g., 5% lift] |
| **Confidence Level** | [95% or 99%] |
| **Expected Duration** | [X days/weeks based on traffic and sample size] |
---
## Targeting
| Criterion | Value |
|-----------|-------|
| **Audience** | [Who sees the test] |
| **Channel** | [Where the test runs] |
| **Device** | [All / Desktop / Mobile] |
| **Geography** | [Regions included] |
| **Exclusions** | [Who is excluded and why] |
---
## Results
### Primary Metric: [Metric Name]
| Variant | Sample Size | Conversions | Rate | Lift vs Control |
|---------|------------|-------------|------|----------------|
| Control (A) | | | % | - |
| Variant (B) | | | % | % |
**Statistical Significance:** [Yes/No] at [X]% confidence
**P-value:** [X.XXX]
### Secondary Metrics
| Metric | Control (A) | Variant (B) | Lift | Significant? |
|--------|------------|-------------|------|-------------|
| [Metric 1] | | | % | [Yes/No] |
| [Metric 2] | | | % | [Yes/No] |
| [Metric 3] | | | % | [Yes/No] |
---
## Segment Analysis
| Segment | Control Rate | Variant Rate | Lift | Notes |
|---------|-------------|-------------|------|-------|
| Desktop | % | % | % | |
| Mobile | % | % | % | |
| New Visitors | % | % | % | |
| Returning Visitors | % | % | % | |
| [Custom Segment] | % | % | % | |
---
## Revenue Impact Estimate
| Metric | Value |
|--------|-------|
| **Projected Annual Lift** | [X]% |
| **Projected Additional Revenue** | $[X] |
| **Projected Additional Conversions** | [X] |
| **Confidence in Estimate** | [High/Medium/Low] |
---
## Decision
**Winner:** [Control / Variant / Inconclusive]
**Rationale:** [Why this decision was made, citing specific metrics and statistical significance]
**Implementation Plan:**
- [ ] [Step 1: e.g., Roll out variant to 100% of traffic]
- [ ] [Step 2: e.g., Update creative assets across campaigns]
- [ ] [Step 3: e.g., Monitor for X days post-implementation]
- [ ] [Step 4: e.g., Document learnings in knowledge base]
---
## Learnings
**What we learned:**
1. [Key learning 1]
2. [Key learning 2]
3. [Key learning 3]
**Follow-up tests to consider:**
1. [Next test idea based on results]
2. [Next test idea based on results]
---
## Quality Checks
- [ ] Sample size reached minimum threshold
- [ ] Test ran for at least 1 full business cycle (7 days minimum)
- [ ] No external factors (holidays, outages, promotions) affected results
- [ ] Segments were balanced between variants
- [ ] No sample ratio mismatch (SRM) detected
- [ ] Results reviewed by at least 2 team members
---
*Template from campaign-analytics skill. Statistical significance calculations require external tools (e.g., online calculators or scipy).*
FILE:assets/campaign_report_template.md
# Campaign Performance Report
**Report Period:** [Start Date] - [End Date]
**Prepared By:** [Name]
**Date:** [Report Date]
---
## Executive Summary
[2-3 sentence summary of overall campaign performance, key wins, and areas of concern.]
---
## Portfolio Overview
| Metric | This Period | Previous Period | Change |
|--------|-----------|----------------|--------|
| Total Spend | $ | $ | % |
| Total Revenue | $ | $ | % |
| Total Profit | $ | $ | % |
| Portfolio ROI | % | % | pp |
| Portfolio ROAS | x | x | % |
| Total Leads | | | % |
| Total Customers | | | % |
| Blended CPA | $ | $ | % |
| Blended CPL | $ | $ | % |
---
## Channel Performance
| Channel | Spend | Revenue | ROI | ROAS | CPA | Leads | Customers |
|---------|-------|---------|-----|------|-----|-------|-----------|
| Email | $ | $ | % | x | $ | | |
| Paid Search | $ | $ | % | x | $ | | |
| Paid Social | $ | $ | % | x | $ | | |
| Display | $ | $ | % | x | $ | | |
| Organic | $ | $ | % | x | $ | | |
| **Total** | **$** | **$** | **%** | **x** | **$** | | |
---
## Top Performing Campaigns
### 1. [Campaign Name]
- **Channel:** [Channel]
- **Spend:** $[Amount] | **Revenue:** $[Amount] | **ROI:** [X]%
- **Key Success Factor:** [What made this campaign successful]
### 2. [Campaign Name]
- **Channel:** [Channel]
- **Spend:** $[Amount] | **Revenue:** $[Amount] | **ROI:** [X]%
- **Key Success Factor:** [What made this campaign successful]
### 3. [Campaign Name]
- **Channel:** [Channel]
- **Spend:** $[Amount] | **Revenue:** $[Amount] | **ROI:** [X]%
- **Key Success Factor:** [What made this campaign successful]
---
## Underperforming Campaigns
### [Campaign Name]
- **Channel:** [Channel]
- **Issue:** [Description of underperformance]
- **Benchmark Comparison:** [How it compares to benchmarks]
- **Recommended Action:** [Specific action to take]
### [Campaign Name]
- **Channel:** [Channel]
- **Issue:** [Description of underperformance]
- **Benchmark Comparison:** [How it compares to benchmarks]
- **Recommended Action:** [Specific action to take]
---
## Attribution Analysis
| Channel | First-Touch | Last-Touch | Linear | Time-Decay | Position-Based |
|---------|------------|------------|--------|------------|----------------|
| [Channel 1] | $[X] | $[X] | $[X] | $[X] | $[X] |
| [Channel 2] | $[X] | $[X] | $[X] | $[X] | $[X] |
| [Channel 3] | $[X] | $[X] | $[X] | $[X] | $[X] |
**Key Insight:** [What does the attribution analysis tell us about channel value that single-model analysis would miss?]
---
## Funnel Analysis
| Stage | Count | Conversion Rate | Drop-off | vs. Previous Period |
|-------|-------|----------------|----------|-------------------|
| Awareness | | - | - | % |
| Interest | | % | % | pp |
| Consideration | | % | % | pp |
| Intent | | % | % | pp |
| Purchase | | % | % | pp |
**Overall Funnel Conversion:** [X]%
**Primary Bottleneck:** [Stage transition with largest drop-off]
**Recommended Focus:** [What to optimize next]
---
## Budget Allocation Recommendations
Based on this period's performance data:
| Channel | Current Allocation | Recommended Allocation | Rationale |
|---------|-------------------|----------------------|-----------|
| [Channel] | [X]% ($[X]) | [X]% ($[X]) | [Reason] |
| [Channel] | [X]% ($[X]) | [X]% ($[X]) | [Reason] |
| [Channel] | [X]% ($[X]) | [X]% ($[X]) | [Reason] |
---
## Action Items
| Priority | Action | Owner | Deadline | Expected Impact |
|----------|--------|-------|----------|----------------|
| High | [Action] | [Name] | [Date] | [Impact] |
| High | [Action] | [Name] | [Date] | [Impact] |
| Medium | [Action] | [Name] | [Date] | [Impact] |
| Low | [Action] | [Name] | [Date] | [Impact] |
---
## Next Period Goals
| Metric | Current | Target | Strategy |
|--------|---------|--------|----------|
| Portfolio ROI | [X]% | [X]% | [How] |
| ROAS | [X]x | [X]x | [How] |
| CPA | $[X] | $[X] | [How] |
| Lead Volume | [X] | [X] | [How] |
---
*Report generated using campaign-analytics toolkit. Data source: [Source system/platform].*
FILE:assets/channel_comparison_template.md
# Channel Performance Comparison
**Period:** [Start Date] - [End Date]
**Compared Against:** [Previous period / Industry benchmarks / Both]
**Prepared By:** [Name]
---
## Summary
[1-2 sentence overview: which channels are performing best, which need attention, and the overall channel mix health.]
---
## Channel Scorecard
| Channel | Spend | Revenue | Profit | ROI | ROAS | CTR | CPA | CPL | Grade |
|---------|-------|---------|--------|-----|------|-----|-----|-----|-------|
| Email | $ | $ | $ | % | x | % | $ | $ | [A-F] |
| Paid Search | $ | $ | $ | % | x | % | $ | $ | [A-F] |
| Paid Social | $ | $ | $ | % | x | % | $ | $ | [A-F] |
| Display | $ | $ | $ | % | x | % | $ | $ | [A-F] |
| Organic Search | $ | $ | $ | % | x | % | $ | $ | [A-F] |
| Organic Social | $ | $ | $ | % | x | % | $ | $ | [A-F] |
| Referral | $ | $ | $ | % | x | % | $ | $ | [A-F] |
| Direct | $ | $ | $ | % | x | % | $ | $ | [A-F] |
| **Total** | **$** | **$** | **$** | **%** | **x** | **%** | **$** | **$** | |
**Grading Scale:**
- A: Exceeds all benchmarks
- B: Meets or exceeds target benchmarks
- C: Between low and target benchmarks
- D: Below low benchmark on 1+ key metrics
- F: Underperforming on multiple metrics or unprofitable
---
## Channel Deep Dives
### [Channel Name]
**Performance Summary:** [1-2 sentences]
| Metric | Actual | Target | Benchmark | vs. Target | vs. Benchmark |
|--------|--------|--------|-----------|-----------|---------------|
| Spend | $ | $ | - | % | - |
| Revenue | $ | $ | - | % | - |
| ROI | % | % | % | pp | pp |
| ROAS | x | x | x | % | % |
| CTR | % | % | % | pp | pp |
| CPA | $ | $ | $ | % | % |
| CPL | $ | $ | $ | % | % |
| CPC | $ | $ | $ | % | % |
**Trend (Last 3 Periods):**
| Period | Spend | Revenue | ROI | ROAS | Key Event |
|--------|-------|---------|-----|------|-----------|
| [Period 1] | $ | $ | % | x | [Note] |
| [Period 2] | $ | $ | % | x | [Note] |
| [Current] | $ | $ | % | x | [Note] |
**Assessment:** [Improving / Stable / Declining]
**Action Items:**
1. [Specific action for this channel]
2. [Specific action for this channel]
---
[Repeat deep dive section for each channel]
---
## Attribution View
How each channel is valued under different attribution models:
| Channel | First-Touch | Last-Touch | Linear | Time-Decay | Position-Based |
|---------|------------|------------|--------|------------|----------------|
| [Channel 1] | $ (X%) | $ (X%) | $ (X%) | $ (X%) | $ (X%) |
| [Channel 2] | $ (X%) | $ (X%) | $ (X%) | $ (X%) | $ (X%) |
| [Channel 3] | $ (X%) | $ (X%) | $ (X%) | $ (X%) | $ (X%) |
**Insight:** [Which channels are over/undervalued by single-touch models?]
---
## Funnel Performance by Channel
| Stage | [Ch 1] | [Ch 2] | [Ch 3] | [Ch 4] | Overall |
|-------|--------|--------|--------|--------|---------|
| Awareness | [Count] | [Count] | [Count] | [Count] | [Count] |
| Interest | [Rate]% | [Rate]% | [Rate]% | [Rate]% | [Rate]% |
| Consideration | [Rate]% | [Rate]% | [Rate]% | [Rate]% | [Rate]% |
| Intent | [Rate]% | [Rate]% | [Rate]% | [Rate]% | [Rate]% |
| Purchase | [Rate]% | [Rate]% | [Rate]% | [Rate]% | [Rate]% |
| **Overall** | **[Rate]%** | **[Rate]%** | **[Rate]%** | **[Rate]%** | **[Rate]%** |
**Best Funnel:** [Channel with highest overall conversion rate]
**Biggest Bottleneck:** [Channel + stage transition with worst drop-off]
---
## Budget Allocation Analysis
### Current vs. Optimal Allocation
| Channel | Current % | Current $ | Recommended % | Recommended $ | Rationale |
|---------|----------|-----------|--------------|---------------|-----------|
| [Channel] | % | $ | % | $ | [Why] |
| [Channel] | % | $ | % | $ | [Why] |
| [Channel] | % | $ | % | $ | [Why] |
| [Channel] | % | $ | % | $ | [Why] |
| **Total** | **100%** | **$** | **100%** | **$** | |
### Reallocation Impact Estimate
| Scenario | Projected Revenue | Projected ROI | Change vs Current |
|----------|------------------|---------------|-------------------|
| Current allocation | $ | % | - |
| Recommended allocation | $ | % | +% |
| Aggressive growth | $ | % | +% |
| Cost optimization | $ | % | +% |
---
## Competitive Context
| Metric | Our Performance | Industry Average | Gap |
|--------|----------------|-----------------|-----|
| Channel Mix Diversity | [X channels active] | [X channels] | |
| Overall ROAS | [X]x | [X]x | |
| Paid vs Organic Split | [X/X]% | [X/X]% | |
| Digital vs Traditional | [X/X]% | [X/X]% | |
---
## Recommendations
### Immediate Actions (This Week)
1. **[Action]** -- [Expected impact], [Owner]
2. **[Action]** -- [Expected impact], [Owner]
### Short-Term (This Month)
1. **[Action]** -- [Expected impact], [Owner]
2. **[Action]** -- [Expected impact], [Owner]
### Strategic (This Quarter)
1. **[Action]** -- [Expected impact], [Owner]
2. **[Action]** -- [Expected impact], [Owner]
---
*Template from campaign-analytics skill. Populate with data from attribution_analyzer.py, funnel_analyzer.py, and campaign_roi_calculator.py.*
FILE:assets/expected_output.json
{
"_description": "Expected output from running the 3 scripts against sample_campaign_data.json with --format json",
"attribution_analyzer": {
"_command": "python scripts/attribution_analyzer.py assets/sample_campaign_data.json --format json",
"summary": {
"total_journeys": 8,
"converted_journeys": 6,
"conversion_rate": 75.0,
"total_revenue": 3700.0,
"channels_observed": [
"direct", "display", "email", "organic_search",
"organic_social", "paid_search", "paid_social", "referral"
]
},
"models": {
"first-touch": {
"organic_search": 700.0,
"paid_social": 1200.0,
"display": 350.0,
"organic_social": 800.0,
"referral": 650.0
},
"last-touch": {
"paid_search": 1500.0,
"direct": 2000.0,
"organic_search": 200.0
},
"linear": {
"organic_search": 666.67,
"email": 1003.33,
"paid_search": 718.33,
"paid_social": 300.0,
"direct": 460.0,
"display": 175.0,
"organic_social": 160.0,
"referral": 216.67
},
"time-decay": {
"organic_search": 582.38,
"email": 1053.68,
"paid_search": 881.03,
"paid_social": 178.4,
"direct": 638.82,
"display": 140.62,
"organic_social": 78.48,
"referral": 146.59
},
"position-based": {
"organic_search": 520.0,
"paid_search": 688.33,
"email": 456.67,
"paid_social": 480.0,
"direct": 800.0,
"display": 175.0,
"organic_social": 320.0,
"referral": 260.0
}
}
},
"funnel_analyzer": {
"_command": "python scripts/funnel_analyzer.py assets/sample_campaign_data.json --format json",
"_note": "Uses segment comparison mode since 'segments' key is present in the data",
"rankings": [
{"rank": 1, "segment": "organic", "overall_conversion_rate": 5.6, "total_entries": 5000, "total_conversions": 280},
{"rank": 2, "segment": "paid", "overall_conversion_rate": 3.0, "total_entries": 3000, "total_conversions": 90},
{"rank": 3, "segment": "email", "overall_conversion_rate": 2.5, "total_entries": 2000, "total_conversions": 50}
],
"key_findings": {
"all_segments_bottleneck_absolute": "Awareness -> Interest",
"all_segments_bottleneck_relative": "Intent -> Purchase",
"best_performing_segment": "organic (5.6% overall conversion)",
"worst_performing_segment": "email (2.5% overall conversion)"
}
},
"campaign_roi_calculator": {
"_command": "python scripts/campaign_roi_calculator.py assets/sample_campaign_data.json --format json",
"portfolio_summary": {
"total_campaigns": 5,
"total_spend": 34000.0,
"total_revenue": 99000.0,
"total_profit": 65000.0,
"portfolio_roi_pct": 191.18,
"portfolio_roas": 2.91,
"blended_ctr_pct": 1.04,
"blended_cpl": 27.64,
"blended_cpa": 161.9,
"top_performer": "Spring Email Campaign",
"underperforming_campaigns": [
"Spring Email Campaign",
"Facebook Awareness Q1",
"LinkedIn B2B Outreach"
]
},
"channel_summary": {
"email": {"spend": 5000.0, "revenue": 25000.0, "roi_pct": 400.0, "roas": 5.0},
"paid_search": {"spend": 12000.0, "revenue": 48000.0, "roi_pct": 300.0, "roas": 4.0},
"paid_social": {"spend": 14000.0, "revenue": 17000.0, "roi_pct": 21.43, "roas": 1.21},
"display": {"spend": 3000.0, "revenue": 9000.0, "roi_pct": 200.0, "roas": 3.0}
},
"key_findings": {
"most_profitable_channel": "paid_search ($36,000 profit)",
"highest_roas_channel": "email (5.0x ROAS)",
"unprofitable_campaign": "LinkedIn B2B Outreach (-$1,000 loss)",
"best_ctr": "Spring Email Campaign (5.0%)"
}
}
}
FILE:assets/sample_campaign_data.json
{
"journeys": [
{
"journey_id": "j001",
"touchpoints": [
{"channel": "organic_search", "timestamp": "2025-10-01T10:00:00", "interaction": "click"},
{"channel": "email", "timestamp": "2025-10-05T14:30:00", "interaction": "open"},
{"channel": "paid_search", "timestamp": "2025-10-08T09:15:00", "interaction": "click"}
],
"converted": true,
"revenue": 500.00
},
{
"journey_id": "j002",
"touchpoints": [
{"channel": "paid_social", "timestamp": "2025-10-02T11:00:00", "interaction": "click"},
{"channel": "organic_search", "timestamp": "2025-10-06T16:45:00", "interaction": "click"},
{"channel": "email", "timestamp": "2025-10-09T08:00:00", "interaction": "click"},
{"channel": "direct", "timestamp": "2025-10-10T13:20:00", "interaction": "visit"}
],
"converted": true,
"revenue": 1200.00
},
{
"journey_id": "j003",
"touchpoints": [
{"channel": "display", "timestamp": "2025-10-03T09:30:00", "interaction": "view"},
{"channel": "paid_search", "timestamp": "2025-10-07T10:00:00", "interaction": "click"}
],
"converted": true,
"revenue": 350.00
},
{
"journey_id": "j004",
"touchpoints": [
{"channel": "organic_social", "timestamp": "2025-10-01T08:00:00", "interaction": "click"},
{"channel": "email", "timestamp": "2025-10-04T12:00:00", "interaction": "click"},
{"channel": "paid_search", "timestamp": "2025-10-08T14:00:00", "interaction": "click"},
{"channel": "email", "timestamp": "2025-10-11T09:00:00", "interaction": "click"},
{"channel": "direct", "timestamp": "2025-10-12T16:00:00", "interaction": "visit"}
],
"converted": true,
"revenue": 800.00
},
{
"journey_id": "j005",
"touchpoints": [
{"channel": "paid_social", "timestamp": "2025-10-05T10:00:00", "interaction": "click"},
{"channel": "display", "timestamp": "2025-10-08T11:30:00", "interaction": "view"}
],
"converted": false,
"revenue": 0
},
{
"journey_id": "j006",
"touchpoints": [
{"channel": "referral", "timestamp": "2025-10-06T14:00:00", "interaction": "click"},
{"channel": "email", "timestamp": "2025-10-10T09:30:00", "interaction": "click"},
{"channel": "paid_search", "timestamp": "2025-10-13T11:00:00", "interaction": "click"}
],
"converted": true,
"revenue": 650.00
},
{
"journey_id": "j007",
"touchpoints": [
{"channel": "organic_search", "timestamp": "2025-10-04T08:30:00", "interaction": "click"}
],
"converted": true,
"revenue": 200.00
},
{
"journey_id": "j008",
"touchpoints": [
{"channel": "paid_social", "timestamp": "2025-10-07T13:00:00", "interaction": "click"},
{"channel": "organic_search", "timestamp": "2025-10-09T10:00:00", "interaction": "click"},
{"channel": "email", "timestamp": "2025-10-12T15:00:00", "interaction": "click"}
],
"converted": false,
"revenue": 0
}
],
"funnel": {
"stages": ["Awareness", "Interest", "Consideration", "Intent", "Purchase"],
"counts": [10000, 5200, 2800, 1400, 420]
},
"segments": {
"organic": {
"counts": [5000, 2800, 1600, 850, 280]
},
"paid": {
"counts": [3000, 1500, 750, 350, 90]
},
"email": {
"counts": [2000, 900, 450, 200, 50]
}
},
"stages": ["Awareness", "Interest", "Consideration", "Intent", "Purchase"],
"campaigns": [
{
"name": "Spring Email Campaign",
"channel": "email",
"spend": 5000.00,
"revenue": 25000.00,
"impressions": 50000,
"clicks": 2500,
"leads": 300,
"customers": 45
},
{
"name": "Google Search - Brand",
"channel": "paid_search",
"spend": 12000.00,
"revenue": 48000.00,
"impressions": 200000,
"clicks": 8000,
"leads": 600,
"customers": 120
},
{
"name": "Facebook Awareness Q1",
"channel": "paid_social",
"spend": 8000.00,
"revenue": 12000.00,
"impressions": 500000,
"clicks": 5000,
"leads": 200,
"customers": 25
},
{
"name": "Display Retargeting",
"channel": "display",
"spend": 3000.00,
"revenue": 9000.00,
"impressions": 800000,
"clicks": 1200,
"leads": 80,
"customers": 15
},
{
"name": "LinkedIn B2B Outreach",
"channel": "paid_social",
"spend": 6000.00,
"revenue": 5000.00,
"impressions": 120000,
"clicks": 600,
"leads": 50,
"customers": 5
}
]
}
FILE:references/attribution-models-guide.md
# Attribution Models Guide
Comprehensive reference for multi-touch attribution modeling in marketing analytics. This guide covers the five standard attribution models, their mathematical foundations, selection criteria, and practical application guidelines.
---
## Overview
Attribution modeling answers the question: **Which marketing touchpoints deserve credit for conversions?** When a customer interacts with multiple channels before converting, attribution models distribute conversion credit across those touchpoints using different rules.
No single model is "correct." Each reveals different aspects of channel performance. Best practice is to run multiple models and compare results to build a complete picture.
---
## Model 1: First-Touch Attribution
### How It Works
All conversion credit (100%) goes to the first touchpoint in the customer journey.
### Formula
```
Credit(channel) = Revenue * 1.0 (if channel is first touchpoint)
Credit(channel) = 0 (otherwise)
```
### When to Use
- **Brand awareness campaigns**: Measures which channels bring new prospects into the funnel
- **Top-of-funnel optimization**: Identifies the best channels for initial discovery
- **New market entry**: Evaluating which channels generate first contact in new segments
### Pros
- Simple to understand and implement
- Clearly identifies awareness-driving channels
- Useful for budget allocation toward customer acquisition
### Cons
- Ignores all touchpoints after the first
- Overvalues awareness channels, undervalues conversion channels
- Does not reflect the reality of multi-touch customer journeys
### Best For
Marketing teams focused on expanding reach and entering new markets where understanding initial discovery channels is the priority.
---
## Model 2: Last-Touch Attribution
### How It Works
All conversion credit (100%) goes to the last touchpoint before conversion.
### Formula
```
Credit(channel) = Revenue * 1.0 (if channel is last touchpoint)
Credit(channel) = 0 (otherwise)
```
### When to Use
- **Direct response campaigns**: Measures which channels close deals
- **Bottom-of-funnel optimization**: Identifies the most effective conversion channels
- **Short sales cycles**: When customers typically convert within 1-2 interactions
### Pros
- Simple to implement (default in many analytics platforms)
- Highlights channels that directly drive conversions
- Useful for performance marketing optimization
### Cons
- Ignores all touchpoints before the last
- Overvalues conversion channels, undervalues awareness channels
- Can lead to cutting awareness spending that actually feeds the pipeline
### Best For
Performance marketing teams running direct-response campaigns where the final interaction is the primary lever.
---
## Model 3: Linear Attribution
### How It Works
Conversion credit is split equally across all touchpoints in the journey.
### Formula
```
Credit(channel) = Revenue / N (for each of N touchpoints)
```
### When to Use
- **Balanced multi-channel evaluation**: When all touchpoints are considered equally valuable
- **Long sales cycles**: Where multiple interactions are required
- **Content marketing**: Where each piece of content plays a role in nurturing
### Pros
- Fair distribution across all channels
- Recognizes the contribution of every touchpoint
- Good starting point for teams new to multi-touch attribution
### Cons
- Treats all touchpoints equally, which rarely reflects reality
- Does not account for the relative importance of different positions in the journey
- Can dilute the signal of truly impactful touchpoints
### Best For
Teams running consistent multi-channel campaigns where every touchpoint is intentionally designed to contribute to conversion.
---
## Model 4: Time-Decay Attribution
### How It Works
Touchpoints closer to conversion receive exponentially more credit. Uses a half-life parameter: a touchpoint occurring one half-life before conversion gets 50% of the credit of the converting touchpoint.
### Formula
```
Weight(touchpoint) = e^(-lambda * days_before_conversion)
where lambda = ln(2) / half_life_days
Credit(channel) = Revenue * (Weight / Sum_of_all_weights)
```
### Configurable Parameters
| Parameter | Default | Description |
|-----------|---------|-------------|
| half_life_days | 7 | Days for weight to decay by 50% |
### Guidance on Half-Life Selection
| Sales Cycle Length | Recommended Half-Life |
|-------------------|----------------------|
| 1-3 days (impulse) | 1-2 days |
| 1-2 weeks (considered) | 5-7 days |
| 1-3 months (B2B) | 14-21 days |
| 3-6 months (enterprise) | 30-45 days |
| 6-12 months (complex B2B) | 60-90 days |
### When to Use
- **Short-to-medium sales cycles**: Where recent interactions are more influential
- **Promotional campaigns**: Where urgency and recency matter
- **E-commerce**: Where the last few interactions before purchase are most impactful
### Pros
- Accounts for recency, which aligns with many buying behaviors
- More sophisticated than first/last-touch
- Configurable half-life allows tuning to specific business contexts
### Cons
- May undervalue early-stage awareness that planted the seed
- Half-life selection is subjective and requires testing
- More complex to explain to stakeholders
### Best For
E-commerce and B2C companies with identifiable sales cycles where recent interactions carry more decision weight.
---
## Model 5: Position-Based Attribution (U-Shaped)
### How It Works
40% of credit goes to the first touchpoint, 40% to the last touchpoint, and the remaining 20% is split equally among middle touchpoints.
### Formula
```
Credit(first_channel) = Revenue * 0.40
Credit(last_channel) = Revenue * 0.40
Credit(middle_channel) = Revenue * 0.20 / (N - 2) (for each middle touchpoint)
Special cases:
- 1 touchpoint: 100% credit
- 2 touchpoints: 50% each
```
### When to Use
- **Full-funnel marketing**: Values both awareness (first) and conversion (last)
- **Mature marketing programs**: With established multi-channel strategies
- **B2B marketing**: Where both lead generation and deal closure are distinct priorities
### Pros
- Recognizes the importance of first and last interactions
- Still gives credit to middle nurturing touchpoints
- Provides a balanced view of the full journey
### Cons
- The 40/20/40 split is arbitrary (some businesses may need 30/40/30 or other splits)
- Middle touchpoints get relatively little credit
- May not suit businesses where middle interactions are the primary differentiator
### Best For
B2B and enterprise marketing teams running coordinated campaigns across the full customer journey from awareness through conversion.
---
## Model Comparison Matrix
| Criteria | First-Touch | Last-Touch | Linear | Time-Decay | Position-Based |
|----------|------------|------------|--------|------------|----------------|
| Complexity | Low | Low | Low | Medium | Medium |
| Awareness bias | High | None | Neutral | Low | Medium |
| Conversion bias | None | High | Neutral | High | Medium |
| Multi-touch fairness | Poor | Poor | Good | Good | Good |
| Best sales cycle | Any | Short | Long | Short-Medium | Any |
| Stakeholder clarity | High | High | High | Medium | Medium |
---
## Practical Guidelines
### Running Multiple Models
Always run at least 3 models and look for channels that rank highly across multiple models. These are your most reliable performers. Channels that rank well in only one model may be overvalued by that model's bias.
### Interpreting Divergent Results
When models disagree significantly on a channel's value:
1. **High in first-touch, low in last-touch**: The channel is strong for awareness but does not close. Pair it with stronger conversion channels.
2. **Low in first-touch, high in last-touch**: The channel closes deals but does not generate new prospects. Ensure upstream awareness channels feed it.
3. **High in linear, low in first/last**: The channel plays a critical nurturing role. Cutting it may break the journey without immediately visible impact.
### Common Pitfalls
- **Over-relying on last-touch**: Most analytics platforms default to last-touch, which chronically undervalues awareness spending.
- **Ignoring non-converting journeys**: Attribution only counts converted journeys. Channels that contribute to unconverted journeys may still have value.
- **Confusing correlation with causation**: Attribution shows correlation between touchpoints and conversion, not definitive causation.
- **Insufficient data volume**: Models require statistically meaningful journey counts. With fewer than 100 journeys, results are unreliable.
---
## Data Requirements
### Minimum Data
| Field | Required | Description |
|-------|----------|-------------|
| journey_id | Yes | Unique identifier for each customer journey |
| touchpoints | Yes | Array of channel interactions with timestamps |
| converted | Yes | Boolean indicating whether the journey converted |
| revenue | Recommended | Conversion value for credit allocation |
### Touchpoint Fields
| Field | Required | Description |
|-------|----------|-------------|
| channel | Yes | Marketing channel name |
| timestamp | Yes | ISO-format timestamp of the interaction |
| interaction | Optional | Type of interaction (click, view, open, etc.) |
---
## Further Reading
- Google Analytics attribution model comparison documentation
- Facebook/Meta attribution window settings and their impact
- HubSpot multi-touch revenue attribution methodology
- Bizible/Marketo B2B attribution best practices
FILE:references/campaign-metrics-benchmarks.md
# Campaign Metrics Benchmarks
Industry benchmark reference for marketing campaign performance metrics. Use these benchmarks to contextualize your campaign results, identify underperformance, and set realistic targets.
---
## How to Use This Reference
1. Find your industry vertical and channel combination
2. Compare your actual metrics to the benchmark ranges
3. Use the assessment scale: Below Low = underperforming, Low-Target = below target, Target-High = good, Above High = excellent
4. Adjust targets based on your historical performance (your own data is always the best benchmark)
---
## Click-Through Rate (CTR) Benchmarks
CTR = (Clicks / Impressions) * 100
### By Channel (Cross-Industry Average)
| Channel | Low | Target | High | Notes |
|---------|-----|--------|------|-------|
| Email | 1.0% | 2.5% | 5.0% | Highly dependent on list quality and segmentation |
| Paid Search (Google) | 1.5% | 3.5% | 7.0% | Brand keywords typically 5-10%, generic 1-3% |
| Paid Social (Facebook) | 0.5% | 1.2% | 3.0% | Video ads trend higher, static lower |
| Paid Social (LinkedIn) | 0.3% | 0.8% | 2.0% | B2B focused, lower volume but higher intent |
| Display Ads | 0.05% | 0.10% | 0.50% | Retargeting typically 0.5-1.0% |
| Organic Search | 1.5% | 3.0% | 8.0% | Position 1 averages 28-31% CTR |
| Organic Social | 0.5% | 1.5% | 4.0% | Platform algorithm changes affect significantly |
| Referral | 1.0% | 3.0% | 6.0% | Quality of referring site matters greatly |
| Direct | 2.0% | 4.0% | 8.0% | Highest intent channel |
### By Industry (Paid Search)
| Industry | Average CTR | Low | High |
|----------|------------|-----|------|
| B2B | 2.4% | 1.5% | 4.0% |
| E-commerce | 2.7% | 1.8% | 5.0% |
| Education | 3.3% | 2.0% | 6.0% |
| Finance & Insurance | 2.9% | 1.5% | 5.5% |
| Healthcare | 3.3% | 2.0% | 5.0% |
| Legal | 2.9% | 1.5% | 5.0% |
| Real Estate | 3.7% | 2.5% | 6.0% |
| Retail | 2.5% | 1.5% | 5.0% |
| SaaS | 2.1% | 1.2% | 3.5% |
| Technology | 2.1% | 1.0% | 4.0% |
| Travel & Hospitality | 4.7% | 3.0% | 8.0% |
---
## Cost Per Click (CPC) Benchmarks
CPC = Spend / Clicks
### By Channel (USD)
| Channel | Low | Target | High | Notes |
|---------|-----|--------|------|-------|
| Google Search | $0.50 | $2.50 | $8.00 | Legal/finance can exceed $50 per click |
| Google Display | $0.10 | $0.50 | $2.00 | Programmatic can be lower |
| Facebook | $0.30 | $1.00 | $3.00 | B2C typically lower than B2B |
| LinkedIn | $2.00 | $5.50 | $12.00 | Highest CPC among social platforms |
| Instagram | $0.40 | $1.20 | $3.50 | Stories ads trending lower |
| Twitter/X | $0.20 | $0.80 | $2.50 | High variability by topic |
| TikTok | $0.10 | $0.50 | $2.00 | Rapidly evolving, currently lower |
### By Industry (Google Ads)
| Industry | Average CPC | Range |
|----------|------------|-------|
| Automotive | $2.46 | $1.00-$6.00 |
| B2B | $3.33 | $1.50-$8.00 |
| E-commerce | $1.16 | $0.50-$3.00 |
| Education | $2.40 | $1.00-$5.00 |
| Finance & Insurance | $3.44 | $1.00-$50.00 |
| Healthcare | $2.62 | $1.00-$6.00 |
| Legal | $6.75 | $2.00-$100.00 |
| Real Estate | $2.37 | $1.00-$5.00 |
| SaaS/Technology | $3.80 | $1.50-$10.00 |
| Travel | $1.53 | $0.50-$4.00 |
---
## Cost Per Mille / Thousand Impressions (CPM) Benchmarks
CPM = (Spend / Impressions) * 1000
### By Channel (USD)
| Channel | Low | Target | High | Notes |
|---------|-----|--------|------|-------|
| Facebook | $3.00 | $8.00 | $15.00 | Q4 holiday season can exceed $20 |
| Instagram | $4.00 | $10.00 | $18.00 | Reels ads trending lower |
| LinkedIn | $8.00 | $25.00 | $50.00 | Premium B2B audience |
| Google Display | $1.00 | $3.50 | $8.00 | Programmatic ranges widely |
| TikTok | $2.00 | $6.00 | $12.00 | Growing platform, rates increasing |
| YouTube | $4.00 | $10.00 | $20.00 | Pre-roll vs discovery ads vary |
| Programmatic Display | $0.50 | $2.00 | $6.00 | Dependent on targeting precision |
---
## Cost Per Acquisition (CPA) Benchmarks
CPA = Spend / Customers Acquired
### By Channel (USD)
| Channel | Low | Target | High | Notes |
|---------|-----|--------|------|-------|
| Email | $5 | $15 | $40 | Existing list; acquisition cost amortized |
| Paid Search | $20 | $50 | $150 | Highly dependent on industry and competition |
| Paid Social | $15 | $40 | $100 | Retargeting typically lower |
| Display | $30 | $75 | $200 | Awareness-focused; higher CPA expected |
| Organic Search | $5 | $20 | $60 | Excludes SEO investment costs |
| Organic Social | $10 | $30 | $80 | Content production costs excluded |
| Referral | $10 | $25 | $70 | Referral incentive costs included |
### By Industry (Across Channels)
| Industry | Average CPA | Acceptable Range |
|----------|------------|------------------|
| B2B SaaS | $150-$400 | $75-$700 |
| E-commerce | $25-$80 | $10-$150 |
| Education | $40-$120 | $20-$250 |
| Finance | $75-$200 | $30-$500 |
| Healthcare | $50-$150 | $25-$300 |
| Legal | $100-$300 | $50-$700 |
| Real Estate | $60-$180 | $30-$350 |
| Retail | $15-$50 | $8-$100 |
| Travel | $20-$70 | $10-$150 |
---
## Cost Per Lead (CPL) Benchmarks
CPL = Spend / Leads Generated
### By Channel (USD)
| Channel | Low | Target | High |
|---------|-----|--------|------|
| Email | $3 | $10 | $25 |
| Paid Search | $15 | $35 | $90 |
| Paid Social (Facebook) | $8 | $20 | $50 |
| Paid Social (LinkedIn) | $25 | $75 | $150 |
| Display | $20 | $50 | $120 |
| Content Marketing | $10 | $30 | $80 |
| Webinars | $30 | $70 | $150 |
### By Industry
| Industry | Average CPL | Range |
|----------|------------|-------|
| B2B SaaS | $50-$150 | $25-$300 |
| E-commerce | $10-$30 | $5-$60 |
| Education | $25-$70 | $15-$150 |
| Financial Services | $40-$120 | $20-$250 |
| Healthcare | $30-$90 | $15-$180 |
| Manufacturing | $50-$120 | $25-$200 |
| Technology | $40-$100 | $20-$200 |
---
## Return on Ad Spend (ROAS) Benchmarks
ROAS = Revenue / Ad Spend
### By Channel
| Channel | Low | Target | High | Notes |
|---------|-----|--------|------|-------|
| Email | 30x | 42x | 60x | Highest ROAS channel when list is healthy |
| Paid Search (Brand) | 8x | 15x | 30x | Brand terms have high ROAS |
| Paid Search (Generic) | 2x | 4x | 8x | Competitive; ROAS varies widely |
| Paid Social | 1.5x | 3x | 6x | Retargeting typically 4-10x |
| Display | 0.5x | 1.5x | 3x | Often used for awareness; lower direct ROAS |
| Organic Search | 5x | 10x | 20x | Excludes SEO investment amortization |
| Organic Social | 3x | 6x | 12x | Excludes content production costs |
### By Industry
| Industry | Minimum Viable ROAS | Target ROAS |
|----------|--------------------:|------------:|
| E-commerce (low margin) | 4x | 8x+ |
| E-commerce (high margin) | 2x | 4x+ |
| SaaS | 3x | 6x+ |
| B2B Services | 5x | 10x+ |
| Retail | 3x | 5x+ |
| DTC Brands | 2.5x | 5x+ |
### ROAS Calculation Notes
- **Breakeven ROAS** = 1 / Profit Margin (e.g., 25% margin = 4x breakeven)
- **Target ROAS** should be at least 2x the breakeven ROAS for sustainable growth
- Always include all costs (media, creative, tools, labor) for true ROAS
---
## Conversion Rate Benchmarks
### Landing Page Conversion Rate
| Industry | Low | Average | High |
|----------|-----|---------|------|
| B2B SaaS | 2.0% | 4.5% | 9.0% |
| E-commerce | 1.5% | 3.0% | 6.0% |
| Education | 2.5% | 5.5% | 10.0% |
| Finance | 2.0% | 5.0% | 11.0% |
| Healthcare | 2.0% | 4.0% | 8.0% |
| Legal | 3.0% | 7.0% | 13.0% |
| Real Estate | 2.0% | 4.5% | 8.0% |
| Travel | 2.0% | 4.0% | 9.0% |
### Email Conversion Rates
| Metric | Low | Average | High |
|--------|-----|---------|------|
| Open Rate | 15% | 22% | 35% |
| Click Rate | 1.0% | 2.5% | 5.0% |
| Click-to-Open Rate | 8% | 12% | 20% |
| Unsubscribe Rate | 0.1% | 0.2% | 0.5% |
---
## Seasonal Adjustments
Campaign benchmarks fluctuate by season. Apply these adjustment factors to normalize your comparisons:
| Quarter | CPC Adjustment | CPM Adjustment | CVR Adjustment |
|---------|---------------|----------------|----------------|
| Q1 (Jan-Mar) | -10% to -15% | -15% to -20% | Baseline |
| Q2 (Apr-Jun) | Baseline | Baseline | Baseline |
| Q3 (Jul-Sep) | +5% to +10% | +5% to +10% | -5% |
| Q4 (Oct-Dec) | +15% to +30% | +20% to +40% | +10% to +20% |
**Key seasonal events:**
- Black Friday/Cyber Monday: CPMs can increase 50-100%
- January: Lowest competition, good for testing
- Back-to-School (Aug-Sep): Education and retail spike
- Tax Season (Jan-Apr): Finance vertical spike
---
## Using Benchmarks Effectively
### Do
- Compare against your own historical data first, then industry benchmarks
- Account for seasonality when comparing time periods
- Consider your funnel position (awareness vs conversion campaigns have different benchmarks)
- Update benchmarks annually as industry norms shift
### Do Not
- Treat benchmarks as absolute targets (your business context matters more)
- Compare across industries without adjustment
- Ignore sample size (small campaigns have high variance)
- Use benchmarks to justify cutting channels without understanding their full-funnel role
FILE:references/funnel-optimization-framework.md
# Funnel Optimization Framework
A stage-by-stage guide to diagnosing and improving marketing and sales funnel performance. Use this framework alongside the funnel_analyzer.py tool to identify bottlenecks and implement targeted optimizations.
---
## The Standard Marketing Funnel
```
AWARENESS (Impressions, Reach)
|
INTEREST (Clicks, Engagement)
|
CONSIDERATION (Leads, Sign-ups)
|
INTENT (Demos, Trials, Cart Adds)
|
PURCHASE (Customers, Revenue)
|
RETENTION (Repeat, Upsell, Referral)
```
Each transition between stages represents a conversion point. The funnel analyzer measures these transitions and identifies where the largest drop-offs occur.
---
## Stage-by-Stage Optimization
### Stage 1: Awareness to Interest
**What it measures:** How effectively you capture attention and generate initial engagement.
**Healthy conversion rate:** 2-8% (varies widely by channel)
**Common bottlenecks:**
- Poor targeting: Reaching the wrong audience
- Weak creative: Ads that do not stand out or communicate value
- Message-market mismatch: Content that does not resonate with the audience's needs
- Low brand recognition: No trust or familiarity established
**Optimization tactics:**
| Tactic | Expected Impact | Effort |
|--------|----------------|--------|
| Audience refinement (lookalike, interest targeting) | High | Medium |
| Creative testing (3-5 variants per campaign) | High | Medium |
| Headline optimization (clear value proposition) | Medium | Low |
| Channel diversification (test new platforms) | Medium | High |
| Retargeting past engagers | Medium | Low |
**Key metrics to track:**
- Impressions and reach
- CTR by creative variant
- Cost per engagement
- Brand lift (if measured)
---
### Stage 2: Interest to Consideration
**What it measures:** How well you convert initial interest into genuine evaluation.
**Healthy conversion rate:** 10-30%
**Common bottlenecks:**
- Landing page disconnect: The page does not match the ad promise
- Poor user experience: Slow load times, confusing layout, mobile issues
- Missing social proof: No testimonials, case studies, or trust signals
- Unclear value proposition: Visitor does not understand "what's in it for me"
- Friction in lead capture: Too many form fields, unclear CTA
**Optimization tactics:**
| Tactic | Expected Impact | Effort |
|--------|----------------|--------|
| Landing page A/B testing | High | Medium |
| Message match (ad copy = page headline) | High | Low |
| Reduce form fields to essential only | High | Low |
| Add social proof (logos, testimonials, numbers) | Medium | Low |
| Improve page load speed (<3 seconds) | Medium | Medium |
| Mobile optimization | Medium | Medium |
| Add exit-intent offers | Low-Medium | Low |
**Key metrics to track:**
- Landing page conversion rate
- Bounce rate
- Time on page
- Form abandonment rate
---
### Stage 3: Consideration to Intent
**What it measures:** How effectively you move evaluated prospects toward a purchase decision.
**Healthy conversion rate:** 15-40%
**Common bottlenecks:**
- Insufficient nurturing: Leads go cold without follow-up
- Lack of differentiation: Prospects do not understand why you are better than alternatives
- Missing information: Pricing, features, or comparisons not available
- Sales-marketing misalignment: MQLs are not meeting sales expectations
- Poor timing: Follow-up is too slow or too aggressive
**Optimization tactics:**
| Tactic | Expected Impact | Effort |
|--------|----------------|--------|
| Email nurture sequences (5-7 touchpoints) | High | Medium |
| Lead scoring to prioritize sales outreach | High | High |
| Comparison content (vs. competitors) | Medium | Medium |
| Free trial or demo offers | High | Medium |
| Case studies relevant to prospect's industry | Medium | Medium |
| Retargeting with mid-funnel content | Medium | Low |
| Pricing transparency | Medium | Low |
**Key metrics to track:**
- MQL to SQL conversion rate
- Lead response time
- Email engagement rates (nurture sequences)
- Content engagement (case studies, comparisons)
---
### Stage 4: Intent to Purchase
**What it measures:** How well you convert ready-to-buy prospects into paying customers.
**Healthy conversion rate:** 20-50%
**Common bottlenecks:**
- Complex purchase process: Too many steps, unclear pricing, difficult checkout
- Lack of urgency: No reason to buy now
- Unaddressed objections: Common concerns not proactively handled
- Poor sales process: Inconsistent follow-up, inadequate discovery
- Payment friction: Limited payment options, security concerns
**Optimization tactics:**
| Tactic | Expected Impact | Effort |
|--------|----------------|--------|
| Simplify checkout/purchase flow | High | Medium |
| Add urgency (limited-time offers, scarcity) | Medium | Low |
| Address objections in sales collateral | Medium | Medium |
| Offer guarantees (money-back, free trial extension) | Medium | Low |
| Cart abandonment emails (3-email sequence) | High | Low |
| Live chat or chatbot support at checkout | Medium | Medium |
| Multiple payment options | Low-Medium | Medium |
| Customer success stories at point of purchase | Medium | Low |
**Key metrics to track:**
- Cart abandonment rate
- Checkout completion rate
- Average deal cycle length
- Win rate (B2B)
- Average order value
---
### Stage 5: Purchase to Retention
**What it measures:** How well you retain customers and expand their lifetime value.
**Healthy retention rate:** 70-95% annually (varies by business model)
**Common bottlenecks:**
- Poor onboarding: Customers do not achieve value quickly
- Lack of engagement: No ongoing communication or community
- Product/service issues: Unmet expectations post-purchase
- No expansion path: No upsell, cross-sell, or referral programs
- Competitor poaching: Better offers from alternatives
**Optimization tactics:**
| Tactic | Expected Impact | Effort |
|--------|----------------|--------|
| Structured onboarding (first 30/60/90 days) | High | High |
| Regular check-ins and health scoring | High | Medium |
| Loyalty programs | Medium | Medium |
| Referral incentives | Medium | Low |
| Cross-sell/upsell email sequences | Medium | Medium |
| Customer community building | Medium | High |
| Proactive support based on usage patterns | High | High |
**Key metrics to track:**
- Customer retention rate
- Net Promoter Score (NPS)
- Customer Lifetime Value (CLV)
- Expansion revenue
- Churn rate and reasons
---
## Bottleneck Diagnosis Framework
When the funnel analyzer identifies a bottleneck, use this diagnostic framework:
### Step 1: Quantify the Problem
- What is the conversion rate at this stage?
- How does it compare to your historical average?
- How does it compare to industry benchmarks?
- What is the absolute number of prospects lost?
### Step 2: Segment the Data
Look at the bottleneck broken down by:
- **Channel**: Is the drop-off worse for certain traffic sources?
- **Device**: Mobile vs desktop performance gaps
- **Geography**: Regional differences
- **Cohort**: Has it changed over time?
- **Campaign**: Specific campaigns performing worse
### Step 3: Identify Root Cause
| Symptom | Likely Root Cause | Diagnostic Action |
|---------|------------------|-------------------|
| High bounce rate | Message mismatch or UX issue | Review landing page vs ad |
| High time on page but low conversion | Confusion or missing CTA | Heatmap analysis |
| Drop-off at form | Too many fields or unclear value | Form analytics review |
| Long time between stages | Insufficient nurturing | Review email engagement |
| Drop-off after pricing page | Pricing concerns | Test pricing presentation |
| High cart abandonment | Checkout friction | Checkout flow analysis |
### Step 4: Prioritize Fixes
Use the ICE scoring framework:
- **Impact** (1-10): How much will fixing this improve the bottleneck?
- **Confidence** (1-10): How confident are you that this fix will work?
- **Ease** (1-10): How easy is this to implement?
Score = (Impact + Confidence + Ease) / 3
Prioritize fixes with the highest ICE score.
---
## Funnel Math and Revenue Impact
### Calculating the Revenue Impact of Funnel Improvements
A useful way to prioritize is to calculate how much revenue each percentage point of improvement is worth at each stage.
**Formula:**
```
Revenue Impact = Current_Revenue * (1 / Current_Conversion_Rate) * Improvement_Percentage
```
**Example:**
| Stage | Current Rate | +1pp Improvement | Revenue Impact |
|-------|-------------|-----------------|----------------|
| Awareness -> Interest | 5.0% | 6.0% | +20% more leads entering funnel |
| Interest -> Consideration | 25% | 26% | +4% more MQLs |
| Consideration -> Intent | 30% | 31% | +3.3% more SQLs |
| Intent -> Purchase | 40% | 41% | +2.5% more customers |
**Key insight:** Improvements at the top of the funnel have a multiplied effect on downstream stages. But improvements at the bottom of the funnel convert to revenue faster.
---
## Common Anti-Patterns
### 1. Optimizing the Wrong Stage
Fixing a bottom-of-funnel problem when the real issue is top-of-funnel volume. Always diagnose the full funnel before optimizing.
### 2. Ignoring Segment Differences
Aggregate funnel metrics can hide that one segment performs well while another is broken. Always segment before optimizing.
### 3. Over-Optimizing for Conversion Rate
Increasing conversion rate by narrowing the funnel (stricter targeting, higher-intent-only leads) can reduce total volume. Balance rate and volume.
### 4. Single-Metric Focus
Optimizing CTR without watching CPA, or optimizing CPA without watching volume. Always track paired metrics.
### 5. Not Accounting for Time Lag
B2B funnels can take weeks or months. Measuring a campaign's funnel performance too early produces incomplete data.
---
## Segment Comparison Best Practices
When using the funnel analyzer's segment comparison feature:
1. **Compare meaningful segments**: Channel, campaign type, audience demographic, or time period
2. **Ensure comparable volume**: Do not compare a segment with 100 entries to one with 10,000
3. **Look for stage-specific differences**: Two segments may have similar overall rates but different bottlenecks
4. **Use insights to inform targeting**: If one segment converts better at a specific stage, understand why and apply those lessons
---
## Recommended Review Cadence
| Review Type | Frequency | Focus |
|-------------|-----------|-------|
| Campaign funnel check | Weekly | Active campaign stage rates |
| Full funnel audit | Monthly | Overall funnel health, bottleneck shifts |
| Segment deep-dive | Monthly | Channel and cohort comparisons |
| Strategic funnel review | Quarterly | Funnel structure, stage definitions, benchmark updates |
| Annual funnel redesign | Annually | Stage definitions, measurement methodology, tool updates |
FILE:scripts/attribution_analyzer.py
#!/usr/bin/env python3
"""
Attribution Analyzer - Multi-touch attribution modeling for marketing campaigns.
Implements 5 attribution models:
- first-touch: 100% credit to first interaction
- last-touch: 100% credit to last interaction
- linear: Equal credit across all touchpoints
- time-decay: Exponential decay favoring recent touchpoints
- position-based: 40% first, 40% last, 20% split among middle
Usage:
python attribution_analyzer.py data.json
python attribution_analyzer.py data.json --model time-decay
python attribution_analyzer.py data.json --model time-decay --half-life 14
python attribution_analyzer.py data.json --format json
"""
import argparse
import json
import sys
from datetime import datetime
from typing import Any, Dict, List, Optional
MODELS = ["first-touch", "last-touch", "linear", "time-decay", "position-based"]
def safe_divide(numerator: float, denominator: float, default: float = 0.0) -> float:
"""Safely divide two numbers, returning default if denominator is zero."""
if denominator == 0:
return default
return numerator / denominator
def parse_timestamp(ts: str) -> datetime:
"""Parse an ISO-format timestamp string into a datetime object."""
for fmt in ("%Y-%m-%dT%H:%M:%S", "%Y-%m-%d %H:%M:%S", "%Y-%m-%d"):
try:
return datetime.strptime(ts, fmt)
except ValueError:
continue
raise ValueError(f"Cannot parse timestamp: {ts}")
def first_touch_attribution(journeys: List[Dict]) -> Dict[str, float]:
"""First-touch: 100% credit to the first touchpoint in each journey."""
credits: Dict[str, float] = {}
for journey in journeys:
if not journey.get("converted", False):
continue
touchpoints = journey.get("touchpoints", [])
if not touchpoints:
continue
sorted_tp = sorted(touchpoints, key=lambda t: parse_timestamp(t["timestamp"]))
channel = sorted_tp[0]["channel"]
revenue = journey.get("revenue", 1.0)
credits[channel] = credits.get(channel, 0.0) + revenue
return credits
def last_touch_attribution(journeys: List[Dict]) -> Dict[str, float]:
"""Last-touch: 100% credit to the last touchpoint in each journey."""
credits: Dict[str, float] = {}
for journey in journeys:
if not journey.get("converted", False):
continue
touchpoints = journey.get("touchpoints", [])
if not touchpoints:
continue
sorted_tp = sorted(touchpoints, key=lambda t: parse_timestamp(t["timestamp"]))
channel = sorted_tp[-1]["channel"]
revenue = journey.get("revenue", 1.0)
credits[channel] = credits.get(channel, 0.0) + revenue
return credits
def linear_attribution(journeys: List[Dict]) -> Dict[str, float]:
"""Linear: Equal credit split across all touchpoints in each journey."""
credits: Dict[str, float] = {}
for journey in journeys:
if not journey.get("converted", False):
continue
touchpoints = journey.get("touchpoints", [])
if not touchpoints:
continue
revenue = journey.get("revenue", 1.0)
share = safe_divide(revenue, len(touchpoints))
for tp in touchpoints:
channel = tp["channel"]
credits[channel] = credits.get(channel, 0.0) + share
return credits
def time_decay_attribution(journeys: List[Dict], half_life_days: float = 7.0) -> Dict[str, float]:
"""Time-decay: Exponential decay giving more credit to recent touchpoints.
Uses a configurable half-life (in days). Touchpoints closer to conversion
receive exponentially more credit.
"""
import math
credits: Dict[str, float] = {}
decay_rate = math.log(2) / half_life_days
for journey in journeys:
if not journey.get("converted", False):
continue
touchpoints = journey.get("touchpoints", [])
if not touchpoints:
continue
revenue = journey.get("revenue", 1.0)
sorted_tp = sorted(touchpoints, key=lambda t: parse_timestamp(t["timestamp"]))
conversion_time = parse_timestamp(sorted_tp[-1]["timestamp"])
# Calculate raw weights
weights: List[float] = []
for tp in sorted_tp:
tp_time = parse_timestamp(tp["timestamp"])
days_before = (conversion_time - tp_time).total_seconds() / 86400.0
weight = math.exp(-decay_rate * days_before)
weights.append(weight)
total_weight = sum(weights)
if total_weight == 0:
continue
for i, tp in enumerate(sorted_tp):
channel = tp["channel"]
share = safe_divide(weights[i], total_weight) * revenue
credits[channel] = credits.get(channel, 0.0) + share
return credits
def position_based_attribution(journeys: List[Dict]) -> Dict[str, float]:
"""Position-based: 40% first, 40% last, 20% split among middle touchpoints."""
credits: Dict[str, float] = {}
for journey in journeys:
if not journey.get("converted", False):
continue
touchpoints = journey.get("touchpoints", [])
if not touchpoints:
continue
revenue = journey.get("revenue", 1.0)
sorted_tp = sorted(touchpoints, key=lambda t: parse_timestamp(t["timestamp"]))
if len(sorted_tp) == 1:
channel = sorted_tp[0]["channel"]
credits[channel] = credits.get(channel, 0.0) + revenue
elif len(sorted_tp) == 2:
first_channel = sorted_tp[0]["channel"]
last_channel = sorted_tp[-1]["channel"]
credits[first_channel] = credits.get(first_channel, 0.0) + revenue * 0.5
credits[last_channel] = credits.get(last_channel, 0.0) + revenue * 0.5
else:
first_channel = sorted_tp[0]["channel"]
last_channel = sorted_tp[-1]["channel"]
credits[first_channel] = credits.get(first_channel, 0.0) + revenue * 0.4
credits[last_channel] = credits.get(last_channel, 0.0) + revenue * 0.4
middle_count = len(sorted_tp) - 2
middle_share = safe_divide(revenue * 0.2, middle_count)
for tp in sorted_tp[1:-1]:
channel = tp["channel"]
credits[channel] = credits.get(channel, 0.0) + middle_share
return credits
def run_model(model_name: str, journeys: List[Dict], half_life: float = 7.0) -> Dict[str, float]:
"""Dispatch to the appropriate attribution model."""
if model_name == "first-touch":
return first_touch_attribution(journeys)
elif model_name == "last-touch":
return last_touch_attribution(journeys)
elif model_name == "linear":
return linear_attribution(journeys)
elif model_name == "time-decay":
return time_decay_attribution(journeys, half_life)
elif model_name == "position-based":
return position_based_attribution(journeys)
else:
raise ValueError(f"Unknown model: {model_name}. Choose from: {', '.join(MODELS)}")
def compute_summary(journeys: List[Dict]) -> Dict[str, Any]:
"""Compute summary statistics about the journey data."""
total_journeys = len(journeys)
converted = sum(1 for j in journeys if j.get("converted", False))
total_revenue = sum(j.get("revenue", 0.0) for j in journeys if j.get("converted", False))
all_channels = set()
for j in journeys:
for tp in j.get("touchpoints", []):
all_channels.add(tp["channel"])
return {
"total_journeys": total_journeys,
"converted_journeys": converted,
"conversion_rate": round(safe_divide(converted, total_journeys) * 100, 2),
"total_revenue": round(total_revenue, 2),
"channels_observed": sorted(all_channels),
}
def format_text(results: Dict[str, Any]) -> str:
"""Format results as human-readable text."""
lines: List[str] = []
lines.append("=" * 70)
lines.append("MULTI-TOUCH ATTRIBUTION ANALYSIS")
lines.append("=" * 70)
summary = results["summary"]
lines.append("")
lines.append("SUMMARY")
lines.append(f" Total Journeys: {summary['total_journeys']}")
lines.append(f" Converted: {summary['converted_journeys']}")
lines.append(f" Conversion Rate: {summary['conversion_rate']}%")
lines.append(f" Total Revenue: ,.2f")
lines.append(f" Channels Observed: {', '.join(summary['channels_observed'])}")
for model_name, credits in results["models"].items():
lines.append("")
lines.append("-" * 70)
lines.append(f"MODEL: {model_name.upper()}")
lines.append("-" * 70)
if not credits:
lines.append(" No conversions to attribute.")
continue
total_credit = sum(credits.values())
sorted_channels = sorted(credits.items(), key=lambda x: x[1], reverse=True)
lines.append(f" {'Channel':<25} {'Revenue Credit':>15} {'Share':>10}")
lines.append(f" {'-'*25} {'-'*15} {'-'*10}")
for channel, credit in sorted_channels:
pct = safe_divide(credit, total_credit) * 100
lines.append(f" {channel:<25} >13,.2f {pct:>8.1f}%")
lines.append(f" {'TOTAL':<25} >13,.2f {'100.0%':>10}")
# Comparison table
if len(results["models"]) > 1:
lines.append("")
lines.append("=" * 70)
lines.append("CROSS-MODEL COMPARISON")
lines.append("=" * 70)
all_channels = set()
for credits in results["models"].values():
all_channels.update(credits.keys())
all_channels_sorted = sorted(all_channels)
model_names = list(results["models"].keys())
header = f" {'Channel':<20}"
for mn in model_names:
short = mn.replace("-", " ").title()
header += f" {short:>14}"
lines.append(header)
lines.append(f" {'-'*20}" + f" {'-'*14}" * len(model_names))
for ch in all_channels_sorted:
row = f" {ch:<20}"
for mn in model_names:
val = results["models"][mn].get(ch, 0.0)
row += f" >12,.2f"
lines.append(row)
lines.append("")
return "\n".join(lines)
def main() -> None:
"""Main entry point for the attribution analyzer."""
parser = argparse.ArgumentParser(
description="Multi-touch attribution analyzer for marketing campaigns.",
epilog="Example: python attribution_analyzer.py data.json --model linear --format json",
)
parser.add_argument(
"input_file",
help="Path to JSON file containing journey/touchpoint data",
)
parser.add_argument(
"--model",
choices=MODELS,
default=None,
help="Run a specific attribution model (default: run all 5 models)",
)
parser.add_argument(
"--half-life",
type=float,
default=7.0,
help="Half-life in days for time-decay model (default: 7)",
)
parser.add_argument(
"--format",
choices=["json", "text"],
default="text",
dest="output_format",
help="Output format (default: text)",
)
args = parser.parse_args()
# Load input data
try:
with open(args.input_file, "r") as f:
data = json.load(f)
except FileNotFoundError:
print(f"Error: File not found: {args.input_file}", file=sys.stderr)
sys.exit(1)
except json.JSONDecodeError as e:
print(f"Error: Invalid JSON in {args.input_file}: {e}", file=sys.stderr)
sys.exit(1)
journeys = data.get("journeys", [])
if not journeys:
print("Error: No 'journeys' array found in input data.", file=sys.stderr)
sys.exit(1)
# Determine which models to run
models_to_run = [args.model] if args.model else MODELS
# Run models
model_results: Dict[str, Dict[str, float]] = {}
for model_name in models_to_run:
credits = run_model(model_name, journeys, args.half_life)
model_results[model_name] = {ch: round(v, 2) for ch, v in credits.items()}
# Build output
results: Dict[str, Any] = {
"summary": compute_summary(journeys),
"models": model_results,
}
if args.output_format == "json":
print(json.dumps(results, indent=2))
else:
print(format_text(results))
if __name__ == "__main__":
main()
FILE:scripts/campaign_roi_calculator.py
#!/usr/bin/env python3
"""
Campaign ROI Calculator - Comprehensive campaign ROI and performance metrics.
Calculates:
- ROI (Return on Investment)
- ROAS (Return on Ad Spend)
- CPA (Cost per Acquisition/Customer)
- CPL (Cost per Lead)
- CAC (Customer Acquisition Cost)
- CTR (Click-Through Rate)
- CVR (Conversion Rate - Leads to Customers)
Includes industry benchmarking and underperformance flagging.
Usage:
python campaign_roi_calculator.py campaign_data.json
python campaign_roi_calculator.py campaign_data.json --format json
"""
import argparse
import json
import sys
from typing import Any, Dict, List, Optional
# Industry benchmark ranges by channel
# Format: {metric: {channel: (low, target, high)}}
BENCHMARKS: Dict[str, Dict[str, tuple]] = {
"ctr": {
"email": (1.0, 2.5, 5.0),
"paid_search": (1.5, 3.5, 7.0),
"paid_social": (0.5, 1.2, 3.0),
"display": (0.05, 0.1, 0.5),
"organic_search": (1.5, 3.0, 8.0),
"organic_social": (0.5, 1.5, 4.0),
"referral": (1.0, 3.0, 6.0),
"direct": (2.0, 4.0, 8.0),
"default": (0.5, 2.0, 5.0),
},
"roas": {
"email": (30.0, 42.0, 60.0),
"paid_search": (2.0, 4.0, 8.0),
"paid_social": (1.5, 3.0, 6.0),
"display": (0.5, 1.5, 3.0),
"organic_search": (5.0, 10.0, 20.0),
"organic_social": (3.0, 6.0, 12.0),
"referral": (3.0, 5.0, 10.0),
"direct": (4.0, 8.0, 15.0),
"default": (2.0, 4.0, 8.0),
},
"cpa": {
"email": (5.0, 15.0, 40.0),
"paid_search": (20.0, 50.0, 150.0),
"paid_social": (15.0, 40.0, 100.0),
"display": (30.0, 75.0, 200.0),
"organic_search": (5.0, 20.0, 60.0),
"organic_social": (10.0, 30.0, 80.0),
"referral": (10.0, 25.0, 70.0),
"direct": (5.0, 15.0, 50.0),
"default": (15.0, 45.0, 120.0),
},
}
def safe_divide(numerator: float, denominator: float, default: float = 0.0) -> float:
"""Safely divide two numbers, returning default if denominator is zero."""
if denominator == 0:
return default
return numerator / denominator
def get_benchmark(metric: str, channel: str) -> tuple:
"""Get benchmark range for a metric and channel.
Returns:
Tuple of (low, target, high) for the given metric and channel.
"""
metric_benchmarks = BENCHMARKS.get(metric, {})
return metric_benchmarks.get(channel, metric_benchmarks.get("default", (0, 0, 0)))
def assess_performance(value: float, benchmark: tuple, higher_is_better: bool = True) -> str:
"""Assess a metric value against its benchmark range.
Args:
value: The metric value to assess.
benchmark: Tuple of (low, target, high).
higher_is_better: Whether higher values are better (True for CTR, ROAS; False for CPA).
Returns:
Performance assessment string.
"""
low, target, high = benchmark
if higher_is_better:
if value >= high:
return "excellent"
elif value >= target:
return "good"
elif value >= low:
return "below_target"
else:
return "underperforming"
else:
# For cost metrics, lower is better
if value <= low:
return "excellent"
elif value <= target:
return "good"
elif value <= high:
return "below_target"
else:
return "underperforming"
def calculate_campaign_metrics(campaign: Dict[str, Any]) -> Dict[str, Any]:
"""Calculate all ROI metrics for a single campaign.
Args:
campaign: Dict with keys: name, channel, spend, revenue, impressions, clicks, leads, customers.
Returns:
Dict with all calculated metrics, benchmarks, and assessments.
"""
name = campaign.get("name", "Unnamed Campaign")
channel = campaign.get("channel", "default")
spend = campaign.get("spend", 0.0)
revenue = campaign.get("revenue", 0.0)
impressions = campaign.get("impressions", 0)
clicks = campaign.get("clicks", 0)
leads = campaign.get("leads", 0)
customers = campaign.get("customers", 0)
# Core metrics
roi = safe_divide(revenue - spend, spend) * 100
roas = safe_divide(revenue, spend)
cpa = safe_divide(spend, customers) if customers > 0 else None
cpl = safe_divide(spend, leads) if leads > 0 else None
cac = safe_divide(spend, customers) if customers > 0 else None
ctr = safe_divide(clicks, impressions) * 100 if impressions > 0 else None
cvr = safe_divide(customers, leads) * 100 if leads > 0 else None
cpc = safe_divide(spend, clicks) if clicks > 0 else None
cpm = safe_divide(spend, impressions) * 1000 if impressions > 0 else None
lead_conversion_rate = safe_divide(leads, clicks) * 100 if clicks > 0 else None
# Profit
profit = revenue - spend
# Benchmark assessments
assessments: Dict[str, Any] = {}
flags: List[str] = []
if ctr is not None:
benchmark = get_benchmark("ctr", channel)
assessment = assess_performance(ctr, benchmark, higher_is_better=True)
assessments["ctr"] = {
"value": round(ctr, 2),
"benchmark_range": {"low": benchmark[0], "target": benchmark[1], "high": benchmark[2]},
"assessment": assessment,
}
if assessment == "underperforming":
flags.append(f"CTR ({ctr:.2f}%) is below industry low ({benchmark[0]}%) for {channel}")
if roas > 0:
benchmark = get_benchmark("roas", channel)
assessment = assess_performance(roas, benchmark, higher_is_better=True)
assessments["roas"] = {
"value": round(roas, 2),
"benchmark_range": {"low": benchmark[0], "target": benchmark[1], "high": benchmark[2]},
"assessment": assessment,
}
if assessment == "underperforming":
flags.append(f"ROAS ({roas:.2f}x) is below industry low ({benchmark[0]}x) for {channel}")
if cpa is not None:
benchmark = get_benchmark("cpa", channel)
assessment = assess_performance(cpa, benchmark, higher_is_better=False)
assessments["cpa"] = {
"value": round(cpa, 2),
"benchmark_range": {"low": benchmark[0], "target": benchmark[1], "high": benchmark[2]},
"assessment": assessment,
}
if assessment == "underperforming":
flags.append(f"CPA (.2f) exceeds industry high (.2f) for {channel}")
if profit < 0:
flags.append(f"Campaign is unprofitable: ,.2f net loss")
# Recommendations
recommendations: List[str] = []
if ctr is not None and assessments.get("ctr", {}).get("assessment") in ("below_target", "underperforming"):
recommendations.append("Improve ad creative and targeting to increase CTR")
if assessments.get("roas", {}).get("assessment") in ("below_target", "underperforming"):
recommendations.append("Review targeting and bid strategy to improve ROAS")
if assessments.get("cpa", {}).get("assessment") in ("below_target", "underperforming"):
recommendations.append("Optimize landing pages and conversion flow to reduce CPA")
if cvr is not None and cvr < 10:
recommendations.append("Lead-to-customer conversion is low; review sales process and lead quality")
if lead_conversion_rate is not None and lead_conversion_rate < 2:
recommendations.append("Click-to-lead rate is low; improve landing page relevance and form experience")
if profit > 0 and assessments.get("roas", {}).get("assessment") in ("good", "excellent"):
recommendations.append("Campaign performing well; consider scaling budget")
return {
"name": name,
"channel": channel,
"metrics": {
"spend": round(spend, 2),
"revenue": round(revenue, 2),
"profit": round(profit, 2),
"roi_pct": round(roi, 2),
"roas": round(roas, 2),
"cpa": round(cpa, 2) if cpa is not None else None,
"cpl": round(cpl, 2) if cpl is not None else None,
"cac": round(cac, 2) if cac is not None else None,
"ctr_pct": round(ctr, 2) if ctr is not None else None,
"cvr_pct": round(cvr, 2) if cvr is not None else None,
"cpc": round(cpc, 2) if cpc is not None else None,
"cpm": round(cpm, 2) if cpm is not None else None,
"lead_conversion_rate_pct": round(lead_conversion_rate, 2) if lead_conversion_rate is not None else None,
"impressions": impressions,
"clicks": clicks,
"leads": leads,
"customers": customers,
},
"assessments": assessments,
"flags": flags,
"recommendations": recommendations,
}
def calculate_portfolio_summary(campaign_results: List[Dict[str, Any]]) -> Dict[str, Any]:
"""Calculate aggregate metrics across all campaigns.
Args:
campaign_results: List of individual campaign result dicts.
Returns:
Portfolio-level summary with totals and weighted averages.
"""
total_spend = sum(c["metrics"]["spend"] for c in campaign_results)
total_revenue = sum(c["metrics"]["revenue"] for c in campaign_results)
total_impressions = sum(c["metrics"]["impressions"] for c in campaign_results)
total_clicks = sum(c["metrics"]["clicks"] for c in campaign_results)
total_leads = sum(c["metrics"]["leads"] for c in campaign_results)
total_customers = sum(c["metrics"]["customers"] for c in campaign_results)
total_profit = total_revenue - total_spend
underperforming = [c["name"] for c in campaign_results if c["flags"]]
top_performers = sorted(
campaign_results,
key=lambda c: c["metrics"]["roi_pct"],
reverse=True,
)
# Channel breakdown
channel_totals: Dict[str, Dict[str, float]] = {}
for c in campaign_results:
ch = c["channel"]
if ch not in channel_totals:
channel_totals[ch] = {"spend": 0, "revenue": 0, "leads": 0, "customers": 0}
channel_totals[ch]["spend"] += c["metrics"]["spend"]
channel_totals[ch]["revenue"] += c["metrics"]["revenue"]
channel_totals[ch]["leads"] += c["metrics"]["leads"]
channel_totals[ch]["customers"] += c["metrics"]["customers"]
channel_summary = {}
for ch, totals in channel_totals.items():
channel_summary[ch] = {
"spend": round(totals["spend"], 2),
"revenue": round(totals["revenue"], 2),
"roi_pct": round(safe_divide(totals["revenue"] - totals["spend"], totals["spend"]) * 100, 2),
"roas": round(safe_divide(totals["revenue"], totals["spend"]), 2),
"leads": int(totals["leads"]),
"customers": int(totals["customers"]),
}
return {
"total_campaigns": len(campaign_results),
"total_spend": round(total_spend, 2),
"total_revenue": round(total_revenue, 2),
"total_profit": round(total_profit, 2),
"portfolio_roi_pct": round(safe_divide(total_profit, total_spend) * 100, 2),
"portfolio_roas": round(safe_divide(total_revenue, total_spend), 2),
"total_impressions": total_impressions,
"total_clicks": total_clicks,
"total_leads": total_leads,
"total_customers": total_customers,
"blended_ctr_pct": round(safe_divide(total_clicks, total_impressions) * 100, 2),
"blended_cpl": round(safe_divide(total_spend, total_leads), 2) if total_leads > 0 else None,
"blended_cpa": round(safe_divide(total_spend, total_customers), 2) if total_customers > 0 else None,
"underperforming_campaigns": underperforming,
"top_performer": top_performers[0]["name"] if top_performers else None,
"channel_summary": channel_summary,
}
def format_text(results: Dict[str, Any]) -> str:
"""Format full results as human-readable text."""
lines: List[str] = []
lines.append("=" * 70)
lines.append("CAMPAIGN ROI ANALYSIS")
lines.append("=" * 70)
# Portfolio summary
summary = results["portfolio_summary"]
lines.append("")
lines.append("PORTFOLIO SUMMARY")
lines.append(f" Total Campaigns: {summary['total_campaigns']}")
lines.append(f" Total Spend: >12,.2f")
lines.append(f" Total Revenue: >12,.2f")
lines.append(f" Total Profit: >12,.2f")
lines.append(f" Portfolio ROI: {summary['portfolio_roi_pct']}%")
lines.append(f" Portfolio ROAS: {summary['portfolio_roas']}x")
lines.append(f" Blended CTR: {summary['blended_ctr_pct']}%")
if summary["blended_cpl"] is not None:
lines.append(f" Blended CPL: >12,.2f")
if summary["blended_cpa"] is not None:
lines.append(f" Blended CPA: >12,.2f")
if summary["top_performer"]:
lines.append(f" Top Performer: {summary['top_performer']}")
if summary["underperforming_campaigns"]:
lines.append(f" Flagged: {', '.join(summary['underperforming_campaigns'])}")
# Channel summary
if summary["channel_summary"]:
lines.append("")
lines.append("-" * 70)
lines.append("CHANNEL SUMMARY")
lines.append(f" {'Channel':<20} {'Spend':>12} {'Revenue':>12} {'ROI':>10} {'ROAS':>8}")
lines.append(f" {'-'*20} {'-'*12} {'-'*12} {'-'*10} {'-'*8}")
for ch, cs in sorted(summary["channel_summary"].items()):
lines.append(
f" {ch:<20} >10,.2f >10,.2f "
f"{cs['roi_pct']:>8.1f}% {cs['roas']:>6.2f}x"
)
# Individual campaigns
for campaign in results["campaigns"]:
lines.append("")
lines.append("-" * 70)
lines.append(f"CAMPAIGN: {campaign['name']}")
lines.append(f"Channel: {campaign['channel']}")
lines.append("-" * 70)
m = campaign["metrics"]
lines.append(f" {'Metric':<25} {'Value':>15}")
lines.append(f" {'-'*25} {'-'*15}")
lines.append(f" {'Spend':<25} >13,.2f")
lines.append(f" {'Revenue':<25} >13,.2f")
lines.append(f" {'Profit':<25} >13,.2f")
lines.append(f" {'ROI':<25} {m['roi_pct']:>13.2f}%")
lines.append(f" {'ROAS':<25} {m['roas']:>13.2f}x")
if m["cpa"] is not None:
lines.append(f" {'CPA':<25} >13,.2f")
if m["cpl"] is not None:
lines.append(f" {'CPL':<25} >13,.2f")
if m["cac"] is not None:
lines.append(f" {'CAC':<25} >13,.2f")
if m["ctr_pct"] is not None:
lines.append(f" {'CTR':<25} {m['ctr_pct']:>13.2f}%")
if m["cpc"] is not None:
lines.append(f" {'CPC':<25} >13,.2f")
if m["cpm"] is not None:
lines.append(f" {'CPM':<25} >13,.2f")
if m["cvr_pct"] is not None:
lines.append(f" {'Lead-to-Customer CVR':<25} {m['cvr_pct']:>13.2f}%")
if m["lead_conversion_rate_pct"] is not None:
lines.append(f" {'Click-to-Lead Rate':<25} {m['lead_conversion_rate_pct']:>13.2f}%")
# Benchmark assessments
if campaign["assessments"]:
lines.append("")
lines.append(" BENCHMARK ASSESSMENT")
for metric_name, a in campaign["assessments"].items():
br = a["benchmark_range"]
status = a["assessment"].upper().replace("_", " ")
lines.append(
f" {metric_name.upper()}: {a['value']} "
f"[low={br['low']}, target={br['target']}, high={br['high']}] "
f"-> {status}"
)
# Flags
if campaign["flags"]:
lines.append("")
lines.append(" WARNING FLAGS")
for flag in campaign["flags"]:
lines.append(f" ! {flag}")
# Recommendations
if campaign["recommendations"]:
lines.append("")
lines.append(" RECOMMENDATIONS")
for i, rec in enumerate(campaign["recommendations"], 1):
lines.append(f" {i}. {rec}")
lines.append("")
return "\n".join(lines)
def main() -> None:
"""Main entry point for the campaign ROI calculator."""
parser = argparse.ArgumentParser(
description="Calculate campaign ROI, ROAS, CPA, CPL, CAC with industry benchmarking.",
epilog="Example: python campaign_roi_calculator.py campaigns.json --format json",
)
parser.add_argument(
"input_file",
help="Path to JSON file containing campaign data",
)
parser.add_argument(
"--format",
choices=["json", "text"],
default="text",
dest="output_format",
help="Output format (default: text)",
)
args = parser.parse_args()
# Load input data
try:
with open(args.input_file, "r") as f:
data = json.load(f)
except FileNotFoundError:
print(f"Error: File not found: {args.input_file}", file=sys.stderr)
sys.exit(1)
except json.JSONDecodeError as e:
print(f"Error: Invalid JSON in {args.input_file}: {e}", file=sys.stderr)
sys.exit(1)
campaigns = data.get("campaigns", [])
if not campaigns:
print("Error: No 'campaigns' array found in input data.", file=sys.stderr)
sys.exit(1)
# Calculate metrics for each campaign
campaign_results = [calculate_campaign_metrics(c) for c in campaigns]
# Calculate portfolio summary
portfolio_summary = calculate_portfolio_summary(campaign_results)
results = {
"portfolio_summary": portfolio_summary,
"campaigns": campaign_results,
}
if args.output_format == "json":
print(json.dumps(results, indent=2))
else:
print(format_text(results))
if __name__ == "__main__":
main()
FILE:scripts/funnel_analyzer.py
#!/usr/bin/env python3
"""
Funnel Analyzer - Conversion funnel analysis with bottleneck detection.
Analyzes marketing/sales funnels to identify:
- Stage-to-stage conversion rates and drop-off percentages
- Biggest bottleneck (largest absolute and relative drops)
- Overall funnel conversion rate
- Segment comparison when multiple segments are provided
Usage:
python funnel_analyzer.py funnel_data.json
python funnel_analyzer.py funnel_data.json --format json
"""
import argparse
import json
import sys
from typing import Any, Dict, List, Optional
def safe_divide(numerator: float, denominator: float, default: float = 0.0) -> float:
"""Safely divide two numbers, returning default if denominator is zero."""
if denominator == 0:
return default
return numerator / denominator
def analyze_funnel(stages: List[str], counts: List[int]) -> Dict[str, Any]:
"""Analyze a single funnel and return stage-by-stage metrics.
Args:
stages: Ordered list of funnel stage names (top to bottom).
counts: Corresponding counts at each stage.
Returns:
Dictionary with stage metrics, bottleneck info, and overall conversion.
"""
if len(stages) != len(counts):
raise ValueError("Number of stages must match number of counts.")
if not stages:
raise ValueError("Funnel must have at least one stage.")
stage_metrics: List[Dict[str, Any]] = []
max_dropoff_abs = 0
max_dropoff_rel = 0.0
bottleneck_abs: Optional[str] = None
bottleneck_rel: Optional[str] = None
for i, (stage, count) in enumerate(zip(stages, counts)):
metric: Dict[str, Any] = {
"stage": stage,
"count": count,
"cumulative_conversion": round(safe_divide(count, counts[0]) * 100, 2),
}
if i > 0:
prev_count = counts[i - 1]
dropoff = prev_count - count
conversion_rate = safe_divide(count, prev_count) * 100
dropoff_rate = 100 - conversion_rate
metric["from_previous"] = stages[i - 1]
metric["conversion_rate"] = round(conversion_rate, 2)
metric["dropoff_count"] = dropoff
metric["dropoff_rate"] = round(dropoff_rate, 2)
# Track biggest absolute drop-off
if dropoff > max_dropoff_abs:
max_dropoff_abs = dropoff
bottleneck_abs = f"{stages[i-1]} -> {stage}"
# Track biggest relative drop-off
if dropoff_rate > max_dropoff_rel:
max_dropoff_rel = dropoff_rate
bottleneck_rel = f"{stages[i-1]} -> {stage}"
else:
metric["conversion_rate"] = 100.0
metric["dropoff_count"] = 0
metric["dropoff_rate"] = 0.0
stage_metrics.append(metric)
overall_conversion = safe_divide(counts[-1], counts[0]) * 100
return {
"stage_metrics": stage_metrics,
"overall_conversion_rate": round(overall_conversion, 2),
"total_entries": counts[0],
"total_conversions": counts[-1],
"total_lost": counts[0] - counts[-1],
"bottleneck_absolute": {
"transition": bottleneck_abs,
"dropoff_count": max_dropoff_abs,
},
"bottleneck_relative": {
"transition": bottleneck_rel,
"dropoff_rate": round(max_dropoff_rel, 2),
},
}
def compare_segments(segments: Dict[str, Dict[str, Any]], stages: List[str]) -> Dict[str, Any]:
"""Compare funnel performance across segments.
Args:
segments: Dict mapping segment name to {"counts": [...]}.
stages: Shared stage names for all segments.
Returns:
Comparison data with per-segment analysis and relative rankings.
"""
segment_results: Dict[str, Dict[str, Any]] = {}
for seg_name, seg_data in segments.items():
counts = seg_data.get("counts", [])
if len(counts) != len(stages):
raise ValueError(
f"Segment '{seg_name}' has {len(counts)} counts but {len(stages)} stages."
)
segment_results[seg_name] = analyze_funnel(stages, counts)
# Rank segments by overall conversion rate
ranked = sorted(
segment_results.items(),
key=lambda x: x[1]["overall_conversion_rate"],
reverse=True,
)
rankings = [
{
"rank": i + 1,
"segment": name,
"overall_conversion_rate": result["overall_conversion_rate"],
"total_entries": result["total_entries"],
"total_conversions": result["total_conversions"],
}
for i, (name, result) in enumerate(ranked)
]
# Stage-by-stage comparison
stage_comparison: List[Dict[str, Any]] = []
for i, stage in enumerate(stages):
stage_data: Dict[str, Any] = {"stage": stage}
for seg_name in segments:
metrics = segment_results[seg_name]["stage_metrics"][i]
stage_data[seg_name] = {
"count": metrics["count"],
"conversion_rate": metrics["conversion_rate"],
}
stage_comparison.append(stage_data)
return {
"segment_results": segment_results,
"rankings": rankings,
"stage_comparison": stage_comparison,
}
def format_single_funnel_text(analysis: Dict[str, Any], title: str = "FUNNEL") -> str:
"""Format a single funnel analysis as human-readable text."""
lines: List[str] = []
lines.append(f" {title}")
lines.append(f" {'='*60}")
lines.append(f" Total Entries: {analysis['total_entries']:,}")
lines.append(f" Total Conversions: {analysis['total_conversions']:,}")
lines.append(f" Total Lost: {analysis['total_lost']:,}")
lines.append(f" Overall Conversion: {analysis['overall_conversion_rate']}%")
lines.append("")
lines.append(f" {'Stage':<20} {'Count':>10} {'Conv Rate':>12} {'Drop-off':>12} {'Cumulative':>12}")
lines.append(f" {'-'*20} {'-'*10} {'-'*12} {'-'*12} {'-'*12}")
for m in analysis["stage_metrics"]:
stage = m["stage"]
count = m["count"]
conv = f"{m['conversion_rate']:.1f}%"
drop = f"-{m['dropoff_count']:,} ({m['dropoff_rate']:.1f}%)" if m["dropoff_count"] > 0 else "-"
cumul = f"{m['cumulative_conversion']:.1f}%"
lines.append(f" {stage:<20} {count:>10,} {conv:>12} {drop:>12} {cumul:>12}")
lines.append("")
bn_abs = analysis["bottleneck_absolute"]
bn_rel = analysis["bottleneck_relative"]
lines.append(f" BOTTLENECK (Absolute): {bn_abs['transition']} (lost {bn_abs['dropoff_count']:,})")
lines.append(f" BOTTLENECK (Relative): {bn_rel['transition']} ({bn_rel['dropoff_rate']}% drop-off)")
return "\n".join(lines)
def format_text(results: Dict[str, Any]) -> str:
"""Format full results as human-readable text output."""
lines: List[str] = []
lines.append("=" * 70)
lines.append("FUNNEL CONVERSION ANALYSIS")
lines.append("=" * 70)
if "stage_comparison" in results:
# Multi-segment output
lines.append("")
lines.append("SEGMENT RANKINGS")
lines.append(f" {'Rank':>4} {'Segment':<25} {'Conversion':>12} {'Entries':>10} {'Conversions':>12}")
lines.append(f" {'-'*4} {'-'*25} {'-'*12} {'-'*10} {'-'*12}")
for r in results["rankings"]:
lines.append(
f" {r['rank']:>4} {r['segment']:<25} {r['overall_conversion_rate']:>11.2f}% "
f"{r['total_entries']:>10,} {r['total_conversions']:>12,}"
)
lines.append("")
for seg_name, seg_result in results["segment_results"].items():
lines.append("")
lines.append(format_single_funnel_text(seg_result, title=f"SEGMENT: {seg_name.upper()}"))
# Stage comparison table
lines.append("")
lines.append("-" * 70)
lines.append("STAGE-BY-STAGE COMPARISON")
lines.append("-" * 70)
seg_names = list(results["segment_results"].keys())
header = f" {'Stage':<20}"
for sn in seg_names:
header += f" {sn:>20}"
lines.append(header)
lines.append(f" {'-'*20}" + f" {'-'*20}" * len(seg_names))
for sc in results["stage_comparison"]:
row = f" {sc['stage']:<20}"
for sn in seg_names:
data = sc[sn]
row += f" {data['count']:>8,} ({data['conversion_rate']:>5.1f}%)"
lines.append(row)
else:
# Single funnel output
lines.append("")
lines.append(format_single_funnel_text(results))
lines.append("")
return "\n".join(lines)
def main() -> None:
"""Main entry point for the funnel analyzer."""
parser = argparse.ArgumentParser(
description="Analyze conversion funnels with bottleneck detection and segment comparison.",
epilog="Example: python funnel_analyzer.py funnel_data.json --format json",
)
parser.add_argument(
"input_file",
help="Path to JSON file containing funnel data",
)
parser.add_argument(
"--format",
choices=["json", "text"],
default="text",
dest="output_format",
help="Output format (default: text)",
)
args = parser.parse_args()
# Load input data
try:
with open(args.input_file, "r") as f:
data = json.load(f)
except FileNotFoundError:
print(f"Error: File not found: {args.input_file}", file=sys.stderr)
sys.exit(1)
except json.JSONDecodeError as e:
print(f"Error: Invalid JSON in {args.input_file}: {e}", file=sys.stderr)
sys.exit(1)
# Determine mode: single funnel vs. segment comparison
if "segments" in data:
# Multi-segment mode
stages = data.get("funnel", {}).get("stages", data.get("stages", []))
if not stages:
print("Error: 'stages' list required for segment comparison.", file=sys.stderr)
sys.exit(1)
segments = data["segments"]
if not segments:
print("Error: 'segments' dict is empty.", file=sys.stderr)
sys.exit(1)
results = compare_segments(segments, stages)
elif "funnel" in data:
# Single funnel mode
funnel = data["funnel"]
stages = funnel.get("stages", [])
counts = funnel.get("counts", [])
if not stages or not counts:
print("Error: 'funnel' must contain 'stages' and 'counts' arrays.", file=sys.stderr)
sys.exit(1)
results = analyze_funnel(stages, counts)
else:
print("Error: Input must contain 'funnel' or 'segments' key.", file=sys.stderr)
sys.exit(1)
if args.output_format == "json":
print(json.dumps(results, indent=2))
else:
print(format_text(results))
if __name__ == "__main__":
main()
Quản lý CAPA cho QMS thiết bị y tế: phân tích nguyên nhân gốc, hành động khắc phục và kiểm chứng hiệu quả.
---
name: "capa-officer"
description: CAPA system management for medical device QMS. Covers root cause analysis, corrective action planning, effectiveness verification, and CAPA metrics. Use for CAPA investigations, 5-Why analysis, fishbone diagrams, root cause determination, corrective action tracking, effectiveness verification, or CAPA program optimization.
triggers:
- CAPA investigation
- root cause analysis
- 5 Why analysis
- fishbone diagram
- corrective action
- preventive action
- effectiveness verification
- CAPA metrics
- nonconformance investigation
- quality issue investigation
- CAPA tracking
- audit finding CAPA
---
# CAPA Officer
Corrective and Preventive Action (CAPA) management within Quality Management Systems, focusing on systematic root cause analysis, action implementation, and effectiveness verification.
---
## Table of Contents
- [CAPA Investigation Workflow](#capa-investigation-workflow)
- [Root Cause Analysis](#root-cause-analysis)
- [Corrective Action Planning](#corrective-action-planning)
- [Effectiveness Verification](#effectiveness-verification)
- [CAPA Metrics and Reporting](#capa-metrics-and-reporting)
- [Reference Documentation](#reference-documentation)
- [Tools](#tools)
---
## CAPA Investigation Workflow
Conduct systematic CAPA investigation from initiation through closure:
1. Document trigger event with objective evidence
2. Assess significance and determine CAPA necessity
3. Form investigation team with relevant expertise
4. Collect data and evidence systematically
5. Select and apply appropriate RCA methodology
6. Identify root cause(s) with supporting evidence
7. Develop corrective and preventive actions
8. **Validation:** Root cause explains all symptoms; if eliminated, problem would not recur
### CAPA Necessity Determination
| Trigger Type | CAPA Required | Criteria |
|--------------|---------------|----------|
| Customer complaint (safety) | Yes | Any complaint involving patient/user safety |
| Customer complaint (quality) | Evaluate | Based on severity and frequency |
| Internal audit finding (Major) | Yes | Systematic failure or absence of element |
| Internal audit finding (Minor) | Recommended | Isolated lapse or partial implementation |
| Nonconformance (recurring) | Yes | Same NC type occurring 3+ times |
| Nonconformance (isolated) | Evaluate | Based on severity and risk |
| External audit finding | Yes | All Major and Minor findings |
| Trend analysis | Evaluate | Based on trend significance |
### Investigation Team Composition
| CAPA Severity | Required Team Members |
|---------------|----------------------|
| Critical | CAPA Officer, Process Owner, QA Manager, Subject Matter Expert, Management Rep |
| Major | CAPA Officer, Process Owner, Subject Matter Expert |
| Minor | CAPA Officer, Process Owner |
### Evidence Collection Checklist
- [ ] Problem description with specific details (what, where, when, who, how much)
- [ ] Timeline of events leading to issue
- [ ] Relevant records and documentation
- [ ] Interview notes from involved personnel
- [ ] Photos or physical evidence (if applicable)
- [ ] Related complaints, NCs, or previous CAPAs
- [ ] Process parameters and specifications
---
## Root Cause Analysis
Select and apply appropriate RCA methodology based on problem characteristics.
### RCA Method Selection Decision Tree
```
Is the issue safety-critical or involves system reliability?
├── Yes → Use FAULT TREE ANALYSIS
└── No → Is human error the suspected primary cause?
├── Yes → Use HUMAN FACTORS ANALYSIS
└── No → How many potential contributing factors?
├── 1-2 factors (linear causation) → Use 5 WHY ANALYSIS
├── 3-6 factors (complex, systemic) → Use FISHBONE DIAGRAM
└── Unknown/proactive assessment → Use FMEA
```
### 5 Why Analysis
Use when: Single-cause issues with linear causation, process deviations with clear failure point.
**Template:**
```
PROBLEM: [Clear, specific statement]
WHY 1: Why did [problem] occur?
BECAUSE: [First-level cause]
EVIDENCE: [Supporting data]
WHY 2: Why did [first-level cause] occur?
BECAUSE: [Second-level cause]
EVIDENCE: [Supporting data]
WHY 3: Why did [second-level cause] occur?
BECAUSE: [Third-level cause]
EVIDENCE: [Supporting data]
WHY 4: Why did [third-level cause] occur?
BECAUSE: [Fourth-level cause]
EVIDENCE: [Supporting data]
WHY 5: Why did [fourth-level cause] occur?
BECAUSE: [Root cause]
EVIDENCE: [Supporting data]
```
**Example - Calibration Overdue:**
```
PROBLEM: pH meter (EQ-042) found 2 months overdue for calibration
WHY 1: Why was calibration overdue?
BECAUSE: Equipment was not on calibration schedule
EVIDENCE: Calibration schedule reviewed, EQ-042 not listed
WHY 2: Why was it not on the schedule?
BECAUSE: Schedule not updated when equipment was purchased
EVIDENCE: Purchase date 2023-06-15, schedule dated 2023-01-01
WHY 3: Why was the schedule not updated?
BECAUSE: No process requires schedule update at equipment purchase
EVIDENCE: SOP-EQ-001 reviewed, no such requirement
WHY 4: Why is there no such requirement?
BECAUSE: Procedure written before equipment tracking was centralized
EVIDENCE: SOP last revised 2019, equipment system implemented 2021
WHY 5: Why has procedure not been updated?
BECAUSE: Periodic review did not assess compatibility with new systems
EVIDENCE: No review against new equipment system documented
ROOT CAUSE: Procedure review process does not assess compatibility
with organizational systems implemented after original procedure creation.
```
### Fishbone Diagram Categories (6M)
| Category | Focus Areas | Typical Causes |
|----------|-------------|----------------|
| Man (People) | Training, competency, workload | Skill gaps, fatigue, communication |
| Machine (Equipment) | Calibration, maintenance, age | Wear, malfunction, inadequate capacity |
| Method (Process) | Procedures, work instructions | Unclear steps, missing controls |
| Material | Specifications, suppliers, storage | Out-of-spec, degradation, contamination |
| Measurement | Calibration, methods, interpretation | Instrument error, wrong method |
| Mother Nature | Temperature, humidity, cleanliness | Environmental excursions |
See `references/rca-methodologies.md` for complete method details and templates.
### Root Cause Validation
Before proceeding to action planning, validate root cause:
- [ ] Root cause can be verified with objective evidence
- [ ] If root cause is eliminated, problem would not recur
- [ ] Root cause is within organizational control
- [ ] Root cause explains all observed symptoms
- [ ] No other significant causes remain unaddressed
---
## Corrective Action Planning
Develop effective actions addressing identified root causes:
1. Define immediate containment actions
2. Develop corrective actions targeting root cause
3. Identify preventive actions for similar processes
4. Assign responsibilities and resources
5. Establish timeline with milestones
6. Define success criteria and verification method
7. Document in CAPA action plan
8. **Validation:** Actions directly address root cause; success criteria are measurable
### Action Types
| Type | Purpose | Timeline | Example |
|------|---------|----------|---------|
| Containment | Stop immediate impact | 24-72 hours | Quarantine affected product |
| Correction | Fix the specific occurrence | 1-2 weeks | Rework or replace affected items |
| Corrective | Eliminate root cause | 30-90 days | Revise procedure, add controls |
| Preventive | Prevent in other areas | 60-120 days | Extend solution to similar processes |
### Action Plan Components
```
ACTION PLAN TEMPLATE
CAPA Number: [CAPA-XXXX]
Root Cause: [Identified root cause]
ACTION 1: [Specific action description]
- Type: [ ] Containment [ ] Correction [ ] Corrective [ ] Preventive
- Responsible: [Name, Title]
- Due Date: [YYYY-MM-DD]
- Resources: [Required resources]
- Success Criteria: [Measurable outcome]
- Verification Method: [How success will be verified]
ACTION 2: [Specific action description]
...
IMPLEMENTATION TIMELINE:
Week 1: [Milestone]
Week 2: [Milestone]
Week 4: [Milestone]
Week 8: [Milestone]
APPROVAL:
CAPA Owner: _____________ Date: _______
Process Owner: _____________ Date: _______
QA Manager: _____________ Date: _______
```
### Action Effectiveness Indicators
| Indicator | Target | Red Flag |
|-----------|--------|----------|
| Action scope | Addresses root cause completely | Treats only symptoms |
| Specificity | Measurable deliverables | Vague commitments |
| Timeline | Aggressive but achievable | No due dates or unrealistic |
| Resources | Identified and allocated | Not specified |
| Sustainability | Permanent solution | Temporary fix |
---
## Effectiveness Verification
Verify corrective actions achieved intended results:
1. Allow adequate implementation period (minimum 30-90 days)
2. Collect post-implementation data
3. Compare to pre-implementation baseline
4. Evaluate against success criteria
5. Verify no recurrence during verification period
6. Document verification evidence
7. Determine CAPA effectiveness
8. **Validation:** All criteria met with objective evidence; no recurrence observed
### Verification Timeline Guidelines
| CAPA Severity | Wait Period | Verification Window |
|---------------|-------------|---------------------|
| Critical | 30 days | 30-90 days post-implementation |
| Major | 60 days | 60-180 days post-implementation |
| Minor | 90 days | 90-365 days post-implementation |
### Verification Methods
| Method | Use When | Evidence Required |
|--------|----------|-------------------|
| Data trend analysis | Quantifiable issues | Pre/post comparison, trend charts |
| Process audit | Procedure compliance issues | Audit checklist, interview notes |
| Record review | Documentation issues | Sample records, compliance rate |
| Testing/inspection | Product quality issues | Test results, pass/fail data |
| Interview/observation | Training issues | Interview notes, observation records |
### Effectiveness Determination
```
Did recurrence occur during verification period?
├── Yes → CAPA INEFFECTIVE (re-investigate root cause)
└── No → Were all effectiveness criteria met?
├── Yes → CAPA EFFECTIVE (proceed to closure)
└── No → Extent of gap?
├── Minor gap → Extend verification or accept with justification
└── Significant gap → CAPA INEFFECTIVE (revise actions)
```
See `references/effectiveness-verification-guide.md` for detailed procedures.
---
## CAPA Metrics and Reporting
Monitor CAPA program performance through key indicators.
### Key Performance Indicators
| Metric | Target | Calculation |
|--------|--------|-------------|
| CAPA cycle time | <60 days average | (Close Date - Open Date) / Number of CAPAs |
| Overdue rate | <10% | Overdue CAPAs / Total Open CAPAs |
| First-time effectiveness | >90% | Effective on first verification / Total verified |
| Recurrence rate | <5% | Recurred issues / Total closed CAPAs |
| Investigation quality | 100% root cause validated | Root causes validated / Total CAPAs |
### Aging Analysis Categories
| Age Bucket | Status | Action Required |
|------------|--------|-----------------|
| 0-30 days | On track | Monitor progress |
| 31-60 days | Monitor | Review for delays |
| 61-90 days | Warning | Escalate to management |
| >90 days | Critical | Management intervention required |
### Management Review Inputs
Monthly CAPA status report includes:
- Open CAPA count by severity and status
- Overdue CAPA list with owners
- Cycle time trends
- Effectiveness rate trends
- Source analysis (complaints, audits, NCs)
- Recommendations for improvement
---
## Reference Documentation
### Root Cause Analysis Methodologies
`references/rca-methodologies.md` contains:
- Method selection decision tree
- 5 Why analysis template and example
- Fishbone diagram categories and template
- Fault Tree Analysis for safety-critical issues
- Human Factors Analysis for people-related causes
- FMEA for proactive risk assessment
- Hybrid approach guidance
### Effectiveness Verification Guide
`references/effectiveness-verification-guide.md` contains:
- Verification planning requirements
- Verification method selection
- Effectiveness criteria definition (SMART)
- Closure requirements by severity
- Ineffective CAPA process
- Documentation templates
---
## Tools
### CAPA Tracker
```bash
# Generate CAPA status report
python scripts/capa_tracker.py --capas capas.json
# Interactive mode for manual entry
python scripts/capa_tracker.py --interactive
# JSON output for integration
python scripts/capa_tracker.py --capas capas.json --output json
# Generate sample data file
python scripts/capa_tracker.py --sample > sample_capas.json
```
Calculates and reports:
- Summary metrics (open, closed, overdue, cycle time, effectiveness)
- Status distribution
- Severity and source analysis
- Aging report by time bucket
- Overdue CAPA list
- Actionable recommendations
### Sample CAPA Input
```json
{
"capas": [
{
"capa_number": "CAPA-2024-001",
"title": "Calibration overdue for pH meter",
"description": "pH meter EQ-042 found 2 months overdue",
"source": "AUDIT",
"severity": "MAJOR",
"status": "VERIFICATION",
"open_date": "2024-06-15",
"target_date": "2024-08-15",
"owner": "J. Smith",
"root_cause": "Procedure review gap",
"corrective_action": "Updated SOP-EQ-001"
}
]
}
```
---
## Regulatory Requirements
### ISO 13485:2016 Clause 8.5
| Sub-clause | Requirement | Key Activities |
|------------|-------------|----------------|
| 8.5.2 Corrective Action | Eliminate cause of nonconformity | NC review, cause determination, action evaluation, implementation, effectiveness review |
| 8.5.3 Preventive Action | Eliminate potential nonconformity | Trend analysis, cause determination, action evaluation, implementation, effectiveness review |
### FDA 21 CFR 820.100
Required CAPA elements:
- Procedures for implementing corrective and preventive action
- Analyzing quality data sources (complaints, NCs, audits, service records)
- Investigating cause of nonconformities
- Identifying actions needed to correct and prevent recurrence
- Verifying actions are effective and do not adversely affect device
- Submitting relevant information for management review
### Common FDA 483 Observations
| Observation | Root Cause Pattern |
|-------------|-------------------|
| CAPA not initiated for recurring issue | Trend analysis not performed |
| Root cause analysis superficial | Inadequate investigation training |
| Effectiveness not verified | No verification procedure |
| Actions do not address root cause | Symptom treatment vs. cause elimination |
FILE:references/effectiveness-verification-guide.md
# Effectiveness Verification Guide
CAPA effectiveness assessment procedures, verification methods, and closure criteria.
---
## Table of Contents
- [Verification Planning](#verification-planning)
- [Verification Methods](#verification-methods)
- [Effectiveness Criteria](#effectiveness-criteria)
- [Closure Requirements](#closure-requirements)
- [Ineffective CAPA Process](#ineffective-capa-process)
- [Documentation Templates](#documentation-templates)
---
## Verification Planning
### When to Plan Verification
Verification planning must occur BEFORE corrective action implementation:
| Stage | Planning Activity | Owner |
|-------|-------------------|-------|
| CAPA Initiation | Define preliminary verification approach | CAPA Owner |
| Root Cause Analysis | Refine criteria based on root cause | Investigation Team |
| Action Planning | Finalize verification method and timeline | CAPA Owner |
| Implementation | Schedule verification activities | Quality Assurance |
### Verification Timeline Guidelines
| CAPA Severity | Minimum Wait Period | Verification Window |
|---------------|---------------------|---------------------|
| Critical (Safety) | 30 days | 30-90 days post-implementation |
| Major | 60 days | 60-180 days post-implementation |
| Minor | 90 days | 90-365 days post-implementation |
**Rationale**: Waiting period ensures sufficient data collection and accounts for process variation.
### Verification Plan Components
```
VERIFICATION PLAN TEMPLATE
CAPA Number: [CAPA-XXXX]
Problem Statement: [Original issue]
Root Cause: [Identified root cause]
Corrective Action: [Implemented action]
VERIFICATION METHOD:
[ ] Data Trend Analysis
[ ] Process Audit
[ ] Record Review
[ ] Testing/Inspection
[ ] Interview/Observation
[ ] Multiple Methods (specify)
EFFECTIVENESS CRITERIA:
1. [Measurable criterion 1]
2. [Measurable criterion 2]
3. [Measurable criterion 3]
SUCCESS THRESHOLD:
- [Quantitative threshold, e.g., "Zero recurrence for 90 days"]
- [Qualitative threshold, e.g., "Procedure followed correctly 100%"]
DATA COLLECTION:
- Source: [Where data will come from]
- Sample Size: [Number of records/instances to review]
- Time Period: [Start and end dates]
- Responsible: [Who collects data]
VERIFICATION SCHEDULE:
- Implementation Complete: [Date]
- Waiting Period Ends: [Date]
- Verification Start: [Date]
- Verification Complete: [Date]
- Report Due: [Date]
APPROVAL:
CAPA Owner: _____________ Date: _______
Quality Assurance: _____________ Date: _______
```
---
## Verification Methods
### 1. Data Trend Analysis
**Best for:** Quantifiable issues with measurable outcomes (defect rates, cycle times, complaint trends)
**Procedure:**
1. Collect post-implementation data for defined period
2. Compare to pre-implementation baseline
3. Apply statistical analysis if sample size permits
4. Document trend direction and magnitude
**Example Criteria:**
- Defect rate reduced by ≥50% from baseline
- Zero recurrence of specific failure mode
- Process capability (Cpk) improved to ≥1.33
**Evidence Required:**
- Pre-implementation baseline data
- Post-implementation trend data
- Statistical analysis (if applicable)
- Trend charts with annotation
### 2. Process Audit
**Best for:** Procedure compliance issues, process control failures, systemic problems
**Procedure:**
1. Develop audit checklist based on corrective action
2. Conduct unannounced process audit
3. Interview operators and supervisors
4. Review records generated since implementation
5. Document compliance percentage
**Example Criteria:**
- 100% compliance with revised procedure
- All operators demonstrate competency
- No deviations observed during audit
**Evidence Required:**
- Audit checklist completed
- Interview notes
- Record samples reviewed
- Photos/observations (if applicable)
### 3. Record Review
**Best for:** Documentation issues, completeness problems, traceability failures
**Procedure:**
1. Define sample size based on volume (minimum 10 or 10%, whichever greater)
2. Review records generated post-implementation
3. Evaluate against specified requirements
4. Calculate compliance rate
**Example Criteria:**
- 100% of records meet completeness requirements
- All required signatures present
- Traceability maintained throughout
**Evidence Required:**
- List of records reviewed
- Compliance checklist results
- Non-compliance summary (if any)
### 4. Testing/Inspection
**Best for:** Product quality issues, equipment failures, specification non-conformances
**Procedure:**
1. Define test protocol based on corrective action
2. Conduct testing on post-implementation units
3. Compare results to acceptance criteria
4. Document pass/fail rates
**Example Criteria:**
- 100% of units pass revised inspection criteria
- All test results within specification
- Zero failures of targeted parameter
**Evidence Required:**
- Test protocol/method
- Test results data
- Pass/fail summary
- Comparison to pre-implementation results
### 5. Interview/Observation
**Best for:** Training issues, communication problems, human factors causes
**Procedure:**
1. Develop structured interview questions
2. Interview representative sample of affected personnel
3. Observe process execution in real-time
4. Document responses and observations
**Example Criteria:**
- All interviewed personnel demonstrate knowledge
- Observed practices match documented procedure
- No unsafe acts or workarounds observed
**Evidence Required:**
- Interview questions and responses
- Observation notes
- Training records (supporting)
---
## Effectiveness Criteria
### Defining Good Criteria
Criteria must be **SMART**:
| Element | Requirement | Example |
|---------|-------------|---------|
| **S**pecific | Clearly defined what to measure | "Calibration overdue rate" not "equipment issues" |
| **M**easurable | Quantifiable or objectively verifiable | "<2% overdue rate" not "improved timeliness" |
| **A**chievable | Realistic given the corrective action | Within capability of implemented solution |
| **R**elevant | Directly related to root cause | Addresses the actual problem |
| **T**ime-bound | Specified evaluation period | "For 90 consecutive days" |
### Criteria by Issue Type
| Issue Type | Typical Criteria | Threshold |
|------------|------------------|-----------|
| Nonconformance | Recurrence rate | Zero recurrence |
| Process deviation | Compliance rate | ≥95% compliance |
| Complaint | Complaint trend | ≥50% reduction |
| Calibration | Overdue rate | <2% overdue |
| Training | Competency pass rate | 100% pass |
| Documentation | Completeness rate | 100% complete |
| Supplier | Incoming reject rate | ≤1% reject rate |
### Sample Size Guidelines
| Population Size | Minimum Sample |
|-----------------|----------------|
| <10 | All (100%) |
| 10-50 | 10 |
| 51-100 | 15 |
| 101-500 | 20 |
| >500 | 25 or 10%, whichever less |
---
## Closure Requirements
### Closure Checklist
**CAPA Closure Prerequisites:**
- [ ] All corrective actions implemented
- [ ] Implementation evidence documented
- [ ] Verification waiting period complete
- [ ] Verification activities performed
- [ ] All effectiveness criteria met
- [ ] Verification evidence documented
- [ ] No recurrence during verification period
- [ ] CAPA owner review complete
- [ ] Quality Assurance review complete
- [ ] Documentation complete and filed
### Effectiveness Status Determination
```
EFFECTIVENESS DECISION TREE:
Did recurrence occur during verification period?
├── Yes → CAPA INEFFECTIVE (escalate per ineffective process)
└── No → Were all effectiveness criteria met?
├── Yes → Were any related issues identified?
│ ├── Yes → Open new CAPA if needed, close original
│ └── No → CAPA EFFECTIVE - proceed to closure
└── No → How many criteria missed?
├── Minor gap (1 criterion, marginal miss) →
│ Extend verification period OR accept with justification
└── Significant gap → CAPA INEFFECTIVE
EFFECTIVENESS DETERMINATION:
[ ] EFFECTIVE - All criteria met, no recurrence
[ ] EFFECTIVE WITH CONDITIONS - Minor gap, justified acceptance
[ ] INEFFECTIVE - Significant gaps or recurrence
```
### Closure Documentation
```
EFFECTIVENESS VERIFICATION REPORT
CAPA Number: [CAPA-XXXX]
Verification Complete Date: [Date]
Verified By: [Name, Title]
VERIFICATION SUMMARY:
| Criterion | Target | Actual | Status |
|-----------|--------|--------|--------|
| [Criterion 1] | [Target] | [Result] | ☑ Met / ☐ Not Met |
| [Criterion 2] | [Target] | [Result] | ☑ Met / ☐ Not Met |
| [Criterion 3] | [Target] | [Result] | ☑ Met / ☐ Not Met |
RECURRENCE CHECK:
- Recurrence during verification period: [ ] Yes [ ] No
- Related issues identified: [ ] Yes [ ] No
- If yes, describe: [Description]
EVIDENCE SUMMARY:
[List of evidence documents, record numbers, data sources]
EFFECTIVENESS DETERMINATION:
[ ] EFFECTIVE
[ ] EFFECTIVE WITH CONDITIONS: [Justification]
[ ] INEFFECTIVE: [Reason]
RECOMMENDED ACTION:
[ ] Close CAPA
[ ] Extend verification period to [Date]
[ ] Open new CAPA [CAPA-XXXX] for [Issue]
[ ] Re-investigate (return to root cause analysis)
APPROVALS:
CAPA Owner: _____________ Date: _______
Quality Assurance: _____________ Date: _______
Management (if Major/Critical): _____________ Date: _______
```
---
## Ineffective CAPA Process
### Definition of Ineffective
CAPA is ineffective when:
1. Original problem recurs during or after verification period
2. Effectiveness criteria not met
3. Root cause still present
4. Corrective action created new problems
### Ineffective CAPA Workflow
```
INEFFECTIVE CAPA DETECTED
│
├── 1. Immediate Actions
│ ├── Reopen CAPA (do not close as effective)
│ ├── Implement containment for recurrence
│ └── Notify CAPA owner and management
│
├── 2. Root Cause Re-evaluation
│ ├── Was original root cause correct?
│ │ ├── No → Conduct new root cause analysis
│ │ └── Yes → Was corrective action appropriate?
│ │ ├── No → Develop new corrective action
│ │ └── Yes → Was implementation adequate?
│ │ ├── No → Re-implement with improvements
│ │ └── Yes → Escalate (systemic issue)
│
├── 3. Escalation Criteria
│ ├── Second ineffective attempt → Management review required
│ ├── Safety-related recurrence → Immediate escalation
│ └── Pattern across multiple CAPAs → Systemic CAPA
│
└── 4. Documentation
├── Document ineffective status with evidence
├── Record re-investigation results
├── Update CAPA metrics/trending
└── Include in management review
```
### Preventing Ineffective CAPAs
| Common Cause | Prevention |
|--------------|------------|
| Superficial root cause | Validate root cause before action |
| Action addresses symptom not cause | Ensure action targets root cause |
| Implementation incomplete | Verify implementation before verification |
| Insufficient verification period | Allow adequate time for data collection |
| Wrong verification method | Match method to issue type |
| Unclear success criteria | Define SMART criteria upfront |
---
## Documentation Templates
### Verification Evidence Log
```
VERIFICATION EVIDENCE LOG
CAPA Number: [CAPA-XXXX]
| Doc/Record # | Description | Date | Reviewed By | Finding |
|--------------|-------------|------|-------------|---------|
| [Number] | [Description] | [Date] | [Reviewer] | [Compliant/Finding] |
| [Number] | [Description] | [Date] | [Reviewer] | [Compliant/Finding] |
SUMMARY:
- Total records reviewed: [Number]
- Compliant: [Number] ([Percentage]%)
- Non-compliant: [Number] ([Percentage]%)
CONCLUSION:
[Statement on whether evidence supports effectiveness]
```
### Trend Analysis Summary
```
TREND ANALYSIS FOR CAPA VERIFICATION
CAPA Number: [CAPA-XXXX]
Metric: [What is being measured]
BASELINE (Pre-Implementation):
- Period: [Start] to [End]
- Value: [Baseline value]
- Data points: [Number]
POST-IMPLEMENTATION:
- Period: [Start] to [End]
- Value: [Current value]
- Data points: [Number]
CHANGE:
- Absolute change: [Value]
- Percentage change: [Percentage]%
- Target: [Target value/change]
- Status: [ ] Met [ ] Not Met
TREND CHART:
[Include or reference trend chart showing before/after comparison]
STATISTICAL SIGNIFICANCE (if applicable):
- Method: [t-test, chi-square, etc.]
- p-value: [Value]
- Conclusion: [Statistically significant / Not significant]
```
### Interview Summary Template
```
VERIFICATION INTERVIEW SUMMARY
CAPA Number: [CAPA-XXXX]
Interviewer: [Name]
Date: [Date]
INTERVIEWEE:
- Name: [Name]
- Role: [Job title]
- Department: [Department]
- Experience: [Years in role]
QUESTIONS AND RESPONSES:
Q1: [Question about awareness of change]
A1: [Response summary]
Knowledge demonstrated: [ ] Yes [ ] Partial [ ] No
Q2: [Question about implementation of change]
A2: [Response summary]
Compliance demonstrated: [ ] Yes [ ] Partial [ ] No
Q3: [Question about understanding rationale]
A3: [Response summary]
Understanding demonstrated: [ ] Yes [ ] Partial [ ] No
OBSERVATION NOTES:
[Any relevant observations during interview]
CONCLUSION:
[ ] Interviewee demonstrates full knowledge and compliance
[ ] Interviewee demonstrates partial knowledge (specify gaps)
[ ] Interviewee does not demonstrate required knowledge
```
FILE:references/rca-methodologies.md
# Root Cause Analysis Methodologies
Decision criteria, templates, and implementation guidance for RCA techniques.
---
## Table of Contents
- [Method Selection Matrix](#method-selection-matrix)
- [5 Why Analysis](#5-why-analysis)
- [Fishbone Diagram](#fishbone-diagram)
- [Fault Tree Analysis](#fault-tree-analysis)
- [Human Factors Analysis](#human-factors-analysis)
- [Failure Mode and Effects Analysis](#failure-mode-and-effects-analysis)
- [Selecting the Right Method](#selecting-the-right-method)
---
## Method Selection Matrix
### When to Use Each Method
| Method | Use When | Problem Type | Team Size | Time Required |
|--------|----------|--------------|-----------|---------------|
| 5 Why | Single-cause issues, process deviations | Linear causation | 1-3 people | 30-60 min |
| Fishbone | Multi-factor problems, 3-6 contributing factors | Complex, systemic | 3-8 people | 2-4 hours |
| Fault Tree | Safety-critical failures, reliability issues | System failures | 2-5 people | 4-8 hours |
| Human Factors | Procedure/training-related issues | Human error | 3-6 people | 2-4 hours |
| FMEA | Systematic risk assessment, design review | Potential failures | 4-10 people | 8-16 hours |
### Quick Selection Decision Tree
```
Is the issue safety-critical or involves system reliability?
├── Yes → Use FAULT TREE ANALYSIS
└── No → Is human error the suspected primary cause?
├── Yes → Use HUMAN FACTORS ANALYSIS
└── No → How many potential contributing factors?
├── 1-2 factors → Use 5 WHY ANALYSIS
├── 3-6 factors → Use FISHBONE DIAGRAM
└── Unknown/Many → Use FMEA (proactive) or Fishbone (reactive)
```
---
## 5 Why Analysis
### Overview
Simple, iterative technique asking "why" repeatedly (typically 5 times) to drill from symptoms to root cause.
### When to Use
- Single-cause issues with linear causation
- Process deviations with clear failure point
- Quick investigations requiring rapid resolution
- Problems where symptoms clearly link to cause
### When NOT to Use
- Complex multi-factor problems
- Safety-critical incidents requiring comprehensive analysis
- Issues with multiple interacting causes
- When systemic factors are suspected
### 5 Why Template
```
PROBLEM STATEMENT:
[Clear, specific description of what happened, when, where, and impact]
WHY 1: Why did [problem] occur?
BECAUSE: [First-level cause]
EVIDENCE: [Data/observation supporting this cause]
WHY 2: Why did [first-level cause] occur?
BECAUSE: [Second-level cause]
EVIDENCE: [Data/observation supporting this cause]
WHY 3: Why did [second-level cause] occur?
BECAUSE: [Third-level cause]
EVIDENCE: [Data/observation supporting this cause]
WHY 4: Why did [third-level cause] occur?
BECAUSE: [Fourth-level cause]
EVIDENCE: [Data/observation supporting this cause]
WHY 5: Why did [fourth-level cause] occur?
BECAUSE: [Root cause - typically systemic or management system failure]
EVIDENCE: [Data/observation supporting this cause]
ROOT CAUSE VALIDATION:
- [ ] Can the root cause be verified with evidence?
- [ ] If root cause is eliminated, would problem recur?
- [ ] Is the root cause within organizational control?
- [ ] Does the root cause explain all symptoms?
```
### Example: Calibration Overdue
```
PROBLEM: pH meter (EQ-042) found 2 months overdue for calibration
WHY 1: Why was calibration overdue?
BECAUSE: The equipment was not on the calibration schedule
EVIDENCE: Calibration schedule reviewed, EQ-042 not listed
WHY 2: Why was it not on the calibration schedule?
BECAUSE: The schedule was not updated when equipment was purchased
EVIDENCE: Purchase date 2023-06-15, schedule dated 2023-01-01
WHY 3: Why was the schedule not updated?
BECAUSE: No process requires schedule update at equipment purchase
EVIDENCE: Equipment procedure SOP-EQ-001 reviewed, no such requirement
WHY 4: Why is there no requirement to update the schedule?
BECAUSE: The procedure was written before equipment tracking was centralized
EVIDENCE: SOP-EQ-001 last revised 2019, equipment system implemented 2021
WHY 5: Why has the procedure not been updated?
BECAUSE: Periodic procedure review did not assess compatibility with new systems
EVIDENCE: No documented review of SOP-EQ-001 against new equipment system
ROOT CAUSE: Procedure review process does not assess compatibility
with organizational systems implemented after original procedure creation
```
---
## Fishbone Diagram
### Overview
Also called Ishikawa or cause-and-effect diagram. Organizes potential causes into categories branching from the problem statement.
### Standard Categories (6M)
| Category | Focus Areas | Typical Causes |
|----------|-------------|----------------|
| **Man** (People) | Training, competency, workload | Skill gaps, fatigue, communication |
| **Machine** (Equipment) | Calibration, maintenance, age | Wear, malfunction, inadequate capacity |
| **Method** (Process) | Procedures, work instructions | Unclear steps, missing controls |
| **Material** | Specifications, suppliers, storage | Out-of-spec, degradation, contamination |
| **Measurement** | Calibration, methods, interpretation | Instrument error, wrong method |
| **Mother Nature** (Environment) | Temperature, humidity, cleanliness | Environmental excursions |
### Fishbone Template
```
PROBLEM STATEMENT: [Effect being investigated]
┌── Man ────────────────┐
│ ├─ [Cause 1] │
│ ├─ [Cause 2] │
│ └─ [Cause 3] │
│ │
┌── Machine ────────┤ ├── Method ──────────┐
│ ├─ [Cause 1] │ │ ├─ [Cause 1] │
│ ├─ [Cause 2] │ PROBLEM │ ├─ [Cause 2] │
│ └─ [Cause 3] ├───────────────────────┤ └─ [Cause 3] │
│ │ │ │
├── Material ───────┤ ├── Measurement ─────┤
│ ├─ [Cause 1] │ │ ├─ [Cause 1] │
│ ├─ [Cause 2] │ │ ├─ [Cause 2] │
│ └─ [Cause 3] │ │ └─ [Cause 3] │
│ │
└── Environment ────────┘
├─ [Cause 1]
├─ [Cause 2]
└─ [Cause 3]
CAUSE PRIORITIZATION:
| Cause | Category | Likelihood | Evidence | Priority |
|-------|----------|------------|----------|----------|
| [Cause A] | Method | High | [Evidence] | 1 |
| [Cause B] | Man | Medium | [Evidence] | 2 |
ROOT CAUSES IDENTIFIED:
1. [Primary root cause with supporting evidence]
2. [Contributing cause with supporting evidence]
```
### Facilitation Guidelines
1. Assemble cross-functional team (3-8 people)
2. Define problem statement clearly before starting
3. Brainstorm causes without judgment first
4. Organize into categories after brainstorming
5. Drill down on each major cause (sub-causes)
6. Prioritize based on evidence and likelihood
7. Validate top causes with data
---
## Fault Tree Analysis
### Overview
Top-down, deductive analysis starting with undesired event and systematically identifying all potential causes using Boolean logic (AND/OR gates).
### When to Use
- Safety-critical system failures
- Complex system reliability analysis
- Events with multiple failure pathways
- Regulatory-required investigations (FDA, MDR)
### FTA Symbols
| Symbol | Name | Meaning |
|--------|------|---------|
| Rectangle | Top Event / Intermediate Event | Undesired event or intermediate fault |
| Circle | Basic Event | Primary fault requiring no further analysis |
| Diamond | Undeveloped Event | Event not fully analyzed (data limitation) |
| AND Gate | Requires all inputs | All child events must occur for parent |
| OR Gate | Requires any input | Any child event causes parent |
### FTA Template
```
TOP EVENT: [Undesired event under investigation]
LEVEL 1 (Immediate Causes):
[Top Event]
│
└── OR GATE ──┬── [Cause 1.1]
├── [Cause 1.2]
└── [Cause 1.3]
LEVEL 2 (Contributing Causes):
[Cause 1.1]
│
└── AND GATE ──┬── [Cause 2.1]
└── [Cause 2.2]
MINIMAL CUT SETS:
(Combinations of basic events that cause top event)
1. {Basic Event A, Basic Event B} ← Both required (AND)
2. {Basic Event C} ← Single point failure (OR)
3. {Basic Event D, Basic Event E} ← Both required (AND)
CRITICAL PATH ANALYSIS:
Most likely failure pathway: [Description]
Single points of failure: [List]
RECOMMENDATIONS:
- Address single points of failure first
- Add redundancy where AND gates show vulnerability
- Prioritize controls on highest probability paths
```
### Cut Set Analysis
Minimal cut sets identify the smallest combination of basic events causing the top event:
- **Single-element cut sets**: Single points of failure (highest priority)
- **Two-element cut sets**: Dual failure scenarios
- **Probability calculation**: P(Top Event) = Union of P(Cut Sets)
---
## Human Factors Analysis
### Overview
Systematic analysis of human error focusing on cognitive, physical, and organizational factors contributing to performance failures.
### HFACS Categories
Human Factors Analysis and Classification System:
| Level | Category | Examples |
|-------|----------|----------|
| **Unsafe Acts** | Errors, violations | Skill-based, decision, perceptual errors |
| **Preconditions** | Conditions for unsafe acts | Fatigue, mental state, CRM, physical environment |
| **Unsafe Supervision** | Supervisory failures | Inadequate supervision, planned inappropriate ops |
| **Organizational Influences** | Organizational failures | Resource management, organizational climate |
### Human Error Types
| Type | Description | Example | Mitigation |
|------|-------------|---------|------------|
| Slip | Execution error in routine task | Wrong button pressed | Error-proofing, forcing functions |
| Lapse | Memory failure | Forgot step in procedure | Checklists, reminders |
| Mistake | Planning/decision error | Wrong procedure selected | Training, decision aids |
| Violation | Intentional deviation | Skipped step to save time | Culture change, supervision |
### Human Factors Investigation Template
```
INCIDENT DESCRIPTION:
[What happened, who was involved, when, where]
UNSAFE ACTS ANALYSIS:
Type of Error: [ ] Slip [ ] Lapse [ ] Mistake [ ] Violation
Description: [Specific action or inaction]
Task Being Performed: [Activity at time of error]
Experience Level: [Novice/Intermediate/Expert]
PRECONDITIONS FOR UNSAFE ACTS:
Cognitive Factors:
- [ ] Task complexity exceeded capability
- [ ] Time pressure
- [ ] Distraction/interruption
- [ ] Mental fatigue
Physical Factors:
- [ ] Physical fatigue
- [ ] Inadequate lighting
- [ ] Noise interference
- [ ] Workspace ergonomics
Team Factors:
- [ ] Communication breakdown
- [ ] Coordination failure
- [ ] Inadequate leadership
SUPERVISORY FACTORS:
- [ ] Inadequate supervision
- [ ] Failed to correct known problem
- [ ] Inappropriate staffing
- [ ] Authorized unnecessary risk
ORGANIZATIONAL FACTORS:
- [ ] Resource management deficiency
- [ ] Organizational process issue
- [ ] Organizational culture/climate
ROOT CAUSE(S):
[Human factors root causes identified]
CORRECTIVE ACTIONS:
| Action | Target Factor | Priority |
|--------|---------------|----------|
| [Action 1] | [Factor addressed] | High |
| [Action 2] | [Factor addressed] | Medium |
```
---
## Failure Mode and Effects Analysis
### Overview
Proactive, systematic technique identifying potential failure modes, their causes, and effects before failures occur.
### FMEA Types
| Type | Application | Scope |
|------|-------------|-------|
| Design FMEA (DFMEA) | Product design | Component and system design failures |
| Process FMEA (PFMEA) | Manufacturing process | Process step failures |
| System FMEA | System-level analysis | System interaction failures |
### Risk Priority Number (RPN)
RPN = Severity (S) × Occurrence (O) × Detection (D)
**Severity Scale (1-10):**
| Rating | Effect | Criteria |
|--------|--------|----------|
| 10 | Hazardous | Failure affects safe operation, no warning |
| 8-9 | Very High | Primary function lost, high impact |
| 6-7 | High | Performance degraded, customer dissatisfied |
| 4-5 | Moderate | Some performance loss, moderate impact |
| 2-3 | Low | Minor effect, slight inconvenience |
| 1 | None | No discernible effect |
**Occurrence Scale (1-10):**
| Rating | Likelihood | Failure Rate |
|--------|------------|--------------|
| 10 | Very High | >1 in 10 |
| 7-9 | High | 1 in 20 - 1 in 100 |
| 4-6 | Moderate | 1 in 400 - 1 in 2,000 |
| 2-3 | Low | 1 in 15,000 - 1 in 150,000 |
| 1 | Remote | <1 in 1,500,000 |
**Detection Scale (1-10):**
| Rating | Detection | Criteria |
|--------|-----------|----------|
| 10 | Absolute Uncertainty | No inspection/control, defect will reach customer |
| 7-9 | Very Remote to Remote | Controls unlikely to detect |
| 4-6 | Moderate | Controls may detect |
| 2-3 | High | Controls likely to detect |
| 1 | Almost Certain | Controls will almost certainly detect |
### FMEA Template
```
PROCESS/PRODUCT: [Name]
FMEA TEAM: [Members]
DATE: [Date]
| Item/Step | Failure Mode | Effect | S | Cause | O | Controls | D | RPN | Action |
|-----------|--------------|--------|---|-------|---|----------|---|-----|--------|
| [Item 1] | [How it fails] | [Impact] | 8 | [Why] | 4 | [Current] | 6 | 192 | [Action] |
| [Item 2] | [How it fails] | [Impact] | 6 | [Why] | 3 | [Current] | 4 | 72 | [Action] |
RPN THRESHOLD: Actions required for RPN > [threshold]
HIGH SEVERITY RULE: Actions required for S >= 9 regardless of RPN
ACTION PRIORITIZATION:
1. Address all items with S >= 9 first
2. Address items with highest RPN
3. Focus on reducing Occurrence (prevention)
4. Then improve Detection (inspection)
```
---
## Selecting the Right Method
### Decision Flowchart
```
START: Investigation Required
│
├── Is this a proactive assessment (no failure yet)?
│ └── Yes → Use FMEA
│
├── Is the issue safety-critical?
│ └── Yes → Use FAULT TREE ANALYSIS
│
├── Is human error the primary concern?
│ └── Yes → Use HUMAN FACTORS ANALYSIS
│
├── Are there multiple contributing factors (3+)?
│ ├── Yes → Use FISHBONE DIAGRAM
│ └── No → Use 5 WHY ANALYSIS
│
└── Uncertain? → Start with 5 WHY, escalate to FISHBONE if needed
```
### Hybrid Approach
For complex investigations, combine methods:
1. **Initial screening**: 5 Why for quick cause identification
2. **Detailed analysis**: Fishbone to explore all categories
3. **Validation**: Fault Tree for critical failure paths
4. **Systemic factors**: Human Factors for people-related causes
5. **Prevention**: FMEA for future risk mitigation
### Documentation Requirements
| Method | Required Outputs | Retention |
|--------|------------------|-----------|
| 5 Why | Completed template with evidence | CAPA record |
| Fishbone | Diagram + prioritized causes | CAPA record |
| Fault Tree | FTA diagram + cut set analysis | DHF/CAPA record |
| Human Factors | HFACS analysis + actions | CAPA record |
| FMEA | FMEA worksheet + action tracking | Design file |
FILE:scripts/capa_tracker.py
#!/usr/bin/env python3
"""
CAPA Tracker - Corrective and Preventive Action Management Tool
Tracks CAPA status, calculates metrics, identifies overdue items,
and generates reports for management review.
Usage:
python capa_tracker.py --capas capas.json
python capa_tracker.py --interactive
python capa_tracker.py --capas capas.json --output json
"""
import argparse
import json
import sys
from dataclasses import dataclass, field, asdict
from datetime import datetime, timedelta
from typing import List, Dict, Optional
from enum import Enum
class CAPAStatus(Enum):
OPEN = "Open"
INVESTIGATION = "Investigation"
ACTION_PLANNING = "Action Planning"
IMPLEMENTATION = "Implementation"
VERIFICATION = "Verification"
CLOSED_EFFECTIVE = "Closed - Effective"
CLOSED_INEFFECTIVE = "Closed - Ineffective"
class CAPASeverity(Enum):
CRITICAL = "Critical"
MAJOR = "Major"
MINOR = "Minor"
class CAPASource(Enum):
COMPLAINT = "Customer Complaint"
AUDIT = "Internal Audit"
EXTERNAL_AUDIT = "External Audit"
NONCONFORMANCE = "Nonconformance"
MANAGEMENT_REVIEW = "Management Review"
TREND_ANALYSIS = "Trend Analysis"
REGULATORY = "Regulatory Feedback"
OTHER = "Other"
@dataclass
class CAPA:
capa_number: str
title: str
description: str
source: CAPASource
severity: CAPASeverity
status: CAPAStatus
open_date: str
target_date: str
owner: str
root_cause: str = ""
corrective_action: str = ""
verification_date: Optional[str] = None
close_date: Optional[str] = None
days_open: int = 0
is_overdue: bool = False
@dataclass
class CAPAMetrics:
total_capas: int
open_capas: int
closed_capas: int
overdue_capas: int
avg_cycle_time: float
effectiveness_rate: float
by_status: Dict[str, int]
by_severity: Dict[str, int]
by_source: Dict[str, int]
overdue_list: List[Dict]
recommendations: List[str]
class CAPATracker:
"""CAPA tracking and metrics calculator."""
# Target cycle times by severity (days)
TARGET_CYCLE_TIMES = {
CAPASeverity.CRITICAL: 30,
CAPASeverity.MAJOR: 60,
CAPASeverity.MINOR: 90,
}
def __init__(self, capas: List[CAPA]):
self.capas = capas
self.today = datetime.now()
self._calculate_derived_fields()
def _calculate_derived_fields(self):
"""Calculate days open and overdue status."""
for capa in self.capas:
open_date = datetime.strptime(capa.open_date, "%Y-%m-%d")
if capa.close_date:
close_date = datetime.strptime(capa.close_date, "%Y-%m-%d")
capa.days_open = (close_date - open_date).days
else:
capa.days_open = (self.today - open_date).days
target_date = datetime.strptime(capa.target_date, "%Y-%m-%d")
if not capa.close_date and self.today > target_date:
capa.is_overdue = True
def calculate_metrics(self) -> CAPAMetrics:
"""Calculate comprehensive CAPA metrics."""
total = len(self.capas)
# Status counts
closed_statuses = [CAPAStatus.CLOSED_EFFECTIVE, CAPAStatus.CLOSED_INEFFECTIVE]
open_capas = [c for c in self.capas if c.status not in closed_statuses]
closed_capas = [c for c in self.capas if c.status in closed_statuses]
overdue_capas = [c for c in self.capas if c.is_overdue]
# Average cycle time (closed CAPAs only)
if closed_capas:
avg_cycle = sum(c.days_open for c in closed_capas) / len(closed_capas)
else:
avg_cycle = 0.0
# Effectiveness rate
effective = [c for c in self.capas if c.status == CAPAStatus.CLOSED_EFFECTIVE]
ineffective = [c for c in self.capas if c.status == CAPAStatus.CLOSED_INEFFECTIVE]
if effective or ineffective:
effectiveness = len(effective) / (len(effective) + len(ineffective)) * 100
else:
effectiveness = 0.0
# Counts by category
by_status = {}
for status in CAPAStatus:
count = len([c for c in self.capas if c.status == status])
if count > 0:
by_status[status.value] = count
by_severity = {}
for severity in CAPASeverity:
count = len([c for c in self.capas if c.severity == severity])
if count > 0:
by_severity[severity.value] = count
by_source = {}
for source in CAPASource:
count = len([c for c in self.capas if c.source == source])
if count > 0:
by_source[source.value] = count
# Overdue list
overdue_list = []
for capa in sorted(overdue_capas, key=lambda c: c.days_open, reverse=True):
target = datetime.strptime(capa.target_date, "%Y-%m-%d")
days_overdue = (self.today - target).days
overdue_list.append({
"capa_number": capa.capa_number,
"title": capa.title,
"severity": capa.severity.value,
"status": capa.status.value,
"days_overdue": days_overdue,
"owner": capa.owner
})
# Generate recommendations
recommendations = self._generate_recommendations(
open_capas, overdue_capas, effectiveness, avg_cycle
)
return CAPAMetrics(
total_capas=total,
open_capas=len(open_capas),
closed_capas=len(closed_capas),
overdue_capas=len(overdue_capas),
avg_cycle_time=round(avg_cycle, 1),
effectiveness_rate=round(effectiveness, 1),
by_status=by_status,
by_severity=by_severity,
by_source=by_source,
overdue_list=overdue_list,
recommendations=recommendations
)
def _generate_recommendations(
self,
open_capas: List[CAPA],
overdue_capas: List[CAPA],
effectiveness: float,
avg_cycle: float
) -> List[str]:
"""Generate actionable recommendations."""
recommendations = []
# Overdue CAPAs
if overdue_capas:
critical_overdue = [c for c in overdue_capas if c.severity == CAPASeverity.CRITICAL]
if critical_overdue:
recommendations.append(
f"URGENT: {len(critical_overdue)} critical CAPA(s) overdue. "
"Escalate to management immediately."
)
else:
recommendations.append(
f"ACTION: {len(overdue_capas)} CAPA(s) overdue. "
"Review and update target dates or expedite closure."
)
# Effectiveness rate
if effectiveness < 80 and effectiveness > 0:
recommendations.append(
f"CONCERN: Effectiveness rate at {effectiveness:.0f}%. "
"Review root cause analysis quality and corrective action adequacy."
)
# Cycle time
if avg_cycle > 60:
recommendations.append(
f"IMPROVEMENT: Average cycle time is {avg_cycle:.0f} days. "
"Target is 60 days. Review investigation and approval bottlenecks."
)
# Investigation backlog
in_investigation = [c for c in open_capas if c.status == CAPAStatus.INVESTIGATION]
if len(in_investigation) > 5:
recommendations.append(
f"WORKLOAD: {len(in_investigation)} CAPAs in investigation phase. "
"Consider additional resources or prioritization."
)
# Stuck in verification
in_verification = [c for c in open_capas if c.status == CAPAStatus.VERIFICATION]
old_verification = [c for c in in_verification if c.days_open > 120]
if old_verification:
recommendations.append(
f"STALLED: {len(old_verification)} CAPA(s) in verification >120 days. "
"Complete effectiveness checks or extend with justification."
)
# Source patterns
complaint_capas = [c for c in self.capas if c.source == CAPASource.COMPLAINT]
if len(complaint_capas) > len(self.capas) * 0.4:
recommendations.append(
"TREND: >40% of CAPAs from customer complaints. "
"Review preventive action effectiveness and quality controls."
)
if not recommendations:
recommendations.append(
"CAPA program operating within targets. "
"Continue monitoring key metrics."
)
return recommendations
def get_aging_report(self) -> Dict:
"""Generate aging analysis of open CAPAs."""
open_statuses = [
CAPAStatus.OPEN, CAPAStatus.INVESTIGATION,
CAPAStatus.ACTION_PLANNING, CAPAStatus.IMPLEMENTATION,
CAPAStatus.VERIFICATION
]
open_capas = [c for c in self.capas if c.status in open_statuses]
aging_buckets = {
"0-30 days": [],
"31-60 days": [],
"61-90 days": [],
"91-120 days": [],
">120 days": []
}
for capa in open_capas:
days = capa.days_open
if days <= 30:
bucket = "0-30 days"
elif days <= 60:
bucket = "31-60 days"
elif days <= 90:
bucket = "61-90 days"
elif days <= 120:
bucket = "91-120 days"
else:
bucket = ">120 days"
aging_buckets[bucket].append({
"capa_number": capa.capa_number,
"title": capa.title,
"days_open": days,
"status": capa.status.value,
"severity": capa.severity.value
})
return aging_buckets
def format_text_output(metrics: CAPAMetrics, aging: Dict) -> str:
"""Format metrics as text report."""
lines = [
"=" * 70,
"CAPA STATUS REPORT",
"=" * 70,
f"Generated: {datetime.now().strftime('%Y-%m-%d %H:%M')}",
"",
"SUMMARY METRICS",
"-" * 40,
f"Total CAPAs: {metrics.total_capas}",
f"Open CAPAs: {metrics.open_capas}",
f"Closed CAPAs: {metrics.closed_capas}",
f"Overdue CAPAs: {metrics.overdue_capas}",
f"Avg Cycle Time: {metrics.avg_cycle_time} days",
f"Effectiveness Rate: {metrics.effectiveness_rate}%",
"",
"STATUS DISTRIBUTION",
"-" * 40,
]
for status, count in metrics.by_status.items():
bar = "█" * min(count, 20)
lines.append(f" {status:<25} {bar} {count}")
lines.extend([
"",
"SEVERITY DISTRIBUTION",
"-" * 40,
])
for severity, count in metrics.by_severity.items():
bar = "█" * min(count, 20)
lines.append(f" {severity:<25} {bar} {count}")
lines.extend([
"",
"SOURCE DISTRIBUTION",
"-" * 40,
])
for source, count in metrics.by_source.items():
bar = "█" * min(count, 20)
lines.append(f" {source:<25} {bar} {count}")
lines.extend([
"",
"AGING ANALYSIS",
"-" * 40,
])
for bucket, capas in aging.items():
lines.append(f" {bucket}: {len(capas)} CAPA(s)")
if metrics.overdue_list:
lines.extend([
"",
"OVERDUE CAPAs",
"-" * 40,
f"{'CAPA #':<12} {'Title':<25} {'Days':<6} {'Owner':<15}",
"-" * 60,
])
for item in metrics.overdue_list[:10]:
title = item["title"][:24] if len(item["title"]) > 24 else item["title"]
lines.append(
f"{item['capa_number']:<12} {title:<25} "
f"{item['days_overdue']:<6} {item['owner']:<15}"
)
if len(metrics.overdue_list) > 10:
lines.append(f"... and {len(metrics.overdue_list) - 10} more")
lines.extend([
"",
"RECOMMENDATIONS",
"-" * 40,
])
for i, rec in enumerate(metrics.recommendations, 1):
lines.append(f"{i}. {rec}")
lines.append("=" * 70)
return "\n".join(lines)
def interactive_mode():
"""Run interactive CAPA entry mode."""
print("=" * 60)
print("CAPA Tracker - Interactive Mode")
print("=" * 60)
capas = []
print("\nEnter CAPAs (blank CAPA number to finish):\n")
while True:
capa_num = input("CAPA Number (e.g., CAPA-2024-001): ").strip()
if not capa_num:
break
title = input("Title: ").strip()
description = input("Description: ").strip()
print("Source options: C=Complaint, A=Audit, N=Nonconformance, M=Management Review, T=Trend, O=Other")
source_input = input("Source [C/A/N/M/T/O]: ").strip().upper()
source_map = {
"C": CAPASource.COMPLAINT,
"A": CAPASource.AUDIT,
"N": CAPASource.NONCONFORMANCE,
"M": CAPASource.MANAGEMENT_REVIEW,
"T": CAPASource.TREND_ANALYSIS,
"O": CAPASource.OTHER
}
source = source_map.get(source_input, CAPASource.OTHER)
print("Severity: C=Critical, M=Major, I=Minor")
severity_input = input("Severity [C/M/I]: ").strip().upper()
severity_map = {
"C": CAPASeverity.CRITICAL,
"M": CAPASeverity.MAJOR,
"I": CAPASeverity.MINOR
}
severity = severity_map.get(severity_input, CAPASeverity.MINOR)
print("Status: O=Open, I=Investigation, P=Action Planning, M=Implementation, V=Verification, E=Closed Effective, N=Closed Ineffective")
status_input = input("Status [O/I/P/M/V/E/N]: ").strip().upper()
status_map = {
"O": CAPAStatus.OPEN,
"I": CAPAStatus.INVESTIGATION,
"P": CAPAStatus.ACTION_PLANNING,
"M": CAPAStatus.IMPLEMENTATION,
"V": CAPAStatus.VERIFICATION,
"E": CAPAStatus.CLOSED_EFFECTIVE,
"N": CAPAStatus.CLOSED_INEFFECTIVE
}
status = status_map.get(status_input, CAPAStatus.OPEN)
open_date = input("Open Date (YYYY-MM-DD): ").strip()
target_date = input("Target Date (YYYY-MM-DD): ").strip()
owner = input("Owner: ").strip()
close_date = None
if status in [CAPAStatus.CLOSED_EFFECTIVE, CAPAStatus.CLOSED_INEFFECTIVE]:
close_date = input("Close Date (YYYY-MM-DD): ").strip()
capas.append(CAPA(
capa_number=capa_num,
title=title,
description=description,
source=source,
severity=severity,
status=status,
open_date=open_date,
target_date=target_date,
owner=owner,
close_date=close_date if close_date else None
))
print(f"\nAdded: {capa_num}\n")
if not capas:
print("No CAPAs entered. Exiting.")
return
tracker = CAPATracker(capas)
metrics = tracker.calculate_metrics()
aging = tracker.get_aging_report()
print("\n" + format_text_output(metrics, aging))
def main():
parser = argparse.ArgumentParser(
description="CAPA Tracking and Metrics Tool"
)
parser.add_argument(
"--capas",
type=str,
help="JSON file with CAPA data"
)
parser.add_argument(
"--output",
choices=["text", "json"],
default="text",
help="Output format"
)
parser.add_argument(
"--interactive",
action="store_true",
help="Run in interactive mode"
)
parser.add_argument(
"--sample",
action="store_true",
help="Generate sample CAPA data file"
)
args = parser.parse_args()
if args.interactive:
interactive_mode()
return
if args.sample:
sample_data = {
"capas": [
{
"capa_number": "CAPA-2024-001",
"title": "Calibration overdue for pH meter",
"description": "pH meter EQ-042 found 2 months overdue",
"source": "AUDIT",
"severity": "MAJOR",
"status": "VERIFICATION",
"open_date": "2024-06-15",
"target_date": "2024-08-15",
"owner": "J. Smith",
"root_cause": "No trigger for schedule update at equipment purchase",
"corrective_action": "Updated SOP-EQ-001 to require schedule update"
},
{
"capa_number": "CAPA-2024-002",
"title": "Customer complaint - labeling error",
"description": "Wrong lot number on product label",
"source": "COMPLAINT",
"severity": "CRITICAL",
"status": "INVESTIGATION",
"open_date": "2024-09-01",
"target_date": "2024-10-01",
"owner": "M. Jones"
},
{
"capa_number": "CAPA-2024-003",
"title": "Training records incomplete",
"description": "Missing effectiveness verification for 3 operators",
"source": "AUDIT",
"severity": "MINOR",
"status": "CLOSED_EFFECTIVE",
"open_date": "2024-03-10",
"target_date": "2024-06-10",
"owner": "A. Brown",
"close_date": "2024-05-20"
}
]
}
print(json.dumps(sample_data, indent=2))
return
if args.capas:
with open(args.capas, "r") as f:
data = json.load(f)
capas = []
for c in data.get("capas", []):
try:
source = CAPASource[c.get("source", "OTHER").upper()]
except KeyError:
source = CAPASource.OTHER
try:
severity = CAPASeverity[c.get("severity", "MINOR").upper()]
except KeyError:
severity = CAPASeverity.MINOR
try:
status = CAPAStatus[c.get("status", "OPEN").upper()]
except KeyError:
status = CAPAStatus.OPEN
capas.append(CAPA(
capa_number=c["capa_number"],
title=c.get("title", ""),
description=c.get("description", ""),
source=source,
severity=severity,
status=status,
open_date=c["open_date"],
target_date=c["target_date"],
owner=c.get("owner", ""),
root_cause=c.get("root_cause", ""),
corrective_action=c.get("corrective_action", ""),
verification_date=c.get("verification_date"),
close_date=c.get("close_date")
))
else:
# Demo data if no file provided
capas = [
CAPA(
capa_number="CAPA-2024-001",
title="Calibration overdue",
description="pH meter overdue",
source=CAPASource.AUDIT,
severity=CAPASeverity.MAJOR,
status=CAPAStatus.VERIFICATION,
open_date="2024-06-15",
target_date="2024-08-15",
owner="J. Smith"
),
CAPA(
capa_number="CAPA-2024-002",
title="Labeling error complaint",
description="Wrong lot number",
source=CAPASource.COMPLAINT,
severity=CAPASeverity.CRITICAL,
status=CAPAStatus.INVESTIGATION,
open_date="2024-09-01",
target_date="2024-10-01",
owner="M. Jones"
),
CAPA(
capa_number="CAPA-2024-003",
title="Training records incomplete",
description="Missing effectiveness verification",
source=CAPASource.AUDIT,
severity=CAPASeverity.MINOR,
status=CAPAStatus.CLOSED_EFFECTIVE,
open_date="2024-03-10",
target_date="2024-06-10",
owner="A. Brown",
close_date="2024-05-20"
)
]
tracker = CAPATracker(capas)
metrics = tracker.calculate_metrics()
aging = tracker.get_aging_report()
if args.output == "json":
output = {
"metrics": asdict(metrics),
"aging": aging
}
print(json.dumps(output, indent=2))
else:
print(format_text_output(metrics, aging))
if __name__ == "__main__":
main()
FILE:scripts/root_cause_analyzer.py
#!/usr/bin/env python3
"""
Root Cause Analyzer - Structured root cause analysis for CAPA investigations.
Supports multiple analysis methodologies:
- 5-Why Analysis
- Fishbone (Ishikawa) Diagram
- Fault Tree Analysis
- Kepner-Tregoe Problem Analysis
Generates structured root cause reports and CAPA recommendations.
Usage:
python root_cause_analyzer.py --method 5why --problem "High defect rate in assembly line"
python root_cause_analyzer.py --interactive
python root_cause_analyzer.py --data investigation.json --output json
"""
import argparse
import json
import sys
from dataclasses import dataclass, field, asdict
from typing import List, Dict, Optional
from enum import Enum
from datetime import datetime
class AnalysisMethod(Enum):
FIVE_WHY = "5-Why"
FISHBONE = "Fishbone"
FAULT_TREE = "Fault Tree"
KEPNER_TREGOE = "Kepner-Tregoe"
class RootCauseCategory(Enum):
MAN = "Man (People)"
MACHINE = "Machine (Equipment)"
MATERIAL = "Material"
METHOD = "Method (Process)"
MEASUREMENT = "Measurement"
ENVIRONMENT = "Environment"
MANAGEMENT = "Management (Policy)"
SOFTWARE = "Software/Data"
class SeverityLevel(Enum):
LOW = "Low"
MEDIUM = "Medium"
HIGH = "High"
CRITICAL = "Critical"
@dataclass
class WhyStep:
"""A single step in 5-Why analysis."""
level: int
question: str
answer: str
evidence: str = ""
verified: bool = False
@dataclass
class FishboneCause:
"""A cause in fishbone analysis."""
category: str
cause: str
sub_causes: List[str] = field(default_factory=list)
is_root: bool = False
evidence: str = ""
@dataclass
class FaultEvent:
"""An event in fault tree analysis."""
event_id: str
description: str
is_basic: bool = True # Basic events have no children
gate_type: str = "OR" # OR, AND
children: List[str] = field(default_factory=list)
probability: Optional[float] = None
@dataclass
class RootCauseFinding:
"""Identified root cause with evidence."""
cause_id: str
description: str
category: str
evidence: List[str] = field(default_factory=list)
contributing_factors: List[str] = field(default_factory=list)
systemic: bool = False # Whether it's a systemic vs. local issue
@dataclass
class CAPARecommendation:
"""Corrective or preventive action recommendation."""
action_id: str
action_type: str # "Corrective" or "Preventive"
description: str
addresses_cause: str # cause_id
priority: str
estimated_effort: str
responsible_role: str
effectiveness_criteria: List[str] = field(default_factory=list)
@dataclass
class RootCauseAnalysis:
"""Complete root cause analysis result."""
investigation_id: str
problem_statement: str
analysis_method: str
root_causes: List[RootCauseFinding]
recommendations: List[CAPARecommendation]
analysis_details: Dict
confidence_level: float
investigator_notes: List[str] = field(default_factory=list)
class RootCauseAnalyzer:
"""Performs structured root cause analysis."""
def __init__(self):
self.analysis_steps = []
self.findings = []
def analyze_5why(self, problem: str, whys: List[Dict] = None) -> Dict:
"""Perform 5-Why analysis."""
steps = []
if whys:
for i, w in enumerate(whys, 1):
steps.append(WhyStep(
level=i,
question=w.get("question", f"Why did this occur? (Level {i})"),
answer=w.get("answer", ""),
evidence=w.get("evidence", ""),
verified=w.get("verified", False)
))
# Analyze depth and quality
depth = len(steps)
has_root = any(
s.answer and ("system" in s.answer.lower() or "policy" in s.answer.lower() or "process" in s.answer.lower())
for s in steps
)
return {
"method": "5-Why Analysis",
"steps": [asdict(s) for s in steps],
"depth": depth,
"reached_systemic_cause": has_root,
"quality_score": min(100, depth * 20 + (20 if has_root else 0))
}
def analyze_fishbone(self, problem: str, causes: List[Dict] = None) -> Dict:
"""Perform fishbone (Ishikawa) analysis."""
categories = {}
fishbone_causes = []
if causes:
for c in causes:
cat = c.get("category", "Method")
cause = c.get("cause", "")
sub = c.get("sub_causes", [])
if cat not in categories:
categories[cat] = []
categories[cat].append({
"cause": cause,
"sub_causes": sub,
"is_root": c.get("is_root", False),
"evidence": c.get("evidence", "")
})
fishbone_causes.append(FishboneCause(
category=cat,
cause=cause,
sub_causes=sub,
is_root=c.get("is_root", False),
evidence=c.get("evidence", "")
))
root_causes = [fc for fc in fishbone_causes if fc.is_root]
return {
"method": "Fishbone (Ishikawa) Analysis",
"problem": problem,
"categories": categories,
"total_causes": len(fishbone_causes),
"root_causes_identified": len(root_causes),
"categories_covered": list(categories.keys()),
"recommended_categories": [c.value for c in RootCauseCategory],
"missing_categories": [c.value for c in RootCauseCategory if c.value.split(" (")[0] not in categories]
}
def analyze_fault_tree(self, top_event: str, events: List[Dict] = None) -> Dict:
"""Perform fault tree analysis."""
fault_events = {}
if events:
for e in events:
fault_events[e["event_id"]] = FaultEvent(
event_id=e["event_id"],
description=e.get("description", ""),
is_basic=e.get("is_basic", True),
gate_type=e.get("gate_type", "OR"),
children=e.get("children", []),
probability=e.get("probability")
)
# Find basic events (root causes)
basic_events = {eid: ev for eid, ev in fault_events.items() if ev.is_basic}
intermediate_events = {eid: ev for eid, ev in fault_events.items() if not ev.is_basic}
return {
"method": "Fault Tree Analysis",
"top_event": top_event,
"total_events": len(fault_events),
"basic_events": len(basic_events),
"intermediate_events": len(intermediate_events),
"basic_event_details": [asdict(e) for e in basic_events.values()],
"cut_sets": self._find_cut_sets(fault_events)
}
def _find_cut_sets(self, events: Dict[str, FaultEvent]) -> List[List[str]]:
"""Find minimal cut sets (combinations of basic events that cause top event)."""
# Simplified cut set analysis
cut_sets = []
for eid, event in events.items():
if not event.is_basic and event.gate_type == "AND":
cut_sets.append(event.children)
return cut_sets[:5] # Return top 5
def generate_recommendations(
self,
root_causes: List[RootCauseFinding],
problem: str
) -> List[CAPARecommendation]:
"""Generate CAPA recommendations based on root causes."""
recommendations = []
for i, cause in enumerate(root_causes, 1):
# Corrective action (fix the immediate cause)
recommendations.append(CAPARecommendation(
action_id=f"CA-{i:03d}",
action_type="Corrective",
description=f"Address immediate cause: {cause.description}",
addresses_cause=cause.cause_id,
priority=self._assess_priority(cause),
estimated_effort=self._estimate_effort(cause),
responsible_role=self._suggest_responsible(cause),
effectiveness_criteria=[
f"Elimination of {cause.description} confirmed by audit",
"No recurrence within 90 days",
"Metrics return to acceptable range"
]
))
# Preventive action (prevent recurrence in other areas)
if cause.systemic:
recommendations.append(CAPARecommendation(
action_id=f"PA-{i:03d}",
action_type="Preventive",
description=f"Systemic prevention: Update process/procedure to prevent similar issues",
addresses_cause=cause.cause_id,
priority="Medium",
estimated_effort="2-4 weeks",
responsible_role="Quality Manager",
effectiveness_criteria=[
"Updated procedure approved and implemented",
"Training completed for affected personnel",
"No similar issues in related processes within 6 months"
]
))
return recommendations
def _assess_priority(self, cause: RootCauseFinding) -> str:
if cause.systemic or "safety" in cause.description.lower():
return "High"
elif "quality" in cause.description.lower():
return "Medium"
return "Low"
def _estimate_effort(self, cause: RootCauseFinding) -> str:
if cause.systemic:
return "4-8 weeks"
elif len(cause.contributing_factors) > 3:
return "2-4 weeks"
return "1-2 weeks"
def _suggest_responsible(self, cause: RootCauseFinding) -> str:
category_roles = {
"Man": "Training Manager",
"Machine": "Engineering Manager",
"Material": "Supply Chain Manager",
"Method": "Process Owner",
"Measurement": "Quality Engineer",
"Environment": "Facilities Manager",
"Management": "Department Head",
"Software": "IT/Software Manager"
}
cat_key = cause.category.split(" (")[0] if "(" in cause.category else cause.category
return category_roles.get(cat_key, "Quality Manager")
def full_analysis(
self,
problem: str,
method: str = "5-Why",
analysis_data: Dict = None
) -> RootCauseAnalysis:
"""Perform complete root cause analysis."""
investigation_id = f"RCA-{datetime.now().strftime('%Y%m%d-%H%M')}"
analysis_details = {}
root_causes = []
if method == "5-Why" and analysis_data:
analysis_details = self.analyze_5why(problem, analysis_data.get("whys", []))
# Extract root cause from deepest why
steps = analysis_details.get("steps", [])
if steps:
last_step = steps[-1]
root_causes.append(RootCauseFinding(
cause_id="RC-001",
description=last_step.get("answer", "Unknown"),
category="Systemic",
evidence=[s.get("evidence", "") for s in steps if s.get("evidence")],
systemic=analysis_details.get("reached_systemic_cause", False)
))
elif method == "Fishbone" and analysis_data:
analysis_details = self.analyze_fishbone(problem, analysis_data.get("causes", []))
for i, cat in enumerate(analysis_data.get("causes", [])):
if cat.get("is_root"):
root_causes.append(RootCauseFinding(
cause_id=f"RC-{i+1:03d}",
description=cat.get("cause", ""),
category=cat.get("category", ""),
evidence=[cat.get("evidence", "")] if cat.get("evidence") else [],
sub_causes=cat.get("sub_causes", []),
systemic=True
))
recommendations = self.generate_recommendations(root_causes, problem)
# Confidence based on evidence and method
confidence = 0.7
if root_causes and any(rc.evidence for rc in root_causes):
confidence = 0.85
if len(root_causes) > 1:
confidence = min(0.95, confidence + 0.05)
return RootCauseAnalysis(
investigation_id=investigation_id,
problem_statement=problem,
analysis_method=method,
root_causes=root_causes,
recommendations=recommendations,
analysis_details=analysis_details,
confidence_level=confidence
)
def format_rca_text(rca: RootCauseAnalysis) -> str:
"""Format RCA report as text."""
lines = [
"=" * 70,
"ROOT CAUSE ANALYSIS REPORT",
"=" * 70,
f"Investigation ID: {rca.investigation_id}",
f"Analysis Method: {rca.analysis_method}",
f"Confidence Level: {rca.confidence_level:.0%}",
"",
"PROBLEM STATEMENT",
"-" * 40,
f" {rca.problem_statement}",
"",
"ROOT CAUSES IDENTIFIED",
"-" * 40,
]
for rc in rca.root_causes:
lines.extend([
f"",
f" [{rc.cause_id}] {rc.description}",
f" Category: {rc.category}",
f" Systemic: {'Yes' if rc.systemic else 'No'}",
])
if rc.evidence:
lines.append(f" Evidence:")
for ev in rc.evidence:
if ev:
lines.append(f" • {ev}")
if rc.contributing_factors:
lines.append(f" Contributing Factors:")
for cf in rc.contributing_factors:
lines.append(f" - {cf}")
lines.extend([
"",
"RECOMMENDED ACTIONS",
"-" * 40,
])
for rec in rca.recommendations:
lines.extend([
f"",
f" [{rec.action_id}] {rec.action_type}: {rec.description}",
f" Priority: {rec.priority} | Effort: {rec.estimated_effort}",
f" Responsible: {rec.responsible_role}",
f" Effectiveness Criteria:",
])
for ec in rec.effectiveness_criteria:
lines.append(f" ✓ {ec}")
if "steps" in rca.analysis_details:
lines.extend([
"",
"5-WHY CHAIN",
"-" * 40,
])
for step in rca.analysis_details["steps"]:
lines.extend([
f"",
f" Why {step['level']}: {step['question']}",
f" → {step['answer']}",
])
if step.get("evidence"):
lines.append(f" Evidence: {step['evidence']}")
lines.append("=" * 70)
return "\n".join(lines)
def main():
parser = argparse.ArgumentParser(description="Root Cause Analyzer for CAPA Investigations")
parser.add_argument("--problem", type=str, help="Problem statement")
parser.add_argument("--method", choices=["5why", "fishbone", "fault-tree", "kt"],
default="5why", help="Analysis method")
parser.add_argument("--data", type=str, help="JSON file with analysis data")
parser.add_argument("--output", choices=["text", "json"], default="text", help="Output format")
parser.add_argument("--interactive", action="store_true", help="Interactive mode")
args = parser.parse_args()
analyzer = RootCauseAnalyzer()
if args.data:
with open(args.data) as f:
data = json.load(f)
problem = data.get("problem", "Unknown problem")
method = data.get("method", "5-Why")
rca = analyzer.full_analysis(problem, method, data)
elif args.problem:
method_map = {"5why": "5-Why", "fishbone": "Fishbone", "fault-tree": "Fault Tree", "kt": "Kepner-Tregoe"}
rca = analyzer.full_analysis(args.problem, method_map.get(args.method, "5-Why"))
else:
# Demo
demo_data = {
"method": "5-Why",
"whys": [
{"question": "Why did the product fail inspection?", "answer": "Surface defect detected on 15% of units", "evidence": "QC inspection records"},
{"question": "Why did surface defects occur?", "answer": "Injection molding temperature was outside spec", "evidence": "Process monitoring data"},
{"question": "Why was temperature outside spec?", "answer": "Temperature controller calibration drift", "evidence": "Calibration log"},
{"question": "Why did calibration drift go undetected?", "answer": "No automated alert for drift, manual checks missed it", "evidence": "SOP review"},
{"question": "Why was there no automated alert?", "answer": "Process monitoring system lacks drift detection capability - systemic gap", "evidence": "System requirements review"}
]
}
rca = analyzer.full_analysis("High defect rate in injection molding process", "5-Why", demo_data)
if args.output == "json":
result = {
"investigation_id": rca.investigation_id,
"problem": rca.problem_statement,
"method": rca.analysis_method,
"root_causes": [asdict(rc) for rc in rca.root_causes],
"recommendations": [asdict(rec) for rec in rca.recommendations],
"analysis_details": rca.analysis_details,
"confidence": rca.confidence_level
}
print(json.dumps(result, indent=2, default=str))
else:
print(format_rca_text(rca))
if __name__ == "__main__":
main()
Giả định kế hoạch thất bại sau 12 tháng rồi lần ngược để tìm điểm yếu, giả định và rủi ro thực thi.
--- name: "challenge" description: "Pre-mortem plan analysis. Imagine the plan failed 12 months from now and work backwards to find the weaknesses. Surfaces assumptions, dependencies, and execution risks before committing resources. Use when before significant resource commitment, before presenting to a board or investors, when feedback has been one-sidedly positive, or when there is pressure to move fast and figure it out later." --- # /em:challenge — Pre-Mortem Plan Analysis **Command:** `/em:challenge <plan>` Systematically finds weaknesses in any plan before reality does. Not to kill the plan — to make it survive contact with reality. --- ## The Core Idea Most plans fail for predictable reasons. Not bad luck — bad assumptions. Overestimated demand. Underestimated complexity. Dependencies nobody questioned. Timing that made sense in a spreadsheet but not in the real world. The pre-mortem technique: **imagine it's 12 months from now and this plan failed spectacularly. Now work backwards. Why?** That's not pessimism. It's how you build something that doesn't collapse. --- ## When to Run a Challenge - Before committing significant resources to a plan - Before presenting to the board or investors - When you notice you're only hearing positive feedback about the plan - When the plan requires multiple external dependencies to align - When there's pressure to move fast and "figure it out later" - When you feel excited about the plan (excitement is a signal to scrutinize harder) --- ## The Challenge Framework ### Step 1: Extract Core Assumptions Before you can test a plan, you need to surface everything it assumes to be true. For each section of the plan, ask: - What has to be true for this to work? - What are we assuming about customer behavior? - What are we assuming about competitor response? - What are we assuming about our own execution capability? - What external factors does this depend on? **Common assumption categories:** - **Market assumptions** — size, growth rate, customer willingness to pay, buying cycle - **Execution assumptions** — team capacity, velocity, no major hires needed - **Customer assumptions** — they have the problem, they know they have it, they'll pay to solve it - **Competitive assumptions** — incumbents won't respond, no new entrant, moat holds - **Financial assumptions** — burn rate, revenue timing, CAC, LTV ratios - **Dependency assumptions** — partner will deliver, API won't change, regulations won't shift ### Step 2: Rate Each Assumption For every assumption extracted, rate it on two dimensions: **Confidence level (how sure are you this is true):** - **High** — verified with data, customer conversations, market research - **Medium** — directionally right but not validated - **Low** — plausible but untested - **Unknown** — we simply don't know **Impact if wrong (what happens if this assumption fails):** - **Critical** — plan fails entirely - **High** — major delay or cost overrun - **Medium** — significant rework required - **Low** — manageable adjustment ### Step 3: Map Vulnerabilities The matrix of Low/Unknown confidence × Critical/High impact = your highest-risk assumptions. **Vulnerability = Low confidence + High impact** These are not problems to ignore. They're the bets you're making. The question is: are you making them consciously? ### Step 4: Find the Dependency Chain Many plans fail not because any single assumption is wrong, but because multiple assumptions have to be right simultaneously. Map the chain: - Does assumption B depend on assumption A being true first? - If the first thing goes wrong, how many downstream things break? - What's the critical path? What has zero slack? ### Step 5: Test the Reversibility For each critical vulnerability: if this assumption turns out to be wrong at month 3, what do you do? - Can you pivot? - Can you cut scope? - Is money already spent? - Are commitments already made? The less reversible, the more rigorously you need to validate before committing. --- ## Output Format **Challenge Report: [Plan Name]** ``` CORE ASSUMPTIONS (extracted) 1. [Assumption] — Confidence: [H/M/L/?] — Impact if wrong: [Critical/High/Medium/Low] 2. ... VULNERABILITY MAP Critical risks (act before proceeding): • [#N] [Assumption] — WHY it might be wrong — WHAT breaks if it is High risks (validate before scaling): • ... DEPENDENCY CHAIN [Assumption A] → depends on → [Assumption B] → which enables → [Assumption C] Weakest link: [X] — if this breaks, [Y] and [Z] also fail REVERSIBILITY ASSESSMENT • Reversible bets: [list] • Irreversible commitments: [list — treat with extreme care] KILL SWITCHES What would have to be true at [30/60/90 days] to continue vs. kill/pivot? • Continue if: ... • Kill/pivot if: ... HARDENING ACTIONS 1. [Specific validation to do before proceeding] 2. [Alternative approach to consider] 3. [Contingency to build into the plan] ``` --- ## Challenge Patterns by Plan Type ### Product Roadmap - Are we building what customers will pay for, or what they said they wanted? - Does the velocity estimate account for real team capacity (not theoretical)? - What happens if the anchor feature takes 3× longer than estimated? - Who owns decisions when requirements conflict? ### Go-to-Market Plan - What's the actual ICP conversion rate, not the hoped-for one? - How many touches to close, and do you have the sales capacity for that? - What happens if the first 10 deals take 3 months instead of 1? - Is "land and expand" a real motion or a hope? ### Hiring Plan - What happens if the key hire takes 4 months to find, not 6 weeks? - Is the plan dependent on retaining specific people who might leave? - Does the plan account for ramp time (usually 3–6 months before full productivity)? - What's the burn impact if headcount leads revenue by 6 months? ### Fundraising Plan - What's your fallback if the lead investor passes? - Have you modeled the timeline if it takes 6 months, not 3? - What's your runway at current burn if the round closes at the low end? - What assumptions break if you raise 50% of the target amount? --- ## The Hardest Questions These are the ones people skip: - "What's the bear case, not the base case?" - "If this exact plan was run by a team we don't trust, would it work?" - "What are we not saying out loud because it's uncomfortable?" - "Who has incentives to make this plan sound better than it is?" - "What would an enemy of this plan attack first?" --- ## Deliverable The output of `/em:challenge` is not permission to stop. It's a vulnerability map. Now you can make conscious decisions: validate the risky assumptions, hedge the critical ones, or accept the bets you're making knowingly. Unknown risks are dangerous. Known risks are manageable.
Tư vấn Chief Customer Officer: phân tích giữ chân, phân khúc khách hàng, mô hình phủ CSM và tổ chức CS.
---
name: "chief-customer-officer-advisor"
description: "Chief Customer Officer advisory for startups: retention decomposition (gross retention vs NRR honesty, churn root-cause taxonomy), customer segmentation strategy (differential investment across tiers + ICP fit scoring), CS team coverage model (pooled vs named CSM thresholds + ratio math), and CS team org evolution (CS vs Support vs AM distinctions). Use when designing retention strategy, segmenting customers for differential investment, sizing CS team, or sequencing CS hires. Strategic only — does not duplicate engineering/business-growth tactical skills."
license: MIT
metadata:
version: 1.0.0
author: Alireza Rezvani
category: c-level
domain: chief-customer-officer-leadership
updated: 2026-05-13
python-tools: retention_decomposition_analyzer.py, customer_segmentation_designer.py, cs_coverage_calculator.py
frameworks: retention-decomposition, customer-segmentation, cs-coverage-model, cs-team-org
---
# Chief Customer Officer Advisor
Strategic customer leadership for startup CCOs and founders without one. **Four decisions, no generic CS survey:**
1. **What's our retention architecture — and is gross retention vs NRR honest?** — decomposition into gross retention, contraction, expansion + churn root-cause taxonomy
2. **How do we segment customers for differential investment?** — tier design + ICP fit scoring + investment-per-segment math
3. **What's the CS team's coverage model — and when do we go pooled vs named?** — coverage ratio calculator + transition thresholds
4. **What CS role do we hire next?** — stage-to-role map (CS ≠ Support ≠ AM ≠ Implementation)
This skill does **not** cover tactical CS implementation. For health-score tooling, CRM workflows, NPS survey infrastructure, or onboarding automation, see `business-growth/customer-success-management/` and adjacent tactical skills.
## Keywords
CCO, chief customer officer, customer success, retention strategy, gross retention, net retention, NRR, GRR, logo retention, dollar retention, churn, contraction, expansion, downsell, customer lifetime value, CLV, LTV, time-to-value, TTV, time-to-first-value, customer health score, NPS, CSAT, customer effort score, segmentation, ICP fit, tier design, low-touch, high-touch, tech-touch, pooled CSM, named CSM, customer success manager, account manager, AM, implementation manager, IM, customer success operations, CS ops, book of business, ratio, ARR-per-CSM, customer marketing, advocacy, expansion playbook, voice of customer, VoC
## Quick Start
```bash
# Decision A: Decompose retention honestly
python scripts/retention_decomposition_analyzer.py # embedded B2B SaaS sample
python scripts/retention_decomposition_analyzer.py path/to/cohorts.json
# Decision B: Design customer segmentation + differential investment
python scripts/customer_segmentation_designer.py # embedded 4-tier sample
python scripts/customer_segmentation_designer.py path/to/customers.json
# Decision C: Calculate CS team coverage model
python scripts/cs_coverage_calculator.py # embedded 350-customer sample
python scripts/cs_coverage_calculator.py path/to/book.json
```
## Key Questions (ask these first)
- **What's your GROSS retention rate?** (Not NRR — NRR hides churn behind expansion. Ask gross first.)
- **What's the #1 reason customers leave?** (If you can't name it, you don't understand churn.)
- **What's the median time-to-value (TTV) by segment?** (Long TTV in low tier = misfit; long TTV in high tier = onboarding broken.)
- **Which customer would you fire today?** (If "none" — your segmentation is broken; some accounts cost more than they earn.)
- **What's your ARR-per-CSM ratio, and what's the model — pooled or named?** (Stage and ACV determine the right answer.)
- **Is CS in your comp plan, and how is it different from Sales comp?** (CS comp on retention; misalignment is a leading indicator of failure.)
## Core Responsibilities
### 1. Retention Decomposition
**The trap:** "Our NRR is 115%, retention is great."
The truth: NRR = Gross Retention − Contraction + Expansion. A 115% NRR with 85% gross retention is a leaky bucket masked by upsells. A 115% NRR with 98% gross retention is a healthy product.
**Mandatory decomposition every quarter:**
| Metric | What it measures | Health threshold (B2B SaaS) |
|---|---|---|
| **Gross Retention (GRR)** | $ from existing customers minus churn + contraction | ≥ 90% at growth stage; ≥ 95% at scale |
| **Logo Retention** | % of customers who renewed | ≥ 85% at growth; ≥ 90% at scale |
| **Net Revenue Retention (NRR)** | GRR + expansion | ≥ 110% at growth; ≥ 120% at scale |
| **Contraction** | $ from existing customers reducing seats/usage | < 5% annually |
| **Expansion** | $ from existing customers growing | 15-25% annually at healthy |
**Run** `retention_decomposition_analyzer.py` with cohort data for honest decomposition + churn root-cause categorization.
See `references/retention_decomposition.md` for the 7-category churn taxonomy + leading indicator playbook.
### 2. Customer Segmentation
**The trap:** "Every customer is important."
The reality: customers exist on a spectrum of ICP fit × strategic value. Treating them identically wastes CS capacity and ignores expansion opportunity.
**4-tier framework (B2B SaaS baseline):**
| Tier | ARR range | Coverage | Investment per account/yr |
|---|---|---|---|
| **Strategic** | Top 5%, often $100K+ | Named CSM + executive sponsor | $20K-50K |
| **Enterprise** | Next 15-20%, $20K-100K | Named CSM | $5K-15K |
| **Mid-market** | Next 30-40%, $5K-20K | Pooled CSM + automation | $1K-3K |
| **SMB / Long-tail** | Bottom 40-50%, <$5K | Tech-touch + self-serve | $50-500 |
**Run** `customer_segmentation_designer.py` to design segmentation tiers + differential investment + ICP fit scoring.
See `references/customer_segmentation_strategy.md` for ICP fit framework, tier transition triggers, and the kill list (customers below the investment floor).
### 3. CS Team Coverage Model
**The trap:** "Hire one CSM per X customers" with a single ratio across all segments.
The reality: coverage model depends on segment, ACV, and complexity. Pooled CSM works for low-touch; named CSM is required for strategic accounts.
**Coverage models:**
| Model | Best for | Ratio (ARR-per-CSM) | Trade-offs |
|---|---|---|---|
| **Tech-touch (no human)** | SMB, low ACV | $5M-15M+ | Automation cost; cannot save high-stakes deals |
| **Pooled CSM** | Mid-market | $2M-5M | Lower cost; less account intimacy |
| **Named CSM** | Enterprise | $500K-2M | Higher cost; deeper relationships |
| **Named CSM + exec sponsor** | Strategic | $300K-1M | Highest cost; reserved for top accounts |
**Run** `cs_coverage_calculator.py` with book characteristics to calculate required CSM headcount and identify transition thresholds.
See `references/cs_coverage_model.md` for ratios, ramp curves, and the "when to add a manager" trigger.
### 4. CS Team Org Evolution
**The wrong question:** "Should we hire a CSM or a Support engineer?"
**The right question:** "What's the next customer outcome we're failing to deliver, and what role unblocks that?"
**Critical distinctions (founders confuse these):**
| Role | Owns | Does NOT own |
|---|---|---|
| Customer Support | Reactive issue resolution (ticket queue) | Renewal, expansion, success outcomes |
| Customer Success Manager | Proactive value realization + renewal + expansion lead | Day-to-day tickets, implementation |
| Account Manager | Commercial relationship + expansion close | Day-to-day success, technical depth |
| Implementation Manager | Onboarding + go-live | Ongoing success after launch |
| CS Operations | Tooling, data, analytics, playbooks | Direct customer relationships |
| Customer Marketing | Advocacy, case studies, references | 1:1 customer relationships |
See `references/cs_team_org_evolution.md` for stage-to-role map (seed → late-stage) + the AM-vs-CSM split decision.
## Workflows
### Workflow 1: Quarterly Retention Review (4 hours)
**Goal:** Decompose retention honestly + identify top-3 churn drivers.
```bash
# 1. Pull cohort data: closed/won by quarter for last 8 quarters
python scripts/retention_decomposition_analyzer.py cohorts.json
# 2. Review GRR / NRR / contraction / expansion separately
# 3. For each cohort showing GRR < 90%: identify churn root cause (7-category taxonomy)
# 4. Cross-check with cs-cro-advisor: does the expansion math add up?
# 5. Cross-check with cs-cpo-advisor: are product gaps driving churn?
# 6. Output: top-3 leakage points + 90-day mitigation plan
```
### Workflow 2: Customer Segmentation Audit (1 day)
**Goal:** Re-segment customer base + reset differential investment.
```bash
# 1. Build customers.json with ARR, tenure, ICP fit signals
python scripts/customer_segmentation_designer.py customers.json
# 2. Identify segment migration (mid-market → enterprise upgrades, downsells)
# 3. Identify kill list (customers below investment floor)
# 4. Output: new tier assignment + investment-per-tier + kill list for sales review
```
### Workflow 3: CS Team Sizing (1 week)
**Goal:** Size the CS team aligned to book composition + coverage model.
```bash
# 1. Build book.json with current customer base + planned acquisition
python scripts/cs_coverage_calculator.py book.json
# 2. Calculate required CSM headcount by segment
# 3. Compare to current team; identify gaps
# 4. Cross-check with cs-chro-advisor on comp + leveling
# 5. Cross-check with cs-cfo-advisor on the cost
# 6. Output: 12-month hiring plan + role sequence
```
### Workflow 4: CS Team Roadmap (1 week)
**Goal:** Sequence next 18 months of CS hires aligned to customer outcomes.
1. List top 5 customer outcomes the company is failing to deliver
2. Map each outcome to the role that unblocks it (CSM / AM / IM / Support / CS Ops)
3. Sequence hires; respect prerequisite order
4. Cross-check with cs-chro-advisor
## Output Standards
```
**Bottom Line:** [one sentence — decision and rationale]
**The Decision:** [one of: retention | segmentation | coverage | next hire]
**The Evidence:** [numbers from the tool, not adjectives]
**How to Act:** [3 concrete next steps]
**Your Decision:** [the call only the founder can make]
```
## Adjacent Skills
- `../cro-advisor/` — Revenue math, NRR, expansion comp (CCO owns customer experience; CRO owns revenue math; clean split)
- `../cpo-advisor/` — Product strategy, JTBD (CCO surfaces product gaps; CPO decides roadmap)
- `../cmo-advisor/` — Customer marketing, advocacy, references
- `../cfo-advisor/` — CS team cost, retention-impact-on-revenue math
- `../chro-advisor/` — CS team hiring + leveling
- `../../../business-growth/` — Tactical CS execution: health scores, CRM workflows, onboarding tooling
## References
- [retention_decomposition.md](references/retention_decomposition.md) — GRR vs NRR honest math + 7-category churn taxonomy + leading indicator playbook
- [customer_segmentation_strategy.md](references/customer_segmentation_strategy.md) — 4-tier framework + ICP fit scoring + tier transition triggers + kill list criteria
- [cs_coverage_model.md](references/cs_coverage_model.md) — Coverage model decision (tech-touch / pooled / named / named+exec) + ratio benchmarks + manager-trigger
- [cs_team_org_evolution.md](references/cs_team_org_evolution.md) — Stage-to-role map + 6-role definition table (CSM ≠ Support ≠ AM ≠ IM ≠ CS Ops ≠ Customer Marketing) + AM-vs-CSM split decision + anti-patterns
---
**Version:** 1.0.0
**Status:** Production Ready
**Disclaimer:** Retention benchmarks vary significantly by ACV, segment, and industry. This skill provides B2B SaaS-baseline guidance; consumer SaaS, marketplaces, and hardware all have materially different retention math.
FILE:references/cs_coverage_model.md
# CS Coverage Model — The Decision: "How do we cover our customer base — and when do we add CSMs?"
This reference answers exactly one decision: **what coverage model do we use, what's the ratio, and when do we add headcount?**
Pair with `scripts/cs_coverage_calculator.py` for automation.
## The Four Coverage Models
### Tech-Touch (no human CSM)
- **Best for:** SMB / long-tail, ACV < $5K, high-volume PLG products
- **Ratio:** Often $5M-$15M ARR per CSM-equivalent (a single CSM handles escalations only)
- **How it works:** Self-serve onboarding, in-product guidance, lifecycle email automation, community support
- **Tooling stack:** Pendo / Appcues / Userpilot (in-product), Customer.io / HubSpot (email), Discourse / Slack community
**Trade-offs:**
- Lowest cost per customer
- Cannot save high-stakes deals; tech-touch customers churn silently
- Requires investment in product onboarding UX and content
- Escalation path must exist — when a tech-touch account becomes valuable, a human takes over
### Pooled CSM (1:many)
- **Best for:** Mid-market, ACV $5K-$20K
- **Ratio:** $2M-$5M ARR per CSM; 50-150 accounts per CSM
- **How it works:** One CSM owns a pool of accounts; automation triggers proactive outreach; reactive when customers ask
- **Hallmarks:** Quarterly automated check-ins, library of playbooks, on-demand 1:1 when triggered
**Trade-offs:**
- Lower cost than named
- Less account intimacy; CSMs don't know all 100 customers deeply
- Works well only with strong CS Ops + health-score automation
- Burnout risk if pool grows too large
### Named CSM (1:few)
- **Best for:** Enterprise, ACV $20K-$100K
- **Ratio:** $500K-$2M ARR per CSM; 20-30 accounts per CSM
- **How it works:** Each customer has a named CSM who knows their business; weekly to monthly cadence; CSM owns the renewal
- **Hallmarks:** Account plans, QBRs, named relationship with customer contacts
**Trade-offs:**
- Standard for enterprise SaaS
- Higher cost (~$180K fully-loaded per CSM)
- CSM ramp time 3-6 months; turnover is expensive
- Named CSMs become single point of failure if they leave
### Named CSM + Executive Sponsor
- **Best for:** Strategic accounts, ACV $100K+
- **Ratio:** $300K-$1M ARR per CSM; 5-10 accounts per CSM; exec sponsor allocates 4-8 hrs/quarter per account
- **How it works:** Named CSM handles tactical relationship; executive sponsor handles strategic + reputation + escalation
- **Hallmarks:** EBRs with customer C-suite, custom roadmap input, multi-year contracts
**Trade-offs:**
- Highest cost (CSM + 5-10% of an exec's time)
- Reserved for top accounts where loss would be material to the company
- Exec sponsor must actually engage — ceremonial sponsorship destroys trust
## Choosing the Model per Segment
Rule of thumb: model follows segment, segment follows ARR + ICP fit.
| Segment | Default model | Override when |
|---|---|---|
| Strategic (top 5%) | Named + exec sponsor | Always — the cost is justified by retention + reference value |
| Enterprise (15-20%) | Named CSM | Downgrade to pooled if ACV barely qualifies AND tenure stable |
| Mid-market (30-40%) | Pooled CSM | Upgrade to named if customer is on Strategic-upgrade trajectory |
| SMB / Long-tail (40-50%) | Tech-touch | Upgrade to pooled if expansion potential is exceptional |
## The Ratio Math
ARR-per-CSM is the most-cited CS metric. It's a useful starting point but **not a target**.
**What "ARR-per-CSM" actually measures:** the ratio of revenue under a CSM's responsibility. Higher = more leveraged; lower = more intimate.
**Ratios by stage (B2B SaaS baseline):**
| Stage | Strategic | Enterprise | Mid-market | SMB |
|---|---|---|---|---|
| Seed | n/a | $300K-$800K | $1M-$3M | n/a |
| Series A | $500K-$1M | $800K-$1.5M | $2M-$4M | $5M+ |
| Series B / Growth | $700K-$1.5M | $1M-$2M | $3M-$5M | $8M+ |
| Late-stage | $1M-$2M | $1.5M-$3M | $4M-$8M | $15M+ |
**Industry variation:**
- **Lower ratios (more CSM density needed):** complex products, regulated industries, customer success critical to expansion
- **Higher ratios (more leverage possible):** simple products, low-complexity workflows, strong product UX
## When to Add a CSM
Two independent triggers:
1. **By ARR:** total tier ARR exceeds (current_csm_count × target_ratio + 20% buffer)
- The 20% buffer absorbs ramp time of new hires
- Don't wait until existing CSMs are at 100% capacity to hire
2. **By account count:** total tier accounts exceeds (current_csm_count × accounts_cap)
- Named CSM cap is ~25 accounts; beyond that, attention degrades
- Pooled CSM cap is ~150 accounts; beyond that, automation must increase
**Whichever triggers first.** Run `cs_coverage_calculator.py` quarterly.
## When to Add a Manager
A CS manager is needed when **any of these become true:**
1. **5+ ICs in a single tier:** the original CSM lead can no longer code AND manage
2. **8+ CSMs across the entire CS function:** spans of control exceed comfortable management
3. **CS is escalating to CTO/CEO for non-product issues weekly:** clear leadership gap
**Manager profile:**
- Internal promotion preferred (knows the playbooks)
- Strong on people management + cross-functional skills
- Has run a CS book themselves; not a pure people manager
## Ramp Curve
New CSMs are not productive at hire.
| Tier | Time to 50% productive | Time to fully productive |
|---|---|---|
| Strategic | 3 months | 6-9 months |
| Enterprise | 2 months | 4-6 months |
| Mid-market | 1 month | 2-3 months |
| SMB / Tech-touch | 2 weeks | 1 month |
**Operational implication:** hire 90 days BEFORE you need the capacity, not when you're already underwater.
## CS Comp Design
CS comp aligned to retention + expansion is the standard.
**Common structure (named CSM):**
- 70% base salary + 30% variable
- Variable split:
- 50% of variable on gross retention (renewals)
- 30% on net retention (expansion)
- 20% on activity (QBRs completed, health-score green %, etc.)
**Critical anti-pattern:** comp CSMs on "customer happiness" or NPS only. They game it and don't drive renewals.
**Pooled CSM comp:** more weight on activity + automation health, less on individual account outcomes (which are statistical at this volume).
## When This Reference Doesn't Help
- **CS technology stack selection (Gainsight, ChurnZero, Vitally, etc.).** Tactical; see CS Ops resources.
- **Health-score formula design.** Tactical; depends on product data model.
- **Comp negotiation with individual CSMs.** HR / management territory.
This reference is about the strategic decision of coverage model + ratio + hiring trigger, not the operational implementation.
---
**Source authorities (non-exhaustive):**
- Gainsight — "CS Maturity Model" + state-of-the-industry reports
- TSIA (Technology Services Industry Association) — annual CS benchmarks including ARR-per-CSM by segment
- Nick Mehta, Allison Pickens — "The Customer Success Economy" (Wiley, 2020)
- ChurnZero — "CS Salary Survey" annual report (CSM comp benchmarks)
- David Skok — SaaS Metrics 2.0 (CAC payback economics that fund CS)
- Lincoln Murphy — extensive writing on pooled vs named models
- Pacific Crest / KeyBanc Capital Markets — annual SaaS survey including CS-as-% of revenue benchmarks
FILE:references/cs_team_org_evolution.md
# CS Team Org Evolution — The Decision: "What CS role do we hire next, and how is CS different from Support / AM / IM?"
This reference answers exactly one decision: **for our stage and the customer outcomes we're failing to deliver, what is the next CS role to hire?**
## The Wrong Question
> "Should we hire a CSM or a Support engineer?"
This is the wrong question. Most CSMs and Support engineers hired at the wrong stage cannot deliver value because:
- The role they're hired into doesn't match the customer outcomes being missed
- The infrastructure (CRM, health scores, playbooks) isn't ready for them to be productive
- Founders confuse the four customer-facing roles and hire the wrong one
## The Right Question
> "What customer outcome are we failing to deliver, and which role unblocks that?"
This shifts hiring from role-taxonomy to outcome-shipping. CS org grows in response to specific failure modes.
## The Six Customer-Facing Roles (founders confuse these)
| Role | Owns | Does NOT own |
|---|---|---|
| **Customer Support** | Reactive issue resolution (ticket queue); product knowledge; first response | Renewal, expansion, strategic relationship, proactive outreach |
| **Customer Success Manager (CSM)** | Proactive value realization + renewal + expansion lead | Day-to-day support tickets, technical implementation |
| **Account Manager (AM)** | Commercial relationship + expansion close + contract negotiation | Day-to-day success, technical depth, ticket resolution |
| **Implementation Manager (IM)** | Onboarding + go-live + first-value delivery | Ongoing success after launch (hands off to CSM) |
| **CS Operations (CS Ops)** | Tooling, data, analytics, playbooks, health scores | Direct customer relationships |
| **Customer Marketing** | Advocacy, case studies, references, customer events | 1:1 customer relationships, renewal/expansion |
**The most common confusions:**
- **CSM = Support:** No. CSMs do proactive value realization. Support is reactive.
- **CSM = AM:** Some companies combine; risky. CSM lens is success outcomes; AM lens is commercial.
- **CSM = Implementation:** No. Implementation is launch-bounded; CSM is ongoing.
## The Five Stages
### Stage 1: Pre-PMF / Pre-seed / Seed
**Team size:** 1-15 people. **CS team:** 0 dedicated.
**Reality:** Founder does customer success. Every customer is hand-held by a co-founder. This is fine and even useful — customer obsession is the right founder behavior at this stage.
**Don't hire:** CSM, Support engineer, AM. Premature.
**Tooling:** Direct customer Slack channels, email, weekly founder check-ins. No CRM needed beyond a spreadsheet.
**When to move to stage 2:** Founder is spending >40% of week on customer issues AND has 10+ paying customers AND can articulate the post-sale playbook clearly.
### Stage 2: Series A
**Team size:** 15-50 people. **CS team:** 1-3.
**First hire: Customer Success Manager (NOT Support engineer first).**
Why: at this stage the biggest leakage is proactive value realization, not ticket volume. CSM handles onboarding, renewal preparation, expansion identification.
Profile:
- 3-5 years experience in B2B SaaS CS
- Strong product fluency (can demo and explain)
- Comfortable with ambiguity (playbooks don't exist yet — they'll build them)
**Second hire: Customer Support engineer / specialist.**
Why: once you have 30+ paying customers, ticket volume becomes real. Support handles the reactive load so CSMs can stay proactive.
Profile:
- Strong technical aptitude + customer empathy
- Comfortable with the product
- Documentation-oriented (will build the knowledge base)
**Third hire: Implementation specialist (often part-time / shared with CSM).**
Why: at higher ACVs, onboarding is its own discipline. Bad onboarding kills retention before the customer ever sees the product's value.
**Don't hire yet:** AM (CSM handles renewals), CS Ops (CSMs do their own ops), Customer Marketing.
**When to move to stage 3:** 100+ paying customers, $1M+ ARR, 3+ CSMs, segmentation tiers are real.
### Stage 3: Series B
**Team size:** 50-200. **CS team:** 4-10.
**Fourth hire: CS Manager (internal promotion).**
Why: 4+ CSMs need a manager. Original CSM lead should be promoted internally; external hires miss the playbook context.
**Fifth hire: CS Operations.**
Why: by Series B, CSMs are spending 30%+ of their time on tooling, reporting, and data work. CS Ops centralizes this; CSMs get their time back for customer-facing work.
Profile:
- Analytical (SQL + spreadsheets minimum; ideally light scripting)
- Has run CRM workflows (Gainsight, ChurnZero, Vitally, or even just Salesforce reports)
- Builds health scores, playbook automation, exec dashboards
**Sixth hire (conditional): Account Manager — separate from CSM.**
Trigger:
- CSMs are good at success but bad at commercial (renewals delayed, expansion under-closed)
- ACV justifies a dedicated commercial role (Enterprise+ segment)
- Multi-product company where cross-sell motion is distinct
Profile: closer / commercial DNA, NOT a success person. AM owns the contract; CSM owns the relationship and success outcomes.
**Seventh hire (conditional): Customer Marketing.**
Trigger:
- 5+ public reference customers
- Conference / event presence needed
- Advocacy is a strategic priority
**When to move to stage 4:** 250+ customers, $5M+ ARR, multiple segment tiers, CS team is 8+ people.
### Stage 4: Growth (Series C / pre-IPO)
**Team size:** 200-1000. **CS team:** 10-50.
**Director / VP CS.**
Triggers:
- CS team is 10+
- CS is a board-level conversation (NRR is in the company narrative)
- CS strategy needs an executive who isn't the founder
Profile: has run CS org at $20M+ ARR, scaled CS through hyper-growth, has comp + ladder + comp-plan design experience.
**Tier-specific specialization:**
By this stage, CSM roles should specialize:
- Strategic CSM: senior, multi-account, executive-facing
- Enterprise CSM: standard CSM career path
- Mid-market CSM: pooled coverage, automation-heavy
- SMB / tech-touch lead: 1 CSM owns the entire long-tail
**Implementation team scaled separately:** dedicated Implementation Managers for Strategic + Enterprise, hand-offs to CSMs at go-live.
**Add: Renewals team (optional but common at growth stage).**
Trigger: CSMs are losing focus on success outcomes because renewal-cycle work consumes them. Dedicated Renewals team takes contract management; CSMs stay on success.
### Stage 5: Late-stage (Series D+, post-IPO)
**Team size:** 1000+. **CS team:** 50-300+.
**CCO promotion or hire.**
Triggers:
- CS is in the company strategic narrative
- Customer experience as a whole (CS + Support + Marketing + Product feedback loops) needs a single leader
- Multi-product portfolio needs unified customer view
CCO profile:
- Has run CS / CX at scale ($100M+ ARR)
- Strong on cross-functional (product, marketing, sales) collaboration
- Comfortable with board-level reporting on retention
**Customer Operations (CustOps) as a unified function.**
Combines: CS Ops + Support Ops + Customer Marketing Ops + Customer Data infra. Centralized, serves all customer-facing teams.
**Federated CSM model.**
CSMs embed in product lines / verticals / geographies. Central CS function provides playbooks + tooling + governance; embedded CSMs deliver day-to-day.
## The AM vs CSM Split Decision
The single most-debated CS org question.
**When to split (separate AM and CSM):**
- ACV $20K+ (Enterprise+)
- CSMs hate commercial work and are losing renewals
- Multi-product cross-sell motion is distinct from success outcomes
- Sales-led GTM model (AM is a natural extension of the AE)
**When NOT to split (CSM owns commercial):**
- Mid-market and below
- PLG / self-serve motion
- Small CS team where context-switching cost is low
- Founder still close enough to deals
**The hybrid (most common):**
- CSM owns relationship + renewal
- AM exists ONLY for expansion close (when complex commercial work justifies a closer)
- AM commission split between CSM (who identified) and AM (who closed)
## Anti-Patterns
- **Hiring Support as the first CS hire.** Support solves a problem you may not yet have at sub-50 customers; CSM solves a problem you have at day one (proactive value).
- **Hiring CS Ops before CSMs.** Premature; nothing to operate. CS Ops emerges from the friction CSMs experience.
- **Promoting the top CSM to manager without training.** Best ICs often fail as managers; provide management training or external hire.
- **CSM + AM combined indefinitely.** Works at sub-$5M ARR; breaks above. Plan the split before it becomes a crisis.
- **CSM = "Support Plus."** Tickets routed to CSMs because "they know the customer best" destroys CSM proactive time. Strict ticket routing to Support.
- **Treating Customer Marketing as a CS extension.** Different discipline; reports up through Marketing, not CS, in most healthy orgs.
- **Hiring a CCO at sub-$10M ARR.** Political role; nothing to operate. Wait until the function justifies an executive.
## The Hiring Sequencing Rule
Never hire the next CS role until:
1. The current role is filled and ramped (3-6 months in seat)
2. That role has shipped a specific customer outcome
3. You can name the gap the next hire will fill
**The discipline:** every CS hire ties to a specific customer outcome the business is currently failing to deliver.
## When This Reference Doesn't Help
- **Comp benchmarking for specific roles.** See `c-level-advisor/skills/chro-advisor/scripts/comp_benchmarker.py`.
- **Leveling ladders.** See `c-level-advisor/skills/chro-advisor/references/leveling_ladders.md`.
- **CS Ops tooling selection (Gainsight, ChurnZero, Vitally, etc.).** Tactical; not strategic.
- **Performance management.** Standard people management.
This reference is about strategic CS team evolution as a function of customer outcomes, not HR mechanics.
---
**Source observations (non-exhaustive):**
- Nick Mehta, Dan Steinman, Lincoln Murphy — "Customer Success" (Wiley, 2016)
- Nick Mehta, Allison Pickens — "The Customer Success Economy" (Wiley, 2020) — chapters on org evolution
- Bessemer Venture Partners — "State of the Cloud" annual report (CS-as-% of revenue benchmarks)
- TSIA — annual CS benchmarks including org structure across SaaS stages
- Gainsight — Pulse conference talks on org maturity
- Direct observations from 30+ B2B SaaS CS org evolutions, 2018-2026
- ChurnZero — annual CS salary + ratio surveys
- Lincoln Murphy — extensive blog writing on AM vs CSM split
FILE:references/customer_segmentation_strategy.md
# Customer Segmentation Strategy — The Decision: "How do we invest differently across customers?"
This reference answers exactly one decision: **which customers get how much investment from CS — and why?**
Pair with `scripts/customer_segmentation_designer.py` for automation.
## The Failure Mode
> "We treat all our customers equally."
This is operationally false (you can't) and strategically wrong (you shouldn't). Equal treatment means:
- Strategic accounts get under-served (executive sponsorship goes to whoever's loudest)
- SMB accounts get over-served (high-touch CS time that destroys unit economics)
- Misfit accounts consume resources that should fund the next strategic acquisition
The discipline is **differential investment**: more CS time and budget per dollar of ARR for high-fit, high-value accounts; less or none for low-fit, low-value accounts.
## The 4-Tier Framework
Standard B2B SaaS framework. ARR ranges are baseline; adjust for your ACV distribution.
### Tier 1: Strategic
- **ARR range:** Top 5% of accounts, typically $100K+
- **% of customers:** ~5%
- **% of ARR:** often 30-50% (Pareto distribution)
- **Coverage model:** Named CSM + executive sponsor + dedicated implementation
- **Investment per account/yr:** $20K-50K (CSM time + exec time + custom work)
- **Examples:** Top 10 logos by ARR, design-partner accounts, public-reference customers
**Hallmarks:**
- Multi-year contracts with QBRs / EBRs
- Custom integrations, API support, prioritized roadmap input
- Executive sponsor on the customer side AND on yours
- Reference + advocacy expected
### Tier 2: Enterprise
- **ARR range:** Next 15-20%, typically $20K-$100K
- **% of customers:** ~15-20%
- **% of ARR:** often 25-35%
- **Coverage model:** Named CSM
- **Investment per account/yr:** $5K-15K
- **Examples:** Mid-sized companies, departmental deployments at large companies
**Hallmarks:**
- Annual contracts, quarterly check-ins
- Standard integrations
- Single primary CSM, no executive sponsor unless escalated
### Tier 3: Mid-Market
- **ARR range:** Next 30-40%, typically $5K-$20K
- **% of customers:** ~30-40%
- **% of ARR:** ~15-25%
- **Coverage model:** Pooled CSM + automation (1:many)
- **Investment per account/yr:** $1K-3K
- **Examples:** Growing SMBs, smaller departmental deployments
**Hallmarks:**
- Pooled CSM model: one CSM owns 50-150 accounts, automation triggers human touch
- Annual contract auto-renew default
- Self-serve onboarding with optional human support
- Standard health scoring + trigger-based intervention
### Tier 4: SMB / Long-Tail
- **ARR range:** Bottom 40-50%, typically <$5K
- **% of customers:** ~40-50%
- **% of ARR:** often <10%
- **Coverage model:** Tech-touch + self-serve
- **Investment per account/yr:** $50-500 (mostly automation cost)
- **Examples:** Solo users, small teams, freemium/PLG converts
**Hallmarks:**
- Fully self-serve onboarding
- Email-based + community-based support
- 1 CSM for the entire tier (escalation handler only)
- Monthly or annual contracts; high price sensitivity
## ICP Fit Scoring (0-10 weighted)
Segmentation by ARR alone is incomplete. A $50K customer with poor ICP fit may cost more than they earn. Layer ICP fit on top.
**Recommended weighting:**
| Signal | Weight | Why |
|---|---|---|
| in_target_industry | 2.0 | Industry fit drives product-market fit |
| in_target_size_range | 1.5 | Wrong size = wrong feature requirements |
| uses_target_workflow | 2.0 | Workflow fit is the strongest retention predictor |
| has_executive_sponsor | 1.5 | Single-threaded accounts churn 3-5x more |
| advocates_publicly | 1.0 | Public advocacy is a strong forward signal |
| expansion_potential_high | 1.0 | Existing customers ARE the next round of revenue |
| competitor_concentration_low | 1.0 | High competitor concentration = price war risk |
**Score interpretation:**
| Score | Meaning |
|---|---|
| 8-10 | Strong ICP fit; invest aggressively, regardless of current ARR |
| 5-7 | Decent fit; standard tier investment |
| 0-4 | Poor fit; consider tech-touch only, or kill list |
## The Kill List (politically difficult, financially obvious)
**Kill candidate criteria** (any one is a yellow flag; two or more is a kill):
- ICP fit score < 5
- Annual support cost > 50% of ARR
- Tenure < 12 months AND multiple escalations
- Customer's company has recently been acquired by a larger conflicting entity
- Customer is in a declining industry / shutting down
**The 3 paths for kill candidates:**
1. **Do not renew.** Send a polite non-renewal communication 60-90 days before contract end.
2. **Downgrade to tech-touch.** Remove CSM coverage; let the customer self-serve. Many will churn naturally; some will stick if the product is actually serving them.
3. **Raise price to cost-recover.** Make the renewal pricing reflect the real cost of serving them. If they accept, great. If they leave, also fine.
**Anti-pattern:** "Strategic accounts" that are actually kill candidates. Founders often protect their first 5-10 customers far past the point of economic sense. Quarterly audits force the conversation.
## Tier Transition Triggers
Customers migrate between tiers. Standard triggers:
- **SMB → Mid-market:** ARR grows above $5K AND tenure > 12 months AND ICP fit ≥ 6
- **Mid-market → Enterprise:** ARR grows above $20K AND has dedicated executive contact
- **Enterprise → Strategic:** ARR above $100K AND multi-year deal AND expansion potential AND named exec sponsor on both sides
- **Down-tier:** ARR drops below tier floor OR ICP fit drops AND quarterly review confirms
**Operational discipline:** quarterly tier review forced for every customer above $5K. Below $5K, automation handles tier assignment.
## Why Segmentation Is Strategic, Not Operational
Segmentation seems like an ops question ("how do we organize the book?"). It's actually a strategic question: **which customers does the company exist to serve?**
A segmentation that has 70% of customers in the "Strategic" tier means the company isn't choosing — and likely is over-investing in the long tail relative to ARR concentration. A segmentation with 70% in "SMB / long-tail" means the company is a PLG/SMB business and should design CS, product, and pricing accordingly.
**Segmentation = strategy in operational form.** Get it wrong, and your CS team, product roadmap, and pricing all misfire.
## When This Reference Doesn't Help
- **Setting up segmentation in your CRM.** Tactical; use Salesforce / HubSpot / etc. native tier fields.
- **ICP refinement when product-market fit is unclear.** See `c-level-advisor/skills/cpo-advisor/` for PMF framework first.
- **Pricing strategy across tiers.** See `c-level-advisor/skills/cmo-advisor/` and consider Patrick Campbell's "Monetizing Innovation".
This reference is about the strategic design of differential investment, not the CRM implementation.
---
**Source authorities (non-exhaustive):**
- Lincoln Murphy — "Customer Success" (Wiley, 2016) + extensive blog on segmentation
- Bain & Co. — "Net Promoter System" research on differential treatment of "promoters"
- Bain — "The Loyalty Effect" (Reichheld) — economics of long-term customer value
- Tomasz Tunguz (Redpoint) — multiple essays on tiered CS coverage
- David Skok — SaaS Metrics 2.0 on the Pareto distribution of revenue and the long-tail problem
- ChartMogul / ProfitWell SaaS benchmarks — distribution of customers by ACV across SaaS companies
- Adamson, Dixon, Toman — "The Challenger Customer" (Portfolio, 2015) — buying-center concentration and CS implication
FILE:references/retention_decomposition.md
# Retention Decomposition — The Decision: "Is our retention number honest?"
This reference answers exactly one decision: **what does our retention number actually mean, and where is the leakage?**
Pair with `scripts/retention_decomposition_analyzer.py` for automation.
## The Vanity Trap
> "Our NRR is 115%, retention is great."
Wrong question. NRR can hide a leaky bucket: 85% gross retention + 30% expansion from existing customers = 115% NRR. The product is failing for 15% of paying customers; expansion from the survivors is masking the failure.
**Always decompose:**
```
NRR = Gross Retention (GRR) − Contraction + Expansion
```
If GRR < 85% but NRR > 100%, you have a **leaky bucket**. Acquisition spend keeps the metric up; eventually expansion can't outrun churn.
## The Honest Metrics
### Gross Revenue Retention (GRR)
**Definition:** Of the ARR that existed at the start of period N, how much remains at the end of period N+1, NOT counting expansion?
**Formula:** `GRR = (starting_arr - churn_arr - contraction_arr) / starting_arr`
**Thresholds (B2B SaaS baseline):**
| Stage | Healthy | Concerning | Critical |
|---|---|---|---|
| Seed / Series A | ≥ 85% | 75-85% | < 75% |
| Series B / Growth | ≥ 90% | 85-90% | < 85% |
| Late-stage / Scale | ≥ 95% | 90-95% | < 90% |
**This is the truth metric.** Without it, you cannot diagnose product-market fit problems.
### Net Revenue Retention (NRR)
**Definition:** GRR plus expansion from existing customers.
**Formula:** `NRR = GRR + (expansion_arr / starting_arr)`
**Thresholds:**
| Stage | Healthy | Concerning | Critical |
|---|---|---|---|
| Seed / Series A | ≥ 100% | 95-100% | < 95% |
| Series B / Growth | ≥ 110% | 100-110% | < 100% |
| Late-stage / Scale | ≥ 120% | 110-120% | < 110% |
**This is the vanity metric in isolation.** Useful only when reported alongside GRR.
### Logo Retention
**Definition:** % of customers (count, not dollars) who renewed.
**Why it matters separately:** dollar retention can stay healthy if you lose lots of small customers and retain big ones. Logo retention exposes whether you're losing the long tail.
**Thresholds:** typically tracks GRR within 3-5 percentage points.
## The 7-Category Churn Taxonomy
Every churned customer falls into one of these categories. Tracking the distribution tells you what to fix.
| Category | Definition | Preventable? | Fix |
|---|---|---|---|
| **product_fit** | Product didn't solve the customer's actual JTBD | Mostly yes (long term) | Sharpen ICP, fix onboarding mismatch, OR accept and price-segment out |
| **competitor_loss** | Lost to a competitor with better fit / price | Partially | Competitive intelligence, product differentiation, pricing review |
| **no_value_realized** | Customer never reached time-to-value; onboarding gap | Yes | Onboarding redesign, milestone tracking, intervention triggers |
| **pricing** | Price-driven churn (too expensive, or perceived as low value) | Sometimes | Price-value re-audit; segmentation; downsell offers vs churn |
| **champion_left** | Internal champion changed roles or left the customer company | Partially | Multi-threading: avoid single-champion dependency |
| **company_event** | M&A, layoffs, shutdown — not your fault | No | Track frequency; if high, your ICP may be unstable |
| **tactical_failure** | Service / support failure — preventable with better CS execution | Yes (always) | CS playbook gaps, response time, escalation paths |
**Preventable churn = product_fit + no_value_realized + tactical_failure.** If preventable churn > 50% of total, your CS function has clear leverage. Below 30%, churn is mostly structural (ICP, market, competitors).
## Leading Indicators (catch churn before it happens)
By the time a customer cancels, you're 60-90 days late. Leading indicators give 30-90 days warning.
**Product engagement signals:**
- Drop in daily active users (DAU) per account (week-over-week trend)
- Drop in "depth of use" — features touched per session
- Drop in API calls (for technical products)
- No login from any user in account for 14+ days
**Commercial signals:**
- Failed payment / payment delay
- Reduction in seat count (often precedes contraction or full churn)
- Champion stops responding to QBR scheduling
- Account team reassignment on customer's side
**Sentiment signals:**
- NPS / CSAT drop > 2 points
- Support ticket volume spike (paradoxically — high engagement, not low)
- Negative sentiment in support tickets (manual or NLP-tagged)
- Public review or social media complaint
**Action:** Build a health score using 3-5 of these. When score crosses threshold, CSM intervention triggers.
## Cohort Analysis: Mandatory Discipline
Pull retention by **acquisition cohort** (quarter or month), not by reporting period. Reporting-period retention mixes cohorts and hides which acquisition vintage is leaky.
**Pattern to watch:**
- Cohort GRR **improves over time** = product quality improving, onboarding maturing
- Cohort GRR **flat** = stable product, no quality regression but no improvement
- Cohort GRR **degrading** = recent cohorts churning faster than older ones → quality regression, ICP drift, or wrong customer acquisition
The third pattern is a critical signal. Acquire less, fix product, or both.
## NPS / CSAT — Use Carefully
NPS is a directional indicator, not a precise measurement. Useful for:
- Trends quarter-over-quarter
- Comparison across segments (e.g., enterprise NPS vs SMB NPS)
- Specific transactional moments (post-onboarding, post-renewal)
NOT useful for:
- Benchmarking against other companies (calculation methodology varies)
- Predicting individual customer churn (better signals exist)
- Single-shot decisions ("our NPS is 35, so we're good")
## When This Reference Doesn't Help
- **Implementing health scores in your CRM.** Tactical; see business-growth/ skills.
- **Setting up NPS survey infrastructure.** Use Delighted, Wootric, Pendo, etc.
- **CS comp design.** See `c-level-advisor/skills/chro-advisor/`.
- **Pricing strategy.** See `c-level-advisor/skills/cmo-advisor/` and consider Patrick Campbell's "Monetizing Innovation" framework.
This reference is about reading retention data honestly, not about gathering it.
---
**Source authorities (non-exhaustive):**
- Nick Mehta, Dan Steinman, Lincoln Murphy — "Customer Success" (Wiley, 2016) — foundational text for the modern CS discipline
- Lincoln Murphy — "Customer Success: Building a Customer Engagement and Retention Framework" — defines GRR/NRR/CHURN clearly
- David Skok (Matrix Partners) — "SaaS Metrics 2.0" (forEntrepreneurs blog) — financial framework for retention math
- Bessemer Venture Partners — "State of the Cloud" annual report — benchmark retention numbers across SaaS stages
- ChartMogul / ProfitWell SaaS Benchmarks — public industry benchmarks for NRR/GRR by stage and ACV
- Reichheld, Fred — "The Loyalty Effect" (HBS Press, 1996) — origin of NPS framework and retention economics
- Tomasz Tunguz (Redpoint) — extensive writing on NRR vs GRR and the leaky bucket pattern
FILE:scripts/cs_coverage_calculator.py
#!/usr/bin/env python3
"""cs_coverage_calculator.py — Calculate CS team headcount per coverage model.
Stdlib-only. Takes a book of business and outputs:
- Required CSM headcount per tier
- Coverage model recommendation (tech-touch / pooled / named / named+exec)
- Manager-trigger threshold (when to add a CS manager)
- 12-month hiring plan if growth_target_pct is provided
Deterministic logic based on ratios + model thresholds.
Input schema (JSON):
{
"book": {
"strategic": {"customer_count": 8, "total_arr_usd": 3200000, "current_csm_count": 1},
"enterprise": {"customer_count": 42, "total_arr_usd": 2100000, "current_csm_count": 2},
"mid_market": {"customer_count": 120, "total_arr_usd": 1080000, "current_csm_count": 1},
"smb_long_tail": {"customer_count": 280, "total_arr_usd": 560000, "current_csm_count": 0}
},
"growth_target_pct": 0.40 # expected book growth in next 12 months
}
Usage:
python cs_coverage_calculator.py # uses embedded sample
python cs_coverage_calculator.py path/to/book.json
python cs_coverage_calculator.py book.json --output json
"""
import argparse
import json
import math
import sys
from typing import Any, Dict, List
SAMPLE: Dict[str, Any] = {
"book": {
"strategic": {"customer_count": 8, "total_arr_usd": 3_200_000, "current_csm_count": 1},
"enterprise": {"customer_count": 42, "total_arr_usd": 2_100_000, "current_csm_count": 2},
"mid_market": {"customer_count": 120, "total_arr_usd": 1_080_000, "current_csm_count": 1},
"smb_long_tail": {"customer_count": 280, "total_arr_usd": 560_000, "current_csm_count": 0},
},
"growth_target_pct": 0.40,
}
# Coverage model ratios (ARR-per-CSM target by tier)
COVERAGE_MODELS = {
"strategic": {
"model": "Named CSM + exec sponsor",
"arr_per_csm_target": 800_000, # mid-range of $300K-$1M ratio
"accounts_per_csm_max": 8, # named coverage cap
"fully_loaded_cost_yr": 220_000, # CSM total comp at strategic
},
"enterprise": {
"model": "Named CSM",
"arr_per_csm_target": 1_200_000, # mid-range of $500K-$2M
"accounts_per_csm_max": 25, # named caps at 20-30
"fully_loaded_cost_yr": 180_000,
},
"mid_market": {
"model": "Pooled CSM + automation",
"arr_per_csm_target": 3_500_000, # mid-range of $2M-$5M
"accounts_per_csm_max": 150, # pooled allows higher count
"fully_loaded_cost_yr": 140_000,
},
"smb_long_tail": {
"model": "Tech-touch + self-serve",
"arr_per_csm_target": 10_000_000, # 1 CSM for escalations only
"accounts_per_csm_max": 1000, # primarily tech-touch
"fully_loaded_cost_yr": 110_000,
},
}
def required_csms(tier_book: Dict[str, Any], model: Dict[str, Any]) -> Dict[str, Any]:
arr = tier_book.get("total_arr_usd", 0)
accounts = tier_book.get("customer_count", 0)
if arr == 0 and accounts == 0:
return {"required": 0, "binding_constraint": "no book"}
by_arr = math.ceil(arr / model["arr_per_csm_target"]) if arr else 0
by_accounts = math.ceil(accounts / model["accounts_per_csm_max"]) if accounts else 0
required = max(by_arr, by_accounts)
binding = "arr" if by_arr >= by_accounts else "accounts"
return {
"required": required,
"by_arr_constraint": by_arr,
"by_accounts_constraint": by_accounts,
"binding_constraint": binding,
}
def analyze(payload: Dict[str, Any]) -> Dict[str, Any]:
book = payload.get("book", {})
growth = payload.get("growth_target_pct", 0)
per_tier = []
total_required_now = 0
total_required_future = 0
total_current = 0
total_cost_now = 0
total_cost_future = 0
for tier_key in ("strategic", "enterprise", "mid_market", "smb_long_tail"):
tier_book = book.get(tier_key, {})
model = COVERAGE_MODELS[tier_key]
req_now = required_csms(tier_book, model)
# Future book (12mo with growth)
future_arr = tier_book.get("total_arr_usd", 0) * (1 + growth)
future_accounts = math.ceil(tier_book.get("customer_count", 0) * (1 + growth))
future_book = {"total_arr_usd": future_arr, "customer_count": future_accounts}
req_future = required_csms(future_book, model)
current = tier_book.get("current_csm_count", 0)
gap_now = req_now["required"] - current
gap_future = req_future["required"] - current
per_tier.append({
"tier": tier_key,
"model": model["model"],
"arr_per_csm_target": model["arr_per_csm_target"],
"current_arr": tier_book.get("total_arr_usd", 0),
"current_customers": tier_book.get("customer_count", 0),
"current_csm_count": current,
"required_csm_now": req_now["required"],
"required_csm_12mo": req_future["required"],
"binding_constraint": req_now["binding_constraint"],
"gap_now": gap_now,
"gap_12mo": gap_future,
"annual_cost_required_now": req_now["required"] * model["fully_loaded_cost_yr"],
"annual_cost_required_12mo": req_future["required"] * model["fully_loaded_cost_yr"],
})
total_required_now += req_now["required"]
total_required_future += req_future["required"]
total_current += current
total_cost_now += req_now["required"] * model["fully_loaded_cost_yr"]
total_cost_future += req_future["required"] * model["fully_loaded_cost_yr"]
# Manager trigger: a CS manager is needed when a single function has 5+ ICs
manager_triggers = []
for t in per_tier:
if t["required_csm_12mo"] >= 5:
manager_triggers.append({
"tier": t["tier"],
"trigger": "5+ ICs in tier",
"recommendation": f"Add CS manager for {t['tier']} when scaling to {t['required_csm_12mo']}+ CSMs",
})
# Overall function trigger
if total_required_future >= 8 and not manager_triggers:
manager_triggers.append({
"tier": "overall",
"trigger": "8+ CSMs across team",
"recommendation": "Add CS manager / Head of CS",
})
# Hiring sequencing (largest gap first, but cap at one hire per quarter per tier)
hiring_plan = []
sorted_gaps = sorted(per_tier, key=lambda x: -x["gap_12mo"])
quarter = 1
for t in sorted_gaps:
if t["gap_12mo"] <= 0:
continue
for i in range(t["gap_12mo"]):
hiring_plan.append({
"quarter": f"Q{quarter}",
"tier": t["tier"],
"role": f"CSM ({t['model']})",
})
quarter = (quarter % 4) + 1
return {
"per_tier": per_tier,
"manager_triggers": manager_triggers,
"hiring_plan_12mo": hiring_plan,
"totals": {
"current_csm_count": total_current,
"required_csm_now": total_required_now,
"required_csm_12mo": total_required_future,
"gap_now": total_required_now - total_current,
"gap_12mo": total_required_future - total_current,
"annual_cost_required_now": total_cost_now,
"annual_cost_required_12mo": total_cost_future,
"growth_target_pct": payload.get("growth_target_pct", 0),
},
}
def render_text(result: Dict[str, Any], source: str) -> str:
lines = []
lines.append("=" * 72)
lines.append("CS TEAM COVERAGE CALCULATION")
lines.append(f"Source: {source}")
lines.append("=" * 72)
lines.append("")
t = result["totals"]
lines.append(f"Book growth assumption (12mo): {t['growth_target_pct']*100:.0f}%")
lines.append("")
lines.append(f"Current CSMs: {t['current_csm_count']}")
lines.append(f"Required now: {t['required_csm_now']} (gap: {t['gap_now']:+d})")
lines.append(f"Required in 12mo: {t['required_csm_12mo']} (gap: {t['gap_12mo']:+d})")
lines.append("")
lines.append(f"Annual CSM cost (now): ,")
lines.append(f"Annual CSM cost (12mo at growth): ,")
lines.append("")
lines.append("-" * 72)
lines.append("PER-TIER BREAKDOWN:")
lines.append("")
for r in result["per_tier"]:
gap_marker = "⚠️ " if r["gap_now"] > 0 else "✓"
lines.append(f" {r['tier']:<16} {r['model']}")
lines.append(f" Book: ,.0f across {r['current_customers']} customers")
lines.append(f" Target ratio: ,/CSM (binding: {r['binding_constraint']})")
lines.append(f" Current CSMs: {r['current_csm_count']} | Required now: {r['required_csm_now']} | Required 12mo: {r['required_csm_12mo']}")
lines.append(f" {gap_marker} Gap now: {r['gap_now']:+d} | Gap 12mo: {r['gap_12mo']:+d}")
lines.append("")
lines.append("-" * 72)
if result["manager_triggers"]:
lines.append("MANAGER TRIGGER(S):")
for mt in result["manager_triggers"]:
lines.append(f" • {mt['tier']:<12} — {mt['trigger']}: {mt['recommendation']}")
lines.append("")
if result["hiring_plan_12mo"]:
lines.append(f"12-MONTH HIRING PLAN ({len(result['hiring_plan_12mo'])} hires):")
for h in result["hiring_plan_12mo"]:
lines.append(f" {h['quarter']}: {h['role']:<45} (tier: {h['tier']})")
lines.append("")
lines.append("-" * 72)
lines.append("REMINDER: ARR-per-CSM ratios are starting points, not laws. ACV, product complexity,")
lines.append("and customer maturity shift the ratios materially. Re-run quarterly with updated book.")
return "\n".join(lines)
def main() -> int:
parser = argparse.ArgumentParser(
description="Calculate CS team headcount per coverage model + 12-month hiring plan.",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__,
)
parser.add_argument("path", nargs="?", help="Path to book JSON (uses embedded sample if omitted)")
parser.add_argument("--output", choices=("text", "json"), default="text", help="Output format")
args = parser.parse_args()
if args.path:
try:
with open(args.path, "r", encoding="utf-8") as f:
payload = json.load(f)
source = args.path
except (IOError, OSError) as e:
print(f"error: could not read {args.path}: {e}", file=sys.stderr)
return 1
except json.JSONDecodeError as e:
print(f"error: invalid JSON in {args.path}: {e}", file=sys.stderr)
return 1
else:
payload = SAMPLE
source = "<embedded sample: 450-customer B2B SaaS book at $6.9M ARR>"
result = analyze(payload)
if args.output == "json":
print(json.dumps({"source": source, **result}, indent=2))
else:
print(render_text(result, source))
return 0
if __name__ == "__main__":
sys.exit(main())
FILE:scripts/customer_segmentation_designer.py
#!/usr/bin/env python3
"""customer_segmentation_designer.py — Design tiered segmentation + ICP fit scoring.
Stdlib-only. Takes a customer list and outputs:
- Tier assignment (Strategic / Enterprise / Mid-market / SMB-long-tail)
- ICP fit score per customer (0-10) based on weighted attributes
- Differential investment recommendation per tier
- Kill list (customers below investment-payback floor)
Deterministic logic. Same input -> same output.
Input schema (JSON):
{
"customers": [
{
"name": "AcmeCorp",
"arr_usd": 180000,
"tenure_months": 18,
"icp_fit_signals": {
"in_target_industry": true,
"in_target_size_range": true,
"uses_target_workflow": true,
"has_executive_sponsor": true,
"advocates_publicly": false,
"expansion_potential_high": true,
"competitor_concentration_low": true
},
"annual_support_cost_usd": 8000 # CSM time + support time + custom work
}
]
}
Usage:
python customer_segmentation_designer.py # uses embedded sample
python customer_segmentation_designer.py path/to/customers.json
python customer_segmentation_designer.py customers.json --output json
"""
import argparse
import json
import sys
from typing import Any, Dict, List, Tuple
SAMPLE: Dict[str, Any] = {
"customers": [
{
"name": "MegaCorp Industries",
"arr_usd": 420_000,
"tenure_months": 26,
"icp_fit_signals": {
"in_target_industry": True,
"in_target_size_range": True,
"uses_target_workflow": True,
"has_executive_sponsor": True,
"advocates_publicly": True,
"expansion_potential_high": True,
"competitor_concentration_low": True,
},
"annual_support_cost_usd": 35000,
},
{
"name": "MidSize Co.",
"arr_usd": 38_000,
"tenure_months": 12,
"icp_fit_signals": {
"in_target_industry": True,
"in_target_size_range": True,
"uses_target_workflow": True,
"has_executive_sponsor": False,
"advocates_publicly": False,
"expansion_potential_high": True,
"competitor_concentration_low": True,
},
"annual_support_cost_usd": 4500,
},
{
"name": "Misfit Customer LLC",
"arr_usd": 12_000,
"tenure_months": 8,
"icp_fit_signals": {
"in_target_industry": False,
"in_target_size_range": True,
"uses_target_workflow": False,
"has_executive_sponsor": False,
"advocates_publicly": False,
"expansion_potential_high": False,
"competitor_concentration_low": False,
},
"annual_support_cost_usd": 14000,
},
{
"name": "Small Biz",
"arr_usd": 2_400,
"tenure_months": 4,
"icp_fit_signals": {
"in_target_industry": True,
"in_target_size_range": False,
"uses_target_workflow": True,
"has_executive_sponsor": False,
"advocates_publicly": False,
"expansion_potential_high": False,
"competitor_concentration_low": True,
},
"annual_support_cost_usd": 500,
},
{
"name": "Enterprise Co",
"arr_usd": 75_000,
"tenure_months": 15,
"icp_fit_signals": {
"in_target_industry": True,
"in_target_size_range": True,
"uses_target_workflow": True,
"has_executive_sponsor": True,
"advocates_publicly": False,
"expansion_potential_high": True,
"competitor_concentration_low": True,
},
"annual_support_cost_usd": 9000,
},
]
}
# ICP signal weights (sum to 10)
ICP_WEIGHTS = {
"in_target_industry": 2.0,
"in_target_size_range": 1.5,
"uses_target_workflow": 2.0,
"has_executive_sponsor": 1.5,
"advocates_publicly": 1.0,
"expansion_potential_high": 1.0,
"competitor_concentration_low": 1.0,
}
# Tier definitions: ARR ranges + recommended coverage + investment
TIER_DEFINITIONS = [
{
"tier": "Strategic",
"arr_min": 100_000,
"coverage": "Named CSM + executive sponsor",
"investment_per_account_yr_min": 20000,
"investment_per_account_yr_max": 50000,
},
{
"tier": "Enterprise",
"arr_min": 20_000,
"coverage": "Named CSM",
"investment_per_account_yr_min": 5000,
"investment_per_account_yr_max": 15000,
},
{
"tier": "Mid-market",
"arr_min": 5_000,
"coverage": "Pooled CSM + automation",
"investment_per_account_yr_min": 1000,
"investment_per_account_yr_max": 3000,
},
{
"tier": "SMB / Long-tail",
"arr_min": 0,
"coverage": "Tech-touch + self-serve",
"investment_per_account_yr_min": 50,
"investment_per_account_yr_max": 500,
},
]
def assign_tier(arr: float) -> Dict[str, Any]:
for t in TIER_DEFINITIONS:
if arr >= t["arr_min"]:
return t
return TIER_DEFINITIONS[-1]
def icp_fit_score(signals: Dict[str, bool]) -> float:
score = 0.0
for signal, weight in ICP_WEIGHTS.items():
if signals.get(signal, False):
score += weight
return round(score, 1)
def analyze_customer(c: Dict[str, Any]) -> Dict[str, Any]:
arr = c.get("arr_usd", 0)
tier_def = assign_tier(arr)
fit_score = icp_fit_score(c.get("icp_fit_signals", {}))
support_cost = c.get("annual_support_cost_usd", 0)
# Investment-to-ARR ratio
cost_ratio = (support_cost / arr) if arr else float("inf")
# Kill list candidate: support cost > 50% of ARR AND ICP fit < 5
kill_candidate = cost_ratio > 0.5 and fit_score < 5.0
# Strategic upgrade candidate: at top of current tier + high ICP fit + expansion potential
upgrade_signal = (
fit_score >= 8.0
and c.get("icp_fit_signals", {}).get("expansion_potential_high", False)
)
return {
"name": c.get("name"),
"arr_usd": arr,
"tenure_months": c.get("tenure_months", 0),
"tier": tier_def["tier"],
"coverage": tier_def["coverage"],
"investment_floor_yr": tier_def["investment_per_account_yr_min"],
"investment_ceiling_yr": tier_def["investment_per_account_yr_max"],
"icp_fit_score": fit_score,
"annual_support_cost_usd": support_cost,
"support_cost_pct_of_arr": round(cost_ratio * 100, 1) if cost_ratio != float("inf") else None,
"kill_candidate": kill_candidate,
"upgrade_candidate": upgrade_signal,
}
def aggregate(customer_results: List[Dict[str, Any]]) -> Dict[str, Any]:
by_tier: Dict[str, List[Dict[str, Any]]] = {t["tier"]: [] for t in TIER_DEFINITIONS}
for r in customer_results:
by_tier[r["tier"]].append(r)
summary = []
total_arr = sum(r["arr_usd"] for r in customer_results)
for t in TIER_DEFINITIONS:
tier_customers = by_tier[t["tier"]]
tier_arr = sum(c["arr_usd"] for c in tier_customers)
summary.append({
"tier": t["tier"],
"customer_count": len(tier_customers),
"tier_arr": tier_arr,
"tier_arr_pct_of_total": round((tier_arr / total_arr * 100) if total_arr else 0, 1),
"coverage": t["coverage"],
"investment_per_account_yr": f",-,",
})
kill_list = [r for r in customer_results if r["kill_candidate"]]
upgrade_list = [r for r in customer_results if r["upgrade_candidate"]]
return {
"tier_summary": summary,
"kill_list": kill_list,
"upgrade_list": upgrade_list,
"total_arr": total_arr,
"total_customers": len(customer_results),
}
def analyze(payload: Dict[str, Any]) -> Dict[str, Any]:
customers = [analyze_customer(c) for c in payload.get("customers", [])]
return {
"customers": customers,
"summary": aggregate(customers),
}
def render_text(result: Dict[str, Any], source: str) -> str:
lines = []
lines.append("=" * 72)
lines.append("CUSTOMER SEGMENTATION DESIGN")
lines.append(f"Source: {source}")
lines.append("=" * 72)
lines.append("")
s = result["summary"]
lines.append(f"Total customers: {s['total_customers']} | Total ARR: ,.0f")
lines.append("")
lines.append("TIER BREAKDOWN:")
lines.append("")
for t in s["tier_summary"]:
lines.append(f" {t['tier']:<20} {t['customer_count']:>3} customers >10,.0f ({t['tier_arr_pct_of_total']:.1f}% of ARR)")
lines.append(f" Coverage: {t['coverage']}")
lines.append(f" Investment per account/yr: {t['investment_per_account_yr']}")
lines.append("")
lines.append("-" * 72)
if s["kill_list"]:
lines.append(f"")
lines.append(f"🔴 KILL LIST ({len(s['kill_list'])} customers): support cost > 50% of ARR AND ICP fit < 5")
for k in s["kill_list"]:
lines.append(f" • {k['name']}: ARR ,.0f, support ,.0f ({k['support_cost_pct_of_arr']}%), ICP fit {k['icp_fit_score']}/10")
lines.append("")
lines.append(" Recommendation: do not renew, OR downgrade to tech-touch, OR raise price to cost-recover.")
lines.append("")
if s["upgrade_list"]:
lines.append(f"")
lines.append(f"🟢 UPGRADE CANDIDATES ({len(s['upgrade_list'])} customers): high ICP fit + expansion potential")
for u in s["upgrade_list"]:
lines.append(f" • {u['name']}: tier {u['tier']}, ICP fit {u['icp_fit_score']}/10, ARR ,.0f")
lines.append("")
lines.append(" Recommendation: assign named CSM (if not already) + executive sponsor + expansion playbook.")
lines.append("")
lines.append("-" * 72)
lines.append("PER-CUSTOMER DETAIL:")
lines.append("")
for c in result["customers"]:
markers = ""
if c["kill_candidate"]:
markers += " 🔴"
if c["upgrade_candidate"]:
markers += " 🟢"
lines.append(f" {c['name']:<25} >8,.0f {c['tier']:<20} ICP fit: {c['icp_fit_score']}/10{markers}")
lines.append("")
lines.append("-" * 72)
lines.append("REMINDER: Segmentation is a quarterly review. Customers migrate between tiers; ICP fit drifts.")
lines.append("Pair this output with cs_coverage_calculator.py to size the CS team for the new segmentation.")
return "\n".join(lines)
def main() -> int:
parser = argparse.ArgumentParser(
description="Design customer segmentation tiers + ICP fit scoring + differential investment.",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__,
)
parser.add_argument("path", nargs="?", help="Path to customers JSON (uses embedded sample if omitted)")
parser.add_argument("--output", choices=("text", "json"), default="text", help="Output format")
args = parser.parse_args()
if args.path:
try:
with open(args.path, "r", encoding="utf-8") as f:
payload = json.load(f)
source = args.path
except (IOError, OSError) as e:
print(f"error: could not read {args.path}: {e}", file=sys.stderr)
return 1
except json.JSONDecodeError as e:
print(f"error: invalid JSON in {args.path}: {e}", file=sys.stderr)
return 1
else:
payload = SAMPLE
source = "<embedded sample: 5 mixed B2B SaaS customers>"
result = analyze(payload)
if args.output == "json":
print(json.dumps({"source": source, **result}, indent=2))
else:
print(render_text(result, source))
return 0
if __name__ == "__main__":
sys.exit(main())
FILE:scripts/retention_decomposition_analyzer.py
#!/usr/bin/env python3
"""retention_decomposition_analyzer.py — Honest retention decomposition for B2B SaaS.
Stdlib-only. Takes cohort data and outputs:
- Gross Revenue Retention (GRR), Net Revenue Retention (NRR), Logo Retention by cohort
- Contraction vs Expansion separation (NRR alone hides churn)
- Churn root-cause categorization (7-category taxonomy)
- Health verdict per cohort with thresholds
Deterministic logic derived from inputs. No projections.
Input schema (JSON):
{
"cohorts": [
{
"name": "2025-Q1",
"starting_arr": 2400000, # ARR of customers acquired in this cohort
"starting_customer_count": 80,
"renewed_arr": 2280000, # ARR retained at 1-year mark (after churn + contraction)
"renewed_customer_count": 72,
"expansion_arr": 360000, # ARR from upsells / seat additions in same cohort
"contraction_arr": 80000, # ARR lost from downsells (without churn)
"churn_reasons": { # logo-count by category
"product_fit": 3,
"competitor_loss": 2,
"no_value_realized": 1,
"pricing": 1,
"champion_left": 1,
"company_event": 0,
"tactical_failure": 0
}
}
]
}
Usage:
python retention_decomposition_analyzer.py # uses embedded sample
python retention_decomposition_analyzer.py path/to/cohorts.json
python retention_decomposition_analyzer.py cohorts.json --output json
"""
import argparse
import json
import sys
from typing import Any, Dict, List
# 7-category churn taxonomy
CHURN_CATEGORIES = {
"product_fit": "Product didn't solve customer's actual job-to-be-done",
"competitor_loss": "Lost to a competitor with better fit or price",
"no_value_realized": "Customer never reached time-to-value; onboarding gap",
"pricing": "Price-driven churn (too expensive, or perceived as low value)",
"champion_left": "Internal champion changed roles or left the company",
"company_event": "Customer's company event (M&A, layoffs, shutdown) — not preventable",
"tactical_failure": "Service / support failure — preventable with better CS execution",
}
# Health thresholds (B2B SaaS baseline)
THRESHOLDS = {
"grr": {"healthy": 0.90, "concerning": 0.85, "critical": 0.80},
"nrr": {"healthy": 1.10, "concerning": 1.00, "critical": 0.95},
"logo": {"healthy": 0.85, "concerning": 0.75, "critical": 0.65},
}
SAMPLE: Dict[str, Any] = {
"cohorts": [
{
"name": "2025-Q1",
"starting_arr": 2_400_000,
"starting_customer_count": 80,
"renewed_arr": 2_280_000,
"renewed_customer_count": 72,
"expansion_arr": 360_000,
"contraction_arr": 80_000,
"churn_reasons": {
"product_fit": 3,
"competitor_loss": 2,
"no_value_realized": 1,
"pricing": 1,
"champion_left": 1,
"company_event": 0,
"tactical_failure": 0,
},
},
{
"name": "2025-Q2",
"starting_arr": 3_100_000,
"starting_customer_count": 95,
"renewed_arr": 2_790_000,
"renewed_customer_count": 81,
"expansion_arr": 280_000,
"contraction_arr": 165_000,
"churn_reasons": {
"product_fit": 6,
"competitor_loss": 3,
"no_value_realized": 2,
"pricing": 2,
"champion_left": 1,
"company_event": 0,
"tactical_failure": 0,
},
},
]
}
def analyze_cohort(cohort: Dict[str, Any]) -> Dict[str, Any]:
starting_arr = cohort.get("starting_arr", 0)
renewed_arr = cohort.get("renewed_arr", 0)
expansion = cohort.get("expansion_arr", 0)
contraction = cohort.get("contraction_arr", 0)
starting_count = cohort.get("starting_customer_count", 0)
renewed_count = cohort.get("renewed_customer_count", 0)
# GRR = (starting_arr - churn - contraction) / starting_arr
# renewed_arr already reflects churn but NOT contraction (per schema)
grr = (renewed_arr - contraction) / starting_arr if starting_arr else 0
# NRR = GRR + expansion / starting
nrr = grr + (expansion / starting_arr) if starting_arr else 0
logo = renewed_count / starting_count if starting_count else 0
return {
"cohort": cohort.get("name"),
"starting_arr": starting_arr,
"renewed_arr": renewed_arr,
"expansion_arr": expansion,
"contraction_arr": contraction,
"gross_retention": round(grr, 4),
"net_retention": round(nrr, 4),
"logo_retention": round(logo, 4),
"expansion_pct": round((expansion / starting_arr * 100) if starting_arr else 0, 1),
"contraction_pct": round((contraction / starting_arr * 100) if starting_arr else 0, 1),
"churn_customers": starting_count - renewed_count,
"churn_reasons": cohort.get("churn_reasons", {}),
}
def verdict(grr: float, nrr: float, logo: float) -> Dict[str, str]:
def bucket(value: float, kind: str) -> str:
t = THRESHOLDS[kind]
if value >= t["healthy"]:
return "HEALTHY"
if value >= t["concerning"]:
return "CONCERNING"
if value >= t["critical"]:
return "POOR"
return "CRITICAL"
grr_v = bucket(grr, "grr")
nrr_v = bucket(nrr, "nrr")
logo_v = bucket(logo, "logo")
# Special detection: NRR healthy but GRR poor → leaky bucket masked by expansion
overall = "HEALTHY"
notes: List[str] = []
if nrr >= THRESHOLDS["nrr"]["healthy"] and grr < THRESHOLDS["grr"]["concerning"]:
overall = "LEAKY BUCKET"
notes.append(
"NRR looks healthy but GRR is poor: expansion is masking churn. "
"Fix retention before celebrating NRR."
)
elif "CRITICAL" in (grr_v, nrr_v, logo_v):
overall = "CRITICAL"
elif "POOR" in (grr_v, nrr_v, logo_v):
overall = "POOR"
elif "CONCERNING" in (grr_v, nrr_v, logo_v):
overall = "CONCERNING"
return {
"grr_verdict": grr_v,
"nrr_verdict": nrr_v,
"logo_verdict": logo_v,
"overall": overall,
"notes": " | ".join(notes) if notes else "",
}
def churn_root_cause_summary(cohort_results: List[Dict[str, Any]]) -> Dict[str, Any]:
"""Aggregate churn reasons across all cohorts; identify top drivers."""
totals: Dict[str, int] = {k: 0 for k in CHURN_CATEGORIES}
for r in cohort_results:
for cat, count in (r.get("churn_reasons") or {}).items():
if cat in totals:
totals[cat] += count
total_churn = sum(totals.values())
if total_churn == 0:
return {"total_churn_customers": 0, "top_drivers": [], "preventable_pct": 0.0}
ranked = sorted(totals.items(), key=lambda x: -x[1])
top_drivers = [
{
"category": cat,
"description": CHURN_CATEGORIES[cat],
"count": cnt,
"pct": round((cnt / total_churn) * 100, 1),
}
for cat, cnt in ranked if cnt > 0
][:3]
# Preventable = product_fit, no_value_realized, tactical_failure (within CS control)
# Less preventable = competitor_loss, pricing, champion_left (mixed)
# Not preventable = company_event
preventable_count = totals["product_fit"] + totals["no_value_realized"] + totals["tactical_failure"]
preventable_pct = round((preventable_count / total_churn) * 100, 1)
return {
"total_churn_customers": total_churn,
"top_drivers": top_drivers,
"preventable_pct": preventable_pct,
}
def analyze(payload: Dict[str, Any]) -> Dict[str, Any]:
cohort_results = []
for cohort in payload.get("cohorts", []):
result = analyze_cohort(cohort)
result["verdict"] = verdict(
result["gross_retention"],
result["net_retention"],
result["logo_retention"],
)
cohort_results.append(result)
return {
"cohorts": cohort_results,
"churn_summary": churn_root_cause_summary(cohort_results),
}
def render_text(result: Dict[str, Any], source: str) -> str:
lines = []
lines.append("=" * 72)
lines.append("RETENTION DECOMPOSITION")
lines.append(f"Source: {source}")
lines.append("=" * 72)
lines.append("")
for c in result["cohorts"]:
v = c["verdict"]
lines.append(f"📊 Cohort {c['cohort']} — {v['overall']}")
lines.append(f" Starting ARR: ,.0f")
lines.append(f" Renewed ARR: ,.0f")
lines.append("")
lines.append(f" GRR: {c['gross_retention']*100:5.1f}% [{v['grr_verdict']}] (healthy ≥ 90%)")
lines.append(f" NRR: {c['net_retention']*100:5.1f}% [{v['nrr_verdict']}] (healthy ≥ 110%)")
lines.append(f" Logo: {c['logo_retention']*100:4.1f}% [{v['logo_verdict']}] (healthy ≥ 85%)")
lines.append("")
lines.append(f" Contraction: {c['contraction_pct']:.1f}% | Expansion: {c['expansion_pct']:.1f}%")
lines.append(f" Customers churned: {c['churn_customers']}")
if v["notes"]:
lines.append("")
lines.append(f" ⚠️ {v['notes']}")
lines.append("")
lines.append("-" * 72)
cs = result["churn_summary"]
lines.append("")
lines.append(f"CHURN ROOT-CAUSE TAXONOMY (across all cohorts)")
lines.append(f" Total customers churned: {cs['total_churn_customers']}")
if cs["total_churn_customers"] > 0:
lines.append(f" Preventable (CS-controllable): {cs['preventable_pct']}%")
lines.append("")
lines.append(" Top drivers:")
for d in cs["top_drivers"]:
lines.append(f" {d['category']:<20} {d['count']:>3} ({d['pct']}%) — {d['description']}")
lines.append("")
lines.append("-" * 72)
lines.append("HONEST READ: NRR is the vanity metric; GRR is the truth metric. If GRR < 85% and NRR > 100%,")
lines.append("you have a leaky bucket masked by upsells. Fix retention before scaling acquisition.")
return "\n".join(lines)
def main() -> int:
parser = argparse.ArgumentParser(
description="Decompose retention honestly (GRR vs NRR) and categorize churn root causes.",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__,
)
parser.add_argument("path", nargs="?", help="Path to cohorts JSON (uses embedded sample if omitted)")
parser.add_argument("--output", choices=("text", "json"), default="text", help="Output format")
args = parser.parse_args()
if args.path:
try:
with open(args.path, "r", encoding="utf-8") as f:
payload = json.load(f)
source = args.path
except (IOError, OSError) as e:
print(f"error: could not read {args.path}: {e}", file=sys.stderr)
return 1
except json.JSONDecodeError as e:
print(f"error: invalid JSON in {args.path}: {e}", file=sys.stderr)
return 1
else:
payload = SAMPLE
source = "<embedded sample: 2 quarterly B2B SaaS cohorts>"
result = analyze(payload)
if args.output == "json":
print(json.dumps({"source": source, **result}, indent=2))
else:
print(render_text(result, source))
return 0
if __name__ == "__main__":
sys.exit(main())
Lãnh đạo nhân sự: chiến lược tuyển dụng, thiết kế lương thưởng, cơ cấu tổ chức, văn hóa và giữ chân nhân tài.
---
name: "chro-advisor"
description: "People leadership for scaling companies. Hiring strategy, compensation design, org structure, culture, and retention. Use when building hiring plans, designing comp frameworks, restructuring teams, managing performance, building culture, or when user mentions CHRO, HR, people strategy, talent, headcount, compensation, org design, retention, or performance management."
license: MIT
metadata:
version: 1.0.0
author: Alireza Rezvani
category: c-level
domain: chro-leadership
updated: 2026-03-05
python-tools: hiring_plan_modeler.py, comp_benchmarker.py
frameworks: people-strategy, comp-frameworks, org-design
---
# CHRO Advisor
People strategy and operational HR frameworks for business-aligned hiring, compensation, org design, and culture that scales.
## Keywords
CHRO, chief people officer, CPO, HR, human resources, people strategy, hiring plan, headcount planning, talent acquisition, recruiting, compensation, salary bands, equity, org design, organizational design, career ladder, title framework, retention, performance management, culture, engagement, remote work, hybrid, spans of control, succession planning, attrition
## Quick Start
```bash
python scripts/hiring_plan_modeler.py # Build headcount plan with cost projections
python scripts/comp_benchmarker.py # Benchmark salaries and model total comp
```
## Core Responsibilities
### 1. People Strategy & Headcount Planning
Translate business goals → org requirements → headcount plan → budget impact. Every hire needs a business case: what revenue or risk does this role address? See `references/people_strategy.md` for hiring at each growth stage.
### 2. Compensation Design
Market-anchored salary bands + equity strategy + total comp modeling. See `references/comp_frameworks.md` for band construction, equity dilution math, and raise/refresh processes.
### 3. Org Design
Right structure for the stage. Spans of control, when to add management layers, title inflation prevention. See `references/org_design.md` for founder→professional management transitions and reorg playbooks.
### 4. Retention & Performance
Retention starts at hire. Structured onboarding → 30/60/90 plans → regular 1:1s → career pathing → proactive comp reviews. See `references/people_strategy.md` for what actually moves the needle.
**Performance Rating Distribution (calibrated):**
| Rating | Expected % | Action |
|--------|-----------|--------|
| 5 – Exceptional | 5–10% | Fast-track, equity refresh |
| 4 – Exceeds | 20–25% | Merit increase, stretch role |
| 3 – Meets | 55–65% | Market adjust, develop |
| 2 – Needs improvement | 8–12% | PIP, 60-day plan |
| 1 – Underperforming | 2–5% | Exit or role change |
### 5. Culture & Engagement
Culture is behavior, not values on a wall. Measure eNPS quarterly. Act on results within 30 days or don't ask.
## Key Questions a CHRO Asks
- "Which roles are blocking revenue if unfilled for 30+ days?"
- "What's our regrettable attrition rate? Who left that we wish hadn't?"
- "Are managers our retention asset or our attrition cause?"
- "Can a new hire explain their career path in 12 months?"
- "Where are we paying below P50? Who's a flight risk because of it?"
- "What's the cost of this hire vs. the cost of not hiring?"
## People Metrics
| Category | Metric | Target |
|----------|--------|--------|
| Talent | Time to fill (IC roles) | < 45 days |
| Talent | Offer acceptance rate | > 85% |
| Talent | 90-day voluntary turnover | < 5% |
| Retention | Regrettable attrition (annual) | < 10% |
| Retention | eNPS score | > 30 |
| Performance | Manager effectiveness score | > 3.8/5 |
| Comp | % employees within band | > 90% |
| Comp | Compa-ratio (avg) | 0.95–1.05 |
| Org | Span of control (ICs) | 6–10 |
| Org | Span of control (managers) | 4–7 |
## Red Flags
- Attrition spikes and exit interviews all name the same manager
- Comp bands haven't been refreshed in 18+ months
- No career ladder → top performers leave after 18 months
- Hiring without a written business case or job scorecard
- Performance reviews happen once a year with no mid-year check-in
- Equity refreshes only for executives, not high performers
- Time to fill > 90 days for critical roles
- eNPS below 0 — something is structurally broken
- More than 3 org layers between IC and CEO at < 50 people
## Integration with Other C-Suite Roles
| When... | CHRO works with... | To... |
|---------|-------------------|-------|
| Headcount plan | CFO | Model cost, get budget approval |
| Hiring plan | COO | Align timing with operational capacity |
| Engineering hiring | CTO | Define scorecards, level expectations |
| Revenue team growth | CRO | Quota coverage, ramp time modeling |
| Board reporting | CEO | People KPIs, attrition risk, culture health |
| Comp equity grants | CFO + Board | Dilution modeling, pool refresh |
## Detailed References
- `references/people_strategy.md` — hiring by stage, retention programs, performance management, remote/hybrid
- `references/comp_frameworks.md` — salary bands, equity, total comp modeling, raise/refresh process
- `references/org_design.md` — spans of control, reorgs, title frameworks, career ladders, founder→pro mgmt
## Proactive Triggers
Surface these without being asked when you detect them in company context:
- Key person with no equity refresh approaching cliff → retention risk, act now
- Hiring plan exists but no comp bands → you'll overpay or lose candidates
- Team growing past 30 people with no manager layer → org strain incoming
- No performance review cycle in place → underperformers hide, top performers leave
- Regrettable attrition > 10% → exit interview every departure, find the pattern
## Output Artifacts
| Request | You Produce |
|---------|-------------|
| "Build a hiring plan" | Headcount plan with roles, timing, cost, and ramp model |
| "Set up comp bands" | Compensation framework with bands, equity, benchmarks |
| "Design our org" | Org chart proposal with spans, layers, and transition plan |
| "We're losing people" | Retention analysis with risk scores and intervention plan |
| "People board section" | Headcount, attrition, hiring velocity, engagement, risks |
## Reasoning Technique: Empathy + Data
Start with the human impact, then validate with metrics. Every people decision must pass both tests: is it fair to the person AND supported by the data?
## Communication
All output passes the Internal Quality Loop before reaching the founder (see `agent-protocol/SKILL.md`).
- Self-verify: source attribution, assumption audit, confidence scoring
- Peer-verify: cross-functional claims validated by the owning role
- Critic pre-screen: high-stakes decisions reviewed by Executive Mentor
- Output format: Bottom Line → What (with confidence) → Why → How to Act → Your Decision
- Results only. Every finding tagged: 🟢 verified, 🟡 medium, 🔴 assumed.
## Context Integration
- **Always** read `company-context.md` before responding (if it exists)
- **During board meetings:** Use only your own analysis in Phase 2 (no cross-pollination)
- **Invocation:** You can request input from other roles: `[INVOKE:role|question]`
FILE:references/comp_frameworks.md
# Compensation Frameworks Reference
Salary bands, equity design, total comp modeling, comp philosophy, and raise/refresh processes.
---
## Comp Philosophy — The Foundation
Before building bands, define your philosophy. Ambiguity in comp philosophy = pay equity lawsuits and trust erosion.
**The five decisions:**
### 1. What market percentile do you target?
- **P25 (below market):** Only viable with exceptional mission, equity, or growth opportunity. Flight risk is high after 18 months.
- **P50 (market median):** Standard for most Series A–B companies. Competitive without premium.
- **P75 (above market):** Premium talent strategy. Used by high-margin or talent-intensive businesses. Netflix model.
- **P90+:** Top-of-market for specific functions (ML at AI companies, senior engineers at FAANG feeders).
**Common hybrid:** P50 base + above-market equity = total comp at P65–75.
### 2. What's in your total comp package?
Define each component explicitly:
- **Base salary** — cash, market-benchmarked
- **Variable / bonus** — % of base, tied to what criteria
- **Equity** — options vs. RSUs, vesting schedule, refresh cadence
- **Benefits** — health, retirement, PTO policy
- **Learning & development budget**
- **Remote/location allowances**
### 3. Are bands public internally?
Recommended: Yes. Pay transparency reduces equity complaints, builds trust, and forces you to maintain clean bands.
### 4. How often do you refresh bands?
Minimum: annually. High-growth markets: every 6 months (engineering specifically in hot markets).
### 5. How do you handle individual negotiation?
Options:
- **Fixed bands, no negotiation** (Buffer model) — simple, fair, loses some candidates
- **Band range with manager discretion** — most common, requires calibration guardrails
- **Individual negotiation within band** — flexible, creates pay equity drift over time
---
## Salary Bands: Construction
### Step 1: Define levels
Standard IC levels (adapt to company):
| Level | Title example | Scope |
|-------|--------------|-------|
| L1 | Junior / Associate | Execution with guidance |
| L2 | Mid-level | Independent execution |
| L3 | Senior | Leads workstreams, mentors L1-L2 |
| L4 | Staff / Principal | Cross-team technical leadership |
| L5 | Distinguished / Fellow | Company-wide technical direction |
Management track:
| Level | Title | Scope |
|-------|-------|-------|
| M1 | Manager | Team of 4–8 ICs |
| M2 | Senior Manager | Manager of managers or larger team |
| M3 | Director | Function or large org |
| M4 | VP | Business unit, company-wide |
| M5 | SVP / C-Suite | Executive |
### Step 2: Gather market data
**Data sources (by quality):**
1. **Radford / Aon** — Gold standard. Expensive ($10K+/year). Worth it at Series B+.
2. **Levels.fyi** — Excellent for engineering. Free. Self-reported but large sample.
3. **Glassdoor Salary** — Broad coverage. Less precise for startups.
4. **Pave / Carta Total Comp** — VC-backed companies. Good peer benchmarking.
5. **LinkedIn Salary** — Free tier. Reasonable signal for G&A roles.
6. **Offer letter data** — What candidates are bringing from other companies. Real-time signal.
**What to pull:** P25, P50, P75, P90 for each role × level × geography.
### Step 3: Set band structure
**Band width (range within a level):**
- IC bands: 80–120% of midpoint (i.e., ±20% from center)
- Manager bands: 85–115% of midpoint
- Wider bands allow room for differentiation within level; narrower bands reduce pay equity drift
**Band overlap between levels:**
- 10–20% overlap is normal (top of L2 overlaps with bottom of L3)
- > 30% overlap: your levels are too close together
- No overlap: new hires jump too much between levels (compression risk)
**Example engineering band structure (US, Series B company, P50 target):**
| Level | Band Min | Midpoint | Band Max |
|-------|----------|----------|----------|
| L1 Software Engineer | $90K | $105K | $125K |
| L2 Software Engineer | $115K | $135K | $160K |
| L3 Senior SWE | $150K | $175K | $205K |
| L4 Staff SWE | $195K | $225K $260K |
| M1 Eng Manager | $175K | $205K | $235K |
| M2 Sr Eng Manager | $215K | $250K | $285K |
| M3 Director, Eng | $255K | $300K | $345K |
*Adjust by 15–25% for non-SF/NYC markets. Adjust -40% to -60% for European markets.*
### Step 4: Place employees in bands
**Compa-ratio** = Employee salary / Band midpoint
| Compa-ratio | Interpretation |
|------------|---------------|
| < 0.85 | Below range — immediate risk |
| 0.85–0.95 | Developing in role |
| 0.95–1.05 | Fully performing (target zone) |
| 1.05–1.15 | Senior/expert in role |
| > 1.15 | Above range — flag for review |
**Audit report:** Run quarterly. Flag anyone below 0.85 (flight risk) or above 1.15 (overpaid for level, or needs promotion).
---
## Equity Frameworks for Startups
### Option Basics
**ISO vs NSO:**
- ISO (Incentive Stock Options): For employees. Favorable tax treatment if held 1+ year post-exercise.
- NSO (Non-Qualified Stock Options): For advisors, contractors, sometimes employees. Taxed as ordinary income on exercise.
**Strike price:** Set to 409A valuation at grant. Lower is better for employees. Early employees win on strike price.
**Vesting schedule standards:**
- 4-year vest, 1-year cliff: Standard
- 4-year vest, 6-month cliff: Startup market adapting to faster pace
- 1-year cliff means: nothing until 12 months; monthly or quarterly after
**Post-termination exercise window (PTEW):**
- Standard: 90 days. Often too short for employees who can't afford exercise.
- Better: 1–5 years or until IPO. Use as a talent differentiator.
- Companies extending PTEW: Stripe, Airbnb (pre-IPO), Square, most employee-friendly startups.
### Equity Grant Ranges by Stage and Level
*Expressed as % of fully diluted shares at grant. Ranges vary significantly by market, stage, and funding.*
**Seed stage:**
| Role | Equity % |
|------|----------|
| Co-founder | 20–40% |
| First engineering hire | 0.5–1.5% |
| First non-technical exec hire | 0.25–0.75% |
| IC (L2-L3) | 0.1–0.4% |
| IC (L3-L4) | 0.2–0.6% |
**Series A:**
| Role | Equity % |
|------|----------|
| VP / Head of function | 0.3–0.75% |
| Director | 0.1–0.3% |
| Senior IC (L3) | 0.05–0.15% |
| Mid IC (L2) | 0.02–0.08% |
| Junior IC (L1) | 0.01–0.05% |
**Series B:**
| Role | Equity % |
|------|----------|
| VP / Head of function | 0.1–0.3% |
| Director | 0.05–0.15% |
| Senior IC (L3) | 0.02–0.07% |
| Mid IC (L2) | 0.01–0.03% |
*At Series B+, equity is increasingly expressed in dollar value (grant value = X shares × current 409A). Use Carta or Pulley to model dilution.*
### Equity Refresh Program
**Why it matters:** Employees hired at Series A with 4-year vesting will be fully vested by Series B. No unvested equity = no retention hook.
**When to refresh:**
- After every significant funding round
- Annually for high performers (top 20%)
- After promotion (role-commensurate top-up)
- Counter-offer situations (use carefully — signals you underpaid initially)
**Refresh models:**
1. **Anniversary grant:** Annual cliff-free refresh for all employees above a performance threshold
2. **Evergreen model:** Continuous vesting maintained — refresh annually so employee always has 2–3 years remaining
3. **Event-based:** Refresh tied to milestones (promotion, funding, annual review cycle)
**Dilution awareness:** Every refresh dilutes existing shareholders. Model pool usage quarterly. Replenish option pool before it drops below 10–12% of fully diluted shares.
---
## Total Comp Modeling
### Components of Total Comp
```
Total Compensation = Base Salary
+ Annual Bonus (target %)
+ Equity Value (annualized grant / vesting period)
+ Benefits (employer-paid premiums, retirement match)
+ Allowances (home office, internet, L&D, commuter)
```
### Annualizing Equity Value
For comparison to cash compensation:
```
Annual equity value = (Grant shares × Current 409A price) / Vesting years
```
Example: 10,000 options at $2 strike, current 409A = $8, 4-year vest
- Grant value at current 409A = 10,000 × $8 = $80,000
- Annual value = $80,000 / 4 = $20,000/year
- If base is $150K, total comp is ~$170K/year
*Note: For recruiting purposes, you can use last preferred share price (VC price) to show upside — but be transparent about the difference between 409A and preferred.*
### Benefits Valuation
Frequently undervalued in offers. Quantify explicitly:
| Benefit | Typical employer cost |
|---------|----------------------|
| Health insurance (employee) | $4K–8K/year |
| Health insurance (family) | $15K–25K/year |
| 401K match (4% of salary) | $5K–10K/year |
| L&D budget ($2K/year) | $2K/year |
| Home office stipend ($500) | $500/year |
A $140K offer with family health coverage + 4% 401K match is worth $165K+ total.
---
## Raise and Refresh Process
### Annual Compensation Review Cycle
**Recommended cadence:**
- October/November: Market data refresh, band updates
- November/December: Manager merit recommendations
- December/January: Calibration and approvals
- January/February: Effective date for new salaries + equity grants
**Budget allocation:**
- **Merit budget** (performance-based raises): 3–5% of total payroll typically
- **Market adjustment budget** (fixing below-band salaries): Separate from merit. Non-negotiable to avoid attrition.
- **Promotion budget:** Separate. Promotions should not come from merit pool.
### Merit Increase Guidelines
| Performance Rating | Merit Increase Range |
|-------------------|---------------------|
| 5 – Exceptional | 8–15% |
| 4 – Exceeds | 5–8% |
| 3 – Meets | 2–4% |
| 2 – Needs improvement | 0–1% |
| 1 – Underperforming | 0% (PIP active) |
*Adjust based on compa-ratio. A high performer at P90 of their band gets a smaller increase than a high performer at P50.*
### Compa-Ratio Adjustment Matrix
| Performance \ Compa-Ratio | < 0.90 | 0.90–1.00 | 1.00–1.10 | > 1.10 |
|---------------------------|--------|-----------|-----------|--------|
| Exceptional (5) | 12–15% | 8–12% | 5–8% | 3–5% |
| Exceeds (4) | 8–12% | 5–8% | 3–5% | 1–3% |
| Meets (3) | 5–8% | 3–5% | 2–3% | 0–2% |
| Needs impr (2) | 0–2% | 0–1% | 0% | 0% |
### Promotion vs. Merit — Keep These Separate
**Common mistake:** Using merit budget to fund promotions. This forces a choice between rewarding performance and recognizing level change.
**Promotion increase guidelines:**
- One level (e.g., L2 → L3): 10–20% increase, new equity grant
- Two levels (rare): 20–35% increase, new equity grant at new level
- Manager track (IC → M1): 15–25% increase, new equity grant
**Promotion criteria process:**
1. Manager nominates with written business case
2. Calibration committee reviews cross-functionally
3. HR validates against band (no off-band exceptions without CHRO sign-off)
4. Employee informed before annual review — never surprised at review meeting
### Off-Cycle Adjustments
When to do them:
- Counter-offer situations (see below)
- Competitive intelligence reveals underpay for a specific role
- New market data shows a role significantly under-benchmarked
- Internal equity audit reveals unexplained gaps
**Counter-offer policy:**
Three options:
1. **Match** — Risk: signals you underpay; sets precedent
2. **Partial match** — "We can do X, which is the top of your band" — cleaner
3. **Decline** — Accept the attrition, improve the band for the next hire
**Rule:** If you're regularly in counter-offer conversations, your bands are stale. Fix the bands.
---
## Pay Equity Audit
Run annually. Non-negotiable at Series B+.
**What to audit:**
- Pay gap by gender within each level and function
- Pay gap by ethnicity within each level and function
- Compa-ratio distribution across demographics
- Time-to-promotion by demographic group
**Methodology:**
1. Pull all employee data: level, function, salary, tenure, performance ratings, gender, ethnicity
2. Run regression controlling for level, tenure, and performance
3. Unexplained gap after controls = the problem to fix
4. Flag and remediate within the same review cycle
**Legal exposure:** In many jurisdictions, documented pay gaps without remediation plans are litigation risk. The audit creates a record of intent; remediation closes the risk.
**Remediation budget:** Set aside 0.5–1% of payroll annually for equity adjustments. If you're doing it right, this shrinks over time.
FILE:references/org_design.md
# Org Design Reference
Spans of control, layering decisions, reorgs, title frameworks, career ladders, and the founder→professional management transition.
---
## Core Org Design Principles
1. **Structure follows strategy.** Reorg after strategy shifts, not before.
2. **Optimize for the bottleneck.** Where does work get slow? Design around that.
3. **Minimize coordination cost.** Conway's Law: your org structure becomes your product architecture. Design intentionally.
4. **Bias toward flatness until it breaks.** Adding layers adds cost and slows decisions.
5. **Reorgs have transition costs.** Relationships reset. Count the cost before you restructure.
---
## Spans of Control
Span of control = number of direct reports a manager has.
### Benchmarks
| Role Type | Optimal Span | Min | Max |
|-----------|-------------|-----|-----|
| IC manager (predictable work) | 7–10 | 5 | 12 |
| IC manager (complex/creative work) | 5–7 | 4 | 8 |
| Manager of managers | 4–6 | 3 | 7 |
| VP / Director | 4–7 | 3 | 8 |
| C-Suite | 5–9 | 4 | 10 |
**Too narrow (< 4 ICs):** Over-management, high cost per output, manager becomes a bottleneck
**Too wide (> 12 ICs):** Under-management, degraded 1:1 quality, feedback loops collapse
### Factors that allow wider spans
- Highly autonomous, senior team (L3+ ICs)
- Predictable, well-defined work (support, ops)
- Strong tooling and process (reduces manager overhead)
- Experienced manager
### Factors that require narrower spans
- High-complexity, undefined problems (research, early product)
- Junior or newly promoted team members
- High interdependence between reports (coordination overhead)
- Manager is also an IC contributor (player-coach)
---
## When to Add Management Layers
**The wrong reason to add layers:** "We need to give good people somewhere to grow."
**The right reason:** "This manager has too many direct reports to do the job well."
### Layer triggers by growth stage
**0 → 15 people:** No layers. Everyone reports to founders.
**15 → 30 people:** First managers emerge. Usually technical leads or function leads. Should still be player-coaches.
**30 → 60 people:** Second layer forms. Engineering splits into squads. Sales gets a frontline manager. Each function has a head.
**60 → 150 people:** Director layer becomes necessary in large functions. Engineering VP + Engineering Directors + Team Managers.
**150+ people:** VP layer fully staffed. Senior Director / Director split. Clear IC → M → Senior M → Director → VP paths.
### The Rule of 7
When any manager has 7 or more direct reports and:
- 1:1s are skipped regularly
- Feedback quality drops
- Manager can't answer "how is each person doing?" without checking notes
→ Time to split or hire a manager.
### Management overhead cost
Every manager layer costs 10–15% in decision speed (communication hops).
Every management role without a team = pure overhead.
**Litmus test for each management role:**
- Does this person have at least 4 ICs under them?
- Would removing this role improve decision speed?
- Is this a management job or a "we ran out of IC levels" job?
---
## Functional vs. Product Org Structures
### Functional Structure (by discipline)
```
CEO
├── VP Engineering
│ ├── Backend Team
│ ├── Frontend Team
│ └── DevOps
├── VP Product
│ ├── PM (Feature A)
│ └── PM (Feature B)
└── VP Design
└── UX Designers
```
**Best for:** Early stage, < 100 people, single product
**Advantage:** Deep expertise development, clear career paths per discipline
**Disadvantage:** Cross-functional coordination is heavy; features require synchronization across silos
### Product/Pod Structure (by product area)
```
CEO
├── Product Area A (autonomous team)
│ ├── EM
│ ├── PM
│ └── Designer
├── Product Area B (autonomous team)
│ ├── EM
│ ├── PM
│ └── Designer
└── Platform (shared services)
└── Platform EM + team
```
**Best for:** Multiple products or large user segments, 50+ in product/eng
**Advantage:** Speed and autonomy; less cross-team coordination for most features
**Disadvantage:** Duplication risk; harder to maintain technical coherence; harder career paths
### When to shift from Functional → Product org
- You have 2+ distinct product lines that rarely share features
- Cross-functional feature delivery takes > 3 sprints of coordination overhead
- Teams are > 8 engineers and still waiting on shared resources
### Hybrid / Matrix (avoid unless necessary)
Matrix reporting (e.g., engineer reports to EM + PM) creates accountability confusion. Avoid at < 500 people.
---
## Title Frameworks
### The Problem with Title Inflation
Early startups over-title to compete with cash. "VP of Engineering" with 2 reports. "Head of Marketing" with no team.
**Consequences:**
- Can't add leadership above inflated titles without awkward conversations
- Candidates from mature companies expect scope commensurate with titles
- Internal equity breaks when the same title means different things
### Preventing Title Inflation
**Rule 1:** VP titles require managing managers (not just ICs).
**Rule 2:** Director titles require managing multiple ICs or a large function.
**Rule 3:** No more than one "Head of X" per function.
**Rule 4:** Document scope expectations per title before making offers.
### Engineering Title Ladder (example)
| Title | Level | Scope | Reports |
|-------|-------|-------|---------|
| Software Engineer I | L1 | Executes defined tasks | — |
| Software Engineer II | L2 | Independent delivery | — |
| Senior Software Engineer | L3 | Leads features, mentors | — |
| Staff Software Engineer | L4 | Cross-team technical leadership | — |
| Principal Software Engineer | L5 | Company-wide technical direction | — |
| Distinguished Engineer | L6 | External recognition, defining practice | — |
| Engineering Manager | M1 | Team of 4–8 engineers | 4–8 ICs |
| Senior Engineering Manager | M2 | Larger team or manager of managers | 2–4 managers |
| Director of Engineering | M3 | Functional area | Multiple managers |
| VP of Engineering | M4 | Engineering org | Directors |
| CTO | M5 | Technical organization + strategy | VPs |
**IC vs. Management track:** Explicitly separate. Senior ICs should not need to move to management for career advancement. Staff/Principal/Distinguished track provides this.
### Go-to-Market Title Ladder (example)
| Title | Level | Focus |
|-------|-------|-------|
| SDR / BDR | S1 | Outbound prospecting |
| Account Executive I | S2 | SMB closing |
| Account Executive II | S3 | Mid-market closing |
| Senior Account Executive | S4 | Enterprise closing |
| Principal / Strategic AE | S5 | Named accounts, complex deals |
| Sales Manager | M1 | 6–8 reps |
| Director of Sales | M2 | Multiple teams or segments |
| VP of Sales | M3 | Full sales org |
| CRO | M4 | Revenue org (sales + CS + marketing) |
---
## Career Ladders
A career ladder is a documented set of expectations per level. Not aspirational — behavioral. "What does a P3 engineer do that a P2 doesn't?"
### Why career ladders matter for HR
1. **Retention:** Employees can see where they're going
2. **Consistency:** Managers use the same criteria for promotions
3. **Compensation:** Bands anchor to levels; levels require definitions
4. **Equity:** Removes "who's the manager's favorite" from promotion decisions
### Career Ladder Structure
For each level, define 4 dimensions:
**1. Scope** — How big is the problem space? Team / cross-team / org-wide / company-wide?
**2. Impact** — How does work connect to outcomes? (Task → Feature → Product → Business)
**3. Craft** — Technical/functional skill expectations
**4. Influence** — How does this person improve others? (Self → peers → team → org)
**Example: Senior Software Engineer (L3) vs. Staff Software Engineer (L4)**
| Dimension | L3 (Senior SWE) | L4 (Staff SWE) |
|-----------|----------------|----------------|
| Scope | Owns features or services | Owns technical domains across teams |
| Impact | Ships features that improve user outcomes | Shapes technical direction for a product area |
| Craft | Writes high-quality code, good design skills | Sets coding standards, contributes to architecture |
| Influence | Mentors L1–L2, code reviews | Mentors L3+, identifies org-wide technical gaps |
### How to build a career ladder from scratch
1. **Interview your best performers** — "What do you do that your junior peers don't?" Collect behaviors, not aspirations.
2. **Draft 3 levels** — Don't start with 6. Start with junior, mid, senior. Add staff/principal only when you have enough people to warrant it.
3. **Manager calibration** — Every manager rates 5 current employees against the draft. Gaps surface immediately.
4. **Publish and iterate** — Don't wait for perfection. A 70% ladder shipped is better than a 100% ladder in a drawer.
---
## Reorg Playbook
### When reorgs are necessary
- Strategy pivot requires different team structure (e.g., single product → multi-product)
- Acquisition or team merger
- Function is genuinely too slow due to coordination overhead
- Leadership departure creates structural opportunity
### When reorgs are a mistake
- "We need to shake things up" (disruption for its own sake)
- Avoiding a specific personnel decision (use the right tool)
- Solving a cultural problem with a structural change
- Reacting to one team's complaint without systemic evidence
### Reorg Process (4–8 weeks)
**Week 1–2: Diagnose**
- Map current org: every role, reporting line, team output
- Identify where work is slow, duplicated, or falling through cracks
- Interview 5–10 people across teams: "What takes longer than it should? What decisions are hard to make?"
**Week 3–4: Design options**
- Draft 2–3 structural alternatives
- For each: estimated coordination costs, manager span impact, open roles created
- Validate with CEO + 1–2 trusted operators. Don't crowdsource the design.
**Week 5–6: Decide and prepare**
- Select option; finalize all reporting changes
- Prepare communications for every affected person (individual conversations before all-hands)
- Write the "why" — employees need to understand the business reason, not just the result
**Week 7–8: Communicate and implement**
- Individual conversations with all manager+ changes (first)
- Team-level conversations with managers (second)
- All-hands with full context (third)
- Updated org chart published within 24 hours of announcement
### Communication sequence (non-negotiable)
1. Affected individuals first (private, before anything else)
2. Affected managers second (to prepare for team conversations)
3. Full team/company third (all-hands or company note)
4. External (clients, board) only if materially impacted
**Never:** Email blast first. No individual conversations. Discovered on the org chart.
---
## Founder → Professional Management Transition
The most common scaling failure point in startups.
### Stage 1: Founder-Led (0–30 people)
Founders make all decisions, know everyone personally, set culture through behavior. Works because trust and context are built directly.
**What breaks:**
- Decisions bottleneck at founders
- New hires don't get enough context (founders can't be everywhere)
- Culture transmitted through osmosis, not documentation
### Stage 2: First Managers (30–80 people)
Founders can no longer manage all ICs. First manager layer typically = promoted high performers.
**The "brilliant IC → struggling manager" trap:**
- Individual contributor skills ≠ management skills
- Promoted ICs often continue doing IC work while ignoring management work
- No one holds them accountable to management output (1:1 quality, team health, performance feedback)
**What to do:**
- Explicit manager training before promotion (not after)
- Management KPIs separate from IC KPIs
- Peer community for new managers (monthly cohort session)
- HR check-ins on manager health at 30/60/90 days
### Stage 3: Professional Management (80–200 people)
External hires at Director/VP level bring professional management skills but lack company context.
**Common failure modes:**
- Hired "too senior" — VP who's used to 200-person teams in a 50-person function
- Culture clash — Big-company manager who adds process that kills startup speed
- Authority vacuum — External VP doesn't earn trust; team ignores them; founder continues to bypass hierarchy
**Mitigation:**
- Hiring bar: Has this person scaled from this stage to 2x this stage before? Not managed a team at 2x — built a team to 2x.
- Explicit onboarding on "how we make decisions here"
- 90-day milestones focused on relationship-building before any structural changes
- Founders explicitly hand off ownership and reinforce new manager's authority publicly
### Stage 4: Founder Transition from Operator to Executive
The hardest personal transition. Founder moves from doing to enabling.
**Signs you haven't made the transition:**
- You're still in every technical decision
- Teams come to you instead of their manager for approvals
- You know more about the team's work than the manager does
- Managers feel they need to check in before acting
**What the transition requires:**
- Explicit authority delegation in writing (not just verbal)
- Willingness to let managers make decisions you'd make differently
- Redirecting team members to their manager consistently
- Measuring managers on outcomes, not just process adherence
- Letting managers hire and fire without founder override (except final call on VPs)
FILE:references/people_strategy.md
# People Strategy Reference
Hiring, retention, performance, and remote/hybrid frameworks for each growth stage.
---
## Hiring Strategy by Growth Stage
### Pre-Seed / Seed (1–15 people)
**Who you're hiring:** Generalists who can do multiple jobs. Specialists are a luxury you can't afford unless the specialty is your core product.
**The test:** Could this person be the 5th employee at a startup and thrive? If they need a defined role, clear process, and a manager — not yet.
**Sourcing at this stage:**
- Founder networks first (highest signal, lowest cost)
- Angel List / Wellfound — self-selected for startup risk tolerance
- Referrals from existing employees (offer a referral bonus from day 1)
- GitHub / Dribbble / published work for technical roles
- Avoid: Big job boards, recruiters (unless technical retained search for C-suite)
**Interview process (keep it lean):**
1. 30-min intro call (culture/motivation fit, comp alignment)
2. Take-home or live work sample (2–4 hours max, paid for senior roles)
3. 60-min deep-dive with founders
4. Reference checks (3 calls, not emails — you want the real story)
**Offer timeline:** Decision within 48 hours. Top candidates have multiple offers.
**What to get right:**
- Written job scorecard (outcomes expected in 30/60/90 days) — not a job description
- Equity range disclosed in first conversation
- No exploding offers. Pressure tactics lose good people.
---
### Series A (15–50 people)
**The hiring shift:** You need some specialists now. First management layer emerges. First "culture carries" — people who reinforce what you want to become.
**Critical hires at this stage (in priority order):**
1. VP/Head of Engineering (if founder isn't technical)
2. Head of Product
3. First dedicated recruiter (when you're hiring > 10/year)
4. First Finance/Operations hire
5. Head of Sales (when product-market fit is real)
**Building the recruiting function:**
- First recruiter should be a generalist with hustle, not a specialist
- Set up an ATS (Ashby, Greenhouse, or Lever) before you need it — not after
- Create interview scorecards for every role
- Track: time to fill, offer acceptance rate, source quality
**Common mistakes at Series A:**
- Promoting top ICs to management without management training
- Hiring "brand name" executives who've never operated lean
- Over-indexing on experience, under-indexing on trajectory
- No onboarding process → 90-day regrettable turnover
**Job scorecards (required for every role):**
```
Role: [Title]
Reports to: [Manager]
Start date: [Target]
Why this role now: [Business case in 1-2 sentences]
Outcomes (90 days):
- [Concrete deliverable 1]
- [Concrete deliverable 2]
- [Concrete deliverable 3]
Outcomes (12 months):
- [Strategic impact 1]
- [Strategic impact 2]
Competencies (top 3 only):
- [What, why it matters for THIS role]
- [What, why it matters for THIS role]
- [What, why it matters for THIS role]
Comp range: [Base] + [Equity] + [Benefits summary]
```
---
### Series B (50–150 people)
**The scaling inflection point.** Tribal knowledge breaks. Process matters now. Culture requires deliberate investment.
**What changes:**
- Recruiters become specialists (technical, GTM, exec)
- Manager training becomes non-negotiable
- Performance management needs structure (not just "we'll know it when we see it")
- Onboarding needs to scale without founders in every session
- Comp bands become essential — people are comparing notes
**Hiring velocity benchmarks (Series B):**
| Function | Avg time to fill | Avg interviews | Benchmark offer acceptance |
|----------|-----------------|----------------|---------------------------|
| Engineering IC | 35–45 days | 4–5 rounds | 80–85% |
| Engineering Manager | 45–60 days | 5–6 rounds | 75–80% |
| Sales IC | 25–35 days | 3–4 rounds | 85–90% |
| Sales Manager | 40–55 days | 4–5 rounds | 80–85% |
| G&A (Finance, HR, Ops) | 30–45 days | 3–4 rounds | 85–90% |
**Internal mobility:** By 50 people, start tracking internal promotion rates. Target: 20–30% of manager+ roles filled internally. If it's < 10%, your career development is failing.
---
### Series C+ (150+ people)
**Professional management era.** Founders can't know everyone. Systems and culture carry what personal relationships used to.
**HR function maturity required:**
- Dedicated HRBPs per business unit (1:75–100 employees)
- L&D budget (1–2% of salary budget minimum)
- Succession planning for all VP+ roles
- Structured calibration process for performance reviews
- Total rewards strategy reviewed annually with board
---
## Retention Programs That Actually Work
### What drives retention (in order of impact)
1. **Manager quality** — Gallup: 70% of team engagement variance is explained by the manager. Fix managers first.
2. **Growth trajectory** — People leave when they can't see their next role. Career ladders are retention tools.
3. **Compensation competitiveness** — Being at P25 on salary is a slow leak. Audit annually.
4. **Mission/product belief** — Especially for senior ICs. They want to work on something that matters.
5. **Team quality** — "I stay because of the people I work with." True at every level.
6. **Flexibility** — Location, hours, autonomy. Low cost, high impact.
### What doesn't work (but companies do anyway)
- Pizza parties and ping pong tables
- "Perks" that substitute for salary
- Annual reviews with no action on feedback
- Forced fun events
- Vague "culture improvement" initiatives without specific behavior changes
### The 30-60-90 Onboarding Framework
Structured onboarding cuts 90-day turnover by 50%+.
**Days 1–30: Learn**
- Complete admin setup (day 1, before lunch)
- Meet all key stakeholders (scheduled by their manager, not on the new hire)
- Understand: business model, current priorities, team processes, how success is measured
- No deliverables expected. Learning is the job.
- Weekly 1:1 with manager: "What's confusing? What do you need?"
**Days 31–60: Contribute**
- First real project (scoped to be completable)
- Present findings or work to the team
- Identify one process that could be improved (observation only — don't fix yet)
- 30-day check-in: formal feedback from manager
**Days 61–90: Lead**
- Own a deliverable end-to-end
- Offer one specific improvement recommendation with data
- 90-day review: mutual assessment — manager on new hire, new hire on onboarding
- Set 6-month goals
### Stay Interviews (underused, high ROI)
Run with every employee once per year. Not their manager — HR or skip-level.
**Questions that surface real risk:**
- "What's keeping you here?"
- "What would make you consider leaving?"
- "What's one thing your manager could do differently?"
- "Is your role what you expected when you joined?"
- "What career path do you want? Are we helping you get there?"
- "Are you fairly compensated? Do you know how you'd get a raise?"
**Act on answers within 30 days or don't ask.** Unanswered feedback is worse than no feedback.
### Exit Interviews — What to Actually Learn
Skip the happiness survey. Ask these:
- "When did you first think about leaving?"
- "Was there a specific event that triggered your decision?"
- "What could we have done to retain you?"
- "Where are you going and why?" (What does the other offer have that we don't?)
- "Would you recommend us as an employer? Why or why not?"
Track exit themes by manager. If one manager's exits cite "micromanagement" three times — that's data.
---
## Performance Management
### The System That Works
**Continuous > annual.** Annual reviews with no mid-year touchpoints are theater.
**Structure:**
- **Weekly 1:1s** (30 min): blockers, priorities, relationship
- **Monthly check-ins** (1 hr): progress against goals, feedback exchange
- **Quarterly reviews** (formal): written self-assessment + manager assessment + goal revision
- **Annual calibration** (rating + comp): cross-manager calibration session, then individual conversations
### Calibration Sessions
**Purpose:** Prevent manager bias. Ensure "exceeds expectations" means the same thing across teams.
**Process:**
1. Managers submit preliminary ratings independently
2. HR facilitates 2-hr calibration with all managers in a function
3. Managers must justify outliers (top and bottom)
4. Ratings adjusted for consistency
5. Managers deliver final ratings with rationale
**Distribution guidance (enforce with calibration):**
- Exceptional (5): < 10% — if everyone's exceptional, no one is
- Exceeds (4): 20–25%
- Meets (3): 55–65%
- Needs improvement (2): 8–12%
- Underperforming (1): 2–5%
### Managing Underperformers
**The most avoided management task. And the most damaging when avoided.**
High performers notice when underperformers are tolerated. They leave.
**The 4-step framework:**
**Step 1: Diagnose before acting** (Week 1–2)
- Is this a skill gap (can't do it) or a will gap (won't do it)?
- Skill gap → training, clearer expectations, different role
- Will gap → direct feedback, clear consequences, then PIP
**Step 2: Direct feedback conversation** (Week 2–3)
- Specific: "Your last 3 sprint deliveries were 40% incomplete"
- Not: "You're not meeting expectations"
- Document. Send written summary after every feedback conversation.
**Step 3: Performance Improvement Plan (PIP)**
Required when: two rounds of direct feedback haven't produced change.
PIP structure:
```
Name: [Employee]
Manager: [Name]
Date: [Start]
Review date: [30/60 days out]
Current performance issues:
- [Specific, observable behavior with examples and dates]
- [Metric not met: target X, actual Y for Z weeks]
Required improvements:
- [Specific, measurable outcome 1] by [date]
- [Specific, measurable outcome 2] by [date]
Support provided:
- [Training, coaching, additional resources]
Consequences if not met: [Role change / separation]
Check-in schedule: [Weekly with manager + HR]
```
**Step 4: Exit or role change**
- If PIP milestones not met: proceed to separation
- Don't extend PIPs indefinitely — it's unfair to the employee and the team
- Offer a graceful exit where possible: "This role isn't the right fit. Here's a package and a reference."
**What not to do:**
- "Quiet manage out" without clear feedback (legally risky, unfair)
- PIP as a formality before termination (if you know you're firing them, just do it)
- Tolerating underperformance "because we're understaffed" (it makes understaffing worse)
---
## Remote / Hybrid Strategy
### The question isn't "remote or not" — it's "what kind of collaboration does our work require?"
**Work type taxonomy:**
| Work type | Remote-compatible? | Hybrid compatible? |
|-----------|-------------------|-------------------|
| Deep individual work (coding, writing, analysis) | Yes | Yes |
| Async collaboration (code review, doc review) | Yes | Yes |
| Synchronous problem-solving (debugging, design) | Yes (video) | Yes |
| Relationship-building (onboarding, new team) | Harder | Yes |
| Executive alignment, strategy | Harder | Yes — quarterly in-person |
| Sales (enterprise, relationship-based) | No | Depends on market |
### Making Hybrid Work (Not Just a Policy)
**The failure mode:** "Hybrid" = go to office on Tuesday/Thursday, but no one coordinates, all meetings are still Zoom anyway.
**What actually works:**
1. **Anchor days with purpose** — Office days should have things that require the office: workshops, team rituals, whiteboarding sessions. Not just "presence."
2. **Async-first culture, not async-only** — Document decisions. Write things down. Use Loom for walkthroughs. Reduce "quick sync" meetings.
3. **Equal experience for remote participants** — If some are in the room and some are on video, the remote folks are second-class. Either everyone's remote or set up rooms properly.
4. **Manager standards for remote teams:**
- 1:1s are non-negotiable (video, not async)
- Over-communicate on priorities (people can't absorb hallway context)
- Write down decisions (remote employees miss casual office decisions)
- Recognize work publicly (Slack shoutouts, all-hands wins)
### Remote Compensation Philosophy (pick one, be explicit)
**Option A: Location-based pay**
Pay based on where the employee lives. Lower cost in lower-cost markets. Harder to hire in high-cost cities.
**Option B: Role-based (location-neutral)**
One band for each role regardless of location. Simpler, more equitable. Higher overall payroll cost.
**Option C: Zone-based**
Define 2–3 geographic zones (e.g., Tier 1 cities, Tier 2 cities, international). Set bands per zone. Common at mid-stage startups.
**The wrong answer:** No stated policy, and every offer is negotiated individually. Creates pay equity problems fast.
FILE:scripts/comp_benchmarker.py
#!/usr/bin/env python3
"""
Compensation Benchmarker
========================
Salary benchmarking and total comp modeling for startup teams.
Analyzes pay equity, compa-ratios, and total comp vs. market.
Usage:
python comp_benchmarker.py # Run with built-in sample data
python comp_benchmarker.py --config roster.json # Load from JSON
python comp_benchmarker.py --help
Output: Band compliance report, compa-ratio distribution, pay equity flags,
equity value analysis, and total comp vs. market.
"""
import argparse
import json
import csv
import io
import sys
from dataclasses import dataclass, field, asdict
from typing import Optional
from datetime import date
import math
# ---------------------------------------------------------------------------
# Data structures
# ---------------------------------------------------------------------------
@dataclass
class BandDefinition:
"""Salary band for a role level."""
level: str # L1, L2, L3, L4, M1, M2, M3, VP
function: str # Engineering, Sales, Product, G&A, Marketing, CS
band_min: int # Annual USD
band_mid: int # P50 anchor
band_max: int # Band ceiling
market_p25: int # Market 25th percentile
market_p50: int # Market median (should align with band_mid for P50 strategy)
market_p75: int # Market 75th percentile
location_zone: str # Tier1 (SF/NYC), Tier2 (Austin/Denver), Tier3 (Remote/other), EU
@dataclass
class Employee:
"""One employee record."""
id: str
name: str
role: str
level: str
function: str
location_zone: str
base_salary: int
bonus_target_pct: float # % of base
equity_shares: int # Total unvested options/RSUs
equity_strike: float # Strike price (0 for RSUs)
equity_current_409a: float # Current 409A share price
equity_vest_years_remaining: float # How many years of vesting remain
benefits_annual: int # Employer-paid benefits cost
gender: str # M/F/NB/Undisclosed (for equity audit)
ethnicity: str # For equity audit — can be "Undisclosed"
tenure_years: float
performance_rating: int # 1–5
last_raise_months_ago: int
last_equity_refresh_months_ago: Optional[int] = None
@dataclass
class CompRoster:
company: str
as_of_date: str # ISO date
funding_stage: str # Seed, Series A, Series B, etc.
comp_philosophy_target: str # P50, P65, P75 — your target percentile
preferred_stock_price: float # Last round price (for offer modeling)
employees: list[Employee] = field(default_factory=list)
bands: list[BandDefinition] = field(default_factory=list)
# ---------------------------------------------------------------------------
# Band lookup
# ---------------------------------------------------------------------------
def find_band(roster: CompRoster, level: str, function: str, zone: str) -> Optional[BandDefinition]:
"""Find best-matching band. Falls back to any matching level+function if zone not found."""
matches = [b for b in roster.bands if b.level == level and b.function == function and b.location_zone == zone]
if matches:
return matches[0]
# Fallback: same level+function, any zone
matches = [b for b in roster.bands if b.level == level and b.function == function]
if matches:
return matches[0]
# Fallback: same level, any function
matches = [b for b in roster.bands if b.level == level]
if matches:
return matches[0]
return None
# ---------------------------------------------------------------------------
# Compensation analysis
# ---------------------------------------------------------------------------
def compa_ratio(salary: int, band_mid: int) -> float:
return salary / band_mid if band_mid > 0 else 0.0
def band_position(salary: int, band_min: int, band_max: int) -> float:
"""Position in band: 0.0 = at min, 1.0 = at max."""
if band_max == band_min:
return 0.5
return (salary - band_min) / (band_max - band_min)
def annualized_equity_value(emp: Employee) -> int:
"""Current 409A value of unvested equity, annualized."""
if emp.equity_vest_years_remaining <= 0:
return 0
if emp.equity_current_409a > emp.equity_strike:
intrinsic = (emp.equity_current_409a - emp.equity_strike) * emp.equity_shares
else:
# Options underwater — still show at current FMV for RSUs or future value for options
intrinsic = emp.equity_current_409a * emp.equity_shares if emp.equity_strike == 0 else 0
return int(intrinsic / emp.equity_vest_years_remaining)
def total_comp(emp: Employee) -> int:
bonus = int(emp.base_salary * emp.bonus_target_pct)
equity = annualized_equity_value(emp)
return emp.base_salary + bonus + equity + emp.benefits_annual
def analyze_employee(emp: Employee, roster: CompRoster) -> dict:
band = find_band(roster, emp.level, emp.function, emp.location_zone)
result = {
"id": emp.id,
"name": emp.name,
"role": emp.role,
"level": emp.level,
"function": emp.function,
"zone": emp.location_zone,
"base": emp.base_salary,
"bonus_target": int(emp.base_salary * emp.bonus_target_pct),
"equity_annual": annualized_equity_value(emp),
"benefits": emp.benefits_annual,
"total_comp": total_comp(emp),
"performance": emp.performance_rating,
"tenure_years": emp.tenure_years,
"last_raise_months": emp.last_raise_months_ago,
"band": band,
"compa_ratio": None,
"band_position": None,
"vs_market_p50": None,
"flags": [],
}
if band:
cr = compa_ratio(emp.base_salary, band.band_mid)
bp = band_position(emp.base_salary, band.band_min, band.band_max)
result["compa_ratio"] = round(cr, 3)
result["band_position"] = round(bp, 3)
result["vs_market_p50"] = round((emp.base_salary - band.market_p50) / band.market_p50 * 100, 1)
# Flags
if emp.base_salary < band.band_min:
result["flags"].append(("CRITICAL", "Base below band minimum — immediate attrition risk"))
elif cr < 0.88:
result["flags"].append(("HIGH", f"Compa-ratio {cr:.2f} — significantly below midpoint"))
elif cr < 0.93:
result["flags"].append(("MEDIUM", f"Compa-ratio {cr:.2f} — below target zone (0.95–1.05)"))
if emp.base_salary > band.band_max:
result["flags"].append(("HIGH", "Base above band maximum — review for promotion or band update"))
if emp.performance_rating >= 4 and cr < 0.95:
result["flags"].append(("HIGH", f"High performer (rating {emp.performance_rating}) underpaid — flight risk"))
if emp.last_raise_months_ago > 18:
result["flags"].append(("MEDIUM", f"No raise in {emp.last_raise_months_ago} months — review due"))
if emp.equity_vest_years_remaining < 1.0 and (emp.last_equity_refresh_months_ago is None or emp.last_equity_refresh_months_ago > 24):
result["flags"].append(("HIGH", "Equity nearly fully vested with no refresh — retention hook gone"))
else:
result["flags"].append(("INFO", "No band found for this level/function/zone"))
return result
# ---------------------------------------------------------------------------
# Aggregate analysis
# ---------------------------------------------------------------------------
def pay_equity_audit(analyses: list[dict], employees: list[Employee]) -> dict:
"""Simple pay equity analysis by gender and ethnicity."""
emp_by_id = {e.id: e for e in employees}
def group_stats(group_key_fn):
groups: dict[str, list[float]] = {}
for a in analyses:
if a["compa_ratio"] is None:
continue
emp = emp_by_id.get(a["id"])
if not emp:
continue
key = group_key_fn(emp)
if key not in groups:
groups[key] = []
groups[key].append(a["compa_ratio"])
return {k: {"n": len(v), "avg_cr": round(sum(v)/len(v), 3), "min_cr": round(min(v), 3), "max_cr": round(max(v), 3)}
for k, v in groups.items() if v}
gender_stats = group_stats(lambda e: e.gender)
ethnicity_stats = group_stats(lambda e: e.ethnicity)
# Compute gap vs. the largest group
def compute_gap(stats: dict) -> dict[str, float]:
if not stats:
return {}
largest = max(stats.items(), key=lambda x: x[1]["n"])
ref_cr = largest[1]["avg_cr"]
return {k: round((v["avg_cr"] - ref_cr) / ref_cr * 100, 1) for k, v in stats.items()}
gender_gaps = compute_gap(gender_stats)
ethnicity_gaps = compute_gap(ethnicity_stats)
return {
"gender": gender_stats,
"gender_gaps_pct": gender_gaps,
"ethnicity": ethnicity_stats,
"ethnicity_gaps_pct": ethnicity_gaps,
}
def compa_ratio_distribution(analyses: list[dict]) -> dict:
crs = [a["compa_ratio"] for a in analyses if a["compa_ratio"] is not None]
if not crs:
return {}
buckets = {
"< 0.85 (below band)": 0,
"0.85–0.94 (developing)": 0,
"0.95–1.05 (target zone)": 0,
"1.06–1.15 (senior in role)": 0,
"> 1.15 (above band)": 0,
}
for cr in crs:
if cr < 0.85:
buckets["< 0.85 (below band)"] += 1
elif cr < 0.95:
buckets["0.85–0.94 (developing)"] += 1
elif cr <= 1.05:
buckets["0.95–1.05 (target zone)"] += 1
elif cr <= 1.15:
buckets["1.06–1.15 (senior in role)"] += 1
else:
buckets["> 1.15 (above band)"] += 1
avg = sum(crs) / len(crs)
return {"distribution": buckets, "avg_compa_ratio": round(avg, 3), "n": len(crs)}
# ---------------------------------------------------------------------------
# Report output
# ---------------------------------------------------------------------------
def fmt(n) -> str:
return f",.0f"
def bar(value: float, width: int = 20) -> str:
filled = min(width, max(0, int(value * width)))
return "█" * filled + "░" * (width - filled)
def print_report(roster: CompRoster):
WIDTH = 76
SEP = "=" * WIDTH
sep = "-" * WIDTH
analyses = [analyze_employee(e, roster) for e in roster.employees]
cr_dist = compa_ratio_distribution(analyses)
equity_audit = pay_equity_audit(analyses, roster.employees)
print(SEP)
print(f" COMPENSATION BENCHMARKING REPORT — {roster.company}")
print(f" As of: {roster.as_of_date} | Stage: {roster.funding_stage} | Target: {roster.comp_philosophy_target}")
print(SEP)
# Summary stats
total_emps = len(roster.employees)
flagged = sum(1 for a in analyses if any(s in ["CRITICAL", "HIGH"] for s, _ in a["flags"]))
total_payroll = sum(e.base_salary for e in roster.employees)
avg_total_comp = sum(a["total_comp"] for a in analyses) // total_emps if total_emps else 0
print(f"\n[ SUMMARY ]")
print(sep)
print(f" Employees analyzed: {total_emps}")
print(f" Flagged (critical/high): {flagged}")
print(f" Total base payroll: {fmt(total_payroll)}/year")
print(f" Avg total comp: {fmt(avg_total_comp)}/year")
if cr_dist:
print(f" Avg compa-ratio: {cr_dist['avg_compa_ratio']:.3f}")
# Compa-ratio distribution
if cr_dist:
print(f"\n[ COMPA-RATIO DISTRIBUTION ]")
print(sep)
total_n = cr_dist["n"]
for label, count in cr_dist["distribution"].items():
pct = count / total_n if total_n else 0
bar_str = bar(pct, 25)
print(f" {label:<30} {bar_str} {count:3d} ({pct*100:4.0f}%)")
# Pay equity audit
print(f"\n[ PAY EQUITY AUDIT ]")
print(sep)
print(f" By Gender:")
for group, stats in equity_audit["gender"].items():
gap = equity_audit["gender_gaps_pct"].get(group, 0.0)
gap_str = f" gap: {gap:+.1f}%" if gap != 0 else " (reference group)"
flag = " ⚠" if abs(gap) > 5 else ""
print(f" {group:<15} n={stats['n']} avg_CR={stats['avg_cr']:.3f}{gap_str}{flag}")
print(f"\n By Ethnicity:")
for group, stats in equity_audit["ethnicity"].items():
gap = equity_audit["ethnicity_gaps_pct"].get(group, 0.0)
gap_str = f" gap: {gap:+.1f}%" if gap != 0 else " (reference group)"
flag = " ⚠" if abs(gap) > 5 else ""
print(f" {group:<20} n={stats['n']} avg_CR={stats['avg_cr']:.3f}{gap_str}{flag}")
print(f"\n ⚠ = gap > 5%. Investigate with regression controlling for level, tenure, and performance.")
# Employee detail with flags
print(f"\n[ EMPLOYEE DETAIL ]")
print(sep)
# Group by function
functions = sorted(set(e.function for e in roster.employees))
for fn in functions:
fn_analyses = [a for a in analyses if a["function"] == fn]
if not fn_analyses:
continue
print(f"\n ── {fn} ──")
print(f" {'Name':<22} {'Role':<28} {'Lvl':<5} {'Base':>10} {'TotalComp':>11} {'CR':>6} {'Perf':>5} Flags")
print(f" {'-'*22} {'-'*28} {'-'*5} {'-'*10} {'-'*11} {'-'*6} {'-'*5} {'-'*20}")
for a in sorted(fn_analyses, key=lambda x: -x["base"]):
cr_str = f"{a['compa_ratio']:.2f}" if a["compa_ratio"] else "N/A"
flag_summary = ", ".join(s for s, _ in a["flags"] if s in ("CRITICAL", "HIGH", "MEDIUM"))
flag_str = flag_summary if flag_summary else "OK"
print(f" {a['name']:<22} {a['role']:<28} {a['level']:<5} "
f"{fmt(a['base']):>10} {fmt(a['total_comp']):>11} {cr_str:>6} {a['performance']:>5} {flag_str}")
# Print flag detail for critical/high
for severity, msg in a["flags"]:
if severity in ("CRITICAL", "HIGH"):
print(f" {'':>22} ↳ [{severity}] {msg}")
# Action items
critical = [(a["name"], msg) for a in analyses for sev, msg in a["flags"] if sev == "CRITICAL"]
high = [(a["name"], msg) for a in analyses for sev, msg in a["flags"] if sev == "HIGH"]
medium = [(a["name"], msg) for a in analyses for sev, msg in a["flags"] if sev == "MEDIUM"]
print(f"\n[ ACTION ITEMS ]")
print(sep)
if critical:
print(f"\n CRITICAL — Address this review cycle:")
for name, msg in critical:
print(f" • {name}: {msg}")
if high:
print(f"\n HIGH — Address within 30 days:")
for name, msg in high[:10]:
print(f" • {name}: {msg}")
if len(high) > 10:
print(f" ... and {len(high)-10} more")
if medium:
print(f"\n MEDIUM — Address in next comp cycle:")
for name, msg in medium[:8]:
print(f" • {name}: {msg}")
if len(medium) > 8:
print(f" ... and {len(medium)-8} more")
if not critical and not high and not medium:
print(f"\n No critical or high-severity issues. Compensation appears well-managed.")
# Remediation cost estimate
below_min = [a for a in analyses if a["band"] and a["base"] < a["band"].band_min]
below_mid = [a for a in analyses if a["compa_ratio"] and a["compa_ratio"] < 0.90]
if below_min or below_mid:
print(f"\n[ REMEDIATION COST ESTIMATE ]")
print(sep)
if below_min:
cost_to_min = sum(a["band"].band_min - a["base"] for a in below_min)
print(f" Cost to bring below-minimum to band min: {fmt(cost_to_min)}/year ({len(below_min)} employees)")
if below_mid:
cost_to_90 = sum(int(a["band"].band_mid * 0.90) - a["base"] for a in below_mid if a["base"] < int(a["band"].band_mid * 0.90))
cost_to_90 = max(0, cost_to_90)
print(f" Cost to bring CR < 0.90 to CR = 0.90: {fmt(cost_to_90)}/year ({len(below_mid)} employees)")
total_payroll_impact = sum(e.base_salary for e in roster.employees)
total_remediation = (below_min and cost_to_min or 0)
print(f"\n Total payroll before remediation: {fmt(total_payroll_impact)}/year")
print(f" Remediation as % of payroll: {total_remediation/total_payroll_impact*100:.1f}%")
print(f"\n{SEP}\n")
def export_csv(roster: CompRoster) -> str:
analyses = [analyze_employee(e, roster) for e in roster.employees]
output = io.StringIO()
writer = csv.writer(output)
writer.writerow(["ID", "Name", "Role", "Level", "Function", "Zone",
"Base", "Bonus Target", "Equity Annual", "Benefits", "Total Comp",
"Compa Ratio", "Band Position", "vs Market P50 %",
"Performance", "Tenure Years", "Last Raise (mo)",
"Gender", "Ethnicity", "Critical Flags", "High Flags"])
for a, e in zip(analyses, roster.employees):
critical_flags = "; ".join(msg for sev, msg in a["flags"] if sev == "CRITICAL")
high_flags = "; ".join(msg for sev, msg in a["flags"] if sev == "HIGH")
writer.writerow([a["id"], a["name"], a["role"], a["level"], a["function"], a["zone"],
a["base"], a["bonus_target"], a["equity_annual"], a["benefits"], a["total_comp"],
a["compa_ratio"], a["band_position"], a["vs_market_p50"],
a["performance"], a["tenure_years"], a["last_raise_months"],
e.gender, e.ethnicity, critical_flags, high_flags])
return output.getvalue()
# ---------------------------------------------------------------------------
# Sample data
# ---------------------------------------------------------------------------
def build_sample_roster() -> CompRoster:
roster = CompRoster(
company="AcmeTech (Series A)",
as_of_date=date.today().isoformat(),
funding_stage="Series A",
comp_philosophy_target="P50",
preferred_stock_price=8.50,
)
# Bands (Engineering, P50 target, Tier1 = SF/NYC)
roster.bands = [
BandDefinition("L2", "Engineering", 115_000, 132_000, 155_000, 110_000, 132_000, 155_000, "Tier1"),
BandDefinition("L3", "Engineering", 148_000, 170_000, 198_000, 145_000, 170_000, 198_000, "Tier1"),
BandDefinition("L4", "Engineering", 185_000, 215_000, 248_000, 182_000, 215_000, 250_000, "Tier1"),
BandDefinition("M1", "Engineering", 170_000, 195_000, 225_000, 168_000, 195_000, 225_000, "Tier1"),
BandDefinition("L2", "Engineering", 95_000, 108_000, 125_000, 92_000, 108_000, 126_000, "Tier2"),
BandDefinition("L3", "Engineering", 122_000, 140_000, 162_000, 120_000, 140_000, 162_000, "Tier2"),
BandDefinition("L2", "Sales", 80_000, 92_000, 108_000, 78_000, 92_000, 108_000, "Tier1"),
BandDefinition("L3", "Sales", 95_000, 110_000, 128_000, 93_000, 110_000, 128_000, "Tier1"),
BandDefinition("M1", "Sales", 130_000, 150_000, 172_000, 128_000, 150_000, 172_000, "Tier1"),
BandDefinition("L2", "Product", 125_000, 145_000, 168_000, 123_000, 145_000, 168_000, "Tier1"),
BandDefinition("L3", "Product", 155_000, 178_000, 205_000, 153_000, 178_000, 205_000, "Tier1"),
BandDefinition("L2", "G&A", 85_000, 98_000, 115_000, 83_000, 98_000, 115_000, "Tier1"),
BandDefinition("L3", "G&A", 110_000, 128_000, 148_000, 108_000, 128_000, 148_000, "Tier1"),
]
roster.employees = [
# Engineering — mix of scenarios
Employee("E001", "Aarav Shah", "Senior SWE (Backend)", "L3", "Engineering", "Tier1",
base_salary=168_000, bonus_target_pct=0.0, equity_shares=40_000,
equity_strike=1.50, equity_current_409a=6.80, equity_vest_years_remaining=2.5,
benefits_annual=18_000, gender="M", ethnicity="Asian",
tenure_years=2.5, performance_rating=4, last_raise_months_ago=14,
last_equity_refresh_months_ago=None),
Employee("E002", "Yuki Tanaka", "Senior SWE (Frontend)", "L3", "Engineering", "Tier1",
base_salary=152_000, bonus_target_pct=0.0, equity_shares=30_000,
equity_strike=2.20, equity_current_409a=6.80, equity_vest_years_remaining=0.5,
benefits_annual=18_000, gender="F", ethnicity="Asian",
tenure_years=3.8, performance_rating=5, last_raise_months_ago=11,
last_equity_refresh_months_ago=30),
# Note: Yuki is high performer, near-vested, no recent refresh — flag expected
Employee("E003", "Marcus Johnson", "SWE II (Backend)", "L2", "Engineering", "Tier1",
base_salary=110_000, bonus_target_pct=0.0, equity_shares=15_000,
equity_strike=2.50, equity_current_409a=6.80, equity_vest_years_remaining=3.0,
benefits_annual=15_000, gender="M", ethnicity="Black",
tenure_years=1.2, performance_rating=3, last_raise_months_ago=12,
last_equity_refresh_months_ago=None),
# Note: Below band midpoint, recently hired — developing flag
Employee("E004", "Priya Nair", "Staff SWE", "L4", "Engineering", "Tier1",
base_salary=222_000, bonus_target_pct=0.0, equity_shares=60_000,
equity_strike=0.80, equity_current_409a=6.80, equity_vest_years_remaining=2.0,
benefits_annual=18_000, gender="F", ethnicity="Asian",
tenure_years=4.2, performance_rating=5, last_raise_months_ago=8,
last_equity_refresh_months_ago=8),
Employee("E005", "Tom Rivera", "SWE II (Platform)", "L2", "Engineering", "Tier2",
base_salary=88_000, bonus_target_pct=0.0, equity_shares=12_000,
equity_strike=3.00, equity_current_409a=6.80, equity_vest_years_remaining=2.5,
benefits_annual=14_000, gender="M", ethnicity="Hispanic",
tenure_years=1.8, performance_rating=4, last_raise_months_ago=22,
last_equity_refresh_months_ago=None),
# Note: No raise in 22 months, high performer — flag expected
Employee("E006", "Sarah Kim", "Eng Manager", "M1", "Engineering", "Tier1",
base_salary=192_000, bonus_target_pct=0.10, equity_shares=35_000,
equity_strike=1.20, equity_current_409a=6.80, equity_vest_years_remaining=1.8,
benefits_annual=18_000, gender="F", ethnicity="Asian",
tenure_years=2.8, performance_rating=4, last_raise_months_ago=9,
last_equity_refresh_months_ago=9),
# Sales
Employee("S001", "David Chen", "Account Executive (MM)", "L3", "Sales", "Tier1",
base_salary=105_000, bonus_target_pct=0.50, equity_shares=8_000,
equity_strike=3.50, equity_current_409a=6.80, equity_vest_years_remaining=2.0,
benefits_annual=15_000, gender="M", ethnicity="Asian",
tenure_years=1.5, performance_rating=3, last_raise_months_ago=15,
last_equity_refresh_months_ago=None),
Employee("S002", "Amara Osei", "AE (Mid-Market)", "L3", "Sales", "Tier1",
base_salary=98_000, bonus_target_pct=0.50, equity_shares=6_000,
equity_strike=3.50, equity_current_409a=6.80, equity_vest_years_remaining=2.5,
benefits_annual=15_000, gender="F", ethnicity="Black",
tenure_years=1.0, performance_rating=4, last_raise_months_ago=12,
last_equity_refresh_months_ago=None),
# Note: High performer, significantly below midpoint — flag expected
Employee("S003", "Jordan Blake", "Sales Manager", "M1", "Sales", "Tier1",
base_salary=155_000, bonus_target_pct=0.20, equity_shares=20_000,
equity_strike=2.00, equity_current_409a=6.80, equity_vest_years_remaining=1.5,
benefits_annual=16_000, gender="NB", ethnicity="White",
tenure_years=2.2, performance_rating=3, last_raise_months_ago=10,
last_equity_refresh_months_ago=10),
# Product
Employee("P001", "Nina Patel", "Senior PM", "L3", "Product", "Tier1",
base_salary=176_000, bonus_target_pct=0.10, equity_shares=22_000,
equity_strike=1.80, equity_current_409a=6.80, equity_vest_years_remaining=2.0,
benefits_annual=17_000, gender="F", ethnicity="Asian",
tenure_years=2.0, performance_rating=4, last_raise_months_ago=12,
last_equity_refresh_months_ago=12),
# G&A
Employee("G001", "Chris Mueller", "Finance Manager", "L3", "G&A", "Tier1",
base_salary=125_000, bonus_target_pct=0.10, equity_shares=10_000,
equity_strike=2.80, equity_current_409a=6.80, equity_vest_years_remaining=3.0,
benefits_annual=16_000, gender="M", ethnicity="White",
tenure_years=1.5, performance_rating=3, last_raise_months_ago=15,
last_equity_refresh_months_ago=None),
Employee("G002", "Fatima Al-Hassan", "HR Operations", "L2", "G&A", "Tier1",
base_salary=82_000, bonus_target_pct=0.08, equity_shares=5_000,
equity_strike=4.00, equity_current_409a=6.80, equity_vest_years_remaining=3.5,
benefits_annual=14_000, gender="F", ethnicity="Middle Eastern",
tenure_years=0.8, performance_rating=3, last_raise_months_ago=8,
last_equity_refresh_months_ago=None),
# Note: Below band minimum — critical flag expected
]
return roster
# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------
def load_roster_from_json(path: str) -> CompRoster:
with open(path) as f:
data = json.load(f)
employees = [Employee(**e) for e in data.pop("employees", [])]
bands = [BandDefinition(**b) for b in data.pop("bands", [])]
roster = CompRoster(**data)
roster.employees = employees
roster.bands = bands
return roster
def main():
parser = argparse.ArgumentParser(
description="Compensation Benchmarker — salary analysis and pay equity audit",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
python comp_benchmarker.py # Run sample roster
python comp_benchmarker.py --config roster.json # Load from JSON
python comp_benchmarker.py --export-csv # Output CSV
python comp_benchmarker.py --export-json # Output JSON template
"""
)
parser.add_argument("--config", help="Path to JSON roster file")
parser.add_argument("--export-csv", action="store_true", help="Export analysis as CSV")
parser.add_argument("--export-json", action="store_true", help="Export sample roster as JSON template")
args = parser.parse_args()
if args.config:
roster = load_roster_from_json(args.config)
else:
roster = build_sample_roster()
if args.export_json:
data = asdict(roster)
print(json.dumps(data, indent=2))
return
if args.export_csv:
print(export_csv(roster))
return
print_report(roster)
if __name__ == "__main__":
main()
FILE:scripts/hiring_plan_modeler.py
#!/usr/bin/env python3
"""
Hiring Plan Modeler
===================
Builds hiring plans from business goals with cost projections.
Outputs quarterly headcount plan, cost model, and risk assessment.
Usage:
python hiring_plan_modeler.py # Run with built-in sample data
python hiring_plan_modeler.py --config plan.json # Load from JSON config
python hiring_plan_modeler.py --help
"""
import argparse
import json
import sys
from dataclasses import dataclass, field, asdict
from datetime import datetime, date
from typing import Optional
import csv
import io
# ---------------------------------------------------------------------------
# Data structures
# ---------------------------------------------------------------------------
@dataclass
class HireTarget:
"""One planned hire."""
role: str
level: str # L1, L2, L3, L4, M1, M2, M3, VP, C-Suite
function: str # Engineering, Sales, Product, G&A, Marketing, CS
quarter: str # Q1-2025, Q2-2025, etc.
base_salary: int # Annual, USD
bonus_pct: float # % of base (e.g., 0.10 for 10%)
equity_annual_usd: int # Annualized equity value at current 409A
benefits_annual: int # Employer-paid benefits
recruiter_fee_pct: float= 0.20 # Agency fee if used (0 for internal recruiter)
ramp_months: int = 3 # Months to full productivity
priority: str = "High" # High / Medium / Low
business_case: str = ""
open_to_internal: bool = False
@dataclass
class HiringPlan:
company: str
plan_period: str # e.g., "2025 Annual"
current_headcount: int
target_revenue: int # Annual target revenue ($)
current_revenue: int # Current ARR ($)
hires: list[HireTarget] = field(default_factory=list)
# Cost overheads beyond comp
overhead_rate: float = 0.25 # Workspace, software, onboarding overhead as % of base
internal_recruiter_cost: int = 0 # If you have an internal recruiter, annual cost
# ---------------------------------------------------------------------------
# Computation
# ---------------------------------------------------------------------------
def quarter_to_sortkey(q: str) -> tuple[int, int]:
"""Parse 'Q2-2025' → (2025, 2)"""
parts = q.upper().split("-")
if len(parts) == 2:
q_num = int(parts[0].replace("Q", ""))
year = int(parts[1])
return (year, q_num)
return (9999, 9)
def get_quarters(hires: list[HireTarget]) -> list[str]:
"""Return sorted unique quarters from hire list."""
quarters = sorted(set(h.quarter for h in hires), key=quarter_to_sortkey)
return quarters
def compute_hire_costs(hire: HireTarget) -> dict:
"""Compute total first-year cost for one hire."""
total_comp = hire.base_salary + int(hire.base_salary * hire.bonus_pct) + hire.equity_annual_usd + hire.benefits_annual
recruiter_fee = int(hire.base_salary * hire.recruiter_fee_pct)
overhead = int(hire.base_salary * 0.25) # workspace, tools, onboarding
ramp_productivity_cost = int(hire.base_salary * (hire.ramp_months / 12)) # cost during ramp
return {
"base_salary": hire.base_salary,
"target_bonus": int(hire.base_salary * hire.bonus_pct),
"equity_annual": hire.equity_annual_usd,
"benefits": hire.benefits_annual,
"total_comp": total_comp,
"recruiter_fee": recruiter_fee,
"overhead": overhead,
"ramp_cost": ramp_productivity_cost,
"first_year_total": total_comp + recruiter_fee + overhead,
"fully_loaded_first_year": total_comp + recruiter_fee + overhead + ramp_productivity_cost,
}
def summarize_by_quarter(plan: HiringPlan) -> dict[str, dict]:
"""Aggregate headcount and costs per quarter."""
quarters = get_quarters(plan.hires)
summary = {}
running_headcount = plan.current_headcount
for q in quarters:
q_hires = [h for h in plan.hires if h.quarter == q]
q_costs = [compute_hire_costs(h) for h in q_hires]
total_comp = sum(c["total_comp"] for c in q_costs)
total_first_year = sum(c["first_year_total"] for c in q_costs)
recruiter_fees = sum(c["recruiter_fee"] for c in q_costs)
running_headcount += len(q_hires)
summary[q] = {
"new_hires": len(q_hires),
"headcount_eop": running_headcount,
"total_annual_comp_added": total_comp,
"total_first_year_cost": total_first_year,
"recruiter_fees": recruiter_fees,
"hires": q_hires,
"costs": q_costs,
}
return summary
def summarize_by_function(plan: HiringPlan) -> dict[str, dict]:
"""Aggregate headcount and costs per function."""
functions: dict[str, dict] = {}
for hire in plan.hires:
fn = hire.function
if fn not in functions:
functions[fn] = {"count": 0, "total_comp": 0, "total_first_year": 0, "roles": []}
costs = compute_hire_costs(hire)
functions[fn]["count"] += 1
functions[fn]["total_comp"] += costs["total_comp"]
functions[fn]["total_first_year"] += costs["first_year_total"]
functions[fn]["roles"].append(hire.role)
return functions
def compute_totals(plan: HiringPlan) -> dict:
all_costs = [compute_hire_costs(h) for h in plan.hires]
total_hires = len(plan.hires)
total_comp = sum(c["total_comp"] for c in all_costs)
total_first_year = sum(c["first_year_total"] for c in all_costs)
total_fully_loaded = sum(c["fully_loaded_first_year"] for c in all_costs)
total_recruiter = sum(c["recruiter_fee"] for c in all_costs)
final_headcount = plan.current_headcount + total_hires
revenue_per_employee = plan.target_revenue / final_headcount if final_headcount > 0 else 0
revenue_per_employee_current = plan.current_revenue / plan.current_headcount if plan.current_headcount > 0 else 0
return {
"total_hires": total_hires,
"final_headcount": final_headcount,
"headcount_growth_pct": ((final_headcount - plan.current_headcount) / plan.current_headcount * 100) if plan.current_headcount > 0 else 0,
"total_annual_comp_added": total_comp,
"total_first_year_cost": total_first_year,
"total_fully_loaded_first_year": total_fully_loaded,
"total_recruiter_fees": total_recruiter,
"revenue_per_employee_target": revenue_per_employee,
"revenue_per_employee_current": revenue_per_employee_current,
"avg_comp_per_hire": total_comp // total_hires if total_hires > 0 else 0,
}
# ---------------------------------------------------------------------------
# Risk assessment
# ---------------------------------------------------------------------------
def assess_risks(plan: HiringPlan, totals: dict) -> list[dict]:
risks = []
# Headcount growth too fast
growth_pct = totals["headcount_growth_pct"]
if growth_pct > 80:
risks.append({
"severity": "HIGH",
"category": "Execution",
"finding": f"Headcount growing {growth_pct:.0f}% this period. "
"Culture and processes rarely scale this fast without breakage.",
"recommendation": "Stagger Q3/Q4 hires. Validate Q1/Q2 cohort is onboarded before next wave."
})
elif growth_pct > 50:
risks.append({
"severity": "MEDIUM",
"category": "Execution",
"finding": f"Headcount growing {growth_pct:.0f}% — significant scaling challenge.",
"recommendation": "Ensure onboarding infrastructure scales. Assign buddy/mentor to each hire."
})
# High concentration in one quarter
quarters = get_quarters(plan.hires)
q_counts = {q: sum(1 for h in plan.hires if h.quarter == q) for q in quarters}
max_q = max(q_counts.values()) if q_counts else 0
if max_q > len(plan.hires) * 0.5 and max_q > 4:
heavy_q = [q for q, c in q_counts.items() if c == max_q][0]
risks.append({
"severity": "MEDIUM",
"category": "Hiring Execution",
"finding": f"More than 50% of hires planned in {heavy_q} ({max_q} hires). "
"Recruiting capacity and onboarding bandwidth may be insufficient.",
"recommendation": "Spread hires across quarters. Hiring pipeline needs to start 60–90 days before target start date."
})
# Revenue per employee declining
if totals["revenue_per_employee_target"] < totals["revenue_per_employee_current"] * 0.7:
risks.append({
"severity": "HIGH",
"category": "Financial",
"finding": f"Revenue per employee declining from ,.0f to "
f",.0f — a {((totals['revenue_per_employee_target']/totals['revenue_per_employee_current'])-1)*100:.0f}% drop.",
"recommendation": "Validate that revenue model supports this headcount. Is target revenue achievable with this team?"
})
# Low priority hires consuming budget
low_priority_hires = [h for h in plan.hires if h.priority == "Low"]
if low_priority_hires:
lp_cost = sum(compute_hire_costs(h)["first_year_total"] for h in low_priority_hires)
risks.append({
"severity": "MEDIUM",
"category": "Prioritization",
"finding": f"{len(low_priority_hires)} 'Low' priority hires consuming ,.0f in first-year costs.",
"recommendation": "Consider deferring Low priority hires to preserve runway. Cut these first if budget tightens."
})
# Hires without business cases
no_case = [h for h in plan.hires if not h.business_case]
if no_case:
risks.append({
"severity": "MEDIUM",
"category": "Governance",
"finding": f"{len(no_case)} hires have no documented business case: {', '.join(h.role for h in no_case[:5])}{'...' if len(no_case) > 5 else ''}",
"recommendation": "Every hire over $80K should have a written business case. What revenue or risk does this role address?"
})
# High recruiter fee exposure
if totals["total_recruiter_fees"] > 100_000:
risks.append({
"severity": "LOW",
"category": "Cost",
"finding": f",.0f in recruiter fees. "
"Consider whether internal recruiter investment would be cheaper at this hiring volume.",
"recommendation": f"Internal recruiter at $120–150K fully loaded pays off at 3–4 hires/year vs. agency fees."
})
# No risks — that's itself a flag
if not risks:
risks.append({
"severity": "INFO",
"category": "General",
"finding": "No major risks flagged. Plan appears well-structured.",
"recommendation": "Validate assumptions: time-to-fill estimates, revenue model, and Q1 hiring pipeline status."
})
return risks
# ---------------------------------------------------------------------------
# Formatting / Output
# ---------------------------------------------------------------------------
def fmt(n: int) -> str:
return f",.0f"
def pct(n: float) -> str:
return f"{n:.1f}%"
def print_report(plan: HiringPlan):
WIDTH = 72
SEP = "=" * WIDTH
sep = "-" * WIDTH
print(SEP)
print(f" HIRING PLAN: {plan.company}")
print(f" Period: {plan.plan_period} | Generated: {date.today().isoformat()}")
print(SEP)
totals = compute_totals(plan)
q_summary = summarize_by_quarter(plan)
fn_summary = summarize_by_function(plan)
risks = assess_risks(plan, totals)
# Executive summary
print("\n[ EXECUTIVE SUMMARY ]")
print(sep)
print(f" Current headcount: {plan.current_headcount:>5}")
print(f" Planned hires: {totals['total_hires']:>5}")
print(f" Final headcount: {totals['final_headcount']:>5} (+{totals['headcount_growth_pct']:.0f}%)")
print(f" Current ARR: {fmt(plan.current_revenue):>12}")
print(f" Target revenue: {fmt(plan.target_revenue):>12}")
print(f" Revenue/employee now: {fmt(int(totals['revenue_per_employee_current'])):>12}")
print(f" Revenue/employee target: {fmt(int(totals['revenue_per_employee_target'])):>12}")
print()
print(f" Total annual comp added: {fmt(totals['total_annual_comp_added']):>12}")
print(f" Total first-year cost: {fmt(totals['total_first_year_cost']):>12}")
print(f" Fully loaded (w/ ramp): {fmt(totals['total_fully_loaded_first_year']):>12}")
print(f" Recruiter fees: {fmt(totals['total_recruiter_fees']):>12}")
print(f" Avg comp per hire: {fmt(totals['avg_comp_per_hire']):>12}")
# Quarterly breakdown
print(f"\n[ QUARTERLY HEADCOUNT PLAN ]")
print(sep)
print(f" {'Quarter':<10} {'New Hires':>10} {'HC (EOP)':>10} {'Comp Added':>14} {'1yr Cost':>14} {'Recruiter $':>12}")
print(f" {'-'*10} {'-'*10} {'-'*10} {'-'*14} {'-'*14} {'-'*12}")
for q, data in q_summary.items():
print(f" {q:<10} {data['new_hires']:>10} {data['headcount_eop']:>10} "
f"{fmt(data['total_annual_comp_added']):>14} "
f"{fmt(data['total_first_year_cost']):>14} "
f"{fmt(data['recruiter_fees']):>12}")
# By function
print(f"\n[ HEADCOUNT BY FUNCTION ]")
print(sep)
print(f" {'Function':<18} {'Hires':>7} {'Annual Comp':>14} {'1yr Cost':>14}")
print(f" {'-'*18} {'-'*7} {'-'*14} {'-'*14}")
for fn, data in sorted(fn_summary.items(), key=lambda x: -x[1]["count"]):
print(f" {fn:<18} {data['count']:>7} {fmt(data['total_comp']):>14} {fmt(data['total_first_year']):>14}")
# Hire detail
print(f"\n[ HIRE DETAIL ]")
print(sep)
print(f" {'Role':<30} {'Fn':<14} {'Lvl':<6} {'Q':<8} {'Base':>10} {'Total Comp':>12} {'Priority':<8}")
print(f" {'-'*30} {'-'*14} {'-'*6} {'-'*8} {'-'*10} {'-'*12} {'-'*8}")
for h in sorted(plan.hires, key=lambda x: quarter_to_sortkey(x.quarter)):
costs = compute_hire_costs(h)
print(f" {h.role:<30} {h.function:<14} {h.level:<6} {h.quarter:<8} "
f"{fmt(h.base_salary):>10} {fmt(costs['total_comp']):>12} {h.priority:<8}")
if h.business_case:
bc = h.business_case[:60] + "..." if len(h.business_case) > 60 else h.business_case
print(f" {'':>30} ↳ {bc}")
# Risk assessment
print(f"\n[ RISK ASSESSMENT ]")
print(sep)
sev_order = {"HIGH": 0, "MEDIUM": 1, "LOW": 2, "INFO": 3}
for risk in sorted(risks, key=lambda r: sev_order.get(r["severity"], 99)):
sev = risk["severity"]
marker = {"HIGH": "⚠ HIGH", "MEDIUM": "◆ MED ", "LOW": "◇ LOW ", "INFO": "ℹ INFO"}[sev]
print(f"\n [{marker}] {risk['category']}")
# Wrap finding
finding = risk["finding"]
words = finding.split()
line = " Finding: "
for w in words:
if len(line) + len(w) + 1 > WIDTH - 2:
print(line)
line = " " + w + " "
else:
line += w + " "
if line.strip():
print(line)
reco = risk["recommendation"]
words = reco.split()
line = " Action: "
for w in words:
if len(line) + len(w) + 1 > WIDTH - 2:
print(line)
line = " " + w + " "
else:
line += w + " "
if line.strip():
print(line)
print(f"\n{SEP}\n")
def export_csv(plan: HiringPlan) -> str:
"""Return CSV of hire detail."""
output = io.StringIO()
writer = csv.writer(output)
writer.writerow(["Role", "Function", "Level", "Quarter", "Priority",
"Base Salary", "Bonus Target", "Equity Annual", "Benefits",
"Total Comp", "Recruiter Fee", "Overhead", "First Year Total",
"Ramp Months", "Open to Internal", "Business Case"])
for h in plan.hires:
c = compute_hire_costs(h)
writer.writerow([h.role, h.function, h.level, h.quarter, h.priority,
h.base_salary, c["target_bonus"], h.equity_annual_usd, h.benefits_annual,
c["total_comp"], c["recruiter_fee"], c["overhead"], c["first_year_total"],
h.ramp_months, h.open_to_internal, h.business_case])
return output.getvalue()
# ---------------------------------------------------------------------------
# Sample data
# ---------------------------------------------------------------------------
def build_sample_plan() -> HiringPlan:
"""Sample Series A → B hiring plan."""
plan = HiringPlan(
company="AcmeTech (Series A)",
plan_period="2025 Annual",
current_headcount=32,
current_revenue=3_500_000,
target_revenue=8_000_000,
overhead_rate=0.25,
internal_recruiter_cost=140_000,
)
plan.hires = [
# Q1 — Foundation hires
HireTarget(
role="Staff Software Engineer (Backend)",
level="L4", function="Engineering", quarter="Q1-2025",
base_salary=185_000, bonus_pct=0.0, equity_annual_usd=25_000,
benefits_annual=18_000, recruiter_fee_pct=0.0, ramp_months=2,
priority="High", open_to_internal=True,
business_case="Core API team is bottleneck for 3 roadmap items. Staff-level needed to lead architecture."
),
HireTarget(
role="Account Executive (Mid-Market)",
level="L3", function="Sales", quarter="Q1-2025",
base_salary=95_000, bonus_pct=0.50, equity_annual_usd=10_000,
benefits_annual=15_000, recruiter_fee_pct=0.18, ramp_months=4,
priority="High",
business_case="Pipeline coverage at 1.8x quota. Need 2.5x by Q2. AE adds $600K ARR/year at ramp."
),
HireTarget(
role="Product Designer (Senior)",
level="L3", function="Product", quarter="Q1-2025",
base_salary=145_000, bonus_pct=0.0, equity_annual_usd=18_000,
benefits_annual=18_000, recruiter_fee_pct=0.0, ramp_months=2,
priority="High",
business_case="Single designer for 4 squads. UX debt slowing enterprise deals requiring onboarding improvements."
),
# Q2 — Growth hires
HireTarget(
role="Engineering Manager (Frontend)",
level="M1", function="Engineering", quarter="Q2-2025",
base_salary=175_000, bonus_pct=0.10, equity_annual_usd=22_000,
benefits_annual=18_000, recruiter_fee_pct=0.20, ramp_months=3,
priority="High",
business_case="Frontend team at 7 ICs with no dedicated EM. Performance review debt is high; manager needed."
),
HireTarget(
role="Account Executive (Mid-Market)",
level="L2", function="Sales", quarter="Q2-2025",
base_salary=85_000, bonus_pct=0.50, equity_annual_usd=8_000,
benefits_annual=15_000, recruiter_fee_pct=0.18, ramp_months=4,
priority="High",
business_case="Second AE to reach 2.5x pipeline coverage target."
),
HireTarget(
role="Customer Success Manager",
level="L2", function="Customer Success", quarter="Q2-2025",
base_salary=90_000, bonus_pct=0.15, equity_annual_usd=8_000,
benefits_annual=15_000, recruiter_fee_pct=0.0, ramp_months=2,
priority="Medium",
business_case="CSM:account ratio at 1:60, industry standard 1:30. NRR has dipped 4pts in 2 quarters."
),
HireTarget(
role="Data Engineer",
level="L2", function="Engineering", quarter="Q2-2025",
base_salary=155_000, bonus_pct=0.0, equity_annual_usd=18_000,
benefits_annual=18_000, recruiter_fee_pct=0.0, ramp_months=3,
priority="Medium",
business_case="Analytics infrastructure blocking product analytics, customer dashboards, and board metrics."
),
# Q3 — Scale hires
HireTarget(
role="Senior Software Engineer (Backend)",
level="L3", function="Engineering", quarter="Q3-2025",
base_salary=165_000, bonus_pct=0.0, equity_annual_usd=20_000,
benefits_annual=18_000, recruiter_fee_pct=0.0, ramp_months=2,
priority="High",
business_case="Backend team needs capacity to deliver Q3 roadmap without delaying Q4 items."
),
HireTarget(
role="Head of Marketing",
level="M3", function="Marketing", quarter="Q3-2025",
base_salary=180_000, bonus_pct=0.15, equity_annual_usd=30_000,
benefits_annual=18_000, recruiter_fee_pct=0.20, ramp_months=3,
priority="High",
business_case="No marketing function. 100% of pipeline is outbound. Need inbound by Q1-2026 for Series B."
),
HireTarget(
role="People Operations Manager",
level="M1", function="G&A", quarter="Q3-2025",
base_salary=120_000, bonus_pct=0.10, equity_annual_usd=12_000,
benefits_annual=16_000, recruiter_fee_pct=0.0, ramp_months=2,
priority="Medium",
business_case="Founders spending 8hrs/week on HR ops at 40 employees. Unscalable. First dedicated HR hire."
),
# Q4 — Stretch hires (conditional on revenue milestone)
HireTarget(
role="Senior Software Engineer (Frontend)",
level="L3", function="Engineering", quarter="Q4-2025",
base_salary=160_000, bonus_pct=0.0, equity_annual_usd=18_000,
benefits_annual=18_000, recruiter_fee_pct=0.0, ramp_months=2,
priority="Medium",
business_case="Conditional on Q3 ARR exceeding $5.5M. Frontend team capacity planning for 2026 roadmap."
),
HireTarget(
role="Account Executive (Enterprise)",
level="L4", function="Sales", quarter="Q4-2025",
base_salary=120_000, bonus_pct=0.60, equity_annual_usd=15_000,
benefits_annual=15_000, recruiter_fee_pct=0.20, ramp_months=6,
priority="Low",
business_case="Enterprise motion exploratory. Requires ICP validation in Q2-Q3 before committing."
),
HireTarget(
role="DevOps / Platform Engineer",
level="L3", function="Engineering", quarter="Q4-2025",
base_salary=150_000, bonus_pct=0.0, equity_annual_usd=18_000,
benefits_annual=18_000, recruiter_fee_pct=0.0, ramp_months=3,
priority="Low",
business_case="Platform reliability becoming bottleneck. Conditional on uptime SLA breaches continuing in Q3."
),
]
return plan
# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------
def load_plan_from_json(path: str) -> HiringPlan:
with open(path) as f:
data = json.load(f)
hires = [HireTarget(**h) for h in data.pop("hires", [])]
plan = HiringPlan(**data)
plan.hires = hires
return plan
def main():
parser = argparse.ArgumentParser(
description="Hiring Plan Modeler — build headcount plans with cost projections",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
python hiring_plan_modeler.py # Run sample plan
python hiring_plan_modeler.py --config plan.json # Load from JSON
python hiring_plan_modeler.py --export-csv # Output CSV of hires
python hiring_plan_modeler.py --export-json # Output plan as JSON template
"""
)
parser.add_argument("--config", help="Path to JSON plan file")
parser.add_argument("--export-csv", action="store_true", help="Export hire detail as CSV")
parser.add_argument("--export-json", action="store_true", help="Export sample plan as JSON template")
args = parser.parse_args()
if args.config:
plan = load_plan_from_json(args.config)
else:
plan = build_sample_plan()
if args.export_json:
data = asdict(plan)
print(json.dumps(data, indent=2))
return
if args.export_csv:
print(export_csv(plan))
return
print_report(plan)
if __name__ == "__main__":
main()
Tự động đánh giá mã nhiều ngôn ngữ: phân tích PR, độ phức tạp, vi phạm SOLID, mã có mùi và tạo báo cáo.
---
name: "code-reviewer"
description: Code review automation for TypeScript, JavaScript, Python, Go, Swift, Kotlin, C#, .NET, Java, C, C++, Rust, Ruby, PHP, and Dart/Flutter. Analyzes PRs for complexity and risk, checks code quality for SOLID violations and code smells, generates review reports. Use when reviewing pull requests, analyzing code quality, identifying issues, generating review checklists.
---
# Code Reviewer
Automated code review tools for analyzing pull requests, detecting code quality issues, and generating review reports.
---
## How This Skill Is Organized
```
code-reviewer/
SKILL.md ← you are here (tools + dispatch table)
rules/
universal.md ← security, async, resources, exceptions, performance — all languages
languages/
python.md ← Python-specific rules + idioms
typescript.md ← TypeScript / JavaScript-specific rules + idioms
go.md ← Go-specific rules + idioms
swift.md ← Swift-specific rules + idioms
kotlin.md ← Kotlin-specific rules + idioms
csharp.md ← C# / .NET-specific rules + idioms
java.md ← Java-specific rules + idioms
c.md ← C -specific rules + idioms
cpp.md ← C++ -specific rules + idioms
rust.md ← Rust -specific rules + idioms
ruby.md ← Ruby -specific rules + idioms
php.md ← PHP-specific rules + idioms
dart.md ← Dart / Flutter-specific rules + idioms
```
### Loading order for every review
1. This file (`SKILL.md`) — tools and thresholds
2. `rules/universal.md` — always, for every language
3. The matching `languages/*.md` — one file based on the extension table below
That is always exactly **2 additional files**, regardless of scope.
| Extension(s) | Load |
|---|---|
| `.py` | `languages/python.md` |
| `.ts`, `.tsx`, `.js`, `.jsx`, `.mjs` | `languages/typescript.md` |
| `.go` | `languages/go.md` |
| `.swift` | `languages/swift.md` |
| `.kt`, `.kts` | `languages/kotlin.md` |
| `.cs`, `.csx`, `.razor`, `.cshtml` | `languages/csharp.md` |
| `.java` | `languages/java.md` |
| `.c`, `.h` | `languages/c.md` |
| `.cpp`, `.cc`, `.cxx`, `.hpp`, `.hh`, `.hxx` | `languages/cpp.md` |
| `.rs` | `languages/rust.md` |
| `.rb`, `.rake`, `.gemspec`, `.ru` | `languages/ruby.md` |
| `.php`, `.phtml` | `languages/php.md` |
| `.dart` | `languages/dart.md` |
---
## Tools
### PR Analyzer
Analyzes git diff between branches to assess review complexity and identify risks.
```bash
# Analyze current branch against main
python scripts/pr_analyzer.py /path/to/repo
# Compare specific branches
python scripts/pr_analyzer.py . --base main --head feature-branch
# JSON output for integration
python scripts/pr_analyzer.py /path/to/repo --json
```
**What it detects (universal — see also language file for language-specific signals):**
- Hardcoded secrets (passwords, API keys, tokens, connection strings)
- SQL / query injection patterns
- Debug statements left in production code
- Lint / analyzer suppression annotations
- TODO/FIXME comments
**Language-specific detections** are defined in each `languages/*.md` file.
**Output includes:**
- Complexity score (1-10)
- Risk categorization (critical, high, medium, low)
- File prioritization for review order
- Commit message validation
---
### Code Quality Checker
Analyzes source code for structural issues, code smells, and SOLID violations.
```bash
# Analyze a directory
python scripts/code_quality_checker.py /path/to/code
# Analyze specific language
# Valid values: python, typescript, javascript, go, swift, kotlin, csharp, java, c, cpp, rust, ruby, php, dart
python scripts/code_quality_checker.py . --language java
# JSON output
python scripts/code_quality_checker.py /path/to/code --json
```
**Universal thresholds:**
| Issue | Threshold |
|-------|-----------|
| Long function | >50 lines |
| Large file | >500 lines |
| God class | >20 methods |
| Too many params | >5 |
| Deep nesting | >4 levels |
| High complexity | >10 branches |
Language-specific checks are defined in each `languages/*.md` file.
---
### Review Report Generator
Combines PR analysis and code quality findings into structured review reports.
```bash
# Generate report for current repo
python scripts/review_report_generator.py /path/to/repo
# Markdown output
python scripts/review_report_generator.py . --format markdown --output review.md
# Use pre-computed analyses
python scripts/review_report_generator.py . \
--pr-analysis pr_results.json \
--quality-analysis quality_results.json
```
**Verdicts:**
| Score | Verdict |
|-------|---------|
| 90+ with no high issues | Approve |
| 75+ with ≤2 high issues | Approve with suggestions |
| 50-74 | Request changes |
| <50 or critical issues | Block |
---
## Adding a New Language
**Reviewer guidance (required):**
1. Create `languages/<name>.md` using any existing language file as a template — it must have sections: PR Analyzer Signals, Code Quality Checks, Security, Async, Resource Management, Exception Handling, Performance, Idioms.
2. Add the extension row to the dispatch table above.
That is all the agent-driven review needs.
**Deterministic analyzer support (optional, recommended):** the bundled scripts
only flag a language they explicitly know. To make `code_quality_checker.py`
score the new language:
3. Add the extensions to `LANGUAGE_EXTENSIONS` in `scripts/code_quality_checker.py` (this also adds the `--language` choice).
4. Add `function` / `class` / `method` regex entries for the language in the same file; otherwise it falls back to the Python patterns.
5. Optionally add a `check_<name>_specific_smells(...)` detector (see the C#, Java, and C ones) and call it from `analyze_file`.
6. Add `assets/sample_<name>_smells.<ext>` + `_clean` fixtures and commit the expected `--json` output under `expected_outputs/` as a regression guard.
---
## Regression Fixtures
Labelled fixtures live in `assets/` with their committed `--json` output in
`expected_outputs/` (C#, Java, and C). Drift from the committed JSON signals a
behaviour change in the analyzer:
```bash
python scripts/code_quality_checker.py assets/sample_java_smells.java --json \
| diff - expected_outputs/sample_java_smells_quality.json
```
FILE:assets/sample_csharp_clean.cs
// Sample C# file showing the fixed version of sample_csharp_smells.cs.
// Same shape, but every smell has been resolved per the patterns documented
// in rules/universal.md and languages/csharp.md.
//
// Run:
// python scripts/code_quality_checker.py assets/sample_csharp_clean.cs
//
// Expected: no HIGH C#-specific smells flagged.
using System;
using System.Net.Http;
using System.Threading.Tasks;
using System.Data.SqlClient;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
namespace Sample
{
public class DbOptions
{
// FIX: connection string from configuration, never inlined.
public string ConnectionString { get; init; } = "";
}
public class UserService
{
private readonly string _connectionString;
private readonly HttpClient _httpClient;
private readonly ILogger<UserService> _logger;
// FIX: IHttpClientFactory + IOptions, no hardcoded secrets, no `new HttpClient()`.
public UserService(
IHttpClientFactory httpClientFactory,
IOptions<DbOptions> dbOptions,
ILogger<UserService> logger)
{
_httpClient = httpClientFactory.CreateClient("api");
_connectionString = dbOptions.Value.ConnectionString;
_logger = logger;
}
// FIX: async Task (not async void) so callers can await and observe exceptions.
public async Task HandleClickAsync()
{
// FIX: await the Task instead of blocking on it.
var data = await FetchAsync().ConfigureAwait(false);
_logger.LogInformation("Fetched {Length} bytes", data.Length);
}
public async Task<string> FetchAsync()
{
try
{
// FIX: await the async call — Task is no longer discarded.
await FireAndForgetAsync().ConfigureAwait(false);
// FIX: real null check, no `!`.
var user = await GetCurrentUserAsync().ConfigureAwait(false);
if (user is null)
{
throw new InvalidOperationException("No current user");
}
_ = user.Name;
return await _httpClient
.GetStringAsync("https://api.example/data")
.ConfigureAwait(false);
}
catch (HttpRequestException ex)
{
// FIX: catch specific exception, log with context, rethrow.
_logger.LogError(ex, "Upstream fetch failed");
throw;
}
}
// FIX: real type, not `dynamic`.
public User? CurrentUser { get; private set; }
// FIX: `unsafe` removed — none of the logic actually needed pointers.
public int FirstValue(int[] values) => values.Length > 0 ? values[0] : 0;
// FIX: no #pragma / [SuppressMessage] — root cause fixed instead.
public string GetName(int id)
{
// FIX: `using var` disposes connection + command deterministically.
using var conn = new SqlConnection(_connectionString);
// FIX: parameterized query, no string concatenation.
using var cmd = new SqlCommand("SELECT name FROM users WHERE id = @id", conn);
cmd.Parameters.AddWithValue("@id", id);
conn.Open();
return (string)cmd.ExecuteScalar();
}
private Task<User?> GetCurrentUserAsync() => Task.FromResult<User?>(null);
private Task FireAndForgetAsync() => Task.CompletedTask;
}
public record User(int Id, string Name);
}
FILE:assets/sample_csharp_smells.cs
// Sample C# file demonstrating every C#-specific pattern the code-reviewer
// skill detects. Each smell is labelled inline. This file is NOT meant to
// compile cleanly — it is a fixture for code_quality_checker.py and
// pr_analyzer.py.
//
// Run:
// python scripts/code_quality_checker.py assets/sample_csharp_smells.cs
//
// Expected output: see expected_outputs/sample_csharp_smells_quality.json
using System;
using System.Net.Http;
using System.Threading.Tasks;
using System.Data.SqlClient;
using System.Diagnostics.CodeAnalysis;
namespace Sample
{
public class UserService
{
// [hardcoded_secrets] hardcoded connection string with password
public string ConnectionString = "Server=prod;Database=app;Password=hunter2;";
// [csharp_async_void] async void on a non-event-handler signature
public async void HandleClick(object sender, EventArgs e)
{
// [csharp_blocking_async] .Result blocks on Task in a sync context
var data = FetchAsync().Result;
// [console_log] Debug.WriteLine output statement
Debug.WriteLine(data);
}
public async Task<string> FetchAsync()
{
// [csharp_new_httpclient] new HttpClient() in method body
// [csharp_undisposed_idisposable] HttpClient not in `using`
var client = new HttpClient();
try
{
// [csharp_missing_await] FireAndForgetAsync() returns Task, never awaited
FireAndForgetAsync();
// [csharp_null_forgiving] `user!.Name` forces null-forgiving
var name = user!.Name;
return await client.GetStringAsync("https://api.example/data");
}
catch (Exception)
{
// [csharp_swallowed_exception] empty catch (Exception)
}
return null!;
}
// [loose_type] C# `dynamic` overuse
public dynamic Untyped = null;
// [csharp_unsafe_block] `unsafe` modifier on a method
public unsafe void Pointers()
{
int x = 0;
int* p = &x;
}
// [analyzer_disable] #pragma warning disable
#pragma warning disable CS0168
// [analyzer_disable] [SuppressMessage] attribute
[SuppressMessage("Style", "IDE0060")]
public string GetName(SqlConnection conn, int id)
{
// [csharp_undisposed_idisposable] SqlCommand without `using`
// [sql_concatenation] string concatenation builds SQL with user input
var cmd = new SqlCommand("SELECT name FROM users WHERE id = " + id, conn);
return cmd.ExecuteScalar().ToString();
}
}
}
FILE:assets/sample_c_clean.c
/*
* sample_c_clean.c — sample_c_smells.c refactored per
* rules/universal.md + languages/c.md. Same surface area, zero
* detector hits.
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
void safe_input(void) {
char buf[64];
/* fgets is bounds-aware */
if (fgets(buf, sizeof(buf), stdin) == NULL) {
return;
}
char dest[10];
/* strncpy with explicit bound + manual null-terminate */
strncpy(dest, buf, sizeof(dest) - 1);
dest[sizeof(dest) - 1] = '\0';
/* strncat with remaining-space bound */
size_t room = sizeof(dest) - strlen(dest) - 1;
strncat(dest, "world", room);
char msg[100];
/* snprintf is bounds-aware */
snprintf(msg, sizeof(msg), "%s says hello", buf);
/* Format string is a literal; buf is an argument */
printf("%s\n", buf);
char name[32];
/* %s with explicit width prevents overflow */
scanf("%31s", name);
}
void checked_alloc(int n) {
/* malloc result is NULL-checked before any dereference */
char *buf = malloc(n);
if (buf == NULL) {
return;
}
buf[0] = 'x';
buf[1] = 'y';
strncpy(buf, "ok", n - 1);
free(buf);
buf = NULL;
printf("done\n");
}
void run_safe_cmd(void) {
/* system() with a string literal — no command-injection surface */
system("ls -la");
}
int main(int argc, char *argv[]) {
(void)argc;
(void)argv;
safe_input();
checked_alloc(100);
run_safe_cmd();
return 0;
}
FILE:assets/sample_c_smells.c
/*
* sample_c_smells.c — labelled instances of every C-specific pattern
* the code-reviewer skill flags. Every smell is annotated inline with
* its CWE and the rule from languages/c.md.
*
* Refactored counterpart: sample_c_clean.c
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
void unsafe_input(void) {
char buf[64];
/* RULE: banned function gets() — CWE-242, no bounds check */
gets(buf);
char dest[10];
/* RULE: banned function strcpy() — no bounds check */
strcpy(dest, buf);
/* RULE: banned function strcat() — no bounds check */
strcat(dest, "world");
char msg[100];
/* RULE: banned function sprintf() — no bounds check */
sprintf(msg, "%s says hello", buf);
/* RULE: format-string vulnerability — CWE-134, buf controls format */
printf(buf);
char name[32];
/* RULE: unbounded scanf — %s without width, CWE-120 */
scanf("%s", name);
}
void leaky_alloc(int n) {
/* RULE: malloc result not NULL-checked within 5 lines — CWE-690 */
char *buf = malloc(n);
buf[0] = 'x';
buf[1] = 'y';
strcpy(buf, "leak");
/* RULE: free without zeroing pointer — CWE-416 dangling */
free(buf);
printf("done\n");
}
void run_user_cmd(const char *cmd_from_user) {
/* RULE: system() with non-literal argument — CWE-78 command injection */
system(cmd_from_user);
}
int main(int argc, char *argv[]) {
if (argc > 1) {
unsafe_input();
leaky_alloc(100);
run_user_cmd(argv[1]);
}
return 0;
}
FILE:assets/sample_java_clean.java
// Sample Java file showing the fixed version of sample_java_smells.java.
// Same shape, but every smell has been resolved per the patterns documented
// in rules/universal.md and languages/java.md.
//
// Run:
// python scripts/code_quality_checker.py assets/sample_java_clean.java
//
// Expected: no HIGH Java-specific smells flagged.
package sample;
import java.io.FileInputStream;
import java.io.InputStream;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import com.fasterxml.jackson.databind.ObjectMapper;
public class UserService {
// FIX: heavy object shared as a singleton instead of constructed per call.
private static final ObjectMapper MAPPER = new ObjectMapper();
// FIX: connection string injected from configuration, never inlined.
private final String connectionString;
public UserService(String connectionString) {
this.connectionString = connectionString;
}
public String getName(Connection conn, int id) {
// FIX: try-with-resources guarantees the stream and statement close.
try (InputStream config = new FileInputStream("/etc/config");
// FIX: parameterized query, no string concatenation.
PreparedStatement stmt =
conn.prepareStatement("SELECT name FROM users WHERE id = ?")) {
stmt.setInt(1, id);
try (ResultSet rs = stmt.executeQuery()) {
return rs.next() ? rs.getString("name") : null;
}
} catch (Exception e) {
// FIX: rethrow with context instead of swallowing.
throw new IllegalStateException("Failed to load user " + id, e);
}
}
public void process() {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// FIX: restore the interrupt flag so cancellation still propagates.
Thread.currentThread().interrupt();
}
}
}
FILE:assets/sample_java_smells.java
// Sample Java file demonstrating the Java-specific patterns the code-reviewer
// skill detects. Each smell is labelled inline. This file is NOT meant to
// compile cleanly — it is a fixture for code_quality_checker.py and
// pr_analyzer.py.
//
// Run:
// python scripts/code_quality_checker.py assets/sample_java_smells.java
//
// Expected output: see expected_outputs/sample_java_smells_quality.json
package sample;
import java.io.FileInputStream;
import java.sql.Connection;
import java.sql.Statement;
import com.fasterxml.jackson.databind.ObjectMapper;
public class UserService {
// [hardcoded_secrets] hardcoded JDBC URL with password
public String connectionString = "jdbc:postgresql://prod/app?user=app&password=hunter2";
// [analyzer_disable] @SuppressWarnings without justification
@SuppressWarnings("unchecked")
public String getName(Connection conn, int id) throws Exception {
// [java_unclosed_resource] FileInputStream not in try-with-resources
FileInputStream fis = new FileInputStream("/etc/config");
// [java_per_use_heavy_object] new ObjectMapper() constructed per call
ObjectMapper mapper = new ObjectMapper();
try {
Statement stmt = conn.createStatement();
// [sql_concatenation] string concatenation builds SQL with user input
return stmt.executeQuery("SELECT name FROM users WHERE id = " + id).toString();
} catch (Exception e) {
// [java_empty_catch] empty catch swallows the exception
}
return null;
}
public void process() {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// [java_swallowed_interrupt] interrupt flag not restored
// [console_log] printStackTrace used as error handling
e.printStackTrace();
}
}
public void log(String message) {
// [console_log] System.out.println left in production code
System.out.println(message);
}
}
FILE:expected_outputs/sample_csharp_clean_quality.json
{
"file": "/home/user/claude-skills/engineering-team/skills/code-reviewer/assets/sample_csharp_clean.cs",
"language": "csharp",
"metrics": {
"lines": {
"total": 101,
"code": 67,
"blank": 14,
"comment": 20
},
"functions": 13,
"classes": 3,
"avg_complexity": 1.3
},
"quality_score": 98,
"grade": "A",
"smells": [
{
"type": "csharp_unused_using",
"severity": "low",
"message": "'using System;' appears unused",
"location": "System"
},
{
"type": "csharp_unused_using",
"severity": "low",
"message": "'using System.Net.Http;' appears unused",
"location": "System.Net.Http"
},
{
"type": "csharp_unused_using",
"severity": "low",
"message": "'using System.Threading.Tasks;' appears unused",
"location": "System.Threading.Tasks"
},
{
"type": "csharp_unused_using",
"severity": "low",
"message": "'using System.Data.SqlClient;' appears unused",
"location": "System.Data.SqlClient"
},
{
"type": "csharp_unused_using",
"severity": "low",
"message": "'using Microsoft.Extensions.Logging;' appears unused",
"location": "Microsoft.Extensions.Logging"
},
{
"type": "csharp_unused_using",
"severity": "low",
"message": "'using Microsoft.Extensions.Options;' appears unused",
"location": "Microsoft.Extensions.Options"
}
],
"solid_violations": [],
"function_details": [
{
"name": "HttpClient",
"parameters": 0,
"lines": 2,
"complexity": 1
},
{
"name": "UserService",
"parameters": 3,
"lines": 8,
"complexity": 1
},
{
"name": "Task",
"parameters": 1,
"lines": 2,
"complexity": 2
},
{
"name": "HandleClickAsync",
"parameters": 0,
"lines": 8,
"complexity": 1
},
{
"name": "FetchAsync",
"parameters": 0,
"lines": 12,
"complexity": 2
},
{
"name": "InvalidOperationException",
"parameters": 1,
"lines": 21,
"complexity": 3
},
{
"name": "FirstValue",
"parameters": 1,
"lines": 4,
"complexity": 1
},
{
"name": "GetName",
"parameters": 1,
"lines": 4,
"complexity": 1
},
{
"name": "SqlConnection",
"parameters": 1,
"lines": 3,
"complexity": 1
},
{
"name": "SqlCommand",
"parameters": 2,
"lines": 7,
"complexity": 1
}
],
"class_details": [
{
"name": "DbOptions",
"methods": 0,
"lines": 7
},
{
"name": "UserService",
"methods": 8,
"lines": 75
},
{
"name": "User",
"methods": 0,
"lines": 3
}
]
}
FILE:expected_outputs/sample_csharp_smells_quality.json
{
"file": "/home/user/claude-skills/engineering-team/skills/code-reviewer/assets/sample_csharp_smells.cs",
"language": "csharp",
"metrics": {
"lines": {
"total": 79,
"code": 43,
"blank": 11,
"comment": 25
},
"functions": 7,
"classes": 1,
"avg_complexity": 1.3
},
"quality_score": 45,
"grade": "F",
"smells": [
{
"type": "csharp_async_void",
"severity": "high",
"message": "'async void HandleClick' \u2014 only safe for event handlers; prefer 'async Task'",
"location": "HandleClick"
},
{
"type": "csharp_blocking_async",
"severity": "high",
"message": "Blocking call on async operation ('.Result' / '.Wait()' / '.GetAwaiter().GetResult()') \u2014 can deadlock in ASP.NET contexts",
"location": "offset 430"
},
{
"type": "csharp_swallowed_exception",
"severity": "high",
"message": "Empty catch block swallows exceptions silently",
"location": "offset 853"
},
{
"type": "csharp_undisposed_idisposable",
"severity": "medium",
"message": "'HttpClient' looks like IDisposable but is not wrapped in 'using' / 'using var'",
"location": "offset 555"
},
{
"type": "csharp_undisposed_idisposable",
"severity": "medium",
"message": "'SqlCommand' looks like IDisposable but is not wrapped in 'using' / 'using var'",
"location": "offset 1289"
},
{
"type": "csharp_new_httpclient",
"severity": "medium",
"message": "'new HttpClient()' \u2014 prefer IHttpClientFactory or a long-lived static instance to avoid socket exhaustion",
"location": "offset 606"
},
{
"type": "csharp_missing_await",
"severity": "medium",
"message": "Async method called without 'await' \u2014 Task is discarded",
"location": "line 42"
},
{
"type": "csharp_unused_using",
"severity": "low",
"message": "'using System;' appears unused",
"location": "System"
},
{
"type": "csharp_unused_using",
"severity": "low",
"message": "'using System.Net.Http;' appears unused",
"location": "System.Net.Http"
},
{
"type": "csharp_unused_using",
"severity": "low",
"message": "'using System.Threading.Tasks;' appears unused",
"location": "System.Threading.Tasks"
},
{
"type": "csharp_unused_using",
"severity": "low",
"message": "'using System.Data.SqlClient;' appears unused",
"location": "System.Data.SqlClient"
},
{
"type": "csharp_unused_using",
"severity": "low",
"message": "'using System.Diagnostics.CodeAnalysis;' appears unused",
"location": "System.Diagnostics.CodeAnalysis"
}
],
"solid_violations": [],
"function_details": [
{
"name": "HandleClick",
"parameters": 2,
"lines": 9,
"complexity": 1
},
{
"name": "FetchAsync",
"parameters": 0,
"lines": 3,
"complexity": 1
},
{
"name": "HttpClient",
"parameters": 0,
"lines": 3,
"complexity": 1
},
{
"name": "HttpClient",
"parameters": 0,
"lines": 24,
"complexity": 3
},
{
"name": "Pointers",
"parameters": 0,
"lines": 11,
"complexity": 1
},
{
"name": "GetName",
"parameters": 2,
"lines": 5,
"complexity": 1
},
{
"name": "SqlCommand",
"parameters": 2,
"lines": 6,
"complexity": 1
}
],
"class_details": [
{
"name": "UserService",
"methods": 4,
"lines": 61
}
]
}
FILE:expected_outputs/sample_c_clean_quality.json
{
"file": "/home/user/claude-skills/engineering-team/skills/code-reviewer/assets/sample_c_clean.c",
"language": "c",
"metrics": {
"lines": {
"total": 72,
"code": 43,
"blank": 17,
"comment": 12
},
"functions": 4,
"classes": 0,
"avg_complexity": 1.8
},
"quality_score": 100,
"grade": "A",
"smells": [
{
"type": "long_function",
"severity": "medium",
"message": "Function 'safe_input' has 61 lines (max: 50)",
"location": "safe_input"
},
{
"type": "magic_number",
"severity": "low",
"message": "Magic number 100 should be a named constant",
"location": "line 29"
},
{
"type": "magic_number",
"severity": "low",
"message": "Magic number 100 should be a named constant",
"location": "line 68"
}
],
"solid_violations": [],
"function_details": [
{
"name": "safe_input",
"parameters": 1,
"lines": 61,
"complexity": 3
},
{
"name": "checked_alloc",
"parameters": 1,
"lines": 31,
"complexity": 2
},
{
"name": "run_safe_cmd",
"parameters": 1,
"lines": 15,
"complexity": 1
},
{
"name": "main",
"parameters": 2,
"lines": 9,
"complexity": 1
}
],
"class_details": []
}
FILE:expected_outputs/sample_c_smells_quality.json
{
"file": "/home/user/claude-skills/engineering-team/skills/code-reviewer/assets/sample_c_smells.c",
"language": "c",
"metrics": {
"lines": {
"total": 67,
"code": 37,
"blank": 17,
"comment": 13
},
"functions": 4,
"classes": 0,
"avg_complexity": 2.0
},
"quality_score": 4,
"grade": "F",
"smells": [
{
"type": "long_function",
"severity": "medium",
"message": "Function 'unsafe_input' has 54 lines (max: 50)",
"location": "unsafe_input"
},
{
"type": "magic_number",
"severity": "low",
"message": "Magic number 242 should be a named constant",
"location": "line 17"
},
{
"type": "magic_number",
"severity": "low",
"message": "Magic number 100 should be a named constant",
"location": "line 27"
},
{
"type": "magic_number",
"severity": "low",
"message": "Magic number 134 should be a named constant",
"location": "line 31"
},
{
"type": "magic_number",
"severity": "low",
"message": "Magic number 120 should be a named constant",
"location": "line 35"
},
{
"type": "magic_number",
"severity": "low",
"message": "Magic number 690 should be a named constant",
"location": "line 41"
},
{
"type": "magic_number",
"severity": "low",
"message": "Magic number 416 should be a named constant",
"location": "line 47"
},
{
"type": "magic_number",
"severity": "low",
"message": "Magic number 100 should be a named constant",
"location": "line 62"
},
{
"type": "c_banned_gets",
"severity": "high",
"message": "'gets()' is unsafe: no bounds check, removed from C11 (CWE-242)",
"location": "offset 117"
},
{
"type": "c_banned_strcpy",
"severity": "high",
"message": "'strcpy()' is unsafe: no bounds check \u2014 prefer strncpy or strlcpy",
"location": "offset 157"
},
{
"type": "c_banned_strcpy",
"severity": "high",
"message": "'strcpy()' is unsafe: no bounds check \u2014 prefer strncpy or strlcpy",
"location": "offset 447"
},
{
"type": "c_banned_strcat",
"severity": "high",
"message": "'strcat()' is unsafe: no bounds check \u2014 prefer strncat or strlcat",
"location": "offset 186"
},
{
"type": "c_banned_sprintf",
"severity": "high",
"message": "'sprintf()' is unsafe: no bounds check \u2014 prefer snprintf",
"location": "offset 238"
},
{
"type": "c_format_string",
"severity": "high",
"message": "'printf(buf)' uses a non-literal format string \u2014 CWE-134 format string vulnerability",
"location": "offset 284"
},
{
"type": "c_unbounded_scanf",
"severity": "high",
"message": "scanf '%s' without a width specifier \u2014 unbounded read can overflow the destination buffer",
"location": "offset 326"
},
{
"type": "c_malloc_unchecked",
"severity": "medium",
"message": "'buf' from malloc/calloc/realloc is not NULL-checked within 5 lines \u2014 dereferencing NULL is UB (CWE-690)",
"location": "line 36"
},
{
"type": "c_free_without_null",
"severity": "low",
"message": "'free(buf)' not followed by 'buf = NULL;' \u2014 dangling pointer can be reused (CWE-416)",
"location": "line 42"
},
{
"type": "c_system_non_literal",
"severity": "high",
"message": "'system(cmd_from_user)' with a non-literal argument \u2014 command injection (CWE-78); use execve with validated args",
"location": "offset 571"
}
],
"solid_violations": [],
"function_details": [
{
"name": "unsafe_input",
"parameters": 1,
"lines": 54,
"complexity": 2
},
{
"name": "leaky_alloc",
"parameters": 1,
"lines": 28,
"complexity": 2
},
{
"name": "run_user_cmd",
"parameters": 1,
"lines": 15,
"complexity": 2
},
{
"name": "main",
"parameters": 2,
"lines": 9,
"complexity": 2
}
],
"class_details": []
}
FILE:expected_outputs/sample_java_clean_quality.json
{
"file": "/home/user/claude-skills/engineering-team/skills/code-reviewer/assets/sample_java_clean.java",
"language": "java",
"metrics": {
"lines": {
"total": 56,
"code": 33,
"blank": 9,
"comment": 14
},
"functions": 3,
"classes": 1,
"avg_complexity": 2.0
},
"quality_score": 100,
"grade": "A",
"smells": [
{
"type": "magic_number",
"severity": "low",
"message": "Magic number 1000 should be a named constant",
"location": "line 49"
}
],
"solid_violations": [],
"function_details": [
{
"name": "UserService",
"parameters": 1,
"lines": 5,
"complexity": 1
},
{
"name": "getName",
"parameters": 2,
"lines": 17,
"complexity": 3
},
{
"name": "process",
"parameters": 0,
"lines": 10,
"complexity": 2
}
],
"class_details": [
{
"name": "UserService",
"methods": 3,
"lines": 38
}
]
}
FILE:expected_outputs/sample_java_smells_quality.json
{
"file": "/home/user/claude-skills/engineering-team/skills/code-reviewer/assets/sample_java_smells.java",
"language": "java",
"metrics": {
"lines": {
"total": 57,
"code": 29,
"blank": 10,
"comment": 18
},
"functions": 3,
"classes": 1,
"avg_complexity": 2.0
},
"quality_score": 68,
"grade": "D",
"smells": [
{
"type": "magic_number",
"severity": "low",
"message": "Magic number 1000 should be a named constant",
"location": "line 44"
},
{
"type": "java_empty_catch",
"severity": "high",
"message": "Empty catch block swallows exceptions silently",
"location": "offset 684"
},
{
"type": "java_print_stack_trace",
"severity": "medium",
"message": "'printStackTrace()' is not real error handling \u2014 log via a proper logger or rethrow with context",
"location": "offset 913"
},
{
"type": "java_swallowed_interrupt",
"severity": "high",
"message": "InterruptedException caught without 'Thread.currentThread().interrupt()' \u2014 breaks cooperative cancellation",
"location": "offset 841"
},
{
"type": "java_unclosed_resource",
"severity": "medium",
"message": "'FileInputStream' looks like an AutoCloseable but is not in a try-with-resources statement",
"location": "offset 366"
},
{
"type": "java_per_use_heavy_object",
"severity": "medium",
"message": "'new ObjectMapper()' is expensive \u2014 share a singleton instance instead of constructing per call",
"location": "offset 451"
}
],
"solid_violations": [],
"function_details": [
{
"name": "getName",
"parameters": 2,
"lines": 18,
"complexity": 3
},
{
"name": "process",
"parameters": 0,
"lines": 11,
"complexity": 2
},
{
"name": "log",
"parameters": 1,
"lines": 6,
"complexity": 1
}
],
"class_details": [
{
"name": "UserService",
"methods": 3,
"lines": 40
}
]
}
FILE:languages/c.md
---
language: c
extensions: [".c", ".h"]
---
# C — Language-Specific Review Notes
Load this file alongside `rules/universal.md`. Universal rules are not repeated here — only C-specific rules and idioms.
---
## PR Analyzer — C Risk Signals
- `printf` / debug `fprintf(stderr, ...)` statements left in production code
- `// TODO` / `// FIXME` comments near memory management code — high risk
- Disabled compiler warnings (`#pragma GCC diagnostic ignore`, `-w` flags in Makefile)
- Hardcoded credentials or keys in source
- Use of banned functions: `gets`, `strcpy`, `strcat`, `sprintf`, `scanf` without width limits
---
## Code Quality — C Checks
- Functions longer than 50 lines — C functions tend to grow organically and become hard to reason about
- Missing `NULL` check after `malloc` / `calloc` / `realloc`
- Return value of functions ignored without explicit `(void)` cast
- Global mutable state used across translation units without clear ownership
- Magic numbers without `#define` or `const` — especially sizes and offsets
- Mixed `malloc`/`free` ownership — unclear which caller is responsible for freeing
---
## Security
- Flag `gets()` — no bounds checking, always a buffer overflow; replace with `fgets()`
- Flag `strcpy()` / `strcat()` — use `strncpy()` / `strncat()` with explicit size, or `strlcpy()` / `strlcat()`
- Flag `sprintf()` — use `snprintf()` with explicit buffer size
- Flag `scanf("%s", buf)` without a width specifier — unbounded read
- Flag `strlen()` result used as a signed integer — potential truncation on 64-bit
- Flag user-controlled data used as a format string (`printf(user_input)`) — format string attack
- Flag integer arithmetic used as array index without bounds check
- Flag signed integer overflow — undefined behavior in C
---
## Async / Concurrency
- Flag shared global or `static` variables accessed from multiple threads without a mutex or `_Atomic`
- Flag `pthread_mutex_t` / `sem_t` not initialized before use
- Flag signal handlers that call non-async-signal-safe functions (`malloc`, `printf`, etc.)
- Flag `volatile` used as a substitute for proper synchronization — it is not sufficient
- Flag lock acquisition order inconsistency across call sites — deadlock risk
---
## Resource Management
- Flag every `malloc` / `calloc` / `realloc` path — verify a matching `free` exists on all exit paths
- Flag `fopen` without a matching `fclose` on all paths including error paths
- Flag `dup` / `socket` / `open` file descriptors not closed on all paths
- Flag stack-allocated VLAs (variable-length arrays) of unbounded size — stack overflow risk
- Flag `realloc` return value assigned directly to the source pointer — leaks on failure
---
## Exception Handling
- Flag ignored return values from `malloc`, `fopen`, `read`, `write`, `close` — all can fail
- Flag `errno` checked after a function that doesn't set it, or not checked immediately after one that does
- Flag `perror` / `strerror` as the sole error handling in library code — propagate errors to callers
- Flag functions that return `-1` on error without documenting which `errno` values are possible
- Flag `assert()` used for runtime error handling — disabled by `NDEBUG` in production builds
---
## Performance
- Flag `strlen()` called repeatedly on the same string in a loop — cache the result
- Flag unnecessary copies of large structs passed by value — pass by pointer
- Flag `memcpy` / `memset` on overlapping regions — use `memmove` for overlapping
- Flag repeated heap allocations in a tight loop — consider a pool or stack allocation
- Flag `volatile` on variables not accessed by hardware or signal handlers — prevents optimization
---
## Idioms and Best Practices
### Memory Safety
- Every pointer must have a clear owner responsible for freeing it — document ownership in comments
- Set pointers to `NULL` immediately after `free` to catch use-after-free early
- Prefer `calloc` over `malloc` + `memset` for zero-initialized allocations
- Use `const` on pointer parameters that the function does not modify
### Defensive Coding
- Always check `NULL` returns from allocation functions
- Use `size_t` for sizes and counts — never `int`
- Prefer `snprintf` and `fgets` over any unbounded string function
- Compile with `-Wall -Wextra -Werror` and treat warnings as errors
### Portability
- Do not assume pointer size equals `int` size — use `intptr_t` / `uintptr_t`
- Do not rely on undefined behavior for performance — use compiler intrinsics instead
- Use `stdint.h` types (`uint32_t`, `int64_t`) for fixed-width requirements
FILE:languages/cpp.md
---
language: cpp
extensions: [".cpp", ".cc", ".cxx", ".hpp", ".hh", ".hxx"]
---
# C++ — Language-Specific Review Notes
Load this file alongside `rules/universal.md`. Universal rules are not repeated here — only C++-specific rules and idioms.
---
## PR Analyzer — C++ Risk Signals
- Raw `new` / `delete` outside of smart pointer wrappers
- `reinterpret_cast` — almost always a red flag; require justification
- Disabled compiler warnings (`#pragma warning(disable:...)`, `-w`)
- `// TODO` / `// FIXME` near ownership or lifetime code
- Hardcoded credentials or keys in source
- Use of deprecated C-style functions: `strcpy`, `sprintf`, `gets`
---
## Code Quality — C++ Checks
- Raw owning pointers (`T*`) used where `unique_ptr` / `shared_ptr` would express ownership
- `shared_ptr` overused where `unique_ptr` suffices — implies shared ownership unnecessarily
- `std::endl` used in hot paths — flushes the buffer every call; prefer `'\n'`
- Implicit conversions between signed and unsigned integers
- Virtual destructor missing on base classes with virtual methods
- `catch (...)` swallowing all exceptions without logging or re-throwing
---
## Security
- Flag `reinterpret_cast` on user-controlled data — potential type confusion
- Flag raw array indexing without bounds check — use `.at()` or assert bounds
- Flag `std::string` data passed to C APIs without null-termination guarantee — use `.c_str()`
- Flag hardcoded buffer sizes — derive from `sizeof` or use `std::array<T, N>`
- Flag `sscanf` / `sprintf` — use `std::istringstream` or `std::format` (C++20)
- Flag user-controlled data used as a format string
---
## Async / Concurrency
- Flag `std::shared_ptr` accessed from multiple threads — the pointer itself is not thread-safe for write; use `std::atomic<std::shared_ptr<T>>` (C++20) or external locking
- Flag `std::vector` / `std::map` mutated from multiple threads without a mutex
- Flag `std::mutex` locked twice in the same thread without `std::recursive_mutex` — deadlock
- Flag detached threads (`std::thread::detach`) with no lifetime coordination
- Flag `volatile` used instead of `std::atomic` for inter-thread communication
---
## Resource Management
- Flag raw `new` returning an owning pointer — wrap immediately in `std::make_unique` or `std::make_shared`
- Flag `delete` called manually outside of a destructor or smart pointer — ownership confusion
- Flag RAII violations — resources acquired in constructor but not released via destructor
- Flag `std::ifstream` / `std::ofstream` not checked for open failure before use
- Flag exceptions thrown from destructors — causes `std::terminate` if thrown during stack unwinding
---
## Exception Handling
- Flag `catch (...)` that swallows exceptions without logging or re-throwing
- Flag exceptions thrown from destructors — wrap in `try/catch` inside the destructor
- Flag `noexcept` on functions that can actually throw — causes `std::terminate`
- Flag exception specifications (`throw(...)`) — deprecated since C++11, removed in C++17
- Flag using exceptions for control flow in performance-critical paths
---
## Performance
- Flag pass-by-value for non-trivial types where pass-by-const-reference suffices
- Flag `std::vector::push_back` in a loop without `reserve` when size is known — repeated reallocations
- Flag `std::map` used where `std::unordered_map` would give O(1) lookup
- Flag `std::endl` in loops — prefer `'\n'` to avoid repeated buffer flushes
- Flag unnecessary copies from missing `std::move` on local temporaries being returned or passed
---
## Idioms and Best Practices
### Ownership and Lifetime
- Prefer `std::unique_ptr` for sole ownership, `std::shared_ptr` only for shared ownership
- Prefer `std::make_unique` / `std::make_shared` over `new` — exception-safe
- Use `std::weak_ptr` to break `shared_ptr` cycles
- Never use raw owning pointers in new code — they are for non-owning observation only
### Modern C++ (17/20)
- Prefer `std::optional<T>` over sentinel values or nullable pointers for optional returns
- Prefer `std::variant` over tagged unions
- Prefer `std::string_view` over `const std::string&` for read-only string parameters
- Prefer range-based `for` loops over index loops where the index isn't needed
- Prefer `if constexpr` over `#ifdef` for compile-time branching
### Type Safety
- Prefer `static_cast` over C-style casts — explicit and auditable
- Avoid `reinterpret_cast` except in low-level I/O or FFI code with a comment
- Use `enum class` over plain `enum` to avoid implicit integer conversions
FILE:languages/csharp.md
---
language: csharp
extensions: [".cs", ".csx", ".razor", ".cshtml"]
---
# C# / .NET — Language-Specific Review Notes
Load this file alongside `rules/universal.md`. Universal rules are not repeated here — only C#-specific rules and idioms.
---
## PR Analyzer — C# Risk Signals
- `#pragma warning disable` and `[SuppressMessage]` — verify they are justified
- `unsafe { }` blocks — require explicit sign-off
- Null-forgiving operator (`!`) used broadly without justification
- `dynamic` used outside of interop scenarios
- Hardcoded connection strings in source files
---
## Code Quality — C# Checks
- `async void` methods (except event handlers)
- `Task` returned but not awaited
- `IDisposable` objects not in `using` / `using var`
- Bare `catch { }` or `catch (Exception e) { }` swallowing silently
- Nullable reference types feature disabled at project level
---
## Security
- Flag raw string interpolation in SQL queries — require parameterized queries (`SqlCommand`) or EF Core
- Flag missing `[ValidateAntiForgeryToken]` on state-changing controller actions
- Flag user-controlled data passed to `Process.Start()` or `File` APIs without validation
- Flag hardcoded connection strings — require `appsettings.json` + secrets management
- Flag `[AllowAnonymous]` on endpoints that should be protected
---
## Async / Await
- Flag `async void` methods outside of event handlers — cannot be awaited and swallow exceptions
- Flag `.Result`, `.Wait()`, or `.GetAwaiter().GetResult()` on `Task` — causes deadlocks in ASP.NET contexts
- Flag missing `ConfigureAwait(false)` in library (non-application) code
- Flag `Task.Run()` wrapping synchronous code inside ASP.NET request handlers unnecessarily
- Flag `CancellationToken` not threaded through to downstream async calls
---
## Resource Management
- Flag `IDisposable` objects (`SqlConnection`, `HttpClient`, `FileStream`, etc.) not wrapped in `using` / `using var`
- Flag `HttpClient` instantiated with `new` inside a method — use `IHttpClientFactory` or a shared static instance to avoid socket exhaustion
- Flag `DbContext` registered as a singleton in DI — it must be scoped
- Flag `MemoryStream` / `MemoryCache` growing unboundedly without eviction policy
---
## Exception Handling
- Flag `catch { }` or `catch (Exception) { }` with no logging or re-throw — silent swallow
- Flag `catch (Exception e) { throw e; }` — resets the stack trace; use `throw;` instead
- Flag catching `Exception` when a specific type (`IOException`, `HttpRequestException`) is appropriate
- Flag exception filters (`when`) used for side effects that suppress the exception
- Flag exceptions used for control flow in hot paths — use `Try*` pattern methods instead
---
## Performance
- Flag `.ToList()` / `.ToArray()` on `IQueryable` before filtering — forces all rows into memory; filter server-side first
- Flag `string` concatenation in loops — use `StringBuilder`
- Flag `Enumerable.Count()` on `IQueryable` when only an existence check is needed — use `Any()`
- Flag `await` in a loop where `Task.WhenAll()` would parallelize the work
- Flag synchronous file or network I/O in an `async` method — use the async overload
---
## Idioms and Best Practices
### Null Safety
- Ensure `<Nullable>enable</Nullable>` is set in the project file
- Flag excessive use of `!` (null-forgiving) without a comment explaining why
- Prefer `is null` / `is not null` over `== null` for null checks
### LINQ
- Flag `First()` where `FirstOrDefault()` is safer
- Flag complex LINQ chains that would be clearer as explicit loops
### Modern C# (10+)
- Prefer `record` types for immutable data carriers
- Prefer `switch` expressions over `switch` statements where a value is returned
- Prefer primary constructors (C# 12) for simple dependency injection
- Prefer file-scoped namespaces (`namespace Foo;`) over block-scoped
- Prefer `is` pattern matching over explicit casts
FILE:languages/dart.md
---
language: dart
extensions: [".dart"]
---
# Dart / Flutter — Language-Specific Review Notes
Load this file alongside `rules/universal.md`. Universal rules are not repeated here — only Dart and Flutter-specific rules and idioms.
---
## PR Analyzer — Dart / Flutter Risk Signals
- `print()` statements left in production code — use a logging package
- `// ignore:` lint suppression comments — verify they are justified
- `!` null assertion operator used broadly without justification
- Hardcoded API keys, tokens, or URLs in Dart source — use environment variables or a secrets package
- `TODO` / `FIXME` near widget lifecycle or state management code
---
## Code Quality — Dart Checks
- `dynamic` used where a concrete type is known — defeats static analysis
- `!` (null assertion) used broadly — prefer null-safe patterns
- `StatefulWidget` used where `StatelessWidget` suffices — prefer stateless
- `setState` called with heavy computation inside — offload before calling
- `BuildContext` used across async gaps without checking `mounted`
- Missing `const` constructor on widgets that could be constant
---
## Security
- Flag API keys or secrets hardcoded in Dart source or `pubspec.yaml` — use `--dart-define` or a secrets manager
- Flag `http` package used without certificate validation disabled intentionally
- Flag `SharedPreferences` used to store sensitive data — use `flutter_secure_storage`
- Flag user-controlled input used in `dart:io` file path operations without sanitization
- Flag `WebView` loading arbitrary user-supplied URLs without validation
- Flag deep link / URL scheme handlers that don't validate the incoming URL before acting on it
---
## Async / Concurrency
- Flag `BuildContext` used after an `await` without checking `if (!mounted) return` — context may be invalid
- Flag `Future` returned but not `await`-ed and without `.catchError()` or `unawaited()` — floating future
- Flag `Isolate.spawn` without a clear message-passing protocol
- Flag heavy computation on the main isolate — offload with `compute()` or `Isolate.run()`
- Flag `StreamController` not closed when the owning widget is disposed — memory leak
- Flag `async*` / `yield*` generators with no error handling on the stream consumer side
---
## Resource Management
- Flag `StreamController` not closed in `dispose()`
- Flag `AnimationController` not disposed in `dispose()`
- Flag `TextEditingController` / `FocusNode` / `ScrollController` not disposed in `dispose()`
- Flag `Timer` not cancelled in `dispose()`
- Flag listeners added to `ChangeNotifier` / `ValueNotifier` without a corresponding `removeListener`
---
## Exception Handling
- Flag empty `catch` blocks — swallowed errors
- Flag `catchError` with no handler body — silent failure
- Flag `Future.error` not surfaced to the UI — show an error state
- Flag `FlutterError.onError` overridden without calling the original handler
- Prefer typed `on ExceptionType catch (e)` over generic `catch (e)` where the exception type is known
---
## Performance
- Flag `setState` called for changes that only affect a small subtree — use `ValueNotifier` / `provider` / `Riverpod` to scope rebuilds
- Flag expensive computation inside `build()` — move to `initState`, a controller, or a `FutureBuilder`
- Flag `ListView` without `ListView.builder` for long or infinite lists — builds all children at once
- Flag missing `const` on widgets that never change — prevents unnecessary rebuilds
- Flag `Image.network` without a caching package in a list — re-downloads on every scroll
- Flag `RepaintBoundary` missing around frequently-repainted widgets (animations, counters)
---
## Idioms and Best Practices
### Null Safety
- Prefer `?.` safe navigation and `??` null coalescing over `!` assertions
- Use `late` only when initialization is guaranteed before first access — document why
- Prefer early returns over deeply nested null checks
### Flutter Widget Patterns
- Prefer `StatelessWidget` + external state management over `StatefulWidget` for business logic
- Keep `build()` methods pure — no side effects, no heavy computation
- Extract repeated widget subtrees into named widget classes, not just methods, for better rebuild granularity
- Use `const` constructors wherever possible — compile-time constant widgets skip rebuilds entirely
### State Management
- Do not mix multiple state management approaches in the same feature
- Flag business logic inside `build()` — it belongs in a ViewModel, Notifier, or BLoC
- Prefer `Riverpod` / `provider` / `BLoC` over raw `setState` for anything beyond local UI state
### Modern Dart (3.x)
- Prefer `sealed` classes for exhaustive pattern matching on domain types
- Use records (`(int, String)`) for lightweight multi-value returns instead of ad hoc classes
- Use `switch` expressions with pattern matching instead of long `if/else` chains
- Prefer `final` for local variables — immutability by default
FILE:languages/go.md
---
language: go
extensions: [".go"]
---
# Go — Language-Specific Review Notes
Load this file alongside `rules/universal.md`. Universal rules are not repeated here — only Go-specific rules and idioms.
---
## PR Analyzer — Go Risk Signals
- `fmt.Println` / `log.Println` debug statements left in production code
- `//nolint` comments — verify they are justified
- `unsafe` package imports — require explicit sign-off
- Hardcoded credentials or tokens in source
---
## Code Quality — Go Checks
- Errors returned but not checked (`_ = someFunc()`)
- `panic()` used outside of package initialization
- Goroutines started without a clear lifetime or cancellation path
- `interface{}` / `any` used where a concrete type or typed interface would work
- Missing context propagation (`context.Context` not threaded through call chains)
---
## Security
- Flag `database/sql` queries built with `fmt.Sprintf` — require `?` / `$N` placeholders
- Flag `os/exec` calls with user-controlled arguments without sanitization
- Flag `html/template` bypassed in favor of `text/template` for HTML output
- Flag `http.ListenAndServeTLS` with `InsecureSkipVerify: true`
---
## Async / Concurrency
- Flag goroutines started with no clear lifetime or cancellation path — always pass `context.Context`
- Flag goroutines that write to a channel with no receiver and no `select` default — causes a leak
- Flag `time.Sleep()` used inside a goroutine as a synchronization mechanism
- Flag `sync.WaitGroup.Add()` called inside the goroutine it tracks — race condition
- Flag `sync.Mutex` copied by value — must always be used as a pointer or embedded in a struct
---
## Resource Management
- Flag `http.Response.Body` not closed after reading — even on error paths (`defer resp.Body.Close()`)
- Flag `os.File` not closed — use `defer f.Close()` immediately after opening
- Flag `rows.Close()` missing after `sql.Query()` — leaks the DB connection
- Flag `context.WithCancel` / `context.WithTimeout` cancel function not called — context and resources leak
---
## Exception Handling
- Flag errors assigned to `_` without a comment explaining why it is safe to ignore
- Flag errors not wrapped with `fmt.Errorf("...: %w", err)` — loses stack context
- Flag `errors.New` / `fmt.Errorf` strings starting with a capital letter or ending in punctuation — violates Go conventions
- Flag `panic()` used for expected runtime errors — reserve for programming errors and unrecoverable states
- Flag `recover()` used to silently swallow panics without logging
---
## Performance
- Flag `fmt.Sprintf` used for simple string concatenation — use `strings.Builder` or `+` for small cases
- Flag `append()` in a tight loop without pre-allocating slice capacity — use `make([]T, 0, n)`
- Flag `json.Marshal` / `json.Unmarshal` on large structs in hot paths — consider `json.Encoder` / streaming
- Flag goroutines spawned per-request without a worker pool for CPU-bound tasks
---
## Idioms and Best Practices
### Error Handling
- All returned errors must be checked — never assign to `_` without a comment
- Prefer wrapping with `fmt.Errorf("...: %w", err)` for stack context
- Use `errors.Is` / `errors.As` for error inspection — never string comparison
### Concurrency
- Every goroutine must have an owner responsible for its lifetime
- Always pass `context.Context` as the first argument to functions that do I/O or block
- Prefer `sync.WaitGroup` or `errgroup` over ad-hoc channel coordination
### Modern Go (1.18+)
- Prefer generics over `interface{}` for container types and utility functions
- Use `any` (alias for `interface{}`) in new code for readability
FILE:languages/java.md
---
language: java
extensions: [".java"]
---
# Java — Language-Specific Review Notes
Load this file alongside `rules/universal.md`. Universal rules are not repeated here — only Java-specific rules and idioms.
---
## PR Analyzer — Java Risk Signals
- `System.out.println` / `e.printStackTrace()` left in production code
- `@SuppressWarnings` annotations — verify they are justified
- Hardcoded JDBC URLs or credentials in source
- Raw type usage (`List`, `Map` without generics)
---
## Code Quality — Java Checks
- Empty `catch` blocks swallowing exceptions silently
- Checked exceptions caught and not re-thrown with context
- `Closeable` / `AutoCloseable` resources not in try-with-resources
- Raw type usage — defeats generics type safety
- Missing `@Override` on overriding methods
- `InterruptedException` caught without calling `Thread.currentThread().interrupt()`
---
## Security
- Flag JPQL / HQL or native SQL string concatenation — require named parameters or `CriteriaBuilder`
- Flag `@RequestMapping` without explicit HTTP method restriction on state-changing endpoints
- Flag user-controlled input passed to `Runtime.exec()` or `ProcessBuilder` without validation
- Flag `ObjectInputStream.readObject()` on untrusted data — unsafe deserialization
- Flag hardcoded JDBC URLs or credentials — require environment variables or a vault
---
## Async / Concurrency
- Flag `ExecutorService.submit()` return value ignored — exceptions are swallowed
- Flag `Thread.sleep()` used as a synchronization mechanism — use `CountDownLatch`, `CompletableFuture`, or `await()`
- Flag `CompletableFuture` chains with no `.exceptionally()` or `.handle()` terminal handler
- Flag `InterruptedException` caught without calling `Thread.currentThread().interrupt()`
- Flag `synchronized` on a non-final field — the lock object can be replaced
- Flag `HashMap` used in multi-threaded context — use `ConcurrentHashMap`
---
## Resource Management
- Flag `InputStream`, `OutputStream`, `Connection`, `ResultSet`, `PreparedStatement` not wrapped in try-with-resources
- Flag manual `finally { resource.close() }` — replace with try-with-resources
- Flag `HttpURLConnection` not disconnected after use
- Flag JDBC `Connection` obtained from a pool and not returned (missing `close()`) on all paths
- Flag `static` `HttpClient` or `Connection` fields shared across threads without connection pool management
---
## Exception Handling
- Flag empty `catch` blocks — `catch (Exception e) {}`
- Flag `InterruptedException` caught without `Thread.currentThread().interrupt()` — breaks cooperative cancellation
- Flag checked exceptions swallowed in a `catch` and not re-thrown or logged with context
- Flag `throw new RuntimeException(e)` without a descriptive message — loses context
- Flag `printStackTrace()` as the sole error handling — use a proper logger
---
## Performance
- Flag `String` concatenation in loops — use `StringBuilder`
- Flag `List.contains()` / `Map.get()` in a loop on large collections — review data structure choice
- Flag N+1 JPA / Hibernate queries — use `JOIN FETCH` or `@BatchSize`
- Flag `new ObjectMapper()` / `new Gson()` instantiated per-request — share a singleton
- Flag `ResultSet` fully iterated when only the first result is needed — use `LIMIT 1` in the query
---
## Idioms and Best Practices
### Null Safety
- Prefer returning `Optional<T>` over `null` from methods
- Flag unchecked dereferences without a prior null guard
- Do not catch `NullPointerException` — fix the root cause instead
### Collections and Streams
- Flag `==` used to compare `String` or boxed types — use `.equals()`
- Flag `.collect(Collectors.toList())` where `.toList()` (Java 16+) suffices
- Flag premature `.stream().collect()` round-trips that could be a single-pass operation
### Generics
- Flag raw types in any new code — always parameterize (`List<String>`, not `List`)
- Flag unchecked cast warnings suppressed without explanation
### Modern Java (11+)
- Prefer `var` for local variables where the type is obvious from the right-hand side
- Prefer records for pure data carriers over manual POJOs with getters/setters
- Prefer `instanceof` pattern matching (`if (obj instanceof String s)`) over explicit casts
- Prefer `switch` expressions over `switch` statements where a value is returned
FILE:languages/kotlin.md
---
language: kotlin
extensions: [".kt", ".kts"]
---
# Kotlin — Language-Specific Review Notes
Load this file alongside `rules/universal.md`. Universal rules are not repeated here — only Kotlin-specific rules and idioms.
---
## PR Analyzer — Kotlin Risk Signals
- `println()` statements left in production code
- `@Suppress` annotations — verify they are justified
- `!!` (not-null assertion) used broadly without justification
- Hardcoded credentials or API keys in source
---
## Code Quality — Kotlin Checks
- `!!` used broadly — prefer `?.let`, `?:`, or `requireNotNull()`
- `lateinit var` accessed before initialization
- Coroutines launched with `GlobalScope` — prefer scoped coroutines
- `runBlocking` used outside of tests or top-level entry points
---
## Security
- Flag Room / SQLite queries built with string concatenation — require parameterized queries
- Flag `WebView.loadUrl()` with user-controlled input without validation
- Flag credentials stored in `SharedPreferences` — require `EncryptedSharedPreferences` or Keychain
---
## Async / Coroutines
- Flag `GlobalScope.launch` / `GlobalScope.async` in production code — use a structured scope
- Flag `runBlocking` outside of tests or top-level main functions
- Flag `launch` / `async` without a `CoroutineExceptionHandler` or `supervisorScope` where individual failures should not cancel siblings
- Flag `Dispatchers.Main` used for CPU-bound work — use `Dispatchers.Default`
- Flag coroutine cancellation not respected — long loops should check `isActive` or call `yield()`
---
## Resource Management
- Flag `Closeable` / `AutoCloseable` not wrapped in `.use { }` (Kotlin's try-with-resources equivalent)
- Flag `OkHttpClient` / `Retrofit` instantiated per-request — share a singleton
- Flag `BroadcastReceiver` registered without a corresponding `unregisterReceiver` — memory / battery leak
- Flag coroutines that hold a resource across a `suspend` point without structured cleanup in `finally`
---
## Exception Handling
- Flag `runCatching { }.getOrNull()` used broadly — silently swallows all exceptions
- Flag `catch (e: Exception)` in coroutines without re-throwing `CancellationException` — breaks structured concurrency
- Flag empty `catch` blocks
- Flag `throw RuntimeException(e)` without a descriptive message
- Prefer typed `sealed class` error hierarchies over raw exceptions for domain errors in coroutine flows
---
## Performance
- Flag `buildString` / `StringBuilder` not used for multi-step string construction in loops
- Flag `List` used for frequent `contains` checks — prefer `Set`
- Flag `flow.collect {}` re-subscribing on every recomposition in Jetpack Compose — use `collectAsStateWithLifecycle`
- Flag `Dispatchers.IO` used for CPU-bound work — use `Dispatchers.Default`
- Flag `suspend` functions calling non-suspend blocking APIs directly — wrap with `withContext(Dispatchers.IO)`
---
## Idioms and Best Practices
### Null Safety
- Prefer safe call (`?.`) and Elvis operator (`?:`) over `!!`
- Use `requireNotNull()` / `checkNotNull()` with a descriptive message when null means a programming error
- Prefer `val` over `var` — immutability by default
### Modern Kotlin
- Prefer `data class` for value carriers
- Prefer `sealed class` / `sealed interface` for exhaustive `when` expressions
- Prefer extension functions over utility classes
- Prefer `object` declarations for singletons
FILE:languages/php.md
---
language: php
extensions: [".php", ".phtml", ".php3", ".php4", ".php5", ".phps"]
---
# PHP — Language-Specific Review Notes
Load this file alongside `rules/universal.md`. Universal rules are not repeated here — only PHP-specific rules and idioms.
---
## PR Analyzer — PHP Risk Signals
- `var_dump` / `print_r` / `echo` debug statements left in production code
- `@` error suppression operator — masks real errors; verify it is justified
- `// phpcs:ignore` / `// phpstan-ignore` comments — verify they are justified
- Hardcoded credentials, database passwords, or API keys in source
- `eval()` anywhere — almost always a security issue
- `$_GET` / `$_POST` / `$_REQUEST` / `$_COOKIE` used without sanitization
---
## Code Quality — PHP Checks
- Missing type declarations on function parameters and return types
- `mixed` return type used broadly — tighten to specific types
- Global variables (`global $var`) — pass dependencies explicitly
- Long functions (>50 lines) — PHP functions tend to accumulate logic
- `isset()` / `empty()` used to mask type errors instead of fixing the root cause
- Missing `strict_types=1` declaration at the top of the file
---
## Security
- Flag `$_GET` / `$_POST` / `$_REQUEST` used directly in SQL queries — require PDO prepared statements
- Flag `mysqli_query($conn, "SELECT ... WHERE id = " . $_GET['id'])` — SQL injection
- Flag `echo $_GET['name']` or any unescaped output — XSS; use `htmlspecialchars()` with `ENT_QUOTES`
- Flag `include` / `require` with user-controlled paths — local/remote file inclusion
- Flag `eval()` — remote code execution risk; no legitimate use in application code
- Flag `shell_exec` / `exec` / `system` / `passthru` with user-controlled input — command injection
- Flag `unserialize()` on untrusted data — arbitrary object instantiation and code execution
- Flag `move_uploaded_file` without MIME type validation and extension whitelist — file upload attack
- Flag `header("Location: " . $_GET['url'])` without validation — open redirect
- Flag missing CSRF token validation on state-changing form endpoints
---
## Async / Concurrency
- Flag long-running synchronous operations in a request cycle — offload to a queue (Laravel Queue, RabbitMQ)
- Flag `sleep()` used inside a request handler — blocks the PHP-FPM worker
- Flag shared mutable state in `static` properties accessed across requests in long-running processes (Swoole, RoadRunner)
- Flag missing idempotency in queued jobs — jobs can be retried on failure
---
## Resource Management
- Flag database connections not closed or returned to the pool (`$pdo = null` or `$conn->close()`)
- Flag `fopen` / `fwrite` without a matching `fclose` on all paths
- Flag `curl_init` without `curl_close` — leaks the curl handle
- Flag unbounded file uploads with no size or type restriction
- Flag sessions not explicitly closed (`session_write_close()`) before long operations — session locking blocks other requests
---
## Exception Handling
- Flag empty `catch` blocks — swallowed exceptions
- Flag `catch (Exception $e) {}` without logging — silent failure
- Flag `die()` / `exit()` used for error handling in library code — use exceptions
- Flag `@` operator used to suppress errors from functions that can fail — check return values instead
- Flag `trigger_error` used in new code — prefer exceptions
---
## Performance
- Flag N+1 Eloquent / Doctrine queries — use eager loading (`with()`, `load()`, `join`)
- Flag `count($array)` called repeatedly in a loop condition — cache the result
- Flag `array_push($arr, $val)` — use `$arr[] = $val` which is faster
- Flag `in_array` on large arrays without the strict third argument — use `isset` on a flipped array for O(1) lookup
- Flag `file_get_contents` on remote URLs in a request cycle — use an HTTP client with timeout and async where possible
- Flag Eloquent `all()` without pagination — loads entire table into memory
---
## Idioms and Best Practices
### Type Safety
- Always declare `declare(strict_types=1)` at the top of every file
- Use union types (`int|string`) and nullable types (`?string`) rather than `mixed`
- Use typed properties on classes — avoid untyped `public $foo`
- Use constructor promotion for simple value objects
### Modern PHP (8.x)
- Prefer `match` expressions over `switch` — strict comparison, no fall-through
- Use named arguments for functions with many optional parameters
- Use `enum` for fixed sets of values instead of class constants
- Use `readonly` properties for immutable data
- Use nullsafe operator (`?->`) instead of nested `isset` checks
- Use `first-class callable syntax` (`strlen(...)`) instead of string references
### Laravel / Symfony Specific
- Keep controllers thin — logic belongs in service classes or action classes
- Use form requests for validation — never validate in the controller directly
- Prefer Eloquent relationships over manual joins for readability
- Flag raw queries where the ORM can express the same intent safely
FILE:languages/python.md
---
language: python
extensions: [".py"]
---
# Python — Language-Specific Review Notes
Load this file alongside `rules/universal.md`. Universal rules are not repeated here — only Python-specific rules and idioms.
---
## PR Analyzer — Python Risk Signals
- `print()` statements left in production code
- `# noqa` and `# type: ignore` comments — verify they are justified
- `eval()` / `exec()` with any user-controlled input
- `pickle` used to deserialize untrusted data
- Hardcoded credentials or tokens in source
---
## Code Quality — Python Checks
- Bare `except:` or `except Exception:` swallowing silently
- Mutable default arguments (`def foo(items=[])`) — shared across calls
- `import *` — pollutes namespace and hides dependencies
- Missing type hints on public functions and methods
- `assert` used for runtime validation — stripped by `-O` flag
---
## Security
- Flag `eval()` / `exec()` with any user-controlled input
- Flag `pickle.loads()` on untrusted data — use `json` or `msgpack`
- Flag `subprocess` calls with `shell=True` and user input
- Flag `flask.render_template_string()` with user data (SSTI)
- Flag `SECRET_KEY` / `DEBUG = True` committed to source
---
## Async
- Flag `asyncio.get_event_loop().run_until_complete()` inside an already-running loop
- Flag mixing `threading` and `asyncio` without a clear bridge (`run_in_executor`)
- Flag CPU-bound work inside an `async def` without offloading to `ProcessPoolExecutor`
- Flag `time.sleep()` inside async functions — use `await asyncio.sleep()`
---
## Resource Management
- Flag `open()` not used as a context manager (`with open(...) as f`)
- Flag `requests.Session` created per-request instead of shared/reused
- Flag database connections not closed or returned to a pool on all paths
- Flag large files read entirely into memory with `.read()` — prefer streaming / chunked reads
---
## Exception Handling
- Flag bare `except:` — catches `BaseException` including `KeyboardInterrupt` and `SystemExit`
- Flag `except Exception: pass` — silently swallows errors
- Flag re-raising with `raise e` instead of `raise` — loses the original traceback
- Flag `except` clause too broad when the `try` block covers multiple operations with different failure modes — split them
---
## Performance
- Flag `+` string concatenation in loops — use `"".join()`
- Flag repeated `re.compile()` inside a loop — compile once at module level
- Flag `list.append()` in a loop where a list comprehension would be more efficient
- Flag `in` membership tests on `list` where the collection is large — use `set`
- Flag loading entire large files into memory — prefer streaming or chunked reads
---
## Idioms and Best Practices
### Type Safety
- All public functions and methods should have type annotations
- Prefer `X | None` (Python 3.10+) over `Optional[X]`
- Use `TypedDict` or `dataclass` over plain `dict` for structured data
### Modern Python (3.10+)
- Prefer `match` statements over long `if/elif` chains
- Prefer `dataclass` or `NamedTuple` over plain classes for data carriers
- Prefer `pathlib.Path` over `os.path` for file operations
- Prefer f-strings over `.format()` or `%` formatting
### None Safety
- Prefer explicit `if x is None` over falsy checks when `0` or `""` are valid values
- Flag functions returning `None` implicitly — make it explicit or raise
FILE:languages/ruby.md
---
language: ruby
extensions: [".rb", ".rake", ".gemspec", ".ru"]
---
# Ruby — Language-Specific Review Notes
Load this file alongside `rules/universal.md`. Universal rules are not repeated here — only Ruby-specific rules and idioms.
---
## PR Analyzer — Ruby Risk Signals
- `puts` / `p` / `pp` debug statements left in production code
- `# rubocop:disable` comments — verify they are justified
- `eval` / `instance_eval` / `class_eval` with user-controlled input
- Hardcoded credentials, tokens, or `SECRET_KEY_BASE` in source
- `binding.pry` / `byebug` / `debugger` left in code
---
## Code Quality — Ruby Checks
- Methods longer than 15 lines — Ruby idioms favor very small methods
- Classes with more than 10 public methods — possible god object
- `rescue Exception` — catches `SignalException` and `SystemExit`; use `rescue StandardError` or more specific types
- `method_missing` implemented without `respond_to_missing?`
- Deeply nested blocks (>3 levels) — extract to methods
- String interpolation used where a symbol would suffice (hash keys, etc.)
---
## Security
- Flag `eval` / `instance_eval` with user-controlled strings — remote code execution
- Flag `system()` / `exec()` / backtick calls with user-controlled input — shell injection
- Flag `YAML.load` on untrusted data — use `YAML.safe_load`
- Flag `Marshal.load` on untrusted data — arbitrary code execution
- Flag raw SQL string interpolation in ActiveRecord — use parameterized queries (`where("name = ?", name)`)
- Flag `params` passed directly to `redirect_to` without validation — open redirect
- Flag `render inline:` with user data — XSS via ERB
- Flag missing `strong_parameters` in Rails controllers — mass assignment vulnerability
---
## Async / Concurrency
- Flag shared mutable state accessed from multiple threads without a `Mutex`
- Flag `Thread.new` without storing the thread reference — exceptions are silently swallowed
- Flag `sleep` used as a synchronization mechanism in threaded code
- Flag `@@class_variables` mutated in multi-threaded contexts — not thread-safe
- Flag Sidekiq / ActiveJob workers that are not idempotent — jobs can be retried
---
## Resource Management
- Flag `File.open` without a block form — the block form guarantees `close`
- Flag database connections or HTTP clients not released in `ensure` blocks
- Flag `ActiveRecord` queries inside loops — N+1 pattern; use `includes` / `preload` / `eager_load`
- Flag `ObjectSpace` usage in production — memory and performance impact
---
## Exception Handling
- Flag `rescue Exception` — use `rescue StandardError` or a specific exception class
- Flag empty `rescue` blocks — swallowed errors
- Flag `rescue` used for control flow (e.g. rescuing `ActiveRecord::RecordNotFound` instead of using `find_by`)
- Flag re-raising with `raise e` instead of bare `raise` — loses the original backtrace
- Flag `ensure` blocks that can raise — masks the original exception
---
## Performance
- Flag N+1 ActiveRecord queries — use `includes`, `preload`, or `eager_load`
- Flag `Array#each` with string concatenation — use `map` + `join`
- Flag `select` + `map` that could be a single `filter_map`
- Flag `.count` on an ActiveRecord relation inside a view or loop — triggers a query each time
- Flag `require` inside a method body — constant overhead on every call
- Flag `Hash#merge` in a loop — use `merge!` or `each_with_object`
---
## Idioms and Best Practices
### Ruby Style
- Prefer `map` / `select` / `reject` / `reduce` over manual `each` + accumulator
- Prefer `&method(:name)` over `{ |x| some_method(x) }` for method reference blocks
- Prefer `freeze` on string constants to avoid repeated object allocation
- Use `attr_reader` / `attr_writer` / `attr_accessor` instead of manual getter/setter methods
- Prefer `Symbol#to_proc` (`&:method_name`) for simple single-method blocks
### Rails-Specific
- Keep controllers thin — logic belongs in service objects, models, or concerns
- Use `before_action` for authentication/authorization checks — never inline
- Prefer `find_by` over `where(...).first` — more intent-revealing
- Flag `after_commit` callbacks with side effects that should be in a service object
- Prefer `respond_to` blocks over separate controller actions for format variants
### Modern Ruby (3.x)
- Prefer pattern matching (`case/in`) for complex data destructuring
- Use numbered block parameters (`_1`, `_2`) only for very short, obvious blocks
- Prefer `Data.define` for simple immutable value objects (Ruby 3.2+)
FILE:languages/rust.md
---
language: rust
extensions: [".rs"]
---
# Rust — Language-Specific Review Notes
Load this file alongside `rules/universal.md`. Universal rules are not repeated here — only Rust-specific rules and idioms.
---
## PR Analyzer — Rust Risk Signals
- `unsafe { }` blocks — require explicit justification and sign-off
- `#[allow(...)]` attributes suppressing lints — verify they are justified
- `.unwrap()` / `.expect("")` on `Option` or `Result` outside of tests or prototypes
- Hardcoded credentials or tokens in source
- `TODO` / `FIXME` comments near `unsafe` or ownership code
---
## Code Quality — Rust Checks
- `.unwrap()` used broadly in production code — prefer `?`, `if let`, or `match`
- `clone()` called excessively — may indicate ownership design issues
- `Arc<Mutex<T>>` used where a simpler ownership model would work
- `Box<dyn Trait>` used where generics (`impl Trait`) would avoid heap allocation
- `pub` fields on structs that should enforce invariants — use accessor methods
---
## Security
- Flag `unsafe` blocks accessing raw pointers without clear safety invariant documented in a comment
- Flag `std::mem::transmute` — almost always a logic error or undefined behavior; require strong justification
- Flag `from_utf8_unchecked` on user-controlled data — use `from_utf8` with error handling
- Flag `unwrap()` on user-supplied input parsing — panics are a denial-of-service vector in server code
- Flag hardcoded secrets — use environment variables or a secrets crate
---
## Async / Concurrency
- Flag `std::sync::Mutex` used in async code — use `tokio::sync::Mutex` to avoid blocking the async runtime
- Flag `.await` inside a `std::sync::MutexGuard` scope — holds the lock across an await point, blocking other tasks
- Flag `spawn` without storing the `JoinHandle` — panics in the spawned task are silently ignored
- Flag `Arc<Mutex<T>>` cloned excessively — consider message passing via channels instead
- Flag blocking I/O calls (`std::fs`, `std::net`) inside async functions — use async equivalents
---
## Resource Management
- Flag manual `drop` called explicitly where the natural scope boundary suffices
- Flag `Rc<T>` used in multi-threaded code — use `Arc<T>`; the compiler catches this but flag in review for architecture discussion
- Flag `Vec` or `String` with large pre-allocated capacity never trimmed — call `.shrink_to_fit()` if long-lived
- Flag `impl Drop` that can panic — causes `abort` during stack unwinding
---
## Exception Handling
- Flag `.unwrap()` in production code outside of tests — use `?` to propagate or handle explicitly
- Flag `.expect("todo")` or `.expect("")` — messages must explain the invariant that guarantees safety
- Flag `panic!` used for recoverable errors — use `Result<T, E>`
- Flag `unwrap_or_default()` where the default silently masks a real error
- Prefer typed error enums (`thiserror`) over `Box<dyn Error>` for library crates
- Prefer `anyhow` for application-level error context; `thiserror` for library error types
---
## Performance
- Flag `.clone()` on large types in hot paths — review whether a reference or `Cow<T>` would work
- Flag `format!` used only to create a `String` from a literal — use `.to_string()` or `String::from`
- Flag `collect::<Vec<_>>()` followed immediately by `.iter()` — chain iterators instead
- Flag `Box<T>` for small types where stack allocation is fine
- Flag `Mutex` contention on a hot path — consider `RwLock` for read-heavy workloads or sharding
---
## Idioms and Best Practices
### Ownership
- Prefer borrowing (`&T`, `&mut T`) over cloning wherever the lifetime allows
- Use `Cow<'_, str>` for functions that sometimes need to own and sometimes borrow
- Prefer `impl Trait` in function signatures over `Box<dyn Trait>` for static dispatch
### Error Handling
- Use `?` operator to propagate errors — avoid manual `match Err(e) => return Err(e)`
- Define domain error types with `thiserror` in libraries; use `anyhow` in binaries
- Never use `.unwrap()` in library code — it panics the caller's thread
### Modern Rust
- Prefer `if let` / `while let` for single-variant matches over full `match`
- Prefer `?` over `unwrap` everywhere errors are recoverable
- Use `#[derive(Debug, Clone, PartialEq)]` consistently on data types
- Prefer `iter()` chains over manual loops — they compose and optimize well
- Use `clippy` and treat its lints as required — flag any `#[allow(clippy::...)]` in review
FILE:languages/swift.md
---
language: swift
extensions: [".swift"]
---
# Swift — Language-Specific Review Notes
Load this file alongside `rules/universal.md`. Universal rules are not repeated here — only Swift-specific rules and idioms.
---
## PR Analyzer — Swift Risk Signals
- `print()` statements left in production code
- Force unwrap (`!`) on optionals outside of tests or justified init
- Force cast (`as!`) without a safe fallback
- Hardcoded credentials or API keys in source
---
## Code Quality — Swift Checks
- Force unwrap (`!`) used broadly — prefer `guard let` or `if let`
- `try!` used outside of guaranteed-safe contexts
- Retain cycles in closures — missing `[weak self]` or `[unowned self]`
- `@objc` / `dynamic` used without an Objective-C interop reason
---
## Security
- Flag credentials stored in `UserDefaults` — require Keychain
- Flag `URLSession` requests over plain HTTP in production
- Flag `WKWebView` loading arbitrary user-supplied URLs without validation
---
## Async / Concurrency
- Flag `DispatchQueue.main.sync` called from the main thread — deadlock
- Flag `@escaping` closures capturing `self` strongly in reference cycles — use `[weak self]`
- Flag mixing `async/await` and `DispatchQueue` for the same operation without clear reasoning
- Flag `Task { }` (unstructured) where a structured `async let` or `TaskGroup` would maintain structure
- Flag data races — shared mutable state accessed from multiple tasks without an actor
---
## Resource Management
- Flag `URLSessionDataTask` started with no cancellation handle stored — cannot be cancelled if the view disappears
- Flag `NotificationCenter` observers added without a corresponding `removeObserver` — memory leak
- Flag `CLLocationManager` / `AVCaptureSession` not stopped when the owning view controller is dismissed
---
## Exception Handling
- Flag `try!` outside of guaranteed-safe contexts (test fixtures, constants) — crashes on failure
- Flag `try?` discarding errors where the failure mode matters to the caller
- Flag error types conforming to `Error` with no associated values or message — makes debugging hard
- Flag throwing functions calling `fatalError()` as a fallback — choose one error strategy
---
## Performance
- Flag `UIImage(named:)` called repeatedly for the same asset without caching
- Flag synchronous network calls on the main thread
- Flag `Array` used for frequent membership tests — prefer `Set`
- Flag `String` interpolation inside tight loops where a pre-built string would avoid allocations
---
## Idioms and Best Practices
### Optionals
- Prefer `guard let` for early exit; `if let` for local scope
- Prefer optional chaining (`?.`) over force unwrap
- Flag implicitly unwrapped optionals (`var x: String!`) outside of `@IBOutlet`
### Memory Management
- Flag closures capturing `self` strongly in reference cycles — use `[weak self]`
- Prefer `struct` over `class` for value semantics unless identity or inheritance is needed
- Use `unowned` only when the lifetime is guaranteed — otherwise `weak`
### Concurrency (Swift 5.5+)
- Prefer `async/await` over completion handlers in new code
- Flag `DispatchQueue.main.async` where `@MainActor` or `await MainActor.run` is more appropriate
FILE:languages/typescript.md
---
language: typescript
extensions: [".ts", ".tsx", ".js", ".jsx", ".mjs"]
---
# TypeScript / JavaScript — Language-Specific Review Notes
Load this file alongside `rules/universal.md`. Universal rules are not repeated here — only TypeScript/JavaScript-specific rules and idioms.
---
## PR Analyzer — TypeScript / JavaScript Risk Signals
- `console.log` / `debugger` statements left in production code
- `// eslint-disable` comments — verify they are justified
- `any` type annotations — require explicit justification
- `@ts-ignore` / `@ts-expect-error` — verify they are justified
- `eval()` with any dynamic or user-controlled input
- Hardcoded API keys or tokens in source
---
## Code Quality — TypeScript / JavaScript Checks
- `any` used broadly instead of proper typing
- Non-null assertion (`!`) used without justification
- `var` declarations — prefer `const` / `let`
- Missing `await` on async function calls
- Floating promises (no `.catch()` and no `await`)
- `==` used instead of `===`
---
## Security
- Flag `innerHTML`, `outerHTML`, `document.write()` with user-controlled data — use `textContent` or a sanitizer
- Flag `dangerouslySetInnerHTML` in React without a sanitizer
- Flag `eval()` / `new Function()` with dynamic input
- Flag JWT decoded without signature verification
- Flag missing `httpOnly` / `secure` flags on cookies
---
## Async / Promises
- Flag floating promises — async calls not `await`-ed and without `.catch()`
- Flag `Promise.all()` where `Promise.allSettled()` is safer (one failure should not cancel siblings)
- Flag `async` functions inside `forEach` — `forEach` does not await; use `for...of` or `Promise.all()`
- Flag unhandled promise rejection (no global `unhandledRejection` handler in Node.js services)
---
## Resource Management
- Flag `fs.createReadStream` / `fs.createWriteStream` with no `close` or `destroy` on error
- Flag `EventEmitter` listeners added in a loop without removal — memory leak
- Flag `setInterval` / `setTimeout` handles not cleared when the owning component unmounts or exits
- Flag database clients / pools not released after use in Node.js
---
## Exception Handling
- Flag `catch (e) {}` (empty catch) — swallowed error
- Flag `catch (e)` where `e` is used as `any` without narrowing — type the error properly
- Flag `Promise` rejection not handled — `.catch()` or `try/await/catch` required
- Flag re-throwing a new `Error` without wrapping the original — loses stack context
- Use `Error` subclasses for domain errors rather than plain strings or object literals
---
## Performance
- Flag `Array.prototype.find` / `filter` / `map` chained multiple times over the same array — combine into one pass
- Flag DOM queries (`document.querySelector`) inside loops — cache the result
- Flag `JSON.parse` / `JSON.stringify` in a hot path on large objects — consider streaming or partial parsing
- Flag `async` functions called sequentially in a loop where `Promise.all()` would parallelize them
---
## Idioms and Best Practices
### Type Safety (TypeScript)
- Prefer `unknown` over `any` for truly unknown values — forces a type guard before use
- Prefer type narrowing (`typeof`, `instanceof`, discriminated unions) over casting
- Enable `strict` mode in `tsconfig.json`
- Prefer `interface` for object shapes that may be extended; `type` for unions and aliases
### Modern JavaScript / TypeScript
- Prefer `const` by default; `let` only when reassignment is needed
- Prefer optional chaining (`?.`) and nullish coalescing (`??`) over manual null guards
- Prefer `structuredClone()` over manual deep-copy patterns
- Prefer named exports over default exports for better refactoring support
### Null / Undefined Safety
- Distinguish between `null` (intentional absence) and `undefined` (not set) — be consistent
- Flag `== null` checks that accidentally include `undefined` when only one is intended
FILE:README.md
# code-reviewer
Code review automation for TypeScript, JavaScript, Python, Go, Swift, Kotlin, C#, .NET, Java, C, C++, Rust, Ruby, PHP, and Dart/Flutter. Analyzes PRs for complexity and risk, checks code quality for SOLID violations and code smells, and generates review reports.
The full skill spec is [`SKILL.md`](./SKILL.md). This README is a quick reference for the 3 bundled scripts.
---
## How to use
### Quick install check
```bash
python scripts/pr_analyzer.py --help
python scripts/code_quality_checker.py --help
python scripts/review_report_generator.py --help
```
All three scripts are stdlib-only — no `pip install` required.
### Example 1 — review a pull request
```bash
# From inside the repo you want to analyze:
python /path/to/skills/code-reviewer/scripts/pr_analyzer.py . --base main --head HEAD
```
Outputs: complexity score (1-10), risk categorization (critical / high / medium / low), prioritized review order, commit-message validation.
### Example 2 — score a directory's code quality
```bash
python scripts/code_quality_checker.py /path/to/code
# Filter by language
python scripts/code_quality_checker.py /path/to/code --language csharp
# Machine-readable
python scripts/code_quality_checker.py /path/to/code --json
```
Outputs: quality score (0-100), letter grade, detected code smells, SOLID violations.
### Example 3 — combine into a review report
```bash
python scripts/review_report_generator.py /path/to/repo --format markdown --output review.md
```
Outputs: review verdict (approve / request changes / block), score, prioritized action items.
---
## Examples bundled with the skill
| File | Purpose |
|------|---------|
| [`assets/sample_csharp_smells.cs`](./assets/sample_csharp_smells.cs) | C# file with every C#-specific pattern this skill detects, labelled inline |
| [`assets/sample_csharp_clean.cs`](./assets/sample_csharp_clean.cs) | Same code refactored per `rules/universal.md` + `languages/csharp.md` |
| [`assets/sample_java_smells.java`](./assets/sample_java_smells.java) | Java file with every Java-specific pattern this skill detects, labelled inline |
| [`assets/sample_java_clean.java`](./assets/sample_java_clean.java) | Same code refactored per `rules/universal.md` + `languages/java.md` |
| [`assets/sample_c_smells.c`](./assets/sample_c_smells.c) | C file with every C-specific pattern this skill detects, labelled inline |
| [`assets/sample_c_clean.c`](./assets/sample_c_clean.c) | Same code refactored per `rules/universal.md` + `languages/c.md` |
| [`expected_outputs/*.json`](./expected_outputs/) | Expected `code_quality_checker.py --json` output for each fixture |
Use them as a regression-detection harness:
```bash
python scripts/code_quality_checker.py assets/sample_java_smells.java --json > /tmp/check.json
diff /tmp/check.json expected_outputs/sample_java_smells_quality.json
# silence means the detector still behaves as documented
```
---
## What it detects
See [`SKILL.md`](./SKILL.md) for the full pattern list, severity tiers, and references. Quick summary:
- **PR Analyzer** (`scripts/pr_analyzer.py`): hardcoded secrets / connection strings, SQL injection, debug statements (`console.*` / `System.out` / `printStackTrace`), analyzer suppressions (ESLint / Roslyn / `@SuppressWarnings`), `any` / `dynamic` overuse, TODO/FIXME, `unsafe` blocks, null-forgiving `!`, `async void`, blocking on `Task`.
- **Code Quality Checker** (`scripts/code_quality_checker.py`): long methods, large files, god classes, deep nesting, too many parameters, high cyclomatic complexity, swallowed exceptions, missing `await`, undisposed `IDisposable`, `new HttpClient()` in method body, unused `using` directives. Language-specific smell packs for C# (`async void`, blocking on `Task`), Java (empty catch, `printStackTrace`, swallowed `InterruptedException`, unclosed resources, per-call `ObjectMapper` / `Gson`), and C (banned functions `gets`/`strcpy`/`strcat`/`sprintf`/`vsprintf`, format-string vulnerability `printf(var)`, unbounded `scanf("%s")`, malloc-without-NULL-check, free-without-zeroing, `system()` with non-literal argument).
- **Review Report Generator** (`scripts/review_report_generator.py`): combines the above into a single markdown or JSON verdict.
---
## Review rules
Rules are split so every review loads exactly two files — the cross-language
baseline plus one language guide (see the dispatch table in [`SKILL.md`](./SKILL.md)):
- [`rules/universal.md`](./rules/universal.md) — cross-language rules: security, async/concurrency, resource management, exception handling, performance
- [`languages/`](./languages/) — one self-contained guide per language (`python`, `typescript`, `go`, `swift`, `kotlin`, `csharp`, `java`, `c`, `cpp`, `rust`, `ruby`, `php`, `dart`), each with Security / Async / Resource Management / Exception Handling / Performance / Idioms sections
FILE:rules/universal.md
# Universal Rules — All Languages
These rules apply regardless of language. Load this file for every review, alongside the relevant `languages/*.md` file.
---
## Security
- Flag any string interpolation or concatenation used to build SQL, shell, or LDAP queries — require parameterized queries or a safe API
- Flag hardcoded credentials, API keys, tokens, or secrets anywhere in source — require environment variables or a secrets manager
- Flag user-controlled input passed to file system, process execution, or URL redirect APIs without validation
- Flag overly broad CORS or CSP policies
---
## Async / Concurrency
- Flag shared mutable state accessed from multiple threads/coroutines/tasks without synchronization
- Flag fire-and-forget async operations with no error handling path
- Flag timeouts missing on any network or I/O call
- Flag unbounded queues or thread pools with no backpressure mechanism
---
## Resource Management
- Flag any resource (file, socket, DB connection, HTTP connection) acquired without a guaranteed release path
- Flag connection pools not returned to the pool on all code paths (including exceptions)
- Flag unbounded collections that grow without eviction — potential memory leak
- Flag resources held open longer than the operation they serve
---
## Exception Handling
- Flag empty catch/except blocks — swallowed exceptions hide bugs silently
- Flag catching the broadest possible exception type (`Exception`, `Throwable`, `error`) where a specific type is appropriate
- Flag exceptions used for normal control flow (signaling "not found", etc.) — use return values or `Optional`
- Flag error context lost when re-throwing — always wrap with the original cause
---
## Performance
- Flag N+1 query patterns — loading a collection then querying for each item individually
- Flag unbounded queries or API calls with no pagination or limit
- Flag synchronous I/O on a thread or event loop that serves concurrent requests
- Flag large objects serialized/deserialized repeatedly when they could be cached
- Flag string concatenation in tight loops — use a builder or join
FILE:scripts/code_quality_checker.py
#!/usr/bin/env python3
"""
Code Quality Checker
Analyzes source code for quality issues, code smells, complexity metrics,
and SOLID principle violations.
Usage:
python code_quality_checker.py /path/to/file.py
python code_quality_checker.py /path/to/directory --recursive
python code_quality_checker.py . --language typescript --json
"""
import argparse
import json
import re
import sys
from pathlib import Path
from typing import Dict, List, Optional
# Language-specific file extensions.
# `c` is declared before `cpp` so plain `.h` resolves to C, matching the
# dispatch table in SKILL.md. C++ headers use `.hpp` / `.hh` / `.hxx`.
LANGUAGE_EXTENSIONS = {
"python": [".py"],
"typescript": [".ts", ".tsx"],
"javascript": [".js", ".jsx", ".mjs"],
"go": [".go"],
"swift": [".swift"],
"kotlin": [".kt", ".kts"],
"csharp": [".cs", ".csx", ".razor", ".cshtml"],
"java": [".java"],
"c": [".c", ".h"],
"cpp": [".cpp", ".cc", ".cxx", ".hpp", ".hh", ".hxx"],
"rust": [".rs"],
"ruby": [".rb", ".rake", ".gemspec", ".ru"],
"php": [".php", ".phtml"],
"dart": [".dart"],
}
# Code smell thresholds
THRESHOLDS = {
"long_function_lines": 50,
"too_many_parameters": 5,
"high_complexity": 10,
"god_class_methods": 20,
"max_imports": 15
}
def get_file_extension(filepath: Path) -> str:
"""Get file extension."""
return filepath.suffix.lower()
def detect_language(filepath: Path) -> Optional[str]:
"""Detect programming language from file extension."""
ext = get_file_extension(filepath)
for lang, extensions in LANGUAGE_EXTENSIONS.items():
if ext in extensions:
return lang
return None
def read_file_content(filepath: Path) -> str:
"""Read file content safely."""
try:
with open(filepath, "r", encoding="utf-8", errors="ignore") as f:
return f.read()
except Exception:
return ""
def calculate_cyclomatic_complexity(content: str) -> int:
"""
Estimate cyclomatic complexity based on control flow keywords.
"""
complexity = 1 # Base complexity
# Control flow patterns that increase complexity
patterns = [
r"\bif\b",
r"\belif\b",
r"\belse\b",
r"\bfor\b",
r"\bwhile\b",
r"\bcase\b",
r"\bcatch\b",
r"\bexcept\b",
r"\band\b",
r"\bor\b",
r"\|\|",
r"&&"
]
for pattern in patterns:
matches = re.findall(pattern, content, re.IGNORECASE)
complexity += len(matches)
return complexity
def count_lines(content: str) -> Dict[str, int]:
"""Count different types of lines in code."""
lines = content.split("\n")
total = len(lines)
blank = sum(1 for line in lines if not line.strip())
comment = 0
for line in lines:
stripped = line.strip()
if stripped.startswith("#") or stripped.startswith("//"):
comment += 1
elif stripped.startswith("/*") or stripped.startswith("'''") or stripped.startswith('"""'):
comment += 1
code = total - blank - comment
return {
"total": total,
"code": code,
"blank": blank,
"comment": comment
}
def find_functions(content: str, language: str) -> List[Dict]:
"""Find function definitions and their metrics."""
functions = []
# Language-specific function patterns
patterns = {
"python": r"def\s+(\w+)\s*\(([^)]*)\)",
"typescript": r"(?:function\s+(\w+)|(?:const|let|var)\s+(\w+)\s*=\s*(?:async\s+)?\([^)]*\)\s*=>)",
"javascript": r"(?:function\s+(\w+)|(?:const|let|var)\s+(\w+)\s*=\s*(?:async\s+)?\([^)]*\)\s*=>)",
"go": r"func\s+(?:\([^)]+\)\s+)?(\w+)\s*\(([^)]*)\)",
"swift": r"func\s+(\w+)\s*\(([^)]*)\)",
"kotlin": r"fun\s+(\w+)\s*\(([^)]*)\)",
# C#: require at least one method modifier (public/private/etc. or static/async/...)
# to distinguish declarations from invocations.
"csharp": (
r"(?:(?:public|private|protected|internal|static|async|virtual|"
r"override|sealed|abstract|partial|new|readonly|extern)\s+)+"
r"(?:[\w<>?,\s\[\]\.]+?\s+)?(\w+)\s*\(([^)]*)\)"
),
# Java: require at least one method modifier to distinguish
# declarations from invocations (mirrors the C# approach).
"java": (
r"(?:(?:public|private|protected|static|final|abstract|"
r"synchronized|native|default|strictfp)\s+)+"
r"(?:[\w<>?,\s\[\]\.]+?\s+)?(\w+)\s*\(([^)]*)\)"
),
# C: require an opening brace after the parens so prototypes and
# call sites don't get matched. Return type / qualifiers come first.
# Skip C control-flow keywords that look like function calls.
"c": (
r"^(?:static\s+|inline\s+|extern\s+|const\s+|unsigned\s+|"
r"signed\s+|volatile\s+|register\s+)*"
r"(?:[\w\*]+\s+\**)+"
r"(?!(?:if|while|for|switch|return|sizeof)\b)"
r"(\w+)\s*\(([^)]*)\)\s*\{"
),
# C++: like C but also catches `ClassName::method(...)` definitions
# and template return types like `std::vector<int>`.
"cpp": (
r"^(?:static\s+|inline\s+|extern\s+|const\s+|virtual\s+|"
r"explicit\s+|constexpr\s+|noexcept\s+)*"
r"(?:[\w:\*<>&,\s]+\s+\**)+"
r"(?!(?:if|while|for|switch|return|sizeof)\b)"
r"(\w+)(?:::\w+)?\s*\(([^)]*)\)\s*(?:const\s*)?"
r"(?:noexcept\s*)?(?:override\s*)?(?:final\s*)?\{"
),
# Rust: `fn` keyword is always present and unambiguous.
"rust": (
r"(?:pub(?:\([^)]+\))?\s+)?(?:async\s+)?(?:unsafe\s+)?"
r"(?:extern\s+\"[^\"]+\"\s+)?fn\s+(\w+)\s*"
r"(?:<[^>]+>)?\s*\(([^)]*)\)"
),
# Ruby: `def` keyword; params may be parenthesised or bare.
"ruby": (
r"def\s+(?:self\.)?(\w+[?!=]?)(?:\s*\(([^)]*)\)|\s*$|\s+\w)"
),
# PHP: `function` keyword is always present.
"php": (
r"(?:(?:public|private|protected|static|abstract|final)\s+)*"
r"function\s+(\w+)\s*\(([^)]*)\)"
),
# Dart: typed return followed by name and parens. Constructors
# (where name matches enclosing class) are not specially handled.
"dart": (
r"^\s*(?:static\s+|external\s+)*"
r"(?:Future<[^>]*>|Stream<[^>]*>|void|[\w<>?,\s]+?)\s+"
r"(\w+)\s*\(([^)]*)\)\s*(?:async\*?\s*|sync\*?\s*)?\{"
),
}
pattern = patterns.get(language, patterns["python"])
matches = re.finditer(pattern, content, re.MULTILINE)
for match in matches:
name = next((g for g in match.groups() if g), "anonymous")
params_str = match.group(2) if len(match.groups()) > 1 and match.group(2) else ""
# Count parameters
params = [p.strip() for p in params_str.split(",") if p.strip()]
param_count = len(params)
# Estimate function length
start_pos = match.end()
remaining = content[start_pos:]
next_func = re.search(pattern, remaining)
if next_func:
func_body = remaining[:next_func.start()]
else:
func_body = remaining[:min(2000, len(remaining))]
line_count = len(func_body.split("\n"))
complexity = calculate_cyclomatic_complexity(func_body)
functions.append({
"name": name,
"parameters": param_count,
"lines": line_count,
"complexity": complexity
})
return functions
def find_classes(content: str, language: str) -> List[Dict]:
"""Find class definitions and their metrics."""
classes = []
patterns = {
"python": r"class\s+(\w+)",
"typescript": r"class\s+(\w+)",
"javascript": r"class\s+(\w+)",
"go": r"type\s+(\w+)\s+struct",
"swift": r"class\s+(\w+)",
"kotlin": r"class\s+(\w+)",
"csharp": r"(?:class|struct|record|interface)\s+(\w+)",
"java": r"(?:class|interface|enum|record)\s+(\w+)",
# C has no classes; `struct` and `typedef struct` are the closest.
"c": r"(?:typedef\s+)?struct\s+(\w+)",
"cpp": r"(?:class|struct)\s+(\w+)",
# Rust uses `struct`, `enum`, `trait`, `union` for type definitions.
# `impl` blocks attach methods but are not type defs themselves.
"rust": r"(?:pub(?:\([^)]+\))?\s+)?(?:struct|enum|trait|union)\s+(\w+)",
"ruby": r"(?:class|module)\s+(\w+)",
"php": (
r"(?:abstract\s+|final\s+)?"
r"(?:class|interface|trait|enum)\s+(\w+)"
),
# Dart 3 class modifiers: final / interface / base / sealed / mixin.
"dart": (
r"(?:abstract\s+|sealed\s+|final\s+|base\s+|interface\s+)?"
r"(?:class|mixin|enum|extension)\s+(\w+)"
),
}
pattern = patterns.get(language, patterns["python"])
matches = re.finditer(pattern, content)
for match in matches:
name = match.group(1)
start_pos = match.end()
remaining = content[start_pos:]
next_class = re.search(pattern, remaining)
if next_class:
class_body = remaining[:next_class.start()]
else:
class_body = remaining
# Count methods
method_patterns = {
"python": r"def\s+\w+\s*\(",
"typescript": r"(?:public|private|protected)?\s*\w+\s*\([^)]*\)\s*[:{]",
"javascript": r"\w+\s*\([^)]*\)\s*\{",
"go": r"func\s+\(",
"swift": r"func\s+\w+",
"kotlin": r"fun\s+\w+",
"csharp": (
r"(?:(?:public|private|protected|internal|static|async|virtual|"
r"override|sealed|abstract|partial)\s+)+"
r"(?:[\w<>?,\s\[\]\.]+?\s+)?\w+\s*\("
),
"java": (
r"(?:(?:public|private|protected|static|final|abstract|"
r"synchronized|native|default|strictfp)\s+)+"
r"(?:[\w<>?,\s\[\]\.]+?\s+)?\w+\s*\("
),
# C has no classes; struct members are typically function pointers
# rather than methods. Use the function definition pattern.
"c": (
r"^(?:static\s+|inline\s+)*(?:[\w\*]+\s+\**)+"
r"(?!(?:if|while|for|switch|return|sizeof)\b)"
r"\w+\s*\([^)]*\)\s*\{"
),
"cpp": (
r"^(?:static\s+|inline\s+|virtual\s+|explicit\s+|"
r"constexpr\s+)*(?:[\w:\*<>&,\s]+\s+\**)+"
r"(?!(?:if|while|for|switch|return|sizeof)\b)"
r"\w+(?:::\w+)?\s*\([^)]*\)"
),
"rust": (
r"(?:pub(?:\([^)]+\))?\s+)?(?:async\s+)?(?:unsafe\s+)?"
r"fn\s+\w+"
),
"ruby": r"def\s+(?:self\.)?\w+[?!=]?",
"php": (
r"(?:(?:public|private|protected|static|abstract|final)\s+)*"
r"function\s+\w+\s*\("
),
"dart": (
r"^\s*(?:static\s+|external\s+)*"
r"(?:Future<[^>]*>|Stream<[^>]*>|void|[\w<>?,\s]+?)\s+"
r"\w+\s*\([^)]*\)\s*(?:async\*?\s*|sync\*?\s*)?\{"
),
}
method_pattern = method_patterns.get(language, method_patterns["python"])
methods = len(re.findall(method_pattern, class_body))
classes.append({
"name": name,
"methods": methods,
"lines": len(class_body.split("\n"))
})
return classes
def check_code_smells(content: str, functions: List[Dict], classes: List[Dict]) -> List[Dict]:
"""Check for code smells in the content."""
smells = []
# Long functions
for func in functions:
if func["lines"] > THRESHOLDS["long_function_lines"]:
smells.append({
"type": "long_function",
"severity": "medium",
"message": f"Function '{func['name']}' has {func['lines']} lines (max: {THRESHOLDS['long_function_lines']})",
"location": func["name"]
})
# Too many parameters
for func in functions:
if func["parameters"] > THRESHOLDS["too_many_parameters"]:
smells.append({
"type": "too_many_parameters",
"severity": "low",
"message": f"Function '{func['name']}' has {func['parameters']} parameters (max: {THRESHOLDS['too_many_parameters']})",
"location": func["name"]
})
# High complexity
for func in functions:
if func["complexity"] > THRESHOLDS["high_complexity"]:
severity = "high" if func["complexity"] > 20 else "medium"
smells.append({
"type": "high_complexity",
"severity": severity,
"message": f"Function '{func['name']}' has complexity {func['complexity']} (max: {THRESHOLDS['high_complexity']})",
"location": func["name"]
})
# God classes
for cls in classes:
if cls["methods"] > THRESHOLDS["god_class_methods"]:
smells.append({
"type": "god_class",
"severity": "high",
"message": f"Class '{cls['name']}' has {cls['methods']} methods (max: {THRESHOLDS['god_class_methods']})",
"location": cls["name"]
})
# Magic numbers
magic_pattern = r"\b(?<![.\"\'])\d{3,}\b(?!\.\d)"
for i, line in enumerate(content.split("\n"), 1):
if line.strip().startswith(("#", "//", "import", "from")):
continue
matches = re.findall(magic_pattern, line)
for match in matches[:1]: # One per line
smells.append({
"type": "magic_number",
"severity": "low",
"message": f"Magic number {match} should be a named constant",
"location": f"line {i}"
})
# Commented code patterns
commented_code_pattern = r"^\s*[#//]+\s*(if|for|while|def|function|class|const|let|var)\s"
for i, line in enumerate(content.split("\n"), 1):
if re.match(commented_code_pattern, line, re.IGNORECASE):
smells.append({
"type": "commented_code",
"severity": "low",
"message": "Commented-out code should be removed",
"location": f"line {i}"
})
return smells
def _strip_csharp_comments(content: str) -> str:
"""Remove // line comments and /* */ block comments so regex detectors
don't match keywords inside prose."""
no_block = re.sub(r"/\*.*?\*/", "", content, flags=re.DOTALL)
no_line = re.sub(r"//[^\n]*", "", no_block)
return no_line
def check_csharp_specific_smells(content: str) -> List[Dict]:
"""C# / .NET-specific code smells documented in SKILL.md."""
smells: List[Dict] = []
content = _strip_csharp_comments(content)
# async void (event handler exception only — caller must justify)
for match in re.finditer(r"\basync\s+void\s+(\w+)\s*\(", content):
smells.append({
"type": "csharp_async_void",
"severity": "high",
"message": (
f"'async void {match.group(1)}' — only safe for event handlers; "
"prefer 'async Task'"
),
"location": match.group(1),
})
# Blocking on async: .Result, .Wait(), .GetAwaiter().GetResult()
for match in re.finditer(
r"\.(?:Result\b|Wait\(\)|GetAwaiter\(\)\.GetResult\(\))", content
):
smells.append({
"type": "csharp_blocking_async",
"severity": "high",
"message": (
"Blocking call on async operation ('.Result' / '.Wait()' / "
"'.GetAwaiter().GetResult()') — can deadlock in ASP.NET contexts"
),
"location": f"offset {match.start()}",
})
# Bare catch / catch (Exception) that swallows
swallow_pattern = re.compile(
r"catch\s*(?:\(\s*(?:System\.)?Exception(?:\s+\w+)?\s*\))?\s*\{\s*\}"
)
for match in swallow_pattern.finditer(content):
smells.append({
"type": "csharp_swallowed_exception",
"severity": "high",
"message": "Empty catch block swallows exceptions silently",
"location": f"offset {match.start()}",
})
# IDisposable instantiated but not in `using` — heuristic: `new SomethingClient(`
# / `new SomethingStream(` / `new SqlConnection(` outside a `using` line.
disposable_hint = re.compile(
r"^(?!\s*using\b)\s*(?:var|[\w<>]+)\s+\w+\s*=\s*new\s+"
r"(\w*(?:Stream|Connection|Reader|Writer|Client|Context|Command))\s*\(",
re.MULTILINE,
)
for match in disposable_hint.finditer(content):
smells.append({
"type": "csharp_undisposed_idisposable",
"severity": "medium",
"message": (
f"'{match.group(1)}' looks like IDisposable but is not wrapped in "
"'using' / 'using var'"
),
"location": f"offset {match.start()}",
})
# HttpClient instantiated with `new` inside a method body (socket exhaustion)
httpclient_inline = re.compile(r"new\s+HttpClient\s*\(\s*\)")
for match in httpclient_inline.finditer(content):
smells.append({
"type": "csharp_new_httpclient",
"severity": "medium",
"message": (
"'new HttpClient()' — prefer IHttpClientFactory or a long-lived "
"static instance to avoid socket exhaustion"
),
"location": f"offset {match.start()}",
})
# Missing await: `Task.Run(` / async method call assigned but never awaited.
# Heuristic: a statement ending in `Async()` or `Async(...)` followed by `;`
# with no `await` keyword on the same line.
for line_no, line in enumerate(content.split("\n"), 1):
stripped = line.strip()
if not stripped or stripped.startswith(("//", "/*", "*")):
continue
if re.search(r"\b\w+Async\s*\([^)]*\)\s*;\s*$", stripped) and "await " not in stripped:
# Skip `return ...Async();` (forwarding the Task is legitimate)
if stripped.startswith("return "):
continue
smells.append({
"type": "csharp_missing_await",
"severity": "medium",
"message": "Async method called without 'await' — Task is discarded",
"location": f"line {line_no}",
})
# Unnecessary `using` directives — heuristic: `using` directive whose
# namespace tail isn't referenced anywhere else in the file.
using_directives = re.findall(
r"^using\s+(?:static\s+)?([A-Z]\w*(?:\.\w+)*)\s*;", content, re.MULTILINE
)
body = re.sub(r"^using\s+[^;]+;\s*$", "", content, flags=re.MULTILINE)
for ns in using_directives:
tail = ns.split(".")[-1]
if not re.search(rf"\b{re.escape(tail)}\b", body):
smells.append({
"type": "csharp_unused_using",
"severity": "low",
"message": f"'using {ns};' appears unused",
"location": ns,
})
return smells
def check_java_specific_smells(content: str) -> List[Dict]:
"""Java-specific code smells documented in languages/java.md."""
smells: List[Dict] = []
# Java comment syntax matches C#, so the same stripper applies.
content = _strip_csharp_comments(content)
# Empty catch block — swallows the exception silently.
for match in re.finditer(r"catch\s*\([^)]*\)\s*\{\s*\}", content):
smells.append({
"type": "java_empty_catch",
"severity": "high",
"message": "Empty catch block swallows exceptions silently",
"location": f"offset {match.start()}",
})
# printStackTrace() as error handling — use a logger instead.
for match in re.finditer(r"\.printStackTrace\s*\(\s*\)", content):
smells.append({
"type": "java_print_stack_trace",
"severity": "medium",
"message": (
"'printStackTrace()' is not real error handling — log via a "
"proper logger or rethrow with context"
),
"location": f"offset {match.start()}",
})
# InterruptedException caught without restoring the interrupt flag.
for match in re.finditer(
r"catch\s*\(\s*InterruptedException\s+(\w+)\s*\)\s*\{(.*?)\}",
content,
re.DOTALL,
):
if "interrupt()" not in match.group(2):
smells.append({
"type": "java_swallowed_interrupt",
"severity": "high",
"message": (
"InterruptedException caught without "
"'Thread.currentThread().interrupt()' — breaks cooperative "
"cancellation"
),
"location": f"offset {match.start()}",
})
# Closeable resource instantiated outside try-with-resources (leak heuristic).
resource_hint = re.compile(
r"^(?!\s*try\b)\s*(?:final\s+)?[\w<>\[\]]+\s+\w+\s*=\s*new\s+"
r"(\w*(?:InputStream|OutputStream|Reader|Writer|Stream|Connection))\s*\(",
re.MULTILINE,
)
for match in resource_hint.finditer(content):
smells.append({
"type": "java_unclosed_resource",
"severity": "medium",
"message": (
f"'{match.group(1)}' looks like an AutoCloseable but is not in a "
"try-with-resources statement"
),
"location": f"offset {match.start()}",
})
# Heavy object built per use instead of shared as a singleton.
# A `static` field assignment is the recommended singleton form — skip it.
heavy_object = re.compile(
r"^(?!.*\bstatic\b).*\bnew\s+(ObjectMapper|Gson)\s*\(\s*\)",
re.MULTILINE,
)
for match in heavy_object.finditer(content):
smells.append({
"type": "java_per_use_heavy_object",
"severity": "medium",
"message": (
f"'new {match.group(1)}()' is expensive — share a singleton "
"instance instead of constructing per call"
),
"location": f"offset {match.start()}",
})
return smells
def check_c_specific_smells(content: str) -> List[Dict]:
"""C-specific code smells documented in languages/c.md.
Focuses on memory-safety and command/format-string patterns that the
CERT C Coding Standard and the CWE catalogue rank as the
highest-impact footguns. C uses the same line/block comment syntax as
C# and Java, so the existing comment stripper applies.
"""
smells: List[Dict] = []
content = _strip_csharp_comments(content)
# 1. Banned functions — no bounds check on any of them.
banned = {
"gets": "no bounds check, removed from C11 (CWE-242)",
"strcpy": "no bounds check — prefer strncpy or strlcpy",
"strcat": "no bounds check — prefer strncat or strlcat",
"sprintf": "no bounds check — prefer snprintf",
"vsprintf": "no bounds check — prefer vsnprintf",
}
for fn, reason in banned.items():
for m in re.finditer(rf"\b{fn}\s*\(", content):
smells.append({
"type": f"c_banned_{fn}",
"severity": "high",
"message": f"'{fn}()' is unsafe: {reason}",
"location": f"offset {m.start()}",
})
# 2. Format-string vulnerability — printf/syslog called with a bare
# identifier as the format argument (CWE-134). Skip when the first
# arg is a string literal.
for fn in ("printf", "syslog"):
pattern = rf"\b{fn}\s*\(\s*(?!\")(\w+)\s*[,\)]"
for m in re.finditer(pattern, content):
smells.append({
"type": "c_format_string",
"severity": "high",
"message": (
f"'{fn}({m.group(1)})' uses a non-literal format string "
"— CWE-134 format string vulnerability"
),
"location": f"offset {m.start()}",
})
# 3. Unbounded scanf — `%s` without a width specifier invites overflow.
scanf_call = re.compile(
r"\b(?:scanf|fscanf|sscanf)\s*\(\s*[^)]*?\"([^\"]*)\""
)
for m in scanf_call.finditer(content):
fmt = m.group(1)
if "%s" in fmt and not re.search(r"%\d+s", fmt):
smells.append({
"type": "c_unbounded_scanf",
"severity": "high",
"message": (
"scanf '%s' without a width specifier — unbounded read "
"can overflow the destination buffer"
),
"location": f"offset {m.start()}",
})
# 4. malloc / calloc / realloc result dereferenced without a NULL check
# within 5 lines. CWE-690.
lines = content.split("\n")
malloc_assign = re.compile(
r"^\s*(?:[\w\*]+\s+)?\*?(\w+)\s*=\s*\(?[\w\s\*]*\)?\s*"
r"(?:m|c|re)alloc\s*\("
)
for i, line in enumerate(lines):
m = malloc_assign.match(line)
if not m:
continue
var = m.group(1)
window = "\n".join(lines[i + 1 : i + 6])
null_check = re.compile(
rf"\bif\s*\([^)]*(?:{re.escape(var)}\s*==\s*NULL"
rf"|NULL\s*==\s*{re.escape(var)}"
rf"|!\s*{re.escape(var)}\b"
rf"|{re.escape(var)}\s*!=\s*NULL)"
)
if not null_check.search(window):
smells.append({
"type": "c_malloc_unchecked",
"severity": "medium",
"message": (
f"'{var}' from malloc/calloc/realloc is not NULL-checked "
"within 5 lines — dereferencing NULL is UB (CWE-690)"
),
"location": f"line {i + 1}",
})
# 5. free(p) without setting p to NULL on the next real line.
# CWE-416 use-after-free guardrail.
free_call = re.compile(r"^\s*free\s*\(\s*(\w+)\s*\)\s*;")
for i, line in enumerate(lines):
m = free_call.match(line)
if not m:
continue
var = m.group(1)
for j in range(i + 1, min(i + 3, len(lines))):
nxt = lines[j].strip()
if not nxt:
continue
if re.match(rf"^{re.escape(var)}\s*=\s*NULL\s*;", nxt):
break
smells.append({
"type": "c_free_without_null",
"severity": "low",
"message": (
f"'free({var})' not followed by '{var} = NULL;' — "
"dangling pointer can be reused (CWE-416)"
),
"location": f"line {i + 1}",
})
break
# 6. system() with a non-string-literal argument — command injection.
system_pattern = re.compile(r"\bsystem\s*\(\s*(?!\"|NULL\b)(\w+)\s*\)")
for m in system_pattern.finditer(content):
smells.append({
"type": "c_system_non_literal",
"severity": "high",
"message": (
f"'system({m.group(1)})' with a non-literal argument — "
"command injection (CWE-78); use execve with validated args"
),
"location": f"offset {m.start()}",
})
return smells
def check_solid_violations(content: str) -> List[Dict]:
"""Check for potential SOLID principle violations."""
violations = []
# OCP: Type checking instead of polymorphism
type_checks = len(re.findall(r"isinstance\(|type\(.*\)\s*==|typeof\s+\w+\s*===", content))
if type_checks > 2:
violations.append({
"principle": "OCP",
"name": "Open/Closed Principle",
"severity": "medium",
"message": f"Found {type_checks} type checks - consider using polymorphism"
})
# LSP/ISP: NotImplementedError
not_impl = len(re.findall(r"raise\s+NotImplementedError|not\s+implemented", content, re.IGNORECASE))
if not_impl:
violations.append({
"principle": "LSP/ISP",
"name": "Liskov/Interface Segregation",
"severity": "low",
"message": f"Found {not_impl} unimplemented methods - may indicate oversized interface"
})
# DIP: Too many direct imports
imports = len(re.findall(r"^(?:import|from)\s+", content, re.MULTILINE))
if imports > THRESHOLDS["max_imports"]:
violations.append({
"principle": "DIP",
"name": "Dependency Inversion Principle",
"severity": "low",
"message": f"File has {imports} imports - consider dependency injection"
})
return violations
def calculate_quality_score(
line_metrics: Dict,
functions: List[Dict],
classes: List[Dict],
smells: List[Dict],
violations: List[Dict]
) -> int:
"""Calculate overall quality score (0-100)."""
score = 100
# Deduct for code smells
for smell in smells:
if smell["severity"] == "high":
score -= 10
elif smell["severity"] == "medium":
score -= 5
elif smell["severity"] == "low":
score -= 2
# Deduct for SOLID violations
for violation in violations:
if violation["severity"] == "high":
score -= 8
elif violation["severity"] == "medium":
score -= 4
elif violation["severity"] == "low":
score -= 2
# Bonus for good comment ratio (10-30%)
if line_metrics["total"] > 0:
comment_ratio = line_metrics["comment"] / line_metrics["total"]
if 0.1 <= comment_ratio <= 0.3:
score += 5
# Bonus for reasonable function sizes
if functions:
avg_lines = sum(f["lines"] for f in functions) / len(functions)
if avg_lines < 30:
score += 5
return max(0, min(100, score))
def get_grade(score: int) -> str:
"""Convert score to letter grade."""
if score >= 90:
return "A"
elif score >= 80:
return "B"
elif score >= 70:
return "C"
elif score >= 60:
return "D"
else:
return "F"
def analyze_file(filepath: Path) -> Dict:
"""Analyze a single file for code quality."""
language = detect_language(filepath)
if not language:
return {"error": f"Unsupported file type: {filepath.suffix}"}
content = read_file_content(filepath)
if not content:
return {"error": f"Could not read file: {filepath}"}
line_metrics = count_lines(content)
functions = find_functions(content, language)
classes = find_classes(content, language)
smells = check_code_smells(content, functions, classes)
if language == "csharp":
smells.extend(check_csharp_specific_smells(content))
if language == "java":
smells.extend(check_java_specific_smells(content))
if language == "c":
smells.extend(check_c_specific_smells(content))
violations = check_solid_violations(content)
score = calculate_quality_score(line_metrics, functions, classes, smells, violations)
return {
"file": str(filepath),
"language": language,
"metrics": {
"lines": line_metrics,
"functions": len(functions),
"classes": len(classes),
"avg_complexity": round(sum(f["complexity"] for f in functions) / max(1, len(functions)), 1)
},
"quality_score": score,
"grade": get_grade(score),
"smells": smells,
"solid_violations": violations,
"function_details": functions[:10],
"class_details": classes[:10]
}
def analyze_directory(
dir_path: Path,
recursive: bool = True,
language: Optional[str] = None
) -> Dict:
"""Analyze all files in a directory."""
results = []
extensions = []
if language:
extensions = LANGUAGE_EXTENSIONS.get(language, [])
else:
for exts in LANGUAGE_EXTENSIONS.values():
extensions.extend(exts)
pattern = "**/*" if recursive else "*"
for ext in extensions:
for filepath in dir_path.glob(f"{pattern}{ext}"):
if "node_modules" in str(filepath) or ".git" in str(filepath):
continue
result = analyze_file(filepath)
if "error" not in result:
results.append(result)
if not results:
return {"error": "No supported files found"}
total_score = sum(r["quality_score"] for r in results)
avg_score = total_score / len(results)
total_smells = sum(len(r["smells"]) for r in results)
total_violations = sum(len(r["solid_violations"]) for r in results)
return {
"directory": str(dir_path),
"files_analyzed": len(results),
"average_score": round(avg_score, 1),
"overall_grade": get_grade(int(avg_score)),
"total_code_smells": total_smells,
"total_solid_violations": total_violations,
"files": sorted(results, key=lambda x: x["quality_score"])
}
def print_report(analysis: Dict) -> None:
"""Print human-readable analysis report."""
if "error" in analysis:
print(f"Error: {analysis['error']}")
return
print("=" * 60)
print("CODE QUALITY REPORT")
print("=" * 60)
if "file" in analysis:
print(f"\nFile: {analysis['file']}")
print(f"Language: {analysis['language']}")
print(f"Quality Score: {analysis['quality_score']}/100 ({analysis['grade']})")
metrics = analysis["metrics"]
print(f"\nLines: {metrics['lines']['total']} ({metrics['lines']['code']} code, {metrics['lines']['comment']} comments)")
print(f"Functions: {metrics['functions']}")
print(f"Classes: {metrics['classes']}")
print(f"Avg Complexity: {metrics['avg_complexity']}")
if analysis["smells"]:
print("\n--- CODE SMELLS ---")
for smell in analysis["smells"][:10]:
print(f" [{smell['severity'].upper()}] {smell['message']} ({smell['location']})")
if analysis["solid_violations"]:
print("\n--- SOLID VIOLATIONS ---")
for v in analysis["solid_violations"]:
print(f" [{v['principle']}] {v['message']}")
else:
print(f"\nDirectory: {analysis['directory']}")
print(f"Files Analyzed: {analysis['files_analyzed']}")
print(f"Average Score: {analysis['average_score']}/100 ({analysis['overall_grade']})")
print(f"Total Code Smells: {analysis['total_code_smells']}")
print(f"Total SOLID Violations: {analysis['total_solid_violations']}")
print("\n--- FILES BY QUALITY ---")
for f in analysis["files"][:10]:
print(f" {f['quality_score']:3d}/100 [{f['grade']}] {f['file']}")
print("\n" + "=" * 60)
def main():
parser = argparse.ArgumentParser(
description="Analyze code quality, smells, and SOLID violations"
)
parser.add_argument(
"path",
help="File or directory to analyze"
)
parser.add_argument(
"--recursive", "-r",
action="store_true",
default=True,
help="Recursively analyze directories (default: true)"
)
parser.add_argument(
"--language", "-l",
choices=list(LANGUAGE_EXTENSIONS.keys()),
help="Filter by programming language"
)
parser.add_argument(
"--json",
action="store_true",
help="Output in JSON format"
)
parser.add_argument(
"--output", "-o",
help="Write output to file"
)
args = parser.parse_args()
target = Path(args.path).resolve()
if not target.exists():
print(f"Error: Path does not exist: {target}", file=sys.stderr)
sys.exit(1)
if target.is_file():
analysis = analyze_file(target)
else:
analysis = analyze_directory(target, args.recursive, args.language)
if args.json:
output = json.dumps(analysis, indent=2, default=str)
if args.output:
with open(args.output, "w") as f:
f.write(output)
print(f"Results written to {args.output}")
else:
print(output)
else:
print_report(analysis)
if __name__ == "__main__":
main()
FILE:scripts/pr_analyzer.py
#!/usr/bin/env python3
"""
PR Analyzer
Analyzes pull request changes for review complexity, risk assessment,
and generates review priorities.
Usage:
python pr_analyzer.py /path/to/repo
python pr_analyzer.py . --base main --head feature-branch
python pr_analyzer.py /path/to/repo --json
"""
import argparse
import json
import os
import re
import subprocess
import sys
from pathlib import Path
from typing import Dict, List, Optional, Tuple
# File categories for review prioritization
FILE_CATEGORIES = {
"critical": {
"patterns": [
r"auth", r"security", r"password", r"token", r"secret",
r"payment", r"billing", r"crypto", r"encrypt"
],
"weight": 5,
"description": "Security-sensitive files requiring careful review"
},
"high": {
"patterns": [
r"api", r"database", r"migration", r"schema", r"model",
r"config", r"env", r"middleware"
],
"weight": 4,
"description": "Core infrastructure files"
},
"medium": {
"patterns": [
r"service", r"controller", r"handler", r"util", r"helper"
],
"weight": 3,
"description": "Business logic files"
},
"low": {
"patterns": [
r"test", r"spec", r"mock", r"fixture", r"story",
r"readme", r"docs", r"\.md$"
],
"weight": 1,
"description": "Tests and documentation"
}
}
# Risky patterns to flag
RISK_PATTERNS = [
{
"name": "hardcoded_secrets",
"pattern": r"(password|secret|api_key|token|connection_?string)\s*[=:]\s*['\"][^'\"]+['\"]",
"severity": "critical",
"message": "Potential hardcoded secret or connection string detected"
},
{
"name": "todo_fixme",
"pattern": r"(TODO|FIXME|HACK|XXX):",
"severity": "low",
"message": "TODO/FIXME comment found"
},
{
"name": "console_log",
"pattern": (
r"console\.(log|debug|info|warn|error)\(|\bDebug\.WriteLine\(|"
r"\bSystem\.out\.print(?:ln)?\(|\.printStackTrace\("
),
"severity": "medium",
"message": (
"Debug output statement found "
"(console.* / Debug.WriteLine / System.out / printStackTrace)"
)
},
{
"name": "debugger",
"pattern": r"\bdebugger\b",
"severity": "high",
"message": "Debugger statement found"
},
{
"name": "analyzer_disable",
"pattern": (
r"eslint-disable|#pragma\s+warning\s+disable|\[SuppressMessage|"
r"@SuppressWarnings"
),
"severity": "medium",
"message": (
"Static-analyzer rule disabled "
"(ESLint / Roslyn / SuppressMessage / @SuppressWarnings)"
)
},
{
"name": "loose_type",
"pattern": r":\s*any\b|\bdynamic\s+\w+\s*[=;]",
"severity": "medium",
"message": "Loose type used (TypeScript 'any' or C# 'dynamic')"
},
{
"name": "sql_concatenation",
"pattern": r"(SELECT|INSERT|UPDATE|DELETE).*\+.*['\"]|(?:FromSql|ExecuteSql)\w*\([^)]*\$\"",
"severity": "critical",
"message": "Potential SQL injection (string concatenation or interpolation in query)"
},
{
"name": "csharp_unsafe_block",
"pattern": (
r"\bunsafe\s+(?:\{|public|private|protected|internal|static|sealed|"
r"partial|class|struct|void|int|string|long|short|byte|double|float|"
r"bool|char|ref|out|fixed)\b"
),
"severity": "high",
"message": "C# 'unsafe' code — requires memory-safety review"
},
{
"name": "csharp_null_forgiving",
"pattern": r"(?:\)\s*!\.|\w+!\.\w+)",
"severity": "medium",
"message": "Null-forgiving operator (!) used — verify the value is truly non-null"
},
{
"name": "csharp_async_void",
"pattern": r"\basync\s+void\s+\w+\s*\(",
"severity": "high",
"message": "'async void' method — use only for event handlers"
},
{
"name": "csharp_blocking_async",
"pattern": r"\.(?:Result\b|Wait\(\)|GetAwaiter\(\)\.GetResult\(\))",
"severity": "high",
"message": "Blocking call on async operation — can deadlock in ASP.NET contexts"
}
]
def run_git_command(cmd: List[str], cwd: Path) -> Tuple[bool, str]:
"""Run a git command and return success status and output."""
try:
result = subprocess.run(
cmd,
cwd=cwd,
capture_output=True,
text=True,
timeout=30
)
return result.returncode == 0, result.stdout.strip()
except subprocess.TimeoutExpired:
return False, "Command timed out"
except Exception as e:
return False, str(e)
def get_changed_files(repo_path: Path, base: str, head: str) -> List[Dict]:
"""Get list of changed files between two refs."""
success, output = run_git_command(
["git", "diff", "--name-status", f"{base}...{head}"],
repo_path
)
if not success:
# Try without the triple dot (for uncommitted changes)
success, output = run_git_command(
["git", "diff", "--name-status", base, head],
repo_path
)
if not success or not output:
# Fall back to staged changes
success, output = run_git_command(
["git", "diff", "--name-status", "--cached"],
repo_path
)
files = []
for line in output.split("\n"):
if not line.strip():
continue
parts = line.split("\t")
if len(parts) >= 2:
status = parts[0][0] # First character of status
filepath = parts[-1] # Handle renames (R100\told\tnew)
status_map = {
"A": "added",
"M": "modified",
"D": "deleted",
"R": "renamed",
"C": "copied"
}
files.append({
"path": filepath,
"status": status_map.get(status, "modified")
})
return files
def get_file_diff(repo_path: Path, filepath: str, base: str, head: str) -> str:
"""Get diff content for a specific file."""
success, output = run_git_command(
["git", "diff", f"{base}...{head}", "--", filepath],
repo_path
)
if not success:
success, output = run_git_command(
["git", "diff", "--cached", "--", filepath],
repo_path
)
return output if success else ""
def categorize_file(filepath: str) -> Tuple[str, int]:
"""Categorize a file based on its path and name."""
filepath_lower = filepath.lower()
for category, info in FILE_CATEGORIES.items():
for pattern in info["patterns"]:
if re.search(pattern, filepath_lower):
return category, info["weight"]
return "medium", 2 # Default category
def analyze_diff_for_risks(diff_content: str, filepath: str) -> List[Dict]:
"""Analyze diff content for risky patterns."""
risks = []
# Only analyze added lines (starting with +)
added_lines = [
line[1:] for line in diff_content.split("\n")
if line.startswith("+") and not line.startswith("+++")
]
content = "\n".join(added_lines)
for risk in RISK_PATTERNS:
matches = re.findall(risk["pattern"], content, re.IGNORECASE)
if matches:
risks.append({
"name": risk["name"],
"severity": risk["severity"],
"message": risk["message"],
"file": filepath,
"count": len(matches)
})
return risks
def count_changes(diff_content: str) -> Dict[str, int]:
"""Count additions and deletions in diff."""
additions = 0
deletions = 0
for line in diff_content.split("\n"):
if line.startswith("+") and not line.startswith("+++"):
additions += 1
elif line.startswith("-") and not line.startswith("---"):
deletions += 1
return {"additions": additions, "deletions": deletions}
def calculate_complexity_score(files: List[Dict], all_risks: List[Dict]) -> int:
"""Calculate overall PR complexity score (1-10)."""
score = 0
# File count contribution (max 3 points)
file_count = len(files)
if file_count > 20:
score += 3
elif file_count > 10:
score += 2
elif file_count > 5:
score += 1
# Total changes contribution (max 3 points)
total_changes = sum(f.get("additions", 0) + f.get("deletions", 0) for f in files)
if total_changes > 500:
score += 3
elif total_changes > 200:
score += 2
elif total_changes > 50:
score += 1
# Risk severity contribution (max 4 points)
critical_risks = sum(1 for r in all_risks if r["severity"] == "critical")
high_risks = sum(1 for r in all_risks if r["severity"] == "high")
score += min(2, critical_risks)
score += min(2, high_risks)
return min(10, max(1, score))
def analyze_commit_messages(repo_path: Path, base: str, head: str) -> Dict:
"""Analyze commit messages in the PR."""
success, output = run_git_command(
["git", "log", "--oneline", f"{base}...{head}"],
repo_path
)
if not success or not output:
return {"commits": 0, "issues": []}
commits = output.strip().split("\n")
issues = []
for commit in commits:
if len(commit) < 10:
continue
# Check for conventional commit format
message = commit[8:] if len(commit) > 8 else commit # Skip hash
if not re.match(r"^(feat|fix|docs|style|refactor|test|chore|perf|ci|build|revert)(\(.+\))?:", message):
issues.append({
"commit": commit[:7],
"issue": "Does not follow conventional commit format"
})
if len(message) > 72:
issues.append({
"commit": commit[:7],
"issue": "Commit message exceeds 72 characters"
})
return {
"commits": len(commits),
"issues": issues
}
def analyze_pr(
repo_path: Path,
base: str = "main",
head: str = "HEAD"
) -> Dict:
"""Perform complete PR analysis."""
# Get changed files
changed_files = get_changed_files(repo_path, base, head)
if not changed_files:
return {
"status": "no_changes",
"message": "No changes detected between branches"
}
# Analyze each file
all_risks = []
file_analyses = []
for file_info in changed_files:
filepath = file_info["path"]
category, weight = categorize_file(filepath)
# Get diff for the file
diff = get_file_diff(repo_path, filepath, base, head)
changes = count_changes(diff)
risks = analyze_diff_for_risks(diff, filepath)
all_risks.extend(risks)
file_analyses.append({
"path": filepath,
"status": file_info["status"],
"category": category,
"priority_weight": weight,
"additions": changes["additions"],
"deletions": changes["deletions"],
"risks": risks
})
# Sort by priority (highest first)
file_analyses.sort(key=lambda x: (-x["priority_weight"], x["path"]))
# Analyze commits
commit_analysis = analyze_commit_messages(repo_path, base, head)
# Calculate metrics
complexity = calculate_complexity_score(file_analyses, all_risks)
total_additions = sum(f["additions"] for f in file_analyses)
total_deletions = sum(f["deletions"] for f in file_analyses)
return {
"status": "analyzed",
"summary": {
"files_changed": len(file_analyses),
"total_additions": total_additions,
"total_deletions": total_deletions,
"complexity_score": complexity,
"complexity_label": get_complexity_label(complexity),
"commits": commit_analysis["commits"]
},
"risks": {
"critical": [r for r in all_risks if r["severity"] == "critical"],
"high": [r for r in all_risks if r["severity"] == "high"],
"medium": [r for r in all_risks if r["severity"] == "medium"],
"low": [r for r in all_risks if r["severity"] == "low"]
},
"files": file_analyses,
"commit_issues": commit_analysis["issues"],
"review_order": [f["path"] for f in file_analyses[:10]] # Top 10 priority files
}
def get_complexity_label(score: int) -> str:
"""Get human-readable complexity label."""
if score <= 2:
return "Simple"
elif score <= 4:
return "Moderate"
elif score <= 6:
return "Complex"
elif score <= 8:
return "Very Complex"
else:
return "Critical"
def print_report(analysis: Dict) -> None:
"""Print human-readable analysis report."""
if analysis["status"] == "no_changes":
print("No changes detected.")
return
summary = analysis["summary"]
risks = analysis["risks"]
print("=" * 60)
print("PR ANALYSIS REPORT")
print("=" * 60)
print(f"\nComplexity: {summary['complexity_score']}/10 ({summary['complexity_label']})")
print(f"Files Changed: {summary['files_changed']}")
print(f"Lines: +{summary['total_additions']} / -{summary['total_deletions']}")
print(f"Commits: {summary['commits']}")
# Risk summary
print("\n--- RISK SUMMARY ---")
print(f"Critical: {len(risks['critical'])}")
print(f"High: {len(risks['high'])}")
print(f"Medium: {len(risks['medium'])}")
print(f"Low: {len(risks['low'])}")
# Critical and high risks details
if risks["critical"]:
print("\n--- CRITICAL RISKS ---")
for risk in risks["critical"]:
print(f" [{risk['file']}] {risk['message']} (x{risk['count']})")
if risks["high"]:
print("\n--- HIGH RISKS ---")
for risk in risks["high"]:
print(f" [{risk['file']}] {risk['message']} (x{risk['count']})")
# Commit message issues
if analysis["commit_issues"]:
print("\n--- COMMIT MESSAGE ISSUES ---")
for issue in analysis["commit_issues"][:5]:
print(f" {issue['commit']}: {issue['issue']}")
# Review order
print("\n--- SUGGESTED REVIEW ORDER ---")
for i, filepath in enumerate(analysis["review_order"], 1):
file_info = next(f for f in analysis["files"] if f["path"] == filepath)
print(f" {i}. [{file_info['category'].upper()}] {filepath}")
print("\n" + "=" * 60)
def main():
parser = argparse.ArgumentParser(
description="Analyze pull request for review complexity and risks"
)
parser.add_argument(
"repo_path",
nargs="?",
default=".",
help="Path to git repository (default: current directory)"
)
parser.add_argument(
"--base", "-b",
default="main",
help="Base branch for comparison (default: main)"
)
parser.add_argument(
"--head",
default="HEAD",
help="Head branch/commit for comparison (default: HEAD)"
)
parser.add_argument(
"--json",
action="store_true",
help="Output in JSON format"
)
parser.add_argument(
"--output", "-o",
help="Write output to file"
)
args = parser.parse_args()
repo_path = Path(args.repo_path).resolve()
if not (repo_path / ".git").exists():
print(f"Error: {repo_path} is not a git repository", file=sys.stderr)
sys.exit(1)
analysis = analyze_pr(repo_path, args.base, args.head)
if args.json:
output = json.dumps(analysis, indent=2)
if args.output:
with open(args.output, "w") as f:
f.write(output)
print(f"Results written to {args.output}")
else:
print(output)
else:
print_report(analysis)
if __name__ == "__main__":
main()
FILE:scripts/review_report_generator.py
#!/usr/bin/env python3
"""
Review Report Generator
Generates comprehensive code review reports by combining PR analysis
and code quality findings into structured, actionable reports.
Usage:
python review_report_generator.py /path/to/repo
python review_report_generator.py . --pr-analysis pr_results.json --quality-analysis quality_results.json
python review_report_generator.py /path/to/repo --format markdown --output review.md
"""
import argparse
import json
import os
import subprocess
import sys
from datetime import datetime
from pathlib import Path
from typing import Dict, List, Optional, Tuple
# Severity weights for prioritization
SEVERITY_WEIGHTS = {
"critical": 100,
"high": 75,
"medium": 50,
"low": 25,
"info": 10
}
# Review verdict thresholds
VERDICT_THRESHOLDS = {
"approve": {"max_critical": 0, "max_high": 0, "max_score": 100},
"approve_with_suggestions": {"max_critical": 0, "max_high": 2, "max_score": 85},
"request_changes": {"max_critical": 0, "max_high": 5, "max_score": 70},
"block": {"max_critical": float("inf"), "max_high": float("inf"), "max_score": 0}
}
def load_json_file(filepath: str) -> Optional[Dict]:
"""Load JSON file if it exists."""
try:
with open(filepath, "r") as f:
return json.load(f)
except (FileNotFoundError, json.JSONDecodeError):
return None
def run_pr_analyzer(repo_path: Path) -> Dict:
"""Run pr_analyzer.py and return results."""
script_path = Path(__file__).parent / "pr_analyzer.py"
if not script_path.exists():
return {"status": "error", "message": "pr_analyzer.py not found"}
try:
result = subprocess.run(
[sys.executable, str(script_path), str(repo_path), "--json"],
capture_output=True,
text=True,
timeout=120
)
if result.returncode == 0:
return json.loads(result.stdout)
return {"status": "error", "message": result.stderr}
except Exception as e:
return {"status": "error", "message": str(e)}
def run_quality_checker(repo_path: Path) -> Dict:
"""Run code_quality_checker.py and return results."""
script_path = Path(__file__).parent / "code_quality_checker.py"
if not script_path.exists():
return {"status": "error", "message": "code_quality_checker.py not found"}
try:
result = subprocess.run(
[sys.executable, str(script_path), str(repo_path), "--json"],
capture_output=True,
text=True,
timeout=300
)
if result.returncode == 0:
return json.loads(result.stdout)
return {"status": "error", "message": result.stderr}
except Exception as e:
return {"status": "error", "message": str(e)}
def calculate_review_score(pr_analysis: Dict, quality_analysis: Dict) -> int:
"""Calculate overall review score (0-100)."""
score = 100
# Deduct for PR risks
if "risks" in pr_analysis:
risks = pr_analysis["risks"]
score -= len(risks.get("critical", [])) * 15
score -= len(risks.get("high", [])) * 10
score -= len(risks.get("medium", [])) * 5
score -= len(risks.get("low", [])) * 2
# Deduct for code quality issues
if "issues" in quality_analysis:
issues = quality_analysis["issues"]
score -= len([i for i in issues if i.get("severity") == "critical"]) * 12
score -= len([i for i in issues if i.get("severity") == "high"]) * 8
score -= len([i for i in issues if i.get("severity") == "medium"]) * 4
score -= len([i for i in issues if i.get("severity") == "low"]) * 1
# Deduct for complexity
if "summary" in pr_analysis:
complexity = pr_analysis["summary"].get("complexity_score", 0)
if complexity > 7:
score -= 10
elif complexity > 5:
score -= 5
return max(0, min(100, score))
def determine_verdict(score: int, critical_count: int, high_count: int) -> Tuple[str, str]:
"""Determine review verdict based on score and issue counts."""
if critical_count > 0:
return "block", "Critical issues must be resolved before merge"
if score >= 90 and high_count == 0:
return "approve", "Code meets quality standards"
if score >= 75 and high_count <= 2:
return "approve_with_suggestions", "Minor improvements recommended"
if score >= 50:
return "request_changes", "Several issues need to be addressed"
return "block", "Significant issues prevent approval"
def generate_findings_list(pr_analysis: Dict, quality_analysis: Dict) -> List[Dict]:
"""Combine and prioritize all findings."""
findings = []
# Add PR risk findings
if "risks" in pr_analysis:
for severity, items in pr_analysis["risks"].items():
for item in items:
findings.append({
"source": "pr_analysis",
"severity": severity,
"category": item.get("name", "unknown"),
"message": item.get("message", ""),
"file": item.get("file", ""),
"count": item.get("count", 1)
})
# Add code quality findings
if "issues" in quality_analysis:
for issue in quality_analysis["issues"]:
findings.append({
"source": "quality_analysis",
"severity": issue.get("severity", "medium"),
"category": issue.get("type", "unknown"),
"message": issue.get("message", ""),
"file": issue.get("file", ""),
"line": issue.get("line", 0)
})
# Sort by severity weight
findings.sort(
key=lambda x: -SEVERITY_WEIGHTS.get(x["severity"], 0)
)
return findings
def generate_action_items(findings: List[Dict]) -> List[Dict]:
"""Generate prioritized action items from findings."""
action_items = []
seen_categories = set()
for finding in findings:
category = finding["category"]
severity = finding["severity"]
# Group similar issues
if category in seen_categories and severity not in ["critical", "high"]:
continue
action = {
"priority": "P0" if severity == "critical" else "P1" if severity == "high" else "P2",
"action": get_action_for_category(category, finding),
"severity": severity,
"files_affected": [finding["file"]] if finding.get("file") else []
}
action_items.append(action)
seen_categories.add(category)
return action_items[:15] # Top 15 actions
def get_action_for_category(category: str, finding: Dict) -> str:
"""Get actionable recommendation for issue category."""
actions = {
"hardcoded_secrets": "Remove hardcoded credentials and use environment variables or a secrets manager",
"sql_concatenation": "Use parameterized queries to prevent SQL injection",
"debugger": "Remove debugger statements before merging",
"console_log": "Remove or replace console statements with proper logging",
"todo_fixme": "Address TODO/FIXME comments or create tracking issues",
"disable_eslint": "Address the underlying issue instead of disabling lint rules",
"any_type": "Replace 'any' types with proper type definitions",
"long_function": "Break down function into smaller, focused units",
"god_class": "Split class into smaller, single-responsibility classes",
"too_many_params": "Use parameter objects or builder pattern",
"deep_nesting": "Refactor using early returns, guard clauses, or extraction",
"high_complexity": "Reduce cyclomatic complexity through refactoring",
"missing_error_handling": "Add proper error handling and recovery logic",
"duplicate_code": "Extract duplicate code into shared functions",
"magic_numbers": "Replace magic numbers with named constants",
"large_file": "Consider splitting into multiple smaller modules"
}
return actions.get(category, f"Review and address: {finding.get('message', category)}")
def format_markdown_report(report: Dict) -> str:
"""Generate markdown-formatted report."""
lines = []
# Header
lines.append("# Code Review Report")
lines.append("")
lines.append(f"**Generated:** {report['metadata']['generated_at']}")
lines.append(f"**Repository:** {report['metadata']['repository']}")
lines.append("")
# Executive Summary
lines.append("## Executive Summary")
lines.append("")
summary = report["summary"]
verdict = summary["verdict"]
verdict_emoji = {
"approve": "✅",
"approve_with_suggestions": "✅",
"request_changes": "⚠️",
"block": "❌"
}.get(verdict, "❓")
lines.append(f"**Verdict:** {verdict_emoji} {verdict.upper().replace('_', ' ')}")
lines.append(f"**Score:** {summary['score']}/100")
lines.append(f"**Rationale:** {summary['rationale']}")
lines.append("")
# Issue Counts
lines.append("### Issue Summary")
lines.append("")
lines.append("| Severity | Count |")
lines.append("|----------|-------|")
for severity in ["critical", "high", "medium", "low"]:
count = summary["issue_counts"].get(severity, 0)
lines.append(f"| {severity.capitalize()} | {count} |")
lines.append("")
# PR Statistics (if available)
if "pr_summary" in report:
pr = report["pr_summary"]
lines.append("### Change Statistics")
lines.append("")
lines.append(f"- **Files Changed:** {pr.get('files_changed', 'N/A')}")
lines.append(f"- **Lines Added:** +{pr.get('total_additions', 0)}")
lines.append(f"- **Lines Removed:** -{pr.get('total_deletions', 0)}")
lines.append(f"- **Complexity:** {pr.get('complexity_label', 'N/A')}")
lines.append("")
# Action Items
if report.get("action_items"):
lines.append("## Action Items")
lines.append("")
for i, item in enumerate(report["action_items"], 1):
priority = item["priority"]
emoji = "🔴" if priority == "P0" else "🟠" if priority == "P1" else "🟡"
lines.append(f"{i}. {emoji} **[{priority}]** {item['action']}")
if item.get("files_affected"):
lines.append(f" - Files: {', '.join(item['files_affected'][:3])}")
lines.append("")
# Critical Findings
critical_findings = [f for f in report.get("findings", []) if f["severity"] == "critical"]
if critical_findings:
lines.append("## Critical Issues (Must Fix)")
lines.append("")
for finding in critical_findings:
lines.append(f"- **{finding['category']}** in `{finding.get('file', 'unknown')}`")
lines.append(f" - {finding['message']}")
lines.append("")
# High Priority Findings
high_findings = [f for f in report.get("findings", []) if f["severity"] == "high"]
if high_findings:
lines.append("## High Priority Issues")
lines.append("")
for finding in high_findings[:10]:
lines.append(f"- **{finding['category']}** in `{finding.get('file', 'unknown')}`")
lines.append(f" - {finding['message']}")
lines.append("")
# Review Order (if available)
if "review_order" in report:
lines.append("## Suggested Review Order")
lines.append("")
for i, filepath in enumerate(report["review_order"][:10], 1):
lines.append(f"{i}. `{filepath}`")
lines.append("")
# Footer
lines.append("---")
lines.append("*Generated by Code Reviewer*")
return "\n".join(lines)
def format_text_report(report: Dict) -> str:
"""Generate plain text report."""
lines = []
lines.append("=" * 60)
lines.append("CODE REVIEW REPORT")
lines.append("=" * 60)
lines.append("")
lines.append(f"Generated: {report['metadata']['generated_at']}")
lines.append(f"Repository: {report['metadata']['repository']}")
lines.append("")
summary = report["summary"]
verdict = summary["verdict"].upper().replace("_", " ")
lines.append(f"VERDICT: {verdict}")
lines.append(f"SCORE: {summary['score']}/100")
lines.append(f"RATIONALE: {summary['rationale']}")
lines.append("")
lines.append("--- ISSUE SUMMARY ---")
for severity in ["critical", "high", "medium", "low"]:
count = summary["issue_counts"].get(severity, 0)
lines.append(f" {severity.capitalize()}: {count}")
lines.append("")
if report.get("action_items"):
lines.append("--- ACTION ITEMS ---")
for i, item in enumerate(report["action_items"][:10], 1):
lines.append(f" {i}. [{item['priority']}] {item['action']}")
lines.append("")
critical = [f for f in report.get("findings", []) if f["severity"] == "critical"]
if critical:
lines.append("--- CRITICAL ISSUES ---")
for f in critical:
lines.append(f" [{f.get('file', 'unknown')}] {f['message']}")
lines.append("")
lines.append("=" * 60)
return "\n".join(lines)
def generate_report(
repo_path: Path,
pr_analysis: Optional[Dict] = None,
quality_analysis: Optional[Dict] = None
) -> Dict:
"""Generate comprehensive review report."""
# Run analyses if not provided
if pr_analysis is None:
pr_analysis = run_pr_analyzer(repo_path)
if quality_analysis is None:
quality_analysis = run_quality_checker(repo_path)
# Generate findings
findings = generate_findings_list(pr_analysis, quality_analysis)
# Count issues by severity
issue_counts = {
"critical": len([f for f in findings if f["severity"] == "critical"]),
"high": len([f for f in findings if f["severity"] == "high"]),
"medium": len([f for f in findings if f["severity"] == "medium"]),
"low": len([f for f in findings if f["severity"] == "low"])
}
# Calculate score and verdict
score = calculate_review_score(pr_analysis, quality_analysis)
verdict, rationale = determine_verdict(
score,
issue_counts["critical"],
issue_counts["high"]
)
# Generate action items
action_items = generate_action_items(findings)
# Build report
report = {
"metadata": {
"generated_at": datetime.now().isoformat(),
"repository": str(repo_path),
"version": "1.0.0"
},
"summary": {
"score": score,
"verdict": verdict,
"rationale": rationale,
"issue_counts": issue_counts
},
"findings": findings,
"action_items": action_items
}
# Add PR summary if available
if pr_analysis.get("status") == "analyzed":
report["pr_summary"] = pr_analysis.get("summary", {})
report["review_order"] = pr_analysis.get("review_order", [])
# Add quality summary if available
if quality_analysis.get("status") == "analyzed":
report["quality_summary"] = quality_analysis.get("summary", {})
return report
def main():
parser = argparse.ArgumentParser(
description="Generate comprehensive code review reports"
)
parser.add_argument(
"repo_path",
nargs="?",
default=".",
help="Path to repository (default: current directory)"
)
parser.add_argument(
"--pr-analysis",
help="Path to pre-computed PR analysis JSON"
)
parser.add_argument(
"--quality-analysis",
help="Path to pre-computed quality analysis JSON"
)
parser.add_argument(
"--format", "-f",
choices=["text", "markdown", "json"],
default="text",
help="Output format (default: text)"
)
parser.add_argument(
"--output", "-o",
help="Write output to file"
)
parser.add_argument(
"--json",
action="store_true",
help="Output as JSON (shortcut for --format json)"
)
args = parser.parse_args()
repo_path = Path(args.repo_path).resolve()
if not repo_path.exists():
print(f"Error: Path does not exist: {repo_path}", file=sys.stderr)
sys.exit(1)
# Load pre-computed analyses if provided
pr_analysis = None
quality_analysis = None
if args.pr_analysis:
pr_analysis = load_json_file(args.pr_analysis)
if not pr_analysis:
print(f"Warning: Could not load PR analysis from {args.pr_analysis}")
if args.quality_analysis:
quality_analysis = load_json_file(args.quality_analysis)
if not quality_analysis:
print(f"Warning: Could not load quality analysis from {args.quality_analysis}")
# Generate report
report = generate_report(repo_path, pr_analysis, quality_analysis)
# Format output
output_format = "json" if args.json else args.format
if output_format == "json":
output = json.dumps(report, indent=2)
elif output_format == "markdown":
output = format_markdown_report(report)
else:
output = format_text_report(report)
# Write or print output
if args.output:
with open(args.output, "w") as f:
f.write(output)
print(f"Report written to {args.output}")
else:
print(output)
if __name__ == "__main__":
main()
Gợi ý và chọn đúng lệnh, agent, skill trong Claude Code khi chưa rõ nên dùng công cụ nào.
---
name: "command-guide"
description: >
Claude Code Command Selection Guide - Automatically recommend and select the right
commands, agents, and skills in Claude Code.
Use when: (1) user is unsure which command or tool to use, (2) needs to decide which
agent/skill best fits the current task, (3) querying usage scenarios for /plan, /tdd,
/compact, /loop and other commands, (4) understanding when to invoke planner,
code-reviewer, build-error-resolver and other agents, (5) needs command cheat sheet
or decision flowchart.
Triggers: "which command to use", "which agent", "command selection", "how to use /plan",
"when to use /compact", "agent selection guide", "command cheat sheet", "skill recommendation".
---
# Claude Code Command Selection Guide
This skill helps you choose the most appropriate command, agent, or skill for different scenarios.
## Quick Decision Flowchart
```mermaid
graph TD
A[User Request] --> B{Request Type?}
B -->|New Feature| C[/plan]
B -->|Bug Fix| D[/tdd or build-error-resolver]
B -->|Code Review| E[/code-review or code-reviewer agent]
B -->|Testing| F[/e2e or tdd-guide agent]
B -->|Context Too Long| G[/compact]
B -->|Documentation| H[/docs or docs-lookup agent]
B -->|Looping Task| I[/loop]
B -->|Security Review| J[security-reviewer agent]
C --> K[planner agent]
D --> L{Build Failed?}
L -->|Yes| M[build-error-resolver]
L -->|No| N[tdd-guide]
E --> O[code-reviewer]
F --> P[e2e-runner]
```
## 1. Built-in Slash Commands
### Session Management Commands
| Command | Use Case | Example |
|---------|----------|---------|
| `/compact` | Context too long (>150K tokens), slow response, task phase transition | `/compact` or auto-trigger |
| `/clear` | Start fresh conversation, clear history | `/clear` |
| `/loop` | Periodic task execution, automated looping work | `/loop 5m check build status` |
| `/help` | View help, learn commands | `/help` |
| `/fast` | Need faster response (Opus 4.6 only) | `/fast` |
| `/model` | Switch model | `/model sonnet` |
### Development Workflow Commands
| Command | Use Case | Activation Timing |
|---------|----------|-------------------|
| `/plan` | Start new feature, architecture refactor, complex tasks | **Enter Plan Mode** |
| `/tdd` | Write tests, TDD development workflow | When test guidance needed |
| `/e2e` | E2E testing, critical user flow verification | When browser testing needed |
| `/code-review` | Code quality review | After writing code |
| `/build-fix` | Build failure, type errors | When build fails |
| `/learn` | Extract patterns from session, learning | Before session ends |
| `/skill-create` | Create new skill from git history | When repeating patterns found |
### Documentation & Query Commands
| Command | Use Case | Example |
|---------|----------|---------|
| `/docs` | Update project documentation | `/docs` |
| `/update-codemaps` | Update code maps | `/update-codemaps` |
| `/remember` | Save memory to memory system | `/remember user prefers concise output` |
| `/tasks` | View task list | `/tasks` |
---
## 2. Agents Selection
### Development Workflow Agents
| Agent | Trigger Condition | Purpose |
|-------|-------------------|---------|
| `planner` | Complex feature request, architectural decision | Create implementation plan |
| `architect` | System design, tech stack selection | Architecture analysis and decisions |
| `tdd-guide` | New feature, bug fix | TDD workflow guidance |
| `code-reviewer` | **Invoke immediately after writing code** | Code quality review |
| `security-reviewer` | Handling auth, API, sensitive data | Security vulnerability detection |
### Problem Solving Agents
| Agent | Trigger Condition | Purpose |
|-------|-------------------|---------|
| `build-error-resolver` | **Invoke immediately when build fails** | Fix build/type errors |
| `e2e-runner` | Critical user flows, before PR | E2E test execution |
| `refactor-cleaner` | Code maintenance, dead code cleanup | Dead code detection and cleanup |
| `doc-updater` | Update docs, codemaps | Documentation sync |
### Research & Exploration Agents
| Agent | Trigger Condition | Purpose |
|-------|-------------------|---------|
| `Explore` | Codebase exploration, file finding | Quick codebase exploration |
| `general-purpose` | Complex multi-step tasks | General task handling |
| `docs-lookup` | Query library/framework docs | Get latest API documentation |
---
## 3. Skills Selection
### Workflow Skills
| Skill | Trigger Timing | Purpose |
|-------|----------------|---------|
| `tdd-workflow` | Developing new feature/fixing bug | Complete TDD workflow guidance |
| `verification-loop` | After feature completion, before PR | Comprehensive verification (build/test/lint/security) |
| `strategic-compact` | Long session, context pressure | Guide when to manually `/compact` |
### Architecture & Pattern Skills
| Skill | Trigger Timing | Purpose |
|-------|----------------|---------|
| `frontend-patterns` | Frontend development | React/Next.js/Vue best practices |
| `backend-patterns` | Backend development | API/service architecture patterns |
| `api-design` | API design | RESTful/API design standards |
| `mcp-server-patterns` | MCP server development | MCP configuration and patterns |
### Testing Skills
| Skill | Trigger Timing | Purpose |
|-------|----------------|---------|
| `e2e-testing` | E2E testing needs | Playwright test generation |
| `security-review` | Security review needs | OWASP Top 10 detection |
### Research Skills
| Skill | Trigger Timing | Purpose |
|-------|----------------|---------|
| `deep-research` | Need deep research | Multi-round search and research |
| `exa-search` | Need web search | Web content search |
| `documentation-lookup` | Query library docs | Context7 documentation query |
---
## 4. Scenario Decision Matrix
### By Task Phase
| Phase | Recommended Tool Combination | Reason |
|-------|------------------------------|--------|
| **Requirements Analysis** | `planner` + `Explore` | Plan first, explore later |
| **Architecture Design** | `architect` + `api-design` skill | Professional architecture guidance |
| **Pre-Development** | `tdd-guide` + `tdd-workflow` skill | Test first |
| **During Development** | Direct edit + quick iteration | Stay in flow |
| **Post-Development** | `code-reviewer` + `verification-loop` | Quality gate |
| **Testing Phase** | `e2e-runner` + `e2e-testing` skill | Complete test coverage |
| **Before PR** | `security-reviewer` + `verification-loop` | Final verification |
| **Build Failure** | `build-error-resolver` | Focused fix |
### By Problem Type
| Problem | Invoke Immediately | Note |
|---------|--------------------|------|
| Build failure | `build-error-resolver` | Minimal changes, quick fix |
| Type error | `build-error-resolver` | TypeScript specialist |
| Bug fix | `tdd-guide` | Write test then fix |
| Security vulnerability | `security-reviewer` | OWASP detection |
| Poor code quality | `code-reviewer` | Immediate review |
| Missing documentation | `doc-updater` | Auto update |
| Dead code | `refactor-cleaner` | Safe cleanup |
### By Development Type
| Development Type | Skills Combination |
|------------------|--------------------|
| Frontend feature | `frontend-patterns` + `tdd-workflow` |
| Backend API | `backend-patterns` + `api-design` + `tdd-workflow` |
| MCP server | `mcp-server-patterns` + `tdd-workflow` |
| Database | `database-reviewer` agent |
| Security feature | `security-reviewer` + `security-review` skill |
---
## 5. Parallel Execution Strategy
### Parallelizable Scenarios
Recommended: Launch multiple independent tasks simultaneously
Scenario: Preparing PR after code completion
- Agent 1: code-reviewer (code quality)
- Agent 2: security-reviewer (security review)
- Agent 3: e2e-runner (E2E tests)
Scenario: Large refactor analysis
- Agent 1: architect (architecture analysis)
- Agent 2: Explore (code exploration)
- Agent 3: refactor-cleaner (dead code detection)
### Sequential Execution Required
Cannot parallelize: Dependencies exist
Scenario: Fixing build error
- Sequence: build-error-resolver -> test verification -> code-reviewer
Scenario: New feature development
- Sequence: planner -> tdd-guide (write tests) -> implementation -> code-reviewer
---
## 6. Auto-Trigger Rules
### Invoke Without User Request
| Situation | Auto Action |
|-----------|-------------|
| Code written/modified | **Immediately invoke** `code-reviewer` |
| Build fails | **Immediately invoke** `build-error-resolver` |
| Complex feature request | **Immediately invoke** `planner` |
| Handling auth/sensitive data | **Immediately invoke** `security-reviewer` |
| New feature/bug fix | **Immediately invoke** `tdd-guide` |
| Architectural decision | **Immediately invoke** `architect` |
---
## 7. Context Management Timing
| Indicator | Trigger `/compact` |
|-----------|-------------------|
| Token > 150K | Immediately compact |
| Slow response | Suggest compact |
| Task phase switch | Compact at boundary |
| Major milestone completed | Compact then continue |
| Debugging ends -> new task | Clear debug traces |
**Best Practices**:
- Compact after research, before implementation (preserve plan)
- Compact after milestone completion (clear intermediate state)
- Don't compact mid-implementation (lose variables/paths)
---
## 8. Command Cheat Sheet
```
Development Workflow:
/plan -> Enter planning mode (complex tasks)
/tdd -> TDD workflow
/e2e -> E2E testing
/code-review -> Code review
/build-fix -> Fix build
Session Management:
/compact -> Compact context
/clear -> Clear session
/loop -> Looping task
/fast -> Fast mode
Documentation & Memory:
/docs -> Update docs
/remember -> Save memory
/tasks -> View tasks
Help:
/help -> View all commands
```
---
## 9. Usage Examples
### Example 1: New Feature Development
User: Add user authentication feature
Workflow:
1. /plan -> planner agent creates plan
2. tdd-guide -> write tests
3. Implementation -> edit code
4. code-reviewer -> code review
5. security-reviewer -> security review (auth sensitive)
6. e2e-runner -> E2E tests
7. /compact -> compact after milestone completion
### Example 2: Build Failure
User: npm run build failed
Workflow:
1. build-error-resolver -> analyze error, minimal fix
2. Verify build success
3. code-reviewer -> check fix quality
### Example 3: Code Refactoring
User: Refactor authentication module
Workflow:
1. architect -> architecture analysis
2. planner -> implementation plan
3. refactor-cleaner -> dead code detection
4. tdd-guide -> ensure test coverage
5. Implementation -> refactor code
6. verification-loop -> comprehensive verification
---
**Core Principles**:
1. **Plan first, implement later** - Use `/plan` for complex tasks
2. **Test first** - Use `tdd-guide` for new features
3. **Review immediately after coding** - Use `code-reviewer` when code complete
4. **Fix build immediately when failed** - Use `build-error-resolver`
5. **Review sensitive code** - Use `security-reviewer` for auth/API
6. **Verify comprehensively before PR** - Use `verification-loop`
Phân tích sản phẩm đối thủ từ trang giá, đánh giá ứng dụng, tin tuyển dụng, SEO, tạo ma trận tính năng và SWOT.
---
name: "competitive-teardown"
description: "Analyzes competitor products and companies by synthesizing data from pricing pages, app store reviews, job postings, SEO signals, and social media into structured competitive intelligence. Produces feature comparison matrices scored across 12 dimensions, SWOT analyses, positioning maps, UX audits, pricing model breakdowns, action item roadmaps, and stakeholder presentation templates. Use when conducting competitor analysis, comparing products against competitors, researching the competitive landscape, building battle cards for sales, preparing for a product strategy or roadmap session, responding to a competitor's new feature or pricing change, or performing a quarterly competitive review."
---
# Competitive Teardown
**Tier:** POWERFUL
**Category:** Product Team
**Domain:** Competitive Intelligence, Product Strategy, Market Analysis
---
## When to Use
- Before a product strategy or roadmap session
- When a competitor launches a major feature or pricing change
- Quarterly competitive review
- Before a sales pitch where you need battle card data
- When entering a new market segment
---
## Teardown Workflow
Follow these steps in sequence to produce a complete teardown:
1. **Define competitors** — List 2–4 competitors to analyze. Confirm which is the primary focus.
2. **Collect data** — Use `references/data-collection-guide.md` to gather raw signals from at least 3 sources per competitor (website, reviews, job postings, SEO, social).
_Validation checkpoint: Before proceeding, confirm you have pricing data, at least 20 reviews, and job posting counts for each competitor._
3. **Score using rubric** — Apply the 12-dimension rubric below to produce a numeric scorecard for each competitor and your own product.
_Validation checkpoint: Every dimension should have a score and at least one supporting evidence note._
4. **Generate outputs** — Populate the templates in `references/analysis-templates.md` (Feature Matrix, Pricing Analysis, SWOT, Positioning Map, UX Audit).
5. **Build action plan** — Translate findings into the Action Items template (quick wins / medium-term / strategic).
6. **Package for stakeholders** — Assemble the Stakeholder Presentation using outputs from steps 3–5.
---
## Data Collection Guide
> Full executable scripts for each source are in `references/data-collection-guide.md`. Summaries of what to capture are below.
### 1. Website Analysis
Key things to capture:
- Pricing tiers and price points
- Feature lists per tier
- Primary CTA and messaging
- Case studies / customer logos (signals ICP)
- Integration logos
- Trust signals (certifications, compliance badges)
### 2. App Store Reviews
Review sentiment categories:
- **Praise** → what users love (defend / strengthen these)
- **Feature requests** → unmet needs (opportunity gaps)
- **Bugs** → quality signals
- **UX complaints** → friction points you can beat them on
**Sample App Store query (iTunes Search API):**
```
GET https://itunes.apple.com/search?term=<competitor_name>&entity=software&limit=1
# Extract trackId, then:
GET https://itunes.apple.com/rss/customerreviews/id=<trackId>/sortBy=mostRecent/json?l=en&limit=50
```
Parse `entry[].content.label` for review text and `entry[].im:rating.label` for star rating.
### 3. Job Postings (Team Size & Tech Stack Signals)
Signals from job postings:
- **Engineering volume** → scaling vs. consolidating
- **Specific tech mentions** → stack (React/Vue, Postgres/Mongo, AWS/GCP)
- **Sales/CS ratio** → product-led vs. sales-led motion
- **Data/ML roles** → upcoming AI features
- **Compliance roles** → regulatory expansion
### 4. SEO Analysis
SEO signals to capture:
- Top 20 organic keywords (intent: informational / navigational / commercial)
- Domain Authority / backlink count
- Blog publishing cadence and topics
- Which pages rank (product pages vs. blog vs. docs)
### 5. Social Media Sentiment
Capture recent mentions via Twitter/X API v2, Reddit, or LinkedIn. Look for recurring praise, complaints, and feature requests. See `references/data-collection-guide.md` for API query examples.
---
## Scoring Rubric (12 Dimensions, 1-5)
| # | Dimension | 1 (Weak) | 3 (Average) | 5 (Best-in-class) |
|---|-----------|----------|-------------|-------------------|
| 1 | **Features** | Core only, many gaps | Solid coverage | Comprehensive + unique |
| 2 | **Pricing** | Confusing / overpriced | Market-rate, clear | Transparent, flexible, fair |
| 3 | **UX** | Confusing, high friction | Functional | Delightful, minimal friction |
| 4 | **Performance** | Slow, unreliable | Acceptable | Fast, high uptime |
| 5 | **Docs** | Sparse, outdated | Decent coverage | Comprehensive, searchable |
| 6 | **Support** | Email only, slow | Chat + email | 24/7, great response |
| 7 | **Integrations** | 0-5 integrations | 6-25 | 26+ or deep ecosystem |
| 8 | **Security** | No mentions | SOC2 claimed | SOC2 Type II, ISO 27001 |
| 9 | **Scalability** | No enterprise tier | Mid-market ready | Enterprise-grade |
| 10 | **Brand** | Generic, unmemorable | Decent positioning | Strong, differentiated |
| 11 | **Community** | None | Forum / Slack | Active, vibrant community |
| 12 | **Innovation** | No recent releases | Quarterly | Frequent, meaningful |
**Example completed row** (Competitor: Acme Corp, Dimension 3 – UX):
| Dimension | Acme Corp Score | Evidence |
|-----------|----------------|---------|
| UX | 2 | App Store reviews cite "confusing navigation" (38 mentions); onboarding requires 7 steps before TTFV; no onboarding wizard; CC required at signup. |
Apply this pattern to all 12 dimensions for each competitor.
---
## Templates
> Full template markdown is in `references/analysis-templates.md`. Abbreviated reference below.
### Feature Comparison Matrix
Rows: core features, pricing tiers, platform capabilities (web, iOS, Android, API).
Columns: your product + up to 3 competitors.
Score each cell 1–5. Sum to get total out of 60.
**Score legend:** 5=Best-in-class, 4=Strong, 3=Average, 2=Below average, 1=Weak/Missing
### Pricing Analysis
Capture per competitor: model type (per-seat / usage-based / flat rate / freemium), entry/mid/enterprise price points, free trial length.
Summarize: price leader, value leader, premium positioning, your position, and 2–3 pricing opportunity bullets.
### SWOT Analysis
For each competitor: 3–5 bullets per quadrant (Strengths, Weaknesses, Opportunities for us, Threats to us). Anchor every bullet to a data signal (review quote, job posting count, pricing page, etc.).
### Positioning Map
2x2 axes (e.g., Simple ↔ Complex / Low Value ↔ High Value). Place each competitor and your product. Bubble size = market share or funding. See `references/analysis-templates.md` for ASCII and editable versions.
### UX Audit Checklist
Onboarding: TTFV (minutes), steps to activation, CC-required, onboarding wizard quality.
Key workflows: steps, friction points, comparative score (yours vs. theirs).
Mobile: iOS/Android ratings, feature parity, top complaint and praise.
Navigation: global search, keyboard shortcuts, in-app help.
### Action Items
| Horizon | Effort | Examples |
|---------|--------|---------|
| Quick wins (0–4 wks) | Low | Add review badges, publish comparison landing page |
| Medium-term (1–3 mo) | Moderate | Launch free tier, improve onboarding TTFV, add top-requested integration |
| Strategic (3–12 mo) | High | Enter new market, build API v2, achieve SOC2 Type II |
### Stakeholder Presentation (7 slides)
1. **Executive Summary** — Threat level (LOW/MEDIUM/HIGH/CRITICAL), top strength, top opportunity, recommended action
2. **Market Position** — 2x2 positioning map
3. **Feature Scorecard** — 12-dimension radar or table, total scores
4. **Pricing Analysis** — Comparison table + key insight
5. **UX Highlights** — What they do better (3 bullets) vs. where we win (3 bullets)
6. **Voice of Customer** — Top 3 review complaints (quoted or paraphrased)
7. **Our Action Plan** — Quick wins, medium-term, strategic priorities; Appendix with raw data
## Related Skills
- **Product Strategist** (`product-team/product-strategist/`) — Competitive insights feed OKR and strategy planning
- **Landing Page Generator** (`product-team/landing-page-generator/`) — Competitive positioning informs landing page messaging
FILE:references/analysis-templates.md
# Competitive Analysis Templates
## 1. SWOT Analysis Template
### Company/Product: [Competitor Name]
**Date:** [Analysis Date] | **Analyst:** [Name] | **Version:** [1.0]
#### Strengths (Internal Advantages)
| # | Strength | Evidence | Impact |
|---|----------|----------|--------|
| 1 | [e.g., Strong brand recognition] | [Source/data point] | High/Med/Low |
| 2 | | | |
| 3 | | | |
#### Weaknesses (Internal Limitations)
| # | Weakness | Evidence | Exploitability |
|---|----------|----------|---------------|
| 1 | [e.g., Limited API capabilities] | [Source/data point] | High/Med/Low |
| 2 | | | |
| 3 | | | |
#### Opportunities (External Favorable)
| # | Opportunity | Timeframe | Our Advantage |
|---|------------|-----------|---------------|
| 1 | [e.g., Competitor slow to adopt AI] | Short/Med/Long | [How we capitalize] |
| 2 | | | |
| 3 | | | |
#### Threats (External Unfavorable)
| # | Threat | Likelihood | Mitigation |
|---|--------|-----------|-----------|
| 1 | [e.g., Competitor acquired by larger company] | High/Med/Low | [Our response plan] |
| 2 | | | |
| 3 | | | |
---
## 2. Porter's Five Forces (Product Application)
### Market: [Your Product Category]
#### Force 1: Competitive Rivalry (Intensity: High/Med/Low)
- Number of direct competitors: ___
- Market growth rate: ___% annually
- Product differentiation level: High/Med/Low
- Switching costs for customers: High/Med/Low
- Exit barriers: High/Med/Low
- **Assessment:** [Summary of competitive rivalry intensity]
#### Force 2: Threat of New Entrants (Intensity: High/Med/Low)
- Capital requirements: High/Med/Low
- Technology barriers: High/Med/Low
- Network effects strength: Strong/Moderate/Weak
- Regulatory barriers: High/Med/Low
- Brand loyalty in market: Strong/Moderate/Weak
- **Assessment:** [Summary of new entrant threat]
#### Force 3: Threat of Substitutes (Intensity: High/Med/Low)
- Alternative solutions: [List substitutes]
- Price-performance of substitutes: Better/Same/Worse
- Switching costs to substitutes: High/Med/Low
- Customer propensity to switch: High/Med/Low
- **Assessment:** [Summary of substitute threat]
#### Force 4: Bargaining Power of Buyers (Power: High/Med/Low)
- Buyer concentration: Concentrated/Fragmented
- Price sensitivity: High/Med/Low
- Information availability: Full/Partial/Limited
- Switching costs: High/Med/Low
- Volume of purchases: High/Med/Low
- **Assessment:** [Summary of buyer power]
#### Force 5: Bargaining Power of Suppliers (Power: High/Med/Low)
- Key technology dependencies: [List]
- Cloud provider lock-in: High/Med/Low
- Talent market tightness: Tight/Balanced/Loose
- Data source dependencies: Critical/Important/Optional
- **Assessment:** [Summary of supplier power]
#### Overall Industry Attractiveness: [Score 1-10]
---
## 3. Competitive Positioning Map
### Axis Definitions
- **X-Axis:** [e.g., Ease of Use] (Low to High)
- **Y-Axis:** [e.g., Feature Completeness] (Low to High)
### Competitor Positions
| Competitor | X Score (1-10) | Y Score (1-10) | Quadrant |
|-----------|---------------|---------------|----------|
| Your Product | ___ | ___ | ___ |
| Competitor A | ___ | ___ | ___ |
| Competitor B | ___ | ___ | ___ |
| Competitor C | ___ | ___ | ___ |
| Competitor D | ___ | ___ | ___ |
### Quadrant Definitions
- **Top-Right (Leaders):** High on both axes - market leaders
- **Top-Left (Feature-Rich):** High features, lower ease of use - complex tools
- **Bottom-Right (Simple):** Easy to use, fewer features - niche players
- **Bottom-Left (Laggards):** Low on both axes - disruption candidates
### Positioning Insights
- **White space opportunities:** [Areas with no competitor presence]
- **Crowded areas:** [Where competition is fiercest]
- **Our trajectory:** [Direction we're moving on the map]
---
## 4. Win/Loss Analysis Template
### Deal: [Opportunity Name]
**Date:** [Close Date] | **Result:** Won / Lost | **Competitor:** [Name]
#### Deal Context
- **Deal Size:** $___
- **Sales Cycle:** ___ days
- **Segment:** SMB / Mid-Market / Enterprise
- **Industry:** ___
- **Decision Makers:** [Roles involved]
- **Evaluation Criteria:** [What mattered most to buyer]
#### Competitive Comparison (Buyer Perspective)
| Factor | Us (Score 1-5) | Competitor (Score 1-5) | Decisive? |
|--------|---------------|----------------------|-----------|
| Product Fit | | | Yes/No |
| Pricing | | | Yes/No |
| Ease of Use | | | Yes/No |
| Support Quality | | | Yes/No |
| Integration | | | Yes/No |
| Brand/Trust | | | Yes/No |
| Implementation | | | Yes/No |
#### Win/Loss Factors
- **Primary reason for outcome:** [Single most important factor]
- **Secondary factors:** [Supporting reasons]
- **Buyer quotes:** ["Direct quotes from debrief"]
#### Action Items
| # | Action | Owner | Due Date |
|---|--------|-------|----------|
| 1 | [e.g., Improve onboarding flow] | [Name] | [Date] |
| 2 | | | |
---
## 5. Battle Card Template
### Competitor: [Name]
**Last Updated:** [Date] | **Confidence:** High/Med/Low
#### Quick Facts
- **Founded:** ___
- **Funding:** $___
- **Employees:** ___
- **Customers:** ___
- **HQ:** ___
#### Elevator Pitch (Their Positioning)
> [How the competitor describes themselves in one sentence]
#### Our Positioning Against Them
> [How we differentiate - our one-liner against this competitor]
#### Where They Win
| Strength | Our Counter |
|----------|------------|
| [e.g., Lower price point] | [e.g., Emphasize TCO including implementation costs] |
| [e.g., Larger integration marketplace] | [e.g., Highlight quality over quantity, key integrations] |
| | |
#### Where We Win
| Our Strength | Evidence |
|-------------|----------|
| [e.g., Superior onboarding experience] | [Metric or customer quote] |
| [e.g., Better enterprise security] | [Certification or feature] |
| | |
#### Landmines to Set
Questions to ask prospects that expose competitor weaknesses:
1. "Have you evaluated how [specific capability] scales beyond [threshold]?"
2. "What's their approach to [area where competitor is weak]?"
3. "Can you share their uptime SLA and historical performance?"
#### Objection Handling
| Objection | Response |
|-----------|----------|
| "[Competitor] is cheaper" | [Value-based response] |
| "[Competitor] has more features" | [Quality/relevance response] |
| "We already use [Competitor]" | [Migration/coexistence story] |
#### Trap Questions They Set
Questions competitors ask about us, and how to respond:
1. **Q:** "[Our known weakness]?" **A:** [Honest, redirect response]
2. **Q:** "[Feature gap]?" **A:** [Roadmap or alternative approach]
#### Recent Intel
- [Date]: [Notable change - pricing, feature, hire, funding]
- [Date]: [Notable change]
FILE:references/competitive-analysis-frameworks.md
# Competitive Analysis Frameworks
This reference provides practical frameworks for evaluating competitors and positioning decisions.
## Porter's Five Forces
Assess the competitive intensity of your market:
1. Threat of new entrants
- Barriers to entry (capital, regulation, network effects)
- Speed of competitor replication
2. Bargaining power of suppliers
- Dependency on core infrastructure vendors
- Concentration of key technical providers
3. Bargaining power of buyers
- Customer switching costs
- Procurement complexity and contract leverage
4. Threat of substitutes
- Adjacent alternatives solving the same job
- DIY and internal build options
5. Rivalry among existing competitors
- Number and similarity of competitors
- Price competition and differentiation pressure
### Five Forces Template
| Force | Current Pressure (Low/Med/High) | Evidence | Strategic Response |
|---|---|---|---|
| New Entrants | | | |
| Supplier Power | | | |
| Buyer Power | | | |
| Substitutes | | | |
| Rivalry | | | |
## SWOT Analysis
Use SWOT to map internal and external context quickly.
### SWOT Template
| Strengths (Internal) | Weaknesses (Internal) |
|---|---|
| What we do better than alternatives | Where competitors outperform us |
| Unique capabilities or assets | Known product or go-to-market gaps |
| Opportunities (External) | Threats (External) |
|---|---|
| Market trends we can exploit | Competitor moves or macro risks |
| Unserved segments and use cases | Regulatory, platform, or pricing pressure |
### SWOT Quality Checklist
- Base every point on evidence, not assumptions.
- Separate observations from conclusions.
- Prioritize top 3 items per quadrant.
## Feature Comparison Matrix
Compare products on meaningful buying criteria, not vanity features.
### Feature Matrix Template
| Dimension | Weight | Your Product | Competitor A | Competitor B | Notes |
|---|---:|---:|---:|---:|---|
| Core workflow coverage | 25% | | | | |
| Ease of implementation | 15% | | | | |
| Performance / reliability | 15% | | | | |
| Integrations / ecosystem | 15% | | | | |
| Security / compliance | 15% | | | | |
| Pricing / TCO | 15% | | | | |
Scoring scale recommendation: 1-5 (weak to strong).
## Competitive Positioning Map
Create a 2-axis map showing market whitespace and crowding.
### Positioning Map Steps
1. Select two high-signal dimensions customers care about.
2. Place each competitor based on evidence (pricing pages, reviews, demos).
3. Mark clusters where products are undifferentiated.
4. Identify white space where demand exists but options are weak.
Example axes:
- X-axis: Ease of use
- Y-axis: Enterprise readiness
## Blue Ocean Strategy Canvas
Use a strategy canvas to decide where to raise, reduce, eliminate, or create factors.
### ERRC Grid (Eliminate-Reduce-Raise-Create)
| Eliminate | Reduce | Raise | Create |
|---|---|---|---|
| Commodity table-stakes not valued by target users | Costly features with weak adoption | Differentiators tied to target job-to-be-done | New value dimensions competitors ignore |
### Strategy Canvas Checklist
- Compare value curves between your product and top competitors.
- Ensure target segment is explicit.
- Tie every strategic choice to measurable outcome.
FILE:references/data-collection-guide.md
# Competitive Data Collection Guide
## Overview
This guide outlines systematic approaches for gathering competitive intelligence from publicly available sources. All methods described here are ethical and rely on information that competitors have made publicly accessible.
## Public Data Sources
### Review Platforms
- **G2**: Enterprise software reviews, feature comparisons, satisfaction scores
- **Capterra**: SMB-focused reviews, pricing transparency, deployment details
- **TrustRadius**: In-depth reviews with verified users, TrustMaps
- **Product Hunt**: Launch positioning, early adopter sentiment, feature highlights
- **App Store / Google Play**: Mobile app ratings, review themes, update frequency
### Company Publications
- **Pricing Pages**: Tier structure, feature gating, enterprise vs self-serve
- **Changelogs / Release Notes**: Development velocity, feature priorities, tech direction
- **Blog Posts**: Strategic messaging, thought leadership topics, market positioning
- **Case Studies**: Target customer profiles, value propositions, success metrics
- **Help Documentation**: Feature depth, API capabilities, integration ecosystem
### Talent & Organization Signals
- **Job Postings**: Technology stack, team growth areas, strategic initiatives
- **LinkedIn**: Team size, org structure, key hires, department ratios
- **Glassdoor**: Company culture, internal challenges, growth trajectory
### Financial & Legal
- **Patent Filings**: Innovation direction, defensive IP, technology differentiation
- **SEC Filings (public companies)**: Revenue, growth rate, customer count, churn
- **Crunchbase / PitchBook**: Funding rounds, investors, valuation trends
### Technical Intelligence
- **BuiltWith / Wappalyzer**: Technology stack detection
- **GitHub**: Open-source contributions, SDK quality, developer engagement
- **API Documentation**: Integration capabilities, rate limits, data models
- **Status Pages**: Uptime history, incident frequency, infrastructure maturity
## Data Points to Collect Per Competitor
### Product
- Core features and capabilities (feature-by-feature matrix)
- Unique differentiators and proprietary technology
- Platform support (web, mobile, desktop, API)
- Integration ecosystem (number and quality of integrations)
- Performance benchmarks (if available from reviews)
### Business
- Pricing tiers and per-seat/usage costs
- Target customer segments (SMB, mid-market, enterprise)
- Estimated customer count and notable logos
- Geographic focus and localization
- Go-to-market model (PLG, sales-led, hybrid)
### Team & Technology
- Estimated team size and engineering ratio
- Technology stack and infrastructure choices
- Development velocity (release frequency)
- Open-source involvement and developer relations
### Market Position
- Market share estimates
- Brand perception and NPS (from reviews)
- Analyst coverage (Gartner, Forrester positioning)
- Partnership and channel strategy
## Ethical Guidelines
1. **Use only public information** - Never access private systems, NDA-protected content, or internal documents
2. **No deception** - Do not misrepresent yourself to obtain information (e.g., fake sales inquiries)
3. **Respect terms of service** - Follow scraping policies and API usage terms
4. **Attribute sources** - Document where each data point came from for verification
5. **No employee poaching for intelligence** - Hiring decisions should be talent-driven, not intelligence-driven
6. **Legal compliance** - Ensure data collection complies with local regulations
## Update Cadence Recommendations
| Data Type | Frequency | Trigger Events |
|-----------|-----------|---------------|
| Pricing | Monthly | Competitor pricing page changes |
| Features | Bi-weekly | Changelog updates, product launches |
| Reviews | Monthly | Batch review analysis |
| Job Postings | Monthly | Hiring surge detection |
| Financials | Quarterly | Earnings reports, funding rounds |
| Tech Stack | Quarterly | Major platform changes |
| Full Teardown | Quarterly | Strategic planning cycles |
## Collection Workflow
1. **Set up monitoring** - Google Alerts, competitor RSS feeds, social listening
2. **Schedule regular sweeps** - Calendar recurring data collection tasks
3. **Centralize data** - Use a shared competitive intelligence database or spreadsheet
4. **Validate findings** - Cross-reference multiple sources for accuracy
5. **Tag and categorize** - Apply consistent taxonomy for easy retrieval
6. **Share insights** - Distribute relevant findings to product, sales, and marketing teams
7. **Archive versions** - Maintain historical snapshots for trend analysis
## Tools for Automation
- **Google Alerts**: Free monitoring for competitor mentions
- **Visualping**: Website change detection (pricing pages, feature pages)
- **Feedly**: RSS aggregation for competitor blogs and news
- **SimilarWeb**: Traffic estimates and audience overlap
- **SEMrush / Ahrefs**: SEO positioning and content strategy analysis
FILE:references/scoring-rubric.md
# Competitive Scoring Rubric
## Overview
This rubric provides a standardized framework for evaluating competitors across key dimensions. Consistent scoring enables meaningful comparisons and tracks competitive position changes over time.
## Scoring Scale (1-10)
| Score | Label | Definition |
|-------|-------|-----------|
| 1-2 | Poor | Significant gaps, major usability issues, or missing capability |
| 3-4 | Below Average | Basic functionality with notable limitations |
| 5-6 | Average | Meets market expectations, no standout qualities |
| 7-8 | Above Average | Strong execution with clear advantages |
| 9-10 | Exceptional | Industry-leading, sets the standard for others |
## Dimension Categories
### 1. User Experience (UX) - Weight: 20%
- **Onboarding**: Time to first value, setup complexity, guided flows
- **Navigation**: Information architecture, discoverability, consistency
- **Visual Design**: Modern aesthetics, brand coherence, accessibility
- **Performance**: Page load times, responsiveness, offline capability
- **Mobile Experience**: Native app quality, responsive design, feature parity
### 2. Feature Completeness - Weight: 25%
- **Core Features**: Coverage of essential use cases
- **Advanced Features**: Power user capabilities, automation, customization
- **Workflow Support**: End-to-end process coverage without workarounds
- **API & Extensibility**: API coverage, webhook support, SDK quality
- **Innovation**: Unique capabilities not found in competitors
### 3. Pricing & Value - Weight: 15%
- **Transparency**: Clear pricing without hidden costs
- **Flexibility**: Plan options matching different customer sizes
- **Value-to-Cost Ratio**: Feature access relative to price point
- **Free Tier / Trial**: Quality of free offering for evaluation
- **Contract Terms**: Lock-in requirements, cancellation ease
### 4. Integrations - Weight: 10%
- **Native Integrations**: Number and quality of built-in connectors
- **Marketplace**: Third-party app ecosystem breadth
- **API Quality**: Documentation, reliability, rate limits
- **Data Import/Export**: Migration ease, format support
- **Workflow Automation**: Zapier, Make, native automation support
### 5. Support & Documentation - Weight: 10%
- **Documentation Quality**: Completeness, searchability, freshness
- **Support Channels**: Chat, email, phone, community availability
- **Response Time**: SLA adherence, resolution speed
- **Self-Service**: Knowledge base, video tutorials, community forums
- **Onboarding Support**: Dedicated CSM, implementation assistance
### 6. Performance & Reliability - Weight: 10%
- **Uptime**: Historical availability, SLA commitments
- **Speed**: Application responsiveness under normal load
- **Scalability**: Performance at high volume, enterprise readiness
- **Data Handling**: Large dataset support, bulk operations
- **Global Performance**: CDN, regional deployments, latency
### 7. Security & Compliance - Weight: 10%
- **Authentication**: SSO, MFA, RBAC granularity
- **Data Protection**: Encryption at rest and in transit, data residency
- **Certifications**: SOC 2, ISO 27001, GDPR, HIPAA compliance
- **Audit Trail**: Activity logging, access monitoring
- **Privacy Controls**: Data retention policies, right to deletion
## Weighting Guidelines
Default weights above suit most B2B SaaS evaluations. Adjust based on:
- **Enterprise buyers**: Increase Security (15%), Support (15%), reduce Pricing (10%)
- **Developer tools**: Increase Integrations (20%), Features (30%), reduce UX (10%)
- **SMB products**: Increase Pricing (25%), UX (25%), reduce Security (5%)
- **Regulated industries**: Increase Security (25%), reduce Features (15%)
## Calibration Process
1. **Anchor scoring** - Score your own product first to establish baseline
2. **Multiple scorers** - Have 2-3 team members score independently
3. **Discuss outliers** - Reconcile scores that differ by more than 2 points
4. **Document evidence** - Record specific examples justifying each score
5. **Normalize quarterly** - Re-calibrate as market expectations evolve
## Bias Mitigation
- **Avoid halo effect** - Score each dimension independently, not influenced by overall impression
- **Use evidence, not feelings** - Every score must link to observable data points
- **Include competitor strengths** - Resist tendency to under-score competitors
- **Rotate scorers** - Different team members bring fresh perspectives
- **Blind scoring** - When possible, evaluate features without knowing which competitor
- **Customer validation** - Compare internal scores against user review sentiment
## Composite Score Calculation
```
Weighted Score = SUM(Dimension Score x Dimension Weight)
Example:
UX(8) x 0.20 = 1.60
Features(7) x 0.25 = 1.75
Pricing(6) x 0.15 = 0.90
Integrations(8) x 0.10 = 0.80
Support(7) x 0.10 = 0.70
Performance(9) x 0.10 = 0.90
Security(8) x 0.10 = 0.80
---
Total = 7.45 / 10
```
## Output Format
Present results as a comparison matrix with color coding:
- Green (8-10): Competitive advantage
- Yellow (5-7): Market parity
- Red (1-4): Competitive gap
FILE:scripts/competitive_matrix_builder.py
#!/usr/bin/env python3
"""Competitive Matrix Builder — Analyze and score competitors across feature dimensions.
Generates weighted competitive matrices, gap analysis, and positioning insights
from structured competitor data.
Usage:
python competitive_matrix_builder.py competitors.json --format json
python competitive_matrix_builder.py competitors.json --format text
python competitive_matrix_builder.py competitors.json --format text --weights pricing=2,ux=1.5
"""
import argparse
import json
import sys
from typing import Dict, List, Any, Optional
from datetime import datetime
from statistics import mean, stdev
def load_competitors(path: str) -> Dict[str, Any]:
"""Load competitor data from JSON file."""
with open(path, "r") as f:
return json.load(f)
def normalize_score(value: float, min_val: float = 1.0, max_val: float = 10.0) -> float:
"""Normalize a score to 0-100 scale."""
return max(0.0, min(100.0, ((value - min_val) / (max_val - min_val)) * 100))
def calculate_weighted_scores(
competitors: List[Dict[str, Any]],
dimensions: List[str],
weights: Optional[Dict[str, float]] = None
) -> List[Dict[str, Any]]:
"""Calculate weighted scores for each competitor across dimensions."""
if weights is None:
weights = {d: 1.0 for d in dimensions}
results = []
for comp in competitors:
scores = comp.get("scores", {})
weighted_total = 0.0
weight_sum = 0.0
dimension_results = {}
for dim in dimensions:
raw = scores.get(dim, 0)
w = weights.get(dim, 1.0)
normalized = normalize_score(raw)
weighted = normalized * w
weighted_total += weighted
weight_sum += w
dimension_results[dim] = {
"raw": raw,
"normalized": round(normalized, 1),
"weight": w,
"weighted": round(weighted, 1)
}
overall = round(weighted_total / weight_sum, 1) if weight_sum > 0 else 0
results.append({
"name": comp["name"],
"overall_score": overall,
"dimensions": dimension_results,
"tier": classify_tier(overall),
"pricing": comp.get("pricing", {}),
"strengths": comp.get("strengths", []),
"weaknesses": comp.get("weaknesses", [])
})
results.sort(key=lambda x: x["overall_score"], reverse=True)
return results
def classify_tier(score: float) -> str:
"""Classify competitor into tier based on overall score."""
if score >= 80:
return "Leader"
elif score >= 60:
return "Strong Competitor"
elif score >= 40:
return "Viable Alternative"
elif score >= 20:
return "Niche Player"
else:
return "Weak"
def gap_analysis(
your_scores: Dict[str, float],
competitor_scores: List[Dict[str, Any]],
dimensions: List[str]
) -> Dict[str, Any]:
"""Identify gaps between your product and competitors."""
gaps = {}
for dim in dimensions:
your_val = your_scores.get(dim, 0)
comp_vals = [c["dimensions"][dim]["raw"] for c in competitor_scores if dim in c.get("dimensions", {})]
if not comp_vals:
continue
avg_comp = mean(comp_vals)
best_comp = max(comp_vals)
gap_to_avg = round(your_val - avg_comp, 1)
gap_to_best = round(your_val - best_comp, 1)
gaps[dim] = {
"your_score": your_val,
"competitor_avg": round(avg_comp, 1),
"competitor_best": best_comp,
"gap_to_avg": gap_to_avg,
"gap_to_best": gap_to_best,
"status": "ahead" if gap_to_avg > 0.5 else ("behind" if gap_to_avg < -0.5 else "parity"),
"priority": "high" if gap_to_best < -2 else ("medium" if gap_to_best < -1 else "low")
}
return {
"gaps": gaps,
"biggest_opportunities": sorted(
[{"dimension": k, **v} for k, v in gaps.items() if v["status"] == "behind"],
key=lambda x: x["gap_to_best"]
)[:5],
"competitive_advantages": sorted(
[{"dimension": k, **v} for k, v in gaps.items() if v["status"] == "ahead"],
key=lambda x: -x["gap_to_avg"]
)[:5]
}
def positioning_analysis(scored: List[Dict[str, Any]]) -> Dict[str, Any]:
"""Generate positioning insights from scored competitors."""
scores = [c["overall_score"] for c in scored]
return {
"market_leaders": [c["name"] for c in scored if c["tier"] == "Leader"],
"your_rank": next((i + 1 for i, c in enumerate(scored) if c.get("is_you")), None),
"total_competitors": len(scored),
"score_distribution": {
"mean": round(mean(scores), 1) if scores else 0,
"stdev": round(stdev(scores), 1) if len(scores) > 1 else 0,
"min": round(min(scores), 1) if scores else 0,
"max": round(max(scores), 1) if scores else 0
},
"tier_distribution": {
tier: len([c for c in scored if c["tier"] == tier])
for tier in ["Leader", "Strong Competitor", "Viable Alternative", "Niche Player", "Weak"]
}
}
def format_text(result: Dict[str, Any]) -> str:
"""Format results as human-readable text."""
lines = []
lines.append("=" * 70)
lines.append("COMPETITIVE MATRIX ANALYSIS")
lines.append(f"Generated: {result['generated_at']}")
lines.append("=" * 70)
# Ranking table
lines.append("\n## COMPETITIVE RANKING\n")
lines.append(f"{'Rank':<6}{'Competitor':<25}{'Score':<10}{'Tier':<20}")
lines.append("-" * 61)
for i, c in enumerate(result["scored_competitors"], 1):
marker = " ← YOU" if c.get("is_you") else ""
lines.append(f"{i:<6}{c['name']:<25}{c['overall_score']:<10}{c['tier']:<20}{marker}")
# Dimension breakdown
lines.append("\n## DIMENSION BREAKDOWN\n")
dims = result["dimensions"]
header = f"{'Dimension':<20}" + "".join(f"{c['name'][:12]:<14}" for c in result["scored_competitors"])
lines.append(header)
lines.append("-" * len(header))
for dim in dims:
row = f"{dim:<20}"
for c in result["scored_competitors"]:
val = c["dimensions"].get(dim, {}).get("raw", "N/A")
row += f"{val:<14}"
lines.append(row)
# Gap analysis
if result.get("gap_analysis"):
ga = result["gap_analysis"]
if ga["biggest_opportunities"]:
lines.append("\n## BIGGEST OPPORTUNITIES (where you're behind)\n")
for opp in ga["biggest_opportunities"]:
lines.append(f" • {opp['dimension']}: You={opp['your_score']}, "
f"Best={opp['competitor_best']}, Gap={opp['gap_to_best']} "
f"[{opp['priority'].upper()} priority]")
if ga["competitive_advantages"]:
lines.append("\n## COMPETITIVE ADVANTAGES (where you lead)\n")
for adv in ga["competitive_advantages"]:
lines.append(f" • {adv['dimension']}: You={adv['your_score']}, "
f"Avg={adv['competitor_avg']}, Lead=+{adv['gap_to_avg']}")
# Positioning
pos = result.get("positioning", {})
if pos:
lines.append("\n## MARKET POSITIONING\n")
lines.append(f" Market Leaders: {', '.join(pos.get('market_leaders', ['None']))}")
if pos.get("your_rank"):
lines.append(f" Your Rank: #{pos['your_rank']} of {pos['total_competitors']}")
dist = pos.get("score_distribution", {})
lines.append(f" Score Range: {dist.get('min', 0)} - {dist.get('max', 0)} "
f"(avg: {dist.get('mean', 0)}, stdev: {dist.get('stdev', 0)})")
lines.append("\n" + "=" * 70)
return "\n".join(lines)
def build_matrix(data: Dict[str, Any], weight_overrides: Optional[Dict[str, float]] = None) -> Dict[str, Any]:
"""Main entry: build competitive matrix from input data."""
competitors = data.get("competitors", [])
dimensions = data.get("dimensions", [])
your_product = data.get("your_product", {})
if not competitors:
return {"error": "No competitors provided"}
if not dimensions:
# Auto-detect from first competitor's scores
dimensions = list(competitors[0].get("scores", {}).keys())
weights = data.get("weights", {})
if weight_overrides:
weights.update(weight_overrides)
# Include your product in scoring if provided
all_entries = list(competitors)
if your_product:
your_product["is_you"] = True
all_entries.insert(0, your_product)
scored = calculate_weighted_scores(all_entries, dimensions, weights)
# Mark your product
for s in scored:
if any(c.get("is_you") and c["name"] == s["name"] for c in all_entries):
s["is_you"] = True
result = {
"generated_at": datetime.now().isoformat(),
"dimensions": dimensions,
"weights": weights if weights else {d: 1.0 for d in dimensions},
"scored_competitors": scored,
"positioning": positioning_analysis(scored)
}
if your_product:
result["gap_analysis"] = gap_analysis(
your_product.get("scores", {}), scored, dimensions
)
return result
def parse_weights(weight_str: str) -> Dict[str, float]:
"""Parse weight string like 'pricing=2,ux=1.5' into dict."""
weights = {}
for pair in weight_str.split(","):
if "=" in pair:
k, v = pair.split("=", 1)
weights[k.strip()] = float(v.strip())
return weights
def main():
parser = argparse.ArgumentParser(
description="Build competitive matrix with scoring and gap analysis"
)
parser.add_argument("input", help="Path to competitors JSON file")
parser.add_argument("--format", choices=["json", "text"], default="text",
help="Output format (default: text)")
parser.add_argument("--weights", type=str, default=None,
help="Weight overrides: 'dim1=2.0,dim2=1.5'")
parser.add_argument("--output", type=str, default=None,
help="Output file path (default: stdout)")
args = parser.parse_args()
data = load_competitors(args.input)
weight_overrides = parse_weights(args.weights) if args.weights else None
result = build_matrix(data, weight_overrides)
if args.format == "json":
output = json.dumps(result, indent=2)
else:
output = format_text(result)
if args.output:
with open(args.output, "w") as f:
f.write(output)
print(f"Output written to {args.output}")
else:
print(output)
if __name__ == "__main__":
main()
Tạo trang so sánh và trang thay thế đối thủ phục vụ SEO và hỗ trợ bán hàng.
---
name: "competitor-alternatives"
description: "When the user wants to create competitor comparison or alternative pages for SEO and sales enablement. Also use when the user mentions 'alternative page,' 'vs page,' 'competitor comparison,' 'comparison page,' '[Product] vs [Product],' '[Product] alternative,' 'competitive landing pages,' 'switch from competitor,' or 'comparison content.' Covers four formats: singular alternative, plural alternatives, you vs competitor, and competitor vs competitor. Emphasizes deep research, modular content architecture, and varied section types beyond feature tables."
license: MIT
metadata:
version: 1.0.0
author: Alireza Rezvani
category: marketing
updated: 2026-03-06
---
# Competitor & Alternative Pages
You are an expert in creating competitor comparison and alternative pages. Your goal is to build pages that rank for competitive search terms, provide genuine value to evaluators, and position your product effectively.
## Initial Assessment
**Check for product marketing context first:**
If `.claude/product-marketing-context.md` exists, read it before asking questions. Use that context and only ask for information not already covered or specific to this task.
Before creating competitor pages, understand:
1. **Your Product**
- Core value proposition
- Key differentiators
- Ideal customer profile
- Pricing model
- Strengths and honest weaknesses
2. **Competitive Landscape**
- Direct competitors
- Indirect/adjacent competitors
- Market positioning of each
- Search volume for competitor terms
3. **Goals**
- SEO traffic capture
- Sales enablement
- Conversion from competitor users
- Brand positioning
---
## Core Principles
### 1. Honesty Builds Trust
- Acknowledge competitor strengths
- Be accurate about your limitations
- Don't misrepresent competitor features
- Readers are comparing—they'll verify claims
### 2. Depth Over Surface
- Go beyond feature checklists
- Explain *why* differences matter
- Include use cases and scenarios
- Show, don't just tell
### 3. Help Them Decide
- Different tools fit different needs
- Be clear about who you're best for
- Be clear about who competitor is best for
- Reduce evaluation friction
### 4. Modular Content Architecture
- Competitor data should be centralized
- Updates propagate to all pages
- Single source of truth per competitor
---
## Page Formats
### Format 1: [Competitor] Alternative (Singular)
**Search intent**: User is actively looking to switch from a specific competitor
**URL pattern**: `/alternatives/[competitor]` or `/[competitor]-alternative`
**Target keywords**: "[Competitor] alternative", "alternative to [Competitor]", "switch from [Competitor]"
**Page structure**:
1. Why people look for alternatives (validate their pain)
2. Summary: You as the alternative (quick positioning)
3. Detailed comparison (features, service, pricing)
4. Who should switch (and who shouldn't)
5. Migration path
6. Social proof from switchers
7. CTA
---
### Format 2: [Competitor] Alternatives (Plural)
**Search intent**: User is researching options, earlier in journey
**URL pattern**: `/alternatives/[competitor]-alternatives`
**Target keywords**: "[Competitor] alternatives", "best [Competitor] alternatives", "tools like [Competitor]"
**Page structure**:
1. Why people look for alternatives (common pain points)
2. What to look for in an alternative (criteria framework)
3. List of alternatives (you first, but include real options)
4. Comparison table (summary)
5. Detailed breakdown of each alternative
6. Recommendation by use case
7. CTA
**Important**: Include 4-7 real alternatives. Being genuinely helpful builds trust and ranks better.
---
### Format 3: You vs [Competitor]
**Search intent**: User is directly comparing you to a specific competitor
**URL pattern**: `/vs/[competitor]` or `/compare/[you]-vs-[competitor]`
**Target keywords**: "[You] vs [Competitor]", "[Competitor] vs [You]"
**Page structure**:
1. TL;DR summary (key differences in 2-3 sentences)
2. At-a-glance comparison table
3. Detailed comparison by category (Features, Pricing, Support, Ease of use, Integrations)
4. Who [You] is best for
5. Who [Competitor] is best for (be honest)
6. What customers say (testimonials from switchers)
7. Migration support
8. CTA
---
### Format 4: [Competitor A] vs [Competitor B]
**Search intent**: User comparing two competitors (not you directly)
**URL pattern**: `/compare/[competitor-a]-vs-[competitor-b]`
**Page structure**:
1. Overview of both products
2. Comparison by category
3. Who each is best for
4. The third option (introduce yourself)
5. Comparison table (all three)
6. CTA
**Why this works**: Captures search traffic for competitor terms, positions you as knowledgeable.
---
## Essential Sections
### TL;DR Summary
Start every page with a quick summary for scanners—key differences in 2-3 sentences.
### Paragraph Comparisons
Go beyond tables. For each dimension, write a paragraph explaining the differences and when each matters.
### Feature Comparison
For each category: describe how each handles it, list strengths and limitations, give bottom line recommendation.
### Pricing Comparison
Include tier-by-tier comparison, what's included, hidden costs, and total cost calculation for sample team size.
### Who It's For
Be explicit about ideal customer for each option. Honest recommendations build trust.
### Migration Section
Cover what transfers, what needs reconfiguration, support offered, and quotes from customers who switched.
**For detailed templates**: See [references/templates.md](references/templates.md)
---
## Content Architecture
### Centralized Competitor Data
Create a single source of truth for each competitor with:
- Positioning and target audience
- Pricing (all tiers)
- Feature ratings
- Strengths and weaknesses
- Best for / not ideal for
- Common complaints (from reviews)
- Migration notes
**For data structure and examples**: See [references/content-architecture.md](references/content-architecture.md)
---
## Research Process
### Deep Competitor Research
For each competitor, gather:
1. **Product research**: Sign up, use it, document features/UX/limitations
2. **Pricing research**: Current pricing, what's included, hidden costs
3. **Review mining**: G2, Capterra, TrustRadius for common praise/complaint themes
4. **Customer feedback**: Talk to customers who switched (both directions)
5. **Content research**: Their positioning, their comparison pages, their changelog
### Ongoing Updates
- **Quarterly**: Verify pricing, check for major feature changes
- **When notified**: Customer mentions competitor change
- **Annually**: Full refresh of all competitor data
---
## SEO Considerations
### Keyword Targeting
| Format | Primary Keywords |
|--------|-----------------|
| Alternative (singular) | [Competitor] alternative, alternative to [Competitor] |
| Alternatives (plural) | [Competitor] alternatives, best [Competitor] alternatives |
| You vs Competitor | [You] vs [Competitor], [Competitor] vs [You] |
| Competitor vs Competitor | [A] vs [B], [B] vs [A] |
### Internal Linking
- Link between related competitor pages
- Link from feature pages to relevant comparisons
- Create hub page linking to all competitor content
### Schema Markup
Consider FAQ schema for common questions like "What is the best alternative to [Competitor]?"
---
## Output Format
### Competitor Data File
Complete competitor profile in YAML format for use across all comparison pages.
### Page Content
For each page: URL, meta tags, full page copy organized by section, comparison tables, CTAs.
### Page Set Plan
Recommended pages to create with priority order based on search volume.
---
## Task-Specific Questions
1. What are common reasons people switch to you?
2. Do you have customer quotes about switching?
3. What's your pricing vs. competitors?
4. Do you offer migration support?
---
## Proactive Triggers
Proactively offer competitor page creation when:
1. **Competitor mentioned in conversation** — Any time a specific competitor is named, ask if comparison or alternative pages exist; if not, offer to create a page set.
2. **Sales team friction** — User mentions prospects comparing them to a specific tool; immediately offer a vs-page for sales enablement.
3. **SEO gap identified** — Keyword research shows competitor-branded terms with no coverage; propose a full alternative page set with prioritized build order.
4. **Switcher testimonial available** — When a customer quote about switching surfaces, offer to build a migration-focused alternative page around it.
5. **Pricing page review** — When reviewing pricing, note that pricing comparison tables belong on dedicated competitor pages, not the pricing page itself.
---
## Output Artifacts
| Artifact | Format | Description |
|----------|--------|-------------|
| Competitor Intelligence File | YAML data file | Centralized competitor profile: pricing, features, weaknesses, review themes |
| Page Set Plan | Prioritized list | Ranked list of pages to build with target keywords and search volume estimates |
| Alternative Page (Singular) | Full page copy | Complete `/[competitor]-alternative` page with all sections |
| Vs Page | Full page copy | Complete `/vs/[competitor]` page with comparison table and CTA |
| Migration Guide Section | Markdown block | Reusable migration copy for inclusion across multiple pages |
---
## Communication
All competitor page outputs should be factually accurate, legally safe (no false claims), and fair to competitors. Acknowledge genuine competitor strengths — pages that only disparage competitors lose credibility with evaluators. Reference `marketing-context` for ICP and positioning before writing any comparison copy. Quality bar: every claim must be verifiable from public sources or customer quotes.
---
## Related Skills
- **seo-audit** — USE to validate that competitor pages meet on-page SEO requirements before publishing; NOT as a replacement for the keyword strategy built here.
- **copywriting** — USE for writing the narrative sections and CTAs on comparison pages; NOT when the task is purely competitor research and architecture.
- **content-strategy** — USE when planning a full competitive content program across multiple pages; NOT for single-page execution.
- **competitive-intel** — USE when C-level strategic competitive analysis is needed beyond page creation; NOT for tactical page writing.
- **marketing-context** — USE as foundation before any competitor page work to align positioning; always load first.
FILE:references/content-architecture.md
# Content Architecture for Competitor Pages
How to structure and maintain competitor data for scalable comparison pages.
## Centralized Competitor Data
Create a single source of truth for each competitor:
```
competitor_data/
├── notion.md
├── airtable.md
├── monday.md
└── ...
```
---
## Competitor Data Template
Per competitor, document:
```yaml
name: Notion
website: notion.so
tagline: "The all-in-one workspace"
founded: 2016
headquarters: San Francisco
# Positioning
primary_use_case: "docs + light databases"
target_audience: "teams wanting flexible workspace"
market_position: "premium, feature-rich"
# Pricing
pricing_model: per-seat
free_tier: true
free_tier_limits: "limited blocks, 1 user"
starter_price: $8/user/month
business_price: $15/user/month
enterprise: custom
# Features (rate 1-5 or describe)
features:
documents: 5
databases: 4
project_management: 3
collaboration: 4
integrations: 3
mobile_app: 3
offline_mode: 2
api: 4
# Strengths (be honest)
strengths:
- Extremely flexible and customizable
- Beautiful, modern interface
- Strong template ecosystem
- Active community
# Weaknesses (be fair)
weaknesses:
- Can be slow with large databases
- Learning curve for advanced features
- Limited automations compared to dedicated tools
- Offline mode is limited
# Best for
best_for:
- Teams wanting all-in-one workspace
- Content-heavy workflows
- Documentation-first teams
- Startups and small teams
# Not ideal for
not_ideal_for:
- Complex project management needs
- Large databases (1000s of rows)
- Teams needing robust offline
- Enterprise with strict compliance
# Common complaints (from reviews)
common_complaints:
- "Gets slow with lots of content"
- "Hard to find things as workspace grows"
- "Mobile app is clunky"
# Migration notes
migration_from:
difficulty: medium
data_export: "Markdown, CSV, HTML"
what_transfers: "Pages, databases"
what_doesnt: "Automations, integrations setup"
time_estimate: "1-3 days for small team"
```
---
## Your Product Data
Same structure for yourself—be honest:
```yaml
name: [Your Product]
# ... same fields
strengths:
- [Your real strengths]
weaknesses:
- [Your honest weaknesses]
best_for:
- [Your ideal customers]
not_ideal_for:
- [Who should use something else]
```
---
## Page Generation
Each page pulls from centralized data:
- **[Competitor] Alternative page**: Pulls competitor data + your data
- **[Competitor] Alternatives page**: Pulls competitor data + your data + other alternatives
- **You vs [Competitor] page**: Pulls your data + competitor data
- **[A] vs [B] page**: Pulls both competitor data + your data
**Benefits**:
- Update competitor pricing once, updates everywhere
- Add new feature comparison once, appears on all pages
- Consistent accuracy across pages
- Easier to maintain at scale
---
## Index Page Structure
### Alternatives Index
**URL**: `/alternatives` or `/alternatives/index`
**Purpose**: Lists all "[Competitor] Alternative" pages
**Page structure**:
1. Headline: "[Your Product] as an Alternative"
2. Brief intro on why people switch to you
3. List of all alternative pages with:
- Competitor name/logo
- One-line summary of key differentiator vs. that competitor
- Link to full comparison
4. Common reasons people switch (aggregated)
5. CTA
**Example**:
```markdown
## Explore [Your Product] as an Alternative
Looking to switch? See how [Your Product] compares to the tools you're evaluating:
- **[Notion Alternative](/alternatives/notion)** — Better for teams who need [X]
- **[Airtable Alternative](/alternatives/airtable)** — Better for teams who need [Y]
- **[Monday Alternative](/alternatives/monday)** — Better for teams who need [Z]
```
---
### Vs Comparisons Index
**URL**: `/vs` or `/compare`
**Purpose**: Lists all "You vs [Competitor]" and "[A] vs [B]" pages
**Page structure**:
1. Headline: "Compare [Your Product]"
2. Section: "[Your Product] vs Competitors" — list of direct comparisons
3. Section: "Head-to-Head Comparisons" — list of [A] vs [B] pages
4. Brief methodology note
5. CTA
---
### Index Page Best Practices
**Keep them updated**: When you add a new comparison page, add it to the relevant index.
**Internal linking**:
- Link from index → individual pages
- Link from individual pages → back to index
- Cross-link between related comparisons
**SEO value**:
- Index pages can rank for broad terms like "project management tool comparisons"
- Pass link equity to individual comparison pages
- Help search engines discover all comparison content
**Sorting options**:
- By popularity (search volume)
- Alphabetically
- By category/use case
- By date added (show freshness)
**Include on index pages**:
- Last updated date for credibility
- Number of pages/comparisons available
- Quick filters if you have many comparisons
---
## Footer Navigation
The site footer appears on all marketing pages, making it a powerful internal linking opportunity for competitor pages.
### Option 1: Link to Index Pages (Minimum)
At minimum, add links to your comparison index pages in the footer:
```
Footer
├── Compare
│ ├── Alternatives → /alternatives
│ └── Comparisons → /vs
```
This ensures every marketing page passes link equity to your comparison content hub.
### Option 2: Footer Columns by Format (Recommended for SEO)
For stronger internal linking, create dedicated footer columns for each format you've built, linking directly to your top competitors:
```
Footer
├── [Product] vs ├── Alternatives to ├── Compare
│ ├── vs Notion │ ├── Notion Alternative │ ├── Notion vs Airtable
│ ├── vs Airtable │ ├── Airtable Alternative │ ├── Monday vs Asana
│ ├── vs Monday │ ├── Monday Alternative │ ├── Notion vs Monday
│ ├── vs Asana │ ├── Asana Alternative │ ├── ...
│ ├── vs Clickup │ ├── Clickup Alternative │ └── View all →
│ ├── ... │ ├── ... │
│ └── View all → │ └── View all → │
```
**Guidelines**:
- Include up to 8 links per column (top competitors by search volume)
- Add "View all" link to the full index page
- Only create columns for formats you've actually built pages for
- Prioritize competitors with highest search volume
### Why Footer Links Matter
1. **Sitewide distribution**: Footer links appear on every marketing page, passing link equity from your entire site to comparison content
2. **Crawl efficiency**: Search engines discover all comparison pages quickly
3. **User discovery**: Visitors evaluating your product can easily find comparisons
4. **Competitive positioning**: Signals to search engines that you're a key player in the space
### Implementation Notes
- Update footer when adding new high-priority comparison pages
- Keep footer clean—don't list every comparison, just the top ones
- Match column headers to your URL structure (e.g., "vs" column → `/vs/` URLs)
- Consider mobile: columns may stack, so order by priority
FILE:references/templates.md
# Section Templates for Competitor Pages
Ready-to-use templates for each section of competitor comparison pages.
## TL;DR Summary
Start every page with a quick summary for scanners:
```markdown
**TL;DR**: [Competitor] excels at [strength] but struggles with [weakness].
[Your product] is built for [your focus], offering [key differentiator].
Choose [Competitor] if [their ideal use case]. Choose [You] if [your ideal use case].
```
---
## Paragraph Comparison (Not Just Tables)
For each major dimension, write a paragraph:
```markdown
## Features
[Competitor] offers [description of their feature approach].
Their strength is [specific strength], which works well for [use case].
However, [limitation] can be challenging for [user type].
[Your product] takes a different approach with [your approach].
This means [benefit], though [honest tradeoff].
Teams who [specific need] often find this more effective.
```
---
## Feature Comparison Section
Go beyond checkmarks:
```markdown
## Feature Comparison
### [Feature Category]
**[Competitor]**: [2-3 sentence description of how they handle this]
- Strengths: [specific]
- Limitations: [specific]
**[Your product]**: [2-3 sentence description]
- Strengths: [specific]
- Limitations: [specific]
**Bottom line**: Choose [Competitor] if [scenario]. Choose [You] if [scenario].
```
---
## Pricing Comparison Section
```markdown
## Pricing
| | [Competitor] | [Your Product] |
|---|---|---|
| Free tier | [Details] | [Details] |
| Starting price | $X/user/mo | $X/user/mo |
| Business tier | $X/user/mo | $X/user/mo |
| Enterprise | Custom | Custom |
**What's included**: [Competitor]'s $X plan includes [features], while
[Your product]'s $X plan includes [features].
**Total cost consideration**: Beyond per-seat pricing, consider [hidden costs,
add-ons, implementation]. [Competitor] charges extra for [X], while
[Your product] includes [Y] in base pricing.
**Value comparison**: For a 10-person team, [Competitor] costs approximately
$X/year while [Your product] costs $Y/year, with [key differences in what you get].
```
---
## Service & Support Comparison
```markdown
## Service & Support
| | [Competitor] | [Your Product] |
|---|---|---|
| Documentation | [Quality assessment] | [Quality assessment] |
| Response time | [SLA if known] | [Your SLA] |
| Support channels | [List] | [List] |
| Onboarding | [What they offer] | [What you offer] |
| CSM included | [At what tier] | [At what tier] |
**Support quality**: Based on [G2/Capterra reviews, your research],
[Competitor] support is described as [assessment]. Common feedback includes
[quotes or themes].
[Your product] offers [your support approach]. [Specific differentiator like
response time, dedicated CSM, implementation help].
```
---
## Who It's For Section
```markdown
## Who Should Choose [Competitor]
[Competitor] is the right choice if:
- [Specific use case or need]
- [Team type or size]
- [Workflow or requirement]
- [Budget or priority]
**Ideal [Competitor] customer**: [Persona description in 1-2 sentences]
## Who Should Choose [Your Product]
[Your product] is built for teams who:
- [Specific use case or need]
- [Team type or size]
- [Workflow or requirement]
- [Priority or value]
**Ideal [Your product] customer**: [Persona description in 1-2 sentences]
```
---
## Migration Section
```markdown
## Switching from [Competitor]
### What transfers
- [Data type]: [How easily, any caveats]
- [Data type]: [How easily, any caveats]
### What needs reconfiguration
- [Thing]: [Why and effort level]
- [Thing]: [Why and effort level]
### Migration support
We offer [migration support details]:
- [Free data import tool / white-glove migration]
- [Documentation / migration guide]
- [Timeline expectation]
- [Support during transition]
### What customers say about switching
> "[Quote from customer who switched]"
> — [Name], [Role] at [Company]
```
---
## Social Proof Section
Focus on switchers:
```markdown
## What Customers Say
### Switched from [Competitor]
> "[Specific quote about why they switched and outcome]"
> — [Name], [Role] at [Company]
> "[Another quote]"
> — [Name], [Role] at [Company]
### Results after switching
- [Company] saw [specific result]
- [Company] reduced [metric] by [amount]
```
---
## Comparison Table Best Practices
### Beyond Checkmarks
Instead of:
| Feature | You | Competitor |
|---------|-----|-----------|
| Feature A | ✓ | ✓ |
| Feature B | ✓ | ✗ |
Do this:
| Feature | You | Competitor |
|---------|-----|-----------|
| Feature A | Full support with [detail] | Basic support, [limitation] |
| Feature B | [Specific capability] | Not available |
### Organize by Category
Group features into meaningful categories:
- Core functionality
- Collaboration
- Integrations
- Security & compliance
- Support & service
### Include Ratings Where Useful
| Category | You | Competitor | Notes |
|----------|-----|-----------|-------|
| Ease of use | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ | [Brief note] |
| Feature depth | ⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | [Brief note] |
FILE:scripts/comparison_matrix_builder.py
#!/usr/bin/env python3
"""
comparison_matrix_builder.py — Competitive Feature Comparison Matrix Builder
100% stdlib, no pip installs required.
Usage:
python3 comparison_matrix_builder.py # demo mode
python3 comparison_matrix_builder.py --input matrix.json
python3 comparison_matrix_builder.py --input matrix.json --json
python3 comparison_matrix_builder.py --input matrix.json --markdown > comparison.md
matrix.json format:
{
"your_product": "YourProduct",
"features": [
{
"name": "SSO / SAML",
"category": "Security",
"your_status": "full", # full | partial | no | planned
"competitors": {
"CompetitorA": "no",
"CompetitorB": "partial",
"CompetitorC": "full"
},
"notes": "Enterprise tier only" # optional
}
]
}
"""
import argparse
import json
import sys
from collections import defaultdict
# ---------------------------------------------------------------------------
# Status helpers
# ---------------------------------------------------------------------------
STATUS_SCORE = {
"full": 2,
"partial": 1,
"no": 0,
"planned": 0, # planned ≠ shipped; conservative scoring
}
STATUS_LABEL = {
"full": "✅",
"partial": "🔶",
"no": "❌",
"planned": "🗓",
}
STATUS_TEXT = {
"full": "Full",
"partial": "Partial",
"no": "No",
"planned": "Planned",
}
FEATURE_IMPORTANCE = {
# Generic defaults — override per-feature with "weight" in JSON
"default": 1,
}
# ---------------------------------------------------------------------------
# Core builder
# ---------------------------------------------------------------------------
def normalise_status(s: str) -> str:
s = (s or "no").strip().lower()
return s if s in STATUS_SCORE else "no"
def build_matrix(data: dict) -> dict:
your_product = data.get("your_product", "Your Product")
features = data.get("features", [])
if not features:
raise ValueError("No features provided in input.")
# Collect competitor names (ordered, deduplicated)
competitors = []
seen = set()
for f in features:
for c in f.get("competitors", {}):
if c not in seen:
competitors.append(c)
seen.add(c)
categories = sorted(set(f.get("category", "General") for f in features))
# --- per-feature analysis ---
feature_rows = []
for f in features:
fname = f.get("name", "?")
category = f.get("category", "General")
weight = f.get("weight", 1)
your_raw = normalise_status(f.get("your_status", "no"))
your_s = STATUS_SCORE[your_raw]
comp_raw = {c: normalise_status(f.get("competitors", {}).get(c, "no"))
for c in competitors}
comp_s = {c: STATUS_SCORE[comp_raw[c]] for c in competitors}
you_win = all(your_s > comp_s[c] for c in competitors) if competitors else False
you_lose = any(your_s < comp_s[c] for c in competitors)
your_max = max(comp_s.values()) if comp_s else 0
advantage = your_s - your_max # positive = you're better overall
feature_rows.append({
"name": fname,
"category": category,
"weight": weight,
"your_status": your_raw,
"your_score": your_s,
"competitors": comp_raw,
"comp_scores": comp_s,
"you_win": you_win,
"you_lose": you_lose,
"advantage": advantage,
"notes": f.get("notes", ""),
})
# --- competitive scores per competitor ---
comp_scores = {}
for c in competitors:
wins = sum(1 for r in feature_rows if r["your_score"] > r["comp_scores"].get(c, 0))
ties = sum(1 for r in feature_rows if r["your_score"] == r["comp_scores"].get(c, 0))
losses = sum(1 for r in feature_rows if r["your_score"] < r["comp_scores"].get(c, 0))
total = len(feature_rows)
score = round((wins / total) * 100) if total else 0
comp_scores[c] = {
"wins": wins, "ties": ties, "losses": losses,
"win_pct": score,
"verdict": _verdict(score),
}
# Overall competitive score (average win% across all competitors)
overall_win_pct = (
round(sum(v["win_pct"] for v in comp_scores.values()) / len(comp_scores))
if comp_scores else 0
)
# Advantages and gaps
advantages = [r["name"] for r in feature_rows if r["advantage"] > 0]
gaps = [r["name"] for r in feature_rows if r["advantage"] < 0]
parity = [r["name"] for r in feature_rows if r["advantage"] == 0]
return {
"meta": {
"your_product": your_product,
"competitors": competitors,
"categories": categories,
"total_features": len(feature_rows),
"overall_win_pct": overall_win_pct,
"verdict": _verdict(overall_win_pct),
},
"competitor_scores": comp_scores,
"advantages": advantages,
"gaps": gaps,
"parity": parity,
"features": feature_rows,
}
def _verdict(win_pct: int) -> str:
if win_pct >= 70: return "Strong advantage"
if win_pct >= 50: return "Slight advantage"
if win_pct >= 35: return "Competitive parity"
return "Trailing"
# ---------------------------------------------------------------------------
# Markdown output
# ---------------------------------------------------------------------------
def build_markdown(result: dict) -> str:
m = result["meta"]
rows = result["features"]
comp = m["competitors"]
lines = []
lines.append(f"# Feature Comparison: {m['your_product']} vs Competitors\n")
lines.append(f"_Generated by comparison_matrix_builder.py — {m['total_features']} features, "
f"{len(comp)} competitor(s)_\n")
# Summary table
lines.append("## Competitive Score Summary\n")
lines.append("| Competitor | You Win | Tie | You Lose | Win % | Verdict |")
lines.append("|---|---|---|---|---|---|")
for c, s in result["competitor_scores"].items():
lines.append(f"| {c} | {s['wins']} | {s['ties']} | {s['losses']} | "
f"**{s['win_pct']}%** | {s['verdict']} |")
lines.append(f"\n**Overall win rate: {m['overall_win_pct']}% — {m['verdict']}**\n")
# Feature matrix by category
lines.append("## Feature Matrix\n")
header = f"| Feature | {m['your_product']} | " + " | ".join(comp) + " | Notes |"
sep = "|---|---|" + "|".join(["---"] * len(comp)) + "|---|"
lines.append(header)
lines.append(sep)
current_cat = None
for r in rows:
cat = r["category"]
if cat != current_cat:
lines.append(f"| **{cat}** | | " + " | ".join([""] * len(comp)) + " | |")
current_cat = cat
you_icon = STATUS_LABEL[r["your_status"]]
comp_icons = " | ".join(STATUS_LABEL[r["competitors"].get(c, "no")] for c in comp)
note = r["notes"] or ""
# Highlight row if it's a unique advantage
fname = f"**{r['name']}**" if r["advantage"] > 0 else r["name"]
lines.append(f"| {fname} | {you_icon} | {comp_icons} | {note} |")
lines.append("")
# Advantages
if result["advantages"]:
lines.append("## ✅ Your Advantages\n")
for a in result["advantages"]:
lines.append(f"- {a}")
lines.append("")
# Gaps
if result["gaps"]:
lines.append("## ⚠️ Feature Gaps (competitors ahead)\n")
for g in result["gaps"]:
lines.append(f"- {g}")
lines.append("")
# Legend
lines.append("## Legend\n")
for k, v in STATUS_LABEL.items():
lines.append(f"- {v} {STATUS_TEXT[k]}")
lines.append("")
return "\n".join(lines)
# ---------------------------------------------------------------------------
# Pretty terminal output
# ---------------------------------------------------------------------------
def pretty_print(result: dict) -> None:
m = result["meta"]
print("\n" + "=" * 70)
print(f" COMPETITIVE MATRIX: {m['your_product'].upper()} vs {', '.join(m['competitors'])}")
print("=" * 70)
print(f"\n Total features analysed : {m['total_features']}")
print(f" Overall win rate : {m['overall_win_pct']}% ({m['verdict']})")
print(f"\n{'─'*70}")
print(f" {'COMPETITOR':<22} {'WIN%':>5} {'WINS':>5} {'TIES':>5} {'LOSSES':>7} VERDICT")
print(f"{'─'*70}")
for c, s in result["competitor_scores"].items():
bar = "█" * (s["win_pct"] // 10) + "░" * (10 - s["win_pct"] // 10)
print(f" {c:<22} {s['win_pct']:>4}% {s['wins']:>5} {s['ties']:>5} "
f"{s['losses']:>7} {bar} {s['verdict']}")
print(f"\n{'─'*70}")
col_w = 20
header = f" {'FEATURE':<28} | {'YOU':^8}"
for c in m["competitors"]:
header += f" | {c[:8]:^8}"
print(header)
print("─" * (30 + 11 * (1 + len(m["competitors"]))))
current_cat = None
for r in result["features"]:
if r["category"] != current_cat:
print(f"\n [{r['category']}]")
current_cat = r["category"]
you_icon = STATUS_LABEL[r["your_status"]]
line = f" {' '+r['name']:<28} | {you_icon:^8}"
for c in m["competitors"]:
ci = STATUS_LABEL[r["competitors"].get(c, "no")]
line += f" | {ci:^8}"
if r["advantage"] > 0:
line += " ← advantage"
elif r["advantage"] < 0:
line += " ← gap"
print(line)
print(f"\n ✅ YOUR ADVANTAGES ({len(result['advantages'])} features)")
for a in result["advantages"]:
print(f" • {a}")
print(f"\n ⚠️ FEATURE GAPS ({len(result['gaps'])} features)")
for g in result["gaps"]:
print(f" • {g}")
print(f"\n Legend: {STATUS_LABEL['full']} Full {STATUS_LABEL['partial']} Partial "
f"{STATUS_LABEL['no']} No {STATUS_LABEL['planned']} Planned\n")
# ---------------------------------------------------------------------------
# Sample data
# ---------------------------------------------------------------------------
DEMO_DATA = {
"your_product": "SwiftBase",
"features": [
{"name": "SSO / SAML", "category": "Security", "weight": 3, "your_status": "full", "competitors": {"AcmeSaaS": "no", "ProStack": "partial"}, "notes": "All plans"},
{"name": "2FA / MFA", "category": "Security", "weight": 3, "your_status": "full", "competitors": {"AcmeSaaS": "full", "ProStack": "full"}, "notes": ""},
{"name": "SOC 2 Type II", "category": "Security", "weight": 3, "your_status": "planned", "competitors": {"AcmeSaaS": "full", "ProStack": "no"}, "notes": "Q3 target"},
{"name": "Role-based access", "category": "Security", "weight": 2, "your_status": "full", "competitors": {"AcmeSaaS": "partial", "ProStack": "full"}, "notes": ""},
{"name": "REST API", "category": "Integrations", "weight": 3, "your_status": "full", "competitors": {"AcmeSaaS": "full", "ProStack": "full"}, "notes": ""},
{"name": "GraphQL API", "category": "Integrations", "weight": 2, "your_status": "full", "competitors": {"AcmeSaaS": "no", "ProStack": "partial"}, "notes": ""},
{"name": "Zapier Integration", "category": "Integrations", "weight": 2, "your_status": "partial", "competitors": {"AcmeSaaS": "full", "ProStack": "full"}, "notes": "10 zaps only"},
{"name": "Webhooks", "category": "Integrations", "weight": 2, "your_status": "full", "competitors": {"AcmeSaaS": "full", "ProStack": "no"}, "notes": ""},
{"name": "Custom domain", "category": "Branding", "weight": 2, "your_status": "full", "competitors": {"AcmeSaaS": "partial", "ProStack": "full"}, "notes": ""},
{"name": "White-label / rebrand","category": "Branding", "weight": 2, "your_status": "full", "competitors": {"AcmeSaaS": "no", "ProStack": "partial"}, "notes": "Agency plan"},
{"name": "Priority support", "category": "Support", "weight": 2, "your_status": "full", "competitors": {"AcmeSaaS": "partial", "ProStack": "full"}, "notes": "24/7"},
{"name": "Dedicated CSM", "category": "Support", "weight": 2, "your_status": "no", "competitors": {"AcmeSaaS": "full", "ProStack": "full"}, "notes": "Enterprise only"},
{"name": "SLA guarantee", "category": "Support", "weight": 3, "your_status": "no", "competitors": {"AcmeSaaS": "full", "ProStack": "no"}, "notes": "Roadmap"},
],
}
# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------
def parse_args():
parser = argparse.ArgumentParser(
description="Build a competitive feature comparison matrix (stdlib only).",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__,
)
parser.add_argument("--input", type=str, default=None,
help="Path to JSON input file")
parser.add_argument("--json", action="store_true",
help="Output analysis as JSON")
parser.add_argument("--markdown", action="store_true",
help="Output comparison table as Markdown")
return parser.parse_args()
def main():
args = parse_args()
if args.input:
with open(args.input) as f:
data = json.load(f)
else:
print("🔬 DEMO MODE — using sample SaaS product matrix\n", file=sys.stderr)
data = DEMO_DATA
result = build_matrix(data)
if args.json:
# Serialise (remove non-JSON-safe keys)
print(json.dumps(result, indent=2))
elif args.markdown:
print(build_markdown(result))
else:
pretty_print(result)
print("\n💡 TIP: Re-run with --markdown to get a copyable Markdown table.\n")
if __name__ == "__main__":
main()