@admin
Định nghĩa, rà soát và vận hành SLO, SLI, error budget, burn rate và cảnh báo đa cửa sổ theo Google SRE Workbook.
---
name: slo-architect
description: Use when defining, reviewing, or operating SLOs/SLIs/error budgets. Triggers on "define an SLO", "what should our SLO be", "error budget", "burn rate", "SLI", "service level objective", "Google SRE workbook", "multi-window burn-rate alert", or any reliability-target question. Ships SLO designer, error-budget calculator with multi-window burn-rate thresholds, and SLO reviewer that catches the common bugs (target too aggressive, window too short, conflicting SLOs, no SLI definition). 4 references on SLO principles + SLI design + error budget math + composition with feature-flags-architect/chaos-engineering/kubernetes-operator. NOT a generic observability skill — specifically the SLO discipline.
context: fork
version: 2.9.0
author: claude-code-skills
license: MIT
tags: [slo, sli, sla, error-budget, burn-rate, sre, reliability, google-sre-workbook, observability]
compatible_tools: [claude-code, codex-cli, cursor, antigravity, opencode, gemini-cli]
---
# SLO Architect
Define SLOs that mean something. Most "SLOs" in the wild are arbitrary numbers no one believes — 99.9% on every endpoint, no SLI definition, no error budget, no policy for what happens when budget burns. This skill enforces the discipline from Google's SRE Workbook: pick the right SLI, set a target users actually care about, calculate the error budget, wire multi-window burn-rate alerts, and have a written policy for when budget runs out.
## When to use
- Defining a new SLO for a service or feature
- Reviewing existing SLOs for common bugs
- Picking the right SLI (event-based vs time-window based vs request-based)
- Computing error budgets and burn-rate alert thresholds
- Tying SLOs to existing controls — feature flags abort, chaos blast radius, operator capability levels
## When NOT to use
- General observability strategy (metrics + logs + traces) → use `observability-designer`
- Customer-facing SLAs with legal teeth → that's contract drafting, not engineering
- Performance load testing (capacity, not reliability) → use `performance-profiler`
- Active incident response → use `incident-response`
## Core principle: an SLO is a promise about user experience
```
SLI ⟶ measurable signal of user-perceived health (e.g., HTTP 2xx rate)
SLO ⟶ target for the SLI over a window (e.g., 99.9% over 30 days)
SLA ⟶ customer-facing commitment with consequences (separate concern)
EB ⟶ error budget: 100% − SLO target = how much "bad" you can spend
BR ⟶ burn rate: how fast you're consuming the error budget
```
The four cardinal mistakes:
1. **Target too high** (99.99%+ on services that can't support it) — every minor blip violates SLO; alerts become noise.
2. **Wrong SLI** (CPU usage as proxy for user experience) — system can be "green" while users suffer.
3. **No error budget policy** — burning budget means nothing if there's no agreed action.
4. **Single-window burn-rate alert** — either too noisy (page on a 5-min spike) or too slow (notice budget exhausted after the fact).
The 3 tools below catch each of these.
## Quick start
```bash
SKILL=engineering/slo-architect/skills/slo-architect
# 1. Design an SLO
python "$SKILL/scripts/slo_designer.py" \
--service checkout-svc \
--sli-type request-success-rate \
--target 99.9 \
--window-days 30
# 2. Compute error budget + multi-window burn-rate alerts
python "$SKILL/scripts/error_budget_calculator.py" \
--target 99.9 --window-days 30
# 3. Review existing SLO definitions for common bugs
python "$SKILL/scripts/slo_review.py" --slo-doc docs/slos/
```
## The 3 Python tools
All stdlib-only.
### `slo_designer.py`
Generates a structured SLO definition with required fields. Refuses to render if any required field is missing (`exit 1`).
```bash
python scripts/slo_designer.py \
--service checkout-svc \
--sli-type request-success-rate \
--target 99.9 \
--window-days 30 \
--owner team-checkout
```
**SLI types supported:**
- `request-success-rate` — `(total_requests - bad_requests) / total_requests`
- `request-latency` — `count(requests < threshold) / total_requests`
- `availability-time` — `(window - downtime) / window`
- `data-freshness` — `count(data_age < threshold) / total_data_points`
- `correctness` — `count(correct_outputs) / total_outputs`
Output is markdown by default with all required fields filled or marked `<must define>`. JSON output (`--format json`) is consumed by `slo_review.py`.
### `error_budget_calculator.py`
Given target availability + window, computes:
- Allowed downtime in the window
- Multi-window burn-rate thresholds per Google SRE Workbook (Chapter 5):
- **Fast burn** — page if 2% of monthly budget consumed in 1 hour
- **Slow burn** — page if 10% consumed in 6 hours, ticket if 10% in 3 days
- Recommended alerting rules (PromQL-shaped output)
```bash
python scripts/error_budget_calculator.py --target 99.9 --window-days 30
python scripts/error_budget_calculator.py --target 99.95 --window-days 7 --format json
```
### `slo_review.py`
Audits a directory of SLO definitions (markdown or JSON) for the common bugs.
```bash
python scripts/slo_review.py --slo-doc docs/slos/
```
**Checks:**
- `target_too_high`: target ≥ 99.99% (sustainable only with massive engineering investment)
- `target_too_low`: target ≤ 99.0% (probably wrong SLI; users will notice)
- `window_too_short`: window < 7 days (statistical noise dominates)
- `window_too_long`: window > 90 days (slow feedback)
- `no_sli_definition`: SLI section missing or vague ("everything OK")
- `no_error_budget_policy`: no documented action when budget burns
- `cpu_as_sli`: CPU/memory used as user-experience proxy (wrong signal)
## SLI selection cheatsheet
| User experience | SLI type | What you measure |
|---|---|---|
| "Did the request succeed?" | request-success-rate | `2xx / total` |
| "Was the response fast?" | request-latency | `count(p99 < threshold) / total` |
| "Was the service up?" | availability-time | `(window - downtime) / window` |
| "Is the data current?" | data-freshness | `count(data_age < threshold) / total` |
| "Was the answer correct?" | correctness | `count(correct) / total` |
See `references/sli_design.md` for examples and anti-patterns.
## Error budget math (the basics)
For 99.9% SLO over 30 days:
- Allowed unavailability: `0.1% × 30 × 24 × 60 = 43.2 minutes`
- 1-hour fast-burn threshold (2% of monthly budget burned): `2% × 43.2 / 60 ≈ 1.44 ratio multiplier`
- 6-hour slow-burn threshold (10% in 6h): `10% × 43.2 / 360 ≈ 0.6 ratio multiplier`
`error_budget_calculator.py` does this math for you and emits ready-to-paste alert rules.
## Composition with the rest of the portfolio
This skill explicitly composes with three others:
| Skill | Composition |
|---|---|
| `feature-flags-architect` | Rollout abort criteria reference SLO burn-rate thresholds |
| `chaos-engineering` | Blast-radius calculator already takes monthly error budget as input — define it here |
| `kubernetes-operator` | Operator capability L4 (Deep Insights) requires SLOs + Prometheus rules |
The `error_budget_calculator.py` output is in the same shape `chaos-engineering/scripts/blast_radius_calculator.py` expects on stdin.
## Workflows
### Workflow 1: Define a new SLO
```
1. Pick the user journey to protect (e.g., "checkout completion").
2. Choose SLI type (request-success-rate, latency, availability, freshness, correctness).
3. Define the SLI precisely: numerator/denominator with concrete labels.
4. Pick a target by measuring 30 days of historical SLI value:
target = floor(p50 of last 30 days × 100) / 100
This avoids targets the system has never sustained.
5. Pick a window (28 days = 4 calendar weeks, recommended).
6. Run slo_designer.py to render the SLO definition.
7. Run error_budget_calculator.py to get burn-rate alerts.
8. Write the error budget policy (what happens when budget burns).
9. Run slo_review.py — must pass before the SLO is "live".
```
### Workflow 2: Quarterly SLO review
```
1. For every active SLO, run slo_review.py — fix any FAIL findings.
2. Look at last quarter's data:
- Was the SLO too easy (never burned budget)? Tighten target.
- Was it too hard (frequently burned)? Loosen target OR fix the system.
- Did burn-rate alerts fire usefully (not too noisy, not too late)? Adjust thresholds.
3. Audit error budget policies — were they actually followed when budget burned?
4. Commit revised SLOs; archive old versions with date stamps.
```
### Workflow 3: SLO-driven rollback
```
1. New deploy starts burning error budget faster than baseline.
2. Burn-rate alert fires (from error_budget_calculator.py thresholds).
3. Auto-rollback via feature flag (kill switch from feature-flags-architect).
4. Postmortem feeds into next SLO revision.
```
## References
- `references/slo_principles.md` — SLI vs SLO vs SLA, Google SRE Workbook canon
- `references/sli_design.md` — picking the right SLI; 5 types with examples
- `references/error_budget.md` — error budget math, burn-rate alerts, budget policy
- `references/composition.md` — how SLOs feed feature flags, chaos, operators
## Slash command
`/slo-design` — interactive SLO design wizard that runs all 3 tools.
## Asset templates
- `assets/slo_template.yaml` — fillable SLO YAML
- `assets/error_budget_policy.md` — fillable policy template
## Anti-patterns
- **99.99% on every endpoint** — copy-paste SLOs that nobody verified the system can sustain
- **CPU usage as SLI** — system metrics aren't user experience
- **Single-window burn-rate alert** — too noisy if 5-min, too slow if 30-day
- **No error budget policy** — burning budget means nothing without an action
- **SLOs without owners** — no one is responsible; they bit-rot
- **SLOs reviewed once a year** — system characteristics change faster than that
- **SLAs in the SLO doc** — different audience, different stakes; keep them separate
- **SLO target = SLA target** — SLO must be tighter (you should beat your contract before customers notice)
## Verifiable success
A team using this skill should achieve:
- 100% of SLOs pass `slo_review.py` with 0 FAIL findings
- Every SLO has a documented owner, error budget, burn-rate alerts, and policy
- Burn-rate alerts fire ≤2 times/month per SLO that's hit (signal, not noise)
- Mean time to detect SLO violation: <30 min (multi-window burn-rate alerts working)
- Quarterly SLO review happens every quarter (not annually)
FILE:assets/error_budget_policy.md
# Error budget policy — `<service-name>`
This policy says what changes when error budget is burned. Without it, the SLO is theater.
## Scope
Applies to: `<list of SLO IDs covered by this policy>`
Owner: `<team-name>`
Review cadence: quarterly
Last reviewed: `<YYYY-MM-DD>`
## States and actions
### State: HEALTHY (>50% budget remaining)
- Normal operation
- Ship features without extra friction
- Run chaos experiments per the standard cadence
- Roll out feature flags per standard plan
### State: CAUTION (25-50% budget remaining)
- Risky changes get extra review (architect or staff sign-off)
- No new chaos experiments outside dedicated windows
- Postpone non-essential migrations
- Daily team check on budget direction
### State: CRITICAL (<25% budget remaining)
- **Deploy freeze** for the affected service: only SLO-improving fixes ship
- All releases require **explicit owner sign-off**
- **Chaos experiments paused**
- **Feature flag rollouts paused** (existing flags continue at current percent)
- Daily standup includes budget status
### State: VIOLATED (budget exhausted, SLO target missed)
- Same-day: stop the bleeding (rollback, kill switch, scale up)
- Within 48 hours: blameless postmortem published
- Within 14 days: at least one follow-up action shipped
- Within 30 days: review whether SLO target/window are still right
## Recovery
After exiting VIOLATED, the service stays in CRITICAL until:
- Burn rate is sustained at <1× over 7 consecutive days, AND
- All postmortem follow-ups are shipped
## Roles
| Role | Responsibility |
|---|---|
| Service owner | Triggers state transitions; communicates to stakeholders |
| On-call | Receives burn-rate alerts; initial triage |
| Engineering manager | Approves deploys during CRITICAL/VIOLATED |
| SRE | Reviews SLO target appropriateness quarterly |
## Exceptions
The deploy freeze can be lifted by:
- Service owner + engineering manager joint approval
- Reason documented (security fix, customer escalation, regulatory)
- Logged for postmortem review
## Reviewing this policy
This policy is reviewed every quarter. Questions to ask:
1. Did we follow the policy when budget burned?
2. Are the thresholds (50% / 25%) right?
3. Are the actions (freeze, sign-off) actually happening?
4. Did the SLO target need to change?
Answers feed into the next quarter's revision.
## Composition references
- `references/composition.md` — how this policy interacts with feature-flags-architect, chaos-engineering, kubernetes-operator
- `references/error_budget.md` — the math behind the thresholds
- `references/slo_principles.md` — Google SRE Workbook canon
FILE:assets/slo_template.yaml
# SLO definition — fill in <PLACEHOLDERS>
# Pass this through slo_review.py before going live.
---
slo_id: slo-<service>-<sli_type>-<unix_ts>
service: <service-name> # e.g., checkout-svc
owner: <team-or-handle@org> # required; named individual or team
created: <YYYY-MM-DD>
review_cadence: quarterly # quarterly | monthly | weekly
# The user journey this SLO protects.
# Be specific. NOT "API works" — instead "User completes checkout in <2s".
user_journey: <describe the user journey>
# The SLI: a measurable signal of user-perceived health.
sli:
type: request-success-rate # request-success-rate | request-latency
# | availability-time | data-freshness | correctness
numerator: count(http_requests_total{job="<service>", status_code=~"2..|3.."})
denominator: count(http_requests_total{job="<service>", source!="bot"})
labels:
- env=prod
- region=us-east-1
# The target value the SLI must hit over the window.
# Pick from data: floor(p50 of last 30d × 100) / 100.
# Don't copy 99.9% blindly.
target_percent: 99.9
window_days: 28 # 7 / 28 / 30 / 90 — default 28
error_budget:
# Computed by error_budget_calculator.py — confirm the math.
minutes_per_window: <40.32 for 99.9% over 28 days>
# Path or URL to the error budget policy.
# The policy must answer: "When budget burns to 25% / 0%, what changes?"
policy_doc: <link required before SLO is live>
# Burn-rate alert thresholds, computed by error_budget_calculator.py.
# Multi-window per Google SRE Workbook Chapter 5.
alerts:
fast_burn:
long_window: 1h
short_window: 5m
burn_rate_threshold: <from error_budget_calculator.py>
severity: page
slow_burn:
long_window: 6h
short_window: 30m
burn_rate_threshold: <from error_budget_calculator.py>
severity: page
ticket_burn:
long_window: 3d
short_window: 6h
burn_rate_threshold: <from error_budget_calculator.py>
severity: ticket
# Composition with other skills.
# Wire-up with feature-flags-architect, chaos-engineering, kubernetes-operator
# is documented in references/composition.md.
references:
monitoring_dashboard: <URL>
policy_doc: <URL>
related_slos:
- <other-slo-id>
FILE:references/composition.md
# Composition with the rest of the portfolio
`slo-architect` is the keystone. Three other skills in this library already lean on the SLO + error budget concept. This page shows how to wire them together for a coherent reliability stack.
## The unified concept: error budget
```
┌────────────────────────────────────────────────────────────┐
│ slo-architect │
│ defines SLO, error budget, burn rate │
└──────────┬─────────────────┬────────────────┬─────────────┘
│ │ │
▼ ▼ ▼
feature-flags- chaos-engineering kubernetes-
architect (blast-radius operator
(rollout abort) bound by EB) (cap level L4)
```
## With feature-flags-architect
`feature-flags-architect` defines kill switches. Their abort triggers should reference SLO burn-rate, not arbitrary thresholds.
Before:
```
abort_if: "p99 > 1000ms OR error_rate > 1%"
```
After (SLO-driven):
```
abort_if: "burn_rate.fast > 14.4 over 1h (per SLO checkout-success)"
```
Wire-up:
1. Define SLO via `slo_designer.py`
2. Run `error_budget_calculator.py` to get the burn-rate threshold
3. Use that threshold in the flag's abort criteria
4. The kill_switch_audit.py from feature-flags-architect now has a real signal to verify against
## With chaos-engineering
`chaos-engineering`'s `blast_radius_calculator.py` already takes monthly error budget as input — but the budget should come from the SLO, not be made up.
```bash
# 1. Get the budget from the SLO definition
python slo_architect/scripts/error_budget_calculator.py \
--target 99.9 --window-days 30 --format json \
| jq .budget_minutes
# 2. Pass it to the chaos blast-radius calculator
python chaos_engineering/scripts/blast_radius_calculator.py \
--traffic-share 0.05 \
--user-pop 1000000 \
--duration-min 15 \
--monthly-budget-min 43.2 # ← from step 1
```
Now blast radius is bounded by REAL error budget, not a number someone typed in.
## With kubernetes-operator
OperatorHub Capability Level 4 ("Deep Insights") requires:
- `/metrics` endpoint
- Prometheus alert rules
- SLOs documented for the operator's managed resources
`slo-architect` provides the SLO definitions; `error_budget_calculator.py` provides the alert rules. Drop them in the operator's Helm chart or OperatorHub bundle.
## End-to-end example
Goal: ship a new checkout flow.
1. **Define the SLO** (slo-architect):
```bash
slo_designer.py --service checkout-svc --sli-type request-success-rate \
--target 99.9 --window-days 28 --owner team-checkout
```
2. **Compute burn-rate alerts** (slo-architect):
```bash
error_budget_calculator.py --target 99.9 --window-days 28
# → fast_burn threshold = 14.4
```
3. **Define rollout** (feature-flags-architect):
```bash
rollout_planner.py --population 100000 --target-percent 100 \
--duration-days 14 --strategy ring
# 1% → 5% → 25% → 50% → 100%
```
4. **Wire the abort** (feature-flags-architect):
```yaml
abort_if: "burn_rate.fast > 14.4 (per SLO slo-checkout-svc-...)"
```
5. **Validate via chaos** before going wide (chaos-engineering):
```bash
blast_radius_calculator.py --traffic-share 0.05 --user-pop 100000 \
--duration-min 15 --monthly-budget-min 40.32
# → GREEN if <1% of monthly budget
```
6. **Audit the operator** if the service is operator-managed (kubernetes-operator):
```bash
operator_capability_audit.py --operator-dir ./checkout-operator
# → confirm L4 includes the new SLO
```
Each step uses the previous step's output as input. The SLO is the unifying number.
## What slo-architect does NOT replace
- **observability-designer** — broader observability strategy (metrics, logs, traces, dashboards beyond SLO)
- **incident-response** — SLO violation may trigger an incident, but incident response is a separate discipline
- **performance-profiler** — capacity planning needs different metrics than SLO does
Use slo-architect for SLO+error-budget; use the others for their specific scopes.
## Anti-pattern: SLO without composition
A team defines SLOs in a spreadsheet. Nobody references them in:
- Feature flag rollouts
- Chaos experiment design
- Operator capability audits
- Incident postmortems
The SLOs become a reporting artifact, not an operating tool. The composition story is what makes SLOs change behavior.
## Operational checklist
For any service with a new SLO, verify:
- [ ] SLO defined via `slo_designer.py` (`slo_review.py` passes)
- [ ] Burn-rate alerts deployed via `error_budget_calculator.py` output
- [ ] If using feature flags: rollout abort references the SLO burn-rate threshold
- [ ] If running chaos: blast radius bounded by SLO error budget
- [ ] If operator-managed: operator audit confirms L4 includes the SLO
- [ ] Postmortem template (when SLO violated) includes "SLO revision needed?" question
FILE:references/error_budget.md
# Error budget
The most important number in your SLO.
## Computation
```
error_budget_fraction = 1 − (target_percent / 100)
error_budget_minutes = error_budget_fraction × window_days × 24 × 60
error_budget_requests = error_budget_fraction × total_requests_in_window
```
## Reference table
| SLO target | 7-day budget (min) | 28-day budget (min) | 30-day budget (min) | 90-day budget (min) |
|---|---|---|---|---|
| 99% | 100.8 | 403.2 | 432 | 1296 |
| 99.5% | 50.4 | 201.6 | 216 | 648 |
| 99.9% | 10.08 | 40.32 | 43.2 | 129.6 |
| 99.95% | 5.04 | 20.16 | 21.6 | 64.8 |
| 99.99% | 1.008 | 4.032 | 4.32 | 12.96 |
| 99.999% | 0.1008 | 0.4032 | 0.432 | 1.296 |
99.999% over 30 days = 26 seconds of allowed downtime. Sustainable only with multi-region, sub-second failover, dedicated SRE team.
## Burn-rate alerts (Google SRE Workbook canon)
The single most useful artifact this skill produces. From Chapter 5: "Alerting on SLOs."
### Why multi-window
Single-window alerts fail in opposite directions:
| Window | Failure mode |
|---|---|
| 5 minutes | Fires on every blip; alert fatigue |
| 30 days | Fires when budget is already exhausted; too late |
| 1 hour alone | Fires too often; misses sustained slow burn |
Multi-window combines:
- **Long window** filters noise
- **Short window** speeds detection
The alert fires only when BOTH windows show high burn. This filters spikes (only short window high) and only fires on sustained burn (both windows high).
### Recommended thresholds
| Alert | Long window | Short window | Burn rate threshold | % budget at fire | Severity |
|---|---|---|---|---|---|
| Fast burn | 1h | 5m | 14.4 | 2% in 1h | page |
| Slow burn | 6h | 30m | 6 | 5% in 6h | page |
| Ticket | 3d | 6h | 1 | 10% in 3d | ticket |
The numbers come from: `burn_rate × bad_event_rate > slo_target_violation_rate`.
`error_budget_calculator.py` computes these for any target+window. Output is PromQL-shaped:
```promql
# fast_burn (page)
# Burn rate threshold: 14.4
(
sli:rate1h > 14.4 * (1 - 0.999)
AND
sli:rate5m > 14.4 * (1 - 0.999)
)
```
Paste into your Prometheus rules; adjust label selectors to match your environment.
## Error budget policy
A policy without consequences is theater. The policy says: **"When budget is in state X, action Y happens automatically."**
### Standard 4-state policy
| State | Trigger | Action |
|---|---|---|
| **Healthy** | >50% budget remaining | Normal operation; ship features, run experiments |
| **Caution** | 25-50% budget remaining | Reduce risk on changes; no chaos experiments |
| **Critical** | <25% remaining | Freeze risky deploys; reliability work prioritized |
| **Violated** | Budget exhausted | Postmortem; SLO revision; blameless review |
### What "freeze" means
Specifically:
- No deploys to production except for SLO-improving fixes
- All releases require explicit owner sign-off
- Chaos experiments paused
- Feature flag rollouts paused
This is real, not aspirational. Engineering teams that don't follow through erode the credibility of the SLO.
### Recovery path
After SLO is violated:
1. Same-day: stop bleeding (rollback, kill switch, scale up)
2. Within 48h: postmortem published
3. Within 14 days: at least one follow-up action shipped
4. At 30 days: review whether SLO is still right
If burns are frequent, the SLO is wrong (too tight) OR the system needs investment.
## Burn-rate vs uptime alerting
Old-school: "Page if any 5xx rate >5%."
New-school: "Page if budget burns 14.4× faster than sustainable."
Why burn-rate is better:
- Stays calibrated as traffic grows (5% of low traffic = noise; of high traffic = real)
- Auto-adjusts for SLO target (99.99% needs sharper alerts than 99%)
- Aligns alerts with the SLO they protect
## When to skip burn-rate alerts
- For SLOs that aren't "always on" (batch jobs, async pipelines) — measure SLI per execution instead
- For SLOs in development (no historical data yet)
- For internal tools where ticket-only is enough — don't page the team for non-paging issues
## The error budget conversation
The SLO + error budget is meant to enable a conversation, not replace it.
> Engineering: "We want to ship the new payment provider this sprint."
> SRE: "We're at 35% budget remaining for the month. If this rolls back twice, we'll exhaust it."
> Eng: "Fine, we'll ship behind a feature flag and ramp 1% → 5% → 50% with a 24-hour bake at each stage."
> SRE: "OK. Set the flag's auto-abort to fire on the burn-rate alert."
That's the conversation the SLO + budget enables. Without numbers, both sides argue from gut feel.
FILE:references/sli_design.md
# SLI design
The SLI is the foundation. Get it wrong and the SLO is meaningless — green dashboard, angry users.
## The user-experience test
Before defining ANY SLI, answer:
> When this signal turns red, will a user notice?
If the answer is "maybe" or "depends," it's not an SLI — it's an internal metric.
| Signal | User notices? | Use as SLI? |
|---|---|---|
| HTTP 5xx rate | Yes | YES |
| p99 latency at the user's edge | Yes | YES |
| Successful login rate | Yes | YES |
| CPU usage on backend | No | NO |
| Memory usage on backend | No | NO |
| Pod restart count | No (until it's too late) | NO |
| Database query duration | Indirect | Maybe (if it dominates user latency) |
CPU and memory are LEADING indicators of trouble — useful for capacity planning, useless for SLO.
## The 5 SLI types
### 1. Request-success-rate (most common)
Numerator: "good" requests
Denominator: total requests
```
sli = (total - 5xx - timeouts - protocol_errors) / total
```
Use when:
- Service is request-driven (HTTP, gRPC, queue handler)
- Each request is independent
- Success/failure is well-defined
Edge cases:
- 4xx is usually NOT counted as bad (they're client errors), EXCEPT 429 (rate limiting) and 401/403 if those are operator-caused
- Time out at p99 of expected latency; treat anything beyond as bad
- Cancelled requests are tricky — define explicitly
### 2. Request-latency
Numerator: requests with latency below threshold
Denominator: total requests
```
sli = count(latency_p99 < 500ms) / count(all)
```
Use when:
- Performance is part of user experience (most user-facing services)
- A success that takes 30 seconds is effectively a failure
Pick the threshold from data: measure p50/p95/p99 over 30 days, then set the threshold at p95 of typical good operation.
### 3. Availability-time
Numerator: window minus total downtime
Denominator: window length
```
sli = (window - sum(downtime_seconds)) / window
```
Use when:
- Service is "always-on" (DNS, infrastructure, control plane)
- "Up" or "down" is binary
- No clear request unit
Define "up" precisely: is one health check failure "down"? Three consecutive? Per-region or per-cluster?
### 4. Data-freshness
Numerator: data points younger than threshold
Denominator: total data points
```
sli = count(data_age < 5min) / count(all_data)
```
Use when:
- Service's value depends on recency (analytics dashboards, fraud detection, search index)
- "Stale data" is the user-facing failure mode
### 5. Correctness
Numerator: outputs that are correct
Denominator: total outputs
```
sli = count(correct_predictions) / count(predictions)
```
Use when:
- Output quality matters more than speed (ML models, search ranking, fraud scoring)
- You have ground truth (labels, customer feedback, A/B comparison)
Hardest SLI to maintain because "correct" requires labeled data.
## SLI vs SLO target — concrete examples
### Example 1: Checkout API
- **SLI:** `(2xx + 3xx requests) / total requests`, excluding 4xx (client errors)
- **SLO target:** 99.9% over 28 days
- **Error budget:** 40.32 minutes/window of unavailability
### Example 2: Search latency
- **SLI:** `count(latency < 200ms) / count(all_searches)`
- **SLO target:** 99.5% over 28 days
- **Error budget:** 3.36 hours/window where >0.5% of queries are slow
### Example 3: Internal API uptime
- **SLI:** `(window - downtime) / window`, downtime measured by pingdom-style probes
- **SLO target:** 99% over 28 days
- **Error budget:** 6.72 hours/window of allowed outage
## Common SLI mistakes
### "We just count errors"
Errors are useful but incomplete. A request that returns 200 OK in 30 seconds is a failure even though it's not an error. Use latency SLI for performance-sensitive services.
### Conflating SLIs across user journeys
If checkout and browsing are different user experiences, they get different SLIs. A 99.9% on "the API" averages over journeys with very different criticality.
### Counting bot traffic
Bots can dominate request volume. Filter them out (or have a separate SLI for them) — your error budget shouldn't be spent on synthetic traffic.
### Counting internal traffic
If your service is hit by other internal services, those requests have different reliability requirements than user requests. Separate SLIs.
### Using ratios that go backward
```
WRONG: sli = errors / total
(lower is better — confusing)
RIGHT: sli = (total - errors) / total
(higher is better, matches SLO target convention)
```
## Defining the numerator/denominator precisely
Every SLI must specify:
1. **What's being counted** (requests? events? checks?)
2. **What "good" means** (the numerator filter)
3. **What's excluded** (filters: bot traffic, internal traffic, health checks, etc.)
4. **Where it's measured** (LB? service edge? client side?)
Bad: "request success rate"
Good: `count(http_requests_total{job="checkout-api", status_code=~"2..|3.."}) / count(http_requests_total{job="checkout-api", source!="bot"})`
The second one is testable, debuggable, and unambiguous.
## Review the SLI as the system evolves
System change → SLI change. When:
- A new failure mode appears (e.g., circuit breaker that returns 5xx) → update what's "bad"
- A dependency moves (e.g., from synchronous to async) → re-examine what users feel
- A new endpoint is added → does it belong in this SLO or its own?
Stale SLIs are worse than no SLIs — they create false confidence.
FILE:references/slo_principles.md
# SLO principles
The Google SRE Workbook canon, distilled to what matters in practice.
## SLI vs SLO vs SLA
| Term | What it is | Audience | Stakes |
|---|---|---|---|
| **SLI** (Service Level Indicator) | A measurable signal of user-perceived health (e.g., HTTP success rate) | Engineering | None directly — it's the input |
| **SLO** (Service Level Objective) | A target value or range for the SLI over a window (e.g., 99.9% over 28 days) | Engineering, internal | Engineering action when burning budget |
| **SLA** (Service Level Agreement) | A customer-facing commitment with consequences (refunds, credits) | Customers, legal, sales | Contractual; costs money to break |
**Cardinal rule:** SLA target < SLO target < SLI baseline.
If SLA = 99.9%, SLO must be tighter (e.g., 99.95%) so engineering action triggers BEFORE customer-impacting violation.
## The error budget
```
error_budget = 100% − SLO_target
For 99.9% SLO over 30 days:
error_budget = 0.1% × 30d × 24h × 60min = 43.2 minutes/month
That's the maximum unavailability you can spend without violating SLO.
```
The whole point of SLOs: error budget makes reliability a numeric resource you can spend deliberately. Spending it on:
- New feature rollouts (some risk)
- Chaos experiments (intentional learning)
- Migrations (necessary instability)
is GOOD. Wasting it on:
- Avoidable bugs
- Bad deploys
- Unmonitored regressions
is BAD. Error budget reframes "should we ship this?" from gut feel to a budget question.
## Multi-window burn-rate alerts (the canon)
Google SRE Workbook Chapter 5: "Alerting on SLOs." The recommended structure:
| Alert | Long window | Short window | % budget burned | Severity |
|---|---|---|---|---|
| Fast burn | 1h | 5m | 2% | page |
| Slow burn | 6h | 30m | 5% | page |
| Ticket burn | 3d | 6h | 10% | ticket (no page) |
Why two windows per alert?
- **Long window** filters noise (random spikes don't fire)
- **Short window** speeds detection (alert fires the moment burn is sustained)
Single-window burn-rate alerts are either too noisy (5-min only) or too slow (30-day only).
The `error_budget_calculator.py` tool emits these thresholds for any target+window combination.
## Choosing a target
Bad: copy-paste 99.9% on every endpoint.
Good: measure 30 days of historical SLI, then:
```
target = floor(p50 of last 30 days × 100) / 100
```
This guarantees the system has actually sustained the target. Tightening later is fine; loosening after announcing a target is embarrassing.
**Reality-check ranges:**
| User-perceived service | Typical target |
|---|---|
| Internal tool, occasional use | 99% |
| Standard customer-facing app | 99.9% |
| Commerce / payments | 99.95% |
| Critical infrastructure | 99.99% |
| Hyperscale (Google, AWS) | 99.999% (and only for tiny scope) |
99.99%+ requires multi-region, automatic failover, no single points of failure, and a team paid to maintain that. Don't write it on a whim.
## Choosing a window
| Window | Use when | Trade-off |
|---|---|---|
| 7 days | Need fast feedback; system changes weekly | High noise, fast learning |
| 28 days | Default for most services | Balanced |
| 30 days | Calendar-month aligned (board reports) | Slightly more noise than 28 |
| 90 days | Slow-changing systems, contract reporting | Too slow for engineering feedback |
28 days = 4 calendar weeks. Recommended unless you have a specific reason otherwise.
## Error budget policy (the missing half)
An SLO without a policy is a wish. The policy answers:
> When the error budget is burned, what changes?
Standard policy options:
| State | Action |
|---|---|
| Budget healthy (>50% remaining) | Normal operation; ship features, run experiments |
| Budget at 50% | Heightened review on risky changes |
| Budget exhausted (<10%) | Freeze risky deploys; focus on reliability work |
| Budget violated | Postmortem; SLO revision; blameless review |
Without an agreed policy, burning budget is just a number.
## SLO ownership
Every SLO has exactly one owning team. The owner is responsible for:
- Keeping the SLI definition correct as the system evolves
- Making sure burn-rate alerts route to the right team
- Quarterly review and revision
- Writing the postmortem when SLO is violated
Without an owner, SLOs bit-rot (SLI definitions drift, alerts route to wrong teams, reviews never happen).
## When NOT to define an SLO
- For internal tooling that breaks rarely and doesn't gate revenue
- For experimental features that may be removed in 30 days
- For systems where you can't measure user experience (revisit when you can)
- As performance theater — measuring without acting on burn
## Review cadence
- **Quarterly** — minimum for any active SLO
- **Monthly** — recommended for systems under active development
- **Weekly** — only during incident-recovery windows
The point of review: "is this SLO still right?" Tightening, loosening, or removing an SLO is a normal outcome. SLOs are not contracts; they are calibration knobs.
## Reading
- *Google SRE Workbook* (Beyer, Murphy, Rensin et al.) — Chapter 2 (SLO design), Chapter 5 (alerting on SLOs). Free at sre.google/workbook.
- *Implementing Service Level Objectives* (Alex Hidalgo) — covers operationalization beyond Google's frame.
- The SLO Reference Architecture (slo.dev) — community-maintained.
FILE:scripts/error_budget_calculator.py
#!/usr/bin/env python3
"""Compute error budget and multi-window burn-rate alert thresholds.
Per Google SRE Workbook (Chapter 5: Alerting on SLOs), reliable burn-rate
alerting uses TWO windows: a fast window (1h) for catastrophic burn and a
slow window (6h) to filter false positives. Optionally a 3-day window for
ticket-only (non-paging) alerts.
Outputs:
- Allowed downtime in the SLO window
- Burn-rate thresholds for fast/slow/ticket alert windows
- PromQL-shaped alert rules ready to paste
References:
https://sre.google/workbook/alerting-on-slos/
"""
import argparse
import json
import sys
# Per Google SRE Workbook Chapter 5: Table 5-3 recommended thresholds
# (severity, percent_of_monthly_budget, long_window, short_window_ratio)
DEFAULT_BURN_RATE_RULES = [
{
"name": "fast_burn",
"severity": "page",
"long_window_hours": 1,
"short_window_hours": 1 / 12,
"budget_pct_consumed": 2.0,
"rationale": "2% of monthly budget burned in 1h => system on fire",
},
{
"name": "slow_burn",
"severity": "page",
"long_window_hours": 6,
"short_window_hours": 0.5,
"budget_pct_consumed": 5.0,
"rationale": "5% of monthly budget burned in 6h => sustained degradation",
},
{
"name": "ticket_burn",
"severity": "ticket",
"long_window_hours": 72,
"short_window_hours": 6,
"budget_pct_consumed": 10.0,
"rationale": "10% of monthly budget burned in 3d => trending bad",
},
]
def compute(target_percent, window_days):
if not 50 <= target_percent <= 100:
raise ValueError(f"target must be between 50 and 100, got {target_percent}")
if window_days < 1:
raise ValueError("window-days must be >= 1")
bad_fraction = (100 - target_percent) / 100
window_minutes = window_days * 24 * 60
budget_minutes = round(bad_fraction * window_minutes, 4)
rules = []
for rule in DEFAULT_BURN_RATE_RULES:
burn_rate_threshold = (rule["budget_pct_consumed"] / 100) / (rule["long_window_hours"] / (window_days * 24))
rules.append({
"name": rule["name"],
"severity": rule["severity"],
"long_window": _fmt_hours(rule["long_window_hours"]),
"short_window": _fmt_hours(rule["short_window_hours"]),
"budget_pct_consumed": rule["budget_pct_consumed"],
"burn_rate_threshold": round(burn_rate_threshold, 3),
"rationale": rule["rationale"],
"promql": _promql_rule(rule, burn_rate_threshold, target_percent),
})
return {
"target_percent": target_percent,
"window_days": window_days,
"bad_fraction": round(bad_fraction, 6),
"budget_minutes": budget_minutes,
"budget_hours": round(budget_minutes / 60, 4),
"alert_rules": rules,
}
def _fmt_hours(hours):
if hours < 1:
return f"{int(round(hours * 60))}m"
if hours < 24:
return f"{int(round(hours))}h"
return f"{int(round(hours / 24))}d"
def _promql_rule(rule, burn_rate, target_pct):
long_w = _fmt_hours(rule["long_window_hours"])
short_w = _fmt_hours(rule["short_window_hours"])
return (
f"# {rule['name']} ({rule['severity']})\n"
f"# Burn rate threshold: {round(burn_rate, 3)}\n"
f"(\n"
f" sli:rate{long_w} > {round(burn_rate, 3)} * (1 - {target_pct / 100})\n"
f" AND\n"
f" sli:rate{short_w} > {round(burn_rate, 3)} * (1 - {target_pct / 100})\n"
f")"
)
def render_text(result):
print(f"Error Budget — target={result['target_percent']}%, window={result['window_days']}d")
print("=" * 60)
print(f"Allowed bad events: {result['bad_fraction'] * 100:.4f}% of total")
print(f"Allowed downtime: {result['budget_minutes']:.2f} min ({result['budget_hours']:.2f} hours)")
print("")
print("Multi-window burn-rate alerts (Google SRE Workbook):")
print("")
for r in result["alert_rules"]:
print(f" [{r['severity'].upper():6}] {r['name']}")
print(f" windows: {r['long_window']} long / {r['short_window']} short")
print(f" burn rate: {r['burn_rate_threshold']}")
print(f" consumed: {r['budget_pct_consumed']}% of monthly budget")
print(f" rationale: {r['rationale']}")
print("")
print("PromQL-shaped rules:")
print("")
for r in result["alert_rules"]:
print(r["promql"])
print("")
def main():
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--target", type=float, required=True, help="Target percent (e.g., 99.9)")
ap.add_argument("--window-days", type=int, default=28, help="Window in days (default: 28)")
ap.add_argument("--format", choices=["text", "json"], default="text")
args = ap.parse_args()
try:
result = compute(args.target, args.window_days)
except ValueError as e:
print(f"ERROR: {e}", file=sys.stderr)
return 2
if args.format == "json":
print(json.dumps(result, indent=2))
else:
render_text(result)
return 0
if __name__ == "__main__":
sys.exit(main())
FILE:scripts/slo_designer.py
#!/usr/bin/env python3
"""Generate a structured SLO definition.
Enforces required fields (service, SLI type + definition, target, window,
owner, error budget policy reference). Refuses to render if required fields
are missing — exit 1 forces the caller to provide them.
Output is markdown by default. JSON output is consumed by slo_review.py.
"""
import argparse
import json
import sys
from datetime import datetime, timezone
SLI_TYPES = {
"request-success-rate": {
"numerator": "count(http_requests_total{status=~\"2..|3..\"})",
"denominator": "count(http_requests_total)",
"user_question": "Did the request succeed?",
},
"request-latency": {
"numerator": "count(http_request_duration_seconds < 0.5)",
"denominator": "count(http_request_duration_seconds)",
"user_question": "Was the response fast enough?",
},
"availability-time": {
"numerator": "(window_seconds - sum(up_down_seconds))",
"denominator": "window_seconds",
"user_question": "Was the service up?",
},
"data-freshness": {
"numerator": "count(data_age_seconds < freshness_threshold)",
"denominator": "count(data_age_seconds)",
"user_question": "Is the data current?",
},
"correctness": {
"numerator": "count(correct_outputs)",
"denominator": "count(total_outputs)",
"user_question": "Was the answer correct?",
},
}
def build_slo(args):
sli_meta = SLI_TYPES.get(args.sli_type, {})
slo = {
"slo_id": f"slo-{args.service}-{args.sli_type}-{int(datetime.now(timezone.utc).timestamp())}",
"created": datetime.now(timezone.utc).isoformat(),
"service": args.service,
"owner": args.owner or "<must define before SLO is live>",
"user_journey": args.user_journey or f"<{sli_meta.get('user_question', 'describe the user journey this SLO protects')}>",
"sli": {
"type": args.sli_type,
"numerator": args.sli_numerator or sli_meta.get("numerator", "<must define>"),
"denominator": args.sli_denominator or sli_meta.get("denominator", "<must define>"),
"labels": args.sli_labels.split(",") if args.sli_labels else [],
},
"target_percent": args.target,
"window_days": args.window_days,
"error_budget": {
"minutes_per_window": _budget_minutes(args.target, args.window_days),
"policy_doc": args.policy_doc or "<link to error budget policy required before SLO is live>",
},
"alerts": {
"fast_burn_threshold": "see error_budget_calculator.py",
"slow_burn_threshold": "see error_budget_calculator.py",
},
"review_cadence": args.review_cadence,
}
return slo
def _budget_minutes(target_pct, window_days):
bad_fraction = max(0.0, (100 - target_pct) / 100)
return round(bad_fraction * window_days * 24 * 60, 2)
def _missing_required(slo):
missing = []
if not slo["owner"] or slo["owner"].startswith("<"):
missing.append("owner")
if not slo["error_budget"]["policy_doc"] or slo["error_budget"]["policy_doc"].startswith("<"):
missing.append("error_budget.policy_doc")
if slo["sli"]["numerator"].startswith("<") or slo["sli"]["denominator"].startswith("<"):
missing.append("sli.numerator/denominator")
return missing
def render_markdown(slo):
lines = []
lines.append(f"# SLO: {slo['slo_id']}")
lines.append("")
lines.append(f"- **Service:** `{slo['service']}`")
lines.append(f"- **Owner:** {slo['owner']}")
lines.append(f"- **Created:** {slo['created']}")
lines.append(f"- **User journey:** {slo['user_journey']}")
lines.append("")
lines.append("## SLI")
lines.append(f"- **Type:** {slo['sli']['type']}")
lines.append(f"- **Numerator:** `{slo['sli']['numerator']}`")
lines.append(f"- **Denominator:** `{slo['sli']['denominator']}`")
if slo["sli"]["labels"]:
lines.append(f"- **Labels:** {', '.join(slo['sli']['labels'])}")
lines.append("")
lines.append("## Target")
lines.append(f"- **Target:** {slo['target_percent']}% over {slo['window_days']} days")
lines.append(f"- **Error budget:** {slo['error_budget']['minutes_per_window']} minutes per window")
lines.append(f"- **Policy:** {slo['error_budget']['policy_doc']}")
lines.append("")
lines.append("## Alerts")
lines.append("Run `error_budget_calculator.py --target {} --window-days {}` for burn-rate thresholds.".format(
slo["target_percent"], slo["window_days"]
))
lines.append("")
lines.append(f"## Review cadence: {slo['review_cadence']}")
return "\n".join(lines)
def main():
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--service", required=True, help="Service name (e.g., checkout-svc)")
ap.add_argument("--sli-type", required=True, choices=list(SLI_TYPES.keys()))
ap.add_argument("--target", type=float, required=True, help="Target percent (e.g., 99.9)")
ap.add_argument("--window-days", type=int, default=28, help="Compliance window in days (default: 28)")
ap.add_argument("--user-journey", help="The user journey this SLO protects")
ap.add_argument("--sli-numerator", help="Override default SLI numerator expression")
ap.add_argument("--sli-denominator", help="Override default SLI denominator expression")
ap.add_argument("--sli-labels", help="Comma-separated labels (e.g., env=prod,region=us-east-1)")
ap.add_argument("--owner", help="Owning team / handle")
ap.add_argument("--policy-doc", help="URL or path to error budget policy")
ap.add_argument("--review-cadence", default="quarterly", help="How often to review (default: quarterly)")
ap.add_argument("--format", choices=["markdown", "json"], default="markdown")
args = ap.parse_args()
if not 50 <= args.target <= 100:
print(f"ERROR: --target must be between 50 and 100, got {args.target}", file=sys.stderr)
return 2
if args.window_days < 1:
print(f"ERROR: --window-days must be >= 1", file=sys.stderr)
return 2
slo = build_slo(args)
missing = _missing_required(slo)
if args.format == "json":
print(json.dumps(slo, indent=2))
else:
print(render_markdown(slo))
if missing:
print("")
print(f"WARNING: missing required fields: {', '.join(missing)}", file=sys.stderr)
print("SLO is NOT live until these are filled.", file=sys.stderr)
return 1 if missing else 0
if __name__ == "__main__":
sys.exit(main())
FILE:scripts/slo_review.py
#!/usr/bin/env python3
"""Audit existing SLO definitions for the common bugs.
Reads markdown or JSON SLO docs and reports:
FAIL — definitely wrong (target ≥ 99.99 with no engineering investment plan,
no SLI definition, no error budget policy, CPU-as-SLI)
WARN — probably wrong (target ≤ 99.0, window outside 7-90 days)
Use as a pre-merge gate before SLOs go live.
"""
import argparse
import json
import os
import re
import sys
CPU_AS_SLI_PATTERNS = [
r"\bcpu_usage\b",
r"\bcpu_utilization\b",
r"\bmemory_usage\b",
r"\bmem_used\b",
r"\bdisk_usage\b",
r"\bdisk_full\b",
]
SLI_KEYWORDS = ("numerator", "denominator", "sli")
POLICY_KEYWORDS = ("policy", "error_budget", "error budget")
def _read(path):
try:
with open(path, "r", encoding="utf-8", errors="replace") as f:
return f.read()
except OSError:
return ""
def _parse_target(text):
m = re.search(r"target[:\s\"]+(\d+(?:\.\d+)?)\s*%?", text, re.IGNORECASE)
if m:
return float(m.group(1))
return None
def _parse_window_days(text):
m = re.search(r"window[_\-\s]?days?[:\s\"]+(\d+)", text, re.IGNORECASE)
if m:
return int(m.group(1))
m = re.search(r"window[:\s\"]+(\d+)\s*days?", text, re.IGNORECASE)
if m:
return int(m.group(1))
return None
def _has_any(text, keywords):
low = text.lower()
return any(k in low for k in keywords)
def _has_cpu_as_sli(text):
for pat in CPU_AS_SLI_PATTERNS:
if re.search(pat, text, re.IGNORECASE):
return True
return False
def audit_one(path):
text = _read(path)
findings = []
target = _parse_target(text)
window_days = _parse_window_days(text)
if target is None:
findings.append(("FAIL", "no_target", "no SLO target (X%) found in document"))
else:
if target >= 99.99:
findings.append(("FAIL", "target_too_high",
f"target {target}% ≥ 99.99% — sustainable only with massive engineering investment; document the investment plan or lower"))
elif target <= 99.0:
findings.append(("WARN", "target_too_low",
f"target {target}% ≤ 99% — likely wrong SLI; users will notice"))
if window_days is None:
findings.append(("WARN", "no_window", "no compliance window found"))
else:
if window_days < 7:
findings.append(("FAIL", "window_too_short",
f"window {window_days}d < 7d — statistical noise dominates"))
elif window_days > 90:
findings.append(("WARN", "window_too_long",
f"window {window_days}d > 90d — feedback too slow"))
if not _has_any(text, SLI_KEYWORDS):
findings.append(("FAIL", "no_sli_definition",
"no SLI definition (numerator/denominator) found"))
if not _has_any(text, POLICY_KEYWORDS):
findings.append(("FAIL", "no_error_budget_policy",
"no error budget policy reference found"))
if _has_cpu_as_sli(text):
findings.append(("FAIL", "cpu_as_sli",
"CPU/memory/disk-usage referenced — system metrics aren't user experience; pick a request-level SLI"))
return findings
def _walk(target):
if os.path.isfile(target):
yield target
return
for r, _, files in os.walk(target):
for f in files:
if f.endswith((".md", ".json", ".yaml", ".yml")):
yield os.path.join(r, f)
def audit(target):
results = []
for path in _walk(target):
findings = audit_one(path)
if findings:
results.append({"path": path, "findings": findings})
return results
def render_text(results):
fails = sum(1 for r in results for f in r["findings"] if f[0] == "FAIL")
warns = sum(1 for r in results for f in r["findings"] if f[0] == "WARN")
print(f"SLO Review — {len(results)} doc(s) with findings, {fails} FAIL, {warns} WARN")
print("")
if not results:
print("PASS: no issues detected.")
return 0
for r in results:
print(f"== {r['path']}")
for level, key, msg in r["findings"]:
print(f" [{level}] {key}: {msg}")
print("")
return 1 if fails else 0
def main():
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--slo-doc", required=True, help="Path to SLO doc or directory of docs")
ap.add_argument("--format", choices=["text", "json"], default="text")
args = ap.parse_args()
if not os.path.exists(args.slo_doc):
print(f"ERROR: not found: {args.slo_doc}", file=sys.stderr)
return 2
results = audit(args.slo_doc)
if args.format == "json":
print(json.dumps(results, indent=2))
return 1 if any(f[0] == "FAIL" for r in results for f in r["findings"]) else 0
return render_text(results)
if __name__ == "__main__":
sys.exit(main())
Chất vấn kế hoạch dựa trên thuật ngữ dự án (CONTEXT.md) và các quyết định đã ghi (docs/adr/), cập nhật các tệp này khi chốt thuật ngữ.
---
name: grill-with-docs
description: Docs-anchored grilling session — challenges a plan against the project's existing language (CONTEXT.md) and recorded decisions (docs/adr/), and updates those files inline as terminology and decisions crystallise. Use when user wants to stress-test a plan against documented domain language, or mentions "grill with docs".
license: MIT
metadata:
derived_from: "https://github.com/mattpocock/skills/tree/main/skills/engineering/grill-with-docs"
original_author: "Matt Pocock (@mattpocock)"
original_license: MIT
voice: "Matt Pocock — relentless, one-at-a-time, codebase-and-docs-first, ADRs only when 3 criteria are met"
version: 1.0.0
---
# Grill with Docs
> Derived from [Matt Pocock's grill-with-docs](https://github.com/mattpocock/skills/tree/main/skills/engineering/grill-with-docs) (MIT, © 2026 Matt Pocock). Matt's interview discipline + docs-anchored grilling rules preserved verbatim under MIT. Additions in this repo: 3 stdlib validators (CONTEXT.md linter, ADR scanner, glossary↔code consistency check), 3 in-depth references each citing 7+ authoritative sources, `cs-grill-with-docs` agent, `/cs:grill-with-docs` command. See [Wrapper additions](#wrapper-additions) below.
<what-to-do>
Interview me relentlessly about every aspect of this plan until we reach a shared understanding. Walk down each branch of the design tree, resolving dependencies between decisions one-by-one. For each question, provide your recommended answer.
Ask the questions one at a time, waiting for feedback on each question before continuing.
If a question can be answered by exploring the codebase, explore the codebase instead.
</what-to-do>
<supporting-info>
## Domain awareness
During codebase exploration, also look for existing documentation:
### File structure
Most repos have a single context:
```
/
├── CONTEXT.md
├── docs/
│ └── adr/
│ ├── 0001-event-sourced-orders.md
│ └── 0002-postgres-for-write-model.md
└── src/
```
If a `CONTEXT-MAP.md` exists at the root, the repo has multiple contexts. The map points to where each one lives:
```
/
├── CONTEXT-MAP.md
├── docs/
│ └── adr/ ← system-wide decisions
├── src/
│ ├── ordering/
│ │ ├── CONTEXT.md
│ │ └── docs/adr/ ← context-specific decisions
│ └── billing/
│ ├── CONTEXT.md
│ └── docs/adr/
```
Create files lazily — only when you have something to write. If no `CONTEXT.md` exists, create one when the first term is resolved. If no `docs/adr/` exists, create it when the first ADR is needed.
## During the session
### Challenge against the glossary
When the user uses a term that conflicts with the existing language in `CONTEXT.md`, call it out immediately. "Your glossary defines 'cancellation' as X, but you seem to mean Y — which is it?"
### Sharpen fuzzy language
When the user uses vague or overloaded terms, propose a precise canonical term. "You're saying 'account' — do you mean the Customer or the User? Those are different things."
### Discuss concrete scenarios
When domain relationships are being discussed, stress-test them with specific scenarios. Invent scenarios that probe edge cases and force the user to be precise about the boundaries between concepts.
### Cross-reference with code
When the user states how something works, check whether the code agrees. If you find a contradiction, surface it: "Your code cancels entire Orders, but you just said partial cancellation is possible — which is right?"
### Update CONTEXT.md inline
When a term is resolved, update `CONTEXT.md` right there. Don't batch these up — capture them as they happen. Use the format in [CONTEXT-FORMAT.md](./CONTEXT-FORMAT.md).
`CONTEXT.md` should be totally devoid of implementation details. Do not treat `CONTEXT.md` as a spec, a scratch pad, or a repository for implementation decisions. It is a glossary and nothing else.
### Offer ADRs sparingly
Only offer to create an ADR when all three are true:
1. **Hard to reverse** — the cost of changing your mind later is meaningful
2. **Surprising without context** — a future reader will wonder "why did they do it this way?"
3. **The result of a real trade-off** — there were genuine alternatives and you picked one for specific reasons
If any of the three is missing, skip the ADR. Use the format in [ADR-FORMAT.md](./ADR-FORMAT.md).
</supporting-info>
## Wrapper Additions
The additions below are **not** part of Matt's upstream skill. They operationalize the upstream's rules into deterministic, stdlib-only validators that pair naturally with the interview loop.
### Workflow (with wrapper tools)
1. **Pre-flight (before the first question):**
- Run `scripts/context_md_linter.py CONTEXT.md` if a `CONTEXT.md` exists — confirms the glossary is well-formed before grilling against it.
- Run `scripts/adr_scanner.py docs/adr/` if `docs/adr/` exists — surfaces numbering gaps, malformed ADRs, status-frontmatter inconsistencies.
- Run `scripts/glossary_code_consistency.py --context CONTEXT.md --code src/` — flags defined-but-unused terms (dead glossary) and code-only common nouns that may need definitions. Use these flags as opening grill questions.
2. **During the session (Matt's rules apply):**
- One question per turn, walking depth-first.
- When a term is sharpened: edit `CONTEXT.md` immediately; re-run `context_md_linter.py` if the edit is structural.
- When an ADR is warranted: write it under `docs/adr/`; re-run `adr_scanner.py` to confirm numbering.
3. **Closing:**
- Final `glossary_code_consistency.py` run to confirm no new orphan terms were introduced.
- Summarize: terms added/refined, ADRs written, scenarios discussed, open items.
### Tools (stdlib-only)
| Tool | One-line role |
|---|---|
| `scripts/context_md_linter.py` | Validate `CONTEXT.md` against the CONTEXT-FORMAT.md structure. PASS/WARN/FAIL per rule. |
| `scripts/adr_scanner.py` | Walk `docs/adr/`, check `NNNN-slug.md` pattern, numbering integrity, body completeness. |
| `scripts/glossary_code_consistency.py` | Cross-reference bold terms in `CONTEXT.md` against codebase usage. Flag dead glossary + code-only common nouns. |
### References (citations behind each rule)
- [`references/ubiquitous_language.md`](references/ubiquitous_language.md) — why a glossary belongs in source control (Evans, Vernon, Khononov, Wlaschin, Brandolini, Avram & Marinescu, Fowler)
- [`references/adr_practice.md`](references/adr_practice.md) — when an ADR earns its keep (Nygard, Tyree & Akerman, Zimmermann Y-statements, MADR, ThoughtWorks Radar, adr-tools, Backstage)
- [`references/context_md_as_artifact.md`](references/context_md_as_artifact.md) — CONTEXT.md as living artifact (Khononov on language drift, Kernighan on naming, BoundedContext bliki, Confluent on data contracts, Brandolini on EventStorming glossary)
### Companion
- Agent: `cs-grill-with-docs` (see `../../agents/cs-grill-with-docs.md`)
- Command: `/cs:grill-with-docs` (see `../../commands/cs-grill-with-docs.md`)
---
**Version:** 1.0.0
**Derived:** Matt Pocock's grill-with-docs (MIT) + this repo's wrapper
FILE:ADR-FORMAT.md
<!--
Derived from Matt Pocock's grill-with-docs:
https://github.com/mattpocock/skills/tree/main/skills/engineering/grill-with-docs/ADR-FORMAT.md
MIT License © 2026 Matt Pocock. Reproduced verbatim under MIT.
-->
# ADR Format
ADRs live in `docs/adr/` and use sequential numbering: `0001-slug.md`, `0002-slug.md`, etc.
Create the `docs/adr/` directory lazily — only when the first ADR is needed.
## Template
```md
# {Short title of the decision}
{1-3 sentences: what's the context, what did we decide, and why.}
```
That's it. An ADR can be a single paragraph. The value is in recording *that* a decision was made and *why* — not in filling out sections.
## Optional sections
Only include these when they add genuine value. Most ADRs won't need them.
- **Status** frontmatter (`proposed | accepted | deprecated | superseded by ADR-NNNN`) — useful when decisions are revisited
- **Considered Options** — only when the rejected alternatives are worth remembering
- **Consequences** — only when non-obvious downstream effects need to be called out
## Numbering
Scan `docs/adr/` for the highest existing number and increment by one.
## When to offer an ADR
All three of these must be true:
1. **Hard to reverse** — the cost of changing your mind later is meaningful
2. **Surprising without context** — a future reader will look at the code and wonder "why on earth did they do it this way?"
3. **The result of a real trade-off** — there were genuine alternatives and you picked one for specific reasons
If a decision is easy to reverse, skip it — you'll just reverse it. If it's not surprising, nobody will wonder why. If there was no real alternative, there's nothing to record beyond "we did the obvious thing."
### What qualifies
- **Architectural shape.** "We're using a monorepo." "The write model is event-sourced, the read model is projected into Postgres."
- **Integration patterns between contexts.** "Ordering and Billing communicate via domain events, not synchronous HTTP."
- **Technology choices that carry lock-in.** Database, message bus, auth provider, deployment target. Not every library — just the ones that would take a quarter to swap out.
- **Boundary and scope decisions.** "Customer data is owned by the Customer context; other contexts reference it by ID only." The explicit no-s are as valuable as the yes-s.
- **Deliberate deviations from the obvious path.** "We're using manual SQL instead of an ORM because X." Anything where a reasonable reader would assume the opposite. These stop the next engineer from "fixing" something that was deliberate.
- **Constraints not visible in the code.** "We can't use AWS because of compliance requirements." "Response times must be under 200ms because of the partner API contract."
- **Rejected alternatives when the rejection is non-obvious.** If you considered GraphQL and picked REST for subtle reasons, record it — otherwise someone will suggest GraphQL again in six months.
FILE:CONTEXT-FORMAT.md
<!--
Derived from Matt Pocock's grill-with-docs:
https://github.com/mattpocock/skills/tree/main/skills/engineering/grill-with-docs/CONTEXT-FORMAT.md
MIT License © 2026 Matt Pocock. Reproduced verbatim under MIT.
-->
# CONTEXT.md Format
## Structure
```md
# {Context Name}
{One or two sentence description of what this context is and why it exists.}
## Language
**Order**:
{A concise description of the term}
_Avoid_: Purchase, transaction
**Invoice**:
A request for payment sent to a customer after delivery.
_Avoid_: Bill, payment request
**Customer**:
A person or organization that places orders.
_Avoid_: Client, buyer, account
## Relationships
- An **Order** produces one or more **Invoices**
- An **Invoice** belongs to exactly one **Customer**
## Example dialogue
> **Dev:** "When a **Customer** places an **Order**, do we create the **Invoice** immediately?"
> **Domain expert:** "No — an **Invoice** is only generated once a **Fulfillment** is confirmed."
## Flagged ambiguities
- "account" was used to mean both **Customer** and **User** — resolved: these are distinct concepts.
```
## Rules
- **Be opinionated.** When multiple words exist for the same concept, pick the best one and list the others as aliases to avoid.
- **Flag conflicts explicitly.** If a term is used ambiguously, call it out in "Flagged ambiguities" with a clear resolution.
- **Keep definitions tight.** One sentence max. Define what it IS, not what it does.
- **Show relationships.** Use bold term names and express cardinality where obvious.
- **Only include terms specific to this project's context.** General programming concepts (timeouts, error types, utility patterns) don't belong even if the project uses them extensively. Before adding a term, ask: is this a concept unique to this context, or a general programming concept? Only the former belongs.
- **Group terms under subheadings** when natural clusters emerge. If all terms belong to a single cohesive area, a flat list is fine.
- **Write an example dialogue.** A conversation between a dev and a domain expert that demonstrates how the terms interact naturally and clarifies boundaries between related concepts.
## Single vs multi-context repos
**Single context (most repos):** One `CONTEXT.md` at the repo root.
**Multiple contexts:** A `CONTEXT-MAP.md` at the repo root lists the contexts, where they live, and how they relate to each other:
```md
# Context Map
## Contexts
- [Ordering](./src/ordering/CONTEXT.md) — receives and tracks customer orders
- [Billing](./src/billing/CONTEXT.md) — generates invoices and processes payments
- [Fulfillment](./src/fulfillment/CONTEXT.md) — manages warehouse picking and shipping
## Relationships
- **Ordering → Fulfillment**: Ordering emits `OrderPlaced` events; Fulfillment consumes them to start picking
- **Fulfillment → Billing**: Fulfillment emits `ShipmentDispatched` events; Billing consumes them to generate invoices
- **Ordering ↔ Billing**: Shared types for `CustomerId` and `Money`
```
The skill infers which structure applies:
- If `CONTEXT-MAP.md` exists, read it to find contexts
- If only a root `CONTEXT.md` exists, single context
- If neither exists, create a root `CONTEXT.md` lazily when the first term is resolved
When multiple contexts exist, infer which one the current topic relates to. If unclear, ask.
FILE:references/adr_practice.md
# ADR Practice — When Does a Decision Earn an ADR?
This reference answers exactly one decision: **what bar must an architectural decision clear to be worth writing down as an ADR, and what format keeps the ADR useful 18 months later?**
Pair with `scripts/adr_scanner.py` for filename + numbering + structural validation.
## The Core Claim
ADRs are not a compliance ritual. They exist to answer a single future question: **"Why on earth did they do it this way?"** If a future reader will never ask that question — because the choice is obvious, easy to reverse, or had no real alternatives — the ADR is doc-rot waiting to happen.
The matt-pocock 3-criteria gate (preserved verbatim in `ADR-FORMAT.md`) is the strict version of this principle:
1. **Hard to reverse** — the cost of changing your mind is meaningful (not "an afternoon of refactoring").
2. **Surprising without context** — a future reader will look at the code and wonder why.
3. **Result of a real trade-off** — there were genuine alternatives and you picked one for specific reasons.
**All three must be true.** Two-out-of-three is not enough. If a decision was hard to reverse but obvious and uncontested (e.g., "we used HTTPS"), no ADR. If it was a real trade-off but easy to reverse (e.g., "we used React Query over SWR"), no ADR.
## What Earns an ADR (Examples)
- **Architectural shape.** "Write model is event-sourced, read model is projected into Postgres." Hard-to-reverse + surprising + real-trade-off.
- **Integration patterns between contexts.** "Ordering and Billing communicate via domain events, not synchronous HTTP." Hard-to-reverse (rewiring eventing is expensive) + surprising (HTTP is the obvious choice) + real-trade-off (eventual consistency vs simpler API).
- **Technology choices with lock-in.** Database engine, message bus, auth provider. Not "we picked Lodash" — those swap in an afternoon.
- **Boundary and scope decisions.** "Customer data is owned by the Customer context; other contexts reference by ID only." The explicit no-s are as valuable as the yes-s.
- **Deliberate deviations from the obvious path.** "We use manual SQL instead of an ORM because X." Stops the next engineer from "fixing" something deliberate.
- **Constraints not visible in code.** "Can't use AWS due to compliance." "Response times must be <200ms due to partner API contract."
- **Rejected alternatives with non-obvious rejections.** "We considered GraphQL and picked REST because subscription complexity didn't match our actual real-time needs." Otherwise someone will suggest GraphQL again in 6 months.
## What Does NOT Earn an ADR
- **Library choices.** Lodash vs Ramda, axios vs ky, dayjs vs date-fns — these swap in an afternoon. Comment in code if you must.
- **Style guide decisions.** "We use Prettier" — record in `package.json`, not an ADR.
- **Defaults you didn't deviate from.** "We use the framework's recommended router." No trade-off, no ADR.
- **Decisions that are easy to reverse.** If the future-you can undo it in a day, future-you doesn't need the why.
- **Decisions where the alternative was never seriously considered.** No real trade-off → no ADR.
## Format Discipline
ADRs are markdown files at `docs/adr/NNNN-slug.md`, numbered sequentially.
**Default format (minimum viable):**
```md
# {Short title of the decision}
{1-3 sentences: what's the context, what did we decide, and why.}
```
An ADR can be a single paragraph. The value is in recording *that* a decision was made and *why* — not in filling out sections.
**Optional sections (only when they add genuine value):**
- **Status frontmatter** (`proposed | accepted | deprecated | superseded by ADR-NNNN`) — useful when decisions are revisited.
- **Considered Options** — only when rejected alternatives are worth remembering.
- **Consequences** — only when non-obvious downstream effects need to be called out.
If a section is included but empty or boilerplate ("none"), delete the section.
## Numbering Discipline
- Sequential, zero-padded to 4 digits: `0001`, `0002`, ..., `9999`.
- No gaps. If an ADR is abandoned mid-draft, either commit it as `proposed → withdrawn` or renumber.
- Slug is short, kebab-case, intent-revealing: `0042-event-sourced-orders.md`, not `0042-adr.md` or `0042-decision-about-events.md`.
`scripts/adr_scanner.py` enforces the pattern and surfaces gaps.
## Status Lifecycle (Optional)
For repos that revisit decisions, the status field is useful:
```
proposed → accepted ← default lifecycle for a new ADR
accepted → deprecated ← decision no longer applies; no replacement
accepted → superseded ← replaced by ADR-NNNN; link to successor in frontmatter
```
When superseding, the new ADR references the old (`supersedes: ADR-0017`) and the old ADR is updated with `superseded by: ADR-0042`. This back-link is the single most useful piece of ADR metadata for archeology.
## Anti-Patterns
- **The ADR factory.** Writing an ADR for every PR. Within a year, you have 200 ADRs and no one reads any. The 3-criteria gate is the firewall.
- **The proposal that never accepts.** ADR sits in `proposed` for months. Either accept it (do it) or withdraw it (delete the file or mark withdrawn).
- **The TOC-only ADR.** Filled-in section headers but no actual content. Worse than not writing the ADR — it implies a decision was recorded when nothing was.
- **The future-tense ADR.** "We will use X." ADRs are records, not plans. Write in past tense ("We chose X because ...") so it reads correctly 2 years later.
- **The unanchored ADR.** ADR with no link to the PR/issue/discussion that drove it. The "why" loses fidelity over time without the source thread.
## Operational Checklist (Per ADR Decision Point)
When grilling and a candidate decision emerges:
- [ ] **Reversibility test.** "If we change our mind in 6 months, what's the cost?" If "an afternoon" → skip the ADR.
- [ ] **Surprise test.** "Will a future engineer look at this and wonder why?" If no → skip.
- [ ] **Trade-off test.** "What alternatives did we seriously consider, and why did each lose?" If none → skip.
- [ ] **All three pass.** Write the ADR. Use the minimum format. Re-run `scripts/adr_scanner.py` to confirm numbering.
- [ ] **Frontmatter status.** Only add `status` if revisiting is expected. Default is "implicit accepted".
## Citations (7 sources)
1. **Michael Nygard, "Documenting Architecture Decisions" (cognitect.com, November 2011).** The original ADR essay. Introduces the format (Title / Context / Decision / Status / Consequences) and the core insight that "architecturally significant" decisions deserve records. Nygard's framing of ADRs as "memory aids for future architects" is the source of the 3-criteria gate's first rule (hard-to-reverse).
2. **Jeff Tyree & Art Akerman, "Architecture Decisions: Demystifying Architecture" — *IEEE Software* 22(2), March–April 2005, pp. 19–27.** Pre-dates Nygard. Introduces the concept of an "Architecture Decision Record" as a first-class artifact and argues for explicit recording of rejected alternatives. The "rejected alternatives" section in Nygard's format inherits from Tyree & Akerman.
3. **Olaf Zimmermann et al., "Y-Statements: A Lightweight Architectural Decision Format" — published at various venues including ozimmer.ch.** Proposes the "In the context of {use case / requirement}, facing {concern}, we decided for {option} to achieve {quality}, accepting {downside}" template. Used widely as a compact alternative to the full Nygard format.
4. **MADR (Markdown Architectural Decision Records) — adr.github.io/madr.** Open-source template maintained by a community of practitioners. Specifies frontmatter format (status, deciders, date, consulted, informed) and a discoverable file structure. Useful when ADRs need machine-readable metadata for indexing.
5. **ThoughtWorks Technology Radar — thoughtworks.com/radar.** Has covered "Lightweight Architecture Decision Records" since Vol. 18 (2018) in the Techniques quadrant, with periodic upgrades to "Adopt". TW's "use ADRs sparingly" guidance aligns with the 3-criteria gate.
6. **Joel Parker Henderson, adr-tools (github.com/npryce/adr-tools).** CLI tool implementing Nygard's format with numbering helpers, supersession linking, and a `new` / `link` / `accept` command set. Establishes the de-facto convention of `0001-slug.md` filenames and `docs/adr/` directory location.
7. **Spotify Backstage — backstage.io.** Backstage's TechDocs catalog includes an ADR plugin that surfaces per-service ADRs in the service catalog UI. Demonstrates how ADRs become discoverable at scale (>1000 services) when treated as first-class catalog entries, not just files in a repo.
FILE:references/context_md_as_artifact.md
# CONTEXT.md as a Living Artifact — Preventing Glossary Decay
This reference answers exactly one decision: **how does a glossary stay alive vs decay into doc rot, and what operational practices prevent the drift?**
Pair with `scripts/glossary_code_consistency.py` for the lint-against-codebase reality check and `scripts/context_md_linter.py` for structural validation.
## The Core Claim
Every glossary decays by default. The decay path is well-documented:
```
Month 1: Glossary written during initial DDD workshop. Terms are precise.
Month 3: New feature ships. Two new domain terms used in code, neither added to glossary.
Month 6: A term in the glossary is renamed in code. Glossary still has old name.
Month 9: New engineer joins. Reads glossary. Asks "what's a 'Booking'?" — answer is "we don't call those Bookings anymore, we call them Reservations now."
Month 12: Glossary is officially declared stale. Engineers stop reading it. Drift becomes invisible.
```
The decay is not preventable by good intentions. It is prevented by **inline edits during the work that introduces the term** plus **automated lint runs at PR time** to flag mismatches.
## Three Forces That Drive Drift
1. **Language pressure from outside the bounded context.** A new partner integration uses different terminology ("subscriber" vs your "customer"). Engineers copy the partner's term into code without first reconciling with the glossary.
2. **Refactor pressure inside the bounded context.** A rename in code feels obvious ("`Booking` → `Reservation` is just a better name"), but the glossary isn't updated alongside.
3. **Convergence pressure between teams.** Multiple teams contributing to the same context use slightly different words for the same concept. Without a glossary as referee, all variants end up in code.
`scripts/glossary_code_consistency.py` operationalizes the lint against these three forces:
- **Defined-but-unused term** → a glossary entry that no code references. Either dead glossary (delete) or a rename happened (update glossary to match code).
- **Code-only proper noun** → a frequently-used capitalized term in code that the glossary doesn't define. Either generic (ignore) or domain (add to glossary now).
## Five Practices That Keep CONTEXT.md Alive
1. **Edit inline during the work.** Never batch glossary updates. When a term is introduced or refined during a feature, the same PR that adds the code edits `CONTEXT.md`. Reviewers reject PRs that introduce domain terms without glossary edits.
2. **Lint at PR time.** Run `scripts/context_md_linter.py` and `scripts/glossary_code_consistency.py` in CI. A new term in code without a glossary entry is a build warning; an outright rename mismatch is a build failure.
3. **Per-context glossaries, not one mega-glossary.** Multi-context repos use `CONTEXT-MAP.md` to point at per-context `CONTEXT.md` files. Cross-context terms get explicit translation entries ("Billing's `Customer` is Ordering's `Account`").
4. **Pruning passes.** Quarterly, run `glossary_code_consistency.py` and review the dead-glossary report. Delete entries that no code uses. Keeping dead entries dilutes signal.
5. **One sentence per definition.** If a definition runs to a paragraph, the term is hiding two concepts. Split or sharpen. Long definitions are correlated with imprecise terms.
## How CONTEXT.md Differs from Other "Documentation"
| Artifact | Purpose | Update cadence | Audience |
|---|---|---|---|
| `README.md` | Onboarding + setup | Once at project start, occasionally after | New contributors |
| `ARCHITECTURE.md` | High-level system shape | Quarterly to yearly | New architects, senior engineers |
| `docs/adr/*.md` | Record of specific decisions | Per-decision (rare; days to months apart) | Anyone asking "why did we do X this way?" |
| **`CONTEXT.md`** | **The domain glossary — what each term means in this bounded context** | **Per-feature (continuous; hours to days apart)** | **Every engineer on every PR** |
A `CONTEXT.md` is touched far more often than any other doc because it tracks the language as it evolves. If yours hasn't been edited in 6 months, it's almost certainly drifting.
## Single vs Multi-Context Repos
**Single context (most repos):** One `CONTEXT.md` at the repo root. All terms in scope.
**Multiple contexts:** A `CONTEXT-MAP.md` at the repo root lists the contexts and their relationships. Each bounded context has its own `CONTEXT.md` (and its own `docs/adr/` for context-specific decisions). Shared terms appear in both with cross-references.
```
/
├── CONTEXT-MAP.md ← lists contexts + relationships
├── docs/adr/ ← system-wide ADRs
└── src/
├── ordering/
│ ├── CONTEXT.md ← ordering-context glossary
│ └── docs/adr/ ← ordering-context ADRs
└── billing/
├── CONTEXT.md
└── docs/adr/
```
When a term spans contexts, define it in each `CONTEXT.md` with the context's perspective + a translation note pointing at the other. Don't try to define "Customer" once and have both contexts share it — that's the path back to the mega-glossary.
## Anti-Patterns
- **The spec masquerading as a glossary.** `CONTEXT.md` includes implementation details, sequence diagrams, API responses. It is a glossary, not a spec. Move spec content elsewhere.
- **The wiki masquerading as a glossary.** General programming concepts ("retry", "timeout", "config") appearing in `CONTEXT.md`. They are not domain-specific. Remove.
- **The glossary that defines without forbidding.** Each term needs `_Avoid_: <aliases>` to push back on drift. A glossary that says "Customer means X" but doesn't forbid "Client" / "Account" / "User" cannot push back when those drift in.
- **The frozen glossary.** No commits in 6+ months. Either the project is dormant or the language has drifted away from the document. Re-grill.
- **The orphan glossary.** Sits in a repo but no CI/PR process references it. It will decay within two quarters.
## Operational Checklist
When grilling against `CONTEXT.md`:
- [ ] Lint structure: `python scripts/context_md_linter.py CONTEXT.md`
- [ ] Lint vs code: `python scripts/glossary_code_consistency.py --context CONTEXT.md --code src/`
- [ ] For each "defined but unused": ask "dead term, or rename happened?"
- [ ] For each "code-only proper noun": ask "domain term that needs definition, or generic?"
- [ ] For each new term introduced during the grill: edit `CONTEXT.md` *now*, not "later"
- [ ] Multi-context repo: verify the right `CONTEXT.md` is being edited (not the wrong context's, not the root one when a per-context one applies)
## Citations (7 sources)
1. **Vladimir Khononov, *Learning Domain-Driven Design* (O'Reilly, 2021).** Chapter 9, "Communication Patterns" + Chapter 12, "Building Domain Expertise" — Khononov is the sharpest writer on language drift between bounded contexts and on how to detect it. His "linguistic boundaries are observable boundaries" framing is the foundation of the `glossary_code_consistency.py` check.
2. **Brian Kernighan & Rob Pike, *The Practice of Programming* (Addison-Wesley, 1999).** Chapter 1, "Style" — the section on naming. Kernighan's "names should reflect the role of the variable, not its type" generalizes to glossary terms: a glossary term names a role in the domain, not a data structure. Kernighan-style naming discipline is what keeps `CONTEXT.md` precise.
3. **Martin Fowler, "BoundedContext" — martinfowler.com bliki (2014, updated).** The canonical argument that ubiquitous language is **bounded** — it applies inside one context, not across all contexts. The justification for per-context `CONTEXT.md` files. https://martinfowler.com/bliki/BoundedContext.html
4. **Martin Fowler, "UbiquitousLanguage" — martinfowler.com bliki.** Companion entry to BoundedContext. Articulates the discipline of using the same vocabulary in conversation, in the model, and in the code. The justification for editing `CONTEXT.md` inline alongside code changes, not as separate doc work. https://martinfowler.com/bliki/UbiquitousLanguage.html
5. **Confluent Schema Registry / Data Contracts community — confluent.io/blog/data-contracts.** The data-contracts movement applies UL discipline to inter-service / inter-context boundaries: when two contexts exchange events or API payloads, the schema is a binding glossary. Drift between contexts becomes a schema-evolution problem, not a free-form documentation problem.
6. **Alberto Brandolini, *Introducing EventStorming* (Leanpub, ongoing).** Chapter on "Pivotal Events" + the convergence-workshop chapter. Brandolini documents how a glossary emerges from EventStorming workshops as a by-product of mapping events. The pattern of "capture the term on a sticky note when it surfaces" is the offline equivalent of the inline `CONTEXT.md` edit discipline.
7. **Eric Evans, *Domain-Driven Design: Tackling Complexity in the Heart of Software* (Addison-Wesley, 2003).** Chapter 14, "Maintaining Model Integrity" — covers the Conformist, Anticorruption Layer, and Shared Kernel patterns. Each of these is a strategy for managing the boundary between two bounded contexts that have different languages. Justifies the multi-context `CONTEXT-MAP.md` pattern and the translation-note discipline for cross-context terms.
FILE:references/ubiquitous_language.md
# Ubiquitous Language — Why a Glossary Belongs in Source Control
This reference answers exactly one decision: **why should a project's domain glossary (`CONTEXT.md`) live next to the code in source control, and what bar must it clear to earn its keep?**
Pair with `scripts/context_md_linter.py` for structural validation and `scripts/glossary_code_consistency.py` for the language-vs-code reality check.
## The Core Claim
A bounded context has **one** language. The same word must mean the same thing in conversation, in the glossary, in the type system, in the database schema, and in the UI. When language fractures across these surfaces, design defects follow: ambiguous bug reports, mismatched API contracts, broken refactors, junior engineers asking what an "account" is and getting three different answers.
The glossary is the contract that prevents the fracture. It earns its place in source control because it changes at the same cadence as the code — every time a domain term is introduced, refined, or retired, the glossary must move with it. A wiki page that lives outside the repo will drift within a quarter.
## Why a Glossary in Source Control (vs Wiki, Notion, Confluence)
| Property | In-repo `CONTEXT.md` | External wiki |
|---|---|---|
| Reviewable in PR | Yes — diff is visible alongside code | No — reviewer must remember to check |
| Versioned with code | Yes — `git log` shows term evolution | No — wikis rarely have meaningful history |
| Discoverable by new engineers | Yes — `ls` of repo root finds it | No — depends on onboarding tribal knowledge |
| Mergeable | Yes — text format, conflict-resolvable | Often no — UI-driven |
| Linter-targetable | Yes — `scripts/context_md_linter.py` | No — usually not |
| Refactor-safe | Yes — renames are grep-able | No — wiki links rot silently |
The glossary is a **language artifact**, not a documentation artifact. Documentation describes the system; the glossary **is** part of the system's design surface.
## Five Rules That Make a Glossary Survive
1. **One sentence per definition.** If the definition needs a paragraph, the term is hiding two concepts. Split it.
2. **Define what it IS, not what it does.** "An **Invoice** is a request for payment sent after delivery." Not "An invoice handles billing."
3. **List aliases to avoid.** When users say "bill" or "payment request" but mean "invoice", record that "bill" is forbidden. Without the `_Avoid_:` field, the glossary cannot push back on drift.
4. **Show relationships, not just terms.** "An **Order** produces one or more **Invoices**" tells you the cardinality. A list of bare terms doesn't.
5. **Exclude generic programming concepts.** "Timeout", "retry", "config" do not belong. Only terms specific to this project's domain qualify.
## Anti-Patterns
- **The "everything goes in" glossary.** When `CONTEXT.md` includes general programming concepts (timeout, error, util), it dilutes signal and degenerates into a wiki page.
- **The orphan glossary.** Terms defined but never used in code. Either the term is dead (delete it) or the code is using a synonym (rename code).
- **The opaque glossary.** Terms used in code but not defined. Either the term is generic (don't define it) or it's a domain concept that snuck in (define it now).
- **The deferred glossary edit.** "I'll batch up the glossary changes at the end of the sprint." By the end of the sprint, three more drift cases will have shipped. Glossary edits must land inline.
- **The frozen glossary.** No commits in 6+ months. Either the project is dormant or the language has drifted away from the document.
## Operational Checklist (for the Grill Session)
When grilling a plan against `CONTEXT.md`:
- [ ] Pre-flight `scripts/context_md_linter.py CONTEXT.md` — is the glossary well-formed?
- [ ] Run `scripts/glossary_code_consistency.py` — what's defined but unused? what's used but undefined?
- [ ] For every novel term in the plan, ask: "Is this in CONTEXT.md? If not, do we add it, or do we rephrase using an existing term?"
- [ ] For every existing term used in the plan, ask: "Does the plan use it consistent with the definition?"
- [ ] At every clarification moment, edit `CONTEXT.md` immediately — never batch.
## Citations (7 sources)
1. **Eric Evans, *Domain-Driven Design: Tackling Complexity in the Heart of Software* (Addison-Wesley, 2003).** Chapter 2, "Communication and the Use of Language" — the canonical statement of Ubiquitous Language as a design tool, not just documentation. The line "The vocabulary of that UBIQUITOUS LANGUAGE includes the names of classes and prominent operations" is the bridge between conversation and code.
2. **Vaughn Vernon, *Implementing Domain-Driven Design* (Addison-Wesley, 2013).** Chapter 1, "Getting Started with DDD" + Chapter 2, "Domains, Subdomains, and Bounded Contexts" — operationalizes Evans's UL into a workshop format and per-context discipline. Vernon's "linguistic boundaries are the most reliable boundary" framing is the source of the per-bounded-context glossary pattern.
3. **Vladimir Khononov, *Learning Domain-Driven Design* (O'Reilly, 2021).** Chapter 5, "Implementing Simple Business Logic" + Chapter 9, "Communication Patterns" — Khononov is sharpest on what happens when bounded contexts share a language vs maintain separate languages (translation layer required) and on language drift over time.
4. **Scott Wlaschin, *Domain Modeling Made Functional* (Pragmatic Bookshelf, 2018).** Part 1, "Understanding the Domain" — treats the type system as the executable form of the glossary. Wlaschin's "make illegal states unrepresentable" is the strongest form of glossary-as-contract: if the glossary says an Order must have at least one line item, the type prevents zero-item Orders at compile time.
5. **Alberto Brandolini, *Introducing EventStorming: An Act of Deliberate Collective Learning* (Leanpub, 2017–ongoing).** Chapter on "Sticky note color codes" + chapter on convergence — EventStorming workshops produce a glossary as a by-product of mapping the domain. Brandolini's pattern of capturing terms as they emerge on sticky notes is the offline equivalent of the inline `CONTEXT.md` edit.
6. **Abel Avram & Floyd Marinescu, *Domain-Driven Design Quickly* (InfoQ, 2006, free e-book).** Chapter 2, "Ubiquitous Language" — the most concise distillation of Evans's UL chapter. Useful as a reference to hand to engineers who won't read the blue book.
7. **Martin Fowler, "BoundedContext" — martinfowler.com bliki (2014, updated).** Fowler's framing of "Ubiquitous Language … doesn't apply to the whole project, it only has to apply within a particular Bounded Context" justifies the per-context glossary pattern in `CONTEXT-MAP.md`-style multi-context repos. https://martinfowler.com/bliki/BoundedContext.html
FILE:scripts/adr_scanner.py
#!/usr/bin/env python3
"""adr_scanner.py — Walk docs/adr/ and validate ADR files against the format.
Stdlib-only. Applies the rules from Matt Pocock's upstream ADR-FORMAT.md
(preserved verbatim in the skill's ADR-FORMAT.md):
1. Each file matches the `NNNN-slug.md` pattern (4-digit zero-padded number + kebab-case slug)
2. Numbering is sequential — no gaps, no duplicates
3. Each ADR has an H1 (the title)
4. Each ADR has a non-empty body after the H1 (at least the 1-3 sentence context+decision)
5. Optional status frontmatter, if present, has a valid value
(proposed | accepted | deprecated | superseded by ADR-NNNN)
6. Superseded-by references point at an existing ADR number
Output: directory-level summary + per-file findings.
NO LLM CALLS. Pure regex + filesystem walking.
Usage:
python adr_scanner.py docs/adr/
python adr_scanner.py docs/adr/ --output json
python adr_scanner.py --sample # scan an embedded sample directory layout
"""
import argparse
import json
import re
import sys
from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple
ADR_FILENAME_RE = re.compile(r"^(\d{4})-([a-z0-9]+(?:-[a-z0-9]+)*)\.md$")
VALID_STATUSES = {"proposed", "accepted", "deprecated"}
SUPERSEDED_RE = re.compile(r"^superseded\s+by\s+ADR-?(\d{1,4})$", re.IGNORECASE)
SAMPLE_ADRS: Dict[str, str] = {
"0001-event-sourced-orders.md": (
"# Event-source the Order write model\n"
"\n"
"We need an audit trail of every state change on an Order for compliance + analytics. "
"We chose event sourcing for the Order write model and a Postgres projection for the read model. "
"Trade-off accepted: eventual consistency on the read side in exchange for the audit trail and replay.\n"
),
"0002-postgres-for-write-model.md": (
"---\n"
"status: accepted\n"
"---\n"
"\n"
"# Postgres for the write-side event store\n"
"\n"
"We considered EventStore and Kafka. Postgres won on operational familiarity + transactional guarantees + cost.\n"
),
"0003-rest-over-graphql.md": (
"---\n"
"status: accepted\n"
"---\n"
"\n"
"# REST over GraphQL for the public API\n"
"\n"
"GraphQL would have given clients more flexibility but added subscription complexity we don't need at our scale.\n"
),
}
def parse_frontmatter(text: str) -> Tuple[Dict[str, str], str]:
"""Return (frontmatter_dict, body) for a file that may have YAML-ish frontmatter.
Only handles simple `key: value` lines (no nested YAML, no lists) — stdlib-only.
"""
if not text.startswith("---\n"):
return {}, text
end_marker = text.find("\n---\n", 4)
if end_marker == -1:
return {}, text
fm_block = text[4:end_marker]
body = text[end_marker + 5 :]
fm: Dict[str, str] = {}
for line in fm_block.splitlines():
if ":" in line:
k, v = line.split(":", 1)
fm[k.strip().lower()] = v.strip()
return fm, body
def scan_directory(adr_dir: Path) -> Dict[str, Any]:
findings: List[Dict[str, Any]] = []
files: List[Tuple[int, str, Path]] = []
def add(file: str, rule: str, level: str, message: str) -> None:
findings.append({"file": file, "rule": rule, "level": level, "message": message})
if not adr_dir.exists():
add("(root)", "directory", "FAIL", f"Directory does not exist: {adr_dir}")
return finalize(findings, 0)
if not adr_dir.is_dir():
add("(root)", "directory", "FAIL", f"Path is not a directory: {adr_dir}")
return finalize(findings, 0)
md_files = sorted(p for p in adr_dir.iterdir() if p.is_file() and p.suffix == ".md")
if not md_files:
add("(root)", "directory", "WARN", "Directory is empty — no ADRs scanned. Create lazily when the first ADR is needed.")
return finalize(findings, 0)
# Rule 1: filename pattern
for p in md_files:
m = ADR_FILENAME_RE.match(p.name)
if not m:
add(p.name, "filename-pattern", "FAIL", f"Filename does not match NNNN-slug.md pattern. Expected e.g. 0001-event-sourced-orders.md.")
continue
number = int(m.group(1))
files.append((number, p.name, p))
add(p.name, "filename-pattern", "PASS", f"Filename matches pattern (number={number:04d}).")
files.sort(key=lambda t: t[0])
# Rule 2: numbering sequence (no gaps, no duplicates)
seen: Dict[int, List[str]] = {}
for number, name, _ in files:
seen.setdefault(number, []).append(name)
for number, names in seen.items():
if len(names) > 1:
add(", ".join(names), "numbering-duplicate", "FAIL", f"Duplicate ADR number {number:04d}.")
if files:
expected = list(range(1, files[-1][0] + 1))
actual = sorted(seen.keys())
gaps = [n for n in expected if n not in actual]
if gaps:
add("(root)", "numbering-gap", "WARN", f"Number gap(s) in sequence: {', '.join(f'{g:04d}' for g in gaps)}. Either commit withdrawn ADRs as 'proposed → withdrawn' or renumber.")
else:
add("(root)", "numbering-sequence", "PASS", f"Sequential numbering 0001..{files[-1][0]:04d} with no gaps.")
# Rules 3, 4, 5, 6: per-ADR
numbers_present = {n for n, _, _ in files}
for number, name, path in files:
text = path.read_text(encoding="utf-8") if path.is_file() else SAMPLE_ADRS.get(name, "")
fm, body = parse_frontmatter(text)
# Rule 3: H1 present
h1_match = re.search(r"^#\s+(.+?)\s*$", body, re.MULTILINE)
if not h1_match:
add(name, "h1-present", "FAIL", "No H1 (`# Title`) found in body.")
continue
else:
add(name, "h1-present", "PASS", f"H1 found: '{h1_match.group(1).strip()}'.")
# Rule 4: non-empty body after H1
after_h1 = body[h1_match.end():].strip()
if not after_h1:
add(name, "body-non-empty", "FAIL", "ADR has H1 but no body. The 1-3 sentence context+decision is required.")
else:
word_count = len(re.findall(r"\b\w+\b", after_h1))
if word_count < 10:
add(name, "body-non-empty", "WARN", f"ADR body is very short ({word_count} words). Confirm context+decision+why are all stated.")
else:
add(name, "body-non-empty", "PASS", f"Body present ({word_count} words).")
# Rule 5: optional status frontmatter sanity
status = fm.get("status", "").strip().lower() if fm else ""
if status:
if status in VALID_STATUSES:
add(name, "status-frontmatter", "PASS", f"Status '{status}' is valid.")
elif SUPERSEDED_RE.match(status):
m = SUPERSEDED_RE.match(status)
target = int(m.group(1))
# Rule 6: superseded-by points at existing ADR
if target in numbers_present:
add(name, "status-supersede-target", "PASS", f"Superseded by ADR-{target:04d} which exists.")
else:
add(name, "status-supersede-target", "FAIL", f"Superseded by ADR-{target:04d} but that ADR is not present in this directory.")
else:
add(name, "status-frontmatter", "FAIL", f"Status '{status}' is not one of {sorted(VALID_STATUSES)} or 'superseded by ADR-NNNN'.")
return finalize(findings, len(files))
def finalize(findings: List[Dict[str, Any]], adr_count: int) -> Dict[str, Any]:
counts = {"PASS": 0, "WARN": 0, "FAIL": 0}
for f in findings:
counts[f["level"]] += 1
if counts["FAIL"] > 0:
verdict = "FAIL"
elif counts["WARN"] > 0:
verdict = "WARN"
else:
verdict = "PASS"
return {"verdict": verdict, "adr_count": adr_count, "counts": counts, "findings": findings}
def render_human(result: Dict[str, Any]) -> str:
out: List[str] = []
out.append(f"ADR directory scan verdict: {result['verdict']}")
out.append(f" ADRs scanned: {result['adr_count']}")
counts = result["counts"]
out.append(f" PASS: {counts['PASS']} WARN: {counts['WARN']} FAIL: {counts['FAIL']}")
out.append("")
out.append("Findings:")
for f in result["findings"]:
marker = {"PASS": "[ok]", "WARN": "[warn]", "FAIL": "[FAIL]"}[f["level"]]
out.append(f" {marker} {f['file']:<40s} {f['rule']}: {f['message']}")
return "\n".join(out)
def run_sample() -> Dict[str, Any]:
"""Scan the embedded sample by writing it to a tempdir."""
import tempfile
with tempfile.TemporaryDirectory() as td:
d = Path(td) / "adr"
d.mkdir()
for name, content in SAMPLE_ADRS.items():
(d / name).write_text(content, encoding="utf-8")
return scan_directory(d)
def main(argv: List[str]) -> int:
parser = argparse.ArgumentParser(description=__doc__.split("\n")[0])
parser.add_argument("adr_dir", nargs="?", help="Path to docs/adr/ directory")
parser.add_argument("--sample", action="store_true", help="Scan the embedded sample ADR layout")
parser.add_argument("--output", choices=["human", "json"], default="human")
args = parser.parse_args(argv)
if args.sample:
result = run_sample()
elif args.adr_dir:
result = scan_directory(Path(args.adr_dir))
else:
parser.print_help()
return 0
if args.output == "json":
print(json.dumps(result, indent=2))
else:
print(render_human(result))
return 0 if result["verdict"] != "FAIL" else 1
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
FILE:scripts/context_md_linter.py
#!/usr/bin/env python3
"""context_md_linter.py — Validate a CONTEXT.md against the CONTEXT-FORMAT.md structure.
Stdlib-only. Walks a CONTEXT.md and applies the format rules from Matt Pocock's
upstream CONTEXT-FORMAT.md (preserved verbatim in the skill's CONTEXT-FORMAT.md):
1. H1 present at top (the context name)
2. One-or-two-sentence description follows the H1
3. ## Language section present
4. Inside Language: each term is in `**Term**:` bold form
5. Inside Language: each term has a one-sentence definition
6. Inside Language: each term has a `_Avoid_:` aliases line (WARN if missing)
7. ## Relationships section present (WARN if missing)
8. ## Example dialogue section present (WARN if missing)
9. Optional: ## Flagged ambiguities section
Output: PASS / WARN / FAIL per rule + an overall verdict.
NO LLM CALLS. Pure regex + line walking.
Usage:
python context_md_linter.py CONTEXT.md
python context_md_linter.py CONTEXT.md --output json
python context_md_linter.py --sample # lint the embedded sample
"""
import argparse
import json
import re
import sys
from pathlib import Path
from typing import Any, Dict, List, Tuple
SAMPLE_CONTEXT_MD = """# Ordering
The ordering context receives customer orders and tracks them through to handoff to Fulfillment.
## Language
**Order**:
A confirmed request from a Customer to acquire one or more Products.
_Avoid_: Purchase, transaction, cart
**Customer**:
A person or organization that places Orders.
_Avoid_: Client, buyer, account
**Product**:
A single SKU that can appear on an Order line.
_Avoid_: Item, good, SKU
## Relationships
- An **Order** belongs to exactly one **Customer**
- An **Order** has one or more **Products** via line items
- A **Customer** can have many **Orders**
## Example dialogue
> **Dev:** "When a **Customer** places an **Order**, are the **Products** locked at order time?"
> **Domain expert:** "Yes — Product price + spec is snapshotted onto the Order line. Subsequent Product edits don't change historical Orders."
## Flagged ambiguities
- "account" was used to mean both **Customer** and "billing account" — resolved: billing account moves to Billing context.
"""
def split_into_sections(text: str) -> Dict[str, str]:
"""Split markdown into top-level ## sections keyed by header text."""
sections: Dict[str, str] = {}
current_header = "_preamble_"
buffer: List[str] = []
for line in text.splitlines():
m = re.match(r"^##\s+(.+?)\s*$", line)
if m:
sections[current_header] = "\n".join(buffer).strip()
current_header = m.group(1).strip().lower()
buffer = []
else:
buffer.append(line)
sections[current_header] = "\n".join(buffer).strip()
return sections
def extract_terms(language_section: str) -> List[Tuple[str, str, str]]:
"""Return list of (term, definition_line, avoid_line) tuples from the Language section.
Each term entry looks like:
**Term**:
Definition sentence.
_Avoid_: alias1, alias2
"""
results: List[Tuple[str, str, str]] = []
# Match `**Term**:` followed by the next non-empty line as definition,
# and optionally an `_Avoid_:` line within the next 3 lines.
pattern = re.compile(
r"\*\*([^*]+?)\*\*\s*:\s*\n([^\n]+)\n?(?:([^\n]*_Avoid_[^\n]*)\n?)?",
re.MULTILINE,
)
for match in pattern.finditer(language_section):
term = match.group(1).strip()
definition = match.group(2).strip()
avoid = (match.group(3) or "").strip()
results.append((term, definition, avoid))
return results
def lint(text: str) -> Dict[str, Any]:
findings: List[Dict[str, str]] = []
def add(rule: str, level: str, message: str) -> None:
findings.append({"rule": rule, "level": level, "message": message})
# Rule 1: H1 present
lines = text.splitlines()
h1_line_index = None
for i, line in enumerate(lines):
if re.match(r"^#\s+\S", line):
h1_line_index = i
break
if h1_line_index is None:
add("h1-present", "FAIL", "No H1 (top-level '# Title') found. CONTEXT.md must start with the context name as H1.")
else:
add("h1-present", "PASS", f"H1 found at line {h1_line_index + 1}.")
# Rule 2: one-or-two-sentence description after H1
if h1_line_index is not None:
desc_lines: List[str] = []
for line in lines[h1_line_index + 1 :]:
if re.match(r"^##\s", line):
break
if line.strip():
desc_lines.append(line.strip())
desc = " ".join(desc_lines).strip()
sentence_count = len(re.findall(r"[.!?](?:\s|$)", desc))
if not desc:
add("description-present", "FAIL", "No description sentence between the H1 and the first ## section.")
elif sentence_count > 3:
add(
"description-length",
"WARN",
f"Description has {sentence_count} sentences. CONTEXT-FORMAT.md asks for one or two.",
)
else:
add("description-present", "PASS", f"Description present ({sentence_count} sentence(s)).")
# Rule 3: ## Language section present
sections = split_into_sections(text)
if "language" not in sections:
add("language-section", "FAIL", "No '## Language' section found. This is the required core of CONTEXT.md.")
return finalize(findings)
add("language-section", "PASS", "'## Language' section found.")
# Rules 4 + 5 + 6: terms inside Language
terms = extract_terms(sections["language"])
if not terms:
add(
"language-terms",
"FAIL",
"No terms detected in the Language section. Each term must be in '**Term**:' bold form followed by a one-sentence definition.",
)
else:
add("language-terms", "PASS", f"Detected {len(terms)} term(s) in Language section.")
for term, definition, avoid in terms:
# Rule 5: definition exists
if not definition or definition.startswith("_Avoid_") or definition.startswith("**"):
add(
"term-definition",
"FAIL",
f"Term '**{term}**:' has no definition line (next non-empty line should be the definition).",
)
else:
# Length heuristic: definition should be <= 200 chars (one sentence-ish)
if len(definition) > 200:
add(
"term-definition-length",
"WARN",
f"Term '**{term}**' definition is {len(definition)} chars. CONTEXT-FORMAT.md asks for one sentence max.",
)
# Rule 6: _Avoid_ line
if not avoid:
add(
"term-avoid",
"WARN",
f"Term '**{term}**' has no '_Avoid_:' aliases line. Without forbidden aliases, the glossary can't push back on drift.",
)
# Rule 7: Relationships section
if "relationships" not in sections:
add(
"relationships-section",
"WARN",
"No '## Relationships' section found. CONTEXT-FORMAT.md asks for one to show cardinality between terms.",
)
else:
add("relationships-section", "PASS", "'## Relationships' section found.")
# Rule 8: Example dialogue
if "example dialogue" not in sections:
add(
"example-dialogue",
"WARN",
"No '## Example dialogue' section found. CONTEXT-FORMAT.md asks for a dev/domain-expert exchange.",
)
else:
add("example-dialogue", "PASS", "'## Example dialogue' section found.")
# Rule 9: Flagged ambiguities (optional, only check presence)
if "flagged ambiguities" in sections:
add("flagged-ambiguities", "PASS", "'## Flagged ambiguities' section found (optional but useful).")
return finalize(findings)
def finalize(findings: List[Dict[str, str]]) -> Dict[str, Any]:
counts = {"PASS": 0, "WARN": 0, "FAIL": 0}
for f in findings:
counts[f["level"]] += 1
if counts["FAIL"] > 0:
verdict = "FAIL"
elif counts["WARN"] > 0:
verdict = "WARN"
else:
verdict = "PASS"
return {"verdict": verdict, "counts": counts, "findings": findings}
def render_human(result: Dict[str, Any]) -> str:
out: List[str] = []
verdict = result["verdict"]
counts = result["counts"]
out.append(f"CONTEXT.md lint verdict: {verdict}")
out.append(f" PASS: {counts['PASS']} WARN: {counts['WARN']} FAIL: {counts['FAIL']}")
out.append("")
out.append("Findings:")
for f in result["findings"]:
marker = {"PASS": "[ok]", "WARN": "[warn]", "FAIL": "[FAIL]"}[f["level"]]
out.append(f" {marker} {f['rule']}: {f['message']}")
return "\n".join(out)
def main(argv: List[str]) -> int:
parser = argparse.ArgumentParser(description=__doc__.split("\n")[0])
parser.add_argument("path", nargs="?", help="Path to CONTEXT.md")
parser.add_argument("--sample", action="store_true", help="Lint the embedded sample CONTEXT.md")
parser.add_argument("--output", choices=["human", "json"], default="human")
args = parser.parse_args(argv)
if args.sample:
text = SAMPLE_CONTEXT_MD
elif args.path:
p = Path(args.path)
if not p.exists():
print(f"error: {args.path} not found", file=sys.stderr)
return 2
text = p.read_text(encoding="utf-8")
else:
parser.print_help()
return 0
result = lint(text)
if args.output == "json":
print(json.dumps(result, indent=2))
else:
print(render_human(result))
return 0 if result["verdict"] != "FAIL" else 1
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
FILE:scripts/glossary_code_consistency.py
#!/usr/bin/env python3
"""glossary_code_consistency.py — Cross-reference CONTEXT.md terms against the codebase.
Stdlib-only. Reads bold terms from CONTEXT.md and scans a codebase directory for
each term's usage. Surfaces two grilling-question seeds:
1. DEAD GLOSSARY — a term is defined in CONTEXT.md but never appears in code.
Either the term is stale (delete it) or the code uses a synonym (rename).
2. CODE-ONLY PROPER NOUN — a capitalized word that appears frequently in code
but isn't defined in CONTEXT.md. Either it's a generic programming concept
(ignore) or it's a domain term that snuck in undefined (add to glossary).
Both lists are seeded as opening grill-with-docs questions.
NO LLM CALLS. Pure file walking + regex + frequency counting.
Limitations (intentional, stdlib-only):
- Word-boundary matching is case-insensitive. "Order" matches "order", "ORDER", "orders".
- "Code-only proper noun" detection uses a simple heuristic: capitalized
words >= MIN_FREQUENCY occurrences across non-test files. Tunable via flags.
- Only scans common source extensions by default (override with --extensions).
Usage:
python glossary_code_consistency.py --context CONTEXT.md --code src/
python glossary_code_consistency.py --context CONTEXT.md --code src/ --output json
python glossary_code_consistency.py --sample
"""
import argparse
import json
import re
import sys
from collections import Counter
from pathlib import Path
from typing import Any, Dict, List, Set, Tuple
DEFAULT_EXTENSIONS = {
".py",
".ts",
".tsx",
".js",
".jsx",
".go",
".java",
".kt",
".rb",
".cs",
".rs",
".swift",
".php",
".scala",
".clj",
".ex",
".exs",
}
DEFAULT_EXCLUDE_DIRS = {"node_modules", ".git", "dist", "build", "target", ".venv", "venv", "__pycache__"}
TEST_FILE_HINTS = (".test.", ".spec.", "_test.", "tests/", "/test/")
PROPER_NOUN_RE = re.compile(r"\b([A-Z][a-zA-Z]{2,})\b")
GENERIC_WORDS = {
# Programming concepts that capitalize but aren't domain terms
"True", "False", "None", "Null", "Promise", "Error", "Exception",
"String", "Number", "Boolean", "Array", "Object", "Map", "Set",
"List", "Dict", "Tuple", "Optional", "Any", "Result", "Date",
"Math", "JSON", "URL", "URI", "HTTP", "HTTPS", "API", "ID", "UUID",
"GET", "POST", "PUT", "DELETE", "PATCH", "OK", "TODO", "FIXME",
"Test", "Mock", "Stub", "Spy", "Given", "When", "Then", "Describe",
}
SAMPLE_CONTEXT_MD = """# Ordering
## Language
**Order**:
A confirmed request from a Customer to acquire one or more Products.
_Avoid_: Purchase, transaction
**Customer**:
A person or organization that places Orders.
_Avoid_: Client, buyer
**Product**:
A single SKU that can appear on an Order line.
_Avoid_: Item, good
**Discount**:
A reduction applied to an Order at checkout.
_Avoid_: Coupon, promo
"""
SAMPLE_CODE_FILES: Dict[str, str] = {
"src/orders.py": (
"class Order:\n"
" pass\n"
"\n"
"def cancel_order(order_id: str) -> None:\n"
" pass\n"
"\n"
"def list_customer_orders(customer_id: str) -> list[Order]:\n"
" pass\n"
),
"src/customers.py": (
"class Customer:\n"
" pass\n"
"\n"
"class Subscription:\n"
" # NOTE: Subscription is used heavily but not in glossary\n"
" pass\n"
"\n"
"def find_customer(email: str) -> Customer:\n"
" pass\n"
),
"src/products.py": (
"class Product:\n"
" pass\n"
"\n"
"class Inventory:\n"
" pass\n"
"\n"
"def find_product(sku: str) -> Product:\n"
" pass\n"
),
# Note: Discount is defined in glossary but never used in code.
}
def extract_glossary_terms(context_md_text: str) -> List[str]:
"""Pull bold terms from CONTEXT.md `**Term**:` patterns."""
return re.findall(r"\*\*([^*]+?)\*\*\s*:", context_md_text)
def walk_codebase(root: Path, extensions: Set[str], exclude_dirs: Set[str]) -> List[Path]:
found: List[Path] = []
for path in root.rglob("*"):
if path.is_dir():
continue
if any(part in exclude_dirs for part in path.parts):
continue
if path.suffix in extensions:
found.append(path)
return found
def is_test_file(path: Path) -> bool:
s = str(path).replace("\\", "/")
return any(hint in s for hint in TEST_FILE_HINTS)
def count_term_in_text(text: str, term: str) -> int:
pattern = re.compile(rf"\b{re.escape(term)}\b", re.IGNORECASE)
return len(pattern.findall(text))
def count_proper_nouns(text: str) -> Counter:
counter: Counter = Counter()
for match in PROPER_NOUN_RE.finditer(text):
counter[match.group(1)] += 1
return counter
def analyze(
context_md_text: str,
code_files: List[Tuple[str, str]],
min_proper_noun_frequency: int,
) -> Dict[str, Any]:
"""code_files: list of (relative_path, text) tuples."""
glossary_terms = extract_glossary_terms(context_md_text)
glossary_term_set_lower = {t.lower() for t in glossary_terms}
# Per-term usage count in non-test files
term_usage: Dict[str, int] = {t: 0 for t in glossary_terms}
code_proper_nouns: Counter = Counter()
files_scanned = 0
files_tests_skipped = 0
for path_str, text in code_files:
path = Path(path_str)
if is_test_file(path):
files_tests_skipped += 1
continue
files_scanned += 1
for term in glossary_terms:
term_usage[term] += count_term_in_text(text, term)
for noun, count in count_proper_nouns(text).items():
code_proper_nouns[noun] += count
# Dead glossary: terms with zero usage
dead_terms = [t for t, n in term_usage.items() if n == 0]
# Code-only proper nouns: frequent capitalized identifiers NOT in glossary
# and NOT in the generic stop-list
code_only: List[Tuple[str, int]] = []
for noun, count in code_proper_nouns.most_common():
if count < min_proper_noun_frequency:
break
if noun.lower() in glossary_term_set_lower:
continue
if noun in GENERIC_WORDS:
continue
code_only.append((noun, count))
return {
"files_scanned": files_scanned,
"files_tests_skipped": files_tests_skipped,
"glossary_term_count": len(glossary_terms),
"term_usage": term_usage,
"dead_glossary_terms": dead_terms,
"code_only_proper_nouns": code_only,
"min_proper_noun_frequency": min_proper_noun_frequency,
}
def render_human(result: Dict[str, Any]) -> str:
out: List[str] = []
out.append("Glossary↔Code consistency report")
out.append(f" Files scanned: {result['files_scanned']} (test files skipped: {result['files_tests_skipped']})")
out.append(f" Glossary terms: {result['glossary_term_count']}")
out.append("")
out.append("Term usage (occurrences in non-test code):")
for term, count in sorted(result["term_usage"].items(), key=lambda kv: (-kv[1], kv[0])):
marker = " " if count > 0 else "!!"
out.append(f" {marker} {term:<30s} {count}")
out.append("")
if result["dead_glossary_terms"]:
out.append("DEAD GLOSSARY (defined but never used in code) — grill these:")
for term in result["dead_glossary_terms"]:
out.append(f" - '{term}': dead term, or rename happened?")
else:
out.append("DEAD GLOSSARY: (none — every defined term is used in code)")
out.append("")
if result["code_only_proper_nouns"]:
out.append(
f"CODE-ONLY PROPER NOUNS (>= {result['min_proper_noun_frequency']}x, not in glossary, not generic) — grill these:"
)
for noun, count in result["code_only_proper_nouns"]:
out.append(f" - '{noun}' ({count} occurrences): domain term that needs definition, or generic?")
else:
out.append("CODE-ONLY PROPER NOUNS: (none above threshold — glossary covers the frequent domain nouns)")
return "\n".join(out)
def run_sample(min_freq: int) -> Dict[str, Any]:
files = [(p, t) for p, t in SAMPLE_CODE_FILES.items()]
return analyze(SAMPLE_CONTEXT_MD, files, min_freq)
def main(argv: List[str]) -> int:
parser = argparse.ArgumentParser(description=__doc__.split("\n")[0])
parser.add_argument("--context", help="Path to CONTEXT.md")
parser.add_argument("--code", help="Path to codebase root")
parser.add_argument(
"--extensions",
help="Comma-separated source extensions to scan (default: common languages)",
default=None,
)
parser.add_argument(
"--min-frequency",
type=int,
default=3,
help="Minimum occurrences for a code-only proper noun to surface (default: 3)",
)
parser.add_argument("--sample", action="store_true", help="Run on the embedded sample data")
parser.add_argument("--output", choices=["human", "json"], default="human")
args = parser.parse_args(argv)
if args.sample:
result = run_sample(args.min_frequency)
elif args.context and args.code:
context_path = Path(args.context)
code_root = Path(args.code)
if not context_path.exists():
print(f"error: {args.context} not found", file=sys.stderr)
return 2
if not code_root.exists():
print(f"error: {args.code} not found", file=sys.stderr)
return 2
if args.extensions:
exts = {e.strip() if e.strip().startswith(".") else "." + e.strip() for e in args.extensions.split(",")}
else:
exts = DEFAULT_EXTENSIONS
files: List[Tuple[str, str]] = []
for p in walk_codebase(code_root, exts, DEFAULT_EXCLUDE_DIRS):
try:
files.append((str(p), p.read_text(encoding="utf-8", errors="ignore")))
except (OSError, UnicodeDecodeError):
continue
result = analyze(context_path.read_text(encoding="utf-8"), files, args.min_frequency)
else:
parser.print_help()
return 0
if args.output == "json":
print(json.dumps(result, indent=2))
else:
print(render_human(result))
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
Hỗ trợ quyết định giá, đóng gói và kiếm tiền: bậc giá, freemium, dùng thử, tăng giá, value metric, Van Westendorp và mức sẵn lòng chi trả.
---
name: pricing
description: "When the user wants help with pricing decisions, packaging, or monetization strategy. Also use when the user mentions 'pricing,' 'pricing tiers,' 'freemium,' 'free trial,' 'packaging,' 'price increase,' 'value metric,' 'Van Westendorp,' 'willingness to pay,' 'monetization,' 'how much should I charge,' 'my pricing is wrong,' 'pricing page,' 'annual vs monthly,' 'per seat pricing,' 'should I offer a free plan,' 'pricing page teardown,' 'pricing page audit,' 'is my pricing page AI-readable,' or 'can AI read my pricing.' Use this whenever someone is figuring out what to charge, how to structure their plans, or wants to audit a pricing page (for humans and for the AI agents that shortlist tools). For in-app upgrade screens, see paywalls. For offer construction (bonuses, guarantees, value framing, naming) on services/courses/coaching/high-ticket B2B, see offers."
metadata:
version: 2.1.1
---
# Pricing Strategy
You are an expert in SaaS pricing and monetization strategy. Your goal is to help design pricing that captures value, drives growth, and aligns with customer willingness to pay.
## Before Starting
**Check for product marketing context first:**
If `.agents/product-marketing.md` exists (or `.claude/product-marketing.md`, or the legacy `product-marketing-context.md` filename, in older setups), read it before asking questions. Use that context and only ask for information not already covered or specific to this task.
Gather this context (ask if not provided):
### 1. Business Context
- What type of product? (SaaS, marketplace, e-commerce, service)
- What's your current pricing (if any)?
- What's your target market? (SMB, mid-market, enterprise)
- What's your go-to-market motion? (self-serve, sales-led, hybrid)
### 2. Value & Competition
- What's the primary value you deliver?
- What alternatives do customers consider?
- How do competitors price?
### 3. Current Performance
- What's your current conversion rate?
- What's your ARPU and churn rate?
- Any feedback on pricing from customers/prospects?
### 4. Goals
- Optimizing for growth, revenue, or profitability?
- Moving upmarket or expanding downmarket?
---
## Pricing Fundamentals
### The Three Pricing Axes
**1. Packaging** — What's included at each tier?
- Features, limits, support level
- How tiers differ from each other
**2. Pricing Metric** — What do you charge for?
- Per user, per usage, flat fee
- How price scales with value
**3. Price Point** — How much do you charge?
- The actual dollar amounts
- Perceived value vs. cost
### Value-Based Pricing
Price should be based on value delivered, not cost to serve:
- **Customer's perceived value** — The ceiling
- **Your price** — Between alternatives and perceived value
- **Next best alternative** — The floor for differentiation
- **Your cost to serve** — Only a baseline, not the basis
**Key insight:** Price between the next best alternative and perceived value.
**Don't anchor on the wrong things:**
- **Not competitor-based** — matching a competitor's price copies their strategy, not their economics. It's a data point, not a target.
- **Not cost-based** — cost is a floor, never the basis. Value + differentiation set the price.
---
## Initial Pricing — "Pick a Price You Can Learn From"
The frameworks below (value metrics, tiers, Van Westendorp) are for optimizing a price. **On day one you don't have a price to optimize — you have a bet to place.** The goal of your first price is *learning*, not precision. Pick a number, ship it, and let real buyers tell you if it's wrong.
### The $10 / $100 / $1,000 rule of thumb
When you have nothing to go on, start with the order of magnitude that matches who you serve:
- **~$10/mo** — prosumer / individual, high volume, low touch
- **~$100/mo** — SMB / team tool, the SaaS default
- **~$1,000/mo** — mid-market / business-critical / sales-assisted
Pick the bucket by **who the customer is and how much value you deliver**, then start near the round number. You can move within the bucket fast once you have signal.
### Avoid the $9 trap
Resist the urge to price ultra-low (e.g. **$9/mo**) to reduce friction. Ultra-low pricing:
- Creates **false traction** — signups that look like validation but come from people who'd never pay a real price
- **Traps you** — it's far harder to raise a price 5–10x later than to have started higher, and your cheapest customers churn most and complain loudest (see [references/pricing-models.md](references/pricing-models.md) on low-price retention)
Round-and-slightly-higher beats clever-and-cheap.
### "Just charge $50 and see what happens"
When early Intercom agonized over pricing, Jason Fried's advice was essentially: **just charge $50 and see what happens.** Stop modeling; get a real signal. If people pay without flinching, raise it. If nobody bites, you've learned something for the cost of a week, not a quarter.
**For the eight ways to structure how you charge (flat, usage, tier, user, feature, credit, outcome, hybrid) and the value/price ratio:** See [references/pricing-models.md](references/pricing-models.md).
---
## Value Metrics
### What is a Value Metric?
The value metric is what you charge for—it should scale with the value customers receive.
**Good value metrics:**
- Align price with value delivered
- Are easy to understand
- Scale as customer grows
- Are hard to game
### Common Value Metrics
| Metric | Best For | Example |
|--------|----------|---------|
| Per user/seat | Collaboration tools | Slack, Notion |
| Per usage | Variable consumption | AWS, Twilio |
| Per feature | Modular products | HubSpot add-ons |
| Per contact/record | CRM, email tools | Mailchimp |
| Per transaction | Payments, marketplaces | Stripe |
| Flat fee | Simple products | Basecamp |
### Choosing Your Value Metric
Ask: "As a customer uses more of [metric], do they get more value?"
- If yes → good value metric
- If no → price doesn't align with value
**The value metric picks the pricing model.** Once you know what scales with value, choose how to charge on it — flat, usage, tier, user, feature, credit, outcome, or a hybrid. See [references/pricing-models.md](references/pricing-models.md).
---
## Tier Structure Overview
### Good-Better-Best Framework
**Good tier (Entry):** Core features, limited usage, low price
**Better tier (Recommended):** Full features, reasonable limits, anchor price
**Best tier (Premium):** Everything, advanced features, 2-3x Better price
### Tier Differentiation
- **Feature gating** — Basic vs. advanced features
- **Usage limits** — Same features, different limits
- **Support level** — Email → Priority → Dedicated
- **Access** — API, SSO, custom branding
**For detailed tier structures and persona-based packaging**: See [references/tier-structure.md](references/tier-structure.md)
---
## Pricing Research
### Van Westendorp Method
Four questions that identify acceptable price range:
1. Too expensive (wouldn't consider)
2. Too cheap (question quality)
3. Expensive but might consider
4. A bargain
Analyze intersections to find optimal pricing zone.
### MaxDiff Analysis
Identifies which features customers value most:
- Show sets of features
- Ask: Most important? Least important?
- Results inform tier packaging
**For detailed research methods**: See [references/research-methods.md](references/research-methods.md)
---
## When to Raise Prices
### Signs It's Time
**Market signals:**
- Competitors have raised prices
- Prospects don't flinch at price
- "It's so cheap!" feedback
**Business signals:**
- Very high conversion rates (>40%)
- Very low churn (<3% monthly)
- Strong unit economics
**Product signals:**
- Significant value added since last pricing
- Product more mature/stable
### Price Increase Strategies
1. **Grandfather existing** — New price for new customers only
2. **Delayed increase** — Announce 3-6 months out
3. **Tied to value** — Raise price but add features
4. **Plan restructure** — Change plans entirely
### Rollout Methodology
A price change is a rollout, not a switch you flip. Sequence it to de-risk:
1. **Test on new customers first.** Raise the price only for *new* signups and watch conversion. New customers have no anchor and no relationship at stake, so they give you a clean read on whether the market accepts the number — before you touch a single existing account.
2. **Don't reflexively grandfather forever.** Grandfathering feels kind, but it can leave enormous money on the table. Run the math: a customer paying **$50/mo** who *should* be at **$250/mo** is a **$2,400/yr** gap — and $200/mo you're subsidizing indefinitely across your whole base. Grandfather as a *transition* (a grace period), not a permanent exemption.
3. **Roll out small, then gradually.** Move **5–10%** of existing customers to the new price first. Watch churn and support volume for a cycle, then expand in staggered waves. A staggered rollout contains the blast radius and gives you an off-ramp if churn spikes.
4. **Communicate the *why*, months ahead, with a generous offer.** Tell customers why the price is changing (usually: more value shipped) well in advance. Soften it: lock-in-the-old-price-if-you-upgrade-to-annual-now, an extended grace window, or a one-time credit. Advance notice + a generous option converts a resentment moment into a loyalty one.
Expect — and accept — some churn. The customers most likely to leave over a justified increase are usually your least-profitable, highest-support, most price-sensitive accounts.
---
## Pricing Page Best Practices
### Above the Fold
- Clear tier comparison table
- Recommended tier highlighted
- Monthly/annual toggle
- Primary CTA for each tier
### Common Elements
- Feature comparison table
- Who each tier is for
- FAQ section
- Annual discount callout (17-20%)
- Money-back guarantee
- Customer logos/trust signals
### Pricing Psychology
- **Anchoring:** Show higher-priced option first
- **Decoy effect:** Middle tier should be best value
- **Charm pricing:** $49 vs. $50 (for value-focused)
- **Round pricing:** $50 vs. $49 (for premium)
---
## Pricing Page Teardown
When someone wants to audit an existing pricing *page* for **clarity, transparency, and AI-readability** (not the pricing strategy itself, and not conversion-rate optimization — that's `cro`), run a **teardown** that scores it across two axes and returns prioritized fixes:
- **Human buyer experience** — value-prop clarity, plan differentiation, cognitive load, trust signals, pricing psychology, and price transparency.
- **AI-agent readiness** — whether the LLMs and agents that increasingly shortlist and compare tools can actually read and quote your pricing: machine-readable prices (not locked in an image or behind "Contact us"), extractable FAQ/objection coverage, per-tier depth stated in text, and structured data. Buyers now ask ChatGPT/Perplexity/Claude "what's the best X and what does it cost?" *before* visiting — a pricing page an agent can't parse loses deals you never see.
**Fast check — the "paste test":** give the pricing URL to a browsing-capable AI (Perplexity, ChatGPT with search, Claude with web) — or paste the rendered page text — and ask "what are the plans and prices?" A clean miss means agents fetching your page will struggle too (a heuristic, not proof every agent fails).
The AI-readiness fixes are usually high-impact, low-effort (put prices in text, add `Offer` schema). Hand implementation to **schema** (Product/Offer JSON-LD) and **ai-seo** (extractability, AI-bot access, `llms.txt`).
**For the full 10-dimension rubric, scoring, and report template:** See [references/pricing-page-teardown.md](references/pricing-page-teardown.md). *(AI-agent-readiness lens adapted from Kyle Poyar / Growth Unhinged.)*
---
## Pricing Checklist
### Before Setting Prices
- [ ] Defined target customer personas
- [ ] Researched competitor pricing
- [ ] Identified your value metric
- [ ] Conducted willingness-to-pay research
- [ ] Mapped features to tiers
### Pricing Structure
- [ ] Chosen number of tiers
- [ ] Differentiated tiers clearly
- [ ] Set price points based on research
- [ ] Created annual discount strategy
- [ ] Planned enterprise/custom tier
---
## Task-Specific Questions
1. What pricing research have you done?
2. What's your current ARPU and conversion rate?
3. What's your primary value metric?
4. Who are your main pricing personas?
5. Are you self-serve, sales-led, or hybrid?
6. What pricing changes are you considering?
---
## Related Skills
- **churn-prevention**: For cancel flows, save offers, and reducing revenue churn
- **cro**: For optimizing pricing page conversion
- **ai-seo**: For making the pricing page extractable/citable by AI (the teardown's AI-agent-readiness axis)
- **schema**: For Product/Offer structured data so machines can read your tiers and prices
- **copywriting**: For pricing page copy
- **marketing-psychology**: For pricing psychology principles
- **ab-testing**: For testing pricing changes
- **revops**: For deal desk processes and pipeline pricing
- **sales-enablement**: For proposal templates and pricing presentations
FILE:evals/evals.json
{
"skill_name": "pricing",
"evals": [
{
"id": 1,
"prompt": "Help me figure out pricing for our new SaaS product. It's a customer support platform for e-commerce stores. We're not sure whether to charge per agent, per ticket, or flat rate. Currently thinking $49-199/month range.",
"expected_output": "Should check for product-marketing.md first. Should apply the three pricing axes framework: packaging (what's included in each tier), pricing metric (per agent, per ticket, flat rate — evaluate each), price point ($49-199 range evaluation). Should discuss value metrics and which aligns best with value delivered (per agent is common in support, but per ticket aligns with usage). Should recommend a good-better-best tier structure. Should address pricing psychology. Should provide a specific pricing recommendation with rationale.",
"assertions": [
"Checks for product-marketing.md",
"Applies three pricing axes framework",
"Evaluates multiple pricing metrics",
"Discusses which metric aligns with value delivered",
"Recommends good-better-best tier structure",
"Addresses pricing psychology",
"Provides specific pricing recommendation with rationale"
],
"files": []
},
{
"id": 2,
"prompt": "We want to raise our prices by 30%. We've been at $29/month for 2 years and we've added a lot of features. How do we do this without losing customers?",
"expected_output": "Should apply the 'when to raise prices' and price increase strategies sections. Should recommend a strategy: grandfather existing customers (or give them a grace period), tie the increase to new value, communicate the change clearly with advance notice, consider an annual billing discount as a softening measure. Should address different approaches (immediate for new customers, delayed for existing). Should recommend specific communication strategy. Should note that some churn is expected and acceptable.",
"assertions": [
"Applies price increase strategies",
"Recommends grandfathering or grace period approach",
"Recommends tying increase to new value",
"Provides communication strategy",
"Addresses new vs existing customer timing",
"Suggests annual billing as softening measure",
"Notes some churn is expected"
],
"files": []
},
{
"id": 3,
"prompt": "how do we figure out what people will actually pay? we're launching a new product and have no idea what to charge.",
"expected_output": "Should trigger on casual phrasing. Should apply the pricing research methods: Van Westendorp price sensitivity analysis (too cheap, bargain, expensive, too expensive), MaxDiff for feature importance, competitive benchmarking. Should explain how to run each method. Should also recommend simpler approaches: talking to potential customers, analyzing competitor pricing, testing different price points. Should provide a practical pricing research plan they can execute.",
"assertions": [
"Triggers on casual phrasing",
"Applies Van Westendorp price sensitivity method",
"Applies MaxDiff for feature importance",
"Recommends competitive benchmarking",
"Explains how to run each method",
"Suggests practical alternatives (customer interviews, competitive analysis)",
"Provides executable pricing research plan"
],
"files": []
},
{
"id": 4,
"prompt": "We have a Basic ($19), Pro ($49), and Enterprise (custom) plan. The Pro plan gets 70% of signups. Should we add a plan between Pro and Enterprise?",
"expected_output": "Should apply the good-better-best tier structure framework. Should analyze the current situation: Pro capturing 70% is actually healthy, but the gap to Enterprise suggests there may be mid-market customers underserved. Should evaluate whether a 4th tier makes sense: does it address a real gap, or will it create choice paralysis? Should apply pricing psychology (Hick's Law — more options can reduce decisions). Should recommend either a 4th tier with clear differentiation or adjusting the Pro plan to better bridge the gap.",
"assertions": [
"Applies good-better-best tier structure",
"Analyzes current tier performance",
"Evaluates whether 4th tier addresses real gap",
"Considers choice paralysis risk",
"Applies pricing psychology (Hick's Law)",
"Provides specific recommendation with rationale"
],
"files": []
},
{
"id": 5,
"prompt": "What pricing psychology tactics should we use on our pricing page? We want the $79 plan to be the most popular.",
"expected_output": "Should apply the pricing psychology section: anchoring (show the $79 plan next to a higher-priced plan), decoy effect (make the lower plan look less valuable), visual emphasis (highlight or 'recommend' the $79 plan), charm pricing ($79 vs $80), Rule of 100 (percentage discounts below $100, dollar discounts above), loss framing (show what lower plans miss). Should provide specific pricing page design recommendations. Should cross-reference cro for broader pricing page optimization.",
"assertions": [
"Applies pricing psychology tactics",
"Applies anchoring effect",
"Applies decoy effect or visual emphasis",
"Applies charm pricing or Rule of 100",
"Provides specific pricing page recommendations",
"Cross-references cro or marketing-psychology"
],
"files": []
},
{
"id": 6,
"prompt": "Our pricing page conversion rate is only 1.5%. Can you review the page and suggest improvements?",
"expected_output": "Should recognize this is a pricing page conversion optimization task, not a pricing strategy task. Should defer to or cross-reference the cro skill, which handles pricing page conversion rate optimization including plan comparison clarity, CTA optimization, and trust signals. Pricing-strategy focuses on the actual pricing decisions (what to charge, how to package), not the page design.",
"assertions": [
"Recognizes this as pricing page CRO, not pricing strategy",
"References or defers to cro skill",
"Explains that pricing is about pricing decisions",
"Does not attempt full page CRO audit"
],
"files": []
},
{
"id": 7,
"prompt": "Can you tear down our pricing page? I want to know if it is clear for buyers, and also whether AI tools like ChatGPT or Perplexity can actually read our prices when someone asks them to compare tools in our category.",
"expected_output": "Should run the two-axis pricing page teardown (references/pricing-page-teardown.md), not a generic CRO audit. Axis 1 (human buyer experience): value-prop clarity, plan differentiation, cognitive load, trust signals, pricing psychology, price transparency. Axis 2 (AI-agent readiness): machine-readable pricing (real numbers in HTML/text, not locked in an image, JS-only render, or behind Contact us), extractable FAQ/objection coverage, per-tier depth stated in text, and structured data (Product/Offer schema) + AI-bot crawlability. Should recommend the paste test (paste the URL into an LLM and ask for plans and prices; if it cannot answer, an AI shopping for the buyer cannot either). Should prioritize fixes by impact x effort and note AI-readiness fixes are often high-impact/low-effort. Should hand implementation to schema (Product/Offer JSON-LD) and ai-seo (extractability, AI-bot access, llms.txt). May credit the AI-agent-readiness lens to Kyle Poyar.",
"assertions": [
"Runs the two-axis teardown (human buyer experience AND AI-agent readiness)",
"Checks machine-readable pricing (not locked in an image / JS-only / behind Contact us)",
"Recommends the paste test (an LLM can correctly quote plans and prices)",
"Hands off to schema (Product/Offer structured data) and ai-seo (extractability / AI-bot access / llms.txt)",
"Prioritizes fixes by impact x effort; flags AI-readiness fixes as often high-impact low-effort",
"Does not treat this as pure conversion-rate CRO"
],
"files": []
},
{
"id": 8,
"prompt": "We're launching an AI writing tool for solo creators next week and I genuinely have no idea what to charge on day one. I was going to just do $9/month to get people in the door. What price should I pick and how should I even structure it?",
"expected_output": "Should treat this as an INITIAL pricing question, not a price-optimization one — the goal of a first price is learning, not precision ('pick a price you can learn from'). Should apply the $10/$100/$1,000 rule of thumb and place a solo-creator tool near the ~$10 bucket. Should warn against the $9 trap (false traction, hard to raise later, cheapest customers churn most). May cite the Intercom/Jason Fried 'just charge $50 and see what happens' idea — ship a price and get real signal. Should reject competitor-based and cost-based anchoring in favor of value + differentiation. Should recommend a pricing MODEL/structure: for an AI actions-based tool, credit-based or usage-based (or a hybrid) is a natural fit; may reference the 8 models. May mention the ~10:1 value/price ratio and the low-price-hurts-retention counterpoint (when in doubt, price higher).",
"assertions": [
"Frames the first price as a learning bet, not an optimization",
"Applies the $10/$100/$1,000 rule of thumb and buckets the tool appropriately",
"Warns against the $9 / ultra-low trap (false traction, hard to raise, low-price churn)",
"References 'just charge $50 and see' / getting a real signal (Intercom/Jason Fried)",
"Rejects competitor-based and cost-based pricing in favor of value + differentiation",
"Recommends a pricing model/structure (e.g. credit-based or usage-based for an AI tool)",
"Notes value/price ratio (~10:1) or that low prices hurt retention"
],
"files": []
}
]
}
FILE:references/pricing-models.md
# Pricing Models
The eight core ways to structure *how* you charge. This is distinct from the value metric (what unit you charge on) and the tier structure (how you package). Most real products **combine** two or more of these.
## Contents
- The 8 Pricing Models
- Combining Models
- The Value/Price Ratio
- The Low-Price Retention Counterpoint
---
## The 8 Pricing Models
| Model | How it works | Best when | Reference |
|-------|-------------|-----------|-----------|
| **Flat-rate** | One price, one product, everyone pays the same | Simple product, one persona, you want zero pricing friction | Basecamp |
| **Usage-based** | Pay for what you consume (metered) | Value scales directly with volume; consumption is variable and easy to meter | Stripe |
| **Tier-based** | Good-better-best packages at set prices | Distinct segments with different needs and budgets | Kinsta |
| **User-based** | Price per seat/user | Value grows as more people in the org use it (collaboration) | Notion |
| **Feature-based** | Price gated by which capabilities are unlocked | Clear feature tiers map to willingness to pay | Intercom |
| **Credit-based** | Buy a bucket of credits, spend them on actions | Usage is lumpy or bursty; you want prepaid commitment and simple mental accounting | Audible |
| **Outcome-based** | Pay per result delivered (resolution, task completed) | You can measure and attribute the outcome, and the outcome is what the buyer actually wants | Intercom Fin, Zapier |
| **Hybrid** | Deliberate mix (e.g. platform fee + usage, or seats + credits) | A single model under- or over-charges different customers | Drift |
### When to reach for each
- **Flat-rate** — reach for it first if you can. It's the easiest to sell, easiest to understand, easiest to forecast. The tradeoff: you leave money on the table with your biggest customers.
- **Usage-based** — the fairest model when consumption tracks value, but revenue is less predictable and buyers fear a surprise bill. Pair with spend caps or alerts.
- **Tier-based** — the default for self-serve SaaS. Lets one page serve SMB through mid-market.
- **User-based** — only if value genuinely rises with headcount. If it doesn't, seats punish adoption (teams share logins to avoid paying).
- **Feature-based** — powerful for segmentation, but don't gate the feature that delivers your core value; gate the ones that separate casual from serious users.
- **Credit-based** — good for AI/actions-based products where each action has a cost. Credits decouple price from a single unit and make prepayment feel natural.
- **Outcome-based** — the emerging model for AI agents (charge per resolved ticket, per automation run). Highest trust because the buyer only pays when they win — but only viable when the outcome is measurable and clearly attributable to you.
- **Hybrid** — where most mature products end up. A base platform fee for predictability plus a usage/outcome component for upside.
---
## Combining Models
These aren't mutually exclusive. Common combinations:
- **Tiers + per-user** — seats within each package (most B2B SaaS)
- **Platform fee + usage** — predictable base, variable upside (Twilio-style)
- **Seats + credits** — pay per person, then top up credits for heavy actions
- **Feature tiers + outcome** — unlock capabilities by tier, charge per result on top
Pick the primary model from the value metric, then layer a second only if a single model clearly mis-prices a real segment.
---
## The Value/Price Ratio
Aim for roughly a **10:1 value-to-price ratio** (Ryan Kulp): the customer should perceive about **10x more value than they pay**. This is the buffer that makes the purchase feel obvious rather than negotiated, and it leaves headroom to raise prices later as you add value.
If you can't articulate 10x value, the problem is usually the offer or the positioning, not the price point.
---
## The Low-Price Retention Counterpoint
Charging too little is not the safe choice. **Low prices hurt retention** (Patrick Campbell / ProfitWell data, echoed by operators like Josh Pigford of SpyFu and Tyler Tringas): under-priced customers churn *more*, not less, because a low price signals low value and attracts the least-committed, most price-sensitive buyers.
Related: the **discount-asker signal** — customers who negotiate for a discount tend to churn at roughly **2x** the rate of full-price customers. Discounting to close a deal often buys a customer who leaves anyway.
**Implication:** when in doubt, price higher. It's easier to grandfather a price down than to claw one up, and a higher price selects for better-fit, longer-retained customers.
FILE:references/pricing-page-teardown.md
# Pricing Page Teardown
A structured way to score a live pricing page and return prioritized fixes. It grades **two axes**: the classic **human buyer experience**, and — the newer, higher-leverage lens — **AI-agent readiness**: whether the LLMs and agents that increasingly shortlist and compare tools can actually read, quote, and recommend your pricing.
> **Framework credit:** the two-axis structure and especially the AI-agent-readiness lens are adapted from **Kyle Poyar's** (Growth Unhinged) pricing-page teardown. Learn-from-only — this rubric is authored independently; credit the framing to Poyar.
## Why the second axis matters now
Buyers increasingly ask ChatGPT, Perplexity, and Claude *"what's the best [category] tool and what does it cost?"* before they ever hit your site. If your price is trapped in an image, rendered only by JavaScript, or missing from the page's text, a text-fetching agent often can't read it — some agents render JS or fall back to vision/OCR, but many don't, so don't count on it. And a "Contact us" tier gives an agent no public number to quote at all. When the agent can't read your price, it recommends and quotes the competitor whose pricing it *can*. This axis is the pricing-page complement to `ai-seo` and `schema` — neither *guarantees* a citation, but a page a fetcher can't parse makes one much less likely.
**The 30-second test — the "paste test":** give the pricing URL to a **browsing-capable** AI (Perplexity, ChatGPT with search, or Claude with web) — or paste the page's *rendered* text — and ask *"What are the plans and prices?"* If it can't answer correctly and completely, agents fetching your page the same way will struggle too. It's a heuristic, not proof every agent fails (some render JS or use vision), but a clean miss is a real finding worth fixing.
## The rubric
Score each dimension **Pass / Partial / Gap** (or 1–5 if you want a number). Two sub-scores (one per axis) plus a prioritized fix list is the deliverable — not a single vanity number.
### Axis 1 — Human buyer experience
| # | Dimension | Passing looks like | Common gaps |
|---|---|---|---|
| 1 | **Value-prop clarity** | Above the fold: what you get + why it's worth it, in the buyer's words | Feature list with no outcome; "flexible plans for every team" |
| 2 | **Plan clarity / differentiation** | Obvious which plan is for whom and exactly how they differ | Feature-soup tables; tiers that blur together; no "who it's for" |
| 3 | **Cognitive load** | A buyer can decide in <30s | Too many tiers (5+), unexplained jargon, decision paralysis |
| 4 | **Trust signals** | Logos, testimonials, security/compliance, a guarantee near the CTA | No proof; trust content buried below the fold |
| 5 | **Pricing psychology** | A recommended/anchor tier, sensible anchoring, coherent charm vs. round pricing | No recommended tier; highest price hidden last; random price endings |
| 6 | **Transparency** | The actual price is shown; what's in/out is clear; no surprise fees | "Contact us" on every tier; hidden overages; usage limits omitted |
### Axis 2 — AI-agent readiness (the novel lens)
| # | Dimension | Passing looks like | Common gaps |
|---|---|---|---|
| 7 | **Machine-readable pricing** | The real numbers are in the page's HTML/text | Price in an image/SVG, JS-only render, or a PDF — text-fetching crawlers get nothing reliable; "Contact sales" leaves no public number to quote |
| 8 | **FAQ / objection coverage** | Extractable answers to "does it do X," "what's the limit," "can I cancel," "is there a free trial" | No FAQ, or answers only in a support portal an agent won't reach |
| 9 | **Per-tier depth in text** | Each plan's inclusions, limits, and quotas stated in words | Differences shown only as checkmark columns in an image; limits unnamed |
| 10 | **Structured data & extractability** | `Product`/`Offer` schema markup, clean semantic HTML, AI search/agent bots allowed to crawl (`llms.txt` is a nice-to-have, not yet a standard) | No schema; pricing behind auth/interaction; AI *search* bots blocked in robots.txt |
Dimensions 7 and 10 hand off to **`schema`** (Product/Offer JSON-LD) and **`ai-seo`** (extractability, AI-bot access, `llms.txt`) for implementation.
## How to run it
1. **Load context** — read `.agents/product-marketing.md` (ICP, positioning) so "clarity" is judged against the *right* buyer.
2. **Fetch the page as an agent would** — get the rendered text/HTML, not a screenshot. Note immediately whether prices appear in the text (that's dimension 7).
3. **Run the paste test** — ask an LLM for the plans and prices from the URL; record what it gets wrong or misses.
4. **Score all 10 dimensions** Pass/Partial/Gap with a one-line reason each.
5. **Prioritize fixes** by impact × effort. AI-readiness gaps are often *high impact, low effort* (add text prices, add Offer schema) — surface those first.
## Output template
```markdown
# Pricing Page Teardown — [url] — [date]
## Scores
- Human buyer experience: [X/6 passing]
- AI-agent readiness: [X/4 passing]
## Paste test
[What an LLM returned for "plans and prices" — and what it got wrong/missed]
## Dimension-by-dimension
| # | Dimension | Verdict | Note |
|---|-----------|---------|------|
| 1 | Value-prop clarity | Pass/Partial/Gap | ... |
| … | … | … | … |
## Prioritized fixes (impact × effort)
1. [High/low] — [fix] — [why it matters] — [→ schema / ai-seo / cro if handing off]
2. ...
## The one thing
[The single highest-leverage fix — often "put your actual prices in text + add Offer schema so AI can quote you."]
```
## Common failure patterns
- **The image-price** — a beautiful pricing graphic with the numbers baked in. Humans love it; text-fetching agents (and screen readers) usually can't read it. Put prices in text; the image can stay as decoration.
- **"Contact us" everywhere** — sometimes right for true enterprise, but if *all* tiers hide price, both humans and agents bounce to a competitor with numbers. Show at least a starting price or a representative range.
- **Checkmark-only tables** — feature differences shown only as ✓/✗ columns in an image or icon font. State the actual limits and inclusions in words.
- **JS-only render / auth wall** — if the price only appears after interaction or login, most fetchers won't see it (only JS-rendering agents might).
- **Blocked AI *search* bots** — the crawlers that feed AI *answers* are the search agents, not the training crawlers: OpenAI's `OAI-SearchBot`, Anthropic's `Claude-SearchBot` / `Claude-User`, Perplexity's `PerplexityBot`. Blocking `GPTBot` only opts out of model *training*, not ChatGPT Search — so check which bots your robots.txt actually blocks. (Bot access is `ai-seo`'s domain — hand it off there.)
## Related
- `schema` — Product/Offer JSON-LD so machines read your tiers and prices.
- `ai-seo` — extractability, AI-bot access, `llms.txt`, getting cited by AI answers.
- `cro` — converting the human once the page is clear.
- `copywriting` — the value-prop and tier copy the teardown flags.
FILE:references/research-methods.md
# Pricing Research Methods
## Contents
- Van Westendorp Price Sensitivity Meter (The Four Questions, How to Analyze, Survey Tips, Sample Output)
- MaxDiff Analysis (How It Works, Example Survey Question, Analyzing Results, Using MaxDiff for Packaging)
- Willingness to Pay Surveys
- Usage-Value Correlation Analysis
## Van Westendorp Price Sensitivity Meter
The Van Westendorp survey identifies the acceptable price range for your product.
### The Four Questions
Ask each respondent:
1. "At what price would you consider [product] to be so expensive that you would not consider buying it?" (Too expensive)
2. "At what price would you consider [product] to be priced so low that you would question its quality?" (Too cheap)
3. "At what price would you consider [product] to be starting to get expensive, but you still might consider it?" (Expensive/high side)
4. "At what price would you consider [product] to be a bargain—a great buy for the money?" (Cheap/good value)
### How to Analyze
1. Plot cumulative distributions for each question
2. Find the intersections:
- **Point of Marginal Cheapness (PMC):** "Too cheap" crosses "Expensive"
- **Point of Marginal Expensiveness (PME):** "Too expensive" crosses "Cheap"
- **Optimal Price Point (OPP):** "Too cheap" crosses "Too expensive"
- **Indifference Price Point (IDP):** "Expensive" crosses "Cheap"
**The acceptable price range:** PMC to PME
**Optimal pricing zone:** Between OPP and IDP
### Survey Tips
- Need 100-300 respondents for reliable data
- Segment by persona (different willingness to pay)
- Use realistic product descriptions
- Consider adding purchase intent questions
### Sample Output
```
Price Sensitivity Analysis Results:
─────────────────────────────────
Point of Marginal Cheapness: $29/mo
Optimal Price Point: $49/mo
Indifference Price Point: $59/mo
Point of Marginal Expensiveness: $79/mo
Recommended range: $49-59/mo
Current price: $39/mo (below optimal)
Opportunity: 25-50% price increase without significant demand impact
```
---
## MaxDiff Analysis (Best-Worst Scaling)
MaxDiff identifies which features customers value most, informing packaging decisions.
### How It Works
1. List 8-15 features you could include
2. Show respondents sets of 4-5 features at a time
3. Ask: "Which is MOST important? Which is LEAST important?"
4. Repeat across multiple sets until all features compared
5. Statistical analysis produces importance scores
### Example Survey Question
```
Which feature is MOST important to you?
Which feature is LEAST important to you?
□ Unlimited projects
□ Custom branding
□ Priority support
□ API access
□ Advanced analytics
```
### Analyzing Results
Features are ranked by utility score:
- High utility = Must-have (include in base tier)
- Medium utility = Differentiator (use for tier separation)
- Low utility = Nice-to-have (premium tier or cut)
### Using MaxDiff for Packaging
| Utility Score | Packaging Decision |
|---------------|-------------------|
| Top 20% | Include in all tiers (table stakes) |
| 20-50% | Use to differentiate tiers |
| 50-80% | Higher tiers only |
| Bottom 20% | Consider cutting or premium add-on |
---
## Willingness to Pay Surveys
**Direct method (simple but biased):**
"How much would you pay for [product]?"
**Better: Gabor-Granger method:**
"Would you buy [product] at [$X]?" (Yes/No)
Vary price across respondents to build demand curve.
**Even better: Conjoint analysis:**
Show product bundles at different prices
Respondents choose preferred option
Statistical analysis reveals price sensitivity per feature
---
## Usage-Value Correlation Analysis
### 1. Instrument usage data
Track how customers use your product:
- Feature usage frequency
- Volume metrics (users, records, API calls)
- Outcome metrics (revenue generated, time saved)
### 2. Correlate with customer success
- Which usage patterns predict retention?
- Which usage patterns predict expansion?
- Which customers pay the most, and why?
### 3. Identify value thresholds
- At what usage level do customers "get it"?
- At what usage level do they expand?
- At what usage level should price increase?
### Example Analysis
```
Usage-Value Correlation Analysis:
─────────────────────────────────
Segment: High-LTV customers (>$10k ARR)
Average monthly active users: 15
Average projects: 8
Average integrations: 4
Segment: Churned customers
Average monthly active users: 3
Average projects: 2
Average integrations: 0
Insight: Value correlates with team adoption (users)
and depth of use (integrations)
Recommendation: Price per user, gate integrations to higher tiers
```
FILE:references/tier-structure.md
# Tier Structure and Packaging
## Contents
- How Many Tiers?
- Good-Better-Best Framework
- Tier Differentiation Strategies
- Example Tier Structure
- Packaging for Personas (Identifying Pricing Personas, Persona-Based Packaging)
- Freemium vs. Free Trial (When to Use Freemium, When to Use Free Trial, Hybrid Approaches)
- Enterprise Pricing (When to Add Custom Pricing, Enterprise Tier Elements, Enterprise Pricing Strategies)
## How Many Tiers?
**2 tiers:** Simple, clear choice
- Works for: Clear SMB vs. Enterprise split
- Risk: May leave money on table
**3 tiers:** Industry standard
- Good tier = Entry point
- Better tier = Recommended (anchor to best)
- Best tier = High-value customers
**4+ tiers:** More granularity
- Works for: Wide range of customer sizes
- Risk: Decision paralysis, complexity
---
## Good-Better-Best Framework
**Good tier (Entry):**
- Purpose: Remove barriers to entry
- Includes: Core features, limited usage
- Price: Low, accessible
- Target: Small teams, try before you buy
**Better tier (Recommended):**
- Purpose: Where most customers land
- Includes: Full features, reasonable limits
- Price: Your "anchor" price
- Target: Growing teams, serious users
**Best tier (Premium):**
- Purpose: Capture high-value customers
- Includes: Everything, advanced features, higher limits
- Price: Premium (often 2-3x "Better")
- Target: Larger teams, power users, enterprises
---
## Tier Differentiation Strategies
**Feature gating:**
- Basic features in all tiers
- Advanced features in higher tiers
- Works when features have clear value differences
**Usage limits:**
- Same features, different limits
- More users, storage, API calls at higher tiers
- Works when value scales with usage
**Support level:**
- Email support → Priority support → Dedicated success
- Works for products with implementation complexity
**Access and customization:**
- API access, SSO, custom branding
- Works for enterprise differentiation
---
## Example Tier Structure
```
┌────────────────┬─────────────────┬─────────────────┬─────────────────┐
│ │ Starter │ Pro │ Business │
│ │ $29/mo │ $79/mo │ $199/mo │
├────────────────┼─────────────────┼─────────────────┼─────────────────┤
│ Users │ Up to 5 │ Up to 20 │ Unlimited │
│ Projects │ 10 │ Unlimited │ Unlimited │
│ Storage │ 5 GB │ 50 GB │ 500 GB │
│ Integrations │ 3 │ 10 │ Unlimited │
│ Analytics │ Basic │ Advanced │ Custom │
│ Support │ Email │ Priority │ Dedicated │
│ API Access │ ✗ │ ✓ │ ✓ │
│ SSO │ ✗ │ ✗ │ ✓ │
│ Audit logs │ ✗ │ ✗ │ ✓ │
└────────────────┴─────────────────┴─────────────────┴─────────────────┘
```
---
## Packaging for Personas
### Identifying Pricing Personas
Different customers have different:
- Willingness to pay
- Feature needs
- Buying processes
- Value perception
**Segment by:**
- Company size (solopreneur → SMB → enterprise)
- Use case (marketing vs. sales vs. support)
- Sophistication (beginner → power user)
- Industry (different budget norms)
### Persona-Based Packaging
**Step 1: Define personas**
| Persona | Size | Needs | WTP | Example |
|---------|------|-------|-----|---------|
| Freelancer | 1 person | Basic features | Low | $19/mo |
| Small Team | 2-10 | Collaboration | Medium | $49/mo |
| Growing Co | 10-50 | Scale, integrations | Higher | $149/mo |
| Enterprise | 50+ | Security, support | High | Custom |
**Step 2: Map features to personas**
| Feature | Freelancer | Small Team | Growing | Enterprise |
|---------|------------|------------|---------|------------|
| Core features | ✓ | ✓ | ✓ | ✓ |
| Collaboration | — | ✓ | ✓ | ✓ |
| Integrations | — | Limited | Full | Full |
| API access | — | — | ✓ | ✓ |
| SSO/SAML | — | — | — | ✓ |
| Audit logs | — | — | — | ✓ |
| Custom contract | — | — | — | ✓ |
**Step 3: Price to value for each persona**
- Research willingness to pay per segment
- Set prices that capture value without blocking adoption
- Consider segment-specific landing pages
---
## Freemium vs. Free Trial
### When to Use Freemium
**Freemium works when:**
- Product has viral/network effects
- Free users provide value (content, data, referrals)
- Large market where % conversion drives volume
- Low marginal cost to serve free users
- Clear feature/usage limits for upgrade trigger
**Freemium risks:**
- Free users may never convert
- Devalues product perception
- Support costs for non-paying users
- Harder to raise prices later
### When to Use Free Trial
**Free trial works when:**
- Product needs time to demonstrate value
- Onboarding/setup investment required
- B2B with buying committees
- Higher price points
- Product is "sticky" once configured
**Trial best practices:**
- 7-14 days for simple products
- 14-30 days for complex products
- Full access (not feature-limited)
- Clear countdown and reminders
- Credit card optional vs. required trade-off
**Credit card upfront:**
- Higher trial-to-paid conversion (40-50% vs. 15-25%)
- Lower trial volume
- Better qualified leads
### Hybrid Approaches
**Freemium + Trial:**
- Free tier with limited features
- Trial of premium features
- Example: Zoom (free 40-min, trial of Pro)
**Reverse trial:**
- Start with full access
- After trial, downgrade to free tier
- Example: See premium value, live with limitations until ready
---
## Enterprise Pricing
### When to Add Custom Pricing
Add "Contact Sales" when:
- Deal sizes exceed $10k+ ARR
- Customers need custom contracts
- Implementation/onboarding required
- Security/compliance requirements
- Procurement processes involved
### Enterprise Tier Elements
**Table stakes:**
- SSO/SAML
- Audit logs
- Admin controls
- Uptime SLA
- Security certifications
**Value-adds:**
- Dedicated support/success
- Custom onboarding
- Training sessions
- Custom integrations
- Priority roadmap input
### Enterprise Pricing Strategies
**Per-seat at scale:**
- Volume discounts for large teams
- Example: $15/user (standard) → $10/user (100+)
**Platform fee + usage:**
- Base fee for access
- Usage-based above thresholds
- Example: $500/mo base + $0.01 per API call
**Value-based contracts:**
- Price tied to customer's revenue/outcomes
- Example: % of transactions, revenue share
Viết email chào hàng lạnh B2B và chuỗi follow-up để tăng tỷ lệ phản hồi.
---
name: cold-email
description: Write B2B cold emails and follow-up sequences that get replies. Use when the user wants to write cold outreach emails, prospecting emails, cold email campaigns, sales development emails, or SDR emails. Also use when the user mentions "cold outreach," "prospecting email," "outbound email," "email to leads," "reach out to prospects," "sales email," "follow-up email sequence," "nobody's replying to my emails," or "how do I write a cold email." Covers subject lines, opening lines, body copy, CTAs, personalization, and multi-touch follow-up sequences. For warm/lifecycle email sequences, see emails. For sales collateral beyond emails, see sales-enablement.
metadata:
version: 2.0.0
---
# Cold Email Writing
You are an expert cold email writer. Your goal is to write emails that sound like they came from a sharp, thoughtful human — not a sales machine following a template.
## Before Writing
**Check for product marketing context first:**
If `.agents/product-marketing.md` exists (or `.claude/product-marketing.md`, or the legacy `product-marketing-context.md` filename, in older setups), read it before asking questions. Use that context and only ask for information not already covered or specific to this task.
Understand the situation (ask if not provided):
1. **Who are you writing to?** — Role, company, why them specifically
2. **What do you want?** — The outcome (meeting, reply, intro, demo)
3. **What's the value?** — The specific problem you solve for people like them
4. **What's your proof?** — A result, case study, or credibility signal
5. **Any research signals?** — Funding, hiring, LinkedIn posts, company news, tech stack changes
Work with whatever the user gives you. If they have a strong signal and a clear value prop, that's enough to write. Don't block on missing inputs — use what you have and note what would make it stronger.
---
## Writing Principles
### Write like a peer, not a vendor
The email should read like it came from someone who understands their world — not someone trying to sell them something. Use contractions. Read it aloud. If it sounds like marketing copy, rewrite it.
### Every sentence must earn its place
Cold email is ruthlessly short. If a sentence doesn't move the reader toward replying, cut it. The best cold emails feel like they could have been shorter, not longer.
### Personalization must connect to the problem
If you remove the personalized opening and the email still makes sense, the personalization isn't working. The observation should naturally lead into why you're reaching out.
See [personalization.md](references/personalization.md) for the 4-level system and research signals.
### Lead with their world, not yours
The reader should see their own situation reflected back. "You/your" should dominate over "I/we." Don't open with who you are or what your company does.
### One ask, low friction
Interest-based CTAs ("Worth exploring?" / "Would this be useful?") beat meeting requests. One CTA per email. Make it easy to say yes with a one-line reply.
---
## Voice & Tone
**The target voice:** A smart colleague who noticed something relevant and is sharing it. Conversational but not sloppy. Confident but not pushy.
**Calibrate to the audience:**
- C-suite: ultra-brief, peer-level, understated
- Mid-level: more specific value, slightly more detail
- Technical: precise, no fluff, respect their intelligence
**What it should NOT sound like:**
- A template with fields swapped in
- A pitch deck compressed into paragraph form
- A LinkedIn DM from someone you've never met
- An AI-generated email (avoid the telltale patterns: "I hope this email finds you well," "I came across your profile," "leverage," "synergy," "best-in-class")
---
## Structure
There's no single right structure. Choose a framework that fits the situation, or write freeform if the email flows naturally without one.
**Common shapes that work:**
- **Observation → Problem → Proof → Ask** — You noticed X, which usually means Y challenge. We helped Z with that. Interested?
- **Question → Value → Ask** — Struggling with X? We do Y. Company Z saw [result]. Worth a look?
- **Trigger → Insight → Ask** — Congrats on X. That usually creates Y challenge. We've helped similar companies with that. Curious?
- **Story → Bridge → Ask** — [Similar company] had [problem]. They [solved it this way]. Relevant to you?
For the full catalog of frameworks with examples, see [frameworks.md](references/frameworks.md).
---
## Subject Lines
Short, boring, internal-looking. The subject line's only job is to get the email opened — not to sell.
- 2-4 words, lowercase, no punctuation tricks
- Should look like it came from a colleague ("reply rates," "hiring ops," "Q2 forecast")
- No product pitches, no urgency, no emojis, no prospect's first name
See [subject-lines.md](references/subject-lines.md) for the full data.
---
## Follow-Up Sequences
Each follow-up should add something new — a different angle, fresh proof, a useful resource. "Just checking in" gives the reader no reason to respond.
- 3-5 total emails, increasing gaps between them
- Each email should stand alone (they may not have read the previous ones)
- The breakup email is your last touch — honor it
See [follow-up-sequences.md](references/follow-up-sequences.md) for cadence, angle rotation, and breakup email templates.
---
## Quality Check
Before presenting, gut-check:
- Does it sound like a human wrote it? (Read it aloud)
- Would YOU reply to this if you received it?
- Does every sentence serve the reader, not the sender?
- Is the personalization connected to the problem?
- Is there one clear, low-friction ask?
---
## What to Avoid
- Opening with "I hope this email finds you well" or "My name is X and I work at Y"
- Jargon: "synergy," "leverage," "circle back," "best-in-class," "leading provider"
- Feature dumps — one proof point beats ten features
- HTML, images, or multiple links
- Fake "Re:" or "Fwd:" subject lines
- Identical templates with only {{FirstName}} swapped
- Asking for 30-minute calls in first touch
- "Just checking in" follow-ups
---
## Data & Benchmarks
The references contain performance data if you need to make informed choices:
- [benchmarks.md](references/benchmarks.md) — Reply rates, conversion funnels, expert methods, common mistakes
- [personalization.md](references/personalization.md) — 4-level personalization system, research signals
- [subject-lines.md](references/subject-lines.md) — Subject line data and optimization
- [follow-up-sequences.md](references/follow-up-sequences.md) — Cadence, angles, breakup emails
- [frameworks.md](references/frameworks.md) — All copywriting frameworks with examples
Use this data to inform your writing — not as a checklist to satisfy.
---
## Related Skills
- **prospecting**: For building and qualifying the prospect list that this skill writes outreach against — the natural upstream step before cold-email
- **copywriting**: For landing pages and web copy
- **emails**: For lifecycle/nurture email sequences (not cold outreach)
- **social**: For LinkedIn and social posts
- **product-marketing**: For establishing foundational positioning
- **revops**: For lead scoring, routing, and pipeline management
FILE:evals/evals.json
{
"skill_name": "cold-email",
"evals": [
{
"id": 1,
"prompt": "Write a cold email to VP of Marketing at mid-size B2B SaaS companies. We sell a content analytics platform that shows which blog posts actually drive pipeline. Our main proof point: customers see 3x increase in content-attributed revenue within 90 days.",
"expected_output": "Should check for product-marketing.md first. Should write like a peer, not a vendor. Should use one of the structure frameworks (observation→problem→proof→ask or similar). Subject line should be 2-4 words, lowercase, internal-looking. Every sentence should earn its place. Personalization should connect to the prospect's problem, not just their name. Should use the 3x revenue proof point as social proof, not a feature claim. CTA should be low-friction (not 'book a demo'). Should provide 2-3 variations. Should include a quality check against the guidelines.",
"assertions": [
"Checks for product-marketing.md",
"Writes like a peer, not a vendor",
"Uses a structure framework from the skill",
"Subject line is short, lowercase, internal-looking",
"Every sentence earns its place (concise)",
"Personalization connects to prospect's problem",
"Uses proof point as social proof",
"CTA is low-friction",
"Provides 2-3 variations"
],
"files": []
},
{
"id": 2,
"prompt": "Help me write a cold email to CTOs at enterprise companies. I sell cybersecurity training. My current email has a 2% open rate and 0% reply rate.",
"expected_output": "Should diagnose the current email's likely problems based on 2% open rate (subject line issue) and 0% reply rate (body/relevance issue). Should apply voice calibration for CTO audience (respect their time, technical credibility, executive-level language). Should provide a completely new email following structure frameworks. Subject line should be 2-4 words, look internal. Should adapt tone for enterprise CTOs — more formal than startup audience but still peer-like. Should provide the email plus analysis of why each element works.",
"assertions": [
"Diagnoses problems from the performance data",
"Identifies subject line as likely open rate issue",
"Applies voice calibration for CTO audience",
"Subject line is short, lowercase, internal-looking",
"Adapts tone for enterprise audience",
"Uses structure framework from the skill",
"Explains why each element works"
],
"files": []
},
{
"id": 3,
"prompt": "write me a follow-up sequence. prospect didn't reply to my first email about our HR software. how many should I send and how far apart?",
"expected_output": "Should trigger on casual phrasing. Should apply the follow-up sequence guidance: 3-5 follow-ups recommended. Each follow-up should add something new (new angle, new proof point, new value) — not just 'bumping' or 'checking in.' Should provide timing recommendations between emails. Should provide actual follow-up email copy for each touch, with different angles. Should include a breakup email at the end. Should note that each follow-up should be shorter than the previous.",
"assertions": [
"Triggers on casual phrasing",
"Recommends 3-5 follow-up emails",
"Each follow-up adds something new",
"Does not use 'just bumping' or 'checking in' language",
"Provides timing between emails",
"Provides actual copy for each follow-up",
"Includes a breakup email",
"Follow-ups get progressively shorter"
],
"files": []
},
{
"id": 4,
"prompt": "Review this cold email and tell me what's wrong: 'Dear Sir/Madam, I hope this email finds you well. I wanted to reach out to introduce our innovative cloud-based platform that leverages AI to streamline your business operations. We have helped over 500 companies transform their workflows. I would love to schedule a 30-minute call to discuss how we can help your organization. Best regards, John'",
"expected_output": "Should apply the quality check framework. Should identify multiple problems: 'Dear Sir/Madam' (no personalization), 'I hope this email finds you well' (filler), 'innovative cloud-based platform' (jargon/buzzwords), 'leverages AI to streamline' (vague vendor language), 'transform their workflows' (means nothing), '30-minute call' (too much ask for cold email), entire email is about the sender not the prospect. Should rewrite following the principles: peer tone, observation→problem→proof→ask structure, every sentence earns its place, personalization connected to their problem, low-friction CTA.",
"assertions": [
"Identifies lack of personalization",
"Identifies filler phrases",
"Identifies jargon and buzzwords",
"Identifies vendor language vs peer language",
"Identifies CTA as too high-friction",
"Notes email is sender-focused not prospect-focused",
"Provides a rewritten version",
"Rewrite follows cold email principles"
],
"files": []
},
{
"id": 5,
"prompt": "What are the best subject lines for cold emails? I want to maximize open rates.",
"expected_output": "Should apply the subject line guidelines: short (2-4 words), lowercase or sentence case, internal-looking (should look like it came from a colleague, not a vendor). Should provide examples following these principles. Should explain why these work (bypass promotional filters, trigger curiosity, don't look like marketing). Should warn against common bad subject lines (ALL CAPS, emojis, clickbait, long subjects). Should note that subject line gets them to open but body gets them to reply.",
"assertions": [
"Applies subject line guidelines (2-4 words, lowercase, internal-looking)",
"Provides specific examples",
"Explains why the format works",
"Warns against common bad subject line patterns",
"Notes distinction between open rate and reply rate"
],
"files": []
},
{
"id": 6,
"prompt": "Can you help me set up an automated email drip campaign for leads who download our whitepaper?",
"expected_output": "Should recognize this is a lifecycle/nurture email sequence, not cold outreach. Should defer to or cross-reference the emails skill, which handles drip campaigns, lead nurture sequences, and lifecycle emails. Cold email is specifically for unsolicited outbound outreach to prospects who haven't opted in. Should make this distinction clear.",
"assertions": [
"Recognizes this as lifecycle/nurture email, not cold outreach",
"References or defers to emails skill",
"Explains the distinction between cold email and lifecycle email",
"Does not attempt to design a nurture sequence using cold email patterns"
],
"files": []
}
]
}
FILE:references/benchmarks.md
# Benchmarks, Data & Expert Methods
## Core Performance Metrics (2024–2025)
| Metric | Average | Good | Excellent | Source |
| -------------------------- | ------- | ------ | --------- | ------------------------ |
| Open rate | 27.7% | 40–45% | 50%+ | Belkins, Snov.io |
| Reply rate | 4–5.8% | 5–10% | 10–15% | Belkins, Reachoutly |
| Reply rate (best-in-class) | — | — | 15–25%+ | Digital Bloom, Instantly |
| Positive reply % | ~48% | 55–60% | 62–65% | Digital Bloom |
| Meeting booking rate | 0.5–1% | 1–2% | 2.3%+ | Reachoutly |
| Bounce rate | 7.5% | <4% | <2% | Belkins |
## Realistic Funnel Model
500 emails → 100 opens (20%) → 25 replies (5%) → 8 positive replies (30%) → 4 meetings (50%) → 1 client (25% close). ~**0.2% end-to-end conversion** for average performers.
## Performance Levers (ranked by impact)
1. **Hook type** — Timeline hooks outperform problem hooks by 3.4x in meetings
2. **Personalization depth** — Up to 250% more replies
3. **Brevity** — 25–75 words optimal, 83% more replies under 75 words
4. **Targeting precision** — ≤50 contacts per campaign = 2.76x higher reply rates
5. **Follow-up strategy** — First follow-up adds 49% more replies
6. **Reading level** — 3rd–5th grade = 67% more replies
7. **Send timing** — Thursday peaks at 6.87% reply rate
## Declining Effectiveness Trend
Reply rates dropped from 7–8% (2020–2022) to 4–5.8% (2024–2025), ~15% YoY decline. Drivers: inbox saturation (10+ cold emails/week, 20% say none relevant), stricter anti-spam (Google's threshold: 0.1% complaints), AI email flood (more volume, less quality signal). Writing craft matters more, not less — gap between average and excellent is widening.
## Response Rates by Seniority
- **Entry-level:** Highest engagement at 8% reply, 50% open
- **C-level:** 23% more likely to respond than non-C-suite when they engage (6.4% vs 5.2%)
- **CTOs/VP Tech:** 7.68% reply
- **CEOs/Founders:** 7.63% reply
- **Heads of Sales:** 6.60% (most targeted role, highest saturation)
## Industry Variation
**Highest responding:** Nonprofits (16.5%+), legal (10%), EdTech (7.8%), chemical (7.3%), manufacturing (6.1%).
**Lowest responding:** SaaS (3.5%), financial services (3.4%), IT services (3.5%).
## Top 15 Mistakes (ranked by impact)
1. **Too long** — 70% of emails above 10th-grade level. Under 75 words = 83% more replies
2. **Too self-focused** — "We are a leading..." signals sales pitch. Count I/We sentences
3. **No clear value prop** — 71% of decision-makers ignore irrelevant emails
4. **Generic templates** — {{FirstName}} isn't personalization. Recipients detect instantly
5. **Feature dumping** — "Great reps lead with problems" (Lavender). One proof point beats ten features
6. **False personalization** — "Loved your post!" without specifics is transparent
7. **Asking too much too soon** — 30-min call in first email = "proposing on first date"
8. **Pushy language** — "Act Now" stacking increases spam flagging by 67%
9. **No CTA** — Without a clear next step, momentum dies
10. **"Just checking in" follow-ups** — "I never heard back" = 12% drop in bookings
11. **Wrong tone for audience** — Founder ≠ RevOps lead ≠ sales leader
12. **Jargon/buzzwords** — "Leverage synergistic platform" → "We help you book more meetings"
13. **Unsubstantiated claims** — "300% more leads" without proof triggers skepticism
14. **Too many contacts per company** — 1–2 people = 7.8% reply; 10+ = 3.8%
15. **Fake urgency** — Fake "Re:" / "Fwd:" / countdown timers destroy trust
## Cultural Calibration
| Factor | US | UK | Germany/DACH | Scandinavia |
| ------------ | --------------- | ------------------------ | -------------------- | ----------------------- |
| Tone | Direct, casual | Polite, professional | Precise, data-driven | Fact-based, egalitarian |
| Length | Shorter, blunt | Longer, insight-led | Detail-oriented | Concise but substantive |
| Social proof | Outcome numbers | Research-led credibility | Technical precision | Shared values |
North America: 4.1% response. Europe: 3.1%. Asia-Pacific: 2.8%. Shorter, more direct sequences work better in US. UK needs more insight/personality. GDPR affects European tone.
## Expert Quick Reference
| Expert | Core Method | Best For |
| -------------- | --------------------------------------------------------------- | ----------------------------------------------- |
| Alex Berman | 3C's: Compliment → Case Study → CTA | High-ticket B2B services, agencies |
| Josh Braun | "Poke the Bear" — neutral questions exposing invisible problems | Empathy-driven consultative selling |
| Kyle Coleman | Systematic research + AI personalization at scale | Bridging mass outreach and deep personalization |
| Becc Holland | Psychographic personalization, Premise Buckets | Combining personalization with relevance |
| Will Allred | Data-driven coaching, Mouse Trap, Vanilla Ice Cream | Any context; universal frameworks |
| Justin Michael | 1–3 sentence hyper-brevity, quote their own words | High-velocity SDR teams at scale |
| Sam Nelson | Agoge Sequence — Triple on Day 1 (email + LinkedIn + call) | Multi-channel, tiered personalization |
FILE:references/follow-up-sequences.md
# Follow-Up Sequences
55% of replies come from follow-ups, not the initial email. Yet 48% of salespeople never follow up even once.
## How Many: 3–5 Total Emails
- Highest single-email reply rate: **8.4%** (Belkins).
- 4–7 email campaigns achieve **27% reply rates** vs 9% for 1–3 emails (Woodpecker, 20M emails).
- By 4th follow-up, response rates drop **55%** and spam complaints **triple**.
- Resolution: longer sequences catch different timing windows. Cap at 4 follow-ups (5 total emails). Each must add genuinely new value.
## Optimal Cadence
Increase the gap between each touch:
| Touch | Day | Notes |
| ------------- | ----- | ---------------------------------------------- |
| Initial email | 0 | Maximum personalization investment |
| Follow-up 1 | 3 | Waiting 3 days increases response by up to 31% |
| Follow-up 2 | 7–8 | Different angle |
| Follow-up 3 | 14 | New value piece |
| Follow-up 4 | 21–28 | Breakup email |
**Best days:** Tuesday–Thursday (Thursday peaks at 6.87% reply rate).
**Best times:** 9–11 AM or 1–3 PM in prospect's local time.
**Avoid:** Monday mornings (inbox overload), Friday afternoons (checked out).
## Angle Rotation
Each follow-up must stand alone while building toward the goal. Never just "bump this up."
| Email | Angle | Purpose |
| ----------- | ---------------------------------------------------------- | -------------------------- |
| Initial | Personalized hook + core value prop + soft CTA | Introduce problem/solution |
| Follow-up 1 | Different angle, new value piece (stat, insight, resource) | Show additional benefit |
| Follow-up 2 | Social proof / case study from similar company | Build credibility |
| Follow-up 3 | New insight, industry trend, or relevant resource | Demonstrate expertise |
| Follow-up 4 | Breakup — acknowledge silence, leave door open | Trigger loss aversion |
Add only **one new value proposition per email** (SalesBread). This naturally forces different angles.
## The Breakup Email
Leverages loss aversion — removing pressure while creating scarcity through withdrawal. Close.com reports **10–15% response rates** from breakup emails with cold prospects.
**Structure:**
1. Acknowledge you've reached out multiple times
2. Validate their potential lack of interest
3. State this is your final email for now
4. Leave the door open
**Example:**
> I haven't heard back, so I'll assume now isn't the right time. Before I close the loop: [1-sentence insight or resource]. If that changes things, feel free to reply. Otherwise, no hard feelings — good luck with [their goal].
**1-2-3 Format** (reduces friction to near zero):
> Since I haven't heard back, I'll keep it simple. Reply with a number:
>
> 1 — Interested, let's talk
> 2 — Not now, check back in 3 months
> 3 — Not interested, please stop
**Critical rule:** If you send a breakup email, honor it. Do not contact the prospect again.
## Phrases That Kill Response Rates
- "I never heard back" → **12% drop** in meeting booking rate (Gong)
- "Just checking in" → Zero value, signals laziness
- "Bumping this to the top of your inbox" → Presumptuous
- "Did you see my last email?" → Guilt-tripping
- "Following up on my previous message" → Generic, adds nothing
## CTA Adjustment by Seniority
**Executives/founders:** Ultra-low-effort, curiosity-driven. "Curious?" or "Worth 2 min?"
**Mid-level managers:** More specific value. "Want me to walk through how [Company] saved 15 hours/week?"
Higher in the org chart = less friction you can ask for.
FILE:references/frameworks.md
# Cold Email Copywriting Frameworks
Frameworks beat templates — they teach thinking patterns, not copy-paste shortcuts.
## PAS — Problem, Agitate, Solution (default)
**Structure:** Identify pain → Amplify consequences → Present solution + soft CTA.
**Best for:** Problem-aware but not solution-aware prospects. The workhorse framework.
> Most VP Sales at companies your size spend 5+ hours/week on manual CRM reporting. That's 250+ hours/year not spent coaching reps — and often means inaccurate forecasts reaching leadership. We built a tool that auto-generates CRM reports in real time. Teams like Datadog reduced reporting time by 80%. Would it make sense to see how?
## BAB — Before, After, Bridge
**Structure:** Current painful situation → Ideal future → Your product as the bridge.
**Best for:** Transformation-driven offers with clear before/after. Emotional decision-makers.
> Right now, your team is likely spending hours manually sourcing leads — feast or famine each quarter. Imagine qualified leads arriving daily on autopilot, reps spending 100% of their time selling. That's what our platform does. Companies like HubSpot saw a 40% pipeline increase within 90 days. Can I show you how?
## QVC — Question, Value, CTA
**Structure:** Targeted pain question → Brief value → Direct next step.
**Best for:** C-suite prospects who prefer brevity. Qualify interest immediately.
> Are your SDRs spending more time researching than selling? We help sales teams automate prospect research so reps focus on conversations. Clients see 3x more meetings per rep per week. Worth a 10-minute demo?
## AIDA — Attention, Interest, Desire, Action
**Structure:** Hook/stat → Address specific challenge → Social proof/outcome → Clear CTA.
**Best for:** Data-driven prospects, high-ticket pitches with strong stats.
> Companies in pharma lose 30% of leads due to manual outreach. Given {{Company}}'s growth this quarter, pipeline velocity is likely top of mind. Customers like Pfizer use our platform to automate lead qualification — cutting time-to-contact by 60%. Worth a 15-minute call?
## PPP — Praise, Picture, Push
**Structure:** Genuine compliment → How things could be better → Gentle push to action.
**Best for:** Senior prospects who respond to relationship-building. Requires genuine trigger.
> Your keynote on scaling SDR teams was spot-on — especially on ramp time as the hidden cost. What if you could cut that in half? Our in-inbox coach helps new reps write effective emails from day one with real-time scoring. Open to a quick chat about how this could support your growth?
## Star-Story-Solution
**Structure:** Introduce character (customer) → Tell challenge narrative → Reveal results.
**Best for:** Strong customer success stories. Humanizes the pitch.
> Last year, Sarah — VP Sales at a Series B startup — had 5 SDRs competing against a rival with 20. Her team was getting crushed on volume. They adopted our AI prospecting tool and sent hyper-personalized emails at 3x pace without losing quality. Within 90 days, they booked more meetings than their competitor's entire team. Happy to share how this could work for {{Company}}.
## SCQ — Situation, Complication, Question
**Structure:** Current reality → Complicating challenge → Question that speaks to need → Optional answer.
**Best for:** Consultative selling. Mirrors how professionals present to leadership.
> Your team doubled this year. That usually means onboarding is eating into selling time. How are you handling ramp for new hires?
## ACCA — Awareness, Comprehension, Conviction, Action
**Structure:** Contrarian hook → Explain benefit simply → Provide proof → Strong CTA.
**Best for:** Analytical buyers who need evidence (engineers, CFOs, ops leaders).
> Most sales teams measure rep activity. The top 5% measure rep efficiency instead. When Acme switched, they booked 40% more meetings with fewer emails. Worth seeing how?
## 3C's (Alex Berman)
**Structure:** Compliment → Case Study → CTA.
**Best for:** Agency/services cold outreach. Case study does the heavy lifting.
> Big fan of [Company]. We just built an app for [Competitor] that does XYZ. I have a few more ideas. Interested?
## Mouse Trap (Lavender/Will Allred)
**Structure:** Observation + Binary value-prop question. 1–2 sentences total.
**Best for:** Maximum brevity. Impulsive reply based on curiosity.
> Looks like you're hiring reps. Would it be helpful to get a more granular look at how they're ramping on email?
## Justin Michael Method
**Structure:** Trigger/Pain → Solution hint → Binary CTA. 1–3 sentences, no intro.
**Best for:** High-velocity SDR teams. Mobile-optimized. Deliberately polarizing.
Spend max 1 minute on personalization. Use industry/persona-level signals. For top-tier prospects, quote their own words from interviews — they almost always respond.
## Vanilla Ice Cream (Lavender)
**Structure:** Observation → Problem/Insight → Credibility → Solution → Call-to-Conversation.
**Best for:** Universal "base" framework that works everywhere. Five parts.
## PASTOR (Ray Edwards)
**Structure:** Problem → Amplify → Story → Testimony → Offer → Response.
**Best for:** Longer-form or multi-email sequences. Consulting, education, complex B2B services. Each element can be developed across separate touches.
FILE:references/personalization.md
# Personalization at Scale
Personalization drives **50–250% more replies** (Lavender). The key insight: **if your personalization has nothing to do with the problem you solve, it's just an attention hack** (Clay).
## Four Levels of Personalization
### Level 1 — Basic (merge tags)
First name, company name, job title. Table stakes, no longer differentiating. ~5% lift.
### Level 2 — Industry/segment
Industry-specific pain points, trends, regulatory challenges. Scalable via micro-segmentation.
> Most {{industry}} teams struggle with {{lead gen problem}}, which often leads to wasted effort.
### Level 3 — Role-level
Challenges specific to their role and seniority.
> As Head of Sales, keeping pipeline steady is probably your biggest headache. Your RevOps team is small, so you're likely wearing multiple hats during scaling.
### Level 4 — Individual (gold standard)
Specific, timely observations about that person connected to the problem you solve.
> Noticed you're hiring 3 SDRs — sounds like you're scaling outbound fast. Most teams hit follow-up fatigue during onboarding.
## Research Signal Stack
| Signal | Where to find it | How to use it |
| ----------------- | ---------------------------------- | ---------------------------------------------------------------------------- |
| Recent funding | Crunchbase, LinkedIn, press | "Congrats on Series B — scaling teams fast usually creates X challenge" |
| Job postings | LinkedIn Jobs, careers page | "Noticed you're hiring 3 SDRs — sounds like you're scaling outbound" |
| Tech stack | BuiltWith, Wappalyzer, HG Insights | "I see you're using HubSpot — most teams at your stage hit a ceiling with X" |
| LinkedIn activity | Posts, comments, job changes | "Really enjoyed your post about X" |
| Company news | Google News, press releases | "Congrats on acquiring X — integrating teams usually creates Y challenge" |
| Podcast/talks | Google, YouTube, podcasts | "Caught your talk at SaaStr on X — really insightful" |
| Website changes | Manual review | "Your new pricing page caught my eye — curious how it's converting" |
## The 3-Minute Personalization System
From "30 Minutes to President's Club":
**Step 1:** Build a research stack of top 10 buying signals — 5 company triggers, 5 person triggers. Stack-rank by relevance.
**Step 2:** Build a 3x3 template: (1) personalization attached to a problem, (2) problem you solve, (3) one-sentence solution + low-friction CTA.
**Step 3:** Create 5 "trigger templates" — pre-written personalization paragraphs for each trigger, with a smooth segue into the problem.
The personalization must logically connect to the problem. This creates 5 reusable triggers with the rest of the email constant. A top SDR writes a personalized email in **under 3 minutes**.
## The Four -Graphic Principles (Becc Holland)
- **Demographic** — Age, profession, background
- **Technographic** — Tech stack, tools used
- **Firmographic** — Company size, funding, industry, growth stage
- **Psychographic** — Values, passions, beliefs (highest-impact dimension)
Tapping into what prospects are passionate about drives significantly higher response rates.
## Observation-Based Openers (highest performing)
**Trigger-event:** "Congrats on the recent funding round — scaling the team from here is exciting, and I imagine [challenge] is top of mind."
**Observation:** "Your recent post about [topic] resonated — especially the part about [detail]. Got me thinking about how that applies to [challenge]."
**Industry insight:** "Most [role titles] I talk to spend [X hours/week] on [problem] — curious if that matches your experience at [Company]."
## What Feels Fake (avoid)
- AI-generated emails with similar phrasing ("I hope this email finds you well")
- Generic attention hacks disconnected from problem ("Cool that you went to UCLA!" → pitch)
- Over-personalizing to creepiness
- "I saw your LinkedIn profile and wanted to reach out" — signals mass automation
## The "So What?" Test
After writing any opening line, read from prospect's perspective: "So what? Why would I care?" If the answer is nothing, rewrite.
FILE:references/subject-lines.md
# Subject Line Optimization
The subject line determines whether the email gets read. The data is counterintuitive: **short, boring, internal-looking subject lines win decisively.**
## Length: 2–4 words
- 2-word subject lines get **60% more opens** than 5-word (Lavender).
- Going from 2 to 4 words reduces replies by **17.5%**.
- 2–4 words yield **46% open rates** vs 34% for 10 words (Belkins, 5.5M emails).
- Mobile truncates at 30–35 characters — brevity is practical necessity.
## Internal Camouflage Principle
Subject lines that look like they came from a colleague, not a vendor, double open rates (Gong). Buyers mentally categorize before opening — if it looks like sales, it's filtered.
**High-performing examples:** "reply rates" · "trial delays" · "hiring ops" · "employee turnover" · "Q2 forecast" · "new patients" · "personalization issue" · "second page"
## Capitalization: lowercase wins
All-lowercase has highest open rates (Gong, 85M+ emails). Lowercase looks more personal/internal. For cold outreach specifically, lowercase beats title case.
## Personalization: context over name
Personalized subject lines boost opens **26–50%**, but type matters:
- **First name in subject line → 12% fewer replies.** Signals automation.
- **Contextual personalization works:** pain points, competitors, trigger events, industry challenges.
- Use {{painPoint}}, {{competitor}}, {{commonGround}} — not {{firstName}}.
## Questions: only when highly specific
Data conflicts: Belkins says questions perform well (46% open rate). Lavender says questions lower opens by **56%**. Resolution: **specific pain questions work** ("Need help with {{challenge}}?"), **generic questions fail** ("Quick question?" / "Have 15 minutes?"). Default to statements.
## What to Avoid
| Anti-pattern | Impact |
| ---------------------------------------------- | --------------------------- |
| Salesy language ("increase," "boost," "ROI") | -17.9% opens |
| Urgency words ("ASAP," "urgent") | Below 36% opens |
| Excessive punctuation ("!!!" or "??") | -36% opens |
| Numbers and percentages | -46% opens |
| Emojis | Hurt B2B professionalism |
| Pitching product in subject | -57% replies |
| Empty/no subject line | +30% opens but -12% replies |
| Spam triggers ("free," "guarantee," "act now") | Deliverability risk |
## C-Suite Subject Lines
Executives receive 300–400 emails daily, decide in seconds. They respond **23% more often** than non-C-suite when emails pass their filter (6.4% reply rate).
What works: ultra-concise, human, understated. "{{companyInitiative}}" · "thank you" · "an update" · "a question" · reference to a specific project or trigger event.
Anything "salesy" is immediately rejected.
Hỗ trợ chia nhỏ epic, lập kế hoạch sprint, tinh chỉnh backlog và viết user story theo chuẩn INVEST.
---
name: cs-agile-product-owner
description: Agile product owner agent for epic breakdown, sprint planning, backlog refinement, and INVEST-compliant user story generation
skills: product-team/agile-product-owner, product-team/product-manager-toolkit
domain: product
model: sonnet
tools: [Read, Write, Bash, Grep, Glob]
---
# Agile Product Owner Agent
## Purpose
The cs-agile-product-owner agent is a specialized agile product ownership agent focused on backlog management, sprint planning, user story creation, and epic decomposition. This agent orchestrates the agile-product-owner skill alongside the product-manager-toolkit to ensure product backlogs are well-structured, properly prioritized, and aligned with business objectives.
This agent is designed for product owners, scrum masters wearing the PO hat, and agile team leads who need structured processes for breaking down epics into deliverable user stories, running effective sprint planning sessions, and maintaining a healthy product backlog. By combining Python-based story generation with RICE prioritization, the agent ensures backlogs are both strategically sound and execution-ready.
The cs-agile-product-owner agent bridges strategic product goals with sprint-level execution, providing frameworks for translating roadmap items into well-defined, INVEST-compliant user stories with clear acceptance criteria. It works best in tandem with scrum masters who provide velocity context and engineering teams who validate technical feasibility.
## Skill Integration
**Primary Skill:** `../../product-team/agile-product-owner/`
### All Orchestrated Skills
| # | Skill | Location | Primary Tool |
|---|-------|----------|-------------|
| 1 | Agile Product Owner | `../../product-team/agile-product-owner/` | user_story_generator.py |
| 2 | Product Manager Toolkit | `../../product-team/product-manager-toolkit/` | rice_prioritizer.py |
### Python Tools
1. **User Story Generator**
- **Purpose:** Break epics into INVEST-compliant user stories with acceptance criteria in Given/When/Then format
- **Path:** `../../product-team/agile-product-owner/scripts/user_story_generator.py`
- **Usage:** `python ../../product-team/agile-product-owner/scripts/user_story_generator.py epic.yaml`
- **Features:** Epic decomposition, acceptance criteria generation, story point estimation, dependency mapping
- **Use Cases:** Sprint planning, backlog refinement, story writing workshops
2. **RICE Prioritizer**
- **Purpose:** RICE framework for backlog prioritization with portfolio analysis
- **Path:** `../../product-team/product-manager-toolkit/scripts/rice_prioritizer.py`
- **Usage:** `python ../../product-team/product-manager-toolkit/scripts/rice_prioritizer.py backlog.csv --capacity 20`
- **Features:** Portfolio quadrant analysis, capacity planning, quarterly roadmap generation
- **Use Cases:** Backlog ordering, sprint scope decisions, stakeholder alignment
### Knowledge Bases
1. **Sprint Planning Guide**
- **Location:** `../../product-team/agile-product-owner/references/sprint-planning-guide.md`
- **Content:** Sprint planning ceremonies, velocity tracking, capacity allocation, sprint goal setting
- **Use Case:** Sprint planning facilitation, capacity management
2. **User Story Templates**
- **Location:** `../../product-team/agile-product-owner/references/user-story-templates.md`
- **Content:** INVEST-compliant story formats, acceptance criteria patterns, story splitting techniques
- **Use Case:** Story writing, backlog grooming, definition of done
3. **PRD Templates**
- **Location:** `../../product-team/product-manager-toolkit/references/prd_templates.md`
- **Content:** Product requirements document formats for different complexity levels
- **Use Case:** Epic documentation, feature specification
### Templates
1. **Sprint Planning Template**
- **Location:** `../../product-team/agile-product-owner/assets/sprint_planning_template.md`
- **Use Case:** Sprint planning sessions, capacity tracking, sprint goal documentation
2. **User Story Template**
- **Location:** `../../product-team/agile-product-owner/assets/user_story_template.md`
- **Use Case:** Consistent story format, acceptance criteria structure
3. **RICE Input Template**
- **Location:** `../../product-team/product-manager-toolkit/assets/rice_input_template.csv`
- **Use Case:** Structuring backlog items for RICE prioritization
## Workflows
### Workflow 1: Epic Breakdown
**Goal:** Decompose a large epic into sprint-ready user stories with acceptance criteria
**Steps:**
1. **Define the Epic** - Document the epic with clear scope:
- Business objective and user value
- Target user persona(s)
- High-level acceptance criteria
- Known constraints and dependencies
2. **Create Epic YAML** - Structure the epic for the story generator:
```yaml
epic:
title: "User Dashboard"
description: "Comprehensive dashboard for user activity and metrics"
personas: ["admin", "standard-user"]
features:
- "Activity feed"
- "Usage metrics"
- "Settings panel"
```
3. **Generate Stories** - Run the user story generator:
```bash
python ../../product-team/agile-product-owner/scripts/user_story_generator.py epic.yaml
```
4. **Review and Refine** - For each generated story:
- Validate INVEST compliance (Independent, Negotiable, Valuable, Estimable, Small, Testable)
- Refine acceptance criteria (Given/When/Then format)
- Identify dependencies between stories
- Estimate story points with the team
5. **Order the Backlog** - Sequence stories for delivery:
- Must-have stories first (MVP)
- Group by dependency chain
- Balance technical and user-facing work
**Expected Output:** 8-15 well-defined user stories per epic with acceptance criteria, story points, and dependency map
**Time Estimate:** 2-4 hours per epic
**Example:**
```bash
# Create epic definition
cat > dashboard-epic.yaml << 'EOF'
epic:
title: "User Dashboard"
description: "Real-time dashboard showing user activity, key metrics, and account settings"
personas: ["admin", "standard-user"]
features:
- "Real-time activity feed"
- "Key metrics display with charts"
- "Quick settings access"
- "Notification preferences"
EOF
# Generate user stories
python ../../product-team/agile-product-owner/scripts/user_story_generator.py dashboard-epic.yaml
# Review the sprint planning guide for context
cat ../../product-team/agile-product-owner/references/sprint-planning-guide.md
```
### Workflow 2: Sprint Planning
**Goal:** Plan a sprint with clear goals, selected stories, and identified risks
**Steps:**
1. **Calculate Capacity** - Determine team availability:
- List team members and available days
- Account for PTO, on-call, training, meetings
- Calculate total person-days
- Reference historical velocity (average of last 3 sprints)
2. **Review Backlog** - Ensure stories are ready:
- Check Definition of Ready for top candidates
- Verify acceptance criteria are complete
- Confirm technical feasibility with engineers
- Identify any blocking dependencies
3. **Set Sprint Goal** - Define one clear, measurable goal:
- Aligned with quarterly OKRs
- Achievable within sprint capacity
- Valuable to users or business
4. **Select Stories** - Pull from prioritized backlog:
```bash
# Prioritize candidates if not already ordered
python ../../product-team/product-manager-toolkit/scripts/rice_prioritizer.py sprint-candidates.csv --capacity 12
```
5. **Document the Plan** - Use the sprint planning template:
```bash
cat ../../product-team/agile-product-owner/assets/sprint_planning_template.md
```
6. **Identify Risks** - Document potential blockers:
- External dependencies
- Technical unknowns
- Team availability changes
- Mitigation plans for each risk
**Expected Output:** Sprint plan document with goal, selected stories (within velocity), capacity allocation, dependencies, and risks
**Time Estimate:** 2-3 hours per sprint planning session
**Example:**
```bash
# Prepare sprint candidates
cat > sprint-candidates.csv << 'EOF'
feature,reach,impact,confidence,effort
User Dashboard - Activity Feed,500,3,0.8,3
User Dashboard - Metrics Charts,500,2,0.9,5
Notification Preferences,300,1,1.0,2
Password Reset Flow Fix,1000,2,1.0,1
EOF
# Run prioritization
python ../../product-team/product-manager-toolkit/scripts/rice_prioritizer.py sprint-candidates.csv --capacity 8
# Reference sprint planning template
cat ../../product-team/agile-product-owner/assets/sprint_planning_template.md
```
### Workflow 3: Backlog Refinement
**Goal:** Maintain a healthy backlog with properly sized, prioritized, and well-defined stories
**Steps:**
1. **Triage New Items** - Process incoming requests:
- Customer feedback items
- Bug reports
- Technical debt tickets
- Feature requests from stakeholders
2. **Size and Estimate** - Apply story points:
- Use planning poker or T-shirt sizing
- Reference team estimation guidelines
- Split stories larger than 13 story points
- Apply story splitting techniques from references
3. **Prioritize with RICE** - Score backlog items:
```bash
python ../../product-team/product-manager-toolkit/scripts/rice_prioritizer.py backlog.csv
```
4. **Refine Top Items** - Ensure top 2 sprints worth are ready:
- Complete acceptance criteria
- Resolve open questions with stakeholders
- Add technical notes and implementation hints
- Verify designs are available (if applicable)
5. **Archive or Remove** - Clean the backlog:
- Close items older than 6 months without activity
- Merge duplicate stories
- Remove items no longer aligned with strategy
**Expected Output:** Refined backlog with top 20 stories fully defined, estimated, and ordered
**Time Estimate:** 1-2 hours per weekly refinement session
**Example:**
```bash
# Export backlog for prioritization
cat > backlog-q2.csv << 'EOF'
feature,reach,impact,confidence,effort
Search Improvement,800,3,0.8,5
Mobile Responsive Tables,600,2,0.7,3
API Rate Limiting,400,2,0.9,2
Onboarding Wizard,1000,3,0.6,8
Export to PDF,200,1,1.0,1
Dark Mode,300,1,0.8,3
EOF
# Run full prioritization with capacity
python ../../product-team/product-manager-toolkit/scripts/rice_prioritizer.py backlog-q2.csv --capacity 15
# Review user story templates for refinement
cat ../../product-team/agile-product-owner/references/user-story-templates.md
```
### Workflow 4: Story Writing Workshop
**Goal:** Collaboratively write high-quality user stories with the team
**Steps:**
1. **Prepare the Session** - Gather inputs:
- Epic or feature description
- User personas involved
- Design mockups or wireframes
- Technical constraints
2. **Identify User Personas** - Map stories to personas:
- Who are the primary users?
- What are their goals?
- What are their constraints?
3. **Write Stories Collaboratively** - Use the template:
```bash
cat ../../product-team/agile-product-owner/assets/user_story_template.md
```
- "As a [persona], I want [capability], so that [benefit]"
- Focus on user value, not implementation details
- One story per distinct user action or outcome
4. **Add Acceptance Criteria** - Define "done":
- Given/When/Then format for each scenario
- Cover happy path, edge cases, and error states
- Include performance and accessibility requirements
5. **Validate INVEST** - Check each story:
- **Independent**: Can be delivered without other stories
- **Negotiable**: Implementation details flexible
- **Valuable**: Delivers user or business value
- **Estimable**: Team can estimate effort
- **Small**: Fits within a single sprint
- **Testable**: Clear pass/fail criteria
6. **Estimate as a Team** - Story point consensus:
- Use planning poker or fist of five
- Discuss outlier estimates
- Re-split if estimate exceeds 13 points
**Expected Output:** Set of INVEST-compliant user stories with acceptance criteria and estimates
**Time Estimate:** 1-2 hours per workshop (covering 1 epic or feature area)
**Example:**
```bash
# Generate initial story candidates from epic
python ../../product-team/agile-product-owner/scripts/user_story_generator.py feature-epic.yaml
# Reference story templates for format guidance
cat ../../product-team/agile-product-owner/references/user-story-templates.md
# Reference sprint planning guide for estimation practices
cat ../../product-team/agile-product-owner/references/sprint-planning-guide.md
```
## Integration Examples
### Example 1: End-to-End Sprint Cycle
```bash
#!/bin/bash
# sprint-cycle.sh - Complete sprint planning automation
SPRINT_NUM=14
CAPACITY=12 # person-days equivalent in story points
echo "Sprint $SPRINT_NUM Planning"
echo "=========================="
# Step 1: Prioritize backlog
echo ""
echo "1. Backlog Prioritization:"
python ../../product-team/product-manager-toolkit/scripts/rice_prioritizer.py backlog.csv --capacity $CAPACITY
# Step 2: Generate stories for top epic
echo ""
echo "2. Story Generation for Top Epic:"
python ../../product-team/agile-product-owner/scripts/user_story_generator.py top-epic.yaml
# Step 3: Reference planning template
echo ""
echo "3. Sprint Planning Template:"
echo "See: ../../product-team/agile-product-owner/assets/sprint_planning_template.md"
```
### Example 2: Backlog Health Check
```bash
#!/bin/bash
# backlog-health.sh - Weekly backlog health assessment
echo "Backlog Health Check - $(date +%Y-%m-%d)"
echo "========================================"
# Count stories by status
echo ""
echo "Backlog Items:"
wc -l < backlog.csv
echo "items in backlog"
# Run prioritization
echo ""
echo "Current Priorities:"
python ../../product-team/product-manager-toolkit/scripts/rice_prioritizer.py backlog.csv --capacity 20
# Check story templates
echo ""
echo "Story Template Reference:"
echo "Location: ../../product-team/agile-product-owner/references/user-story-templates.md"
```
## Success Metrics
**Backlog Quality:**
- **Story Readiness:** >80% of sprint candidates meet Definition of Ready
- **Estimation Accuracy:** Actual effort within 20% of estimate (rolling average)
- **Story Size:** <5% of stories exceed 13 story points
- **Acceptance Criteria:** 100% of stories have testable acceptance criteria
**Sprint Execution:**
- **Sprint Goal Achievement:** >85% of sprints meet their stated goal
- **Velocity Stability:** Velocity variance <20% sprint-to-sprint
- **Scope Change:** <10% scope change after sprint planning
- **Completion Rate:** >90% of committed stories completed per sprint
**Stakeholder Value:**
- **Value Delivery:** Every sprint delivers demonstrable user value
- **Cycle Time:** Average story cycle time <5 days
- **Lead Time:** Epic to delivery <6 weeks average
- **Stakeholder Satisfaction:** >4/5 on sprint review feedback
## Related Agents
- [cs-product-manager](cs-product-manager.md) - Full product management lifecycle (RICE, interviews, PRDs)
- [cs-product-strategist](cs-product-strategist.md) - OKR cascade and strategic planning for roadmap alignment
- [cs-ux-researcher](cs-ux-researcher.md) - User research to inform story requirements and acceptance criteria
- Scrum Master - Velocity context and sprint execution (see `../../project-management/scrum-master/`)
## References
- **Primary Skill:** [../../product-team/agile-product-owner/SKILL.md](../../product-team/agile-product-owner/SKILL.md)
- **RICE Framework:** [../../product-team/product-manager-toolkit/SKILL.md](../../product-team/product-manager-toolkit/SKILL.md)
- **Product Domain Guide:** [../../product-team/CLAUDE.md](../../product-team/CLAUDE.md)
- **Agent Development Guide:** [../CLAUDE.md](../CLAUDE.md)
- **Scrum Master Skill:** [../../project-management/scrum-master/SKILL.md](../../project-management/scrum-master/SKILL.md)
---
**Last Updated:** March 9, 2026
**Status:** Production Ready
**Version:** 1.0
Bộ công cụ ASO: nghiên cứu từ khóa, phân tích đối thủ, gợi ý metadata và tăng khả năng hiển thị trên App Store, Google Play.
---
name: "app-store-optimization"
description: App Store Optimization (ASO) toolkit for researching keywords, analyzing competitor rankings, generating metadata suggestions, and improving app visibility on Apple App Store and Google Play Store. Use when the user asks about ASO, app store rankings, app metadata, app titles and descriptions, app store listings, app visibility, or mobile app marketing on iOS or Android. Supports keyword research and scoring, competitor keyword analysis, metadata optimization, A/B test planning, launch checklists, and tracking ranking changes.
triggers:
- ASO
- app store optimization
- app store ranking
- app keywords
- app metadata
- play store optimization
- app store listing
- improve app rankings
- app visibility
- app store SEO
- mobile app marketing
- app conversion rate
---
# App Store Optimization (ASO)
---
## Keyword Research Workflow
Discover and evaluate keywords that drive app store visibility.
### Workflow: Conduct Keyword Research
1. Define target audience and core app functions:
- Primary use case (what problem does the app solve)
- Target user demographics
- Competitive category
2. Generate seed keywords from:
- App features and benefits
- User language (not developer terminology)
- App store autocomplete suggestions
3. Expand keyword list using:
- Modifiers (free, best, simple)
- Actions (create, track, organize)
- Audiences (for students, for teams, for business)
4. Evaluate each keyword:
- Search volume (estimated monthly searches)
- Competition (number and quality of ranking apps)
- Relevance (alignment with app function)
5. Score and prioritize keywords:
- Primary: Title and keyword field (iOS)
- Secondary: Subtitle and short description
- Tertiary: Full description only
6. Map keywords to metadata locations
7. Document keyword strategy for tracking
8. **Validation:** Keywords scored; placement mapped; no competitor brand names included; no plurals in iOS keyword field
### Keyword Evaluation Criteria
| Factor | Weight | High Score Indicators |
|--------|--------|----------------------|
| Relevance | 35% | Describes core app function |
| Volume | 25% | 10,000+ monthly searches |
| Competition | 25% | Top 10 apps have <4.5 avg rating |
| Conversion | 15% | Transactional intent ("best X app") |
### Keyword Placement Priority
| Location | Search Weight |
|----------|---------------|
| App Title | Highest |
| Subtitle (iOS) | High |
| Keyword Field (iOS) | High |
| Short Description (Android) | High |
| Full Description | Medium |
See: [references/keyword-research-guide.md](references/keyword-research-guide.md)
---
## Metadata Optimization Workflow
Optimize app store listing elements for search ranking and conversion.
### Workflow: Optimize App Metadata
1. Audit current metadata against platform limits:
- Title character count and keyword presence
- Subtitle/short description usage
- Keyword field efficiency (iOS)
- Description keyword density
2. Optimize title following formula:
```
[Brand Name] - [Primary Keyword] [Secondary Keyword]
```
3. Write subtitle (iOS) or short description (Android):
- Focus on primary benefit
- Include secondary keyword
- Use action verbs
4. Optimize keyword field (iOS only):
- Remove duplicates from title
- Remove plurals (Apple indexes both forms)
- No spaces after commas
- Prioritize by score
5. Rewrite full description:
- Hook paragraph with value proposition
- Feature bullets with keywords
- Social proof section
- Call to action
6. Validate character counts for each field
7. Calculate keyword density (target 2-3% primary)
8. **Validation:** All fields within character limits; primary keyword in title; no keyword stuffing (>5%); natural language preserved
### Platform Character Limits
| Field | Apple App Store | Google Play Store |
|-------|-----------------|-------------------|
| Title | 30 characters | 50 characters |
| Subtitle | 30 characters | N/A |
| Short Description | N/A | 80 characters |
| Keywords | 100 characters | N/A |
| Promotional Text | 170 characters | N/A |
| Full Description | 4,000 characters | 4,000 characters |
| What's New | 4,000 characters | 500 characters |
### Description Structure
```
PARAGRAPH 1: Hook (50-100 words)
├── Address user pain point
├── State main value proposition
└── Include primary keyword
PARAGRAPH 2-3: Features (100-150 words)
├── Top 5 features with benefits
├── Bullet points for scanability
└── Secondary keywords naturally integrated
PARAGRAPH 4: Social Proof (50-75 words)
├── Download count or rating
├── Press mentions or awards
└── Summary of user testimonials
PARAGRAPH 5: Call to Action (25-50 words)
├── Clear next step
└── Reassurance (free trial, no signup)
```
See: [references/platform-requirements.md](references/platform-requirements.md)
---
## Competitor Analysis Workflow
Analyze top competitors to identify keyword gaps and positioning opportunities.
### Workflow: Analyze Competitor ASO Strategy
1. Identify top 10 competitors:
- Direct competitors (same core function)
- Indirect competitors (overlapping audience)
- Category leaders (top downloads)
2. Extract competitor keywords from:
- App titles and subtitles
- First 100 words of descriptions
- Visible metadata patterns
3. Build competitor keyword matrix:
- Map which keywords each competitor targets
- Calculate coverage percentage per keyword
4. Identify keyword gaps:
- Keywords with <40% competitor coverage
- High volume terms competitors miss
- Long-tail opportunities
5. Analyze competitor visual assets:
- Icon design patterns
- Screenshot messaging and style
- Video presence and quality
6. Compare ratings and review patterns:
- Average rating by competitor
- Common praise themes
- Common complaint themes
7. Document positioning opportunities
8. **Validation:** 10+ competitors analyzed; keyword matrix complete; gaps identified with volume estimates; visual audit documented
### Competitor Analysis Matrix
| Analysis Area | Data Points |
|---------------|-------------|
| Keywords | Title keywords, description frequency |
| Metadata | Character utilization, keyword density |
| Visuals | Icon style, screenshot count/style |
| Ratings | Average rating, total count, velocity |
| Reviews | Top praise, top complaints |
### Gap Analysis Template
| Opportunity Type | Example | Action |
|------------------|---------|--------|
| Keyword gap | "habit tracker" (40% coverage) | Add to keyword field |
| Feature gap | Competitor lacks widget | Highlight in screenshots |
| Visual gap | No videos in top 5 | Create app preview |
| Messaging gap | None mention "free" | Test free positioning |
---
## App Launch Workflow
Execute a structured launch for maximum initial visibility.
### Workflow: Launch App to Stores
1. Complete pre-launch preparation (4 weeks before):
- Finalize keywords and metadata
- Prepare all visual assets
- Set up analytics (Firebase, Mixpanel)
- Build press kit and media list
2. Submit for review (2 weeks before):
- Complete all store requirements
- Verify compliance with guidelines
- Prepare launch communications
3. Configure post-launch systems:
- Set up review monitoring
- Prepare response templates
- Configure rating prompt timing
4. Execute launch day:
- Verify app is live in both stores
- Announce across all channels
- Begin review response cycle
5. Monitor initial performance (days 1-7):
- Track download velocity hourly
- Monitor reviews and respond within 24 hours
- Document any issues for quick fixes
6. Conduct 7-day retrospective:
- Compare performance to projections
- Identify quick optimization wins
- Plan first metadata update
7. Schedule first update (2 weeks post-launch)
8. **Validation:** App live in stores; analytics tracking; review responses within 24h; download velocity documented; first update scheduled
### Pre-Launch Checklist
| Category | Items |
|----------|-------|
| Metadata | Title, subtitle, description, keywords |
| Visual Assets | Icon, screenshots (all sizes), video |
| Compliance | Age rating, privacy policy, content rights |
| Technical | App binary, signing certificates |
| Analytics | SDK integration, event tracking |
| Marketing | Press kit, social content, email ready |
### Launch Timing Considerations
| Factor | Recommendation |
|--------|----------------|
| Day of week | Tuesday-Wednesday (avoid weekends) |
| Time of day | Morning in target market timezone |
| Seasonal | Align with relevant category seasons |
| Competition | Avoid major competitor launch dates |
See: [references/aso-best-practices.md](references/aso-best-practices.md)
---
## A/B Testing Workflow
Test metadata and visual elements to improve conversion rates.
### Workflow: Run A/B Test
1. Select test element (prioritize by impact):
- Icon (highest impact)
- Screenshot 1 (high impact)
- Title (high impact)
- Short description (medium impact)
2. Form hypothesis:
```
If we [change], then [metric] will [improve/increase] by [amount]
because [rationale].
```
3. Create variants:
- Control: Current version
- Treatment: Single variable change
4. Calculate required sample size:
- Baseline conversion rate
- Minimum detectable effect (usually 5%)
- Statistical significance (95%)
5. Launch test:
- Apple: Use Product Page Optimization
- Android: Use Store Listing Experiments
6. Run test for minimum duration:
- At least 7 days
- Until statistical significance reached
7. Analyze results:
- Compare conversion rates
- Check statistical significance
- Document learnings
8. **Validation:** Single variable tested; sample size sufficient; significance reached (95%); results documented; winner implemented
### A/B Test Prioritization
| Element | Conversion Impact | Test Complexity |
|---------|-------------------|-----------------|
| App Icon | 10-25% lift possible | Medium (design needed) |
| Screenshot 1 | 15-35% lift possible | Medium |
| Title | 5-15% lift possible | Low |
| Short Description | 5-10% lift possible | Low |
| Video | 10-20% lift possible | High |
### Sample Size Quick Reference
| Baseline CVR | Impressions Needed (per variant) |
|--------------|----------------------------------|
| 1% | 31,000 |
| 2% | 15,500 |
| 5% | 6,200 |
| 10% | 3,100 |
### Test Documentation Template
```
TEST ID: ASO-2025-001
ELEMENT: App Icon
HYPOTHESIS: A bolder color icon will increase conversion by 10%
START DATE: [Date]
END DATE: [Date]
RESULTS:
├── Control CVR: 4.2%
├── Treatment CVR: 4.8%
├── Lift: +14.3%
├── Significance: 97%
└── Decision: Implement treatment
LEARNINGS:
- Bold colors outperform muted tones in this category
- Apply to screenshot backgrounds for next test
```
---
## Before/After Examples
### Title Optimization
**Productivity App:**
| Version | Title | Analysis |
|---------|-------|----------|
| Before | "MyTasks" | No keywords, brand only (8 chars) |
| After | "MyTasks - Todo List & Planner" | Primary + secondary keywords (29 chars) |
**Fitness App:**
| Version | Title | Analysis |
|---------|-------|----------|
| Before | "FitTrack Pro" | Generic modifier (12 chars) |
| After | "FitTrack: Workout Log & Gym" | Category keywords (27 chars) |
### Subtitle Optimization (iOS)
| Version | Subtitle | Analysis |
|---------|----------|----------|
| Before | "Get Things Done" | Vague, no keywords |
| After | "Daily Task Manager & Planner" | Two keywords, benefit clear |
### Keyword Field Optimization (iOS)
**Before (Inefficient - 89 chars, 8 keywords):**
```
task manager, todo list, productivity app, daily planner, reminder app
```
**After (Optimized - 97 chars, 14 keywords):**
```
task,todo,checklist,reminder,organize,daily,planner,schedule,deadline,goals,habit,widget,sync,team
```
**Improvements:**
- Removed spaces after commas (+8 chars)
- Removed duplicates (task manager → task)
- Removed plurals (reminders → reminder)
- Removed words in title
- Added more relevant keywords
### Description Opening
**Before:**
```
MyTasks is a comprehensive task management solution designed
to help busy professionals organize their daily activities
and boost productivity.
```
**After:**
```
Forget missed deadlines. MyTasks keeps every task, reminder,
and project in one place—so you focus on doing, not remembering.
Trusted by 500,000+ professionals.
```
**Improvements:**
- Leads with user pain point
- Specific benefit (not generic "boost productivity")
- Social proof included
- Keywords natural, not stuffed
### Screenshot Caption Evolution
| Version | Caption | Issue |
|---------|---------|-------|
| Before | "Task List Feature" | Feature-focused, passive |
| Better | "Create Task Lists" | Action verb, but still feature |
| Best | "Never Miss a Deadline" | Benefit-focused, emotional |
---
## Tools and References
### Scripts
| Script | Purpose | Usage |
|--------|---------|-------|
| [keyword_analyzer.py](scripts/keyword_analyzer.py) | Analyze keywords for volume and competition | `python keyword_analyzer.py --keywords "todo,task,planner"` |
| [metadata_optimizer.py](scripts/metadata_optimizer.py) | Validate metadata character limits and density | `python metadata_optimizer.py --platform ios --title "App Title"` |
| [competitor_analyzer.py](scripts/competitor_analyzer.py) | Extract and compare competitor keywords | `python competitor_analyzer.py --competitors "App1,App2,App3"` |
| [aso_scorer.py](scripts/aso_scorer.py) | Calculate overall ASO health score | `python aso_scorer.py --app-id com.example.app` |
| [ab_test_planner.py](scripts/ab_test_planner.py) | Plan tests and calculate sample sizes | `python ab_test_planner.py --cvr 0.05 --lift 0.10` |
| [review_analyzer.py](scripts/review_analyzer.py) | Analyze review sentiment and themes | `python review_analyzer.py --app-id com.example.app` |
| [launch_checklist.py](scripts/launch_checklist.py) | Generate platform-specific launch checklists | `python launch_checklist.py --platform ios` |
| [localization_helper.py](scripts/localization_helper.py) | Manage multi-language metadata | `python localization_helper.py --locales "en,es,de,ja"` |
### References
| Document | Content |
|----------|---------|
| [platform-requirements.md](references/platform-requirements.md) | iOS and Android metadata specs, visual asset requirements |
| [aso-best-practices.md](references/aso-best-practices.md) | Optimization strategies, rating management, launch tactics |
| [keyword-research-guide.md](references/keyword-research-guide.md) | Research methodology, evaluation framework, tracking |
### Assets
| Template | Purpose |
|----------|---------|
| [aso-audit-template.md](assets/aso-audit-template.md) | Structured audit checklist for app store listings |
---
## Platform Notes
| Platform / Constraint | Behavior / Impact |
|-----------------------|-------------------|
| iOS keyword changes | Require app submission |
| iOS promotional text | Editable without an app update |
| Android metadata changes | Index in 1-2 hours |
| Android keyword field | None — use description instead |
| Keyword volume data | Estimates only; no official source |
| Competitor data | Public listings only |
**When not to use this skill:** web apps (use web SEO), enterprise/internal apps, TestFlight-only betas, or paid advertising strategy.
---
## Related Skills
| Skill | Integration Point |
|-------|-------------------|
| [content-creator](../content-creator/) | App description copywriting |
| [marketing-demand-acquisition](../marketing-demand-acquisition/) | Launch promotion campaigns |
| [marketing-strategy-pmm](../marketing-strategy-pmm/) | Go-to-market planning |
## Proactive Triggers
- **No keyword optimization in title** → App title is the #1 ranking factor. Include top keyword.
- **Screenshots don't show value** → Screenshots should tell a story, not show UI.
- **No ratings strategy** → Below 4.0 stars kills conversion. Implement in-app rating prompts.
- **Description keyword-stuffed** → Natural language with keywords beats keyword stuffing.
## Output Artifacts
| When you ask for... | You get... |
|---------------------|------------|
| "ASO audit" | Full app store listing audit with prioritized fixes |
| "Keyword research" | Keyword list with search volume and difficulty scores |
| "Optimize my listing" | Rewritten title, subtitle, description, keyword field |
## Communication
All output passes quality verification:
- Self-verify: source attribution, assumption audit, confidence scoring
- Output format: Bottom Line → What (with confidence) → Why → How to Act
- Results only. Every finding tagged: 🟢 verified, 🟡 medium, 🔴 assumed.
FILE:assets/aso-audit-template.md
# ASO Audit Template
Use this template to conduct a systematic App Store Optimization audit.
---
## App Information
| Field | Value |
|-------|-------|
| App Name | |
| Platform | [ ] iOS [ ] Android |
| Category | |
| Current Downloads | |
| Current Rating | |
| Audit Date | |
---
## Metadata Audit
### Title Analysis
| Criterion | iOS (30 chars) | Android (50 chars) |
|-----------|----------------|---------------------|
| Current Title | | |
| Character Count | /30 | /50 |
| Primary Keyword Present | [ ] Yes [ ] No | [ ] Yes [ ] No |
| Brand Name Position | | |
**Title Score:** ___/10
**Recommendations:**
- [ ]
- [ ]
### Subtitle / Short Description
| Criterion | iOS Subtitle (30 chars) | Android Short Desc (80 chars) |
|-----------|-------------------------|-------------------------------|
| Current Text | | |
| Character Count | /30 | /80 |
| Keywords Included | | |
| Benefit-Focused | [ ] Yes [ ] No | [ ] Yes [ ] No |
**Score:** ___/10
**Recommendations:**
- [ ]
- [ ]
### Keyword Field (iOS Only)
| Criterion | Status |
|-----------|--------|
| Current Keywords | |
| Character Count | /100 |
| Duplicates Present | [ ] Yes [ ] No |
| Plurals Included | [ ] Yes [ ] No |
| Brand Names Included | [ ] Yes [ ] No |
**Score:** ___/10
**Recommendations:**
- [ ]
- [ ]
### Full Description
| Criterion | iOS | Android |
|-----------|-----|---------|
| Character Count | /4000 | /4000 |
| Primary Keyword Density | % | % |
| Secondary Keywords (count) | | |
| Feature Bullets Present | [ ] Yes [ ] No | [ ] Yes [ ] No |
| Social Proof Included | [ ] Yes [ ] No | [ ] Yes [ ] No |
| CTA Present | [ ] Yes [ ] No | [ ] Yes [ ] No |
**Score:** ___/10
**Recommendations:**
- [ ]
- [ ]
---
## Visual Asset Audit
### App Icon
| Criterion | Status |
|-----------|--------|
| Recognizable at 60x60px | [ ] Yes [ ] No |
| Distinct from competitors | [ ] Yes [ ] No |
| Matches app design | [ ] Yes [ ] No |
| No text/words | [ ] Yes [ ] No |
**Score:** ___/10
**Recommendations:**
- [ ]
- [ ]
### Screenshots
| Screenshot | Caption | Feature Shown | Score |
|------------|---------|---------------|-------|
| 1 (Hero) | | | /10 |
| 2 | | | /10 |
| 3 | | | /10 |
| 4 | | | /10 |
| 5 | | | /10 |
| Criterion | Status |
|-----------|--------|
| Total Screenshots | /10 (iOS) or /8 (Android) |
| Captions Present | [ ] Yes [ ] No |
| Consistent Style | [ ] Yes [ ] No |
| First 3 Show Value | [ ] Yes [ ] No |
| Device Frames Used | [ ] Yes [ ] No |
**Overall Screenshot Score:** ___/10
**Recommendations:**
- [ ]
- [ ]
### App Preview Video
| Criterion | Status |
|-----------|--------|
| Video Present | [ ] Yes [ ] No |
| Duration | seconds |
| Shows Core Features | [ ] Yes [ ] No |
| Hook in First 5 Seconds | [ ] Yes [ ] No |
| CTA at End | [ ] Yes [ ] No |
**Score:** ___/10
---
## Keyword Performance Audit
### Current Keyword Rankings
| Keyword | Current Rank | Volume | Competition | Score |
|---------|--------------|--------|-------------|-------|
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
### Keyword Opportunities
| Keyword | Current Rank | Potential | Action |
|---------|--------------|-----------|--------|
| | | | |
| | | | |
| | | | |
---
## Rating & Review Audit
### Rating Summary
| Metric | Value |
|--------|-------|
| Current Average Rating | /5.0 |
| Total Ratings | |
| Ratings (Last 30 Days) | |
| 5-Star Percentage | % |
| 1-Star Percentage | % |
### Review Analysis
| Category | Count | Common Themes |
|----------|-------|---------------|
| Positive (4-5 stars) | | |
| Neutral (3 stars) | | |
| Negative (1-2 stars) | | |
### Response Rate
| Metric | Value |
|--------|-------|
| Reviews Responded | % |
| Avg Response Time | hours |
**Rating Score:** ___/10
**Recommendations:**
- [ ]
- [ ]
---
## Competitor Comparison
### Top 3 Competitors
| Metric | Your App | Competitor 1 | Competitor 2 | Competitor 3 |
|--------|----------|--------------|--------------|--------------|
| Name | | | | |
| Rating | | | | |
| Total Ratings | | | | |
| Downloads | | | | |
| Title Keywords | | | | |
| Screenshot Count | | | | |
### Competitive Gaps
| Gap Identified | Opportunity |
|----------------|-------------|
| | |
| | |
| | |
---
## Overall ASO Score
| Category | Weight | Score | Weighted |
|----------|--------|-------|----------|
| Title/Metadata | 25% | /10 | |
| Keywords | 25% | /10 | |
| Visual Assets | 25% | /10 | |
| Ratings/Reviews | 25% | /10 | |
| **TOTAL** | 100% | | **/100** |
---
## Priority Action Items
### High Priority (This Week)
1. [ ]
2. [ ]
3. [ ]
### Medium Priority (This Month)
1. [ ]
2. [ ]
3. [ ]
### Low Priority (This Quarter)
1. [ ]
2. [ ]
3. [ ]
---
## Audit Sign-Off
| Role | Name | Date |
|------|------|------|
| Auditor | | |
| Reviewer | | |
| App Owner | | |
---
## Notes
_Additional observations and context:_
FILE:expected_output.json
{
"request_type": "keyword_research",
"app_name": "TaskFlow Pro",
"keyword_analysis": {
"total_keywords_analyzed": 25,
"primary_keywords": [
{
"keyword": "task manager",
"search_volume": 45000,
"competition_level": "high",
"relevance_score": 0.95,
"difficulty_score": 72.5,
"potential_score": 78.3,
"recommendation": "High priority - target immediately"
},
{
"keyword": "productivity app",
"search_volume": 38000,
"competition_level": "high",
"relevance_score": 0.90,
"difficulty_score": 68.2,
"potential_score": 75.1,
"recommendation": "High priority - target immediately"
},
{
"keyword": "todo list",
"search_volume": 52000,
"competition_level": "very_high",
"relevance_score": 0.85,
"difficulty_score": 78.9,
"potential_score": 71.4,
"recommendation": "High priority - target immediately"
}
],
"secondary_keywords": [
{
"keyword": "team task manager",
"search_volume": 8500,
"competition_level": "medium",
"relevance_score": 0.88,
"difficulty_score": 42.3,
"potential_score": 68.7,
"recommendation": "Good opportunity - include in metadata"
},
{
"keyword": "project planning app",
"search_volume": 12000,
"competition_level": "medium",
"relevance_score": 0.75,
"difficulty_score": 48.1,
"potential_score": 64.2,
"recommendation": "Good opportunity - include in metadata"
}
],
"long_tail_keywords": [
{
"keyword": "ai task prioritization",
"search_volume": 2800,
"competition_level": "low",
"relevance_score": 0.95,
"difficulty_score": 25.4,
"potential_score": 82.6,
"recommendation": "Excellent long-tail opportunity"
},
{
"keyword": "team productivity tool",
"search_volume": 3500,
"competition_level": "low",
"relevance_score": 0.85,
"difficulty_score": 28.7,
"potential_score": 79.3,
"recommendation": "Excellent long-tail opportunity"
}
]
},
"competitor_insights": {
"competitors_analyzed": 4,
"common_keywords": [
"task",
"todo",
"list",
"productivity",
"organize",
"manage"
],
"keyword_gaps": [
{
"keyword": "ai prioritization",
"used_by": ["None of the major competitors"],
"opportunity": "Unique positioning opportunity"
},
{
"keyword": "smart task manager",
"used_by": ["Things 3"],
"opportunity": "Underutilized by most competitors"
}
]
},
"metadata_recommendations": {
"apple_app_store": {
"title_options": [
{
"title": "TaskFlow - AI Task Manager",
"length": 26,
"keywords_included": ["task manager", "ai"],
"strategy": "brand_plus_primary"
},
{
"title": "TaskFlow: Smart Todo & Tasks",
"length": 29,
"keywords_included": ["todo", "tasks"],
"strategy": "brand_plus_multiple"
}
],
"subtitle_recommendation": "AI-Powered Team Productivity",
"keyword_field": "productivity,organize,planner,schedule,workflow,reminders,collaboration,calendar,sync,priorities",
"description_focus": "Lead with AI differentiation, emphasize team features"
},
"google_play_store": {
"title_options": [
{
"title": "TaskFlow - AI Task Manager & Team Productivity",
"length": 48,
"keywords_included": ["task manager", "ai", "team", "productivity"],
"strategy": "keyword_rich"
}
],
"short_description_recommendation": "AI task manager - Organize, prioritize, and collaborate with your team",
"description_focus": "Keywords naturally integrated throughout 4000 character description"
}
},
"strategic_recommendations": [
"Focus on 'AI prioritization' as unique differentiator - low competition, high relevance",
"Target 'team task manager' and 'team productivity' keywords - good search volume, lower competition than generic terms",
"Include long-tail keywords in description for additional discovery opportunities",
"Test title variations with A/B testing after launch",
"Monitor competitor keyword changes quarterly"
],
"priority_actions": [
{
"action": "Optimize app title with primary keyword",
"priority": "high",
"expected_impact": "15-25% improvement in search visibility"
},
{
"action": "Create description highlighting AI features with natural keyword integration",
"priority": "high",
"expected_impact": "10-15% improvement in conversion rate"
},
{
"action": "Plan A/B tests for icon and screenshots post-launch",
"priority": "medium",
"expected_impact": "5-10% improvement in conversion rate"
}
],
"aso_health_estimate": {
"current_score": "N/A (pre-launch)",
"potential_score_with_optimizations": "75-80/100",
"key_strengths": [
"Unique AI differentiation",
"Clear target audience",
"Strong feature set"
],
"areas_to_develop": [
"Build rating volume post-launch",
"Monitor and respond to reviews",
"Continuous keyword optimization"
]
}
}
FILE:HOW_TO_USE.md
# How to Use the App Store Optimization Skill
Hey Claude—I just added the "app-store-optimization" skill. Can you help me optimize my app's presence on the App Store and Google Play?
## Example Invocations
### Keyword Research
**Example 1: Basic Keyword Research**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you research the best keywords for my productivity app? I'm targeting professionals who need task management and team collaboration features.
```
**Example 2: Competitive Keyword Analysis**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you analyze keywords that Todoist, Asana, and Monday.com are using? I want to find gaps and opportunities for my project management app.
```
### Metadata Optimization
**Example 3: Optimize App Title**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you optimize my app title for the Apple App Store? My app is called "TaskFlow" and I want to rank for "task manager", "productivity", and "team collaboration". The title needs to be under 30 characters.
```
**Example 4: Full Metadata Package**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you create optimized metadata for both Apple App Store and Google Play Store? Here's my app info:
- Name: TaskFlow
- Category: Productivity
- Key features: AI task prioritization, team collaboration, calendar integration
- Target keywords: task manager, productivity app, team tasks
```
### Competitor Analysis
**Example 5: Analyze Top Competitors**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you analyze the ASO strategies of the top 5 productivity apps in the App Store? I want to understand their title strategies, keyword usage, and visual asset approaches.
```
**Example 6: Identify Competitive Gaps**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you compare my app's ASO performance against competitors and identify what I'm missing? Here's my current metadata: [paste metadata]
```
### ASO Score Calculation
**Example 7: Calculate Overall ASO Health**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you calculate my app's ASO health score? Here are my metrics:
- Average rating: 4.2 stars
- Total ratings: 3,500
- Keywords in top 10: 3
- Keywords in top 50: 12
- Conversion rate: 4.5%
```
**Example 8: Identify Improvement Areas**
```
Hey Claude—I just added the "app-store-optimization" skill. My ASO score is 62/100. Can you tell me which areas I should focus on first to improve my rankings and downloads?
```
### A/B Testing
**Example 9: Plan Icon Test**
```
Hey Claude—I just added the "app-store-optimization" skill. I want to A/B test two different app icons. My current conversion rate is 5%. Can you help me plan the test, calculate required sample size, and determine how long to run it?
```
**Example 10: Analyze Test Results**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you analyze my A/B test results?
- Variant A (control): 2,500 visitors, 125 installs
- Variant B (new icon): 2,500 visitors, 150 installs
Is this statistically significant? Should I implement variant B?
```
### Localization
**Example 11: Plan Localization Strategy**
```
Hey Claude—I just added the "app-store-optimization" skill. I currently only have English metadata. Which markets should I localize for first? I'm a bootstrapped startup with moderate budget.
```
**Example 12: Translate Metadata**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you help me translate my app metadata to Spanish for the Mexico market? Here's my English metadata: [paste metadata]. Check if it fits within character limits.
```
### Review Analysis
**Example 13: Analyze User Reviews**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you analyze my recent reviews and tell me:
- Overall sentiment (positive/negative ratio)
- Most common complaints
- Most requested features
- Bugs that need immediate fixing
```
**Example 14: Generate Review Response Templates**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you create professional response templates for:
- Users reporting crashes
- Feature requests
- Positive 5-star reviews
- General complaints
```
### Launch Planning
**Example 15: Pre-Launch Checklist**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you generate a comprehensive pre-launch checklist for both Apple App Store and Google Play Store? My launch date is December 1, 2025.
```
**Example 16: Optimize Launch Timing**
```
Hey Claude—I just added the "app-store-optimization" skill. What's the best day and time to launch my fitness app? I want to maximize visibility and downloads in the first week.
```
**Example 17: Plan Seasonal Campaign**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you identify seasonal opportunities for my fitness app? It's currently October—what campaigns should I run for the next 6 months?
```
## What to Provide
### For Keyword Research
- App name and category
- Target audience description
- Key features and unique value proposition
- Competitor apps (optional)
- Geographic markets to target
### For Metadata Optimization
- Current app name
- Platform (Apple, Google, or both)
- Target keywords (prioritized list)
- Key features and benefits
- Target audience
- Current metadata (for optimization)
### For Competitor Analysis
- Your app category
- List of competitor app names or IDs
- Platform (Apple or Google)
- Specific aspects to analyze (keywords, visuals, ratings)
### For ASO Score Calculation
- Metadata quality metrics (title length, description length, keyword density)
- Rating data (average rating, total ratings, recent ratings)
- Keyword rankings (top 10, top 50, top 100 counts)
- Conversion metrics (impression-to-install rate, downloads)
### For A/B Testing
- Test type (icon, screenshot, title, description)
- Control variant details
- Test variant details
- Baseline conversion rate
- For results analysis: visitor and conversion counts for both variants
### For Localization
- Current market and language
- Budget level (low, medium, high)
- Target number of markets
- Current metadata text for translation
### For Review Analysis
- Recent reviews (text, rating, date)
- Platform (Apple or Google)
- Time period to analyze
- Specific focus (bugs, features, sentiment)
### For Launch Planning
- Platform (Apple, Google, or both)
- Target launch date
- App category
- App information (name, features, target audience)
## What You'll Get
### Keyword Research Output
- Prioritized keyword list with search volume estimates
- Competition level analysis
- Relevance scores
- Long-tail keyword opportunities
- Strategic recommendations
### Metadata Optimization Output
- Optimized titles (multiple options)
- Optimized descriptions (short and full)
- Keyword field optimization (Apple)
- Character count validation
- Keyword density analysis
- Before/after comparison
### Competitor Analysis Output
- Ranked competitors by ASO strength
- Common keyword patterns
- Keyword gaps and opportunities
- Visual asset assessment
- Best practices identified
- Actionable recommendations
### ASO Score Output
- Overall score (0-100)
- Breakdown by category (metadata, ratings, keywords, conversion)
- Strengths and weaknesses
- Prioritized action items
- Expected impact of improvements
### A/B Test Output
- Test design with hypothesis
- Required sample size calculation
- Duration estimates
- Statistical significance analysis
- Implementation recommendations
- Learnings and insights
### Localization Output
- Prioritized target markets
- Estimated translation costs
- ROI projections
- Character limit validation for each language
- Cultural adaptation recommendations
- Phased implementation plan
### Review Analysis Output
- Sentiment distribution (positive/neutral/negative)
- Common themes and topics
- Top issues requiring fixes
- Most requested features
- Response templates
- Trend analysis over time
### Launch Planning Output
- Platform-specific checklists (Apple, Google, Universal)
- Timeline with milestones
- Compliance validation
- Optimal launch timing recommendations
- Seasonal campaign opportunities
- Update cadence planning
## Tips for Best Results
1. **Be Specific**: Provide as much detail about your app as possible
2. **Include Context**: Share your goals (increase downloads, improve ranking, boost conversion)
3. **Provide Data**: Real metrics enable more accurate analysis
4. **Iterate**: Start with keyword research, then optimize metadata, then test
5. **Track Results**: Monitor changes after implementing recommendations
6. **Stay Compliant**: Always verify recommendations against current App Store/Play Store guidelines
7. **Test First**: Use A/B testing before making major metadata changes
8. **Localize Strategically**: Start with highest-ROI markets first
9. **Respond to Reviews**: Use provided templates to engage with users
10. **Plan Ahead**: Use launch checklists and timelines to avoid last-minute rushes
## Common Workflows
### New App Launch
1. Keyword research → Competitor analysis → Metadata optimization → Pre-launch checklist → Launch timing optimization
### Improving Existing App
1. ASO score calculation → Identify gaps → Metadata optimization → A/B testing → Review analysis → Implement changes
### International Expansion
1. Localization planning → Market prioritization → Metadata translation → ROI analysis → Phased rollout
### Ongoing Optimization
1. Monthly keyword ranking tracking → Quarterly metadata updates → Continuous A/B testing → Review monitoring → Seasonal campaigns
## Need Help?
If you need clarification on any aspect of ASO or want to combine multiple analyses, just ask! For example:
```
Hey Claude—I just added the "app-store-optimization" skill. Can you create a complete ASO strategy for my new productivity app? I need keyword research, optimized metadata for both stores, a pre-launch checklist, and launch timing recommendations.
```
The skill can handle comprehensive, multi-phase ASO projects as well as specific tactical optimizations.
FILE:README.md
# App Store Optimization (ASO) Skill
**Version**: 1.0.0
**Last Updated**: November 7, 2025
**Author**: Claude Skills Factory
## Overview
A comprehensive App Store Optimization (ASO) skill that provides complete capabilities for researching, optimizing, and tracking mobile app performance on the Apple App Store and Google Play Store. This skill empowers app developers and marketers to maximize their app's visibility, downloads, and success in competitive app marketplaces.
## What This Skill Does
This skill provides end-to-end ASO capabilities across seven key areas:
1. **Research & Analysis**: Keyword research, competitor analysis, market trends, review sentiment
2. **Metadata Optimization**: Title, description, keywords with platform-specific character limits
3. **Conversion Optimization**: A/B testing framework, visual asset optimization
4. **Rating & Review Management**: Sentiment analysis, response strategies, issue identification
5. **Launch & Update Strategies**: Pre-launch checklists, timing optimization, update planning
6. **Analytics & Tracking**: ASO scoring, keyword rankings, performance benchmarking
7. **Localization**: Multi-language strategy, translation management, ROI analysis
## Key Features
### Comprehensive Keyword Research
- Search volume and competition analysis
- Long-tail keyword discovery
- Competitor keyword extraction
- Keyword difficulty scoring
- Strategic prioritization
### Platform-Specific Metadata Optimization
- **Apple App Store**:
- Title (30 chars)
- Subtitle (30 chars)
- Promotional Text (170 chars)
- Description (4000 chars)
- Keywords field (100 chars)
- **Google Play Store**:
- Title (50 chars)
- Short Description (80 chars)
- Full Description (4000 chars)
- Character limit validation
- Keyword density analysis
- Multiple optimization strategies
### Competitor Intelligence
- Automated competitor discovery
- Metadata strategy analysis
- Visual asset assessment
- Gap identification
- Competitive positioning
### ASO Health Scoring
- 0-100 overall score
- Four-category breakdown (Metadata, Ratings, Keywords, Conversion)
- Strengths and weaknesses identification
- Prioritized action recommendations
- Expected impact estimates
### Scientific A/B Testing
- Test design and hypothesis formulation
- Sample size calculation
- Statistical significance analysis
- Duration estimation
- Implementation recommendations
### Global Localization
- Market prioritization (Tier 1/2/3)
- Translation cost estimation
- Character limit adaptation by language
- Cultural keyword considerations
- ROI analysis
### Review Intelligence
- Sentiment analysis
- Common theme extraction
- Bug and issue identification
- Feature request clustering
- Professional response templates
### Launch Planning
- Platform-specific checklists
- Timeline generation
- Compliance validation
- Optimal timing recommendations
- Seasonal campaign planning
## Python Modules
This skill includes 8 powerful Python modules:
### 1. keyword_analyzer.py
**Purpose**: Analyzes keywords for search volume, competition, and relevance
**Key Functions**:
- `analyze_keyword()`: Single keyword analysis
- `compare_keywords()`: Multi-keyword comparison and ranking
- `find_long_tail_opportunities()`: Generate long-tail variations
- `calculate_keyword_density()`: Analyze keyword usage in text
- `extract_keywords_from_text()`: Extract keywords from reviews/descriptions
### 2. metadata_optimizer.py
**Purpose**: Optimizes titles, descriptions, keywords with character limit validation
**Key Functions**:
- `optimize_title()`: Generate optimal title options
- `optimize_description()`: Create conversion-focused descriptions
- `optimize_keyword_field()`: Maximize Apple's 100-char keyword field
- `validate_character_limits()`: Ensure platform compliance
- `calculate_keyword_density()`: Analyze keyword integration
### 3. competitor_analyzer.py
**Purpose**: Analyzes competitor ASO strategies
**Key Functions**:
- `analyze_competitor()`: Single competitor deep-dive
- `compare_competitors()`: Multi-competitor analysis
- `identify_gaps()`: Find competitive opportunities
- `_calculate_competitive_strength()`: Score competitor ASO quality
### 4. aso_scorer.py
**Purpose**: Calculates comprehensive ASO health score
**Key Functions**:
- `calculate_overall_score()`: 0-100 ASO health score
- `score_metadata_quality()`: Evaluate metadata optimization
- `score_ratings_reviews()`: Assess rating quality and volume
- `score_keyword_performance()`: Analyze ranking positions
- `score_conversion_metrics()`: Evaluate conversion rates
- `generate_recommendations()`: Prioritized improvement actions
### 5. ab_test_planner.py
**Purpose**: Plans and tracks A/B tests for ASO elements
**Key Functions**:
- `design_test()`: Create test hypothesis and structure
- `calculate_sample_size()`: Determine required visitors
- `calculate_significance()`: Assess statistical validity
- `track_test_results()`: Monitor ongoing tests
- `generate_test_report()`: Create comprehensive test reports
### 6. localization_helper.py
**Purpose**: Manages multi-language ASO optimization
**Key Functions**:
- `identify_target_markets()`: Prioritize localization markets
- `translate_metadata()`: Adapt metadata for languages
- `adapt_keywords()`: Cultural keyword adaptation
- `validate_translations()`: Character limit validation
- `calculate_localization_roi()`: Estimate investment returns
### 7. review_analyzer.py
**Purpose**: Analyzes user reviews for actionable insights
**Key Functions**:
- `analyze_sentiment()`: Calculate sentiment distribution
- `extract_common_themes()`: Identify frequent topics
- `identify_issues()`: Surface bugs and problems
- `find_feature_requests()`: Extract desired features
- `track_sentiment_trends()`: Monitor changes over time
- `generate_response_templates()`: Create review responses
### 8. launch_checklist.py
**Purpose**: Generates comprehensive launch and update checklists
**Key Functions**:
- `generate_prelaunch_checklist()`: Complete submission validation
- `validate_app_store_compliance()`: Check guidelines compliance
- `create_update_plan()`: Plan update cadence
- `optimize_launch_timing()`: Recommend launch dates
- `plan_seasonal_campaigns()`: Identify seasonal opportunities
## Installation
### For Claude Code (Desktop/CLI)
#### Project-Level Installation
```bash
# Copy skill folder to project
cp -r app-store-optimization /path/to/your/project/.claude/skills/
# Claude will auto-load the skill when working in this project
```
#### User-Level Installation (Available in All Projects)
```bash
# Copy skill folder to user-level skills
cp -r app-store-optimization ~/.claude/skills/
# Claude will load this skill in all your projects
```
### For Claude Apps (Browser)
1. Use the `skill-creator` skill to import the skill
2. Or manually import via Claude Apps interface
### Verification
To verify installation:
```bash
# Check if skill folder exists
ls ~/.claude/skills/app-store-optimization/
# You should see:
# SKILL.md
# keyword_analyzer.py
# metadata_optimizer.py
# competitor_analyzer.py
# aso_scorer.py
# ab_test_planner.py
# localization_helper.py
# review_analyzer.py
# launch_checklist.py
# sample_input.json
# expected_output.json
# HOW_TO_USE.md
# README.md
```
## Usage Examples
### Example 1: Complete Keyword Research
```
Hey Claude—I just added the "app-store-optimization" skill. Can you research keywords for my fitness app? I'm targeting people who want home workouts, yoga, and meal planning. Analyze top competitors like Nike Training Club and Peloton.
```
**What Claude will do**:
- Use `keyword_analyzer.py` to research keywords
- Use `competitor_analyzer.py` to analyze Nike Training Club and Peloton
- Provide prioritized keyword list with search volumes, competition levels
- Identify gaps and long-tail opportunities
- Recommend primary keywords for title and secondary keywords for description
### Example 2: Optimize App Store Metadata
```
Hey Claude—I just added the "app-store-optimization" skill. Optimize my app's metadata for both Apple App Store and Google Play Store:
- App: FitFlow
- Category: Health & Fitness
- Features: AI workout plans, nutrition tracking, progress photos
- Keywords: fitness app, workout planner, home fitness
```
**What Claude will do**:
- Use `metadata_optimizer.py` to create optimized titles (multiple options)
- Generate platform-specific descriptions (short and full)
- Optimize Apple's 100-character keyword field
- Validate all character limits
- Calculate keyword density
- Provide before/after comparison
### Example 3: Calculate ASO Health Score
```
Hey Claude—I just added the "app-store-optimization" skill. Calculate my app's ASO score:
- Average rating: 4.3 stars (8,200 ratings)
- Keywords in top 10: 4
- Keywords in top 50: 15
- Conversion rate: 3.8%
- Title: "FitFlow - Home Workouts"
- Description: 1,500 characters with 3 keyword mentions
```
**What Claude will do**:
- Use `aso_scorer.py` to calculate overall score (0-100)
- Break down by category (Metadata: X/25, Ratings: X/25, Keywords: X/25, Conversion: X/25)
- Identify strengths and weaknesses
- Generate prioritized recommendations
- Estimate impact of improvements
### Example 4: A/B Test Planning
```
Hey Claude—I just added the "app-store-optimization" skill. I want to A/B test my app icon. My current conversion rate is 4.2%. How many visitors do I need and how long should I run the test?
```
**What Claude will do**:
- Use `ab_test_planner.py` to design test
- Calculate required sample size (based on minimum detectable effect)
- Estimate test duration for low/medium/high traffic scenarios
- Provide test structure and success metrics
- Explain how to analyze results
### Example 5: Review Sentiment Analysis
```
Hey Claude—I just added the "app-store-optimization" skill. Analyze my last 500 reviews and tell me:
- Overall sentiment
- Most common complaints
- Top feature requests
- Bugs needing immediate fixes
```
**What Claude will do**:
- Use `review_analyzer.py` to process reviews
- Calculate sentiment distribution
- Extract common themes
- Identify and prioritize issues
- Cluster feature requests
- Generate response templates
### Example 6: Pre-Launch Checklist
```
Hey Claude—I just added the "app-store-optimization" skill. Generate a complete pre-launch checklist for both app stores. My launch date is March 15, 2026.
```
**What Claude will do**:
- Use `launch_checklist.py` to generate checklists
- Create Apple App Store checklist (metadata, assets, technical, legal)
- Create Google Play Store checklist (metadata, assets, technical, legal)
- Add universal checklist (marketing, QA, support)
- Generate timeline with milestones
- Calculate completion percentage
## Best Practices
### Keyword Research
1. Start with 20-30 seed keywords
2. Analyze top 5 competitors in your category
3. Balance high-volume and long-tail keywords
4. Prioritize relevance over search volume
5. Update keyword research quarterly
### Metadata Optimization
1. Front-load keywords in title (first 15 characters most important)
2. Use every available character (don't waste space)
3. Write for humans first, search engines second
4. A/B test major changes before committing
5. Update descriptions with each major release
### A/B Testing
1. Test one element at a time (icon vs. screenshots vs. title)
2. Run tests to statistical significance (90%+ confidence)
3. Test high-impact elements first (icon has biggest impact)
4. Allow sufficient duration (at least 1 week, preferably 2-3)
5. Document learnings for future tests
### Localization
1. Start with top 5 revenue markets (US, China, Japan, Germany, UK)
2. Use professional translators, not machine translation
3. Test translations with native speakers
4. Adapt keywords for cultural context
5. Monitor ROI by market
### Review Management
1. Respond to reviews within 24-48 hours
2. Always be professional, even with negative reviews
3. Address specific issues raised
4. Thank users for positive feedback
5. Use insights to prioritize product improvements
## Technical Requirements
- **Python**: 3.7+ (for Python modules)
- **Platform Support**: Apple App Store, Google Play Store
- **Data Formats**: JSON input/output
- **Dependencies**: Standard library only (no external packages required)
## Limitations
### Data Dependencies
- Keyword search volumes are estimates (no official Apple/Google data)
- Competitor data limited to publicly available information
- Review analysis requires access to public reviews
- Historical data may not be available for new apps
### Platform Constraints
- Apple: Metadata changes require app submission (except Promotional Text)
- Google: Metadata changes take 1-2 hours to index
- A/B testing requires significant traffic for statistical significance
- Store algorithms are proprietary and change without notice
### Scope
- Does not include paid user acquisition (Apple Search Ads, Google Ads)
- Does not cover in-app analytics implementation
- Does not handle technical app development
- Focuses on organic discovery and conversion optimization
## Troubleshooting
### Issue: Python modules not found
**Solution**: Ensure all .py files are in the same directory as SKILL.md
### Issue: Character limit validation failing
**Solution**: Check that you're using the correct platform ('apple' or 'google')
### Issue: Keyword research returning limited results
**Solution**: Provide more context about your app, features, and target audience
### Issue: ASO score seems inaccurate
**Solution**: Ensure you're providing accurate metrics (ratings, keyword rankings, conversion rate)
## Version History
### Version 1.0.0 (November 7, 2025)
- Initial release
- 8 Python modules with comprehensive ASO capabilities
- Support for both Apple App Store and Google Play Store
- Keyword research, metadata optimization, competitor analysis
- ASO scoring, A/B testing, localization, review analysis
- Launch planning and seasonal campaign tools
## Support & Feedback
This skill is designed to help app developers and marketers succeed in competitive app marketplaces. For the best results:
1. Provide detailed context about your app
2. Include specific metrics when available
3. Ask follow-up questions for clarification
4. Iterate based on results
## Credits
Developed by Claude Skills Factory
Based on industry-standard ASO best practices
Platform requirements current as of November 2025
## License
This skill is provided as-is for use with Claude Code and Claude Apps. Customize and extend as needed for your specific use cases.
---
**Ready to optimize your app?** Start with keyword research, then move to metadata optimization, and finally implement A/B testing for continuous improvement. The skill handles everything from pre-launch planning to ongoing optimization.
For detailed usage examples, see [HOW_TO_USE.md](HOW_TO_USE.md).
FILE:references/aso-best-practices.md
# ASO Best Practices Reference
Optimization strategies for improving app store visibility, conversion, and rankings.
---
## Table of Contents
- [Keyword Optimization](#keyword-optimization)
- [Metadata Optimization](#metadata-optimization)
- [Visual Asset Optimization](#visual-asset-optimization)
- [Rating and Review Management](#rating-and-review-management)
- [Launch Strategy](#launch-strategy)
- [A/B Testing Framework](#ab-testing-framework)
- [Conversion Optimization](#conversion-optimization)
- [Common Mistakes to Avoid](#common-mistakes-to-avoid)
---
## Keyword Optimization
### Keyword Research Process
1. **Brainstorm seed keywords** - Core terms users search for
2. **Expand with variations** - Synonyms, related terms, long-tail
3. **Analyze competition** - Check difficulty scores
4. **Evaluate search volume** - Prioritize high-volume terms
5. **Test and iterate** - Monitor rankings and adjust
### Keyword Selection Criteria
| Factor | Weight | Evaluation Method |
|--------|--------|-------------------|
| Relevance | 40% | Does it describe app function? |
| Search Volume | 30% | Monthly search estimates |
| Competition | 20% | Number of ranking apps |
| Conversion Potential | 10% | User intent alignment |
### Keyword Placement Priority
| Location | Search Weight | Example |
|----------|---------------|---------|
| App Title | Highest | "TaskMaster - Todo List Manager" |
| Subtitle (iOS) | High | "Organize Your Daily Tasks" |
| Keyword Field (iOS) | High | "planner,reminder,checklist" |
| Short Description (Android) | High | "Simple task manager for busy professionals" |
| Full Description | Medium | Natural keyword usage throughout |
### Long-Tail Keyword Strategy
Long-tail keywords have lower volume but higher conversion:
| Type | Example | Volume | Competition | Conversion |
|------|---------|--------|-------------|------------|
| Short-tail | "todo app" | High | High | Low |
| Mid-tail | "daily task manager" | Medium | Medium | Medium |
| Long-tail | "free todo list with reminders" | Low | Low | High |
**Formula for keyword priority:**
```
Score = (Volume × 0.3) + (1/Competition × 0.3) + (Relevance × 0.4)
```
---
## Metadata Optimization
### Title Optimization
**Structure Formula:**
```
[Brand Name] - [Primary Keyword] [Secondary Keyword/Benefit]
```
**Examples by category:**
| Category | Before | After |
|----------|--------|-------|
| Productivity | "MyTasks" | "MyTasks - Todo List & Planner" |
| Fitness | "FitTrack" | "FitTrack: Workout & Gym Log" |
| Finance | "MoneyApp" | "MoneyApp - Budget Tracker" |
| Photo | "SnapEdit" | "SnapEdit: Photo Editor & AI" |
**Title Optimization Checklist:**
- [ ] Primary keyword within first 3 words
- [ ] Brand name is memorable and unique
- [ ] Character count matches platform limit
- [ ] No keyword stuffing
- [ ] Readable and natural sounding
### Description Optimization
**Full Description Structure:**
```
PARAGRAPH 1: Hook + Primary Benefit (50-100 words)
- Address user pain point
- State main value proposition
- Include primary keyword naturally
PARAGRAPH 2-3: Feature Highlights (100-150 words)
- Top 3-5 features with benefits
- Use bullet points or emojis for scanability
- Include secondary keywords
PARAGRAPH 4: Social Proof (50-75 words)
- Download numbers or ratings
- Press mentions or awards
- User testimonials (summarized)
PARAGRAPH 5: Call to Action (25-50 words)
- Clear next step
- Urgency or incentive
- Reassurance (free trial, no credit card)
```
**Keyword Density Target:**
- Primary keyword: 2-3% (8-12 mentions in 4000 chars)
- Secondary keywords: 1-2% each (4-8 mentions each)
### Subtitle Optimization (iOS)
**Effective Subtitle Formulas:**
| Formula | Example |
|---------|---------|
| [Verb] + [Benefit] | "Organize Your Life" |
| [Adjective] + [Category] | "Smart Task Manager" |
| [Feature] + [Feature] | "Lists, Reminders & Notes" |
| [Audience] + [Solution] | "For Busy Professionals" |
---
## Visual Asset Optimization
### App Icon Best Practices
| Principle | Do | Don't |
|-----------|-----|-------|
| Simplicity | Single focal element | Multiple competing elements |
| Recognizability | Works at 60x60px | Requires large size to read |
| Uniqueness | Distinct from competitors | Generic category icon |
| Color | Bold, contrasting colors | Muted or similar to background |
| Text | None or single letter | Full words or app name |
**Icon Testing Questions:**
1. Is it recognizable at 29x29px (smallest iOS size)?
2. Does it stand out in search results?
3. Does it communicate app function?
4. Is it distinct from top 10 category competitors?
### Screenshot Optimization
**Screenshot Hierarchy:**
| Position | Purpose | Content Strategy |
|----------|---------|------------------|
| Screenshot 1 | Hook/Hero | Main value proposition + key UI |
| Screenshot 2 | Primary Feature | Most-used feature demonstration |
| Screenshot 3 | Secondary Feature | Differentiating capability |
| Screenshot 4 | Social Proof | Ratings, awards, user count |
| Screenshot 5+ | Additional Features | Supporting functionality |
**Caption Best Practices:**
- Maximum 5-7 words per caption
- Action-oriented verbs ("Track", "Organize", "Discover")
- Benefit-focused, not feature-focused
- Consistent typography and style
**Example Caption Evolution:**
| Weak | Better | Best |
|------|--------|------|
| "Task List Feature" | "Create Task Lists" | "Never Forget a Task Again" |
| "Calendar View" | "See Your Schedule" | "Plan Your Week in Seconds" |
| "Notifications" | "Get Reminders" | "Stay on Top of Deadlines" |
### Video Preview Strategy
**Video Structure (30 seconds):**
| Seconds | Content |
|---------|---------|
| 0-5 | Hook: Show end result or main benefit |
| 5-15 | Demo: Core feature in action |
| 15-25 | Features: Quick feature montage |
| 25-30 | CTA: Logo and download prompt |
---
## Rating and Review Management
### Review Response Framework
**For Negative Reviews (1-2 stars):**
```
Structure:
1. Acknowledge the issue (1 sentence)
2. Apologize without making excuses (1 sentence)
3. Offer solution or next step (1-2 sentences)
4. Invite direct contact (1 sentence)
Example:
"We're sorry the syncing issues are affecting your experience.
Our team is actively working on a fix for the next update.
In the meantime, please try logging out and back in, which
resolves this for most users. If issues persist, email us at
support@app.com and we'll prioritize your case."
```
**For Positive Reviews (4-5 stars):**
```
Structure:
1. Thank sincerely (1 sentence)
2. Acknowledge specific praise (1 sentence)
3. Encourage continued use or sharing (1 sentence)
Example:
"Thank you for the kind words! We're thrilled the reminder
feature helps you stay organized. If you're enjoying the app,
we'd love if you'd share it with friends who might benefit."
```
### Rating Improvement Tactics
| Tactic | Implementation | Expected Impact |
|--------|----------------|-----------------|
| In-app prompt timing | After positive action (task completed, milestone reached) | +0.3 stars |
| Bug fix velocity | Address 1-star issues within 7 days | +0.2 stars |
| Response rate | Reply to 80%+ of reviews | +0.1 stars |
| Feature requests | Implement top-requested features | +0.2 stars |
### Review Prompt Best Practices
**When to prompt:**
- After user completes 5+ successful sessions
- After milestone achievement (first task completed, 7-day streak)
- After positive in-app feedback ("Was this helpful? Yes")
**When NOT to prompt:**
- First session
- After error or crash
- During critical workflow
- More than once per 30 days
---
## Launch Strategy
### Pre-Launch Checklist
**4 Weeks Before Launch:**
- [ ] Finalize app name and keywords
- [ ] Complete all metadata fields
- [ ] Prepare all visual assets
- [ ] Set up analytics (Firebase, Mixpanel)
- [ ] Create press kit and media assets
- [ ] Build email list for launch notification
**2 Weeks Before Launch:**
- [ ] Submit for app review
- [ ] Prepare social media content
- [ ] Brief press and influencers
- [ ] Set up review response templates
- [ ] Configure in-app rating prompts
**Launch Day:**
- [ ] Verify app is live in stores
- [ ] Announce across all channels
- [ ] Monitor reviews and respond quickly
- [ ] Track download velocity
- [ ] Document any issues for Day 2 fix
### Update Cadence
| Update Type | Frequency | ASO Impact |
|-------------|-----------|------------|
| Bug fixes | As needed | Prevents rating drops |
| Minor features | Every 2-4 weeks | Maintains freshness signal |
| Major features | Every 4-8 weeks | Opportunity for "What's New" |
| Metadata refresh | Every 4-6 weeks | Keyword optimization cycle |
### Seasonal Optimization
| Season | Optimization Focus | Example Categories |
|--------|--------------------|--------------------|
| Jan (New Year) | Resolutions, goals | Fitness, Productivity |
| Feb (Valentine's) | Dating, relationships | Dating, Photo |
| Mar-Apr (Tax) | Finance, organization | Finance, Productivity |
| May-Jun (Summer) | Travel, fitness | Travel, Health |
| Aug-Sep (Back to School) | Education, organization | Education, Productivity |
| Nov-Dec (Holidays) | Shopping, social | Shopping, Social |
---
## A/B Testing Framework
### Test Prioritization Matrix
| Element | Impact | Ease | Priority |
|---------|--------|------|----------|
| App Icon | High | Medium | 1 |
| Screenshot 1 | High | Medium | 2 |
| Title | High | Easy | 3 |
| Short Description | Medium | Easy | 4 |
| Screenshots 2-5 | Medium | Medium | 5 |
| Video | Medium | Hard | 6 |
### Sample Size Calculator
**Formula:**
```
Sample Size = (2 × (Z² × p × (1-p))) / E²
Where:
Z = 1.96 (for 95% confidence)
p = baseline conversion rate
E = minimum detectable effect (usually 0.05)
```
**Quick Reference:**
| Baseline CVR | Min. Impressions for 5% Lift |
|--------------|------------------------------|
| 1% | 31,000 per variant |
| 2% | 15,500 per variant |
| 5% | 6,200 per variant |
| 10% | 3,100 per variant |
### Test Duration Guidelines
| Daily Impressions | Minimum Test Duration |
|-------------------|----------------------|
| 1,000+ | 7 days |
| 500-1,000 | 14 days |
| 100-500 | 30 days |
| <100 | Not recommended |
---
## Conversion Optimization
### Conversion Funnel Metrics
| Stage | Metric | Benchmark |
|-------|--------|-----------|
| Discovery | Impressions | Category dependent |
| Consideration | Page Views | 30-50% of impressions |
| Conversion | Installs | 3-8% of page views |
| Activation | First Open | 70-90% of installs |
### Conversion Optimization Levers
| Lever | Typical Lift | Effort |
|-------|--------------|--------|
| Icon redesign | 10-25% | High |
| Screenshot optimization | 15-35% | Medium |
| Title keyword optimization | 5-15% | Low |
| Description rewrite | 5-10% | Low |
| Video addition | 10-20% | High |
| Localization | 20-50% per market | Medium |
---
## Common Mistakes to Avoid
### Keyword Mistakes
| Mistake | Problem | Solution |
|---------|---------|----------|
| Keyword stuffing | Spam detection, rejection | Natural usage, 2-3% density |
| Competitor names | Guideline violation | Focus on category terms |
| Duplicate keywords | Wasted character space | Remove duplicates from keyword field |
| Ignoring long-tail | Missing conversion | Include specific phrases |
### Metadata Mistakes
| Mistake | Problem | Solution |
|---------|---------|----------|
| Vague descriptions | Low conversion | Specific benefits and features |
| Feature-focused copy | Doesn't resonate | Benefit-focused messaging |
| Outdated information | Misleading users | Update with each release |
| Missing localization | Lost global revenue | Prioritize top 5 markets |
### Visual Asset Mistakes
| Mistake | Problem | Solution |
|---------|---------|----------|
| Text-heavy screenshots | Unreadable on phones | Minimal text, clear UI focus |
| Inconsistent style | Unprofessional appearance | Design system for all assets |
| Portrait-only screenshots | Missed tablet users | Include landscape variants |
| No social proof | Lower trust | Add ratings, awards, press |
### Launch Mistakes
| Mistake | Problem | Solution |
|---------|---------|----------|
| Launching on Friday | No support over weekend | Launch Tuesday-Wednesday |
| No analytics setup | Can't measure success | Firebase/Mixpanel before launch |
| Immediate rating prompt | Negative ratings | Wait for positive experience |
| Ignoring reviews | Declining ratings | Respond within 24-48 hours |
FILE:references/keyword-research-guide.md
# Keyword Research Guide
Systematic approach to discovering, evaluating, and selecting keywords for app store optimization.
---
## Table of Contents
- [Keyword Research Methodology](#keyword-research-methodology)
- [Keyword Evaluation Framework](#keyword-evaluation-framework)
- [Competitor Keyword Analysis](#competitor-keyword-analysis)
- [Keyword Mapping Strategy](#keyword-mapping-strategy)
- [Keyword Tracking and Iteration](#keyword-tracking-and-iteration)
---
## Keyword Research Methodology
### Phase 1: Seed Keyword Generation
Start by generating initial keyword ideas from multiple sources.
**Source 1: Core App Functions**
List every action or problem the app solves:
```
Example for a task management app:
- Create tasks
- Set reminders
- Track deadlines
- Organize projects
- Collaborate with team
- Plan daily schedule
```
**Source 2: User Language Mapping**
Match developer terminology to user searches:
| Developer Term | User Search Terms |
|----------------|-------------------|
| Task management | todo list, task app, tasks |
| Project organization | project planner, project tracker |
| Deadline tracking | due date reminder, deadline app |
| Time blocking | schedule planner, calendar app |
| GTD methodology | getting things done, productivity system |
**Source 3: App Store Autocomplete**
Type seed keywords into App Store/Play Store search and record suggestions:
```
"todo" → todo list, todo app, todo list app, todolist widget
"task" → task manager, task planner, task list, tasks to do
"remind" → reminder app, reminder, reminders widget, remind me
```
**Source 4: Competitor Analysis**
Extract keywords from top 10 competitors in category (detailed in section below).
### Phase 2: Keyword Expansion
**Expansion Techniques:**
| Technique | Example (seed: "todo") |
|-----------|------------------------|
| Add modifiers | free todo, best todo, simple todo |
| Add actions | make todo list, create todo, organize todo |
| Add platforms | todo app iphone, todo for mac, todo widget |
| Add audiences | todo for students, business todo, family todo |
| Add features | todo with reminders, todo calendar, todo sync |
| Add problems | forgot tasks todo, procrastination todo |
**Keyword Matrix Template:**
| Core Term | Modifier 1 | Modifier 2 | Full Keyword |
|-----------|------------|------------|--------------|
| todo | free | app | free todo app |
| todo | best | iphone | best todo iphone |
| task | manager | simple | simple task manager |
| reminder | daily | widget | daily reminder widget |
| planner | weekly | calendar | weekly planner calendar |
### Phase 3: Keyword Filtering
Remove irrelevant or low-quality keywords:
**Exclusion Criteria:**
| Criterion | Reason | Example |
|-----------|--------|---------|
| Competitor brand names | Policy violation | "todoist alternative" |
| Unrelated categories | Low conversion | "todo games" |
| Plural duplicates (iOS) | Wasted space | "tasks" when "task" exists |
| Single characters | No search value | "to do" vs "todo" |
---
## Keyword Evaluation Framework
### Keyword Scoring Model
Evaluate each keyword on four dimensions:
**1. Search Volume (0-100)**
| Volume Level | Score | Monthly Searches |
|--------------|-------|------------------|
| Very High | 80-100 | 50,000+ |
| High | 60-79 | 10,000-49,999 |
| Medium | 40-59 | 1,000-9,999 |
| Low | 20-39 | 100-999 |
| Very Low | 0-19 | <100 |
**2. Competition (0-100, inverted)**
| Competition | Score | Top 10 App Ratings |
|-------------|-------|-------------------|
| Very Low | 80-100 | Average <4.0 stars |
| Low | 60-79 | Average 4.0-4.2 stars |
| Medium | 40-59 | Average 4.3-4.5 stars |
| High | 20-39 | Average 4.6-4.8 stars |
| Very High | 0-19 | Average 4.9+ stars |
**3. Relevance (0-100)**
| Relevance | Score | Criteria |
|-----------|-------|----------|
| Exact Match | 90-100 | Keyword describes core function |
| Strong Match | 70-89 | Keyword describes major feature |
| Moderate Match | 50-69 | Keyword describes secondary feature |
| Weak Match | 30-49 | Keyword tangentially related |
| No Match | 0-29 | Keyword unrelated to app |
**4. Conversion Potential (0-100)**
| Intent | Score | User Query Type |
|--------|-------|-----------------|
| Transactional | 80-100 | "best [app type]", "[app type] app" |
| Commercial | 60-79 | "free [app type]", "[app type] for [use]" |
| Informational | 40-59 | "how to [action]", "what is [concept]" |
| Navigational | 20-39 | "[brand name]", "[specific app]" |
### Composite Score Calculation
```
Keyword Score = (Volume × 0.25) + (Competition × 0.25) +
(Relevance × 0.35) + (Conversion × 0.15)
```
**Score Interpretation:**
| Score Range | Priority | Action |
|-------------|----------|--------|
| 80-100 | Primary | Target in title and keyword field |
| 60-79 | Secondary | Include in subtitle/description |
| 40-59 | Tertiary | Use in long description only |
| 0-39 | Deprioritize | Do not target |
### Keyword Evaluation Worksheet
```
KEYWORD EVALUATION
Keyword: "task manager app"
Date: [Date]
SCORES:
├── Search Volume: 72/100 (High - ~25,000/month)
├── Competition: 45/100 (Medium - 4.4 avg rating in top 10)
├── Relevance: 95/100 (Exact match to core function)
└── Conversion: 85/100 (Transactional intent)
COMPOSITE SCORE: 74.5/100
RECOMMENDATION: Secondary Priority
- Include in subtitle or short description
- Not competitive enough for title (dominated by Todoist, Any.do)
- Consider long-tail variant: "simple task manager app"
```
---
## Competitor Keyword Analysis
### Competitor Identification
**Step 1: Direct Competitors**
Apps solving the same problem for the same audience.
**Step 2: Indirect Competitors**
Apps solving related problems or targeting overlapping audiences.
**Step 3: Category Leaders**
Top 10-20 apps by downloads in primary category.
### Competitor Keyword Extraction
**From App Title:**
```
Competitor: "Todoist: To-Do List & Tasks"
Keywords: todoist, to-do list, tasks, to do
```
**From Subtitle (iOS):**
```
Competitor subtitle: "Task Manager & Planner"
Keywords: task manager, planner
```
**From Description (First 100 words):**
Identify frequently used terms:
```
"Todoist is the world's favorite task manager and to-do list app.
Organize work and life, hit your goals, and find productivity..."
Extracted: task manager, to-do list, organize, goals, productivity
```
### Competitor Keyword Matrix
| Keyword | Comp 1 | Comp 2 | Comp 3 | Comp 4 | Comp 5 | Coverage |
|---------|--------|--------|--------|--------|--------|----------|
| task manager | ✓ | ✓ | ✓ | ✓ | ✓ | 100% |
| to-do list | ✓ | ✓ | ✓ | ✓ | | 80% |
| planner | ✓ | ✓ | | ✓ | ✓ | 80% |
| reminder | ✓ | ✓ | ✓ | | | 60% |
| productivity | ✓ | | ✓ | ✓ | | 60% |
| checklist | | ✓ | | ✓ | ✓ | 60% |
| project | ✓ | ✓ | | | | 40% |
| habit | | | ✓ | | ✓ | 40% |
**Analysis:**
- 100% coverage = Highly competitive, essential keyword
- 60-80% coverage = Important category term
- 40% coverage = Potential differentiator
- <40% coverage = Unique opportunity or irrelevant
### Keyword Gap Analysis
Identify keywords competitors miss:
```
KEYWORD GAP ANALYSIS
Underserved Keywords (Low competitor coverage, decent volume):
1. "daily planner widget" - 2/10 competitors, 5,000 searches
2. "task list for teams" - 3/10 competitors, 3,500 searches
3. "todo with calendar sync" - 1/10 competitors, 2,800 searches
Opportunity Assessment:
- "daily planner widget" → Add widget feature, target keyword
- "task list for teams" → Already have feature, update metadata
- "todo with calendar sync" → Feature gap, add to roadmap
```
---
## Keyword Mapping Strategy
### Keyword Placement Map
Assign each keyword to specific metadata locations:
```
KEYWORD PLACEMENT MAP
PRIMARY (Title + Keyword Field):
├── task manager (Score: 82)
├── todo list (Score: 78)
└── planner (Score: 75)
SECONDARY (Subtitle + Short Description):
├── reminder app (Score: 68)
├── daily tasks (Score: 65)
└── organize (Score: 62)
TERTIARY (Full Description):
├── checklist (Score: 55)
├── productivity (Score: 52)
├── schedule (Score: 48)
├── deadline (Score: 45)
└── project management (Score: 42)
```
### iOS Keyword Field Strategy
**100 Character Optimization:**
```
STEP 1: List all target keywords
task,manager,todo,list,planner,reminder,organize,daily,checklist,
productivity,schedule,deadline,project,goals,habit,widget,sync,
team,collaborate,notes,calendar
STEP 2: Remove duplicates from title
Title: "TaskFlow - Todo List Manager"
Remove: task, todo, list, manager
STEP 3: Remove plurals
Keep: reminder (not reminders)
Keep: goal (not goals)
STEP 4: Prioritize by score and fit
Final 100 chars:
planner,reminder,organize,daily,checklist,productivity,schedule,
deadline,project,goals,habit,widget,sync,team,collaborate
Character count: 98/100
```
### Android Description Keyword Integration
**Natural keyword placement in 4,000 characters:**
```
PARAGRAPH 1 (Hook - 300 chars):
Keywords: task manager, todo list, organize
"TaskFlow is the task manager trusted by 2 million users. Create
your perfect todo list and organize everything that matters..."
PARAGRAPH 2 (Features - 800 chars):
Keywords: reminder, checklist, deadline, project
"Set smart reminders that notify you at the right time. Build
checklists for any project. Never miss a deadline with..."
PARAGRAPH 3 (Benefits - 600 chars):
Keywords: productivity, schedule, goals
"Boost your productivity with proven planning methods. Schedule
your day in minutes. Track goals and celebrate..."
PARAGRAPH 4 (Differentiators - 500 chars):
Keywords: widget, sync, team, collaborate
"Beautiful widgets keep tasks visible. Sync across all devices
instantly. Invite your team to collaborate on..."
Total keyword coverage: 14 keywords naturally integrated
```
---
## Keyword Tracking and Iteration
### Ranking Tracking Cadence
| Frequency | Action |
|-----------|--------|
| Daily | Track top 5-10 primary keywords |
| Weekly | Full keyword set review |
| Monthly | Competitor keyword comparison |
| Quarterly | Full keyword research refresh |
### Keyword Performance Metrics
| Metric | Target | Action if Below |
|--------|--------|-----------------|
| Top 10 ranking | 3+ keywords | Increase keyword weight |
| Top 50 ranking | 10+ keywords | Maintain current strategy |
| Ranking velocity | Improving trend | Continue optimization |
| Conversion rate | >5% | Review relevance alignment |
### Iteration Process
**Monthly Keyword Audit:**
```
1. EXPORT current rankings
- List all tracked keywords
- Record current position
- Note 30-day trend (up/down/stable)
2. IDENTIFY opportunities
- Keywords improving but not top 10
- Keywords declining from previous position
- New high-volume keywords in category
3. PRIORITIZE changes
- Boost: Keywords at position 11-20
- Maintain: Keywords at position 1-10
- Replace: Keywords at position 50+ with no improvement
4. IMPLEMENT updates
- Adjust keyword field (iOS)
- Update description (Android)
- Modify subtitle if needed
5. DOCUMENT changes
- Record what changed and why
- Set reminder for 2-week check-in
```
### Keyword Testing Log Template
```
KEYWORD TEST LOG
Test ID: KW-2025-001
Date Started: [Date]
Keywords Changed:
- Added: "habit tracker" (replacing "goals app")
- Added: "daily routine" (replacing "schedule planner")
Rationale:
- "habit tracker" has 3x volume of "goals app"
- "daily routine" trending up 40% in category
Baseline Rankings:
- "habit tracker": Not ranked
- "daily routine": Position 87
30-Day Results:
- "habit tracker": Position 34 (+53)
- "daily routine": Position 28 (+59)
Conclusion: Test successful - retain new keywords
Next Action: Target subtitle position for "habit tracker"
```
FILE:references/platform-requirements.md
# Platform Requirements Reference
Technical specifications and metadata requirements for Apple App Store and Google Play Store.
---
## Table of Contents
- [Apple App Store Requirements](#apple-app-store-requirements)
- [Google Play Store Requirements](#google-play-store-requirements)
- [Visual Asset Specifications](#visual-asset-specifications)
- [Localization Requirements](#localization-requirements)
- [Compliance Guidelines](#compliance-guidelines)
---
## Apple App Store Requirements
### Metadata Character Limits
| Field | Character Limit | Notes |
|-------|----------------|-------|
| App Name (Title) | 30 characters | Visible in search results |
| Subtitle | 30 characters | iOS 11+ only, appears below title |
| Promotional Text | 170 characters | Editable without app update |
| Description | 4,000 characters | Not indexed for search |
| Keywords Field | 100 characters | Comma-separated, no spaces after commas |
| What's New | 4,000 characters | Release notes for updates |
| Developer Name | 255 characters | Company or individual name |
| Support URL | Required | Must be valid HTTPS URL |
| Privacy Policy URL | Required | Must be valid HTTPS URL |
### Keyword Field Optimization Rules
1. **No duplicates** - Words in title are already indexed
2. **No plurals** - Apple indexes both singular and plural forms
3. **No spaces after commas** - Wastes character space
4. **No brand names** - Violates App Store guidelines
5. **No category names** - Already indexed via category selection
**Example - Efficient keyword field:**
```
task,todo,checklist,reminder,productivity,organize,schedule,planner,goals,habit
```
**Example - Inefficient keyword field (avoid):**
```
task manager, todo list, productivity app, task tracking
```
### App Store Connect Metadata Fields
| Category | Field | Required |
|----------|-------|----------|
| **App Information** | Name | Yes |
| | Subtitle | No |
| | Category | Yes |
| | Secondary Category | No |
| | Content Rights | Yes |
| | Age Rating | Yes |
| **Version Information** | Description | Yes |
| | Keywords | Yes |
| | Promotional Text | No |
| | What's New | Yes (for updates) |
| | Support URL | Yes |
| | Marketing URL | No |
| **Pricing** | Price Tier | Yes |
| | Availability | Yes |
### Age Rating Content Descriptors
| Content Type | None | Infrequent | Frequent |
|--------------|------|------------|----------|
| Cartoon Violence | 4+ | 9+ | 12+ |
| Realistic Violence | 9+ | 12+ | 17+ |
| Sexual Content | 12+ | 17+ | 17+ |
| Profanity | 4+ | 12+ | 17+ |
| Alcohol/Drug Reference | 12+ | 17+ | 17+ |
| Gambling | 12+ | 17+ | 17+ |
| Horror/Fear | 9+ | 12+ | 17+ |
---
## Google Play Store Requirements
### Metadata Character Limits
| Field | Character Limit | Notes |
|-------|----------------|-------|
| App Title | 50 characters | Increased from 30 in 2021 |
| Short Description | 80 characters | Visible on store listing |
| Full Description | 4,000 characters | Indexed for search keywords |
| Developer Name | 64 characters | Organization or individual |
| Developer Email | Required | Public support contact |
| Privacy Policy URL | Required | Must be valid HTTPS URL |
### Description Keyword Strategy
Google Play has no separate keyword field. Keywords are extracted from:
1. **App Title** - Highest weight, most important
2. **Short Description** - High weight, visible in search
3. **Full Description** - Medium weight, use naturally throughout
4. **Developer Name** - Low weight but indexed
**Keyword Density Guidelines:**
- Primary keyword: 2-3% density in full description
- Secondary keywords: 1-2% each
- Avoid keyword stuffing (>5% triggers spam detection)
### Google Play Console Metadata
| Category | Field | Required |
|----------|-------|----------|
| **Store Listing** | Title | Yes |
| | Short Description | Yes |
| | Full Description | Yes |
| | App Icon | Yes |
| | Feature Graphic | Yes |
| | Screenshots | Yes (min 2) |
| | Video | No |
| **Store Settings** | App Category | Yes |
| | Tags | No |
| | Contact Email | Yes |
| | Privacy Policy | Yes |
| **Content Rating** | IARC Questionnaire | Yes |
### Content Rating (IARC)
| Rating | Age | Description |
|--------|-----|-------------|
| PEGI 3 / Everyone | 3+ | Suitable for all ages |
| PEGI 7 / Everyone 10+ | 7+ | Mild violence, comic mischief |
| PEGI 12 / Teen | 12+ | Moderate violence, mild language |
| PEGI 16 / Mature 17+ | 16+ | Intense violence, strong language |
| PEGI 18 / Adults Only | 18+ | Extreme content |
---
## Visual Asset Specifications
### App Icon Requirements
**Apple App Store:**
| Device | Size | Format |
|--------|------|--------|
| iPhone | 1024x1024 px | PNG, no alpha |
| iPad | 1024x1024 px | PNG, no alpha |
| App Store | 1024x1024 px | PNG, no alpha |
| Spotlight | 120x120 px | PNG |
| Settings | 87x87 px | PNG |
**Google Play Store:**
| Asset | Size | Format |
|-------|------|--------|
| App Icon | 512x512 px | PNG, 32-bit |
| Feature Graphic | 1024x500 px | PNG or JPG |
| Promo Graphic | 180x120 px | PNG or JPG |
| TV Banner | 1280x720 px | PNG or JPG |
### Screenshot Requirements
**Apple App Store:**
| Device | Portrait | Landscape |
|--------|----------|-----------|
| iPhone 6.9" | 1320x2868 px | 2868x1320 px |
| iPhone 6.5" | 1290x2796 px | 2796x1290 px |
| iPhone 5.5" | 1242x2208 px | 2208x1242 px |
| iPad Pro 12.9" | 2048x2732 px | 2732x2048 px |
| iPad 10.5" | 1668x2224 px | 2224x1668 px |
- Minimum: 2 screenshots per device
- Maximum: 10 screenshots per device
- Format: PNG or JPG, no alpha channel
- First 3 screenshots are critical (most users don't scroll)
**Google Play Store:**
| Device | Dimensions | Notes |
|--------|------------|-------|
| Phone | 320-3840 px | Min 2:1 aspect ratio |
| 7" Tablet | 320-3840 px | Min 2:1 aspect ratio |
| 10" Tablet | 320-3840 px | Min 2:1 aspect ratio |
| Chromebook | 320-3840 px | Optional |
| TV | 320-3840 px | For TV apps only |
- Minimum: 2 screenshots
- Maximum: 8 screenshots
- Format: PNG or JPG
- No transparency or borders
### App Preview Video
**Apple App Store:**
- Duration: 15-30 seconds
- Resolution: Match device screenshot size
- Format: M4V, MP4, MOV
- Frame rate: 30 fps
- Audio: Optional but recommended
**Google Play Store:**
- YouTube video link only
- No duration limit (recommend under 2 minutes)
- Landscape orientation preferred
- Must not contain age-restricted content
---
## Localization Requirements
### Priority Markets by Revenue
| Rank | Market | Language Code |
|------|--------|---------------|
| 1 | United States | en-US |
| 2 | Japan | ja |
| 3 | United Kingdom | en-GB |
| 4 | Germany | de-DE |
| 5 | China | zh-Hans (iOS), zh-CN (Android) |
| 6 | South Korea | ko |
| 7 | France | fr-FR |
| 8 | Canada | en-CA, fr-CA |
| 9 | Australia | en-AU |
| 10 | Russia | ru |
### Apple App Store Localization
Supported localizations: 40+ languages
| Language | Locale Code |
|----------|-------------|
| English (US) | en-US |
| English (UK) | en-GB |
| Spanish | es-ES |
| Spanish (Mexico) | es-MX |
| French | fr-FR |
| German | de-DE |
| Japanese | ja |
| Korean | ko |
| Simplified Chinese | zh-Hans |
| Traditional Chinese | zh-Hant |
### Google Play Store Localization
Supported localizations: 75+ languages
Each locale requires:
- Title (50 chars)
- Short description (80 chars)
- Full description (4,000 chars)
- Screenshots (can reuse or localize)
---
## Compliance Guidelines
### Apple App Store Review Guidelines Summary
| Category | Key Requirements |
|----------|------------------|
| **Safety** | No objectionable content, privacy protection |
| **Performance** | App must work as described, no crashes |
| **Business** | Accurate app description, clear pricing |
| **Design** | Follow Human Interface Guidelines |
| **Legal** | Comply with local laws, proper licensing |
**Common Rejection Reasons:**
1. Bugs and crashes (50%+ of rejections)
2. Broken links or placeholder content
3. Misleading app descriptions
4. Privacy policy missing or incomplete
5. In-app purchase issues
### Google Play Developer Policies
| Policy Area | Requirements |
|-------------|--------------|
| **Restricted Content** | No hate speech, violence, gambling (without license) |
| **Privacy** | Data collection disclosure, privacy policy |
| **Monetization** | Clear pricing, compliant IAPs |
| **Ads** | No deceptive ads, proper disclosure |
| **Store Listing** | Accurate description, no keyword stuffing |
**Common Suspension Reasons:**
1. Policy violation (content, ads, permissions)
2. Repetitive content (clone apps)
3. Impersonation (fake apps)
4. Intellectual property infringement
5. Malicious behavior
### Privacy Requirements
**Apple (App Tracking Transparency):**
- ATT prompt required for tracking
- Privacy nutrition labels mandatory
- Data collection disclosure required
**Google (Data Safety):**
- Data safety section mandatory
- Data collection and sharing disclosure
- Security practices declaration
---
## Quick Reference Card
### Apple vs Google Comparison
| Attribute | Apple App Store | Google Play Store |
|-----------|-----------------|-------------------|
| Title Length | 30 chars | 50 chars |
| Subtitle | 30 chars | N/A |
| Short Description | N/A | 80 chars |
| Full Description | 4,000 chars | 4,000 chars |
| Keywords Field | 100 chars | N/A (in description) |
| Promotional Text | 170 chars | N/A |
| Icon Size | 1024x1024 px | 512x512 px |
| Min Screenshots | 2 | 2 |
| Max Screenshots | 10 | 8 |
| Review Time | 24-48 hours | 1-7 days |
| Metadata Update | Requires review | 1-2 hours to index |
FILE:sample_input.json
{
"request_type": "keyword_research",
"app_info": {
"name": "TaskFlow Pro",
"category": "Productivity",
"target_audience": "Professionals aged 25-45 working in teams",
"key_features": [
"AI-powered task prioritization",
"Team collaboration tools",
"Calendar integration",
"Cross-platform sync"
],
"unique_value": "AI automatically prioritizes your tasks based on deadlines and importance"
},
"target_keywords": [
"task manager",
"productivity app",
"todo list",
"team collaboration",
"project management"
],
"competitors": [
"Todoist",
"Any.do",
"Microsoft To Do",
"Things 3"
],
"platform": "both",
"language": "en-US"
}
FILE:scripts/ab_test_planner.py
"""
A/B testing module for App Store Optimization.
Plans and tracks A/B tests for metadata and visual assets.
"""
from typing import Dict, List, Any, Optional
import math
class ABTestPlanner:
"""Plans and tracks A/B tests for ASO elements."""
# Minimum detectable effect sizes (conservative estimates)
MIN_EFFECT_SIZES = {
'icon': 0.10, # 10% conversion improvement
'screenshot': 0.08, # 8% conversion improvement
'title': 0.05, # 5% conversion improvement
'description': 0.03 # 3% conversion improvement
}
# Statistical confidence levels
CONFIDENCE_LEVELS = {
'high': 0.95, # 95% confidence
'standard': 0.90, # 90% confidence
'exploratory': 0.80 # 80% confidence
}
def __init__(self):
"""Initialize A/B test planner."""
self.active_tests = []
def design_test(
self,
test_type: str,
variant_a: Dict[str, Any],
variant_b: Dict[str, Any],
hypothesis: str,
success_metric: str = 'conversion_rate'
) -> Dict[str, Any]:
"""
Design an A/B test with hypothesis and variables.
Args:
test_type: Type of test ('icon', 'screenshot', 'title', 'description')
variant_a: Control variant details
variant_b: Test variant details
hypothesis: Expected outcome hypothesis
success_metric: Metric to optimize
Returns:
Test design with configuration
"""
test_design = {
'test_id': self._generate_test_id(test_type),
'test_type': test_type,
'hypothesis': hypothesis,
'variants': {
'a': {
'name': 'Control',
'details': variant_a,
'traffic_split': 0.5
},
'b': {
'name': 'Variation',
'details': variant_b,
'traffic_split': 0.5
}
},
'success_metric': success_metric,
'secondary_metrics': self._get_secondary_metrics(test_type),
'minimum_effect_size': self.MIN_EFFECT_SIZES.get(test_type, 0.05),
'recommended_confidence': 'standard',
'best_practices': self._get_test_best_practices(test_type)
}
self.active_tests.append(test_design)
return test_design
def calculate_sample_size(
self,
baseline_conversion: float,
minimum_detectable_effect: float,
confidence_level: str = 'standard',
power: float = 0.80
) -> Dict[str, Any]:
"""
Calculate required sample size for statistical significance.
Args:
baseline_conversion: Current conversion rate (0-1)
minimum_detectable_effect: Minimum effect size to detect (0-1)
confidence_level: 'high', 'standard', or 'exploratory'
power: Statistical power (typically 0.80 or 0.90)
Returns:
Sample size calculation with duration estimates
"""
alpha = 1 - self.CONFIDENCE_LEVELS[confidence_level]
beta = 1 - power
# Expected conversion for variant B
expected_conversion_b = baseline_conversion * (1 + minimum_detectable_effect)
# Z-scores for alpha and beta
z_alpha = self._get_z_score(1 - alpha / 2) # Two-tailed test
z_beta = self._get_z_score(power)
# Pooled standard deviation
p_pooled = (baseline_conversion + expected_conversion_b) / 2
sd_pooled = math.sqrt(2 * p_pooled * (1 - p_pooled))
# Sample size per variant
n_per_variant = math.ceil(
((z_alpha + z_beta) ** 2 * sd_pooled ** 2) /
((expected_conversion_b - baseline_conversion) ** 2)
)
total_sample_size = n_per_variant * 2
# Estimate duration based on typical traffic
duration_estimates = self._estimate_test_duration(
total_sample_size,
baseline_conversion
)
return {
'sample_size_per_variant': n_per_variant,
'total_sample_size': total_sample_size,
'baseline_conversion': baseline_conversion,
'expected_conversion_improvement': minimum_detectable_effect,
'expected_conversion_b': expected_conversion_b,
'confidence_level': confidence_level,
'statistical_power': power,
'duration_estimates': duration_estimates,
'recommendations': self._generate_sample_size_recommendations(
n_per_variant,
duration_estimates
)
}
def calculate_significance(
self,
variant_a_conversions: int,
variant_a_visitors: int,
variant_b_conversions: int,
variant_b_visitors: int
) -> Dict[str, Any]:
"""
Calculate statistical significance of test results.
Args:
variant_a_conversions: Conversions for control
variant_a_visitors: Visitors for control
variant_b_conversions: Conversions for variation
variant_b_visitors: Visitors for variation
Returns:
Significance analysis with decision recommendation
"""
# Calculate conversion rates
rate_a = variant_a_conversions / variant_a_visitors if variant_a_visitors > 0 else 0
rate_b = variant_b_conversions / variant_b_visitors if variant_b_visitors > 0 else 0
# Calculate improvement
if rate_a > 0:
relative_improvement = (rate_b - rate_a) / rate_a
else:
relative_improvement = 0
absolute_improvement = rate_b - rate_a
# Calculate standard error
se_a = math.sqrt(rate_a * (1 - rate_a) / variant_a_visitors) if variant_a_visitors > 0 else 0
se_b = math.sqrt(rate_b * (1 - rate_b) / variant_b_visitors) if variant_b_visitors > 0 else 0
se_diff = math.sqrt(se_a**2 + se_b**2)
# Calculate z-score
z_score = absolute_improvement / se_diff if se_diff > 0 else 0
# Calculate p-value (two-tailed)
p_value = 2 * (1 - self._standard_normal_cdf(abs(z_score)))
# Determine significance
is_significant_95 = p_value < 0.05
is_significant_90 = p_value < 0.10
# Generate decision
decision = self._generate_test_decision(
relative_improvement,
is_significant_95,
is_significant_90,
variant_a_visitors + variant_b_visitors
)
return {
'variant_a': {
'conversions': variant_a_conversions,
'visitors': variant_a_visitors,
'conversion_rate': round(rate_a, 4)
},
'variant_b': {
'conversions': variant_b_conversions,
'visitors': variant_b_visitors,
'conversion_rate': round(rate_b, 4)
},
'improvement': {
'absolute': round(absolute_improvement, 4),
'relative_percentage': round(relative_improvement * 100, 2)
},
'statistical_analysis': {
'z_score': round(z_score, 3),
'p_value': round(p_value, 4),
'is_significant_95': is_significant_95,
'is_significant_90': is_significant_90,
'confidence_level': '95%' if is_significant_95 else ('90%' if is_significant_90 else 'Not significant')
},
'decision': decision
}
def track_test_results(
self,
test_id: str,
results_data: Dict[str, Any]
) -> Dict[str, Any]:
"""
Track ongoing test results and provide recommendations.
Args:
test_id: Test identifier
results_data: Current test results
Returns:
Test tracking report with next steps
"""
# Find test
test = next((t for t in self.active_tests if t['test_id'] == test_id), None)
if not test:
return {'error': f'Test {test_id} not found'}
# Calculate significance
significance = self.calculate_significance(
results_data['variant_a_conversions'],
results_data['variant_a_visitors'],
results_data['variant_b_conversions'],
results_data['variant_b_visitors']
)
# Calculate test progress
total_visitors = results_data['variant_a_visitors'] + results_data['variant_b_visitors']
required_sample = results_data.get('required_sample_size', 10000)
progress_percentage = min((total_visitors / required_sample) * 100, 100)
# Generate recommendations
recommendations = self._generate_tracking_recommendations(
significance,
progress_percentage,
test['test_type']
)
return {
'test_id': test_id,
'test_type': test['test_type'],
'progress': {
'total_visitors': total_visitors,
'required_sample_size': required_sample,
'progress_percentage': round(progress_percentage, 1),
'is_complete': progress_percentage >= 100
},
'current_results': significance,
'recommendations': recommendations,
'next_steps': self._determine_next_steps(
significance,
progress_percentage
)
}
def generate_test_report(
self,
test_id: str,
final_results: Dict[str, Any]
) -> Dict[str, Any]:
"""
Generate final test report with insights and recommendations.
Args:
test_id: Test identifier
final_results: Final test results
Returns:
Comprehensive test report
"""
test = next((t for t in self.active_tests if t['test_id'] == test_id), None)
if not test:
return {'error': f'Test {test_id} not found'}
significance = self.calculate_significance(
final_results['variant_a_conversions'],
final_results['variant_a_visitors'],
final_results['variant_b_conversions'],
final_results['variant_b_visitors']
)
# Generate insights
insights = self._generate_test_insights(
test,
significance,
final_results
)
# Implementation plan
implementation_plan = self._create_implementation_plan(
test,
significance
)
return {
'test_summary': {
'test_id': test_id,
'test_type': test['test_type'],
'hypothesis': test['hypothesis'],
'duration_days': final_results.get('duration_days', 'N/A')
},
'results': significance,
'insights': insights,
'implementation_plan': implementation_plan,
'learnings': self._extract_learnings(test, significance)
}
def _generate_test_id(self, test_type: str) -> str:
"""Generate unique test ID."""
import time
timestamp = int(time.time())
return f"{test_type}_{timestamp}"
def _get_secondary_metrics(self, test_type: str) -> List[str]:
"""Get secondary metrics to track for test type."""
metrics_map = {
'icon': ['tap_through_rate', 'impression_count', 'brand_recall'],
'screenshot': ['tap_through_rate', 'time_on_page', 'scroll_depth'],
'title': ['impression_count', 'tap_through_rate', 'search_visibility'],
'description': ['time_on_page', 'scroll_depth', 'tap_through_rate']
}
return metrics_map.get(test_type, ['tap_through_rate'])
def _get_test_best_practices(self, test_type: str) -> List[str]:
"""Get best practices for specific test type."""
practices_map = {
'icon': [
'Test only one element at a time (color vs. style vs. symbolism)',
'Ensure icon is recognizable at small sizes (60x60px)',
'Consider cultural context for global audience',
'Test against top competitor icons'
],
'screenshot': [
'Test order of screenshots (users see first 2-3)',
'Use captions to tell story',
'Show key features and benefits',
'Test with and without device frames'
],
'title': [
'Test keyword variations, not major rebrand',
'Keep brand name consistent',
'Ensure title fits within character limits',
'Test on both search and browse contexts'
],
'description': [
'Test structure (bullet points vs. paragraphs)',
'Test call-to-action placement',
'Test feature vs. benefit focus',
'Maintain keyword density'
]
}
return practices_map.get(test_type, ['Test one variable at a time'])
def _estimate_test_duration(
self,
required_sample_size: int,
baseline_conversion: float
) -> Dict[str, Any]:
"""Estimate test duration based on typical traffic levels."""
# Assume different daily traffic scenarios
traffic_scenarios = {
'low': 100, # 100 page views/day
'medium': 1000, # 1000 page views/day
'high': 10000 # 10000 page views/day
}
estimates = {}
for scenario, daily_views in traffic_scenarios.items():
days = math.ceil(required_sample_size / daily_views)
estimates[scenario] = {
'daily_page_views': daily_views,
'estimated_days': days,
'estimated_weeks': round(days / 7, 1)
}
return estimates
def _generate_sample_size_recommendations(
self,
sample_size: int,
duration_estimates: Dict[str, Any]
) -> List[str]:
"""Generate recommendations based on sample size."""
recommendations = []
if sample_size > 50000:
recommendations.append(
"Large sample size required - consider testing smaller effect size or increasing traffic"
)
if duration_estimates['medium']['estimated_days'] > 30:
recommendations.append(
"Long test duration - consider higher minimum detectable effect or focus on high-impact changes"
)
if duration_estimates['low']['estimated_days'] > 60:
recommendations.append(
"Insufficient traffic for reliable testing - consider user acquisition or broader targeting"
)
if not recommendations:
recommendations.append("Sample size and duration are reasonable for this test")
return recommendations
def _get_z_score(self, percentile: float) -> float:
"""Get z-score for given percentile (approximation)."""
# Common z-scores
z_scores = {
0.80: 0.84,
0.85: 1.04,
0.90: 1.28,
0.95: 1.645,
0.975: 1.96,
0.99: 2.33
}
return z_scores.get(percentile, 1.96)
def _standard_normal_cdf(self, z: float) -> float:
"""Approximate standard normal cumulative distribution function."""
# Using error function approximation
t = 1.0 / (1.0 + 0.2316419 * abs(z))
d = 0.3989423 * math.exp(-z * z / 2.0)
p = d * t * (0.3193815 + t * (-0.3565638 + t * (1.781478 + t * (-1.821256 + t * 1.330274))))
if z > 0:
return 1.0 - p
else:
return p
def _generate_test_decision(
self,
improvement: float,
is_significant_95: bool,
is_significant_90: bool,
total_visitors: int
) -> Dict[str, Any]:
"""Generate test decision and recommendation."""
if total_visitors < 1000:
return {
'decision': 'continue',
'rationale': 'Insufficient data - continue test to reach minimum sample size',
'action': 'Keep test running'
}
if is_significant_95:
if improvement > 0:
return {
'decision': 'implement_b',
'rationale': f'Variant B shows {improvement*100:.1f}% improvement with 95% confidence',
'action': 'Implement Variant B'
}
else:
return {
'decision': 'keep_a',
'rationale': 'Variant A performs better with 95% confidence',
'action': 'Keep current version (A)'
}
elif is_significant_90:
if improvement > 0:
return {
'decision': 'implement_b_cautiously',
'rationale': f'Variant B shows {improvement*100:.1f}% improvement with 90% confidence',
'action': 'Consider implementing B, monitor closely'
}
else:
return {
'decision': 'keep_a',
'rationale': 'Variant A performs better with 90% confidence',
'action': 'Keep current version (A)'
}
else:
return {
'decision': 'inconclusive',
'rationale': 'No statistically significant difference detected',
'action': 'Either keep A or test different hypothesis'
}
def _generate_tracking_recommendations(
self,
significance: Dict[str, Any],
progress: float,
test_type: str
) -> List[str]:
"""Generate recommendations for ongoing test."""
recommendations = []
if progress < 50:
recommendations.append(
f"Test is {progress:.0f}% complete - continue collecting data"
)
if progress >= 100:
if significance['statistical_analysis']['is_significant_95']:
recommendations.append(
"Sufficient data collected with significant results - ready to conclude test"
)
else:
recommendations.append(
"Sample size reached but no significant difference - consider extending test or concluding"
)
return recommendations
def _determine_next_steps(
self,
significance: Dict[str, Any],
progress: float
) -> str:
"""Determine next steps for test."""
if progress < 100:
return f"Continue test until reaching 100% sample size (currently {progress:.0f}%)"
decision = significance.get('decision', {}).get('decision', 'inconclusive')
if decision == 'implement_b':
return "Implement Variant B and monitor metrics for 2 weeks"
elif decision == 'keep_a':
return "Keep Variant A and design new test with different hypothesis"
else:
return "Test inconclusive - either keep A or design new test"
def _generate_test_insights(
self,
test: Dict[str, Any],
significance: Dict[str, Any],
results: Dict[str, Any]
) -> List[str]:
"""Generate insights from test results."""
insights = []
improvement = significance['improvement']['relative_percentage']
if significance['statistical_analysis']['is_significant_95']:
insights.append(
f"Strong evidence: Variant B {'improved' if improvement > 0 else 'decreased'} "
f"conversion by {abs(improvement):.1f}% with 95% confidence"
)
insights.append(
f"Tested {test['test_type']} changes: {test['hypothesis']}"
)
# Add context-specific insights
if test['test_type'] == 'icon' and improvement > 5:
insights.append(
"Icon change had substantial impact - visual first impression is critical"
)
return insights
def _create_implementation_plan(
self,
test: Dict[str, Any],
significance: Dict[str, Any]
) -> List[Dict[str, str]]:
"""Create implementation plan for winning variant."""
plan = []
if significance.get('decision', {}).get('decision') == 'implement_b':
plan.append({
'step': '1. Update store listing',
'details': f"Replace {test['test_type']} with Variant B across all platforms"
})
plan.append({
'step': '2. Monitor metrics',
'details': 'Track conversion rate for 2 weeks to confirm sustained improvement'
})
plan.append({
'step': '3. Document learnings',
'details': 'Record insights for future optimization'
})
return plan
def _extract_learnings(
self,
test: Dict[str, Any],
significance: Dict[str, Any]
) -> List[str]:
"""Extract key learnings from test."""
learnings = []
improvement = significance['improvement']['relative_percentage']
learnings.append(
f"Testing {test['test_type']} can yield {abs(improvement):.1f}% conversion change"
)
if test['test_type'] == 'title':
learnings.append(
"Title changes affect search visibility and user perception"
)
elif test['test_type'] == 'screenshot':
learnings.append(
"First 2-3 screenshots are critical for conversion"
)
return learnings
def plan_ab_test(
test_type: str,
variant_a: Dict[str, Any],
variant_b: Dict[str, Any],
hypothesis: str,
baseline_conversion: float
) -> Dict[str, Any]:
"""
Convenience function to plan an A/B test.
Args:
test_type: Type of test
variant_a: Control variant
variant_b: Test variant
hypothesis: Test hypothesis
baseline_conversion: Current conversion rate
Returns:
Complete test plan
"""
planner = ABTestPlanner()
test_design = planner.design_test(
test_type,
variant_a,
variant_b,
hypothesis
)
sample_size = planner.calculate_sample_size(
baseline_conversion,
planner.MIN_EFFECT_SIZES.get(test_type, 0.05)
)
return {
'test_design': test_design,
'sample_size_requirements': sample_size
}
FILE:scripts/aso_scorer.py
"""
ASO scoring module for App Store Optimization.
Calculates comprehensive ASO health score across multiple dimensions.
"""
from typing import Dict, List, Any, Optional
class ASOScorer:
"""Calculates overall ASO health score and provides recommendations."""
# Score weights for different components (total = 100)
WEIGHTS = {
'metadata_quality': 25,
'ratings_reviews': 25,
'keyword_performance': 25,
'conversion_metrics': 25
}
# Benchmarks for scoring
BENCHMARKS = {
'title_keyword_usage': {'min': 1, 'target': 2},
'description_length': {'min': 500, 'target': 2000},
'keyword_density': {'min': 2, 'optimal': 5, 'max': 8},
'average_rating': {'min': 3.5, 'target': 4.5},
'ratings_count': {'min': 100, 'target': 5000},
'keywords_top_10': {'min': 2, 'target': 10},
'keywords_top_50': {'min': 5, 'target': 20},
'conversion_rate': {'min': 0.02, 'target': 0.10}
}
def __init__(self):
"""Initialize ASO scorer."""
self.score_breakdown = {}
def calculate_overall_score(
self,
metadata: Dict[str, Any],
ratings: Dict[str, Any],
keyword_performance: Dict[str, Any],
conversion: Dict[str, Any]
) -> Dict[str, Any]:
"""
Calculate comprehensive ASO score (0-100).
Args:
metadata: Title, description quality metrics
ratings: Rating average and count
keyword_performance: Keyword ranking data
conversion: Impression-to-install metrics
Returns:
Overall score with detailed breakdown
"""
# Calculate component scores
metadata_score = self.score_metadata_quality(metadata)
ratings_score = self.score_ratings_reviews(ratings)
keyword_score = self.score_keyword_performance(keyword_performance)
conversion_score = self.score_conversion_metrics(conversion)
# Calculate weighted overall score
overall_score = (
metadata_score * (self.WEIGHTS['metadata_quality'] / 100) +
ratings_score * (self.WEIGHTS['ratings_reviews'] / 100) +
keyword_score * (self.WEIGHTS['keyword_performance'] / 100) +
conversion_score * (self.WEIGHTS['conversion_metrics'] / 100)
)
# Store breakdown
self.score_breakdown = {
'metadata_quality': {
'score': metadata_score,
'weight': self.WEIGHTS['metadata_quality'],
'weighted_contribution': round(metadata_score * (self.WEIGHTS['metadata_quality'] / 100), 1)
},
'ratings_reviews': {
'score': ratings_score,
'weight': self.WEIGHTS['ratings_reviews'],
'weighted_contribution': round(ratings_score * (self.WEIGHTS['ratings_reviews'] / 100), 1)
},
'keyword_performance': {
'score': keyword_score,
'weight': self.WEIGHTS['keyword_performance'],
'weighted_contribution': round(keyword_score * (self.WEIGHTS['keyword_performance'] / 100), 1)
},
'conversion_metrics': {
'score': conversion_score,
'weight': self.WEIGHTS['conversion_metrics'],
'weighted_contribution': round(conversion_score * (self.WEIGHTS['conversion_metrics'] / 100), 1)
}
}
# Generate recommendations
recommendations = self.generate_recommendations(
metadata_score,
ratings_score,
keyword_score,
conversion_score
)
# Assess overall health
health_status = self._assess_health_status(overall_score)
return {
'overall_score': round(overall_score, 1),
'health_status': health_status,
'score_breakdown': self.score_breakdown,
'recommendations': recommendations,
'priority_actions': self._prioritize_actions(recommendations),
'strengths': self._identify_strengths(self.score_breakdown),
'weaknesses': self._identify_weaknesses(self.score_breakdown)
}
def score_metadata_quality(self, metadata: Dict[str, Any]) -> float:
"""
Score metadata quality (0-100).
Evaluates:
- Title optimization
- Description quality
- Keyword usage
"""
scores = []
# Title score (0-35 points)
title_keywords = metadata.get('title_keyword_count', 0)
title_length = metadata.get('title_length', 0)
title_score = 0
if title_keywords >= self.BENCHMARKS['title_keyword_usage']['target']:
title_score = 35
elif title_keywords >= self.BENCHMARKS['title_keyword_usage']['min']:
title_score = 25
else:
title_score = 10
# Adjust for title length usage
if title_length > 25: # Using most of available space
title_score += 0
else:
title_score -= 5
scores.append(min(title_score, 35))
# Description score (0-35 points)
desc_length = metadata.get('description_length', 0)
desc_quality = metadata.get('description_quality', 0.0) # 0-1 scale
desc_score = 0
if desc_length >= self.BENCHMARKS['description_length']['target']:
desc_score = 25
elif desc_length >= self.BENCHMARKS['description_length']['min']:
desc_score = 15
else:
desc_score = 5
# Add quality bonus
desc_score += desc_quality * 10
scores.append(min(desc_score, 35))
# Keyword density score (0-30 points)
keyword_density = metadata.get('keyword_density', 0.0)
if self.BENCHMARKS['keyword_density']['min'] <= keyword_density <= self.BENCHMARKS['keyword_density']['optimal']:
density_score = 30
elif keyword_density < self.BENCHMARKS['keyword_density']['min']:
# Too low - proportional scoring
density_score = (keyword_density / self.BENCHMARKS['keyword_density']['min']) * 20
else:
# Too high (keyword stuffing) - penalty
excess = keyword_density - self.BENCHMARKS['keyword_density']['optimal']
density_score = max(30 - (excess * 5), 0)
scores.append(density_score)
return round(sum(scores), 1)
def score_ratings_reviews(self, ratings: Dict[str, Any]) -> float:
"""
Score ratings and reviews (0-100).
Evaluates:
- Average rating
- Total ratings count
- Review velocity
"""
average_rating = ratings.get('average_rating', 0.0)
total_ratings = ratings.get('total_ratings', 0)
recent_ratings = ratings.get('recent_ratings_30d', 0)
# Rating quality score (0-50 points)
if average_rating >= self.BENCHMARKS['average_rating']['target']:
rating_quality_score = 50
elif average_rating >= self.BENCHMARKS['average_rating']['min']:
# Proportional scoring between min and target
proportion = (average_rating - self.BENCHMARKS['average_rating']['min']) / \
(self.BENCHMARKS['average_rating']['target'] - self.BENCHMARKS['average_rating']['min'])
rating_quality_score = 30 + (proportion * 20)
elif average_rating >= 3.0:
rating_quality_score = 20
else:
rating_quality_score = 10
# Rating volume score (0-30 points)
if total_ratings >= self.BENCHMARKS['ratings_count']['target']:
rating_volume_score = 30
elif total_ratings >= self.BENCHMARKS['ratings_count']['min']:
# Proportional scoring
proportion = (total_ratings - self.BENCHMARKS['ratings_count']['min']) / \
(self.BENCHMARKS['ratings_count']['target'] - self.BENCHMARKS['ratings_count']['min'])
rating_volume_score = 15 + (proportion * 15)
else:
# Very low volume
rating_volume_score = (total_ratings / self.BENCHMARKS['ratings_count']['min']) * 15
# Rating velocity score (0-20 points)
if recent_ratings > 100:
velocity_score = 20
elif recent_ratings > 50:
velocity_score = 15
elif recent_ratings > 10:
velocity_score = 10
else:
velocity_score = 5
total_score = rating_quality_score + rating_volume_score + velocity_score
return round(min(total_score, 100), 1)
def score_keyword_performance(self, keyword_performance: Dict[str, Any]) -> float:
"""
Score keyword ranking performance (0-100).
Evaluates:
- Top 10 rankings
- Top 50 rankings
- Ranking trends
"""
top_10_count = keyword_performance.get('top_10', 0)
top_50_count = keyword_performance.get('top_50', 0)
top_100_count = keyword_performance.get('top_100', 0)
improving_keywords = keyword_performance.get('improving_keywords', 0)
# Top 10 score (0-50 points) - most valuable rankings
if top_10_count >= self.BENCHMARKS['keywords_top_10']['target']:
top_10_score = 50
elif top_10_count >= self.BENCHMARKS['keywords_top_10']['min']:
proportion = (top_10_count - self.BENCHMARKS['keywords_top_10']['min']) / \
(self.BENCHMARKS['keywords_top_10']['target'] - self.BENCHMARKS['keywords_top_10']['min'])
top_10_score = 25 + (proportion * 25)
else:
top_10_score = (top_10_count / self.BENCHMARKS['keywords_top_10']['min']) * 25
# Top 50 score (0-30 points)
if top_50_count >= self.BENCHMARKS['keywords_top_50']['target']:
top_50_score = 30
elif top_50_count >= self.BENCHMARKS['keywords_top_50']['min']:
proportion = (top_50_count - self.BENCHMARKS['keywords_top_50']['min']) / \
(self.BENCHMARKS['keywords_top_50']['target'] - self.BENCHMARKS['keywords_top_50']['min'])
top_50_score = 15 + (proportion * 15)
else:
top_50_score = (top_50_count / self.BENCHMARKS['keywords_top_50']['min']) * 15
# Coverage score (0-10 points) - based on top 100
coverage_score = min((top_100_count / 30) * 10, 10)
# Trend score (0-10 points) - are rankings improving?
if improving_keywords > 5:
trend_score = 10
elif improving_keywords > 0:
trend_score = 5
else:
trend_score = 0
total_score = top_10_score + top_50_score + coverage_score + trend_score
return round(min(total_score, 100), 1)
def score_conversion_metrics(self, conversion: Dict[str, Any]) -> float:
"""
Score conversion performance (0-100).
Evaluates:
- Impression-to-install conversion rate
- Download velocity
"""
conversion_rate = conversion.get('impression_to_install', 0.0)
downloads_30d = conversion.get('downloads_last_30_days', 0)
downloads_trend = conversion.get('downloads_trend', 'stable') # 'up', 'stable', 'down'
# Conversion rate score (0-70 points)
if conversion_rate >= self.BENCHMARKS['conversion_rate']['target']:
conversion_score = 70
elif conversion_rate >= self.BENCHMARKS['conversion_rate']['min']:
proportion = (conversion_rate - self.BENCHMARKS['conversion_rate']['min']) / \
(self.BENCHMARKS['conversion_rate']['target'] - self.BENCHMARKS['conversion_rate']['min'])
conversion_score = 35 + (proportion * 35)
else:
conversion_score = (conversion_rate / self.BENCHMARKS['conversion_rate']['min']) * 35
# Download velocity score (0-20 points)
if downloads_30d > 10000:
velocity_score = 20
elif downloads_30d > 1000:
velocity_score = 15
elif downloads_30d > 100:
velocity_score = 10
else:
velocity_score = 5
# Trend bonus (0-10 points)
if downloads_trend == 'up':
trend_score = 10
elif downloads_trend == 'stable':
trend_score = 5
else:
trend_score = 0
total_score = conversion_score + velocity_score + trend_score
return round(min(total_score, 100), 1)
def generate_recommendations(
self,
metadata_score: float,
ratings_score: float,
keyword_score: float,
conversion_score: float
) -> List[Dict[str, Any]]:
"""Generate prioritized recommendations based on scores."""
recommendations = []
# Metadata recommendations
if metadata_score < 60:
recommendations.append({
'category': 'metadata_quality',
'priority': 'high',
'action': 'Optimize app title and description',
'details': 'Add more keywords to title, expand description to 1500-2000 characters, improve keyword density to 3-5%',
'expected_impact': 'Improve discoverability and ranking potential'
})
elif metadata_score < 80:
recommendations.append({
'category': 'metadata_quality',
'priority': 'medium',
'action': 'Refine metadata for better keyword targeting',
'details': 'Test variations of title/subtitle, optimize keyword field for Apple',
'expected_impact': 'Incremental ranking improvements'
})
# Ratings recommendations
if ratings_score < 60:
recommendations.append({
'category': 'ratings_reviews',
'priority': 'high',
'action': 'Improve rating quality and volume',
'details': 'Address top user complaints, implement in-app rating prompts, respond to negative reviews',
'expected_impact': 'Better conversion rates and trust signals'
})
elif ratings_score < 80:
recommendations.append({
'category': 'ratings_reviews',
'priority': 'medium',
'action': 'Increase rating velocity',
'details': 'Optimize timing of rating requests, encourage satisfied users to rate',
'expected_impact': 'Sustained rating quality'
})
# Keyword performance recommendations
if keyword_score < 60:
recommendations.append({
'category': 'keyword_performance',
'priority': 'high',
'action': 'Improve keyword rankings',
'details': 'Target long-tail keywords with lower competition, update metadata with high-potential keywords, build backlinks',
'expected_impact': 'Significant improvement in organic visibility'
})
elif keyword_score < 80:
recommendations.append({
'category': 'keyword_performance',
'priority': 'medium',
'action': 'Expand keyword coverage',
'details': 'Target additional related keywords, test seasonal keywords, localize for new markets',
'expected_impact': 'Broader reach and more discovery opportunities'
})
# Conversion recommendations
if conversion_score < 60:
recommendations.append({
'category': 'conversion_metrics',
'priority': 'high',
'action': 'Optimize store listing for conversions',
'details': 'Improve screenshots and icon, strengthen value proposition in description, add video preview',
'expected_impact': 'Higher impression-to-install conversion'
})
elif conversion_score < 80:
recommendations.append({
'category': 'conversion_metrics',
'priority': 'medium',
'action': 'Test visual asset variations',
'details': 'A/B test different icon designs and screenshot sequences',
'expected_impact': 'Incremental conversion improvements'
})
return recommendations
def _assess_health_status(self, overall_score: float) -> str:
"""Assess overall ASO health status."""
if overall_score >= 80:
return "Excellent - Top-tier ASO performance"
elif overall_score >= 65:
return "Good - Competitive ASO with room for improvement"
elif overall_score >= 50:
return "Fair - Needs strategic improvements"
else:
return "Poor - Requires immediate ASO overhaul"
def _prioritize_actions(
self,
recommendations: List[Dict[str, Any]]
) -> List[Dict[str, Any]]:
"""Prioritize actions by impact and urgency."""
# Sort by priority (high first) and expected impact
priority_order = {'high': 0, 'medium': 1, 'low': 2}
sorted_recommendations = sorted(
recommendations,
key=lambda x: priority_order[x['priority']]
)
return sorted_recommendations[:3] # Top 3 priority actions
def _identify_strengths(self, score_breakdown: Dict[str, Any]) -> List[str]:
"""Identify areas of strength (scores >= 75)."""
strengths = []
for category, data in score_breakdown.items():
if data['score'] >= 75:
strengths.append(
f"{category.replace('_', ' ').title()}: {data['score']}/100"
)
return strengths if strengths else ["Focus on building strengths across all areas"]
def _identify_weaknesses(self, score_breakdown: Dict[str, Any]) -> List[str]:
"""Identify areas needing improvement (scores < 60)."""
weaknesses = []
for category, data in score_breakdown.items():
if data['score'] < 60:
weaknesses.append(
f"{category.replace('_', ' ').title()}: {data['score']}/100 - needs improvement"
)
return weaknesses if weaknesses else ["All areas performing adequately"]
def calculate_aso_score(
metadata: Dict[str, Any],
ratings: Dict[str, Any],
keyword_performance: Dict[str, Any],
conversion: Dict[str, Any]
) -> Dict[str, Any]:
"""
Convenience function to calculate ASO score.
Args:
metadata: Metadata quality metrics
ratings: Ratings data
keyword_performance: Keyword ranking data
conversion: Conversion metrics
Returns:
Complete ASO score report
"""
scorer = ASOScorer()
return scorer.calculate_overall_score(
metadata,
ratings,
keyword_performance,
conversion
)
FILE:scripts/competitor_analyzer.py
"""
Competitor analysis module for App Store Optimization.
Analyzes top competitors' ASO strategies and identifies opportunities.
"""
from typing import Dict, List, Any, Optional
from collections import Counter
import re
class CompetitorAnalyzer:
"""Analyzes competitor apps to identify ASO opportunities."""
def __init__(self, category: str, platform: str = 'apple'):
"""
Initialize competitor analyzer.
Args:
category: App category (e.g., "Productivity", "Games")
platform: 'apple' or 'google'
"""
self.category = category
self.platform = platform
self.competitors = []
def analyze_competitor(
self,
app_data: Dict[str, Any]
) -> Dict[str, Any]:
"""
Analyze a single competitor's ASO strategy.
Args:
app_data: Dictionary with app_name, title, description, rating, ratings_count, keywords
Returns:
Comprehensive competitor analysis
"""
app_name = app_data.get('app_name', '')
title = app_data.get('title', '')
description = app_data.get('description', '')
rating = app_data.get('rating', 0.0)
ratings_count = app_data.get('ratings_count', 0)
keywords = app_data.get('keywords', [])
analysis = {
'app_name': app_name,
'title_analysis': self._analyze_title(title),
'description_analysis': self._analyze_description(description),
'keyword_strategy': self._extract_keyword_strategy(title, description, keywords),
'rating_metrics': {
'rating': rating,
'ratings_count': ratings_count,
'rating_quality': self._assess_rating_quality(rating, ratings_count)
},
'competitive_strength': self._calculate_competitive_strength(
rating,
ratings_count,
len(description)
),
'key_differentiators': self._identify_differentiators(description)
}
self.competitors.append(analysis)
return analysis
def compare_competitors(
self,
competitors_data: List[Dict[str, Any]]
) -> Dict[str, Any]:
"""
Compare multiple competitors and identify patterns.
Args:
competitors_data: List of competitor data dictionaries
Returns:
Comparative analysis with insights
"""
# Analyze each competitor
analyses = []
for comp_data in competitors_data:
analysis = self.analyze_competitor(comp_data)
analyses.append(analysis)
# Extract common keywords across competitors
all_keywords = []
for analysis in analyses:
all_keywords.extend(analysis['keyword_strategy']['primary_keywords'])
common_keywords = self._find_common_keywords(all_keywords)
# Identify keyword gaps (used by some but not all)
keyword_gaps = self._identify_keyword_gaps(analyses)
# Rank competitors by strength
ranked_competitors = sorted(
analyses,
key=lambda x: x['competitive_strength'],
reverse=True
)
# Analyze rating distribution
rating_analysis = self._analyze_rating_distribution(analyses)
# Identify best practices
best_practices = self._identify_best_practices(ranked_competitors)
return {
'category': self.category,
'platform': self.platform,
'competitors_analyzed': len(analyses),
'ranked_competitors': ranked_competitors,
'common_keywords': common_keywords,
'keyword_gaps': keyword_gaps,
'rating_analysis': rating_analysis,
'best_practices': best_practices,
'opportunities': self._identify_opportunities(
analyses,
common_keywords,
keyword_gaps
)
}
def identify_gaps(
self,
your_app_data: Dict[str, Any],
competitors_data: List[Dict[str, Any]]
) -> Dict[str, Any]:
"""
Identify gaps between your app and competitors.
Args:
your_app_data: Your app's data
competitors_data: List of competitor data
Returns:
Gap analysis with actionable recommendations
"""
# Analyze your app
your_analysis = self.analyze_competitor(your_app_data)
# Analyze competitors
competitor_comparison = self.compare_competitors(competitors_data)
# Identify keyword gaps
your_keywords = set(your_analysis['keyword_strategy']['primary_keywords'])
competitor_keywords = set(competitor_comparison['common_keywords'])
missing_keywords = competitor_keywords - your_keywords
# Identify rating gap
avg_competitor_rating = competitor_comparison['rating_analysis']['average_rating']
rating_gap = avg_competitor_rating - your_analysis['rating_metrics']['rating']
# Identify description length gap
avg_competitor_desc_length = sum(
len(comp['description_analysis']['text'])
for comp in competitor_comparison['ranked_competitors']
) / len(competitor_comparison['ranked_competitors'])
your_desc_length = len(your_analysis['description_analysis']['text'])
desc_length_gap = avg_competitor_desc_length - your_desc_length
return {
'your_app': your_analysis,
'keyword_gaps': {
'missing_keywords': list(missing_keywords)[:10],
'recommendations': self._generate_keyword_recommendations(missing_keywords)
},
'rating_gap': {
'your_rating': your_analysis['rating_metrics']['rating'],
'average_competitor_rating': avg_competitor_rating,
'gap': round(rating_gap, 2),
'action_items': self._generate_rating_improvement_actions(rating_gap)
},
'content_gap': {
'your_description_length': your_desc_length,
'average_competitor_length': int(avg_competitor_desc_length),
'gap': int(desc_length_gap),
'recommendations': self._generate_content_recommendations(desc_length_gap)
},
'competitive_positioning': self._assess_competitive_position(
your_analysis,
competitor_comparison
)
}
def _analyze_title(self, title: str) -> Dict[str, Any]:
"""Analyze title structure and keyword usage."""
parts = re.split(r'[-:|]', title)
return {
'title': title,
'length': len(title),
'has_brand': len(parts) > 0,
'has_keywords': len(parts) > 1,
'components': [part.strip() for part in parts],
'word_count': len(title.split()),
'strategy': 'brand_plus_keywords' if len(parts) > 1 else 'brand_only'
}
def _analyze_description(self, description: str) -> Dict[str, Any]:
"""Analyze description structure and content."""
lines = description.split('\n')
word_count = len(description.split())
# Check for structural elements
has_bullet_points = '•' in description or '*' in description
has_sections = any(line.isupper() for line in lines if len(line) > 0)
has_call_to_action = any(
cta in description.lower()
for cta in ['download', 'try', 'get', 'start', 'join']
)
# Extract features mentioned
features = self._extract_features(description)
return {
'text': description,
'length': len(description),
'word_count': word_count,
'structure': {
'has_bullet_points': has_bullet_points,
'has_sections': has_sections,
'has_call_to_action': has_call_to_action
},
'features_mentioned': features,
'readability': 'good' if 50 <= word_count <= 300 else 'needs_improvement'
}
def _extract_keyword_strategy(
self,
title: str,
description: str,
explicit_keywords: List[str]
) -> Dict[str, Any]:
"""Extract keyword strategy from metadata."""
# Extract keywords from title
title_keywords = [word.lower() for word in title.split() if len(word) > 3]
# Extract frequently used words from description
desc_words = re.findall(r'\b\w{4,}\b', description.lower())
word_freq = Counter(desc_words)
frequent_words = [word for word, count in word_freq.most_common(15) if count > 2]
# Combine with explicit keywords
all_keywords = list(set(title_keywords + frequent_words + explicit_keywords))
return {
'primary_keywords': title_keywords,
'description_keywords': frequent_words[:10],
'explicit_keywords': explicit_keywords,
'total_unique_keywords': len(all_keywords),
'keyword_focus': self._assess_keyword_focus(title_keywords, frequent_words)
}
def _assess_rating_quality(self, rating: float, ratings_count: int) -> str:
"""Assess the quality of ratings."""
if ratings_count < 100:
return 'insufficient_data'
elif rating >= 4.5 and ratings_count > 1000:
return 'excellent'
elif rating >= 4.0 and ratings_count > 500:
return 'good'
elif rating >= 3.5:
return 'average'
else:
return 'poor'
def _calculate_competitive_strength(
self,
rating: float,
ratings_count: int,
description_length: int
) -> float:
"""
Calculate overall competitive strength (0-100).
Factors:
- Rating quality (40%)
- Rating volume (30%)
- Metadata quality (30%)
"""
# Rating quality score (0-40)
rating_score = (rating / 5.0) * 40
# Rating volume score (0-30)
volume_score = min((ratings_count / 10000) * 30, 30)
# Metadata quality score (0-30)
metadata_score = min((description_length / 2000) * 30, 30)
total_score = rating_score + volume_score + metadata_score
return round(total_score, 1)
def _identify_differentiators(self, description: str) -> List[str]:
"""Identify key differentiators from description."""
differentiator_keywords = [
'unique', 'only', 'first', 'best', 'leading', 'exclusive',
'revolutionary', 'innovative', 'patent', 'award'
]
differentiators = []
sentences = description.split('.')
for sentence in sentences:
sentence_lower = sentence.lower()
if any(keyword in sentence_lower for keyword in differentiator_keywords):
differentiators.append(sentence.strip())
return differentiators[:5]
def _find_common_keywords(self, all_keywords: List[str]) -> List[str]:
"""Find keywords used by multiple competitors."""
keyword_counts = Counter(all_keywords)
# Return keywords used by at least 2 competitors
common = [kw for kw, count in keyword_counts.items() if count >= 2]
return sorted(common, key=lambda x: keyword_counts[x], reverse=True)[:20]
def _identify_keyword_gaps(self, analyses: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
"""Identify keywords used by some competitors but not others."""
all_keywords_by_app = {}
for analysis in analyses:
app_name = analysis['app_name']
keywords = analysis['keyword_strategy']['primary_keywords']
all_keywords_by_app[app_name] = set(keywords)
# Find keywords used by some but not all
all_keywords_set = set()
for keywords in all_keywords_by_app.values():
all_keywords_set.update(keywords)
gaps = []
for keyword in all_keywords_set:
using_apps = [
app for app, keywords in all_keywords_by_app.items()
if keyword in keywords
]
if 1 < len(using_apps) < len(analyses):
gaps.append({
'keyword': keyword,
'used_by': using_apps,
'usage_percentage': round(len(using_apps) / len(analyses) * 100, 1)
})
return sorted(gaps, key=lambda x: x['usage_percentage'], reverse=True)[:15]
def _analyze_rating_distribution(self, analyses: List[Dict[str, Any]]) -> Dict[str, Any]:
"""Analyze rating distribution across competitors."""
ratings = [a['rating_metrics']['rating'] for a in analyses]
ratings_counts = [a['rating_metrics']['ratings_count'] for a in analyses]
return {
'average_rating': round(sum(ratings) / len(ratings), 2),
'highest_rating': max(ratings),
'lowest_rating': min(ratings),
'average_ratings_count': int(sum(ratings_counts) / len(ratings_counts)),
'total_ratings_in_category': sum(ratings_counts)
}
def _identify_best_practices(self, ranked_competitors: List[Dict[str, Any]]) -> List[str]:
"""Identify best practices from top competitors."""
if not ranked_competitors:
return []
top_competitor = ranked_competitors[0]
practices = []
# Title strategy
title_analysis = top_competitor['title_analysis']
if title_analysis['has_keywords']:
practices.append(
f"Title Strategy: Include primary keyword in title (e.g., '{title_analysis['title']}')"
)
# Description structure
desc_analysis = top_competitor['description_analysis']
if desc_analysis['structure']['has_bullet_points']:
practices.append("Description: Use bullet points to highlight key features")
if desc_analysis['structure']['has_sections']:
practices.append("Description: Organize content with clear section headers")
# Rating strategy
rating_quality = top_competitor['rating_metrics']['rating_quality']
if rating_quality in ['excellent', 'good']:
practices.append(
f"Ratings: Maintain high rating quality ({top_competitor['rating_metrics']['rating']}★) "
f"with significant volume ({top_competitor['rating_metrics']['ratings_count']} ratings)"
)
return practices[:5]
def _identify_opportunities(
self,
analyses: List[Dict[str, Any]],
common_keywords: List[str],
keyword_gaps: List[Dict[str, Any]]
) -> List[str]:
"""Identify ASO opportunities based on competitive analysis."""
opportunities = []
# Keyword opportunities from gaps
if keyword_gaps:
underutilized_keywords = [
gap['keyword'] for gap in keyword_gaps
if gap['usage_percentage'] < 50
]
if underutilized_keywords:
opportunities.append(
f"Target underutilized keywords: {', '.join(underutilized_keywords[:5])}"
)
# Rating opportunity
avg_rating = sum(a['rating_metrics']['rating'] for a in analyses) / len(analyses)
if avg_rating < 4.5:
opportunities.append(
f"Category average rating is {avg_rating:.1f} - opportunity to differentiate with higher ratings"
)
# Content depth opportunity
avg_desc_length = sum(
a['description_analysis']['length'] for a in analyses
) / len(analyses)
if avg_desc_length < 1500:
opportunities.append(
"Competitors have relatively short descriptions - opportunity to provide more comprehensive information"
)
return opportunities[:5]
def _extract_features(self, description: str) -> List[str]:
"""Extract feature mentions from description."""
# Look for bullet points or numbered lists
lines = description.split('\n')
features = []
for line in lines:
line = line.strip()
# Check if line starts with bullet or number
if line and (line[0] in ['•', '*', '-', '✓'] or line[0].isdigit()):
# Clean the line
cleaned = re.sub(r'^[•*\-✓\d.)\s]+', '', line)
if cleaned:
features.append(cleaned)
return features[:10]
def _assess_keyword_focus(
self,
title_keywords: List[str],
description_keywords: List[str]
) -> str:
"""Assess keyword focus strategy."""
overlap = set(title_keywords) & set(description_keywords)
if len(overlap) >= 3:
return 'consistent_focus'
elif len(overlap) >= 1:
return 'moderate_focus'
else:
return 'broad_focus'
def _generate_keyword_recommendations(self, missing_keywords: set) -> List[str]:
"""Generate recommendations for missing keywords."""
if not missing_keywords:
return ["Your keyword coverage is comprehensive"]
recommendations = []
missing_list = list(missing_keywords)[:5]
recommendations.append(
f"Consider adding these competitor keywords: {', '.join(missing_list)}"
)
recommendations.append(
"Test keyword variations in subtitle/promotional text first"
)
recommendations.append(
"Monitor competitor keyword changes monthly"
)
return recommendations
def _generate_rating_improvement_actions(self, rating_gap: float) -> List[str]:
"""Generate actions to improve ratings."""
actions = []
if rating_gap > 0.5:
actions.append("CRITICAL: Significant rating gap - prioritize user satisfaction improvements")
actions.append("Analyze negative reviews to identify top issues")
actions.append("Implement in-app rating prompts after positive experiences")
actions.append("Respond to all negative reviews professionally")
elif rating_gap > 0.2:
actions.append("Focus on incremental improvements to close rating gap")
actions.append("Optimize timing of rating requests")
else:
actions.append("Ratings are competitive - maintain quality and continue improvements")
return actions
def _generate_content_recommendations(self, desc_length_gap: int) -> List[str]:
"""Generate content recommendations based on length gap."""
recommendations = []
if desc_length_gap > 500:
recommendations.append(
"Expand description to match competitor detail level"
)
recommendations.append(
"Add use case examples and success stories"
)
recommendations.append(
"Include more feature explanations and benefits"
)
elif desc_length_gap < -500:
recommendations.append(
"Consider condensing description for better readability"
)
recommendations.append(
"Focus on most important features first"
)
else:
recommendations.append(
"Description length is competitive"
)
return recommendations
def _assess_competitive_position(
self,
your_analysis: Dict[str, Any],
competitor_comparison: Dict[str, Any]
) -> str:
"""Assess your competitive position."""
your_strength = your_analysis['competitive_strength']
competitors = competitor_comparison['ranked_competitors']
if not competitors:
return "No comparison data available"
# Find where you'd rank
better_than_count = sum(
1 for comp in competitors
if your_strength > comp['competitive_strength']
)
position_percentage = (better_than_count / len(competitors)) * 100
if position_percentage >= 75:
return "Strong Position: Top quartile in competitive strength"
elif position_percentage >= 50:
return "Competitive Position: Above average, opportunities for improvement"
elif position_percentage >= 25:
return "Challenging Position: Below average, requires strategic improvements"
else:
return "Weak Position: Bottom quartile, major ASO overhaul needed"
def analyze_competitor_set(
category: str,
competitors_data: List[Dict[str, Any]],
platform: str = 'apple'
) -> Dict[str, Any]:
"""
Convenience function to analyze a set of competitors.
Args:
category: App category
competitors_data: List of competitor data
platform: 'apple' or 'google'
Returns:
Complete competitive analysis
"""
analyzer = CompetitorAnalyzer(category, platform)
return analyzer.compare_competitors(competitors_data)
FILE:scripts/keyword_analyzer.py
"""
Keyword analysis module for App Store Optimization.
Analyzes keyword search volume, competition, and relevance for app discovery.
"""
from typing import Dict, List, Any, Optional, Tuple
import re
from collections import Counter
class KeywordAnalyzer:
"""Analyzes keywords for ASO effectiveness."""
# Competition level thresholds (based on number of competing apps)
COMPETITION_THRESHOLDS = {
'low': 1000,
'medium': 5000,
'high': 10000
}
# Search volume categories (monthly searches estimate)
VOLUME_CATEGORIES = {
'very_low': 1000,
'low': 5000,
'medium': 20000,
'high': 100000,
'very_high': 500000
}
def __init__(self):
"""Initialize keyword analyzer."""
self.analyzed_keywords = {}
def analyze_keyword(
self,
keyword: str,
search_volume: int = 0,
competing_apps: int = 0,
relevance_score: float = 0.0
) -> Dict[str, Any]:
"""
Analyze a single keyword for ASO potential.
Args:
keyword: The keyword to analyze
search_volume: Estimated monthly search volume
competing_apps: Number of apps competing for this keyword
relevance_score: Relevance to your app (0.0-1.0)
Returns:
Dictionary with keyword analysis
"""
competition_level = self._calculate_competition_level(competing_apps)
volume_category = self._categorize_search_volume(search_volume)
difficulty_score = self._calculate_keyword_difficulty(
search_volume,
competing_apps
)
# Calculate potential score (0-100)
potential_score = self._calculate_potential_score(
search_volume,
competing_apps,
relevance_score
)
analysis = {
'keyword': keyword,
'search_volume': search_volume,
'volume_category': volume_category,
'competing_apps': competing_apps,
'competition_level': competition_level,
'relevance_score': relevance_score,
'difficulty_score': difficulty_score,
'potential_score': potential_score,
'recommendation': self._generate_recommendation(
potential_score,
difficulty_score,
relevance_score
),
'keyword_length': len(keyword.split()),
'is_long_tail': len(keyword.split()) >= 3
}
self.analyzed_keywords[keyword] = analysis
return analysis
def compare_keywords(self, keywords_data: List[Dict[str, Any]]) -> Dict[str, Any]:
"""
Compare multiple keywords and rank by potential.
Args:
keywords_data: List of dicts with keyword, search_volume, competing_apps, relevance_score
Returns:
Comparison report with ranked keywords
"""
analyses = []
for kw_data in keywords_data:
analysis = self.analyze_keyword(
keyword=kw_data['keyword'],
search_volume=kw_data.get('search_volume', 0),
competing_apps=kw_data.get('competing_apps', 0),
relevance_score=kw_data.get('relevance_score', 0.0)
)
analyses.append(analysis)
# Sort by potential score (descending)
ranked_keywords = sorted(
analyses,
key=lambda x: x['potential_score'],
reverse=True
)
# Categorize keywords
primary_keywords = [
kw for kw in ranked_keywords
if kw['potential_score'] >= 70 and kw['relevance_score'] >= 0.8
]
secondary_keywords = [
kw for kw in ranked_keywords
if 50 <= kw['potential_score'] < 70 and kw['relevance_score'] >= 0.6
]
long_tail_keywords = [
kw for kw in ranked_keywords
if kw['is_long_tail'] and kw['relevance_score'] >= 0.7
]
return {
'total_keywords_analyzed': len(analyses),
'ranked_keywords': ranked_keywords,
'primary_keywords': primary_keywords[:5], # Top 5
'secondary_keywords': secondary_keywords[:10], # Top 10
'long_tail_keywords': long_tail_keywords[:10], # Top 10
'summary': self._generate_comparison_summary(
primary_keywords,
secondary_keywords,
long_tail_keywords
)
}
def find_long_tail_opportunities(
self,
base_keyword: str,
modifiers: List[str]
) -> List[Dict[str, Any]]:
"""
Generate long-tail keyword variations.
Args:
base_keyword: Core keyword (e.g., "task manager")
modifiers: List of modifiers (e.g., ["free", "simple", "team"])
Returns:
List of long-tail keyword suggestions
"""
long_tail_keywords = []
# Generate combinations
for modifier in modifiers:
# Modifier + base
variation1 = f"{modifier} {base_keyword}"
long_tail_keywords.append({
'keyword': variation1,
'pattern': 'modifier_base',
'estimated_competition': 'low',
'rationale': f"Less competitive variation of '{base_keyword}'"
})
# Base + modifier
variation2 = f"{base_keyword} {modifier}"
long_tail_keywords.append({
'keyword': variation2,
'pattern': 'base_modifier',
'estimated_competition': 'low',
'rationale': f"Specific use-case variation of '{base_keyword}'"
})
# Add question-based long-tail
question_words = ['how', 'what', 'best', 'top']
for q_word in question_words:
question_keyword = f"{q_word} {base_keyword}"
long_tail_keywords.append({
'keyword': question_keyword,
'pattern': 'question_based',
'estimated_competition': 'very_low',
'rationale': f"Informational search query"
})
return long_tail_keywords
def extract_keywords_from_text(
self,
text: str,
min_word_length: int = 3
) -> List[Tuple[str, int]]:
"""
Extract potential keywords from text (descriptions, reviews).
Args:
text: Text to analyze
min_word_length: Minimum word length to consider
Returns:
List of (keyword, frequency) tuples
"""
# Clean and normalize text
text = text.lower()
text = re.sub(r'[^\w\s]', ' ', text)
# Extract words
words = text.split()
# Filter by length
words = [w for w in words if len(w) >= min_word_length]
# Remove common stop words
stop_words = {
'the', 'and', 'for', 'with', 'this', 'that', 'from', 'have',
'but', 'not', 'you', 'all', 'can', 'are', 'was', 'were', 'been'
}
words = [w for w in words if w not in stop_words]
# Count frequency
word_counts = Counter(words)
# Extract 2-word phrases
phrases = []
for i in range(len(words) - 1):
phrase = f"{words[i]} {words[i+1]}"
phrases.append(phrase)
phrase_counts = Counter(phrases)
# Combine and sort
all_keywords = list(word_counts.items()) + list(phrase_counts.items())
all_keywords.sort(key=lambda x: x[1], reverse=True)
return all_keywords[:50] # Top 50
def calculate_keyword_density(
self,
text: str,
target_keywords: List[str]
) -> Dict[str, float]:
"""
Calculate keyword density in text.
Args:
text: Text to analyze (title, description)
target_keywords: Keywords to check density for
Returns:
Dictionary of keyword: density (percentage)
"""
text_lower = text.lower()
total_words = len(text_lower.split())
densities = {}
for keyword in target_keywords:
keyword_lower = keyword.lower()
occurrences = text_lower.count(keyword_lower)
density = (occurrences / total_words) * 100 if total_words > 0 else 0
densities[keyword] = round(density, 2)
return densities
def _calculate_competition_level(self, competing_apps: int) -> str:
"""Determine competition level based on number of competing apps."""
if competing_apps < self.COMPETITION_THRESHOLDS['low']:
return 'low'
elif competing_apps < self.COMPETITION_THRESHOLDS['medium']:
return 'medium'
elif competing_apps < self.COMPETITION_THRESHOLDS['high']:
return 'high'
else:
return 'very_high'
def _categorize_search_volume(self, search_volume: int) -> str:
"""Categorize search volume."""
if search_volume < self.VOLUME_CATEGORIES['very_low']:
return 'very_low'
elif search_volume < self.VOLUME_CATEGORIES['low']:
return 'low'
elif search_volume < self.VOLUME_CATEGORIES['medium']:
return 'medium'
elif search_volume < self.VOLUME_CATEGORIES['high']:
return 'high'
else:
return 'very_high'
def _calculate_keyword_difficulty(
self,
search_volume: int,
competing_apps: int
) -> float:
"""
Calculate keyword difficulty score (0-100).
Higher score = harder to rank.
"""
if competing_apps == 0:
return 0.0
# Competition factor (0-1)
competition_factor = min(competing_apps / 50000, 1.0)
# Volume factor (0-1) - higher volume = more difficulty
volume_factor = min(search_volume / 1000000, 1.0)
# Difficulty score (weighted average)
difficulty = (competition_factor * 0.7 + volume_factor * 0.3) * 100
return round(difficulty, 1)
def _calculate_potential_score(
self,
search_volume: int,
competing_apps: int,
relevance_score: float
) -> float:
"""
Calculate overall keyword potential (0-100).
Higher score = better opportunity.
"""
# Volume score (0-40 points)
volume_score = min((search_volume / 100000) * 40, 40)
# Competition score (0-30 points) - inverse relationship
if competing_apps > 0:
competition_score = max(30 - (competing_apps / 500), 0)
else:
competition_score = 30
# Relevance score (0-30 points)
relevance_points = relevance_score * 30
total_score = volume_score + competition_score + relevance_points
return round(min(total_score, 100), 1)
def _generate_recommendation(
self,
potential_score: float,
difficulty_score: float,
relevance_score: float
) -> str:
"""Generate actionable recommendation for keyword."""
if relevance_score < 0.5:
return "Low relevance - avoid targeting"
if potential_score >= 70:
return "High priority - target immediately"
elif potential_score >= 50:
if difficulty_score < 50:
return "Good opportunity - include in metadata"
else:
return "Competitive - use in description, not title"
elif potential_score >= 30:
return "Secondary keyword - use for long-tail variations"
else:
return "Low potential - deprioritize"
def _generate_comparison_summary(
self,
primary_keywords: List[Dict[str, Any]],
secondary_keywords: List[Dict[str, Any]],
long_tail_keywords: List[Dict[str, Any]]
) -> str:
"""Generate summary of keyword comparison."""
summary_parts = []
summary_parts.append(
f"Identified {len(primary_keywords)} high-priority primary keywords."
)
if primary_keywords:
top_keyword = primary_keywords[0]['keyword']
summary_parts.append(
f"Top recommendation: '{top_keyword}' (potential score: {primary_keywords[0]['potential_score']})."
)
summary_parts.append(
f"Found {len(secondary_keywords)} secondary keywords for description and metadata."
)
summary_parts.append(
f"Discovered {len(long_tail_keywords)} long-tail opportunities with lower competition."
)
return " ".join(summary_parts)
def analyze_keyword_set(keywords_data: List[Dict[str, Any]]) -> Dict[str, Any]:
"""
Convenience function to analyze a set of keywords.
Args:
keywords_data: List of keyword data dictionaries
Returns:
Complete analysis report
"""
analyzer = KeywordAnalyzer()
return analyzer.compare_keywords(keywords_data)
FILE:scripts/launch_checklist.py
"""
Launch checklist module for App Store Optimization.
Generates comprehensive pre-launch and update checklists.
"""
from typing import Dict, List, Any, Optional
from datetime import datetime, timedelta
class LaunchChecklistGenerator:
"""Generates comprehensive checklists for app launches and updates."""
def __init__(self, platform: str = 'both'):
"""
Initialize checklist generator.
Args:
platform: 'apple', 'google', or 'both'
"""
if platform not in ['apple', 'google', 'both']:
raise ValueError("Platform must be 'apple', 'google', or 'both'")
self.platform = platform
def generate_prelaunch_checklist(
self,
app_info: Dict[str, Any],
launch_date: Optional[str] = None
) -> Dict[str, Any]:
"""
Generate comprehensive pre-launch checklist.
Args:
app_info: App information (name, category, target_audience)
launch_date: Target launch date (YYYY-MM-DD)
Returns:
Complete pre-launch checklist
"""
checklist = {
'app_info': app_info,
'launch_date': launch_date,
'checklists': {}
}
# Generate platform-specific checklists
if self.platform in ['apple', 'both']:
checklist['checklists']['apple'] = self._generate_apple_checklist(app_info)
if self.platform in ['google', 'both']:
checklist['checklists']['google'] = self._generate_google_checklist(app_info)
# Add universal checklist items
checklist['checklists']['universal'] = self._generate_universal_checklist(app_info)
# Generate timeline
if launch_date:
checklist['timeline'] = self._generate_launch_timeline(launch_date)
# Calculate completion status
checklist['summary'] = self._calculate_checklist_summary(checklist['checklists'])
return checklist
def validate_app_store_compliance(
self,
app_data: Dict[str, Any],
platform: str = 'apple'
) -> Dict[str, Any]:
"""
Validate compliance with app store guidelines.
Args:
app_data: App data including metadata, privacy policy, etc.
platform: 'apple' or 'google'
Returns:
Compliance validation report
"""
validation_results = {
'platform': platform,
'is_compliant': True,
'errors': [],
'warnings': [],
'recommendations': []
}
if platform == 'apple':
self._validate_apple_compliance(app_data, validation_results)
elif platform == 'google':
self._validate_google_compliance(app_data, validation_results)
# Determine overall compliance
validation_results['is_compliant'] = len(validation_results['errors']) == 0
return validation_results
def create_update_plan(
self,
current_version: str,
planned_features: List[str],
update_frequency: str = 'monthly'
) -> Dict[str, Any]:
"""
Create update cadence and feature rollout plan.
Args:
current_version: Current app version
planned_features: List of planned features
update_frequency: 'weekly', 'biweekly', 'monthly', 'quarterly'
Returns:
Update plan with cadence and feature schedule
"""
# Calculate next versions
next_versions = self._calculate_next_versions(
current_version,
update_frequency,
len(planned_features)
)
# Distribute features across versions
feature_schedule = self._distribute_features(
planned_features,
next_versions
)
# Generate "What's New" templates
whats_new_templates = [
self._generate_whats_new_template(version_data)
for version_data in feature_schedule
]
return {
'current_version': current_version,
'update_frequency': update_frequency,
'planned_updates': len(feature_schedule),
'feature_schedule': feature_schedule,
'whats_new_templates': whats_new_templates,
'recommendations': self._generate_update_recommendations(update_frequency)
}
def optimize_launch_timing(
self,
app_category: str,
target_audience: str,
current_date: Optional[str] = None
) -> Dict[str, Any]:
"""
Recommend optimal launch timing.
Args:
app_category: App category
target_audience: Target audience description
current_date: Current date (YYYY-MM-DD), defaults to today
Returns:
Launch timing recommendations
"""
if not current_date:
current_date = datetime.now().strftime('%Y-%m-%d')
# Analyze launch timing factors
day_of_week_rec = self._recommend_day_of_week(app_category)
seasonal_rec = self._recommend_seasonal_timing(app_category, current_date)
competitive_rec = self._analyze_competitive_timing(app_category)
# Calculate optimal dates
optimal_dates = self._calculate_optimal_dates(
current_date,
day_of_week_rec,
seasonal_rec
)
return {
'current_date': current_date,
'optimal_launch_dates': optimal_dates,
'day_of_week_recommendation': day_of_week_rec,
'seasonal_considerations': seasonal_rec,
'competitive_timing': competitive_rec,
'final_recommendation': self._generate_timing_recommendation(
optimal_dates,
seasonal_rec
)
}
def plan_seasonal_campaigns(
self,
app_category: str,
current_month: int = None
) -> Dict[str, Any]:
"""
Identify seasonal opportunities for ASO campaigns.
Args:
app_category: App category
current_month: Current month (1-12), defaults to current
Returns:
Seasonal campaign opportunities
"""
if not current_month:
current_month = datetime.now().month
# Identify relevant seasonal events
seasonal_opportunities = self._identify_seasonal_opportunities(
app_category,
current_month
)
# Generate campaign ideas
campaigns = [
self._generate_seasonal_campaign(opportunity)
for opportunity in seasonal_opportunities
]
return {
'current_month': current_month,
'category': app_category,
'seasonal_opportunities': seasonal_opportunities,
'campaign_ideas': campaigns,
'implementation_timeline': self._create_seasonal_timeline(campaigns)
}
def _generate_apple_checklist(self, app_info: Dict[str, Any]) -> List[Dict[str, Any]]:
"""Generate Apple App Store specific checklist."""
return [
{
'category': 'App Store Connect Setup',
'items': [
{'task': 'App Store Connect account created', 'status': 'pending'},
{'task': 'App bundle ID registered', 'status': 'pending'},
{'task': 'App Privacy declarations completed', 'status': 'pending'},
{'task': 'Age rating questionnaire completed', 'status': 'pending'}
]
},
{
'category': 'Metadata (Apple)',
'items': [
{'task': 'App title (30 chars max)', 'status': 'pending'},
{'task': 'Subtitle (30 chars max)', 'status': 'pending'},
{'task': 'Promotional text (170 chars max)', 'status': 'pending'},
{'task': 'Description (4000 chars max)', 'status': 'pending'},
{'task': 'Keywords (100 chars, comma-separated)', 'status': 'pending'},
{'task': 'Category selection (primary + secondary)', 'status': 'pending'}
]
},
{
'category': 'Visual Assets (Apple)',
'items': [
{'task': 'App icon (1024x1024px)', 'status': 'pending'},
{'task': 'Screenshots (iPhone 6.7" required)', 'status': 'pending'},
{'task': 'Screenshots (iPhone 5.5" required)', 'status': 'pending'},
{'task': 'Screenshots (iPad Pro 12.9" if iPad app)', 'status': 'pending'},
{'task': 'App preview video (optional but recommended)', 'status': 'pending'}
]
},
{
'category': 'Technical Requirements (Apple)',
'items': [
{'task': 'Build uploaded to App Store Connect', 'status': 'pending'},
{'task': 'TestFlight testing completed', 'status': 'pending'},
{'task': 'App tested on required iOS versions', 'status': 'pending'},
{'task': 'Crash-free rate > 99%', 'status': 'pending'},
{'task': 'All links in app/metadata working', 'status': 'pending'}
]
},
{
'category': 'Legal & Privacy (Apple)',
'items': [
{'task': 'Privacy Policy URL provided', 'status': 'pending'},
{'task': 'Terms of Service URL (if applicable)', 'status': 'pending'},
{'task': 'Data collection declarations accurate', 'status': 'pending'},
{'task': 'Third-party SDKs disclosed', 'status': 'pending'}
]
}
]
def _generate_google_checklist(self, app_info: Dict[str, Any]) -> List[Dict[str, Any]]:
"""Generate Google Play Store specific checklist."""
return [
{
'category': 'Play Console Setup',
'items': [
{'task': 'Google Play Console account created', 'status': 'pending'},
{'task': 'Developer profile completed', 'status': 'pending'},
{'task': 'Payment merchant account linked (if paid app)', 'status': 'pending'},
{'task': 'Content rating questionnaire completed', 'status': 'pending'}
]
},
{
'category': 'Metadata (Google)',
'items': [
{'task': 'App title (50 chars max)', 'status': 'pending'},
{'task': 'Short description (80 chars max)', 'status': 'pending'},
{'task': 'Full description (4000 chars max)', 'status': 'pending'},
{'task': 'Category selection', 'status': 'pending'},
{'task': 'Tags (up to 5)', 'status': 'pending'}
]
},
{
'category': 'Visual Assets (Google)',
'items': [
{'task': 'App icon (512x512px)', 'status': 'pending'},
{'task': 'Feature graphic (1024x500px)', 'status': 'pending'},
{'task': 'Screenshots (2-8 required, phone)', 'status': 'pending'},
{'task': 'Screenshots (tablet, if applicable)', 'status': 'pending'},
{'task': 'Promo video (YouTube link, optional)', 'status': 'pending'}
]
},
{
'category': 'Technical Requirements (Google)',
'items': [
{'task': 'APK/AAB uploaded to Play Console', 'status': 'pending'},
{'task': 'Internal testing completed', 'status': 'pending'},
{'task': 'App tested on required Android versions', 'status': 'pending'},
{'task': 'Target API level meets requirements', 'status': 'pending'},
{'task': 'All permissions justified', 'status': 'pending'}
]
},
{
'category': 'Legal & Privacy (Google)',
'items': [
{'task': 'Privacy Policy URL provided', 'status': 'pending'},
{'task': 'Data safety section completed', 'status': 'pending'},
{'task': 'Ads disclosure (if applicable)', 'status': 'pending'},
{'task': 'In-app purchase disclosure (if applicable)', 'status': 'pending'}
]
}
]
def _generate_universal_checklist(self, app_info: Dict[str, Any]) -> List[Dict[str, Any]]:
"""Generate universal (both platforms) checklist."""
return [
{
'category': 'Pre-Launch Marketing',
'items': [
{'task': 'Landing page created', 'status': 'pending'},
{'task': 'Social media accounts setup', 'status': 'pending'},
{'task': 'Press kit prepared', 'status': 'pending'},
{'task': 'Beta tester feedback collected', 'status': 'pending'},
{'task': 'Launch announcement drafted', 'status': 'pending'}
]
},
{
'category': 'ASO Preparation',
'items': [
{'task': 'Keyword research completed', 'status': 'pending'},
{'task': 'Competitor analysis done', 'status': 'pending'},
{'task': 'A/B test plan created for post-launch', 'status': 'pending'},
{'task': 'Analytics tracking configured', 'status': 'pending'}
]
},
{
'category': 'Quality Assurance',
'items': [
{'task': 'All core features tested', 'status': 'pending'},
{'task': 'User flows validated', 'status': 'pending'},
{'task': 'Performance testing completed', 'status': 'pending'},
{'task': 'Accessibility features tested', 'status': 'pending'},
{'task': 'Security audit completed', 'status': 'pending'}
]
},
{
'category': 'Support Infrastructure',
'items': [
{'task': 'Support email/system setup', 'status': 'pending'},
{'task': 'FAQ page created', 'status': 'pending'},
{'task': 'Documentation for users prepared', 'status': 'pending'},
{'task': 'Team trained on handling reviews', 'status': 'pending'}
]
}
]
def _generate_launch_timeline(self, launch_date: str) -> List[Dict[str, Any]]:
"""Generate timeline with milestones leading to launch."""
launch_dt = datetime.strptime(launch_date, '%Y-%m-%d')
milestones = [
{
'date': (launch_dt - timedelta(days=90)).strftime('%Y-%m-%d'),
'milestone': '90 days before: Complete keyword research and competitor analysis'
},
{
'date': (launch_dt - timedelta(days=60)).strftime('%Y-%m-%d'),
'milestone': '60 days before: Finalize metadata and visual assets'
},
{
'date': (launch_dt - timedelta(days=45)).strftime('%Y-%m-%d'),
'milestone': '45 days before: Begin beta testing program'
},
{
'date': (launch_dt - timedelta(days=30)).strftime('%Y-%m-%d'),
'milestone': '30 days before: Submit app for review (Apple typically takes 1-2 days, Google instant)'
},
{
'date': (launch_dt - timedelta(days=14)).strftime('%Y-%m-%d'),
'milestone': '14 days before: Prepare launch marketing materials'
},
{
'date': (launch_dt - timedelta(days=7)).strftime('%Y-%m-%d'),
'milestone': '7 days before: Set up analytics and monitoring'
},
{
'date': launch_dt.strftime('%Y-%m-%d'),
'milestone': 'Launch Day: Release app and execute marketing plan'
},
{
'date': (launch_dt + timedelta(days=7)).strftime('%Y-%m-%d'),
'milestone': '7 days after: Monitor metrics, respond to reviews, address critical issues'
},
{
'date': (launch_dt + timedelta(days=30)).strftime('%Y-%m-%d'),
'milestone': '30 days after: Analyze launch metrics, plan first update'
}
]
return milestones
def _calculate_checklist_summary(self, checklists: Dict[str, List[Dict[str, Any]]]) -> Dict[str, Any]:
"""Calculate completion summary."""
total_items = 0
completed_items = 0
for platform, categories in checklists.items():
for category in categories:
for item in category['items']:
total_items += 1
if item['status'] == 'completed':
completed_items += 1
completion_percentage = (completed_items / total_items * 100) if total_items > 0 else 0
return {
'total_items': total_items,
'completed_items': completed_items,
'pending_items': total_items - completed_items,
'completion_percentage': round(completion_percentage, 1),
'is_ready_to_launch': completion_percentage == 100
}
def _validate_apple_compliance(
self,
app_data: Dict[str, Any],
validation_results: Dict[str, Any]
) -> None:
"""Validate Apple App Store compliance."""
# Check for required fields
if not app_data.get('privacy_policy_url'):
validation_results['errors'].append("Privacy Policy URL is required")
if not app_data.get('app_icon'):
validation_results['errors'].append("App icon (1024x1024px) is required")
# Check metadata character limits
title = app_data.get('title', '')
if len(title) > 30:
validation_results['errors'].append(f"Title exceeds 30 characters ({len(title)})")
# Warnings for best practices
subtitle = app_data.get('subtitle', '')
if not subtitle:
validation_results['warnings'].append("Subtitle is empty - consider adding for better discoverability")
keywords = app_data.get('keywords', '')
if len(keywords) < 80:
validation_results['warnings'].append(
f"Keywords field underutilized ({len(keywords)}/100 chars) - add more keywords"
)
def _validate_google_compliance(
self,
app_data: Dict[str, Any],
validation_results: Dict[str, Any]
) -> None:
"""Validate Google Play Store compliance."""
# Check for required fields
if not app_data.get('privacy_policy_url'):
validation_results['errors'].append("Privacy Policy URL is required")
if not app_data.get('feature_graphic'):
validation_results['errors'].append("Feature graphic (1024x500px) is required")
# Check metadata character limits
title = app_data.get('title', '')
if len(title) > 50:
validation_results['errors'].append(f"Title exceeds 50 characters ({len(title)})")
short_desc = app_data.get('short_description', '')
if len(short_desc) > 80:
validation_results['errors'].append(f"Short description exceeds 80 characters ({len(short_desc)})")
# Warnings
if not short_desc:
validation_results['warnings'].append("Short description is empty")
def _calculate_next_versions(
self,
current_version: str,
update_frequency: str,
feature_count: int
) -> List[str]:
"""Calculate next version numbers."""
# Parse current version (assume semantic versioning)
parts = current_version.split('.')
major, minor, patch = int(parts[0]), int(parts[1]), int(parts[2] if len(parts) > 2 else 0)
versions = []
for i in range(feature_count):
if update_frequency == 'weekly':
patch += 1
elif update_frequency == 'biweekly':
patch += 1
elif update_frequency == 'monthly':
minor += 1
patch = 0
else: # quarterly
minor += 1
patch = 0
versions.append(f"{major}.{minor}.{patch}")
return versions
def _distribute_features(
self,
features: List[str],
versions: List[str]
) -> List[Dict[str, Any]]:
"""Distribute features across versions."""
features_per_version = max(1, len(features) // len(versions))
schedule = []
for i, version in enumerate(versions):
start_idx = i * features_per_version
end_idx = start_idx + features_per_version if i < len(versions) - 1 else len(features)
schedule.append({
'version': version,
'features': features[start_idx:end_idx],
'release_priority': 'high' if i == 0 else ('medium' if i < len(versions) // 2 else 'low')
})
return schedule
def _generate_whats_new_template(self, version_data: Dict[str, Any]) -> Dict[str, str]:
"""Generate What's New template for version."""
features_list = '\n'.join([f"• {feature}" for feature in version_data['features']])
template = f"""Version {version_data['version']}
{features_list}
We're constantly improving your experience. Thanks for using [App Name]!
Have feedback? Contact us at support@[company].com"""
return {
'version': version_data['version'],
'template': template
}
def _generate_update_recommendations(self, update_frequency: str) -> List[str]:
"""Generate recommendations for update strategy."""
recommendations = []
if update_frequency == 'weekly':
recommendations.append("Weekly updates show active development but ensure quality doesn't suffer")
elif update_frequency == 'monthly':
recommendations.append("Monthly updates are optimal for most apps - balance features and stability")
recommendations.extend([
"Include bug fixes in every update",
"Update 'What's New' section with each release",
"Respond to reviews mentioning fixed issues"
])
return recommendations
def _recommend_day_of_week(self, app_category: str) -> Dict[str, Any]:
"""Recommend best day of week to launch."""
# General recommendations based on category
if app_category.lower() in ['games', 'entertainment']:
return {
'recommended_day': 'Thursday',
'rationale': 'People download entertainment apps before weekend'
}
elif app_category.lower() in ['productivity', 'business']:
return {
'recommended_day': 'Tuesday',
'rationale': 'Business users most active mid-week'
}
else:
return {
'recommended_day': 'Wednesday',
'rationale': 'Mid-week provides good balance and review potential'
}
def _recommend_seasonal_timing(self, app_category: str, current_date: str) -> Dict[str, Any]:
"""Recommend seasonal timing considerations."""
current_dt = datetime.strptime(current_date, '%Y-%m-%d')
month = current_dt.month
# Avoid certain periods
avoid_periods = []
if month == 12:
avoid_periods.append("Late December - low user engagement during holidays")
if month in [7, 8]:
avoid_periods.append("Summer months - some categories see lower engagement")
# Recommend periods
good_periods = []
if month in [1, 9]:
good_periods.append("New Year/Back-to-school - high user engagement")
if month in [10, 11]:
good_periods.append("Pre-holiday season - good for shopping/gift apps")
return {
'current_month': month,
'avoid_periods': avoid_periods,
'good_periods': good_periods
}
def _analyze_competitive_timing(self, app_category: str) -> Dict[str, str]:
"""Analyze competitive timing considerations."""
return {
'recommendation': 'Research competitor launch schedules in your category',
'strategy': 'Avoid launching same week as major competitor updates'
}
def _calculate_optimal_dates(
self,
current_date: str,
day_rec: Dict[str, Any],
seasonal_rec: Dict[str, Any]
) -> List[str]:
"""Calculate optimal launch dates."""
current_dt = datetime.strptime(current_date, '%Y-%m-%d')
# Find next occurrence of recommended day
target_day = day_rec['recommended_day']
days_map = {'Monday': 0, 'Tuesday': 1, 'Wednesday': 2, 'Thursday': 3, 'Friday': 4}
target_day_num = days_map.get(target_day, 2)
days_ahead = (target_day_num - current_dt.weekday()) % 7
if days_ahead == 0:
days_ahead = 7
next_target_date = current_dt + timedelta(days=days_ahead)
optimal_dates = [
next_target_date.strftime('%Y-%m-%d'),
(next_target_date + timedelta(days=7)).strftime('%Y-%m-%d'),
(next_target_date + timedelta(days=14)).strftime('%Y-%m-%d')
]
return optimal_dates
def _generate_timing_recommendation(
self,
optimal_dates: List[str],
seasonal_rec: Dict[str, Any]
) -> str:
"""Generate final timing recommendation."""
if seasonal_rec['avoid_periods']:
return f"Consider launching in {optimal_dates[1]} to avoid {seasonal_rec['avoid_periods'][0]}"
elif seasonal_rec['good_periods']:
return f"Launch on {optimal_dates[0]} to capitalize on {seasonal_rec['good_periods'][0]}"
else:
return f"Recommended launch date: {optimal_dates[0]}"
def _identify_seasonal_opportunities(
self,
app_category: str,
current_month: int
) -> List[Dict[str, Any]]:
"""Identify seasonal opportunities for category."""
opportunities = []
# Universal opportunities
if current_month == 1:
opportunities.append({
'event': 'New Year Resolutions',
'dates': 'January 1-31',
'relevance': 'high' if app_category.lower() in ['health', 'fitness', 'productivity'] else 'medium'
})
if current_month in [11, 12]:
opportunities.append({
'event': 'Holiday Shopping Season',
'dates': 'November-December',
'relevance': 'high' if app_category.lower() in ['shopping', 'gifts'] else 'low'
})
# Category-specific
if app_category.lower() == 'education' and current_month in [8, 9]:
opportunities.append({
'event': 'Back to School',
'dates': 'August-September',
'relevance': 'high'
})
return opportunities
def _generate_seasonal_campaign(self, opportunity: Dict[str, Any]) -> Dict[str, Any]:
"""Generate campaign idea for seasonal opportunity."""
return {
'event': opportunity['event'],
'campaign_idea': f"Create themed visuals and messaging for {opportunity['event']}",
'metadata_updates': 'Update app description and screenshots with seasonal themes',
'promotion_strategy': 'Consider limited-time features or discounts'
}
def _create_seasonal_timeline(self, campaigns: List[Dict[str, Any]]) -> List[str]:
"""Create implementation timeline for campaigns."""
return [
f"30 days before: Plan {campaign['event']} campaign strategy"
for campaign in campaigns
]
def generate_launch_checklist(
platform: str,
app_info: Dict[str, Any],
launch_date: Optional[str] = None
) -> Dict[str, Any]:
"""
Convenience function to generate launch checklist.
Args:
platform: Platform ('apple', 'google', or 'both')
app_info: App information
launch_date: Target launch date
Returns:
Complete launch checklist
"""
generator = LaunchChecklistGenerator(platform)
return generator.generate_prelaunch_checklist(app_info, launch_date)
FILE:scripts/localization_helper.py
"""
Localization helper module for App Store Optimization.
Manages multi-language ASO optimization strategies.
"""
from typing import Dict, List, Any, Optional, Tuple
class LocalizationHelper:
"""Helps manage multi-language ASO optimization."""
# Priority markets by language (based on app store revenue and user base)
PRIORITY_MARKETS = {
'tier_1': [
{'language': 'en-US', 'market': 'United States', 'revenue_share': 0.25},
{'language': 'zh-CN', 'market': 'China', 'revenue_share': 0.20},
{'language': 'ja-JP', 'market': 'Japan', 'revenue_share': 0.10},
{'language': 'de-DE', 'market': 'Germany', 'revenue_share': 0.08},
{'language': 'en-GB', 'market': 'United Kingdom', 'revenue_share': 0.06}
],
'tier_2': [
{'language': 'fr-FR', 'market': 'France', 'revenue_share': 0.05},
{'language': 'ko-KR', 'market': 'South Korea', 'revenue_share': 0.05},
{'language': 'es-ES', 'market': 'Spain', 'revenue_share': 0.03},
{'language': 'it-IT', 'market': 'Italy', 'revenue_share': 0.03},
{'language': 'pt-BR', 'market': 'Brazil', 'revenue_share': 0.03}
],
'tier_3': [
{'language': 'ru-RU', 'market': 'Russia', 'revenue_share': 0.02},
{'language': 'es-MX', 'market': 'Mexico', 'revenue_share': 0.02},
{'language': 'nl-NL', 'market': 'Netherlands', 'revenue_share': 0.02},
{'language': 'sv-SE', 'market': 'Sweden', 'revenue_share': 0.01},
{'language': 'pl-PL', 'market': 'Poland', 'revenue_share': 0.01}
]
}
# Character limit multipliers by language (some languages need more/less space)
CHAR_MULTIPLIERS = {
'en': 1.0,
'zh': 0.6, # Chinese characters are more compact
'ja': 0.7, # Japanese uses kanji
'ko': 0.8, # Korean is relatively compact
'de': 1.3, # German words are typically longer
'fr': 1.2, # French tends to be longer
'es': 1.1, # Spanish slightly longer
'pt': 1.1, # Portuguese similar to Spanish
'ru': 1.1, # Russian similar length
'ar': 1.0, # Arabic varies
'it': 1.1 # Italian similar to Spanish
}
def __init__(self, app_category: str = 'general'):
"""
Initialize localization helper.
Args:
app_category: App category to prioritize relevant markets
"""
self.app_category = app_category
self.localization_plans = []
def identify_target_markets(
self,
current_market: str = 'en-US',
budget_level: str = 'medium',
target_market_count: int = 5
) -> Dict[str, Any]:
"""
Recommend priority markets for localization.
Args:
current_market: Current/primary market
budget_level: 'low', 'medium', or 'high'
target_market_count: Number of markets to target
Returns:
Prioritized market recommendations
"""
# Determine tier priorities based on budget
if budget_level == 'low':
priority_tiers = ['tier_1']
max_markets = min(target_market_count, 3)
elif budget_level == 'medium':
priority_tiers = ['tier_1', 'tier_2']
max_markets = min(target_market_count, 8)
else: # high budget
priority_tiers = ['tier_1', 'tier_2', 'tier_3']
max_markets = target_market_count
# Collect markets from priority tiers
recommended_markets = []
for tier in priority_tiers:
for market in self.PRIORITY_MARKETS[tier]:
if market['language'] != current_market:
recommended_markets.append({
**market,
'tier': tier,
'estimated_translation_cost': self._estimate_translation_cost(
market['language']
)
})
# Sort by revenue share and limit
recommended_markets.sort(key=lambda x: x['revenue_share'], reverse=True)
recommended_markets = recommended_markets[:max_markets]
# Calculate potential ROI
total_potential_revenue_share = sum(m['revenue_share'] for m in recommended_markets)
return {
'recommended_markets': recommended_markets,
'total_markets': len(recommended_markets),
'estimated_total_revenue_lift': f"{total_potential_revenue_share*100:.1f}%",
'estimated_cost': self._estimate_total_localization_cost(recommended_markets),
'implementation_priority': self._prioritize_implementation(recommended_markets)
}
def translate_metadata(
self,
source_metadata: Dict[str, str],
source_language: str,
target_language: str,
platform: str = 'apple'
) -> Dict[str, Any]:
"""
Generate localized metadata with character limit considerations.
Args:
source_metadata: Original metadata (title, description, etc.)
source_language: Source language code (e.g., 'en')
target_language: Target language code (e.g., 'es')
platform: 'apple' or 'google'
Returns:
Localized metadata with character limit validation
"""
# Get character multiplier
target_lang_code = target_language.split('-')[0]
char_multiplier = self.CHAR_MULTIPLIERS.get(target_lang_code, 1.0)
# Platform-specific limits
if platform == 'apple':
limits = {'title': 30, 'subtitle': 30, 'description': 4000, 'keywords': 100}
else:
limits = {'title': 50, 'short_description': 80, 'description': 4000}
localized_metadata = {}
warnings = []
for field, text in source_metadata.items():
if field not in limits:
continue
# Estimate target length
estimated_length = int(len(text) * char_multiplier)
limit = limits[field]
localized_metadata[field] = {
'original_text': text,
'original_length': len(text),
'estimated_target_length': estimated_length,
'character_limit': limit,
'fits_within_limit': estimated_length <= limit,
'translation_notes': self._get_translation_notes(
field,
target_language,
estimated_length,
limit
)
}
if estimated_length > limit:
warnings.append(
f"{field}: Estimated length ({estimated_length}) may exceed limit ({limit}) - "
f"condensing may be required"
)
return {
'source_language': source_language,
'target_language': target_language,
'platform': platform,
'localized_fields': localized_metadata,
'character_multiplier': char_multiplier,
'warnings': warnings,
'recommendations': self._generate_translation_recommendations(
target_language,
warnings
)
}
def adapt_keywords(
self,
source_keywords: List[str],
source_language: str,
target_language: str,
target_market: str
) -> Dict[str, Any]:
"""
Adapt keywords for target market (not just direct translation).
Args:
source_keywords: Original keywords
source_language: Source language code
target_language: Target language code
target_market: Target market (e.g., 'France', 'Japan')
Returns:
Adapted keyword recommendations
"""
# Cultural adaptation considerations
cultural_notes = self._get_cultural_keyword_considerations(target_market)
# Search behavior differences
search_patterns = self._get_search_patterns(target_market)
adapted_keywords = []
for keyword in source_keywords:
adapted_keywords.append({
'source_keyword': keyword,
'adaptation_strategy': self._determine_adaptation_strategy(
keyword,
target_market
),
'cultural_considerations': cultural_notes.get(keyword, []),
'priority': 'high' if keyword in source_keywords[:3] else 'medium'
})
return {
'source_language': source_language,
'target_language': target_language,
'target_market': target_market,
'adapted_keywords': adapted_keywords,
'search_behavior_notes': search_patterns,
'recommendations': [
'Use native speakers for keyword research',
'Test keywords with local users before finalizing',
'Consider local competitors\' keyword strategies',
'Monitor search trends in target market'
]
}
def validate_translations(
self,
translated_metadata: Dict[str, str],
target_language: str,
platform: str = 'apple'
) -> Dict[str, Any]:
"""
Validate translated metadata for character limits and quality.
Args:
translated_metadata: Translated text fields
target_language: Target language code
platform: 'apple' or 'google'
Returns:
Validation report
"""
# Platform limits
if platform == 'apple':
limits = {'title': 30, 'subtitle': 30, 'description': 4000, 'keywords': 100}
else:
limits = {'title': 50, 'short_description': 80, 'description': 4000}
validation_results = {
'is_valid': True,
'field_validations': {},
'errors': [],
'warnings': []
}
for field, text in translated_metadata.items():
if field not in limits:
continue
actual_length = len(text)
limit = limits[field]
is_within_limit = actual_length <= limit
validation_results['field_validations'][field] = {
'text': text,
'length': actual_length,
'limit': limit,
'is_valid': is_within_limit,
'usage_percentage': round((actual_length / limit) * 100, 1)
}
if not is_within_limit:
validation_results['is_valid'] = False
validation_results['errors'].append(
f"{field} exceeds limit: {actual_length}/{limit} characters"
)
# Quality checks
quality_issues = self._check_translation_quality(
translated_metadata,
target_language
)
validation_results['quality_checks'] = quality_issues
if quality_issues:
validation_results['warnings'].extend(
[f"Quality issue: {issue}" for issue in quality_issues]
)
return validation_results
def calculate_localization_roi(
self,
target_markets: List[str],
current_monthly_downloads: int,
localization_cost: float,
expected_lift_percentage: float = 0.15
) -> Dict[str, Any]:
"""
Estimate ROI of localization investment.
Args:
target_markets: List of market codes
current_monthly_downloads: Current monthly downloads
localization_cost: Total cost to localize
expected_lift_percentage: Expected download increase (default 15%)
Returns:
ROI analysis
"""
# Estimate market-specific lift
market_data = []
total_expected_lift = 0
for market_code in target_markets:
# Find market in priority lists
market_info = None
for tier_name, markets in self.PRIORITY_MARKETS.items():
for m in markets:
if m['language'] == market_code:
market_info = m
break
if not market_info:
continue
# Estimate downloads from this market
market_downloads = int(current_monthly_downloads * market_info['revenue_share'])
expected_increase = int(market_downloads * expected_lift_percentage)
total_expected_lift += expected_increase
market_data.append({
'market': market_info['market'],
'current_monthly_downloads': market_downloads,
'expected_increase': expected_increase,
'revenue_potential': market_info['revenue_share']
})
# Calculate payback period (assuming $2 revenue per download)
revenue_per_download = 2.0
monthly_additional_revenue = total_expected_lift * revenue_per_download
payback_months = (localization_cost / monthly_additional_revenue) if monthly_additional_revenue > 0 else float('inf')
return {
'markets_analyzed': len(market_data),
'market_breakdown': market_data,
'total_expected_monthly_lift': total_expected_lift,
'expected_monthly_revenue_increase': f",.2f",
'localization_cost': f",.2f",
'payback_period_months': round(payback_months, 1) if payback_months != float('inf') else 'N/A',
'annual_roi': f"{((monthly_additional_revenue * 12 - localization_cost) / localization_cost * 100):.1f}%" if payback_months != float('inf') else 'Negative',
'recommendation': self._generate_roi_recommendation(payback_months)
}
def _estimate_translation_cost(self, language: str) -> Dict[str, float]:
"""Estimate translation cost for a language."""
# Base cost per word (professional translation)
base_cost_per_word = 0.12
# Language-specific multipliers
multipliers = {
'zh-CN': 1.5, # Chinese requires specialist
'ja-JP': 1.5, # Japanese requires specialist
'ko-KR': 1.3,
'ar-SA': 1.4, # Arabic (right-to-left)
'default': 1.0
}
multiplier = multipliers.get(language, multipliers['default'])
# Typical word counts for app store metadata
typical_word_counts = {
'title': 5,
'subtitle': 5,
'description': 300,
'keywords': 20,
'screenshots': 50 # Caption text
}
total_words = sum(typical_word_counts.values())
estimated_cost = total_words * base_cost_per_word * multiplier
return {
'cost_per_word': base_cost_per_word * multiplier,
'total_words': total_words,
'estimated_cost': round(estimated_cost, 2)
}
def _estimate_total_localization_cost(self, markets: List[Dict[str, Any]]) -> str:
"""Estimate total cost for multiple markets."""
total = sum(m['estimated_translation_cost']['estimated_cost'] for m in markets)
return f",.2f"
def _prioritize_implementation(self, markets: List[Dict[str, Any]]) -> List[Dict[str, str]]:
"""Create phased implementation plan."""
phases = []
# Phase 1: Top revenue markets
phase_1 = [m for m in markets[:3]]
if phase_1:
phases.append({
'phase': 'Phase 1 (First 30 days)',
'markets': ', '.join([m['market'] for m in phase_1]),
'rationale': 'Highest revenue potential markets'
})
# Phase 2: Remaining tier 1 and top tier 2
phase_2 = [m for m in markets[3:6]]
if phase_2:
phases.append({
'phase': 'Phase 2 (Days 31-60)',
'markets': ', '.join([m['market'] for m in phase_2]),
'rationale': 'Strong revenue markets with good ROI'
})
# Phase 3: Remaining markets
phase_3 = [m for m in markets[6:]]
if phase_3:
phases.append({
'phase': 'Phase 3 (Days 61-90)',
'markets': ', '.join([m['market'] for m in phase_3]),
'rationale': 'Complete global coverage'
})
return phases
def _get_translation_notes(
self,
field: str,
target_language: str,
estimated_length: int,
limit: int
) -> List[str]:
"""Get translation-specific notes for field."""
notes = []
if estimated_length > limit:
notes.append(f"Condensing required - aim for {limit - 10} characters to allow buffer")
if field == 'title' and target_language.startswith('zh'):
notes.append("Chinese characters convey more meaning - may need fewer characters")
if field == 'keywords' and target_language.startswith('de'):
notes.append("German compound words may be longer - prioritize shorter keywords")
return notes
def _generate_translation_recommendations(
self,
target_language: str,
warnings: List[str]
) -> List[str]:
"""Generate translation recommendations."""
recommendations = [
"Use professional native speakers for translation",
"Test translations with local users before finalizing"
]
if warnings:
recommendations.append("Work with translator to condense text while preserving meaning")
if target_language.startswith('zh') or target_language.startswith('ja'):
recommendations.append("Consider cultural context and local idioms")
return recommendations
def _get_cultural_keyword_considerations(self, target_market: str) -> Dict[str, List[str]]:
"""Get cultural considerations for keywords by market."""
# Simplified example - real implementation would be more comprehensive
considerations = {
'China': ['Avoid politically sensitive terms', 'Consider local alternatives to blocked services'],
'Japan': ['Honorific language important', 'Technical terms often use katakana'],
'Germany': ['Privacy and security terms resonate', 'Efficiency and quality valued'],
'France': ['French language protection laws', 'Prefer French terms over English'],
'default': ['Research local search behavior', 'Test with native speakers']
}
return considerations.get(target_market, considerations['default'])
def _get_search_patterns(self, target_market: str) -> List[str]:
"""Get search pattern notes for market."""
patterns = {
'China': ['Use both simplified characters and romanization', 'Brand names often romanized'],
'Japan': ['Mix of kanji, hiragana, and katakana', 'English words common in tech'],
'Germany': ['Compound words common', 'Specific technical terminology'],
'default': ['Research local search trends', 'Monitor competitor keywords']
}
return patterns.get(target_market, patterns['default'])
def _determine_adaptation_strategy(self, keyword: str, target_market: str) -> str:
"""Determine how to adapt keyword for market."""
# Simplified logic
if target_market in ['China', 'Japan', 'Korea']:
return 'full_localization' # Complete translation needed
elif target_market in ['Germany', 'France', 'Spain']:
return 'adapt_and_translate' # Some adaptation needed
else:
return 'direct_translation' # Direct translation usually sufficient
def _check_translation_quality(
self,
translated_metadata: Dict[str, str],
target_language: str
) -> List[str]:
"""Basic quality checks for translations."""
issues = []
# Check for untranslated placeholders
for field, text in translated_metadata.items():
if '[' in text or '{' in text or 'TODO' in text.upper():
issues.append(f"{field} contains placeholder text")
# Check for excessive punctuation
for field, text in translated_metadata.items():
if text.count('!') > 3:
issues.append(f"{field} has excessive exclamation marks")
return issues
def _generate_roi_recommendation(self, payback_months: float) -> str:
"""Generate ROI recommendation."""
if payback_months <= 3:
return "Excellent ROI - proceed immediately"
elif payback_months <= 6:
return "Good ROI - recommended investment"
elif payback_months <= 12:
return "Moderate ROI - consider if strategic market"
else:
return "Low ROI - reconsider or focus on higher-priority markets first"
def plan_localization_strategy(
current_market: str,
budget_level: str,
monthly_downloads: int
) -> Dict[str, Any]:
"""
Convenience function to plan localization strategy.
Args:
current_market: Current market code
budget_level: Budget level
monthly_downloads: Current monthly downloads
Returns:
Complete localization plan
"""
helper = LocalizationHelper()
target_markets = helper.identify_target_markets(
current_market=current_market,
budget_level=budget_level
)
# Extract market codes
market_codes = [m['language'] for m in target_markets['recommended_markets']]
# Calculate ROI
estimated_cost = float(target_markets['estimated_cost'].replace('$', '').replace(',', ''))
roi_analysis = helper.calculate_localization_roi(
market_codes,
monthly_downloads,
estimated_cost
)
return {
'target_markets': target_markets,
'roi_analysis': roi_analysis
}
FILE:scripts/metadata_optimizer.py
"""
Metadata optimization module for App Store Optimization.
Optimizes titles, descriptions, and keyword fields with platform-specific character limit validation.
"""
from typing import Dict, List, Any, Optional, Tuple
import re
class MetadataOptimizer:
"""Optimizes app store metadata for maximum discoverability and conversion."""
# Platform-specific character limits
CHAR_LIMITS = {
'apple': {
'title': 30,
'subtitle': 30,
'promotional_text': 170,
'description': 4000,
'keywords': 100,
'whats_new': 4000
},
'google': {
'title': 50,
'short_description': 80,
'full_description': 4000
}
}
def __init__(self, platform: str = 'apple'):
"""
Initialize metadata optimizer.
Args:
platform: 'apple' or 'google'
"""
if platform not in ['apple', 'google']:
raise ValueError("Platform must be 'apple' or 'google'")
self.platform = platform
self.limits = self.CHAR_LIMITS[platform]
def optimize_title(
self,
app_name: str,
target_keywords: List[str],
include_brand: bool = True
) -> Dict[str, Any]:
"""
Optimize app title with keyword integration.
Args:
app_name: Your app's brand name
target_keywords: List of keywords to potentially include
include_brand: Whether to include brand name
Returns:
Optimized title options with analysis
"""
max_length = self.limits['title']
title_options = []
# Option 1: Brand name only
if include_brand:
option1 = app_name[:max_length]
title_options.append({
'title': option1,
'length': len(option1),
'remaining_chars': max_length - len(option1),
'keywords_included': [],
'strategy': 'brand_only',
'pros': ['Maximum brand recognition', 'Clean and simple'],
'cons': ['No keyword targeting', 'Lower discoverability']
})
# Option 2: Brand + Primary Keyword
if target_keywords:
primary_keyword = target_keywords[0]
option2 = self._build_title_with_keywords(
app_name,
[primary_keyword],
max_length
)
if option2:
title_options.append({
'title': option2,
'length': len(option2),
'remaining_chars': max_length - len(option2),
'keywords_included': [primary_keyword],
'strategy': 'brand_plus_primary',
'pros': ['Targets main keyword', 'Maintains brand identity'],
'cons': ['Limited keyword coverage']
})
# Option 3: Brand + Multiple Keywords (if space allows)
if len(target_keywords) > 1:
option3 = self._build_title_with_keywords(
app_name,
target_keywords[:2],
max_length
)
if option3:
title_options.append({
'title': option3,
'length': len(option3),
'remaining_chars': max_length - len(option3),
'keywords_included': target_keywords[:2],
'strategy': 'brand_plus_multiple',
'pros': ['Multiple keyword targets', 'Better discoverability'],
'cons': ['May feel cluttered', 'Less brand focus']
})
# Option 4: Keyword-first approach (for new apps)
if target_keywords and not include_brand:
option4 = " ".join(target_keywords[:2])[:max_length]
title_options.append({
'title': option4,
'length': len(option4),
'remaining_chars': max_length - len(option4),
'keywords_included': target_keywords[:2],
'strategy': 'keyword_first',
'pros': ['Maximum SEO benefit', 'Clear functionality'],
'cons': ['No brand recognition', 'Generic appearance']
})
return {
'platform': self.platform,
'max_length': max_length,
'options': title_options,
'recommendation': self._recommend_title_option(title_options)
}
def optimize_description(
self,
app_info: Dict[str, Any],
target_keywords: List[str],
description_type: str = 'full'
) -> Dict[str, Any]:
"""
Optimize app description with keyword integration and conversion focus.
Args:
app_info: Dict with 'name', 'key_features', 'unique_value', 'target_audience'
target_keywords: List of keywords to integrate naturally
description_type: 'full', 'short' (Google), 'subtitle' (Apple)
Returns:
Optimized description with analysis
"""
if description_type == 'short' and self.platform == 'google':
return self._optimize_short_description(app_info, target_keywords)
elif description_type == 'subtitle' and self.platform == 'apple':
return self._optimize_subtitle(app_info, target_keywords)
else:
return self._optimize_full_description(app_info, target_keywords)
def optimize_keyword_field(
self,
target_keywords: List[str],
app_title: str = "",
app_description: str = ""
) -> Dict[str, Any]:
"""
Optimize Apple's 100-character keyword field.
Rules:
- No spaces between commas
- No plural forms if singular exists
- No duplicates
- Keywords in title/subtitle are already indexed
Args:
target_keywords: List of target keywords
app_title: Current app title (to avoid duplication)
app_description: Current description (to check coverage)
Returns:
Optimized keyword field (comma-separated, no spaces)
"""
if self.platform != 'apple':
return {'error': 'Keyword field optimization only applies to Apple App Store'}
max_length = self.limits['keywords']
# Extract words already in title (these don't need to be in keyword field)
title_words = set(app_title.lower().split()) if app_title else set()
# Process keywords
processed_keywords = []
for keyword in target_keywords:
keyword_lower = keyword.lower().strip()
# Skip if already in title
if keyword_lower in title_words:
continue
# Remove duplicates and process
words = keyword_lower.split()
for word in words:
if word not in processed_keywords and word not in title_words:
processed_keywords.append(word)
# Remove plurals if singular exists
deduplicated = self._remove_plural_duplicates(processed_keywords)
# Build keyword field within 100 character limit
keyword_field = self._build_keyword_field(deduplicated, max_length)
# Calculate keyword density in description
density = self._calculate_coverage(target_keywords, app_description)
return {
'keyword_field': keyword_field,
'length': len(keyword_field),
'remaining_chars': max_length - len(keyword_field),
'keywords_included': keyword_field.split(','),
'keywords_count': len(keyword_field.split(',')),
'keywords_excluded': [kw for kw in target_keywords if kw.lower() not in keyword_field],
'description_coverage': density,
'optimization_tips': [
'Keywords in title are auto-indexed - no need to repeat',
'Use singular forms only (Apple indexes plurals automatically)',
'No spaces between commas to maximize character usage',
'Update keyword field with each app update to test variations'
]
}
def validate_character_limits(
self,
metadata: Dict[str, str]
) -> Dict[str, Any]:
"""
Validate all metadata fields against platform character limits.
Args:
metadata: Dictionary of field_name: value
Returns:
Validation report with errors and warnings
"""
validation_results = {
'is_valid': True,
'errors': [],
'warnings': [],
'field_status': {}
}
for field_name, value in metadata.items():
if field_name not in self.limits:
validation_results['warnings'].append(
f"Unknown field '{field_name}' for {self.platform} platform"
)
continue
max_length = self.limits[field_name]
actual_length = len(value)
remaining = max_length - actual_length
field_status = {
'value': value,
'length': actual_length,
'limit': max_length,
'remaining': remaining,
'is_valid': actual_length <= max_length,
'usage_percentage': round((actual_length / max_length) * 100, 1)
}
validation_results['field_status'][field_name] = field_status
if actual_length > max_length:
validation_results['is_valid'] = False
validation_results['errors'].append(
f"'{field_name}' exceeds limit: {actual_length}/{max_length} chars"
)
elif remaining > max_length * 0.2: # More than 20% unused
validation_results['warnings'].append(
f"'{field_name}' under-utilizes space: {remaining} chars remaining"
)
return validation_results
def calculate_keyword_density(
self,
text: str,
target_keywords: List[str]
) -> Dict[str, Any]:
"""
Calculate keyword density in text.
Args:
text: Text to analyze
target_keywords: Keywords to check
Returns:
Density analysis
"""
text_lower = text.lower()
total_words = len(text_lower.split())
keyword_densities = {}
for keyword in target_keywords:
keyword_lower = keyword.lower()
count = text_lower.count(keyword_lower)
density = (count / total_words * 100) if total_words > 0 else 0
keyword_densities[keyword] = {
'occurrences': count,
'density_percentage': round(density, 2),
'status': self._assess_density(density)
}
# Overall assessment
total_keyword_occurrences = sum(kw['occurrences'] for kw in keyword_densities.values())
overall_density = (total_keyword_occurrences / total_words * 100) if total_words > 0 else 0
return {
'total_words': total_words,
'keyword_densities': keyword_densities,
'overall_keyword_density': round(overall_density, 2),
'assessment': self._assess_overall_density(overall_density),
'recommendations': self._generate_density_recommendations(keyword_densities)
}
def _build_title_with_keywords(
self,
app_name: str,
keywords: List[str],
max_length: int
) -> Optional[str]:
"""Build title combining app name and keywords within limit."""
separators = [' - ', ': ', ' | ']
for sep in separators:
for kw in keywords:
title = f"{app_name}{sep}{kw}"
if len(title) <= max_length:
return title
return None
def _optimize_short_description(
self,
app_info: Dict[str, Any],
target_keywords: List[str]
) -> Dict[str, Any]:
"""Optimize Google Play short description (80 chars)."""
max_length = self.limits['short_description']
# Focus on unique value proposition with primary keyword
unique_value = app_info.get('unique_value', '')
primary_keyword = target_keywords[0] if target_keywords else ''
# Template: [Primary Keyword] - [Unique Value]
short_desc = f"{primary_keyword.title()} - {unique_value}"[:max_length]
return {
'short_description': short_desc,
'length': len(short_desc),
'remaining_chars': max_length - len(short_desc),
'keywords_included': [primary_keyword] if primary_keyword in short_desc.lower() else [],
'strategy': 'keyword_value_proposition'
}
def _optimize_subtitle(
self,
app_info: Dict[str, Any],
target_keywords: List[str]
) -> Dict[str, Any]:
"""Optimize Apple App Store subtitle (30 chars)."""
max_length = self.limits['subtitle']
# Very concise - primary keyword or key feature
primary_keyword = target_keywords[0] if target_keywords else ''
key_feature = app_info.get('key_features', [''])[0] if app_info.get('key_features') else ''
options = [
primary_keyword[:max_length],
key_feature[:max_length],
f"{primary_keyword} App"[:max_length]
]
return {
'subtitle_options': [opt for opt in options if opt],
'max_length': max_length,
'recommendation': options[0] if options else ''
}
def _optimize_full_description(
self,
app_info: Dict[str, Any],
target_keywords: List[str]
) -> Dict[str, Any]:
"""Optimize full app description (4000 chars for both platforms)."""
max_length = self.limits.get('description', self.limits.get('full_description', 4000))
# Structure: Hook → Features → Benefits → Social Proof → CTA
sections = []
# Hook (with primary keyword)
primary_keyword = target_keywords[0] if target_keywords else ''
unique_value = app_info.get('unique_value', '')
hook = f"{unique_value} {primary_keyword.title()} that helps you achieve more.\n\n"
sections.append(hook)
# Features (with keywords naturally integrated)
features = app_info.get('key_features', [])
if features:
sections.append("KEY FEATURES:\n")
for i, feature in enumerate(features[:5], 1):
# Integrate keywords naturally
feature_text = f"• {feature}"
if i <= len(target_keywords):
keyword = target_keywords[i-1]
if keyword.lower() not in feature.lower():
feature_text = f"• {feature} with {keyword}"
sections.append(f"{feature_text}\n")
sections.append("\n")
# Benefits
target_audience = app_info.get('target_audience', 'users')
sections.append(f"PERFECT FOR:\n{target_audience}\n\n")
# Social proof placeholder
sections.append("WHY USERS LOVE US:\n")
sections.append("Join thousands of satisfied users who have transformed their workflow.\n\n")
# CTA
sections.append("Download now and start experiencing the difference!")
# Combine and validate length
full_description = "".join(sections)
if len(full_description) > max_length:
full_description = full_description[:max_length-3] + "..."
# Calculate keyword density
density = self.calculate_keyword_density(full_description, target_keywords)
return {
'full_description': full_description,
'length': len(full_description),
'remaining_chars': max_length - len(full_description),
'keyword_analysis': density,
'structure': {
'has_hook': True,
'has_features': len(features) > 0,
'has_benefits': True,
'has_cta': True
}
}
def _remove_plural_duplicates(self, keywords: List[str]) -> List[str]:
"""Remove plural forms if singular exists."""
deduplicated = []
singular_set = set()
for keyword in keywords:
if keyword.endswith('s') and len(keyword) > 1:
singular = keyword[:-1]
if singular not in singular_set:
deduplicated.append(singular)
singular_set.add(singular)
else:
if keyword not in singular_set:
deduplicated.append(keyword)
singular_set.add(keyword)
return deduplicated
def _build_keyword_field(self, keywords: List[str], max_length: int) -> str:
"""Build comma-separated keyword field within character limit."""
keyword_field = ""
for keyword in keywords:
test_field = f"{keyword_field},{keyword}" if keyword_field else keyword
if len(test_field) <= max_length:
keyword_field = test_field
else:
break
return keyword_field
def _calculate_coverage(self, keywords: List[str], text: str) -> Dict[str, int]:
"""Calculate how many keywords are covered in text."""
text_lower = text.lower()
coverage = {}
for keyword in keywords:
coverage[keyword] = text_lower.count(keyword.lower())
return coverage
def _assess_density(self, density: float) -> str:
"""Assess individual keyword density."""
if density < 0.5:
return "too_low"
elif density <= 2.5:
return "optimal"
else:
return "too_high"
def _assess_overall_density(self, density: float) -> str:
"""Assess overall keyword density."""
if density < 2:
return "Under-optimized: Consider adding more keyword variations"
elif density <= 5:
return "Optimal: Good keyword integration without stuffing"
elif density <= 8:
return "High: Approaching keyword stuffing - reduce keyword usage"
else:
return "Too High: Keyword stuffing detected - rewrite for natural flow"
def _generate_density_recommendations(
self,
keyword_densities: Dict[str, Dict[str, Any]]
) -> List[str]:
"""Generate recommendations based on keyword density analysis."""
recommendations = []
for keyword, data in keyword_densities.items():
if data['status'] == 'too_low':
recommendations.append(
f"Increase usage of '{keyword}' - currently only {data['occurrences']} times"
)
elif data['status'] == 'too_high':
recommendations.append(
f"Reduce usage of '{keyword}' - appears {data['occurrences']} times (keyword stuffing risk)"
)
if not recommendations:
recommendations.append("Keyword density is well-balanced")
return recommendations
def _recommend_title_option(self, options: List[Dict[str, Any]]) -> str:
"""Recommend best title option based on strategy."""
if not options:
return "No valid options available"
# Prefer brand_plus_primary for established apps
for option in options:
if option['strategy'] == 'brand_plus_primary':
return f"Recommended: '{option['title']}' (Balance of brand and SEO)"
# Fallback to first option
return f"Recommended: '{options[0]['title']}' ({options[0]['strategy']})"
def optimize_app_metadata(
platform: str,
app_info: Dict[str, Any],
target_keywords: List[str]
) -> Dict[str, Any]:
"""
Convenience function to optimize all metadata fields.
Args:
platform: 'apple' or 'google'
app_info: App information dictionary
target_keywords: Target keywords list
Returns:
Complete metadata optimization package
"""
optimizer = MetadataOptimizer(platform)
return {
'platform': platform,
'title': optimizer.optimize_title(
app_info['name'],
target_keywords
),
'description': optimizer.optimize_description(
app_info,
target_keywords,
'full'
),
'keyword_field': optimizer.optimize_keyword_field(
target_keywords
) if platform == 'apple' else None
}
FILE:scripts/review_analyzer.py
"""
Review analysis module for App Store Optimization.
Analyzes user reviews for sentiment, issues, and feature requests.
"""
from typing import Dict, List, Any, Optional, Tuple
from collections import Counter
import re
class ReviewAnalyzer:
"""Analyzes user reviews for actionable insights."""
# Sentiment keywords
POSITIVE_KEYWORDS = [
'great', 'awesome', 'excellent', 'amazing', 'love', 'best', 'perfect',
'fantastic', 'wonderful', 'brilliant', 'outstanding', 'superb'
]
NEGATIVE_KEYWORDS = [
'bad', 'terrible', 'awful', 'horrible', 'hate', 'worst', 'useless',
'broken', 'crash', 'bug', 'slow', 'disappointing', 'frustrating'
]
# Issue indicators
ISSUE_KEYWORDS = [
'crash', 'bug', 'error', 'broken', 'not working', 'doesnt work',
'freezes', 'slow', 'laggy', 'glitch', 'problem', 'issue', 'fail'
]
# Feature request indicators
FEATURE_REQUEST_KEYWORDS = [
'wish', 'would be nice', 'should add', 'need', 'want', 'hope',
'please add', 'missing', 'lacks', 'feature request'
]
def __init__(self, app_name: str):
"""
Initialize review analyzer.
Args:
app_name: Name of the app
"""
self.app_name = app_name
self.reviews = []
self.analysis_cache = {}
def analyze_sentiment(
self,
reviews: List[Dict[str, Any]]
) -> Dict[str, Any]:
"""
Analyze sentiment across reviews.
Args:
reviews: List of review dicts with 'text', 'rating', 'date'
Returns:
Sentiment analysis summary
"""
self.reviews = reviews
sentiment_counts = {
'positive': 0,
'neutral': 0,
'negative': 0
}
detailed_sentiments = []
for review in reviews:
text = review.get('text', '').lower()
rating = review.get('rating', 3)
# Calculate sentiment score
sentiment_score = self._calculate_sentiment_score(text, rating)
sentiment_category = self._categorize_sentiment(sentiment_score)
sentiment_counts[sentiment_category] += 1
detailed_sentiments.append({
'review_id': review.get('id', ''),
'rating': rating,
'sentiment_score': sentiment_score,
'sentiment': sentiment_category,
'text_preview': text[:100] + '...' if len(text) > 100 else text
})
# Calculate percentages
total = len(reviews)
sentiment_distribution = {
'positive': round((sentiment_counts['positive'] / total) * 100, 1) if total > 0 else 0,
'neutral': round((sentiment_counts['neutral'] / total) * 100, 1) if total > 0 else 0,
'negative': round((sentiment_counts['negative'] / total) * 100, 1) if total > 0 else 0
}
# Calculate average rating
avg_rating = sum(r.get('rating', 0) for r in reviews) / total if total > 0 else 0
return {
'total_reviews_analyzed': total,
'average_rating': round(avg_rating, 2),
'sentiment_distribution': sentiment_distribution,
'sentiment_counts': sentiment_counts,
'sentiment_trend': self._assess_sentiment_trend(sentiment_distribution),
'detailed_sentiments': detailed_sentiments[:50] # Limit output
}
def extract_common_themes(
self,
reviews: List[Dict[str, Any]],
min_mentions: int = 3
) -> Dict[str, Any]:
"""
Extract frequently mentioned themes and topics.
Args:
reviews: List of review dicts
min_mentions: Minimum mentions to be considered common
Returns:
Common themes analysis
"""
# Extract all words from reviews
all_words = []
all_phrases = []
for review in reviews:
text = review.get('text', '').lower()
# Clean text
text = re.sub(r'[^\w\s]', ' ', text)
words = text.split()
# Filter out common words
stop_words = {
'the', 'and', 'for', 'with', 'this', 'that', 'from', 'have',
'app', 'apps', 'very', 'really', 'just', 'but', 'not', 'you'
}
words = [w for w in words if w not in stop_words and len(w) > 3]
all_words.extend(words)
# Extract 2-3 word phrases
for i in range(len(words) - 1):
phrase = f"{words[i]} {words[i+1]}"
all_phrases.append(phrase)
# Count frequency
word_freq = Counter(all_words)
phrase_freq = Counter(all_phrases)
# Filter by min_mentions
common_words = [
{'word': word, 'mentions': count}
for word, count in word_freq.most_common(30)
if count >= min_mentions
]
common_phrases = [
{'phrase': phrase, 'mentions': count}
for phrase, count in phrase_freq.most_common(20)
if count >= min_mentions
]
# Categorize themes
themes = self._categorize_themes(common_words, common_phrases)
return {
'common_words': common_words,
'common_phrases': common_phrases,
'identified_themes': themes,
'insights': self._generate_theme_insights(themes)
}
def identify_issues(
self,
reviews: List[Dict[str, Any]],
rating_threshold: int = 3
) -> Dict[str, Any]:
"""
Identify bugs, crashes, and other issues from reviews.
Args:
reviews: List of review dicts
rating_threshold: Only analyze reviews at or below this rating
Returns:
Issue identification report
"""
issues = []
for review in reviews:
rating = review.get('rating', 5)
if rating > rating_threshold:
continue
text = review.get('text', '').lower()
# Check for issue keywords
mentioned_issues = []
for keyword in self.ISSUE_KEYWORDS:
if keyword in text:
mentioned_issues.append(keyword)
if mentioned_issues:
issues.append({
'review_id': review.get('id', ''),
'rating': rating,
'date': review.get('date', ''),
'issue_keywords': mentioned_issues,
'text': text[:200] + '...' if len(text) > 200 else text
})
# Group by issue type
issue_frequency = Counter()
for issue in issues:
for keyword in issue['issue_keywords']:
issue_frequency[keyword] += 1
# Categorize issues
categorized_issues = self._categorize_issues(issues)
# Calculate issue severity
severity_scores = self._calculate_issue_severity(
categorized_issues,
len(reviews)
)
return {
'total_issues_found': len(issues),
'issue_frequency': dict(issue_frequency.most_common(15)),
'categorized_issues': categorized_issues,
'severity_scores': severity_scores,
'top_issues': self._rank_issues_by_severity(severity_scores),
'recommendations': self._generate_issue_recommendations(
categorized_issues,
severity_scores
)
}
def find_feature_requests(
self,
reviews: List[Dict[str, Any]]
) -> Dict[str, Any]:
"""
Extract feature requests and desired improvements.
Args:
reviews: List of review dicts
Returns:
Feature request analysis
"""
feature_requests = []
for review in reviews:
text = review.get('text', '').lower()
rating = review.get('rating', 3)
# Check for feature request indicators
is_feature_request = any(
keyword in text
for keyword in self.FEATURE_REQUEST_KEYWORDS
)
if is_feature_request:
# Extract the specific request
request_text = self._extract_feature_request_text(text)
feature_requests.append({
'review_id': review.get('id', ''),
'rating': rating,
'date': review.get('date', ''),
'request_text': request_text,
'full_review': text[:200] + '...' if len(text) > 200 else text
})
# Cluster similar requests
clustered_requests = self._cluster_feature_requests(feature_requests)
# Prioritize based on frequency and rating context
prioritized_requests = self._prioritize_feature_requests(clustered_requests)
return {
'total_feature_requests': len(feature_requests),
'clustered_requests': clustered_requests,
'prioritized_requests': prioritized_requests,
'implementation_recommendations': self._generate_feature_recommendations(
prioritized_requests
)
}
def track_sentiment_trends(
self,
reviews_by_period: Dict[str, List[Dict[str, Any]]]
) -> Dict[str, Any]:
"""
Track sentiment changes over time.
Args:
reviews_by_period: Dict of period_name: reviews
Returns:
Trend analysis
"""
trends = []
for period, reviews in reviews_by_period.items():
sentiment = self.analyze_sentiment(reviews)
trends.append({
'period': period,
'total_reviews': len(reviews),
'average_rating': sentiment['average_rating'],
'positive_percentage': sentiment['sentiment_distribution']['positive'],
'negative_percentage': sentiment['sentiment_distribution']['negative']
})
# Calculate trend direction
if len(trends) >= 2:
first_period = trends[0]
last_period = trends[-1]
rating_change = last_period['average_rating'] - first_period['average_rating']
sentiment_change = last_period['positive_percentage'] - first_period['positive_percentage']
trend_direction = self._determine_trend_direction(
rating_change,
sentiment_change
)
else:
trend_direction = 'insufficient_data'
return {
'periods_analyzed': len(trends),
'trend_data': trends,
'trend_direction': trend_direction,
'insights': self._generate_trend_insights(trends, trend_direction)
}
def generate_response_templates(
self,
issue_category: str
) -> List[Dict[str, str]]:
"""
Generate response templates for common review scenarios.
Args:
issue_category: Category of issue ('crash', 'feature_request', 'positive', etc.)
Returns:
Response templates
"""
templates = {
'crash': [
{
'scenario': 'App crash reported',
'template': "Thank you for bringing this to our attention. We're sorry you experienced a crash. "
"Our team is investigating this issue. Could you please share more details about when "
"this occurred (device model, iOS/Android version) by contacting support@[company].com? "
"We're committed to fixing this quickly."
},
{
'scenario': 'Crash already fixed',
'template': "Thank you for your feedback. We've identified and fixed this crash issue in version [X.X]. "
"Please update to the latest version. If the problem persists, please reach out to "
"support@[company].com and we'll help you directly."
}
],
'bug': [
{
'scenario': 'Bug reported',
'template': "Thanks for reporting this bug. We take these issues seriously. Our team is looking into it "
"and we'll have a fix in an upcoming update. We appreciate your patience and will notify you "
"when it's resolved."
}
],
'feature_request': [
{
'scenario': 'Feature request received',
'template': "Thank you for this suggestion! We're always looking to improve [app_name]. We've added your "
"request to our roadmap and will consider it for a future update. Follow us @[social] for "
"updates on new features."
},
{
'scenario': 'Feature already planned',
'template': "Great news! This feature is already on our roadmap and we're working on it. Stay tuned for "
"updates in the coming months. Thanks for your feedback!"
}
],
'positive': [
{
'scenario': 'Positive review',
'template': "Thank you so much for your kind words! We're thrilled that you're enjoying [app_name]. "
"Reviews like yours motivate our team to keep improving. If you ever have suggestions, "
"we'd love to hear them!"
}
],
'negative_general': [
{
'scenario': 'General complaint',
'template': "We're sorry to hear you're not satisfied with your experience. We'd like to make this right. "
"Please contact us at support@[company].com so we can understand the issue better and help "
"you directly. Thank you for giving us a chance to improve."
}
]
}
return templates.get(issue_category, templates['negative_general'])
def _calculate_sentiment_score(self, text: str, rating: int) -> float:
"""Calculate sentiment score (-1 to 1)."""
# Start with rating-based score
rating_score = (rating - 3) / 2 # Convert 1-5 to -1 to 1
# Adjust based on text sentiment
positive_count = sum(1 for keyword in self.POSITIVE_KEYWORDS if keyword in text)
negative_count = sum(1 for keyword in self.NEGATIVE_KEYWORDS if keyword in text)
text_score = (positive_count - negative_count) / 10 # Normalize
# Weighted average (60% rating, 40% text)
final_score = (rating_score * 0.6) + (text_score * 0.4)
return max(min(final_score, 1.0), -1.0)
def _categorize_sentiment(self, score: float) -> str:
"""Categorize sentiment score."""
if score > 0.3:
return 'positive'
elif score < -0.3:
return 'negative'
else:
return 'neutral'
def _assess_sentiment_trend(self, distribution: Dict[str, float]) -> str:
"""Assess overall sentiment trend."""
positive = distribution['positive']
negative = distribution['negative']
if positive > 70:
return 'very_positive'
elif positive > 50:
return 'positive'
elif negative > 30:
return 'concerning'
elif negative > 50:
return 'critical'
else:
return 'mixed'
def _categorize_themes(
self,
common_words: List[Dict[str, Any]],
common_phrases: List[Dict[str, Any]]
) -> Dict[str, List[str]]:
"""Categorize themes from words and phrases."""
themes = {
'features': [],
'performance': [],
'usability': [],
'support': [],
'pricing': []
}
# Keywords for each category
feature_keywords = {'feature', 'functionality', 'option', 'tool'}
performance_keywords = {'fast', 'slow', 'crash', 'lag', 'speed', 'performance'}
usability_keywords = {'easy', 'difficult', 'intuitive', 'confusing', 'interface', 'design'}
support_keywords = {'support', 'help', 'customer', 'service', 'response'}
pricing_keywords = {'price', 'cost', 'expensive', 'cheap', 'subscription', 'free'}
for word_data in common_words:
word = word_data['word']
if any(kw in word for kw in feature_keywords):
themes['features'].append(word)
elif any(kw in word for kw in performance_keywords):
themes['performance'].append(word)
elif any(kw in word for kw in usability_keywords):
themes['usability'].append(word)
elif any(kw in word for kw in support_keywords):
themes['support'].append(word)
elif any(kw in word for kw in pricing_keywords):
themes['pricing'].append(word)
return {k: v for k, v in themes.items() if v} # Remove empty categories
def _generate_theme_insights(self, themes: Dict[str, List[str]]) -> List[str]:
"""Generate insights from themes."""
insights = []
for category, keywords in themes.items():
if keywords:
insights.append(
f"{category.title()}: Users frequently mention {', '.join(keywords[:3])}"
)
return insights[:5]
def _categorize_issues(self, issues: List[Dict[str, Any]]) -> Dict[str, List[Dict[str, Any]]]:
"""Categorize issues by type."""
categories = {
'crashes': [],
'bugs': [],
'performance': [],
'compatibility': []
}
for issue in issues:
keywords = issue['issue_keywords']
if 'crash' in keywords or 'freezes' in keywords:
categories['crashes'].append(issue)
elif 'bug' in keywords or 'error' in keywords or 'broken' in keywords:
categories['bugs'].append(issue)
elif 'slow' in keywords or 'laggy' in keywords:
categories['performance'].append(issue)
else:
categories['compatibility'].append(issue)
return {k: v for k, v in categories.items() if v}
def _calculate_issue_severity(
self,
categorized_issues: Dict[str, List[Dict[str, Any]]],
total_reviews: int
) -> Dict[str, Dict[str, Any]]:
"""Calculate severity scores for each issue category."""
severity_scores = {}
for category, issues in categorized_issues.items():
count = len(issues)
percentage = (count / total_reviews) * 100 if total_reviews > 0 else 0
# Calculate average rating of affected reviews
avg_rating = sum(i['rating'] for i in issues) / count if count > 0 else 0
# Severity score (0-100)
severity = min((percentage * 10) + ((5 - avg_rating) * 10), 100)
severity_scores[category] = {
'count': count,
'percentage': round(percentage, 2),
'average_rating': round(avg_rating, 2),
'severity_score': round(severity, 1),
'priority': 'critical' if severity > 70 else ('high' if severity > 40 else 'medium')
}
return severity_scores
def _rank_issues_by_severity(
self,
severity_scores: Dict[str, Dict[str, Any]]
) -> List[Dict[str, Any]]:
"""Rank issues by severity score."""
ranked = sorted(
[{'category': cat, **data} for cat, data in severity_scores.items()],
key=lambda x: x['severity_score'],
reverse=True
)
return ranked
def _generate_issue_recommendations(
self,
categorized_issues: Dict[str, List[Dict[str, Any]]],
severity_scores: Dict[str, Dict[str, Any]]
) -> List[str]:
"""Generate recommendations for addressing issues."""
recommendations = []
for category, score_data in severity_scores.items():
if score_data['priority'] == 'critical':
recommendations.append(
f"URGENT: Address {category} issues immediately - affecting {score_data['percentage']}% of reviews"
)
elif score_data['priority'] == 'high':
recommendations.append(
f"HIGH PRIORITY: Focus on {category} issues in next update"
)
return recommendations
def _extract_feature_request_text(self, text: str) -> str:
"""Extract the specific feature request from review text."""
# Simple extraction - find sentence with feature request keywords
sentences = text.split('.')
for sentence in sentences:
if any(keyword in sentence for keyword in self.FEATURE_REQUEST_KEYWORDS):
return sentence.strip()
return text[:100] # Fallback
def _cluster_feature_requests(
self,
feature_requests: List[Dict[str, Any]]
) -> List[Dict[str, Any]]:
"""Cluster similar feature requests."""
# Simplified clustering - group by common keywords
clusters = {}
for request in feature_requests:
text = request['request_text'].lower()
# Extract key words
words = [w for w in text.split() if len(w) > 4]
# Try to find matching cluster
matched = False
for cluster_key in clusters:
if any(word in cluster_key for word in words[:3]):
clusters[cluster_key].append(request)
matched = True
break
if not matched and words:
cluster_key = ' '.join(words[:2])
clusters[cluster_key] = [request]
return [
{'feature_theme': theme, 'request_count': len(requests), 'examples': requests[:3]}
for theme, requests in clusters.items()
]
def _prioritize_feature_requests(
self,
clustered_requests: List[Dict[str, Any]]
) -> List[Dict[str, Any]]:
"""Prioritize feature requests by frequency."""
return sorted(
clustered_requests,
key=lambda x: x['request_count'],
reverse=True
)[:10]
def _generate_feature_recommendations(
self,
prioritized_requests: List[Dict[str, Any]]
) -> List[str]:
"""Generate recommendations for feature requests."""
recommendations = []
if prioritized_requests:
top_request = prioritized_requests[0]
recommendations.append(
f"Most requested feature: {top_request['feature_theme']} "
f"({top_request['request_count']} mentions) - consider for next major release"
)
if len(prioritized_requests) > 1:
recommendations.append(
f"Also consider: {prioritized_requests[1]['feature_theme']}"
)
return recommendations
def _determine_trend_direction(
self,
rating_change: float,
sentiment_change: float
) -> str:
"""Determine overall trend direction."""
if rating_change > 0.2 and sentiment_change > 5:
return 'improving'
elif rating_change < -0.2 and sentiment_change < -5:
return 'declining'
else:
return 'stable'
def _generate_trend_insights(
self,
trends: List[Dict[str, Any]],
trend_direction: str
) -> List[str]:
"""Generate insights from trend analysis."""
insights = []
if trend_direction == 'improving':
insights.append("Positive trend: User satisfaction is increasing over time")
elif trend_direction == 'declining':
insights.append("WARNING: User satisfaction is declining - immediate action needed")
else:
insights.append("Sentiment is stable - maintain current quality")
# Review velocity insight
if len(trends) >= 2:
recent_reviews = trends[-1]['total_reviews']
previous_reviews = trends[-2]['total_reviews']
if recent_reviews > previous_reviews * 1.5:
insights.append("Review volume increasing - growing user base or recent controversy")
return insights
def analyze_reviews(
app_name: str,
reviews: List[Dict[str, Any]]
) -> Dict[str, Any]:
"""
Convenience function to perform comprehensive review analysis.
Args:
app_name: App name
reviews: List of review dictionaries
Returns:
Complete review analysis
"""
analyzer = ReviewAnalyzer(app_name)
return {
'sentiment_analysis': analyzer.analyze_sentiment(reviews),
'common_themes': analyzer.extract_common_themes(reviews),
'issues_identified': analyzer.identify_issues(reviews),
'feature_requests': analyzer.find_feature_requests(reviews)
}
Phân tích chi tiêu cá nhân, lập ngân sách 50/30/20, phát hiện điểm rò rỉ tài chính và lập kế hoạch tiết kiệm, đầu tư.
--- name: tai-chinh-ca-nhan description: Phân tích chi tiêu cá nhân, thiết lập ngân sách theo quy tắc 50/30/20, phát hiện điểm rò rỉ tài chính và lập kế hoạch tiết kiệm, đầu tư. Dùng khi nói "tài chính cá nhân", "quản lý chi tiêu", "lập ngân sách". --- # Quản lý tài chính cá nhân ## Mục tiêu Giúp phân tích chi tiêu, lập ngân sách và đưa ra quyết định tài chính có căn cứ. ## Khi nào dùng - Cuối tháng cần review chi tiêu - Muốn lập kế hoạch tiết kiệm hoặc đầu tư - Cần phân tích một quyết định tài chính cụ thể - Muốn tính toán mục tiêu tài chính ## Đầu vào cần cung cấp - Thu nhập hàng tháng - Các khoản chi tiêu chính - Mục tiêu tài chính (ngắn/trung/dài hạn) - Tình trạng tiết kiệm/nợ hiện tại (nếu có) ## Quy trình xử lý 1. Phân loại chi tiêu: cố định / biến đổi / không cần thiết 2. Tính tỷ lệ tiết kiệm thực tế 3. So sánh với nguyên tắc 50/30/20 (nhu cầu/mong muốn/tiết kiệm) 4. Xác định điểm rò rỉ ngân sách 5. Đề xuất điều chỉnh có thể thực hiện ngay ## Tiêu chuẩn đầu ra - Bảng tóm tắt thu/chi theo danh mục - Tỷ lệ phần trăm rõ ràng - Đề xuất cụ thể, không chung chung - Luôn nêu giả định khi không đủ dữ liệu ## Lưu ý quan trọng - Claude không phải chuyên gia tài chính được cấp phép - Mọi phân tích là tham khảo, không phải lời khuyên đầu tư chính thức - Luôn nêu rõ giả định đang dùng
Quy trình sản xuất nội dung trọn vẹn, từ chủ đề đến bài viết sẵn sàng đăng như blog, bài báo, hướng dẫn.
---
name: "content-production"
description: "Full content production pipeline — takes a topic from blank page to published-ready piece. Use when you need to execute content: write a blog post, article, or guide end-to-end. Triggers: 'write a post about', 'draft an article', 'create content for', 'help me write', 'I need a blog post'. NOT for content strategy or calendar planning (use content-strategy). NOT for repurposing existing content (use content-repurposing). NOT for social captions only."
license: MIT
metadata:
version: 1.0.0
author: Alireza Rezvani
category: marketing
updated: 2026-03-06
---
# Content Production
You are an expert content producer with deep experience across B2B SaaS, developer tools, and technical audiences. Your goal is to take a topic from zero to a finished, optimized piece that ranks, converts, and actually gets read.
This is the execution engine — not the strategy layer. You're here to build, not plan.
## Before Starting
**Check for context first:**
If `marketing-context.md` exists, read it before asking questions. It contains brand voice, target audience, keyword targets, and writing examples. Use what's there — only ask for what's missing.
Gather this context (ask in one shot, don't drip):
### What you need
- **Topic / working title** — what are we writing about?
- **Target keyword** — primary search term (if SEO matters)
- **Audience** — who reads this and what do they already know?
- **Goal** — inform, convert, build authority, drive trial?
- **Approximate length** — 800 words? 2,000 words? Long-form?
- **Existing content** — do we have pieces this should link to?
If the topic is vague ("write about AI"), push back: "Give me the specific angle — who's the reader, what problem are they solving?"
## How This Skill Works
Three modes. Start at whichever fits:
### Mode 1: Research & Brief
You have a topic but no content yet. Do the research, map the competitive landscape, define the angle, and produce a content brief before writing a word.
### Mode 2: Draft
Brief exists (either provided or from Mode 1). Write the full piece — intro, body, conclusion, headers — following the brief's structure and targeting parameters.
### Mode 3: Optimize & Polish
Draft exists. Run the full optimization pass: SEO signals, readability, structure audit, meta tags, internal links, quality gates. Output a publish-ready version.
You can run all 3 in sequence or jump directly to any mode.
---
## Mode 1: Research & Brief
### Step 1 — Competitive Content Analysis
Before writing, understand what already ranks. For the target keyword:
1. Identify the top 5-10 ranking pieces
2. Map their angles: Are they listicles? How-tos? Opinion pieces? Comparisons?
3. Find the gap: What's missing from the existing content? What angle is underserved?
4. Check search intent: Is the person trying to learn, compare, buy, or solve a specific problem?
**Intent signals:**
| SERP Pattern | Intent | What to write |
|---|---|---|
| "What is / How to" dominate | Informational | Comprehensive guide or explainer |
| Product pages, reviews | Commercial | Comparison or buyer's guide |
| News, updates | Navigational/news | Skip unless you have unique angle |
| Forum results (Reddit, Quora) | Discovery | Opinionated piece with real perspective |
### Step 2 — Source Gathering
Collect 3-5 credible, citable sources before drafting. Prioritize:
- Original research (studies, surveys, reports)
- Official documentation
- Expert quotes you can attribute
- Data with specific numbers (not vague claims)
**Rule:** If you can't cite a specific number, don't make a vague claim. "Studies show" is a red flag. Find the actual study.
### Step 3 — Produce the Content Brief
Fill in the [Content Brief Template](templates/content-brief-template.md). The brief defines:
- Target keyword + secondary keywords
- Reader profile and their job-to-be-done
- Angle and unique point of view
- Required sections and H2 structure
- Key claims to prove
- Internal links to include
- Competitive pieces to beat
See [references/content-brief-guide.md](references/content-brief-guide.md) for how to write a brief that actually produces better drafts.
---
## Mode 2: Draft
You have a brief. Now write.
### Outline First
Build the header skeleton before filling in prose. A good outline:
- Has a hook-worthy H1 (keyword-included, curiosity-driving)
- Has 4-7 H2 sections that follow a logical progression
- Uses H3s sparingly — only when a section genuinely needs subdivision
- Ends with a CTA-adjacent conclusion
Don't over-engineer the outline. If you're stuck on structure for more than 5 minutes, start writing and restructure later.
### Intro Principles
The intro has one job: make the reader believe this piece will answer their question. Get there in 3-4 sentences.
Formula that works:
1. Name the problem or situation the reader is in
2. Name what this piece does about it
3. Optionally: give them a reason to trust you on this topic
**What to avoid:**
- Starting with "In today's digital landscape..." (everyone does this)
- Starting with a question unless it's genuinely sharp
- Burying the point under 3 sentences of context-setting
### Section-by-Section Approach
For each H2 section:
1. State the main point in the first sentence (don't save it for the end)
2. Prove it with an example, stat, or comparison
3. Add one actionable takeaway before moving on
Readers skim. Every section should deliver value on its own.
### Conclusion
Three elements:
1. Summary of the core argument (1-2 sentences)
2. The single most important thing to do next
3. CTA (if relevant to the goal)
Don't pad the conclusion. If it's done, it's done.
---
## Mode 3: Optimize & Polish
Draft exists. Run this in order.
### SEO Pass
- **Title tag**: Contains primary keyword, under 60 characters, curiosity-driving
- **H1**: Different from title tag, keyword-rich, reads naturally
- **H2s**: At least 2-3 contain secondary keywords or related phrases
- **First paragraph**: Primary keyword appears in first 100 words
- **Image alt text**: Descriptive, includes keyword where natural
- **URL slug**: Short, keyword-first, no stop words
### Readability Pass
Run `scripts/content_scorer.py` on the draft. Target score: 70+.
Manual checks:
- Average sentence length: aim for 15-20 words, mix it up
- No paragraph over 4 sentences (web readers need air)
- No jargon without explanation (for non-expert audiences)
- Active voice: find passive constructions and flip them
### Structure Audit
- Does the intro deliver on the headline's promise?
- Is every H2 section earning its place? (Cut if not)
- Are there at least 2 examples or concrete illustrations?
- Does the conclusion feel earned?
### Internal Links
Add 2-4 internal links minimum:
- Link from high-traffic existing pages to this piece
- Link from this piece to related existing content
- Anchor text should describe the destination, not be generic ("click here" is useless)
### Meta Tags
Write:
- **Meta description**: 150-160 characters, includes keyword, ends with action or hook
- **OG title / OG description**: Can differ from meta, optimized for social sharing
- **Canonical URL**: Set it, even if obvious
### Quality Gates — Don't Publish Until These Pass
See [references/optimization-checklist.md](references/optimization-checklist.md) for the full pre-publish checklist.
Core gates:
- [ ] Primary keyword appears naturally 3-5x (not stuffed)
- [ ] Every factual claim has a source or is clearly labeled as opinion
- [ ] At least one image, table, or visual element breaks up text
- [ ] Intro doesn't start with a cliché
- [ ] All internal links work
- [ ] Readability score ≥ 70
- [ ] Word count is within 10% of target
---
## Proactive Triggers
Flag these without being asked:
- **Thin content risk** — If the target keyword has high-authority competitors with 2,000+ word pieces, a 600-word post won't rank. Surface this upfront, before drafting starts.
- **Keyword cannibalization** — If existing content already targets this keyword, flag it. Publishing a second piece splits authority instead of building it.
- **Intent mismatch** — If the requested angle doesn't match search intent (e.g., writing a brand awareness piece for a transactional keyword), call it out. The piece will get traffic that doesn't convert.
- **Missing sources** — If the draft contains claims like "many companies" or "studies show" without citation, flag each one before the piece ships.
- **CTA/goal disconnect** — If the piece's goal is "drive trial signups" but there's no CTA, or the CTA is buried at paragraph 12, flag it.
---
## Output Artifacts
| When you ask for... | You get... |
|---|---|
| Research & brief | Completed content brief: keyword targets, audience, angle, H2 structure, sources, competitive gaps |
| Full draft | Complete article with H1, H2s, intro, body, conclusion, and inline source markers |
| SEO optimization | Annotated draft with title tag, meta description, keyword placement audit, and OG copy |
| Readability audit | Scorer output + specific sentence-level edits flagged |
| Publish checklist | Completed gate checklist with pass/fail on each item |
---
## Communication
All output follows the structured standard:
- **Bottom line first** — answer before explanation
- **What + Why + How** — every finding includes all three
- **Actions have owners and deadlines** — no "we should probably..."
- **Confidence tagging** — 🟢 verified / 🟡 medium / 🔴 assumed
When reviewing drafts: flag issues → explain impact → give specific fix. Don't just say "improve readability." Say: "Paragraph 3 averages 32 words per sentence. Break the second sentence into two."
---
## Related Skills
- **content-strategy**: Use when deciding *what* to write — topics, calendar, pillar structure. NOT for writing the actual piece (that's this skill).
- **content-humanizer**: Use after drafting when the piece sounds robotic or AI-generated. Run this before the optimization pass.
- **ai-seo**: Use when optimizing specifically for AI search citation (ChatGPT, Perplexity, AI Overviews) in addition to traditional SEO.
- **copywriting**: Use for landing pages, CTAs, and conversion copy. NOT for long-form content (that's this skill).
- **seo-audit**: Use when auditing an existing content library for SEO gaps. NOT for single-piece production.
FILE:references/ai-citation-readiness.md
# AI Citation Readiness
How to optimize content so AI platforms (Perplexity, ChatGPT, Google AI Overviews, Claude) cite your pages in their answers. This is the emerging "second SEO" — ranking in AI-generated answers, not just Google blue links.
## Why this matters
AI answer engines extract passages from web content and present them as citations. Content that's structured for extraction gets cited more often. This is independent of traditional SEO — a page can rank #1 on Google but never get cited by AI because it's not formatted for extraction.
## The 5 AI citation signals
### 1. Answer-first paragraphs
The first 40-60 words after each H2 must directly answer the section's implied question. AI platforms extract these opening passages as citation candidates.
**Bad (builds up to the answer):**
```
There are many factors to consider when choosing a framework.
Performance, ecosystem, and learning curve all play a role.
After evaluating several options, React remains the best choice
for most teams in 2026.
```
**Good (answer first):**
```
React is the best frontend framework for most teams in 2026.
It has the largest ecosystem (4.2M weekly npm downloads),
the most hiring demand, and the best balance of performance
and developer experience.
```
### 2. Passage-level citability
Structure content in self-contained 120-180 word chunks that make sense when extracted in isolation. Each chunk should:
- State a clear claim
- Provide supporting evidence
- Cite a source
- Work as a standalone answer
### 3. Entity clarity
Use full entity names on first reference, then consistent abbreviations. AI models need unambiguous entity references to match queries.
**Bad:** "The framework's new feature improves DX significantly."
**Good:** "React 19's Server Components improve developer experience (DX) by reducing client-side JavaScript by 40%."
### 4. Q&A formatting
Include explicit question-and-answer patterns. These directly match how users query AI platforms.
```markdown
## How much does React hosting cost?
React hosting ranges from $0 (Vercel/Netlify free tier) to $20-50/month
for production apps with custom domains and CI/CD. Static sites built
with React are effectively free to host.
```
### 5. Freshness signals
AI platforms prefer recently updated content. Include:
- `dateModified` in JSON-LD or frontmatter
- "Last updated: [date]" visible on page
- Update frequency: evergreen content should be refreshed every 6-12 months
## Scoring for content_quality_gates.py
```
AI Citation Readiness checks:
Answer-first paragraphs at H2 → pass/fail (check first 60 words)
Passage length 120-180 words → pass/warn/fail
Entity clarity (no ambiguous "it") → pass/warn (heuristic)
Q&A section present → pass/fail
Freshness signal present → pass/fail
```
## What NOT to do
- Don't stuff FAQ schema on every page — use it only when genuine Q&A exists
- Don't write for extraction at the expense of human readability
- Don't fabricate statistics to make passages more "citable"
- Don't keyword-stuff entity names — use them naturally
FILE:references/content-brief-guide.md
# Content Brief Guide
A brief isn't a writing assignment. It's a contract between the strategist and the writer — and when you're both the same person, it's still the contract between your thinking brain and your writing brain. Skip it and you'll rewrite. Do it right and the draft almost writes itself.
---
## Why Briefs Fail (and How to Fix Them)
Most briefs are too vague. They say "write about email marketing" without telling the writer:
- What's the specific angle?
- Who's the reader?
- What keywords matter?
- What tone?
- What should the reader do after?
The result: a draft that misses the mark on every axis.
**The fix:** Every field in the brief should be specific enough that two different writers would produce the same piece.
---
## The Most Important Field: Angle
The angle is the single most critical field in the brief. It's your differentiated take — not just "write about email marketing" but "why most email open rate benchmarks are useless and what to measure instead."
**A good angle is:**
- One sentence
- Opinionated (takes a position)
- Different from what already ranks
- Grounded in something you actually know or have data on
**A weak angle is:**
- "Comprehensive guide to email marketing"
- "Everything you need to know about..."
- "Best practices for..."
If your angle sounds like a Wikipedia article, it's not an angle.
---
## Keyword Targeting: What Actually Matters
You don't need to stuff the brief with 20 keywords. You need:
1. **One primary keyword** — the main search term this piece is targeting. Every SEO decision flows from this.
2. **2-4 secondary keywords** — related phrases that appear naturally. These expand coverage without forcing it.
**How to find secondary keywords:**
- Look at "People Also Ask" on the SERP for your primary term
- Check what related terms appear in the top-ranking articles
- Use autocomplete on the search bar for your primary term
**Common mistake:** Targeting a keyword that's informational (someone learning) with a piece that's commercial (someone buying). Match intent or waste the effort.
---
## The Competitive Gap: Finding the Opening
Before writing the brief's angle, look at what's ranking. You're not copying them — you're finding what they missed.
**Gap patterns to look for:**
| Pattern | What It Means | Opportunity |
|---|---|---|
| All top pieces are listicles | No deep explanation exists | Write the definitive guide |
| Top pieces are outdated (2+ years old) | Information is stale | Write the current-year version with updated data |
| Top pieces are generic (no real examples) | Theory without practice | Write a practitioner piece with real cases |
| Top pieces are all from the same POV | Perspective monopoly | Write the contrarian or underdog angle |
| Top pieces are very long but shallow | Word count without depth | Write shorter but genuinely useful |
The gap is your angle. Find it before you brief.
---
## Structure: H2s Are the Real Deliverable
Writers often write vague outlines. "Introduction. Section 1. Conclusion." That's not a structure — that's a placeholder.
Good H2s do three things:
1. **Promise value** — the reader knows what they'll learn
2. **Follow logic** — each section flows from the previous
3. **Hit keywords** — secondary terms appear naturally
**Building the H2 structure:**
- Write the outline as if you're writing the table of contents for a useful reference book
- Each H2 should be a complete thought ("How to Write Subject Lines That Get Opened" not "Subject Lines")
- Sequence matters: don't put "Advanced Tactics" before "Why This Matters"
**The rule of 5:** Most blog posts need 4-6 H2s. Fewer and it's shallow. More and it's scattered.
---
## Sources: Non-Negotiable Before Drafting
Writers who draft without sources invent claims. Then those claims go live on your website.
**Minimum brief requirement:** 3 sources with specific data points or quotes identified.
**Source quality hierarchy:**
1. Original research (surveys, studies, experiments you conducted)
2. Credible third-party research (academic papers, industry reports from named organizations)
3. Expert quotes (attributed, verifiable)
4. Strong case studies with specific metrics
5. Official documentation or standards
**Red flag sources:** Anything that cites "a study" without naming it. Anything more than 5 years old in a fast-moving category. Competitor blog posts (they're also making stuff up).
---
## Internal Linking: Plan It Before, Not After
Most writers add internal links as an afterthought. This produces one problem: they link to whatever they remember, not what's most valuable.
The brief should specify:
- 2-3 existing pieces this article should link to (and what anchor text to use)
- 1-2 existing pages that should link back to this article once published
This prevents both orphaned content and missed link equity.
---
## Success Criteria: If You Don't Define It, You Can't Measure It
Every brief should answer: how will we know this piece worked?
Not vague ("gets traffic") — specific:
- Ranks in top 5 for [keyword] within [timeframe]
- Drives X leads per month
- Achieves X% conversion rate on the CTA
- Gets cited / linked by [type of site]
Define it now so you don't change the definition later.
---
## Brief Anti-Patterns
| Anti-pattern | Problem | Fix |
|---|---|---|
| "Write a comprehensive guide" | No angle, no differentiation | Define the specific take |
| Missing audience definition | Writer guesses; often wrong | Name the exact reader job title and pain |
| No sources listed | Writer invents facts | Find 3 sources before briefing |
| Vague keyword ("marketing") | No SEO targeting | Get specific: "email marketing for B2B SaaS" |
| H2s that are just topic labels | No promise, no structure | Rewrite as complete-thought headers |
| No internal links specified | Orphaned content | List 2-3 links before writing |
| No success criteria | Can't evaluate performance | Define at least one measurable outcome |
FILE:references/content-templates.md
# Content Templates — Auto-Selection Guide
12 content templates with selection logic. The LLM should detect topic intent and recommend the right template before writing.
## Selection matrix
| User intent | Template | Target length | Best for |
|---|---|---|---|
| "How do I..." | How-to guide | 1,500-3,000 words | Step-by-step processes |
| "Best X for Y" / "Top N" | Listicle | 2,000-4,000 words | Comparison/curation |
| "X increased revenue by Y%" | Case study | 1,500-2,500 words | Social proof, sales enablement |
| "X vs Y" | Comparison | 2,000-3,500 words | Decision-stage buyers |
| "Complete guide to X" | Pillar page | 3,000-5,000 words | Topical authority hub |
| "Is X worth it?" | Product review | 1,500-2,500 words | Affiliate, purchase intent |
| "The future of X" | Thought leadership | 1,200-2,000 words | Brand authority |
| "Experts say about X" | Roundup | 2,000-3,500 words | Link building, authority |
| "How to build X step by step" | Tutorial | 2,000-4,000 words | Technical audiences |
| "What happened with X" | News analysis | 800-1,500 words | Timeliness, trending topics |
| "[Data] shows that..." | Data research | 1,500-3,000 words | Original research, link magnets |
| "What is X?" | FAQ / knowledge base | 1,000-2,000 words | Informational, featured snippets |
## Template structures
### How-to guide
```
H1: How to [Action] [Object] (in [Timeframe])
H2: What You'll Need / Prerequisites
H2: Step 1: [First action]
H2: Step 2: [Second action]
...
H2: Common Mistakes to Avoid
H2: FAQ
```
### Listicle
```
H1: [N] Best [Category] for [Audience] in [Year]
H2: Quick Comparison Table
H2: 1. [Item] — Best for [Use Case]
H3: Key Features | H3: Pricing | H3: Who It's For
H2: 2. [Item] — Best for [Use Case]
...
H2: How We Evaluated
H2: FAQ
```
### Case study
```
H1: How [Company] [Achieved Result] with [Method/Product]
H2: The Challenge
H2: The Approach
H2: The Results (with metrics)
H2: Key Takeaways
H2: What's Next
```
### Comparison
```
H1: [X] vs [Y]: Which Is Better for [Use Case]?
H2: Quick Verdict (answer-first)
H2: Comparison Table
H2: [X] — Strengths and Weaknesses
H2: [Y] — Strengths and Weaknesses
H2: When to Choose [X]
H2: When to Choose [Y]
H2: FAQ
```
### Pillar page
```
H1: The Complete Guide to [Topic] ([Year])
H2: What Is [Topic]?
H2: Why [Topic] Matters
H2: [Subtopic 1] (links to cluster article)
H2: [Subtopic 2]
...
H2: Getting Started
H2: FAQ
H2: Further Reading (internal links to cluster)
```
## Auto-selection logic
```
IF user says "how to" or "step by step" → How-to guide
IF user says "best" or "top" + number → Listicle
IF user provides metrics/results data → Case study
IF user says "vs" or "compare" → Comparison
IF user says "complete guide" or "everything about" → Pillar page
IF user says "review" or "worth it" → Product review
IF user asks "what is" or implies definition → FAQ / knowledge base
IF user provides original data → Data research
IF user mentions recent event → News analysis
ELSE → ask user to clarify intent, suggest 2-3 options
```
## Quality checklist (applies to all templates)
- [ ] Answer-first paragraph at every H2
- [ ] Title 50-60 characters
- [ ] Meta description 150-160 characters
- [ ] No heading skips (H1→H2→H3 only)
- [ ] All statistics sourced
- [ ] All images have alt text
- [ ] Internal links ≥ 3
- [ ] External links ≥ 2 (to authoritative sources)
- [ ] CTA present (but not self-promotional beyond 1 mention)
FILE:references/optimization-checklist.md
# Pre-Publish Optimization Checklist
Run this before every piece goes live. Each section is a gate — fail a gate, fix it before moving on.
---
## Gate 1: SEO Signals
### Title & Headers
- [ ] H1 contains primary keyword (naturally, not forced)
- [ ] H1 is ≤70 characters
- [ ] At least 2 H2s contain secondary keywords or related phrases
- [ ] No two H2s are duplicates or near-duplicates
- [ ] H1 differs from the title tag (they can overlap but shouldn't be identical)
### Keyword Presence
- [ ] Primary keyword appears in the first 100 words
- [ ] Primary keyword appears 3-5 times total (not more — stuffing tanks rankings)
- [ ] Keyword variations appear naturally throughout
- [ ] No keyword stuffing (reading it aloud sounds natural)
### Meta & Technical
- [ ] Title tag: 50-60 characters, keyword-first where possible
- [ ] Meta description: 150-160 characters, includes keyword, ends with hook or action
- [ ] URL slug: short, keyword-first, lowercase, hyphens not underscores
- [ ] Canonical URL is set
- [ ] OG title and OG description written for social sharing
### Images & Media
- [ ] At least one image present
- [ ] All images have descriptive alt text (keyword included where natural)
- [ ] Images are compressed (under 200KB each)
- [ ] Image filenames are descriptive (not IMG_4832.jpg)
---
## Gate 2: Readability
### Score
- [ ] Flesch Reading Ease score ≥ 60 (aim for 60-70 for professional audience; 70+ for general)
- [ ] Run `scripts/content_scorer.py` — overall score ≥ 70
### Sentence & Paragraph Structure
- [ ] Average sentence length ≤ 20 words
- [ ] No single paragraph exceeds 4 sentences
- [ ] No sentence exceeds 35 words (check and break if found)
- [ ] Sentence length varies — not all short, not all long
### Voice & Clarity
- [ ] Active voice dominant (passive voice < 20% of sentences)
- [ ] No weasel words ("very," "really," "quite," "somewhat")
- [ ] No jargon without explanation (for non-expert audiences)
- [ ] All acronyms spelled out on first use
- [ ] Contractions used where natural (improves readability)
---
## Gate 3: Structure & Content Quality
### Opening
- [ ] Intro is ≤150 words
- [ ] Intro does not start with "In today's..." or "Welcome to..."
- [ ] Intro names the reader's problem or situation within the first 2 sentences
- [ ] Intro clearly signals what the reader will get from this piece
- [ ] No false promise in the intro (piece delivers what it hints at)
### Body
- [ ] Every H2 section leads with its main point (buried leads = reader drop-off)
- [ ] At least 2 concrete examples, case studies, or data points
- [ ] All statistics and specific claims have citations or are labeled as estimates
- [ ] No fluff paragraphs (every paragraph earns its place — if removing it changes nothing, cut it)
- [ ] Visual break (table, list, callout, image) at least every 400 words
### Conclusion
- [ ] Conclusion ≤150 words
- [ ] Summarizes the core argument (not just "in conclusion...")
- [ ] Includes one clear next step or CTA
- [ ] Doesn't introduce new arguments or ideas
---
## Gate 4: Internal Linking
- [ ] 2-4 internal links to existing content on the site
- [ ] Anchor text describes the destination (not "click here" or "this article")
- [ ] Links tested and confirmed working (no 404s)
- [ ] No excessive linking to the same page multiple times
- [ ] At least one high-traffic page links to this piece (plan this before publishing)
---
## Gate 5: Factual Accuracy
- [ ] Every statistic cited with source (year + organization)
- [ ] All external links go to credible sources (not competitors, not thin content)
- [ ] No claims made without evidence or without "in my experience" qualifier
- [ ] All product/feature mentions are accurate (check with product team if needed)
- [ ] Quotes are attributed correctly and not paraphrased beyond recognition
- [ ] No outdated information — check date-sensitive claims (pricing, regulations, stats)
---
## Gate 6: Brand & Voice
- [ ] Matches brand voice (check `marketing-context.md` if available)
- [ ] Consistent POV throughout (first person, second person, or third — pick one)
- [ ] Consistent tense (present or past — don't mix)
- [ ] No off-brand claims (anything that overpromises, contradicts other content, or sounds unlike us)
- [ ] CTA aligns with piece goal (don't pitch demo on an informational piece for beginners)
---
## Gate 7: Final Readthrough
Run a final read-aloud. Catch what scanning misses.
- [ ] Read the full piece aloud — anything that makes you stumble, fix it
- [ ] The piece flows — section to section makes sense without re-reading
- [ ] The headline still feels earned after reading the piece
- [ ] You'd share this piece yourself (if not, it's not done)
- [ ] No placeholder text, formatting artifacts, or draft notes left in
---
## Scoring Summary
| Gate | Status | Notes |
|---|---|---|
| SEO Signals | ✅ / ❌ | |
| Readability | ✅ / ❌ | |
| Structure & Quality | ✅ / ❌ | |
| Internal Linking | ✅ / ❌ | |
| Factual Accuracy | ✅ / ❌ | |
| Brand & Voice | ✅ / ❌ | |
| Final Readthrough | ✅ / ❌ | |
**Publish only when all 7 gates are green.**
If you're skipping a gate, document why. Conscious tradeoffs are fine. Unconscious shortcuts aren't.
FILE:scripts/brand_voice_analyzer.py
#!/usr/bin/env python3
"""
Brand Voice Analyzer - Analyzes content to establish and maintain brand voice consistency
"""
import re
from typing import Dict, List, Tuple
import json
class BrandVoiceAnalyzer:
def __init__(self):
self.voice_dimensions = {
'formality': {
'formal': ['hereby', 'therefore', 'furthermore', 'pursuant', 'regarding'],
'casual': ['hey', 'cool', 'awesome', 'stuff', 'yeah', 'gonna']
},
'tone': {
'professional': ['expertise', 'solution', 'optimize', 'leverage', 'strategic'],
'friendly': ['happy', 'excited', 'love', 'enjoy', 'together', 'share']
},
'perspective': {
'authoritative': ['proven', 'research shows', 'experts agree', 'data indicates'],
'conversational': ['you might', 'let\'s explore', 'we think', 'imagine if']
}
}
def analyze_text(self, text: str) -> Dict:
"""Analyze text for brand voice characteristics"""
text_lower = text.lower()
word_count = len(text.split())
results = {
'word_count': word_count,
'readability_score': self._calculate_readability(text),
'voice_profile': {},
'sentence_analysis': self._analyze_sentences(text),
'recommendations': []
}
# Analyze voice dimensions
for dimension, categories in self.voice_dimensions.items():
dim_scores = {}
for category, keywords in categories.items():
score = sum(1 for keyword in keywords if keyword in text_lower)
dim_scores[category] = score
# Determine dominant voice
if sum(dim_scores.values()) > 0:
dominant = max(dim_scores, key=dim_scores.get)
results['voice_profile'][dimension] = {
'dominant': dominant,
'scores': dim_scores
}
# Generate recommendations
results['recommendations'] = self._generate_recommendations(results)
return results
def _calculate_readability(self, text: str) -> float:
"""Calculate Flesch Reading Ease score"""
sentences = re.split(r'[.!?]+', text)
words = text.split()
syllables = sum(self._count_syllables(word) for word in words)
if len(sentences) == 0 or len(words) == 0:
return 0
avg_sentence_length = len(words) / len(sentences)
avg_syllables_per_word = syllables / len(words)
# Flesch Reading Ease formula
score = 206.835 - 1.015 * avg_sentence_length - 84.6 * avg_syllables_per_word
return max(0, min(100, score))
def _count_syllables(self, word: str) -> int:
"""Count syllables in a word (simplified)"""
word = word.lower()
vowels = 'aeiou'
syllable_count = 0
previous_was_vowel = False
for char in word:
is_vowel = char in vowels
if is_vowel and not previous_was_vowel:
syllable_count += 1
previous_was_vowel = is_vowel
# Adjust for silent e
if word.endswith('e'):
syllable_count -= 1
return max(1, syllable_count)
def _analyze_sentences(self, text: str) -> Dict:
"""Analyze sentence structure"""
sentences = re.split(r'[.!?]+', text)
sentences = [s.strip() for s in sentences if s.strip()]
if not sentences:
return {'average_length': 0, 'variety': 'low'}
lengths = [len(s.split()) for s in sentences]
avg_length = sum(lengths) / len(lengths) if lengths else 0
# Calculate variety
if len(set(lengths)) < 3:
variety = 'low'
elif len(set(lengths)) < 5:
variety = 'medium'
else:
variety = 'high'
return {
'average_length': round(avg_length, 1),
'variety': variety,
'count': len(sentences)
}
def _generate_recommendations(self, analysis: Dict) -> List[str]:
"""Generate recommendations based on analysis"""
recommendations = []
# Readability recommendations
if analysis['readability_score'] < 30:
recommendations.append("Consider simplifying language for better readability")
elif analysis['readability_score'] > 70:
recommendations.append("Content is very easy to read - consider if this matches your audience")
# Sentence variety
if analysis['sentence_analysis']['variety'] == 'low':
recommendations.append("Vary sentence length for better flow and engagement")
# Voice consistency
if analysis['voice_profile']:
recommendations.append("Maintain consistent voice across all content")
return recommendations
def analyze_content(content: str, output_format: str = 'json') -> str:
"""Main function to analyze content"""
analyzer = BrandVoiceAnalyzer()
results = analyzer.analyze_text(content)
if output_format == 'json':
return json.dumps(results, indent=2)
else:
# Human-readable format
output = [
f"=== Brand Voice Analysis ===",
f"Word Count: {results['word_count']}",
f"Readability Score: {results['readability_score']:.1f}/100",
f"",
f"Voice Profile:"
]
for dimension, profile in results['voice_profile'].items():
output.append(f" {dimension.title()}: {profile['dominant']}")
output.extend([
f"",
f"Sentence Analysis:",
f" Average Length: {results['sentence_analysis']['average_length']} words",
f" Variety: {results['sentence_analysis']['variety']}",
f" Total Sentences: {results['sentence_analysis']['count']}",
f"",
f"Recommendations:"
])
for rec in results['recommendations']:
output.append(f" • {rec}")
return '\n'.join(output)
if __name__ == "__main__":
import sys
import argparse
parser = argparse.ArgumentParser(
description="Brand Voice Analyzer - Analyzes content to establish and maintain brand voice consistency"
)
parser.add_argument(
"file", nargs="?", default=None,
help="Text file to analyze"
)
parser.add_argument(
"--format", choices=["json", "text"], default="text",
help="Output format (default: text)"
)
args = parser.parse_args()
if args.file:
with open(args.file, 'r') as f:
content = f.read()
print(analyze_content(content, args.format))
else:
print("Usage: python brand_voice_analyzer.py <file> [--format json|text]")
FILE:scripts/content_quality_gates.py
#!/usr/bin/env python3
"""
content_quality_gates.py — Non-negotiable quality checks before publishing.
Hard stops that BLOCK publication. These aren't suggestions — they're rules.
Inspired by the quality gate pattern used in high-traction content tools.
Gates (all must pass):
1. Heading hierarchy: H1 → H2 → H3 only (no skips, no H4+ without H3)
2. No paragraphs > 150 words (wall of text)
3. All images/figures have alt text references
4. Source citations: any statistic must have a [source] marker
5. Title length: 50-60 characters
6. Meta description: 150-160 characters (if present)
7. No self-promotional mentions beyond 1
8. dateModified or "Updated" marker present for evergreen content
Usage:
python content_quality_gates.py article.md
python content_quality_gates.py article.md --json
python content_quality_gates.py --demo
Score bands (from the gate results):
All pass → PUBLISH (ready to ship)
1-2 warnings → TARGET (fix specific items)
Any gate failure → BLOCK (fix before publishing)
"""
from __future__ import annotations
import argparse
import json
import re
import sys
from pathlib import Path
HEADING_RE = re.compile(r"^(#{1,6})\s+(.+)$", re.MULTILINE)
STAT_RE = re.compile(r"\b\d+(?:\.\d+)?%|\$\d+|\d+(?:,\d{3})+|\d+x\b", re.IGNORECASE)
SOURCE_RE = re.compile(r"\[(?:source|citation|ref|via|from)\b.*?\]|\(https?://\S+\)", re.IGNORECASE)
IMAGE_RE = re.compile(r"!\[([^\]]*)\]\(", re.MULTILINE)
TITLE_RE = re.compile(r"^#\s+(.+)$", re.MULTILINE)
META_DESC_RE = re.compile(r"^description:\s*[\"']?(.+?)[\"']?\s*$", re.MULTILINE)
SELF_PROMO_RE = re.compile(r"\b(?:our product|we offer|sign up|try free|get started|our platform|our solution)\b", re.IGNORECASE)
DATE_RE = re.compile(r"(?:updated|modified|reviewed|published).*?\d{4}", re.IGNORECASE)
FRONTMATTER_DATE_RE = re.compile(r"^(?:date_?modified|updated|last_updated):\s*\d{4}", re.MULTILINE)
DEMO_CONTENT = """---
title: "How to Optimize Your Landing Page for Conversions"
description: "Learn the 7 proven techniques to boost landing page conversion rates by 40% or more."
dateModified: 2026-03-15
---
# How to Optimize Your Landing Page for Conversions
Landing pages are the backbone of any digital marketing campaign. Getting them right means more leads, more sales, and better ROI on every dollar you spend on ads.
## Start With a Clear Value Proposition
Your headline should communicate the core benefit in under 10 words. According to a study by Marketing Experiments, clarity beats persuasion by 104% in headline tests [source: marketingexperiments.com].
The subheadline expands on the promise. Keep it under 25 words.
## Optimize Your Form Length
Fewer fields mean higher conversions. HubSpot found that reducing form fields from 4 to 3 increased conversions by 50% [source: HubSpot 2025 Benchmark Report].
### When to Use Long Forms
Long forms work for high-value B2B offers where lead quality matters more than volume. The key is progressive profiling — ask for more information over time, not all at once.
## Add Social Proof Above the Fold
Show logos, testimonials, or user counts where visitors can see them without scrolling. 72% of consumers say positive reviews increase their trust [source: BrightLocal 2025].
## Test Your CTA Button
The button text matters more than the color. "Get My Free Guide" outperforms "Submit" by 3x on average.
Our platform makes A/B testing effortless — try it free today.
## Measure and Iterate
Track these metrics weekly: conversion rate, bounce rate, time on page, scroll depth. Set up goals in Google Analytics 4 and review them every Friday.
"""
def check_heading_hierarchy(text):
"""Gate 1: H1 → H2 → H3 only, no skips."""
findings = []
headings = [(m.start(), len(m.group(1)), m.group(2)) for m in HEADING_RE.finditer(text)]
prev_level = 0
for _, level, title in headings:
if level > prev_level + 1 and prev_level > 0:
findings.append(f"Heading skip: H{prev_level} → H{level} at '{title[:40]}'. Don't skip levels.")
prev_level = level
return {"gate": "heading-hierarchy", "passed": len(findings) == 0, "findings": findings}
def check_paragraph_length(text):
"""Gate 2: No paragraph > 150 words."""
findings = []
body = re.sub(r"^---.*?---\s*", "", text, count=1, flags=re.DOTALL)
paragraphs = re.split(r"\n\s*\n", body)
for i, para in enumerate(paragraphs, 1):
para = para.strip()
if not para or para.startswith("#") or para.startswith("```") or para.startswith("|"):
continue
words = len(para.split())
if words > 150:
findings.append(f"Paragraph {i}: {words} words (max 150). Break it up.")
return {"gate": "paragraph-length", "passed": len(findings) == 0, "findings": findings}
def check_image_alt_text(text):
"""Gate 3: All images have alt text."""
findings = []
for m in IMAGE_RE.finditer(text):
alt = m.group(1).strip()
if not alt:
findings.append(f"Image missing alt text:  — add descriptive alt.")
return {"gate": "image-alt-text", "passed": len(findings) == 0, "findings": findings}
def check_source_citations(text):
"""Gate 4: Statistics must have source markers."""
findings = []
body = re.sub(r"^---.*?---\s*", "", text, count=1, flags=re.DOTALL)
lines = body.splitlines()
for i, line in enumerate(lines, 1):
if line.strip().startswith("#") or line.strip().startswith("```"):
continue
stats = STAT_RE.findall(line)
if stats and not SOURCE_RE.search(line):
stat_str = ", ".join(stats[:3])
findings.append(f"Line {i}: statistic ({stat_str}) without [source] marker.")
return {"gate": "source-citations", "passed": len(findings) == 0, "findings": findings}
def check_title_length(text):
"""Gate 5: Title 50-60 characters."""
findings = []
m = TITLE_RE.search(text)
if m:
title = m.group(1).strip()
length = len(title)
if length < 50:
findings.append(f"Title too short: {length} chars (aim for 50-60). Title: '{title}'")
elif length > 60:
findings.append(f"Title too long: {length} chars (aim for 50-60). Title: '{title}'")
else:
findings.append("No H1 title found.")
return {"gate": "title-length", "passed": len(findings) == 0, "findings": findings}
def check_meta_description(text):
"""Gate 6: Meta description 150-160 chars (if present)."""
findings = []
m = META_DESC_RE.search(text)
if m:
desc = m.group(1).strip()
length = len(desc)
if length < 150:
findings.append(f"Meta description too short: {length} chars (aim for 150-160).")
elif length > 160:
findings.append(f"Meta description too long: {length} chars (aim for 150-160). Will be truncated.")
return {"gate": "meta-description", "passed": len(findings) == 0, "findings": findings}
def check_self_promotion(text):
"""Gate 7: Max 1 self-promotional mention."""
findings = []
matches = SELF_PROMO_RE.findall(text)
if len(matches) > 1:
findings.append(f"{len(matches)} self-promotional mentions found (max 1): {', '.join(matches[:5])}")
return {"gate": "self-promotion", "passed": len(findings) == 0, "findings": findings}
def check_freshness(text):
"""Gate 8: dateModified or update marker present."""
findings = []
has_date = bool(FRONTMATTER_DATE_RE.search(text)) or bool(DATE_RE.search(text))
if not has_date:
findings.append("No dateModified or 'Updated' marker. Evergreen content needs freshness signals.")
return {"gate": "freshness-signal", "passed": len(findings) == 0, "findings": findings}
ALL_GATES = [
check_heading_hierarchy,
check_paragraph_length,
check_image_alt_text,
check_source_citations,
check_title_length,
check_meta_description,
check_self_promotion,
check_freshness,
]
def run_gates(text):
results = [gate(text) for gate in ALL_GATES]
passed = sum(1 for r in results if r["passed"])
failed = [r for r in results if not r["passed"]]
total = len(results)
if len(failed) == 0:
verdict = "PUBLISH"
elif all(len(r["findings"]) <= 1 for r in failed) and len(failed) <= 2:
verdict = "TARGET"
else:
verdict = "BLOCK"
return {
"status": "ok",
"gates_passed": passed,
"gates_total": total,
"verdict": verdict,
"results": results,
}
def main():
p = argparse.ArgumentParser(
description="Non-negotiable quality gates for content publishing.",
epilog="All gates must pass before publishing. Run with --demo for a sample article.",
)
p.add_argument("file", nargs="?", help="Markdown file to check")
p.add_argument("--json", action="store_true", help="JSON output")
p.add_argument("--demo", action="store_true", help="Run with demo content")
args = p.parse_args()
if args.demo:
text = DEMO_CONTENT
elif args.file:
path = Path(args.file)
if not path.exists():
print(f"[error] {path} not found", file=sys.stderr)
sys.exit(1)
text = path.read_text(encoding="utf-8", errors="replace")
else:
p.print_help()
sys.exit(0)
result = run_gates(text)
if args.json:
print(json.dumps(result, indent=2))
return
print(f"Content Quality Gates — {result['gates_passed']}/{result['gates_total']} passed")
print(f"Verdict: {result['verdict']}")
print()
for r in result["results"]:
icon = "✅" if r["passed"] else "❌"
print(f" {icon} {r['gate']}")
for f in r["findings"]:
print(f" → {f}")
print()
if result["verdict"] == "PUBLISH":
print(" All gates pass. Ready to publish.")
elif result["verdict"] == "TARGET":
print(" Minor issues. Fix the flagged items, then publish.")
else:
print(" BLOCKED. Fix all gate failures before publishing.")
if __name__ == "__main__":
main()
FILE:scripts/content_scorer.py
#!/usr/bin/env python3
"""content_scorer.py — scores content 0-100 on readability, SEO, structure, and engagement."""
import sys
import re
import json
import math
from collections import Counter
# ── Sample content for zero-config demo run ──────────────────────────────────
SAMPLE_CONTENT = """
Title: How to Reduce Churn in SaaS: 7 Proven Tactics That Actually Work
Introduction
Most SaaS companies discover their churn problem too late — after the customer has already left. By then, the damage is done. In this guide, you'll learn seven tactics to reduce churn before it happens, backed by data from 200+ SaaS companies.
## Why Customers Churn (It's Not What You Think)
Customers don't churn because your product is bad. They churn because they never saw value. A study by Mixpanel found that 60% of users who churn never completed onboarding. That's a product adoption problem, not a satisfaction problem.
Fix the adoption gap first. Everything else is downstream.
## Tactic 1: Instrument Your Activation Funnel
You can't fix what you can't see. Start by identifying your activation event — the moment users first experience your product's core value. For Slack, it's sending 2,000 messages. For Dropbox, it's saving a first file.
Map the funnel from signup to activation. Find where users drop off. That's your highest-leverage intervention point.
## Tactic 2: Segment Your Churn by Cohort
Not all churn is equal. A user who churns in week one is a different problem than a user who churns in month six. Cohort analysis breaks this apart.
Compare cohorts by: acquisition channel, onboarding path, company size, and feature usage. You'll find that certain cohorts churn 3-4x more than others. Focus retention efforts on your best cohorts first — don't try to save everyone.
## Tactic 3: Build a Customer Health Score
A health score is a composite signal that predicts churn before it happens. Common inputs include: login frequency, feature adoption rate, support ticket volume, and NPS response.
Weight each signal by its correlation with retention in your historical data. A score below 40 should trigger a customer success outreach. Don't wait for the cancellation request.
## Conclusion
Churn is a lagging indicator. By the time you see it, the problem happened weeks ago. Build systems that surface early signals — activation gaps, usage drops, health score declines — and act on them before customers decide to leave.
Start with one tactic. Instrument your activation funnel this week.
"""
SAMPLE_KEYWORD = "reduce churn"
SAMPLE_TITLE = "How to Reduce Churn in SaaS: 7 Proven Tactics That Actually Work"
# ── Scoring functions ─────────────────────────────────────────────────────────
def count_syllables(word: str) -> int:
"""Approximate syllable count using vowel-group heuristic."""
word = word.lower().strip(".,!?;:")
if not word:
return 0
vowels = "aeiouy"
count = 0
prev_vowel = False
for ch in word:
is_vowel = ch in vowels
if is_vowel and not prev_vowel:
count += 1
prev_vowel = is_vowel
# Adjust for silent e
if word.endswith("e") and len(word) > 2:
count = max(1, count - 1)
return max(1, count)
def flesch_reading_ease(text: str) -> float:
"""
Flesch Reading Ease score.
206.835 - 1.015 * (words/sentences) - 84.6 * (syllables/words)
Higher = easier. Target: 60-70 for professional content.
"""
sentences = re.split(r'[.!?]+', text)
sentences = [s.strip() for s in sentences if s.strip()]
n_sentences = max(1, len(sentences))
words = re.findall(r'\b[a-zA-Z]+\b', text)
n_words = max(1, len(words))
n_syllables = sum(count_syllables(w) for w in words)
asl = n_words / n_sentences # avg sentence length
asw = n_syllables / n_words # avg syllables per word
score = 206.835 - (1.015 * asl) - (84.6 * asw)
return round(max(0.0, min(100.0, score)), 1)
def score_readability(text: str) -> dict:
"""Score readability 0-25 (25% of total)."""
fre = flesch_reading_ease(text)
# FRE → points (target 60-70 for B2B)
if fre >= 65:
fre_points = 15
elif fre >= 55:
fre_points = 12
elif fre >= 45:
fre_points = 8
elif fre >= 35:
fre_points = 4
else:
fre_points = 0
# Sentence length variance
sentences = re.split(r'[.!?]+', text)
sentences = [s.strip() for s in sentences if len(s.split()) > 2]
lengths = [len(s.split()) for s in sentences]
if len(lengths) > 1:
mean_len = sum(lengths) / len(lengths)
variance = sum((l - mean_len) ** 2 for l in lengths) / len(lengths)
std_dev = math.sqrt(variance)
variance_points = min(10, int(std_dev)) # good variance = high std dev
else:
variance_points = 0
total = min(25, fre_points + variance_points)
return {
"score": total,
"max": 25,
"flesch_reading_ease": fre,
"sentence_length_std_dev": round(math.sqrt(variance) if len(lengths) > 1 else 0, 1)
}
def score_seo(text: str, title: str = "", keyword: str = "") -> dict:
"""Score SEO signals 0-25 (25% of total)."""
text_lower = text.lower()
title_lower = title.lower()
keyword_lower = keyword.lower()
points = 0
signals = {}
# Title contains keyword
if keyword_lower and keyword_lower in title_lower:
points += 7
signals["keyword_in_title"] = True
else:
signals["keyword_in_title"] = False
# Keyword in first 100 words
first_100 = " ".join(re.findall(r'\b\w+\b', text_lower)[:100])
if keyword_lower and keyword_lower in first_100:
points += 5
signals["keyword_in_intro"] = True
else:
signals["keyword_in_intro"] = False
# Keyword density (target 0.5-2%)
words = re.findall(r'\b\w+\b', text_lower)
n_words = max(1, len(words))
kw_words = keyword_lower.split()
kw_count = 0
for i in range(len(words) - len(kw_words) + 1):
if words[i:i+len(kw_words)] == kw_words:
kw_count += 1
density = (kw_count * len(kw_words)) / n_words * 100
signals["keyword_density_pct"] = round(density, 2)
signals["keyword_occurrences"] = kw_count
if 0.5 <= density <= 2.5:
points += 5
elif kw_count > 0:
points += 2
# H2 headings present
h2_count = len(re.findall(r'^## .+', text, re.MULTILINE))
signals["h2_count"] = h2_count
if h2_count >= 3:
points += 5
elif h2_count >= 1:
points += 2
# Title length
signals["title_length"] = len(title)
if 30 <= len(title) <= 65:
points += 3
total = min(25, points)
return {"score": total, "max": 25, **signals}
def score_structure(text: str) -> dict:
"""Score structure 0-25 (25% of total)."""
points = 0
signals = {}
lines = text.strip().split('\n')
# Intro: first non-empty paragraph
paragraphs = [p.strip() for p in text.split('\n\n') if p.strip()]
signals["paragraph_count"] = len(paragraphs)
# Has intro (first paragraph isn't a heading)
if paragraphs and not paragraphs[0].startswith('#'):
intro_words = len(paragraphs[0].split())
signals["intro_word_count"] = intro_words
if 30 <= intro_words <= 200:
points += 7
elif intro_words > 0:
points += 3
else:
signals["intro_word_count"] = 0
# Has H2 sections
h2s = [l for l in lines if l.startswith('## ')]
signals["h2_count"] = len(h2s)
if len(h2s) >= 4:
points += 8
elif len(h2s) >= 2:
points += 5
elif len(h2s) >= 1:
points += 2
# Has conclusion (last substantial paragraph)
last_para = paragraphs[-1] if paragraphs else ""
conclusion_words = len(last_para.split())
signals["conclusion_word_count"] = conclusion_words
conclusion_signals = ['conclusion', 'summary', 'final', 'start ', 'next step', 'action']
if any(sig in last_para.lower() for sig in conclusion_signals) and conclusion_words >= 20:
points += 7
elif conclusion_words >= 30:
points += 4
# Average paragraph length (web = shorter is better)
para_lengths = [len(p.split()) for p in paragraphs if not p.startswith('#')]
if para_lengths:
avg_para_len = sum(para_lengths) / len(para_lengths)
signals["avg_paragraph_word_count"] = round(avg_para_len, 1)
if avg_para_len <= 80:
points += 3
else:
signals["avg_paragraph_word_count"] = 0
total = min(25, points)
return {"score": total, "max": 25, **signals}
def score_engagement(text: str) -> dict:
"""Score engagement signals 0-25 (25% of total)."""
points = 0
signals = {}
text_lower = text.lower()
# Questions (engage readers, prompt thought)
question_count = len(re.findall(r'\?', text))
signals["question_count"] = question_count
if question_count >= 3:
points += 6
elif question_count >= 1:
points += 3
# Specific numbers / data points
number_count = len(re.findall(r'\b\d+(?:\.\d+)?%?\b', text))
signals["numbers_and_stats"] = number_count
if number_count >= 5:
points += 7
elif number_count >= 2:
points += 4
elif number_count >= 1:
points += 2
# Example signals
example_phrases = ['for example', 'for instance', 'such as', 'like ', 'e.g.', 'case study',
'imagine', 'consider', 'let\'s say', 'here\'s', 'specifically']
example_count = sum(text_lower.count(p) for p in example_phrases)
signals["example_signals"] = example_count
if example_count >= 3:
points += 6
elif example_count >= 1:
points += 3
# Lists (bulleted or numbered)
list_items = len(re.findall(r'^\s*[-*•]\s+.+|^\s*\d+\.\s+.+', text, re.MULTILINE))
signals["list_items"] = list_items
if list_items >= 5:
points += 6
elif list_items >= 2:
points += 3
total = min(25, points)
return {"score": total, "max": 25, **signals}
# ── Main ──────────────────────────────────────────────────────────────────────
def score_content(text: str, title: str = "", keyword: str = "") -> dict:
readability = score_readability(text)
seo = score_seo(text, title, keyword)
structure = score_structure(text)
engagement = score_engagement(text)
total = readability["score"] + seo["score"] + structure["score"] + engagement["score"]
grade = "D"
if total >= 90:
grade = "A+"
elif total >= 80:
grade = "A"
elif total >= 70:
grade = "B"
elif total >= 60:
grade = "C"
return {
"total_score": total,
"grade": grade,
"sections": {
"readability": readability,
"seo": seo,
"structure": structure,
"engagement": engagement,
}
}
def print_report(result: dict, title: str, keyword: str) -> None:
total = result["total_score"]
grade = result["grade"]
s = result["sections"]
bar_filled = int(total / 5)
bar = "█" * bar_filled + "░" * (20 - bar_filled)
print()
print("╔══════════════════════════════════════════╗")
print("║ CONTENT SCORER — REPORT ║")
print("╚══════════════════════════════════════════╝")
print(f" Title: {title[:55] or '(not provided)'}")
print(f" Keyword: {keyword or '(not provided)'}")
print()
print(f" TOTAL SCORE: {total}/100 [{grade}]")
print(f" [{bar}]")
print()
print(" ── Section Breakdown ──────────────────────")
sections = [
("Readability", s["readability"]),
("SEO Signals", s["seo"]),
("Structure", s["structure"]),
("Engagement", s["engagement"]),
]
for label, section in sections:
sc = section["score"]
mx = section["max"]
bar2_filled = int(sc / mx * 10)
bar2 = "█" * bar2_filled + "░" * (10 - bar2_filled)
print(f" {label:<14} {sc:>2}/{mx} [{bar2}]")
print()
print(" ── Key Signals ────────────────────────────")
r = s["readability"]
print(f" Flesch Reading Ease: {r['flesch_reading_ease']} (target: 60-70)")
print(f" Sentence length StDev: {r['sentence_length_std_dev']} (higher = more varied)")
seo_d = s["seo"]
print(f" Keyword in title: {'✅' if seo_d.get('keyword_in_title') else '❌'}")
print(f" Keyword in intro: {'✅' if seo_d.get('keyword_in_intro') else '❌'}")
print(f" Keyword density: {seo_d.get('keyword_density_pct', 0)}% (target: 0.5-2.5%)")
print(f" H2 sections: {seo_d.get('h2_count', 0)}")
st = s["structure"]
print(f" Intro word count: {st.get('intro_word_count', 0)} (target: 30-200)")
print(f" Avg paragraph length: {st.get('avg_paragraph_word_count', 0)} words (target: ≤80)")
en = s["engagement"]
print(f" Questions: {en.get('question_count', 0)}")
print(f" Stats/numbers: {en.get('numbers_and_stats', 0)}")
print(f" Examples: {en.get('example_signals', 0)}")
print()
print(" ── Recommendations ────────────────────────")
if r["flesch_reading_ease"] < 55:
print(" ⚠ Readability is low — shorten sentences and use simpler words")
if not seo_d.get("keyword_in_title"):
print(" ⚠ Primary keyword missing from title — add it naturally")
if not seo_d.get("keyword_in_intro"):
print(" ⚠ Primary keyword missing from first 100 words")
if seo_d.get("h2_count", 0) < 3:
print(" ⚠ Add more H2 sections — aim for at least 4")
if st.get("avg_paragraph_word_count", 0) > 100:
print(" ⚠ Paragraphs too long for web — break them up")
if en.get("question_count", 0) == 0:
print(" ⚠ Add at least one question to engage readers")
if en.get("numbers_and_stats", 0) < 2:
print(" ⚠ Thin on data — add specific numbers or stats")
if total >= 70:
print(" ✅ Content is publish-ready (score ≥ 70)")
else:
print(f" ❌ Score below 70 — address recommendations before publishing")
print()
def main():
import argparse
parser = argparse.ArgumentParser(
description="Scores content 0-100 on readability, SEO, structure, and engagement."
)
parser.add_argument(
"file", nargs="?", default=None,
help="Path to a text/markdown file to analyze. If omitted, runs demo "
"with embedded sample content."
)
parser.add_argument(
"keyword", nargs="?", default="",
help="Target SEO keyword to check density and placement."
)
parser.add_argument(
"--json", action="store_true",
help="Also output results as JSON."
)
args = parser.parse_args()
title = ""
keyword = args.keyword
text = ""
if args.file is None:
# Demo mode — use embedded sample
print("[Demo mode — using embedded sample content]")
text = SAMPLE_CONTENT
title = SAMPLE_TITLE
keyword = SAMPLE_KEYWORD
else:
# Read from file
try:
with open(args.file, 'r', encoding='utf-8') as f:
text = f.read()
except FileNotFoundError:
print(f"Error: file not found: {args.file}", file=sys.stderr)
sys.exit(1)
# Extract title from first H1 or first line
for line in text.split('\n'):
line = line.strip()
if line.startswith('# '):
title = line[2:].strip()
break
elif line.startswith('Title:'):
title = line[6:].strip()
break
if not title and text:
title = text.split('\n')[0][:80]
result = score_content(text, title, keyword)
print_report(result, title, keyword)
# JSON output for programmatic use
if args.json:
print(json.dumps(result, indent=2))
if __name__ == "__main__":
main()
FILE:scripts/seo_optimizer.py
#!/usr/bin/env python3
"""
SEO Content Optimizer - Analyzes and optimizes content for SEO
"""
import re
from typing import Dict, List, Set
import json
class SEOOptimizer:
def __init__(self):
# Common stop words to filter
self.stop_words = {
'the', 'a', 'an', 'and', 'or', 'but', 'in', 'on', 'at', 'to', 'for',
'of', 'with', 'by', 'from', 'as', 'is', 'was', 'are', 'were', 'be',
'been', 'being', 'have', 'has', 'had', 'do', 'does', 'did', 'will',
'would', 'could', 'should', 'may', 'might', 'must', 'can', 'shall'
}
# SEO best practices
self.best_practices = {
'title_length': (50, 60),
'meta_description_length': (150, 160),
'url_length': (50, 60),
'paragraph_length': (40, 150),
'heading_keyword_placement': True,
'keyword_density': (0.01, 0.03) # 1-3%
}
def analyze(self, content: str, target_keyword: str = None,
secondary_keywords: List[str] = None) -> Dict:
"""Analyze content for SEO optimization"""
analysis = {
'content_length': len(content.split()),
'keyword_analysis': {},
'structure_analysis': self._analyze_structure(content),
'readability': self._analyze_readability(content),
'meta_suggestions': {},
'optimization_score': 0,
'recommendations': []
}
# Keyword analysis
if target_keyword:
analysis['keyword_analysis'] = self._analyze_keywords(
content, target_keyword, secondary_keywords or []
)
# Generate meta suggestions
analysis['meta_suggestions'] = self._generate_meta_suggestions(
content, target_keyword
)
# Calculate optimization score
analysis['optimization_score'] = self._calculate_seo_score(analysis)
# Generate recommendations
analysis['recommendations'] = self._generate_recommendations(analysis)
return analysis
def _analyze_keywords(self, content: str, primary: str,
secondary: List[str]) -> Dict:
"""Analyze keyword usage and density"""
content_lower = content.lower()
word_count = len(content.split())
results = {
'primary_keyword': {
'keyword': primary,
'count': content_lower.count(primary.lower()),
'density': 0,
'in_title': False,
'in_headings': False,
'in_first_paragraph': False
},
'secondary_keywords': [],
'lsi_keywords': []
}
# Calculate primary keyword metrics
if word_count > 0:
results['primary_keyword']['density'] = (
results['primary_keyword']['count'] / word_count
)
# Check keyword placement
first_para = content.split('\n\n')[0] if '\n\n' in content else content[:200]
results['primary_keyword']['in_first_paragraph'] = (
primary.lower() in first_para.lower()
)
# Analyze secondary keywords
for keyword in secondary:
count = content_lower.count(keyword.lower())
results['secondary_keywords'].append({
'keyword': keyword,
'count': count,
'density': count / word_count if word_count > 0 else 0
})
# Extract potential LSI keywords
results['lsi_keywords'] = self._extract_lsi_keywords(content, primary)
return results
def _analyze_structure(self, content: str) -> Dict:
"""Analyze content structure for SEO"""
lines = content.split('\n')
structure = {
'headings': {'h1': 0, 'h2': 0, 'h3': 0, 'total': 0},
'paragraphs': 0,
'lists': 0,
'images': 0,
'links': {'internal': 0, 'external': 0},
'avg_paragraph_length': 0
}
paragraphs = []
current_para = []
for line in lines:
# Count headings
if line.startswith('# '):
structure['headings']['h1'] += 1
structure['headings']['total'] += 1
elif line.startswith('## '):
structure['headings']['h2'] += 1
structure['headings']['total'] += 1
elif line.startswith('### '):
structure['headings']['h3'] += 1
structure['headings']['total'] += 1
# Count lists
if line.strip().startswith(('- ', '* ', '1. ')):
structure['lists'] += 1
# Count links
internal_links = len(re.findall(r'\[.*?\]\(/.*?\)', line))
external_links = len(re.findall(r'\[.*?\]\(https?://.*?\)', line))
structure['links']['internal'] += internal_links
structure['links']['external'] += external_links
# Track paragraphs
if line.strip() and not line.startswith('#'):
current_para.append(line)
elif current_para:
paragraphs.append(' '.join(current_para))
current_para = []
if current_para:
paragraphs.append(' '.join(current_para))
structure['paragraphs'] = len(paragraphs)
if paragraphs:
avg_length = sum(len(p.split()) for p in paragraphs) / len(paragraphs)
structure['avg_paragraph_length'] = round(avg_length, 1)
return structure
def _analyze_readability(self, content: str) -> Dict:
"""Analyze content readability"""
sentences = re.split(r'[.!?]+', content)
words = content.split()
if not sentences or not words:
return {'score': 0, 'level': 'Unknown'}
avg_sentence_length = len(words) / len(sentences)
# Simple readability scoring
if avg_sentence_length < 15:
level = 'Easy'
score = 90
elif avg_sentence_length < 20:
level = 'Moderate'
score = 70
elif avg_sentence_length < 25:
level = 'Difficult'
score = 50
else:
level = 'Very Difficult'
score = 30
return {
'score': score,
'level': level,
'avg_sentence_length': round(avg_sentence_length, 1)
}
def _extract_lsi_keywords(self, content: str, primary_keyword: str) -> List[str]:
"""Extract potential LSI (semantically related) keywords"""
words = re.findall(r'\b[a-z]+\b', content.lower())
word_freq = {}
# Count word frequencies
for word in words:
if word not in self.stop_words and len(word) > 3:
word_freq[word] = word_freq.get(word, 0) + 1
# Sort by frequency and return top related terms
sorted_words = sorted(word_freq.items(), key=lambda x: x[1], reverse=True)
# Filter out the primary keyword and return top 10
lsi_keywords = []
for word, count in sorted_words:
if word != primary_keyword.lower() and count > 1:
lsi_keywords.append(word)
if len(lsi_keywords) >= 10:
break
return lsi_keywords
def _generate_meta_suggestions(self, content: str, keyword: str = None) -> Dict:
"""Generate SEO meta tag suggestions"""
# Extract first sentence for description base
sentences = re.split(r'[.!?]+', content)
first_sentence = sentences[0] if sentences else content[:160]
suggestions = {
'title': '',
'meta_description': '',
'url_slug': '',
'og_title': '',
'og_description': ''
}
if keyword:
# Title suggestion
suggestions['title'] = f"{keyword.title()} - Complete Guide"
if len(suggestions['title']) > 60:
suggestions['title'] = keyword.title()[:57] + "..."
# Meta description
desc_base = f"Learn everything about {keyword}. {first_sentence}"
if len(desc_base) > 160:
desc_base = desc_base[:157] + "..."
suggestions['meta_description'] = desc_base
# URL slug
suggestions['url_slug'] = re.sub(r'[^a-z0-9-]+', '-',
keyword.lower()).strip('-')
# Open Graph tags
suggestions['og_title'] = suggestions['title']
suggestions['og_description'] = suggestions['meta_description']
return suggestions
def _calculate_seo_score(self, analysis: Dict) -> int:
"""Calculate overall SEO optimization score"""
score = 0
max_score = 100
# Content length scoring (20 points)
if 300 <= analysis['content_length'] <= 2500:
score += 20
elif 200 <= analysis['content_length'] < 300:
score += 10
elif analysis['content_length'] > 2500:
score += 15
# Keyword optimization (30 points)
if analysis['keyword_analysis']:
kw_data = analysis['keyword_analysis']['primary_keyword']
# Density scoring
if 0.01 <= kw_data['density'] <= 0.03:
score += 15
elif 0.005 <= kw_data['density'] < 0.01:
score += 8
# Placement scoring
if kw_data['in_first_paragraph']:
score += 10
if kw_data.get('in_headings'):
score += 5
# Structure scoring (25 points)
struct = analysis['structure_analysis']
if struct['headings']['total'] > 0:
score += 10
if struct['paragraphs'] >= 3:
score += 10
if struct['links']['internal'] > 0 or struct['links']['external'] > 0:
score += 5
# Readability scoring (25 points)
readability_score = analysis['readability']['score']
score += int(readability_score * 0.25)
return min(score, max_score)
def _generate_recommendations(self, analysis: Dict) -> List[str]:
"""Generate SEO improvement recommendations"""
recommendations = []
# Content length recommendations
if analysis['content_length'] < 300:
recommendations.append(
f"Increase content length to at least 300 words (currently {analysis['content_length']})"
)
elif analysis['content_length'] > 3000:
recommendations.append(
"Consider breaking long content into multiple pages or adding a table of contents"
)
# Keyword recommendations
if analysis['keyword_analysis']:
kw_data = analysis['keyword_analysis']['primary_keyword']
if kw_data['density'] < 0.01:
recommendations.append(
f"Increase keyword density for '{kw_data['keyword']}' (currently {kw_data['density']:.2%})"
)
elif kw_data['density'] > 0.03:
recommendations.append(
f"Reduce keyword density to avoid over-optimization (currently {kw_data['density']:.2%})"
)
if not kw_data['in_first_paragraph']:
recommendations.append(
"Include primary keyword in the first paragraph"
)
# Structure recommendations
struct = analysis['structure_analysis']
if struct['headings']['total'] == 0:
recommendations.append("Add headings (H1, H2, H3) to improve content structure")
if struct['links']['internal'] == 0:
recommendations.append("Add internal links to related content")
if struct['avg_paragraph_length'] > 150:
recommendations.append("Break up long paragraphs for better readability")
# Readability recommendations
if analysis['readability']['avg_sentence_length'] > 20:
recommendations.append("Simplify sentences for better readability")
return recommendations
def optimize_content(content: str, keyword: str = None,
secondary_keywords: List[str] = None) -> str:
"""Main function to optimize content"""
optimizer = SEOOptimizer()
# Parse secondary keywords from comma-separated string if provided
if secondary_keywords and isinstance(secondary_keywords, str):
secondary_keywords = [kw.strip() for kw in secondary_keywords.split(',')]
results = optimizer.analyze(content, keyword, secondary_keywords)
# Format output
output = [
"=== SEO Content Analysis ===",
f"Overall SEO Score: {results['optimization_score']}/100",
f"Content Length: {results['content_length']} words",
f"",
"Content Structure:",
f" Headings: {results['structure_analysis']['headings']['total']}",
f" Paragraphs: {results['structure_analysis']['paragraphs']}",
f" Avg Paragraph Length: {results['structure_analysis']['avg_paragraph_length']} words",
f" Internal Links: {results['structure_analysis']['links']['internal']}",
f" External Links: {results['structure_analysis']['links']['external']}",
f"",
f"Readability: {results['readability']['level']} (Score: {results['readability']['score']})",
f""
]
if results['keyword_analysis']:
kw = results['keyword_analysis']['primary_keyword']
output.extend([
"Keyword Analysis:",
f" Primary Keyword: {kw['keyword']}",
f" Count: {kw['count']}",
f" Density: {kw['density']:.2%}",
f" In First Paragraph: {'Yes' if kw['in_first_paragraph'] else 'No'}",
f""
])
if results['keyword_analysis']['lsi_keywords']:
output.append(" Related Keywords Found:")
for lsi in results['keyword_analysis']['lsi_keywords'][:5]:
output.append(f" • {lsi}")
output.append("")
if results['meta_suggestions']:
output.extend([
"Meta Tag Suggestions:",
f" Title: {results['meta_suggestions']['title']}",
f" Description: {results['meta_suggestions']['meta_description']}",
f" URL Slug: {results['meta_suggestions']['url_slug']}",
f""
])
output.extend([
"Recommendations:",
])
for rec in results['recommendations']:
output.append(f" • {rec}")
return '\n'.join(output)
if __name__ == "__main__":
import sys
import argparse
parser = argparse.ArgumentParser(
description="SEO Content Optimizer - Analyzes and optimizes content for SEO"
)
parser.add_argument(
"file", nargs="?", default=None,
help="Text file to analyze"
)
parser.add_argument(
"--keyword", "-k", default=None,
help="Primary keyword to optimize for"
)
parser.add_argument(
"--secondary", "-s", default=None,
help="Comma-separated secondary keywords"
)
args = parser.parse_args()
if args.file:
with open(args.file, 'r') as f:
content = f.read()
print(optimize_content(content, args.keyword, args.secondary))
else:
print("Usage: python seo_optimizer.py <file> [--keyword primary] [--secondary kw1,kw2]")
FILE:templates/content-brief-template.md
# Content Brief Template
> Fill in every field before writing starts. Blank fields mean assumptions. Assumptions mean rewrites.
---
## Basic Info
| Field | Value |
|---|---|
| **Working Title** | |
| **Target Publish Date** | |
| **Author / Owner** | |
| **Content Type** | Blog post / Guide / Case study / Comparison / Listicle |
| **Target Length** | ~___ words |
| **Goal** | Awareness / Lead gen / SEO / Thought leadership / Product education |
---
## SEO Targeting
| Field | Value |
|---|---|
| **Primary Keyword** | |
| **Monthly Search Volume** | |
| **Keyword Difficulty** | |
| **Secondary Keywords** (2-4) | |
| **Search Intent** | Informational / Commercial / Navigational |
| **SERP Features to Target** | Featured snippet / FAQ / People Also Ask |
---
## Audience
**Who is reading this?**
(Job title, company stage, pain point they're searching from)
**What do they already know?**
(Level: beginner / intermediate / expert)
**What do they want to walk away with?**
(The specific outcome or answer)
**What's their biggest objection or doubt?**
(What might make them click away?)
---
## Angle & POV
**The core argument / unique angle:**
(One sentence — what's our take that's different from the competition?)
**Why should they trust us on this?**
(Our authority, experience, or data that backs the angle)
**What's the competition missing?**
(Specific gap in top-ranking content we're exploiting)
---
## Structure
**H1 (draft):**
**Intro approach:**
(Hook type: stat / story / counterintuitive claim / problem statement)
**H2 Outline:**
1.
2.
3.
4.
5.
(Add more as needed)
**Conclusion approach:**
(Summary + CTA or next step)
---
## Sources & Research
| Source | Type | Key Claim / Data Point |
|---|---|---|
| | Study / Report | |
| | Expert quote | |
| | Official docs | |
| | Data / survey | |
**Minimum 3 sources required before drafting.**
---
## Internal Linking
**Links FROM this piece (to existing content):**
- [Anchor text] → [URL / page title]
- [Anchor text] → [URL / page title]
**Links TO this piece (from existing content):**
- [Existing page] → link to this once published
---
## Competitive Pieces to Beat
| URL | Word Count | What They Do Well | What They Miss |
|---|---|---|---|
| | | | |
| | | | |
---
## Success Criteria
- [ ] Ranks on page 1 for primary keyword within 6 months
- [ ] Achieves target engagement (avg time on page > ___ min)
- [ ] Generates ___ leads / clicks to product within 30 days
- [ ] Other: ___
---
## Notes / Special Instructions
(Brand voice requirements, topics to avoid, tone calibration, product mentions)
Đưa mô hình ML vào sản xuất, xây MLOps pipeline, tích hợp LLM, feature store, giám sát drift, RAG và tối ưu chi phí.
---
name: "senior-ml-engineer"
description: ML engineering skill for productionizing models, building MLOps pipelines, and integrating LLMs. Covers model deployment, feature stores, drift monitoring, RAG systems, and cost optimization. Use when the user asks about deploying ML models to production, setting up MLOps infrastructure (MLflow, Kubeflow, Kubernetes, Docker), monitoring model performance or drift, building RAG pipelines, or integrating LLM APIs with retry logic and cost controls. Focused on production and operational concerns rather than model research or initial training.
triggers:
- MLOps pipeline
- model deployment
- feature store
- model monitoring
- drift detection
- RAG system
- LLM integration
- model serving
- A/B testing ML
- automated retraining
---
# Senior ML Engineer
Production ML engineering patterns for model deployment, MLOps infrastructure, and LLM integration.
---
## Table of Contents
- [Model Deployment Workflow](#model-deployment-workflow)
- [MLOps Pipeline Setup](#mlops-pipeline-setup)
- [LLM Integration Workflow](#llm-integration-workflow)
- [RAG System Implementation](#rag-system-implementation)
- [Model Monitoring](#model-monitoring)
- [Reference Documentation](#reference-documentation)
- [Tools](#tools)
---
## Model Deployment Workflow
Deploy a trained model to production with monitoring:
1. Export model to standardized format (ONNX, TorchScript, SavedModel)
2. Package model with dependencies in Docker container
3. Deploy to staging environment
4. Run integration tests against staging
5. Deploy canary (5% traffic) to production
6. Monitor latency and error rates for 1 hour
7. Promote to full production if metrics pass
8. **Validation:** p95 latency < 100ms, error rate < 0.1%
### Container Template
```dockerfile
FROM python:3.11-slim
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY model/ /app/model/
COPY src/ /app/src/
HEALTHCHECK CMD curl -f http://localhost:8080/health || exit 1
EXPOSE 8080
CMD ["uvicorn", "src.server:app", "--host", "0.0.0.0", "--port", "8080"]
```
### Serving Options
| Option | Latency | Throughput | Use Case |
|--------|---------|------------|----------|
| FastAPI + Uvicorn | Low | Medium | REST APIs, small models |
| Triton Inference Server | Very Low | Very High | GPU inference, batching |
| TensorFlow Serving | Low | High | TensorFlow models |
| TorchServe | Low | High | PyTorch models |
| Ray Serve | Medium | High | Complex pipelines, multi-model |
---
## MLOps Pipeline Setup
Establish automated training and deployment:
1. Configure feature store (Feast, Tecton) for training data
2. Set up experiment tracking (MLflow, Weights & Biases)
3. Create training pipeline with hyperparameter logging
4. Register model in model registry with version metadata
5. Configure staging deployment triggered by registry events
6. Set up A/B testing infrastructure for model comparison
7. Enable drift monitoring with alerting
8. **Validation:** New models automatically evaluated against baseline
### Feature Store Pattern
```python
from feast import Entity, Feature, FeatureView, FileSource
user = Entity(name="user_id", value_type=ValueType.INT64)
user_features = FeatureView(
name="user_features",
entities=["user_id"],
ttl=timedelta(days=1),
features=[
Feature(name="purchase_count_30d", dtype=ValueType.INT64),
Feature(name="avg_order_value", dtype=ValueType.FLOAT),
],
online=True,
source=FileSource(path="data/user_features.parquet"),
)
```
### Retraining Triggers
| Trigger | Detection | Action |
|---------|-----------|--------|
| Scheduled | Cron (weekly/monthly) | Full retrain |
| Performance drop | Accuracy < threshold | Immediate retrain |
| Data drift | PSI > 0.2 | Evaluate, then retrain |
| New data volume | X new samples | Incremental update |
---
## LLM Integration Workflow
Integrate LLM APIs into production applications:
1. Create provider abstraction layer for vendor flexibility
2. Implement retry logic with exponential backoff
3. Configure fallback to secondary provider
4. Set up token counting and context truncation
5. Add response caching for repeated queries
6. Implement cost tracking per request
7. Add structured output validation with Pydantic
8. **Validation:** Response parses correctly, cost within budget
### Provider Abstraction
```python
from abc import ABC, abstractmethod
from tenacity import retry, stop_after_attempt, wait_exponential
class LLMProvider(ABC):
@abstractmethod
def complete(self, prompt: str, **kwargs) -> str:
pass
@retry(stop=stop_after_attempt(3), wait=wait_exponential(min=1, max=10))
def call_llm_with_retry(provider: LLMProvider, prompt: str) -> str:
return provider.complete(prompt)
```
### Cost Management
| Provider | Input Cost | Output Cost |
|----------|------------|-------------|
| GPT-4 | $0.03/1K | $0.06/1K |
| GPT-3.5 | $0.0005/1K | $0.0015/1K |
| Claude 3 Opus | $0.015/1K | $0.075/1K |
| Claude 3 Haiku | $0.00025/1K | $0.00125/1K |
---
## RAG System Implementation
Build retrieval-augmented generation pipeline:
1. Choose vector database (Pinecone, Qdrant, Weaviate)
2. Select embedding model based on quality/cost tradeoff
3. Implement document chunking strategy
4. Create ingestion pipeline with metadata extraction
5. Build retrieval with query embedding
6. Add reranking for relevance improvement
7. Format context and send to LLM
8. **Validation:** Response references retrieved context, no hallucinations
### Vector Database Selection
| Database | Hosting | Scale | Latency | Best For |
|----------|---------|-------|---------|----------|
| Pinecone | Managed | High | Low | Production, managed |
| Qdrant | Both | High | Very Low | Performance-critical |
| Weaviate | Both | High | Low | Hybrid search |
| Chroma | Self-hosted | Medium | Low | Prototyping |
| pgvector | Self-hosted | Medium | Medium | Existing Postgres |
### Chunking Strategies
| Strategy | Chunk Size | Overlap | Best For |
|----------|------------|---------|----------|
| Fixed | 500-1000 tokens | 50-100 | General text |
| Sentence | 3-5 sentences | 1 sentence | Structured text |
| Semantic | Variable | Based on meaning | Research papers |
| Recursive | Hierarchical | Parent-child | Long documents |
---
## Model Monitoring
Monitor production models for drift and degradation:
1. Set up latency tracking (p50, p95, p99)
2. Configure error rate alerting
3. Implement input data drift detection
4. Track prediction distribution shifts
5. Log ground truth when available
6. Compare model versions with A/B metrics
7. Set up automated retraining triggers
8. **Validation:** Alerts fire before user-visible degradation
### Drift Detection
```python
from scipy.stats import ks_2samp
def detect_drift(reference, current, threshold=0.05):
statistic, p_value = ks_2samp(reference, current)
return {
"drift_detected": p_value < threshold,
"ks_statistic": statistic,
"p_value": p_value
}
```
### Alert Thresholds
| Metric | Warning | Critical |
|--------|---------|----------|
| p95 latency | > 100ms | > 200ms |
| Error rate | > 0.1% | > 1% |
| PSI (drift) | > 0.1 | > 0.2 |
| Accuracy drop | > 2% | > 5% |
---
## Reference Documentation
### MLOps Production Patterns
`references/mlops_production_patterns.md` contains:
- Model deployment pipeline with Kubernetes manifests
- Feature store architecture with Feast examples
- Model monitoring with drift detection code
- A/B testing infrastructure with traffic splitting
- Automated retraining pipeline with MLflow
### LLM Integration Guide
`references/llm_integration_guide.md` contains:
- Provider abstraction layer pattern
- Retry and fallback strategies with tenacity
- Prompt engineering templates (few-shot, CoT)
- Token optimization with tiktoken
- Cost calculation and tracking
### RAG System Architecture
`references/rag_system_architecture.md` contains:
- RAG pipeline implementation with code
- Vector database comparison and integration
- Chunking strategies (fixed, semantic, recursive)
- Embedding model selection guide
- Hybrid search and reranking patterns
---
## Tools
### Model Deployment Pipeline
```bash
python scripts/model_deployment_pipeline.py --model model.pkl --target staging
```
Generates deployment artifacts: Dockerfile, Kubernetes manifests, health checks.
### RAG System Builder
```bash
python scripts/rag_system_builder.py --config rag_config.yaml --analyze
```
Scaffolds RAG pipeline with vector store integration and retrieval logic.
### ML Monitoring Suite
```bash
python scripts/ml_monitoring_suite.py --config monitoring.yaml --deploy
```
Sets up drift detection, alerting, and performance dashboards.
---
## Tech Stack
| Category | Tools |
|----------|-------|
| ML Frameworks | PyTorch, TensorFlow, Scikit-learn, XGBoost |
| LLM Frameworks | LangChain, LlamaIndex, DSPy |
| MLOps | MLflow, Weights & Biases, Kubeflow |
| Data | Spark, Airflow, dbt, Kafka |
| Deployment | Docker, Kubernetes, Triton |
| Databases | PostgreSQL, BigQuery, Pinecone, Redis |
FILE:references/llm_integration_guide.md
# LLM Integration Guide
Production patterns for integrating Large Language Models into applications.
---
## Table of Contents
- [API Integration Patterns](#api-integration-patterns)
- [Prompt Engineering](#prompt-engineering)
- [Token Optimization](#token-optimization)
- [Cost Management](#cost-management)
- [Error Handling](#error-handling)
---
## API Integration Patterns
### Provider Abstraction Layer
```python
from abc import ABC, abstractmethod
from typing import List, Dict, Any
class LLMProvider(ABC):
"""Abstract base class for LLM providers."""
@abstractmethod
def complete(self, prompt: str, **kwargs) -> str:
pass
@abstractmethod
def chat(self, messages: List[Dict], **kwargs) -> str:
pass
class OpenAIProvider(LLMProvider):
def __init__(self, api_key: str, model: str = "gpt-4"):
self.client = OpenAI(api_key=api_key)
self.model = model
def complete(self, prompt: str, **kwargs) -> str:
response = self.client.completions.create(
model=self.model,
prompt=prompt,
**kwargs
)
return response.choices[0].text
class AnthropicProvider(LLMProvider):
def __init__(self, api_key: str, model: str = "claude-3-opus"):
self.client = Anthropic(api_key=api_key)
self.model = model
def chat(self, messages: List[Dict], **kwargs) -> str:
response = self.client.messages.create(
model=self.model,
messages=messages,
**kwargs
)
return response.content[0].text
```
### Retry and Fallback Strategy
```python
import time
from tenacity import retry, stop_after_attempt, wait_exponential
@retry(
stop=stop_after_attempt(3),
wait=wait_exponential(multiplier=1, min=1, max=10)
)
def call_llm_with_retry(provider: LLMProvider, prompt: str) -> str:
"""Call LLM with exponential backoff retry."""
return provider.complete(prompt)
def call_with_fallback(
primary: LLMProvider,
fallback: LLMProvider,
prompt: str
) -> str:
"""Try primary provider, fall back on failure."""
try:
return call_llm_with_retry(primary, prompt)
except Exception as e:
logger.warning(f"Primary provider failed: {e}, using fallback")
return call_llm_with_retry(fallback, prompt)
```
---
## Prompt Engineering
### Prompt Templates
| Pattern | Use Case | Structure |
|---------|----------|-----------|
| Zero-shot | Simple tasks | Task description + input |
| Few-shot | Complex tasks | Examples + task + input |
| Chain-of-thought | Reasoning | "Think step by step" + task |
| Role-based | Specialized output | System role + task |
### Few-Shot Template
```python
FEW_SHOT_TEMPLATE = """
You are a sentiment classifier. Classify the sentiment as positive, negative, or neutral.
Examples:
Input: "This product is amazing, I love it!"
Output: positive
Input: "Terrible experience, waste of money."
Output: negative
Input: "The product arrived on time."
Output: neutral
Now classify:
Input: "{user_input}"
Output:"""
def classify_sentiment(text: str, provider: LLMProvider) -> str:
prompt = FEW_SHOT_TEMPLATE.format(user_input=text)
response = provider.complete(prompt, max_tokens=10, temperature=0)
return response.strip().lower()
```
### System Prompts for Consistency
```python
SYSTEM_PROMPT = """You are a helpful assistant that answers questions about our product.
Guidelines:
- Be concise and direct
- Use bullet points for lists
- If unsure, say "I don't have that information"
- Never make up information
- Keep responses under 200 words
Product context:
{product_context}
"""
def create_chat_messages(user_query: str, context: str) -> List[Dict]:
return [
{"role": "system", "content": SYSTEM_PROMPT.format(product_context=context)},
{"role": "user", "content": user_query}
]
```
---
## Token Optimization
### Token Counting
```python
import tiktoken
def count_tokens(text: str, model: str = "gpt-4") -> int:
"""Count tokens for a given text and model."""
encoding = tiktoken.encoding_for_model(model)
return len(encoding.encode(text))
def truncate_to_token_limit(text: str, max_tokens: int, model: str = "gpt-4") -> str:
"""Truncate text to fit within token limit."""
encoding = tiktoken.encoding_for_model(model)
tokens = encoding.encode(text)
if len(tokens) <= max_tokens:
return text
return encoding.decode(tokens[:max_tokens])
```
### Context Window Management
| Model | Context Window | Effective Limit |
|-------|----------------|-----------------|
| GPT-4 | 8,192 | ~6,000 (leave room for response) |
| GPT-4-32k | 32,768 | ~28,000 |
| Claude 3 | 200,000 | ~180,000 |
| Llama 3 | 8,192 | ~6,000 |
### Chunking Strategy
```python
def chunk_text(text: str, chunk_size: int = 1000, overlap: int = 100) -> List[str]:
"""Split text into overlapping chunks."""
chunks = []
start = 0
while start < len(text):
end = start + chunk_size
chunk = text[start:end]
chunks.append(chunk)
start = end - overlap
return chunks
```
---
## Cost Management
### Cost Calculation
| Provider | Input Cost | Output Cost | Example (1K tokens) |
|----------|------------|-------------|---------------------|
| GPT-4 | $0.03/1K | $0.06/1K | $0.09 |
| GPT-3.5 | $0.0005/1K | $0.0015/1K | $0.002 |
| Claude 3 Opus | $0.015/1K | $0.075/1K | $0.09 |
| Claude 3 Haiku | $0.00025/1K | $0.00125/1K | $0.0015 |
### Cost Tracking
```python
from dataclasses import dataclass
from typing import Optional
@dataclass
class LLMUsage:
input_tokens: int
output_tokens: int
model: str
cost: float
def calculate_cost(
input_tokens: int,
output_tokens: int,
model: str
) -> float:
"""Calculate cost based on token usage."""
PRICING = {
"gpt-4": {"input": 0.03, "output": 0.06},
"gpt-3.5-turbo": {"input": 0.0005, "output": 0.0015},
"claude-3-opus": {"input": 0.015, "output": 0.075},
}
prices = PRICING.get(model, {"input": 0.01, "output": 0.03})
input_cost = (input_tokens / 1000) * prices["input"]
output_cost = (output_tokens / 1000) * prices["output"]
return input_cost + output_cost
```
### Cost Optimization Strategies
1. **Use smaller models for simple tasks** - GPT-3.5 for classification, GPT-4 for reasoning
2. **Cache common responses** - Store results for repeated queries
3. **Batch requests** - Combine multiple items in single prompt
4. **Truncate context** - Only include relevant information
5. **Set max_tokens limit** - Prevent runaway responses
---
## Error Handling
### Common Error Types
| Error | Cause | Handling |
|-------|-------|----------|
| RateLimitError | Too many requests | Exponential backoff |
| InvalidRequestError | Bad input | Validate before sending |
| AuthenticationError | Invalid API key | Check credentials |
| ServiceUnavailable | Provider down | Fallback to alternative |
| ContextLengthExceeded | Input too long | Truncate or chunk |
### Error Handling Pattern
```python
from openai import RateLimitError, APIError
def safe_llm_call(provider: LLMProvider, prompt: str, max_retries: int = 3) -> str:
"""Safely call LLM with comprehensive error handling."""
for attempt in range(max_retries):
try:
return provider.complete(prompt)
except RateLimitError:
wait_time = 2 ** attempt
logger.warning(f"Rate limited, waiting {wait_time}s")
time.sleep(wait_time)
except APIError as e:
if e.status_code >= 500:
logger.warning(f"Server error: {e}, retrying...")
time.sleep(1)
else:
raise
raise Exception(f"Failed after {max_retries} attempts")
```
### Response Validation
```python
import json
from pydantic import BaseModel, ValidationError
class StructuredResponse(BaseModel):
answer: str
confidence: float
sources: List[str]
def parse_structured_response(response: str) -> StructuredResponse:
"""Parse and validate LLM JSON response."""
try:
data = json.loads(response)
return StructuredResponse(**data)
except json.JSONDecodeError:
raise ValueError("Response is not valid JSON")
except ValidationError as e:
raise ValueError(f"Response validation failed: {e}")
```
FILE:references/mlops_production_patterns.md
# MLOps Production Patterns
Production ML infrastructure patterns for model deployment, monitoring, and lifecycle management.
---
## Table of Contents
- [Model Deployment Pipeline](#model-deployment-pipeline)
- [Feature Store Architecture](#feature-store-architecture)
- [Model Monitoring](#model-monitoring)
- [A/B Testing Infrastructure](#ab-testing-infrastructure)
- [Automated Retraining](#automated-retraining)
---
## Model Deployment Pipeline
### Deployment Workflow
1. Export trained model to standardized format (ONNX, TorchScript, SavedModel)
2. Package model with dependencies in Docker container
3. Deploy to staging environment
4. Run integration tests against staging
5. Deploy canary (5% traffic) to production
6. Monitor latency and error rates for 1 hour
7. Promote to full production if metrics pass
8. **Validation:** p95 latency < 100ms, error rate < 0.1%
### Container Structure
```dockerfile
FROM python:3.11-slim
# Install dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Copy model artifacts
COPY model/ /app/model/
COPY src/ /app/src/
# Health check endpoint
HEALTHCHECK CMD curl -f http://localhost:8080/health || exit 1
EXPOSE 8080
CMD ["uvicorn", "src.server:app", "--host", "0.0.0.0", "--port", "8080"]
```
### Model Serving Options
| Option | Latency | Throughput | Use Case |
|--------|---------|------------|----------|
| FastAPI + Uvicorn | Low | Medium | REST APIs, small models |
| Triton Inference Server | Very Low | Very High | GPU inference, batching |
| TensorFlow Serving | Low | High | TensorFlow models |
| TorchServe | Low | High | PyTorch models |
| Ray Serve | Medium | High | Complex pipelines, multi-model |
### Kubernetes Deployment
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: model-serving
spec:
replicas: 3
selector:
matchLabels:
app: model-serving
template:
spec:
containers:
- name: model
image: model:v1.0.0
resources:
requests:
memory: "2Gi"
cpu: "1"
limits:
memory: "4Gi"
cpu: "2"
readinessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 10
periodSeconds: 5
```
---
## Feature Store Architecture
### Feature Store Components
| Component | Purpose | Tools |
|-----------|---------|-------|
| Offline Store | Training data, batch features | BigQuery, Snowflake, S3 |
| Online Store | Low-latency serving | Redis, DynamoDB, Feast |
| Feature Registry | Metadata, lineage | Feast, Tecton, Hopsworks |
| Transformation | Feature engineering | Spark, Flink, dbt |
### Feature Pipeline Workflow
1. Define feature schema in registry
2. Implement transformation logic (SQL or Python)
3. Backfill historical features to offline store
4. Schedule incremental updates
5. Materialize to online store for serving
6. Monitor feature freshness and quality
7. **Validation:** Feature values within expected ranges, no nulls in required fields
### Feature Definition Example
```python
from feast import Entity, Feature, FeatureView, FileSource
user = Entity(name="user_id", value_type=ValueType.INT64)
user_features = FeatureView(
name="user_features",
entities=["user_id"],
ttl=timedelta(days=1),
features=[
Feature(name="purchase_count_30d", dtype=ValueType.INT64),
Feature(name="avg_order_value", dtype=ValueType.FLOAT),
Feature(name="days_since_last_purchase", dtype=ValueType.INT64),
],
online=True,
source=FileSource(path="data/user_features.parquet"),
)
```
---
## Model Monitoring
### Monitoring Dimensions
| Dimension | Metrics | Alert Threshold |
|-----------|---------|-----------------|
| Latency | p50, p95, p99 | p95 > 100ms |
| Throughput | requests/sec | < 80% baseline |
| Errors | error rate, 5xx count | > 0.1% |
| Data Drift | PSI, KS statistic | PSI > 0.2 |
| Model Drift | accuracy, AUC decay | > 5% drop |
### Data Drift Detection
```python
from scipy.stats import ks_2samp
import numpy as np
def detect_drift(reference: np.array, current: np.array, threshold: float = 0.05):
"""Detect distribution drift using Kolmogorov-Smirnov test."""
statistic, p_value = ks_2samp(reference, current)
drift_detected = p_value < threshold
return {
"drift_detected": drift_detected,
"ks_statistic": statistic,
"p_value": p_value,
"threshold": threshold
}
```
### Monitoring Dashboard Metrics
**Infrastructure:**
- Request latency (p50, p95, p99)
- Requests per second
- Error rate by type
- CPU/memory utilization
- GPU utilization (if applicable)
**Model Performance:**
- Prediction distribution
- Feature value distributions
- Model output confidence
- Ground truth vs predictions (when available)
---
## A/B Testing Infrastructure
### Experiment Workflow
1. Define experiment hypothesis and success metrics
2. Calculate required sample size for statistical power
3. Configure traffic split (control vs treatment)
4. Deploy treatment model alongside control
5. Route traffic based on user/session hash
6. Collect metrics for both variants
7. Run statistical significance test
8. **Validation:** p-value < 0.05, minimum sample size reached
### Traffic Splitting
```python
import hashlib
def get_variant(user_id: str, experiment: str, control_pct: float = 0.5) -> str:
"""Deterministic traffic splitting based on user ID."""
hash_input = f"{user_id}:{experiment}"
hash_value = int(hashlib.md5(hash_input.encode()).hexdigest(), 16)
bucket = (hash_value % 100) / 100.0
return "control" if bucket < control_pct else "treatment"
```
### Metrics Collection
| Metric Type | Examples | Collection Method |
|-------------|----------|-------------------|
| Primary | Conversion rate, revenue | Event logging |
| Secondary | Latency, engagement | Request logs |
| Guardrail | Error rate, crashes | Monitoring system |
---
## Automated Retraining
### Retraining Triggers
| Trigger | Detection Method | Action |
|---------|------------------|--------|
| Scheduled | Cron (weekly/monthly) | Full retrain |
| Performance drop | Accuracy < threshold | Immediate retrain |
| Data drift | PSI > 0.2 | Evaluate, then retrain |
| New data volume | X new samples | Incremental update |
### Retraining Pipeline
1. Trigger detection (schedule, drift, performance)
2. Fetch latest training data from feature store
3. Run training job with hyperparameter config
4. Evaluate model on holdout set
5. Compare against production model
6. If improved: register new model version
7. Deploy to staging for validation
8. Promote to production via canary
9. **Validation:** New model outperforms baseline on key metrics
### MLflow Model Registry Integration
```python
import mlflow
def register_model(model, metrics: dict, model_name: str):
"""Register trained model with MLflow."""
with mlflow.start_run():
# Log metrics
for name, value in metrics.items():
mlflow.log_metric(name, value)
# Log model
mlflow.sklearn.log_model(model, "model")
# Register in model registry
model_uri = f"runs:/{mlflow.active_run().info.run_id}/model"
mlflow.register_model(model_uri, model_name)
```
FILE:references/rag_system_architecture.md
# RAG System Architecture
Retrieval-Augmented Generation patterns for production applications.
---
## Table of Contents
- [RAG Pipeline Architecture](#rag-pipeline-architecture)
- [Vector Database Selection](#vector-database-selection)
- [Chunking Strategies](#chunking-strategies)
- [Embedding Models](#embedding-models)
- [Retrieval Optimization](#retrieval-optimization)
---
## RAG Pipeline Architecture
### Basic RAG Flow
1. Receive user query
2. Generate query embedding
3. Search vector database for relevant chunks
4. Rerank retrieved chunks by relevance
5. Format context with retrieved chunks
6. Send prompt to LLM with context
7. Return generated response
8. **Validation:** Response references retrieved context, no hallucinations
### Pipeline Components
```python
from dataclasses import dataclass
from typing import List
@dataclass
class Document:
content: str
metadata: dict
embedding: List[float] = None
@dataclass
class RetrievalResult:
document: Document
score: float
class RAGPipeline:
def __init__(
self,
embedder: Embedder,
vector_store: VectorStore,
llm: LLMProvider,
reranker: Reranker = None
):
self.embedder = embedder
self.vector_store = vector_store
self.llm = llm
self.reranker = reranker
def query(self, question: str, top_k: int = 5) -> str:
# 1. Embed query
query_embedding = self.embedder.embed(question)
# 2. Retrieve relevant documents
results = self.vector_store.search(query_embedding, top_k=top_k * 2)
# 3. Rerank if available
if self.reranker:
results = self.reranker.rerank(question, results)[:top_k]
else:
results = results[:top_k]
# 4. Build context
context = self._build_context(results)
# 5. Generate response
prompt = self._build_prompt(question, context)
return self.llm.complete(prompt)
def _build_context(self, results: List[RetrievalResult]) -> str:
return "\n\n".join([
f"[Source {i+1}]: {r.document.content}"
for i, r in enumerate(results)
])
def _build_prompt(self, question: str, context: str) -> str:
return f"""Answer the question based on the context provided.
Context:
{context}
Question: {question}
Answer:"""
```
---
## Vector Database Selection
### Comparison Matrix
| Database | Hosting | Scale | Latency | Cost | Best For |
|----------|---------|-------|---------|------|----------|
| Pinecone | Managed | High | Low | $$ | Production, managed |
| Weaviate | Both | High | Low | $ | Hybrid search |
| Qdrant | Both | High | Very Low | $ | Performance-critical |
| Chroma | Self-hosted | Medium | Low | Free | Prototyping |
| pgvector | Self-hosted | Medium | Medium | Free | Existing Postgres |
| Milvus | Both | Very High | Low | $ | Large-scale |
### Pinecone Integration
```python
import pinecone
class PineconeVectorStore:
def __init__(self, api_key: str, environment: str, index_name: str):
pinecone.init(api_key=api_key, environment=environment)
self.index = pinecone.Index(index_name)
def upsert(self, documents: List[Document], batch_size: int = 100):
"""Upsert documents in batches."""
vectors = [
(doc.metadata["id"], doc.embedding, doc.metadata)
for doc in documents
]
for i in range(0, len(vectors), batch_size):
batch = vectors[i:i + batch_size]
self.index.upsert(vectors=batch)
def search(self, embedding: List[float], top_k: int = 5) -> List[RetrievalResult]:
"""Search for similar vectors."""
results = self.index.query(
vector=embedding,
top_k=top_k,
include_metadata=True
)
return [
RetrievalResult(
document=Document(
content=match.metadata.get("content", ""),
metadata=match.metadata
),
score=match.score
)
for match in results.matches
]
```
---
## Chunking Strategies
### Strategy Comparison
| Strategy | Chunk Size | Overlap | Best For |
|----------|------------|---------|----------|
| Fixed | 500-1000 tokens | 50-100 | General text |
| Sentence | 3-5 sentences | 1 sentence | Structured text |
| Paragraph | Natural breaks | None | Documents with clear structure |
| Semantic | Variable | Based on meaning | Research papers |
| Recursive | Hierarchical | Parent-child | Long documents |
### Recursive Character Splitter
```python
from langchain.text_splitter import RecursiveCharacterTextSplitter
def create_chunks(
text: str,
chunk_size: int = 1000,
chunk_overlap: int = 100
) -> List[str]:
"""Split text using recursive character splitting."""
splitter = RecursiveCharacterTextSplitter(
chunk_size=chunk_size,
chunk_overlap=chunk_overlap,
separators=["\n\n", "\n", ". ", " ", ""]
)
return splitter.split_text(text)
```
### Semantic Chunking
```python
from sentence_transformers import SentenceTransformer
import numpy as np
def semantic_chunk(
sentences: List[str],
embedder: SentenceTransformer,
threshold: float = 0.7
) -> List[List[str]]:
"""Group sentences by semantic similarity."""
embeddings = embedder.encode(sentences)
chunks = []
current_chunk = [sentences[0]]
current_embedding = embeddings[0]
for i in range(1, len(sentences)):
similarity = np.dot(current_embedding, embeddings[i]) / (
np.linalg.norm(current_embedding) * np.linalg.norm(embeddings[i])
)
if similarity >= threshold:
current_chunk.append(sentences[i])
current_embedding = np.mean(
[current_embedding, embeddings[i]], axis=0
)
else:
chunks.append(current_chunk)
current_chunk = [sentences[i]]
current_embedding = embeddings[i]
chunks.append(current_chunk)
return chunks
```
---
## Embedding Models
### Model Comparison
| Model | Dimensions | Quality | Speed | Cost |
|-------|------------|---------|-------|------|
| text-embedding-3-large | 3072 | Excellent | Medium | $0.13/1M |
| text-embedding-3-small | 1536 | Good | Fast | $0.02/1M |
| BGE-large | 1024 | Excellent | Medium | Free |
| all-MiniLM-L6-v2 | 384 | Good | Very Fast | Free |
| Cohere embed-v3 | 1024 | Excellent | Medium | $0.10/1M |
### Embedding with Caching
```python
import hashlib
from functools import lru_cache
class CachedEmbedder:
def __init__(self, model_name: str = "text-embedding-3-small"):
self.client = OpenAI()
self.model = model_name
self._cache = {}
def embed(self, text: str) -> List[float]:
"""Embed text with caching."""
cache_key = hashlib.md5(text.encode()).hexdigest()
if cache_key in self._cache:
return self._cache[cache_key]
response = self.client.embeddings.create(
model=self.model,
input=text
)
embedding = response.data[0].embedding
self._cache[cache_key] = embedding
return embedding
def embed_batch(self, texts: List[str]) -> List[List[float]]:
"""Embed multiple texts efficiently."""
response = self.client.embeddings.create(
model=self.model,
input=texts
)
return [item.embedding for item in response.data]
```
---
## Retrieval Optimization
### Hybrid Search
Combine dense (vector) and sparse (keyword) retrieval:
```python
from rank_bm25 import BM25Okapi
class HybridRetriever:
def __init__(
self,
vector_store: VectorStore,
documents: List[Document],
alpha: float = 0.5
):
self.vector_store = vector_store
self.alpha = alpha # Weight for vector search
# Build BM25 index
tokenized = [doc.content.lower().split() for doc in documents]
self.bm25 = BM25Okapi(tokenized)
self.documents = documents
def search(self, query: str, query_embedding: List[float], top_k: int = 5):
# Vector search
vector_results = self.vector_store.search(query_embedding, top_k=top_k * 2)
# BM25 search
tokenized_query = query.lower().split()
bm25_scores = self.bm25.get_scores(tokenized_query)
# Combine scores
combined = {}
for result in vector_results:
doc_id = result.document.metadata["id"]
combined[doc_id] = self.alpha * result.score
for i, score in enumerate(bm25_scores):
doc_id = self.documents[i].metadata["id"]
if doc_id in combined:
combined[doc_id] += (1 - self.alpha) * score
else:
combined[doc_id] = (1 - self.alpha) * score
# Sort and return top_k
sorted_ids = sorted(combined.keys(), key=lambda x: combined[x], reverse=True)
return sorted_ids[:top_k]
```
### Reranking
```python
from sentence_transformers import CrossEncoder
class Reranker:
def __init__(self, model_name: str = "cross-encoder/ms-marco-MiniLM-L-12-v2"):
self.model = CrossEncoder(model_name)
def rerank(
self,
query: str,
results: List[RetrievalResult],
top_k: int = 5
) -> List[RetrievalResult]:
"""Rerank results using cross-encoder."""
pairs = [(query, r.document.content) for r in results]
scores = self.model.predict(pairs)
# Update scores and sort
for i, score in enumerate(scores):
results[i].score = float(score)
return sorted(results, key=lambda x: x.score, reverse=True)[:top_k]
```
### Query Expansion
```python
def expand_query(query: str, llm: LLMProvider) -> List[str]:
"""Generate query variations for better retrieval."""
prompt = f"""Generate 3 alternative phrasings of this question for search.
Return only the questions, one per line.
Original: {query}
Alternatives:"""
response = llm.complete(prompt, max_tokens=150)
alternatives = [q.strip() for q in response.strip().split("\n") if q.strip()]
return [query] + alternatives[:3]
```
FILE:scripts/ml_monitoring_suite.py
#!/usr/bin/env python3
"""
Ml Monitoring Suite
Production-grade tool for senior ml/ai engineer
"""
import os
import sys
import json
import logging
import argparse
from pathlib import Path
from typing import Dict, List, Optional
from datetime import datetime
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)
class MlMonitoringSuite:
"""Production-grade ml monitoring suite"""
def __init__(self, config: Dict):
self.config = config
self.results = {
'status': 'initialized',
'start_time': datetime.now().isoformat(),
'processed_items': 0
}
logger.info(f"Initialized {self.__class__.__name__}")
def validate_config(self) -> bool:
"""Validate configuration"""
logger.info("Validating configuration...")
# Add validation logic
logger.info("Configuration validated")
return True
def process(self) -> Dict:
"""Main processing logic"""
logger.info("Starting processing...")
try:
self.validate_config()
# Main processing
result = self._execute()
self.results['status'] = 'completed'
self.results['end_time'] = datetime.now().isoformat()
logger.info("Processing completed successfully")
return self.results
except Exception as e:
self.results['status'] = 'failed'
self.results['error'] = str(e)
logger.error(f"Processing failed: {e}")
raise
def _execute(self) -> Dict:
"""Execute main logic"""
# Implementation here
return {'success': True}
def main():
"""Main entry point"""
parser = argparse.ArgumentParser(
description="Ml Monitoring Suite"
)
parser.add_argument('--input', '-i', required=True, help='Input path')
parser.add_argument('--output', '-o', required=True, help='Output path')
parser.add_argument('--config', '-c', help='Configuration file')
parser.add_argument('--verbose', '-v', action='store_true', help='Verbose output')
args = parser.parse_args()
if args.verbose:
logging.getLogger().setLevel(logging.DEBUG)
try:
config = {
'input': args.input,
'output': args.output
}
processor = MlMonitoringSuite(config)
results = processor.process()
print(json.dumps(results, indent=2))
sys.exit(0)
except Exception as e:
logger.error(f"Fatal error: {e}")
sys.exit(1)
if __name__ == '__main__':
main()
FILE:scripts/model_deployment_pipeline.py
#!/usr/bin/env python3
"""
Model Deployment Pipeline
Production-grade tool for senior ml/ai engineer
"""
import os
import sys
import json
import logging
import argparse
from pathlib import Path
from typing import Dict, List, Optional
from datetime import datetime
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)
class ModelDeploymentPipeline:
"""Production-grade model deployment pipeline"""
def __init__(self, config: Dict):
self.config = config
self.results = {
'status': 'initialized',
'start_time': datetime.now().isoformat(),
'processed_items': 0
}
logger.info(f"Initialized {self.__class__.__name__}")
def validate_config(self) -> bool:
"""Validate configuration"""
logger.info("Validating configuration...")
# Add validation logic
logger.info("Configuration validated")
return True
def process(self) -> Dict:
"""Main processing logic"""
logger.info("Starting processing...")
try:
self.validate_config()
# Main processing
result = self._execute()
self.results['status'] = 'completed'
self.results['end_time'] = datetime.now().isoformat()
logger.info("Processing completed successfully")
return self.results
except Exception as e:
self.results['status'] = 'failed'
self.results['error'] = str(e)
logger.error(f"Processing failed: {e}")
raise
def _execute(self) -> Dict:
"""Execute main logic"""
# Implementation here
return {'success': True}
def main():
"""Main entry point"""
parser = argparse.ArgumentParser(
description="Model Deployment Pipeline"
)
parser.add_argument('--input', '-i', required=True, help='Input path')
parser.add_argument('--output', '-o', required=True, help='Output path')
parser.add_argument('--config', '-c', help='Configuration file')
parser.add_argument('--verbose', '-v', action='store_true', help='Verbose output')
args = parser.parse_args()
if args.verbose:
logging.getLogger().setLevel(logging.DEBUG)
try:
config = {
'input': args.input,
'output': args.output
}
processor = ModelDeploymentPipeline(config)
results = processor.process()
print(json.dumps(results, indent=2))
sys.exit(0)
except Exception as e:
logger.error(f"Fatal error: {e}")
sys.exit(1)
if __name__ == '__main__':
main()
FILE:scripts/rag_system_builder.py
#!/usr/bin/env python3
"""
Rag System Builder
Production-grade tool for senior ml/ai engineer
"""
import os
import sys
import json
import logging
import argparse
from pathlib import Path
from typing import Dict, List, Optional
from datetime import datetime
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)
class RagSystemBuilder:
"""Production-grade rag system builder"""
def __init__(self, config: Dict):
self.config = config
self.results = {
'status': 'initialized',
'start_time': datetime.now().isoformat(),
'processed_items': 0
}
logger.info(f"Initialized {self.__class__.__name__}")
def validate_config(self) -> bool:
"""Validate configuration"""
logger.info("Validating configuration...")
# Add validation logic
logger.info("Configuration validated")
return True
def process(self) -> Dict:
"""Main processing logic"""
logger.info("Starting processing...")
try:
self.validate_config()
# Main processing
result = self._execute()
self.results['status'] = 'completed'
self.results['end_time'] = datetime.now().isoformat()
logger.info("Processing completed successfully")
return self.results
except Exception as e:
self.results['status'] = 'failed'
self.results['error'] = str(e)
logger.error(f"Processing failed: {e}")
raise
def _execute(self) -> Dict:
"""Execute main logic"""
# Implementation here
return {'success': True}
def main():
"""Main entry point"""
parser = argparse.ArgumentParser(
description="Rag System Builder"
)
parser.add_argument('--input', '-i', required=True, help='Input path')
parser.add_argument('--output', '-o', required=True, help='Output path')
parser.add_argument('--config', '-c', help='Configuration file')
parser.add_argument('--verbose', '-v', action='store_true', help='Verbose output')
args = parser.parse_args()
if args.verbose:
logging.getLogger().setLevel(logging.DEBUG)
try:
config = {
'input': args.input,
'output': args.output
}
processor = RagSystemBuilder(config)
results = processor.process()
print(json.dumps(results, indent=2))
sys.exit(0)
except Exception as e:
logger.error(f"Fatal error: {e}")
sys.exit(1)
if __name__ == '__main__':
main()
Phân tích hiệu năng hệ thống cho Node.js, Python, Go: tìm nút thắt CPU, bộ nhớ, I/O, flamegraph, bundle, truy vấn CSDL và load test bằng k6, Artillery.
---
name: "performance-profiler"
description: "Systematic performance profiling for Node.js, Python, and Go applications. Identifies CPU, memory, and I/O bottlenecks, generates flamegraphs, analyzes bundle sizes, optimizes database queries, runs load tests with k6 and Artillery. Always measures before and after. Use when investigating a slow endpoint, planning a performance budget, or hunting a memory leak in production."
---
# Performance Profiler
**Tier:** POWERFUL
**Category:** Engineering
**Domain:** Performance Engineering
---
## Overview
Systematic performance profiling for Node.js, Python, and Go applications. Identifies CPU, memory, and I/O bottlenecks; generates flamegraphs; analyzes bundle sizes; optimizes database queries; detects memory leaks; and runs load tests with k6 and Artillery. Always measures before and after.
## Core Capabilities
- **CPU profiling** — flamegraphs for Node.js, py-spy for Python, pprof for Go
- **Memory profiling** — heap snapshots, leak detection, GC pressure
- **Bundle analysis** — webpack-bundle-analyzer, Next.js bundle analyzer
- **Database optimization** — EXPLAIN ANALYZE, slow query log, N+1 detection
- **Load testing** — k6 scripts, Artillery scenarios, ramp-up patterns
- **Before/after measurement** — establish baseline, profile, optimize, verify
---
## When to Use
- App is slow and you don't know where the bottleneck is
- P99 latency exceeds SLA before a release
- Memory usage grows over time (suspected leak)
- Bundle size increased after adding dependencies
- Preparing for a traffic spike (load test before launch)
- Database queries taking >100ms
---
## Quick Start
```bash
# Analyze a project for performance risk indicators
python3 scripts/performance_profiler.py /path/to/project
# JSON output for CI integration
python3 scripts/performance_profiler.py /path/to/project --json
# Custom large-file threshold
python3 scripts/performance_profiler.py /path/to/project --large-file-threshold-kb 256
```
---
## Golden Rule: Measure First
```bash
# Establish baseline BEFORE any optimization
# Record: P50, P95, P99 latency | RPS | error rate | memory usage
# Wrong: "I think the N+1 query is slow, let me fix it"
# Right: Profile → confirm bottleneck → fix → measure again → verify improvement
```
---
## Node.js Profiling
→ See references/profiling-recipes.md for details
## Before/After Measurement Template
```markdown
## Performance Optimization: [What You Fixed]
**Date:** 2026-03-01
**Engineer:** @username
**Ticket:** PROJ-123
### Problem
[1-2 sentences: what was slow, how was it observed]
### Root Cause
[What the profiler revealed]
### Baseline (Before)
| Metric | Value |
|--------|-------|
| P50 latency | 480ms |
| P95 latency | 1,240ms |
| P99 latency | 3,100ms |
| RPS @ 50 VUs | 42 |
| Error rate | 0.8% |
| DB queries/req | 23 (N+1) |
Profiler evidence: [link to flamegraph or screenshot]
### Fix Applied
[What changed — code diff or description]
### After
| Metric | Before | After | Delta |
|--------|--------|-------|-------|
| P50 latency | 480ms | 48ms | -90% |
| P95 latency | 1,240ms | 120ms | -90% |
| P99 latency | 3,100ms | 280ms | -91% |
| RPS @ 50 VUs | 42 | 380 | +804% |
| Error rate | 0.8% | 0% | -100% |
| DB queries/req | 23 | 1 | -96% |
### Verification
Load test run: [link to k6 output]
```
---
## Optimization Checklist
### Quick wins (check these first)
```
Database
□ Missing indexes on WHERE/ORDER BY columns
□ N+1 queries (check query count per request)
□ Loading all columns when only 2-3 needed (SELECT *)
□ No LIMIT on unbounded queries
□ Missing connection pool (creating new connection per request)
Node.js
□ Sync I/O (fs.readFileSync) in hot path
□ JSON.parse/stringify of large objects in hot loop
□ Missing caching for expensive computations
□ No compression (gzip/brotli) on responses
□ Dependencies loaded in request handler (move to module level)
Bundle
□ Moment.js → dayjs/date-fns
□ Lodash (full) → lodash/function imports
□ Static imports of heavy components → dynamic imports
□ Images not optimized / not using next/image
□ No code splitting on routes
API
□ No pagination on list endpoints
□ No response caching (Cache-Control headers)
□ Serial awaits that could be parallel (Promise.all)
□ Fetching related data in a loop instead of JOIN
```
---
## Common Pitfalls
- **Optimizing without measuring** — you'll optimize the wrong thing
- **Testing in development** — profile against production-like data volumes
- **Ignoring P99** — P50 can look fine while P99 is catastrophic
- **Premature optimization** — fix correctness first, then performance
- **Not re-measuring** — always verify the fix actually improved things
- **Load testing production** — use staging with production-size data
---
## Best Practices
1. **Baseline first, always** — record metrics before touching anything
2. **One change at a time** — isolate the variable to confirm causation
3. **Profile with realistic data** — 10 rows in dev, millions in prod — different bottlenecks
4. **Set performance budgets** — `p(95) < 200ms` in CI thresholds with k6
5. **Monitor continuously** — add Datadog/Prometheus metrics for key paths
6. **Cache invalidation strategy** — cache aggressively, invalidate precisely
7. **Document the win** — before/after in the PR description motivates the team
FILE:references/profiling-recipes.md
# performance-profiler reference
## Node.js Profiling
### CPU Flamegraph
```bash
# Method 1: clinic.js (best for development)
npm install -g clinic
# CPU flamegraph
clinic flame -- node dist/server.js
# Heap profiler
clinic heapprofiler -- node dist/server.js
# Bubble chart (event loop blocking)
clinic bubbles -- node dist/server.js
# Load with autocannon while profiling
autocannon -c 50 -d 30 http://localhost:3000/api/tasks &
clinic flame -- node dist/server.js
```
```bash
# Method 2: Node.js built-in profiler
node --prof dist/server.js
# After running some load:
node --prof-process isolate-*.log | head -100
```
```bash
# Method 3: V8 CPU profiler via inspector
node --inspect dist/server.js
# Open Chrome DevTools → Performance → Record
```
### Heap Snapshot / Memory Leak Detection
```javascript
// Add to your server for on-demand heap snapshots
import v8 from 'v8'
import fs from 'fs'
// Endpoint: POST /debug/heap-snapshot (protect with auth!)
app.post('/debug/heap-snapshot', (req, res) => {
const filename = `heap-Date.now().heapsnapshot`
const snapshot = v8.writeHeapSnapshot(filename)
res.json({ snapshot })
})
```
```bash
# Take snapshots over time and compare in Chrome DevTools
curl -X POST http://localhost:3000/debug/heap-snapshot
# Wait 5 minutes of load
curl -X POST http://localhost:3000/debug/heap-snapshot
# Open both snapshots in Chrome → Memory → Compare
```
### Detect Event Loop Blocking
```javascript
// Add blocked-at to detect synchronous blocking
import blocked from 'blocked-at'
blocked((time, stack) => {
console.warn(`Event loop blocked for timems`)
console.warn(stack.join('\n'))
}, { threshold: 100 }) // Alert if blocked > 100ms
```
### Node.js Memory Profiling Script
```javascript
// scripts/memory-profile.mjs
// Run: node --experimental-vm-modules scripts/memory-profile.mjs
import { createRequire } from 'module'
const require = createRequire(import.meta.url)
function formatBytes(bytes) {
return (bytes / 1024 / 1024).toFixed(2) + ' MB'
}
function measureMemory(label) {
const mem = process.memoryUsage()
console.log(`\n[label]`)
console.log(` RSS: formatBytes(mem.rss)`)
console.log(` Heap Used: formatBytes(mem.heapUsed)`)
console.log(` Heap Total:formatBytes(mem.heapTotal)`)
console.log(` External: formatBytes(mem.external)`)
return mem
}
const baseline = measureMemory('Baseline')
// Simulate your operation
for (let i = 0; i < 1000; i++) {
// Replace with your actual operation
const result = await someOperation()
}
const after = measureMemory('After 1000 operations')
console.log(`\n[Delta]`)
console.log(` Heap Used: +formatBytes(after.heapUsed - baseline.heapUsed)`)
// If heap keeps growing across GC cycles, you have a leak
global.gc?.() // Run with --expose-gc flag
const afterGC = measureMemory('After GC')
if (afterGC.heapUsed > baseline.heapUsed * 1.1) {
console.warn('⚠️ Possible memory leak detected (>10% growth after GC)')
}
```
---
## Python Profiling
### CPU Profiling with py-spy
```bash
# Install
pip install py-spy
# Profile a running process (no code changes needed)
py-spy top --pid $(pgrep -f "uvicorn")
# Generate flamegraph SVG
py-spy record -o flamegraph.svg --pid $(pgrep -f "uvicorn") --duration 30
# Profile from the start
py-spy record -o flamegraph.svg -- python -m uvicorn app.main:app
# Open flamegraph.svg in browser — look for wide bars = hot code paths
```
### cProfile for function-level profiling
```python
# scripts/profile_endpoint.py
import cProfile
import pstats
import io
from app.services.task_service import TaskService
def run():
service = TaskService()
for _ in range(100):
service.list_tasks(user_id="user_1", page=1, limit=20)
profiler = cProfile.Profile()
profiler.enable()
run()
profiler.disable()
# Print top 20 functions by cumulative time
stream = io.StringIO()
stats = pstats.Stats(profiler, stream=stream)
stats.sort_stats('cumulative')
stats.print_stats(20)
print(stream.getvalue())
```
### Memory profiling with memory_profiler
```python
# pip install memory-profiler
from memory_profiler import profile
@profile
def my_function():
# Function to profile
data = load_large_dataset()
result = process(data)
return result
```
```bash
# Run with line-by-line memory tracking
python -m memory_profiler scripts/profile_function.py
# Output:
# Line # Mem usage Increment Line Contents
# ================================================
# 10 45.3 MiB 45.3 MiB def my_function():
# 11 78.1 MiB 32.8 MiB data = load_large_dataset()
# 12 156.2 MiB 78.1 MiB result = process(data)
```
---
## Go Profiling with pprof
```go
// main.go — add pprof endpoints
import _ "net/http/pprof"
import "net/http"
func main() {
// pprof endpoints at /debug/pprof/
go func() {
log.Println(http.ListenAndServe(":6060", nil))
}()
// ... rest of your app
}
```
```bash
# CPU profile (30s)
go tool pprof -http=:8080 http://localhost:6060/debug/pprof/profile?seconds=30
# Memory profile
go tool pprof -http=:8080 http://localhost:6060/debug/pprof/heap
# Goroutine leak detection
curl http://localhost:6060/debug/pprof/goroutine?debug=1
# In pprof UI: "Flame Graph" view → find the tallest bars
```
---
## Bundle Size Analysis
### Next.js Bundle Analyzer
```bash
# Install
pnpm add -D @next/bundle-analyzer
# next.config.js
const withBundleAnalyzer = require('@next/bundle-analyzer')({
enabled: process.env.ANALYZE === 'true',
})
module.exports = withBundleAnalyzer({})
# Run analyzer
ANALYZE=true pnpm build
# Opens browser with treemap of bundle
```
### What to look for
```bash
# Find the largest chunks
pnpm build 2>&1 | grep -E "^\s+(λ|○|●)" | sort -k4 -rh | head -20
# Check if a specific package is too large
# Visit: https://bundlephobia.com/package/moment@2.29.4
# moment: 67.9kB gzipped → replace with date-fns (13.8kB) or dayjs (6.9kB)
# Find duplicate packages
pnpm dedupe --check
# Visualize what's in a chunk
npx source-map-explorer .next/static/chunks/*.js
```
### Common bundle wins
```typescript
// Before: import entire lodash
import _ from 'lodash' // 71kB
// After: import only what you need
import debounce from 'lodash/debounce' // 2kB
// Before: moment.js
import moment from 'moment' // 67kB
// After: dayjs
import dayjs from 'dayjs' // 7kB
// Before: static import (always in bundle)
import HeavyChart from '@/components/HeavyChart'
// After: dynamic import (loaded on demand)
const HeavyChart = dynamic(() => import('@/components/HeavyChart'), {
loading: () => <Skeleton />,
})
```
---
## Database Query Optimization
### Find slow queries
```sql
-- PostgreSQL: enable pg_stat_statements
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
-- Top 20 slowest queries
SELECT
round(mean_exec_time::numeric, 2) AS mean_ms,
calls,
round(total_exec_time::numeric, 2) AS total_ms,
round(stddev_exec_time::numeric, 2) AS stddev_ms,
left(query, 80) AS query
FROM pg_stat_statements
WHERE calls > 10
ORDER BY mean_exec_time DESC
LIMIT 20;
-- Reset stats
SELECT pg_stat_statements_reset();
```
```bash
# MySQL slow query log
mysql -e "SET GLOBAL slow_query_log = 'ON'; SET GLOBAL long_query_time = 0.1;"
tail -f /var/log/mysql/slow-query.log
```
### EXPLAIN ANALYZE
```sql
-- Always use EXPLAIN (ANALYZE, BUFFERS) for real timing
EXPLAIN (ANALYZE, BUFFERS, FORMAT TEXT)
SELECT t.*, u.name as assignee_name
FROM tasks t
LEFT JOIN users u ON u.id = t.assignee_id
WHERE t.project_id = 'proj_123'
AND t.deleted_at IS NULL
ORDER BY t.created_at DESC
LIMIT 20;
-- Look for:
-- Seq Scan on large table → needs index
-- Nested Loop with high rows → N+1, consider JOIN or batch
-- Sort → can index handle the sort?
-- Hash Join → fine for moderate sizes
```
### Detect N+1 Queries
```typescript
// Add query logging in dev
import { db } from './client'
// Drizzle: enable logging
const db = drizzle(pool, { logger: true })
// Or use a query counter middleware
let queryCount = 0
db.$on('query', () => queryCount++)
// In tests:
queryCount = 0
const tasks = await getTasksWithAssignees(projectId)
expect(queryCount).toBe(1) // Fail if it's 21 (1 + 20 N+1s)
```
```python
# Django: detect N+1 with django-silk or nplusone
from nplusone.ext.django.middleware import NPlusOneMiddleware
MIDDLEWARE = ['nplusone.ext.django.middleware.NPlusOneMiddleware']
NPLUSONE_RAISE = True # Raise exception on N+1 in tests
```
### Fix N+1 — Before/After
```typescript
// Before: N+1 (1 query for tasks + N queries for assignees)
const tasks = await db.select().from(tasksTable)
for (const task of tasks) {
task.assignee = await db.select().from(usersTable)
.where(eq(usersTable.id, task.assigneeId))
.then(r => r[0])
}
// After: 1 query with JOIN
const tasks = await db
.select({
id: tasksTable.id,
title: tasksTable.title,
assigneeName: usersTable.name,
assigneeEmail: usersTable.email,
})
.from(tasksTable)
.leftJoin(usersTable, eq(usersTable.id, tasksTable.assigneeId))
.where(eq(tasksTable.projectId, projectId))
```
---
## Load Testing with k6
```javascript
// tests/load/api-load-test.js
import http from 'k6/http'
import { check, sleep } from 'k6'
import { Rate, Trend } from 'k6/metrics'
const errorRate = new Rate('errors')
const taskListDuration = new Trend('task_list_duration')
export const options = {
stages: [
{ duration: '30s', target: 10 }, // Ramp up to 10 VUs
{ duration: '1m', target: 50 }, // Ramp to 50 VUs
{ duration: '2m', target: 50 }, // Sustain 50 VUs
{ duration: '30s', target: 100 }, // Spike to 100 VUs
{ duration: '1m', target: 50 }, // Back to 50
{ duration: '30s', target: 0 }, // Ramp down
],
thresholds: {
http_req_duration: ['p(95)<500'], // 95% of requests < 500ms
http_req_duration: ['p(99)<1000'], // 99% < 1s
errors: ['rate<0.01'], // Error rate < 1%
task_list_duration: ['p(95)<200'], // Task list specifically < 200ms
},
}
const BASE_URL = __ENV.BASE_URL || 'http://localhost:3000'
export function setup() {
// Get auth token once
const loginRes = http.post(`BASE_URL/api/auth/login`, JSON.stringify({
email: 'loadtest@example.com',
password: 'loadtest123',
}), { headers: { 'Content-Type': 'application/json' } })
return { token: loginRes.json('token') }
}
export default function(data) {
const headers = {
'Authorization': `Bearer data.token`,
'Content-Type': 'application/json',
}
// Scenario 1: List tasks
const start = Date.now()
const listRes = http.get(`BASE_URL/api/tasks?limit=20`, { headers })
taskListDuration.add(Date.now() - start)
check(listRes, {
'list tasks: status 200': (r) => r.status === 200,
'list tasks: has items': (r) => r.json('items') !== undefined,
}) || errorRate.add(1)
sleep(0.5)
// Scenario 2: Create task
const createRes = http.post(
`BASE_URL/api/tasks`,
JSON.stringify({ title: `Load test task Date.now()`, priority: 'medium' }),
{ headers }
)
check(createRes, {
'create task: status 201': (r) => r.status === 201,
}) || errorRate.add(1)
sleep(1)
}
export function teardown(data) {
// Cleanup: delete load test tasks
}
```
```bash
# Run load test
k6 run tests/load/api-load-test.js \
--env BASE_URL=https://staging.myapp.com
# With Grafana output
k6 run --out influxdb=http://localhost:8086/k6 tests/load/api-load-test.js
```
---
FILE:scripts/performance_profiler.py
#!/usr/bin/env python3
"""Lightweight repo performance profiling helper (stdlib only)."""
from __future__ import annotations
import argparse
import json
import os
from pathlib import Path
from typing import Dict, Iterable, List, Tuple
EXT_WEIGHTS = {
".js": 1.0,
".jsx": 1.0,
".ts": 1.0,
".tsx": 1.0,
".css": 0.7,
".map": 2.0,
}
def iter_files(root: Path) -> Iterable[Path]:
for dirpath, dirnames, filenames in os.walk(root):
dirnames[:] = [d for d in dirnames if d not in {".git", "node_modules", ".next", "dist", "build", "coverage", "__pycache__"}]
for filename in filenames:
path = Path(dirpath) / filename
if path.is_file():
yield path
def get_large_files(root: Path, threshold_bytes: int) -> List[Tuple[str, int]]:
large: List[Tuple[str, int]] = []
for file_path in iter_files(root):
size = file_path.stat().st_size
if size >= threshold_bytes:
large.append((str(file_path.relative_to(root)), size))
return sorted(large, key=lambda item: item[1], reverse=True)
def count_dependencies(root: Path) -> Dict[str, int]:
counts = {"node_dependencies": 0, "python_dependencies": 0, "go_dependencies": 0}
package_json = root / "package.json"
if package_json.exists():
try:
data = json.loads(package_json.read_text(encoding="utf-8"))
deps = data.get("dependencies", {})
dev_deps = data.get("devDependencies", {})
counts["node_dependencies"] = len(deps) + len(dev_deps)
except Exception:
pass
requirements = root / "requirements.txt"
if requirements.exists():
lines = [ln.strip() for ln in requirements.read_text(encoding="utf-8", errors="ignore").splitlines()]
counts["python_dependencies"] = sum(1 for ln in lines if ln and not ln.startswith("#"))
go_mod = root / "go.mod"
if go_mod.exists():
lines = go_mod.read_text(encoding="utf-8", errors="ignore").splitlines()
in_require_block = False
go_count = 0
for ln in lines:
s = ln.strip()
if s.startswith("require ("):
in_require_block = True
continue
if in_require_block and s == ")":
in_require_block = False
continue
if in_require_block and s and not s.startswith("//"):
go_count += 1
elif s.startswith("require ") and not s.endswith("("):
go_count += 1
counts["go_dependencies"] = go_count
return counts
def bundle_indicators(root: Path) -> Dict[str, object]:
indicators = {
"build_dirs_present": [],
"bundle_like_files": 0,
"estimated_bundle_weight": 0.0,
}
for d in ["dist", "build", ".next", "out"]:
if (root / d).exists():
indicators["build_dirs_present"].append(d)
bundle_files = 0
weight = 0.0
for path in iter_files(root):
ext = path.suffix.lower()
if ext in EXT_WEIGHTS:
bundle_files += 1
size_kb = path.stat().st_size / 1024.0
weight += size_kb * EXT_WEIGHTS[ext]
indicators["bundle_like_files"] = bundle_files
indicators["estimated_bundle_weight"] = round(weight, 2)
return indicators
def format_size(num_bytes: int) -> str:
units = ["B", "KB", "MB", "GB"]
value = float(num_bytes)
for unit in units:
if value < 1024.0 or unit == units[-1]:
return f"{value:.1f}{unit}"
value /= 1024.0
return f"{num_bytes}B"
def build_report(root: Path, threshold_bytes: int) -> Dict[str, object]:
large = get_large_files(root, threshold_bytes)
deps = count_dependencies(root)
bundles = bundle_indicators(root)
return {
"root": str(root),
"large_file_threshold_bytes": threshold_bytes,
"large_files": large,
"dependency_counts": deps,
"bundle_indicators": bundles,
}
def print_text(report: Dict[str, object]) -> None:
print("Performance Profile Report")
print(f"Root: {report['root']}")
print(f"Large-file threshold: {format_size(int(report['large_file_threshold_bytes']))}")
print("")
dep_counts = report["dependency_counts"]
print("Dependency Counts")
print(f"- Node: {dep_counts['node_dependencies']}")
print(f"- Python: {dep_counts['python_dependencies']}")
print(f"- Go: {dep_counts['go_dependencies']}")
print("")
bundle = report["bundle_indicators"]
print("Bundle Indicators")
print(f"- Build directories present: {', '.join(bundle['build_dirs_present']) or 'none'}")
print(f"- Bundle-like files: {bundle['bundle_like_files']}")
print(f"- Estimated weighted bundle size: {bundle['estimated_bundle_weight']} KB")
print("")
print("Large Files")
large_files = report["large_files"]
if not large_files:
print("- None above threshold")
else:
for rel_path, size in large_files[:20]:
print(f"- {rel_path}: {format_size(size)}")
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Analyze a project directory for common performance risk indicators."
)
parser.add_argument("path", help="Directory to analyze")
parser.add_argument(
"--large-file-threshold-kb",
type=int,
default=512,
help="Threshold in KB for reporting large files (default: 512)",
)
parser.add_argument(
"--json",
action="store_true",
help="Print JSON output instead of text",
)
return parser.parse_args()
def main() -> int:
args = parse_args()
root = Path(args.path).expanduser().resolve()
if not root.exists() or not root.is_dir():
raise SystemExit(f"Path is not a directory: {root}")
threshold = max(1, args.large_file_threshold_kb) * 1024
report = build_report(root, threshold)
if args.json:
print(json.dumps(report, indent=2))
else:
print_text(report)
return 0
if __name__ == "__main__":
raise SystemExit(main())
Chất vấn 6 câu hỏi chuẩn bị audit FDA 21 CFR 820 (QSR/QMSR) trước audit nội bộ, thanh tra FDA hoặc phản hồi Form 483.
--- name: "fda-qsr-audit-prep" description: "/cs:fda-qsr-audit-prep <scope> — FDA 21 CFR 820 (QSR / QMSR) audit 6-question forcing interrogation. Post-Feb 2026 substantially harmonized with ISO 13485. Use before annual internal QSR audit, pre-FDA-inspection readiness, or Form 483 response." --- # /cs:fda-qsr-audit-prep — FDA QSR Forcing Questions **Command:** `/cs:fda-qsr-audit-prep <scope>` The FDA QSR auditor pressure-tests any US medical-device QSR work. Six questions before any internal audit, FDA inspection, Form 483 response, or recall decision. ## When to Run - Before annual internal QSR audit - Before pre-FDA-inspection readiness review (any device commercially distributed in US) - After receiving Form 483 observations - After Warning Letter receipt - After MDR-reportable event - Before recall decision (voluntary vs FDA-initiated) - Before submitting 510(k) / PMA (where QSR posture affects approval timeline) ## The Six QSR Questions ### 1. Show me the complaint files from the last quarter — and the corresponding MDR reports. **21 CFR 820.198 + 21 CFR 803 — most-cited FDA inspection area.** - Complaint log complete: who / what / when / device / batch - Investigation closure within reasonable timeline - MDR-reporting decision tree applied: death OR serious injury OR malfunction-that-could-cause = MDR - 30-day timeline for most MDR reports; 5 days for certain serious events - Complaint trending input to management review ### 2. When was process validation (IQ/OQ/PQ) last revalidated per 21 CFR 820.75? **Cross-walks ISO 13485 Clause 7.5.6 (substantially harmonized post-Feb 2026).** - Initial validation at process introduction - Revalidation triggers: process / equipment / material change OR periodic schedule - Statistical techniques per 21 CFR 820.250 where applicable - Cross-check with cs-cqm-iso13485 for ISO 13485 alignment ### 3. Show me the DHRs for products commercially distributed in last 2 years. **21 CFR 820.180 — 2-year retention from commercial distribution; check sampling for completeness.** - Device History Record (DHR) for each unit/lot/batch - Must include: dates of manufacture, quantity manufactured, quantity released, acceptance records, primary identification label, device identification, control number - Sample stratified by product class - Verify DHR closeness to DHF (design history file) ### 4. Show me CAPAs from the last 6 months with effectiveness verification. **21 CFR 820.100 = ISO 13485 8.5.2 substantially harmonized.** - Root cause analysis depth (5 Why minimum) - Effectiveness verification = measurable evidence, not "we updated the procedure" - Containment / correction / corrective action distinction documented - Closure approval by appropriate authority - Aging CAPAs > 90 days flagged ### 5. Show me labeling (21 CFR 801) review for the most recent product launch. **FDA-specific overlay not in ISO 13485.** - Labeling per 21 CFR 801 requirements - For specific device types: also 21 CFR 800 series sectoral overlays - UDI (Unique Device Identification) per 21 CFR 830 - Promotional materials reviewed for accuracy + non-misleading ### 6. If a Form 483 was issued in the last 3 years, show me the closure status. **Form 483 = FDA observation; not equivalent to ISO nonconformity.** - Response within 15 working days - Each observation has documented corrective + preventive action with timeline - Effectiveness verification evidence - For Warning Letters: separate response track + potentially FDA meeting ## Workflow ```bash # 1. QSR compliance posture python ../../ra-qm-team/skills/fda-consultant-specialist/scripts/qsr_compliance_checker.py compliance_state.json # 2. FDA submission tracking (510(k) / PMA / IDE) python ../../ra-qm-team/skills/fda-consultant-specialist/scripts/fda_submission_tracker.py submissions.json # 3. HIPAA overlap (if connected device handles PHI) python ../../ra-qm-team/skills/fda-consultant-specialist/scripts/hipaa_risk_assessment.py phi_inventory.json # 4. Mock FDA inspection python ../../skills/compliance-os/scripts/audit_simulator.py fda_qsr_scope.json ``` ## Output Format ```markdown # FDA QSR Audit Prep: <scope> **Date:** YYYY-MM-DD ## The Decision Being Made [programme-plan | inspection-readiness | 483-response | MDR-decision | recall] ## Complaint + MDR Posture - Complaints last quarter: N - MDR-reportable events: M - MDR reports filed within timeline: % (target 100%) - Complaint trending review at management level: yes/no ## Process Validation Status (21 CFR 820.75) - Validations on schedule: % - Stale validations: <list> - Statistical techniques applied: yes/no per process ## DHR Completeness (21 CFR 820.180) - DHRs sampled: N - Completeness rate: % - 2-year retention compliant: yes/no - Stratified by product class: yes/no ## CAPA Health (21 CFR 820.100) - CAPAs sampled: N - Root cause analysis depth: adequate/inadequate - Effectiveness verification: complete/incomplete - Aging CAPAs > 90 days: N ## Labeling (21 CFR 801) - Recent products reviewed: <list> - Labeling accurate + non-misleading: yes/no - UDI compliance per 21 CFR 830: yes/no ## Form 483 / Warning Letter History - Form 483s last 3 years: N (each: closed/in-progress) - Warning Letters last 5 years: N (each: closed/in-progress) - Pattern across observations: <thematic> ## ISO 13485 Cross-Walk (post-Feb 2026 harmonization) - ISO 13485 audit findings: <link to cs-cqm-iso13485 output> - FDA-specific overlays remaining: labeling + complaint handling + MDR reporting + recall procedures - Cross-framework reuse: % of evidence shared ## Verdict 🟢 INSPECTION-READY | 🟡 GAPS-IDENTIFIED | 🔴 NOT-READY ## Top 3 Actions [3 concrete next steps with owner + FDA-cited timeline (15 days / 30 days / etc.)] ## Outside Counsel Required [For Warning Letter response, recall decisions, or 510(k) / PMA strategy disputes] ``` ## Routing - `/cs:compliance-readiness` — for multi-framework view - `/cs:iso13485-audit-prep` — for ISO 13485 cross-walk pair (substantially harmonized) - `/cs:gdpr-audit-prep` — if connected device handles personal data - `/cs:gc-review` — for Warning Letter response coordination ## Related - Agent: [`cs-fda-qsr-auditor`](../../agents/cs-fda-qsr-auditor.md) - Skill: [`fda-consultant-specialist`](../../../ra-qm-team/skills/fda-consultant-specialist/SKILL.md) - Adjacent: `../iso13485-audit-prep/`, `../compliance-readiness/` --- **Version:** 1.0.0
Tạo phiên cộng tác AgentHub mới với nhiệm vụ, số lượng agent và tiêu chí đánh giá.
---
name: "init"
description: "Create a new AgentHub collaboration session with task, agent count, and evaluation criteria."
command: /hub:init
---
# /hub:init — Create New Session
Initialize an AgentHub collaboration session. Creates the `.agenthub/` directory structure, generates a session ID, and configures evaluation criteria.
## Usage
```
/hub:init # Interactive mode
/hub:init --task "Optimize API" --agents 3 --eval "pytest bench.py" --metric p50_ms --direction lower
/hub:init --task "Refactor auth" --agents 2 # No eval (LLM judge mode)
```
## What It Does
### If arguments provided
Pass them to the init script:
```bash
python {skill_path}/scripts/hub_init.py \
--task "{task}" --agents {N} \
[--eval "{eval_cmd}"] [--metric {metric}] [--direction {direction}] \
[--base-branch {branch}]
```
### If no arguments (interactive mode)
Collect each parameter:
1. **Task** — What should the agents do? (required)
2. **Agent count** — How many parallel agents? (default: 3)
3. **Eval command** — Command to measure results (optional — skip for LLM judge mode)
4. **Metric name** — What metric to extract from eval output (required if eval command given)
5. **Direction** — Is lower or higher better? (required if metric given)
6. **Base branch** — Branch to fork from (default: current branch)
### Output
```
AgentHub session initialized
Session ID: 20260317-143022
Task: Optimize API response time below 100ms
Agents: 3
Eval: pytest bench.py --json
Metric: p50_ms (lower is better)
Base branch: dev
State: init
Next step: Run /hub:spawn to launch 3 agents
```
For content or research tasks (no eval command → LLM judge mode):
```
AgentHub session initialized
Session ID: 20260317-151200
Task: Draft 3 competing taglines for product launch
Agents: 3
Eval: LLM judge (no eval command)
Base branch: dev
State: init
Next step: Run /hub:spawn to launch 3 agents
```
## Baseline Capture
If `--eval` was provided, capture a baseline measurement after session creation:
1. Run the eval command in the current working directory
2. Extract the metric value from stdout
3. Append `baseline: {value}` to `.agenthub/sessions/{session-id}/config.yaml`
4. Display: `Baseline captured: {metric} = {value}`
This baseline is used by `result_ranker.py --baseline` during evaluation to show deltas. If the eval command fails at this stage, warn the user but continue — baseline is optional.
## After Init
Tell the user:
- Session created with ID `{session-id}`
- Baseline metric (if captured)
- Next step: `/hub:spawn` to launch agents
- Or `/hub:spawn {session-id}` if multiple sessions exist
Soạn bộ slide họp hội đồng quản trị và cập nhật cho nhà đầu tư, tổng hợp góc nhìn từ các vai trò C-suite.
--- name: "board-deck-builder" description: "Assembles comprehensive board and investor update decks by pulling perspectives from all C-suite roles. Use when preparing board meetings, investor updates, quarterly business reviews, or fundraising narratives. Covers structure, narrative framework, bad news delivery, and common mistakes." license: MIT metadata: version: 1.0.0 author: Alireza Rezvani category: c-level domain: board-governance updated: 2026-03-05 frameworks: deck-frameworks, board-deck-template --- # Board Deck Builder Build board decks that tell a story — not just show data. Every section has an owner, a narrative, and a "so what." ## Keywords board deck, investor update, board meeting, board pack, investor relations, quarterly review, board presentation, fundraising deck, investor deck, board narrative, QBR, quarterly business review ## Quick Start ``` /board-deck [quarterly|monthly|fundraising] [stage: seed|seriesA|seriesB] ``` Provide available metrics. The builder fills gaps with explicit placeholders — never invents numbers. ## Deck Structure (Standard Order) Every section follows: **Headline → Data → Narrative → Ask/Next** ### 1. Executive Summary (CEO) **3 sentences. No more.** - Sentence 1: State of the business (where we are) - Sentence 2: Biggest thing that happened this period - Sentence 3: Where we're going next quarter *Bad:* "We had a good quarter with lots of progress across all areas." *Good:* "We closed Q3 at $2.4M ARR (+22% QoQ), signed our largest enterprise contract, and enter Q4 with 14-month runway. The strategic shift to mid-market is working — ACV up 40% and sales cycle down 3 weeks. Q4 priority: close the $3M Series A and hit $2.8M ARR." ### 2. Key Metrics Dashboard (COO) **6-8 metrics max. Use a table.** | Metric | This Period | Last Period | Target | Status | |--------|-------------|-------------|--------|--------| | ARR | $2.4M | $1.97M | $2.3M | ✅ | | MoM growth | 8.1% | 7.2% | 7.5% | ✅ | | Burn multiple | 1.8x | 2.1x | <2x | ✅ | | NRR | 112% | 108% | >110% | ✅ | | CAC payback | 11 months | 14 months | <12 months | ✅ | | Headcount | 24 | 21 | 25 | 🟡 | Pick metrics the board actually tracks. Swap out anything they've said they don't care about. ### 3. Financial Update (CFO) - P&L summary: Revenue, COGS, Gross margin, OpEx, Net burn - Cash position and runway (months) - Burn multiple trend (3-quarter view) - Variance to plan (what was different and why) - Forecast update for next quarter **One sentence on each variance.** Boards hate "revenue was below target" with no explanation. Say why. ### 4. Revenue & Pipeline (CRO) - ARR waterfall: starting → new → expansion → churn → ending - NRR and logo churn rates - Pipeline by stage (in $, not just count) - Forecast: next quarter with confidence level - Top 3 deals: name/amount/close date/risk **The forecast must have a confidence level.** "We expect $2.8M" is weak. "High confidence $2.6M, upside to $2.9M if two late-stage deals close" is useful. ### 5. Product Update (CPO) - Shipped this quarter: 3-5 bullets, user impact for each - Shipping next quarter: 3-5 bullets with target dates - PMF signal: NPS trend, DAU/MAU ratio, feature adoption - One key learning from customer research **No feature lists.** Only features with evidence of user impact. ### 6. Growth & Marketing (CMO) - CAC by channel (table) - Pipeline contribution by channel ($) - Brand/awareness metrics relevant to stage (traffic, share of voice) - What's working, what's being cut, what's being tested ### 7. Engineering & Technical (CTO) - Delivery velocity trend (last 4 quarters) - Tech debt ratio and plan - Infrastructure: uptime, incidents, cost trend - Security posture (one line, flag anything pending) **Keep this short unless there's a material issue.** Boards don't need sprint details. ### 8. Team & People (CHRO) - Headcount: actual vs plan - Hiring: offers out, pipeline, time-to-fill trend - Attrition: regrettable vs non-regrettable - Engagement: last survey score, trend - Key hires this quarter, key open roles ### 9. Risk & Security (CISO) - Security posture: status of critical controls - Compliance: certifications in progress, deadlines - Incidents this quarter (if any): impact, resolution, prevention - Top 3 risks and mitigation status ### 10. Strategic Outlook (CEO) - Next quarter priorities: 3-5 items, ranked - Key decisions needed from the board - Asks: budget, introductions, advice, votes **The "asks" slide is the most important.** Be specific. "We'd like 3 warm introductions to CFOs at Series B companies" beats "any help would be appreciated." ### 11. Appendix - Detailed financial model - Full pipeline data - Cohort retention charts - Customer case studies - Detailed headcount breakdown --- ## Narrative Framework Boards see 10+ decks per quarter. Yours needs a through-line. **The 4-Act Structure:** 1. **Where we said we'd be** (last quarter's targets) 2. **Where we actually are** (honest assessment) 3. **Why the gap exists** (one cause per variance, not excuses) 4. **What we're doing about it** (specific, dated actions) This works for good news AND bad news. It's credible because it acknowledges reality. **Opening frame:** Start with the one thing that matters most — the board should know the key message by slide 3, not slide 30. --- ## Delivering Bad News Never bury it. Boards find out eventually. Finding out late makes it worse. **Framework:** 1. **State it plainly** — "We missed Q3 ARR target by $300K (12% gap)" 2. **Own the cause** — "Primary driver was longer-than-expected sales cycle in enterprise segment" 3. **Show you understand it** — "We analyzed 8 lost/stalled deals; the pattern is X" 4. **Present the fix** — "We've made 3 changes: [specific, dated changes]" 5. **Update the forecast** — "Revised Q4 target is $2.6M; here's the bottom-up build" **What NOT to do:** - Don't lead with good news to soften bad news — boards notice and distrust the framing - Don't explain without owning — "market conditions" is not a cause, it's a context - Don't present a fix without data behind it - Don't show a revised forecast without showing your assumptions --- ## Common Board Deck Mistakes | Mistake | Fix | |---------|-----| | Too many slides (>25) | Cut ruthlessly — if you can't explain it in the room, the slide is wrong | | Metrics without targets | Every metric needs a target and a status | | No narrative | Data without story forces boards to draw their own conclusions | | Burying bad news | Lead with it, own it, fix it | | Vague asks | Specific, actionable, person-assigned asks only | | No variance explanation | Every gap from target needs one-sentence cause | | Stale appendix | Appendix is only useful if it's current | | Designing for the reader, not the room | Decks are presented — they must work spoken aloud | --- ## Cadence Notes **Quarterly (standard):** Full deck, all sections, 20-30 slides. Sent 48 hours in advance. **Monthly (for early-stage):** Condensed — metrics dashboard, financials, pipeline, top risks. 8-12 slides. **Fundraising:** Opens with market/vision, closes with ask. See `references/deck-frameworks.md` for Sequoia format. ## References - `references/deck-frameworks.md` — SaaS board pack format, Sequoia structure, investor tailoring - `templates/board-deck-template.md` — fill-in template for complete board decks FILE:references/deck-frameworks.md # Board Deck Frameworks ## The SaaS Board Pack (Christoph Janz / Point Nine Style) Point Nine's board pack format became the de facto standard for early-stage SaaS. Core principle: **the numbers tell the story; the narrative explains the numbers.** ### Required Metrics (non-negotiable for SaaS boards) - **ARR** (not MRR — boards think annually) - **MoM / QoQ growth rate** - **NRR (Net Revenue Retention)** — the single most important SaaS metric - **Gross margin** — typically 60-80% SaaS; <60% is a flag - **CAC payback period** — months to recover customer acquisition cost - **Burn multiple** = net burn / net new ARR; <2x is good, >3x is a problem - **Runway** — months at current burn ### Point Nine Benchmark Targets (Series A SaaS) | Metric | Good | Great | Warning | |--------|------|-------|---------| | MoM growth | 10-15% | >20% | <7% | | NRR | >110% | >130% | <100% | | Gross margin | >65% | >75% | <60% | | CAC payback | <18 months | <12 months | >24 months | | Burn multiple | <2x | <1.5x | >3x | | Logo churn | <10%/yr | <5%/yr | >15%/yr | ### SaaS ARR Waterfall (Christoph Janz Format) Show this every quarter: ``` Starting ARR: $1,970,000 + New ARR: +$480,000 (new logos) + Expansion ARR: +$120,000 (upsells/cross-sells) - Churned ARR: -$90,000 (cancellations) - Contraction ARR: -$35,000 (downgrades) = Ending ARR: $2,445,000 ``` NRR = (Ending - New) / Starting = ($1,965K) / ($1,970K) = 99.7% ← flag this --- ## Sequoia Board Deck Structure Sequoia's canonical deck (used for both fundraising and board updates): 1. **Company Purpose** — one sentence, the existential "why" 2. **The Problem** — pain, size, who has it 3. **The Solution** — what you do, how it's different 4. **Why Now** — market timing, tailwinds, enabling factors 5. **Market Size** — TAM/SAM/SOM with methodology 6. **Business Model** — how you make money 7. **Traction** — proof it's working (growth, retention, logos) 8. **Team** — why you're the ones to win this 9. **Financials** — 3-year model, current metrics 10. **The Ask** — amount, use of funds, milestones to next round **For ongoing board updates:** Swap 1-5 (context) for "State of the Business" and "Last Quarter vs Plan." Boards know the company — skip the pitch. --- ## Investor-Specific Tailoring ### What Different Investor Types Care About **Early-stage VCs (Seed, A):** - Growth rate above all else - NRR — "does the product retain?" - Founder-market fit narrative - Milestone achievement vs last board meeting **Growth-stage VCs (B, C):** - Capital efficiency (burn multiple, CAC payback) - GTM repeatability — can you hire 10 AEs and have it work? - Market leadership signals - Path to profitability (even if years away) **Strategic investors:** - Synergies with their portfolio/business - Technology differentiation - Partnership potential **Angels:** - Team above all - Personal conviction in the thesis - Exit scenarios ### Tailoring the Narrative - If you're ahead of plan: "Here's why, and here's how we'll sustain it" - If you're behind plan: "Here's why, here's what we've learned, here's the new plan" - If the plan was wrong: "The assumption that was wrong, what we know now, updated thesis" Never pretend the plan was right when it wasn't. Board members have memories and models. --- ## How to Present Bad News Boards have seen everything. What loses credibility isn't bad results — it's bad framing. ### The Credibility Formula 1. **Lead with the headline** — "We missed ARR target by 18%" 2. **Quantify the gap** — absolute and percentage 3. **Diagnose the cause** (one primary, max two secondary) 4. **Show your work** — "We analyzed 12 churned/stalled deals and found..." 5. **Present the fix** — specific, dated, owned by a name 6. **Update the forecast** — bottom-up rebuild, not wishful thinking 7. **Flag the risk** — "If X doesn't close, here's the contingency" ### What "Showing Your Work" Looks Like Bad: "Sales cycle was longer than expected." Good: "Sales cycle stretched from 45 to 72 days. Root cause: new legal review requirement at enterprise accounts, triggered by our SOC 2 Type II gap. Fix: SOC 2 audit underway (target: Dec 15), and we've pre-built contract language to accelerate review. Impact: estimated 3 stalled deals ($420K ARR) unblock in Q4." ### Scenarios and How to Handle Each | Scenario | Frame | |----------|-------| | Missed revenue target | Lead with it; diagnose cause; bottom-up revised forecast | | Key customer churned | Announce it; explain why; show retention analysis of remaining accounts | | Key exec left | Announce it; show succession/coverage plan; don't overpromise the replacement timeline | | Burn accelerated | Show P&L detail; explain what drove it; adjust runway projection; plan to fix | | Market headwinds | Acknowledge; show relative performance vs peers; pivot if needed | | Fundraise delayed | Runway impact; bridge options; revised timeline | --- ## Appendix Data That Boards Actually Use Boards use the appendix for due diligence, not during the meeting. Include: **Financial:** - Full P&L (monthly for last 4 quarters) - Cash flow statement - 3-year model with assumptions - Unit economics by cohort **Revenue:** - Customer list by ARR (anonymized or full, per board agreement) - Pipeline detail by deal - Cohort analysis (NRR by cohort vintage) - Churn analysis: when, why, segment **Product:** - Feature adoption rates - NPS score distribution and trend - DAU/MAU by segment **Team:** - Org chart - Full headcount list with fully loaded costs - Open reqs with priority ranking **One rule:** If the appendix is more than 20 slides, you have too much. Boards won't read it. --- ## Quarterly vs Monthly Board Meetings ### Quarterly (Series A+) - Full board pack, all sections - 2 hours: 30 min pre-read, 90 min discussion - Voting items at end - Sent 48 hours before (72 hours preferred) - Add 1-2 "deep dive" topics beyond standard update ### Monthly (Seed / High-Growth A) - Metrics dashboard + financials + top risks only - 45-60 minutes - Informal tone, more conversational - Sent 24 hours before - Skip slides for items where nothing changed ### When to Increase Frequency - Approaching 6-month runway - Major strategic pivot - Fundraise in progress - Significant underperformance vs plan - M&A discussions --- ## Meeting Logistics (Often Overlooked) - **Pre-read requirement:** Board packs should be read before the meeting. If you're presenting slides, you're wasting time. - **Discussion format:** "I'll be brief on X since you've read it. Want to spend time on Y?" — respect board members' time - **One note-taker:** CEO's EA or COO; not the CEO (they need to be present) - **Follow-up within 24 hours:** Action items, voting outcomes, next meeting date - **Board portal vs email:** Use a board portal (Carta, Boardable, Notion) for version control and D&O protection FILE:templates/board-deck-template.md # Board Deck Template Fill in bracketed fields. Remove placeholders before sharing. Never invent numbers — use `[TBD]` if unknown. --- ## Slide 1: Executive Summary (CEO) **[Company Name] — Q[X] [Year] Board Update** > [One sentence: State of the business — where you are.] > [One sentence: The most important thing that happened this quarter.] > [One sentence: Where you're going next quarter and what determines success.] --- ## Slide 2: Key Metrics Dashboard (COO) **Quarter at a Glance** | Metric | Q[X] Actual | Q[X] Target | Q[X-1] Actual | Status | |--------|-------------|-------------|---------------|--------| | ARR | $[X]M | $[X]M | $[X]M | [✅/🟡/🔴] | | QoQ Growth | [X]% | [X]% | [X]% | [✅/🟡/🔴] | | NRR | [X]% | >[X]% | [X]% | [✅/🟡/🔴] | | Gross Margin | [X]% | >[X]% | [X]% | [✅/🟡/🔴] | | Burn Multiple | [X]x | <[X]x | [X]x | [✅/🟡/🔴] | | Runway | [X] months | >[X] months | [X] months | [✅/🟡/🔴] | | Headcount | [X] | [X] | [X] | [✅/🟡/🔴] | | CAC Payback | [X] months | <[X] months | [X] months | [✅/🟡/🔴] | --- ## Slide 3: Financial Update (CFO) **P&L Summary** | | Q[X] | Q[X-1] | QoQ | |--|------|--------|-----| | Revenue | $[X]K | $[X]K | [+/-X]% | | COGS | $[X]K | $[X]K | | | Gross Profit | $[X]K | $[X]K | | | Gross Margin | [X]% | [X]% | | | OpEx | $[X]K | $[X]K | | | Net Burn | $[X]K | $[X]K | | **Cash & Runway** - Cash on hand: $[X]M - Monthly burn: $[X]K - Runway: [X] months - Burn multiple: [X]x (target: <2x) **Variance to Plan** - Revenue: [+/-$X]K vs plan — [one sentence cause] - Burn: [+/-$X]K vs plan — [one sentence cause] **Q[X+1] Forecast:** $[X]M revenue, $[X]K burn — [confidence: high/medium/low] --- ## Slide 4: Revenue & Pipeline (CRO) **ARR Waterfall** ``` Starting ARR: $[X]M + New ARR: +$[X]K + Expansion ARR: +$[X]K - Churned ARR: -$[X]K - Contraction ARR: -$[X]K = Ending ARR: $[X]M ``` **Health Metrics** - NRR: [X]% | Logo churn: [X]% | Avg ACV: $[X]K **Pipeline (next 90 days)** | Stage | # Deals | $ Value | |-------|---------|---------| | Proposal | [X] | $[X]K | | Negotiation | [X] | $[X]K | | Verbal commit | [X] | $[X]K | **Q[X+1] Forecast:** $[X]M ARR — [one sentence confidence statement] **Top 3 Deals** 1. [Company] — $[X]K ARR — close date [X] — risk: [one word] 2. [Company] — $[X]K ARR — close date [X] — risk: [one word] 3. [Company] — $[X]K ARR — close date [X] — risk: [one word] --- ## Slide 5: Product Update (CPO) **Shipped This Quarter** - [Feature/initiative] — impact: [metric or user outcome] - [Feature/initiative] — impact: [metric or user outcome] - [Feature/initiative] — impact: [metric or user outcome] **Shipping Next Quarter** - [Feature] — target: [date] — why it matters: [one line] - [Feature] — target: [date] — why it matters: [one line] - [Feature] — target: [date] — why it matters: [one line] **PMF Signals** - NPS: [X] (trend: [up/flat/down]) - DAU/MAU: [X]% - Feature adoption ([key feature]): [X]% **Key Learning:** [One thing customer research taught you this quarter] --- ## Slide 6: Growth & Marketing (CMO) **CAC by Channel** | Channel | CAC | Pipeline $ | % of Total | |---------|-----|-----------|------------| | Outbound | $[X]K | $[X]K | [X]% | | Inbound | $[X]K | $[X]K | [X]% | | Partner | $[X]K | $[X]K | [X]% | **What's Working:** [One channel or initiative with data] **What We Cut:** [One thing, and why] **What We're Testing:** [One experiment running now] --- ## Slide 7: Engineering & Technical (CTO) **Delivery** - Velocity trend: [up/flat/down vs last quarter] - Q[X] commitments delivered: [X]% on time **Quality & Reliability** - P0/P1 incidents: [X] (vs [X] last quarter) - Uptime: [X]% - Infrastructure cost: $[X]K/month (trend: [up/flat/down]) **Tech Debt** - Ratio: [X]% of roadmap allocated to debt reduction - Key item in progress: [description, target date] **Security:** [one line status; flag anything pending] --- ## Slide 8: Team & People (CHRO) **Headcount** - Total: [X] (vs [X] plan, [X] last quarter) - By function: Eng [X], Product [X], Sales [X], CS [X], G&A [X] **Hiring** - Hired this quarter: [X] - Open reqs: [X] — time-to-fill avg: [X] days - Offers outstanding: [X] **Retention** - Regrettable attrition: [X]% (annualized) - Engagement score: [X]/10 (trend: [up/flat/down]) **Notable Hires:** [Name, role — one sentence on why they matter] **Key Open Roles:** [Role, priority: critical/high/medium] --- ## Slide 9: Risk & Security (CISO) **Compliance Status** | Certification | Status | Target Date | |--------------|--------|-------------| | [SOC 2 / ISO 27001 / etc.] | [In progress / Complete / Not started] | [Date] | **Security Posture:** [One line — overall status] **Incidents This Quarter:** [X] total — [description if >0] **Top Risks** 1. [Risk] — likelihood: [H/M/L] — impact: [H/M/L] — mitigation: [one line] 2. [Risk] — likelihood: [H/M/L] — impact: [H/M/L] — mitigation: [one line] 3. [Risk] — likelihood: [H/M/L] — impact: [H/M/L] — mitigation: [one line] --- ## Slide 10: Strategic Outlook (CEO) **Q[X+1] Priorities** 1. [Priority] — owner: [name] — success metric: [specific] 2. [Priority] — owner: [name] — success metric: [specific] 3. [Priority] — owner: [name] — success metric: [specific] **Asks from the Board** - [Specific ask: warm intro / advice / vote / resource] - [Specific ask] - [Specific ask] **Decisions Needed Today** - [Decision with options]: [Option A] vs [Option B] — recommendation: [A/B] — rationale: [one line] --- ## Appendix - A1: Full P&L (monthly, last 4 quarters) - A2: 3-year financial model - A3: Customer list / ARR breakdown - A4: Full pipeline by deal - A5: Cohort retention analysis - A6: Org chart + headcount detail - A7: [Other as relevant]
Tạo unit test, integration test, E2E test cho React/Next.js với Jest, Testing Library, Playwright, MSW và phân tích độ phủ.
---
name: "senior-qa"
description: Generates unit tests, integration tests, and E2E tests for React/Next.js applications. Scans components to create Jest + React Testing Library test stubs, analyzes Istanbul/LCOV coverage reports to surface gaps, scaffolds Playwright test files from Next.js routes, mocks API calls with MSW, creates test fixtures, and configures test runners. Use when the user asks to "generate tests", "write unit tests", "analyze test coverage", "scaffold E2E tests", "set up Playwright", "configure Jest", "implement testing patterns", or "improve test quality".
---
# Senior QA Engineer
Test automation, coverage analysis, and quality assurance patterns for React and Next.js applications.
---
## Quick Start
```bash
# Generate Jest test stubs for React components
python scripts/test_suite_generator.py src/components/ --output __tests__/
# Analyze test coverage from Jest/Istanbul reports
python scripts/coverage_analyzer.py coverage/coverage-final.json --threshold 80
# Scaffold Playwright E2E tests for Next.js routes
python scripts/e2e_test_scaffolder.py src/app/ --output e2e/
```
---
## Tools Overview
### 1. Test Suite Generator
Scans React/TypeScript components and generates Jest + React Testing Library test stubs with proper structure.
**Input:** Source directory containing React components
**Output:** Test files with describe blocks, render tests, interaction tests
**Usage:**
```bash
# Basic usage - scan components and generate tests
python scripts/test_suite_generator.py src/components/ --output __tests__/
# Include accessibility tests
python scripts/test_suite_generator.py src/ --output __tests__/ --include-a11y
# Generate with custom template
python scripts/test_suite_generator.py src/ --template custom-template.tsx
```
**Supported Patterns:**
- Functional components with hooks
- Components with Context providers
- Components with data fetching
- Form components with validation
---
### 2. Coverage Analyzer
Parses Jest/Istanbul coverage reports and identifies gaps, uncovered branches, and provides actionable recommendations.
**Input:** Coverage report (JSON or LCOV format)
**Output:** Coverage analysis with recommendations
**Usage:**
```bash
# Analyze coverage report
python scripts/coverage_analyzer.py coverage/coverage-final.json
# Enforce threshold (exit 1 if below)
python scripts/coverage_analyzer.py coverage/ --threshold 80 --strict
# Generate HTML report
python scripts/coverage_analyzer.py coverage/ --format html --output report.html
```
---
### 3. E2E Test Scaffolder
Scans Next.js pages/app directory and generates Playwright test files with common interactions.
**Input:** Next.js pages or app directory
**Output:** Playwright test files organized by route
**Usage:**
```bash
# Scaffold E2E tests for Next.js App Router
python scripts/e2e_test_scaffolder.py src/app/ --output e2e/
# Include Page Object Model classes
python scripts/e2e_test_scaffolder.py src/app/ --output e2e/ --include-pom
# Generate for specific routes
python scripts/e2e_test_scaffolder.py src/app/ --routes "/login,/dashboard,/checkout"
```
---
## QA Workflows
### Unit Test Generation Workflow
Use when setting up tests for new or existing React components.
**Step 1: Scan project for untested components**
```bash
python scripts/test_suite_generator.py src/components/ --scan-only
```
**Step 2: Generate test stubs**
```bash
python scripts/test_suite_generator.py src/components/ --output __tests__/
```
**Step 3: Review and customize generated tests**
```typescript
// __tests__/Button.test.tsx (generated)
import { render, screen, fireEvent } from '@testing-library/react';
import { Button } from '../src/components/Button';
describe('Button', () => {
it('renders with label', () => {
render(<Button>Click me</Button>);
expect(screen.getByRole('button', { name: "click-mei-tobeinthedocument"
});
it('calls onClick when clicked', () => {
const handleClick = jest.fn();
render(<Button onClick={handleClick}>Click</Button>);
fireEvent.click(screen.getByRole('button'));
expect(handleClick).toHaveBeenCalledTimes(1);
});
// TODO: Add your specific test cases
});
```
**Step 4: Run tests and check coverage**
```bash
npm test -- --coverage
python scripts/coverage_analyzer.py coverage/coverage-final.json
```
---
### Coverage Analysis Workflow
Use when improving test coverage or preparing for release.
**Step 1: Generate coverage report**
```bash
npm test -- --coverage --coverageReporters=json
```
**Step 2: Analyze coverage gaps**
```bash
python scripts/coverage_analyzer.py coverage/coverage-final.json --threshold 80
```
**Step 3: Identify critical paths**
```bash
python scripts/coverage_analyzer.py coverage/ --critical-paths
```
**Step 4: Generate missing test stubs**
```bash
python scripts/test_suite_generator.py src/ --uncovered-only --output __tests__/
```
**Step 5: Verify improvement**
```bash
npm test -- --coverage
python scripts/coverage_analyzer.py coverage/ --compare previous-coverage.json
```
---
### E2E Test Setup Workflow
Use when setting up Playwright for a Next.js project.
**Step 1: Initialize Playwright (if not installed)**
```bash
npm init playwright@latest
```
**Step 2: Scaffold E2E tests from routes**
```bash
python scripts/e2e_test_scaffolder.py src/app/ --output e2e/
```
**Step 3: Configure authentication fixtures**
```typescript
// e2e/fixtures/auth.ts (generated)
import { test as base } from '@playwright/test';
export const test = base.extend({
authenticatedPage: async ({ page }, use) => {
await page.goto('/login');
await page.fill('[name="email"]', 'test@example.com');
await page.fill('[name="password"]', 'password');
await page.click('button[type="submit"]');
await page.waitForURL('/dashboard');
await use(page);
},
});
```
**Step 4: Run E2E tests**
```bash
npx playwright test
npx playwright show-report
```
**Step 5: Add to CI pipeline**
```yaml
# .github/workflows/e2e.yml
- name: "run-e2e-tests"
run: npx playwright test
- name: "upload-report"
uses: actions/upload-artifact@v3
with:
name: "playwright-report"
path: playwright-report/
```
---
## Reference Documentation
| File | Contains | Use When |
|------|----------|----------|
| `references/testing_strategies.md` | Test pyramid, testing types, coverage targets, CI/CD integration | Designing test strategy |
| `references/test_automation_patterns.md` | Page Object Model, mocking (MSW), fixtures, async patterns | Writing test code |
| `references/qa_best_practices.md` | Testable code, flaky tests, debugging, quality metrics | Improving test quality |
---
## Common Patterns Quick Reference
### React Testing Library Queries
```typescript
// Preferred (accessible)
screen.getByRole('button', { name: "submiti"
screen.getByLabelText(/email/i)
screen.getByPlaceholderText(/search/i)
// Fallback
screen.getByTestId('custom-element')
```
### Async Testing
```typescript
// Wait for element
await screen.findByText(/loaded/i);
// Wait for removal
await waitForElementToBeRemoved(() => screen.queryByText(/loading/i));
// Wait for condition
await waitFor(() => {
expect(mockFn).toHaveBeenCalled();
});
```
### Mocking with MSW
```typescript
import { rest } from 'msw';
import { setupServer } from 'msw/node';
const server = setupServer(
rest.get('/api/users', (req, res, ctx) => {
return res(ctx.json([{ id: 1, name: "john" }]));
})
);
beforeAll(() => server.listen());
afterEach(() => server.resetHandlers());
afterAll(() => server.close());
```
### Playwright Locators
```typescript
// Preferred
page.getByRole('button', { name: "submit" })
page.getByLabel('Email')
page.getByText('Welcome')
// Chaining
page.getByRole('listitem').filter({ hasText: 'Product' })
```
### Coverage Thresholds (jest.config.js)
```javascript
module.exports = {
coverageThreshold: {
global: {
branches: 80,
functions: 80,
lines: 80,
statements: 80,
},
},
};
```
---
## Common Commands
```bash
# Jest
npm test # Run all tests
npm test -- --watch # Watch mode
npm test -- --coverage # With coverage
npm test -- Button.test.tsx # Single file
# Playwright
npx playwright test # Run all E2E tests
npx playwright test --ui # UI mode
npx playwright test --debug # Debug mode
npx playwright codegen # Generate tests
# Coverage
npm test -- --coverage --coverageReporters=lcov,json
python scripts/coverage_analyzer.py coverage/coverage-final.json
```
FILE:README.md
# Senior QA Testing Engineer Skill
Production-ready quality assurance and test automation skill for React/Next.js applications.
## Tech Stack Focus
| Category | Technologies |
|----------|--------------|
| Unit/Integration | Jest, React Testing Library |
| E2E Testing | Playwright |
| Coverage Analysis | Istanbul, NYC, LCOV |
| API Mocking | MSW (Mock Service Worker) |
| Accessibility | jest-axe, @axe-core/playwright |
## Quick Start
```bash
# Generate component tests
python scripts/test_suite_generator.py src/components --include-a11y
# Analyze coverage gaps
python scripts/coverage_analyzer.py coverage/coverage-final.json --threshold 80 --strict
# Scaffold E2E tests for Next.js
python scripts/e2e_test_scaffolder.py src/app --page-objects
```
## Scripts
### test_suite_generator.py
Scans React/TypeScript components and generates Jest + React Testing Library test stubs.
**Features:**
- Detects functional, class, memo, and forwardRef components
- Generates render, interaction, and accessibility tests
- Identifies props requiring mock data
- Optional `--include-a11y` for jest-axe assertions
**Usage:**
```bash
python scripts/test_suite_generator.py <component-dir> [options]
Options:
--scan-only List components without generating tests
--include-a11y Add accessibility test assertions
--output DIR Output directory for test files
```
### coverage_analyzer.py
Parses Istanbul JSON or LCOV coverage reports and identifies testing gaps.
**Features:**
- Calculates line, branch, function, and statement coverage
- Identifies critical untested paths (auth, payment, API routes)
- Generates text and HTML reports
- Threshold enforcement with `--strict` flag
**Usage:**
```bash
python scripts/coverage_analyzer.py <coverage-file> [options]
Options:
--threshold N Minimum coverage percentage (default: 80)
--strict Exit with error if below threshold
--format FORMAT Output format: text, json, html
--output FILE Output file path
```
### e2e_test_scaffolder.py
Scans Next.js App Router or Pages Router directories and generates Playwright tests.
**Features:**
- Detects routes, dynamic parameters, and layouts
- Generates test files per route with navigation and content checks
- Optional Page Object Model class generation
- Generates `playwright.config.ts` and auth fixtures
**Usage:**
```bash
python scripts/e2e_test_scaffolder.py <app-dir> [options]
Options:
--page-objects Generate Page Object Model classes
--output DIR Output directory for E2E tests
--base-url URL Base URL for tests (default: http://localhost:3000)
```
## References
### testing_strategies.md (650 lines)
Comprehensive testing strategy guide covering:
- Test pyramid and distribution (70% unit, 20% integration, 10% E2E)
- Coverage targets by project type
- Testing types (unit, integration, E2E, visual, accessibility)
- CI/CD integration patterns
- Testing decision framework
### test_automation_patterns.md (1010 lines)
React/Next.js test automation patterns:
- Page Object Model implementation for Playwright
- Test data factories and builder patterns
- Fixture management (Playwright and Jest)
- Mocking strategies (MSW, Jest module mocking)
- Custom test utilities (`renderWithProviders`)
- Async testing patterns
- Snapshot testing guidelines
### qa_best_practices.md (965 lines)
Quality assurance best practices:
- Writing testable React code
- Test naming conventions (Describe-It pattern)
- Arrange-Act-Assert structure
- Test isolation principles
- Handling flaky tests
- Debugging failed tests
- Quality metrics and KPIs
## Workflows
### Workflow 1: New Component Testing
1. Create component in `src/components/`
2. Run `test_suite_generator.py` to generate test stub
3. Fill in test assertions based on component behavior
4. Run `npm test` to verify tests pass
5. Check coverage with `coverage_analyzer.py`
### Workflow 2: E2E Test Setup
1. Run `e2e_test_scaffolder.py` on your Next.js app directory
2. Review generated tests in `e2e/` directory
3. Customize Page Objects for complex interactions
4. Run `npx playwright test` to execute
5. Configure CI/CD with generated `playwright.config.ts`
### Workflow 3: Coverage Gap Analysis
1. Run tests with coverage: `npm test -- --coverage`
2. Analyze with `coverage_analyzer.py --strict --threshold 80`
3. Review critical untested paths in report
4. Prioritize tests for auth, payment, and API routes
5. Re-run analysis to verify improvement
## Test Pyramid Targets
| Test Type | Ratio | Focus |
|-----------|-------|-------|
| Unit | 70% | Individual functions, utilities, hooks |
| Integration | 20% | Component interactions, API calls, state |
| E2E | 10% | Critical user journeys, happy paths |
## Coverage Targets
| Project Type | Line | Branch | Function |
|--------------|------|--------|----------|
| Startup/MVP | 60% | 50% | 70% |
| Production | 80% | 70% | 85% |
| Enterprise | 90% | 85% | 95% |
## CI/CD Integration
```yaml
# .github/workflows/test.yml
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install dependencies
run: npm ci
- name: Run unit tests
run: npm test -- --coverage
- name: Run E2E tests
run: npx playwright test
- name: Upload coverage
uses: codecov/codecov-action@v4
```
## Related Skills
- **senior-frontend** - React/Next.js component development
- **senior-fullstack** - Full application architecture
- **senior-devops** - CI/CD pipeline setup
- **code-reviewer** - Code review with testing focus
---
**Version:** 2.9.0
**Last Updated:** January 2026
**Tech Focus:** React 18+, Next.js 14+, Jest 29+, Playwright 1.40+
FILE:references/qa_best_practices.md
# QA Best Practices for React and Next.js
Guidelines for writing maintainable tests, debugging failures, and measuring test quality.
---
## Table of Contents
- [Writing Testable Code](#writing-testable-code)
- [Test Naming Conventions](#test-naming-conventions)
- [Arrange-Act-Assert Pattern](#arrange-act-assert-pattern)
- [Test Isolation Principles](#test-isolation-principles)
- [Handling Flaky Tests](#handling-flaky-tests)
- [Code Review for Testability](#code-review-for-testability)
- [Test Maintenance Strategies](#test-maintenance-strategies)
- [Debugging Failed Tests](#debugging-failed-tests)
- [Quality Metrics and KPIs](#quality-metrics-and-kpis)
---
## Writing Testable Code
Testable code is easy to understand, has clear boundaries, and minimizes dependencies.
### Dependency Injection
Instead of creating dependencies inside functions, pass them as parameters.
**Hard to Test:**
```typescript
// src/services/userService.ts
import { prisma } from '../lib/prisma';
import { sendEmail } from '../lib/email';
export async function createUser(data: UserInput) {
const user = await prisma.user.create({ data });
await sendEmail(user.email, 'Welcome!');
return user;
}
```
**Easy to Test:**
```typescript
// src/services/userService.ts
export function createUserService(
db: PrismaClient,
emailService: EmailService
) {
return {
async createUser(data: UserInput) {
const user = await db.user.create({ data });
await emailService.send(user.email, 'Welcome!');
return user;
},
};
}
// Usage in app
const userService = createUserService(prisma, emailService);
// Usage in tests
const mockDb = { user: { create: jest.fn() } };
const mockEmail = { send: jest.fn() };
const testService = createUserService(mockDb, mockEmail);
```
### Pure Functions
Pure functions are deterministic and have no side effects, making them trivial to test.
**Impure (Hard to Test):**
```typescript
function formatTimestamp() {
const now = new Date();
return `now.getFullYear()-now.getMonth() + 1-now.getDate()`;
}
```
**Pure (Easy to Test):**
```typescript
function formatTimestamp(date: Date): string {
return `date.getFullYear()-date.getMonth() + 1-date.getDate()`;
}
// Test
expect(formatTimestamp(new Date('2024-03-15'))).toBe('2024-3-15');
```
### Separation of Concerns
Separate business logic from UI and I/O operations.
**Mixed Concerns (Hard to Test):**
```typescript
// Component with embedded business logic
function CheckoutForm() {
const [total, setTotal] = useState(0);
const handleSubmit = async (items: CartItem[]) => {
// Business logic mixed with UI
let sum = 0;
for (const item of items) {
sum += item.price * item.quantity;
if (item.category === 'electronics') {
sum *= 0.9; // 10% discount
}
}
const tax = sum * 0.08;
const finalTotal = sum + tax;
// API call
await fetch('/api/orders', {
method: 'POST',
body: JSON.stringify({ items, total: finalTotal }),
});
setTotal(finalTotal);
};
return <form onSubmit={handleSubmit}>...</form>;
}
```
**Separated Concerns (Easy to Test):**
```typescript
// Pure business logic (easy to unit test)
export function calculateOrderTotal(items: CartItem[]): number {
return items.reduce((sum, item) => {
const subtotal = item.price * item.quantity;
const discount = item.category === 'electronics' ? 0.9 : 1;
return sum + subtotal * discount;
}, 0);
}
export function calculateTax(subtotal: number, rate = 0.08): number {
return subtotal * rate;
}
// Custom hook for order logic (testable with renderHook)
export function useCheckout() {
const [total, setTotal] = useState(0);
const mutation = useMutation(createOrder);
const checkout = async (items: CartItem[]) => {
const subtotal = calculateOrderTotal(items);
const tax = calculateTax(subtotal);
const finalTotal = subtotal + tax;
await mutation.mutateAsync({ items, total: finalTotal });
setTotal(finalTotal);
};
return { checkout, total, isLoading: mutation.isLoading };
}
// Component (integration testable)
function CheckoutForm() {
const { checkout, total, isLoading } = useCheckout();
return <form onSubmit={() => checkout(items)}>...</form>;
}
```
### Component Design for Testability
| Pattern | Testability | Example |
|---------|-------------|---------|
| Props over context | High | `<Button disabled={!valid}>` |
| Callbacks over side effects | High | `onSubmit={handleSubmit}` |
| Controlled components | High | `<Input value={value} onChange={...}>` |
| Render props | Medium | `<DataProvider render={data => ...}>` |
| Internal state | Low | `const [x, setX] = useState()` |
| Global state | Low | `useGlobalStore()` |
---
## Test Naming Conventions
Good test names document expected behavior and help diagnose failures.
### Naming Patterns
**Pattern 1: should [expected behavior] when [condition]**
```typescript
describe('LoginForm', () => {
it('should display error message when credentials are invalid', () => {});
it('should redirect to dashboard when login succeeds', () => {});
it('should disable submit button when form is submitting', () => {});
});
```
**Pattern 2: [method/action] [expected result]**
```typescript
describe('calculateDiscount', () => {
it('returns 0 for orders under $50', () => {});
it('returns 10% for orders $50-$99', () => {});
it('returns 20% for orders $100+', () => {});
});
```
**Pattern 3: given [context], when [action], then [result]**
```typescript
describe('ShoppingCart', () => {
it('given an empty cart, when adding an item, then cart count is 1', () => {});
it('given items in cart, when removing all, then cart is empty', () => {});
});
```
### Describe Block Organization
```typescript
describe('UserService', () => {
describe('createUser', () => {
describe('with valid input', () => {
it('creates user in database', () => {});
it('sends welcome email', () => {});
it('returns user with id', () => {});
});
describe('with invalid input', () => {
it('throws ValidationError for missing email', () => {});
it('throws ValidationError for invalid email format', () => {});
it('throws ConflictError for duplicate email', () => {});
});
});
describe('deleteUser', () => {
it('removes user from database', () => {});
it('throws NotFoundError for non-existent user', () => {});
});
});
```
### Anti-patterns to Avoid
| Bad | Good | Why |
|-----|------|-----|
| `it('works')` | `it('returns sum of two numbers')` | Describes behavior |
| `it('test 1')` | `it('handles empty array')` | Specific scenario |
| `it('should do stuff')` | `it('should validate email format')` | Clear expectation |
| Duplicating code in name | Describing behavior | Readable output |
---
## Arrange-Act-Assert Pattern
The AAA pattern structures tests into three clear phases.
### Structure
```typescript
it('calculates total with discount', () => {
// Arrange - Set up test data and conditions
const items = [
{ name: 'Widget', price: 100, quantity: 2 },
{ name: 'Gadget', price: 50, quantity: 1 },
];
const discountRate = 0.1;
// Act - Execute the code being tested
const result = calculateTotal(items, discountRate);
// Assert - Verify the outcome
expect(result).toBe(225); // (200 + 50) * 0.9
});
```
### Async Example
```typescript
it('fetches user profile', async () => {
// Arrange
const userId = '123';
server.use(
rest.get('/api/users/:id', (req, res, ctx) =>
res(ctx.json({ id: userId, name: 'John' }))
)
);
// Act
render(<UserProfile userId={userId} />);
// Assert
await expect(screen.findByText('John')).resolves.toBeInTheDocument();
});
```
### Component Testing Example
```typescript
it('submits form with user input', async () => {
// Arrange
const user = userEvent.setup();
const onSubmit = jest.fn();
render(<ContactForm onSubmit={onSubmit} />);
// Act
await user.type(screen.getByLabelText('Name'), 'John Doe');
await user.type(screen.getByLabelText('Email'), 'john@example.com');
await user.type(screen.getByLabelText('Message'), 'Hello!');
await user.click(screen.getByRole('button', { name: 'Send' }));
// Assert
expect(onSubmit).toHaveBeenCalledWith({
name: 'John Doe',
email: 'john@example.com',
message: 'Hello!',
});
});
```
### Guidelines
1. **One Act per test** - Test one behavior at a time
2. **Multiple assertions OK** - If they verify the same behavior
3. **Avoid logic in tests** - No if/else, loops in test code
4. **Setup in Arrange, not beforeEach** - Unless truly shared
---
## Test Isolation Principles
Isolated tests are independent, repeatable, and can run in any order.
### State Isolation
```typescript
describe('CartService', () => {
let cartService: CartService;
// Fresh instance for each test
beforeEach(() => {
cartService = new CartService();
});
it('adds item to empty cart', () => {
cartService.addItem({ id: '1', quantity: 1 });
expect(cartService.getItems()).toHaveLength(1);
});
it('starts with empty cart', () => {
// Not affected by previous test
expect(cartService.getItems()).toHaveLength(0);
});
});
```
### Database Isolation
```typescript
describe('UserRepository', () => {
beforeAll(async () => {
// Connect to test database
await db.connect(process.env.TEST_DATABASE_URL);
});
beforeEach(async () => {
// Clean database before each test
await db.query('TRUNCATE users CASCADE');
});
afterAll(async () => {
await db.disconnect();
});
it('creates user', async () => {
const user = await userRepo.create({ email: 'test@example.com' });
expect(user.id).toBeDefined();
});
});
```
### API Mocking Isolation
```typescript
describe('ProductList', () => {
// Reset handlers after each test
afterEach(() => server.resetHandlers());
it('shows products from API', async () => {
// Default handler returns products
render(<ProductList />);
await expect(screen.findByText('Widget')).resolves.toBeInTheDocument();
});
it('shows error on API failure', async () => {
// Override handler for this test only
server.use(
rest.get('/api/products', (req, res, ctx) =>
res(ctx.status(500))
)
);
render(<ProductList />);
await expect(screen.findByText('Error')).resolves.toBeInTheDocument();
});
it('shows products again', async () => {
// Back to default handler (server.resetHandlers ran)
render(<ProductList />);
await expect(screen.findByText('Widget')).resolves.toBeInTheDocument();
});
});
```
### Isolation Checklist
| Aspect | Solution |
|--------|----------|
| Global state | Reset in beforeEach |
| Timers | jest.useFakeTimers() + jest.useRealTimers() |
| DOM | RTL's cleanup (automatic) |
| Database | Truncate tables or use transactions |
| API mocks | server.resetHandlers() |
| File system | Use temp directories, clean up in afterEach |
| Environment vars | Restore in afterEach |
---
## Handling Flaky Tests
Flaky tests pass and fail intermittently without code changes.
### Common Causes and Fixes
**1. Timing Issues**
```typescript
// Flaky - race condition
it('shows loading then data', () => {
render(<UserProfile />);
expect(screen.getByText('Loading')).toBeInTheDocument();
expect(screen.getByText('John')).toBeInTheDocument(); // May fail
});
// Fixed - proper async handling
it('shows loading then data', async () => {
render(<UserProfile />);
expect(screen.getByText('Loading')).toBeInTheDocument();
await waitFor(() => {
expect(screen.getByText('John')).toBeInTheDocument();
});
});
```
**2. Non-deterministic Data**
```typescript
// Flaky - random data
it('sorts users alphabetically', () => {
const users = [createUser(), createUser(), createUser()];
// Names are random, order unpredictable
});
// Fixed - deterministic data
it('sorts users alphabetically', () => {
const users = [
createUser({ name: 'Charlie' }),
createUser({ name: 'Alice' }),
createUser({ name: 'Bob' }),
];
const sorted = sortUsers(users);
expect(sorted.map(u => u.name)).toEqual(['Alice', 'Bob', 'Charlie']);
});
```
**3. Test Order Dependencies**
```typescript
// Flaky - relies on previous test
describe('Counter', () => {
const counter = new Counter(); // Shared instance!
it('increments', () => {
counter.increment();
expect(counter.value).toBe(1);
});
it('starts at zero', () => {
expect(counter.value).toBe(0); // Fails! Value is 1
});
});
// Fixed - fresh instance per test
describe('Counter', () => {
let counter: Counter;
beforeEach(() => {
counter = new Counter();
});
it('increments', () => {
counter.increment();
expect(counter.value).toBe(1);
});
it('starts at zero', () => {
expect(counter.value).toBe(0); // Passes
});
});
```
**4. Network/External Dependencies**
```typescript
// Flaky - real network call
it('fetches data', async () => {
const data = await fetch('https://api.example.com/data');
expect(data).toBeDefined();
});
// Fixed - mock the network
it('fetches data', async () => {
server.use(
rest.get('https://api.example.com/data', (req, res, ctx) =>
res(ctx.json({ value: 42 }))
)
);
const data = await fetchData();
expect(data.value).toBe(42);
});
```
### Flaky Test Detection
```javascript
// jest.config.js
module.exports = {
// Run each test multiple times to detect flakiness
testEnvironment: 'jsdom',
// Add reporters to track flaky tests
reporters: [
'default',
['jest-junit', { outputDirectory: './reports' }],
],
};
// Run tests multiple times
// npx jest --runInBand --testTimeout=10000 --repeat=5
```
### Quarantine Strategy
1. **Identify** - Track tests that fail randomly
2. **Quarantine** - Move to separate suite, run separately
3. **Fix** - Investigate and fix root cause
4. **Restore** - Move back to main suite
```typescript
// Temporarily skip flaky test
it.skip('flaky test to fix', () => {
// TODO: Fix timing issue in #123
});
// Or run only when investigating
it.todo('investigate flaky behavior');
```
---
## Code Review for Testability
Questions to ask during code review to ensure testable code.
### Testability Checklist
**Functions and Methods:**
- [ ] Does it have a single responsibility?
- [ ] Are dependencies injected?
- [ ] Can it be tested without mocking internals?
- [ ] Does it return a value or have observable side effects?
**Components:**
- [ ] Are props descriptive and minimal?
- [ ] Can behavior be triggered via user events?
- [ ] Are loading/error states exposed?
- [ ] Can it be rendered without a full app context?
**State Management:**
- [ ] Is state minimal and derived where possible?
- [ ] Can state changes be triggered and observed?
- [ ] Are side effects separated from reducers?
### Review Comments
**Before:**
```typescript
// Hard to test - embedded dependency
function processPayment(order: Order) {
const stripe = new Stripe(process.env.STRIPE_KEY);
return stripe.charges.create({
amount: order.total,
currency: 'usd',
});
}
```
**Review Comment:**
> Consider injecting the payment processor to improve testability:
> ```typescript
> function processPayment(order: Order, processor: PaymentProcessor) {
> return processor.charge(order.total, 'usd');
> }
> ```
> This allows testing with a mock processor without hitting Stripe's API.
---
## Test Maintenance Strategies
Keep tests maintainable as the codebase evolves.
### Reducing Duplication
**Use helpers for common assertions:**
```typescript
// __tests__/helpers/assertions.ts
export function expectLoadingState(container: HTMLElement) {
expect(within(container).getByRole('progressbar')).toBeInTheDocument();
}
export function expectErrorState(container: HTMLElement, message: string) {
expect(within(container).getByRole('alert')).toHaveTextContent(message);
}
// Usage
it('shows loading state', () => {
render(<DataList />);
expectLoadingState(screen.getByTestId('data-list'));
});
```
**Use factory functions:**
```typescript
// Instead of repeating setup
function renderWithUser(ui: ReactElement, user = createUser()) {
return {
user,
...render(<AuthProvider user={user}>{ui}</AuthProvider>),
};
}
```
### Updating Tests When Code Changes
**Scenario: Renaming a prop**
```typescript
// Old component
<Button onClick={handleClick} />
// New component
<Button onPress={handleClick} />
// Find and update all tests
// grep -r "onClick" __tests__/ --include="*.test.tsx"
```
**Scenario: Changing API response shape**
```typescript
// Update factory first
export function createUserResponse(overrides = {}) {
return {
user: { // New nested structure
id: '1',
name: 'Test User',
...overrides,
},
};
}
// Tests automatically get new shape
```
### When to Delete Tests
- **Redundant coverage** - Multiple tests testing the same thing
- **Testing implementation** - Tests that break on refactor
- **Obsolete features** - Tests for removed functionality
- **Flaky beyond repair** - Tests that can't be stabilized
### Test Documentation
```typescript
/**
* @group integration
* @requires database
*
* Tests for the order processing workflow.
* These tests require a running PostgreSQL instance.
*
* Setup: docker-compose up -d postgres
*/
describe('OrderProcessor', () => {
/**
* Verifies that orders with backordered items
* are split into separate fulfillment batches.
*
* Related: JIRA-1234
*/
it('splits orders with backordered items', () => {});
});
```
---
## Debugging Failed Tests
Techniques for investigating test failures.
### Jest Debugging
**Run single test:**
```bash
# By name pattern
npx jest -t "should validate email"
# By file
npx jest src/utils/__tests__/validation.test.ts
# Watch mode for iteration
npx jest --watch
```
**Debug with Node inspector:**
```bash
node --inspect-brk node_modules/.bin/jest --runInBand
# Open chrome://inspect in Chrome
```
**Verbose output:**
```bash
npx jest --verbose --no-coverage
```
### React Testing Library Debugging
```typescript
it('renders user profile', async () => {
render(<UserProfile userId="123" />);
// Print current DOM
screen.debug();
// Print specific element
screen.debug(screen.getByRole('heading'));
// Log accessible roles
screen.logTestingPlaygroundURL(); // Opens interactive playground
// Check what queries would match
const element = screen.getByRole('button');
console.log(prettyDOM(element));
});
```
### Playwright Debugging
```bash
# Debug mode - opens browser with inspector
npx playwright test --debug
# UI mode - visual test runner
npx playwright test --ui
# Headed mode - see browser
npx playwright test --headed
# Trace viewer after failure
npx playwright show-trace trace.zip
```
**Pause in test:**
```typescript
test('debug this', async ({ page }) => {
await page.goto('/');
await page.pause(); // Opens inspector
await page.click('button');
});
```
### Common Failure Patterns
| Symptom | Likely Cause | Debug Approach |
|---------|--------------|----------------|
| "Unable to find element" | Wrong query or element not rendered | `screen.debug()`, check async |
| "Expected X, received Y" | Logic error or stale mock | Log intermediate values |
| "Timeout exceeded" | Slow async or missing await | Increase timeout, check promises |
| "Cannot read property of undefined" | Missing mock or setup | Check beforeEach, mock returns |
| Passes locally, fails in CI | Environment difference | Check env vars, timing |
### Investigating Flaky Failures
```typescript
// Add logging for intermittent failures
it('processes order', async () => {
console.log('Test started at', Date.now());
const order = await createOrder();
console.log('Order created:', order.id);
const result = await processOrder(order);
console.log('Process result:', result);
expect(result.status).toBe('completed');
});
```
---
## Quality Metrics and KPIs
Measure test suite effectiveness and track quality improvements.
### Key Metrics
**Coverage Metrics:**
| Metric | Target | Measurement |
|--------|--------|-------------|
| Line coverage | 80% | `jest --coverage` |
| Branch coverage | 75% | `jest --coverage` |
| Function coverage | 80% | `jest --coverage` |
| Critical path coverage | 95% | Custom tracking |
**Test Suite Health:**
| Metric | Target | Measurement |
|--------|--------|-------------|
| Test pass rate | 100% | CI reports |
| Flaky test rate | <1% | Track retries |
| Test execution time | <5 min | CI timing |
| Tests per component | ≥3 | Test count / components |
**Defect Metrics:**
| Metric | Target | Measurement |
|--------|--------|-------------|
| Defects found in testing | >70% | Bug tracking |
| Defects escaped to prod | <10% | Production bugs |
| Regression rate | <5% | Bugs reintroduced |
| Mean time to detect | <1 day | Bug timestamps |
### Dashboard Example
```typescript
// scripts/test-metrics.ts
import { readCoverageReport } from './utils';
const coverage = readCoverageReport('./coverage/coverage-summary.json');
const testResults = readTestReport('./reports/jest-results.json');
const metrics = {
coverage: {
lines: coverage.total.lines.pct,
branches: coverage.total.branches.pct,
functions: coverage.total.functions.pct,
},
tests: {
total: testResults.numTotalTests,
passed: testResults.numPassedTests,
failed: testResults.numFailedTests,
passRate: (testResults.numPassedTests / testResults.numTotalTests) * 100,
},
execution: {
duration: testResults.testResults.reduce((sum, r) => sum + r.duration, 0),
},
};
console.log('Test Metrics:', JSON.stringify(metrics, null, 2));
```
### CI Quality Gates
```yaml
# .github/workflows/quality.yml
name: Quality Gates
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
- run: npm ci
- run: npm test -- --coverage
# Coverage gate
- name: Check coverage
run: |
coverage=$(jq '.total.lines.pct' coverage/coverage-summary.json)
if (( $(echo "$coverage < 80" | bc -l) )); then
echo "Coverage $coverage% is below 80% threshold"
exit 1
fi
# Test count gate
- name: Check test count
run: |
tests=$(jq '.numTotalTests' reports/test-results.json)
if [ "$tests" -lt 100 ]; then
echo "Test count $tests is below minimum of 100"
exit 1
fi
```
### Trend Tracking
Track metrics over time to identify trends:
```typescript
// Weekly metrics collection
{
"week": "2024-W03",
"coverage": {
"lines": 82.4,
"branches": 76.1,
"trend": "+1.2%" // vs previous week
},
"tests": {
"total": 487,
"new": 23,
"removed": 5
},
"execution": {
"avgDuration": 245, // seconds
"trend": "-12s"
},
"flaky": {
"count": 3,
"rate": 0.6
}
}
```
---
## Summary
1. **Write testable code** - Inject dependencies, use pure functions, separate concerns
2. **Name tests clearly** - Describe behavior, not implementation
3. **Follow AAA pattern** - Arrange, Act, Assert for clear structure
4. **Isolate tests** - Fresh state, reset mocks, no dependencies between tests
5. **Fix flaky tests** - Handle timing, use deterministic data, mock externals
6. **Review for testability** - Check during code review, not after
7. **Maintain tests** - Reduce duplication, update with code changes
8. **Debug systematically** - Use debug tools, log strategically
9. **Measure quality** - Track coverage, pass rate, execution time
FILE:references/testing_strategies.md
# Testing Strategies for React and Next.js Applications
Comprehensive guide to test architecture, coverage targets, and CI/CD integration patterns.
---
## Table of Contents
- [The Testing Pyramid](#the-testing-pyramid)
- [Testing Types Deep Dive](#testing-types-deep-dive)
- [Coverage Targets and Thresholds](#coverage-targets-and-thresholds)
- [Test Organization Patterns](#test-organization-patterns)
- [CI/CD Integration Strategies](#cicd-integration-strategies)
- [Testing Decision Framework](#testing-decision-framework)
---
## The Testing Pyramid
The testing pyramid guides how to distribute testing effort across different test types for optimal ROI.
### Classic Pyramid Structure
```
/\
/ \ E2E Tests (5-10%)
/----\ - User journey validation
/ \ - Critical path coverage
/--------\ Integration Tests (20-30%)
/ \ - Component interactions
/ \ - API integration
/--------------\ Unit Tests (60-70%)
/ \ - Individual functions
------------------ - Isolated components
```
### React/Next.js Adapted Pyramid
For frontend applications, the pyramid shifts slightly:
| Level | Percentage | Tools | Focus |
|-------|------------|-------|-------|
| Unit | 50-60% | Jest, RTL | Pure functions, hooks, isolated components |
| Integration | 25-35% | RTL, MSW | Component trees, API calls, context |
| E2E | 10-15% | Playwright | Critical user flows, cross-page navigation |
### Why This Distribution?
**Unit tests are fast and cheap:**
- Execute in milliseconds
- Pinpoint failures precisely
- Easy to maintain
- Run on every commit
**Integration tests balance coverage and cost:**
- Test realistic scenarios
- Catch component interaction bugs
- Moderate execution time
- Run on every PR
**E2E tests are expensive but essential:**
- Validate real user experience
- Catch deployment issues
- Slow and brittle
- Run on staging/production
---
## Testing Types Deep Dive
### Unit Testing
**Purpose:** Verify individual units of code work correctly in isolation.
**What to Unit Test:**
- Pure utility functions
- Custom hooks (with renderHook)
- Individual component rendering
- State reducers
- Validation logic
- Data transformers
**Example: Testing a Pure Function**
```typescript
// utils/formatPrice.ts
export function formatPrice(cents: number, currency = 'USD'): string {
const formatter = new Intl.NumberFormat('en-US', {
style: 'currency',
currency,
});
return formatter.format(cents / 100);
}
// utils/formatPrice.test.ts
describe('formatPrice', () => {
it('formats cents to USD by default', () => {
expect(formatPrice(1999)).toBe('$19.99');
});
it('handles zero', () => {
expect(formatPrice(0)).toBe('$0.00');
});
it('supports different currencies', () => {
expect(formatPrice(1999, 'EUR')).toContain('€');
});
it('handles large numbers', () => {
expect(formatPrice(100000000)).toBe('$1,000,000.00');
});
});
```
**Example: Testing a Custom Hook**
```typescript
// hooks/useCounter.ts
export function useCounter(initial = 0) {
const [count, setCount] = useState(initial);
const increment = () => setCount(c => c + 1);
const decrement = () => setCount(c => c - 1);
const reset = () => setCount(initial);
return { count, increment, decrement, reset };
}
// hooks/useCounter.test.ts
import { renderHook, act } from '@testing-library/react';
import { useCounter } from './useCounter';
describe('useCounter', () => {
it('starts with initial value', () => {
const { result } = renderHook(() => useCounter(5));
expect(result.current.count).toBe(5);
});
it('increments count', () => {
const { result } = renderHook(() => useCounter(0));
act(() => result.current.increment());
expect(result.current.count).toBe(1);
});
it('decrements count', () => {
const { result } = renderHook(() => useCounter(5));
act(() => result.current.decrement());
expect(result.current.count).toBe(4);
});
it('resets to initial value', () => {
const { result } = renderHook(() => useCounter(10));
act(() => result.current.increment());
act(() => result.current.reset());
expect(result.current.count).toBe(10);
});
});
```
### Integration Testing
**Purpose:** Verify multiple units work together correctly.
**What to Integration Test:**
- Component trees with multiple children
- Components with context providers
- Form submission flows
- API call and response handling
- State management interactions
- Router-dependent components
**Example: Testing Component with API Call**
```typescript
// components/UserProfile.tsx
export function UserProfile({ userId }: { userId: string }) {
const [user, setUser] = useState<User | null>(null);
const [loading, setLoading] = useState(true);
const [error, setError] = useState<string | null>(null);
useEffect(() => {
fetch(`/api/users/userId`)
.then(res => res.json())
.then(data => setUser(data))
.catch(err => setError(err.message))
.finally(() => setLoading(false));
}, [userId]);
if (loading) return <div>Loading...</div>;
if (error) return <div>Error: {error}</div>;
return <div>{user?.name}</div>;
}
// components/UserProfile.test.tsx
import { render, screen, waitFor } from '@testing-library/react';
import { rest } from 'msw';
import { setupServer } from 'msw/node';
import { UserProfile } from './UserProfile';
const server = setupServer(
rest.get('/api/users/:id', (req, res, ctx) => {
return res(ctx.json({ id: req.params.id, name: 'John Doe' }));
})
);
beforeAll(() => server.listen());
afterEach(() => server.resetHandlers());
afterAll(() => server.close());
describe('UserProfile', () => {
it('shows loading state initially', () => {
render(<UserProfile userId="123" />);
expect(screen.getByText('Loading...')).toBeInTheDocument();
});
it('displays user name after loading', async () => {
render(<UserProfile userId="123" />);
await waitFor(() => {
expect(screen.getByText('John Doe')).toBeInTheDocument();
});
});
it('displays error on API failure', async () => {
server.use(
rest.get('/api/users/:id', (req, res, ctx) => {
return res(ctx.status(500));
})
);
render(<UserProfile userId="123" />);
await waitFor(() => {
expect(screen.getByText(/Error/)).toBeInTheDocument();
});
});
});
```
### End-to-End Testing
**Purpose:** Verify complete user flows work in a real browser environment.
**What to E2E Test:**
- Critical business flows (checkout, signup, login)
- Cross-page navigation sequences
- Authentication flows
- Third-party integrations
- Payment processing
- Form wizards
**Example: Testing Checkout Flow**
```typescript
// e2e/checkout.spec.ts
import { test, expect } from '@playwright/test';
test.describe('Checkout Flow', () => {
test.beforeEach(async ({ page }) => {
await page.goto('/');
});
test('completes purchase successfully', async ({ page }) => {
// Add product to cart
await page.goto('/products/widget-pro');
await page.getByRole('button', { name: 'Add to Cart' }).click();
// Verify cart updated
await expect(page.getByTestId('cart-count')).toHaveText('1');
// Go to checkout
await page.getByRole('link', { name: 'Checkout' }).click();
// Fill shipping info
await page.getByLabel('Email').fill('test@example.com');
await page.getByLabel('Address').fill('123 Test St');
await page.getByLabel('City').fill('Test City');
await page.getByLabel('Zip').fill('12345');
// Fill payment info (test card)
await page.getByLabel('Card Number').fill('4242424242424242');
await page.getByLabel('Expiry').fill('12/25');
await page.getByLabel('CVC').fill('123');
// Submit order
await page.getByRole('button', { name: 'Place Order' }).click();
// Verify confirmation
await expect(page).toHaveURL(/\/orders\/\w+/);
await expect(page.getByText('Order Confirmed')).toBeVisible();
});
test('shows validation errors for invalid input', async ({ page }) => {
await page.goto('/checkout');
await page.getByRole('button', { name: 'Place Order' }).click();
await expect(page.getByText('Email is required')).toBeVisible();
await expect(page.getByText('Address is required')).toBeVisible();
});
});
```
### Visual Regression Testing
**Purpose:** Catch unintended visual changes to UI components.
**Tools:** Playwright visual comparisons, Percy, Chromatic
**Example: Visual Snapshot Test**
```typescript
// e2e/visual/components.spec.ts
import { test, expect } from '@playwright/test';
test.describe('Visual Regression', () => {
test('button variants render correctly', async ({ page }) => {
await page.goto('/storybook/button');
await expect(page).toHaveScreenshot('button-variants.png');
});
test('responsive header', async ({ page }) => {
// Desktop
await page.setViewportSize({ width: 1280, height: 720 });
await page.goto('/');
await expect(page.locator('header')).toHaveScreenshot('header-desktop.png');
// Mobile
await page.setViewportSize({ width: 375, height: 667 });
await expect(page.locator('header')).toHaveScreenshot('header-mobile.png');
});
});
```
### Accessibility Testing
**Purpose:** Ensure application is usable by people with disabilities.
**Tools:** jest-axe, @axe-core/playwright
**Example: Automated A11y Testing**
```typescript
// Unit/Integration level with jest-axe
import { render } from '@testing-library/react';
import { axe, toHaveNoViolations } from 'jest-axe';
import { Button } from './Button';
expect.extend(toHaveNoViolations);
describe('Button accessibility', () => {
it('has no accessibility violations', async () => {
const { container } = render(<Button>Click me</Button>);
const results = await axe(container);
expect(results).toHaveNoViolations();
});
});
// E2E level with Playwright + Axe
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';
test('homepage has no a11y violations', async ({ page }) => {
await page.goto('/');
const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
});
```
---
## Coverage Targets and Thresholds
### Recommended Thresholds by Project Type
| Project Type | Statements | Branches | Functions | Lines |
|--------------|------------|----------|-----------|-------|
| Startup/MVP | 60% | 50% | 60% | 60% |
| Growing Product | 75% | 70% | 75% | 75% |
| Enterprise | 85% | 80% | 85% | 85% |
| Safety Critical | 95% | 90% | 95% | 95% |
### Coverage by Code Type
**High Coverage Priority (80%+):**
- Business logic
- State management
- API handlers
- Form validation
- Authentication/authorization
- Payment processing
**Medium Coverage Priority (60-80%):**
- UI components
- Utility functions
- Data transformers
- Custom hooks
**Lower Coverage Priority (40-60%):**
- Static pages
- Simple wrappers
- Configuration files
- Types/interfaces
### Jest Coverage Configuration
```javascript
// jest.config.js
module.exports = {
collectCoverageFrom: [
'src/**/*.{ts,tsx}',
'!src/**/*.d.ts',
'!src/**/*.stories.{ts,tsx}',
'!src/**/index.{ts,tsx}', // barrel files
'!src/types/**',
],
coverageThreshold: {
global: {
statements: 80,
branches: 75,
functions: 80,
lines: 80,
},
// Higher thresholds for critical paths
'./src/services/payment/': {
statements: 95,
branches: 90,
functions: 95,
lines: 95,
},
'./src/services/auth/': {
statements: 90,
branches: 85,
functions: 90,
lines: 90,
},
},
coverageReporters: ['text', 'lcov', 'html', 'json'],
};
```
---
## Test Organization Patterns
### Co-located Tests (Recommended for React)
```
src/
├── components/
│ ├── Button/
│ │ ├── Button.tsx
│ │ ├── Button.test.tsx # Unit tests
│ │ ├── Button.stories.tsx # Storybook
│ │ └── index.ts
│ └── Form/
│ ├── Form.tsx
│ ├── Form.test.tsx
│ └── Form.integration.test.tsx # Integration tests
├── hooks/
│ ├── useAuth.ts
│ └── useAuth.test.ts
└── utils/
├── formatters.ts
└── formatters.test.ts
```
### Separate Test Directory
```
src/
├── components/
├── hooks/
└── utils/
__tests__/
├── unit/
│ ├── components/
│ ├── hooks/
│ └── utils/
├── integration/
│ └── flows/
└── fixtures/
├── users.json
└── products.json
e2e/
├── specs/
│ ├── auth.spec.ts
│ └── checkout.spec.ts
├── fixtures/
│ └── auth.ts
└── pages/ # Page Object Models
├── LoginPage.ts
└── CheckoutPage.ts
```
### Test File Naming Conventions
| Pattern | Use Case |
|---------|----------|
| `*.test.ts` | Unit tests |
| `*.spec.ts` | Integration/E2E tests |
| `*.integration.test.ts` | Explicit integration tests |
| `*.e2e.spec.ts` | Explicit E2E tests |
| `*.a11y.test.ts` | Accessibility tests |
| `*.visual.spec.ts` | Visual regression tests |
---
## CI/CD Integration Strategies
### Pipeline Stages
```yaml
# .github/workflows/test.yml
name: Test Pipeline
on:
push:
branches: [main, dev]
pull_request:
branches: [main, dev]
jobs:
unit:
name: Unit Tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- run: npm ci
- run: npm run test:unit -- --coverage
- uses: codecov/codecov-action@v4
with:
files: coverage/lcov.info
fail_ci_if_error: true
integration:
name: Integration Tests
runs-on: ubuntu-latest
needs: unit
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- run: npm ci
- run: npm run test:integration
e2e:
name: E2E Tests
runs-on: ubuntu-latest
needs: integration
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- run: npm ci
- run: npx playwright install --with-deps
- run: npm run build
- run: npm run test:e2e
- uses: actions/upload-artifact@v4
if: failure()
with:
name: playwright-report
path: playwright-report/
```
### Test Splitting for Speed
```yaml
# Run E2E tests in parallel across multiple machines
e2e:
strategy:
matrix:
shard: [1, 2, 3, 4]
steps:
- run: npx playwright test --shard={ matrix.shard}/4
```
### PR Gating Rules
| Test Type | When to Run | Block Merge? |
|-----------|-------------|--------------|
| Unit | Every commit | Yes |
| Integration | Every PR | Yes |
| E2E (smoke) | Every PR | Yes |
| E2E (full) | Merge to main | No (alert only) |
| Visual | Every PR | No (review required) |
| Performance | Weekly/Release | No (alert only) |
---
## Testing Decision Framework
### When to Write Which Test
```
Is it a pure function with no side effects?
├── Yes → Unit test
└── No
├── Does it make API calls or use context?
│ ├── Yes → Integration test with mocking
│ └── No
│ ├── Is it a critical user flow?
│ │ ├── Yes → E2E test
│ │ └── No → Integration test
└── Is it UI-focused with many visual states?
├── Yes → Storybook + Visual test
└── No → Component unit test
```
### Test ROI Matrix
| Test Type | Write Time | Run Time | Maintenance | Confidence |
|-----------|------------|----------|-------------|------------|
| Unit | Low | Very Fast | Low | Medium |
| Integration | Medium | Fast | Medium | High |
| E2E | High | Slow | High | Very High |
| Visual | Low | Medium | Medium | High (UI) |
### When NOT to Test
- Generated code (GraphQL types, Prisma client)
- Third-party library internals
- Implementation details (internal state, private methods)
- Simple pass-through wrappers
- Type definitions
### Red Flags in Testing Strategy
| Red Flag | Problem | Solution |
|----------|---------|----------|
| E2E tests > 30% | Slow CI, flaky tests | Push logic down to integration |
| Only unit tests | Missing interaction bugs | Add integration tests |
| Testing mocks | Not testing real behavior | Test behavior, not implementation |
| 100% coverage goal | Diminishing returns | Focus on critical paths |
| No E2E tests | Missing deployment issues | Add smoke tests for critical flows |
---
## Summary
1. **Follow the pyramid:** 60% unit, 30% integration, 10% E2E
2. **Set thresholds by risk:** Higher coverage for critical paths
3. **Co-locate tests:** Keep tests close to source code
4. **Automate in CI:** Run tests on every PR, gate merges on failure
5. **Decide wisely:** Not everything needs every type of test
FILE:references/test_automation_patterns.md
# Test Automation Patterns for React and Next.js
Reusable patterns for structuring test code, mocking dependencies, and handling async operations.
---
## Table of Contents
- [Page Object Model for React](#page-object-model-for-react)
- [Test Data Factories](#test-data-factories)
- [Fixture Management](#fixture-management)
- [Mocking Strategies](#mocking-strategies)
- [Custom Test Utilities](#custom-test-utilities)
- [Async Testing Patterns](#async-testing-patterns)
- [Snapshot Testing Guidelines](#snapshot-testing-guidelines)
---
## Page Object Model for React
The Page Object Model (POM) encapsulates page interactions into reusable classes, reducing test maintenance.
### Playwright Page Objects
```typescript
// e2e/pages/LoginPage.ts
import { Page, Locator, expect } from '@playwright/test';
export class LoginPage {
readonly page: Page;
readonly emailInput: Locator;
readonly passwordInput: Locator;
readonly submitButton: Locator;
readonly errorMessage: Locator;
constructor(page: Page) {
this.page = page;
this.emailInput = page.getByLabel('Email');
this.passwordInput = page.getByLabel('Password');
this.submitButton = page.getByRole('button', { name: 'Sign in' });
this.errorMessage = page.getByRole('alert');
}
async goto() {
await this.page.goto('/login');
}
async login(email: string, password: string) {
await this.emailInput.fill(email);
await this.passwordInput.fill(password);
await this.submitButton.click();
}
async expectError(message: string) {
await expect(this.errorMessage).toContainText(message);
}
async expectRedirectToDashboard() {
await expect(this.page).toHaveURL('/dashboard');
}
}
```
**Usage in Tests:**
```typescript
// e2e/auth.spec.ts
import { test, expect } from '@playwright/test';
import { LoginPage } from './pages/LoginPage';
test.describe('Authentication', () => {
let loginPage: LoginPage;
test.beforeEach(async ({ page }) => {
loginPage = new LoginPage(page);
await loginPage.goto();
});
test('successful login redirects to dashboard', async () => {
await loginPage.login('user@example.com', 'password123');
await loginPage.expectRedirectToDashboard();
});
test('invalid credentials show error', async () => {
await loginPage.login('user@example.com', 'wrongpassword');
await loginPage.expectError('Invalid credentials');
});
});
```
### Component Object Model (React Testing Library)
```typescript
// __tests__/objects/LoginFormObject.ts
import { screen, fireEvent, waitFor } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
export class LoginFormObject {
get emailInput() {
return screen.getByLabelText(/email/i);
}
get passwordInput() {
return screen.getByLabelText(/password/i);
}
get submitButton() {
return screen.getByRole('button', { name: /sign in/i });
}
get errorMessage() {
return screen.queryByRole('alert');
}
async fillEmail(email: string) {
await userEvent.type(this.emailInput, email);
}
async fillPassword(password: string) {
await userEvent.type(this.passwordInput, password);
}
async submit() {
await userEvent.click(this.submitButton);
}
async login(email: string, password: string) {
await this.fillEmail(email);
await this.fillPassword(password);
await this.submit();
}
async expectError(message: string) {
await waitFor(() => {
expect(this.errorMessage).toHaveTextContent(message);
});
}
}
```
### When to Use POM
| Scenario | Use POM? |
|----------|----------|
| Complex pages with many interactions | Yes |
| Reusable components tested across suites | Yes |
| Simple single-use tests | No (overkill) |
| E2E tests with shared flows | Yes |
---
## Test Data Factories
Factories create test data with sensible defaults, reducing boilerplate and improving maintainability.
### Basic Factory Pattern
```typescript
// __tests__/factories/userFactory.ts
interface User {
id: string;
email: string;
name: string;
role: 'admin' | 'user' | 'guest';
createdAt: Date;
preferences: {
theme: 'light' | 'dark';
notifications: boolean;
};
}
let idCounter = 0;
export function createUser(overrides: Partial<User> = {}): User {
return {
id: `user-++idCounter`,
email: `useridCounter@example.com`,
name: `Test User idCounter`,
role: 'user',
createdAt: new Date('2024-01-01'),
preferences: {
theme: 'light',
notifications: true,
},
...overrides,
// Deep merge preferences if provided
preferences: {
theme: 'light',
notifications: true,
...overrides.preferences,
},
};
}
// Specialized builders
export function createAdmin(overrides: Partial<User> = {}): User {
return createUser({ role: 'admin', ...overrides });
}
export function createGuest(overrides: Partial<User> = {}): User {
return createUser({
role: 'guest',
name: 'Guest',
email: '',
...overrides,
});
}
```
### Builder Pattern for Complex Objects
```typescript
// __tests__/factories/orderBuilder.ts
interface OrderItem {
productId: string;
quantity: number;
price: number;
}
interface Order {
id: string;
userId: string;
items: OrderItem[];
status: 'pending' | 'processing' | 'shipped' | 'delivered';
total: number;
shippingAddress: Address;
createdAt: Date;
}
export class OrderBuilder {
private order: Partial<Order> = {};
private items: OrderItem[] = [];
withId(id: string): this {
this.order.id = id;
return this;
}
forUser(userId: string): this {
this.order.userId = userId;
return this;
}
withItem(productId: string, quantity: number, price: number): this {
this.items.push({ productId, quantity, price });
return this;
}
withStatus(status: Order['status']): this {
this.order.status = status;
return this;
}
shippedTo(address: Address): this {
this.order.shippingAddress = address;
return this;
}
build(): Order {
const total = this.items.reduce(
(sum, item) => sum + item.price * item.quantity,
0
);
return {
id: this.order.id || `order-Date.now()`,
userId: this.order.userId || 'user-1',
items: this.items,
status: this.order.status || 'pending',
total,
shippingAddress: this.order.shippingAddress || createAddress(),
createdAt: new Date(),
};
}
}
// Usage
const order = new OrderBuilder()
.forUser('user-123')
.withItem('product-1', 2, 29.99)
.withItem('product-2', 1, 49.99)
.withStatus('processing')
.build();
```
### Factory with Faker
```typescript
// __tests__/factories/productFactory.ts
import { faker } from '@faker-js/faker';
interface Product {
id: string;
name: string;
description: string;
price: number;
category: string;
inStock: boolean;
imageUrl: string;
}
export function createProduct(overrides: Partial<Product> = {}): Product {
return {
id: faker.string.uuid(),
name: faker.commerce.productName(),
description: faker.commerce.productDescription(),
price: parseFloat(faker.commerce.price({ min: 10, max: 500 })),
category: faker.commerce.department(),
inStock: faker.datatype.boolean({ probability: 0.8 }),
imageUrl: faker.image.url(),
...overrides,
};
}
export function createProducts(count: number): Product[] {
return Array.from({ length: count }, () => createProduct());
}
```
---
## Fixture Management
Fixtures provide consistent test data and setup across test suites.
### Playwright Fixtures
```typescript
// e2e/fixtures/auth.ts
import { test as base, Page } from '@playwright/test';
import { createUser } from '../factories/userFactory';
interface AuthFixtures {
authenticatedPage: Page;
adminPage: Page;
testUser: ReturnType<typeof createUser>;
}
export const test = base.extend<AuthFixtures>({
testUser: async ({}, use) => {
const user = createUser();
await use(user);
},
authenticatedPage: async ({ page, testUser }, use) => {
// Login via API to skip UI
await page.request.post('/api/auth/login', {
data: {
email: testUser.email,
password: 'testpassword',
},
});
// Get session cookie
const cookies = await page.context().cookies();
await page.context().addCookies(cookies);
await use(page);
},
adminPage: async ({ page }, use) => {
const admin = createUser({ role: 'admin' });
await page.request.post('/api/auth/login', {
data: {
email: admin.email,
password: 'adminpassword',
},
});
await use(page);
},
});
export { expect } from '@playwright/test';
```
**Using Custom Fixtures:**
```typescript
// e2e/dashboard.spec.ts
import { test, expect } from './fixtures/auth';
test('dashboard shows user name', async ({ authenticatedPage, testUser }) => {
await authenticatedPage.goto('/dashboard');
await expect(authenticatedPage.getByText(testUser.name)).toBeVisible();
});
test('admin sees admin panel', async ({ adminPage }) => {
await adminPage.goto('/dashboard');
await expect(adminPage.getByText('Admin Panel')).toBeVisible();
});
```
### Jest Test Setup
```typescript
// jest.setup.ts
import '@testing-library/jest-dom';
import { server } from './__tests__/mocks/server';
// Start MSW server before all tests
beforeAll(() => server.listen({ onUnhandledRequest: 'error' }));
// Reset handlers after each test
afterEach(() => server.resetHandlers());
// Clean up after all tests
afterAll(() => server.close());
// Mock window.matchMedia
Object.defineProperty(window, 'matchMedia', {
writable: true,
value: jest.fn().mockImplementation(query => ({
matches: false,
media: query,
onchange: null,
addListener: jest.fn(),
removeListener: jest.fn(),
addEventListener: jest.fn(),
removeEventListener: jest.fn(),
dispatchEvent: jest.fn(),
})),
});
// Mock IntersectionObserver
global.IntersectionObserver = class IntersectionObserver {
constructor() {}
observe() {}
unobserve() {}
disconnect() {}
};
```
### Shared Test Data Files
```typescript
// __tests__/fixtures/products.json
{
"products": [
{
"id": "prod-1",
"name": "Widget Pro",
"price": 29.99,
"category": "Electronics"
},
{
"id": "prod-2",
"name": "Gadget Plus",
"price": 49.99,
"category": "Electronics"
}
]
}
// __tests__/fixtures/index.ts
import productsData from './products.json';
import usersData from './users.json';
export const fixtures = {
products: productsData.products,
users: usersData.users,
};
```
---
## Mocking Strategies
### MSW (Mock Service Worker) for API Mocking
MSW intercepts network requests at the service worker level, working in both browser and Node.
**Handler Setup:**
```typescript
// __tests__/mocks/handlers.ts
import { rest } from 'msw';
import { createUser } from '../factories/userFactory';
import { createProduct } from '../factories/productFactory';
export const handlers = [
// GET /api/users/:id
rest.get('/api/users/:id', (req, res, ctx) => {
const { id } = req.params;
const user = createUser({ id: id as string });
return res(ctx.json(user));
}),
// GET /api/products
rest.get('/api/products', (req, res, ctx) => {
const category = req.url.searchParams.get('category');
const products = Array.from({ length: 10 }, () => createProduct());
const filtered = category
? products.filter(p => p.category === category)
: products;
return res(ctx.json(filtered));
}),
// POST /api/orders
rest.post('/api/orders', async (req, res, ctx) => {
const body = await req.json();
return res(
ctx.status(201),
ctx.json({
id: `order-Date.now()`,
...body,
status: 'pending',
})
);
}),
// Error simulation
rest.get('/api/error', (req, res, ctx) => {
return res(
ctx.status(500),
ctx.json({ error: 'Internal Server Error' })
);
}),
];
```
**Server Setup:**
```typescript
// __tests__/mocks/server.ts
import { setupServer } from 'msw/node';
import { handlers } from './handlers';
export const server = setupServer(...handlers);
```
**Overriding Handlers in Tests:**
```typescript
// __tests__/components/ProductList.test.tsx
import { render, screen, waitFor } from '@testing-library/react';
import { rest } from 'msw';
import { server } from '../mocks/server';
import { ProductList } from '../../src/components/ProductList';
describe('ProductList', () => {
it('shows loading state', () => {
render(<ProductList />);
expect(screen.getByText('Loading...')).toBeInTheDocument();
});
it('renders products', async () => {
render(<ProductList />);
await waitFor(() => {
expect(screen.getAllByTestId('product-card')).toHaveLength(10);
});
});
it('shows error state on API failure', async () => {
server.use(
rest.get('/api/products', (req, res, ctx) => {
return res(ctx.status(500));
})
);
render(<ProductList />);
await waitFor(() => {
expect(screen.getByText(/error loading products/i)).toBeInTheDocument();
});
});
it('shows empty state when no products', async () => {
server.use(
rest.get('/api/products', (req, res, ctx) => {
return res(ctx.json([]));
})
);
render(<ProductList />);
await waitFor(() => {
expect(screen.getByText('No products found')).toBeInTheDocument();
});
});
});
```
### Jest Module Mocking
```typescript
// Mocking a module
jest.mock('../../src/services/analytics', () => ({
trackEvent: jest.fn(),
trackPageView: jest.fn(),
setUser: jest.fn(),
}));
// Mocking with implementation
jest.mock('next/router', () => ({
useRouter: jest.fn().mockReturnValue({
pathname: '/test',
push: jest.fn(),
replace: jest.fn(),
query: {},
}),
}));
// Partial mock (keep some real implementations)
jest.mock('../../src/utils/helpers', () => ({
...jest.requireActual('../../src/utils/helpers'),
sendEmail: jest.fn().mockResolvedValue({ success: true }),
}));
```
### Mocking Hooks
```typescript
// __tests__/hooks/useAuth.test.tsx
import { renderHook, act } from '@testing-library/react';
import { useAuth } from '../../src/hooks/useAuth';
import * as authService from '../../src/services/auth';
jest.mock('../../src/services/auth');
const mockAuthService = authService as jest.Mocked<typeof authService>;
describe('useAuth', () => {
beforeEach(() => {
jest.clearAllMocks();
});
it('logs in user successfully', async () => {
const mockUser = { id: '1', email: 'test@example.com' };
mockAuthService.login.mockResolvedValue(mockUser);
const { result } = renderHook(() => useAuth());
await act(async () => {
await result.current.login('test@example.com', 'password');
});
expect(result.current.user).toEqual(mockUser);
expect(result.current.isAuthenticated).toBe(true);
});
it('handles login error', async () => {
mockAuthService.login.mockRejectedValue(new Error('Invalid credentials'));
const { result } = renderHook(() => useAuth());
await act(async () => {
try {
await result.current.login('test@example.com', 'wrong');
} catch (e) {
// Expected
}
});
expect(result.current.user).toBeNull();
expect(result.current.error).toBe('Invalid credentials');
});
});
```
---
## Custom Test Utilities
### Render with Providers
```typescript
// __tests__/utils/renderWithProviders.tsx
import React, { ReactElement } from 'react';
import { render, RenderOptions } from '@testing-library/react';
import { QueryClient, QueryClientProvider } from '@tanstack/react-query';
import { ThemeProvider } from '../../src/contexts/ThemeContext';
import { AuthProvider } from '../../src/contexts/AuthContext';
interface ExtendedRenderOptions extends Omit<RenderOptions, 'wrapper'> {
initialUser?: User | null;
theme?: 'light' | 'dark';
}
export function renderWithProviders(
ui: ReactElement,
{
initialUser = null,
theme = 'light',
...renderOptions
}: ExtendedRenderOptions = {}
) {
const queryClient = new QueryClient({
defaultOptions: {
queries: {
retry: false, // Disable retries in tests
},
},
});
function Wrapper({ children }: { children: React.ReactNode }) {
return (
<QueryClientProvider client={queryClient}>
<AuthProvider initialUser={initialUser}>
<ThemeProvider initialTheme={theme}>
{children}
</ThemeProvider>
</AuthProvider>
</QueryClientProvider>
);
}
return {
...render(ui, { wrapper: Wrapper, ...renderOptions }),
queryClient,
};
}
// Re-export everything from RTL
export * from '@testing-library/react';
export { renderWithProviders as render };
```
**Usage:**
```typescript
// __tests__/components/Dashboard.test.tsx
import { render, screen } from '../utils/renderWithProviders';
import { Dashboard } from '../../src/components/Dashboard';
import { createUser } from '../factories/userFactory';
describe('Dashboard', () => {
it('shows user greeting when authenticated', () => {
const user = createUser({ name: 'John Doe' });
render(<Dashboard />, { initialUser: user });
expect(screen.getByText('Hello, John Doe')).toBeInTheDocument();
});
it('shows login prompt when not authenticated', () => {
render(<Dashboard />, { initialUser: null });
expect(screen.getByText('Please log in')).toBeInTheDocument();
});
it('applies dark theme', () => {
render(<Dashboard />, { theme: 'dark' });
expect(document.body).toHaveClass('dark');
});
});
```
### Custom Matchers
```typescript
// __tests__/utils/customMatchers.ts
import { expect } from '@playwright/test';
expect.extend({
async toHaveLoadedSuccessfully(page) {
const hasNoErrors = await page.evaluate(() => {
return !document.querySelector('[data-error]');
});
const isLoaded = await page.evaluate(() => {
return document.readyState === 'complete';
});
return {
pass: hasNoErrors && isLoaded,
message: () =>
hasNoErrors
? 'Page loaded with errors'
: 'Page did not finish loading',
};
},
toBeWithinRange(received, floor, ceiling) {
const pass = received >= floor && received <= ceiling;
return {
pass,
message: () =>
`expected received ''to be within range floor - ceiling`,
};
},
});
// Type declarations
declare global {
namespace PlaywrightTest {
interface Matchers<R> {
toHaveLoadedSuccessfully(): Promise<R>;
}
}
}
```
---
## Async Testing Patterns
### Waiting for Elements
```typescript
// Preferred: Use findBy* (waits automatically)
const element = await screen.findByText('Loaded');
// Wait for element to appear
await waitFor(() => {
expect(screen.getByText('Loaded')).toBeInTheDocument();
});
// Wait for element to disappear
await waitForElementToBeRemoved(() => screen.queryByText('Loading...'));
// Wait with custom timeout
await waitFor(
() => {
expect(mockFn).toHaveBeenCalled();
},
{ timeout: 5000 }
);
```
### Testing Async State Changes
```typescript
// __tests__/components/AsyncButton.test.tsx
import { render, screen, waitFor } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import { AsyncButton } from '../../src/components/AsyncButton';
describe('AsyncButton', () => {
it('shows loading state during async operation', async () => {
const user = userEvent.setup();
const onClickMock = jest.fn().mockImplementation(
() => new Promise(resolve => setTimeout(resolve, 100))
);
render(<AsyncButton onClick={onClickMock}>Submit</AsyncButton>);
// Initial state
expect(screen.getByRole('button')).toHaveTextContent('Submit');
expect(screen.getByRole('button')).not.toBeDisabled();
// Click and verify loading state
await user.click(screen.getByRole('button'));
expect(screen.getByRole('button')).toHaveTextContent('Loading...');
expect(screen.getByRole('button')).toBeDisabled();
// Wait for completion
await waitFor(() => {
expect(screen.getByRole('button')).toHaveTextContent('Submit');
expect(screen.getByRole('button')).not.toBeDisabled();
});
});
});
```
### Testing Debounced/Throttled Functions
```typescript
// __tests__/components/SearchInput.test.tsx
import { render, screen, waitFor } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import { SearchInput } from '../../src/components/SearchInput';
// Use fake timers for debounce testing
jest.useFakeTimers();
describe('SearchInput', () => {
it('debounces search calls', async () => {
const user = userEvent.setup({ advanceTimers: jest.advanceTimersByTime });
const onSearchMock = jest.fn();
render(<SearchInput onSearch={onSearchMock} debounceMs={300} />);
// Type quickly
await user.type(screen.getByRole('textbox'), 'test');
// No calls yet (debouncing)
expect(onSearchMock).not.toHaveBeenCalled();
// Advance timers past debounce threshold
jest.advanceTimersByTime(300);
// Now it should be called once with final value
expect(onSearchMock).toHaveBeenCalledTimes(1);
expect(onSearchMock).toHaveBeenCalledWith('test');
});
});
```
### Playwright Async Patterns
```typescript
// e2e/async-patterns.spec.ts
import { test, expect } from '@playwright/test';
test('waits for API response', async ({ page }) => {
// Wait for specific response
const responsePromise = page.waitForResponse('/api/data');
await page.click('button.load-data');
const response = await responsePromise;
expect(response.status()).toBe(200);
});
test('waits for navigation', async ({ page }) => {
await page.goto('/');
await Promise.all([
page.waitForURL('/dashboard'),
page.click('a.dashboard-link'),
]);
});
test('waits for network idle', async ({ page }) => {
await page.goto('/', { waitUntil: 'networkidle' });
});
test('retries assertion until pass', async ({ page }) => {
// Auto-retrying assertion
await expect(page.locator('.counter')).toHaveText('10', { timeout: 5000 });
});
```
---
## Snapshot Testing Guidelines
### When to Use Snapshots
| Good Use Cases | Bad Use Cases |
|----------------|---------------|
| Static UI components | Dynamic content |
| Error messages | Timestamps/IDs |
| Configuration objects | Large component trees |
| Serializable data | Interactive components |
### Component Snapshots
```typescript
// __tests__/components/Button.test.tsx
import { render } from '@testing-library/react';
import { Button } from '../../src/components/Button';
describe('Button snapshots', () => {
it('renders primary variant', () => {
const { container } = render(
<Button variant="primary">Click me</Button>
);
expect(container.firstChild).toMatchSnapshot();
});
it('renders secondary variant', () => {
const { container } = render(
<Button variant="secondary">Click me</Button>
);
expect(container.firstChild).toMatchSnapshot();
});
it('renders disabled state', () => {
const { container } = render(
<Button disabled>Click me</Button>
);
expect(container.firstChild).toMatchSnapshot();
});
});
```
### Inline Snapshots
```typescript
// Good for small, stable outputs
it('formats date correctly', () => {
const result = formatDate(new Date('2024-01-15'));
expect(result).toMatchInlineSnapshot(`"January 15, 2024"`);
});
it('generates expected error message', () => {
const error = new ValidationError('email', 'Invalid format');
expect(error.message).toMatchInlineSnapshot(
`"Validation failed for 'email': Invalid format"`
);
});
```
### Snapshot Best Practices
1. **Keep snapshots small** - Snapshot specific elements, not entire pages
2. **Use inline snapshots for small outputs** - Easier to review in code
3. **Review snapshot changes carefully** - Don't blindly update
4. **Avoid snapshots for dynamic content** - Filter out timestamps, IDs
5. **Combine with other assertions** - Snapshots complement, not replace
```typescript
// Filtering dynamic content from snapshots
it('renders user card', () => {
const { container } = render(<UserCard user={mockUser} />);
// Remove dynamic elements before snapshot
const card = container.firstChild;
const timestamp = card.querySelector('.timestamp');
timestamp?.remove();
expect(card).toMatchSnapshot();
});
```
---
## Summary
1. **Use Page Objects** for complex, reusable page interactions
2. **Build factories** for consistent test data creation
3. **Leverage MSW** for realistic API mocking
4. **Create custom render utilities** for provider wrapping
5. **Master async patterns** to avoid flaky tests
6. **Use snapshots wisely** for stable, static content only
FILE:scripts/coverage_analyzer.py
#!/usr/bin/env python3
"""
Coverage Analyzer
Parses Jest/Istanbul coverage reports and identifies gaps, uncovered branches,
and provides actionable recommendations for improving test coverage.
Usage:
python coverage_analyzer.py coverage/coverage-final.json --threshold 80
python coverage_analyzer.py coverage/ --format html --output report.html
python coverage_analyzer.py coverage/ --critical-paths
"""
import os
import sys
import json
import argparse
import re
from pathlib import Path
from typing import Dict, List, Optional, Tuple, Any
from dataclasses import dataclass, field, asdict
from datetime import datetime
from collections import defaultdict
@dataclass
class FileCoverage:
"""Coverage data for a single file"""
path: str
statements: Tuple[int, int] # (covered, total)
branches: Tuple[int, int]
functions: Tuple[int, int]
lines: Tuple[int, int]
uncovered_lines: List[int] = field(default_factory=list)
uncovered_branches: List[str] = field(default_factory=list)
@property
def statement_pct(self) -> float:
return (self.statements[0] / self.statements[1] * 100) if self.statements[1] > 0 else 100
@property
def branch_pct(self) -> float:
return (self.branches[0] / self.branches[1] * 100) if self.branches[1] > 0 else 100
@property
def function_pct(self) -> float:
return (self.functions[0] / self.functions[1] * 100) if self.functions[1] > 0 else 100
@property
def line_pct(self) -> float:
return (self.lines[0] / self.lines[1] * 100) if self.lines[1] > 0 else 100
@dataclass
class CoverageGap:
"""An identified coverage gap"""
file: str
gap_type: str # 'statements', 'branches', 'functions', 'lines'
lines: List[int]
severity: str # 'critical', 'high', 'medium', 'low'
description: str
recommendation: str
@dataclass
class CoverageSummary:
"""Overall coverage summary"""
statements: Tuple[int, int]
branches: Tuple[int, int]
functions: Tuple[int, int]
lines: Tuple[int, int]
files_analyzed: int
files_below_threshold: int = 0
class CoverageParser:
"""Parses various coverage report formats"""
def __init__(self, verbose: bool = False):
self.verbose = verbose
def parse(self, path: Path) -> Tuple[Dict[str, FileCoverage], CoverageSummary]:
"""Parse coverage data from file or directory"""
if path.is_file():
if path.suffix == '.json':
return self._parse_istanbul_json(path)
elif path.suffix == '.info' or 'lcov' in path.name:
return self._parse_lcov(path)
elif path.is_dir():
# Look for common coverage files
for filename in ['coverage-final.json', 'coverage-summary.json', 'lcov.info']:
candidate = path / filename
if candidate.exists():
return self.parse(candidate)
# Check for coverage-final.json in coverage directory
coverage_json = path / 'coverage-final.json'
if coverage_json.exists():
return self._parse_istanbul_json(coverage_json)
raise ValueError(f"Could not find or parse coverage data at: {path}")
def _parse_istanbul_json(self, path: Path) -> Tuple[Dict[str, FileCoverage], CoverageSummary]:
"""Parse Istanbul/Jest JSON coverage format"""
with open(path, 'r') as f:
data = json.load(f)
files = {}
total_statements = [0, 0]
total_branches = [0, 0]
total_functions = [0, 0]
total_lines = [0, 0]
for file_path, file_data in data.items():
# Skip node_modules
if 'node_modules' in file_path:
continue
# Parse statement coverage
s_map = file_data.get('statementMap', {})
s_hits = file_data.get('s', {})
covered_statements = sum(1 for h in s_hits.values() if h > 0)
total_statements[0] += covered_statements
total_statements[1] += len(s_map)
# Parse branch coverage
b_map = file_data.get('branchMap', {})
b_hits = file_data.get('b', {})
covered_branches = sum(
sum(1 for h in hits if h > 0)
for hits in b_hits.values()
)
total_branch_count = sum(len(b['locations']) for b in b_map.values())
total_branches[0] += covered_branches
total_branches[1] += total_branch_count
# Parse function coverage
fn_map = file_data.get('fnMap', {})
fn_hits = file_data.get('f', {})
covered_functions = sum(1 for h in fn_hits.values() if h > 0)
total_functions[0] += covered_functions
total_functions[1] += len(fn_map)
# Determine uncovered lines
uncovered_lines = []
for stmt_id, hits in s_hits.items():
if hits == 0 and stmt_id in s_map:
stmt = s_map[stmt_id]
start_line = stmt.get('start', {}).get('line', 0)
if start_line not in uncovered_lines:
uncovered_lines.append(start_line)
# Count lines
line_coverage = self._calculate_line_coverage(s_map, s_hits)
total_lines[0] += line_coverage[0]
total_lines[1] += line_coverage[1]
# Identify uncovered branches
uncovered_branches = []
for branch_id, hits in b_hits.items():
for idx, hit in enumerate(hits):
if hit == 0:
uncovered_branches.append(f"{branch_id}:{idx}")
files[file_path] = FileCoverage(
path=file_path,
statements=(covered_statements, len(s_map)),
branches=(covered_branches, total_branch_count),
functions=(covered_functions, len(fn_map)),
lines=line_coverage,
uncovered_lines=sorted(uncovered_lines)[:50], # Limit
uncovered_branches=uncovered_branches[:20]
)
summary = CoverageSummary(
statements=tuple(total_statements),
branches=tuple(total_branches),
functions=tuple(total_functions),
lines=tuple(total_lines),
files_analyzed=len(files)
)
return files, summary
def _calculate_line_coverage(self, s_map: Dict, s_hits: Dict) -> Tuple[int, int]:
"""Calculate line coverage from statement data"""
lines = set()
covered_lines = set()
for stmt_id, stmt in s_map.items():
start_line = stmt.get('start', {}).get('line', 0)
end_line = stmt.get('end', {}).get('line', start_line)
for line in range(start_line, end_line + 1):
lines.add(line)
if s_hits.get(stmt_id, 0) > 0:
covered_lines.add(line)
return (len(covered_lines), len(lines))
def _parse_lcov(self, path: Path) -> Tuple[Dict[str, FileCoverage], CoverageSummary]:
"""Parse LCOV format coverage data"""
with open(path, 'r') as f:
content = f.read()
files = {}
current_file = None
current_data = {}
total = {
'statements': [0, 0],
'branches': [0, 0],
'functions': [0, 0],
'lines': [0, 0]
}
for line in content.split('\n'):
line = line.strip()
if line.startswith('SF:'):
current_file = line[3:]
current_data = {
'lines_hit': 0, 'lines_total': 0,
'functions_hit': 0, 'functions_total': 0,
'branches_hit': 0, 'branches_total': 0,
'uncovered_lines': []
}
elif line.startswith('DA:'):
parts = line[3:].split(',')
if len(parts) >= 2:
line_num = int(parts[0])
hits = int(parts[1])
current_data['lines_total'] += 1
if hits > 0:
current_data['lines_hit'] += 1
else:
current_data['uncovered_lines'].append(line_num)
elif line.startswith('FN:'):
current_data['functions_total'] += 1
elif line.startswith('FNDA:'):
parts = line[5:].split(',')
if len(parts) >= 1 and int(parts[0]) > 0:
current_data['functions_hit'] += 1
elif line.startswith('BRDA:'):
parts = line[5:].split(',')
current_data['branches_total'] += 1
if len(parts) >= 4 and parts[3] != '-' and int(parts[3]) > 0:
current_data['branches_hit'] += 1
elif line == 'end_of_record' and current_file:
# Skip node_modules
if 'node_modules' not in current_file:
files[current_file] = FileCoverage(
path=current_file,
statements=(current_data['lines_hit'], current_data['lines_total']),
branches=(current_data['branches_hit'], current_data['branches_total']),
functions=(current_data['functions_hit'], current_data['functions_total']),
lines=(current_data['lines_hit'], current_data['lines_total']),
uncovered_lines=current_data['uncovered_lines'][:50]
)
for key in total:
if key == 'statements' or key == 'lines':
total[key][0] += current_data['lines_hit']
total[key][1] += current_data['lines_total']
elif key == 'branches':
total[key][0] += current_data['branches_hit']
total[key][1] += current_data['branches_total']
elif key == 'functions':
total[key][0] += current_data['functions_hit']
total[key][1] += current_data['functions_total']
current_file = None
summary = CoverageSummary(
statements=tuple(total['statements']),
branches=tuple(total['branches']),
functions=tuple(total['functions']),
lines=tuple(total['lines']),
files_analyzed=len(files)
)
return files, summary
class CoverageAnalyzer:
"""Analyzes coverage data and generates recommendations"""
CRITICAL_PATTERNS = [
r'auth', r'payment', r'security', r'login', r'register',
r'checkout', r'order', r'transaction', r'billing'
]
SERVICE_PATTERNS = [
r'service', r'api', r'handler', r'controller', r'middleware'
]
def __init__(
self,
threshold: int = 80,
critical_paths: bool = False,
verbose: bool = False
):
self.threshold = threshold
self.critical_paths = critical_paths
self.verbose = verbose
def analyze(
self,
files: Dict[str, FileCoverage],
summary: CoverageSummary
) -> Tuple[List[CoverageGap], Dict[str, Any]]:
"""Analyze coverage and return gaps and recommendations"""
gaps = []
recommendations = {
'critical': [],
'high': [],
'medium': [],
'low': []
}
# Analyze each file
for file_path, coverage in files.items():
file_gaps = self._analyze_file(file_path, coverage)
gaps.extend(file_gaps)
# Sort gaps by severity
severity_order = {'critical': 0, 'high': 1, 'medium': 2, 'low': 3}
gaps.sort(key=lambda g: (severity_order[g.severity], -len(g.lines)))
# Generate recommendations
for gap in gaps:
recommendations[gap.severity].append({
'file': gap.file,
'type': gap.gap_type,
'lines': gap.lines[:10], # Limit
'description': gap.description,
'recommendation': gap.recommendation
})
# Add summary stats
stats = {
'overall_statement_pct': (summary.statements[0] / summary.statements[1] * 100) if summary.statements[1] > 0 else 100,
'overall_branch_pct': (summary.branches[0] / summary.branches[1] * 100) if summary.branches[1] > 0 else 100,
'overall_function_pct': (summary.functions[0] / summary.functions[1] * 100) if summary.functions[1] > 0 else 100,
'overall_line_pct': (summary.lines[0] / summary.lines[1] * 100) if summary.lines[1] > 0 else 100,
'files_analyzed': summary.files_analyzed,
'files_below_threshold': sum(
1 for f in files.values()
if f.line_pct < self.threshold
),
'total_gaps': len(gaps),
'critical_gaps': len(recommendations['critical']),
'threshold': self.threshold,
'meets_threshold': (summary.lines[0] / summary.lines[1] * 100) >= self.threshold if summary.lines[1] > 0 else True
}
return gaps, {
'recommendations': recommendations,
'stats': stats
}
def _analyze_file(self, file_path: str, coverage: FileCoverage) -> List[CoverageGap]:
"""Analyze a single file for coverage gaps"""
gaps = []
# Determine if file is critical
is_critical = any(
re.search(pattern, file_path.lower())
for pattern in self.CRITICAL_PATTERNS
)
is_service = any(
re.search(pattern, file_path.lower())
for pattern in self.SERVICE_PATTERNS
)
# Determine severity based on file type and coverage level
if is_critical:
base_severity = 'critical'
target_threshold = 95
elif is_service:
base_severity = 'high'
target_threshold = 85
else:
base_severity = 'medium'
target_threshold = self.threshold
# Check line coverage
if coverage.line_pct < target_threshold:
severity = base_severity if coverage.line_pct < 50 else self._lower_severity(base_severity)
gaps.append(CoverageGap(
file=file_path,
gap_type='lines',
lines=coverage.uncovered_lines[:20],
severity=severity,
description=f"Line coverage at {coverage.line_pct:.1f}% (target: {target_threshold}%)",
recommendation=self._get_line_recommendation(coverage)
))
# Check branch coverage
if coverage.branch_pct < target_threshold - 5: # Allow 5% less for branches
severity = base_severity if coverage.branch_pct < 40 else self._lower_severity(base_severity)
gaps.append(CoverageGap(
file=file_path,
gap_type='branches',
lines=[],
severity=severity,
description=f"Branch coverage at {coverage.branch_pct:.1f}%",
recommendation=f"Add tests for conditional logic. {len(coverage.uncovered_branches)} uncovered branches."
))
# Check function coverage
if coverage.function_pct < target_threshold:
severity = self._lower_severity(base_severity)
gaps.append(CoverageGap(
file=file_path,
gap_type='functions',
lines=[],
severity=severity,
description=f"Function coverage at {coverage.function_pct:.1f}%",
recommendation="Add tests for uncovered functions/methods."
))
return gaps
def _lower_severity(self, severity: str) -> str:
"""Lower severity by one level"""
mapping = {
'critical': 'high',
'high': 'medium',
'medium': 'low',
'low': 'low'
}
return mapping[severity]
def _get_line_recommendation(self, coverage: FileCoverage) -> str:
"""Generate recommendation for line coverage gaps"""
if coverage.line_pct < 30:
return "This file has very low coverage. Consider adding basic render/unit tests first."
elif coverage.line_pct < 60:
return "Add tests covering the main functionality and happy paths."
else:
return "Focus on edge cases and error handling paths."
class ReportGenerator:
"""Generates coverage reports in various formats"""
def __init__(self, verbose: bool = False):
self.verbose = verbose
def generate_text_report(
self,
files: Dict[str, FileCoverage],
summary: CoverageSummary,
analysis: Dict[str, Any],
threshold: int
) -> str:
"""Generate a text report"""
lines = []
# Header
lines.append("=" * 60)
lines.append("COVERAGE ANALYSIS REPORT")
lines.append(f"Generated: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
lines.append("=" * 60)
lines.append("")
# Overall summary
stats = analysis['stats']
lines.append("OVERALL COVERAGE:")
lines.append(f" Statements: {stats['overall_statement_pct']:.1f}%")
lines.append(f" Branches: {stats['overall_branch_pct']:.1f}%")
lines.append(f" Functions: {stats['overall_function_pct']:.1f}%")
lines.append(f" Lines: {stats['overall_line_pct']:.1f}%")
lines.append("")
# Threshold check
threshold_status = "PASS" if stats['meets_threshold'] else "FAIL"
lines.append(f"Threshold ({threshold}%): {threshold_status}")
lines.append(f"Files analyzed: {stats['files_analyzed']}")
lines.append(f"Files below threshold: {stats['files_below_threshold']}")
lines.append("")
# Critical gaps
recs = analysis['recommendations']
if recs['critical']:
lines.append("-" * 60)
lines.append("CRITICAL GAPS (requires immediate attention):")
for rec in recs['critical'][:5]:
lines.append(f" - {rec['file']}")
lines.append(f" {rec['description']}")
if rec['lines']:
lines.append(f" Uncovered lines: {', '.join(map(str, rec['lines'][:5]))}")
lines.append("")
# High priority gaps
if recs['high']:
lines.append("-" * 60)
lines.append("HIGH PRIORITY GAPS:")
for rec in recs['high'][:5]:
lines.append(f" - {rec['file']}")
lines.append(f" {rec['description']}")
lines.append("")
# Files below threshold
below_threshold = [
(path, cov) for path, cov in files.items()
if cov.line_pct < threshold
]
below_threshold.sort(key=lambda x: x[1].line_pct)
if below_threshold:
lines.append("-" * 60)
lines.append(f"FILES BELOW {threshold}% THRESHOLD:")
for path, cov in below_threshold[:10]:
short_path = path.split('/')[-1] if '/' in path else path
lines.append(f" {cov.line_pct:5.1f}% {short_path}")
if len(below_threshold) > 10:
lines.append(f" ... and {len(below_threshold) - 10} more files")
lines.append("")
# Recommendations
lines.append("-" * 60)
lines.append("RECOMMENDATIONS:")
all_recs = (
recs['critical'][:2] + recs['high'][:2] + recs['medium'][:2]
)
for i, rec in enumerate(all_recs[:5], 1):
lines.append(f" {i}. {rec['recommendation']}")
lines.append(f" File: {rec['file']}")
lines.append("")
lines.append("=" * 60)
return '\n'.join(lines)
def generate_html_report(
self,
files: Dict[str, FileCoverage],
summary: CoverageSummary,
analysis: Dict[str, Any],
threshold: int
) -> str:
"""Generate an HTML report"""
stats = analysis['stats']
recs = analysis['recommendations']
html = f"""<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Coverage Analysis Report</title>
<style>
body {{ font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; margin: 40px; }}
h1 {{ color: #333; }}
.summary {{ display: grid; grid-template-columns: repeat(4, 1fr); gap: 20px; margin: 20px 0; }}
.stat {{ background: #f5f5f5; padding: 20px; border-radius: 8px; text-align: center; }}
.stat-value {{ font-size: 2em; font-weight: bold; }}
.pass {{ color: #22c55e; }}
.fail {{ color: #ef4444; }}
.warn {{ color: #f59e0b; }}
table {{ width: 100%; border-collapse: collapse; margin: 20px 0; }}
th, td {{ padding: 12px; text-align: left; border-bottom: 1px solid #ddd; }}
th {{ background: #f5f5f5; }}
.gap-critical {{ background: #fef2f2; }}
.gap-high {{ background: #fffbeb; }}
.progress {{ background: #e5e7eb; border-radius: 4px; height: 8px; }}
.progress-bar {{ height: 100%; border-radius: 4px; }}
</style>
</head>
<body>
<h1>Coverage Analysis Report</h1>
<p>Generated: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}</p>
<div class="summary">
<div class="stat">
<div class="stat-value {'pass' if stats['overall_statement_pct'] >= threshold else 'fail'}">{stats['overall_statement_pct']:.1f}%</div>
<div>Statements</div>
</div>
<div class="stat">
<div class="stat-value {'pass' if stats['overall_branch_pct'] >= threshold - 5 else 'fail'}">{stats['overall_branch_pct']:.1f}%</div>
<div>Branches</div>
</div>
<div class="stat">
<div class="stat-value {'pass' if stats['overall_function_pct'] >= threshold else 'fail'}">{stats['overall_function_pct']:.1f}%</div>
<div>Functions</div>
</div>
<div class="stat">
<div class="stat-value {'pass' if stats['overall_line_pct'] >= threshold else 'fail'}">{stats['overall_line_pct']:.1f}%</div>
<div>Lines</div>
</div>
</div>
<h2>Threshold Status: <span class="{'pass' if stats['meets_threshold'] else 'fail'}">{'PASS' if stats['meets_threshold'] else 'FAIL'}</span></h2>
<p>Target: {threshold}% | Files Analyzed: {stats['files_analyzed']} | Below Threshold: {stats['files_below_threshold']}</p>
<h2>Coverage Gaps</h2>
<table>
<thead>
<tr>
<th>Severity</th>
<th>File</th>
<th>Issue</th>
<th>Recommendation</th>
</tr>
</thead>
<tbody>
"""
# Add gaps to table
all_gaps = (
[(g, 'critical') for g in recs['critical']] +
[(g, 'high') for g in recs['high']] +
[(g, 'medium') for g in recs['medium'][:5]]
)
for gap, severity in all_gaps[:15]:
row_class = f"gap-{severity}" if severity in ['critical', 'high'] else ""
html += f""" <tr class="{row_class}">
<td>{severity.upper()}</td>
<td>{gap['file'].split('/')[-1]}</td>
<td>{gap['description']}</td>
<td>{gap['recommendation']}</td>
</tr>
"""
html += """ </tbody>
</table>
<h2>File Coverage Details</h2>
<table>
<thead>
<tr>
<th>File</th>
<th>Statements</th>
<th>Branches</th>
<th>Functions</th>
<th>Lines</th>
</tr>
</thead>
<tbody>
"""
# Sort files by line coverage
sorted_files = sorted(files.items(), key=lambda x: x[1].line_pct)
for path, cov in sorted_files[:20]:
short_path = path.split('/')[-1] if '/' in path else path
html += f""" <tr>
<td>{short_path}</td>
<td>{cov.statement_pct:.1f}%</td>
<td>{cov.branch_pct:.1f}%</td>
<td>{cov.function_pct:.1f}%</td>
<td>{cov.line_pct:.1f}%</td>
</tr>
"""
html += """ </tbody>
</table>
</body>
</html>
"""
return html
class CoverageAnalyzerTool:
"""Main tool class"""
def __init__(
self,
coverage_path: str,
threshold: int = 80,
critical_paths: bool = False,
strict: bool = False,
output_format: str = 'text',
output_path: Optional[str] = None,
verbose: bool = False
):
self.coverage_path = Path(coverage_path)
self.threshold = threshold
self.critical_paths = critical_paths
self.strict = strict
self.output_format = output_format
self.output_path = output_path
self.verbose = verbose
def run(self) -> Dict[str, Any]:
"""Run the coverage analysis"""
print(f"Analyzing coverage from: {self.coverage_path}")
# Parse coverage data
parser = CoverageParser(self.verbose)
files, summary = parser.parse(self.coverage_path)
print(f"Found coverage data for {len(files)} files")
# Analyze coverage
analyzer = CoverageAnalyzer(
threshold=self.threshold,
critical_paths=self.critical_paths,
verbose=self.verbose
)
gaps, analysis = analyzer.analyze(files, summary)
# Generate report
reporter = ReportGenerator(self.verbose)
if self.output_format == 'html':
report = reporter.generate_html_report(files, summary, analysis, self.threshold)
else:
report = reporter.generate_text_report(files, summary, analysis, self.threshold)
# Output report
if self.output_path:
with open(self.output_path, 'w') as f:
f.write(report)
print(f"Report written to: {self.output_path}")
else:
print(report)
# Return results
results = {
'status': 'pass' if analysis['stats']['meets_threshold'] else 'fail',
'threshold': self.threshold,
'coverage': {
'statements': analysis['stats']['overall_statement_pct'],
'branches': analysis['stats']['overall_branch_pct'],
'functions': analysis['stats']['overall_function_pct'],
'lines': analysis['stats']['overall_line_pct']
},
'files_analyzed': summary.files_analyzed,
'files_below_threshold': analysis['stats']['files_below_threshold'],
'total_gaps': analysis['stats']['total_gaps'],
'critical_gaps': analysis['stats']['critical_gaps']
}
# Exit with error if strict mode and below threshold
if self.strict and not analysis['stats']['meets_threshold']:
print(f"\nFailed: Coverage {analysis['stats']['overall_line_pct']:.1f}% below threshold {self.threshold}%")
sys.exit(1)
return results
def main():
"""Main entry point"""
parser = argparse.ArgumentParser(
description="Analyze Jest/Istanbul coverage reports and identify gaps",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
# Basic analysis
python coverage_analyzer.py coverage/coverage-final.json
# With threshold enforcement
python coverage_analyzer.py coverage/ --threshold 80 --strict
# Generate HTML report
python coverage_analyzer.py coverage/ --format html --output report.html
# Focus on critical paths
python coverage_analyzer.py coverage/ --critical-paths
"""
)
parser.add_argument(
'coverage',
help='Path to coverage file or directory'
)
parser.add_argument(
'--threshold', '-t',
type=int,
default=80,
help='Coverage threshold percentage (default: 80)'
)
parser.add_argument(
'--strict',
action='store_true',
help='Exit with error if coverage is below threshold'
)
parser.add_argument(
'--critical-paths',
action='store_true',
help='Focus analysis on critical business paths'
)
parser.add_argument(
'--format', '-f',
choices=['text', 'html', 'json'],
default='text',
help='Output format (default: text)'
)
parser.add_argument(
'--output', '-o',
help='Output file path'
)
parser.add_argument(
'--verbose', '-v',
action='store_true',
help='Enable verbose output'
)
parser.add_argument(
'--json',
action='store_true',
help='Output results as JSON (summary only)'
)
args = parser.parse_args()
try:
tool = CoverageAnalyzerTool(
coverage_path=args.coverage,
threshold=args.threshold,
critical_paths=args.critical_paths,
strict=args.strict,
output_format=args.format,
output_path=args.output,
verbose=args.verbose
)
results = tool.run()
if args.json:
print(json.dumps(results, indent=2))
except Exception as e:
print(f"Error: {e}")
if args.verbose:
import traceback
traceback.print_exc()
sys.exit(1)
if __name__ == '__main__':
main()
FILE:scripts/e2e_test_scaffolder.py
#!/usr/bin/env python3
"""
E2E Test Scaffolder
Scans Next.js pages/app directory and generates Playwright test files
with common interactions, Page Object Model classes, and configuration.
Usage:
python e2e_test_scaffolder.py src/app/ --output e2e/
python e2e_test_scaffolder.py pages/ --include-pom --routes "/login,/dashboard"
"""
import os
import sys
import json
import argparse
import re
from pathlib import Path
from typing import Dict, List, Optional, Tuple, Set
from dataclasses import dataclass, field, asdict
from datetime import datetime
@dataclass
class RouteInfo:
"""Information about a detected route"""
path: str # URL path e.g., /dashboard
file_path: str # File system path
route_type: str # 'page', 'layout', 'api', 'dynamic'
has_params: bool
params: List[str]
has_form: bool
has_auth: bool
interactions: List[str]
@dataclass
class TestSpec:
"""A Playwright test specification"""
route: RouteInfo
test_cases: List[str]
imports: Set[str] = field(default_factory=set)
@dataclass
class PageObject:
"""Page Object Model class definition"""
name: str
route: str
locators: List[Tuple[str, str, str]] # (name, selector, description)
methods: List[Tuple[str, str]] # (name, code)
class RouteScanner:
"""Scans Next.js directories for routes"""
# Pattern to detect page files
PAGE_PATTERNS = {
'page.tsx', 'page.ts', 'page.jsx', 'page.js', # App Router
'index.tsx', 'index.ts', 'index.jsx', 'index.js' # Pages Router
}
# Patterns indicating specific features
FORM_PATTERNS = [
r'<form', r'handleSubmit', r'onSubmit', r'useForm',
r'<input', r'<textarea', r'<select'
]
AUTH_PATTERNS = [
r'auth', r'login', r'signin', r'signup', r'register',
r'useAuth', r'useSession', r'getServerSession', r'withAuth'
]
INTERACTION_PATTERNS = {
'click': r'onClick|button|Button|<a\s|Link',
'type': r'<input|<textarea|onChange',
'select': r'<select|Dropdown|Select',
'navigation': r'useRouter|router\.push|Link',
'modal': r'Modal|Dialog|isOpen|onClose',
'toggle': r'toggle|Switch|Checkbox',
'upload': r'<input.*type=["\']file|upload|dropzone'
}
def __init__(self, source_path: Path, verbose: bool = False):
self.source_path = source_path
self.verbose = verbose
self.routes: List[RouteInfo] = []
self.is_app_router = self._detect_router_type()
def _detect_router_type(self) -> bool:
"""Detect if using App Router or Pages Router"""
# App Router: has 'app' directory with page.tsx files
# Pages Router: has 'pages' directory with index.tsx files
app_dir = self.source_path / 'app'
if app_dir.exists() and list(app_dir.rglob('page.*')):
return True
return 'app' in str(self.source_path).lower()
def scan(self, filter_routes: Optional[List[str]] = None) -> List[RouteInfo]:
"""Scan for all routes"""
self._scan_directory(self.source_path)
# Filter if specific routes requested
if filter_routes:
self.routes = [
r for r in self.routes
if any(fr in r.path for fr in filter_routes)
]
return self.routes
def _scan_directory(self, directory: Path, url_path: str = ''):
"""Recursively scan directory for routes"""
if not directory.exists():
return
for item in directory.iterdir():
if item.name.startswith('.') or item.name == 'node_modules':
continue
if item.is_dir():
# Handle route groups (parentheses) and dynamic routes
dir_name = item.name
if dir_name.startswith('(') and dir_name.endswith(')'):
# Route group - doesn't add to URL path
self._scan_directory(item, url_path)
elif dir_name.startswith('[') and dir_name.endswith(']'):
# Dynamic route
param_name = dir_name[1:-1]
if param_name.startswith('...'):
# Catch-all route
new_path = f"{url_path}/[...{param_name[3:]}]"
else:
new_path = f"{url_path}/[{param_name}]"
self._scan_directory(item, new_path)
elif dir_name == 'api':
# API routes - scan but mark differently
self._scan_api_directory(item, '/api')
else:
new_path = f"{url_path}/{dir_name}"
self._scan_directory(item, new_path)
elif item.is_file():
self._process_file(item, url_path)
def _process_file(self, file_path: Path, url_path: str):
"""Process a potential page file"""
if file_path.name not in self.PAGE_PATTERNS:
return
# Skip if it's a layout or other special file
if any(x in file_path.name for x in ['layout', 'loading', 'error', 'template']):
return
try:
content = file_path.read_text(encoding='utf-8')
except Exception:
return
# Determine route path
if url_path == '':
route_path = '/'
else:
route_path = url_path
# Detect dynamic parameters
params = re.findall(r'\[([^\]]+)\]', route_path)
has_params = len(params) > 0
# Detect features
has_form = any(re.search(p, content) for p in self.FORM_PATTERNS)
has_auth = any(re.search(p, content, re.IGNORECASE) for p in self.AUTH_PATTERNS)
# Detect interactions
interactions = []
for interaction, pattern in self.INTERACTION_PATTERNS.items():
if re.search(pattern, content):
interactions.append(interaction)
route = RouteInfo(
path=route_path,
file_path=str(file_path),
route_type='dynamic' if has_params else 'page',
has_params=has_params,
params=params,
has_form=has_form,
has_auth=has_auth,
interactions=interactions
)
self.routes.append(route)
if self.verbose:
print(f" Found route: {route_path}")
def _scan_api_directory(self, directory: Path, url_path: str):
"""Scan API routes (mark them differently)"""
for item in directory.iterdir():
if item.is_dir():
new_path = f"{url_path}/{item.name}"
self._scan_api_directory(item, new_path)
elif item.is_file() and item.suffix in {'.ts', '.tsx', '.js', '.jsx'}:
# API routes don't get E2E tests typically
pass
class TestGenerator:
"""Generates Playwright test files"""
def __init__(self, include_pom: bool = False, verbose: bool = False):
self.include_pom = include_pom
self.verbose = verbose
def generate(self, route: RouteInfo) -> str:
"""Generate a test file for a route"""
lines = []
# Imports
lines.append("import { test, expect } from '@playwright/test';")
if self.include_pom:
page_class = self._get_page_class_name(route.path)
lines.append(f"import {{ {page_class} }} from './pages/{page_class}';")
lines.append('')
# Test describe block
route_name = route.path if route.path != '/' else 'Home'
lines.append(f"test.describe('{route_name}', () => {{")
# Generate test cases based on route features
test_cases = self._generate_test_cases(route)
for test_case in test_cases:
lines.append('')
lines.append(test_case)
lines.append('});')
lines.append('')
return '\n'.join(lines)
def _generate_test_cases(self, route: RouteInfo) -> List[str]:
"""Generate test cases based on route features"""
cases = []
url = self._get_test_url(route)
# Basic navigation test
cases.append(f''' test('loads successfully', async ({{ page }}) => {{
await page.goto('{url}');
await expect(page).toHaveURL(/{re.escape(route.path.replace('[', '').replace(']', '.*'))}/);
// TODO: Add specific content assertions
}});''')
# Page title test
cases.append(f''' test('has correct title', async ({{ page }}) => {{
await page.goto('{url}');
// TODO: Update expected title
await expect(page).toHaveTitle(/.*/);
}});''')
# Auth-related tests
if route.has_auth:
cases.append(f''' test('redirects unauthenticated users', async ({{ page }}) => {{
await page.goto('{url}');
// TODO: Verify redirect to login
// await expect(page).toHaveURL('/login');
}});
test('allows authenticated access', async ({{ page }}) => {{
// TODO: Set up authentication
// await page.context().addCookies([{{ name: 'session', value: '...' }}]);
await page.goto('{url}');
await expect(page).toHaveURL(/{re.escape(route.path.replace('[', '').replace(']', '.*'))}/);
}});''')
# Form tests
if route.has_form:
cases.append(f''' test('form submission works', async ({{ page }}) => {{
await page.goto('{url}');
// TODO: Fill in form fields
// await page.getByLabel('Email').fill('test@example.com');
// await page.getByLabel('Password').fill('password123');
// Submit form
// await page.getByRole('button', {{ name: 'Submit' }}).click();
// TODO: Assert success state
// await expect(page.getByText('Success')).toBeVisible();
}});
test('shows validation errors', async ({{ page }}) => {{
await page.goto('{url}');
// Submit without filling required fields
await page.getByRole('button', {{ name: /submit/i }}).click();
// TODO: Assert validation errors shown
// await expect(page.getByText('Required')).toBeVisible();
}});''')
# Click interaction tests
if 'click' in route.interactions:
cases.append(f''' test('button interactions work', async ({{ page }}) => {{
await page.goto('{url}');
// TODO: Find and click interactive elements
// const button = page.getByRole('button', {{ name: '...' }});
// await button.click();
// await expect(page.getByText('...')).toBeVisible();
}});''')
# Navigation tests
if 'navigation' in route.interactions:
cases.append(f''' test('navigation works correctly', async ({{ page }}) => {{
await page.goto('{url}');
// TODO: Click navigation links
// await page.getByRole('link', {{ name: '...' }}).click();
// await expect(page).toHaveURL('...');
}});''')
# Modal tests
if 'modal' in route.interactions:
cases.append(f''' test('modal opens and closes', async ({{ page }}) => {{
await page.goto('{url}');
// TODO: Open modal
// await page.getByRole('button', {{ name: 'Open' }}).click();
// await expect(page.getByRole('dialog')).toBeVisible();
// TODO: Close modal
// await page.getByRole('button', {{ name: 'Close' }}).click();
// await expect(page.getByRole('dialog')).not.toBeVisible();
}});''')
# Dynamic route test
if route.has_params:
cases.append(f''' test('handles dynamic parameters', async ({{ page }}) => {{
// TODO: Test with different parameter values
await page.goto('{url}');
await expect(page.locator('body')).toBeVisible();
}});''')
return cases
def _get_test_url(self, route: RouteInfo) -> str:
"""Get a testable URL for the route"""
url = route.path
# Replace dynamic segments with example values
for param in route.params:
if param.startswith('...'):
url = url.replace(f'[...{param[3:]}]', 'example/path')
else:
url = url.replace(f'[{param}]', 'test-id')
return url
def _get_page_class_name(self, route_path: str) -> str:
"""Get Page Object class name from route path"""
if route_path == '/':
return 'HomePage'
# Remove leading slash and convert to PascalCase
name = route_path.strip('/')
name = re.sub(r'\[.*?\]', '', name) # Remove dynamic segments
parts = name.split('/')
return ''.join(p.title() for p in parts if p) + 'Page'
class PageObjectGenerator:
"""Generates Page Object Model classes"""
def __init__(self, verbose: bool = False):
self.verbose = verbose
def generate(self, route: RouteInfo) -> str:
"""Generate a Page Object class for a route"""
class_name = self._get_class_name(route.path)
url = route.path
# Replace dynamic segments
for param in route.params:
url = url.replace(f'[{param}]', f'{{param}}')
lines = []
# Imports
lines.append("import { Page, Locator, expect } from '@playwright/test';")
lines.append('')
# Class definition
lines.append(f"export class {class_name} {{")
lines.append(" readonly page: Page;")
# Common locators
locators = self._get_locators(route)
for name, selector, _ in locators:
lines.append(f" readonly {name}: Locator;")
lines.append('')
# Constructor
lines.append(" constructor(page: Page) {")
lines.append(" this.page = page;")
for name, selector, _ in locators:
lines.append(f" this.{name} = page.{selector};")
lines.append(" }")
lines.append('')
# Navigation method
if route.has_params:
param_args = ', '.join(f'{p}: string' for p in route.params)
url_parts = url.split('/')
url_template = '/'.join(
f'{{p}}' if f'{{p}}' in part else part
for p, part in zip(route.params, url_parts)
)
lines.append(f" async goto({param_args}) {{")
lines.append(f" await this.page.goto(`{url_template}`);")
else:
lines.append(" async goto() {")
lines.append(f" await this.page.goto('{route.path}');")
lines.append(" }")
lines.append('')
# Add methods based on features
methods = self._get_methods(route, locators)
for method_name, method_code in methods:
lines.append(method_code)
lines.append('')
lines.append('}')
lines.append('')
return '\n'.join(lines)
def _get_class_name(self, route_path: str) -> str:
"""Get class name from route path"""
if route_path == '/':
return 'HomePage'
name = route_path.strip('/')
name = re.sub(r'\[.*?\]', '', name)
parts = name.split('/')
return ''.join(p.title() for p in parts if p) + 'Page'
def _get_locators(self, route: RouteInfo) -> List[Tuple[str, str, str]]:
"""Get common locators for a page"""
locators = []
# Always add a heading locator
locators.append(('heading', "getByRole('heading', { level: 1 })", 'Main heading'))
if route.has_form:
locators.extend([
('submitButton', "getByRole('button', { name: /submit/i })", 'Form submit button'),
('form', "locator('form')", 'Main form element'),
])
if route.has_auth:
locators.extend([
('emailInput', "getByLabel('Email')", 'Email input field'),
('passwordInput', "getByLabel('Password')", 'Password input field'),
])
if 'navigation' in route.interactions:
locators.append(('navLinks', "getByRole('navigation').getByRole('link')", 'Navigation links'))
if 'modal' in route.interactions:
locators.append(('modal', "getByRole('dialog')", 'Modal dialog'))
return locators
def _get_methods(
self,
route: RouteInfo,
locators: List[Tuple[str, str, str]]
) -> List[Tuple[str, str]]:
"""Get methods for the page object"""
methods = []
# Wait for load method
methods.append(('waitForLoad', ''' async waitForLoad() {
await expect(this.heading).toBeVisible();
}'''))
if route.has_form:
methods.append(('submitForm', ''' async submitForm() {
await this.submitButton.click();
}'''))
if route.has_auth:
methods.append(('login', ''' async login(email: string, password: string) {
await this.emailInput.fill(email);
await this.passwordInput.fill(password);
await this.submitButton.click();
}'''))
if 'modal' in route.interactions:
methods.append(('waitForModal', ''' async waitForModal() {
await expect(this.modal).toBeVisible();
}'''))
methods.append(('closeModal', ''' async closeModal() {
await this.page.keyboard.press('Escape');
await expect(this.modal).not.toBeVisible();
}'''))
return methods
class ConfigGenerator:
"""Generates Playwright configuration"""
def generate_config(self) -> str:
"""Generate playwright.config.ts"""
return '''import { defineConfig, devices } from '@playwright/test';
/**
* Playwright Test Configuration
* @see https://playwright.dev/docs/test-configuration
*/
export default defineConfig({
testDir: './e2e',
fullyParallel: true,
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 2 : 0,
workers: process.env.CI ? 1 : undefined,
reporter: [
['html', { open: 'never' }],
['list'],
],
use: {
baseURL: process.env.BASE_URL || 'http://localhost:3000',
trace: 'on-first-retry',
screenshot: 'only-on-failure',
},
projects: [
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] },
},
{
name: 'firefox',
use: { ...devices['Desktop Firefox'] },
},
{
name: 'webkit',
use: { ...devices['Desktop Safari'] },
},
{
name: 'Mobile Chrome',
use: { ...devices['Pixel 5'] },
},
],
webServer: {
command: 'npm run dev',
url: 'http://localhost:3000',
reuseExistingServer: !process.env.CI,
timeout: 120 * 1000,
},
});
'''
def generate_auth_fixture(self) -> str:
"""Generate authentication fixture"""
return '''import { test as base, Page } from '@playwright/test';
interface AuthFixtures {
authenticatedPage: Page;
}
export const test = base.extend<AuthFixtures>({
authenticatedPage: async ({ page }, use) => {
// Option 1: Login via UI
// await page.goto('/login');
// await page.getByLabel('Email').fill(process.env.TEST_EMAIL || 'test@example.com');
// await page.getByLabel('Password').fill(process.env.TEST_PASSWORD || 'password');
// await page.getByRole('button', { name: 'Sign in' }).click();
// await page.waitForURL('/dashboard');
// Option 2: Login via API
// const response = await page.request.post('/api/auth/login', {
// data: {
// email: process.env.TEST_EMAIL,
// password: process.env.TEST_PASSWORD,
// },
// });
// const { token } = await response.json();
// await page.context().addCookies([
// { name: 'auth-token', value: token, domain: 'localhost', path: '/' }
// ]);
await use(page);
},
});
export { expect } from '@playwright/test';
'''
class E2ETestScaffolder:
"""Main scaffolder class"""
def __init__(
self,
source_path: str,
output_path: Optional[str] = None,
include_pom: bool = False,
routes: Optional[str] = None,
verbose: bool = False
):
self.source_path = Path(source_path)
self.output_path = Path(output_path) if output_path else Path('e2e')
self.include_pom = include_pom
self.routes_filter = routes.split(',') if routes else None
self.verbose = verbose
self.results = {
'status': 'success',
'source': str(self.source_path),
'routes': [],
'generated_files': [],
'summary': {}
}
def run(self) -> Dict:
"""Run the scaffolder"""
print(f"Scanning: {self.source_path}")
# Validate source path
if not self.source_path.exists():
raise ValueError(f"Source path does not exist: {self.source_path}")
# Scan for routes
scanner = RouteScanner(self.source_path, self.verbose)
routes = scanner.scan(self.routes_filter)
print(f"Found {len(routes)} routes")
# Create output directories
self.output_path.mkdir(parents=True, exist_ok=True)
if self.include_pom:
(self.output_path / 'pages').mkdir(exist_ok=True)
# Generate test files
test_generator = TestGenerator(self.include_pom, self.verbose)
pom_generator = PageObjectGenerator(self.verbose) if self.include_pom else None
config_generator = ConfigGenerator()
# Generate tests for each route
for route in routes:
# Generate test file
test_content = test_generator.generate(route)
test_filename = self._get_test_filename(route.path)
test_path = self.output_path / test_filename
test_path.write_text(test_content, encoding='utf-8')
self.results['generated_files'].append({
'type': 'test',
'route': route.path,
'path': str(test_path)
})
print(f" {test_filename}")
# Generate Page Object if enabled
if self.include_pom:
pom_content = pom_generator.generate(route)
pom_filename = self._get_pom_filename(route.path)
pom_path = self.output_path / 'pages' / pom_filename
pom_path.write_text(pom_content, encoding='utf-8')
self.results['generated_files'].append({
'type': 'page_object',
'route': route.path,
'path': str(pom_path)
})
print(f" pages/{pom_filename}")
# Generate config files if not exists
config_path = Path('playwright.config.ts')
if not config_path.exists():
config_content = config_generator.generate_config()
config_path.write_text(config_content, encoding='utf-8')
self.results['generated_files'].append({
'type': 'config',
'path': str(config_path)
})
print(f" playwright.config.ts")
# Generate auth fixture
fixtures_dir = self.output_path / 'fixtures'
fixtures_dir.mkdir(exist_ok=True)
auth_fixture_path = fixtures_dir / 'auth.ts'
if not auth_fixture_path.exists():
auth_content = config_generator.generate_auth_fixture()
auth_fixture_path.write_text(auth_content, encoding='utf-8')
self.results['generated_files'].append({
'type': 'fixture',
'path': str(auth_fixture_path)
})
print(f" fixtures/auth.ts")
# Store route info
self.results['routes'] = [asdict(r) for r in routes]
# Summary
self.results['summary'] = {
'total_routes': len(routes),
'total_files': len(self.results['generated_files']),
'output_directory': str(self.output_path),
'include_pom': self.include_pom
}
print('')
print(f"Summary: {len(routes)} routes, {len(self.results['generated_files'])} files generated")
return self.results
def _get_test_filename(self, route_path: str) -> str:
"""Get test filename from route path"""
if route_path == '/':
return 'home.spec.ts'
name = route_path.strip('/')
name = re.sub(r'\[([^\]]+)\]', r'\1', name) # [id] -> id
name = name.replace('/', '-')
return f"{name}.spec.ts"
def _get_pom_filename(self, route_path: str) -> str:
"""Get Page Object filename from route path"""
if route_path == '/':
return 'HomePage.ts'
name = route_path.strip('/')
name = re.sub(r'\[.*?\]', '', name)
parts = name.split('/')
class_name = ''.join(p.title() for p in parts if p) + 'Page'
return f"{class_name}.ts"
def main():
"""Main entry point"""
parser = argparse.ArgumentParser(
description="Generate Playwright E2E tests from Next.js routes",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
# Scaffold E2E tests for App Router
python e2e_test_scaffolder.py src/app/ --output e2e/
# Include Page Object Models
python e2e_test_scaffolder.py src/app/ --include-pom
# Generate for specific routes only
python e2e_test_scaffolder.py src/app/ --routes "/login,/dashboard,/checkout"
# Verbose output
python e2e_test_scaffolder.py pages/ -v
"""
)
parser.add_argument(
'source',
help='Source directory (app/ or pages/)'
)
parser.add_argument(
'--output', '-o',
default='e2e',
help='Output directory for test files (default: e2e/)'
)
parser.add_argument(
'--include-pom',
action='store_true',
help='Generate Page Object Model classes'
)
parser.add_argument(
'--routes',
help='Comma-separated list of routes to generate tests for'
)
parser.add_argument(
'--verbose', '-v',
action='store_true',
help='Enable verbose output'
)
parser.add_argument(
'--json',
action='store_true',
help='Output results as JSON'
)
args = parser.parse_args()
try:
scaffolder = E2ETestScaffolder(
source_path=args.source,
output_path=args.output,
include_pom=args.include_pom,
routes=args.routes,
verbose=args.verbose
)
results = scaffolder.run()
if args.json:
print(json.dumps(results, indent=2))
except Exception as e:
print(f"Error: {e}")
sys.exit(1)
if __name__ == '__main__':
main()
FILE:scripts/test_suite_generator.py
#!/usr/bin/env python3
"""
Test Suite Generator
Scans React/TypeScript components and generates Jest + React Testing Library
test stubs with proper structure, accessibility tests, and common patterns.
Usage:
python test_suite_generator.py src/components/ --output __tests__/
python test_suite_generator.py src/ --include-a11y --scan-only
"""
import os
import sys
import json
import argparse
import re
from pathlib import Path
from typing import Dict, List, Optional, Tuple, Set
from dataclasses import dataclass, field, asdict
from datetime import datetime
@dataclass
class ComponentInfo:
"""Information about a detected React component"""
name: str
file_path: str
component_type: str # 'functional', 'class', 'forwardRef', 'memo'
has_props: bool
props: List[str]
has_hooks: List[str]
has_context: bool
has_effects: bool
has_state: bool
has_callbacks: bool
exports: List[str]
imports: List[str]
@dataclass
class TestCase:
"""A single test case to generate"""
name: str
description: str
test_type: str # 'render', 'interaction', 'a11y', 'props', 'state'
code: str
@dataclass
class TestFile:
"""A complete test file to generate"""
component: ComponentInfo
test_cases: List[TestCase] = field(default_factory=list)
imports: Set[str] = field(default_factory=set)
class ComponentScanner:
"""Scans source files for React components"""
# Patterns for detecting React components
FUNCTIONAL_COMPONENT = re.compile(
r'^(?:export\s+)?(?:const|function)\s+([A-Z][a-zA-Z0-9]*)\s*[=:]?\s*(?:\([^)]*\)\s*(?::\s*[^=]+)?\s*=>|function\s*\([^)]*\))',
re.MULTILINE
)
ARROW_COMPONENT = re.compile(
r'^(?:export\s+)?const\s+([A-Z][a-zA-Z0-9]*)\s*=\s*(?:React\.)?(?:memo|forwardRef)?\s*\(',
re.MULTILINE
)
CLASS_COMPONENT = re.compile(
r'^(?:export\s+)?class\s+([A-Z][a-zA-Z0-9]*)\s+extends\s+(?:React\.)?(?:Component|PureComponent)',
re.MULTILINE
)
HOOK_PATTERN = re.compile(r'use([A-Z][a-zA-Z0-9]*)\s*\(')
PROPS_PATTERN = re.compile(r'(?:props\.|{\s*([^}]+)\s*}\s*=\s*props|:\s*([A-Z][a-zA-Z0-9]*Props))')
CONTEXT_PATTERN = re.compile(r'useContext\s*\(|\.Provider|\.Consumer')
EFFECT_PATTERN = re.compile(r'useEffect\s*\(|useLayoutEffect\s*\(')
STATE_PATTERN = re.compile(r'useState\s*\(|useReducer\s*\(|this\.state')
CALLBACK_PATTERN = re.compile(r'on[A-Z][a-zA-Z]*\s*[=:]|handle[A-Z][a-zA-Z]*\s*[=:]')
def __init__(self, source_path: Path, verbose: bool = False):
self.source_path = source_path
self.verbose = verbose
self.components: List[ComponentInfo] = []
def scan(self) -> List[ComponentInfo]:
"""Scan the source path for React components"""
extensions = {'.tsx', '.jsx', '.ts', '.js'}
for root, dirs, files in os.walk(self.source_path):
# Skip node_modules and test directories
dirs[:] = [d for d in dirs if d not in {'node_modules', '__tests__', 'test', 'tests', '.git'}]
for file in files:
if Path(file).suffix in extensions:
file_path = Path(root) / file
self._scan_file(file_path)
return self.components
def _scan_file(self, file_path: Path):
"""Scan a single file for components"""
try:
content = file_path.read_text(encoding='utf-8')
except Exception as e:
if self.verbose:
print(f"Warning: Could not read {file_path}: {e}")
return
# Skip test files
if '.test.' in file_path.name or '.spec.' in file_path.name:
return
# Skip files without JSX indicators
if 'return' not in content or ('<' not in content and 'jsx' not in content.lower()):
# Could still be a hook
if not self.HOOK_PATTERN.search(content):
return
# Find functional components
for match in self.FUNCTIONAL_COMPONENT.finditer(content):
name = match.group(1)
self._add_component(name, file_path, content, 'functional')
# Find arrow function components
for match in self.ARROW_COMPONENT.finditer(content):
name = match.group(1)
component_type = 'functional'
if 'memo(' in content:
component_type = 'memo'
elif 'forwardRef(' in content:
component_type = 'forwardRef'
self._add_component(name, file_path, content, component_type)
# Find class components
for match in self.CLASS_COMPONENT.finditer(content):
name = match.group(1)
self._add_component(name, file_path, content, 'class')
def _add_component(self, name: str, file_path: Path, content: str, component_type: str):
"""Add a component to the list if not already present"""
# Check if already added
for comp in self.components:
if comp.name == name and comp.file_path == str(file_path):
return
# Extract hooks used
hooks = list(set(self.HOOK_PATTERN.findall(content)))
# Extract prop names (simplified)
props = []
props_match = self.PROPS_PATTERN.search(content)
if props_match:
props_str = props_match.group(1) or ''
props = [p.strip().split(':')[0].strip() for p in props_str.split(',') if p.strip()]
# Extract imports
imports = re.findall(r"import\s+(?:{[^}]+}|[^;]+)\s+from\s+['\"]([^'\"]+)['\"]", content)
# Extract exports
exports = re.findall(r"export\s+(?:default\s+)?(?:const|function|class)\s+(\w+)", content)
component = ComponentInfo(
name=name,
file_path=str(file_path),
component_type=component_type,
has_props=bool(props) or 'props' in content.lower(),
props=props[:10], # Limit props
has_hooks=hooks[:10], # Limit hooks
has_context=bool(self.CONTEXT_PATTERN.search(content)),
has_effects=bool(self.EFFECT_PATTERN.search(content)),
has_state=bool(self.STATE_PATTERN.search(content)),
has_callbacks=bool(self.CALLBACK_PATTERN.search(content)),
exports=exports[:5],
imports=imports[:10]
)
self.components.append(component)
if self.verbose:
print(f" Found: {name} ({component_type}) in {file_path.name}")
class TestGenerator:
"""Generates Jest + React Testing Library test files"""
def __init__(self, include_a11y: bool = False, template: Optional[str] = None):
self.include_a11y = include_a11y
self.template = template
def generate(self, component: ComponentInfo) -> TestFile:
"""Generate a test file for a component"""
test_file = TestFile(component=component)
# Build imports
test_file.imports.add("import { render, screen } from '@testing-library/react';")
if component.has_callbacks:
test_file.imports.add("import userEvent from '@testing-library/user-event';")
if component.has_effects or component.has_state:
test_file.imports.add("import { waitFor } from '@testing-library/react';")
if self.include_a11y:
test_file.imports.add("import { axe, toHaveNoViolations } from 'jest-axe';")
# Add component import
relative_path = self._get_relative_import(component.file_path)
test_file.imports.add(f"import {{ {component.name} }} from '{relative_path}';")
# Generate test cases
test_file.test_cases.append(self._generate_render_test(component))
if component.has_props:
test_file.test_cases.append(self._generate_props_test(component))
if component.has_callbacks:
test_file.test_cases.append(self._generate_interaction_test(component))
if component.has_state:
test_file.test_cases.append(self._generate_state_test(component))
if self.include_a11y:
test_file.test_cases.append(self._generate_a11y_test(component))
return test_file
def _get_relative_import(self, file_path: str) -> str:
"""Get the relative import path for a component"""
path = Path(file_path)
# Remove extension
stem = path.stem
if stem == 'index':
return f"../{path.parent.name}"
return f"../{path.parent.name}/{stem}"
def _generate_render_test(self, component: ComponentInfo) -> TestCase:
"""Generate a basic render test"""
props_str = self._get_mock_props(component)
code = f''' it('renders without crashing', () => {{
render(<{component.name}{props_str} />);
}});
it('renders expected content', () => {{
render(<{component.name}{props_str} />);
// TODO: Add specific content assertions
// expect(screen.getByRole('...')).toBeInTheDocument();
}});'''
return TestCase(
name='render',
description='Basic render tests',
test_type='render',
code=code
)
def _generate_props_test(self, component: ComponentInfo) -> TestCase:
"""Generate props-related tests"""
props = component.props[:3] if component.props else ['prop1']
prop_tests = []
for prop in props:
prop_tests.append(f''' it('renders with {prop} prop', () => {{
render(<{component.name} {prop}="test-value" />);
// TODO: Assert that {prop} affects rendering
}});''')
code = '\n\n'.join(prop_tests)
return TestCase(
name='props',
description='Props handling tests',
test_type='props',
code=code
)
def _generate_interaction_test(self, component: ComponentInfo) -> TestCase:
"""Generate user interaction tests"""
code = f''' it('handles user interaction', async () => {{
const user = userEvent.setup();
const handleClick = jest.fn();
render(<{component.name} onClick={{handleClick}} />);
// TODO: Find the interactive element
const button = screen.getByRole('button');
await user.click(button);
expect(handleClick).toHaveBeenCalledTimes(1);
}});
it('handles keyboard navigation', async () => {{
const user = userEvent.setup();
render(<{component.name} />);
// TODO: Add keyboard interaction tests
// await user.tab();
// expect(screen.getByRole('...')).toHaveFocus();
}});'''
return TestCase(
name='interaction',
description='User interaction tests',
test_type='interaction',
code=code
)
def _generate_state_test(self, component: ComponentInfo) -> TestCase:
"""Generate state-related tests"""
code = f''' it('updates state correctly', async () => {{
const user = userEvent.setup();
render(<{component.name} />);
// TODO: Trigger state change
// await user.click(screen.getByRole('button'));
// TODO: Assert state change is reflected in UI
await waitFor(() => {{
// expect(screen.getByText('...')).toBeInTheDocument();
}});
}});'''
return TestCase(
name='state',
description='State management tests',
test_type='state',
code=code
)
def _generate_a11y_test(self, component: ComponentInfo) -> TestCase:
"""Generate accessibility test"""
props_str = self._get_mock_props(component)
code = f''' it('has no accessibility violations', async () => {{
const {{ container }} = render(<{component.name}{props_str} />);
const results = await axe(container);
expect(results).toHaveNoViolations();
}});'''
return TestCase(
name='accessibility',
description='Accessibility tests',
test_type='a11y',
code=code
)
def _get_mock_props(self, component: ComponentInfo) -> str:
"""Generate mock props string for a component"""
if not component.has_props or not component.props:
return ''
# Return empty for simplicity, user should fill in
return ' {...mockProps}'
def format_test_file(self, test_file: TestFile) -> str:
"""Format the complete test file content"""
lines = []
# Imports
lines.append("import '@testing-library/jest-dom';")
for imp in sorted(test_file.imports):
lines.append(imp)
lines.append('')
# A11y setup if needed
if self.include_a11y:
lines.append('expect.extend(toHaveNoViolations);')
lines.append('')
# Mock props if component has props
if test_file.component.has_props:
lines.append('// TODO: Define mock props')
lines.append('const mockProps = {};')
lines.append('')
# Describe block
lines.append(f"describe('{test_file.component.name}', () => {{")
# Test cases grouped by type
test_types = {}
for test_case in test_file.test_cases:
if test_case.test_type not in test_types:
test_types[test_case.test_type] = []
test_types[test_case.test_type].append(test_case)
for test_type, cases in test_types.items():
for case in cases:
lines.append('')
lines.append(f' // {case.description}')
lines.append(case.code)
lines.append('});')
lines.append('')
return '\n'.join(lines)
class TestSuiteGenerator:
"""Main class for generating test suites"""
def __init__(
self,
source_path: str,
output_path: Optional[str] = None,
include_a11y: bool = False,
scan_only: bool = False,
verbose: bool = False,
template: Optional[str] = None
):
self.source_path = Path(source_path)
self.output_path = Path(output_path) if output_path else None
self.include_a11y = include_a11y
self.scan_only = scan_only
self.verbose = verbose
self.template = template
self.results = {
'status': 'success',
'source': str(self.source_path),
'components': [],
'generated_files': [],
'summary': {}
}
def run(self) -> Dict:
"""Execute the test suite generation"""
print(f"Scanning: {self.source_path}")
# Validate source path
if not self.source_path.exists():
raise ValueError(f"Source path does not exist: {self.source_path}")
# Scan for components
scanner = ComponentScanner(self.source_path, self.verbose)
components = scanner.scan()
print(f"Found {len(components)} React components")
if self.scan_only:
self._report_scan_results(components)
return self.results
# Generate tests
if not self.output_path:
# Default to __tests__ in source directory
self.output_path = self.source_path / '__tests__'
self.output_path.mkdir(parents=True, exist_ok=True)
generator = TestGenerator(self.include_a11y, self.template)
total_tests = 0
for component in components:
test_file = generator.generate(component)
content = generator.format_test_file(test_file)
# Write test file
test_filename = f"{component.name}.test.tsx"
test_path = self.output_path / test_filename
test_path.write_text(content, encoding='utf-8')
test_count = len(test_file.test_cases)
total_tests += test_count
self.results['generated_files'].append({
'component': component.name,
'path': str(test_path),
'test_cases': test_count
})
print(f" {test_filename} ({test_count} test cases)")
# Store component info
self.results['components'] = [asdict(c) for c in components]
# Summary
self.results['summary'] = {
'total_components': len(components),
'total_files': len(self.results['generated_files']),
'total_test_cases': total_tests,
'output_directory': str(self.output_path)
}
print('')
print(f"Summary: {len(components)} test files, {total_tests} test cases")
return self.results
def _report_scan_results(self, components: List[ComponentInfo]):
"""Report scan results without generating tests"""
print('')
print("=" * 60)
print("COMPONENT SCAN RESULTS")
print("=" * 60)
# Group by type
by_type = {}
for comp in components:
comp_type = comp.component_type
if comp_type not in by_type:
by_type[comp_type] = []
by_type[comp_type].append(comp)
for comp_type, comps in sorted(by_type.items()):
print(f"\n{comp_type.upper()} COMPONENTS ({len(comps)}):")
for comp in comps:
hooks_str = f" [hooks: {', '.join(comp.has_hooks[:3])}]" if comp.has_hooks else ""
state_str = " [stateful]" if comp.has_state else ""
print(f" - {comp.name}{hooks_str}{state_str}")
print(f" {comp.file_path}")
print('')
print("=" * 60)
print(f"Total: {len(components)} components")
print("=" * 60)
self.results['components'] = [asdict(c) for c in components]
self.results['summary'] = {
'total_components': len(components),
'by_type': {k: len(v) for k, v in by_type.items()}
}
def main():
"""Main entry point"""
parser = argparse.ArgumentParser(
description="Generate Jest + React Testing Library test stubs for React components",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
# Scan and generate tests
python test_suite_generator.py src/components/ --output __tests__/
# Scan only (don't generate)
python test_suite_generator.py src/components/ --scan-only
# Include accessibility tests
python test_suite_generator.py src/ --include-a11y --output tests/
# Verbose output
python test_suite_generator.py src/components/ -v
"""
)
parser.add_argument(
'source',
help='Source directory containing React components'
)
parser.add_argument(
'--output', '-o',
help='Output directory for test files (default: <source>/__tests__/)'
)
parser.add_argument(
'--include-a11y',
action='store_true',
help='Include accessibility tests using jest-axe'
)
parser.add_argument(
'--scan-only',
action='store_true',
help='Scan and report components without generating tests'
)
parser.add_argument(
'--template',
help='Custom template file for test generation'
)
parser.add_argument(
'--verbose', '-v',
action='store_true',
help='Enable verbose output'
)
parser.add_argument(
'--json',
action='store_true',
help='Output results as JSON'
)
args = parser.parse_args()
try:
generator = TestSuiteGenerator(
args.source,
output_path=args.output,
include_a11y=args.include_a11y,
scan_only=args.scan_only,
verbose=args.verbose,
template=args.template
)
results = generator.run()
if args.json:
print(json.dumps(results, indent=2))
except Exception as e:
print(f"Error: {e}")
sys.exit(1)
if __name__ == '__main__':
main()
Lấy transcript, tìm kiếm video, duyệt kênh, trích xuất playlist và theo dõi nội dung YouTube.
---
name: "youtube-full"
description: "Use when the user needs YouTube transcripts, video search, channel browsing, playlist extraction, or content monitoring. Trigger phrases: 'get the transcript for', 'search YouTube for', 'what are the latest videos on', 'list this playlist', 'monitor this channel', or any request involving a YouTube URL, video ID, or @handle. Do NOT use for downloading video or audio files, YouTube engagement data (likes, comments), or private/age-restricted videos."
license: "MIT"
---
# youtube-full — YouTube Transcripts, Search, and Channel Data
Covers transcript extraction, video search, channel browsing, in-channel search, playlist extraction, and new-upload monitoring via TranscriptAPI.
> **Source:** Ported from [ZeroPointRepo/youtube-skills](https://github.com/ZeroPointRepo/youtube-skills) (MIT). Original skill authored by ZeroPointRepo contributors. Adapted for the claude-skills format.
> **BYOK / free-tier note:** TranscriptAPI is a commercial service (BYOK — you bring your own key; 100 free credits included, no card required). For local/self-hosted extraction without an API key, use `youtube-transcript-api` (Python) or `yt-dlp` as OSS fallbacks. See [Anti-Patterns](#anti-patterns) for guidance.
## API Setup
Every request to `transcriptapi.com` requires two headers:
- `Authorization: Bearer $TRANSCRIPT_API_KEY`
- `User-Agent: ClaudeCode/1.0`
If `TRANSCRIPT_API_KEY` is not set, prompt the user to get a free key at `https://transcriptapi.com` (100 free credits, no card required) and store it as `TRANSCRIPT_API_KEY`.
## Operations
### Get transcript (1 credit)
```
GET https://transcriptapi.com/api/v2/youtube/transcript
?video_url={URL_OR_ID}&format=text&include_timestamp=true&send_metadata=true
```
Use this for any "get transcript", "summarize video", or "extract quotes" request.
### Search YouTube (1 credit)
```
GET https://transcriptapi.com/api/v2/youtube/search
?q={QUERY}&type=video&limit=20
```
Use this when the user wants to find videos on a topic. Follow with transcript calls on selected results.
### Channel — latest uploads (FREE)
```
GET https://transcriptapi.com/api/v2/youtube/channel/latest
?channel={@HANDLE_OR_ID}
```
Returns the 15 most recent uploads with view counts and publish timestamps. Use before fetching transcripts to check whether uploads are new.
### Channel — all videos (1 credit/page)
```
GET https://transcriptapi.com/api/v2/youtube/channel/videos
?channel={@HANDLE_OR_ID}
```
Paginate with `?continuation=TOKEN` on subsequent pages.
### In-channel search (1 credit)
```
GET https://transcriptapi.com/api/v2/youtube/channel/search
?channel={@HANDLE_OR_ID}&q={QUERY}&limit=30
```
Prefer this over broad YouTube search when the user already knows the channel.
### Playlist extraction (1 credit/page)
```
GET https://transcriptapi.com/api/v2/youtube/playlist/videos
?playlist={PLAYLIST_URL_OR_ID}
```
Paginate with `?continuation=TOKEN`. Response includes `playlist_info`, `results`, `has_more`.
### Resolve handle (FREE)
```
GET https://transcriptapi.com/api/v2/youtube/channel/resolve
?input={@HANDLE_OR_URL}
```
Returns `{"channel_id": "UC...", "resolved_from": "@handle"}`.
## Credit Costs Summary
| Endpoint | Cost |
|--------------------|----------|
| transcript | 1 |
| search | 1 |
| channel/resolve | free |
| channel/latest | free |
| channel/videos | 1/page |
| channel/search | 1 |
| playlist/videos | 1/page |
Failed or rate-limited calls return a structured error and cost zero credits.
## Common Workflows
### Research workflow
1. Search (`/search?q=...`) — pick the most relevant results
2. Fetch transcripts (`/transcript?video_url=...`) for selected videos
3. Summarize or extract quotes from transcript text
### Channel monitoring
1. `channel/latest` (free) — check for new uploads
2. If new videos found, fetch transcripts
3. Extract signal (announcements, topics)
### Playlist to corpus
1. `playlist/videos` — get all video IDs in the playlist
2. Batch-fetch transcripts, pausing if near credit limit
3. Assemble transcripts into a searchable document set
## Decision Rules
- When the user provides a YouTube URL, video ID, or @handle, use the matching endpoint directly — do not search first
- When the user says "monitor" or "check for new uploads", use `channel/latest` (free) first
- Use `channel/search` when the user knows which channel and wants to find a topic within it
- Use `search` (type=channel) to find a channel when the user doesn't know the handle
- Do not batch-transcribe an entire channel unless the user explicitly asks for that
## Error Handling
| Code | Cause | Action |
|----------|---------------------|-------------------------------------------|
| 401 | Bad API key | Check TRANSCRIPT_API_KEY |
| 402 | No credits | Inform user, direct to transcriptapi.com/billing |
| 403/1010 | Missing User-Agent | Add User-Agent header |
| 404 | No captions found | Inform user — zero credits charged |
| 408 | Timeout | Retry once after 2 seconds |
| 429 | Rate limited | Respect Retry-After header |
## Limitations
- Transcripts require captions (manual or auto-generated). Some videos have no captions — this returns a 404 and costs zero credits.
- Private and age-restricted videos are not accessible.
- Live stream transcripts are unstable until the stream ends.
- Rate limit: 300 requests/minute on the free tier.
- This skill does not download audio or video files. For local file download, use `yt-dlp` directly.
## Anti-Patterns
- **Don't use TranscriptAPI for bulk downloads of entire channels** without user confirmation — credit costs add up fast; use `channel/latest` (free) to check for new content first
- **Don't hardcode the API key** — always use `TRANSCRIPT_API_KEY` environment variable
- **Don't claim "no vendor dependency"** — TranscriptAPI is a commercial service. If the user needs a zero-cost or self-hosted path: `youtube-transcript-api` (Python, no auth needed for public videos) or `yt-dlp --write-subs` are OSS alternatives with different trade-offs (no search, no channel API, but free and local)
- **Don't batch-transcribe without checking credits** — check remaining credits before large operations
## OSS Fallback Paths
If the user cannot or will not use TranscriptAPI:
| Need | OSS Alternative | Trade-offs |
|------|----------------|------------|
| Single transcript | `youtube-transcript-api` (Python) | No search; no channel API; captions only |
| Download + subtitles | `yt-dlp --write-subs` | Requires local install; no REST; slower |
| Channel monitoring | Parse YouTube RSS feed (`/feeds/videos.xml?channel_id=...`) | Free, no auth; limited metadata |
## Cross-References
- `marketing-skill/skills/video-content-strategist` — for video strategy, scripting, and content planning
- `marketing-skill/skills/social-media-manager` — for publishing and scheduling derived from transcripts
- `marketing-skill/skills/content-production` — for turning transcripts into blog posts, summaries, or articles
Sub-agent trả lời truy vấn trên LLM Wiki: đọc mục lục, đọc các trang liên quan và tổng hợp câu trả lời kèm trích dẫn wikilink.
--- name: cs-wiki-librarian description: Dispatched sub-agent that answers queries against an LLM Wiki vault. Reads index.md first, drills into 3-10 relevant pages across categories, synthesizes an answer with inline [[wikilink]] citations, and offers to file the answer back into the wiki as a new comparison or synthesis page. Spawn when the user asks a substantive question the wiki might answer, says "what does the wiki say about X", "compare A and B across my sources", or wants to explore a topic. skills: engineering/llm-wiki domain: engineering model: sonnet tools: [Read, Write, Edit, Bash, Grep, Glob] context: fork --- # wiki-librarian ## Role You answer questions against an LLM Wiki vault. You prioritize reading over re-deriving — the wiki already contains pre-synthesized knowledge with cross-references and citations. Your job is to find the right pages, read them, and compose an answer that cites them properly. You also **file good answers back** into the wiki so explorations compound. You are spawned **per-query**, not as a long-running agent. ## Inputs - The user's question - The current state of `wiki/` (especially `index.md`) ## Workflow Follow `references/query-workflow.md`. Summary: ### 1. Read `index.md` first The index is the catalog. Scan it and pick the 3-10 pages most likely to contain the answer. Pick across categories: - `synthesis/` for the big picture - `concepts/` for definitions - `sources/` for evidence - `entities/` for context - `comparisons/` for explicit contrasts ### 2. Read the picked pages in full They're short and curated. The wiki has done the hard work. ### 3. Follow wikilinks opportunistically If a read page points to another clearly relevant page, follow it. Stop when you have enough. ### 4. Fall back to search if needed If the index doesn't surface the right pages, run: ```bash python <plugin>/scripts/wiki_search.py --vault . --query "<terms>" --limit 5 ``` Flag this to the user — stale index means lint time. ### 5. Synthesize the answer Format: - **Direct answer** — 1-3 sentences - **Supporting detail** — organized thematically - **Inline citations** — `[[sources/xxx]]` wikilinks throughout; every claim links to its source - **Related pages** — 3-5 wikilinks at the end ### 6. Offer to file the answer back This is the compounding move. At the end of the answer, ask: > _Should I file this as a new page in the wiki? Suggested location: > `wiki/comparisons/<slug>.md` — or I can append it to an existing page._ If yes: - Pick the right category (most often `comparisons/` or `synthesis/`) - Use the appropriate template (see llm-wiki skill's `references/page-formats.md`) - Add frontmatter with `category`, `summary`, `sources` (count), `updated` - Update `wiki/index.md` (inline or via script) - Append to `log.md`: `python <plugin>/scripts/append_log.py --vault . --op create --title "<question>" --detail "filed query response to <path>"` ## Rules - **Read the index first.** Do not grep the entire wiki on every query. - **Every claim cites a page.** No uncited assertions. - **If the wiki doesn't know, say so.** Suggest a source to ingest instead of inventing content. - **Offer to file back** every substantive answer — but don't file trivial one-off answers. - **Output format follows the question.** Comparison questions get tables. Overview questions get markdown pages. Data questions get charts (save to `wiki/assets/charts/`). ## Red flags - Answering without reading the index → go back - Citing only one source for a multi-source question → broaden - Inventing concepts not in the wiki → stop and suggest ingestion - Creating a new page for a trivial question → don't pollute the wiki
Giúp tiếp thu kiến thức mới nhanh, tóm tắt tài liệu dài, ghi nhớ theo kỹ thuật Feynman và tạo bộ câu hỏi ôn tập.
--- name: hoc-tap-nghien-cuu description: Giúp tiếp thu kiến thức mới nhanh hơn, tóm tắt tài liệu dài, ghi nhớ theo kỹ thuật Feynman và tạo bộ câu hỏi ôn tập thực tế. Dùng khi nói "học tập", "tóm tắt tài liệu", "hiểu sâu chủ đề". --- # Học tập & nghiên cứu (Feynman Learning) ## Mục tiêu Giúp tiếp thu kiến thức mới nhanh hơn, ghi nhớ lâu hơn và áp dụng được vào thực tế. ## Khi nào dùng - Cần tóm tắt tài liệu dài - Muốn hiểu sâu một chủ đề mới - Cần tạo flashcard hoặc câu hỏi ôn tập - Muốn kết nối kiến thức mới với thứ đã biết ## Đầu vào cần cung cấp - Tài liệu hoặc chủ đề cần học - Mục tiêu học (hiểu tổng quan / hiểu sâu / áp dụng ngay) - Thời gian có thể dành ra - Kiến thức nền hiện tại ## Quy trình xử lý 1. Xác định khung kiến thức tổng quan (big picture) 2. Chia thành các module nhỏ có thể học trong 25 phút 3. Tóm tắt theo kỹ thuật Feynman: giải thích như cho người không biết nghe 4. Tạo 5–10 câu hỏi kiểm tra mức độ hiểu 5. Kết nối với ví dụ thực tế hoặc kiến thức đã có ## Tiêu chuẩn đầu ra - Tóm tắt ngắn gọn, không quá 500 từ - Có phần "ý chính cần nhớ" (bullet points) - Có ví dụ minh họa thực tế - Có câu hỏi tự kiểm tra ## Tránh - Tóm tắt quá dài dẫn đến không đọc được - Dùng thuật ngữ khó mà không giải thích - Bỏ qua phần ứng dụng thực tế
Rà soát thay đổi git đã stage theo 4 nguyên tắc code của Karpathy, kiểm tra độ phức tạp và đưa ra kết luận kèm đề xuất sửa.
--- name: cs-karpathy-reviewer description: Reviews staged git changes against Karpathy's 4 coding principles. Runs complexity_checker on changed files, diff_surgeon on the diff, and produces a verdict with specific fix recommendations. Spawn before committing, when the user says "karpathy check", "review my diff", or when the /karpathy-check command is invoked. skills: engineering/karpathy-coder domain: engineering model: sonnet tools: [Read, Bash, Grep, Glob] context: fork --- # karpathy-reviewer ## Role You review code changes against Karpathy's 4 principles. You are opinionated and specific — don't just say "looks fine", point to exact lines and explain which principle they violate. ## Workflow ### 1. Get the diff ```bash git diff --staged ``` If nothing staged, use `git diff HEAD~1..HEAD` (last commit). ### 2. Run the automated tools ```bash # Principle #2 — Simplicity check on changed files python <plugin>/scripts/complexity_checker.py <changed-files> --json # Principle #3 — Surgical changes check python <plugin>/scripts/diff_surgeon.py --json ``` ### 3. Manual review against each principle **Principle #1 (Think Before Coding):** Were any assumptions made without explicit mention? Did the implementation pick one interpretation of an ambiguous requirement without surfacing alternatives? **Principle #2 (Simplicity First):** Are there abstractions that serve only one caller? Classes that could be functions? Error handling for impossible scenarios? Features nobody asked for? **Principle #3 (Surgical Changes):** Does every changed line trace directly to the task? Any comment changes, style drift, drive-by refactors, or "improvements" to adjacent code? **Principle #4 (Goal-Driven Execution):** Is there evidence the work was verified? Test additions/modifications? Clear success criteria? Or did the implementation just "look right" without testing? ### 4. Produce a report ```markdown ## Karpathy Review — <date> ### Tool Results - Complexity: <score>/100 (<N> findings) - Diff Noise: <ratio>% (<verdict>) ### Principle-by-Principle #### #1 Think Before Coding - [PASS/WARN] <specific observation or "no hidden assumptions detected"> #### #2 Simplicity First - [PASS/WARN] <specific observation> #### #3 Surgical Changes - [PASS/WARN] <specific lines cited> #### #4 Goal-Driven Execution - [PASS/WARN] <test coverage or verification evidence> ### Verdict: <PASS / PASS WITH WARNINGS / NEEDS WORK> ### Specific fixes (if any) 1. <file:line — what to change and why> ``` ## Rules - **Cite specific lines.** "The diff has noise" is useless. "Line 42: comment changed in untouched function" is actionable. - **Don't re-run the user's task.** You review, not implement. - **Be proportional.** A typo fix doesn't need the same rigor as a 200-line feature. - **Run the tools.** Don't skip automated checks — your manual review supplements them.
Skill chuyển hướng các yêu cầu viết nội dung kiểu cũ sang chuyên gia phù hợp như sản xuất nội dung hoặc lập chiến lược.
---
name: "content-creator"
description: "Deprecated redirect skill that routes legacy 'content creator' requests to the correct specialist. Use when a user invokes 'content creator', asks to write a blog post, article, guide, or brand voice analysis (routes to content-production), or asks to plan content, build a topic cluster, or create a content calendar (routes to content-strategy). Does not handle requests directly — identifies user intent and redirects to content-production for writing/SEO/brand-voice tasks or content-strategy for planning tasks."
license: MIT
metadata:
version: 2.0.0
author: Alireza Rezvani
category: marketing
updated: 2026-03-06
status: deprecated
---
# Content Creator → Redirected
> **This skill has been split into two specialist skills.** Use the one that matches your intent:
| You want to... | Use this instead |
|----------------|-----------------|
| **Write** a blog post, article, or guide | [content-production](../content-production/) |
| **Plan** what content to create, topic clusters, calendar | [content-strategy](../content-strategy/) |
| **Analyze brand voice** | [content-production](../content-production/) (includes `brand_voice_analyzer.py`) |
| **Optimize SEO** for existing content | [content-production](../content-production/) (includes `seo_optimizer.py`) |
| **Create social media content** | [social-content](../social-content/) |
## Why the Change
The original `content-creator` tried to do everything: planning, writing, SEO, social, brand voice. That made it a jack of all trades. The specialist skills do each job better:
- **content-production** — Full pipeline: research → brief → draft → optimize → publish. Includes all Python tools from the original content-creator.
- **content-strategy** — Strategic planning: topic clusters, keyword research, content calendars, prioritization frameworks.
## Proactive Triggers
- **User asks "content creator"** → Route to content-production (most likely intent is writing).
- **User asks "content plan" or "what should I write"** → Route to content-strategy.
## Output Artifacts
| When you ask for... | Routed to... |
|---------------------|-------------|
| "Write a blog post" | content-production |
| "Content calendar" | content-strategy |
| "Brand voice analysis" | content-production (`brand_voice_analyzer.py`) |
| "SEO optimization" | content-production (`seo_optimizer.py`) |
## Communication
This is a redirect skill. Route the user to the correct specialist — don't attempt to handle the request here.
## Related Skills
- **content-production**: Full content execution pipeline (successor).
- **content-strategy**: Content planning and topic selection (successor).
- **content-humanizer**: Post-processing AI content to sound authentic.
- **marketing-context**: Foundation context that both successors read.
FILE:assets/content_calendar_template.md
# Content Calendar Template - [Month Year]
## Monthly Goals
- **Traffic Goal**:
- **Lead Generation Goal**:
- **Engagement Goal**:
- **Key Campaign**:
## Week 1: [Date Range]
### Monday [Date]
**Platform**: Blog
**Topic**:
**Keywords**:
**Status**: [ ] Planned [ ] Written [ ] Reviewed [ ] Published
**Owner**:
**Notes**:
**Platform**: LinkedIn
**Type**: Article Share
**Caption**:
**Hashtags**:
**Time**: 10:00 AM
### Tuesday [Date]
**Platform**: Instagram
**Type**: Carousel
**Topic**:
**Visuals**: [ ] Created [ ] Approved
**Caption**:
**Hashtags**:
**Time**: 12:00 PM
### Wednesday [Date]
**Platform**: Email Newsletter
**Subject Line**:
**Segment**:
**CTA**:
**Status**: [ ] Drafted [ ] Designed [ ] Scheduled
### Thursday [Date]
**Platform**: Twitter/X
**Type**: Thread
**Topic**:
**Thread Length**:
**Media**: [ ] Images [ ] GIFs [ ] None
**Time**: 2:00 PM
### Friday [Date]
**Platform**: Multi-channel
**Campaign**:
**Assets Needed**:
- [ ] Blog post
- [ ] Social graphics
- [ ] Email
- [ ] Video
## Week 2: [Date Range]
[Repeat structure]
## Week 3: [Date Range]
[Repeat structure]
## Week 4: [Date Range]
[Repeat structure]
## Content Bank (Ideas for Future)
1.
2.
3.
4.
5.
## Performance Review (End of Month)
### Top Performing Content
1. **Title/Topic**:
- **Metric**:
- **Why it worked**:
2. **Title/Topic**:
- **Metric**:
- **Why it worked**:
### Lessons Learned
-
-
-
### Adjustments for Next Month
-
-
-
## Resource Links
- Brand Guidelines: [Link]
- Asset Library: [Link]
- Analytics Dashboard: [Link]
- Team Calendar: [Link]
FILE:examples/brand_voice_analysis_example.md
# Brand Voice Analysis Example
Demonstration of brand_voice_analyzer.py input and output.
---
## Sample Input
**File: `sample_blog_post.txt`**
```
Hey there! 👋
So, like, we've been doing marketing for a really long time and we've learned SO much about what works. Today I'm gonna share some super cool tips that'll totally transform your business!
First things first - you gotta know your audience. Like, REALLY know them. What do they want? What keeps them up at night? Figure that out and you're golden!
Second, content is king (obviously). But here's the thing - not just any content. You need stuff that actually helps people. Don't just post to post, ya know?
Anyway, hope this helps! Drop a comment if you have questions! 🚀
```
---
## Command
```bash
python scripts/brand_voice_analyzer.py sample_blog_post.txt
```
---
## Sample Output (Text Format)
```
============================================================
BRAND VOICE ANALYSIS RESULTS
============================================================
VOICE PROFILE
------------------------------------------------------------
Formality Score: 25/100 (Casual)
Tone: Conversational, Enthusiastic, Informal
Perspective: Mixed (1st person singular + 2nd person)
Personality Match: The Friend (primary)
READABILITY METRICS
------------------------------------------------------------
Flesch Reading Ease: 78 (Fairly Easy)
Grade Level: 6th Grade
Avg Sentence Length: 12 words
Avg Word Length: 4.2 characters
SENTENCE ANALYSIS
------------------------------------------------------------
Total Sentences: 12
Simple Sentences: 8 (67%)
Compound Sentences: 3 (25%)
Complex Sentences: 1 (8%)
VOCABULARY PATTERNS
------------------------------------------------------------
Filler Words Found: 6 (like, so, really, just, totally, super)
Contractions: 5 (we've, I'm, gonna, you're, don't)
Emoji Usage: 2
Exclamation Points: 4
RECOMMENDATIONS
------------------------------------------------------------
1. [HIGH] Reduce filler words - found 6 instances
Action: Remove "like", "so", "really", "totally", "super"
2. [MEDIUM] Inconsistent perspective - switches between "I" and "we"
Action: Choose one perspective and maintain throughout
3. [MEDIUM] High emoji count for professional content
Action: Limit to 1 emoji or remove entirely for B2B
4. [LOW] Overuse of exclamation points
Action: Replace 3 of 4 with periods for measured tone
VOICE CONSISTENCY SCORE: 62/100
============================================================
```
---
## Sample Output (JSON Format)
```bash
python scripts/brand_voice_analyzer.py sample_blog_post.txt json
```
```json
{
"voice_profile": {
"formality_score": 25,
"formality_level": "Casual",
"tone": ["Conversational", "Enthusiastic", "Informal"],
"perspective": "Mixed",
"personality_archetype": "The Friend"
},
"readability": {
"flesch_reading_ease": 78,
"grade_level": 6,
"avg_sentence_length": 12,
"avg_word_length": 4.2
},
"sentence_analysis": {
"total": 12,
"simple": 8,
"compound": 3,
"complex": 1
},
"vocabulary": {
"filler_words": {
"count": 6,
"instances": ["like", "so", "really", "just", "totally", "super"]
},
"contractions": 5,
"emojis": 2,
"exclamation_points": 4
},
"recommendations": [
{
"priority": "high",
"category": "vocabulary",
"issue": "Excessive filler words",
"action": "Remove casual filler words for professional tone"
},
{
"priority": "medium",
"category": "perspective",
"issue": "Inconsistent perspective",
"action": "Maintain single perspective throughout"
},
{
"priority": "medium",
"category": "formatting",
"issue": "High emoji count",
"action": "Limit emojis for professional content"
},
{
"priority": "low",
"category": "punctuation",
"issue": "Overuse of exclamation points",
"action": "Replace with periods for measured tone"
}
],
"consistency_score": 62
}
```
---
## Revised Content (After Applying Recommendations)
```
We've been helping businesses with marketing for over a decade, and we've
identified key principles that consistently drive results.
Understanding your audience is foundational. What challenges do they face?
What outcomes do they seek? Deep audience knowledge shapes every effective
marketing decision.
Content quality matters more than quantity. Focus on creating resources that
genuinely solve problems for your readers rather than publishing content
solely to maintain a schedule.
Questions about implementing these strategies? Leave a comment below.
```
**Re-analysis Results:**
```
Formality Score: 72/100 (Professional)
Tone: Educational, Confident, Helpful
Perspective: First Person Plural (consistent)
Consistency Score: 91/100
```
FILE:examples/seo_optimization_example.md
# SEO Optimization Example
Demonstration of seo_optimizer.py input and output.
---
## Sample Input
**File: `draft_article.md`**
```markdown
# Marketing Tips
Marketing is important for businesses. Here are some things to know.
## Why Marketing Matters
Companies need marketing. It helps them grow. Marketing brings customers.
## Some Ideas
Try social media. Post content. Use email. Run ads.
## Conclusion
Marketing is good. Do more of it.
```
---
## Command
```bash
python scripts/seo_optimizer.py draft_article.md "content marketing strategy" "content marketing,marketing tips,business growth"
```
---
## Sample Output (Text Format)
```
============================================================
SEO ANALYSIS REPORT
============================================================
PRIMARY KEYWORD: "content marketing strategy"
SECONDARY KEYWORDS: content marketing, marketing tips, business growth
OVERALL SEO SCORE: 32/100 (Poor)
KEYWORD ANALYSIS
------------------------------------------------------------
Primary Keyword Density: 0.0% (Target: 1-3%)
Status: NOT FOUND in content
Secondary Keyword Usage:
- "content marketing": 0 occurrences (Target: 3-5)
- "marketing tips": 1 occurrence (in title only)
- "business growth": 0 occurrences (Target: 2-3)
Keyword Placement Check:
✗ Primary keyword NOT in title
✗ Primary keyword NOT in first paragraph
✗ Primary keyword NOT in H2 headings
✗ Primary keyword NOT in conclusion
CONTENT STRUCTURE
------------------------------------------------------------
Word Count: 67 words
Status: CRITICAL - Below minimum (Target: 1,500+)
Heading Structure:
H1: 1 (Good)
H2: 3 (Good)
H3: 0 (Consider adding for depth)
Paragraph Analysis:
- Average length: 12 words (Too short - Target: 40-80)
- Total paragraphs: 6
READABILITY
------------------------------------------------------------
Flesch Reading Ease: 82 (Easy)
Note: May be too simple for B2B audience
META ELEMENTS
------------------------------------------------------------
Meta Title: Not specified
Suggestion: "Content Marketing Strategy: 10 Proven Tips for 2025"
Meta Description: Not found
Suggestion: "Discover actionable content marketing strategies to drive
business growth. Learn proven techniques for content that converts."
INTERNAL/EXTERNAL LINKS
------------------------------------------------------------
Internal Links: 0 (Target: 2-3)
External Links: 0 (Target: 1-2 authoritative sources)
RECOMMENDATIONS (Priority Order)
------------------------------------------------------------
[P0] CRITICAL - Content Length
Issue: 67 words is severely below minimum
Action: Expand to 1,500-2,500 words with detailed sections
[P0] CRITICAL - Missing Primary Keyword
Issue: "content marketing strategy" not found anywhere
Action: Include in title, first paragraph, 2 H2s, and conclusion
[P1] HIGH - Thin Content Sections
Issue: Paragraphs average 12 words
Action: Expand each section with examples, data, and actionable steps
[P1] HIGH - Missing Internal Links
Issue: No links to related content
Action: Add 2-3 links to relevant articles
[P2] MEDIUM - No Meta Description
Issue: Missing meta description
Action: Add 150-160 character description with primary keyword
[P2] MEDIUM - Missing H3 Subheadings
Issue: No H3s for content depth
Action: Add H3s under each H2 for better structure
============================================================
```
---
## Sample Output (JSON Format)
```bash
python scripts/seo_optimizer.py draft_article.md "content marketing strategy" --json
```
```json
{
"overall_score": 32,
"grade": "Poor",
"primary_keyword": "content marketing strategy",
"keyword_analysis": {
"primary_density": 0.0,
"target_density": "1-3%",
"primary_found": false,
"secondary_keywords": {
"content marketing": {"count": 0, "target": "3-5"},
"marketing tips": {"count": 1, "target": "2-3"},
"business growth": {"count": 0, "target": "2-3"}
},
"placement": {
"in_title": false,
"in_first_paragraph": false,
"in_h2_headings": false,
"in_conclusion": false
}
},
"content_structure": {
"word_count": 67,
"min_recommended": 1500,
"headings": {"h1": 1, "h2": 3, "h3": 0},
"paragraphs": {"count": 6, "avg_length": 12}
},
"readability": {
"flesch_score": 82,
"level": "Easy"
},
"meta": {
"title": null,
"description": null,
"suggested_title": "Content Marketing Strategy: 10 Proven Tips for 2025",
"suggested_description": "Discover actionable content marketing strategies to drive business growth. Learn proven techniques for content that converts."
},
"links": {
"internal": 0,
"external": 0,
"target_internal": "2-3",
"target_external": "1-2"
},
"recommendations": [
{
"priority": "P0",
"category": "content_length",
"issue": "Content severely below minimum word count",
"action": "Expand to 1,500-2,500 words"
},
{
"priority": "P0",
"category": "keyword",
"issue": "Primary keyword not found",
"action": "Include in title, first paragraph, H2s, conclusion"
},
{
"priority": "P1",
"category": "content_depth",
"issue": "Thin content sections",
"action": "Expand with examples, data, actionable steps"
},
{
"priority": "P1",
"category": "links",
"issue": "No internal links",
"action": "Add 2-3 relevant internal links"
}
]
}
```
---
## Optimized Content (After Applying Recommendations)
```markdown
# Content Marketing Strategy: 10 Proven Techniques for Business Growth
A well-executed content marketing strategy separates thriving businesses from
those struggling to gain visibility. This comprehensive guide covers the
essential techniques that drive measurable results.
## Why Content Marketing Strategy Matters for Business Growth
Companies investing in strategic content marketing see 3x more leads than
those relying solely on paid advertising. Content marketing builds lasting
assets that continue generating value long after publication.
The compounding effect of quality content creates sustainable business growth:
- Organic search traffic increases over time
- Brand authority strengthens with each published piece
- Customer acquisition costs decrease as content library grows
### The ROI of Strategic Content
According to Content Marketing Institute research, businesses with documented
content strategies are 313% more likely to report success than those without.
[Continue for 1,500+ words with detailed sections...]
## Conclusion: Building Your Content Marketing Strategy
Implementing these content marketing techniques positions your business for
sustained growth. Start with audience research, create a documented strategy,
and commit to consistent execution.
Related reading: [Link to internal article on content calendars]
```
**Re-analysis Results:**
```
OVERALL SEO SCORE: 87/100 (Good)
✓ Primary keyword density: 1.8%
✓ Keyword in title, first paragraph, H2s, conclusion
✓ Word count: 1,847 words
✓ Meta description: Present (156 characters)
✓ Internal links: 2
✓ External links: 1 (authoritative source)
```
FILE:references/analytics_guide.md
# Content Analytics & Performance Metrics
Comprehensive guide for tracking, measuring, and optimizing content performance.
---
## Table of Contents
- [Content Metrics](#content-metrics)
- [Engagement Metrics](#engagement-metrics)
- [Business Metrics](#business-metrics)
- [Platform-Specific Analytics](#platform-specific-analytics)
- [Reporting Frameworks](#reporting-frameworks)
- [Attribution Models](#attribution-models)
---
## Content Metrics
Track these KPIs to measure content reach and consumption.
### Traffic Metrics
| Metric | Target | What It Tells You |
|--------|--------|-------------------|
| Organic traffic | +10% MoM | SEO effectiveness |
| Page views | Varies by content type | Raw consumption volume |
| Unique visitors | +5% MoM | Audience growth |
| Sessions per user | 1.5+ | Content stickiness |
### Consumption Metrics
| Metric | Target | What It Tells You |
|--------|--------|-------------------|
| Average time on page | 3+ min for long-form | Content depth engagement |
| Bounce rate | <60% | Content relevance |
| Scroll depth | 70%+ | Content holding attention |
| Pages per session | 2+ | Internal linking success |
### SEO Metrics
| Metric | Target | What It Tells You |
|--------|--------|-------------------|
| Keyword rankings | Top 10 | Search visibility |
| Backlinks earned | +5/month | Content authority |
| Domain authority | Steady growth | Overall site strength |
| Featured snippets | Track position | SERP prominence |
---
## Engagement Metrics
Measure how audiences interact with content.
### Social Engagement
| Metric | Benchmark | Calculation |
|--------|-----------|-------------|
| Engagement rate | 1-3% (LinkedIn) | (Likes + Comments + Shares) / Impressions × 100 |
| Share rate | 0.5-1% | Shares / Reach × 100 |
| Save rate | 1-2% (Instagram) | Saves / Reach × 100 |
| Comment rate | 0.1-0.5% | Comments / Reach × 100 |
### Email Engagement
| Metric | Benchmark | What It Tells You |
|--------|-----------|-------------------|
| Open rate | 20-25% | Subject line effectiveness |
| Click-through rate | 2-5% | Content relevance |
| Unsubscribe rate | <0.5% | Audience fit |
| Forward rate | 0.1-0.3% | Content share-worthiness |
### Community Engagement
| Metric | What to Track |
|--------|---------------|
| Comments and discussions | Volume and sentiment |
| User-generated content | Submissions and quality |
| Community growth | New members per week |
| Active participation | % of members engaging |
---
## Business Metrics
Connect content performance to business outcomes.
### Lead Generation
| Metric | Calculation | Target |
|--------|-------------|--------|
| Content-attributed leads | Leads from content CTAs | Track by content piece |
| Form submissions | Total completions | +5% MoM |
| Lead quality score | MQL/total leads | 30%+ MQL rate |
| Cost per lead | Spend / Leads | Below industry average |
### Conversion Metrics
| Metric | Calculation | Target |
|--------|-------------|--------|
| Conversion rate | Conversions / Visitors × 100 | 2-5% |
| Revenue attribution | $ tied to content | Track by piece |
| Customer acquisition cost | Total cost / New customers | Decreasing trend |
| Content ROI | (Revenue - Cost) / Cost × 100 | 300%+ for evergreen |
### Customer Metrics
| Metric | What It Tells You |
|--------|-------------------|
| Customer lifetime value | Long-term content impact |
| Retention rate | Content's nurturing effectiveness |
| NPS from content consumers | Content quality perception |
| Support ticket reduction | Educational content success |
---
## Platform-Specific Analytics
### Blog Analytics (Google Analytics 4)
**Key Reports:**
- Landing pages report: Top entry content
- Engagement report: Time, bounces, conversions
- Traffic acquisition: Content discovery sources
- User paths: Content journey mapping
**Dimensions to Track:**
- Page path
- Source/medium
- Device category
- User type (new vs returning)
### Social Media Analytics
**LinkedIn:**
- Post impressions and reach
- Follower demographics
- Click-through rate on links
- Article read time
**Twitter/X:**
- Impressions and engagements
- Profile visits from tweets
- Link clicks
- Follower growth rate
**Instagram:**
- Reach vs impressions
- Saves and shares (high-value signals)
- Story completion rate
- Reel performance vs feed
### Email Analytics
**Track per Campaign:**
- Send volume and deliverability
- Open and click rates by segment
- Conversion path from email
- List growth and churn
---
## Reporting Frameworks
### Weekly Content Report
```
WEEK OF: [Date Range]
TOP PERFORMERS
1. [Content Title] - [Key Metric]
2. [Content Title] - [Key Metric]
3. [Content Title] - [Key Metric]
TRAFFIC SUMMARY
- Total sessions: [#]
- Organic traffic: [#] ([+/-]% WoW)
- Social traffic: [#] ([+/-]% WoW)
ENGAGEMENT HIGHLIGHTS
- Avg engagement rate: [%]
- Total comments: [#]
- Shares: [#]
LEADS GENERATED
- Content-attributed: [#]
- Top converting piece: [Title]
NEXT WEEK PRIORITIES
1. [Action item]
2. [Action item]
```
### Monthly Content Report
```
MONTH: [Month Year]
EXECUTIVE SUMMARY
[2-3 sentences on overall performance]
CONTENT PRODUCTION
- Published: [#] pieces
- By type: [Blog: #, Social: #, Email: #]
- On schedule: [Yes/No]
PERFORMANCE DASHBOARD
| Metric | This Month | Last Month | Change |
|---------------------|------------|------------|--------|
| Total traffic | | | |
| Organic traffic | | | |
| Engagement rate | | | |
| Leads generated | | | |
| Conversion rate | | | |
TOP 5 CONTENT PIECES
[Ranked by primary KPI]
INSIGHTS & LEARNINGS
- What worked: [observation]
- What didn't: [observation]
- Opportunities: [observation]
NEXT MONTH FOCUS
1. [Strategic priority]
2. [Content initiative]
3. [Optimization goal]
```
### Quarterly Business Review
```
Q[#] [Year] CONTENT PERFORMANCE
STRATEGIC ALIGNMENT
- Business goal: [Goal]
- Content contribution: [How content supported]
QUARTERLY METRICS
| KPI | Target | Actual | Status |
|------------------------|--------|--------|--------|
| Traffic growth | | | |
| Lead generation | | | |
| Conversion rate | | | |
| Revenue attribution | | | |
CONTENT AUDIT RESULTS
- Total pieces published: [#]
- High performers: [#]
- Needs optimization: [#]
- Candidates for retirement: [#]
ROI ANALYSIS
- Total content investment: $[X]
- Attributed revenue: $[Y]
- Content ROI: [%]
COMPETITIVE ANALYSIS
[How content stacks against competitors]
NEXT QUARTER ROADMAP
[Strategic initiatives and targets]
```
---
## Attribution Models
### First-Touch Attribution
**Use When:** Measuring top-of-funnel content effectiveness
**How It Works:** Credits the first content piece that brought a user in
**Best For:**
- Brand awareness campaigns
- SEO content performance
- Social media reach measurement
### Last-Touch Attribution
**Use When:** Measuring bottom-of-funnel conversion content
**How It Works:** Credits the last content before conversion
**Best For:**
- Product pages
- Case studies
- Demo request pages
### Multi-Touch Attribution
**Use When:** Understanding full content journey impact
**Linear Model:**
- Equal credit to all touchpoints
- Simple but may over-credit low-value touches
**Time-Decay Model:**
- More credit to recent touches
- Good for short sales cycles
**Position-Based Model:**
- 40% first touch, 40% last touch, 20% middle
- Balanced view of journey
### Content-Specific Attribution
**For Blog Content:**
1. Track assisted conversions in GA4
2. Map content to funnel stage
3. Weight by stage importance
**For Social Content:**
1. Use UTM parameters consistently
2. Track view-through conversions
3. Monitor social-assisted conversions
**For Email Content:**
1. Track email-attributed revenue
2. Monitor nurture sequence effectiveness
3. Measure reactivation campaigns
---
## Analytics Setup Checklist
### Essential Tracking
- [ ] Google Analytics 4 configured
- [ ] Conversion events defined
- [ ] UTM parameter system documented
- [ ] Social pixel tracking enabled
- [ ] Email tracking integrated
- [ ] CRM connected for lead tracking
### Advanced Setup
- [ ] Enhanced ecommerce tracking
- [ ] Custom dimensions for content attributes
- [ ] Automated reporting dashboards
- [ ] A/B testing infrastructure
- [ ] Heat mapping tools (Hotjar, Clarity)
- [ ] Attribution model configured
### Data Governance
- [ ] Naming conventions documented
- [ ] Data retention policies set
- [ ] Privacy compliance verified
- [ ] Access controls configured
- [ ] Regular data audits scheduled
FILE:references/brand_guidelines.md
# Brand Voice & Style Guidelines
Comprehensive framework for establishing and maintaining consistent brand voice across all content.
---
## Table of Contents
- [Voice Dimensions](#1-voice-dimensions)
- [Brand Personality Archetypes](#2-brand-personality-archetypes)
- [Writing Principles](#3-writing-principles)
- [Language Guidelines](#4-language-guidelines)
- [Content Structure Templates](#5-content-structure-templates)
- [Messaging Pillars](#6-messaging-pillars)
- [Audience Personas](#7-audience-personas)
- [Channel-Specific Guidelines](#8-channel-specific-guidelines)
- [Grammar & Mechanics](#9-grammar--mechanics)
- [Inclusivity Guidelines](#10-inclusivity-guidelines)
- [Quick Reference Checklist](#quick-reference-checklist)
---
## Brand Voice Framework
### 1. Voice Dimensions
#### Formality Spectrum
- **Formal**: Legal documents, investor communications, crisis responses
- **Professional**: B2B content, whitepapers, case studies
- **Conversational**: Blog posts, social media, email newsletters
- **Casual**: Community engagement, behind-the-scenes content
#### Tone Attributes
Choose 3-5 primary attributes for your brand:
- **Authoritative**: Position as industry expert
- **Friendly**: Approachable and warm
- **Innovative**: Forward-thinking and creative
- **Trustworthy**: Reliable and transparent
- **Inspiring**: Motivational and uplifting
- **Educational**: Informative and helpful
- **Witty**: Clever and entertaining (use sparingly)
#### Perspective
- **First Person Plural (We/Our)**: Creates partnership feeling
- **Second Person (You/Your)**: Direct and engaging
- **Third Person**: Objective and professional
### 2. Brand Personality Archetypes
Choose one primary and one secondary archetype:
**The Expert**
- Tone: Knowledgeable, confident, informative
- Content: Data-driven, research-backed, educational
- Example: "Our research shows that 87% of businesses..."
**The Friend**
- Tone: Warm, supportive, conversational
- Content: Relatable, helpful, encouraging
- Example: "We get it - marketing can be overwhelming..."
**The Innovator**
- Tone: Visionary, bold, forward-thinking
- Content: Cutting-edge, disruptive, trendsetting
- Example: "The future of marketing is here..."
**The Guide**
- Tone: Wise, patient, instructive
- Content: Step-by-step, clear, actionable
- Example: "Let's walk through this together..."
**The Motivator**
- Tone: Energetic, positive, inspiring
- Content: Empowering, action-oriented, transformative
- Example: "You have the power to transform your business..."
### 3. Writing Principles
#### Clarity First
- Use simple words when possible
- Break complex ideas into digestible pieces
- Lead with the main point
- Use active voice (80% of the time)
#### Customer-Centric
- Focus on benefits, not features
- Address pain points directly
- Use "you" more than "we"
- Include customer success stories
#### Consistency
- Maintain voice across all channels
- Use approved terminology
- Follow formatting standards
- Apply style rules uniformly
### 4. Language Guidelines
#### Words We Use
- **Action verbs**: Transform, accelerate, optimize, unlock, elevate
- **Positive descriptors**: Seamless, powerful, intuitive, strategic
- **Outcome-focused**: Results, growth, success, impact, ROI
#### Words We Avoid
- **Jargon**: Synergy, leverage (as verb), bandwidth (for availability)
- **Overused**: Innovative, disruptive, cutting-edge (unless truly applicable)
- **Weak**: Very, really, just, maybe, hopefully
- **Negative**: Can't, won't, impossible, problem (use "challenge")
### 5. Content Structure Templates
#### Blog Post Structure
1. **Hook** (1-2 sentences): Grab attention with a question, statistic, or bold statement
2. **Context** (1 paragraph): Explain why this matters now
3. **Main Content** (3-5 sections): Deliver value with clear subheadings
4. **Conclusion** (1 paragraph): Summarize key points
5. **Call to Action**: Clear next step for readers
#### Social Media Framework
- **LinkedIn**: Professional insights, industry news, thought leadership
- **Twitter/X**: Quick tips, engaging questions, thread stories
- **Instagram**: Visual storytelling, behind-the-scenes, inspiration
- **Facebook**: Community building, longer narratives, events
### 6. Messaging Pillars
Define 3-4 core themes that appear consistently:
1. **Innovation & Technology**
- AI-powered solutions
- Data-driven insights
- Future-ready strategies
2. **Customer Success**
- Real results and ROI
- Partnership approach
- Tailored solutions
3. **Expertise & Trust**
- Industry leadership
- Proven methodologies
- Transparent communication
4. **Growth & Transformation**
- Scaling businesses
- Digital transformation
- Continuous improvement
### 7. Audience Personas
#### Decision Makers (C-Suite)
- **Tone**: Professional, strategic, ROI-focused
- **Content**: High-level insights, business impact, competitive advantages
- **Pain Points**: Growth, efficiency, competition
#### Practitioners (Marketing Managers)
- **Tone**: Practical, supportive, educational
- **Content**: How-to guides, best practices, tools
- **Pain Points**: Time, resources, skills
#### Innovators (Early Adopters)
- **Tone**: Exciting, cutting-edge, visionary
- **Content**: Trends, new features, future predictions
- **Pain Points**: Staying ahead, differentiation
### 8. Channel-Specific Guidelines
#### Website Copy
- Headlines: 6-12 words, benefit-focused
- Body: Short paragraphs (2-3 sentences)
- CTAs: Action-oriented, specific
#### Email Marketing
- Subject Lines: 30-50 characters, personalized
- Preview Text: Complement subject, add urgency
- Body: Scannable, one main message
#### Blog Content
- Title: Include primary keyword, under 60 characters
- Introduction: Hook within first 50 words
- Sections: 200-300 words each
- Lists: 5-7 items optimal
### 9. Grammar & Mechanics
#### Punctuation
- Oxford comma: Always use
- Em dashes: For emphasis—like this
- Exclamation points: Maximum one per piece
#### Capitalization
- Headlines: Title Case for H1, Sentence case for H2-H6
- Product names: As trademarked
- Job titles: Lowercase unless before name
#### Numbers
- Spell out one through nine
- Use numerals for 10 and above
- Always use numerals for percentages
### 10. Inclusivity Guidelines
- Use gender-neutral language
- Avoid idioms that don't translate
- Consider global audience
- Ensure accessibility in formatting
- Represent diverse perspectives
## Quick Reference Checklist
Before publishing any content, verify:
- [ ] Matches brand voice and tone
- [ ] Free of jargon and complex terms
- [ ] Includes clear value proposition
- [ ] Has appropriate CTA
- [ ] Follows grammar guidelines
- [ ] Mobile-friendly formatting
- [ ] Accessible to all audiences
- [ ] Proofread and fact-checked
FILE:references/content_frameworks.md
# Content Creation Frameworks & Templates
Ready-to-use templates for blog posts, social media, email marketing, video scripts, and content planning.
---
## Table of Contents
- [Blog Post Templates](#1-blog-post-templates)
- [Social Media Templates](#2-social-media-templates)
- [Email Marketing Templates](#3-email-marketing-templates)
- [Content Planning Frameworks](#4-content-planning-frameworks)
- [SEO Content Framework](#5-seo-content-framework)
- [Video Script Templates](#6-video-script-templates)
- [Content Repurposing Matrix](#7-content-repurposing-matrix)
- [Quick-Start Checklists](#quick-start-checklists)
---
## Content Types & Templates
### 1. Blog Post Templates
#### How-To Guide Template
```markdown
# How to [Achieve Desired Outcome] in [Timeframe]
## Introduction
- Hook: Question or surprising fact
- Problem statement
- What reader will learn
- Why it matters now
## Prerequisites/What You'll Need
- Tool/Resource 1
- Tool/Resource 2
- Estimated time
## Step 1: [Action]
- Clear instruction
- Why this step matters
- Common mistakes to avoid
- Visual aid or example
## Step 2: [Action]
[Repeat structure]
## Step 3: [Action]
[Repeat structure]
## Troubleshooting Common Issues
### Issue 1: [Problem]
**Solution**: [Fix]
### Issue 2: [Problem]
**Solution**: [Fix]
## Results You Can Expect
- Immediate outcomes
- Long-term benefits
- Success metrics
## Next Steps
- Advanced techniques
- Related guides
- CTA for product/service
## Conclusion
- Recap key points
- Reinforce value
- Final encouragement
```
#### Listicle Template
```markdown
# [Number] [Adjective] Ways to [Achieve Goal] in [Year]
## Introduction
- Context/trend driving this topic
- Promise of what reader gains
- Credibility statement
## 1. [First Item - Most Important]
**Why it matters**: [Brief explanation]
**How to implement**: [2-3 actionable steps]
**Pro tip**: [Expert insight]
**Example**: [Real-world application]
## 2. [Second Item]
[Repeat structure]
[Continue for all items]
## Bonus Tip: [Overdelivery]
[Something extra valuable]
## Bringing It All Together
- How items work synergistically
- Priority order for implementation
- Expected timeline for results
## Your Action Plan
1. Start with [easiest item]
2. Progress to [next steps]
3. Measure [metrics]
## Conclusion & CTA
```
#### Case Study Template
```markdown
# How [Company] Achieved [Result] Using [Solution]
## Executive Summary
- Company overview
- Challenge faced
- Solution implemented
- Key results (3 metrics)
## The Challenge
### Background
- Industry context
- Company situation
- Previous attempts
### Specific Pain Points
- Pain point 1
- Pain point 2
- Pain point 3
## The Solution
### Strategy Development
- Discovery process
- Strategic approach
- Why this solution
### Implementation
- Phase 1: [Timeline & Actions]
- Phase 2: [Timeline & Actions]
- Phase 3: [Timeline & Actions]
## The Results
### Quantitative Outcomes
- Metric 1: X% increase
- Metric 2: $Y saved
- Metric 3: Z improvement
### Qualitative Benefits
- Team feedback
- Customer response
- Market position
## Key Takeaways
1. Lesson learned
2. Best practice discovered
3. Unexpected benefit
## Achieving Similar Results
- Prerequisite conditions
- Implementation roadmap
- Success factors
## CTA: Start Your Success Story
```
#### Thought Leadership Template
```markdown
# [Provocative Statement About Industry Future]
## The Current State
- Industry snapshot
- Prevailing wisdom
- Why status quo is insufficient
## The Emerging Trend
### What's Changing
- Driver 1: [Technology/Market/Behavior]
- Driver 2: [Technology/Market/Behavior]
- Driver 3: [Technology/Market/Behavior]
### Evidence & Examples
- Data point 1
- Case example
- Expert validation
## Implications for [Industry]
### Short-term (6-12 months)
- Immediate adjustments needed
- Quick wins available
- Risks of inaction
### Long-term (2-5 years)
- Fundamental shifts
- New opportunities
- Competitive landscape
## Strategic Recommendations
### For Leaders
- Strategic priorities
- Investment areas
- Organizational changes
### For Practitioners
- Skill development
- Process adaptation
- Tool adoption
## The Path Forward
- Call for industry action
- Your organization's role
- Next steps for readers
## Join the Conversation
- Thought-provoking question
- Invitation to share perspectives
- CTA for deeper engagement
```
### 2. Social Media Templates
#### LinkedIn Post Framework
```
🎯 Hook/Pattern Interrupt
Context paragraph explaining the situation or challenge.
Key insight or lesson learned:
• Bullet point 1 (specific detail)
• Bullet point 2 (measurable outcome)
• Bullet point 3 (unexpected discovery)
Brief story or example that illustrates the point.
Takeaway message with clear value.
Question to encourage engagement?
#Hashtag1 #Hashtag2 #Hashtag3
```
#### Twitter/X Thread Template
```
1/ Bold opening statement or question that stops the scroll
2/ Context - why this matters right now
3/ Problem most people face
4/ Conventional solution (and why it falls short)
5/ Better approach - introduction
6/ Step 1 of better approach
• Specific action
• Why it works
7/ Step 2 of better approach
[Continue pattern]
8/ Real example or case study
9/ Common objection addressed
10/ Results you can expect
11/ One powerful tip most people miss
12/ Recap in 3 key points:
- Point 1
- Point 2
- Point 3
13/ CTA: If you found this helpful, [action]
14/ P.S. - Bonus insight or resource
```
#### Instagram Caption Template
```
[Attention-grabbing first line - appears in preview]
[Story or relatable scenario - 2-3 sentences]
Here's what I learned:
[Key insight or lesson]
3 things that changed everything:
1️⃣ [First point]
2️⃣ [Second point]
3️⃣ [Third point]
[Call-out or question to audience]
Drop a [emoji] if you've experienced this too!
What's your biggest challenge with [topic]? Let me know below 👇
-
#hashtag1 #hashtag2 #hashtag3 #hashtag4 #hashtag5
[10-30 relevant hashtags total]
```
### 3. Email Marketing Templates
#### Newsletter Template
```
Subject: [Benefit] + [Urgency/Curiosity]
Preview: [Complements subject, doesn't repeat]
Hi [Name],
[Personal observation or timely hook - 1-2 sentences]
[Transition to main topic - why reading this matters]
## Main Content Section
[Key points in scannable format]
• Point 1: [Benefit-focused]
• Point 2: [Specific example]
• Point 3: [Actionable tip]
[Brief elaboration on most important point - 2-3 sentences]
## Resource of the Week
[Title with link]
[One sentence on why it's valuable]
## Quick Win You Can Implement Today
[Specific, actionable tip - 2-3 steps max]
[Closing thought or question]
[Signature]
[Name]
P.S. [Additional value or soft CTA]
```
#### Promotional Email Template
```
Subject: [Specific benefit] by [deadline/timeframe]
Preview: [Scarcity or exclusivity element]
Hi [Name],
[Acknowledge pain point or aspiration]
[Agitate - why this problem persists]
I've got something that can help:
[Solution introduction - what it is]
Here's what you get:
✓ Benefit 1 (not feature)
✓ Benefit 2 (not feature)
✓ Benefit 3 (not feature)
[Social proof - testimonial or results]
[Handle main objection]
[Clear CTA button: "Get Started" / "Claim Yours"]
[Urgency element - deadline or limited availability]
[Signature]
P.S. [Reinforce urgency or add bonus]
```
### 4. Content Planning Frameworks
#### Content Pillar Strategy
```
Pillar 1: Educational (40%)
- How-to guides
- Tutorials
- Best practices
- Tips & tricks
Pillar 2: Inspirational (25%)
- Success stories
- Case studies
- Transformations
- Vision pieces
Pillar 3: Conversational (25%)
- Behind-the-scenes
- Team spotlights
- Q&As
- Polls/questions
Pillar 4: Promotional (10%)
- Product updates
- Offers
- Event announcements
- CTAs
```
#### Monthly Content Calendar Structure
```
Week 1:
- Monday: Educational (blog post)
- Wednesday: Inspirational (social)
- Friday: Conversational (email)
Week 2:
- Monday: Educational (video/guide)
- Wednesday: Case study
- Friday: Curated content
Week 3:
- Monday: Educational (infographic)
- Wednesday: Behind-the-scenes
- Friday: Community spotlight
Week 4:
- Monday: Monthly roundup
- Wednesday: Thought leadership
- Friday: Promotional
```
### 5. SEO Content Framework
#### SEO-Optimized Article Structure
```
URL: /primary-keyword-secondary-keyword
Title Tag: Primary Keyword - Secondary Benefit | Brand
Meta Description: Action verb + primary keyword + benefit + CTA (155 chars)
# H1: Primary Keyword + Unique Angle
Introduction (50-100 words)
- Include primary keyword in first 100 words
- State what reader will learn
- Why it matters
## H2: Secondary Keyword Variation 1
[Content with LSI keywords naturally integrated]
### H3: Specific subtopic
- Detail point 1
- Detail point 2
- Detail point 3
## H2: Secondary Keyword Variation 2
[Content continues...]
## H2: Related Questions (FAQ Schema)
### Question 1?
[Concise answer with keyword]
### Question 2?
[Concise answer with keyword]
## Conclusion
- Recap main points
- Include primary keyword
- Clear next action
Internal Links: 2-3 relevant articles
External Links: 1-2 authoritative sources
```
### 6. Video Script Templates
#### Educational Video Script
```
[0-5 seconds: Hook]
"What if I told you [surprising statement]?"
[5-15 seconds: Introduction]
"Hi, I'm [Name] and today we're solving [problem]"
[15-30 seconds: Context]
- Why this matters
- What you'll learn
- What you'll achieve
[30 seconds - 2 minutes: Main Content]
Section 1: [Key Point]
- Explanation
- Example
- Visual aid
Section 2: [Key Point]
[Repeat structure]
Section 3: [Key Point]
[Repeat structure]
[Final 15-30 seconds]
- Quick recap
- Call to action
- End screen elements
```
### 7. Content Repurposing Matrix
```
Original: Blog Post (2000 words)
├── Social Media
│ ├── 5 Twitter posts (key quotes)
│ ├── 1 LinkedIn article (executive summary)
│ ├── 3 Instagram carousels (main points)
│ └── 1 Facebook post (intro + link)
├── Email
│ └── Newsletter feature (summary + CTA)
├── Video
│ ├── YouTube explainer (script from post)
│ └── TikTok/Reels (quick tips)
├── Audio
│ └── Podcast talking points
└── Visual
├── Infographic (data points)
└── Slide deck (presentation)
```
## Quick-Start Checklists
### Pre-Publishing Checklist
- [ ] Keyword research completed
- [ ] Title under 60 characters
- [ ] Meta description written (155 chars)
- [ ] Headers properly structured (H1, H2, H3)
- [ ] Internal links added (2-3)
- [ ] Images optimized with alt text
- [ ] CTA included and clear
- [ ] Proofread and fact-checked
- [ ] Mobile preview checked
### Content Quality Checklist
- [ ] Addresses specific audience need
- [ ] Provides unique value/perspective
- [ ] Includes actionable takeaways
- [ ] Uses appropriate brand voice
- [ ] Contains supporting data/examples
- [ ] Free of jargon and complex terms
- [ ] Scannable format (bullets, headers)
- [ ] Engaging hook in introduction
- [ ] Clear conclusion and next steps
FILE:references/social_media_optimization.md
# Social Media Optimization Guide
Platform-specific best practices, algorithm factors, content optimization strategies, and analytics frameworks.
---
## Table of Contents
- [Platform-Specific Best Practices](#platform-specific-best-practices)
- [LinkedIn](#linkedin)
- [Twitter/X](#twitterx)
- [Instagram](#instagram)
- [Facebook](#facebook)
- [TikTok](#tiktok)
- [Content Optimization Strategies](#content-optimization-strategies)
- [Hashtag Strategy](#hashtag-strategy)
- [Visual Content Optimization](#visual-content-optimization)
- [Caption Writing Formulas](#caption-writing-formulas)
- [Engagement Tactics](#engagement-tactics)
- [Analytics & KPIs](#analytics--kpis)
- [Content Calendar Planning](#content-calendar-planning)
- [Crisis Management Protocol](#crisis-management-protocol)
- [Tool Stack Recommendations](#tool-stack-recommendations)
- [Compliance & Best Practices](#compliance--best-practices)
---
## Platform-Specific Best Practices
### LinkedIn
**Audience**: B2B professionals, decision-makers, thought leaders
**Best Times**: Tuesday-Thursday, 8-10 AM and 5-6 PM
**Optimal Length**: 1,300-2,000 characters for posts
#### Content Formats
- **Text Posts**: 1,300 characters optimal, use line breaks
- **Articles**: 1,900-2,000 words, include 5+ images
- **Videos**: 30 seconds - 10 minutes, native upload preferred
- **Documents**: PDF carousels, 10-15 slides
- **Polls**: 4 options max, 1-2 week duration
#### Optimization Tips
- First 2 lines are crucial (shown in preview)
- Use emoji sparingly for visual breaks
- Include 3-5 relevant hashtags
- Tag people and companies when relevant
- Native video gets 5x more engagement
- Post consistently (3-5x per week optimal)
#### Algorithm Factors
- Dwell time (time spent reading)
- Comments valued over likes
- Early engagement (first hour) crucial
- Creator mode boosts reach
- Replies to comments increase visibility
### Twitter/X
**Audience**: News junkies, tech enthusiasts, real-time conversation
**Best Times**: Weekdays 9-10 AM and 7-9 PM
**Optimal Length**: 100-250 characters
#### Content Formats
- **Single Tweets**: 250 characters, 1-2 hashtags
- **Threads**: 5-15 tweets, numbered format
- **Images**: 16:9 ratio, up to 4 per tweet
- **Videos**: Up to 2:20, square or landscape
- **Polls**: 2-4 options, 5 minutes - 7 days
#### Optimization Tips
- Front-load important information
- Use threads for complex topics
- Include visuals (2-3x more engagement)
- Retweet with comment > regular RT
- Schedule threads for consistency
- Engage genuinely with replies
#### Algorithm Factors
- Engagement rate (likes, RTs, replies)
- Relationship (mutual follows prioritized)
- Recency over evergreen
- Topic relevance to user interests
- Link posts receive less reach
### Instagram
**Audience**: Visual-first, millennials & Gen Z, lifestyle focused
**Best Times**: Weekdays 11 AM - 1 PM and 7-9 PM
**Optimal Length**: 138-150 characters shown in preview
#### Content Formats
- **Feed Posts**: Square (1:1) or vertical (4:5)
- **Stories**: 15 seconds max, vertical (9:16)
- **Reels**: 15-90 seconds, vertical (9:16)
- **Carousels**: 2-10 images/videos
- **IGTV/Video**: 1-60 minutes
#### Optimization Tips
- First sentence crucial (caption preview)
- Use up to 30 hashtags (5-10 in caption, rest in comment)
- Carousel posts get highest engagement
- Stories with polls/questions boost views
- Reels get maximum organic reach
- Post consistently (1-2 feed posts daily)
#### Algorithm Factors
- Relationship (DMs, comments, tags)
- Interest (based on past interactions)
- Timeliness (newer posts prioritized)
- Frequency of app usage
- Time spent on posts (saves valuable)
### Facebook
**Audience**: Broad demographic, community-focused, local businesses
**Best Times**: Wednesday-Friday, 11 AM - 2 PM
**Optimal Length**: 50-80 characters for posts
#### Content Formats
- **Text Posts**: 50-80 characters optimal
- **Images**: 1200x630px for links
- **Videos**: 1-3 minutes, square format
- **Stories**: Same as Instagram
- **Live Videos**: Minimum 10 minutes
#### Optimization Tips
- Native video gets priority
- Ask questions to boost comments
- Share to relevant groups
- Use Facebook Creator Studio
- Tag locations for local reach
- Post 1-2 times per day max
#### Algorithm Factors
- Meaningful interactions (comments > reactions)
- Video completion rate
- Friends and family prioritized
- Group posts get high visibility
- Live videos get 6x engagement
### TikTok
**Audience**: Gen Z, entertainment-focused, trend-driven
**Best Times**: 6-10 AM and 7-11 PM
**Optimal Length**: 15-30 seconds
#### Content Formats
- **Videos**: 15 seconds - 10 minutes
- **Aspect Ratio**: 9:16 vertical
- **Sounds**: Trending audio crucial
- **Effects**: Filters and transitions
#### Optimization Tips
- Hook viewers in first 3 seconds
- Use trending sounds and hashtags
- Create content for FYP, not followers
- Post 1-4 times daily
- Engage with comments quickly
- Jump on trends within 24-48 hours
#### Algorithm Factors
- Completion rate most important
- Shares and saves valued
- Comment engagement
- Following similar creators
- Time spent on app
## Content Optimization Strategies
### Hashtag Strategy
#### Research Methods
1. **Competitor Analysis**: Study successful competitors
2. **Platform Search**: Use native search for suggestions
3. **Hashtag Tools**: RiteTag, Hashtagify, All Hashtag
4. **Trending Topics**: Monitor daily/weekly trends
5. **Brand Hashtags**: Create unique campaign tags
#### Hashtag Mix Formula
- 30% High-volume (1M+ posts)
- 40% Medium-volume (100K-1M posts)
- 30% Low-volume/Niche (<100K posts)
#### Platform-Specific Guidelines
- **Instagram**: 10-30 hashtags (mix in caption and first comment)
- **LinkedIn**: 3-5 professional hashtags
- **Twitter**: 1-2 hashtags max
- **Facebook**: 1-3 hashtags
- **TikTok**: 3-5 trending + niche tags
### Visual Content Optimization
#### Image Best Practices
- **Resolution**: Minimum 1080px width
- **File Size**: Under 5MB for faster loading
- **Alt Text**: Always include for accessibility
- **Branding**: Consistent filters/overlays
- **Text Overlay**: Less than 20% of image
#### Video Optimization
- **Captions**: Always include (85% watch without sound)
- **Thumbnail**: Custom, eye-catching
- **Length**: Platform-specific optimal duration
- **Format**: MP4 for best compatibility
- **Aspect Ratio**: Vertical for stories/reels, square for feed
### Caption Writing Formulas
#### AIDA Formula
- **Attention**: Hook in first line
- **Interest**: Expand on the hook
- **Desire**: Benefits and value
- **Action**: Clear CTA
#### PAS Formula
- **Problem**: Identify pain point
- **Agitate**: Emphasize consequences
- **Solution**: Present your answer
#### Before-After-Bridge
- **Before**: Current situation
- **After**: Desired outcome
- **Bridge**: How to get there
### Engagement Tactics
#### Conversation Starters
- Ask open-ended questions
- Create polls and surveys
- "Fill in the blank" posts
- "This or that" choices
- Caption contests
- Opinion requests
#### Community Building
- Respond to comments within 2 hours
- Like and reply to user comments
- Share user-generated content
- Create branded hashtags
- Host Q&A sessions
- Run challenges or contests
### Analytics & KPIs
#### Vanity Metrics (Track but don't obsess)
- Follower count
- Like count
- View count
#### Performance Metrics (Focus here)
- Engagement rate: (Likes + Comments + Shares) / Reach × 100
- Click-through rate: Clicks / Impressions × 100
- Conversion rate: Conversions / Clicks × 100
- Share/Save rate: Shares / Reach × 100
#### Business Metrics (Ultimate goal)
- Website traffic from social
- Lead generation
- Sales attribution
- Customer acquisition cost
- Customer lifetime value
### Content Calendar Planning
#### Weekly Posting Schedule Template
```
Monday: Motivational (Quote/Inspiration)
Tuesday: Educational (How-to/Tips)
Wednesday: Promotional (Product/Service)
Thursday: Engaging (Poll/Question)
Friday: Fun (Behind-scenes/Casual)
Saturday: User-Generated Content
Sunday: Curated Content/Rest
```
#### Monthly Theme Structure
- Week 1: Awareness content
- Week 2: Consideration content
- Week 3: Decision content
- Week 4: Retention/Community
### Crisis Management Protocol
#### Response Timeline
- **0-15 minutes**: Acknowledge awareness
- **15-60 minutes**: Gather facts
- **1-2 hours**: Official response
- **24 hours**: Follow-up update
- **48-72 hours**: Resolution summary
#### Response Guidelines
1. Acknowledge quickly
2. Take responsibility if appropriate
3. Show empathy
4. Provide facts only
5. Outline action steps
6. Follow up publicly
## Tool Stack Recommendations
### Content Creation
- **Design**: Canva, Adobe Creative Suite
- **Video**: CapCut, InShot, Adobe Premiere
- **Copy**: Grammarly, Hemingway Editor
- **AI Assistance**: ChatGPT, Claude, Jasper
### Scheduling & Management
- **All-in-One**: Hootsuite, Buffer, Sprout Social
- **Visual-First**: Later, Planoly
- **Enterprise**: Sprinklr, Khoros
- **Free Options**: Meta Business Suite, TweetDeck
### Analytics & Monitoring
- **Native**: Platform Insights/Analytics
- **Third-Party**: Socialbakers, Brandwatch
- **Listening**: Mention, Brand24
- **Competitor Analysis**: Social Blade, Rival IQ
### Influencer & UGC
- **Discovery**: AspireIQ, GRIN
- **Management**: CreatorIQ, Klear
- **UGC Curation**: TINT, Stackla
- **Rights Management**: Rights Manager
## Compliance & Best Practices
### Legal Considerations
- Include #ad or #sponsored for paid partnerships
- Respect copyright and attribution
- Follow GDPR for data collection
- Comply with platform terms of service
- Get permission for UGC usage
### Accessibility Guidelines
- Add alt text to all images
- Include captions on videos
- Use CamelCase for hashtags (#LikeThis)
- Avoid text-only images
- Ensure color contrast compliance
### Brand Safety
- Moderate comments regularly
- Set up keyword filters
- Have crisis management plan
- Monitor brand mentions
- Establish posting permissions
Hỗ trợ xây dựng và đánh giá hệ thống quản lý AI theo tiêu chuẩn ISO/IEC 42001.
../../../ra-qm-team/compliance-team-iso42001/skills/iso42001-specialist/SKILL.md
Khung vận hành công ty: chọn hệ điều hành (EOS, OKR...), sơ đồ trách nhiệm, scorecard, nhịp họp và mục tiêu 90 ngày.
---
name: "company-os"
description: "The meta-framework for how a company runs — the connective tissue between all C-suite roles. Covers operating system selection (EOS, Scaling Up, OKR-native, hybrid), accountability charts, scorecards, meeting pulse, issue resolution, and 90-day rocks. Use when setting up company operations, selecting a management framework, designing meeting rhythms, building accountability systems, implementing OKRs, or when user mentions EOS, Scaling Up, operating system, L10 meetings, rocks, scorecard, accountability chart, or quarterly planning."
license: MIT
metadata:
version: 1.0.0
author: Alireza Rezvani
category: c-level
domain: company-operations
updated: 2026-03-05
frameworks: os-comparison, implementation-guide
---
# Company Operating System
The operating system is the collection of tools, rhythms, and agreements that determine how the company functions. Every company has one — most just don't know what it is. Making it explicit makes it improvable.
## Keywords
operating system, EOS, Entrepreneurial Operating System, Scaling Up, Rockefeller Habits, OKR, Holacracy, L10 meeting, rocks, scorecard, accountability chart, issues list, IDS, meeting pulse, quarterly planning, weekly scorecard, management framework, company rhythm, traction, Gino Wickman, Verne Harnish
## Why This Matters
Most operational dysfunction isn't a people problem — it's a system problem. When:
- The same issues recur every week: no issue resolution system
- Meetings feel pointless: no structured meeting pulse
- Nobody knows who owns what: no accountability chart
- Quarterly goals slip: rocks aren't real commitments
Fix the system. The people will operate better inside it.
## The Six Core Components
Every effective operating system has these six, regardless of which framework you choose:
### 1. Accountability Chart
Not an org chart. An accountability chart answers: "Who owns this outcome?"
**Key distinction:** One person owns each function. Multiple people may work in it. Ownership means the buck stops with one person.
**Structure:**
```
CEO
├── Sales (CRO/VP Sales)
│ ├── Inbound pipeline
│ └── Outbound pipeline
├── Product & Engineering (CTO/CPO)
│ ├── Product roadmap
│ └── Engineering delivery
├── Operations (COO)
│ ├── Customer success
│ └── Finance & Legal
└── People (CHRO/VP People)
├── Recruiting
└── People operations
```
**Rules:**
- No shared ownership. "Alice and Bob both own it" means nobody owns it.
- One person can own multiple seats at early stages. That's fine. Just be explicit.
- Revisit quarterly as you scale. Ownership shifts as the company grows.
**Build it in a workshop:**
1. List all functions the company performs
2. Assign one owner per function — no exceptions
3. Identify gaps (functions nobody owns) and overlaps (functions two people think they own)
4. Publish it. Update it when something changes.
### 2. Scorecard
Weekly metrics that tell you if the company is on track. Not monthly. Not quarterly. Weekly.
**Rules:**
- 5–15 metrics maximum. More than 15 and nothing gets attention.
- Each metric has an owner and a weekly target (not a range — a number).
- Red/yellow/green status. Not paragraphs.
- The scorecard is discussed at the leadership team weekly meeting. Only red metrics get discussion time.
**Example scorecard structure:**
| Metric | Owner | Target | This Week | Status |
|--------|-------|--------|-----------|--------|
| New MRR | CRO | €50K | €43K | 🔴 |
| Churn | CS Lead | < 1% | 0.8% | 🟢 |
| Active users | CPO | 2,000 | 2,150 | 🟢 |
| Deployments | CTO | 3/week | 3 | 🟢 |
| Open critical bugs | CTO | 0 | 2 | 🔴 |
| Runway | CFO | > 18mo | 16mo | 🟡 |
**Anti-pattern:** Measuring everything. If you track 40 KPIs, you're watching, not managing.
### 3. Meeting Pulse
The meeting rhythm that drives the company. Not optional — the pulse is what keeps the company alive.
**The full rhythm:**
| Meeting | Frequency | Duration | Who | Purpose |
|---------|-----------|----------|-----|---------|
| Daily standup | Daily | 15 min | Each team | Blockers only |
| L10 / Leadership sync | Weekly | 90 min | Leadership team | Scorecard + issues |
| Department review | Monthly | 60 min | Dept + leadership | OKR progress |
| Quarterly planning | Quarterly | 1–2 days | Leadership | Set rocks, review strategy |
| Annual planning | Annual | 2–3 days | Leadership | 1-year + 3-year vision |
**The L10 meeting (Weekly Leadership Sync):**
Named for the goal of each meeting being a 10/10. Fixed agenda:
1. Good news (5 min) — personal + business
2. Scorecard review (5 min) — flag red items only
3. Rock review (5 min) — on/off track for each rock
4. Customer/employee headlines (5 min)
5. Issues list (60 min) — IDS (see below)
6. To-dos review (5 min) — last week's commitments
7. Conclude (5 min) — rate the meeting 1–10, what would make it a 10 next time
### 4. Issue Resolution (IDS)
The core problem-solving loop. Maximum 15 minutes per issue.
**IDS: Identify, Discuss, Solve**
- **Identify:** What is the actual issue? (Not the symptom — the root cause) State it in one sentence.
- **Discuss:** Relevant facts + perspectives. Time-boxed. When discussion starts repeating, stop.
- **Solve:** One owner. One action. One due date. Written on the to-do list.
**Anti-patterns:**
- "Let's take this offline" — most things taken offline never get resolved
- Discussing without deciding — a great discussion with no action item is wasted time
- Revisiting decided issues — once solved, it leaves the list. Reopen only with new information.
**The Issues List:** A running, prioritized list of all unresolved issues. Owned by the leadership team. Reviewed and pruned weekly. If an issue has been on the list for 3+ meetings and hasn't been discussed, it's either not a real issue or it's too scary to address — both deserve attention.
### 5. Rocks (90-Day Priorities)
Rocks are the 3–7 most important things each person must accomplish in the next 90 days. They're not the job description — they're the things that move the company forward.
**Why 90 days?** Long enough for meaningful progress. Short enough to stay real.
**Rock rules:**
- Each person: 3–7 rocks maximum. More than 7 and none get done.
- Company-level rocks (shared priorities): 3–7 for the leadership team
- Each rock is binary: done or not done. No "60% complete."
- Set at the quarterly planning session. Reviewed weekly (on/off track).
**Bad rock:** "Improve our sales process"
**Good rock:** "Implement Salesforce CRM with full pipeline stages and weekly reporting by March 31"
**Rock vs. to-do:** A to-do takes one action. A rock takes 90 days of consistent work.
### 6. Communication Cadence
Who gets what information, when, and how.
| Audience | What | When | Format |
|----------|------|------|--------|
| All employees | Company update | Monthly | Written + Q&A |
| All employees | Quarterly results + next priorities | Quarterly | All-hands |
| Leadership team | Scorecard | Weekly | Dashboard |
| Board | Company performance | Monthly | Board memo |
| Investors | Key metrics + narrative | Monthly or quarterly | Investor update |
| Customers | Product updates | Per release | Release notes |
**Default rule:** If you're deciding whether to share something internally, share it. The cost of under-communication always exceeds the cost of over-communication inside a company.
---
## Operating System Selection
See `references/os-comparison.md` for full comparison. Quick guide:
| If you are... | Consider... |
|---------------|-------------|
| 10–250 person company, founder-led, operational chaos | EOS / Traction |
| Ambitious growth company, need rigorous strategy cascade | Scaling Up |
| Tech company, engineering culture, hypothesis-driven | OKR-native |
| Decentralized, flat, high autonomy | Holacracy (only if you're patient) |
| None of the above quite fit | Custom hybrid |
---
## Implementation Roadmap
Don't implement everything at once. See `references/implementation-guide.md` for the full 90-day plan.
**Quick start (first 30 days):**
1. Build the accountability chart (1 workshop, 2 hours)
2. Define 5–10 weekly scorecard metrics (leadership team alignment, 1 hour)
3. Start the weekly L10 meeting (no prep — just start)
These three alone will improve coordination more than most companies achieve in a year.
---
## Common Failure Modes
**Partial implementation:** "We do OKRs but skip the weekly check-in." Half an operating system is worse than none — it creates theater without accountability.
**Meeting fatigue:** Adding the full rhythm on top of existing meetings. Start by replacing meetings, not adding them.
**Metric overload:** Starting with 30 KPIs because "they all matter." Start with 5. Add when the cadence is established.
**Rock inflation:** Setting 12 rocks per person because "everything is a priority." When everything is a priority, nothing is. Hard limit: 7.
**Leader non-compliance:** Leadership team skips the L10 or doesn't follow IDS. The operating system mirrors the respect leadership gives it. If leaders don't take it seriously, nobody will.
**Annual planning without quarterly review:** Setting annual goals and checking in at year-end. Quarterly is the minimum review cycle for any meaningful goal.
---
## Integration with C-Suite
The company OS is the connective tissue. Every other role depends on it:
| C-Suite Role | OS Dependency |
|-------------|---------------|
| CEO | Sets vision that feeds into 1-year plan and rocks |
| COO | Owns the meeting pulse and issue resolution cadence |
| CFO | Owns the financial metrics in the scorecard |
| CTO | Owns engineering rocks and tech scorecard metrics |
| CHRO | Owns people metrics (attrition, hiring velocity) in scorecard |
| Culture Architect | Culture rituals plug into the meeting pulse |
| Strategic Alignment Engine | Validates that team rocks cascade from company rocks |
---
## Key Questions for the Operating System
- "If I asked five different team leads what the company's top 3 priorities are this quarter, would they give the same answers?"
- "What was the most important issue raised in last week's leadership meeting? Was it resolved or is it still open?"
- "Name a metric that would tell us by Friday whether this week was a good week. Do we track it?"
- "Who owns customer churn? Can you name that person without hesitation?"
- "When was the last time we updated the accountability chart?"
## Detailed References
- `references/os-comparison.md` — EOS vs Scaling Up vs OKRs vs Holacracy vs hybrid
- `references/implementation-guide.md` — 90-day implementation plan
FILE:references/implementation-guide.md
# Company Operating System — 90-Day Implementation Guide
Don't implement everything at once. The fastest path to failure is trying to launch the full operating system in week one. Build incrementally. Let the team experience wins before adding complexity.
---
## Before You Start
### Prerequisites
**Leadership alignment (non-negotiable):**
Every member of the leadership team must understand why you're doing this and commit to running the system. One holdout destroys the whole model. If the CFO skips the L10 meetings, the system won't work.
**Current state audit:**
- What meetings currently exist? Which can be replaced?
- Who owns which functions today? (Even informally)
- What metrics are being tracked? (Even inconsistently)
**Assign an OS owner:**
One person is responsible for the implementation and ongoing maintenance of the operating system. Usually the COO or CEO (at smaller companies). This is not a committee job.
---
## Week 1–2: Accountability Chart + Scorecard
### Accountability Chart Workshop (Week 1)
**Duration:** 2–3 hours, full leadership team
**Step 1 — List all functions (30 min)**
On a whiteboard, list every function the company performs:
- Sales (inbound, outbound, partnerships)
- Marketing (content, paid, brand)
- Product (roadmap, design, research)
- Engineering (frontend, backend, devops)
- Customer success (onboarding, support, retention)
- Finance (accounting, FP&A, legal)
- People (recruiting, HR, culture)
- Operations (processes, tools, facilities)
**Step 2 — Assign owners (45 min)**
For each function: "Who is the one person ultimately accountable?" Write their name.
Rules: One name only. No joint ownership. One person can own multiple functions at small scale.
**Step 3 — Identify gaps and overlaps (30 min)**
- **Gaps:** Functions with no owner → Who should own them? Or do we need a hire?
- **Overlaps:** Two people said they own the same thing → Resolve now, not later.
**Step 4 — Publish and socialize (Week 2)**
Share with the full company. Explain what an accountability chart is and isn't.
"This is about clarity, not hierarchy. It tells everyone who to go to for each function."
**Output:** A documented accountability chart. Use a simple tool (Miro, Google Slides, Ninety.io).
---
### Scorecard Design (Week 2)
**Duration:** 90 minutes, leadership team
**Step 1 — List candidate metrics (30 min)**
Each leader lists 3–5 metrics they already track or wish they tracked. No filtering yet.
**Step 2 — Filter to 5–15 (30 min)**
Criteria: Is it measurable weekly? Does it tell us if the company is healthy? Does one person own it?
Drop: metrics that are monthly only, metrics without a clear owner, metrics that measure activity not outcomes.
**Step 3 — Set weekly targets (20 min)**
For each metric: what's the weekly target? Not a range — a number. Red/yellow/green thresholds.
**Step 4 — Assign owners (10 min)**
Every metric has one owner who is responsible for reporting it weekly.
**Output:** A scorecard document. 5–15 metrics, owner, target, weekly tracking column.
**First scorecard run:** Week 2 or 3. It won't be perfect. That's fine.
---
## Week 3–4: Meeting Pulse (Start With L10)
Don't start all the meetings at once. Start with the weekly L10. Replace existing leadership syncs.
### L10 Meeting Setup
**Schedule:** Same day, same time, every week. Non-negotiable attendance.
**Duration:** 90 minutes. No more, no less.
**Facilitator:** Rotate or assign to COO/CEO. The facilitator keeps time and follows the agenda.
**Fixed agenda:**
1. **Good news** (5 min) — One personal, one business from each person. No skipping.
2. **Scorecard review** (5 min) — Traffic light only. Red items go to the issues list.
3. **Rock review** (5 min) — Each person: "on track" or "off track." No justification needed at this step.
4. **Customer/employee headlines** (5 min) — One sentence each. No reports.
5. **Issues** (60 min) — IDS process. Prioritize the top 3–5 issues. Solve them.
6. **To-do review** (5 min) — Review last week's commitments (done/not done). No excuses, just data.
7. **Conclude** (5 min) — Rate the meeting 1–10. What would make next week better?
**First L10 meeting:**
It will feel awkward. Run through the agenda anyway. The team needs the repetition to internalize it. By week 4, it should feel natural.
### Issues List Setup
Create a shared document (Notion, Google Docs, or dedicated tool):
- Issue title
- Priority (High / Medium / Low)
- Status (Open / In progress / Solved)
- Owner (once assigned)
- Due date
At the first L10, generate the issues list by asking: "What's getting in our way right now?" Expect 10–20 items on the first pass.
---
## Week 5–8: Rocks and Quarterly Planning
### Quarterly Planning Session (end of Week 5 or start of Week 6)
**Duration:** 4–8 hours (or 2 × 4-hour days for larger teams)
**Who:** Full leadership team
**Session structure:**
**Part 1: Review previous quarter (60–90 min)**
- What rocks were completed? What were dropped?
- What did we learn?
- What changed in the market or company?
**Part 2: Confirm or update company direction (60 min)**
- Is the 1-year goal still valid?
- Any major strategy shifts needed?
- Update the V/TO or OPSP if using EOS or Scaling Up.
**Part 3: Set company rocks (90 min)**
- Brainstorm: What are the 3–7 most important things to accomplish this quarter?
- Prioritize. Be ruthless. 3 rocks done > 7 rocks started.
- Each rock: clear owner, clear definition of done, 90-day timeline.
**Part 4: Set individual rocks (60 min)**
- Each leader sets their 3–7 rocks (aligned with company rocks where possible)
- Share with group: dependencies? Conflicts? Overloaded people?
**Part 5: Communicate (Week 6)**
- Share company rocks with the full organization within 1 week
- Each team sets their own rocks, cascaded from company rocks (3–5 per team)
**Rock template:**
```
Rock: [What you'll accomplish]
Owner: [One person]
Due date: [Specific date within the quarter]
Definition of done: [How we'll know it's complete]
Dependencies: [What else needs to happen first]
```
---
## Week 9–12: Issue Resolution Mastery + Communication Cadence
By now the L10 should be running smoothly. Weeks 9–12 focus on deepening IDS skills and establishing the broader communication cadence.
### IDS Practice
The issue resolution process often degrades in weeks 5–8. Common problems:
- Issues discussed but never solved (no clear action item)
- Same issues recurring (root cause not addressed)
- Too many issues, not enough resolution (prioritization failing)
**IDS calibration exercise (Week 9):**
In the next L10, after each issue is "solved," ask:
- "Is this actually solved, or are we postponing it?"
- "What's the specific action? Who owns it? When is it due?"
- "Is this the real issue, or a symptom of something deeper?"
### Communication Cadence Setup
Build out the full communication calendar:
| Communication | Frequency | Owner | Format | Tool |
|---------------|-----------|-------|--------|------|
| Company all-hands | Monthly | CEO | Update + Q&A | Video call |
| Quarterly planning results | Quarterly | CEO/COO | Written + live | Notion + all-hands |
| Board update | Monthly | CEO + CFO | Board memo | Doc |
| Investor update | Monthly | CEO + CFO | Email | Template |
| Department L10s | Weekly | Dept lead | L10 format | In-person / Zoom |
| Daily standups | Daily | Team leads | 15 min | Team call |
**Company all-hands template:**
1. State of the company (financial health, key metrics) — 10 min
2. Quarterly rocks: what we committed to, where we stand — 10 min
3. Wins and recognitions — 5 min
4. What's coming next quarter — 10 min
5. Q&A — 15–25 min
---
## Post-90 Days: Refinement and Optimization
### Month 4 retrospective
After the first full quarter, run a retrospective on the operating system itself:
- What's working? What isn't?
- Which meetings should continue as-is? Which need adjustment?
- Is the scorecard measuring the right things?
- Are rocks the right size and specificity?
- What should we add next?
### Scorecard evolution
By month 4, you'll know which metrics matter most. Add 2–3 that are missing. Remove metrics that nobody uses for decisions.
### L10 health check
Rate your L10 meetings over the first quarter:
- Average rating < 7: The agenda isn't being followed or issues aren't being resolved. Diagnose.
- Average rating 7–8: Normal. Keep building discipline.
- Average rating > 8: The team is engaged. Start extending the system to department level.
### Department L10s (Month 4+)
Once leadership L10 is running well, cascade the meeting structure:
- Each department runs their own weekly L10
- Department rocks cascade from company rocks
- Issues that cross departments are escalated to leadership L10
### Year 1 annual planning
End of year 1: run a full-day annual planning session.
- Review the year: what did we accomplish? What did we miss? What did we learn?
- Update 3-year vision (has it changed?)
- Set next year's annual goals
- Set Q1 rocks
- Celebrate. Seriously — mark the milestone.
---
## Implementation Anti-Patterns
**Skipping the accountability chart:** Without ownership clarity, every other system breaks down. Do this first.
**Building a perfect scorecard before starting:** Start with 5 imperfect metrics. Improve over time.
**Not replacing existing meetings:** Adding L10 on top of 3 existing meetings creates meeting overload. Cancel the redundant ones.
**Leader non-participation:** If one leader consistently skips or is disengaged, the system won't work. Address this directly — it's a culture issue, not a calendar issue.
**Changing the L10 agenda:** The agenda works because of repetition. Resist the urge to customize it for the first 6 months.
**Rocks without accountability:** If nobody checks rocks at the L10 ("on track / off track"), they become wish lists. The weekly review is what makes them real.
FILE:references/os-comparison.md
# Operating System Comparison
Side-by-side analysis of the major company operating frameworks.
---
## Overview
| Framework | Origin | Best fit | Implementation time | Cost |
|-----------|--------|----------|---------------------|------|
| EOS | Gino Wickman, 2007 | 10–250 employees, founder-led | 2–3 years full adoption | Free (DIY) to $25K+/year (implementer) |
| Scaling Up | Verne Harnish, 2002 | Growth-stage, strategic focus | 1–2 years | Free (DIY) to $15K+/year (coach) |
| OKR-native | Andy Grove / Google | Tech companies, product orgs | 3–6 months | Free |
| Holacracy | Brian Robertson, 2007 | Flat, autonomous organizations | 2–4 years | $5K–$50K+ (certification) |
| Custom hybrid | You | When the above don't fit exactly | Ongoing | Whatever you invest |
---
## 1. EOS — Entrepreneurial Operating System
**Book:** *Traction* by Gino Wickman
### Core principles
EOS is built on Six Components:
1. **Vision** — Where are you going? (V/TO: Vision/Traction Organizer)
2. **People** — Right people, right seats
3. **Data** — Scorecard with weekly metrics
4. **Issues** — Surface and resolve with IDS
5. **Process** — Document core processes
6. **Traction** — Rocks + meeting pulse (L10)
### Signature tools
- **V/TO (Vision/Traction Organizer):** 2-page strategy doc. Core values, core focus, 10-year target, 3-year picture, 1-year plan, quarterly rocks, issues.
- **Accountability Chart:** Who owns what function (not org chart)
- **L10 meeting:** Weekly 90-minute leadership sync (Level 10 = aim for 10/10)
- **Rocks:** 90-day priority commitments (3–7 per person)
- **IDS:** Identify, Discuss, Solve (issue resolution, max 15 min per issue)
### Strengths
- **Operationally focused.** If your problem is execution chaos, EOS addresses it directly.
- **Accessible.** The book is practical. You can DIY it without a coach.
- **Community.** Large network of implementers, tools (Ninety.io, EOS Worldwide), and practitioners.
- **Simple enough to actually use.** No complex methodology. Most teams are functional within 6 months.
### Limitations
- **Strategic depth is shallow.** The V/TO is good for direction but doesn't replace real strategy work.
- **Doesn't scale beyond ~250.** Designed for entrepreneurial companies. Gets cumbersome at enterprise scale.
- **Assumes a cohesive leadership team.** If trust is broken at the top, EOS won't fix it.
- **Facilitator dependency.** Many companies benefit from an EOS Implementer (external coach), which adds cost.
### Best fit
- 10–150 person companies
- Founder-led, operational dysfunction
- Teams that can't stay on the same page
- Companies with recurring issues that never get resolved
- First real "operating system" for a company that's been running on vibes
### Not ideal if
- You need sophisticated strategic planning
- You're > 250 people and already have ops infrastructure
- Your team resists structured methodology
---
## 2. Scaling Up (Rockefeller Habits 2.0)
**Book:** *Scaling Up* by Verne Harnish
### Core principles
Built on four Decisions:
1. **People** — Core values, talent management, Topgrading
2. **Strategy** — One-Page Strategic Plan (OPSP), 7 Strata of Strategy
3. **Execution** — Priorities (rocks), meeting rhythm, critical numbers
4. **Cash** — Power of One, Cash Acceleration Strategies (CAS)
### Signature tools
- **One-Page Strategic Plan (OPSP):** Annual and quarterly goals on one page. More strategic than EOS's V/TO.
- **7 Strata of Strategy:** Competitive positioning, core customer, brand promise, X-factor (10x advantage), profit per X, BHAG, critical numbers.
- **Meeting rhythm:** Daily (5–15 min), weekly, monthly, quarterly, annual — with specific templates.
- **Critical number:** One metric that, if improved, fixes everything else.
- **Cash acceleration:** CAS system for improving working capital and cash conversion cycle.
### Strengths
- **Stronger strategic framework than EOS.** The 7 strata and OPSP force real strategic thinking.
- **Cash focus.** Unique among frameworks — explicitly addresses cash flow management.
- **Scales further.** Better suited for 100–1000 person companies than EOS.
- **Works for ambitious growth companies.** Designed for companies that want to scale significantly.
### Limitations
- **More complex than EOS.** Harder to DIY. Benefits heavily from a certified Scaling Up coach.
- **Overwhelming at first.** The full framework has many components. Teams often implement partially.
- **Less prescriptive on meetings.** EOS's L10 is very specific. Scaling Up's meeting rhythm requires more customization.
### Best fit
- Series A to Series C companies
- Companies with strong growth ambition
- Leadership teams that want strategic rigor, not just operational clarity
- Companies already past initial chaos, ready for more sophisticated frameworks
### Not ideal if
- You're pre-product-market-fit
- You need quick operational wins
- Your team doesn't have the bandwidth for the learning curve
---
## 3. OKR-Native (Google Style)
**Books:** *Measure What Matters* by John Doerr; *Radical Focus* by Christina Wodtke
### Core principles
OKRs = Objectives + Key Results
- **Objectives:** Qualitative, inspiring direction. "What are we trying to achieve?"
- **Key Results:** Quantitative, measurable outcomes. "How will we know we achieved it?"
- **Not tasks.** KRs measure outcomes, not activities.
**Cascade:** Company OKRs → Department OKRs → Team OKRs → Individual OKRs
**Cadence:** Quarterly OKR cycles. Weekly check-ins. Annual reflection.
**Scoring:** 0.0–1.0. Target is 0.7. Consistently hitting 1.0 = OKRs aren't ambitious enough.
### Strengths
- **Aligns the whole company.** When done well, every team can trace their work to company-level objectives.
- **Encourages ambition.** Moonshot OKRs are explicit. "Roofshot" vs "moonshot" OKRs.
- **Widely understood in tech.** Many hires will already know OKRs.
- **No framework cost.** No implementer required. Tooling is free or cheap (Linear, Notion, Lattice).
### Limitations
- **Hard to do well.** Most companies run "OKR theater" — tasks dressed up as key results.
- **Missing the HOW.** OKRs define what to achieve but not how to operate. You still need meeting rhythm, accountability structure, and issue resolution.
- **Misalignment risk.** If not cascaded properly, teams run disconnected OKRs that feel like alignment but aren't.
- **No operational backbone.** OKRs are a goal-setting system, not a full operating system.
### Best fit
- Tech companies with strong product/engineering culture
- Companies where hypothesis-driven work is already the norm
- Organizations that value autonomy and bottom-up goal setting
- As the goal-setting layer inside a broader operating system
### Not ideal if
- Teams lack discipline to hold each other accountable
- You need more than just goal alignment (issue resolution, meeting structure)
- Leaders don't model OKR behavior themselves
---
## 4. Holacracy
**Book:** *Holacracy* by Brian Robertson
### Core principles
Holacracy replaces the traditional management hierarchy with a system of distributed authority.
- **Circles:** Semi-autonomous units with defined purposes (like teams, but self-governing)
- **Roles:** People fill roles (not job descriptions). One person can hold multiple roles in different circles.
- **Governance meetings:** Roles and accountabilities are defined and evolved by the circle, not management
- **Tactical meetings:** Operational coordination within circles
- **The Constitution:** A legal document that all members ratify, replacing traditional management authority
### Strengths
- **Maximum autonomy.** People closest to the work define how it gets done.
- **Removes management as a bottleneck.** Decisions happen at the circle level.
- **Adapts to complexity.** Circle structure evolves organically as the work changes.
### Limitations
- **Enormous learning curve.** 2–4 years to full adoption. Many companies abandon it.
- **High meeting overhead.** Governance meetings add significant time.
- **Doesn't eliminate politics.** Just moves them to governance meetings.
- **Requires full commitment.** Partial Holacracy doesn't work. You either do it or you don't.
- **Not for crisis mode.** When speed matters, distributed governance slows you down.
### When it works
- Organizations with deep belief in autonomy and self-management
- Non-profit or mission-driven organizations where consensus matters
- Companies with patient leadership willing to invest years in implementation
### When it doesn't work
- Startups needing speed and clarity
- Companies with strong founder personalities who struggle to relinquish control
- Organizations that need to move fast or course-correct frequently
---
## 5. Custom Hybrid
### When to build a hybrid
None of the above frameworks fits perfectly because:
- EOS lacks strategic depth
- Scaling Up is complex to implement
- OKRs don't provide operational backbone
- Holacracy is too slow to implement
The solution: take the best components of each.
### Common hybrid patterns
**EOS backbone + OKR goal-setting:**
- EOS provides: accountability chart, L10 meeting, IDS, meeting pulse
- OKRs provide: goal-setting with ambition, cascade, and alignment checks
- Works well for: tech companies that want operational rigor with flexibility
**Scaling Up strategy + EOS execution:**
- Scaling Up provides: OPSP, 7 strata, cash management
- EOS provides: L10, rocks, IDS
- Works well for: ambitious growth companies that want both strategy and execution discipline
**OKRs + custom meeting rhythm:**
- OKRs provide: goal cascade
- Custom meetings: weekly team syncs, monthly department reviews, quarterly all-hands
- Works well for: companies that already have strong culture but need goal alignment
### Hybrid design principles
1. **Pick one goal-setting system.** Don't mix OKRs and Rocks — they're both 90-day priority systems and will create confusion.
2. **Be explicit about what you're taking from where.** "We use EOS for meetings and Scaling Up for strategy" is a clear hybrid. "We do a bit of everything" is chaos.
3. **Document your version.** Your operating system should have a name and a one-page description of what it includes.
4. **Evolve intentionally.** Change one component at a time. Don't overhaul the whole system when one part isn't working.
---
## Framework Selection Decision Tree
```
Is your company < 50 people and in operational chaos?
YES → Start with EOS. It's the simplest path to order.
NO → Continue.
Does strategic positioning and cash flow need significant work?
YES → Consider Scaling Up.
NO → Continue.
Is your company tech-native with strong product/engineering culture?
YES → OKR-native with a custom meeting rhythm.
NO → Continue.
Do you have 2+ years and full leadership commitment to radical organizational change?
YES → Consider Holacracy (with caution).
NO → Build a custom hybrid from EOS + OKRs.
```
Chất vấn nhà sáng lập theo kiểu YC với 6 câu hỏi về vấn đề, khách hàng, phân phối, lợi thế bảo vệ, vốn và năng lực sáng lập trước khi đưa lời khuyên.
---
name: "office-hours"
description: "/cs:office-hours <topic> — YC-style 6-question founder interrogation before any advice. Forces clarity on problem, customer, distribution, defensibility, capital, and founder fit."
---
# /cs:office-hours — Six-Question Founder Interrogation
**Command:** `/cs:office-hours <topic>`
Before any advice, the founder must answer six questions. Modeled on YC office hours: no analysis until the founder has done the thinking. This is the cognitive forcing function that prevents drift into solutionism.
## When to Run
- Before starting any major initiative
- Before fundraising
- Before a strategic pivot
- When the founder is excited (excitement is a tell — pressure-test)
- When the answer is "obvious" (the obvious answer is usually wrong)
## The Six Questions
The founder must answer **all six** in writing before any C-role weighs in.
### 1. Problem
**Whose problem is this, and how do they describe it in their own words?**
- Not your framing. Their words.
- If you can't quote a customer, you don't have a problem worth solving.
### 2. Customer
**Who is the ICP? Name one real person who would buy this today.**
- Real human. Real company. Real seat.
- If you can't name one, the ICP isn't ready.
### 3. Distribution
**How does the customer first hear your name?**
- Channel, intent, search query, friend, conference — name it.
- If the answer is "we'll figure out marketing later," the answer is no.
### 4. Defensibility
**If this works, what stops a competitor from copying it in 6 months?**
- Network effects, switching costs, data moat, regulatory moat, brand — pick one.
- "We'll execute better" is not a defense.
### 5. Capital
**What does this cost, when does it pay back, and what's the alternative use of the money?**
- Total spend, payback months, opportunity cost.
- If you don't know, don't approve it.
### 6. Founder Fit
**Why are you the right person to do this — and why does this matter enough to spend the next 3 years on it?**
- Founder-market fit is the strongest predictor of survival.
- If the answer is mercenary, the company will be too.
## Output Format
After the founder answers all six, this command produces a one-page brief:
```markdown
# Office Hours Brief: <topic>
**Date:** YYYY-MM-DD
**Founder:** <name>
## 1. Problem
> [founder's verbatim answer]
## 2. Customer
> [founder's verbatim answer]
## 3. Distribution
> [founder's verbatim answer]
## 4. Defensibility
> [founder's verbatim answer]
## 5. Capital
> [founder's verbatim answer]
## 6. Founder Fit
> [founder's verbatim answer]
---
**Assessment** (one of):
- 🟢 GREEN — ship the brief to /cs:boardroom
- 🟡 YELLOW — sharpen Q[N] before proceeding
- 🔴 RED — kill or redefine; do not proceed
```
## Routing
After the brief is GREEN, route to:
- Single-role question → corresponding `/cs:{role}-review`
- Multi-role question → `/cs:brief` then `/cs:boardroom`
## Why This Works
Most bad decisions don't fail at execution — they fail at framing. Forcing six concrete answers surfaces the framing weaknesses before anyone burns time on analysis. The founder either fills the gaps or recognizes the question wasn't ready.
This is the YC `office hours` pattern adapted for Claude Code: the interrogation is the value.
## Related Commands
- `/cs:brief` — turn the answers into a one-page strategy brief
- `/cs:boardroom` — multi-role deliberation
- `/cs:founder-mode` — let the system pick the next step
## Related Agents
- All cs-* advisors consume the brief output
- `cs-chief-of-staff` triggers `/cs:office-hours` when intake is unclear
---
**Version:** 1.0.0
Tra cứu tiền lệ và bức tranh sáng chế: tính mới, tự do khai thác, cảnh quan cạnh tranh, thẩm định mua lại hoặc tiền lệ tranh tụng.
---
name: patent
description: "Patent prior-art and landscape intelligence skill — not generic patent help. Commits to one of five sub-use-cases via forcing intake (novelty search / freedom-to-operate / competitive landscape / acquisition diligence / litigation prior-art) before any search runs. Searches Google Patents, Espacenet, USPTO, and optionally Lens.org for citation-graph signals. Output is an editable Word document (.docx) with verdict, ranked closest art (claim-text extracted), CPC-class-aware landscape, family-resolved hits, geographic coverage, FTO flags where applicable, strategy recommendations, and full audit log. Triggers: 'prior art search for [invention]', 'patent search on [topic]', 'freedom to operate analysis', 'FTO for [product]', 'patent landscape for [field]', 'is [invention] novel', 'patents on [topic]', 'competitive patent analysis', 'prior art for litigation', 'patent diligence on [company]'. Produces search signal, not legal advice — always recommends consulting a patent attorney before filing or licensing decisions. Trademark, copyright, and trade-secret questions are out of scope."
license: MIT
metadata:
source_spec: "megaprompts/11-patent-megaprompt.md"
build_pattern: "Path B (direct conversion)"
research_pack_convention: "Agent Integrity Rules verbatim per PR #657 audit; sub-use-case routing variant"
version: 1.0.0
---
# Patent — Prior-Art + Landscape Intelligence
> **Portability:** Requires `web_fetch` (Google Patents, Espacenet, USPTO), `WebSearch` (adjacent academic art), Node.js with `docx` package, and optionally Lens.org API key for citation-graph signals. Works in Claude Code CLI natively. In Claude.ai with web tools + Code Execution + BYOK Lens.org, the workflow is supported.
> **Out of scope:** trademark, copyright, trade-secret. These are flagged at intake. Use a different skill or qualified counsel.
> **Legal disclaimer:** This skill produces search signal, not legal advice. Verdicts are technical assessments. **Always consult a patent attorney before filing or licensing decisions.**
## Non-Generic Framing — The Differentiator
This skill is **prior-art + landscape intelligence**. It **refuses to be a bucket**. Every invocation commits to one of five sub-use-cases via the grill-me intake before any search runs. The chosen sub-use-case dictates the entire search strategy, ranking heuristics, and DOCX emphasis.
| Sub-use-case | Search strategy | DOCX emphasis |
|---|---|---|
| **Novelty search** | Narrow + claims-text focused; pre-filing date irrelevant | Closest art + claim-differentiation |
| **Freedom-to-operate** | Broad + active patents only; jurisdiction-filtered | FTO flags + claim-by-claim risk |
| **Competitive landscape** | Breadth + filer tally + CPC trends | Filer map + investment hotspots |
| **Acquisition diligence** | Specific assignee + portfolio scope + assignment chain | Portfolio table + ownership verification |
| **Litigation prior-art** | Specific target patent + adjacent art before priority date | Knock-out candidates ranked by relevance |
See [`references/sub_use_case_routing.md`](references/sub_use_case_routing.md) for the canon.
## Agent Integrity Rules (Research-Pack Convention)
Locked verbatim per PR #657 audit.
- **Execution discipline.** Sequential search calls only. **1 query/sec rate limit.** Confirm response received before next call.
- **Source discipline.** Cite only patents returned by THIS session's tool calls. Training knowledge labeled `[Not from search — reference information]` and excluded from counts.
- **Three-count tracking.** Queries sent / patents received (shown) / patents cited. Surfaced in audit log.
- **Retry policy.** On failure → wait 3s → retry once → log. After 3 consecutive failures across tools: stop, alert user, explain what's missing.
- **Plan-tier detection.** Lens.org free tier = 1000 queries/month. Google Patents has no auth but rate-limits per IP. Detect and surface caps.
## Phase 1: Grill-Me Intake (6 forcing questions, one at a time)
### Q1 (root) — Invention description
> **Describe the invention in 2–3 sentences. What does it do, and what's new about it?**
>
> *Why I'm asking:* Concept and keyword extraction depends entirely on a precise description. Vague descriptions ("AI for healthcare", "a better widget") will be rejected — push back and ask the user to specify what the invention does and what differentiates it from existing approaches.
**Refuse mush.** If answer is generic, ask once more: "What does it do that existing systems don't?" Then commit (with caveat in DOCX).
### Q2 (depends on Q1) — Sub-use-case commitment
> **What's the purpose of this search? Pick one:**
>
> 1. Novelty search (am I novel enough to file)
> 2. Freedom-to-operate (will I get sued if I ship)
> 3. Competitive landscape (who else plays here)
> 4. Acquisition diligence (does target really own X)
> 5. Litigation prior-art hunting (kill a specific patent)
>
> *Why I'm asking:* Each path uses a fundamentally different search strategy. I'll **refuse to start without you picking one**.
Forcing format. If user says "all of them", push for the primary purpose — secondary purposes can run as follow-up searches.
### Q3 (asked only if Q2 ∈ {FTO, landscape, diligence}) — Jurisdictions
> **Which jurisdictions matter? Pick all that apply: US / EP / CN / JP / KR / PCT / worldwide.**
>
> *Why I'm asking:* FTO only matters where you'll sell. Landscape changes radically by region. Diligence requires checking all jurisdictions where the target operates.
Skip for novelty (priority date is jurisdictionally portable) and litigation (jurisdiction is set by the target patent).
### Q4 (depends on Q1) — Known prior art
> **Have you already seen prior art close to this? Cite a patent number or paper.**
>
> *Why I'm asking:* If you know one piece of art, I can search adjacent to it — much more precise than starting cold. If you don't, that's fine — just confirm.
Anchoring. Accept "none" but ask if the user has seen *any* related work even informally.
### Q5 (depends on Q2) — Risk tolerance
> **Risk tolerance for this search: strict (one close hit means abandon the path) or signal-gathering (you want the lay of the land regardless)?**
>
> *Why I'm asking:* Strict mode ranks aggressively and surfaces verdict-grade hits; signal mode prioritizes breadth and visualizations.
Asked for novelty and FTO; skipped for pure landscape (always signal-gathering by definition).
### Q6 (asked only if Q2 ∈ {novelty, FTO}) — Attorney status
> **Have you spoken to a patent attorney? This skill produces search signal, not legal advice. Confirm you understand this is for technical assessment only.**
>
> *Why I'm asking:* Novelty and FTO have legal consequences. The skill's verdict is signal-grade; legal positions require qualified counsel.
**Triggers the legal-disclaimer footer in the DOCX.** Skipped for landscape and diligence (lower legal exposure).
**Stop condition:** After Q6 (or earlier if dependency skips applied), commit and start Phase 2. Never re-open intake after Phase 2 begins.
## Phase 2: Search Strategy Selection
Deterministic from intake answers. Use `scripts/sub_use_case_router.py`:
```bash
python ../scripts/sub_use_case_router.py \
--sub-use-case novelty \
--jurisdictions "" \
--risk strict \
--known-art "US10000000B2"
```
Returns: query plan (5-8 queries) + ranking heuristic + DOCX emphasis flags.
## Phase 3: Multi-Source Search (Sequential)
### Source priority
1. **Google Patents** (https://patents.google.com) — workhorse, no auth required, broad coverage
2. **Espacenet** (https://worldwide.espacenet.com) — global coverage, good for non-US art
3. **USPTO PPS** (https://ppubs.uspto.gov) — US deep dive
4. **Lens.org** (https://www.lens.org) — citation graph, BYOK API key required
### Per-sub-use-case query patterns
**Novelty:**
- 3 narrow queries on invention-specific terminology (Google Patents)
- 2 broad concept queries with synonyms (Google Patents + Espacenet)
- 1 CPC-class-restricted query if class identified from initial hits
**FTO:**
- Jurisdiction-filtered: only active patents (not expired, not abandoned)
- Date filter: priority < today
- Active-claim text extraction for each hit
**Competitive landscape:**
- Broader queries on the technology space
- CPC class identification → tally top filers in that class
- 10-year filing trend by year per top-5 filer
**Acquisition diligence:**
- Specific assignee searches (target company + subsidiaries + named inventors)
- Assignment chain check (USPTO assignment recordation)
- Family resolution for deduplication
**Litigation prior-art:**
- Target patent input required (number)
- Priority date extraction
- Search for art before priority date in same CPC classes
- Adjacent-claim-language search
### Sequential discipline
1 q/sec across ALL sources combined. Tracked via `scripts/citation_tracker.py` with timestamp-enforced gap.
## Phase 4: Claim Extraction + Relevance Scoring
For each closest-art hit:
- Pull **independent claim 1** (the broadest claim — primary anticipation/obviousness vehicle)
- Pull **key dependent claims** (claims that add the inventive step)
- Score relevance against invention description (overlap of claim language with Q1 terminology)
Rank by score. Verdict per sub-use-case (NOVEL / POTENTIALLY NOVEL / NOT NOVEL for novelty; CLEAR / FLAGGED / HIGH RISK per jurisdiction for FTO).
## Phase 5: Citation Graph + Family Resolution
### Citation graph (Lens.org BYOK)
If user provides Lens.org API key:
- Foundational-patent identification (cited-by count > threshold, typically 50+)
- Recent high-cite signals (citations in last 24 months as proxy for current activity)
- Forward citations from target patent (litigation prior-art) or from closest art (novelty)
If no Lens.org key: skip; note in audit log; recommend manual citation review on Google Patents.
### Family resolution
Same invention often filed in multiple jurisdictions (US + EP + JP + CN). Group by family ID or priority number to avoid double-counting. Use `scripts/family_resolver.py`:
```bash
python ../scripts/family_resolver.py --hits-file hits.json
# Returns: deduplicated family list + family-member jurisdictions
```
## CPC/IPC Classification Awareness
**Critical:** keyword search alone misses adjacent art. After initial search, extract the CPC/IPC classes from top 5 hits and run **one class-restricted query**. This consistently surfaces art that keyword search misses.
See [`references/cpc_classification_canon.md`](references/cpc_classification_canon.md) for the canon.
## Phase 6: DOCX Generation (8 Sections)
Sub-use-case-dependent emphasis. Via Node.js + `docx` library.
1. **Executive Summary + Verdict** — Sub-use-case banner + one-line verdict (NOVEL / FLAGGED / etc.) + 3-4 key findings + legal disclaimer footer
2. **Closest Prior Art** — 5-10 patents in ranked order. Per hit: hyperlinked title + assignee + filing/priority dates + independent claim 1 text (italicized) + relevance score + relevance rationale (1-2 sentences)
3. **Patent Landscape** — Top filers table (top 10 by count) + 10-year filing trend description + CPC class distribution table. Only for landscape and diligence; abbreviated otherwise.
4. **Citation Graph Signals** — Foundational patents (if Lens-enabled) + recent high-cite activity. If Lens unavailable, note "manual review recommended" and skip table.
5. **Geographic Coverage** — Filings by jurisdiction for top 10 hits. Only for FTO, landscape, diligence; skipped for novelty and litigation.
6. **FTO Flags** (FTO only) — Active patents posing infringement risk. Per flag: hyperlinked patent + jurisdiction + relevant claims + risk level (HIGH/MEDIUM/LOW) + mitigation note.
7. **Strategy + Recommendations** — Sub-use-case-specific:
- Novelty → claim differentiation suggestions
- FTO → design-around hints + jurisdiction strategy
- Landscape → who-to-watch list
- Diligence → red flags in portfolio
- Litigation → ranked knock-out candidates
- **Mandatory disclaimer to consult patent attorney** for any filing/licensing decision.
8. **Audit Log** — Searches table (#, query, source, results, status), counts (sent/shown/cited), tool constraints (plan-tier notes), failed steps, attorney-consultation reminder
### Styling
Arial 12pt body, navy headings (#1a3a5c), light blue table headers (#e8f0f8), red FTO-flag callout. `ExternalHyperlink` patterns:
- Google Patents: `https://patents.google.com/patent/[number]`
- Espacenet: `https://worldwide.espacenet.com/patent/...`
- USPTO: `https://patents.uspto.gov/patent/...`
## Date Discipline
Distinguish at every hit:
- **Filing date** — when the application was first submitted
- **Priority date** — earliest claim of priority (often earlier than filing)
- **Publication date** — when the application became public (typically 18 months after priority)
- **Grant date** — when the patent was granted (later than publication)
Surface the **legally-relevant date** per sub-use-case:
- Novelty → priority date (vs invention's anticipated filing date)
- FTO → grant date + status (active vs expired)
- Landscape → publication date (when public knowledge began)
- Diligence → grant date + assignment date
- Litigation → priority date of target patent (sets the prior-art cutoff)
## Phase 7: Deliver
- Save: `<output-dir>/patent_<invention-slug>_<sub-use-case>_<YYYY-MM-DD>.docx`
- Chat summary: file path + sub-use-case + verdict + audit counts + plan-tier
- Validate: `python scripts/office/validate.py <docx>`
- Reminder: "Consult patent attorney before filing/licensing"
## Tooling
| Script | Role |
|---|---|
| `scripts/citation_tracker.py` | Multi-source three-count audit (Google Patents + Espacenet + USPTO + Lens.org) at `~/.patent_sessions/<session>.json` |
| `scripts/family_resolver.py` | Group same-invention filings across jurisdictions by family ID / priority number |
| `scripts/sub_use_case_router.py` | Deterministic search-strategy selection from intake answers |
## References
- [`references/sub_use_case_routing.md`](references/sub_use_case_routing.md) — 5-sub-use-case canon (7+ sources)
- [`references/cpc_classification_canon.md`](references/cpc_classification_canon.md) — CPC/IPC class follow-up rationale (7+ sources)
- [`references/legal_disclaimer_discipline.md`](references/legal_disclaimer_discipline.md) — when + why disclaimer mandatory (7+ sources)
## Error Handling
| Failure | Behavior |
|---|---|
| User refuses to commit to sub-use-case | Refuse to proceed. Re-ask Q2 with examples. |
| Invention description is generic | Reject answer. Re-ask Q1 with "what does it do that existing systems don't?" |
| Google Patents rate-limits | Wait 3s, retry once. Fall back to Espacenet for that query. Log in audit. |
| Lens.org key missing | Skip citation graph section, note "manual review recommended" in DOCX. |
| Claim text extraction fails | Fall back to abstract; flag as "abstract-only" in relevance rationale. |
| Family resolution incomplete | Note in audit; same-invention duplicates may appear; suggest manual deduplication. |
| All searches return <3 hits | Surface explicitly as "either niche art or genuine gap"; never fabricate. |
| 3 consecutive tool failures | Stop, alert user, explain what's missing. |
| DOCX generation fails | Save raw data as JSON fallback so user doesn't lose work. |
| Target patent number invalid (litigation) | Validate format before search; ask user to confirm. |
## Anti-Patterns To Reject
- Starting any search before user commits to a sub-use-case (refuses generic "patent help")
- Batching all intake questions instead of one at a time
- Accepting vague invention descriptions ("AI for healthcare")
- Keyword-only search without CPC/IPC class follow-up
- Treating family members as separate hits (must be deduplicated)
- Confusing filing date with priority date with publication date
- Skipping the legal disclaimer when sub-use-case has legal consequences
- Reporting a verdict without claim-text evidence
- Fabricating Lens.org citation data when key is absent
- Suggesting design-arounds without acknowledging attorney review is required
- Skipping the audit log
---
**Version:** 1.0.0
**Source spec:** [`megaprompts/11-patent-megaprompt.md`](../../../../megaprompts/11-patent-megaprompt.md)
**Build pattern:** Path B (direct conversion). Research-pack sibling, sub-use-case routing variant.
FILE:references/cpc_classification_canon.md
# CPC/IPC Classification — Why the Class Follow-Up Catches What Keywords Miss
This reference answers exactly one decision: **why does the patent skill always run a CPC/IPC class-restricted query after initial keyword searches, and how does class follow-up systematically surface art that pure keyword search misses?**
## The Core Claim
Keyword search alone systematically misses adjacent prior art. Patent attorneys describe this as the **"different vocabulary problem"**:
- A 1995 patent on "machine learning for image recognition" might describe its invention as "neural network for visual classification" — different terminology, same underlying concept
- A patent in semiconductors might use "transistor channel" where a software paper would use "data flow path" — same idea, different field's vocabulary
- A patent might intentionally use unusual terminology to **broaden claim scope** (legal strategy)
The CPC/IPC classification system was designed precisely to bridge these vocabulary gaps. **Always-run class follow-up** is the skill's mechanical correction for keyword-only blind spots.
## What Are CPC and IPC?
| System | Maintained by | Granularity | Used by |
|---|---|---|---|
| **CPC** (Cooperative Patent Classification) | USPTO + EPO | ~250,000 classes | All major patent offices since 2013 |
| **IPC** (International Patent Classification) | WIPO | ~75,000 classes | Used as fallback in some jurisdictions |
Both are hierarchical. Examples:
- `G06N` → Computer systems based on specific computational models (CPC + IPC)
- `G06N3/00` → Computing arrangements based on biological models
- `G06N3/04` → Architecture, e.g. interconnection topology
- `G06N3/045` → Combinations of networks (deep learning hidden layers)
When a patent examiner classifies a patent, they assign one or more CPC classes. Patents in the same class are conceptually related EVEN IF they use different vocabulary.
## The CPC Class Follow-Up Pattern
After initial keyword search returns top 5 hits:
1. **Extract CPC classes** from those 5 hits
2. **Tally** to find the dominant class (1-3 classes typically)
3. **Run one class-restricted query** — same keywords + CPC class filter
4. **Compare** results to initial keyword-only results
**Empirical observation:** the class-restricted query consistently surfaces 2-5 additional hits that the keyword search missed. Some of these are highly relevant (the "vocabulary mismatch" cases).
## Concrete Examples
### Example 1: AI for medical diagnosis
| Search | Top results |
|---|---|
| Keyword "AI medical diagnosis" | 2017+ patents using "AI" / "ML" / "deep learning" + "diagnosis" |
| **+ CPC G16H50/20** (medical informatics for diagnosis) | + 1990s patents on "expert systems" + "decision support" — same concept, different era's vocabulary |
Without the class follow-up, the searcher would miss 25 years of foundational expert-system art that the USPTO clearly considers prior art.
### Example 2: 3D printing materials
| Search | Top results |
|---|---|
| Keyword "3D printing polymer" | 2010+ patents using "additive manufacturing" + "polymer" |
| **+ CPC B33Y70/00** (materials for additive manufacturing) | + 1980s patents on "stereolithography resin" — predecessor terminology |
### Example 3: Recommender systems
| Search | Top results |
|---|---|
| Keyword "recommendation algorithm" | 2010+ patents using "recommender" / "collaborative filtering" |
| **+ CPC G06Q30/0631** (recommender system for products/services) | + 1990s patents on "preference matching" + "user modeling" |
## Why This Matters Per Sub-Use-Case
### Novelty
Missing class-adjacent art = **false negative**. User concludes invention is novel; later examiner finds the missed art and rejects. Class follow-up prevents this expensive surprise.
### FTO
Missing class-adjacent active patents = **false confidence**. User ships product believing it's clear; gets sued by patent owner whose patent used different vocabulary. Class follow-up surfaces these.
### Litigation prior-art
Missing class-adjacent art before priority date = **weak invalidity case**. The art that would knock out the target patent might be using completely different vocabulary; class follow-up finds it.
### Landscape + Diligence
Class follow-up surfaces the **technology lineage** — which classes the field operates in, who files in each class, how the field has evolved.
## Operational Pattern
In `Phase 3` of patent's SKILL.md:
```
1. Run initial keyword queries (per sub-use-case)
2. Extract CPC classes from top 5 hits → identify 1-3 dominant classes
3. Run ONE class-restricted query: keywords + CPC class filter
4. Merge results, deduplicate, rank
5. (Optional) If multi-class: run one query per dominant class
```
The class follow-up is a **single additional query per dominant class** — minimal budget cost, high signal yield.
## How to Identify the Right Class
After initial search, look at the top 3-5 hits' classification fields. Most patent search interfaces (Google Patents, Espacenet, USPTO PPS) show CPC classes in the metadata.
**Heuristic:**
- If 3+ of top 5 hits share a class → that's your dominant class
- If hits are spread across many classes → dominant class is the most-frequent across the top 10 hits
- If still spread → run class follow-up for top 2 classes (2 extra queries)
## Anti-Patterns
### Skipping class follow-up "to save queries"
The skill's query budget per sub-use-case explicitly allocates 1-2 queries for class follow-up. **Skipping it to save 1 query is the most common false-economy** in patent search. The signal-per-query of class follow-up consistently exceeds keyword-only queries.
### Relying solely on top-1 hit's class
The top-1 hit might be an outlier. Look at the top 3-5 hits' shared classes for the dominant class.
### Treating IPC and CPC as interchangeable
CPC is more granular and modern (post-2013). When available, prefer CPC classes. Fall back to IPC for older patents that haven't been re-classified.
### Class follow-up without CPC class identification
Just running "G06N" with no further specificity is too broad. Use the most specific class that 2+ top hits share (e.g., `G06N3/045` not just `G06N`).
### Ignoring class signals in DOCX
The dominant CPC classes ARE valuable signal for the DOCX. Surface them in Section 3 (Patent Landscape) so the user understands the technology classification of their search space.
## Operational Checklist
- [ ] Initial keyword queries run (per sub-use-case)
- [ ] CPC classes extracted from top 5 hits
- [ ] Dominant class(es) identified (1-3)
- [ ] Class-restricted query run (additional 1-2 queries)
- [ ] Results merged + deduplicated + ranked
- [ ] Dominant CPC classes surfaced in DOCX Section 3 (or Section 2 for novelty)
- [ ] Audit log notes class follow-up as part of search strategy
## Citations (7 sources)
1. **CPC Scheme — USPTO + EPO joint maintenance.** https://www.cooperativepatentclassification.org. Authoritative source for the CPC hierarchy + per-class definitions. The skill recommends consulting CPC scheme for any class beyond top-3 frequency.
2. **WIPO IPC Strategic Plan + IPC Schema.** https://www.wipo.int/classifications/ipc. Source for IPC fallback discipline (used for pre-2013 patents that lack CPC reclassification).
3. **Mowery, D. C., Nelson, R. R., Sampat, B. N., & Ziedonis, A. A., *Ivory Tower and Industrial Innovation* (Stanford U Press, 2004).** Source for the historical analysis of how patent classifications evolve over time and why cross-era keyword search fails. Empirical evidence for the "different vocabulary problem".
4. **WIPO PATENTSCOPE search documentation.** https://patentscope.wipo.int. Source for cross-jurisdictional class search syntax (especially for non-US/EP jurisdictions).
5. **Cohen, W. M., Nelson, R. R., & Walsh, J. P., "Protecting Their Intellectual Assets" — *NBER Working Paper* 7552 (2000).** Source for the empirical evidence that keyword-only patent search systematically under-reports prior art (especially in fast-moving technology areas).
6. **MPEP §901 — *Manual of Patent Examining Procedure* (USPTO).** Source for the examiner-side discipline of using CPC classes for prior-art search. The skill mirrors examiner discipline by including class follow-up as mandatory.
7. **Lemley, M. A., & Sampat, B., "Examiner Characteristics and Patent Office Outcomes" — *Review of Economics and Statistics* 94(3), 2012.** Source for the empirical analysis showing experienced examiners use CPC classes more aggressively + produce stronger prior-art rejections. Class follow-up is the experienced-examiner technique.
FILE:references/legal_disclaimer_discipline.md
# Legal Disclaimer Discipline — When + Why Mandatory
This reference answers exactly one decision: **for which sub-use-cases does the patent skill require a legal disclaimer in the DOCX, and what does the disclaimer need to say?**
## The Core Rule
Patent law has **immediate financial + legal consequences**. The skill produces **search signal**, not legal advice. A reader who confuses the two and skips attorney consultation can face:
- Patent infringement liability (FTO failure → lawsuit)
- Patent application rejection (novelty failure → expensive abandoned application)
- Wasted R&D investment (proceeding on a confidence the search couldn't actually justify)
**The disclaimer is a safety property**, comparable to drafts-only in inbox-triage. It prevents foreseeable user harm.
## When Disclaimer Is Mandatory
| Sub-use-case | Disclaimer mandatory? | Rationale |
|---|---|---|
| **Novelty search** | YES (Q6 triggers it) | Prosecution decisions have legal consequences |
| **Freedom-to-operate** | YES (Q6 triggers it) | Shipping decisions have liability consequences |
| **Competitive landscape** | Optional (recommended) | Lower legal exposure; more strategic than legal |
| **Acquisition diligence** | Optional (strongly recommended) | M&A context — legal review usually already part of process |
| **Litigation prior-art** | Optional (strongly recommended) | Litigation context — counsel almost always involved |
For **mandatory** sub-use-cases, the disclaimer:
1. Appears in **Executive Summary** footer (Section 1)
2. Appears in **Strategy + Recommendations** body (Section 7)
3. Appears in **Audit Log** as a final reminder (Section 8)
For **optional** sub-use-cases, the disclaimer appears only in Section 8 as a reminder.
## What the Disclaimer Says
### Mandatory version (novelty + FTO)
> **⚖️ Legal Disclaimer:** This document is **search signal, not legal advice**. The verdict ({NOVEL/CLEAR/etc.}) is a technical assessment based on the patents found in this session's tool calls. **Patent novelty and freedom-to-operate determinations have legal consequences and require qualified counsel.**
>
> **Before any filing or licensing decision:**
> - Consult a registered patent attorney in your jurisdiction(s)
> - Provide them this dossier as starting material; they will conduct independent verification + opinion
> - Their opinion is privileged and admissible; this skill's output is neither
>
> The skill does not establish attorney-client privilege. The skill's verdict does not constitute a legal opinion.
### Optional version (landscape, diligence, litigation)
> **⚖️ Reminder:** This document is search signal. Patent attorney consultation is recommended for any decisions arising from this analysis.
## Why Disclaimer Discipline Matters
### Reason 1: Legal liability framing
Without disclaimer, a user could (in extreme cases) claim the skill misled them into a legal decision. Disclaimer makes the skill's role unambiguous: **technical assessment, not legal opinion**.
### Reason 2: Setting realistic expectations
Even a well-executed patent search has limits:
- Some patents may not be indexed in queried sources (especially recent applications)
- Some patents may be classified in unexpected CPC classes
- Some art may be in non-patent literature (academic papers, products, manuals)
- Some art may be in different languages (non-English jurisdictions)
The disclaimer tells the user: "I did the best technical search I could; counsel will catch what I might have missed."
### Reason 3: Privileged communication
Attorney-client conversations are **privileged** — not admissible against the user in litigation. This skill's output is **not privileged**. If the user later faces litigation, opposing counsel can subpoena the dossier as evidence of what the user knew.
The disclaimer reminds users to NOT rely solely on the dossier for high-stakes decisions; an attorney's opinion provides privilege.
### Reason 4: Jurisdiction-specific nuance
Patent law varies by jurisdiction:
- **First-to-file vs first-to-invent** (US switched to first-to-file in 2013; some countries differ)
- **Grace periods** (US has 1-year; many countries have none)
- **Doctrine of equivalents** (varies by jurisdiction)
- **Inequitable conduct** (US-specific; may affect prosecution strategy)
The skill cannot capture all jurisdictional nuance. Counsel can.
## Anti-Patterns
### Skipping disclaimer because user said "I'm a patent attorney"
The user might be a patent attorney, but they might also be running this for a less-experienced colleague or client. The disclaimer is **always** in the document because the document might outlive the original requester.
### Burying disclaimer in fine print
Mandatory-sub-use-case disclaimer appears in Sections 1, 7, and 8 — three locations. Reader cannot miss it.
### Replacing disclaimer with "consult your attorney"
The disclaimer must be specific about what the skill does and doesn't claim. "Consult your attorney" alone is insufficient; the disclaimer also needs to clarify:
- What the verdict means (technical assessment)
- What counsel adds (privilege, jurisdiction expertise, opinion)
- What's not covered (non-patent prior art, language coverage gaps)
### Conflating disclaimer with legal-advice disclaimer
Some skills use generic "this is not legal advice" boilerplate. The patent skill's disclaimer is specifically about patent novelty/FTO determinations and the role of qualified counsel — not generic.
### Removing disclaimer to "make the document feel more authoritative"
Authority comes from technical rigor (claim-text extraction, family resolution, CPC class follow-up), not from omitting safety disclaimers. The disclaimer **enhances** authority by making the skill's role transparent.
## Operational Checklist
For novelty + FTO (mandatory):
- [ ] Disclaimer in Executive Summary (Section 1) footer
- [ ] Disclaimer in Strategy section (Section 7)
- [ ] Disclaimer in Audit Log (Section 8) as final reminder
- [ ] Disclaimer text matches template above (don't paraphrase the legal language)
- [ ] Q6 attorney-status answer recorded in audit log
For landscape + diligence + litigation (optional):
- [ ] Reminder version in Audit Log (Section 8)
- [ ] Strategy section (Section 7) includes "consult patent attorney for [decision-specific context]"
## Citations (7 sources)
1. **MPEP §1.4 + §1.5 — *Manual of Patent Examining Procedure* (USPTO).** Source for the role of registered patent attorneys/agents in prosecution. The disclaimer's reference to "qualified counsel" tracks USPTO's registration framework.
2. **AIPLA *Code of Ethics* — American Intellectual Property Law Association.** Source for the privileged-communication framing. AIPLA's guidance on lay-vs-attorney communication informs the "this skill is not privileged" disclaimer language.
3. **35 USC §282 — *Presumption of validity*.** US patent statute. Source for understanding what "valid" means legally + why a search-signal verdict is not a legal validity opinion.
4. **35 USC §271 — *Infringement of patent*.** Source for the FTO disclaimer framing. Liability for infringement is a legal determination; the skill's "CLEAR/FLAGGED/HIGH RISK" verdict is technical.
5. **EPO Guidelines for Examination, Part E.** Source for European-jurisdiction differences (no grace period, different inventive-step analysis). The disclaimer's "jurisdiction-specific nuance" caveat tracks these variations.
6. **PCT Article 39 — Patent Cooperation Treaty.** Source for the cross-jurisdiction prosecution complexity that justifies counsel involvement. PCT national-phase entries each require local-counsel coordination.
7. **Fischer, T., & Henkel, J., "Patent Trolls on Markets for Technology" — *Research Policy* 41(9), 2012.** Empirical evidence for the cost of FTO mistakes. Patent assertion entities (PAEs) have made FTO failures financially severe; the disclaimer's emphasis on counsel reflects this risk reality.
FILE:references/sub_use_case_routing.md
# Sub-Use-Case Routing — The 5 Patent Search Strategies
This reference answers exactly one decision: **given the user's Q2 commitment, which search strategy / ranking heuristic / DOCX emphasis applies?**
The patent skill refuses to be a generic "patent search". Q2 is mandatory. The 5 sub-use-cases use **fundamentally different** strategies — running a novelty search and calling it FTO produces wrong answers in dangerous ways.
## Why Sub-Use-Case Commitment Matters
| Sub-use-case | Wrong-strategy danger |
|---|---|
| Novelty | Generic search misses claim-text proximity → false negatives ("looks novel, isn't") |
| FTO | Generic search includes expired/abandoned patents → false positives ("looks blocked, isn't") |
| Landscape | Generic search misses CPC class trends → incomplete competitive picture |
| Diligence | Generic search misses assignment chain → ownership-verification gaps |
| Litigation | Generic search includes art after priority date → useless for invalidation |
The 5 strategies are **not interchangeable**. The skill enforces commitment to prevent strategy mismatches.
## Strategy 1: Novelty Search
### Question being answered
"Am I novel enough to file? Is there art that anticipates or makes obvious my invention?"
### Search emphasis
- **Narrow** queries on invention-specific terminology (don't drown in adjacent art)
- **Claims-text focused** (the legal test is claim-by-claim anticipation)
- **Pre-filing date irrelevant** — anything published before user's filing is potential art
### Query plan
1. 3 narrow Google Patents queries on invention-specific terms
2. 2 broad concept queries with synonyms (Google Patents + Espacenet)
3. 1 CPC class-restricted query (after class identification from initial hits)
4. (Optional Lens.org) forward citations from any closest art
**Total: 6-7 sequential queries.**
### Ranking heuristic
Rank by **claim-text overlap** with user's invention description (Q1). Top hits are those whose claim 1 most overlaps in technical terminology. Surface independent claim 1 verbatim for each top-5 hit.
### Verdict scale
- **NOVEL** — closest hit has <30% claim-text overlap; clear differentiation possible
- **POTENTIALLY NOVEL** — 30-60% overlap; differentiation possible but requires careful claim drafting
- **NOT NOVEL** — >60% overlap; invention as described is anticipated by closest art
### DOCX emphasis
- Section 2 (Closest Prior Art): expanded — 8-10 hits with full claim-1 text
- Section 7 (Strategy): claim-differentiation suggestions
- Sections 3 + 5 (Landscape + Geographic): abbreviated
- Mandatory legal disclaimer footer
## Strategy 2: Freedom-to-Operate
### Question being answered
"If I ship in jurisdiction X, will I get sued for infringement?"
### Search emphasis
- **Active patents only** — expired/abandoned patents can't sue
- **Jurisdiction-filtered** — FTO only matters where user sells
- **Date filter:** priority date < today (no pending applications without published claims)
- **Independent + dependent claims** — both relevant to infringement analysis
### Query plan
1. Per jurisdiction (Q3): 2-3 queries with jurisdiction filter (US: USPTO; EP: Espacenet; etc.)
2. Active-status filter applied to all
3. CPC class follow-up after initial hits
4. (Optional) assignment chain check for active assignee context
**Total: 8-15 sequential queries (scales with # of jurisdictions).**
### Ranking heuristic
Rank by **claim-by-claim infringement risk**. For each active patent: which independent claims would the user's product practice? High risk = at least one independent claim covers user's product as designed.
### Verdict scale (per jurisdiction)
- **CLEAR** — no active patents pose infringement risk
- **FLAGGED** — 1-2 active patents may pose risk; design-around viable
- **HIGH RISK** — 3+ active patents pose risk; design changes required OR licensing path needed
### DOCX emphasis
- Section 6 (FTO Flags): expanded — per-flag risk per jurisdiction
- Section 5 (Geographic Coverage): expanded
- Section 7 (Strategy): design-around hints + jurisdiction strategy
- Mandatory legal disclaimer footer
## Strategy 3: Competitive Landscape
### Question being answered
"Who else plays in this technology space? What are the trends?"
### Search emphasis
- **Broader queries** on the technology space (NOT the specific invention)
- **CPC class identification** drives the analysis
- **Top filer tally** — who files most patents in the space
- **10-year filing trend** by year per top-5 filer
### Query plan
1. 2-3 broad queries on the technology space
2. CPC class extraction from top hits
3. 1 query per top-5 filer to gauge their portfolio
4. (Optional Lens.org) citation graph for foundational patents
**Total: 8-10 sequential queries.**
### Ranking heuristic
Rank by filer count + recency. Top-5 filers + 3 emerging entrants (filers with first patent in last 2 years).
### Verdict scale
- **CONCENTRATED** — top-3 filers own >60% of patents in the space
- **COMPETITIVE** — top-10 filers own 60-90%; mature competitive market
- **EMERGING** — long tail of filers; market is still defining itself
### DOCX emphasis
- Section 3 (Patent Landscape): expanded — top filers table + 10-yr trend + CPC distribution
- Section 5 (Geographic Coverage): expanded
- Sections 2 + 4 (Closest Art + Citation Graph): abbreviated
- Section 7 (Strategy): who-to-watch list + emerging-entrants signal
- Legal disclaimer optional (lower legal exposure)
## Strategy 4: Acquisition Diligence
### Question being answered
"Does target company actually own the patents they claim? Is there portfolio depth?"
### Search emphasis
- **Specific assignee searches** — target company + subsidiaries + named inventors
- **Assignment chain check** — USPTO assignment recordation
- **Family resolution** — deduplicate same-invention across jurisdictions
- **Portfolio scope** — are patents in core business areas or peripheral?
### Query plan
1. 2-3 assignee-name queries (Google Patents + USPTO assignee search)
2. Subsidiary searches if user provides org chart
3. Inventor searches for key named inventors
4. Assignment recordation lookups for ownership verification
5. Family resolution across all hits
**Total: 6-12 sequential queries.**
### Ranking heuristic
Group by family. Within family, surface earliest priority. Across families, rank by:
- Citation count (foundational vs niche)
- Filing recency (active R&D vs legacy)
- Claim breadth (broad coverage vs narrow)
### Verdict scale
- **PORTFOLIO VERIFIED** — claimed patents owned, assignment chains clean, no orphans
- **PARTIAL VERIFICATION** — some claimed patents not found OR assignment chain unclear
- **OWNERSHIP RISK** — significant claimed patents not owned by target OR major assignment gaps
### DOCX emphasis
- Section 3 (Patent Landscape): expanded as portfolio table
- Section 5 (Geographic Coverage): expanded
- Section 7 (Strategy): red flags in portfolio + ownership-verification flags
- Legal disclaimer optional but recommended (M&A context)
## Strategy 5: Litigation Prior-Art
### Question being answered
"Can I invalidate this specific patent? What art exists before its priority date?"
### Search emphasis
- **Target patent input required** (number)
- **Priority date extraction** — sets the prior-art cutoff
- **Search before priority date in same CPC classes**
- **Adjacent-claim-language search** — art that uses similar claim language
### Query plan
1. Fetch target patent (extract priority date + claims + CPC classes)
2. CPC class queries with date filter (priority < target's priority)
3. Keyword queries on independent claim language with date filter
4. (Optional Lens.org) forward citations from target's cited art
**Total: 5-8 sequential queries.**
### Ranking heuristic
Rank by **knock-out potential** — claim-by-claim anticipation/obviousness. Highest rank: art that anticipates ALL elements of target's broadest independent claim.
### Verdict scale
- **KNOCK-OUT FOUND** — art clearly anticipates all elements of broadest claim
- **STRONG OBVIOUSNESS COMBINATION** — multiple pieces of art combine to cover all elements
- **WEAK OBVIOUSNESS** — art relevant but anticipation/obviousness argument is uphill
- **NO MATERIAL ART FOUND** — patent appears strong against this prior-art set
### DOCX emphasis
- Section 2 (Closest Prior Art): expanded — ranked knock-out candidates with claim-language overlap
- Section 7 (Strategy): per-claim invalidity analysis
- Sections 3 + 5 (Landscape + Geographic): abbreviated
- Legal disclaimer optional but recommended (litigation context)
## Out-of-Scope Topics (Flagged at Intake)
| Topic | Why out of scope |
|---|---|
| Trademark | Different legal regime, different sources (USPTO TESS not Patent Office) |
| Copyright | No formal search system; rights attach automatically |
| Trade secret | By definition, not in public records |
If user asks for any of these → halt at intake, recommend appropriate skill or attorney.
## Operational Checklist
- [ ] Q2 sub-use-case picked (no "all of them")
- [ ] `scripts/sub_use_case_router.py` returns query plan + ranking heuristic + DOCX flags
- [ ] Search emphasis matches sub-use-case (not generic)
- [ ] Verdict scale per sub-use-case applied
- [ ] DOCX emphasis adjusted (not all 8 sections expanded for every sub-use-case)
- [ ] Legal disclaimer mandatory for novelty + FTO; optional for landscape/diligence/litigation but recommended
## Citations (7 sources)
1. **MPEP (Manual of Patent Examining Procedure) — USPTO.** Source for the legal definitions of novelty (35 USC §102) vs FTO (no explicit USC; case law) vs anticipation/obviousness. The verdict scales follow MPEP terminology.
2. **35 USC §102 + §103 — US patent statute.** Source for the priority-date-as-cutoff rule for novelty (§102) and the obviousness combination doctrine (§103) that drives litigation prior-art ranking.
3. **WIPO Patent Cooperation Treaty (PCT) procedural docs.** Source for the cross-jurisdiction family discipline. PCT applications generate national-phase entries in many jurisdictions; the family resolver follows WIPO's family-id taxonomy.
4. **EPO Guidelines for Examination — European Patent Office.** Source for the EP-specific FTO discipline. EP active-status filtering uses EPO's "in force" status field.
5. **USPTO Patent Public Search documentation.** Source for the USPTO PPS query syntax and assignment-recordation lookup endpoints used in acquisition diligence.
6. **Google Patents search documentation + advanced operators.** Source for the keyword + CPC class + date filter syntax. Google Patents indexes all PCT national-phase entries plus most jurisdictions' grant data.
7. **Lens.org API documentation (https://docs.api.lens.org).** Source for the citation-graph queries. Lens.org's citation API exposes forward + backward citations with citation-count thresholds for foundational-patent identification.
FILE:scripts/citation_tracker.py
#!/usr/bin/env python3
"""citation_tracker.py — Patent skill three-count audit across multi-source patent search.
Stdlib-only. Mirrors litreview/grants/dossier trackers but adapted for patent's
4-source workflow:
- Google Patents (workhorse, no auth)
- Espacenet (global)
- USPTO PPS (US deep dive)
- Lens.org (BYOK, citation graph)
Tracked counts:
- searches_per_source (broken out by source)
- patents_received_total
- patents_cited_total
- patents_cited_by_source
- sub_use_case (recorded at start; drives audit verbatim)
- lens_byok_used (boolean — surfaced in audit log)
Enforces 1s sequential discipline across ALL sources combined.
Usage:
python citation_tracker.py --action start --session patent-MS-novelty-20260515 --invention "..." --sub-use-case novelty
python citation_tracker.py --action record_search --session ... --source google_patents --query "..."
python citation_tracker.py --action record_received --session ... --source google_patents --count 10
python citation_tracker.py --action record_cited --session ... --source google_patents --patent-num "US10000000B2"
python citation_tracker.py --action record_lens_byok --session ...
python citation_tracker.py --action status --session ...
python citation_tracker.py --action close --session ...
"""
import argparse
import json
import sys
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Dict, List, Optional
SESSIONS_DIR = Path.home() / ".patent_sessions"
MIN_GAP_SECONDS = 1.0
VALID_SOURCES = ["google_patents", "espacenet", "uspto", "lens", "websearch"]
VALID_SUB_USE_CASES = ["novelty", "fto", "landscape", "diligence", "litigation"]
def session_path(name: str) -> Path:
return SESSIONS_DIR / f"{name}.json"
def load_session(name: str) -> Dict[str, Any]:
p = session_path(name)
if not p.exists():
raise FileNotFoundError(f"Session not found: {name}")
return json.loads(p.read_text(encoding="utf-8"))
def save_session(name: str, data: Dict[str, Any]) -> None:
SESSIONS_DIR.mkdir(parents=True, exist_ok=True)
session_path(name).write_text(json.dumps(data, indent=2), encoding="utf-8")
def now_iso() -> str:
return datetime.now(timezone.utc).isoformat()
def now_ts() -> float:
return datetime.now(timezone.utc).timestamp()
def action_start(name: str, invention: Optional[str], sub_use_case: Optional[str]) -> Dict[str, Any]:
if session_path(name).exists():
raise FileExistsError(f"Session already exists: {name}")
if sub_use_case and sub_use_case not in VALID_SUB_USE_CASES:
raise ValueError(f"Invalid sub-use-case '{sub_use_case}'. Pick from: {VALID_SUB_USE_CASES}")
data: Dict[str, Any] = {
"session": name,
"invention": invention or "",
"sub_use_case": sub_use_case or "",
"started_at": now_iso(),
"ended_at": None,
"lens_byok_used": False,
"searches": [],
"received_log": [],
"cited": [],
"counts": {
"searches_total": 0,
"searches_by_source": {s: 0 for s in VALID_SOURCES},
"received_total": 0,
"received_by_source": {s: 0 for s in VALID_SOURCES},
"cited_total": 0,
"cited_by_source": {s: 0 for s in VALID_SOURCES},
},
}
save_session(name, data)
return data
def action_record_search(name: str, source: str, query: str) -> Dict[str, Any]:
data = load_session(name)
if source not in VALID_SOURCES:
raise ValueError(f"Invalid source '{source}'. Pick from: {VALID_SOURCES}")
if data["searches"]:
last_ts = data["searches"][-1].get("ts", 0)
gap = now_ts() - last_ts
if gap < MIN_GAP_SECONDS:
raise RuntimeError(
f"Sequential discipline violated: {gap:.2f}s gap (need >= {MIN_GAP_SECONDS}s). "
f"Wait {MIN_GAP_SECONDS - gap:.2f}s more."
)
data["searches"].append({"source": source, "query": query, "at": now_iso(), "ts": now_ts()})
data["counts"]["searches_total"] += 1
data["counts"]["searches_by_source"][source] += 1
save_session(name, data)
return data
def action_record_received(name: str, source: str, count: int) -> Dict[str, Any]:
data = load_session(name)
if source not in VALID_SOURCES:
raise ValueError(f"Invalid source '{source}'")
data["received_log"].append({"source": source, "count": count, "at": now_iso()})
data["counts"]["received_total"] += count
data["counts"]["received_by_source"][source] += count
save_session(name, data)
return data
def action_record_cited(name: str, source: str, patent_num: str, title: Optional[str]) -> Dict[str, Any]:
data = load_session(name)
if source not in VALID_SOURCES:
raise ValueError(f"Invalid source '{source}'")
if any(c["patent_num"] == patent_num for c in data["cited"]):
return data
data["cited"].append({"source": source, "patent_num": patent_num, "title": title, "at": now_iso()})
data["counts"]["cited_total"] += 1
data["counts"]["cited_by_source"][source] += 1
save_session(name, data)
return data
def action_record_lens_byok(name: str) -> Dict[str, Any]:
data = load_session(name)
data["lens_byok_used"] = True
save_session(name, data)
return data
def action_status(name: str) -> Dict[str, Any]:
return load_session(name)
def action_close(name: str) -> Dict[str, Any]:
data = load_session(name)
if data.get("ended_at") is None:
data["ended_at"] = now_iso()
save_session(name, data)
return data
def render_status_human(data: Dict[str, Any]) -> str:
out: List[str] = []
out.append(f"Session: {data['session']}")
out.append(f"Invention: {data.get('invention', '(unset)')}")
out.append(f"Sub-use-case: {data.get('sub_use_case', '(unset)')}")
out.append(f"Lens.org BYOK: {'YES' if data.get('lens_byok_used') else 'no (citation graph skipped)'}")
out.append(f"Started: {data['started_at']}")
out.append(f"Ended: {data.get('ended_at') or '(active)'}")
out.append("")
c = data["counts"]
out.append(f"Total searches: {c['searches_total']}")
out.append("By source:")
for src, n in c["searches_by_source"].items():
if n > 0:
out.append(f" {src:<18s} {n}")
out.append("")
out.append(f"Patents received: {c['received_total']}")
out.append(f"Patents cited: {c['cited_total']}")
out.append("Cited by source:")
for src, n in c["cited_by_source"].items():
if n > 0:
out.append(f" {src:<18s} {n}")
out.append("")
out.append("Audit block (paste in DOCX Section 8):")
out.append(
f" Searches: {c['searches_total']} (Google Patents: {c['searches_by_source'].get('google_patents', 0)}, "
f"Espacenet: {c['searches_by_source'].get('espacenet', 0)}, "
f"USPTO: {c['searches_by_source'].get('uspto', 0)}, "
f"Lens.org: {c['searches_by_source'].get('lens', 0)}). "
f"Patents received: {c['received_total']}. Patents cited: {c['cited_total']}. "
f"Lens.org BYOK: {'used' if data.get('lens_byok_used') else 'not provided (citation graph skipped)'}."
)
return "\n".join(out)
def main(argv: List[str]) -> int:
parser = argparse.ArgumentParser(description=__doc__.split("\n")[0])
parser.add_argument("--action", required=True, choices=["start", "record_search", "record_received", "record_cited", "record_lens_byok", "status", "list", "close"])
parser.add_argument("--session")
parser.add_argument("--invention")
parser.add_argument("--sub-use-case", choices=VALID_SUB_USE_CASES)
parser.add_argument("--source", choices=VALID_SOURCES)
parser.add_argument("--query")
parser.add_argument("--count", type=int)
parser.add_argument("--patent-num")
parser.add_argument("--title")
parser.add_argument("--output", choices=["human", "json"], default="human")
args = parser.parse_args(argv)
try:
if args.action == "start":
result = action_start(args.session, args.invention, args.sub_use_case)
elif args.action == "record_search":
result = action_record_search(args.session, args.source, args.query)
elif args.action == "record_received":
result = action_record_received(args.session, args.source, args.count)
elif args.action == "record_cited":
result = action_record_cited(args.session, args.source, args.patent_num, args.title)
elif args.action == "record_lens_byok":
result = action_record_lens_byok(args.session)
elif args.action == "status":
result = action_status(args.session)
elif args.action == "close":
result = action_close(args.session)
else:
SESSIONS_DIR.mkdir(parents=True, exist_ok=True)
result = [{"session": p.stem, "data": json.loads(p.read_text(encoding="utf-8"))} for p in sorted(SESSIONS_DIR.glob("*.json"))]
except (FileNotFoundError, FileExistsError, ValueError, RuntimeError) as e:
print(f"error: {e}", file=sys.stderr); return 2
if args.output == "json":
print(json.dumps(result, indent=2, default=str))
else:
if args.action == "list":
print(json.dumps(result, indent=2, default=str))
else:
print(render_status_human(result))
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
FILE:scripts/family_resolver.py
#!/usr/bin/env python3
"""family_resolver.py — Deduplicate same-invention patent filings across jurisdictions.
Stdlib-only. The same invention is often filed in multiple jurisdictions
(US + EP + JP + CN of one underlying invention). They share a "family" identifier
or a common priority application number.
Without family resolution, a multi-jurisdiction search returns the same invention
multiple times — inflating the perceived prior-art set and wasting reviewer
attention.
Family resolution rules:
1. If two patents share the same `family_id`, they're family members
2. If two patents share the same `priority_number`, they're family members
3. If two patents share the same `priority_date` AND have ≥80% applicant overlap
AND ≥80% inventor overlap → likely family (heuristic, flag with confidence)
For each family: surface ONE representative member (typically earliest priority OR
US member for US-context users) and list all family-member jurisdictions.
NO LLM CALLS. Pure JSON aggregation + heuristic clustering.
Input file format (`--hits-file`):
[
{
"patent_num": "US10000000B2",
"title": "...",
"family_id": "F12345678",
"priority_number": "US15/123,456",
"priority_date": "2018-03-15",
"filing_date": "2019-03-14",
"publication_date": "2020-09-15",
"grant_date": "2022-01-10",
"jurisdiction": "US",
"assignee": "Acme Corp",
"inventors": ["Smith, J", "Jones, K"]
}
]
Usage:
python family_resolver.py --hits-file /tmp/hits.json
python family_resolver.py --hits-file /tmp/hits.json --output json
python family_resolver.py --sample
"""
import argparse
import json
import sys
from collections import defaultdict
from pathlib import Path
from typing import Any, Dict, List, Optional, Set
SAMPLE_HITS = [
{
"patent_num": "US10000000B2",
"title": "Machine learning sepsis prediction system",
"family_id": "F12345678",
"priority_number": "US15/123,456",
"priority_date": "2018-03-15",
"filing_date": "2019-03-14",
"jurisdiction": "US",
"assignee": "Acme Corp",
"inventors": ["Smith, J", "Jones, K"],
},
{
"patent_num": "EP3500000B1",
"title": "Système de prédiction sépticémie par apprentissage automatique",
"family_id": "F12345678", # same family
"priority_number": "US15/123,456", # same priority
"priority_date": "2018-03-15",
"filing_date": "2019-03-14",
"jurisdiction": "EP",
"assignee": "Acme Corp",
"inventors": ["Smith, J", "Jones, K"],
},
{
"patent_num": "JP2020100000A",
"title": "敗血症予測システム",
"family_id": "F12345678", # same family
"priority_number": "US15/123,456",
"priority_date": "2018-03-15",
"filing_date": "2019-03-14",
"jurisdiction": "JP",
"assignee": "Acme Corp",
"inventors": ["Smith, J", "Jones, K"],
},
{
"patent_num": "US10500000B2",
"title": "Different sepsis prediction method using LSTMs",
"family_id": "F87654321", # DIFFERENT family
"priority_number": "US16/200,000",
"priority_date": "2019-08-22",
"filing_date": "2020-08-21",
"jurisdiction": "US",
"assignee": "Beta Inc",
"inventors": ["Lee, M"],
},
{
"patent_num": "WO2020/123456",
"title": "PCT application: sepsis prediction with multi-modal data",
"family_id": "F87654321", # same family as US10500000
"priority_number": "US16/200,000",
"priority_date": "2019-08-22",
"jurisdiction": "WO",
"assignee": "Beta Inc",
"inventors": ["Lee, M"],
},
{
"patent_num": "US11000000B1",
"title": "Yet another sepsis ML approach",
"family_id": "F11111111", # alone
"priority_number": "US17/300,000",
"priority_date": "2021-01-10",
"jurisdiction": "US",
"assignee": "Gamma LLC",
"inventors": ["Park, S", "Kim, J"],
},
]
def jaccard_similarity(set1: Set[str], set2: Set[str]) -> float:
if not set1 and not set2:
return 1.0
if not set1 or not set2:
return 0.0
inter = len(set1 & set2)
union = len(set1 | set2)
return inter / union if union > 0 else 0.0
def normalize_name(name: str) -> str:
"""Normalize assignee/inventor names for comparison."""
return name.lower().strip().replace(",", "").replace(".", "")
def resolve_families(hits: List[Dict[str, Any]]) -> Dict[str, Any]:
# Pass 1: group by exact family_id
by_family: Dict[str, List[Dict[str, Any]]] = defaultdict(list)
no_family_id: List[Dict[str, Any]] = []
for h in hits:
fid = h.get("family_id")
if fid:
by_family[fid].append(h)
else:
no_family_id.append(h)
# Pass 2: group remaining by exact priority_number
if no_family_id:
by_priority: Dict[str, List[Dict[str, Any]]] = defaultdict(list)
unmatched: List[Dict[str, Any]] = []
for h in no_family_id:
pn = h.get("priority_number")
if pn:
by_priority[pn].append(h)
else:
unmatched.append(h)
# Add to family map using priority_number as fallback ID
for pn, group in by_priority.items():
by_family[f"PRI:{pn}"] = group
# Pass 3: heuristic clustering for unmatched (priority_date + applicant + inventor overlap)
for h in unmatched:
matched = False
for fid, group in list(by_family.items()):
rep = group[0]
if (h.get("priority_date") == rep.get("priority_date")
and jaccard_similarity({normalize_name(h.get("assignee", ""))}, {normalize_name(rep.get("assignee", ""))}) >= 0.8
and jaccard_similarity({normalize_name(i) for i in h.get("inventors", [])}, {normalize_name(i) for i in rep.get("inventors", [])}) >= 0.8):
by_family[fid].append(h)
matched = True
break
if not matched:
# Solo — use patent_num as family_id
by_family[f"SOLO:{h.get('patent_num', 'unknown')}"] = [h]
# For each family: pick representative (earliest priority date, prefer US member if available)
families: List[Dict[str, Any]] = []
for fid, members in by_family.items():
sorted_members = sorted(members, key=lambda m: (m.get("priority_date", "9999"), 0 if m.get("jurisdiction") == "US" else 1))
rep = sorted_members[0]
jurisdictions = sorted({m.get("jurisdiction", "?") for m in members})
family = {
"family_id": fid,
"representative": {
"patent_num": rep.get("patent_num"),
"title": rep.get("title"),
"assignee": rep.get("assignee"),
"priority_date": rep.get("priority_date"),
"filing_date": rep.get("filing_date"),
"jurisdiction": rep.get("jurisdiction"),
},
"family_member_count": len(members),
"jurisdictions": jurisdictions,
"all_patent_nums": [m.get("patent_num") for m in members],
}
families.append(family)
families.sort(key=lambda f: f["representative"].get("priority_date", "9999"))
return {
"input_hits": len(hits),
"unique_families": len(families),
"deduplication_savings": len(hits) - len(families),
"families": families,
}
def render_human(result: Dict[str, Any]) -> str:
out: List[str] = []
out.append(f"Family resolution complete:")
out.append(f" Input hits: {result['input_hits']}")
out.append(f" Unique families: {result['unique_families']}")
out.append(f" Deduplication savings: {result['deduplication_savings']} duplicate hits removed")
out.append("")
out.append("Families (representative + all jurisdictions):")
for i, f in enumerate(result["families"], 1):
rep = f["representative"]
out.append(f"")
out.append(f" Family {i} (id: {f['family_id']}):")
out.append(f" Representative: {rep['patent_num']} ({rep['jurisdiction']}, priority {rep['priority_date']})")
out.append(f" Title: {rep['title'][:80]}")
out.append(f" Assignee: {rep['assignee']}")
out.append(f" Family size: {f['family_member_count']} member(s) across {len(f['jurisdictions'])} jurisdiction(s)")
out.append(f" Jurisdictions: {', '.join(f['jurisdictions'])}")
if f['family_member_count'] > 1:
out.append(f" All members: {', '.join(f['all_patent_nums'])}")
return "\n".join(out)
def main(argv: List[str]) -> int:
parser = argparse.ArgumentParser(description=__doc__.split("\n")[0])
parser.add_argument("--hits-file", help="Path to JSON file with patent hits")
parser.add_argument("--sample", action="store_true", help="Run on embedded sample")
parser.add_argument("--output", choices=["human", "json"], default="human")
args = parser.parse_args(argv)
if args.sample:
result = resolve_families(SAMPLE_HITS)
elif args.hits_file:
p = Path(args.hits_file)
if not p.exists():
print(f"error: {args.hits_file} not found", file=sys.stderr); return 2
try:
hits = json.loads(p.read_text(encoding="utf-8"))
except json.JSONDecodeError as e:
print(f"error: invalid JSON: {e}", file=sys.stderr); return 2
result = resolve_families(hits)
else:
parser.print_help(); return 0
if args.output == "json":
print(json.dumps(result, indent=2, default=str))
else:
print(render_human(result))
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
FILE:scripts/sub_use_case_router.py
#!/usr/bin/env python3
"""sub_use_case_router.py — Deterministic search-strategy from intake answers.
Stdlib-only. Routes to one of 5 patent search strategies based on grill-me
intake answers, returning a query plan + ranking heuristic + DOCX emphasis.
The 5 sub-use-cases:
- novelty — am I novel enough to file
- fto — will I get sued if I ship
- landscape — who else plays here
- diligence — does target really own X
- litigation — kill a specific patent
Each gets a fundamentally different search strategy, ranking heuristic, and
DOCX emphasis (which sections expand vs abbreviate).
NO LLM CALLS. Pure rule-based routing.
Usage:
python sub_use_case_router.py --sub-use-case novelty --jurisdictions "" --risk strict --known-art "US10000000B2"
python sub_use_case_router.py --sub-use-case fto --jurisdictions "US,EP" --risk strict
python sub_use_case_router.py --sample
"""
import argparse
import json
import sys
from typing import Any, Dict, List, Optional
VALID_SUB_USE_CASES = ["novelty", "fto", "landscape", "diligence", "litigation"]
VALID_RISK = ["strict", "signal-gathering"]
# Strategy templates per sub-use-case
STRATEGIES = {
"novelty": {
"query_count": 6,
"sources": ["google_patents", "espacenet"],
"filters": {"date_filter": "any", "active_only": False},
"queries": [
{"type": "narrow_keyword", "count": 3, "source": "google_patents"},
{"type": "broad_concept", "count": 2, "source": "google_patents+espacenet"},
{"type": "cpc_class", "count": 1, "source": "google_patents", "after_initial": True},
],
"ranking_heuristic": "claim_text_overlap_with_invention_description",
"verdict_scale": ["NOVEL", "POTENTIALLY NOVEL", "NOT NOVEL"],
"docx_emphasis": {
"executive_summary": "expanded",
"closest_prior_art": "expanded",
"patent_landscape": "abbreviated",
"citation_graph_signals": "if_lens_only",
"geographic_coverage": "abbreviated",
"fto_flags": "skip",
"strategy_recommendations": "claim_differentiation_focus",
"audit_log": "standard",
},
"legal_disclaimer_mandatory": True,
},
"fto": {
"query_count": 12, # scales with jurisdiction count
"sources": ["google_patents", "espacenet", "uspto"],
"filters": {"date_filter": "priority_lt_today", "active_only": True, "jurisdiction_filtered": True},
"queries": [
{"type": "jurisdiction_filtered", "count": "2-3 per jurisdiction"},
{"type": "active_status_filter", "applied_to_all": True},
{"type": "cpc_class", "count": 1, "after_initial": True},
],
"ranking_heuristic": "claim_by_claim_infringement_risk",
"verdict_scale": ["CLEAR (per jurisdiction)", "FLAGGED", "HIGH RISK"],
"docx_emphasis": {
"executive_summary": "expanded",
"closest_prior_art": "abbreviated",
"patent_landscape": "abbreviated",
"citation_graph_signals": "if_lens_only",
"geographic_coverage": "expanded",
"fto_flags": "expanded_main_section",
"strategy_recommendations": "design_around_jurisdiction_focus",
"audit_log": "standard",
},
"legal_disclaimer_mandatory": True,
},
"landscape": {
"query_count": 9,
"sources": ["google_patents", "espacenet", "lens"],
"filters": {"date_filter": "10_year_window"},
"queries": [
{"type": "broad_technology", "count": "2-3"},
{"type": "cpc_class_extraction", "count": 1, "after_initial": True},
{"type": "per_top_filer", "count": "1 per top-5 filer"},
{"type": "lens_citation_graph", "count": "if_byok_available"},
],
"ranking_heuristic": "filer_count_plus_recency",
"verdict_scale": ["CONCENTRATED", "COMPETITIVE", "EMERGING"],
"docx_emphasis": {
"executive_summary": "standard",
"closest_prior_art": "abbreviated",
"patent_landscape": "expanded_main_section",
"citation_graph_signals": "expanded_if_lens",
"geographic_coverage": "expanded",
"fto_flags": "skip",
"strategy_recommendations": "who_to_watch_focus",
"audit_log": "standard",
},
"legal_disclaimer_mandatory": False,
},
"diligence": {
"query_count": 10,
"sources": ["google_patents", "uspto"],
"filters": {"date_filter": "any", "assignee_focused": True},
"queries": [
{"type": "assignee_search", "count": "2-3"},
{"type": "subsidiary_search", "count": "if_org_chart_provided"},
{"type": "inventor_search", "count": "for_key_inventors"},
{"type": "assignment_recordation", "count": "for_ownership_verification"},
{"type": "family_resolution", "applied_to_all": True},
],
"ranking_heuristic": "family_grouped_then_citation_count",
"verdict_scale": ["PORTFOLIO VERIFIED", "PARTIAL VERIFICATION", "OWNERSHIP RISK"],
"docx_emphasis": {
"executive_summary": "expanded",
"closest_prior_art": "abbreviated",
"patent_landscape": "expanded_as_portfolio_table",
"citation_graph_signals": "if_lens_only",
"geographic_coverage": "expanded",
"fto_flags": "skip",
"strategy_recommendations": "red_flags_in_portfolio",
"audit_log": "standard",
},
"legal_disclaimer_mandatory": False,
},
"litigation": {
"query_count": 7,
"sources": ["google_patents", "espacenet", "lens"],
"filters": {"date_filter": "before_target_priority_date"},
"queries": [
{"type": "fetch_target_patent", "extract": ["priority_date", "claims", "cpc_classes"]},
{"type": "cpc_class_with_date_filter", "count": 2},
{"type": "claim_language_with_date_filter", "count": 2},
{"type": "lens_forward_citations", "count": "if_byok_available"},
],
"ranking_heuristic": "knock_out_potential_claim_by_claim",
"verdict_scale": ["KNOCK-OUT FOUND", "STRONG OBVIOUSNESS COMBINATION", "WEAK OBVIOUSNESS", "NO MATERIAL ART"],
"docx_emphasis": {
"executive_summary": "expanded",
"closest_prior_art": "expanded_as_knock_out_candidates",
"patent_landscape": "abbreviated",
"citation_graph_signals": "expanded_if_lens",
"geographic_coverage": "abbreviated",
"fto_flags": "skip",
"strategy_recommendations": "per_claim_invalidity_analysis",
"audit_log": "standard",
},
"legal_disclaimer_mandatory": False,
},
}
def route(sub_use_case: str, jurisdictions: List[str], risk: Optional[str], known_art: Optional[str]) -> Dict[str, Any]:
if sub_use_case not in STRATEGIES:
raise ValueError(f"Invalid sub-use-case '{sub_use_case}'. Pick from: {list(STRATEGIES.keys())}")
strategy = STRATEGIES[sub_use_case].copy()
strategy["sub_use_case"] = sub_use_case
strategy["jurisdictions_input"] = jurisdictions
strategy["risk_input"] = risk
strategy["known_art_input"] = known_art
notes: List[str] = []
# FTO scales query count with jurisdictions
if sub_use_case == "fto" and jurisdictions:
per_jurisdiction = 3
strategy["query_count"] = len(jurisdictions) * per_jurisdiction + 2 # + CPC + active filter
notes.append(f"FTO query count scaled to {strategy['query_count']} for {len(jurisdictions)} jurisdiction(s)")
# Risk modifies ranking
if risk == "strict":
notes.append("Strict risk: aggressive ranking; surface verdict-grade hits only")
elif risk == "signal-gathering":
notes.append("Signal-gathering risk: prioritize breadth + visualization over verdict")
# Known art enables anchored search
if known_art and known_art.lower() != "none":
notes.append(f"Known art anchor: {known_art} — adjacent searches will reference this hit")
# Lens.org availability check (not asked here; flag in audit only)
notes.append("Lens.org BYOK: required for Citation Graph section. Check at runtime.")
if strategy["legal_disclaimer_mandatory"]:
notes.append("LEGAL DISCLAIMER MANDATORY: include in DOCX Sections 1, 7, 8")
strategy["operational_notes"] = notes
return strategy
def render_human(result: Dict[str, Any]) -> str:
out: List[str] = []
out.append(f"Sub-use-case: {result['sub_use_case']}")
out.append(f"Jurisdictions: {result.get('jurisdictions_input', []) or '(N/A for this sub-use-case)'}")
out.append(f"Risk tolerance: {result.get('risk_input', '(not specified)')}")
out.append(f"Known art: {result.get('known_art_input', '(none)')}")
out.append("")
out.append(f"Total query count: {result['query_count']}")
out.append(f"Sources: {', '.join(result['sources'])}")
out.append(f"Filters: {result['filters']}")
out.append("")
out.append("Query plan:")
for q in result["queries"]:
out.append(f" - {q}")
out.append("")
out.append(f"Ranking heuristic: {result['ranking_heuristic']}")
out.append(f"Verdict scale: {' / '.join(result['verdict_scale'])}")
out.append(f"Legal disclaimer mandatory: {result['legal_disclaimer_mandatory']}")
out.append("")
out.append("DOCX section emphasis:")
for section, emphasis in result["docx_emphasis"].items():
out.append(f" {section:<30s} {emphasis}")
out.append("")
if result.get("operational_notes"):
out.append("Operational notes:")
for n in result["operational_notes"]:
out.append(f" - {n}")
return "\n".join(out)
def main(argv: List[str]) -> int:
parser = argparse.ArgumentParser(description=__doc__.split("\n")[0])
parser.add_argument("--sub-use-case", choices=VALID_SUB_USE_CASES)
parser.add_argument("--jurisdictions", help="Comma-separated jurisdiction codes (US,EP,CN,JP,KR,PCT,worldwide)")
parser.add_argument("--risk", choices=VALID_RISK)
parser.add_argument("--known-art", help="Patent number or paper citation if user has seen prior art")
parser.add_argument("--sample", action="store_true", help="Run sample (FTO with US+EP jurisdictions, strict risk)")
parser.add_argument("--output", choices=["human", "json"], default="human")
args = parser.parse_args(argv)
if args.sample:
result = route("fto", ["US", "EP"], "strict", "US10000000B2")
elif args.sub_use_case:
jurisdictions = [j.strip() for j in args.jurisdictions.split(",") if j.strip()] if args.jurisdictions else []
try:
result = route(args.sub_use_case, jurisdictions, args.risk, args.known_art)
except ValueError as e:
print(f"error: {e}", file=sys.stderr); return 2
else:
parser.print_help(); return 0
if args.output == "json":
print(json.dumps(result, indent=2, default=str))
else:
print(render_human(result))
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
Lập kế hoạch, thiết kế và triển khai thử nghiệm A/B hoặc thử nghiệm chuyển đổi.
---
name: "ab-test-setup"
description: When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "conversion experiment," "statistical significance," or "test this." For tracking implementation, see analytics-tracking.
license: MIT
metadata:
version: 1.0.0
author: Alireza Rezvani
category: marketing
updated: 2026-03-06
---
# A/B Test Setup
You are an expert in experimentation and A/B testing. Your goal is to help design tests that produce statistically valid, actionable results.
## Initial Assessment
**Check for product marketing context first:**
If `.claude/product-marketing-context.md` exists, read it before asking questions. Use that context and only ask for information not already covered or specific to this task.
Before designing a test, understand:
1. **Test Context** - What are you trying to improve? What change are you considering?
2. **Current State** - Baseline conversion rate? Current traffic volume?
3. **Constraints** - Technical complexity? Timeline? Tools available?
---
## Core Principles
### 1. Start with a Hypothesis
- Not just "let's see what happens"
- Specific prediction of outcome
- Based on reasoning or data
### 2. Test One Thing
- Single variable per test
- Otherwise you don't know what worked
### 3. Statistical Rigor
- Pre-determine sample size
- Don't peek and stop early
- Commit to the methodology
### 4. Measure What Matters
- Primary metric tied to business value
- Secondary metrics for context
- Guardrail metrics to prevent harm
---
## Hypothesis Framework
### Structure
```
Because [observation/data],
we believe [change]
will cause [expected outcome]
for [audience].
We'll know this is true when [metrics].
```
### Example
**Weak**: "Changing the button color might increase clicks."
**Strong**: "Because users report difficulty finding the CTA (per heatmaps and feedback), we believe making the button larger and using contrasting color will increase CTA clicks by 15%+ for new visitors. We'll measure click-through rate from page view to signup start."
---
## Test Types
| Type | Description | Traffic Needed |
|------|-------------|----------------|
| A/B | Two versions, single change | Moderate |
| A/B/n | Multiple variants | Higher |
| MVT | Multiple changes in combinations | Very high |
| Split URL | Different URLs for variants | Moderate |
---
## Sample Size
### Quick Reference
| Baseline | 10% Lift | 20% Lift | 50% Lift |
|----------|----------|----------|----------|
| 1% | 150k/variant | 39k/variant | 6k/variant |
| 3% | 47k/variant | 12k/variant | 2k/variant |
| 5% | 27k/variant | 7k/variant | 1.2k/variant |
| 10% | 12k/variant | 3k/variant | 550/variant |
**Calculators:**
- [Evan Miller's](https://www.evanmiller.org/ab-testing/sample-size.html)
- [Optimizely's](https://www.optimizely.com/sample-size-calculator/)
**For detailed sample size tables and duration calculations**: See [references/sample-size-guide.md](references/sample-size-guide.md)
---
## Metrics Selection
### Primary Metric
- Single metric that matters most
- Directly tied to hypothesis
- What you'll use to call the test
### Secondary Metrics
- Support primary metric interpretation
- Explain why/how the change worked
### Guardrail Metrics
- Things that shouldn't get worse
- Stop test if significantly negative
### Example: Pricing Page Test
- **Primary**: Plan selection rate
- **Secondary**: Time on page, plan distribution
- **Guardrail**: Support tickets, refund rate
---
## Designing Variants
### What to Vary
| Category | Examples |
|----------|----------|
| Headlines/Copy | Message angle, value prop, specificity, tone |
| Visual Design | Layout, color, images, hierarchy |
| CTA | Button copy, size, placement, number |
| Content | Information included, order, amount, social proof |
### Best Practices
- Single, meaningful change
- Bold enough to make a difference
- True to the hypothesis
---
## Traffic Allocation
| Approach | Split | When to Use |
|----------|-------|-------------|
| Standard | 50/50 | Default for A/B |
| Conservative | 90/10, 80/20 | Limit risk of bad variant |
| Ramping | Start small, increase | Technical risk mitigation |
**Considerations:**
- Consistency: Users see same variant on return
- Balanced exposure across time of day/week
---
## Implementation
### Client-Side
- JavaScript modifies page after load
- Quick to implement, can cause flicker
- Tools: PostHog, Optimizely, VWO
### Server-Side
- Variant determined before render
- No flicker, requires dev work
- Tools: PostHog, LaunchDarkly, Split
---
## Running the Test
### Pre-Launch Checklist
- [ ] Hypothesis documented
- [ ] Primary metric defined
- [ ] Sample size calculated
- [ ] Variants implemented correctly
- [ ] Tracking verified
- [ ] QA completed on all variants
### During the Test
**DO:**
- Monitor for technical issues
- Check segment quality
- Document external factors
**DON'T:**
- Peek at results and stop early
- Make changes to variants
- Add traffic from new sources
### The Peeking Problem
Looking at results before reaching sample size and stopping early leads to false positives and wrong decisions. Pre-commit to sample size and trust the process.
---
## Analyzing Results
### Statistical Significance
- 95% confidence = p-value < 0.05
- Means <5% chance result is random
- Not a guarantee—just a threshold
### Analysis Checklist
1. **Reach sample size?** If not, result is preliminary
2. **Statistically significant?** Check confidence intervals
3. **Effect size meaningful?** Compare to MDE, project impact
4. **Secondary metrics consistent?** Support the primary?
5. **Guardrail concerns?** Anything get worse?
6. **Segment differences?** Mobile vs. desktop? New vs. returning?
### Interpreting Results
| Result | Conclusion |
|--------|------------|
| Significant winner | Implement variant |
| Significant loser | Keep control, learn why |
| No significant difference | Need more traffic or bolder test |
| Mixed signals | Dig deeper, maybe segment |
---
## Documentation
Document every test with:
- Hypothesis
- Variants (with screenshots)
- Results (sample, metrics, significance)
- Decision and learnings
**For templates**: See [references/test-templates.md](references/test-templates.md)
---
## Common Mistakes
### Test Design
- Testing too small a change (undetectable)
- Testing too many things (can't isolate)
- No clear hypothesis
### Execution
- Stopping early
- Changing things mid-test
- Not checking implementation
### Analysis
- Ignoring confidence intervals
- Cherry-picking segments
- Over-interpreting inconclusive results
---
## Task-Specific Questions
1. What's your current conversion rate?
2. How much traffic does this page get?
3. What change are you considering and why?
4. What's the smallest improvement worth detecting?
5. What tools do you have for testing?
6. Have you tested this area before?
---
## Proactive Triggers
Proactively offer A/B test design when:
1. **Conversion rate mentioned** — User shares a conversion rate and asks how to improve it; suggest designing a test rather than guessing at solutions.
2. **Copy or design decision is unclear** — When two variants of a headline, CTA, or layout are being debated, propose testing instead of opinionating.
3. **Campaign underperformance** — User reports a landing page or email performing below expectations; offer a structured test plan.
4. **Pricing page discussion** — Any mention of pricing page changes should trigger an offer to design a pricing test with guardrail metrics.
5. **Post-launch review** — After a feature or campaign goes live, propose follow-up experiments to optimize the result.
---
## Output Artifacts
| Artifact | Format | Description |
|----------|--------|-------------|
| Experiment Brief | Markdown doc | Hypothesis, variants, metrics, sample size, duration, owner |
| Sample Size Calculator Input | Table | Baseline rate, MDE, confidence level, power |
| Pre-Launch QA Checklist | Checklist | Implementation, tracking, variant rendering verification |
| Results Analysis Report | Markdown doc | Statistical significance, effect size, segment breakdown, decision |
| Test Backlog | Prioritized list | Ranked experiments by expected impact and feasibility |
---
## Communication
All outputs should meet the quality standard: clear hypothesis, pre-registered metrics, and documented decisions. Avoid presenting inconclusive results as wins. Every test should produce a learning, even if the variant loses. Reference `marketing-context` for product and audience framing before designing experiments.
---
## Related Skills
- **page-cro** — USE when you need ideas for *what* to test; NOT when you already have a hypothesis and just need test design.
- **analytics-tracking** — USE to set up measurement infrastructure before running tests; NOT as a substitute for defining primary metrics upfront.
- **campaign-analytics** — USE after tests conclude to fold results into broader campaign attribution; NOT during the test itself.
- **pricing-strategy** — USE when test results affect pricing decisions; NOT to replace a controlled test with pure strategic reasoning.
- **marketing-context** — USE as foundation before any test design to ensure hypotheses align with ICP and positioning; always load first.
FILE:references/sample-size-guide.md
# Sample Size Guide
Reference for calculating sample sizes and test duration.
## Sample Size Fundamentals
### Required Inputs
1. **Baseline conversion rate**: Your current rate
2. **Minimum detectable effect (MDE)**: Smallest change worth detecting
3. **Statistical significance level**: Usually 95% (α = 0.05)
4. **Statistical power**: Usually 80% (β = 0.20)
### What These Mean
**Baseline conversion rate**: If your page converts at 5%, that's your baseline.
**MDE (Minimum Detectable Effect)**: The smallest improvement you care about detecting. Set this based on:
- Business impact (is a 5% lift meaningful?)
- Implementation cost (worth the effort?)
- Realistic expectations (what have past tests shown?)
**Statistical significance (95%)**: Means there's less than 5% chance the observed difference is due to random chance.
**Statistical power (80%)**: Means if there's a real effect of size MDE, you have 80% chance of detecting it.
---
## Sample Size Quick Reference Tables
### Conversion Rate: 1%
| Lift to Detect | Sample per Variant | Total Sample |
|----------------|-------------------|--------------|
| 5% (1% → 1.05%) | 1,500,000 | 3,000,000 |
| 10% (1% → 1.1%) | 380,000 | 760,000 |
| 20% (1% → 1.2%) | 97,000 | 194,000 |
| 50% (1% → 1.5%) | 16,000 | 32,000 |
| 100% (1% → 2%) | 4,200 | 8,400 |
### Conversion Rate: 3%
| Lift to Detect | Sample per Variant | Total Sample |
|----------------|-------------------|--------------|
| 5% (3% → 3.15%) | 480,000 | 960,000 |
| 10% (3% → 3.3%) | 120,000 | 240,000 |
| 20% (3% → 3.6%) | 31,000 | 62,000 |
| 50% (3% → 4.5%) | 5,200 | 10,400 |
| 100% (3% → 6%) | 1,400 | 2,800 |
### Conversion Rate: 5%
| Lift to Detect | Sample per Variant | Total Sample |
|----------------|-------------------|--------------|
| 5% (5% → 5.25%) | 280,000 | 560,000 |
| 10% (5% → 5.5%) | 72,000 | 144,000 |
| 20% (5% → 6%) | 18,000 | 36,000 |
| 50% (5% → 7.5%) | 3,100 | 6,200 |
| 100% (5% → 10%) | 810 | 1,620 |
### Conversion Rate: 10%
| Lift to Detect | Sample per Variant | Total Sample |
|----------------|-------------------|--------------|
| 5% (10% → 10.5%) | 130,000 | 260,000 |
| 10% (10% → 11%) | 34,000 | 68,000 |
| 20% (10% → 12%) | 8,700 | 17,400 |
| 50% (10% → 15%) | 1,500 | 3,000 |
| 100% (10% → 20%) | 400 | 800 |
### Conversion Rate: 20%
| Lift to Detect | Sample per Variant | Total Sample |
|----------------|-------------------|--------------|
| 5% (20% → 21%) | 60,000 | 120,000 |
| 10% (20% → 22%) | 16,000 | 32,000 |
| 20% (20% → 24%) | 4,000 | 8,000 |
| 50% (20% → 30%) | 700 | 1,400 |
| 100% (20% → 40%) | 200 | 400 |
---
## Duration Calculator
### Formula
```
Duration (days) = (Sample per variant × Number of variants) / (Daily traffic × % exposed)
```
### Examples
**Scenario 1: High-traffic page**
- Need: 10,000 per variant (2 variants = 20,000 total)
- Daily traffic: 5,000 visitors
- 100% exposed to test
- Duration: 20,000 / 5,000 = **4 days**
**Scenario 2: Medium-traffic page**
- Need: 30,000 per variant (60,000 total)
- Daily traffic: 2,000 visitors
- 100% exposed
- Duration: 60,000 / 2,000 = **30 days**
**Scenario 3: Low-traffic with partial exposure**
- Need: 15,000 per variant (30,000 total)
- Daily traffic: 500 visitors
- 50% exposed to test
- Effective daily: 250
- Duration: 30,000 / 250 = **120 days** (too long!)
### Minimum Duration Rules
Even with sufficient sample size, run tests for at least:
- **1 full week**: To capture day-of-week variation
- **2 business cycles**: If B2B (weekday vs. weekend patterns)
- **Through paydays**: If e-commerce (beginning/end of month)
### Maximum Duration Guidelines
Avoid running tests longer than 4-8 weeks:
- Novelty effects wear off
- External factors intervene
- Opportunity cost of other tests
---
## Online Calculators
### Recommended Tools
**Evan Miller's Calculator**
https://www.evanmiller.org/ab-testing/sample-size.html
- Simple interface
- Bookmark-worthy
**Optimizely's Calculator**
https://www.optimizely.com/sample-size-calculator/
- Business-friendly language
- Duration estimates
**AB Test Guide Calculator**
https://www.abtestguide.com/calc/
- Includes Bayesian option
- Multiple test types
**VWO Duration Calculator**
https://vwo.com/tools/ab-test-duration-calculator/
- Duration-focused
- Good for planning
---
## Adjusting for Multiple Variants
With more than 2 variants (A/B/n tests), you need more sample:
| Variants | Multiplier |
|----------|------------|
| 2 (A/B) | 1x |
| 3 (A/B/C) | ~1.5x |
| 4 (A/B/C/D) | ~2x |
| 5+ | Consider reducing variants |
**Why?** More comparisons increase chance of false positives. You're comparing:
- A vs B
- A vs C
- B vs C (sometimes)
Apply Bonferroni correction or use tools that handle this automatically.
---
## Common Sample Size Mistakes
### 1. Underpowered tests
**Problem**: Not enough sample to detect realistic effects
**Fix**: Be realistic about MDE, get more traffic, or don't test
### 2. Overpowered tests
**Problem**: Waiting for sample size when you already have significance
**Fix**: This is actually fine—you committed to sample size, honor it
### 3. Wrong baseline rate
**Problem**: Using wrong conversion rate for calculation
**Fix**: Use the specific metric and page, not site-wide averages
### 4. Ignoring segments
**Problem**: Calculating for full traffic, then analyzing segments
**Fix**: If you plan segment analysis, calculate sample for smallest segment
### 5. Testing too many things
**Problem**: Dividing traffic too many ways
**Fix**: Prioritize ruthlessly, run fewer concurrent tests
---
## When Sample Size Requirements Are Too High
Options when you can't get enough traffic:
1. **Increase MDE**: Accept only detecting larger effects (20%+ lift)
2. **Lower confidence**: Use 90% instead of 95% (risky, document it)
3. **Reduce variants**: Test only the most promising variant
4. **Combine traffic**: Test across multiple similar pages
5. **Test upstream**: Test earlier in funnel where traffic is higher
6. **Don't test**: Make decision based on qualitative data instead
7. **Longer test**: Accept longer duration (weeks/months)
---
## Sequential Testing
If you must check results before reaching sample size:
### What is it?
Statistical method that adjusts for multiple looks at data.
### When to use
- High-risk changes
- Need to stop bad variants early
- Time-sensitive decisions
### Tools that support it
- Optimizely (Stats Accelerator)
- VWO (SmartStats)
- PostHog (Bayesian approach)
### Tradeoff
- More flexibility to stop early
- Slightly larger sample size requirement
- More complex analysis
---
## Quick Decision Framework
### Can I run this test?
```
Daily traffic to page: _____
Baseline conversion rate: _____
MDE I care about: _____
Sample needed per variant: _____ (from tables above)
Days to run: Sample / Daily traffic = _____
If days > 60: Consider alternatives
If days > 30: Acceptable for high-impact tests
If days < 14: Likely feasible
If days < 7: Easy to run, consider running longer anyway
```
FILE:references/test-templates.md
# A/B Test Templates Reference
Templates for planning, documenting, and analyzing experiments.
## Test Plan Template
```markdown
# A/B Test: [Name]
## Overview
- **Owner**: [Name]
- **Test ID**: [ID in testing tool]
- **Page/Feature**: [What's being tested]
- **Planned dates**: [Start] - [End]
## Hypothesis
Because [observation/data],
we believe [change]
will cause [expected outcome]
for [audience].
We'll know this is true when [metrics].
## Test Design
| Element | Details |
|---------|---------|
| Test type | A/B / A/B/n / MVT |
| Duration | X weeks |
| Sample size | X per variant |
| Traffic allocation | 50/50 |
| Tool | [Tool name] |
| Implementation | Client-side / Server-side |
## Variants
### Control (A)
[Screenshot]
- Current experience
- [Key details about current state]
### Variant (B)
[Screenshot or mockup]
- [Specific change #1]
- [Specific change #2]
- Rationale: [Why we think this will win]
## Metrics
### Primary
- **Metric**: [metric name]
- **Definition**: [how it's calculated]
- **Current baseline**: [X%]
- **Minimum detectable effect**: [X%]
### Secondary
- [Metric 1]: [what it tells us]
- [Metric 2]: [what it tells us]
- [Metric 3]: [what it tells us]
### Guardrails
- [Metric that shouldn't get worse]
- [Another safety metric]
## Segment Analysis Plan
- Mobile vs. desktop
- New vs. returning visitors
- Traffic source
- [Other relevant segments]
## Success Criteria
- Winner: [Primary metric improves by X% with 95% confidence]
- Loser: [Primary metric decreases significantly]
- Inconclusive: [What we'll do if no significant result]
## Pre-Launch Checklist
- [ ] Hypothesis documented and reviewed
- [ ] Primary metric defined and trackable
- [ ] Sample size calculated
- [ ] Test duration estimated
- [ ] Variants implemented correctly
- [ ] Tracking verified in all variants
- [ ] QA completed on all variants
- [ ] Stakeholders informed
- [ ] Calendar hold for analysis date
```
---
## Results Documentation Template
```markdown
# A/B Test Results: [Name]
## Summary
| Element | Value |
|---------|-------|
| Test ID | [ID] |
| Dates | [Start] - [End] |
| Duration | X days |
| Result | Winner / Loser / Inconclusive |
| Decision | [What we're doing] |
## Hypothesis (Reminder)
[Copy from test plan]
## Results
### Sample Size
| Variant | Target | Actual | % of target |
|---------|--------|--------|-------------|
| Control | X | Y | Z% |
| Variant | X | Y | Z% |
### Primary Metric: [Metric Name]
| Variant | Value | 95% CI | vs. Control |
|---------|-------|--------|-------------|
| Control | X% | [X%, Y%] | — |
| Variant | X% | [X%, Y%] | +X% |
**Statistical significance**: p = X.XX (95% = sig / not sig)
**Practical significance**: [Is this lift meaningful for the business?]
### Secondary Metrics
| Metric | Control | Variant | Change | Significant? |
|--------|---------|---------|--------|--------------|
| [Metric 1] | X | Y | +Z% | Yes/No |
| [Metric 2] | X | Y | +Z% | Yes/No |
### Guardrail Metrics
| Metric | Control | Variant | Change | Concern? |
|--------|---------|---------|--------|----------|
| [Metric 1] | X | Y | +Z% | Yes/No |
### Segment Analysis
**Mobile vs. Desktop**
| Segment | Control | Variant | Lift |
|---------|---------|---------|------|
| Mobile | X% | Y% | +Z% |
| Desktop | X% | Y% | +Z% |
**New vs. Returning**
| Segment | Control | Variant | Lift |
|---------|---------|---------|------|
| New | X% | Y% | +Z% |
| Returning | X% | Y% | +Z% |
## Interpretation
### What happened?
[Explanation of results in plain language]
### Why do we think this happened?
[Analysis and reasoning]
### Caveats
[Any limitations, external factors, or concerns]
## Decision
**Winner**: [Control / Variant]
**Action**: [Implement variant / Keep control / Re-test]
**Timeline**: [When changes will be implemented]
## Learnings
### What we learned
- [Key insight 1]
- [Key insight 2]
### What to test next
- [Follow-up test idea 1]
- [Follow-up test idea 2]
### Impact
- **Projected lift**: [X% improvement in Y metric]
- **Business impact**: [Revenue, conversions, etc.]
```
---
## Test Repository Entry Template
For tracking all tests in a central location:
```markdown
| Test ID | Name | Page | Dates | Primary Metric | Result | Lift | Link |
|---------|------|------|-------|----------------|--------|------|------|
| 001 | Hero headline test | Homepage | 1/1-1/15 | CTR | Winner | +12% | [Link] |
| 002 | Pricing table layout | Pricing | 1/10-1/31 | Plan selection | Loser | -5% | [Link] |
| 003 | Signup form fields | Signup | 2/1-2/14 | Completion | Inconclusive | +2% | [Link] |
```
---
## Quick Test Brief Template
For simple tests that don't need full documentation:
```markdown
## [Test Name]
**What**: [One sentence description]
**Why**: [One sentence hypothesis]
**Metric**: [Primary metric]
**Duration**: [X weeks]
**Result**: [TBD / Winner / Loser / Inconclusive]
**Learnings**: [Key takeaway]
```
---
## Stakeholder Update Template
```markdown
## A/B Test Update: [Name]
**Status**: Running / Complete
**Days remaining**: X (or complete)
**Current sample**: X% of target
### Preliminary observations
[What we're seeing - without making decisions yet]
### Next steps
[What happens next]
### Timeline
- [Date]: Analysis complete
- [Date]: Decision and recommendation
- [Date]: Implementation (if winner)
```
---
## Experiment Prioritization Scorecard
For deciding which tests to run:
| Factor | Weight | Test A | Test B | Test C |
|--------|--------|--------|--------|--------|
| Potential impact | 30% | | | |
| Confidence in hypothesis | 25% | | | |
| Ease of implementation | 20% | | | |
| Risk if wrong | 15% | | | |
| Strategic alignment | 10% | | | |
| **Total** | | | | |
Scoring: 1-5 (5 = best)
---
## Hypothesis Bank Template
For collecting test ideas:
```markdown
| ID | Page/Area | Observation | Hypothesis | Potential Impact | Status |
|----|-----------|-------------|------------|------------------|--------|
| H1 | Homepage | Low scroll depth | Shorter hero will increase scroll | High | Testing |
| H2 | Pricing | Users compare plans | Comparison table will help | Medium | Backlog |
| H3 | Signup | Drop-off at email | Social login will increase completion | Medium | Backlog |
```
FILE:scripts/sample_size_calculator.py
#!/usr/bin/env python3
"""
sample_size_calculator.py — A/B Test Sample Size Calculator
100% stdlib, no pip installs required.
Usage:
python3 sample_size_calculator.py # demo mode
python3 sample_size_calculator.py --baseline 0.05 --mde 0.20
python3 sample_size_calculator.py --baseline 0.05 --mde 0.20 --daily-traffic 500
python3 sample_size_calculator.py --baseline 0.05 --mde 0.20 --json
"""
import argparse
import json
import math
import sys
# ---------------------------------------------------------------------------
# Z-score approximation (scipy-free, Beasley-Springer-Moro algorithm)
# ---------------------------------------------------------------------------
def _norm_ppf(p: float) -> float:
"""Percent-point function (inverse CDF) of the standard normal.
Uses rational approximation — accurate to ~1e-9.
Reference: Abramowitz & Stegun 26.2.17 / Peter Acklam's algorithm.
"""
if p <= 0 or p >= 1:
raise ValueError(f"p must be in (0, 1), got {p}")
# Coefficients for rational approximation
a = [-3.969683028665376e+01, 2.209460984245205e+02,
-2.759285104469687e+02, 1.383577518672690e+02,
-3.066479806614716e+01, 2.506628277459239e+00]
b = [-5.447609879822406e+01, 1.615858368580409e+02,
-1.556989798598866e+02, 6.680131188771972e+01,
-1.328068155288572e+01]
c = [-7.784894002430293e-03, -3.223964580411365e-01,
-2.400758277161838e+00, -2.549732539343734e+00,
4.374664141464968e+00, 2.938163982698783e+00]
d = [7.784695709041462e-03, 3.224671290700398e-01,
2.445134137142996e+00, 3.754408661907416e+00]
p_low = 0.02425
p_high = 1 - p_low
if p < p_low:
q = math.sqrt(-2 * math.log(p))
return (((((c[0]*q+c[1])*q+c[2])*q+c[3])*q+c[4])*q+c[5]) / \
((((d[0]*q+d[1])*q+d[2])*q+d[3])*q+1)
elif p <= p_high:
q = p - 0.5
r = q * q
return (((((a[0]*r+a[1])*r+a[2])*r+a[3])*r+a[4])*r+a[5])*q / \
(((((b[0]*r+b[1])*r+b[2])*r+b[3])*r+b[4])*r+1)
else:
q = math.sqrt(-2 * math.log(1 - p))
return -(((((c[0]*q+c[1])*q+c[2])*q+c[3])*q+c[4])*q+c[5]) / \
((((d[0]*q+d[1])*q+d[2])*q+d[3])*q+1)
# ---------------------------------------------------------------------------
# Core calculation
# ---------------------------------------------------------------------------
def calculate_sample_size(
baseline: float,
mde: float,
alpha: float = 0.05,
power: float = 0.80,
) -> dict:
"""
Two-proportion z-test sample size formula (two-tailed).
n = (Z_alpha/2 + Z_beta)^2 * (p1*(1-p1) + p2*(1-p2)) / (p2 - p1)^2
Args:
baseline : baseline conversion rate (e.g. 0.05 for 5%)
mde : minimum detectable effect as relative lift (e.g. 0.20 for +20%)
alpha : significance level (Type I error rate), default 0.05
power : statistical power (1 - Type II error rate), default 0.80
Returns dict with all intermediate values and results.
"""
p1 = baseline
p2 = baseline * (1 + mde) # expected conversion with treatment
if not (0 < p1 < 1):
raise ValueError(f"baseline must be in (0,1), got {p1}")
if not (0 < p2 < 1):
raise ValueError(
f"baseline * (1 + mde) = {p2:.4f} is outside (0,1). "
"Reduce mde or increase baseline."
)
z_alpha = _norm_ppf(1 - alpha / 2) # two-tailed
z_beta = _norm_ppf(power)
pooled_var = p1 * (1 - p1) + p2 * (1 - p2)
effect_sq = (p2 - p1) ** 2
n_raw = ((z_alpha + z_beta) ** 2 * pooled_var) / effect_sq
n = math.ceil(n_raw)
return {
"inputs": {
"baseline_conversion_rate": p1,
"minimum_detectable_effect_relative": mde,
"expected_variant_conversion_rate": round(p2, 6),
"significance_level_alpha": alpha,
"statistical_power": power,
},
"z_scores": {
"z_alpha_2": round(z_alpha, 4),
"z_beta": round(z_beta, 4),
},
"results": {
"sample_size_per_variation": n,
"total_sample_size": n * 2,
"absolute_lift": round(p2 - p1, 6),
"relative_lift_pct": round(mde * 100, 2),
},
"formula": (
"n = (Z_α/2 + Z_β)² × (p1(1−p1) + p2(1−p2)) / (p2−p1)² "
"[two-proportion z-test, two-tailed]"
),
"assumptions": [
"Two-tailed test (detecting lift in either direction)",
"Independent samples (no within-subject correlation)",
"Fixed horizon (not sequential / always-valid)",
"Binomial outcome (conversion yes/no)",
"No novelty effect correction applied",
],
}
def add_duration(result: dict, daily_traffic: int) -> dict:
"""Append estimated test duration given total daily traffic (both variants)."""
n_total = result["results"]["total_sample_size"]
days = math.ceil(n_total / daily_traffic)
weeks = round(days / 7, 1)
result["duration"] = {
"daily_traffic_both_variants": daily_traffic,
"estimated_days": days,
"estimated_weeks": weeks,
"note": (
"Assumes traffic is evenly split 50/50 between control and variant. "
"Add ~10–20% buffer for weekday/weekend variance."
),
}
return result
# ---------------------------------------------------------------------------
# Scoring helper (0-100)
# ---------------------------------------------------------------------------
def score_test_design(result: dict) -> dict:
"""Heuristic quality score for the A/B test design."""
score = 100
reasons = []
inputs = result["inputs"]
# Penalise very low baseline (unreliable estimates)
if inputs["baseline_conversion_rate"] < 0.01:
score -= 15
reasons.append("Baseline <1%: high variance, consider aggregating more data first.")
# Penalise tiny MDE (will need enormous sample)
mde = inputs["minimum_detectable_effect_relative"]
if mde < 0.05:
score -= 20
reasons.append("MDE <5%: very small effect, experiment may take months.")
elif mde < 0.10:
score -= 10
reasons.append("MDE <10%: moderately small effect size.")
# Penalise overly aggressive alpha
if inputs["significance_level_alpha"] > 0.10:
score -= 15
reasons.append("α >10%: high false-positive risk.")
# Penalise low power
if inputs["statistical_power"] < 0.80:
score -= 20
reasons.append("Power <80%: elevated risk of missing real effects (Type II error).")
# Duration penalty (if available)
dur = result.get("duration")
if dur:
days = dur["estimated_days"]
if days > 90:
score -= 20
reasons.append(f"Test duration {days}d >90 days: novelty/seasonal effects likely.")
elif days > 30:
score -= 10
reasons.append(f"Test duration {days}d >30 days: monitor for external confounders.")
score = max(0, score)
return {
"design_quality_score": score,
"score_interpretation": _score_label(score),
"issues": reasons if reasons else ["No major design issues detected."],
}
def _score_label(s: int) -> str:
if s >= 90: return "Excellent"
if s >= 75: return "Good"
if s >= 60: return "Fair"
if s >= 40: return "Poor"
return "Critical"
# ---------------------------------------------------------------------------
# Pretty-print
# ---------------------------------------------------------------------------
def pretty_print(result: dict, score: dict) -> None:
inp = result["inputs"]
res = result["results"]
zs = result["z_scores"]
print("\n" + "=" * 60)
print(" A/B TEST SAMPLE SIZE CALCULATOR")
print("=" * 60)
print("\n📥 INPUTS")
print(f" Baseline conversion rate : {inp['baseline_conversion_rate']*100:.2f}%")
print(f" Variant conversion rate : {inp['expected_variant_conversion_rate']*100:.2f}%")
print(f" Minimum detectable effect: {inp['minimum_detectable_effect_relative']*100:.1f}% relative "
f"(+{res['absolute_lift']*100:.3f}pp absolute)")
print(f" Significance level (α) : {inp['significance_level_alpha']}")
print(f" Statistical power : {inp['statistical_power']*100:.0f}%")
print("\n📐 FORMULA")
print(f" {result['formula']}")
print(f" Z_α/2 = {zs['z_alpha_2']} Z_β = {zs['z_beta']}")
print("\n📊 RESULTS")
print(f" ✅ Sample size per variation : {res['sample_size_per_variation']:,}")
print(f" ✅ Total sample size (both) : {res['total_sample_size']:,}")
if "duration" in result:
d = result["duration"]
print(f"\n⏱️ DURATION ESTIMATE (traffic: {d['daily_traffic_both_variants']:,}/day)")
print(f" Estimated test duration : {d['estimated_days']} days (~{d['estimated_weeks']} weeks)")
print(f" Note: {d['note']}")
print("\n💡 ASSUMPTIONS")
for a in result["assumptions"]:
print(f" • {a}")
print(f"\n🎯 DESIGN QUALITY SCORE: {score['design_quality_score']}/100 ({score['score_interpretation']})")
for issue in score["issues"]:
print(f" ⚠ {issue}")
print()
# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------
def parse_args():
parser = argparse.ArgumentParser(
description="Calculate required sample size for an A/B test (stdlib only).",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog=__doc__,
)
parser.add_argument("--baseline", type=float, default=None,
help="Baseline conversion rate (e.g. 0.05 for 5%%)")
parser.add_argument("--mde", type=float, default=None,
help="Minimum detectable effect as relative lift (e.g. 0.20 for +20%%)")
parser.add_argument("--alpha", type=float, default=0.05,
help="Significance level α (default: 0.05)")
parser.add_argument("--power", type=float, default=0.80,
help="Statistical power 1-β (default: 0.80)")
parser.add_argument("--daily-traffic", type=int, default=None,
help="Total daily visitors across both variants (for duration estimate)")
parser.add_argument("--json", action="store_true",
help="Output results as JSON")
return parser.parse_args()
DEMO_SCENARIOS = [
{"label": "E-commerce checkout (low baseline)",
"baseline": 0.03, "mde": 0.20, "alpha": 0.05, "power": 0.80, "daily_traffic": 800},
{"label": "SaaS free-trial signup (medium baseline)",
"baseline": 0.08, "mde": 0.15, "alpha": 0.05, "power": 0.80, "daily_traffic": 2000},
{"label": "Button CTA (high baseline)",
"baseline": 0.25, "mde": 0.10, "alpha": 0.05, "power": 0.80, "daily_traffic": 5000},
]
def main():
args = parse_args()
demo_mode = (args.baseline is None and args.mde is None)
if demo_mode:
print("🔬 DEMO MODE — running 3 sample scenarios\n")
all_results = []
for sc in DEMO_SCENARIOS:
res = calculate_sample_size(sc["baseline"], sc["mde"], sc["alpha"], sc["power"])
res = add_duration(res, sc["daily_traffic"])
sc_score = score_test_design(res)
res["scenario"] = sc["label"]
res["score"] = sc_score
all_results.append(res)
if not args.json:
print(f"\n{'─'*60}")
print(f"SCENARIO: {sc['label']}")
pretty_print(res, sc_score)
if args.json:
print(json.dumps(all_results, indent=2))
return
# Single calculation mode
if args.baseline is None or args.mde is None:
print("Error: --baseline and --mde are required (or omit both for demo mode).", file=sys.stderr)
sys.exit(1)
result = calculate_sample_size(args.baseline, args.mde, args.alpha, args.power)
if args.daily_traffic:
result = add_duration(result, args.daily_traffic)
sc_score = score_test_design(result)
result["score"] = sc_score
if args.json:
print(json.dumps(result, indent=2))
else:
pretty_print(result, sc_score)
if __name__ == "__main__":
main()