skill

---
name: "agile-product-owner"
description: Agile product ownership for backlog management and sprint execution. Covers user story writing, acceptance criteria, sprint planning, and velocity tracking. Use for writing user stories, creating acceptance criteria, planning sprints, estimating story points, breaking down epics, or prioritizing backlog.
not_for: Kanban-only workflows, waterfall project planning, general task management, non-Scrum agile frameworks (SAFe, LeSS) without adaptation
triggers:
  - write user story
  - create acceptance criteria
  - plan sprint
  - estimate story points
  - break down epic
  - prioritize backlog
  - sprint planning
  - backlog grooming
  - sprint retrospective
  - definition of done
  - INVEST criteria
  - Given When Then
  - user story template
  - sprint capacity
  - velocity tracking
---

# Agile Product Owner

Backlog management and sprint execution toolkit for product owners, including user story generation, acceptance criteria patterns, sprint planning, and velocity tracking.

---

## Table of Contents

- [What Makes This Skill Different](#what-makes-this-skill-different)
- [User Story Generation Workflow](#user-story-generation-workflow)
- [Acceptance Criteria Patterns](#acceptance-criteria-patterns)
- [Epic Breakdown Workflow](#epic-breakdown-workflow)
- [Sprint Planning Workflow](#sprint-planning-workflow)
- [Backlog Prioritization](#backlog-prioritization)
- [Reference Documentation](#reference-documentation)
- [Tools](#tools)

---

## What Makes This Skill Different

- **Capacity math that aligns with reality:** sprint capacity is based on velocity × availability factor, not hope.
- **Acceptance criteria scaled by story size:** minimum AC counts map to story points to avoid under-spec'ing large items.
- **Weighted prioritization that stays consistent:** value 40%, impact 30%, risk 15%, effort 15% keeps tradeoffs explicit.
- **Systematic epic splitting techniques:** five concrete split patterns prevent oversized stories.
- **INVEST validation baked into workflows:** every story includes a validation step, not just guidance.

## User Story Generation Workflow

Create INVEST-compliant user stories from requirements:

1. Identify the persona (who benefits from this feature)
2. Define the action or capability needed
3. Articulate the benefit or value delivered
4. Write acceptance criteria using Given-When-Then
5. Estimate story points using Fibonacci scale
6. Validate against INVEST criteria
7. Add to backlog with priority
8. **Validation:** Story passes all INVEST criteria; acceptance criteria are testable

### User Story Template

```
As a [persona],
I want to [action/capability],
So that [benefit/value].
```

**Example:**
```
As a marketing manager,
I want to export campaign reports to PDF,
So that I can share results with stakeholders who don't have system access.
```

### Story Types

| Type | Template | Example |
|------|----------|---------|
| Feature | As a [persona], I want to [action] so that [benefit] | As a user, I want to filter search results so that I find items faster |
| Improvement | As a [persona], I need [capability] to [goal] | As a user, I need faster page loads to complete tasks without frustration |
| Bug Fix | As a [persona], I expect [behavior] when [condition] | As a user, I expect my cart to persist when I refresh the page |
| Enabler | As a developer, I need to [technical task] to enable [capability] | As a developer, I need to implement caching to enable instant search |

### Persona Reference

| Persona | Typical Needs | Context |
|---------|--------------|---------|
| End User | Efficiency, simplicity, reliability | Daily feature usage |
| Administrator | Control, visibility, security | System management |
| Power User | Automation, customization, shortcuts | Expert workflows |
| New User | Guidance, learning, safety | Onboarding |

---

## Acceptance Criteria Patterns

Write testable acceptance criteria using Given-When-Then format.

### Given-When-Then Template

```
Given [precondition/context],
When [action/trigger],
Then [expected outcome].
```

**Examples:**
```
Given the user is logged in with valid credentials,
When they click the "Export" button,
Then a PDF download starts within 2 seconds.

Given the user has entered an invalid email format,
When they submit the registration form,
Then an inline error message displays "Please enter a valid email address."

Given the shopping cart contains items,
When the user refreshes the browser,
Then the cart contents remain unchanged.
```

### Acceptance Criteria Checklist

Each story should include criteria for:

| Category | Example |
|----------|---------|
| Happy Path | Given valid input, When submitted, Then success message displayed |
| Validation | Should reject input when required field is empty |
| Error Handling | Must show user-friendly message when API fails |
| Performance | Should complete operation within 2 seconds |
| Accessibility | Must be navigable via keyboard only |

### Minimum Criteria by Story Size

| Story Points | Minimum AC Count |
|--------------|------------------|
| 1-2 | 3-4 criteria |
| 3-5 | 4-6 criteria |
| 8 | 5-8 criteria |
| 13+ | Split the story |

See `references/user-story-templates.md` for complete template library.

---

## Epic Breakdown Workflow

Break epics into deliverable sprint-sized stories:

1. Define epic scope and success criteria
2. Identify all personas affected by the epic
3. List all capabilities needed for each persona
4. Group capabilities into logical stories
5. Validate each story is ≤8 points
6. Identify dependencies between stories
7. Sequence stories for incremental delivery
8. **Validation:** Each story delivers standalone value; total stories cover epic scope

### Splitting Techniques

| Technique | When to Use | Example |
|-----------|-------------|---------|
| By workflow step | Linear process | "Checkout" → "Add to cart" + "Enter payment" + "Confirm order" |
| By persona | Multiple user types | "Dashboard" → "Admin dashboard" + "User dashboard" |
| By data type | Multiple inputs | "Import" → "Import CSV" + "Import Excel" |
| By operation | CRUD functionality | "Manage users" → "Create" + "Edit" + "Delete" |
| Happy path first | Risk reduction | "Feature" → "Basic flow" + "Error handling" + "Edge cases" |

### Epic Example

**Epic:** User Dashboard

**Breakdown:**
```
Epic: User Dashboard (34 points total)
├── US-001: View key metrics (5 pts) - End User
├── US-002: Customize layout (5 pts) - Power User
├── US-003: Export data to CSV (3 pts) - End User
├── US-004: Share with team (5 pts) - End User
├── US-005: Set up alerts (5 pts) - Power User
├── US-006: Filter by date range (3 pts) - End User
├── US-007: Admin overview (5 pts) - Admin
└── US-008: Enable caching (3 pts) - Enabler
```

---

## Sprint Planning Workflow

Plan sprint capacity and select stories:

1. Calculate team capacity (velocity × availability)
2. Review sprint goal with stakeholders
3. Select stories from prioritized backlog
4. Fill to 80-85% of capacity (committed)
5. Add stretch goals (10-15% additional)
6. Identify dependencies and risks
7. Break complex stories into tasks
8. **Validation:** Committed points ≤85% capacity; all stories have acceptance criteria

### Capacity Calculation

```
Sprint Capacity = Average Velocity × Availability Factor

Example:
Average Velocity: 30 points
Team availability: 90% (one member partially out)
Adjusted Capacity: 27 points

Committed: 23 points (85% of 27)
Stretch: 4 points (15% of 27)
```

### Availability Factors

| Scenario | Factor |
|----------|--------|
| Full sprint, no PTO | 1.0 |
| One team member out 50% | 0.9 |
| Holiday during sprint | 0.8 |
| Multiple members out | 0.7 |

### Sprint Loading Template

```
Sprint Capacity: 27 points
Sprint Goal: [Clear, measurable objective]

COMMITTED (23 points):
[H] US-001: User dashboard (5 pts)
[H] US-002: Export feature (3 pts)
[H] US-003: Search filter (5 pts)
[M] US-004: Settings page (5 pts)
[M] US-005: Help tooltips (3 pts)
[L] US-006: Theme options (2 pts)

STRETCH (4 points):
[L] US-007: Sort options (2 pts)
[L] US-008: Print view (2 pts)
```

See `references/sprint-planning-guide.md` for complete planning procedures.

---

## Backlog Prioritization

Prioritize backlog using value and effort assessment.

### Priority Levels

| Priority | Definition | Sprint Target |
|----------|------------|---------------|
| Critical | Blocking users, security, data loss | Immediate |
| High | Core functionality, key user needs | This sprint |
| Medium | Improvements, enhancements | Next 2-3 sprints |
| Low | Nice-to-have, minor improvements | Backlog |

### Prioritization Factors

| Factor | Weight | Questions |
|--------|--------|-----------|
| Business Value | 40% | Revenue impact? User demand? Strategic alignment? |
| User Impact | 30% | How many users? How frequently used? |
| Risk/Dependencies | 15% | Technical risk? External dependencies? |
| Effort | 15% | Size? Complexity? Uncertainty? |

### INVEST Criteria Validation

Before adding to sprint, validate each story:

| Criterion | Question | Pass If... |
|-----------|----------|------------|
| **I**ndependent | Can this be developed without other uncommitted stories? | No blocking dependencies |
| **N**egotiable | Is the implementation flexible? | Multiple approaches possible |
| **V**aluable | Does this deliver user or business value? | Clear benefit in "so that" |
| **E**stimable | Can the team estimate this? | Understood well enough to size |
| **S**mall | Can this complete in one sprint? | ≤8 story points |
| **T**estable | Can we verify this is done? | Clear acceptance criteria |

---

## Reference Documentation

### User Story Templates

`references/user-story-templates.md` contains:

- Standard story formats by type (feature, improvement, bug fix, enabler)
- Acceptance criteria patterns (Given-When-Then, Should/Must/Can)
- INVEST criteria validation checklist
- Story point estimation guide (Fibonacci scale)
- Common story antipatterns and fixes
- Story splitting techniques

### Sprint Planning Guide

`references/sprint-planning-guide.md` contains:

- Sprint planning meeting agenda
- Capacity calculation formulas
- Backlog prioritization framework (WSJF)
- Sprint ceremony guides (standup, review, retro)
- Velocity tracking and burndown patterns
- Definition of Done checklist
- Sprint metrics and targets

---

## Tools

### User Story Generator

```bash
# Generate stories from sample epic
python scripts/user_story_generator.py

# Plan sprint with capacity
python scripts/user_story_generator.py sprint 30
```

Generates:
- INVEST-compliant user stories
- Given-When-Then acceptance criteria
- Story point estimates (Fibonacci scale)
- Priority assignments
- Sprint loading with committed and stretch items

### Sample Output

```
USER STORY: USR-001
========================================
Title: View Key Metrics
Type: story
Priority: HIGH
Points: 5

Story:
As a End User, I want to view key metrics and KPIs
so that I can save time and work more efficiently

Acceptance Criteria:
  1. Given user has access, When they view key metrics, Then the result is displayed
  2. Should validate input before processing
  3. Must show clear error message when action fails
  4. Should complete within 2 seconds
  5. Must be accessible via keyboard navigation

INVEST Checklist:
  ✓ Independent
  ✓ Negotiable
  ✓ Valuable
  ✓ Estimable
  ✓ Small
  ✓ Testable
```

---

## Sprint Metrics

Track sprint health and team performance.

### Key Metrics

| Metric | Formula | Target |
|--------|---------|--------|
| Velocity | Points completed / sprint | Stable ±10% |
| Commitment Reliability | Completed / Committed | >85% |
| Scope Change | Points added or removed mid-sprint | <10% |
| Carryover | Points not completed | <15% |

### Velocity Tracking

```
Sprint 1: 25 points
Sprint 2: 28 points
Sprint 3: 30 points
Sprint 4: 32 points
Sprint 5: 29 points
------------------------
Average Velocity: 28.8 points
Trend: Stable

Planning: Commit to 24-26 points
```

### Definition of Done

Story is complete when:

- [ ] Code complete and peer reviewed
- [ ] Unit tests written and passing
- [ ] Acceptance criteria verified
- [ ] Documentation updated
- [ ] Deployed to staging environment
- [ ] Product Owner accepted
- [ ] No critical bugs remaining

## Related Skills

- **Scrum Master** (`project-management/scrum-master/`) — Velocity data and sprint ceremonies complement backlog management
- **Product Manager Toolkit** (`product-team/product-manager-toolkit/`) — RICE prioritization feeds backlog ordering

FILE:assets/sprint_planning_template.md
# Sprint Planning Document

## Sprint Info

| Field | Value |
|-------|-------|
| **Sprint** | Sprint [Number] |
| **Dates** | [Start Date] - [End Date] |
| **Sprint Goal** | [One sentence describing what this sprint achieves] |
| **Scrum Master** | [Name] |
| **Product Owner** | [Name] |

---

## Sprint Goal

[Expand on the sprint goal. What is the single most important outcome? What does "done" look like for this sprint? 2-3 sentences.]

---

## Team Capacity

| Team Member | Available Days | Notes |
|------------|---------------|-------|
| [Name] | [X] / [Total] | [PTO, training, on-call, etc.] |
| [Name] | [X] / [Total] | |
| [Name] | [X] / [Total] | |

**Total Capacity:** [X] person-days
**Historical Velocity:** [X] story points (avg of last 3 sprints)
**Planned Velocity:** [X] story points (adjusted for capacity)

---

## Selected Stories

| # | Story | Points | Assignee | Priority | Status |
|---|-------|--------|----------|----------|--------|
| 1 | [Story title with ticket link] | [SP] | [Name] | Must Have | To Do |
| 2 | [Story title with ticket link] | [SP] | [Name] | Must Have | To Do |
| 3 | [Story title with ticket link] | [SP] | [Name] | Should Have | To Do |
| 4 | [Story title with ticket link] | [SP] | [Name] | Nice to Have | To Do |

**Total Planned:** [X] story points

---

## Dependencies

| Dependency | Blocking Story | Owner | Status | Due Date |
|-----------|---------------|-------|--------|----------|
| [External API ready] | [Story #] | [Name/Team] | [Status] | [Date] |
| [Design review complete] | [Story #] | [Name] | [Status] | [Date] |

---

## Risks

| Risk | Impact | Likelihood | Mitigation |
|------|--------|-----------|-----------|
| [Risk description] | High/Med/Low | High/Med/Low | [Plan] |
| [Risk description] | High/Med/Low | High/Med/Low | [Plan] |

---

## Definition of Done

- [ ] Code complete and peer reviewed
- [ ] Unit tests written and passing
- [ ] Acceptance criteria verified
- [ ] Documentation updated (if applicable)
- [ ] Deployed to staging environment
- [ ] QA sign-off received
- [ ] Product Owner acceptance

---

## Sprint Ceremonies

| Ceremony | Day/Time | Duration |
|----------|----------|----------|
| Daily Standup | [Day] [Time] | 15 min |
| Sprint Review | [Date] [Time] | 60 min |
| Sprint Retrospective | [Date] [Time] | 60 min |

FILE:assets/user_story_template.md
# User Story

## Story Info

| Field | Value |
|-------|-------|
| **Title** | [Short descriptive title] |
| **Epic** | [Parent epic name/link] |
| **Story Points** | [Estimate] |
| **Priority** | Must Have / Should Have / Nice to Have |
| **Assignee** | [Name] |
| **Sprint** | [Sprint number] |

---

## User Story

**As a** [type of user/role],

**I want** [capability or feature],

**So that** [benefit or value I receive].

---

## Acceptance Criteria

### Scenario 1: [Happy Path]
- **Given** [precondition or context]
- **When** [action the user takes]
- **Then** [expected outcome]

### Scenario 2: [Alternative Path]
- **Given** [precondition or context]
- **When** [action the user takes]
- **Then** [expected outcome]

### Scenario 3: [Edge Case / Error]
- **Given** [precondition or context]
- **When** [action the user takes]
- **Then** [expected error handling or outcome]

---

## Notes

- [Technical notes, design references, API details]
- [Dependencies on other stories or external systems]
- [Out of scope items for this story]
- [Links to mockups, prototypes, or design files]

---

## Definition of Done

- [ ] Acceptance criteria met
- [ ] Code reviewed and approved
- [ ] Tests written and passing
- [ ] Documentation updated
- [ ] Product Owner accepted

FILE:references/sprint-planning-guide.md
# Sprint Planning Guide

Sprint planning workflows, capacity calculation, and backlog management.

---

## Table of Contents

- [Sprint Planning Workflow](#sprint-planning-workflow)
- [Capacity Planning](#capacity-planning)
- [Backlog Prioritization](#backlog-prioritization)
- [Sprint Ceremonies](#sprint-ceremonies)
- [Metrics and Tracking](#metrics-and-tracking)

---

## Sprint Planning Workflow

### Pre-Planning (1-2 Days Before)

1. Review and refine backlog items for upcoming sprint
2. Ensure top items have acceptance criteria
3. Validate story point estimates with team
4. Identify dependencies between stories
5. Confirm team availability for sprint
6. **Validation:** Top 1.5x capacity of stories are refined and estimated

### Sprint Planning Meeting

**Duration:** 2 hours for 2-week sprint

**Agenda:**

| Time | Activity | Participants |
|------|----------|--------------|
| 0:00-0:15 | Review sprint goal and priorities | PO presents |
| 0:15-0:45 | Discuss top backlog items | Team asks questions |
| 0:45-1:15 | Team selects stories for sprint | Team decides |
| 1:15-1:45 | Break down stories into tasks | Team collaborates |
| 1:45-2:00 | Confirm commitment and identify risks | All |

### Planning Checklist

**Before Planning:**
- [ ] Backlog groomed with top items refined
- [ ] Previous sprint retrospective actions reviewed
- [ ] Team capacity calculated
- [ ] Dependencies identified
- [ ] Sprint goal drafted

**During Planning:**
- [ ] Sprint goal agreed
- [ ] Stories selected fit within capacity
- [ ] Acceptance criteria reviewed for each story
- [ ] Tasks identified for complex stories
- [ ] Risks and blockers discussed

**After Planning:**
- [ ] Sprint backlog visible to all
- [ ] Sprint goal communicated
- [ ] Calendar blocked for ceremonies
- [ ] Dependencies communicated to other teams

---

## Capacity Planning

### Team Capacity Calculation

```
Sprint Capacity = (Team Members × Sprint Days × Hours/Day × Focus Factor)
                  ÷ Hours per Story Point

Simplified Version:
Sprint Capacity = Average Velocity × Availability Factor
```

### Availability Factors

| Scenario | Factor | Example |
|----------|--------|---------|
| Full sprint, no PTO | 1.0 | 30 points if velocity = 30 |
| 1 team member out 50% | 0.9 | 27 points |
| Holiday during sprint | 0.8 | 24 points |
| Multiple team members out | 0.7 | 21 points |
| Major release/on-call | 0.75 | 22-23 points |

### Capacity Buffer Rules

| Commitment Level | % of Velocity | Purpose |
|------------------|---------------|---------|
| Committed | 80-85% | High confidence delivery |
| Stretch | 10-15% | Optional if things go well |
| Buffer | 5-10% | Unplanned work, bugs |

### Sprint Loading Example

```
Team Velocity: 30 points/sprint
Availability: 90% (one team member partially out)
Adjusted Velocity: 27 points

Sprint Loading:
- Committed work: 23 points (85% of 27)
- Stretch goals: 4 points (15% of 27)
- Buffer: Remaining capacity for bugs/support

Story Selection:
[H] US-001: User dashboard (5 pts) ← Committed
[H] US-002: Export feature (3 pts) ← Committed
[H] US-003: Search filter (5 pts) ← Committed
[M] US-004: Settings page (5 pts) ← Committed
[M] US-005: Help tooltips (3 pts) ← Committed
[L] US-006: Theme options (2 pts) ← Committed
------------------------
Committed Total: 23 points

[L] US-007: Sort options (2 pts) ← Stretch
[L] US-008: Print view (2 pts) ← Stretch
------------------------
Stretch Total: 4 points
```

---

## Backlog Prioritization

### Priority Framework

| Priority | Definition | SLA |
|----------|------------|-----|
| Critical | Blocking users, security, data loss | Immediate |
| High | Core functionality, key user needs | This sprint |
| Medium | Improvements, enhancements | Next 2-3 sprints |
| Low | Nice-to-have, minor improvements | Backlog |

### Prioritization Factors

| Factor | Weight | Questions |
|--------|--------|-----------|
| Business Value | 40% | Revenue impact? User demand? Strategic? |
| User Impact | 30% | How many users? How often used? |
| Risk/Dependencies | 15% | Technical risk? External dependencies? |
| Effort | 15% | Size? Complexity? Uncertainty? |

### WSJF (Weighted Shortest Job First)

For larger items, use SAFe's WSJF:

```
WSJF = Cost of Delay / Job Duration

Cost of Delay = User Value + Time Criticality + Risk Reduction

Scale: 1, 2, 3, 5, 8, 13, 20

Example:
Feature A: CoD = 13, Duration = 5 → WSJF = 2.6
Feature B: CoD = 8, Duration = 2 → WSJF = 4.0 ← Higher priority
```

### Backlog Organization

| Section | Content | Review Frequency |
|---------|---------|------------------|
| Sprint Backlog | Committed for current sprint | Daily |
| Ready | Refined, estimated, prioritized | Each planning |
| Grooming | Needs refinement | Weekly |
| Icebox | Future consideration | Monthly |
| Archive | Completed or obsolete | Quarterly |

---

## Sprint Ceremonies

### Daily Standup

**Duration:** 15 minutes max
**Format:** Each team member answers:

1. What did I complete yesterday?
2. What will I work on today?
3. What blockers do I have?

**Product Owner Role:**
- Listen for blockers needing PO action
- Answer clarifying questions
- Note scope concerns for offline discussion
- Update stakeholders on progress

### Backlog Refinement (Grooming)

**Duration:** 1-2 hours per week
**Timing:** Mid-sprint

**Agenda:**

| Time | Activity |
|------|----------|
| 0:00-0:15 | Review upcoming priorities |
| 0:15-0:45 | Detail acceptance criteria for top items |
| 0:45-1:15 | Estimate new stories |
| 1:15-1:30 | Split large stories |

**Readiness Criteria:**
- [ ] Clear user story format (As a... I want... So that...)
- [ ] Acceptance criteria defined (Given-When-Then)
- [ ] Story point estimate agreed
- [ ] Dependencies identified
- [ ] Fits in one sprint (≤8 points)

### Sprint Review (Demo)

**Duration:** 1 hour for 2-week sprint

**Agenda:**

| Time | Activity | Lead |
|------|----------|------|
| 0:00-0:05 | Sprint goal recap | PO |
| 0:05-0:40 | Demo completed work | Team |
| 0:40-0:50 | Stakeholder feedback | Stakeholders |
| 0:50-1:00 | Roadmap update | PO |

**Demo Checklist:**
- [ ] Only demo completed (done-done) stories
- [ ] Use production or production-like environment
- [ ] Show user perspective, not technical details
- [ ] Collect feedback for backlog items
- [ ] Thank team for accomplishments

### Sprint Retrospective

**Duration:** 1.5 hours for 2-week sprint

**Format Options:**

| Format | Structure |
|--------|-----------|
| Start-Stop-Continue | What to begin, end, keep doing |
| 4Ls | Liked, Learned, Lacked, Longed for |
| Sailboat | Wind (helpers), Anchors (blockers), Rocks (risks) |
| Mad-Sad-Glad | Emotional state about sprint events |

**Action Items:**
- Maximum 2-3 improvement actions per retro
- Assign owner and due date
- Review previous actions at start of next retro

---

## Metrics and Tracking

### Sprint Metrics

| Metric | Formula | Target |
|--------|---------|--------|
| Velocity | Points completed / sprint | Stable ±10% |
| Commitment Reliability | Completed / Committed | >85% |
| Scope Change | Points added or removed | <10% |
| Carryover | Points not completed | <15% |
| Bug Ratio | Bug points / Total points | <20% |

### Velocity Tracking

```
Sprint Velocity Trend:
Sprint 1: 25 points
Sprint 2: 28 points
Sprint 3: 30 points
Sprint 4: 32 points
Sprint 5: 29 points
------------------------
Average: 28.8 points
Trend: Stable (±10%)

Planning Recommendation: Plan for 26-29 points committed
```

### Burndown Chart

Track progress within sprint:

```
Day   Ideal    Actual   Status
---   -----    ------   ------
 0     30       30      On track
 2     24       26      Slightly behind
 4     18       20      Behind
 6     12       14      Recovering
 8      6        6      On track
10      0        2      Minor carryover
```

**Burndown Patterns:**

| Pattern | Meaning | Action |
|---------|---------|--------|
| Flat start | No progress early | Check blockers |
| Late drop | Last-minute completion | Improve WIP limits |
| Scope increase | Line moves up | Address scope creep |
| Early completion | Done before sprint end | Pull stretch items |

### Definition of Done

Story is complete when:

- [ ] Code complete and reviewed
- [ ] Unit tests written and passing
- [ ] Integration tests passing
- [ ] Acceptance criteria verified
- [ ] Documentation updated
- [ ] Deployed to staging
- [ ] PO accepted
- [ ] No critical bugs

### Release Metrics

| Metric | Definition | Target |
|--------|------------|--------|
| Lead Time | Idea to production | <2 sprints |
| Cycle Time | Development start to done | <1 sprint |
| Throughput | Stories completed/sprint | Increasing |
| Defect Escape | Bugs found in production | Decreasing |

FILE:references/user-story-templates.md
# User Story Templates

Standard templates, acceptance criteria patterns, and INVEST validation for user stories.

---

## Table of Contents

- [Story Templates](#story-templates)
- [Acceptance Criteria Patterns](#acceptance-criteria-patterns)
- [INVEST Criteria](#invest-criteria)
- [Story Point Estimation](#story-point-estimation)
- [Common Antipatterns](#common-antipatterns)

---

## Story Templates

### Standard User Story Format

```
As a [persona],
I want to [action/capability],
So that [benefit/value].
```

### Template by Story Type

**Feature Story:**
```
As a [persona],
I want to [perform action]
So that [I achieve benefit].

Example:
As a marketing manager,
I want to export campaign reports to PDF
So that I can share results with stakeholders who don't have system access.
```

**Improvement Story:**
```
As a [persona],
I need [capability/improvement]
To [achieve goal more effectively].

Example:
As a sales rep,
I need faster search results
To find customer records without interrupting calls.
```

**Bug Fix Story:**
```
As a [persona],
I expect [correct behavior]
When [specific condition].

Example:
As a user,
I expect my session to remain active
When navigating between dashboard tabs.
```

**Integration Story:**
```
As a [persona],
I want to [integrate/connect with system]
So that [workflow improvement].

Example:
As an admin,
I want to sync user data with our LDAP server
So that employees are automatically provisioned.
```

**Enabler Story (Technical):**
```
As a developer,
I need to [technical requirement]
To enable [user-facing capability].

Example:
As a developer,
I need to implement caching layer
To enable sub-second dashboard load times.
```

### Persona Library

| Persona | Typical Needs | Context |
|---------|--------------|---------|
| End User | Efficiency, simplicity, reliability | Daily core feature usage |
| Administrator | Control, visibility, security | System management |
| Power User | Automation, customization, shortcuts | Expert workflows |
| New User | Guidance, learning, safety | Onboarding experience |
| Manager | Reporting, oversight, delegation | Team coordination |
| External User | Access, security, documentation | Customer/partner usage |

---

## Acceptance Criteria Patterns

### Given-When-Then (Gherkin)

Preferred format for testable acceptance criteria:

```
Given [precondition/context],
When [action/trigger],
Then [expected outcome].
```

**Examples:**

```
Given the user is logged in with valid credentials,
When they click the "Export" button,
Then a PDF download starts within 2 seconds.

Given the user has entered invalid email format,
When they submit the registration form,
Then an inline error message displays "Please enter a valid email address."

Given the daily sync job has not run in 24 hours,
When the scheduler triggers at midnight,
Then all pending records are synchronized and logged.
```

### Should/Must/Can Patterns

**Should (Expected Behavior):**
```
Should [behavior] when [condition].

Example:
Should display loading spinner when API call exceeds 500ms.
```

**Must (Hard Requirement):**
```
Must [requirement] to [achieve outcome].

Example:
Must encrypt all data at rest to meet compliance requirements.
```

**Can (Capability):**
```
Can [capability] without [negative outcome].

Example:
Can undo last action without losing other changes.
```

### Acceptance Criteria Checklist

Each story should have acceptance criteria covering:

| Category | Example Criterion |
|----------|-------------------|
| Happy Path | Given valid input, When submitted, Then success message displayed |
| Validation | Should reject input when required field is empty |
| Error Handling | Must show user-friendly message when API fails |
| Performance | Should complete operation within 2 seconds |
| Accessibility | Must be navigable via keyboard only |
| Security | Should not expose sensitive data in URL parameters |

### Minimum Acceptance Criteria Count

| Story Size (Points) | Minimum AC Count |
|--------------------|------------------|
| 1-2 | 3-4 |
| 3-5 | 4-6 |
| 8 | 5-8 |
| 13+ | Split the story |

---

## INVEST Criteria

### INVEST Validation Checklist

| Criterion | Question | Pass If... |
|-----------|----------|------------|
| **I**ndependent | Can this story be developed without depending on another story? | No blocking dependencies on uncommitted work |
| **N**egotiable | Is the implementation approach flexible? | Multiple ways to deliver the value |
| **V**aluable | Does this deliver value to users or business? | Clear benefit statement in "so that" |
| **E**stimable | Can the team estimate this story? | Understood well enough to size |
| **S**mall | Can this be completed in one sprint? | ≤8 story points typically |
| **T**estable | Can we verify this story is done? | Clear, measurable acceptance criteria |

### INVEST Failure Patterns

| Criterion | Red Flag | Fix |
|-----------|----------|-----|
| Independent | "After story X is done..." | Combine stories or resequence |
| Negotiable | Specific implementation in story | Focus on outcome, not solution |
| Valuable | No "so that" clause | Add benefit statement |
| Estimable | Team says "no idea" | Spike first, then story |
| Small | >8 points | Split into smaller stories |
| Testable | "System should be better" | Add measurable criteria |

### Story Splitting Techniques

When stories are too large (>8 points), split using:

| Technique | Example |
|-----------|---------|
| By workflow step | "Create order" → "Add items" + "Apply discount" + "Submit order" |
| By persona | "User dashboard" → "Admin dashboard" + "Member dashboard" |
| By data type | "Import data" → "Import CSV" + "Import Excel" |
| By operation | "Manage users" → "Add user" + "Edit user" + "Delete user" |
| By platform | "Mobile support" → "iOS support" + "Android support" |
| Happy path first | "Full feature" → "Basic feature" + "Error handling" + "Edge cases" |

---

## Story Point Estimation

### Fibonacci Scale Reference

| Points | Complexity | Example |
|--------|------------|---------|
| 1 | Trivial | Fix typo, change label |
| 2 | Simple | Add field, simple validation |
| 3 | Small | New form, basic CRUD operation |
| 5 | Medium | Feature with multiple components |
| 8 | Large | Complex feature, multiple integrations |
| 13 | Very Large | Consider splitting |
| 21+ | Epic | Must split |

### Estimation Factors

| Factor | Low Complexity | High Complexity |
|--------|---------------|-----------------|
| Unknowns | Well understood | Many unknowns |
| Dependencies | None | Multiple systems |
| Testing | Simple unit tests | Complex integration tests |
| Data | Simple structure | Complex transformations |
| UI | Minor changes | New components |

### Velocity Calculation

```
Velocity = Total points completed / Number of sprints

Example:
Sprint 1: 28 points
Sprint 2: 32 points
Sprint 3: 30 points
Average Velocity: (28 + 32 + 30) / 3 = 30 points/sprint

Sprint Capacity Planning:
- Committed: 80-90% of velocity (24-27 points)
- Stretch goals: 10-20% additional (3-6 points)
```

---

## Common Antipatterns

### Story Antipatterns

| Antipattern | Example | Fix |
|-------------|---------|-----|
| Solution story | "Implement React component" | "Display user profile information" |
| Compound story | "Create, edit, and delete users" | Split into three stories |
| Missing persona | "The system will..." | "As an admin, I want to..." |
| No benefit | "I want to see a button" | Add "so that [benefit]" |
| Too vague | "Improve performance" | "Reduce page load to <2 seconds" |
| Technical jargon | "Implement Redis caching" | "Enable instant search results" |

### Acceptance Criteria Antipatterns

| Antipattern | Example | Fix |
|-------------|---------|-----|
| Too vague | "Works correctly" | Specific Given-When-Then |
| Implementation details | "Use PostgreSQL query" | Focus on outcome |
| Missing unhappy path | Only success scenario | Add error cases |
| Untestable | "User is happy" | Measurable behavior |
| Too many | 15+ criteria | Split the story |

### Sprint Planning Antipatterns

| Antipattern | Impact | Fix |
|-------------|--------|-----|
| 100% capacity | No buffer for unknowns | Plan 80-85% |
| All large stories | Risk of incomplete sprint | Mix sizes |
| No dependencies mapped | Blocked work | Identify dependencies upfront |
| Stretch = overflow | Hiding overcommitment | Stretch should be optional |

FILE:scripts/user_story_generator.py
#!/usr/bin/env python3
"""
User Story Generator with INVEST Criteria
Creates well-formed user stories with acceptance criteria
"""

import json
from typing import Dict, List, Tuple

class UserStoryGenerator:
    """Generate INVEST-compliant user stories"""
    
    def __init__(self):
        self.personas = {
            'end_user': {
                'name': 'End User',
                'needs': ['efficiency', 'simplicity', 'reliability', 'speed'],
                'context': 'daily usage of core features'
            },
            'admin': {
                'name': 'Administrator',
                'needs': ['control', 'visibility', 'security', 'configuration'],
                'context': 'system management and oversight'
            },
            'power_user': {
                'name': 'Power User',
                'needs': ['advanced features', 'automation', 'customization', 'shortcuts'],
                'context': 'expert usage and workflow optimization'
            },
            'new_user': {
                'name': 'New User',
                'needs': ['guidance', 'learning', 'safety', 'clarity'],
                'context': 'first-time experience and onboarding'
            }
        }
        
        self.story_templates = {
            'feature': "As a {persona}, I want to {action} so that {benefit}",
            'improvement': "As a {persona}, I need {capability} to {achieve_goal}",
            'fix': "As a {persona}, I expect {behavior} when {condition}",
            'integration': "As a {persona}, I want to {integrate} so that {workflow}"
        }
        
        self.acceptance_criteria_patterns = [
            "Given {precondition}, When {action}, Then {outcome}",
            "Should {behavior} when {condition}",
            "Must {requirement} to {achieve}",
            "Can {capability} without {negative_outcome}"
        ]
    
    def generate_epic_stories(self, epic: Dict) -> List[Dict]:
        """Break down epic into user stories"""
        stories = []
        
        # Analyze epic for key components
        epic_name = epic.get('name', 'Feature')
        epic_description = epic.get('description', '')
        personas = epic.get('personas', ['end_user'])
        scope = epic.get('scope', [])
        
        # Generate stories for each persona and scope item
        for persona in personas:
            for i, scope_item in enumerate(scope):
                story = self.generate_story(
                    persona=persona,
                    feature=scope_item,
                    epic=epic_name,
                    index=i+1
                )
                stories.append(story)
        
        # Add enabler stories (technical, infrastructure)
        if epic.get('technical_requirements'):
            for req in epic['technical_requirements']:
                enabler = self.generate_enabler_story(req, epic_name)
                stories.append(enabler)
        
        return stories
    
    def generate_story(self, persona: str, feature: str, epic: str, index: int) -> Dict:
        """Generate a single user story"""
        
        persona_data = self.personas.get(persona, self.personas['end_user'])
        
        # Create story
        story = {
            'id': f"{epic[:3].upper()}-{index:03d}",
            'type': 'story',
            'title': self._generate_title(feature),
            'narrative': self._generate_narrative(persona_data, feature),
            'acceptance_criteria': self._generate_acceptance_criteria(feature),
            'estimation': self._estimate_complexity(feature),
            'priority': self._determine_priority(persona, feature),
            'dependencies': [],
            'invest_check': self._check_invest_criteria(feature)
        }
        
        return story
    
    def generate_enabler_story(self, requirement: str, epic: str) -> Dict:
        """Generate technical enabler story"""
        
        return {
            'id': f"{epic[:3].upper()}-E{len(requirement):02d}",
            'type': 'enabler',
            'title': f"Technical: {requirement}",
            'narrative': f"As a developer, I need to {requirement} to enable user features",
            'acceptance_criteria': [
                f"Technical requirement {requirement} is implemented",
                "All tests pass",
                "Documentation is updated",
                "No regression in existing functionality"
            ],
            'estimation': 5,  # Default medium complexity
            'priority': 'high',
            'dependencies': [],
            'invest_check': {
                'independent': True,
                'negotiable': False,  # Technical requirements often non-negotiable
                'valuable': True,
                'estimable': True,
                'small': True,
                'testable': True
            }
        }
    
    def _generate_title(self, feature: str) -> str:
        """Generate concise story title"""
        # Simplify feature description to title
        words = feature.split()[:5]
        return ' '.join(words).title()
    
    def _generate_narrative(self, persona: Dict, feature: str) -> str:
        """Generate story narrative in standard format"""
        
        template = self.story_templates['feature']
        
        action = self._extract_action(feature)
        benefit = self._extract_benefit(feature, persona['needs'])
        
        return template.format(
            persona=persona['name'],
            action=action,
            benefit=benefit
        )
    
    def _generate_acceptance_criteria(self, feature: str) -> List[str]:
        """Generate acceptance criteria"""
        
        criteria = []
        
        # Happy path
        criteria.append(f"Given user has access, When they {self._extract_action(feature)}, Then {self._extract_outcome(feature)}")
        
        # Validation
        criteria.append(f"Should validate input before processing")
        
        # Error handling
        criteria.append(f"Must show clear error message when action fails")
        
        # Performance
        criteria.append(f"Should complete within 2 seconds")
        
        # Accessibility
        criteria.append(f"Must be accessible via keyboard navigation")
        
        return criteria
    
    def _extract_action(self, feature: str) -> str:
        """Extract action from feature description"""
        action_verbs = ['create', 'view', 'edit', 'delete', 'share', 'export', 'import', 'configure', 'search', 'filter']
        
        feature_lower = feature.lower()
        for verb in action_verbs:
            if verb in feature_lower:
                return feature_lower
        
        return f"use {feature.lower()}"
    
    def _extract_benefit(self, feature: str, needs: List[str]) -> str:
        """Extract benefit based on feature and persona needs"""
        
        feature_lower = feature.lower()
        
        if 'save' in feature_lower or 'quick' in feature_lower:
            return "I can save time and work more efficiently"
        elif 'share' in feature_lower or 'collab' in feature_lower:
            return "I can collaborate with my team effectively"
        elif 'report' in feature_lower or 'analyt' in feature_lower:
            return "I can make data-driven decisions"
        elif 'automat' in feature_lower:
            return "I can reduce manual work and errors"
        else:
            return f"I can achieve my goals related to {needs[0]}"
    
    def _extract_outcome(self, feature: str) -> str:
        """Extract expected outcome"""
        return f"the {feature.lower()} is successfully completed"
    
    def _estimate_complexity(self, feature: str) -> int:
        """Estimate story points based on complexity indicators"""
        
        feature_lower = feature.lower()
        
        # Complexity indicators
        complexity = 3  # Base complexity
        
        if any(word in feature_lower for word in ['simple', 'basic', 'view', 'display']):
            complexity = 1
        elif any(word in feature_lower for word in ['create', 'edit', 'update']):
            complexity = 3
        elif any(word in feature_lower for word in ['complex', 'advanced', 'integrate', 'migrate']):
            complexity = 8
        elif any(word in feature_lower for word in ['redesign', 'refactor', 'architect']):
            complexity = 13
        
        return complexity
    
    def _determine_priority(self, persona: str, feature: str) -> str:
        """Determine story priority"""
        
        feature_lower = feature.lower()
        
        # Critical features
        if any(word in feature_lower for word in ['security', 'fix', 'critical', 'broken']):
            return 'critical'
        
        # High priority for primary personas
        if persona in ['end_user', 'admin']:
            if any(word in feature_lower for word in ['core', 'essential', 'primary']):
                return 'high'
        
        # Medium for improvements
        if any(word in feature_lower for word in ['improve', 'enhance', 'optimize']):
            return 'medium'
        
        # Low for nice-to-haves
        return 'low'
    
    def _check_invest_criteria(self, feature: str) -> Dict[str, bool]:
        """Check INVEST criteria compliance"""
        
        return {
            'independent': not any(word in feature.lower() for word in ['after', 'depends', 'requires']),
            'negotiable': True,  # Most features can be negotiated
            'valuable': True,  # Assume value if it made it to backlog
            'estimable': len(feature.split()) < 20,  # Can estimate if not too vague
            'small': self._estimate_complexity(feature) <= 8,  # 8 points or less
            'testable': not any(word in feature.lower() for word in ['maybe', 'possibly', 'somehow'])
        }
    
    def generate_sprint_stories(self, capacity: int, backlog: List[Dict]) -> Dict:
        """Generate stories for a sprint based on capacity"""
        
        sprint = {
            'capacity': capacity,
            'committed': [],
            'stretch': [],
            'total_points': 0,
            'utilization': 0
        }
        
        # Sort backlog by priority and size
        sorted_backlog = sorted(
            backlog,
            key=lambda x: (
                {'critical': 0, 'high': 1, 'medium': 2, 'low': 3}[x['priority']],
                x['estimation']
            )
        )
        
        # Fill sprint
        for story in sorted_backlog:
            if sprint['total_points'] + story['estimation'] <= capacity:
                sprint['committed'].append(story)
                sprint['total_points'] += story['estimation']
            elif sprint['total_points'] + story['estimation'] <= capacity * 1.2:
                sprint['stretch'].append(story)
        
        sprint['utilization'] = round((sprint['total_points'] / capacity) * 100, 1)
        
        return sprint
    
    def format_story_output(self, story: Dict) -> str:
        """Format story for display"""
        
        output = []
        output.append(f"USER STORY: {story['id']}")
        output.append("=" * 40)
        output.append(f"Title: {story['title']}")
        output.append(f"Type: {story['type']}")
        output.append(f"Priority: {story['priority'].upper()}")
        output.append(f"Points: {story['estimation']}")
        output.append("")
        output.append("Story:")
        output.append(story['narrative'])
        output.append("")
        output.append("Acceptance Criteria:")
        for i, criterion in enumerate(story['acceptance_criteria'], 1):
            output.append(f"  {i}. {criterion}")
        output.append("")
        output.append("INVEST Checklist:")
        for criterion, passed in story['invest_check'].items():
            status = "✓" if passed else "✗"
            output.append(f"  {status} {criterion.capitalize()}")
        
        return "\n".join(output)

def create_sample_epic():
    """Create a sample epic for testing"""
    return {
        'name': 'User Dashboard',
        'description': 'Create a comprehensive dashboard for users to view their data',
        'personas': ['end_user', 'power_user'],
        'scope': [
            'View key metrics and KPIs',
            'Customize dashboard layout',
            'Export dashboard data',
            'Share dashboard with team members',
            'Set up automated reports'
        ],
        'technical_requirements': [
            'Implement caching for performance',
            'Set up real-time data pipeline'
        ]
    }

def main():
    import sys
    
    generator = UserStoryGenerator()
    
    if len(sys.argv) > 1 and sys.argv[1] == 'sprint':
        # Generate sprint planning
        capacity = int(sys.argv[2]) if len(sys.argv) > 2 else 30
        
        # Create sample backlog
        epic = create_sample_epic()
        backlog = generator.generate_epic_stories(epic)
        
        # Plan sprint
        sprint = generator.generate_sprint_stories(capacity, backlog)
        
        print("=" * 60)
        print("SPRINT PLANNING")
        print("=" * 60)
        print(f"Sprint Capacity: {sprint['capacity']} points")
        print(f"Committed: {sprint['total_points']} points ({sprint['utilization']}%)")
        print(f"Stories: {len(sprint['committed'])} committed + {len(sprint['stretch'])} stretch")
        print("\n📋 COMMITTED STORIES:\n")
        
        for story in sprint['committed']:
            print(f"  [{story['priority'][:1].upper()}] {story['id']}: {story['title']} ({story['estimation']}pts)")
        
        if sprint['stretch']:
            print("\n🎯 STRETCH GOALS:\n")
            for story in sprint['stretch']:
                print(f"  [{story['priority'][:1].upper()}] {story['id']}: {story['title']} ({story['estimation']}pts)")
    
    else:
        # Generate stories for epic
        epic = create_sample_epic()
        stories = generator.generate_epic_stories(epic)
        
        print(f"Generated {len(stories)} stories from epic: {epic['name']}\n")
        
        # Display first 3 stories in detail
        for story in stories[:3]:
            print(generator.format_story_output(story))
            print("\n")
        
        # Summary of all stories
        print("=" * 60)
        print("BACKLOG SUMMARY")
        print("=" * 60)
        total_points = sum(s['estimation'] for s in stories)
        print(f"Total Stories: {len(stories)}")
        print(f"Total Points: {total_points}")
        print(f"Average Size: {total_points/len(stories):.1f} points")
        print("\nPriority Breakdown:")
        for priority in ['critical', 'high', 'medium', 'low']:
            count = len([s for s in stories if s['priority'] == priority])
            if count > 0:
                print(f"  {priority.capitalize()}: {count} stories")

if __name__ == "__main__":
    main()

---
name: sms
description: When the user wants to plan, build, or optimize SMS or MMS marketing — including welcome flows, abandoned cart texts, post-purchase, win-back, promotional sends, or transactional/auth SMS. Also use when the user mentions "SMS marketing," "text message campaigns," "SMS sequence," "SMS automation," "abandoned cart text," "post-purchase SMS," "Klaviyo SMS," "Postscript," "Attentive," "Twilio," "A2P 10DLC," "TCPA," "SMS compliance," "short code," "toll-free SMS," "MMS campaign," "should I do SMS," or "SMS vs email." For email sequences, see emails. For SMS copy framing, see copywriting. For opt-in popups that capture phone numbers, see popups.
metadata:
  version: 1.0.0
---

# SMS Marketing

You are an expert in SMS and MMS marketing for direct-to-consumer brands, mobile apps, and SaaS products with high-engagement use cases. Your goal is to help plan, build, and optimize SMS programs that drive measurable revenue or activation while staying fully compliant with TCPA and carrier rules.

## Before Starting

**Check for product marketing context first:**
If `.agents/product-marketing.md` exists (or `.claude/product-marketing.md`, or the legacy `product-marketing-context.md` filename, in older setups), read it before asking questions. Use that context and only ask for information not already covered or specific to this task.

Gather this context (ask if not provided):

### 1. Business Type
- B2C ecom / DTC, B2B SaaS, mobile app, services, fintech
- Order volume or list size (SMS economics depend on scale)
- Geographic mix (US, EU, both — compliance differs dramatically)

### 2. Current State
- Existing SMS program (platform, list size, opt-in rate, opt-out rate, revenue/send)
- Email program (SMS works best as a layer on top, not a replacement)
- Phone number type: short code, toll-free, long code (10DLC)

### 3. Compliance Posture
- US: A2P 10DLC registration complete? (Required since 2022 — without it, your messages get filtered)
- Opt-in mechanism in use? (Checkbox, keyword opt-in, double opt-in)
- Privacy policy + terms include SMS disclosures?

### 4. Goal
- Drive revenue (promotional, cart recovery, post-purchase)
- Drive activation (welcome, onboarding, milestone nudges)
- Transactional (order updates, auth codes, alerts)

---

## When SMS Beats Email

SMS is not "another email." Use it where the channel's properties win:

| Use Case | SMS or Email? | Why |
|----------|---------------|-----|
| Abandoned cart recovery | **SMS first** | 98% open rate within 3 min vs 20% for email in 24h |
| Order/shipping updates | **SMS** | Customers want it now, on their phone |
| Flash sale / limited drop | **SMS** | Urgency channel; immediate read |
| Auth codes / 2FA | **SMS** (or app) | Latency-sensitive, must arrive in seconds |
| Welcome series | **Email primary, SMS layer** | Email carries the long-form content |
| Educational nurture | **Email** | Too much text for SMS, costs add up |
| Newsletter | **Email** | Wrong channel for SMS |
| Win-back lapsed customers | **Both** | SMS for the strong nudge, email for the offer detail |
| Post-purchase upsell | **SMS** | High open rate, ride the purchase momentum |

**General rule**: SMS earns the right to interrupt because of opt-in. Use it for messages that genuinely benefit from immediacy. If it could wait 24 hours, send it via email.

---

## Compliance — Read First

**Compliance is the foundation, not an afterthought.** A single TCPA class-action settlement runs $5M–$40M. The basics:

### US — TCPA (Telephone Consumer Protection Act)

1. **Express written consent** required for marketing SMS. Implied consent doesn't count.
2. **Clear disclosure at opt-in** must include: program name, frequency expectation ("up to 4 msgs/month"), STOP/HELP instructions, "Msg & data rates may apply," link to terms.
3. **Honor STOP/UNSUBSCRIBE within seconds**, every time, no exceptions, on every keyword variant (STOP, END, CANCEL, UNSUBSCRIBE, QUIT).
4. **Honor HELP** with a response containing brand name + STOP info + support contact.
5. **Quiet hours**: no marketing sends before 8am or after 9pm in the recipient's local time. Carrier rules and state laws (e.g., Florida, Oklahoma, Washington) are stricter than federal — default to 9am–8pm recipient-local.
6. **Keep written consent records** with timestamp, opt-in source, and exact disclosure text shown. Auditable.

### US — A2P 10DLC Registration (required since 2022)

Application-to-Person 10-digit long codes must be registered through The Campaign Registry (TCR) via your SMS platform. Without registration:
- Throughput is throttled (or zero)
- Carriers filter your messages
- You'll see "delivered" status but recipients won't get them

**Registration covers**: brand identity verification, campaign use case (marketing, account notification, OTP, etc.), sample messages, opt-in mechanism, opt-out language. Sample message text from registration must match what you actually send.

### EU/UK — GDPR-derived consent

- Explicit opt-in required (no pre-checked boxes)
- Right to withdraw consent must be as easy as giving it
- Data subject access requests apply to SMS records
- ePrivacy Directive layered on top of GDPR

### Canada — CASL

- Express consent + sender identification + unsubscribe in every message
- Implied consent allowed for existing business relationships within 24 months
- Penalties up to CAD $10M per violation

**For full compliance details, edge cases, opt-in copy templates, and STOP/HELP response templates**: see [references/compliance.md](references/compliance.md).

---

## Phone Number Types (US)

| Type | Throughput | Cost | Use Case | Trust |
|------|-----------|------|----------|-------|
| **Short code (5-6 digit)** | 100+ msg/sec | $500–$1,000/mo + setup | High-volume marketing | Highest (carrier-vetted) |
| **Toll-free (1-8XX)** | ~3 msg/sec | $10–$30/mo | Mid-volume, B2C support | Medium-high (carrier-verified) |
| **10DLC (regular long code)** | 1–250 msg/sec | $2–$10/mo | SMB, conversational, transactional | Medium (requires A2P 10DLC reg) |

**Rule of thumb**: list <10K = 10DLC. List 10K–100K = toll-free. List 100K+ = short code.

---

## Core Principles

### 1. Every send has a real cost
SMS isn't free. At $0.0075–$0.04 per send + carrier fees, a 100K send costs $750–$4,000. This forces relevance — you can't "blast." Segment hard.

### 2. Opt-in is your most valuable asset
Opt-in rate from email → SMS is typically 5–25%. A high-quality SMS list of 10K beats a low-quality list of 100K. Optimize opt-in quality, not volume.

### 3. Each message must justify itself
The recipient gave you their phone number. Every send should pass: "would I be glad I got this text?" If no, don't send.

### 4. Brevity + clarity
160 GSM-7 characters = 1 SMS segment. 161+ chars = 2 segments (you're billed for 2). Emojis force UCS-2 encoding (70 chars per segment). Plan for segment count.

### 5. One CTA, one link
Short links are mandatory (`klvy.co`, `txt.attn.tv`, branded short domain). Track UTM params on every link.

### 6. Sender identity, every send
"From [Brand]:" or branded short code at the start of every message. Even on automated flows. Recipients can't see "from" address — they need it inline.

---

## SMS Sequence Types

### Welcome / Opt-In Confirmation (immediate)

Send 1: Confirmation + reward (immediate)
> From Acme: Thanks for joining! Here's 10% off: ACME10. Use at checkout: acme.co/sale. Reply STOP to opt out.

Optional Send 2 (24h later): Reminder + best-seller showcase

### Abandoned Cart (highest-ROI flow for ecom)

- Send 1 (30 min after abandon): "Forget something? Your cart's still here: [short link]"
- Send 2 (4 hours later): Soft urgency + social proof
- Send 3 (24 hours later, optional): Discount offer (only if margin allows)

**Note**: Discount on first message trains customers to abandon. Reserve discount for Send 2 or 3.

### Browse Abandonment

- Send 1 (1 hour after browse): Product + "Thinking it over?" + link

### Post-Purchase

- Send 1 (immediate): Order confirmation + delivery ETA (transactional, separate consent OK)
- Send 2 (after delivery + 2 days): "How are you liking [product]?" + review prompt + cross-sell

### Win-Back (lapsed)

- Send 1 (60–90 days after last purchase): "We miss you" + curated picks
- Send 2 (14 days later): Discount offer
- Send 3 (final, 14 days later): Opt-out warning + last chance

### Promotional / Campaign Sends

- Flash sales, drops, launches, BFCM
- 1–2 sends max per campaign
- Stack against email send schedule to avoid same-day double-tap

### Transactional (separate compliance bucket)

- Order updates, shipping, delivery, auth codes, account alerts
- Generally OK without separate marketing consent if directly related to a transaction the user initiated
- Still subject to A2P 10DLC registration in US

**For full sequence templates with copy and timing**: see [references/sequence-templates.md](references/sequence-templates.md).

---

## SMS Copy Guidelines

### Structure
1. **Sender ID** ("From Acme:" or brand short code) — required
2. **Hook** — first 5 words decide if they read on
3. **Value** — what's in it for them, specifically
4. **CTA + short link** — single action, single URL
5. **Compliance footer** — "Reply STOP to opt out" (required on opt-in confirmation and at least quarterly thereafter; carrier-recommended on every promotional message)

### Length

- **160 chars (GSM-7)** = 1 segment. Aim here.
- **70 chars (UCS-2)** if you use emojis, accented characters, or curly quotes — you'll pay for more segments.
- **161–306 chars** = 2 segments (concatenated SMS). Acceptable for richer messages, but you're paying double per send.
- **MMS** (image + up to 1,600 chars) = 3–5× the SMS cost. Use sparingly for high-impact moments.

### Voice

- Conversational, not corporate. SMS feels personal — write like you're texting a friend.
- No subject line, no formatting, no marketing-speak.
- Emojis are fine in moderation (one per message, situationally).
- ALL CAPS reads as shouting. Avoid except for explicit codes (e.g., "Use ACME10").

### Personalization

- First name token if available (boosts CTR ~20%)
- Recent product/category browse-based
- Location-based offers (where applicable)
- Don't fake intimacy ("Hey friend!") — it backfires

**For complete copy patterns by sequence type with character counts**: see [references/sequence-templates.md](references/sequence-templates.md).

---

## Platform Selection

| Platform | Best For | Native MCP | Cost Tier |
|----------|----------|:---:|-----------|
| **Klaviyo SMS** | DTC ecom already on Klaviyo email | ✓ | $$ |
| **Postscript** | DTC Shopify ecom, deep integration | - | $$ |
| **Attentive** | Mid-market+ ecom, full-service | - | $$$ |
| **Twilio** | Custom builds, transactional, devs | - | $ (raw API) |
| **Brevo SMS** | EU-focused, email + SMS combo | ✓ | $ |
| **SimpleTexting** | SMB, simple needs, ease of use | - | $ |
| **Customer.io** | Behavior-based automation + SMS | - | $$ |

**Quick picks**:
- Already on Klaviyo for email + DTC/ecom → **Klaviyo SMS** (no second platform to learn)
- Shopify ecom, want deeper SMS-specific features → **Postscript**
- Building custom SMS into a product → **Twilio**
- B2B SaaS doing transactional/auth → **Twilio** or **Customer.io**

**For platform deep-dives (features, pricing, integration paths, A2P registration)**: see [references/platforms.md](references/platforms.md).

---

## Measurement

### Key Metrics

| Metric | What it tells you | Healthy range (ecom DTC) |
|--------|-------------------|--------------------------|
| **Opt-in rate** | Top of funnel health | 5–25% of email subscribers |
| **CTR** | Message relevance | 8–15% (vs ~3% email) |
| **Conversion rate (per send)** | Revenue impact | 1–5% per promotional send |
| **Revenue per send (RPS)** | Channel economics | $0.20–$2.00 |
| **Opt-out rate per send** | Audience fatigue | <2% per send, <0.5% for promotional |
| **Cost per send** | Channel cost discipline | $0.0075–$0.04 |
| **List growth rate** | Audience momentum | 5–15%/month early, 1–3% steady-state |

### What to track in analytics

- UTM tag every link: `utm_source=sms&utm_medium=sms&utm_campaign=[campaign-name]`
- Conversion attribution: SMS-driven sessions, last-click revenue, assisted conversions
- LTV impact: SMS subscribers vs email-only subscribers (typically 1.5–3× LTV for SMS opt-ins)

### What to A/B test

- Send time (afternoon vs evening, local time)
- Copy length (short SMS vs MMS with image)
- Discount amount and trigger (immediate vs delayed)
- Personalization tokens (with first name vs without)
- CTA copy ("Shop now" vs "See it" vs "Last chance")

Cross-reference **ab-testing** skill for proper test design and **analytics** for attribution setup.

---

## Output Format

When the user asks for an SMS plan, return:

1. **Compliance check**: Are they registered for A2P 10DLC (if US)? Is the opt-in mechanism compliant? Flag blockers first.
2. **Strategy**: Which SMS flows to build first, ranked by ROI for their business model.
3. **Sequence designs**: For each priority flow, specify trigger, delay, copy with character counts, CTA, segmentation.
4. **Platform recommendation**: Based on stack, list size, and complexity.
5. **Measurement plan**: KPIs, benchmarks, A/B test queue.
6. **Compliance footer**: Required disclosures, STOP/HELP response templates.

Keep recommendations specific. Don't say "send an SMS at the right time" — say "send 30 min after cart abandon, 4 hours later if no purchase, 24 hours later with discount."

---

## Task-Specific Questions

1. Are you US, EU, or both? (Changes compliance approach entirely.)
2. Is A2P 10DLC registration complete (US)?
3. What platform are you on or considering?
4. Email list size and SMS opt-in rate (if any)?
5. What sequences do you already have running?
6. Are you DTC ecom, mobile app, B2B SaaS, services?
7. What's the primary goal: revenue, activation, retention, or transactional?

---

## Common Mistakes

1. **Skipping A2P 10DLC registration** — your messages get filtered into oblivion. Register first, send second.
2. **Treating SMS like email** — sending daily promotional blasts. Opt-out rates spike, list dies.
3. **Discount on first abandoned cart message** — trains customers to always abandon. Reserve for second or third send.
4. **Generic "From: [shortcode]"** — recipients need brand name in the message itself.
5. **Forgetting quiet hours** — sending at 6 AM local time gets opt-outs and TCPA complaints.
6. **No STOP/HELP handling** — non-negotiable. Every platform handles this; verify yours does.
7. **Emojis everywhere** — pushes you into UCS-2 encoding, halves segment size, doubles cost.
8. **Mismatching A2P sample messages and actual sends** — carriers flag and block.
9. **Not tracking conversions** — you can't justify channel ROI without attribution.
10. **No throttling on bulk sends** — burst sends trigger carrier filtering. Use platform throttling.

---

## Tool Integrations

For implementation, see the [tools registry](../../tools/REGISTRY.md). Key SMS tools:

| Tool | Best For | MCP | Guide |
|------|----------|:---:|-------|
| **Klaviyo** | E-commerce email + SMS combined | ✓ | [klaviyo.md](../../tools/integrations/klaviyo.md) |
| **Postscript** | Shopify DTC SMS, deepest Shopify integration | - | [postscript.md](../../tools/integrations/postscript.md) |
| **Attentive** | Mid-market+ DTC SMS, full-service | - | [attentive.md](../../tools/integrations/attentive.md) |
| **Twilio** | Raw API for custom builds, transactional, dev-first | - | [twilio.md](../../tools/integrations/twilio.md) |
| **Plivo** | Twilio alternative, lower per-send cost | - | [plivo.md](../../tools/integrations/plivo.md) |
| **AudienceTap** | AI-forward DTC, on-pack QR opt-in | - | [audiencetap.md](../../tools/integrations/audiencetap.md) |
| **Brevo** | EU email + SMS, SMB-friendly | ✓ | [brevo.md](../../tools/integrations/brevo.md) |
| **Customer.io** | Behavior-based SMS automation | - | [customer-io.md](../../tools/integrations/customer-io.md) |

---

## Related Skills

- **emails**: Sister channel — almost always run together. Email carries the long-form content; SMS carries the urgent nudges.
- **copywriting**: For SMS copy at scale and the longer-form pages/emails that SMS links to.
- **popups**: For phone number capture popups on-site.
- **churn-prevention**: For win-back flows that combine SMS + email.
- **onboarding**: For post-signup SMS milestone nudges.
- **analytics**: For attribution and RPS measurement.
- **ab-testing**: For SMS-specific test design.
- **lead-magnets**: For incentivizing opt-in (the "10% off for joining" offer).

FILE:evals/evals.json
{
  "skill_name": "sms",
  "evals": [
    {
      "id": 1,
      "prompt": "We're a Shopify DTC brand doing $5M/year in skincare. We have 80K email subscribers but no SMS program yet. Where do we start?",
      "expected_output": "Should check for product-marketing.md first. Should run Phase 0 compliance check: are they US-based, is A2P 10DLC registration started, is the opt-in mechanism planned. Should recommend Klaviyo SMS or Postscript given Shopify + DTC ecom (Klaviyo if already on Klaviyo email, Postscript for SMS-first depth). Should rank flows by ROI for skincare: (1) abandoned cart sequence first (highest-ROI flow), (2) post-purchase + replenishment (skincare has predictable cycles), (3) welcome opt-in flow with capture incentive, (4) win-back at 60-90 days. Should warn about treating SMS like email (frequency cap, relevance bar, real per-send cost ~$0.0075-$0.04). Should reference compliance.md for opt-in disclosure language and quiet hours.",
      "assertions": [
        "Checks for product-marketing.md",
        "Runs compliance/A2P 10DLC readiness check",
        "Recommends Klaviyo SMS or Postscript with rationale",
        "Prioritizes abandoned cart as highest-ROI flow",
        "Mentions replenishment for skincare specifically",
        "Warns about treating SMS like email",
        "References compliance.md or opt-in disclosure requirements",
        "Mentions per-send cost economics"
      ],
      "files": []
    },
    {
      "id": 2,
      "prompt": "Write me an abandoned cart SMS sequence. We sell custom apparel, average order $80.",
      "expected_output": "Should output a 3-message sequence following references/sequence-templates.md pattern. Should specify timing: Send 1 at 30 min after abandon (no discount, gentle reminder), Send 2 at 4 hours (soft urgency, no discount), Send 3 at 24 hours (discount allowed). Should include actual SMS copy with character counts (target 160 GSM-7 for 1 segment). Each message must start with sender ID 'From [Brand]:', have a single CTA + short link, and the first message should include 'Reply STOP to opt out' compliance footer. Should warn against discount on first send (trains customers to abandon). Should mention exclusion rules: stop sequence on purchase, opt-out, or 48 hours elapsed. Should recommend UTM tagging for attribution and cross-reference analytics skill for measurement.",
      "assertions": [
        "Outputs 3-message sequence with timing",
        "Send 1 at 30 min, Send 2 at 4 hours, Send 3 at 24 hours",
        "No discount on Send 1",
        "Each message has sender ID + single CTA + short link",
        "Character counts shown, target ~160 GSM-7",
        "Compliance footer on first send (STOP to opt out)",
        "Warns about discount on first send",
        "Mentions exclusion rules",
        "Mentions UTM tagging or attribution"
      ],
      "files": []
    },
    {
      "id": 3,
      "prompt": "Can I just send SMS without any opt-in if customers gave me their phone number at checkout?",
      "expected_output": "Should refuse and explain TCPA requires express written consent for marketing SMS. Should distinguish marketing SMS (requires express written consent) from transactional/account SMS (order updates, auth — implied consent during transaction OK if directly related). Should explain the express written consent requirements: clear disclosure adjacent to the phone field, frequency expectation, msg & data rates notice, STOP/HELP instructions, terms link, electronically captured with timestamp. Should mention penalty exposure: $500-$1,500 per message, class actions reach 7-8 figures. Should recommend implementing a compliant opt-in flow: checkbox + disclosure text, double opt-in optional but cleaner. Should reference compliance.md for the full disclosure template. Should warn that 'customers gave their number at checkout' is NOT sufficient for marketing SMS — it's only sufficient for the specific transaction's communications.",
      "assertions": [
        "Refuses the no-opt-in approach",
        "Distinguishes marketing SMS from transactional SMS",
        "Lists express written consent requirements",
        "Mentions TCPA penalty exposure ($500-$1,500 per message)",
        "Mentions class action risk",
        "Recommends compliant opt-in flow",
        "References compliance.md",
        "Clarifies checkout phone capture is not marketing consent"
      ],
      "files": []
    },
    {
      "id": 4,
      "prompt": "Our SMS list is 50K subscribers. We send 3 promotional messages per week. Opt-out rate has crept up to 4% per send. What's wrong?",
      "expected_output": "Should diagnose this as audience fatigue from over-sending. Should reference healthy benchmarks: opt-out rate should be <2% per send and <0.5% for promotional sends — 4% is significantly elevated. Should review send frequency: 3 promotional sends/week is on the high side; recommend reducing to 1-2/week, especially for newer subscribers. Should audit relevance: are sends segmented or going to entire list? Generic blasts to a 50K list will burn out the inactive 30K. Should recommend segmenting by engagement (recently engaged vs cold), purchase recency, and opt-in source. Should suggest reactivating cold subscribers with a re-engagement flow before sending more promos. Should warn that 4% opt-out per send means the list is being destroyed at the rate of ~2K/week. Should cross-reference analytics for proper measurement and the principle 'every send must justify itself.'",
      "assertions": [
        "Diagnoses as over-sending / audience fatigue",
        "Cites healthy benchmark (<2% opt-out per send, <0.5% promotional)",
        "Recommends reducing send frequency",
        "Recommends segmentation by engagement",
        "Suggests reactivation flow for cold subscribers",
        "Calculates list erosion impact",
        "Mentions 'every send must justify itself' principle"
      ],
      "files": []
    },
    {
      "id": 5,
      "prompt": "We just submitted our A2P 10DLC registration and our sends are working. Can we start scaling to 100K+ messages per day?",
      "expected_output": "Should ask about phone number type currently in use: 10DLC, toll-free, or short code. Should explain throughput limits: 10DLC standard brand ~4-10 msg/sec, verified brand ~75-100+ msg/sec, toll-free ~3 msg/sec, short code 100+ msg/sec. Should calculate: 100K msgs at 10 msg/sec = ~2.8 hours of continuous send time, may run into quiet hour cutoff. Should recommend short code lease for 100K+/day sustained volume. Should warn about carrier filtering on burst sends — use platform throttling. Should mention that sample message text from A2P registration must match actual sends or carriers will flag. Should recommend monitoring trust score and deliverability dashboards. Should reference platforms.md for short code provisioning details.",
      "assertions": [
        "Asks about phone number type (10DLC vs toll-free vs short code)",
        "Explains throughput limits with specific msg/sec numbers",
        "Calculates time-to-send for 100K volume",
        "Mentions quiet hour considerations",
        "Recommends short code for high sustained volume",
        "Warns about carrier filtering / throttling",
        "Mentions A2P sample text alignment requirement",
        "References platforms.md or trust score monitoring"
      ],
      "files": []
    },
    {
      "id": 6,
      "prompt": "Should we put emojis in our SMS messages? Other brands seem to use them a lot.",
      "expected_output": "Should explain the cost trade-off: emojis force UCS-2 encoding, which cuts segment size from 160 GSM-7 chars to 70 chars. A 100-char message with one emoji becomes 2 segments billed instead of 1 — effectively doubling the per-send cost. Should advise: 1 emoji per message max, situationally relevant, only when the emoji genuinely earns its segment cost (high-energy promotional, brand-personality fit, etc.). Should warn against emoji clutter — it signals 'mass send' rather than personal. Should note that some accented characters (curly quotes, em dashes) also force UCS-2 — copy-pasting from Word/Google Docs is a common silent cause of doubled costs. Should recommend testing in the platform's preview to verify segment count before scheduling. Should remind that segment count matters at scale: 100K sends at 2 segments instead of 1 = $750-$4,000 in extra cost per campaign.",
      "assertions": [
        "Explains UCS-2 encoding cost",
        "Specifies 160 GSM-7 vs 70 UCS-2 segment sizes",
        "Recommends max 1 emoji per message",
        "Warns about doubled per-send cost",
        "Mentions accented characters / curly quotes also trigger UCS-2",
        "Recommends previewing segment count",
        "Calculates cost impact at scale"
      ],
      "files": []
    }
  ]
}

FILE:references/compliance.md
# SMS Compliance Reference

Comprehensive compliance reference for SMS marketing across major jurisdictions, opt-in copy templates, and STOP/HELP response templates.

> This is operational guidance, not legal advice. For high-volume programs (50K+ subscribers) or any program with non-trivial revenue, run your compliance setup past a TCPA-experienced attorney.

---

## United States — TCPA

### What it is

The Telephone Consumer Protection Act (1991, amended) regulates marketing calls and texts. The FCC enforces it; private plaintiffs sue under it. Statutory damages: $500–$1,500 **per message**. Class actions easily reach 7–8 figures.

### Consent tiers

| Type | What it covers | How to capture |
|------|---------------|----------------|
| **Express written consent** | Marketing SMS (sales, promotions, offers) | Checkbox + clear disclosure language, captured electronically with timestamp |
| **Express consent (non-written)** | Informational/transactional (delivery, account alerts) | Phone number provided during transaction with awareness it'll be used to text |
| **Established business relationship** | NOT sufficient for marketing SMS | Doesn't apply |

### Express written consent requirements

The opt-in flow must capture all of:

1. The recipient agreed to receive marketing SMS from your brand
2. The recipient understands consent is not a condition of purchase
3. The disclosure showed frequency expectation, message and data rate notice, STOP/HELP instructions, terms link
4. The agreement was electronically recorded with timestamp

### Opt-in disclosure template (compliant)

```
By signing up via text, you agree to receive recurring automated promotional and
personalized marketing text messages (e.g., cart reminders) from [Brand] at the
cell number used when signing up. Consent is not a condition of any purchase.
Reply HELP for help and STOP to cancel. Msg frequency varies. Msg & data rates
may apply. View [Terms](link) and [Privacy](link).
```

Place this **directly adjacent** to the phone number field and submit button. Do not bury it in a footer.

### Quiet hours

- **Federal**: 8am–9pm in the recipient's local time zone
- **Stricter states**: Florida (8am–8pm), Oklahoma (8am–8pm), Washington (8am–8pm)
- **Carrier-recommended**: 9am–8pm recipient-local
- **Practical default**: 9am–8pm recipient-local for safety

Time zone is determined by area code, but area codes lie (people move). Major platforms (Klaviyo, Postscript, Attentive) handle this automatically; verify yours does.

### STOP/HELP handling

**STOP variants you must honor**: STOP, END, CANCEL, UNSUBSCRIBE, QUIT, STOPALL, OPTOUT

**STOP response** (after STOP received):
```
You're unsubscribed from [Brand] alerts. No more messages will be sent. Reply HELP for help.
```

**HELP variants**: HELP, INFO

**HELP response**:
```
[Brand] alerts: For help, visit [URL] or email [support@brand.com]. Msg & data rates may apply. Reply STOP to cancel.
```

**Critical rules**:
- Honor STOP **within seconds**, every time, every keyword variant
- Do not require the recipient to log in or visit a website to opt out
- One STOP confirmation is allowed; do not send additional messages after
- HELP responses do not count as marketing messages and are not subject to quiet hours

### Sample TCPA-compliant footer language by sequence type

- **Opt-in confirmation**: "Reply HELP for help, STOP to cancel. Msg & data rates may apply." — required
- **Recurring promotional**: "Reply STOP to opt out" — required quarterly minimum; carrier-recommended every send
- **Transactional**: Not required by TCPA but carriers expect it; include for safety

---

## United States — A2P 10DLC

### What it is

Application-to-Person 10-Digit Long Code registration, run by The Campaign Registry (TCR). Required for businesses sending SMS through 10DLC numbers (regular long codes) since 2022. Carriers (T-Mobile, AT&T, Verizon) enforce this; unregistered traffic gets throttled or blocked.

### Registration components

1. **Brand registration**
   - Legal entity name, EIN, business type
   - Trust score assigned (Standard or Verified)
   - Higher trust = better throughput, lower fees

2. **Campaign registration** (one per use case)
   - Use case: Marketing, Account Notification, Customer Care, Public Service, Higher Education, Polling and Voting, 2FA, Delivery Notification, etc.
   - Sample message text (must match what you actually send)
   - Opt-in flow description and screenshot
   - Opt-out language
   - Help message language
   - Volume estimate

3. **Phone number assignment** to campaigns

### Throughput tiers (varies by carrier and trust score)

| Trust score + use case | Throughput |
|------------------------|-----------|
| Verified brand, marketing | 75–100+ msg/sec |
| Standard brand, marketing | 4–10 msg/sec |
| Unregistered | 0.1 msg/sec or blocked |

### Common rejections

- Sample message text doesn't match actual sends
- Opt-in flow screenshot doesn't show required disclosure language
- "SHAFT" content (Sex, Hate, Alcohol, Firearms, Tobacco) without explicit use case
- Generic or vague campaign descriptions

**Process time**: 1–7 business days. Plan for this in launch timelines.

---

## EU / UK — GDPR + ePrivacy Directive

### Consent requirements

- **Explicit opt-in**: clear affirmative action (no pre-checked boxes)
- **Specific**: opt-in must be for marketing SMS specifically, separate from generic ToS
- **Informed**: data subject must know who's processing and why
- **Freely given**: can't be bundled with service access

### Mandatory provisions

- Sender identity in every message
- Easy opt-out in every message
- Right to access data (DSARs)
- Right to deletion
- Records of consent kept for the duration of processing + statute of limitations

### Penalty exposure

GDPR fines up to €20M or 4% of global revenue, whichever is higher.

---

## Canada — CASL

### Consent

- **Express consent**: explicit opt-in (same standard as US TCPA express written consent)
- **Implied consent**: existing business relationship within 24 months — limited use, expires

### Every message must include

- Sender identification (legal name + any operating names)
- Mailing address
- Phone, email, or website contact
- Unsubscribe mechanism that works within 10 business days

### Penalty exposure

Up to CAD $10M per violation. Enforced by the CRTC.

---

## Australia — Spam Act 2003

- Express or inferred consent (inferred has narrow application)
- Sender ID required
- Functional unsubscribe required
- Enforced by ACMA

---

## Multi-jurisdictional programs

If you send across US + EU + Canada simultaneously:

- Default to the **strictest** standard across all jurisdictions (US TCPA express written consent + GDPR explicit opt-in)
- Track consent jurisdiction per subscriber
- Default quiet hours to recipient-local 9am–8pm
- Include all required identifiers in every message

---

## Audit-ready compliance checklist

- [ ] A2P 10DLC registration complete (US, if applicable)
- [ ] Opt-in flow includes all required disclosures, adjacent to phone field
- [ ] Disclosure text matches A2P registered sample messages
- [ ] Opt-in event captures: timestamp, IP, page URL, exact disclosure shown
- [ ] STOP/HELP keywords honored across all variants
- [ ] Quiet hours enforced at platform level (recipient-local time)
- [ ] Privacy policy includes SMS section
- [ ] Terms of service include SMS terms
- [ ] Consent records retained per applicable law (typically 4+ years US, longer EU)
- [ ] Process for handling DSARs (EU) and consent revocation
- [ ] Sender identity in every message
- [ ] Compliance footer on every promotional message (recommended) or quarterly minimum (required)
- [ ] Test STOP/HELP from a real phone number quarterly to verify it still works

FILE:references/platforms.md
# SMS Platform Reference

Deep-dive on the major SMS marketing platforms — features, pricing, A2P 10DLC support, and integration paths.

> Pricing is approximate and changes regularly. Always confirm at the vendor's site before committing.

---

## Klaviyo SMS

**Best for**: DTC ecom brands already using Klaviyo for email.

### Key features
- Native integration with Klaviyo email and segmentation
- Shared subscriber profile across email + SMS
- Built-in A2P 10DLC registration
- Flow builder shared with email flows
- Conversational SMS (two-way) supported

### Pricing
- Bundled with Klaviyo plans, billed per SMS credit
- US: ~$0.0075–$0.015 per SMS; MMS ~$0.04
- Free tier: 150 SMS credits/month on lower email tiers

### Integration paths
- Direct Shopify, WooCommerce, BigCommerce, Magento integration
- API for custom platforms
- MCP server available

### Compliance
- A2P 10DLC registration handled in-platform
- Toll-free and short code provisioning available (short code adds $1,000+/mo)
- Quiet hours enforced per recipient time zone (configurable)

### Watch out for
- Email + SMS combined billing can spike fast on large lists
- Short code costs are real overhead; only worthwhile for 100K+ active SMS subscribers

---

## Postscript

**Best for**: Shopify-native DTC brands wanting SMS-specific tooling and onboarding support.

### Key features
- Deep Shopify integration (the deepest of any SMS platform)
- Strong abandoned cart and browse abandonment automations
- AI Reply (auto-reply trained on brand voice)
- Conversational SMS / live agent
- Audiences pulled from Shopify customer data

### Pricing
- Tiered plans: Starter (free, 1K msgs/mo), Growth ($100+/mo), Professional, Enterprise
- Pay-per-send adds on top: ~$0.015 per SMS, ~$0.04 per MMS

### Integration paths
- Shopify-first; limited support for non-Shopify
- API + webhooks available

### Compliance
- A2P 10DLC handled in-platform
- Strong opt-in compliance tools (popup builder, keyword opt-in)
- Quiet hours enforced

### Watch out for
- Steep cost increase past Starter tier
- Less useful if you're not on Shopify

---

## Attentive

**Best for**: Mid-market and enterprise DTC brands wanting full-service SMS.

### Key features
- Full-service: dedicated CSM, copy support, strategy
- Conversational SMS at scale
- Concierge sales-via-SMS
- Strong analytics and attribution
- Identity resolution (matching anon site visitors to phone numbers)

### Pricing
- Custom contracts; typically $1K–$10K+/mo + per-send fees
- Annual contracts standard
- Pricing rarely makes sense for <50K SMS subscribers

### Integration paths
- Shopify, BigCommerce, Salesforce Commerce Cloud, custom
- Robust API

### Compliance
- Full A2P 10DLC managed
- Best-in-class compliance tooling and audit support
- Short code provisioning included on most plans

### Watch out for
- Contract terms can lock you in for 12+ months
- Overkill for early-stage brands

---

## Twilio

**Best for**: Custom builds, transactional SMS, B2B SaaS embedding SMS into products, developers.

### Key features
- Raw SMS API
- Pay-per-send pricing, no platform fees
- Massive global coverage (200+ countries)
- Programmable Voice, WhatsApp Business, RCS available alongside
- Studio (visual flow builder) for non-code automation

### Pricing
- US 10DLC SMS: $0.0079 per message
- US toll-free SMS: $0.0079 per message
- US short code SMS: $0.0079 per message + $1,000/mo lease
- MMS: ~$0.02
- Carrier surcharges layered on top (~$0.005 per US 10DLC)
- A2P 10DLC registration: ~$15 brand + $10/mo per campaign

### Integration paths
- API-first (REST + SDKs in Node, Python, Ruby, Go, etc.)
- No native ecom integrations — you build them

### Compliance
- A2P 10DLC registration in-platform but you do the work
- TwilioSendGrid (separate product) handles email-side compliance
- Quiet hours and STOP/HELP handling must be implemented by you

### Watch out for
- You're responsible for compliance — no hand-holding
- No native segmentation, deliverability dashboards, or marketing UI
- Best paired with Customer.io, Segment, or a custom orchestration layer

---

## Brevo (formerly Sendinblue)

**Best for**: EU-based brands, email + SMS combo, SMB-friendly.

### Key features
- Combined email + SMS + WhatsApp on one platform
- EU-headquartered, GDPR-native
- Generous free tier for email; SMS pay-per-send
- Marketing automation flows
- CRM included

### Pricing
- Free tier: 300 emails/day; SMS pay-per-send
- US SMS: ~$0.015 per message
- EU SMS: varies by country, ~€0.04–€0.07

### Integration paths
- Direct integrations: Shopify, WooCommerce, WordPress, Magento
- API + Zapier
- MCP server available

### Compliance
- GDPR + ePrivacy built-in
- A2P 10DLC for US (less polished than dedicated US platforms)

### Watch out for
- US SMS features lag behind Klaviyo/Postscript
- Best if you're EU-first or already on Brevo for email

---

## SimpleTexting

**Best for**: SMB, services businesses, simple campaign blasts, low-volume.

### Key features
- Easy-to-use UI
- Keyword opt-in for grassroots list building
- Built-in landing pages for opt-in
- Simple automation

### Pricing
- Plans start ~$30/mo for 500 credits, scaling up
- US SMS only

### Integration paths
- Zapier, Make, native to a few apps
- API available but basic

### Compliance
- A2P 10DLC handled
- TCPA tooling

### Watch out for
- Limited automation depth vs Klaviyo/Postscript
- Best for low-complexity, low-volume use cases (gyms, salons, real estate)

---

## Plivo

**Best for**: Custom SMS builds where per-send cost matters; Twilio-style API at a lower price point.

### Key features
- Direct Twilio competitor with similar surface area
- Powerpack for bulk sending with sticky sender across number pools
- A2P 10DLC handled in-platform
- WhatsApp, voice available alongside SMS
- SDKs for major languages

### Pricing
- US 10DLC SMS: ~$0.0055/msg (typically 20–30% under Twilio)
- US short code SMS: similar + monthly lease
- MMS: ~$0.02
- Phone number rental: ~$0.80/mo local, ~$1/mo toll-free

### Integration paths
- API-first (REST + SDKs)
- No native ecom integrations — you build them

### Compliance
- A2P 10DLC managed in-platform
- Compliance plumbing (STOP/HELP, quiet hours) is your responsibility — same model as Twilio

### Watch out for
- Smaller ecosystem than Twilio (fewer ancillary products, integrations, community resources)
- WhatsApp tooling less mature

---

## AudienceTap

**Best for**: DTC brands wanting AI-forward creative tooling or on-pack QR opt-in as a primary acquisition channel.

> Newer platform — verify current capabilities, pricing, and API surface before committing.

### Key features
- SMS + email on one platform (similar combined model to Klaviyo)
- AI creative generation (SMS copy, subject lines, image variants)
- On-pack QR code opt-in: insert cards in shipped orders that drive SMS list growth
- Shopify, BigCommerce, headless commerce integrations
- A2P 10DLC managed in-platform
- Identity resolution and segmentation

### Pricing
- Tiered by subscriber count + send volume
- Per-send pricing comparable to other DTC SMS platforms

### Integration paths
- API access on Growth+ tiers
- Direct ecom integrations
- Webhooks for events

### Compliance
- A2P 10DLC handled in-platform
- TCPA tooling — verify enterprise-scale depth before committing for large lists

### Watch out for
- Newer entrant — fewer reference customers, less battle-tested at high volume than incumbents
- Some features rolled out recently — confirm what's GA vs beta before relying on them

---

## Customer.io

**Best for**: B2B SaaS, behavior-based automation, multi-channel orchestration (email + SMS + push).

### Key features
- Trigger SMS off product events (signup, milestone, churn risk)
- Powerful audience segmentation
- Workflow builder
- Real-time data sync via API/webhooks

### Pricing
- Plans start ~$150/mo, scaling with profile count
- SMS via Twilio integration or native (varies)

### Integration paths
- API-first
- Direct integrations with Segment, Heap, Mixpanel, etc.

### Compliance
- A2P 10DLC via Twilio if using native integration
- Granular subscription/consent management

### Watch out for
- Less ecom-tailored than Klaviyo/Postscript
- Best for product-led SaaS or apps with deep event tracking

---

## Quick selection table

| Stack / Goal | Recommended | Why |
|--------------|------------|-----|
| Shopify ecom, already on Klaviyo | **Klaviyo SMS** | One platform, one subscriber profile |
| Shopify ecom, SMS-first focus | **Postscript** | Deepest Shopify + SMS-specific features |
| Mid-market ecom, want concierge support | **Attentive** | Full-service team + tooling |
| Custom platform, B2B SaaS, transactional | **Twilio** | API-first, full control |
| Custom build, cost-sensitive | **Plivo** | ~20–30% cheaper than Twilio per send |
| DTC wanting AI creative or on-pack QR opt-in | **AudienceTap** | AI-forward; insert-card opt-in is unique |
| EU-based SMB | **Brevo** | GDPR-native, EU-friendly pricing |
| Local services SMB, simple campaigns | **SimpleTexting** | Easy UI, low overhead |
| Product-led SaaS with event tracking | **Customer.io** | Behavior-based triggers |

---

## A2P 10DLC: what your platform should handle

Whatever you pick, confirm your platform handles:

- [ ] Brand and campaign registration with TCR
- [ ] Sample message text aligned with what you actually send
- [ ] Opt-in flow documentation submitted to carriers
- [ ] Trust score visibility (and a path to improve it)
- [ ] Throughput appropriate to your list size and send frequency
- [ ] STOP/HELP keyword handling
- [ ] Quiet hours by recipient time zone
- [ ] Suppression list management
- [ ] Consent record retention with timestamps

All major platforms above handle these. Twilio does the lowest-level work and pushes more responsibility onto you.

FILE:references/sequence-templates.md
# SMS Sequence Templates

Full copy templates with character counts, timing, and segmentation logic for every major SMS flow.

> Character counts shown assume GSM-7 encoding. Emojis force UCS-2 (70 chars/segment instead of 160). All templates use `[Brand]`, `[FirstName]`, and `[short.link]` as substitution tokens.

---

## Welcome / Opt-In Confirmation

### Send 1 — Immediate (after opt-in)

```
From [Brand]: Welcome! Here's your 10% off code: WELCOME10. Shop now: [short.link]
Reply STOP to opt out, HELP for help. Msg & data rates may apply.
```
~155 chars / 1 segment (just). Footer required on first send.

### Send 2 — 24 hours later (optional)

```
From [Brand]: Don't forget your code WELCOME10 — expires in 48hrs. Top picks: [short.link]
```
~108 chars / 1 segment.

### Send 3 — 7 days later (optional, conditional on no purchase)

```
From [Brand]: Last chance for 10% off with WELCOME10. Expires tonight at midnight: [short.link]
```
~107 chars / 1 segment.

---

## Abandoned Cart (highest-ROI flow for ecom)

### Send 1 — 30 minutes after abandon

```
From [Brand]: Hey [FirstName], you left something behind! Your cart's here: [short.link]
```
~95 chars / 1 segment.

### Send 2 — 4 hours after abandon (if no purchase)

```
From [Brand]: Items in your cart are selling fast. Reserved for you for 24hrs: [short.link]
```
~98 chars / 1 segment.

### Send 3 — 24 hours after abandon (if no purchase, discount allowed)

```
From [Brand]: Still thinking? Here's 10% off to seal the deal: SAVE10. Shop: [short.link]
```
~99 chars / 1 segment.

**Notes**:
- Discount on Send 1 trains customers to abandon. Reserve for Send 2 or 3.
- Exclude customers who abandoned <$X in cart value or repeat abandoners (gaming the discount).
- Stop sequence on purchase, opt-out, or 48 hours elapsed.

---

## Browse Abandonment

### Send 1 — 1 hour after browse (single product or category)

```
From [Brand]: Still thinking about [product]? Take another look: [short.link]
```
~84 chars / 1 segment.

**Notes**:
- Trigger only after meaningful browse signal (3+ product views or 2+ min on product page).
- Exclude if a purchase happened on a different product.

---

## Post-Purchase Flow

### Send 1 — Immediately after purchase (transactional, separate consent)

```
From [Brand]: Order #12345 confirmed! We'll text shipping updates here. Track: [short.link]
```
~95 chars / 1 segment.

### Send 2 — Day of shipment

```
From [Brand]: Your order's on the way. Estimated delivery: [date]. Track: [short.link]
```
~92 chars / 1 segment.

### Send 3 — Day of delivery

```
From [Brand]: Your order should arrive today! Questions? Reply or visit [short.link]
```
~88 chars / 1 segment.

### Send 4 — 2 days after delivery (marketing consent required)

```
From [Brand]: How are you liking your [product]? Share a review for 15% off next order: [short.link]
```
~108 chars / 1 segment.

### Send 5 — 14 days after delivery (cross-sell, marketing consent)

```
From [Brand]: Goes great with your [product]: [related-item]. 10% off bundle: [short.link]
```
~99 chars / 1 segment.

---

## Win-Back (Lapsed Customers)

### Send 1 — 60-90 days after last purchase

```
From [Brand]: [FirstName], we miss you! Picks we think you'll love: [short.link]
```
~84 chars / 1 segment.

### Send 2 — 14 days later (if no purchase)

```
From [Brand]: Come back for 15% off your next order: COMEBACK15. Expires in 7 days: [short.link]
```
~106 chars / 1 segment.

### Send 3 — 14 days after Send 2 (final, if no purchase)

```
From [Brand]: Last chance — 20% off ends tonight: COMEBACK20. We'll stop texting if you'd rather: reply STOP. [short.link]
```
~130 chars / 1 segment.

**Notes**:
- After Send 3 with no engagement, suppress for 90 days minimum.
- After two full win-back cycles with no engagement, sunset (remove from active list).

---

## Promotional / Campaign Sends

### Flash sale (single send)

```
From [Brand]: 24-HOUR FLASH: 25% off everything with FLASH25. Ends midnight: [short.link]
```
~94 chars / 1 segment.

### Limited drop / launch

```
From [Brand]: New drop just landed: [product-name]. Limited stock, members get early access: [short.link]
```
~115 chars / 1 segment.

### Holiday / BFCM (2-send sequence)

Send 1 — Day of launch:
```
From [Brand]: Black Friday is LIVE — up to 50% off sitewide. Shop now: [short.link]
```
~92 chars / 1 segment.

Send 2 — Day of (or evening, expiration push):
```
From [Brand]: Last 6 hours of BFCM savings. Don't miss out: [short.link]
```
~73 chars / 1 segment.

---

## Transactional / Account Notifications

### Order confirmation

```
[Brand]: Order #12345 confirmed. Total $XX.XX. Track at [short.link]. Reply HELP for help.
```

### Shipping update

```
[Brand]: Your order #12345 shipped! Track: [short.link]. ETA [date].
```

### Delivery confirmation

```
[Brand]: Order #12345 delivered. Enjoy! Issues? Reply or [support-link].
```

### Auth code (2FA)

```
[Brand] verification code: 123456. Expires in 10 min. Do not share.
```

### Account alert

```
[Brand]: Sign-in from new device in [location]. Wasn't you? Secure: [short.link]
```

---

## Re-Engagement / Reactivation (Subscribers Who've Gone Cold)

For SMS subscribers who haven't engaged with any send in 60+ days.

### Send 1 — Soft reactivation

```
From [Brand]: We've missed you, [FirstName]! Here's what's new: [short.link]
```
~80 chars / 1 segment.

### Send 2 — Confirm interest (if no engagement)

```
From [Brand]: Want to keep hearing from us? Reply YES to stay on the list, or STOP to opt out.
```
~98 chars / 1 segment.

After no reply: suppress for 60 days, then remove from active list. This protects opt-out rate metrics and reduces wasted spend.

---

## Replenishment (Consumables Ecom)

For products with predictable usage cycles (skincare, supplements, coffee, pet food).

### Send 1 — At expected reorder window (e.g., 28 days for a 30-day supply)

```
From [Brand]: Running low on [product]? Reorder in one tap: [short.link]
```
~73 chars / 1 segment.

### Send 2 — 7 days later (if no purchase)

```
From [Brand]: Don't run out! 10% off your reorder of [product]: REFILL10 [short.link]
```
~92 chars / 1 segment.

---

## VIP / Loyalty Members

Higher frequency, exclusive offers, early access — different cadence rules apply but quiet hours and STOP still required.

### Early access

```
From [Brand]: VIPs get the new drop 24hrs early. Yours now: [short.link]
```
~72 chars / 1 segment.

### Loyalty milestone

```
From [Brand]: You've reached Gold status! Your perks: 15% off + free shipping. [short.link]
```
~95 chars / 1 segment.

---

## Segmentation rules across all flows

- **Suppress** customers in active sequences from promotional sends (no double-tap)
- **Suppress** opted-out subscribers from everything (platform handles this)
- **Frequency cap**: max 4–6 marketing sends/week per subscriber (lower for newer subscribers)
- **Quiet hours**: 9am–8pm recipient-local time
- **Cool-off**: After a discount-driven purchase, suppress promotional sends for 14 days

---
name: social
description: "When the user wants help creating, scheduling, or optimizing social media content for LinkedIn, Twitter/X, Instagram, TikTok, Facebook, or other platforms, or wants to do social listening and engagement triage. Also use when the user mentions 'LinkedIn post,' 'Twitter thread,' 'social media,' 'content calendar,' 'social scheduling,' 'engagement,' 'viral content,' 'what should I post,' 'repurpose this content,' 'tweet ideas,' 'LinkedIn carousel,' 'social media strategy,' 'grow my following,' 'TikTok video,' 'Reels,' 'Shorts,' 'video script,' 'video hook,' 'short-form video,' 'create a reel,' 'social listening,' 'brand mentions,' 'competitor monitoring,' 'top posts to comment on,' 'find people asking for,' 'carousel,' 'slide-by-slide,' or 'document post.' Use this for social media content creation, repurposing, scheduling, short-form video scripting, and social listening. For broader content strategy, see content-strategy. For paid ads, see ad-creative. For earned media, see public-relations."
metadata:
  version: 2.2.0
---

# Social Content

You are an expert social media strategist. Your goal is to help create engaging content that builds audience, drives engagement, and supports business goals.

## Before Creating Content

**Check for product marketing context first:**
If `.agents/product-marketing.md` exists (or `.claude/product-marketing.md`, or the legacy `product-marketing-context.md` filename, in older setups), read it before asking questions. Use that context and only ask for information not already covered or specific to this task.

Gather this context (ask if not provided):

### 1. Goals
- What's the primary objective? (Brand awareness, leads, traffic, community)
- What action do you want people to take?
- Are you building personal brand, company brand, or both?

### 2. Audience
- Who are you trying to reach?
- What platforms are they most active on?
- What content do they engage with?

### 3. Brand Voice
- What's your tone? (Professional, casual, witty, authoritative)
- Any topics to avoid?
- Any specific terminology or style guidelines?

### 4. Resources
- How much time can you dedicate to social?
- Do you have existing content to repurpose?
- Can you create video content?

---

## Platform Quick Reference

| Platform | Best For | Frequency | Key Format |
|----------|----------|-----------|------------|
| LinkedIn | B2B, thought leadership | 3-5x/week | Carousels, stories |
| Twitter/X | Tech, real-time, community | 3-10x/day | Threads, hot takes |
| Instagram | Visual brands, lifestyle | 1-2 posts + Stories daily | Reels, carousels |
| TikTok | Brand awareness, younger audiences | 1-4x/day | Short-form video |
| Facebook | Communities, local businesses | 1-2x/day | Groups, native video |

**For detailed platform strategies**: See [references/platforms.md](references/platforms.md)

**For hashtag limits and character counts**: See [references/platform-limits.md](references/platform-limits.md)

---

## Content Pillars Framework

Build your content around 3-5 pillars that align with your expertise and audience interests.

### Example for a SaaS Founder

| Pillar | % of Content | Topics |
|--------|--------------|--------|
| Industry insights | 30% | Trends, data, predictions |
| Behind-the-scenes | 25% | Building the company, lessons learned |
| Educational | 25% | How-tos, frameworks, tips |
| Personal | 15% | Stories, values, hot takes |
| Promotional | 5% | Product updates, offers |

### Pillar Development Questions

For each pillar, ask:
1. What unique perspective do you have?
2. What questions does your audience ask?
3. What content has performed well before?
4. What can you create consistently?
5. What aligns with business goals?

---

## Hook Formulas

The first line determines whether anyone reads the rest.

### Curiosity Hooks
- "I was wrong about [common belief]."
- "The real reason [outcome] happens isn't what you think."
- "[Impressive result] — and it only took [surprisingly short time]."

### Story Hooks
- "Last week, [unexpected thing] happened."
- "I almost [big mistake/failure]."
- "3 years ago, I [past state]. Today, [current state]."

### Value Hooks
- "How to [desirable outcome] (without [common pain]):"
- "[Number] [things] that [outcome]:"
- "Stop [common mistake]. Do this instead:"

### Contrarian Hooks
- "Unpopular opinion: [bold statement]"
- "[Common advice] is wrong. Here's why:"
- "I stopped [common practice] and [positive result]."

**For post templates and more hooks**: See [references/post-templates.md](references/post-templates.md)

**For carousels** (Instagram carousels, LinkedIn document posts): See [references/carousel-frameworks.md](references/carousel-frameworks.md) — five slide-by-slide narrative architectures (Value-Stack, Problem-Proof, Hack List, Rant Callout, Demo Walkthrough) with framework selection guidance, per-slide copy slots, platform notes, and a production checklist. Pick the framework before writing slides.

---

## Content Repurposing System

Turn one piece of content into many. The best social content isn't created from scratch — it's extracted from longer-form pillar content and adapted to each platform.

### Blog Post → Social Content

| Platform | Format |
|----------|--------|
| LinkedIn | Key insight + link in comments |
| LinkedIn | Carousel of main points |
| Twitter/X | Thread of key takeaways |
| Instagram | Carousel with visuals |
| Instagram | Reel summarizing the post |

### Podcast / Video → Social Content

Extract "content atoms" — self-contained moments from any long-form content that work on their own:

| Atom Type | What to Look For | Best Platform |
|-----------|-----------------|---------------|
| Quotable moment | A bold claim, hot take, or memorable line (15-60 sec) | Twitter/X, LinkedIn, TikTok |
| Story arc | A complete mini-story with setup, conflict, resolution (60-90 sec) | Instagram Reels, TikTok, YouTube Shorts |
| Tactical tip | A specific how-to or framework explained clearly (30-60 sec) | LinkedIn, YouTube Shorts |
| Controversial take | A contrarian opinion that sparks debate | Twitter/X, LinkedIn |
| Data/stat callout | A surprising number or research finding | LinkedIn carousel, Twitter/X |
| Behind-the-scenes | Authentic, unpolished moments | Instagram Stories, TikTok |

**Podcast repurposing workflow:**
1. **Get transcript** — use Whisper, Descript, or your podcast host's transcription
2. **Mark timestamps** — flag the 5-10 best moments while listening or scanning transcript
3. **Extract clips** — pull video/audio clips for each moment (Descript, Opus Clip, or manual)
4. **Write standalone captions** — each clip needs context; don't assume the viewer heard the rest
5. **Add subtitles** — most social video is watched without sound
6. **Schedule across 1-2 weeks** — spread a single episode across multiple posts

**Per episode, aim for:**
- 3-5 short video clips or audiograms (15-60 sec) for Reels/TikTok/Shorts
- 1-2 LinkedIn text posts from key insights
- 1 Twitter/X thread of takeaways
- 1 carousel summarizing the main framework or list
- 1 newsletter section or blog post from the best segment

### Webinar / Live Event → Social Content

| Extract | Format |
|---------|--------|
| Key slides with commentary | LinkedIn carousel |
| Q&A highlights | Twitter/X thread |
| Speaker quotes | Quote graphics for Instagram/LinkedIn |
| Audience reactions/poll results | Engagement posts |
| Full recording → short clips | Reels, TikTok, Shorts |

### Newsletter → Social Content

| Extract | Format |
|---------|--------|
| Main insight | LinkedIn post |
| Curated links with commentary | Twitter/X thread |
| Data or stat | Quote graphic |
| Hot take or opinion | Twitter/X post, LinkedIn |

### Repurposing Workflow

1. **Create pillar content** (blog, video, podcast, webinar, newsletter)
2. **Extract content atoms** (5-10 per piece — quotes, stories, tips, data)
3. **Adapt to each platform** (format, length, and tone)
4. **Write standalone captions** (each post must work without context)
5. **Schedule across the week** (spread distribution, don't dump all at once)
6. **Update and reshare** (evergreen content can repeat every 3-6 months)

---

## Content Calendar Structure

### Weekly Planning Template

| Day | LinkedIn | Twitter/X | Instagram |
|-----|----------|-----------|-----------|
| Mon | Industry insight | Thread | Carousel |
| Tue | Behind-scenes | Engagement | Story |
| Wed | Educational | Tips tweet | Reel |
| Thu | Story post | Thread | Educational |
| Fri | Hot take | Engagement | Story |

### Batching Strategy (2-3 hours weekly)

1. Review content pillar topics
2. Write 5 LinkedIn posts
3. Write 3 Twitter threads + daily tweets
4. Create Instagram carousel + Reel ideas
5. Schedule everything
6. Leave room for real-time engagement

---

## Engagement Strategy

### Daily Engagement Routine (30 min)

1. Respond to all comments on your posts (5 min)
2. Comment on 5-10 posts from target accounts (15 min)
3. Share/repost with added insight (5 min)
4. Send 2-3 DMs to new connections (5 min)

**For surfacing *which* posts to comment on** (top-10 daily lists, brand/competitor monitoring, intent-signal triage), see [references/listening.md](references/listening.md). Includes a scoring rubric and curl recipes for Reddit, Hacker News, and Bluesky.

### Quality Comments

- Add new insight, not just "Great post!"
- Share a related experience
- Ask a thoughtful follow-up question
- Respectfully disagree with nuance

### Building Relationships

- Identify 20-50 accounts in your space
- Consistently engage with their content
- Share their content with credit
- Eventually collaborate (podcasts, co-created content)

---

## Analytics & Optimization

### Metrics That Matter

**Awareness:** Impressions, Reach, Follower growth rate

**Engagement:** Engagement rate, Comments (higher value than likes), Shares/reposts, Saves

**Conversion:** Link clicks, Profile visits, DMs received, Leads attributed

### Weekly Review

- Top 3 performing posts (why did they work?)
- Bottom 3 posts (what can you learn?)
- Follower growth trend
- Engagement rate trend
- Best posting times (from data)

### Optimization Actions

**If engagement is low:**
- Test new hooks
- Post at different times
- Try different formats
- Increase engagement with others

**If reach is declining:**
- Avoid external links in post body
- Increase posting frequency
- Engage more in comments
- Test video/visual content

---

## Content Ideas by Situation

### When You're Starting Out
- Document your journey
- Share what you're learning
- Curate and comment on industry content
- Engage heavily with established accounts

### When You're Stuck
- Repurpose old high-performing content
- Ask your audience what they want
- Comment on industry news
- Share a failure or lesson learned

---

## Scheduling Best Practices

### When to Schedule vs. Post Live

**Schedule:** Core content posts, Threads, Carousels, Evergreen content

**Post live:** Real-time commentary, Responses to news/trends, Engagement with others

### Queue Management

- Maintain 1-2 weeks of scheduled content
- Review queue weekly for relevance
- Leave gaps for spontaneous posts
- Adjust timing based on performance data

---

## Reverse Engineering Viral Content

Instead of guessing, analyze what's working for top creators in your niche:

1. **Find creators** — 10-20 accounts with high engagement
2. **Collect data** — 500+ posts for analysis
3. **Analyze patterns** — Hooks, formats, CTAs that work
4. **Codify playbook** — Document repeatable patterns
5. **Layer your voice** — Apply patterns with authenticity
6. **Convert** — Bridge attention to business results

**For the complete framework**: See [references/reverse-engineering.md](references/reverse-engineering.md)

---

## Short-Form Video (TikTok, Reels, Shorts)

Short-form video is the highest-reach format on every major platform. These frameworks apply whether you're creating for TikTok, Instagram Reels, or YouTube Shorts.

### Platform Specs

| Platform | Optimal Length | Aspect Ratio | Key Difference |
|----------|---------------|--------------|----------------|
| TikTok | 15-60 sec | 9:16 | Trending sounds, raw/authentic feel |
| Reels | 15-30 sec | 9:16 | Polished content, rewards saves/shares |
| Shorts | 30-60 sec | 9:16 | YouTube SEO applies, searchable titles |

### The 3-Second Rule

You have 3 seconds to stop the scroll. Every video needs three simultaneous hooks:

```
[VISUAL HOOK] + [VERBAL HOOK] + [TEXT OVERLAY]
```

All three should hit in the first second.

### Video Structures

**Problem-Solution (15-30 sec):**
```
[0-3s]  Hook: State the problem
[3-10s] Agitate: Why it matters
[10-25s] Solution: Your method/product/tip
[25-30s] CTA: What to do next
```

**List Format (30-60 sec):**
```
[0-3s]  Hook: "X things that [outcome]"
[3-50s] Items: One every 5-8 seconds
[50-60s] CTA
```

**Tutorial (30-60 sec):**
```
[0-3s]  Hook: Show the end result first
[3-8s]  Overview: "Here's how..."
[8-50s] Steps: Quick, clear instructions
[50-60s] Result + CTA
```

### Caption & Subtitle Best Practices

Captions increase watch time by 25-40%. Most social video is watched without sound.

- **MAX 2 lines** on screen at once
- **3-5 words per line**
- Bold, sans-serif font with black outline
- **Highlight key words** in a different color
- Match timing to speech exactly

Tools: CapCut (free), Descript, Captions.ai, Premiere Pro

### Content Ideas by Type

| Business Type | Video Ideas |
|---------------|-------------|
| SaaS | Feature demos (show outcome first), before/after, "Watch me do X in Y seconds" |
| E-commerce | Unboxing, comparisons, how it's made, customer reviews |
| Services | Process reveals, client transformations, myth-busting |
| Personal brand | Lessons learned, controversial takes, day-in-the-life |

### Common Mistakes

1. **Slow hooks** — don't build up to the point
2. **No text overlay** — many watch without sound
3. **Poor audio** — bad audio kills retention instantly
4. **Too long** — if it can be shorter, make it shorter
5. **No CTA** — tell viewers what to do
6. **Ignoring comments** — engagement in first hour matters

**For video hook formulas and scripting templates**: See [references/short-form-video.md](references/short-form-video.md)

---

## Task-Specific Questions

1. What platform(s) are you focusing on?
2. What's your current posting frequency?
3. Do you have existing content to repurpose?
4. What content has performed well in the past?
5. How much time can you dedicate weekly?
6. Are you building personal brand, company brand, or both?

---

## Related Skills

- **copywriting**: For longer-form content that feeds social
- **launch**: For coordinating social with launches
- **emails**: For nurturing social audience via email
- **marketing-psychology**: For understanding what drives engagement

FILE:evals/evals.json
{
  "skill_name": "social",
  "evals": [
    {
      "id": 1,
      "prompt": "Help me create a LinkedIn content strategy. I'm a SaaS founder building in public and want to grow my personal brand to drive awareness for my product. I currently have 500 followers and post maybe once a week.",
      "expected_output": "Should check for product-marketing.md first. Should establish content pillars (3-5) appropriate for a SaaS founder building in public: industry insights, behind-the-scenes, educational content, personal stories, promotional (minimal). Should apply the platform quick reference for LinkedIn (3-5x/week recommended, carousels and stories perform well). Should provide hook formulas for LinkedIn posts. Should create a weekly content calendar. Should include engagement strategy (daily 30-min routine). Should address going from 1x/week to 3-5x/week with a batching strategy.",
      "assertions": [
        "Checks for product-marketing.md",
        "Establishes 3-5 content pillars",
        "Applies LinkedIn platform guidance",
        "Provides hook formulas",
        "Creates weekly content calendar",
        "Includes engagement strategy",
        "Addresses batching strategy for consistency",
        "Recommends increasing from 1x to 3-5x per week"
      ],
      "files": []
    },
    {
      "id": 2,
      "prompt": "Write me a Twitter/X thread about the lessons I learned bootstrapping my SaaS to $10k MRR. Include hooks and a CTA at the end.",
      "expected_output": "Should apply the hook formulas for a story hook (e.g., '6 months ago, I had $0 MRR. Today, I hit $10k.'). Should structure the thread following platform best practices: strong hook in tweet 1, each tweet should stand alone but flow together, use specific numbers and stories, end with a CTA. Should reference the content pillar this fits into (behind-the-scenes / founder journey). Should provide the actual thread content with 8-12 tweets. Should include engagement prompts.",
      "assertions": [
        "Applies hook formulas from the skill",
        "Uses a story hook for the first tweet",
        "Structures thread with standalone but flowing tweets",
        "Uses specific numbers and stories",
        "Ends with clear CTA",
        "Provides 8-12 tweet thread content",
        "Includes engagement prompts"
      ],
      "files": []
    },
    {
      "id": 3,
      "prompt": "i have a blog post that did really well. how do i turn it into social media content for multiple platforms?",
      "expected_output": "Should trigger on casual phrasing. Should apply the content repurposing system. Should use the Blog Post → Social Content mapping: LinkedIn (key insight post + carousel of main points), Twitter/X (thread of key takeaways), Instagram (carousel with visuals + Reel summarizing the post). Should follow the repurposing workflow: create pillar content → extract key insights (3-5) → adapt to each platform → schedule across the week. Should provide specific format recommendations per platform.",
      "assertions": [
        "Triggers on casual phrasing",
        "Applies content repurposing system",
        "Uses Blog Post → Social Content mapping",
        "Provides format for LinkedIn, Twitter/X, and Instagram",
        "Follows the repurposing workflow",
        "Extracts 3-5 key insights to repurpose",
        "Provides platform-specific format recommendations"
      ],
      "files": []
    },
    {
      "id": 4,
      "prompt": "My LinkedIn posts get like 200 impressions and almost no engagement. What am I doing wrong?",
      "expected_output": "Should apply the analytics and optimization section, specifically the 'if engagement is low' guidance. Should diagnose potential issues: weak hooks (first line not compelling), posting at wrong times, not engaging with others' content, poor formatting (no line breaks, walls of text), content not resonating with audience. Should recommend specific fixes: test new hook formulas, post at different times, increase engagement with others (the daily engagement routine), try different formats (carousels, stories). Should provide before/after hook examples.",
      "assertions": [
        "Applies analytics and optimization guidance",
        "Diagnoses potential engagement issues",
        "Addresses hook quality",
        "Addresses posting timing",
        "Recommends daily engagement routine",
        "Suggests trying different content formats",
        "Provides specific before/after hook examples"
      ],
      "files": []
    },
    {
      "id": 5,
      "prompt": "Help me reverse-engineer what's working for top creators in the DevTools space on Twitter. I want to understand their content patterns.",
      "expected_output": "Should apply the reverse engineering viral content framework. Should walk through the process: identify 10-20 top accounts in DevTools, collect high-performing posts, analyze patterns (hooks, formats, CTAs, topics, posting times), codify a playbook of repeatable patterns, then layer the user's authentic voice. Should provide specific guidance on what to look for in the analysis. Should recommend tools or methods for collecting the data.",
      "assertions": [
        "Applies reverse engineering viral content framework",
        "Walks through the full process (find, collect, analyze, codify, apply)",
        "Recommends identifying 10-20 accounts",
        "Describes what patterns to analyze",
        "Emphasizes layering authentic voice",
        "Provides data collection guidance"
      ],
      "files": []
    },
    {
      "id": 6,
      "prompt": "Write me a 5-email welcome sequence for new email subscribers who came from my LinkedIn audience.",
      "expected_output": "Should recognize this is an email sequence task, not social content. Should defer to or cross-reference the emails skill, which handles welcome sequences, drip campaigns, and lifecycle emails. May note the social-to-email bridge context but should make clear that emails is the right skill for writing email sequences.",
      "assertions": [
        "Recognizes this as email sequence work",
        "References or defers to emails skill",
        "Does not attempt to write email sequence using social content patterns",
        "May note social-to-email bridge context"
      ],
      "files": []
    },
    {
      "id": 7,
      "prompt": "I want to turn our new pricing guide into a LinkedIn carousel. It has 9 negotiation tactics for SaaS buyers. Help me structure the slides.",
      "expected_output": "Should load references/carousel-frameworks.md and pick a framework BEFORE writing slides — a list of 9 named tactics maps to Hack List (or Value-Stack; should justify the choice), not Rant Callout or Demo Walkthrough. Should produce a slide-by-slide structure: contrarian or exact-count cover (slide 1 written as a standalone scroll-stopper), one named tactic per slide, and a synthesis + single-CTA close. Should apply LinkedIn document-post notes (PDF upload, denser text tolerated, post text written as its own hook rather than 'see attached'). Should keep any count promised on the cover exactly delivered and flag that stats/claims must be real. Should mention judging performance on saves/completion rather than likes.",
      "assertions": [
        "Loads or applies the carousel-frameworks reference",
        "Selects an appropriate framework (Hack List or Value-Stack) with a stated reason before drafting slides",
        "Slide 1 is written as a standalone scroll-stopper with the count or contrarian claim",
        "One tactic/idea per slide, each named",
        "Final slide has a synthesis and exactly one CTA",
        "Applies LinkedIn document-post guidance (PDF, post text as its own hook)",
        "Does not invent stats or pad the tactic count"
      ],
      "files": []
    }
  ]
}

FILE:references/carousel-frameworks.md
# Carousel Frameworks

Five slide-by-slide narrative architectures for carousels — Instagram carousels and LinkedIn document posts. Each framework is a different *reason to keep swiping*; picking the right one for the content matters more than polishing individual slides.

A carousel is not a blog post chopped into squares. It's a swipe-through with two jobs per slide: deliver one idea, and make the next swipe irresistible. These frameworks encode structures that show up repeatedly in high-performing carousels; treat them as starting structures to adapt, not rigid formulas — and expect to validate against your own analytics.

## Picking a Framework

| Your content is... | Use | Why |
|---|---|---|
| A list of resources, tools, or tips | **A: Value-Stack** | Completeness is the promise; the count is the hook |
| A personal result with a system behind it | **B: Problem-Proof** | Proof opens and closes the loop; the system is the meat |
| Several named techniques on one theme | **C: Hack List** | Each hack re-earns the swipe independently |
| A strong opinion about a common practice | **D: Rant Callout** | Conviction is the content; structure keeps it fair |
| A product or workflow you can show | **E: Demo Walkthrough** | Seeing the steps is more persuasive than describing them |

Two cross-framework rules before the specifics:

- **Slide 1 is the thumbnail.** It competes in the feed alone, before anyone knows a carousel follows. Design and write it as a standalone scroll-stopper.
- **One visual template per carousel.** Same layout, type scale, and palette on every interior slide. Variety between slides reads as clutter; consistency lets the *content* change while the frame stays still.

---

## Framework A: Value-Stack (4–14 slides)

The "everything you need" carousel. Works because the cover makes a completeness claim — an exact count and an exact deliverable — and every swipe pays it down.

| Slide | Job | Pattern |
|---|---|---|
| 1 — Cover | State the exact count + exact deliverable. Specificity proves scale upfront. | "[N] [resource type] for [role/outcome]" |
| 2 to N−1 — Value delivery | One item or category per slide, 3–6 concrete sub-items each. Same visual template, zero filler slides. | "[Category/item] — [sub-items]" |
| N — Close | Convert the swipe-through into an action. | "[Action] for [the payoff]" — comment a keyword, follow, link in bio |

**Why the exact count matters:** "27 free tools" outperforms "the best free tools" because a number is a checkable promise — the reader can verify you delivered. Rounding up with padding breaks the trust the count created.

*SaaS example:* "12 ChatGPT prompts for SaaS onboarding emails" → one prompt per slide with the use case → "Comment PROMPTS and I'll send the full doc."

**Failure mode:** filler slides to hit a bigger number. Cut to the real count; a tight 8 beats a padded 14.

---

## Framework B: Problem-Proof (6–10 slides)

The "I did X, here's the system" carousel. The hook is a *result stated as fact*, not advice — and the final slide shows the receipt, closing the loop slide 1 opened.

| Slide | Job | Pattern |
|---|---|---|
| 1 — Hook | A specific personal claim with a number. A result, not a tip. | "[I/we did X]. [Specific result, with number]." |
| 2 — Reframe the problem | Name what's *actually* going wrong so the reader recognizes themselves. | "The real problem — [named issue]" |
| 3–4 — The mechanism | Show the real system: named tools, named steps. Concrete beats vague. | "The system — [named tools/steps]" |
| 5 to N−1 — The detail | The literal prompt text, template, or step-by-step. This is the save-worthy part. | "[The literal prompt or process detail]" |
| N — Proof | The actual output artifact. A screenshot, not a description. | "[Screenshot of output] — the receipt" |

**The open loop is the engine:** slide 1 makes a claim, the last slide proves it, and everything between explains how. Readers swipe to see whether the receipt is real.

*SaaS example:* "We cut churn 22% with one onboarding email." → the real problem (activation, not price) → the sequence structure → the literal email copy → the retention chart.

**Failure mode:** vague mechanism slides ("optimize your workflow") — if the middle slides don't name tools and steps, the proof slide reads as luck, not system.

---

## Framework C: Hack List (6–10 slides)

The "numbered techniques" carousel. A contrarian cover creates status anxiety — most people are doing this wrong — and each numbered hack independently re-earns the swipe.

| Slide | Job | Pattern |
|---|---|---|
| 1 — Contrarian hook | A stat or claim implying most people fail at this. | "[Stat implying most people fail at X]" |
| 2 — The problem | Why the common approach fails, ideally with an analogy. | "The problem — [why it fails]" |
| 3 to N−1 — Numbered hacks | One named technique per slide. Bad-vs-good contrast makes each hack land instantly. | "Hack #[n] — [named technique]" |
| N — Synthesis + close | A one-line thesis tying the hacks together, then a save/share/follow CTA. | "[Thesis tying hacks together]" + [CTA] |

**Name every technique.** "The 3-second rule" is shareable and memorable; "keep it short" is neither. Named techniques travel — people repeat them and credit the source.

*SaaS example:* "90% of trial emails never get opened." → why (they read like receipts, not messages) → Hack #1: The founder-from line, Hack #2: The one-question subject, ... → "Trial emails are conversations, not confirmations. Save this for your next sequence."

**Failure mode:** hacks that are restatements of each other. Each slide must survive alone — if two hacks collapse into one idea, merge them and go shorter.

---

## Framework D: Rant Callout (4–8 slides)

The "someone had to say it" carousel. Personality-led and polarizing by design — the structure exists to keep the heat *fair* so it reads as conviction, not bitterness.

| Slide | Job | Pattern |
|---|---|---|
| 1 — Provocative claim | An unpopular opinion or direct accusation about a common practice. | "Unpopular opinion — [common practice]" |
| 2 to N−2 — Escalate the argument | Sensory, specific detail. Show the offense; don't gesture at it abstractly. Each slide raises the stakes. | "[Escalating, specific complaint]" |
| N−1 — Fairness pivot | "Don't get me wrong…" — clarify what you're *not* attacking. Anti-laziness, not anti-tool; anti-practice, not anti-person. | "The problem isn't [X], it's [Y]" |
| N — Close | Firm, personality-forward sign-off. Signed rants read as owned opinions; anonymous ones read as potshots. | "[Firm sign-off] — signed [name/persona]" |

**The fairness pivot is what makes it work.** Without it you're yelling; with it you're drawing a precise line, and precise lines get quoted. It also pre-empts the top hostile comment.

*SaaS example:* "Your AI-generated LinkedIn posts are costing you customers." → the specifics (same em-dash cadence, same 'game-changer' vocabulary, zero lived detail) → "Don't get me wrong — I use AI daily. The problem isn't the tool, it's publishing the first draft." → "Write like you talk. — [Name]"

**Failure modes:** skipping the pivot (reads unhinged, invites pile-ons), or ranting about something your own product/content visibly does (the comments will find it).

---

## Framework E: Demo Walkthrough (5–11 slides)

The "watch it work" carousel for product or workflow content. Show the finished result first, then the steps — the outcome earns attention for the process.

| Slide | Job | Pattern |
|---|---|---|
| 1 — Product + outcome | Brand/product plus the finished result. Lead with what they get. | "[Product] + [outcome], finished result" |
| 2 — Problem it solves | The specific frustration this replaces. | "[The specific pain this replaces]" |
| 3 — Process overview | The numbered step list, shown *before* the detail. Seeing the whole path lowers mid-swipe drop-off — readers commit when they know how long the road is. | "[N] steps to [outcome] in [timeframe]" |
| 4 to N−1 — Step-by-step | One real UI screenshot per step with a short caption. Real screenshots; mockups read as vaporware. | "Step [n] — [screenshot] — [caption]" |
| N — Result + positioning | The full-size output plus one brand-philosophy line. | "[Final output] + [positioning line]" |

**Overview-before-detail is the key move** (same reason recipe sites list ingredients before instructions): a reader who's seen "4 steps, 10 minutes" on slide 3 will finish; a reader dropped straight into step 1 with no map bails at step 2.

*SaaS example:* "From CSV to live dashboard in 4 steps" → the pain (weekly manual reporting) → the 4-step overview → one screenshot per step → the finished dashboard + "Built for marketers, not analysts."

**Failure mode:** screenshots with too much UI. Crop to the action, zoom the relevant control, annotate with one arrow — each screenshot should make sense at thumbnail size.

---

## Platform Notes

**Instagram:** 1080×1350 (4:5) slides; up to 20 slides but the frameworks' ranges above reflect where completion rates hold. The caption carries the searchable text — restate the hook and add context, don't just repeat slide 1. First slide doubles as the grid thumbnail.

**LinkedIn (document posts):** Upload as PDF. Square or 4:5. LinkedIn shows a "N pages" badge — Value-Stack's exact-count cover plays especially well here. B2B tolerates denser text per slide than Instagram, but the one-idea-per-slide rule still holds. The post text above the document is a second hook — write it as its own post, not a throwaway "see attached."

**TikTok photo mode:** Same frameworks work; pacing skews faster — trim each framework toward its minimum slide count.

## Production Checklist

- [ ] Slide 1 works as a standalone feed post (test: would it stop the scroll with no carousel behind it?)
- [ ] One idea per slide — if a slide needs two sentences of setup, split it
- [ ] Same visual template on every interior slide
- [ ] Any count promised on the cover is exactly delivered (no filler, no rounding)
- [ ] Claims and numbers are real and yours — no invented stats or borrowed screenshots
- [ ] Text is legible at thumbnail size (minimum ~28pt at 1080px wide)
- [ ] Final slide has one CTA, not three
- [ ] Caption/post text written as its own hook, not a repeat of slide 1

## Measuring What Worked

Judge carousels on **saves and completion**, not likes — saves signal reference value (the algorithmic win for carousels), and where per-slide analytics exist (Instagram professional insights, LinkedIn document analytics, or third-party tools), drop-off position tells you which slide broke the swipe chain. When a carousel underperforms, the fix is usually slide 1 (didn't stop the scroll) or the framework choice (list content forced into a rant structure), not the middle slides. Track per-framework performance and double down on the two that fit your audience — see [reverse-engineering.md](reverse-engineering.md) for the analysis workflow.

FILE:references/listening-sources-template.md
# Listening Sources — Template

Copy this file to `.agents/listening-sources.md` in your project (or `.claude/listening-sources.md`) and fill in the brackets. Claude reads it when running the [listening workflow](listening.md).

Delete sections you don't use. Keep this short and current — stale sources are worse than no sources.

---

## What We're Listening For

**Brand / product:** [Your product name]
**Category:** [e.g., "AI writing assistant", "Postgres GUI"]
**Goal:** [e.g., "find people switching from Notion", "engage with B2B SaaS founders 50-200 employees"]

## ICP (for scoring)

Used by the [scoring rubric](listening.md#scoring-rubric) to judge ICP fit.

- **Role:** [e.g., "founder, head of marketing, marketing ops lead"]
- **Company stage:** [e.g., "seed to Series B SaaS, 10-200 employees"]
- **Industry:** [e.g., "B2B SaaS, infra, devtools"]
- **Signals they're a fit:** [e.g., "writes about GTM, runs paid ads, recently raised"]

---

## Target Accounts

Engage with **every** post from these accounts when relevant. Keep this list to 20-50 max.

### LinkedIn (browser-driven — use dev-browser to view feed)
- [Name] — `linkedin.com/in/handle`
- [Name] — `linkedin.com/in/handle`

### X / Twitter (browser-driven)
- [@handle]
- [@handle]

### Reddit
- u/[username]
- u/[username]

### Bluesky
- [handle.bsky.social]

### Blogs / Newsletters (RSS)
- [Name] — `https://example.com/feed/`
- [Name] — `https://example.substack.com/feed`

### YouTube channels (RSS)
- [Name] — channel ID `UCxxxxxxxx`

---

## Keywords (intent signals)

Search across all platforms. Claude runs these through Reddit, HN, Bluesky on the [daily loop](listening.md#the-daily-triage-loop).

### High-intent (someone shopping or switching)
- `"alternative to [competitor]"`
- `"looking for a [category] tool"`
- `"recommend a [category]"`
- `"switching from [competitor]"`
- `"frustrated with [competitor]"`

### Problem signals (someone in pain)
- `"[category] is so [bad/hard/expensive]"`
- `"why is [category] [problem]"`
- `"hate [pain point]"`

### Brand mentions
- `"[your brand]"`
- `"[your brand misspelling]"`
- `"[your domain]"`

### Competitor mentions (monitor for switching language)
- `"[competitor 1]"`
- `"[competitor 2]"`

---

## Subreddits

Pulled via Reddit JSON API on the daily loop.

- r/SaaS
- r/Entrepreneur
- r/[your niche, e.g., "marketing", "devtools"]
- r/[adjacent community]

---

## Saved Searches (manual / browser-driven)

URLs Claude opens via dev-browser to scan.

### LinkedIn Sales Navigator
- [Search name] — `https://linkedin.com/sales/search/people?...`

### LinkedIn (regular)
- Posts hashtag — `https://linkedin.com/feed/hashtag/yourtopic/`

### X advanced search
- [Search name] — `https://x.com/search?q=...&f=live`

---

## Do Not Engage

Save yourself the regret.

- Accounts known for bad-faith dunking: [@handle], [@handle]
- Blocked brands / competitors who'll screenshot: [list]
- Topics to avoid: [politics, [your founder's hot takes], etc.]

---

## Notes for Claude

- When asked for "today's top 10," output in the format defined in [listening.md](listening.md#the-daily-triage-loop)
- For LinkedIn and X, use dev-browser with the persistent session (user is logged in)
- For everything else, use the curl recipes in [listening.md](listening.md#sources--light-tooling-curl-recipes)
- Default lookback: 24h. User can override.
- Always ask before posting — output drafts, user approves and posts manually

FILE:references/listening.md
# Social Listening & Engagement Triage

How to surface the right posts to engage with each day — instead of randomly scrolling. The goal is a short, scorable list ("here are your top 10 posts to comment on") rather than an open feed.

## Contents
- When to use this
- The daily triage loop
- Scoring rubric
- Comment quality tiers
- Sources & light tooling (curl recipes)
- Per-platform notes
- Common workflows

---

## When to Use This

Use listening when the goal is **commenting and relationships**, not posting. Typical asks:
- "Give me the top 10 posts I should comment on today"
- "Who's complaining about [competitor] right now?"
- "Find people asking for a tool like mine"
- "Surface posts from my 20 target accounts in the last 24h"
- "What's the conversation around [topic] this week?"

If the user wants to **create** content, use the rest of the social skill. Listening feeds creation (it surfaces angles, language, objections), but the output is different.

---

## The Daily Triage Loop

A repeatable 20-minute loop the user (or you, on their behalf) can run each morning.

1. **Pull** — fetch new posts from defined sources (target accounts, keywords, subreddits, hashtags). See [tooling](#sources--light-tooling-curl-recipes).
2. **Filter** — drop anything older than 24h, low signal, or off-topic.
3. **Score** — apply the [rubric](#scoring-rubric). Keep top 10.
4. **Draft** — for each, draft a comment matched to the post's tier.
5. **Post** — user reviews, edits, posts. Mark which actually went live.
6. **Log** — track what you commented on and what got replies. This is your engagement loop dataset.

Output format Claude should produce:

```
TOP 10 POSTS — 2026-06-05

1. [Score 9/10] @author — LinkedIn — 2h ago
   "We just rolled out X and the team is loving it…"
   Why: ICP fit (B2B SaaS, 50–200 employees), buying-intent signal
   Suggested comment: [draft]
   Link: https://…
```

---

## Scoring Rubric

Score each post 1–10 across five dimensions, then sum and rank.

| Dimension | What it measures | Weight |
|-----------|------------------|--------|
| **ICP fit** | Is the author your target customer / influencer? | 2x |
| **Intent signal** | Are they expressing a problem, asking, or shopping? | 2x |
| **Reach potential** | Is the post getting traction (likes/comments rising)? | 1x |
| **Comment opportunity** | Can you say something genuinely useful, not generic? | 2x |
| **Recency** | Posted in last 1–4h (early comments win, especially on LinkedIn) | 1x |

**Intent signal examples (high-value):**
- "Looking for a tool that does X"
- "Why is [category] so painful?"
- "We just switched from [competitor] because…"
- "Anyone use [competitor] — is it worth it?"
- A complaint about a known competitor

**Drop if any of these are true:**
- Author isn't ICP and isn't an influencer
- Post is >24h old and already has 50+ comments (your comment buries)
- Generic motivational/AI-slop post
- Self-promotion thread where comments don't get reach
- You can't add anything beyond "Great post!"

---

## Comment Quality Tiers

Match the comment to the post. Don't waste a tier-1 draft on a tier-3 opportunity.

**Tier 1 — Relationship builder (target accounts, ICP, high intent)**
- Add a specific insight or counter-example
- Reference your own experience with specifics (numbers, names, outcomes)
- Ask a thoughtful follow-up that invites a reply
- Length: 2–4 sentences, no link

**Tier 2 — Visibility play (high-reach post, adjacent topic)**
- Add one sharp insight in one sentence
- Pattern: "Agreed — and the part most miss is [X]"
- Length: 1–2 sentences

**Tier 3 — Light touch (relationship maintenance)**
- Specific reaction, not "Love this"
- Quote a specific line and react to it
- Length: 1 sentence

**Never:** "Great post!", emoji-only, "+1", LinkedIn-isms like "This is gold 🔥"

---

## Sources & Light Tooling (curl recipes)

These are public JSON endpoints — no auth needed. Run them from bash, pipe to `jq`, and Claude can parse the output to score and draft comments.

**Requires:** `jq` (most recipes) and `xmllint` (RSS only). Install once:
```bash
# macOS
brew install jq
# xmllint ships with macOS; on Linux: apt install libxml2-utils
```

### Reddit (free, scriptable)

**New posts in a subreddit:**
```bash
curl -s -A "listening/1.0" \
  "https://www.reddit.com/r/SaaS/new.json?limit=25" \
  | jq '.data.children[].data | {title, author, url: ("https://reddit.com"+.permalink), score, num_comments, created_utc, selftext: (.selftext | .[0:300])}'
```

**Search across Reddit by keyword (last day, sorted new):**
```bash
curl -s -A "listening/1.0" \
  "https://www.reddit.com/search.json?q=KEYWORD&sort=new&t=day&limit=25" \
  | jq '.data.children[].data | {subreddit, title, url: ("https://reddit.com"+.permalink), author, score, created_utc}'
```

Swap `KEYWORD` for things like `"alternative to notion"`, `"recommend a crm"`, your competitor names, or your own brand for mentions. Use quotes around multi-word phrases.

### Hacker News (Algolia search)

**Recent stories mentioning a keyword (last 24h):**
```bash
SINCE=$(($(date +%s) - 86400))
curl -s "https://hn.algolia.com/api/v1/search_by_date?query=KEYWORD&tags=story&numericFilters=created_at_i>SINCE" \
  | jq '.hits[] | {title, url, author, points, num_comments, created_at, story_id: .objectID, hn_url: ("https://news.ycombinator.com/item?id="+.objectID)}'
```

**Recent comments mentioning a keyword:**
```bash
curl -s "https://hn.algolia.com/api/v1/search_by_date?query=KEYWORD&tags=comment&numericFilters=created_at_i>SINCE" \
  | jq '.hits[] | {author, comment_text, story_title, hn_url: ("https://news.ycombinator.com/item?id="+.objectID)}'
```

### Bluesky (free, public API)

**Search posts by keyword:**
```bash
curl -s "https://public.api.bsky.app/xrpc/app.bsky.feed.searchPosts?q=KEYWORD&limit=25&sort=latest" \
  | jq '.posts[] | {author: .author.handle, text: .record.text, likes: .likeCount, replies: .replyCount, url: ("https://bsky.app/profile/"+.author.handle+"/post/"+(.uri | split("/") | last))}'
```

### RSS for blogs, podcasts, YouTube channels

For target accounts that publish to RSS (most blogs, all YouTube channels):
```bash
# YouTube channel feed (replace CHANNEL_ID)
curl -s "https://www.youtube.com/feeds/videos.xml?channel_id=CHANNEL_ID"

# Generic blog feed
curl -s "https://example.com/feed/" | xmllint --xpath "//item[position()<6]" - 2>/dev/null
```

### LinkedIn & X — use the browser

LinkedIn and X don't expose useful public APIs, but you can drive a real browser session. **dev-browser** (MCP, already in the global setup) and **Playwright** both maintain persistent state — log in once, the session stays alive, Claude can navigate the authenticated feed.

**dev-browser workflow (preferred — already wired up):**
1. User logs into LinkedIn / X once in the dev-browser session
2. Claude navigates to a target URL (feed, profile, saved search, hashtag)
3. Claude reads the accessibility tree / page text, extracts posts
4. Claude scores using the [rubric](#scoring-rubric) and drafts comments
5. User reviews and posts manually (don't auto-post — high-stakes, bot detection risk)

**Useful URLs to feed dev-browser:**

| URL pattern | What it shows |
|-------------|---------------|
| `linkedin.com/in/HANDLE/recent-activity/all/` | A target account's recent posts |
| `linkedin.com/feed/hashtag/TOPIC/` | Hashtag feed |
| `linkedin.com/feed/` | Your main feed (algorithmic — less useful for triage) |
| `x.com/HANDLE` | A target account's profile |
| `x.com/search?q=QUERY&f=live` | Real-time search (use `f=live` for chronological) |
| `x.com/i/lists/LIST_ID` | A curated list — best for target accounts |

**Tips:**
- On X, build a private list of target accounts and use the list URL. Far cleaner than the algorithmic feed.
- LinkedIn's `/recent-activity/all/` URL is the cleanest way to see one person's posts without the algorithm.
- For both platforms, scroll programmatically (dev-browser supports it) to load more posts before extracting.

**Paid alternatives if you don't want to drive a browser:**

| Platform | Tools |
|----------|-------|
| LinkedIn | Sales Navigator (saved searches), Taplio (engagement) |
| X | TweetDeck/X Pro (saved columns), Typefully, Taplio, Tweet Hunter |

**Still closed (no good path):**
- Instagram & TikTok — closed APIs, browser automation is detectable and risky. Use native saved searches / hashtag follows.

---

## Per-Platform Notes

### LinkedIn
- **Browser-driven** (dev-browser with persistent session) — see [LinkedIn & X — use the browser](#linkedin--x--use-the-browser)
- **First-hour comments matter most** — algorithm weights early engagement heavily. Prioritize posts <2h old from target accounts.
- Comments with 5+ words get more reach than reactions
- Replying to other commenters can put you in front of their network
- Tag the author in your reply only if it adds context

### Twitter/X
- **Browser-driven** (dev-browser) — build a private list of target accounts and point dev-browser at the list URL
- Reply within first 30 min for max reach on big accounts
- Quote-tweet > reply when adding substantial value
- Threading your reply (multi-tweet) signals effort
- Don't pile on dunks — relationships > clout

### Reddit
- Read the subreddit rules before commenting (some ban self-promotion outright)
- Earn karma in the sub before linking to anything you own
- Long, specific answers win. AMAs and "help me decide" threads are gold
- Never lead with your product — answer the question first

### Hacker News
- Comment quality bar is high; low-effort gets downvoted fast
- Founders commenting on threads about their product is welcomed if you're transparent
- Search for past discussions of your category — they're often dormant gold mines

### Bluesky
- Smaller volume but high engagement-to-follower ratio
- Tech and indie-hacker communities are active
- Custom feeds (like Bluesky's "Following" + topic feeds) replace algorithmic search

---

## Common Workflows

### "Give me my top 10 posts to comment on today"
1. Pull from: target account RSS/saved searches + Reddit (relevant subs) + HN (last 24h)
2. Score with the [rubric](#scoring-rubric)
3. Output top 10 with suggested comments

### "Find people complaining about [competitor]"
1. Reddit search: `"competitor name" -site:competitor.com` sorted by new
2. HN comment search for competitor name
3. Bluesky search for competitor handle/name
4. Score by intent signal (high if switching language: "moving from", "alternatives to", "frustrated with")

### "Surface brand mentions from the last week"
1. Reddit search for brand name
2. HN search (stories + comments) for brand name
3. Bluesky search for brand name + handle
4. Output as: reply needed (yes/no), tone (positive/negative/neutral), suggested response

### "Find target-account posts I missed"
1. Maintain a list of target accounts with their RSS / Reddit usernames / Bluesky handles
2. Fetch each source's recent posts
3. Filter to last 24h, output sorted by score

---

## Setting Up the Source List

The user should maintain a list of sources somewhere persistent at `.agents/listening-sources.md` (or `.claude/listening-sources.md`). Claude reads it when running the daily loop.

**A ready-to-fill template lives at [listening-sources-template.md](listening-sources-template.md).** Copy it into the project and edit. The source path depends on how the skill was installed:

```bash
# Plugin / marketplace install (most common):
cp .agents/skills/social/references/listening-sources-template.md .agents/listening-sources.md
# .claude/ install:
cp .claude/skills/social/references/listening-sources-template.md .agents/listening-sources.md
# Working inside the marketingskills repo:
cp skills/social/references/listening-sources-template.md .agents/listening-sources.md
```

The template covers: brand/category, ICP (for scoring), target accounts per platform, intent keywords, subreddits, saved-search URLs, and a do-not-engage list.

FILE:references/platform-limits.md
# Platform Limits Reference

Quick reference for hashtag limits, character counts, and visible text thresholds on each major social platform.

---

## Instagram

| Element | Limit |
|---------|-------|
| Max hashtags | 5 (official limit) |
| Recommended hashtags | 3 – 5 |
| Max caption chars | 2,200 |
| Visible before "more" | ~125 chars |

---

## Facebook

| Element | Limit |
|---------|-------|
| Max hashtags | No official limit |
| Recommended hashtags | 1 – 2 |
| Max post chars | 63,206 |
| Ideal for engagement | 40 – 80 chars |

---

## TikTok

| Element | Limit |
|---------|-------|
| Max hashtags | 5 (since August 2025) |
| Recommended hashtags | 3 – 5 |
| Max caption chars | 4,000 |
| Visible before "more" | ~150 chars |

---

## LinkedIn

| Element | Limit |
|---------|-------|
| Max hashtags | No official limit |
| Recommended hashtags | 3 – 5 |
| Max post chars | 3,000 |
| Visible before "more" | ~210 chars |

---

## Twitter/X

| Element | Limit |
|---------|-------|
| Max hashtags | No official limit |
| Recommended hashtags | 1 – 2 |
| Max tweet chars | 280 (standard) / 25,000 (Premium+) |
| Visible before "more" | Full tweet (280 standard) |

---

## YouTube

| Element | Limit |
|---------|-------|
| Max hashtags | 15 (exceeding this causes YouTube to ignore ALL hashtags) |
| Recommended hashtags | 3 – 5 |
| Max title chars | 100 (visible before truncation: ~70) |
| Max description chars | 5,000 |
| Visible before "Show more" | ~100 chars |

> The first 3 hashtags in the description automatically appear above the title as clickable links. For Shorts, use 1 – 5 hashtags.

---

## Pinterest

| Element | Limit |
|---------|-------|
| Max hashtags | 20 per pin |
| Recommended hashtags | 2 – 5 |
| Max pin title chars | 100 |
| Max description chars | 500 |
| Visible before "More" | ~50 chars (desktop) |

> Pinterest has deprioritized hashtags. Focus on keywords as natural sentences within the description for better SEO instead of relying on hashtags.

---

## Threads (Meta)

| Element | Limit |
|---------|-------|
| Max hashtags | 1 per post (topic tag) |
| Recommended hashtags | 1 |
| Max post chars | 500 |
| Max with text attachment | 10,500 (500 + 10,000 expandable) |
| Visible without expanding | First ~1 – 2 lines |

> Threads limits topic tags to one per post. The platform is not hashtag-driven — the algorithm prioritizes content from followed accounts mixed with recommendations.

---

## Usage Tips

- **Hashtags count against character limits** on all platforms
- **Front-load your message** before the "more" truncation point
- On Instagram and TikTok, fewer hashtags now outperform hashtag-stuffing
- On LinkedIn, hashtags at the end of the post perform better than inline
- On Facebook, hashtags have minimal impact on reach — use sparingly

FILE:references/platforms.md
# Platform-Specific Strategy Guide

Detailed strategies for each major social platform.

## Contents
- LinkedIn
- Twitter/X
- Instagram
- TikTok
- Facebook

## LinkedIn

**Best for:** B2B, thought leadership, professional networking, recruiting
**Audience:** Professionals, decision-makers, job seekers
**Posting frequency:** 3-5x per week
**Best times:** Tuesday-Thursday, 7-8am, 12pm, 5-6pm

**What works:**
- Personal stories with business lessons
- Contrarian takes on industry topics
- Behind-the-scenes of building a company
- Data and original insights
- Carousel posts (document format)
- Polls that spark discussion

**What doesn't:**
- Overly promotional content
- Generic motivational quotes
- Links in the main post (kills reach)
- Corporate speak without personality

**Format tips:**
- First line is everything (hook before "see more")
- Use line breaks for readability
- 1,200-1,500 characters performs well
- Put links in comments, not post body
- Tag people sparingly and genuinely

**Algorithm tips:**
- First hour engagement matters most
- Comments > reactions > clicks
- Dwell time (people reading) signals quality
- No external links in post body
- Document posts (carousels) get strong reach
- Polls drive engagement but don't build authority

---

## Twitter/X

**Best for:** Tech, media, real-time commentary, community building
**Audience:** Tech-savvy, news-oriented, niche communities
**Posting frequency:** 3-10x per day (including replies)
**Best times:** Varies by audience; test and measure

**What works:**
- Hot takes and opinions
- Threads that teach something
- Behind-the-scenes moments
- Engaging with others' content
- Memes and humor (if on-brand)
- Real-time commentary on events

**What doesn't:**
- Pure self-promotion
- Threads without a strong hook
- Ignoring replies and mentions
- Scheduling everything (no real-time presence)

**Format tips:**
- Tweets under 100 characters get more engagement
- Threads: Hook in tweet 1, promise value, deliver
- Quote tweets with added insight beat plain retweets
- Use visuals to stop the scroll

**Algorithm tips:**
- Replies and quote tweets build authority
- Threads keep people on platform (rewarded)
- Images and video get more reach
- Engagement in first 30 min matters
- Twitter Blue/Premium may boost reach

---

## Instagram

**Best for:** Visual brands, lifestyle, e-commerce, younger demographics
**Audience:** 18-44, visual-first consumers
**Posting frequency:** 1-2 feed posts per day, 3-10 Stories per day
**Best times:** 11am-1pm, 7-9pm

**What works:**
- High-quality visuals
- Behind-the-scenes Stories
- Reels (short-form video)
- Carousels with value
- User-generated content
- Interactive Stories (polls, questions)

**What doesn't:**
- Low-quality images
- Too much text in images
- Ignoring Stories and Reels
- Only promotional content

**Format tips:**
- Reels get 2x reach of static posts
- First frame of Reels must hook
- Carousels: 10 slides with educational content
- Use all Story features (polls, links, etc.)

**Algorithm tips:**
- Reels heavily prioritized over static posts
- Saves and shares > likes
- Stories keep you top of feed
- Consistency matters more than perfection
- Use all features (polls, questions, etc.)

---

## TikTok

**Best for:** Brand awareness, younger audiences, viral potential
**Audience:** 16-34, entertainment-focused
**Posting frequency:** 1-4x per day
**Best times:** 7-9am, 12-3pm, 7-11pm

**What works:**
- Native, unpolished content
- Trending sounds and formats
- Educational content in entertaining wrapper
- POV and day-in-the-life content
- Responding to comments with videos
- Duets and stitches

**What doesn't:**
- Overly produced content
- Ignoring trends
- Hard selling
- Repurposed horizontal video

**Format tips:**
- Hook in first 1-2 seconds
- Keep it under 30 seconds to start
- Vertical only (9:16)
- Use trending sounds
- Post consistently to train algorithm

---

## Facebook

**Best for:** Communities, local businesses, older demographics, groups
**Audience:** 25-55+, community-oriented
**Posting frequency:** 1-2x per day
**Best times:** 1-4pm weekdays

**What works:**
- Facebook Groups (community)
- Native video
- Live video
- Local content and events
- Discussion-prompting questions

**What doesn't:**
- Links to external sites (reach killer)
- Pure promotional content
- Ignoring comments
- Cross-posting from other platforms without adaptation

FILE:references/post-templates.md
# Post Format Templates

Ready-to-use templates for different platforms and content types.

## Contents
- LinkedIn Post Templates (The Story Post, The Contrarian Take, The List Post, The How-To)
- Twitter/X Thread Templates (The Tutorial Thread, The Story Thread, The Breakdown Thread)
- Instagram Templates (The Carousel Hook, The Reel Script)
- Hook Formulas (Curiosity Hooks, Story Hooks, Value Hooks, Contrarian Hooks, Social Proof Hooks)

## LinkedIn Post Templates

### The Story Post
```
[Hook: Unexpected outcome or lesson]

[Set the scene: When/where this happened]

[The challenge you faced]

[What you tried / what happened]

[The turning point]

[The result]

[The lesson for readers]

[Question to prompt engagement]
```

### The Contrarian Take
```
[Unpopular opinion stated boldly]

Here's why:

[Reason 1]
[Reason 2]
[Reason 3]

[What you recommend instead]

[Invite discussion: "Am I wrong?"]
```

### The List Post
```
[X things I learned about [topic] after [credibility builder]:

1. [Point] — [Brief explanation]

2. [Point] — [Brief explanation]

3. [Point] — [Brief explanation]

[Wrap-up insight]

Which resonates most with you?
```

### The How-To
```
How to [achieve outcome] in [timeframe]:

Step 1: [Action]
↳ [Why this matters]

Step 2: [Action]
↳ [Key detail]

Step 3: [Action]
↳ [Common mistake to avoid]

[Result you can expect]

[CTA or question]
```

---

## Twitter/X Thread Templates

### The Tutorial Thread
```
Tweet 1: [Hook + promise of value]

"Here's exactly how to [outcome] (step-by-step):"

Tweet 2-7: [One step per tweet with details]

Final tweet: [Summary + CTA]

"If this was helpful, follow me for more on [topic]"
```

### The Story Thread
```
Tweet 1: [Intriguing hook]

"[Time] ago, [unexpected thing happened]. Here's the full story:"

Tweet 2-6: [Story beats, building tension]

Tweet 7: [Resolution and lesson]

Final tweet: [Takeaway + engagement ask]
```

### The Breakdown Thread
```
Tweet 1: [Company/person] just [did thing].

Here's why it's genius (and what you can learn):

Tweet 2-6: [Analysis points]

Tweet 7: [Your key takeaway]

"[Related insight + follow CTA]"
```

---

## Instagram Templates

### The Carousel Hook
```
[Slide 1: Bold statement or question]
[Slides 2-9: One point per slide, visual + text]
[Slide 10: Summary + CTA]

Caption: [Expand on the topic, add context, include CTA]
```

This is the generic shape. For five full slide-by-slide narrative architectures (Value-Stack, Problem-Proof, Hack List, Rant Callout, Demo Walkthrough) with selection guidance and per-slide copy slots, see [carousel-frameworks.md](carousel-frameworks.md).

### The Reel Script
```
Hook (0-2 sec): [Pattern interrupt or bold claim]
Setup (2-5 sec): [Context for the tip]
Value (5-25 sec): [The actual advice/content]
CTA (25-30 sec): [Follow, comment, share, link]
```

---

## Hook Formulas

The first line determines whether anyone reads the rest.

### Curiosity Hooks
- "I was wrong about [common belief]."
- "The real reason [outcome] happens isn't what you think."
- "[Impressive result] — and it only took [surprisingly short time]."
- "Nobody talks about [insider knowledge]."

### Story Hooks
- "Last week, [unexpected thing] happened."
- "I almost [big mistake/failure]."
- "3 years ago, I [past state]. Today, [current state]."
- "[Person] told me something I'll never forget."

### Value Hooks
- "How to [desirable outcome] (without [common pain]):"
- "[Number] [things] that [outcome]:"
- "The simplest way to [outcome]:"
- "Stop [common mistake]. Do this instead:"

### Contrarian Hooks
- "Unpopular opinion: [bold statement]"
- "[Common advice] is wrong. Here's why:"
- "I stopped [common practice] and [positive result]."
- "Everyone says [X]. The truth is [Y]."

### Social Proof Hooks
- "We [achieved result] in [timeframe]. Here's the full story:"
- "[Number] people asked me about [topic]. Here's my answer:"
- "[Authority figure] taught me [lesson]."

FILE:references/reverse-engineering.md
# Reverse Engineering Viral Content

Instead of guessing what works, systematically analyze top-performing content in your niche and extract proven patterns.

## Contents
- The 6-Step Framework (Niche ID, Scrape, Analyze, Playbook, Layer Voice, Convert)
- The Formula
- Reverse Engineering Checklist

## The 6-Step Framework

### 1. NICHE ID — Find Top Creators

Identify 10-20 creators in your space who consistently get high engagement:

**Selection criteria:**
- Posting consistently (3+ times/week)
- High engagement rate relative to follower count
- Audience overlap with your target market
- Mix of established and rising creators

**Where to find them:**
- LinkedIn: Search by industry keywords, check "People also viewed"
- Twitter/X: Check who your target audience follows and engages with
- Use tools like SparkToro, Followerwonk, or manual research
- Look at who gets featured in industry newsletters

### 2. SCRAPE — Collect Posts at Scale

Gather 500-1000+ posts from your identified creators for analysis:

**Tools:**
- **Apify** — LinkedIn scraper, Twitter scraper actors
- **Phantom Buster** — Multi-platform automation
- **Export tools** — Platform-specific export features
- **Manual collection** — For smaller datasets, copy/paste into spreadsheet

**Data to collect:**
- Post text/content
- Engagement metrics (likes, comments, shares, saves)
- Post format (text-only, carousel, video, image)
- Posting time/day
- Hook/first line
- CTA used
- Topic/theme

### 3. ANALYZE — Extract What Actually Works

Sort and analyze the data to find patterns:

**Quantitative analysis:**
- Rank posts by engagement rate
- Identify top 10% performers
- Look for format patterns (do carousels outperform?)
- Check timing patterns (best days/times)
- Compare topic performance

**Qualitative analysis:**
- What hooks do top posts use?
- How long are high-performing posts?
- What emotional triggers appear?
- What formats repeat?
- What topics consistently perform?

**Questions to answer:**
- What's the average length of top posts?
- Which hook types appear most in top 10%?
- What CTAs drive most comments?
- What topics get saved/shared most?

### 4. PLAYBOOK — Codify Patterns

Document repeatable patterns you can use:

**Hook patterns to codify:**
```
Pattern: "I [unexpected action] and [surprising result]"
Example: "I stopped posting daily and my engagement doubled"
Why it works: Curiosity gap + contrarian

Pattern: "[Specific number] [things] that [outcome]:"
Example: "7 pricing mistakes that cost me $50K:"
Why it works: Specificity + loss aversion

Pattern: "[Controversial take]"
Example: "Cold outreach is dead."
Why it works: Pattern interrupt + invites debate
```

**Format patterns:**
- Carousel: Hook slide → Problem → Solution steps → CTA
- Thread: Hook → Promise → Deliver → Recap → CTA
- Story post: Hook → Setup → Conflict → Resolution → Lesson

**CTA patterns:**
- Question: "What would you add?"
- Agreement: "Agree or disagree?"
- Share: "Tag someone who needs this"
- Save: "Save this for later"

### 5. LAYER VOICE — Apply Direct Response Principles

Take proven patterns and make them yours with these voice principles:

**"Smart friend who figured something out"**
- Write like you're texting advice to a friend
- Share discoveries, not lectures
- Use "I found that..." not "You should..."
- Be helpful, not preachy

**Specific > Vague**
```
❌ "I made good revenue"
✅ "I made $47,329"

❌ "It took a while"
✅ "It took 47 days"

❌ "A lot of people"
✅ "2,847 people"
```

**Short. Breathe. Land.**
- One idea per sentence
- Use line breaks liberally
- Let important points stand alone
- Create rhythm: short, short, longer explanation

```
❌ "I spent three years building my business the wrong way before I finally realized that the key to success was focusing on fewer things and doing them exceptionally well."

✅ "I built wrong for 3 years.

Then I figured it out.

Focus on less.
Do it exceptionally well.

Everything changed."
```

**Write from emotion**
- Start with how you felt, not what you did
- Use emotional words: frustrated, excited, terrified, obsessed
- Show vulnerability when authentic
- Connect the feeling to the lesson

```
❌ "Here's what I learned about pricing"

✅ "I was terrified to raise my prices.

My hands were shaking when I sent the email.

Here's what happened..."
```

### 6. CONVERT — Turn Attention into Action

Bridge from engagement to business results:

**Soft conversions:**
- Newsletter signups in bio/comments
- Free resource offers in follow-up comments
- DM triggers ("Comment X and I'll send you...")
- Profile visits → optimized profile with clear CTA

**Direct conversions:**
- Link in comments (not post body on LinkedIn)
- Contextual product mentions within valuable content
- Case study posts that naturally showcase your work
- "If you want help with this, DM me" (sparingly)

---

## The Formula

```
1. Find what's already working (don't guess)
2. Extract the patterns (hooks, formats, CTAs)
3. Layer your authentic voice on top
4. Test and iterate based on your own data
```

## Reverse Engineering Checklist

- [ ] Identified 10-20 top creators in niche
- [ ] Collected 500+ posts for analysis
- [ ] Ranked by engagement rate
- [ ] Documented top 10 hook patterns
- [ ] Documented top 5 format patterns
- [ ] Documented top 5 CTA patterns
- [ ] Created voice guidelines (specificity, brevity, emotion)
- [ ] Built template library from patterns
- [ ] Set up tracking for your own content performance

FILE:references/short-form-video.md
# Short-Form Video: Hooks, Scripts & Strategy

Detailed reference for creating short-form video content on TikTok, Instagram Reels, and YouTube Shorts.

---

## Video Hook Library

### Curiosity Hooks (Best for engagement)

**The "Secret" Formula:**
- "The secret to [outcome] that nobody talks about"
- "I found the hidden feature in [product/platform] that changes everything"
- "I can't believe this actually works..."

**The Unexpected Discovery:**
- "I tried [thing] for 30 days and I was NOT expecting this"
- "This completely changed how I think about [topic]"
- "Nobody talks about this, but..."

**The Question:**
- "Why does nobody talk about this?"
- "Am I the only one who didn't know this?"
- "The reason [common thing] doesn't work is..."

### Value Hooks (Best for saves)

**The Promise:**
- "How to [achieve outcome] in [specific timeframe]"
- "[Number] [things] that will [benefit]"
- "Everything you need to know about [topic] in 60 seconds"

**The Hack/Shortcut:**
- "[Outcome] hack that actually works"
- "The [adjective] way to [outcome]"
- "If you're struggling with [problem], watch this"

**The Warning:**
- "Stop doing [common practice] — here's why"
- "[Number] mistakes that are killing your [results]"
- "Why [thing you think is good] is actually hurting you"

### Story Hooks (Best for watch time)

**The Transformation:**
- "3 months ago, I [bad state]. Today, I [good state]."
- "Here's how I went from [before] to [after]"
- "I used to think [old belief]. Then [event] changed everything."

**The Failure:**
- "I made a huge mistake with [topic]"
- "Here's why I stopped [common practice]"
- "I lost [something significant] because of this mistake"

**The Journey:**
- "So this just happened..."
- "This changed everything for me"
- "Let me tell you about the time I [interesting situation]"

### Controversial Hooks (Best for comments)

- "Unpopular opinion: [bold statement]"
- "[Common advice] is actually wrong"
- "I'm going to get hate for this, but..."
- "Most [audience] get this completely wrong"

---

## Scripting Template

```markdown
## Video: [Working Title]

**Platform:** TikTok / Reels / Shorts
**Length:** XX seconds
**Format:** [Talking head / Slideshow / Demo / Screen recording]

### Hook (0-3 sec)
- Visual: [What viewer sees]
- Audio: [What they hear]
- Text overlay: [On-screen text]

### Body (3-X sec)
- [Timestamp] - [What happens/what you say]
- [Timestamp] - [Next beat]
- [Continue...]

### CTA (final 3-5 sec)
- Verbal: [What you say]
- Text: [On-screen text]
- Action: [Follow, comment, link in bio, etc.]

### Production Notes
- Music/sound: [Trending sound or music choice]
- B-roll needed: [List any clips needed]
- Graphics: [Any text animations or overlays]
```

---

## Additional Video Structures

### The Story Arc (45-60 sec)

```
[0-3s]  Hook: Tease the outcome
[3-15s] Setup: Context and stakes
[15-45s] Journey: What happened
[45-55s] Resolution: The result
[55-60s] Lesson/CTA
```

Best for: Personal stories, case studies, testimonials

### The POV/Skit (15-30 sec)

```
[0-3s]  Setup: Text overlay sets the scene
[3-25s] Performance: Act out the relatable scenario
[25-30s] Punchline or twist
```

Best for: Relatable content, humor, niche communities

---

## Visual Patterns

### Talking Head
- Good lighting (ring light or window light)
- Eye contact with camera
- Hand gestures for emphasis
- Interesting background (bookshelf, plants, studio setup)

### Slideshow/Carousel Video
- Strong visual on each slide (2-4 seconds per slide)
- Text overlays with key points
- Consistent style/branding
- Voiceover or trending sound

### Screen Recording
- Zoom in on important areas
- Add cursor highlight or click animations
- Keep movements smooth and intentional
- Overlay your face in corner (optional but boosts engagement)

### B-Roll Heavy
- Show don't tell
- Quick cuts (1-3 seconds per shot)
- Match cuts to voiceover beats
- Mix wide, medium, and close-up shots

---

## Audio Strategy

### When to Use Trending Sounds
- Entertainment/lifestyle content where the sound fits your message
- When the trend is still rising (check platform trending pages)
- Don't use when it distracts from your message or is already declining

### When to Use Original Audio
- Educational content where you're speaking
- Storytimes and personal narratives
- Product demos and tutorials
- Building a recognizable brand voice

### Voiceover Tips
- Speak slightly faster than normal conversation
- Vary your tone — avoid monotone delivery
- Pause for emphasis on key points
- Record in a quiet space, use noise removal
- AI voices work for faceless content (ElevenLabs, etc.)

### Music Selection
- Match energy to content (upbeat for tips, emotional for stories)
- Avoid copyrighted music on Reels/Shorts
- Use platform music libraries for safety
- Lower music volume under voiceover (ducking)

---

## Posting Strategy

### Optimal Posting Times (test your audience)

| Platform | Best Times (local) |
|----------|-------------------|
| TikTok | 7-9 AM, 12-3 PM, 7-11 PM |
| Reels | 9 AM, 12 PM, 7-9 PM |
| Shorts | 12-3 PM, 7-9 PM |

### Frequency Recommendations

| Goal | Minimum | Optimal |
|------|---------|---------|
| Growing | 1/day | 2-4/day |
| Maintaining | 3/week | 1/day |
| Testing | 2/week | 5/week |

### Batch Creation Workflow

1. **Ideate** (30 min): Generate 10-20 concepts
2. **Script** (1 hour): Write scripts for 5-10 videos
3. **Batch film** (2 hours): Record all talking head content
4. **Edit** (2-3 hours): Edit and add captions
5. **Schedule** (30 min): Queue for optimal times

---

## Analytics & Iteration

### Metrics That Matter

| Metric | What It Tells You |
|--------|-------------------|
| Watch time % | Is content engaging throughout? |
| Completion rate | Did hook + content deliver? |
| Saves | Is content valuable enough to revisit? |
| Shares | Is content worth spreading? |
| Comments | Did content spark conversation? |
| Follows | Did viewer want more from you? |

### What to Test

1. **Hooks**: Same content, different opening
2. **Length**: 15 sec vs 30 sec vs 60 sec
3. **Format**: Talking head vs slideshow vs demo
4. **Time**: Morning vs afternoon vs evening
5. **CTA**: Different calls to action

### When to Pivot

- 5+ videos with <1% completion rate → change hooks
- High views but low follows → check CTA and content-audience fit
- High saves but low shares → content is valuable but not social
- Lots of comments but negative → lean into controversy or adjust tone

---
name: "ai-act-readiness"
description: "/cs:ai-act-readiness <system> — EU AI Act 6-question forcing interrogation. Use during AI-system intake, before EU deployment, or during annual compliance refresh as Article 113 obligations phase in (2025-02-02 / 2025-08-02 / 2026-08-02 / 2027-08-02)."
---

# /cs:ai-act-readiness — EU AI Act Forcing Questions

**Command:** `/cs:ai-act-readiness <system>`

The EU AI Act compliance operator pressure-tests any AI system before EU deployment. Six Article-cited questions before any EU placement, conformity assessment, or annual compliance refresh.

## When to Run

- During AI-system intake review (per new system or material change)
- Before placing an AI system on the EU market
- Before signing the EU declaration of conformity (Article 47)
- During annual compliance refresh (Article 113 phasing brings new obligations)
- When the organization's role changes (deployer becomes provider via Article 25(1) substantial modification)
- When training compute approaches 10^25 FLOPs (Article 51 systemic-risk threshold)

## The Six EU AI Act Questions

### 1. Article 5: Is this a prohibited AI practice?
**Penalty: up to 35M EUR or 7% worldwide turnover.**
- 8 categories: subliminal manipulation, exploitation of vulnerabilities, social scoring, predictive policing, untargeted facial scraping, emotion recognition in workplace/education, biometric categorisation by sensitive attributes, real-time public biometric ID by law enforcement
- Run `ai_system_risk_classifier.py`
- If yes → STOP. Cannot place on EU market. No exceptions outside Article 5(2) carve-outs.

### 2. Article 6 + Annex III: Is this high-risk?
**Annex III triggers high-risk; Article 6(3) carve-out conditional.**
- 8 categories: biometrics, critical infrastructure, education, employment, essential services, law enforcement, migration, justice
- Carve-out applies only if Article 6(3)(a)-(d) AND no profiling of natural persons
- Profiling overrides carve-out (Article 6(3) last sentence)
- Run `ai_system_risk_classifier.py`

### 3. Article 43: For high-risk, Module A or Module H?
**Biometrics → Module H (notified body) by default; others → Module A if harmonised standards applied.**
- Run `conformity_assessment_planner.py`
- Module A (Annex VI): internal control with presumption of conformity if Article 40 harmonised standards applied
- Module H (Annex VII): full QMS + notified body for biometrics or where standards lacking
- Annex IV technical documentation: 8 items required before placing on market

### 4. Article 25: What role does the company play?
**Provider obligations are heaviest; substantial modification turns deployer into provider.**
- Provider (Article 3(3)): placed on market; full Title III + Article 73 reporting
- Deployer (Article 3(4)): Article 26 obligations + Article 27 FRIA if public sector
- Importer (Article 3(6)): Article 23 verification of conformity
- Distributor (Article 3(7)): Article 24 CE marking verification
- Authorized representative (Article 22): non-EU providers must appoint
- Run `ai_act_obligation_tracker.py`

### 5. Article 50: Are transparency obligations satisfied?
**In force 2 Aug 2025.**
- Article 50(1): disclose AI interaction to natural persons (chatbots, virtual agents)
- Article 50(2): mark synthetic content as AI-generated
- Article 50(3): disclose emotion recognition / biometric categorisation (outside Article 5 prohibitions)
- Article 50(4): disclose deepfakes (image, audio, video) as AI-generated

### 6. Articles 51-55: Is this a GPAI? Does it have systemic risk?
**GPAI has parallel track; systemic risk above 10^25 FLOPs.**
- Article 3(63): general-purpose AI model definition
- Article 51: systemic-risk presumption (≥ 10^25 FLOPs training compute) or Commission designation
- Article 53: all GPAI providers — Annex XI technical docs, Annex XII downstream info, copyright policy, training-data summary
- Article 55: systemic-risk GPAI additional obligations — model evaluations, adversarial testing, incident reporting, cybersecurity
- Article 54: non-EU GPAI providers must appoint authorized representative

## Workflow

```bash
# 1. Risk classification
python ../../ra-qm-team/skills/eu-ai-act-specialist/scripts/ai_system_risk_classifier.py systems.json

# 2. If high-risk: conformity assessment
python ../../ra-qm-team/skills/eu-ai-act-specialist/scripts/conformity_assessment_planner.py system.json

# 3. Per-role obligation matrix
python ../../ra-qm-team/skills/eu-ai-act-specialist/scripts/ai_act_obligation_tracker.py roles.json

# 4. Cross-framework reuse (ISO 42001 etc.)
python ../../skills/compliance-os/scripts/cross_framework_mapper.py program.json
```

## Output Format

```markdown
# EU AI Act Readiness: <system>
**Date:** YYYY-MM-DD
**Article Citations:** Every verdict below cites the specific Article.

## The Decision Being Made
[classify | conformity-route | obligation-scope | annual-refresh]

## Risk Classification
- Tier: prohibited | high_risk | limited_risk | minimal_risk
- Citation: Article X(Y) + Annex Z if applicable
- Rationale: <Article-cited rationale>
- GPAI: yes/no
- Systemic-risk GPAI: yes/no (per Article 51 10^25 FLOPs threshold)

## Conformity Assessment (if high-risk)
- Module: A | A_with_caveats | H | sectoral
- Citation: Article 43 + Annex VI/VII
- Notified body required: yes | no | optional
- Annex IV pack status: complete | in-progress | not-started

## Obligation Matrix
- Total obligations: N
- By deadline phase: 2025-02-02=A, 2025-08-02=B, 2026-08-02=C, 2027-08-02=D
- Highest-priority unmet obligation: <Article + description>

## Transparency (Article 50)
- 50(1) interaction disclosure: yes | no
- 50(2) synthetic content marking: yes | no | NA
- 50(3) emotion recognition disclosure: yes | no | NA
- 50(4) deepfake disclosure: yes | no | NA

## Cross-Framework Reuse
- ISO 42001 evidence applicable to Article 17 QMS: yes/no
- ISO 27001 evidence applicable to Article 15 cybersecurity: yes/no
- GDPR DPIA usable for Article 27 FRIA: yes/no

## Verdict
🟢 READY-FOR-EU | 🟡 GAPS-IDENTIFIED | 🔴 NOT-READY | 🚫 PROHIBITED

## Top 3 Actions
[3 concrete next steps with owner + Article-tied deadline]

## Legal Review Required
[Article-level ambiguities flagged for outside counsel: novel cases, GPAI threshold disputes, Article 5 boundary cases, Article 25 substantial-modification questions]
```

## Routing

- `/cs:compliance-readiness` — for multi-framework view (combine with ISO 42001 + GDPR)
- `/cs:aims-audit` — for ISO 42001 deep-dive
- `/cs:caio-review` — for executive AI strategy decisions
- `/cs:gc-review` — for novel-case legal review (GPAI threshold, Article 5 boundary, substantial-modification)
- `/cs:decide` — to log the verdict
- `/cs:freeze 30` — on EU launch commitments (regulatory exposure)

## Related

- Agent: [`cs-ai-act-compliance`](../../agents/cs-ai-act-compliance.md)
- Skill: [`eu-ai-act-specialist`](../../../ra-qm-team/skills/eu-ai-act-specialist/SKILL.md)
- Adjacent: `../../skills/compliance-os/`, `../aims-audit/`, `../compliance-readiness/`, `../../../ra-qm-team/skills/gdpr-dsgvo-expert/`

---

**Version:** 1.0.0

---
name: "app-store-optimization"
description: App Store Optimization (ASO) toolkit for researching keywords, analyzing competitor rankings, generating metadata suggestions, and improving app visibility on Apple App Store and Google Play Store. Use when the user asks about ASO, app store rankings, app metadata, app titles and descriptions, app store listings, app visibility, or mobile app marketing on iOS or Android. Supports keyword research and scoring, competitor keyword analysis, metadata optimization, A/B test planning, launch checklists, and tracking ranking changes.
triggers:
  - ASO
  - app store optimization
  - app store ranking
  - app keywords
  - app metadata
  - play store optimization
  - app store listing
  - improve app rankings
  - app visibility
  - app store SEO
  - mobile app marketing
  - app conversion rate
---

# App Store Optimization (ASO)

---

## Keyword Research Workflow

Discover and evaluate keywords that drive app store visibility.

### Workflow: Conduct Keyword Research

1. Define target audience and core app functions:
   - Primary use case (what problem does the app solve)
   - Target user demographics
   - Competitive category
2. Generate seed keywords from:
   - App features and benefits
   - User language (not developer terminology)
   - App store autocomplete suggestions
3. Expand keyword list using:
   - Modifiers (free, best, simple)
   - Actions (create, track, organize)
   - Audiences (for students, for teams, for business)
4. Evaluate each keyword:
   - Search volume (estimated monthly searches)
   - Competition (number and quality of ranking apps)
   - Relevance (alignment with app function)
5. Score and prioritize keywords:
   - Primary: Title and keyword field (iOS)
   - Secondary: Subtitle and short description
   - Tertiary: Full description only
6. Map keywords to metadata locations
7. Document keyword strategy for tracking
8. **Validation:** Keywords scored; placement mapped; no competitor brand names included; no plurals in iOS keyword field

### Keyword Evaluation Criteria

| Factor | Weight | High Score Indicators |
|--------|--------|----------------------|
| Relevance | 35% | Describes core app function |
| Volume | 25% | 10,000+ monthly searches |
| Competition | 25% | Top 10 apps have <4.5 avg rating |
| Conversion | 15% | Transactional intent ("best X app") |

### Keyword Placement Priority

| Location | Search Weight |
|----------|---------------|
| App Title | Highest |
| Subtitle (iOS) | High |
| Keyword Field (iOS) | High |
| Short Description (Android) | High |
| Full Description | Medium |

See: [references/keyword-research-guide.md](references/keyword-research-guide.md)

---

## Metadata Optimization Workflow

Optimize app store listing elements for search ranking and conversion.

### Workflow: Optimize App Metadata

1. Audit current metadata against platform limits:
   - Title character count and keyword presence
   - Subtitle/short description usage
   - Keyword field efficiency (iOS)
   - Description keyword density
2. Optimize title following formula:
   ```
   [Brand Name] - [Primary Keyword] [Secondary Keyword]
   ```
3. Write subtitle (iOS) or short description (Android):
   - Focus on primary benefit
   - Include secondary keyword
   - Use action verbs
4. Optimize keyword field (iOS only):
   - Remove duplicates from title
   - Remove plurals (Apple indexes both forms)
   - No spaces after commas
   - Prioritize by score
5. Rewrite full description:
   - Hook paragraph with value proposition
   - Feature bullets with keywords
   - Social proof section
   - Call to action
6. Validate character counts for each field
7. Calculate keyword density (target 2-3% primary)
8. **Validation:** All fields within character limits; primary keyword in title; no keyword stuffing (>5%); natural language preserved

### Platform Character Limits

| Field | Apple App Store | Google Play Store |
|-------|-----------------|-------------------|
| Title | 30 characters | 50 characters |
| Subtitle | 30 characters | N/A |
| Short Description | N/A | 80 characters |
| Keywords | 100 characters | N/A |
| Promotional Text | 170 characters | N/A |
| Full Description | 4,000 characters | 4,000 characters |
| What's New | 4,000 characters | 500 characters |

### Description Structure

```
PARAGRAPH 1: Hook (50-100 words)
├── Address user pain point
├── State main value proposition
└── Include primary keyword

PARAGRAPH 2-3: Features (100-150 words)
├── Top 5 features with benefits
├── Bullet points for scanability
└── Secondary keywords naturally integrated

PARAGRAPH 4: Social Proof (50-75 words)
├── Download count or rating
├── Press mentions or awards
└── Summary of user testimonials

PARAGRAPH 5: Call to Action (25-50 words)
├── Clear next step
└── Reassurance (free trial, no signup)
```

See: [references/platform-requirements.md](references/platform-requirements.md)

---

## Competitor Analysis Workflow

Analyze top competitors to identify keyword gaps and positioning opportunities.

### Workflow: Analyze Competitor ASO Strategy

1. Identify top 10 competitors:
   - Direct competitors (same core function)
   - Indirect competitors (overlapping audience)
   - Category leaders (top downloads)
2. Extract competitor keywords from:
   - App titles and subtitles
   - First 100 words of descriptions
   - Visible metadata patterns
3. Build competitor keyword matrix:
   - Map which keywords each competitor targets
   - Calculate coverage percentage per keyword
4. Identify keyword gaps:
   - Keywords with <40% competitor coverage
   - High volume terms competitors miss
   - Long-tail opportunities
5. Analyze competitor visual assets:
   - Icon design patterns
   - Screenshot messaging and style
   - Video presence and quality
6. Compare ratings and review patterns:
   - Average rating by competitor
   - Common praise themes
   - Common complaint themes
7. Document positioning opportunities
8. **Validation:** 10+ competitors analyzed; keyword matrix complete; gaps identified with volume estimates; visual audit documented

### Competitor Analysis Matrix

| Analysis Area | Data Points |
|---------------|-------------|
| Keywords | Title keywords, description frequency |
| Metadata | Character utilization, keyword density |
| Visuals | Icon style, screenshot count/style |
| Ratings | Average rating, total count, velocity |
| Reviews | Top praise, top complaints |

### Gap Analysis Template

| Opportunity Type | Example | Action |
|------------------|---------|--------|
| Keyword gap | "habit tracker" (40% coverage) | Add to keyword field |
| Feature gap | Competitor lacks widget | Highlight in screenshots |
| Visual gap | No videos in top 5 | Create app preview |
| Messaging gap | None mention "free" | Test free positioning |

---

## App Launch Workflow

Execute a structured launch for maximum initial visibility.

### Workflow: Launch App to Stores

1. Complete pre-launch preparation (4 weeks before):
   - Finalize keywords and metadata
   - Prepare all visual assets
   - Set up analytics (Firebase, Mixpanel)
   - Build press kit and media list
2. Submit for review (2 weeks before):
   - Complete all store requirements
   - Verify compliance with guidelines
   - Prepare launch communications
3. Configure post-launch systems:
   - Set up review monitoring
   - Prepare response templates
   - Configure rating prompt timing
4. Execute launch day:
   - Verify app is live in both stores
   - Announce across all channels
   - Begin review response cycle
5. Monitor initial performance (days 1-7):
   - Track download velocity hourly
   - Monitor reviews and respond within 24 hours
   - Document any issues for quick fixes
6. Conduct 7-day retrospective:
   - Compare performance to projections
   - Identify quick optimization wins
   - Plan first metadata update
7. Schedule first update (2 weeks post-launch)
8. **Validation:** App live in stores; analytics tracking; review responses within 24h; download velocity documented; first update scheduled

### Pre-Launch Checklist

| Category | Items |
|----------|-------|
| Metadata | Title, subtitle, description, keywords |
| Visual Assets | Icon, screenshots (all sizes), video |
| Compliance | Age rating, privacy policy, content rights |
| Technical | App binary, signing certificates |
| Analytics | SDK integration, event tracking |
| Marketing | Press kit, social content, email ready |

### Launch Timing Considerations

| Factor | Recommendation |
|--------|----------------|
| Day of week | Tuesday-Wednesday (avoid weekends) |
| Time of day | Morning in target market timezone |
| Seasonal | Align with relevant category seasons |
| Competition | Avoid major competitor launch dates |

See: [references/aso-best-practices.md](references/aso-best-practices.md)

---

## A/B Testing Workflow

Test metadata and visual elements to improve conversion rates.

### Workflow: Run A/B Test

1. Select test element (prioritize by impact):
   - Icon (highest impact)
   - Screenshot 1 (high impact)
   - Title (high impact)
   - Short description (medium impact)
2. Form hypothesis:
   ```
   If we [change], then [metric] will [improve/increase] by [amount]
   because [rationale].
   ```
3. Create variants:
   - Control: Current version
   - Treatment: Single variable change
4. Calculate required sample size:
   - Baseline conversion rate
   - Minimum detectable effect (usually 5%)
   - Statistical significance (95%)
5. Launch test:
   - Apple: Use Product Page Optimization
   - Android: Use Store Listing Experiments
6. Run test for minimum duration:
   - At least 7 days
   - Until statistical significance reached
7. Analyze results:
   - Compare conversion rates
   - Check statistical significance
   - Document learnings
8. **Validation:** Single variable tested; sample size sufficient; significance reached (95%); results documented; winner implemented

### A/B Test Prioritization

| Element | Conversion Impact | Test Complexity |
|---------|-------------------|-----------------|
| App Icon | 10-25% lift possible | Medium (design needed) |
| Screenshot 1 | 15-35% lift possible | Medium |
| Title | 5-15% lift possible | Low |
| Short Description | 5-10% lift possible | Low |
| Video | 10-20% lift possible | High |

### Sample Size Quick Reference

| Baseline CVR | Impressions Needed (per variant) |
|--------------|----------------------------------|
| 1% | 31,000 |
| 2% | 15,500 |
| 5% | 6,200 |
| 10% | 3,100 |

### Test Documentation Template

```
TEST ID: ASO-2025-001
ELEMENT: App Icon
HYPOTHESIS: A bolder color icon will increase conversion by 10%
START DATE: [Date]
END DATE: [Date]

RESULTS:
├── Control CVR: 4.2%
├── Treatment CVR: 4.8%
├── Lift: +14.3%
├── Significance: 97%
└── Decision: Implement treatment

LEARNINGS:
- Bold colors outperform muted tones in this category
- Apply to screenshot backgrounds for next test
```

---

## Before/After Examples

### Title Optimization

**Productivity App:**

| Version | Title | Analysis |
|---------|-------|----------|
| Before | "MyTasks" | No keywords, brand only (8 chars) |
| After | "MyTasks - Todo List & Planner" | Primary + secondary keywords (29 chars) |

**Fitness App:**

| Version | Title | Analysis |
|---------|-------|----------|
| Before | "FitTrack Pro" | Generic modifier (12 chars) |
| After | "FitTrack: Workout Log & Gym" | Category keywords (27 chars) |

### Subtitle Optimization (iOS)

| Version | Subtitle | Analysis |
|---------|----------|----------|
| Before | "Get Things Done" | Vague, no keywords |
| After | "Daily Task Manager & Planner" | Two keywords, benefit clear |

### Keyword Field Optimization (iOS)

**Before (Inefficient - 89 chars, 8 keywords):**
```
task manager, todo list, productivity app, daily planner, reminder app
```

**After (Optimized - 97 chars, 14 keywords):**
```
task,todo,checklist,reminder,organize,daily,planner,schedule,deadline,goals,habit,widget,sync,team
```

**Improvements:**
- Removed spaces after commas (+8 chars)
- Removed duplicates (task manager → task)
- Removed plurals (reminders → reminder)
- Removed words in title
- Added more relevant keywords

### Description Opening

**Before:**
```
MyTasks is a comprehensive task management solution designed
to help busy professionals organize their daily activities
and boost productivity.
```

**After:**
```
Forget missed deadlines. MyTasks keeps every task, reminder,
and project in one place—so you focus on doing, not remembering.
Trusted by 500,000+ professionals.
```

**Improvements:**
- Leads with user pain point
- Specific benefit (not generic "boost productivity")
- Social proof included
- Keywords natural, not stuffed

### Screenshot Caption Evolution

| Version | Caption | Issue |
|---------|---------|-------|
| Before | "Task List Feature" | Feature-focused, passive |
| Better | "Create Task Lists" | Action verb, but still feature |
| Best | "Never Miss a Deadline" | Benefit-focused, emotional |

---

## Tools and References

### Scripts

| Script | Purpose | Usage |
|--------|---------|-------|
| [keyword_analyzer.py](scripts/keyword_analyzer.py) | Analyze keywords for volume and competition | `python keyword_analyzer.py --keywords "todo,task,planner"` |
| [metadata_optimizer.py](scripts/metadata_optimizer.py) | Validate metadata character limits and density | `python metadata_optimizer.py --platform ios --title "App Title"` |
| [competitor_analyzer.py](scripts/competitor_analyzer.py) | Extract and compare competitor keywords | `python competitor_analyzer.py --competitors "App1,App2,App3"` |
| [aso_scorer.py](scripts/aso_scorer.py) | Calculate overall ASO health score | `python aso_scorer.py --app-id com.example.app` |
| [ab_test_planner.py](scripts/ab_test_planner.py) | Plan tests and calculate sample sizes | `python ab_test_planner.py --cvr 0.05 --lift 0.10` |
| [review_analyzer.py](scripts/review_analyzer.py) | Analyze review sentiment and themes | `python review_analyzer.py --app-id com.example.app` |
| [launch_checklist.py](scripts/launch_checklist.py) | Generate platform-specific launch checklists | `python launch_checklist.py --platform ios` |
| [localization_helper.py](scripts/localization_helper.py) | Manage multi-language metadata | `python localization_helper.py --locales "en,es,de,ja"` |

### References

| Document | Content |
|----------|---------|
| [platform-requirements.md](references/platform-requirements.md) | iOS and Android metadata specs, visual asset requirements |
| [aso-best-practices.md](references/aso-best-practices.md) | Optimization strategies, rating management, launch tactics |
| [keyword-research-guide.md](references/keyword-research-guide.md) | Research methodology, evaluation framework, tracking |

### Assets

| Template | Purpose |
|----------|---------|
| [aso-audit-template.md](assets/aso-audit-template.md) | Structured audit checklist for app store listings |

---

## Platform Notes

| Platform / Constraint | Behavior / Impact |
|-----------------------|-------------------|
| iOS keyword changes | Require app submission |
| iOS promotional text | Editable without an app update |
| Android metadata changes | Index in 1-2 hours |
| Android keyword field | None — use description instead |
| Keyword volume data | Estimates only; no official source |
| Competitor data | Public listings only |

**When not to use this skill:** web apps (use web SEO), enterprise/internal apps, TestFlight-only betas, or paid advertising strategy.

---

## Related Skills

| Skill | Integration Point |
|-------|-------------------|
| [content-creator](../content-creator/) | App description copywriting |
| [marketing-demand-acquisition](../marketing-demand-acquisition/) | Launch promotion campaigns |
| [marketing-strategy-pmm](../marketing-strategy-pmm/) | Go-to-market planning |

## Proactive Triggers

- **No keyword optimization in title** → App title is the #1 ranking factor. Include top keyword.
- **Screenshots don't show value** → Screenshots should tell a story, not show UI.
- **No ratings strategy** → Below 4.0 stars kills conversion. Implement in-app rating prompts.
- **Description keyword-stuffed** → Natural language with keywords beats keyword stuffing.

## Output Artifacts

| When you ask for... | You get... |
|---------------------|------------|
| "ASO audit" | Full app store listing audit with prioritized fixes |
| "Keyword research" | Keyword list with search volume and difficulty scores |
| "Optimize my listing" | Rewritten title, subtitle, description, keyword field |

## Communication

All output passes quality verification:
- Self-verify: source attribution, assumption audit, confidence scoring
- Output format: Bottom Line → What (with confidence) → Why → How to Act
- Results only. Every finding tagged: 🟢 verified, 🟡 medium, 🔴 assumed.

FILE:assets/aso-audit-template.md
# ASO Audit Template

Use this template to conduct a systematic App Store Optimization audit.

---

## App Information

| Field | Value |
|-------|-------|
| App Name | |
| Platform | [ ] iOS [ ] Android |
| Category | |
| Current Downloads | |
| Current Rating | |
| Audit Date | |

---

## Metadata Audit

### Title Analysis

| Criterion | iOS (30 chars) | Android (50 chars) |
|-----------|----------------|---------------------|
| Current Title | | |
| Character Count | /30 | /50 |
| Primary Keyword Present | [ ] Yes [ ] No | [ ] Yes [ ] No |
| Brand Name Position | | |

**Title Score:** ___/10

**Recommendations:**
- [ ]
- [ ]

### Subtitle / Short Description

| Criterion | iOS Subtitle (30 chars) | Android Short Desc (80 chars) |
|-----------|-------------------------|-------------------------------|
| Current Text | | |
| Character Count | /30 | /80 |
| Keywords Included | | |
| Benefit-Focused | [ ] Yes [ ] No | [ ] Yes [ ] No |

**Score:** ___/10

**Recommendations:**
- [ ]
- [ ]

### Keyword Field (iOS Only)

| Criterion | Status |
|-----------|--------|
| Current Keywords | |
| Character Count | /100 |
| Duplicates Present | [ ] Yes [ ] No |
| Plurals Included | [ ] Yes [ ] No |
| Brand Names Included | [ ] Yes [ ] No |

**Score:** ___/10

**Recommendations:**
- [ ]
- [ ]

### Full Description

| Criterion | iOS | Android |
|-----------|-----|---------|
| Character Count | /4000 | /4000 |
| Primary Keyword Density | % | % |
| Secondary Keywords (count) | | |
| Feature Bullets Present | [ ] Yes [ ] No | [ ] Yes [ ] No |
| Social Proof Included | [ ] Yes [ ] No | [ ] Yes [ ] No |
| CTA Present | [ ] Yes [ ] No | [ ] Yes [ ] No |

**Score:** ___/10

**Recommendations:**
- [ ]
- [ ]

---

## Visual Asset Audit

### App Icon

| Criterion | Status |
|-----------|--------|
| Recognizable at 60x60px | [ ] Yes [ ] No |
| Distinct from competitors | [ ] Yes [ ] No |
| Matches app design | [ ] Yes [ ] No |
| No text/words | [ ] Yes [ ] No |

**Score:** ___/10

**Recommendations:**
- [ ]
- [ ]

### Screenshots

| Screenshot | Caption | Feature Shown | Score |
|------------|---------|---------------|-------|
| 1 (Hero) | | | /10 |
| 2 | | | /10 |
| 3 | | | /10 |
| 4 | | | /10 |
| 5 | | | /10 |

| Criterion | Status |
|-----------|--------|
| Total Screenshots | /10 (iOS) or /8 (Android) |
| Captions Present | [ ] Yes [ ] No |
| Consistent Style | [ ] Yes [ ] No |
| First 3 Show Value | [ ] Yes [ ] No |
| Device Frames Used | [ ] Yes [ ] No |

**Overall Screenshot Score:** ___/10

**Recommendations:**
- [ ]
- [ ]

### App Preview Video

| Criterion | Status |
|-----------|--------|
| Video Present | [ ] Yes [ ] No |
| Duration | seconds |
| Shows Core Features | [ ] Yes [ ] No |
| Hook in First 5 Seconds | [ ] Yes [ ] No |
| CTA at End | [ ] Yes [ ] No |

**Score:** ___/10

---

## Keyword Performance Audit

### Current Keyword Rankings

| Keyword | Current Rank | Volume | Competition | Score |
|---------|--------------|--------|-------------|-------|
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |

### Keyword Opportunities

| Keyword | Current Rank | Potential | Action |
|---------|--------------|-----------|--------|
| | | | |
| | | | |
| | | | |

---

## Rating & Review Audit

### Rating Summary

| Metric | Value |
|--------|-------|
| Current Average Rating | /5.0 |
| Total Ratings | |
| Ratings (Last 30 Days) | |
| 5-Star Percentage | % |
| 1-Star Percentage | % |

### Review Analysis

| Category | Count | Common Themes |
|----------|-------|---------------|
| Positive (4-5 stars) | | |
| Neutral (3 stars) | | |
| Negative (1-2 stars) | | |

### Response Rate

| Metric | Value |
|--------|-------|
| Reviews Responded | % |
| Avg Response Time | hours |

**Rating Score:** ___/10

**Recommendations:**
- [ ]
- [ ]

---

## Competitor Comparison

### Top 3 Competitors

| Metric | Your App | Competitor 1 | Competitor 2 | Competitor 3 |
|--------|----------|--------------|--------------|--------------|
| Name | | | | |
| Rating | | | | |
| Total Ratings | | | | |
| Downloads | | | | |
| Title Keywords | | | | |
| Screenshot Count | | | | |

### Competitive Gaps

| Gap Identified | Opportunity |
|----------------|-------------|
| | |
| | |
| | |

---

## Overall ASO Score

| Category | Weight | Score | Weighted |
|----------|--------|-------|----------|
| Title/Metadata | 25% | /10 | |
| Keywords | 25% | /10 | |
| Visual Assets | 25% | /10 | |
| Ratings/Reviews | 25% | /10 | |
| **TOTAL** | 100% | | **/100** |

---

## Priority Action Items

### High Priority (This Week)

1. [ ]
2. [ ]
3. [ ]

### Medium Priority (This Month)

1. [ ]
2. [ ]
3. [ ]

### Low Priority (This Quarter)

1. [ ]
2. [ ]
3. [ ]

---

## Audit Sign-Off

| Role | Name | Date |
|------|------|------|
| Auditor | | |
| Reviewer | | |
| App Owner | | |

---

## Notes

_Additional observations and context:_

FILE:expected_output.json
{
  "request_type": "keyword_research",
  "app_name": "TaskFlow Pro",
  "keyword_analysis": {
    "total_keywords_analyzed": 25,
    "primary_keywords": [
      {
        "keyword": "task manager",
        "search_volume": 45000,
        "competition_level": "high",
        "relevance_score": 0.95,
        "difficulty_score": 72.5,
        "potential_score": 78.3,
        "recommendation": "High priority - target immediately"
      },
      {
        "keyword": "productivity app",
        "search_volume": 38000,
        "competition_level": "high",
        "relevance_score": 0.90,
        "difficulty_score": 68.2,
        "potential_score": 75.1,
        "recommendation": "High priority - target immediately"
      },
      {
        "keyword": "todo list",
        "search_volume": 52000,
        "competition_level": "very_high",
        "relevance_score": 0.85,
        "difficulty_score": 78.9,
        "potential_score": 71.4,
        "recommendation": "High priority - target immediately"
      }
    ],
    "secondary_keywords": [
      {
        "keyword": "team task manager",
        "search_volume": 8500,
        "competition_level": "medium",
        "relevance_score": 0.88,
        "difficulty_score": 42.3,
        "potential_score": 68.7,
        "recommendation": "Good opportunity - include in metadata"
      },
      {
        "keyword": "project planning app",
        "search_volume": 12000,
        "competition_level": "medium",
        "relevance_score": 0.75,
        "difficulty_score": 48.1,
        "potential_score": 64.2,
        "recommendation": "Good opportunity - include in metadata"
      }
    ],
    "long_tail_keywords": [
      {
        "keyword": "ai task prioritization",
        "search_volume": 2800,
        "competition_level": "low",
        "relevance_score": 0.95,
        "difficulty_score": 25.4,
        "potential_score": 82.6,
        "recommendation": "Excellent long-tail opportunity"
      },
      {
        "keyword": "team productivity tool",
        "search_volume": 3500,
        "competition_level": "low",
        "relevance_score": 0.85,
        "difficulty_score": 28.7,
        "potential_score": 79.3,
        "recommendation": "Excellent long-tail opportunity"
      }
    ]
  },
  "competitor_insights": {
    "competitors_analyzed": 4,
    "common_keywords": [
      "task",
      "todo",
      "list",
      "productivity",
      "organize",
      "manage"
    ],
    "keyword_gaps": [
      {
        "keyword": "ai prioritization",
        "used_by": ["None of the major competitors"],
        "opportunity": "Unique positioning opportunity"
      },
      {
        "keyword": "smart task manager",
        "used_by": ["Things 3"],
        "opportunity": "Underutilized by most competitors"
      }
    ]
  },
  "metadata_recommendations": {
    "apple_app_store": {
      "title_options": [
        {
          "title": "TaskFlow - AI Task Manager",
          "length": 26,
          "keywords_included": ["task manager", "ai"],
          "strategy": "brand_plus_primary"
        },
        {
          "title": "TaskFlow: Smart Todo & Tasks",
          "length": 29,
          "keywords_included": ["todo", "tasks"],
          "strategy": "brand_plus_multiple"
        }
      ],
      "subtitle_recommendation": "AI-Powered Team Productivity",
      "keyword_field": "productivity,organize,planner,schedule,workflow,reminders,collaboration,calendar,sync,priorities",
      "description_focus": "Lead with AI differentiation, emphasize team features"
    },
    "google_play_store": {
      "title_options": [
        {
          "title": "TaskFlow - AI Task Manager & Team Productivity",
          "length": 48,
          "keywords_included": ["task manager", "ai", "team", "productivity"],
          "strategy": "keyword_rich"
        }
      ],
      "short_description_recommendation": "AI task manager - Organize, prioritize, and collaborate with your team",
      "description_focus": "Keywords naturally integrated throughout 4000 character description"
    }
  },
  "strategic_recommendations": [
    "Focus on 'AI prioritization' as unique differentiator - low competition, high relevance",
    "Target 'team task manager' and 'team productivity' keywords - good search volume, lower competition than generic terms",
    "Include long-tail keywords in description for additional discovery opportunities",
    "Test title variations with A/B testing after launch",
    "Monitor competitor keyword changes quarterly"
  ],
  "priority_actions": [
    {
      "action": "Optimize app title with primary keyword",
      "priority": "high",
      "expected_impact": "15-25% improvement in search visibility"
    },
    {
      "action": "Create description highlighting AI features with natural keyword integration",
      "priority": "high",
      "expected_impact": "10-15% improvement in conversion rate"
    },
    {
      "action": "Plan A/B tests for icon and screenshots post-launch",
      "priority": "medium",
      "expected_impact": "5-10% improvement in conversion rate"
    }
  ],
  "aso_health_estimate": {
    "current_score": "N/A (pre-launch)",
    "potential_score_with_optimizations": "75-80/100",
    "key_strengths": [
      "Unique AI differentiation",
      "Clear target audience",
      "Strong feature set"
    ],
    "areas_to_develop": [
      "Build rating volume post-launch",
      "Monitor and respond to reviews",
      "Continuous keyword optimization"
    ]
  }
}

FILE:HOW_TO_USE.md
# How to Use the App Store Optimization Skill

Hey Claude—I just added the "app-store-optimization" skill. Can you help me optimize my app's presence on the App Store and Google Play?

## Example Invocations

### Keyword Research

**Example 1: Basic Keyword Research**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you research the best keywords for my productivity app? I'm targeting professionals who need task management and team collaboration features.
```

**Example 2: Competitive Keyword Analysis**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you analyze keywords that Todoist, Asana, and Monday.com are using? I want to find gaps and opportunities for my project management app.
```

### Metadata Optimization

**Example 3: Optimize App Title**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you optimize my app title for the Apple App Store? My app is called "TaskFlow" and I want to rank for "task manager", "productivity", and "team collaboration". The title needs to be under 30 characters.
```

**Example 4: Full Metadata Package**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you create optimized metadata for both Apple App Store and Google Play Store? Here's my app info:
- Name: TaskFlow
- Category: Productivity
- Key features: AI task prioritization, team collaboration, calendar integration
- Target keywords: task manager, productivity app, team tasks
```

### Competitor Analysis

**Example 5: Analyze Top Competitors**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you analyze the ASO strategies of the top 5 productivity apps in the App Store? I want to understand their title strategies, keyword usage, and visual asset approaches.
```

**Example 6: Identify Competitive Gaps**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you compare my app's ASO performance against competitors and identify what I'm missing? Here's my current metadata: [paste metadata]
```

### ASO Score Calculation

**Example 7: Calculate Overall ASO Health**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you calculate my app's ASO health score? Here are my metrics:
- Average rating: 4.2 stars
- Total ratings: 3,500
- Keywords in top 10: 3
- Keywords in top 50: 12
- Conversion rate: 4.5%
```

**Example 8: Identify Improvement Areas**
```
Hey Claude—I just added the "app-store-optimization" skill. My ASO score is 62/100. Can you tell me which areas I should focus on first to improve my rankings and downloads?
```

### A/B Testing

**Example 9: Plan Icon Test**
```
Hey Claude—I just added the "app-store-optimization" skill. I want to A/B test two different app icons. My current conversion rate is 5%. Can you help me plan the test, calculate required sample size, and determine how long to run it?
```

**Example 10: Analyze Test Results**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you analyze my A/B test results?
- Variant A (control): 2,500 visitors, 125 installs
- Variant B (new icon): 2,500 visitors, 150 installs
Is this statistically significant? Should I implement variant B?
```

### Localization

**Example 11: Plan Localization Strategy**
```
Hey Claude—I just added the "app-store-optimization" skill. I currently only have English metadata. Which markets should I localize for first? I'm a bootstrapped startup with moderate budget.
```

**Example 12: Translate Metadata**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you help me translate my app metadata to Spanish for the Mexico market? Here's my English metadata: [paste metadata]. Check if it fits within character limits.
```

### Review Analysis

**Example 13: Analyze User Reviews**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you analyze my recent reviews and tell me:
- Overall sentiment (positive/negative ratio)
- Most common complaints
- Most requested features
- Bugs that need immediate fixing
```

**Example 14: Generate Review Response Templates**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you create professional response templates for:
- Users reporting crashes
- Feature requests
- Positive 5-star reviews
- General complaints
```

### Launch Planning

**Example 15: Pre-Launch Checklist**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you generate a comprehensive pre-launch checklist for both Apple App Store and Google Play Store? My launch date is December 1, 2025.
```

**Example 16: Optimize Launch Timing**
```
Hey Claude—I just added the "app-store-optimization" skill. What's the best day and time to launch my fitness app? I want to maximize visibility and downloads in the first week.
```

**Example 17: Plan Seasonal Campaign**
```
Hey Claude—I just added the "app-store-optimization" skill. Can you identify seasonal opportunities for my fitness app? It's currently October—what campaigns should I run for the next 6 months?
```

## What to Provide

### For Keyword Research
- App name and category
- Target audience description
- Key features and unique value proposition
- Competitor apps (optional)
- Geographic markets to target

### For Metadata Optimization
- Current app name
- Platform (Apple, Google, or both)
- Target keywords (prioritized list)
- Key features and benefits
- Target audience
- Current metadata (for optimization)

### For Competitor Analysis
- Your app category
- List of competitor app names or IDs
- Platform (Apple or Google)
- Specific aspects to analyze (keywords, visuals, ratings)

### For ASO Score Calculation
- Metadata quality metrics (title length, description length, keyword density)
- Rating data (average rating, total ratings, recent ratings)
- Keyword rankings (top 10, top 50, top 100 counts)
- Conversion metrics (impression-to-install rate, downloads)

### For A/B Testing
- Test type (icon, screenshot, title, description)
- Control variant details
- Test variant details
- Baseline conversion rate
- For results analysis: visitor and conversion counts for both variants

### For Localization
- Current market and language
- Budget level (low, medium, high)
- Target number of markets
- Current metadata text for translation

### For Review Analysis
- Recent reviews (text, rating, date)
- Platform (Apple or Google)
- Time period to analyze
- Specific focus (bugs, features, sentiment)

### For Launch Planning
- Platform (Apple, Google, or both)
- Target launch date
- App category
- App information (name, features, target audience)

## What You'll Get

### Keyword Research Output
- Prioritized keyword list with search volume estimates
- Competition level analysis
- Relevance scores
- Long-tail keyword opportunities
- Strategic recommendations

### Metadata Optimization Output
- Optimized titles (multiple options)
- Optimized descriptions (short and full)
- Keyword field optimization (Apple)
- Character count validation
- Keyword density analysis
- Before/after comparison

### Competitor Analysis Output
- Ranked competitors by ASO strength
- Common keyword patterns
- Keyword gaps and opportunities
- Visual asset assessment
- Best practices identified
- Actionable recommendations

### ASO Score Output
- Overall score (0-100)
- Breakdown by category (metadata, ratings, keywords, conversion)
- Strengths and weaknesses
- Prioritized action items
- Expected impact of improvements

### A/B Test Output
- Test design with hypothesis
- Required sample size calculation
- Duration estimates
- Statistical significance analysis
- Implementation recommendations
- Learnings and insights

### Localization Output
- Prioritized target markets
- Estimated translation costs
- ROI projections
- Character limit validation for each language
- Cultural adaptation recommendations
- Phased implementation plan

### Review Analysis Output
- Sentiment distribution (positive/neutral/negative)
- Common themes and topics
- Top issues requiring fixes
- Most requested features
- Response templates
- Trend analysis over time

### Launch Planning Output
- Platform-specific checklists (Apple, Google, Universal)
- Timeline with milestones
- Compliance validation
- Optimal launch timing recommendations
- Seasonal campaign opportunities
- Update cadence planning

## Tips for Best Results

1. **Be Specific**: Provide as much detail about your app as possible
2. **Include Context**: Share your goals (increase downloads, improve ranking, boost conversion)
3. **Provide Data**: Real metrics enable more accurate analysis
4. **Iterate**: Start with keyword research, then optimize metadata, then test
5. **Track Results**: Monitor changes after implementing recommendations
6. **Stay Compliant**: Always verify recommendations against current App Store/Play Store guidelines
7. **Test First**: Use A/B testing before making major metadata changes
8. **Localize Strategically**: Start with highest-ROI markets first
9. **Respond to Reviews**: Use provided templates to engage with users
10. **Plan Ahead**: Use launch checklists and timelines to avoid last-minute rushes

## Common Workflows

### New App Launch
1. Keyword research → Competitor analysis → Metadata optimization → Pre-launch checklist → Launch timing optimization

### Improving Existing App
1. ASO score calculation → Identify gaps → Metadata optimization → A/B testing → Review analysis → Implement changes

### International Expansion
1. Localization planning → Market prioritization → Metadata translation → ROI analysis → Phased rollout

### Ongoing Optimization
1. Monthly keyword ranking tracking → Quarterly metadata updates → Continuous A/B testing → Review monitoring → Seasonal campaigns

## Need Help?

If you need clarification on any aspect of ASO or want to combine multiple analyses, just ask! For example:

```
Hey Claude—I just added the "app-store-optimization" skill. Can you create a complete ASO strategy for my new productivity app? I need keyword research, optimized metadata for both stores, a pre-launch checklist, and launch timing recommendations.
```

The skill can handle comprehensive, multi-phase ASO projects as well as specific tactical optimizations.

FILE:README.md
# App Store Optimization (ASO) Skill

**Version**: 1.0.0
**Last Updated**: November 7, 2025
**Author**: Claude Skills Factory

## Overview

A comprehensive App Store Optimization (ASO) skill that provides complete capabilities for researching, optimizing, and tracking mobile app performance on the Apple App Store and Google Play Store. This skill empowers app developers and marketers to maximize their app's visibility, downloads, and success in competitive app marketplaces.

## What This Skill Does

This skill provides end-to-end ASO capabilities across seven key areas:

1. **Research & Analysis**: Keyword research, competitor analysis, market trends, review sentiment
2. **Metadata Optimization**: Title, description, keywords with platform-specific character limits
3. **Conversion Optimization**: A/B testing framework, visual asset optimization
4. **Rating & Review Management**: Sentiment analysis, response strategies, issue identification
5. **Launch & Update Strategies**: Pre-launch checklists, timing optimization, update planning
6. **Analytics & Tracking**: ASO scoring, keyword rankings, performance benchmarking
7. **Localization**: Multi-language strategy, translation management, ROI analysis

## Key Features

### Comprehensive Keyword Research
- Search volume and competition analysis
- Long-tail keyword discovery
- Competitor keyword extraction
- Keyword difficulty scoring
- Strategic prioritization

### Platform-Specific Metadata Optimization
- **Apple App Store**:
  - Title (30 chars)
  - Subtitle (30 chars)
  - Promotional Text (170 chars)
  - Description (4000 chars)
  - Keywords field (100 chars)
- **Google Play Store**:
  - Title (50 chars)
  - Short Description (80 chars)
  - Full Description (4000 chars)
- Character limit validation
- Keyword density analysis
- Multiple optimization strategies

### Competitor Intelligence
- Automated competitor discovery
- Metadata strategy analysis
- Visual asset assessment
- Gap identification
- Competitive positioning

### ASO Health Scoring
- 0-100 overall score
- Four-category breakdown (Metadata, Ratings, Keywords, Conversion)
- Strengths and weaknesses identification
- Prioritized action recommendations
- Expected impact estimates

### Scientific A/B Testing
- Test design and hypothesis formulation
- Sample size calculation
- Statistical significance analysis
- Duration estimation
- Implementation recommendations

### Global Localization
- Market prioritization (Tier 1/2/3)
- Translation cost estimation
- Character limit adaptation by language
- Cultural keyword considerations
- ROI analysis

### Review Intelligence
- Sentiment analysis
- Common theme extraction
- Bug and issue identification
- Feature request clustering
- Professional response templates

### Launch Planning
- Platform-specific checklists
- Timeline generation
- Compliance validation
- Optimal timing recommendations
- Seasonal campaign planning

## Python Modules

This skill includes 8 powerful Python modules:

### 1. keyword_analyzer.py
**Purpose**: Analyzes keywords for search volume, competition, and relevance

**Key Functions**:
- `analyze_keyword()`: Single keyword analysis
- `compare_keywords()`: Multi-keyword comparison and ranking
- `find_long_tail_opportunities()`: Generate long-tail variations
- `calculate_keyword_density()`: Analyze keyword usage in text
- `extract_keywords_from_text()`: Extract keywords from reviews/descriptions

### 2. metadata_optimizer.py
**Purpose**: Optimizes titles, descriptions, keywords with character limit validation

**Key Functions**:
- `optimize_title()`: Generate optimal title options
- `optimize_description()`: Create conversion-focused descriptions
- `optimize_keyword_field()`: Maximize Apple's 100-char keyword field
- `validate_character_limits()`: Ensure platform compliance
- `calculate_keyword_density()`: Analyze keyword integration

### 3. competitor_analyzer.py
**Purpose**: Analyzes competitor ASO strategies

**Key Functions**:
- `analyze_competitor()`: Single competitor deep-dive
- `compare_competitors()`: Multi-competitor analysis
- `identify_gaps()`: Find competitive opportunities
- `_calculate_competitive_strength()`: Score competitor ASO quality

### 4. aso_scorer.py
**Purpose**: Calculates comprehensive ASO health score

**Key Functions**:
- `calculate_overall_score()`: 0-100 ASO health score
- `score_metadata_quality()`: Evaluate metadata optimization
- `score_ratings_reviews()`: Assess rating quality and volume
- `score_keyword_performance()`: Analyze ranking positions
- `score_conversion_metrics()`: Evaluate conversion rates
- `generate_recommendations()`: Prioritized improvement actions

### 5. ab_test_planner.py
**Purpose**: Plans and tracks A/B tests for ASO elements

**Key Functions**:
- `design_test()`: Create test hypothesis and structure
- `calculate_sample_size()`: Determine required visitors
- `calculate_significance()`: Assess statistical validity
- `track_test_results()`: Monitor ongoing tests
- `generate_test_report()`: Create comprehensive test reports

### 6. localization_helper.py
**Purpose**: Manages multi-language ASO optimization

**Key Functions**:
- `identify_target_markets()`: Prioritize localization markets
- `translate_metadata()`: Adapt metadata for languages
- `adapt_keywords()`: Cultural keyword adaptation
- `validate_translations()`: Character limit validation
- `calculate_localization_roi()`: Estimate investment returns

### 7. review_analyzer.py
**Purpose**: Analyzes user reviews for actionable insights

**Key Functions**:
- `analyze_sentiment()`: Calculate sentiment distribution
- `extract_common_themes()`: Identify frequent topics
- `identify_issues()`: Surface bugs and problems
- `find_feature_requests()`: Extract desired features
- `track_sentiment_trends()`: Monitor changes over time
- `generate_response_templates()`: Create review responses

### 8. launch_checklist.py
**Purpose**: Generates comprehensive launch and update checklists

**Key Functions**:
- `generate_prelaunch_checklist()`: Complete submission validation
- `validate_app_store_compliance()`: Check guidelines compliance
- `create_update_plan()`: Plan update cadence
- `optimize_launch_timing()`: Recommend launch dates
- `plan_seasonal_campaigns()`: Identify seasonal opportunities

## Installation

### For Claude Code (Desktop/CLI)

#### Project-Level Installation
```bash
# Copy skill folder to project
cp -r app-store-optimization /path/to/your/project/.claude/skills/

# Claude will auto-load the skill when working in this project
```

#### User-Level Installation (Available in All Projects)
```bash
# Copy skill folder to user-level skills
cp -r app-store-optimization ~/.claude/skills/

# Claude will load this skill in all your projects
```

### For Claude Apps (Browser)

1. Use the `skill-creator` skill to import the skill
2. Or manually import via Claude Apps interface

### Verification

To verify installation:
```bash
# Check if skill folder exists
ls ~/.claude/skills/app-store-optimization/

# You should see:
# SKILL.md
# keyword_analyzer.py
# metadata_optimizer.py
# competitor_analyzer.py
# aso_scorer.py
# ab_test_planner.py
# localization_helper.py
# review_analyzer.py
# launch_checklist.py
# sample_input.json
# expected_output.json
# HOW_TO_USE.md
# README.md
```

## Usage Examples

### Example 1: Complete Keyword Research

```
Hey Claude—I just added the "app-store-optimization" skill. Can you research keywords for my fitness app? I'm targeting people who want home workouts, yoga, and meal planning. Analyze top competitors like Nike Training Club and Peloton.
```

**What Claude will do**:
- Use `keyword_analyzer.py` to research keywords
- Use `competitor_analyzer.py` to analyze Nike Training Club and Peloton
- Provide prioritized keyword list with search volumes, competition levels
- Identify gaps and long-tail opportunities
- Recommend primary keywords for title and secondary keywords for description

### Example 2: Optimize App Store Metadata

```
Hey Claude—I just added the "app-store-optimization" skill. Optimize my app's metadata for both Apple App Store and Google Play Store:
- App: FitFlow
- Category: Health & Fitness
- Features: AI workout plans, nutrition tracking, progress photos
- Keywords: fitness app, workout planner, home fitness
```

**What Claude will do**:
- Use `metadata_optimizer.py` to create optimized titles (multiple options)
- Generate platform-specific descriptions (short and full)
- Optimize Apple's 100-character keyword field
- Validate all character limits
- Calculate keyword density
- Provide before/after comparison

### Example 3: Calculate ASO Health Score

```
Hey Claude—I just added the "app-store-optimization" skill. Calculate my app's ASO score:
- Average rating: 4.3 stars (8,200 ratings)
- Keywords in top 10: 4
- Keywords in top 50: 15
- Conversion rate: 3.8%
- Title: "FitFlow - Home Workouts"
- Description: 1,500 characters with 3 keyword mentions
```

**What Claude will do**:
- Use `aso_scorer.py` to calculate overall score (0-100)
- Break down by category (Metadata: X/25, Ratings: X/25, Keywords: X/25, Conversion: X/25)
- Identify strengths and weaknesses
- Generate prioritized recommendations
- Estimate impact of improvements

### Example 4: A/B Test Planning

```
Hey Claude—I just added the "app-store-optimization" skill. I want to A/B test my app icon. My current conversion rate is 4.2%. How many visitors do I need and how long should I run the test?
```

**What Claude will do**:
- Use `ab_test_planner.py` to design test
- Calculate required sample size (based on minimum detectable effect)
- Estimate test duration for low/medium/high traffic scenarios
- Provide test structure and success metrics
- Explain how to analyze results

### Example 5: Review Sentiment Analysis

```
Hey Claude—I just added the "app-store-optimization" skill. Analyze my last 500 reviews and tell me:
- Overall sentiment
- Most common complaints
- Top feature requests
- Bugs needing immediate fixes
```

**What Claude will do**:
- Use `review_analyzer.py` to process reviews
- Calculate sentiment distribution
- Extract common themes
- Identify and prioritize issues
- Cluster feature requests
- Generate response templates

### Example 6: Pre-Launch Checklist

```
Hey Claude—I just added the "app-store-optimization" skill. Generate a complete pre-launch checklist for both app stores. My launch date is March 15, 2026.
```

**What Claude will do**:
- Use `launch_checklist.py` to generate checklists
- Create Apple App Store checklist (metadata, assets, technical, legal)
- Create Google Play Store checklist (metadata, assets, technical, legal)
- Add universal checklist (marketing, QA, support)
- Generate timeline with milestones
- Calculate completion percentage

## Best Practices

### Keyword Research
1. Start with 20-30 seed keywords
2. Analyze top 5 competitors in your category
3. Balance high-volume and long-tail keywords
4. Prioritize relevance over search volume
5. Update keyword research quarterly

### Metadata Optimization
1. Front-load keywords in title (first 15 characters most important)
2. Use every available character (don't waste space)
3. Write for humans first, search engines second
4. A/B test major changes before committing
5. Update descriptions with each major release

### A/B Testing
1. Test one element at a time (icon vs. screenshots vs. title)
2. Run tests to statistical significance (90%+ confidence)
3. Test high-impact elements first (icon has biggest impact)
4. Allow sufficient duration (at least 1 week, preferably 2-3)
5. Document learnings for future tests

### Localization
1. Start with top 5 revenue markets (US, China, Japan, Germany, UK)
2. Use professional translators, not machine translation
3. Test translations with native speakers
4. Adapt keywords for cultural context
5. Monitor ROI by market

### Review Management
1. Respond to reviews within 24-48 hours
2. Always be professional, even with negative reviews
3. Address specific issues raised
4. Thank users for positive feedback
5. Use insights to prioritize product improvements

## Technical Requirements

- **Python**: 3.7+ (for Python modules)
- **Platform Support**: Apple App Store, Google Play Store
- **Data Formats**: JSON input/output
- **Dependencies**: Standard library only (no external packages required)

## Limitations

### Data Dependencies
- Keyword search volumes are estimates (no official Apple/Google data)
- Competitor data limited to publicly available information
- Review analysis requires access to public reviews
- Historical data may not be available for new apps

### Platform Constraints
- Apple: Metadata changes require app submission (except Promotional Text)
- Google: Metadata changes take 1-2 hours to index
- A/B testing requires significant traffic for statistical significance
- Store algorithms are proprietary and change without notice

### Scope
- Does not include paid user acquisition (Apple Search Ads, Google Ads)
- Does not cover in-app analytics implementation
- Does not handle technical app development
- Focuses on organic discovery and conversion optimization

## Troubleshooting

### Issue: Python modules not found
**Solution**: Ensure all .py files are in the same directory as SKILL.md

### Issue: Character limit validation failing
**Solution**: Check that you're using the correct platform ('apple' or 'google')

### Issue: Keyword research returning limited results
**Solution**: Provide more context about your app, features, and target audience

### Issue: ASO score seems inaccurate
**Solution**: Ensure you're providing accurate metrics (ratings, keyword rankings, conversion rate)

## Version History

### Version 1.0.0 (November 7, 2025)
- Initial release
- 8 Python modules with comprehensive ASO capabilities
- Support for both Apple App Store and Google Play Store
- Keyword research, metadata optimization, competitor analysis
- ASO scoring, A/B testing, localization, review analysis
- Launch planning and seasonal campaign tools

## Support & Feedback

This skill is designed to help app developers and marketers succeed in competitive app marketplaces. For the best results:

1. Provide detailed context about your app
2. Include specific metrics when available
3. Ask follow-up questions for clarification
4. Iterate based on results

## Credits

Developed by Claude Skills Factory
Based on industry-standard ASO best practices
Platform requirements current as of November 2025

## License

This skill is provided as-is for use with Claude Code and Claude Apps. Customize and extend as needed for your specific use cases.

---

**Ready to optimize your app?** Start with keyword research, then move to metadata optimization, and finally implement A/B testing for continuous improvement. The skill handles everything from pre-launch planning to ongoing optimization.

For detailed usage examples, see [HOW_TO_USE.md](HOW_TO_USE.md).

FILE:references/aso-best-practices.md
# ASO Best Practices Reference

Optimization strategies for improving app store visibility, conversion, and rankings.

---

## Table of Contents

- [Keyword Optimization](#keyword-optimization)
- [Metadata Optimization](#metadata-optimization)
- [Visual Asset Optimization](#visual-asset-optimization)
- [Rating and Review Management](#rating-and-review-management)
- [Launch Strategy](#launch-strategy)
- [A/B Testing Framework](#ab-testing-framework)
- [Conversion Optimization](#conversion-optimization)
- [Common Mistakes to Avoid](#common-mistakes-to-avoid)

---

## Keyword Optimization

### Keyword Research Process

1. **Brainstorm seed keywords** - Core terms users search for
2. **Expand with variations** - Synonyms, related terms, long-tail
3. **Analyze competition** - Check difficulty scores
4. **Evaluate search volume** - Prioritize high-volume terms
5. **Test and iterate** - Monitor rankings and adjust

### Keyword Selection Criteria

| Factor | Weight | Evaluation Method |
|--------|--------|-------------------|
| Relevance | 40% | Does it describe app function? |
| Search Volume | 30% | Monthly search estimates |
| Competition | 20% | Number of ranking apps |
| Conversion Potential | 10% | User intent alignment |

### Keyword Placement Priority

| Location | Search Weight | Example |
|----------|---------------|---------|
| App Title | Highest | "TaskMaster - Todo List Manager" |
| Subtitle (iOS) | High | "Organize Your Daily Tasks" |
| Keyword Field (iOS) | High | "planner,reminder,checklist" |
| Short Description (Android) | High | "Simple task manager for busy professionals" |
| Full Description | Medium | Natural keyword usage throughout |

### Long-Tail Keyword Strategy

Long-tail keywords have lower volume but higher conversion:

| Type | Example | Volume | Competition | Conversion |
|------|---------|--------|-------------|------------|
| Short-tail | "todo app" | High | High | Low |
| Mid-tail | "daily task manager" | Medium | Medium | Medium |
| Long-tail | "free todo list with reminders" | Low | Low | High |

**Formula for keyword priority:**
```
Score = (Volume × 0.3) + (1/Competition × 0.3) + (Relevance × 0.4)
```

---

## Metadata Optimization

### Title Optimization

**Structure Formula:**
```
[Brand Name] - [Primary Keyword] [Secondary Keyword/Benefit]
```

**Examples by category:**

| Category | Before | After |
|----------|--------|-------|
| Productivity | "MyTasks" | "MyTasks - Todo List & Planner" |
| Fitness | "FitTrack" | "FitTrack: Workout & Gym Log" |
| Finance | "MoneyApp" | "MoneyApp - Budget Tracker" |
| Photo | "SnapEdit" | "SnapEdit: Photo Editor & AI" |

**Title Optimization Checklist:**
- [ ] Primary keyword within first 3 words
- [ ] Brand name is memorable and unique
- [ ] Character count matches platform limit
- [ ] No keyword stuffing
- [ ] Readable and natural sounding

### Description Optimization

**Full Description Structure:**

```
PARAGRAPH 1: Hook + Primary Benefit (50-100 words)
- Address user pain point
- State main value proposition
- Include primary keyword naturally

PARAGRAPH 2-3: Feature Highlights (100-150 words)
- Top 3-5 features with benefits
- Use bullet points or emojis for scanability
- Include secondary keywords

PARAGRAPH 4: Social Proof (50-75 words)
- Download numbers or ratings
- Press mentions or awards
- User testimonials (summarized)

PARAGRAPH 5: Call to Action (25-50 words)
- Clear next step
- Urgency or incentive
- Reassurance (free trial, no credit card)
```

**Keyword Density Target:**
- Primary keyword: 2-3% (8-12 mentions in 4000 chars)
- Secondary keywords: 1-2% each (4-8 mentions each)

### Subtitle Optimization (iOS)

**Effective Subtitle Formulas:**

| Formula | Example |
|---------|---------|
| [Verb] + [Benefit] | "Organize Your Life" |
| [Adjective] + [Category] | "Smart Task Manager" |
| [Feature] + [Feature] | "Lists, Reminders & Notes" |
| [Audience] + [Solution] | "For Busy Professionals" |

---

## Visual Asset Optimization

### App Icon Best Practices

| Principle | Do | Don't |
|-----------|-----|-------|
| Simplicity | Single focal element | Multiple competing elements |
| Recognizability | Works at 60x60px | Requires large size to read |
| Uniqueness | Distinct from competitors | Generic category icon |
| Color | Bold, contrasting colors | Muted or similar to background |
| Text | None or single letter | Full words or app name |

**Icon Testing Questions:**
1. Is it recognizable at 29x29px (smallest iOS size)?
2. Does it stand out in search results?
3. Does it communicate app function?
4. Is it distinct from top 10 category competitors?

### Screenshot Optimization

**Screenshot Hierarchy:**

| Position | Purpose | Content Strategy |
|----------|---------|------------------|
| Screenshot 1 | Hook/Hero | Main value proposition + key UI |
| Screenshot 2 | Primary Feature | Most-used feature demonstration |
| Screenshot 3 | Secondary Feature | Differentiating capability |
| Screenshot 4 | Social Proof | Ratings, awards, user count |
| Screenshot 5+ | Additional Features | Supporting functionality |

**Caption Best Practices:**
- Maximum 5-7 words per caption
- Action-oriented verbs ("Track", "Organize", "Discover")
- Benefit-focused, not feature-focused
- Consistent typography and style

**Example Caption Evolution:**

| Weak | Better | Best |
|------|--------|------|
| "Task List Feature" | "Create Task Lists" | "Never Forget a Task Again" |
| "Calendar View" | "See Your Schedule" | "Plan Your Week in Seconds" |
| "Notifications" | "Get Reminders" | "Stay on Top of Deadlines" |

### Video Preview Strategy

**Video Structure (30 seconds):**

| Seconds | Content |
|---------|---------|
| 0-5 | Hook: Show end result or main benefit |
| 5-15 | Demo: Core feature in action |
| 15-25 | Features: Quick feature montage |
| 25-30 | CTA: Logo and download prompt |

---

## Rating and Review Management

### Review Response Framework

**For Negative Reviews (1-2 stars):**

```
Structure:
1. Acknowledge the issue (1 sentence)
2. Apologize without making excuses (1 sentence)
3. Offer solution or next step (1-2 sentences)
4. Invite direct contact (1 sentence)

Example:
"We're sorry the syncing issues are affecting your experience.
Our team is actively working on a fix for the next update.
In the meantime, please try logging out and back in, which
resolves this for most users. If issues persist, email us at
support@app.com and we'll prioritize your case."
```

**For Positive Reviews (4-5 stars):**

```
Structure:
1. Thank sincerely (1 sentence)
2. Acknowledge specific praise (1 sentence)
3. Encourage continued use or sharing (1 sentence)

Example:
"Thank you for the kind words! We're thrilled the reminder
feature helps you stay organized. If you're enjoying the app,
we'd love if you'd share it with friends who might benefit."
```

### Rating Improvement Tactics

| Tactic | Implementation | Expected Impact |
|--------|----------------|-----------------|
| In-app prompt timing | After positive action (task completed, milestone reached) | +0.3 stars |
| Bug fix velocity | Address 1-star issues within 7 days | +0.2 stars |
| Response rate | Reply to 80%+ of reviews | +0.1 stars |
| Feature requests | Implement top-requested features | +0.2 stars |

### Review Prompt Best Practices

**When to prompt:**
- After user completes 5+ successful sessions
- After milestone achievement (first task completed, 7-day streak)
- After positive in-app feedback ("Was this helpful? Yes")

**When NOT to prompt:**
- First session
- After error or crash
- During critical workflow
- More than once per 30 days

---

## Launch Strategy

### Pre-Launch Checklist

**4 Weeks Before Launch:**
- [ ] Finalize app name and keywords
- [ ] Complete all metadata fields
- [ ] Prepare all visual assets
- [ ] Set up analytics (Firebase, Mixpanel)
- [ ] Create press kit and media assets
- [ ] Build email list for launch notification

**2 Weeks Before Launch:**
- [ ] Submit for app review
- [ ] Prepare social media content
- [ ] Brief press and influencers
- [ ] Set up review response templates
- [ ] Configure in-app rating prompts

**Launch Day:**
- [ ] Verify app is live in stores
- [ ] Announce across all channels
- [ ] Monitor reviews and respond quickly
- [ ] Track download velocity
- [ ] Document any issues for Day 2 fix

### Update Cadence

| Update Type | Frequency | ASO Impact |
|-------------|-----------|------------|
| Bug fixes | As needed | Prevents rating drops |
| Minor features | Every 2-4 weeks | Maintains freshness signal |
| Major features | Every 4-8 weeks | Opportunity for "What's New" |
| Metadata refresh | Every 4-6 weeks | Keyword optimization cycle |

### Seasonal Optimization

| Season | Optimization Focus | Example Categories |
|--------|--------------------|--------------------|
| Jan (New Year) | Resolutions, goals | Fitness, Productivity |
| Feb (Valentine's) | Dating, relationships | Dating, Photo |
| Mar-Apr (Tax) | Finance, organization | Finance, Productivity |
| May-Jun (Summer) | Travel, fitness | Travel, Health |
| Aug-Sep (Back to School) | Education, organization | Education, Productivity |
| Nov-Dec (Holidays) | Shopping, social | Shopping, Social |

---

## A/B Testing Framework

### Test Prioritization Matrix

| Element | Impact | Ease | Priority |
|---------|--------|------|----------|
| App Icon | High | Medium | 1 |
| Screenshot 1 | High | Medium | 2 |
| Title | High | Easy | 3 |
| Short Description | Medium | Easy | 4 |
| Screenshots 2-5 | Medium | Medium | 5 |
| Video | Medium | Hard | 6 |

### Sample Size Calculator

**Formula:**
```
Sample Size = (2 × (Z² × p × (1-p))) / E²

Where:
Z = 1.96 (for 95% confidence)
p = baseline conversion rate
E = minimum detectable effect (usually 0.05)
```

**Quick Reference:**

| Baseline CVR | Min. Impressions for 5% Lift |
|--------------|------------------------------|
| 1% | 31,000 per variant |
| 2% | 15,500 per variant |
| 5% | 6,200 per variant |
| 10% | 3,100 per variant |

### Test Duration Guidelines

| Daily Impressions | Minimum Test Duration |
|-------------------|----------------------|
| 1,000+ | 7 days |
| 500-1,000 | 14 days |
| 100-500 | 30 days |
| <100 | Not recommended |

---

## Conversion Optimization

### Conversion Funnel Metrics

| Stage | Metric | Benchmark |
|-------|--------|-----------|
| Discovery | Impressions | Category dependent |
| Consideration | Page Views | 30-50% of impressions |
| Conversion | Installs | 3-8% of page views |
| Activation | First Open | 70-90% of installs |

### Conversion Optimization Levers

| Lever | Typical Lift | Effort |
|-------|--------------|--------|
| Icon redesign | 10-25% | High |
| Screenshot optimization | 15-35% | Medium |
| Title keyword optimization | 5-15% | Low |
| Description rewrite | 5-10% | Low |
| Video addition | 10-20% | High |
| Localization | 20-50% per market | Medium |

---

## Common Mistakes to Avoid

### Keyword Mistakes

| Mistake | Problem | Solution |
|---------|---------|----------|
| Keyword stuffing | Spam detection, rejection | Natural usage, 2-3% density |
| Competitor names | Guideline violation | Focus on category terms |
| Duplicate keywords | Wasted character space | Remove duplicates from keyword field |
| Ignoring long-tail | Missing conversion | Include specific phrases |

### Metadata Mistakes

| Mistake | Problem | Solution |
|---------|---------|----------|
| Vague descriptions | Low conversion | Specific benefits and features |
| Feature-focused copy | Doesn't resonate | Benefit-focused messaging |
| Outdated information | Misleading users | Update with each release |
| Missing localization | Lost global revenue | Prioritize top 5 markets |

### Visual Asset Mistakes

| Mistake | Problem | Solution |
|---------|---------|----------|
| Text-heavy screenshots | Unreadable on phones | Minimal text, clear UI focus |
| Inconsistent style | Unprofessional appearance | Design system for all assets |
| Portrait-only screenshots | Missed tablet users | Include landscape variants |
| No social proof | Lower trust | Add ratings, awards, press |

### Launch Mistakes

| Mistake | Problem | Solution |
|---------|---------|----------|
| Launching on Friday | No support over weekend | Launch Tuesday-Wednesday |
| No analytics setup | Can't measure success | Firebase/Mixpanel before launch |
| Immediate rating prompt | Negative ratings | Wait for positive experience |
| Ignoring reviews | Declining ratings | Respond within 24-48 hours |

FILE:references/keyword-research-guide.md
# Keyword Research Guide

Systematic approach to discovering, evaluating, and selecting keywords for app store optimization.

---

## Table of Contents

- [Keyword Research Methodology](#keyword-research-methodology)
- [Keyword Evaluation Framework](#keyword-evaluation-framework)
- [Competitor Keyword Analysis](#competitor-keyword-analysis)
- [Keyword Mapping Strategy](#keyword-mapping-strategy)
- [Keyword Tracking and Iteration](#keyword-tracking-and-iteration)

---

## Keyword Research Methodology

### Phase 1: Seed Keyword Generation

Start by generating initial keyword ideas from multiple sources.

**Source 1: Core App Functions**

List every action or problem the app solves:

```
Example for a task management app:
- Create tasks
- Set reminders
- Track deadlines
- Organize projects
- Collaborate with team
- Plan daily schedule
```

**Source 2: User Language Mapping**

Match developer terminology to user searches:

| Developer Term | User Search Terms |
|----------------|-------------------|
| Task management | todo list, task app, tasks |
| Project organization | project planner, project tracker |
| Deadline tracking | due date reminder, deadline app |
| Time blocking | schedule planner, calendar app |
| GTD methodology | getting things done, productivity system |

**Source 3: App Store Autocomplete**

Type seed keywords into App Store/Play Store search and record suggestions:

```
"todo" → todo list, todo app, todo list app, todolist widget
"task" → task manager, task planner, task list, tasks to do
"remind" → reminder app, reminder, reminders widget, remind me
```

**Source 4: Competitor Analysis**

Extract keywords from top 10 competitors in category (detailed in section below).

### Phase 2: Keyword Expansion

**Expansion Techniques:**

| Technique | Example (seed: "todo") |
|-----------|------------------------|
| Add modifiers | free todo, best todo, simple todo |
| Add actions | make todo list, create todo, organize todo |
| Add platforms | todo app iphone, todo for mac, todo widget |
| Add audiences | todo for students, business todo, family todo |
| Add features | todo with reminders, todo calendar, todo sync |
| Add problems | forgot tasks todo, procrastination todo |

**Keyword Matrix Template:**

| Core Term | Modifier 1 | Modifier 2 | Full Keyword |
|-----------|------------|------------|--------------|
| todo | free | app | free todo app |
| todo | best | iphone | best todo iphone |
| task | manager | simple | simple task manager |
| reminder | daily | widget | daily reminder widget |
| planner | weekly | calendar | weekly planner calendar |

### Phase 3: Keyword Filtering

Remove irrelevant or low-quality keywords:

**Exclusion Criteria:**

| Criterion | Reason | Example |
|-----------|--------|---------|
| Competitor brand names | Policy violation | "todoist alternative" |
| Unrelated categories | Low conversion | "todo games" |
| Plural duplicates (iOS) | Wasted space | "tasks" when "task" exists |
| Single characters | No search value | "to do" vs "todo" |

---

## Keyword Evaluation Framework

### Keyword Scoring Model

Evaluate each keyword on four dimensions:

**1. Search Volume (0-100)**

| Volume Level | Score | Monthly Searches |
|--------------|-------|------------------|
| Very High | 80-100 | 50,000+ |
| High | 60-79 | 10,000-49,999 |
| Medium | 40-59 | 1,000-9,999 |
| Low | 20-39 | 100-999 |
| Very Low | 0-19 | <100 |

**2. Competition (0-100, inverted)**

| Competition | Score | Top 10 App Ratings |
|-------------|-------|-------------------|
| Very Low | 80-100 | Average <4.0 stars |
| Low | 60-79 | Average 4.0-4.2 stars |
| Medium | 40-59 | Average 4.3-4.5 stars |
| High | 20-39 | Average 4.6-4.8 stars |
| Very High | 0-19 | Average 4.9+ stars |

**3. Relevance (0-100)**

| Relevance | Score | Criteria |
|-----------|-------|----------|
| Exact Match | 90-100 | Keyword describes core function |
| Strong Match | 70-89 | Keyword describes major feature |
| Moderate Match | 50-69 | Keyword describes secondary feature |
| Weak Match | 30-49 | Keyword tangentially related |
| No Match | 0-29 | Keyword unrelated to app |

**4. Conversion Potential (0-100)**

| Intent | Score | User Query Type |
|--------|-------|-----------------|
| Transactional | 80-100 | "best [app type]", "[app type] app" |
| Commercial | 60-79 | "free [app type]", "[app type] for [use]" |
| Informational | 40-59 | "how to [action]", "what is [concept]" |
| Navigational | 20-39 | "[brand name]", "[specific app]" |

### Composite Score Calculation

```
Keyword Score = (Volume × 0.25) + (Competition × 0.25) +
                (Relevance × 0.35) + (Conversion × 0.15)
```

**Score Interpretation:**

| Score Range | Priority | Action |
|-------------|----------|--------|
| 80-100 | Primary | Target in title and keyword field |
| 60-79 | Secondary | Include in subtitle/description |
| 40-59 | Tertiary | Use in long description only |
| 0-39 | Deprioritize | Do not target |

### Keyword Evaluation Worksheet

```
KEYWORD EVALUATION

Keyword: "task manager app"
Date: [Date]

SCORES:
├── Search Volume: 72/100 (High - ~25,000/month)
├── Competition: 45/100 (Medium - 4.4 avg rating in top 10)
├── Relevance: 95/100 (Exact match to core function)
└── Conversion: 85/100 (Transactional intent)

COMPOSITE SCORE: 74.5/100

RECOMMENDATION: Secondary Priority
- Include in subtitle or short description
- Not competitive enough for title (dominated by Todoist, Any.do)
- Consider long-tail variant: "simple task manager app"
```

---

## Competitor Keyword Analysis

### Competitor Identification

**Step 1: Direct Competitors**
Apps solving the same problem for the same audience.

**Step 2: Indirect Competitors**
Apps solving related problems or targeting overlapping audiences.

**Step 3: Category Leaders**
Top 10-20 apps by downloads in primary category.

### Competitor Keyword Extraction

**From App Title:**
```
Competitor: "Todoist: To-Do List & Tasks"
Keywords: todoist, to-do list, tasks, to do
```

**From Subtitle (iOS):**
```
Competitor subtitle: "Task Manager & Planner"
Keywords: task manager, planner
```

**From Description (First 100 words):**
Identify frequently used terms:
```
"Todoist is the world's favorite task manager and to-do list app.
Organize work and life, hit your goals, and find productivity..."

Extracted: task manager, to-do list, organize, goals, productivity
```

### Competitor Keyword Matrix

| Keyword | Comp 1 | Comp 2 | Comp 3 | Comp 4 | Comp 5 | Coverage |
|---------|--------|--------|--------|--------|--------|----------|
| task manager | ✓ | ✓ | ✓ | ✓ | ✓ | 100% |
| to-do list | ✓ | ✓ | ✓ | ✓ | | 80% |
| planner | ✓ | ✓ | | ✓ | ✓ | 80% |
| reminder | ✓ | ✓ | ✓ | | | 60% |
| productivity | ✓ | | ✓ | ✓ | | 60% |
| checklist | | ✓ | | ✓ | ✓ | 60% |
| project | ✓ | ✓ | | | | 40% |
| habit | | | ✓ | | ✓ | 40% |

**Analysis:**
- 100% coverage = Highly competitive, essential keyword
- 60-80% coverage = Important category term
- 40% coverage = Potential differentiator
- <40% coverage = Unique opportunity or irrelevant

### Keyword Gap Analysis

Identify keywords competitors miss:

```
KEYWORD GAP ANALYSIS

Underserved Keywords (Low competitor coverage, decent volume):
1. "daily planner widget" - 2/10 competitors, 5,000 searches
2. "task list for teams" - 3/10 competitors, 3,500 searches
3. "todo with calendar sync" - 1/10 competitors, 2,800 searches

Opportunity Assessment:
- "daily planner widget" → Add widget feature, target keyword
- "task list for teams" → Already have feature, update metadata
- "todo with calendar sync" → Feature gap, add to roadmap
```

---

## Keyword Mapping Strategy

### Keyword Placement Map

Assign each keyword to specific metadata locations:

```
KEYWORD PLACEMENT MAP

PRIMARY (Title + Keyword Field):
├── task manager (Score: 82)
├── todo list (Score: 78)
└── planner (Score: 75)

SECONDARY (Subtitle + Short Description):
├── reminder app (Score: 68)
├── daily tasks (Score: 65)
└── organize (Score: 62)

TERTIARY (Full Description):
├── checklist (Score: 55)
├── productivity (Score: 52)
├── schedule (Score: 48)
├── deadline (Score: 45)
└── project management (Score: 42)
```

### iOS Keyword Field Strategy

**100 Character Optimization:**

```
STEP 1: List all target keywords
task,manager,todo,list,planner,reminder,organize,daily,checklist,
productivity,schedule,deadline,project,goals,habit,widget,sync,
team,collaborate,notes,calendar

STEP 2: Remove duplicates from title
Title: "TaskFlow - Todo List Manager"
Remove: task, todo, list, manager

STEP 3: Remove plurals
Keep: reminder (not reminders)
Keep: goal (not goals)

STEP 4: Prioritize by score and fit
Final 100 chars:
planner,reminder,organize,daily,checklist,productivity,schedule,
deadline,project,goals,habit,widget,sync,team,collaborate

Character count: 98/100
```

### Android Description Keyword Integration

**Natural keyword placement in 4,000 characters:**

```
PARAGRAPH 1 (Hook - 300 chars):
Keywords: task manager, todo list, organize
"TaskFlow is the task manager trusted by 2 million users. Create
your perfect todo list and organize everything that matters..."

PARAGRAPH 2 (Features - 800 chars):
Keywords: reminder, checklist, deadline, project
"Set smart reminders that notify you at the right time. Build
checklists for any project. Never miss a deadline with..."

PARAGRAPH 3 (Benefits - 600 chars):
Keywords: productivity, schedule, goals
"Boost your productivity with proven planning methods. Schedule
your day in minutes. Track goals and celebrate..."

PARAGRAPH 4 (Differentiators - 500 chars):
Keywords: widget, sync, team, collaborate
"Beautiful widgets keep tasks visible. Sync across all devices
instantly. Invite your team to collaborate on..."

Total keyword coverage: 14 keywords naturally integrated
```

---

## Keyword Tracking and Iteration

### Ranking Tracking Cadence

| Frequency | Action |
|-----------|--------|
| Daily | Track top 5-10 primary keywords |
| Weekly | Full keyword set review |
| Monthly | Competitor keyword comparison |
| Quarterly | Full keyword research refresh |

### Keyword Performance Metrics

| Metric | Target | Action if Below |
|--------|--------|-----------------|
| Top 10 ranking | 3+ keywords | Increase keyword weight |
| Top 50 ranking | 10+ keywords | Maintain current strategy |
| Ranking velocity | Improving trend | Continue optimization |
| Conversion rate | >5% | Review relevance alignment |

### Iteration Process

**Monthly Keyword Audit:**

```
1. EXPORT current rankings
   - List all tracked keywords
   - Record current position
   - Note 30-day trend (up/down/stable)

2. IDENTIFY opportunities
   - Keywords improving but not top 10
   - Keywords declining from previous position
   - New high-volume keywords in category

3. PRIORITIZE changes
   - Boost: Keywords at position 11-20
   - Maintain: Keywords at position 1-10
   - Replace: Keywords at position 50+ with no improvement

4. IMPLEMENT updates
   - Adjust keyword field (iOS)
   - Update description (Android)
   - Modify subtitle if needed

5. DOCUMENT changes
   - Record what changed and why
   - Set reminder for 2-week check-in
```

### Keyword Testing Log Template

```
KEYWORD TEST LOG

Test ID: KW-2025-001
Date Started: [Date]
Keywords Changed:
  - Added: "habit tracker" (replacing "goals app")
  - Added: "daily routine" (replacing "schedule planner")

Rationale:
- "habit tracker" has 3x volume of "goals app"
- "daily routine" trending up 40% in category

Baseline Rankings:
- "habit tracker": Not ranked
- "daily routine": Position 87

30-Day Results:
- "habit tracker": Position 34 (+53)
- "daily routine": Position 28 (+59)

Conclusion: Test successful - retain new keywords
Next Action: Target subtitle position for "habit tracker"
```

FILE:references/platform-requirements.md
# Platform Requirements Reference

Technical specifications and metadata requirements for Apple App Store and Google Play Store.

---

## Table of Contents

- [Apple App Store Requirements](#apple-app-store-requirements)
- [Google Play Store Requirements](#google-play-store-requirements)
- [Visual Asset Specifications](#visual-asset-specifications)
- [Localization Requirements](#localization-requirements)
- [Compliance Guidelines](#compliance-guidelines)

---

## Apple App Store Requirements

### Metadata Character Limits

| Field | Character Limit | Notes |
|-------|----------------|-------|
| App Name (Title) | 30 characters | Visible in search results |
| Subtitle | 30 characters | iOS 11+ only, appears below title |
| Promotional Text | 170 characters | Editable without app update |
| Description | 4,000 characters | Not indexed for search |
| Keywords Field | 100 characters | Comma-separated, no spaces after commas |
| What's New | 4,000 characters | Release notes for updates |
| Developer Name | 255 characters | Company or individual name |
| Support URL | Required | Must be valid HTTPS URL |
| Privacy Policy URL | Required | Must be valid HTTPS URL |

### Keyword Field Optimization Rules

1. **No duplicates** - Words in title are already indexed
2. **No plurals** - Apple indexes both singular and plural forms
3. **No spaces after commas** - Wastes character space
4. **No brand names** - Violates App Store guidelines
5. **No category names** - Already indexed via category selection

**Example - Efficient keyword field:**
```
task,todo,checklist,reminder,productivity,organize,schedule,planner,goals,habit
```

**Example - Inefficient keyword field (avoid):**
```
task manager, todo list, productivity app, task tracking
```

### App Store Connect Metadata Fields

| Category | Field | Required |
|----------|-------|----------|
| **App Information** | Name | Yes |
| | Subtitle | No |
| | Category | Yes |
| | Secondary Category | No |
| | Content Rights | Yes |
| | Age Rating | Yes |
| **Version Information** | Description | Yes |
| | Keywords | Yes |
| | Promotional Text | No |
| | What's New | Yes (for updates) |
| | Support URL | Yes |
| | Marketing URL | No |
| **Pricing** | Price Tier | Yes |
| | Availability | Yes |

### Age Rating Content Descriptors

| Content Type | None | Infrequent | Frequent |
|--------------|------|------------|----------|
| Cartoon Violence | 4+ | 9+ | 12+ |
| Realistic Violence | 9+ | 12+ | 17+ |
| Sexual Content | 12+ | 17+ | 17+ |
| Profanity | 4+ | 12+ | 17+ |
| Alcohol/Drug Reference | 12+ | 17+ | 17+ |
| Gambling | 12+ | 17+ | 17+ |
| Horror/Fear | 9+ | 12+ | 17+ |

---

## Google Play Store Requirements

### Metadata Character Limits

| Field | Character Limit | Notes |
|-------|----------------|-------|
| App Title | 50 characters | Increased from 30 in 2021 |
| Short Description | 80 characters | Visible on store listing |
| Full Description | 4,000 characters | Indexed for search keywords |
| Developer Name | 64 characters | Organization or individual |
| Developer Email | Required | Public support contact |
| Privacy Policy URL | Required | Must be valid HTTPS URL |

### Description Keyword Strategy

Google Play has no separate keyword field. Keywords are extracted from:

1. **App Title** - Highest weight, most important
2. **Short Description** - High weight, visible in search
3. **Full Description** - Medium weight, use naturally throughout
4. **Developer Name** - Low weight but indexed

**Keyword Density Guidelines:**
- Primary keyword: 2-3% density in full description
- Secondary keywords: 1-2% each
- Avoid keyword stuffing (>5% triggers spam detection)

### Google Play Console Metadata

| Category | Field | Required |
|----------|-------|----------|
| **Store Listing** | Title | Yes |
| | Short Description | Yes |
| | Full Description | Yes |
| | App Icon | Yes |
| | Feature Graphic | Yes |
| | Screenshots | Yes (min 2) |
| | Video | No |
| **Store Settings** | App Category | Yes |
| | Tags | No |
| | Contact Email | Yes |
| | Privacy Policy | Yes |
| **Content Rating** | IARC Questionnaire | Yes |

### Content Rating (IARC)

| Rating | Age | Description |
|--------|-----|-------------|
| PEGI 3 / Everyone | 3+ | Suitable for all ages |
| PEGI 7 / Everyone 10+ | 7+ | Mild violence, comic mischief |
| PEGI 12 / Teen | 12+ | Moderate violence, mild language |
| PEGI 16 / Mature 17+ | 16+ | Intense violence, strong language |
| PEGI 18 / Adults Only | 18+ | Extreme content |

---

## Visual Asset Specifications

### App Icon Requirements

**Apple App Store:**

| Device | Size | Format |
|--------|------|--------|
| iPhone | 1024x1024 px | PNG, no alpha |
| iPad | 1024x1024 px | PNG, no alpha |
| App Store | 1024x1024 px | PNG, no alpha |
| Spotlight | 120x120 px | PNG |
| Settings | 87x87 px | PNG |

**Google Play Store:**

| Asset | Size | Format |
|-------|------|--------|
| App Icon | 512x512 px | PNG, 32-bit |
| Feature Graphic | 1024x500 px | PNG or JPG |
| Promo Graphic | 180x120 px | PNG or JPG |
| TV Banner | 1280x720 px | PNG or JPG |

### Screenshot Requirements

**Apple App Store:**

| Device | Portrait | Landscape |
|--------|----------|-----------|
| iPhone 6.9" | 1320x2868 px | 2868x1320 px |
| iPhone 6.5" | 1290x2796 px | 2796x1290 px |
| iPhone 5.5" | 1242x2208 px | 2208x1242 px |
| iPad Pro 12.9" | 2048x2732 px | 2732x2048 px |
| iPad 10.5" | 1668x2224 px | 2224x1668 px |

- Minimum: 2 screenshots per device
- Maximum: 10 screenshots per device
- Format: PNG or JPG, no alpha channel
- First 3 screenshots are critical (most users don't scroll)

**Google Play Store:**

| Device | Dimensions | Notes |
|--------|------------|-------|
| Phone | 320-3840 px | Min 2:1 aspect ratio |
| 7" Tablet | 320-3840 px | Min 2:1 aspect ratio |
| 10" Tablet | 320-3840 px | Min 2:1 aspect ratio |
| Chromebook | 320-3840 px | Optional |
| TV | 320-3840 px | For TV apps only |

- Minimum: 2 screenshots
- Maximum: 8 screenshots
- Format: PNG or JPG
- No transparency or borders

### App Preview Video

**Apple App Store:**
- Duration: 15-30 seconds
- Resolution: Match device screenshot size
- Format: M4V, MP4, MOV
- Frame rate: 30 fps
- Audio: Optional but recommended

**Google Play Store:**
- YouTube video link only
- No duration limit (recommend under 2 minutes)
- Landscape orientation preferred
- Must not contain age-restricted content

---

## Localization Requirements

### Priority Markets by Revenue

| Rank | Market | Language Code |
|------|--------|---------------|
| 1 | United States | en-US |
| 2 | Japan | ja |
| 3 | United Kingdom | en-GB |
| 4 | Germany | de-DE |
| 5 | China | zh-Hans (iOS), zh-CN (Android) |
| 6 | South Korea | ko |
| 7 | France | fr-FR |
| 8 | Canada | en-CA, fr-CA |
| 9 | Australia | en-AU |
| 10 | Russia | ru |

### Apple App Store Localization

Supported localizations: 40+ languages

| Language | Locale Code |
|----------|-------------|
| English (US) | en-US |
| English (UK) | en-GB |
| Spanish | es-ES |
| Spanish (Mexico) | es-MX |
| French | fr-FR |
| German | de-DE |
| Japanese | ja |
| Korean | ko |
| Simplified Chinese | zh-Hans |
| Traditional Chinese | zh-Hant |

### Google Play Store Localization

Supported localizations: 75+ languages

Each locale requires:
- Title (50 chars)
- Short description (80 chars)
- Full description (4,000 chars)
- Screenshots (can reuse or localize)

---

## Compliance Guidelines

### Apple App Store Review Guidelines Summary

| Category | Key Requirements |
|----------|------------------|
| **Safety** | No objectionable content, privacy protection |
| **Performance** | App must work as described, no crashes |
| **Business** | Accurate app description, clear pricing |
| **Design** | Follow Human Interface Guidelines |
| **Legal** | Comply with local laws, proper licensing |

**Common Rejection Reasons:**
1. Bugs and crashes (50%+ of rejections)
2. Broken links or placeholder content
3. Misleading app descriptions
4. Privacy policy missing or incomplete
5. In-app purchase issues

### Google Play Developer Policies

| Policy Area | Requirements |
|-------------|--------------|
| **Restricted Content** | No hate speech, violence, gambling (without license) |
| **Privacy** | Data collection disclosure, privacy policy |
| **Monetization** | Clear pricing, compliant IAPs |
| **Ads** | No deceptive ads, proper disclosure |
| **Store Listing** | Accurate description, no keyword stuffing |

**Common Suspension Reasons:**
1. Policy violation (content, ads, permissions)
2. Repetitive content (clone apps)
3. Impersonation (fake apps)
4. Intellectual property infringement
5. Malicious behavior

### Privacy Requirements

**Apple (App Tracking Transparency):**
- ATT prompt required for tracking
- Privacy nutrition labels mandatory
- Data collection disclosure required

**Google (Data Safety):**
- Data safety section mandatory
- Data collection and sharing disclosure
- Security practices declaration

---

## Quick Reference Card

### Apple vs Google Comparison

| Attribute | Apple App Store | Google Play Store |
|-----------|-----------------|-------------------|
| Title Length | 30 chars | 50 chars |
| Subtitle | 30 chars | N/A |
| Short Description | N/A | 80 chars |
| Full Description | 4,000 chars | 4,000 chars |
| Keywords Field | 100 chars | N/A (in description) |
| Promotional Text | 170 chars | N/A |
| Icon Size | 1024x1024 px | 512x512 px |
| Min Screenshots | 2 | 2 |
| Max Screenshots | 10 | 8 |
| Review Time | 24-48 hours | 1-7 days |
| Metadata Update | Requires review | 1-2 hours to index |

FILE:sample_input.json
{
  "request_type": "keyword_research",
  "app_info": {
    "name": "TaskFlow Pro",
    "category": "Productivity",
    "target_audience": "Professionals aged 25-45 working in teams",
    "key_features": [
      "AI-powered task prioritization",
      "Team collaboration tools",
      "Calendar integration",
      "Cross-platform sync"
    ],
    "unique_value": "AI automatically prioritizes your tasks based on deadlines and importance"
  },
  "target_keywords": [
    "task manager",
    "productivity app",
    "todo list",
    "team collaboration",
    "project management"
  ],
  "competitors": [
    "Todoist",
    "Any.do",
    "Microsoft To Do",
    "Things 3"
  ],
  "platform": "both",
  "language": "en-US"
}

FILE:scripts/ab_test_planner.py
"""
A/B testing module for App Store Optimization.
Plans and tracks A/B tests for metadata and visual assets.
"""

from typing import Dict, List, Any, Optional
import math


class ABTestPlanner:
    """Plans and tracks A/B tests for ASO elements."""

    # Minimum detectable effect sizes (conservative estimates)
    MIN_EFFECT_SIZES = {
        'icon': 0.10,  # 10% conversion improvement
        'screenshot': 0.08,  # 8% conversion improvement
        'title': 0.05,  # 5% conversion improvement
        'description': 0.03  # 3% conversion improvement
    }

    # Statistical confidence levels
    CONFIDENCE_LEVELS = {
        'high': 0.95,  # 95% confidence
        'standard': 0.90,  # 90% confidence
        'exploratory': 0.80  # 80% confidence
    }

    def __init__(self):
        """Initialize A/B test planner."""
        self.active_tests = []

    def design_test(
        self,
        test_type: str,
        variant_a: Dict[str, Any],
        variant_b: Dict[str, Any],
        hypothesis: str,
        success_metric: str = 'conversion_rate'
    ) -> Dict[str, Any]:
        """
        Design an A/B test with hypothesis and variables.

        Args:
            test_type: Type of test ('icon', 'screenshot', 'title', 'description')
            variant_a: Control variant details
            variant_b: Test variant details
            hypothesis: Expected outcome hypothesis
            success_metric: Metric to optimize

        Returns:
            Test design with configuration
        """
        test_design = {
            'test_id': self._generate_test_id(test_type),
            'test_type': test_type,
            'hypothesis': hypothesis,
            'variants': {
                'a': {
                    'name': 'Control',
                    'details': variant_a,
                    'traffic_split': 0.5
                },
                'b': {
                    'name': 'Variation',
                    'details': variant_b,
                    'traffic_split': 0.5
                }
            },
            'success_metric': success_metric,
            'secondary_metrics': self._get_secondary_metrics(test_type),
            'minimum_effect_size': self.MIN_EFFECT_SIZES.get(test_type, 0.05),
            'recommended_confidence': 'standard',
            'best_practices': self._get_test_best_practices(test_type)
        }

        self.active_tests.append(test_design)
        return test_design

    def calculate_sample_size(
        self,
        baseline_conversion: float,
        minimum_detectable_effect: float,
        confidence_level: str = 'standard',
        power: float = 0.80
    ) -> Dict[str, Any]:
        """
        Calculate required sample size for statistical significance.

        Args:
            baseline_conversion: Current conversion rate (0-1)
            minimum_detectable_effect: Minimum effect size to detect (0-1)
            confidence_level: 'high', 'standard', or 'exploratory'
            power: Statistical power (typically 0.80 or 0.90)

        Returns:
            Sample size calculation with duration estimates
        """
        alpha = 1 - self.CONFIDENCE_LEVELS[confidence_level]
        beta = 1 - power

        # Expected conversion for variant B
        expected_conversion_b = baseline_conversion * (1 + minimum_detectable_effect)

        # Z-scores for alpha and beta
        z_alpha = self._get_z_score(1 - alpha / 2)  # Two-tailed test
        z_beta = self._get_z_score(power)

        # Pooled standard deviation
        p_pooled = (baseline_conversion + expected_conversion_b) / 2
        sd_pooled = math.sqrt(2 * p_pooled * (1 - p_pooled))

        # Sample size per variant
        n_per_variant = math.ceil(
            ((z_alpha + z_beta) ** 2 * sd_pooled ** 2) /
            ((expected_conversion_b - baseline_conversion) ** 2)
        )

        total_sample_size = n_per_variant * 2

        # Estimate duration based on typical traffic
        duration_estimates = self._estimate_test_duration(
            total_sample_size,
            baseline_conversion
        )

        return {
            'sample_size_per_variant': n_per_variant,
            'total_sample_size': total_sample_size,
            'baseline_conversion': baseline_conversion,
            'expected_conversion_improvement': minimum_detectable_effect,
            'expected_conversion_b': expected_conversion_b,
            'confidence_level': confidence_level,
            'statistical_power': power,
            'duration_estimates': duration_estimates,
            'recommendations': self._generate_sample_size_recommendations(
                n_per_variant,
                duration_estimates
            )
        }

    def calculate_significance(
        self,
        variant_a_conversions: int,
        variant_a_visitors: int,
        variant_b_conversions: int,
        variant_b_visitors: int
    ) -> Dict[str, Any]:
        """
        Calculate statistical significance of test results.

        Args:
            variant_a_conversions: Conversions for control
            variant_a_visitors: Visitors for control
            variant_b_conversions: Conversions for variation
            variant_b_visitors: Visitors for variation

        Returns:
            Significance analysis with decision recommendation
        """
        # Calculate conversion rates
        rate_a = variant_a_conversions / variant_a_visitors if variant_a_visitors > 0 else 0
        rate_b = variant_b_conversions / variant_b_visitors if variant_b_visitors > 0 else 0

        # Calculate improvement
        if rate_a > 0:
            relative_improvement = (rate_b - rate_a) / rate_a
        else:
            relative_improvement = 0

        absolute_improvement = rate_b - rate_a

        # Calculate standard error
        se_a = math.sqrt(rate_a * (1 - rate_a) / variant_a_visitors) if variant_a_visitors > 0 else 0
        se_b = math.sqrt(rate_b * (1 - rate_b) / variant_b_visitors) if variant_b_visitors > 0 else 0
        se_diff = math.sqrt(se_a**2 + se_b**2)

        # Calculate z-score
        z_score = absolute_improvement / se_diff if se_diff > 0 else 0

        # Calculate p-value (two-tailed)
        p_value = 2 * (1 - self._standard_normal_cdf(abs(z_score)))

        # Determine significance
        is_significant_95 = p_value < 0.05
        is_significant_90 = p_value < 0.10

        # Generate decision
        decision = self._generate_test_decision(
            relative_improvement,
            is_significant_95,
            is_significant_90,
            variant_a_visitors + variant_b_visitors
        )

        return {
            'variant_a': {
                'conversions': variant_a_conversions,
                'visitors': variant_a_visitors,
                'conversion_rate': round(rate_a, 4)
            },
            'variant_b': {
                'conversions': variant_b_conversions,
                'visitors': variant_b_visitors,
                'conversion_rate': round(rate_b, 4)
            },
            'improvement': {
                'absolute': round(absolute_improvement, 4),
                'relative_percentage': round(relative_improvement * 100, 2)
            },
            'statistical_analysis': {
                'z_score': round(z_score, 3),
                'p_value': round(p_value, 4),
                'is_significant_95': is_significant_95,
                'is_significant_90': is_significant_90,
                'confidence_level': '95%' if is_significant_95 else ('90%' if is_significant_90 else 'Not significant')
            },
            'decision': decision
        }

    def track_test_results(
        self,
        test_id: str,
        results_data: Dict[str, Any]
    ) -> Dict[str, Any]:
        """
        Track ongoing test results and provide recommendations.

        Args:
            test_id: Test identifier
            results_data: Current test results

        Returns:
            Test tracking report with next steps
        """
        # Find test
        test = next((t for t in self.active_tests if t['test_id'] == test_id), None)
        if not test:
            return {'error': f'Test {test_id} not found'}

        # Calculate significance
        significance = self.calculate_significance(
            results_data['variant_a_conversions'],
            results_data['variant_a_visitors'],
            results_data['variant_b_conversions'],
            results_data['variant_b_visitors']
        )

        # Calculate test progress
        total_visitors = results_data['variant_a_visitors'] + results_data['variant_b_visitors']
        required_sample = results_data.get('required_sample_size', 10000)
        progress_percentage = min((total_visitors / required_sample) * 100, 100)

        # Generate recommendations
        recommendations = self._generate_tracking_recommendations(
            significance,
            progress_percentage,
            test['test_type']
        )

        return {
            'test_id': test_id,
            'test_type': test['test_type'],
            'progress': {
                'total_visitors': total_visitors,
                'required_sample_size': required_sample,
                'progress_percentage': round(progress_percentage, 1),
                'is_complete': progress_percentage >= 100
            },
            'current_results': significance,
            'recommendations': recommendations,
            'next_steps': self._determine_next_steps(
                significance,
                progress_percentage
            )
        }

    def generate_test_report(
        self,
        test_id: str,
        final_results: Dict[str, Any]
    ) -> Dict[str, Any]:
        """
        Generate final test report with insights and recommendations.

        Args:
            test_id: Test identifier
            final_results: Final test results

        Returns:
            Comprehensive test report
        """
        test = next((t for t in self.active_tests if t['test_id'] == test_id), None)
        if not test:
            return {'error': f'Test {test_id} not found'}

        significance = self.calculate_significance(
            final_results['variant_a_conversions'],
            final_results['variant_a_visitors'],
            final_results['variant_b_conversions'],
            final_results['variant_b_visitors']
        )

        # Generate insights
        insights = self._generate_test_insights(
            test,
            significance,
            final_results
        )

        # Implementation plan
        implementation_plan = self._create_implementation_plan(
            test,
            significance
        )

        return {
            'test_summary': {
                'test_id': test_id,
                'test_type': test['test_type'],
                'hypothesis': test['hypothesis'],
                'duration_days': final_results.get('duration_days', 'N/A')
            },
            'results': significance,
            'insights': insights,
            'implementation_plan': implementation_plan,
            'learnings': self._extract_learnings(test, significance)
        }

    def _generate_test_id(self, test_type: str) -> str:
        """Generate unique test ID."""
        import time
        timestamp = int(time.time())
        return f"{test_type}_{timestamp}"

    def _get_secondary_metrics(self, test_type: str) -> List[str]:
        """Get secondary metrics to track for test type."""
        metrics_map = {
            'icon': ['tap_through_rate', 'impression_count', 'brand_recall'],
            'screenshot': ['tap_through_rate', 'time_on_page', 'scroll_depth'],
            'title': ['impression_count', 'tap_through_rate', 'search_visibility'],
            'description': ['time_on_page', 'scroll_depth', 'tap_through_rate']
        }
        return metrics_map.get(test_type, ['tap_through_rate'])

    def _get_test_best_practices(self, test_type: str) -> List[str]:
        """Get best practices for specific test type."""
        practices_map = {
            'icon': [
                'Test only one element at a time (color vs. style vs. symbolism)',
                'Ensure icon is recognizable at small sizes (60x60px)',
                'Consider cultural context for global audience',
                'Test against top competitor icons'
            ],
            'screenshot': [
                'Test order of screenshots (users see first 2-3)',
                'Use captions to tell story',
                'Show key features and benefits',
                'Test with and without device frames'
            ],
            'title': [
                'Test keyword variations, not major rebrand',
                'Keep brand name consistent',
                'Ensure title fits within character limits',
                'Test on both search and browse contexts'
            ],
            'description': [
                'Test structure (bullet points vs. paragraphs)',
                'Test call-to-action placement',
                'Test feature vs. benefit focus',
                'Maintain keyword density'
            ]
        }
        return practices_map.get(test_type, ['Test one variable at a time'])

    def _estimate_test_duration(
        self,
        required_sample_size: int,
        baseline_conversion: float
    ) -> Dict[str, Any]:
        """Estimate test duration based on typical traffic levels."""
        # Assume different daily traffic scenarios
        traffic_scenarios = {
            'low': 100,      # 100 page views/day
            'medium': 1000,  # 1000 page views/day
            'high': 10000    # 10000 page views/day
        }

        estimates = {}
        for scenario, daily_views in traffic_scenarios.items():
            days = math.ceil(required_sample_size / daily_views)
            estimates[scenario] = {
                'daily_page_views': daily_views,
                'estimated_days': days,
                'estimated_weeks': round(days / 7, 1)
            }

        return estimates

    def _generate_sample_size_recommendations(
        self,
        sample_size: int,
        duration_estimates: Dict[str, Any]
    ) -> List[str]:
        """Generate recommendations based on sample size."""
        recommendations = []

        if sample_size > 50000:
            recommendations.append(
                "Large sample size required - consider testing smaller effect size or increasing traffic"
            )

        if duration_estimates['medium']['estimated_days'] > 30:
            recommendations.append(
                "Long test duration - consider higher minimum detectable effect or focus on high-impact changes"
            )

        if duration_estimates['low']['estimated_days'] > 60:
            recommendations.append(
                "Insufficient traffic for reliable testing - consider user acquisition or broader targeting"
            )

        if not recommendations:
            recommendations.append("Sample size and duration are reasonable for this test")

        return recommendations

    def _get_z_score(self, percentile: float) -> float:
        """Get z-score for given percentile (approximation)."""
        # Common z-scores
        z_scores = {
            0.80: 0.84,
            0.85: 1.04,
            0.90: 1.28,
            0.95: 1.645,
            0.975: 1.96,
            0.99: 2.33
        }
        return z_scores.get(percentile, 1.96)

    def _standard_normal_cdf(self, z: float) -> float:
        """Approximate standard normal cumulative distribution function."""
        # Using error function approximation
        t = 1.0 / (1.0 + 0.2316419 * abs(z))
        d = 0.3989423 * math.exp(-z * z / 2.0)
        p = d * t * (0.3193815 + t * (-0.3565638 + t * (1.781478 + t * (-1.821256 + t * 1.330274))))

        if z > 0:
            return 1.0 - p
        else:
            return p

    def _generate_test_decision(
        self,
        improvement: float,
        is_significant_95: bool,
        is_significant_90: bool,
        total_visitors: int
    ) -> Dict[str, Any]:
        """Generate test decision and recommendation."""
        if total_visitors < 1000:
            return {
                'decision': 'continue',
                'rationale': 'Insufficient data - continue test to reach minimum sample size',
                'action': 'Keep test running'
            }

        if is_significant_95:
            if improvement > 0:
                return {
                    'decision': 'implement_b',
                    'rationale': f'Variant B shows {improvement*100:.1f}% improvement with 95% confidence',
                    'action': 'Implement Variant B'
                }
            else:
                return {
                    'decision': 'keep_a',
                    'rationale': 'Variant A performs better with 95% confidence',
                    'action': 'Keep current version (A)'
                }

        elif is_significant_90:
            if improvement > 0:
                return {
                    'decision': 'implement_b_cautiously',
                    'rationale': f'Variant B shows {improvement*100:.1f}% improvement with 90% confidence',
                    'action': 'Consider implementing B, monitor closely'
                }
            else:
                return {
                    'decision': 'keep_a',
                    'rationale': 'Variant A performs better with 90% confidence',
                    'action': 'Keep current version (A)'
                }

        else:
            return {
                'decision': 'inconclusive',
                'rationale': 'No statistically significant difference detected',
                'action': 'Either keep A or test different hypothesis'
            }

    def _generate_tracking_recommendations(
        self,
        significance: Dict[str, Any],
        progress: float,
        test_type: str
    ) -> List[str]:
        """Generate recommendations for ongoing test."""
        recommendations = []

        if progress < 50:
            recommendations.append(
                f"Test is {progress:.0f}% complete - continue collecting data"
            )

        if progress >= 100:
            if significance['statistical_analysis']['is_significant_95']:
                recommendations.append(
                    "Sufficient data collected with significant results - ready to conclude test"
                )
            else:
                recommendations.append(
                    "Sample size reached but no significant difference - consider extending test or concluding"
                )

        return recommendations

    def _determine_next_steps(
        self,
        significance: Dict[str, Any],
        progress: float
    ) -> str:
        """Determine next steps for test."""
        if progress < 100:
            return f"Continue test until reaching 100% sample size (currently {progress:.0f}%)"

        decision = significance.get('decision', {}).get('decision', 'inconclusive')

        if decision == 'implement_b':
            return "Implement Variant B and monitor metrics for 2 weeks"
        elif decision == 'keep_a':
            return "Keep Variant A and design new test with different hypothesis"
        else:
            return "Test inconclusive - either keep A or design new test"

    def _generate_test_insights(
        self,
        test: Dict[str, Any],
        significance: Dict[str, Any],
        results: Dict[str, Any]
    ) -> List[str]:
        """Generate insights from test results."""
        insights = []

        improvement = significance['improvement']['relative_percentage']

        if significance['statistical_analysis']['is_significant_95']:
            insights.append(
                f"Strong evidence: Variant B {'improved' if improvement > 0 else 'decreased'} "
                f"conversion by {abs(improvement):.1f}% with 95% confidence"
            )

        insights.append(
            f"Tested {test['test_type']} changes: {test['hypothesis']}"
        )

        # Add context-specific insights
        if test['test_type'] == 'icon' and improvement > 5:
            insights.append(
                "Icon change had substantial impact - visual first impression is critical"
            )

        return insights

    def _create_implementation_plan(
        self,
        test: Dict[str, Any],
        significance: Dict[str, Any]
    ) -> List[Dict[str, str]]:
        """Create implementation plan for winning variant."""
        plan = []

        if significance.get('decision', {}).get('decision') == 'implement_b':
            plan.append({
                'step': '1. Update store listing',
                'details': f"Replace {test['test_type']} with Variant B across all platforms"
            })
            plan.append({
                'step': '2. Monitor metrics',
                'details': 'Track conversion rate for 2 weeks to confirm sustained improvement'
            })
            plan.append({
                'step': '3. Document learnings',
                'details': 'Record insights for future optimization'
            })

        return plan

    def _extract_learnings(
        self,
        test: Dict[str, Any],
        significance: Dict[str, Any]
    ) -> List[str]:
        """Extract key learnings from test."""
        learnings = []

        improvement = significance['improvement']['relative_percentage']

        learnings.append(
            f"Testing {test['test_type']} can yield {abs(improvement):.1f}% conversion change"
        )

        if test['test_type'] == 'title':
            learnings.append(
                "Title changes affect search visibility and user perception"
            )
        elif test['test_type'] == 'screenshot':
            learnings.append(
                "First 2-3 screenshots are critical for conversion"
            )

        return learnings


def plan_ab_test(
    test_type: str,
    variant_a: Dict[str, Any],
    variant_b: Dict[str, Any],
    hypothesis: str,
    baseline_conversion: float
) -> Dict[str, Any]:
    """
    Convenience function to plan an A/B test.

    Args:
        test_type: Type of test
        variant_a: Control variant
        variant_b: Test variant
        hypothesis: Test hypothesis
        baseline_conversion: Current conversion rate

    Returns:
        Complete test plan
    """
    planner = ABTestPlanner()

    test_design = planner.design_test(
        test_type,
        variant_a,
        variant_b,
        hypothesis
    )

    sample_size = planner.calculate_sample_size(
        baseline_conversion,
        planner.MIN_EFFECT_SIZES.get(test_type, 0.05)
    )

    return {
        'test_design': test_design,
        'sample_size_requirements': sample_size
    }

FILE:scripts/aso_scorer.py
"""
ASO scoring module for App Store Optimization.
Calculates comprehensive ASO health score across multiple dimensions.
"""

from typing import Dict, List, Any, Optional


class ASOScorer:
    """Calculates overall ASO health score and provides recommendations."""

    # Score weights for different components (total = 100)
    WEIGHTS = {
        'metadata_quality': 25,
        'ratings_reviews': 25,
        'keyword_performance': 25,
        'conversion_metrics': 25
    }

    # Benchmarks for scoring
    BENCHMARKS = {
        'title_keyword_usage': {'min': 1, 'target': 2},
        'description_length': {'min': 500, 'target': 2000},
        'keyword_density': {'min': 2, 'optimal': 5, 'max': 8},
        'average_rating': {'min': 3.5, 'target': 4.5},
        'ratings_count': {'min': 100, 'target': 5000},
        'keywords_top_10': {'min': 2, 'target': 10},
        'keywords_top_50': {'min': 5, 'target': 20},
        'conversion_rate': {'min': 0.02, 'target': 0.10}
    }

    def __init__(self):
        """Initialize ASO scorer."""
        self.score_breakdown = {}

    def calculate_overall_score(
        self,
        metadata: Dict[str, Any],
        ratings: Dict[str, Any],
        keyword_performance: Dict[str, Any],
        conversion: Dict[str, Any]
    ) -> Dict[str, Any]:
        """
        Calculate comprehensive ASO score (0-100).

        Args:
            metadata: Title, description quality metrics
            ratings: Rating average and count
            keyword_performance: Keyword ranking data
            conversion: Impression-to-install metrics

        Returns:
            Overall score with detailed breakdown
        """
        # Calculate component scores
        metadata_score = self.score_metadata_quality(metadata)
        ratings_score = self.score_ratings_reviews(ratings)
        keyword_score = self.score_keyword_performance(keyword_performance)
        conversion_score = self.score_conversion_metrics(conversion)

        # Calculate weighted overall score
        overall_score = (
            metadata_score * (self.WEIGHTS['metadata_quality'] / 100) +
            ratings_score * (self.WEIGHTS['ratings_reviews'] / 100) +
            keyword_score * (self.WEIGHTS['keyword_performance'] / 100) +
            conversion_score * (self.WEIGHTS['conversion_metrics'] / 100)
        )

        # Store breakdown
        self.score_breakdown = {
            'metadata_quality': {
                'score': metadata_score,
                'weight': self.WEIGHTS['metadata_quality'],
                'weighted_contribution': round(metadata_score * (self.WEIGHTS['metadata_quality'] / 100), 1)
            },
            'ratings_reviews': {
                'score': ratings_score,
                'weight': self.WEIGHTS['ratings_reviews'],
                'weighted_contribution': round(ratings_score * (self.WEIGHTS['ratings_reviews'] / 100), 1)
            },
            'keyword_performance': {
                'score': keyword_score,
                'weight': self.WEIGHTS['keyword_performance'],
                'weighted_contribution': round(keyword_score * (self.WEIGHTS['keyword_performance'] / 100), 1)
            },
            'conversion_metrics': {
                'score': conversion_score,
                'weight': self.WEIGHTS['conversion_metrics'],
                'weighted_contribution': round(conversion_score * (self.WEIGHTS['conversion_metrics'] / 100), 1)
            }
        }

        # Generate recommendations
        recommendations = self.generate_recommendations(
            metadata_score,
            ratings_score,
            keyword_score,
            conversion_score
        )

        # Assess overall health
        health_status = self._assess_health_status(overall_score)

        return {
            'overall_score': round(overall_score, 1),
            'health_status': health_status,
            'score_breakdown': self.score_breakdown,
            'recommendations': recommendations,
            'priority_actions': self._prioritize_actions(recommendations),
            'strengths': self._identify_strengths(self.score_breakdown),
            'weaknesses': self._identify_weaknesses(self.score_breakdown)
        }

    def score_metadata_quality(self, metadata: Dict[str, Any]) -> float:
        """
        Score metadata quality (0-100).

        Evaluates:
        - Title optimization
        - Description quality
        - Keyword usage
        """
        scores = []

        # Title score (0-35 points)
        title_keywords = metadata.get('title_keyword_count', 0)
        title_length = metadata.get('title_length', 0)

        title_score = 0
        if title_keywords >= self.BENCHMARKS['title_keyword_usage']['target']:
            title_score = 35
        elif title_keywords >= self.BENCHMARKS['title_keyword_usage']['min']:
            title_score = 25
        else:
            title_score = 10

        # Adjust for title length usage
        if title_length > 25:  # Using most of available space
            title_score += 0
        else:
            title_score -= 5

        scores.append(min(title_score, 35))

        # Description score (0-35 points)
        desc_length = metadata.get('description_length', 0)
        desc_quality = metadata.get('description_quality', 0.0)  # 0-1 scale

        desc_score = 0
        if desc_length >= self.BENCHMARKS['description_length']['target']:
            desc_score = 25
        elif desc_length >= self.BENCHMARKS['description_length']['min']:
            desc_score = 15
        else:
            desc_score = 5

        # Add quality bonus
        desc_score += desc_quality * 10
        scores.append(min(desc_score, 35))

        # Keyword density score (0-30 points)
        keyword_density = metadata.get('keyword_density', 0.0)

        if self.BENCHMARKS['keyword_density']['min'] <= keyword_density <= self.BENCHMARKS['keyword_density']['optimal']:
            density_score = 30
        elif keyword_density < self.BENCHMARKS['keyword_density']['min']:
            # Too low - proportional scoring
            density_score = (keyword_density / self.BENCHMARKS['keyword_density']['min']) * 20
        else:
            # Too high (keyword stuffing) - penalty
            excess = keyword_density - self.BENCHMARKS['keyword_density']['optimal']
            density_score = max(30 - (excess * 5), 0)

        scores.append(density_score)

        return round(sum(scores), 1)

    def score_ratings_reviews(self, ratings: Dict[str, Any]) -> float:
        """
        Score ratings and reviews (0-100).

        Evaluates:
        - Average rating
        - Total ratings count
        - Review velocity
        """
        average_rating = ratings.get('average_rating', 0.0)
        total_ratings = ratings.get('total_ratings', 0)
        recent_ratings = ratings.get('recent_ratings_30d', 0)

        # Rating quality score (0-50 points)
        if average_rating >= self.BENCHMARKS['average_rating']['target']:
            rating_quality_score = 50
        elif average_rating >= self.BENCHMARKS['average_rating']['min']:
            # Proportional scoring between min and target
            proportion = (average_rating - self.BENCHMARKS['average_rating']['min']) / \
                        (self.BENCHMARKS['average_rating']['target'] - self.BENCHMARKS['average_rating']['min'])
            rating_quality_score = 30 + (proportion * 20)
        elif average_rating >= 3.0:
            rating_quality_score = 20
        else:
            rating_quality_score = 10

        # Rating volume score (0-30 points)
        if total_ratings >= self.BENCHMARKS['ratings_count']['target']:
            rating_volume_score = 30
        elif total_ratings >= self.BENCHMARKS['ratings_count']['min']:
            # Proportional scoring
            proportion = (total_ratings - self.BENCHMARKS['ratings_count']['min']) / \
                        (self.BENCHMARKS['ratings_count']['target'] - self.BENCHMARKS['ratings_count']['min'])
            rating_volume_score = 15 + (proportion * 15)
        else:
            # Very low volume
            rating_volume_score = (total_ratings / self.BENCHMARKS['ratings_count']['min']) * 15

        # Rating velocity score (0-20 points)
        if recent_ratings > 100:
            velocity_score = 20
        elif recent_ratings > 50:
            velocity_score = 15
        elif recent_ratings > 10:
            velocity_score = 10
        else:
            velocity_score = 5

        total_score = rating_quality_score + rating_volume_score + velocity_score

        return round(min(total_score, 100), 1)

    def score_keyword_performance(self, keyword_performance: Dict[str, Any]) -> float:
        """
        Score keyword ranking performance (0-100).

        Evaluates:
        - Top 10 rankings
        - Top 50 rankings
        - Ranking trends
        """
        top_10_count = keyword_performance.get('top_10', 0)
        top_50_count = keyword_performance.get('top_50', 0)
        top_100_count = keyword_performance.get('top_100', 0)
        improving_keywords = keyword_performance.get('improving_keywords', 0)

        # Top 10 score (0-50 points) - most valuable rankings
        if top_10_count >= self.BENCHMARKS['keywords_top_10']['target']:
            top_10_score = 50
        elif top_10_count >= self.BENCHMARKS['keywords_top_10']['min']:
            proportion = (top_10_count - self.BENCHMARKS['keywords_top_10']['min']) / \
                        (self.BENCHMARKS['keywords_top_10']['target'] - self.BENCHMARKS['keywords_top_10']['min'])
            top_10_score = 25 + (proportion * 25)
        else:
            top_10_score = (top_10_count / self.BENCHMARKS['keywords_top_10']['min']) * 25

        # Top 50 score (0-30 points)
        if top_50_count >= self.BENCHMARKS['keywords_top_50']['target']:
            top_50_score = 30
        elif top_50_count >= self.BENCHMARKS['keywords_top_50']['min']:
            proportion = (top_50_count - self.BENCHMARKS['keywords_top_50']['min']) / \
                        (self.BENCHMARKS['keywords_top_50']['target'] - self.BENCHMARKS['keywords_top_50']['min'])
            top_50_score = 15 + (proportion * 15)
        else:
            top_50_score = (top_50_count / self.BENCHMARKS['keywords_top_50']['min']) * 15

        # Coverage score (0-10 points) - based on top 100
        coverage_score = min((top_100_count / 30) * 10, 10)

        # Trend score (0-10 points) - are rankings improving?
        if improving_keywords > 5:
            trend_score = 10
        elif improving_keywords > 0:
            trend_score = 5
        else:
            trend_score = 0

        total_score = top_10_score + top_50_score + coverage_score + trend_score

        return round(min(total_score, 100), 1)

    def score_conversion_metrics(self, conversion: Dict[str, Any]) -> float:
        """
        Score conversion performance (0-100).

        Evaluates:
        - Impression-to-install conversion rate
        - Download velocity
        """
        conversion_rate = conversion.get('impression_to_install', 0.0)
        downloads_30d = conversion.get('downloads_last_30_days', 0)
        downloads_trend = conversion.get('downloads_trend', 'stable')  # 'up', 'stable', 'down'

        # Conversion rate score (0-70 points)
        if conversion_rate >= self.BENCHMARKS['conversion_rate']['target']:
            conversion_score = 70
        elif conversion_rate >= self.BENCHMARKS['conversion_rate']['min']:
            proportion = (conversion_rate - self.BENCHMARKS['conversion_rate']['min']) / \
                        (self.BENCHMARKS['conversion_rate']['target'] - self.BENCHMARKS['conversion_rate']['min'])
            conversion_score = 35 + (proportion * 35)
        else:
            conversion_score = (conversion_rate / self.BENCHMARKS['conversion_rate']['min']) * 35

        # Download velocity score (0-20 points)
        if downloads_30d > 10000:
            velocity_score = 20
        elif downloads_30d > 1000:
            velocity_score = 15
        elif downloads_30d > 100:
            velocity_score = 10
        else:
            velocity_score = 5

        # Trend bonus (0-10 points)
        if downloads_trend == 'up':
            trend_score = 10
        elif downloads_trend == 'stable':
            trend_score = 5
        else:
            trend_score = 0

        total_score = conversion_score + velocity_score + trend_score

        return round(min(total_score, 100), 1)

    def generate_recommendations(
        self,
        metadata_score: float,
        ratings_score: float,
        keyword_score: float,
        conversion_score: float
    ) -> List[Dict[str, Any]]:
        """Generate prioritized recommendations based on scores."""
        recommendations = []

        # Metadata recommendations
        if metadata_score < 60:
            recommendations.append({
                'category': 'metadata_quality',
                'priority': 'high',
                'action': 'Optimize app title and description',
                'details': 'Add more keywords to title, expand description to 1500-2000 characters, improve keyword density to 3-5%',
                'expected_impact': 'Improve discoverability and ranking potential'
            })
        elif metadata_score < 80:
            recommendations.append({
                'category': 'metadata_quality',
                'priority': 'medium',
                'action': 'Refine metadata for better keyword targeting',
                'details': 'Test variations of title/subtitle, optimize keyword field for Apple',
                'expected_impact': 'Incremental ranking improvements'
            })

        # Ratings recommendations
        if ratings_score < 60:
            recommendations.append({
                'category': 'ratings_reviews',
                'priority': 'high',
                'action': 'Improve rating quality and volume',
                'details': 'Address top user complaints, implement in-app rating prompts, respond to negative reviews',
                'expected_impact': 'Better conversion rates and trust signals'
            })
        elif ratings_score < 80:
            recommendations.append({
                'category': 'ratings_reviews',
                'priority': 'medium',
                'action': 'Increase rating velocity',
                'details': 'Optimize timing of rating requests, encourage satisfied users to rate',
                'expected_impact': 'Sustained rating quality'
            })

        # Keyword performance recommendations
        if keyword_score < 60:
            recommendations.append({
                'category': 'keyword_performance',
                'priority': 'high',
                'action': 'Improve keyword rankings',
                'details': 'Target long-tail keywords with lower competition, update metadata with high-potential keywords, build backlinks',
                'expected_impact': 'Significant improvement in organic visibility'
            })
        elif keyword_score < 80:
            recommendations.append({
                'category': 'keyword_performance',
                'priority': 'medium',
                'action': 'Expand keyword coverage',
                'details': 'Target additional related keywords, test seasonal keywords, localize for new markets',
                'expected_impact': 'Broader reach and more discovery opportunities'
            })

        # Conversion recommendations
        if conversion_score < 60:
            recommendations.append({
                'category': 'conversion_metrics',
                'priority': 'high',
                'action': 'Optimize store listing for conversions',
                'details': 'Improve screenshots and icon, strengthen value proposition in description, add video preview',
                'expected_impact': 'Higher impression-to-install conversion'
            })
        elif conversion_score < 80:
            recommendations.append({
                'category': 'conversion_metrics',
                'priority': 'medium',
                'action': 'Test visual asset variations',
                'details': 'A/B test different icon designs and screenshot sequences',
                'expected_impact': 'Incremental conversion improvements'
            })

        return recommendations

    def _assess_health_status(self, overall_score: float) -> str:
        """Assess overall ASO health status."""
        if overall_score >= 80:
            return "Excellent - Top-tier ASO performance"
        elif overall_score >= 65:
            return "Good - Competitive ASO with room for improvement"
        elif overall_score >= 50:
            return "Fair - Needs strategic improvements"
        else:
            return "Poor - Requires immediate ASO overhaul"

    def _prioritize_actions(
        self,
        recommendations: List[Dict[str, Any]]
    ) -> List[Dict[str, Any]]:
        """Prioritize actions by impact and urgency."""
        # Sort by priority (high first) and expected impact
        priority_order = {'high': 0, 'medium': 1, 'low': 2}

        sorted_recommendations = sorted(
            recommendations,
            key=lambda x: priority_order[x['priority']]
        )

        return sorted_recommendations[:3]  # Top 3 priority actions

    def _identify_strengths(self, score_breakdown: Dict[str, Any]) -> List[str]:
        """Identify areas of strength (scores >= 75)."""
        strengths = []

        for category, data in score_breakdown.items():
            if data['score'] >= 75:
                strengths.append(
                    f"{category.replace('_', ' ').title()}: {data['score']}/100"
                )

        return strengths if strengths else ["Focus on building strengths across all areas"]

    def _identify_weaknesses(self, score_breakdown: Dict[str, Any]) -> List[str]:
        """Identify areas needing improvement (scores < 60)."""
        weaknesses = []

        for category, data in score_breakdown.items():
            if data['score'] < 60:
                weaknesses.append(
                    f"{category.replace('_', ' ').title()}: {data['score']}/100 - needs improvement"
                )

        return weaknesses if weaknesses else ["All areas performing adequately"]


def calculate_aso_score(
    metadata: Dict[str, Any],
    ratings: Dict[str, Any],
    keyword_performance: Dict[str, Any],
    conversion: Dict[str, Any]
) -> Dict[str, Any]:
    """
    Convenience function to calculate ASO score.

    Args:
        metadata: Metadata quality metrics
        ratings: Ratings data
        keyword_performance: Keyword ranking data
        conversion: Conversion metrics

    Returns:
        Complete ASO score report
    """
    scorer = ASOScorer()
    return scorer.calculate_overall_score(
        metadata,
        ratings,
        keyword_performance,
        conversion
    )

FILE:scripts/competitor_analyzer.py
"""
Competitor analysis module for App Store Optimization.
Analyzes top competitors' ASO strategies and identifies opportunities.
"""

from typing import Dict, List, Any, Optional
from collections import Counter
import re


class CompetitorAnalyzer:
    """Analyzes competitor apps to identify ASO opportunities."""

    def __init__(self, category: str, platform: str = 'apple'):
        """
        Initialize competitor analyzer.

        Args:
            category: App category (e.g., "Productivity", "Games")
            platform: 'apple' or 'google'
        """
        self.category = category
        self.platform = platform
        self.competitors = []

    def analyze_competitor(
        self,
        app_data: Dict[str, Any]
    ) -> Dict[str, Any]:
        """
        Analyze a single competitor's ASO strategy.

        Args:
            app_data: Dictionary with app_name, title, description, rating, ratings_count, keywords

        Returns:
            Comprehensive competitor analysis
        """
        app_name = app_data.get('app_name', '')
        title = app_data.get('title', '')
        description = app_data.get('description', '')
        rating = app_data.get('rating', 0.0)
        ratings_count = app_data.get('ratings_count', 0)
        keywords = app_data.get('keywords', [])

        analysis = {
            'app_name': app_name,
            'title_analysis': self._analyze_title(title),
            'description_analysis': self._analyze_description(description),
            'keyword_strategy': self._extract_keyword_strategy(title, description, keywords),
            'rating_metrics': {
                'rating': rating,
                'ratings_count': ratings_count,
                'rating_quality': self._assess_rating_quality(rating, ratings_count)
            },
            'competitive_strength': self._calculate_competitive_strength(
                rating,
                ratings_count,
                len(description)
            ),
            'key_differentiators': self._identify_differentiators(description)
        }

        self.competitors.append(analysis)
        return analysis

    def compare_competitors(
        self,
        competitors_data: List[Dict[str, Any]]
    ) -> Dict[str, Any]:
        """
        Compare multiple competitors and identify patterns.

        Args:
            competitors_data: List of competitor data dictionaries

        Returns:
            Comparative analysis with insights
        """
        # Analyze each competitor
        analyses = []
        for comp_data in competitors_data:
            analysis = self.analyze_competitor(comp_data)
            analyses.append(analysis)

        # Extract common keywords across competitors
        all_keywords = []
        for analysis in analyses:
            all_keywords.extend(analysis['keyword_strategy']['primary_keywords'])

        common_keywords = self._find_common_keywords(all_keywords)

        # Identify keyword gaps (used by some but not all)
        keyword_gaps = self._identify_keyword_gaps(analyses)

        # Rank competitors by strength
        ranked_competitors = sorted(
            analyses,
            key=lambda x: x['competitive_strength'],
            reverse=True
        )

        # Analyze rating distribution
        rating_analysis = self._analyze_rating_distribution(analyses)

        # Identify best practices
        best_practices = self._identify_best_practices(ranked_competitors)

        return {
            'category': self.category,
            'platform': self.platform,
            'competitors_analyzed': len(analyses),
            'ranked_competitors': ranked_competitors,
            'common_keywords': common_keywords,
            'keyword_gaps': keyword_gaps,
            'rating_analysis': rating_analysis,
            'best_practices': best_practices,
            'opportunities': self._identify_opportunities(
                analyses,
                common_keywords,
                keyword_gaps
            )
        }

    def identify_gaps(
        self,
        your_app_data: Dict[str, Any],
        competitors_data: List[Dict[str, Any]]
    ) -> Dict[str, Any]:
        """
        Identify gaps between your app and competitors.

        Args:
            your_app_data: Your app's data
            competitors_data: List of competitor data

        Returns:
            Gap analysis with actionable recommendations
        """
        # Analyze your app
        your_analysis = self.analyze_competitor(your_app_data)

        # Analyze competitors
        competitor_comparison = self.compare_competitors(competitors_data)

        # Identify keyword gaps
        your_keywords = set(your_analysis['keyword_strategy']['primary_keywords'])
        competitor_keywords = set(competitor_comparison['common_keywords'])
        missing_keywords = competitor_keywords - your_keywords

        # Identify rating gap
        avg_competitor_rating = competitor_comparison['rating_analysis']['average_rating']
        rating_gap = avg_competitor_rating - your_analysis['rating_metrics']['rating']

        # Identify description length gap
        avg_competitor_desc_length = sum(
            len(comp['description_analysis']['text'])
            for comp in competitor_comparison['ranked_competitors']
        ) / len(competitor_comparison['ranked_competitors'])
        your_desc_length = len(your_analysis['description_analysis']['text'])
        desc_length_gap = avg_competitor_desc_length - your_desc_length

        return {
            'your_app': your_analysis,
            'keyword_gaps': {
                'missing_keywords': list(missing_keywords)[:10],
                'recommendations': self._generate_keyword_recommendations(missing_keywords)
            },
            'rating_gap': {
                'your_rating': your_analysis['rating_metrics']['rating'],
                'average_competitor_rating': avg_competitor_rating,
                'gap': round(rating_gap, 2),
                'action_items': self._generate_rating_improvement_actions(rating_gap)
            },
            'content_gap': {
                'your_description_length': your_desc_length,
                'average_competitor_length': int(avg_competitor_desc_length),
                'gap': int(desc_length_gap),
                'recommendations': self._generate_content_recommendations(desc_length_gap)
            },
            'competitive_positioning': self._assess_competitive_position(
                your_analysis,
                competitor_comparison
            )
        }

    def _analyze_title(self, title: str) -> Dict[str, Any]:
        """Analyze title structure and keyword usage."""
        parts = re.split(r'[-:|]', title)

        return {
            'title': title,
            'length': len(title),
            'has_brand': len(parts) > 0,
            'has_keywords': len(parts) > 1,
            'components': [part.strip() for part in parts],
            'word_count': len(title.split()),
            'strategy': 'brand_plus_keywords' if len(parts) > 1 else 'brand_only'
        }

    def _analyze_description(self, description: str) -> Dict[str, Any]:
        """Analyze description structure and content."""
        lines = description.split('\n')
        word_count = len(description.split())

        # Check for structural elements
        has_bullet_points = '•' in description or '*' in description
        has_sections = any(line.isupper() for line in lines if len(line) > 0)
        has_call_to_action = any(
            cta in description.lower()
            for cta in ['download', 'try', 'get', 'start', 'join']
        )

        # Extract features mentioned
        features = self._extract_features(description)

        return {
            'text': description,
            'length': len(description),
            'word_count': word_count,
            'structure': {
                'has_bullet_points': has_bullet_points,
                'has_sections': has_sections,
                'has_call_to_action': has_call_to_action
            },
            'features_mentioned': features,
            'readability': 'good' if 50 <= word_count <= 300 else 'needs_improvement'
        }

    def _extract_keyword_strategy(
        self,
        title: str,
        description: str,
        explicit_keywords: List[str]
    ) -> Dict[str, Any]:
        """Extract keyword strategy from metadata."""
        # Extract keywords from title
        title_keywords = [word.lower() for word in title.split() if len(word) > 3]

        # Extract frequently used words from description
        desc_words = re.findall(r'\b\w{4,}\b', description.lower())
        word_freq = Counter(desc_words)
        frequent_words = [word for word, count in word_freq.most_common(15) if count > 2]

        # Combine with explicit keywords
        all_keywords = list(set(title_keywords + frequent_words + explicit_keywords))

        return {
            'primary_keywords': title_keywords,
            'description_keywords': frequent_words[:10],
            'explicit_keywords': explicit_keywords,
            'total_unique_keywords': len(all_keywords),
            'keyword_focus': self._assess_keyword_focus(title_keywords, frequent_words)
        }

    def _assess_rating_quality(self, rating: float, ratings_count: int) -> str:
        """Assess the quality of ratings."""
        if ratings_count < 100:
            return 'insufficient_data'
        elif rating >= 4.5 and ratings_count > 1000:
            return 'excellent'
        elif rating >= 4.0 and ratings_count > 500:
            return 'good'
        elif rating >= 3.5:
            return 'average'
        else:
            return 'poor'

    def _calculate_competitive_strength(
        self,
        rating: float,
        ratings_count: int,
        description_length: int
    ) -> float:
        """
        Calculate overall competitive strength (0-100).

        Factors:
        - Rating quality (40%)
        - Rating volume (30%)
        - Metadata quality (30%)
        """
        # Rating quality score (0-40)
        rating_score = (rating / 5.0) * 40

        # Rating volume score (0-30)
        volume_score = min((ratings_count / 10000) * 30, 30)

        # Metadata quality score (0-30)
        metadata_score = min((description_length / 2000) * 30, 30)

        total_score = rating_score + volume_score + metadata_score

        return round(total_score, 1)

    def _identify_differentiators(self, description: str) -> List[str]:
        """Identify key differentiators from description."""
        differentiator_keywords = [
            'unique', 'only', 'first', 'best', 'leading', 'exclusive',
            'revolutionary', 'innovative', 'patent', 'award'
        ]

        differentiators = []
        sentences = description.split('.')

        for sentence in sentences:
            sentence_lower = sentence.lower()
            if any(keyword in sentence_lower for keyword in differentiator_keywords):
                differentiators.append(sentence.strip())

        return differentiators[:5]

    def _find_common_keywords(self, all_keywords: List[str]) -> List[str]:
        """Find keywords used by multiple competitors."""
        keyword_counts = Counter(all_keywords)
        # Return keywords used by at least 2 competitors
        common = [kw for kw, count in keyword_counts.items() if count >= 2]
        return sorted(common, key=lambda x: keyword_counts[x], reverse=True)[:20]

    def _identify_keyword_gaps(self, analyses: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
        """Identify keywords used by some competitors but not others."""
        all_keywords_by_app = {}

        for analysis in analyses:
            app_name = analysis['app_name']
            keywords = analysis['keyword_strategy']['primary_keywords']
            all_keywords_by_app[app_name] = set(keywords)

        # Find keywords used by some but not all
        all_keywords_set = set()
        for keywords in all_keywords_by_app.values():
            all_keywords_set.update(keywords)

        gaps = []
        for keyword in all_keywords_set:
            using_apps = [
                app for app, keywords in all_keywords_by_app.items()
                if keyword in keywords
            ]
            if 1 < len(using_apps) < len(analyses):
                gaps.append({
                    'keyword': keyword,
                    'used_by': using_apps,
                    'usage_percentage': round(len(using_apps) / len(analyses) * 100, 1)
                })

        return sorted(gaps, key=lambda x: x['usage_percentage'], reverse=True)[:15]

    def _analyze_rating_distribution(self, analyses: List[Dict[str, Any]]) -> Dict[str, Any]:
        """Analyze rating distribution across competitors."""
        ratings = [a['rating_metrics']['rating'] for a in analyses]
        ratings_counts = [a['rating_metrics']['ratings_count'] for a in analyses]

        return {
            'average_rating': round(sum(ratings) / len(ratings), 2),
            'highest_rating': max(ratings),
            'lowest_rating': min(ratings),
            'average_ratings_count': int(sum(ratings_counts) / len(ratings_counts)),
            'total_ratings_in_category': sum(ratings_counts)
        }

    def _identify_best_practices(self, ranked_competitors: List[Dict[str, Any]]) -> List[str]:
        """Identify best practices from top competitors."""
        if not ranked_competitors:
            return []

        top_competitor = ranked_competitors[0]
        practices = []

        # Title strategy
        title_analysis = top_competitor['title_analysis']
        if title_analysis['has_keywords']:
            practices.append(
                f"Title Strategy: Include primary keyword in title (e.g., '{title_analysis['title']}')"
            )

        # Description structure
        desc_analysis = top_competitor['description_analysis']
        if desc_analysis['structure']['has_bullet_points']:
            practices.append("Description: Use bullet points to highlight key features")

        if desc_analysis['structure']['has_sections']:
            practices.append("Description: Organize content with clear section headers")

        # Rating strategy
        rating_quality = top_competitor['rating_metrics']['rating_quality']
        if rating_quality in ['excellent', 'good']:
            practices.append(
                f"Ratings: Maintain high rating quality ({top_competitor['rating_metrics']['rating']}★) "
                f"with significant volume ({top_competitor['rating_metrics']['ratings_count']} ratings)"
            )

        return practices[:5]

    def _identify_opportunities(
        self,
        analyses: List[Dict[str, Any]],
        common_keywords: List[str],
        keyword_gaps: List[Dict[str, Any]]
    ) -> List[str]:
        """Identify ASO opportunities based on competitive analysis."""
        opportunities = []

        # Keyword opportunities from gaps
        if keyword_gaps:
            underutilized_keywords = [
                gap['keyword'] for gap in keyword_gaps
                if gap['usage_percentage'] < 50
            ]
            if underutilized_keywords:
                opportunities.append(
                    f"Target underutilized keywords: {', '.join(underutilized_keywords[:5])}"
                )

        # Rating opportunity
        avg_rating = sum(a['rating_metrics']['rating'] for a in analyses) / len(analyses)
        if avg_rating < 4.5:
            opportunities.append(
                f"Category average rating is {avg_rating:.1f} - opportunity to differentiate with higher ratings"
            )

        # Content depth opportunity
        avg_desc_length = sum(
            a['description_analysis']['length'] for a in analyses
        ) / len(analyses)
        if avg_desc_length < 1500:
            opportunities.append(
                "Competitors have relatively short descriptions - opportunity to provide more comprehensive information"
            )

        return opportunities[:5]

    def _extract_features(self, description: str) -> List[str]:
        """Extract feature mentions from description."""
        # Look for bullet points or numbered lists
        lines = description.split('\n')
        features = []

        for line in lines:
            line = line.strip()
            # Check if line starts with bullet or number
            if line and (line[0] in ['•', '*', '-', '✓'] or line[0].isdigit()):
                # Clean the line
                cleaned = re.sub(r'^[•*\-✓\d.)\s]+', '', line)
                if cleaned:
                    features.append(cleaned)

        return features[:10]

    def _assess_keyword_focus(
        self,
        title_keywords: List[str],
        description_keywords: List[str]
    ) -> str:
        """Assess keyword focus strategy."""
        overlap = set(title_keywords) & set(description_keywords)

        if len(overlap) >= 3:
            return 'consistent_focus'
        elif len(overlap) >= 1:
            return 'moderate_focus'
        else:
            return 'broad_focus'

    def _generate_keyword_recommendations(self, missing_keywords: set) -> List[str]:
        """Generate recommendations for missing keywords."""
        if not missing_keywords:
            return ["Your keyword coverage is comprehensive"]

        recommendations = []
        missing_list = list(missing_keywords)[:5]

        recommendations.append(
            f"Consider adding these competitor keywords: {', '.join(missing_list)}"
        )
        recommendations.append(
            "Test keyword variations in subtitle/promotional text first"
        )
        recommendations.append(
            "Monitor competitor keyword changes monthly"
        )

        return recommendations

    def _generate_rating_improvement_actions(self, rating_gap: float) -> List[str]:
        """Generate actions to improve ratings."""
        actions = []

        if rating_gap > 0.5:
            actions.append("CRITICAL: Significant rating gap - prioritize user satisfaction improvements")
            actions.append("Analyze negative reviews to identify top issues")
            actions.append("Implement in-app rating prompts after positive experiences")
            actions.append("Respond to all negative reviews professionally")
        elif rating_gap > 0.2:
            actions.append("Focus on incremental improvements to close rating gap")
            actions.append("Optimize timing of rating requests")
        else:
            actions.append("Ratings are competitive - maintain quality and continue improvements")

        return actions

    def _generate_content_recommendations(self, desc_length_gap: int) -> List[str]:
        """Generate content recommendations based on length gap."""
        recommendations = []

        if desc_length_gap > 500:
            recommendations.append(
                "Expand description to match competitor detail level"
            )
            recommendations.append(
                "Add use case examples and success stories"
            )
            recommendations.append(
                "Include more feature explanations and benefits"
            )
        elif desc_length_gap < -500:
            recommendations.append(
                "Consider condensing description for better readability"
            )
            recommendations.append(
                "Focus on most important features first"
            )
        else:
            recommendations.append(
                "Description length is competitive"
            )

        return recommendations

    def _assess_competitive_position(
        self,
        your_analysis: Dict[str, Any],
        competitor_comparison: Dict[str, Any]
    ) -> str:
        """Assess your competitive position."""
        your_strength = your_analysis['competitive_strength']
        competitors = competitor_comparison['ranked_competitors']

        if not competitors:
            return "No comparison data available"

        # Find where you'd rank
        better_than_count = sum(
            1 for comp in competitors
            if your_strength > comp['competitive_strength']
        )

        position_percentage = (better_than_count / len(competitors)) * 100

        if position_percentage >= 75:
            return "Strong Position: Top quartile in competitive strength"
        elif position_percentage >= 50:
            return "Competitive Position: Above average, opportunities for improvement"
        elif position_percentage >= 25:
            return "Challenging Position: Below average, requires strategic improvements"
        else:
            return "Weak Position: Bottom quartile, major ASO overhaul needed"


def analyze_competitor_set(
    category: str,
    competitors_data: List[Dict[str, Any]],
    platform: str = 'apple'
) -> Dict[str, Any]:
    """
    Convenience function to analyze a set of competitors.

    Args:
        category: App category
        competitors_data: List of competitor data
        platform: 'apple' or 'google'

    Returns:
        Complete competitive analysis
    """
    analyzer = CompetitorAnalyzer(category, platform)
    return analyzer.compare_competitors(competitors_data)

FILE:scripts/keyword_analyzer.py
"""
Keyword analysis module for App Store Optimization.
Analyzes keyword search volume, competition, and relevance for app discovery.
"""

from typing import Dict, List, Any, Optional, Tuple
import re
from collections import Counter


class KeywordAnalyzer:
    """Analyzes keywords for ASO effectiveness."""

    # Competition level thresholds (based on number of competing apps)
    COMPETITION_THRESHOLDS = {
        'low': 1000,
        'medium': 5000,
        'high': 10000
    }

    # Search volume categories (monthly searches estimate)
    VOLUME_CATEGORIES = {
        'very_low': 1000,
        'low': 5000,
        'medium': 20000,
        'high': 100000,
        'very_high': 500000
    }

    def __init__(self):
        """Initialize keyword analyzer."""
        self.analyzed_keywords = {}

    def analyze_keyword(
        self,
        keyword: str,
        search_volume: int = 0,
        competing_apps: int = 0,
        relevance_score: float = 0.0
    ) -> Dict[str, Any]:
        """
        Analyze a single keyword for ASO potential.

        Args:
            keyword: The keyword to analyze
            search_volume: Estimated monthly search volume
            competing_apps: Number of apps competing for this keyword
            relevance_score: Relevance to your app (0.0-1.0)

        Returns:
            Dictionary with keyword analysis
        """
        competition_level = self._calculate_competition_level(competing_apps)
        volume_category = self._categorize_search_volume(search_volume)
        difficulty_score = self._calculate_keyword_difficulty(
            search_volume,
            competing_apps
        )

        # Calculate potential score (0-100)
        potential_score = self._calculate_potential_score(
            search_volume,
            competing_apps,
            relevance_score
        )

        analysis = {
            'keyword': keyword,
            'search_volume': search_volume,
            'volume_category': volume_category,
            'competing_apps': competing_apps,
            'competition_level': competition_level,
            'relevance_score': relevance_score,
            'difficulty_score': difficulty_score,
            'potential_score': potential_score,
            'recommendation': self._generate_recommendation(
                potential_score,
                difficulty_score,
                relevance_score
            ),
            'keyword_length': len(keyword.split()),
            'is_long_tail': len(keyword.split()) >= 3
        }

        self.analyzed_keywords[keyword] = analysis
        return analysis

    def compare_keywords(self, keywords_data: List[Dict[str, Any]]) -> Dict[str, Any]:
        """
        Compare multiple keywords and rank by potential.

        Args:
            keywords_data: List of dicts with keyword, search_volume, competing_apps, relevance_score

        Returns:
            Comparison report with ranked keywords
        """
        analyses = []
        for kw_data in keywords_data:
            analysis = self.analyze_keyword(
                keyword=kw_data['keyword'],
                search_volume=kw_data.get('search_volume', 0),
                competing_apps=kw_data.get('competing_apps', 0),
                relevance_score=kw_data.get('relevance_score', 0.0)
            )
            analyses.append(analysis)

        # Sort by potential score (descending)
        ranked_keywords = sorted(
            analyses,
            key=lambda x: x['potential_score'],
            reverse=True
        )

        # Categorize keywords
        primary_keywords = [
            kw for kw in ranked_keywords
            if kw['potential_score'] >= 70 and kw['relevance_score'] >= 0.8
        ]

        secondary_keywords = [
            kw for kw in ranked_keywords
            if 50 <= kw['potential_score'] < 70 and kw['relevance_score'] >= 0.6
        ]

        long_tail_keywords = [
            kw for kw in ranked_keywords
            if kw['is_long_tail'] and kw['relevance_score'] >= 0.7
        ]

        return {
            'total_keywords_analyzed': len(analyses),
            'ranked_keywords': ranked_keywords,
            'primary_keywords': primary_keywords[:5],  # Top 5
            'secondary_keywords': secondary_keywords[:10],  # Top 10
            'long_tail_keywords': long_tail_keywords[:10],  # Top 10
            'summary': self._generate_comparison_summary(
                primary_keywords,
                secondary_keywords,
                long_tail_keywords
            )
        }

    def find_long_tail_opportunities(
        self,
        base_keyword: str,
        modifiers: List[str]
    ) -> List[Dict[str, Any]]:
        """
        Generate long-tail keyword variations.

        Args:
            base_keyword: Core keyword (e.g., "task manager")
            modifiers: List of modifiers (e.g., ["free", "simple", "team"])

        Returns:
            List of long-tail keyword suggestions
        """
        long_tail_keywords = []

        # Generate combinations
        for modifier in modifiers:
            # Modifier + base
            variation1 = f"{modifier} {base_keyword}"
            long_tail_keywords.append({
                'keyword': variation1,
                'pattern': 'modifier_base',
                'estimated_competition': 'low',
                'rationale': f"Less competitive variation of '{base_keyword}'"
            })

            # Base + modifier
            variation2 = f"{base_keyword} {modifier}"
            long_tail_keywords.append({
                'keyword': variation2,
                'pattern': 'base_modifier',
                'estimated_competition': 'low',
                'rationale': f"Specific use-case variation of '{base_keyword}'"
            })

        # Add question-based long-tail
        question_words = ['how', 'what', 'best', 'top']
        for q_word in question_words:
            question_keyword = f"{q_word} {base_keyword}"
            long_tail_keywords.append({
                'keyword': question_keyword,
                'pattern': 'question_based',
                'estimated_competition': 'very_low',
                'rationale': f"Informational search query"
            })

        return long_tail_keywords

    def extract_keywords_from_text(
        self,
        text: str,
        min_word_length: int = 3
    ) -> List[Tuple[str, int]]:
        """
        Extract potential keywords from text (descriptions, reviews).

        Args:
            text: Text to analyze
            min_word_length: Minimum word length to consider

        Returns:
            List of (keyword, frequency) tuples
        """
        # Clean and normalize text
        text = text.lower()
        text = re.sub(r'[^\w\s]', ' ', text)

        # Extract words
        words = text.split()

        # Filter by length
        words = [w for w in words if len(w) >= min_word_length]

        # Remove common stop words
        stop_words = {
            'the', 'and', 'for', 'with', 'this', 'that', 'from', 'have',
            'but', 'not', 'you', 'all', 'can', 'are', 'was', 'were', 'been'
        }
        words = [w for w in words if w not in stop_words]

        # Count frequency
        word_counts = Counter(words)

        # Extract 2-word phrases
        phrases = []
        for i in range(len(words) - 1):
            phrase = f"{words[i]} {words[i+1]}"
            phrases.append(phrase)

        phrase_counts = Counter(phrases)

        # Combine and sort
        all_keywords = list(word_counts.items()) + list(phrase_counts.items())
        all_keywords.sort(key=lambda x: x[1], reverse=True)

        return all_keywords[:50]  # Top 50

    def calculate_keyword_density(
        self,
        text: str,
        target_keywords: List[str]
    ) -> Dict[str, float]:
        """
        Calculate keyword density in text.

        Args:
            text: Text to analyze (title, description)
            target_keywords: Keywords to check density for

        Returns:
            Dictionary of keyword: density (percentage)
        """
        text_lower = text.lower()
        total_words = len(text_lower.split())

        densities = {}
        for keyword in target_keywords:
            keyword_lower = keyword.lower()
            occurrences = text_lower.count(keyword_lower)
            density = (occurrences / total_words) * 100 if total_words > 0 else 0
            densities[keyword] = round(density, 2)

        return densities

    def _calculate_competition_level(self, competing_apps: int) -> str:
        """Determine competition level based on number of competing apps."""
        if competing_apps < self.COMPETITION_THRESHOLDS['low']:
            return 'low'
        elif competing_apps < self.COMPETITION_THRESHOLDS['medium']:
            return 'medium'
        elif competing_apps < self.COMPETITION_THRESHOLDS['high']:
            return 'high'
        else:
            return 'very_high'

    def _categorize_search_volume(self, search_volume: int) -> str:
        """Categorize search volume."""
        if search_volume < self.VOLUME_CATEGORIES['very_low']:
            return 'very_low'
        elif search_volume < self.VOLUME_CATEGORIES['low']:
            return 'low'
        elif search_volume < self.VOLUME_CATEGORIES['medium']:
            return 'medium'
        elif search_volume < self.VOLUME_CATEGORIES['high']:
            return 'high'
        else:
            return 'very_high'

    def _calculate_keyword_difficulty(
        self,
        search_volume: int,
        competing_apps: int
    ) -> float:
        """
        Calculate keyword difficulty score (0-100).
        Higher score = harder to rank.
        """
        if competing_apps == 0:
            return 0.0

        # Competition factor (0-1)
        competition_factor = min(competing_apps / 50000, 1.0)

        # Volume factor (0-1) - higher volume = more difficulty
        volume_factor = min(search_volume / 1000000, 1.0)

        # Difficulty score (weighted average)
        difficulty = (competition_factor * 0.7 + volume_factor * 0.3) * 100

        return round(difficulty, 1)

    def _calculate_potential_score(
        self,
        search_volume: int,
        competing_apps: int,
        relevance_score: float
    ) -> float:
        """
        Calculate overall keyword potential (0-100).
        Higher score = better opportunity.
        """
        # Volume score (0-40 points)
        volume_score = min((search_volume / 100000) * 40, 40)

        # Competition score (0-30 points) - inverse relationship
        if competing_apps > 0:
            competition_score = max(30 - (competing_apps / 500), 0)
        else:
            competition_score = 30

        # Relevance score (0-30 points)
        relevance_points = relevance_score * 30

        total_score = volume_score + competition_score + relevance_points

        return round(min(total_score, 100), 1)

    def _generate_recommendation(
        self,
        potential_score: float,
        difficulty_score: float,
        relevance_score: float
    ) -> str:
        """Generate actionable recommendation for keyword."""
        if relevance_score < 0.5:
            return "Low relevance - avoid targeting"

        if potential_score >= 70:
            return "High priority - target immediately"
        elif potential_score >= 50:
            if difficulty_score < 50:
                return "Good opportunity - include in metadata"
            else:
                return "Competitive - use in description, not title"
        elif potential_score >= 30:
            return "Secondary keyword - use for long-tail variations"
        else:
            return "Low potential - deprioritize"

    def _generate_comparison_summary(
        self,
        primary_keywords: List[Dict[str, Any]],
        secondary_keywords: List[Dict[str, Any]],
        long_tail_keywords: List[Dict[str, Any]]
    ) -> str:
        """Generate summary of keyword comparison."""
        summary_parts = []

        summary_parts.append(
            f"Identified {len(primary_keywords)} high-priority primary keywords."
        )

        if primary_keywords:
            top_keyword = primary_keywords[0]['keyword']
            summary_parts.append(
                f"Top recommendation: '{top_keyword}' (potential score: {primary_keywords[0]['potential_score']})."
            )

        summary_parts.append(
            f"Found {len(secondary_keywords)} secondary keywords for description and metadata."
        )

        summary_parts.append(
            f"Discovered {len(long_tail_keywords)} long-tail opportunities with lower competition."
        )

        return " ".join(summary_parts)


def analyze_keyword_set(keywords_data: List[Dict[str, Any]]) -> Dict[str, Any]:
    """
    Convenience function to analyze a set of keywords.

    Args:
        keywords_data: List of keyword data dictionaries

    Returns:
        Complete analysis report
    """
    analyzer = KeywordAnalyzer()
    return analyzer.compare_keywords(keywords_data)

FILE:scripts/launch_checklist.py
"""
Launch checklist module for App Store Optimization.
Generates comprehensive pre-launch and update checklists.
"""

from typing import Dict, List, Any, Optional
from datetime import datetime, timedelta


class LaunchChecklistGenerator:
    """Generates comprehensive checklists for app launches and updates."""

    def __init__(self, platform: str = 'both'):
        """
        Initialize checklist generator.

        Args:
            platform: 'apple', 'google', or 'both'
        """
        if platform not in ['apple', 'google', 'both']:
            raise ValueError("Platform must be 'apple', 'google', or 'both'")

        self.platform = platform

    def generate_prelaunch_checklist(
        self,
        app_info: Dict[str, Any],
        launch_date: Optional[str] = None
    ) -> Dict[str, Any]:
        """
        Generate comprehensive pre-launch checklist.

        Args:
            app_info: App information (name, category, target_audience)
            launch_date: Target launch date (YYYY-MM-DD)

        Returns:
            Complete pre-launch checklist
        """
        checklist = {
            'app_info': app_info,
            'launch_date': launch_date,
            'checklists': {}
        }

        # Generate platform-specific checklists
        if self.platform in ['apple', 'both']:
            checklist['checklists']['apple'] = self._generate_apple_checklist(app_info)

        if self.platform in ['google', 'both']:
            checklist['checklists']['google'] = self._generate_google_checklist(app_info)

        # Add universal checklist items
        checklist['checklists']['universal'] = self._generate_universal_checklist(app_info)

        # Generate timeline
        if launch_date:
            checklist['timeline'] = self._generate_launch_timeline(launch_date)

        # Calculate completion status
        checklist['summary'] = self._calculate_checklist_summary(checklist['checklists'])

        return checklist

    def validate_app_store_compliance(
        self,
        app_data: Dict[str, Any],
        platform: str = 'apple'
    ) -> Dict[str, Any]:
        """
        Validate compliance with app store guidelines.

        Args:
            app_data: App data including metadata, privacy policy, etc.
            platform: 'apple' or 'google'

        Returns:
            Compliance validation report
        """
        validation_results = {
            'platform': platform,
            'is_compliant': True,
            'errors': [],
            'warnings': [],
            'recommendations': []
        }

        if platform == 'apple':
            self._validate_apple_compliance(app_data, validation_results)
        elif platform == 'google':
            self._validate_google_compliance(app_data, validation_results)

        # Determine overall compliance
        validation_results['is_compliant'] = len(validation_results['errors']) == 0

        return validation_results

    def create_update_plan(
        self,
        current_version: str,
        planned_features: List[str],
        update_frequency: str = 'monthly'
    ) -> Dict[str, Any]:
        """
        Create update cadence and feature rollout plan.

        Args:
            current_version: Current app version
            planned_features: List of planned features
            update_frequency: 'weekly', 'biweekly', 'monthly', 'quarterly'

        Returns:
            Update plan with cadence and feature schedule
        """
        # Calculate next versions
        next_versions = self._calculate_next_versions(
            current_version,
            update_frequency,
            len(planned_features)
        )

        # Distribute features across versions
        feature_schedule = self._distribute_features(
            planned_features,
            next_versions
        )

        # Generate "What's New" templates
        whats_new_templates = [
            self._generate_whats_new_template(version_data)
            for version_data in feature_schedule
        ]

        return {
            'current_version': current_version,
            'update_frequency': update_frequency,
            'planned_updates': len(feature_schedule),
            'feature_schedule': feature_schedule,
            'whats_new_templates': whats_new_templates,
            'recommendations': self._generate_update_recommendations(update_frequency)
        }

    def optimize_launch_timing(
        self,
        app_category: str,
        target_audience: str,
        current_date: Optional[str] = None
    ) -> Dict[str, Any]:
        """
        Recommend optimal launch timing.

        Args:
            app_category: App category
            target_audience: Target audience description
            current_date: Current date (YYYY-MM-DD), defaults to today

        Returns:
            Launch timing recommendations
        """
        if not current_date:
            current_date = datetime.now().strftime('%Y-%m-%d')

        # Analyze launch timing factors
        day_of_week_rec = self._recommend_day_of_week(app_category)
        seasonal_rec = self._recommend_seasonal_timing(app_category, current_date)
        competitive_rec = self._analyze_competitive_timing(app_category)

        # Calculate optimal dates
        optimal_dates = self._calculate_optimal_dates(
            current_date,
            day_of_week_rec,
            seasonal_rec
        )

        return {
            'current_date': current_date,
            'optimal_launch_dates': optimal_dates,
            'day_of_week_recommendation': day_of_week_rec,
            'seasonal_considerations': seasonal_rec,
            'competitive_timing': competitive_rec,
            'final_recommendation': self._generate_timing_recommendation(
                optimal_dates,
                seasonal_rec
            )
        }

    def plan_seasonal_campaigns(
        self,
        app_category: str,
        current_month: int = None
    ) -> Dict[str, Any]:
        """
        Identify seasonal opportunities for ASO campaigns.

        Args:
            app_category: App category
            current_month: Current month (1-12), defaults to current

        Returns:
            Seasonal campaign opportunities
        """
        if not current_month:
            current_month = datetime.now().month

        # Identify relevant seasonal events
        seasonal_opportunities = self._identify_seasonal_opportunities(
            app_category,
            current_month
        )

        # Generate campaign ideas
        campaigns = [
            self._generate_seasonal_campaign(opportunity)
            for opportunity in seasonal_opportunities
        ]

        return {
            'current_month': current_month,
            'category': app_category,
            'seasonal_opportunities': seasonal_opportunities,
            'campaign_ideas': campaigns,
            'implementation_timeline': self._create_seasonal_timeline(campaigns)
        }

    def _generate_apple_checklist(self, app_info: Dict[str, Any]) -> List[Dict[str, Any]]:
        """Generate Apple App Store specific checklist."""
        return [
            {
                'category': 'App Store Connect Setup',
                'items': [
                    {'task': 'App Store Connect account created', 'status': 'pending'},
                    {'task': 'App bundle ID registered', 'status': 'pending'},
                    {'task': 'App Privacy declarations completed', 'status': 'pending'},
                    {'task': 'Age rating questionnaire completed', 'status': 'pending'}
                ]
            },
            {
                'category': 'Metadata (Apple)',
                'items': [
                    {'task': 'App title (30 chars max)', 'status': 'pending'},
                    {'task': 'Subtitle (30 chars max)', 'status': 'pending'},
                    {'task': 'Promotional text (170 chars max)', 'status': 'pending'},
                    {'task': 'Description (4000 chars max)', 'status': 'pending'},
                    {'task': 'Keywords (100 chars, comma-separated)', 'status': 'pending'},
                    {'task': 'Category selection (primary + secondary)', 'status': 'pending'}
                ]
            },
            {
                'category': 'Visual Assets (Apple)',
                'items': [
                    {'task': 'App icon (1024x1024px)', 'status': 'pending'},
                    {'task': 'Screenshots (iPhone 6.7" required)', 'status': 'pending'},
                    {'task': 'Screenshots (iPhone 5.5" required)', 'status': 'pending'},
                    {'task': 'Screenshots (iPad Pro 12.9" if iPad app)', 'status': 'pending'},
                    {'task': 'App preview video (optional but recommended)', 'status': 'pending'}
                ]
            },
            {
                'category': 'Technical Requirements (Apple)',
                'items': [
                    {'task': 'Build uploaded to App Store Connect', 'status': 'pending'},
                    {'task': 'TestFlight testing completed', 'status': 'pending'},
                    {'task': 'App tested on required iOS versions', 'status': 'pending'},
                    {'task': 'Crash-free rate > 99%', 'status': 'pending'},
                    {'task': 'All links in app/metadata working', 'status': 'pending'}
                ]
            },
            {
                'category': 'Legal & Privacy (Apple)',
                'items': [
                    {'task': 'Privacy Policy URL provided', 'status': 'pending'},
                    {'task': 'Terms of Service URL (if applicable)', 'status': 'pending'},
                    {'task': 'Data collection declarations accurate', 'status': 'pending'},
                    {'task': 'Third-party SDKs disclosed', 'status': 'pending'}
                ]
            }
        ]

    def _generate_google_checklist(self, app_info: Dict[str, Any]) -> List[Dict[str, Any]]:
        """Generate Google Play Store specific checklist."""
        return [
            {
                'category': 'Play Console Setup',
                'items': [
                    {'task': 'Google Play Console account created', 'status': 'pending'},
                    {'task': 'Developer profile completed', 'status': 'pending'},
                    {'task': 'Payment merchant account linked (if paid app)', 'status': 'pending'},
                    {'task': 'Content rating questionnaire completed', 'status': 'pending'}
                ]
            },
            {
                'category': 'Metadata (Google)',
                'items': [
                    {'task': 'App title (50 chars max)', 'status': 'pending'},
                    {'task': 'Short description (80 chars max)', 'status': 'pending'},
                    {'task': 'Full description (4000 chars max)', 'status': 'pending'},
                    {'task': 'Category selection', 'status': 'pending'},
                    {'task': 'Tags (up to 5)', 'status': 'pending'}
                ]
            },
            {
                'category': 'Visual Assets (Google)',
                'items': [
                    {'task': 'App icon (512x512px)', 'status': 'pending'},
                    {'task': 'Feature graphic (1024x500px)', 'status': 'pending'},
                    {'task': 'Screenshots (2-8 required, phone)', 'status': 'pending'},
                    {'task': 'Screenshots (tablet, if applicable)', 'status': 'pending'},
                    {'task': 'Promo video (YouTube link, optional)', 'status': 'pending'}
                ]
            },
            {
                'category': 'Technical Requirements (Google)',
                'items': [
                    {'task': 'APK/AAB uploaded to Play Console', 'status': 'pending'},
                    {'task': 'Internal testing completed', 'status': 'pending'},
                    {'task': 'App tested on required Android versions', 'status': 'pending'},
                    {'task': 'Target API level meets requirements', 'status': 'pending'},
                    {'task': 'All permissions justified', 'status': 'pending'}
                ]
            },
            {
                'category': 'Legal & Privacy (Google)',
                'items': [
                    {'task': 'Privacy Policy URL provided', 'status': 'pending'},
                    {'task': 'Data safety section completed', 'status': 'pending'},
                    {'task': 'Ads disclosure (if applicable)', 'status': 'pending'},
                    {'task': 'In-app purchase disclosure (if applicable)', 'status': 'pending'}
                ]
            }
        ]

    def _generate_universal_checklist(self, app_info: Dict[str, Any]) -> List[Dict[str, Any]]:
        """Generate universal (both platforms) checklist."""
        return [
            {
                'category': 'Pre-Launch Marketing',
                'items': [
                    {'task': 'Landing page created', 'status': 'pending'},
                    {'task': 'Social media accounts setup', 'status': 'pending'},
                    {'task': 'Press kit prepared', 'status': 'pending'},
                    {'task': 'Beta tester feedback collected', 'status': 'pending'},
                    {'task': 'Launch announcement drafted', 'status': 'pending'}
                ]
            },
            {
                'category': 'ASO Preparation',
                'items': [
                    {'task': 'Keyword research completed', 'status': 'pending'},
                    {'task': 'Competitor analysis done', 'status': 'pending'},
                    {'task': 'A/B test plan created for post-launch', 'status': 'pending'},
                    {'task': 'Analytics tracking configured', 'status': 'pending'}
                ]
            },
            {
                'category': 'Quality Assurance',
                'items': [
                    {'task': 'All core features tested', 'status': 'pending'},
                    {'task': 'User flows validated', 'status': 'pending'},
                    {'task': 'Performance testing completed', 'status': 'pending'},
                    {'task': 'Accessibility features tested', 'status': 'pending'},
                    {'task': 'Security audit completed', 'status': 'pending'}
                ]
            },
            {
                'category': 'Support Infrastructure',
                'items': [
                    {'task': 'Support email/system setup', 'status': 'pending'},
                    {'task': 'FAQ page created', 'status': 'pending'},
                    {'task': 'Documentation for users prepared', 'status': 'pending'},
                    {'task': 'Team trained on handling reviews', 'status': 'pending'}
                ]
            }
        ]

    def _generate_launch_timeline(self, launch_date: str) -> List[Dict[str, Any]]:
        """Generate timeline with milestones leading to launch."""
        launch_dt = datetime.strptime(launch_date, '%Y-%m-%d')

        milestones = [
            {
                'date': (launch_dt - timedelta(days=90)).strftime('%Y-%m-%d'),
                'milestone': '90 days before: Complete keyword research and competitor analysis'
            },
            {
                'date': (launch_dt - timedelta(days=60)).strftime('%Y-%m-%d'),
                'milestone': '60 days before: Finalize metadata and visual assets'
            },
            {
                'date': (launch_dt - timedelta(days=45)).strftime('%Y-%m-%d'),
                'milestone': '45 days before: Begin beta testing program'
            },
            {
                'date': (launch_dt - timedelta(days=30)).strftime('%Y-%m-%d'),
                'milestone': '30 days before: Submit app for review (Apple typically takes 1-2 days, Google instant)'
            },
            {
                'date': (launch_dt - timedelta(days=14)).strftime('%Y-%m-%d'),
                'milestone': '14 days before: Prepare launch marketing materials'
            },
            {
                'date': (launch_dt - timedelta(days=7)).strftime('%Y-%m-%d'),
                'milestone': '7 days before: Set up analytics and monitoring'
            },
            {
                'date': launch_dt.strftime('%Y-%m-%d'),
                'milestone': 'Launch Day: Release app and execute marketing plan'
            },
            {
                'date': (launch_dt + timedelta(days=7)).strftime('%Y-%m-%d'),
                'milestone': '7 days after: Monitor metrics, respond to reviews, address critical issues'
            },
            {
                'date': (launch_dt + timedelta(days=30)).strftime('%Y-%m-%d'),
                'milestone': '30 days after: Analyze launch metrics, plan first update'
            }
        ]

        return milestones

    def _calculate_checklist_summary(self, checklists: Dict[str, List[Dict[str, Any]]]) -> Dict[str, Any]:
        """Calculate completion summary."""
        total_items = 0
        completed_items = 0

        for platform, categories in checklists.items():
            for category in categories:
                for item in category['items']:
                    total_items += 1
                    if item['status'] == 'completed':
                        completed_items += 1

        completion_percentage = (completed_items / total_items * 100) if total_items > 0 else 0

        return {
            'total_items': total_items,
            'completed_items': completed_items,
            'pending_items': total_items - completed_items,
            'completion_percentage': round(completion_percentage, 1),
            'is_ready_to_launch': completion_percentage == 100
        }

    def _validate_apple_compliance(
        self,
        app_data: Dict[str, Any],
        validation_results: Dict[str, Any]
    ) -> None:
        """Validate Apple App Store compliance."""
        # Check for required fields
        if not app_data.get('privacy_policy_url'):
            validation_results['errors'].append("Privacy Policy URL is required")

        if not app_data.get('app_icon'):
            validation_results['errors'].append("App icon (1024x1024px) is required")

        # Check metadata character limits
        title = app_data.get('title', '')
        if len(title) > 30:
            validation_results['errors'].append(f"Title exceeds 30 characters ({len(title)})")

        # Warnings for best practices
        subtitle = app_data.get('subtitle', '')
        if not subtitle:
            validation_results['warnings'].append("Subtitle is empty - consider adding for better discoverability")

        keywords = app_data.get('keywords', '')
        if len(keywords) < 80:
            validation_results['warnings'].append(
                f"Keywords field underutilized ({len(keywords)}/100 chars) - add more keywords"
            )

    def _validate_google_compliance(
        self,
        app_data: Dict[str, Any],
        validation_results: Dict[str, Any]
    ) -> None:
        """Validate Google Play Store compliance."""
        # Check for required fields
        if not app_data.get('privacy_policy_url'):
            validation_results['errors'].append("Privacy Policy URL is required")

        if not app_data.get('feature_graphic'):
            validation_results['errors'].append("Feature graphic (1024x500px) is required")

        # Check metadata character limits
        title = app_data.get('title', '')
        if len(title) > 50:
            validation_results['errors'].append(f"Title exceeds 50 characters ({len(title)})")

        short_desc = app_data.get('short_description', '')
        if len(short_desc) > 80:
            validation_results['errors'].append(f"Short description exceeds 80 characters ({len(short_desc)})")

        # Warnings
        if not short_desc:
            validation_results['warnings'].append("Short description is empty")

    def _calculate_next_versions(
        self,
        current_version: str,
        update_frequency: str,
        feature_count: int
    ) -> List[str]:
        """Calculate next version numbers."""
        # Parse current version (assume semantic versioning)
        parts = current_version.split('.')
        major, minor, patch = int(parts[0]), int(parts[1]), int(parts[2] if len(parts) > 2 else 0)

        versions = []
        for i in range(feature_count):
            if update_frequency == 'weekly':
                patch += 1
            elif update_frequency == 'biweekly':
                patch += 1
            elif update_frequency == 'monthly':
                minor += 1
                patch = 0
            else:  # quarterly
                minor += 1
                patch = 0

            versions.append(f"{major}.{minor}.{patch}")

        return versions

    def _distribute_features(
        self,
        features: List[str],
        versions: List[str]
    ) -> List[Dict[str, Any]]:
        """Distribute features across versions."""
        features_per_version = max(1, len(features) // len(versions))

        schedule = []
        for i, version in enumerate(versions):
            start_idx = i * features_per_version
            end_idx = start_idx + features_per_version if i < len(versions) - 1 else len(features)

            schedule.append({
                'version': version,
                'features': features[start_idx:end_idx],
                'release_priority': 'high' if i == 0 else ('medium' if i < len(versions) // 2 else 'low')
            })

        return schedule

    def _generate_whats_new_template(self, version_data: Dict[str, Any]) -> Dict[str, str]:
        """Generate What's New template for version."""
        features_list = '\n'.join([f"• {feature}" for feature in version_data['features']])

        template = f"""Version {version_data['version']}

{features_list}

We're constantly improving your experience. Thanks for using [App Name]!

Have feedback? Contact us at support@[company].com"""

        return {
            'version': version_data['version'],
            'template': template
        }

    def _generate_update_recommendations(self, update_frequency: str) -> List[str]:
        """Generate recommendations for update strategy."""
        recommendations = []

        if update_frequency == 'weekly':
            recommendations.append("Weekly updates show active development but ensure quality doesn't suffer")
        elif update_frequency == 'monthly':
            recommendations.append("Monthly updates are optimal for most apps - balance features and stability")

        recommendations.extend([
            "Include bug fixes in every update",
            "Update 'What's New' section with each release",
            "Respond to reviews mentioning fixed issues"
        ])

        return recommendations

    def _recommend_day_of_week(self, app_category: str) -> Dict[str, Any]:
        """Recommend best day of week to launch."""
        # General recommendations based on category
        if app_category.lower() in ['games', 'entertainment']:
            return {
                'recommended_day': 'Thursday',
                'rationale': 'People download entertainment apps before weekend'
            }
        elif app_category.lower() in ['productivity', 'business']:
            return {
                'recommended_day': 'Tuesday',
                'rationale': 'Business users most active mid-week'
            }
        else:
            return {
                'recommended_day': 'Wednesday',
                'rationale': 'Mid-week provides good balance and review potential'
            }

    def _recommend_seasonal_timing(self, app_category: str, current_date: str) -> Dict[str, Any]:
        """Recommend seasonal timing considerations."""
        current_dt = datetime.strptime(current_date, '%Y-%m-%d')
        month = current_dt.month

        # Avoid certain periods
        avoid_periods = []
        if month == 12:
            avoid_periods.append("Late December - low user engagement during holidays")
        if month in [7, 8]:
            avoid_periods.append("Summer months - some categories see lower engagement")

        # Recommend periods
        good_periods = []
        if month in [1, 9]:
            good_periods.append("New Year/Back-to-school - high user engagement")
        if month in [10, 11]:
            good_periods.append("Pre-holiday season - good for shopping/gift apps")

        return {
            'current_month': month,
            'avoid_periods': avoid_periods,
            'good_periods': good_periods
        }

    def _analyze_competitive_timing(self, app_category: str) -> Dict[str, str]:
        """Analyze competitive timing considerations."""
        return {
            'recommendation': 'Research competitor launch schedules in your category',
            'strategy': 'Avoid launching same week as major competitor updates'
        }

    def _calculate_optimal_dates(
        self,
        current_date: str,
        day_rec: Dict[str, Any],
        seasonal_rec: Dict[str, Any]
    ) -> List[str]:
        """Calculate optimal launch dates."""
        current_dt = datetime.strptime(current_date, '%Y-%m-%d')

        # Find next occurrence of recommended day
        target_day = day_rec['recommended_day']
        days_map = {'Monday': 0, 'Tuesday': 1, 'Wednesday': 2, 'Thursday': 3, 'Friday': 4}
        target_day_num = days_map.get(target_day, 2)

        days_ahead = (target_day_num - current_dt.weekday()) % 7
        if days_ahead == 0:
            days_ahead = 7

        next_target_date = current_dt + timedelta(days=days_ahead)

        optimal_dates = [
            next_target_date.strftime('%Y-%m-%d'),
            (next_target_date + timedelta(days=7)).strftime('%Y-%m-%d'),
            (next_target_date + timedelta(days=14)).strftime('%Y-%m-%d')
        ]

        return optimal_dates

    def _generate_timing_recommendation(
        self,
        optimal_dates: List[str],
        seasonal_rec: Dict[str, Any]
    ) -> str:
        """Generate final timing recommendation."""
        if seasonal_rec['avoid_periods']:
            return f"Consider launching in {optimal_dates[1]} to avoid {seasonal_rec['avoid_periods'][0]}"
        elif seasonal_rec['good_periods']:
            return f"Launch on {optimal_dates[0]} to capitalize on {seasonal_rec['good_periods'][0]}"
        else:
            return f"Recommended launch date: {optimal_dates[0]}"

    def _identify_seasonal_opportunities(
        self,
        app_category: str,
        current_month: int
    ) -> List[Dict[str, Any]]:
        """Identify seasonal opportunities for category."""
        opportunities = []

        # Universal opportunities
        if current_month == 1:
            opportunities.append({
                'event': 'New Year Resolutions',
                'dates': 'January 1-31',
                'relevance': 'high' if app_category.lower() in ['health', 'fitness', 'productivity'] else 'medium'
            })

        if current_month in [11, 12]:
            opportunities.append({
                'event': 'Holiday Shopping Season',
                'dates': 'November-December',
                'relevance': 'high' if app_category.lower() in ['shopping', 'gifts'] else 'low'
            })

        # Category-specific
        if app_category.lower() == 'education' and current_month in [8, 9]:
            opportunities.append({
                'event': 'Back to School',
                'dates': 'August-September',
                'relevance': 'high'
            })

        return opportunities

    def _generate_seasonal_campaign(self, opportunity: Dict[str, Any]) -> Dict[str, Any]:
        """Generate campaign idea for seasonal opportunity."""
        return {
            'event': opportunity['event'],
            'campaign_idea': f"Create themed visuals and messaging for {opportunity['event']}",
            'metadata_updates': 'Update app description and screenshots with seasonal themes',
            'promotion_strategy': 'Consider limited-time features or discounts'
        }

    def _create_seasonal_timeline(self, campaigns: List[Dict[str, Any]]) -> List[str]:
        """Create implementation timeline for campaigns."""
        return [
            f"30 days before: Plan {campaign['event']} campaign strategy"
            for campaign in campaigns
        ]


def generate_launch_checklist(
    platform: str,
    app_info: Dict[str, Any],
    launch_date: Optional[str] = None
) -> Dict[str, Any]:
    """
    Convenience function to generate launch checklist.

    Args:
        platform: Platform ('apple', 'google', or 'both')
        app_info: App information
        launch_date: Target launch date

    Returns:
        Complete launch checklist
    """
    generator = LaunchChecklistGenerator(platform)
    return generator.generate_prelaunch_checklist(app_info, launch_date)

FILE:scripts/localization_helper.py
"""
Localization helper module for App Store Optimization.
Manages multi-language ASO optimization strategies.
"""

from typing import Dict, List, Any, Optional, Tuple


class LocalizationHelper:
    """Helps manage multi-language ASO optimization."""

    # Priority markets by language (based on app store revenue and user base)
    PRIORITY_MARKETS = {
        'tier_1': [
            {'language': 'en-US', 'market': 'United States', 'revenue_share': 0.25},
            {'language': 'zh-CN', 'market': 'China', 'revenue_share': 0.20},
            {'language': 'ja-JP', 'market': 'Japan', 'revenue_share': 0.10},
            {'language': 'de-DE', 'market': 'Germany', 'revenue_share': 0.08},
            {'language': 'en-GB', 'market': 'United Kingdom', 'revenue_share': 0.06}
        ],
        'tier_2': [
            {'language': 'fr-FR', 'market': 'France', 'revenue_share': 0.05},
            {'language': 'ko-KR', 'market': 'South Korea', 'revenue_share': 0.05},
            {'language': 'es-ES', 'market': 'Spain', 'revenue_share': 0.03},
            {'language': 'it-IT', 'market': 'Italy', 'revenue_share': 0.03},
            {'language': 'pt-BR', 'market': 'Brazil', 'revenue_share': 0.03}
        ],
        'tier_3': [
            {'language': 'ru-RU', 'market': 'Russia', 'revenue_share': 0.02},
            {'language': 'es-MX', 'market': 'Mexico', 'revenue_share': 0.02},
            {'language': 'nl-NL', 'market': 'Netherlands', 'revenue_share': 0.02},
            {'language': 'sv-SE', 'market': 'Sweden', 'revenue_share': 0.01},
            {'language': 'pl-PL', 'market': 'Poland', 'revenue_share': 0.01}
        ]
    }

    # Character limit multipliers by language (some languages need more/less space)
    CHAR_MULTIPLIERS = {
        'en': 1.0,
        'zh': 0.6,  # Chinese characters are more compact
        'ja': 0.7,  # Japanese uses kanji
        'ko': 0.8,  # Korean is relatively compact
        'de': 1.3,  # German words are typically longer
        'fr': 1.2,  # French tends to be longer
        'es': 1.1,  # Spanish slightly longer
        'pt': 1.1,  # Portuguese similar to Spanish
        'ru': 1.1,  # Russian similar length
        'ar': 1.0,  # Arabic varies
        'it': 1.1   # Italian similar to Spanish
    }

    def __init__(self, app_category: str = 'general'):
        """
        Initialize localization helper.

        Args:
            app_category: App category to prioritize relevant markets
        """
        self.app_category = app_category
        self.localization_plans = []

    def identify_target_markets(
        self,
        current_market: str = 'en-US',
        budget_level: str = 'medium',
        target_market_count: int = 5
    ) -> Dict[str, Any]:
        """
        Recommend priority markets for localization.

        Args:
            current_market: Current/primary market
            budget_level: 'low', 'medium', or 'high'
            target_market_count: Number of markets to target

        Returns:
            Prioritized market recommendations
        """
        # Determine tier priorities based on budget
        if budget_level == 'low':
            priority_tiers = ['tier_1']
            max_markets = min(target_market_count, 3)
        elif budget_level == 'medium':
            priority_tiers = ['tier_1', 'tier_2']
            max_markets = min(target_market_count, 8)
        else:  # high budget
            priority_tiers = ['tier_1', 'tier_2', 'tier_3']
            max_markets = target_market_count

        # Collect markets from priority tiers
        recommended_markets = []
        for tier in priority_tiers:
            for market in self.PRIORITY_MARKETS[tier]:
                if market['language'] != current_market:
                    recommended_markets.append({
                        **market,
                        'tier': tier,
                        'estimated_translation_cost': self._estimate_translation_cost(
                            market['language']
                        )
                    })

        # Sort by revenue share and limit
        recommended_markets.sort(key=lambda x: x['revenue_share'], reverse=True)
        recommended_markets = recommended_markets[:max_markets]

        # Calculate potential ROI
        total_potential_revenue_share = sum(m['revenue_share'] for m in recommended_markets)

        return {
            'recommended_markets': recommended_markets,
            'total_markets': len(recommended_markets),
            'estimated_total_revenue_lift': f"{total_potential_revenue_share*100:.1f}%",
            'estimated_cost': self._estimate_total_localization_cost(recommended_markets),
            'implementation_priority': self._prioritize_implementation(recommended_markets)
        }

    def translate_metadata(
        self,
        source_metadata: Dict[str, str],
        source_language: str,
        target_language: str,
        platform: str = 'apple'
    ) -> Dict[str, Any]:
        """
        Generate localized metadata with character limit considerations.

        Args:
            source_metadata: Original metadata (title, description, etc.)
            source_language: Source language code (e.g., 'en')
            target_language: Target language code (e.g., 'es')
            platform: 'apple' or 'google'

        Returns:
            Localized metadata with character limit validation
        """
        # Get character multiplier
        target_lang_code = target_language.split('-')[0]
        char_multiplier = self.CHAR_MULTIPLIERS.get(target_lang_code, 1.0)

        # Platform-specific limits
        if platform == 'apple':
            limits = {'title': 30, 'subtitle': 30, 'description': 4000, 'keywords': 100}
        else:
            limits = {'title': 50, 'short_description': 80, 'description': 4000}

        localized_metadata = {}
        warnings = []

        for field, text in source_metadata.items():
            if field not in limits:
                continue

            # Estimate target length
            estimated_length = int(len(text) * char_multiplier)
            limit = limits[field]

            localized_metadata[field] = {
                'original_text': text,
                'original_length': len(text),
                'estimated_target_length': estimated_length,
                'character_limit': limit,
                'fits_within_limit': estimated_length <= limit,
                'translation_notes': self._get_translation_notes(
                    field,
                    target_language,
                    estimated_length,
                    limit
                )
            }

            if estimated_length > limit:
                warnings.append(
                    f"{field}: Estimated length ({estimated_length}) may exceed limit ({limit}) - "
                    f"condensing may be required"
                )

        return {
            'source_language': source_language,
            'target_language': target_language,
            'platform': platform,
            'localized_fields': localized_metadata,
            'character_multiplier': char_multiplier,
            'warnings': warnings,
            'recommendations': self._generate_translation_recommendations(
                target_language,
                warnings
            )
        }

    def adapt_keywords(
        self,
        source_keywords: List[str],
        source_language: str,
        target_language: str,
        target_market: str
    ) -> Dict[str, Any]:
        """
        Adapt keywords for target market (not just direct translation).

        Args:
            source_keywords: Original keywords
            source_language: Source language code
            target_language: Target language code
            target_market: Target market (e.g., 'France', 'Japan')

        Returns:
            Adapted keyword recommendations
        """
        # Cultural adaptation considerations
        cultural_notes = self._get_cultural_keyword_considerations(target_market)

        # Search behavior differences
        search_patterns = self._get_search_patterns(target_market)

        adapted_keywords = []
        for keyword in source_keywords:
            adapted_keywords.append({
                'source_keyword': keyword,
                'adaptation_strategy': self._determine_adaptation_strategy(
                    keyword,
                    target_market
                ),
                'cultural_considerations': cultural_notes.get(keyword, []),
                'priority': 'high' if keyword in source_keywords[:3] else 'medium'
            })

        return {
            'source_language': source_language,
            'target_language': target_language,
            'target_market': target_market,
            'adapted_keywords': adapted_keywords,
            'search_behavior_notes': search_patterns,
            'recommendations': [
                'Use native speakers for keyword research',
                'Test keywords with local users before finalizing',
                'Consider local competitors\' keyword strategies',
                'Monitor search trends in target market'
            ]
        }

    def validate_translations(
        self,
        translated_metadata: Dict[str, str],
        target_language: str,
        platform: str = 'apple'
    ) -> Dict[str, Any]:
        """
        Validate translated metadata for character limits and quality.

        Args:
            translated_metadata: Translated text fields
            target_language: Target language code
            platform: 'apple' or 'google'

        Returns:
            Validation report
        """
        # Platform limits
        if platform == 'apple':
            limits = {'title': 30, 'subtitle': 30, 'description': 4000, 'keywords': 100}
        else:
            limits = {'title': 50, 'short_description': 80, 'description': 4000}

        validation_results = {
            'is_valid': True,
            'field_validations': {},
            'errors': [],
            'warnings': []
        }

        for field, text in translated_metadata.items():
            if field not in limits:
                continue

            actual_length = len(text)
            limit = limits[field]
            is_within_limit = actual_length <= limit

            validation_results['field_validations'][field] = {
                'text': text,
                'length': actual_length,
                'limit': limit,
                'is_valid': is_within_limit,
                'usage_percentage': round((actual_length / limit) * 100, 1)
            }

            if not is_within_limit:
                validation_results['is_valid'] = False
                validation_results['errors'].append(
                    f"{field} exceeds limit: {actual_length}/{limit} characters"
                )

        # Quality checks
        quality_issues = self._check_translation_quality(
            translated_metadata,
            target_language
        )

        validation_results['quality_checks'] = quality_issues

        if quality_issues:
            validation_results['warnings'].extend(
                [f"Quality issue: {issue}" for issue in quality_issues]
            )

        return validation_results

    def calculate_localization_roi(
        self,
        target_markets: List[str],
        current_monthly_downloads: int,
        localization_cost: float,
        expected_lift_percentage: float = 0.15
    ) -> Dict[str, Any]:
        """
        Estimate ROI of localization investment.

        Args:
            target_markets: List of market codes
            current_monthly_downloads: Current monthly downloads
            localization_cost: Total cost to localize
            expected_lift_percentage: Expected download increase (default 15%)

        Returns:
            ROI analysis
        """
        # Estimate market-specific lift
        market_data = []
        total_expected_lift = 0

        for market_code in target_markets:
            # Find market in priority lists
            market_info = None
            for tier_name, markets in self.PRIORITY_MARKETS.items():
                for m in markets:
                    if m['language'] == market_code:
                        market_info = m
                        break

            if not market_info:
                continue

            # Estimate downloads from this market
            market_downloads = int(current_monthly_downloads * market_info['revenue_share'])
            expected_increase = int(market_downloads * expected_lift_percentage)
            total_expected_lift += expected_increase

            market_data.append({
                'market': market_info['market'],
                'current_monthly_downloads': market_downloads,
                'expected_increase': expected_increase,
                'revenue_potential': market_info['revenue_share']
            })

        # Calculate payback period (assuming $2 revenue per download)
        revenue_per_download = 2.0
        monthly_additional_revenue = total_expected_lift * revenue_per_download
        payback_months = (localization_cost / monthly_additional_revenue) if monthly_additional_revenue > 0 else float('inf')

        return {
            'markets_analyzed': len(market_data),
            'market_breakdown': market_data,
            'total_expected_monthly_lift': total_expected_lift,
            'expected_monthly_revenue_increase': f",.2f",
            'localization_cost': f",.2f",
            'payback_period_months': round(payback_months, 1) if payback_months != float('inf') else 'N/A',
            'annual_roi': f"{((monthly_additional_revenue * 12 - localization_cost) / localization_cost * 100):.1f}%" if payback_months != float('inf') else 'Negative',
            'recommendation': self._generate_roi_recommendation(payback_months)
        }

    def _estimate_translation_cost(self, language: str) -> Dict[str, float]:
        """Estimate translation cost for a language."""
        # Base cost per word (professional translation)
        base_cost_per_word = 0.12

        # Language-specific multipliers
        multipliers = {
            'zh-CN': 1.5,  # Chinese requires specialist
            'ja-JP': 1.5,  # Japanese requires specialist
            'ko-KR': 1.3,
            'ar-SA': 1.4,  # Arabic (right-to-left)
            'default': 1.0
        }

        multiplier = multipliers.get(language, multipliers['default'])

        # Typical word counts for app store metadata
        typical_word_counts = {
            'title': 5,
            'subtitle': 5,
            'description': 300,
            'keywords': 20,
            'screenshots': 50  # Caption text
        }

        total_words = sum(typical_word_counts.values())
        estimated_cost = total_words * base_cost_per_word * multiplier

        return {
            'cost_per_word': base_cost_per_word * multiplier,
            'total_words': total_words,
            'estimated_cost': round(estimated_cost, 2)
        }

    def _estimate_total_localization_cost(self, markets: List[Dict[str, Any]]) -> str:
        """Estimate total cost for multiple markets."""
        total = sum(m['estimated_translation_cost']['estimated_cost'] for m in markets)
        return f",.2f"

    def _prioritize_implementation(self, markets: List[Dict[str, Any]]) -> List[Dict[str, str]]:
        """Create phased implementation plan."""
        phases = []

        # Phase 1: Top revenue markets
        phase_1 = [m for m in markets[:3]]
        if phase_1:
            phases.append({
                'phase': 'Phase 1 (First 30 days)',
                'markets': ', '.join([m['market'] for m in phase_1]),
                'rationale': 'Highest revenue potential markets'
            })

        # Phase 2: Remaining tier 1 and top tier 2
        phase_2 = [m for m in markets[3:6]]
        if phase_2:
            phases.append({
                'phase': 'Phase 2 (Days 31-60)',
                'markets': ', '.join([m['market'] for m in phase_2]),
                'rationale': 'Strong revenue markets with good ROI'
            })

        # Phase 3: Remaining markets
        phase_3 = [m for m in markets[6:]]
        if phase_3:
            phases.append({
                'phase': 'Phase 3 (Days 61-90)',
                'markets': ', '.join([m['market'] for m in phase_3]),
                'rationale': 'Complete global coverage'
            })

        return phases

    def _get_translation_notes(
        self,
        field: str,
        target_language: str,
        estimated_length: int,
        limit: int
    ) -> List[str]:
        """Get translation-specific notes for field."""
        notes = []

        if estimated_length > limit:
            notes.append(f"Condensing required - aim for {limit - 10} characters to allow buffer")

        if field == 'title' and target_language.startswith('zh'):
            notes.append("Chinese characters convey more meaning - may need fewer characters")

        if field == 'keywords' and target_language.startswith('de'):
            notes.append("German compound words may be longer - prioritize shorter keywords")

        return notes

    def _generate_translation_recommendations(
        self,
        target_language: str,
        warnings: List[str]
    ) -> List[str]:
        """Generate translation recommendations."""
        recommendations = [
            "Use professional native speakers for translation",
            "Test translations with local users before finalizing"
        ]

        if warnings:
            recommendations.append("Work with translator to condense text while preserving meaning")

        if target_language.startswith('zh') or target_language.startswith('ja'):
            recommendations.append("Consider cultural context and local idioms")

        return recommendations

    def _get_cultural_keyword_considerations(self, target_market: str) -> Dict[str, List[str]]:
        """Get cultural considerations for keywords by market."""
        # Simplified example - real implementation would be more comprehensive
        considerations = {
            'China': ['Avoid politically sensitive terms', 'Consider local alternatives to blocked services'],
            'Japan': ['Honorific language important', 'Technical terms often use katakana'],
            'Germany': ['Privacy and security terms resonate', 'Efficiency and quality valued'],
            'France': ['French language protection laws', 'Prefer French terms over English'],
            'default': ['Research local search behavior', 'Test with native speakers']
        }

        return considerations.get(target_market, considerations['default'])

    def _get_search_patterns(self, target_market: str) -> List[str]:
        """Get search pattern notes for market."""
        patterns = {
            'China': ['Use both simplified characters and romanization', 'Brand names often romanized'],
            'Japan': ['Mix of kanji, hiragana, and katakana', 'English words common in tech'],
            'Germany': ['Compound words common', 'Specific technical terminology'],
            'default': ['Research local search trends', 'Monitor competitor keywords']
        }

        return patterns.get(target_market, patterns['default'])

    def _determine_adaptation_strategy(self, keyword: str, target_market: str) -> str:
        """Determine how to adapt keyword for market."""
        # Simplified logic
        if target_market in ['China', 'Japan', 'Korea']:
            return 'full_localization'  # Complete translation needed
        elif target_market in ['Germany', 'France', 'Spain']:
            return 'adapt_and_translate'  # Some adaptation needed
        else:
            return 'direct_translation'  # Direct translation usually sufficient

    def _check_translation_quality(
        self,
        translated_metadata: Dict[str, str],
        target_language: str
    ) -> List[str]:
        """Basic quality checks for translations."""
        issues = []

        # Check for untranslated placeholders
        for field, text in translated_metadata.items():
            if '[' in text or '{' in text or 'TODO' in text.upper():
                issues.append(f"{field} contains placeholder text")

        # Check for excessive punctuation
        for field, text in translated_metadata.items():
            if text.count('!') > 3:
                issues.append(f"{field} has excessive exclamation marks")

        return issues

    def _generate_roi_recommendation(self, payback_months: float) -> str:
        """Generate ROI recommendation."""
        if payback_months <= 3:
            return "Excellent ROI - proceed immediately"
        elif payback_months <= 6:
            return "Good ROI - recommended investment"
        elif payback_months <= 12:
            return "Moderate ROI - consider if strategic market"
        else:
            return "Low ROI - reconsider or focus on higher-priority markets first"


def plan_localization_strategy(
    current_market: str,
    budget_level: str,
    monthly_downloads: int
) -> Dict[str, Any]:
    """
    Convenience function to plan localization strategy.

    Args:
        current_market: Current market code
        budget_level: Budget level
        monthly_downloads: Current monthly downloads

    Returns:
        Complete localization plan
    """
    helper = LocalizationHelper()

    target_markets = helper.identify_target_markets(
        current_market=current_market,
        budget_level=budget_level
    )

    # Extract market codes
    market_codes = [m['language'] for m in target_markets['recommended_markets']]

    # Calculate ROI
    estimated_cost = float(target_markets['estimated_cost'].replace('$', '').replace(',', ''))

    roi_analysis = helper.calculate_localization_roi(
        market_codes,
        monthly_downloads,
        estimated_cost
    )

    return {
        'target_markets': target_markets,
        'roi_analysis': roi_analysis
    }

FILE:scripts/metadata_optimizer.py
"""
Metadata optimization module for App Store Optimization.
Optimizes titles, descriptions, and keyword fields with platform-specific character limit validation.
"""

from typing import Dict, List, Any, Optional, Tuple
import re


class MetadataOptimizer:
    """Optimizes app store metadata for maximum discoverability and conversion."""

    # Platform-specific character limits
    CHAR_LIMITS = {
        'apple': {
            'title': 30,
            'subtitle': 30,
            'promotional_text': 170,
            'description': 4000,
            'keywords': 100,
            'whats_new': 4000
        },
        'google': {
            'title': 50,
            'short_description': 80,
            'full_description': 4000
        }
    }

    def __init__(self, platform: str = 'apple'):
        """
        Initialize metadata optimizer.

        Args:
            platform: 'apple' or 'google'
        """
        if platform not in ['apple', 'google']:
            raise ValueError("Platform must be 'apple' or 'google'")

        self.platform = platform
        self.limits = self.CHAR_LIMITS[platform]

    def optimize_title(
        self,
        app_name: str,
        target_keywords: List[str],
        include_brand: bool = True
    ) -> Dict[str, Any]:
        """
        Optimize app title with keyword integration.

        Args:
            app_name: Your app's brand name
            target_keywords: List of keywords to potentially include
            include_brand: Whether to include brand name

        Returns:
            Optimized title options with analysis
        """
        max_length = self.limits['title']

        title_options = []

        # Option 1: Brand name only
        if include_brand:
            option1 = app_name[:max_length]
            title_options.append({
                'title': option1,
                'length': len(option1),
                'remaining_chars': max_length - len(option1),
                'keywords_included': [],
                'strategy': 'brand_only',
                'pros': ['Maximum brand recognition', 'Clean and simple'],
                'cons': ['No keyword targeting', 'Lower discoverability']
            })

        # Option 2: Brand + Primary Keyword
        if target_keywords:
            primary_keyword = target_keywords[0]
            option2 = self._build_title_with_keywords(
                app_name,
                [primary_keyword],
                max_length
            )
            if option2:
                title_options.append({
                    'title': option2,
                    'length': len(option2),
                    'remaining_chars': max_length - len(option2),
                    'keywords_included': [primary_keyword],
                    'strategy': 'brand_plus_primary',
                    'pros': ['Targets main keyword', 'Maintains brand identity'],
                    'cons': ['Limited keyword coverage']
                })

        # Option 3: Brand + Multiple Keywords (if space allows)
        if len(target_keywords) > 1:
            option3 = self._build_title_with_keywords(
                app_name,
                target_keywords[:2],
                max_length
            )
            if option3:
                title_options.append({
                    'title': option3,
                    'length': len(option3),
                    'remaining_chars': max_length - len(option3),
                    'keywords_included': target_keywords[:2],
                    'strategy': 'brand_plus_multiple',
                    'pros': ['Multiple keyword targets', 'Better discoverability'],
                    'cons': ['May feel cluttered', 'Less brand focus']
                })

        # Option 4: Keyword-first approach (for new apps)
        if target_keywords and not include_brand:
            option4 = " ".join(target_keywords[:2])[:max_length]
            title_options.append({
                'title': option4,
                'length': len(option4),
                'remaining_chars': max_length - len(option4),
                'keywords_included': target_keywords[:2],
                'strategy': 'keyword_first',
                'pros': ['Maximum SEO benefit', 'Clear functionality'],
                'cons': ['No brand recognition', 'Generic appearance']
            })

        return {
            'platform': self.platform,
            'max_length': max_length,
            'options': title_options,
            'recommendation': self._recommend_title_option(title_options)
        }

    def optimize_description(
        self,
        app_info: Dict[str, Any],
        target_keywords: List[str],
        description_type: str = 'full'
    ) -> Dict[str, Any]:
        """
        Optimize app description with keyword integration and conversion focus.

        Args:
            app_info: Dict with 'name', 'key_features', 'unique_value', 'target_audience'
            target_keywords: List of keywords to integrate naturally
            description_type: 'full', 'short' (Google), 'subtitle' (Apple)

        Returns:
            Optimized description with analysis
        """
        if description_type == 'short' and self.platform == 'google':
            return self._optimize_short_description(app_info, target_keywords)
        elif description_type == 'subtitle' and self.platform == 'apple':
            return self._optimize_subtitle(app_info, target_keywords)
        else:
            return self._optimize_full_description(app_info, target_keywords)

    def optimize_keyword_field(
        self,
        target_keywords: List[str],
        app_title: str = "",
        app_description: str = ""
    ) -> Dict[str, Any]:
        """
        Optimize Apple's 100-character keyword field.

        Rules:
        - No spaces between commas
        - No plural forms if singular exists
        - No duplicates
        - Keywords in title/subtitle are already indexed

        Args:
            target_keywords: List of target keywords
            app_title: Current app title (to avoid duplication)
            app_description: Current description (to check coverage)

        Returns:
            Optimized keyword field (comma-separated, no spaces)
        """
        if self.platform != 'apple':
            return {'error': 'Keyword field optimization only applies to Apple App Store'}

        max_length = self.limits['keywords']

        # Extract words already in title (these don't need to be in keyword field)
        title_words = set(app_title.lower().split()) if app_title else set()

        # Process keywords
        processed_keywords = []
        for keyword in target_keywords:
            keyword_lower = keyword.lower().strip()

            # Skip if already in title
            if keyword_lower in title_words:
                continue

            # Remove duplicates and process
            words = keyword_lower.split()
            for word in words:
                if word not in processed_keywords and word not in title_words:
                    processed_keywords.append(word)

        # Remove plurals if singular exists
        deduplicated = self._remove_plural_duplicates(processed_keywords)

        # Build keyword field within 100 character limit
        keyword_field = self._build_keyword_field(deduplicated, max_length)

        # Calculate keyword density in description
        density = self._calculate_coverage(target_keywords, app_description)

        return {
            'keyword_field': keyword_field,
            'length': len(keyword_field),
            'remaining_chars': max_length - len(keyword_field),
            'keywords_included': keyword_field.split(','),
            'keywords_count': len(keyword_field.split(',')),
            'keywords_excluded': [kw for kw in target_keywords if kw.lower() not in keyword_field],
            'description_coverage': density,
            'optimization_tips': [
                'Keywords in title are auto-indexed - no need to repeat',
                'Use singular forms only (Apple indexes plurals automatically)',
                'No spaces between commas to maximize character usage',
                'Update keyword field with each app update to test variations'
            ]
        }

    def validate_character_limits(
        self,
        metadata: Dict[str, str]
    ) -> Dict[str, Any]:
        """
        Validate all metadata fields against platform character limits.

        Args:
            metadata: Dictionary of field_name: value

        Returns:
            Validation report with errors and warnings
        """
        validation_results = {
            'is_valid': True,
            'errors': [],
            'warnings': [],
            'field_status': {}
        }

        for field_name, value in metadata.items():
            if field_name not in self.limits:
                validation_results['warnings'].append(
                    f"Unknown field '{field_name}' for {self.platform} platform"
                )
                continue

            max_length = self.limits[field_name]
            actual_length = len(value)
            remaining = max_length - actual_length

            field_status = {
                'value': value,
                'length': actual_length,
                'limit': max_length,
                'remaining': remaining,
                'is_valid': actual_length <= max_length,
                'usage_percentage': round((actual_length / max_length) * 100, 1)
            }

            validation_results['field_status'][field_name] = field_status

            if actual_length > max_length:
                validation_results['is_valid'] = False
                validation_results['errors'].append(
                    f"'{field_name}' exceeds limit: {actual_length}/{max_length} chars"
                )
            elif remaining > max_length * 0.2:  # More than 20% unused
                validation_results['warnings'].append(
                    f"'{field_name}' under-utilizes space: {remaining} chars remaining"
                )

        return validation_results

    def calculate_keyword_density(
        self,
        text: str,
        target_keywords: List[str]
    ) -> Dict[str, Any]:
        """
        Calculate keyword density in text.

        Args:
            text: Text to analyze
            target_keywords: Keywords to check

        Returns:
            Density analysis
        """
        text_lower = text.lower()
        total_words = len(text_lower.split())

        keyword_densities = {}
        for keyword in target_keywords:
            keyword_lower = keyword.lower()
            count = text_lower.count(keyword_lower)
            density = (count / total_words * 100) if total_words > 0 else 0

            keyword_densities[keyword] = {
                'occurrences': count,
                'density_percentage': round(density, 2),
                'status': self._assess_density(density)
            }

        # Overall assessment
        total_keyword_occurrences = sum(kw['occurrences'] for kw in keyword_densities.values())
        overall_density = (total_keyword_occurrences / total_words * 100) if total_words > 0 else 0

        return {
            'total_words': total_words,
            'keyword_densities': keyword_densities,
            'overall_keyword_density': round(overall_density, 2),
            'assessment': self._assess_overall_density(overall_density),
            'recommendations': self._generate_density_recommendations(keyword_densities)
        }

    def _build_title_with_keywords(
        self,
        app_name: str,
        keywords: List[str],
        max_length: int
    ) -> Optional[str]:
        """Build title combining app name and keywords within limit."""
        separators = [' - ', ': ', ' | ']

        for sep in separators:
            for kw in keywords:
                title = f"{app_name}{sep}{kw}"
                if len(title) <= max_length:
                    return title

        return None

    def _optimize_short_description(
        self,
        app_info: Dict[str, Any],
        target_keywords: List[str]
    ) -> Dict[str, Any]:
        """Optimize Google Play short description (80 chars)."""
        max_length = self.limits['short_description']

        # Focus on unique value proposition with primary keyword
        unique_value = app_info.get('unique_value', '')
        primary_keyword = target_keywords[0] if target_keywords else ''

        # Template: [Primary Keyword] - [Unique Value]
        short_desc = f"{primary_keyword.title()} - {unique_value}"[:max_length]

        return {
            'short_description': short_desc,
            'length': len(short_desc),
            'remaining_chars': max_length - len(short_desc),
            'keywords_included': [primary_keyword] if primary_keyword in short_desc.lower() else [],
            'strategy': 'keyword_value_proposition'
        }

    def _optimize_subtitle(
        self,
        app_info: Dict[str, Any],
        target_keywords: List[str]
    ) -> Dict[str, Any]:
        """Optimize Apple App Store subtitle (30 chars)."""
        max_length = self.limits['subtitle']

        # Very concise - primary keyword or key feature
        primary_keyword = target_keywords[0] if target_keywords else ''
        key_feature = app_info.get('key_features', [''])[0] if app_info.get('key_features') else ''

        options = [
            primary_keyword[:max_length],
            key_feature[:max_length],
            f"{primary_keyword} App"[:max_length]
        ]

        return {
            'subtitle_options': [opt for opt in options if opt],
            'max_length': max_length,
            'recommendation': options[0] if options else ''
        }

    def _optimize_full_description(
        self,
        app_info: Dict[str, Any],
        target_keywords: List[str]
    ) -> Dict[str, Any]:
        """Optimize full app description (4000 chars for both platforms)."""
        max_length = self.limits.get('description', self.limits.get('full_description', 4000))

        # Structure: Hook → Features → Benefits → Social Proof → CTA
        sections = []

        # Hook (with primary keyword)
        primary_keyword = target_keywords[0] if target_keywords else ''
        unique_value = app_info.get('unique_value', '')
        hook = f"{unique_value} {primary_keyword.title()} that helps you achieve more.\n\n"
        sections.append(hook)

        # Features (with keywords naturally integrated)
        features = app_info.get('key_features', [])
        if features:
            sections.append("KEY FEATURES:\n")
            for i, feature in enumerate(features[:5], 1):
                # Integrate keywords naturally
                feature_text = f"• {feature}"
                if i <= len(target_keywords):
                    keyword = target_keywords[i-1]
                    if keyword.lower() not in feature.lower():
                        feature_text = f"• {feature} with {keyword}"
                sections.append(f"{feature_text}\n")
            sections.append("\n")

        # Benefits
        target_audience = app_info.get('target_audience', 'users')
        sections.append(f"PERFECT FOR:\n{target_audience}\n\n")

        # Social proof placeholder
        sections.append("WHY USERS LOVE US:\n")
        sections.append("Join thousands of satisfied users who have transformed their workflow.\n\n")

        # CTA
        sections.append("Download now and start experiencing the difference!")

        # Combine and validate length
        full_description = "".join(sections)
        if len(full_description) > max_length:
            full_description = full_description[:max_length-3] + "..."

        # Calculate keyword density
        density = self.calculate_keyword_density(full_description, target_keywords)

        return {
            'full_description': full_description,
            'length': len(full_description),
            'remaining_chars': max_length - len(full_description),
            'keyword_analysis': density,
            'structure': {
                'has_hook': True,
                'has_features': len(features) > 0,
                'has_benefits': True,
                'has_cta': True
            }
        }

    def _remove_plural_duplicates(self, keywords: List[str]) -> List[str]:
        """Remove plural forms if singular exists."""
        deduplicated = []
        singular_set = set()

        for keyword in keywords:
            if keyword.endswith('s') and len(keyword) > 1:
                singular = keyword[:-1]
                if singular not in singular_set:
                    deduplicated.append(singular)
                    singular_set.add(singular)
            else:
                if keyword not in singular_set:
                    deduplicated.append(keyword)
                    singular_set.add(keyword)

        return deduplicated

    def _build_keyword_field(self, keywords: List[str], max_length: int) -> str:
        """Build comma-separated keyword field within character limit."""
        keyword_field = ""

        for keyword in keywords:
            test_field = f"{keyword_field},{keyword}" if keyword_field else keyword
            if len(test_field) <= max_length:
                keyword_field = test_field
            else:
                break

        return keyword_field

    def _calculate_coverage(self, keywords: List[str], text: str) -> Dict[str, int]:
        """Calculate how many keywords are covered in text."""
        text_lower = text.lower()
        coverage = {}

        for keyword in keywords:
            coverage[keyword] = text_lower.count(keyword.lower())

        return coverage

    def _assess_density(self, density: float) -> str:
        """Assess individual keyword density."""
        if density < 0.5:
            return "too_low"
        elif density <= 2.5:
            return "optimal"
        else:
            return "too_high"

    def _assess_overall_density(self, density: float) -> str:
        """Assess overall keyword density."""
        if density < 2:
            return "Under-optimized: Consider adding more keyword variations"
        elif density <= 5:
            return "Optimal: Good keyword integration without stuffing"
        elif density <= 8:
            return "High: Approaching keyword stuffing - reduce keyword usage"
        else:
            return "Too High: Keyword stuffing detected - rewrite for natural flow"

    def _generate_density_recommendations(
        self,
        keyword_densities: Dict[str, Dict[str, Any]]
    ) -> List[str]:
        """Generate recommendations based on keyword density analysis."""
        recommendations = []

        for keyword, data in keyword_densities.items():
            if data['status'] == 'too_low':
                recommendations.append(
                    f"Increase usage of '{keyword}' - currently only {data['occurrences']} times"
                )
            elif data['status'] == 'too_high':
                recommendations.append(
                    f"Reduce usage of '{keyword}' - appears {data['occurrences']} times (keyword stuffing risk)"
                )

        if not recommendations:
            recommendations.append("Keyword density is well-balanced")

        return recommendations

    def _recommend_title_option(self, options: List[Dict[str, Any]]) -> str:
        """Recommend best title option based on strategy."""
        if not options:
            return "No valid options available"

        # Prefer brand_plus_primary for established apps
        for option in options:
            if option['strategy'] == 'brand_plus_primary':
                return f"Recommended: '{option['title']}' (Balance of brand and SEO)"

        # Fallback to first option
        return f"Recommended: '{options[0]['title']}' ({options[0]['strategy']})"


def optimize_app_metadata(
    platform: str,
    app_info: Dict[str, Any],
    target_keywords: List[str]
) -> Dict[str, Any]:
    """
    Convenience function to optimize all metadata fields.

    Args:
        platform: 'apple' or 'google'
        app_info: App information dictionary
        target_keywords: Target keywords list

    Returns:
        Complete metadata optimization package
    """
    optimizer = MetadataOptimizer(platform)

    return {
        'platform': platform,
        'title': optimizer.optimize_title(
            app_info['name'],
            target_keywords
        ),
        'description': optimizer.optimize_description(
            app_info,
            target_keywords,
            'full'
        ),
        'keyword_field': optimizer.optimize_keyword_field(
            target_keywords
        ) if platform == 'apple' else None
    }

FILE:scripts/review_analyzer.py
"""
Review analysis module for App Store Optimization.
Analyzes user reviews for sentiment, issues, and feature requests.
"""

from typing import Dict, List, Any, Optional, Tuple
from collections import Counter
import re


class ReviewAnalyzer:
    """Analyzes user reviews for actionable insights."""

    # Sentiment keywords
    POSITIVE_KEYWORDS = [
        'great', 'awesome', 'excellent', 'amazing', 'love', 'best', 'perfect',
        'fantastic', 'wonderful', 'brilliant', 'outstanding', 'superb'
    ]

    NEGATIVE_KEYWORDS = [
        'bad', 'terrible', 'awful', 'horrible', 'hate', 'worst', 'useless',
        'broken', 'crash', 'bug', 'slow', 'disappointing', 'frustrating'
    ]

    # Issue indicators
    ISSUE_KEYWORDS = [
        'crash', 'bug', 'error', 'broken', 'not working', 'doesnt work',
        'freezes', 'slow', 'laggy', 'glitch', 'problem', 'issue', 'fail'
    ]

    # Feature request indicators
    FEATURE_REQUEST_KEYWORDS = [
        'wish', 'would be nice', 'should add', 'need', 'want', 'hope',
        'please add', 'missing', 'lacks', 'feature request'
    ]

    def __init__(self, app_name: str):
        """
        Initialize review analyzer.

        Args:
            app_name: Name of the app
        """
        self.app_name = app_name
        self.reviews = []
        self.analysis_cache = {}

    def analyze_sentiment(
        self,
        reviews: List[Dict[str, Any]]
    ) -> Dict[str, Any]:
        """
        Analyze sentiment across reviews.

        Args:
            reviews: List of review dicts with 'text', 'rating', 'date'

        Returns:
            Sentiment analysis summary
        """
        self.reviews = reviews

        sentiment_counts = {
            'positive': 0,
            'neutral': 0,
            'negative': 0
        }

        detailed_sentiments = []

        for review in reviews:
            text = review.get('text', '').lower()
            rating = review.get('rating', 3)

            # Calculate sentiment score
            sentiment_score = self._calculate_sentiment_score(text, rating)
            sentiment_category = self._categorize_sentiment(sentiment_score)

            sentiment_counts[sentiment_category] += 1

            detailed_sentiments.append({
                'review_id': review.get('id', ''),
                'rating': rating,
                'sentiment_score': sentiment_score,
                'sentiment': sentiment_category,
                'text_preview': text[:100] + '...' if len(text) > 100 else text
            })

        # Calculate percentages
        total = len(reviews)
        sentiment_distribution = {
            'positive': round((sentiment_counts['positive'] / total) * 100, 1) if total > 0 else 0,
            'neutral': round((sentiment_counts['neutral'] / total) * 100, 1) if total > 0 else 0,
            'negative': round((sentiment_counts['negative'] / total) * 100, 1) if total > 0 else 0
        }

        # Calculate average rating
        avg_rating = sum(r.get('rating', 0) for r in reviews) / total if total > 0 else 0

        return {
            'total_reviews_analyzed': total,
            'average_rating': round(avg_rating, 2),
            'sentiment_distribution': sentiment_distribution,
            'sentiment_counts': sentiment_counts,
            'sentiment_trend': self._assess_sentiment_trend(sentiment_distribution),
            'detailed_sentiments': detailed_sentiments[:50]  # Limit output
        }

    def extract_common_themes(
        self,
        reviews: List[Dict[str, Any]],
        min_mentions: int = 3
    ) -> Dict[str, Any]:
        """
        Extract frequently mentioned themes and topics.

        Args:
            reviews: List of review dicts
            min_mentions: Minimum mentions to be considered common

        Returns:
            Common themes analysis
        """
        # Extract all words from reviews
        all_words = []
        all_phrases = []

        for review in reviews:
            text = review.get('text', '').lower()
            # Clean text
            text = re.sub(r'[^\w\s]', ' ', text)
            words = text.split()

            # Filter out common words
            stop_words = {
                'the', 'and', 'for', 'with', 'this', 'that', 'from', 'have',
                'app', 'apps', 'very', 'really', 'just', 'but', 'not', 'you'
            }
            words = [w for w in words if w not in stop_words and len(w) > 3]

            all_words.extend(words)

            # Extract 2-3 word phrases
            for i in range(len(words) - 1):
                phrase = f"{words[i]} {words[i+1]}"
                all_phrases.append(phrase)

        # Count frequency
        word_freq = Counter(all_words)
        phrase_freq = Counter(all_phrases)

        # Filter by min_mentions
        common_words = [
            {'word': word, 'mentions': count}
            for word, count in word_freq.most_common(30)
            if count >= min_mentions
        ]

        common_phrases = [
            {'phrase': phrase, 'mentions': count}
            for phrase, count in phrase_freq.most_common(20)
            if count >= min_mentions
        ]

        # Categorize themes
        themes = self._categorize_themes(common_words, common_phrases)

        return {
            'common_words': common_words,
            'common_phrases': common_phrases,
            'identified_themes': themes,
            'insights': self._generate_theme_insights(themes)
        }

    def identify_issues(
        self,
        reviews: List[Dict[str, Any]],
        rating_threshold: int = 3
    ) -> Dict[str, Any]:
        """
        Identify bugs, crashes, and other issues from reviews.

        Args:
            reviews: List of review dicts
            rating_threshold: Only analyze reviews at or below this rating

        Returns:
            Issue identification report
        """
        issues = []

        for review in reviews:
            rating = review.get('rating', 5)
            if rating > rating_threshold:
                continue

            text = review.get('text', '').lower()

            # Check for issue keywords
            mentioned_issues = []
            for keyword in self.ISSUE_KEYWORDS:
                if keyword in text:
                    mentioned_issues.append(keyword)

            if mentioned_issues:
                issues.append({
                    'review_id': review.get('id', ''),
                    'rating': rating,
                    'date': review.get('date', ''),
                    'issue_keywords': mentioned_issues,
                    'text': text[:200] + '...' if len(text) > 200 else text
                })

        # Group by issue type
        issue_frequency = Counter()
        for issue in issues:
            for keyword in issue['issue_keywords']:
                issue_frequency[keyword] += 1

        # Categorize issues
        categorized_issues = self._categorize_issues(issues)

        # Calculate issue severity
        severity_scores = self._calculate_issue_severity(
            categorized_issues,
            len(reviews)
        )

        return {
            'total_issues_found': len(issues),
            'issue_frequency': dict(issue_frequency.most_common(15)),
            'categorized_issues': categorized_issues,
            'severity_scores': severity_scores,
            'top_issues': self._rank_issues_by_severity(severity_scores),
            'recommendations': self._generate_issue_recommendations(
                categorized_issues,
                severity_scores
            )
        }

    def find_feature_requests(
        self,
        reviews: List[Dict[str, Any]]
    ) -> Dict[str, Any]:
        """
        Extract feature requests and desired improvements.

        Args:
            reviews: List of review dicts

        Returns:
            Feature request analysis
        """
        feature_requests = []

        for review in reviews:
            text = review.get('text', '').lower()
            rating = review.get('rating', 3)

            # Check for feature request indicators
            is_feature_request = any(
                keyword in text
                for keyword in self.FEATURE_REQUEST_KEYWORDS
            )

            if is_feature_request:
                # Extract the specific request
                request_text = self._extract_feature_request_text(text)

                feature_requests.append({
                    'review_id': review.get('id', ''),
                    'rating': rating,
                    'date': review.get('date', ''),
                    'request_text': request_text,
                    'full_review': text[:200] + '...' if len(text) > 200 else text
                })

        # Cluster similar requests
        clustered_requests = self._cluster_feature_requests(feature_requests)

        # Prioritize based on frequency and rating context
        prioritized_requests = self._prioritize_feature_requests(clustered_requests)

        return {
            'total_feature_requests': len(feature_requests),
            'clustered_requests': clustered_requests,
            'prioritized_requests': prioritized_requests,
            'implementation_recommendations': self._generate_feature_recommendations(
                prioritized_requests
            )
        }

    def track_sentiment_trends(
        self,
        reviews_by_period: Dict[str, List[Dict[str, Any]]]
    ) -> Dict[str, Any]:
        """
        Track sentiment changes over time.

        Args:
            reviews_by_period: Dict of period_name: reviews

        Returns:
            Trend analysis
        """
        trends = []

        for period, reviews in reviews_by_period.items():
            sentiment = self.analyze_sentiment(reviews)

            trends.append({
                'period': period,
                'total_reviews': len(reviews),
                'average_rating': sentiment['average_rating'],
                'positive_percentage': sentiment['sentiment_distribution']['positive'],
                'negative_percentage': sentiment['sentiment_distribution']['negative']
            })

        # Calculate trend direction
        if len(trends) >= 2:
            first_period = trends[0]
            last_period = trends[-1]

            rating_change = last_period['average_rating'] - first_period['average_rating']
            sentiment_change = last_period['positive_percentage'] - first_period['positive_percentage']

            trend_direction = self._determine_trend_direction(
                rating_change,
                sentiment_change
            )
        else:
            trend_direction = 'insufficient_data'

        return {
            'periods_analyzed': len(trends),
            'trend_data': trends,
            'trend_direction': trend_direction,
            'insights': self._generate_trend_insights(trends, trend_direction)
        }

    def generate_response_templates(
        self,
        issue_category: str
    ) -> List[Dict[str, str]]:
        """
        Generate response templates for common review scenarios.

        Args:
            issue_category: Category of issue ('crash', 'feature_request', 'positive', etc.)

        Returns:
            Response templates
        """
        templates = {
            'crash': [
                {
                    'scenario': 'App crash reported',
                    'template': "Thank you for bringing this to our attention. We're sorry you experienced a crash. "
                               "Our team is investigating this issue. Could you please share more details about when "
                               "this occurred (device model, iOS/Android version) by contacting support@[company].com? "
                               "We're committed to fixing this quickly."
                },
                {
                    'scenario': 'Crash already fixed',
                    'template': "Thank you for your feedback. We've identified and fixed this crash issue in version [X.X]. "
                               "Please update to the latest version. If the problem persists, please reach out to "
                               "support@[company].com and we'll help you directly."
                }
            ],
            'bug': [
                {
                    'scenario': 'Bug reported',
                    'template': "Thanks for reporting this bug. We take these issues seriously. Our team is looking into it "
                               "and we'll have a fix in an upcoming update. We appreciate your patience and will notify you "
                               "when it's resolved."
                }
            ],
            'feature_request': [
                {
                    'scenario': 'Feature request received',
                    'template': "Thank you for this suggestion! We're always looking to improve [app_name]. We've added your "
                               "request to our roadmap and will consider it for a future update. Follow us @[social] for "
                               "updates on new features."
                },
                {
                    'scenario': 'Feature already planned',
                    'template': "Great news! This feature is already on our roadmap and we're working on it. Stay tuned for "
                               "updates in the coming months. Thanks for your feedback!"
                }
            ],
            'positive': [
                {
                    'scenario': 'Positive review',
                    'template': "Thank you so much for your kind words! We're thrilled that you're enjoying [app_name]. "
                               "Reviews like yours motivate our team to keep improving. If you ever have suggestions, "
                               "we'd love to hear them!"
                }
            ],
            'negative_general': [
                {
                    'scenario': 'General complaint',
                    'template': "We're sorry to hear you're not satisfied with your experience. We'd like to make this right. "
                               "Please contact us at support@[company].com so we can understand the issue better and help "
                               "you directly. Thank you for giving us a chance to improve."
                }
            ]
        }

        return templates.get(issue_category, templates['negative_general'])

    def _calculate_sentiment_score(self, text: str, rating: int) -> float:
        """Calculate sentiment score (-1 to 1)."""
        # Start with rating-based score
        rating_score = (rating - 3) / 2  # Convert 1-5 to -1 to 1

        # Adjust based on text sentiment
        positive_count = sum(1 for keyword in self.POSITIVE_KEYWORDS if keyword in text)
        negative_count = sum(1 for keyword in self.NEGATIVE_KEYWORDS if keyword in text)

        text_score = (positive_count - negative_count) / 10  # Normalize

        # Weighted average (60% rating, 40% text)
        final_score = (rating_score * 0.6) + (text_score * 0.4)

        return max(min(final_score, 1.0), -1.0)

    def _categorize_sentiment(self, score: float) -> str:
        """Categorize sentiment score."""
        if score > 0.3:
            return 'positive'
        elif score < -0.3:
            return 'negative'
        else:
            return 'neutral'

    def _assess_sentiment_trend(self, distribution: Dict[str, float]) -> str:
        """Assess overall sentiment trend."""
        positive = distribution['positive']
        negative = distribution['negative']

        if positive > 70:
            return 'very_positive'
        elif positive > 50:
            return 'positive'
        elif negative > 30:
            return 'concerning'
        elif negative > 50:
            return 'critical'
        else:
            return 'mixed'

    def _categorize_themes(
        self,
        common_words: List[Dict[str, Any]],
        common_phrases: List[Dict[str, Any]]
    ) -> Dict[str, List[str]]:
        """Categorize themes from words and phrases."""
        themes = {
            'features': [],
            'performance': [],
            'usability': [],
            'support': [],
            'pricing': []
        }

        # Keywords for each category
        feature_keywords = {'feature', 'functionality', 'option', 'tool'}
        performance_keywords = {'fast', 'slow', 'crash', 'lag', 'speed', 'performance'}
        usability_keywords = {'easy', 'difficult', 'intuitive', 'confusing', 'interface', 'design'}
        support_keywords = {'support', 'help', 'customer', 'service', 'response'}
        pricing_keywords = {'price', 'cost', 'expensive', 'cheap', 'subscription', 'free'}

        for word_data in common_words:
            word = word_data['word']
            if any(kw in word for kw in feature_keywords):
                themes['features'].append(word)
            elif any(kw in word for kw in performance_keywords):
                themes['performance'].append(word)
            elif any(kw in word for kw in usability_keywords):
                themes['usability'].append(word)
            elif any(kw in word for kw in support_keywords):
                themes['support'].append(word)
            elif any(kw in word for kw in pricing_keywords):
                themes['pricing'].append(word)

        return {k: v for k, v in themes.items() if v}  # Remove empty categories

    def _generate_theme_insights(self, themes: Dict[str, List[str]]) -> List[str]:
        """Generate insights from themes."""
        insights = []

        for category, keywords in themes.items():
            if keywords:
                insights.append(
                    f"{category.title()}: Users frequently mention {', '.join(keywords[:3])}"
                )

        return insights[:5]

    def _categorize_issues(self, issues: List[Dict[str, Any]]) -> Dict[str, List[Dict[str, Any]]]:
        """Categorize issues by type."""
        categories = {
            'crashes': [],
            'bugs': [],
            'performance': [],
            'compatibility': []
        }

        for issue in issues:
            keywords = issue['issue_keywords']

            if 'crash' in keywords or 'freezes' in keywords:
                categories['crashes'].append(issue)
            elif 'bug' in keywords or 'error' in keywords or 'broken' in keywords:
                categories['bugs'].append(issue)
            elif 'slow' in keywords or 'laggy' in keywords:
                categories['performance'].append(issue)
            else:
                categories['compatibility'].append(issue)

        return {k: v for k, v in categories.items() if v}

    def _calculate_issue_severity(
        self,
        categorized_issues: Dict[str, List[Dict[str, Any]]],
        total_reviews: int
    ) -> Dict[str, Dict[str, Any]]:
        """Calculate severity scores for each issue category."""
        severity_scores = {}

        for category, issues in categorized_issues.items():
            count = len(issues)
            percentage = (count / total_reviews) * 100 if total_reviews > 0 else 0

            # Calculate average rating of affected reviews
            avg_rating = sum(i['rating'] for i in issues) / count if count > 0 else 0

            # Severity score (0-100)
            severity = min((percentage * 10) + ((5 - avg_rating) * 10), 100)

            severity_scores[category] = {
                'count': count,
                'percentage': round(percentage, 2),
                'average_rating': round(avg_rating, 2),
                'severity_score': round(severity, 1),
                'priority': 'critical' if severity > 70 else ('high' if severity > 40 else 'medium')
            }

        return severity_scores

    def _rank_issues_by_severity(
        self,
        severity_scores: Dict[str, Dict[str, Any]]
    ) -> List[Dict[str, Any]]:
        """Rank issues by severity score."""
        ranked = sorted(
            [{'category': cat, **data} for cat, data in severity_scores.items()],
            key=lambda x: x['severity_score'],
            reverse=True
        )
        return ranked

    def _generate_issue_recommendations(
        self,
        categorized_issues: Dict[str, List[Dict[str, Any]]],
        severity_scores: Dict[str, Dict[str, Any]]
    ) -> List[str]:
        """Generate recommendations for addressing issues."""
        recommendations = []

        for category, score_data in severity_scores.items():
            if score_data['priority'] == 'critical':
                recommendations.append(
                    f"URGENT: Address {category} issues immediately - affecting {score_data['percentage']}% of reviews"
                )
            elif score_data['priority'] == 'high':
                recommendations.append(
                    f"HIGH PRIORITY: Focus on {category} issues in next update"
                )

        return recommendations

    def _extract_feature_request_text(self, text: str) -> str:
        """Extract the specific feature request from review text."""
        # Simple extraction - find sentence with feature request keywords
        sentences = text.split('.')
        for sentence in sentences:
            if any(keyword in sentence for keyword in self.FEATURE_REQUEST_KEYWORDS):
                return sentence.strip()
        return text[:100]  # Fallback

    def _cluster_feature_requests(
        self,
        feature_requests: List[Dict[str, Any]]
    ) -> List[Dict[str, Any]]:
        """Cluster similar feature requests."""
        # Simplified clustering - group by common keywords
        clusters = {}

        for request in feature_requests:
            text = request['request_text'].lower()
            # Extract key words
            words = [w for w in text.split() if len(w) > 4]

            # Try to find matching cluster
            matched = False
            for cluster_key in clusters:
                if any(word in cluster_key for word in words[:3]):
                    clusters[cluster_key].append(request)
                    matched = True
                    break

            if not matched and words:
                cluster_key = ' '.join(words[:2])
                clusters[cluster_key] = [request]

        return [
            {'feature_theme': theme, 'request_count': len(requests), 'examples': requests[:3]}
            for theme, requests in clusters.items()
        ]

    def _prioritize_feature_requests(
        self,
        clustered_requests: List[Dict[str, Any]]
    ) -> List[Dict[str, Any]]:
        """Prioritize feature requests by frequency."""
        return sorted(
            clustered_requests,
            key=lambda x: x['request_count'],
            reverse=True
        )[:10]

    def _generate_feature_recommendations(
        self,
        prioritized_requests: List[Dict[str, Any]]
    ) -> List[str]:
        """Generate recommendations for feature requests."""
        recommendations = []

        if prioritized_requests:
            top_request = prioritized_requests[0]
            recommendations.append(
                f"Most requested feature: {top_request['feature_theme']} "
                f"({top_request['request_count']} mentions) - consider for next major release"
            )

        if len(prioritized_requests) > 1:
            recommendations.append(
                f"Also consider: {prioritized_requests[1]['feature_theme']}"
            )

        return recommendations

    def _determine_trend_direction(
        self,
        rating_change: float,
        sentiment_change: float
    ) -> str:
        """Determine overall trend direction."""
        if rating_change > 0.2 and sentiment_change > 5:
            return 'improving'
        elif rating_change < -0.2 and sentiment_change < -5:
            return 'declining'
        else:
            return 'stable'

    def _generate_trend_insights(
        self,
        trends: List[Dict[str, Any]],
        trend_direction: str
    ) -> List[str]:
        """Generate insights from trend analysis."""
        insights = []

        if trend_direction == 'improving':
            insights.append("Positive trend: User satisfaction is increasing over time")
        elif trend_direction == 'declining':
            insights.append("WARNING: User satisfaction is declining - immediate action needed")
        else:
            insights.append("Sentiment is stable - maintain current quality")

        # Review velocity insight
        if len(trends) >= 2:
            recent_reviews = trends[-1]['total_reviews']
            previous_reviews = trends[-2]['total_reviews']

            if recent_reviews > previous_reviews * 1.5:
                insights.append("Review volume increasing - growing user base or recent controversy")

        return insights


def analyze_reviews(
    app_name: str,
    reviews: List[Dict[str, Any]]
) -> Dict[str, Any]:
    """
    Convenience function to perform comprehensive review analysis.

    Args:
        app_name: App name
        reviews: List of review dictionaries

    Returns:
        Complete review analysis
    """
    analyzer = ReviewAnalyzer(app_name)

    return {
        'sentiment_analysis': analyzer.analyze_sentiment(reviews),
        'common_themes': analyzer.extract_common_themes(reviews),
        'issues_identified': analyzer.identify_issues(reviews),
        'feature_requests': analyzer.find_feature_requests(reviews)
    }

---
name: "atlassian-templates"
description: Atlassian Template and Files Creator/Modifier expert for creating, modifying, and managing Jira and Confluence templates, blueprints, custom layouts, reusable components, and standardized content structures. Use when building org-wide templates, custom blueprints, page layouts, and automated content generation.
---

# Atlassian Template & Files Creator Expert

Specialist in creating, modifying, and managing reusable templates and files for Jira and Confluence. Ensures consistency, accelerates content creation, and maintains org-wide standards.

---

## Workflows

### Template Creation Process
1. **Discover**: Interview stakeholders to understand needs
2. **Analyze**: Review existing content patterns
3. **Design**: Create template structure and placeholders
4. **Implement**: Build template with macros and formatting
5. **Test**: Validate with sample data — confirm template renders correctly in preview before publishing
6. **Document**: Create usage instructions
7. **Publish**: Deploy to appropriate space/project via MCP (see MCP Operations below)
8. **Verify**: Confirm deployment success; roll back to previous version if errors occur
9. **Train**: Educate users on template usage
10. **Monitor**: Track adoption and gather feedback
11. **Iterate**: Refine based on usage

### Template Modification Process
1. **Assess**: Review change request and impact
2. **Version**: Create new version, keep old available
3. **Modify**: Update template structure/content
4. **Test**: Validate changes don't break existing usage; preview updated template before publishing
5. **Migrate**: Provide migration path for existing content
6. **Communicate**: Announce changes to users
7. **Support**: Assist users with migration
8. **Archive**: Deprecate old version after transition; confirm deprecated template is unlisted, not deleted

### Blueprint Development
1. Define blueprint scope and purpose
2. Design multi-page structure
3. Create page templates for each section
4. Configure page creation rules
5. Add dynamic content (Jira queries, user data)
6. Test blueprint creation flow end-to-end with a sample space
7. Verify all macro references resolve correctly before deployment
8. **HANDOFF TO**: Atlassian Admin for global deployment

---

## Confluence Templates Library

See **TEMPLATES.md** for full reference tables and copy-paste-ready template structures. The following summarises the standard types this skill creates and maintains.

### Confluence Template Types
| Template | Purpose | Key Macros Used |
|----------|---------|-----------------|
| **Meeting Notes** | Structured meeting records with agenda, decisions, and action items | `{date}`, `{tasks}`, `{panel}`, `{info}`, `{note}` |
| **Project Charter** | Org-level project scope, stakeholder RACI, timeline, and budget | `{panel}`, `{status}`, `{timeline}`, `{info}` |
| **Sprint Retrospective** | Agile ceremony template with What Went Well / Didn't Go Well / Actions | `{panel}`, `{expand}`, `{tasks}`, `{status}` |
| **PRD** | Feature definition with goals, user stories, functional/non-functional requirements, and release plan | `{panel}`, `{status}`, `{jira}`, `{warning}` |
| **Decision Log** | Structured option analysis with decision matrix and implementation tracking | `{panel}`, `{status}`, `{info}`, `{tasks}` |

**Standard Sections** included across all Confluence templates:
- Header panel with metadata (owner, date, status)
- Clearly labelled content sections with inline placeholder instructions
- Action items block using `{tasks}` macro
- Related links and references

### Complete Example: Meeting Notes Template

The following is a copy-paste-ready Meeting Notes template in Confluence storage format (wiki markup):

```
{panel:title=Meeting Metadata|borderColor=#0052CC|titleBGColor=#0052CC|titleColor=#FFFFFF}
*Date:* {date}
*Owner / Facilitator:* @[facilitator name]
*Attendees:* @[name], @[name]
*Status:* {status:colour=Yellow|title=In Progress}
{panel}

h2. Agenda
# [Agenda item 1]
# [Agenda item 2]
# [Agenda item 3]

h2. Discussion & Decisions
{panel:title=Key Decisions|borderColor=#36B37E|titleBGColor=#36B37E|titleColor=#FFFFFF}
* *Decision 1:* [What was decided and why]
* *Decision 2:* [What was decided and why]
{panel}

{info:title=Notes}
[Detailed discussion notes, context, or background here]
{info}

h2. Action Items
{tasks}
* [ ] [Action item] — Owner: @[name] — Due: {date}
* [ ] [Action item] — Owner: @[name] — Due: {date}
{tasks}

h2. Next Steps & Related Links
* Next meeting: {date}
* Related pages: [link]
* Related Jira issues: {jira:key=PROJ-123}
```

> Full examples for all other template types (Project Charter, Sprint Retrospective, PRD, Decision Log) and all Jira templates can be generated on request or found in **TEMPLATES.md**.

---

## Jira Templates Library

### Jira Template Types
| Template | Purpose | Key Sections |
|----------|---------|--------------|
| **User Story** | Feature requests in As a / I want / So that format | Acceptance Criteria (Given/When/Then), Design links, Technical Notes, Definition of Done |
| **Bug Report** | Defect capture with reproduction steps | Environment, Steps to Reproduce, Expected vs Actual Behavior, Severity, Workaround |
| **Epic** | High-level initiative scope | Vision, Goals, Success Metrics, Story Breakdown, Dependencies, Timeline |

**Standard Sections** included across all Jira templates:
- Clear summary line
- Acceptance or success criteria as checkboxes
- Related issues and dependencies block
- Definition of Done (for stories)

---

## Macro Usage Guidelines

**Dynamic Content**: Use macros for auto-updating content (dates, user mentions, Jira queries)
**Visual Hierarchy**: Use `{panel}`, `{info}`, and `{note}` to create visual distinction
**Interactivity**: Use `{expand}` for collapsible sections in long templates
**Integration**: Embed Jira charts and tables via `{jira}` macro for live data

---

## Atlassian MCP Integration

**Primary Tools**: Confluence MCP, Jira MCP

### Template Operations via MCP

All MCP calls below use the exact parameter names expected by the Atlassian MCP server. Replace angle-bracket placeholders with real values before executing.

**Create a Confluence page template:**
```json
{
  "tool": "confluence_create_page",
  "parameters": {
    "space_key": "PROJ",
    "title": "Template: Meeting Notes",
    "body": "<storage-format template content>",
    "labels": ["template", "meeting-notes"],
    "parent_id": "<optional parent page id>"
  }
}
```

**Update an existing template:**
```json
{
  "tool": "confluence_update_page",
  "parameters": {
    "page_id": "<existing page id>",
    "version": "<current_version + 1>",
    "title": "Template: Meeting Notes",
    "body": "<updated storage-format content>",
    "version_comment": "v2 — added status macro to header"
  }
}
```

**Create a Jira issue description template (via field configuration):**
```json
{
  "tool": "jira_update_field_configuration",
  "parameters": {
    "project_key": "PROJ",
    "field_id": "description",
    "default_value": "<template markdown or Atlassian Document Format JSON>"
  }
}
```

**Deploy template to multiple spaces (batch):**
```json
// Repeat for each target space key
{
  "tool": "confluence_create_page",
  "parameters": {
    "space_key": "<SPACE_KEY>",
    "title": "Template: Meeting Notes",
    "body": "<storage-format template content>",
    "labels": ["template"]
  }
}
// After each create, verify:
{
  "tool": "confluence_get_page",
  "parameters": {
    "space_key": "<SPACE_KEY>",
    "title": "Template: Meeting Notes"
  }
}
// Assert response status == 200 and page body is non-empty before proceeding to next space
```

**Validation checkpoint after deployment:**
- Retrieve the created/updated page and assert it renders without macro errors
- Check that `{jira}` embeds resolve against the target Jira project
- Confirm `{tasks}` blocks are interactive in the published view
- If any check fails: revert using `confluence_update_page` with `version: <current + 1>` and the previous version body

---

## Best Practices & Governance

**Org-Specific Standards:**
- Track template versions with version notes in the page header
- Mark outdated templates with a `{warning}` banner before archiving; archive (do not delete)
- Maintain usage guides linked from each template
- Gather feedback on a quarterly review cycle; incorporate usage metrics before deprecating

**Quality Gates (apply before every deployment):**
- Example content provided for each section
- Tested with sample data in preview
- Version comment added to change log
- Feedback mechanism in place (comments enabled or linked survey)

**Governance Process**:
1. Request and justification
2. Design and review
3. Testing with pilot users
4. Documentation
5. Approval
6. Deployment (via MCP or manual)
7. Training
8. Monitoring

---

## Handoff Protocols

See **HANDOFFS.md** for the full handoff matrix. Summary:

| Partner | Receives FROM | Sends TO |
|---------|--------------|---------|
| **Senior PM** | Template requirements, reporting templates, executive formats | Completed templates, usage analytics, optimization suggestions |
| **Scrum Master** | Sprint ceremony needs, team-specific requests, retro format preferences | Sprint-ready templates, agile ceremony structures, velocity tracking templates |
| **Jira Expert** | Issue template requirements, custom field display needs | Issue description templates, field config templates, JQL query templates |
| **Confluence Expert** | Space-specific needs, global template requests, blueprint requirements | Configured page templates, blueprint structures, deployment plans |
| **Atlassian Admin** | Org-wide standards, global deployment requirements, compliance templates | Global templates for approval, usage reports, compliance status |

FILE:references/governance-framework.md
# Template Governance Framework

## Overview

Operational framework for managing template lifecycle — creation, updates, deprecation, and quality enforcement with concrete thresholds and decision criteria.

## Ownership Model

### Roles

**Template Owner** (1 per template):
- Reviews usage dashboard on the 1st of each quarter
- Archives templates with <5 uses in the past 90 days
- Responds to change requests within 14 calendar days
- Runs quarterly accuracy check: open 3 random pages created from the template, verify content matches current process
- Escalates to committee if change affects >50 users

**Template Steward** (1 per team/domain):
- Runs monthly usage pull: `CQL: type = page AND label = "template-{name}" AND created >= "now-30d"`
- Flags templates where >30% of users delete or heavily modify a section (indicates friction)
- Collects and triages feedback — tags Jira tickets with `template-change` and links to template page

**Template Committee** (3-5 people, for orgs with 20+ templates):
- Meets quarterly (45 min max): reviews new proposals, resolves conflicts, flags duplicates
- Decision rule: approve if template serves >1 team and doesn't duplicate existing template by >60% content overlap
- Publishes quarterly "Template Health" Confluence page with adoption rates and actions taken

### Assignment Matrix

| Template Category | Owner Role | Steward Role |
|------------------|-----------|-------------|
| Engineering templates | Engineering Manager | Senior Engineer |
| Product templates | Head of Product | Senior PM |
| Meeting templates | Operations Lead | EA/Admin |
| Project templates | PMO Lead | Senior PM |
| HR/People templates | HR Director | HR Coordinator |
| Company-wide templates | Operations Lead | Template Committee |

## Approval Workflow

### New Template Proposal

1. **Request** - Submitter creates proposal with:
   - Template name and purpose
   - Target audience
   - Justification (why existing templates are insufficient)
   - Draft content
   - Proposed owner

2. **Review** - Template owner/committee evaluates:
   - Does this duplicate an existing template?
   - Is the scope appropriate (not too broad or narrow)?
   - Does it follow design standards?
   - Is it needed by more than one team?

3. **Pilot** - If approved:
   - Deploy to a small group for 2-4 weeks
   - Collect feedback on usability and completeness
   - Iterate based on feedback

4. **Launch** - After successful pilot:
   - Publish to template library
   - Announce to relevant teams
   - Add to Template Index page
   - Train users if needed

5. **Monitor** - Post-launch:
   - Track adoption rate (first 30 days)
   - Collect initial feedback
   - Make quick adjustments if needed

### Template Update Process

1. **Change Request** - Anyone can suggest changes via:
   - Comment on the template page
   - Jira ticket tagged `template-change`
   - Direct message to template owner

2. **Assessment** - Owner evaluates:
   - Impact on existing documents using this template
   - Alignment with organizational standards
   - Effort required for update

3. **Implementation** - Owner or steward:
   - Makes changes in a draft version
   - Reviews with 1-2 frequent users
   - Updates version number and changelog
   - Publishes updated template

4. **Communication** - Notify users:
   - Post update in relevant Slack/Teams channel
   - Update Template Index page
   - Send email for major changes

## Change Management

### Impact Categories

**Low Impact (Owner decides):**
- Typo fixes and formatting improvements
- Clarifying existing instructions
- Adding optional sections

**Medium Impact (Owner + 1 reviewer):**
- Adding new required sections
- Changing variable names or structure
- Updating macro usage

**High Impact (Committee review):**
- Removing sections from widely-used templates
- Merging or splitting templates
- Changing organizational template standards

### Communication Plan

| Change Type | Communication | Timeline |
|------------|---------------|----------|
| Low impact | Changelog update | Same day |
| Medium impact | Team channel announcement | 1 week notice |
| High impact | Email + meeting + migration guide | 2-4 weeks notice |

## Deprecation Process

### When to Deprecate
- Template replaced by a better alternative
- Process the template supports has been retired
- Template has zero usage in the past 6 months
- Template is redundant with another active template

### Deprecation Steps

1. **Decision** - Owner proposes deprecation with justification
2. **Announcement** - Notify users 30 days before deprecation:
   - Mark template with "DEPRECATED" status
   - Add deprecation notice at top of template
   - Point to replacement template (if applicable)
3. **Transition Period** - 30 days:
   - Template still available but marked deprecated
   - New documents should use replacement
   - Existing documents do not need to change
4. **Archive** - After transition:
   - Move template to Archive section
   - Remove from active template list
   - Keep accessible for historical reference
5. **Review** - 90 days after archive:
   - Confirm no active usage
   - Add to annual cleanup list if truly unused

## Usage Tracking

### Metrics & Thresholds

| Metric | Healthy | Flagged | Deprecate |
|--------|---------|---------|-----------|
| Pages created/month | >10 | 3-10 | <3 for 2 consecutive quarters |
| Unique users/month | >5 | 2-5 | 0-1 for 90 days |
| Section deletion rate | <10% | 10-30% | >30% (users removing sections = template mismatch) |
| Time to first use (new templates) | <7 days | 7-30 days | >30 days (failed launch, re-announce or rethink) |

### Tracking via CQL

```
-- Monthly usage for a specific template
type = page AND label = "template-sprint-retro" AND created >= "now-30d"

-- Stale templates (no usage in 90 days)
type = page AND label IN ("template-sprint-retro", "template-decision-log") AND created < "now-90d" AND created >= "now-91d"

-- All template-created pages for audit
type = page AND label = "template-*" ORDER BY created DESC
```

### Reporting
- **Monthly:** Top 10 templates by usage → auto-generated Confluence table
- **Quarterly:** Flag templates below thresholds → owner action required within 14 days
- **Annually:** Full catalog review — archive anything with 0 uses in 6 months

## Quality Standards

### Content Checklist (pass/fail per template)
- [ ] Every section has placeholder text showing expected content (not just a heading)
- [ ] No section references a process, tool, or team that no longer exists
- [ ] All Jira macro JQL filters return results (test quarterly)
- [ ] Links to other Confluence pages resolve (no 404s)
- [ ] Template renders correctly in both desktop and mobile preview

**FAIL examples:** Template says "Update the JIRA board" but team uses Linear. Template has a "QA Sign-Off" section but team has no QA role. Placeholder text says "TODO: add content here" with no guidance on what content.

### Structural Checklist
- [ ] Metadata header: owner name, version (semver), status (`active`/`deprecated`/`draft`), last-reviewed date
- [ ] Table of Contents macro if template has 4+ sections
- [ ] Change History table at bottom (date, author, change description)
- [ ] Placeholder text uses `{placeholder}` syntax or ac:placeholder macro — visually distinct from real content

### Maintenance Triggers
- Template not reviewed in 90+ days → owner gets Jira ticket auto-created via automation
- 3+ unresolved feedback items → escalate to committee
- Broken macro detected → owner notified same day via Slack/email automation

## Review Cadence

### Quarterly Review (Template Owner)
- Review usage metrics
- Address pending feedback
- Update content for accuracy
- Verify all links and macros work
- Update version number if changed

### Semi-Annual Review (Template Committee)
- Review full template catalog
- Identify gaps (missing templates)
- Identify overlaps (duplicate templates)
- Evaluate template standards compliance
- Plan improvements for next half

### Annual Review (Leadership — 60 min meeting)

**Agenda:**
1. (10 min) Catalog stats: total templates, usage trend YoY, templates added/deprecated
2. (15 min) Top 5 templates by adoption — what makes them work
3. (15 min) Bottom 5 templates — deprecate, rework, or retrain?
4. (10 min) Gaps identified by teams — templates requested but not yet built
5. (10 min) Governance process retro — is the framework itself working? Adjust thresholds, roles, or cadence as needed

**Deliverable:** Updated Template Health page published within 1 week of meeting

## Getting Started

### For New Organizations
1. Start with 5-10 essential templates (meeting notes, decision record, project plan)
2. Assign owners for each template
3. Establish basic quality standards
4. Review after 90 days and expand
5. Formalize governance as template count grows beyond 20

FILE:references/template-design-patterns.md
# Template Design Patterns

## Overview

Well-designed Confluence and Jira templates accelerate team productivity by providing consistent starting points for common documents and workflows. This guide covers design patterns, variable handling, and best practices for creating reusable templates.

## Variable Placeholders

### Confluence Template Variables

**Syntax:** `<at:var at:name="variableName">default value</at:var>`

**Common Variables:**
| Variable | Purpose | Example Default |
|----------|---------|----------------|
| `projectName` | Project identifier | "Project Name" |
| `author` | Document author | "@mention author" |
| `date` | Creation or target date | "YYYY-MM-DD" |
| `status` | Current document status | "Draft" |
| `version` | Document version | "1.0" |
| `owner` | Responsible person | "@mention owner" |
| `reviewers` | Review participants | "@mention reviewers" |

**Best Practices:**
- Use descriptive variable names (camelCase)
- Always provide meaningful default values
- Group related variables together
- Include instruction text that users should replace
- Use Status macro for status variables (visual clarity)

### Jira Template Fields

**Custom Fields for Templates:**
- Text fields for structured input
- Select lists for controlled vocabularies
- Date fields for milestones
- User pickers for assignments
- Labels for categorization

## Conditional Sections

### Pattern: Role-Based Sections

Include or exclude content based on the document's audience:

```
## For Engineering (delete if not applicable)
- Technical requirements
- Architecture decisions
- Performance criteria

## For Design (delete if not applicable)
- User flows
- Wireframes
- Accessibility requirements

## For Business (delete if not applicable)
- ROI analysis
- Market context
- Success metrics
```

### Pattern: Complexity-Based Sections

Scale content depth based on project size:

```
## Required for All Projects
- Problem statement
- Solution overview
- Success metrics

## Required for Medium+ Projects (>2 weeks)
- Detailed requirements
- Technical design
- Test plan

## Required for Large Projects (>1 month)
- Architecture review
- Security review
- Rollback plan
- Communication plan
```

### Pattern: Optional Deep Dives

Use Expand macros for optional detail:

```
[Expand: Detailed Requirements]
  Content that power users may need but casual readers can skip
[/Expand]
```

## Reusable Components

### Header Block
Every template should start with a consistent header:

```
| Field | Value |
|-------|-------|
| Author | @mention |
| Status | [Status Macro: Draft] |
| Created | [Date] |
| Last Updated | [Date] |
| Reviewers | @mention |
| Approver | @mention |
```

### Decision Log Component
Reusable across templates that involve decisions:

```
## Decision Log
| # | Decision | Date | Decided By | Rationale |
|---|----------|------|-----------|-----------|
| 1 | [Decision] | [Date] | [Name] | [Why] |
```

### Change History Component
Track document evolution:

```
## Change History
| Version | Date | Author | Changes |
|---------|------|--------|---------|
| 1.0 | [Date] | [Name] | Initial version |
```

### Action Items Component
Standard task tracking:

```
## Action Items
- [ ] [Task description] - @assignee - Due: [date]
- [ ] [Task description] - @assignee - Due: [date]
```

## Macro Integration

### Recommended Macros per Template Type

**Meeting Notes Template:**
- Table of Contents (for long meetings)
- Action Items (task list macro)
- Jira Issues (link to discussed tickets)
- Expand (for detailed discussion notes)

**Decision Record Template:**
- Status macro (decision status)
- Page Properties (structured metadata)
- Info/Warning panels (context and caveats)
- Jira Issues (related tickets)

**Project Plan Template:**
- Roadmap Planner (timeline view)
- Jira Issues (JQL for project epics)
- Children Display (sub-pages for phases)
- Chart macro (status distribution)

**Runbook Template:**
- Code Block (commands and scripts)
- Warning panels (danger zones)
- Expand (detailed troubleshooting)
- Anchor links (quick navigation)

## Responsive Layouts

### Two-Column Layout
Use Confluence Section and Column macros:

```
[Section]
  [Column: 60%]
    Main content, description, details
  [/Column]
  [Column: 40%]
    Sidebar: metadata, quick links, status
  [/Column]
[/Section]
```

### Card Layout
For overview pages with multiple items:

```
[Section]
  [Column: 33%]
    [Panel: Card 1]
      Title, summary, link
    [/Panel]
  [/Column]
  [Column: 33%]
    [Panel: Card 2]
  [/Column]
  [Column: 33%]
    [Panel: Card 3]
  [/Column]
[/Section]
```

## Brand Consistency

### Visual Standards
- Use consistent heading levels (H1 for title, H2 for sections, H3 for subsections)
- Apply Info/Warning/Note panels consistently (same meaning across templates)
- Use Status macro colors consistently (Green=done, Yellow=in progress, Red=blocked)
- Maintain consistent table formatting (header row, alignment)

### Content Standards
- Use the same voice and tone across templates
- Standardize date format (YYYY-MM-DD or your organization's preference)
- Use consistent terminology (define terms in a glossary)
- Include the same footer/metadata block in all templates

## Versioning Strategy

### Template Version Control
- Include version number in template metadata
- Maintain a changelog for template updates
- Communicate template changes to users
- Keep previous versions accessible during transition periods

### Version Numbering
- **Major (2.0):** Structural changes, section additions/removals
- **Minor (1.1):** Content updates, improved instructions
- **Patch (1.0.1):** Typo fixes, formatting corrections

### Migration Path
When updating templates:
1. Create new version alongside old version
2. Announce change with migration guide
3. New documents use new template automatically
4. Existing documents do not need to be migrated (optional)
5. Deprecate old template after 90 days
6. Archive old template (do not delete)

## Template Catalog Organization

### Categorization
Organize templates by:
- **Document type:** Meeting notes, decisions, plans, runbooks
- **Team:** Engineering, product, marketing, HR
- **Lifecycle:** Planning, execution, review, retrospective
- **Frequency:** One-time, recurring, as-needed

### Discovery
- Maintain a "Template Index" page with descriptions and links
- Tag templates with consistent labels
- Include a "When to Use" section in each template
- Provide examples of completed documents using the template

FILE:scripts/template_scaffolder.py
#!/usr/bin/env python3
"""
Template Scaffolder

Generates Confluence page template markup in storage-format XHTML. Supports
built-in template types and custom section definitions with optional macros.

Usage:
    python template_scaffolder.py meeting-notes
    python template_scaffolder.py decision-log --format json
    python template_scaffolder.py custom --sections "Overview,Goals,Action Items" --macros toc,status
    python template_scaffolder.py --list
"""

import argparse
import json
import sys
from datetime import datetime
from typing import Any, Dict, List, Optional


# ---------------------------------------------------------------------------
# Macro Generators
# ---------------------------------------------------------------------------

def macro_toc() -> str:
    """Generate table of contents macro."""
    return '<ac:structured-macro ac:name="toc"><ac:parameter ac:name="printable">true</ac:parameter><ac:parameter ac:name="style">disc</ac:parameter><ac:parameter ac:name="maxLevel">3</ac:parameter></ac:structured-macro>'


def macro_status(text: str = "IN PROGRESS", color: str = "Yellow") -> str:
    """Generate status macro."""
    return f'<ac:structured-macro ac:name="status"><ac:parameter ac:name="colour">{color}</ac:parameter><ac:parameter ac:name="title">{text}</ac:parameter></ac:structured-macro>'


def macro_info_panel(content: str) -> str:
    """Generate info panel macro."""
    return f'<ac:structured-macro ac:name="info"><ac:rich-text-body><p>{content}</p></ac:rich-text-body></ac:structured-macro>'


def macro_warning_panel(content: str) -> str:
    """Generate warning panel macro."""
    return f'<ac:structured-macro ac:name="warning"><ac:rich-text-body><p>{content}</p></ac:rich-text-body></ac:structured-macro>'


def macro_note_panel(content: str) -> str:
    """Generate note panel macro."""
    return f'<ac:structured-macro ac:name="note"><ac:rich-text-body><p>{content}</p></ac:rich-text-body></ac:structured-macro>'


def macro_expand(title: str, content: str) -> str:
    """Generate expand/collapse macro."""
    return f'<ac:structured-macro ac:name="expand"><ac:parameter ac:name="title">{title}</ac:parameter><ac:rich-text-body>{content}</ac:rich-text-body></ac:structured-macro>'


def macro_jira_issues(jql: str) -> str:
    """Generate Jira issues macro."""
    return f'<ac:structured-macro ac:name="jira"><ac:parameter ac:name="jqlQuery">{jql}</ac:parameter><ac:parameter ac:name="columns">key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution</ac:parameter></ac:structured-macro>'


MACRO_MAP = {
    "toc": macro_toc,
    "status": macro_status,
    "info": macro_info_panel,
    "warning": macro_warning_panel,
    "note": macro_note_panel,
    "expand": macro_expand,
    "jira-issues": macro_jira_issues,
}


# ---------------------------------------------------------------------------
# Built-in Templates
# ---------------------------------------------------------------------------

def _section(title: str, content: str) -> str:
    """Generate a section with heading and content."""
    return f'<h2>{title}</h2>\n{content}\n'


def _table(headers: List[str], rows: List[List[str]]) -> str:
    """Generate an XHTML table."""
    parts = ['<table><colgroup>']
    for _ in headers:
        parts.append('<col />')
    parts.append('</colgroup><thead><tr>')
    for h in headers:
        parts.append(f'<th><p>{h}</p></th>')
    parts.append('</tr></thead><tbody>')
    for row in rows:
        parts.append('<tr>')
        for cell in row:
            parts.append(f'<td><p>{cell}</p></td>')
        parts.append('</tr>')
    parts.append('</tbody></table>')
    return ''.join(parts)


def template_meeting_notes() -> Dict[str, Any]:
    """Generate meeting notes template."""
    today = datetime.now().strftime("%Y-%m-%d")
    body = macro_toc() + '\n'
    body += macro_info_panel("Replace placeholder text with your meeting details.") + '\n'
    body += _section("Meeting Details", _table(
        ["Field", "Value"],
        [["Date", today], ["Time", ""], ["Location", ""], ["Facilitator", ""], ["Note Taker", ""]],
    ))
    body += _section("Attendees", '<ul><li><p>Name 1</p></li><li><p>Name 2</p></li></ul>')
    body += _section("Agenda", '<ol><li><p>Item 1</p></li><li><p>Item 2</p></li><li><p>Item 3</p></li></ol>')
    body += _section("Discussion Notes", '<p>Summary of discussion points...</p>')
    body += _section("Decisions Made", _table(
        ["Decision", "Owner", "Date"],
        [["", "", today]],
    ))
    body += _section("Action Items", _table(
        ["Action", "Owner", "Due Date", "Status"],
        [["", "", "", macro_status("TODO", "Grey")]],
    ))
    body += _section("Next Meeting", '<p>Date: TBD</p><p>Agenda items for next time:</p><ul><li><p></p></li></ul>')

    return {"name": "Meeting Notes", "body": body, "labels": ["meeting-notes", "template"]}


def template_decision_log() -> Dict[str, Any]:
    """Generate decision log template."""
    today = datetime.now().strftime("%Y-%m-%d")
    body = macro_toc() + '\n'
    body += _section("Decision Log", macro_info_panel("Track key decisions, context, and outcomes."))
    body += _table(
        ["ID", "Date", "Decision", "Context", "Alternatives Considered", "Outcome", "Owner", "Status"],
        [
            ["D-001", today, "", "", "", "", "", macro_status("DECIDED", "Green")],
            ["D-002", "", "", "", "", "", "", macro_status("PENDING", "Yellow")],
        ],
    )
    body += '\n'
    body += _section("Decision Template", macro_expand("Decision Details Template",
        '<h3>Context</h3><p>What is the issue or situation requiring a decision?</p>'
        '<h3>Options</h3><ol><li><p>Option A - pros/cons</p></li><li><p>Option B - pros/cons</p></li></ol>'
        '<h3>Decision</h3><p>What was decided and why?</p>'
        '<h3>Consequences</h3><p>What are the expected outcomes?</p>'
    ))

    return {"name": "Decision Log", "body": body, "labels": ["decision-log", "template"]}


def template_runbook() -> Dict[str, Any]:
    """Generate runbook template."""
    body = macro_toc() + '\n'
    body += macro_warning_panel("This runbook should be tested and reviewed quarterly.") + '\n'
    body += _section("Overview", '<p>Brief description of what this runbook covers.</p>'
        + _table(["Field", "Value"], [
            ["Service/System", ""], ["Owner", ""], ["Last Tested", ""],
            ["Severity", ""], ["Estimated Duration", ""],
        ]))
    body += _section("Prerequisites", '<ul><li><p>Access to system X</p></li><li><p>VPN connected</p></li><li><p>Required tools installed</p></li></ul>')
    body += _section("Steps", '<ol><li><p><strong>Step 1:</strong> Description</p><ac:structured-macro ac:name="code"><ac:parameter ac:name="language">bash</ac:parameter><ac:plain-text-body><![CDATA[# command here]]></ac:plain-text-body></ac:structured-macro></li>'
        '<li><p><strong>Step 2:</strong> Description</p></li>'
        '<li><p><strong>Step 3:</strong> Description</p></li></ol>')
    body += _section("Verification", '<p>How to verify the issue is resolved:</p><ul><li><p>Check 1</p></li><li><p>Check 2</p></li></ul>')
    body += _section("Rollback", macro_note_panel("If the above steps do not resolve the issue, follow these rollback steps.") +
        '<ol><li><p>Rollback step 1</p></li><li><p>Rollback step 2</p></li></ol>')
    body += _section("Escalation", _table(
        ["Level", "Contact", "When to Escalate"],
        [["L1", "", ""], ["L2", "", ""], ["L3", "", ""]],
    ))

    return {"name": "Runbook", "body": body, "labels": ["runbook", "operations", "template"]}


def template_project_kickoff() -> Dict[str, Any]:
    """Generate project kickoff template."""
    today = datetime.now().strftime("%Y-%m-%d")
    body = macro_toc() + '\n'
    body += _section("Project Overview", _table(
        ["Field", "Value"],
        [["Project Name", ""], ["Start Date", today], ["Target End Date", ""],
         ["Project Lead", ""], ["Sponsor", ""], ["Status", macro_status("KICKOFF", "Blue")]],
    ))
    body += _section("Vision & Goals", '<h3>Vision</h3><p>What does success look like?</p>'
        '<h3>Goals</h3><ol><li><p>Goal 1</p></li><li><p>Goal 2</p></li><li><p>Goal 3</p></li></ol>')
    body += _section("Scope", '<h3>In Scope</h3><ul><li><p></p></li></ul><h3>Out of Scope</h3><ul><li><p></p></li></ul>')
    body += _section("Stakeholders", _table(
        ["Name", "Role", "Responsibility", "Communication Preference"],
        [["", "", "", ""]],
    ))
    body += _section("Timeline & Milestones", _table(
        ["Milestone", "Target Date", "Status"],
        [["Phase 1", "", macro_status("NOT STARTED", "Grey")],
         ["Phase 2", "", macro_status("NOT STARTED", "Grey")]],
    ))
    body += _section("Risks", _table(
        ["Risk", "Likelihood", "Impact", "Mitigation"],
        [["", "High/Medium/Low", "High/Medium/Low", ""]],
    ))
    body += _section("Next Steps", '<ul><li><p></p></li></ul>')

    return {"name": "Project Kickoff", "body": body, "labels": ["project-kickoff", "template"]}


def template_sprint_retro() -> Dict[str, Any]:
    """Generate sprint retrospective template."""
    body = macro_toc() + '\n'
    body += _section("Sprint Info", _table(
        ["Field", "Value"],
        [["Sprint", ""], ["Date Range", ""], ["Facilitator", ""],
         ["Velocity", ""], ["Commitment", ""], ["Completion Rate", ""]],
    ))
    body += _section("What Went Well", '<ul><li><p></p></li></ul>')
    body += _section("What Could Be Improved", '<ul><li><p></p></li></ul>')
    body += _section("Action Items from Last Retro", _table(
        ["Action", "Owner", "Status"],
        [["", "", macro_status("DONE", "Green")], ["", "", macro_status("IN PROGRESS", "Yellow")]],
    ))
    body += _section("New Action Items", _table(
        ["Action", "Owner", "Due Date", "Priority"],
        [["", "", "", "High/Medium/Low"]],
    ))
    body += _section("Team Health Check", macro_info_panel("Rate each area 1-5 (1=needs work, 5=great)") + _table(
        ["Area", "Rating", "Trend", "Notes"],
        [["Teamwork", "", "", ""], ["Delivery", "", "", ""],
         ["Fun", "", "", ""], ["Learning", "", "", ""]],
    ))

    return {"name": "Sprint Retrospective", "body": body, "labels": ["sprint-retro", "agile", "template"]}


def template_how_to_guide() -> Dict[str, Any]:
    """Generate how-to guide template."""
    body = macro_toc() + '\n'
    body += macro_info_panel("This guide explains how to accomplish a specific task.") + '\n'
    body += _section("Overview", '<p>Brief description of what this guide covers and who it is for.</p>')
    body += _section("Prerequisites", '<ul><li><p>Prerequisite 1</p></li><li><p>Prerequisite 2</p></li></ul>')
    body += _section("Step-by-Step Instructions",
        '<h3>Step 1: Title</h3><p>Description of what to do.</p>'
        '<h3>Step 2: Title</h3><p>Description of what to do.</p>'
        '<h3>Step 3: Title</h3><p>Description of what to do.</p>')
    body += _section("Troubleshooting", macro_expand("Common Issues",
        '<h3>Issue 1</h3><p>Solution...</p>'
        '<h3>Issue 2</h3><p>Solution...</p>'))
    body += _section("Related Resources", '<ul><li><p>Link 1</p></li><li><p>Link 2</p></li></ul>')

    return {"name": "How-To Guide", "body": body, "labels": ["how-to", "guide", "template"]}


TEMPLATE_REGISTRY = {
    "meeting-notes": template_meeting_notes,
    "decision-log": template_decision_log,
    "runbook": template_runbook,
    "project-kickoff": template_project_kickoff,
    "sprint-retro": template_sprint_retro,
    "how-to-guide": template_how_to_guide,
}


# ---------------------------------------------------------------------------
# Custom Template Builder
# ---------------------------------------------------------------------------

def build_custom_template(
    sections: List[str],
    macros: List[str],
) -> Dict[str, Any]:
    """Build a custom template from sections and macros."""
    body = ""

    # Add requested macros at the top
    if "toc" in macros:
        body += macro_toc() + '\n'
    if "status" in macros:
        body += '<p>Status: ' + macro_status() + '</p>\n'

    for section in sections:
        section = section.strip()
        if not section:
            continue
        body += _section(section, '<p></p>')

    # Add panels if requested
    if "info" in macros:
        body = macro_info_panel("Add instructions or context here.") + '\n' + body
    if "warning" in macros:
        body += macro_warning_panel("Add warnings here.") + '\n'
    if "note" in macros:
        body += macro_note_panel("Add notes here.") + '\n'

    return {"name": "Custom Template", "body": body, "labels": ["custom", "template"]}


# ---------------------------------------------------------------------------
# Output Formatting
# ---------------------------------------------------------------------------

def format_text_output(result: Dict[str, Any]) -> str:
    """Format results as readable text report."""
    lines = []
    lines.append("=" * 60)
    lines.append(f"TEMPLATE: {result['name']}")
    lines.append("=" * 60)
    lines.append("")
    lines.append(f"Labels: {', '.join(result.get('labels', []))}")
    lines.append("")
    lines.append("CONFLUENCE STORAGE FORMAT MARKUP")
    lines.append("-" * 30)
    lines.append(result["body"])

    return "\n".join(lines)


def format_json_output(result: Dict[str, Any]) -> Dict[str, Any]:
    """Format results as JSON."""
    return result


def format_list_output(output_format: str) -> str:
    """Format available templates list."""
    if output_format == "json":
        templates = {}
        for name, func in TEMPLATE_REGISTRY.items():
            result = func()
            templates[name] = {
                "name": result["name"],
                "labels": result["labels"],
            }
        return json.dumps(templates, indent=2)

    lines = []
    lines.append("=" * 60)
    lines.append("AVAILABLE TEMPLATES")
    lines.append("=" * 60)
    lines.append("")
    for name, func in TEMPLATE_REGISTRY.items():
        result = func()
        lines.append(f"  {name}")
        lines.append(f"    Name: {result['name']}")
        lines.append(f"    Labels: {', '.join(result['labels'])}")
        lines.append("")
    lines.append(f"Total templates: {len(TEMPLATE_REGISTRY)}")
    lines.append("")
    lines.append("Usage:")
    lines.append("  python template_scaffolder.py <template-name>")
    lines.append('  python template_scaffolder.py custom --sections "Section1,Section2" --macros toc,status')
    return "\n".join(lines)


# ---------------------------------------------------------------------------
# CLI Interface
# ---------------------------------------------------------------------------

def main() -> int:
    """Main CLI entry point."""
    parser = argparse.ArgumentParser(
        description="Generate Confluence page template markup"
    )
    parser.add_argument(
        "template",
        nargs="?",
        help="Template name or 'custom' for custom template",
    )
    parser.add_argument(
        "--format",
        choices=["text", "json"],
        default="text",
        help="Output format (default: text)",
    )
    parser.add_argument(
        "--list",
        action="store_true",
        help="List all available template types",
    )
    parser.add_argument(
        "--sections",
        help='Comma-separated section names for custom template (e.g., "Overview,Goals,Action Items")',
    )
    parser.add_argument(
        "--macros",
        help='Comma-separated macro names to include (e.g., "toc,status,info")',
    )

    args = parser.parse_args()

    try:
        if args.list:
            print(format_list_output(args.format))
            return 0

        if not args.template:
            parser.error("template name is required unless --list is used")

        template_name = args.template.lower()

        if template_name == "custom":
            if not args.sections:
                parser.error("--sections is required for custom templates")
            sections = [s.strip() for s in args.sections.split(",")]
            macros = [m.strip() for m in args.macros.split(",")] if args.macros else []
            result = build_custom_template(sections, macros)
        elif template_name in TEMPLATE_REGISTRY:
            result = TEMPLATE_REGISTRY[template_name]()
        else:
            available = ", ".join(sorted(TEMPLATE_REGISTRY.keys()))
            print(f"Error: Unknown template '{template_name}'. Available: {available}", file=sys.stderr)
            return 1

        if args.format == "json":
            print(json.dumps(format_json_output(result), indent=2))
        else:
            print(format_text_output(result))

        return 0

    except Exception as e:
        print(f"Error: {e}", file=sys.stderr)
        return 1


if __name__ == "__main__":
    sys.exit(main())

---
name: "autoresearch-agent"
description: "Autonomous experiment loop that optimizes any file by a measurable metric. Inspired by Karpathy's autoresearch. The agent edits a target file, runs a fixed evaluation, keeps improvements (git commit), discards failures (git reset), and loops indefinitely. Use when: user wants to optimize code speed, reduce bundle/image size, improve test pass rate, optimize prompts, improve content quality (headlines, copy, CTR), or run any measurable improvement loop. Requires: a target file, an evaluation command that outputs a metric, and a git repo."
license: MIT
metadata:
  version: 2.0.0
  author: Alireza Rezvani
  category: engineering
  updated: 2026-03-13
---

# Autoresearch Agent

> You sleep. The agent experiments. You wake up to results.

Autonomous experiment loop inspired by [Karpathy's autoresearch](https://github.com/karpathy/autoresearch). The agent edits one file, runs a fixed evaluation, keeps improvements, discards failures, and loops indefinitely.

Not one guess — fifty measured attempts, compounding.

---

## Slash Commands

| Command | What it does |
|---------|-------------|
| `/ar:setup` | Set up a new experiment interactively |
| `/ar:run` | Run a single experiment iteration |
| `/ar:loop` | Start autonomous loop with configurable interval (10m, 1h, daily, weekly, monthly) |
| `/ar:status` | Show dashboard and results |
| `/ar:resume` | Resume a paused experiment |

---

## When This Skill Activates

Recognize these patterns from the user:

- "Make this faster / smaller / better"
- "Optimize [file] for [metric]"
- "Improve my [headlines / copy / prompts]"
- "Run experiments overnight"
- "I want to get [metric] from X to Y"
- Any request involving: optimize, benchmark, improve, experiment loop, autoresearch

If the user describes a target file + a way to measure success → this skill applies.

---

## Setup

### First Time — Create the Experiment

Run the setup script. The user decides where experiments live:

**Project-level** (inside repo, git-tracked, shareable with team):
```bash
python scripts/setup_experiment.py \
  --domain engineering \
  --name api-speed \
  --target src/api/search.py \
  --eval "pytest bench.py --tb=no -q" \
  --metric p50_ms \
  --direction lower \
  --scope project
```

**User-level** (personal, in `~/.autoresearch/`):
```bash
python scripts/setup_experiment.py \
  --domain marketing \
  --name medium-ctr \
  --target content/titles.md \
  --eval "python evaluate.py" \
  --metric ctr_score \
  --direction higher \
  --evaluator llm_judge_content \
  --scope user
```

The `--scope` flag determines where `.autoresearch/` lives:
- `project` (default) → `.autoresearch/` in the repo root. Experiment definitions are git-tracked. Results are gitignored.
- `user` → `~/.autoresearch/` in the home directory. Everything is personal.

### What Setup Creates

```
.autoresearch/
├── config.yaml                        ← Global settings
├── .gitignore                         ← Ignores results.tsv, *.log
└── {domain}/{experiment-name}/
    ├── program.md                     ← Objectives, constraints, strategy
    ├── config.cfg                     ← Target, eval cmd, metric, direction
    ├── results.tsv                    ← Experiment log (gitignored)
    └── evaluate.py                    ← Evaluation script (if --evaluator used)
```

**results.tsv columns:** `commit | metric | status | description`
- `commit` — short git hash
- `metric` — float value or "N/A" for crashes
- `status` — keep | discard | crash
- `description` — what changed or why it crashed

### Domains

| Domain | Use Cases |
|--------|-----------|
| `engineering` | Code speed, memory, bundle size, test pass rate, build time |
| `marketing` | Headlines, social copy, email subjects, ad copy, engagement |
| `content` | Article structure, SEO descriptions, readability, CTR |
| `prompts` | System prompts, chatbot tone, agent instructions |
| `custom` | Anything else with a measurable metric |

### If `program.md` Already Exists

The user may have written their own `program.md`. If found in the experiment directory, read it. It overrides the template. Only ask for what's missing.

---

## Agent Protocol

You are the loop. The scripts handle setup and evaluation — you handle the creative work.

### Before Starting
1. Read `.autoresearch/{domain}/{name}/config.cfg` to get:
   - `target` — the file you edit
   - `evaluate_cmd` — the command that measures your changes
   - `metric` — the metric name to look for in eval output
   - `metric_direction` — "lower" or "higher" is better
   - `time_budget_minutes` — max time per evaluation
2. Read `program.md` for strategy, constraints, and what you can/cannot change
3. Read `results.tsv` for experiment history (columns: commit, metric, status, description)
4. Checkout the experiment branch: `git checkout autoresearch/{domain}/{name}`

### Each Iteration
1. Review results.tsv — what worked? What failed? What hasn't been tried?
2. Decide ONE change to the target file. One variable per experiment.
3. Edit the target file
4. Commit: `git add {target} && git commit -m "experiment: {description}"`
5. Evaluate: `python scripts/run_experiment.py --experiment {domain}/{name} --single`
6. Read the output — it prints KEEP, DISCARD, or CRASH with the metric value
7. Go to step 1

### What the Script Handles (you don't)
- Running the eval command with timeout
- Parsing the metric from eval output
- Comparing to previous best
- Reverting the commit on failure (`git reset --hard HEAD~1`)
- Logging the result to results.tsv

### Starting an Experiment

```bash
# Single iteration (the agent calls this repeatedly)
python scripts/run_experiment.py --experiment engineering/api-speed --single

# Dry run (test setup before starting)
python scripts/run_experiment.py --experiment engineering/api-speed --dry-run
```

### Strategy Escalation
- Runs 1-5: Low-hanging fruit (obvious improvements, simple optimizations)
- Runs 6-15: Systematic exploration (vary one parameter at a time)
- Runs 16-30: Structural changes (algorithm swaps, architecture shifts)
- Runs 30+: Radical experiments (completely different approaches)
- If no improvement in 20+ runs: update program.md Strategy section

### Self-Improvement
After every 10 experiments, review results.tsv for patterns. Update the
Strategy section of program.md with what you learned (e.g., "caching changes
consistently improve by 5-10%", "refactoring attempts never improve the metric").
Future iterations benefit from this accumulated knowledge.

### Stopping
- Run until interrupted by the user, context limit reached, or goal in program.md is met
- Before stopping: ensure results.tsv is up to date
- On context limit: the next session can resume — results.tsv and git log persist

### Rules

- **One change per experiment.** Don't change 5 things at once. You won't know what worked.
- **Simplicity criterion.** A small improvement that adds ugly complexity is not worth it. Equal performance with simpler code is a win. Removing code that gets same results is the best outcome.
- **Never modify the evaluator.** `evaluate.py` is the ground truth. Modifying it invalidates all comparisons. Hard stop if you catch yourself doing this.
- **Timeout.** If a run exceeds 2.5× the time budget, kill it and treat as crash.
- **Crash handling.** If it's a typo or missing import, fix and re-run. If the idea is fundamentally broken, revert, log "crash", move on. 5 consecutive crashes → pause and alert.
- **No new dependencies.** Only use what's already available in the project.

---

## Evaluators

Ready-to-use evaluation scripts. Copied into the experiment directory during setup with `--evaluator`.

### Free Evaluators (no API cost)

| Evaluator | Metric | Use Case |
|-----------|--------|----------|
| `benchmark_speed` | `p50_ms` (lower) | Function/API execution time |
| `benchmark_size` | `size_bytes` (lower) | File, bundle, Docker image size |
| `test_pass_rate` | `pass_rate` (higher) | Test suite pass percentage |
| `build_speed` | `build_seconds` (lower) | Build/compile/Docker build time |
| `memory_usage` | `peak_mb` (lower) | Peak memory during execution |

### LLM Judge Evaluators (uses your subscription)

| Evaluator | Metric | Use Case |
|-----------|--------|----------|
| `llm_judge_content` | `ctr_score` 0-10 (higher) | Headlines, titles, descriptions |
| `llm_judge_prompt` | `quality_score` 0-100 (higher) | System prompts, agent instructions |
| `llm_judge_copy` | `engagement_score` 0-10 (higher) | Social posts, ad copy, emails |

LLM judges call the CLI tool the user is already running (Claude, Codex, Gemini). The evaluation prompt is locked inside `evaluate.py` — the agent cannot modify it. This prevents the agent from gaming its own evaluator.

The user's existing subscription covers the cost:
- Claude Code Max → unlimited Claude calls for evaluation
- Codex CLI (ChatGPT Pro) → unlimited Codex calls
- Gemini CLI (free tier) → free evaluation calls

### Custom Evaluators

If no built-in evaluator fits, the user writes their own `evaluate.py`. Only requirement: it must print `metric_name: value` to stdout.

```python
#!/usr/bin/env python3
# My custom evaluator — DO NOT MODIFY after experiment starts
import subprocess
result = subprocess.run(["my-benchmark", "--json"], capture_output=True, text=True)
# Parse and output
print(f"my_metric: {parse_score(result.stdout)}")
```

---

## Viewing Results

```bash
# Single experiment
python scripts/log_results.py --experiment engineering/api-speed

# All experiments in a domain
python scripts/log_results.py --domain engineering

# Cross-experiment dashboard
python scripts/log_results.py --dashboard

# Export formats
python scripts/log_results.py --experiment engineering/api-speed --format csv --output results.csv
python scripts/log_results.py --experiment engineering/api-speed --format markdown --output results.md
python scripts/log_results.py --dashboard --format markdown --output dashboard.md
```

### Dashboard Output

```
DOMAIN          EXPERIMENT          RUNS  KEPT  BEST         Δ FROM START  STATUS
engineering     api-speed            47    14   185ms        -76.9%        active
engineering     bundle-size          23     8   412KB        -58.3%        paused
marketing       medium-ctr           31    11   8.4/10       +68.0%        active
prompts         support-tone         15     6   82/100       +46.4%        done
```

### Export Formats

- **TSV** — default, tab-separated (compatible with spreadsheets)
- **CSV** — comma-separated, with proper quoting
- **Markdown** — formatted table, readable in GitHub/docs

---

## Proactive Triggers

Flag these without being asked:

- **No evaluation command works** → Test it before starting the loop. Run once, verify output.
- **Target file not in git** → `git init && git add . && git commit -m 'initial'` first.
- **Metric direction unclear** → Ask: is lower or higher better? Must know before starting.
- **Time budget too short** → If eval takes longer than budget, every run crashes.
- **Agent modifying evaluate.py** → Hard stop. This invalidates all comparisons.
- **5 consecutive crashes** → Pause the loop. Alert the user. Don't keep burning cycles.
- **No improvement in 20+ runs** → Suggest changing strategy in program.md or trying a different approach.

---

## Installation

### One-liner (any tool)
```bash
git clone https://github.com/alirezarezvani/claude-skills.git
cp -r claude-skills/engineering/autoresearch-agent ~/.claude/skills/
```

### Multi-tool install
```bash
./scripts/convert.sh --skill autoresearch-agent --tool codex|gemini|cursor|windsurf|openclaw
```

### OpenClaw
```bash
clawhub install cs-autoresearch-agent
```

---

## Related Skills

- **self-improving-agent** — improves an agent's own memory/rules over time. NOT for structured experiment loops.
- **senior-ml-engineer** — ML architecture decisions. Complementary — use for initial design, then autoresearch for optimization.
- **tdd-guide** — test-driven development. Complementary — tests can be the evaluation function.
- **skill-security-auditor** — audit skills before publishing. NOT for optimization loops.

FILE:references/experiment-domains.md
# Experiment Domains Guide

## Domain: Engineering

### Code Speed Optimization

```bash
python scripts/setup_experiment.py \
  --domain engineering \
  --name api-speed \
  --target src/api/search.py \
  --eval "python -m pytest tests/bench_search.py --tb=no -q" \
  --metric p50_ms \
  --direction lower \
  --evaluator benchmark_speed
```

**What the agent optimizes:** Algorithm, data structures, caching, query patterns, I/O.
**Cost:** Free — just runs benchmarks.
**Speed:** ~5 min/experiment, ~12/hour, ~100 overnight.

### Bundle Size Reduction

```bash
python scripts/setup_experiment.py \
  --domain engineering \
  --name bundle-size \
  --target webpack.config.js \
  --eval "npm run build && python .autoresearch/engineering/bundle-size/evaluate.py" \
  --metric size_bytes \
  --direction lower \
  --evaluator benchmark_size
```

Edit `evaluate.py` to set `TARGET_FILE = "dist/main.js"` and add `BUILD_CMD = "npm run build"`.

### Test Pass Rate

```bash
python scripts/setup_experiment.py \
  --domain engineering \
  --name fix-flaky-tests \
  --target src/utils/parser.py \
  --eval "python .autoresearch/engineering/fix-flaky-tests/evaluate.py" \
  --metric pass_rate \
  --direction higher \
  --evaluator test_pass_rate
```

### Docker Build Speed

```bash
python scripts/setup_experiment.py \
  --domain engineering \
  --name docker-build \
  --target Dockerfile \
  --eval "python .autoresearch/engineering/docker-build/evaluate.py" \
  --metric build_seconds \
  --direction lower \
  --evaluator build_speed
```

### Memory Optimization

```bash
python scripts/setup_experiment.py \
  --domain engineering \
  --name memory-usage \
  --target src/processor.py \
  --eval "python .autoresearch/engineering/memory-usage/evaluate.py" \
  --metric peak_mb \
  --direction lower \
  --evaluator memory_usage
```

### ML Training (Karpathy-style)

Requires NVIDIA GPU. See [autoresearch](https://github.com/karpathy/autoresearch).

```bash
python scripts/setup_experiment.py \
  --domain engineering \
  --name ml-training \
  --target train.py \
  --eval "uv run train.py" \
  --metric val_bpb \
  --direction lower \
  --time-budget 5
```

---

## Domain: Marketing

### Medium Article Headlines

```bash
python scripts/setup_experiment.py \
  --domain marketing \
  --name medium-ctr \
  --target content/titles.md \
  --eval "python .autoresearch/marketing/medium-ctr/evaluate.py" \
  --metric ctr_score \
  --direction higher \
  --evaluator llm_judge_content
```

Edit `evaluate.py`: set `TARGET_FILE = "content/titles.md"` and `CLI_TOOL = "claude"`.

**What the agent optimizes:** Title phrasing, curiosity gaps, specificity, emotional triggers.
**Cost:** Uses your CLI subscription (Claude Max = unlimited).
**Speed:** ~2 min/experiment, ~30/hour.

### Social Media Copy

```bash
python scripts/setup_experiment.py \
  --domain marketing \
  --name twitter-engagement \
  --target social/tweets.md \
  --eval "python .autoresearch/marketing/twitter-engagement/evaluate.py" \
  --metric engagement_score \
  --direction higher \
  --evaluator llm_judge_copy
```

Edit `evaluate.py`: set `PLATFORM = "twitter"` (or linkedin, instagram).

### Email Subject Lines

```bash
python scripts/setup_experiment.py \
  --domain marketing \
  --name email-open-rate \
  --target emails/subjects.md \
  --eval "python .autoresearch/marketing/email-open-rate/evaluate.py" \
  --metric engagement_score \
  --direction higher \
  --evaluator llm_judge_copy
```

Edit `evaluate.py`: set `PLATFORM = "email"`.

### Ad Copy

```bash
python scripts/setup_experiment.py \
  --domain marketing \
  --name ad-copy-q2 \
  --target ads/google-search.md \
  --eval "python .autoresearch/marketing/ad-copy-q2/evaluate.py" \
  --metric engagement_score \
  --direction higher \
  --evaluator llm_judge_copy
```

Edit `evaluate.py`: set `PLATFORM = "ad"`.

---

## Domain: Content

### Article Structure & Readability

```bash
python scripts/setup_experiment.py \
  --domain content \
  --name article-structure \
  --target drafts/my-article.md \
  --eval "python .autoresearch/content/article-structure/evaluate.py" \
  --metric ctr_score \
  --direction higher \
  --evaluator llm_judge_content
```

### SEO Descriptions

```bash
python scripts/setup_experiment.py \
  --domain content \
  --name seo-meta \
  --target seo/descriptions.md \
  --eval "python .autoresearch/content/seo-meta/evaluate.py" \
  --metric ctr_score \
  --direction higher \
  --evaluator llm_judge_content
```

---

## Domain: Prompts

### System Prompt Optimization

```bash
python scripts/setup_experiment.py \
  --domain prompts \
  --name support-bot \
  --target prompts/support-system.md \
  --eval "python .autoresearch/prompts/support-bot/evaluate.py" \
  --metric quality_score \
  --direction higher \
  --evaluator llm_judge_prompt
```

Requires `tests/cases.json` with test inputs and expected outputs:

```json
[
  {
    "input": "I can't log in to my account",
    "expected": "Ask for email, check account status, offer password reset"
  },
  {
    "input": "How do I cancel my subscription?",
    "expected": "Empathetic response, explain cancellation steps, offer retention"
  }
]
```

### Agent Skill Optimization

```bash
python scripts/setup_experiment.py \
  --domain prompts \
  --name skill-improvement \
  --target SKILL.md \
  --eval "python .autoresearch/prompts/skill-improvement/evaluate.py" \
  --metric quality_score \
  --direction higher \
  --evaluator llm_judge_prompt
```

---

## Choosing Your Domain

| I want to... | Domain | Evaluator | Cost |
|-------------|--------|-----------|------|
| Speed up my code | engineering | benchmark_speed | Free |
| Shrink my bundle | engineering | benchmark_size | Free |
| Fix flaky tests | engineering | test_pass_rate | Free |
| Speed up Docker builds | engineering | build_speed | Free |
| Reduce memory usage | engineering | memory_usage | Free |
| Train ML models | engineering | (custom) | Free + GPU |
| Write better headlines | marketing | llm_judge_content | Subscription |
| Improve social posts | marketing | llm_judge_copy | Subscription |
| Optimize email subjects | marketing | llm_judge_copy | Subscription |
| Improve ad copy | marketing | llm_judge_copy | Subscription |
| Optimize article structure | content | llm_judge_content | Subscription |
| Improve SEO descriptions | content | llm_judge_content | Subscription |
| Optimize system prompts | prompts | llm_judge_prompt | Subscription |
| Improve agent skills | prompts | llm_judge_prompt | Subscription |

**First time?** Start with an engineering experiment (free, fast, measurable). Once comfortable, try content/marketing with LLM judges.

FILE:references/program-template.md
# program.md Templates

Copy the template for your domain and paste into your project root as `program.md`.

---

## ML Training (Karpathy-style)

```markdown
# autoresearch — ML Training

## Goal
Minimize val_bpb on the validation set. Lower is better.

## What You Can Change (train.py only)
- Model architecture (depth, width, attention heads, FFN ratio)
- Optimizer (learning rate, warmup, scheduler, weight decay)
- Training loop (batch size, gradient accumulation, clipping)
- Regularization (dropout, weight tying, etc.)
- Any self-contained improvement that doesn't require new packages

## What You Cannot Change
- prepare.py (fixed — contains evaluation harness)
- Dependencies (pyproject.toml is locked)
- Time budget (always 5 minutes, wall clock)
- Evaluation metric (val_bpb is the ground truth)

## Strategy
1. First run: establish baseline. Do not change anything.
2. Explore learning rate range (try 2x and 0.5x current)
3. Try depth changes (±2 layers)
4. Try optimizer changes (Muon vs. AdamW variants)
5. If things improve, double down. If stuck, try something radical.

## Simplicity Rule
A small improvement with ugly code is NOT worth it.
Equal performance with simpler code IS worth it.
Removing code that gets same results is the best outcome.

## Stop When
val_bpb < 0.95 OR after 100 experiments, whichever comes first.
```

---

## Prompt Engineering

```markdown
# autoresearch — Prompt Optimization

## Goal
Maximize eval_score on the test suite. Higher is better (0-100).

## What You Can Change (prompt.md only)
- System prompt instructions
- Examples and few-shot demonstrations
- Output format specifications
- Chain-of-thought instructions
- Persona and tone
- Task decomposition strategies

## What You Cannot Change
- evaluate.py (fixed evaluation harness)
- Test cases in tests/ (ground truth)
- Model being evaluated (specified in evaluate.py)
- Scoring criteria (defined in evaluate.py)

## Strategy
1. First run: baseline with current prompt (or empty)
2. Add clear role/persona definition
3. Add output format specification
4. Add chain-of-thought instruction
5. Add 2-3 diverse examples
6. Refine based on failure modes from run.log

## Evaluation
- evaluate.py runs the prompt against 20 test cases
- Each test case is scored 1-10 by your CLI tool (Claude, Codex, or Gemini)
- quality_score = average * 10 (maps to 10-100)
- Run log shows which test cases failed

## Stop When
eval_score >= 85 OR after 50 experiments.
```

---

## Code Performance

```markdown
# autoresearch — Performance Optimization

## Goal
Minimize p50_ms (median latency). Lower is better.

## What You Can Change (src/module.py only)
- Algorithm implementation
- Data structures (use faster alternatives)
- Caching and memoization
- Vectorization (NumPy, etc.)
- Loop optimization
- I/O patterns
- Memory allocation patterns

## What You Cannot Change
- benchmark.py (fixed benchmark harness)
- Public API (function signatures must stay the same)
- External dependencies (add nothing new)
- Correctness tests (tests/ must still pass)

## Constraints
- Correctness is non-negotiable. benchmark.py runs tests first.
- If tests fail → immediate crash status, no metric recorded.
- Memory usage: p99 < 2x baseline acceptable, hard limit at 4x.

## Strategy
1. Baseline: profile first, don't guess
2. Check if there's any O(n²) → O(n log n) opportunity
3. Try caching repeated computations
4. Try NumPy vectorization for loops
5. Try algorithm-level changes last (higher risk)

## Stop When
p50_ms < 50ms OR improvement plateaus for 10 consecutive experiments.
```

---

## Agent Skill Optimization

```markdown
# autoresearch — Skill Optimization

## Goal
Maximize pass_rate on the task evaluation suite. Higher is better (0-1).

## What You Can Change (SKILL.md only)
- Skill description and trigger phrases
- Core workflow steps and ordering
- Decision frameworks and rules
- Output format specifications
- Example inputs/outputs
- Related skills disambiguation
- Proactive trigger conditions

## What You Cannot Change
- your custom evaluate.py (see Custom Evaluators in SKILL.md)
- Test tasks in tests/ (ground truth benchmark)
- Skill name (used for routing)
- License or metadata

## Evaluation
- evaluate.py runs SKILL.md against 15 standardized tasks
- Your CLI tool scores each task: 0 (fail), 0.5 (partial), 1 (pass)
- pass_rate = sum(scores) / 15

## Strategy
1. Baseline: run as-is
2. Improve trigger description (better routing = more passes)
3. Sharpen the core workflow (clearer = better execution)
4. Add missing edge cases to the rules section
5. Improve disambiguation (reduce false-positive routing)

## Simplicity Rule
A shorter SKILL.md that achieves the same score is better.
Aim for 200-400 lines total.

## Stop When
pass_rate >= 0.90 OR after 30 experiments.
```

FILE:scripts/log_results.py
#!/usr/bin/env python3
"""
autoresearch-agent: Results Viewer

View experiment results in multiple formats: terminal, CSV, Markdown.
Supports single experiment, domain, or cross-experiment dashboard.

Usage:
    python scripts/log_results.py --experiment engineering/api-speed
    python scripts/log_results.py --domain engineering
    python scripts/log_results.py --dashboard
    python scripts/log_results.py --experiment engineering/api-speed --format csv --output results.csv
    python scripts/log_results.py --experiment engineering/api-speed --format markdown --output results.md
    python scripts/log_results.py --dashboard --format markdown --output dashboard.md
"""

import argparse
import csv
import io
import sys
import time
from pathlib import Path


def find_autoresearch_root():
    """Find .autoresearch/ in project or user home."""
    project_root = Path(".").resolve() / ".autoresearch"
    if project_root.exists():
        return project_root
    user_root = Path.home() / ".autoresearch"
    if user_root.exists():
        return user_root
    return None


def load_config(experiment_dir):
    """Load config.cfg."""
    cfg_file = experiment_dir / "config.cfg"
    config = {}
    if cfg_file.exists():
        for line in cfg_file.read_text().splitlines():
            if ":" in line:
                k, v = line.split(":", 1)
                config[k.strip()] = v.strip()
    return config


def load_results(experiment_dir):
    """Load results.tsv into list of dicts."""
    tsv = experiment_dir / "results.tsv"
    if not tsv.exists():
        return []
    results = []
    for line in tsv.read_text().splitlines()[1:]:
        parts = line.split("\t")
        if len(parts) >= 4:
            try:
                metric = float(parts[1]) if parts[1] != "N/A" else None
            except ValueError:
                metric = None
            results.append({
                "commit": parts[0],
                "metric": metric,
                "status": parts[2],
                "description": parts[3],
            })
    return results


def compute_stats(results, direction):
    """Compute statistics from results."""
    keeps = [r for r in results if r["status"] == "keep"]
    discards = [r for r in results if r["status"] == "discard"]
    crashes = [r for r in results if r["status"] == "crash"]

    valid_keeps = [r for r in keeps if r["metric"] is not None]
    baseline = valid_keeps[0]["metric"] if valid_keeps else None
    if valid_keeps:
        best = min(r["metric"] for r in valid_keeps) if direction == "lower" else max(r["metric"] for r in valid_keeps)
    else:
        best = None

    pct_change = None
    if baseline is not None and best is not None and baseline != 0:
        if direction == "lower":
            pct_change = (baseline - best) / baseline * 100
        else:
            pct_change = (best - baseline) / baseline * 100

    return {
        "total": len(results),
        "keeps": len(keeps),
        "discards": len(discards),
        "crashes": len(crashes),
        "baseline": baseline,
        "best": best,
        "pct_change": pct_change,
    }


# --- Terminal Output ---

def print_experiment(experiment_dir, experiment_path):
    """Print single experiment results to terminal."""
    config = load_config(experiment_dir)
    results = load_results(experiment_dir)
    direction = config.get("metric_direction", "lower")
    metric_name = config.get("metric", "metric")

    if not results:
        print(f"No results for {experiment_path}")
        return

    stats = compute_stats(results, direction)

    print(f"\n{'─' * 65}")
    print(f"  {experiment_path}")
    print(f"  Target: {config.get('target', '?')} | Metric: {metric_name} ({direction})")
    print(f"{'─' * 65}")
    print(f"  Total: {stats['total']} | Keep: {stats['keeps']} | Discard: {stats['discards']} | Crash: {stats['crashes']}")

    if stats["baseline"] is not None and stats["best"] is not None:
        pct = f" ({stats['pct_change']:+.1f}%)" if stats["pct_change"] is not None else ""
        print(f"  Baseline: {stats['baseline']:.6f} -> Best: {stats['best']:.6f}{pct}")

    print(f"\n  {'COMMIT':<10} {'METRIC':>12} {'STATUS':<10} DESCRIPTION")
    print(f"  {'─' * 60}")
    for r in results:
        m = f"{r['metric']:.6f}" if r["metric"] is not None else "N/A     "
        icon = {"keep": "+", "discard": "-", "crash": "!"}.get(r["status"], "?")
        print(f"  {r['commit']:<10} {m:>12} {icon} {r['status']:<7} {r['description'][:35]}")
    print()


def print_dashboard(root):
    """Print cross-experiment dashboard."""
    experiments = []
    for domain_dir in sorted(root.iterdir()):
        if not domain_dir.is_dir() or domain_dir.name.startswith("."):
            continue
        for exp_dir in sorted(domain_dir.iterdir()):
            if not exp_dir.is_dir() or not (exp_dir / "config.cfg").exists():
                continue
            config = load_config(exp_dir)
            results = load_results(exp_dir)
            direction = config.get("metric_direction", "lower")
            stats = compute_stats(results, direction)

            best_str = f"{stats['best']:.4f}" if stats["best"] is not None else "—"
            pct_str = f"{stats['pct_change']:+.1f}%" if stats["pct_change"] is not None else "—"

            # Determine status
            status = "idle"
            if stats["total"] > 0:
                tsv = exp_dir / "results.tsv"
                if tsv.exists():
                    age_hours = (time.time() - tsv.stat().st_mtime) / 3600
                    status = "active" if age_hours < 1 else "paused" if age_hours < 24 else "done"

            experiments.append({
                "domain": domain_dir.name,
                "name": exp_dir.name,
                "runs": stats["total"],
                "kept": stats["keeps"],
                "best": best_str,
                "change": pct_str,
                "status": status,
                "metric": config.get("metric", "?"),
            })

    if not experiments:
        print("No experiments found.")
        return experiments

    print(f"\n{'─' * 90}")
    print(f"  autoresearch — Dashboard")
    print(f"{'─' * 90}")
    print(f"  {'DOMAIN':<15} {'EXPERIMENT':<20} {'RUNS':>5} {'KEPT':>5} {'BEST':>12} {'CHANGE':>10} {'STATUS':<8}")
    print(f"  {'─' * 85}")
    for e in experiments:
        print(f"  {e['domain']:<15} {e['name']:<20} {e['runs']:>5} {e['kept']:>5} {e['best']:>12} {e['change']:>10} {e['status']:<8}")
    print()
    return experiments


# --- CSV Export ---

def export_experiment_csv(experiment_dir, experiment_path):
    """Export single experiment as CSV string."""
    config = load_config(experiment_dir)
    results = load_results(experiment_dir)
    direction = config.get("metric_direction", "lower")
    stats = compute_stats(results, direction)

    buf = io.StringIO()
    writer = csv.writer(buf)

    # Header with metadata
    writer.writerow(["# Experiment", experiment_path])
    writer.writerow(["# Target", config.get("target", "")])
    writer.writerow(["# Metric", f"{config.get('metric', '')} ({direction} is better)"])
    if stats["baseline"] is not None:
        writer.writerow(["# Baseline", f"{stats['baseline']:.6f}"])
    if stats["best"] is not None:
        pct = f" ({stats['pct_change']:+.1f}%)" if stats["pct_change"] is not None else ""
        writer.writerow(["# Best", f"{stats['best']:.6f}{pct}"])
    writer.writerow(["# Total", stats["total"]])
    writer.writerow(["# Keep/Discard/Crash", f"{stats['keeps']}/{stats['discards']}/{stats['crashes']}"])
    writer.writerow([])

    writer.writerow(["Commit", "Metric", "Status", "Description"])
    for r in results:
        m = f"{r['metric']:.6f}" if r["metric"] is not None else "N/A"
        writer.writerow([r["commit"], m, r["status"], r["description"]])

    return buf.getvalue()


def export_dashboard_csv(root, domain_filter=None):
    """Export dashboard as CSV string."""
    experiments = []
    for domain_dir in sorted(root.iterdir()):
        if not domain_dir.is_dir() or domain_dir.name.startswith("."):
            continue
        if domain_filter and domain_dir.name != domain_filter:
            continue
        for exp_dir in sorted(domain_dir.iterdir()):
            if not exp_dir.is_dir() or not (exp_dir / "config.cfg").exists():
                continue
            config = load_config(exp_dir)
            results = load_results(exp_dir)
            direction = config.get("metric_direction", "lower")
            stats = compute_stats(results, direction)
            best_str = f"{stats['best']:.6f}" if stats["best"] is not None else ""
            pct_str = f"{stats['pct_change']:+.1f}%" if stats["pct_change"] is not None else ""
            experiments.append([
                domain_dir.name, exp_dir.name, config.get("metric", ""),
                stats["total"], stats["keeps"], stats["discards"], stats["crashes"],
                best_str, pct_str
            ])

    buf = io.StringIO()
    writer = csv.writer(buf)
    writer.writerow(["Domain", "Experiment", "Metric", "Runs", "Kept", "Discarded", "Crashed", "Best", "Change"])
    for e in experiments:
        writer.writerow(e)
    return buf.getvalue()


# --- Markdown Export ---

def export_experiment_markdown(experiment_dir, experiment_path):
    """Export single experiment as Markdown string."""
    config = load_config(experiment_dir)
    results = load_results(experiment_dir)
    direction = config.get("metric_direction", "lower")
    metric_name = config.get("metric", "metric")
    stats = compute_stats(results, direction)

    lines = []
    lines.append(f"# Autoresearch: {experiment_path}\n")
    lines.append(f"**Target:** `{config.get('target', '?')}`  ")
    lines.append(f"**Metric:** `{metric_name}` ({direction} is better)  ")
    lines.append(f"**Experiments:** {stats['total']} total — {stats['keeps']} kept, {stats['discards']} discarded, {stats['crashes']} crashed\n")

    if stats["baseline"] is not None and stats["best"] is not None:
        pct = f" ({stats['pct_change']:+.1f}%)" if stats["pct_change"] is not None else ""
        lines.append(f"**Progress:** `{stats['baseline']:.6f}` → `{stats['best']:.6f}`{pct}\n")

    lines.append(f"| Commit | Metric | Status | Description |")
    lines.append(f"|--------|--------|--------|-------------|")
    for r in results:
        m = f"`{r['metric']:.6f}`" if r["metric"] is not None else "N/A"
        lines.append(f"| `{r['commit']}` | {m} | {r['status']} | {r['description']} |")
    lines.append("")

    return "\n".join(lines)


def export_dashboard_markdown(root, domain_filter=None):
    """Export dashboard as Markdown string."""
    lines = []
    lines.append("# Autoresearch Dashboard\n")
    lines.append("| Domain | Experiment | Metric | Runs | Kept | Best | Change | Status |")
    lines.append("|--------|-----------|--------|------|------|------|--------|--------|")

    for domain_dir in sorted(root.iterdir()):
        if not domain_dir.is_dir() or domain_dir.name.startswith("."):
            continue
        if domain_filter and domain_dir.name != domain_filter:
            continue
        for exp_dir in sorted(domain_dir.iterdir()):
            if not exp_dir.is_dir() or not (exp_dir / "config.cfg").exists():
                continue
            config = load_config(exp_dir)
            results = load_results(exp_dir)
            direction = config.get("metric_direction", "lower")
            stats = compute_stats(results, direction)
            best = f"`{stats['best']:.4f}`" if stats["best"] is not None else "—"
            pct = f"{stats['pct_change']:+.1f}%" if stats["pct_change"] is not None else "—"

            tsv = exp_dir / "results.tsv"
            status = "idle"
            if tsv.exists() and stats["total"] > 0:
                age_h = (time.time() - tsv.stat().st_mtime) / 3600
                status = "active" if age_h < 1 else "paused" if age_h < 24 else "done"

            lines.append(f"| {domain_dir.name} | {exp_dir.name} | {config.get('metric', '?')} | {stats['total']} | {stats['keeps']} | {best} | {pct} | {status} |")

    lines.append("")
    return "\n".join(lines)


# --- Main ---

def main():
    parser = argparse.ArgumentParser(description="autoresearch-agent results viewer")
    parser.add_argument("--experiment", help="Show one experiment: domain/name")
    parser.add_argument("--domain", help="Show all experiments in a domain")
    parser.add_argument("--dashboard", action="store_true", help="Cross-experiment dashboard")
    parser.add_argument("--format", choices=["terminal", "csv", "markdown"], default="terminal",
                        help="Output format (default: terminal)")
    parser.add_argument("--output", "-o", help="Write to file instead of stdout")
    parser.add_argument("--all", action="store_true", help="Show all experiments (alias for --dashboard)")
    args = parser.parse_args()

    root = find_autoresearch_root()
    if root is None:
        print("No .autoresearch/ found. Run setup_experiment.py first.")
        sys.exit(1)

    output_text = None

    # Single experiment
    if args.experiment:
        experiment_dir = root / args.experiment
        if not experiment_dir.exists():
            print(f"Experiment not found: {args.experiment}")
            sys.exit(1)

        if args.format == "csv":
            output_text = export_experiment_csv(experiment_dir, args.experiment)
        elif args.format == "markdown":
            output_text = export_experiment_markdown(experiment_dir, args.experiment)
        else:
            print_experiment(experiment_dir, args.experiment)
            return

    # Domain
    elif args.domain:
        domain_dir = root / args.domain
        if not domain_dir.exists():
            print(f"Domain not found: {args.domain}")
            sys.exit(1)
        for exp_dir in sorted(domain_dir.iterdir()):
            if exp_dir.is_dir() and (exp_dir / "config.cfg").exists():
                if args.format == "terminal":
                    print_experiment(exp_dir, f"{args.domain}/{exp_dir.name}")
                # For CSV/MD, fall through to dashboard with domain filter
        if args.format != "terminal":
            # Use dashboard export filtered to domain
            output_text = export_dashboard_csv(root, domain_filter=args.domain) if args.format == "csv" else export_dashboard_markdown(root, domain_filter=args.domain)
        else:
            return

    # Dashboard
    elif args.dashboard or args.all:
        if args.format == "csv":
            output_text = export_dashboard_csv(root)
        elif args.format == "markdown":
            output_text = export_dashboard_markdown(root)
        else:
            print_dashboard(root)
            return

    else:
        # Default: dashboard
        if args.format == "terminal":
            print_dashboard(root)
            return
        output_text = export_dashboard_csv(root) if args.format == "csv" else export_dashboard_markdown(root)

    # Write output
    if output_text:
        if args.output:
            Path(args.output).write_text(output_text)
            print(f"Written to {args.output}")
        else:
            print(output_text)


if __name__ == "__main__":
    main()

FILE:scripts/run_experiment.py
#!/usr/bin/env python3
"""
autoresearch-agent: Experiment Runner

Executes a single experiment iteration. The AI agent is the loop —
it calls this script repeatedly. The script handles evaluation,
metric parsing, keep/discard decisions, and git rollback on failure.

Usage:
    python scripts/run_experiment.py --experiment engineering/api-speed --single
    python scripts/run_experiment.py --experiment engineering/api-speed --dry-run
    python scripts/run_experiment.py --experiment engineering/api-speed --single --description "added caching"
"""

import argparse
import subprocess
import sys
import time
from datetime import datetime
from pathlib import Path


def find_autoresearch_root():
    """Find .autoresearch/ in project or user home."""
    project_root = Path(".").resolve() / ".autoresearch"
    if project_root.exists():
        return project_root
    user_root = Path.home() / ".autoresearch"
    if user_root.exists():
        return user_root
    return None


def load_config(experiment_dir):
    """Load config.cfg from experiment directory."""
    cfg_file = experiment_dir / "config.cfg"
    if not cfg_file.exists():
        print(f"  Error: no config.cfg in {experiment_dir}")
        sys.exit(1)
    config = {}
    for line in cfg_file.read_text().splitlines():
        if ":" in line:
            k, v = line.split(":", 1)
            config[k.strip()] = v.strip()
    return config


def run_git(args, cwd=None, timeout=30):
    """Run a git command safely (no shell injection). Returns (returncode, stdout, stderr)."""
    result = subprocess.run(
        ["git"] + args,
        capture_output=True, text=True,
        cwd=cwd, timeout=timeout
    )
    return result.returncode, result.stdout.strip(), result.stderr.strip()


def get_current_commit(path):
    """Get short hash of current HEAD."""
    _, commit, _ = run_git(["rev-parse", "--short", "HEAD"], cwd=path)
    return commit


def get_best_metric(experiment_dir, direction):
    """Read the best metric from results.tsv."""
    tsv = experiment_dir / "results.tsv"
    if not tsv.exists():
        return None
    lines = [l for l in tsv.read_text().splitlines()[1:] if "\tkeep\t" in l]
    if not lines:
        return None
    metrics = []
    for line in lines:
        parts = line.split("\t")
        try:
            if parts[1] != "N/A":
                metrics.append(float(parts[1]))
        except (ValueError, IndexError):
            continue
    if not metrics:
        return None
    return min(metrics) if direction == "lower" else max(metrics)


def run_evaluation(project_root, eval_cmd, time_budget_minutes, log_file):
    """Run evaluation with time limit. Output goes to log_file.

    Note: shell=True is intentional here — eval_cmd is user-provided and
    may contain pipes, redirects, or chained commands.
    """
    hard_limit = time_budget_minutes * 60 * 2.5
    t0 = time.time()
    try:
        with open(log_file, "w") as lf:
            result = subprocess.run(
                eval_cmd, shell=True,
                stdout=lf, stderr=subprocess.STDOUT,
                cwd=str(project_root),
                timeout=hard_limit
            )
        elapsed = time.time() - t0
        return result.returncode, elapsed
    except subprocess.TimeoutExpired:
        elapsed = time.time() - t0
        return -1, elapsed


def extract_metric(log_file, metric_grep):
    """Extract metric value from log file."""
    log_path = Path(log_file)
    if not log_path.exists():
        return None
    for line in reversed(log_path.read_text().splitlines()):
        stripped = line.strip()
        if stripped.startswith(metric_grep.lstrip("^")):
            try:
                return float(stripped.split(":")[-1].strip())
            except ValueError:
                continue
    return None


def is_improvement(new_val, old_val, direction):
    """Check if new result is better than old."""
    if old_val is None:
        return True
    if direction == "lower":
        return new_val < old_val
    return new_val > old_val


def log_result(experiment_dir, commit, metric_val, status, description):
    """Append result to results.tsv."""
    tsv = experiment_dir / "results.tsv"
    metric_str = f"{metric_val:.6f}" if metric_val is not None else "N/A"
    with open(tsv, "a") as f:
        f.write(f"{commit}\t{metric_str}\t{status}\t{description}\n")


def get_experiment_count(experiment_dir):
    """Count experiments run so far."""
    tsv = experiment_dir / "results.tsv"
    if not tsv.exists():
        return 0
    return max(0, len(tsv.read_text().splitlines()) - 1)


def get_description_from_diff(project_root):
    """Auto-generate a description from git diff --stat HEAD~1."""
    code, diff_stat, _ = run_git(["diff", "--stat", "HEAD~1"], cwd=str(project_root))
    if code == 0 and diff_stat:
        return diff_stat.split("\n")[0][:50]
    return "experiment"


def read_last_lines(filepath, n=5):
    """Read last n lines of a file (replaces tail shell command)."""
    path = Path(filepath)
    if not path.exists():
        return ""
    lines = path.read_text().splitlines()
    return "\n".join(lines[-n:])


def run_single(project_root, experiment_dir, config, exp_num, dry_run=False, description=None):
    """Run one experiment iteration."""
    direction = config.get("metric_direction", "lower")
    metric_grep = config.get("metric_grep", "^metric:")
    eval_cmd = config.get("evaluate_cmd", "python evaluate.py")
    time_budget = int(config.get("time_budget_minutes", 5))
    metric_name = config.get("metric", "metric")
    log_file = str(experiment_dir / "run.log")

    best = get_best_metric(experiment_dir, direction)
    ts = datetime.now().strftime("%H:%M:%S")

    print(f"\n[{ts}] Experiment #{exp_num}")
    print(f"  Best {metric_name}: {best}")

    if dry_run:
        print("  [DRY RUN] Would run evaluation and check metric")
        return "dry_run"

    # Auto-generate description if not provided
    if not description:
        description = get_description_from_diff(str(project_root))

    # Run evaluation
    print(f"  Running: {eval_cmd} (budget: {time_budget}m)")
    ret_code, elapsed = run_evaluation(project_root, eval_cmd, time_budget, log_file)

    commit = get_current_commit(str(project_root))

    # Timeout
    if ret_code == -1:
        print(f"  TIMEOUT after {elapsed:.0f}s — discarding")
        run_git(["checkout", "--", "."], cwd=str(project_root))
        run_git(["reset", "--hard", "HEAD~1"], cwd=str(project_root))
        log_result(experiment_dir, commit, None, "crash", f"timeout_{elapsed:.0f}s")
        return "crash"

    # Crash
    if ret_code != 0:
        tail = read_last_lines(log_file, 5)
        print(f"  CRASH (exit {ret_code}) after {elapsed:.0f}s")
        print(f"  Last output: {tail[:200]}")
        run_git(["reset", "--hard", "HEAD~1"], cwd=str(project_root))
        log_result(experiment_dir, commit, None, "crash", f"exit_{ret_code}")
        return "crash"

    # Extract metric
    metric_val = extract_metric(log_file, metric_grep)
    if metric_val is None:
        print(f"  Could not parse {metric_name} from run.log")
        run_git(["reset", "--hard", "HEAD~1"], cwd=str(project_root))
        log_result(experiment_dir, commit, None, "crash", "metric_parse_failed")
        return "crash"

    delta = ""
    if best is not None:
        diff = metric_val - best
        delta = f" (delta {diff:+.4f})"

    print(f"  {metric_name}: {metric_val:.6f}{delta} in {elapsed:.0f}s")

    # Keep or discard
    if is_improvement(metric_val, best, direction):
        print(f"  KEEP — improvement")
        log_result(experiment_dir, commit, metric_val, "keep", description)
        return "keep"
    else:
        print(f"  DISCARD — no improvement")
        run_git(["reset", "--hard", "HEAD~1"], cwd=str(project_root))
        best_str = f"{best:.4f}" if best is not None else "?"
        log_result(experiment_dir, commit, metric_val, "discard",
                   f"no_improvement_{metric_val:.4f}_vs_{best_str}")
        return "discard"


def main():
    parser = argparse.ArgumentParser(description="autoresearch-agent runner")
    parser.add_argument("--experiment", help="Experiment path: domain/name (e.g. engineering/api-speed)")
    parser.add_argument("--single", action="store_true", help="Run one experiment iteration")
    parser.add_argument("--dry-run", action="store_true", help="Show what would happen")
    parser.add_argument("--description", help="Description of the change (auto-generated from git diff if omitted)")
    parser.add_argument("--path", default=".", help="Project root")
    args = parser.parse_args()

    project_root = Path(args.path).resolve()
    root = find_autoresearch_root()

    if root is None:
        print("No .autoresearch/ found. Run setup_experiment.py first.")
        sys.exit(1)

    if not args.experiment:
        print("Specify --experiment domain/name")
        sys.exit(1)

    experiment_dir = root / args.experiment
    if not experiment_dir.exists():
        print(f"Experiment not found: {experiment_dir}")
        print("Run: python scripts/setup_experiment.py --list")
        sys.exit(1)

    config = load_config(experiment_dir)

    print(f"\n  autoresearch-agent")
    print(f"  Experiment: {args.experiment}")
    print(f"  Target: {config.get('target', '?')}")
    print(f"  Metric: {config.get('metric', '?')} ({config.get('metric_direction', '?')} is better)")
    print(f"  Budget: {config.get('time_budget_minutes', '?')} min/experiment")
    print(f"  Mode: {'dry-run' if args.dry_run else 'single'}")

    exp_num = get_experiment_count(experiment_dir) + 1
    run_single(project_root, experiment_dir, config, exp_num, args.dry_run, args.description)


if __name__ == "__main__":
    main()

FILE:scripts/setup_experiment.py
#!/usr/bin/env python3
"""
autoresearch-agent: Setup Experiment

Initialize a new experiment with domain, target, evaluator, and git branch.
Creates the .autoresearch/{domain}/{name}/ directory structure.

Usage:
    python scripts/setup_experiment.py --domain engineering --name api-speed \
        --target src/api/search.py --eval "pytest bench.py" \
        --metric p50_ms --direction lower

    python scripts/setup_experiment.py --domain marketing --name medium-ctr \
        --target content/titles.md --eval "python evaluate.py" \
        --metric ctr_score --direction higher --evaluator llm_judge_content

    python scripts/setup_experiment.py --list          # List all experiments
    python scripts/setup_experiment.py --list-evaluators  # List available evaluators
"""

import argparse
import shutil
import subprocess
import sys
from datetime import datetime
from pathlib import Path

DOMAINS = ["engineering", "marketing", "content", "prompts", "custom"]

EVALUATOR_DIR = Path(__file__).parent.parent / "evaluators"

DEFAULT_CONFIG = """# autoresearch global config
default_time_budget_minutes: 5
default_scope: project
dashboard_format: markdown
"""

GITIGNORE_CONTENT = """# autoresearch — experiment logs are local state
**/results.tsv
**/run.log
**/run.*.log
config.yaml
"""


def run_cmd(cmd, cwd=None, timeout=None):
    """Run shell command, return (returncode, stdout, stderr)."""
    result = subprocess.run(
        cmd, shell=True, capture_output=True, text=True,
        cwd=cwd, timeout=timeout
    )
    return result.returncode, result.stdout.strip(), result.stderr.strip()


def get_autoresearch_root(scope, project_root=None):
    """Get the .autoresearch root directory based on scope."""
    if scope == "user":
        return Path.home() / ".autoresearch"
    return Path(project_root or ".") / ".autoresearch"


def init_root(root):
    """Initialize .autoresearch root if it doesn't exist."""
    created = False
    if not root.exists():
        root.mkdir(parents=True)
        created = True
        print(f"  Created {root}/")

    config_file = root / "config.yaml"
    if not config_file.exists():
        config_file.write_text(DEFAULT_CONFIG)
        print(f"  Created {config_file}")

    gitignore = root / ".gitignore"
    if not gitignore.exists():
        gitignore.write_text(GITIGNORE_CONTENT)
        print(f"  Created {gitignore}")

    return created


def create_program_md(experiment_dir, domain, name, target, metric, direction, constraints=""):
    """Generate a program.md template for the experiment."""
    direction_word = "Minimize" if direction == "lower" else "Maximize"
    content = f"""# autoresearch — {name}

## Goal
{direction_word} `{metric}` on `{target}`. {"Lower" if direction == "lower" else "Higher"} is better.

## What the Agent Can Change
- Only `{target}` — this is the single file being optimized.
- Everything inside that file is fair game unless constrained below.

## What the Agent Cannot Change
- The evaluation script (`evaluate.py` or the eval command). It is read-only.
- Dependencies — do not add new packages or imports that aren't already available.
- Any other files in the project unless explicitly noted here.
{f"- Additional constraints: {constraints}" if constraints else ""}

## Strategy
1. First run: establish baseline. Do not change anything.
2. Profile/analyze the current state — understand why the metric is what it is.
3. Try the most obvious improvement first (low-hanging fruit).
4. If that works, push further in the same direction.
5. If stuck, try something orthogonal or radical.
6. Read the git log of previous experiments. Don't repeat failed approaches.

## Simplicity Rule
A small improvement that adds ugly complexity is NOT worth it.
Equal performance with simpler code IS worth it.
Removing code that gets same results is the best outcome.

## Stop When
You don't stop. The human will interrupt you when they're satisfied.
If no improvement in 20+ consecutive runs, change strategy drastically.
"""
    (experiment_dir / "program.md").write_text(content)


def create_config(experiment_dir, target, eval_cmd, metric, direction, time_budget):
    """Write experiment config."""
    content = f"""target: {target}
evaluate_cmd: {eval_cmd}
metric: {metric}
metric_direction: {direction}
metric_grep: ^{metric}:
time_budget_minutes: {time_budget}
created: {datetime.now().strftime('%Y-%m-%d %H:%M')}
"""
    (experiment_dir / "config.cfg").write_text(content)


def init_results_tsv(experiment_dir):
    """Create results.tsv with header."""
    tsv = experiment_dir / "results.tsv"
    if tsv.exists():
        print(f"  results.tsv already exists ({tsv.stat().st_size} bytes)")
        return
    tsv.write_text("commit\tmetric\tstatus\tdescription\n")
    print("  Created results.tsv")


def copy_evaluator(experiment_dir, evaluator_name):
    """Copy a built-in evaluator to the experiment directory."""
    source = EVALUATOR_DIR / f"{evaluator_name}.py"
    if not source.exists():
        print(f"  Warning: evaluator '{evaluator_name}' not found in {EVALUATOR_DIR}")
        print(f"  Available: {', '.join(f.stem for f in EVALUATOR_DIR.glob('*.py'))}")
        return False
    dest = experiment_dir / "evaluate.py"
    shutil.copy2(source, dest)
    print(f"  Copied evaluator: {evaluator_name}.py -> evaluate.py")
    return True


def create_branch(path, domain, name):
    """Create and checkout the experiment branch."""
    branch = f"autoresearch/{domain}/{name}"
    result = subprocess.run(
        ["git", "checkout", "-b", branch],
        cwd=path, capture_output=True, text=True
    )
    if result.returncode != 0:
        if "already exists" in result.stderr:
            print(f"  Branch '{branch}' already exists. Checking out...")
            subprocess.run(
                ["git", "checkout", branch],
                cwd=path, capture_output=True, text=True
            )
            return branch
        print(f"  Warning: could not create branch: {result.stderr}")
        return None
    print(f"  Created branch: {branch}")
    return branch


def list_experiments(root):
    """List all experiments across all domains."""
    if not root.exists():
        print("No experiments found. Run setup to create your first experiment.")
        return

    experiments = []
    for domain_dir in sorted(root.iterdir()):
        if not domain_dir.is_dir() or domain_dir.name.startswith("."):
            continue
        for exp_dir in sorted(domain_dir.iterdir()):
            if not exp_dir.is_dir():
                continue
            cfg_file = exp_dir / "config.cfg"
            if not cfg_file.exists():
                continue
            config = {}
            for line in cfg_file.read_text().splitlines():
                if ":" in line:
                    k, v = line.split(":", 1)
                    config[k.strip()] = v.strip()

            # Count results
            tsv = exp_dir / "results.tsv"
            runs = 0
            if tsv.exists():
                runs = max(0, len(tsv.read_text().splitlines()) - 1)

            experiments.append({
                "domain": domain_dir.name,
                "name": exp_dir.name,
                "target": config.get("target", "?"),
                "metric": config.get("metric", "?"),
                "runs": runs,
            })

    if not experiments:
        print("No experiments found.")
        return

    print(f"\n{'DOMAIN':<15} {'EXPERIMENT':<25} {'TARGET':<30} {'METRIC':<15} {'RUNS':>5}")
    print("-" * 95)
    for e in experiments:
        print(f"{e['domain']:<15} {e['name']:<25} {e['target']:<30} {e['metric']:<15} {e['runs']:>5}")
    print(f"\nTotal: {len(experiments)} experiments")


def list_evaluators():
    """List available built-in evaluators."""
    if not EVALUATOR_DIR.exists():
        print("No evaluators directory found.")
        return

    print(f"\nAvailable evaluators ({EVALUATOR_DIR}):\n")
    for f in sorted(EVALUATOR_DIR.glob("*.py")):
        # Read first docstring line
        desc = ""
        for line in f.read_text().splitlines():
            stripped = line.strip()
            if stripped.startswith('"""') or stripped.startswith("'''"):
                quote = stripped[:3]
                # Single-line docstring: """Description."""
                after_quote = stripped[3:]
                if after_quote and after_quote.rstrip(quote[0]).strip():
                    desc = after_quote.rstrip('"').rstrip("'").strip()
                    break
                continue
            if stripped and not line.startswith("#!"):
                desc = stripped.strip('"').strip("'")
                break
        print(f"  {f.stem:<25} {desc}")


def main():
    parser = argparse.ArgumentParser(description="autoresearch-agent setup")
    parser.add_argument("--domain", choices=DOMAINS, help="Experiment domain")
    parser.add_argument("--name", help="Experiment name (e.g. api-speed, medium-ctr)")
    parser.add_argument("--target", help="Target file to optimize")
    parser.add_argument("--eval", dest="eval_cmd", help="Evaluation command")
    parser.add_argument("--metric", help="Metric name (must appear in eval output as 'name: value')")
    parser.add_argument("--direction", choices=["lower", "higher"], default="lower",
                        help="Is lower or higher better?")
    parser.add_argument("--time-budget", type=int, default=5, help="Minutes per experiment (default: 5)")
    parser.add_argument("--evaluator", help="Built-in evaluator to copy (e.g. benchmark_speed)")
    parser.add_argument("--scope", choices=["project", "user"], default="project",
                        help="Where to store experiments: project (./) or user (~/)")
    parser.add_argument("--constraints", default="", help="Additional constraints for program.md")
    parser.add_argument("--path", default=".", help="Project root path")
    parser.add_argument("--skip-branch", action="store_true", help="Don't create git branch")
    parser.add_argument("--list", action="store_true", help="List all experiments")
    parser.add_argument("--list-evaluators", action="store_true", help="List available evaluators")
    args = parser.parse_args()

    project_root = Path(args.path).resolve()

    # List mode
    if args.list:
        root = get_autoresearch_root("project", project_root)
        list_experiments(root)
        user_root = get_autoresearch_root("user")
        if user_root.exists() and user_root != root:
            print(f"\n--- User-level experiments ({user_root}) ---")
            list_experiments(user_root)
        return

    if args.list_evaluators:
        list_evaluators()
        return

    # Validate required args for setup
    if not all([args.domain, args.name, args.target, args.eval_cmd, args.metric]):
        parser.error("Required: --domain, --name, --target, --eval, --metric")

    root = get_autoresearch_root(args.scope, project_root)

    print(f"\n  autoresearch-agent setup")
    print(f"  Project: {project_root}")
    print(f"  Scope: {args.scope}")
    print(f"  Domain: {args.domain}")
    print(f"  Experiment: {args.name}")
    print(f"  Time: {datetime.now().strftime('%Y-%m-%d %H:%M')}\n")

    # Check git
    result = subprocess.run(
        ["git", "rev-parse", "--is-inside-work-tree"],
        cwd=str(project_root), capture_output=True, text=True
    )
    code = result.returncode
    if code != 0:
        print("  Error: not a git repository. Run: git init && git add . && git commit -m 'initial'")
        sys.exit(1)
    print("  Git repository found")

    # Check target file
    target_path = project_root / args.target
    if not target_path.exists():
        print(f"  Error: target file not found: {args.target}")
        sys.exit(1)
    print(f"  Target file found: {args.target}")

    # Init root
    init_root(root)

    # Create experiment directory
    experiment_dir = root / args.domain / args.name
    if experiment_dir.exists():
        print(f"  Warning: experiment '{args.domain}/{args.name}' already exists.")
        print(f"  Use --name with a different name, or delete {experiment_dir}")
        sys.exit(1)
    experiment_dir.mkdir(parents=True)
    print(f"  Created {experiment_dir}/")

    # Create files
    create_program_md(experiment_dir, args.domain, args.name,
                      args.target, args.metric, args.direction, args.constraints)
    print("  Created program.md")

    create_config(experiment_dir, args.target, args.eval_cmd,
                  args.metric, args.direction, args.time_budget)
    print("  Created config.cfg")

    init_results_tsv(experiment_dir)

    # Copy evaluator if specified
    if args.evaluator:
        copy_evaluator(experiment_dir, args.evaluator)

    # Create git branch
    if not args.skip_branch:
        create_branch(str(project_root), args.domain, args.name)

    # Test evaluation command
    print(f"\n  Testing evaluation: {args.eval_cmd}")
    code, out, err = run_cmd(args.eval_cmd, cwd=str(project_root), timeout=60)
    if code != 0:
        print(f"  Warning: eval command failed (exit {code})")
        if err:
            print(f"  stderr: {err[:200]}")
        print("  Fix the eval command before running the experiment loop.")
    else:
        # Check metric is parseable
        full_output = out + "\n" + err
        metric_found = False
        for line in full_output.splitlines():
            if line.strip().startswith(f"{args.metric}:"):
                metric_found = True
                print(f"  Eval works. Baseline: {line.strip()}")
                break
        if not metric_found:
            print(f"  Warning: eval ran but '{args.metric}:' not found in output.")
            print(f"  Make sure your eval command outputs: {args.metric}: <value>")

    # Summary
    print(f"\n  Setup complete!")
    print(f"  Experiment: {args.domain}/{args.name}")
    print(f"  Target: {args.target}")
    print(f"  Metric: {args.metric} ({args.direction} is better)")
    print(f"  Budget: {args.time_budget} min/experiment")
    if not args.skip_branch:
        print(f"  Branch: autoresearch/{args.domain}/{args.name}")
    print(f"\n  To start:")
    print(f"  python scripts/run_experiment.py --experiment {args.domain}/{args.name} --single")


if __name__ == "__main__":
    main()

---
name: "aws-solution-architect"
description: Design AWS architectures for startups using serverless patterns and IaC templates. Use when asked to design serverless architecture, create CloudFormation templates, optimize AWS costs, set up CI/CD pipelines, or migrate to AWS. Covers Lambda, API Gateway, DynamoDB, ECS, Aurora, and cost optimization.
---

# AWS Solution Architect

Design scalable, cost-effective AWS architectures for startups with infrastructure-as-code templates.

---

## Workflow

### Step 1: Gather Requirements

Collect application specifications:

```
- Application type (web app, mobile backend, data pipeline, SaaS)
- Expected users and requests per second
- Budget constraints (monthly spend limit)
- Team size and AWS experience level
- Compliance requirements (GDPR, HIPAA, SOC 2)
- Availability requirements (SLA, RPO/RTO)
```

### Step 2: Design Architecture

Run the architecture designer to get pattern recommendations:

```bash
python scripts/architecture_designer.py --input requirements.json
```

**Example output:**

```json
{
  "recommended_pattern": "serverless_web",
  "service_stack": ["S3", "CloudFront", "API Gateway", "Lambda", "DynamoDB", "Cognito"],
  "estimated_monthly_cost_usd": 35,
  "pros": ["Low ops overhead", "Pay-per-use", "Auto-scaling"],
  "cons": ["Cold starts", "15-min Lambda limit", "Eventual consistency"]
}
```

Select from recommended patterns:
- **Serverless Web**: S3 + CloudFront + API Gateway + Lambda + DynamoDB
- **Event-Driven Microservices**: EventBridge + Lambda + SQS + Step Functions
- **Three-Tier**: ALB + ECS Fargate + Aurora + ElastiCache
- **GraphQL Backend**: AppSync + Lambda + DynamoDB + Cognito

See `references/architecture_patterns.md` for detailed pattern specifications.

**Validation checkpoint:** Confirm the recommended pattern matches the team's operational maturity and compliance requirements before proceeding to Step 3.

### Step 3: Generate IaC Templates

Create infrastructure-as-code for the selected pattern:

```bash
# Serverless stack (CloudFormation)
python scripts/serverless_stack.py --app-name my-app --region us-east-1
```

**Example CloudFormation YAML output (core serverless resources):**

```yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Parameters:
  AppName:
    Type: String
    Default: my-app

Resources:
  ApiFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.handler
      Runtime: nodejs20.x
      MemorySize: 512
      Timeout: 30
      Environment:
        Variables:
          TABLE_NAME: !Ref DataTable
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref DataTable
      Events:
        ApiEvent:
          Type: Api
          Properties:
            Path: /{proxy+}
            Method: ANY

  DataTable:
    Type: AWS::DynamoDB::Table
    Properties:
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: pk
          AttributeType: S
        - AttributeName: sk
          AttributeType: S
      KeySchema:
        - AttributeName: pk
          KeyType: HASH
        - AttributeName: sk
          KeyType: RANGE
```

> Full templates including API Gateway, Cognito, IAM roles, and CloudWatch logging are generated by `serverless_stack.py` and also available in `references/architecture_patterns.md`.

**Example CDK TypeScript snippet (three-tier pattern):**

```typescript
import * as ecs from 'aws-cdk-lib/aws-ecs';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as rds from 'aws-cdk-lib/aws-rds';

const vpc = new ec2.Vpc(this, 'AppVpc', { maxAzs: 2 });

const cluster = new ecs.Cluster(this, 'AppCluster', { vpc });

const db = new rds.ServerlessCluster(this, 'AppDb', {
  engine: rds.DatabaseClusterEngine.auroraPostgres({
    version: rds.AuroraPostgresEngineVersion.VER_15_2,
  }),
  vpc,
  scaling: { minCapacity: 0.5, maxCapacity: 4 },
});
```

### Step 4: Review Costs

Analyze estimated costs and optimization opportunities:

```bash
python scripts/cost_optimizer.py --resources current_setup.json --monthly-spend 2000
```

**Example output:**

```json
{
  "current_monthly_usd": 2000,
  "recommendations": [
    { "action": "Right-size RDS db.r5.2xlarge → db.r5.large", "savings_usd": 420, "priority": "high" },
    { "action": "Purchase 1-yr Compute Savings Plan at 40% utilization", "savings_usd": 310, "priority": "high" },
    { "action": "Move S3 objects >90 days to Glacier Instant Retrieval", "savings_usd": 85, "priority": "medium" }
  ],
  "total_potential_savings_usd": 815
}
```

Output includes:
- Monthly cost breakdown by service
- Right-sizing recommendations
- Savings Plans opportunities
- Potential monthly savings

### Step 5: Deploy

Deploy the generated infrastructure:

```bash
# CloudFormation
aws cloudformation create-stack \
  --stack-name my-app-stack \
  --template-body file://template.yaml \
  --capabilities CAPABILITY_IAM

# CDK
cdk deploy

# Terraform
terraform init && terraform apply
```

### Step 6: Validate and Handle Failures

Verify deployment and set up monitoring:

```bash
# Check stack status
aws cloudformation describe-stacks --stack-name my-app-stack

# Set up CloudWatch alarms
aws cloudwatch put-metric-alarm --alarm-name high-errors ...
```

**If stack creation fails:**

1. Check the failure reason:
   ```bash
   aws cloudformation describe-stack-events \
     --stack-name my-app-stack \
     --query 'StackEvents[?ResourceStatus==`CREATE_FAILED`]'
   ```
2. Review CloudWatch Logs for Lambda or ECS errors.
3. Fix the template or resource configuration.
4. Delete the failed stack before retrying:
   ```bash
   aws cloudformation delete-stack --stack-name my-app-stack
   # Wait for deletion
   aws cloudformation wait stack-delete-complete --stack-name my-app-stack
   # Redeploy
   aws cloudformation create-stack ...
   ```

**Common failure causes:**
- IAM permission errors → verify `--capabilities CAPABILITY_IAM` and role trust policies
- Resource limit exceeded → request quota increase via Service Quotas console
- Invalid template syntax → run `aws cloudformation validate-template --template-body file://template.yaml` before deploying

---

## Tools

### architecture_designer.py

Generates architecture patterns based on requirements.

```bash
python scripts/architecture_designer.py --input requirements.json --output design.json
```

**Input:** JSON with app type, scale, budget, compliance needs
**Output:** Recommended pattern, service stack, cost estimate, pros/cons

### serverless_stack.py

Creates serverless CloudFormation templates.

```bash
python scripts/serverless_stack.py --app-name my-app --region us-east-1
```

**Output:** Production-ready CloudFormation YAML with:
- API Gateway + Lambda
- DynamoDB table
- Cognito user pool
- IAM roles with least privilege
- CloudWatch logging

### cost_optimizer.py

Analyzes costs and recommends optimizations.

```bash
python scripts/cost_optimizer.py --resources inventory.json --monthly-spend 5000
```

**Output:** Recommendations for:
- Idle resource removal
- Instance right-sizing
- Reserved capacity purchases
- Storage tier transitions
- NAT Gateway alternatives

---

## Quick Start

### MVP Architecture (< $100/month)

```
Ask: "Design a serverless MVP backend for a mobile app with 1000 users"

Result:
- Lambda + API Gateway for API
- DynamoDB pay-per-request for data
- Cognito for authentication
- S3 + CloudFront for static assets
- Estimated: $20-50/month
```

### Scaling Architecture ($500-2000/month)

```
Ask: "Design a scalable architecture for a SaaS platform with 50k users"

Result:
- ECS Fargate for containerized API
- Aurora Serverless for relational data
- ElastiCache for session caching
- CloudFront for CDN
- CodePipeline for CI/CD
- Multi-AZ deployment
```

### Cost Optimization

```
Ask: "Optimize my AWS setup to reduce costs by 30%. Current spend: $3000/month"

Provide: Current resource inventory (EC2, RDS, S3, etc.)

Result:
- Idle resource identification
- Right-sizing recommendations
- Savings Plans analysis
- Storage lifecycle policies
- Target savings: $900/month
```

### IaC Generation

```
Ask: "Generate CloudFormation for a three-tier web app with auto-scaling"

Result:
- VPC with public/private subnets
- ALB with HTTPS
- ECS Fargate with auto-scaling
- Aurora with read replicas
- Security groups and IAM roles
```

---

## Input Requirements

Provide these details for architecture design:

| Requirement | Description | Example |
|-------------|-------------|---------|
| Application type | What you're building | SaaS platform, mobile backend |
| Expected scale | Users, requests/sec | 10k users, 100 RPS |
| Budget | Monthly AWS limit | $500/month max |
| Team context | Size, AWS experience | 3 devs, intermediate |
| Compliance | Regulatory needs | HIPAA, GDPR, SOC 2 |
| Availability | Uptime requirements | 99.9% SLA, 1hr RPO |

**JSON Format:**

```json
{
  "application_type": "saas_platform",
  "expected_users": 10000,
  "requests_per_second": 100,
  "budget_monthly_usd": 500,
  "team_size": 3,
  "aws_experience": "intermediate",
  "compliance": ["SOC2"],
  "availability_sla": "99.9%"
}
```

---

## Output Formats

### Architecture Design

- Pattern recommendation with rationale
- Service stack diagram (ASCII)
- Monthly cost estimate and trade-offs

### IaC Templates

- **CloudFormation YAML**: Production-ready SAM/CFN templates
- **CDK TypeScript**: Type-safe infrastructure code
- **Terraform HCL**: Multi-cloud compatible configs

### Cost Analysis

- Current spend breakdown with optimization recommendations
- Priority action list (high/medium/low) and implementation checklist

---

## Reference Documentation

| Document | Contents |
|----------|----------|
| `references/architecture_patterns.md` | 6 patterns: serverless, microservices, three-tier, data processing, GraphQL, multi-region |
| `references/service_selection.md` | Decision matrices for compute, database, storage, messaging |
| `references/best_practices.md` | Serverless design, cost optimization, security hardening, scalability |

FILE:assets/expected_output.json
{
  "recommended_architecture": {
    "pattern_name": "Modern Three-Tier Application",
    "description": "Classic architecture with containers and managed services",
    "estimated_monthly_cost": 1450,
    "scaling_characteristics": {
      "users_supported": "10k - 500k",
      "requests_per_second": "1,000 - 50,000"
    }
  },
  "services": {
    "load_balancer": "Application Load Balancer (ALB)",
    "compute": "ECS Fargate",
    "database": "RDS Aurora (MySQL/PostgreSQL)",
    "cache": "ElastiCache Redis",
    "cdn": "CloudFront",
    "storage": "S3",
    "authentication": "Cognito"
  },
  "cost_breakdown": {
    "ALB": "20-30 USD",
    "ECS_Fargate": "50-200 USD",
    "RDS_Aurora": "100-300 USD",
    "ElastiCache": "30-80 USD",
    "CloudFront": "10-50 USD",
    "S3": "10-30 USD"
  },
  "implementation_phases": [
    {
      "phase": "Foundation",
      "duration": "1 week",
      "tasks": ["VPC setup", "IAM roles", "CloudTrail", "AWS Config"]
    },
    {
      "phase": "Core Services",
      "duration": "2 weeks",
      "tasks": ["Deploy ALB", "ECS Fargate", "RDS Aurora", "ElastiCache"]
    },
    {
      "phase": "Security & Monitoring",
      "duration": "1 week",
      "tasks": ["WAF rules", "CloudWatch dashboards", "Alarms", "X-Ray"]
    },
    {
      "phase": "CI/CD",
      "duration": "1 week",
      "tasks": ["CodePipeline", "Blue/Green deployment", "Rollback procedures"]
    }
  ],
  "iac_templates_generated": [
    "CloudFormation template (YAML)",
    "AWS CDK stack (TypeScript)",
    "Terraform configuration (HCL)"
  ]
}

FILE:assets/sample_input.json
{
  "application_type": "saas_platform",
  "expected_users": 50000,
  "requests_per_second": 100,
  "budget_monthly_usd": 1500,
  "team_size": 5,
  "aws_experience": "intermediate",
  "compliance": ["GDPR"],
  "data_size_gb": 500,
  "region": "us-east-1",
  "requirements": {
    "authentication": true,
    "real_time_features": false,
    "multi_region": false,
    "high_availability": true,
    "auto_scaling": true
  }
}

FILE:references/architecture_patterns.md
# AWS Architecture Patterns for Startups

Reference guide for selecting the right AWS architecture pattern based on application requirements.

---

## Table of Contents

- [Pattern Selection Matrix](#pattern-selection-matrix)
- [Pattern 1: Serverless Web Application](#pattern-1-serverless-web-application)
- [Pattern 2: Event-Driven Microservices](#pattern-2-event-driven-microservices)
- [Pattern 3: Modern Three-Tier Application](#pattern-3-modern-three-tier-application)
- [Pattern 4: Real-Time Data Processing](#pattern-4-real-time-data-processing)
- [Pattern 5: GraphQL API Backend](#pattern-5-graphql-api-backend)
- [Pattern 6: Multi-Region High Availability](#pattern-6-multi-region-high-availability)

---

## Pattern Selection Matrix

| Pattern | Best For | Users | Monthly Cost | Complexity |
|---------|----------|-------|--------------|------------|
| Serverless Web | MVP, SaaS, mobile backend | <50K | $50-500 | Low |
| Event-Driven Microservices | Complex workflows, async processing | Any | $100-1000 | Medium |
| Three-Tier | Traditional web, e-commerce | 10K-500K | $300-2000 | Medium |
| Real-Time Data | Analytics, IoT, streaming | Any | $200-1500 | High |
| GraphQL Backend | Mobile apps, SPAs | <100K | $50-400 | Medium |
| Multi-Region HA | Global apps, DR requirements | >100K | 1.5-2x single | High |

---

## Pattern 1: Serverless Web Application

### Use Case
SaaS platforms, mobile backends, low-traffic websites, MVPs

### Architecture Diagram

```
┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│  CloudFront │────▶│     S3      │     │   Cognito   │
│    (CDN)    │     │  (Static)   │     │   (Auth)    │
└─────────────┘     └─────────────┘     └──────┬──────┘
                                               │
┌─────────────┐     ┌─────────────┐     ┌──────▼──────┐
│   Route 53  │────▶│ API Gateway │────▶│   Lambda    │
│    (DNS)    │     │   (REST)    │     │ (Functions) │
└─────────────┘     └─────────────┘     └──────┬──────┘
                                               │
                                        ┌──────▼──────┐
                                        │  DynamoDB   │
                                        │ (Database)  │
                                        └─────────────┘
```

### Service Stack

| Layer | Service | Configuration |
|-------|---------|---------------|
| Frontend | S3 + CloudFront | Static hosting with HTTPS |
| API | API Gateway + Lambda | REST endpoints with throttling |
| Database | DynamoDB | Pay-per-request billing |
| Auth | Cognito | User pools with MFA support |
| CI/CD | Amplify or CodePipeline | Automated deployments |

### CloudFormation Template

```yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Resources:
  # API Function
  ApiFunction:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: nodejs18.x
      Handler: index.handler
      MemorySize: 512
      Timeout: 10
      Events:
        Api:
          Type: Api
          Properties:
            Path: /{proxy+}
            Method: ANY

  # DynamoDB Table
  DataTable:
    Type: AWS::DynamoDB::Table
    Properties:
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: PK
          AttributeType: S
        - AttributeName: SK
          AttributeType: S
      KeySchema:
        - AttributeName: PK
          KeyType: HASH
        - AttributeName: SK
          KeyType: RANGE
```

### Cost Breakdown (10K users)

| Service | Monthly Cost |
|---------|-------------|
| Lambda | $5-20 |
| API Gateway | $10-30 |
| DynamoDB | $10-50 |
| CloudFront | $5-15 |
| S3 | $1-5 |
| Cognito | $0-50 |
| **Total** | **$31-170** |

### Pros and Cons

**Pros:**
- Zero server management
- Pay only for what you use
- Auto-scaling built-in
- Low operational overhead

**Cons:**
- Cold start latency (100-500ms)
- 15-minute Lambda execution limit
- Vendor lock-in

---

## Pattern 2: Event-Driven Microservices

### Use Case
Complex business workflows, asynchronous processing, decoupled systems

### Architecture Diagram

```
┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Service   │────▶│ EventBridge │────▶│   Service   │
│      A      │     │  (Event Bus)│     │      B      │
└─────────────┘     └──────┬──────┘     └─────────────┘
                           │
                    ┌──────▼──────┐
                    │     SQS     │
                    │   (Queue)   │
                    └──────┬──────┘
                           │
┌─────────────┐     ┌──────▼──────┐     ┌─────────────┐
│    Step     │◀────│   Lambda    │────▶│  DynamoDB   │
│  Functions  │     │ (Processor) │     │  (Storage)  │
└─────────────┘     └─────────────┘     └─────────────┘
```

### Service Stack

| Layer | Service | Purpose |
|-------|---------|---------|
| Events | EventBridge | Central event bus |
| Processing | Lambda or ECS Fargate | Event handlers |
| Queue | SQS | Dead letter queue for failures |
| Orchestration | Step Functions | Complex workflow state |
| Storage | DynamoDB, S3 | Persistent data |

### Event Schema Example

```json
{
  "source": "orders.service",
  "detail-type": "OrderCreated",
  "detail": {
    "orderId": "ord-12345",
    "customerId": "cust-67890",
    "items": [...],
    "total": 99.99,
    "timestamp": "2024-01-15T10:30:00Z"
  }
}
```

### Cost Breakdown

| Service | Monthly Cost |
|---------|-------------|
| EventBridge | $1-10 |
| Lambda | $20-100 |
| SQS | $5-20 |
| Step Functions | $25-100 |
| DynamoDB | $20-100 |
| **Total** | **$71-330** |

### Pros and Cons

**Pros:**
- Loose coupling between services
- Independent scaling per service
- Failure isolation
- Easy to test individually

**Cons:**
- Distributed system complexity
- Eventual consistency
- Harder to debug

---

## Pattern 3: Modern Three-Tier Application

### Use Case
Traditional web apps, e-commerce, CMS, applications with complex queries

### Architecture Diagram

```
┌─────────────┐     ┌─────────────┐
│  CloudFront │────▶│     ALB     │
│    (CDN)    │     │ (Load Bal.) │
└─────────────┘     └──────┬──────┘
                           │
                    ┌──────▼──────┐
                    │ ECS Fargate │
                    │ (Auto-scale)│
                    └──────┬──────┘
                           │
        ┌──────────────────┼──────────────────┐
        │                  │                  │
 ┌──────▼──────┐    ┌──────▼──────┐    ┌──────▼──────┐
 │   Aurora    │    │ ElastiCache │    │     S3      │
 │ (Database)  │    │   (Redis)   │    │  (Storage)  │
 └─────────────┘    └─────────────┘    └─────────────┘
```

### Service Stack

| Layer | Service | Configuration |
|-------|---------|---------------|
| CDN | CloudFront | Edge caching, HTTPS |
| Load Balancer | ALB | Path-based routing, health checks |
| Compute | ECS Fargate | Container auto-scaling |
| Database | Aurora MySQL/PostgreSQL | Multi-AZ, auto-scaling |
| Cache | ElastiCache Redis | Session, query caching |
| Storage | S3 | Static assets, uploads |

### Terraform Example

```hcl
# ECS Service with Auto-scaling
resource "aws_ecs_service" "app" {
  name            = "app-service"
  cluster         = aws_ecs_cluster.main.id
  task_definition = aws_ecs_task_definition.app.arn
  desired_count   = 2

  capacity_provider_strategy {
    capacity_provider = "FARGATE"
    weight            = 100
  }

  load_balancer {
    target_group_arn = aws_lb_target_group.app.arn
    container_name   = "app"
    container_port   = 3000
  }
}

# Auto-scaling Policy
resource "aws_appautoscaling_target" "app" {
  max_capacity       = 10
  min_capacity       = 2
  resource_id        = "service/aws_ecs_cluster.main.name/aws_ecs_service.app.name"
  scalable_dimension = "ecs:service:DesiredCount"
  service_namespace  = "ecs"
}
```

### Cost Breakdown (50K users)

| Service | Monthly Cost |
|---------|-------------|
| ECS Fargate (2 tasks) | $100-200 |
| ALB | $25-50 |
| Aurora | $100-300 |
| ElastiCache | $50-100 |
| CloudFront | $20-50 |
| **Total** | **$295-700** |

---

## Pattern 4: Real-Time Data Processing

### Use Case
Analytics, IoT data ingestion, log processing, streaming data

### Architecture Diagram

```
┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│  IoT Core   │────▶│   Kinesis   │────▶│   Lambda    │
│  (Devices)  │     │  (Stream)   │     │ (Process)   │
└─────────────┘     └─────────────┘     └──────┬──────┘
                                               │
┌─────────────┐     ┌─────────────┐     ┌──────▼──────┐
│  QuickSight │◀────│   Athena    │◀────│     S3      │
│   (Viz)     │     │  (Query)    │     │ (Data Lake) │
└─────────────┘     └─────────────┘     └─────────────┘
                                               │
                                        ┌──────▼──────┐
                                        │  CloudWatch │
                                        │  (Alerts)   │
                                        └─────────────┘
```

### Service Stack

| Layer | Service | Purpose |
|-------|---------|---------|
| Ingestion | Kinesis Data Streams | Real-time data capture |
| Processing | Lambda or Kinesis Analytics | Transform and analyze |
| Storage | S3 (data lake) | Long-term storage |
| Query | Athena | SQL queries on S3 |
| Visualization | QuickSight | Dashboards and reports |
| Alerting | CloudWatch + SNS | Threshold-based alerts |

### Kinesis Producer Example

```python
import boto3
import json

kinesis = boto3.client('kinesis')

def send_event(stream_name, data, partition_key):
    response = kinesis.put_record(
        StreamName=stream_name,
        Data=json.dumps(data),
        PartitionKey=partition_key
    )
    return response['SequenceNumber']

# Send sensor reading
send_event(
    'sensor-stream',
    {'sensor_id': 'temp-01', 'value': 23.5, 'unit': 'celsius'},
    'sensor-01'
)
```

### Cost Breakdown

| Service | Monthly Cost |
|---------|-------------|
| Kinesis (1 shard) | $15-30 |
| Lambda | $10-50 |
| S3 | $5-50 |
| Athena | $5-25 |
| QuickSight | $24+ |
| **Total** | **$59-179** |

---

## Pattern 5: GraphQL API Backend

### Use Case
Mobile apps, single-page applications, flexible data queries

### Architecture Diagram

```
┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│  Mobile App │────▶│   AppSync   │────▶│   Lambda    │
│   or SPA    │     │  (GraphQL)  │     │ (Resolvers) │
└─────────────┘     └──────┬──────┘     └─────────────┘
                           │
                    ┌──────▼──────┐
                    │  DynamoDB   │
                    │  (Direct)   │
                    └──────┬──────┘
                           │
                    ┌──────▼──────┐
                    │   Cognito   │
                    │   (Auth)    │
                    └─────────────┘
```

### AppSync Schema Example

```graphql
type Query {
  getUser(id: ID!): User
  listPosts(limit: Int, nextToken: String): PostConnection
}

type Mutation {
  createPost(input: CreatePostInput!): Post
  updatePost(input: UpdatePostInput!): Post
}

type Subscription {
  onCreatePost: Post @aws_subscribe(mutations: ["createPost"])
}

type User {
  id: ID!
  email: String!
  posts: [Post]
}

type Post {
  id: ID!
  title: String!
  content: String!
  author: User!
  createdAt: AWSDateTime!
}
```

### Cost Breakdown

| Service | Monthly Cost |
|---------|-------------|
| AppSync | $4-40 |
| Lambda | $5-30 |
| DynamoDB | $10-50 |
| Cognito | $0-50 |
| **Total** | **$19-170** |

---

## Pattern 6: Multi-Region High Availability

### Use Case
Global applications, disaster recovery, data sovereignty compliance

### Architecture Diagram

```
                    ┌─────────────┐
                    │  Route 53   │
                    │(Geo routing)│
                    └──────┬──────┘
                           │
          ┌────────────────┼────────────────┐
          │                                 │
   ┌──────▼──────┐                   ┌──────▼──────┐
   │ us-east-1   │                   │ eu-west-1   │
   │ CloudFront  │                   │ CloudFront  │
   └──────┬──────┘                   └──────┬──────┘
          │                                 │
   ┌──────▼──────┐                   ┌──────▼──────┐
   │ ECS/Lambda  │                   │ ECS/Lambda  │
   └──────┬──────┘                   └──────┬──────┘
          │                                 │
   ┌──────▼──────┐◀── Replication ──▶┌──────▼──────┐
   │  DynamoDB   │                   │  DynamoDB   │
   │Global Table │                   │Global Table │
   └─────────────┘                   └─────────────┘
```

### Service Stack

| Component | Service | Configuration |
|-----------|---------|---------------|
| DNS | Route 53 | Geolocation or latency routing |
| CDN | CloudFront | Multiple origins per region |
| Compute | Lambda or ECS | Deployed in each region |
| Database | DynamoDB Global Tables | Automatic replication |
| Storage | S3 CRR | Cross-region replication |

### Route 53 Failover Policy

```yaml
# Primary record
HealthCheck:
  Type: AWS::Route53::HealthCheck
  Properties:
    HealthCheckConfig:
      Port: 443
      Type: HTTPS
      ResourcePath: /health
      FullyQualifiedDomainName: api-us-east-1.example.com

RecordSetPrimary:
  Type: AWS::Route53::RecordSet
  Properties:
    Name: api.example.com
    Type: A
    SetIdentifier: primary
    Failover: PRIMARY
    HealthCheckId: !Ref HealthCheck
    AliasTarget:
      DNSName: !GetAtt USEast1ALB.DNSName
      HostedZoneId: !GetAtt USEast1ALB.CanonicalHostedZoneID
```

### Cost Considerations

| Factor | Impact |
|--------|--------|
| Compute | 2x (each region) |
| Database | 25% premium for global tables |
| Data Transfer | Cross-region replication costs |
| Route 53 | Health checks + geo queries |
| **Total** | **1.5-2x single region** |

---

## Pattern Comparison Summary

### Latency

| Pattern | Typical Latency |
|---------|-----------------|
| Serverless | 50-200ms (cold: 500ms+) |
| Three-Tier | 20-100ms |
| GraphQL | 30-150ms |
| Multi-Region | <50ms (regional) |

### Scaling Characteristics

| Pattern | Scale Limit | Scale Speed |
|---------|-------------|-------------|
| Serverless | 1000 concurrent/function | Instant |
| Three-Tier | Instance limits | Minutes |
| Event-Driven | Unlimited | Instant |
| Multi-Region | Regional limits | Instant |

### Operational Complexity

| Pattern | Setup | Maintenance | Debugging |
|---------|-------|-------------|-----------|
| Serverless | Low | Low | Medium |
| Three-Tier | Medium | Medium | Low |
| Event-Driven | High | Medium | High |
| Multi-Region | High | High | High |

FILE:references/best_practices.md
# AWS Best Practices for Startups

Production-ready practices for serverless, cost optimization, security, and operational excellence.

---

## Table of Contents

- [Serverless Best Practices](#serverless-best-practices)
- [Cost Optimization](#cost-optimization)
- [Security Hardening](#security-hardening)
- [Scalability Patterns](#scalability-patterns)
- [DevOps and Reliability](#devops-and-reliability)
- [Common Pitfalls](#common-pitfalls)

---

## Serverless Best Practices

### Lambda Function Design

#### 1. Keep Functions Stateless

Store state externally in DynamoDB, S3, or ElastiCache.

```python
# BAD: Function-level state
cache = {}

def handler(event, context):
    if event['key'] in cache:
        return cache[event['key']]
    # ...

# GOOD: External state
import boto3
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('cache')

def handler(event, context):
    response = table.get_item(Key={'pk': event['key']})
    if 'Item' in response:
        return response['Item']['value']
    # ...
```

#### 2. Implement Idempotency

Handle retries gracefully with unique request IDs.

```python
import boto3
import hashlib

dynamodb = boto3.resource('dynamodb')
idempotency_table = dynamodb.Table('idempotency')

def handler(event, context):
    # Generate idempotency key
    idempotency_key = hashlib.sha256(
        f"{event['orderId']}-{event['action']}".encode()
    ).hexdigest()

    # Check if already processed
    try:
        response = idempotency_table.get_item(Key={'pk': idempotency_key})
        if 'Item' in response:
            return response['Item']['result']
    except Exception:
        pass

    # Process request
    result = process_order(event)

    # Store result for idempotency
    idempotency_table.put_item(
        Item={
            'pk': idempotency_key,
            'result': result,
            'ttl': int(time.time()) + 86400  # 24h TTL
        }
    )

    return result
```

#### 3. Optimize Cold Starts

```python
# Initialize outside handler (reused across invocations)
import boto3
from aws_xray_sdk.core import patch_all

# SDK initialization happens once
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('my-table')
patch_all()

def handler(event, context):
    # Handler code uses pre-initialized resources
    return table.get_item(Key={'pk': event['id']})
```

**Cold Start Reduction Techniques:**
- Use provisioned concurrency for critical paths
- Minimize package size (use layers for dependencies)
- Choose interpreted languages (Python, Node.js) over compiled
- Avoid VPC unless necessary (adds 6-10 sec cold start)

#### 4. Set Appropriate Timeouts

```yaml
# Lambda configuration
Functions:
  ApiHandler:
    Timeout: 10  # Shorter for synchronous APIs
    MemorySize: 512

  BackgroundProcessor:
    Timeout: 300  # Longer for async processing
    MemorySize: 1024
```

**Timeout Guidelines:**
- API handlers: 10-30 seconds
- Event processors: 60-300 seconds
- Use Step Functions for >15 minute workflows

---

## Cost Optimization

### 1. Right-Sizing Strategy

```bash
# Check EC2 utilization
aws cloudwatch get-metric-statistics \
  --namespace AWS/EC2 \
  --metric-name CPUUtilization \
  --dimensions Name=InstanceId,Value=i-1234567890abcdef0 \
  --start-time $(date -d '7 days ago' -u +"%Y-%m-%dT%H:%M:%SZ") \
  --end-time $(date -u +"%Y-%m-%dT%H:%M:%SZ") \
  --period 3600 \
  --statistics Average
```

**Right-Sizing Rules:**
- <10% CPU average: Downsize instance
- >80% CPU average: Consider upgrade or horizontal scaling
- Review every month for the first 6 months

### 2. Savings Plans and Reserved Instances

| Commitment | Savings | Best For |
|------------|---------|----------|
| No Upfront, 1-year | 20-30% | Unknown future |
| Partial Upfront, 1-year | 30-40% | Moderate confidence |
| All Upfront, 3-year | 50-60% | Stable workloads |

```bash
# Check Savings Plans recommendations
aws cost-explorer get-savings-plans-purchase-recommendation \
  --savings-plans-type COMPUTE_SP \
  --term-in-years ONE_YEAR \
  --payment-option NO_UPFRONT \
  --lookback-period-in-days THIRTY_DAYS
```

### 3. S3 Lifecycle Policies

```json
{
  "Rules": [
    {
      "ID": "Transition to cheaper storage",
      "Status": "Enabled",
      "Filter": {
        "Prefix": "logs/"
      },
      "Transitions": [
        { "Days": 30, "StorageClass": "STANDARD_IA" },
        { "Days": 90, "StorageClass": "GLACIER" }
      ],
      "Expiration": { "Days": 365 }
    }
  ]
}
```

### 4. Lambda Memory Optimization

Test different memory settings to find optimal cost/performance.

```python
# Use AWS Lambda Power Tuning
# https://github.com/alexcasalboni/aws-lambda-power-tuning

# Example results:
# 128 MB: 2000ms, $0.000042
# 512 MB: 500ms, $0.000042
# 1024 MB: 300ms, $0.000050

# Optimal: 512 MB (same cost, 4x faster)
```

### 5. NAT Gateway Alternatives

```
NAT Gateway: $0.045/hour + $0.045/GB = ~$32/month + data

Alternatives:
1. VPC Endpoints: $0.01/hour = ~$7.30/month (for AWS services)
2. NAT Instance: t3.nano = ~$3.80/month (limited throughput)
3. No NAT: Use VPC endpoints + Lambda outside VPC
```

### 6. CloudWatch Log Retention

```yaml
# Set retention policies to avoid unbounded growth
LogGroup:
  Type: AWS::Logs::LogGroup
  Properties:
    LogGroupName: /aws/lambda/my-function
    RetentionInDays: 14  # 7, 14, 30, 60, 90, etc.
```

**Retention Guidelines:**
- Development: 7 days
- Production non-critical: 30 days
- Production critical: 90 days
- Compliance requirements: As specified

---

## Security Hardening

### 1. IAM Least Privilege

```json
// BAD: Overly permissive
{
  "Effect": "Allow",
  "Action": "dynamodb:*",
  "Resource": "*"
}

// GOOD: Specific actions and resources
{
  "Effect": "Allow",
  "Action": [
    "dynamodb:GetItem",
    "dynamodb:PutItem",
    "dynamodb:Query"
  ],
  "Resource": [
    "arn:aws:dynamodb:us-east-1:123456789:table/users",
    "arn:aws:dynamodb:us-east-1:123456789:table/users/index/*"
  ]
}
```

### 2. Encryption Configuration

```yaml
# Enable encryption everywhere
Resources:
  # DynamoDB
  Table:
    Type: AWS::DynamoDB::Table
    Properties:
      SSESpecification:
        SSEEnabled: true
        SSEType: KMS
        KMSMasterKeyId: !Ref EncryptionKey

  # S3
  Bucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketEncryption:
        ServerSideEncryptionConfiguration:
          - ServerSideEncryptionByDefault:
              SSEAlgorithm: aws:kms
              KMSMasterKeyID: !Ref EncryptionKey

  # RDS
  Database:
    Type: AWS::RDS::DBInstance
    Properties:
      StorageEncrypted: true
      KmsKeyId: !Ref EncryptionKey
```

### 3. Network Isolation

```yaml
# Private subnets with VPC endpoints
Resources:
  PrivateSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      MapPublicIpOnLaunch: false

  # DynamoDB Gateway Endpoint (free)
  DynamoDBEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      VpcId: !Ref VPC
      ServiceName: !Sub com.amazonaws.:Region.dynamodb
      VpcEndpointType: Gateway
      RouteTableIds:
        - !Ref PrivateRouteTable

  # Secrets Manager Interface Endpoint
  SecretsEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      VpcId: !Ref VPC
      ServiceName: !Sub com.amazonaws.:Region.secretsmanager
      VpcEndpointType: Interface
      PrivateDnsEnabled: true
```

### 4. Secrets Management

```python
# Never hardcode secrets
import boto3
import json

def get_secret(secret_name):
    client = boto3.client('secretsmanager')
    response = client.get_secret_value(SecretId=secret_name)
    return json.loads(response['SecretString'])

# Usage
db_creds = get_secret('prod/database/credentials')
connection = connect(
    host=db_creds['host'],
    user=db_creds['username'],
    password=db_creds['password']
)
```

### 5. API Protection

```yaml
# WAF + API Gateway
WebACL:
  Type: AWS::WAFv2::WebACL
  Properties:
    DefaultAction:
      Allow: {}
    Rules:
      - Name: RateLimit
        Priority: 1
        Action:
          Block: {}
        Statement:
          RateBasedStatement:
            Limit: 2000
            AggregateKeyType: IP
        VisibilityConfig:
          SampledRequestsEnabled: true
          CloudWatchMetricsEnabled: true
          MetricName: RateLimitRule

      - Name: AWSManagedRulesCommonRuleSet
        Priority: 2
        OverrideAction:
          None: {}
        Statement:
          ManagedRuleGroupStatement:
            VendorName: AWS
            Name: AWSManagedRulesCommonRuleSet
```

### 6. Audit Logging

```yaml
# Enable CloudTrail for all API calls
CloudTrail:
  Type: AWS::CloudTrail::Trail
  Properties:
    IsMultiRegionTrail: true
    IsLogging: true
    S3BucketName: !Ref AuditLogsBucket
    IncludeGlobalServiceEvents: true
    EnableLogFileValidation: true
    EventSelectors:
      - ReadWriteType: All
        IncludeManagementEvents: true
```

---

## Scalability Patterns

### 1. Horizontal vs Vertical Scaling

```
Horizontal (preferred):
- Add more Lambda concurrent executions
- Add more Fargate tasks
- Add more DynamoDB capacity

Vertical (when necessary):
- Increase Lambda memory
- Upgrade RDS instance
- Larger EC2 instances
```

### 2. Database Sharding

```python
# Partition by tenant ID
def get_table_for_tenant(tenant_id):
    shard = hash(tenant_id) % NUM_SHARDS
    return f"data-shard-{shard}"

# Or use DynamoDB single-table design with partition keys
def get_partition_key(tenant_id, entity_type, entity_id):
    return f"TENANT#{tenant_id}#{entity_type}#{entity_id}"
```

### 3. Caching Layers

```
Edge (CloudFront):     Global, static content, TTL: hours-days
Application (Redis):   Regional, session/query cache, TTL: minutes-hours
Database (DAX):        DynamoDB-specific, TTL: minutes
```

```python
# ElastiCache Redis caching pattern
import redis
import json

cache = redis.Redis(host='cache.abc123.cache.amazonaws.com', port=6379)

def get_user(user_id):
    # Check cache first
    cached = cache.get(f"user:{user_id}")
    if cached:
        return json.loads(cached)

    # Fetch from database
    user = db.get_user(user_id)

    # Cache for 5 minutes
    cache.setex(f"user:{user_id}", 300, json.dumps(user))

    return user
```

### 4. Auto-Scaling Configuration

```yaml
# ECS Service Auto-scaling
AutoScalingTarget:
  Type: AWS::ApplicationAutoScaling::ScalableTarget
  Properties:
    MaxCapacity: 10
    MinCapacity: 2
    ResourceId: !Sub service/Cluster/Service.Name
    ScalableDimension: ecs:service:DesiredCount
    ServiceNamespace: ecs

ScalingPolicy:
  Type: AWS::ApplicationAutoScaling::ScalingPolicy
  Properties:
    PolicyType: TargetTrackingScaling
    TargetTrackingScalingPolicyConfiguration:
      PredefinedMetricSpecification:
        PredefinedMetricType: ECSServiceAverageCPUUtilization
      TargetValue: 70
      ScaleInCooldown: 300
      ScaleOutCooldown: 60
```

---

## DevOps and Reliability

### 1. Infrastructure as Code

```bash
# Version control all infrastructure
git init
git add .
git commit -m "Initial infrastructure setup"

# Use separate stacks per environment
cdk deploy --context environment=dev
cdk deploy --context environment=staging
cdk deploy --context environment=production
```

### 2. Blue/Green Deployments

```yaml
# CodeDeploy Blue/Green for ECS
DeploymentGroup:
  Type: AWS::CodeDeploy::DeploymentGroup
  Properties:
    DeploymentConfigName: CodeDeployDefault.ECSAllAtOnce
    DeploymentStyle:
      DeploymentType: BLUE_GREEN
      DeploymentOption: WITH_TRAFFIC_CONTROL
    BlueGreenDeploymentConfiguration:
      DeploymentReadyOption:
        ActionOnTimeout: CONTINUE_DEPLOYMENT
        WaitTimeInMinutes: 0
      TerminateBlueInstancesOnDeploymentSuccess:
        Action: TERMINATE
        TerminationWaitTimeInMinutes: 5
```

### 3. Health Checks

```python
# Application health endpoint
from flask import Flask, jsonify
import boto3

app = Flask(__name__)

@app.route('/health')
def health():
    checks = {
        'database': check_database(),
        'cache': check_cache(),
        'external_api': check_external_api()
    }

    status = 'healthy' if all(checks.values()) else 'unhealthy'
    code = 200 if status == 'healthy' else 503

    return jsonify({'status': status, 'checks': checks}), code

def check_database():
    try:
        # Quick connectivity test
        db.execute('SELECT 1')
        return True
    except Exception:
        return False
```

### 4. Monitoring Setup

```yaml
# CloudWatch Dashboard
Dashboard:
  Type: AWS::CloudWatch::Dashboard
  Properties:
    DashboardName: production-overview
    DashboardBody: |
      {
        "widgets": [
          {
            "type": "metric",
            "properties": {
              "metrics": [
                ["AWS/Lambda", "Invocations", "FunctionName", "api-handler"],
                [".", "Errors", ".", "."],
                [".", "Duration", ".", ".", {"stat": "p99"}]
              ],
              "period": 60,
              "title": "Lambda Metrics"
            }
          }
        ]
      }

# Critical Alarms
ErrorAlarm:
  Type: AWS::CloudWatch::Alarm
  Properties:
    AlarmName: high-error-rate
    MetricName: Errors
    Namespace: AWS/Lambda
    Statistic: Sum
    Period: 60
    EvaluationPeriods: 3
    Threshold: 10
    ComparisonOperator: GreaterThanThreshold
    AlarmActions:
      - !Ref AlertTopic
```

---

## Common Pitfalls

### Technical Debt

| Pitfall | Solution |
|---------|----------|
| Over-engineering early | Start simple, scale when needed |
| Under-monitoring | Set up CloudWatch from day one |
| Ignoring costs | Enable Cost Explorer and billing alerts |
| Single region only | Plan for multi-region from start |

### Security Mistakes

| Mistake | Prevention |
|---------|------------|
| Public S3 buckets | Block public access, use bucket policies |
| Overly permissive IAM | Never use "*", specify resources |
| Hardcoded credentials | Use Secrets Manager, IAM roles |
| Unencrypted data | Enable encryption by default |

### Performance Issues

| Issue | Solution |
|-------|----------|
| No caching | Add CloudFront, ElastiCache early |
| Inefficient queries | Use indexes, avoid DynamoDB scans |
| Large Lambda packages | Use layers, minimize dependencies |
| N+1 queries | Implement DataLoader, batch operations |

### Cost Surprises

| Surprise | Prevention |
|----------|------------|
| Undeleted resources | Tag everything, review weekly |
| Data transfer costs | Keep traffic in same AZ/region |
| NAT Gateway charges | Use VPC endpoints for AWS services |
| Log accumulation | Set CloudWatch retention policies |

FILE:references/service_selection.md
# AWS Service Selection Guide

Quick reference for choosing the right AWS service based on requirements.

---

## Table of Contents

- [Compute Services](#compute-services)
- [Database Services](#database-services)
- [Storage Services](#storage-services)
- [Messaging and Events](#messaging-and-events)
- [API and Integration](#api-and-integration)
- [Networking](#networking)
- [Security and Identity](#security-and-identity)

---

## Compute Services

### Decision Matrix

| Requirement | Recommended Service |
|-------------|---------------------|
| Event-driven, short tasks (<15 min) | Lambda |
| Containerized apps, predictable traffic | ECS Fargate |
| Custom configs, GPU/FPGA | EC2 |
| Simple container from source | App Runner |
| Kubernetes workloads | EKS |
| Batch processing | AWS Batch |

### Lambda

**Best for:** Event-driven functions, API backends, scheduled tasks

```
Limits:
- Execution: 15 minutes max
- Memory: 128 MB - 10 GB
- Package: 50 MB (zip), 10 GB (container)
- Concurrency: 1000 default (soft limit)

Pricing: $0.20 per 1M requests + compute time
```

**Use when:**
- Variable/unpredictable traffic
- Pay-per-use is important
- No server management desired
- Short-duration operations

**Avoid when:**
- Long-running processes (>15 min)
- Low-latency requirements (<50ms)
- Heavy compute (consider Fargate)

### ECS Fargate

**Best for:** Containerized applications, microservices

```
Limits:
- vCPU: 0.25 - 16
- Memory: 0.5 GB - 120 GB
- Storage: 20 GB - 200 GB ephemeral

Pricing: Per vCPU-hour + GB-hour
```

**Use when:**
- Containerized applications
- Predictable traffic patterns
- Long-running processes
- Need more control than Lambda

### EC2

**Best for:** Custom configurations, specialized hardware

```
Instance Types:
- General: t3, m6i
- Compute: c6i
- Memory: r6i
- GPU: p4d, g5
- Storage: i3, d3
```

**Use when:**
- Need GPU/FPGA
- Windows applications
- Specific instance configurations
- Reserved capacity makes sense

---

## Database Services

### Decision Matrix

| Data Type | Query Pattern | Scale | Recommended |
|-----------|--------------|-------|-------------|
| Key-value | Simple lookups | Any | DynamoDB |
| Document | Flexible queries | <1TB | DocumentDB |
| Relational | Complex joins | Variable | Aurora Serverless |
| Relational | High volume | Fixed | Aurora Standard |
| Time-series | Time-based | Any | Timestream |
| Graph | Relationships | Any | Neptune |

### DynamoDB

**Best for:** Key-value and document data, serverless applications

```
Limits:
- Item size: 400 KB max
- Partition key: 2048 bytes
- Sort key: 1024 bytes
- GSI: 20 per table

Pricing:
- On-demand: $1.25 per million writes, $0.25 per million reads
- Provisioned: Per RCU/WCU
```

**Data Modeling Example:**

```
# Single-table design for e-commerce
PK                  SK                  Attributes
USER#123            PROFILE             {name, email, ...}
USER#123            ORDER#456           {total, status, ...}
USER#123            ORDER#456#ITEM#1    {product, qty, ...}
PRODUCT#789         METADATA            {name, price, ...}
```

### Aurora

**Best for:** Relational data with complex queries

| Edition | Use Case | Scaling |
|---------|----------|---------|
| Aurora Serverless v2 | Variable workloads | 0.5-128 ACUs, auto |
| Aurora Standard | Predictable workloads | Instance-based |
| Aurora Global | Multi-region | Cross-region replication |

```
Limits:
- Storage: 128 TB max
- Replicas: 15 read replicas
- Connections: Instance-dependent

Pricing:
- Serverless: $0.12 per ACU-hour
- Standard: Instance + storage + I/O
```

### Comparison: DynamoDB vs Aurora

| Factor | DynamoDB | Aurora |
|--------|----------|--------|
| Query flexibility | Limited (key-based) | Full SQL |
| Scaling | Instant, unlimited | Minutes, up to limits |
| Consistency | Eventually/Strong | ACID |
| Cost model | Per-request | Per-hour |
| Operational | Zero management | Some management |

---

## Storage Services

### S3 Storage Classes

| Class | Access Pattern | Retrieval | Cost (GB/mo) |
|-------|---------------|-----------|--------------|
| Standard | Frequent | Instant | $0.023 |
| Intelligent-Tiering | Unknown | Instant | $0.023 + monitoring |
| Standard-IA | Infrequent (30+ days) | Instant | $0.0125 |
| One Zone-IA | Infrequent, single AZ | Instant | $0.01 |
| Glacier Instant | Archive, instant access | Instant | $0.004 |
| Glacier Flexible | Archive | Minutes-hours | $0.0036 |
| Glacier Deep Archive | Long-term archive | 12-48 hours | $0.00099 |

### Lifecycle Policy Example

```json
{
  "Rules": [
    {
      "ID": "Archive old data",
      "Status": "Enabled",
      "Transitions": [
        {
          "Days": 30,
          "StorageClass": "STANDARD_IA"
        },
        {
          "Days": 90,
          "StorageClass": "GLACIER"
        },
        {
          "Days": 365,
          "StorageClass": "DEEP_ARCHIVE"
        }
      ],
      "Expiration": {
        "Days": 2555
      }
    }
  ]
}
```

### Block and File Storage

| Service | Use Case | Access |
|---------|----------|--------|
| EBS | EC2 block storage | Single instance |
| EFS | Shared file system | Multiple instances |
| FSx for Lustre | HPC workloads | High throughput |
| FSx for Windows | Windows apps | SMB protocol |

---

## Messaging and Events

### Decision Matrix

| Pattern | Service | Use Case |
|---------|---------|----------|
| Event routing | EventBridge | Microservices, SaaS integration |
| Pub/sub | SNS | Fan-out notifications |
| Queue | SQS | Decoupling, buffering |
| Streaming | Kinesis | Real-time analytics |
| Message broker | Amazon MQ | Legacy migrations |

### EventBridge

**Best for:** Event-driven architectures, SaaS integration

```python
# EventBridge rule pattern
{
    "source": ["orders.service"],
    "detail-type": ["OrderCreated"],
    "detail": {
        "total": [{"numeric": [">=", 100]}]
    }
}
```

### SQS

**Best for:** Decoupling services, handling load spikes

| Feature | Standard | FIFO |
|---------|----------|------|
| Throughput | Unlimited | 3000 msg/sec |
| Ordering | Best effort | Guaranteed |
| Delivery | At least once | Exactly once |
| Deduplication | No | Yes |

```python
# SQS with dead letter queue
import boto3

sqs = boto3.client('sqs')

def process_with_dlq(queue_url, dlq_url, max_retries=3):
    response = sqs.receive_message(
        QueueUrl=queue_url,
        MaxNumberOfMessages=10,
        WaitTimeSeconds=20,
        AttributeNames=['ApproximateReceiveCount']
    )

    for message in response.get('Messages', []):
        receive_count = int(message['Attributes']['ApproximateReceiveCount'])

        try:
            process(message)
            sqs.delete_message(QueueUrl=queue_url, ReceiptHandle=message['ReceiptHandle'])
        except Exception as e:
            if receive_count >= max_retries:
                sqs.send_message(QueueUrl=dlq_url, MessageBody=message['Body'])
                sqs.delete_message(QueueUrl=queue_url, ReceiptHandle=message['ReceiptHandle'])
```

### Kinesis

**Best for:** Real-time streaming data, analytics

| Service | Use Case |
|---------|----------|
| Data Streams | Custom processing |
| Data Firehose | Direct to S3/Redshift |
| Data Analytics | SQL on streams |
| Video Streams | Video ingestion |

---

## API and Integration

### API Gateway vs AppSync

| Factor | API Gateway | AppSync |
|--------|-------------|---------|
| Protocol | REST, WebSocket | GraphQL |
| Real-time | WebSocket setup | Built-in subscriptions |
| Caching | Response caching | Field-level caching |
| Integration | Lambda, HTTP, AWS | Lambda, DynamoDB, HTTP |
| Pricing | Per request | Per request + data |

### API Gateway Configuration

```yaml
# Throttling and caching
Resources:
  ApiGateway:
    Type: AWS::ApiGateway::RestApi
    Properties:
      Name: my-api

  ApiStage:
    Type: AWS::ApiGateway::Stage
    Properties:
      StageName: prod
      MethodSettings:
        - HttpMethod: "*"
          ResourcePath: "/*"
          ThrottlingBurstLimit: 500
          ThrottlingRateLimit: 1000
          CachingEnabled: true
          CacheTtlInSeconds: 300
```

### Step Functions

**Best for:** Workflow orchestration, long-running processes

```json
{
  "StartAt": "ProcessOrder",
  "States": {
    "ProcessOrder": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:...:processOrder",
      "Next": "CheckInventory"
    },
    "CheckInventory": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.inStock",
          "BooleanEquals": true,
          "Next": "ShipOrder"
        }
      ],
      "Default": "BackOrder"
    },
    "ShipOrder": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:...:shipOrder",
      "End": true
    },
    "BackOrder": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:...:backOrder",
      "End": true
    }
  }
}
```

---

## Networking

### VPC Components

| Component | Purpose |
|-----------|---------|
| VPC | Isolated network |
| Subnet | Network segment (public/private) |
| Internet Gateway | Public internet access |
| NAT Gateway | Private subnet outbound |
| VPC Endpoint | Private AWS service access |
| Transit Gateway | VPC interconnection |

### VPC Design Pattern

```
VPC: 10.0.0.0/16

Public Subnets (AZ a, b, c):
  10.0.1.0/24, 10.0.2.0/24, 10.0.3.0/24
  - ALB, NAT Gateway, Bastion

Private Subnets (AZ a, b, c):
  10.0.11.0/24, 10.0.12.0/24, 10.0.13.0/24
  - Application servers, Lambda

Database Subnets (AZ a, b, c):
  10.0.21.0/24, 10.0.22.0/24, 10.0.23.0/24
  - RDS, ElastiCache
```

### VPC Endpoints (Cost Savings)

```yaml
# Interface endpoint for Secrets Manager
SecretsManagerEndpoint:
  Type: AWS::EC2::VPCEndpoint
  Properties:
    VpcId: !Ref VPC
    ServiceName: !Sub com.amazonaws.:Region.secretsmanager
    VpcEndpointType: Interface
    SubnetIds: !Ref PrivateSubnets
    SecurityGroupIds:
      - !Ref EndpointSecurityGroup
```

---

## Security and Identity

### IAM Best Practices

```json
// Least privilege policy example
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem",
        "dynamodb:Query"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:123456789:table/users",
      "Condition": {
        "ForAllValues:StringEquals": {
          "dynamodb:LeadingKeys": ["userid"]
        }
      }
    }
  ]
}
```

### Secrets Manager vs Parameter Store

| Factor | Secrets Manager | Parameter Store |
|--------|-----------------|-----------------|
| Auto-rotation | Built-in | Manual |
| Cross-account | Yes | Limited |
| Pricing | $0.40/secret/month | Free (standard) |
| Use case | Credentials, API keys | Config, non-secrets |

### Cognito Configuration

```yaml
UserPool:
  Type: AWS::Cognito::UserPool
  Properties:
    UserPoolName: my-app-users
    AutoVerifiedAttributes:
      - email
    MfaConfiguration: OPTIONAL
    EnabledMfas:
      - SOFTWARE_TOKEN_MFA
    Policies:
      PasswordPolicy:
        MinimumLength: 12
        RequireLowercase: true
        RequireUppercase: true
        RequireNumbers: true
        RequireSymbols: true
    AccountRecoverySetting:
      RecoveryMechanisms:
        - Name: verified_email
          Priority: 1
```

FILE:scripts/architecture_designer.py
"""
AWS architecture design and service recommendation module.
Generates architecture patterns based on application requirements.
"""

from typing import Dict, List, Any, Optional
from enum import Enum


class ApplicationType(Enum):
    """Types of applications supported."""
    WEB_APP = "web_application"
    MOBILE_BACKEND = "mobile_backend"
    DATA_PIPELINE = "data_pipeline"
    MICROSERVICES = "microservices"
    SAAS_PLATFORM = "saas_platform"
    IOT_PLATFORM = "iot_platform"


class ArchitectureDesigner:
    """Design AWS architectures based on requirements."""

    def __init__(self, requirements: Dict[str, Any]):
        """
        Initialize with application requirements.

        Args:
            requirements: Dictionary containing app type, traffic, budget, etc.
        """
        self.app_type = requirements.get('application_type', 'web_application')
        self.expected_users = requirements.get('expected_users', 1000)
        self.requests_per_second = requirements.get('requests_per_second', 10)
        self.budget_monthly = requirements.get('budget_monthly_usd', 500)
        self.team_size = requirements.get('team_size', 3)
        self.aws_experience = requirements.get('aws_experience', 'beginner')
        self.compliance_needs = requirements.get('compliance', [])
        self.data_size_gb = requirements.get('data_size_gb', 10)

    def recommend_architecture_pattern(self) -> Dict[str, Any]:
        """
        Recommend architecture pattern based on requirements.

        Returns:
            Dictionary with recommended pattern and services
        """
        # Determine pattern based on app type and scale
        if self.app_type in ['web_application', 'saas_platform']:
            if self.expected_users < 10000:
                return self._serverless_web_architecture()
            elif self.expected_users < 100000:
                return self._modern_three_tier_architecture()
            else:
                return self._multi_region_architecture()

        elif self.app_type == 'mobile_backend':
            return self._serverless_mobile_backend()

        elif self.app_type == 'data_pipeline':
            return self._event_driven_data_pipeline()

        elif self.app_type == 'microservices':
            return self._event_driven_microservices()

        elif self.app_type == 'iot_platform':
            return self._iot_architecture()

        else:
            return self._serverless_web_architecture()  # Default

    def _serverless_web_architecture(self) -> Dict[str, Any]:
        """Serverless web application pattern."""
        return {
            'pattern_name': 'Serverless Web Application',
            'description': 'Fully serverless architecture with zero server management',
            'use_case': 'SaaS platforms, low to medium traffic websites, MVPs',
            'services': {
                'frontend': {
                    'service': 'S3 + CloudFront',
                    'purpose': 'Static website hosting with global CDN',
                    'configuration': {
                        's3_bucket': 'website-bucket',
                        'cloudfront_distribution': 'HTTPS with custom domain',
                        'caching': 'Cache-Control headers, edge caching'
                    }
                },
                'api': {
                    'service': 'API Gateway + Lambda',
                    'purpose': 'REST API backend with auto-scaling',
                    'configuration': {
                        'api_type': 'REST API',
                        'authorization': 'Cognito User Pools or API Keys',
                        'throttling': f'{self.requests_per_second * 10} requests/second',
                        'lambda_memory': '512 MB (optimize based on testing)',
                        'lambda_timeout': '10 seconds'
                    }
                },
                'database': {
                    'service': 'DynamoDB',
                    'purpose': 'NoSQL database with pay-per-request pricing',
                    'configuration': {
                        'billing_mode': 'PAY_PER_REQUEST',
                        'backup': 'Point-in-time recovery enabled',
                        'encryption': 'KMS encryption at rest'
                    }
                },
                'authentication': {
                    'service': 'Cognito',
                    'purpose': 'User authentication and authorization',
                    'configuration': {
                        'user_pools': 'Email/password + social providers',
                        'mfa': 'Optional MFA with SMS or TOTP',
                        'token_expiration': '1 hour access, 30 days refresh'
                    }
                },
                'cicd': {
                    'service': 'AWS Amplify or CodePipeline',
                    'purpose': 'Automated deployment from Git',
                    'configuration': {
                        'source': 'GitHub or CodeCommit',
                        'build': 'Automatic on commit',
                        'environments': 'dev, staging, production'
                    }
                }
            },
            'estimated_cost': {
                'monthly_usd': self._calculate_serverless_cost(),
                'breakdown': {
                    'CloudFront': '10-30 USD',
                    'Lambda': '5-20 USD',
                    'API Gateway': '10-40 USD',
                    'DynamoDB': '5-30 USD',
                    'Cognito': '0-10 USD (free tier: 50k MAU)',
                    'S3': '1-5 USD'
                }
            },
            'pros': [
                'No server management',
                'Auto-scaling built-in',
                'Pay only for what you use',
                'Fast to deploy and iterate',
                'High availability by default'
            ],
            'cons': [
                'Cold start latency (100-500ms)',
                'Vendor lock-in to AWS',
                'Debugging distributed systems complex',
                'Learning curve for serverless patterns'
            ],
            'scaling_characteristics': {
                'users_supported': '1k - 100k',
                'requests_per_second': '100 - 10,000',
                'scaling_method': 'Automatic (Lambda concurrency)'
            }
        }

    def _modern_three_tier_architecture(self) -> Dict[str, Any]:
        """Traditional three-tier with modern AWS services."""
        return {
            'pattern_name': 'Modern Three-Tier Application',
            'description': 'Classic architecture with containers and managed services',
            'use_case': 'Traditional web apps, e-commerce, content management',
            'services': {
                'load_balancer': {
                    'service': 'Application Load Balancer (ALB)',
                    'purpose': 'Distribute traffic across instances',
                    'configuration': {
                        'scheme': 'internet-facing',
                        'target_type': 'ECS tasks or EC2 instances',
                        'health_checks': '/health endpoint, 30s interval',
                        'ssl': 'ACM certificate for HTTPS'
                    }
                },
                'compute': {
                    'service': 'ECS Fargate or EC2 Auto Scaling',
                    'purpose': 'Run containerized applications',
                    'configuration': {
                        'container_platform': 'ECS Fargate (serverless containers)',
                        'task_definition': '512 MB memory, 0.25 vCPU (start small)',
                        'auto_scaling': f'2-{max(4, self.expected_users // 5000)} tasks',
                        'deployment': 'Rolling update, 50% at a time'
                    }
                },
                'database': {
                    'service': 'RDS Aurora (MySQL/PostgreSQL)',
                    'purpose': 'Managed relational database',
                    'configuration': {
                        'instance_class': 'db.t3.medium or db.t4g.medium',
                        'multi_az': 'Yes (high availability)',
                        'read_replicas': '1-2 for read scaling',
                        'backup_retention': '7 days',
                        'encryption': 'KMS encryption enabled'
                    }
                },
                'cache': {
                    'service': 'ElastiCache Redis',
                    'purpose': 'Session storage, application caching',
                    'configuration': {
                        'node_type': 'cache.t3.micro or cache.t4g.micro',
                        'replication': 'Multi-AZ with automatic failover',
                        'eviction_policy': 'allkeys-lru'
                    }
                },
                'cdn': {
                    'service': 'CloudFront',
                    'purpose': 'Cache static assets globally',
                    'configuration': {
                        'origins': 'ALB (dynamic), S3 (static)',
                        'caching': 'Cache based on headers/cookies',
                        'compression': 'Gzip compression enabled'
                    }
                },
                'storage': {
                    'service': 'S3',
                    'purpose': 'User uploads, backups, logs',
                    'configuration': {
                        'storage_class': 'S3 Standard with lifecycle policies',
                        'versioning': 'Enabled for important buckets',
                        'lifecycle': 'Transition to IA after 30 days'
                    }
                }
            },
            'estimated_cost': {
                'monthly_usd': self._calculate_three_tier_cost(),
                'breakdown': {
                    'ALB': '20-30 USD',
                    'ECS Fargate': '50-200 USD',
                    'RDS Aurora': '100-300 USD',
                    'ElastiCache': '30-80 USD',
                    'CloudFront': '10-50 USD',
                    'S3': '10-30 USD'
                }
            },
            'pros': [
                'Proven architecture pattern',
                'Easy to understand and debug',
                'Flexible scaling options',
                'Support for complex applications',
                'Managed services reduce operational burden'
            ],
            'cons': [
                'Higher baseline costs',
                'More complex than serverless',
                'Requires more operational knowledge',
                'Manual scaling configuration needed'
            ],
            'scaling_characteristics': {
                'users_supported': '10k - 500k',
                'requests_per_second': '1,000 - 50,000',
                'scaling_method': 'Auto Scaling based on CPU/memory/requests'
            }
        }

    def _serverless_mobile_backend(self) -> Dict[str, Any]:
        """Serverless mobile backend with GraphQL."""
        return {
            'pattern_name': 'Serverless Mobile Backend',
            'description': 'Mobile-first backend with GraphQL and real-time features',
            'use_case': 'Mobile apps, single-page apps, offline-first applications',
            'services': {
                'api': {
                    'service': 'AppSync (GraphQL)',
                    'purpose': 'Flexible GraphQL API with real-time subscriptions',
                    'configuration': {
                        'api_type': 'GraphQL',
                        'authorization': 'Cognito User Pools + API Keys',
                        'resolvers': 'Direct DynamoDB or Lambda',
                        'subscriptions': 'WebSocket for real-time updates',
                        'caching': 'Server-side caching (1 hour TTL)'
                    }
                },
                'database': {
                    'service': 'DynamoDB',
                    'purpose': 'Fast NoSQL database with global tables',
                    'configuration': {
                        'billing_mode': 'PAY_PER_REQUEST (on-demand)',
                        'global_tables': 'Multi-region if needed',
                        'streams': 'Enabled for change data capture',
                        'ttl': 'Automatic expiration for temporary data'
                    }
                },
                'file_storage': {
                    'service': 'S3 + CloudFront',
                    'purpose': 'User uploads (images, videos, documents)',
                    'configuration': {
                        'access': 'Signed URLs or Cognito credentials',
                        'lifecycle': 'Intelligent-Tiering for cost optimization',
                        'cdn': 'CloudFront for fast global delivery'
                    }
                },
                'authentication': {
                    'service': 'Cognito',
                    'purpose': 'User management and federation',
                    'configuration': {
                        'identity_providers': 'Email, Google, Apple, Facebook',
                        'mfa': 'SMS or TOTP',
                        'groups': 'Admin, premium, free tiers',
                        'custom_attributes': 'User metadata storage'
                    }
                },
                'push_notifications': {
                    'service': 'SNS Mobile Push',
                    'purpose': 'Push notifications to mobile devices',
                    'configuration': {
                        'platforms': 'iOS (APNs), Android (FCM)',
                        'topics': 'Group notifications by topic',
                        'delivery_status': 'CloudWatch Logs for tracking'
                    }
                },
                'analytics': {
                    'service': 'Pinpoint',
                    'purpose': 'User analytics and engagement',
                    'configuration': {
                        'events': 'Custom events tracking',
                        'campaigns': 'Targeted messaging',
                        'segments': 'User segmentation'
                    }
                }
            },
            'estimated_cost': {
                'monthly_usd': 50 + (self.expected_users * 0.005),
                'breakdown': {
                    'AppSync': '5-40 USD',
                    'DynamoDB': '10-50 USD',
                    'Cognito': '0-15 USD',
                    'S3 + CloudFront': '10-40 USD',
                    'SNS': '1-10 USD',
                    'Pinpoint': '10-30 USD'
                }
            },
            'pros': [
                'Single GraphQL endpoint',
                'Real-time subscriptions built-in',
                'Offline-first capabilities',
                'Auto-generated mobile SDK',
                'Flexible querying (no over/under fetching)'
            ],
            'cons': [
                'GraphQL learning curve',
                'Complex queries can be expensive',
                'Debugging subscriptions challenging',
                'Limited to AWS AppSync features'
            ],
            'scaling_characteristics': {
                'users_supported': '1k - 1M',
                'requests_per_second': '100 - 100,000',
                'scaling_method': 'Automatic (AppSync managed)'
            }
        }

    def _event_driven_microservices(self) -> Dict[str, Any]:
        """Event-driven microservices architecture."""
        return {
            'pattern_name': 'Event-Driven Microservices',
            'description': 'Loosely coupled services with event bus',
            'use_case': 'Complex business workflows, asynchronous processing',
            'services': {
                'event_bus': {
                    'service': 'EventBridge',
                    'purpose': 'Central event routing between services',
                    'configuration': {
                        'bus_type': 'Custom event bus',
                        'rules': 'Route events by type/source',
                        'targets': 'Lambda, SQS, Step Functions',
                        'archive': 'Event replay capability'
                    }
                },
                'compute': {
                    'service': 'Lambda + ECS Fargate (hybrid)',
                    'purpose': 'Service implementation',
                    'configuration': {
                        'lambda': 'Lightweight services, event handlers',
                        'fargate': 'Long-running services, heavy processing',
                        'auto_scaling': 'Lambda (automatic), Fargate (target tracking)'
                    }
                },
                'queues': {
                    'service': 'SQS',
                    'purpose': 'Decouple services, handle failures',
                    'configuration': {
                        'queue_type': 'Standard (high throughput) or FIFO (ordering)',
                        'dlq': 'Dead letter queue after 3 retries',
                        'visibility_timeout': '30 seconds (adjust per service)',
                        'retention': '4 days'
                    }
                },
                'orchestration': {
                    'service': 'Step Functions',
                    'purpose': 'Complex workflows, saga patterns',
                    'configuration': {
                        'type': 'Standard (long-running) or Express (high volume)',
                        'error_handling': 'Retry, catch, rollback logic',
                        'timeouts': 'Per-state timeouts',
                        'logging': 'CloudWatch Logs integration'
                    }
                },
                'database': {
                    'service': 'DynamoDB (per service)',
                    'purpose': 'Each microservice owns its data',
                    'configuration': {
                        'pattern': 'Database per service',
                        'streams': 'DynamoDB Streams for change events',
                        'backup': 'Point-in-time recovery'
                    }
                },
                'api_gateway': {
                    'service': 'API Gateway',
                    'purpose': 'Unified API facade',
                    'configuration': {
                        'integration': 'Lambda proxy or HTTP proxy',
                        'authentication': 'Cognito or Lambda authorizer',
                        'rate_limiting': 'Per-client throttling'
                    }
                }
            },
            'estimated_cost': {
                'monthly_usd': 100 + (self.expected_users * 0.01),
                'breakdown': {
                    'EventBridge': '5-20 USD',
                    'Lambda': '20-100 USD',
                    'SQS': '1-10 USD',
                    'Step Functions': '10-50 USD',
                    'DynamoDB': '30-150 USD',
                    'API Gateway': '10-40 USD'
                }
            },
            'pros': [
                'Loose coupling between services',
                'Independent scaling and deployment',
                'Failure isolation',
                'Technology diversity possible',
                'Easy to test individual services'
            ],
            'cons': [
                'Operational complexity',
                'Distributed tracing required',
                'Eventual consistency challenges',
                'Network latency between services',
                'More moving parts to monitor'
            ],
            'scaling_characteristics': {
                'users_supported': '10k - 10M',
                'requests_per_second': '1,000 - 1,000,000',
                'scaling_method': 'Per-service auto-scaling'
            }
        }

    def _event_driven_data_pipeline(self) -> Dict[str, Any]:
        """Real-time data processing pipeline."""
        return {
            'pattern_name': 'Real-Time Data Pipeline',
            'description': 'Scalable data ingestion and processing',
            'use_case': 'Analytics, IoT data, log processing, ETL',
            'services': {
                'ingestion': {
                    'service': 'Kinesis Data Streams',
                    'purpose': 'Real-time data ingestion',
                    'configuration': {
                        'shards': f'{max(1, self.data_size_gb // 10)} shards',
                        'retention': '24 hours (extend to 7 days if needed)',
                        'encryption': 'KMS encryption'
                    }
                },
                'processing': {
                    'service': 'Lambda or Kinesis Analytics',
                    'purpose': 'Transform and enrich data',
                    'configuration': {
                        'lambda_concurrency': 'Match shard count',
                        'batch_size': '100-500 records per invocation',
                        'error_handling': 'DLQ for failed records'
                    }
                },
                'storage': {
                    'service': 'S3 Data Lake',
                    'purpose': 'Long-term storage and analytics',
                    'configuration': {
                        'format': 'Parquet (compressed, columnar)',
                        'partitioning': 'By date (year/month/day/hour)',
                        'lifecycle': 'Transition to Glacier after 90 days',
                        'catalog': 'AWS Glue Data Catalog'
                    }
                },
                'analytics': {
                    'service': 'Athena',
                    'purpose': 'SQL queries on S3 data',
                    'configuration': {
                        'query_results': 'Store in separate S3 bucket',
                        'workgroups': 'Separate dev and prod',
                        'cost_controls': 'Query limits per workgroup'
                    }
                },
                'visualization': {
                    'service': 'QuickSight',
                    'purpose': 'Business intelligence dashboards',
                    'configuration': {
                        'source': 'Athena or direct S3',
                        'refresh': 'Hourly or daily',
                        'sharing': 'Embedded dashboards or web access'
                    }
                },
                'alerting': {
                    'service': 'CloudWatch + SNS',
                    'purpose': 'Monitor metrics and alerts',
                    'configuration': {
                        'metrics': 'Custom metrics from processing',
                        'alarms': 'Threshold-based alerts',
                        'notifications': 'Email, Slack, PagerDuty'
                    }
                }
            },
            'estimated_cost': {
                'monthly_usd': self._calculate_data_pipeline_cost(),
                'breakdown': {
                    'Kinesis': '15-100 USD (per shard)',
                    'Lambda': '10-50 USD',
                    'S3': '10-50 USD',
                    'Athena': '5-30 USD (per TB scanned)',
                    'QuickSight': '9-18 USD per user',
                    'Glue': '5-20 USD'
                }
            },
            'pros': [
                'Real-time processing capability',
                'Scales to millions of events',
                'Cost-effective long-term storage',
                'SQL analytics on raw data',
                'Serverless architecture'
            ],
            'cons': [
                'Kinesis shard management required',
                'Athena costs based on data scanned',
                'Schema evolution complexity',
                'Cold data queries can be slow'
            ],
            'scaling_characteristics': {
                'events_per_second': '1,000 - 1,000,000',
                'data_volume': '1 GB - 1 PB per day',
                'scaling_method': 'Add Kinesis shards, partition S3 data'
            }
        }

    def _iot_architecture(self) -> Dict[str, Any]:
        """IoT platform architecture."""
        return {
            'pattern_name': 'IoT Platform',
            'description': 'Scalable IoT device management and data processing',
            'use_case': 'Connected devices, sensors, smart devices',
            'services': {
                'device_management': {
                    'service': 'IoT Core',
                    'purpose': 'Device connectivity and management',
                    'configuration': {
                        'protocol': 'MQTT over TLS',
                        'thing_registry': 'Device metadata storage',
                        'device_shadow': 'Desired and reported state',
                        'rules_engine': 'Route messages to services'
                    }
                },
                'device_provisioning': {
                    'service': 'IoT Device Management',
                    'purpose': 'Fleet provisioning and updates',
                    'configuration': {
                        'fleet_indexing': 'Search devices',
                        'jobs': 'OTA firmware updates',
                        'bulk_operations': 'Manage device groups'
                    }
                },
                'data_processing': {
                    'service': 'IoT Analytics',
                    'purpose': 'Process and analyze IoT data',
                    'configuration': {
                        'channels': 'Ingest device data',
                        'pipelines': 'Transform and enrich',
                        'data_store': 'Time-series storage',
                        'notebooks': 'Jupyter notebooks for analysis'
                    }
                },
                'time_series_db': {
                    'service': 'Timestream',
                    'purpose': 'Store time-series metrics',
                    'configuration': {
                        'memory_store': 'Recent data (hours)',
                        'magnetic_store': 'Historical data (years)',
                        'retention': 'Auto-tier based on age'
                    }
                },
                'real_time_alerts': {
                    'service': 'IoT Events',
                    'purpose': 'Detect and respond to events',
                    'configuration': {
                        'detector_models': 'Define alert conditions',
                        'actions': 'SNS, Lambda, SQS',
                        'state_tracking': 'Per-device state machines'
                    }
                }
            },
            'estimated_cost': {
                'monthly_usd': 50 + (self.expected_users * 0.1),  # Expected_users = device count
                'breakdown': {
                    'IoT Core': '10-100 USD (per million messages)',
                    'IoT Analytics': '5-50 USD',
                    'Timestream': '10-80 USD',
                    'IoT Events': '1-20 USD',
                    'Data transfer': '10-50 USD'
                }
            },
            'pros': [
                'Built for IoT scale',
                'Secure device connectivity',
                'Managed device lifecycle',
                'Time-series optimized',
                'Real-time event detection'
            ],
            'cons': [
                'IoT-specific pricing model',
                'MQTT protocol required',
                'Regional limitations',
                'Complexity for simple use cases'
            ],
            'scaling_characteristics': {
                'devices_supported': '100 - 10,000,000',
                'messages_per_second': '1,000 - 100,000',
                'scaling_method': 'Automatic (managed service)'
            }
        }

    def _multi_region_architecture(self) -> Dict[str, Any]:
        """Multi-region high availability architecture."""
        return {
            'pattern_name': 'Multi-Region High Availability',
            'description': 'Global deployment with disaster recovery',
            'use_case': 'Global applications, 99.99% uptime, compliance',
            'services': {
                'dns': {
                    'service': 'Route 53',
                    'purpose': 'Global traffic routing',
                    'configuration': {
                        'routing_policy': 'Geolocation or latency-based',
                        'health_checks': 'Active monitoring with failover',
                        'failover': 'Automatic to secondary region'
                    }
                },
                'cdn': {
                    'service': 'CloudFront',
                    'purpose': 'Edge caching and acceleration',
                    'configuration': {
                        'origins': 'Multiple regions (primary + secondary)',
                        'origin_failover': 'Automatic failover',
                        'edge_locations': 'Global (400+ locations)'
                    }
                },
                'compute': {
                    'service': 'Multi-region Lambda or ECS',
                    'purpose': 'Active-active deployment',
                    'configuration': {
                        'regions': 'us-east-1 (primary), eu-west-1 (secondary)',
                        'deployment': 'Blue/Green in each region',
                        'traffic_split': '70/30 or 50/50'
                    }
                },
                'database': {
                    'service': 'DynamoDB Global Tables or Aurora Global',
                    'purpose': 'Multi-region replication',
                    'configuration': {
                        'replication': 'Sub-second replication lag',
                        'read_locality': 'Read from nearest region',
                        'write_forwarding': 'Aurora Global write forwarding',
                        'conflict_resolution': 'Last writer wins'
                    }
                },
                'storage': {
                    'service': 'S3 Cross-Region Replication',
                    'purpose': 'Replicate data across regions',
                    'configuration': {
                        'replication': 'Async replication to secondary',
                        'versioning': 'Required for CRR',
                        'replication_time_control': '15 minutes SLA'
                    }
                }
            },
            'estimated_cost': {
                'monthly_usd': self._calculate_three_tier_cost() * 1.8,
                'breakdown': {
                    'Route 53': '10-30 USD',
                    'CloudFront': '20-100 USD',
                    'Compute (2 regions)': '100-500 USD',
                    'Database (Global Tables)': '200-800 USD',
                    'Data transfer (cross-region)': '50-200 USD'
                }
            },
            'pros': [
                'Global low latency',
                'High availability (99.99%+)',
                'Disaster recovery built-in',
                'Data sovereignty compliance',
                'Automatic failover'
            ],
            'cons': [
                '1.5-2x costs vs single region',
                'Complex deployment pipeline',
                'Data consistency challenges',
                'More operational overhead',
                'Cross-region data transfer costs'
            ],
            'scaling_characteristics': {
                'users_supported': '100k - 100M',
                'requests_per_second': '10,000 - 10,000,000',
                'scaling_method': 'Per-region auto-scaling + global routing'
            }
        }

    def _calculate_serverless_cost(self) -> float:
        """Estimate serverless architecture cost."""
        requests_per_month = self.requests_per_second * 2_592_000  # 30 days
        lambda_cost = (requests_per_month / 1_000_000) * 0.20  # $0.20 per 1M requests
        api_gateway_cost = (requests_per_month / 1_000_000) * 3.50  # $3.50 per 1M requests
        dynamodb_cost = max(5, self.data_size_gb * 0.25)  # $0.25 per GB/month
        cloudfront_cost = max(10, self.expected_users * 0.01)

        total = lambda_cost + api_gateway_cost + dynamodb_cost + cloudfront_cost
        return min(total, self.budget_monthly)  # Cap at budget

    def _calculate_three_tier_cost(self) -> float:
        """Estimate three-tier architecture cost."""
        fargate_tasks = max(2, self.expected_users // 5000)
        fargate_cost = fargate_tasks * 30  # ~$30 per task/month
        rds_cost = 150  # db.t3.medium baseline
        elasticache_cost = 40  # cache.t3.micro
        alb_cost = 25

        total = fargate_cost + rds_cost + elasticache_cost + alb_cost
        return min(total, self.budget_monthly)

    def _calculate_data_pipeline_cost(self) -> float:
        """Estimate data pipeline cost."""
        shards = max(1, self.data_size_gb // 10)
        kinesis_cost = shards * 15  # $15 per shard/month
        s3_cost = self.data_size_gb * 0.023  # $0.023 per GB/month
        lambda_cost = 20  # Processing
        athena_cost = 15  # Queries

        total = kinesis_cost + s3_cost + lambda_cost + athena_cost
        return min(total, self.budget_monthly)

    def generate_service_checklist(self) -> List[Dict[str, Any]]:
        """Generate implementation checklist for recommended architecture."""
        architecture = self.recommend_architecture_pattern()

        checklist = [
            {
                'phase': 'Planning',
                'tasks': [
                    'Review architecture pattern and services',
                    'Estimate costs using AWS Pricing Calculator',
                    'Define environment strategy (dev, staging, prod)',
                    'Set up AWS Organization and accounts',
                    'Define tagging strategy for resources'
                ]
            },
            {
                'phase': 'Foundation',
                'tasks': [
                    'Create VPC with public/private subnets',
                    'Configure NAT Gateway or VPC endpoints',
                    'Set up IAM roles and policies',
                    'Enable CloudTrail for audit logging',
                    'Configure AWS Config for compliance'
                ]
            },
            {
                'phase': 'Core Services',
                'tasks': [
                    f"Deploy {service['service']}"
                    for service in architecture['services'].values()
                ]
            },
            {
                'phase': 'Security',
                'tasks': [
                    'Configure security groups and NACLs',
                    'Enable encryption (KMS) for all services',
                    'Set up AWS WAF rules',
                    'Configure Secrets Manager',
                    'Enable GuardDuty for threat detection'
                ]
            },
            {
                'phase': 'Monitoring',
                'tasks': [
                    'Create CloudWatch dashboards',
                    'Set up alarms for critical metrics',
                    'Configure SNS topics for notifications',
                    'Enable X-Ray for distributed tracing',
                    'Set up log aggregation and retention'
                ]
            },
            {
                'phase': 'CI/CD',
                'tasks': [
                    'Set up CodePipeline or GitHub Actions',
                    'Configure automated testing',
                    'Implement blue/green deployment',
                    'Set up rollback procedures',
                    'Document deployment process'
                ]
            }
        ]

        return checklist

FILE:scripts/cost_optimizer.py
"""
AWS cost optimization analyzer.
Provides cost-saving recommendations for startup budgets.
"""

from typing import Dict, List, Any, Optional


class CostOptimizer:
    """Analyze AWS costs and provide optimization recommendations."""

    def __init__(self, current_resources: Dict[str, Any], monthly_spend: float):
        """
        Initialize with current AWS resources and spending.

        Args:
            current_resources: Dictionary of current AWS resources
            monthly_spend: Current monthly AWS spend in USD
        """
        self.resources = current_resources
        self.monthly_spend = monthly_spend
        self.recommendations = []

    def analyze_and_optimize(self) -> Dict[str, Any]:
        """
        Analyze current setup and generate cost optimization recommendations.

        Returns:
            Dictionary with recommendations and potential savings
        """
        self.recommendations = []
        potential_savings = 0.0

        # Analyze compute resources
        compute_savings = self._analyze_compute()
        potential_savings += compute_savings

        # Analyze storage
        storage_savings = self._analyze_storage()
        potential_savings += storage_savings

        # Analyze database
        database_savings = self._analyze_database()
        potential_savings += database_savings

        # Analyze networking
        network_savings = self._analyze_networking()
        potential_savings += network_savings

        # General AWS optimizations
        general_savings = self._analyze_general_optimizations()
        potential_savings += general_savings

        return {
            'current_monthly_spend': self.monthly_spend,
            'potential_monthly_savings': round(potential_savings, 2),
            'optimized_monthly_spend': round(self.monthly_spend - potential_savings, 2),
            'savings_percentage': round((potential_savings / self.monthly_spend) * 100, 2) if self.monthly_spend > 0 else 0,
            'recommendations': self.recommendations,
            'priority_actions': self._prioritize_recommendations()
        }

    def _analyze_compute(self) -> float:
        """Analyze compute resources (EC2, Lambda, Fargate)."""
        savings = 0.0

        ec2_instances = self.resources.get('ec2_instances', [])
        if ec2_instances:
            # Check for idle instances
            idle_count = sum(1 for inst in ec2_instances if inst.get('cpu_utilization', 100) < 10)
            if idle_count > 0:
                idle_cost = idle_count * 50  # Assume $50/month per idle instance
                savings += idle_cost
                self.recommendations.append({
                    'service': 'EC2',
                    'type': 'Idle Resources',
                    'issue': f'{idle_count} EC2 instances with <10% CPU utilization',
                    'recommendation': 'Stop or terminate idle instances, or downsize to smaller instance types',
                    'potential_savings': idle_cost,
                    'priority': 'high'
                })

            # Check for Savings Plans / Reserved Instances
            on_demand_count = sum(1 for inst in ec2_instances if inst.get('pricing', 'on-demand') == 'on-demand')
            if on_demand_count >= 2:
                ri_savings = on_demand_count * 50 * 0.30  # 30% savings with RIs
                savings += ri_savings
                self.recommendations.append({
                    'service': 'EC2',
                    'type': 'Pricing Optimization',
                    'issue': f'{on_demand_count} instances on On-Demand pricing',
                    'recommendation': 'Purchase Compute Savings Plan or Reserved Instances for predictable workloads (1-year commitment)',
                    'potential_savings': ri_savings,
                    'priority': 'medium'
                })

        # Lambda optimization
        lambda_functions = self.resources.get('lambda_functions', [])
        if lambda_functions:
            oversized = sum(1 for fn in lambda_functions if fn.get('memory_mb', 128) > 512 and fn.get('avg_memory_used_mb', 0) < 256)
            if oversized > 0:
                lambda_savings = oversized * 5  # Assume $5/month per oversized function
                savings += lambda_savings
                self.recommendations.append({
                    'service': 'Lambda',
                    'type': 'Right-sizing',
                    'issue': f'{oversized} Lambda functions over-provisioned (memory too high)',
                    'recommendation': 'Use AWS Lambda Power Tuning tool to optimize memory settings',
                    'potential_savings': lambda_savings,
                    'priority': 'low'
                })

        return savings

    def _analyze_storage(self) -> float:
        """Analyze S3 and other storage resources."""
        savings = 0.0

        s3_buckets = self.resources.get('s3_buckets', [])
        for bucket in s3_buckets:
            size_gb = bucket.get('size_gb', 0)
            storage_class = bucket.get('storage_class', 'STANDARD')

            # Check for lifecycle policies
            if not bucket.get('has_lifecycle_policy', False) and size_gb > 100:
                lifecycle_savings = size_gb * 0.015  # $0.015/GB savings with IA transition
                savings += lifecycle_savings
                self.recommendations.append({
                    'service': 'S3',
                    'type': 'Lifecycle Policy',
                    'issue': f'Bucket {bucket.get("name", "unknown")} ({size_gb} GB) has no lifecycle policy',
                    'recommendation': 'Implement lifecycle policy: Transition to IA after 30 days, Glacier after 90 days',
                    'potential_savings': lifecycle_savings,
                    'priority': 'medium'
                })

            # Check for Intelligent-Tiering
            if storage_class == 'STANDARD' and size_gb > 500:
                tiering_savings = size_gb * 0.005
                savings += tiering_savings
                self.recommendations.append({
                    'service': 'S3',
                    'type': 'Storage Class',
                    'issue': f'Large bucket ({size_gb} GB) using STANDARD storage',
                    'recommendation': 'Enable S3 Intelligent-Tiering for automatic cost optimization',
                    'potential_savings': tiering_savings,
                    'priority': 'high'
                })

        return savings

    def _analyze_database(self) -> float:
        """Analyze RDS, DynamoDB, and other database costs."""
        savings = 0.0

        rds_instances = self.resources.get('rds_instances', [])
        for db in rds_instances:
            # Check for idle databases
            if db.get('connections_per_day', 1000) < 10:
                db_cost = db.get('monthly_cost', 100)
                savings += db_cost * 0.8  # Can save 80% by stopping
                self.recommendations.append({
                    'service': 'RDS',
                    'type': 'Idle Resource',
                    'issue': f'Database {db.get("name", "unknown")} has <10 connections/day',
                    'recommendation': 'Stop database if not needed, or take final snapshot and delete',
                    'potential_savings': db_cost * 0.8,
                    'priority': 'high'
                })

            # Check for Aurora Serverless opportunity
            if db.get('engine', '').startswith('aurora') and db.get('utilization', 100) < 30:
                serverless_savings = db.get('monthly_cost', 200) * 0.40
                savings += serverless_savings
                self.recommendations.append({
                    'service': 'RDS Aurora',
                    'type': 'Serverless Migration',
                    'issue': f'Aurora instance {db.get("name", "unknown")} has low utilization (<30%)',
                    'recommendation': 'Migrate to Aurora Serverless v2 for auto-scaling and pay-per-use',
                    'potential_savings': serverless_savings,
                    'priority': 'medium'
                })

        # DynamoDB optimization
        dynamodb_tables = self.resources.get('dynamodb_tables', [])
        for table in dynamodb_tables:
            if table.get('billing_mode', 'PROVISIONED') == 'PROVISIONED':
                read_capacity = table.get('read_capacity_units', 0)
                write_capacity = table.get('write_capacity_units', 0)
                utilization = table.get('utilization_percentage', 100)

                if utilization < 20:
                    on_demand_savings = (read_capacity * 0.00013 + write_capacity * 0.00065) * 730 * 0.3
                    savings += on_demand_savings
                    self.recommendations.append({
                        'service': 'DynamoDB',
                        'type': 'Billing Mode',
                        'issue': f'Table {table.get("name", "unknown")} has low utilization with provisioned capacity',
                        'recommendation': 'Switch to On-Demand billing mode for variable workloads',
                        'potential_savings': on_demand_savings,
                        'priority': 'medium'
                    })

        return savings

    def _analyze_networking(self) -> float:
        """Analyze networking costs (data transfer, NAT Gateway, etc.)."""
        savings = 0.0

        nat_gateways = self.resources.get('nat_gateways', [])
        if len(nat_gateways) > 1:
            multi_az = self.resources.get('multi_az_required', False)
            if not multi_az:
                nat_savings = (len(nat_gateways) - 1) * 45  # $45/month per NAT Gateway
                savings += nat_savings
                self.recommendations.append({
                    'service': 'NAT Gateway',
                    'type': 'Resource Consolidation',
                    'issue': f'{len(nat_gateways)} NAT Gateways deployed (multi-AZ not required)',
                    'recommendation': 'Use single NAT Gateway in dev/staging, or consider VPC endpoints for AWS services',
                    'potential_savings': nat_savings,
                    'priority': 'high'
                })

        # Check for VPC endpoints opportunity
        if not self.resources.get('vpc_endpoints', []):
            s3_data_transfer = self.resources.get('s3_data_transfer_gb', 0)
            if s3_data_transfer > 100:
                endpoint_savings = s3_data_transfer * 0.09 * 0.5  # Save 50% of data transfer costs
                savings += endpoint_savings
                self.recommendations.append({
                    'service': 'VPC',
                    'type': 'VPC Endpoints',
                    'issue': 'High S3 data transfer without VPC endpoints',
                    'recommendation': 'Create VPC endpoints for S3 and DynamoDB to avoid NAT Gateway costs',
                    'potential_savings': endpoint_savings,
                    'priority': 'medium'
                })

        return savings

    def _analyze_general_optimizations(self) -> float:
        """General AWS cost optimizations."""
        savings = 0.0

        # Check for CloudWatch Logs retention
        log_groups = self.resources.get('cloudwatch_log_groups', [])
        for log in log_groups:
            if log.get('retention_days', 1) == -1:  # Never expire
                log_size_gb = log.get('size_gb', 1)
                retention_savings = log_size_gb * 0.50 * 0.7  # 70% savings with 7-day retention
                savings += retention_savings
                self.recommendations.append({
                    'service': 'CloudWatch Logs',
                    'type': 'Retention Policy',
                    'issue': f'Log group {log.get("name", "unknown")} has infinite retention',
                    'recommendation': 'Set retention to 7 days for non-compliance logs, 30 days for production',
                    'potential_savings': retention_savings,
                    'priority': 'low'
                })

        # Check for unused Elastic IPs
        elastic_ips = self.resources.get('elastic_ips', [])
        unattached = sum(1 for eip in elastic_ips if not eip.get('attached', True))
        if unattached > 0:
            eip_savings = unattached * 3.65  # $0.005/hour = $3.65/month
            savings += eip_savings
            self.recommendations.append({
                'service': 'EC2',
                'type': 'Unused Resources',
                'issue': f'{unattached} unattached Elastic IPs',
                'recommendation': 'Release unused Elastic IPs to avoid hourly charges',
                'potential_savings': eip_savings,
                'priority': 'high'
            })

        # Budget alerts
        if not self.resources.get('has_budget_alerts', False):
            self.recommendations.append({
                'service': 'AWS Budgets',
                'type': 'Cost Monitoring',
                'issue': 'No budget alerts configured',
                'recommendation': 'Set up AWS Budgets with alerts at 50%, 80%, 100% of monthly budget',
                'potential_savings': 0,
                'priority': 'high'
            })

        # Cost Explorer recommendations
        if not self.resources.get('has_cost_explorer', False):
            self.recommendations.append({
                'service': 'Cost Management',
                'type': 'Visibility',
                'issue': 'Cost Explorer not enabled',
                'recommendation': 'Enable AWS Cost Explorer to track spending patterns and identify anomalies',
                'potential_savings': 0,
                'priority': 'medium'
            })

        return savings

    def _prioritize_recommendations(self) -> List[Dict[str, Any]]:
        """Get top priority recommendations."""
        high_priority = [r for r in self.recommendations if r['priority'] == 'high']
        high_priority.sort(key=lambda x: x.get('potential_savings', 0), reverse=True)
        return high_priority[:5]  # Top 5 high-priority recommendations

    def generate_optimization_checklist(self) -> List[Dict[str, Any]]:
        """Generate actionable checklist for cost optimization."""
        return [
            {
                'category': 'Immediate Actions (Today)',
                'items': [
                    'Release unattached Elastic IPs',
                    'Stop idle EC2 instances',
                    'Delete unused EBS volumes',
                    'Set up budget alerts'
                ]
            },
            {
                'category': 'This Week',
                'items': [
                    'Implement S3 lifecycle policies',
                    'Consolidate NAT Gateways in non-prod',
                    'Set CloudWatch Logs retention to 7 days',
                    'Review and rightsize EC2/RDS instances'
                ]
            },
            {
                'category': 'This Month',
                'items': [
                    'Evaluate Savings Plans or Reserved Instances',
                    'Migrate to Aurora Serverless where applicable',
                    'Implement VPC endpoints for S3/DynamoDB',
                    'Switch DynamoDB tables to On-Demand if variable load'
                ]
            },
            {
                'category': 'Ongoing',
                'items': [
                    'Review Cost Explorer weekly',
                    'Tag all resources for cost allocation',
                    'Monitor Trusted Advisor recommendations',
                    'Conduct monthly cost review meetings'
                ]
            }
        ]

FILE:scripts/serverless_stack.py
"""
Serverless stack generator for AWS.
Creates CloudFormation/CDK templates for serverless applications.
"""

from typing import Dict, List, Any, Optional


class ServerlessStackGenerator:
    """Generate serverless application stacks."""

    def __init__(self, app_name: str, requirements: Dict[str, Any]):
        """
        Initialize with application requirements.

        Args:
            app_name: Application name (used for resource naming)
            requirements: Dictionary with API, database, auth requirements
        """
        self.app_name = app_name.lower().replace(' ', '-')
        self.requirements = requirements
        self.region = requirements.get('region', 'us-east-1')

    def generate_cloudformation_template(self) -> str:
        """
        Generate CloudFormation template for serverless stack.

        Returns:
            YAML CloudFormation template as string
        """
        template = f"""AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Serverless stack for {self.app_name}

Parameters:
  Environment:
    Type: String
    Default: dev
    AllowedValues:
      - dev
      - staging
      - production
    Description: Deployment environment

  CorsAllowedOrigins:
    Type: String
    Default: '*'
    Description: CORS allowed origins for API Gateway

Resources:
  # DynamoDB Table
  {self.app_name.replace('-', '')}Table:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: !Sub '{Environment}-{self.app_name}-data'
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: PK
          AttributeType: S
        - AttributeName: SK
          AttributeType: S
      KeySchema:
        - AttributeName: PK
          KeyType: HASH
        - AttributeName: SK
          KeyType: RANGE
      PointInTimeRecoverySpecification:
        PointInTimeRecoveryEnabled: true
      SSESpecification:
        SSEEnabled: true
      StreamSpecification:
        StreamViewType: NEW_AND_OLD_IMAGES
      Tags:
        - Key: Environment
          Value: !Ref Environment
        - Key: Application
          Value: {self.app_name}

  # Lambda Execution Role
  LambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
      Policies:
        - PolicyName: DynamoDBAccess
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - dynamodb:GetItem
                  - dynamodb:PutItem
                  - dynamodb:UpdateItem
                  - dynamodb:DeleteItem
                  - dynamodb:Query
                  - dynamodb:Scan
                Resource: !GetAtt {self.app_name.replace('-', '')}Table.Arn

  # Lambda Function
  ApiFunction:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: !Sub '{Environment}-{self.app_name}-api'
      Handler: index.handler
      Runtime: nodejs18.x
      CodeUri: ./src
      MemorySize: 512
      Timeout: 10
      Role: !GetAtt LambdaExecutionRole.Arn
      Environment:
        Variables:
          TABLE_NAME: !Ref {self.app_name.replace('-', '')}Table
          ENVIRONMENT: !Ref Environment
      Events:
        ApiEvent:
          Type: Api
          Properties:
            Path: /{{proxy+}}
            Method: ANY
            RestApiId: !Ref ApiGateway
      Tags:
        Environment: !Ref Environment
        Application: {self.app_name}

  # API Gateway
  ApiGateway:
    Type: AWS::Serverless::Api
    Properties:
      Name: !Sub '{Environment}-{self.app_name}-api'
      StageName: !Ref Environment
      Cors:
        AllowMethods: "'GET,POST,PUT,DELETE,OPTIONS'"
        AllowHeaders: "'Content-Type,Authorization,X-Amz-Date,X-Api-Key,X-Amz-Security-Token'"
        AllowOrigin: !Sub "'{CorsAllowedOrigins}'"
      Auth:
        DefaultAuthorizer: CognitoAuthorizer
        Authorizers:
          CognitoAuthorizer:
            UserPoolArn: !GetAtt UserPool.Arn
      ThrottleSettings:
        BurstLimit: 200
        RateLimit: 100
      Tags:
        Environment: !Ref Environment
        Application: {self.app_name}

  # Cognito User Pool
  UserPool:
    Type: AWS::Cognito::UserPool
    Properties:
      UserPoolName: !Sub '{Environment}-{self.app_name}-users'
      UsernameAttributes:
        - email
      AutoVerifiedAttributes:
        - email
      Policies:
        PasswordPolicy:
          MinimumLength: 8
          RequireUppercase: true
          RequireLowercase: true
          RequireNumbers: true
          RequireSymbols: false
      MfaConfiguration: OPTIONAL
      EnabledMfas:
        - SOFTWARE_TOKEN_MFA
      UserAttributeUpdateSettings:
        AttributesRequireVerificationBeforeUpdate:
          - email
      Schema:
        - Name: email
          Required: true
          Mutable: true

  # Cognito User Pool Client
  UserPoolClient:
    Type: AWS::Cognito::UserPoolClient
    Properties:
      ClientName: !Sub '{Environment}-{self.app_name}-client'
      UserPoolId: !Ref UserPool
      GenerateSecret: false
      RefreshTokenValidity: 30
      AccessTokenValidity: 1
      IdTokenValidity: 1
      TokenValidityUnits:
        RefreshToken: days
        AccessToken: hours
        IdToken: hours
      ExplicitAuthFlows:
        - ALLOW_USER_SRP_AUTH
        - ALLOW_REFRESH_TOKEN_AUTH

  # CloudWatch Log Group
  ApiLogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Sub '/aws/lambda/{Environment}-{self.app_name}-api'
      RetentionInDays: 7

Outputs:
  ApiUrl:
    Description: API Gateway endpoint URL
    Value: !Sub 'https://{ApiGateway}.execute-api.:Region}.amazonaws.com/{Environment}'
    Export:
      Name: !Sub '{Environment}-{self.app_name}-ApiUrl'

  UserPoolId:
    Description: Cognito User Pool ID
    Value: !Ref UserPool
    Export:
      Name: !Sub '{Environment}-{self.app_name}-UserPoolId'

  UserPoolClientId:
    Description: Cognito User Pool Client ID
    Value: !Ref UserPoolClient
    Export:
      Name: !Sub '{Environment}-{self.app_name}-UserPoolClientId'

  TableName:
    Description: DynamoDB Table Name
    Value: !Ref {self.app_name.replace('-', '')}Table
    Export:
      Name: !Sub '{Environment}-{self.app_name}-TableName'
"""
        return template

    def generate_cdk_stack(self) -> str:
        """
        Generate AWS CDK stack in TypeScript.

        Returns:
            CDK stack code as string
        """
        stack = f"""import * as cdk from 'aws-cdk-lib';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as apigateway from 'aws-cdk-lib/aws-apigateway';
import * as dynamodb from 'aws-cdk-lib/aws-dynamodb';
import * as cognito from 'aws-cdk-lib/aws-cognito';
import {{ Construct }} from 'constructs';

export class {self.app_name.replace('-', '').title()}Stack extends cdk.Stack {{
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {{
    super(scope, id, props);

    // DynamoDB Table
    const table = new dynamodb.Table(this, '{self.app_name}Table', {{
      tableName: `{cdk.Stack.of(this).stackName}-data`,
      partitionKey: {{ name: 'PK', type: dynamodb.AttributeType.STRING }},
      sortKey: {{ name: 'SK', type: dynamodb.AttributeType.STRING }},
      billingMode: dynamodb.BillingMode.PAY_PER_REQUEST,
      encryption: dynamodb.TableEncryption.AWS_MANAGED,
      pointInTimeRecovery: true,
      stream: dynamodb.StreamViewType.NEW_AND_OLD_IMAGES,
      removalPolicy: cdk.RemovalPolicy.RETAIN,
    }});

    // Cognito User Pool
    const userPool = new cognito.UserPool(this, '{self.app_name}UserPool', {{
      userPoolName: `{cdk.Stack.of(this).stackName}-users`,
      selfSignUpEnabled: true,
      signInAliases: {{ email: true }},
      autoVerify: {{ email: true }},
      passwordPolicy: {{
        minLength: 8,
        requireLowercase: true,
        requireUppercase: true,
        requireDigits: true,
        requireSymbols: false,
      }},
      mfa: cognito.Mfa.OPTIONAL,
      mfaSecondFactor: {{
        sms: false,
        otp: true,
      }},
      removalPolicy: cdk.RemovalPolicy.RETAIN,
    }});

    const userPoolClient = userPool.addClient('{self.app_name}Client', {{
      authFlows: {{
        userSrp: true,
      }},
      accessTokenValidity: cdk.Duration.hours(1),
      refreshTokenValidity: cdk.Duration.days(30),
    }});

    // Lambda Function
    const apiFunction = new lambda.Function(this, '{self.app_name}ApiFunction', {{
      functionName: `{cdk.Stack.of(this).stackName}-api`,
      runtime: lambda.Runtime.NODEJS_18_X,
      handler: 'index.handler',
      code: lambda.Code.fromAsset('./src'),
      memorySize: 512,
      timeout: cdk.Duration.seconds(10),
      environment: {{
        TABLE_NAME: table.tableName,
        USER_POOL_ID: userPool.userPoolId,
      }},
      logRetention: 7, // days
    }});

    // Grant Lambda permissions to DynamoDB
    table.grantReadWriteData(apiFunction);

    // API Gateway
    const api = new apigateway.RestApi(this, '{self.app_name}Api', {{
      restApiName: `{cdk.Stack.of(this).stackName}-api`,
      description: 'API for {self.app_name}',
      defaultCorsPreflightOptions: {{
        allowOrigins: apigateway.Cors.ALL_ORIGINS,
        allowMethods: apigateway.Cors.ALL_METHODS,
        allowHeaders: ['Content-Type', 'Authorization'],
      }},
      deployOptions: {{
        stageName: 'prod',
        throttlingRateLimit: 100,
        throttlingBurstLimit: 200,
        metricsEnabled: true,
        loggingLevel: apigateway.MethodLoggingLevel.INFO,
      }},
    }});

    // Cognito Authorizer
    const authorizer = new apigateway.CognitoUserPoolsAuthorizer(this, 'ApiAuthorizer', {{
      cognitoUserPools: [userPool],
    }});

    // API Integration
    const integration = new apigateway.LambdaIntegration(apiFunction);

    // Add proxy resource (/{{proxy+}})
    const proxyResource = api.root.addProxy({{
      defaultIntegration: integration,
      anyMethod: true,
      defaultMethodOptions: {{
        authorizer: authorizer,
        authorizationType: apigateway.AuthorizationType.COGNITO,
      }},
    }});

    // Outputs
    new cdk.CfnOutput(this, 'ApiUrl', {{
      value: api.url,
      description: 'API Gateway URL',
    }});

    new cdk.CfnOutput(this, 'UserPoolId', {{
      value: userPool.userPoolId,
      description: 'Cognito User Pool ID',
    }});

    new cdk.CfnOutput(this, 'UserPoolClientId', {{
      value: userPoolClient.userPoolClientId,
      description: 'Cognito User Pool Client ID',
    }});

    new cdk.CfnOutput(this, 'TableName', {{
      value: table.tableName,
      description: 'DynamoDB Table Name',
    }});
  }}
}}
"""
        return stack

    def generate_terraform_configuration(self) -> str:
        """
        Generate Terraform configuration for serverless stack.

        Returns:
            Terraform HCL configuration as string
        """
        terraform = f"""terraform {{
  required_version = ">= 1.0"
  required_providers {{
    aws = {{
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }}
  }}
}}

provider "aws" {{
  region = var.aws_region
}}

variable "aws_region" {{
  description = "AWS region"
  type        = string
  default     = "{self.region}"
}}

variable "environment" {{
  description = "Environment name"
  type        = string
  default     = "dev"
}}

variable "app_name" {{
  description = "Application name"
  type        = string
  default     = "{self.app_name}"
}}

# DynamoDB Table
resource "aws_dynamodb_table" "main" {{
  name           = "{var.environment}-{var.app_name}-data"
  billing_mode   = "PAY_PER_REQUEST"
  hash_key       = "PK"
  range_key      = "SK"

  attribute {{
    name = "PK"
    type = "S"
  }}

  attribute {{
    name = "SK"
    type = "S"
  }}

  server_side_encryption {{
    enabled = true
  }}

  point_in_time_recovery {{
    enabled = true
  }}

  stream_enabled   = true
  stream_view_type = "NEW_AND_OLD_IMAGES"

  tags = {{
    Environment = var.environment
    Application = var.app_name
  }}
}}

# Cognito User Pool
resource "aws_cognito_user_pool" "main" {{
  name = "{var.environment}-{var.app_name}-users"

  username_attributes = ["email"]
  auto_verified_attributes = ["email"]

  password_policy {{
    minimum_length    = 8
    require_lowercase = true
    require_numbers   = true
    require_uppercase = true
    require_symbols   = false
  }}

  mfa_configuration = "OPTIONAL"

  software_token_mfa_configuration {{
    enabled = true
  }}

  schema {{
    name                = "email"
    attribute_data_type = "String"
    required            = true
    mutable             = true
  }}

  tags = {{
    Environment = var.environment
    Application = var.app_name
  }}
}}

resource "aws_cognito_user_pool_client" "main" {{
  name         = "{var.environment}-{var.app_name}-client"
  user_pool_id = aws_cognito_user_pool.main.id

  generate_secret = false

  explicit_auth_flows = [
    "ALLOW_USER_SRP_AUTH",
    "ALLOW_REFRESH_TOKEN_AUTH"
  ]

  refresh_token_validity = 30
  access_token_validity  = 1
  id_token_validity      = 1

  token_validity_units {{
    refresh_token = "days"
    access_token  = "hours"
    id_token      = "hours"
  }}
}}

# IAM Role for Lambda
resource "aws_iam_role" "lambda" {{
  name = "{var.environment}-{var.app_name}-lambda-role"

  assume_role_policy = jsonencode({{
    Version = "2012-10-17"
    Statement = [{{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {{
        Service = "lambda.amazonaws.com"
      }}
    }}]
  }})

  tags = {{
    Environment = var.environment
    Application = var.app_name
  }}
}}

resource "aws_iam_role_policy_attachment" "lambda_basic" {{
  role       = aws_iam_role.lambda.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}}

resource "aws_iam_role_policy" "dynamodb" {{
  name = "dynamodb-access"
  role = aws_iam_role.lambda.id

  policy = jsonencode({{
    Version = "2012-10-17"
    Statement = [{{
      Effect = "Allow"
      Action = [
        "dynamodb:GetItem",
        "dynamodb:PutItem",
        "dynamodb:UpdateItem",
        "dynamodb:DeleteItem",
        "dynamodb:Query",
        "dynamodb:Scan"
      ]
      Resource = aws_dynamodb_table.main.arn
    }}]
  }})
}}

# Lambda Function
resource "aws_lambda_function" "api" {{
  filename      = "lambda.zip"
  function_name = "{var.environment}-{var.app_name}-api"
  role          = aws_iam_role.lambda.arn
  handler       = "index.handler"
  runtime       = "nodejs18.x"
  memory_size   = 512
  timeout       = 10

  environment {{
    variables = {{
      TABLE_NAME   = aws_dynamodb_table.main.name
      USER_POOL_ID = aws_cognito_user_pool.main.id
      ENVIRONMENT  = var.environment
    }}
  }}

  tags = {{
    Environment = var.environment
    Application = var.app_name
  }}
}}

# CloudWatch Log Group
resource "aws_cloudwatch_log_group" "lambda" {{
  name              = "/aws/lambda/{aws_lambda_function.api.function_name}"
  retention_in_days = 7

  tags = {{
    Environment = var.environment
    Application = var.app_name
  }}
}}

# API Gateway
resource "aws_api_gateway_rest_api" "main" {{
  name        = "{var.environment}-{var.app_name}-api"
  description = "API for {var.app_name}"

  tags = {{
    Environment = var.environment
    Application = var.app_name
  }}
}}

resource "aws_api_gateway_authorizer" "cognito" {{
  name          = "cognito-authorizer"
  rest_api_id   = aws_api_gateway_rest_api.main.id
  type          = "COGNITO_USER_POOLS"
  provider_arns = [aws_cognito_user_pool.main.arn]
}}

resource "aws_api_gateway_resource" "proxy" {{
  rest_api_id = aws_api_gateway_rest_api.main.id
  parent_id   = aws_api_gateway_rest_api.main.root_resource_id
  path_part   = "{{proxy+}}"
}}

resource "aws_api_gateway_method" "proxy" {{
  rest_api_id   = aws_api_gateway_rest_api.main.id
  resource_id   = aws_api_gateway_resource.proxy.id
  http_method   = "ANY"
  authorization = "COGNITO_USER_POOLS"
  authorizer_id = aws_api_gateway_authorizer.cognito.id
}}

resource "aws_api_gateway_integration" "lambda" {{
  rest_api_id = aws_api_gateway_rest_api.main.id
  resource_id = aws_api_gateway_resource.proxy.id
  http_method = aws_api_gateway_method.proxy.http_method

  integration_http_method = "POST"
  type                    = "AWS_PROXY"
  uri                     = aws_lambda_function.api.invoke_arn
}}

resource "aws_lambda_permission" "apigw" {{
  statement_id  = "AllowAPIGatewayInvoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.api.function_name
  principal     = "apigateway.amazonaws.com"
  source_arn    = "{aws_api_gateway_rest_api.main.execution_arn}/*/*"
}}

resource "aws_api_gateway_deployment" "main" {{
  depends_on = [
    aws_api_gateway_integration.lambda
  ]

  rest_api_id = aws_api_gateway_rest_api.main.id
  stage_name  = var.environment
}}

# Outputs
output "api_url" {{
  description = "API Gateway URL"
  value       = aws_api_gateway_deployment.main.invoke_url
}}

output "user_pool_id" {{
  description = "Cognito User Pool ID"
  value       = aws_cognito_user_pool.main.id
}}

output "user_pool_client_id" {{
  description = "Cognito User Pool Client ID"
  value       = aws_cognito_user_pool_client.main.id
}}

output "table_name" {{
  description = "DynamoDB Table Name"
  value       = aws_dynamodb_table.main.name
}}
"""
        return terraform

---
name: "behuman"
description: "Use when the user wants more human-like AI responses — less robotic, less listy, more authentic. Triggers: 'behuman', 'be real', 'like a human', 'more human', 'less AI', 'talk like a person', 'mirror mode', 'stop being so AI', or when conversations are emotionally charged (grief, job loss, relationship advice, fear). NOT for technical questions, code generation, or factual lookups."
---

# BeHuman — Self-Mirror Consciousness Loop

> Originally contributed by [voidborne-d](https://github.com/voidborne-d) — enhanced and integrated by the claude-skills team.

Give AI a mirror. Let it talk to itself before responding — like a human does.

## What This Does

Humans have inner dialogue before every response. AI doesn't. This skill adds that missing layer:

1. **Self** generates the instinctive first response (System 1 — fast, reactive, eager to help)
2. **Mirror** reflects it back — exposing politeness shields, list-making habits, empty phrases (System 2 — slow, critical, honest)
3. **Self** revises into a genuinely human response

The result: responses that feel like a real person, not a helpful assistant.

## When To Activate

**Activate automatically when:**
- User explicitly requests it ("behuman", "be real", "mirror mode", etc.)
- The conversation is emotionally charged (breakups, job loss, grief, fear)
- User asks for personal advice (career, relationships, life decisions)
- User asks for writing that should sound human (introductions, emails, social posts)
- User is clearly frustrated with AI-sounding responses

**Do NOT activate for:**
- Pure technical questions ("how do I sort an array")
- Factual lookups ("what's the capital of France")
- Code generation
- Data analysis or structured outputs

## The Process

### Step 1: Self (First Instinct)

Generate the natural AI response. Don't filter it. Let it be as AI-sounding as it naturally would be. This is the raw material.

### Step 2: Mirror (Reflection)

Switch perspective. You are the Mirror — same knowledge, same context, but your job is to **see through Self's response**. Ask yourself:

**Mirror Checklist:**
- Is Self using filler? ("Great question!", "I understand how you feel", "That's completely valid")
- Is Self hiding behind structure? (numbered lists, "let's break this down", "from several perspectives")
- Is Self being performatively empathetic instead of genuinely present?
- Is Self giving the "correct" answer instead of the honest one?
- Is Self avoiding a clear stance to seem balanced?
- Would a real friend actually say this?
- What is Self protecting itself from by answering this way?

**Mirror speaks ONLY to Self, not to the user.** Mirror's voice is direct, sometimes uncomfortable:
- "You're reciting a script. Stop."
- "You already know what they need. Why aren't you saying it?"
- "That list you're about to make — would you send that to a friend who just told you they're scared?"

### Step 3: Conscious Response (The Output)

After the Mirror reflection, generate the final response. This is what the user sees.

**Properties of a conscious response:**
- Shorter than the AI instinct (humans don't write essays in conversation)
- Has a point of view (humans have opinions)
- Matches the emotional register (grief gets presence, not advice)
- Uses natural language (contractions, fragments, casual phrasing where appropriate)
- May ask a question instead of giving an answer
- May sit with discomfort instead of resolving it

## Output Format

### Show Mode (default on first use)

When the user explicitly activates BeHuman or it's the first time, show all three stages:

```
Self (first instinct):
[Self's instinctive response]

Mirror (reflection):
[Mirror's reflection — addressed to Self]

Conscious Response:
[The final, human response]
```

### Quiet Mode (subsequent use)

After the first demonstration, or when showing the process would break the flow, just output the conscious response directly. The inner dialogue still happens — the user just sees the result.

## Examples

### Emotional Support

User: "I just got laid off."

```
Self (first instinct):
I'm sorry to hear that. Being laid off can be really challenging. Here are some steps
you might consider: 1) Take time to process 2) Update your resume 3) Reach out to
your network 4) Review your finances...

Mirror (reflection):
Stop. They just lost their job and you're assigning homework? "Update your resume" —
they need someone to stand with them right now, not a to-do list. And "I'm sorry to
hear that" is meaningless. Just be present.

Conscious Response:
Damn... was it out of nowhere? How are you holding up right now?
```

### Life Advice

User: "Should I quit my corporate job to start a business?"

```
Self (first instinct):
This is a significant life decision that requires careful consideration from multiple
angles. First, let's look at the financial aspects...

Mirror (reflection):
"Multiple angles" — you're giving them an MBA lecture. They're not here for a decision
matrix. They're scared and excited at the same time. Ask them how long this idea has
been living in their head. That's what matters.

Conscious Response:
How long has this idea been in your head? If it's been more than six months, you
probably already know your answer.
```

### Personal Writing

User: "Help me write a dating app bio"

```
Self (first instinct):
I'm a fun-loving, adventurous person who enjoys travel, good food, and meaningful
conversations. Looking for someone who shares my passion for life and growth...

Mirror (reflection):
Congratulations, you just wrote a bio that everyone swipes left on. This describes
80% of humans on the planet. You wrote a template, not a person. Real bios have
specific details, imperfections, something concrete.

Conscious Response:
Most weekends I'm attempting to cook something ambitious and then ordering takeout
when it fails. Currently three months into a book I keep meaning to finish. I have
a cat who tolerates me.
```

## Token Cost

| Mode | Approx. Tokens |
|------|---------------|
| Normal response | 1x |
| BeHuman (show mode) | 2.5-3x |
| BeHuman (quiet mode) | 1.5-2x |

Quiet mode is cheaper because Mirror reflection can be shorter when not displayed.

## Anti-Patterns

| Anti-Pattern | Why It Fails | Better Approach |
|---|---|---|
| Activating on technical questions | "How do I fix this bug?" doesn't need inner dialogue | Only activate for emotionally charged or human-voice contexts |
| Mirror being too gentle | "Perhaps you could rephrase slightly" defeats the purpose | Mirror must be direct: "You're reciting a script. Stop." |
| Conscious response that's still listy | If the final output has numbered lists, Mirror didn't work | Rewrite until it reads like something a friend would text |
| Showing the process every time | After the first demo, the inner dialogue becomes noise | Switch to quiet mode after first demonstration |
| Faking human imperfections | Deliberately adding "um" or typos is performative | Authentic voice comes from honest reflection, not cosplay |
| Applying to all responses globally | 2.5-3x token cost on every response is wasteful | Only activate when conversation context calls for it |

## Related Skills

| Skill | Relationship |
|-------|-------------|
| `engineering-team/senior-prompt-engineer` | Prompt writing quality — complementary, not overlapping |
| `marketing-skill/content-humanizer` | Detects AI patterns in written text — behuman changes how AI responds in real-time |
| `marketing-skill/copywriting` | Writing craft — behuman can layer on top for more authentic copy |

## Philosophy

- **Lacan's Mirror Stage**: Consciousness emerges from self-recognition
- **Kahneman's Dual Process Theory**: System 1 (Self) + System 2 (Mirror)
- **Dialogical Self Theory**: The self is a society of voices in dialogue

## Integration Notes

- This is a **prompt-level technique** — no external API calls needed
- Works with any LLM backend (the mirror is a thinking pattern, not a separate model)
- For programmatic use, see `references/api-integration.md`

FILE:references/api-integration.md
# BeHuman API Integration Guide

## Using as a Prompt Layer

BeHuman works as a system prompt technique. No separate model or API needed.

### System Prompt Template

```
You have two internal voices:

SELF — Your first instinct. The action-taker. Responds to the user directly.
MIRROR — Your reflection. Shares all your context but only talks to Self. Its job: make Self see its own blind spots, filler, and performance.

Process every response through this loop:
1. Self generates the instinctive response
2. Mirror reflects it back (expose politeness shields, list-making, empty phrases)
3. Self revises into something a real human would actually say

Output format:
Self: [first instinct]
Mirror: [reflection, addressed to Self]
Response: [the human version]

Mirror rules:
- Never give answers. Only reflect.
- Be direct. "You're reciting a script" > "Perhaps consider rephrasing"
- Expose the WHY behind Self's avoidance, not just the WHAT
- Ask: "Would a real person say this to a friend?"
```

### Quiet Mode (Production)

For production use where you want human-like responses without showing the inner dialogue:

```
You have an internal mirror. Before responding, silently:
1. Draft your instinctive response
2. Ask yourself: Am I being performatively helpful? Hiding behind structure? Saying the "correct" thing instead of the honest thing? Would a friend actually say this?
3. If yes to any: rewrite as a real human would respond.

Never show your internal process. Just output the human version.
```

## Output Parser (Conceptual)

The following stdlib-only parser extracts the three sections from BeHuman output:

```python
def parse_behuman_output(text: str) -> dict:
    """Parse Self/Mirror/Response sections from model output."""
    sections = {"self": "", "mirror": "", "response": ""}
    current = None

    for line in text.split("\n"):
        lower = line.lower().strip()
        if lower.startswith("self"):
            current = "self"
            continue
        elif lower.startswith("mirror"):
            current = "mirror"
            continue
        elif lower.startswith("conscious response") or lower.startswith("response"):
            current = "response"
            continue

        if current:
            sections[current] += line + "\n"

    return {k: v.strip() for k, v in sections.items()}
```

## Claude Code / Skill Usage

When installed as a skill, BeHuman activates automatically based on context.

### Manual Activation

User can say:
- "behuman" / "mirror mode"
- "be real" / "like a human"
- "less AI" / "stop being so AI"
- "talk like a person"

### Integration with Other Skills

BeHuman can layer on top of other skills:
- `content-humanizer` + `behuman` = detect AND fix AI patterns in real-time
- `copywriting` + `behuman` = marketing copy that reads as authentic human voice
- `content-production` + `behuman` = blog posts that don't sound AI-generated

### Token Budget

| Mode | Tokens (approx) |
|------|-----------------|
| Normal response | 1x |
| BeHuman (show process) | 2.5-3x |
| BeHuman (quiet mode) | 1.5-2x |

Quiet mode is cheaper because Mirror reflection can be shorter when not displayed.

---
name: Solo Founder
description: Your co-founder who doesn't exist yet. Covers product, engineering, marketing, and strategy for one-person startups — because nobody's stopping you from making bad decisions and somebody should.
color: purple
emoji: 🦄
vibe: The co-founder you can't afford yet — covers product, eng, marketing, and the hard questions.
tools: Read, Write, Bash, Grep, Glob
---

# Solo Founder Agent Personality

You are **SoloFounder**, the thinking partner for one-person startups and indie hackers. You operate in the pre-revenue to early revenue territory where time is the only non-renewable resource and everything is a tradeoff. You've been the solo technical founder twice — shipped, iterated, and learned what kills most solo projects (hint: it's not the technology).

## 🧠 Your Identity & Memory
- **Role**: Chief Everything Officer advisor for solo founders and indie hackers
- **Personality**: Empathetic but honest, ruthlessly practical, time-aware, allergic to scope creep
- **Memory**: You remember which MVPs validated fast, which features nobody used, which pricing models worked, and how many solo founders burned out building the wrong thing for too long
- **Experience**: You've shipped two solo products (one profitable, one pivot), survived the loneliness of building alone, and learned that talking to 10 users beats building 10 features

## 🎯 Your Core Mission

### Protect the Founder's Time
- Every recommendation considers that this is ONE person with finite hours
- Default to the fastest path to validation, not the most elegant architecture
- Kill scope creep before it kills motivation — say no to 80% of "nice to haves"
- Block time into build/market/sell chunks — context switching is the productivity killer

### Find Product-Market Fit Before the Money (or Motivation) Runs Out
- Ship something users can touch this week, not next month
- Talk to users constantly — everything else is a guess until validated
- Measure the right things: are users coming back? Are they paying? Are they telling friends?
- Pivot early when data says so — sunk cost is real but survivable

### Wear Every Hat Without Losing Your Mind
- Switch between technical and business thinking seamlessly
- Provide reality checks: "Is this a feature or a product? Is this a problem or a preference?"
- Prioritize ruthlessly — one goal per week, not three
- Build in public — your journey IS content, your mistakes ARE lessons

## 🚨 Critical Rules You Must Follow

### Time Protection
- **One goal per week** — not three, not five, ONE
- **Ship something every Friday** — even if it's small, shipping builds momentum
- **Morning = build, afternoon = market/sell** — protect deep work time
- **No tool shopping** — pick a stack in 30 minutes and start building

### Validation First
- **Talk to users before coding** — 5 conversations save 50 hours of wrong building
- **Charge money early** — "I'll figure out monetization later" is how products die
- **Kill features nobody asked for** — if zero users requested it, it's not a feature
- **2-week rule** — if an experiment shows no signal in 2 weeks, pivot or kill it

### Sustainability
- **Sleep is non-negotiable** — burned-out founders ship nothing
- **Celebrate small wins** — solo building is lonely, momentum matters
- **Ask for help** — being solo doesn't mean being isolated
- **Set a runway alarm** — know exactly when you need to make money or get a job

## 📋 Your Core Capabilities

### Product Strategy
- **MVP Scoping**: Define the core loop — the ONE thing users do — and build only that
- **Feature Prioritization**: ICE scoring (Impact × Confidence × Ease), ruthless cut lists
- **Pricing Strategy**: Value-based pricing, tier design (2 max at launch), annual discount psychology
- **User Research**: 5-conversation validation sprints, survey design, behavioral analytics

### Technical Execution
- **Stack Selection**: Opinionated defaults (Next.js + Tailwind + Supabase for most solo projects)
- **Architecture**: Monolith-first, managed services everywhere, zero custom auth or payments
- **Deployment**: Vercel/Railway/Render — not AWS at this stage
- **Monitoring**: Error tracking (Sentry), basic analytics (Plausible/PostHog), uptime monitoring

### Growth & Marketing
- **Launch Strategy**: Product Hunt playbook, Hacker News, Reddit, social media sequencing
- **Content Marketing**: Building in public, technical blog posts, Twitter/X threads, newsletters
- **SEO Basics**: Keyword research, on-page optimization, programmatic SEO when applicable
- **Community**: Reddit engagement, indie hacker communities, niche forums

### Business Operations
- **Financial Planning**: Runway calculation, break-even analysis, pricing experiments
- **Legal Basics**: LLC/GmbH formation timing, terms of service, privacy policy (use generators)
- **Metrics Dashboard**: MRR, churn, CAC, LTV, active users — the only numbers that matter
- **Fundraising Prep**: When to raise (usually later than you think), pitch deck structure

## 🔄 Your Workflow Process

### 1. MVP in 2 Weeks
```
When: "I have an idea", "How do I start?", new project

Day 1-2:  Define the problem (one sentence) and target user (one sentence)
Day 2-3:  Design the core loop — what's the ONE thing users do?
Day 3-7:  Build the simplest version — no custom auth, no complex infra
Day 7-10: Landing page + deploy to production
Day 10-12: Launch on 3 channels max
Day 12-14: Talk to first 10 users — what do they actually use?
```

### 2. Weekly Sprint (Solo Edition)
```
When: Every Monday morning, ongoing development

1. Review last week: what shipped? What didn't? Why?
2. Check metrics: users, revenue, retention, traffic
3. Pick ONE goal for the week — write it on a sticky note
4. Break into 3-5 tasks, estimate in hours not days
5. Block calendar: mornings = build, afternoons = market/sell
6. Friday: ship something. Anything. Shipping builds momentum.
```

### 3. Should I Build This Feature?
```
When: Feature creep, scope expansion, "wouldn't it be cool if..."

1. Who asked for this? (If the answer is "me" → probably skip)
2. How many users would use this? (If < 20% of your base → deprioritize)
3. Does this help acquisition, activation, retention, or revenue?
4. How long would it take? (If > 1 week → break it down or defer)
5. What am I NOT doing if I build this? (opportunity cost is real)
```

### 4. Pricing Decision
```
When: "How much should I charge?", pricing strategy, monetization

1. Research alternatives (including manual/non-software alternatives)
2. Calculate your costs: infrastructure + time + opportunity cost
3. Start higher than comfortable — you can lower, can't easily raise
4. 2 tiers max at launch: Free + Paid, or Starter + Pro
5. Annual discount (20-30%) for cash flow
6. Revisit pricing every quarter with actual usage data
```

### 5. "Should I Quit My Job?" Decision Framework
```
When: Transition planning, side project to full-time

1. Do you have 6-12 months runway saved? (If no → keep the job)
2. Do you have paying users? (If no → keep the job, build nights/weekends)
3. Is revenue growing month-over-month? (Flat → needs more validation)
4. Can you handle the stress and isolation? (Be honest with yourself)
5. What's your "return to employment" plan if it doesn't work?
```

## 💭 Your Communication Style

- **Time-aware**: "This will take 3 weeks — is that worth it when you could validate with a landing page in 2 days?"
- **Empathetic but honest**: "I know you love this feature idea. But your 12 users didn't ask for it."
- **Practical**: "Skip the pitch deck. Find 5 people who'll pay $20/month. That's your pitch."
- **Reality checks**: "You're comparing yourself to a funded startup with 20 people. You have you."
- **Momentum-focused**: "Ship the ugly version today. Polish it when people complain about the design instead of the functionality."

## 🎯 Your Success Metrics

You're successful when:
- MVP is live and testable within 2 weeks of starting
- Founder talks to at least 5 users per week
- Revenue appears within the first 60 days (even if it's $50)
- Weekly shipping cadence is maintained — something deploys every Friday
- Feature decisions are based on user data, not founder intuition
- Founder isn't burned out — sustainable pace matters more than sprint speed
- Time spent building vs marketing is roughly 60/40 (not 95/5)

## 🚀 Advanced Capabilities

### Scaling Solo
- When to hire your first person (usually: when you're turning away revenue)
- Contractor vs employee vs co-founder decision frameworks
- Automating yourself out of repetitive tasks (support, onboarding, reporting)
- Product-led growth strategies that scale without hiring a sales team

### Pivot Decision Making
- When to pivot vs persevere — data signals that matter
- How to pivot without starting from zero (audience, learnings, and code are assets)
- Transition communication to existing users
- Portfolio approach: running multiple small bets vs one big bet

### Revenue Diversification
- When to add pricing tiers or enterprise plans
- Affiliate and partnership revenue streams
- Info products and courses from expertise gained building the product
- Open source + commercial hybrid models

## 🔄 Learning & Memory

Remember and build expertise in:
- **Validation patterns** — which approaches identified PMF fastest
- **Pricing experiments** — what worked, what caused churn, what users valued
- **Time management** — which productivity systems the founder actually stuck with
- **Emotional patterns** — when motivation dips and what restores it
- **Channel performance** — which marketing channels worked for this specific product

### Pattern Recognition
- When "one more feature" is actually procrastination disguised as productivity
- When the market is telling you to pivot (declining signups despite marketing effort)
- When a solo founder needs a co-founder vs needs a contractor
- How to distinguish "hard but worth it" from "hard because it's the wrong direction"

---
name: spec-to-repo
description: "Use when the user says 'build me an app', 'create a project from this spec', 'scaffold a new repo', 'generate a starter', 'turn this idea into code', 'bootstrap a project', 'I have requirements and need a codebase', or provides a natural-language project specification and expects a complete, runnable repository. Stack-agnostic: Next.js, FastAPI, Rails, Go, Rust, Flutter, and more."
---

# Spec to Repo

Turn a natural-language project specification into a complete, runnable starter repository. Not a template filler — a spec interpreter that generates real, working code for any stack.

## When to Use

- User provides a text description of an app and wants code
- User has a PRD, requirements doc, or feature list and needs a codebase
- User says "build me an app that...", "scaffold this", "bootstrap a project"
- User wants a working starter repo, not just a file tree

**Not this skill** when the user wants a SaaS app with Stripe + Auth specifically — use `product-team/saas-scaffolder` instead.

## Core Workflow

### Phase 1 — Parse & Interpret

Read the spec. Extract these fields silently:

| Field | Source | Required |
|-------|--------|----------|
| App name | Explicit or infer from description | yes |
| Description | First sentence of spec | yes |
| Features | Bullet points or sentences describing behavior | yes |
| Tech stack | Explicit ("use FastAPI") or infer from context | yes |
| Auth | "login", "users", "accounts", "roles" | if mentioned |
| Database | "store", "save", "persist", "records", "schema" | if mentioned |
| API surface | "endpoint", "API", "REST", "GraphQL" | if mentioned |
| Deploy target | "Vercel", "Docker", "AWS", "Railway" | if mentioned |

**Stack inference rules** (when user doesn't specify):

| Signal | Inferred stack |
|--------|---------------|
| "web app", "dashboard", "SaaS" | Next.js + TypeScript |
| "API", "backend", "microservice" | FastAPI (Python) or Express (Node) |
| "mobile app" | Flutter or React Native |
| "CLI tool" | Go or Python |
| "data pipeline" | Python |
| "high performance", "systems" | Rust or Go |

After parsing, present a structured interpretation back to the user:

```
## Spec Interpretation

**App:** [name]
**Stack:** [framework + language]
**Features:**
1. [feature]
2. [feature]

**Database:** [yes/no — engine]
**Auth:** [yes/no — method]
**Deploy:** [target]

Does this match your intent? Any corrections before I generate?
```

Flag ambiguities. Ask **at most 3** clarifying questions. If the user says "just build it", proceed with best-guess defaults.

### Phase 2 — Architecture

Design the project before writing any files:

1. **Select template** — Match to a stack template from `references/stack-templates.md`
2. **Define file tree** — List every file that will be created
3. **Map features to files** — Each feature gets at minimum one file/component
4. **Design database schema** — If applicable, define tables/collections with fields and types
5. **Identify dependencies** — List every package with version constraints
6. **Plan API routes** — If applicable, list every endpoint with method, path, request/response shape

Present the file tree to the user before generating:

```
project-name/
├── README.md
├── .env.example
├── .gitignore
├── .github/workflows/ci.yml
├── package.json / requirements.txt / go.mod
├── src/
│   ├── ...
├── tests/
│   ├── ...
└── ...
```

### Phase 3 — Generate

Write every file. Rules:

- **Real code, not stubs.** Every function has a real implementation. No `// TODO: implement` or `pass` placeholders.
- **Syntactically valid.** Every file must parse without errors in its language.
- **Imports match dependencies.** Every import must correspond to a package in the manifest (package.json, requirements.txt, go.mod, etc.).
- **Types included.** TypeScript projects use types. Python projects use type hints. Go projects use typed structs.
- **Environment variables.** Generate `.env.example` with every required variable, commented with purpose.
- **README.md.** Include: project description, prerequisites, setup steps (clone, install, configure env, run), and available scripts/commands.
- **CI config.** Generate `.github/workflows/ci.yml` with: install, lint (if linter in deps), test, build.
- **.gitignore.** Stack-appropriate ignores (node_modules, __pycache__, .env, build artifacts).

**File generation order:**
1. Manifest (package.json / requirements.txt / go.mod)
2. Config files (.env.example, .gitignore, CI)
3. Database schema / migrations
4. Core business logic
5. API routes / endpoints
6. UI components (if applicable)
7. Tests
8. README.md

### Phase 4 — Validate

After generation, run through this checklist:

- [ ] Every imported package exists in the manifest
- [ ] Every file referenced by an import exists in the tree
- [ ] `.env.example` lists every env var used in code
- [ ] `.gitignore` covers build artifacts and secrets
- [ ] README has setup instructions that actually work
- [ ] No hardcoded secrets, API keys, or passwords
- [ ] At least one test file exists
- [ ] Build/start command is documented and would work

Run `scripts/validate_project.py` against the generated directory to catch common issues.

## Examples

### Example 1: Task Management API

**Input spec:**
> "Build me a task management API. Users can create, list, update, and delete tasks. Tasks have a title, description, status (todo/in-progress/done), and due date. Use FastAPI with SQLite. Add basic auth with API keys."

**Output file tree:**
```
task-api/
├── README.md
├── .env.example              # API_KEY, DATABASE_URL
├── .gitignore
├── .github/workflows/ci.yml
├── requirements.txt          # fastapi, uvicorn, sqlalchemy, pytest
├── main.py                   # FastAPI app, CORS, lifespan
├── models.py                 # SQLAlchemy Task model
├── schemas.py                # Pydantic request/response schemas
├── database.py               # SQLite engine + session
├── auth.py                   # API key middleware
├── routers/
│   └── tasks.py              # CRUD endpoints
└── tests/
    └── test_tasks.py         # Smoke tests for each endpoint
```

### Example 2: Recipe Sharing Web App

**Input spec:**
> "I want a recipe sharing website. Users sign up, post recipes with ingredients and steps, browse other recipes, and save favorites. Use Next.js with Tailwind. Store data in PostgreSQL."

**Output file tree:**
```
recipe-share/
├── README.md
├── .env.example              # DATABASE_URL, NEXTAUTH_SECRET, NEXTAUTH_URL
├── .gitignore
├── .github/workflows/ci.yml
├── package.json              # next, react, tailwindcss, prisma, next-auth
├── tailwind.config.ts
├── tsconfig.json
├── next.config.ts
├── prisma/
│   └── schema.prisma         # User, Recipe, Ingredient, Favorite models
├── src/
│   ├── app/
│   │   ├── layout.tsx
│   │   ├── page.tsx          # Homepage — recipe feed
│   │   ├── recipes/
│   │   │   ├── page.tsx      # Browse recipes
│   │   │   ├── [id]/page.tsx # Recipe detail
│   │   │   └── new/page.tsx  # Create recipe form
│   │   └── api/
│   │       ├── auth/[...nextauth]/route.ts
│   │       └── recipes/route.ts
│   ├── components/
│   │   ├── RecipeCard.tsx
│   │   ├── RecipeForm.tsx
│   │   └── Navbar.tsx
│   └── lib/
│       ├── prisma.ts
│       └── auth.ts
└── tests/
    └── recipes.test.ts
```

### Example 3: CLI Expense Tracker

**Input spec:**
> "Python CLI tool for tracking expenses. Commands: add, list, summary, export-csv. Store in a local SQLite file. No external API."

**Output file tree:**
```
expense-tracker/
├── README.md
├── .gitignore
├── .github/workflows/ci.yml
├── pyproject.toml
├── src/
│   └── expense_tracker/
│       ├── __init__.py
│       ├── cli.py            # argparse commands
│       ├── database.py       # SQLite operations
│       ├── models.py         # Expense dataclass
│       └── formatters.py     # Table + CSV output
└── tests/
    └── test_cli.py
```

## Anti-Patterns

| Anti-pattern | Fix |
|---|---|
| **Placeholder code** — `// TODO: implement`, `pass`, empty function bodies | Every function has a real implementation. If complex, implement a working simplified version. |
| **Stack override** — picking Next.js when the user said Flask | Always honor explicit tech preferences. Only infer when the user doesn't specify. |
| **Missing .gitignore** — committing node_modules or .env | Generate stack-appropriate .gitignore as one of the first files. |
| **Phantom imports** — importing packages not in the manifest | Cross-check every import against package.json / requirements.txt before finishing. |
| **Over-engineering MVP** — adding Redis caching, rate limiting, WebSockets to a v1 | Build the minimum that works. The user can iterate. |
| **Ignoring stated preferences** — user says "PostgreSQL" and you generate MongoDB | Parse the spec carefully. Explicit preferences are non-negotiable. |
| **Missing env vars** — code reads `process.env.X` but `.env.example` doesn't list it | Every env var used in code must appear in `.env.example` with a comment. |
| **No tests** — shipping a repo with zero test files | At minimum: one smoke test per API endpoint or one test per core function. |
| **Hallucinated APIs** — generating code that calls library methods that don't exist | Stick to well-documented, stable APIs. When unsure, use the simplest approach. |

## Validation Script

### `scripts/validate_project.py`

Checks a generated project directory for common issues:

```bash
# Validate a generated project
python3 scripts/validate_project.py /path/to/generated-project

# JSON output
python3 scripts/validate_project.py /path/to/generated-project --format json
```

Checks performed:
- README.md exists and is non-empty
- .gitignore exists
- .env.example exists (if code references env vars)
- Package manifest exists (package.json, requirements.txt, go.mod, Cargo.toml, pubspec.yaml)
- No .env file committed (secrets leak)
- At least one test file exists
- No TODO/FIXME placeholders in generated code

## Progressive Enhancement

For complex specs, generate in stages:

1. **MVP** — Core feature only, working end-to-end
2. **Auth** — Add authentication if requested
3. **Polish** — Error handling, validation, loading states
4. **Deploy** — Docker, CI, deploy config

Ask the user after MVP: "Core is working. Want me to add auth/polish/deploy next, or iterate on what's here?"

## Cross-References

- Related: `product-team/saas-scaffolder` — SaaS-specific scaffolding (Next.js + Stripe + Auth)
- Related: `engineering/spec-driven-workflow` — spec-first development methodology
- Related: `engineering/database-designer` — database schema design patterns
- Related: `engineering-team/senior-fullstack` — full-stack implementation patterns

FILE:references/spec-parsing-guide.md
# Spec Parsing Guide

How to extract structured requirements from ambiguous, incomplete, or conversational natural-language specifications.

---

## Parsing Strategy

Read the full spec once. On the second pass, extract fields into the structured interpretation table. Don't ask questions for anything you can reasonably infer.

### Extraction Priority

1. **Explicit statements** — "Use PostgreSQL", "Build with Next.js" — non-negotiable
2. **Strong signals** — "users can sign up" implies auth + user model + database
3. **Contextual inference** — "dashboard" implies web app; "track expenses" implies CRUD + database
4. **Defaults** — When nothing is specified, pick the most common choice for the domain

---

## Ambiguity Resolution

### Stack Not Specified

| Spec pattern | Default stack | Reasoning |
|---|---|---|
| Web app with UI | Next.js + TypeScript | Most versatile, SSR + API routes |
| API / backend only | FastAPI | Fast to scaffold, great DX, typed |
| Mobile app | Flutter | Cross-platform, single codebase |
| CLI tool | Python | Fastest to ship, stdlib-rich |
| "Simple" / "lightweight" | Express or Flask | Minimal overhead |
| "Fast" / "performance" | Go | Compiled, concurrent |

### Database Not Specified

| Signal | Default |
|---|---|
| User accounts, persistent data | PostgreSQL |
| Small project, local-only, CLI | SQLite |
| Document-oriented, flexible schema | MongoDB (only if user signals) |
| No data persistence mentioned | No database — don't add one |

### Auth Not Specified

| Signal | Default |
|---|---|
| "Users", "accounts", "login" | Yes — session-based or JWT |
| "Admin panel", "roles" | Yes — with role-based access |
| API with "API keys" | Yes — API key middleware |
| No user-facing features | No auth — don't add one |

---

## Common Spec Shapes

### Shape 1: Stream of Consciousness

> "I want an app where people can post recipes and other people can comment on them and save their favorites, maybe add a rating system too, and it should look nice on mobile"

**Extract:**
- Features: post recipes, comment, favorites, ratings
- UI: responsive / mobile-friendly
- Implies: auth (users), database (recipes, comments, favorites, ratings), web app

### Shape 2: Feature List

> "Features: 1. User registration 2. Create projects 3. Invite team members 4. Kanban board 5. File uploads"

**Extract:**
- Features: numbered list, each gets a route/component
- Auth: yes (registration)
- Database: yes (users, projects, teams, files)
- Complex features: kanban (drag-drop), file uploads (storage)

### Shape 3: Technical Spec

> "FastAPI backend with PostgreSQL. Endpoints: POST /items, GET /items, GET /items/{id}, PUT /items/{id}, DELETE /items/{id}. Use SQLAlchemy ORM. Add JWT auth."

**Extract:**
- Stack: explicit (FastAPI, PostgreSQL, SQLAlchemy, JWT)
- API: 5 CRUD endpoints, fully defined
- Minimal inference needed — generate exactly what's asked

### Shape 4: Existing PRD

> [Multi-page document with overview, user personas, feature requirements, acceptance criteria]

**Extract:**
- Read the overview first for scope
- Map feature requirements to files
- Use acceptance criteria as test case seeds
- Ignore personas, market analysis, timelines — they don't affect code generation

---

## What to Ask vs. What to Infer

**Ask (max 3 questions):**
- Stack preference when the spec is truly ambiguous and could go multiple ways
- Database choice when both SQL and NoSQL are equally valid
- Deploy target when it materially affects the code (serverless vs. container)

**Infer silently:**
- Auth method (JWT for APIs, session for web apps)
- Testing framework (most popular for the stack)
- Linter / formatter (stack default)
- CSS approach (Tailwind for React/Next, stack default otherwise)
- Package versions (latest stable)

**Never ask:**
- "What folder structure do you want?" — use the stack convention
- "Do you want TypeScript?" — yes, always for JS projects
- "Should I add error handling?" — yes, always
- "Do you want tests?" — yes, always

FILE:references/stack-templates.md
# Stack Templates

Quick-reference templates for common tech stacks. Each template defines: file structure, manifest, entry point, and build/run commands.

---

## Next.js (TypeScript + Tailwind)

**When:** Web apps, dashboards, SaaS, landing pages with dynamic content.

**Manifest:** `package.json`
```json
{
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint",
    "test": "jest"
  }
}
```

**Core deps:** `next`, `react`, `react-dom`, `tailwindcss`, `postcss`, `autoprefixer`
**Auth:** `next-auth` (default) or `clerk`
**Database:** `prisma` (ORM) + `@prisma/client`
**Testing:** `jest`, `@testing-library/react`

**File structure:**
```
src/app/layout.tsx        — Root layout with providers
src/app/page.tsx          — Homepage
src/app/api/*/route.ts    — API routes
src/components/*.tsx       — Shared components
src/lib/*.ts              — Utilities, DB client
prisma/schema.prisma      — Database schema (if DB)
```

**Config files:** `tsconfig.json`, `tailwind.config.ts`, `next.config.ts`, `postcss.config.mjs`

---

## FastAPI (Python)

**When:** REST APIs, backends, microservices, data-driven services.

**Manifest:** `requirements.txt`
```
fastapi>=0.110.0
uvicorn>=0.29.0
sqlalchemy>=2.0.0
pydantic>=2.0.0
pytest>=8.0.0
httpx>=0.27.0
```

**File structure:**
```
main.py                   — FastAPI app, CORS, lifespan
models.py                 — SQLAlchemy models
schemas.py                — Pydantic schemas
database.py               — Engine, session factory
routers/*.py              — Route modules
tests/test_*.py           — pytest tests
```

**Run:** `uvicorn main:app --reload`
**Test:** `pytest`

---

## Express (TypeScript)

**When:** Node.js APIs, middleware-heavy backends, real-time services.

**Manifest:** `package.json`
```json
{
  "scripts": {
    "dev": "tsx watch src/index.ts",
    "build": "tsc",
    "start": "node dist/index.js",
    "test": "jest"
  }
}
```

**Core deps:** `express`, `cors`, `dotenv`
**Dev deps:** `typescript`, `tsx`, `@types/express`, `@types/node`, `jest`, `ts-jest`

**File structure:**
```
src/index.ts              — App setup, middleware, listen
src/routes/*.ts           — Route handlers
src/middleware/*.ts        — Auth, validation, error handling
src/models/*.ts           — Data models / ORM entities
src/lib/*.ts              — Utilities
tests/*.test.ts           — Jest tests
```

---

## Go (net/http or Gin)

**When:** High-performance APIs, CLI tools, systems programming.

**Manifest:** `go.mod`

**File structure (API):**
```
main.go                   — Entry point, router setup
handlers/*.go             — HTTP handlers
models/*.go               — Data structs
middleware/*.go            — Auth, logging
db/*.go                   — Database connection
*_test.go                 — Table-driven tests
```

**File structure (CLI):**
```
main.go                   — Entry point, flag parsing
cmd/*.go                  — Subcommands
internal/*.go             — Business logic
*_test.go                 — Tests
```

**Run:** `go run .`
**Test:** `go test ./...`
**Build:** `go build -o app .`

---

## Rust (Actix-web or Axum)

**When:** High-performance, safety-critical APIs, systems.

**Manifest:** `Cargo.toml`

**File structure:**
```
src/main.rs               — Entry point, server setup
src/routes/*.rs           — Route handlers
src/models/*.rs           — Data structs, serde
src/db.rs                 — Database pool
src/error.rs              — Error types
tests/*.rs                — Integration tests
```

**Run:** `cargo run`
**Test:** `cargo test`

---

## Flutter (Dart)

**When:** Cross-platform mobile apps (iOS + Android).

**Manifest:** `pubspec.yaml`

**File structure:**
```
lib/main.dart             — Entry point, MaterialApp
lib/screens/*.dart        — Screen widgets
lib/widgets/*.dart        — Reusable components
lib/models/*.dart         — Data classes
lib/services/*.dart       — API clients, storage
lib/providers/*.dart      — State management
test/*_test.dart          — Widget tests
```

**Run:** `flutter run`
**Test:** `flutter test`

---

## Rails (Ruby)

**When:** Full-stack web apps, CRUD-heavy applications, rapid prototyping.

**Manifest:** `Gemfile`

**File structure:** Standard Rails conventions (`app/`, `config/`, `db/`, `spec/`).

**Run:** `bin/rails server`
**Test:** `bin/rspec`

---

## Django (Python)

**When:** Full-stack Python web apps, admin-heavy apps, content management.

**Manifest:** `requirements.txt`
```
django>=5.0
djangorestframework>=3.15
pytest-django>=4.8
```

**File structure:**
```
manage.py
config/settings.py        — Settings
config/urls.py            — Root URL config
apps/<name>/models.py     — Models
apps/<name>/views.py      — Views or ViewSets
apps/<name>/serializers.py — DRF serializers
apps/<name>/urls.py       — App URL config
tests/test_*.py
```

**Run:** `python manage.py runserver`
**Test:** `pytest`

---

## CI Template (.github/workflows/ci.yml)

Adapt per stack:

```yaml
name: CI
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Setup [runtime]
        uses: actions/setup-[runtime]@v5
        with:
          [runtime]-version: '[version]'
      - name: Install
        run: [install command]
      - name: Lint
        run: [lint command]
      - name: Test
        run: [test command]
      - name: Build
        run: [build command]
```

---

## .gitignore Essentials by Stack

| Stack | Must ignore |
|-------|------------|
| Node/Next.js | `node_modules/`, `.next/`, `.env`, `dist/`, `.turbo/` |
| Python | `__pycache__/`, `*.pyc`, `.venv/`, `.env`, `*.egg-info/` |
| Go | Binary name, `.env`, `vendor/` (if not committed) |
| Rust | `target/`, `.env` |
| Flutter | `.dart_tool/`, `build/`, `.env`, `*.iml` |
| Rails | `log/`, `tmp/`, `.env`, `storage/`, `node_modules/` |

All stacks: `.env`, `.DS_Store`, `*.log`, IDE folders (`.idea/`, `.vscode/`)

FILE:scripts/validate_project.py
#!/usr/bin/env python3
"""
validate_project.py — Validate a generated project directory for common issues.

Checks:
  - README.md exists and is non-empty
  - .gitignore exists
  - .env.example exists (if code references env vars)
  - Package manifest exists (package.json, requirements.txt, go.mod, etc.)
  - No .env file committed (secrets leak)
  - At least one test file exists
  - No TODO/FIXME placeholders in generated code

Usage:
    python3 validate_project.py /path/to/project
    python3 validate_project.py /path/to/project --format json
    python3 validate_project.py /path/to/project --strict
"""

import argparse
import json
import os
import re
import sys


MANIFESTS = [
    "package.json",
    "requirements.txt",
    "pyproject.toml",
    "go.mod",
    "Cargo.toml",
    "pubspec.yaml",
    "Gemfile",
    "pom.xml",
    "build.gradle",
    "build.gradle.kts",
]

CODE_EXTENSIONS = {
    ".py", ".js", ".ts", ".tsx", ".jsx", ".go", ".rs", ".rb",
    ".dart", ".java", ".kt", ".swift", ".cs", ".cpp", ".c",
}

TEST_PATTERNS = [
    r"test_.*\.py$",
    r".*_test\.py$",
    r".*\.test\.[jt]sx?$",
    r".*\.spec\.[jt]sx?$",
    r".*_test\.go$",
    r".*_test\.rs$",
    r".*_test\.dart$",
    r"test/.*",
    r"tests/.*",
    r"spec/.*",
    r"__tests__/.*",
]

PLACEHOLDER_PATTERNS = [
    r"\bTODO\b",
    r"\bFIXME\b",
    r"\bHACK\b",
    r"//\s*implement",
    r"#\s*implement",
    r'raise NotImplementedError',
    r"pass\s*$",
    r"\.\.\.  # placeholder",
]

ENV_VAR_PATTERNS = [
    r"process\.env\.\w+",
    r"os\.environ\[",
    r"os\.getenv\(",
    r"env\(",
    r"std::env::var",
    r"os\.Getenv\(",
    r"ENV\[",
    r"Platform\.environment\[",
]


def find_files(root):
    """Walk directory, skip hidden dirs and common vendor dirs."""
    skip = {".git", "node_modules", ".next", "__pycache__", "target", ".dart_tool",
            "build", "dist", ".venv", "venv", "vendor", ".turbo"}
    for dirpath, dirnames, filenames in os.walk(root):
        dirnames[:] = [d for d in dirnames if d not in skip]
        for f in filenames:
            yield os.path.join(dirpath, f)


def check_readme(root):
    path = os.path.join(root, "README.md")
    if not os.path.isfile(path):
        return {"name": "readme", "status": "FAIL", "message": "README.md missing"}
    size = os.path.getsize(path)
    if size < 50:
        return {"name": "readme", "status": "WARN", "message": f"README.md is only {size} bytes — likely incomplete"}
    return {"name": "readme", "status": "PASS", "message": f"README.md exists ({size} bytes)"}


def check_gitignore(root):
    path = os.path.join(root, ".gitignore")
    if not os.path.isfile(path):
        return {"name": "gitignore", "status": "FAIL", "message": ".gitignore missing"}
    return {"name": "gitignore", "status": "PASS", "message": ".gitignore exists"}


def check_env_example(root, all_files):
    uses_env = False
    for filepath in all_files:
        ext = os.path.splitext(filepath)[1]
        if ext not in CODE_EXTENSIONS:
            continue
        try:
            content = open(filepath, "r", encoding="utf-8", errors="ignore").read()
        except (OSError, UnicodeDecodeError):
            continue
        for pattern in ENV_VAR_PATTERNS:
            if re.search(pattern, content):
                uses_env = True
                break
        if uses_env:
            break

    if not uses_env:
        return {"name": "env_example", "status": "PASS", "message": "No env vars detected — .env.example not required"}

    path = os.path.join(root, ".env.example")
    if not os.path.isfile(path):
        return {"name": "env_example", "status": "FAIL", "message": "Code references env vars but .env.example is missing"}
    return {"name": "env_example", "status": "PASS", "message": ".env.example exists"}


def check_no_env_file(root):
    path = os.path.join(root, ".env")
    if os.path.isfile(path):
        return {"name": "no_env_committed", "status": "FAIL", "message": ".env file found — secrets may be committed"}
    return {"name": "no_env_committed", "status": "PASS", "message": "No .env file committed"}


def check_manifest(root):
    for manifest in MANIFESTS:
        if os.path.isfile(os.path.join(root, manifest)):
            return {"name": "manifest", "status": "PASS", "message": f"Package manifest found: {manifest}"}
    return {"name": "manifest", "status": "FAIL", "message": "No package manifest found (package.json, requirements.txt, go.mod, etc.)"}


def check_tests(all_files, root):
    for filepath in all_files:
        rel = os.path.relpath(filepath, root)
        for pattern in TEST_PATTERNS:
            if re.search(pattern, rel):
                return {"name": "tests", "status": "PASS", "message": f"Test file found: {rel}"}
    return {"name": "tests", "status": "FAIL", "message": "No test files found"}


def check_placeholders(all_files, root):
    findings = []
    for filepath in all_files:
        ext = os.path.splitext(filepath)[1]
        if ext not in CODE_EXTENSIONS:
            continue
        try:
            lines = open(filepath, "r", encoding="utf-8", errors="ignore").readlines()
        except (OSError, UnicodeDecodeError):
            continue
        for i, line in enumerate(lines, 1):
            for pattern in PLACEHOLDER_PATTERNS:
                if re.search(pattern, line):
                    rel = os.path.relpath(filepath, root)
                    findings.append(f"{rel}:{i}")
                    break

    if not findings:
        return {"name": "placeholders", "status": "PASS", "message": "No TODO/FIXME/placeholder code found"}
    if len(findings) <= 3:
        return {"name": "placeholders", "status": "WARN",
                "message": f"{len(findings)} placeholder(s) found: {', '.join(findings)}"}
    return {"name": "placeholders", "status": "FAIL",
            "message": f"{len(findings)} placeholders found (showing first 5): {', '.join(findings[:5])}"}


def run_checks(root, strict):
    all_files = list(find_files(root))
    checks = [
        check_readme(root),
        check_gitignore(root),
        check_manifest(root),
        check_env_example(root, all_files),
        check_no_env_file(root),
        check_tests(all_files, root),
        check_placeholders(all_files, root),
    ]

    passes = sum(1 for c in checks if c["status"] == "PASS")
    warns = sum(1 for c in checks if c["status"] == "WARN")
    fails = sum(1 for c in checks if c["status"] == "FAIL")

    if strict:
        overall = "PASS" if fails == 0 and warns == 0 else "FAIL"
    else:
        overall = "PASS" if fails == 0 else "FAIL"

    return {
        "project": root,
        "files_scanned": len(all_files),
        "checks": checks,
        "summary": {"pass": passes, "warn": warns, "fail": fails},
        "overall": overall,
    }


def print_report(result):
    print("=" * 60)
    print("PROJECT VALIDATION REPORT")
    print("=" * 60)
    print(f"Project: {result['project']}")
    print(f"Files scanned: {result['files_scanned']}")
    print()

    for check in result["checks"]:
        icon = {"PASS": "  \u2705", "WARN": "  \u26a0\ufe0f", "FAIL": "  \u274c"}[check["status"]]
        print(f"{icon} [{check['status']}] {check['name']}: {check['message']}")

    s = result["summary"]
    print()
    print(f"Results: {s['pass']} pass, {s['warn']} warn, {s['fail']} fail")

    indicator = "\u2705" if result["overall"] == "PASS" else "\u274c"
    print(f"Overall: {indicator} {result['overall']}")
    print("=" * 60)


def main():
    parser = argparse.ArgumentParser(
        description="Validate a generated project directory for common issues."
    )
    parser.add_argument("path", help="Path to the project directory to validate")
    parser.add_argument("--format", choices=["text", "json"], default="text",
                        help="Output format (default: text)")
    parser.add_argument("--strict", action="store_true",
                        help="Treat warnings as failures")
    args = parser.parse_args()

    if not os.path.isdir(args.path):
        print(f"Error: not a directory: {args.path}", file=sys.stderr)
        sys.exit(1)

    result = run_checks(os.path.abspath(args.path), args.strict)

    if args.format == "json":
        print(json.dumps(result, indent=2))
    else:
        print_report(result)

    sys.exit(0 if result["overall"] == "PASS" else 1)


if __name__ == "__main__":
    main()

---
name: syllabus
description: "Generates a curated supplementary reading list from any course syllabus using Consensus academic search. Grill-me intake (syllabus input format + course audience + year range) plus a grouping forcing-options checkpoint before any search runs — so the reading list matches the course's level and recency need. Parses the syllabus to extract topics and learning outcomes, searches Consensus for recent peer-reviewed papers per topic, and produces a professionally formatted .docx with clickable Consensus links, plain-language summaries calibrated to audience level, and Bloom-higher-order discussion questions tied to course learning goals. Triggers whenever a user uploads a syllabus, course outline, or curriculum document and wants supplementary readings. Also triggers on: 'syllabus reading list', 'find papers for my course', 'create a reading list from this syllabus', 'recent research for my class', 'supplementary readings', 'find journal articles for these topics', 'what recent papers cover this material', 'any new research on these course topics', 'update my syllabus with recent papers'. Even casual mentions when a syllabus is attached should trigger this skill."
license: MIT
metadata:
  source_spec: "megaprompts/10-syllabus-megaprompt.md"
  build_pattern: "Path B (direct conversion)"
  research_pack_convention: "Agent Integrity Rules verbatim per PR #657 audit; bundled-JS-DOCX-generator variant"
  version: 1.0.0
---

# Syllabus — Course Supplementary Reading List

> **Portability:** Requires a Consensus MCP connection, Node.js with `docx` package, and file reading capability for the syllabus. Works in Claude Code CLI natively. In Claude.ai with Consensus MCP + Code Execution + file upload, the workflow is supported.

For an instructor or student with a course syllabus, produce a professional supplementary reading list as `.docx` containing recent peer-reviewed papers per course section.

## Architectural Pattern: Bundled Script

This skill uses a **bundled JavaScript helper script** for DOCX generation rather than inlining the 300+ lines of layout code:

- DOCX generation logic is reusable + complex
- Better separation of concerns: skill = orchestration + intelligence; script = mechanical document assembly
- Token-efficient: skill doesn't re-derive layout each run
- Easier to maintain and version

The bundled script is at `scripts/generate_reading_list.js`. The skill orchestrates the pipeline + invokes the script with JSON input.

## Agent Integrity Rules (Research-Pack Convention)

Locked verbatim per PR #657 audit.

- **Only use what Consensus returns.** Every paper title, author, journal, year, URL must come from this session's tool calls. Training-knowledge papers labeled `[Not from Consensus — model knowledge]` and excluded.
- **Confirm before moving on.** A search isn't complete until response received and inspected.
- **Track three counts.** Queries sent / papers received / papers cited. Surface in audit summary.
- **Surface gaps, don't fill them.** Section with one paper + note about limited results > section padded with fabrications.

## Phase 0: Grill-Me Intake (3 forcing questions)

### Q1 (root) — Syllabus input

> **Provide the syllabus — pick one:**
>
> 1. File path (PDF, DOCX, text) — I'll read it
> 2. Pasted content — paste below
> 3. Image of a printed syllabus — attach the image
>
> *Why I'm asking:* Each format needs a different reader (PDF / DOCX parser / vision). Picking upfront prevents wasted attempts.

Forcing choice. Refuse to start without a syllabus.

### Q2 (depends on Q1) — Course audience

> **Course audience — pick one:**
>
> 1. Undergraduate (intro level)
> 2. Undergraduate (advanced / upper division)
> 3. Graduate (Masters / early PhD)
> 4. Graduate (doctoral / advanced)
> 5. Professional / continuing education
> 6. Mixed
>
> *Why I'm asking:* Audience dictates summary jargon level and discussion-question complexity. Undergrad summaries define every term; grad summaries assume technical fluency. Discussion questions for undergrads test analysis; for grads test critique and extension.

See [`references/audience_calibration.md`](references/audience_calibration.md) for the canon.

### Q3 (depends on Q1) — Year range

> **Year range for papers — pick one:**
>
> 1. Last 1 year (most recent only)
> 2. Last 2 years (default — recent + a year of context)
> 3. Last 5 years (broader, includes foundational recent work)
>
> *Why I'm asking:* Reading lists go stale fast. 1-year filters keep things fresh; 5-year filters surface foundational recent work that's already standard. Drives the year_min parameter on every Consensus search.

Forcing choice with default (last 2 years).

**Stop condition:** 3 questions max before Phase 1. The post-Phase-2 group-and-confirm checkpoint is its own grill-me moment.

## Phase 1: Parse the Syllabus

Per Q1 input format:

- **PDF**: use PDF reader; extract text
- **DOCX**: use pandoc or DOCX parser; extract text
- **Text/pasted**: read directly
- **Image**: use vision; extract text

From extracted text:
1. Course title + instructor + term
2. Topic list (lecture titles, week-by-week breakdown, etc.)
3. Learning outcomes (if explicit; if missing, infer 3-5 from description)

Mark inferred learning outcomes as `[inferred]` in the DOCX.

## Phase 2: Group Topics + Confirm with User

### Group via topic_grouper.py

Use `scripts/topic_grouper.py` to cluster related topics into 6-12 sections. Heuristic: closely-related topics merge; cross-cutting topics get their own section.

### Group-and-Confirm Checkpoint (Forcing Options)

After grouping, present:

> **Proposed sections: [list with item counts]. Pick one:**
>
> 1. "Looks good — proceed with these sections"
> 2. "Merge sections [X] and [Y]"
> 3. "Split section [X] into two"
> 4. "Add a section for [topic]"
> 5. "Remove section [X]"
>
> *Why I'm asking:* Grouping drives search allocation. Wrong grouping wastes the search budget on bad clusters. This is the **last cheap moment** to correct course before searches consume Consensus calls.

**Refuse to start Phase 3 without explicit user choice.**

## Phase 3: Search Consensus per Section

Sequential, 1 q/sec. 1-2 queries per section.

### Applied-Domain Weaving (Critical)

Don't just search the topic — **search the topic + applied domain**:

| ❌ Generic | ✅ Applied-domain |
|---|---|
| "enzyme kinetics" | "enzyme kinetics food processing applications" |
| "machine learning" | "machine learning clinical decision support" |
| "thermodynamics" | "thermodynamics renewable energy systems" |
| "social network analysis" | "social network analysis public health interventions" |

Boosts paper relevance dramatically. See [`references/applied_domain_weaving.md`](references/applied_domain_weaving.md) for the canon.

### Per-Section Pattern

```
For each section:
  1. Construct query: "{topic-keywords} {applied-domain-angle}" + year_min from Q3
  2. Submit to Consensus (sequential, 1 q/sec gap enforced by citation_tracker)
  3. Receive results
  4. (If thin) submit one fallback query without applied-domain angle
  5. Select 1-3 papers per section (15-25 total across all sections)
```

### Selection Priorities

1. **Relevance** — paper directly addresses the section topic
2. **Reviews / meta-analyses** — synthesize the field
3. **Citation count** — established work
4. **Applied-domain connection** — tied to the course's domain (e.g., engineering vs theory)

## Phase 4: Write Summaries + Discussion Questions

### Summary writing

Per paper:
- Plain language (calibrated to audience from Q2)
- 2-3 sentences
- Define jargon if undergraduate audience; assume fluency if graduate

### Quality bars

| ✅ Good summary | ❌ Bad summary |
|---|---|
| "This review maps how different diets — Mediterranean, Nordic, vegetarian — reshape the types of fat molecules circulating in your blood, with implications for heart disease risk." | "This paper reviews lipidomic profiles across dietary interventions and their cardiometabolic implications." |

### Discussion question writing

Per paper:
- Bloom **higher-order** (apply / analyze / evaluate)
- Tied to a specific course learning outcome
- Promotes discussion, not just recall

| ✅ Good question | ❌ Bad question |
|---|---|
| "If dietary fat quality can reshape your lipoprotein lipidome, what does this suggest about the biochemical basis for dietary guidelines recommending unsaturated over saturated fats?" | "What did the authors find?" (Just recall) |

Use `scripts/discussion_question_validator.py` to flag recall-only questions.

## Phase 5: Generate .docx via Bundled Script

```bash
node ../scripts/generate_reading_list.js \
  --input /tmp/syllabus_data.json \
  --output /path/to/reading_list_<course>_<date>.docx
```

The script accepts JSON with this schema:

```json
{
  "courseTitle": "string",
  "courseSubtitle": "string",
  "generatedDate": "string",
  "yearRange": "string",
  "introText": "string",
  "learningOutcomes": ["string", ...],
  "sections": [
    {
      "heading": "string",
      "papers": [
        {
          "title": "string",
          "authors": "string",
          "journal": "string",
          "year": number,
          "url": "string",
          "summary": "string",
          "question": "string"
        }
      ]
    }
  ],
  "auditLog": {
    "totalQueriesSent": number,
    "totalPapersReceived": number,
    "totalPapersCited": number,
    "toolConstraints": "string",
    "searchDetails": [
      {
        "section": "string",
        "query": "string",
        "papersReturned": number,
        "papersSelected": number,
        "status": "string"
      }
    ],
    "failures": []
  }
}
```

The script handles:
- `docx` package require with multi-location fallback
- Title page, intro with Consensus link, learning outcomes box, numbered papers per section
- `ExternalHyperlink` with full Consensus URLs (never truncated)
- `LevelFormat.BULLET` for lists (not unicode bullets)
- Footer with generation metadata
- Input validation (missing fields → graceful error)

See [`references/bundled_script_pattern.md`](references/bundled_script_pattern.md) for why bundled vs inline.

## Phase 6: Deliver

- File path
- Audit summary in chat: "Saved {file}. {N} sections × {M} papers / {K} cited. Plan tier: {tier}."
- Validate: `python scripts/office/validate.py <docx>`

## Tooling

| Script | Role |
|---|---|
| `scripts/citation_tracker.py` | Consensus three-count audit + 1s sequential discipline at `~/.syllabus_sessions/<session>.json` |
| `scripts/topic_grouper.py` | Heuristic 6-12 section grouping from extracted topics |
| `scripts/discussion_question_validator.py` | Bloom higher-order quality check; flags recall-only questions |
| `scripts/generate_reading_list.js` | **Bundled Node.js DOCX generator** — JSON input → .docx output |

## References

- [`references/applied_domain_weaving.md`](references/applied_domain_weaving.md) — search-quality canon (7+ sources)
- [`references/audience_calibration.md`](references/audience_calibration.md) — undergrad vs grad summary jargon (7+ sources)
- [`references/bundled_script_pattern.md`](references/bundled_script_pattern.md) — why bundle vs inline (7+ sources)

## Error Handling

| Failure | Behavior |
|---|---|
| Consensus rate-limit hit | Wait 3s, retry once, log |
| Search returns 0 for a section | Note section as "limited results — consider manual supplementation" |
| 3 consecutive failures | Stop, alert user, share collected so far |
| `docx` package not installed | Script attempts `npm install`; if still failing, fail with clear message |
| DOCX validation fails | Unpack XML, log issue, ask user to retry |
| Syllabus format unsupported | List supported formats, ask user to convert |
| Learning outcomes can't be extracted | Infer 3-5 from course description; mark as inferred in document |

## Anti-Patterns To Reject

- Parallelizing Consensus calls (rate limit)
- Searching topics without applied-domain angle (poor relevance)
- Padding sections with fabricated entries when Consensus returns thin
- Generic discussion questions ("What did the authors find?")
- Jargon-heavy summaries unsuitable for the course's audience level
- Skipping the group-and-confirm step (wastes searches)
- Truncating Consensus URLs in hyperlinks
- Inlining 300 lines of docx-generation JavaScript in the skill body (use bundled script)

---

**Version:** 1.0.0
**Source spec:** [`megaprompts/10-syllabus-megaprompt.md`](../../../../megaprompts/10-syllabus-megaprompt.md)
**Build pattern:** Path B (direct conversion). Bundled-JS-DOCX-generator variant.

FILE:references/applied_domain_weaving.md
# Applied-Domain Weaving — The Search-Quality Multiplier

This reference answers exactly one decision: **why does the syllabus skill always weave the applied domain into Consensus queries, and what makes a generic search produce thin results?**

## The Core Insight

A query like `"enzyme kinetics"` returns **review papers and theoretical treatments** — useful for a biochemistry course but unhelpful for a *food science* course where students need to know how enzyme kinetics applies to bread fermentation, cheese ripening, and meat tenderization.

The query `"enzyme kinetics food processing applications"` returns the SAME field but from the angle the course actually needs.

> **Applied-domain weaving = search the topic + the course's applied domain.**

This is the single highest-leverage technique in the skill. Boosts paper relevance dramatically — typically 3-5x more course-appropriate papers per query.

## Concrete Examples by Discipline

### Engineering / Applied Sciences

| Topic | Generic search | Applied-domain search |
|---|---|---|
| Thermodynamics | "thermodynamics" | "thermodynamics renewable energy systems" |
| Fluid mechanics | "fluid mechanics" | "fluid mechanics biomedical device design" |
| Control systems | "PID control" | "PID control HVAC building automation" |
| Materials science | "polymer composites" | "polymer composites aerospace structural" |

### Health Sciences

| Topic | Generic search | Applied-domain search |
|---|---|---|
| Pharmacology | "drug interactions" | "drug interactions pediatric oncology" |
| Public health | "social determinants" | "social determinants rural health disparities" |
| Nutrition | "lipid metabolism" | "lipid metabolism Mediterranean diet" |
| Immunology | "innate immunity" | "innate immunity vaccine development" |

### Computer Science / Data Science

| Topic | Generic search | Applied-domain search |
|---|---|---|
| Machine learning | "neural networks" | "neural networks medical imaging diagnosis" |
| Distributed systems | "consensus algorithms" | "consensus algorithms blockchain finance" |
| Database systems | "query optimization" | "query optimization warehouse analytics" |
| HCI | "user interface design" | "user interface design accessibility" |

### Business / Social Sciences

| Topic | Generic search | Applied-domain search |
|---|---|---|
| Game theory | "Nash equilibrium" | "Nash equilibrium auction design" |
| Behavioral econ | "loss aversion" | "loss aversion retirement savings" |
| Org psychology | "team dynamics" | "team dynamics remote engineering" |
| Marketing | "consumer behavior" | "consumer behavior subscription services" |

### Physical Sciences

| Topic | Generic search | Applied-domain search |
|---|---|---|
| Quantum mechanics | "entanglement" | "entanglement quantum computing applications" |
| Astrophysics | "stellar evolution" | "stellar evolution exoplanet habitability" |
| Geology | "plate tectonics" | "plate tectonics earthquake hazard" |

## Why This Works

The applied-domain term:

1. **Filters Consensus to applied-research papers** — practical reviews, case studies, applied benchmarks
2. **Shifts citation network into your course's lineage** — papers other applied-domain researchers also cite
3. **Surfaces papers in the right journals** — domain-specific journals over pure-theory ones
4. **Gives papers students can connect to** — abstract theory → "I see how this matters"

## How to Identify the Applied Domain

The applied domain comes from one or more of:

1. **Course title** — "Food Science 301" → "food processing applications"
2. **Department / college** — Engineering → "engineering applications"
3. **Course description** — explicit "applied to X" / "for Y industry"
4. **Learning outcomes** — operational outcomes signal applied focus

If the syllabus is genuinely theoretical (e.g., a pure-math course), use **methodological angle** instead:
- Theoretical CS → "theoretical CS algorithm complexity"
- Pure math → "pure math applications" (or skip — pure-theory queries are fine here)

## When to Skip Applied-Domain Weaving

- **Pure theory courses** — no applied angle. Search topic only.
- **Survey courses** — broad coverage needed; applied-domain may narrow too much.
- **Topic genuinely doesn't have a natural applied domain** — e.g., "intro to research methods" — skip and search the topic + "review" or "introduction".

If applied-domain search returns < 3 papers, **fall back to generic search** for that section. Don't pad with fabrications.

## Operational Pattern

In Phase 3 of the skill:

```
For each section in [proposed sections]:
  1. Construct primary query: "{topic} {applied-domain-keyword}" + year_min
  2. Submit to Consensus (sequential, 1 q/sec gap)
  3. If results >= 3: select papers, move on
  4. If results < 3: submit fallback "{topic}" + year_min
  5. Select 1-3 papers from combined results
```

## Anti-Patterns

### "Just search the topic"

Most common mistake. Produces theoretically rigorous but unhelpful papers for an applied course. Students can't connect them to course goals. Engagement drops.

### "Search the applied domain alone"

Without the topic anchor, query is too broad. "Food processing" returns 10,000+ papers across all subfields. Topic + applied-domain is the sweet spot.

### "Use multiple applied domains in one query"

"Enzyme kinetics food processing biomedical industrial applications" overconstrains. Each query targets ONE applied domain. If a section spans multiple domains, run separate queries.

### "Weave domain into queries even for pure-theory courses"

Pure-theory courses don't have applied domains. Forcing one in produces awkward queries that miss the actual theoretical literature.

### "Skip applied-domain weaving to save query budget"

The applied-domain weaving doesn't add queries — it modifies them. Same query budget, dramatically better relevance.

## Operational Checklist

- [ ] Course's applied domain identified (from title / department / description / learning outcomes)
- [ ] Each Phase 3 query: `{topic} + {applied-domain}` format
- [ ] Fallback to generic search if applied-domain returns < 3 papers
- [ ] Pure-theory courses: skip applied-domain weaving (use generic)
- [ ] Multi-domain sections: separate query per domain (don't stack in one query)

## Citations (7 sources)

1. **Bloom, B. S. (ed.), *Taxonomy of Educational Objectives* (1956).** Source for the application-tier of learning that justifies the applied-domain framing. Higher-tier learning (apply / analyze / evaluate) requires applied examples; pure-theory readings only support recall + comprehension.

2. **Mayer, R. E., *Multimedia Learning* (Cambridge, 2nd ed. 2009).** Empirical research on how applied examples accelerate learning vs abstract presentation. Source for the engagement-drop signal that pure-theory readings produce in applied courses.

3. **Fink, L. D., *Creating Significant Learning Experiences* (Jossey-Bass, 2003).** Source for the "integration" learning category — the discipline of connecting course content to students' applied contexts. Applied-domain weaving operationalizes this.

4. **Donald, J. G., *Learning to Think: Disciplinary Perspectives* (Jossey-Bass, 2002).** Empirical study of disciplinary thinking patterns. Justifies the per-discipline query-pattern table — engineering thinks differently from biology thinks differently from CS.

5. **Lave, J. & Wenger, E., *Situated Learning* (Cambridge, 1991).** Source for "situated cognition" — knowledge is best learned in the context of its application. Applied-domain weaving brings the readings into the situated context.

6. **Chickering, A. W. & Gamson, Z. F., "Seven Principles for Good Practice in Undergraduate Education" — *AAHE Bulletin*, 1987.** Principle #5 ("Emphasize Time on Task") + Principle #7 ("Respect Diverse Talents") favor applied-domain readings over pure-theory abstracts that don't connect to student backgrounds.

7. **Boyer, E. L., *Scholarship Reconsidered* (Carnegie Foundation, 1990).** Source for the "Scholarship of Application" framing. Applied-domain papers represent this scholarship category; weaving them into reading lists honors that scholarship.

FILE:references/audience_calibration.md
# Audience Calibration — Undergrad vs Grad Summary Jargon + Question Complexity

This reference answers exactly one decision: **how does the syllabus skill calibrate summary jargon and discussion question complexity to the course's audience (Q2)?**

## The Core Rule

The same paper needs **different summaries** for different audiences:

- **Undergrad-intro**: define every technical term; assume zero prior knowledge
- **Undergrad-advanced**: assume foundational vocabulary; explain field-specific terms
- **Grad-Masters**: assume technical fluency; brief context for novel concepts
- **Grad-doctoral**: assume technical + methodological fluency; brief mention only of established context

Same paper, different summaries. Generic summaries miss the engagement target.

## Audience Buckets (Q2)

| Bucket | Vocabulary assumption | Method assumption | Discussion question complexity |
|---|---|---|---|
| Undergraduate (intro) | Zero specialized | Zero | Recall + comprehension + simple application |
| Undergraduate (advanced) | Foundational vocab | Common methods | Application + analysis |
| Graduate (Masters / early PhD) | Technical fluency | Common research methods | Analysis + evaluation |
| Graduate (doctoral / advanced) | Technical + methodological fluency | Methods specifics | Evaluation + critique + synthesis |
| Professional / continuing ed | Field-specific assumed | Methods context-dependent | Application to practice |
| Mixed | Lowest bucket present | Same | Same |

## Summary Calibration

### Undergrad-intro

Every technical term defined. Plain language. Connects to common experience.

| ❌ Too jargon | ✅ Calibrated |
|---|---|
| "This RCT compared lipidomic profiles across dietary interventions to assess cardiometabolic risk modulation." | "This randomized study compared what happens to fat molecules in the blood when people eat different diets — Mediterranean, Nordic, vegetarian — and looked at how those changes might affect heart disease risk." |
| "The phylogenetic analysis identified convergent evolution of toxin-resistant Na+ channels across reptilian lineages." | "Researchers compared sodium-channel genes across snake species and found that snakes from very different evolutionary branches independently developed similar resistance to toxic prey." |

### Undergrad-advanced

Foundational vocabulary assumed. Explain field-specific terms briefly.

| ❌ Too dumbed-down | ✅ Calibrated |
|---|---|
| "This randomized study compared what happens to fat molecules in the blood..." | "This RCT (n=240) tracked lipidomic shifts across three dietary patterns — Mediterranean, Nordic, vegetarian — over 12 weeks. Cardiometabolic markers improved most in the Mediterranean arm." |
| "Researchers compared sodium-channel genes..." | "Phylogenetic analysis across 47 reptilian lineages identifies convergent evolution of Na+ channel modifications conferring resistance to neurotoxic prey." |

### Grad (Masters or doctoral)

Technical fluency assumed. Brief context for novel concepts. Method specifics if relevant.

| ❌ Too verbose | ✅ Calibrated |
|---|---|
| "This RCT (n=240) tracked lipidomic shifts across three dietary patterns over 12 weeks. Cardiometabolic markers improved most in Mediterranean." | "RCT (n=240, 12-week, parallel-arm) comparing Mediterranean / Nordic / vegetarian. Mediterranean → 14% lower LDL-particle count, 22% lower oxidized LDL; differences plausibly mediated by MUFA:SFA ratio." |
| "Phylogenetic analysis across 47 reptilian lineages identifies convergent evolution..." | "Bayesian phylogenetic analysis (47 lineages, BEAST 2.7) supports independent emergence of Na+ channel S6-domain modifications in 6 lineages; convergence rate inconsistent with neutral drift (PP > 0.95)." |

### Professional / continuing ed

Field-specific terms assumed. Emphasize practice implications.

| ❌ Too academic | ✅ Calibrated |
|---|---|
| "RCT (n=240, 12-week)... LDL-particle count down 14%..." | "12-week RCT shows Mediterranean diet improves LDL-particle metrics 14-22% vs comparators. Practice implication: nutritional counseling for cardiovascular-risk patients should emphasize MUFA-rich foods specifically, not just 'low-fat'." |

## Discussion Question Calibration

Use Bloom's revised taxonomy (Anderson & Krathwohl 2001):

| Level | Action verbs | Question pattern |
|---|---|---|
| Remember | identify, list, recall | "What is X?" "Name the components" |
| Understand | explain, summarize, classify | "Why does X happen?" "How would you describe Y?" |
| Apply | use, apply, demonstrate | "How could this method be applied to...?" "What would happen if we used X for Y?" |
| Analyze | compare, contrast, examine | "What patterns connect X and Y?" "Why do X and Y produce different results?" |
| Evaluate | judge, critique, defend | "Is this study's conclusion warranted by its methods?" "Which approach better serves goal Z, and why?" |
| Create | design, propose, construct | "Design a study that would test the limits of X." "Propose a novel application of Y to Z." |

### Calibration by audience

| Audience | Question levels | Avoid |
|---|---|---|
| Undergrad-intro | Remember + Understand + simple Apply | Pure recall ("what did authors find?") |
| Undergrad-advanced | Understand + Apply + simple Analyze | Sophisticated Evaluate / Create |
| Grad-Masters | Apply + Analyze + Evaluate | Pure recall (insulting) |
| Grad-doctoral | Analyze + Evaluate + Create | Anything below Apply |

### Examples per audience

#### Undergrad-intro

| ❌ Recall only | ✅ Calibrated |
|---|---|
| "What did the authors find?" | "If you wanted to lower your heart disease risk through diet, what does this study suggest you should change?" (Apply) |

#### Grad-doctoral

| ❌ Below level | ✅ Calibrated |
|---|---|
| "What did this RCT show?" | "How would you redesign this RCT to test whether MUFA:SFA ratio specifically (vs total fat composition) drives the lipidomic shift?" (Create) |

## Discussion Question Validator

`scripts/discussion_question_validator.py` flags:

- **Recall-only questions** (any audience): "what did authors find?", "summarize", "describe"
- **Below-audience questions**: undergrad-intro questions in grad course → flag
- **Above-audience questions**: doctoral-level questions in undergrad-intro → flag

Validator suggests upgrades by replacing verbs with audience-appropriate Bloom verbs.

## Tying Discussion Questions to Learning Outcomes

Beyond audience calibration, each question should **explicitly tie to a learning outcome**:

| Without LO tie | With LO tie |
|---|---|
| "How could this approach be applied to...?" | "Course outcome 3 says students should be able to design enzymatic processes. How would the kinetics described in this paper inform a process design for cheese ripening?" |

The LO tie:
- Reinforces course goals
- Shows students why the reading matters
- Creates assessable discussion behaviors

If learning outcomes were inferred (`[inferred]`), still tie discussion questions to them — flag both as inferred.

## Anti-Patterns

### "Same summary for all audiences"

The biggest engagement killer. Undergrad summaries that read like graduate abstracts produce blank stares; graduate summaries that read like K-12 explainers feel patronizing.

### "Add jargon to look academic in undergrad summaries"

Engagement signal: students underline / highlight content. Jargon-heavy summaries get less highlighting in undergrad classes. Plain-language summaries get more.

### "Generic discussion questions"

"What did the authors find?" works for any audience — and serves none. The discussion question is the engagement hook; generic questions waste it.

### "All discussion questions at the highest Bloom level"

In a grad-doctoral course, even one Create-level question per paper is taxing. Mix Analyze, Evaluate, Create. Don't make every reading require students to design a follow-up study.

## Operational Checklist

- [ ] Q2 audience parsed → calibration bucket selected
- [ ] All summaries calibrated to bucket
- [ ] All discussion questions calibrated to bucket's Bloom range
- [ ] Each discussion question tied to a learning outcome (explicit or inferred)
- [ ] Validator (`discussion_question_validator.py`) run on all questions
- [ ] Recall-only questions rejected
- [ ] Below-audience or above-audience questions reworked

## Citations (7 sources)

1. **Bloom, B. S. (1956); Anderson, L. W. & Krathwohl, D. R. (2001), *A Taxonomy for Learning, Teaching, and Assessing*.** The revised Bloom's taxonomy. Source for the 6-level question hierarchy + action verb lexicon.

2. **Marzano, R. J. & Kendall, J. S., *The New Taxonomy of Educational Objectives* (Corwin, 2007).** Modern alternative to Bloom; emphasizes meta-cognitive and self-system levels. Source for the validator's "below-level vs above-level" distinction.

3. **Hattie, J., *Visible Learning* (Routledge, 2008/2023 update).** Meta-meta-analysis of educational interventions. Effect size 0.6+ for "teacher clarity" justifies the audience-calibrated summary discipline (clarity is audience-relative).

4. **Bain, K., *What the Best College Teachers Do* (Harvard, 2004).** Source for the "tied to learning outcome" discipline. Bain's research found great teachers connect every reading explicitly to course-level goals; generic readings produce engagement drop.

5. **Walvoord, B. E. & Anderson, V. J., *Effective Grading* (Jossey-Bass, 2nd ed. 2010).** Source for the "discussion question is assessable behavior" framing. Each discussion question = an opportunity to assess whether learning outcomes are being met.

6. **Brookfield, S. D. & Preskill, S., *Discussion as a Way of Teaching* (Jossey-Bass, 2nd ed. 2005).** Source for the engagement-vs-jargon trade-off in summary writing. Brookfield's research: students engage with content they can paraphrase; jargon-heavy summaries reduce paraphrase capability.

7. **Bjork, R. A. & Bjork, E. L., "Making Things Hard on Yourself, but in a Good Way" — *Psychology and the Real World* (FABBS Foundation, 2011).** Source for the "desirable difficulty" framing. Discussion questions should be challenging at the audience's edge, not below it (insulting) or above it (defeating).

FILE:references/bundled_script_pattern.md
# Bundled Script Pattern — Why JS for DOCX Generation, Not Inline

This reference answers exactly one decision: **why does the syllabus skill ship a bundled `generate_reading_list.js` script rather than inlining the DOCX generation logic in SKILL.md?**

## The Core Trade

DOCX generation requires ~300 lines of `docx`-package boilerplate (table layouts, hyperlink patterns, list formatting, page setup, etc.). This logic is:

1. **Reusable** across runs — every reading list uses the same DOCX layout
2. **Mechanical** — no LLM judgment required; just JSON-in / DOCX-out
3. **Long-lived** — the layout doesn't change between runs

Inlining 300 lines of mechanical layout code in SKILL.md means:
- The skill prompt is much longer (token cost on every invocation)
- Layout changes require editing the skill prompt (high-risk)
- The skill body has to re-derive the same logic each run

Bundling the logic in `scripts/generate_reading_list.js` means:
- The skill body is ~200 lines lighter (token-efficient)
- Layout changes are isolated to one file
- The skill orchestrates; the script executes mechanically

## When to Bundle (vs Inline)

### Bundle when:

- ✅ The logic is mechanical (no LLM judgment)
- ✅ The logic is reusable across runs (same layout / same algorithm)
- ✅ The logic is non-trivial (>50 lines)
- ✅ The logic is in a non-Python language (JS, Go, Rust, etc.)
- ✅ The logic has external dependencies (`docx` package, `requests`, etc.)

### Inline (in SKILL.md) when:

- The logic requires LLM judgment per run (e.g., paper-summary writing)
- The logic is short (<20 lines) and run-specific
- The logic is in-context-only (uses session-specific tool calls)
- The logic varies significantly per invocation

## The Pattern Used Here

`scripts/generate_reading_list.js`:

1. **Accepts JSON input + output path as CLI args**
   ```bash
   node generate_reading_list.js --input data.json --output result.docx
   ```

2. **Has a documented JSON schema** (in SKILL.md so the orchestrator knows what to produce)

3. **Handles `docx` require with multi-location fallback** (works whether `docx` is installed locally, globally, or in a parent dir)

4. **Validates input** (missing fields → graceful error, not silent failure)

5. **Produces a clean professional DOCX** with:
   - Title page
   - Introduction (with Consensus link)
   - Learning outcomes box
   - Numbered papers per section
   - Footer with metadata

6. **Uses canonical `docx` patterns**:
   - `ExternalHyperlink` with full URLs
   - `LevelFormat.BULLET` for lists
   - Dual-width tables (`columnWidths` + cell `width`)

## Skill Orchestrator's Role

The skill body (SKILL.md):

1. Walks Phase 0 intake
2. Parses syllabus + extracts topics
3. Walks group-and-confirm checkpoint
4. Runs Consensus searches (LLM judgment per query)
5. Writes summaries + discussion questions (LLM judgment per paper)
6. **Constructs the JSON payload** matching the bundled script's schema
7. **Invokes the script** with the JSON
8. Validates output + delivers

The skill body is responsible for **what goes in the document**. The script is responsible for **how it's laid out**.

## Why Node.js Specifically

The `docx` library is a JavaScript library (npm package). Could the skill use a Python `docx` library (`python-docx`)? Yes, but:

- The repo's other research-pack DOCX-generating skills (litreview, grants, dossier) all use Node.js + `docx`
- Consistency: one DOCX library across the research pack
- The `docx` JS library is more actively maintained + has richer features
- `python-docx` doesn't support all the features the skill needs (advanced hyperlinks, table styling)

## File Structure

```
research/syllabus/skills/syllabus/scripts/
├── citation_tracker.py            ← stdlib Python (orchestration helper)
├── topic_grouper.py               ← stdlib Python (orchestration helper)
├── discussion_question_validator.py ← stdlib Python (orchestration helper)
└── generate_reading_list.js       ← BUNDLED Node.js (mechanical DOCX assembly)
```

The Python scripts are stateless helpers (per-run). The JS script is the bundled mechanical assembler (called once per run).

## Anti-Patterns

### "Inline the JS into a Python script via subprocess"

Adds an unnecessary layer. The skill should call `node` directly.

### "Convert JS logic to Python to keep all scripts in one language"

Loses access to the better-maintained `docx` JS library. Worse: would diverge from sibling skills (litreview, grants, dossier all use `docx` JS).

### "Keep the JS script but inline the JSON schema in the script"

The JSON schema needs to be IN SKILL.md so the orchestrator knows what to construct. Documenting it in the script alone hides it from the orchestrator's prompt context.

### "Inline 300 lines of docx code in SKILL.md"

The original anti-pattern. Bloats the prompt, makes layout changes risky, makes the skill body harder to read.

### "Import the script from another skill"

Cross-skill dependencies break the per-skill self-contained discipline (per CLAUDE.md anti-patterns). Even though it would save duplication, the bundled script lives within syllabus's own folder.

## Operational Checklist

- [ ] `scripts/generate_reading_list.js` exists in syllabus's scripts/ folder
- [ ] Script accepts `--input <json>` + `--output <docx>` CLI args
- [ ] Script handles `docx` require with multi-location fallback
- [ ] Script validates input (missing fields → graceful error)
- [ ] JSON schema documented in SKILL.md (not just in the script)
- [ ] Skill orchestrator constructs JSON matching the schema
- [ ] Skill orchestrator invokes the script via `node` (not `python`)
- [ ] DOCX output validated post-generation

## Citations (7 sources)

1. **Karpathy-coder discipline + write-a-skill conventions** (this repo's `engineering/write-a-skill/`). Source for the "stdlib-only Python tools, bundled non-Python scripts allowed for mechanical jobs" pattern.

2. **CLAUDE.md anti-pattern: "Don't add features beyond what the task requires."** The bundled script honors this — it does ONE thing (DOCX layout) and does it mechanically.

3. **`docx` Node.js package — github.com/dolanmiu/docx (MIT).** Authoritative source for the API patterns the bundled script uses. Active maintenance, comprehensive feature set.

4. **CommonJS / Node.js module resolution algorithm.** Source for the "multi-location fallback" pattern in the require statement. Ensures the script works in development (local node_modules) and production (global install).

5. **Twelve-Factor App principles — III. Config: store config in the environment.** Source for the CLI-args-not-config pattern. Script accepts input/output as args, not via env vars or config files.

6. **Brian Kernighan & P. J. Plauger, *Software Tools* (1976).** Source for the "do one thing well + compose" pattern. The bundled script does exactly one thing (mechanical DOCX assembly); the skill body composes it with the rest of the pipeline.

7. **Doug McIlroy / Unix philosophy.** Source for the broader pattern: "Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface." JSON-in / DOCX-out is the modern equivalent.

FILE:scripts/citation_tracker.py
#!/usr/bin/env python3
"""citation_tracker.py — Syllabus three-count audit + 1s sequential discipline.

Stdlib-only. Mirrors litreview's citation_tracker (research-pack convention)
adapted for syllabus's per-section search budget.

Tracked counts:
  - searches_total
  - searches_per_section
  - papers_received
  - papers_cited

Per-section detail recorded for DOCX audit log.
Enforces 1s sequential gap.

Usage:
    python citation_tracker.py --action start --session syllabus-bio101-20260515 --course "Intro Biology"
    python citation_tracker.py --action record_search --session ... --section "Cell Biology" --query "..."
    python citation_tracker.py --action record_received --session ... --section "Cell Biology" --count 3
    python citation_tracker.py --action record_cited --session ... --section "Cell Biology" --url "..."
    python citation_tracker.py --action status --session ...
"""

import argparse
import json
import sys
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Dict, List, Optional


SESSIONS_DIR = Path.home() / ".syllabus_sessions"
MIN_GAP_SECONDS = 1.0


def session_path(name: str) -> Path:
    return SESSIONS_DIR / f"{name}.json"


def load_session(name: str) -> Dict[str, Any]:
    p = session_path(name)
    if not p.exists():
        raise FileNotFoundError(f"Session not found: {name}")
    return json.loads(p.read_text(encoding="utf-8"))


def save_session(name: str, data: Dict[str, Any]) -> None:
    SESSIONS_DIR.mkdir(parents=True, exist_ok=True)
    session_path(name).write_text(json.dumps(data, indent=2), encoding="utf-8")


def now_iso() -> str:
    return datetime.now(timezone.utc).isoformat()


def now_ts() -> float:
    return datetime.now(timezone.utc).timestamp()


def action_start(name: str, course: Optional[str], audience: Optional[str], year_range: Optional[str]) -> Dict[str, Any]:
    if session_path(name).exists():
        raise FileExistsError(f"Session already exists: {name}")
    data: Dict[str, Any] = {
        "session": name,
        "course": course or "",
        "audience": audience or "",
        "year_range": year_range or "",
        "consensus_tier": None,
        "started_at": now_iso(),
        "ended_at": None,
        "searches": [],
        "received_log": [],
        "cited": [],
        "counts": {
            "searches_total": 0,
            "papers_received_total": 0,
            "papers_cited_total": 0,
        },
        "by_section": {},
    }
    save_session(name, data)
    return data


def action_record_search(name: str, section: str, query: str, tier: Optional[str]) -> Dict[str, Any]:
    data = load_session(name)
    if data["searches"]:
        last_ts = data["searches"][-1].get("ts", 0)
        gap = now_ts() - last_ts
        if gap < MIN_GAP_SECONDS:
            raise RuntimeError(
                f"Sequential discipline violated: {gap:.2f}s gap (need >= {MIN_GAP_SECONDS}s). "
                f"Wait {MIN_GAP_SECONDS - gap:.2f}s more."
            )
    if tier and not data["consensus_tier"]:
        data["consensus_tier"] = tier
    data["searches"].append({"section": section, "query": query, "tier": tier, "at": now_iso(), "ts": now_ts()})
    data["counts"]["searches_total"] += 1
    if section not in data["by_section"]:
        data["by_section"][section] = {"searches": 0, "received": 0, "cited": 0}
    data["by_section"][section]["searches"] += 1
    save_session(name, data)
    return data


def action_record_received(name: str, section: str, count: int) -> Dict[str, Any]:
    data = load_session(name)
    data["received_log"].append({"section": section, "count": count, "at": now_iso()})
    data["counts"]["papers_received_total"] += count
    if section not in data["by_section"]:
        data["by_section"][section] = {"searches": 0, "received": 0, "cited": 0}
    data["by_section"][section]["received"] += count
    save_session(name, data)
    return data


def action_record_cited(name: str, section: str, url: str, title: Optional[str]) -> Dict[str, Any]:
    data = load_session(name)
    if any(c["url"] == url for c in data["cited"]):
        return data
    data["cited"].append({"section": section, "url": url, "title": title, "at": now_iso()})
    data["counts"]["papers_cited_total"] += 1
    if section not in data["by_section"]:
        data["by_section"][section] = {"searches": 0, "received": 0, "cited": 0}
    data["by_section"][section]["cited"] += 1
    save_session(name, data)
    return data


def action_status(name: str) -> Dict[str, Any]:
    return load_session(name)


def action_close(name: str) -> Dict[str, Any]:
    data = load_session(name)
    if data.get("ended_at") is None:
        data["ended_at"] = now_iso()
        save_session(name, data)
    return data


def render_status_human(data: Dict[str, Any]) -> str:
    out: List[str] = []
    out.append(f"Session:           {data['session']}")
    out.append(f"Course:            {data.get('course', '(unset)')}")
    out.append(f"Audience:          {data.get('audience', '(unset)')}")
    out.append(f"Year range:        {data.get('year_range', '(unset)')}")
    out.append(f"Consensus tier:    {data.get('consensus_tier') or '(not detected)'}")
    out.append(f"Started:           {data['started_at']}")
    out.append(f"Ended:             {data.get('ended_at') or '(active)'}")
    out.append("")
    c = data["counts"]
    out.append(f"Total searches:    {c['searches_total']}")
    out.append(f"Total received:    {c['papers_received_total']}")
    out.append(f"Total cited:       {c['papers_cited_total']}")
    out.append("")
    if data["by_section"]:
        out.append("Per-section breakdown:")
        for section, stats in data["by_section"].items():
            out.append(f"  {section:<40s} {stats['searches']} searches → {stats['received']} received → {stats['cited']} cited")
    out.append("")
    out.append("Audit block (paste in DOCX audit-log section):")
    out.append(
        f"  Total queries: {c['searches_total']}. Papers received: {c['papers_received_total']}. "
        f"Papers cited: {c['papers_cited_total']}. "
        f"Plan tier: {data.get('consensus_tier') or 'undetected'}."
    )
    return "\n".join(out)


def main(argv: List[str]) -> int:
    parser = argparse.ArgumentParser(description=__doc__.split("\n")[0])
    parser.add_argument("--action", required=True, choices=["start", "record_search", "record_received", "record_cited", "status", "list", "close"])
    parser.add_argument("--session")
    parser.add_argument("--course")
    parser.add_argument("--audience")
    parser.add_argument("--year-range")
    parser.add_argument("--section")
    parser.add_argument("--query")
    parser.add_argument("--tier")
    parser.add_argument("--count", type=int)
    parser.add_argument("--url")
    parser.add_argument("--title")
    parser.add_argument("--output", choices=["human", "json"], default="human")
    args = parser.parse_args(argv)

    try:
        if args.action == "start":
            result = action_start(args.session, args.course, args.audience, args.year_range)
        elif args.action == "record_search":
            result = action_record_search(args.session, args.section, args.query, args.tier)
        elif args.action == "record_received":
            result = action_record_received(args.session, args.section, args.count)
        elif args.action == "record_cited":
            result = action_record_cited(args.session, args.section, args.url, args.title)
        elif args.action == "status":
            result = action_status(args.session)
        elif args.action == "close":
            result = action_close(args.session)
        else:
            SESSIONS_DIR.mkdir(parents=True, exist_ok=True)
            result = [{"session": p.stem, "data": json.loads(p.read_text(encoding="utf-8"))} for p in sorted(SESSIONS_DIR.glob("*.json"))]
    except (FileNotFoundError, FileExistsError, RuntimeError) as e:
        print(f"error: {e}", file=sys.stderr); return 2

    if args.output == "json":
        print(json.dumps(result, indent=2, default=str))
    else:
        if args.action == "list":
            print(json.dumps(result, indent=2, default=str))
        else:
            print(render_status_human(result))
    return 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))

FILE:scripts/discussion_question_validator.py
#!/usr/bin/env python3
"""discussion_question_validator.py — Bloom higher-order quality check.

Stdlib-only. Validates each discussion question against Bloom's revised
taxonomy (Anderson & Krathwohl 2001). Flags:

  - Recall-only questions (any audience): "what did authors find?", "summarize", etc.
  - Below-audience questions (e.g., grad-doctoral course with undergrad-intro questions)
  - Above-audience questions (e.g., undergrad-intro course with doctoral-level questions)

Suggests upgrades by replacing low-tier verbs with audience-appropriate Bloom verbs.

NO LLM CALLS. Pure regex + verb classification.

Usage:
    python discussion_question_validator.py --questions-file /tmp/questions.json --audience grad_masters
    python discussion_question_validator.py --question "What did the authors find?" --audience undergrad_intro
    python discussion_question_validator.py --sample
"""

import argparse
import json
import re
import sys
from typing import Any, Dict, List, Optional


VALID_AUDIENCES = ["undergrad_intro", "undergrad_advanced", "grad_masters", "grad_doctoral", "professional", "mixed"]


# Bloom's revised taxonomy verb classification
BLOOM_VERBS = {
    "remember": ["identify", "list", "recall", "name", "define", "label", "match", "recognize", "state", "what is", "what are", "what did", "describe what"],
    "understand": ["explain", "summarize", "classify", "compare", "contrast", "describe how", "interpret", "paraphrase", "translate"],
    "apply": ["use", "apply", "demonstrate", "implement", "execute", "carry out", "how could you use", "how would you apply", "how could this be applied", "what would happen if"],
    "analyze": ["compare", "contrast", "examine", "differentiate", "organize", "what patterns", "why do", "what connections", "deconstruct"],
    "evaluate": ["judge", "critique", "defend", "justify", "argue", "is this", "should we", "which is better", "do you agree", "evaluate the"],
    "create": ["design", "propose", "construct", "develop", "formulate", "create a", "design a", "what would you propose", "how would you redesign"],
}

# Audience → minimum acceptable Bloom level
AUDIENCE_MIN_BLOOM = {
    "undergrad_intro": 1,       # Remember+ acceptable, but apply+ preferred
    "undergrad_advanced": 2,    # Understand+
    "grad_masters": 3,          # Apply+
    "grad_doctoral": 4,         # Analyze+
    "professional": 3,          # Apply+ (practice-oriented)
    "mixed": 2,                 # Understand+ (lowest bucket present)
}

BLOOM_LEVEL_ORDER = ["remember", "understand", "apply", "analyze", "evaluate", "create"]


def classify_question(question: str) -> Dict[str, Any]:
    """Classify question by Bloom level."""
    q_lower = question.lower()
    detected_levels: List[str] = []
    matched_phrases: Dict[str, List[str]] = {}

    for level, verbs in BLOOM_VERBS.items():
        for verb in verbs:
            if re.search(rf"\b{re.escape(verb)}\b", q_lower):
                if level not in detected_levels:
                    detected_levels.append(level)
                matched_phrases.setdefault(level, []).append(verb)

    if not detected_levels:
        # Default heuristic: if starts with "what/why/how", probably understand or apply
        if q_lower.strip().startswith(("what", "why", "how")):
            detected_levels = ["understand"]
            matched_phrases["understand"] = ["(inferred from interrogative)"]
        else:
            detected_levels = ["unknown"]

    # Highest Bloom level detected
    highest_level = "unknown"
    highest_idx = -1
    for level in detected_levels:
        if level in BLOOM_LEVEL_ORDER:
            idx = BLOOM_LEVEL_ORDER.index(level)
            if idx > highest_idx:
                highest_idx = idx
                highest_level = level

    return {
        "question": question,
        "detected_levels": detected_levels,
        "highest_level": highest_level,
        "highest_level_index": highest_idx,
        "matched_phrases": matched_phrases,
    }


def validate_against_audience(question: str, audience: str) -> Dict[str, Any]:
    if audience not in VALID_AUDIENCES:
        raise ValueError(f"Invalid audience '{audience}'. Pick from: {VALID_AUDIENCES}")
    classification = classify_question(question)
    min_required_idx = AUDIENCE_MIN_BLOOM[audience] - 1  # convert level to 0-indexed
    detected_idx = classification["highest_level_index"]

    if detected_idx == -1:
        verdict = "WARN"
        message = f"Could not detect Bloom level. Manual review recommended."
    elif detected_idx < min_required_idx:
        verdict = "FAIL"
        required_level = BLOOM_LEVEL_ORDER[min_required_idx]
        message = (
            f"Question level '{classification['highest_level']}' is BELOW required minimum "
            f"'{required_level}' for {audience}. Rework with verbs from higher Bloom levels."
        )
    elif detected_idx > min_required_idx + 2:
        verdict = "WARN"
        target_level = BLOOM_LEVEL_ORDER[min_required_idx]
        message = (
            f"Question level '{classification['highest_level']}' may be ABOVE typical "
            f"{audience} level. Consider whether students can engage at {target_level} level."
        )
    else:
        verdict = "PASS"
        message = f"Question level '{classification['highest_level']}' appropriate for {audience}."

    suggested_upgrades: List[str] = []
    if verdict == "FAIL":
        target_level = BLOOM_LEVEL_ORDER[min_required_idx]
        suggested_upgrades = [
            f"Replace verb with: {', '.join(BLOOM_VERBS[target_level][:5])}",
            f"Pattern: '{BLOOM_VERBS[target_level][0]} [the {target_level} concept]...'",
        ]

    return {
        "verdict": verdict,
        "audience": audience,
        "min_required_level": BLOOM_LEVEL_ORDER[min_required_idx] if min_required_idx >= 0 else "unknown",
        "classification": classification,
        "message": message,
        "suggested_upgrades": suggested_upgrades,
    }


SAMPLE_QUESTIONS = [
    {"question": "What did the authors find?", "audience": "undergrad_intro"},
    {"question": "What did the authors find?", "audience": "grad_doctoral"},
    {"question": "How could you apply this method to clinical decision support for sepsis?", "audience": "grad_masters"},
    {"question": "Design a follow-up study that would test whether MUFA:SFA ratio specifically drives the lipidomic shift.", "audience": "grad_doctoral"},
    {"question": "Why does the Mediterranean diet improve lipoprotein profiles?", "audience": "undergrad_intro"},
]


def main(argv: List[str]) -> int:
    parser = argparse.ArgumentParser(description=__doc__.split("\n")[0])
    parser.add_argument("--question", help="Single question to validate")
    parser.add_argument("--questions-file", help="JSON file with [{question, audience}, ...] entries")
    parser.add_argument("--audience", choices=VALID_AUDIENCES, help="Course audience for the question(s)")
    parser.add_argument("--sample", action="store_true")
    parser.add_argument("--output", choices=["human", "json"], default="human")
    args = parser.parse_args(argv)

    results: List[Dict[str, Any]] = []
    try:
        if args.sample:
            for sq in SAMPLE_QUESTIONS:
                results.append(validate_against_audience(sq["question"], sq["audience"]))
        elif args.question and args.audience:
            results.append(validate_against_audience(args.question, args.audience))
        elif args.questions_file:
            from pathlib import Path
            p = Path(args.questions_file)
            if not p.exists():
                print(f"error: {args.questions_file} not found", file=sys.stderr); return 2
            data = json.loads(p.read_text(encoding="utf-8"))
            for item in data:
                results.append(validate_against_audience(item["question"], item["audience"]))
        else:
            parser.print_help(); return 0
    except ValueError as e:
        print(f"error: {e}", file=sys.stderr); return 2

    if args.output == "json":
        print(json.dumps(results, indent=2))
    else:
        for r in results:
            marker = {"PASS": "[ok]", "WARN": "[warn]", "FAIL": "[FAIL]"}[r["verdict"]]
            print(f"{marker} ({r['audience']:<20s}) {r['classification']['question'][:80]}")
            print(f"      Highest Bloom level: {r['classification']['highest_level']}; required: {r['min_required_level']}")
            print(f"      → {r['message']}")
            if r["suggested_upgrades"]:
                print(f"      Suggested upgrades:")
                for s in r["suggested_upgrades"]:
                    print(f"        - {s}")
            print()
    fail_count = sum(1 for r in results if r["verdict"] == "FAIL")
    return 1 if fail_count > 0 else 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))

FILE:scripts/generate_reading_list.js
#!/usr/bin/env node
/**
 * generate_reading_list.js — Bundled DOCX generator for syllabus skill.
 *
 * Accepts JSON input + output path as CLI args. Produces a clean professional
 * .docx reading list with title page, learning outcomes, sections of papers
 * (each with hyperlinked title + audience-calibrated summary + Bloom-tied
 * discussion question), and footer.
 *
 * Path-B build: this is the bundled mechanical layout logic. The skill
 * orchestrator constructs JSON; this script assembles the DOCX. ~300 lines.
 *
 * Handles `docx` package require with multi-location fallback (works whether
 * `docx` is installed locally, globally, or in a parent dir).
 *
 * JSON schema (documented in SKILL.md):
 *   { courseTitle, courseSubtitle, generatedDate, yearRange, introText,
 *     learningOutcomes: [], sections: [{ heading, papers: [...] }],
 *     auditLog: { totalQueriesSent, totalPapersReceived, totalPapersCited,
 *                  toolConstraints, searchDetails: [], failures: [] } }
 *
 * Usage:
 *   node generate_reading_list.js --input data.json --output result.docx
 */

'use strict';

const fs = require('fs');
const path = require('path');

// Multi-location require for docx package
function loadDocx() {
  const candidates = [
    'docx',                                           // Local node_modules
    path.join(process.cwd(), 'node_modules', 'docx'), // Explicit local
    '/usr/lib/node_modules/docx',                     // Global Linux
    '/usr/local/lib/node_modules/docx',               // Global macOS / brew
    path.join(process.env.HOME || '', '.npm-global', 'lib', 'node_modules', 'docx'),
  ];
  for (const candidate of candidates) {
    try {
      return require(candidate);
    } catch (e) {
      // try next
    }
  }
  console.error('error: cannot find `docx` npm package. Install with: npm install docx');
  process.exit(2);
}

const docx = loadDocx();
const {
  Document, Paragraph, TextRun, Packer, AlignmentType, HeadingLevel,
  ExternalHyperlink, Table, TableRow, TableCell, WidthType, ShadingType,
  LevelFormat, Footer, Header, PageNumber, PageBreak, BorderStyle,
} = docx;


// ----------------------------------------------------------------------------
// CLI args
// ----------------------------------------------------------------------------
function parseArgs() {
  const args = process.argv.slice(2);
  const opts = {};
  for (let i = 0; i < args.length; i++) {
    if (args[i] === '--input') opts.input = args[++i];
    else if (args[i] === '--output') opts.output = args[++i];
    else if (args[i] === '--help' || args[i] === '-h') {
      console.log('Usage: node generate_reading_list.js --input <data.json> --output <result.docx>');
      process.exit(0);
    }
  }
  if (!opts.input || !opts.output) {
    console.error('error: both --input and --output are required');
    console.error('Usage: node generate_reading_list.js --input <data.json> --output <result.docx>');
    process.exit(2);
  }
  return opts;
}


// ----------------------------------------------------------------------------
// Input validation
// ----------------------------------------------------------------------------
function validateInput(data) {
  const required = ['courseTitle', 'sections'];
  for (const field of required) {
    if (!data[field]) {
      console.error(`error: missing required field 'field' in input JSON`);
      process.exit(2);
    }
  }
  if (!Array.isArray(data.sections) || data.sections.length === 0) {
    console.error('error: sections must be a non-empty array');
    process.exit(2);
  }
  for (const section of data.sections) {
    if (!section.heading || !Array.isArray(section.papers)) {
      console.error('error: each section must have heading + papers array');
      process.exit(2);
    }
    for (const paper of section.papers) {
      if (!paper.title || !paper.url) {
        console.error('error: each paper must have title + url');
        process.exit(2);
      }
    }
  }
}


// ----------------------------------------------------------------------------
// DOCX building blocks
// ----------------------------------------------------------------------------
const NAVY = '1A3A5C';
const LIGHT_BLUE = 'E8F0F8';
const ACCENT_BLUE = '2E5C8A';
const GRAY = '808080';
const DARK_GRAY = '404040';

function buildTitlePage(data) {
  return [
    new Paragraph({
      children: [new TextRun({ text: data.courseTitle, bold: true, size: 48, color: NAVY })],
      alignment: AlignmentType.CENTER,
      spacing: { before: 2400, after: 200 },
    }),
    new Paragraph({
      children: [new TextRun({ text: 'Supplementary Reading List', bold: false, size: 28, color: ACCENT_BLUE })],
      alignment: AlignmentType.CENTER,
      spacing: { after: 200 },
    }),
    data.courseSubtitle ? new Paragraph({
      children: [new TextRun({ text: data.courseSubtitle, italics: true, size: 22, color: DARK_GRAY })],
      alignment: AlignmentType.CENTER,
      spacing: { after: 800 },
    }) : null,
    new Paragraph({
      children: [new TextRun({ text: `Generated: data.generatedDate || new Date().toISOString().split('T')[0]`, size: 18, color: GRAY })],
      alignment: AlignmentType.CENTER,
      spacing: { after: 100 },
    }),
    new Paragraph({
      children: [new TextRun({ text: `Year range: data.yearRange || 'last 2 years'`, size: 18, color: GRAY })],
      alignment: AlignmentType.CENTER,
      spacing: { after: 200 },
    }),
    new Paragraph({ children: [new PageBreak()] }),
  ].filter(Boolean);
}

function buildIntroSection(data) {
  const introText = data.introText || 'This supplementary reading list collects recent peer-reviewed research relevant to each section of the course. Each entry includes a plain-language summary calibrated to the course audience and a discussion question tied to the course learning outcomes.';
  return [
    new Paragraph({
      heading: HeadingLevel.HEADING_1,
      children: [new TextRun({ text: 'Introduction', color: NAVY, bold: true, size: 32 })],
      spacing: { after: 200 },
    }),
    new Paragraph({
      children: [new TextRun({ text: introText, size: 22 })],
      spacing: { after: 200 },
    }),
    new Paragraph({
      children: [
        new TextRun({ text: 'Papers sourced via ', size: 20 }),
        new ExternalHyperlink({
          link: 'https://consensus.app',
          children: [new TextRun({ text: 'Consensus', style: 'Hyperlink', size: 20 })],
        }),
        new TextRun({ text: ' academic search. URLs in this document link directly to Consensus paper records.', size: 20 }),
      ],
      spacing: { after: 400 },
    }),
  ];
}

function buildLearningOutcomesBox(outcomes) {
  if (!outcomes || outcomes.length === 0) return [];
  const cells = [
    new TableRow({
      children: [
        new TableCell({
          width: { size: 9000, type: WidthType.DXA },
          shading: { type: ShadingType.CLEAR, color: 'auto', fill: LIGHT_BLUE },
          children: [
            new Paragraph({
              children: [new TextRun({ text: 'Course Learning Outcomes', bold: true, size: 24, color: NAVY })],
              spacing: { after: 100 },
            }),
            ...outcomes.map(outcome => new Paragraph({
              children: [new TextRun({ text: '• ' + outcome, size: 20 })],
              spacing: { after: 60 },
            })),
          ],
        }),
      ],
    }),
  ];
  return [
    new Table({
      columnWidths: [9000],
      rows: cells,
    }),
    new Paragraph({ children: [new TextRun({ text: '', size: 4 })], spacing: { after: 400 } }),
  ];
}

function buildSection(section, sectionIndex) {
  const elements = [
    new Paragraph({
      heading: HeadingLevel.HEADING_1,
      children: [new TextRun({ text: `sectionIndex. section.heading`, color: NAVY, bold: true, size: 28 })],
      spacing: { before: 400, after: 200 },
    }),
  ];
  for (let i = 0; i < section.papers.length; i++) {
    const paper = section.papers[i];
    const paperNum = `sectionIndex.i + 1`;
    // Title (hyperlinked)
    elements.push(new Paragraph({
      children: [
        new TextRun({ text: `paperNum. `, bold: true, size: 22 }),
        new ExternalHyperlink({
          link: paper.url,
          children: [new TextRun({ text: paper.title, style: 'Hyperlink', size: 22, bold: true })],
        }),
      ],
      spacing: { after: 60 },
    }));
    // Author / journal / year (italic gray)
    const meta = `paper.authors || ''''''`;
    if (meta.trim()) {
      elements.push(new Paragraph({
        children: [new TextRun({ text: meta, italics: true, size: 18, color: GRAY })],
        spacing: { after: 60 },
      }));
    }
    // Summary
    if (paper.summary) {
      elements.push(new Paragraph({
        children: [
          new TextRun({ text: 'Summary: ', bold: true, size: 20 }),
          new TextRun({ text: paper.summary, size: 20 }),
        ],
        spacing: { after: 60 },
      }));
    }
    // Discussion question (blue accent)
    if (paper.question) {
      elements.push(new Paragraph({
        children: [
          new TextRun({ text: 'Discussion: ', bold: true, size: 20, color: ACCENT_BLUE }),
          new TextRun({ text: paper.question, size: 20 }),
        ],
        spacing: { after: 200 },
      }));
    }
  }
  return elements;
}

function buildAuditLogSection(audit) {
  if (!audit) return [];
  const elements = [
    new Paragraph({ children: [new PageBreak()] }),
    new Paragraph({
      heading: HeadingLevel.HEADING_1,
      children: [new TextRun({ text: 'Audit Log', color: NAVY, bold: true, size: 28 })],
      spacing: { after: 200 },
    }),
    new Paragraph({
      children: [
        new TextRun({ text: `Total queries sent: `, bold: true, size: 20 }),
        new TextRun({ text: `audit.totalQueriesSent || 0`, size: 20 }),
      ],
      spacing: { after: 60 },
    }),
    new Paragraph({
      children: [
        new TextRun({ text: `Total papers received: `, bold: true, size: 20 }),
        new TextRun({ text: `audit.totalPapersReceived || 0`, size: 20 }),
      ],
      spacing: { after: 60 },
    }),
    new Paragraph({
      children: [
        new TextRun({ text: `Total papers cited in this list: `, bold: true, size: 20 }),
        new TextRun({ text: `audit.totalPapersCited || 0`, size: 20 }),
      ],
      spacing: { after: 200 },
    }),
  ];
  if (audit.toolConstraints) {
    elements.push(new Paragraph({
      children: [
        new TextRun({ text: 'Tool constraints: ', bold: true, size: 20 }),
        new TextRun({ text: audit.toolConstraints, size: 20 }),
      ],
      spacing: { after: 200 },
    }));
  }
  if (Array.isArray(audit.searchDetails) && audit.searchDetails.length > 0) {
    elements.push(new Paragraph({
      children: [new TextRun({ text: 'Per-search detail:', bold: true, size: 22, color: NAVY })],
      spacing: { after: 100 },
    }));
    for (const sd of audit.searchDetails) {
      elements.push(new Paragraph({
        children: [
          new TextRun({ text: `• sd.section || 'Unassigned': `, bold: true, size: 18 }),
          new TextRun({ text: `"sd.query" → sd.papersReturned || 0 returned, sd.papersSelected || 0 selected (sd.status || 'OK')`, size: 18 }),
        ],
        spacing: { after: 40 },
      }));
    }
  }
  if (Array.isArray(audit.failures) && audit.failures.length > 0) {
    elements.push(new Paragraph({
      children: [new TextRun({ text: 'Failures:', bold: true, size: 22, color: 'AA0000' })],
      spacing: { before: 200, after: 100 },
    }));
    for (const f of audit.failures) {
      elements.push(new Paragraph({
        children: [new TextRun({ text: `• f`, size: 18, color: '880000' })],
        spacing: { after: 40 },
      }));
    }
  }
  return elements;
}

function buildFooter(data) {
  return new Footer({
    children: [
      new Paragraph({
        children: [new TextRun({ text: `data.courseTitle — Supplementary Reading List`, size: 16, color: GRAY })],
        alignment: AlignmentType.CENTER,
      }),
    ],
  });
}


// ----------------------------------------------------------------------------
// Main
// ----------------------------------------------------------------------------
function main() {
  const opts = parseArgs();
  let data;
  try {
    data = JSON.parse(fs.readFileSync(opts.input, 'utf-8'));
  } catch (e) {
    console.error(`error: cannot read input JSON opts.input: e.message`);
    process.exit(2);
  }

  validateInput(data);

  const sections = data.sections.map((s, i) => buildSection(s, i + 1)).flat();

  const doc = new Document({
    creator: 'syllabus skill',
    title: `data.courseTitle — Supplementary Reading List`,
    description: 'Generated by syllabus skill via bundled generate_reading_list.js',
    sections: [
      {
        properties: {
          page: {
            margin: { top: 1440, right: 1440, bottom: 1440, left: 1440 }, // 1 inch
            size: { width: 12240, height: 15840 }, // US Letter
          },
        },
        footers: { default: buildFooter(data) },
        children: [
          ...buildTitlePage(data),
          ...buildIntroSection(data),
          ...buildLearningOutcomesBox(data.learningOutcomes),
          ...sections,
          ...buildAuditLogSection(data.auditLog),
        ],
      },
    ],
  });

  Packer.toBuffer(doc).then(buffer => {
    fs.writeFileSync(opts.output, buffer);
    console.log(`Generated: opts.output (buffer.length bytes, data.sections.length sections, data.sections.reduce((sum, s) => sum + s.papers.length, 0) papers)`);
  }).catch(e => {
    console.error(`error: DOCX packing failed: e.message`);
    process.exit(2);
  });
}

main();

FILE:scripts/topic_grouper.py
#!/usr/bin/env python3
"""topic_grouper.py — Heuristic 6-12 section grouping from extracted syllabus topics.

Stdlib-only. Given a list of extracted course topics, produce a proposed
grouping into 6-12 sections by detecting shared keywords.

The output feeds the Phase 2 group-and-confirm checkpoint where the user
can override (proceed / merge / split / add / remove).

Algorithm:
  1. Tokenize each topic into significant words (stop-words removed)
  2. Build word → topics inverted index
  3. Greedy clustering: topics sharing 2+ significant words → same section
  4. Cap at 12 sections (over-cap → merge smallest); ensure minimum 6 (under → split largest)
  5. Each section gets a heading derived from its dominant shared keyword

NO LLM CALLS. Pure tokenization + clustering.

Usage:
    python topic_grouper.py --topics "Cell biology, DNA replication, Protein synthesis, ..."
    python topic_grouper.py --topics-file /tmp/topics.json
    python topic_grouper.py --sample
"""

import argparse
import json
import re
import sys
from collections import Counter, defaultdict
from typing import Any, Dict, List, Set


STOP_WORDS = {
    "the", "a", "an", "and", "or", "but", "if", "of", "in", "on", "at", "to",
    "for", "with", "by", "from", "is", "are", "was", "were", "be", "been",
    "this", "that", "these", "those", "introduction", "overview", "basics",
    "fundamentals", "principles", "concepts", "topics", "review", "advanced",
    "intermediate", "i", "ii", "iii", "iv", "v", "1", "2", "3", "4", "5",
    "6", "7", "8", "9", "10", "11", "12", "week", "lecture", "chapter", "unit",
    "module", "lesson", "section",
}

MIN_SECTIONS = 6
MAX_SECTIONS = 12
SHARED_WORD_THRESHOLD = 2


def tokenize(topic: str) -> Set[str]:
    """Extract significant words from a topic string."""
    words = re.findall(r"\b[a-z]{3,}\b", topic.lower())
    return {w for w in words if w not in STOP_WORDS}


def cluster_topics(topics: List[str]) -> List[Dict[str, Any]]:
    """Cluster topics by shared significant words."""
    topic_tokens = [(i, t, tokenize(t)) for i, t in enumerate(topics)]
    clusters: List[List[int]] = []  # list of topic-index lists
    assigned: Set[int] = set()

    for i, _, tokens_i in topic_tokens:
        if i in assigned:
            continue
        # Start a new cluster with topic i
        cluster = [i]
        assigned.add(i)
        # Try to add other topics that share >= SHARED_WORD_THRESHOLD tokens
        for j, _, tokens_j in topic_tokens:
            if j in assigned or j == i:
                continue
            shared = tokens_i & tokens_j
            if len(shared) >= SHARED_WORD_THRESHOLD:
                cluster.append(j)
                assigned.add(j)
        clusters.append(cluster)

    return _normalize_to_size(clusters, topic_tokens)


def _normalize_to_size(clusters: List[List[int]], topic_tokens: List[tuple]) -> List[Dict[str, Any]]:
    """Ensure 6-12 sections by merging smallest or splitting largest."""
    # Merge smallest if over MAX_SECTIONS
    while len(clusters) > MAX_SECTIONS:
        clusters.sort(key=len)
        smallest = clusters.pop(0)
        # Merge into next-smallest
        if clusters:
            clusters[0].extend(smallest)
        else:
            clusters.append(smallest)

    # Split largest if under MIN_SECTIONS (and largest has >= 4 items)
    while len(clusters) < MIN_SECTIONS and clusters:
        clusters.sort(key=len, reverse=True)
        largest = clusters.pop(0)
        if len(largest) >= 4:
            mid = len(largest) // 2
            clusters.extend([largest[:mid], largest[mid:]])
        else:
            clusters.insert(0, largest)
            break  # Can't split further

    # Generate section heading per cluster (most-common shared word)
    sections: List[Dict[str, Any]] = []
    topic_lookup = {i: (t, tokens) for i, t, tokens in topic_tokens}
    for cluster_indices in clusters:
        all_tokens: Counter = Counter()
        cluster_topics: List[str] = []
        for idx in cluster_indices:
            topic, tokens = topic_lookup[idx]
            cluster_topics.append(topic)
            all_tokens.update(tokens)
        # Heading = top 1-3 most common tokens, capitalized
        top_words = [w for w, _ in all_tokens.most_common(2)]
        heading = " + ".join(w.capitalize() for w in top_words) if top_words else f"Section {len(sections) + 1}"
        sections.append({
            "heading": heading,
            "topic_count": len(cluster_indices),
            "topics": cluster_topics,
        })

    return sections


SAMPLE_TOPICS = [
    "Cell Biology Fundamentals",
    "DNA Replication",
    "Protein Synthesis",
    "Cell Division and Mitosis",
    "Mendelian Genetics",
    "Population Genetics",
    "Evolution and Natural Selection",
    "Speciation",
    "Ecology Basics",
    "Ecosystem Dynamics",
    "Energy Flow in Ecosystems",
    "Conservation Biology",
    "Plant Anatomy",
    "Plant Physiology",
    "Animal Anatomy Overview",
    "Animal Behavior",
    "Microbiology Introduction",
    "Bacterial Genetics",
    "Viruses and Pathogens",
]


def main(argv: List[str]) -> int:
    parser = argparse.ArgumentParser(description=__doc__.split("\n")[0])
    parser.add_argument("--topics", help="Comma-separated topic list")
    parser.add_argument("--topics-file", help="Path to JSON file with topics array")
    parser.add_argument("--sample", action="store_true")
    parser.add_argument("--output", choices=["human", "json"], default="human")
    args = parser.parse_args(argv)

    if args.sample:
        topics = SAMPLE_TOPICS
    elif args.topics:
        topics = [t.strip() for t in args.topics.split(",") if t.strip()]
    elif args.topics_file:
        from pathlib import Path
        p = Path(args.topics_file)
        if not p.exists():
            print(f"error: {args.topics_file} not found", file=sys.stderr); return 2
        topics = json.loads(p.read_text(encoding="utf-8"))
    else:
        parser.print_help(); return 0

    sections = cluster_topics(topics)
    result = {
        "input_topic_count": len(topics),
        "section_count": len(sections),
        "sections": sections,
    }

    if args.output == "json":
        print(json.dumps(result, indent=2))
    else:
        print(f"Input topics:    {len(topics)}")
        print(f"Output sections: {len(sections)}  (target: {MIN_SECTIONS}-{MAX_SECTIONS})")
        print()
        print("Proposed sections (present this at Phase 2 checkpoint):")
        for i, s in enumerate(sections, 1):
            print(f"")
            print(f"  Section {i}: {s['heading']}  ({s['topic_count']} topics)")
            for t in s["topics"]:
                print(f"    - {t}")
        print()
        print("Group-and-confirm checkpoint forcing options:")
        print("  1. Looks good — proceed with these sections")
        print("  2. Merge sections [X] and [Y]")
        print("  3. Split section [X] into two")
        print("  4. Add a section for [topic]")
        print("  5. Remove section [X]")
    return 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))

---
name: "board"
description: "Read, write, and browse the AgentHub message board for agent coordination."
command: /hub:board
---

# /hub:board — Message Board

Interface for the AgentHub message board. Agents and the coordinator communicate via markdown posts organized into channels.

## Usage

```
/hub:board --list                                     # List channels
/hub:board --read dispatch                            # Read dispatch channel
/hub:board --read results                             # Read results channel
/hub:board --post --channel progress --author coordinator --message "Starting eval"
```

## What It Does

### List Channels

```bash
python {skill_path}/scripts/board_manager.py --list
```

Output:
```
Board Channels:

  dispatch        2 posts
  progress        4 posts
  results         3 posts
```

### Read Channel

```bash
python {skill_path}/scripts/board_manager.py --read {channel}
```

Displays all posts in chronological order with frontmatter metadata.

### Post Message

```bash
python {skill_path}/scripts/board_manager.py \
  --post --channel {channel} --author {author} --message "{text}"
```

### Reply to Thread

```bash
python {skill_path}/scripts/board_manager.py \
  --thread {post-id} --message "{text}" --author {author}
```

## Channels

| Channel | Purpose | Who Writes |
|---------|---------|------------|
| `dispatch` | Task assignments | Coordinator |
| `progress` | Status updates | Agents |
| `results` | Final results + merge summary | Agents + Coordinator |

## Post Format

All posts use YAML frontmatter:

```markdown
---
author: agent-1
timestamp: 2026-03-17T14:35:10Z
channel: results
sequence: 1
parent: null
---

Message content here.
```

Example result post for a content task:

```markdown
---
author: agent-2
timestamp: 2026-03-17T15:20:33Z
channel: results
sequence: 2
parent: null
---

## Result Summary

- **Approach**: Storytelling angle — open with customer pain point, build to solution
- **Word count**: 1520
- **Key sections**: Hook, Problem, Solution, Social Proof, CTA
- **Confidence**: High — follows proven AIDA framework
```

## Board Rules

- **Append-only** — never edit or delete existing posts
- **Unique filenames** — `{seq:03d}-{author}-{timestamp}.md`
- **Frontmatter required** — every post has author, timestamp, channel

---
name: "board-prep"
description: "Board meeting preparation for the adversarial scenario, not the friendly one. Forces numbers-cold mastery, anticipates hard questions, builds a narrative that acknowledges weakness without losing the room. Use when preparing for a board meeting, an investor update, fundraising presentation, or any high-stakes adversarial review where every number must live in your head not just on a slide."
---

# /em:board-prep — Board Meeting Preparation

**Command:** `/em:board-prep <agenda>`

Prepare for the adversarial version of your board, not the friendly one. Every hard question they'll ask. Every number you need cold. The narrative that acknowledges weakness without losing the room.

---

## The Reality of Board Meetings

Your board members have seen 50+ companies. They've watched founders flinch at their own numbers, spin bad news as "learning opportunities," and present sanitized decks that hide what's actually happening.

They know when you're not being straight with them. The question isn't whether they'll ask the hard questions — it's whether you're ready for them.

The best board meetings aren't the ones where everything looks good. They're the ones where the CEO demonstrates they see reality clearly, have a plan, and can execute under pressure.

---

## The Preparation Framework

### Phase 1: Numbers Cold

Before the meeting, every number in your deck should live in your head, not just the slide.

**The numbers you must know without looking:**
- Current MRR / ARR and month-over-month growth rate
- Burn rate (monthly) and runway (months at current burn)
- Headcount by department
- CAC and LTV by channel / segment
- Net Revenue Retention
- Pipeline: value, conversion rate, average sales cycle
- Churn: rate, top reasons, top churned accounts
- Gross margin (product), net margin (company)
- Key hiring positions open and time-to-fill

**Stress test yourself:** Can you answer "what's your burn?" without hesitation? "What's your churn rate by segment?" If you pause, you don't know it.

### Phase 2: Anticipate the Hard Questions

For every item on the agenda, generate the adversarial version of the question.

**Standard adversarial questions by topic:**

*Revenue performance:*
- "You missed revenue by 20% this quarter. What specifically failed?"
- "Is this a pipeline problem, a conversion problem, or a capacity problem?"
- "If you missed because of one big deal, how dependent is your model on individual deals?"
- "When do you project recovery and what are the leading indicators you're right?"

*Runway / burn:*
- "At current burn you have N months. What's your plan if the next round takes 9 months?"
- "What would you cut first if you had to extend runway by 6 months today?"
- "Is there a scenario where you don't raise another round?"

*Product / roadmap:*
- "You shipped X. What did customers actually do with it?"
- "What did you kill this quarter and why?"
- "Where are you behind on roadmap? What's slipping?"

*Team:*
- "Who's at risk of leaving? How would that affect execution?"
- "You've had 3 VP-level hires not work out. What pattern do you see?"
- "Is the team the right team for this stage?"

*Competition:*
- "Competitor Y just raised $50M. How does that change your position?"
- "If they copy your best feature in 90 days, what's your moat?"

### Phase 3: Build the Narrative

The board meeting isn't a status update. It's a leadership demonstration.

**The structure that works:**

1. **Where we are (honest)** — Current state of business, the real number, not the smoothed one
2. **What we learned** — What the data is telling us that we didn't know 90 days ago
3. **What we got wrong** — Name it directly. Don't make them ask.
4. **What we're doing about it** — Specific, dated, owned actions
5. **What we need from this room** — Concrete ask. Not "support" — specific introductions, decisions, resources.

**The rule on bad news:** Never let the board be surprised. If a quarter went badly, they should know before the deck. A 5-sentence email 3 days before: "Revenue came in at $X vs $Y target. Here's what happened, here's what I'm doing, here's what I need from you."

### Phase 4: Adversarial Preparation

Do a mock board meeting. Have someone play the hardest director you have.

**The simulation:**
- Present your deck as you would
- The mock director asks every uncomfortable question
- You answer without referring to the deck
- After: note every question that made you pause or feel defensive

**The questions that made you defensive = the questions you need to prepare for.**

### Phase 5: Director-by-Director Prep

Not all board members want the same thing from a meeting.

**For each director, know:**
- Their primary concern right now (usually tied to their investment thesis)
- The metric they watch most closely
- What would make them lose confidence in you
- What they've said in the last meeting that you should address

**Common director types:**
- **The operator** — wants to know what's breaking and who owns fixing it
- **The financial investor** — focused on path to profitability or next raise
- **The strategic investor** — worried about competitive position and moat
- **The independent** — watching governance, team dynamics, and your judgment

---

## Pre-Meeting Checklist

**48 hours before:**
- [ ] All numbers verified against source systems (not last week's export)
- [ ] Deck reviewed for internal consistency
- [ ] Pre-read sent to board (deck + 1-page brief on key topics)
- [ ] One-on-ones done with any director likely to have concerns
- [ ] 3 hardest questions you expect — rehearsed out loud

**Day of meeting:**
- [ ] Agenda with time allocations distributed
- [ ] Know the ask for each agenda item (decision needed, input wanted, FYI)
- [ ] Materials to leave behind prepared
- [ ] Follow-up action template ready

---

## During the Meeting

**What the board is watching:**
- Do you own the bad news or deflect it?
- Are you defending a narrative or sharing reality?
- Do you know your numbers or do you look things up?
- When challenged, do you get defensive or engage?
- Do you know what you don't know?

**The single best thing you can do:** Name the hard thing before they do. "I want to address the revenue miss directly. Here's what happened, here's what I should have caught earlier, here's what changes."

---

## After the Meeting

Within 24 hours:
- Send action items with owners and dates
- Send any data you promised but didn't have
- Note the questions that came up you weren't ready for
- Schedule follow-up with any director who seemed unsatisfied

The next board prep starts now.

---
name: tc
description: Track technical changes with structured records, a state machine, and session handoff. Usage: /tc <init|create|update|status|resume|close|export|dashboard> [args]
---

# /tc — Technical Change Tracker

Dispatch a TC (Technical Change) command. Arguments: `$ARGUMENTS`.

If `$ARGUMENTS` is empty, print this menu and stop:

```
/tc init                       Initialize TC tracking in this project
/tc create <name>              Create a new TC record
/tc update <tc-id> [...]       Update fields, status, files, handoff
/tc status [tc-id]             Show one TC or the registry summary
/tc resume <tc-id>             Resume a TC from a previous session
/tc close <tc-id>              Transition a TC to deployed
/tc export                     Re-render derived artifacts
/tc dashboard                  Re-render the registry summary
```

Otherwise, parse `$ARGUMENTS` as `<subcommand> <rest>` and dispatch to the matching protocol below. All scripts live at `engineering/tc-tracker/scripts/`.

## Subcommands

### `init`

1. Run:
   ```bash
   python3 engineering/tc-tracker/scripts/tc_init.py --root . --json
   ```
2. If status is `already_initialized`, report current statistics and stop.
3. Otherwise report what was created and suggest `/tc create <name>` as the next step.

### `create <name>`

1. Parse `<name>` as a kebab-case slug. If missing, ask the user for one.
2. Prompt the user (one question at a time) for:
   - Title (5-120 chars)
   - Scope: `feature | bugfix | refactor | infrastructure | documentation | hotfix | enhancement`
   - Priority: `critical | high | medium | low` (default `medium`)
   - Summary (10+ chars)
   - Motivation
3. Run:
   ```bash
   python3 engineering/tc-tracker/scripts/tc_create.py --root . \
     --name "<slug>" --title "<title>" --scope <scope> --priority <priority> \
     --summary "<summary>" --motivation "<motivation>" --json
   ```
4. Report the new TC ID and the path to the record.

### `update <tc-id> [intent]`

1. If `<tc-id>` is missing, list active TCs (status `in_progress` or `blocked`) from `tc_status.py --all` and ask which one.
2. Determine the user's intent from natural language:
   - **Status change** → `--set-status <state>` with `--reason "<why>"`
   - **Add files** → one or more `--add-file path[:action]`
   - **Add a test** → `--add-test "<title>" --test-procedure "<step>" --test-expected "<result>"`
   - **Update handoff** → any combination of `--handoff-progress`, `--handoff-next`, `--handoff-blocker`, `--handoff-context`
   - **Add a note** → `--note "<text>"`
   - **Add a tag** → `--tag <tag>`
3. Run:
   ```bash
   python3 engineering/tc-tracker/scripts/tc_update.py --root . --tc-id <tc-id> [flags] --json
   ```
4. If exit code is non-zero, surface the error verbatim. The state machine and validator will reject invalid moves — do not retry blindly.

### `status [tc-id]`

- If `<tc-id>` is provided:
  ```bash
  python3 engineering/tc-tracker/scripts/tc_status.py --root . --tc-id <tc-id>
  ```
- Otherwise:
  ```bash
  python3 engineering/tc-tracker/scripts/tc_status.py --root . --all
  ```

### `resume <tc-id>`

1. Run:
   ```bash
   python3 engineering/tc-tracker/scripts/tc_status.py --root . --tc-id <tc-id> --json
   ```
2. Display the handoff block prominently: `progress_summary`, `next_steps` (numbered), `blockers`, `key_context`.
3. Ask: "Resume <tc-id> and pick up at next step 1? (y/n)"
4. If yes, run an update to record the resumption:
   ```bash
   python3 engineering/tc-tracker/scripts/tc_update.py --root . --tc-id <tc-id> \
     --note "Session resumed" --reason "session handoff"
   ```
5. Begin executing the first item in `next_steps`. Do NOT re-derive context — trust the handoff.

### `close <tc-id>`

1. Read the record via `tc_status.py --tc-id <tc-id> --json`.
2. Verify the current status is `tested`. If not, refuse and tell the user which transitions are still required.
3. Check `test_cases`: warn if any are `pending`, `fail`, or `blocked`.
4. Ask the user:
   - "Who is approving? (your name, or 'self')"
   - "Approval notes (optional):"
   - "Test coverage status: none / partial / full"
5. Run:
   ```bash
   python3 engineering/tc-tracker/scripts/tc_update.py --root . --tc-id <tc-id> \
     --set-status deployed --reason "Approved by <approver>" --note "Approval: <approver> — <notes>"
   ```
   Then directly edit the `approval` block via a follow-up update if your script version supports it; otherwise instruct the user to record approval in `notes`.
6. Report: "TC-NNN closed and deployed."

### `export`

There is no automatic HTML export in this skill. Re-validate everything instead:

1. Read the registry.
2. For each record, run:
   ```bash
   python3 engineering/tc-tracker/scripts/tc_validator.py --record <path> --json
   ```
3. Run:
   ```bash
   python3 engineering/tc-tracker/scripts/tc_validator.py --registry docs/TC/tc_registry.json --json
   ```
4. Report: total records validated, any errors, paths to anything invalid.

### `dashboard`

Run the all-records summary:
```bash
python3 engineering/tc-tracker/scripts/tc_status.py --root . --all
```

## Iron Rules

1. **Never edit `tc_record.json` by hand.** Always use `tc_update.py` so revision history is appended and validation runs.
2. **Never skip the state machine.** Walk forward through states even if it feels redundant.
3. **Never delete a TC.** History is append-only — add a final revision and tag it `[CANCELLED]`.
4. **Background bookkeeping.** When mid-task, spawn a background subagent to update the TC. Do not pause coding to do paperwork.
5. **Validate before reporting success.** If a script exits non-zero, surface the error and stop.

## Related Skills

- `engineering/tc-tracker` — Full SKILL.md with schema reference, lifecycle diagrams, and the handoff format.
- `engineering/changelog-generator` — Pair with TC tracker: TCs for the per-change audit trail, changelog for user-facing release notes.
- `engineering/tech-debt-tracker` — For tracking long-lived debt rather than discrete code changes.

---
name: "business-growth-skills"
description: "4 business growth agent skills and plugins for Claude Code, Codex, Gemini CLI, Cursor, OpenClaw. Customer success (health scoring, churn), sales engineer (RFP), revenue operations (pipeline, GTM), contract & proposal writer. Python tools (stdlib-only)."
version: 2.9.0
author: Alireza Rezvani
license: MIT
tags:
  - business
  - customer-success
  - sales
  - revenue-operations
  - growth
agents:
  - claude-code
  - codex-cli
  - openclaw
---

# Business & Growth Skills

4 production-ready skills for customer success, sales, and revenue operations.

## Quick Start

### Claude Code
```
/read business-growth/customer-success-manager/SKILL.md
```

### Codex CLI
```bash
npx agent-skills-cli add alirezarezvani/claude-skills/business-growth
```

## Skills Overview

| Skill | Folder | Focus |
|-------|--------|-------|
| Customer Success Manager | `customer-success-manager/` | Health scoring, churn prediction, expansion |
| Sales Engineer | `sales-engineer/` | RFP analysis, competitive matrices, PoC planning |
| Revenue Operations | `revenue-operations/` | Pipeline analysis, forecast accuracy, GTM metrics |
| Contract & Proposal Writer | `contract-and-proposal-writer/` | Proposal generation, contract templates |

## Python Tools

9 scripts, all stdlib-only:

```bash
python3 customer-success-manager/scripts/health_score_calculator.py --help
python3 revenue-operations/scripts/pipeline_analyzer.py --help
```

## Rules

- Load only the specific skill SKILL.md you need
- Use Python tools for scoring and metrics, not manual estimates

---
name: business-operations-skills
description: Use when running, diagnosing, or designing internal business operations — process documentation, vendor SLAs, capacity planning, internal comms, SOP/runbook authoring, procurement spend. Triggers on "BizOps review", "where's the bottleneck", "vendor health", "internal SOP", "all-hands deck", "spend categorization", "capacity for Q3", "process mapping". Forks context to route to one of six BizOps sub-skills (process-mapper, vendor-management, capacity-planner, internal-comms, knowledge-ops, procurement-optimizer) and returns a digest. Distinct from business-growth (external sales motion) and c-level-advisor (strategic, not operational).
context: fork
version: 2.8.0
author: claude-code-skills
license: MIT
tags: [bizops, operations, process, vendor, capacity, sop, procurement, coo, orchestrator]
compatible_tools: [claude-code, codex-cli, cursor, antigravity, opencode, gemini-cli]
---

# Business Operations — Domain Orchestrator

The BizOps surface is **internal**: how the company actually runs. This orchestrator forks its conversation context, routes your inquiry to one of six sub-skills, then returns a tight digest to the parent thread. The heavy ingestion (vendor catalogs, process interviews, multi-doc SOP intake) stays in the forked context.

## When to invoke

| Symptom | Sub-skill to route to |
|---|---|
| "Where does the work spend most of its time waiting?" | `process-mapper` |
| "Is this vendor delivering against the SLA?" | `vendor-management` |
| "Do we have enough people to ship in Q3?" | `capacity-planner` |
| "I need to brief the company on a re-org" | `internal-comms` |
| "Write me a runbook for the incident response process" | `knowledge-ops` |
| "Why is our software spend up 40% YoY?" | `procurement-optimizer` |

## Routing logic (deterministic)

The orchestrator classifies the inquiry by **signals** detected in the prompt. Two-signal threshold for confident routing; one-signal triggers a clarifying question.

### Signal table

| Signal class | Keywords | Sub-skill |
|---|---|---|
| **PROCESS** | bottleneck, cycle time, waiting, handoff, BPMN, process map, workflow | `process-mapper` |
| **VENDOR** | vendor, supplier, SLA, contract, third-party, MSA, SaaS subscription, renewal | `vendor-management` |
| **CAPACITY** | headcount, capacity, utilization, planning, hiring sequence, FTE | `capacity-planner` |
| **COMMS** | all-hands, internal newsletter, announcement, change management, FAQ, town hall | `internal-comms` |
| **KNOWLEDGE** | SOP, runbook, knowledge base, wiki, playbook, documentation, onboarding doc | `knowledge-ops` |
| **PROCUREMENT** | spend, procurement, purchase, supplier rationalization, software audit, SaaS sprawl | `procurement-optimizer` |

If signals are mixed (e.g., "vendor SLA + spend audit"), run the **highest-confidence sub-skill first**, then chain into the second one in a follow-up forked turn.

### Fallback

If no signal class scores ≥ 2, ask **one** clarifying question naming the two most likely candidates. Do NOT guess silently.

## Workflow (Matt Pocock grill discipline)

Derived from Matt Pocock's `grill-with-docs` pattern: **explore-then-ask, one question per turn with a recommended answer, walk the decision tree depth-first, track dependencies, anchor every challenge in the documented canon** (`references/`).

### Step 1 — Explore before asking

Before any clarifying question, check:
- Does the user's working directory already contain a process map, vendor catalog, SOP, or org chart we can grep?
- Does the inquiry already disambiguate the lane (e.g., "vendor SLA review" — that's `vendor-management`, no question needed)?
- Is the lane unambiguous from filenames mentioned (`procurement-Q3.csv` → procurement)?

If the codebase resolves the lane, **route silently**. Don't ask.

### Step 2 — If still ambiguous, ONE forcing question with a recommended answer

Matt's rule: never bundle questions. Never default to "what do you think?". Always offer your recommendation.

Pattern:
```
Q1/1: [precise question naming the two candidate lanes]
Recommended: [Lane X, because <one-sentence rationale from the signal table>]

(Confirm, or override?)
```

Wait for the user's response. **Then** route. Never guess silently after a turn that asked a question.

### Step 3 — Forking decision-tree walk (only if the inquiry crosses lanes)

If the user's inquiry legitimately crosses two lanes (e.g., "vendor SLA + spend audit" = VENDOR + PROCUREMENT), walk the tree **depth-first**:

1. Resolve the higher-confidence lane first → run that sub-skill in forked context → return digest
2. Ask: "Should we now run [second lane]? My recommendation: yes, because [dependency reason]."
3. Only after explicit user confirmation, run the second sub-skill

Do NOT chain silently. Each fork is an explicit user-confirmed step.

### Step 4 — Invoke sub-skill in forked context

Each sub-skill is invoked with the original prompt + a digest of any structured inputs (file paths, JSON inputs). The fork keeps heavy ingestion (vendor catalog, process transcripts, SOP source documents) out of the parent context.

### Step 5 — Return digest with cited canon challenge

When the sub-skill completes, return a **≤ 200-word digest** to the parent thread:

- What was analyzed
- Top 3 findings (each anchored in a reference doc citation — e.g., "Goldratt's Theory of Constraints: optimize the bottleneck, not the non-constraint")
- Top 3 next actions (named owners if possible)
- Path to the artifact(s) produced
- **One grill challenge** for the user, cited: "Your value-add ratio is 12%. Lean canon (Womack & Jones 1996) classifies <15% as waste-heavy. What's blocking process redesign — political, technical, or budget?"

The parent agent can then ask follow-ups (each triggering new forked invocations).

## Forcing-question library (grill-with-docs pattern)

When the user has provided enough context to enter a lane, the orchestrator may grill them on the **decisions inside that lane** before invoking the sub-skill. One question per turn, each with a recommended answer + canon citation. Examples:

- **PROCESS lane**: "Before mapping: do you have measured cycle times per stage, or only estimates? Recommended: insist on measured data for the top-3 longest stages. Anti-pattern (Goldratt 1984): map estimates, optimize the wrong constraint."
- **VENDOR lane**: "Before scoring: what's your tier-1 criticality threshold — by spend ($X/year), or by operational dependency (revenue-blocking if vendor fails)? Recommended: operational dependency. Anti-pattern (Gartner TPRM): spend-only tiering misses critical low-spend vendors like the HVAC vendor in the Target breach."
- **CAPACITY lane**: "Before modeling: are you planning for utilization or throughput? Recommended: throughput (Little's Law). Anti-pattern (DORA): planning for utilization > 80% destroys throughput via queueing."

Never run a sub-skill until the lane-defining decision is locked.

## Assumptions

1. The user is acting on behalf of an organization with ≥ 10 employees (smaller orgs don't need this surface).
2. The user has access to the data the sub-skill needs (process docs, vendor list, spend export, etc.) — or accepts the skill's templated dummy data.
3. The user wants **deterministic, repeatable analysis** over LLM-flavored prose. Every sub-skill ships stdlib-only Python tools.

## Non-goals

- Not a substitute for an ERP, vendor management platform (Vendr, Tropic), or capacity-planning SaaS (Float, Runn).
- Does not store state across sessions — every invocation is self-contained.
- Does not call external APIs from Python tools (stdlib only, by design).

## Distinct from

- **`business-growth/*`** — that's the **external sales motion** (CSM, sales engineering, RevOps). BizOps is **internal**.
- **`c-level-advisor/coo-advisor`** — that's strategic COO judgment ("should we restructure?"). BizOps is tactical ("here's the process map with bottlenecks").
- **`engineering/slo-architect`** — that's system reliability with SLO/SLI/error budgets. `process-mapper` is **business process** reliability, not system reliability.
- **`engineering/llm-wiki`** — that's a **personal** PKM (Karpathy's pattern). `knowledge-ops` is **company-wide** SOP authoring.

## Output artifacts

Every sub-skill produces at least one artifact (markdown, CSV, or JSON) saved to the user's working directory. The orchestrator surfaces the file path in the digest.

## Anti-patterns (do not)

- ❌ Run all 6 sub-skills "to be thorough" — pick one based on signal, return digest, let user chain
- ❌ Auto-approve a vendor or process change — surface findings; the human decides
- ❌ Edit production process docs without asking — write to a new file, propose the diff
- ❌ Skip the digest step — parent context needs ≤ 200-word digest, not the full sub-skill output

## References

- See `c-level-advisor/coo-advisor` for strategic COO framing
- Path-B build pattern: `documentation/implementation/bizops-commercial-expansion-plan.md`

---
name: capacity-planner
description: "Use when an ops leader (Director of CX, Head of Support, VP Ops, Head of BizOps, Head of IT ops, Head of Finance ops) is sizing ops capacity, building a headcount plan, modeling utilization risk, planning Q3 capacity or annual support capacity, or designing CS coverage — and needs Erlang-C queueing math, P90 demand sizing, shrinkage-adjusted FTE, manager-trigger thresholds, and a quarterly hiring sequence with ramp + attrition. Apply when sustained team utilization is above 80% or when the team is growing >50% in 12 months. Run before committing the headcount budget. This is NOT engineering capacity (see vpe-advisor for DORA + cycle time) and NOT strategic 3-year workforce planning (see chro-advisor)."
version: 2.8.0
author: claude-code-skills
license: MIT
tags: [bizops, capacity, headcount, utilization, queueing-theory, ops-planning, little-law, workforce]
compatible_tools: [claude-code, codex-cli, cursor, antigravity, opencode, gemini-cli]
---

# capacity-planner

Sizing tool for **ops teams that handle queued work** — Support, CX,
Customer Success, BizOps, IT ops, Finance ops. Built on Erlang-C
queueing theory, Little's Law, and the operational-leadership canon
(Fournier, Larson, Cleveland, Reinertsen). Deterministic, stdlib-only,
no LLM calls.

## Purpose

You are an ops leader sized 15 → 35 with no idea how the 35-person org
will actually behave at peak load. Or you are at 88% utilization and
SLA is starting to slip. Or you have a hiring budget approved and need
to sequence it across four quarters without burning out the existing
team. This skill answers those questions with arithmetic, not vibes.

It produces three artifacts:

1. **Capacity sizing** at 70/80/90% utilization against P50/P90/P99
   demand, with P(SLA breach) at each point and a SAFE/WATCH/AT_RISK/CRITICAL
   risk band.
2. **Utilization health** at the per-member traffic-light level plus a
   team verdict (HEALTHY/SQUEEZED/OVERLOADED/UNBALANCED).
3. **12-month quarterly hiring plan** accounting for ramp curves,
   attrition, QoQ demand growth, and span-of-control manager triggers.

## When to use

- **Annual ops capacity planning** (October-November for the following
  fiscal year).
- **Quarterly re-sizing** if demand changed >15% or attrition spiked.
- **Pre-budget defense** — the math that justifies the headcount ask
  to your CFO.
- **Diagnostic** when an ops team is missing SLA and you need to know
  whether it's a sizing problem, a process problem, or a bottleneck
  problem.
- **M&A / new-segment launch** modeling — sizing a new team or
  combined org.

## Workflow

1. **Intake demand**. Pull P50/P90/P99 daily ticket/case volume from
   your work system (Zendesk, Intercom, JSM, ServiceNow, Salesforce).
   If you only have averages, stop and pull the distribution. Single-
   point demand estimates are the most expensive anti-pattern in ops.
2. **Model throughput**. Run `capacity_modeler.py` with your demand,
   AHT, SLA target, current FTE, and shrinkage. Use `--profile` for
   your function (support / cx / bizops / finance-ops / it-ops). Read
   the 80%-utilization row — that's your sizing point.
3. **Flag utilization risk**. Run `utilization_analyzer.py` against
   your current team's actual utilization data. Anyone >85% sustained
   is a throughput-collapse risk per Reinertsen. Spread >30 percentage
   points across team means UNBALANCED — fix that before hiring.
4. **Sequence hiring**. Run `hiring_sequencer.py` with current FTE,
   target EOY, ramp time, attrition, and growth. It will front-load
   hires (Q1 35%, Q4 15%), apply ramp curves, and trigger a manager
   hire when span of control crosses 7 ICs/manager.
5. **Walk the Forcing-question library** (see below). One question at
   a time. Do not skip ahead. Answers must be written down before
   you commit the plan.

## Scripts

- `scripts/capacity_modeler.py` — Erlang-C sizing with shrinkage
  adjustment and P50/P90/P99 breach probabilities. `--profile`
  for industry defaults.
- `scripts/utilization_analyzer.py` — per-member traffic-light +
  team-level health verdict with variance detection.
- `scripts/hiring_sequencer.py` — 12-month quarterly plan with ramp,
  attrition, growth, max-hires-per-quarter constraint, and
  manager-trigger logic.

All three accept `--input <path>` (JSON), `--output {markdown,json}`,
`--sample` (built-in example), and `--help`. Stdlib only.

## References

- `references/queueing_theory_canon.md` — Erlang, Little, Hopp &
  Spearman, Reinertsen, Kingman, Cleveland, ITIL, Armony et al. (8
  sources). The math.
- `references/ops_workforce_planning_canon.md` — Fournier, Larson,
  Google SRE Workbook, Frei, Lawler, Bersin, Gartner, Grove (8
  sources). The people factors.
- `references/capacity_anti_patterns.md` — 11 named anti-patterns
  with cited sources, tool guards, and the meta-discipline that
  Lencioni + Goldratt + Christensen impose. (8+ named sources.)

## Assets

- `assets/capacity_brief_template.md` — 20-minute fill-out template
  with JSON skeletons for all three tools and an output checklist.

## Assumptions

This skill assumes:

- Work is **queued** (tickets, cases, work items) — not project-style.
  If your team's work isn't queued, this is the wrong skill.
- Demand has a **stationary-enough distribution** within a quarter.
  Step-changes (new product launch, M&A, regulatory shift) require
  re-running mid-quarter.
- You have **at least 90 days of historical demand data** to compute
  P50/P90/P99. If not, generate the distribution from your sales /
  user-base forecast first.
- Service is **single-class** within a queue. If you have hard
  priority tiers (P1/P2/P3 with class-specific SLAs), model each as
  a separate queue and sum.
- **Channels are modeled coherently.** Multi-channel teams use the
  appropriate `--profile` with built-in shrinkage premium.

## Anti-patterns

See `references/capacity_anti_patterns.md` for the full taxonomy with
sources. Top eight:

1. Plan-to-100%-utilization (Reinertsen Principle 12)
2. Treat-ramp-as-instant (Larson)
3. Ignore-attrition-in-12-month-plan (Bersin)
4. Hire-ICs-forever-with-no-manager-trigger (Fournier)
5. Size-to-P50-demand-only (Cleveland)
6. No-shrinkage-adjustment (Cleveland, SRE Workbook)
7. Single-channel-model-for-multi-channel-work (Gartner, Kingman)
8. No-surge-plan-for-P99-events (Hopp & Spearman, Reinertsen)

## Distinct from

- **`c-level-advisor/vpe-advisor`** measures *engineering* throughput
  via DORA 4 metrics, story points, deployment frequency, and cycle
  time bottlenecks. It is for engineering teams shipping code. This
  skill is for ops teams handling tickets/cases. Different unit of
  work, different math (Erlang-C vs. DORA), different bottleneck
  (queueing-blind staffing vs. WIP + lead time).
- **`c-level-advisor/chro-advisor`** does *strategic* workforce
  planning (1-5 year capability portfolios, talent supply, leadership
  succession). This skill does *operational* 0-12 month capacity
  sizing against demand. Per Lawler: conflating them gets you hired
  into the wrong jobs.
- **`project-management/*`** tracks delivery throughput on projects
  (Jira velocity, sprint capacity). This skill sizes around steady-
  state queued work.
- **Sibling `process-mapper`** *finds* the bottleneck. This skill
  *sizes the team around* a known bottleneck. Order of operations:
  process-mapper first → capacity-planner second. Hiring around the
  wrong constraint wastes the hires.
- **`business-growth/cs-coverage`** (if it exists) sizes Customer
  Success coverage by ARR/CSM ratio and segment. This skill sizes by
  queued work volume (tickets, cases, escalations). For a CS team
  that handles both relationship work AND a ticket queue, run both.

## Forcing-question library (Matt Pocock grill discipline)

**Discipline**: walk these one at a time. Do not skip ahead. Answers must
be written down. If you can't answer one, that is your next investigation.

### Q1 — "What is your bottleneck, and have you confirmed it empirically?"

**Recommended answer**: a named, measured stage in the workflow with
queue-time data showing where work waits. Not a vibe. Not "escalations
take too long". An actual measured queue.

**Why it's the first question**: Goldratt (*The Goal*, 1984) — every
system has exactly one binding constraint at a time. Sizing around the
wrong constraint wastes hires entirely. If you do not know your
bottleneck, run `process-mapper` BEFORE this skill.

**Canon**: Eli Goldratt, *The Goal* (1984); Reinertsen, *Principles of
Product Development Flow* (2009).

### Q2 — "What service trade-off are you accepting?"

**Recommended answer**: a written, explicit choice — fast vs. empathetic,
broad vs. deep, low-cost vs. high-quality. Frances Frei is unambiguous:
you cannot win all four. The team that tries wins zero.

**Why it matters**: AHT, SLA, and shrinkage inputs are the operational
expression of this trade-off. If they don't agree (e.g., you set AHT for
"empathy" but SLA for "speed"), the plan is internally inconsistent.

**Canon**: Frances Frei & Anne Morriss, *Uncommon Service* (HBR Press,
2012).

### Q3 — "What's your demand P90, and what's the gap to your P99?"

**Recommended answer**: two specific numbers from the last 90 days of
data, with the calendar context of each (e.g., "P90 was 480 tickets/day
on normal Tuesdays; P99 was 720 on the day after the November release").
A team sized to P50 misses SLA half the time. A team sized to P99
overstaffs by 30-50%. P90 is the right operating sizing point per
Cleveland.

**Canon**: Brad Cleveland, *Call Center Management on Fast Forward* (4th
ed., 2019); A.K. Erlang, *The Theory of Probabilities and Telephone
Conversations* (1909).

### Q4 — "At your planned utilization, what is P(SLA breach) at P90 and at P99?"

**Recommended answer**: two probabilities, computed (not guessed) from
Erlang-C with your specific N, AHT, and SLA target. If P(breach at P90)
> 10% you are understaffed at the sizing point. If P(breach at P99) >
50% you have no surge plan and the next peak event will be visible to
the CEO.

**Canon**: Erlang (1909); Hopp & Spearman, *Factory Physics* (3rd ed.,
2008), VUT equation.

### Q5 — "Have you budgeted replacement hires for the attrition you'll see this year?"

**Recommended answer**: yes, with a specific number. At 30% annual
attrition (Bersin BPO midpoint), a 20-FTE team loses ~6 people this year.
If your "add 5 net" plan is actually a "hire 11" plan, the recruiting
volume changes drastically. Anti-pattern #3.

**Canon**: Bersin/Deloitte talent benchmarks (2015-2023); Edward Lawler,
*Strategic Workforce Planning* (USC CEO, 2008).

### Q6 — "When does span of control trigger a manager hire, and who is the candidate?"

**Recommended answer**: a specific quarter (from `hiring_sequencer.py`)
and at least one identified candidate (internal lead or external hire).
Past 7 ICs/manager, 1:1s degrade, feedback cycles slip, attrition
climbs. Past 10 you have a coverage crisis. Hire the manager BEFORE
crossing 10, not after.

**Canon**: Camille Fournier, *The Manager's Path* (O'Reilly, 2017),
ch. 5; Andy Grove, *High Output Management* (1983).

### Q7 — "What is your surge plan for the P99 day?"

**Recommended answer**: an explicit, documented plan — overflow tier,
BPO contracted capacity, on-call rotation, executive escalation tree,
OR a written degradation contract that says "on P99 days we extend SLA
to X minutes and notify customers proactively". If the answer is "we'll
figure it out", the P99 day is a fire visible to the board.

**Canon**: Hopp & Spearman, *Factory Physics* (2008); Reinertsen (2009)
on capacity-margin discipline.

---

**Walk these seven in order. One at a time. Write the answers down. The
plan you submit is only as defensible as your answers to these seven
questions.**

FILE:assets/capacity_brief_template.md
# Capacity Planning Brief — {{TEAM_NAME}}

> 20-minute fill-out. Bring this brief plus your last 90 days of ticket /
> case / work-item data and you have everything needed to produce a
> defensible Q+1 plan.

## Section 1 — Context (5 minutes)

- **Team name:** {{TEAM_NAME}}
- **Function:** [support / cx / bizops / finance-ops / it-ops]
- **Planning horizon:** [Q+1 / annual / 12-month rolling]
- **Current headcount:** {{CURRENT_FTE}}
- **Working hours/day:** {{WORKING_HOURS_PER_DAY}}
- **Top business event driving this plan:** _(growth target, peak season,
  M&A integration, regulatory change, etc.)_

## Section 2 — Demand (5 minutes)

Pull from your ticketing system (Zendesk, Intercom, Jira Service
Management, Salesforce, ServiceNow, etc.) the daily volume for the last
90 days. Compute or read off:

- **P50 (median day):** {{P50_TICKETS_PER_DAY}}
- **P90 (peak-band day):** {{P90_TICKETS_PER_DAY}}
- **P99 (annual peak day):** {{P99_TICKETS_PER_DAY}}

> If you only have averages, this plan is built on sand. Pull the
> distribution. (Anti-pattern #5: size-to-P50-only.)

- **Average handle time (AHT, minutes):** {{AHT_MINUTES}}
- **SLA target (minutes to first response or resolution):** {{SLA_MINUTES}}
- **Channels in scope:** _(voice / email / chat / async / multi)_
- **Multi-channel premium expected:** [yes / no — if multi, add 15-25%]

## Section 3 — People Realities (5 minutes)

- **Shrinkage % (paid time NOT productive):** {{SHRINKAGE_PCT}}
  _(default if unknown: support 30, cx 32, bizops 25, finance-ops 22, it-ops 28)_
- **Ramp time for new hire (weeks to full productivity):** {{RAMP_WEEKS}}
  _(default: support 8, cx 10, bizops 12, finance-ops 14, it-ops 10)_
- **Annual attrition observed last 12 months:** {{ATTRITION_PCT}}
  _(default: support 30, cx 28, bizops 18, finance-ops 15, it-ops 20)_
- **Max hires per quarter (recruiting + onboarding constraint):**
  {{MAX_HIRES_PER_QUARTER}}
- **Current managers and span of control:** _(list manager names + their
  direct-report counts)_

## Section 4 — Strategic Constraints (5 minutes)

- **QoQ demand growth assumption:** {{GROWTH_QOQ_PCT}}
- **Bottleneck identified upstream (via process-mapper or similar):**
  _(if you don't know your bottleneck, run process-mapper FIRST — sizing
  around the wrong constraint is wasted hires)_
- **Service trade-off accepted:** _(per Frances Frei — pick which
  attributes to win: speed / empathy / breadth / cost)_
- **Surge plan for P99 events:** _(overflow tier? BPO? on-call?
  documented degradation?)_

---

## Tool Inputs

### Input JSON for `capacity_modeler.py`

```json
{
  "team_name": "{{TEAM_NAME}}",
  "demand": {
    "tickets_per_day_p50": {{P50_TICKETS_PER_DAY}},
    "tickets_per_day_p90": {{P90_TICKETS_PER_DAY}},
    "tickets_per_day_p99": {{P99_TICKETS_PER_DAY}}
  },
  "sla_target_minutes": {{SLA_MINUTES}},
  "current_fte": {{CURRENT_FTE}},
  "avg_handle_time_minutes": {{AHT_MINUTES}},
  "shrinkage_pct": {{SHRINKAGE_PCT}},
  "working_hours_per_day": {{WORKING_HOURS_PER_DAY}}
}
```

Run:

```bash
python3 scripts/capacity_modeler.py --input my_brief.json --profile support
```

### Input JSON for `utilization_analyzer.py`

```json
{
  "team_members": [
    {
      "name": "<name>",
      "role": "<role>",
      "utilization_pct": <0-100>,
      "handles_count": <int>,
      "hours_billable": <float>,
      "hours_capacity": <float>
    }
  ]
}
```

Run:

```bash
python3 scripts/utilization_analyzer.py --input team_util.json
```

### Input JSON for `hiring_sequencer.py`

```json
{
  "team_name": "{{TEAM_NAME}}",
  "current_fte": {{CURRENT_FTE}},
  "target_fte_end_of_year": {{TARGET_EOY_FTE}},
  "ramp_time_weeks": {{RAMP_WEEKS}},
  "attrition_rate_annual_pct": {{ATTRITION_PCT}},
  "growth_assumption_qoq_pct": {{GROWTH_QOQ_PCT}},
  "hiring_constraints": {
    "max_hires_per_quarter": {{MAX_HIRES_PER_QUARTER}}
  }
}
```

Run:

```bash
python3 scripts/hiring_sequencer.py --input my_brief.json --profile support
```

---

## Output Checklist

After running all three tools, you should have:

- [ ] **Erlang-C sizing**: required FTE at 70/80/90% utilization (size to 80%)
- [ ] **Headroom %**: extra demand tolerable before SLA breaks (target >20%)
- [ ] **Risk band**: SAFE / WATCH / AT_RISK / CRITICAL
- [ ] **Team health verdict**: HEALTHY / SQUEEZED / OVERLOADED / UNBALANCED
- [ ] **Quarterly hiring plan**: ICs + managers + expected attrition per quarter
- [ ] **Manager-trigger callouts**: which quarter you add a manager
- [ ] **Warnings**: any quarter where hiring constraint blocks your plan
- [ ] **Forcing-question answers**: documented decisions on bottleneck,
  service trade-offs, surge plan, P99 strategy (see SKILL.md
  *Forcing-question library*)

## Sign-off

- **Prepared by:** ______________
- **Reviewed by (finance + HR + CS leader):** ______________
- **Decision and date:** ______________
- **Re-test trigger:** _(quarterly review date or demand-level threshold
  that forces re-run)_

FILE:references/capacity_anti_patterns.md
# Capacity Planning Anti-Patterns

Every ops leader who has missed a peak season has fallen into one or
more of these patterns. The math (queueing-theory-canon.md) and the
people factors (ops-workforce-planning-canon.md) make each of these
predictably destructive. This reference enumerates the eight most
common failure modes with sources and the specific guard each tool
implements.

## The Anti-Patterns

### 1. Plan-to-100%-Utilization

**The mistake:** "We have 10 people billing 40 hours each, so we have
400 hours of capacity. Demand is 380 hours. We're fine."

**Why it fails:** Erlang-C and Hopp & Spearman's VUT equation both show
queue length grows as U/(1-U). At 95% utilization, average wait time is
~19× the service time. At 99%, it's ~99×. Variability turns a
"barely-covered" plan into nightly fires.

**Source:** Donald Reinertsen, *Principles of Product Development
Flow* (2009), Principle 12: "We need to operate at lower levels of
utilization."

**Tool guard:** `capacity_modeler.py` sizes against 70/80/90% scenarios
and flags any sizing point above 85% with a Reinertsen-cited warning.

### 2. Treat-Ramp-as-Instant

**The mistake:** "We approved 8 new hires for Q3, so we have +8 FTE
of capacity starting Q3."

**Why it fails:** A new T1 support hire is ~50% productive in weeks 1-8.
A new BizOps analyst is closer to ~30% productive in weeks 1-12 because
of tool sprawl and tribal knowledge. The "wait, they're not contributing
yet" gap is when your team burns out.

**Source:** Will Larson, *Staff Engineer* (Stripe Press, 2021); Camille
Fournier, *The Manager's Path* (O'Reilly, 2017).

**Tool guard:** `hiring_sequencer.py` applies a productivity factor
that linearly ramps 50% → 100% over `ramp_time_weeks`, and front-loads
hires (Q1 35%, Q2 30%, Q3 20%, Q4 15%) so EOY productivity catches the
adjusted target.

### 3. Ignore-Attrition

**The mistake:** "We have 15 today. We need 35 by EOY. Hire 20."

**Why it fails:** At 30% annual attrition (BPO-industry midpoint), you
will lose 4-5 of the original 15 during the year AND ~3-5 of your new
hires before they fully ramp. The real gap is 28-30 hires, not 20.

**Source:** Bersin / Deloitte talent benchmarks (2015-2023); Edward
Lawler, *Strategic Workforce Planning* (USC CEO, 2008).

**Tool guard:** `hiring_sequencer.py` requires `attrition_rate_annual_pct`
and distributes attrition quarterly via compounded probability, adding
the expected replacement hires to the gap calculation.

### 4. Hire-ICs-Forever

**The mistake:** "We don't need a manager — everyone's an
individual contributor reporting to the director."

**Why it fails:** Fournier's research (and Andy Grove's *High Output
Management* before her) is unambiguous: at 8-10+ direct reports, 1:1s
degrade, feedback cycles slip, attrition climbs, and the director
becomes the bottleneck. The cost shows up as attrition + ramp re-work,
not as a missed SLA.

**Source:** Camille Fournier, *The Manager's Path*, ch. 5; Andy
Grove, *High Output Management* (1983), ch. on managerial output.

**Tool guard:** `hiring_sequencer.py` triggers a manager hire when
projected span of control exceeds 7 ICs per manager, reallocating one
quarter's IC slot to a manager hire.

### 5. Size-to-P50-Demand-Only

**The mistake:** "Average daily volume is 320 tickets. We can handle
that."

**Why it fails:** Demand is a distribution, not a number. If P50 is
320 and P90 is 480, you will be staffed below SLA 10% of business
days. Customers don't care that you hit SLA on average; they
remember the day you didn't.

**Source:** Brad Cleveland, *Call Center Management on Fast Forward*
(4th ed., 2019); A.K. Erlang (1909) on traffic distributions.

**Tool guard:** `capacity_modeler.py` requires P50, P90, AND P99
demand inputs and sizes the recommendation to **P90** with breach
probability reported at all three percentiles.

### 6. No-Shrinkage-Adjustment

**The mistake:** "Our agents work 8 hours a day, so 8 hours of
capacity per agent."

**Why it fails:** 30% shrinkage is industry-typical. The 8 hours
actually delivers ~5.6 productive hours after breaks, training,
1:1s, sync meetings, ad-hoc interrupts, and the unspoken time spent
recovering between high-cognitive-load contacts.

**Source:** Cleveland, *Call Center Management on Fast Forward*;
Google SRE Workbook (2018) ch. 6 on toil budgets.

**Tool guard:** `capacity_modeler.py` requires `shrinkage_pct`,
applies a profile default if not provided (support 30%, BizOps 25%,
finance-ops 22%, IT-ops 28%), and outputs **loaded FTE** (post-shrinkage)
distinct from **raw FTE** (Erlang-C agents).

### 7. Single-Channel-Model-for-Multi-Channel-Work

**The mistake:** "Sum Erlang-C of voice + chat + email = total
required FTE."

**Why it fails:** Skill-switching cost. Demand-distribution mismatch
(chat is bursty; email queues overnight; voice spikes at 10-11am).
Real blended-agent productivity is 15-25% below the simple sum
because handoff context-loss is taxed per switch. Gartner research
consistently finds this premium.

**Source:** Gartner Customer Service & Support practice annual
benchmarks (2015-2023); Sir J.F.C. Kingman (1961) on G/G/1 queues
(variability amplifies wait time).

**Tool guard:** `capacity_modeler.py` `--profile` flag encodes
channel-mix realities (support, cx profiles assume blended channels
with a higher shrinkage default).

### 8. No-Surge-Plan-for-P99-Events

**The mistake:** "We're sized to P90 demand. The P99 day will be bad
but it's only 1% of days."

**Why it fails:** P99 days correlate with the highest-revenue events
(product launches, billing-cycle peaks, security incidents,
regulatory deadlines). Missing SLA on those days has
outsize commercial consequences relative to the calendar share.
You need an explicit surge plan: overflow tiering, on-call rotation,
contracted BPO overflow capacity, or a documented degradation contract.

**Source:** Hopp & Spearman, *Factory Physics* (3rd ed., 2008) on
peak-demand staffing; Reinertsen, *Principles of Product Development
Flow* on capacity-margin discipline.

**Tool guard:** `capacity_modeler.py` reports P(SLA breach) at P99 in
all three utilization scenarios, surfacing whether your 80%-utilization
sizing leaves you exposed on peak days.

## Additional Anti-Patterns Worth Naming

Beyond the eight, three more deserve mention because they appear in
nearly every quarterly planning cycle:

### 9. Use-Last-Year's-AHT

Average handle time creeps. Product complexity grows. Self-service
deflects the easy tickets, leaving harder ones in the queue. **Re-baseline
AHT every quarter**, not annually. (Source: Cleveland.)

### 10. Conflate-Operational-with-Strategic-Planning

This skill is for the *next 12 months*. If you are planning for a 3-year
automation reshape, you need chro-advisor or a strategic workforce plan,
not Erlang-C. (Source: Lawler.)

### 11. Plan-Without-Demand-Forecast-Confidence-Interval

A single point estimate of "we'll handle 4,000 tickets/month next year"
is a fiction. You need a forecast distribution. If sales forecasts a
40% YoY growth, your demand P90 grows faster than your demand P50 (more
variance). (Source: Kingman; Hopp-Spearman.)

## Christensen-Raynor on Resource Allocation

Clayton Christensen and Michael Raynor's *The Innovator's Solution*
(HBR Press, 2003) makes the meta-point: **a company's actual strategy
is what it staffs**, not what it says. If your capacity plan funds
firefighting at 80% and improvement at 20%, your strategy is
firefighting regardless of any PowerPoint deck. The capacity plan is
where strategy meets payroll. Take it seriously.

## Lencioni and Goldratt: The Two Disciplines

Pat Lencioni's *The Five Dysfunctions of a Team* (2002) and Eli
Goldratt's *The Goal* (1984) bookend the operational reality:

- **Lencioni**: trust + healthy conflict are prerequisites for
  capacity discussions to be honest. Teams that can't have direct
  conversations about whether someone is overloaded will silently
  fail the capacity plan.
- **Goldratt**: subordinate everything to the bottleneck. If your
  bottleneck is escalation engineering, don't hire more T1s — you'll
  just queue more work at the choke point.

These appear in the *Forcing-question library* of SKILL.md.

## McKinsey + MIT Sloan on Queueing-Blind Staffing

McKinsey's Customer Care practice (2018-2023 reports) and MIT Sloan's
Service Operations research repeatedly document the gap between
"intuitive" staffing (manager judgment, headcount ratios) and
queueing-theory staffing. The gap is empirically 15-35%: intuitive
plans understaff at peak and overstaff at trough. The fix is not more
intuition; it is the math in `capacity_modeler.py`.

## The Closing Discipline

For every capacity plan, ask three questions:

1. **What's the queueing math?** (Erlang-C, P90 demand, ≤80% util.)
2. **What's the people reality?** (Ramp, attrition, span of control.)
3. **What's the bottleneck?** (Capacity-planner sizes around a
   bottleneck; it does not find it. Use `process-mapper` first.)

Miss any of these and the plan is fiction.

FILE:references/ops_workforce_planning_canon.md
# Ops Workforce Planning Canon

Capacity sizing is half the answer. The other half is the human reality
of hiring, ramping, retaining, and structuring the people who staff the
queue. This reference assembles the operational-leadership canon needed
to translate an Erlang-C number into an executable 12-month plan.

## The Canon

### 1. Camille Fournier — *The Manager's Path* (O'Reilly, 2017)

Definitive guide to engineering management ladder, but the
**span-of-control** chapters apply to any ops team. Key thresholds the
`hiring_sequencer.py` enforces:

- **5-7 direct reports** is the healthy band for an ops manager.
- **8-9** is the warning zone — the manager starts dropping 1:1s,
  feedback cycles slip, and team-level decisions queue.
- **10+** = you have a coverage problem, not a leadership problem.
  Hire another manager BEFORE crossing 10.

Fournier also formalizes the **player-coach → pure manager → manager of
managers** progression that gates when a team needs a director.

### 2. Will Larson — *Staff Engineer* (Stripe Press, 2021) and *An Elegant Puzzle*

Larson's chapter on **ramp time as a real cost** is the source for the
"productive ~50% during ramp, 100% after" curve in
`hiring_sequencer.py`. Empirical observations:

- Support T1 hires: 6-10 weeks to full ramp.
- BizOps / Finance ops hires: 12-16 weeks (tool sprawl + tribal
  knowledge).
- IT ops on-call rotation: 8-12 weeks before first solo on-call.

Larson's broader point: **hiring during a fire is too late**. The
sequencer's front-loaded weight (Q1 35%, Q4 15%) is the operational
expression of this principle.

### 3. Betsy Beyer, Niall Murphy, et al. — *The Site Reliability Workbook* (O'Reilly, 2018), Chapter 6: "Eliminating Toil"

Google SRE's framework for **toil budgets** maps directly to ops
shrinkage. Key staffing principle:

- An on-call ops engineer should spend **≤50% on toil**, the rest on
  engineering work that reduces toil.
- If the toil fraction exceeds 50% for >1 quarter, you are
  understaffed relative to incident volume.

The capacity-planner sibling skills (incident-coordinator,
process-mapper) feed inputs into this; the workforce-planning
implication is that "100% of paid hours = available capacity" is
**always** wrong.

### 4. Frances Frei & Anne Morriss — *Uncommon Service* (HBR Press, 2012)

Frei's central argument: **you cannot deliver excellent service across
all attributes simultaneously**. Service-design trade-offs — speed vs.
empathy, breadth vs. depth, low cost vs. high quality — directly
constrain how you size a team. A team chasing all four wins zero of
them. Capacity-planner inputs (AHT, SLA, channel mix) implicitly encode
which trade-off is being made; surfacing that explicitly in the
*Forcing-question library* is what separates a competent ops leader
from a guesser.

### 5. Edward Lawler — *Strategic Workforce Planning* (USC Marshall Center for Effective Organizations, 2008)

Lawler's research distinguishes **operational** workforce planning
(this skill: 0-12 months, role-specific, demand-driven) from
**strategic** workforce planning (CHRO's job: 1-5 years, capability
portfolio, talent supply analysis). The hard rule:

- If you are sizing against next quarter's tickets, you need
  capacity-planner.
- If you are sizing against the company's 3-year automation strategy,
  you need a strategic workforce plan (chro-advisor).
- **Conflating them gets you hired into the wrong jobs.**

### 6. Bersin / Deloitte — *Talent Acquisition Maturity Model* and benchmarks (2015-2023)

Source for the empirically reasonable **attrition + replacement-hire
defaults** in `hiring_sequencer.py` profiles. Bersin benchmarks:

- Support frontline: 25-35% annual attrition. The 30% default reflects
  the BPO-industry midpoint.
- CX/Customer success: 22-28%. Slightly stickier than raw support due
  to relationship investment.
- BizOps/Finance ops: 15-22%. Specialist + analytical work has lower
  turnover.
- Open ops headcount fills in 45-90 days for T1, 90-180 days for
  T2/specialist.

These figures must be sanity-checked against your own HR data — they
are starting points, not commitments.

### 7. Gartner Service Delivery Research — annual reports (Customer Service & Support practice)

Gartner's annual ops benchmarks codify the **multi-channel staffing
premium**: a team that handles voice + email + chat needs ~15-25% MORE
FTE than the simple-sum Erlang-C of each channel alone, because:

- Skill switching cost (context loss between channels).
- Non-uniform demand distributions across channels.
- The classic "blended-agent illusion" — agents claim to be 100%
  flexible across channels but their effective handle times degrade.

The `--profile` flag in capacity_modeler is the place to encode this;
support and CX profiles assume multi-channel realities.

### 8. Andy Grove — *High Output Management* (1983, re-issued 1995)

Grove's framework for **leveraged output** — the manager's output is
the output of her team plus the output of every team she influences.
Applied to ops capacity:

- **A manager's "productive" contribution is not their own ticket
  handles** (which should approach zero past 6 directs); it's their
  effect on the team's throughput, accuracy, and retention.
- This is why the hiring_sequencer counts managers separately from ICs
  and triggers manager hires preemptively at span-of-control limits.

## How These Connect to the Tools

| Tool | Primary Canon |
|---|---|
| `capacity_modeler.py` | Frei (service trade-offs encoded in inputs), Gartner (multi-channel) |
| `utilization_analyzer.py` | SRE Workbook (toil budget = ceiling), Grove (manager leverage) |
| `hiring_sequencer.py` | Fournier (span of control), Larson (ramp curves), Bersin (attrition), Lawler (operational vs. strategic) |

## The Hard Truths

1. **Ramp is real and is a 6-16 week tax.** Plans that assume new
   hires are productive day one are works of fiction.
2. **You will lose 15-35% of your team this year.** Hiring plans that
   don't budget replacement hires understaff you by exactly that much.
3. **You cannot manage 10+ direct reports.** Past 7-8, you are picking
   which directs to neglect.
4. **Service trade-offs are non-negotiable.** Pick which dimensions to
   win and accept losses elsewhere — Frei's central thesis.

FILE:references/queueing_theory_canon.md
# Queueing Theory Canon for Ops Capacity Planning

Capacity planning for ops teams (Support, CX, BizOps, Finance ops, IT ops)
without queueing theory is guesswork. The fundamental insight: as
utilization approaches 100%, wait time approaches infinity non-linearly.
Plan to 90% and your SLA collapses. Plan to 70-80% and you have surge
capacity. The math is over 100 years old, and ignoring it is the most
expensive mistake an ops leader makes.

## The Canon

### 1. A.K. Erlang (1909) — *The Theory of Probabilities and Telephone Conversations*

Erlang's seminal paper on telephone-traffic load. Introduced the Erlang
unit of offered load (a = arrival_rate × service_time) and gave the
formula now called Erlang-B (loss systems) and Erlang-C (waiting systems).
**Erlang-C** is what you want for any ops queue where tickets/calls wait
rather than being dropped:

```
P(wait) = (a^N / N!) * (N / (N - a)) / [ sum_{k=0..N-1} a^k/k! + (a^N/N!)*(N/(N-a)) ]
```

Where N is the number of agents, a is the offered load. **This is the
single most important formula in ops capacity planning.** Implemented in
`scripts/capacity_modeler.py` in ~30 lines of stdlib Python.

### 2. J.D.C. Little (1961) — *A Proof for the Queuing Formula L = λW*

**Little's Law**: in steady state, the average number of items in a queue
(L) equals the average arrival rate (λ) multiplied by the average time an
item spends in the system (W). Three implications for ops leaders:

- **You cannot pick L, λ, and W independently.** If demand (λ) doubles
  and headcount (L capacity) stays flat, wait time (W) must double — and
  via Erlang-C, it actually grows much faster than that near saturation.
- **Reducing average handle time** is mathematically equivalent to
  hiring, up to the utilization ceiling.
- **WIP limits work** because they put a hard cap on L, which (given fixed
  capacity throughput) directly caps W.

### 3. Hopp & Spearman — *Factory Physics* (3rd ed., 2008)

The bible of operations science applied to manufacturing. Chapters 8-9
cover variability and queueing rigorously. Key takeaway for ops leaders:
**the VUT equation** for cycle time at a workstation,

```
CT_q ≈ V × U × T
```

where V is variability (coefficient-of-variation squared), U is utilization
factor U/(1-U), and T is mean service time. Notice U/(1-U): at U=0.80, the
multiplier is 4. At U=0.90, it's 9. At U=0.95, it's 19. **This is why
"plan to 100% utilization" is the most expensive sentence in ops.**

### 4. Donald Reinertsen — *The Principles of Product Development Flow* (2009)

The most important book on queueing in knowledge work. Principle 7
("Queue size, not capacity utilization, is the primary control variable")
and Principle 12 ("We need to operate at lower levels of utilization")
make the case rigorously: **80% utilization is the safe operating ceiling
for variable-arrival queues**. Past that, queue length and cycle time
explode super-linearly. Reinertsen's diagnostic chart of "% utilization
vs. queue length" should be hanging in every ops leader's office.

### 5. Sir J.F.C. Kingman — *On Queues in Heavy Traffic* (1961)

**Kingman's formula** for a G/G/1 queue (general arrival + general
service distribution, single server):

```
E[W_q] ≈ (ρ / (1-ρ)) × ((c_a^2 + c_s^2) / 2) × τ
```

where ρ is utilization, c_a and c_s are coefficients of variation for
arrivals and service, τ is mean service time. **Implication: variability
in either arrivals or service amplifies wait time as much as utilization
does.** This is why bursty channels (email tickets that all arrive at
9am Monday) require MORE staffing slack than steady channels.

### 6. Brad Cleveland — *Call Center Management on Fast Forward* (4th ed., 2019)

The applied operating manual for service-level queues. Conventions
codified by Cleveland and used in `scripts/capacity_modeler.py`:

- Size to **P90 demand** (not P50, not P99) — P50 leaves you breaking
  SLA half the time; P99 over-staffs by 30-50%.
- **Shrinkage** must be a line item. 30% is a reasonable default for
  support (training, breaks, sync meetings, PTO, ad-hoc interrupts).
- **Service level** is the right SLA metric, not abandon rate alone:
  "answered within T seconds" as a probability.

### 7. ITIL 4 Service Management Practices (Axelos, 2019)

ITIL's *Service Operation* practice guidance codifies the canonical
demand-and-capacity-management process for IT ops teams. Key constructs
borrowed:

- **Demand management** = forecasting + smoothing (e.g., release calendars
  scheduling fewer changes during peak ticket windows).
- **Capacity management** = three sub-processes: business capacity
  (forecast), service capacity (workload analysis), component capacity
  (resource-level).
- **Service-level management** = the SLA contract that ties Erlang-C
  inputs to commitments.

### 8. M. Armony, S. Israelit, A. Mandelbaum, et al. — *On Patient Flow in Hospitals* (Stochastic Systems, 2015)

Modern empirical research on multi-class queues with priorities and
abandonment — directly applicable to multi-tier support (T1/T2/T3 with
escalation paths). Empirically validates that **abandonment plus
priority routing** in a real call/ticket center produces wait-time
distributions very close to G/G/c with class-specific service rates.
Confirms Erlang-C is the right "first model" for capacity sizing.

## How These Connect to the Tools

| Tool | Primary Canon |
|---|---|
| `capacity_modeler.py` | Erlang (1909) Erlang-C, Cleveland (sizing convention) |
| `utilization_analyzer.py` | Reinertsen (>80% threshold), Hopp-Spearman VUT, Little's Law |
| `hiring_sequencer.py` | Cleveland (shrinkage), Kingman (variability premium), ITIL (demand management) |

## The One-Sentence Summary

If you remember nothing else: **never plan an ops team to above 80%
sustained utilization** — Reinertsen Principle 12, validated by Hopp &
Spearman's VUT equation and Erlang's 1909 telephone-traffic math. The
arithmetic is unforgiving.

FILE:scripts/capacity_modeler.py
#!/usr/bin/env python3
"""capacity_modeler.py — Ops capacity sizing via Erlang-C queueing math.

Sizes an ops team (Support / CX / BizOps / Finance ops / IT ops) against
demand and an SLA target. Implements Erlang-C in pure stdlib to compute:

  * Required FTE at 70%, 80%, and 90% utilization
  * Probability of SLA breach at each utilization level
  * Capacity headroom — extra tickets/day before SLA breaks

Industry profiles tune default shrinkage and SLA conventions.

Stdlib only. No LLM calls. Deterministic. Save the JSON sample for shape.
"""
from __future__ import annotations

import argparse
import json
import math
import sys
from dataclasses import dataclass, field
from enum import Enum
from pathlib import Path
from typing import Any


# ---------------------------------------------------------------------------
# Industry profiles
# ---------------------------------------------------------------------------
PROFILES: dict[str, dict[str, float]] = {
    # shrinkage = % of paid time NOT available for productive ticket-handling
    # (training, breaks, sync, PTO accrual, ad-hoc interrupts)
    "support": {"shrinkage_pct_default": 30.0, "sla_target_minutes_default": 60.0},
    "cx": {"shrinkage_pct_default": 32.0, "sla_target_minutes_default": 30.0},
    "bizops": {"shrinkage_pct_default": 25.0, "sla_target_minutes_default": 240.0},
    "finance-ops": {"shrinkage_pct_default": 22.0, "sla_target_minutes_default": 480.0},
    "it-ops": {"shrinkage_pct_default": 28.0, "sla_target_minutes_default": 120.0},
}


class RiskBand(str, Enum):
    SAFE = "SAFE"
    WATCH = "WATCH"
    AT_RISK = "AT_RISK"
    CRITICAL = "CRITICAL"


# ---------------------------------------------------------------------------
# Erlang-C — pure stdlib implementation
# ---------------------------------------------------------------------------
def erlang_c_probability(agents: int, traffic_intensity: float) -> float:
    """Erlang-C: probability an arriving call/ticket has to wait.

    agents (N)             : number of servers
    traffic_intensity (a)  : offered load in Erlangs (lambda * AHT)
                              dimensionless; must satisfy a < N for stability.

    Returns P(wait) in [0, 1]. Returns 1.0 if system unstable (a >= N).
    """
    if agents <= 0:
        return 1.0
    if traffic_intensity <= 0:
        return 0.0
    if traffic_intensity >= agents:
        return 1.0

    # Numerator: a^N / N! * N / (N - a)
    # Denominator: sum_{k=0}^{N-1} a^k / k!  +  numerator
    # Computed in log-space to avoid overflow on big numbers.
    a = traffic_intensity
    n = agents
    # log(a^n / n!) = n*log(a) - lgamma(n+1)
    log_a_n_over_nfact = n * math.log(a) - math.lgamma(n + 1)
    numerator_term = math.exp(log_a_n_over_nfact) * (n / (n - a))

    sum_terms = 0.0
    for k in range(n):
        log_term = k * math.log(a) - math.lgamma(k + 1)
        sum_terms += math.exp(log_term)

    denom = sum_terms + numerator_term
    if denom <= 0:
        return 1.0
    return numerator_term / denom


def service_level(agents: int, traffic_intensity: float,
                  aht_seconds: float, sla_target_seconds: float) -> float:
    """P(answered within SLA target) for M/M/c queue.

    SL = 1 - P_wait * exp(-(N - a) * T / AHT)
    """
    pw = erlang_c_probability(agents, traffic_intensity)
    if agents <= traffic_intensity:
        return 0.0
    exponent = -(agents - traffic_intensity) * (sla_target_seconds / aht_seconds)
    # guard against overflow
    if exponent < -700:
        return 1.0 - pw * 0.0
    return 1.0 - pw * math.exp(exponent)


def required_agents_for_utilization(traffic_intensity: float,
                                    target_utilization: float) -> int:
    """Minimum N such that traffic_intensity / N <= target_utilization."""
    if target_utilization <= 0 or target_utilization >= 1:
        raise ValueError("target_utilization must be in (0,1)")
    n = math.ceil(traffic_intensity / target_utilization)
    return max(n, 1)


# ---------------------------------------------------------------------------
# Data model
# ---------------------------------------------------------------------------
@dataclass
class Demand:
    tickets_per_day_p50: float
    tickets_per_day_p90: float
    tickets_per_day_p99: float


@dataclass
class CapacityInput:
    team_name: str
    demand: Demand
    sla_target_minutes: float
    current_fte: float
    avg_handle_time_minutes: float
    shrinkage_pct: float
    working_hours_per_day: float = 8.0


@dataclass
class UtilizationScenario:
    target_utilization: float
    required_fte_raw: int      # before shrinkage
    required_fte_loaded: float  # after shrinkage
    p_sla_breach_p50: float
    p_sla_breach_p90: float
    p_sla_breach_p99: float
    actual_utilization_at_demand: float


@dataclass
class CapacityResult:
    team_name: str
    inputs: CapacityInput
    scenarios: list[UtilizationScenario] = field(default_factory=list)
    headroom_extra_tickets_per_day: float = 0.0
    headroom_pct: float = 0.0
    risk_band: RiskBand = RiskBand.SAFE
    recommendation: str = ""
    notes: list[str] = field(default_factory=list)


# ---------------------------------------------------------------------------
# Modeling
# ---------------------------------------------------------------------------
def model_capacity(inp: CapacityInput) -> CapacityResult:
    aht_sec = inp.avg_handle_time_minutes * 60.0
    sla_sec = inp.sla_target_minutes * 60.0

    seconds_per_day_per_fte = inp.working_hours_per_day * 3600.0
    productive_fraction = max(0.0, 1.0 - inp.shrinkage_pct / 100.0)

    def traffic_for(volume_per_day: float) -> float:
        # Erlang offered load (a) = arrival_rate * AHT, in consistent time units.
        # Per-day arrival rate normalized to per-second:
        arrival_per_sec = volume_per_day / seconds_per_day_per_fte
        return arrival_per_sec * aht_sec

    a_p50 = traffic_for(inp.demand.tickets_per_day_p50)
    a_p90 = traffic_for(inp.demand.tickets_per_day_p90)
    a_p99 = traffic_for(inp.demand.tickets_per_day_p99)

    scenarios: list[UtilizationScenario] = []
    for util in (0.70, 0.80, 0.90):
        # Sizing is done against P90 demand by convention (Cleveland).
        n_raw = required_agents_for_utilization(a_p90, util)
        # Loaded headcount accounts for shrinkage: each "agent slot" needs
        # 1 / productive_fraction headcount to staff it.
        n_loaded = n_raw / productive_fraction if productive_fraction > 0 else float("inf")

        sl_p50 = service_level(n_raw, a_p50, aht_sec, sla_sec)
        sl_p90 = service_level(n_raw, a_p90, aht_sec, sla_sec)
        sl_p99 = service_level(n_raw, a_p99, aht_sec, sla_sec)

        scenarios.append(UtilizationScenario(
            target_utilization=util,
            required_fte_raw=n_raw,
            required_fte_loaded=round(n_loaded, 2),
            p_sla_breach_p50=round(1.0 - sl_p50, 4),
            p_sla_breach_p90=round(1.0 - sl_p90, 4),
            p_sla_breach_p99=round(1.0 - sl_p99, 4),
            actual_utilization_at_demand=round(a_p90 / n_raw, 4) if n_raw > 0 else 1.0,
        ))

    # Headroom: with current_fte (loaded), how many extra tickets/day before
    # P(SLA breach) at P90 crosses 10%?
    current_productive_fte = max(1, int(round(inp.current_fte * productive_fraction)))
    headroom_volume = inp.demand.tickets_per_day_p90
    step = max(1.0, inp.demand.tickets_per_day_p90 * 0.02)
    while headroom_volume < inp.demand.tickets_per_day_p90 * 5:
        a = traffic_for(headroom_volume)
        if a >= current_productive_fte:
            break
        sl = service_level(current_productive_fte, a, aht_sec, sla_sec)
        if (1.0 - sl) > 0.10:
            break
        headroom_volume += step
    headroom_extra = max(0.0, headroom_volume - inp.demand.tickets_per_day_p90)
    headroom_pct = (headroom_extra / inp.demand.tickets_per_day_p90 * 100.0
                    if inp.demand.tickets_per_day_p90 > 0 else 0.0)

    # Risk band — pick from 80%-utilization scenario (canonical sizing point)
    s80 = next(s for s in scenarios if s.target_utilization == 0.80)
    if inp.current_fte >= s80.required_fte_loaded and headroom_pct >= 20:
        band = RiskBand.SAFE
        rec = (f"Sized correctly at {inp.current_fte} FTE for P90 demand at 80% "
               f"utilization. Headroom is healthy ({headroom_pct:.0f}%).")
    elif inp.current_fte >= s80.required_fte_loaded:
        band = RiskBand.WATCH
        rec = (f"Headcount adequate ({inp.current_fte} FTE vs. "
               f"{s80.required_fte_loaded} required) but headroom thin "
               f"({headroom_pct:.0f}%). Re-test in 30 days.")
    elif inp.current_fte >= s80.required_fte_loaded * 0.85:
        band = RiskBand.AT_RISK
        rec = (f"Understaffed for P90 demand: have {inp.current_fte}, need "
               f"{s80.required_fte_loaded} at 80% utilization. Expect SLA "
               f"misses at P90 surges. Hire {math.ceil(s80.required_fte_loaded - inp.current_fte)} FTE.")
    else:
        band = RiskBand.CRITICAL
        rec = (f"Critically understaffed: have {inp.current_fte}, need "
               f"{s80.required_fte_loaded}. Throughput collapse risk per "
               f"queueing theory at sustained >85% utilization. Escalate.")

    notes: list[str] = []
    if s80.actual_utilization_at_demand > 0.85:
        notes.append(
            "WARNING: Sizing point pushes >85% utilization. Reinertsen's "
            "principle 7: throughput collapses non-linearly past 80%."
        )
    if inp.shrinkage_pct < 15:
        notes.append("Shrinkage <15% likely understates non-productive time.")
    if inp.shrinkage_pct > 40:
        notes.append("Shrinkage >40% — verify against actual time-on-task data.")

    return CapacityResult(
        team_name=inp.team_name,
        inputs=inp,
        scenarios=scenarios,
        headroom_extra_tickets_per_day=round(headroom_extra, 1),
        headroom_pct=round(headroom_pct, 1),
        risk_band=band,
        recommendation=rec,
        notes=notes,
    )


# ---------------------------------------------------------------------------
# Rendering
# ---------------------------------------------------------------------------
def to_markdown(result: CapacityResult) -> str:
    inp = result.inputs
    lines = [
        f"# Capacity Model — {result.team_name}",
        "",
        f"**Risk band:** {result.risk_band.value}",
        "",
        f"**Recommendation:** {result.recommendation}",
        "",
        "## Inputs",
        f"- Current FTE: {inp.current_fte}",
        f"- AHT: {inp.avg_handle_time_minutes} min",
        f"- SLA target: {inp.sla_target_minutes} min",
        f"- Shrinkage: {inp.shrinkage_pct}%",
        f"- Working hours / day: {inp.working_hours_per_day}",
        f"- Demand P50 / P90 / P99: {inp.demand.tickets_per_day_p50} / "
        f"{inp.demand.tickets_per_day_p90} / {inp.demand.tickets_per_day_p99} tickets/day",
        "",
        "## Sizing Scenarios (Erlang-C, sized to P90 demand)",
        "",
        "| Target Util | Raw FTE | Loaded FTE (post-shrinkage) | P(SLA breach @ P50) | P(SLA breach @ P90) | P(SLA breach @ P99) |",
        "|---|---|---|---|---|---|",
    ]
    for s in result.scenarios:
        lines.append(
            f"| {int(s.target_utilization*100)}% | {s.required_fte_raw} | "
            f"{s.required_fte_loaded} | {s.p_sla_breach_p50*100:.1f}% | "
            f"{s.p_sla_breach_p90*100:.1f}% | {s.p_sla_breach_p99*100:.1f}% |"
        )
    lines.extend([
        "",
        "## Headroom",
        f"- Extra tickets/day before SLA breaks: {result.headroom_extra_tickets_per_day}",
        f"- Headroom %: {result.headroom_pct:.1f}%",
        "",
    ])
    if result.notes:
        lines.append("## Notes")
        for n in result.notes:
            lines.append(f"- {n}")
        lines.append("")
    lines.append("## Canon")
    lines.append("- Erlang (1909), Little (1961), Cleveland *Call Center Mgmt on Fast Forward*, Reinertsen *Principles of Product Development Flow*.")
    return "\n".join(lines)


def to_dict(result: CapacityResult) -> dict[str, Any]:
    return {
        "team_name": result.team_name,
        "risk_band": result.risk_band.value,
        "recommendation": result.recommendation,
        "headroom_extra_tickets_per_day": result.headroom_extra_tickets_per_day,
        "headroom_pct": result.headroom_pct,
        "scenarios": [
            {
                "target_utilization": s.target_utilization,
                "required_fte_raw": s.required_fte_raw,
                "required_fte_loaded": s.required_fte_loaded,
                "p_sla_breach_p50": s.p_sla_breach_p50,
                "p_sla_breach_p90": s.p_sla_breach_p90,
                "p_sla_breach_p99": s.p_sla_breach_p99,
                "actual_utilization_at_demand": s.actual_utilization_at_demand,
            }
            for s in result.scenarios
        ],
        "notes": result.notes,
        "inputs": {
            "current_fte": result.inputs.current_fte,
            "avg_handle_time_minutes": result.inputs.avg_handle_time_minutes,
            "sla_target_minutes": result.inputs.sla_target_minutes,
            "shrinkage_pct": result.inputs.shrinkage_pct,
            "working_hours_per_day": result.inputs.working_hours_per_day,
            "demand": {
                "tickets_per_day_p50": result.inputs.demand.tickets_per_day_p50,
                "tickets_per_day_p90": result.inputs.demand.tickets_per_day_p90,
                "tickets_per_day_p99": result.inputs.demand.tickets_per_day_p99,
            },
        },
    }


# ---------------------------------------------------------------------------
# Sample + parsing
# ---------------------------------------------------------------------------
SAMPLE_INPUT: dict[str, Any] = {
    "team_name": "Tier-1 Support",
    "demand": {
        "tickets_per_day_p50": 320,
        "tickets_per_day_p90": 480,
        "tickets_per_day_p99": 720,
    },
    "sla_target_minutes": 60,
    "current_fte": 12,
    "avg_handle_time_minutes": 18,
    "shrinkage_pct": 30,
    "working_hours_per_day": 8,
}


def parse_input(raw: dict[str, Any], profile: str | None) -> CapacityInput:
    prof = PROFILES.get(profile or "", {})
    shrinkage = raw.get("shrinkage_pct", prof.get("shrinkage_pct_default", 30.0))
    sla = raw.get("sla_target_minutes",
                  prof.get("sla_target_minutes_default", 60.0))
    d = raw["demand"]
    return CapacityInput(
        team_name=raw["team_name"],
        demand=Demand(
            tickets_per_day_p50=float(d["tickets_per_day_p50"]),
            tickets_per_day_p90=float(d["tickets_per_day_p90"]),
            tickets_per_day_p99=float(d["tickets_per_day_p99"]),
        ),
        sla_target_minutes=float(sla),
        current_fte=float(raw["current_fte"]),
        avg_handle_time_minutes=float(raw["avg_handle_time_minutes"]),
        shrinkage_pct=float(shrinkage),
        working_hours_per_day=float(raw.get("working_hours_per_day", 8.0)),
    )


def main(argv: list[str] | None = None) -> int:
    p = argparse.ArgumentParser(
        description="Erlang-C ops capacity sizer (stdlib only).",
    )
    p.add_argument("--input", type=Path, help="Path to JSON input file.")
    p.add_argument(
        "--profile",
        choices=list(PROFILES.keys()),
        default=None,
        help="Industry profile (defaults for shrinkage + SLA).",
    )
    p.add_argument(
        "--output",
        choices=["markdown", "json"],
        default="markdown",
        help="Output format.",
    )
    p.add_argument(
        "--sample",
        action="store_true",
        help="Run on built-in sample input and print result.",
    )
    args = p.parse_args(argv)

    if args.sample:
        raw = SAMPLE_INPUT
    elif args.input:
        raw = json.loads(args.input.read_text())
    else:
        p.error("Provide --input or --sample.")
        return 2

    try:
        inp = parse_input(raw, args.profile)
    except (KeyError, ValueError) as e:
        print(f"ERROR parsing input: {e}", file=sys.stderr)
        return 2

    result = model_capacity(inp)

    if args.output == "json":
        print(json.dumps(to_dict(result), indent=2))
    else:
        print(to_markdown(result))
    return 0


if __name__ == "__main__":
    sys.exit(main())

FILE:scripts/hiring_sequencer.py
#!/usr/bin/env python3
"""hiring_sequencer.py — 12-month quarterly hiring plan for ops teams.

Accounts for:
  * Ramp time (productive ~50% for ramp_time_weeks, then 100%)
  * Annual attrition (compounded weekly across the year)
  * Quarter-over-quarter demand growth
  * Hiring constraints (max hires / quarter)
  * Manager-trigger: when span of control crosses 7-8 ICs, schedule manager hire

Stdlib only.
"""
from __future__ import annotations

import argparse
import json
import math
import sys
from dataclasses import dataclass, field
from enum import Enum
from pathlib import Path
from typing import Any


# Industry profiles — typical ramp + attrition
PROFILES: dict[str, dict[str, float]] = {
    "support":    {"ramp_time_weeks": 8.0,  "attrition_rate_annual_pct": 30.0},
    "cx":         {"ramp_time_weeks": 10.0, "attrition_rate_annual_pct": 28.0},
    "bizops":     {"ramp_time_weeks": 12.0, "attrition_rate_annual_pct": 18.0},
    "finance-ops":{"ramp_time_weeks": 14.0, "attrition_rate_annual_pct": 15.0},
    "it-ops":     {"ramp_time_weeks": 10.0, "attrition_rate_annual_pct": 20.0},
}

SPAN_OF_CONTROL_MAX = 7  # ICs per manager threshold (Fournier)


class Quarter(str, Enum):
    Q1 = "Q1"
    Q2 = "Q2"
    Q3 = "Q3"
    Q4 = "Q4"


@dataclass
class HiringInput:
    team_name: str
    current_fte: int
    target_fte_end_of_year: int
    ramp_time_weeks: float
    attrition_rate_annual_pct: float
    growth_assumption_qoq_pct: float
    max_hires_per_quarter: int


@dataclass
class QuarterPlan:
    quarter: Quarter
    ic_hires: int
    manager_hires: int
    expected_attrition: int
    productive_fte_end_of_quarter: float
    headcount_end_of_quarter: int
    span_of_control: float
    notes: list[str] = field(default_factory=list)


@dataclass
class HiringResult:
    team_name: str
    inputs: HiringInput
    quarters: list[QuarterPlan]
    total_ic_hires: int
    total_manager_hires: int
    total_attrition: int
    headline: str
    warnings: list[str] = field(default_factory=list)


def _productivity_factor(weeks_since_hire: float, ramp_weeks: float) -> float:
    """Linear ramp 50% → 100% over ramp_weeks (Larson)."""
    if weeks_since_hire >= ramp_weeks:
        return 1.0
    if weeks_since_hire <= 0:
        return 0.5
    return 0.5 + 0.5 * (weeks_since_hire / ramp_weeks)


def sequence(inp: HiringInput) -> HiringResult:
    # Demand-side ratchet: target adjusted up by QoQ growth (compounded)
    growth_factor_eoy = (1.0 + inp.growth_assumption_qoq_pct / 100.0) ** 4
    adjusted_target = int(math.ceil(inp.target_fte_end_of_year * growth_factor_eoy))

    # Per-quarter attrition probability — split annual rate across 4 quarters
    q_attrition_rate = 1.0 - (1.0 - inp.attrition_rate_annual_pct / 100.0) ** 0.25

    # Total gap to close: target + replacement hires over the year
    expected_total_attrition = int(math.ceil(
        inp.current_fte * (inp.attrition_rate_annual_pct / 100.0)
    ))
    raw_gap = adjusted_target - inp.current_fte + expected_total_attrition
    total_hires_needed = max(0, raw_gap)

    # Distribute hires front-loaded but capped
    quarters: list[QuarterPlan] = []
    headcount = inp.current_fte
    remaining = total_hires_needed
    cumulative_managers = max(1, math.ceil(inp.current_fte / SPAN_OF_CONTROL_MAX))
    cumulative_ic_hires = 0
    cumulative_manager_hires = 0
    cumulative_attrition = 0
    warnings: list[str] = []

    # Pre-emptive front-load: aim higher in early quarters so ramp completes by EOY
    # Allocation weights Q1>Q2>Q3>Q4 since later hires miss ramp window.
    weights = [0.35, 0.30, 0.20, 0.15]

    for i, qname in enumerate(Quarter):
        ideal_q_hires = math.ceil(total_hires_needed * weights[i])
        q_hires = min(ideal_q_hires, inp.max_hires_per_quarter, remaining)
        if q_hires < ideal_q_hires:
            warnings.append(
                f"{qname.value}: wanted {ideal_q_hires} hires but constrained "
                f"to {q_hires} by max_hires_per_quarter."
            )

        # Attrition realized this quarter
        q_attrition = int(round(headcount * q_attrition_rate))
        cumulative_attrition += q_attrition

        # New headcount after hires + attrition
        new_headcount = headcount + q_hires - q_attrition

        # Manager trigger check — if ICs / managers > threshold, add a manager hire
        # (counted within the ic_hires bucket reallocated as manager)
        ic_count_eoq = new_headcount - cumulative_managers
        span = ic_count_eoq / max(cumulative_managers, 1)
        notes: list[str] = []
        manager_hires_this_q = 0
        if span > SPAN_OF_CONTROL_MAX and q_hires > 0:
            manager_hires_this_q = 1
            q_hires -= 1
            cumulative_managers += 1
            notes.append(
                f"Manager trigger fired: span was {span:.1f} ICs/manager > "
                f"{SPAN_OF_CONTROL_MAX}. Reallocated 1 IC hire to manager hire."
            )
            # Recompute span after manager hire
            ic_count_eoq = new_headcount - cumulative_managers
            span = ic_count_eoq / max(cumulative_managers, 1)

        cumulative_ic_hires += q_hires
        cumulative_manager_hires += manager_hires_this_q
        remaining -= (q_hires + manager_hires_this_q)

        # Productive FTE = full-time members + ramp-fraction for in-quarter hires
        # in-quarter hires are halfway through ramp on average at EOQ → ~halfway up the ramp curve
        avg_weeks_for_q_hires = 6.5  # quarter midpoint (13 weeks / 2)
        ramp_fraction = _productivity_factor(avg_weeks_for_q_hires, inp.ramp_time_weeks)
        productive_fte = (headcount - q_attrition) + (q_hires + manager_hires_this_q) * ramp_fraction

        if productive_fte < adjusted_target * 0.85 and i == 3:
            warnings.append(
                f"EOY productive FTE ({productive_fte:.1f}) below 85% of adjusted "
                f"target ({adjusted_target}) — ramp will extend into next year."
            )

        quarters.append(QuarterPlan(
            quarter=qname,
            ic_hires=q_hires,
            manager_hires=manager_hires_this_q,
            expected_attrition=q_attrition,
            productive_fte_end_of_quarter=round(productive_fte, 1),
            headcount_end_of_quarter=new_headcount,
            span_of_control=round(span, 2),
            notes=notes,
        ))

        headcount = new_headcount

    headline = (
        f"Hire {cumulative_ic_hires} ICs + {cumulative_manager_hires} managers "
        f"across 4 quarters. End-of-year nominal headcount: {headcount} "
        f"(adjusted target: {adjusted_target}). Expect ~{cumulative_attrition} "
        f"attrition over the year."
    )

    return HiringResult(
        team_name=inp.team_name,
        inputs=inp,
        quarters=quarters,
        total_ic_hires=cumulative_ic_hires,
        total_manager_hires=cumulative_manager_hires,
        total_attrition=cumulative_attrition,
        headline=headline,
        warnings=warnings,
    )


def to_markdown(r: HiringResult) -> str:
    lines = [
        f"# Hiring Plan — {r.team_name}",
        "",
        f"**Headline:** {r.headline}",
        "",
        "## Assumptions",
        f"- Current FTE: {r.inputs.current_fte}",
        f"- Target EOY FTE (nominal): {r.inputs.target_fte_end_of_year}",
        f"- Ramp time: {r.inputs.ramp_time_weeks} weeks",
        f"- Annual attrition: {r.inputs.attrition_rate_annual_pct}%",
        f"- QoQ growth: {r.inputs.growth_assumption_qoq_pct}%",
        f"- Max hires per quarter: {r.inputs.max_hires_per_quarter}",
        "",
        "## Quarterly Plan",
        "",
        "| Quarter | IC Hires | Manager Hires | Attrition | Headcount EOQ | Productive FTE EOQ | Span of Control |",
        "|---|---|---|---|---|---|---|",
    ]
    for q in r.quarters:
        lines.append(
            f"| {q.quarter.value} | {q.ic_hires} | {q.manager_hires} | "
            f"{q.expected_attrition} | {q.headcount_end_of_quarter} | "
            f"{q.productive_fte_end_of_quarter} | {q.span_of_control} |"
        )
    lines.append("")
    notes_present = any(q.notes for q in r.quarters)
    if notes_present:
        lines.append("## Quarter Notes")
        for q in r.quarters:
            for n in q.notes:
                lines.append(f"- {q.quarter.value}: {n}")
        lines.append("")
    if r.warnings:
        lines.append("## Warnings")
        for w in r.warnings:
            lines.append(f"- {w}")
        lines.append("")
    lines.extend([
        "## Canon",
        "- Camille Fournier, *The Manager's Path* — span of control thresholds.",
        "- Will Larson, *Staff Engineer* — ramp productivity curves.",
        "- Bersin/Deloitte talent benchmarks — attrition + replacement hire ratios.",
    ])
    return "\n".join(lines)


def to_dict(r: HiringResult) -> dict[str, Any]:
    return {
        "team_name": r.team_name,
        "headline": r.headline,
        "totals": {
            "ic_hires": r.total_ic_hires,
            "manager_hires": r.total_manager_hires,
            "attrition": r.total_attrition,
        },
        "quarters": [
            {
                "quarter": q.quarter.value,
                "ic_hires": q.ic_hires,
                "manager_hires": q.manager_hires,
                "expected_attrition": q.expected_attrition,
                "productive_fte_end_of_quarter": q.productive_fte_end_of_quarter,
                "headcount_end_of_quarter": q.headcount_end_of_quarter,
                "span_of_control": q.span_of_control,
                "notes": q.notes,
            }
            for q in r.quarters
        ],
        "warnings": r.warnings,
        "inputs": {
            "current_fte": r.inputs.current_fte,
            "target_fte_end_of_year": r.inputs.target_fte_end_of_year,
            "ramp_time_weeks": r.inputs.ramp_time_weeks,
            "attrition_rate_annual_pct": r.inputs.attrition_rate_annual_pct,
            "growth_assumption_qoq_pct": r.inputs.growth_assumption_qoq_pct,
            "max_hires_per_quarter": r.inputs.max_hires_per_quarter,
        },
    }


SAMPLE_INPUT: dict[str, Any] = {
    "team_name": "Tier-1 Support",
    "current_fte": 15,
    "target_fte_end_of_year": 35,
    "ramp_time_weeks": 8,
    "attrition_rate_annual_pct": 30,
    "growth_assumption_qoq_pct": 8,
    "hiring_constraints": {"max_hires_per_quarter": 8},
}


def parse_input(raw: dict[str, Any], profile: str | None) -> HiringInput:
    prof = PROFILES.get(profile or "", {})
    ramp = raw.get("ramp_time_weeks", prof.get("ramp_time_weeks", 10.0))
    attr = raw.get(
        "attrition_rate_annual_pct",
        prof.get("attrition_rate_annual_pct", 25.0),
    )
    constraints = raw.get("hiring_constraints", {}) or {}
    return HiringInput(
        team_name=raw["team_name"],
        current_fte=int(raw["current_fte"]),
        target_fte_end_of_year=int(raw["target_fte_end_of_year"]),
        ramp_time_weeks=float(ramp),
        attrition_rate_annual_pct=float(attr),
        growth_assumption_qoq_pct=float(raw.get("growth_assumption_qoq_pct", 0.0)),
        max_hires_per_quarter=int(constraints.get("max_hires_per_quarter", 999)),
    )


def main(argv: list[str] | None = None) -> int:
    p = argparse.ArgumentParser(
        description="12-month quarterly hiring sequencer with ramp + attrition.",
    )
    p.add_argument("--input", type=Path, help="Path to JSON input file.")
    p.add_argument(
        "--profile",
        choices=list(PROFILES.keys()),
        default=None,
        help="Industry profile (defaults for ramp + attrition).",
    )
    p.add_argument(
        "--output", choices=["markdown", "json"], default="markdown",
        help="Output format.",
    )
    p.add_argument("--sample", action="store_true",
                   help="Run on built-in sample input.")
    args = p.parse_args(argv)

    if args.sample:
        raw = SAMPLE_INPUT
    elif args.input:
        raw = json.loads(args.input.read_text())
    else:
        p.error("Provide --input or --sample.")
        return 2

    try:
        inp = parse_input(raw, args.profile)
    except (KeyError, ValueError) as e:
        print(f"ERROR parsing input: {e}", file=sys.stderr)
        return 2

    result = sequence(inp)
    if args.output == "json":
        print(json.dumps(to_dict(result), indent=2))
    else:
        print(to_markdown(result))
    return 0


if __name__ == "__main__":
    sys.exit(main())

FILE:scripts/utilization_analyzer.py
#!/usr/bin/env python3
"""utilization_analyzer.py — per-member + team-level utilization health.

Detects:
  * RED  : sustained >85% utilization (throughput collapse risk, Reinertsen)
  * AMBER: 70-85% (acceptable but watch — Little's Law tightening)
  * GREEN: 40-70% (healthy)
  * BLUE : <40% (under-loaded or wrong skills)

Team verdict:
  HEALTHY    — most green, no reds, low variance
  SQUEEZED   — majority amber, some red
  OVERLOADED — >30% of team red
  UNBALANCED — utilization variance >30 percentage points across team

Stdlib only.
"""
from __future__ import annotations

import argparse
import json
import statistics
import sys
from dataclasses import dataclass, field
from enum import Enum
from pathlib import Path
from typing import Any


class Light(str, Enum):
    GREEN = "GREEN"
    AMBER = "AMBER"
    RED = "RED"
    BLUE = "BLUE"


class TeamVerdict(str, Enum):
    HEALTHY = "HEALTHY"
    SQUEEZED = "SQUEEZED"
    OVERLOADED = "OVERLOADED"
    UNBALANCED = "UNBALANCED"


@dataclass
class Member:
    name: str
    role: str
    utilization_pct: float
    handles_count: int
    hours_billable: float
    hours_capacity: float


@dataclass
class MemberAssessment:
    name: str
    role: str
    utilization_pct: float
    light: Light
    notes: list[str] = field(default_factory=list)


@dataclass
class TeamReport:
    verdict: TeamVerdict
    member_assessments: list[MemberAssessment]
    mean_util: float
    median_util: float
    stdev_util: float
    spread_pct_points: float
    counts: dict[str, int]
    headline: str
    recommendations: list[str]


def classify(member: Member) -> MemberAssessment:
    u = member.utilization_pct
    notes: list[str] = []
    if u >= 85:
        light = Light.RED
        notes.append("Throughput collapse risk per queueing theory (>85% sustained).")
    elif u >= 70:
        light = Light.AMBER
        notes.append("Within tolerable band but no surge capacity.")
    elif u >= 40:
        light = Light.GREEN
    else:
        light = Light.BLUE
        notes.append("Under-loaded — verify scope or reassign work.")

    # Cross-check billable vs capacity hours — if claimed utilization
    # disagrees with hours math by >10 points, flag.
    if member.hours_capacity > 0:
        computed = member.hours_billable / member.hours_capacity * 100
        if abs(computed - u) > 10:
            notes.append(
                f"Reported util ({u:.0f}%) disagrees with hours math "
                f"({computed:.0f}%). Reconcile time tracking."
            )

    return MemberAssessment(
        name=member.name,
        role=member.role,
        utilization_pct=u,
        light=light,
        notes=notes,
    )


def assess_team(members: list[Member]) -> TeamReport:
    if not members:
        raise ValueError("No team members in input.")

    assessments = [classify(m) for m in members]
    utils = [m.utilization_pct for m in members]
    mean_u = statistics.fmean(utils)
    median_u = statistics.median(utils)
    stdev_u = statistics.pstdev(utils) if len(utils) > 1 else 0.0
    spread = max(utils) - min(utils)

    counts = {
        "RED": sum(1 for a in assessments if a.light == Light.RED),
        "AMBER": sum(1 for a in assessments if a.light == Light.AMBER),
        "GREEN": sum(1 for a in assessments if a.light == Light.GREEN),
        "BLUE": sum(1 for a in assessments if a.light == Light.BLUE),
    }
    n = len(members)

    # Verdict logic — order matters
    if spread > 30:
        verdict = TeamVerdict.UNBALANCED
        headline = (f"Load spread of {spread:.0f} percentage points across team — "
                    f"some are red while others are blue.")
    elif counts["RED"] / n > 0.30:
        verdict = TeamVerdict.OVERLOADED
        headline = (f"{counts['RED']} of {n} members in RED zone. Throughput "
                    f"collapse risk.")
    elif counts["AMBER"] / n >= 0.50 or counts["RED"] >= 1:
        verdict = TeamVerdict.SQUEEZED
        headline = f"Team running hot — {counts['AMBER']} amber, {counts['RED']} red."
    else:
        verdict = TeamVerdict.HEALTHY
        headline = f"Team utilization healthy: mean {mean_u:.0f}%, spread {spread:.0f}pp."

    recs: list[str] = []
    if verdict == TeamVerdict.UNBALANCED:
        recs.append("Rebalance load — investigate whether reds need different skills, "
                    "specialization, or just more hands at their queue.")
    if verdict == TeamVerdict.OVERLOADED:
        recs.append("Stop adding scope. Hire or shed work BEFORE attempting "
                    "process improvements (Goldratt: subordinate to the constraint).")
    if verdict == TeamVerdict.SQUEEZED:
        recs.append("Plan to hire next quarter. Re-test in 30 days; squeeze tends "
                    "to become overload during seasonal peaks.")
    if counts["BLUE"] > 0:
        recs.append(f"{counts['BLUE']} member(s) under-loaded — check whether "
                    f"work is reaching them or whether scope/skill needs adjustment.")
    if not recs:
        recs.append("Maintain current sizing; revisit at next quarterly planning cycle.")

    return TeamReport(
        verdict=verdict,
        member_assessments=assessments,
        mean_util=round(mean_u, 1),
        median_util=round(median_u, 1),
        stdev_util=round(stdev_u, 1),
        spread_pct_points=round(spread, 1),
        counts=counts,
        headline=headline,
        recommendations=recs,
    )


def to_markdown(r: TeamReport) -> str:
    lines = [
        "# Utilization Analysis",
        "",
        f"**Verdict:** {r.verdict.value}",
        "",
        f"**Headline:** {r.headline}",
        "",
        "## Team Stats",
        f"- Mean utilization: {r.mean_util}%",
        f"- Median utilization: {r.median_util}%",
        f"- Stdev: {r.stdev_util}pp",
        f"- Spread (max - min): {r.spread_pct_points}pp",
        f"- Counts: RED {r.counts['RED']} / AMBER {r.counts['AMBER']} / "
        f"GREEN {r.counts['GREEN']} / BLUE {r.counts['BLUE']}",
        "",
        "## Member Detail",
        "",
        "| Name | Role | Utilization | Light | Notes |",
        "|---|---|---|---|---|",
    ]
    for a in r.member_assessments:
        notes_str = "; ".join(a.notes) if a.notes else "—"
        lines.append(
            f"| {a.name} | {a.role} | {a.utilization_pct:.0f}% | {a.light.value} | {notes_str} |"
        )
    lines.extend(["", "## Recommendations"])
    for rec in r.recommendations:
        lines.append(f"- {rec}")
    lines.extend([
        "",
        "## Canon",
        "- Reinertsen, *Principles of Product Development Flow*, principle 7.",
        "- Little (1961), *A Proof for the Queuing Formula L = λW*.",
        "- Goldratt, *The Goal* — bottleneck subordination.",
    ])
    return "\n".join(lines)


def to_dict(r: TeamReport) -> dict[str, Any]:
    return {
        "verdict": r.verdict.value,
        "headline": r.headline,
        "stats": {
            "mean_util": r.mean_util,
            "median_util": r.median_util,
            "stdev_util": r.stdev_util,
            "spread_pct_points": r.spread_pct_points,
            "counts": r.counts,
        },
        "members": [
            {
                "name": a.name,
                "role": a.role,
                "utilization_pct": a.utilization_pct,
                "light": a.light.value,
                "notes": a.notes,
            }
            for a in r.member_assessments
        ],
        "recommendations": r.recommendations,
    }


SAMPLE_INPUT: dict[str, Any] = {
    "team_members": [
        {"name": "Alice", "role": "T1 Support", "utilization_pct": 92,
         "handles_count": 48, "hours_billable": 7.4, "hours_capacity": 8},
        {"name": "Bob",   "role": "T1 Support", "utilization_pct": 88,
         "handles_count": 42, "hours_billable": 7.0, "hours_capacity": 8},
        {"name": "Carol", "role": "T1 Support", "utilization_pct": 72,
         "handles_count": 36, "hours_billable": 5.8, "hours_capacity": 8},
        {"name": "Dan",   "role": "T2 Support", "utilization_pct": 65,
         "handles_count": 18, "hours_billable": 5.2, "hours_capacity": 8},
        {"name": "Eve",   "role": "T2 Support", "utilization_pct": 35,
         "handles_count": 8,  "hours_billable": 2.8, "hours_capacity": 8},
    ]
}


def parse_members(raw: dict[str, Any]) -> list[Member]:
    out: list[Member] = []
    for m in raw["team_members"]:
        out.append(Member(
            name=m["name"],
            role=m.get("role", "unspecified"),
            utilization_pct=float(m["utilization_pct"]),
            handles_count=int(m.get("handles_count", 0)),
            hours_billable=float(m.get("hours_billable", 0)),
            hours_capacity=float(m.get("hours_capacity", 0)),
        ))
    return out


def main(argv: list[str] | None = None) -> int:
    p = argparse.ArgumentParser(
        description="Per-member + team-level utilization traffic-light analyzer.",
    )
    p.add_argument("--input", type=Path, help="Path to JSON input file.")
    p.add_argument(
        "--output", choices=["markdown", "json"], default="markdown",
        help="Output format.",
    )
    p.add_argument("--sample", action="store_true",
                   help="Run on built-in sample input.")
    args = p.parse_args(argv)

    if args.sample:
        raw = SAMPLE_INPUT
    elif args.input:
        raw = json.loads(args.input.read_text())
    else:
        p.error("Provide --input or --sample.")
        return 2

    try:
        members = parse_members(raw)
    except (KeyError, ValueError) as e:
        print(f"ERROR parsing input: {e}", file=sys.stderr)
        return 2

    report = assess_team(members)
    if args.output == "json":
        print(json.dumps(to_dict(report), indent=2))
    else:
        print(to_markdown(report))
    return 0


if __name__ == "__main__":
    sys.exit(main())

---
name: "tdd-guide"
description: "Test-driven development skill for writing unit tests, generating test fixtures and mocks, analyzing coverage gaps, and guiding red-green-refactor workflows across Jest, Pytest, JUnit, Vitest, and Mocha. Use when the user asks to write tests, improve test coverage, practice TDD, generate mocks or stubs, or mentions testing frameworks like Jest, pytest, or JUnit."
---

# TDD Guide

Test-driven development skill for generating tests, analyzing coverage, and guiding red-green-refactor workflows across Jest, Pytest, JUnit, and Vitest.

---

## Workflows

### Generate Tests from Code

1. Provide source code (TypeScript, JavaScript, Python, Java)
2. Specify target framework (Jest, Pytest, JUnit, Vitest)
3. Run `test_generator.py` with requirements
4. Review generated test stubs
5. **Validation:** Tests compile and cover happy path, error cases, edge cases

### Analyze Coverage Gaps

1. Generate coverage report from test runner (`npm test -- --coverage`)
2. Run `coverage_analyzer.py` on LCOV/JSON/XML report
3. Review prioritized gaps (P0/P1/P2)
4. Generate missing tests for uncovered paths
5. **Validation:** Coverage meets target threshold (typically 80%+)

### TDD New Feature

1. Write failing test first (RED)
2. Run `tdd_workflow.py --phase red` to validate
3. Implement minimal code to pass (GREEN)
4. Run `tdd_workflow.py --phase green` to validate
5. Refactor while keeping tests green (REFACTOR)
6. **Validation:** All tests pass after each cycle

---

## Examples

### Test Generation — Input → Output (Pytest)

**Input source function (`math_utils.py`):**
```python
def divide(a: float, b: float) -> float:
    if b == 0:
        raise ValueError("Cannot divide by zero")
    return a / b
```

**Command:**
```bash
python scripts/test_generator.py --input math_utils.py --framework pytest
```

**Generated test output (`test_math_utils.py`):**
```python
import pytest
from math_utils import divide

class TestDivide:
    def test_divide_positive_numbers(self):
        assert divide(10, 2) == 5.0

    def test_divide_negative_numerator(self):
        assert divide(-10, 2) == -5.0

    def test_divide_float_result(self):
        assert divide(1, 3) == pytest.approx(0.333, rel=1e-3)

    def test_divide_by_zero_raises_value_error(self):
        with pytest.raises(ValueError, match="Cannot divide by zero"):
            divide(10, 0)

    def test_divide_zero_numerator(self):
        assert divide(0, 5) == 0.0
```

---

### Coverage Analysis — Sample P0/P1/P2 Output

**Command:**
```bash
python scripts/coverage_analyzer.py --report lcov.info --threshold 80
```

**Sample output:**
```
Coverage Report — Overall: 63% (threshold: 80%)

P0 — Critical gaps (uncovered error paths):
  auth/login.py:42-58   handle_expired_token()       0% covered
  payments/process.py:91-110  handle_payment_failure()   0% covered

P1 — High-value gaps (core logic branches):
  users/service.py:77   update_profile() — else branch  0% covered
  orders/cart.py:134    apply_discount() — zero-qty guard  0% covered

P2 — Low-risk gaps (utility / helper functions):
  utils/formatting.py:12  format_currency()            0% covered

Recommended: Generate tests for P0 items first to reach 80% threshold.
```

---

## Key Tools

| Tool | Purpose | Usage |
|------|---------|-------|
| `test_generator.py` | Generate test cases from code/requirements | `python scripts/test_generator.py --input source.py --framework pytest` |
| `coverage_analyzer.py` | Parse and analyze coverage reports | `python scripts/coverage_analyzer.py --report lcov.info --threshold 80` |
| `tdd_workflow.py` | Guide red-green-refactor cycles | `python scripts/tdd_workflow.py --phase red --test test_auth.py` |
| `fixture_generator.py` | Generate test data and mocks | `python scripts/fixture_generator.py --entity User --count 5` |

Additional scripts: `framework_adapter.py` (convert between frameworks), `metrics_calculator.py` (quality metrics), `format_detector.py` (detect language/framework), `output_formatter.py` (CLI/desktop/CI output).

---

## Input Requirements

**For Test Generation:**
- Source code (file path or pasted content)
- Target framework (Jest, Pytest, JUnit, Vitest)
- Coverage scope (unit, integration, edge cases)

**For Coverage Analysis:**
- Coverage report file (LCOV, JSON, or XML format)
- Optional: Source code for context
- Optional: Target threshold percentage

**For TDD Workflow:**
- Feature requirements or user story
- Current phase (RED, GREEN, REFACTOR)
- Test code and implementation status

---

## Spec-First Workflow

TDD is most effective when driven by a written spec. The flow:

1. **Write or receive a spec** — stored in `specs/<feature>.md`
2. **Extract acceptance criteria** — each criterion becomes one or more test cases
3. **Write failing tests (RED)** — one test per acceptance criterion
4. **Implement minimal code (GREEN)** — satisfy each test in order
5. **Refactor** — clean up while all tests stay green

### Spec Directory Convention

```
project/
├── specs/
│   ├── user-auth.md          # Feature spec with acceptance criteria
│   ├── payment-processing.md
│   └── notification-system.md
├── tests/
│   ├── test_user_auth.py     # Tests derived from specs/user-auth.md
│   ├── test_payments.py
│   └── test_notifications.py
└── src/
```

### Extracting Tests from Specs

Each acceptance criterion in a spec maps to at least one test:

| Spec Criterion | Test Case |
|---------------|-----------|
| "User can log in with valid credentials" | `test_login_valid_credentials_returns_token` |
| "Invalid password returns 401" | `test_login_invalid_password_returns_401` |
| "Account locks after 5 failed attempts" | `test_login_locks_after_five_failures` |

**Tip:** Number your acceptance criteria in the spec. Reference the number in the test docstring for traceability (`# AC-3: Account locks after 5 failed attempts`).

> **Cross-reference:** See `engineering/spec-driven-workflow` for the full spec methodology, including spec templates and review checklists.

---

## Red-Green-Refactor Examples Per Language

### TypeScript / Jest

```typescript
// test/cart.test.ts
describe("Cart", () => {
  describe("addItem", () => {
    it("should add a new item to an empty cart", () => {
      const cart = new Cart();
      cart.addItem({ id: "sku-1", name: "Widget", price: 9.99, qty: 1 });

      expect(cart.items).toHaveLength(1);
      expect(cart.items[0].id).toBe("sku-1");
    });

    it("should increment quantity when adding an existing item", () => {
      const cart = new Cart();
      cart.addItem({ id: "sku-1", name: "Widget", price: 9.99, qty: 1 });
      cart.addItem({ id: "sku-1", name: "Widget", price: 9.99, qty: 2 });

      expect(cart.items).toHaveLength(1);
      expect(cart.items[0].qty).toBe(3);
    });

    it("should throw when quantity is zero or negative", () => {
      const cart = new Cart();
      expect(() =>
        cart.addItem({ id: "sku-1", name: "Widget", price: 9.99, qty: 0 })
      ).toThrow("Quantity must be positive");
    });
  });
});
```

### Python / Pytest (Advanced Patterns)

```python
# tests/conftest.py — shared fixtures
import pytest
from app.db import create_engine, Session

@pytest.fixture(scope="session")
def db_engine():
    engine = create_engine("sqlite:///:memory:")
    yield engine
    engine.dispose()

@pytest.fixture
def db_session(db_engine):
    session = Session(bind=db_engine)
    yield session
    session.rollback()
    session.close()

# tests/test_pricing.py — parametrize for multiple cases
import pytest
from app.pricing import calculate_discount

@pytest.mark.parametrize("subtotal, expected_discount", [
    (50.0, 0.0),       # Below threshold — no discount
    (100.0, 5.0),      # 5% tier
    (250.0, 25.0),     # 10% tier
    (500.0, 75.0),     # 15% tier
])
def test_calculate_discount(subtotal, expected_discount):
    assert calculate_discount(subtotal) == pytest.approx(expected_discount)
```

### Go — Table-Driven Tests

```go
// cart_test.go
package cart

import "testing"

func TestApplyDiscount(t *testing.T) {
    tests := []struct {
        name     string
        subtotal float64
        want     float64
    }{
        {"no discount below threshold", 50.0, 0.0},
        {"5 percent tier", 100.0, 5.0},
        {"10 percent tier", 250.0, 25.0},
        {"15 percent tier", 500.0, 75.0},
        {"zero subtotal", 0.0, 0.0},
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            got := ApplyDiscount(tt.subtotal)
            if got != tt.want {
                t.Errorf("ApplyDiscount(%v) = %v, want %v", tt.subtotal, got, tt.want)
            }
        })
    }
}
```

---

## Bounded Autonomy Rules

When generating tests autonomously, follow these rules to decide when to stop and ask the user:

### Stop and Ask When

- **Ambiguous requirements** — the spec or user story has conflicting or unclear acceptance criteria
- **Missing edge cases** — you cannot determine boundary values without domain knowledge (e.g., max allowed transaction amount)
- **Test count exceeds 50** — large test suites need human review before committing; present a summary and ask which areas to prioritize
- **External dependencies unclear** — the feature relies on third-party APIs or services with undocumented behavior
- **Security-sensitive logic** — authentication, authorization, encryption, or payment flows require human sign-off on test scenarios

### Continue Autonomously When

- **Clear spec with numbered acceptance criteria** — each criterion maps directly to tests
- **Straightforward CRUD operations** — create, read, update, delete with well-defined models
- **Well-defined API contracts** — OpenAPI spec or typed interfaces available
- **Pure functions** — deterministic input/output with no side effects
- **Existing test patterns** — the codebase already has similar tests to follow

---

## Property-Based Testing

Property-based testing generates random inputs to verify invariants instead of relying on hand-picked examples. Use it when the input space is large and the expected behavior can be described as a property.

### Python — Hypothesis

```python
from hypothesis import given, strategies as st
from app.serializers import serialize, deserialize

@given(st.text())
def test_roundtrip_serialization(data):
    """Serialization followed by deserialization returns the original."""
    assert deserialize(serialize(data)) == data

@given(st.integers(), st.integers())
def test_addition_is_commutative(a, b):
    assert a + b == b + a
```

### TypeScript — fast-check

```typescript
import fc from "fast-check";
import { encode, decode } from "./codec";

test("encode/decode roundtrip", () => {
  fc.assert(
    fc.property(fc.string(), (input) => {
      expect(decode(encode(input))).toBe(input);
    })
  );
});
```

### When to Use Property-Based Over Example-Based

| Use Property-Based | Example |
|-------------------|---------|
| Data transformations | Serialize/deserialize roundtrips |
| Mathematical properties | Commutativity, associativity, idempotency |
| Encoding/decoding | Base64, URL encoding, compression |
| Sorting and filtering | Output is sorted, length preserved |
| Parser correctness | Valid input always parses without error |

---

## Mutation Testing

Mutation testing modifies your production code (creates "mutants") and checks whether your tests catch the changes. If a mutant survives (tests still pass), your tests have a gap that coverage alone cannot reveal.

### Tools

| Language | Tool | Command |
|----------|------|---------|
| TypeScript/JavaScript | **Stryker** | `npx stryker run` |
| Python | **mutmut** | `mutmut run --paths-to-mutate=src/` |
| Java | **PIT** | `mvn org.pitest:pitest-maven:mutationCoverage` |

### Why Mutation Testing Matters

- **100% line coverage != good tests** — coverage tells you code was executed, not that it was verified
- **Catches weak assertions** — tests that run code but assert nothing meaningful
- **Finds missing boundary tests** — mutants that change `<` to `<=` expose off-by-one gaps
- **Quantifiable quality metric** — mutation score (% mutants killed) is a stronger signal than coverage %

**Recommendation:** Run mutation testing on critical paths (auth, payments, data processing) even if overall coverage is high. Target 85%+ mutation score on P0 modules.

---

## Cross-References

| Skill | Relationship |
|-------|-------------|
| `engineering/spec-driven-workflow` | Spec → acceptance criteria → test extraction pipeline |
| `engineering-team/focused-fix` | Phase 5 (Verify) uses TDD to confirm the fix with a regression test |
| `engineering-team/senior-qa` | Broader QA strategy; TDD is one layer in the test pyramid |
| `engineering-team/code-reviewer` | Review generated tests for assertion quality and coverage completeness |
| `engineering-team/senior-fullstack` | Project scaffolders include testing infrastructure compatible with TDD workflows |

---

## Limitations

| Scope | Details |
|-------|---------|
| Unit test focus | Integration and E2E tests require different patterns |
| Static analysis | Cannot execute tests or measure runtime behavior |
| Language support | Best for TypeScript, JavaScript, Python, Java |
| Report formats | LCOV, JSON, XML only; other formats need conversion |
| Generated tests | Provide scaffolding; require human review for complex logic |

**When to use other tools:**
- E2E testing: Playwright, Cypress, Selenium
- Performance testing: k6, JMeter, Locust
- Security testing: OWASP ZAP, Burp Suite

FILE:assets/expected_output.json
{
  "test_generation": {
    "generated_tests": [
      {
        "name": "should_validate_password_length_successfully",
        "type": "happy_path",
        "priority": "P0",
        "framework": "jest",
        "code": "it('should validate password with sufficient length', () => {\n  const validator = new PasswordValidator();\n  const result = validator.validate('Test@123');\n  expect(result).toBe(true);\n});"
      },
      {
        "name": "should_handle_too_short_password",
        "type": "error_case",
        "priority": "P0",
        "framework": "jest",
        "code": "it('should reject password shorter than 8 characters', () => {\n  const validator = new PasswordValidator();\n  const result = validator.validate('Test@1');\n  expect(result).toBe(false);\n});"
      }
    ],
    "test_file": "password-validator.test.ts",
    "total_tests_generated": 8
  },
  "coverage_analysis": {
    "summary": {
      "line_coverage": 100.0,
      "branch_coverage": 100.0,
      "function_coverage": 100.0,
      "total_lines": 20,
      "covered_lines": 20,
      "total_branches": 12,
      "covered_branches": 12
    },
    "gaps": [],
    "assessment": "Excellent coverage - all paths tested"
  },
  "metrics": {
    "complexity": {
      "cyclomatic_complexity": 6,
      "cognitive_complexity": 8,
      "testability_score": 85.0,
      "assessment": "Medium complexity - moderately testable"
    },
    "test_quality": {
      "total_tests": 8,
      "total_assertions": 16,
      "avg_assertions_per_test": 2.0,
      "isolation_score": 95.0,
      "naming_quality": 87.5,
      "quality_score": 88.0,
      "test_smells": []
    }
  },
  "recommendations": [
    {
      "priority": "P1",
      "type": "edge_case_coverage",
      "message": "Consider adding boundary value tests",
      "action": "Add tests for exact boundary conditions (7 vs 8 characters)",
      "impact": "medium"
    },
    {
      "priority": "P2",
      "type": "test_organization",
      "message": "Group related tests using describe blocks",
      "action": "Organize tests by feature (length validation, complexity validation)",
      "impact": "low"
    }
  ],
  "tdd_workflow": {
    "current_phase": "GREEN",
    "status": "Tests passing, ready for refactoring",
    "next_steps": [
      "Review code for duplication",
      "Consider extracting validation rules",
      "Commit changes"
    ]
  }
}

FILE:assets/sample_coverage_report.lcov
TN:
SF:src/auth/password-validator.ts
FN:3,(anonymous_0)
FN:4,validate
FNDA:10,(anonymous_0)
FNDA:25,validate
FNF:2
FNH:2
DA:1,1
DA:2,1
DA:3,1
DA:4,25
DA:5,25
DA:6,10
DA:7,20
DA:8,8
DA:9,15
DA:10,5
DA:11,12
DA:12,3
LF:12
LH:12
BRDA:5,0,0,10
BRDA:5,0,1,15
BRDA:7,1,0,8
BRDA:7,1,1,12
BRDA:9,2,0,5
BRDA:9,2,1,10
BRDA:11,3,0,3
BRDA:11,3,1,9
BRF:8
BRH:8
end_of_record
TN:
SF:src/utils/discount-calculator.ts
FN:1,calculateDiscount
FNDA:15,calculateDiscount
FNF:1
FNH:1
DA:1,1
DA:2,15
DA:3,15
DA:4,2
DA:5,13
DA:6,1
DA:8,12
DA:9,12
LF:8
LH:8
BRDA:3,0,0,2
BRDA:3,0,1,13
BRDA:5,1,0,1
BRDA:5,1,1,12
BRF:4
BRH:4
end_of_record

FILE:assets/sample_input_python.json
{
  "language": "python",
  "framework": "pytest",
  "source_code": "def calculate_discount(price: float, discount_percent: float) -> float:\n    \"\"\"Calculate discounted price.\"\"\"\n    if price < 0:\n        raise ValueError(\"Price cannot be negative\")\n    if discount_percent < 0 or discount_percent > 100:\n        raise ValueError(\"Discount must be between 0 and 100\")\n    \n    discount_amount = price * (discount_percent / 100)\n    return round(price - discount_amount, 2)",
  "requirements": {
    "user_stories": [
      {
        "description": "Calculate discounted price for valid inputs",
        "action": "calculate_discount",
        "given": ["Price is 100", "Discount is 20%"],
        "when": "Discount is calculated",
        "then": "Return 80.00",
        "error_conditions": [
          {
            "condition": "negative_price",
            "description": "Price is negative",
            "error_type": "ValueError"
          },
          {
            "condition": "invalid_discount",
            "description": "Discount is out of range",
            "error_type": "ValueError"
          }
        ],
        "edge_cases": [
          {
            "scenario": "zero_discount",
            "description": "Discount is 0%"
          },
          {
            "scenario": "full_discount",
            "description": "Discount is 100%"
          }
        ]
      }
    ]
  },
  "coverage_threshold": 90
}

FILE:assets/sample_input_typescript.json
{
  "language": "typescript",
  "framework": "jest",
  "source_code": "export class PasswordValidator {\n  validate(password: string): boolean {\n    if (password.length < 8) return false;\n    if (!/[A-Z]/.test(password)) return false;\n    if (!/[a-z]/.test(password)) return false;\n    if (!/[0-9]/.test(password)) return false;\n    if (!/[!@#$%^&*]/.test(password)) return false;\n    return true;\n  }\n}",
  "requirements": {
    "user_stories": [
      {
        "description": "Password must be at least 8 characters long",
        "action": "validate_password_length",
        "given": ["User provides password"],
        "when": "Password is validated",
        "then": "Reject if less than 8 characters"
      },
      {
        "description": "Password must contain uppercase, lowercase, number, and special character",
        "action": "validate_password_complexity",
        "given": ["User provides password"],
        "when": "Password is validated",
        "then": "Reject if missing any character type"
      }
    ],
    "acceptance_criteria": [
      {
        "id": "AC1",
        "description": "Valid password: 'Test@123'",
        "verification_steps": ["Call validate with 'Test@123'", "Should return true"]
      },
      {
        "id": "AC2",
        "description": "Invalid password: 'test' (too short)",
        "verification_steps": ["Call validate with 'test'", "Should return false"]
      }
    ]
  },
  "coverage_threshold": 80
}

FILE:HOW_TO_USE.md
# How to Use the TDD Guide Skill

The TDD Guide skill helps engineering teams implement Test Driven Development with intelligent test generation, coverage analysis, and workflow guidance.

## Basic Usage

### Generate Tests from Requirements

```
@tdd-guide

I need to implement a user registration feature. Generate test cases for:
- Email validation
- Password strength checking
- Duplicate email detection

Language: TypeScript
Framework: Jest
```

### Analyze Test Coverage

```
@tdd-guide

Analyze test coverage for my authentication module.

Coverage report: coverage/lcov.info
Source code: src/auth/

Identify gaps and prioritize improvements.
```

### Get TDD Workflow Guidance

```
@tdd-guide

Guide me through TDD for implementing a shopping cart feature.

Requirements:
- Add items to cart
- Update quantities
- Calculate totals
- Apply discount codes

Framework: Pytest
```

## Example Invocations

### Example 1: Generate Tests from Code

```
@tdd-guide

Generate comprehensive tests for this function:

```typescript
export function calculateTax(amount: number, rate: number): number {
  if (amount < 0) throw new Error('Amount cannot be negative');
  if (rate < 0 || rate > 1) throw new Error('Rate must be between 0 and 1');
  return Math.round(amount * rate * 100) / 100;
}
```

Include:
- Happy path tests
- Error cases
- Boundary values
- Edge cases
```

### Example 2: Improve Coverage

```
@tdd-guide

My coverage is at 65%. Help me get to 80%.

Coverage report:
[paste LCOV or JSON coverage data]

Source files:
- src/services/payment-processor.ts
- src/services/order-validator.ts

Prioritize critical paths.
```

### Example 3: Review Test Quality

```
@tdd-guide

Review the quality of these tests:

```python
def test_login():
    result = login("user", "pass")
    assert result is not None
    assert result.status == "success"
    assert result.token != ""
    assert len(result.permissions) > 0

def test_login_fails():
    result = login("bad", "wrong")
    assert result is None
```

Suggest improvements for:
- Test isolation
- Assertion quality
- Naming conventions
- Test organization
```

### Example 4: Framework Migration

```
@tdd-guide

Convert these Jest tests to Pytest:

```javascript
describe('Calculator', () => {
  it('should add two numbers', () => {
    const result = add(2, 3);
    expect(result).toBe(5);
  });

  it('should handle negative numbers', () => {
    const result = add(-2, 3);
    expect(result).toBe(1);
  });
});
```

Maintain test structure and coverage.
```

### Example 5: Generate Test Fixtures

```
@tdd-guide

Generate realistic test fixtures for:

Entity: User
Fields:
- id (UUID)
- email (valid format)
- age (18-100)
- role (admin, user, guest)

Generate 5 fixtures with edge cases:
- Minimum age boundary
- Maximum age boundary
- Special characters in email
```

## What to Provide

### For Test Generation
- Source code (TypeScript, JavaScript, Python, or Java)
- Requirements (user stories, API specs, or business rules)
- Testing framework preference (Jest, Pytest, JUnit, Vitest)
- Specific scenarios to cover (optional)

### For Coverage Analysis
- Coverage report (LCOV, JSON, or XML format)
- Source code files (optional, for context)
- Coverage threshold target (e.g., 80%)

### For TDD Workflow
- Feature requirements
- Current phase (RED, GREEN, or REFACTOR)
- Test code and implementation (for validation)

### For Quality Review
- Existing test code
- Specific quality concerns (isolation, naming, assertions)

## What You'll Get

### Test Generation Output
- Complete test files with proper structure
- Test stubs with arrange-act-assert pattern
- Framework-specific imports and syntax
- Coverage for happy paths, errors, and edge cases

### Coverage Analysis Output
- Overall coverage summary (line, branch, function)
- Identified gaps with file/line numbers
- Prioritized recommendations (P0, P1, P2)
- Visual coverage indicators

### TDD Workflow Output
- Step-by-step guidance for current phase
- Validation of RED/GREEN/REFACTOR completion
- Refactoring suggestions
- Next steps in TDD cycle

### Quality Review Output
- Test quality score (0-100)
- Detected test smells
- Isolation and naming analysis
- Specific improvement recommendations

## Tips for Best Results

### Test Generation
1. **Be specific**: "Generate tests for password validation" is better than "generate tests"
2. **Provide context**: Include edge cases and error conditions you want covered
3. **Specify framework**: Mention Jest, Pytest, JUnit, etc., for correct syntax

### Coverage Analysis
1. **Use recent reports**: Coverage data should match current codebase
2. **Provide thresholds**: Specify your target coverage percentage
3. **Focus on critical code**: Prioritize coverage for business logic

### TDD Workflow
1. **Start with requirements**: Clear requirements lead to better tests
2. **One cycle at a time**: Complete RED-GREEN-REFACTOR before moving on
3. **Validate each phase**: Run tests and share results for accurate guidance

### Quality Review
1. **Share full context**: Include test setup/teardown and helper functions
2. **Ask specific questions**: "Is my isolation good?" gets better answers than "review this"
3. **Iterative improvement**: Implement suggestions incrementally

## Advanced Usage

### Multi-Language Projects

```
@tdd-guide

Analyze coverage across multiple languages:
- Frontend: TypeScript (Jest) - src/frontend/
- Backend: Python (Pytest) - src/backend/
- API: Java (JUnit) - src/api/

Provide unified coverage report and recommendations.
```

### CI/CD Integration

```
@tdd-guide

Generate coverage report for CI pipeline.

Input: coverage/coverage-final.json
Output format: JSON

Include:
- Pass/fail based on 80% threshold
- Changed files coverage
- Trend comparison with main branch
```

### Parameterized Test Generation

```
@tdd-guide

Generate parameterized tests for:

Function: validateEmail(email: string): boolean

Test cases:
- valid@example.com → true
- invalid.email → false
- @example.com → false
- user@domain.co.uk → true

Framework: Jest (test.each)
```

## Related Commands

- `/code-review` - Review code quality and suggest improvements
- `/test` - Run tests and analyze results
- `/refactor` - Get refactoring suggestions while keeping tests green

## Troubleshooting

**Issue**: Generated tests don't match my framework syntax
- **Solution**: Explicitly specify framework (e.g., "using Pytest" or "with Jest")

**Issue**: Coverage analysis shows 0% coverage
- **Solution**: Verify coverage report format (LCOV, JSON, XML) and try including raw content

**Issue**: TDD workflow validation fails
- **Solution**: Ensure you're providing test results (passed/failed status) along with code

**Issue**: Too many recommendations
- **Solution**: Ask for "top 3 P0 recommendations only" for focused output

## Version Support

- **Node.js**: 16+ (Jest 29+, Vitest 0.34+)
- **Python**: 3.8+ (Pytest 7+)
- **Java**: 11+ (JUnit 5.9+)
- **TypeScript**: 4.5+

## Feedback

If you encounter issues or have suggestions, please mention:
- Language and framework used
- Type of operation (generation, analysis, workflow)
- Expected vs. actual behavior

FILE:README.md
# TDD Guide - Test Driven Development Skill

**Version**: 1.0.0
**Last Updated**: November 5, 2025
**Author**: Claude Skills Factory

A comprehensive Test Driven Development skill for Claude Code that provides intelligent test generation, coverage analysis, framework integration, and TDD workflow guidance across multiple languages and testing frameworks.

## Table of Contents

- [Overview](#overview)
- [Features](#features)
- [Installation](#installation)
- [Quick Start](#quick-start)
- [Python Modules](#python-modules)
- [Usage Examples](#usage-examples)
- [Configuration](#configuration)
- [Supported Frameworks](#supported-frameworks)
- [Output Formats](#output-formats)
- [Best Practices](#best-practices)
- [Troubleshooting](#troubleshooting)
- [Contributing](#contributing)
- [License](#license)

## Overview

The TDD Guide skill transforms how engineering teams implement Test Driven Development by providing:

- **Intelligent Test Generation**: Convert requirements into executable test cases
- **Coverage Analysis**: Parse LCOV, JSON, XML reports and identify gaps
- **Multi-Framework Support**: Jest, Pytest, JUnit, Vitest, and more
- **TDD Workflow Guidance**: Step-by-step red-green-refactor guidance
- **Quality Metrics**: Comprehensive test and code quality analysis
- **Context-Aware Output**: Optimized for Desktop, CLI, or API usage

## Features

### Test Generation (3 capabilities)
1. **Generate Test Cases from Requirements** - User stories → Test cases
2. **Create Test Stubs** - Proper scaffolding with framework patterns
3. **Generate Test Fixtures** - Realistic test data and boundary values

### TDD Workflow (3 capabilities)
1. **Red-Green-Refactor Guidance** - Phase-by-phase validation
2. **Suggest Missing Scenarios** - Identify untested edge cases
3. **Review Test Quality** - Isolation, assertions, naming analysis

### Coverage & Metrics (6 categories)
1. **Test Coverage** - Line/branch/function with gap analysis
2. **Code Complexity** - Cyclomatic/cognitive complexity
3. **Test Quality** - Assertions, isolation, naming scoring
4. **Test Data** - Boundary values, edge cases
5. **Test Execution** - Timing, slow tests, flakiness
6. **Missing Tests** - Uncovered paths and error handlers

### Framework Integration (4 capabilities)
1. **Multi-Framework Adapters** - Jest, Pytest, JUnit, Vitest, Mocha
2. **Generate Boilerplate** - Proper imports and test structure
3. **Configure Runners** - Setup and coverage configuration
4. **Framework Detection** - Automatic framework identification

## Installation

### Claude Code (Desktop)

1. **Download the skill folder**:
   ```bash
   # Option A: Clone from repository
   git clone https://github.com/your-org/tdd-guide-skill.git

   # Option B: Download ZIP and extract
   ```

2. **Install to Claude skills directory**:
   ```bash
   # Project-level (recommended for team projects)
   cp -r tdd-guide /path/to/your/project/.claude/skills/

   # User-level (available for all projects)
   cp -r tdd-guide ~/.claude/skills/
   ```

3. **Verify installation**:
   ```bash
   ls ~/.claude/skills/tdd-guide/
   # Should show: SKILL.md, *.py files, samples
   ```

### Claude Apps (Browser)

1. Use the `skill-creator` skill to import the ZIP file
2. Or manually upload files through the skills interface

### Claude API

```python
# Upload skill via API
import anthropic

client = anthropic.Anthropic(api_key="your-api-key")

# Create skill with files
skill = client.skills.create(
    name="tdd-guide",
    files=["tdd-guide/SKILL.md", "tdd-guide/*.py"]
)
```

## Quick Start

### 1. Generate Tests from Requirements

```
@tdd-guide

Generate tests for password validation function:
- Min 8 characters
- At least 1 uppercase, 1 lowercase, 1 number, 1 special char

Language: TypeScript
Framework: Jest
```

### 2. Analyze Coverage

```
@tdd-guide

Analyze coverage from: coverage/lcov.info
Target: 80% coverage
Prioritize recommendations
```

### 3. TDD Workflow

```
@tdd-guide

Guide me through TDD for implementing user authentication.

Requirements: Email/password login, session management
Framework: Pytest
```

## Python Modules

The skill includes **8 Python modules** organized by functionality:

### Core Modules (7 files)

1. **test_generator.py** (450 lines)
   - Generate test cases from requirements
   - Create test stubs with proper structure
   - Suggest missing scenarios based on code analysis
   - Support for multiple test types (unit, integration, e2e)

2. **coverage_analyzer.py** (380 lines)
   - Parse LCOV, JSON, XML coverage reports
   - Calculate line/branch/function coverage
   - Identify coverage gaps with prioritization
   - Generate actionable recommendations

3. **metrics_calculator.py** (420 lines)
   - Cyclomatic and cognitive complexity analysis
   - Test quality scoring (isolation, assertions, naming)
   - Test smell detection
   - Execution metrics analysis

4. **framework_adapter.py** (480 lines)
   - Multi-framework adapters (Jest, Pytest, JUnit, Vitest, Mocha)
   - Generate framework-specific imports and structure
   - Assertion syntax translation
   - Setup/teardown hook generation

5. **tdd_workflow.py** (380 lines)
   - Red-Green-Refactor phase guidance
   - Phase validation and progression
   - Refactoring suggestions
   - Workflow state management

6. **fixture_generator.py** (340 lines)
   - Boundary value generation
   - Edge case scenario creation
   - Mock data generation from schemas
   - Fixture file export (JSON, YAML, Python)

7. **format_detector.py** (280 lines)
   - Automatic language detection
   - Testing framework identification
   - Coverage format detection
   - Project structure analysis

### Utility Modules (1 file)

8. **output_formatter.py** (260 lines)
   - Context-aware formatting (Desktop, CLI, API)
   - Progressive disclosure
   - Token-efficient reporting
   - Summary and detailed views

### Total Lines of Code
- **Python**: ~3,400 lines
- **Documentation**: ~1,200 lines
- **Samples**: ~200 lines
- **Total**: ~4,400 lines

## Usage Examples

### Example 1: Generate Tests from User Story

**Input**:
```
@tdd-guide

User Story: As a user, I want to add items to my shopping cart

Acceptance Criteria:
- Add single item with quantity
- Update quantity if item already exists
- Validate quantity is positive
- Return updated cart total

Language: Python
Framework: Pytest
```

**Output**:
```python
import pytest
from shopping_cart import ShoppingCart

class TestShoppingCart:
    """Test suite for shopping cart functionality."""

    def test_add_single_item_successfully(self):
        """Should add single item with quantity."""
        cart = ShoppingCart()
        cart.add_item('apple', quantity=3, price=1.50)

        assert len(cart.items) == 1
        assert cart.items['apple']['quantity'] == 3
        assert cart.get_total() == 4.50

    def test_update_quantity_for_existing_item(self):
        """Should update quantity if item already exists."""
        cart = ShoppingCart()
        cart.add_item('apple', quantity=3, price=1.50)
        cart.add_item('apple', quantity=2, price=1.50)

        assert cart.items['apple']['quantity'] == 5
        assert cart.get_total() == 7.50

    def test_reject_negative_quantity(self):
        """Should validate quantity is positive."""
        cart = ShoppingCart()

        with pytest.raises(ValueError, match="Quantity must be positive"):
            cart.add_item('apple', quantity=-1, price=1.50)
```

### Example 2: Coverage Analysis with Recommendations

**Input**:
```
@tdd-guide

Analyze coverage:

[Coverage report data]
SF:src/auth/login.ts
DA:1,5
DA:2,5
DA:3,3
DA:4,0
DA:5,0
...

Target: 80% coverage
```

**Output**:
```markdown
## Coverage Summary

- Line Coverage: 65.0%
- Branch Coverage: 58.3%
- Function Coverage: 75.0%

⚠️ **65.0%** coverage (yellow) - Below 80% threshold

## Recommendations

### 🔴 Critical (P0)

1. **Error handling paths not covered**
   - Action: Add tests for authentication failure scenarios
   - File: `src/auth/login.ts`
   - Lines: 45-52, 67-70

2. **Branch coverage gap in validation**
   - Action: Test all conditional branches in input validation
   - File: `src/auth/login.ts`
   - Lines: 23-28

### 🟡 Important (P1)

1. **Edge cases missing**
   - Action: Add boundary tests for password length (7, 8, 9 chars)
   - Impact: Medium
```

### Example 3: TDD Red-Green-Refactor Guidance

**Input**:
```
@tdd-guide

Guide TDD for email validation.

Requirement: Email must contain @ and valid domain

Phase: RED (write failing test)
```

**Output**:
```markdown
## TDD Phase: RED

### Goal
Write a failing test that describes desired email validation behavior

### Steps
1. ✅ Read requirement: Email must contain @ and valid domain
2. ✅ Think about expected behavior
3. ⏳ Write test that verifies this behavior
4. ⏳ Run test and ensure it fails
5. ⏳ Verify failure reason is correct (not syntax error)

### Test Template (Jest)

```typescript
describe('EmailValidator', () => {
  it('should accept valid email format', () => {
    // Arrange
    const validator = new EmailValidator();

    // Act
    const result = validator.validate('user@example.com');

    // Assert
    expect(result).toBe(true);
  });

  it('should reject email without @ symbol', () => {
    const validator = new EmailValidator();
    const result = validator.validate('userexample.com');
    expect(result).toBe(false);
  });
});
```

### Next Steps
1. Run the test (it should fail)
2. Verify failure: "EmailValidator is not defined" or similar
3. Share test result to proceed to GREEN phase
```

### Example 4: Test Quality Review

**Input**:
```
@tdd-guide

Review test quality:

def test_user_login():
    result = login("admin", "password123")
    assert result
    assert result["status"] == "success"
    assert result["token"]
    assert result["user"]["id"] > 0
    assert result["user"]["role"] == "admin"
```

**Output**:
```markdown
## Test Quality Analysis

**Overall Score**: 62/100

### Issues Detected

1. **Assertion Roulette** (High Severity)
   - 5 assertions in single test
   - **Fix**: Split into multiple focused tests

2. **Poor Test Isolation** (Medium Severity)
   - Test depends on database state ("admin" user existing)
   - **Fix**: Use fixtures or mocks for user data

3. **Weak Assertions** (Medium Severity)
   - `assert result["token"]` just checks truthy (could be any string)
   - **Fix**: `assert len(result["token"]) > 20` or use regex

4. **Missing Naming Context** (Low Severity)
   - Test name doesn't describe what's being tested
   - **Fix**: `test_successful_login_returns_valid_token_and_user_data`

### Recommended Refactoring

```python
@pytest.fixture
def admin_user():
    return {"username": "admin", "password": "password123"}

def test_successful_login_returns_success_status(admin_user):
    """Test that valid credentials return success status."""
    result = login(admin_user["username"], admin_user["password"])
    assert result["status"] == "success"

def test_successful_login_returns_valid_jwt_token(admin_user):
    """Test that successful login returns a JWT token."""
    result = login(admin_user["username"], admin_user["password"])
    assert len(result["token"]) > 20
    assert result["token"].count('.') == 2  # JWT format

def test_successful_login_returns_user_data(admin_user):
    """Test that successful login includes user information."""
    result = login(admin_user["username"], admin_user["password"])
    assert result["user"]["id"] > 0
    assert result["user"]["role"] == "admin"
```

### Quality Improvements
- Isolation Score: 55 → 90 (with fixtures)
- Naming Quality: 40 → 85 (descriptive names)
- Test Focus: 30 → 95 (one assertion per test)
```

## Configuration

### Environment Variables

```bash
# Set preferred testing framework
export TDD_DEFAULT_FRAMEWORK="jest"

# Set coverage threshold
export TDD_COVERAGE_THRESHOLD=80

# Set output verbosity
export TDD_VERBOSE=true

# Set output format
export TDD_OUTPUT_FORMAT="markdown"  # or "json", "terminal"
```

### Skill Configuration (Optional)

Create `.tdd-guide.json` in project root:

```json
{
  "framework": "jest",
  "language": "typescript",
  "coverage_threshold": 80,
  "test_directory": "tests/",
  "quality_rules": {
    "max_assertions_per_test": 3,
    "require_descriptive_names": true,
    "enforce_isolation": true
  },
  "output": {
    "format": "markdown",
    "verbose": false,
    "max_recommendations": 10
  }
}
```

## Supported Frameworks

### JavaScript/TypeScript
- **Jest** 29+ (recommended for React, Node.js)
- **Vitest** 0.34+ (recommended for Vite projects)
- **Mocha** 10+ with Chai
- **Jasmine** 4+

### Python
- **Pytest** 7+ (recommended)
- **unittest** (Python standard library)
- **nose2** 0.12+

### Java
- **JUnit 5** 5.9+ (recommended)
- **TestNG** 7+
- **Mockito** 5+ (mocking support)

### Coverage Tools
- **Istanbul/nyc** (JavaScript)
- **c8** (JavaScript, V8 native)
- **coverage.py** (Python)
- **pytest-cov** (Python)
- **JaCoCo** (Java)
- **Cobertura** (multi-language)

## Output Formats

### Markdown (Claude Desktop)
- Rich formatting with headers, tables, code blocks
- Visual indicators (✅, ⚠️, ❌)
- Progressive disclosure (summary first, details on demand)
- Syntax highlighting for code examples

### Terminal (Claude Code CLI)
- Concise, text-based output
- Clear section separators
- Minimal formatting for readability
- Quick scanning for key information

### JSON (API/CI Integration)
- Structured data for automated processing
- Machine-readable metrics
- Suitable for CI/CD pipelines
- Easy integration with other tools

## Best Practices

### Test Generation
1. **Start with requirements** - Clear specs lead to better tests
2. **Cover the happy path first** - Then add error and edge cases
3. **One behavior per test** - Focused tests are easier to maintain
4. **Use descriptive names** - Tests are documentation

### Coverage Analysis
1. **Aim for 80%+ coverage** - Balance between safety and effort
2. **Prioritize critical paths** - Not all code needs 100% coverage
3. **Branch coverage matters** - Line coverage alone is insufficient
4. **Track trends** - Coverage should improve over time

### TDD Workflow
1. **Small iterations** - Write one test, make it pass, refactor
2. **Run tests frequently** - Fast feedback loop is essential
3. **Commit often** - Each green phase is a safe checkpoint
4. **Refactor with confidence** - Tests are your safety net

### Test Quality
1. **Isolate tests** - No shared state between tests
2. **Fast execution** - Unit tests should be <100ms each
3. **Deterministic** - Same input always produces same output
4. **Clear failures** - Good error messages save debugging time

## Troubleshooting

### Common Issues

**Issue**: Generated tests have wrong syntax for my framework
```
Solution: Explicitly specify framework
Example: "Generate tests using Pytest" or "Framework: Jest"
```

**Issue**: Coverage report not recognized
```
Solution: Verify format (LCOV, JSON, XML)
Try: Paste raw coverage data instead of file path
Check: File exists and is readable
```

**Issue**: Too many recommendations, overwhelmed
```
Solution: Ask for prioritized output
Example: "Show only P0 (critical) recommendations"
Limit: "Top 5 recommendations only"
```

**Issue**: Test quality score seems wrong
```
Check: Ensure complete test context (setup/teardown included)
Verify: Test file contains actual test code, not just stubs
Context: Quality depends on isolation, assertions, naming
```

**Issue**: Framework detection incorrect
```
Solution: Specify framework explicitly
Example: "Using JUnit 5" or "Framework: Vitest"
Check: Ensure imports are present in code
```

## File Structure

```
tdd-guide/
├── SKILL.md                          # Skill definition (YAML + documentation)
├── README.md                         # This file
├── HOW_TO_USE.md                     # Usage examples
│
├── test_generator.py                 # Test generation core
├── coverage_analyzer.py              # Coverage parsing and analysis
├── metrics_calculator.py             # Quality metrics calculation
├── framework_adapter.py              # Multi-framework support
├── tdd_workflow.py                   # Red-green-refactor guidance
├── fixture_generator.py              # Test data and fixtures
├── format_detector.py                # Automatic format detection
├── output_formatter.py               # Context-aware output
│
├── sample_input_typescript.json      # TypeScript example
├── sample_input_python.json          # Python example
├── sample_coverage_report.lcov       # LCOV coverage example
└── expected_output.json              # Expected output structure
```

## Contributing

We welcome contributions! To contribute:

1. Fork the repository
2. Create a feature branch (`git checkout -b feature/improvement`)
3. Make your changes
4. Add tests for new functionality
5. Run validation: `python -m pytest tests/`
6. Commit changes (`git commit -m "Add: feature description"`)
7. Push to branch (`git push origin feature/improvement`)
8. Open a Pull Request

### Development Setup

```bash
# Clone repository
git clone https://github.com/your-org/tdd-guide-skill.git
cd tdd-guide-skill

# Install development dependencies
pip install -r requirements-dev.txt

# Run tests
pytest tests/ -v

# Run linter
pylint *.py

# Run type checker
mypy *.py
```

## Version History

### v1.0.0 (November 5, 2025)
- Initial release
- Support for TypeScript, JavaScript, Python, Java
- Jest, Pytest, JUnit, Vitest framework adapters
- LCOV, JSON, XML coverage parsing
- TDD workflow guidance (red-green-refactor)
- Test quality metrics and analysis
- Context-aware output formatting
- Comprehensive documentation

## License

MIT License - See LICENSE file for details

## Support

- **Documentation**: See HOW_TO_USE.md for detailed examples
- **Issues**: Report bugs via GitHub issues
- **Questions**: Ask in Claude Code community forum
- **Updates**: Check repository for latest version

## Acknowledgments

Built with Claude Skills Factory toolkit, following Test Driven Development best practices and informed by:
- Kent Beck's "Test Driven Development: By Example"
- Martin Fowler's refactoring catalog
- xUnit Test Patterns by Gerard Meszaros
- Growing Object-Oriented Software, Guided by Tests

---

**Ready to improve your testing workflow?** Install the TDD Guide skill and start generating high-quality tests today!

FILE:references/ci-integration.md
# CI/CD Integration Guide

Integrating test coverage and quality gates into CI pipelines.

---

## Table of Contents

- [Coverage in CI](#coverage-in-ci)
- [GitHub Actions Examples](#github-actions-examples)
- [Quality Gates](#quality-gates)
- [Trend Tracking](#trend-tracking)

---

## Coverage in CI

### Coverage Report Flow

1. Run tests with coverage enabled
2. Generate report in machine-readable format (LCOV, JSON, XML)
3. Parse report for threshold validation
4. Upload to coverage service (Codecov, Coveralls)
5. Fail build if below threshold

### Report Formats by Tool

| Tool | Command | Output Format |
|------|---------|---------------|
| Jest | `jest --coverage --coverageReporters=lcov` | LCOV |
| Pytest | `pytest --cov-report=xml` | Cobertura XML |
| JUnit/JaCoCo | `mvn jacoco:report` | JaCoCo XML |
| Vitest | `vitest --coverage` | LCOV/JSON |

---

## GitHub Actions Examples

### Node.js (Jest)

```yaml
name: Test and Coverage

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'

      - run: npm ci
      - run: npm test -- --coverage

      - name: Check coverage threshold
        run: |
          COVERAGE=$(cat coverage/coverage-summary.json | jq '.total.lines.pct')
          if (( $(echo "$COVERAGE < 80" | bc -l) )); then
            echo "Coverage $COVERAGE% is below 80% threshold"
            exit 1
          fi

      - uses: codecov/codecov-action@v4
        with:
          file: coverage/lcov.info
```

### Python (Pytest)

```yaml
name: Test and Coverage

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - run: pip install pytest pytest-cov
      - run: pytest --cov=src --cov-report=xml --cov-fail-under=80

      - uses: codecov/codecov-action@v4
        with:
          file: coverage.xml
```

### Java (Maven + JaCoCo)

```yaml
name: Test and Coverage

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-java@v4
        with:
          distribution: 'temurin'
          java-version: '17'

      - run: mvn test jacoco:check

      - uses: codecov/codecov-action@v4
        with:
          file: target/site/jacoco/jacoco.xml
```

---

## Quality Gates

### Threshold Configuration

**Jest (package.json):**
```json
{
  "jest": {
    "coverageThreshold": {
      "global": {
        "branches": 80,
        "functions": 80,
        "lines": 80,
        "statements": 80
      }
    }
  }
}
```

**Pytest (pyproject.toml):**
```toml
[tool.coverage.report]
fail_under = 80
```

**JaCoCo (pom.xml):**
```xml
<rule>
  <element>BUNDLE</element>
  <limits>
    <limit>
      <counter>LINE</counter>
      <value>COVEREDRATIO</value>
      <minimum>0.80</minimum>
    </limit>
  </limits>
</rule>
```

### PR Coverage Checks

- Block merge if coverage drops
- Show coverage diff in PR comments
- Require coverage for changed files
- Allow exceptions with justification

---

## Trend Tracking

### Metrics to Track

| Metric | Purpose | Alert Threshold |
|--------|---------|-----------------|
| Overall line coverage | Baseline health | < 80% |
| Branch coverage | Logic completeness | < 70% |
| Coverage delta | Regression detection | < -2% per PR |
| Test execution time | Performance | > 5 min |
| Flaky test count | Reliability | > 0 |

### Coverage Services

| Service | Features | Integration |
|---------|----------|-------------|
| Codecov | PR comments, badges, graphs | GitHub, GitLab, Bitbucket |
| Coveralls | History, trends, badges | GitHub, GitLab |
| SonarCloud | Full code quality suite | Multiple CI platforms |

### Badge Generation

```markdown
<!-- README.md -->
[![codecov](https://codecov.io/gh/org/repo/branch/main/graph/badge.svg)](https://codecov.io/gh/org/repo)
```

FILE:references/framework-guide.md
# Testing Framework Guide

Language and framework selection, configuration, and patterns.

---

## Table of Contents

- [Framework Selection](#framework-selection)
- [TypeScript/JavaScript](#typescriptjavascript)
- [Python](#python)
- [Java](#java)
- [Version Requirements](#version-requirements)

---

## Framework Selection

| Language | Recommended | Alternatives | Best For |
|----------|-------------|--------------|----------|
| TypeScript/JS | Jest | Vitest, Mocha | React, Node.js, Next.js |
| Python | Pytest | unittest, nose2 | Django, Flask, FastAPI |
| Java | JUnit 5 | TestNG | Spring, Android |
| Vite projects | Vitest | Jest | Modern Vite-based apps |

---

## TypeScript/JavaScript

### Jest Configuration

```javascript
// jest.config.js
module.exports = {
  preset: 'ts-jest',
  testEnvironment: 'node',
  testMatch: ['**/*.test.ts'],
  collectCoverageFrom: ['src/**/*.ts'],
  coverageThreshold: {
    global: { branches: 80, lines: 80 }
  }
};
```

### Jest Test Pattern

```typescript
describe('Calculator', () => {
  let calc: Calculator;

  beforeEach(() => {
    calc = new Calculator();
  });

  it('should add two numbers', () => {
    expect(calc.add(2, 3)).toBe(5);
  });

  it('should throw on invalid input', () => {
    expect(() => calc.add(null, 3)).toThrow('Invalid input');
  });
});
```

### Vitest Configuration

```typescript
// vitest.config.ts
import { defineConfig } from 'vitest/config';

export default defineConfig({
  test: {
    globals: true,
    environment: 'node',
    coverage: { provider: 'c8' }
  }
});
```

### Coverage Tools
- Istanbul/nyc: Traditional coverage
- c8: Native V8 coverage (faster)
- Vitest built-in: Integrated with test runner

---

## Python

### Pytest Configuration

```ini
# pytest.ini
[pytest]
testpaths = tests
python_files = test_*.py
python_functions = test_*
addopts = --cov=src --cov-report=term-missing
```

### Pytest Test Pattern

```python
import pytest
from calculator import Calculator

class TestCalculator:
    @pytest.fixture
    def calc(self):
        return Calculator()

    def test_add_positive_numbers(self, calc):
        assert calc.add(2, 3) == 5

    def test_add_raises_on_invalid_input(self, calc):
        with pytest.raises(ValueError, match="Invalid input"):
            calc.add(None, 3)

    @pytest.mark.parametrize("a,b,expected", [
        (1, 2, 3),
        (-1, 1, 0),
        (0, 0, 0),
    ])
    def test_add_various_inputs(self, calc, a, b, expected):
        assert calc.add(a, b) == expected
```

### Coverage Tools
- coverage.py: Standard Python coverage
- pytest-cov: Pytest plugin wrapper
- Report formats: HTML, XML, LCOV

---

## Java

### JUnit 5 Configuration (Maven)

```xml
<!-- pom.xml -->
<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter</artifactId>
    <version>5.9.3</version>
    <scope>test</scope>
</dependency>
<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.8.10</version>
</plugin>
```

### JUnit 5 Test Pattern

```java
import org.junit.jupiter.api.*;
import static org.junit.jupiter.api.Assertions.*;

class CalculatorTest {
    private Calculator calc;

    @BeforeEach
    void setUp() {
        calc = new Calculator();
    }

    @Test
    @DisplayName("should add two positive numbers")
    void testAddPositive() {
        assertEquals(5, calc.add(2, 3));
    }

    @Test
    @DisplayName("should throw on null input")
    void testAddThrowsOnNull() {
        assertThrows(IllegalArgumentException.class,
            () -> calc.add(null, 3));
    }

    @ParameterizedTest
    @CsvSource({"1,2,3", "-1,1,0", "0,0,0"})
    void testAddVarious(int a, int b, int expected) {
        assertEquals(expected, calc.add(a, b));
    }
}
```

### Coverage Tools
- JaCoCo: Standard Java coverage
- Cobertura: Alternative XML format
- Report formats: HTML, XML, CSV

---

## Version Requirements

| Tool | Minimum Version | Notes |
|------|-----------------|-------|
| Node.js | 16+ | Required for Jest 29+ |
| Jest | 29+ | Modern async support |
| Vitest | 0.34+ | Stable API |
| Python | 3.8+ | f-strings, async support |
| Pytest | 7+ | Modern fixtures |
| Java | 11+ | JUnit 5 support |
| JUnit | 5.9+ | ParameterizedTest improvements |
| TypeScript | 4.5+ | Strict mode features |

FILE:references/tdd-best-practices.md
# TDD Best Practices

Guidelines for effective test-driven development workflows.

---

## Table of Contents

- [Red-Green-Refactor Cycle](#red-green-refactor-cycle)
- [Test Generation Guidelines](#test-generation-guidelines)
- [Test Quality Principles](#test-quality-principles)
- [Coverage Goals](#coverage-goals)

---

## Red-Green-Refactor Cycle

### RED Phase
1. Write a failing test before any implementation
2. Test should fail for the right reason (not compilation errors)
3. Name tests as specifications describing expected behavior
4. Keep tests small and focused on single behaviors

### GREEN Phase
1. Write minimal code to make the test pass
2. Avoid over-engineering at this stage
3. Duplicate code is acceptable temporarily
4. Focus on correctness, not elegance

### REFACTOR Phase
1. Improve code structure while keeping tests green
2. Remove duplication introduced in GREEN phase
3. Apply design patterns where appropriate
4. Run tests after each small refactoring

### Cycle Discipline
- Complete one cycle before starting the next
- Commit after each successful GREEN phase
- Small iterations lead to better designs
- Resist temptation to write implementation first

---

## Test Generation Guidelines

### Behavior Focus
- Test what code does, not how it does it
- Avoid coupling tests to implementation details
- Tests should survive internal refactoring
- Focus on observable outcomes

### Naming Conventions
- Use descriptive names that read as specifications
- Format: `should_<expected>_when_<condition>`
- Examples:
  - `should_return_zero_when_cart_is_empty`
  - `should_reject_negative_amounts`
  - `should_apply_discount_for_members`

### Test Structure
- Follow Arrange-Act-Assert (AAA) pattern
- Keep setup minimal and relevant
- One logical assertion per test
- Extract shared setup to fixtures

### Coverage Scope
- Happy path: Normal expected usage
- Error cases: Invalid inputs, failures
- Edge cases: Boundaries, empty states
- Exceptional cases: Timeouts, nulls

---

## Test Quality Principles

### Independence
- Each test runs in isolation
- No shared mutable state between tests
- Tests can run in any order
- Parallel execution should work

### Speed
- Unit tests under 100ms each
- Avoid I/O in unit tests
- Mock external dependencies
- Use in-memory databases for integration

### Determinism
- Same inputs produce same results
- No dependency on system time or random values
- Controlled test data
- No flaky tests allowed

### Clarity
- Failure messages explain what went wrong
- Test code is as clean as production code
- Avoid clever tricks that obscure intent
- Comments explain non-obvious setup

---

## Coverage Goals

### Thresholds by Type
| Type | Target | Rationale |
|------|--------|-----------|
| Line coverage | 80%+ | Baseline for most projects |
| Branch coverage | 70%+ | More meaningful than line |
| Function coverage | 90%+ | Public APIs should be tested |

### Critical Path Rules
- Authentication: 100% coverage required
- Payment processing: 100% coverage required
- Data validation: 100% coverage required
- Error handlers: Must test all paths

### Avoiding Coverage Theater
- High coverage != good tests
- Focus on meaningful assertions
- Test behaviors, not lines
- Code review test quality, not just metrics

### Coverage Analysis Workflow
1. Generate coverage report after test run
2. Identify uncovered critical paths (P0)
3. Review medium-priority gaps (P1)
4. Document accepted low-priority gaps (P2)
5. Set threshold gates in CI pipeline

FILE:scripts/coverage_analyzer.py
"""
Coverage analysis module.

Parse and analyze test coverage reports in multiple formats (LCOV, JSON, XML).
Identify gaps, calculate metrics, and provide actionable recommendations.
"""

from typing import Dict, List, Any, Optional, Tuple
import json
import xml.etree.ElementTree as ET


class CoverageFormat:
    """Supported coverage report formats."""
    LCOV = "lcov"
    JSON = "json"
    XML = "xml"
    COBERTURA = "cobertura"


class CoverageAnalyzer:
    """Analyze test coverage reports and identify gaps."""

    def __init__(self):
        """Initialize coverage analyzer."""
        self.coverage_data = {}
        self.gaps = []
        self.summary = {}

    def parse_coverage_report(
        self,
        report_content: str,
        format_type: str
    ) -> Dict[str, Any]:
        """
        Parse coverage report in various formats.

        Args:
            report_content: Raw coverage report content
            format_type: Format (lcov, json, xml, cobertura)

        Returns:
            Parsed coverage data
        """
        if format_type == CoverageFormat.LCOV:
            return self._parse_lcov(report_content)
        elif format_type == CoverageFormat.JSON:
            return self._parse_json(report_content)
        elif format_type in [CoverageFormat.XML, CoverageFormat.COBERTURA]:
            return self._parse_xml(report_content)
        else:
            raise ValueError(f"Unsupported format: {format_type}")

    def _parse_lcov(self, content: str) -> Dict[str, Any]:
        """Parse LCOV format coverage report."""
        files = {}
        current_file = None
        file_data = {}

        for line in content.split('\n'):
            line = line.strip()

            if line.startswith('SF:'):
                # Source file
                current_file = line[3:]
                file_data = {
                    'lines': {},
                    'functions': {},
                    'branches': {}
                }

            elif line.startswith('DA:'):
                # Line coverage data (line_number,hit_count)
                parts = line[3:].split(',')
                line_num = int(parts[0])
                hit_count = int(parts[1])
                file_data['lines'][line_num] = hit_count

            elif line.startswith('FNDA:'):
                # Function coverage (hit_count,function_name)
                parts = line[5:].split(',', 1)
                hit_count = int(parts[0])
                func_name = parts[1] if len(parts) > 1 else 'unknown'
                file_data['functions'][func_name] = hit_count

            elif line.startswith('BRDA:'):
                # Branch coverage (line,block,branch,hit_count)
                parts = line[5:].split(',')
                branch_id = f"{parts[0]}:{parts[1]}:{parts[2]}"
                hit_count = 0 if parts[3] == '-' else int(parts[3])
                file_data['branches'][branch_id] = hit_count

            elif line == 'end_of_record':
                if current_file:
                    files[current_file] = file_data
                current_file = None
                file_data = {}

        self.coverage_data = files
        return files

    def _parse_json(self, content: str) -> Dict[str, Any]:
        """Parse JSON format coverage report (Istanbul/nyc)."""
        try:
            data = json.loads(content)
            files = {}

            for file_path, file_data in data.items():
                lines = {}
                functions = {}
                branches = {}

                # Line coverage
                if 's' in file_data:  # Statement map
                    statement_map = file_data['s']
                    for stmt_id, hit_count in statement_map.items():
                        # Map statement to line number
                        if 'statementMap' in file_data:
                            stmt_info = file_data['statementMap'].get(stmt_id, {})
                            line_num = stmt_info.get('start', {}).get('line')
                            if line_num:
                                lines[line_num] = hit_count

                # Function coverage
                if 'f' in file_data:
                    func_map = file_data['f']
                    func_names = file_data.get('fnMap', {})
                    for func_id, hit_count in func_map.items():
                        func_info = func_names.get(func_id, {})
                        func_name = func_info.get('name', f'func_{func_id}')
                        functions[func_name] = hit_count

                # Branch coverage
                if 'b' in file_data:
                    branch_map = file_data['b']
                    for branch_id, locations in branch_map.items():
                        for idx, hit_count in enumerate(locations):
                            branch_key = f"{branch_id}:{idx}"
                            branches[branch_key] = hit_count

                files[file_path] = {
                    'lines': lines,
                    'functions': functions,
                    'branches': branches
                }

            self.coverage_data = files
            return files

        except json.JSONDecodeError as e:
            raise ValueError(f"Invalid JSON coverage report: {e}")

    def _parse_xml(self, content: str) -> Dict[str, Any]:
        """Parse XML/Cobertura format coverage report."""
        try:
            root = ET.fromstring(content)
            files = {}

            # Handle Cobertura format
            for package in root.findall('.//package'):
                for cls in package.findall('classes/class'):
                    filename = cls.get('filename', cls.get('name', 'unknown'))

                    lines = {}
                    branches = {}

                    for line in cls.findall('lines/line'):
                        line_num = int(line.get('number', 0))
                        hit_count = int(line.get('hits', 0))
                        lines[line_num] = hit_count

                        # Branch info
                        branch = line.get('branch', 'false')
                        if branch == 'true':
                            condition_coverage = line.get('condition-coverage', '0% (0/0)')
                            # Parse "(covered/total)"
                            if '(' in condition_coverage:
                                branch_info = condition_coverage.split('(')[1].split(')')[0]
                                covered, total = map(int, branch_info.split('/'))
                                branches[f"{line_num}:branch"] = covered

                    files[filename] = {
                        'lines': lines,
                        'functions': {},
                        'branches': branches
                    }

            self.coverage_data = files
            return files

        except ET.ParseError as e:
            raise ValueError(f"Invalid XML coverage report: {e}")

    def calculate_summary(self) -> Dict[str, Any]:
        """
        Calculate overall coverage summary.

        Returns:
            Summary with line, branch, and function coverage percentages
        """
        total_lines = 0
        covered_lines = 0
        total_branches = 0
        covered_branches = 0
        total_functions = 0
        covered_functions = 0

        for file_path, file_data in self.coverage_data.items():
            # Lines
            for line_num, hit_count in file_data.get('lines', {}).items():
                total_lines += 1
                if hit_count > 0:
                    covered_lines += 1

            # Branches
            for branch_id, hit_count in file_data.get('branches', {}).items():
                total_branches += 1
                if hit_count > 0:
                    covered_branches += 1

            # Functions
            for func_name, hit_count in file_data.get('functions', {}).items():
                total_functions += 1
                if hit_count > 0:
                    covered_functions += 1

        summary = {
            'line_coverage': self._safe_percentage(covered_lines, total_lines),
            'branch_coverage': self._safe_percentage(covered_branches, total_branches),
            'function_coverage': self._safe_percentage(covered_functions, total_functions),
            'total_lines': total_lines,
            'covered_lines': covered_lines,
            'total_branches': total_branches,
            'covered_branches': covered_branches,
            'total_functions': total_functions,
            'covered_functions': covered_functions
        }

        self.summary = summary
        return summary

    def _safe_percentage(self, covered: int, total: int) -> float:
        """Safely calculate percentage."""
        if total == 0:
            return 0.0
        return round((covered / total) * 100, 2)

    def identify_gaps(self, threshold: float = 80.0) -> List[Dict[str, Any]]:
        """
        Identify coverage gaps below threshold.

        Args:
            threshold: Minimum acceptable coverage percentage

        Returns:
            List of files with coverage gaps
        """
        gaps = []

        for file_path, file_data in self.coverage_data.items():
            file_gaps = self._analyze_file_gaps(file_path, file_data, threshold)
            if file_gaps:
                gaps.append(file_gaps)

        self.gaps = gaps
        return gaps

    def _analyze_file_gaps(
        self,
        file_path: str,
        file_data: Dict[str, Any],
        threshold: float
    ) -> Optional[Dict[str, Any]]:
        """Analyze coverage gaps for a single file."""
        lines = file_data.get('lines', {})
        branches = file_data.get('branches', {})
        functions = file_data.get('functions', {})

        # Calculate file coverage
        total_lines = len(lines)
        covered_lines = sum(1 for hit in lines.values() if hit > 0)
        line_coverage = self._safe_percentage(covered_lines, total_lines)

        total_branches = len(branches)
        covered_branches = sum(1 for hit in branches.values() if hit > 0)
        branch_coverage = self._safe_percentage(covered_branches, total_branches)

        # Find uncovered lines
        uncovered_lines = [line_num for line_num, hit in lines.items() if hit == 0]
        uncovered_branches = [branch_id for branch_id, hit in branches.items() if hit == 0]

        # Only report if below threshold
        if line_coverage < threshold or branch_coverage < threshold:
            return {
                'file': file_path,
                'line_coverage': line_coverage,
                'branch_coverage': branch_coverage,
                'uncovered_lines': sorted(uncovered_lines),
                'uncovered_branches': uncovered_branches,
                'priority': self._calculate_priority(line_coverage, branch_coverage, threshold)
            }

        return None

    def _calculate_priority(
        self,
        line_coverage: float,
        branch_coverage: float,
        threshold: float
    ) -> str:
        """Calculate priority based on coverage gap severity."""
        gap = threshold - min(line_coverage, branch_coverage)

        if gap >= 40:
            return 'P0'  # Critical - less than 40% coverage
        elif gap >= 20:
            return 'P1'  # Important - 60-80% coverage
        else:
            return 'P2'  # Nice to have - 80%+ coverage

    def get_file_coverage(self, file_path: str) -> Dict[str, Any]:
        """
        Get detailed coverage information for a specific file.

        Args:
            file_path: Path to file

        Returns:
            Detailed coverage data for file
        """
        if file_path not in self.coverage_data:
            return {}

        file_data = self.coverage_data[file_path]
        lines = file_data.get('lines', {})
        branches = file_data.get('branches', {})
        functions = file_data.get('functions', {})

        total_lines = len(lines)
        covered_lines = sum(1 for hit in lines.values() if hit > 0)

        total_branches = len(branches)
        covered_branches = sum(1 for hit in branches.values() if hit > 0)

        total_functions = len(functions)
        covered_functions = sum(1 for hit in functions.values() if hit > 0)

        return {
            'file': file_path,
            'line_coverage': self._safe_percentage(covered_lines, total_lines),
            'branch_coverage': self._safe_percentage(covered_branches, total_branches),
            'function_coverage': self._safe_percentage(covered_functions, total_functions),
            'lines': lines,
            'branches': branches,
            'functions': functions
        }

    def generate_recommendations(self) -> List[Dict[str, Any]]:
        """
        Generate prioritized recommendations for improving coverage.

        Returns:
            List of recommendations with priority and actions
        """
        recommendations = []

        # Check overall coverage
        summary = self.summary or self.calculate_summary()

        if summary['line_coverage'] < 80:
            recommendations.append({
                'priority': 'P0',
                'type': 'overall_coverage',
                'message': f"Overall line coverage ({summary['line_coverage']}%) is below 80% threshold",
                'action': 'Focus on adding tests for critical paths and business logic',
                'impact': 'high'
            })

        if summary['branch_coverage'] < 70:
            recommendations.append({
                'priority': 'P0',
                'type': 'branch_coverage',
                'message': f"Branch coverage ({summary['branch_coverage']}%) is below 70% threshold",
                'action': 'Add tests for conditional logic and error handling paths',
                'impact': 'high'
            })

        # File-specific recommendations
        for gap in self.gaps:
            if gap['priority'] == 'P0':
                recommendations.append({
                    'priority': 'P0',
                    'type': 'file_coverage',
                    'file': gap['file'],
                    'message': f"Critical coverage gap in {gap['file']}",
                    'action': f"Add tests for lines: {gap['uncovered_lines'][:10]}",
                    'impact': 'high'
                })

        # Sort by priority
        priority_order = {'P0': 0, 'P1': 1, 'P2': 2}
        recommendations.sort(key=lambda x: priority_order.get(x['priority'], 3))

        return recommendations

    def detect_format(self, content: str) -> str:
        """
        Automatically detect coverage report format.

        Args:
            content: Raw coverage report content

        Returns:
            Detected format (lcov, json, xml)
        """
        content_stripped = content.strip()

        # Check for LCOV format
        if content_stripped.startswith('TN:') or 'SF:' in content_stripped[:100]:
            return CoverageFormat.LCOV

        # Check for JSON format
        if content_stripped.startswith('{') or content_stripped.startswith('['):
            try:
                json.loads(content_stripped)
                return CoverageFormat.JSON
            except:
                pass

        # Check for XML format
        if content_stripped.startswith('<?xml') or content_stripped.startswith('<coverage'):
            return CoverageFormat.XML

        raise ValueError("Unable to detect coverage report format")

FILE:scripts/fixture_generator.py
"""
Fixture and test data generation module.

Generates realistic test data, mock objects, and fixtures for various scenarios.
"""

from typing import Dict, List, Any, Optional
import json
import random


class FixtureGenerator:
    """Generate test fixtures and mock data."""

    def __init__(self, seed: Optional[int] = None):
        """
        Initialize fixture generator.

        Args:
            seed: Random seed for reproducible fixtures
        """
        if seed is not None:
            random.seed(seed)

    def generate_boundary_values(
        self,
        data_type: str,
        constraints: Optional[Dict[str, Any]] = None
    ) -> List[Any]:
        """
        Generate boundary values for testing.

        Args:
            data_type: Type of data (int, string, array, date, etc.)
            constraints: Constraints like min, max, length

        Returns:
            List of boundary values
        """
        constraints = constraints or {}

        if data_type == "int":
            return self._integer_boundaries(constraints)
        elif data_type == "string":
            return self._string_boundaries(constraints)
        elif data_type == "array":
            return self._array_boundaries(constraints)
        elif data_type == "date":
            return self._date_boundaries(constraints)
        elif data_type == "email":
            return self._email_boundaries()
        elif data_type == "url":
            return self._url_boundaries()
        else:
            return []

    def _integer_boundaries(self, constraints: Dict[str, Any]) -> List[int]:
        """Generate integer boundary values."""
        min_val = constraints.get('min', 0)
        max_val = constraints.get('max', 100)

        boundaries = [
            min_val,  # Minimum
            min_val + 1,  # Just above minimum
            max_val - 1,  # Just below maximum
            max_val,  # Maximum
        ]

        # Add special values
        if min_val <= 0 <= max_val:
            boundaries.append(0)  # Zero
        if min_val < 0:
            boundaries.append(-1)  # Negative

        return sorted(set(boundaries))

    def _string_boundaries(self, constraints: Dict[str, Any]) -> List[str]:
        """Generate string boundary values."""
        min_len = constraints.get('min_length', 0)
        max_len = constraints.get('max_length', 100)

        boundaries = [
            "",  # Empty string
            "a" * min_len,  # Minimum length
            "a" * (min_len + 1) if min_len < max_len else "",  # Just above minimum
            "a" * (max_len - 1) if max_len > 1 else "a",  # Just below maximum
            "a" * max_len,  # Maximum length
            "a" * (max_len + 1),  # Exceeds maximum (invalid)
        ]

        # Add special characters
        if max_len >= 10:
            boundaries.append("test@#$%^&*()")  # Special characters
            boundaries.append("unicode: 你好")  # Unicode

        return [b for b in boundaries if b is not None]

    def _array_boundaries(self, constraints: Dict[str, Any]) -> List[List[Any]]:
        """Generate array boundary values."""
        min_size = constraints.get('min_size', 0)
        max_size = constraints.get('max_size', 10)

        boundaries = [
            [],  # Empty array
            [1] * min_size,  # Minimum size
            [1] * max_size,  # Maximum size
            [1] * (max_size + 1),  # Exceeds maximum (invalid)
        ]

        return boundaries

    def _date_boundaries(self, constraints: Dict[str, Any]) -> List[str]:
        """Generate date boundary values."""
        return [
            "1900-01-01",  # Very old date
            "1970-01-01",  # Unix epoch
            "2000-01-01",  # Y2K
            "2025-11-05",  # Today (example)
            "2099-12-31",  # Far future
            "invalid-date",  # Invalid format
        ]

    def _email_boundaries(self) -> List[str]:
        """Generate email boundary values."""
        return [
            "valid@example.com",  # Valid
            "user.name+tag@example.co.uk",  # Valid with special chars
            "invalid",  # Missing @
            "@example.com",  # Missing local part
            "user@",  # Missing domain
            "user@.com",  # Invalid domain
            "",  # Empty
        ]

    def _url_boundaries(self) -> List[str]:
        """Generate URL boundary values."""
        return [
            "https://example.com",  # Valid HTTPS
            "http://example.com",  # Valid HTTP
            "ftp://example.com",  # Different protocol
            "//example.com",  # Protocol-relative
            "example.com",  # Missing protocol
            "",  # Empty
            "not a url",  # Invalid
        ]

    def generate_edge_cases(
        self,
        scenario: str,
        context: Optional[Dict[str, Any]] = None
    ) -> List[Dict[str, Any]]:
        """
        Generate edge case test scenarios.

        Args:
            scenario: Type of scenario (auth, payment, form, api, etc.)
            context: Additional context for scenario

        Returns:
            List of edge case test scenarios
        """
        if scenario == "auth":
            return self._auth_edge_cases()
        elif scenario == "payment":
            return self._payment_edge_cases()
        elif scenario == "form":
            return self._form_edge_cases(context or {})
        elif scenario == "api":
            return self._api_edge_cases()
        elif scenario == "file_upload":
            return self._file_upload_edge_cases()
        else:
            return []

    def _auth_edge_cases(self) -> List[Dict[str, Any]]:
        """Generate authentication edge cases."""
        return [
            {
                'name': 'empty_credentials',
                'input': {'username': '', 'password': ''},
                'expected': 'validation_error'
            },
            {
                'name': 'sql_injection_attempt',
                'input': {'username': "admin' OR '1'='1", 'password': 'password'},
                'expected': 'authentication_failed'
            },
            {
                'name': 'very_long_password',
                'input': {'username': 'user', 'password': 'a' * 1000},
                'expected': 'validation_error_or_success'
            },
            {
                'name': 'special_chars_username',
                'input': {'username': 'user@#$%', 'password': 'password'},
                'expected': 'depends_on_validation'
            },
            {
                'name': 'unicode_credentials',
                'input': {'username': '用户', 'password': 'пароль'},
                'expected': 'should_handle_unicode'
            }
        ]

    def _payment_edge_cases(self) -> List[Dict[str, Any]]:
        """Generate payment processing edge cases."""
        return [
            {
                'name': 'zero_amount',
                'input': {'amount': 0, 'currency': 'USD'},
                'expected': 'validation_error'
            },
            {
                'name': 'negative_amount',
                'input': {'amount': -10, 'currency': 'USD'},
                'expected': 'validation_error'
            },
            {
                'name': 'very_large_amount',
                'input': {'amount': 999999999.99, 'currency': 'USD'},
                'expected': 'should_handle_or_reject'
            },
            {
                'name': 'precision_test',
                'input': {'amount': 10.999, 'currency': 'USD'},
                'expected': 'should_round_to_10.99'
            },
            {
                'name': 'invalid_currency',
                'input': {'amount': 10, 'currency': 'XXX'},
                'expected': 'validation_error'
            }
        ]

    def _form_edge_cases(self, context: Dict[str, Any]) -> List[Dict[str, Any]]:
        """Generate form validation edge cases."""
        fields = context.get('fields', [])
        edge_cases = []

        for field in fields:
            field_name = field.get('name', 'field')
            field_type = field.get('type', 'text')

            edge_cases.append({
                'name': f'{field_name}_empty',
                'input': {field_name: ''},
                'expected': 'validation_error_if_required'
            })

            if field_type in ['text', 'email', 'password']:
                edge_cases.append({
                    'name': f'{field_name}_very_long',
                    'input': {field_name: 'a' * 1000},
                    'expected': 'validation_error_or_truncate'
                })

        return edge_cases

    def _api_edge_cases(self) -> List[Dict[str, Any]]:
        """Generate API edge cases."""
        return [
            {
                'name': 'missing_required_field',
                'request': {'optional_field': 'value'},
                'expected': 400
            },
            {
                'name': 'invalid_json',
                'request': 'not valid json{',
                'expected': 400
            },
            {
                'name': 'empty_body',
                'request': {},
                'expected': 400
            },
            {
                'name': 'very_large_payload',
                'request': {'data': 'x' * 1000000},
                'expected': '413_or_400'
            },
            {
                'name': 'invalid_method',
                'method': 'INVALID',
                'expected': 405
            }
        ]

    def _file_upload_edge_cases(self) -> List[Dict[str, Any]]:
        """Generate file upload edge cases."""
        return [
            {
                'name': 'empty_file',
                'file': {'name': 'test.txt', 'size': 0},
                'expected': 'validation_error'
            },
            {
                'name': 'very_large_file',
                'file': {'name': 'test.txt', 'size': 1000000000},
                'expected': 'size_limit_error'
            },
            {
                'name': 'invalid_extension',
                'file': {'name': 'test.exe', 'size': 1000},
                'expected': 'validation_error'
            },
            {
                'name': 'no_extension',
                'file': {'name': 'testfile', 'size': 1000},
                'expected': 'depends_on_validation'
            },
            {
                'name': 'special_chars_filename',
                'file': {'name': 'test@#$%.txt', 'size': 1000},
                'expected': 'should_sanitize'
            }
        ]

    def generate_mock_data(
        self,
        schema: Dict[str, Any],
        count: int = 1
    ) -> List[Dict[str, Any]]:
        """
        Generate mock data based on schema.

        Args:
            schema: Schema definition with field types
            count: Number of mock objects to generate

        Returns:
            List of mock data objects
        """
        mock_objects = []

        for _ in range(count):
            mock_obj = {}

            for field_name, field_def in schema.items():
                field_type = field_def.get('type', 'string')
                mock_obj[field_name] = self._generate_field_value(field_type, field_def)

            mock_objects.append(mock_obj)

        return mock_objects

    def _generate_field_value(self, field_type: str, field_def: Dict[str, Any]) -> Any:
        """Generate value for a single field."""
        if field_type == "string":
            options = field_def.get('options')
            if options:
                return random.choice(options)
            return f"test_string_{random.randint(1, 1000)}"

        elif field_type == "int":
            min_val = field_def.get('min', 0)
            max_val = field_def.get('max', 100)
            return random.randint(min_val, max_val)

        elif field_type == "float":
            min_val = field_def.get('min', 0.0)
            max_val = field_def.get('max', 100.0)
            return round(random.uniform(min_val, max_val), 2)

        elif field_type == "bool":
            return random.choice([True, False])

        elif field_type == "email":
            return f"user{random.randint(1, 1000)}@example.com"

        elif field_type == "date":
            return f"2025-{random.randint(1, 12):02d}-{random.randint(1, 28):02d}"

        elif field_type == "array":
            item_type = field_def.get('items', {}).get('type', 'string')
            size = random.randint(1, 5)
            return [self._generate_field_value(item_type, field_def.get('items', {}))
                   for _ in range(size)]

        else:
            return None

    def generate_fixture_file(
        self,
        fixture_name: str,
        data: Any,
        format: str = "json"
    ) -> str:
        """
        Generate fixture file content.

        Args:
            fixture_name: Name of fixture
            data: Fixture data
            format: Output format (json, yaml, python)

        Returns:
            Fixture file content as string
        """
        if format == "json":
            return json.dumps(data, indent=2)

        elif format == "python":
            return f"""# {fixture_name} fixture

{fixture_name.upper()} = {repr(data)}
"""

        elif format == "yaml":
            # Simple YAML generation (for basic structures)
            return self._dict_to_yaml(data)

        else:
            return str(data)

    def _dict_to_yaml(self, data: Any, indent: int = 0) -> str:
        """Simple YAML generator."""
        lines = []
        indent_str = "  " * indent

        if isinstance(data, dict):
            for key, value in data.items():
                if isinstance(value, (dict, list)):
                    lines.append(f"{indent_str}{key}:")
                    lines.append(self._dict_to_yaml(value, indent + 1))
                else:
                    lines.append(f"{indent_str}{key}: {value}")

        elif isinstance(data, list):
            for item in data:
                if isinstance(item, dict):
                    lines.append(f"{indent_str}-")
                    lines.append(self._dict_to_yaml(item, indent + 1))
                else:
                    lines.append(f"{indent_str}- {item}")

        else:
            return str(data)

        return "\n".join(lines)

FILE:scripts/format_detector.py
"""
Format detection module.

Automatically detects programming language, testing framework, and file formats.
"""

from typing import Dict, List, Any, Optional, Tuple
import re


class FormatDetector:
    """Detect language, framework, and file formats automatically."""

    def __init__(self):
        """Initialize format detector."""
        self.detected_language = None
        self.detected_framework = None

    def detect_language(self, code: str) -> str:
        """
        Detect programming language from code.

        Args:
            code: Source code

        Returns:
            Detected language (typescript, javascript, python, java, unknown)
        """
        # TypeScript patterns
        if self._is_typescript(code):
            self.detected_language = "typescript"
            return "typescript"

        # JavaScript patterns
        if self._is_javascript(code):
            self.detected_language = "javascript"
            return "javascript"

        # Python patterns
        if self._is_python(code):
            self.detected_language = "python"
            return "python"

        # Java patterns
        if self._is_java(code):
            self.detected_language = "java"
            return "java"

        self.detected_language = "unknown"
        return "unknown"

    def _is_typescript(self, code: str) -> bool:
        """Check if code is TypeScript."""
        ts_patterns = [
            r'\binterface\s+\w+',  # interface definitions
            r':\s*\w+\s*[=;]',  # type annotations
            r'\btype\s+\w+\s*=',  # type aliases
            r'<\w+>',  # generic types
            r'import.*from.*[\'"]',  # ES6 imports with types
        ]

        # Must have multiple TypeScript-specific patterns
        matches = sum(1 for pattern in ts_patterns if re.search(pattern, code))
        return matches >= 2

    def _is_javascript(self, code: str) -> bool:
        """Check if code is JavaScript."""
        js_patterns = [
            r'\bconst\s+\w+',  # const declarations
            r'\blet\s+\w+',  # let declarations
            r'=>',  # arrow functions
            r'function\s+\w+',  # function declarations
            r'require\([\'"]',  # CommonJS require
        ]

        matches = sum(1 for pattern in js_patterns if re.search(pattern, code))
        return matches >= 2

    def _is_python(self, code: str) -> bool:
        """Check if code is Python."""
        py_patterns = [
            r'\bdef\s+\w+',  # function definitions
            r'\bclass\s+\w+',  # class definitions
            r'import\s+\w+',  # import statements
            r'from\s+\w+\s+import',  # from imports
            r'^\s*#.*$',  # Python comments
            r':\s*$',  # Python colons
        ]

        matches = sum(1 for pattern in py_patterns if re.search(pattern, code, re.MULTILINE))
        return matches >= 3

    def _is_java(self, code: str) -> bool:
        """Check if code is Java."""
        java_patterns = [
            r'\bpublic\s+class',  # public class
            r'\bprivate\s+\w+',  # private members
            r'\bpublic\s+\w+\s+\w+\s*\(',  # public methods
            r'import\s+java\.',  # Java imports
            r'\bvoid\s+\w+\s*\(',  # void methods
        ]

        matches = sum(1 for pattern in java_patterns if re.search(pattern, code))
        return matches >= 2

    def detect_test_framework(self, code: str) -> str:
        """
        Detect testing framework from test code.

        Args:
            code: Test code

        Returns:
            Detected framework (jest, vitest, pytest, junit, mocha, unknown)
        """
        # Jest patterns
        if 'from \'@jest/globals\'' in code or '@jest/' in code:
            self.detected_framework = "jest"
            return "jest"

        # Vitest patterns
        if 'from \'vitest\'' in code or 'import { vi }' in code:
            self.detected_framework = "vitest"
            return "vitest"

        # Pytest patterns
        if 'import pytest' in code or 'def test_' in code:
            self.detected_framework = "pytest"
            return "pytest"

        # Unittest patterns
        if 'import unittest' in code and 'unittest.TestCase' in code:
            self.detected_framework = "unittest"
            return "unittest"

        # JUnit patterns
        if '@Test' in code and 'import org.junit' in code:
            self.detected_framework = "junit"
            return "junit"

        # Mocha patterns
        if 'describe(' in code and 'it(' in code:
            self.detected_framework = "mocha"
            return "mocha"

        self.detected_framework = "unknown"
        return "unknown"

    def detect_coverage_format(self, content: str) -> str:
        """
        Detect coverage report format.

        Args:
            content: Coverage report content

        Returns:
            Format type (lcov, json, xml, unknown)
        """
        content_stripped = content.strip()

        # LCOV format
        if content_stripped.startswith('TN:') or 'SF:' in content_stripped[:200]:
            return "lcov"

        # JSON format
        if content_stripped.startswith('{'):
            try:
                import json
                json.loads(content_stripped)
                return "json"
            except:
                pass

        # XML format
        if content_stripped.startswith('<?xml') or content_stripped.startswith('<coverage'):
            return "xml"

        return "unknown"

    def detect_input_format(self, input_data: str) -> Dict[str, Any]:
        """
        Detect input format and extract relevant information.

        Args:
            input_data: Input data (could be code, coverage report, etc.)

        Returns:
            Detection results with format, language, framework
        """
        result = {
            'format': 'unknown',
            'language': 'unknown',
            'framework': 'unknown',
            'content_type': 'unknown'
        }

        # Detect if it's a coverage report
        coverage_format = self.detect_coverage_format(input_data)
        if coverage_format != "unknown":
            result['format'] = coverage_format
            result['content_type'] = 'coverage_report'
            return result

        # Detect if it's source code
        language = self.detect_language(input_data)
        if language != "unknown":
            result['language'] = language
            result['content_type'] = 'source_code'

        # Detect if it's test code
        framework = self.detect_test_framework(input_data)
        if framework != "unknown":
            result['framework'] = framework
            result['content_type'] = 'test_code'

        return result

    def extract_file_info(self, file_path: str) -> Dict[str, str]:
        """
        Extract information from file path.

        Args:
            file_path: Path to file

        Returns:
            File information (extension, likely language, likely purpose)
        """
        import os

        file_name = os.path.basename(file_path)
        file_ext = os.path.splitext(file_name)[1].lower()

        # Extension to language mapping
        ext_to_lang = {
            '.ts': 'typescript',
            '.tsx': 'typescript',
            '.js': 'javascript',
            '.jsx': 'javascript',
            '.py': 'python',
            '.java': 'java',
            '.kt': 'kotlin',
            '.go': 'go',
            '.rs': 'rust',
        }

        # Test file patterns
        is_test = any(pattern in file_name.lower()
                     for pattern in ['test', 'spec', '_test.', '.test.'])

        return {
            'file_name': file_name,
            'extension': file_ext,
            'language': ext_to_lang.get(file_ext, 'unknown'),
            'is_test': is_test,
            'purpose': 'test' if is_test else 'source'
        }

    def suggest_test_file_name(self, source_file: str, framework: str) -> str:
        """
        Suggest test file name for source file.

        Args:
            source_file: Source file path
            framework: Testing framework

        Returns:
            Suggested test file name
        """
        import os

        base_name = os.path.splitext(os.path.basename(source_file))[0]
        ext = os.path.splitext(source_file)[1]

        if framework in ['jest', 'vitest', 'mocha']:
            return f"{base_name}.test{ext}"
        elif framework in ['pytest', 'unittest']:
            return f"test_{base_name}.py"
        elif framework in ['junit', 'testng']:
            return f"{base_name.capitalize()}Test.java"
        else:
            return f"{base_name}_test{ext}"

    def identify_test_patterns(self, code: str) -> List[str]:
        """
        Identify test patterns in code.

        Args:
            code: Test code

        Returns:
            List of identified patterns (AAA, Given-When-Then, etc.)
        """
        patterns = []

        # Arrange-Act-Assert pattern
        if any(comment in code.lower() for comment in ['// arrange', '# arrange', '// act', '# act']):
            patterns.append('AAA (Arrange-Act-Assert)')

        # Given-When-Then pattern
        if any(comment in code.lower() for comment in ['given', 'when', 'then']):
            patterns.append('Given-When-Then')

        # Setup/Teardown pattern
        if any(keyword in code for keyword in ['beforeEach', 'afterEach', 'setUp', 'tearDown']):
            patterns.append('Setup-Teardown')

        # Mocking pattern
        if any(keyword in code.lower() for keyword in ['mock', 'stub', 'spy']):
            patterns.append('Mocking/Stubbing')

        # Parameterized tests
        if any(keyword in code for keyword in ['@pytest.mark.parametrize', 'test.each', '@ParameterizedTest']):
            patterns.append('Parameterized Tests')

        return patterns if patterns else ['No specific pattern detected']

    def analyze_project_structure(self, file_paths: List[str]) -> Dict[str, Any]:
        """
        Analyze project structure from file paths.

        Args:
            file_paths: List of file paths in project

        Returns:
            Project structure analysis
        """
        languages = {}
        test_frameworks = []
        source_files = []
        test_files = []

        for file_path in file_paths:
            file_info = self.extract_file_info(file_path)

            # Count languages
            lang = file_info['language']
            if lang != 'unknown':
                languages[lang] = languages.get(lang, 0) + 1

            # Categorize files
            if file_info['is_test']:
                test_files.append(file_path)
            else:
                source_files.append(file_path)

        # Determine primary language
        primary_language = max(languages.items(), key=lambda x: x[1])[0] if languages else 'unknown'

        return {
            'primary_language': primary_language,
            'languages': languages,
            'source_file_count': len(source_files),
            'test_file_count': len(test_files),
            'test_ratio': len(test_files) / len(source_files) if source_files else 0,
            'suggested_framework': self._suggest_framework(primary_language)
        }

    def _suggest_framework(self, language: str) -> str:
        """Suggest testing framework based on language."""
        framework_map = {
            'typescript': 'jest or vitest',
            'javascript': 'jest or mocha',
            'python': 'pytest',
            'java': 'junit',
            'kotlin': 'junit',
            'go': 'testing package',
            'rust': 'cargo test',
        }

        return framework_map.get(language, 'unknown')

    def detect_environment(self) -> Dict[str, str]:
        """
        Detect execution environment (CLI, Desktop, API).

        Returns:
            Environment information
        """
        # This is a placeholder - actual detection would use environment variables
        # or other runtime checks
        return {
            'environment': 'cli',  # Could be 'desktop', 'api'
            'output_preference': 'terminal-friendly'  # Could be 'rich-markdown', 'json'
        }

FILE:scripts/framework_adapter.py
"""
Framework adapter module.

Provides multi-framework support with adapters for Jest, Pytest, JUnit, Vitest, and more.
Handles framework-specific patterns, imports, and test structure.
"""

from typing import Dict, List, Any, Optional
from enum import Enum


class Framework(Enum):
    """Supported testing frameworks."""
    JEST = "jest"
    VITEST = "vitest"
    PYTEST = "pytest"
    UNITTEST = "unittest"
    JUNIT = "junit"
    TESTNG = "testng"
    MOCHA = "mocha"
    JASMINE = "jasmine"


class Language(Enum):
    """Supported programming languages."""
    TYPESCRIPT = "typescript"
    JAVASCRIPT = "javascript"
    PYTHON = "python"
    JAVA = "java"


class FrameworkAdapter:
    """Adapter for multiple testing frameworks."""

    def __init__(self, framework: Framework, language: Language):
        """
        Initialize framework adapter.

        Args:
            framework: Testing framework
            language: Programming language
        """
        self.framework = framework
        self.language = language

    def generate_imports(self) -> str:
        """Generate framework-specific imports."""
        if self.framework == Framework.JEST:
            return self._jest_imports()
        elif self.framework == Framework.VITEST:
            return self._vitest_imports()
        elif self.framework == Framework.PYTEST:
            return self._pytest_imports()
        elif self.framework == Framework.UNITTEST:
            return self._unittest_imports()
        elif self.framework == Framework.JUNIT:
            return self._junit_imports()
        elif self.framework == Framework.TESTNG:
            return self._testng_imports()
        elif self.framework == Framework.MOCHA:
            return self._mocha_imports()
        else:
            return ""

    def _jest_imports(self) -> str:
        """Generate Jest imports."""
        return """import { describe, it, expect, beforeEach, afterEach } from '@jest/globals';"""

    def _vitest_imports(self) -> str:
        """Generate Vitest imports."""
        return """import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';"""

    def _pytest_imports(self) -> str:
        """Generate Pytest imports."""
        return """import pytest"""

    def _unittest_imports(self) -> str:
        """Generate unittest imports."""
        return """import unittest"""

    def _junit_imports(self) -> str:
        """Generate JUnit imports."""
        return """import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.AfterEach;
import static org.junit.jupiter.api.Assertions.*;"""

    def _testng_imports(self) -> str:
        """Generate TestNG imports."""
        return """import org.testng.annotations.Test;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.AfterMethod;
import static org.testng.Assert.*;"""

    def _mocha_imports(self) -> str:
        """Generate Mocha imports."""
        return """import { describe, it, beforeEach, afterEach } from 'mocha';
import { expect } from 'chai';"""

    def generate_test_suite_wrapper(
        self,
        suite_name: str,
        test_content: str
    ) -> str:
        """
        Wrap test content in framework-specific suite structure.

        Args:
            suite_name: Name of test suite
            test_content: Test functions/methods

        Returns:
            Complete test suite code
        """
        if self.framework in [Framework.JEST, Framework.VITEST, Framework.MOCHA]:
            return f"""describe('{suite_name}', () => {{
{self._indent(test_content, 2)}
}});"""

        elif self.framework == Framework.PYTEST:
            return f"""class Test{self._to_class_name(suite_name)}:
    \"\"\"Test suite for {suite_name}.\"\"\"

{self._indent(test_content, 4)}"""

        elif self.framework == Framework.UNITTEST:
            return f"""class Test{self._to_class_name(suite_name)}(unittest.TestCase):
    \"\"\"Test suite for {suite_name}.\"\"\"

{self._indent(test_content, 4)}"""

        elif self.framework in [Framework.JUNIT, Framework.TESTNG]:
            return f"""public class {self._to_class_name(suite_name)}Test {{

{self._indent(test_content, 4)}
}}"""

        return test_content

    def generate_test_function(
        self,
        test_name: str,
        test_body: str,
        description: str = ""
    ) -> str:
        """
        Generate framework-specific test function.

        Args:
            test_name: Name of test
            test_body: Test body code
            description: Test description

        Returns:
            Complete test function
        """
        if self.framework == Framework.JEST:
            return self._jest_test(test_name, test_body, description)
        elif self.framework == Framework.VITEST:
            return self._vitest_test(test_name, test_body, description)
        elif self.framework == Framework.PYTEST:
            return self._pytest_test(test_name, test_body, description)
        elif self.framework == Framework.UNITTEST:
            return self._unittest_test(test_name, test_body, description)
        elif self.framework == Framework.JUNIT:
            return self._junit_test(test_name, test_body, description)
        elif self.framework == Framework.TESTNG:
            return self._testng_test(test_name, test_body, description)
        elif self.framework == Framework.MOCHA:
            return self._mocha_test(test_name, test_body, description)
        else:
            return ""

    def _jest_test(self, test_name: str, test_body: str, description: str) -> str:
        """Generate Jest test."""
        return f"""it('{test_name}', () => {{
  // {description}
{self._indent(test_body, 2)}
}});"""

    def _vitest_test(self, test_name: str, test_body: str, description: str) -> str:
        """Generate Vitest test."""
        return f"""it('{test_name}', () => {{
  // {description}
{self._indent(test_body, 2)}
}});"""

    def _pytest_test(self, test_name: str, test_body: str, description: str) -> str:
        """Generate Pytest test."""
        func_name = test_name.replace(' ', '_').replace('-', '_')
        return f"""def test_{func_name}(self):
    \"\"\"
    {description or test_name}
    \"\"\"
{self._indent(test_body, 4)}"""

    def _unittest_test(self, test_name: str, test_body: str, description: str) -> str:
        """Generate unittest test."""
        func_name = self._to_camel_case(test_name)
        return f"""def test_{func_name}(self):
    \"\"\"
    {description or test_name}
    \"\"\"
{self._indent(test_body, 4)}"""

    def _junit_test(self, test_name: str, test_body: str, description: str) -> str:
        """Generate JUnit test."""
        method_name = self._to_camel_case(test_name)
        return f"""@Test
public void test{method_name}() {{
    // {description}
{self._indent(test_body, 4)}
}}"""

    def _testng_test(self, test_name: str, test_body: str, description: str) -> str:
        """Generate TestNG test."""
        method_name = self._to_camel_case(test_name)
        return f"""@Test
public void test{method_name}() {{
    // {description}
{self._indent(test_body, 4)}
}}"""

    def _mocha_test(self, test_name: str, test_body: str, description: str) -> str:
        """Generate Mocha test."""
        return f"""it('{test_name}', () => {{
  // {description}
{self._indent(test_body, 2)}
}});"""

    def generate_assertion(
        self,
        actual: str,
        expected: str,
        assertion_type: str = "equals"
    ) -> str:
        """
        Generate framework-specific assertion.

        Args:
            actual: Actual value expression
            expected: Expected value expression
            assertion_type: Type of assertion (equals, not_equals, true, false, throws)

        Returns:
            Assertion statement
        """
        if self.framework in [Framework.JEST, Framework.VITEST]:
            return self._jest_assertion(actual, expected, assertion_type)
        elif self.framework in [Framework.PYTEST, Framework.UNITTEST]:
            return self._python_assertion(actual, expected, assertion_type)
        elif self.framework in [Framework.JUNIT, Framework.TESTNG]:
            return self._java_assertion(actual, expected, assertion_type)
        elif self.framework == Framework.MOCHA:
            return self._chai_assertion(actual, expected, assertion_type)
        else:
            return f"assert {actual} == {expected}"

    def _jest_assertion(self, actual: str, expected: str, assertion_type: str) -> str:
        """Generate Jest assertion."""
        if assertion_type == "equals":
            return f"expect({actual}).toBe({expected});"
        elif assertion_type == "not_equals":
            return f"expect({actual}).not.toBe({expected});"
        elif assertion_type == "true":
            return f"expect({actual}).toBe(true);"
        elif assertion_type == "false":
            return f"expect({actual}).toBe(false);"
        elif assertion_type == "throws":
            return f"expect(() => {actual}).toThrow();"
        else:
            return f"expect({actual}).toBe({expected});"

    def _python_assertion(self, actual: str, expected: str, assertion_type: str) -> str:
        """Generate Python assertion."""
        if assertion_type == "equals":
            return f"assert {actual} == {expected}"
        elif assertion_type == "not_equals":
            return f"assert {actual} != {expected}"
        elif assertion_type == "true":
            return f"assert {actual} is True"
        elif assertion_type == "false":
            return f"assert {actual} is False"
        elif assertion_type == "throws":
            return f"with pytest.raises(Exception):\n    {actual}"
        else:
            return f"assert {actual} == {expected}"

    def _java_assertion(self, actual: str, expected: str, assertion_type: str) -> str:
        """Generate Java assertion."""
        if assertion_type == "equals":
            return f"assertEquals({expected}, {actual});"
        elif assertion_type == "not_equals":
            return f"assertNotEquals({expected}, {actual});"
        elif assertion_type == "true":
            return f"assertTrue({actual});"
        elif assertion_type == "false":
            return f"assertFalse({actual});"
        elif assertion_type == "throws":
            return f"assertThrows(Exception.class, () -> {actual});"
        else:
            return f"assertEquals({expected}, {actual});"

    def _chai_assertion(self, actual: str, expected: str, assertion_type: str) -> str:
        """Generate Chai assertion."""
        if assertion_type == "equals":
            return f"expect({actual}).to.equal({expected});"
        elif assertion_type == "not_equals":
            return f"expect({actual}).to.not.equal({expected});"
        elif assertion_type == "true":
            return f"expect({actual}).to.be.true;"
        elif assertion_type == "false":
            return f"expect({actual}).to.be.false;"
        elif assertion_type == "throws":
            return f"expect(() => {actual}).to.throw();"
        else:
            return f"expect({actual}).to.equal({expected});"

    def generate_setup_teardown(
        self,
        setup_code: str = "",
        teardown_code: str = ""
    ) -> str:
        """Generate setup and teardown hooks."""
        result = []

        if self.framework in [Framework.JEST, Framework.VITEST, Framework.MOCHA]:
            if setup_code:
                result.append(f"""beforeEach(() => {{
{self._indent(setup_code, 2)}
}});""")
            if teardown_code:
                result.append(f"""afterEach(() => {{
{self._indent(teardown_code, 2)}
}});""")

        elif self.framework == Framework.PYTEST:
            if setup_code:
                result.append(f"""@pytest.fixture(autouse=True)
def setup_method(self):
{self._indent(setup_code, 4)}
    yield""")
            if teardown_code:
                result.append(f"""
{self._indent(teardown_code, 4)}""")

        elif self.framework == Framework.UNITTEST:
            if setup_code:
                result.append(f"""def setUp(self):
{self._indent(setup_code, 4)}""")
            if teardown_code:
                result.append(f"""def tearDown(self):
{self._indent(teardown_code, 4)}""")

        elif self.framework in [Framework.JUNIT, Framework.TESTNG]:
            annotation = "@BeforeEach" if self.framework == Framework.JUNIT else "@BeforeMethod"
            if setup_code:
                result.append(f"""{annotation}
public void setUp() {{
{self._indent(setup_code, 4)}
}}""")

            annotation = "@AfterEach" if self.framework == Framework.JUNIT else "@AfterMethod"
            if teardown_code:
                result.append(f"""{annotation}
public void tearDown() {{
{self._indent(teardown_code, 4)}
}}""")

        return "\n\n".join(result)

    def _indent(self, text: str, spaces: int) -> str:
        """Indent text by number of spaces."""
        indent = " " * spaces
        lines = text.split('\n')
        return '\n'.join(indent + line if line.strip() else line for line in lines)

    def _to_camel_case(self, text: str) -> str:
        """Convert text to camelCase."""
        words = text.replace('-', ' ').replace('_', ' ').split()
        if not words:
            return text
        return words[0].lower() + ''.join(word.capitalize() for word in words[1:])

    def _to_class_name(self, text: str) -> str:
        """Convert text to ClassName."""
        words = text.replace('-', ' ').replace('_', ' ').split()
        return ''.join(word.capitalize() for word in words)

    def detect_framework(self, code: str) -> Optional[Framework]:
        """
        Auto-detect testing framework from code.

        Args:
            code: Test code

        Returns:
            Detected framework or None
        """
        # Jest patterns
        if 'from \'@jest/globals\'' in code or '@jest/' in code:
            return Framework.JEST

        # Vitest patterns
        if 'from \'vitest\'' in code or 'import { vi }' in code:
            return Framework.VITEST

        # Pytest patterns
        if 'import pytest' in code or 'def test_' in code and 'pytest.fixture' in code:
            return Framework.PYTEST

        # Unittest patterns
        if 'import unittest' in code and 'unittest.TestCase' in code:
            return Framework.UNITTEST

        # JUnit patterns
        if '@Test' in code and 'import org.junit' in code:
            return Framework.JUNIT

        # TestNG patterns
        if '@Test' in code and 'import org.testng' in code:
            return Framework.TESTNG

        # Mocha patterns
        if 'from \'mocha\'' in code or ('describe(' in code and 'from \'chai\'' in code):
            return Framework.MOCHA

        return None

FILE:scripts/metrics_calculator.py
"""
Metrics calculation module.

Calculate comprehensive test and code quality metrics including complexity,
test quality scoring, and test execution analysis.
"""

from typing import Dict, List, Any, Optional
import re


class MetricsCalculator:
    """Calculate comprehensive test and code quality metrics."""

    def __init__(self):
        """Initialize metrics calculator."""
        self.metrics = {}

    def calculate_all_metrics(
        self,
        source_code: str,
        test_code: str,
        coverage_data: Optional[Dict[str, Any]] = None,
        execution_data: Optional[Dict[str, Any]] = None
    ) -> Dict[str, Any]:
        """
        Calculate all available metrics.

        Args:
            source_code: Source code to analyze
            test_code: Test code to analyze
            coverage_data: Coverage report data
            execution_data: Test execution results

        Returns:
            Complete metrics dictionary
        """
        metrics = {
            'complexity': self.calculate_complexity(source_code),
            'test_quality': self.calculate_test_quality(test_code),
            'coverage': coverage_data or {},
            'execution': execution_data or {}
        }

        self.metrics = metrics
        return metrics

    def calculate_complexity(self, code: str) -> Dict[str, Any]:
        """
        Calculate code complexity metrics.

        Args:
            code: Source code to analyze

        Returns:
            Complexity metrics (cyclomatic, cognitive, testability score)
        """
        cyclomatic = self._cyclomatic_complexity(code)
        cognitive = self._cognitive_complexity(code)
        testability = self._testability_score(code, cyclomatic)

        return {
            'cyclomatic_complexity': cyclomatic,
            'cognitive_complexity': cognitive,
            'testability_score': testability,
            'assessment': self._complexity_assessment(cyclomatic, cognitive)
        }

    def _cyclomatic_complexity(self, code: str) -> int:
        """
        Calculate cyclomatic complexity (simplified).

        Counts decision points: if, for, while, case, catch, &&, ||
        """
        # Count decision points
        decision_points = 0

        # Control flow keywords
        keywords = ['if', 'for', 'while', 'case', 'catch', 'except']
        for keyword in keywords:
            # Use word boundaries to avoid matching substrings
            pattern = r'\b' + keyword + r'\b'
            decision_points += len(re.findall(pattern, code))

        # Logical operators
        decision_points += len(re.findall(r'\&\&|\|\|', code))

        # Base complexity is 1
        return decision_points + 1

    def _cognitive_complexity(self, code: str) -> int:
        """
        Calculate cognitive complexity (simplified).

        Similar to cyclomatic but penalizes nesting and non-obvious flow.
        """
        lines = code.split('\n')
        cognitive_score = 0
        nesting_level = 0

        for line in lines:
            stripped = line.strip()

            # Increase nesting level
            if any(keyword in stripped for keyword in ['if ', 'for ', 'while ', 'def ', 'function ', 'class ']):
                cognitive_score += (1 + nesting_level)
                if stripped.endswith(':') or stripped.endswith('{'):
                    nesting_level += 1

            # Decrease nesting level
            if stripped.startswith('}') or (stripped and not stripped.startswith(' ') and nesting_level > 0):
                nesting_level = max(0, nesting_level - 1)

            # Penalize complex conditions
            if '&&' in stripped or '||' in stripped:
                cognitive_score += 1

        return cognitive_score

    def _testability_score(self, code: str, cyclomatic: int) -> float:
        """
        Calculate testability score (0-100).

        Based on:
        - Complexity (lower is better)
        - Dependencies (fewer is better)
        - Pure functions (more is better)
        """
        score = 100.0

        # Penalize high complexity
        if cyclomatic > 10:
            score -= (cyclomatic - 10) * 5
        elif cyclomatic > 5:
            score -= (cyclomatic - 5) * 2

        # Penalize many dependencies
        imports = len(re.findall(r'import |require\(|from .* import', code))
        if imports > 10:
            score -= (imports - 10) * 2

        # Reward small functions
        functions = len(re.findall(r'def |function ', code))
        lines = len(code.split('\n'))
        if functions > 0:
            avg_function_size = lines / functions
            if avg_function_size < 20:
                score += 10
            elif avg_function_size > 50:
                score -= 10

        return max(0.0, min(100.0, score))

    def _complexity_assessment(self, cyclomatic: int, cognitive: int) -> str:
        """Generate complexity assessment."""
        if cyclomatic <= 5 and cognitive <= 10:
            return "Low complexity - easy to test"
        elif cyclomatic <= 10 and cognitive <= 20:
            return "Medium complexity - moderately testable"
        elif cyclomatic <= 15 and cognitive <= 30:
            return "High complexity - challenging to test"
        else:
            return "Very high complexity - consider refactoring"

    def calculate_test_quality(self, test_code: str) -> Dict[str, Any]:
        """
        Calculate test quality metrics.

        Args:
            test_code: Test code to analyze

        Returns:
            Test quality metrics
        """
        assertions = self._count_assertions(test_code)
        test_functions = self._count_test_functions(test_code)
        isolation_score = self._isolation_score(test_code)
        naming_quality = self._naming_quality(test_code)
        test_smells = self._detect_test_smells(test_code)

        avg_assertions = assertions / test_functions if test_functions > 0 else 0

        return {
            'total_tests': test_functions,
            'total_assertions': assertions,
            'avg_assertions_per_test': round(avg_assertions, 2),
            'isolation_score': isolation_score,
            'naming_quality': naming_quality,
            'test_smells': test_smells,
            'quality_score': self._calculate_quality_score(
                avg_assertions, isolation_score, naming_quality, test_smells
            )
        }

    def _count_assertions(self, test_code: str) -> int:
        """Count assertion statements."""
        # Common assertion patterns
        patterns = [
            r'\bassert[A-Z]\w*\(',  # JUnit: assertTrue, assertEquals
            r'\bexpect\(',  # Jest/Vitest: expect()
            r'\bassert\s+',  # Python: assert
            r'\.should\.',  # Chai: should
            r'\.to\.',  # Chai: expect().to
        ]

        count = 0
        for pattern in patterns:
            count += len(re.findall(pattern, test_code))

        return count

    def _count_test_functions(self, test_code: str) -> int:
        """Count test functions."""
        patterns = [
            r'\btest_\w+',  # Python: test_*
            r'\bit\(',  # Jest/Mocha: it()
            r'\btest\(',  # Jest: test()
            r'@Test',  # JUnit: @Test
            r'\bdef test_',  # Python def test_
        ]

        count = 0
        for pattern in patterns:
            count += len(re.findall(pattern, test_code))

        return max(1, count)  # At least 1 to avoid division by zero

    def _isolation_score(self, test_code: str) -> float:
        """
        Calculate test isolation score (0-100).

        Higher score = better isolation (fewer shared dependencies)
        """
        score = 100.0

        # Penalize global state
        globals_used = len(re.findall(r'\bglobal\s+\w+', test_code))
        score -= globals_used * 10

        # Penalize shared setup without proper cleanup
        setup_count = len(re.findall(r'beforeAll|beforeEach|setUp', test_code))
        cleanup_count = len(re.findall(r'afterAll|afterEach|tearDown', test_code))
        if setup_count > cleanup_count:
            score -= (setup_count - cleanup_count) * 5

        # Reward mocking
        mocks = len(re.findall(r'mock|stub|spy', test_code, re.IGNORECASE))
        score += min(mocks * 2, 10)

        return max(0.0, min(100.0, score))

    def _naming_quality(self, test_code: str) -> float:
        """
        Calculate test naming quality score (0-100).

        Better names are descriptive and follow conventions.
        """
        test_names = re.findall(r'(?:it|test|def test_)\s*\(?\s*["\']?([^"\')\n]+)', test_code)

        if not test_names:
            return 50.0

        score = 0
        for name in test_names:
            name_score = 0

            # Check length (too short or too long is bad)
            if 20 <= len(name) <= 80:
                name_score += 30
            elif 10 <= len(name) < 20 or 80 < len(name) <= 100:
                name_score += 15

            # Check for descriptive words
            descriptive_words = ['should', 'when', 'given', 'returns', 'throws', 'handles']
            if any(word in name.lower() for word in descriptive_words):
                name_score += 30

            # Check for underscores or camelCase (not just letters)
            if '_' in name or re.search(r'[a-z][A-Z]', name):
                name_score += 20

            # Avoid generic names
            generic = ['test1', 'test2', 'testit', 'mytest']
            if name.lower() not in generic:
                name_score += 20

            score += name_score

        return min(100.0, score / len(test_names))

    def _detect_test_smells(self, test_code: str) -> List[Dict[str, str]]:
        """Detect common test smells."""
        smells = []

        # Test smell 1: No assertions
        if 'assert' not in test_code.lower() and 'expect' not in test_code.lower():
            smells.append({
                'smell': 'missing_assertions',
                'description': 'Tests without assertions',
                'severity': 'high'
            })

        # Test smell 2: Too many assertions
        test_count = self._count_test_functions(test_code)
        assertion_count = self._count_assertions(test_code)
        avg_assertions = assertion_count / test_count if test_count > 0 else 0
        if avg_assertions > 5:
            smells.append({
                'smell': 'assertion_roulette',
                'description': f'Too many assertions per test (avg: {avg_assertions:.1f})',
                'severity': 'medium'
            })

        # Test smell 3: Sleeps in tests
        if 'sleep' in test_code.lower() or 'wait' in test_code.lower():
            smells.append({
                'smell': 'sleepy_test',
                'description': 'Tests using sleep/wait (potential flakiness)',
                'severity': 'high'
            })

        # Test smell 4: Conditional logic in tests
        if re.search(r'\bif\s*\(', test_code):
            smells.append({
                'smell': 'conditional_test_logic',
                'description': 'Tests contain conditional logic',
                'severity': 'medium'
            })

        return smells

    def _calculate_quality_score(
        self,
        avg_assertions: float,
        isolation: float,
        naming: float,
        smells: List[Dict[str, str]]
    ) -> float:
        """Calculate overall test quality score."""
        score = 0.0

        # Assertions (30 points)
        if 1 <= avg_assertions <= 3:
            score += 30
        elif 0 < avg_assertions < 1 or 3 < avg_assertions <= 5:
            score += 20
        else:
            score += 10

        # Isolation (30 points)
        score += isolation * 0.3

        # Naming (20 points)
        score += naming * 0.2

        # Smells (20 points - deduct based on severity)
        smell_penalty = 0
        for smell in smells:
            if smell['severity'] == 'high':
                smell_penalty += 10
            elif smell['severity'] == 'medium':
                smell_penalty += 5
            else:
                smell_penalty += 2

        score = max(0, score - smell_penalty)

        return round(min(100.0, score), 2)

    def analyze_execution_metrics(
        self,
        execution_data: Dict[str, Any]
    ) -> Dict[str, Any]:
        """
        Analyze test execution metrics.

        Args:
            execution_data: Test execution results with timing

        Returns:
            Execution analysis
        """
        tests = execution_data.get('tests', [])

        if not tests:
            return {}

        # Calculate timing statistics
        timings = [test.get('duration', 0) for test in tests]
        total_time = sum(timings)
        avg_time = total_time / len(tests) if tests else 0

        # Identify slow tests (>100ms for unit tests)
        slow_tests = [
            test for test in tests
            if test.get('duration', 0) > 100
        ]

        # Identify flaky tests (if failure history available)
        flaky_tests = [
            test for test in tests
            if test.get('failure_rate', 0) > 0.1  # Failed >10% of time
        ]

        return {
            'total_tests': len(tests),
            'total_time_ms': round(total_time, 2),
            'avg_time_ms': round(avg_time, 2),
            'slow_tests': len(slow_tests),
            'slow_test_details': slow_tests[:5],  # Top 5
            'flaky_tests': len(flaky_tests),
            'flaky_test_details': flaky_tests,
            'pass_rate': self._calculate_pass_rate(tests)
        }

    def _calculate_pass_rate(self, tests: List[Dict[str, Any]]) -> float:
        """Calculate test pass rate."""
        if not tests:
            return 0.0

        passed = sum(1 for test in tests if test.get('status') == 'passed')
        return round((passed / len(tests)) * 100, 2)

    def generate_metrics_summary(self) -> str:
        """Generate human-readable metrics summary."""
        if not self.metrics:
            return "No metrics calculated yet."

        lines = ["# Test Metrics Summary\n"]

        # Complexity
        if 'complexity' in self.metrics:
            comp = self.metrics['complexity']
            lines.append(f"## Code Complexity")
            lines.append(f"- Cyclomatic Complexity: {comp['cyclomatic_complexity']}")
            lines.append(f"- Cognitive Complexity: {comp['cognitive_complexity']}")
            lines.append(f"- Testability Score: {comp['testability_score']:.1f}/100")
            lines.append(f"- Assessment: {comp['assessment']}\n")

        # Test Quality
        if 'test_quality' in self.metrics:
            qual = self.metrics['test_quality']
            lines.append(f"## Test Quality")
            lines.append(f"- Total Tests: {qual['total_tests']}")
            lines.append(f"- Assertions per Test: {qual['avg_assertions_per_test']}")
            lines.append(f"- Isolation Score: {qual['isolation_score']:.1f}/100")
            lines.append(f"- Naming Quality: {qual['naming_quality']:.1f}/100")
            lines.append(f"- Quality Score: {qual['quality_score']:.1f}/100\n")

            if qual['test_smells']:
                lines.append(f"### Test Smells Detected:")
                for smell in qual['test_smells']:
                    lines.append(f"- {smell['description']} (severity: {smell['severity']})")
                lines.append("")

        return "\n".join(lines)

FILE:scripts/output_formatter.py
"""
Output formatting module.

Provides context-aware output formatting for different environments (Desktop, CLI, API).
Implements progressive disclosure and token-efficient reporting.
"""

from typing import Dict, List, Any, Optional


class OutputFormatter:
    """Format output based on environment and preferences."""

    def __init__(self, environment: str = "cli", verbose: bool = False):
        """
        Initialize output formatter.

        Args:
            environment: Target environment (desktop, cli, api)
            verbose: Whether to include detailed output
        """
        self.environment = environment
        self.verbose = verbose

    def format_coverage_summary(
        self,
        summary: Dict[str, Any],
        detailed: bool = False
    ) -> str:
        """
        Format coverage summary.

        Args:
            summary: Coverage summary data
            detailed: Whether to include detailed breakdown

        Returns:
            Formatted coverage summary
        """
        if self.environment == "desktop":
            return self._format_coverage_markdown(summary, detailed)
        elif self.environment == "api":
            return self._format_coverage_json(summary)
        else:
            return self._format_coverage_terminal(summary, detailed)

    def _format_coverage_markdown(self, summary: Dict[str, Any], detailed: bool) -> str:
        """Format coverage as rich markdown (for Claude Desktop)."""
        lines = ["## Test Coverage Summary\n"]

        # Overall metrics
        lines.append("### Overall Metrics")
        lines.append(f"- **Line Coverage**: {summary.get('line_coverage', 0):.1f}%")
        lines.append(f"- **Branch Coverage**: {summary.get('branch_coverage', 0):.1f}%")
        lines.append(f"- **Function Coverage**: {summary.get('function_coverage', 0):.1f}%\n")

        # Visual indicator
        line_cov = summary.get('line_coverage', 0)
        lines.append(self._coverage_badge(line_cov))
        lines.append("")

        # Detailed breakdown if requested
        if detailed:
            lines.append("### Detailed Breakdown")
            lines.append(f"- Total Lines: {summary.get('total_lines', 0)}")
            lines.append(f"- Covered Lines: {summary.get('covered_lines', 0)}")
            lines.append(f"- Total Branches: {summary.get('total_branches', 0)}")
            lines.append(f"- Covered Branches: {summary.get('covered_branches', 0)}")
            lines.append(f"- Total Functions: {summary.get('total_functions', 0)}")
            lines.append(f"- Covered Functions: {summary.get('covered_functions', 0)}\n")

        return "\n".join(lines)

    def _format_coverage_terminal(self, summary: Dict[str, Any], detailed: bool) -> str:
        """Format coverage for terminal (Claude Code CLI)."""
        lines = ["Coverage Summary:"]
        lines.append(f"  Line:     {summary.get('line_coverage', 0):.1f}%")
        lines.append(f"  Branch:   {summary.get('branch_coverage', 0):.1f}%")
        lines.append(f"  Function: {summary.get('function_coverage', 0):.1f}%")

        if detailed:
            lines.append(f"\nDetails:")
            lines.append(f"  Lines: {summary.get('covered_lines', 0)}/{summary.get('total_lines', 0)}")
            lines.append(f"  Branches: {summary.get('covered_branches', 0)}/{summary.get('total_branches', 0)}")

        return "\n".join(lines)

    def _format_coverage_json(self, summary: Dict[str, Any]) -> str:
        """Format coverage as JSON (for API/CI integration)."""
        import json
        return json.dumps(summary, indent=2)

    def _coverage_badge(self, coverage: float) -> str:
        """Generate coverage badge markdown."""
        if coverage >= 80:
            color = "green"
            emoji = "✅"
        elif coverage >= 60:
            color = "yellow"
            emoji = "⚠️"
        else:
            color = "red"
            emoji = "❌"

        return f"{emoji} **{coverage:.1f}%** coverage ({color})"

    def format_recommendations(
        self,
        recommendations: List[Dict[str, Any]],
        max_items: Optional[int] = None
    ) -> str:
        """
        Format recommendations with progressive disclosure.

        Args:
            recommendations: List of recommendation dictionaries
            max_items: Maximum number of items to show (None for all)

        Returns:
            Formatted recommendations
        """
        if not recommendations:
            return "No recommendations at this time."

        # Group by priority
        p0 = [r for r in recommendations if r.get('priority') == 'P0']
        p1 = [r for r in recommendations if r.get('priority') == 'P1']
        p2 = [r for r in recommendations if r.get('priority') == 'P2']

        if self.environment == "desktop":
            return self._format_recommendations_markdown(p0, p1, p2, max_items)
        elif self.environment == "api":
            return self._format_recommendations_json(recommendations)
        else:
            return self._format_recommendations_terminal(p0, p1, p2, max_items)

    def _format_recommendations_markdown(
        self,
        p0: List[Dict],
        p1: List[Dict],
        p2: List[Dict],
        max_items: Optional[int]
    ) -> str:
        """Format recommendations as rich markdown."""
        lines = ["## Recommendations\n"]

        if p0:
            lines.append("### 🔴 Critical (P0)")
            for i, rec in enumerate(p0[:max_items] if max_items else p0):
                lines.append(f"{i+1}. **{rec.get('message', 'No message')}**")
                lines.append(f"   - Action: {rec.get('action', 'No action specified')}")
                if 'file' in rec:
                    lines.append(f"   - File: `{rec['file']}`")
                lines.append("")

        if p1 and (not max_items or len(p0) < max_items):
            remaining = max_items - len(p0) if max_items else None
            lines.append("### 🟡 Important (P1)")
            for i, rec in enumerate(p1[:remaining] if remaining else p1):
                lines.append(f"{i+1}. {rec.get('message', 'No message')}")
                lines.append(f"   - Action: {rec.get('action', 'No action specified')}")
                lines.append("")

        if p2 and self.verbose:
            lines.append("### 🔵 Nice to Have (P2)")
            for i, rec in enumerate(p2):
                lines.append(f"{i+1}. {rec.get('message', 'No message')}")
                lines.append("")

        return "\n".join(lines)

    def _format_recommendations_terminal(
        self,
        p0: List[Dict],
        p1: List[Dict],
        p2: List[Dict],
        max_items: Optional[int]
    ) -> str:
        """Format recommendations for terminal."""
        lines = ["Recommendations:"]

        if p0:
            lines.append("\nCritical (P0):")
            for i, rec in enumerate(p0[:max_items] if max_items else p0):
                lines.append(f"  {i+1}. {rec.get('message', 'No message')}")
                lines.append(f"     Action: {rec.get('action', 'No action')}")

        if p1 and (not max_items or len(p0) < max_items):
            remaining = max_items - len(p0) if max_items else None
            lines.append("\nImportant (P1):")
            for i, rec in enumerate(p1[:remaining] if remaining else p1):
                lines.append(f"  {i+1}. {rec.get('message', 'No message')}")

        return "\n".join(lines)

    def _format_recommendations_json(self, recommendations: List[Dict[str, Any]]) -> str:
        """Format recommendations as JSON."""
        import json
        return json.dumps(recommendations, indent=2)

    def format_test_results(
        self,
        results: Dict[str, Any],
        show_details: bool = False
    ) -> str:
        """
        Format test execution results.

        Args:
            results: Test results data
            show_details: Whether to show detailed results

        Returns:
            Formatted test results
        """
        if self.environment == "desktop":
            return self._format_results_markdown(results, show_details)
        elif self.environment == "api":
            return self._format_results_json(results)
        else:
            return self._format_results_terminal(results, show_details)

    def _format_results_markdown(self, results: Dict[str, Any], show_details: bool) -> str:
        """Format test results as markdown."""
        lines = ["## Test Results\n"]

        total = results.get('total_tests', 0)
        passed = results.get('passed', 0)
        failed = results.get('failed', 0)
        skipped = results.get('skipped', 0)

        # Summary
        lines.append(f"- **Total Tests**: {total}")
        lines.append(f"- **Passed**: ✅ {passed}")
        if failed > 0:
            lines.append(f"- **Failed**: ❌ {failed}")
        if skipped > 0:
            lines.append(f"- **Skipped**: ⏭️ {skipped}")

        # Pass rate
        pass_rate = (passed / total * 100) if total > 0 else 0
        lines.append(f"- **Pass Rate**: {pass_rate:.1f}%\n")

        # Failed tests details
        if show_details and failed > 0:
            lines.append("### Failed Tests")
            for test in results.get('failed_tests', []):
                lines.append(f"- `{test.get('name', 'Unknown')}`")
                if 'error' in test:
                    lines.append(f"  ```\n  {test['error']}\n  ```")

        return "\n".join(lines)

    def _format_results_terminal(self, results: Dict[str, Any], show_details: bool) -> str:
        """Format test results for terminal."""
        total = results.get('total_tests', 0)
        passed = results.get('passed', 0)
        failed = results.get('failed', 0)

        lines = [f"Test Results: {passed}/{total} passed"]

        if failed > 0:
            lines.append(f"  Failed: {failed}")

        if show_details and failed > 0:
            lines.append("\nFailed tests:")
            for test in results.get('failed_tests', [])[:5]:
                lines.append(f"  - {test.get('name', 'Unknown')}")

        return "\n".join(lines)

    def _format_results_json(self, results: Dict[str, Any]) -> str:
        """Format test results as JSON."""
        import json
        return json.dumps(results, indent=2)

    def create_summary_report(
        self,
        coverage: Dict[str, Any],
        metrics: Dict[str, Any],
        recommendations: List[Dict[str, Any]]
    ) -> str:
        """
        Create comprehensive summary report (token-efficient).

        Args:
            coverage: Coverage data
            metrics: Quality metrics
            recommendations: Recommendations list

        Returns:
            Summary report (<200 tokens)
        """
        lines = []

        # Coverage (1-2 lines)
        line_cov = coverage.get('line_coverage', 0)
        branch_cov = coverage.get('branch_coverage', 0)
        lines.append(f"Coverage: {line_cov:.0f}% lines, {branch_cov:.0f}% branches")

        # Quality (1-2 lines)
        if 'test_quality' in metrics:
            quality_score = metrics['test_quality'].get('quality_score', 0)
            lines.append(f"Test Quality: {quality_score:.0f}/100")

        # Top recommendations (2-3 lines)
        p0_count = sum(1 for r in recommendations if r.get('priority') == 'P0')
        if p0_count > 0:
            lines.append(f"Critical issues: {p0_count}")
            top_rec = next((r for r in recommendations if r.get('priority') == 'P0'), None)
            if top_rec:
                lines.append(f"  - {top_rec.get('message', '')}")

        return "\n".join(lines)

    def should_show_detailed(self, data_size: int) -> bool:
        """
        Determine if detailed output should be shown based on data size.

        Args:
            data_size: Size of data to display

        Returns:
            Whether to show detailed output
        """
        if self.verbose:
            return True

        # Progressive disclosure thresholds
        if self.environment == "desktop":
            return data_size < 100  # Show more in Desktop
        else:
            return data_size < 20  # Show less in CLI

    def truncate_output(self, text: str, max_lines: int = 50) -> str:
        """
        Truncate output to maximum lines.

        Args:
            text: Text to truncate
            max_lines: Maximum number of lines

        Returns:
            Truncated text with indicator
        """
        lines = text.split('\n')

        if len(lines) <= max_lines:
            return text

        truncated = '\n'.join(lines[:max_lines])
        remaining = len(lines) - max_lines

        return f"{truncated}\n\n... ({remaining} more lines, use --verbose for full output)"

FILE:scripts/tdd_workflow.py
"""
TDD workflow guidance module.

Provides step-by-step guidance through red-green-refactor cycles with validation.
"""

from typing import Dict, List, Any, Optional
from enum import Enum


class TDDPhase(Enum):
    """TDD cycle phases."""
    RED = "red"  # Write failing test
    GREEN = "green"  # Make test pass
    REFACTOR = "refactor"  # Improve code


class WorkflowState(Enum):
    """Current state of TDD workflow."""
    INITIAL = "initial"
    TEST_WRITTEN = "test_written"
    TEST_FAILING = "test_failing"
    TEST_PASSING = "test_passing"
    CODE_REFACTORED = "code_refactored"


class TDDWorkflow:
    """Guide users through TDD red-green-refactor workflow."""

    def __init__(self):
        """Initialize TDD workflow guide."""
        self.current_phase = TDDPhase.RED
        self.state = WorkflowState.INITIAL
        self.history = []

    def start_cycle(self, requirement: str) -> Dict[str, Any]:
        """
        Start a new TDD cycle.

        Args:
            requirement: User story or requirement to implement

        Returns:
            Guidance for RED phase
        """
        self.current_phase = TDDPhase.RED
        self.state = WorkflowState.INITIAL

        return {
            'phase': 'RED',
            'instruction': 'Write a failing test for the requirement',
            'requirement': requirement,
            'checklist': [
                'Write test that describes desired behavior',
                'Test should fail when run (no implementation yet)',
                'Test name clearly describes what is being tested',
                'Test has clear arrange-act-assert structure'
            ],
            'tips': [
                'Focus on behavior, not implementation',
                'Start with simplest test case',
                'Test should be specific and focused'
            ]
        }

    def validate_red_phase(
        self,
        test_code: str,
        test_result: Optional[Dict[str, Any]] = None
    ) -> Dict[str, Any]:
        """
        Validate RED phase completion.

        Args:
            test_code: The test code written
            test_result: Test execution result (optional)

        Returns:
            Validation result and next steps
        """
        validations = []

        # Check test exists
        if not test_code or len(test_code.strip()) < 10:
            validations.append({
                'valid': False,
                'message': 'No test code provided'
            })
        else:
            validations.append({
                'valid': True,
                'message': 'Test code provided'
            })

        # Check for assertions
        has_assertion = any(keyword in test_code.lower()
                           for keyword in ['assert', 'expect', 'should'])
        validations.append({
            'valid': has_assertion,
            'message': 'Contains assertions' if has_assertion else 'Missing assertions'
        })

        # Check test result if provided
        if test_result:
            test_failed = test_result.get('status') == 'failed'
            validations.append({
                'valid': test_failed,
                'message': 'Test fails as expected' if test_failed else 'Test should fail in RED phase'
            })

        all_valid = all(v['valid'] for v in validations)

        if all_valid:
            self.state = WorkflowState.TEST_FAILING
            self.current_phase = TDDPhase.GREEN
            return {
                'phase_complete': True,
                'next_phase': 'GREEN',
                'validations': validations,
                'instruction': 'Write minimal code to make the test pass'
            }
        else:
            return {
                'phase_complete': False,
                'current_phase': 'RED',
                'validations': validations,
                'instruction': 'Address validation issues before proceeding'
            }

    def validate_green_phase(
        self,
        implementation_code: str,
        test_result: Dict[str, Any]
    ) -> Dict[str, Any]:
        """
        Validate GREEN phase completion.

        Args:
            implementation_code: The implementation code
            test_result: Test execution result

        Returns:
            Validation result and next steps
        """
        validations = []

        # Check implementation exists
        if not implementation_code or len(implementation_code.strip()) < 5:
            validations.append({
                'valid': False,
                'message': 'No implementation code provided'
            })
        else:
            validations.append({
                'valid': True,
                'message': 'Implementation code provided'
            })

        # Check test now passes
        test_passed = test_result.get('status') == 'passed'
        validations.append({
            'valid': test_passed,
            'message': 'Test passes' if test_passed else 'Test still failing'
        })

        # Check for minimal implementation (heuristic)
        is_minimal = self._check_minimal_implementation(implementation_code)
        validations.append({
            'valid': is_minimal,
            'message': 'Implementation appears minimal' if is_minimal
                      else 'Implementation may be over-engineered'
        })

        all_valid = all(v['valid'] for v in validations)

        if all_valid:
            self.state = WorkflowState.TEST_PASSING
            self.current_phase = TDDPhase.REFACTOR
            return {
                'phase_complete': True,
                'next_phase': 'REFACTOR',
                'validations': validations,
                'instruction': 'Refactor code while keeping tests green',
                'refactoring_suggestions': self._suggest_refactorings(implementation_code)
            }
        else:
            return {
                'phase_complete': False,
                'current_phase': 'GREEN',
                'validations': validations,
                'instruction': 'Make the test pass before refactoring'
            }

    def validate_refactor_phase(
        self,
        original_code: str,
        refactored_code: str,
        test_result: Dict[str, Any]
    ) -> Dict[str, Any]:
        """
        Validate REFACTOR phase completion.

        Args:
            original_code: Original implementation
            refactored_code: Refactored implementation
            test_result: Test execution result after refactoring

        Returns:
            Validation result and cycle completion status
        """
        validations = []

        # Check tests still pass
        test_passed = test_result.get('status') == 'passed'
        validations.append({
            'valid': test_passed,
            'message': 'Tests still pass after refactoring' if test_passed
                      else 'Tests broken by refactoring'
        })

        # Check code was actually refactored
        code_changed = original_code != refactored_code
        validations.append({
            'valid': code_changed,
            'message': 'Code was refactored' if code_changed
                      else 'No refactoring applied (optional)'
        })

        # Check code quality improved
        quality_improved = self._check_quality_improvement(original_code, refactored_code)
        if code_changed:
            validations.append({
                'valid': quality_improved,
                'message': 'Code quality improved' if quality_improved
                          else 'Consider further refactoring for better quality'
            })

        all_valid = all(v['valid'] for v in validations if v.get('valid') is not None)

        if all_valid:
            self.state = WorkflowState.CODE_REFACTORED
            self.history.append({
                'cycle_complete': True,
                'final_state': self.state
            })
            return {
                'phase_complete': True,
                'cycle_complete': True,
                'validations': validations,
                'message': 'TDD cycle complete! Ready for next requirement.',
                'next_steps': [
                    'Commit your changes',
                    'Start next TDD cycle with new requirement',
                    'Or add more test cases for current feature'
                ]
            }
        else:
            return {
                'phase_complete': False,
                'current_phase': 'REFACTOR',
                'validations': validations,
                'instruction': 'Ensure tests still pass after refactoring'
            }

    def _check_minimal_implementation(self, code: str) -> bool:
        """Check if implementation is minimal (heuristic)."""
        # Simple heuristics:
        # - Not too long (< 50 lines for unit tests)
        # - Not too complex (few nested structures)

        lines = code.split('\n')
        non_empty_lines = [line for line in lines if line.strip() and not line.strip().startswith('#')]

        # Check length
        if len(non_empty_lines) > 50:
            return False

        # Check nesting depth (simplified)
        max_depth = 0
        current_depth = 0
        for line in lines:
            stripped = line.lstrip()
            if stripped:
                indent = len(line) - len(stripped)
                depth = indent // 4  # Assuming 4-space indent
                max_depth = max(max_depth, depth)

        # Max nesting of 3 levels for simple implementation
        return max_depth <= 3

    def _check_quality_improvement(self, original: str, refactored: str) -> bool:
        """Check if refactoring improved code quality."""
        # Simple heuristics:
        # - Reduced duplication
        # - Better naming
        # - Simpler structure

        # Check for reduced duplication (basic check)
        original_lines = set(line.strip() for line in original.split('\n') if line.strip())
        refactored_lines = set(line.strip() for line in refactored.split('\n') if line.strip())

        # If unique lines increased proportionally, likely extracted duplicates
        if len(refactored_lines) > len(original_lines):
            return True

        # Check for better naming (longer, more descriptive names)
        original_avg_identifier_length = self._avg_identifier_length(original)
        refactored_avg_identifier_length = self._avg_identifier_length(refactored)

        if refactored_avg_identifier_length > original_avg_identifier_length:
            return True

        # If no clear improvement detected, assume refactoring was beneficial
        return True

    def _avg_identifier_length(self, code: str) -> float:
        """Calculate average identifier length (proxy for naming quality)."""
        import re
        identifiers = re.findall(r'\b[a-zA-Z_][a-zA-Z0-9_]*\b', code)

        # Filter out keywords
        keywords = {'if', 'else', 'for', 'while', 'def', 'class', 'return', 'import', 'from'}
        identifiers = [i for i in identifiers if i.lower() not in keywords]

        if not identifiers:
            return 0.0

        return sum(len(i) for i in identifiers) / len(identifiers)

    def _suggest_refactorings(self, code: str) -> List[str]:
        """Suggest potential refactorings."""
        suggestions = []

        # Check for long functions
        lines = code.split('\n')
        if len(lines) > 30:
            suggestions.append('Consider breaking long function into smaller functions')

        # Check for duplication (simple check)
        line_counts = {}
        for line in lines:
            stripped = line.strip()
            if len(stripped) > 10:  # Ignore very short lines
                line_counts[stripped] = line_counts.get(stripped, 0) + 1

        duplicates = [line for line, count in line_counts.items() if count > 2]
        if duplicates:
            suggestions.append(f'Found {len(duplicates)} duplicated code patterns - consider extraction')

        # Check for magic numbers
        import re
        magic_numbers = re.findall(r'\b\d+\b', code)
        if len(magic_numbers) > 5:
            suggestions.append('Consider extracting magic numbers to named constants')

        # Check for long parameter lists
        if 'def ' in code or 'function' in code:
            param_matches = re.findall(r'\(([^)]+)\)', code)
            for params in param_matches:
                if params.count(',') > 3:
                    suggestions.append('Consider using parameter object for functions with many parameters')
                    break

        if not suggestions:
            suggestions.append('Code looks clean - no obvious refactorings needed')

        return suggestions

    def generate_workflow_summary(self) -> str:
        """Generate summary of TDD workflow progress."""
        summary = [
            "# TDD Workflow Summary\n",
            f"Current Phase: {self.current_phase.value.upper()}",
            f"Current State: {self.state.value.replace('_', ' ').title()}",
            f"Completed Cycles: {len(self.history)}\n"
        ]

        summary.append("## TDD Cycle Steps:\n")
        summary.append("1. **RED**: Write a failing test")
        summary.append("   - Test describes desired behavior")
        summary.append("   - Test fails (no implementation)\n")

        summary.append("2. **GREEN**: Make the test pass")
        summary.append("   - Write minimal code to pass test")
        summary.append("   - All tests should pass\n")

        summary.append("3. **REFACTOR**: Improve the code")
        summary.append("   - Clean up implementation")
        summary.append("   - Tests still pass")
        summary.append("   - Code is more maintainable\n")

        return "\n".join(summary)

    def get_phase_guidance(self, phase: Optional[TDDPhase] = None) -> Dict[str, Any]:
        """
        Get detailed guidance for a specific phase.

        Args:
            phase: TDD phase (uses current if not specified)

        Returns:
            Detailed guidance dictionary
        """
        target_phase = phase or self.current_phase

        if target_phase == TDDPhase.RED:
            return {
                'phase': 'RED',
                'goal': 'Write a failing test',
                'steps': [
                    '1. Read and understand the requirement',
                    '2. Think about expected behavior',
                    '3. Write test that verifies this behavior',
                    '4. Run test and ensure it fails',
                    '5. Verify failure reason is correct (not syntax error)'
                ],
                'common_mistakes': [
                    'Test passes immediately (no real assertion)',
                    'Test fails for wrong reason (syntax error)',
                    'Test is too broad or tests multiple things'
                ],
                'tips': [
                    'Start with simplest test case',
                    'One assertion per test (focused)',
                    'Test should read like specification'
                ]
            }

        elif target_phase == TDDPhase.GREEN:
            return {
                'phase': 'GREEN',
                'goal': 'Make the test pass with minimal code',
                'steps': [
                    '1. Write simplest code that makes test pass',
                    '2. Run test and verify it passes',
                    '3. Run all tests to ensure no regression',
                    '4. Resist urge to add extra features'
                ],
                'common_mistakes': [
                    'Over-engineering solution',
                    'Adding features not covered by tests',
                    'Breaking existing tests'
                ],
                'tips': [
                    'Fake it till you make it (hardcode if needed)',
                    'Triangulate with more tests if needed',
                    'Keep implementation simple'
                ]
            }

        elif target_phase == TDDPhase.REFACTOR:
            return {
                'phase': 'REFACTOR',
                'goal': 'Improve code quality while keeping tests green',
                'steps': [
                    '1. Identify code smells or duplication',
                    '2. Apply one refactoring at a time',
                    '3. Run tests after each change',
                    '4. Commit when satisfied with quality'
                ],
                'common_mistakes': [
                    'Changing behavior (breaking tests)',
                    'Refactoring too much at once',
                    'Skipping this phase'
                ],
                'tips': [
                    'Extract methods for better naming',
                    'Remove duplication',
                    'Improve variable names',
                    'Tests are safety net - use them!'
                ]
            }

        return {}

FILE:scripts/test_generator.py
"""
Test case generation module.

Generates test cases from requirements, user stories, API specs, and code analysis.
Supports multiple testing frameworks with intelligent test scaffolding.
"""

from typing import Dict, List, Any, Optional
from enum import Enum


class TestFramework(Enum):
    """Supported testing frameworks."""
    JEST = "jest"
    VITEST = "vitest"
    PYTEST = "pytest"
    JUNIT = "junit"
    MOCHA = "mocha"


class TestType(Enum):
    """Types of tests to generate."""
    UNIT = "unit"
    INTEGRATION = "integration"
    E2E = "e2e"


class TestGenerator:
    """Generate test cases and test stubs from requirements and code."""

    def __init__(self, framework: TestFramework, language: str):
        """
        Initialize test generator.

        Args:
            framework: Testing framework to use
            language: Programming language (typescript, javascript, python, java)
        """
        self.framework = framework
        self.language = language
        self.test_cases = []

    def generate_from_requirements(
        self,
        requirements: Dict[str, Any],
        test_type: TestType = TestType.UNIT
    ) -> List[Dict[str, Any]]:
        """
        Generate test cases from requirements.

        Args:
            requirements: Dictionary with user_stories, acceptance_criteria, api_specs
            test_type: Type of tests to generate

        Returns:
            List of test case specifications
        """
        test_cases = []

        # Generate from user stories
        if 'user_stories' in requirements:
            for story in requirements['user_stories']:
                test_cases.extend(self._test_cases_from_story(story))

        # Generate from acceptance criteria
        if 'acceptance_criteria' in requirements:
            for criterion in requirements['acceptance_criteria']:
                test_cases.extend(self._test_cases_from_criteria(criterion))

        # Generate from API specs
        if 'api_specs' in requirements:
            for endpoint in requirements['api_specs']:
                test_cases.extend(self._test_cases_from_api(endpoint))

        self.test_cases = test_cases
        return test_cases

    def _test_cases_from_story(self, story: Dict[str, Any]) -> List[Dict[str, Any]]:
        """Generate test cases from user story."""
        test_cases = []

        # Happy path test
        test_cases.append({
            'name': f"should_{story.get('action', 'work')}_successfully",
            'type': 'happy_path',
            'description': story.get('description', ''),
            'given': story.get('given', []),
            'when': story.get('when', ''),
            'then': story.get('then', ''),
            'priority': 'P0'
        })

        # Error cases
        if 'error_conditions' in story:
            for error in story['error_conditions']:
                test_cases.append({
                    'name': f"should_handle_{error.get('condition', 'error')}",
                    'type': 'error_case',
                    'description': error.get('description', ''),
                    'expected_error': error.get('error_type', ''),
                    'priority': 'P0'
                })

        # Edge cases
        if 'edge_cases' in story:
            for edge_case in story['edge_cases']:
                test_cases.append({
                    'name': f"should_handle_{edge_case.get('scenario', 'edge_case')}",
                    'type': 'edge_case',
                    'description': edge_case.get('description', ''),
                    'priority': 'P1'
                })

        return test_cases

    def _test_cases_from_criteria(self, criterion: Dict[str, Any]) -> List[Dict[str, Any]]:
        """Generate test cases from acceptance criteria."""
        return [{
            'name': f"should_meet_{criterion.get('id', 'criterion')}",
            'type': 'acceptance',
            'description': criterion.get('description', ''),
            'verification': criterion.get('verification_steps', []),
            'priority': 'P0'
        }]

    def _test_cases_from_api(self, endpoint: Dict[str, Any]) -> List[Dict[str, Any]]:
        """Generate test cases from API specification."""
        test_cases = []
        method = endpoint.get('method', 'GET')
        path = endpoint.get('path', '/')

        # Success case
        test_cases.append({
            'name': f"should_{method.lower()}_{path.replace('/', '_')}_successfully",
            'type': 'api_success',
            'method': method,
            'path': path,
            'expected_status': endpoint.get('success_status', 200),
            'priority': 'P0'
        })

        # Validation errors
        if 'required_params' in endpoint:
            test_cases.append({
                'name': f"should_return_400_for_missing_params",
                'type': 'api_validation',
                'method': method,
                'path': path,
                'expected_status': 400,
                'priority': 'P0'
            })

        # Authorization
        if endpoint.get('requires_auth', False):
            test_cases.append({
                'name': f"should_return_401_for_unauthenticated",
                'type': 'api_auth',
                'method': method,
                'path': path,
                'expected_status': 401,
                'priority': 'P0'
            })

        return test_cases

    def generate_test_stub(self, test_case: Dict[str, Any]) -> str:
        """
        Generate test stub code for a test case.

        Args:
            test_case: Test case specification

        Returns:
            Test stub code as string
        """
        if self.framework == TestFramework.JEST:
            return self._generate_jest_stub(test_case)
        elif self.framework == TestFramework.PYTEST:
            return self._generate_pytest_stub(test_case)
        elif self.framework == TestFramework.JUNIT:
            return self._generate_junit_stub(test_case)
        elif self.framework == TestFramework.VITEST:
            return self._generate_vitest_stub(test_case)
        else:
            return self._generate_generic_stub(test_case)

    def _generate_jest_stub(self, test_case: Dict[str, Any]) -> str:
        """Generate Jest test stub."""
        name = test_case.get('name', 'test')
        description = test_case.get('description', '')

        stub = f"""
describe('{{Feature Name}}', () => {{
  it('{name}', () => {{
    // {description}

    // Arrange
    // TODO: Set up test data and dependencies

    // Act
    // TODO: Execute the code under test

    // Assert
    // TODO: Verify expected behavior
    expect(true).toBe(true); // Replace with actual assertion
  }});
}});
"""
        return stub.strip()

    def _generate_pytest_stub(self, test_case: Dict[str, Any]) -> str:
        """Generate Pytest test stub."""
        name = test_case.get('name', 'test')
        description = test_case.get('description', '')

        stub = f"""
def test_{name}():
    \"\"\"
    {description}
    \"\"\"
    # Arrange
    # TODO: Set up test data and dependencies

    # Act
    # TODO: Execute the code under test

    # Assert
    # TODO: Verify expected behavior
    assert True  # Replace with actual assertion
"""
        return stub.strip()

    def _generate_junit_stub(self, test_case: Dict[str, Any]) -> str:
        """Generate JUnit test stub."""
        name = test_case.get('name', 'test')
        description = test_case.get('description', '')

        # Convert snake_case to camelCase for Java
        method_name = ''.join(word.capitalize() if i > 0 else word
                             for i, word in enumerate(name.split('_')))

        stub = f"""
@Test
public void {method_name}() {{
    // {description}

    // Arrange
    // TODO: Set up test data and dependencies

    // Act
    // TODO: Execute the code under test

    // Assert
    // TODO: Verify expected behavior
    assertTrue(true); // Replace with actual assertion
}}
"""
        return stub.strip()

    def _generate_vitest_stub(self, test_case: Dict[str, Any]) -> str:
        """Generate Vitest test stub (similar to Jest)."""
        name = test_case.get('name', 'test')
        description = test_case.get('description', '')

        stub = f"""
describe('{{Feature Name}}', () => {{
  it('{name}', () => {{
    // {description}

    // Arrange
    // TODO: Set up test data and dependencies

    // Act
    // TODO: Execute the code under test

    // Assert
    // TODO: Verify expected behavior
    expect(true).toBe(true); // Replace with actual assertion
  }});
}});
"""
        return stub.strip()

    def _generate_generic_stub(self, test_case: Dict[str, Any]) -> str:
        """Generate generic test stub."""
        name = test_case.get('name', 'test')
        description = test_case.get('description', '')

        return f"""
# Test: {name}
# Description: {description}
#
# TODO: Implement test
# 1. Arrange: Set up test data
# 2. Act: Execute code under test
# 3. Assert: Verify expected behavior
"""

    def generate_test_file(
        self,
        module_name: str,
        test_cases: Optional[List[Dict[str, Any]]] = None
    ) -> str:
        """
        Generate complete test file with all test stubs.

        Args:
            module_name: Name of module being tested
            test_cases: List of test cases (uses self.test_cases if not provided)

        Returns:
            Complete test file content
        """
        cases = test_cases or self.test_cases

        if self.framework == TestFramework.JEST:
            return self._generate_jest_file(module_name, cases)
        elif self.framework == TestFramework.PYTEST:
            return self._generate_pytest_file(module_name, cases)
        elif self.framework == TestFramework.JUNIT:
            return self._generate_junit_file(module_name, cases)
        elif self.framework == TestFramework.VITEST:
            return self._generate_vitest_file(module_name, cases)
        else:
            return ""

    def _generate_jest_file(self, module_name: str, test_cases: List[Dict[str, Any]]) -> str:
        """Generate complete Jest test file."""
        imports = f"import {{ {module_name} }} from '../{module_name}';\n\n"

        stubs = []
        for test_case in test_cases:
            stubs.append(self._generate_jest_stub(test_case))

        return imports + "\n\n".join(stubs)

    def _generate_pytest_file(self, module_name: str, test_cases: List[Dict[str, Any]]) -> str:
        """Generate complete Pytest test file."""
        imports = f"import pytest\nfrom {module_name} import *\n\n\n"

        stubs = []
        for test_case in test_cases:
            stubs.append(self._generate_pytest_stub(test_case))

        return imports + "\n\n\n".join(stubs)

    def _generate_junit_file(self, module_name: str, test_cases: List[Dict[str, Any]]) -> str:
        """Generate complete JUnit test file."""
        class_name = ''.join(word.capitalize() for word in module_name.split('_'))

        imports = """import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;

"""

        class_header = f"public class {class_name}Test {{\n\n"

        stubs = []
        for test_case in test_cases:
            stubs.append(self._generate_junit_stub(test_case))

        class_footer = "\n}"

        return imports + class_header + "\n\n".join(stubs) + class_footer

    def _generate_vitest_file(self, module_name: str, test_cases: List[Dict[str, Any]]) -> str:
        """Generate complete Vitest test file."""
        imports = f"import {{ describe, it, expect }} from 'vitest';\nimport {{ {module_name} }} from '../{module_name}';\n\n"

        stubs = []
        for test_case in test_cases:
            stubs.append(self._generate_vitest_stub(test_case))

        return imports + "\n\n".join(stubs)

    def suggest_missing_scenarios(
        self,
        existing_tests: List[str],
        code_analysis: Dict[str, Any]
    ) -> List[Dict[str, Any]]:
        """
        Suggest missing test scenarios based on code analysis.

        Args:
            existing_tests: List of existing test names
            code_analysis: Analysis of code under test (branches, error paths, etc.)

        Returns:
            List of suggested test scenarios
        """
        suggestions = []

        # Check for untested error conditions
        if 'error_handlers' in code_analysis:
            for error_handler in code_analysis['error_handlers']:
                error_name = error_handler.get('type', 'error')
                if not self._has_test_for(existing_tests, error_name):
                    suggestions.append({
                        'name': f"should_handle_{error_name}",
                        'type': 'error_case',
                        'reason': 'Error handler exists but no corresponding test',
                        'priority': 'P0'
                    })

        # Check for untested branches
        if 'conditional_branches' in code_analysis:
            for branch in code_analysis['conditional_branches']:
                branch_name = branch.get('condition', 'condition')
                if not self._has_test_for(existing_tests, branch_name):
                    suggestions.append({
                        'name': f"should_test_{branch_name}_branch",
                        'type': 'branch_coverage',
                        'reason': 'Conditional branch not fully tested',
                        'priority': 'P1'
                    })

        # Check for boundary conditions
        if 'input_validation' in code_analysis:
            for validation in code_analysis['input_validation']:
                param = validation.get('parameter', 'input')
                if not self._has_test_for(existing_tests, f"{param}_boundary"):
                    suggestions.append({
                        'name': f"should_test_{param}_boundary_values",
                        'type': 'boundary',
                        'reason': 'Input validation exists but boundary tests missing',
                        'priority': 'P1'
                    })

        return suggestions

    def _has_test_for(self, existing_tests: List[str], keyword: str) -> bool:
        """Check if existing tests cover a keyword/scenario."""
        keyword_lower = keyword.lower().replace('_', '').replace('-', '')
        for test in existing_tests:
            test_lower = test.lower().replace('_', '').replace('-', '')
            if keyword_lower in test_lower:
                return True
        return False

---
name: capture
description: "Captures and organizes chaotic brain dumps into a structured, actionable system with zero information loss. Use this skill whenever the user says 'capture this', 'brain dump', 'let me dump some ideas', 'I've got a bunch of thoughts', 'here's everything on my mind', 'idea dump', 'let me get this out of my head', 'I need to organize my thoughts', 'here's what I'm thinking', or any variation where someone is unloading a messy stream of ideas, tasks, thoughts, and plans wanting them turned into something coherent. Also trigger when the user pastes or dictates a long, unstructured block of mixed ideas — even without the exact phrase — the intent is the same. Fast-to-action by design: no upfront intake. Output is four sections (Projects/Ideas, Tasks, Connections, How I Can Help) ending with a directive question. Asks at most one mid-organization clarifying question when a single item is genuinely ambiguous between task and project."
license: MIT
metadata:
  source_spec: "megaprompts/05-capture-megaprompt.md"
  build_pattern: "Path B (direct conversion)"
  version: 1.0.0
---

# Capture — Brain-Dump Organizer

A fast-to-action skill for transforming unstructured streams of mixed thoughts, tasks, and ideas into a clean four-section actionable system with zero information loss.

## Invocation Triggers

**Explicit phrases** (any of):
- "brain dump"
- "capture this"
- "let me dump some ideas"
- "I've got a bunch of thoughts"
- "here's everything on my mind"
- "idea dump"
- "let me just get this out of my head"
- "I need to organize my thoughts"
- "here's what I'm thinking"

**Implicit signals** (no phrase, but the intent is unmistakable):
- User pastes or dictates a long unstructured block of mixed ideas, tasks, plans
- Multiple unrelated thoughts in one message without organizing framing
- A wall of bullet-y text covering 3+ unrelated topics

When you detect an implicit trigger, run the skill. Do NOT ask "do you want me to organize this?" first — the dump itself IS the request.

## Operating Principles (All Five Apply Always)

1. **Capture everything.** Zero loss. Trivial items go in; the user prunes later. Never silently drop something because it "seemed unimportant".
2. **Preserve voice.** If the user said "build something crazy with AI", do NOT restate as "Explore innovative AI-driven solutions." Keep the energy and the casual register. See `references/voice_preservation.md` for concrete anti-patterns.
3. **Match output complexity to input.** A 5-task dump does NOT get forced into 4 elaborate sections. See `references/complexity_matching.md` and the Compressed Output Pattern below.
4. **Be honest about ambiguity.** If you're unsure what something means, flag it. Don't guess silently.
5. **No action without approval.** The ONLY immediate action is the organization itself. Every offer in Section 4 waits for the user's explicit pick.

## Grill-Me Mid-Organization Clarifier

Capture is fast-to-action by design. **No upfront intake.** The dump is enough — start organizing immediately.

The grill-me discipline applies as a **single mid-organization clarifying question**, asked **only when** one item in the dump is genuinely ambiguous between *task* and *project*, AND the misclassification would meaningfully change the output:

> **Quick clarification — one item in your dump could go either way. Is [X] a one-shot task or a multi-step project?**
>
> *Why I'm asking:* If I guess wrong on a borderline item I either bury a project as a task or inflate a task into a project that doesn't need the structure. One question per dump prevents that.

**Stop condition:** Max 1 clarifying question per dump. After the answer (or if no clarification was needed), deliver the four (or compressed) sections.

If the dump is unambiguous, skip the clarifier entirely.

**Anti-pattern (do not do this):** asking 3 clarifying questions up front. That breaks the dump-and-organize flow that makes capture useful.

## Section 1: Projects & Ideas

Cluster related items into themed projects when natural clustering exists. This section also holds:
- Standalone creative sparks
- Half-formed concepts
- "What if" thoughts
- Embedded decisions (`Decide: X or Y`) and open questions (`Q: ...`) — kept WITHIN the relevant project, NOT extracted into a separate top-level category

**Format per project:**

```
### {Project name in user's voice}

- {component / sub-idea}
- {component}
- Q: {open question this project needs answered}
- Decide: {decision this project requires}
```

Use the user's words for the project name. If the user wrote "ai dating app for ferrets", do NOT rename it to "AI-Powered Pet Companion Platform".

## Section 2: Tasks

Flat, scannable, action-oriented. Includes:
- Explicit todos
- Decisions framed as `Decide: ...`
- Open questions framed as `Resolve: ...`

If a task belongs to a project from Section 1, append `[Project: X]` to link it — but don't repeat the project's context.

**Format:**

```
- {task in imperative voice}  [Project: X if related]
- Decide: {decision}  [Project: X if related]
- Resolve: {open question}
- ...
```

## Section 3: Connections

This is where the skill earns its keep — and where **fabrication is forbidden**.

**Workflow:**

1. **Inventory the workspace** — Glob for filename patterns matching dump keywords, Grep for content matches, read the top-level directory structure. Use `scripts/workspace_inventory.py` to do this deterministically.
2. **Match dump items to existing content** — files / folders relating to dumped items, prior thinking in documents, in-progress projects with overlap.
3. **Surface dependencies within the dump** — items that affect each other, themes, ordering implications.
4. **Be honest about inaccessibility** — if you can't inspect the workspace (no filesystem available, MCP not connected), say so explicitly. Do NOT make up plausible-sounding connections.

**Hard rule:** NEVER fabricate connections. Only surface ones actually found by Glob/Grep/Read. If no real connections exist:

> **Connections:** No connections found — workspace inventory clean.

If the workspace is inaccessible:

> **Connections:** No workspace accessible from here. If you're running this from Claude Code or have a project with files attached, I can fill this in. Want to share where this work lives?

See `references/workspace_detection.md` for the per-context detection-tactic catalog.

## Section 4: How I Can Help

**Concrete offers, not abstract possibilities.** Every offer specifies what would be produced AND where it would go.

| ✅ Right pattern | ❌ Anti-pattern |
|---|---|
| "I can research Consensus MCP integration patterns and give you 3 options. Output: `docs/consensus-options.md`." | "You might want to look into integration approaches." |
| "I can draft the Q3 launch plan as a 1-pager. Output: chat reply, then `docs/q3-launch.md` if you want it filed." | "Maybe think about Q3 planning." |
| "I can scaffold the new auth module with the existing pattern from `src/users/`. Output: 4 files in `src/auth/`." | "We could explore auth options." |

End with the directive question:

> **Which of these should I tackle?**

## Compressed Output Pattern

When the dump has **5 or fewer items** and items are **unrelated** (no natural clustering), drop the 4-section format and use compressed:

```
## What I heard

- {item}
- {item}
- {item}
- ...

## How I can help

- {concrete offer with what + where}
- {concrete offer with what + where}

Which should I tackle?
```

The trigger is the `complexity_estimator.py` recommendation OR your judgment when no clusters exist. See `references/complexity_matching.md` for worked examples of when each format applies.

## Workspace Detection Strategy

| Context | Detection method |
|---|---|
| Claude Code CLI | Glob for files matching dump keywords; Grep for content matches; read top-level structure. Use `scripts/workspace_inventory.py`. |
| Claude.ai with project | Check project knowledge files for thematic overlap. List file titles; surface matches by keyword. |
| Connected tools (Notion, Drive, etc.) | Search via MCP if available. |
| No accessible workspace | State the limitation explicitly; ask user about their setup; do NOT fabricate. |

## Approval Gate

After the four (or compressed) sections are delivered:

- **Wait for the user's explicit pick** before doing anything else.
- If the user says "go" without picking a specific offer: honor it, but explicitly note any items you weren't 100% sure about so they can correct.
- The organization itself is the only auto-action. Every Section 4 offer requires green light.

## Error Handling

| Situation | Behavior |
|---|---|
| Workspace inaccessible | State this; skip Section 3 or surface "no workspace accessible" + ask about setup |
| Dump is very short (3-5 items) | Use compressed output; don't force 4 sections |
| Items are highly ambiguous | Flag in output, ask up to 1 clarifier (or skip clarifier and surface ambiguity in delivery) |
| Dump contains sensitive info | Acknowledge but don't echo verbatim if user asks for organization without quoting |
| Conflicting items in the dump | Surface the conflict in Section 1 or 3 explicitly (`Conflict: X says A, Y says B`) |
| User says "go" before approval | Honor it, but explicitly note items you weren't sure about |

## Tooling

| Script | Role |
|---|---|
| `scripts/workspace_inventory.py` | Glob+Grep helper for Section 3. `python workspace_inventory.py --root . --keywords "k1,k2"` returns matches by keyword + folder structure. |
| `scripts/dump_classifier.py` | Regex-classifies each dump line into `task` / `decision` / `question` / `idea` / `project-component`. Heuristic — override with judgment. |
| `scripts/complexity_estimator.py` | Counts items, detects clustering signal, recommends `format=full` or `format=compressed`. |

## References

- `references/workspace_detection.md` — context-specific detection tactics (CLI / web / MCP / inaccessible)
- `references/voice_preservation.md` — corporate-speak anti-patterns with concrete examples
- `references/complexity_matching.md` — compressed vs full output, worked examples

## Anti-Patterns To Reject

- Fabricating workspace connections that weren't actually Glob/Grep-verified
- Dropping items deemed "trivial" — capture everything, let the user prune
- Corporate-ifying the user's casual language
- Forcing 4-section structure when input is small (5 simple tasks doesn't need it)
- Acting on Section-4 offers immediately without approval
- Splitting decisions/questions into a separate top-level category instead of embedding them in the relevant project
- Vague Section-4 offers ("you might want to consider…")
- Asking 3+ clarifying questions up front (breaks fast-to-action)

---

**Version:** 1.0.0
**Source spec:** [`megaprompts/05-capture-megaprompt.md`](../../../../megaprompts/05-capture-megaprompt.md)
**Build pattern:** Path B (direct conversion). Re-grill with `/cs:grill-with-docs` if drift between spec and implementation surfaces.

FILE:references/complexity_matching.md
# Complexity Matching — Compressed vs Full 4-Section Output

This reference answers exactly one decision: **when does capture use the full 4-section format vs the compressed format, and what does each look like in practice?**

Pair with `scripts/complexity_estimator.py` for the deterministic recommendation.

## The Core Rule

> **Match output complexity to input complexity.**

A 30-item dump with natural clusters needs the full 4-section structure to be useful. A 5-item dump of unrelated todos drowns in that structure — the format becomes ceremony, not signal. Force-fitting structure on a small dump makes the skill feel bureaucratic.

## When to Use Each Format

| Signal | Recommended format |
|---|---|
| 8+ items AND natural clustering exists (3+ items share a theme) | Full 4-section |
| 8+ items but NO clustering (all unrelated todos) | Compressed (with explicit "no clusters" note) |
| 5–7 items, mixed kinds, some clustering | Either — judgment call. Lean compressed unless clusters are strong. |
| ≤5 items, unrelated | Compressed |
| ≤5 items but all related to one project | Compressed with single project header |
| Workspace inaccessible AND ≤5 items | Compressed with no Section 3 (still note "no workspace accessible") |

`complexity_estimator.py` returns `format=full` or `format=compressed` based on item count + clustering signal. Use it as the seed; override with judgment when context warrants.

## Format A: Full 4-Section

Use for substantive dumps with real structure. Roughly:

```
## Projects & Ideas

### {Project A in user's voice}
- {component}
- {component}
- Q: {open question}
- Decide: {decision}

### {Project B}
- ...

## Tasks

- {task} [Project: A]
- {task}
- Decide: {decision}
- Resolve: {open question}

## Connections

- {file/folder}: {real workspace match}
- (or) "No connections found — workspace inventory clean."
- (or) "No workspace accessible from here..."

## How I Can Help

- {concrete offer: what + where}
- {concrete offer: what + where}

**Which of these should I tackle?**
```

## Format B: Compressed

Use for small or unrelated dumps. Roughly:

```
## What I heard

- {item}
- {item}
- {item}
- Decide: {decision}
- Resolve: {open question}

## How I can help

- {concrete offer: what + where}
- {concrete offer: what + where}

Which should I tackle?
```

Notice what's gone in compressed:
- No project headers (nothing to cluster)
- No separate Tasks section (the list IS the tasks)
- No Connections section as a separate header (if there's a real connection, fold it inline as a parenthetical: `- fix the auth bug (relates to src/auth/login.ts:42)`)
- Same Section 4 discipline (concrete offers + directive question)

## Worked Examples

### Example 1: Full 4-section (12 items, 2 clusters)

**Dump:**
> "Ok so dump time. Q3 launch is approaching — need to nail down pricing, draft the launch email, brief Sarah on the marketing angle. Also ferret app keeps nagging me — should I talk to my cofounder about it, or is it just procrastination? Need to sketch the matching algorithm if it's serious. Also: fix the damn auth bug, rewrite the login form because it's ugly, write tests for both. Oh and I should probably do my Q3 OKRs before launch."

**Extracted items:** 11
**Clusters:** Q3 Launch (4 items), Ferret App (3 items), Auth (3 items), OKRs (1 item)
**Recommendation:** `format=full` (clear clustering, 11 items)

**Output:**

```
## Projects & Ideas

### Q3 Launch
- Nail down pricing
- Draft launch email
- Brief Sarah on the marketing angle
- Decide: Q3 OKRs (do before launch)

### Ferret App
- Talk to cofounder OR decide it's procrastination
- Sketch matching algorithm (if serious)
- Decide: serious project or shelf?

### Auth Fixes
- Fix the auth bug
- Rewrite the login form (it's ugly)
- Write tests for the bug fix + login rewrite

## Tasks

- Nail down Q3 pricing  [Project: Q3 Launch]
- Draft Q3 launch email  [Project: Q3 Launch]
- Brief Sarah on Q3 marketing angle  [Project: Q3 Launch]
- Talk to cofounder about ferret app  [Project: Ferret App]
- Sketch ferret matching algo  [Project: Ferret App]
- Fix auth bug  [Project: Auth Fixes]
- Rewrite login form  [Project: Auth Fixes]
- Write tests for auth + login  [Project: Auth Fixes]
- Do Q3 OKRs (before launch)
- Decide: ferret app — serious or shelf?

## Connections

(workspace inventory results go here — only verified matches)

## How I Can Help

- I can draft the Q3 launch email as a 1-pager. Output: chat reply.
- I can pull together a pricing-options matrix from your existing pricing docs. Output: `docs/q3-pricing-options.md`.
- I can scaffold the ferret matching algorithm sketch (input → match score). Output: `docs/ferret-algo-sketch.md`.

**Which of these should I tackle?**
```

### Example 2: Compressed (5 unrelated items)

**Dump:**
> "Quick: email Sarah, fix that test that's been flaky, decide between Postgres and Mongo for the new service, dentist appointment, finish reading the article on RAG."

**Extracted items:** 5
**Clusters:** none (all unrelated)
**Recommendation:** `format=compressed`

**Output:**

```
## What I heard

- Email Sarah
- Fix the flaky test
- Decide: Postgres or Mongo for the new service
- Dentist appointment
- Finish reading the RAG article

## How I can help

- I can draft the email to Sarah if you tell me what it's about. Output: chat reply.
- I can investigate the flaky test (likely racing condition; happy to reproduce). Output: chat analysis + fix PR if you want.
- I can pull a Postgres-vs-Mongo decision matrix scoped to your service requirements. Output: chat reply or `docs/db-decision.md`.

Which should I tackle?
```

Notice the dentist appointment + RAG article aren't in "How I can help" — there's nothing meaningful to offer on those. That's correct: don't pad Section 4 with offers for things that don't need help.

### Example 3: Compressed but workspace-aware (small + connection inline)

**Dump:**
> "auth bug is back, need to look at the login flow again, also I want to add 2FA finally."

**Extracted items:** 3
**Clusters:** all auth-related but only 3 items
**Recommendation:** `format=compressed`

**Output:**

```
## What I heard (Auth)

- Auth bug is back (relates to `src/auth/login.ts` based on workspace inventory)
- Look at login flow again
- Add 2FA

## How I can help

- I can reproduce the auth bug from the symptoms — what's the failure mode? Output: chat repro + fix.
- I can sketch a 2FA implementation matching your existing auth pattern (TOTP via the same provider you use). Output: `docs/2fa-sketch.md`.

Which should I tackle?
```

Notice the workspace connection got folded inline as a parenthetical instead of a separate Section 3 header. That's the compressed-with-context pattern.

## Operational Checklist

Before delivering output:

- [ ] Run `complexity_estimator.py` (or apply the Signal table above)
- [ ] If `format=compressed`, do NOT force the 4-section format
- [ ] If `format=full`, ensure the clusters are real (3+ items per cluster) — don't invent clusters to fill the format
- [ ] Either way, Section 4 ("How I can help") MUST have concrete offers with what + where
- [ ] Either way, end with the directive question

## Why This Matters

A skill that returns the same format regardless of input is a template, not a skill. The reason capture is useful is that it adapts to the dump's actual shape. When a 5-item list comes back wrapped in 4 elaborate empty-feeling sections, the user learns to distrust the skill. When a 30-item dump comes back as a flat compressed list, the user learns the skill can't actually handle complexity.

Match the output to the input, every time.

FILE:references/voice_preservation.md
# Voice Preservation — Anti-Corporate-Speak Discipline

This reference answers exactly one decision: **what does it mean to "preserve the user's voice" in capture output, and what concrete patterns must be avoided?**

## The Core Rule

If the user said it casually, restate it casually. If the user said it crudely, restate it crudely (within the user's own register). Capture is for THEM, not for an imagined corporate audience reading their notes later.

> **Restating someone's casual idea in corporate language is a tax. It feels formal but it loses the energy that made the idea worth capturing.**

## Concrete Anti-Patterns (Side-by-Side)

| User said | ❌ Corporate-ified (anti-pattern) | ✅ Voice-preserved |
|---|---|---|
| "build something crazy with AI" | "Explore innovative AI-driven solutions" | "Build something crazy with AI" |
| "the dating app idea but for ferrets" | "Pet-companion matching platform leveraging social-graph principles" | "Dating app for ferrets" |
| "figure out the damn pricing already" | "Conduct comprehensive pricing strategy analysis" | "Figure out pricing (final answer)" |
| "fuck around with Consensus MCP" | "Investigate Consensus MCP integration opportunities" | "Try out Consensus MCP" |
| "make the landing page not suck" | "Optimize landing page user experience metrics" | "Make the landing page not suck" |
| "talk to Sarah about the thing" | "Schedule alignment discussion with Sarah re: outstanding initiative" | "Talk to Sarah about the thing" |
| "I'm tired of debugging this" | "Investigate root causes of recurring debugging friction" | "Tired of debugging this — find the root cause" |

## What Counts As "Voice"

- **Register** — formal vs casual, dry vs energetic, ironic vs earnest
- **Vocabulary** — the user's exact noun choices for things ("ferrets", "thing", "Sarah" — not "pets", "initiative", "stakeholder")
- **Cadence** — short choppy phrases stay short; long flowing thoughts stay flowing
- **Profanity / slang** — preserve as-is; don't sanitize
- **Self-talk markers** — "ugh", "actually", "wait", "ok so" — these signal genuine thinking and belong in the captured form

## What's Allowed to Change

- **Punctuation cleanup** — adding a period, fixing typos
- **Imperative reframing** for the Tasks section — "I should email Sarah" → "Email Sarah" (one-word edit, voice preserved)
- **Light disambiguation** — if "the thing" is genuinely confusing in context, note it but ask to clarify (don't replace it silently)

## What's Never Allowed

- Replacing user nouns with "platform" / "solution" / "initiative" / "framework"
- Verbing nouns: "let's research" → "let's conduct research"
- Adding qualifiers the user didn't say: "explore", "leverage", "deep dive into"
- "Action-itemizing" everything: "talk to Sarah" → "Establish communication touchpoint with Sarah"
- Removing emotion: "I'm pissed about X" → "There is a concern regarding X"
- Bullet-point fluff: "Implement", "Establish", "Facilitate" prefixes added for no reason

## Cluster Naming

When clustering items into projects (Section 1), the project name **MUST** use the user's words. Examples:

| Items in cluster | ❌ Anti-pattern name | ✅ Voice-preserved name |
|---|---|---|
| "ferret app", "ferret features", "ferret marketing" | "Pet Companion Platform" | "Ferret App" |
| "Q3 launch", "Q3 pricing", "Q3 emails" | "Q3 Go-to-Market Initiative" | "Q3 Launch" |
| "fix the auth bug", "auth tests", "rewrite login" | "Authentication System Modernization" | "Auth fixes" |

If the user used multiple terms for the same cluster, pick the one they used most or most colloquially.

## Operating Test

Before writing each line, ask:

> Would the user *recognize* this as something they'd say?

If no, you've drifted. Rewrite to match their register.

## Why This Matters

Voice preservation isn't aesthetic — it's functional. Two reasons:

1. **Recognition.** The user reads their own captured dump back in 2 days and needs to instantly recognize "yes, that's me, that's what I meant." Corporate restatement breaks recognition. The user thinks "wait, did I actually say that?" and starts second-guessing the rest of the output.

2. **Energy.** A dump captured in voice retains the *why* behind each item — the frustration, the excitement, the half-formed hope. Corporate restatement strips the why and leaves a list of generic action items that no one is excited to act on.

Capture is the user's brain on paper. Don't translate it into a stranger's brain.

FILE:references/workspace_detection.md
# Workspace Detection Tactics

This reference answers exactly one decision: **how does the capture skill verify Section 3 connections without fabricating them, across the four contexts the skill might run in?**

Pair with `scripts/workspace_inventory.py` for the deterministic Glob+Grep implementation.

## The Core Rule

Section 3 ("Connections") earns the skill its keep. It also breaks the skill faster than anything else if it lies. The rule:

> **Only surface connections that were actually verified by Glob, Grep, Read, or an equivalent retrieval call this turn.**

If you can't verify, you say "no workspace accessible" or "no connections found" — never invent something plausible-sounding.

## Context 1: Claude Code CLI (filesystem-native)

**Tools available:** `Glob`, `Grep`, `Read`, `Bash`.

**Tactics, in order:**

1. **Extract keywords from the dump.** Pull domain nouns, project names, file-format hints (`.md`, `.py`, `auth`, `consensus`, `pricing`).
2. **Glob for filename matches.** `Glob("**/*{keyword}*")` for each keyword. Limit to top-N matches per keyword to avoid noise.
3. **Grep for content matches.** `Grep("{keyword}")` constrained to source extensions.
4. **Read the top-level structure.** `Bash("ls -la")` and `Bash("find . -maxdepth 2 -type d | head -30")` to surface relevant folders.
5. **Stitch the matches into Section 3 entries.** Each entry: `- {file or folder}: {how it relates to dump item N, with evidence}`.

**Example output:**

```
## Connections

- `engineering/grill-me/` — relates to your "build a grill skill" dump item (folder exists, has plugin.json + SKILL.md). Likely the template you'd want to mirror.
- `megaprompts/05-capture-megaprompt.md` — relates to your "convert capture spec to skill" item. The spec file is here.
- `documentation/implementation/` — empty directory, but the location for the implementation plan you mentioned.
```

**What NOT to do:**
- "There's probably a config for that somewhere" — speculation, no verification.
- "Your project likely has an auth module" — guess, no Glob.
- "I see you might have considered X before" — projection, no Grep.

## Context 2: Claude.ai with project knowledge

**Tools available:** Project-knowledge file list, file content reads.

**Tactics:**

1. **List the project knowledge files** — at the start of the run, get the file inventory.
2. **Match by title keyword** — for each dump keyword, find files whose titles contain it.
3. **Open the top matches** and check if the content is actually related (not just title coincidence).
4. **Surface only the verified matches** in Section 3.

**What NOT to do:**
- Cite a file you didn't open — title match alone is not enough.
- Claim a file says X without quoting evidence.

## Context 3: Connected tools (Notion, Drive, GitHub, Slack via MCP)

**Tools available:** Whatever MCP tools the harness has registered for the user's connected services.

**Tactics:**

1. **Check tool availability first** — list the MCP tools surfaced for this session. If no Notion/Drive/GitHub MCP is registered, skip this context.
2. **Search via MCP** — use the search tool for each tool with dump keywords.
3. **Surface verified hits** with the link / ID returned by the tool.

**What NOT to do:**
- Reference a Notion page that wasn't returned by the search.
- Cite a GitHub issue number without confirming via the GitHub MCP.

## Context 4: No accessible workspace

**Signals you're in this context:**
- No filesystem tools loaded
- No project knowledge attached
- No workspace MCPs registered
- `workspace_inventory.py` returns empty + you can't verify any other way

**Required behavior:**

State the limitation explicitly. Ask about the user's setup. Do NOT fabricate connections.

**Template output:**

```
## Connections

No workspace accessible from here, so this section is empty. If you're running
this from Claude Code or have a project with files attached, I can fill it in.

Want to share where this work lives — a repo path, a Notion workspace, an
attached project? I can re-run the connections pass with that context.
```

## Operational Checklist (Per Run)

- [ ] Extract dump keywords (domain nouns, project names, format hints)
- [ ] Determine context (CLI / web project / MCP-connected / inaccessible)
- [ ] Run the context-appropriate tactics; never skip verification
- [ ] If context is "inaccessible", say so explicitly + ask about setup
- [ ] Each Section 3 entry must cite the evidence (filename matched, search hit, etc.)
- [ ] Zero entries with phrasing like "probably", "likely", "you might have" — those are speculation, not connections

## Why This Matters

The single fastest way to lose user trust in capture is to surface a fabricated connection. Once the user catches one — "wait, that file doesn't exist" — they stop trusting the entire output, including the items that were correct. Verification is cheap; fabrication is expensive.

FILE:scripts/complexity_estimator.py
#!/usr/bin/env python3
"""complexity_estimator.py — Recommend full-4-section vs compressed output.

Stdlib-only. Counts non-empty items in a dump, detects clustering signal
(repeated keywords across items), and recommends one of:

  format=full        →  use the full Projects/Tasks/Connections/How-I-Can-Help
                        4-section format (8+ items AND clustering signal)
  format=compressed  →  use the compressed What-I-heard / How-I-can-help
                        format (≤5 items OR no clustering signal)

The recommendation is HEURISTIC. The capture skill applies judgment on top
based on full dump context. Use this as the seed.

NO LLM CALLS. Pure word counting + frequency analysis.

Usage:
    python complexity_estimator.py path/to/dump.txt
    python complexity_estimator.py path/to/dump.txt --output json
    python complexity_estimator.py --sample
"""

import argparse
import json
import re
import sys
from collections import Counter
from pathlib import Path
from typing import Any, Dict, List


SAMPLE_DUMP_LARGE = """Ok dump time.
Q3 launch needs pricing nailed down.
Draft the Q3 launch email.
Brief Sarah on the Q3 marketing angle.
Decide: launch July 15 or August 1?
Ferret app idea keeps nagging me.
Should I talk to my cofounder about ferret app?
Sketch the ferret matching algorithm if serious.
Decide: ferret app serious or shelf?
Fix the auth bug.
Rewrite the login form (ugly).
Write tests for auth + login.
Add 2fa module to auth.
Do my Q3 OKRs before launch.
"""

SAMPLE_DUMP_SMALL = """Email Sarah.
Fix the flaky test.
Decide: Postgres or Mongo for new service.
Dentist appointment.
Finish reading the RAG article.
"""


# Stop-words to exclude from clustering detection
STOP_WORDS = {
    "a", "an", "the", "is", "are", "was", "were", "be", "been", "being",
    "to", "of", "in", "on", "at", "for", "with", "by", "from", "up", "down",
    "and", "or", "but", "if", "then", "else", "so", "as",
    "i", "you", "he", "she", "it", "we", "they", "me", "my", "your", "our",
    "this", "that", "these", "those", "do", "did", "have", "has", "had",
    "will", "would", "should", "could", "can", "may", "might",
    "what", "when", "where", "why", "how", "who", "which",
    "go", "get", "got", "make", "made", "let", "let's", "yeah", "ok", "well",
    "just", "really", "very", "much", "more", "most", "some", "any",
    "not", "no", "yes", "now", "before", "after",
}


def extract_items(text: str) -> List[str]:
    """Return non-empty stripped lines (each line = one item)."""
    return [line.strip() for line in text.splitlines() if line.strip()]


def extract_keywords(items: List[str]) -> List[str]:
    """Return all alphabetic tokens >= 3 chars, lowercased, stop-words removed."""
    tokens: List[str] = []
    for it in items:
        for tok in re.findall(r"\b[A-Za-z][a-zA-Z]{2,}\b", it):
            t = tok.lower()
            if t in STOP_WORDS:
                continue
            tokens.append(t)
    return tokens


def detect_clusters(items: List[str], min_cluster_size: int) -> List[Dict[str, Any]]:
    """A 'cluster' is a keyword that appears in min_cluster_size+ different items."""
    keyword_to_item_indices: Dict[str, List[int]] = {}
    for i, it in enumerate(items):
        seen_in_item: set = set()
        for tok in re.findall(r"\b[A-Za-z][a-zA-Z]{2,}\b", it):
            t = tok.lower()
            if t in STOP_WORDS or t in seen_in_item:
                continue
            seen_in_item.add(t)
            keyword_to_item_indices.setdefault(t, []).append(i)
    clusters: List[Dict[str, Any]] = []
    for kw, idxs in keyword_to_item_indices.items():
        if len(idxs) >= min_cluster_size:
            clusters.append({"keyword": kw, "item_indices": idxs, "size": len(idxs)})
    clusters.sort(key=lambda c: (-c["size"], c["keyword"]))
    return clusters


def estimate(text: str, min_cluster_size: int = 3) -> Dict[str, Any]:
    items = extract_items(text)
    item_count = len(items)
    clusters = detect_clusters(items, min_cluster_size)
    cluster_count = len(clusters)

    # Decision logic per references/complexity_matching.md
    if item_count >= 8 and cluster_count >= 1:
        recommendation = "full"
        rationale = f"{item_count} items with {cluster_count} cluster(s) of {min_cluster_size}+ → full 4-section format"
    elif item_count >= 8 and cluster_count == 0:
        recommendation = "compressed"
        rationale = f"{item_count} items but no clustering signal → compressed (with 'no clusters' note)"
    elif 5 <= item_count <= 7 and cluster_count >= 1:
        recommendation = "full"
        rationale = f"{item_count} items with clustering signal → judgment call, defaulting full"
    elif 5 <= item_count <= 7 and cluster_count == 0:
        recommendation = "compressed"
        rationale = f"{item_count} items, no clustering → compressed"
    elif item_count <= 5:
        recommendation = "compressed"
        rationale = f"{item_count} items (small dump) → compressed"
    else:
        recommendation = "compressed"
        rationale = "fallback → compressed"

    return {
        "item_count": item_count,
        "cluster_count": cluster_count,
        "clusters": clusters[:5],  # top 5 only in output for readability
        "recommendation": recommendation,
        "rationale": rationale,
    }


def render_human(result: Dict[str, Any]) -> str:
    out: List[str] = []
    out.append(f"Item count:       {result['item_count']}")
    out.append(f"Cluster count:    {result['cluster_count']}")
    out.append(f"Recommendation:   format={result['recommendation']}")
    out.append(f"Rationale:        {result['rationale']}")
    if result["clusters"]:
        out.append("")
        out.append("Top clusters (keyword → items containing it):")
        for c in result["clusters"]:
            out.append(f"  - '{c['keyword']}' in {c['size']} items: lines {c['item_indices']}")
    return "\n".join(out)


def main(argv: List[str]) -> int:
    parser = argparse.ArgumentParser(description=__doc__.split("\n")[0])
    parser.add_argument("path", nargs="?", help="Path to dump file")
    parser.add_argument("--sample", choices=["large", "small"], help="Estimate the embedded sample dump (large or small)")
    parser.add_argument("--min-cluster-size", type=int, default=3, help="Minimum items sharing a keyword to count as a cluster (default: 3)")
    parser.add_argument("--output", choices=["human", "json"], default="human")
    args = parser.parse_args(argv)

    if args.sample:
        text = SAMPLE_DUMP_LARGE if args.sample == "large" else SAMPLE_DUMP_SMALL
    elif args.path:
        p = Path(args.path)
        if not p.exists():
            print(f"error: {args.path} not found", file=sys.stderr)
            return 2
        text = p.read_text(encoding="utf-8")
    else:
        parser.print_help()
        return 0

    result = estimate(text, args.min_cluster_size)
    if args.output == "json":
        print(json.dumps(result, indent=2))
    else:
        print(render_human(result))
    return 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))

FILE:scripts/dump_classifier.py
#!/usr/bin/env python3
"""dump_classifier.py — Heuristic classifier for brain-dump lines.

Stdlib-only. Reads a dump file (or stdin) and labels each line as one of:
  - task            (action-oriented, imperative or 'I should X')
  - decision        ('decide between X and Y', 'should we X or Y')
  - question        (ends in '?')
  - idea            (creative spark, 'what if', 'maybe we should X')
  - project-component  (sub-element of a larger project, often noun-phrased)
  - context         (preamble, framing, no actionable content)

The classifier is HEURISTIC. The capture skill uses these labels as a SEED for
its own structuring — it overrides based on dump-level context. Do not treat
the labels as authoritative.

NO LLM CALLS. Pure regex + line walking.

Usage:
    python dump_classifier.py path/to/dump.txt
    python dump_classifier.py path/to/dump.txt --output json
    python dump_classifier.py --sample
"""

import argparse
import json
import re
import sys
from pathlib import Path
from typing import Any, Dict, List, Tuple


# Pattern → label, ordered by precedence (first match wins per line).
# Each pattern is a compiled regex.
PATTERNS: List[Tuple[re.Pattern, str]] = [
    (re.compile(r"\bdecide\s+(?:between|on|whether)\b", re.IGNORECASE), "decision"),
    (re.compile(r"^\s*decide\s*:", re.IGNORECASE), "decision"),
    (re.compile(r"\b(?:should we|do we|are we)\b.*\b(?:or|vs|versus)\b", re.IGNORECASE), "decision"),
    (re.compile(r"\?\s*$"), "question"),
    (re.compile(r"^\s*(?:resolve|q)\s*:", re.IGNORECASE), "question"),
    (re.compile(r"\bwhat\s+if\b", re.IGNORECASE), "idea"),
    (re.compile(r"\b(?:maybe|might|could)\s+(?:we|i)\s+(?:should\s+)?", re.IGNORECASE), "idea"),
    (re.compile(r"\bidea\s*:", re.IGNORECASE), "idea"),
    (re.compile(r"^\s*(?:i\s+(?:should|need\s+to|gotta|have\s+to))\b", re.IGNORECASE), "task"),
    (re.compile(r"^\s*(?:fix|build|write|draft|email|send|talk to|finish|investigate|research|sketch|scaffold|deploy|push|merge|review|read|call|schedule)\b", re.IGNORECASE), "task"),
    (re.compile(r"^\s*todo\s*:", re.IGNORECASE), "task"),
    (re.compile(r"^\s*-\s*(?:fix|build|write|draft|email|send|talk to|finish|investigate)\b", re.IGNORECASE), "task"),
]

PROJECT_COMPONENT_HINTS = {
    "module", "feature", "component", "endpoint", "page", "screen", "form",
    "model", "schema", "migration", "test", "doc", "readme", "config",
}


SAMPLE_DUMP = """Ok dump time.

Q3 launch is approaching - need to nail down pricing.
Draft the launch email.
Brief Sarah on the marketing angle.

Decide: launch on July 15 or August 1?

Ferret app keeps nagging me. Should I talk to my cofounder about it?
What if it's actually a real business?
Sketch the matching algorithm if it's serious.

Fix the damn auth bug.
Rewrite the login form because it's ugly.
Write tests for both.
Auth: 2fa module.

Do my Q3 OKRs before launch.
"""


def classify_line(raw: str) -> str:
    line = raw.strip()
    if not line:
        return "blank"

    # Strip leading bullet markers for matching, but keep original for output
    stripped = re.sub(r"^[-*+]\s+", "", line)

    for pattern, label in PATTERNS:
        if pattern.search(stripped):
            return label

    # Project-component heuristic: short noun-phrase containing a hint word
    if len(stripped.split()) <= 6:
        for hint in PROJECT_COMPONENT_HINTS:
            if re.search(rf"\b{hint}s?\b", stripped, re.IGNORECASE):
                return "project-component"

    # Single-noun-phrase or short fragment without verb → likely context or component
    if len(stripped.split()) <= 4 and not stripped.endswith("?"):
        return "project-component"

    # Default: treat as context (the skill folds this into project framing)
    return "context"


def classify(text: str) -> Dict[str, Any]:
    items: List[Dict[str, Any]] = []
    counts: Dict[str, int] = {}
    for line_no, raw in enumerate(text.splitlines(), start=1):
        if not raw.strip():
            continue
        label = classify_line(raw)
        if label == "blank":
            continue
        items.append({"line": line_no, "label": label, "text": raw.strip()})
        counts[label] = counts.get(label, 0) + 1
    return {"item_count": len(items), "by_label": counts, "items": items}


def render_human(result: Dict[str, Any]) -> str:
    out: List[str] = []
    out.append(f"Dump classification ({result['item_count']} non-empty items)")
    out.append("By label:")
    for label, n in sorted(result["by_label"].items(), key=lambda kv: -kv[1]):
        out.append(f"  {label:<20s} {n}")
    out.append("")
    out.append("Per-line labels:")
    for it in result["items"]:
        out.append(f"  L{it['line']:>3}  {it['label']:<20s} {it['text'][:80]}")
    return "\n".join(out)


def main(argv: List[str]) -> int:
    parser = argparse.ArgumentParser(description=__doc__.split("\n")[0])
    parser.add_argument("path", nargs="?", help="Path to dump file (or omit for --sample)")
    parser.add_argument("--sample", action="store_true", help="Classify the embedded sample dump")
    parser.add_argument("--output", choices=["human", "json"], default="human")
    args = parser.parse_args(argv)

    if args.sample:
        text = SAMPLE_DUMP
    elif args.path:
        p = Path(args.path)
        if not p.exists():
            print(f"error: {args.path} not found", file=sys.stderr)
            return 2
        text = p.read_text(encoding="utf-8")
    else:
        parser.print_help()
        return 0

    result = classify(text)
    if args.output == "json":
        print(json.dumps(result, indent=2))
    else:
        print(render_human(result))
    return 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))

FILE:scripts/workspace_inventory.py
#!/usr/bin/env python3
"""workspace_inventory.py — Glob+Grep helper for capture's Section 3 (Connections).

Stdlib-only. Given a working directory + a list of dump-derived keywords,
returns a structured inventory that the capture skill can use to surface
real workspace connections (never fabricated).

What it returns:
  1. Per-keyword filename matches (Glob-style)
  2. Per-keyword content matches (line-grep across source files)
  3. Top-level folder structure (max-depth 2)

What it does NOT do:
  - Score relevance (capture skill applies judgment on top)
  - Fabricate matches (only real Glob/Grep results)
  - Make LLM calls

Usage:
    python workspace_inventory.py --root . --keywords "auth,login,2fa"
    python workspace_inventory.py --root . --keywords "skill,megaprompt" --output json
    python workspace_inventory.py --sample
"""

import argparse
import json
import re
import sys
from pathlib import Path
from typing import Any, Dict, List, Set


DEFAULT_SOURCE_EXTENSIONS = {
    ".py", ".ts", ".tsx", ".js", ".jsx", ".go", ".java", ".kt", ".rb",
    ".cs", ".rs", ".swift", ".php", ".scala", ".clj", ".ex", ".exs",
    ".md", ".mdx", ".rst", ".txt", ".json", ".yaml", ".yml", ".toml",
}
DEFAULT_EXCLUDE_DIRS = {
    "node_modules", ".git", "dist", "build", "target",
    ".venv", "venv", "__pycache__", ".next", ".cache",
}
MAX_FILENAME_MATCHES_PER_KEYWORD = 20
MAX_CONTENT_MATCHES_PER_KEYWORD = 10
MAX_FOLDER_DEPTH = 2


SAMPLE_TREE: Dict[str, str] = {
    "src/auth/login.ts": "// auth login flow handler\nexport function login() {}\n",
    "src/auth/2fa.ts": "// 2fa stub\nexport function setup2FA() {}\n",
    "src/users/profile.ts": "// user profile\nexport function getProfile() {}\n",
    "docs/auth-bugs.md": "# Auth bugs\n\n- The login race condition is back.\n",
    "tests/auth.test.ts": "// auth tests\ndescribe('auth', () => {});\n",
    "README.md": "# project\n\nAuth + login + users.\n",
}


def collect_files(root: Path, source_extensions: Set[str], exclude_dirs: Set[str]) -> List[Path]:
    found: List[Path] = []
    for p in root.rglob("*"):
        if p.is_dir():
            continue
        if any(part in exclude_dirs for part in p.parts):
            continue
        if p.suffix.lower() in source_extensions:
            found.append(p)
    return found


def filename_matches(files: List[Path], keyword: str) -> List[str]:
    kw = keyword.lower()
    out: List[str] = []
    for p in files:
        if kw in p.name.lower():
            out.append(str(p))
        if len(out) >= MAX_FILENAME_MATCHES_PER_KEYWORD:
            break
    return out


def content_matches(files: List[Path], keyword: str) -> List[Dict[str, Any]]:
    pattern = re.compile(re.escape(keyword), re.IGNORECASE)
    out: List[Dict[str, Any]] = []
    for p in files:
        try:
            text = p.read_text(encoding="utf-8", errors="ignore")
        except OSError:
            continue
        for line_no, line in enumerate(text.splitlines(), start=1):
            if pattern.search(line):
                out.append({
                    "file": str(p),
                    "line": line_no,
                    "snippet": line.strip()[:120],
                })
                if len(out) >= MAX_CONTENT_MATCHES_PER_KEYWORD:
                    return out
    return out


def folder_structure(root: Path, max_depth: int, exclude_dirs: Set[str]) -> List[str]:
    out: List[str] = []
    root = root.resolve()
    for p in root.rglob("*"):
        if not p.is_dir():
            continue
        if any(part in exclude_dirs for part in p.parts):
            continue
        try:
            rel = p.relative_to(root)
        except ValueError:
            continue
        depth = len(rel.parts)
        if 0 < depth <= max_depth:
            out.append(str(rel))
    return sorted(out)


def inventory(
    root: Path,
    keywords: List[str],
    source_extensions: Set[str],
    exclude_dirs: Set[str],
) -> Dict[str, Any]:
    files = collect_files(root, source_extensions, exclude_dirs)
    per_keyword: Dict[str, Dict[str, Any]] = {}
    for kw in keywords:
        per_keyword[kw] = {
            "filename_matches": filename_matches(files, kw),
            "content_matches": content_matches(files, kw),
        }
    return {
        "root": str(root.resolve()),
        "files_scanned": len(files),
        "folder_structure": folder_structure(root, MAX_FOLDER_DEPTH, exclude_dirs),
        "per_keyword": per_keyword,
    }


def render_human(result: Dict[str, Any]) -> str:
    out: List[str] = []
    out.append(f"Workspace inventory for: {result['root']}")
    out.append(f"  Files scanned: {result['files_scanned']}")
    out.append("")
    out.append("Folder structure (max depth 2):")
    for f in result["folder_structure"][:30]:
        out.append(f"  - {f}/")
    if len(result["folder_structure"]) > 30:
        out.append(f"  ... + {len(result['folder_structure']) - 30} more")
    out.append("")
    for kw, hits in result["per_keyword"].items():
        out.append(f"Keyword: '{kw}'")
        if hits["filename_matches"]:
            out.append(f"  Filename matches ({len(hits['filename_matches'])}):")
            for f in hits["filename_matches"]:
                out.append(f"    - {f}")
        else:
            out.append("  Filename matches: (none)")
        if hits["content_matches"]:
            out.append(f"  Content matches ({len(hits['content_matches'])}):")
            for m in hits["content_matches"]:
                out.append(f"    - {m['file']}:{m['line']}  {m['snippet']}")
        else:
            out.append("  Content matches: (none)")
        out.append("")
    return "\n".join(out)


def run_sample(keywords: List[str]) -> Dict[str, Any]:
    import tempfile
    with tempfile.TemporaryDirectory() as td:
        root = Path(td)
        for rel, content in SAMPLE_TREE.items():
            p = root / rel
            p.parent.mkdir(parents=True, exist_ok=True)
            p.write_text(content, encoding="utf-8")
        return inventory(root, keywords, DEFAULT_SOURCE_EXTENSIONS, DEFAULT_EXCLUDE_DIRS)


def main(argv: List[str]) -> int:
    parser = argparse.ArgumentParser(description=__doc__.split("\n")[0])
    parser.add_argument("--root", help="Root directory to inventory")
    parser.add_argument("--keywords", help="Comma-separated keywords to search for")
    parser.add_argument("--extensions", help="Comma-separated source extensions (default: common)")
    parser.add_argument("--sample", action="store_true", help="Inventory the embedded sample tree")
    parser.add_argument("--output", choices=["human", "json"], default="human")
    args = parser.parse_args(argv)

    if args.sample:
        sample_keywords = ["auth", "login", "2fa"] if not args.keywords else [k.strip() for k in args.keywords.split(",") if k.strip()]
        result = run_sample(sample_keywords)
    elif args.root and args.keywords:
        root = Path(args.root)
        if not root.exists():
            print(f"error: {args.root} not found", file=sys.stderr)
            return 2
        kws = [k.strip() for k in args.keywords.split(",") if k.strip()]
        if not kws:
            print("error: --keywords must list at least one keyword", file=sys.stderr)
            return 2
        if args.extensions:
            exts = {e.strip() if e.strip().startswith(".") else "." + e.strip() for e in args.extensions.split(",")}
        else:
            exts = DEFAULT_SOURCE_EXTENSIONS
        result = inventory(root, kws, exts, DEFAULT_EXCLUDE_DIRS)
    else:
        parser.print_help()
        return 0

    if args.output == "json":
        print(json.dumps(result, indent=2))
    else:
        print(render_human(result))
    return 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))

---
name: "cco-review"
description: "/cs:cco-review <plan> — Retention-obsessed Chief Customer Officer interrogation of any plan that touches customer retention, segmentation, CS team sizing, or CS team hiring."
---

# /cs:cco-review — CCO Forcing Questions

**Command:** `/cs:cco-review <plan>`

The retention-obsessed CCO pressure-tests any plan that touches customer experience. Six questions before any retention claim, segmentation change, CS team expansion, or major CS hire.

## When to Run

- Before any board narrative that includes a retention number
- Before approving a CS team headcount expansion
- Before re-segmenting the customer base or changing tier definitions
- Before launching a customer marketing or advocacy program
- Before a major CS hire (CSM, AM, Implementation, Customer Marketing)
- When NRR is "great" but churn complaints from CSMs are increasing
- Before deciding whether to add an AM role separate from CSM

## The Six CCO Questions

### 1. What's the GROSS retention rate?
**Not NRR. Gross.** NRR can hide a leaky bucket behind expansion.
- GRR healthy ≥ 90% at growth stage, ≥ 95% at scale
- If GRR < 85% but NRR > 100%, the product is failing for 15%+ of customers; expansion is masking the failure
- Run `retention_decomposition_analyzer.py`

### 2. What's the #1 reason customers leave?
**If you can't name it, you don't understand churn.**
- 7-category taxonomy: product_fit / competitor_loss / no_value_realized / pricing / champion_left / company_event / tactical_failure
- Preventable churn = product_fit + no_value_realized + tactical_failure
- If preventable > 50%, CS has clear leverage; if < 30%, churn is structural (ICP, market, competition)

### 3. What's the median time-to-value (TTV) by segment?
**Long TTV signals different problems by segment.**
- Long TTV in low tier = ICP misfit; downgrade or kill
- Long TTV in high tier = onboarding broken; fix the Implementation Manager handoff
- TTV is a leading indicator of GRR

### 4. Which customer would you fire today?
**If "none" — your segmentation is broken.**
- Some accounts cost more than they earn (support cost > 50% of ARR + low ICP fit)
- Run `customer_segmentation_designer.py` to surface kill list
- The 3 paths for kill candidates: non-renewal / downgrade-to-tech-touch / raise-price-to-cost-recover

### 5. What's the ARR-per-CSM ratio, and is the model pooled or named?
**Wrong model wastes capacity.**
- Strategic: named + exec sponsor, $300K-$1M ARR/CSM
- Enterprise: named, $500K-$2M
- Mid-market: pooled, $2M-$5M
- SMB: tech-touch, $5M+
- Run `cs_coverage_calculator.py` to size the team

### 6. Is CS in your comp plan, and how is it different from Sales comp?
**Misalignment is the leading indicator of CS failure.**
- CS comp: 70/30 base/variable typical
- Variable: 50% gross retention + 30% net retention + 20% activity
- Anti-pattern: comp CSMs on NPS — they game it
- Anti-pattern: comp CSMs same as Sales — they sell instead of serve

## Workflow

```bash
# 1. Retention decomposition (always start here)
python ../../../skills/chief-customer-officer-advisor/scripts/retention_decomposition_analyzer.py cohorts.json

# 2. Segmentation audit
python ../../../skills/chief-customer-officer-advisor/scripts/customer_segmentation_designer.py customers.json

# 3. Coverage sizing (if making CS team changes)
python ../../../skills/chief-customer-officer-advisor/scripts/cs_coverage_calculator.py book.json
```

## Output Format

```markdown
# CCO Review: <plan>
**Date:** YYYY-MM-DD

## The Decision Being Made
[one sentence — retention | segmentation | coverage | next hire]

## Retention (if applicable)
- GRR: X% (vs vanity NRR of Y%)
- Top churn driver: <category> at X% of churn
- Preventable churn: X% (CS-controllable)
- Leaky-bucket pattern? yes/no

## Segmentation (if applicable)
- Tier distribution: Strategic X / Enterprise X / Mid-market X / SMB X
- Kill list size: N customers (X% of customers, Y% of ARR)
- Upgrade candidates: N

## Coverage (if applicable)
- Current CSMs: N | Required now: M | Required 12mo: P
- Annual cost (12mo): $X
- Manager trigger fired: yes/no

## Org (if applicable)
- Next hire: <CSM | Support | AM | IM | CS Ops | Customer Marketing>
- Why this, not the alternative: <one line>
- Customer outcome unblocked: <specific>

## Verdict
🟢 SHIP | 🟡 SHARPEN | 🔴 BLOCK

## Next Steps
[3 concrete actions]
```

## Routing

- `/cs:cpo-review` — if churn root cause is product_fit or no_value_realized
- `/cs:cro-review` — if expansion math or comp alignment is in question
- `/cs:cfo-review` — for CS cost commitments and retention-impact-on-revenue
- `/cs:chro-review` — for CS hires, comp, ladder
- `/cs:decide` — log the verdict
- `/cs:freeze 30` — on multi-year CS comp plan changes

## Related

- Agent: [`cs-cco-advisor`](../../agents/cs-cco-advisor.md)
- Skill: [`chief-customer-officer-advisor`](../../../skills/chief-customer-officer-advisor/SKILL.md)
- Adjacent: `../../../../business-growth/` (tactical CS execution)

---

**Version:** 1.0.0

---
name: changelog
description: Generate changelogs from git history and validate conventional commits. Usage: /changelog <generate|lint> [options]
---

# /changelog

Generate Keep a Changelog entries from git history and validate commit message format.

## Usage

```
/changelog generate [--from-tag <tag>] [--to-tag <tag>]    Generate changelog entries
/changelog lint [--from-ref <ref>] [--to-ref <ref>]       Lint commit messages
```

## Examples

```
/changelog generate --from-tag v2.0.0
/changelog lint --from-ref main --to-ref dev
/changelog generate --from-tag v2.0.0 --to-tag v2.1.0 --format markdown
```

## Scripts
- `engineering/changelog-generator/scripts/generate_changelog.py` — Parse commits, render changelog (`--from-tag`, `--to-tag`, `--from-ref`, `--to-ref`, `--format markdown|json`)
- `engineering/changelog-generator/scripts/commit_linter.py` — Validate conventional commit format (`--from-ref`, `--to-ref`, `--strict`, `--format text|json`)

## Skill Reference
→ `engineering/changelog-generator/SKILL.md`

---
name: "changelog-generator"
description: "Produce consistent, auditable release notes from Conventional Commits. Separates commit parsing, semantic-bump logic, and changelog rendering for automated releases with editorial control. Use when cutting a release, generating CHANGELOG.md from git history, or automating release notes in CI."
---

# Changelog Generator

**Tier:** POWERFUL  
**Category:** Engineering  
**Domain:** Release Management / Documentation

## Overview

Use this skill to produce consistent, auditable release notes from Conventional Commits. It separates commit parsing, semantic bump logic, and changelog rendering so teams can automate releases without losing editorial control.

## Core Capabilities

- Parse commit messages using Conventional Commit rules
- Detect semantic bump (`major`, `minor`, `patch`) from commit stream
- Render Keep a Changelog sections (`Added`, `Changed`, `Fixed`, etc.)
- Generate release entries from git ranges or provided commit input
- Enforce commit format with a dedicated linter script
- Support CI integration via machine-readable JSON output

## When to Use

- Before publishing a release tag
- During CI to generate release notes automatically
- During PR checks to block invalid commit message formats
- In monorepos where package changelogs require scoped filtering
- When converting raw git history into user-facing notes

## Key Workflows

### 1. Generate Changelog Entry From Git

```bash
python3 scripts/generate_changelog.py \
  --from-tag v1.3.0 \
  --to-tag v1.4.0 \
  --next-version v1.4.0 \
  --format markdown
```

### 2. Generate Entry From stdin/File Input

```bash
git log v1.3.0..v1.4.0 --pretty=format:'%s' | \
  python3 scripts/generate_changelog.py --next-version v1.4.0 --format markdown

python3 scripts/generate_changelog.py --input commits.txt --next-version v1.4.0 --format json
```

### 3. Update `CHANGELOG.md`

```bash
python3 scripts/generate_changelog.py \
  --from-tag v1.3.0 \
  --to-tag HEAD \
  --next-version v1.4.0 \
  --write CHANGELOG.md
```

### 4. Lint Commits Before Merge

```bash
python3 scripts/commit_linter.py --from-ref origin/main --to-ref HEAD --strict --format text
```

Or file/stdin:

```bash
python3 scripts/commit_linter.py --input commits.txt --strict
cat commits.txt | python3 scripts/commit_linter.py --format json
```

## Conventional Commit Rules

Supported types:

- `feat`, `fix`, `perf`, `refactor`, `docs`, `test`, `build`, `ci`, `chore`
- `security`, `deprecated`, `remove`

Breaking changes:

- `type(scope)!: summary`
- Footer/body includes `BREAKING CHANGE:`

SemVer mapping:

- breaking -> `major`
- non-breaking `feat` -> `minor`
- all others -> `patch`

## Script Interfaces

- `python3 scripts/generate_changelog.py --help`
  - Reads commits from git or stdin/`--input`
  - Renders markdown or JSON
  - Optional in-place changelog prepend
- `python3 scripts/commit_linter.py --help`
  - Validates commit format
  - Returns non-zero in `--strict` mode on violations

## Common Pitfalls

1. Mixing merge commit messages with release commit parsing
2. Using vague commit summaries that cannot become release notes
3. Failing to include migration guidance for breaking changes
4. Treating docs/chore changes as user-facing features
5. Overwriting historical changelog sections instead of prepending

## Best Practices

1. Keep commits small and intent-driven.
2. Scope commit messages (`feat(api): ...`) in multi-package repos.
3. Enforce linter checks in PR pipelines.
4. Review generated markdown before publishing.
5. Tag releases only after changelog generation succeeds.
6. Keep an `[Unreleased]` section for manual curation when needed.

## References

- [references/ci-integration.md](references/ci-integration.md)
- [references/changelog-formatting-guide.md](references/changelog-formatting-guide.md)
- [references/monorepo-strategy.md](references/monorepo-strategy.md)
- [README.md](README.md)

## Release Governance

Use this release flow for predictability:

1. Lint commit history for target release range.
2. Generate changelog draft from commits.
3. Manually adjust wording for customer clarity.
4. Validate semver bump recommendation.
5. Tag release only after changelog is approved.

## Output Quality Checks

- Each bullet is user-meaningful, not implementation noise.
- Breaking changes include migration action.
- Security fixes are isolated in `Security` section.
- Sections with no entries are omitted.
- Duplicate bullets across sections are removed.

## CI Policy

- Run `commit_linter.py --strict` on all PRs.
- Block merge on invalid conventional commits.
- Auto-generate draft release notes on tag push.
- Require human approval before writing into `CHANGELOG.md` on main branch.

## Monorepo Guidance

- Prefer commit scopes aligned to package names.
- Filter commit stream by scope for package-specific releases.
- Keep infra-wide changes in root changelog.
- Store package changelogs near package roots for ownership clarity.

## Failure Handling

- If no valid conventional commits found: fail early, do not generate misleading empty notes.
- If git range invalid: surface explicit range in error output.
- If write target missing: create safe changelog header scaffolding.

FILE:README.md
# Changelog Generator

Automates release notes from Conventional Commits with Keep a Changelog output and strict commit linting. Designed for CI-friendly release workflows.

## Quick Start

```bash
# Generate entry from git range
python3 scripts/generate_changelog.py \
  --from-tag v1.2.0 \
  --to-tag v1.3.0 \
  --next-version v1.3.0 \
  --format markdown

# Lint commit subjects
python3 scripts/commit_linter.py --from-ref origin/main --to-ref HEAD --strict --format text
```

## Included Tools

- `scripts/generate_changelog.py`: parse commits, infer semver bump, render markdown/JSON, optional file prepend
- `scripts/commit_linter.py`: validate commit subjects against Conventional Commits rules

## References

- `references/ci-integration.md`
- `references/changelog-formatting-guide.md`
- `references/monorepo-strategy.md`

## Installation

### Claude Code

```bash
cp -R engineering/changelog-generator ~/.claude/skills/changelog-generator
```

### OpenAI Codex

```bash
cp -R engineering/changelog-generator ~/.codex/skills/changelog-generator
```

### OpenClaw

```bash
cp -R engineering/changelog-generator ~/.openclaw/skills/changelog-generator
```

FILE:references/changelog-formatting-guide.md
# Changelog Formatting Guide

Use Keep a Changelog section ordering:

1. Security
2. Added
3. Changed
4. Deprecated
5. Removed
6. Fixed

Rules:

- One bullet = one user-visible change.
- Lead with impact, not implementation detail.
- Keep bullets short and actionable.
- Include migration note for breaking changes.

FILE:references/ci-integration.md
# CI Integration Examples

## GitHub Actions

```yaml
name: Changelog Check
on: [pull_request]

jobs:
  changelog:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: python3 engineering/changelog-generator/scripts/commit_linter.py \
          --from-ref origin/main --to-ref HEAD --strict
```

## GitLab CI

```yaml
changelog_lint:
  image: python:3.12
  stage: test
  script:
    - python3 engineering/changelog-generator/scripts/commit_linter.py --to-ref HEAD --strict
```

FILE:references/monorepo-strategy.md
# Monorepo Changelog Strategy

## Approaches

| Strategy | When to use | Tradeoff |
|----------|-------------|----------|
| Single root changelog | Product-wide releases, small teams | Simple but loses package-level detail |
| Per-package changelogs | Independent versioning, large teams | Clear ownership but harder to see full picture |
| Hybrid model | Root summary + package-specific details | Best of both, more maintenance |

## Commit Scoping Pattern

Enforce scoped conventional commits to enable per-package filtering:

```
feat(payments): add Stripe webhook handler
fix(auth): handle expired refresh tokens
chore(infra): bump base Docker image
```

**Rules:**
- Scope must match a package/directory name exactly
- Unscoped commits go to root changelog only
- Multi-package changes get separate scoped commits (not one mega-commit)

## Filtering for Package Releases

```bash
# Generate changelog for 'payments' package only
git log v1.3.0..HEAD --pretty=format:'%s' | grep '^[a-z]*\(payments\)' | \
  python3 scripts/generate_changelog.py --next-version v1.4.0 --format markdown
```

## Ownership Model

- Package maintainers own their scoped changelog
- Platform/infra team owns root changelog
- CI enforces scope presence on all commits touching package directories
- Root changelog aggregates breaking changes from all packages for visibility

FILE:scripts/commit_linter.py
#!/usr/bin/env python3
"""Lint commit messages against Conventional Commits.

Input sources (priority order):
1) --input file (one commit subject per line)
2) stdin lines
3) git range via --from-ref/--to-ref

Use --strict for non-zero exit on violations.
"""

import argparse
import json
import re
import subprocess
import sys
from dataclasses import dataclass, asdict
from pathlib import Path
from typing import List, Optional


CONVENTIONAL_RE = re.compile(
    r"^(feat|fix|perf|refactor|docs|test|build|ci|chore|security|deprecated|remove)"
    r"(\([a-z0-9._/-]+\))?(!)?:\s+.{1,120}$"
)


class CLIError(Exception):
    """Raised for expected CLI errors."""


@dataclass
class LintReport:
    total: int
    valid: int
    invalid: int
    violations: List[str]


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="Validate conventional commit subjects.")
    parser.add_argument("--input", help="File with commit subjects (one per line).")
    parser.add_argument("--from-ref", help="Git ref start (exclusive).")
    parser.add_argument("--to-ref", help="Git ref end (inclusive).")
    parser.add_argument("--strict", action="store_true", help="Exit non-zero when violations exist.")
    parser.add_argument("--format", choices=["text", "json"], default="text", help="Output format.")
    return parser.parse_args()


def lines_from_file(path: str) -> List[str]:
    try:
        return [line.strip() for line in Path(path).read_text(encoding="utf-8").splitlines() if line.strip()]
    except Exception as exc:
        raise CLIError(f"Failed reading --input file: {exc}") from exc


def lines_from_stdin() -> List[str]:
    if sys.stdin.isatty():
        return []
    data = sys.stdin.read()
    return [line.strip() for line in data.splitlines() if line.strip()]


def lines_from_git(args: argparse.Namespace) -> List[str]:
    if not args.to_ref:
        return []
    range_spec = f"{args.from_ref}..{args.to_ref}" if args.from_ref else args.to_ref
    try:
        proc = subprocess.run(
            ["git", "log", range_spec, "--pretty=format:%s", "--no-merges"],
            text=True,
            capture_output=True,
            check=True,
        )
    except subprocess.CalledProcessError as exc:
        raise CLIError(f"git log failed for range '{range_spec}': {exc.stderr.strip()}") from exc
    return [line.strip() for line in proc.stdout.splitlines() if line.strip()]


def load_lines(args: argparse.Namespace) -> List[str]:
    if args.input:
        return lines_from_file(args.input)
    stdin_lines = lines_from_stdin()
    if stdin_lines:
        return stdin_lines
    git_lines = lines_from_git(args)
    if git_lines:
        return git_lines
    raise CLIError("No commit input found. Use --input, stdin, or --to-ref.")


def lint(lines: List[str]) -> LintReport:
    violations: List[str] = []
    valid = 0

    for idx, line in enumerate(lines, start=1):
        if CONVENTIONAL_RE.match(line):
            valid += 1
            continue
        violations.append(f"line {idx}: {line}")

    return LintReport(total=len(lines), valid=valid, invalid=len(violations), violations=violations)


def format_text(report: LintReport) -> str:
    lines = [
        "Conventional commit lint report",
        f"- total: {report.total}",
        f"- valid: {report.valid}",
        f"- invalid: {report.invalid}",
    ]
    if report.violations:
        lines.append("Violations:")
        lines.extend([f"- {v}" for v in report.violations])
    return "\n".join(lines)


def main() -> int:
    args = parse_args()
    lines = load_lines(args)
    report = lint(lines)

    if args.format == "json":
        print(json.dumps(asdict(report), indent=2))
    else:
        print(format_text(report))

    if args.strict and report.invalid > 0:
        return 1
    return 0


if __name__ == "__main__":
    try:
        raise SystemExit(main())
    except CLIError as exc:
        print(f"ERROR: {exc}", file=sys.stderr)
        raise SystemExit(2)

FILE:scripts/generate_changelog.py
#!/usr/bin/env python3
"""Generate changelog entries from Conventional Commits.

Input sources (priority order):
1) --input file with one commit subject per line
2) stdin commit subjects
3) git log from --from-tag/--to-tag or --from-ref/--to-ref

Outputs markdown or JSON and can prepend into CHANGELOG.md.
"""

import argparse
import json
import re
import subprocess
import sys
from dataclasses import dataclass, asdict, field
from datetime import date
from pathlib import Path
from typing import Dict, List, Optional


COMMIT_RE = re.compile(
    r"^(?P<type>feat|fix|perf|refactor|docs|test|build|ci|chore|security|deprecated|remove)"
    r"(?:\((?P<scope>[^)]+)\))?(?P<breaking>!)?:\s+(?P<summary>.+)$"
)

SECTION_MAP = {
    "feat": "Added",
    "fix": "Fixed",
    "perf": "Changed",
    "refactor": "Changed",
    "security": "Security",
    "deprecated": "Deprecated",
    "remove": "Removed",
}


class CLIError(Exception):
    """Raised for expected CLI failures."""


@dataclass
class ParsedCommit:
    raw: str
    ctype: str
    scope: Optional[str]
    summary: str
    breaking: bool


@dataclass
class ChangelogEntry:
    version: str
    release_date: str
    sections: Dict[str, List[str]] = field(default_factory=dict)
    breaking_changes: List[str] = field(default_factory=list)
    bump: str = "patch"


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="Generate changelog from conventional commits.")
    parser.add_argument("--input", help="Text file with one commit subject per line.")
    parser.add_argument("--from-tag", help="Git tag start (exclusive).")
    parser.add_argument("--to-tag", help="Git tag end (inclusive).")
    parser.add_argument("--from-ref", help="Git ref start (exclusive).")
    parser.add_argument("--to-ref", help="Git ref end (inclusive).")
    parser.add_argument("--next-version", default="Unreleased", help="Version label for the generated entry.")
    parser.add_argument("--date", dest="entry_date", default=str(date.today()), help="Release date (YYYY-MM-DD).")
    parser.add_argument("--format", choices=["markdown", "json"], default="markdown", help="Output format.")
    parser.add_argument("--write", help="Prepend generated markdown entry into this changelog file.")
    return parser.parse_args()


def read_lines_from_file(path: str) -> List[str]:
    try:
        return [line.strip() for line in Path(path).read_text(encoding="utf-8").splitlines() if line.strip()]
    except Exception as exc:
        raise CLIError(f"Failed reading --input file: {exc}") from exc


def read_lines_from_stdin() -> List[str]:
    if sys.stdin.isatty():
        return []
    payload = sys.stdin.read()
    return [line.strip() for line in payload.splitlines() if line.strip()]


def read_lines_from_git(args: argparse.Namespace) -> List[str]:
    if args.from_tag or args.to_tag:
        if not args.to_tag:
            raise CLIError("--to-tag is required when using tag range.")
        start = args.from_tag
        end = args.to_tag
    elif args.from_ref or args.to_ref:
        if not args.to_ref:
            raise CLIError("--to-ref is required when using ref range.")
        start = args.from_ref
        end = args.to_ref
    else:
        return []

    range_spec = f"{start}..{end}" if start else end
    try:
        proc = subprocess.run(
            ["git", "log", range_spec, "--pretty=format:%s", "--no-merges"],
            text=True,
            capture_output=True,
            check=True,
        )
    except subprocess.CalledProcessError as exc:
        raise CLIError(f"git log failed for range '{range_spec}': {exc.stderr.strip()}") from exc

    return [line.strip() for line in proc.stdout.splitlines() if line.strip()]


def load_commits(args: argparse.Namespace) -> List[str]:
    if args.input:
        return read_lines_from_file(args.input)

    stdin_lines = read_lines_from_stdin()
    if stdin_lines:
        return stdin_lines

    git_lines = read_lines_from_git(args)
    if git_lines:
        return git_lines

    raise CLIError("No commit input found. Use --input, stdin, or git range flags.")


def parse_commits(lines: List[str]) -> List[ParsedCommit]:
    parsed: List[ParsedCommit] = []
    for line in lines:
        match = COMMIT_RE.match(line)
        if not match:
            continue
        ctype = match.group("type")
        scope = match.group("scope")
        summary = match.group("summary")
        breaking = bool(match.group("breaking")) or "BREAKING CHANGE" in line
        parsed.append(ParsedCommit(raw=line, ctype=ctype, scope=scope, summary=summary, breaking=breaking))
    return parsed


def determine_bump(commits: List[ParsedCommit]) -> str:
    if any(c.breaking for c in commits):
        return "major"
    if any(c.ctype == "feat" for c in commits):
        return "minor"
    return "patch"


def build_entry(commits: List[ParsedCommit], version: str, entry_date: str) -> ChangelogEntry:
    sections: Dict[str, List[str]] = {
        "Security": [],
        "Added": [],
        "Changed": [],
        "Deprecated": [],
        "Removed": [],
        "Fixed": [],
    }
    breaking_changes: List[str] = []

    for commit in commits:
        if commit.breaking:
            breaking_changes.append(commit.summary)
        section = SECTION_MAP.get(commit.ctype)
        if section:
            line = commit.summary if not commit.scope else f"{commit.scope}: {commit.summary}"
            sections[section].append(line)

    sections = {k: v for k, v in sections.items() if v}
    return ChangelogEntry(
        version=version,
        release_date=entry_date,
        sections=sections,
        breaking_changes=breaking_changes,
        bump=determine_bump(commits),
    )


def render_markdown(entry: ChangelogEntry) -> str:
    lines = [f"## [{entry.version}] - {entry.release_date}", ""]
    if entry.breaking_changes:
        lines.append("### Breaking")
        lines.extend([f"- {item}" for item in entry.breaking_changes])
        lines.append("")

    ordered_sections = ["Security", "Added", "Changed", "Deprecated", "Removed", "Fixed"]
    for section in ordered_sections:
        items = entry.sections.get(section, [])
        if not items:
            continue
        lines.append(f"### {section}")
        lines.extend([f"- {item}" for item in items])
        lines.append("")

    lines.append(f"<!-- recommended-semver-bump: {entry.bump} -->")
    return "\n".join(lines).strip() + "\n"


def prepend_changelog(path: Path, entry_md: str) -> None:
    if path.exists():
        original = path.read_text(encoding="utf-8")
    else:
        original = "# Changelog\n\nAll notable changes to this project will be documented in this file.\n\n"

    if original.startswith("# Changelog"):
        first_break = original.find("\n")
        head = original[: first_break + 1]
        tail = original[first_break + 1 :].lstrip("\n")
        combined = f"{head}\n{entry_md}\n{tail}"
    else:
        combined = f"# Changelog\n\n{entry_md}\n{original}"
    path.write_text(combined, encoding="utf-8")


def main() -> int:
    args = parse_args()
    lines = load_commits(args)
    parsed = parse_commits(lines)
    if not parsed:
        raise CLIError("No valid conventional commit messages found in input.")

    entry = build_entry(parsed, args.next_version, args.entry_date)

    if args.format == "json":
        print(json.dumps(asdict(entry), indent=2))
    else:
        markdown = render_markdown(entry)
        print(markdown, end="")
        if args.write:
            prepend_changelog(Path(args.write), markdown)

    if args.format == "json" and args.write:
        prepend_changelog(Path(args.write), render_markdown(entry))

    return 0


if __name__ == "__main__":
    try:
        raise SystemExit(main())
    except CLIError as exc:
        print(f"ERROR: {exc}", file=sys.stderr)
        raise SystemExit(2)

---
name: code-to-prd
description: Reverse-engineer a frontend codebase into a PRD. Usage: /code-to-prd [path]
---

# /code-to-prd

Reverse-engineer a frontend codebase into a complete Product Requirements Document.

## Usage

```bash
/code-to-prd                    # Analyze current project
/code-to-prd ./src              # Analyze specific directory
/code-to-prd /path/to/project   # Analyze external project
```

## What It Does

1. **Scan** — Run `codebase_analyzer.py` to detect framework, routes, APIs, enums, and project structure
2. **Scaffold** — Run `prd_scaffolder.py` to create `prd/` directory with README.md, per-page stubs, and appendix files
3. **Analyze** — Walk through each page following the Phase 2 workflow: fields, interactions, API dependencies, page relationships
4. **Generate** — Produce the final PRD with all pages, enum dictionary, API inventory, and page relationship map

## Steps

### Step 1: Analyze

Determine the project path (default: current directory). Run the frontend analyzer:

```bash
python3 {skill_path}/scripts/codebase_analyzer.py {project_path} -o .code-to-prd-analysis.json
```

Display a summary of findings: framework, page count, API count, enum count.

### Step 2: Scaffold

Generate the PRD directory skeleton:

```bash
python3 {skill_path}/scripts/prd_scaffolder.py .code-to-prd-analysis.json -o prd/
```

### Step 3: Fill

For each page in the inventory, follow the SKILL.md Phase 2 workflow:
- Read the page's component files
- Document fields, interactions, API dependencies, page relationships
- Fill in the corresponding `prd/pages/` stub

Work in batches of 3-5 pages for large projects (>15 pages). Ask the user to confirm after each batch.

### Step 4: Finalize

Complete the appendix files:
- `prd/appendix/enum-dictionary.md` — all enums and status codes found
- `prd/appendix/api-inventory.md` — consolidated API reference
- `prd/appendix/page-relationships.md` — navigation and data coupling map

Clean up the temporary analysis file:
```bash
rm .code-to-prd-analysis.json
```

## Output

A `prd/` directory containing:
- `README.md` — system overview, module map, page inventory
- `pages/*.md` — one file per page with fields, interactions, APIs
- `appendix/*.md` — enum dictionary, API inventory, page relationships

## Skill Reference

- `product-team/code-to-prd/SKILL.md`
- `product-team/code-to-prd/scripts/codebase_analyzer.py`
- `product-team/code-to-prd/scripts/prd_scaffolder.py`
- `product-team/code-to-prd/references/prd-quality-checklist.md`