--- name: "dependency-auditor" description: "Audit and manage dependencies across multi-language projects. Identifies vulnerabilities, license conflicts, transitive dependency risks, and safe-upgrade paths. Use when auditing third-party packages before release, investigating a CVE, planning a major version bump, or running a license-compliance review." --- # Dependency Auditor > **Skill Type:** POWERFUL > **Category:** Engineering > **Domain:** Dependency Management & Security ## Overview The **Dependency Auditor** is a comprehensive toolkit for analyzing, auditing, and managing dependencies across multi-language software projects. This skill provides deep visibility into your project's dependency ecosystem, enabling teams to identify vulnerabilities, ensure license compliance, optimize dependency trees, and plan safe upgrades. In modern software development, dependencies form complex webs that can introduce significant security, legal, and maintenance risks. A single project might have hundreds of direct and transitive dependencies, each potentially introducing vulnerabilities, license conflicts, or maintenance burden. This skill addresses these challenges through automated analysis and actionable recommendations. ## Core Capabilities ### 1. Vulnerability Scanning & CVE Matching **Comprehensive Security Analysis** - Scans dependencies against built-in vulnerability databases - Matches Common Vulnerabilities and Exposures (CVE) patterns - Identifies known security issues across multiple ecosystems - Analyzes transitive dependency vulnerabilities - Provides CVSS scores and exploit assessments - Tracks vulnerability disclosure timelines - Maps vulnerabilities to dependency paths **Multi-Language Support** - **JavaScript/Node.js**: package.json, package-lock.json, yarn.lock - **Python**: requirements.txt, pyproject.toml, Pipfile.lock, poetry.lock - **Go**: go.mod, go.sum - **Rust**: Cargo.toml, Cargo.lock - **Ruby**: Gemfile, Gemfile.lock - **Java/Maven**: pom.xml, gradle.lockfile - **PHP**: composer.json, composer.lock - **C#/.NET**: packages.config, project.assets.json ### 2. License Compliance & Legal Risk Assessment **License Classification System** - **Permissive Licenses**: MIT, Apache 2.0, BSD (2-clause, 3-clause), ISC - **Copyleft (Strong)**: GPL (v2, v3), AGPL (v3) - **Copyleft (Weak)**: LGPL (v2.1, v3), MPL (v2.0) - **Proprietary**: Commercial, custom, or restrictive licenses - **Dual Licensed**: Multi-license scenarios and compatibility - **Unknown/Ambiguous**: Missing or unclear licensing **Conflict Detection** - Identifies incompatible license combinations - Warns about GPL contamination in permissive projects - Analyzes license inheritance through dependency chains - Provides compliance recommendations for distribution - Generates legal risk matrices for decision-making ### 3. Outdated Dependency Detection **Version Analysis** - Identifies dependencies with available updates - Categorizes updates by severity (patch, minor, major) - Detects pinned versions that may be outdated - Analyzes semantic versioning patterns - Identifies floating version specifiers - Tracks release frequencies and maintenance status **Maintenance Status Assessment** - Identifies abandoned or unmaintained packages - Analyzes commit frequency and contributor activity - Tracks last release dates and security patch availability - Identifies packages with known end-of-life dates - Assesses upstream maintenance quality ### 4. Dependency Bloat Analysis **Unused Dependency Detection** - Identifies dependencies that aren't actually imported/used - Analyzes import statements and usage patterns - Detects redundant dependencies with overlapping functionality - Identifies oversized packages for simple use cases - Maps actual vs. declared dependency usage **Redundancy Analysis** - Identifies multiple packages providing similar functionality - Detects version conflicts in transitive dependencies - Analyzes bundle size impact of dependencies - Identifies opportunities for dependency consolidation - Maps dependency overlap and duplication ### 5. Upgrade Path Planning & Breaking Change Risk **Semantic Versioning Analysis** - Analyzes semver patterns to predict breaking changes - Identifies safe upgrade paths (patch/minor versions) - Flags major version updates requiring attention - Tracks breaking changes across dependency updates - Provides rollback strategies for failed upgrades **Risk Assessment Matrix** - Low Risk: Patch updates, security fixes - Medium Risk: Minor updates with new features - High Risk: Major version updates, API changes - Critical Risk: Dependencies with known breaking changes **Upgrade Prioritization** - Security patches: Highest priority - Bug fixes: High priority - Feature updates: Medium priority - Major rewrites: Planned priority - Deprecated features: Immediate attention ### 6. Supply Chain Security **Dependency Provenance** - Verifies package signatures and checksums - Analyzes package download sources and mirrors - Identifies suspicious or compromised packages - Tracks package ownership changes and maintainer shifts - Detects typosquatting and malicious packages **Transitive Risk Analysis** - Maps complete dependency trees - Identifies high-risk transitive dependencies - Analyzes dependency depth and complexity - Tracks influence of indirect dependencies - Provides supply chain risk scoring ### 7. Lockfile Analysis & Deterministic Builds **Lockfile Validation** - Ensures lockfiles are up-to-date with manifests - Validates integrity hashes and version consistency - Identifies drift between environments - Analyzes lockfile conflicts and resolution strategies - Ensures deterministic, reproducible builds **Environment Consistency** - Compares dependencies across environments (dev/staging/prod) - Identifies version mismatches between team members - Validates CI/CD environment consistency - Tracks dependency resolution differences ## Technical Architecture ### Scanner Engine (`dep_scanner.py`) - Multi-format parser supporting 8+ package ecosystems - Built-in vulnerability database with 500+ CVE patterns - Transitive dependency resolution from lockfiles - JSON and human-readable output formats - Configurable scanning depth and exclusion patterns ### License Analyzer (`license_checker.py`) - License detection from package metadata and files - Compatibility matrix with 20+ license types - Conflict detection engine with remediation suggestions - Risk scoring based on distribution and usage context - Export capabilities for legal review ### Upgrade Planner (`upgrade_planner.py`) - Semantic version analysis with breaking change prediction - Dependency ordering based on risk and interdependence - Migration checklists with testing recommendations - Rollback procedures for failed upgrades - Timeline estimation for upgrade cycles ## Use Cases & Applications ### Security Teams - **Vulnerability Management**: Continuous scanning for security issues - **Incident Response**: Rapid assessment of vulnerable dependencies - **Supply Chain Monitoring**: Tracking third-party security posture - **Compliance Reporting**: Automated security compliance documentation ### Legal & Compliance Teams - **License Auditing**: Comprehensive license compliance verification - **Risk Assessment**: Legal risk analysis for software distribution - **Due Diligence**: Dependency licensing for M&A activities - **Policy Enforcement**: Automated license policy compliance ### Development Teams - **Dependency Hygiene**: Regular cleanup of unused dependencies - **Upgrade Planning**: Strategic dependency update scheduling - **Performance Optimization**: Bundle size optimization through dep analysis - **Technical Debt**: Identifying and prioritizing dependency technical debt ### DevOps & Platform Teams - **Build Optimization**: Faster builds through dependency optimization - **Security Automation**: Automated vulnerability scanning in CI/CD - **Environment Consistency**: Ensuring consistent dependencies across environments - **Release Management**: Dependency-aware release planning ## Integration Patterns ### CI/CD Pipeline Integration ```bash # Security gate in CI python dep_scanner.py /project --format json --fail-on-high python license_checker.py /project --policy strict --format json ``` ### Scheduled Audits ```bash # Weekly dependency audit ./audit_dependencies.sh > weekly_report.html python upgrade_planner.py deps.json --timeline 30days ``` ### Development Workflow ```bash # Pre-commit dependency check python dep_scanner.py . --quick-scan python license_checker.py . --warn-conflicts ``` ## Advanced Features ### Custom Vulnerability Databases - Support for internal/proprietary vulnerability feeds - Custom CVE pattern definitions - Organization-specific risk scoring - Integration with enterprise security tools ### Policy-Based Scanning - Configurable license policies by project type - Custom risk thresholds and escalation rules - Automated policy enforcement and notifications - Exception management for approved violations ### Reporting & Dashboards - Executive summaries for management - Technical reports for development teams - Trend analysis and dependency health metrics - Integration with project management tools ### Multi-Project Analysis - Portfolio-level dependency analysis - Shared dependency impact analysis - Organization-wide license compliance - Cross-project vulnerability propagation ## Best Practices ### Scanning Frequency - **Security Scans**: Daily or on every commit - **License Audits**: Weekly or monthly - **Upgrade Planning**: Monthly or quarterly - **Full Dependency Audit**: Quarterly ### Risk Management 1. **Prioritize Security**: Address high/critical CVEs immediately 2. **License First**: Ensure compliance before functionality 3. **Gradual Updates**: Incremental dependency updates 4. **Test Thoroughly**: Comprehensive testing after updates 5. **Monitor Continuously**: Automated monitoring and alerting ### Team Workflows 1. **Security Champions**: Designate dependency security owners 2. **Review Process**: Mandatory review for new dependencies 3. **Update Cycles**: Regular, scheduled dependency updates 4. **Documentation**: Maintain dependency rationale and decisions 5. **Training**: Regular team education on dependency security ## Metrics & KPIs ### Security Metrics - Mean Time to Patch (MTTP) for vulnerabilities - Number of high/critical vulnerabilities - Percentage of dependencies with known vulnerabilities - Security debt accumulation rate ### Compliance Metrics - License compliance percentage - Number of license conflicts - Time to resolve compliance issues - Policy violation frequency ### Maintenance Metrics - Percentage of up-to-date dependencies - Average dependency age - Number of abandoned dependencies - Upgrade success rate ### Efficiency Metrics - Bundle size reduction percentage - Unused dependency elimination rate - Build time improvement - Developer productivity impact ## Troubleshooting Guide ### Common Issues 1. **False Positives**: Tuning vulnerability detection sensitivity 2. **License Ambiguity**: Resolving unclear or multiple licenses 3. **Breaking Changes**: Managing major version upgrades 4. **Performance Impact**: Optimizing scanning for large codebases ### Resolution Strategies - Whitelist false positives with documentation - Contact maintainers for license clarification - Implement feature flags for risky upgrades - Use incremental scanning for large projects ## Future Enhancements ### Planned Features - Machine learning for vulnerability prediction - Automated dependency update pull requests - Integration with container image scanning - Real-time dependency monitoring dashboards - Natural language policy definition ### Ecosystem Expansion - Additional language support (Swift, Kotlin, Dart) - Container and infrastructure dependencies - Development tool and build system dependencies - Cloud service and SaaS dependency tracking --- ## Quick Start ```bash # Scan project for vulnerabilities and licenses python scripts/dep_scanner.py /path/to/project # Check license compliance python scripts/license_checker.py /path/to/project --policy strict # Plan dependency upgrades python scripts/upgrade_planner.py deps.json --risk-threshold medium ``` For detailed usage instructions, see [README.md](README.md). --- *This skill provides comprehensive dependency management capabilities essential for maintaining secure, compliant, and efficient software projects. Regular use helps teams stay ahead of security threats, maintain legal compliance, and optimize their dependency ecosystems.* FILE:assets/sample_go.mod module github.com/example/sample-go-service go 1.20 require ( github.com/gin-gonic/gin v1.9.1 github.com/go-redis/redis/v8 v8.11.5 github.com/golang-jwt/jwt/v4 v4.5.0 github.com/gorilla/mux v1.8.0 github.com/gorilla/websocket v1.5.0 github.com/lib/pq v1.10.9 github.com/stretchr/testify v1.8.2 go.uber.org/zap v1.24.0 golang.org/x/crypto v0.9.0 gopkg.in/yaml.v3 v3.0.1 gorm.io/driver/postgres v1.5.0 gorm.io/gorm v1.25.1 ) require ( github.com/bytedance/sonic v1.8.8 // indirect github.com/cespare/xxhash/v2 v2.2.0 // indirect github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311 // indirect github.com/davecgh/go-spew v1.1.1 // indirect github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect github.com/gabriel-vasile/mimetype v1.4.2 // indirect github.com/gin-contrib/sse v0.1.0 // indirect github.com/go-playground/locales v0.14.1 // indirect github.com/go-playground/universal-translator v0.18.1 // indirect github.com/go-playground/validator/v10 v10.13.0 // indirect github.com/goccy/go-json v0.10.2 // indirect github.com/jackc/pgpassfile v1.0.0 // indirect github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a // indirect github.com/jackc/pgx/v5 v5.3.1 // indirect github.com/jinzhu/inflection v1.0.0 // indirect github.com/jinzhu/now v1.1.5 // indirect github.com/json-iterator/go v1.1.12 // indirect github.com/klauspost/cpuid/v2 v2.2.4 // indirect github.com/leodido/go-urn v1.2.4 // indirect github.com/mattn/go-isatty v0.0.18 // indirect github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect github.com/modern-go/reflect2 v1.0.2 // indirect github.com/pelletier/go-toml/v2 v2.0.7 // indirect github.com/pmezard/go-difflib v1.0.0 // indirect github.com/twitchyliquid64/golang-asm v0.15.1 // indirect github.com/ugorji/go/codec v1.2.11 // indirect go.uber.org/atomic v1.11.0 // indirect go.uber.org/multierr v1.11.0 // indirect golang.org/x/arch v0.3.0 // indirect golang.org/x/net v0.10.0 // indirect golang.org/x/sys v0.8.0 // indirect golang.org/x/text v0.9.0 // indirect ) FILE:assets/sample_package.json { "name": "sample-web-app", "version": "1.2.3", "description": "A sample web application with various dependencies for testing dependency auditing", "main": "index.js", "scripts": { "start": "node index.js", "dev": "nodemon index.js", "build": "webpack --mode production", "test": "jest", "lint": "eslint src/", "audit": "npm audit" }, "keywords": ["web", "app", "sample", "dependency", "audit"], "author": "Claude Skills Team", "license": "MIT", "dependencies": { "express": "4.18.1", "lodash": "4.17.20", "axios": "1.5.0", "jsonwebtoken": "8.5.1", "bcrypt": "5.1.0", "mongoose": "6.10.0", "cors": "2.8.5", "helmet": "6.1.5", "winston": "3.8.2", "dotenv": "16.0.3", "express-rate-limit": "6.7.0", "multer": "1.4.5-lts.1", "sharp": "0.32.1", "nodemailer": "6.9.1", "socket.io": "4.6.1", "redis": "4.6.5", "moment": "2.29.4", "chalk": "4.1.2", "commander": "9.4.1" }, "devDependencies": { "nodemon": "2.0.22", "jest": "29.5.0", "supertest": "6.3.3", "eslint": "8.40.0", "eslint-config-airbnb-base": "15.0.0", "eslint-plugin-import": "2.27.5", "webpack": "5.82.1", "webpack-cli": "5.1.1", "babel-loader": "9.1.2", "@babel/core": "7.22.1", "@babel/preset-env": "7.22.2", "css-loader": "6.7.4", "style-loader": "3.3.3", "html-webpack-plugin": "5.5.1", "mini-css-extract-plugin": "2.7.6", "postcss": "8.4.23", "postcss-loader": "7.3.0", "autoprefixer": "10.4.14", "cross-env": "7.0.3", "rimraf": "5.0.1" }, "engines": { "node": ">=16.0.0", "npm": ">=8.0.0" }, "repository": { "type": "git", "url": "https://github.com/example/sample-web-app.git" }, "bugs": { "url": "https://github.com/example/sample-web-app/issues" }, "homepage": "https://github.com/example/sample-web-app#readme" } FILE:assets/sample_requirements.txt # Core web framework Django==4.1.7 djangorestframework==3.14.0 django-cors-headers==3.14.0 django-environ==0.10.0 django-extensions==3.2.1 # Database and ORM psycopg2-binary==2.9.6 redis==4.5.4 celery==5.2.7 # Authentication and Security django-allauth==0.54.0 djangorestframework-simplejwt==5.2.2 cryptography==40.0.1 bcrypt==4.0.1 # HTTP and API clients requests==2.28.2 httpx==0.24.1 urllib3==1.26.15 # Data processing and analysis pandas==2.0.1 numpy==1.24.3 Pillow==9.5.0 openpyxl==3.1.2 # Monitoring and logging sentry-sdk==1.21.1 structlog==23.1.0 # Testing pytest==7.3.1 pytest-django==4.5.2 pytest-cov==4.0.0 factory-boy==3.2.1 freezegun==1.2.2 # Development tools black==23.3.0 flake8==6.0.0 isort==5.12.0 pre-commit==3.3.2 django-debug-toolbar==4.0.0 # Documentation Sphinx==6.2.1 sphinx-rtd-theme==1.2.0 # Deployment and server gunicorn==20.1.0 whitenoise==6.4.0 # Environment and configuration python-decouple==3.8 pyyaml==6.0 # Utilities click==8.1.3 python-dateutil==2.8.2 pytz==2023.3 six==1.16.0 # AWS integration boto3==1.26.137 botocore==1.29.137 # Email django-anymail==10.0 FILE:expected_outputs/sample_license_report.txt ============================================================ LICENSE COMPLIANCE REPORT ============================================================ Analysis Date: 2024-02-16T15:30:00.000Z Project: /example/sample-web-app Project License: MIT SUMMARY: Total Dependencies: 23 Compliance Score: 92.5/100 Overall Risk: LOW License Conflicts: 0 LICENSE DISTRIBUTION: Permissive: 21 Copyleft_weak: 1 Copyleft_strong: 0 Proprietary: 0 Unknown: 1 RISK BREAKDOWN: Low: 21 Medium: 1 High: 0 Critical: 1 HIGH-RISK DEPENDENCIES: ------------------------------ moment v2.29.4: Unknown (CRITICAL) RECOMMENDATIONS: -------------------- 1. Investigate and clarify licenses for 1 dependencies with unknown licensing 2. Overall compliance score is high - maintain current practices 3. Consider updating moment.js which has been deprecated by maintainers ============================================================ FILE:expected_outputs/sample_upgrade_plan.txt ============================================================ DEPENDENCY UPGRADE PLAN ============================================================ Generated: 2024-02-16T15:30:00.000Z Timeline: 90 days UPGRADE SUMMARY: Total Upgrades Available: 12 Security Updates: 2 Major Version Updates: 3 High Risk Updates: 2 RISK ASSESSMENT: Overall Risk Level: MEDIUM Key Risk Factors: • 2 critical risk upgrades requiring careful planning • Core framework upgrades: ['express', 'webpack', 'eslint'] • 1 major version upgrades with potential breaking changes TOP PRIORITY UPGRADES: ------------------------------ 🔒 lodash: 4.17.20 → 4.17.21 🔒 Type: Patch | Risk: Low | Priority: 95.0 Security: CVE-2021-23337: Prototype pollution vulnerability 🟡 express: 4.18.1 → 4.18.2 Type: Patch | Risk: Low | Priority: 85.0 🟡 webpack: 5.82.1 → 5.88.0 Type: Minor | Risk: Medium | Priority: 75.0 🔴 eslint: 8.40.0 → 9.0.0 Type: Major | Risk: High | Priority: 65.0 🟢 cors: 2.8.5 → 2.8.7 Type: Patch | Risk: Safe | Priority: 80.0 PHASED UPGRADE PLANS: ------------------------------ Phase 1: Security & Safe Updates (30 days) Dependencies: lodash, cors, helmet, dotenv, bcrypt Key Steps: Create feature branch; Update dependency versions in manifest files; Run dependency install/update commands Phase 2: Regular Updates (36 days) Dependencies: express, axios, winston, multer Key Steps: Create feature branch; Update dependency versions in manifest files; Run dependency install/update commands Phase 3: Major Updates (30 days) Dependencies: webpack, eslint, jest ... and 2 more Key Steps: Create feature branch; Update dependency versions in manifest files; Run dependency install/update commands RECOMMENDATIONS: -------------------- 1. URGENT: 2 security updates available - prioritize immediately 2. Quick wins: 6 safe updates can be applied with minimal risk 3. Plan carefully: 2 high-risk upgrades need thorough testing ============================================================ FILE:expected_outputs/sample_vulnerability_report.json { "timestamp": "2024-02-16T15:30:00.000Z", "project_path": "/example/sample-web-app", "dependencies": [ { "name": "lodash", "version": "4.17.20", "ecosystem": "npm", "direct": true, "license": "MIT", "vulnerabilities": [ { "id": "CVE-2021-23337", "summary": "Prototype pollution in lodash", "severity": "HIGH", "cvss_score": 7.2, "affected_versions": "<4.17.21", "fixed_version": "4.17.21", "published_date": "2021-02-15", "references": [ "https://nvd.nist.gov/vuln/detail/CVE-2021-23337" ] } ] }, { "name": "axios", "version": "1.5.0", "ecosystem": "npm", "direct": true, "license": "MIT", "vulnerabilities": [] }, { "name": "express", "version": "4.18.1", "ecosystem": "npm", "direct": true, "license": "MIT", "vulnerabilities": [] }, { "name": "jsonwebtoken", "version": "8.5.1", "ecosystem": "npm", "direct": true, "license": "MIT", "vulnerabilities": [] } ], "vulnerabilities_found": 1, "high_severity_count": 1, "medium_severity_count": 0, "low_severity_count": 0, "ecosystems": ["npm"], "scan_summary": { "total_dependencies": 4, "unique_dependencies": 4, "ecosystems_found": 1, "vulnerable_dependencies": 1, "vulnerability_breakdown": { "high": 1, "medium": 0, "low": 0 } }, "recommendations": [ "URGENT: Address 1 high-severity vulnerabilities immediately", "Update lodash from 4.17.20 to 4.17.21 to fix CVE-2021-23337" ] } FILE:README.md # Dependency Auditor A comprehensive toolkit for analyzing, auditing, and managing dependencies across multi-language software projects. This skill provides vulnerability scanning, license compliance checking, and upgrade path planning with zero external dependencies. ## Overview The Dependency Auditor skill consists of three main Python scripts that work together to provide complete dependency management capabilities: - **`dep_scanner.py`**: Vulnerability scanning and dependency analysis - **`license_checker.py`**: License compliance and conflict detection - **`upgrade_planner.py`**: Upgrade path planning and risk assessment ## Features ### 🔍 Vulnerability Scanning - Multi-language dependency parsing (JavaScript, Python, Go, Rust, Ruby, Java) - Built-in vulnerability database with common CVE patterns - CVSS scoring and risk assessment - JSON and human-readable output formats - CI/CD integration support ### ⚖️ License Compliance - Comprehensive license classification and compatibility analysis - Automatic conflict detection between project and dependency licenses - Risk assessment for commercial usage and distribution - Compliance scoring and reporting ### 📈 Upgrade Planning - Semantic versioning analysis with breaking change prediction - Risk-based upgrade prioritization - Phased migration plans with rollback procedures - Security-focused upgrade recommendations ## Installation No external dependencies required! All scripts use only Python standard library. ```bash # Clone or download the dependency-auditor skill cd engineering/dependency-auditor/scripts # Make scripts executable chmod +x dep_scanner.py license_checker.py upgrade_planner.py ``` ## Quick Start ### 1. Scan for Vulnerabilities ```bash # Basic vulnerability scan python dep_scanner.py /path/to/your/project # JSON output for automation python dep_scanner.py /path/to/your/project --format json --output scan_results.json # Fail CI/CD on high-severity vulnerabilities python dep_scanner.py /path/to/your/project --fail-on-high ``` ### 2. Check License Compliance ```bash # Basic license compliance check python license_checker.py /path/to/your/project # Strict policy enforcement python license_checker.py /path/to/your/project --policy strict # Use existing dependency inventory python license_checker.py /path/to/project --inventory scan_results.json --format json ``` ### 3. Plan Dependency Upgrades ```bash # Generate upgrade plan from dependency inventory python upgrade_planner.py scan_results.json # Custom timeline and risk filtering python upgrade_planner.py scan_results.json --timeline 60 --risk-threshold medium # Security updates only python upgrade_planner.py scan_results.json --security-only --format json ``` ## Detailed Usage ### Dependency Scanner (`dep_scanner.py`) The dependency scanner parses project files to extract dependencies and check them against a built-in vulnerability database. #### Supported File Formats - **JavaScript/Node.js**: package.json, package-lock.json, yarn.lock - **Python**: requirements.txt, pyproject.toml, Pipfile.lock, poetry.lock - **Go**: go.mod, go.sum - **Rust**: Cargo.toml, Cargo.lock - **Ruby**: Gemfile, Gemfile.lock #### Command Line Options ```bash python dep_scanner.py [PROJECT_PATH] [OPTIONS] Required Arguments: PROJECT_PATH Path to the project directory to scan Optional Arguments: --format {text,json} Output format (default: text) --output FILE Output file path (default: stdout) --fail-on-high Exit with error code if high-severity vulnerabilities found --quick-scan Perform quick scan (skip transitive dependencies) Examples: python dep_scanner.py /app python dep_scanner.py . --format json --output results.json python dep_scanner.py /project --fail-on-high --quick-scan ``` #### Output Format **Text Output:** ``` ============================================================ DEPENDENCY SECURITY SCAN REPORT ============================================================ Scan Date: 2024-02-16T15:30:00.000Z Project: /example/sample-web-app SUMMARY: Total Dependencies: 23 Unique Dependencies: 19 Ecosystems: npm Vulnerabilities Found: 1 High Severity: 1 Medium Severity: 0 Low Severity: 0 VULNERABLE DEPENDENCIES: ------------------------------ Package: lodash v4.17.20 (npm) • CVE-2021-23337: Prototype pollution in lodash Severity: HIGH (CVSS: 7.2) Fixed in: 4.17.21 RECOMMENDATIONS: -------------------- 1. URGENT: Address 1 high-severity vulnerabilities immediately 2. Update lodash from 4.17.20 to 4.17.21 to fix CVE-2021-23337 ``` **JSON Output:** ```json { "timestamp": "2024-02-16T15:30:00.000Z", "project_path": "/example/sample-web-app", "dependencies": [ { "name": "lodash", "version": "4.17.20", "ecosystem": "npm", "direct": true, "vulnerabilities": [ { "id": "CVE-2021-23337", "summary": "Prototype pollution in lodash", "severity": "HIGH", "cvss_score": 7.2 } ] } ], "recommendations": [ "Update lodash from 4.17.20 to 4.17.21 to fix CVE-2021-23337" ] } ``` ### License Checker (`license_checker.py`) The license checker analyzes dependency licenses for compliance and detects potential conflicts. #### Command Line Options ```bash python license_checker.py [PROJECT_PATH] [OPTIONS] Required Arguments: PROJECT_PATH Path to the project directory to analyze Optional Arguments: --inventory FILE Path to dependency inventory JSON file --format {text,json} Output format (default: text) --output FILE Output file path (default: stdout) --policy {permissive,strict} License policy strictness (default: permissive) --warn-conflicts Show warnings for potential conflicts Examples: python license_checker.py /app python license_checker.py . --format json --output compliance.json python license_checker.py /app --inventory deps.json --policy strict ``` #### License Classifications The tool classifies licenses into risk categories: - **Permissive (Low Risk)**: MIT, Apache-2.0, BSD, ISC - **Weak Copyleft (Medium Risk)**: LGPL, MPL - **Strong Copyleft (High Risk)**: GPL, AGPL - **Proprietary (High Risk)**: Commercial licenses - **Unknown (Critical Risk)**: Unidentified licenses #### Compatibility Matrix The tool includes a comprehensive compatibility matrix that checks: - Project license vs. dependency licenses - GPL contamination detection - Commercial usage restrictions - Distribution requirements ### Upgrade Planner (`upgrade_planner.py`) The upgrade planner analyzes dependency inventories and creates prioritized upgrade plans. #### Command Line Options ```bash python upgrade_planner.py [INVENTORY_FILE] [OPTIONS] Required Arguments: INVENTORY_FILE Path to dependency inventory JSON file Optional Arguments: --timeline DAYS Timeline for upgrade plan in days (default: 90) --format {text,json} Output format (default: text) --output FILE Output file path (default: stdout) --risk-threshold {safe,low,medium,high,critical} Maximum risk level (default: high) --security-only Only plan upgrades with security fixes Examples: python upgrade_planner.py deps.json python upgrade_planner.py inventory.json --timeline 60 --format json python upgrade_planner.py deps.json --security-only --risk-threshold medium ``` #### Risk Assessment Upgrades are classified by risk level: - **Safe**: Patch updates with no breaking changes - **Low**: Minor updates with backward compatibility - **Medium**: Updates with potential API changes - **High**: Major version updates with breaking changes - **Critical**: Updates affecting core functionality #### Phased Planning The tool creates three-phase upgrade plans: 1. **Phase 1 (30% of timeline)**: Security fixes and safe updates 2. **Phase 2 (40% of timeline)**: Regular maintenance updates 3. **Phase 3 (30% of timeline)**: Major updates requiring careful planning ## Integration Examples ### CI/CD Pipeline Integration #### GitHub Actions Example ```yaml name: Dependency Audit on: [push, pull_request, schedule] jobs: audit: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Setup Python uses: actions/setup-python@v4 with: python-version: '3.9' - name: Run Vulnerability Scan run: | python scripts/dep_scanner.py . --format json --output scan.json python scripts/dep_scanner.py . --fail-on-high - name: Check License Compliance run: | python scripts/license_checker.py . --inventory scan.json --policy strict - name: Generate Upgrade Plan run: | python scripts/upgrade_planner.py scan.json --output upgrade-plan.txt - name: Upload Reports uses: actions/upload-artifact@v3 with: name: dependency-reports path: | scan.json upgrade-plan.txt ``` #### Jenkins Pipeline Example ```groovy pipeline { agent any stages { stage('Dependency Audit') { steps { script { // Vulnerability scan sh 'python scripts/dep_scanner.py . --format json --output scan.json' // License compliance sh 'python scripts/license_checker.py . --inventory scan.json --format json --output compliance.json' // Upgrade planning sh 'python scripts/upgrade_planner.py scan.json --format json --output upgrades.json' } // Archive reports archiveArtifacts artifacts: '*.json', fingerprint: true // Fail build on high-severity vulnerabilities sh 'python scripts/dep_scanner.py . --fail-on-high' } } } post { always { // Publish reports publishHTML([ allowMissing: false, alwaysLinkToLastBuild: true, keepAll: true, reportDir: '.', reportFiles: '*.json', reportName: 'Dependency Audit Report' ]) } } } ``` ### Automated Dependency Updates #### Weekly Security Updates Script ```bash #!/bin/bash # weekly-security-updates.sh set -e echo "Running weekly security dependency updates..." # Scan for vulnerabilities python scripts/dep_scanner.py . --format json --output current-scan.json # Generate security-only upgrade plan python scripts/upgrade_planner.py current-scan.json --security-only --output security-upgrades.txt # Check if security updates are available if grep -q "URGENT" security-upgrades.txt; then echo "Security updates found! Creating automated PR..." # Create branch git checkout -b "automated-security-updates-$(date +%Y%m%d)" # Apply updates (example for npm) npm audit fix --only=prod # Commit and push git add . git commit -m "chore: automated security dependency updates" git push origin HEAD # Create PR (using GitHub CLI) gh pr create \ --title "Automated Security Updates" \ --body-file security-upgrades.txt \ --label "security,dependencies,automated" else echo "No critical security updates found." fi ``` ## Sample Files The `assets/` directory contains sample dependency files for testing: - `sample_package.json`: Node.js project with various dependencies - `sample_requirements.txt`: Python project dependencies - `sample_go.mod`: Go module dependencies The `expected_outputs/` directory contains example reports showing the expected format and content. ## Advanced Usage ### Custom Vulnerability Database You can extend the built-in vulnerability database by modifying the `_load_vulnerability_database()` method in `dep_scanner.py`: ```python def _load_vulnerability_database(self): """Load vulnerability database from multiple sources.""" db = self._load_builtin_database() # Load custom vulnerabilities custom_db_path = os.environ.get('CUSTOM_VULN_DB') if custom_db_path and os.path.exists(custom_db_path): with open(custom_db_path, 'r') as f: custom_vulns = json.load(f) db.update(custom_vulns) return db ``` ### Custom License Policies Create custom license policies by modifying the license database: ```python # Add custom license custom_license = LicenseInfo( name='Custom Internal License', spdx_id='CUSTOM-1.0', license_type=LicenseType.PROPRIETARY, risk_level=RiskLevel.HIGH, description='Internal company license', restrictions=['Internal use only'], obligations=['Attribution required'] ) ``` ### Multi-Project Analysis For analyzing multiple projects, create a wrapper script: ```python #!/usr/bin/env python3 import os import json from pathlib import Path projects = ['/path/to/project1', '/path/to/project2', '/path/to/project3'] results = {} for project in projects: project_name = Path(project).name # Run vulnerability scan scan_result = subprocess.run([ 'python', 'scripts/dep_scanner.py', project, '--format', 'json' ], capture_output=True, text=True) if scan_result.returncode == 0: results[project_name] = json.loads(scan_result.stdout) # Generate consolidated report with open('consolidated-report.json', 'w') as f: json.dump(results, f, indent=2) ``` ## Troubleshooting ### Common Issues 1. **Permission Errors** ```bash chmod +x scripts/*.py ``` 2. **Python Version Compatibility** - Requires Python 3.7 or higher - Uses only standard library modules 3. **Large Projects** - Use `--quick-scan` for faster analysis - Consider excluding large node_modules directories 4. **False Positives** - Review vulnerability matches manually - Consider version range parsing improvements ### Debug Mode Enable debug logging by setting environment variable: ```bash export DEPENDENCY_AUDIT_DEBUG=1 python scripts/dep_scanner.py /your/project ``` ## Contributing 1. **Adding New Package Managers**: Extend the `supported_files` dictionary and add corresponding parsers 2. **Vulnerability Database**: Add new CVE entries to the built-in database 3. **License Support**: Add new license types to the license database 4. **Risk Assessment**: Improve risk scoring algorithms ## References - [SKILL.md](SKILL.md): Comprehensive skill documentation - [references/](references/): Best practices and compatibility guides - [assets/](assets/): Sample dependency files for testing - [expected_outputs/](expected_outputs/): Example reports and outputs ## License This skill is licensed under the MIT License. See the project license file for details. --- **Note**: This tool provides automated analysis to assist with dependency management decisions. Always review recommendations and consult with security and legal teams for critical applications. FILE:references/dependency_management_best_practices.md # Dependency Management Best Practices A comprehensive guide to effective dependency management across the software development lifecycle, covering strategy, governance, security, and operational practices. ## Strategic Foundation ### Dependency Strategy #### Philosophy and Principles 1. **Minimize Dependencies**: Every dependency is a liability - Prefer standard library solutions when possible - Evaluate alternatives before adding new dependencies - Regularly audit and remove unused dependencies 2. **Quality Over Convenience**: Choose well-maintained, secure dependencies - Active maintenance and community - Strong security track record - Comprehensive documentation and testing 3. **Stability Over Novelty**: Prefer proven, stable solutions - Avoid dependencies with frequent breaking changes - Consider long-term support and backwards compatibility - Evaluate dependency maturity and adoption 4. **Transparency and Control**: Understand what you're depending on - Review dependency source code when possible - Understand licensing implications - Monitor dependency behavior and updates #### Decision Framework ##### Evaluation Criteria ``` Dependency Evaluation Scorecard: │ ├── Necessity (25 points) │ ├── Problem complexity (10) │ ├── Standard library alternatives (8) │ └── Internal implementation effort (7) │ ├── Quality (30 points) │ ├── Code quality and architecture (10) │ ├── Test coverage and reliability (10) │ └── Documentation completeness (10) │ ├── Maintenance (25 points) │ ├── Active development and releases (10) │ ├── Issue response time (8) │ └── Community size and engagement (7) │ └── Compatibility (20 points) ├── License compatibility (10) ├── Version stability (5) └── Platform/runtime compatibility (5) Scoring: - 80-100: Excellent choice - 60-79: Good choice with monitoring - 40-59: Acceptable with caution - Below 40: Avoid or find alternatives ``` ### Governance Framework #### Dependency Approval Process ##### New Dependency Approval ``` New Dependency Workflow: │ 1. Developer identifies need ├── Documents use case and requirements ├── Researches available options └── Proposes recommendation ↓ 2. Technical review ├── Architecture team evaluates fit ├── Security team assesses risks └── Legal team reviews licensing ↓ 3. Management approval ├── Low risk: Tech lead approval ├── Medium risk: Architecture board └── High risk: CTO approval ↓ 4. Implementation ├── Add to approved dependencies list ├── Document usage guidelines └── Configure monitoring and alerts ``` ##### Risk Classification - **Low Risk**: Well-known libraries, permissive licenses, stable APIs - **Medium Risk**: Less common libraries, weak copyleft licenses, evolving APIs - **High Risk**: New/experimental libraries, strong copyleft licenses, breaking changes #### Dependency Policies ##### Licensing Policy ```yaml licensing_policy: allowed_licenses: - MIT - Apache-2.0 - BSD-3-Clause - BSD-2-Clause - ISC conditional_licenses: - LGPL-2.1 # Library linking only - LGPL-3.0 # With legal review - MPL-2.0 # File-level copyleft acceptable prohibited_licenses: - GPL-2.0 # Strong copyleft - GPL-3.0 # Strong copyleft - AGPL-3.0 # Network copyleft - SSPL # Server-side public license - Custom # Unknown/proprietary licenses exceptions: process: "Legal and executive approval required" documentation: "Risk assessment and mitigation plan" ``` ##### Security Policy ```yaml security_policy: vulnerability_response: critical: "24 hours" high: "1 week" medium: "1 month" low: "Next release cycle" scanning_requirements: frequency: "Daily automated scans" tools: ["Snyk", "OWASP Dependency Check"] ci_cd_integration: "Mandatory security gates" approval_thresholds: known_vulnerabilities: "Zero tolerance for high/critical" maintenance_status: "Must be actively maintained" community_size: "Minimum 10 contributors or enterprise backing" ``` ## Operational Practices ### Dependency Lifecycle Management #### Addition Process 1. **Research and Evaluation** ```bash # Example evaluation script #!/bin/bash PACKAGE=$1 echo "=== Package Analysis: $PACKAGE ===" # Check package stats npm view $PACKAGE # Security audit npm audit $PACKAGE # License check npm view $PACKAGE license # Dependency tree npm ls $PACKAGE # Recent activity npm view $PACKAGE --json | jq '.time' ``` 2. **Documentation Requirements** - **Purpose**: Why this dependency is needed - **Alternatives**: Other options considered and why rejected - **Risk Assessment**: Security, licensing, maintenance risks - **Usage Guidelines**: How to use safely within the project - **Exit Strategy**: How to remove/replace if needed 3. **Integration Standards** - Pin to specific versions (avoid wildcards) - Document version constraints and reasoning - Configure automated update policies - Add monitoring and alerting #### Update Management ##### Update Strategy ``` Update Prioritization: │ ├── Security Updates (P0) │ ├── Critical vulnerabilities: Immediate │ ├── High vulnerabilities: Within 1 week │ └── Medium vulnerabilities: Within 1 month │ ├── Maintenance Updates (P1) │ ├── Bug fixes: Next minor release │ ├── Performance improvements: Next minor release │ └── Deprecation warnings: Plan for major release │ └── Feature Updates (P2) ├── Minor versions: Quarterly review ├── Major versions: Annual planning cycle └── Breaking changes: Dedicated migration projects ``` ##### Update Process ```yaml update_workflow: automated: patch_updates: enabled: true auto_merge: true conditions: - tests_pass: true - security_scan_clean: true - no_breaking_changes: true minor_updates: enabled: true auto_merge: false requires: "Manual review and testing" major_updates: enabled: false requires: "Full impact assessment and planning" testing_requirements: unit_tests: "100% pass rate" integration_tests: "Full test suite" security_tests: "Vulnerability scan clean" performance_tests: "No regression" rollback_plan: automated: "Failed CI/CD triggers automatic rollback" manual: "Documented rollback procedure" monitoring: "Real-time health checks post-deployment" ``` #### Removal Process 1. **Deprecation Planning** - Identify deprecated/unused dependencies - Assess removal impact and effort - Plan migration timeline and strategy - Communicate to stakeholders 2. **Safe Removal** ```bash # Example removal checklist echo "Dependency Removal Checklist:" echo "1. [ ] Grep codebase for all imports/usage" echo "2. [ ] Check if any other dependencies require it" echo "3. [ ] Remove from package files" echo "4. [ ] Run full test suite" echo "5. [ ] Update documentation" echo "6. [ ] Deploy with monitoring" ``` ### Version Management #### Semantic Versioning Strategy ##### Version Pinning Policies ```yaml version_pinning: production_dependencies: strategy: "Exact pinning" example: "react: 18.2.0" rationale: "Predictable builds, security control" development_dependencies: strategy: "Compatible range" example: "eslint: ^8.0.0" rationale: "Allow bug fixes and improvements" internal_libraries: strategy: "Compatible range" example: "^1.2.0" rationale: "Internal control, faster iteration" ``` ##### Update Windows - **Patch Updates (x.y.Z)**: Allow automatically with testing - **Minor Updates (x.Y.z)**: Review monthly, apply quarterly - **Major Updates (X.y.z)**: Annual review cycle, planned migrations #### Lockfile Management ##### Best Practices 1. **Always Commit Lockfiles** - package-lock.json (npm) - yarn.lock (Yarn) - Pipfile.lock (Python) - Cargo.lock (Rust) - go.sum (Go) 2. **Lockfile Validation** ```bash # Example CI validation - name: Validate lockfile run: | npm ci --audit npm audit --audit-level moderate # Verify lockfile is up to date npm install --package-lock-only git diff --exit-code package-lock.json ``` 3. **Regeneration Policy** - Regenerate monthly or after significant updates - Always regenerate after security updates - Document regeneration in change logs ## Security Management ### Vulnerability Management #### Continuous Monitoring ```yaml monitoring_stack: scanning_tools: - name: "Snyk" scope: "All ecosystems" frequency: "Daily" integration: "CI/CD + IDE" - name: "GitHub Dependabot" scope: "GitHub repositories" frequency: "Real-time" integration: "Pull requests" - name: "OWASP Dependency Check" scope: "Java/.NET focus" frequency: "Build pipeline" integration: "CI/CD gates" alerting: channels: ["Slack", "Email", "PagerDuty"] escalation: critical: "Immediate notification" high: "Within 1 hour" medium: "Daily digest" ``` #### Response Procedures ##### Critical Vulnerability Response ``` Critical Vulnerability (CVSS 9.0+) Response: │ 0-2 hours: Detection & Assessment ├── Automated scan identifies vulnerability ├── Security team notified immediately └── Initial impact assessment started │ 2-6 hours: Planning & Communication ├── Detailed impact analysis completed ├── Fix strategy determined ├── Stakeholder communication initiated └── Emergency change approval obtained │ 6-24 hours: Implementation & Testing ├── Fix implemented in development ├── Security testing performed ├── Limited rollout to staging └── Production deployment prepared │ 24-48 hours: Deployment & Validation ├── Production deployment executed ├── Monitoring and validation performed ├── Post-deployment testing completed └── Incident documentation finalized ``` ### Supply Chain Security #### Source Verification 1. **Package Authenticity** - Verify package signatures when available - Use official package registries - Check package maintainer reputation - Validate download checksums 2. **Build Reproducibility** - Use deterministic builds where possible - Pin dependency versions exactly - Document build environment requirements - Maintain build artifact checksums #### Dependency Provenance ```yaml provenance_tracking: metadata_collection: - package_name: "Library identification" - version: "Exact version used" - source_url: "Official repository" - maintainer: "Package maintainer info" - license: "License verification" - checksum: "Content verification" verification_process: - signature_check: "GPG signature validation" - reputation_check: "Maintainer history review" - content_analysis: "Static code analysis" - behavior_monitoring: "Runtime behavior analysis" ``` ## Multi-Language Considerations ### Ecosystem-Specific Practices #### JavaScript/Node.js ```json { "npm_practices": { "package_json": { "engines": "Specify Node.js version requirements", "dependencies": "Production dependencies only", "devDependencies": "Development tools and testing", "optionalDependencies": "Use sparingly, document why" }, "security": { "npm_audit": "Run in CI/CD pipeline", "package_lock": "Always commit to repository", "registry": "Use official npm registry or approved mirrors" }, "performance": { "bundle_analysis": "Regular bundle size monitoring", "tree_shaking": "Ensure unused code is eliminated", "code_splitting": "Lazy load dependencies when possible" } } } ``` #### Python ```yaml python_practices: dependency_files: requirements.txt: "Pin exact versions for production" requirements-dev.txt: "Development dependencies" setup.py: "Package distribution metadata" pyproject.toml: "Modern Python packaging" virtual_environments: purpose: "Isolate project dependencies" tools: ["venv", "virtualenv", "conda", "poetry"] best_practice: "One environment per project" security: tools: ["safety", "pip-audit", "bandit"] practices: ["Pin versions", "Use private PyPI if needed"] ``` #### Java/Maven ```xml 5.3.21 5.8.2 org.springframework spring-bom ${spring.version} pom import ``` ### Cross-Language Integration #### API Boundaries - Define clear service interfaces - Use standard protocols (HTTP, gRPC) - Document API contracts - Version APIs independently #### Shared Dependencies - Minimize shared dependencies across services - Use containerization for isolation - Document shared dependency policies - Monitor for version conflicts ## Performance and Optimization ### Bundle Size Management #### Analysis Tools ```bash # JavaScript bundle analysis npm install -g webpack-bundle-analyzer webpack-bundle-analyzer dist/main.js # Python package size analysis pip install pip-audit pip-audit --format json | jq '.dependencies[].package_size' # General dependency tree analysis dep-tree analyze --format json --output deps.json ``` #### Optimization Strategies 1. **Tree Shaking**: Remove unused code 2. **Code Splitting**: Load dependencies on demand 3. **Polyfill Optimization**: Only include needed polyfills 4. **Alternative Packages**: Choose smaller alternatives when possible ### Build Performance #### Dependency Caching ```yaml # Example CI/CD caching cache_strategy: node_modules: key: "npm-{{ checksum 'package-lock.json' }}" paths: ["~/.npm", "node_modules"] pip_cache: key: "pip-{{ checksum 'requirements.txt' }}" paths: ["~/.cache/pip"] maven_cache: key: "maven-{{ checksum 'pom.xml' }}" paths: ["~/.m2/repository"] ``` #### Parallel Installation - Configure package managers for parallel downloads - Use local package caches - Consider dependency proxies for enterprise environments ## Monitoring and Metrics ### Key Performance Indicators #### Security Metrics ```yaml security_kpis: vulnerability_metrics: - mean_time_to_detection: "Average time to identify vulnerabilities" - mean_time_to_patch: "Average time to fix vulnerabilities" - vulnerability_density: "Vulnerabilities per 1000 dependencies" - false_positive_rate: "Percentage of false vulnerability reports" compliance_metrics: - license_compliance_rate: "Percentage of compliant dependencies" - policy_violation_rate: "Rate of policy violations" - security_gate_success_rate: "CI/CD security gate pass rate" ``` #### Operational Metrics ```yaml operational_kpis: maintenance_metrics: - dependency_freshness: "Average age of dependencies" - update_frequency: "Rate of dependency updates" - technical_debt: "Number of outdated dependencies" performance_metrics: - build_time: "Time to install/build dependencies" - bundle_size: "Final application size" - dependency_count: "Total number of dependencies" ``` ### Dashboard and Reporting #### Executive Dashboard - Overall risk score and trend - Security compliance status - Cost of dependency management - Policy violation summary #### Technical Dashboard - Vulnerability count by severity - Outdated dependency count - Build performance metrics - License compliance details #### Automated Reports - Weekly security summary - Monthly compliance report - Quarterly dependency review - Annual strategy assessment ## Team Organization and Training ### Roles and Responsibilities #### Security Champions - Monitor security advisories - Review dependency security scans - Coordinate vulnerability responses - Maintain security policies #### Platform Engineers - Maintain dependency management infrastructure - Configure automated scanning and updates - Manage package registries and mirrors - Support development teams #### Development Teams - Follow dependency policies - Perform regular security updates - Document dependency decisions - Participate in security training ### Training Programs #### Security Training - Dependency security fundamentals - Vulnerability assessment and response - Secure coding practices - Supply chain attack awareness #### Tool Training - Package manager best practices - Security scanning tool usage - CI/CD security integration - Incident response procedures ## Conclusion Effective dependency management requires a holistic approach combining technical practices, organizational policies, and cultural awareness. Key success factors: 1. **Proactive Strategy**: Plan dependency management from project inception 2. **Clear Governance**: Establish and enforce dependency policies 3. **Automated Processes**: Use tools to scale security and maintenance 4. **Continuous Monitoring**: Stay informed about dependency risks and updates 5. **Team Training**: Ensure all team members understand security implications 6. **Regular Review**: Periodically assess and improve dependency practices Remember that dependency management is an investment in long-term project health, security, and maintainability. The upfront effort to establish good practices pays dividends in reduced security risks, easier maintenance, and more stable software systems. FILE:references/license_compatibility_matrix.md # License Compatibility Matrix This document provides a comprehensive reference for understanding license compatibility when combining open source software dependencies in your projects. ## Understanding License Types ### Permissive Licenses - **MIT License**: Very permissive, allows commercial use, modification, and distribution - **Apache 2.0**: Permissive with patent grant and trademark restrictions - **BSD 3-Clause**: Permissive with non-endorsement clause - **BSD 2-Clause**: Simple permissive license - **ISC License**: Functionally equivalent to MIT ### Weak Copyleft Licenses - **LGPL 2.1/3.0**: Library-level copyleft, allows linking but requires modifications to be shared - **MPL 2.0**: File-level copyleft, compatible with many licenses ### Strong Copyleft Licenses - **GPL 2.0/3.0**: Requires entire derivative work to be GPL-licensed - **AGPL 3.0**: Extends GPL to network services (SaaS applications) ## Compatibility Matrix | Project License | MIT | Apache-2.0 | BSD-3 | LGPL-2.1 | LGPL-3.0 | MPL-2.0 | GPL-2.0 | GPL-3.0 | AGPL-3.0 | |----------------|-----|------------|-------|----------|----------|---------|---------|---------|----------| | **MIT** | ✅ | ✅ | ✅ | ⚠️ | ⚠️ | ⚠️ | ❌ | ❌ | ❌ | | **Apache-2.0** | ✅ | ✅ | ✅ | ❌ | ⚠️ | ✅ | ❌ | ⚠️ | ⚠️ | | **BSD-3** | ✅ | ✅ | ✅ | ⚠️ | ⚠️ | ⚠️ | ❌ | ❌ | ❌ | | **LGPL-2.1** | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | | **LGPL-3.0** | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ | ✅ | | **MPL-2.0** | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ | ✅ | | **GPL-2.0** | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | | **GPL-3.0** | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ | ✅ | | **AGPL-3.0** | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ | ✅ | **Legend:** - ✅ Generally Compatible - ⚠️ Compatible with conditions/restrictions - ❌ Incompatible ## Detailed Compatibility Rules ### MIT Project with Other Licenses **Compatible:** - MIT, Apache-2.0, BSD (all variants), ISC: Full compatibility - LGPL 2.1/3.0: Can use LGPL libraries via dynamic linking - MPL 2.0: Can use MPL modules, must keep MPL files under MPL **Incompatible:** - GPL 2.0/3.0: GPL requires entire project to be GPL - AGPL 3.0: AGPL extends to network services ### Apache 2.0 Project with Other Licenses **Compatible:** - MIT, BSD, ISC: Full compatibility - LGPL 3.0: Compatible (LGPL 3.0 has Apache compatibility clause) - MPL 2.0: Compatible - GPL 3.0: Compatible (GPL 3.0 has Apache compatibility clause) **Incompatible:** - LGPL 2.1: License incompatibility - GPL 2.0: License incompatibility (no Apache clause) ### GPL Projects **GPL 2.0 Compatible:** - MIT, BSD, ISC: Can incorporate permissive code - LGPL 2.1: Compatible - Other GPL 2.0: Compatible **GPL 2.0 Incompatible:** - Apache 2.0: Different patent clauses - LGPL 3.0: Version incompatibility - GPL 3.0: Version incompatibility **GPL 3.0 Compatible:** - All permissive licenses (MIT, Apache, BSD, ISC) - LGPL 3.0: Version compatibility - MPL 2.0: Explicit compatibility ## Common Compatibility Scenarios ### Scenario 1: Permissive Project with GPL Dependency **Problem:** MIT-licensed project wants to use GPL library **Impact:** Entire project must become GPL-licensed **Solutions:** 1. Find alternative non-GPL library 2. Use dynamic linking (if possible) 3. Change project license to GPL 4. Remove the dependency ### Scenario 2: Apache Project with GPL 2.0 Dependency **Problem:** Apache 2.0 project with GPL 2.0 dependency **Impact:** License incompatibility due to patent clauses **Solutions:** 1. Upgrade to GPL 3.0 if available 2. Find alternative library 3. Use via separate service (API boundary) ### Scenario 3: Commercial Product with AGPL Dependency **Problem:** Proprietary software using AGPL library **Impact:** AGPL copyleft extends to network services **Solutions:** 1. Obtain commercial license 2. Replace with permissive alternative 3. Use via separate service with API boundary 4. Make entire application AGPL ## License Combination Rules ### Safe Combinations 1. **Permissive + Permissive**: Always safe 2. **Permissive + Weak Copyleft**: Usually safe with proper attribution 3. **GPL + Compatible Permissive**: Safe, result is GPL ### Risky Combinations 1. **Apache 2.0 + GPL 2.0**: Incompatible patent terms 2. **Different GPL versions**: Version compatibility issues 3. **Permissive + Strong Copyleft**: Changes project licensing ### Forbidden Combinations 1. **MIT + GPL** (without relicensing) 2. **Proprietary + Any Copyleft** 3. **LGPL 2.1 + Apache 2.0** ## Distribution Considerations ### Binary Distribution - Must include all required license texts - Must preserve copyright notices - Must include source code for copyleft licenses - Must provide installation instructions for LGPL ### Source Distribution - Must include original license files - Must preserve copyright headers - Must document any modifications - Must provide clear licensing information ### SaaS/Network Services - AGPL extends copyleft to network services - GPL/LGPL generally don't apply to network services - Consider service boundaries carefully ## Compliance Best Practices ### 1. License Inventory - Maintain complete list of all dependencies - Track license changes in updates - Document license obligations ### 2. Compatibility Checking - Use automated tools for license scanning - Implement CI/CD license gates - Regular compliance audits ### 3. Documentation - Clear project license declaration - Complete attribution files - License change history ### 4. Legal Review - Consult legal counsel for complex scenarios - Review before major releases - Consider business model implications ## Risk Mitigation Strategies ### High-Risk Licenses - **AGPL**: Avoid in commercial/proprietary projects - **GPL in permissive projects**: Plan migration strategy - **Unknown licenses**: Investigate immediately ### Medium-Risk Scenarios - **Version incompatibilities**: Upgrade when possible - **Patent clause conflicts**: Seek legal advice - **Multiple copyleft licenses**: Verify compatibility ### Risk Assessment Framework 1. **Identify** all dependencies and their licenses 2. **Classify** by license type and risk level 3. **Analyze** compatibility with project license 4. **Document** decisions and rationale 5. **Monitor** for license changes ## Common Misconceptions ### ❌ Wrong Assumptions - "MIT allows everything" (still requires attribution) - "Linking doesn't create derivatives" (depends on license) - "GPL only affects distribution" (AGPL affects network use) - "Commercial use is always forbidden" (most FOSS allows it) ### ✅ Correct Understanding - Each license has specific requirements - Combination creates most restrictive terms - Network use may trigger copyleft (AGPL) - Commercial licensing options often available ## Quick Reference Decision Tree ``` Is the dependency GPL/AGPL? ├─ YES → Is your project commercial/proprietary? │ ├─ YES → ❌ Incompatible (find alternative) │ └─ NO → ✅ Compatible (if same GPL version) └─ NO → Is it permissive (MIT/Apache/BSD)? ├─ YES → ✅ Generally compatible └─ NO → Check specific compatibility matrix ``` ## Tools and Resources ### Automated Tools - **FOSSA**: Commercial license scanning - **WhiteSource**: Enterprise license management - **ORT**: Open source license scanning - **License Finder**: Ruby-based license detection ### Manual Review Resources - **choosealicense.com**: License picker and comparison - **SPDX License List**: Standardized license identifiers - **FSF License List**: Free Software Foundation compatibility - **OSI Approved Licenses**: Open Source Initiative approved licenses ## Conclusion License compatibility is crucial for legal compliance and risk management. When in doubt: 1. **Choose permissive licenses** for maximum compatibility 2. **Avoid strong copyleft** in proprietary projects 3. **Document all license decisions** thoroughly 4. **Consult legal experts** for complex scenarios 5. **Use automated tools** for continuous monitoring Remember: This matrix provides general guidance but legal requirements may vary by jurisdiction and specific use cases. Always consult with legal counsel for important licensing decisions. FILE:references/vulnerability_assessment_guide.md # Vulnerability Assessment Guide A comprehensive guide to assessing, prioritizing, and managing security vulnerabilities in software dependencies. ## Overview Dependency vulnerabilities represent one of the most significant attack vectors in modern software systems. This guide provides a structured approach to vulnerability assessment, risk scoring, and remediation planning. ## Vulnerability Classification System ### Severity Levels (CVSS 3.1) #### Critical (9.0 - 10.0) - **Impact**: Complete system compromise possible - **Examples**: Remote code execution, privilege escalation to admin - **Response Time**: Immediate (within 24 hours) - **Business Risk**: System shutdown, data breach, regulatory violations #### High (7.0 - 8.9) - **Impact**: Significant security impact - **Examples**: SQL injection, authentication bypass, sensitive data exposure - **Response Time**: 7 days maximum - **Business Risk**: Data compromise, service disruption #### Medium (4.0 - 6.9) - **Impact**: Moderate security impact - **Examples**: Cross-site scripting (XSS), information disclosure - **Response Time**: 30 days - **Business Risk**: Limited data exposure, minor service impact #### Low (0.1 - 3.9) - **Impact**: Limited security impact - **Examples**: Denial of service (limited), minor information leakage - **Response Time**: Next planned release cycle - **Business Risk**: Minimal impact on operations ## Vulnerability Types and Patterns ### Code Injection Vulnerabilities #### SQL Injection - **CWE-89**: Improper neutralization of SQL commands - **Common in**: Database interaction libraries, ORM frameworks - **Detection**: Parameter handling analysis, query construction review - **Mitigation**: Parameterized queries, input validation, least privilege DB access #### Command Injection - **CWE-78**: OS command injection - **Common in**: System utilities, file processing libraries - **Detection**: System call analysis, user input handling - **Mitigation**: Input sanitization, avoid system calls, sandboxing #### Code Injection - **CWE-94**: Code injection - **Common in**: Template engines, dynamic code evaluation - **Detection**: eval() usage, dynamic code generation - **Mitigation**: Avoid dynamic code execution, input validation, sandboxing ### Authentication and Authorization #### Authentication Bypass - **CWE-287**: Improper authentication - **Common in**: Authentication libraries, session management - **Detection**: Authentication flow analysis, session handling review - **Mitigation**: Multi-factor authentication, secure session management #### Privilege Escalation - **CWE-269**: Improper privilege management - **Common in**: Authorization frameworks, access control libraries - **Detection**: Permission checking analysis, role validation - **Mitigation**: Principle of least privilege, proper access controls ### Data Exposure #### Sensitive Data Exposure - **CWE-200**: Information exposure - **Common in**: Logging libraries, error handling, API responses - **Detection**: Log output analysis, error message review - **Mitigation**: Data classification, sanitized logging, proper error handling #### Cryptographic Failures - **CWE-327**: Broken cryptography - **Common in**: Cryptographic libraries, hash functions - **Detection**: Algorithm analysis, key management review - **Mitigation**: Modern cryptographic standards, proper key management ### Input Validation Issues #### Cross-Site Scripting (XSS) - **CWE-79**: Improper neutralization of input - **Common in**: Web frameworks, template engines - **Detection**: Input handling analysis, output encoding review - **Mitigation**: Input validation, output encoding, Content Security Policy #### Deserialization Vulnerabilities - **CWE-502**: Deserialization of untrusted data - **Common in**: Serialization libraries, data processing - **Detection**: Deserialization usage analysis - **Mitigation**: Avoid untrusted deserialization, input validation ## Risk Assessment Framework ### CVSS Scoring Components #### Base Metrics 1. **Attack Vector (AV)** - Network (N): 0.85 - Adjacent (A): 0.62 - Local (L): 0.55 - Physical (P): 0.2 2. **Attack Complexity (AC)** - Low (L): 0.77 - High (H): 0.44 3. **Privileges Required (PR)** - None (N): 0.85 - Low (L): 0.62/0.68 - High (H): 0.27/0.50 4. **User Interaction (UI)** - None (N): 0.85 - Required (R): 0.62 5. **Impact Metrics (C/I/A)** - High (H): 0.56 - Low (L): 0.22 - None (N): 0 #### Temporal Metrics - **Exploit Code Maturity**: Proof of concept availability - **Remediation Level**: Official fix availability - **Report Confidence**: Vulnerability confirmation level #### Environmental Metrics - **Confidentiality/Integrity/Availability Requirements**: Business impact - **Modified Base Metrics**: Environment-specific adjustments ### Custom Risk Factors #### Business Context 1. **Data Sensitivity** - Public data: Low risk multiplier (1.0x) - Internal data: Medium risk multiplier (1.2x) - Customer data: High risk multiplier (1.5x) - Regulated data: Critical risk multiplier (2.0x) 2. **System Criticality** - Development: Low impact (1.0x) - Staging: Medium impact (1.3x) - Production: High impact (1.8x) - Core infrastructure: Critical impact (2.5x) 3. **Exposure Level** - Internal systems: Base risk - Partner access: +1 risk level - Public internet: +2 risk levels - High-value target: +3 risk levels #### Technical Factors 1. **Dependency Type** - Direct dependencies: Higher priority - Transitive dependencies: Lower priority (unless critical path) - Development dependencies: Lowest priority 2. **Usage Pattern** - Core functionality: Highest priority - Optional features: Medium priority - Unused code paths: Lowest priority 3. **Fix Availability** - Official patch available: Standard timeline - Workaround available: Extended timeline acceptable - No fix available: Risk acceptance or replacement needed ## Vulnerability Discovery and Monitoring ### Automated Scanning #### Dependency Scanners - **npm audit**: Node.js ecosystem - **pip-audit**: Python ecosystem - **bundler-audit**: Ruby ecosystem - **OWASP Dependency Check**: Multi-language support #### Continuous Monitoring ```bash # Example CI/CD integration name: Security Scan on: [push, pull_request, schedule] jobs: security-scan: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Run dependency audit run: | npm audit --audit-level high python -m pip_audit bundle audit ``` #### Commercial Tools - **Snyk**: Developer-first security platform - **WhiteSource**: Enterprise dependency management - **Veracode**: Application security platform - **Checkmarx**: Static application security testing ### Manual Assessment #### Code Review Checklist 1. **Input Validation** - [ ] All user inputs validated - [ ] Proper sanitization applied - [ ] Length and format restrictions 2. **Authentication/Authorization** - [ ] Proper authentication checks - [ ] Authorization at every access point - [ ] Session management secure 3. **Data Handling** - [ ] Sensitive data protected - [ ] Encryption properly implemented - [ ] Secure data transmission 4. **Error Handling** - [ ] No sensitive info in error messages - [ ] Proper logging without data leaks - [ ] Graceful error handling ## Prioritization Framework ### Priority Matrix | Severity | Exploitability | Business Impact | Priority Level | |----------|---------------|-----------------|---------------| | Critical | High | High | P0 (Immediate) | | Critical | High | Medium | P0 (Immediate) | | Critical | Medium | High | P1 (24 hours) | | High | High | High | P1 (24 hours) | | High | High | Medium | P2 (1 week) | | High | Medium | High | P2 (1 week) | | Medium | High | High | P2 (1 week) | | All Others | - | - | P3 (30 days) | ### Prioritization Factors #### Technical Factors (40% weight) 1. **CVSS Base Score** (15%) 2. **Exploit Availability** (10%) 3. **Fix Complexity** (8%) 4. **Dependency Criticality** (7%) #### Business Factors (35% weight) 1. **Data Impact** (15%) 2. **System Criticality** (10%) 3. **Regulatory Requirements** (5%) 4. **Customer Impact** (5%) #### Operational Factors (25% weight) 1. **Attack Surface** (10%) 2. **Monitoring Coverage** (8%) 3. **Incident Response Capability** (7%) ### Scoring Formula ``` Priority Score = (Technical Score × 0.4) + (Business Score × 0.35) + (Operational Score × 0.25) Where each component is scored 1-10: - 9-10: Critical priority - 7-8: High priority - 5-6: Medium priority - 3-4: Low priority - 1-2: Informational ``` ## Remediation Strategies ### Immediate Actions (P0/P1) #### Hot Fixes 1. **Version Upgrade** - Update to patched version - Test critical functionality - Deploy with rollback plan 2. **Configuration Changes** - Disable vulnerable features - Implement additional access controls - Add monitoring/alerting 3. **Workarounds** - Input validation layers - Network-level protections - Application-level mitigations #### Emergency Response Process ``` 1. Vulnerability Confirmed ↓ 2. Impact Assessment (2 hours) ↓ 3. Mitigation Strategy (4 hours) ↓ 4. Implementation & Testing (12 hours) ↓ 5. Deployment (2 hours) ↓ 6. Monitoring & Validation (ongoing) ``` ### Planned Remediation (P2/P3) #### Standard Update Process 1. **Assessment Phase** - Detailed impact analysis - Testing requirements - Rollback procedures 2. **Planning Phase** - Update scheduling - Resource allocation - Communication plan 3. **Implementation Phase** - Development environment testing - Staging environment validation - Production deployment 4. **Validation Phase** - Functionality verification - Security testing - Performance monitoring ### Alternative Approaches #### Dependency Replacement - **When to Consider**: No fix available, persistent vulnerabilities - **Process**: Impact analysis → Alternative evaluation → Migration planning - **Risks**: API changes, feature differences, stability concerns #### Accept Risk (Last Resort) - **Criteria**: Very low probability, minimal impact, no feasible fix - **Requirements**: Executive approval, documented risk acceptance, monitoring - **Conditions**: Regular re-assessment, alternative solution tracking ## Remediation Tracking ### Metrics and KPIs #### Vulnerability Metrics - **Mean Time to Detection (MTTD)**: Average time from publication to discovery - **Mean Time to Patch (MTTP)**: Average time from discovery to fix deployment - **Vulnerability Density**: Vulnerabilities per 1000 dependencies - **Fix Rate**: Percentage of vulnerabilities fixed within SLA #### Trend Analysis - **Monthly vulnerability counts by severity** - **Average age of unpatched vulnerabilities** - **Remediation timeline trends** - **False positive rates** #### Reporting Dashboard ``` Security Dashboard Components: ├── Current Vulnerability Status │ ├── Critical: 2 (SLA: 24h) │ ├── High: 5 (SLA: 7d) │ └── Medium: 12 (SLA: 30d) ├── Trend Analysis │ ├── New vulnerabilities (last 30 days) │ ├── Fixed vulnerabilities (last 30 days) │ └── Average resolution time └── Risk Assessment ├── Overall risk score ├── Top vulnerable components └── Compliance status ``` ## Documentation Requirements ### Vulnerability Records Each vulnerability should be documented with: - **CVE/Advisory ID**: Official vulnerability identifier - **Discovery Date**: When vulnerability was identified - **CVSS Score**: Base and environmental scores - **Affected Systems**: Components and versions impacted - **Business Impact**: Risk assessment and criticality - **Remediation Plan**: Planned fix approach and timeline - **Resolution Date**: When fix was implemented and verified ### Risk Acceptance Documentation For accepted risks, document: - **Risk Description**: Detailed vulnerability explanation - **Impact Analysis**: Potential business and technical impact - **Mitigation Measures**: Compensating controls implemented - **Acceptance Rationale**: Why risk is being accepted - **Review Schedule**: When risk will be reassessed - **Approver**: Who authorized the risk acceptance ## Integration with Development Workflow ### Shift-Left Security #### Development Phase - **IDE Integration**: Real-time vulnerability detection - **Pre-commit Hooks**: Automated security checks - **Code Review**: Security-focused review criteria #### CI/CD Integration - **Build Stage**: Dependency vulnerability scanning - **Test Stage**: Security test automation - **Deploy Stage**: Final security validation #### Production Monitoring - **Runtime Protection**: Web application firewalls, runtime security - **Continuous Scanning**: Regular dependency updates check - **Incident Response**: Automated vulnerability alert handling ### Security Gates ```yaml security_gates: development: - dependency_scan: true - secret_detection: true - code_quality: true staging: - penetration_test: true - compliance_check: true - performance_test: true production: - final_security_scan: true - change_approval: required - rollback_plan: verified ``` ## Best Practices Summary ### Proactive Measures 1. **Regular Scanning**: Automated daily/weekly scans 2. **Update Schedule**: Regular dependency maintenance 3. **Security Training**: Developer security awareness 4. **Threat Modeling**: Understanding attack vectors ### Reactive Measures 1. **Incident Response**: Well-defined process for critical vulnerabilities 2. **Communication Plan**: Stakeholder notification procedures 3. **Lessons Learned**: Post-incident analysis and improvement 4. **Recovery Procedures**: Rollback and recovery capabilities ### Organizational Considerations 1. **Responsibility Assignment**: Clear ownership of security tasks 2. **Resource Allocation**: Adequate security budget and staffing 3. **Tool Selection**: Appropriate security tools for organization size 4. **Compliance Requirements**: Meeting regulatory and industry standards Remember: Vulnerability management is an ongoing process requiring continuous attention, regular updates to procedures, and organizational commitment to security best practices. FILE:scripts/dep_scanner.py #!/usr/bin/env python3 """ Dependency Scanner - Multi-language dependency vulnerability and analysis tool. This script parses dependency files from various package managers, extracts direct and transitive dependencies, checks against built-in vulnerability databases, and provides comprehensive security analysis with actionable recommendations. Author: Claude Skills Engineering Team License: MIT """ import json import os import re import sys import argparse from typing import Dict, List, Set, Any, Optional, Tuple from pathlib import Path from dataclasses import dataclass, asdict from datetime import datetime import hashlib import subprocess @dataclass class Vulnerability: """Represents a security vulnerability.""" id: str summary: str severity: str cvss_score: float affected_versions: str fixed_version: Optional[str] published_date: str references: List[str] @dataclass class Dependency: """Represents a project dependency.""" name: str version: str ecosystem: str direct: bool license: Optional[str] = None description: Optional[str] = None homepage: Optional[str] = None vulnerabilities: List[Vulnerability] = None def __post_init__(self): if self.vulnerabilities is None: self.vulnerabilities = [] class DependencyScanner: """Main dependency scanner class.""" def __init__(self): self.known_vulnerabilities = self._load_vulnerability_database() self.supported_files = { 'package.json': self._parse_package_json, 'package-lock.json': self._parse_package_lock, 'yarn.lock': self._parse_yarn_lock, 'requirements.txt': self._parse_requirements_txt, 'pyproject.toml': self._parse_pyproject_toml, 'Pipfile.lock': self._parse_pipfile_lock, 'poetry.lock': self._parse_poetry_lock, 'go.mod': self._parse_go_mod, 'go.sum': self._parse_go_sum, 'Cargo.toml': self._parse_cargo_toml, 'Cargo.lock': self._parse_cargo_lock, 'Gemfile': self._parse_gemfile, 'Gemfile.lock': self._parse_gemfile_lock, } def _load_vulnerability_database(self) -> Dict[str, List[Vulnerability]]: """Load built-in vulnerability database with common CVE patterns.""" return { # JavaScript/Node.js vulnerabilities 'lodash': [ Vulnerability( id='CVE-2021-23337', summary='Prototype pollution in lodash', severity='HIGH', cvss_score=7.2, affected_versions='<4.17.21', fixed_version='4.17.21', published_date='2021-02-15', references=['https://nvd.nist.gov/vuln/detail/CVE-2021-23337'] ) ], 'axios': [ Vulnerability( id='CVE-2023-45857', summary='Cross-site request forgery in axios', severity='MEDIUM', cvss_score=6.1, affected_versions='>=1.0.0 <1.6.0', fixed_version='1.6.0', published_date='2023-10-11', references=['https://nvd.nist.gov/vuln/detail/CVE-2023-45857'] ) ], 'express': [ Vulnerability( id='CVE-2022-24999', summary='Open redirect in express', severity='MEDIUM', cvss_score=6.1, affected_versions='<4.18.2', fixed_version='4.18.2', published_date='2022-11-26', references=['https://nvd.nist.gov/vuln/detail/CVE-2022-24999'] ) ], # Python vulnerabilities 'django': [ Vulnerability( id='CVE-2024-27351', summary='SQL injection in Django', severity='HIGH', cvss_score=9.8, affected_versions='>=3.2 <4.2.11', fixed_version='4.2.11', published_date='2024-02-06', references=['https://nvd.nist.gov/vuln/detail/CVE-2024-27351'] ) ], 'requests': [ Vulnerability( id='CVE-2023-32681', summary='Proxy-authorization header leak in requests', severity='MEDIUM', cvss_score=6.1, affected_versions='>=2.3.0 <2.31.0', fixed_version='2.31.0', published_date='2023-05-26', references=['https://nvd.nist.gov/vuln/detail/CVE-2023-32681'] ) ], 'pillow': [ Vulnerability( id='CVE-2023-50447', summary='Arbitrary code execution in Pillow', severity='HIGH', cvss_score=8.8, affected_versions='<10.2.0', fixed_version='10.2.0', published_date='2024-01-02', references=['https://nvd.nist.gov/vuln/detail/CVE-2023-50447'] ) ], # Go vulnerabilities 'github.com/gin-gonic/gin': [ Vulnerability( id='CVE-2023-26125', summary='Path traversal in gin', severity='HIGH', cvss_score=7.5, affected_versions='<1.9.1', fixed_version='1.9.1', published_date='2023-02-28', references=['https://nvd.nist.gov/vuln/detail/CVE-2023-26125'] ) ], # Rust vulnerabilities 'serde': [ Vulnerability( id='RUSTSEC-2022-0061', summary='Deserialization vulnerability in serde', severity='HIGH', cvss_score=8.2, affected_versions='<1.0.152', fixed_version='1.0.152', published_date='2022-12-07', references=['https://rustsec.org/advisories/RUSTSEC-2022-0061'] ) ], # Ruby vulnerabilities 'rails': [ Vulnerability( id='CVE-2023-28362', summary='ReDoS vulnerability in Rails', severity='HIGH', cvss_score=7.5, affected_versions='>=7.0.0 <7.0.4.3', fixed_version='7.0.4.3', published_date='2023-03-13', references=['https://nvd.nist.gov/vuln/detail/CVE-2023-28362'] ) ] } def scan_project(self, project_path: str) -> Dict[str, Any]: """Scan a project directory for dependencies and vulnerabilities.""" project_path = Path(project_path) if not project_path.exists(): raise FileNotFoundError(f"Project path does not exist: {project_path}") scan_results = { 'timestamp': datetime.now().isoformat(), 'project_path': str(project_path), 'dependencies': [], 'vulnerabilities_found': 0, 'high_severity_count': 0, 'medium_severity_count': 0, 'low_severity_count': 0, 'ecosystems': set(), 'scan_summary': {}, 'recommendations': [] } # Find and parse dependency files for file_pattern, parser in self.supported_files.items(): matching_files = list(project_path.rglob(file_pattern)) for dep_file in matching_files: try: dependencies = parser(dep_file) scan_results['dependencies'].extend(dependencies) for dep in dependencies: scan_results['ecosystems'].add(dep.ecosystem) # Check for vulnerabilities vulnerabilities = self._check_vulnerabilities(dep) dep.vulnerabilities = vulnerabilities scan_results['vulnerabilities_found'] += len(vulnerabilities) for vuln in vulnerabilities: if vuln.severity == 'HIGH': scan_results['high_severity_count'] += 1 elif vuln.severity == 'MEDIUM': scan_results['medium_severity_count'] += 1 else: scan_results['low_severity_count'] += 1 except Exception as e: print(f"Error parsing {dep_file}: {e}") continue scan_results['ecosystems'] = list(scan_results['ecosystems']) scan_results['scan_summary'] = self._generate_scan_summary(scan_results) scan_results['recommendations'] = self._generate_recommendations(scan_results) return scan_results def _check_vulnerabilities(self, dependency: Dependency) -> List[Vulnerability]: """Check if a dependency has known vulnerabilities.""" vulnerabilities = [] # Check package name (exact match and common variations) package_names = [dependency.name, dependency.name.lower()] for pkg_name in package_names: if pkg_name in self.known_vulnerabilities: for vuln in self.known_vulnerabilities[pkg_name]: if self._version_matches_vulnerability(dependency.version, vuln.affected_versions): vulnerabilities.append(vuln) return vulnerabilities def _version_matches_vulnerability(self, version: str, affected_pattern: str) -> bool: """Check if a version matches a vulnerability pattern.""" # Simple version matching - in production, use proper semver library try: # Handle common patterns like "<4.17.21", ">=1.0.0 <1.6.0" if '<' in affected_pattern and '>' not in affected_pattern: # Pattern like "<4.17.21" max_version = affected_pattern.replace('<', '').strip() return self._compare_versions(version, max_version) < 0 elif '>=' in affected_pattern and '<' in affected_pattern: # Pattern like ">=1.0.0 <1.6.0" parts = affected_pattern.split('<') min_part = parts[0].replace('>=', '').strip() max_part = parts[1].strip() return (self._compare_versions(version, min_part) >= 0 and self._compare_versions(version, max_part) < 0) except: pass return False def _compare_versions(self, v1: str, v2: str) -> int: """Simple version comparison. Returns -1, 0, or 1.""" try: def normalize(v): return [int(x) for x in re.sub(r'(\.0+)*$','', v).split('.')] v1_parts = normalize(v1) v2_parts = normalize(v2) if v1_parts < v2_parts: return -1 elif v1_parts > v2_parts: return 1 else: return 0 except: return 0 # Package file parsers def _parse_package_json(self, file_path: Path) -> List[Dependency]: """Parse package.json for Node.js dependencies.""" dependencies = [] try: with open(file_path, 'r') as f: data = json.load(f) # Parse dependencies for dep_type in ['dependencies', 'devDependencies']: if dep_type in data: for name, version in data[dep_type].items(): dep = Dependency( name=name, version=version.replace('^', '').replace('~', '').replace('>=', '').replace('<=', ''), ecosystem='npm', direct=True ) dependencies.append(dep) except Exception as e: print(f"Error parsing package.json: {e}") return dependencies def _parse_package_lock(self, file_path: Path) -> List[Dependency]: """Parse package-lock.json for Node.js transitive dependencies.""" dependencies = [] try: with open(file_path, 'r') as f: data = json.load(f) if 'packages' in data: for path, pkg_info in data['packages'].items(): if path == '': # Skip root package continue name = path.split('/')[-1] if '/' in path else path version = pkg_info.get('version', '') dep = Dependency( name=name, version=version, ecosystem='npm', direct=False, description=pkg_info.get('description', '') ) dependencies.append(dep) except Exception as e: print(f"Error parsing package-lock.json: {e}") return dependencies def _parse_yarn_lock(self, file_path: Path) -> List[Dependency]: """Parse yarn.lock for Node.js dependencies.""" dependencies = [] try: with open(file_path, 'r') as f: content = f.read() # Simple yarn.lock parsing packages = re.findall(r'^([^#\s][^:]+):\s*\n(?:\s+.*\n)*?\s+version\s+"([^"]+)"', content, re.MULTILINE) for package_spec, version in packages: name = package_spec.split('@')[0] if '@' in package_spec else package_spec name = name.strip('"') dep = Dependency( name=name, version=version, ecosystem='npm', direct=False ) dependencies.append(dep) except Exception as e: print(f"Error parsing yarn.lock: {e}") return dependencies def _parse_requirements_txt(self, file_path: Path) -> List[Dependency]: """Parse requirements.txt for Python dependencies.""" dependencies = [] try: with open(file_path, 'r') as f: lines = f.readlines() for line in lines: line = line.strip() if line and not line.startswith('#') and not line.startswith('-'): # Parse package==version or package>=version patterns match = re.match(r'^([a-zA-Z0-9_-]+)([><=!]+)(.+)$', line) if match: name, operator, version = match.groups() dep = Dependency( name=name, version=version, ecosystem='pypi', direct=True ) dependencies.append(dep) except Exception as e: print(f"Error parsing requirements.txt: {e}") return dependencies def _parse_pyproject_toml(self, file_path: Path) -> List[Dependency]: """Parse pyproject.toml for Python dependencies.""" dependencies = [] try: with open(file_path, 'r') as f: content = f.read() # Simple TOML parsing for dependencies dep_section = re.search(r'\[tool\.poetry\.dependencies\](.*?)(?=\[|\Z)', content, re.DOTALL) if dep_section: for line in dep_section.group(1).split('\n'): match = re.match(r'^([a-zA-Z0-9_-]+)\s*=\s*["\']([^"\']+)["\']', line.strip()) if match: name, version = match.groups() if name != 'python': dep = Dependency( name=name, version=version.replace('^', '').replace('~', ''), ecosystem='pypi', direct=True ) dependencies.append(dep) except Exception as e: print(f"Error parsing pyproject.toml: {e}") return dependencies def _parse_pipfile_lock(self, file_path: Path) -> List[Dependency]: """Parse Pipfile.lock for Python dependencies.""" dependencies = [] try: with open(file_path, 'r') as f: data = json.load(f) for section in ['default', 'develop']: if section in data: for name, info in data[section].items(): version = info.get('version', '').replace('==', '') dep = Dependency( name=name, version=version, ecosystem='pypi', direct=(section == 'default') ) dependencies.append(dep) except Exception as e: print(f"Error parsing Pipfile.lock: {e}") return dependencies def _parse_poetry_lock(self, file_path: Path) -> List[Dependency]: """Parse poetry.lock for Python dependencies.""" dependencies = [] try: with open(file_path, 'r') as f: content = f.read() # Extract package entries from TOML packages = re.findall(r'\[\[package\]\]\nname\s*=\s*"([^"]+)"\nversion\s*=\s*"([^"]+)"', content) for name, version in packages: dep = Dependency( name=name, version=version, ecosystem='pypi', direct=False ) dependencies.append(dep) except Exception as e: print(f"Error parsing poetry.lock: {e}") return dependencies def _parse_go_mod(self, file_path: Path) -> List[Dependency]: """Parse go.mod for Go dependencies.""" dependencies = [] try: with open(file_path, 'r') as f: content = f.read() # Parse require block require_match = re.search(r'require\s*$(.*?)$', content, re.DOTALL) if require_match: requires = require_match.group(1) for line in requires.split('\n'): match = re.match(r'\s*([^\s]+)\s+v?([^\s]+)', line.strip()) if match: name, version = match.groups() dep = Dependency( name=name, version=version, ecosystem='go', direct=True ) dependencies.append(dep) except Exception as e: print(f"Error parsing go.mod: {e}") return dependencies def _parse_go_sum(self, file_path: Path) -> List[Dependency]: """Parse go.sum for Go dependency checksums.""" return [] # go.sum mainly contains checksums, dependencies are in go.mod def _parse_cargo_toml(self, file_path: Path) -> List[Dependency]: """Parse Cargo.toml for Rust dependencies.""" dependencies = [] try: with open(file_path, 'r') as f: content = f.read() # Parse [dependencies] section dep_section = re.search(r'\[dependencies\](.*?)(?=\[|\Z)', content, re.DOTALL) if dep_section: for line in dep_section.group(1).split('\n'): match = re.match(r'^([a-zA-Z0-9_-]+)\s*=\s*["\']([^"\']+)["\']', line.strip()) if match: name, version = match.groups() dep = Dependency( name=name, version=version, ecosystem='cargo', direct=True ) dependencies.append(dep) except Exception as e: print(f"Error parsing Cargo.toml: {e}") return dependencies def _parse_cargo_lock(self, file_path: Path) -> List[Dependency]: """Parse Cargo.lock for Rust dependencies.""" dependencies = [] try: with open(file_path, 'r') as f: content = f.read() # Parse [[package]] entries packages = re.findall(r'\[\[package\]\]\nname\s*=\s*"([^"]+)"\nversion\s*=\s*"([^"]+)"', content) for name, version in packages: dep = Dependency( name=name, version=version, ecosystem='cargo', direct=False ) dependencies.append(dep) except Exception as e: print(f"Error parsing Cargo.lock: {e}") return dependencies def _parse_gemfile(self, file_path: Path) -> List[Dependency]: """Parse Gemfile for Ruby dependencies.""" dependencies = [] try: with open(file_path, 'r') as f: content = f.read() # Parse gem declarations gems = re.findall(r'gem\s+["\']([^"\']+)["\'](?:\s*,\s*["\']([^"\']+)["\'])?', content) for gem_info in gems: name = gem_info[0] version = gem_info[1] if len(gem_info) > 1 and gem_info[1] else '' dep = Dependency( name=name, version=version, ecosystem='rubygems', direct=True ) dependencies.append(dep) except Exception as e: print(f"Error parsing Gemfile: {e}") return dependencies def _parse_gemfile_lock(self, file_path: Path) -> List[Dependency]: """Parse Gemfile.lock for Ruby dependencies.""" dependencies = [] try: with open(file_path, 'r') as f: content = f.read() # Extract GEM section gem_section = re.search(r'GEM\s*\n(.*?)(?=\n\S|\Z)', content, re.DOTALL) if gem_section: specs = gem_section.group(1) gems = re.findall(r'\s+([a-zA-Z0-9_-]+)\s+$([^)]+)$', specs) for name, version in gems: dep = Dependency( name=name, version=version, ecosystem='rubygems', direct=False ) dependencies.append(dep) except Exception as e: print(f"Error parsing Gemfile.lock: {e}") return dependencies def _generate_scan_summary(self, scan_results: Dict[str, Any]) -> Dict[str, Any]: """Generate a summary of the scan results.""" total_deps = len(scan_results['dependencies']) unique_deps = len(set(dep.name for dep in scan_results['dependencies'])) return { 'total_dependencies': total_deps, 'unique_dependencies': unique_deps, 'ecosystems_found': len(scan_results['ecosystems']), 'vulnerable_dependencies': len([dep for dep in scan_results['dependencies'] if dep.vulnerabilities]), 'vulnerability_breakdown': { 'high': scan_results['high_severity_count'], 'medium': scan_results['medium_severity_count'], 'low': scan_results['low_severity_count'] } } def _generate_recommendations(self, scan_results: Dict[str, Any]) -> List[str]: """Generate actionable recommendations based on scan results.""" recommendations = [] high_count = scan_results['high_severity_count'] medium_count = scan_results['medium_severity_count'] if high_count > 0: recommendations.append(f"URGENT: Address {high_count} high-severity vulnerabilities immediately") if medium_count > 0: recommendations.append(f"Schedule fixes for {medium_count} medium-severity vulnerabilities within 30 days") vulnerable_deps = [dep for dep in scan_results['dependencies'] if dep.vulnerabilities] if vulnerable_deps: for dep in vulnerable_deps[:3]: # Top 3 most critical for vuln in dep.vulnerabilities: if vuln.fixed_version: recommendations.append(f"Update {dep.name} from {dep.version} to {vuln.fixed_version} to fix {vuln.id}") if len(scan_results['ecosystems']) > 3: recommendations.append("Consider consolidating package managers to reduce complexity") return recommendations def generate_report(self, scan_results: Dict[str, Any], format: str = 'text') -> str: """Generate a human-readable or JSON report.""" if format == 'json': # Convert Dependency objects to dicts for JSON serialization serializable_results = scan_results.copy() serializable_results['dependencies'] = [ { 'name': dep.name, 'version': dep.version, 'ecosystem': dep.ecosystem, 'direct': dep.direct, 'license': dep.license, 'vulnerabilities': [asdict(vuln) for vuln in dep.vulnerabilities] } for dep in scan_results['dependencies'] ] return json.dumps(serializable_results, indent=2, default=str) # Text format report report = [] report.append("=" * 60) report.append("DEPENDENCY SECURITY SCAN REPORT") report.append("=" * 60) report.append(f"Scan Date: {scan_results['timestamp']}") report.append(f"Project: {scan_results['project_path']}") report.append("") # Summary summary = scan_results['scan_summary'] report.append("SUMMARY:") report.append(f" Total Dependencies: {summary['total_dependencies']}") report.append(f" Unique Dependencies: {summary['unique_dependencies']}") report.append(f" Ecosystems: {', '.join(scan_results['ecosystems'])}") report.append(f" Vulnerabilities Found: {scan_results['vulnerabilities_found']}") report.append(f" High Severity: {summary['vulnerability_breakdown']['high']}") report.append(f" Medium Severity: {summary['vulnerability_breakdown']['medium']}") report.append(f" Low Severity: {summary['vulnerability_breakdown']['low']}") report.append("") # Vulnerable dependencies vulnerable_deps = [dep for dep in scan_results['dependencies'] if dep.vulnerabilities] if vulnerable_deps: report.append("VULNERABLE DEPENDENCIES:") report.append("-" * 30) for dep in vulnerable_deps: report.append(f"Package: {dep.name} v{dep.version} ({dep.ecosystem})") for vuln in dep.vulnerabilities: report.append(f" • {vuln.id}: {vuln.summary}") report.append(f" Severity: {vuln.severity} (CVSS: {vuln.cvss_score})") if vuln.fixed_version: report.append(f" Fixed in: {vuln.fixed_version}") report.append("") # Recommendations if scan_results['recommendations']: report.append("RECOMMENDATIONS:") report.append("-" * 20) for i, rec in enumerate(scan_results['recommendations'], 1): report.append(f"{i}. {rec}") report.append("") report.append("=" * 60) return '\n'.join(report) def main(): """Main entry point for the dependency scanner.""" parser = argparse.ArgumentParser( description='Scan project dependencies for vulnerabilities and security issues', formatter_class=argparse.RawDescriptionHelpFormatter, epilog=""" Examples: python dep_scanner.py /path/to/project python dep_scanner.py . --format json --output results.json python dep_scanner.py /app --fail-on-high """ ) parser.add_argument('project_path', help='Path to the project directory to scan') parser.add_argument('--format', choices=['text', 'json'], default='text', help='Output format (default: text)') parser.add_argument('--output', '-o', help='Output file path (default: stdout)') parser.add_argument('--fail-on-high', action='store_true', help='Exit with error code if high-severity vulnerabilities found') parser.add_argument('--quick-scan', action='store_true', help='Perform quick scan (skip transitive dependencies)') args = parser.parse_args() try: scanner = DependencyScanner() results = scanner.scan_project(args.project_path) report = scanner.generate_report(results, args.format) if args.output: with open(args.output, 'w') as f: f.write(report) print(f"Report saved to {args.output}") else: print(report) # Exit with error if high-severity vulnerabilities found and --fail-on-high is set if args.fail_on_high and results['high_severity_count'] > 0: sys.exit(1) except Exception as e: print(f"Error: {e}", file=sys.stderr) sys.exit(1) if __name__ == '__main__': main() FILE:scripts/license_checker.py #!/usr/bin/env python3 """ License Checker - Dependency license compliance and conflict analysis tool. This script analyzes dependency licenses from package metadata, classifies them into risk categories, detects license conflicts, and generates compliance reports with actionable recommendations for legal risk management. Author: Claude Skills Engineering Team License: MIT """ import json import os import sys import argparse from typing import Dict, List, Set, Any, Optional, Tuple from pathlib import Path from dataclasses import dataclass, asdict from datetime import datetime import re from enum import Enum class LicenseType(Enum): """License classification types.""" PERMISSIVE = "permissive" COPYLEFT_STRONG = "copyleft_strong" COPYLEFT_WEAK = "copyleft_weak" PROPRIETARY = "proprietary" DUAL = "dual" UNKNOWN = "unknown" class RiskLevel(Enum): """Risk assessment levels.""" LOW = "low" MEDIUM = "medium" HIGH = "high" CRITICAL = "critical" @dataclass class LicenseInfo: """Represents license information for a dependency.""" name: str spdx_id: Optional[str] license_type: LicenseType risk_level: RiskLevel description: str restrictions: List[str] obligations: List[str] compatibility: Dict[str, bool] @dataclass class DependencyLicense: """Represents a dependency with its license information.""" name: str version: str ecosystem: str direct: bool license_declared: Optional[str] license_detected: Optional[LicenseInfo] license_files: List[str] confidence: float @dataclass class LicenseConflict: """Represents a license compatibility conflict.""" dependency1: str license1: str dependency2: str license2: str conflict_type: str severity: RiskLevel description: str resolution_options: List[str] class LicenseChecker: """Main license checking and compliance analysis class.""" def __init__(self): self.license_database = self._build_license_database() self.compatibility_matrix = self._build_compatibility_matrix() self.license_patterns = self._build_license_patterns() def _build_license_database(self) -> Dict[str, LicenseInfo]: """Build comprehensive license database with risk classifications.""" return { # Permissive Licenses (Low Risk) 'MIT': LicenseInfo( name='MIT License', spdx_id='MIT', license_type=LicenseType.PERMISSIVE, risk_level=RiskLevel.LOW, description='Very permissive license with minimal restrictions', restrictions=['Include copyright notice', 'Include license text'], obligations=['Attribution'], compatibility={ 'commercial': True, 'modification': True, 'distribution': True, 'private_use': True, 'patent_grant': False } ), 'Apache-2.0': LicenseInfo( name='Apache License 2.0', spdx_id='Apache-2.0', license_type=LicenseType.PERMISSIVE, risk_level=RiskLevel.LOW, description='Permissive license with patent protection', restrictions=['Include copyright notice', 'Include license text', 'State changes', 'Include NOTICE file'], obligations=['Attribution', 'Patent grant'], compatibility={ 'commercial': True, 'modification': True, 'distribution': True, 'private_use': True, 'patent_grant': True } ), 'BSD-3-Clause': LicenseInfo( name='BSD 3-Clause License', spdx_id='BSD-3-Clause', license_type=LicenseType.PERMISSIVE, risk_level=RiskLevel.LOW, description='Permissive license with non-endorsement clause', restrictions=['Include copyright notice', 'Include license text', 'No endorsement using author names'], obligations=['Attribution'], compatibility={ 'commercial': True, 'modification': True, 'distribution': True, 'private_use': True, 'patent_grant': False } ), 'BSD-2-Clause': LicenseInfo( name='BSD 2-Clause License', spdx_id='BSD-2-Clause', license_type=LicenseType.PERMISSIVE, risk_level=RiskLevel.LOW, description='Very permissive license similar to MIT', restrictions=['Include copyright notice', 'Include license text'], obligations=['Attribution'], compatibility={ 'commercial': True, 'modification': True, 'distribution': True, 'private_use': True, 'patent_grant': False } ), 'ISC': LicenseInfo( name='ISC License', spdx_id='ISC', license_type=LicenseType.PERMISSIVE, risk_level=RiskLevel.LOW, description='Functionally equivalent to MIT license', restrictions=['Include copyright notice'], obligations=['Attribution'], compatibility={ 'commercial': True, 'modification': True, 'distribution': True, 'private_use': True, 'patent_grant': False } ), # Weak Copyleft Licenses (Medium Risk) 'MPL-2.0': LicenseInfo( name='Mozilla Public License 2.0', spdx_id='MPL-2.0', license_type=LicenseType.COPYLEFT_WEAK, risk_level=RiskLevel.MEDIUM, description='File-level copyleft license', restrictions=['Disclose source of modified files', 'Include copyright notice', 'Include license text', 'State changes'], obligations=['Source disclosure (modified files only)'], compatibility={ 'commercial': True, 'modification': True, 'distribution': True, 'private_use': True, 'patent_grant': True } ), 'LGPL-2.1': LicenseInfo( name='GNU Lesser General Public License 2.1', spdx_id='LGPL-2.1', license_type=LicenseType.COPYLEFT_WEAK, risk_level=RiskLevel.MEDIUM, description='Library-level copyleft license', restrictions=['Disclose source of library modifications', 'Include copyright notice', 'Include license text', 'Allow relinking'], obligations=['Source disclosure (library modifications)', 'Dynamic linking preferred'], compatibility={ 'commercial': True, 'modification': True, 'distribution': True, 'private_use': True, 'patent_grant': False } ), 'LGPL-3.0': LicenseInfo( name='GNU Lesser General Public License 3.0', spdx_id='LGPL-3.0', license_type=LicenseType.COPYLEFT_WEAK, risk_level=RiskLevel.MEDIUM, description='Library-level copyleft with patent provisions', restrictions=['Disclose source of library modifications', 'Include copyright notice', 'Include license text', 'Allow relinking', 'Anti-tivoization'], obligations=['Source disclosure (library modifications)', 'Patent grant'], compatibility={ 'commercial': True, 'modification': True, 'distribution': True, 'private_use': True, 'patent_grant': True } ), # Strong Copyleft Licenses (High Risk) 'GPL-2.0': LicenseInfo( name='GNU General Public License 2.0', spdx_id='GPL-2.0', license_type=LicenseType.COPYLEFT_STRONG, risk_level=RiskLevel.HIGH, description='Strong copyleft requiring full source disclosure', restrictions=['Disclose entire source code', 'Include copyright notice', 'Include license text', 'Use same license'], obligations=['Full source disclosure', 'License compatibility'], compatibility={ 'commercial': False, 'modification': True, 'distribution': True, 'private_use': True, 'patent_grant': False } ), 'GPL-3.0': LicenseInfo( name='GNU General Public License 3.0', spdx_id='GPL-3.0', license_type=LicenseType.COPYLEFT_STRONG, risk_level=RiskLevel.HIGH, description='Strong copyleft with patent and hardware provisions', restrictions=['Disclose entire source code', 'Include copyright notice', 'Include license text', 'Use same license', 'Anti-tivoization'], obligations=['Full source disclosure', 'Patent grant', 'License compatibility'], compatibility={ 'commercial': False, 'modification': True, 'distribution': True, 'private_use': True, 'patent_grant': True } ), 'AGPL-3.0': LicenseInfo( name='GNU Affero General Public License 3.0', spdx_id='AGPL-3.0', license_type=LicenseType.COPYLEFT_STRONG, risk_level=RiskLevel.CRITICAL, description='Network copyleft extending GPL to SaaS', restrictions=['Disclose entire source code', 'Include copyright notice', 'Include license text', 'Use same license', 'Network use triggers copyleft'], obligations=['Full source disclosure', 'Network service source disclosure'], compatibility={ 'commercial': False, 'modification': True, 'distribution': True, 'private_use': True, 'patent_grant': True } ), # Proprietary/Commercial Licenses (High Risk) 'PROPRIETARY': LicenseInfo( name='Proprietary License', spdx_id=None, license_type=LicenseType.PROPRIETARY, risk_level=RiskLevel.HIGH, description='Commercial or custom proprietary license', restrictions=['Varies by license', 'Often no redistribution', 'May require commercial license'], obligations=['License agreement compliance', 'Payment obligations'], compatibility={ 'commercial': False, 'modification': False, 'distribution': False, 'private_use': True, 'patent_grant': False } ), # Unknown/Unlicensed (Critical Risk) 'UNKNOWN': LicenseInfo( name='Unknown License', spdx_id=None, license_type=LicenseType.UNKNOWN, risk_level=RiskLevel.CRITICAL, description='No license detected or ambiguous licensing', restrictions=['Unknown', 'Assume no rights granted'], obligations=['Investigate and clarify licensing'], compatibility={ 'commercial': False, 'modification': False, 'distribution': False, 'private_use': False, 'patent_grant': False } ) } def _build_compatibility_matrix(self) -> Dict[str, Dict[str, bool]]: """Build license compatibility matrix.""" return { 'MIT': { 'MIT': True, 'Apache-2.0': True, 'BSD-3-Clause': True, 'BSD-2-Clause': True, 'ISC': True, 'MPL-2.0': True, 'LGPL-2.1': True, 'LGPL-3.0': True, 'GPL-2.0': False, 'GPL-3.0': False, 'AGPL-3.0': False, 'PROPRIETARY': False }, 'Apache-2.0': { 'MIT': True, 'Apache-2.0': True, 'BSD-3-Clause': True, 'BSD-2-Clause': True, 'ISC': True, 'MPL-2.0': True, 'LGPL-2.1': False, 'LGPL-3.0': True, 'GPL-2.0': False, 'GPL-3.0': True, 'AGPL-3.0': True, 'PROPRIETARY': False }, 'GPL-2.0': { 'MIT': True, 'Apache-2.0': False, 'BSD-3-Clause': True, 'BSD-2-Clause': True, 'ISC': True, 'MPL-2.0': False, 'LGPL-2.1': True, 'LGPL-3.0': False, 'GPL-2.0': True, 'GPL-3.0': False, 'AGPL-3.0': False, 'PROPRIETARY': False }, 'GPL-3.0': { 'MIT': True, 'Apache-2.0': True, 'BSD-3-Clause': True, 'BSD-2-Clause': True, 'ISC': True, 'MPL-2.0': True, 'LGPL-2.1': False, 'LGPL-3.0': True, 'GPL-2.0': False, 'GPL-3.0': True, 'AGPL-3.0': True, 'PROPRIETARY': False }, 'AGPL-3.0': { 'MIT': True, 'Apache-2.0': True, 'BSD-3-Clause': True, 'BSD-2-Clause': True, 'ISC': True, 'MPL-2.0': True, 'LGPL-2.1': False, 'LGPL-3.0': True, 'GPL-2.0': False, 'GPL-3.0': True, 'AGPL-3.0': True, 'PROPRIETARY': False } } def _build_license_patterns(self) -> Dict[str, List[str]]: """Build license detection patterns for text analysis.""" return { 'MIT': [ r'MIT License', r'Permission is hereby granted, free of charge', r'THE SOFTWARE IS PROVIDED "AS IS"' ], 'Apache-2.0': [ r'Apache License, Version 2\.0', r'Licensed under the Apache License', r'http://www\.apache\.org/licenses/LICENSE-2\.0' ], 'GPL-2.0': [ r'GNU GENERAL PUBLIC LICENSE\s+Version 2', r'This program is free software.*GPL.*version 2', r'http://www\.gnu\.org/licenses/gpl-2\.0' ], 'GPL-3.0': [ r'GNU GENERAL PUBLIC LICENSE\s+Version 3', r'This program is free software.*GPL.*version 3', r'http://www\.gnu\.org/licenses/gpl-3\.0' ], 'BSD-3-Clause': [ r'BSD 3-Clause License', r'Redistributions of source code must retain', r'Neither the name.*may be used to endorse' ], 'BSD-2-Clause': [ r'BSD 2-Clause License', r'Redistributions of source code must retain.*Redistributions in binary form' ] } def analyze_project(self, project_path: str, dependency_inventory: Optional[str] = None) -> Dict[str, Any]: """Analyze license compliance for a project.""" project_path = Path(project_path) analysis_results = { 'timestamp': datetime.now().isoformat(), 'project_path': str(project_path), 'project_license': self._detect_project_license(project_path), 'dependencies': [], 'license_summary': {}, 'conflicts': [], 'compliance_score': 0.0, 'risk_assessment': {}, 'recommendations': [] } # Load dependencies from inventory or scan project if dependency_inventory: dependencies = self._load_dependency_inventory(dependency_inventory) else: dependencies = self._scan_project_dependencies(project_path) # Analyze each dependency's license for dep in dependencies: license_info = self._analyze_dependency_license(dep, project_path) analysis_results['dependencies'].append(license_info) # Generate license summary analysis_results['license_summary'] = self._generate_license_summary( analysis_results['dependencies'] ) # Detect conflicts analysis_results['conflicts'] = self._detect_license_conflicts( analysis_results['project_license'], analysis_results['dependencies'] ) # Calculate compliance score analysis_results['compliance_score'] = self._calculate_compliance_score( analysis_results['dependencies'], analysis_results['conflicts'] ) # Generate risk assessment analysis_results['risk_assessment'] = self._generate_risk_assessment( analysis_results['dependencies'], analysis_results['conflicts'] ) # Generate recommendations analysis_results['recommendations'] = self._generate_compliance_recommendations( analysis_results ) return analysis_results def _detect_project_license(self, project_path: Path) -> Optional[str]: """Detect the main project license.""" license_files = ['LICENSE', 'LICENSE.txt', 'LICENSE.md', 'COPYING', 'COPYING.txt'] for license_file in license_files: license_path = project_path / license_file if license_path.exists(): try: with open(license_path, 'r', encoding='utf-8') as f: content = f.read() # Analyze license content detected_license = self._detect_license_from_text(content) if detected_license: return detected_license except Exception as e: print(f"Error reading license file {license_path}: {e}") return None def _detect_license_from_text(self, text: str) -> Optional[str]: """Detect license type from text content.""" text_upper = text.upper() for license_id, patterns in self.license_patterns.items(): for pattern in patterns: if re.search(pattern, text, re.IGNORECASE): return license_id # Common license text patterns if 'MIT' in text_upper and 'PERMISSION IS HEREBY GRANTED' in text_upper: return 'MIT' elif 'APACHE LICENSE' in text_upper and 'VERSION 2.0' in text_upper: return 'Apache-2.0' elif 'GPL' in text_upper and 'VERSION 2' in text_upper: return 'GPL-2.0' elif 'GPL' in text_upper and 'VERSION 3' in text_upper: return 'GPL-3.0' return None def _load_dependency_inventory(self, inventory_path: str) -> List[Dict[str, Any]]: """Load dependencies from JSON inventory file.""" try: with open(inventory_path, 'r') as f: data = json.load(f) if 'dependencies' in data: return data['dependencies'] else: return data if isinstance(data, list) else [] except Exception as e: print(f"Error loading dependency inventory: {e}") return [] def _scan_project_dependencies(self, project_path: Path) -> List[Dict[str, Any]]: """Basic dependency scanning - in practice, would integrate with dep_scanner.py.""" dependencies = [] # Simple package.json parsing as example package_json = project_path / 'package.json' if package_json.exists(): try: with open(package_json, 'r') as f: data = json.load(f) for dep_type in ['dependencies', 'devDependencies']: if dep_type in data: for name, version in data[dep_type].items(): dependencies.append({ 'name': name, 'version': version, 'ecosystem': 'npm', 'direct': True }) except Exception as e: print(f"Error parsing package.json: {e}") return dependencies def _analyze_dependency_license(self, dependency: Dict[str, Any], project_path: Path) -> DependencyLicense: """Analyze license information for a single dependency.""" dep_license = DependencyLicense( name=dependency['name'], version=dependency.get('version', ''), ecosystem=dependency.get('ecosystem', ''), direct=dependency.get('direct', False), license_declared=dependency.get('license'), license_detected=None, license_files=[], confidence=0.0 ) # Try to detect license from various sources declared_license = dependency.get('license') if declared_license: license_info = self._resolve_license_info(declared_license) if license_info: dep_license.license_detected = license_info dep_license.confidence = 0.9 # For unknown licenses, try to find license files in node_modules (example) if not dep_license.license_detected and dep_license.ecosystem == 'npm': node_modules_path = project_path / 'node_modules' / dep_license.name if node_modules_path.exists(): license_info = self._scan_package_directory(node_modules_path) if license_info: dep_license.license_detected = license_info dep_license.confidence = 0.7 # Default to unknown if no license detected if not dep_license.license_detected: dep_license.license_detected = self.license_database['UNKNOWN'] dep_license.confidence = 0.0 return dep_license def _resolve_license_info(self, license_string: str) -> Optional[LicenseInfo]: """Resolve license string to LicenseInfo object.""" if not license_string: return None license_string = license_string.strip() # Direct SPDX ID match if license_string in self.license_database: return self.license_database[license_string] # Common variations and mappings license_mappings = { 'mit': 'MIT', 'apache': 'Apache-2.0', 'apache-2.0': 'Apache-2.0', 'apache 2.0': 'Apache-2.0', 'bsd': 'BSD-3-Clause', 'bsd-3-clause': 'BSD-3-Clause', 'bsd-2-clause': 'BSD-2-Clause', 'gpl-2.0': 'GPL-2.0', 'gpl-3.0': 'GPL-3.0', 'lgpl-2.1': 'LGPL-2.1', 'lgpl-3.0': 'LGPL-3.0', 'mpl-2.0': 'MPL-2.0', 'isc': 'ISC', 'unlicense': 'MIT', # Treat as permissive 'public domain': 'MIT', # Treat as permissive 'proprietary': 'PROPRIETARY', 'commercial': 'PROPRIETARY' } license_lower = license_string.lower() for pattern, mapped_license in license_mappings.items(): if pattern in license_lower: return self.license_database.get(mapped_license) return None def _scan_package_directory(self, package_path: Path) -> Optional[LicenseInfo]: """Scan package directory for license information.""" license_files = ['LICENSE', 'LICENSE.txt', 'LICENSE.md', 'COPYING', 'README.md', 'package.json'] for license_file in license_files: file_path = package_path / license_file if file_path.exists(): try: with open(file_path, 'r', encoding='utf-8', errors='ignore') as f: content = f.read() # Try to detect license from content if license_file == 'package.json': # Parse JSON for license field try: data = json.loads(content) license_field = data.get('license') if license_field: return self._resolve_license_info(license_field) except: continue else: # Analyze text content detected_license = self._detect_license_from_text(content) if detected_license: return self.license_database.get(detected_license) except Exception: continue return None def _generate_license_summary(self, dependencies: List[DependencyLicense]) -> Dict[str, Any]: """Generate summary of license distribution.""" summary = { 'total_dependencies': len(dependencies), 'license_types': {}, 'risk_levels': {}, 'unknown_licenses': 0, 'direct_dependencies': 0, 'transitive_dependencies': 0 } for dep in dependencies: # Count by license type license_type = dep.license_detected.license_type.value summary['license_types'][license_type] = summary['license_types'].get(license_type, 0) + 1 # Count by risk level risk_level = dep.license_detected.risk_level.value summary['risk_levels'][risk_level] = summary['risk_levels'].get(risk_level, 0) + 1 # Count unknowns if dep.license_detected.license_type == LicenseType.UNKNOWN: summary['unknown_licenses'] += 1 # Count direct vs transitive if dep.direct: summary['direct_dependencies'] += 1 else: summary['transitive_dependencies'] += 1 return summary def _detect_license_conflicts(self, project_license: Optional[str], dependencies: List[DependencyLicense]) -> List[LicenseConflict]: """Detect license compatibility conflicts.""" conflicts = [] if not project_license: # If no project license detected, flag as potential issue for dep in dependencies: if dep.license_detected.risk_level in [RiskLevel.HIGH, RiskLevel.CRITICAL]: conflicts.append(LicenseConflict( dependency1='Project', license1='Unknown', dependency2=dep.name, license2=dep.license_detected.spdx_id or dep.license_detected.name, conflict_type='Unknown project license', severity=RiskLevel.HIGH, description=f'Project license unknown, dependency {dep.name} has {dep.license_detected.risk_level.value} risk license', resolution_options=['Define project license', 'Review dependency usage'] )) return conflicts project_license_info = self.license_database.get(project_license) if not project_license_info: return conflicts # Check compatibility with project license for dep in dependencies: dep_license_id = dep.license_detected.spdx_id or 'UNKNOWN' # Check compatibility matrix if project_license in self.compatibility_matrix: compatibility = self.compatibility_matrix[project_license].get(dep_license_id, False) if not compatibility: severity = self._determine_conflict_severity(project_license_info, dep.license_detected) conflicts.append(LicenseConflict( dependency1='Project', license1=project_license, dependency2=dep.name, license2=dep_license_id, conflict_type='License incompatibility', severity=severity, description=f'Project license {project_license} is incompatible with dependency license {dep_license_id}', resolution_options=self._generate_conflict_resolutions(project_license, dep_license_id) )) # Check for GPL contamination in permissive projects if project_license_info.license_type == LicenseType.PERMISSIVE: for dep in dependencies: if dep.license_detected.license_type == LicenseType.COPYLEFT_STRONG: conflicts.append(LicenseConflict( dependency1='Project', license1=project_license, dependency2=dep.name, license2=dep.license_detected.spdx_id or dep.license_detected.name, conflict_type='GPL contamination', severity=RiskLevel.CRITICAL, description=f'GPL dependency {dep.name} may contaminate permissive project', resolution_options=['Remove GPL dependency', 'Change project license to GPL', 'Use dynamic linking', 'Find alternative dependency'] )) return conflicts def _determine_conflict_severity(self, project_license: LicenseInfo, dep_license: LicenseInfo) -> RiskLevel: """Determine severity of a license conflict.""" if dep_license.license_type == LicenseType.UNKNOWN: return RiskLevel.CRITICAL elif (project_license.license_type == LicenseType.PERMISSIVE and dep_license.license_type == LicenseType.COPYLEFT_STRONG): return RiskLevel.CRITICAL elif dep_license.license_type == LicenseType.PROPRIETARY: return RiskLevel.HIGH else: return RiskLevel.MEDIUM def _generate_conflict_resolutions(self, project_license: str, dep_license: str) -> List[str]: """Generate resolution options for license conflicts.""" resolutions = [] if 'GPL' in dep_license: resolutions.extend([ 'Find alternative non-GPL dependency', 'Use dynamic linking if possible', 'Consider changing project license to GPL-compatible', 'Remove the dependency if not essential' ]) elif dep_license == 'PROPRIETARY': resolutions.extend([ 'Obtain commercial license', 'Find open-source alternative', 'Remove dependency if not essential', 'Negotiate license terms' ]) else: resolutions.extend([ 'Review license compatibility carefully', 'Consult legal counsel', 'Find alternative dependency', 'Consider license exception' ]) return resolutions def _calculate_compliance_score(self, dependencies: List[DependencyLicense], conflicts: List[LicenseConflict]) -> float: """Calculate overall compliance score (0-100).""" if not dependencies: return 100.0 base_score = 100.0 # Deduct points for unknown licenses unknown_count = sum(1 for dep in dependencies if dep.license_detected.license_type == LicenseType.UNKNOWN) base_score -= (unknown_count / len(dependencies)) * 30 # Deduct points for high-risk licenses high_risk_count = sum(1 for dep in dependencies if dep.license_detected.risk_level in [RiskLevel.HIGH, RiskLevel.CRITICAL]) base_score -= (high_risk_count / len(dependencies)) * 20 # Deduct points for conflicts if conflicts: critical_conflicts = sum(1 for c in conflicts if c.severity == RiskLevel.CRITICAL) high_conflicts = sum(1 for c in conflicts if c.severity == RiskLevel.HIGH) base_score -= critical_conflicts * 15 base_score -= high_conflicts * 10 return max(0.0, base_score) def _generate_risk_assessment(self, dependencies: List[DependencyLicense], conflicts: List[LicenseConflict]) -> Dict[str, Any]: """Generate comprehensive risk assessment.""" return { 'overall_risk': self._calculate_overall_risk(dependencies, conflicts), 'license_risk_breakdown': self._calculate_license_risks(dependencies), 'conflict_summary': { 'total_conflicts': len(conflicts), 'critical_conflicts': len([c for c in conflicts if c.severity == RiskLevel.CRITICAL]), 'high_conflicts': len([c for c in conflicts if c.severity == RiskLevel.HIGH]) }, 'distribution_risks': self._assess_distribution_risks(dependencies), 'commercial_risks': self._assess_commercial_risks(dependencies) } def _calculate_overall_risk(self, dependencies: List[DependencyLicense], conflicts: List[LicenseConflict]) -> str: """Calculate overall project risk level.""" if any(c.severity == RiskLevel.CRITICAL for c in conflicts): return 'CRITICAL' elif any(dep.license_detected.risk_level == RiskLevel.CRITICAL for dep in dependencies): return 'CRITICAL' elif any(c.severity == RiskLevel.HIGH for c in conflicts): return 'HIGH' elif any(dep.license_detected.risk_level == RiskLevel.HIGH for dep in dependencies): return 'HIGH' elif any(dep.license_detected.risk_level == RiskLevel.MEDIUM for dep in dependencies): return 'MEDIUM' else: return 'LOW' def _calculate_license_risks(self, dependencies: List[DependencyLicense]) -> Dict[str, int]: """Calculate breakdown of license risks.""" risks = {'low': 0, 'medium': 0, 'high': 0, 'critical': 0} for dep in dependencies: risk_level = dep.license_detected.risk_level.value risks[risk_level] += 1 return risks def _assess_distribution_risks(self, dependencies: List[DependencyLicense]) -> List[str]: """Assess risks related to software distribution.""" risks = [] gpl_deps = [dep for dep in dependencies if dep.license_detected.license_type == LicenseType.COPYLEFT_STRONG] if gpl_deps: risks.append(f"GPL dependencies require source code disclosure: {[d.name for d in gpl_deps]}") proprietary_deps = [dep for dep in dependencies if dep.license_detected.license_type == LicenseType.PROPRIETARY] if proprietary_deps: risks.append(f"Proprietary dependencies may require commercial licenses: {[d.name for d in proprietary_deps]}") unknown_deps = [dep for dep in dependencies if dep.license_detected.license_type == LicenseType.UNKNOWN] if unknown_deps: risks.append(f"Unknown licenses pose legal uncertainty: {[d.name for d in unknown_deps]}") return risks def _assess_commercial_risks(self, dependencies: List[DependencyLicense]) -> List[str]: """Assess risks for commercial usage.""" risks = [] agpl_deps = [dep for dep in dependencies if dep.license_detected.spdx_id == 'AGPL-3.0'] if agpl_deps: risks.append(f"AGPL dependencies trigger copyleft for network services: {[d.name for d in agpl_deps]}") return risks def _generate_compliance_recommendations(self, analysis_results: Dict[str, Any]) -> List[str]: """Generate actionable compliance recommendations.""" recommendations = [] # Address critical issues first critical_conflicts = [c for c in analysis_results['conflicts'] if c.severity == RiskLevel.CRITICAL] if critical_conflicts: recommendations.append("CRITICAL: Address license conflicts immediately before any distribution") for conflict in critical_conflicts[:3]: # Top 3 recommendations.append(f" • {conflict.description}") # Unknown licenses unknown_count = analysis_results['license_summary']['unknown_licenses'] if unknown_count > 0: recommendations.append(f"Investigate and clarify licenses for {unknown_count} dependencies with unknown licensing") # GPL contamination gpl_deps = [dep for dep in analysis_results['dependencies'] if dep.license_detected.license_type == LicenseType.COPYLEFT_STRONG] if gpl_deps and analysis_results.get('project_license') in ['MIT', 'Apache-2.0', 'BSD-3-Clause']: recommendations.append("Consider removing GPL dependencies or changing project license for permissive project") # Compliance score if analysis_results['compliance_score'] < 70: recommendations.append("Overall compliance score is low - prioritize license cleanup") return recommendations def generate_report(self, analysis_results: Dict[str, Any], format: str = 'text') -> str: """Generate compliance report in specified format.""" if format == 'json': # Convert dataclass objects for JSON serialization serializable_results = analysis_results.copy() serializable_results['dependencies'] = [ { 'name': dep.name, 'version': dep.version, 'ecosystem': dep.ecosystem, 'direct': dep.direct, 'license_declared': dep.license_declared, 'license_detected': asdict(dep.license_detected) if dep.license_detected else None, 'confidence': dep.confidence } for dep in analysis_results['dependencies'] ] serializable_results['conflicts'] = [asdict(conflict) for conflict in analysis_results['conflicts']] return json.dumps(serializable_results, indent=2, default=str) # Text format report report = [] report.append("=" * 60) report.append("LICENSE COMPLIANCE REPORT") report.append("=" * 60) report.append(f"Analysis Date: {analysis_results['timestamp']}") report.append(f"Project: {analysis_results['project_path']}") report.append(f"Project License: {analysis_results['project_license'] or 'Unknown'}") report.append("") # Summary summary = analysis_results['license_summary'] report.append("SUMMARY:") report.append(f" Total Dependencies: {summary['total_dependencies']}") report.append(f" Compliance Score: {analysis_results['compliance_score']:.1f}/100") report.append(f" Overall Risk: {analysis_results['risk_assessment']['overall_risk']}") report.append(f" License Conflicts: {len(analysis_results['conflicts'])}") report.append("") # License distribution report.append("LICENSE DISTRIBUTION:") for license_type, count in summary['license_types'].items(): report.append(f" {license_type.title()}: {count}") report.append("") # Risk breakdown report.append("RISK BREAKDOWN:") for risk_level, count in summary['risk_levels'].items(): report.append(f" {risk_level.title()}: {count}") report.append("") # Conflicts if analysis_results['conflicts']: report.append("LICENSE CONFLICTS:") report.append("-" * 30) for conflict in analysis_results['conflicts']: report.append(f"Conflict: {conflict.dependency2} ({conflict.license2})") report.append(f" Issue: {conflict.description}") report.append(f" Severity: {conflict.severity.value.upper()}") report.append(f" Resolutions: {', '.join(conflict.resolution_options[:2])}") report.append("") # High-risk dependencies high_risk_deps = [dep for dep in analysis_results['dependencies'] if dep.license_detected.risk_level in [RiskLevel.HIGH, RiskLevel.CRITICAL]] if high_risk_deps: report.append("HIGH-RISK DEPENDENCIES:") report.append("-" * 30) for dep in high_risk_deps[:10]: # Top 10 license_name = dep.license_detected.spdx_id or dep.license_detected.name report.append(f" {dep.name} v{dep.version}: {license_name} ({dep.license_detected.risk_level.value.upper()})") report.append("") # Recommendations if analysis_results['recommendations']: report.append("RECOMMENDATIONS:") report.append("-" * 20) for i, rec in enumerate(analysis_results['recommendations'], 1): report.append(f"{i}. {rec}") report.append("") report.append("=" * 60) return '\n'.join(report) def main(): """Main entry point for the license checker.""" parser = argparse.ArgumentParser( description='Analyze dependency licenses for compliance and conflicts', formatter_class=argparse.RawDescriptionHelpFormatter, epilog=""" Examples: python license_checker.py /path/to/project python license_checker.py . --format json --output compliance.json python license_checker.py /app --inventory deps.json --policy strict """ ) parser.add_argument('project_path', help='Path to the project directory to analyze') parser.add_argument('--inventory', help='Path to dependency inventory JSON file') parser.add_argument('--format', choices=['text', 'json'], default='text', help='Output format (default: text)') parser.add_argument('--output', '-o', help='Output file path (default: stdout)') parser.add_argument('--policy', choices=['permissive', 'strict'], default='permissive', help='License policy strictness (default: permissive)') parser.add_argument('--warn-conflicts', action='store_true', help='Show warnings for potential conflicts') args = parser.parse_args() try: checker = LicenseChecker() results = checker.analyze_project(args.project_path, args.inventory) report = checker.generate_report(results, args.format) if args.output: with open(args.output, 'w') as f: f.write(report) print(f"Compliance report saved to {args.output}") else: print(report) # Exit with error code for policy violations if args.policy == 'strict' and results['compliance_score'] < 80: sys.exit(1) if args.warn_conflicts and results['conflicts']: print("\nWARNING: License conflicts detected!") sys.exit(2) except Exception as e: print(f"Error: {e}", file=sys.stderr) sys.exit(1) if __name__ == '__main__': main() FILE:scripts/upgrade_planner.py #!/usr/bin/env python3 """ Upgrade Planner - Dependency upgrade path planning and risk analysis tool. This script analyzes dependency inventories, evaluates semantic versioning patterns, estimates breaking change risks, and generates prioritized upgrade plans with migration checklists and rollback procedures. Author: Claude Skills Engineering Team License: MIT """ import json import os import sys import argparse from typing import Dict, List, Set, Any, Optional, Tuple from pathlib import Path from dataclasses import dataclass, asdict from datetime import datetime, timedelta from enum import Enum import re import subprocess class UpgradeRisk(Enum): """Upgrade risk levels.""" SAFE = "safe" LOW = "low" MEDIUM = "medium" HIGH = "high" CRITICAL = "critical" class UpdateType(Enum): """Semantic versioning update types.""" PATCH = "patch" MINOR = "minor" MAJOR = "major" PRERELEASE = "prerelease" @dataclass class VersionInfo: """Represents version information.""" major: int minor: int patch: int prerelease: Optional[str] = None build: Optional[str] = None def __str__(self): version = f"{self.major}.{self.minor}.{self.patch}" if self.prerelease: version += f"-{self.prerelease}" if self.build: version += f"+{self.build}" return version @dataclass class DependencyUpgrade: """Represents a potential dependency upgrade.""" name: str current_version: str latest_version: str ecosystem: str direct: bool update_type: UpdateType risk_level: UpgradeRisk security_updates: List[str] breaking_changes: List[str] migration_effort: str dependencies_affected: List[str] rollback_complexity: str estimated_time: str priority_score: float @dataclass class UpgradePlan: """Represents a complete upgrade plan.""" name: str description: str phase: int dependencies: List[str] estimated_duration: str prerequisites: List[str] migration_steps: List[str] testing_requirements: List[str] rollback_plan: List[str] success_criteria: List[str] class UpgradePlanner: """Main upgrade planning and risk analysis class.""" def __init__(self): self.breaking_change_patterns = self._build_breaking_change_patterns() self.ecosystem_knowledge = self._build_ecosystem_knowledge() self.security_advisories = self._build_security_advisories() def _build_breaking_change_patterns(self) -> Dict[str, List[str]]: """Build patterns for detecting breaking changes.""" return { 'npm': [ r'BREAKING\s*CHANGE', r'breaking\s*change', r'major\s*version', r'removed.*API', r'deprecated.*removed', r'no\s*longer\s*supported', r'minimum.*node.*version', r'peer.*dependency.*change' ], 'pypi': [ r'BREAKING\s*CHANGE', r'breaking\s*change', r'removed.*function', r'deprecated.*removed', r'minimum.*python.*version', r'incompatible.*change', r'API.*change' ], 'maven': [ r'BREAKING\s*CHANGE', r'breaking\s*change', r'removed.*method', r'deprecated.*removed', r'minimum.*java.*version', r'API.*incompatible' ] } def _build_ecosystem_knowledge(self) -> Dict[str, Dict[str, Any]]: """Build ecosystem-specific upgrade knowledge.""" return { 'npm': { 'typical_major_cycle_months': 12, 'typical_patch_cycle_weeks': 2, 'deprecation_notice_months': 6, 'lts_support_years': 3, 'common_breaking_changes': [ 'Node.js version requirements', 'Peer dependency updates', 'API signature changes', 'Configuration format changes' ] }, 'pypi': { 'typical_major_cycle_months': 18, 'typical_patch_cycle_weeks': 4, 'deprecation_notice_months': 12, 'lts_support_years': 2, 'common_breaking_changes': [ 'Python version requirements', 'Function signature changes', 'Import path changes', 'Configuration changes' ] }, 'maven': { 'typical_major_cycle_months': 24, 'typical_patch_cycle_weeks': 6, 'deprecation_notice_months': 12, 'lts_support_years': 5, 'common_breaking_changes': [ 'Java version requirements', 'Method signature changes', 'Package restructuring', 'Dependency changes' ] }, 'cargo': { 'typical_major_cycle_months': 6, 'typical_patch_cycle_weeks': 2, 'deprecation_notice_months': 3, 'lts_support_years': 1, 'common_breaking_changes': [ 'Rust edition changes', 'Trait changes', 'Module restructuring', 'Macro changes' ] } } def _build_security_advisories(self) -> Dict[str, List[Dict[str, Any]]]: """Build security advisory database for upgrade prioritization.""" return { 'lodash': [ { 'advisory_id': 'CVE-2021-23337', 'severity': 'HIGH', 'fixed_in': '4.17.21', 'description': 'Prototype pollution vulnerability' } ], 'django': [ { 'advisory_id': 'CVE-2024-27351', 'severity': 'HIGH', 'fixed_in': '4.2.11', 'description': 'SQL injection vulnerability' } ], 'express': [ { 'advisory_id': 'CVE-2022-24999', 'severity': 'MEDIUM', 'fixed_in': '4.18.2', 'description': 'Open redirect vulnerability' } ], 'axios': [ { 'advisory_id': 'CVE-2023-45857', 'severity': 'MEDIUM', 'fixed_in': '1.6.0', 'description': 'Cross-site request forgery' } ] } def analyze_upgrades(self, dependency_inventory: str, timeline_days: int = 90) -> Dict[str, Any]: """Analyze potential dependency upgrades and create upgrade plan.""" dependencies = self._load_dependency_inventory(dependency_inventory) analysis_results = { 'timestamp': datetime.now().isoformat(), 'timeline_days': timeline_days, 'dependencies_analyzed': len(dependencies), 'available_upgrades': [], 'upgrade_statistics': {}, 'risk_assessment': {}, 'upgrade_plans': [], 'recommendations': [] } # Analyze each dependency for upgrades for dep in dependencies: upgrade_info = self._analyze_dependency_upgrade(dep) if upgrade_info: analysis_results['available_upgrades'].append(upgrade_info) # Generate upgrade statistics analysis_results['upgrade_statistics'] = self._generate_upgrade_statistics( analysis_results['available_upgrades'] ) # Perform risk assessment analysis_results['risk_assessment'] = self._perform_risk_assessment( analysis_results['available_upgrades'] ) # Create phased upgrade plans analysis_results['upgrade_plans'] = self._create_upgrade_plans( analysis_results['available_upgrades'], timeline_days ) # Generate recommendations analysis_results['recommendations'] = self._generate_upgrade_recommendations( analysis_results ) return analysis_results def _load_dependency_inventory(self, inventory_path: str) -> List[Dict[str, Any]]: """Load dependency inventory from JSON file.""" try: with open(inventory_path, 'r') as f: data = json.load(f) if 'dependencies' in data: return data['dependencies'] elif isinstance(data, list): return data else: print("Warning: Unexpected inventory format") return [] except Exception as e: print(f"Error loading dependency inventory: {e}") return [] def _analyze_dependency_upgrade(self, dependency: Dict[str, Any]) -> Optional[DependencyUpgrade]: """Analyze upgrade possibilities for a single dependency.""" name = dependency.get('name', '') current_version = dependency.get('version', '').replace('^', '').replace('~', '') ecosystem = dependency.get('ecosystem', '') if not name or not current_version: return None # Parse current version current_ver = self._parse_version(current_version) if not current_ver: return None # Get latest version (simulated - in practice would query package registries) latest_version = self._get_latest_version(name, ecosystem) if not latest_version: return None latest_ver = self._parse_version(latest_version) if not latest_ver: return None # Determine if upgrade is needed if self._compare_versions(current_ver, latest_ver) >= 0: return None # Already up to date # Determine update type update_type = self._determine_update_type(current_ver, latest_ver) # Assess upgrade risk risk_level = self._assess_upgrade_risk(name, current_ver, latest_ver, ecosystem, update_type) # Check for security updates security_updates = self._check_security_updates(name, current_version, latest_version) # Analyze breaking changes breaking_changes = self._analyze_breaking_changes(name, current_ver, latest_ver, ecosystem) # Calculate priority score priority_score = self._calculate_priority_score( update_type, risk_level, security_updates, dependency.get('direct', False) ) return DependencyUpgrade( name=name, current_version=current_version, latest_version=latest_version, ecosystem=ecosystem, direct=dependency.get('direct', False), update_type=update_type, risk_level=risk_level, security_updates=security_updates, breaking_changes=breaking_changes, migration_effort=self._estimate_migration_effort(update_type, breaking_changes), dependencies_affected=self._get_affected_dependencies(name, dependency), rollback_complexity=self._assess_rollback_complexity(update_type, risk_level), estimated_time=self._estimate_upgrade_time(update_type, breaking_changes), priority_score=priority_score ) def _parse_version(self, version_string: str) -> Optional[VersionInfo]: """Parse semantic version string.""" # Clean version string version = re.sub(r'[^0-9a-zA-Z.-]', '', version_string) # Basic semver pattern pattern = r'^(\d+)\.(\d+)\.(\d+)(?:-([0-9A-Za-z.-]+))?(?:\+([0-9A-Za-z.-]+))?$' match = re.match(pattern, version) if match: major, minor, patch, prerelease, build = match.groups() return VersionInfo( major=int(major), minor=int(minor), patch=int(patch), prerelease=prerelease, build=build ) # Fallback for simpler version patterns simple_pattern = r'^(\d+)\.(\d+)(?:\.(\d+))?' match = re.match(simple_pattern, version) if match: major, minor, patch = match.groups() return VersionInfo( major=int(major), minor=int(minor), patch=int(patch or 0) ) return None def _compare_versions(self, v1: VersionInfo, v2: VersionInfo) -> int: """Compare two versions. Returns -1, 0, or 1.""" if (v1.major, v1.minor, v1.patch) < (v2.major, v2.minor, v2.patch): return -1 elif (v1.major, v1.minor, v1.patch) > (v2.major, v2.minor, v2.patch): return 1 else: # Handle prerelease comparison if v1.prerelease and not v2.prerelease: return -1 elif not v1.prerelease and v2.prerelease: return 1 elif v1.prerelease and v2.prerelease: if v1.prerelease < v2.prerelease: return -1 elif v1.prerelease > v2.prerelease: return 1 return 0 def _get_latest_version(self, package_name: str, ecosystem: str) -> Optional[str]: """Get latest version from package registry (simulated).""" # Simulated latest versions for common packages mock_versions = { 'lodash': '4.17.21', 'express': '4.18.2', 'react': '18.2.0', 'axios': '1.6.0', 'django': '4.2.11', 'requests': '2.31.0', 'numpy': '1.24.0', 'flask': '2.3.0', 'fastapi': '0.104.0', 'pytest': '7.4.0' } # In production, would query actual package registries: # npm: npm view version # pypi: pip index versions # maven: maven metadata API return mock_versions.get(package_name.lower()) def _determine_update_type(self, current: VersionInfo, latest: VersionInfo) -> UpdateType: """Determine the type of update based on semantic versioning.""" if latest.major > current.major: return UpdateType.MAJOR elif latest.minor > current.minor: return UpdateType.MINOR elif latest.patch > current.patch: return UpdateType.PATCH elif latest.prerelease and not current.prerelease: return UpdateType.PRERELEASE else: return UpdateType.PATCH # Default fallback def _assess_upgrade_risk(self, package_name: str, current: VersionInfo, latest: VersionInfo, ecosystem: str, update_type: UpdateType) -> UpgradeRisk: """Assess the risk level of an upgrade.""" # Base risk assessment on update type base_risk = { UpdateType.PATCH: UpgradeRisk.SAFE, UpdateType.MINOR: UpgradeRisk.LOW, UpdateType.MAJOR: UpgradeRisk.HIGH, UpdateType.PRERELEASE: UpgradeRisk.MEDIUM }.get(update_type, UpgradeRisk.MEDIUM) # Adjust for package-specific factors high_risk_packages = [ 'webpack', 'babel', 'typescript', 'eslint', # Build tools 'react', 'vue', 'angular', # Frameworks 'django', 'flask', 'fastapi', # Web frameworks 'spring-boot', 'hibernate' # Java frameworks ] if package_name.lower() in high_risk_packages and update_type == UpdateType.MAJOR: base_risk = UpgradeRisk.CRITICAL # Check for known breaking changes if self._has_known_breaking_changes(package_name, current, latest): if base_risk in [UpgradeRisk.SAFE, UpgradeRisk.LOW]: base_risk = UpgradeRisk.MEDIUM elif base_risk == UpgradeRisk.MEDIUM: base_risk = UpgradeRisk.HIGH return base_risk def _has_known_breaking_changes(self, package_name: str, current: VersionInfo, latest: VersionInfo) -> bool: """Check if there are known breaking changes between versions.""" # Simulated breaking change detection breaking_change_versions = { 'react': ['16.0.0', '17.0.0', '18.0.0'], 'django': ['2.0.0', '3.0.0', '4.0.0'], 'webpack': ['4.0.0', '5.0.0'], 'babel': ['7.0.0', '8.0.0'], 'typescript': ['4.0.0', '5.0.0'] } package_versions = breaking_change_versions.get(package_name.lower(), []) latest_str = str(latest) return any(latest_str.startswith(v.split('.')[0]) for v in package_versions) def _check_security_updates(self, package_name: str, current_version: str, latest_version: str) -> List[str]: """Check for security updates in the upgrade.""" security_updates = [] if package_name in self.security_advisories: for advisory in self.security_advisories[package_name]: fixed_version = advisory['fixed_in'] # Simple version comparison for security fixes if (self._is_version_greater(fixed_version, current_version) and not self._is_version_greater(fixed_version, latest_version)): security_updates.append(f"{advisory['advisory_id']}: {advisory['description']}") return security_updates def _is_version_greater(self, v1: str, v2: str) -> bool: """Simple version comparison.""" v1_parts = [int(x) for x in v1.split('.')] v2_parts = [int(x) for x in v2.split('.')] # Pad shorter version max_len = max(len(v1_parts), len(v2_parts)) v1_parts.extend([0] * (max_len - len(v1_parts))) v2_parts.extend([0] * (max_len - len(v2_parts))) return v1_parts > v2_parts def _analyze_breaking_changes(self, package_name: str, current: VersionInfo, latest: VersionInfo, ecosystem: str) -> List[str]: """Analyze potential breaking changes.""" breaking_changes = [] # Check if major version change if latest.major > current.major: breaking_changes.append(f"Major version upgrade from {current.major}.x to {latest.major}.x") # Add ecosystem-specific common breaking changes ecosystem_knowledge = self.ecosystem_knowledge.get(ecosystem, {}) common_changes = ecosystem_knowledge.get('common_breaking_changes', []) breaking_changes.extend(common_changes[:2]) # Add top 2 # Check for specific package patterns if package_name.lower() == 'react' and latest.major >= 17: breaking_changes.append("New JSX Transform") if latest.major >= 18: breaking_changes.append("Concurrent Rendering changes") elif package_name.lower() == 'django' and latest.major >= 4: breaking_changes.append("CSRF token changes") breaking_changes.append("Default AUTO_INCREMENT field changes") elif package_name.lower() == 'webpack' and latest.major >= 5: breaking_changes.append("Module Federation support") breaking_changes.append("Asset modules replace file-loader") return breaking_changes def _calculate_priority_score(self, update_type: UpdateType, risk_level: UpgradeRisk, security_updates: List[str], is_direct: bool) -> float: """Calculate priority score for upgrade (0-100).""" score = 50.0 # Base score # Security updates get highest priority if security_updates: score += 30.0 score += len(security_updates) * 5.0 # Multiple security fixes # Update type scoring type_scores = { UpdateType.PATCH: 20.0, UpdateType.MINOR: 10.0, UpdateType.MAJOR: -10.0, UpdateType.PRERELEASE: -5.0 } score += type_scores.get(update_type, 0) # Risk level adjustment risk_adjustments = { UpgradeRisk.SAFE: 15.0, UpgradeRisk.LOW: 5.0, UpgradeRisk.MEDIUM: -5.0, UpgradeRisk.HIGH: -15.0, UpgradeRisk.CRITICAL: -25.0 } score += risk_adjustments.get(risk_level, 0) # Direct dependencies get slightly higher priority if is_direct: score += 5.0 return max(0.0, min(100.0, score)) def _estimate_migration_effort(self, update_type: UpdateType, breaking_changes: List[str]) -> str: """Estimate migration effort level.""" if update_type == UpdateType.PATCH and not breaking_changes: return "Minimal" elif update_type == UpdateType.MINOR and len(breaking_changes) <= 1: return "Low" elif update_type == UpdateType.MAJOR or len(breaking_changes) > 2: return "High" else: return "Medium" def _get_affected_dependencies(self, package_name: str, dependency: Dict[str, Any]) -> List[str]: """Get list of dependencies that might be affected by this upgrade.""" # Simulated dependency impact analysis common_dependencies = { 'react': ['react-dom', 'react-router', 'react-redux'], 'django': ['djangorestframework', 'django-cors-headers', 'celery'], 'webpack': ['webpack-cli', 'webpack-dev-server', 'html-webpack-plugin'], 'babel': ['@babel/core', '@babel/preset-env', '@babel/preset-react'] } return common_dependencies.get(package_name.lower(), []) def _assess_rollback_complexity(self, update_type: UpdateType, risk_level: UpgradeRisk) -> str: """Assess complexity of rolling back the upgrade.""" if update_type == UpdateType.PATCH: return "Simple" elif update_type == UpdateType.MINOR and risk_level in [UpgradeRisk.SAFE, UpgradeRisk.LOW]: return "Simple" elif risk_level in [UpgradeRisk.HIGH, UpgradeRisk.CRITICAL]: return "Complex" else: return "Moderate" def _estimate_upgrade_time(self, update_type: UpdateType, breaking_changes: List[str]) -> str: """Estimate time required for upgrade.""" base_times = { UpdateType.PATCH: "30 minutes", UpdateType.MINOR: "2 hours", UpdateType.MAJOR: "1 day", UpdateType.PRERELEASE: "4 hours" } base_time = base_times.get(update_type, "4 hours") if len(breaking_changes) > 2: if "30 minutes" in base_time: base_time = "2 hours" elif "2 hours" in base_time: base_time = "1 day" elif "1 day" in base_time: base_time = "3 days" return base_time def _generate_upgrade_statistics(self, upgrades: List[DependencyUpgrade]) -> Dict[str, Any]: """Generate statistics about available upgrades.""" if not upgrades: return {} return { 'total_upgrades': len(upgrades), 'by_type': { 'patch': len([u for u in upgrades if u.update_type == UpdateType.PATCH]), 'minor': len([u for u in upgrades if u.update_type == UpdateType.MINOR]), 'major': len([u for u in upgrades if u.update_type == UpdateType.MAJOR]), 'prerelease': len([u for u in upgrades if u.update_type == UpdateType.PRERELEASE]) }, 'by_risk': { 'safe': len([u for u in upgrades if u.risk_level == UpgradeRisk.SAFE]), 'low': len([u for u in upgrades if u.risk_level == UpgradeRisk.LOW]), 'medium': len([u for u in upgrades if u.risk_level == UpgradeRisk.MEDIUM]), 'high': len([u for u in upgrades if u.risk_level == UpgradeRisk.HIGH]), 'critical': len([u for u in upgrades if u.risk_level == UpgradeRisk.CRITICAL]) }, 'security_updates': len([u for u in upgrades if u.security_updates]), 'direct_dependencies': len([u for u in upgrades if u.direct]), 'average_priority': sum(u.priority_score for u in upgrades) / len(upgrades) } def _perform_risk_assessment(self, upgrades: List[DependencyUpgrade]) -> Dict[str, Any]: """Perform comprehensive risk assessment.""" high_risk_upgrades = [u for u in upgrades if u.risk_level in [UpgradeRisk.HIGH, UpgradeRisk.CRITICAL]] security_upgrades = [u for u in upgrades if u.security_updates] major_upgrades = [u for u in upgrades if u.update_type == UpdateType.MAJOR] return { 'overall_risk': self._calculate_overall_upgrade_risk(upgrades), 'high_risk_count': len(high_risk_upgrades), 'security_critical_count': len(security_upgrades), 'major_version_count': len(major_upgrades), 'risk_factors': self._identify_risk_factors(upgrades), 'mitigation_strategies': self._suggest_mitigation_strategies(upgrades) } def _calculate_overall_upgrade_risk(self, upgrades: List[DependencyUpgrade]) -> str: """Calculate overall risk level for all upgrades.""" if not upgrades: return "LOW" risk_scores = { UpgradeRisk.SAFE: 1, UpgradeRisk.LOW: 2, UpgradeRisk.MEDIUM: 3, UpgradeRisk.HIGH: 4, UpgradeRisk.CRITICAL: 5 } total_score = sum(risk_scores.get(u.risk_level, 3) for u in upgrades) average_score = total_score / len(upgrades) if average_score >= 4.0: return "CRITICAL" elif average_score >= 3.0: return "HIGH" elif average_score >= 2.0: return "MEDIUM" else: return "LOW" def _identify_risk_factors(self, upgrades: List[DependencyUpgrade]) -> List[str]: """Identify key risk factors across all upgrades.""" factors = [] major_count = len([u for u in upgrades if u.update_type == UpdateType.MAJOR]) if major_count > 0: factors.append(f"{major_count} major version upgrades with potential breaking changes") critical_count = len([u for u in upgrades if u.risk_level == UpgradeRisk.CRITICAL]) if critical_count > 0: factors.append(f"{critical_count} critical risk upgrades requiring careful planning") framework_upgrades = [u for u in upgrades if any(fw in u.name.lower() for fw in ['react', 'django', 'spring', 'webpack', 'babel'])] if framework_upgrades: factors.append(f"Core framework upgrades: {[u.name for u in framework_upgrades[:3]]}") return factors def _suggest_mitigation_strategies(self, upgrades: List[DependencyUpgrade]) -> List[str]: """Suggest risk mitigation strategies.""" strategies = [] high_risk_count = len([u for u in upgrades if u.risk_level in [UpgradeRisk.HIGH, UpgradeRisk.CRITICAL]]) if high_risk_count > 0: strategies.append("Create comprehensive test suite before high-risk upgrades") strategies.append("Plan rollback procedures for critical upgrades") major_count = len([u for u in upgrades if u.update_type == UpdateType.MAJOR]) if major_count > 3: strategies.append("Phase major upgrades across multiple releases") strategies.append("Use feature flags for gradual rollout") security_count = len([u for u in upgrades if u.security_updates]) if security_count > 0: strategies.append("Prioritize security updates regardless of risk level") return strategies def _create_upgrade_plans(self, upgrades: List[DependencyUpgrade], timeline_days: int) -> List[UpgradePlan]: """Create phased upgrade plans.""" if not upgrades: return [] # Sort upgrades by priority score (descending) sorted_upgrades = sorted(upgrades, key=lambda x: x.priority_score, reverse=True) plans = [] # Phase 1: Security and safe updates (first 30% of timeline) phase1_upgrades = [u for u in sorted_upgrades if u.security_updates or u.risk_level == UpgradeRisk.SAFE][:10] if phase1_upgrades: plans.append(self._create_upgrade_plan( "Phase 1: Security & Safe Updates", "Immediate security fixes and low-risk updates", 1, phase1_upgrades, timeline_days // 3 )) # Phase 2: Low-medium risk updates (middle 40% of timeline) phase2_upgrades = [u for u in sorted_upgrades if u.risk_level in [UpgradeRisk.LOW, UpgradeRisk.MEDIUM] and not u.security_updates][:8] if phase2_upgrades: plans.append(self._create_upgrade_plan( "Phase 2: Regular Updates", "Standard dependency updates with moderate risk", 2, phase2_upgrades, timeline_days * 2 // 5 )) # Phase 3: High-risk and major updates (final 30% of timeline) phase3_upgrades = [u for u in sorted_upgrades if u.risk_level in [UpgradeRisk.HIGH, UpgradeRisk.CRITICAL]][:5] if phase3_upgrades: plans.append(self._create_upgrade_plan( "Phase 3: Major Updates", "High-risk upgrades requiring careful planning", 3, phase3_upgrades, timeline_days // 3 )) return plans def _create_upgrade_plan(self, name: str, description: str, phase: int, upgrades: List[DependencyUpgrade], duration_days: int) -> UpgradePlan: """Create a detailed upgrade plan for a phase.""" dependency_names = [u.name for u in upgrades] # Generate migration steps migration_steps = [] migration_steps.append("1. Create feature branch for upgrades") migration_steps.append("2. Update dependency versions in manifest files") migration_steps.append("3. Run dependency install/update commands") migration_steps.append("4. Fix breaking changes and deprecation warnings") migration_steps.append("5. Update test suite for compatibility") migration_steps.append("6. Run comprehensive test suite") migration_steps.append("7. Update documentation and changelog") migration_steps.append("8. Create pull request for review") # Add phase-specific steps if phase == 1: migration_steps.insert(3, "3a. Verify security fixes are applied") elif phase == 3: migration_steps.insert(5, "5a. Perform extensive integration testing") migration_steps.insert(6, "6a. Test with production-like data") # Generate testing requirements testing_requirements = [ "Unit test suite passes 100%", "Integration tests cover upgrade scenarios", "Performance benchmarks within acceptable range" ] if any(u.risk_level in [UpgradeRisk.HIGH, UpgradeRisk.CRITICAL] for u in upgrades): testing_requirements.extend([ "Manual testing of critical user flows", "Load testing for performance regression", "Security scanning for new vulnerabilities" ]) # Generate rollback plan rollback_plan = [ "1. Revert dependency versions in manifest files", "2. Run dependency install with previous versions", "3. Restore previous configuration files if changed", "4. Run smoke tests to verify rollback success", "5. Monitor system health metrics" ] # Success criteria success_criteria = [ "All tests pass in CI/CD pipeline", "No security vulnerabilities introduced", "Performance metrics within acceptable thresholds", "No critical user workflows broken" ] return UpgradePlan( name=name, description=description, phase=phase, dependencies=dependency_names, estimated_duration=f"{duration_days} days", prerequisites=self._generate_prerequisites(upgrades), migration_steps=migration_steps, testing_requirements=testing_requirements, rollback_plan=rollback_plan, success_criteria=success_criteria ) def _generate_prerequisites(self, upgrades: List[DependencyUpgrade]) -> List[str]: """Generate prerequisites for upgrade phase.""" prerequisites = [ "Comprehensive test suite with good coverage", "Backup of current working state", "Development environment setup" ] if any(u.risk_level in [UpgradeRisk.HIGH, UpgradeRisk.CRITICAL] for u in upgrades): prerequisites.extend([ "Staging environment for testing", "Rollback procedure documented and tested", "Team availability for issue resolution" ]) if any(u.security_updates for u in upgrades): prerequisites.append("Security team notification for validation") return prerequisites def _generate_upgrade_recommendations(self, analysis_results: Dict[str, Any]) -> List[str]: """Generate actionable upgrade recommendations.""" recommendations = [] security_count = analysis_results['upgrade_statistics'].get('security_updates', 0) if security_count > 0: recommendations.append(f"URGENT: {security_count} security updates available - prioritize immediately") safe_count = analysis_results['upgrade_statistics']['by_risk'].get('safe', 0) if safe_count > 0: recommendations.append(f"Quick wins: {safe_count} safe updates can be applied with minimal risk") critical_count = analysis_results['risk_assessment']['high_risk_count'] if critical_count > 0: recommendations.append(f"Plan carefully: {critical_count} high-risk upgrades need thorough testing") major_count = analysis_results['upgrade_statistics']['by_type'].get('major', 0) if major_count > 3: recommendations.append("Consider phasing major upgrades across multiple releases") overall_risk = analysis_results['risk_assessment']['overall_risk'] if overall_risk in ['HIGH', 'CRITICAL']: recommendations.append("Overall upgrade risk is high - recommend gradual approach") return recommendations def generate_report(self, analysis_results: Dict[str, Any], format: str = 'text') -> str: """Generate upgrade plan report in specified format.""" if format == 'json': # Convert dataclass objects for JSON serialization serializable_results = analysis_results.copy() serializable_results['available_upgrades'] = [asdict(upgrade) for upgrade in analysis_results['available_upgrades']] serializable_results['upgrade_plans'] = [asdict(plan) for plan in analysis_results['upgrade_plans']] return json.dumps(serializable_results, indent=2, default=str) # Text format report report = [] report.append("=" * 60) report.append("DEPENDENCY UPGRADE PLAN") report.append("=" * 60) report.append(f"Generated: {analysis_results['timestamp']}") report.append(f"Timeline: {analysis_results['timeline_days']} days") report.append("") # Statistics stats = analysis_results['upgrade_statistics'] report.append("UPGRADE SUMMARY:") report.append(f" Total Upgrades Available: {stats.get('total_upgrades', 0)}") report.append(f" Security Updates: {stats.get('security_updates', 0)}") report.append(f" Major Version Updates: {stats['by_type'].get('major', 0)}") report.append(f" High Risk Updates: {stats['by_risk'].get('high', 0)}") report.append("") # Risk Assessment risk = analysis_results['risk_assessment'] report.append("RISK ASSESSMENT:") report.append(f" Overall Risk Level: {risk['overall_risk']}") if risk.get('risk_factors'): report.append(" Key Risk Factors:") for factor in risk['risk_factors'][:3]: report.append(f" • {factor}") report.append("") # High Priority Upgrades high_priority = sorted([u for u in analysis_results['available_upgrades']], key=lambda x: x.priority_score, reverse=True)[:10] if high_priority: report.append("TOP PRIORITY UPGRADES:") report.append("-" * 30) for upgrade in high_priority: risk_indicator = "🔴" if upgrade.risk_level in [UpgradeRisk.HIGH, UpgradeRisk.CRITICAL] else \ "🟡" if upgrade.risk_level == UpgradeRisk.MEDIUM else "🟢" security_indicator = " 🔒" if upgrade.security_updates else "" report.append(f"{risk_indicator} {upgrade.name}: {upgrade.current_version} → {upgrade.latest_version}{security_indicator}") report.append(f" Type: {upgrade.update_type.value.title()} | Risk: {upgrade.risk_level.value.title()} | Priority: {upgrade.priority_score:.1f}") if upgrade.security_updates: report.append(f" Security: {upgrade.security_updates[0]}") report.append("") # Upgrade Plans if analysis_results['upgrade_plans']: report.append("PHASED UPGRADE PLANS:") report.append("-" * 30) for plan in analysis_results['upgrade_plans']: report.append(f"{plan.name} ({plan.estimated_duration})") report.append(f" Dependencies: {', '.join(plan.dependencies[:5])}") if len(plan.dependencies) > 5: report.append(f" ... and {len(plan.dependencies) - 5} more") report.append(f" Key Steps: {'; '.join(plan.migration_steps[:3])}") report.append("") # Recommendations if analysis_results['recommendations']: report.append("RECOMMENDATIONS:") report.append("-" * 20) for i, rec in enumerate(analysis_results['recommendations'], 1): report.append(f"{i}. {rec}") report.append("") report.append("=" * 60) return '\n'.join(report) def main(): """Main entry point for the upgrade planner.""" parser = argparse.ArgumentParser( description='Analyze dependency upgrades and create migration plans', formatter_class=argparse.RawDescriptionHelpFormatter, epilog=""" Examples: python upgrade_planner.py deps.json python upgrade_planner.py inventory.json --timeline 60 --format json python upgrade_planner.py deps.json --risk-threshold medium --output plan.txt """ ) parser.add_argument('inventory_file', help='Path to dependency inventory JSON file') parser.add_argument('--timeline', type=int, default=90, help='Timeline for upgrade plan in days (default: 90)') parser.add_argument('--format', choices=['text', 'json'], default='text', help='Output format (default: text)') parser.add_argument('--output', '-o', help='Output file path (default: stdout)') parser.add_argument('--risk-threshold', choices=['safe', 'low', 'medium', 'high', 'critical'], default='high', help='Maximum risk level to include (default: high)') parser.add_argument('--security-only', action='store_true', help='Only plan upgrades with security fixes') args = parser.parse_args() try: planner = UpgradePlanner() results = planner.analyze_upgrades(args.inventory_file, args.timeline) # Filter by risk threshold if specified if args.risk_threshold != 'critical': risk_levels = ['safe', 'low', 'medium', 'high', 'critical'] max_index = risk_levels.index(args.risk_threshold) allowed_risks = set(risk_levels[:max_index + 1]) results['available_upgrades'] = [ u for u in results['available_upgrades'] if u.risk_level.value in allowed_risks ] # Filter for security-only if specified if args.security_only: results['available_upgrades'] = [ u for u in results['available_upgrades'] if u.security_updates ] report = planner.generate_report(results, args.format) if args.output: with open(args.output, 'w') as f: f.write(report) print(f"Upgrade plan saved to {args.output}") else: print(report) except Exception as e: print(f"Error: {e}", file=sys.stderr) sys.exit(1) if __name__ == '__main__': main() FILE:test-inventory.json { "timestamp": "2026-02-16T15:42:09.730696", "project_path": "test-project", "dependencies": [ { "name": "express", "version": "4.18.1", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [ { "id": "CVE-2022-24999", "summary": "Open redirect in express", "severity": "MEDIUM", "cvss_score": 6.1, "affected_versions": "<4.18.2", "fixed_version": "4.18.2", "published_date": "2022-11-26", "references": [ "https://nvd.nist.gov/vuln/detail/CVE-2022-24999" ] }, { "id": "CVE-2022-24999", "summary": "Open redirect in express", "severity": "MEDIUM", "cvss_score": 6.1, "affected_versions": "<4.18.2", "fixed_version": "4.18.2", "published_date": "2022-11-26", "references": [ "https://nvd.nist.gov/vuln/detail/CVE-2022-24999" ] } ] }, { "name": "lodash", "version": "4.17.20", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [ { "id": "CVE-2021-23337", "summary": "Prototype pollution in lodash", "severity": "HIGH", "cvss_score": 7.2, "affected_versions": "<4.17.21", "fixed_version": "4.17.21", "published_date": "2021-02-15", "references": [ "https://nvd.nist.gov/vuln/detail/CVE-2021-23337" ] }, { "id": "CVE-2021-23337", "summary": "Prototype pollution in lodash", "severity": "HIGH", "cvss_score": 7.2, "affected_versions": "<4.17.21", "fixed_version": "4.17.21", "published_date": "2021-02-15", "references": [ "https://nvd.nist.gov/vuln/detail/CVE-2021-23337" ] } ] }, { "name": "axios", "version": "1.5.0", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [ { "id": "CVE-2023-45857", "summary": "Cross-site request forgery in axios", "severity": "MEDIUM", "cvss_score": 6.1, "affected_versions": ">=1.0.0 <1.6.0", "fixed_version": "1.6.0", "published_date": "2023-10-11", "references": [ "https://nvd.nist.gov/vuln/detail/CVE-2023-45857" ] }, { "id": "CVE-2023-45857", "summary": "Cross-site request forgery in axios", "severity": "MEDIUM", "cvss_score": 6.1, "affected_versions": ">=1.0.0 <1.6.0", "fixed_version": "1.6.0", "published_date": "2023-10-11", "references": [ "https://nvd.nist.gov/vuln/detail/CVE-2023-45857" ] } ] }, { "name": "jsonwebtoken", "version": "8.5.1", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "bcrypt", "version": "5.1.0", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "mongoose", "version": "6.10.0", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "cors", "version": "2.8.5", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "helmet", "version": "6.1.5", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "winston", "version": "3.8.2", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "dotenv", "version": "16.0.3", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "express-rate-limit", "version": "6.7.0", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "multer", "version": "1.4.5-lts.1", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "sharp", "version": "0.32.1", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "nodemailer", "version": "6.9.1", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "socket.io", "version": "4.6.1", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "redis", "version": "4.6.5", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "moment", "version": "2.29.4", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "chalk", "version": "4.1.2", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "commander", "version": "9.4.1", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "nodemon", "version": "2.0.22", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "jest", "version": "29.5.0", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "supertest", "version": "6.3.3", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "eslint", "version": "8.40.0", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "eslint-config-airbnb-base", "version": "15.0.0", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "eslint-plugin-import", "version": "2.27.5", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "webpack", "version": "5.82.1", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "webpack-cli", "version": "5.1.1", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "babel-loader", "version": "9.1.2", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "@babel/core", "version": "7.22.1", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "@babel/preset-env", "version": "7.22.2", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "css-loader", "version": "6.7.4", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "style-loader", "version": "3.3.3", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "html-webpack-plugin", "version": "5.5.1", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "mini-css-extract-plugin", "version": "2.7.6", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "postcss", "version": "8.4.23", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "postcss-loader", "version": "7.3.0", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "autoprefixer", "version": "10.4.14", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "cross-env", "version": "7.0.3", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] }, { "name": "rimraf", "version": "5.0.1", "ecosystem": "npm", "direct": true, "license": null, "vulnerabilities": [] } ], "vulnerabilities_found": 6, "high_severity_count": 2, "medium_severity_count": 4, "low_severity_count": 0, "ecosystems": [ "npm" ], "scan_summary": { "total_dependencies": 39, "unique_dependencies": 39, "ecosystems_found": 1, "vulnerable_dependencies": 3, "vulnerability_breakdown": { "high": 2, "medium": 4, "low": 0 } }, "recommendations": [ "URGENT: Address 2 high-severity vulnerabilities immediately", "Schedule fixes for 4 medium-severity vulnerabilities within 30 days", "Update express from 4.18.1 to 4.18.2 to fix CVE-2022-24999", "Update express from 4.18.1 to 4.18.2 to fix CVE-2022-24999", "Update lodash from 4.17.20 to 4.17.21 to fix CVE-2021-23337", "Update lodash from 4.17.20 to 4.17.21 to fix CVE-2021-23337", "Update axios from 1.5.0 to 1.6.0 to fix CVE-2023-45857", "Update axios from 1.5.0 to 1.6.0 to fix CVE-2023-45857" ] } FILE:test-project/package.json { "name": "sample-web-app", "version": "1.2.3", "description": "A sample web application with various dependencies for testing dependency auditing", "main": "index.js", "scripts": { "start": "node index.js", "dev": "nodemon index.js", "build": "webpack --mode production", "test": "jest", "lint": "eslint src/", "audit": "npm audit" }, "keywords": ["web", "app", "sample", "dependency", "audit"], "author": "Claude Skills Team", "license": "MIT", "dependencies": { "express": "4.18.1", "lodash": "4.17.20", "axios": "1.5.0", "jsonwebtoken": "8.5.1", "bcrypt": "5.1.0", "mongoose": "6.10.0", "cors": "2.8.5", "helmet": "6.1.5", "winston": "3.8.2", "dotenv": "16.0.3", "express-rate-limit": "6.7.0", "multer": "1.4.5-lts.1", "sharp": "0.32.1", "nodemailer": "6.9.1", "socket.io": "4.6.1", "redis": "4.6.5", "moment": "2.29.4", "chalk": "4.1.2", "commander": "9.4.1" }, "devDependencies": { "nodemon": "2.0.22", "jest": "29.5.0", "supertest": "6.3.3", "eslint": "8.40.0", "eslint-config-airbnb-base": "15.0.0", "eslint-plugin-import": "2.27.5", "webpack": "5.82.1", "webpack-cli": "5.1.1", "babel-loader": "9.1.2", "@babel/core": "7.22.1", "@babel/preset-env": "7.22.2", "css-loader": "6.7.4", "style-loader": "3.3.3", "html-webpack-plugin": "5.5.1", "mini-css-extract-plugin": "2.7.6", "postcss": "8.4.23", "postcss-loader": "7.3.0", "autoprefixer": "10.4.14", "cross-env": "7.0.3", "rimraf": "5.0.1" }, "engines": { "node": ">=16.0.0", "npm": ">=8.0.0" }, "repository": { "type": "git", "url": "https://github.com/example/sample-web-app.git" }, "bugs": { "url": "https://github.com/example/sample-web-app/issues" }, "homepage": "https://github.com/example/sample-web-app#readme" }

Kiểm toán phụ thuộc — dependency-auditor

Bình luận (0)