Skip to content

Test Directory

This directory contains all testing frameworks, quality assurance tools, and automated validation systems for the IT-Journey project.

🧪 Testing Philosophy

The IT-Journey testing framework follows modern DevOps practices with emphasis on: - Automated Quality Assurance: Continuous monitoring and validation - Educational Value: Testing tools serve as learning examples - Proactive Detection: Early identification of issues before they impact users - AI-Enhanced Analysis: Intelligent failure analysis and recommendations

📁 Directory Structure

test/
├── README.md                           # This comprehensive guide
├── quest-solutions/                    # Quest completion validation framework
│   ├── README.md                      # Framework guide and authoring instructions
│   ├── validate-quest-solution.sh     # Main validation entry point
│   ├── _shared/                       # Shared toolkit and templates
│   │   ├── lib/                       # Common library functions
│   │   │   ├── common.sh             # Assertions, logging, platform detection
│   │   │   └── scoring.sh            # Point-based scoring engine
│   │   └── templates/                 # Templates for new quest solutions
│   │       ├── quest-solution-readme-template.md
│   │       ├── answer-key-template.md
│   │       └── validation-script-template.sh
│   ├── 0000/                          # Level 0000 solutions
│   ├── 0001/                          # Level 0001 solutions
│   ├── 0010/                          # Level 0010 solutions
│   │   └── oh-my-zsh-terminal-enchantment/  # Complete solution set
│   ├── 0011/                          # Level 0011 solutions
│   ├── 0100/                          # Level 0100 solutions
│   └── 0101/                          # Level 0101 solutions
└── [future-test-frameworks]/          # Additional testing systems

Link health monitoring is now handled by Link Health Guardian v3.0 (scripts/validation/link-checker.py), which consolidates all link checking into a single Python tool with lychee and curl engines, AI-powered analysis, and GitHub Actions integration.

Quick Start

# Run link health check (uses lychee engine by default)
python3 scripts/validation/link-checker.py --scope repo

# Run with curl fallback engine
python3 scripts/validation/link-checker.py --engine curl --scope repo

# Full check with AI analysis
python3 scripts/validation/link-checker.py --scope repo --ai-analyze

Key Resources

🧪 Quest Solutions Framework

Validates quest completion through answer keys, validation scripts, and reference reports. Organized by level (0000–0101) with a shared toolkit.

Quick Start

# Validate all quest solutions (structural checks)
./test/quest-solutions/validate-quest-solution.sh --all

# Validate a specific quest solution
./test/quest-solutions/validate-quest-solution.sh 0010/oh-my-zsh-terminal-enchantment

# Validate all solutions in a level
./test/quest-solutions/validate-quest-solution.sh --level 0010

Key Features

  • Shared Validation Toolkit: Common library with assertions, logging, and scoring engine
  • Structural Checks: File presence, script syntax, link integrity (CI-safe)
  • Scoring Engine: Point-based scoring with rank determination (LEGENDARY → Keep Questing)
  • Two Challenge Patterns: Supports both Template-style (Novice/Intermediate/Advanced) and Implementation Challenge + Boss Battle quests
  • Templates: Ready-to-use templates for creating new quest solutions

Creating New Quest Solutions

# Scaffold a new quest solution
mkdir -p test/quest-solutions/<LEVEL>/<quest-slug>/{scripts,reports}

# Copy templates
cp test/quest-solutions/_shared/templates/quest-solution-readme-template.md \
   test/quest-solutions/<LEVEL>/<quest-slug>/README.md
cp test/quest-solutions/_shared/templates/answer-key-template.md \
   test/quest-solutions/<LEVEL>/<quest-slug>/answer-key.md

See the Quest Solutions Framework README for the full authoring guide.

🚀 GitHub Actions Integration

The testing framework integrates seamlessly with GitHub Actions:

Workflows: - .github/workflows/link-checker.yml — Link health monitoring (PR + scheduled) - .github/workflows/validate-solutions.yml — Quest solution structural validation

Link Checker Schedule: Weekly (configurable via workflow_dispatch)

Solution Validation Triggers: - Push/PR changes to test/quest-solutions/ - Manual dispatch with optional level filter

Outputs: - Automated GitHub issues with analysis - Workflow artifacts with detailed results - Performance metrics and trends

📊 Output and Reporting

Test Results Structure

Link check results are stored in link-check-results/. Run outputs are gitignored (regenerated every run, uploaded as workflow artifacts); only the baselines and .gitkeep are tracked:

link-check-results/
├── .gitkeep                  # Preserves directory in git (tracked)
├── broken_links_baseline.json # Delta comparison baseline (tracked)
├── posts-external-baseline.md # Curated external link-rot catalog (tracked)
├── lychee_results.json       # Raw lychee output
├── statistics.env            # Machine-readable stats
├── summary.md                # Human-readable summary
├── detailed_analysis.md      # Failure categorization report
├── ai_analysis.md            # AI-generated insights
└── ai_analysis_summary.env   # AI summary metrics

AI Analysis Output

The AI analysis provides: - Executive Summary: High-level assessment of link health - Categorized Analysis: Breakdown by failure type and domain - Root Cause Identification: Technical reasons for failures - Priority Actions: Ranked remediation steps - Preventive Measures: Recommendations for avoiding future issues - Educational Impact: Assessment of learning disruption

🛠️ Development and Customization

Adding New Test Types

  1. Create a new subdirectory under test/
  2. Follow the established structure pattern
  3. Update this README with documentation
  4. Add GitHub Actions integration if needed

The v3.0 tool at scripts/validation/link-checker.py is designed for extensibility:

Custom Engines: Add new link checking engines beyond lychee/curl Enhanced AI Analysis: Extend OpenAI/Anthropic prompts and analysis categories Additional Outputs: Create new report formats and destinations Integration Points: Connect with external monitoring systems

Local Development

# Install dependencies
pip install -r scripts/validation/requirements.txt

# Run link checker locally
python3 scripts/validation/link-checker.py --scope repo --verbose

# Run with AI analysis (requires API key)
export OPENAI_API_KEY="your-key-here"
python3 scripts/validation/link-checker.py --scope repo --ai-analyze

# Run unit tests
python3 -m pytest scripts/validation/ -v

📚 Educational Value

The testing framework serves multiple educational purposes:

DevOps Learning

  • CI/CD Pipeline Design: Complete workflow automation example
  • Quality Assurance: Proactive monitoring and validation strategies
  • Infrastructure as Code: Configuration management and version control
  • Observability: Logging, metrics, and alerting patterns

AI Integration

  • Practical AI Application: Real-world use of GPT-4 for analysis
  • Prompt Engineering: Structured prompts for technical analysis
  • Fallback Strategies: Graceful degradation when AI unavailable
  • Cost Management: Efficient API usage and token optimization

Software Engineering

  • Error Handling: Comprehensive error detection and recovery
  • Performance Optimization: Parallel processing and resource management
  • Code Organization: Modular design and separation of concerns
  • Documentation: Self-documenting code and comprehensive guides

🔧 Configuration Reference

Environment Variables

Variable Description Default Required
SITE_URL Target site URL for testing Auto-detected No
OPENAI_API_KEY OpenAI API key for AI analysis None For AI features
MAX_PARALLEL Maximum parallel link tests 10 No
TIMEOUT Request timeout in seconds 30 No
VERBOSE Enable verbose logging false No

Configuration

Lychee Config (.lychee.toml): 

max_concurrency = 32
timeout = 20
cache = true
max_cache_age = "7d"
exclude = [
  "https://url/",
  "https://github\\.com/.*/blob/.*",
]

See .lychee.toml for complete configuration.

🚨 Troubleshooting

Common Issues

No Links Found: Check .lychee.toml exclude patterns and path settings AI Analysis Fails: Verify OPENAI_API_KEY or ANTHROPIC_API_KEY is set Timeout Errors: Increase timeout in .lychee.toml Lychee Not Found: Install via brew install lychee or cargo install lychee

Debug Mode

# Enable verbose output
python3 scripts/validation/link-checker.py --scope repo --verbose

Getting Help

  1. Tool Docs: See scripts/validation/link-checker.py --help for usage
  2. Issues: Report problems via GitHub Issues with the testing label
  3. Discussions: Join community discussions for questions and improvements
  4. Quest Guide: Follow the Hyperlink Guardian Quest

🤝 Contributing

Testing improvements are always welcome:

Areas for Contribution

  • New Test Types: Additional quality assurance frameworks
  • Performance Optimization: Faster execution and resource efficiency
  • Enhanced Reporting: Better visualizations and insights
  • Platform Support: Improved compatibility across operating systems
  • AI Enhancement: Better prompts and analysis capabilities

Development Guidelines

  1. Follow the established directory structure
  2. Include comprehensive documentation
  3. Add configuration examples
  4. Test across multiple platforms
  5. Consider educational value in design decisions

📈 Future Roadmap

Planned Enhancements

  • Visual Testing: Screenshot comparison and UI validation
  • Performance Testing: Site speed and accessibility monitoring
  • Security Testing: Vulnerability scanning and compliance checks
  • Content Validation: Grammar, spelling, and style checking
  • SEO Analysis: Search engine optimization recommendations

Integration Opportunities

  • External Services: Slack, Discord, email notifications
  • Monitoring Platforms: DataDog, New Relic, CloudWatch integration
  • Documentation Tools: Automated documentation generation
  • Learning Management: Progress tracking and skill assessment

The test directory embodies the IT-Journey commitment to quality, education, and automation - ensuring that every learner has access to reliable, up-to-date educational resources while providing real-world examples of modern testing practices.