Project Overview

A sophisticated homelab automation project that combines the stability and hardware compatibility of Fedora Server with the power of automated virtualization. This setup provides isolated environments for services, development work, and gaming while maintaining reproducible, version-controlled infrastructure.

The project demonstrates advanced DevOps practices applied to homelab infrastructure, featuring multi-VM architecture with GPU passthrough, comprehensive automation, and production-grade monitoring and backup systems.

The Challenge

Traditional homelab management faces several critical issues:

  • Manual Configuration Complexity: Time-consuming manual setup processes prone to human error
  • Configuration Drift: Inconsistencies accumulate across different deployments and updates
  • Poor Recovery Procedures: Difficult system recovery and replication when hardware fails
  • Documentation Decay: Manual processes lead to outdated or missing documentation
  • Service Isolation: Conflicting dependencies and resource contention between services
  • Hardware Compatibility: Limited hardware support with specialized Linux distributions
  • Development Environment: Need for isolated development environments without affecting production services

The Solution

Implemented a comprehensive infrastructure-as-code solution addressing all pain points:

Architecture Design

┌─────────────────────────────────────────────────────────────────────────────────────┐
│                              Fedora Server Host                                     │
│                               192.168.0.30                                          │
│                                                                                     │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐ │
│  │   home-server   │  │  personal-dev   │  │    work-dev     │  │   gaming-vm     │ │
│  │ (Fedora Server) │  │ (Fedora Server) │  │ (Fedora Server) │  │  (Fedora VM)    │ │
│  │  192.168.0.31   │  │  192.168.0.33   │  │  192.168.10.31  │  │  192.168.0.32   │ │
│  │                 │  │                 │  │   (VLAN 10)     │  │                 │ │
│  │  • Services     │  │  • Personal Dev │  │  • Work Dev     │  │  • Gaming       │ │
│  │  • Docker       │  │  • SPICE + HDMI │  │  • VLAN Isolate │  │  • GPU Pass     │ │
│  │  • Monitoring   │  │  • 2 vCPUs      │  │  • 2 vCPUs      │  │  • DP Output    │ │
│  │  • 2 vCPUs      │  │  • 16GB RAM     │  │  • 16GB RAM     │  │  • 4 vCPUs      │ │
│  │  • 16GB RAM     │  │    (planned)    │  │    (planned)    │  │  • 16GB RAM     │ │
│  │    (planned)    │  │                 │  │                 │  │    (planned)    │ │
│  └─────────────────┘  └─────────────────┘  └─────────────────┘  └─────────────────┘ │
│                                                                                     │
│  Network Configuration:                                                             │
│  • Bridge networking (br0) - Main LAN: 192.168.0.x                                  │
│  • VLAN 10 - Work isolation: 192.168.10.x                                           │
│  • GPU passthrough to gaming VM                                                     │
│  • HDMI output to personal-dev VM                                                   │
│                                                                                     │
│  Host Features:                                                                     │
│  • SSH-only access (no GUI)                                                         │
│  • Cockpit web management                                                           │
│  • Automated VM provisioning                                                        │
└─────────────────────────────────────────────────────────────────────────────────────┘

Custom ISO Generation

The project features a sophisticated ISO generation system that creates custom Fedora installation media for each VM with embedded kickstart configurations. This automated process downloads the official Fedora Server ISO, extracts it, embeds VM-specific kickstart files with network settings and user accounts, and rebuilds bootable ISOs that install completely unattended. Each VM boots from its custom ISO and installs automatically with the correct IP address, hostname, disk partitioning, and initial user setup, requiring zero manual intervention. This approach ensures consistent, reproducible VM deployments while maintaining security through unique per-VM configurations.

Automation Framework

Ansible-Based Infrastructure:

  • Version: Ansible 2.18 with Python 3.13 compatibility
  • Collections: community.general, ansible.posix, community.docker, community.vmware
  • Declarative Configuration: All infrastructure defined in version-controlled YAML files
  • Role-based Organization: Modular, reusable automation components for different services
  • Idempotent Operations: Safe to run multiple times with consistent results
  • Testing Pipeline: Automated validation and testing procedures for infrastructure changes

Key Components:

  • Base System Provisioning: Automated Fedora Server installation and configuration
  • VM Management: Automated creation, configuration, and management of virtual machines
  • Network Configuration: Bridge networking setup with proper firewall rules
  • Service Orchestration: Automated deployment and management of containerized services
  • Monitoring Integration: Built-in system and service monitoring with alerting

Monitoring & Backup Systems:

  • Monitoring Stack: Prometheus + Grafana for metrics (planned), Cockpit 344 for system management
  • Backup Strategy: Restic with automated scheduling (planned), LVM snapshots for VM backups (planned)
  • Logging: Systemd journald with centralized log aggregation and rotation (planned)
  • Alerting: Automated notifications for system health, disk space, and service status (planned)

Technical Implementation

Infrastructure Components

Host System (Fedora Server):

  • Hardware Management: Direct hardware access with excellent driver support
  • Virtualization: KVM/QEMU with GPU passthrough capabilities
  • Network Configuration: br0 bridge for main LAN (192.168.0.x) and VLAN 10 for work isolation (192.168.10.x)
  • Storage Management: LVM-based storage with automated backup strategies
  • Management Interface: Cockpit 344 web interface for system monitoring

Virtual Machine Architecture:

  • Home Server VM (Fedora Server): Services, Docker containers, monitoring systems (192.168.0.31, 2 vCPUs, 16GB RAM planned)
  • Personal Dev VM (Fedora Server): Personal development environment with GUI access (192.168.0.33, 2 vCPUs, 16GB RAM planned, SPICE + HDMI)
  • Work Dev VM (Fedora Server): Isolated work development environment on VLAN (192.168.10.31, VLAN 10, 2 vCPUs, 16GB RAM planned)
  • Gaming VM (Fedora): Gaming workstation with GPU passthrough and display output (192.168.0.32, 4 vCPUs, 16GB RAM planned)

Automation Layer:

  • Ansible Playbooks: Comprehensive automation for all infrastructure components
  • Role Library: Reusable roles for common infrastructure patterns
  • Inventory Management: Dynamic inventory with environment-specific configurations
  • Secret Management: Secure handling of sensitive configuration data

Development Practices

Infrastructure as Code:

  • Version Control: All configurations tracked in Git with proper branching strategy
  • Documentation: Comprehensive documentation generated from code and maintained automatically
  • Testing Automation: Automated validation of infrastructure changes before deployment
  • Modular Design: Component-based architecture enabling easy modification and extension

Quality Assurance:

  • Linting: Automated code quality checks for Ansible playbooks and shell scripts
  • Testing Framework: Comprehensive testing of infrastructure automation
  • Validation Procedures: Multi-stage validation ensuring system reliability
  • Monitoring: Continuous monitoring of infrastructure health and performance

Advanced Features

GPU Passthrough System:

  • Hardware Isolation: Dedicated GPU access for gaming VM
  • Display Management: Multiple display outputs for different VMs
  • Performance Optimization: Near-native gaming performance in virtualized environment

Development Environment:

  • Personal Development: Isolated personal development environment with SPICE + HDMI output
  • Work Development: VLAN-isolated work environment for professional projects (VLAN 10)
  • Network Separation: Clear separation between personal and work development networks
  • Resource Management: Dynamic resource allocation based on workload requirements

Key Features

Automated Provisioning

  • Complete Infrastructure: End-to-end server provisioning from bare metal to running services
  • VM Lifecycle Management: Automated creation, configuration, backup, and destruction of VMs
  • Service Deployment: Containerized service deployment with dependency management
  • Configuration Management: Centralized configuration with environment-specific overrides

Monitoring & Maintenance

  • System Monitoring: Comprehensive monitoring of host and VM resources (planned)
  • Service Health Checks: Automated health monitoring for all deployed services (planned)
  • Backup Automation: Automated backup strategies for data protection and disaster recovery (planned)
  • Update Management: Automated system updates with rollback capabilities

Development Workflow

  • Environment Isolation: Complete separation between personal/work development, testing, and production
  • Network Segmentation: VLAN 10-based isolation for work development environment
  • Rapid Provisioning: Quick setup of new development environments for different purposes
  • Resource Scaling: Dynamic resource allocation based on workload requirements
  • Integration Testing: Automated testing of infrastructure changes

Results and Impact

Infrastructure Success

  • Reproducible Deployment: Complete server rebuild achievable in under 45 minutes
  • High Reliability: 99.9%+ uptime with automated recovery procedures
  • Improved Efficiency: 90% reduction in manual configuration tasks
  • Enhanced Security: Automated security hardening and update management
  • Better Documentation: Comprehensive, tested documentation procedures

Technical Achievements

  • Advanced Virtualization: Successfully implemented GPU passthrough for gaming workloads
  • Production Monitoring: Comprehensive monitoring and alerting capabilities (planned)
  • Disaster Recovery: Tested backup and recovery procedures with documented RTO/RPO (planned)
  • Performance Optimization: Optimized resource allocation and system performance

Skills Development

  • DevOps Expertise: Advanced knowledge of modern infrastructure automation practices
  • Virtualization Mastery: Deep understanding of KVM/QEMU and advanced virtualization concepts
  • Ansible Proficiency: Expert-level Ansible usage with complex automation scenarios
  • Linux Administration: Advanced Linux system administration and troubleshooting skills

Lessons Learned

Infrastructure Management

  • Automation Value: Critical importance of infrastructure-as-code for reproducibility and consistency
  • Documentation Strategy: Benefits of auto-generating documentation from configuration code
  • Testing Methodology: Value of automated testing for infrastructure changes and validation
  • Modular Design: Importance of component-based architecture for maintainability and scalability

Technical Implementation

  • Hardware Compatibility: Benefits of choosing mainstream distributions for broader hardware support
  • Resource Planning: Importance of proper resource allocation and capacity planning for multi-VM environments
  • Network Design: Critical role of proper network configuration for VM communication and security
  • Backup Strategy: Essential nature of automated backup systems for data protection and disaster recovery

Development Practices

  • Infrastructure as Code: Version control and testing practices applied to infrastructure configuration
  • Continuous Integration: Automated validation and deployment of infrastructure changes
  • Monitoring Integration: Importance of comprehensive monitoring from the initial design phase
  • Security by Design: Integration of security measures throughout the infrastructure lifecycle

Next Steps

Infrastructure Enhancements

  • Container Orchestration: Migration to Kubernetes for advanced container management
  • Multi-node Clustering: Expansion to multi-node cluster for high availability
  • Advanced Monitoring: Implementation of comprehensive observability stack
  • Security Hardening: Advanced security measures and compliance frameworks

Automation Improvements

  • CI/CD Integration: Infrastructure pipeline integration with application deployments
  • Automated Testing: Expanded testing coverage for infrastructure changes
  • Performance Monitoring: Advanced performance metrics and optimization automation
  • Disaster Recovery: Automated disaster recovery testing and validation

Feature Development

  • Self-service Portal: Web interface for infrastructure self-service capabilities
  • Resource Optimization: AI-driven resource allocation and optimization
  • Cost Management: Resource usage tracking and optimization recommendations
  • Integration Expansion: Integration with cloud services for hybrid infrastructure

This project demonstrates advanced DevOps practices, homelab automation expertise, and the ability to design and implement complex, production-ready systems that solve real-world infrastructure challenges while maintaining high standards for reliability, security, and maintainability.

Planned Features

Resource Management & Optimization

  • Live VM Resource Synchronization: Dynamic CPU and RAM adjustment based on inventory changes without VM restarts
  • SSL/TLS Certificate Management: Automated certificate deployment for HTTPS access to all web interfaces

Automation Framework Integration

Advanced Ansible Implementation

Building upon the core infrastructure, comprehensive automation frameworks ensure reproducible deployments:

  • Complete Automation: End-to-end server provisioning from bare metal to fully configured services
  • Role-based Organization: Modular, reusable Ansible components for different infrastructure aspects
  • Testing Pipeline: Automated validation ensuring infrastructure changes don’t break existing functionality
  • Configuration Management: Declarative infrastructure preventing configuration drift across deployments

Enhanced DevOps Practices

  • Infrastructure as Code: All configurations version-controlled with comprehensive change tracking
  • Automated Backup Strategies: Systematic backup automation with tested recovery procedures (planned)
  • Performance Monitoring: Integrated monitoring stack providing proactive system health insights (planned)
  • Security Automation: Automated security hardening and compliance validation throughout the infrastructure lifecycle