Self-Hosted AI Infrastructure – Reference Architecture
Deploying AI workloads on premises requires careful planning across networking, storage, compute, and identity management. This guide presents a validated architecture for running Homegrown Intelligence workflows in a self-hosted environment. The design assumes a single administrative domain with physical separation between inference nodes and supporting services.
Network Segmentation
Place inference nodes on a dedicated VLAN with firewall rules that permit inbound connections only from authorized clients. The inference VLAN uses RFC 1918 addressing with a /24 subnet. A Traefik reverse proxy terminates TLS at the network edge and routes requests to the appropriate workflow service based on path prefixes. Health-check endpoints are exposed on a separate management interface accessible only from the administration jump box. mDNS advertisements enable automatic service discovery within the local segment.
Storage Architecture
Model weights are stored on a ZFS pool with compression enabled, reducing storage footprint by approximately 18 percent for typical FP16 checkpoints. The pool uses mirrored vdevs for redundancy with a hot spare. Dataset and output directories reside on a separate NFS share exported from a NAS appliance. The NFS mount is configured with noac and actimeo=0 to prevent stale file-handle errors during concurrent write operations from multiple inference nodes. Snapshot schedules run every six hours with a retention window of fourteen days.
Container Orchestration
Each Homegrown Intelligence workflow ships as a Docker image with pinned base layers and a read-only root filesystem. Containers are managed through a lightweight Docker Compose stack with resource reservations for GPU devices, memory, and CPU cores. The NVIDIA container toolkit provides GPU passthrough. Restart policies are set to unless-stopped with a five-second grace period. Logging uses the json-file driver with rotation at 100 MB per file and a maximum of three retained files.
Authentication and Authorization
An internal OAuth 2.0 provider issues short-lived access tokens scoped to individual workflow APIs. Token refresh occurs automatically through a companion sidecar container that monitors the token expiry and reissues credentials before revocation. Role-based access control restricts write operations to admin principals while allowing read-only access for monitoring dashboards. Audit logs capture every authenticated request with client IP, principal identifier, target resource, and HTTP method.
Monitoring Stack
Prometheus scrapes metrics from each workflow container at fifteen-second intervals. Node exporters on inference hosts report CPU temperature, GPU clock speeds, memory bandwidth utilization, and disk I/O latency. Grafana dashboards visualize throughput trends, error rates, and resource saturation. Alertmanager sends notifications to a webhook receiver when GPU temperature exceeds 85 degrees Celsius or when request latency surpasses the 95th percentile threshold of two seconds for more than five consecutive minutes.
Backup and Disaster Recovery
Configuration backups are taken daily and replicated to a secondary storage appliance. Model weights are considered immutable artifacts cached from the Homegrown Intelligence registry and can be re-fetched if lost. The recovery time objective is thirty minutes for the control plane and two hours for full inference capacity restoration. A disaster recovery drill is performed quarterly with documented runbooks covering hardware failure, network partition, and certificate expiration scenarios.