Finally…
In the past most services ran on real hardware.
Only a few had been virtualized - mostly based on Xen.
Less than 10 hosts with Xen, 2 with KVM (libvirt).
More than 200 have been real hardware.
Migration project (since 2013) to switch to entirely new Hardware.
To make it easier, also change the whole IP layout below us.
And even made simpler by a complete change of policies (security/runtime/…).
And of course: Virtualize as much as possible!
2 datacentres (HA requirements).
Big storage in the background (HP 3PAR x 2).
Cisco based Networking (one Nexus 5500 per DC, everything cross-connected).
Some HP Enclosures, filled with HP Blades 465c Gen8.
No local storage, all off into SAN.
Enclosures connect Blades using "HP VirtualConnect" Modules.
As few rack mounted systems as possible.
As many machines as needed - virtual please.
Some TB of disk space in lots of spindles.
A hell of a broken API (Mix of REST and SSH).
A hell of a broken firmware. Easy to crash the REST Interface.
-> Wrote an extstorage Interface for Ganeti (GPL, but not yet published)
Enclosure connected to switch with 4x10GBit
All Blades get 10GBit via "Virtual Connect" Modules from the Enclosure
Lots of different VLANs in use (Prod/Test/Int/Admin/Backup/… seperation)
Blades use OpenVSwitch (currently not centrally controlled)
Ganeti just "attaches" to one of the tagged interfaces in OpenVSwitch
-> NO Routing/Firewall on the Blade!
-> Blade "invisible" for the VM!
DMZ | 5 machines (1 to move to CMS) - all VMS with connections to "Outside CMS" |
CMS | 6 machines. Will get 1 from DMZ, and 2 new small Blades. everything that doesn't need the DMZ |
Test | 2 machines. Need a place to test and play with new features. |
Most everything is additionally seperated into "prod"/"test"/"int", plus a few more VLans.
Blades live in an own "Management" vlan, not directly accessible from anything except very few machines. Thats provided via 2 own (bonded) interfaces, untagged.
Openvswitch on the Blades manages the needed VLans for the VMs.
Seperate Backup VLAN connecting every VM with Backup hosts. Special firewall rules ensure seperation there.
Created by Joerg Jaspert <joerg@nsb-software.de>.