VM-HA with CloudStack and Linstor, Rene Peinthor

Linstor's CloudStack integration uses DRBD's quorum implementation to determine whether a VM currently has problems accessing its disk storage and might need to be restarted on another host. This talk overviewed the implementation and included a live demo of CloudStack’s VM-HA combined with Linstor.

Rene shared what happens on DRBD level if a host node fails and when CloudStack will start doing HA recovery procedures. Rene also went into CloudStack settings that can be used to tune VM HA timeouts.

--

The CloudStack European User Group 2024 took place on September 19th in Frankfurt, Germany. This year's event was another successful User Group for the community. The event drew a strong audience, highlighting the growth of the CloudStack community in the region.

1. VM-HA with CloudStack and Linstor
Rene Peinthor, LINBIT

1

2. 2
LINSTOR - goals
•storage build from generic Linux nodes
•for SDS consumers (Cloudstack, K8s, OpenNebula, ...)
•building on existing Linux storage components
•LVM, thin LVM or ZFS for volume management
•deployment architectures
•distinct storage nodes
•hyperconverged with hypervisors / container hosts
•Open Source, GPL

3. 3

Architecture, CloudStack support

4. 4

5. 5
CloudStack Support
●4.17
○Volume snapshots
○Live migrate
●4.18
○IOPS QoS
○Backup snapshots
●4.19
○Backup snapshots
○VM snapshots (without memory)
○HA support

6. 6

CloudStack VM High Availability


+

7. 7
VM HA or Host HA?
●VM HA with Linstor and DRBD (3 nodes)


●Host HA
○Out-of-band management
○Fencing

8. 8
DRBD quorum
●3 nodes minimum (3rd node can be diskless)
●No split-brain

9. 9
VM quorum settings
●on-no-quorum suspend-io;
●on-no-data-accessible suspend-io;

Linstor commands:
linstor rg sp DrbdOptions/auto-quorum suspend -io cloudstack
linstor rg opt --on-no-data-accessible suspend-io cloudstack

10. 10
1.Host goes down (network or power)
2.Ping timeout
3.checkOnHostCommand to all cluster agents
a.return: Host not responding
b.return: Other hosts notify host is down
4.CloudStack declares: “Host is down”
5.HA VM restart scheduling starting
CloudStack VM High Availability

11. 11
checkingHeartBeat(pool, host);
Check local node?
●Parse drbdsetup events2 –now
○Check if any resource that is primary and
has promotion score 0

Check other node?
●Parse drbdsetup status –json
○Check if local node has any connected resource
to the other node

12. 12
Settings for Linstor VM-HA
●Compute->High Availability (Mostly for Host HA)
○force.ha
●Hypervisor->KVM
○kvm.ha.activity.check.interval
○kvm.ha.activity.check.timeout
○kvm.ha.degraded.max.period
○…
●Miscellaneous->Others
○wait !!!
●ping.interval
●ping.timeout

13. 13
Live Demo
AGENT-71

LINSTOR
AGENT-72

LINSTOR
AGENT-73

LINSTOR
MGMT-70

14. 14
Live Demo
AGENT-71

LINSTOR
AGENT-72

LINSTOR
AGENT-73

LINSTOR
MGMT-70

15. 15
Thank you
https://www.linbit.com