Knowledge transfer from Wayne: https://drupal.star.bnl.gov/STAR/blog/wbetts/online-gluster-and-ovirt-expansion

Problem: Presently Gluster is configured with Distributed-Replica Volume using oVirt nodes 1,2, and 4.
oVirt Engine on oVirt 1, dean on ovirt 2
Replication: 3 * (2+1) => 3 replications on each node, 2 actual data vol and 1 arbiter

When we try to expand:

• Gluster expects a new set of 3 bricks matching the same structure (2 + 1)

• But there’s no built-in logic to automate arbiter awareness during volume expansion

Options:

1. Wayne suggests: Promote arbiter to full replica 3*3 using these instructions: https://lists.gluster.org/pipermail/gluster-users/2018-July/034385.html
Check for any space issues? If not, are there any uses of arbiter v/s full replica? May not be safe conversion..

2. Create new parallel volume with layout 4 * (2+1) = 12 bricks
I read creating a set of 3 bricks; manually setting it as new replica set and maintaining arbiter logic is not officially supported and easy to break... not a good option

Potential option: Install GlusterFS on ovirt3, add it to the pool? 

3. Can you use ovirt4 as VM host? Originally asked by Wayne's blog

ovirt1

ovirt4

Issue: Microarchitecture Gap: Big difference in generations — 2014 vs. 2021 hardware.
oVirt can technically run VMs but it's not the same CPU family as ovirt1/2 (1,2 are same)

Live migration is not compatible without config tweaks.
So it's better to have ovirt4 for Gluster Node only and NO VMs hosted here.

4. About oVirt3:
CPU: Xeon E5-2667 v4 with Broadwell-EP microarch (2016)
x86 architecture match, VT-x supported and Family 6 (therefore fully compatible with ovirt 1/2)

Ok! To add ovirt3 as a node for now (without any GlusterFS, we can do this safely without any downtime on ovirt1/2)

Ovirt1/2 status fine as in UI. Engine running fine.
ssh into ovirt3 from ovirt-Engine: gives Permission denied.
This was not needed for ovirt1,2 as these hosts are managed by VDSM but for new ovirt3, ssh is needed for deployment.
 

Passwordless SSH is NOT working yet between oVirt Engine ➔ ovirt3.
Trying to add ovirt3 to cluster via WebUI.

- Ended up adding the overt-engine's IP to /etc/ssh/sshd_confif on ovirt3 to allow root login from ovirt-engine.
- Then the ovirt3 was added as a compute host but not yet ready to host the oVirt Engine.