Provisioning a VM using terraform with GitHub CI/CD¶

Minimal disk size

Make sure your specified disk size is not smaller than the template VM. 8GB for the Alma cloud-init template

Disk overprovisioning

Proxmox allows for allocating more space for VMs' disks than is physically available. It shouldn't be a problem unless the storage space fills up.

Directory structure¶

\+ infra-clusters
 \\
 \+ terraform
 \|\\
 \|\+ defaults.tf - Default base values for a VM object
 \|\\
 \|\+ main.tf - Provisioning logic
 \|\\
 \|\+ terraform.tfvars - Desired cluster state
 \|\\
 \|\+ vars.tf - Config vars like proxmox connection or interface for optional VM vars
 \\
  \+ scripts
   \\
   \+ user-data.yaml

How to provision a basic VM:¶

Add VM definition to terraform.tfvars
Push changes to a branch
Open PR to main
Wait for plan action to finish
Read the plan output, if OK merge
Wait for apply action to finish
Congrats, you created a new VM

VM attributes¶

Default:¶

Danger

This values are either mandatory for VM to provision or hardcoded "as they are" for a reason and should never be changed during normal operations

target_node - The Proxmox node VM will be provisioned on. Only for initial provisioning, if you move it after provisioning further applies won't move it between the node unless you de provision and re provision the VM.
full_clone - Whether new VM should be linked clone (only saves changes from template) or full (creates whole disk for the VM)
clone - The name of the template to use
os_type - The type of the os, in the case of our Alma template specifically set to "cloud-init"
agent - Set to 1 tells Proxmox that it should look for guest agent inside the VM
onboot - In the event of Proxmox node power down the VM should automatically start itself up after node power on
storage_pool - Storage to use for VMs drives, in our case distributed ceph storage
base_tags - base tags given to every VM to signify being terraform managed, and for now also "alma" as it's the only distro we're using at this point
cicustom - path to cloud init file, in our case set to cephfs under snippets, it's stored on cephfs, because it's crucial, that the file can be accessed from every node and under the same path
timeouts - timeout, how long should terraform wait before aborting, set to 120m because that's how long it took to initially provision 12 vms for 2 Kubernetes clusters in one go

vars¶

Connection vars¶

proxmox_api_url - endpoint on the proxmox for terraform to make requests to, at this moment set to internal DNS record spreading requests on all Proxmox nodes in round robin
proxmox_api_token_id - id of user the Terraform acts at, can be created through Promxmox's gui or terminal
proxmox_api_token_secret - the secret for the aforementioned token, set to sensitive

Variable VM settings¶

Info

This section contains VM parameters that you might want to change

name - name of the machine, required
ip - ip of the machine, required
target_node - target node for initial provisioning see Defaults, can be used f.eg. to spread cluster machines, default: [hades]
gw - ip address of gateway to use, default: [10.1.0.1]
extra_tags - extra tags to append after default tags, default: ["[]"]
disk_size - specifies size of the main disk, should not be smaller than the template's disk see first warning, default: [12G]
storage_pool - storage pool to use for the main disk, can be specified as \<number>\<prefix> example: [12G], default: [ceph]
log_disk_size - size of disk mounted under /var/logs in the VM, default: [3G]
log_storage_pool - storage pool to use for the logs' disk, default: [ceph]
memory - amount of ram to allocate for the VM, default: [2048]
cores - number of CPU cores to allocate, default: [2]
bridge - network bridge the VM will connect to, default: [vmbr20]

tfvars¶

vms - list of vm objects making cluster state

main¶

Danger

Under normal operations this file should never be modified

Info

We're storing the state file in s3 to save on time probing proxmox to rebuild it each time you plan or apply