================================================= Configuring the Compute (nova) service (optional) ================================================= The Compute service (nova) handles the creation of virtual machines within an OpenStack environment. Many of the default options used by OpenStack-Ansible are found within ``defaults/main.yml`` within the nova role. Availability zones ~~~~~~~~~~~~~~~~~~ Deployers with multiple availability zones can set the ``nova_nova_conf_overrides.DEFAULT.default_schedule_zone`` Ansible variable to specify an availability zone for new requests. This is useful in environments with different types of hypervisors, where builds are sent to certain hardware types based on their resource requirements. For example, if you have servers running on two racks without sharing the PDU. These two racks can be grouped into two availability zones. When one rack loses power, the other one still works. By spreading your containers onto the two racks (availability zones), you will improve your service availability. Block device tuning for Ceph (RBD) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Enabling Ceph and defining ``nova_libvirt_images_rbd_pool`` changes two libvirt configurations by default: * hw_disk_discard: ``unmap`` * disk_cachemodes: ``network=writeback`` Setting ``hw_disk_discard`` to ``unmap`` in libvirt enables discard (sometimes called TRIM) support for the underlying block device. This allows reclaiming of unused blocks on the underlying disks. Setting ``disk_cachemodes`` to ``network=writeback`` allows data to be written into a cache on each change, but those changes are flushed to disk at a regular interval. This can increase write performance on Ceph block devices. You have the option to customize these settings using two Ansible variables (defaults shown here): .. code-block:: yaml nova_libvirt_hw_disk_discard: 'unmap' nova_libvirt_disk_cachemodes: 'network=writeback' You can disable discard by setting ``nova_libvirt_hw_disk_discard`` to ``ignore``. The ``nova_libvirt_disk_cachemodes`` can be set to an empty string to disable ``network=writeback``. The following minimal example configuration sets nova to use the ``ephemeral-vms`` Ceph pool. The following example uses cephx authentication, and requires an existing ``cinder`` account for the ``ephemeral-vms`` pool: .. code-block:: console nova_libvirt_images_rbd_pool: ephemeral-vms ceph_mons: - 172.29.244.151 - 172.29.244.152 - 172.29.244.153 If you have a different Ceph username for the pool, use it as: .. code-block:: console cinder_ceph_client: * The `Ceph documentation for OpenStack`_ has additional information about these settings. * `OpenStack-Ansible and Ceph Working Example`_ .. _Ceph documentation for OpenStack: http://docs.ceph.com/docs/master/rbd/rbd-openstack/ .. _OpenStack-Ansible and Ceph Working Example: https://www.openstackfaq.com/openstack-ansible-ceph/ Config drive ~~~~~~~~~~~~ By default, OpenStack-Ansible does not configure nova to force config drives to be provisioned with every instance that nova builds. The metadata service provides configuration information that is used by ``cloud-init`` inside the instance. Config drives are only necessary when an instance does not have ``cloud-init`` installed or does not have support for handling metadata. A deployer can set an Ansible variable to force config drives to be deployed with every virtual machine: .. code-block:: yaml nova_nova_conf_overrides: DEFAULT: force_config_drive: True Certain formats of config drives can prevent instances from migrating properly between hypervisors. If you need forced config drives and the ability to migrate instances, set the config drive format to ``vfat`` using the ``nova_nova_conf_overrides`` variable: .. code-block:: yaml nova_nova_conf_overrides: DEFAULT: config_drive_format: vfat force_config_drive: True Libvirtd connectivity and authentication ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ By default, OpenStack-Ansible configures the libvirt daemon in the following way: * TLS connections are enabled * TCP plaintext connections are disabled * Authentication over TCP connections uses SASL You can customize these settings using the following Ansible variables: .. code-block:: yaml # Enable libvirtd's TLS listener nova_libvirtd_listen_tls: 1 # Disable libvirtd's plaintext TCP listener nova_libvirtd_listen_tcp: 0 # Use SASL for authentication nova_libvirtd_auth_tcp: sasl Multipath ~~~~~~~~~ Nova supports multipath for iSCSI-based storage. Enable multipath support in nova through a configuration override: .. code-block:: yaml nova_nova_conf_overrides: libvirt: iscsi_use_multipath: true Shared storage and synchronized UID/GID ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Specify a custom UID for the nova user and GID for the nova group to ensure they are identical on each host. This is helpful when using shared storage on Compute nodes because it allows instances to migrate without filesystem ownership failures. By default, Ansible creates the nova user and group without specifying the UID or GID. To specify custom values for the UID or GID, set the following Ansible variables: .. warning:: Setting this value after deploying an environment with OpenStack-Ansible can cause failures, errors, and general instability. These values should only be set once before deploying an OpenStack environment and then never changed. .. code-block:: yaml nova_system_user_uid = nova_system_group_gid = Enabling Huge Pages ~~~~~~~~~~~~~~~~~~~ In order to enable Huge Pages for your kernel, as series of actions must be done: .. note:: It is suggested to leverage ``group_vars``, as usually you need to enable Huge Pages only on specific set of hosts. For example, you can use `/etc/openstack_deploy/group_vars/compute_hosts.yml` file for defining variables discussed below. #. In variables define a default size of Huge Pages. .. code-block:: yaml custom_huge_pages_size_mb: 2 #. Define custom GRUB records to enable Huge Pages .. code-block:: yaml openstack_host_grub_options: - key: hugepagesz value: "{{ custom_huge_pages_size_mb }}M" - key: hugepages value: "{{ (ansible_facts['memtotal_mb'] - nova_reserved_host_memory_mb) // custom_huge_pages_size_mb }}" - key: transparent_hugepage value: never #. Define override for the default ``dev-hugepages.mount``: .. code-block:: yaml openstack_hosts_systemd_mounts: - what: dev-hugepages mount_overrides_only: true type: hugetlbfs escape_name: false config_overrides: Mount: Options: "pagesize={{ custom_huge_pages_size_mb }}M" #. Rollout changes to hosts .. code-block:: shell # openstack-ansible openstack.osa.openstack_hosts_setup --limit compute_hosts #. Define flavors with Huge Pages support .. code-block:: yaml openstack_user_compute: flavors: - specs: - name: m1.small vcpus: 2 ram: 4096 disk: 0 - name: m1.large vcpus: 8 ram: 16384 disk: 0 extra_specs: hw:mem_page_size: large hw:cpu_policy: dedicated #. Create defined flavors in region .. code-block:: shell # openstack-ansible openstack.osa.openstack_resources Enabling post-copy for live migrations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ One of methodologies to ensure successful migration of high-loaded instances is to use ``post-copy`` feature of Libvirt/QEMU. When this method is enabled, when instance fails to migrate with provided timeout the migration is forcefully completed, while remaining (untransferred memory) from the original hypervisor are transferred once being an access attempt happens from the instance. For post-copy to work, it is important so satisfy multiple conditions: * Nova configured to send post-copy to Libvirt while asking for migration: * ``live_migration_timeout_action`` should be set to ``force_complete`` instead of default ``abort`` * ``live_migration_permit_post_copy`` should be enabled * It is recommended to tune ``live_migration_completion_timeout``, as the default of 800 seconds might take too long before deciding that post-copy must be initiated. Please note, that the value of timeout is multiplied by amount of RAM and disk for the instance. So instance with 16Gb RAM and 50Gb disk may take over 14 hours before enforcing post-copy mechanism. * Hypervisor needs to have ``vm.unprivileged_userfaultfd`` enabled to allow post-copy to happen * Ensure Open vSwitch is not using ``mlockall`` for startup Below you can find an example configuration for OpenStack-Ansible to enable post-copy migration for your instances. For that, place the following content into ``/etc/openstack_deploy/group_vars/nova_compute.yml``: .. code:: yaml nova_nova_conf_overrides: libvirt: live_migration_permit_post_copy: true live_migration_timeout_action: force_complete live_migration_completion_timeout: 30 openstack_user_kernel_options: - key: vm.unprivileged_userfaultfd value: 1 Once configuration is in place, you need to run following playbooks to apply changes: .. code-block:: console # openstack-ansible openstack.osa.openstack_hosts_setup --tags openstack_hosts-install --limit nova_compute # openstack-ansible openstack.osa.nova --tags post-install --limit nova_compute