Merge "Add spec for downloading images via RBD"
This commit is contained in:
Zuul
201
specs/victoria/approved/nova-image-download-via-rbd.rst
Normal file
201
specs/victoria/approved/nova-image-download-via-rbd.rst
Normal file
@@ -0,0 +1,201 @@
|
||||
..
|
||||
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||
License.
|
||||
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
=====================================================
|
||||
Allow Nova to download Glance images directly via RBD
|
||||
=====================================================
|
||||
|
||||
https://blueprints.launchpad.net/nova/+spec/nova-image-download-via-rbd
|
||||
|
||||
|
||||
Problem description
|
||||
===================
|
||||
|
||||
When using compute-local storage with qcow2 based VM root disks, Glance images
|
||||
are downloaded into the libvirt image store by way of the Glance HTTP API.
|
||||
For images in the 10s-100s of GB, this download can be _very_ slow.
|
||||
If the compute node has access to Ceph, it can instead perform an 'rbd export'
|
||||
on the Glance image, bypassing the Glance API entirely and directly download
|
||||
the image from Ceph. This direct download can result in a drastic reduction
|
||||
in download time, from tens of minutes to tens of seconds.
|
||||
|
||||
Use Cases
|
||||
---------
|
||||
|
||||
As a user with a Ceph-backed image storage, I want to configure some compute
|
||||
hosts for qcow2 images local to the compute host but quickly get the images
|
||||
from Ceph rather than slow downloads from the Glance API.
|
||||
|
||||
Proposed change
|
||||
===============
|
||||
|
||||
A special download handler will be registered for Glance images when the 'rbd'
|
||||
value is present in ``allowed_direct_url_schemes`` option.
|
||||
|
||||
This download handler will be called only when a VM is scheduled on a node and
|
||||
the required Glance image is not already present in the local libvirt image
|
||||
cache. It will execute the OS native 'rbd export' command, using ``privsep``,
|
||||
in order to perform the download operation instead of using the Glance HTTP
|
||||
API.
|
||||
|
||||
The mechanism for per-scheme download handlers was previously available
|
||||
as a plugin point, which is now deprecated, along with the
|
||||
allowed_direct_url_schemes config option. This effort will close out on that
|
||||
deprecation by moving the per-scheme support into the nova.images.glance module
|
||||
itself, undeprecating the allowed_direct_url_schemes config, and removing the
|
||||
old nova.images.download plug point.
|
||||
|
||||
The glance module also never used to perform image signature verification when
|
||||
the per-scheme module was used. Since we are moving this into core code,
|
||||
we will also fix this so that per-scheme images are verified like all the rest.
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
|
||||
VM root disks can be run directly within Ceph as creation of these VM root
|
||||
disks are fast as they are COW clones for the Glance image, also in Ceph.
|
||||
However, running the VM root disks from Ceph introduces additional latency to
|
||||
the running VM and needlessly wastes network bandwidth and Ceph IOPS. This
|
||||
specific functionality was added in Mitaka but is aimed at a different use case
|
||||
where the VM root disks remain in Ceph and are not run as qcow2 local disks.
|
||||
|
||||
https://specs.openstack.org/openstack/nova-specs/specs/mitaka/implemented/rbd-instance-snapshots.html
|
||||
|
||||
The other alternative is to continue with existing approach only.
|
||||
|
||||
Data model impact
|
||||
-----------------
|
||||
|
||||
None
|
||||
|
||||
REST API impact
|
||||
---------------
|
||||
|
||||
None
|
||||
|
||||
Security impact
|
||||
---------------
|
||||
|
||||
None
|
||||
|
||||
Notifications impact
|
||||
--------------------
|
||||
|
||||
None
|
||||
|
||||
Other end user impact
|
||||
---------------------
|
||||
|
||||
None
|
||||
|
||||
Performance Impact
|
||||
------------------
|
||||
|
||||
None
|
||||
|
||||
Other deployer impact
|
||||
---------------------
|
||||
|
||||
As proposed, there are no new configuration items, simply configuration of
|
||||
existing items.
|
||||
|
||||
The following configuration options are required to ensure qcow2 local images
|
||||
are downloaded from Ceph and cached on the local compute host:
|
||||
|
||||
On the Glance API node in glance-api.conf:
|
||||
|
||||
``DEFAULT.show_image_direct_url=true``
|
||||
|
||||
On the Nova compute node in nova.conf:
|
||||
|
||||
``DEFAULT.force_raw_images=false``
|
||||
|
||||
``libvirt.images_type=qcow2``
|
||||
|
||||
``libvirt.images_rbd_ceph_conf=<ceph_config_file>``
|
||||
|
||||
``libvirt.rbd_user=<ceph_user_name>``
|
||||
|
||||
``glance.allowed_direct_url_schemes = rbd``
|
||||
|
||||
Looking ahead, it may be desired to create additional entries in the libvirt
|
||||
section of ``nova.conf`` for this feature as the current implementation assumes
|
||||
that the ``rbd_user`` will have access to the Glance images. This may not be
|
||||
the case depending upon how the Ceph pool permissions are configured.
|
||||
|
||||
Developer impact
|
||||
----------------
|
||||
|
||||
The ``allowed_direct_url_schemes`` option was deprecated in Queens. Proposed
|
||||
implementation of this feature would halt the deprecation of this option and
|
||||
we would need to "un-deprecate" it.
|
||||
|
||||
Upgrade impact
|
||||
--------------
|
||||
|
||||
None
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
|
||||
Primary assignee:
|
||||
Jiri Suchomel <jiri.suchomel@suse.com>
|
||||
|
||||
Feature Liaison
|
||||
---------------
|
||||
|
||||
Feature liaison:
|
||||
Dan Smith (danms)
|
||||
|
||||
Work Items
|
||||
----------
|
||||
|
||||
* Refactor existing in-house out-of-tree implementation and integrate it fully
|
||||
into current codebase
|
||||
* Write tests for implementation
|
||||
* Update the admin guide with the description of how to set up the config if
|
||||
the new feature is required.
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
None
|
||||
|
||||
Testing
|
||||
=======
|
||||
|
||||
* Unit tests
|
||||
* Add an experimental on-demand queue job which uses Ceph with local qcow2
|
||||
images and 'direct from rbd' feature enabled
|
||||
|
||||
Documentation Impact
|
||||
====================
|
||||
|
||||
The admin guide should be updated to call out this use case and how it differs
|
||||
from the Ceph-native snapshot feature. A good place to document this may be:
|
||||
|
||||
https://docs.openstack.org/nova/latest/admin/configuration/hypervisor-kvm.html#configure-compute-backing-storage
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
http://lists.openstack.org/pipermail/openstack-dev/2018-May/131002.html
|
||||
|
||||
http://lists.openstack.org/pipermail/openstack-operators/2018-June/015384.html
|
||||
|
||||
History
|
||||
=======
|
||||
|
||||
.. list-table:: Revisions
|
||||
:header-rows: 1
|
||||
|
||||
* - Release Name
|
||||
- Description
|
||||
* - Victoria
|
||||
- Introduced
|
||||
Reference in New Issue
Block a user