Merge "Support for rollback after the 'software deploy activate' step of upgrade"
This commit is contained in:
@@ -7,459 +7,266 @@
|
||||
Manual Rollback Host Software Deployment
|
||||
========================================
|
||||
|
||||
For a major release software deployment, you can roll back the
|
||||
For a major release software deployment, you can abort and rollback the
|
||||
:ref:`manual-host-software-deployment-ee17ec6f71a4` procedure at any time
|
||||
between :command:`software deploy start` and :command:`software deploy delete`.
|
||||
After the software deploy deletion step, aborting and rolling back of the major
|
||||
release deployment is not possible.
|
||||
|
||||
.. note::
|
||||
|
||||
After the software deploy deletion step, aborting and rolling back of the
|
||||
major release deployment is not possible.
|
||||
|
||||
.. note::
|
||||
This section also covers the abort and rollback of a new patched major
|
||||
release deployment.
|
||||
|
||||
.. note::
|
||||
This section describes different rollback procedures based on where you
|
||||
currently are in the software deployment procedure.
|
||||
|
||||
Currently, software deployments cannot be rolled back after the :command:`software
|
||||
deploy activate` step.
|
||||
.. contents:: |minitoc|
|
||||
:local:
|
||||
:depth: 1
|
||||
|
||||
.. rubric:: |prereq|
|
||||
|
||||
You are in the middle of the
|
||||
:ref:`manual-host-software-deployment-ee17ec6f71a4` procedure for a major
|
||||
release between :command:`software deploy start` and :command:`software deploy
|
||||
delete`.
|
||||
Rollback After Successful Software Deploy Activate
|
||||
--------------------------------------------------
|
||||
|
||||
You can roll back the manual host software deployment after a successful
|
||||
:command:`software deploy activate` in the following scenarios:
|
||||
|
||||
- A Patch Release deployment
|
||||
|
||||
Immediately after the successful :command:`software deploy activate`
|
||||
|
||||
- A major release deployment
|
||||
|
||||
- Immediately after the successful :command:`software deploy activate`, before
|
||||
attempting the Kubernetes upgrade.
|
||||
|
||||
- After attempting a Kubernetes upgrade which failed and was rolled back.
|
||||
|
||||
.. note::
|
||||
|
||||
A rollback of a host major release software deployment can only be done
|
||||
if Kubernetes has been successfully rolled back to its original
|
||||
version.
|
||||
|
||||
.. _rollbackafterdeployactive:
|
||||
|
||||
.. rubric:: |proc|
|
||||
|
||||
#. Abort the current in-progress major release software deployment.
|
||||
#. Verify the current deploy state and Kubernetes current version.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ software deploy abort
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ software deploy show
|
||||
+--------------+------------+------+------------------+
|
||||
| From Release | To Release | RR | State |
|
||||
+--------------+------------+------+------------------+
|
||||
| 24.09.400 | 25.09.0 | True | deploy-completed |
|
||||
+--------------+------------+------+------------------+
|
||||
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ system kube-version-list
|
||||
+---------+--------+-----------+
|
||||
| version | target | state |
|
||||
+---------+--------+-----------+
|
||||
| v1.29.2 | True | active |
|
||||
| v1.30.6 | False | available |
|
||||
| v1.31.5 | False | available |
|
||||
| v1.32.2 | False | available |
|
||||
| v1.33.0 | False | available |
|
||||
+---------+--------+-----------+
|
||||
|
||||
.. _aborthostsoftwaredeploy:
|
||||
|
||||
#. Abort the current host software deployment.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ software deploy abort
|
||||
Deployment has been aborted
|
||||
|
||||
#. If the current deploy state is ``deploy-activate-rollback-pending``, then roll back the
|
||||
activate of the aborted deployment, otherwise proceed to :ref:`3 <manual-rollback-host-software-deployment-9295ce1e6e29-step>`.
|
||||
#. Verify the deploy state.
|
||||
|
||||
.. code-block::
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)]$ software deploy show
|
||||
+------------------+------------+------+----------------------------------+
|
||||
| From Release | To Release | RR | State |
|
||||
+------------------+------------+------+----------------------------------+
|
||||
| <new-maj-rel-id> | 10.0.0 | True | deploy-activate-rollback-pending |
|
||||
+------------------+------------+------+----------------------------------+
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ software deploy show
|
||||
+--------------+------------+------+----------------------------------+
|
||||
| From Release | To Release | RR | State |
|
||||
+--------------+------------+------+----------------------------------+
|
||||
| 25.09.0 | 24.09.400 | True | deploy-activate-rollback-pending |
|
||||
+--------------+------------+------+----------------------------------+
|
||||
|
||||
.. code-block::
|
||||
#. Perform :command:`deploy activate-rollback` and check the deploy and host deploy states.
|
||||
|
||||
~(keystone_admin)]$ software deploy activate-rollback
|
||||
.. code-block:: none
|
||||
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ software deploy activate-rollback
|
||||
Deploy activate-rollback has started
|
||||
|
||||
When running the :command:`software deploy activate-rollback` command, previous
|
||||
configurations are applied to the controller.
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ software deploy show
|
||||
+--------------+------------+------+-------------------------------+
|
||||
| From Release | To Release | RR | State |
|
||||
+--------------+------------+------+-------------------------------+
|
||||
| 25.09.0 | 24.09.400 | True | deploy-activate-rollback-done |
|
||||
+--------------+------------+------+-------------------------------+
|
||||
|
||||
Alarm 250.001 (Configuration is out-of-date) is raised and cleared as the
|
||||
configurations are applied.
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ software deploy host-list
|
||||
+--------------+--------------+------------+------+------------------------------+
|
||||
| Host | From Release | To Release | RR | State |
|
||||
+--------------+--------------+------------+------+------------------------------+
|
||||
| controller-0 | 25.09.0 | 24.09.400 | True | deploy-host-rollback-pending |
|
||||
| controller-1 | 25.09.0 | 24.09.400 | True | deploy-host-rollback-pending |
|
||||
+--------------+--------------+------------+------+------------------------------+
|
||||
|
||||
The software deployment state goes from ``activate-rollback-done`` to ``host-rollback``.
|
||||
.. _rollbackhost:
|
||||
|
||||
This may take up to 30 mins to complete depending on system configuration
|
||||
and hardware.
|
||||
#. Roll back host.
|
||||
|
||||
.. code-block::
|
||||
Perform the following commands in the order: all worker nodes, all
|
||||
storage nodes, controller-1, and controller-0:
|
||||
|
||||
~(keystone_admin)]$ software deploy show
|
||||
+------------------+------------+------+---------------+
|
||||
| From Release | To Release | RR | State |
|
||||
+------------------+------------+------+---------------+
|
||||
| <new-maj-rel-id> | 10.0.0 | True | host-rollback |
|
||||
+------------------+------------+------+---------------+
|
||||
.. code-block:: none
|
||||
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ system host-lock <host>
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ software deploy host-rollback <host>
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ system host-unlock <host>
|
||||
|
||||
.. note::
|
||||
|
||||
If :command:`software deploy activate-rollback` fails, that is, if the state is
|
||||
``activate-rollback-failed``, review ``/var/log/software.log`` on the
|
||||
active controller for failure details, address the issues, and
|
||||
re-execute the :command:`software deploy activate-rollback` command.
|
||||
For controllers, a :command:`system host-swact controller-#` will be
|
||||
required to lock and roll back the active controller.
|
||||
|
||||
#. If the current deploy state is ``host-rollback``, then roll back the
|
||||
deployment of all the hosts.
|
||||
.. _verifydeploystate:
|
||||
|
||||
.. _manual-rollback-host-software-deployment-9295ce1e6e29-step:
|
||||
#. Verify the deploy state and host deploy state.
|
||||
|
||||
.. code-block::
|
||||
All the hosts must be in the ``deploy-host-rollback-deployed`` state and the deploy state
|
||||
should be ``deploy-host-rollback-done``.
|
||||
|
||||
~(keystone_admin)]$ software deploy show
|
||||
+------------------+------------+------+-------------------+
|
||||
| From Release | To Release | RR | State |
|
||||
+------------------+------------+------+-------------------+
|
||||
| <new-maj-rel-id> | 10.0.0 | True | host-rollback |
|
||||
+------------------+------------+------+-------------------+
|
||||
.. code-block:: none
|
||||
|
||||
If the state is ``host-rollback``, then proceed with the rest of this step,
|
||||
otherwise proceed to :ref:`4 <manual-rollback-host-software-deployment-9295ce1e6e29-deletestep>`.
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ software deploy host-list
|
||||
+--------------+--------------+------------+------+-------------------------------+
|
||||
| Host | From Release | To Release | RR | State |
|
||||
+--------------+--------------+------------+------+-------------------------------+
|
||||
| controller-0 | 25.09.0 | 24.09.400 | True | deploy-host-rollback-deployed |
|
||||
| controller-1 | 25.09.0 | 24.09.400 | True | deploy-host-rollback-deployed |
|
||||
+--------------+--------------+------------+------+-------------------------------+
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ software deploy show
|
||||
+--------------+------------+------+---------------------------+
|
||||
| From Release | To Release | RR | State |
|
||||
+--------------+------------+------+---------------------------+
|
||||
| 25.09.0 | 24.09.400 | True | deploy-host-rollback-done |
|
||||
+--------------+------------+------+---------------------------+
|
||||
|
||||
- For an |AIO-SX| system
|
||||
.. _deletecurrentdeploy:
|
||||
|
||||
#. Roll back the software release on controller-0.
|
||||
#. Delete the current deployment.
|
||||
|
||||
#. Lock controller-0.
|
||||
.. code-block:: none
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ system host-lock controller-0
|
||||
|
||||
#. Roll back the software release on controller-0.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ software deploy host-rollback controller-0
|
||||
Host installation request sent to controller-0.
|
||||
Host installation was successful on controller-0
|
||||
|
||||
The host is still running the new software release, however boot
|
||||
parameters have been updated to boot into the previous software
|
||||
release on the next host reboot, which will occur in the next step
|
||||
which unlocks the host.
|
||||
|
||||
#. Unlock controller-0.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ system host-unlock controller-0
|
||||
|
||||
The host will now reboot into the previous software release. Wait for
|
||||
the host to finish rebooting and become available.
|
||||
|
||||
This may take 3-5 mins depending on hardware.
|
||||
|
||||
#. Proceed to step :ref:`4
|
||||
<manual-rollback-host-software-deployment-9295ce1e6e29-deletestep>` of
|
||||
the main procedure.
|
||||
|
||||
- For an |AIO-DX| system or standard system
|
||||
|
||||
#. If worker hosts are present, and one or more are in the ``pending-rollback``
|
||||
state, then roll back the software release on all worker hosts in the
|
||||
``pending-rollback`` state one at a time. Otherwise, proceed to step :ref:`b <manual-rollback-host-software-deployment-9295ce1e6e29-storagehost>`.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ software deploy host-list
|
||||
+--------------+------------------+------------+-------+------------------------------+
|
||||
| Host | From Release | To Release | RR | State |
|
||||
+--------------+------------------+------------+-------+------------------------------+
|
||||
| controller-0 | <new-maj-rel-id> | 10.0.0 | True | deploy-host-rollback-pending |
|
||||
| controller-1 | <new-maj-rel-id> | 10.0.0 | True | deploy-host-rollback-pending |
|
||||
| storage-0 | <new-maj-rel-id> | 10.0.0 | True | deploy-host-rollback-pending |
|
||||
| storage-1 | <new-maj-rel-id> | 10.0.0 | True | deploy-host-rollback-pending |
|
||||
| storage-2 | <new-maj-rel-id> | 10.0.0 | True | deploy-host-rollback-pending |
|
||||
| storage-3 | <new-maj-rel-id> | 10.0.0 | True | deploy-host-rollback-pending |
|
||||
| worker-0 | <new-maj-rel-id> | 10.0.0 | True | deploy-host-rollback-pending |
|
||||
| worker-1 | <new-maj-rel-id> | 10.0.0 | True | deploy-host-rollback-pending |
|
||||
| worker-2 | <new-maj-rel-id> | 10.0.0 | True | deploy-host-rollback-pending |
|
||||
| worker-3 | <new-maj-rel-id> | 10.0.0 | True | deploy-host-rollback-deployed|
|
||||
+--------------+------------------+------------+-------+------------------------------+
|
||||
|
||||
#. Roll back the software release on worker-0.
|
||||
|
||||
#. Lock worker-0.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ system host-lock worker-0
|
||||
|
||||
#. Roll back the software release on worker-0.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ software deploy host-rollback worker-0
|
||||
Host installation request sent to worker-0
|
||||
Host installation was successful on worker-0.
|
||||
|
||||
The host is still running the new software release, however boot parameters
|
||||
have been updated to boot into the previous software release on the next
|
||||
host reboot, which will occur in the next step which unlocks the host.
|
||||
|
||||
#. Unlock worker-0.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ system host-unlock worker-0
|
||||
|
||||
The host will now reboot into the previous software release. Wait
|
||||
for the host to finish rebooting and become available. Wait
|
||||
for all the alarms to clear after the unlock before proceeding to the
|
||||
next worker host.
|
||||
|
||||
This may take 3-5 mins depending on hardware.
|
||||
|
||||
#. Display the state of software deployment.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ software deploy host-list
|
||||
+--------------+------------------+------------+-------+------------------+
|
||||
| Host | From Release | To Release | RR | State |
|
||||
+--------------+------------------+------------+-------+------------------+
|
||||
| controller-0 | <new-maj-rel-id> | 10.0.0 | True | pending-rollback |
|
||||
| controller-1 | <new-maj-rel-id> | 10.0.0 | True | pending-rollback |
|
||||
| storage-0 | <new-maj-rel-id> | 10.0.0 | True | pending-rollback |
|
||||
| storage-1 | <new-maj-rel-id> | 10.0.0 | True | pending-rollback |
|
||||
| storage-2 | <new-maj-rel-id> | 10.0.0 | True | pending-rollback |
|
||||
| storage-3 | <new-maj-rel-id> | 10.0.0 | True | pending-rollback |
|
||||
| worker-0 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| worker-1 | <new-maj-rel-id> | 10.0.0 | True | pending-rollback |
|
||||
| worker-2 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| worker-3 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
+--------------+------------------+------------+-------+------------------+
|
||||
|
||||
#. Repeat the above steps for any remaining worker hosts in the ``pending-rollback`` state.
|
||||
|
||||
#. If storage hosts are present, and one or more are in the ``pending-rollback`` state,
|
||||
then roll back the software release on all storage hosts in the ``pending-rollback`` state,
|
||||
one at a time. Otherwise, proceed to step :ref:`c <manual-rollback-host-software-deployment-9295ce1e6e29-bothcontrollers>`.
|
||||
|
||||
.. _manual-rollback-host-software-deployment-9295ce1e6e29-storagehost:
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ software deploy host-list
|
||||
+--------------+------------------+------------+-------+------------------+
|
||||
| Host | From Release | To Release | RR | State |
|
||||
+--------------+------------------+------------+-------+------------------+
|
||||
| controller-0 | <new-maj-rel-id> | 10.0.0 | True | pending-rollback |
|
||||
| controller-1 | <new-maj-rel-id> | 10.0.0 | True | pending-rollback |
|
||||
| storage-0 | <new-maj-rel-id> | 10.0.0 | True | pending-rollback |
|
||||
| storage-1 | <new-maj-rel-id> | 10.0.0 | True | pending-rollback |
|
||||
| storage-2 | <new-maj-rel-id> | 10.0.0 | True | pending-rollback |
|
||||
| storage-3 | <new-maj-rel-id> | 10.0.0 | True | pending-rollback |
|
||||
| worker-0 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| worker-1 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| worker-2 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| worker-3 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
+--------------+------------------+------------+-------+------------------+
|
||||
|
||||
#. Roll back the software release on storage-0.
|
||||
|
||||
#. Lock storage-0.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ system host-lock storage-0
|
||||
|
||||
#. Roll back the software release on storage-0.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ software deploy host-rollback storage-0
|
||||
Host installation request sent to storage-0
|
||||
Host installation was successful on storage-0.
|
||||
|
||||
The host is still running the new software release,
|
||||
however boot parameters have been updated to boot into
|
||||
the previous software release on the next host reboot, which
|
||||
will occur in the next step which unlocks the host.
|
||||
|
||||
#. Unlock storage-0.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ system host-unlock storage-0
|
||||
|
||||
The host will now reboot into the previous software release. Wait
|
||||
for the host to finish rebooting and become available. Wait for
|
||||
all the alarms to clear after the unlock before proceeding to the next
|
||||
storage host.
|
||||
|
||||
This may take 3-5 mins depending on hardware.
|
||||
|
||||
#. Display the state of software deployment.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ software deploy host-list
|
||||
+--------------+------------------+------------+-------+------------------+
|
||||
| Host | From Release | To Release | RR | State |
|
||||
+--------------+------------------+------------+-------+------------------+
|
||||
| controller-0 | <new-maj-rel-id> | 10.0.0 | True | pending-rollback |
|
||||
| controller-1 | <new-maj-rel-id> | 10.0.0 | True | pending-rollback |
|
||||
| storage-0 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| storage-1 | <new-maj-rel-id> | 10.0.0 | True | pending-rollback |
|
||||
| storage-2 | <new-maj-rel-id> | 10.0.0 | True | pending-rollback |
|
||||
| storage-3 | <new-maj-rel-id> | 10.0.0 | True | pending-rollback |
|
||||
| worker-0 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| worker-1 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| worker-2 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| worker-3 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
+--------------+------------------+------------+-------+------------------+
|
||||
|
||||
#. Repeat the above steps for any remaining storage hosts in the
|
||||
``pending-rollback`` state.
|
||||
|
||||
.. note::
|
||||
|
||||
After rolling back the first storage host, you can expect alarm
|
||||
800.003. The alarm is cleared after all the storage hosts are rolled
|
||||
back.
|
||||
|
||||
#. If both the controllers are in the ``pending-rollback`` state, then roll back
|
||||
controller-0 first.
|
||||
|
||||
.. _manual-rollback-host-software-deployment-9295ce1e6e29-bothcontrollers:
|
||||
|
||||
#. Ensure that controller-1 is active by switching activity from
|
||||
controller-0.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ system host-swact controller-0
|
||||
|
||||
Wait for the activity to switch to controller-1. This may take up to a
|
||||
minute depending on hardware. Reconnect to the system.
|
||||
|
||||
#. Roll back the software release on controller-0 (the standby controller).
|
||||
|
||||
#. Lock controller-0.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ system host-lock controller-0
|
||||
|
||||
#. Rollback the software release on controller-0.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ software deploy host-rollback controller-0
|
||||
Host installation request sent to controller-0.
|
||||
Host installation was successful on controller-0.
|
||||
|
||||
The host is still running the new software release,
|
||||
however boot parameters have been updated to boot into
|
||||
the previous software release on the next host reboot, which
|
||||
will occur in the next step which unlocks the host.
|
||||
|
||||
#. Unlock controller-0.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ system host-unlock controller-0
|
||||
|
||||
The host will now reboot into the new software release. Wait for
|
||||
the host to finish rebooting and become available.
|
||||
|
||||
This may take 3-5 mins depending on hardware.
|
||||
|
||||
#. Display the state of software deployment.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ software deploy host-list
|
||||
+--------------+------------------+------------+-------+------------------+
|
||||
| Host | From Release | To Release | RR | State |
|
||||
+--------------+------------------+------------+-------+------------------+
|
||||
| controller-0 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| controller-1 | <new-maj-rel-id> | 10.0.0 | True | pending-rollback |
|
||||
| storage-0 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| storage-1 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| storage-2 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| storage-3 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| worker-0 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| worker-1 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| worker-2 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| worker-3 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
+--------------+------------------+------------+-------+------------------+
|
||||
|
||||
#. If only controller-1 is in the ``pending-rollback`` state, then roll
|
||||
back controller-1.
|
||||
|
||||
#. Ensure that controller-0 is active by switching activity from
|
||||
controller-1.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ system host-swact controller-1
|
||||
|
||||
Wait for the activity to switch to controller-0.
|
||||
|
||||
This may take up to a minute depending on hardware.
|
||||
|
||||
Reconnect to the system.
|
||||
|
||||
#. Roll back the software release on controller-1 (the standby controller).
|
||||
|
||||
#. Lock controller-1
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ system host-lock controller-1
|
||||
|
||||
#. Roll back the software release on controller-1.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ software deploy host-rollback controller-1
|
||||
Host installation request sent to controller-1.
|
||||
Host installation was successful on controller-1.
|
||||
|
||||
The host is still running the new software release, however boot
|
||||
parameters have been updated to boot into the previous software
|
||||
release on the next host reboot, which will occur in the next step
|
||||
which unlocks the host.
|
||||
|
||||
#. Unlock controller-1.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ system host-unlock controller-1
|
||||
|
||||
The host will now reboot into the new software release. Wait for
|
||||
the host to finish rebooting and become available.
|
||||
|
||||
This may take 3-5 mins depending on hardware.
|
||||
|
||||
#. Display the state of software deployment.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ software deploy host-list
|
||||
+--------------+------------------+------------+-------+------------------+
|
||||
| Host | From Release | To Release | RR | State |
|
||||
+--------------+------------------+------------+-------+------------------+
|
||||
| controller-0 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| controller-1 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| storage-0 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| storage-1 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| storage-2 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| storage-3 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| worker-0 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| worker-1 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| worker-2 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
| worker-3 | <new-maj-rel-id> | 10.0.0 | True | rolled back |
|
||||
+--------------+------------------+------------+-------+------------------+
|
||||
|
||||
#. Delete the software deployment to complete the rollback.
|
||||
|
||||
.. _manual-rollback-host-software-deployment-9295ce1e6e29-deletestep:
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ software deploy delete
|
||||
Deployment has been deleted
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ software deploy show
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ software deploy delete
|
||||
Deploy deleted with success
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ software deploy show
|
||||
No deploy in progress
|
||||
|
||||
#. Confirm that the previous software release is now deployed.
|
||||
Now, the system is ready for the next host software deployment.
|
||||
|
||||
Rollback After Software Deploy Activate Fails
|
||||
---------------------------------------------
|
||||
|
||||
When :command:`software deploy activate` fails, follow the steps below to abort
|
||||
and roll back the manual host software deployment.
|
||||
|
||||
.. rubric:: |proc|
|
||||
|
||||
#. Verify current deploy state.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ software deploy show
|
||||
+--------------+------------+------+------------------------+
|
||||
| From Release | To Release | RR | State |
|
||||
+--------------+------------+------+------------------------+
|
||||
| 24.09.400 | 25.09.0 | True | deploy-activate-failed |
|
||||
+--------------+------------+------+------------------------+
|
||||
|
||||
#. Follow the steps from :ref:`step 2 <aborthostsoftwaredeploy>` to :ref:`step
|
||||
7 <deletecurrentdeploy>` in :ref:`Rollback After Successful Software Deploy
|
||||
Activate <rollbackafterdeployactive>`.
|
||||
|
||||
|
||||
Rollback After Software Deploy Host <hostname> Fails
|
||||
----------------------------------------------------
|
||||
|
||||
When deploying software to a host (:command:`software deploy host <hostname>`) fails,
|
||||
follow the steps below to abort and roll back the manual host software
|
||||
deployment.
|
||||
|
||||
.. rubric:: |proc|
|
||||
|
||||
#. Verify the current deploy state.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ software deploy show
|
||||
+--------------+------------+------+--------------------+
|
||||
| From Release | To Release | RR | State |
|
||||
+--------------+------------+------+--------------------+
|
||||
| 24.09.400 | 25.09.0 | True | deploy-host-failed |
|
||||
+--------------+------------+------+--------------------+
|
||||
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ software deploy host-list
|
||||
+--------------+--------------+------------+------+---------------------+
|
||||
| Host | From Release | To Release | RR | State |
|
||||
+--------------+--------------+------------+------+---------------------+
|
||||
| controller-0 | 25.09.0 | 24.09.400 | True | deploy-host-failed |
|
||||
| controller-1 | 25.09.0 | 24.09.400 | True | deploy-host-done |
|
||||
| compute-0 | 25.09.0 | 24.09.400 | True | deploy-host-pending |
|
||||
+--------------+--------------+------------+------+---------------------+
|
||||
|
||||
#. Abort the deploy.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ software deploy abort
|
||||
Deployment has been aborted
|
||||
|
||||
#. Verify the deploy state.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ software deploy host-list
|
||||
+--------------+--------------+------------+------+------------------------------+
|
||||
| Host | From Release | To Release | RR | State |
|
||||
+--------------+--------------+------------+------+------------------------------+
|
||||
| controller-0 | 25.09.0 | 24.09.400 | True | deploy-host-rollback-pending |
|
||||
+--------------+--------------+------------+------+------------------------------+
|
||||
|
||||
[sysadmin@controller-0 ~(keystone_admin)]$ software deploy host-list
|
||||
+--------------+--------------+------------+------+-------------------------------+
|
||||
| Host | From Release | To Release | RR | State |
|
||||
+--------------+--------------+------------+------+-------------------------------+
|
||||
| controller-0 | 25.09.0 | 24.09.400 | True | deploy-host-rollback-pending |
|
||||
| controller-1 | 25.09.0 | 24.09.400 | True | deploy-host-rollback-pending |
|
||||
| compute-0 | 25.09.0 | 24.09.400 | True | deploy-host-rollback-deployed |
|
||||
+--------------+--------------+------------+------+-------------------------------+
|
||||
|
||||
Follow :ref:`step 5 <rollbackhost>` in :ref:`Rollback After Successful Software
|
||||
Deploy Activate <rollbackafterdeployactive>`. Roll back all the hosts that are
|
||||
in ``deploy-host-rollback-pending`` until all the hosts are in
|
||||
``deploy-host-rollback-deployed``. Then, follow :ref:`step 6
|
||||
<verifydeploystate>` and :ref:`step 7 <deletecurrentdeploy>` to complete the
|
||||
rollback.
|
||||
|
||||
|
||||
Rollback Before Software Deploy Host <hostname> to Any Host
|
||||
-----------------------------------------------------------
|
||||
|
||||
If a host software deployment needs to be cancelled before the software is
|
||||
deployed to any host, that is, if the deploy state is either
|
||||
``deploy-start-done`` or ``deploy-start-failed``, you can delete the pending
|
||||
deployment directly using :command:`software deploy delete`.
|
||||
|
||||
.. code-block::
|
||||
|
||||
~(keystone_admin)]$ software list
|
||||
+--------------------------+-------+-----------+
|
||||
| Release | RR | State |
|
||||
+--------------------------+-------+-----------+
|
||||
| starlingx-10.0.0 | True | deployed |
|
||||
| <new-maj-rel-id> | True | available |
|
||||
+--------------------------+-------+-----------+
|
||||
|
Reference in New Issue
Block a user