diff --git a/doc/source/admin/install/dashboard_guide.rst b/doc/source/admin/install/dashboard_guide.rst index f3ab86861..a645c0045 100644 --- a/doc/source/admin/install/dashboard_guide.rst +++ b/doc/source/admin/install/dashboard_guide.rst @@ -7,59 +7,35 @@ control mistral objects by interacting with web user interface. Setup Instructions ------------------ + This instruction assumes that Horizon is already installed and it's -installation folder is . Detailed information on how to install +installation folder is **. + +Detailed information on how to install Horizon can be found at `Horizon Installation `_ -The installation folder of Mistral Dashboard will be referred to as -. +The following steps should get you started: -The following should get you started: +#. Install mistral-dashboard in horizon virtual env: -1. Clone the repository into your local OpenStack directory:: +.. code-block:: console - $ git clone https://github.com/openstack/mistral-dashboard.git + $ pip install mistral-dashboard -2. Install mistral-dashboard:: +#. Enable the dashboard: - $ sudo pip install -e +.. code-block:: console - Or if you're planning to run Horizon server in a virtual environment - (see below):: - - $ tox -evenv -- pip install -e ../mistral-dashboard/ - - and then:: - - $ cp -b /mistraldashboard/enabled/_50_mistral.py \ + $ cp -b /mistraldashboard/enabled/_50_mistral.py \ /openstack_dashboard/local/enabled/_50_mistral.py -3. Since Mistral only supports Identity v3, you must ensure that the dashboard - points the proper OPENSTACK_KEYSTONE_URL in - /openstack_dashboard/local/local_settings.py file:: +#. When you're ready, you would need to restart horizon, e.g. with apache: - OPENSTACK_API_VERSIONS = { - "identity": 3, - } +.. code-block:: console - OPENSTACK_KEYSTONE_URL = "http://%s:5000/v3" % OPENSTACK_HOST + # Debian based + $ systemctl restart apache2 -4. Also, make sure you have changed OPENSTACK_HOST to point to your Keystone - server and check all endpoints are accessible. You may want to change - OPENSTACK_ENDPOINT_TYPE to "publicURL" if some of them are not. - -5. When you're ready, you would need to either restart your apache:: - - $ sudo service apache2 restart - - or run the development server - (in case you have decided to use local horizon):: - - $ cd ../horizon/ - $ tox -evenv -- python manage.py runserver - -Debug instructions ------------------- -Please refer to :doc:`Mistral Troubleshooting -<../../contributor/debugging_and_testing>` + # Or RHEL/CentOS + $ systemctl restart httpd diff --git a/doc/source/admin/install/get_started.rst b/doc/source/admin/install/get_started.rst deleted file mode 100644 index 15b2b36c5..000000000 --- a/doc/source/admin/install/get_started.rst +++ /dev/null @@ -1,29 +0,0 @@ -========================= -Workflow Service Overview -========================= - - -The Workflow service consists of the following components: - -``Mistral API`` service - Provides a REST API for operating and monitoring workflow executions. - -``mistral-dashboard`` service - Mistral Dashboard is a Horizon plugin. - -``Mistral Engine`` service - Controls workflow executions and handles their data flow, places finished - tasks in a queue, transfers data from task to task, and deals with condition - transitions, and so on. - -``Mistral Executor`` service - Executes task actions, picks up the tasks from the queue, runs actions, and - sends results back to the engine. - -``Mistral Notifier`` service - -``python-mistralclient`` - Python client API and Command Line Interface. - - ``mistral-lib`` - A library to support writing custom Mistral actions. diff --git a/doc/source/admin/install/index.rst b/doc/source/admin/install/index.rst index 5b68a7c97..f787bcf41 100644 --- a/doc/source/admin/install/index.rst +++ b/doc/source/admin/install/index.rst @@ -5,13 +5,9 @@ Installation Guide .. toctree:: :maxdepth: 1 - get_started install verify - next-steps dashboard_guide - installation_guide - mistralclient_guide The Workflow service (mistral) enables setting up task relations that have to be executed in a particular order, called workflows. diff --git a/doc/source/admin/install/install-obs.rst b/doc/source/admin/install/install-obs.rst deleted file mode 100644 index c058eb1ef..000000000 --- a/doc/source/admin/install/install-obs.rst +++ /dev/null @@ -1,10 +0,0 @@ -.. _install-obs: - - -Install and configure for openSUSE and SUSE Linux Enterprise -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -For information on how to install and configure the Workflow service -for openSUSE and SUSE Linux Enterprise, refer to the :doc:`Installation guide -for Ubuntu `. Note that some commands vary by distribution and -might differ from the ones described, for instance, package management. diff --git a/doc/source/admin/install/install-rdo.rst b/doc/source/admin/install/install-rdo.rst deleted file mode 100644 index cd7353e03..000000000 --- a/doc/source/admin/install/install-rdo.rst +++ /dev/null @@ -1,10 +0,0 @@ -.. _install-rdo: - -Install and configure for Red Hat Enterprise Linux and CentOS -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - -For information on how to install and configure the Workflow service -for Red Hat Enterprise Linux 7 and CentOS 7, refer to the :doc:`Installation -guide for Ubuntu `. Note that some commands vary by distribution -and might differ from the ones described, for instance, package management. diff --git a/doc/source/admin/install/install-ubuntu.rst b/doc/source/admin/install/install-ubuntu.rst deleted file mode 100644 index c0f2019dd..000000000 --- a/doc/source/admin/install/install-ubuntu.rst +++ /dev/null @@ -1,212 +0,0 @@ -.. _install-ubuntu: - -================================ -Install and Configure for Ubuntu -================================ - -This section describes how to install and configure the Workflow Service -service for Ubuntu. - -Prerequisites -------------- - -#. Install the packages: - - .. code-block:: console - - # apt-get update - - # apt-get install python-setuptools python-pip libffi-dev libxslt1-dev \ - libxml2-dev libyaml-dev libssl-dev python3-dev tox mistral-common - -Installation ------------- - -**NOTE**: For instructions on how to install Mistral using devstack, refer to -:doc:`Mistral Devstack Installation <../../contributor/devstack>` - -Clone the repo and go to the repo directory: - -.. code-block:: console - - $ git clone https://opendev.org/openstack/mistral - $ cd mistral - -Generate the configuration file: - -.. code-block:: console - - $ tox -egenconfig - -Create the mistral directory and copy the example configuration file: - -.. code-block:: console - - $ mkdir /etc/mistral - $ cp etc/mistral.conf.sample /etc/mistral/mistral.conf - - -Edit the configuration file: - - .. code-block:: console - - $ vi /etc/mistral/mistral.conf - - - -**Virtualenv installation**: - -.. code-block:: console - - $ tox - -This installs the necessary virtual environments and runs all the project -tests. Installing the virtual environments may take significant time (~10-15 -mins). - -**Local installation**: - -.. code-block:: console - - $ pip install -e . - -or: - -.. code-block:: console - - $ pip install -r requirements.txt - $ python setup.py install - -**NOTE**: There are some differences between *pip install -e* and *setup.py -install*. **pip install -e** works similarly to **setup.py install** -or the EasyInstall tool, however, it does not actually install anything. -Instead, it creates a special .egg-link file in the deployment directory that -links to your project’s source code. - -Configuring Mistral -------------------- - -Refer :doc:`../configuration/index` to find general information on how to -configure Mistral server. - - -Before The First Run --------------------- - -After the installation, you will see the **mistral-server** and -**mistral-db-manage** commands in your environment, either in system or virtual -environment. - -**NOTE**: If you use **virtualenv**, all Mistral-related commands can be -accessed with **tox -evenv --**. For example, *mistral-server* is available via -*tox -evenv -- mistral-server*. - -The **mistral-db-manage** command can be used for migrations. - -Updating the database to the latest revision type: - -.. code-block:: console - - $ mistral-db-manage --config-file upgrade head - -Before starting the Mistral server, run the *mistral-db-manage populate* -command. It creates the DB with all the standard actions and standard workflows -that Mistral provides to all Mistral users.: - -.. code-block:: console - - $ mistral-db-manage --config-file populate - -For more detailed information on the *mistral-db-manage* script, see -the :doc:`Mistral Upgrade Guide `. - -**NOTE**: For users who want a dry run with an **SQLite** database backend (not -used in production), the *mistral-db-manage* script is not recommended for -database initialization because of -`SQLite limitations `_. -Use the sync_db script described below for database -initialization instead. - -**If you use virtualenv**: - -.. code-block:: console - - $ tools/sync_db.sh --config-file - -**Or run sync_db directly**: - -.. code-block:: console - - $ python tools/sync_db.py --config-file - -Running Mistral API server --------------------------- - -To run the Mistral API server, execute the following command in a shell: - -.. code-block:: console - - $ mistral-server --server api --config-file - -Running Mistral Engines ------------------------ - -To run the Mistral Engine, execute the following command in a shell: - -.. code-block:: console - - $ mistral-server --server engine --config-file - -Running Mistral Executors -------------------------- -To run the Mistral Executor instance, execute the following command in a -shell: - -.. code-block:: console - - $ mistral-server --server executor --config-file - -Note that at least one Engine instance and one Executor instance should be -running so that workflow tasks are processed by Mistral. - -Mistral Notifier ----------------- - -To run the Mistral Notifier, execute the following command in a shell: - -.. code-block:: console - - $ mistral-server --server notifier --config-file - -Running Multiple Mistral Servers Under the Same Process -------------------------------------------------------- -To run more than one server (API, Engine, or Task Executor) on the same process, -execute the following command in a shell: - -.. code-block:: console - - $ mistral-server --server api,engine --config-file - -The --server command line option can be a comma delimited list. The valid -options are "all" (by default if not specified) or any combination of "api", -"engine", and "executor". It is important to note that the "fake" transport for -the rpc_backend defined in the config file should only be used if "all" the -Mistral servers are launched on the same process. Otherwise, messages do not -get delivered if the Mistral servers are launched on different processes -because the "fake" transport is using an in-process queue. - - -Mistral Client Installation ---------------------------- - -Refer :doc:`/admin/install/mistralclient_guide` to find out how to install -Mistral Client. - -Finalize installation ---------------------- - -Restart the Workflow services: - -.. code-block:: console - - # service openstack-mistral-api restart diff --git a/doc/source/admin/install/install.rst b/doc/source/admin/install/install.rst index 7181c01a7..f4a3cbace 100644 --- a/doc/source/admin/install/install.rst +++ b/doc/source/admin/install/install.rst @@ -1,5 +1,3 @@ -.. _install: - ===================== Install and configure ===================== @@ -7,19 +5,222 @@ Install and configure This section describes how to install and configure the Workflow Service, code-named mistral, on the controller node. -**NOTE:** **Mistral can be used in standalone mode or it can work -with OpenStack.** +.. note:: + + Mistral can be used in standalone mode or it can work with OpenStack. If Mistral is used with OpenStack, you must already have a working OpenStack environment with at least the following components installed: - Keystone with API v3 support -Note that installation and configuration vary by distribution. +Note that installation and configuration may vary by distribution. -.. toctree:: - :maxdepth: 2 +Overview +-------- - install-ubuntu - install-rdo - install-obs +The Workflow service consists of the following components: + +``Mistral API`` service + Provides a REST API for operating and monitoring workflow executions. + +``mistral-dashboard`` service + Mistral Dashboard is a Horizon plugin. + +``Mistral Engine`` service + Controls workflow executions and handles their data flow, places finished + tasks in a queue, transfers data from task to task, and deals with condition + transitions, and so on. + +``Mistral Executor`` service + Executes task actions, picks up the tasks from the queue, runs actions, and + sends results back to the engine. + +``Mistral Notifier`` service + +``python-mistralclient`` + Python client API and Command Line Interface. + + ``mistral-lib`` + A library to support writing custom Mistral actions. + +Prerequisites +------------- + +Install the following dependencies: + +On ``apt`` based distributions: + +.. code-block:: console + + $ apt-get update + $ apt-get install python3 python3-venv python3-pip git + +On ``dnf`` based distributions: + +.. code-block:: console + + $ dnf update + $ dnf install python3 python3-venv python3-pip git + +.. note:: + + you may need to adapt the previous commands based on your distribution. + +Installation +------------ + +.. note:: + + For instructions on how to install Mistral using devstack, refer to + :doc:`Mistral Devstack Installation <../../contributor/devstack>` + +Clone the repo and go to the repo directory: + +.. code-block:: console + + $ git clone https://opendev.org/openstack/mistral + $ cd mistral + +Create a venv: + +.. code-block:: console + + $ python3 -m venv venv + $ source venv/bin/activate + +Now install mistral: + +.. code-block:: console + + $ pip install \ + -c https://releases.openstack.org/constraints/upper/master \ + -r requirements.txt \ + . + +.. note:: + + You may need to adjust the constraints file based on the release + of mistral you are installing + +Generate the configuration file: + +.. code-block:: console + + $ pip install tox + $ tox -egenconfig + +Create the mistral directory and copy the example configuration file: + +.. code-block:: console + + $ mkdir /etc/mistral + $ cp etc/mistral.conf.sample /etc/mistral/mistral.conf + +Edit the configuration file: + +.. code-block:: console + + $ vi /etc/mistral/mistral.conf + +You may also want to install the `mistral-extra` package to have the +opentack actions available (but this is not mandatory): + +.. code-block:: console + + $ pip install mistral-extra + + +Configuring Mistral +------------------- + +Refer :doc:`../configuration/index` to find general information on how to +configure Mistral server. + + +Before The First Run +-------------------- + +After the installation, you will see the **mistral-server** and +**mistral-db-manage** commands in your virtual env. + +The **mistral-db-manage** command can be used for database migrations. + +Update the database to the latest revision: + +.. code-block:: console + + # For MySQL / MariaDB / PostgreSQL + $ mistral-db-manage --config-file /etc/mistral/mistral.conf upgrade head + + # For SQLite - do not use sqlite in production! + # e.g. connection = 'sqlite:////var/lib/mistral.sqlite' + $ python tools/sync_db.py --config-file /etc/mistral/mistral.conf + +Before starting the Mistral server, run the *mistral-db-manage populate* +command. It creates the DB with all the standard actions and standard workflows +that Mistral provides to all Mistral users.: + +.. code-block:: console + + $ mistral-db-manage --config-file /etc/mistral/mistral.conf populate + +For more detailed information on the *mistral-db-manage* script, see +the :doc:`Mistral Upgrade Guide `. + + +Running Mistral API server +-------------------------- + +To run the Mistral API server, execute the following command in a shell: + +.. code-block:: console + + $ mistral-server --server api --config-file /etc/mistral/mistral.conf + +Running Mistral Engines +----------------------- + +To run the Mistral Engine, execute the following command in a shell: + +.. code-block:: console + + $ mistral-server --server engine --config-file /etc/mistral/mistral.conf + +Running Mistral Executors +------------------------- +To run the Mistral Executor instance, execute the following command in a +shell: + +.. code-block:: console + + $ mistral-server --server executor --config-file /etc/mistral/mistral.conf + +Note that at least one Engine instance and one Executor instance should be +running so that workflow tasks are processed by Mistral. + +Mistral Notifier +---------------- + +To run the Mistral Notifier, execute the following command in a shell: + +.. code-block:: console + + $ mistral-server --server notifier --config-file /etc/mistral/mistral.conf + +Running Multiple Mistral Servers Under the Same Process +------------------------------------------------------- +To run more than one server (API, Engine, or Task Executor) on the same process, +execute the following command in a shell: + +.. code-block:: console + + $ mistral-server --server api,engine --config-file /etc/mistral/mistral.conf + +The --server command line option can be a comma delimited list. The valid +options are "all" (by default if not specified) or any combination of "api", +"engine", and "executor". It is important to note that the "fake" transport for +the rpc_backend defined in the config file should only be used if "all" the +Mistral servers are launched on the same process. Otherwise, messages do not +get delivered if the Mistral servers are launched on different processes +because the "fake" transport is using an in-process queue. diff --git a/doc/source/admin/install/installation_guide.rst b/doc/source/admin/install/installation_guide.rst deleted file mode 100644 index cd244dce0..000000000 --- a/doc/source/admin/install/installation_guide.rst +++ /dev/null @@ -1,456 +0,0 @@ -========================== -Mistral Installation Guide -========================== - -Prerequisites -------------- - -It is necessary to install some specific system libs for installing Mistral. -They can be installed on most popular operating system using their package -manager (for Ubuntu - *apt*, for Fedora - *dnf*, CentOS - *yum*, for Mac OS - -*brew* or *macports*). -The list of needed packages is shown below: - -1. **python-dev** -2. **python-setuptools** -3. **python-pip** -4. **libffi-dev** -5. **libxslt1-dev (or libxslt-dev)** -6. **libxml2-dev** -7. **libyaml-dev** -8. **libssl-dev** - -In case of Ubuntu, just run:: - - $ apt-get install python-dev python-setuptools python-pip libffi-dev \ - libxslt1-dev libxml2-dev libyaml-dev libssl-dev - -**NOTE:** **Mistral can be used without authentication at all or it can work -with OpenStack.** In case of OpenStack, it works **only on Keystone v3**, make -sure **Keystone v3** is installed. - -Installation ------------- - -**NOTE**: If it is needed to install Mistral using devstack, please refer to -:doc:`Mistral Devstack Installation <../../contributor/devstack>` - -First of all, clone the repo and go to the repo directory:: - - $ git clone https://github.com/openstack/mistral.git - $ cd mistral - -Install tox:: - - $ pip install tox - -Generate config:: - - $ tox -egenconfig - -Configure Mistral as needed. The configuration file is located in -``etc/mistral.conf.sample``. You will need to modify the configuration options -and then copy it into ``/etc/mistral/mistral.conf``. -For details see :doc:`Mistral Configuration Guide ` - -**Virtualenv installation**:: - - $ tox - -This will install necessary virtual environments and run all the project tests. -Installing virtual environments may take significant time (~10-15 mins). - -**Local installation**:: - - $ pip install -e . - -or:: - - $ pip install -r requirements.txt - $ python setup.py install - -**NOTE**: Differences *pip install -e* and *setup.py install*. -**pip install -e** works very similarly to **setup.py install** or the -EasyInstall tool, except that it doesn't actually install anything. -Instead, it creates a special .egg-link file in the deployment directory, -that links to your project's source code. - -Before the first run --------------------- - -After installation you will see **mistral-server** and **mistral-db-manage** -commands in your environment, either in system or virtual environment. - -**NOTE**: In case of using **virtualenv**, all Mistral related commands -available via **tox -evenv --**. For example, *mistral-server* is available via -*tox -evenv -- mistral-server*. - -**mistral-db-manage** command can be used for migrations. - -For updating the database to the latest revision type:: - - $ mistral-db-manage --config-file upgrade head - -Before starting Mistral server, run *mistral-db-manage populate* command. -It prepares the database with standard actions and workflows which Mistral -will provide for all users.:: - - $ mistral-db-manage --config-file populate - -For more detailed information about *mistral-db-manage* script please -see :doc:`Mistral Upgrade Guide `. - -**NOTE**: For users who want a dry run with **SQLite** database backend(not -used in production), *mistral-db-manage* is not recommended for database -initialization because of -`SQLite limitations `_. -Please use sync_db script described below instead for database initialization. - -**If you use virtualenv**:: - - $ tools/sync_db.sh --config-file - -**Or run sync_db directly**:: - - $ python tools/sync_db.py --config-file - -Running Mistral API server --------------------------- - -To run Mistral API server perform the following command in a shell:: - - $ mistral-server --server api --config-file - -Running Mistral Engines ------------------------ - -To run Mistral Engine perform the following command in a shell:: - - $ mistral-server --server engine --config-file - -Running Mistral Task Executors ------------------------------- -To run Mistral Task Executor instance perform the following command -in a shell:: - - $ mistral-server --server executor --config-file - -Running Mistral Notifier ------------------------- -To run Mistral Notifier perform the following command in a shell:: - - $ mistral-server --server notifier -- config-file - -Note that at least one Engine instance and one Executor instance should be -running so that workflow tasks are processed by Mistral. - -Running Multiple Mistral Servers Under the Same Process -------------------------------------------------------- -To run more than one server (API, Engine, or Task Executor) on the same -process, perform the following command in a shell:: - - $ mistral-server --server api,engine --config-file - -The --server command line option can be a comma delimited list. The valid -options are "all" (by default if not specified) or any combination of "api", -"engine", "notifier" and "executor". It's important to note -that the "fake" transport for -the rpc_backend defined in the config file should only be used if "all" the -Mistral servers are launched on the same process. Otherwise, messages do not -get delivered if the Mistral servers are launched on different processes -because the "fake" transport is using an in process queue. - -Running Mistral By Systemd --------------------------- -#. Create an upstart config, it could be named as - ``/etc/systemd/system/mistral-api.service``: - - .. code-block:: bash - - [Unit] - Description = Openstack Workflow Service API - - [Service] - ExecStart = /usr/bin/mistral-server --server api --config-file /etc/mistral/mistral.conf - User = mistral - - [Install] - WantedBy = multi-user.target - -#. Enable and start mistral-api: - - .. code-block:: console - - # systemctl enable mistral-api - # systemctl start mistral-api - -#. Verify that mistral-api services are running: - - .. code-block:: console - - # systemctl status mistral-api - -#. Create an upstart config, it could be named as - ``/etc/systemd/system/mistral-engine.service``: - - .. code-block:: bash - - [Unit] - Description = Openstack Workflow Service Engine - - [Service] - ExecStart = /usr/bin/mistral-server --server engine --config-file /etc/mistral/mistral.conf - User = mistral - - [Install] - WantedBy = multi-user.target - -#. Enable and start mistral-engine: - - .. code-block:: console - - # systemctl enable mistral-engine - # systemctl start mistral-engine - -#. Verify that mistral-engine services are running: - - .. code-block:: console - - # systemctl status mistral-engine - -#. Create an upstart config, it could be named as - ``/etc/systemd/system/mistral-notifier.service``: - - .. code-block:: bash - - [Unit] - Description = Openstack Workflow Service Notifier - - [Service] - ExecStart = /usr/bin/mistral-server --server notifier --config-file /etc/mistral/mistral.conf - User = mistral - - [Install] - WantedBy = multi-user.target - -#. Enable and start mistral-notifier: - - .. code-block:: console - - # systemctl enable mistral-notifier - # systemctl start mistral-notifier - -#. Verify that mistral-notifier services are running: - - .. code-block:: console - - # systemctl status mistral-notifier - -#. Create an upstart config, it could be named as - ``/etc/systemd/system/mistral-executor.service``: - - .. code-block:: bash - - [Unit] - Description = Openstack Workflow Service Executor - - [Service] - ExecStart = /usr/bin/mistral-server --server executor --config-file /etc/mistral/mistral.conf - User = mistral - - [Install] - WantedBy = multi-user.target - -#. Enable and start mistral-executor: - - .. code-block:: console - - # systemctl enable mistral-executor - # systemctl start mistral-executor - -#. Verify that mistral-executor services are running: - - .. code-block:: console - - # systemctl status mistral-executor - - -Mistral And Docker ------------------- - -Docker containers provide an easy way to quickly deploy independent or -networked Mistral instances in seconds. This guide describes the process -to launch an all-in-one Mistral container. - - -Docker Installation -------------------- - -The following links contain instructions to install latest Docker software: - -* `Docker Engine `_ -* `Docker Compose `_ - - -Build the Mistral Image Manually --------------------------------- - -Execute the following command from the repository top-level directory:: - - docker build -t mistral -f tools/docker/Dockerfile . - -The Mistral Docker image has one build parameter: - -+-------------------------+-------------+--------------------------------------+ -|Name |Default value| Description | -+=========================+=============+======================================+ -|`BUILD_TEST_DEPENDENCIES`|false |If the `BUILD_TEST_DEPENDENCIES` | -| | |equals `true`, the Mistral test | -| | |dependencies will be installed inside | -| | |the Docker image | -+-------------------------+-------------+----------------------+---------------+ - - -Running Mistral using Docker Compose ------------------------------------- - -To launch Mistral in the single node configuration:: - - docker-compose -f tools/docker/docker-compose/infrastructure.yaml \ - -f tools/docker/docker-compose/mistral-single-node.yaml \ - -p mistral up -d - -To launch Mistral in the multi node configuration:: - - docker-compose -f tools/docker/docker-compose/infrastructure.yaml \ - -f tools/docker/docker-compose/mistral-multi-node.yaml \ - -p mistral up -d - -The infrastructure docker-compose file contains examples of RabbitMQ, -PostgreSQL and MySQL configurations. Feel free to modify the docker-compose -files as needed. - -The docker-compose Mistral configurations also include the CloudFlow container. -It is available at `link `_ - -The `--build` option can be used when it is necessary to rebuild the image, -for example:: - - docker-compose -f tools/docker/docker-compose/infrastructure.yaml \ - -f tools/docker/docker-compose/mistral-single-node.yaml \ - -p mistral up -d --build - -Running the Mistral client from the Docker Compose container ------------------------------------------------------------- - -To run the mistral client against the server in the container using the client -present in the container:: - - docker run -it mistral_mistral mistral workflow-list - -Configuring Mistral -------------------- - -The Docker image contains the minimal set of Mistral configuration parameters -by default: - -+--------------------+------------------+--------------------------------------+ -|Name |Default value | Description | -+====================+==================+======================================+ -|`MESSAGE_BROKER_URL`|rabbit://guest:gu\|The message broker URL | -| |est@rabbitmq:5672 | | -+--------------------+------------------+----------------------+---------------+ -|`DATABASE_URL` |sqlite:///mistral\|The database URL | -| |.db | | -+--------------------+------------------+----------------------+---------------+ -|`UPGRADE_DB` |false |If the `UPGRADE_DB` equals `true`, | -| | |a database upgrade will be launched | -| | |before Mistral main process | -+--------------------+------------------+----------------------+---------------+ -|`MISTRAL_SERVER` |all |Specifies which mistral server to | -| | |start by the launch script. | -+--------------------+------------------+----------------------+---------------+ -|`LOG_DEBUG` |false |If set to true, the logging level will| -| | |be set to DEBUG instead of the default| -| | |INFO level. | -+--------------------+------------------+----------------------+---------------+ -|`RUN_TESTS` |false |If the `UPGRADE_DB` equals `true`, | -| | |the Mistral unit tests will be | -| | |launched inside container | -+--------------------+------------------+----------------------+---------------+ - -The `/etc/mistral/mistral.conf` configuration file can be mounted to the Mistral -Docker container by uncommenting and editing the `volumes` sections in the -Mistral docker-compose files. - - -Launch tests inside Container ------------------------------ - -Build mistral:: - - docker build -t mistral -f tools/docker/Dockerfile \ - --build-arg BUILD_TEST_DEPENDENCIES=true . - -Run tests using SQLite:: - - docker run -it -e RUN_TESTS=true mistral - -or PostgreSQL:: - - docker run -it \ - -e DATABASE_URL=postgresql://postgres:postgres@localhost:5432/postgres \ - -e RUN_TESTS=true mistral - - -Keycloak integration --------------------- - -If you set AUTH_ENABLE to True value in the mistral.env file then Mistral will -enable Keycloak integration by default. Keycloak will be deployed with -mistral/mistral credentials. You should uncomment the volume line in the -`infrastructure.yaml` for the CloudFlow. - -Next step you login in the administrative console using the -http://localhost:8080/auth/admin URL. Create a oauth client, you can -specify only a name, for example mistral. - -Specify valid redirect URL: http://localhost:8000/* and turn on the -"Implicit Flow Enabled" in the your client page. Save your changes. - -Add the following line to your /etc/hosts file:: - - 127.0.0.1 keycloak - -Export the following environments variable for mistral cli:: - - export MISTRAL_AUTH_TYPE=keycloak-oidc - export OS_AUTH_URL=http://keycloak:8080/auth - export OS_TENANT_NAME=master - export OS_USERNAME=mistral - export OS_PASSWORD=mistral - export OS_MISTRAL_URL=http://localhost:8989/v2 - export OPENID_CLIENT_ID=mistral - export OPENID_CLIENT_SECRET= - export MISTRALCLIENT_INSECURE=True - -Check your configuration:: - - mistral workflow-list - -Or open a cloud flow page in a browser:: - - http://localhost:8000 - - -Using Mistral Client with Docker --------------------------------- - -The Mistral API will be accessible from the host machine on the default -port 8989. Install `python-mistralclient` on the host machine to -execute mistral commands. - -Mistral Client Installation ---------------------------- - -Please refer to :doc:`Mistral Client / CLI Guide ` diff --git a/doc/source/admin/install/next-steps.rst b/doc/source/admin/install/next-steps.rst deleted file mode 100644 index 511a588c2..000000000 --- a/doc/source/admin/install/next-steps.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. _next-steps: - -Next steps -~~~~~~~~~~ - -Your OpenStack environment now includes the Mistral service. - -To add additional services, see `OpenStack Pike Installation Tutorials and -Guides `_. diff --git a/doc/source/admin/install/verify.rst b/doc/source/admin/install/verify.rst index 202218d70..7882de3db 100644 --- a/doc/source/admin/install/verify.rst +++ b/doc/source/admin/install/verify.rst @@ -5,47 +5,61 @@ Basic verification .. code-block:: console - $ mistral run-action std.noop + $ mistral run-action std.echo '{"output": "Hello world"}' +Should give you something like: -Verify operation of the Workflow service -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. code-block:: console -.. note:: + {"result": "Hello world"} - Perform these commands on the controller node. +Congrats! + +A step further - your first workflow +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #. Create a workflow file: - .. code-block:: console +.. code-block:: console - $ cat >/tmp/test.wf.yaml </tmp/test.wf.yaml < + tasks: + echo_task: + action: std.echo output=<% $.message %> + publish: + output: <% task().result %> + EOL #. Create a workflow from the workflow file: - .. code-block:: console +.. code-block:: console - $ mistral workflow-create /tmp/test.wf.yaml + $ mistral workflow-create /tmp/test.wf.yaml #. Create an execution based on the workflow: - .. code-block:: console +.. code-block:: console - $ mistral execution-create test_wf + $ mistral execution-create test_wf #. Run the execution until its returning state is 'SUCCESS': - .. code-block:: console +.. code-block:: console - $ mistral execution-list + $ mistral execution-list + +#. You can grab the output of the execution using: + +.. code-block:: console + + $ mistral execution-get-output After performing the above steps, the Mistral service is ready for use. diff --git a/doc/source/admin/install/mistralclient_guide.rst b/doc/source/user/cli/advanced.rst similarity index 88% rename from doc/source/admin/install/mistralclient_guide.rst rename to doc/source/user/cli/advanced.rst index 40c328420..d83f5cd42 100644 --- a/doc/source/admin/install/mistralclient_guide.rst +++ b/doc/source/user/cli/advanced.rst @@ -1,23 +1,6 @@ -=========================== -Mistral Client Installation -=========================== - -To install ``python-mistralclient``, it is required to have ``pip`` -(in most cases). Make sure that ``pip`` is installed. Then type: - -.. code-block:: console - - $ pip install python-mistralclient - -Or, if it is needed to install ``python-mistralclient`` from master branch, -type: - -.. code-block:: console - - $ pip install git+https://github.com/openstack/python-mistralclient.git - -After ``python-mistralclient`` is installed you will see command ``mistral`` -in your environment. +============================= +Mistral Client Advanced Usage +============================= Configure Authentication Against Keystone ----------------------------------------- @@ -27,9 +10,9 @@ have auth variables: .. code-block:: console - $ export OS_AUTH_URL=http://:5000/v2.0 + $ export OS_AUTH_URL=http://:5000/v3 $ export OS_TENANT_NAME=tenant - $ export OS_USERNAME=admin + $ export OS_USERNAME=user $ export OS_PASSWORD=secret $ export OS_MISTRAL_URL=http://:8989/v2 ( optional, by default URL=http://localhost:8989/v2) diff --git a/doc/source/user/cli/index.rst b/doc/source/user/cli/index.rst index b195a407c..e5f031215 100644 --- a/doc/source/user/cli/index.rst +++ b/doc/source/user/cli/index.rst @@ -1,7 +1,19 @@ +============================= Mistral Client Commands Guide ============================= -The Mistral CLI can be used with ``mistral`` command or via `OpenStackClient +The Mistral CLI can be used with ``mistral`` command or via ``openstack``. + +Installation +------------ + +To install ``mistral`` and ``openstack`` via ``pip``: + +.. code-block:: console + + $ pip install python-mistralclient python-openstackclient + +See here for more info about `OpenStackClient `_. Mistral Client @@ -10,7 +22,7 @@ Mistral Client The best way to learn about all the commands and arguments that are expected is to use the ``mistral help`` command. -.. code-block:: bash +.. code-block:: console $ mistral help usage: mistral [--version] [-v] [--log-file LOG_FILE] [-q] [-h] [--debug] @@ -21,7 +33,7 @@ is to use the ``mistral help`` command. It can also be used with the name of a sub-command. -.. code-block:: bash +.. code-block:: console $ mistral help execution-create usage: mistral execution-create [-h] [-f {json,shell,table,value,yaml}] @@ -49,7 +61,7 @@ The full list of Mistral commands that are registered with OpenStack client can be listed with ``openstack command list``. By default it will list all commands grouped together, but we can specify only the Mistral command group. -.. code-block:: bash +.. code-block:: console $ openstack command list --group openstack.workflow_engine.v2 +------------------------------+-----------------------------------+ @@ -66,7 +78,7 @@ commands grouped together, but we can specify only the Mistral command group. Then detailed help output can be requested for an individual command. -.. code-block:: bash +.. code-block:: console $ openstack help workflow execution create usage: openstack workflow execution create [-h] diff --git a/doc/source/user/index.rst b/doc/source/user/index.rst index 1360c9846..2fe3d8076 100644 --- a/doc/source/user/index.rst +++ b/doc/source/user/index.rst @@ -25,4 +25,5 @@ info on concrete features. wf_lang_v2 rest_api_v2 cli/index + cli/advanced cookbooks/index diff --git a/doc/source/user/terminology/workflows.rst b/doc/source/user/terminology/workflows.rst index 376be4681..316aefb51 100644 --- a/doc/source/user/terminology/workflows.rst +++ b/doc/source/user/terminology/workflows.rst @@ -1,5 +1,5 @@ -Mistral Workflows -================= +Workflows +========= Workflow is the main building block of Mistral Workflow Language, the reason why the project exists. Workflow represents a process that can be described in diff --git a/tools/docker/DOCKER_README.rst b/tools/docker/DOCKER_README.rst deleted file mode 100644 index 17b62b45b..000000000 --- a/tools/docker/DOCKER_README.rst +++ /dev/null @@ -1,2 +0,0 @@ -You can find the latest documentation of Mistral integration with Docker -`here `_. diff --git a/tools/docker/Dockerfile b/tools/docker/Dockerfile deleted file mode 100644 index f2e8c489f..000000000 --- a/tools/docker/Dockerfile +++ /dev/null @@ -1,68 +0,0 @@ -FROM krallin/ubuntu-tini:16.04 - -LABEL name="Mistral" \ - description="Workflow Service for OpenStack" \ - maintainers="Andras Kovi \ - Vitalii Solodilov " - -RUN apt-get -qq update && \ - apt-get install -y \ - libffi-dev \ - libpq-dev \ - libssl-dev \ - libxml2-dev \ - libxslt1-dev \ - libyaml-dev \ - libmysqlclient-dev \ - python \ - python-dev \ - crudini \ - curl \ - git \ - gcc \ - libuv1 \ - libuv1-dev && \ - curl -f -o /tmp/get-pip.py https://bootstrap.pypa.io/pip/3.2/get-pip.py && \ - python /tmp/get-pip.py && rm /tmp/get-pip.py && \ - pip install --upgrade pip - -RUN pip install pymysql psycopg2 py_mini_racer - -ENV MISTRAL_DIR="/opt/stack/mistral" \ - TMP_CONSTRAINTS="/tmp/upper-constraints.txt" \ - CONFIG_FILE="/etc/mistral/mistral.conf" \ - INI_SET="crudini --set /etc/mistral/mistral.conf" \ - MESSAGE_BROKER_URL="rabbit://guest:guest@rabbitmq:5672/" \ - DATABASE_URL="sqlite:///mistral.db" \ - UPGRADE_DB="false" \ - RUN_TESTS="false" \ - DEBIAN_FRONTEND="noninteractive" \ - MISTRAL_SERVER="all" \ - LOG_DEBUG="false" \ - AUTH_URL="http://keycloak:8080/auth" \ - AUTH_ENABLE="false" \ - AUTH_TYPE="keycloak-oidc" - -# We install dependencies separatly for a caching purpose -COPY requirements.txt "${MISTRAL_DIR}/" -RUN curl -o "${TMP_CONSTRAINTS}" \ - http://opendev.org/openstack/requirements/raw/branch/master/upper-constraints.txt && \ - sed -i "/^mistral.*/d" "${TMP_CONSTRAINTS}" && \ - pip install -r "${MISTRAL_DIR}/requirements.txt" - -ARG BUILD_TEST_DEPENDENCIES="false" -COPY test-requirements.txt "${MISTRAL_DIR}/" -RUN if ${BUILD_TEST_DEPENDENCIES} ; then \ - pip install -r "${MISTRAL_DIR}/test-requirements.txt" ; \ - fi - -COPY . ${MISTRAL_DIR} - -RUN pip install -e "${MISTRAL_DIR}" && \ - mkdir /etc/mistral && \ - rm -rf /var/lib/apt/lists/* && \ - find ${MISTRAL_DIR} -name "*.sh" -exec chmod +x {} \; - -WORKDIR "${MISTRAL_DIR}" -EXPOSE 8989 -CMD "${MISTRAL_DIR}/tools/docker/start.sh" diff --git a/tools/docker/docker-compose/auth.json b/tools/docker/docker-compose/auth.json deleted file mode 100644 index 918ffac3e..000000000 --- a/tools/docker/docker-compose/auth.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "_type": "openid-connect", - "issuer": "http://keycloak:8080/auth/realms/master", - "loginUrl": "http://keycloak:8080/auth/realms/master/protocol/openid-connect/auth", - "logoutUrl": "http://keycloak:8080/auth/realms/master/protocol/openid-connect/logout", - "requireHttps": false, - "clientId": "mistral" -} \ No newline at end of file diff --git a/tools/docker/docker-compose/infrastructure.yaml b/tools/docker/docker-compose/infrastructure.yaml deleted file mode 100644 index 1e478dde8..000000000 --- a/tools/docker/docker-compose/infrastructure.yaml +++ /dev/null @@ -1,79 +0,0 @@ -version: '3' -services: -# postgresql: -# image: postgres:10.1-alpine -# restart: always -# ports: -# - "5432:5432" -# volumes: -# - postgresql:/var/lib/postgresql/data -# networks: -# - database -# environment: -# - POSTGRES_PASSWORD=mistral -# - POSTGRES_USER=mistral -# - POSTGRES_DB=mistral - - rabbitmq: - image: rabbitmq:3.7.2-management-alpine - restart: always - ports: - - "15672:15672" - networks: - - message-broker - hostname: rabbitmq - environment: - - RABBITMQ_VM_MEMORY_HIGH_WATERMARK=0.81 - - RABBITMQ_DEFAULT_USER=mistral - - RABBITMQ_DEFAULT_PASS=mistral - - RABBITMQ_DEFAULT_VHOST=mistral - - mysql: - image: mysql:8.0.3 - restart: always - ports: - - "3306:3306" - volumes: - - mysql:/var/lib/mysql - networks: - - database - environment: - - MYSQL_ROOT_PASSWORD=mistral - - MYSQL_DATABASE=mistral - - MYSQL_USER=mistral - - MYSQL_PASSWORD=mistral - - keycloak: - image: jboss/keycloak:4.1.0.Final - ports: - - "8080:8080" - networks: - - identity-provider - environment: - DB_VENDOR: h2 - KEYCLOAK_USER: mistral - KEYCLOAK_PASSWORD: mistral - - cloud-flow: - image: mcdoker18/cloud-flow:0.5.0 - restart: always - networks: - - cloud-flow - ports: - - "8000:8000" - # TODO: make a Keycloak environment variables for CloudFlow - # volumes: - # - "./auth.json:/opt/CloudFlow/dist/assets/auth.json" - environment: - - CF_MISTRAL_URL=http://mistral:8989 - -volumes: - postgresql: - rabbitmq: - mysql: - -networks: - database: - message-broker: - identity-provider: - cloud-flow: \ No newline at end of file diff --git a/tools/docker/docker-compose/mistral-multi-node.yaml b/tools/docker/docker-compose/mistral-multi-node.yaml deleted file mode 100644 index 55bd74fe7..000000000 --- a/tools/docker/docker-compose/mistral-multi-node.yaml +++ /dev/null @@ -1,96 +0,0 @@ -version: '3' -services: - mistral-api: - build: - context: ../../.. - dockerfile: tools/docker/Dockerfile - args: - BUILD_TEST_DEPENDENCIES: "false" - restart: always - ports: - - "8989:8989" - networks: - database: - message-broker: - identity-provider: - cloud-flow: - aliases: - - mistral - env_file: - - mistral.env -# volumes: -# - "/path/to/mistral.conf:/etc/mistral/mistral.conf" - environment: - - MISTRAL_SERVER=api - - UPGRADE_DB=true - - mistral-engine: - build: - context: ../../.. - dockerfile: tools/docker/Dockerfile - args: - BUILD_TEST_DEPENDENCIES: "false" - restart: always - networks: - - database - - identity-provider - - message-broker - env_file: - - mistral.env -# volumes: -# - "/path/to/mistral.conf:/etc/mistral/mistral.conf" - environment: - - MISTRAL_SERVER=engine - - mistral-executor: - build: - context: ../../.. - dockerfile: tools/docker/Dockerfile - args: - BUILD_TEST_DEPENDENCIES: "false" - restart: always - networks: - - message-broker - - identity-provider - env_file: - - mistral.env -# volumes: -# - "/path/to/mistral.conf:/etc/mistral/mistral.conf" - environment: - - MISTRAL_SERVER=executor - - mistral-event-engine: - build: - context: ../../.. - dockerfile: tools/docker/Dockerfile - args: - BUILD_TEST_DEPENDENCIES: "false" - restart: always - networks: - - database - - message-broker - - identity-provider - env_file: - - mistral.env -# volumes: -# - "/path/to/mistral.conf:/etc/mistral/mistral.conf" - environment: - - MISTRAL_SERVER=event-engine - - mistral-notifier: - build: - context: ../../.. - dockerfile: tools/docker/Dockerfile - args: - BUILD_TEST_DEPENDENCIES: "false" - restart: always - networks: - - database - - message-broker - - identity-provider - env_file: - - mistral.env -# volumes: -# - "/path/to/mistral.conf:/etc/mistral/mistral.conf" - environment: - - MISTRAL_SERVER=notifier diff --git a/tools/docker/docker-compose/mistral-single-node.yaml b/tools/docker/docker-compose/mistral-single-node.yaml deleted file mode 100644 index 1e6a8ef9c..000000000 --- a/tools/docker/docker-compose/mistral-single-node.yaml +++ /dev/null @@ -1,22 +0,0 @@ -version: '3' -services: - mistral: - build: - context: ../../.. - dockerfile: "tools/docker/Dockerfile" - args: - BUILD_TEST_DEPENDENCIES: "false" - restart: always - ports: - - "8989:8989" - networks: - - database - - message-broker - - identity-provider - - cloud-flow - env_file: - - mistral.env -# volumes: -# - "/path/to/mistral.conf:/etc/mistral/mistral.conf" - environment: - - UPGRADE_DB=true \ No newline at end of file diff --git a/tools/docker/docker-compose/mistral.env b/tools/docker/docker-compose/mistral.env deleted file mode 100644 index e82aa732f..000000000 --- a/tools/docker/docker-compose/mistral.env +++ /dev/null @@ -1,4 +0,0 @@ -MESSAGE_BROKER_URL=rabbit://mistral:mistral@rabbitmq:5672/mistral -#DATABASE_URL=postgresql+psycopg2://mistral:mistral@postgresql:5432/mistral -DATABASE_URL=mysql+pymysql://mistral:mistral@mysql:3306/mistral -AUTH_ENABLE=false \ No newline at end of file diff --git a/tools/docker/start.sh b/tools/docker/start.sh deleted file mode 100755 index 306e5ccdd..000000000 --- a/tools/docker/start.sh +++ /dev/null @@ -1,41 +0,0 @@ -#!/bin/bash -set -e - -# If a Mistral config doesn't exist we should create it and fill in with -# parameters -if [ ! -f ${CONFIG_FILE} ]; then - oslo-config-generator \ - --config-file "${MISTRAL_DIR}/tools/config/config-generator.mistral.conf" \ - --output-file "${CONFIG_FILE}" - - ${INI_SET} DEFAULT js_implementation py_mini_racer - ${INI_SET} oslo_policy policy_file "${MISTRAL_DIR}/etc/policy.json" - - ${INI_SET} DEFAULT auth_type ${AUTH_TYPE} - ${INI_SET} pecan auth_enable ${AUTH_ENABLE} - ${INI_SET} keycloak_oidc auth_url ${AUTH_URL} - ${INI_SET} keycloak_oidc insecure true - - ${INI_SET} DEFAULT transport_url "${MESSAGE_BROKER_URL}" - ${INI_SET} database connection "${DATABASE_URL}" - ${INI_SET} DEFAULT debug "${LOG_DEBUG}" -fi - -if [ ${DATABASE_URL} == "sqlite:///mistral.db" -a ! -f ./mistral.db ] -then - python ./tools/sync_db.py --config-file "${CONFIG_FILE}" - mistral-db-manage --config-file "${CONFIG_FILE}" populate -fi - -if "${UPGRADE_DB}"; -then - /usr/local/bin/mistral-db-manage --config-file "${CONFIG_FILE}" upgrade head - mistral-db-manage --config-file "${CONFIG_FILE}" populate -fi -if "${RUN_TESTS}"; -then - cp "${CONFIG_FILE}" .mistral.conf - "${MISTRAL_DIR}/run_tests.sh" -N -else - mistral-server --config-file "${CONFIG_FILE}" --server ${MISTRAL_SERVER} -fi